Chatbots designed to recognize Covid-19 symptoms dispense alarmingly inconsistent recommendations.

What’s new: Given the same symptoms, eight high-profile medical bots responded with divergent, often conflicting advice, according to STAT News.

Conflicting information: Reporters Casey Ross and Erin Brodwin discussed Covid-19 symptoms such as coughing, fever, and shortness of breath with conversational systems offered by government agencies, hospitals, and tech companies.

The CDC’s Coronavirus Self-Checker told the reporters that they had at least one Covid-19 symptom and recommended they isolate themselves and contact a healthcare provider within 24 hours. A tool from Providence St. Joseph Health told the reporters they might have Covid-19 and suggested they call a physician or 911.
Buoy Health, a web-based medical service, suggested the symptoms might be a common cold and didn’t recommend special precautions. Google sister company Verily determined that the reporters’ complaints did not warrant further testing.
In a similar test of popular smart-speakers platforms, Recode found that voice assistants from Amazon, Apple, and Google often answered questions about Covid-19 with information that was overly general, outdated, or lacking context.

Behind the news: A study from Stanford University suggests that symptom checkers built for Covid-19 are flawed partly because the disease’s early signs are similar to those of the common cold or garden-variety influenza. A 2015 study that found that online symptom checkers for a range of conditions often reach faulty conclusions.

Yes, but: Screening tools don’t need to be perfect to add a lot of value. They are statistical tools intended to prioritize quickly and inexpensively which cases should be escalated for deeper examination.

Why it matters: The significant disagreement among these tools means there’s a lot of room for improvement, and bad advice is clearly dangerous in a situation like this. Still, AI-based screening could play a helpful role in this pandemic, especially considering how many countries are short on test kits. It could ease the burden on hospital staff and testing centers, which risk becoming overwhelmed as the pandemic spreads.

We’re thinking: Human doctors’ recommendations aren’t always consistent.

Verily

Chatbots Disagree on Covid-19

Subscribe to The Batch