Digital COVID-19 'symptom checkers' may stop some patients from getting prompt treatment for serious illness, suggests an international case simulation study, published in the online journal BMJ Health & Care Informatics.
Both the US and UK symptom checkers consistently failed to identify the symptoms of severe COVID-19, bacterial pneumonia, and sepsis, frequently advising these cases to stay home, the findings indicate.
The availability and use of symptom checkers is on the rise, and they are currently being used at a national level to pick up COVID-19 infection.
Identifying which patients with COVID-19 require treatment is difficult, because the infection can mimic common conditions that rarely require medical attention and because there are no clinical signs or symptoms that reliably predict who will progress to severe disease, note the researchers.
Digital symptom checkers combine a series of set questions and pre-determined responses to advise a person on the most appropriate course of action.
But as yet, there is hardly any evidence for the effectiveness or safety of symptom checkers for prioritising treatment (triaging) during a pandemic, say the researchers.
They therefore looked at the ability of national government-sponsored digital symptom checkers in use in Singapore, Japan, the US and the UK, to correctly triage people in need of a medical assessment and/or treatment.
The symptom trackers included: the Singapore COVID-19 Symptom Checker; Stop COVID-19 Symptom Checker (Japan); CDC Coronavirus Symptom Checker (US); and 111 COVID-19 Symptom Checker (UK).
COVID-19 death rates in Singapore and Japan are comparatively low; in the US and the UK they are comparatively high.
The researchers wanted to see if the symptom checkers were able to differentiate mild from severe COVID-19, and how well they picked up COVID-19 'mimickers' such as bacterial pneumonia and sepsis in 52 standardised case scenarios.
The cases simulated common COVID-19 related signs/symptoms of varying severity and risk factors. These included: cough and fever; a co-existing condition (high blood pressure) plus cough and fever; suppressed immunity as a result of drug treatment plus cough and fever; and shortness of breath and fever.
These four distinct presentations were then varied in relation to one or more of the following: how long since symptoms had started; the patient's age; and symptom severity.
The symptom checkers in Singapore and Japan triaged twice as many cases for direct clinical assessment as the symptom checkers in the US and UK.
Singapore had the highest overall referral rate at 88%; the US had the lowest at 38%. Among the simulated cases not referred, the USA and UK triaged a significant number that would typically have required early clinical assessment to 'stay home'.
The US symptom checker frequently triaged simulated cases with possible severe COVID-19, bacterial pneumonia, and sepsis to stay home and possible neutropenic (low white cell count after chemotherapy) sepsis to seek medical attention within 24 hours.
The UK's symptom checker frequently triaged possible severe COVID-19 and bacterial pneumonia to stay home with no follow-up and is likely to have delayed treatment for sepsis, severe COVID-19, and severe sepsis, point out the researchers.
While Japan's symptom checker generally performed well, the simulation revealed a potential delay to treatment for very severe sepsis. And none of the symptom checkers triaged simulated neutropenic sepsis to emergency care.
The researchers single out the UK's symptom checker for its complex algorithm, and for attempting to quantify symptoms such as shortness of breath and the overall severity of illness by asking subjective, qualitative questions with multiple choice answers.
The questions used to discriminate between degrees of severity aren't up to the task, they suggest.
Both the US and UK symptom checkers "are likely to delay presentations of serious medical conditions to appropriate care, and as such, are likely to confer an increased risk of morbidity and mortality," they warn.
"Both symptom checkers maintain a high threshold for referring onward to clinical contact, triaging the majority of patients to stay home with no clinical contact."
The researchers acknowledge that by reducing physical contacts, symptom checkers can potentially save valuable resources and reduce viral spread, and they require fewer resources than phone and telemedicine triaging services.
But their findings suggest that "whilst 'symptom checkers' may be of use to the healthcare COVID-19 response, there is the potential for such patient-led assessment tools to worsen outcomes by delaying appropriate clinical assessment," particularly if they are used as the only source of medical advice, they conclude.
"Our results support the recommendation that symptom checkers should be subjected to the same level of evidenced based quality assurance as other diagnostic tests prior to implementation," they add.
###
Externally peer reviewed? Yes
Evidence type: Case simulation study
Subjects: COVID-19 symptom checkers
Journal
BMJ Health & Care Informatics