PITTSBURGH, Nov. 15 -- Two computer-based decision support systems can improve the diagnostic reasoning of clinicians evaluating difficult patient cases, according to a University of Pittsburgh-led study whose results are reported in the Nov. 17 issue of the Journal of the American Medical Association. The finding suggests a role for decision support systems (DSSs) in medicine, particularly in physician training, according to the authors.
DSSs are computer-based systems designed to provide accurate, useful and timely medical information to clinicians who are trying to determine a patient diagnosis given a limited amount of initial information.
"Previous studies have focused on how often DSSs could generate correct diagnoses when placed in the hands of expert users. This study differs because it focuses on a more typical group of clinicians who have been trained to use a DSS but are relatively inexperienced with it. We focus on whether the system is helpful as opposed to whether the system is right, and we look at whether this effect varies with levels of clinical experience," said Charles P. Friedman, Ph.D., associate vice chancellor for biomedical informatics, professor of medicine and director of the Center for Biomedical Informatics at the University of Pittsburgh.
"Using a relatively large sample for this kind of study, we found a modest, but statistically significant, increase in the ability of two systems to improve clinicians' diagnostic reasoning for a set of rather difficult cases. We also believe this is one of the largest studies of diagnostic reasoning conducted to date," he added.
The study, which ran from 1995 to 1998, involved a total of 215 clinicians evaluating 36 cases based on actual patient records. Clinicians were located at the medical centers at the University of Illinois, the University of Michigan and the University of North Carolina. Clinician subjects in the study included faculty-level general internists with at least two years of post-residency clinical experience; internal medicine residents, either late in their second training year or early in their third year; and fourth-year medical students. Initially, participating clinicians received training using either one of two commercially available systems, ILIAD and Quick Medical Reference. Each clinician then was randomly assigned to evaluate nine out of the 36 cases using the DSS on which he/she trained. Subjects evaluated each case, first without, then with, help from a DSS. Their diagnoses generated before and after use of the DSS were compared. Few of the clinicians involved in the study had prior experience with any DSS.
"We found that both DSSs significantly improved the ability of clinicians at each experience level to arrive at a correct diagnosis, but the effect was much larger for medical students than for faculty and residents," remarked Dr. Friedman. "By and large, cases that are very difficult for an expert diagnostician also will be very difficult for the computer. Although a head-to-head-comparison of systems was not our primary objective, we found that Quick Medical Reference was better than ILIAD in assisting the clinicians in the study."
These results suggest that DSSs could be particularly valuable for medical student training, according to Dr. Friedman.
The study was supported by a grant from the National Library of Medicine.
Journal
JAMA