Research spotlight: Machine learning helps identify patients at varying levels of risk for opioid use disorder

Ronen Rozenblum, PhD, MPH, director of the Unit for Innovative Healthcare Practice & Technology and director of Business Development of the Center for Patient Safety Research and Practice at Brigham and Women's Hospital, and an assistant professor at Harvard Medical School, is the principal investigator and senior author of a new study published in JMIR Medical Informatics, “A Machine Learning Application to Classify Patients at Differing Levels of Risk of Opioid Use Disorder: Clinician Based Validation Study.”

How would you summarize your study for a lay audience? 

Our study focused on using advanced machine learning (ML) to help clinicians more accurately identify patients at risk of developing opioid use disorder (OUD). Despite strict guidelines for managing opioids, OUD remains a serious public health issue. We evaluated an ML application called MedAware, which alerts clinicians to patients who may be at higher risk of OUD by analyzing patient records. Our findings showed that ML can provide clinicians with reliable alerts about a patient’s level of risk. This kind of technology has the potential to significantly enhance how physicians and other clinicians assess and treat OUD, with the goal of providing more accurate, safer, and personalized care for patients early in their opioid treatment.

What question were you investigating?

This clinical validation study investigated how well the ML system compared to clinicians’ assessment of a patient’s risk of OUD. We examined the agreement between the ML application and clinicians’ structured review of medical records to classify patients receiving opioid treatment into three distinct categories of OUD risk (i.e., not high risk, high risk, or suspected OUD). We also evaluated the reasons for discrepancies between clinicians’ judgements and ML risk assessment.

What methods or approach did you use?

Outpatient data of 649,504 Mass General Brigham patients and a random sample of 180 patients were used to develop the ML model and the validation study, respectively. We developed an OUD risk classification scheme and data extraction tool to validate these alerts. Clinicians independently conducted a systematic and structured review of medical records and reached a consensus on each patient’s OUD risk level, which was then compared to the ML application’s risk assignments.

What did you find?

Our findings revealed that the ML application successfully classified patients into differing levels of OUD risk and demonstrated substantial agreement with clinicians’ review of medical records. The highest agreement between the two methods was observed for patients classified as high risk for OUD and suspected OUD. Thus, the results of this study demonstrate that this ML application can generate clinically valid and useful alerts for screening patients at risk of OUD. Additionally, we identified key themes explaining disagreements between the ML application and clinician reviews.

What are the implications and next steps?

The significance of these findings lies in the fact that only a limited number of studies have examined the clinical validity and utility of ML applications in distinguishing between various levels of OUD risk in patients. These results suggest that ML applications, such as MedAware, can significantly enhance clinicians’ ability to assess patients’ risk for OUD early in opioid treatment, promoting more personalized and safer care. This capability is expected to complement traditional rule-based approaches in alerting physicians and other clinicians about opioid safety issues.

Authorship: In addition to Rozenblum, Mass General Brigham authors include Tewodros Eguale, Wenyu Song, Yasmin Elsobky, Angela Rui, Marlika Marceau, Clark Davis, Sandya Ganesan, Ava Alsubai, Michele Matthews, Lynn A Volk, and David W Bates. Additional authors include François Bastardot and Daniel Motta-Calderon.

Paper cited: Eguale, T et al. “A Machine Learning Application to Classify Patients at Differing Levels of Risk of Opioid Use Disorder: Clinician-Based Validation Study” JMIR Medical Informatics DOI: 10.2196/53625

Funding: This work was supported in part by MedAware, Ltd.