Large language model shows promise in helping clinicians identify postpartum hemorrhage

Postpartum hemorrhage is the leading cause of maternal mortality and morbidity worldwide and a common pregnancy complication. This serious medical condition is understudied and not universally defined or well represented in health records. A new study by investigators from Brigham and Women’s Hospital, a founding member of the Mass General Brigham healthcare system, used the large language model Flan-T5 to extract medical concepts from electronic health records in order to better define and identify the populations impacted by postpartum hemorrhage. 

The study found the model to be 95 percent accurate in identifying patients with the condition, and resulted in 47 percent more patients identified than when using the standard method of tracking the condition through billing codes. The tool showed great promise for helping clinicians identify subpopulations that are at higher risk of postpartum hemorrhage—and predicting those who are more likely to develop it. The results are published in npj Digital Medicine.

“We need better ways to identify the patients that have this complication, as well as the different clinical factors associated with it,” said corresponding author Vesela Kovacheva, MD, of the Department of Anesthesiology, Perioperative and Pain Medicine. “There are so many amazing large language models being developed right now, and this approach could be used with other conditions and diseases.”

The emergence of artificial intelligence tools in healthcare has been groundbreaking and has the potential to

positively reshape the continuum of care. Mass General Brigham, as one of the nation’s top integrated academic health systems and largest innovation enterprises, is leading the way in conducting rigorous research on new and emerging technologies to inform the responsible incorporation of AI into care delivery, workforce support, and administrative processes.

Because conditions like postpartum hemorrhage include a large spectrum of patients, symptoms, and causes, the research team used the Flan-T5 model to analyze comprehensive information from electronic health records to help them better categorize subpopulations of patients. They prompted the Flan-T5 model with lists of concepts known to be associated with postpartum hemorrhage and then asked it to extract them from the discharge summaries of a cohort of 131,284 patients who gave birth at Mass General Brigham hospitals between 1998-2015. This method achieved rapid and accurate results without the need for manual labeling.

“We looked at all of the patients that Flan-T5 identified as having postpartum hemorrhage and looked at what fraction of those also had the corresponding billing code. It turns out that Flan-T5 was 95 percent accurate and allowed us to identify 47 percent more patients than we would have from the billing codes alone,” said first author Emily Alsentzer, PhD, a research fellow in the Division. “Ideally, we would like to be able to predict who will develop postpartum’ hemorrhage before they do so, and this is a tool that can help us get there.”

Next, the team plans to continue to use this approach to look at other pregnancy complications and hopes their work will help address growing maternal health crises in the United States. 

“This approach can be applied to many future studies,” said Kovacheva. “And it could be used to help guide real-time medical decision making, which is very exciting and valuable to me as a clinician.”

Authorship: Additional Brigham and Women’s authors include Matthew J Rasmussen, Romy Fontoura, Alexis L Cull, Kathryn J Gray, and David W Bates. Additional authors include Brett Beaulieu-Jones.

Disclosures: KJG has served as a consultant to Illumina Inc., Aetion, Roche, and Bil428 lionToOne outside the scope of the submitted work. DWB reports grants and personal fees from EarlySense, personal fees from CDI Negev, equity from Valera Health, equity from CLEW, equity from MDClone, personal fees and equity from AESOP Technology, personal fees and equity from FeelBetter, and grants from IBM Watson Health, outside the submitted work. VPK reports consulting fees from Avania CRO unrelated to the current work.

Funding: This research was funded in part by the National Heart, Lung and Blood Institute (K08 HL146963-02, K08, HL146963-02S1, R03 HL162756, 1K08HL161326-01A1) Anesthesia Patient Safety Foundation (APSF), and BWH IGNITE Award.

Paper cited: Alsentzer, E. et al. “Zero-shot Interpretable Phenotyping of Postpartum Hemorrhage Using Large Language Models.” npj Digital Medicine, DOI: https://www.nature.com/articles/s41746-023-00957-x