Learning to be better doctors through AI

A new application developed at Dartmouth is among the first iterations of an artificial intelligence program that provides independent instruction, roleplaying as a patient to help medical students sharpen their diagnostic and interpersonal skills.

The AI Patient Actor app uses tools from ChatGPT — the chatbot from OpenAI — trained on medical case studies to interact with students through type and voice, responding like a patient by describing symptoms and answering students' questions about how it's feeling. The app provides students with individualized feedback on their performance, aligning with the concept of precision education recently prioritized by the American Medical Association.

Thomas Thesen, an associate professor of medical education in Dartmouth's Geisel School of Medicine who led the creation of AI Patient Actor, said that medical schools usually employ actors to act as patients based on scripts that include a medical history and the symptoms they are experiencing. Medical students are given the opportunity to interact with the actors, learning how to do everything from establishing a bedside rapport and asking the right questions, to developing a final diagnosis.

But hiring and training actors is resource-intensive, which limits the frequency with which these mock interviews are available as learning experiences to budding medical professionals.

That's why Thesen, who is director of the Neuroscience-Informed Learning and Education Lab, hit upon the idea of creating virtual patients when ChatGPT began making waves for its ability to mimic human conversations. His first text-based prototype for the app drew from medical case histories he wrote himself and was powered by ChatGPT's language model to answer students' questions with conversational ease.

AI Patient Actor can be thought of as a customized version of ChatGPT, Thesen said, allowing educators to create a database of tailor-made cases that would be most instructive for students learning the ropes of clinical history-taking and diagnosis.

"As AI continues to evolve, we anticipate a transformative shift in medical education where AI-driven tools like ours enable personalized learning pathways that adapt to each student's unique educational needs and learning pace," Thesen said.

To develop the app further and make it deployable on the web, Thesen teamed up with Simon Stone, a research data science specialist at Dartmouth Library. "Simon had the skills to make the app really user-friendly and take it to the next level. Without that, it would have probably stayed on my laptop," Thesen said.

Thesen debuted AI Patient Actor in 2023 for his Neuroscience and Neurology course that medical students take in their second year. Students have the option to practice diagnosing two or three cases every week that are related to the diseases they learn about in class.

The clinical information behind the app is richly detailed. Not only does it enumerate routine vital signs and relevant symptoms, but the app also reports back lab results for tests that a doctor would likely need to differentiate between diseases that are symptomatically similar.

In Thesen's course, for example, students were learning to diagnose neurological conditions. After asking a set of preliminary questions, the student could request an MRI. The app would then present the test results and the student could be more certain based on the MRI results that the patient is showing signs of Alzheimer's and not Parkinson's disease.

Students receive individualized feedback on their interaction with the patient based on an evaluation rubric built into the app. They can use the feedback to have another go at the interview, improving their clinical and diagnostic skills with each turn.

Modeling a clinical interaction with a virtual patient is an easy first step before graduating to mock interviews with actors or working in an actual clinical setting, said Nsomma Alilonu, a second-year medical student at Dartmouth who worked with Thesen on the experimental design for evaluating AI Patient Actor, creating a feedback rubric, and providing a student's perspective on the app's functionality.

"It's a very good way to practice interviewing patients in a stress-free environment to get formative feedback," she said.

Alilonu majored in computer science as an undergraduate and was thrilled to explore new ways of integrating medicine and technology through this project. "Even if just one patient can benefit from having a doctor who has refined their skills by practicing with our app, it's really worth it," she said.

"While our initial experiences with the AI Patient Actor app have been exceptional, we are undertaking a rigorous scientific experiment to validate its performance," Thesen said. This validation study looks at the accuracy of the patient simulation and the generated feedback, and assesses the implicit bias in the large language model's responses that may occur as a result of inherent biases in the training data.

Since its debut, the team has added several features to the app. Besides English, AI Patient Actor is now available in Spanish and Swahili and can be expanded to other languages as well. They started with these languages, Thesen said, because Geisel offers students a Spanish Medical Pathway as part of its curriculum and the team has begun a partnership to train medical students at Aga Khan University Medical College in Kenya.

Users now have the option to speak to their virtual patients. The latest update allows users to choose between text or voice as input or output, enabling students to have a verbal conversation with their virtual patient.

Eventually, the team hopes to scale the AI Patient Actor app and make it available to medical educators outside the Dartmouth community.

"In higher education there is a prevailing fear that AI will take the human side out of learning," Thesen said. "The beauty of this way of using AI is that it actually helps students become better communicators and, ultimately, connect better with their patients."