Lap Fai (Craig) Yu, Associate Professor, Computer Science, College of Engineering and Computing, and Joel Martin, Associate Professor, Kinesiology, College of Education and Human Development, received funding for the project: “EAGER: TaskDCL: Adapting Mixed Reality Training Programs to Real-World Scenes to enhance Human-AI Teaming in Emergency Responses.”
This EArly-concept Grant for Exploratory Research (EAGER) project funds research that intends to speed up the development of mixed reality and artificial intelligence (AI) technologies to help first responders, aiming to reduce training-related risks and casualties.
The research team will work with the Fairfax Fire and Rescue Department to explore how AI can be used in mixed reality tools to improve training and effectiveness for first responders. By adding virtual elements like fires, hazards, firefighters, robots, and people in need of rescue to real-life scenes, these mixed reality scenarios help first responders practice handling real-world challenges through interactive training.
The project will also involve a postdoctoral researcher and undergraduate students, including those from underrepresented groups in science and technology fields. The team will share their findings at conferences focused on mixed reality and training.
This EAGER project offers a novel interdisciplinary research perspective by integrating concepts and techniques from mixed reality, AI, human-computer interaction, and movement science to advance first responder training.
The researchers aim to devise a novel optimization-based generative framework for adapting mixed reality training scenarios to real scenes, which will offer ample training opportunities for first responders to practice, accomplishing different first-responder tasks (e.g., firefighting, search and rescue) via human-AI collaboration enabled by mixed reality headsets.
To carry out the research, the team will first investigate how AI techniques could be integrated with mixed reality devices to provide first response assistance. The team will then devise a generative framework based on optimization techniques for adapting mixed reality training scenarios to real scenes. They will conduct user studies to evaluate the performance gain brought about by the advanced mixed reality interfaces and the synthesized training scenarios.
Yu received $299,861 from the National Science Foundation for this research. Funding began in Jan. 2025 and will end in late Dec. 2026.
###
ABOUT GEORGE MASON UNIVERSITY
George Mason University is Virginia’s largest public research university. Located near Washington, D.C., Mason enrolls more than 40,000 students from 130 countries and all 50 states. Mason has grown rapidly over the past half-century and is recognized for its innovation and entrepreneurship, remarkable diversity, and commitment to accessibility. In 2023, the university launched Mason Now: Power the Possible, a one-billion-dollar comprehensive campaign to support student success, research, innovation, community, and stewardship. Learn more at gmu.edu.