image: The flight scenario was presented using a head mounted display to create an even more immersive 3D experience of the task. In addition to electrical measures of brain activity from an electroencephalography (EEG) cap, we also recorded other physiological signals like electrodermal activity from the left hand of participants. view more
Credit: Josef Faller/Columbia Engineering
New York, NY--March 12, 2019--Our state of arousal--being fearful, agitated, or calm--can significantly affect our ability to make optimal decisions, judgments, and actions in real-world dynamic environments. Imagine, for instance, walking across a balance beam. Your performance--speed across the beam and the odds of making it across without falling off--are dramatically better if the beam sits a mere six inches off the ground and you are relaxed rather than terror-stricken on a beam 60 feet higher. To keep you in the zone of maximum performance, your arousal needs to be at moderate levels, not so high that it pushes you over the edge.
Biomedical engineers at Columbia Engineering have shown--for the first time--that they can use online neurofeedback to modify an individual's arousal state to improve performance in a demanding sensory motor task, such as flying a plane or driving in suboptimal conditions. The researchers used a brain computer interface (BCI) to monitor, through electroencephalography (EEG) in real time, the arousal states of the study participants when they were engaged in a virtual reality aerial navigation task. The system generated a neurofeedback signal that helped participants to decrease their arousal in particularly difficult flight situations, which in turn improved participants' performance. The study was published today by Proceedings of the National Academy of Sciences.
"The whole question of how you can get into the zone, whether you're a baseball hitter or a stock trader or a fighter pilot, has always been an intriguing one," says Paul Sajda, professor of biomedical engineering (BME), electrical engineering, and radiology, who led the study. "Our work shows that we can use feedback generated from our own brain activity to shift our arousal state in ways that significantly improve our performance in difficult tasks--so we can hit that home run or land on a carrier deck without crashing."
The 20 subjects in the study were immersed in a virtual reality scenario in which they had to navigate a simulated airplane through rectangular boundaries. Known as a boundary avoidance task, this demanding sensory-motor task model created cognitive stresses, such as making the boxes narrower every 30 seconds, that escalated arousal and quickly resulted in task failure--missing or crashing into the boundary. But when the researchers used neurofeedback, the subjects did better, were able to fly longer while performing the difficult tasks that required high levels of visual and motor coordination.
There were three feedback conditions (BCI, sham, and silence) randomly assigned for every new flight attempt. In the BCI condition, subjects heard the sound of a low-rate synthetic heartbeat that was continuously modulated in loudness as a function of the level of inferred task-dependent arousal, as decoded from the EEG. The higher that level of arousal, the louder the feedback and vice versa. Participants' task performance in the BCI condition, measured as time and distance over which the subject can navigate before failure, was increased by around 20 percent.
"Simultaneous measurements of pupil dilation and heart rate variability showed that the neurofeedback indeed reduced arousal, causing the subjects to remain calm and fly beyond the point at which they would normally fail," says Josef Faller, the study's lead author and a postdoctoral research scientist in BME. "Our work is the first demonstration of a BCI system that uses online neurofeedback to shift arousal state and improve task performance in accordance with the Yerkes-Dodson law."
The Yerkes-Dodson law is a well-established and intensively studied law in behavioral psychology about the relationship between arousal and performance. Developed in 1908, it posits an inverse-relationship between arousal and task performance, that there is a state of arousal that is optimal for behavioral performance in a given task. In this new study, the researchers showed that they could use neurofeedback in real time to move an individual's arousal from the right side of the Yerkes-Dodson curve to the left, toward a state of improved performance.
"What's exciting about our new approach is that it is applicable to different task domains," Sajda adds. "This includes clinical applications that use self-regulation as a targeted treatment, such as mental illness."
The researchers are now studying how neurofeedback can be used to regulate arousal and emotion for clinical conditions such as PTSD. They are also exploring how they might use online monitoring of arousal and cognitive control to inform human-agent teaming, when a robot and a human work together in a high-stress situation like a rescue. If the robot has information on the human's arousal state, it could choose its tasks in a way that reduces its teammate's arousal, pushing her/him into an ideal performance zone.
"Good human-agent teams, like the Navy SEALS, do this already, but that is because the human-agents can read facial expressions, voice patterns, etc., of their teammates to infer arousal and stress levels," Sajda says. "We envision our system being a better way to communicate not just this type of information, but much more to a robot-agent."
###
About the Study
The study is titled "Regulation of arousal via online neurofeedback improves human performance in a demanding sensory-motor task."
Authors are: Josef Faller, Jennifer Cummings, Sameer Saproo, and Paul Sajda, Columbia Engineering.
The work was funded by the Defense Advanced Research Projects Agency (DARPA) and Army Research Office (ARO) under grant number W911NF-11-1- 0219, the Army Research Laboratory under Cooperative agreement number W911NF-10-2-0022, the National Science Foundation under grant IIS-1527747 and the UK Economic and Social Research Council under grant number ES/L012995/1.
LINKS:
Paper: https://www.pnas.org/content/early/2019/03/11/1817207116
DOI: https://doi.org/10.1073/pnas.1817207116
http://engineering.columbia.edu/
https://engineering.columbia.edu/faculty/paul-sajda
Columbia Engineering
Columbia Engineering, based in New York City, is one of the top engineering schools in the U.S. and one of the oldest in the nation. Also known as The Fu Foundation School of Engineering and Applied Science, the School expands knowledge and advances technology through the pioneering research of its more than 220 faculty, while educating undergraduate and graduate students in a collaborative environment to become leaders informed by a firm foundation in engineering. The School's faculty are at the center of the University's cross-disciplinary research, contributing to the Data Science Institute, Earth Institute, Zuckerman Mind Brain Behavior Institute, Precision Medicine Initiative, and the Columbia Nano Initiative. Guided by its strategic vision, "Columbia Engineering for Humanity," the School aims to translate ideas into innovations that foster a sustainable, healthy, secure, connected, and creative humanity.
Journal
Proceedings of the National Academy of Sciences