image: This is a graph of audio effects on human emotions. view more
Credit: Graph by science team
Researchers have created a digital audio platform that can modify the emotional tone of people’s voices while they are talking, to make them sound happier, sadder or more fearful. New results show that while listening to their altered voices, participants’ emotional state change in accordance with the new emotion.
"Very little is known about the mechanisms behind the production of vocal emotion", says lead author Jean-Julien Aucouturier from the French National Centre for Scientific Research (CNRS), France.
"Previous research has suggested that people try to manage and control their emotions, for example hold back an expression or reappraise feelings. We wanted to investigate what kind of awareness people have of their own emotional expressions."
In an initial study using a novel digital audio platform, published in Proceedings of the National Academy of Sciences (PNAS), participants read a short story aloud while hearing their own altered voice, sounding happier, sadder or more fearful, through a headset.
The study found that the participants were unaware that their voices were being manipulated, while their emotional state changed in accordance with the manipulated emotion portrayed. This indicates that people do not always control their own voice to meet a specific goal and that people listen to their own voice to learn how they are feeling.
"The relationship between the expression and experience of emotions has been a long-standing topic of disagreement in the field of psychology", says Petter Johansson, one of the authors from Lund University, Sweden. "This is the first evidence of direct feedback effects on emotional experience in the auditory domain."
The emotional manipulations were created by digital audio processing algorithms that simulate acoustic components of emotional vocalisations. For example, the happy manipulation modifies the pitch of a speaker's voice using pitch shifting and inflection to make it sound more positive, modifies its dynamic range using compression to make it sound more confident, and modifies its spectral content using high pass filtering to make it sound more excited.
The researchers believe this novel audio platform opens up many new areas of experimentation.
"Previously, this kind of emotion manipulation has not been done on running speech, only on recorded segments", explains Jean-Julien Aucouturier. "We are making a version of the voice manipulation platform available as open-source on our website, and we invite anyone to download and experiment with the tools."
For applications outside academia, co-author Katsumi Watanabe from Waseda University and the University of Tokyo in Japan considers that the platform could be used for therapeutic purposes, for example for mood disorders by inducing positive attitude change from retelling affective memories or by redescribing emotionally laden events in a modified tone of voice. It might also be possible to enhance the emotional impact of Karaoke or live singing performances, or maybe to alter the emotional atmosphere of conversations in online meetings and gaming.
###
The study was conducted by researchers at the Science and Technology of Music and Sound Lab (STMS), (IRCAM/CNRS/UPMC) and the LEAD Lab (CNRS/University of Burgundy) in France, Lund University in Sweden, and Waseda University and the University of Tokyo in Japan.
Full citation: Covert digital manipulation of vocal emotion alter speakers' emotional states in a congruent direction. Proceedings of the National Academy of Sciences (PNAS). Aucouturier, J-J., Johansson, P., Hall, L., Segnini, R., Mercadi, L., Watanabe, K. (2015).
Download the open-access study: http://www.pnas.org/cgi/doi/10.1073/pnas.1506552113 (Embargoed until Monday 11 January, 15.00 pm US EST)
To download the digital audio software, please visit: http://cream.ircam.fr
About IRCAM: http://www.ircam.fr/
About French National Centre for Scientific Research (CNRS): http://www.cnrs.fr/index.php
About Lund University: http://www.lunduniversity.lu.se
Journal
Proceedings of the National Academy of Sciences