image: This composite image shows a happy face as it was originally photographed (left) and the same photo retouched to emphasize changes in color. Researchers at The Ohio State University found that people are able to correctly identify other people's feelings up to 75 percent of the time -- based solely on subtle shifts in blood flow color around the nose, eyebrows, cheeks or chin. The study demonstrates a never-before-documented connection between the central nervous system and emotional expression in the face. view more
Credit: The Ohio State University
COLUMBUS, Ohio--Our faces broadcast our feelings in living color--even when we don't move a muscle.
That's the conclusion of a groundbreaking study into human expressions of emotion, which found that people are able to correctly identify other people's feelings up to 75 percent of the time--based solely on subtle shifts in blood flow color around the nose, eyebrows, cheeks or chin.
The study, published in the Proceedings of the National Academy of Sciences this week, demonstrates a never-before-documented connection between the central nervous system and emotional expression in the face. It also enabled researchers to construct computer algorithms that correctly recognize human emotion via face color up to 90 percent of the time.
"We identified patterns of facial coloring that are unique to every emotion we studied," said Aleix Martinez, cognitive scientist and professor of electrical and computer engineering at The Ohio State University.
"We believe these color patterns are due to subtle changes in blood flow or blood composition triggered by the central nervous system. Not only do we perceive these changes in facial color, but we use them to correctly identify how other people are feeling, whether we do it consciously or not."
The researchers are patenting the computer algorithms, and hope they will enable future forms of artificial intelligence to recognize and emulate human emotions. They have also formed a spin-off company, Online Emotion, to commercialize the research.
This is the latest in a series of studies in which Martinez and his team have identified unique forms of human facial expression. In prior work, they identified several, previously unknown facial expressions produced though unique patterns of muscle movements, including the "not face," a frown that they determined to be a universal embodiment of negation in human communication. What's unusual about their latest work is that it involves color changes that communicate emotion without any movement of facial muscles.
For this study, the researchers first took hundreds of pictures of facial expressions and separated the images into different color channels that correspond to how human eyes see color--a red-green channel and a blue-yellow channel. Via computer analysis, they found that emotions like "happy" or "sad" formed unique color patterns.
Regardless of gender, ethnicity or overall skin tone, everybody displayed similar patterns when expressing the same emotion.
To test whether colors alone could convey emotions--without smiles or frowns to go along with them--the researchers superimposed the different emotional color patterns on pictures of faces with neutral expressions. They showed the neutral faces to 20 study participants and asked them to guess how the person in the picture was feeling, choosing from a list of 18 emotions. The emotions included basic ones like "happy" and "sad" as well as more complex ones such as "sadly angry" or "happily surprised."
"Admittedly, these images look weird," Martinez said. "But we told people to go ahead and guess from the list of emotions what emotion they thought those faces were conveying. And they guessed right most of the time."
About 70 percent of the time, participants thought that a neutral face that had been colorized to look happy actually conveyed happiness. They thought faces colorized to look sad were actually sad about 75 percent of the time, and neutral faces colorized to look angry were actually angry about 65 percent of the time. They perceived the emotion, even though their only clue was the color superimposed on the face, with no facial movements.
Next, researchers showed participants facial expressions of happiness, sadness and other emotions. This time, however, they mixed up the colors on some of the images. For example, they sometimes took a happy face and put angry colors on it, or vice versa. Participants noticed that something about the mixed up images looked "off," even if they weren't sure what was wrong.
"Participants could clearly identify which images had the congruent versus the incongruent colors," Martinez said.
The researchers used what they learned to develop computer algorithms that could detect emotions via face color. Given photographs of people expressing emotion, the computer could match face color to feeling better than the human study participants could.
Happiness was the easiest emotion for the computer to recognize by color alone, and it detected the emotion with 90 percent accuracy. Several emotions related to happiness, such as "happily surprised," came in second at around 85 percent. Anger was detectable by color alone 80 percent of the time, and sadness 75 percent of the time. Fear was recognizable 70 percent of the time, and the least recognizable emotion was "fearfully disgusted," which registered only 65 percent of the time.
The findings may ultimately inform research in computer science, cognition, neuroscience and even human evolution.
Language is replete with idioms that equate face color to emotion. When we argue until we're "blue in the face," we're angry. If we look "green around the gills," something has triggered our disgust. "Blushing brides" are the very picture of happiness.
This study shows that there is some physiological truth to these old sayings, though the color scheme of human emotion is not as simple as a monochromatic blue, green or red.
"There's a little bit of every color everywhere," Martinez said. Touches of red, green, blue and yellow characterize every emotion--just in slightly different amounts or locations around the face.
Disgust, for instance, creates a blue-yellow cast around the lips, but with a red-green cast around the nose and forehead.
It's all the more impressive, then, that our brains are able to decipher the meaning of these color arrangements in an instant. We see a smiling person with red cheeks and temples (with a little blue around the chin) and we automatically read their emotion as "happy." But the same face with a slightly redder forehead and slightly less blue chin registers as "surprised."
Humans just may be alone among primates for our ability to change facial color due to emotion. It's hard to tell, because other primates' faces are covered by hair.
Yet, in humans, nowhere else on the body are there so many blood vessels so close to the surface of the skin than our faces. The fact that we evolved much less facial hair than the apes suggests that our early ancestors may have found some advantage to letting their blushes show.
What about fake blushes--ones created with makeup?
"People have always said that we use makeup to look beautiful or younger, but I think that it is possible that we actually do it to appear happier or create a positive perception of emotion--or a negative perception, if you wanted to do that," Martinez said.
Now that we know a little more about the varied colors of human facial expression, he suggested an intriguing new possibility: We could make "smart" cosmetics to play up certain emotions--or conceal them.
###
Martinez's co-authors on the study included C. Fabian Benitez-Quiroz, a postdoctoral researcher, and Ramprakash Srinivasan, a doctoral student, both in electrical and computer engineering. All three hold appointments in Ohio State's Center for Cognitive and Brain Sciences. Their research was funded by the National Institutes of Health and the Human Frontier Science Program.
Contact: Aleix Martinez, 614-688-8225; Martinez.158@osu.edu
Written by Pam Frost Gorder, 614-292-9475; Gorder.1@osu.edu
Journal
Proceedings of the National Academy of Sciences