Human Faces Can Express at Least 21 Distinct Emotions Alice Park @aliceparkny March 31, 2014

Distinct facial muscles were used to express
compound emotions

Leading scientific thinkers of their time, such as Aristotle, Rene Descartes, Guillaume Duchenne, and Charles Darwin, have long promoted the idea that there are a handful of basic emotions that people express. In recent decades, that group has crystalized into six core emotions: happiness, surprise, sadness, anger, fear, and disgust.

But there are clearly many shades of gray between those emotions. For example, there’s the happy-because-I’m-eating-ice cream and the happy-because-I-just-learned-I-got-a-surprise-marriage-proposal looks, each of which is slightly different.

That’s what intrigued Aleix Martinez, associate professor of electrical and computer engineering at Ohio State University. “Six seemed a small number given the rainbow of possibilities of feeling and expressing emotions,” he says.

Martinez wanted to know whether compound emotions, such as happy surprise, were expressed using the same muscle movements of both happiness and surprise, or whether the expression involved a unique set of muscles that represented some amalgam of the two.

What he and his colleagues found was that the human face makes 21 different emotional expressions – and each is different from the other. While some represented combinations of emotions, each differed in terms of which muscles were involved.

And surprisingly, these facial expression patterns were remarkably consistent across all 230 volunteers. For example, each showed happy surprise in the same way that was distinct from both happiness and from surprise, and different still from angry surprise.

Martinez broke down the facial expressions of 230 volunteers by applying his engineering strategies. He and his colleagues gave each of the students, staff, or faculty members who enrolled in the study different scenarios and asked them to show how they would react in each one. They were told, for example, that they had just learned they had been accepted to a graduate program, that someone had told them a disgusting, but still funny joke, or that they had just smelled something bad. The volunteers were allowed to practice their facial expressions in front of a mirror before Martinez took pictures of their reactions.

He then computer-analyzed each of the 5,000 images, breaking them down by which facial muscles the participants used. These were first defined in 1978 by psychologist Paul Ekman, who codified facial expressions in the Facial Action Coding System (FACS) by action units, or muscles or groups of muscles that went into making facial expressions – such as lip parts (for showing disgust), showing teeth (for expressing happiness), mouth stretch (for fear), or eyelid tightening (for anger).

Not only are people able to make compound expressions consistently, but we are also able to read these emotions pretty accurately as well. When we see someone who looks angrily surprised, for example, we know that they aren’t happy about whatever unexpected event they just experienced. “We don’t know for sure how much is learned or innate in expressing these emotions,” says Martinez. “But what we do know is that a big component has to be innate because otherwise different people would use different muscles to show the same emotion.”

The results dovetail with recent findings that some cultures may already recognize these more refined, and numerous expressions of emotions. In a study among the remote Himba tribes in Namibia, for example, researchers at Northeastern University reported that when they provided tribe members with images of facial expressions of emotions, rather than creating six neat piles of emotions, the members created many more. For them, happy could be interpreted as anything from happy to laughing to wonder.

That should help to improve how studies of human emotions are conducted in different cultures, says Martinez; his results suggest that the underlying ability to express emotions may be similar around the world, but cultural biases may simply define emotions in different ways – much the same way that babies are born with the capacity to speak and make sounds for any language, but are trained to speak their native tongue by what they hear around them.

Even more exciting for mental health experts is the possibility that this work can teach them about when the processing and expression of human emotions goes awry such as in depression. “It’s important to understand which are categories of emotions that we have,” says Martinez, so that experts can recognize the pathological ones. Expressing emotions is generally a transient exercise, lasting milliseconds or up to a minute. Those that linger longer, for hours or even days, tend to be considered moods, and once these emotions persist for days, weeks or months, they can become the subject of mental illness. So distinguishing how people express themselves, and defining the categories of well-known and well-recognized emotions, could lead to better understanding of which emotions can become detrimental and even harmful.

Twenty facial expressions of basic and compound emotion categories. From left to right and top to bottom, these categories correspond to: happy, sad, fearful, angry, surprised, disgusted, happily surprised, happily disgusted, sadly fearful, sadly angry, sadly surprised, sadly disgusted, fearfully angry, fearfully surprised, fearfully disgusted, angrily surprised, angrily disgusted, disgustedly surprised, hatred, and awed. Credit: Aleix M. Martinez



To Really Read Emotions, Look at Body Language,
Not Facial Expressions
By Laura Blue Nov. 30, 2012


We like to think we can read people like a book, relying mostly on tell-tale facial expressions that give away the emotions inside: the way the brows lift slightly with alarm, or the crow’s feet that crinkle with a wide smile. But when it comes to the strongest emotions, we read much less from facial expressions than we think we do. In fact, even though we believe it’s the face that tells the story, we’re typically reading something very different: body language and social cues.

That’s the new, counterintuitive finding from a study published this week in the journal Science. Researchers from Princeton, New York University, and the Hebrew University of Jerusalem presented volunteer study participants with a series of pictures showing people experiencing extreme emotion, either positive or negative. The images included professional tennis players who had just won or lost a point in a major match, as well as people undergoing nipple piercing, and those in the throes of orgasm.

In some of the images, researchers would only show the study participants a face; in others, only a body; and in others still, both the body and the face. You might think it’d be obvious from a face whether someone is in pain (having a nipple pierced) or whether he has just won Wimbledon. But it turns out it isn’t.

“The striking finding was that our participants had no clue if the emotion was positive or negative, when they were judging isolated faces,” says lead study author Hillel Aviezer from Hebrew University in an email response discussing the findings. “By contrast, when they were judging the body (with no face), or the body with the face, they easily differentiated positive from negative expressions.”

The findings are doubly surprising because the study participants themselves were convinced that they recognized the emotions from the faces, not from body language or contextual cues.

“They even had their own ‘mini theories’ about what part of the face was most important – but this was all an illusion,” Aviezer says.

He adds that we do, of course, read a great deal of salient day-to-day emotional information from faces — but only in certain situations. The reliability of that transmission, for example, appears to break down when emotions are at their strongest. The face contorts. We can tell that something major has happened, but it’s tough to tell that something is dramatically positive or devastatingly negative.

In the article, Aviezer and his colleagues liken the muscles of the face to an audio speaker. As the volume is pushed to its maximum, the quality of the signal becomes poor and the message becomes harder to pick out.

There is a similar analogy to the emotional signals that we hear. We know that the sound of gentle laughing conveys happiness and a gentle sob conveys sorrow. But most of us find it difficult to distinguish a shriek of joy from a shriek of fear. Since we often hear sounds without another contextual clue to explain them — overheard from an adjacent room — we tend to accept that we can’t tell positive from negative emotion in all cases. But we almost never see faces without body language and a setting to accompany them. So we may fail to realize that the face is less informative than usual when emotions run high.

Aviezer says his results suggest that the classic textbook distinction between positive emotions and negative emotions may need to be revised to incorporate the possibility that they may share more underlying physiological features than we thought. That could have implications for theories that link emotions to economics, social psychology, and neuroscience.

“I also think the findings may have some clinical applications,” he says. “Consider populations such as individuals with autism or various neuropsychiatric disorders. We know these people often have difficulties with recognizing facial expressions,” he says. “Until now we have been trying to help them by training them to better understand the isolated faces. But our work suggests that perhaps we should zoom out a bit and teach them how to recognize emotions from the full person in context.” That could provide a broader range of therapies from which autistic children can draw, and, perhaps even benefit.

Emotions May Not Be So Universal After All
Alice Park @aliceparkny March 6, 2014

Our current understanding of facial expressions could
be specific to Western cultures

The Many Faces of Megan

Simon Gerzina Photography—Getty Images/Flickr RF


From a very young age, infants have a way of making their feelings known – contorted faces and howls indicate their displeasure with a meal or a damp diaper, a gummy smile their contentment, and a furrowed brow their puzzlement over a new discovery such as their thumb.

While it seems logical that these expressions are universal, the latest study suggests they may not be. In fact, expressions of the major emotions – happiness, sadness, anger and the like, may be strongly culturally driven.

Maria Gendron, a post doc in the lab of psychology professor Lisa Feldman Barrett at Northeastern University, visited remote tribes in Namibia to come to that conclusion. Gendron spent 18 days with the Himba, a people with little exposure to the Western world. When members were asked to sort photos of six people making six facial expressions of emotions, she expected to see six neat piles of images.

Instead, she found that the tribal members created a multitude of piles, with some images appearing in more than one. The same thing happened when she played vocal sounds of emotions – the same sound appeared joyful to some and more negative to others. When she and Barrett repeated the experiment in Boston, there was more unanimity in the sorting.

That sunk the idea that emotional expressions were universal; the Himba, for example, saw what Westerners would view as happy expressions as reflecting anything from happy to laughing to wonder.

They believe that the notion of universal facial expressions emerged from flaws in the way the original theory was developed in the 1970s, when American psychologist Paul Ekman traveled to Papua New Guinea to conduct a study similar to the one Gendron did. Ekman, however, asked the participants to match images of facial expressions to six words or scenarios depicting the emotions. That constraint, Barrett and Gendron, believe, implied a universality in emotional expressions that may not have existed.

The findings aren’t the first to nibble away at some conventional wisdom about how we express emotions. Recently, Scottish researchers at the University of Glasgow questioned the long-held belief that humans expressed six emotions – happy, surprised, afraid, disgusted, angry and sad. Instead, they found that similarities among some of them mean there are likely only four basic emotional expressions – happy, sad, afraid/surprised, and angry/disgusted.

How we read and express how we feel, it seems, is strongly influenced by how the people around us express how they feel. We feel what we see.