Keywords

Nonverbal communication is often considered merely body language, but researchers have defined nonverbal communication as almost all of human communication except the spoken or written word (Knapp 1972). We broadly define nonverbal communication as the transfer and exchange of messages in any and all modalities that do not involve words. One of the major ways by which nonverbal communication occurs is through nonverbal behaviors, the dynamic behaviors that occur during communication that include facial expressions, gestures, tone of voice, and body postures.

Of the various nonverbal behaviors, facial expressions are one of the most complex signal systems in the body. The face is a channel that can produce both voluntary movements and involuntary reactions. These two facets make research on the face complicated. Facial signals that are involuntarily produced are universal, whereas voluntary or learned facial expressions can vary across cultures. Voluntary and involuntary facial signals often confuse communicators, but they can be differentiated.

This chapter explores the scientific evidence for the universal expression and recognition of facial expressions of emotions. We first discuss the history of research on facial expressions of emotion, including early debates and positions, and review the ample scientific evidence for the universal expression and recognition of facial expressions of emotions. We then discuss how the universality of facial expressions of emotion informed our understanding of emotions, and in particular, a category of emotions known as basic emotions. We also discuss how facial signals of emotions interact with culture to demonstrate how biologically innate expressions can be modified by cultural learning. As a first step of understanding this area, we begin by describing the original debate about the universality of facial expressions of emotion.

3.1 Early Debates Concerning the Universality Versus Culture Specificity of Facial Expressions of Emotion

Modern day work in this area started with Darwin’s (1872) seminal thesis about emotion and expression, who posited that all humans, regardless of race or culture, possessed the ability to express emotions on faces in similar ways because emotions and their expressions are evolutionarily adaptive and biologically innate. Darwin insisted that emotions exist panculturally and that all humans possess the ability to express emotions in the same ways through their faces and other nonverbal channels such as voice. He also claimed, in his principle of serviceable associated habits, that facial expressions are the residual actions of more complete behavioral responses. Relying on advances in photography and anatomy (de Boulogne 1862/1990), Darwin engaged in a detailed study of the muscle actions involved in emotion and concluded that the muscle actions are universal and their precursors can be seen in the expressive behaviors of nonhuman primates and other mammals.

Darwin’s work drew heavy criticism, especially from anthropologists such as Birdwhistell (1970). They noted vast differences in expressive behavior across cultures and concluded that facial expressions could not be universal. Instead they argued that emotional expressions had to be learned differently in every culture, and just as different cultures have different spoken languages they must have different expressive languages of the face as well.

Between Darwin’s original writing and the 1960s, only seven studies attempted to test the universality of facial expression. Unfortunately, these studies were inconclusive (Ekman et al. 1972). Thus, an influential review of the literature (Bruner and Tagiuri 1954) concluded that facial expressions were not universal but learned. It was not until almost a century after Darwin that the first systematic evidence for the universality of facial expressions of emotion appeared.

3.2 Evidence for the Universality of Facial Expressions of Emotions

In the mid-1960s, Sylvan Tomkins resurrected interest in the study of emotions and faces with the publication of his landmark volumes entitled Affect, Imagery, and Consciousness (Tomkins 1962, 1963). Tomkins conducted the first study demonstrating that facial expressions were reliably judged to be associated with certain emotional states (Tomkins and McCarter 1964) and later studies showed consistent findings (Ekman 1972; Ekman et al. 1969; Izard 1971). Those initial findings were criticized, however, because the evidence for universality (i.e., the high levels of cross-cultural agreements in judgments) might have occurred because of influences of social media (e.g., TV) and shared visual input (e.g., Hollywood movies, magazines, etc.). To address these potential limitations, Ekman and his colleagues conducted two studies with two visually isolated, preliterate tribes in the highlands of New Guinea (Ekman and Friesen 1971; Ekman et al. 1969). In the first study, the tribespeople could reliably recognize facial expressions of emotion (anger, disgust, fear, happiness, surprise, sadness) posed by westerners; in the second study, films of the tribespeople expressing emotions were shown to Americans who had never seen New Guineans before, and the Americans were able to recognize the expressions of the New Guineans. Thus, the ability to recognize facial expressions of emotion did not occur because of learning through mass media or other shared visual input as the New Guineans had had no exposure to the outside world.

One limitation of the above studies was that they all examined judgments of facial expressions of emotions and did not investigate their spontaneous production. Friesen’s (1972) study, however, addressed this limitation. In that study, American and Japanese participants were presented with neutral and stressful films. During the experiment, the same expressions associated with the six emotions mentioned previously were identified via facial coding systems. The coded facial behaviors from the participants in the study corresponded to the facial expressions tested in the previous judgment studies supporting the universality of emotions. Members of both the American and Japanese cultures showed the same expressive patterns, providing the first evidence that facial expressions of emotion were universally produced.

Since the original evidence for the universality of facial expressions of emotions described above, numerous studies have replicated their findings (Ekman et al. 1987; Matsumoto 2001; Matsumoto et al. 2002, 2008). These studies have included both studies examining judgments of facial expressions of emotion (Elfenbein and Ambady 2002) as well as studies investigating the production of facial expressions of emotion (Matsumoto et al. 2008a, b). For example, one recent study examined the expressions of 84 judo athletes from 35 countries at the 2004 Athens Olympic Games (Matsumoto and Willingham 2006). The spontaneous facial expressions of winners and losers that were first observed at the completion of their final medal match were consistent with the universal expressions. In particular, winners displayed Duchenne smiles while losers displayed sadness, disgust, anger, and other negative emotions. Duchenne smiles are smiles that involve not only the smiling muscle (zygomatic major), which raises the lip corners, but also the muscles surrounding the eyes (orbicularlis oculi), which raise the cheeks, thin the eyes, and narrow the eye cover fold. That these spontaneous expressions were documented in a real-life, naturalistic setting and were produced by individuals from many different cultures of the world spoke to the universality of those facial expressions of emotion.

In addition to the six universal emotions, contempt was identified as a universal expression in various studies (Ekman and Heider 1988; Matsumoto 1992, 2005). The evidence for the universality of the contempt expression was first documented in a study involving 10 cultures and later replicated in an additional seven cultures, including the Minangkabau in Sumatra, Indonesia (Ekman and Heider 1988; Matsumoto 1992). Thus, today there is strong evidence for the universal facial expression of seven emotions (see Fig. 3.1 for seven basic emotions and their universal expressions). In the next section, the source of the evidence of universal facial expressions of emotion will be introduced in detail.

Fig. 3.1
figure 1

Seven basic emotions and their universal expressions. Reprinted with permission © David Matsumoto 2008

Full size image

3.3 The Source of Universal Facial Expressions of Emotions

Merely documenting the universality of emotional expression in many cultures around the world does not resolve questions concerning the source of the universality. Facial expressions of emotion may be universal because of at least two reasons. First, emotional expressions may be a biologically innate skill that all humans are born with. Or second, they may be a skill that is learned in the same way all around the world in different cultures through a mechanism known as culture constant learning. Demonstrating cross-cultural agreement in either the production or judgment of expressions does not address which of the two sources may produce the agreement; other methodologies are necessary to do so including studies of blind individuals, twins, infants, and animals. In this section, we briefly review some of the representative studies in these areas, all of which point to a biologically innate source of universality.

3.4 Studies with Blind Individuals

One of the critical challenges to the notion of the biological innateness of emotion is that humans can easily (or entirely) learn and imitate emotional expressions from others. Blind individuals who are limited in observing and imitating others’ behaviors compared to sighted people are a suitable group to explore the pure effect of biologically wired systems on the universality of emotions. This is especially true for studies involving congenitally blind individuals because they are expected to be have limited social learning about how to produce sophisticated facial muscle movements of each emotion because they could not visually learn them from birth.

Many similarities between blind and sighted individuals in their spontaneous facial expressions of emotion have been reported in studies of congenitally blind individuals. For example, researchers have measured the spontaneous facial behaviors of blind individuals when emotions were aroused studying blind children (Cole et al. 1989) and adults of many different cultures (Galati et al. 2001; Galati et al. 2003) and have reported similarities in facial expressions between blind individuals and nonblind individuals. This evidence is compelling to show the existence of universal emotions because it is impossible for blind persons to simply imitate others and produce the complicated facial expressions involved in complex muscle combinations fired spontaneously in less than a second when they experience an emotion. They would not have these automatic reactions unless they were born with the capability of experiencing and expressing the emotions in a certain way.

More recent studies reported similar findings when comparing congenitally and noncongenitally blind judo athletes at the 2004 Athens Paralympic Games with the sighted athletes from the 2004 Olympic Games (Matsumoto and Willingham 2009). The blind athletes, who came from 23 cultures, produced the same facial configurations of emotion as sighted athletes in the same emotionally evocative situations. The study also found high concordance between the blind and sighted athletes in their expressions. Winners displayed all types of smiles, especially Duchenne smiles, more frequently than the defeated athletes, who displayed more disgust, sadness, and combined negative emotions. When receiving the medal, all athletes smiled, but winners of the last match (gold and bronze medalists) displayed Duchenne smiles more frequently than did the defeated (silver medalists), who displayed more non-Duchenne smiles. Because congenitally blind individuals could not have possibly learned to produce these expressions by imitation, we believe that these studies provided strong evidence for a biologically based emotion–expression linkage that is universal to all people of all cultures.

3.5 Evidence from Twin and Family Studies

Another source of evidence for the possible biological origins of emotion–expression linkages comes from studies of twins and family relatives. Facial behaviors of blind individuals are more concordant with their kin than with strangers (Peleg et al. 2006); in this study’s facial movement analysis during an individual interview, the correlation between movements of 21 congenitally blind subjects with those of their 30 relatives especially in relation to such expressions as sadness or anger was significantly more similar to each other than with nonfamily members. The results provided evidence for a unique family facial expression signature, indicating a hereditary component for facial expressions. Moreover, some facial expressions in response to emotionally provocative stimuli are more concordant among monozygotic twin pairs than dizygotic twins (Kendler et al. 2008). These studies are strongly suggestive of a hereditable, genetic component to facial expressions of emotion.

3.6 Evidence from the Developmental Literature

More evidence for the biological base of facial expressions of emotion comes from the developmental literature. The same facial musculature that exists in adult humans exists in newborn infants and is fully functional at birth (Ekman and Oster 1979). Infants have a rich and varied repertoire of facial expressions including those that signal not only emotional states but also interest and attention (Oster 2005, 2010). There is widespread consensus that smiling; distaste, the infant precursor of adult disgust; and crying, the universal signal of sadness/distress, occur in neonates (Oster 2005).

There is some controversy as to when other differentiated and discrete negative emotions occur. Some authors suggest that discrete negative emotions exist from birth or shortly thereafter and emerge according to a maturational timetable (Izard 1991; Izard and Malatesta 1987; Tronick 1989). Others suggest that infants, at least within the first year of life, display relatively undifferentiated or modulated negative expressions, which ultimately transform into more differentiated, discrete expressions later (Camras et al. 2003; Oster 2005). Discrete expressions of anger and sadness have been reported in the early part of the second year of life (Hyson and Izard 1985; Shiller et al. 1986). Regardless, by the time of preschool, children display discrete expressions of the other emotions as well (Casey 1993). It is difficult to conceive of how this occurs if the children did not have the biological capability to do so in the first place, which again points to the innateness of facial expressions of emotion.

3.7 Evidence from Nonhuman Primates

The facial expressions considered to be universal among humans also have been observed in nonhuman primates (de Waal 2003). Chimpanzees have a fully functional facial musculature that, while not as differentiated as that of humans, include the same muscles that are used in human emotional expressions (Bard 2003; Burrows et al. 2006). Moreover, the chimpanzee facial musculature produces many of the same appearance changes as does the human musculature, according to a comparison of the human and chimpanzee versions of the Facial Action Coding System (Vick et al. 2007; Shepherd et al. 2012). Chimpanzees as well as Rhesus Macaques can categorize facial expressions of emotion much as humans do (Parr et al. 2008; Parr et al. 2010; Waller et al. 2012). Consistently, chimpanzees produce distinct laughs depending on contexts and interactants like human beings (Davila-Ross et al. 2011).

Following all the evidence stated above, we speculate that the emotions that are universally expressed and recognized are dominantly biologically wired. These findings have led to research that has suggested that the universally expressed and recognized emotions belong to a specific class of emotions that has certain specific and unique characteristics. This class of emotions is known as basic emotions. In the next section, we discuss how the universality of facial expressions of emotion has informed our understanding of emotions in general and of basic emotions in particular.

3.8 Emotions and Basic Emotions

The documentation of the evidence for the universality of some facial expressions of emotion led to the increased study of emotions in general, and especially to the study of universal emotions that have a biological basis, which are called basic emotions. The increased attention to emotion has led to the need for defining “emotion.” However, it is difficult to define emotions in a simple word that can be equally understood by and for everybody even after so much research has been conducted in emotion and nonverbal behavior. For us, emotions are as transient, bio-psycho-social reactions to events that have consequences for our welfare and potentially require immediate action (Matsumoto and Hwang 2012).

Basic emotions include the emotions that have been shown to be universally expressed and recognized and are akin to biological systems and reactions. Basic emotions are discrete, unique, and rapid information processing systems that aid us to act with minimal conscious deliberation (Tooby and Cosmides 2008; Izard 2009). If humans did not have emotions, they could not make rapid decisions concerning whether to attack, defend, flee, care for others, reject food, or approach something useful. Emotion response is adaptive and aids in our ultimate survival and allows us to take action immediately without much thinking, and its expression promptly conveys this valuable information to others. This does not mean that emotions continue to occur all the time because humans consciously or unconsciously scan and evaluate our environments constantly but only selected stimuli evoke an emotional reaction (Ekman 2003; Ellsworth and Scherer 2003; Frijda et al. 1989; Roseman 1984; Roseman et al. 1995; Scherer et al. 2001). If selected events are expected to have any consequences, they trigger emotion in order to prime action and motivate behavior (Frijda et al. 1989; Tomkins 1962, 1963). Once emotions are triggered, they coordinate multiple bodily and psychological systems such as perception, attention, inference, learning, memory, goal choice, motivational priorities, physiological reactions, motor behaviors, and behavioral decision making (Cosmides and Tooby 2000).

The expressions of those biologically innate, basic emotions of course interact with culture, and there are many spaces for cultural variations in how to socially express facial reactions after these emotions have been triggered, based on social/cultural norms. And we also do not mean to imply that basic emotions are the only emotions that humans experience. There are many other emotional states that humans experience that are much more culturally grounded, both in terms of origin as well as moderation. For those emotions that can be culturally and socially cultivated, what triggers them in the first place, what happens when they are triggered, and their meanings afterward are influenced by culture (see Matsumoto and Hwang 2012, for a more detailed discussion). In the next section, we address cultural variations in the display of the universal facial expressions of emotion.

3.9 Cultural Differences in Expressing Facial Emotions

Despite the existence of universal facial expressions of emotion, people around the world use the universal expressions differently. The first evidence for cultural differences in expressions was in a second condition in Friesen’s study (1972). In that study, Americans and Japanese viewed the stressful films alone, and then in the presence of an older, presumably higher-status male experimenter. In the latter condition, the Americans continued to express their negative emotions consistently regardless of the other’s presence, whereas the Japanese were more likely to smile in the presence of others than when they were alone.

The concept of cultural display rules was used to explain these cultural differences in emotional expressions. Display rules are social norms learned early in childhood to help individuals manage and modify their emotional expressions depending on social circumstances. They provide a way of behaving that is consonant with the normative behaviors within a social role. They serve a vitally important function in culture by helping to regulate emotional expressions, which aids social coordination and group survival (Matsumoto and Juang 2013). When the participants in Friesen’s (1972) experiment viewed the stressful films alone in the first condition, there was no reason for display rules to modify the expressions because there was no one else present; thus, the Americans and Japanese produced the same facial expressions (providing evidence for the universality of facial expressions of emotion, as discussed earlier). When viewing the films in the presence of a higher-status person, however, display rules were activated. Because the Japanese had a display rule not to express their negative feelings to a higher-status person, they masked their negative feelings by smiling. Because the Americans did not have such a display rule, they did not change their expressions much. Thus, cultural differences in the expressions were produced because of the different social contexts in which the expressions occurred.

After the original inception of the concept of display rules, cross-cultural research on them was dormant until Matsumoto’s (1990) study examining display rules in Americans and Japanese, and a similar study documenting differences in display rules among four ethnic groups within the USA (Matsumoto 1993). Later Matsumoto and colleagues created the Display Rule Assessment Inventory (DRAI), where participants choose one of six behavioral responses (corresponding to the ways expressions are managed in real life, as described above) when they experience different emotions with family, friends, colleagues, and strangers (Matsumoto et al. 1998, 2005). They demonstrated cultural differences in display rules and provided evidence for its internal and temporal reliability and for its content, convergent, discriminant, external, and concurrent predictive validity with personality.

Matsumoto et al. (2008a, b) then administered a more comprehensive version of the DRAI in over 30 countries, examining universal and culture-specific aspects to display rules, and linking the cultural differences to culture-level individualism (vs. collectivism). Most countries’ means on overall expression endorsement suggested a universal norm for expression management. Individuals of all cultures had a display rule norm for greater expressivity toward in-groups than toward out-groups, indicating another universal effect. Collectivistic cultures were associated with a display rule norm of less expressivity overall than individualistic cultures, suggesting that overall expression management for all emotions is central to the preservation of social order in these cultures (Fig. 3.2). This finding is commensurate with the behavioral findings from previous findings (Friesen 1972; Matsumoto and Kupperbusch 2001; Matsumoto et al. 2009). Individualism was also positively associated with higher expressivity norms in general and for positive emotions in particular. Individualism was positively associated with endorsement of expressions of all emotions toward in-groups, but negatively correlated with all negative emotions and positively correlated with happiness and surprise toward outgroups. Cumulatively, these findings suggest a fairly nuanced view of the relationship between culture and display rules that varies as a function of emotion, interactant, and overall expressivity endorsement levels.

Fig. 3.2
figure 2

Graphical representation of the relationship between individualism and overall expressivity endorsement in Matsumoto et al. (2008a, b)

Full size image

As studies documenting cultural differences in expression peppered the literature (Argyle and Cook 1976; Edelmann et al. 1987; Gudykunst and Nishida 1984; Gudykunst and Ting-Toomey 1988; Matsumoto and Kupperbusch 2001; Noesjirwan 1978; Szarota 2010; Waxer 1985), a consensus emerged that when emotions are aroused, the displays are either universal or culture specific, depending on context. A recent study (Matsumoto et al. 2009), however, showed that emotional displays can be both for the same person in the same context, if displays are examined in sequence across time. In this study, changes in Olympic athletes’ expressions after their initial reactions were classified into one of several regulation strategies, and the relationship between these expressive styles and cultural variables such Hofstede’s (2001) cultural dimensions (i.e., country level scores on the dimensions Individualism, Power Distance, Uncertainty Avoidance, Masculinity, and Long Term Orientation) and country demographics such as population density and affluence were examined. Although the athletes’ initial reactions were universal, their subsequent expressions were culturally regulated and associated with population density, affluence, and individualism. Athletes from urban, individualistic cultures expressed their emotions more; athletes from less urban, more collectivistic cultures masked their emotions more. The average length of time from an initial, universal emotional expression to a culturally moderated modification was less than 1 s.

In summary, culture plays an important role in how to manage emotions and their expressions when they occur. However, spontaneous and immediate reactions of universal emotions are often produced prior to the cultural reactions that modify the initial emotional expressions in socially desirable ways. The universal expressions of emotions are very rapid and often unconsciously produced, mostly occurring in less than a second. These two facets of universal emotions and their expressions highlight the critical part of research on the topic; ongoing arguments about whether biological emotions exist or not have occurred because the subsequent expressions of emotions are easily mistaken as evidence of cultural variations of emotional expressions (Matsumoto and Hwang 2012). How to disentangle these different expressions of emotions would be the key to understanding the universality of emotions in research.

3.10 Future Directions

In the future, researchers need to more clearly define emotions and the specific emotion domains in which they are interested in order to approach conducting research and understanding their findings with greater specificity and sophistication. For example, research based on evolutionary theory, as stated above, may focus on spontaneous expressions on the face that immediately occur right after emotional events because facial expressions are part of the evolutionarily derived, biologically innate package of emotion components. Research based on self-reports of emotions that are based on memory or recall, however, represents a different domain of emotion, measuring emotions after immediate reactions. Thus, comparing findings from the two different approaches may not make sense. Researchers should fairly evaluate studies considering the domain of emotion being examined.

Second, evolutionary theory does not neglect the effects of context. In contrast, evolutionists believe that universal behavioral reactions and cultural-specific management of those reactions interact in specific contexts. Yet, very few studies have systematically attempted to directly examine these effects of context. To disentangle the unidentified and seemingly contradicted layers produced by different contexts, examining how culture specifically interacts with universal behavioral reactions across a wide variety of targeted contexts is an on-going task in future research on facial expressions of emotions.

Furthermore, the stimuli used in studies to test judgments of facial expressions of emotions should be carefully considered. This is a critical issue to resolve because many previous studies showing culture-specific findings in facial expressions of emotions used stimuli that may have been questionable in terms of their ecological validity to portray emotional expressions. Also, the expressive intensities of the stimuli are often not considered, thus rendering definitive conclusions about statistically significant differences in agreement rates among tested emotions is very difficult.

More studies concerning spontaneous facial expressions of emotion in real-life situations should be considered for future research. Although research in laboratory-based, experimental environments is understandable and has a scientifically unique meaning, conducting studies in real-life contexts provide solid and practical results that can enhance the applicability of research findings to the real life. This is especially true of the facial expressions of basic emotions, which are rooted in spontaneous behavioral reactions that usually occur in natural settings than in experimental ones. Therefore, scientists must continue to make efforts to conduct research in real-life contexts.

Finally, examining the training of facial expression recognition in practical applications will be beneficial in a practical research perspective (e.g., see Matsumoto and Hwang 2011, for a study on the benefits of facial expression training). It is a fact that basic emotions are universal and commonly recognizable across cultures. Thus, being skillful in understanding and catching other’s emotions on the face in social interactions may be useful in many practical and applied contexts. As individual variations always exist, there must be some room for people to acquire and develop their ability of recognizing other’s emotions on face. Acknowledging the possible consideration of facial expressions of emotion to real life will elucidate how to apply research findings appropriately in reality.

3.11 Conclusion

Darwin (1872) originally suggested that emotions and their expressions had evolved across species, were evolutionarily adaptive, biologically innate, and universal. Darwin’s idea has been fully examined in numerous studies as discussed above. We started by reviewing the original and subsequent evidence for the universal expression and recognition of facial expressions of emotion. We then reviewed evidence concerning the source of the universality of facial expressions on emotions, examining research findings on blind individuals, twins, infants, and children as well as nonhuman primates. Those reports provided a solid and consistent conclusion that universal facial expressions of emotions are biologically innate. This characteristic of facial expressions of emotion has led to a greater understanding of the class of emotions known as basic emotions, which we then described. At the same time, we did not intend to undervalue the role of culture in moderating emotions and their expressions and discussed cultural display rules that indicated how cultures cause individuals to modify the initial universal facial expressions of emotions on social circumstances.

Not only are the seven basic universal facial expressions panculturally recognized, but cultures are similar in other aspects of emotion judgment as well. For example, there is cultural similarity in judgments of relative intensity among faces; that is, when comparing expressions, people of different countries agree on which is more strongly expressed (Ekman et al. 1987; Matsumoto and Ekman 1989). There is also cross-cultural agreement in the association between perceived expression intensity and inferences about subjective experiences, as well as in the secondary emotions portrayed in an expression (Biehl et al. 1997; Ekman et al. 1987; Matsumoto and Ekman 1989). This agreement may exist because of overlap in the semantics of the emotion categories, antecedents and elicitors of emotion, or in the facial configurations themselves.

There are cultural differences in emotion judgments as well, such as in the absolute levels of recognition across cultures; for example, Americans typically have higher agreement rates when judging emotions than other countries (Biehl et al. 1997; Elfenbein and Ambady 2002; Matsumoto 1992; Matsumoto et al. 2002). There are also cultural differences in ratings of the intensity of expressions; for example, Japanese tend to rate expressions lower in intensity than Americans (Biehl et al. 1997; Ekman et al. 1987; Matsumoto 1992; Matsumoto et al. 2002). Other cultural differences have led to interesting debates and controversies (see in particular Chap. 4 on the possible ingroup advantage of emotion recognition). However, it is extremely important to recognize that the cultural variations in how to display facial expressions is not interpreted as cultural control over immediate behavioral reactions on the face as these are likely very difficult to control. Instead, culture is an essential guideline for people to socially modify their more voluntarily based facial behaviors, which occur after the immediate behavioral reactions, in order to smooth their social interactions for well-being and social survival.