Abstract
Do basic emotions produce their predicted facial expressions in nonlaboratory settings? Available studies in naturalistic settings rarely test causation, but do show a surprisingly weak correlation between emotions and their predicted facial expressions. This evidence from field studies is more consistent with facial behavior having many causes, functions, and meanings, as opposed to their being fixed signals of basic emotion.
The Facial Expression Program (FEP; Russell & Fernández-Dols, 1997) proposed by Tomkins (1962) and those he influenced (Ekman, 1972; Izard, 1971) assumes that certain facial expressions are each accounted for by the occurrence of one category of basic emotion and that they are adaptations (i.e., innate behaviors with a functional value from our evolutionary past) shared with other primate species. For example, “true” or “Duchenne” smiles are accounted for only by happiness, and only “true” smiles indicate happiness. 1
Empirical tests of the FEP have mostly been recognition studies, but research on categorization (Fugate, 2013), language (Lindquist & Gendron, 2013), and recognition across cultures (Nelson & Russell, 2013) raises doubts about the FEP. Another necessary and more straightforward test of the FEP is the study of the occurrence of facial expressions. Do basic emotions and their predicted facial expression co-occur? This article reviews evidence on whether they co-occur in naturalistic or seminaturalistic settings, and thus complements Reisenzein, Studtmann, and Horstmann’s (2013) review of the evidence from laboratory studies.
The Case for Naturalistic Studies
Laboratory experiments are the bread and butter of research on facial expression. Their clear advantages are that they facilitate causal claims by subjecting many participants to an identical situation, obtaining self-reports of emotions, recording facial behavior in a uniform and often unobtrusive manner, and creating situations that minimize participants’ motivation to hide their expressions. On the other hand, laboratory experiments also have severe limitations. One hour in a laboratory is ill-suited to strong or long-term emotions such as grief. Indeed, ethical and practical concerns rule out all but faint versions of, for example, happiness, fear, sadness, or anger (for some exceptions in emotions such as surprise, disgust, or amusement see Reisenzein et al., 2013).
Naturalistic studies can examine the ecological frequency of specific emotions and facial expressions, as well as their co-occurrence—all without the intervention of an experimenter. Their strengths also include their usefulness for studying strong and long-lasting emotions in situations that could not be created in a laboratory.
Experimental findings must be supported by naturalistic studies. Experiments are “clean,” but they are restrictive in studying facial expression. Naturalistic studies are “dirty,” but their findings are about robustness: they test whether the co-occurrence of facial expressions and emotion is part of existing behavioral repertoires, happens in contexts that have not been established for the sole or primary purpose of conducting research, and would have happened with or without the presence of a researcher (Tunnell, 1977).
Of course, naturalistic studies also have their limitations. They are often considered “dirty” because the situations examined are rarely identical for all participants, no self-reports of emotion are available, recording of facial behavior is nonuniform and often difficult, and naturally occurring situations may invoke social norms for hiding facial expressions. On the other hand, the FEP assumes that facial expressions of basic emotion are communicative adaptations, that is, signals that must be salient to observers, recognizable without self-reports, and—most important—often displayed without being hidden. After all, there would be no adaptive advantage from the ability to produce signals that must be hidden from view. If the FEP is correct, naturalistic studies should find evidence for it.
Is there a Robust Correlation between Basic Emotions and their Predicted Facial Expressions?
We now turn to all the studies that we know of that are naturalistic or seminaturalistic tests of the existence of expressions of basic emotion. Such studies cover a number of highly diverse episodes, such as clinical interviews, complaints, getting lost, play, or sex.
Bonanno and Keltner (1997) studied the correlation between the facial expressions of adults experiencing conjugal bereavement and their reported emotions at the time of displaying the expression. Correlations were obtained by aggregating across emotions of the same valence, but even in this way the correlation was significant but moderate (.45) for negative emotions, and it was nonsignificant for positive emotions (.19). In a second study with bereaved widows, Keltner and Bonanno (1997) found a significant but modest correlation (.34) of reported enjoyment with either Duchenne laughter or Duchenne smiling, but not with non-Duchenne laughter or smiling.
In a third study, Bonanno and Keltner (2004) examined correlations among emotional content (measured in terms of the emotional appraisals reflected in their discourse), self-reported emotion, and facial expressions of sadness, anger, enjoyment, and pride. The expressions of sadness and anger correlated modestly with discourse themes about loss and injustice (.33 and .35), but only for anger did the facial expression correlate significantly with self-report (.44). The positive expressions showed a complex and paradoxical pattern: Duchenne smiles showed no significant correlation with happiness appraisals (.15) or self-reports of joy (.24), but Duchenne laughs correlated significantly but positively with a negative appraisal: injustice (.38).
In a field study of passengers who lost baggage at an airport, Scherer and Ceschi (2000) observed facial expressions and obtained self- and other-reports of emotion. The passengers displayed a heterogeneous set of expressions, with frequencies so low that the authors coded only smiles. Frequency of Duchenne smiles correlated significantly but modestly with self- (.27) and other- (.40) attributions of good humor.
Schützwohl and Reisenzein (in press) carried out a natural experiment in which they reproduced an everyday episode of being lost: Participants found themselves in a little room after going through a door that had led to a corridor a few minutes earlier. Even though 83% of the participants reported strong feelings of surprise, only 17% displayed at least two components of the surprise expression. Furthermore, none of the correlations between participants’ reports of surprise and the components of the surprise expression was significant.
Unfortunately, most of the described tests of covariance between emotion and expression have focused on confirming the existence of the expressions of basic emotion (e.g., smiles and happiness). Only a few naturalistic studies have been aimed at testing not only the covariation between emotion and expressions, but also the existence of other, unpredicted facial behaviors that might be related to the experience of emotion.
Chong, Werker, Russell, and Carroll (2003) found three displays characteristic of Chinese-speaking and English-speaking mothers while interacting with their 4- to 7-month-old babies. The mothers did not know that their facial expressions were being studied. The OOCHIEE display consisted of puckered lips and an open mouth—a caricature of a kiss that was presumably meant to convey love, concern, and emotional availability. WOW was an expression of mock surprise that probably conveyed pride and amazement. JOY was an exaggerated version of the expression of happiness, and probably conveyed a message of playful love.
Fernández-Dols and Ruiz-Belda (1995a) found “facial expressions of sadness” (often accompanied by tears) in extremely happy gold medalist athletes in the noninteractive phases of the Olympics awards ceremony, while Fernández-Dols, Carrera, and Crivelli (2011) found that an experience of intense sexual enjoyment consistently coincided with an expression strikingly similar to that of pain (Prkachin, 1992).
Altogether, the available evidence points to weak correlations between emotions and their predicted expressions in natural settings. Some evidence even points to significant correlations between emotions and unpredicted expressions. In addition to the usual problems with field studies, a potential criticism of these findings is that adult senders might be following display rules that would hide their “true” expressions. However, in all of the studies reviewed so far, care was taken to find situations where this criticism is implausible (moreover, the concept of display rules remains today underspecified, and can therefore be invoked post hoc whenever the FEP prediction fails). In any case, we next turn to evidence for which the potential confounding role of display rules is especially implausible: naturalistic studies with young children who lack the cognitive and social skills presumably required to follow display rules.
Naturalistic studies of young children and babies have also found weak or unpredicted correlations between the reported or inferred emotion of the expresser and the FEP’s predicted facial expressions. For example, Camras (1992) observed the facial expressions of her daughter during her first 9 weeks. She found facial expressions that were not predicted by the emotional content of the situation (e.g., surprise expressions elicited by interesting but familiar stimuli, or during oral exploration), or expressions of different emotions that were produced in unexpected situations and in close temporal proximity (e.g., expressions of pain, anger, and sadness produced in a regular sequence during almost all instances of crying, irrespective of the eliciting circumstances). Camras concluded that these latter sequential expressions constituted a global and nonspecific manifestation of distress.
Another seminaturalistic study, with 3.5-, and 5.5-month-old babies (Reissland, Shepherd, & Cowie, 2002), suggested that the “facial expression of surprise” was likely acquired during early childhood. Only 29% of the 3.5-month-old babies displayed the predicted expression of surprise in response to a jack-in-the-box; however, mothers exclaimed with a higher pitch when the child failed to show the expected expression. At 5.5 months, 67% of babies showed the predicted expression.
In another seminaturalistic experiment, Underwood and Bjornstad (2001) observed 8-, 10-, and 12-year-old children playing a dyadic computer game with each other. They recorded self-reported emotions and expressive responses to peer provocation in 565 episodes. Although the coding of the expressions was impressionistic and of low reliability, the correlation between emotion and their predicted expression was low to nonexistent. Correlations between self-reports of emotion (sad, angry/mad) and facial expressions did not exceed .15, and showed unpredicted patterns. For example, reports of sadness yielded a minimal (.09) but statistically significant positive correlation with angry expressions, whereas reports of anger showed no correlation with angry expressions (.03).
Also in the school context, Ahmed, van der Werf, and Minnaert (2010) studied six 12- to 13-year-old students in their classroom. For six affective states (anger, anxiety, boredom, enjoyment, pride, and shame), the authors obtained self-reports and measured facial expressions, physiological reactions, and appraisals. The coding of expressions and the estimates of correspondence between expressions and self-reports were strongly inclusive (frequency of expressions divided by the frequency of reports for the corresponding emotion at any time of one video segment). Nevertheless, the ratios of convergence were, in the authors’ own words, “not conclusive” (.50 for anger, .38 for anxiety, .40 for boredom, .29 for enjoyment, .19 for pride, and .28 for shame).
All in all, the studies reported in this section support a very weak, nonexistent, or unpredicted relationship between emotion and facial expression. The predicted “facial expressions of emotion” did not prove to be robust readouts of basic emotions in natural settings. Interestingly, the same pattern of findings emerged when professional actors tried to portray typical natural expressions of basic emotion (Carroll & Russell, 1997; Scherer & Ellgring, 2007a, 2007b; cf. Gosselin, Kirouac, & Doré, 1995).
Facial Behavior that Does Not Express a Basic Emotion
Every day, we see smiles, frowns, grimaces, and other facial expressions. How can we reconcile this common everyday experience with the surprisingly weak correlation of emotion with the FEP’s predicted facial expression of that emotion in natural settings? We next turn to evidence on the causes and functions of facial expressions, other than the expression of emotion.
Before doing so, however, we also note that some of these sightings of facial expressions could be false positives. For example, observers were shown videotapes of children in an obviously fearful situation, awaiting an inoculation. The videotapes showed none of the predicted facial expressions of fear, and yet observers remembered having seen them (Fernández-Dols, Carrera, Barchard, & Gacitua, 2008).
Several theories offer accounts of facial expressions—although, now, facial behavior would be the better term—not as readouts of emotion (e.g., Fridlund, 1994; Mead, 1967; Seyfarth & Cheney, 2003). These accounts have in common that they view facial behavior as caused by an interaction among sender, receiver, and environment, and as functioning to influence others’ behavior. Fridlund (1994) specifically pointed out that automatically producing an accurate readout of one’s emotion would often be to the sender’s disadvantage. Sender and receiver often have conflicting interests, but still need to coordinate their behavior.
Naturalistic studies have provided evidence supporting this alternative approach to facial expression. Most of the evidence comes from the study of smiles. The smile is the facial expression with the highest ratings of universal recognition, and therefore the apparently ultimate facial expression of basic emotion. But there is robust evidence (a) that smiles are interactive (i.e., dependent on an audience), and (b) that they have multiple meanings and functions, depending on the context.
The Interactive Nature of Smiles
Over the last 30 years, studies have repeatedly confirmed the interactive nature of smiling. Bainum, Lounsbury, and Pollio (1984) observed 1,847 humorous events that elicited laughter or smiling in 86 3-, 4-, and 5-year-old children. Frequencies of laughter and/or smiling were adjusted for the observed duration of solitary and interactive events; only 5% of the laughter or smiling occurred when the child was alone (i.e., 95% of laughter and smiling were in the presence of others). Kraut and Johnston (1979) examined smiling by bowlers who made a good roll, ice hockey fans cheering their team, and pedestrians on a sunny day. The probability of a smile when interacting with others was respectively .42, .27, and .62, whereas the probability of a smile in noninteractive phases was almost negligible (.04, .12, and .12). Following Kraut and Johnston’s rationale, Fernández-Dols and Ruiz-Belda (1995b) observed Olympic Games gold medalists during the awards ceremonies. Gold medalists displayed smiles during the interactive phases of the ceremony, but virtually no smiles during noninteractive phases. Retrospective emotional reports of a subsample of gold medalists showed that the intensity of happiness was similar in the interactive and the noninteractive phases—a finding that undermines any explanation of the observed facial differences in terms of absence or presence (or different intensity) of emotion. In the same vein, Ruiz-Belda, Fernández-Dols, Carrera, and Barchard (2003) recorded facial expressions of bowlers and soccer fans at times at which the participants reported happiness (typically, after a strike or when their team scored). They found that the mean probability of a smile was .78 and .70 for the interactive phases and .09 and .07 for the noninteractive phases.
The dependency of smiling on a social audience was seemingly challenged by Matsumoto and Willingham (2006, 2009) in their studies of sighted and blind judo fighters in the Athens Olympic Games. Matsumoto and Willingham (2006) analyzed a small portion (190) of the 2,735 photographs taken by a professional photographer. They reported that 29 out of 40 judo winners displayed Duchenne smiles at match completion, and that “most expressions occurred before the athlete turned to face the crowd” (2006, p. 577). However, closer examination of their study reveals that Matsumoto and Willingham did not carry out a direct test of the interactive nature of smiles. More specifically, Matsumoto and Willingham’s photographs were taken in a 15-second period between match completion and the referee’s decision on the winner (2006, p. 571). Because 15 seconds is a relatively long period of time, the winner could well have interacted with the opponent, referees, coach, or spectators when the picture was taken.
To examine this question more closely, Crivelli, Carrera, and Fernández-Dols (2012) attempted two replications of Matsumoto and Willingham’s (2006) study. Instead of photographs, they analyzed video recordings of judo fighters’ expressions in observation periods of 5 and 7 seconds after winning a match, and they also coded whether or not the winner was interacting with someone. The proportion of winners who displayed Duchenne smiles (.21 in the first replication and .15 in the second replication) did not reach statistical significance, in contrast to Matsumoto and Willingham’s findings, and smiling was virtually nonexistent when winners were not interacting with the others (2 out of 33 observed Duchenne smiles in both studies).
Smiles Have Multiple Functions and Meanings
A second group of naturalistic studies, most of them in the ethological tradition, suggests that smiles are better explained as behaviors that satisfy diverse adaptive functions and have a flexible referential value, rather than having a fixed meaning as signals of emotion. 2 For example, smiling has also been found in natural encounters with diverse social functions, such as sexual invitation (Moore, 1985), the marking of sociometric status (Cashdan, 1998), or ambivalent teasing (Keltner, Young, Heerey, Oemig, & Monarch, 1998). Eibl-Eibesfeldt (1989) reported a specific facial expression of affection and acceptance during friendly greetings in Western and remote cultures (such as the Eipo, Yanomami, and Trobriand islanders). The expression consisted of a rapid eyebrow raising (the “eyebrow flash”), typically accompanied by smiling. For Eibl-Eibesfeldt, the function of smiles when combined with an eyebrow flash was social invitation.
Mehu, Grammer, and Dunbar (2007; see also Mehu & Dunbar, 2008) hypothesized that Duchenne smiles are messages of cooperative predispositions to share material resources. Such messages would not only communicate the sender’s cooperative disposition, but also induce cooperative responses on the receiver’s side. The study compared two natural situations (rest times between experimental sessions) in which pairs of friends were successively involved in a noncooperative task (choosing a pseudonym for themselves) versus a cooperative task (sharing the experimental fee). The frequency of Duchenne smiles increased while participants were sharing resources, and senders’ Duchenne smiles increased receivers’ reported happiness. On the other hand, senders’ reports of happiness did not predict either their Duchenne or non-Duchenne smiles, whereas sadness showed a significant but modest correlation (.30) with non-Duchenne smiles.
All in all, these studies provide robust evidence, across different natural settings, that at least one conspicuous type of facial expression (Duchenne and non-Duchenne smiling) is an interactive behavior with multiple functions, rather than a simple readout of a basic emotion.
Conclusion
The reported very weak, nonexistent, or unpredicted relationship between emotion and facial expression is coherent with the experimental evidence (see Fernández-Dols & Ruiz-Belda, 1997; Reisenzein et al., 2013). At best one could claim a very weak correlation between some incomplete forms of prototypical emotional expressions and some hypothetical forms of their proposed underlying basic emotion. Furthermore, experimental evidence on audience facilitation effects on facial expression (for a review see Parkinson, 2005) supports the reported interactive nature of some prototypical expressions, such as Duchenne smiles.
This convergent evidence prompts a first reasonable conclusion: facial expressions cannot be defined as crisp, “true” signals of an emotion, but rather as fast, multiple, and imprecise cues which, nevertheless, are adequate (adaptive) for their senders in a particular situation.
A second, no less reasonable conclusion is that such cues are linked to different mental processes. For example, a single emotional episode might include simultaneous or successive facial movements linked to affective reactions, appraisals, social motives, or strategies of regulation, but also to cognitive processes or cultural conventions.
Footnotes
Author note:
This article was funded by the Spanish Government (Grant PSI 2011-28720).
