In Your Face: Startle to Emotional Facial Expressions Depends on Face Direction

Emotional visual stimuli seem to receive a prioritized processing in relation to nonemotional ones (The Cambridge Handbook of Human Affective Neuroscience, 2012). As such, these affective stimuli have been widely used to elicit emotions in research, and most used are pictures of emotional scenes. Negative emotional scenes typically produce more intense reactions compared with neutral and positive scenes (Cuthbert, Bradley, & Lang, 1996; Lang, Bradley, & Cuthbert, 1998; Vrana, Spence, & Lang, 1988). Furthermore, pictures containing emotional faces have often been included in the broader category of emotional scenes (Lang & Greenwald, 1988). However, the effects of emotional facial expressions are not as distinct as the effects of emotional scenes (Alpers, Adolph, & Pauli, 2011; Paulus, Musial, & Renn, 2014). In addition, pictures of faces may vary in relevance to the observer based on head direction. That is, an emotional face directed at you is arguably more relevant than one turned in another direction. In the present study, the effects of different emotional facial expressions as well as the effect of face direction was investigated.

In contrast to the extensive use of emotional scenes in psychophysiological research, there are few published studies utilizing exclusively emotional faces as stimuli. In addition, the limited research on the topic has revealed inconsistent results employing both electrodermal responses (Dimberg, 1982; Merckelbach, van Hout, van den Hout, & Mersch, 1989; Vrana & Gross, 2004) and heart rate (Dimberg, 1982; Johnsen, Thayer, & Hugdahl, 1995; Vrana & Gross, 2004). Utilizing electrodermal responses, some studies have found similar reactions to angry and happy facial expressions (Dimberg, 1982; Dimberg & Petterson, 2000; Vrana & Gross, 2004). Considering heart rate, two studies have found the expected deceleration to angry facial expressions and acceleration to happy expressions (Johnsen et al., 1995; Vrana & Gross, 2004). However, Vrana and Gross (2004) found deceleration also for neutral faces, and Hess, Philippot, and Blairy (1998) did not find any differences between angry and happy expressions.

In the present study, the startle reflex was used as an index of emotional activation. The startle reflex is a simple defensive response to sudden and intense stimuli (e.g., a loud noise) that in humans involves a rapid closure of the eyelids. It has been well demonstrated that the amplitude of the startle eye blink varies with the emotional content of the background stimulus viewed simultaneously (Lang, Bradley, & Cuthbert, 1990), and it seems to be produced by the priming of the startle circuitry by the amygdala (Davis, 1992). Similar, Angrilli et al. (1996) found that the right amygdala was vital for emotional modulation of the startle response. The typical emotional modulation is increased startle to negative pictures and, to some lesser extent, inhibited startle to positive pictures (Cuthbert et al., 1996; Lang et al., 1998; Vrana et al., 1988). As such, there is a close relation between startle potentiation and valence, and in addition, a relation between the amount of potentiation and arousal for negative valence pictures (Bradley, Codispoti, & Lang, 2006). However, research on startle in relation to emotional facial expressions has revealed a less clear pattern (Alpers et al., 2011; Paulus et al., 2014). Anger seems to consistently potentiate startle, but other than that, the question about which facial stimuli potentiates startle is still unsettled.

Some studies have investigated questions related to startle responding to facial stimuli. Among these, Balaban (1995) found potentiated startle to angry faces and inhibited startle to happy faces in human infants, whereas Waters, Neumann, Henry, Craske, and Ornitz (2008) did not find modulated startle to neutral and angry facial expressions in four- to eight-year-old children. Anokhin and Golosheykin (2010) found startle potentiation to negative facial expressions (i.e., angry and fearful), with a stronger effect in females. Hess, Sabourin, and Kleck (2007) and Paulus et al. (2014) found startle potentiation to angry emotional faces, but only when the expressers were males. Springer, Rosas, McGetrick, and Bowers (2007), on the other hand, found potentiated startle to angry faces regardless of the sex of the expresser. Furthermore, Alpers et al. (2011) found increased startle to angry facial expressions compared with neutral, but not compared with happy expressions. Dunning, Auriemmo, Castille, and Hajcak (2010) morphed pictures of faces and found potentiated startle to angry faces compared with neutral ones, but only for maximally angry expressions. Even fewer studies have investigated the psychophysiological effect of fearful facial expressions and the differences between subtypes of negative expressions. Nonetheless, Grillon and Charney (2011) found enhanced startle responses to fearful faces but only in a threat situation. Springer et al. (2007) compared the effect of angry and fearful faces and found potentiated startle to angry faces, but not to fearful ones.

The only startle study we are familiar with investigating the effect of gaze direction found greater responding to pictures where the model gazed directly at the observer (Lass-Hennemann et al., 2009). However, this study investigated the effect of pictures of nude females on the startle reaction in men, thereby investigating the effect of a distinct subtype of pleasant stimuli not directly comparable to the stimuli utilized in the present study. Adams and Kleck (2003) investigated behavioral data and argued that facial displays of emotions and gaze direction combines to signal approach or avoidance. They found that anger and happy expressions were more quickly recognized in combination with a direct gaze, and that fear and sad expressions were more quickly decoded with an averted gaze. Later, Adams, and Kleck (2005) found support for a notion that direct gaze would enhance the perception of approach-oriented emotions, and averted eye gaze would enhance the perception of avoidance-oriented emotions. Following this logic, one could expect larger startle responses to anger and happy facial expressions with a frontal directed face, and larger startle to fearful faces with averted facial direction.

In an exploratory part of the experiment, we investigated the startle response to facial stimuli at early lead intervals. A few studies have investigated early lead interval responses (50–500 ms) to pictures of emotional scenes (Bradley et al., 2006; Bradley, Cuthbert, & Lang, 1993; Duval, Lovelace, Aarant, & Filion, 2013). These studies found smaller startle response to more emotionally arousing stimuli, compared with neutral stimuli. This effect, which is opposite to the larger startle response to emotionally arousing stimuli at late intervals, is probably caused by more prepulse inhibition (PPI) to the emotional stimuli. PPI is the effect of which a nonstartling stimulus precedes the startle probe by a short duration and reduces the startle response (Graham, 1975) and is an index of protective processing of the prepulse. Thus, in the present study, the pictures of emotional faces would serve as the prepulse, and an inhibition of the startle response would be an index of the processing of these pictures. Bradley et al. (2006) suggested that affectively engaging stimuli produces more inhibition as they are more demanding to the attentional system. Bradley et al. (1993) argued that more interesting and complex pictures leads to longer periods of “processing protection” compared with simpler stimuli. Thus, producing more PPI and, as such, less startle. Duval et al. (2013) investigated the startle response to pictures of happy, neutral, and angry faces at 300, 800, and 3500 ms. At the early 300 ms lead interval, they found smaller startle response to angry compared with neutral faces, but only when the expresser were males. For female expressers, the results where opposite, with smaller responses to neutral faces compared with angry.

In the present study, we investigated the startle reflex during presentations of pictures displaying angry, fearful, happy, and neutral facial expressions. Faces were directed (toward the viewer) and averted (away from the viewer), and startle eliciting stimuli were presented at an early lead interval (250 ms) and at a late lead interval (3500 ms). Based on the startle literature reviewed earlier, we hypothesized that startle would be potentiated to angry faces compared with neutral and fearful at a late lead interval when faces were directed forward. In addition, based on the work of Adams and Kleck (2003, 2005), we expected increased startle to forward directed approach motivated emotions (anger and happiness) and increased startle to averted directed avoidance motivated emotions (fear). In addition, at the early lead interval, we expected less startle to emotionally arousing stimuli. However, this part of the experiment was more exploratory as the previous research on this effect is scarce. In addition to the startle test, participants evaluated the pictures on scales for valence, arousal, and domination.

Method

Results

Discussion

In the present study, we compared the startle response to pictures of faces with a forward direction (directed) with faces directed to the side (averted). Our analyses of the long lead interval condition revealed that directed angry faces elicited greater startle than averted angry faces. Larger startle to directed faces was also evident for happy faces. For the fearful faces, there was a trend toward greater startle to the averted compared with the ones directed forward. As far as we know, this is the first study comparing startle reactions to faces directed to the side to the standard pictures of faces directed forward. These results fit well with the findings from Adams and Kleck (2005) that direct gaze enhances the perception of approach-oriented emotions (anger and happiness), and averted eye gaze enhances the perception of avoidance-oriented emotions (fear).

These results could also be in line with findings from the imaging literature: N’Diaye et al. (2009) and Hadjikhani et al. (2008) found stronger amygdala activation to fearful facial expressions with an averted gaze, and N’Diaye et al. (2009) found stronger amygdala activation angry facial expressions with a directed gaze. N’Diaye et al. (2009) argued that amygdala activation follows a self-relevance model, where a forward directed angry face may signal a threat toward the observer, and where a fearful face directed to the side may signal a possible threat in the environment.

Happy faces are not expected to signal threat. However, a happy face directed at you is more relevant to you than one directed to the side. As such, it could be that startle responding to facial stimuli mimics the activation of the amygdala and is in line with the “relevance detector model” (Sander et al., 2003), which postulates a general role for the amygdala in recognizing biologically and socially important information. In addition, Alpers et al. (2011) argued that a facial expression of happiness, without the social context, may not yield enough information about the situation to prime motivated behavior. In contrast to emotional expressions of anger which signal threat and where the organism may be tuned to automatically respond accordingly (Dimberg & Öhman, 1996).

The significant differences between facial expressions with face direction left also fits with both the model from Adams and Kleck (2005) and the self-relevance model (N’Diaye et al., 2009), as fearful facial expressions with left face direction elicited greater startle than happy and angry facial expressions with the same face direction. However, the significant greater startle response to neutral facial expression with left face direction compared with angry expressions with the same direction is harder to explain based on the self-relevance model. These results were not as expected. That is, averted neutral expression produced a similar rate of startle potentiation compared with averted fearful faces, and greater startle than averted anger. As far as we know, this is the first study investigating the startle reaction to faces with a neutral expression directed to the side. As such, we can only speculate to the reason behind this finding. However, Adams and Kleck (2005) found that an averted gaze increased the likeliness for a neutral facial expression to be attributed as a fearful expression. As such, it could be that a neutral facial expression within an averted face was decoded as an avoidance-motivated emotion, and, subsequently, produced similar startle responses as averted fear.

Gaze direction may also have been an important factor in the present study. Indeed, pictures of faces directed to the side indirectly produced an averted gaze in comparison to facial expressions with frontal face direction and forward gaze direction. As it is impossible to disentangle the effect of face direction and gaze direction in the present study, it remains a question for further research whether face or gaze direction is the crucial factor in this regard.

The results for the frontal face direction at the late lead interval revealed greater startle to angry faces, compared with fearful and neutral. These results are similar to the results of Hess et al. (2007) and Springer et al. (2007) who also found potentiated startle to angry facial expressions. Moreover, these results are in line with Alpers et al. (2011), who found increased startle to angry facial expressions compared with neutral ones, but not compared with happy expressions. Similar to Springer et al. (2007), we found potentiated startle to angry faces but not fearful ones. In sum, the results of the present study are in line with previous findings for frontal directed faces and startle probes with standard lead interval.

The results also revealed a significant main effect of lead interval with greater startle to the long lead interval compared with the short lead interval. This is probably caused by prepulse inhibition (PPI; Graham, 1975) reducing the responses for the short lead interval. Bradley et al. (1993) showed that pictures can serve as prepulses and reported PPI to neutral, pleasant, and unpleasant pictures at a 300 ms lead interval. In the present study, follow-up analyses to the interaction of Face Direction by Emotion by Lead Interval revealed greater startle to a neutral facial expression than a fearful facial expression, when faces were directed forward. This is in line with the results from Bradley et al. (1993) and Bradley et al. (2006) who found less PPI, and hence larger startle responses, to neutral stimuli compared with pleasant and unpleasant stimuli.

The notion from Adams and Kleck (2005) that gaze direction signaling the same emotional motivation enhances the perception of emotion in the face could possibly explain the other results from the PPI data. That is, startle responses were larger to neutral facial expression when the face was directed forward compared with the same facial expression when the face was directed to the side. As mentioned earlier, it is possible that a neutral face directed to the side is decoded somewhat similar to a fearful face in the same direction. For face direction to the side, angry facial expression is possibly less emotional significant compared with fearful and neutral expression. However, the larger response to averted angry compared averted happy is not as fitting to the Adams and Kleck (2005) model. In sum, the results from the startle data for the short lead interval seems to add moderate support to the idea that emotional prepulses inhibits startle more than less emotional prepulses. However, as Duval et al. (2013) showed, other aspects not investigated in the present study, such as the sex of the expresser, could influence the results regarding picture of faces as prepulses.

In the valence data, the main effect of Emotion was significant, and contrast analyses revealed that the happy faces was rated as more positive than the neutral ones. Neutral was rated as more positive than fearful, which in turn was rated as more positive than angry. It is notable that angry faces was rated as most negative, and this could possibly explain why angry faces elicited more startle than fearful ones. However, as the startle data showed a much more complicated pattern of results, this is not likely. It is also worth noting that angry and fearful facial expressions with left face direction had a similar valence rating as their frontal-face-direction equivalents. Therefore, it is not likely that the larger startle to facial expressions with frontal face direction could be explained by valence.

Analysis of the arousal data revealed a main effect of emotion. Follow-up analyses revealed more arousal to angry and fearful facial expressions compared with neutral and happy. It is not common for happy facial expressions to be equal in arousal to neutral expressions. However, it is possible that the Norwegian words used in this study describing the end points on the VAS for arousal were not identical in meaning to the English counterparts, which may explain this atypical result. There is no good Norwegian word for arousal and the word used is more akin to “activation.”

As described in the previous two paragraphs, taking face direction into account, the effect of the emotional facial expressions on startle and the subjective evaluations of the same expressions diverged. That is, startle was greater to angry facial expressions with frontal directed face compared with angry facial expression with the face directed to the side. In the subjective data (valence, arousal and dominance), there was no such difference. The same was found for happy faces. For fearful faces, there was a trend toward greater startle when the faces were directed to the side. Such a trend was not apparent in the subjective data. A similar difference between startle and subjective evaluations of valence, arousal, and dominance was reported by Paulus et al. (2014). They found potentiated startle to angry male faces compared with angry female faces, but no difference in subjective evaluations between genders. In the present study, the explanation may be that the participants were unable to report differences in subjective data to the more complex stimuli combining emotional stimuli and face direction. The participants may not have been aware of the emotional difference between these stimuli, despite the difference in startle responding. It has been shown that startle responses have been modulated by negative emotional pictures even when the participants failed to label the valence (Reagh & Knight, 2013). Similarly, studies from our lab has shown potentiated startle to a tone conditioned stimulus following fear conditioning, but no difference in subjective evaluations to the tone following conditioning (Åsli & Flaten, 2008; Åsli, Kulvedrøsten, Solbakken, & Flaten, 2009).

To conclude, the present study showed that pictures of angry facial expressions directed forward elicited greater startle responses than pictures of neutral and fearful expressions. This is in line with previous research reporting greater startle to pictures of angry faces. However, in the present study, pictures of emotional faces were also presented at an angle from the side. Responses to these pictures were different, with greater startle to fearful faces compared with angry and happy. In sum, these results lend support to the notion that startle responses to emotional faces follow the same pattern as behavioral data from Adams and Kleck (2005). That is, a direct gaze enhances the perception of approach-oriented emotions, and a averted gaze enhances the perception of avoidance-oriented emotions. However, as the present study did not differentiate between face and gaze direction, these results should be followed up by further experiments designed to investigate these factors.