Eye-tracking study on facial emotion recognition tasks in individuals with high-functioning autism spectrum disorders

Abstract

The eye-tracking experiment was carried out to assess fixation duration and scan paths that individuals with and without high-functioning autism spectrum disorders employed when identifying simple and complex emotions. Participants viewed human photos of facial expressions and decided on the identification of emotion, the negative–positive emotion orientation, and the degree of emotion intensity. Results showed that there was an atypical emotional processing in the high-functioning autism spectrum disorder group to identify facial emotions when eye-tracking data were compared between groups. We suggest that the high-functioning autism spectrum disorder group prefers to use a rule-bound categorical approach as well as featured processing strategy in the facial emotion recognition tasks. Therefore, the high-functioning autism spectrum disorder group more readily distinguishes overt emotions such as happiness and sadness. However, they perform more inconsistently in covert emotions such as disgust and angry, which demand more cognitive strategy employment during emotional perception. Their fixation time in eye-tracking data demonstrated a significant difference from that of their controls when judging complex emotions, showing reduced “in” gazes and increased “out” gazes. The data were in compliance with the findings in their emotion intensity ratings which showed individuals with autism spectrum disorder misjudge the intensity of complex emotions especially the emotion of fear.

Keywords

autism spectrum disorders emotion perception eye tracking facial emotion recognition social cognition and social behavior

Introduction

Most people are aware of the three characteristic dimensions that mark the diagnosis of autism spectrum disorder (ASD), namely, the typified deficits in social interaction, communication, and restricted or repetitive behaviors (American Psychiatric Association, 2013). However, few have noticed the specific elaboration in the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5) diagnostic criteria on their deficits in regard to their emotional processing as, “… deficits in social-emotional reciprocity … reduced sharing of interests, emotions, or affect ….” Facial expressions reveal an individual’s momentary emotional state. Therefore, the ability to accurately recognize others’ emotions via their facial expressions is an essential social interactive skill (Bradley et al., 2001). Emotional processing deficits can be viewed from at least three aspects, namely, accuracy in identifying emotions, visual processing preference (configural or featural) in emotion perception, and the ability to judge emotional intensity. All these abilities are essential for successful social interaction.

Literature review

Because facial emotion recognition (FER) is emotion-dependent (Herba and Phillips, 2004), it is often used to assess emotional processing and identification. Among the six basic emotions (namely, happiness, sadness, anger, fear, surprise, and disgust), sadness and happiness tend to be recognized earliest and are therefore regarded as simple emotions, whereas surprise and fear tend to be recognized last and are therefore regarded as more complex emotions (Herba and Phillips, 2004). Past research findings on FER in ASD are controversial. Some studies have found intact FER, while others have concluded profound deficits in FER in ASD (Harms et al., 2010). For instance, with regard to complex emotions, intact performance in FER was reported in a high-functioning ASD group (Adolphs et al., 2001; Loveland et al., 2008), whereas others found intact accuracy in FER among low-functioning ASD groups as well (Grossman et al., 2000; Neumann et al., 2006; Ogai et al., 2003; Rosset et al., 2008; Rutherford and Towns, 2008; Teunisse and De Gelder, 2001). On the other hand, a number of experimental studies found reduced accuracy in identifying emotions in adults with ASD, especially for negative emotions (Ashwin et al., 2006; Bal et al., 2010; Corden et al., 2008; Howard et al., 2000; Wallace et al., 2008). In particular, some studies have pointed specifically to the inaccurate identification of fear (Howard et al., 2000; Pelphrey et al., 2002), while others reported abnormal judgments of faces showing complex emotions in general (Adolphs et al., 2005).

Two things stand out about ASD patients’ visual processing of faces. First, ASD groups appear to rely on featural processing, emphasizing individual components of a face (such as the shape or the size of the eyes), rather than configural processing, which emphasizes the spatial relationships among the single facial components (such as distance of the mouth from the nose) (Renzi et al., 2013). Featural processing appears to emerge before configural processing developmentally (Mondloch et al., 2002, 2010). FER becomes more configural and less feature-based in typical development with increases in age and maturation (Herba and Phillips, 2004).

It has been proposed that in the FER process, individuals with ASD tend to use localized featural processing, in contrast to the global configural processing strategy adopted by their controls (Behrmann et al., 2006). Individuals with ASD might learn to match specific emotions with specific facial features and therefore show more consistency in recognizing simple emotions than recognizing complex emotions. For example, it is easier to perform simple matching of curled-up or curled-down lips for happiness and sadness, in contrast to the other less distinguishable emotions, such as surprise and anger, which demand more perception of information conveyed by the eyes (Boraston et al., 2007; Boucher and Lewis, 1992; Humphreys et al., 2007).

For featural processing, attention to the eyes of others is crucial for early social development as well as for lifelong social adaptation, and this ability is already found to be weak in individuals with ASD as early as toddlerhood (Jones et al., 2008). In particular, eye-tracking studies have shown that individuals with ASD show reduced preference for the eye regions, as they spent less time fixating on eyes in FER than controls or did not use information from the upper aspects of the face as effectively during emotion identification (Corden et al., 2008; Gross, 2008; Pelphrey et al., 2002; Spezio et al., 2007). Hence, it has been suggested that the poor performance of individuals with ASD on emotion perception tasks, such as recognizing a facial expression of fear, might be linked to a reduced tendency to look at the eyes because the identification of fear relies more heavily on the eyes than other emotions (Adolphs et al., 2005; Dalton et al., 2005; Pelphrey et al., 2002).

The second thing that stands out about ASD groups’ visual processing of faces is that ASD patients tend to use an inflexible, rule-based approach to making judgments about emotion. Rutherford and McIntosh’s study reported that individuals with ASD were more tolerant of exaggeration in facial expression because they preferred using a rule-based emotion perception strategy. Hence, by analyzing the strategy preference in emotional perception, differences in face-processing performance were expected from various ASD subgroups and their controls (Rutherford and McIntosh, 2007). It was found that typically developing (TD) individuals showed greater acceptance of average facial expressions than exaggerated facial expressions. This could be explained by the fact that people in natural daily interactions often express their emotions in a more subtle manner than those in the photo test, where expressions are exaggerated. Therefore, individuals with ASD are likely to experience problems in everyday social interactions even when they demonstrate intact performance on an FER test because they rely on extreme intensity as a cue for emotion recognition.

Thus, rather than using FER accuracy testing alone, it may be more relevant and valid to assess emotional perception in individuals with ASD by assessing the ability to accurately judge emotional intensity, using expressions of differing intensity (i.e. morphing from neutral to fully expressive) or mixed expressions (i.e. morphing from one emotion to another). The arousal test is one way to assess such an ability, by presenting various degrees of facial expression stimuli to detect differences between the ASD and control groups. Again, conflicting results have been documented. In studies that utilized morphed stimuli, one found diminished FER in ASD (Homer and Rutherford, 2008) and the other found no significant difference between the two groups (Teunisse and De Gelder, 2001).

Study aim

The aim of this study was to compare the emotion perception style and the emotion intensity judgment ability of the TD individuals and those with high-functioning ASD using FER of daily facial expressions in various degrees of intensity. It is commonly known that attention-deficit hyperactivity disorder (ADHD) often exists as a co-morbid condition for ASD. The co-morbid rate of ADHD among individuals with ASD has been estimated as 30%–80%, and the presence of ASD in the context of ADHD has been estimated as 20%–50% (Ames and White, 2011). Therefore, it is possible that differences between ASD and TD groups in earlier research might have been attributable to co-morbid ADHD. In this study, we tested whether the ASD-only group and the co-morbid ASD/ADHD groups differed from their TD peers in terms of emotion perception styles and emotion intensity judgment. First, we investigated the accuracy in emotion recognition via the FER test. Second, we examined the emotional perception style, namely, featural versus configural processing, using eye-tracking data to compare gaze behaviors between groups in terms of which facial parts were the focus of visual processing. Furthermore, emotion perception was examined by comparing judgments of facial emotion intensity with the arousal test.

Methods

Participants

Ethics approval from the Institute and informed consent from the participants were obtained prior to the conduct of the study. The control group was recruited from 10 elementary schools from throughout the three Hong Kong districts by convenience sampling. An invitation letter cum consent form was sent to each parent of children in primary grade 3 through grade 6 in the participating schools. The clinical group was referred by local centers that provided intervention to children with ASD (some of whom had co-morbid ADHD). Once written consent was received by mail or email, the parents were contacted via phone, and a 2-h assessment appointment was scheduled for each child participant. As convenience sampling was adopted, no attempt was made to match the groups on gender or age. If the child was being prescribed medication for controlling their ADHD symptoms, the parents were advised not to administer the medication on the assessment day.

Information about the sample is provided in Table 1. The control group consisted of 29 participants with ages ranging from 8.67 to 11.88 (age mean (M) = 10.04, standard deviation (SD) = 0.94) years. The high-functioning ASD group consisted of 26 participants and was subdivided into two subtype groups, namely, the ASD-only group of 11 participants (age M = 9.63, SD = 1.89, ranging from 7.08 to 12.50 years) and the co-morbid ASD-ADHD group of 15 participants (age M = 9.67, SD = 1.05, ranging from 8.00 to 12.00 years). There was no significant difference in age across groups, although the percentage of males did vary across groups. Participants were local Asian Chinese speakers and passed all screening criteria as explained below. No participants had any visual problems or were in need of corrective eyeglasses.

Table 1.

Details of participants included in the eye-tracking study.

		Group analysis									Group comparison
		Control			ASD-only			ASD-ADHD
		N	Mean	SD	N	Mean	SD	N	Mean	SD
Gender	Male	16			8			15
Gender	Female	13			3			0
Age			10.04	0.94		9.63	1.89		9.67	1.05	F(2, 52) = 0.70, p = 0.50
Raven’s rank			3.72	1.73		3.27	1.74		3.47	2.13	F(2, 52) = 0.27, p = 0.77

SD: standard deviation; ASD: autism spectrum disorder; ADHD: attention-deficit hyperactivity disorder.

Raven’s rank: 1 = excellent ability; 2 = above-average ability; 3 = average ability; 4 = below-average ability; and 5 = weak ability.

Screening measures

Two screening measures were adopted to classify participants into the ASD, ASD-ADHD, and control groups: (1) Raven’s Standard Progressive Matrices (Raven et al., 1983) and (2) medical diagnostic report.

Raven’s Standard Progressive Matrices Test (Raven)

Raven’s test assesses the visual reasoning abilities of children from ages 5.5 to 16 years (Raven et al., 1983). In this study, indexed raw scores relative to chronological age in an Asian population were converted to percentile scores that correspond to five percentile ranks, giving an indication of how well the child performed relative to same-age peers. More specifically, those who score at the percentile rank of IV or V are classified as being at a below-average level, those at the rank of I at an above-average level, and those at the rank of II or III at an average level. In this study, only children who were ranked at the average level of II or III were included in the analyses, so the three groups were matched with an average performance of rank III. Table 1 shows the mean percentile ranks of the three study groups.

Medical diagnostic report

The high-functioning ASD group received a formal medical diagnosis of ASD and co-morbid diagnosis of ADHD from licensed medical consultants using DSM-5 criteria. These assessments were conducted as part of usual diagnostic procedures at the centers where recruitment took place.

Experimental measures

Eye-tracking apparatus

The eye-tracking study used TX300 Eye Tracker with infrared corneal reflectance technology to directly measure the scan path of the participants. It was available in the research laboratory of the Institute where the researchers were teaching from April to December 2014. Raven’s pictures and face emotion photos were presented sequentially on a computer monitor with screen resolution of 1920 × 1080 pixels. Calibration was done in order to follow the participant’s eye movements before each eye-tracking test was recorded. The participant was asked to sit upright and look at specific calibration dots on the screen. The resulting information was then integrated with the eye model, and the gaze point for each image sample was calculated. The eye-tracking calibration procedures were consistent across the experimental and control groups.

Three measures of gaze behaviors were calculated: (1) fixation duration—data from each trial were analyzed to determine how much time the participant’s gaze fell within the eye, mouth, or other region of the face; (2) fixation gaze points—the number of fixations in and out of the stimuli was computed and analyzed; and (3) scan-path pattern—the patterns of visual attention were recorded in visual scan-path plots for further analysis.

Eye-tracking FER tasks

The photo set was produced by local Asian professional actors because most existing photo valence tests used non-local faces, which might add a confounding variable of unfamiliarity to the test results when all the participants were Asian (Ekman and Friesen, 1971; Ekman and Rosenberg, 1997; Miles and Johnston, 2006; Peace et al., 2006; Tottenham et al., 2009). The full photo set consisted of 24 color human face photographs, which was further subdivided into three sets, namely, positive valence, negative valence, and complex emotions. The first two sets consisted of 20 male or female facial emotional expressions (i.e. 10 photos for each gender), which were rated by participants as falling into five ranks of emotional orientation from very positive, positive, neutral to negative and very negative. The third set consisted of four face photographs showing complex emotions of surprise, anger, fear, and disgust. All the photographs in this study were close-cropped to show only the face from the chin to the eyebrows. For each face, the areas of interest (AOIs) were defined prior to data collection, by drawing a rectangular box around each area. Four AOIs were defined for each face as mouth, two eyes, and the rest of the face areas (Figure 1). Gaze points that fell within the four defined AOIs were used to calculate fixation duration. The scan paths of the entire photo test period were also selected for data analysis, and the percentage of time for which participants looked at the AOIs and the number of gaze points in and out of the AOIs were calculated.

Figure 1.

Four areas of interest for face expression recognition task.

Each participant had to answer three questions for each face photo viewed, using adapted FER emotion orientation and intensity ratings that originated from the International Affective Picture System (Lang et al., 2005). There was no objective correct answer to these questions; rather, we were interested in differences in responses across the three study groups. Question 1: Which figure best represents the face emotion seen on the screen? Using the five emotion orientation rating options available, the participant had to choose an emotion orientation from very positive, positive, neutral, negative, or very negative. Question 2: Can you use words to describe the face emotion seen on the screen? Question 3: How “intense” is the emotion expressed on the face? Using the five emotion intensity rating options available, the participant had to choose from the most, much, moderate, little, or least option to assess the degree of intensity of the face emotion perceived.

Procedures

The experiment was carried out on a one-on-one basis. During the experiment, the participant was seated in a room in front of a computer screen and the eye-tracking camera, which recorded the eye movements of the participant using a remote eye-tracker. The eye-tracker was calibrated before viewing the photos by asking the participant to look at nine predefined points on the computer monitor. The participant was given a visual instruction sheet with the three questions illustrated, as shown in Figure 2. After a photo was presented on the screen, the investigator repeated the instruction verbally. The participant verbally indicated the ratings, and the responses to the three questions were recorded on a rating sheet. There was no time limit for the viewing of each photo, but instructions were repeated when no response was made after every 15 s.

Figure 2.

Photo emotion orientation and intensity rating test.

Results

Accuracy in FER

For ratings of emotion orientation in the FER test, analysis of variance (ANOVA) showed significant differences across groups in rating the complex emotion of disgust. Post hoc comparisons showed that the ASD group (M = 2.67) and ASD/ADHD (M = 2.80) gave significantly higher ratings than their controls (M = 1.55) (Table 2). Furthermore, more inconsistent ratings, as indicated by a higher SD, were found in the ASD groups than in the TD group when attempting to identify the emotional orientation of complex facial expressions of disgust, anger, and fear. As seen in Table 2, showing ratings of emotion orientation in FER of complex emotions, the SDs for the control group were all under 1 (i.e. from 0.52 to 0.79). However, the co-morbid ASD-ADHD group when attempting to identify the face of disgust showed an SD of 1.30, and the ASD-only group when attempting to identify the face of anger and fear showed SDs of 1.67 and 1.53, respectively. This suggests that the emotion perception styles of individuals in the ASD group vary a great deal, particularly when it comes to identifying complex emotions, more than was seen in the TD group.

Table 2.

Rating of emotion orientation in facial emotion recognition of complex emotions.

Complex emotion	Group analysis									ANOVA	Post hoc comparison
	Control			ASD-only			ASD-ADHD
	Count	M	SD	Count	M	SD	Count	M	SD
Disgust	29	1.55	0.52	11	2.67	0.58	15	2.80	1.30	F(2, 16) = 5.30, p = 0.02	Control < ASD-ADHD
Angry	29	2.08	0.79	11	2.40	1.67	15	1.80	0.45	F(2, 19) = 0.45, p = 0.64	–
Fear	29	2.17	0.75	11	2.67	1.53	15	3.20	0.84	F(2, 11) = 1.56, p = 0.25	–

M: mean; SD: standard deviation; ASD: autism spectrum disorder; ADHD: attention-deficit hyperactivity disorder; ANOVA: analysis of variance.

For ratings of emotion intensity, ANOVA showed significant differences across groups in rating the emotion of fear. Post hoc comparisons showed that the ASD group (M = 2.75) gave significantly lower ratings than their controls (M = 4.29). Significant results were also found on judging the intensity of the complex and positive photo sets. Post hoc comparisons showed that the ASD group gave significantly lower ratings than their controls on photos of complex emotions (ASD group M = 2.58; control group M = 3.24), whereas the co-morbid ASD/ADHD group gave lower ratings than the control group on photos of positive emotions (ASD/ADHD group M = 3.32; control group M = 4.03) (Table 3).

Table 3.

Rating of emotion intensity in facial emotion recognition of complex emotions.

Rating of emotion intensity	Group analysis									ANOVA	Post hoc comparisons
	Control			ASD-only			ASD-ADHD
	N	M	SD	N	M	SD	N	M	SD
Negative emotion	29	3.34	1.04	11	3.78	1.23	15	3.15	1.20	F(2, 44) = 0.85, p = 0.44
Complex emotion	29	3.24	0.63	11	2.58	0.52	15	3.03	0.55	F(2, 52) = 5.14, p = 0.01	Control > ASD-only
Positive emotion	29	4.03	0.45	11	3.98	0.66	15	3.32	1.23	F(2, 52) = 4.41, p = 0.02	Control > ASD-ADHD
Fear emotion	29	4.29	0.61	11	2.75	0.96	15	4.40	0.55	F(2, 20) = 9.25, p = 0.00	ASD < control
Surprise emotion	29	4.45	0.69	11	5.00	0.00	15	4.00	1.00	F(2, 16) = 1.75, p = 0.21	–
Angry emotion	29	4.33	0.52	11	5.00	0.00	15	4.25	1.50	F(2, 11) = 0.96, p = 0.41	–

M: mean; SD: standard deviation; ASD: autism spectrum disorder; ADHD: attention-deficit hyperactivity disorder; ANOVA: analysis of variance.

Intensity rating options: 5 = most; 4 = much; 3 = moderate; 2 = little; and 1 = least.

Visual processing of emotions

ANOVA showed a significant difference across groups in terms of total duration of “in” fixations. Post hoc comparisons showed that this overall effect was due to a significant difference between the ASD group and the control group. For all photo types, regardless of emotional orientation, the eye-tracking data showed that the control group spent significantly more time in terms of duration of fixations within the face regions than those of the ASD-only group during the FER photo tasks (Figure 3 and Table 4). To be specific, the control group spent 10,882 ms, double the overall time of the ASD group, who spent 5954.37 ms. Two other trends are worth mentioning. First, ANOVA showed a marginally significant difference across groups in the total number of “in” gaze fixations, and this trend appeared to be due to the especially low rates in the ASD group. Second, ANOVA showed a marginally significant difference across groups in the total duration of “out” gaze fixations. In this case, the ASD group spent double the fixation duration viewing “out” of the face regions compared to the control group (i.e. 25599.31 ms in ASD versus 9735.84 ms in control). Together, the results suggest that the high-functioning ASD group spent more time looking outside the face and less time inside the face during FER.

Figure 3.

Gaze plots of an individual with ASD (left) and a control (right) viewing a face showing fear.

Table 4.

Performance of in/out gaze points and fixation duration in FER tasks between groups.

In/out gaze points and duration	Group analysis									ANOVA	Post hoc test
	Control			ASD-only			ASD-ADHD
	N	M (ms)	SD	N	M (ms)	SD	N	M (ms)	SD
Total number of “in” gaze	29	46.97	21.62	11	28.64	17.41	15	47.71	23.86	F(2, 43) = 2.94, p = 0.06
Total duration of “in” fixation	29	10882.15	5348.00	11	5954.37	5089.39	15	9305.07	4552.51	F(2, 43) = 3.30, p = 0.05	ASD < control
Total number of “out” gaze	29	176.08	197.85	11	109.01	172.36	15	135.39	179.27	F(2, 43) = 0.51, p = 0.61	–
Total duration of “out” fixation	29	9735.84	7058.72	11	25599.31	34025.84	15	14869.75	10395.06	F(2, 43) = 3.00, p = 0.06	–

M: mean; SD: standard deviation; ASD: autism spectrum disorder; ADHD: attention-deficit hyperactivity disorder; ANOVA: analysis of variance; ms: milliseconds.

Discussion

This study adds scientific data from the Chinese population regarding the inconsistent findings of FER studies of persons with high-functioning ASD. Moreover, the clinical group was subdivided into those with ASD-only and ASD/ADHD to take into account the common co-morbidity of these conditions when comparing children with ASD to the control group. Although our study results support those of past Western studies that found that the high-functioning ASD group tends to identify positive emotions, such as happiness, more easily than negative emotions, such as anger (De Wit et al., 2008), our interpretation of the findings tends to differ from the conventional reason put forward to explain the reduced attention to the eye regions by individuals with ASD. This new interpretation is due in part to the use of a wider range of measures of emotion facial processing, as in this study we used emotion intensity ratings to complement the emotional orientation ratings. The focus of analysis has so far been more on the regions of face, meaning whether attention is paid more to the eyes or mouth, or to the upper half or lower half of face, to account for the differences in the groups’ ability to perceive emotion orientation, positive or negative. We argue that it is not the physical region of interest that matters so much, but rather the processing style, configural or featural, in emotion perception.

Emotion perception style

Our position is that FER is mediated by the adoption of a rule-bound categorical thinking approach by the high-functioning ASD group. When individuals with high-functioning ASD understand a concept, in this case facial emotions, they tend to abide by categorical rules in relation to the concept. For example, happiness is represented by a curled-up smile, while sadness is represented by a curled-down grimace. It might be due to this reason that some past studies have found the high-functioning ASD group attended to the mouth region more than the control group, as the mouth provides simpler information than that provided by the eyes (Klin et al., 2002; Rutherford and McIntosh, 2007; Rutherford and Towns, 2008). However, using a rule-bound categorical thinking approach for emotional recognition might lead to poor generalization of pre-learned concepts. If individuals with high-functioning ASD overlearn features associated with specific facial emotions, this might delimit their ability to generalize across different faces. That is to say, if specific features of one face look different from those of another face, individuals with high-functioning ASD might not be able to generalize the emotion from one face to the other. Thus, inflexible, rule-bound categorical thinking may more likely impair their performance on FER tasks involving face-to-face emotion matching (Russell and Widen, 2002).

Intensity rating of complex emotions

In our study, there were more inconsistent ratings found in the ASD groups when attempting to rate the intensity of complex emotions as compared to simple emotions. This pattern can be readily explained because simple emotions, such as happiness, can be overtly graded based on the shape of the mouth. The rule of thumb the individuals with ASD have learnt is as follows: the more curled up the mouth is, the happier the person is (Figure 4). However, rating more complex facial expressions creates a problem for the high-functioning ASD individual. For example, the photo of “anger” expression can more readily be perceived through the intenseness and brightness of the actor’s eye. This hypothesis is supported by Spezio’s study, which showed some evidence that adults with high-functioning ASD fail to make use of information from the eyes when identifying facial expressions (Spezio et al., 2007). For similar reasons, more inconsistent results were found in the high-functioning ASD group when recognizing other emotions such as “disgust,” which is also represented with curled-down lips like the “sadness” emotion (Figure 5), and when comparing “anger” against “fear” because both faces were represented with a tight opened mouth.

Figure 4.

Increasing emotion intensity rating of happy emotion from left to right.

Figure 5.

Emotion perception of complex emotions.

Furthermore, our study results showed that there were no significant differences between the groups’ orientation and intensity ratings of positive emotions. However, the control group selected higher intensity ratings on photos of “fear” emotion, with the high-functioning ASD group’s rating being significantly lower. In contrast, for “surprise” and “anger” emotions, the control group made significantly lower ratings than the high-functioning ASD group. These findings suggest that the high-functioning ASD group does not perceive the “fear” emotion as being as intense as the control group does. While the result supported past studies showing that high-functioning individuals with ASD have more difficulties in recognizing fear, the main focus here is that high-functioning individuals with ASD encounter more difficulties in detecting and interpreting accurately any subtle facial expressions. Consequently, they may make atypical judgments of the intensity of emotions in daily social situations, in particular, in response to complex emotions (Mondloch et al., 2002).

Limitations of the study

Past studies reported great variation in findings. Some reported large FER deficits in high-functioning ASD (Lindner and Rosen, 2006), while some revealed no difference in FER performance between the high-functioning ASD group and the control group (Robel et al., 2004). It has been suggested that the inconsistent results might have been caused by procedural bias when matching ASD and control groups on intellectual ability, as FER is more dependent on intellectual ability in children with ASD than in typical development (Burack et al., 2004; Dyck et al., 2006). Furthermore, as some researchers have suggested that individuals with ASD may have a higher nonverbal intelligence quotient (IQ) than their control peers with the same verbal IQ, matching groups on verbal ability as well as nonverbal ability is important. In our study, we attempted to match nonverbal IQ using the Raven’s ranking scores. However, for verbal IQ, only clinical observation was used with no standard testing support. As long as the participant could respond in full sentences verbally and sat through the photo viewing, they were included in the study. As a result, no verbal IQ scores were available for the FER task between-group comparison.

In addition, the overall small sample size might have limited our ability to detect differences between groups in our study. The significant effect sizes of the difference between control and ASD groups were medium, ranging from 0.40 to 0.69 with a mean of 0.50, but the small sample might have limited the ability to identify smaller effects as significant (Cohen, 1992). Furthermore, for the co-morbid ASD/ADHD group, only 15 boys and no girls were recruited. Therefore, no attempt to analyze gender differences was made in this study. This bias in gender recruitment was mainly due to the much higher ADHD prevalence rate found in males than females. Male-to-female ratio has been reported as high as 2.28:1 (Ramtekkar et al., 2010). Very few studies have included a sufficient number of female subjects to warrant gender-based comparisons of children with ADHD (Gaub and Carlson, 1997).

Finally, the nature of the experimental materials should be further studied. For example, children with ASD may misinterpret a task that combines drawings and pictures of facial expressions (as in our pictorial emotion orientation rating scale). Future research may use other indicators of facial processing deficits to distinguish the effects of a general deficit in emotion recognition from the effects of a literal interpretation of the task demands. Measurement issues are also important to consider in terms of the use of the Autism Diagnostic Observation Schedule and the Autism Diagnostic Interview, which were developed in the West but used in a Chinese sample in our study. In future research, it will be important to test the cross-cultural validity of categorization systems, such as the DSM-5, so that research on facial processing deficits in ASD in Asian populations can more easily be compared with research in the West.

Concluding remarks: implications for further research

In summary, the results of this study demonstrated atypical emotional processing in the high-functioning ASD children when identifying facial emotions and in rating their intensity. We suggest that high-functioning ASD children prefer to use a rule-bound categorical approach as well as a featural processing strategy in the FER tasks. Therefore, these children more readily distinguish overt emotions such as demonstrations of happiness and sadness. However, they perform more inconsistently when distinguishing more covert emotions such as disgust and anger, which demand more cognitive strategy employment during emotional perception. Their fixation time in eye-tracking data demonstrated a significant difference from controls when judging complex emotions, showing reduced “in” gazes and increased “out” gazes. The data were consistent with their emotion intensity ratings, which showed that they misjudge the intensity of complex emotions, especially the emotion of fear.

A better understanding of the processing style or strategies adopted by individuals with high-functioning ASD will give us hints on how to make full use of their preferences to help them acquire social interaction skills, such as perception of others’ emotions. Future ASD interventions that include emotion perceptual skill training can narrow the focus on those social situations that trigger understanding of complex emotions. Using a systematic rule-bound analysis of complex everyday social situations, the high-functioning individuals with ASD can be guided through the simulated social episodes and practice emotion perceptual compensatory strategies and social interaction. The employment of cognitive and language-based compensatory strategies can also be incorporated to guide them successfully to meet daily social interaction challenges (Mondloch et al., 2002).

Footnotes

Funding

The author acknowledges that the wider study from which this paper is generated is financially supported by the General Research Fund under University Grants Council of Hong Kong Special Administration Region, China [Grant number: GRF 844813].

References

Adolphs

Gosselin

Buchanan

. (2005) A mechanism for impaired fear recognition after amygdala damage. Nature 433: 68–72.

Adolphs

Sears

Piven

(2001) Abnormal processing of social information from faces in autism. Journal of Cognitive Neuroscience 13: 232–240.

American Psychiatric Association (2013) Diagnostic and Statistical Manual of Mental Disorders (DSM-V). 5th ed. Washington, DC: American Psychiatric Association.

Ames

White

(2011) Are ADHD traits dissociable from the autistic profile? Links between cognition and behaviour. Journal of Autism and Developmental Disorders 41: 357–363.

Ashwin

Chapman

Colle

. (2006) Impaired recognition of negative basic emotions in autism: a test of the amygdala theory. Social Neuroscience 1(3–4): 349–363.

Bal

Harden

Lamb

. (2010) Emotion recognition in children with autism spectrum disorders: relations to eye gaze and autonomic state. Journal of Autism and Developmental Disorders 40(3): 358–370.

Behrmann

Thomas

Humphreys

(2006) Seeing it differently: visual processing in autism. Trends in Cognitive Sciences 10(6): 258–264.

Boraston

Blakemore

Chilvers

. (2007) Impaired sadness recognition is linked to social interaction deficit in autism. Neuropsychologia 45(7): 1501–1510.

Boucher

Lewis

(1992) Unfamiliar face recognition in relatively able autistic children. Journal of Child Psychology and Psychiatry 33: 843–859.

10.

Bradley

Codispoti

Cuthbert

. (2001) Emotion and motivation I: defensive and appetitive reactions in picture processing. Emotion 1: 276–298.

11.

Burack

Iarocci

Flanagan

. (2004) On mosaics and melting pots: conceptual considerations of comparison and matching strategies. Journal of Autism and Developmental Disorders 34(1): 65–73.

12.

Cohen

. (1992) A power primer. Psychological Bulletin 112(1): 155–159.

13.

Corden

Chilvers

Skuse

(2008) Avoidance of emotionally arousing stimuli predicts social-perceptual impairment in Asperger’s syndrome. Neuropsychologia 46(1): 137–147.

14.

Dalton

Nacewicz

Johnstone

. (2005) Gaze fixation and the neural circuitry of face processing in autism. Nature Neuroscience 8: 519–526.

15.

De Wit

TCJ

Falck-Ytter

von Hofsten

(2008) Young children with autism spectrum disorder look differently at positive versus negative emotional faces. Research in Autism Spectrum Disorders 2: 651–659.

16.

Dyck

Piek

Hay

. (2006) Are abilities abnormally interdependent in children with autism? Journal of Clinical Child & Adolescent Psychology 35: 20–33.

17.

Ekman

Friesen

(1971) Constants across cultures in face and emotion. Journal of Personality and Social Psychology 17(2): 124–129.

18.

Ekman

Rosenberg

(1997) What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS). New York: Oxford University Press.

19.

Gaub

Carlson

(1997) Gender differences in ADHD: a meta-analysis and critical review. Journal of the American Academy of Child & Adolescent Psychiatry 36(8): 1036–1045.

20.

Gross

(2008) Recognition of immaturity and emotional expressions in blended faces by children with autism and other developmental disabilities. Journal of Autism and Developmental Disorders 38(2): 297–311.

21.

Grossman

Klin

Carter

. (2000) Verbal bias in recognition of facial emotions in children with Asperger syndrome. Journal of Child Psychology and Psychiatry and Allied Disciplines 41(3): 369–379.

22.

Harms

Martin

Wallace

(2010) Facial emotion recognition in autism spectrum disorders: a review of behavioral and neuroimaging studies. Neuropsychological Review 20: 290–322.

23.

Herba

Phillips

(2004) Annotation: development of facial expression recognition from childhood to adolescence: behavioural and neurological perspectives. Journal of Child Psychology and Psychiatry 45(7): 1185–1198.

24.

Homer

Rutherford

(2008) Individuals with autism can categorize facial expressions. Child Neuropsychology 14(5): 419–437.

25.

Howard

Cowell

Boucher

. (2000) Convergent neuroanatomical and behavioural evidence of an amygdala hypothesis of autism. NeuroReport 11: 2931–2935.

26.

Humphreys

Minshew

Leonard

. (2007) A fine-grained analysis of facial expression processing in high functioning adults with autism. Neuropsychologia 45(4): 685–695.

27.

Jones

Carr

Klin

(2008) Absence of preferential looking to the eyes of approaching adults predicts level of social disability in 2-year-old toddlers with autism spectrum disorder. Archives of General Psychiatry 65(8): 946–954.

28.

Klin

Jones

Schultz

. (2002) Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry 59: 809–816.

29.

Lang

Bradley

Cuthbert

(2005) International Affective Picture System (IAPS): Affective ratings of pictures and instruction manual. Technical Report A-6. Gainesville, FL: University of Florida.

30.

Lindner

Rosen

(2006) Decoding of emotion through facial expression, prosody, and verbal content in children and adolescents with Asperger’s syndrome. Journal of Autism and Developmental Disorders 36: 769–777.

31.

Loveland

Steinberg

Pearson

. (2008) Judgments of auditory-visual affective congruence in adolescents with and without autism: a pilot study of a new task using fMRI. Perceptual and Motor Skills 107(2): 557–575.

32.

Miles

Johnston

(2006) Not all smiles are created equal: perceiver sensitivity to the differences between posed and genuine smiles. In: Haberman

Flethcher-Flinn

(eds) Cognition and Language: Perspectives from New Zealand. Bowen Hills. QLD, Australia: Australian Academic Press, pp.51–64.

33.

Mondloch

Le Grand

Maurer

(2002) Configural face processing develops more slowly than featural face processing. Perception 31(5): 553–566.

34.

Mondloch

Robbins

Maurer

(2010) Discrimination of facial features by adults, 10-year-olds, and cataract-reversal patients. Perception 39(2): 184–194.

35.

Neumann

Spezio

Piven

. (2006) Looking you in the mouth: abnormal gaze in autism resulting from impaired top-down modulation of visual attention. Social Cognitive and Affective Neuroscience 1: 194–202.

36.

Ogai

Matsumoto

Suzuki

. (2003) fMRI study of recognition of facial expressions in high-functioning autistic patients. NeuroReport 14: 559–563.

37.

Peace

Miles

Johnston

(2006) It doesn’t matter what you wear: the impact of posed and genuine expressions of happiness on product evaluation. Social Cognition 24: 137–168.

38.

Pelphrey

Sasson

Reznick

. (2002) Visual scanning of faces in autism. Journal of Autism and Developmental Disorders 32: 249–261.

39.

Ramtekkar

Reiersen

Todorov

. (2010) Sex and age differences in attention-deficit/hyperactivity disorder symptoms and diagnoses: implications for DSM-V and ICD-11. Journal of the American Academy of Child & Adolescent Psychiatry 49(3): 217–228.

40.

Raven

Court

Raven

(1983) Manual for Raven’s Progressive Matrices and Vocabulary Scales. London: Lewis.

41.

Renzi

Schiavi

Carbon

C-C

. (2013) Featural and Configural processing of faces are dissociated in the dorsolateral prefrontal cortex: a TMS study. NeuroImage 74: 45–51.

42.

Robel

Ennouri

Piana

. (2004) Discrimination of face identities and expressions in children with autism: same or different? European Child & Adolescent Psychiatry 13(4): 227–233.

43.

Rosset

Rondan

Da Fonseca

. (2008) Typical emotion processing for cartoon but not for real faces in children with autistic spectrum disorders. Journal of Autism and Developmental Disorders 38(5): 919–925.

44.

Russell

Widen

(2002) Words versus faces in evoking children’s knowledge of the causes of emotions. International Journal of Behavioral Development 26: 97–103.

45.

Rutherford

McIntosh

(2007) Rules versus prototype matching: strategies of perception of emotional facial expressions in the autism spectrum. Journal of Autism and Developmental Disorders 37(2): 187–196.

46.

Rutherford

Towns

(2008) Scan path differences and similarities during emotion perception in those with and without autism spectrum disorders. Journal of Autism and Developmental Disorders 38(7): 1371–1381.

47.

Spezio

Adolphs

Hurley

RSE

. (2007) Abnormal use of facial information in high-functioning autism. Journal of Autism and Developmental Disorders 37: 929–939.

48.

Teunisse

De Gelder

(2001) Impaired categorical perception of facial expressions in high-functioning adolescents with autism. Child Neuropsychology 7(1): 1–14.

49.

Tottenham

Tanaka

Leon

. (2009) The NimStim set of facial expressions: judgments from untrained research participants. Psychiatry Research 168: 242–249.

50.

Wallace

Coleman

Bailey

(2008) An investigation of basic facial emotion recognition in autism spectrum disorders. Cognition & Emotion 22(7): 1353–1380.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.10 MB