Where Children With ADHD Direct Visual Attention During Emotion Knowledge Tasks: Relationships to Accuracy,Response Time,and ADHD Symptoms

Abstract

Objective: Inattention may contribute to emotion recognition deficits in children with ADHD. In the current study, we compared the viewing patterns for emotion stimuli between children with and without ADHD and examined the relationship between viewing patterns, emotion knowledge accuracy, response time, and ADHD symptoms. Method: Eye-tracking technology recorded viewing patterns for emotion stimuli among 45 children (60% male; control n = 26, ADHD n = 19). Results: Overall, viewing patterns of children with and without ADHD were strikingly similar; however, small to large effect sizes (Cohen’s d = −0.73 to 0.93) across emotions suggest that, for some emotions, children with ADHD spend less time viewing relevant areas of images and take longer to respond (i.e., detect an emotion) compared with children without ADHD. Conclusion: Children with ADHD view some emotions differently from children without ADHD. The results provide an important foundation for additional work in this area.

Keywords

ADHD visual attention eye-tracking emotion knowledge

On a daily basis, individuals are required to recognize emotions in facial expressions and body language of others, recognize their own emotions, and understand emotions in situations. These skills represent emotion knowledge (EK). Among typically developing children, greater EK is related to greater social competence, fewer behavioral problems, and less peer rejection (Izard et al., 2008; A. Miller et al., 2005). There is evidence that children with ADHD exhibit deficits in EK, and that these deficits are related to impaired social competence (Kats-Gold, Besser, & Priel, 2007; Kats-Gold & Priel, 2009). The EK deficits in children with ADHD include decreased accuracy in recognizing some emotions in facial expressions and in understanding the role of emotion in situations (Cadesky, Mota, & Schachar, 2000). Given the relationship between EK and social competence, EK deficits may contribute to or exacerbate the social impairment and peer rejection often experienced by children with ADHD (Hoza, 2007). The potential long-term negative impact of EK deficits for children with ADHD and the limited effectiveness of interventions to address social impairment with this population (Hoza et al., 2005) underscore the importance of understanding the factors contributing to these deficits. A better understanding could inform the development or modification of interventions that target the primary contributors to EK deficits in ADHD.

Factors contributing to EK deficits in ADHD are still unknown. Although researchers have hypothesized that inattention is a primary contributor, it is not clear how it affects EK. One hypothesis is that inattention results in missing important cues in emotion stimuli (such as furrowed eyebrows or a slight frown); however, whether inattention actually affects viewing patterns among children with ADHD is unknown. Eye-tracking technology offers an innovative method for examining viewing patterns of emotion stimuli and the relationship between visual inattention (measured via eye-tracking) and EK. Despite its potential utility, the use of eye-tracking to examine EK in individuals with ADHD has been limited. Because little is known about the visual attention of children with ADHD during emotion recognition, an exploratory study was conducted with the goal of better understanding how children with ADHD view emotion stimuli and how their viewing patterns are related to EK and ADHD symptoms. To achieve this goal, we used eye-tracking technology to (a) compare viewing patterns of emotion stimuli between children with and without ADHD and (b) examine the relationship between viewing patterns, EK accuracy, response time, and ADHD symptoms.

EK in Children With ADHD

Emotion Recognition Deficits in Children With ADHD

One component of EK is the ability to recognize emotions in facial expressions. Of the studies that examine emotion recognition in children with ADHD, results indicate that they exhibit deficits in facial expression recognition for one or more emotion (e.g., Boakes, Chapman, Houghton, & West, 2008; Cadesky et al., 2000) and in identifying emotions in themselves and others (Casey, 1996; Norvilitis, Casey, Brooklier, & Bonello, 2000). Effect sizes for group differences are small to large (Cohen’s d); however, there is wide variability in the findings of these studies. In some studies, deficits were observed for only one of the emotions tested, and the emotions for which there were significant deficits varied across studies. The variability in findings may be explained by the inconsistent methods across studies, including differences in the use of standardized versus author-developed stimuli, number of emotions tested, number of test trials, response format, and ages (age range = 5-15). In addition, ceiling effects in emotion recognition were present in a few of the studies. The presence of ceiling effects raises the possibility that some stimuli were not sufficiently difficult to detect group differences. Taken together, it is unclear whether variability in results of these emotion recognition studies is due to methodological differences across the studies or whether EK deficits in children with ADHD are not as pervasive as some results suggest. Despite these inconsistencies, the research supports the notion that EK deficits exist to an extent in individuals with ADHD.

Situational EK Deficits in Children With ADHD

Another component of EK is situational EK, the ability to infer an emotion from situational cues. Although there is some evidence to the contrary (i.e., Lee, Hung, Lam, & Lee, 2009), at least four studies suggest that children with or at risk for ADHD have deficits in situational EK. Two studies found children with ADHD to be less accurate than same-aged and younger non-ADHD peers in identifying the emotion represented in a scenario read aloud (Shin, Lee, Kim, Park, & Lim, 2008; Yuill & Lyon, 2007). Another study found boys at risk for ADHD demonstrated a decreased understanding of how emotions change in general and across situations (Kats-Gold & Priel, 2009). Da Fonseca, Seguier, Santos, Poinso, and Deruelle (2009) assessed situational EK via a task that required children to use context to identify the emotion of a person whose face was obscured in an image. Children in the ADHD group made more errors than children in the control group. Participants also completed a similar, non-emotion, object task, and group differences were not found for this task. Rapport, Friedman, Tzelepis, and Van Voorhis (2002) reported similar findings in adults, with ADHD and control group differences for an emotion task but not for a similar, non-emotion task. These findings suggest that the emotional aspect of the tasks contributed to decreased accuracy in individuals with ADHD, rather than an aspect of the task itself (e.g., attention demands). In sum, results suggest that children with ADHD demonstrate impaired situational EK and are less able to use context to aid emotion identification compared to same-aged and younger children without ADHD. These impairments have negative implications for the social interactions of children with ADHD.

The Relationship Between EK and Social Competence in Children With ADHD

As has been documented in typically developing children (Miller et al., 2005), there is a link between EK and social competence in children with ADHD. Kats-Gold et al. (2007) found that risk of ADHD moderated the relationship between facial expression recognition and social skills, with an increased number of emotion recognition errors negatively related to social skills for boys at risk for ADHD. Emotion recognition errors accounted for 16% of the variance in social skills and 18% of the variance in behavior problems for the at risk group. In addition, their results indicated that, for boys at risk for ADHD, four EK subscale scores were significantly correlated with social skills measures (r range = .26-.39). Pelc, Kornreich, Foisy, and Dan (2006) also found a negative relationship between number of self-rated interpersonal problems and emotion recognition accuracy among children, ages 7 to 12, with ADHD (r = −.67). Given the positive relationship between social competence and EK in typical children (A. Miller et al., 2005) and the contributions that EK deficits can have on social impairment in children with ADHD, understanding the nature of EK deficits and how they relate to social competence may assist in developing social interventions. However, before such interventions can be developed, research is needed to better understand the nature of EK deficits exhibited by children with ADHD.

Purported Contributor to EK Deficits in Children With ADHD

Although EK deficits in children with ADHD are documented, the factors contributing to these deficits are not well understood. Researchers have hypothesized that overall inattention or inattention to the critical aspects of emotion stimuli may contribute to emotion recognition deficits; however, few studies have specifically examined the relationship between inattention measures and EK accuracy. Miller, Hanford, Fassbender, Duke, and Schweitzer (2011) reported a positive relationship between self-rated inattention symptoms and number of sad-affect recognition errors in adults with ADHD. The authors hypothesized that inattention problems may interfere with attending to the critical cues of social situations, thereby affecting the ability to recognize and respond to the emotions of others. However, that study was conducted with adults and used self-report measures of inattention; thus, findings should be replicated with children and with collateral informants of inattention symptoms and/or objective behavioral indicators of inattention.

In a study of children with ADHD and autism spectrum disorder (ASD), scores on a sustained attention and an inhibition task, but not clinician-rated inattention, were related to number correct for the emotion recognition task (rs > .48; Sinzig, Morsch, & Lehmkuhl, 2008). Furthermore, the groups with ADHD (ADHD only and ADHD + ASD) were less accurate in emotion recognition than the two other groups (ASD only and control). Thus, inattention and possibly other symptoms of ADHD may be important contributors to EK deficits. A limitation of that study is that the sustained attention task was auditory in nature, whereas the EK stimuli were visual. Measurement of inattention via the same sensory modality used when identifying emotion provides information on the type of inattention problems most likely to contribute to EK deficits. Shin et al. (2008) reported a relationship between visual attention on a continuous performance test (CPT) and EK, but the CPT task did not require detailed visual attention. Measurement of detailed visual attention is important when examining emotion content because it may be more negatively affected by poor attention (i.e., cues to be detected are more subtle) or may be more difficult to accurately decipher if such subtle cues are missed. Thus, additional data on ADHD viewing patterns and their relationship to EK are needed to understand EK deficits in those with ADHD. Detailed viewing patterns of visual attention would indicate the extent to which children with ADHD fail to devote visual attention to relevant cues in a manner similar to typical children (a foundational problem), or whether they are visually attending to the cues but are not interpreting the cues in a manner similar to typical children (a higher order problem). Although there is likely an interaction between visual attention and higher order processing, deficits in each area specifically may result in differing intervention approaches.

Use of Eye-Tracking in Social and Emotional Research

Measuring visual fixations with eye-tracking technology provides a reliable and valid tool for determining where a child is looking while viewing an image (Karatekin, 2007), and emotion recognition accuracy reveals whether the information gathered from the image resulted in correctly identifying the emotion. The use of eye-tracking and EK tasks together is important because eye-tracking alone only tells where the child looks when viewing the image; it does not offer insight into how the child uses that information. Similarly, EK accuracy alone only reveals the result of information input; it does not offer insight into what information was used to determine that result. Together, the two provide indicators of information input and output.

Pishyareh et al. (2012) are among the few researchers who have used eye-tracking to examine visual patterns to emotion stimuli among children with ADHD. The authors simultaneously presented children with two scenes of differing valence (e.g., pleasant–unpleasant) and examined first gaze duration for each scene of the pair. Children with ADHD spent less time viewing pleasant pictures in the pleasant–unpleasant and pleasant–neutral pairings compared with controls (Pishyareh et al., 2012). These results suggest viewing preferences may differ between children with and without ADHD; however, they do not provide insight into how viewing patterns may differ within a scene. As a result, they do not address the question of the role of visual attention to detail in emotion stimuli and its relationship to EK.

Viewing the relevant areas of faces and situations is essential to emotion recognition. It is not currently known whether children with ADHD view emotion stimuli in a similar or dissimilar manner as children without ADHD. Examining how children with ADHD view these stimuli is important for understanding their EK deficits because it will inform whether children with ADHD are not viewing important cues, are viewing them for a different duration, or if they view the important cues but do not interpret them correctly to yield accurate emotion recognition. If there are deficits at the most basic level of emotion recognition (i.e., viewing relevant areas), interventions can help children with ADHD direct attention to the correct aspects of faces and important areas of scenes. If there are not deficits at this basic level and viewing patterns are similar across children with and without ADHD, this suggests that higher order skills (e.g., information processing, cue interpretation) may be contributing to EK deficits, and interventions can work to address these higher order skills. Examining emotion recognition from the bottom up can provide foundational information to guide where to intervene to address EK and social deficits in children with ADHD. Indeed, encoding cues is the first step in social information processing (Crick & Dodge, 1994); if this first step is disrupted, then later steps of social information processing are likely to be negatively affected. In the current study, eye-tracking technology was used in conjunction with EK tasks to (a) compare viewing patterns with faces and social situations among children with and without ADHD, and (b) examine the relationship between viewing patterns, EK accuracy, response time, and ADHD symptoms.

Method

Participants

Participants were children aged 8 to 12, with an IQ estimate of 79 or greater (Wechsler, 2011), who passed visual acuity, color vision, and ocular-motor screenings. Sample characteristics are included in Table 1. Following evidence-based assessment procedures (Pelham, Fabiano, & Massetti, 2005) and criteria from the Diagnostic and Statistical Manual of Mental Disorders (4th ed., text revision; DSM-IV-TR; American Psychiatric Association [APA], 2000), diagnoses were made using information from a diagnostic interview with the parent (Children’s Interview for Psychiatric Syndromes–Parent Version; Weller, Rooney, Fristad, & Weller, 1999), and parent and teacher rating scales of symptoms and impairment (described below). All children in the ADHD group (n = 19) met DSM-IV-TR criteria for ADHD (any subtype). Participants in the control group (n = 26) were free of any psychiatric disorder based on parent and teacher report. Children taking stimulant medication were required to abstain from this medication for at least 12 hr before testing.

Table 1.

Sample Characteristics.

	ADHD (n = 19)	Control (n = 26)
	M (SD)	M (SD)
Age^a	9.16 (0.90)	9.77 (0.95)
% male^a	78.9	46.2
WASI Composite Score	104 (14)	106 (16)
Has taken medication for ADHD^a	26.3%	0%
Race
Caucasian	47.4%	76.9%
African American	21.1%	3.8%
Other	31.6%	19.2%
Maternal education
Partial HS or HS diploma	10.5%	11.5%
Partial college	26.3%	0%
Associate’s degree	10.5%	19.2%
Bachelor’s degree	31.6%	30.8%
Master’s/doctoral degree	21.1%	38.5%
Paternal education^b
Less than 9th grade	5.3%	0%
Partial HS or HS diploma	31.6%	19.2%
Partial college	0%	7.7%
Associate’s degree	5.3%	3.8%
Bachelor’s degree	5.3%	19.2%
Master’s/doctoral degree	36.8%	50%
Mean income range	$25,000-$49,999	$50,000-$74,999
Public assistance in past year	47.4%	19.2%
Parent DBD Inattention Count^a	4.90 (2.35)	1.50 (2.47)
Parent DBD Hyperactive/Impulsive Count^a	3.68 (2.38)	1.23 (1.37)
Parent DBD ODD Count^a	1.90 (2.16)	0.42 (0.99)
Teacher DBD Inattention Count^a	5.00 (2.71)	1.19 (2.53)
Teacher DBD Hyperactive/Impulsive Count^a	4.16 (3.30)	0.27 (0.67)
Teacher DBD ODD Count^a	2.79 (3.22)	0.35 (0.89)

Note. WASI = Wechsler Abbreviated Scale of Intelligence; DBD = disruptive behavior disorder; ODD = oppositional defiant disorder.

Denotes significant group differences (p < .05) according to t tests or χ² tests.

Values do not add up to 100% due to missing data.

Measures

ADHD symptoms

The Disruptive Behavior Disorders (DBD) Rating Scale (Pelham, Gnagy, Greenslade, & Milich, 1992) is used to assess the presence and severity of Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; APA, 1994) symptoms of ADHD, conduct disorder (CD), and oppositional defiant disorder (ODD). Items are rated on a 4-point scale ranging from 0 (“not at all” present) to 3 (“very much” present). Items rated as “pretty much” or “very much” by either the parent or the teacher were considered “present”; subscale scores represent the total symptom count (i.e., present). Parent and teacher versions are psychometrically sound, with good internal reliability (internal reliability range = .82- .96) and strong concurrent validity with other symptom rating scales (Pelham et al., 2005). Parents and teachers completed all but the CD items for this study. In the current sample, internal reliability ranged from .86 to .96 for parents and teachers.

Impairment

The Impairment Rating Scale (IRS; Fabiano et al., 2006), completed by parents and teachers, assesses a child’s level of impairment across various domains (e.g., peer relations, academics). Anchors for the ratings range from 0 (no problem/definitely does not need treatment or special services) to 6 (extreme problems/definitely needs treatment or special services). Reliability and temporal stability of IRS scores are acceptable for this age group, even when two different teachers complete the measure (r range = .40-.67; Fabiano et al., 2006). The IRS has acceptable convergent and discriminant validity when compared with other measures of parent- and teacher-rated impairment (r range = .58-.85; scores of 3 or higher are considered to be predictive of ADHD status; Fabiano et al., 2006).

Eye-tracking apparatus and EK stimuli

Eye movements were recorded via a LC Technologies EyeGaze EyeFollower binocular eye-tracking system running Interactive Minds Nyan 2.0 Data Analysis Software. The sample rate was 120 Hz, and accuracy was within 1 degree of visual angle. Participants were positioned 24 inches from the display monitor, and a chin rest was used to reduce head movement. A fixation was defined as a relatively stable eye position (within 6 degrees horizontally and 4 degrees vertically) for a minimum of 100 ms (Duchowski, 2007). The eye-tracking dependent measure was the proportion of total fixation duration on all areas of interests (AOIs; total AOI fixation duration divided by total duration of all fixations on each image shown). Such measures have been validated in other studies using eye-tracking to index visual attention (Heuer & Hallowell, 2009). For trials in which the child covered or closed his or her eyes (other than for blinking) or looked away from the screen, eye-tracking data were compared with that child’s data for other trials corresponding to the same emotion and task (e.g., other happy images in the Faces task). If the data for the suspect trial did not lie within the range of other trials, the trial was excluded; if it was within the range, it was included. Of the 2,610 trials across the 45 participants, 20 trials were excluded.

Pictures of Facial Affect (POFA)

The POFA (Ekman & Friesen, 1975) is an image set of facial expressions displaying seven different emotions (happy, sad, angry, disgusted, surprised, fearful, and neutral). The POFA images have been used with children with ADHD to test emotion recognition (Singh et al., 1998). The images selected for use in the current study were selected to comprise a set of six examples of each emotion (three males and three females) with a range of reliability ratings (71%-100% agreement) to increase difficulty and reduce the likelihood of ceiling effects, plus two neutral (“no emotion”) images (both male). Facial expression EK accuracy scores are the percent of correct responses for each of the emotions (i.e., six basic emotions and no emotion). The AOIs for the POFA faces were the eyes and mouth of each face.

Internet images

Standardized images of facial expression in context with discrete emotion labels, child-appropriate content, and all six of the basic emotions are not widely available. Thus, the images were obtained via Google Image searches using emotion-related search terms (e.g., happy child, sad child), and image pilot testing was conducted with 11 graduate students and employees at the author’s institution. Respondents rated the emotion of each image, as well as areas relevant and irrelevant to determining the emotion. Included images had an average reliability of 89% (range = 45%-100%) for emotion detection, with all images having 50% or greater reliability, with the exception of one image. The one emotion with a 45% reliability was flagged for possible removal if control children were split in responses like the pilot testing respondents. However, 69% of control children chose “angry” as the response, which was the a priori classification of the image; thus, it was included in the analyses. Relevant areas were determined by examining the qualitative responses provided for each image. Areas that were identified as relevant to discerning the emotion by at least 6 of the 11 respondents were classified as relevant areas. The relevant areas (i.e., AOIs) were primarily focal persons and objects. Any regions outside these defined areas were considered irrelevant areas. In the images, the face of one individual was obscured, and the child was asked to identify the emotion most likely expressed by the individual whose face was obscured, given the context. This method is similar to that used in Da Fonseca et al. (2009). Situation EK accuracy scores are the percent of correct responses for each of the six basic emotions and no emotion.

Procedure

All procedures were approved by the University Institutional Review Board. Participants were recruited from a rural community via local health clinics, elementary schools, newspaper, email, and flyer advertisements, and from an urban community via a larger grant-funded research project. A phone screen was conducted to evaluate preliminary study eligibility; those determined to be eligible were scheduled for a study session. During the study session, consent and assent procedures were completed. The child was then escorted to a testing room to complete the Wechsler Abbreviated Scale of Intelligence–Second Edition (WASI-II) and eye-tracking task (task order was counterbalanced across participants). The parent was escorted to a different room to complete the diagnostic interview and rating scales.

For the EK and eye-tracking procedures, a brief definition of each emotion and an example of the emotion in context were read to the child for the seven emotions. The child then participated in brief vision, color, and ocular-motor screenings; one was excluded on the basis of failing the color screening. After the screenings, the child completed the eye-tracking calibration process, after which the first EK task was introduced, either emotion recognition in faces (henceforth referred to as the Faces task) or emotion in situations (henceforth referred to as the Situations task). One training trial was completed before each task’s test trials. Images were displayed on a computer screen for 4 s for the Faces task and 7 s for the Situations task, and then replaced by a screen showing the response options. Children were allowed to verbalize or point to indicate their response. Time to respond once the options were shown was recorded. A total of 38 facial expression images were shown (i.e., six trials per basic emotion and 2 neutral). A total of 20 situation images were shown (i.e., two to four trials per basic emotion and one trial for neutral).

Results

Descriptive statistics and effect sizes are reported for proportion of total fixation duration on AOIs (PFDAOI), response time, and EK accuracy across groups. Given the sample size and the exploratory nature of the study, effects sizes, rather than statistical significance, were used to interpret meaningful effects. Effect sizes are interpreted using Cohen’s conventions (Cohen, 1988; that is, small: 0.2; medium: 0.5; large: 0.8). To examine the relationship between viewing patterns, EK accuracy, response, and ADHD symptoms, bivariate correlations and profile plots were analyzed.

Proportion of Time Spent Viewing Relevant Areas

For the Faces task, children with ADHD (M = .59, SD = .11) spent less time viewing the relevant areas of the faces (i.e., eyes and mouth) than did children without ADHD (M = .61, SD = .09; d = −0.20). This same pattern emerged on the Situations task, with children with ADHD (M = .61, SD = .07) spending less time viewing the relevant areas of the social scenes than did children without ADHD (M = .64, SD = .07; d = −0.43; see Table 2 for total faces and total situation).¹ For the Faces task, effect sizes ranged from 0.06 for No Emotion faces to −0.42 for Happy faces, indicating that the emotion represented affected whether children with ADHD viewed the face more or less similarly than children without ADHD. The largest effect sizes were observed for Happy and Mad faces and the smallest effect sizes for No Emotion, Scared, and Sad faces. For the Situations task, the effect sizes ranged from −0.73 to 0.33, with Mad and Scared situations having the largest negative effect sizes (children with ADHD spent less time viewing relevant areas) and Surprised situations having a positive effect size (children with ADHD spent more time viewing the relevant areas).

Table 2.

EK PFDAOI, Response Times, and Accuracy.

	ADHD (n = 19)	Control (n = 26)	Effect size
	M (SD)	M (SD)	(d)
Faces PFDAOI
Disgusted	0.48 (0.13)	0.51 (0.12)	−0.24
Happy	0.50 (0.13)	0.55 (0.11)	−0.42
Mad	0.60 (0.15)	0.65 (0.11)	−0.38
Sad	0.62 (0.12)	0.63 (0.13)	−0.08
Scared	0.67 (0.14)	0.68 (0.10)	−0.08
Surprised	0.65 (0.14)	0.67 (0.13)	−0.15
No Emotion	0.56 (0.20)	0.55 (0.12)	0.06
Total Faces	0.59 (0.11)	0.61 (0.09)	−0.20
Faces Average Response Time (s)
Disgusted	2.41 (1.62)	1.71 (0.79)	0.55
Happy	2.07 (0.72)	1.50 (0.49)	0.93
Mad	2.88 (2.08)	2.08 (0.88)	0.50
Sad	2.89 (1.74)	2.36 (1.18)	0.36
Scared	3.02 (1.31)	2.18 (1.36)	0.63
Surprised	2.59 (1.24)	1.73 (0.48)	0.91
No Emotion	1.64 (0.69)	1.37 (0.61)	0.41
Total Faces	2.60 (1.14)	1.90 (0.53)	0.79
Faces Percent Correct
Disgusted	49.82 (33.62)	61.54 (26.15)	−0.39
Happy	95.61 (7.54)	93.97 (10.20)	0.18
Mad	76.32 (19.50)	77.56 (18.22)	−0.07
Sad	60.18 (23.67)	59.62 (24.12)	0.02
Scared	58.25 (17.86)	62.82 (27.21)	−0.20
Surprised	77.72 (16.63)	82.69 (21.33)	−0.26
No Emotion	84.21 (37.46)	92.31 (23.20)	−0.26
Total Faces	70.21 (9.30)	74.01 (9.36)	−0.41
Situations PFDAOI
Disgusted	0.78 (0.09)	0.79 (0.09)	−0.11
Happy	0.63 (0.12)	0.62 (0.12)	0.08
Mad	0.58 (0.11)	0.66 (0.11)	−0.73
Sad	0.54 (0.11)	0.58 (0.10)	−0.38
Scared	0.62 (0.11)	0.67 (0.09)	−0.50
Surprised	0.64 (0.10)	0.61 (0.08)	0.33
No Emotion	0.53 (0.18)	0.58 (0.14)	−0.31
Total Situations	0.61 (0.07)	0.64 (0.07)	−0.43
Situations Average Response Time (s)
Disgusted	2.44 (1.14)	1.85 (0.75)	0.61
Happy	2.14 (0.88)	1.99 (1.22)	0.14
Mad	4.50 (4.57)	2.29 (2.13)	0.62
Sad	2.67 (1.58)	1.78 (0.71)	0.73
Scared	2.42 (1.16)	1.62 (0.73)	0.83
Surprised	3.54 (3.31)	2.46 (1.33)	0.43
No Emotion	2.96 (1.54)	2.61 (2.09)	0.19
Total Situations	2.94 (1.71)	2.07 (0.79)	0.65
Situations Percent Correct
Disgusted	89.47 (20.94)	94.23 (16.29)	−0.25
Happy	85.96 (19.65)	94.23 (16.29)	−0.46
Mad	71.93 (22.94)	76.92 (20.59)	−0.23
Sad	79.82 (19.90)	79.81 (24.51)	0.00
Scared	84.21 (20.39)	84.62 (19.39)	−0.02
Surprised	57.89 (31.12)	61.54 (29.35)	−0.12
No Emotion	61.11 (50.16)	72.00 (45.82)	−0.23
Total Situations	77.15 (11.67)	81.25 (11.97)	−0.35

Note. Negative effect sizes represent longer reaction times, lower accuracy, and less time focused on AOIs for children with ADHD. EK = emotion knowledge; PFDAOI = proportion of fixation duration on AOIs; AOI = areas of interest.

EK Accuracy and Response Time

On average, overall EK accuracy scores for the Faces (d = −0.41) and Situations task (d = −0.35) revealed that children with ADHD were less accurate than children without ADHD. Consistent with the literature, group differences varied by emotion type (see Table 2).² In addition, children with ADHD took significantly longer to respond (i.e., identify the emotion) than did children without ADHD in both the Faces (effect sizes range from 0.36 to 0.91) and the Situation tasks (effect sizes range from 0.14 to 0.83; see Table 2). Thus, even when children with ADHD performed as well as children without ADHD (e.g., Happy and Sad faces; Sad situations), it took longer for them to identify the correct emotion in the face. As a way to illustrate the pattern of results, we provide profile plots of the all three data components (i.e., PFDAOI, EK accuracy, and response time; see Figure 1). Although the magnitude of the group difference varies by emotion, the overall pattern of response across emotions for these three constructs is strikingly similar. Where differences emerge, children with ADHD are slightly less accurate, view the AOIs less, and take longer to respond.

Figure 1.

Profile plots for PFDAOIs, EK accuracy, and response time for children with and without ADHD.

Relationship Among Viewing Proportion, EK Accuracy, Response Time, and ADHD Symptoms

Correlations were used to examine the relationship among PFDAOI, EK task accuracy, response time, and ADHD symptoms. Due to space constraints, only overall EK task values rather than emotion-specific values are presented in the correlation matrix (see Table 3).³ Primary analyses focused on correlations with the entire sample to allow for greater dimension and range to the variables. Secondary correlation analyses were conducted with the ADHD sample alone to determine whether the relationships were stronger within this subsample. Only 4 of 45 correlations differed by a magnitude of .30 or greater (noted in Table 3), and the vast majority of correlations mirrored that found within the total sample; thus, we report the findings from the total sample. Among data for the Faces task, PFDAOI was correlated with task accuracy (r = .32, p = .03), indicating that more time spent viewing the relevant areas was associated with increased accuracy and with quicker response time (r = −.27, p = .08). Although, not significant, the magnitude indicates a shorter response time was associated with increased emotion recognition accuracy. For the Situations task, PFDAOI was not correlated with accuracy or response time for that task. Situations task accuracy and response time were correlated (r = −.34, p = .02), indicating a quicker response time was associated with increased emotion recognition. Across tasks, response times (r = .47, p = .001) and accuracy rates (r = .31, p = .04) were correlated with each other. Hyperactive/impulsive symptoms were positively correlated with response time and negatively correlated with PFDAOI across tasks; they were not significantly correlated with EK accuracy. Teacher-rated inattention symptoms were correlated with Faces response time (r = .48, p = .001), but with no other variables. Thus, hyperactivity/impulsivity was more strongly associated with EK variables than inattention.

Table 3.

Correlations Between PFDAOI, EK Accuracy, and Response Time.

	1	2	3	4	5	6	7	8	9	10
1. Faces PFDAOI	—	.32*	−.27^†	.20	.05	−.10	−.07	−.28^†	−.19	−.14
2. Faces EK Accuracy		—	−.22	.02	.31*	−.20	.06	−.08^a	−.14	−.19
3. Faces Average Response Time			—	−.26^†	−.22	.47**	.12	.48**	.46**	.66**
4. Situations PFDAOI				—	.17	−.02	−.10	−.27	−.30*	−.34*
5. Situations EK Accuracy					—	−.34*^b	.06	−.07	−.18	−.26^†
6. Situations Average Response Time						—	.15	.17	.27^†	.30*
7. Parent DBD Inattention Count							—	.32*	.57**^c	.35*^d
8. Teacher DBD Inattention Count								—	.35*	.57**
9. Parent DBD Hyperactive/Impulsive Count									—	.57**
10. Teacher DBD Hyperactive/Impulsive Count										—

Note. PFDAOI = proportion of fixation duration on AOI; EK = emotion knowledge; DBD = Disruptive Behavior Disorder; AOI = areas of interest.

r = .57 for ADHD group.

r = −.68 for ADHD group.

r = .17 for ADHD group.

r = −.13 for ADHD group.

†

p < .10. *p < .05. **p < .01.

Discussion

Despite the hypothesized role of inattention in EK deficits in children with ADHD, few studies have examined visual attention patterns in individuals with ADHD. Such work is needed to understand the extent to which children have deficits in visually attending to the cues that provide emotion information, deficits in higher order processing, and interpretation of cues that are viewed, or both. This study takes an innovative exploratory step in this line of research. Broadly speaking, the data provide some optimism that children with ADHD are viewing the most important areas of faces (i.e., eyes and mouth) and scenes (e.g., context cues, body language) to determine the emotion depicted. However, there are several patterns in the data that are noteworthy. First, consistent with the literature, deficits in emotion recognition vary by the emotion type, and this study advances the literature by showing that performance deficits vary across tasks as well (e.g., performance for Faces differs from performance in Situation tasks). Second, effect sizes suggest that, for some emotions, children with ADHD spend less time viewing the relevant areas of facial expressions and social situations. Third, response time was, on average, greater among children with ADHD than those without ADHD. Fourth, in this sample, adult-rated hyperactivity/impulsivity rather than inattention may more greatly affect viewing patterns and EK accuracy. We discuss these patterns, highlighting implications for future research.

Consistent with previous literature, the effect sizes for group differences in emotion recognition vary by the emotion type and task. For example, during the Faces task, relative to control children, children with ADHD showed the weakest recognition of Disgusted (d = −0.39; average percent correct less than 50). This is in line with other studies reporting Disgusted accuracy rates less than 50% among children with ADHD and higher rates among children without ADHD (Boakes et al., 2008; Lee et al., 2009). However, for the Situations task, Disgusted had the highest accuracy rate for children with ADHD and tied with Happy for the highest accuracy rate for children without ADHD. Disgusted images for the Situations task had clear clues, easily recognizable to children (e.g., dirty diaper, smelly shoe); thus, the disgusting nature of those objects may have helped to identify the emotion, whereas identifying the facial cues indicative of Disgusted (e.g., scrunched nose) is more difficult for children regardless of ADHD status. This pattern of better performance on the Situations task compared with the Faces task for an emotion was reversed for Happy stimuli. During the Situations task, children with ADHD showed the weakest relative performance for happy (d = −0.46; although average percent correct was just above 85), and on the Faces task, they were slightly more accurate than children without ADHD (96% vs. 94%). Some Happy situation images had clear cues such as a gold medal and “number 1” finger, whereas others had less clear cues such as children at a beach. Thus, detecting the facial cues indicative of happy (e.g., a smile) is easy for children, but interpreting contextual cues suggesting happy may be more varied and difficult. Together, these results suggest that not only do accuracy rates differ between ADHD and control groups when compared across emotion, they also differ when compared across tasks (e.g., facial emotion recognition vs. situation emotion recognition). In addition, accuracy rates across the tasks were moderately correlated, suggesting that they are distinct but related EK abilities.

Attending to and encoding of cues are the first steps of emotion recognition and social information processing, but it is not known whether there are deficits in these basic-level skills among children with ADHD. For the Faces task, the largest PFDAOI effect sizes were observed for Happy and Mad, whereas for the Situations task, the largest effect sizes were observed for Scared and Mad. These effects fell in the small to medium range, suggesting that children with ADHD spend less time than children without ADHD viewing the relevant areas of the images. However, less than small effect sizes were observed for three of the Faces emotions and two of the Situation emotions, indicating that children with ADHD view the relevant areas of these images more similarly compared with children without ADHD. Thus, whether children with ADHD spend a similar proportion of their viewing time looking at the areas relevant to emotion recognition as children without ADHD depends on emotion and the cues presented. This variability is perhaps not surprising given that variable attention is a hallmark symptom of ADHD. Beyond group differences and similarities, children in the ADHD group spent approximately 50% or more of viewing time on relevant areas across tasks. This is the first study to report such results, and this suggests that complete inattention to areas relevant to emotion recognition is not contributing to EK deficits in ADHD. Furthermore, Faces task accuracy was correlated with Faces PFDAOI (r = .32 for whole sample, r = .35 for ADHD group), and Situations task accuracy was not correlated with Situations PFDAOI. Thus, viewing relevant areas may be related to accuracy for simple tasks such as recognizing emotion in faces; however, when inference is involved (as in the Situations task), viewing the relevant areas may not be sufficient to identify emotion. An important consideration is that children were specifically asked to identify emotion in the images, and this may have affected (i.e., enhanced) the viewing patterns of the children and resulted in greater AOI viewing than usual. In daily social interactions, children are not explicitly instructed to focus on identifying emotion in others or assess the emotion of persons in a situation; thus, viewing patterns may have differed if the task demands were different (e.g., free viewing of the stimuli without the stated goal of emotion recognition). Indeed, a recent review of eye-tracking studies in the ASD literature explicitly included studies of spontaneous visual attention to social stimuli and explicitly excluded emotion recognition studies because eye-tracking patterns vary across these tasks (Guillon, Hadjikhani, Baduel, & Rogé, 2014). However, the explicit emotion identification tasks of the present study were necessary to investigate whether children with ADHD view the relevant areas during emotion recognition.

Given that viewing patterns can differ based on task demand, attending to task demands in future research is important. For example, to better understand the type of inattention likely to affect children with ADHD during social interactions, the next step in this research is to examine visual attention during tasks with greater real-life demands (e.g., recognizing emotion during a conversation). Interestingly, the largest PFDAOI effect size for the Situations task was for Mad. The disruptive behavior of children with ADHD often elicits negative or angry behavior from adults. Thus, the finding that children with ADHD spend less time than typical children viewing relevant areas in angry situations may translate to their real-time interactions with others who are angry, such as avoiding looking at the faces or body language cues of someone who is angry.

Profile plots indicate that overall, children with and without ADHD display remarkably similar patterns of PFDAOI, EK accuracy, and response times across the emotions. These findings indicate that EK deficits in children with ADHD are not simply due to an interpretation bias or a qualitative difference in emotion recognition; if this were the case, the patterns would be more distinct. Rather, children with ADHD appear to demonstrate a quantitative difference from children without ADHD; they are making more errors, viewing relevant areas less, and taking more time to respond. Furthermore, longer response times may indicate that children with ADHD take longer to process emotion cues than children without ADHD. Given that longer response time can be indicative of greater cognitive load (Huang, Eades, & Hong, 2009), children with ADHD may be using more cognitive resources to process the information and arrive at a conclusion than children without ADHD. Furthermore, the longer response time may affect the real-time social dynamics and the ability to generate and execute social behaviors. The present data are not appropriate for drawing conclusions related to cognitive resources. Still, the Situations task response time was negatively correlated with task accuracy; thus, this is a relationship worth exploring in future research. The largest response time effect sizes were observed for Scared situations (d = 0.83) and Surprised faces (d = 0.91), whereas the accuracy effect sizes for these emotions were small, thus reinforcing the point that there is a discrepancy between the amount of effort children with and without ADHD exert to correctly recognize emotions.

Although inattention was hypothesized to affect viewing patterns, bivariate correlations indicated that parent and teacher ratings of hyperactive/impulsive symptoms were more strongly correlated with PFDAOI, EK accuracy, and response times than ratings of inattention symptoms. Although this is inconsistent with our hypothesis, it aligns with a recent study (Zoromski, Owens, Evans, & Brady, 2015) showing that hyperactivity is significantly associated with impairment in social relationships in early childhood and middle childhood samples (K-fifth grade), but that this relationship decreases in adolescence samples. In contrast, inattention does not become more significantly associated with social impairment compared with hyperactivity/impulsivity until children enter middle childhood and adolescence. Taken together, this suggests that inattentive and hyperactive/impulsive symptoms are both important but may vary in their relative contributions to social impairment across development. It also suggests that both are important to explore in relation to how children with ADHD view emotion stimuli.

Limitations and Future Directions

The present study is not without limitations. First, the small sample size limited detection of significance within the statistical tests. Second, a limitation of all eye-tracking studies is the assumption that visual fixations represent where the individual is focusing attention. However, design of the study in terms of stimuli, task goal, and instructions provided help to support the connection between visual and cognitive attention. Third, the children in the ADHD group represent those with moderate (rather than severe) symptom severity and impairment. With a more symptomatic or socially impaired ADHD group, larger group differences may be found. Fourth, we examined all AOIs together. Future analyses to determine whether viewing times differ among the specific AOIs could offer further insight into which cues children use to detect emotion (e.g., mouth vs. eyes), and this has important implications for the quality of their social interactions. In addition, future use of eye-tracking variables that are more closely related to cognitive processing (e.g., pupil dilation) will allow us to determine whether children with ADHD experience emotion recognition as more cognitively taxing than children without ADHD. Fifth, use of eye-tracking during situations with greater ecological validity (e.g., reacting to real-time stimuli, interacting socially) could offer further insight into the social deficits in children with ADHD. Last, simple, goal-oriented tasks with explicit instructions and minimal distractions may not be best-suited to expose the EK deficits and processes underlying these deficits that exist during real-world interactions for children with ADHD. However, starting with a widely used face emotion recognition method and similar methods for a situations task provides a starting point to move up from basic, foundational skills to more advanced skills. If complex or difficult stimuli are used first, the question of whether basic skills are intact remains unanswered.

Summary

Recognizing emotion and using context to infer the emotion of others are EK skills that are important to social functioning and are deficient in children with ADHD. Given the social impairment children with ADHD often experience, investigating factors that contribute to this impairment and methods for reducing the impairment is needed to inform effective treatment for social and emotional impairment in children with ADHD. This exploratory study represents one of the first to use eye-tracking technology to examine whether group differences exist at the most foundational level of emotion recognition. The overall viewing patterns of children with ADHD are strikingly similar to those without ADHD, suggesting that many foundation skills are intact. However, the nuances found in the group differences and the relationships detected among viewing patterns, EK accuracy, response time, and ADHD symptoms confirm that these variables are worthy of further exploration. As such, this study offers a foundation on which additional studies on the role of visual attention in EK and social functioning in children with ADHD can build. Future studies that allow for more detailed analysis of viewing patterns and include proxies for cognitive processing during EK tasks can help further disentangle the complex role of EK in social functioning.

Footnotes

Acknowledgements

The authors extend their gratitude to the children, parents, and teachers who participated in the project. They also thank Steven W. Evans for providing feedback on the article.

Authors’ Note

This research was conducted as the first author’s master’s thesis study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Verenea J. Serrano received a reasearch grant from Ohio University’s Graduate Student Senate to support the project. During the preparation of this article, Julie S. Owens was supported by grants from the Department of Education, Institute of Educational Sciences (R324A120003; R324A120272; R324C080006).

Notes

Author Biographies

Verenea J. Serrano, MS, is a doctoral candidate at Ohio University in the Center for Intervention Research in Schools. Her research focuses on (a) identifying factors contributing to social and emotional deficits in children with ADHD so as to improve social interventions for this population and (b) increasing dissemination of quality interventions to underserved populations by reducing barriers to access and quality implementation.

Julie Sarno Owens, PhD, is an associate professor of psychology and co-director of the Center for Intervention Research in Schools at Ohio University. Her program of research is focused on (a) evaluation of effectiveness, feasibility, and sustainment of evidence-based interventions under typical school conditions; (b) modification of procedures to address barriers to intervention implementation and positive student outcomes; and (c) the assessment of impairment in youth with ADHD to inform future intervention modifications.

Brooke Hallowell, PhD, CCC-SLP, is a professor and coordinator of PhD programs in communication sciences and disorders, the director of the Neurolinguistics Laboratory, and the executive director of the Collaborative on Aging at Ohio University. She has 25 years of experience with eye-tracking research and related application development.

References

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.

Boakes

Chapman

Houghton

West

(2008). Facial affect interpretation in boys with attention deficit/hyperactivity disorder. Child Neuropsychology, 14, 82-96.

Cadesky

Mota

Schachar

(2000). Beyond words: How do problem children with ADHD and/or conduct problems process nonverbal information about affect? Journal of the American Academy of Child & Adolescent Psychiatry, 39, 1160-1167.

Casey

(1996). Emotional competence in children with externalizing and internalizing disorders. In Lewis

Sullivan

M. W.

(Eds.), Emotional development in atypical children (pp. 161-183). Hillsdale, NJ: Lawrence Erlbaum.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Crick

Dodge

(1994). A review and reformulation of social information-processing mechanisms in children’s social adjustment. Psychological Bulletin, 115, 74-101.

Da Fonseca

Seguier

Santos

Poinso

Deruelle

(2009). Emotion understanding in children with ADHD. Child Psychiatry & Human Development, 40, 111-121.

Duchowski

(2007). Eye tracking methodology theory and practice (2nd ed.). London, England: Springer.

10.

Ekman

Friesen

(1975). Pictures of facial affect. Palo Alto, CA: Consulting Psychologists Press.

11.

Fabiano

Pelham

Waschbusch

Gnagy

Lahey

Chronis

. . . Burrows-MacLean

(2006). A practical measure of impairment: Psychometric properties of the Impairment Rating Scale in samples of children with attention deficit hyperactivity disorder and two school-based samples. Journal of Clinical Child & Adolescent Psychology, 35, 369-385.

12.

Guillon

Hadjikhani

Baduel

Rogé

(2014). Visual social attention in autism spectrum disorder: Insights from eye tracking studies. Neuroscience & Biobehavioral Reviews, 42, 279-297.

13.

Heuer

Hallowell

(2009). Visual attention in a multiple-choice task: Influences of image characteristics. Aphasiology, 23, 351-363.

14.

Hoza

(2007). Peer functioning in children with ADHD. Journal of Pediatric Psychology, 32, 655-663.

15.

Hoza

Gerdes

Mrug

Hinshaw

Bukowski

Gold

. . . Wigal

(2005). Peer-assessed outcomes in the multimodal treatment study of children with attention deficit hyperactivity disorder. Journal of Clinical Child & Adolescent Psychology, 34, 74-86.

16.

Huang

Eades

Hong

(2009). Measuring effectiveness of graph visualizations: A cognitive load perspective. Information Visualization, 8, 139-152.

17.

Izard

King

Trentacosta

Morgan

Laurenceau

Krauthamer-Ewing

Finlon

(2008). Accelerating the development of emotion competence in Head Start children: Effects on adaptive and maladaptive behavior. Development and Psychopathology, 20, 369-397.

18.

Karatekin

(2007). Eye tracking studies of normative and atypical development. Developmental Review, 27, 283-348.

19.

Kats-Gold

Besser

Priel

(2007). The role of simple emotion recognition skills among school aged boys at risk of ADHD. Journal of Abnormal Child Psychology, 35, 363-378.

20.

Kats-Gold

Priel

(2009). Emotion, understanding, and social skills among boys at risk of attention deficit hyperactivity disorder. Psychology in the Schools, 46, 658-678.

21.

Lee

Hung

Lam

Lee

(2009). The ability of Hong Kong children with attention-deficit hyperactivity disorder to recognise facial emotion. Hong Kong Journal of Psychiatry, 19, 18-25.

22.

Miller

Gouley

Seifer

Zakriski

Eguia

Vergnani

(2005). Emotion knowledge skills in low-income elementary school children: Associations with social status and peer experiences. Social Development, 14, 637-651.

23.

Miller

Hanford

Fassbender

Duke

Schweitzer

(2011). Affect recognition in adults with ADHD. Journal of Attention Disorders, 15, 452-460.

24.

Norvilitis

Casey

Brooklier

Bonello

(2000). Emotion appraisal in children with attention-deficit/hyperactivity disorder and their parents. Journal of Attention Disorders, 4, 15-26.

25.

Pelc

Kornreich

Foisy

Dan

(2006). Recognition of emotional facial expressions in Attention-Deficit Hyperactivity Disorder. Pediatric Neurology, 35, 93-97.

26.

Pelham

Fabiano

Massetti

(2005). Evidence-based assessment of Attention Deficit Hyperactivity Disorder in children and adolescents. Journal of Clinical Child & Adolescent Psychology, 34, 449-476.

27.

Pelham

Gnagy

Greenslade

Milich

(1992). Teacher ratings of DSM-III-R symptoms for the disruptive behavior disorders. Journal of the American Academy of Child & Adolescent Psychiatry, 31, 210-218.

28.

Pishyareh

Tehrani-Doost

Mahmoodi-Gharaie

Khorrami

Joudi

Ahmadi

(2012). Attentional bias towards emotional scenes in boys with Attention Deficit Hyperactivity Disorder. Iranian Journal of Psychiatry, 7, 93-96.

29.

Rapport

Friedman

Tzelepis

Van Voorhis

(2002). Experienced emotion and affect recognition in adult attention-deficit hyperactivity disorder. Neuropsychology, 16, 102-110.

30.

Shin

Lee

Kim

Park

Lim

(2008). Visual attention deficits contribute to impaired facial emotion recognition in boys with attention-deficit/hyperactivity disorder. Neuropediatrics, 39, 323-327.

31.

Singh

Ellis

Winton

Singh

Leung

Oswald

(1998). Recognition of facial expressions of emotion by children with attention-deficit hyperactivity disorder. Behavior Modification, 22, 128-142.

32.

Sinzig

Morsch

Lehmkuhl

(2008). Do hyperactivity, impulsivity and inattention have an impact on the ability of facial affect recognition in children with autism and ADHD? European Child & Adolescent Psychiatry, 17, 63-72.

33.

Wechsler

(2011). Wechsler Abbreviated Scale of Intelligence–Second edition (WASI-II). San Antonio, TX: Pearson.

34.

Weller

Rooney

Fristad

Weller

(1999). Children’s interview for psychiatric syndromes: Parent version. Washington, DC: American Psychiatric Press.

35.

Yuill

Lyon

(2007). Selective difficulty in recognising facial expressions of emotion in boys with ADHD: General performance impairments or specific problems in social cognition? European Child & Adolescent Psychiatry, 16, 398-404.

36.

Zoromski

Owens

J. S.

Evans

S. W.

Brady

(2015). Identifying ADHD Symptoms that Best Predict Disorder-Related Impairment in Early Childhood, Middle Childhood, and Adolescence. Journal of Abnormal Child Psychology. Advance online publication. doi: 10.10007/s10802-015-0017-8