Feasibility of Ecological Momentary Assessment of Negative Emotion in Girls With ADHD: A Pilot Study

Introduction

Attention-deficit/hyperactivity disorder (ADHD) is diagnosed two to nine times more frequently in boys than girls (Arnold, 1996), and as a result, much less research has been conducted on girls than boys with ADHD. While it had once been assumed that girls manifested a similar or sometimes less severe presentation of ADHD compared to boys with the disorder (Arnold, 1996), emerging work suggests that girls with ADHD demonstrate heightened risk for depression that is greater and more severe than what has been documented among boys with ADHD (Biederman et al., 2008; Chronis-Tuscano et al., 2010). By early adulthood, as many as 67.9% of girls with ADHD have experienced a major depressive episode (Owens, Zalecki, Gillette, & Hinshaw, 2017), which is at least double the prevalence reported in the general population of women (Kessler et al., 2003). The co-occurrence of ADHD and depression in girls is associated with severe outcomes, including suicidality, self-harm, and substance use (Biederman et al., 2008; Chronis-Tuscano et al., 2010). Thus, attention to examining factors associated with depression among girls with ADHD is of great importance.

Accumulating evidence has identified emotion regulation (ER) difficulties as a key mechanism underlying the risk for depression in youth with ADHD (Seymour et al., 2012; Steinberg & Drabick, 2015). While the definition of ER difficulties is broad, ER difficulties typically involve problems modulating the intensity of emotion, particularly problems related to inhibiting and controlling negative emotional reactions to distressing or frustrating situations (Cole, Martin, & Dennis, 2004). Large differences have been documented among youth with and without ADHD in regulating negative emotion (Graziano & Garcia, 2016). However, the majority of work examining ER among youth with ADHD to date has relied on self-reports from parents and youth. While informative, these one-time assessments may mask more nuanced patterns of ER that may emerge over time. Specifically, methodologies that consider both the intensity of emotional response and the duration of the emotional response are needed to comprehensively study ER difficulties in youth with ADHD (Bunford, Evans, & Wymbs, 2015). There has been some work using observational paradigms to assess ER difficulties in youth with ADHD during frustrating tasks, such as attempting to complete a puzzle with missing pieces or peer competitions (e.g., Melnick & Hinshaw, 2000; Seymour et al., 2012). Results from these tasks suggest that compared to youth without ADHD, youth with ADHD exhibit more negative emotion reactivity when frustrated and require more time to regulate this emotion. However, the magnitude of effects between youth with and without ADHD from observational studies has been smaller than those reported from parent and teacher ratings, raising the possibility that alternative methods of assessment may be needed to assess ER difficulties in youth with ADHD. Additionally, girls have typically not been included in these studies, despite the apparent importance of identifying mechanisms of the risk for depression in adolescent girls with ADHD. Thus, exploration of additional methods to assess ER and inclusion of girls is worthwhile.

Ecological momentary assessment (EMA) may provide unique insight into the assessment of ER in girls with ADHD (Heron, Everhart, McHale, & Smyth, 2017). Using mobile phones, EMA can be used to collect frequent and naturalistic assessments of girls’ ER in real time. EMA data collection occurs outside the laboratory setting allowing for an ecologically valid assessment of ER, and collection of ER at multiple time points provides a reliable measure of ER. Such assessment reduces the limitations of traditional self-report methods, such as retrospective recall bias, memory difficulties, and self-perception difficulties (Heron et al., 2017), which have been documented among youth with ADHD (Owens, Goldfine, Evangelista, Hoza, & Kaiser, 2007). A large and growing literature shows that EMA with adults provides useful information, and emerging work suggests that it is useful for collecting information from youth as young as eight years old (Heron et al., 2017).

A few emerging studies have used EMA to examine ER difficulties in samples comprised predominantly of school-aged boys with ADHD (Rosen, Epstein, & Van Orden, 2013; Rosen & Factor, 2015). For example, in one study (Rosen et al., 2013), parents of 11 youth (ages 8–11 years old) with ADHD were asked to complete ratings of their children’s overall mood on an 11-point scale, −5 (much worse mood than usual) to +5 (much better mood than usual), with 0 equivalent to normal mood. Ratings were collected three times per day (i.e., before school, after school, and evening) over the course of 28 days. Parents demonstrated a high level of adherence to study procedures (i.e., 86.53% of ratings were completed), and EMA ratings of greater emotional intensity and variability were significantly associated with internalizing problems. When youth were asked to complete similar assessment procedures across a 28-day period, similarly high adherence (i.e., 84.43%) was reported, and youth reports were significantly correlated with parent EMA ratings (Rosen et al., 2013). However, youth ratings were generally more positive than parent ratings. In addition, there was some evidence that youth were less likely to not complete ratings when they were rated as distressed by their parent, suggesting potential bias in youth EMA ratings, and raising questions about the feasibility and validity of EMA for youth with ADHD. It is not clear whether similar biases would be evident among adolescent girls with ADHD. Although self-perception difficulties have been widely reported among youth, largely school-aged boys with ADHD, emerging research focused on girls with ADHD has not found pronounced self-perception biases (Swanson, Owens, & Hinshaw, 2012). It may be that ratings of mood may be more salient to adolescent girls compared to previous studies of preadolescent youth. This may be due to age-related increases in self-awareness (Sebastian, Burnett, & Blakemore, 2008) or to the increasing salience of mood in adolescence (Biederman et al., 2008; Chronis-Tuscano et al., 2010).

The primary goal of this study was to evaluate the feasibility of using EMA for assessing ER in an exploratory sample of 13 adolescent girls with ADHD. Feasibility was assessed by examining the rate of adherence to EMA prompts by girls and their mothers. Collection of both parent and youth reports is important to provide a comprehensive assessment of youth functioning, particularly given previous work that has raised concerns about the validity of self-report ratings among youth with ADHD (Owens et al., 2017; Rosen et al., 2013). To further evaluate the feasibility of EMA, concordance between youth and maternal EMA ratings of the intensity and variability of negative emotion was also examined as a measure of convergent validity. Finally, the intensity and fluctuation of negative mood were compared between girls with ADHD and comorbid depression versus ADHD alone as a measure of discriminant validity. Based on accumulating evidence demonstrating the feasibility of EMA (Heron et al., 2017; Rosen & Factor, 2015), we hypothesized that EMA would be a feasible method to collect information on ER among adolescent girls with ADHD and their mothers. Specifically, we hypothesized that girls and parents would demonstrate adherence to the EMA protocol. In addition, we hypothesized that maternal and youth EMA ratings of negative emotion would be significantly associated. Finally, based on previous work showing ER dysfunction as a marker of depression (e.g., Seymour et al., 2012), we hypothesized that greater ER difficulties would be demonstrated in girls with versus without comorbid depression based on youth and maternal reports.

Methods

Participants

A total of 17 girls with ADHD were recruited to participate in a pilot study broadly assessing mood and social functioning. Girls were recruited using advertisements posted within an academic medical center and the surrounding community in the mid-Atlantic region of the United States. Girls were required to be between 12 and 16 years old (inclusive) and have an IQ >70 based on the Wechsler Abbreviated Scales of Intelligence (Wechsler, 2011). ADHD, oppositional defiant disorder (ODD), and conduct disorder (CD) diagnoses were assigned based on maternal report on the Disruptive Behavior Disorders Interview (DBD-I; Pelham, Fabiano, & Massetti, 2005). The DBD-I is a semistructured interview examining DSM (Diagnostic and Statistical Manual of Mental Disorders, 5th Edition) symptoms of ADHD, ODD, and CD, with supplemental probes for symptom severity and situational variability; the DBD-I was conducted by a master’s or doctoral level clinician. High levels of internal consistency have been reported for symptoms of ADHD using the DBD-I (alpha = .93 for inattentive symptoms, alpha = .81 for hyperactive/impulsive symptoms; Hartung, McCarthy, Milich, & Martin, 2005). Furthermore, parent ratings on the DBD-I are strongly correlated with ADHD symptom dimensions collected using self-report questionnaires (r = .90 for inattentive symptoms, r = .81 for hyperactive/impulsive symptoms; Hartung et al., 2005). Mothers also completed the Disruptive Behavior Disorders Rating Scale (Pelham et al., 2005). Diagnoses were made if a sufficient number of symptoms were endorsed (considering all available diagnostic information). In addition, mood and anxiety disorder modules of the Schedule for Affective Disorders and Schizophrenia for School-Aged Children (Kaufman et al., 1997) were administered to girls and their mothers to assess current co-occurring psychopathology. Autism, schizophrenia, psychosis, and non-English speaking were exclusionary.

Study procedures were completed in two appointments conducted from September 2016 through January 2017. Diagnostic information was collected in the first appointment, and girls and their mothers were provided with cell phones to collect EMA ratings at the end of the first appointment. Cell phones were returned at the second study visit, when additional measures of social–emotional functioning were collected including self-report rating scales from mothers and girls, behavioral observation, and neurophysiological assessment. Four girls and their mothers did not complete EMA procedures. One family took the phones but did not complete ratings, as they later decided to withdraw from the study. The other three families did not complete initial diagnostic procedures and were lost to follow-up. Thus, a total of 13 girls (5 with a comorbid diagnosis of depression) and their mothers completed EMA study procedures, and their data are described herein. Demographic characteristics are presented in Table 1. One girl with ADHD only was medicated for anxiety, and one girl in the ADHD with comorbid depression group was medicated for depression and anxiety. Girls with ADHD and depression were significantly more likely to be diagnosed with a comorbid anxiety disorder compared to girls with ADHD only. No differences in age, race, ethnicity, IQ, ADHD presentation, ADHD medication status, or comorbid ODD/CD diagnosis emerged between girls with ADHD only and girls with ADHD and depression.

OPEN IN VIEWER

Table 1.

Demographic characteristics.

	ADHD (n = 8)	ADHD + depression (n = 5)	p
Age (M, SD)	14.13 (1.73)	12.80 (0.84)	.14
White race (%)	87.50	80.00	.72
Hispanic/Latino (%)	12.50	20.00	.72
IQ (M, SD)	100.00 (12.55)	102.80 (20.96)	.77
ADHD presentation			.57
Inattentive (%)	75.00	60.00
Combined (%)	25.00	40.00
ADHD medication (%)	37.50	50.00	.43
ODD/CD diagnosis (%)	37.50	20.00	.51
Anxiety disorder (%)	37.50	100.00	.02

ADHD: attention-deficit/hyperactivity disorder; ODD: oppositional defiant disorder; CD: conduct disorder; SD: standard deviation.

EMA of negative emotion

Girls and their mothers were provided with cell phones that were modified to allow access only to the surveys and not internet or call out features. Participants were instructed that the device would signal at random times to complete a rating. Cell phones were programmed so that survey prompts were sent to girls and their mothers for one week (Monday–Friday between 4:00–6:00 pm and 7:30–9:30 pm; Saturday–Sunday between 11:00 am–12:30 pm, 2:00–3:30 pm, 4:30–6:00 pm, and 7:30–9:30 pm). The time frames were selected so that cell phone use would not interfere with girls’ school, and also because this time period allows for variable levels of freedom and social interaction. Participants had the opportunity to use a snooze feature that would postpone the completion of the survey for 10 minutes. After 10 minutes if the survey was not completed, participants were again prompted to complete the survey. The snooze feature could be used multiple times, but phones were programmed so that if participants did not respond to the prompt within 60 minutes, they would no longer have access to the survey. Participants were reminded to charge the cell phones during the week and to keep phones on in order to receive surveys. Technical problems were reported by one family (Participant 03; prompts for EMA collection were not initially received). However, this issue was resolved by resetting the survey programming allowing for adequate data collection. Participants were instructed to complete EMA ratings for one week, although in some cases participants completed additional ratings when their return visit was scheduled later than one week from their initial study visit. The total available data were included due to the exploratory and descriptive nature of this pilot study. Data were collected over an average of 8.3 days (range: 4–14 days).

Intensity of negative emotion

Each phone assessment measured negative emotion intensity using six items (i.e., lonely, sad, worried, bored, hopeless, and in pain (stomachaches, headaches, etc.)), each rated on a scale from 1 (very slightly or not at all) to 5 (very much). Items reflecting sadness and worry were selected from the short form of the Positive and Negative Affect Schedule for Children (Ebesutani et al., 2012; Laurent et al., 1999), which has been used with EMA collection in youth (Dunton et al., 2014; Rosen, Walerius, Fogleman, & Factor, 2015), and additional items reflecting depressive symptomology, including hopelessness, boredom, and loneliness were selected (American Psychiatric Association, 2013). An item assessing pain was also included, as somatic complaints have been shown to be a common manifestation of negative emotion in youth (Terwogt, Rieffe, Miers, Jellesma, & Tolland, 2006). Girls were asked to rate their own negative emotion, and mothers were asked to rate the negative emotion of their daughters (see Table 2). Total negative emotion scores were created by summing scores across all six emotions for girls (Cronbach’s alpha = .84) and their mothers (Cronbach’s alpha = .75). However, to obtain the most comprehensive assessment, both adolescent and maternal ratings were also combined by taking the maximum score on an item-by-item basis as has been done in past research and as recommended on theoretical and empirical grounds (e.g., Rhew et al., 2010). Total negative emotion using combined maternal and youth reports was calculated by summing ratings across all negative emotion ratings (Cronbach’s alpha = .82). Scores ranged from 6 to 30, with higher scores indicating greater negative emotion.

OPEN IN VIEWER

Table 2.

Ecological momentary assessment ratings of negative emotion intensity and variability by youth, maternal, and combined reports.

	Total sample	ADHD	ADHD + depression	ES
Youth report
Intensity of negative emotion
Pain	1.74 (1.26)	1.47 (1.06)	2.34 (1.46)	0.68**
Hopeless	1.34 (0.92)	1.14 (0.65)	1.82 (1.23)	0.69**
Worried	1.37 (0.85)	1.09 (0.36)	2.00 (1.25)	1.00**
Bored	1.91 (1.26)	1.69 (1.11)	2.39 (1.45)	0.54**
Sad	1.37 (0.84)	1.12 (0.45)	1.95 (1.19)	0.92**
Lonely	1.25 (0.73)	1.08 (0.37)	1.66 (1.12)	0.70**
Total negative emotion	8.93 (4.43)	7.62 (2.34)	12.15 (6.22)	0.96**
Variability of negative emotion	15.75 (38.87)	7.32 (16.51)	36.81 (63.98)	0.63*
Maternal report
Intensity of negative emotion
Pain	1.39 (0.90)	1.19 (0.65)	1.78 (1.17)	0.62**
Hopeless	1.13 (0.51)	1.01 (0.12)	1.37 (0.81)	0.62**
Worried	1.42 (0.78)	1.26 (0.63)	1.74 (0.94)	0.60**
Bored	1.79 (1.02)	1.53 (0.76)	2.30 (1.25)	0.74**
Sad	1.31 (0.71)	1.17 (0.53)	1.58 (0.91)	0.55**
Lonely	1.34 (0.68)	1.21 (0.50)	1.59 (0.91)	0.52**
Total negative emotion	8.37 (3.13)	7.39 (1.96)	10.31 (4.00)	0.93**
Variability of negative emotion	9.92 (19.42)	5.18 (11.48)	19.64 (27.44)	0.69**
Combined youth and maternal reports
Intensity of negative emotion
Pain	1.74 (1.23)	1.45 (1.04)	2.27 (1.37)	0.67**
Hopeless	1.31 (0.85)	1.12 (0.58)	1.66 (1.12)	0.61**
Worried	1.53 (0.91)	1.28 (0.65)	1.99 (1.13)	0.77**
Bored	2.08 (1.21)	1.85 (1.07)	2.50 (1.35)	0.53**
Sad	1.47 (0.89)	1.25 (0.62)	1.90 (1.15)	0.70**
Lonely	1.41 (0.80)	1.23 (0.55)	1.74 (1.06)	0.60**
Total negative emotion	9.53 (4.36)	8.18 (2.60)	12.06 (5.69)	0.88**
Variability of negative emotion	17.78 (41.23)	9.52 (16.35)	33.79 (64.37)	0.52*

Note: ES was calculated for comparisons between girls with ADHD versus girls with ADHD and comorbid depression. Combined youth and maternal reports were computed by taking the maximum score on an item-by-item considering youth and maternal ratings. ES: Cohen’s d effect size; ADHD: attention-deficit/hyperactivity disorder.

*p < .05; **p < .01.

Variability of negative emotion

The average Mean Square Successive Difference (MSSD) score was used to assess variability of total negative emotion (Rosen & Factor, 2015; Solhan, Trull, Jahng, & Wood, 2009). The MSSD is calculated by determining the difference in each successive, within day total negative emotion rating, then squaring this difference. Thus, for weekday assessments, the MSSD would be calculated from the afternoon (i.e., 4–6 pm) assessment to the evening (i.e., 7:30–9:30 pm) assessment, and for weekends, the MSSD was calculated from the 11 am–12:30 pm assessment to the 2:00–3:30 pm assessment, from the 2:00–3:30 pm assessment to the 4:30–6:00 pm assessment, and from the 4:30–6:00 pm to 7:30–9:30 pm assessment.

Results

Negative emotion ratings in girls with ADHD only versus ADHD + depression

Intensity of negative emotion

Parent, youth, and combined EMA ratings of negative emotion intensity for girls with ADHD only versus ADHD + depression are reported in Table 2. All negative emotion ratings were significantly different between girls with ADHD alone and ADHD + depression, with ratings for girls with ADHD and depression significantly greater than ratings of girls with ADHD alone.

Variability of negative emotion

Negative emotion variability MSSD scores by maternal, child, and combined reports for girls with and without comorbid depression are presented in Table 2. Given the significant correlation between maternal and youth reports, and consistent with guidelines showing that the combination of parent and youth reports is associated with the most meaningful information (Rhew et al., 2010), combined parent–youth reports are presented visually in figures. Total negative emotion ratings using combined parent and youth reports are presented for youth with ADHD alone in Figure 1 and for youth with ADHD and depression in Figure 2.

OPEN IN VIEWER

Figure 1.

Combined youth and maternal negative emotion ratings for girls with ADHD only. X-axis depicts each time point, and Y-axis depicts the total intensity of negative emotion scored calculated using combined parent and youth reports. Dx: diagnosis; y.o.: years old; ADHD I: ADHD inattentive presentation; ADHD-C: ADHD combined presentation. Response rate was calculated based on combined parent and youth reports. All available data are presented visually due to the exploratory nature of the study. ODD: oppositional defiant disorder; CD: conduct disorder; ADHD: attention-deficit/hyperactivity disorder.

OPEN IN VIEWER

Figure 2.

Combined youth and maternal negative emotion ratings for girls with ADHD and depression. X-axis depicts each time point, and Y-axis depicts the total intensity of negative emotion scored calculated using combined parent and youth reports. Dx: diagnosis; y.o.: years old; ADHD I: ADHD inattentive presentation; ADHD-C: ADHD combined presentation. Response rate was calculated based on combined parent and youth reports. All available data are presented visually due to the exploratory nature of the study. ODD: oppositional defiant disorder; CD: conduct disorder; ADHD: attention-deficit/hyperactivity disorder.

Discussion

This exploratory study examined the feasibility of EMA to assess ER difficulties in adolescent girls with ADHD. Mobile phones were provided to both girls and their mothers to assess ratings of girls’ negative emotions. The intensity of negative emotion and the variability of negative emotion were assessed using maternal and youth reports. The results demonstrated that EMA was a feasible and valid method to assess negative emotion in girls with ADHD. There was a high level of compliance with daily assessment procedures, and youth and maternal ratings of negative emotion were significantly correlated. Furthermore, EMA ratings differentiated girls with ADHD with and without comorbid depression. Specifically, girls with comorbid depression demonstrated greater negative emotion and significantly more variability in negative emotion compared to girls with ADHD only. These findings are discussed herein.

The high rate of adherence among participants was at or better than rates of adherence reported in previous studies of youth (e.g., Heron et al., 2017), and specifically youth with ADHD (Rosen et al., 2013; Rosen & Factor, 2015). The high level of adherence to study procedures is also notable, as procedures in the current study required most parents and some girls to carry around two devices (the study phone and their own cellphone), which may have been cumbersome. Emerging research suggests alternative methods of collecting EMA, including using mobile phone-based applications that can be downloaded onto participants’ own cellphone (Kleiman et al., 2017), which may be useful for future EMA research with adolescent youth with ADHD. Previous EMA studies have focused on elementary-aged youth with ADHD, who may have limited self-control to complete frequent EMA ratings. It is plausible that negative mood was a highly salient topic for girls with ADHD and their mothers, which resulted in high engagement in completing EMA surveys.

The high adherence to the EMA protocol may also be due to the relatively short period of time EMA was collected, as previous studies have collected EMA ratings over the course of intervals lasting as long as a month (Heron et al., 2017; Rosen et al., 2013). Importantly, even with our relatively brief interval of assessment over the course of approximately one week, it appears that valid data were collected. With the exception of girls rating greater pain compared to mothers, and mothers rating endorsing greater loneliness compared to youth report, consistent differences between maternal and youth reports of negative emotion were not identified. The similarity between maternal and youth reports contrasts previous EMA research showing youth with ADHD provide positively skewed ratings of mood (Rosen et al., 2013) and may be due to increasing self-insight in adolescents versus elementary-aged youth (Sebastian et al., 2008). In addition, maternal and youth reports of negative emotion were significantly correlated, and the strength of associations between maternal and youth ratings of negative emotion was in line with previous work showing modest associations between ratings of functioning collected across raters (Verhulst & van der Ende, 1992; Youngstrom, Loeber, & Stouthamer-Loeber, 2000).

Comparisons between girls with ADHD and comorbid depression compared to girls with ADHD alone also provide evidence of the feasibility of collecting valid EMA ratings. Girls with ADHD and comorbid depression rated greater negative emotion intensity and fluctuation by self, maternal, and combined reports. These consistent group differences fit with existing research showing ER difficulties to be a marker of depression in youth with (Seymour et al., 2012) and without ADHD (Silk, Steinberg, & Morris, 2003). Notably, girls with ADHD and depression demonstrated highs and lows in negative emotion, which is somewhat distinct from conceptualizations of depression that involve persistent negative mood (Bennik, Nederhof, Ormel, & Oldehinkel, 2014). As our study is cross-sectional, we are not able to determine whether the greater negative emotion and fluctuations in negative emotion also identify risk for depression in girls prior to the development of depression. For example, some of the girls in the nondepressed group (e.g., participant 11) displayed variability in ratings of negative affect, which may potentially signal risk for the development of depression later in development. However, ER dysregulation has also been associated with ADHD (Barkley, 2015; Martel, 2009), and inclusion of a sample of girls without ADHD is needed to identify whether unique patterns of regulating negative emotion are associated with ADHD versus depression. Similarly, the majority of girls in the ADHD and depression group were also diagnosed with an anxiety disorder, which may also contribute to these findings.

It is important to mention several limitations of the current work. As this is a small pilot study that included only girls with ADHD, generalizability is limited. In addition, we focused only on regulation of six specific negative emotions and attention to other measures of ER, including changes in positive affect, frustration tolerance, or anger may also be relevant to consider. We incorporated maternal and youth EMA ratings in order to provide the most comprehensive evaluation of mood, which has been recommended on empirical and theoretical grounds (e.g., Rhew et al., 2010). However, it may also be useful in a larger sample to consider more nuanced differences in maternal and youth ratings or potential covariates, including ADHD medication use and additional comorbidities (e.g., ODD, anxiety). A larger sample may also afford opportunities to consider alternative methods of data analysis, such as partial least squares algorithms (Hair, Ringle, & Sarstedt, 2011). We did not consider information on other important areas of functioning, particularly social functioning, which may serve to further contextualize these findings. Peer difficulties have been identified in the development of youth with ADHD (Humphreys et al., 2013). Thus, attention to the relationship between peer stress and changes in negative emotion over time may serve to clarify antecedents to mood difficulties in girls.

Despite these limitations, this work provides an important step toward prioritizing the study of girls with ADHD in adolescence when mood difficulties become particularly prominent. Indeed, given these promising preliminary findings, additional research incorporating EMA holds great promise for understanding the real-time functioning of girls with ADHD. Accumulating research shows that ER difficulties among individuals with ADHD are associated with a broad range of psychopathology and functional impairments beyond depression, including social and occupational impairment, as well as substance abuse (Barkley, 2015). Thus, continued attention to assess this construct is of critical importance.