Measurement of Emotion Regulation in a Clinical Sample of Children: The Psychometric Properties of the How I Feel Scale

Abstract

Emotion regulation (ER) is a crucial construct in the study of child education, development, and psychopathology. The How I Feel (HIF) questionnaire, a self-report measure of ER for children, was validated in school-based, nonclinical samples. The current study examined the factor structure of the HIF in a clinical sample of children (N = 195; 73.33% male; 79.49% White; 61.03% diagnosed with ADHD). Confirmatory factor analysis did not support the original 3-factor model. An exploratory factor analysis revealed an altered five-factor model with identical Positive Emotion and Emotion Control factors, but the original Negative Emotion factor was divided into three factors: Mad Emotion, Sad and Scared Frequency, and Sad and Scared Intensity. Results suggest that the HIF may not accurately assess the intensity and frequency of negative emotions in clinical samples.

Keywords

emotion regulation how I feel child assessment confirmatory factor analysis

Emotion regulation (ER) is ubiquitous and constant, prompting sizable research of its correlates, predictors, and consequences (Adrian et al., 2011). In children, ER is important for a range of outcomes, including internalizing and externalizing symptomatology (Compas et al., 2017), academic success (Ivcevic & Brackett, 2014), social functioning (Bunford et al., 2014), and resilience to life stressors (Eisenberg et al., 2003). Therefore, developing a reliable and valid ER measure for children is important for studying child education and development.

Despite the importance of ER in child psychology, it is a challenging construct to measure. ER involves behaviors, subjective internal experience, cognitions, physiological changes, and social interactions (Solomon, 2002). ER is interrelated with emotion intensity, frequency, and expression, as well as heavily dependent on context (Cole et al., 2004). Hence, assessing just one of these domains, which is the easiest and most common approach, is insufficient to fully understand ER (Adrian et al., 2011). In addition, assessing ER in children is particularly complex as they may have less insight into their internal states, have more limited language abilities, and generally demonstrate more variable responding (Zeman et al., 2007). There is a need for multi-domain, developmentally sensitive, and easily used assessment approaches.

The How I Feel Scale (HIF; Walden et al., 2003) is a self-report scale of positive emotion, negative emotion, and emotion control developed to assess both emotional arousal and control. The HIF is a multi-domain scale for 8 to 12-year-old children assessing intensity, frequency, and controllability of “sad,” “mad,” “happy,” “excited,” and “scared” emotions (Walden et al., 2003). Respondents rate the extent to which sentences are true of them over the past three months on a 1 to 5 Likert scale where 1 is “Not at all true of me” and 5 is “Very true of me.” The HIF has demonstrated good test-retest reliability (i.e., alphas ranging from .79 to .92), as well as strong content and convergent validity (Walden et al., 2003). To develop the measure, the authors conducted iterative studies to generate items, gathered expert judgment on content validity of items, and conducted both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) to determine the factor structure of the scale (Walden et al., 2003). The original sample was drawn from a study of school-based mental health in urban schools with a large number of economically marginalized students. In this sample, the factor analysis identified three factors—Negative Emotion, Positive Emotion, and Emotion Control (Walden et al., 2003). Another study using a school-based sample in Italy replicated the same factor structure, invariance across sex and age, and similar levels of stability over time (Ciucci et al., 2016).

Both existing psychometric studies of the HIF have been conducted in school-based samples and nonclinical samples, meaning the psychometric properties of the HIF are unknown in youth seeking psychological services. Given that psychometric properties may differ between clinical and general samples (Page et al., 2007), this should be assessed in both contexts. Therefore, this study sought to assess the psychometric properties of the HIF in a clinical sample.

Method

This project used data that were collected as part of ongoing IRB-approved treatment effectiveness research of a group therapy intervention at a private psychotherapy clinic. Youth were eligible to participate in the larger study if they experienced difficulties with peer relationships and psychosocial functioning. Following consent and assent, youth and their caregivers filled out a battery of questionnaires, including the HIF, before and after treatment, for which youth were compensated $20. Only pre-treatment HIF data were used in the current study.

A total of 484 youth, ages 7 to 15, were eligible for participation in the larger study. The current sample includes only the 195 children (M_age = 10.21, SD = 1.19) who were between the ages of 8 and 12 and completed the HIF measure. The majority of participants were male (n = 143; 73.33%), White (n = 155; 79.49%), lived with both biological parents (n = 143; 73.33%), and reported an annual family income over $100,000 (n = 159; 81.54%). The most common diagnoses were attention deficit/hyperactivity disorder (ADHD; n = 119; 61.03%), generalized anxiety disorder (n = 39; 20.00%), and major depression (n = 13; 6.67%), with 27.18% of participants reporting more than one psychiatric diagnosis (n = 53).

We conducted a confirmatory factor analysis (CFA) to examine the validity of the conceptual model in Walden et al. (2003) using the current clinical youth sample. Data analyses were conducted in R Studio with the package “lavaan” (Rosseel, 2012). We first examined the measurement model with all 30 items loaded on 3 latent emotion factors (Positive Emotion, Negative Emotion, Emotion Control). Then we examined the conceptual model (Figure 1) which had a two-layer latent structure in which the 30 questionnaire items were loaded on three emotion factors (layer-1; Positive Emotion, Negative Emotion, Emotion Control) as well as on two experience factors (layer-2; Frequency, Intensity). This conceptual model was constructed in the same manner as in Walden et al. (2003). The result of the Henze–Zirkler’s Multivariate Normality test suggested the current sample violated the assumption of multivariate normality (Positive Emotion: HZ = 2.09, p < .001; Negative Emotion: HZ = 1.50, p < .001; Emotion Control: HZ = 1.30, p < .001). To account for the non-normality in the questionnaire items, we estimated parameter standard errors with maximum likelihood estimation with robust (Huber-White) standard errors (Lai, 2018). Parameters were estimated using full information maximum likelihood to account for missing values. Given that the model fit of Walden et al.‘s conceptual model (2003) was low in the current sample, we subsequently conducted an exploratory factor analysis (EFA) for model modification. Data analyses were conducted in R Studio using the package “psych” (Revelle, 2020). First, we conducted a parallel analysis to determine the number of components. We then conducted the factor analysis with the number of factors suggested in the previous step. Factor extraction was conducted using Principal Axis Factoring with Minimum Residuals to account for non-normal data (Zygmont & Smith, 2014). Factor rotation was conducted using Promax approach to allow for inter-factor correlations (Zygmont & Smith, 2014).

Figure 1.

Two-layer conceptual model in Walden et al. (2003) with standardized path coefficients estimated using the clinical youth sample in the current study. Exp = Experience. Ctrl = Control. #p < .10. *p < .05. **p < .01. ***p < .001.

Results

Preliminary Analyses

The means, standard deviations, and number of missing values for each item are shown in Table 1. The internal consistency was good to excellent for the three subscales in the current sample. The Positive Emotion subscale had a Cohen’s alpha of .89, 95% CI = [.87, .92]. The Negative Emotion subscale had a Cohen’s alpha of .90, 95% CI = [.88, .92]. The Emotion Control subscale had a Cohen’s alpha of .85, 95% CI = [.82, .88].

Table 1.

Means, Standard Deviations, Number of Missingness for all How I Feel Items.

HIF items	Mean	SD	# of Missingness
Happy experience often (1)	3.79	1.04	0
Happy experience all the time (16)	3.41	1.25	1
Excited experience often (19)	3.21	1.30	2
Excited experience all the time (4)	3.14	1.25	0
Happy experience powerful (11)	3.70	1.18	1
Happy experience strong (26)	3.71	1.15	2
Excited experience powerful (29)	3.56	1.29	3
Excited experience strong (14)	3.57	1.22	4
Sad experience often (7)	2.03	1.14	0
Sad experience all the time (22)	1.78	1.06	2
Mad experience often (13)	2.18	1.13	2
Mad experience all the time (28)	1.79	.96	3
Scared experience often (25)	1.81	1.08	5
Scared experience all the time (10)	1.60	.98	1
Sad experience powerful (17)	2.83	1.26	2
Sad experience strong (2)	3.02	1.25	0
Mad experience powerful (23)	2.98	1.35	2
Mad experience strong (8)	3.05	1.30	0
Scared experience powerful (5)	2.69	1.38	1
Scared experience strong (20)	2.56	1.30	2
Happy control frequency (21)	3.58	1.21	4
Excited control frequency (9)	3.20	1.36	1
Happy control intensity (6)	3.35	1.32	3
Excited control intensity (24)	3.35	1.35	3
Sad control frequency (27)	2.90	1.30	3
Sad control intensity (12)	2.81	1.26	2
Mad control frequency (3)	2.86	1.16	3
Mad control intensity (18)	2.69	1.13	2
Scared control frequency (15)	2.98	1.334	2
Scared control intensity (30)	2.79	1.31	3

ANOVA comparisons of the four diagnostic groups (ADHD, GAD, depression, comorbidity) found nonsignificant differences in HIF scores on Positive Emotion, F (4, 190) = .38, p = .826, Negative Emotion, F (4, 190) = .38, p = .821, and Emotional Control, F (4, 190) = .89, p = .471.

Confirmatory Factor Analyses

Although all items for the CFA measurement model had significant loadings on the measurement emotion factors, the measurement model as determined by Walden et al. (2003) provided a poor fit to the current clinical sample, χ² (402) = 1103.20, p < .001, CFI = .72, TLI = .70, RMSEA = .10 [.10, .11], SRMR = .11. Inclusion of the experience factors (intensity and frequency) significantly improved model fit, χ² (30) = 232.55, p < .001, but the model fit was still poor, χ² (372) = 785.46, p < .001, CFI = .84, TLI = .82, RMSEA = .08 [.07, .09], SRMR = .14. Nonetheless, the conceptual model did not yield significant path coefficients for all factors. While all items on Positive Emotion and Emotion Control had significant coefficients, only items associated with “mad” emotion were significantly correlated with the Negative Emotion factor (Figure 1). Dropping the Negative Emotion factor produced marginally acceptable model fit, χ² (117) = 7236.91, p < .001, CFI = .90, TLI = .87, RMSEA = .07 [.06, .09], SRMR = .07. Thus, a model modification specifically for the Negative Emotion factor might be suggested for the use of the HIF in clinical youth samples.

Exploratory Factor Analysis

The parallel analysis suggested the model might have five factors. Items with factor loadings greater than .40 were retained. Table 2 shows the factor loadings after rotation and the proportion of variance accounted for by each factor. Factor 1 was identical to the Positive Emotion factor in Walden et al. (2003), including eight items related to positive emotion. Factor 1 had factor loadings ranging from .57 to .81, accounting for 15% of the total variance. Factor 2 was identical to the Emotion Control factor in Walden et al. (2003). Factor 2 had 10 items with factor loadings ranging from .43 to .76, accounting for 12% of the total variance. As suggested by our CFA, the Negative Emotion factor in Walden et al. (2003) needed major modifications, and it was divided into Factors 3 to 5 in the current EFA. Factor 3 included four items related to the intensity of sad and scared emotions and had loadings ranging from .65 to .79, accounting for 11% of the total variance. Factor 4 consisted of three items on the frequency of sadness and scaredness, with loadings from .54 to .71, and accounted for 7% of the total variance. Factor 5 included four items related to mad frequency and intensity with loadings from .49 to .90 and explained about 7% of the total variance. All five factors together explained 52% of the total variance.

Table 2.

Factor Loadings for How I Feel Items and Proportion of Variance Explained by Each Factor.

HIF Items	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5
Happy experience often (1)	.57	.06	−.18	−.05	−.07
Happy experience all the time (16)	.75	−.03	−.23	−.05	−.03
Excited experience often (19)	.81	.06	−.07	.17	−.03
Excited experience all the time (4)	.73	.05	−.17	.19	−.09
Happy experience powerful (11)	.76	−.09	.12	−.07	.02
Happy experience strong (26)	.74	−.03	.05	−.12	.08
Excited experience powerful (29)	.72	−.05	.24	.03	.02
Excited experience strong (14)	.72	−.01	.35	.00	.02
aSad experience often (7)	−.16	−.01	.40	.35	.13
Sad experience all the time (22)	−.19	.04	.28	.54	.13
Mad experience often (13)	−.12	.01	−.08	.22	.66
bMad experience all the time (28)	.12	−.06	−.42	.48	.90
Scared experience often (25)	.10	.05	.23	.68	.12
Scared experience all the time (10)	.04	.01	.15	.71	.06
Sad experience powerful (17)	−.06	.08	.76	.07	−.01
Sad experience strong (2)	.06	−.04	.65	.09	−.03
Mad experience powerful (23)	.06	.01	.34	−.12	.65
bMad experience strong (8)	−.02	−.06	.41	−.11	.49
Scared experience powerful (5)	.11	−.04	.75	.14	−.22
Scared experience strong (20)	.09	.00	.79	.13	−.02
Happy control frequency (21)	.30	.45	.06	−.20	.02
Excited control frequency (9)	−.01	.60	.08	−.05	−.03
Happy control intensity (6)	.19	.45	−.01	.11	.11
Excited control intensity (24)	.08	.69	.09	−.12	.21
Sad control frequency (27)	−.04	.70	−.02	−.15	.04
Sad control intensity (12)	−.02	.62	−.12	.10	−.02
Mad control frequency (3)	−.05	.43	−.02	.09	−.13
Mad control intensity (18)	.02	.67	.02	.21	−.11
Scared control frequency (15)	−.12	.58	−.08	−.14	−.03
Scared control intensity (30)	−.11	.76	.00	−.02	−.03
Proportion of variance explained	.15	.12	.11	.07	.07

Note. Bolded values indicate items that loaded onto specific factors.

^aIndicates the item has no factor loading above .40.

^bCross-loadings above .40 on two factors.

Discussion

This study examined the psychometric properties of the HIF scale, a self-report measure of ER for children, in a clinical sample. Previous studies of the HIF, conducted in nonclinical school-based populations, revealed a three-factor structure—Positive Emotion, Negative Emotion, and Emotion Control (Ciucci et al., 2016; Walden et al., 2003). Our analyses, using a clinical sample of children from a large private psychotherapy practice, revealed a five-factor structure for the HIF. While the first and second factors found in our study are equivalent to the Positive Emotion and Emotion Control factors, our study, which used a clinical sample, revealed three separate factors related to Negative Emotion. One of these factors corresponds to the HIF items related to the intensity of sad and scared emotions, another corresponds to the frequency of sad and scared emotions, and the final corresponds to both the frequency and intensity of mad emotions. This finding suggests that a unitary construct for frequency and intensity of multiple negative emotions (i.e., sad, scared, and mad), as measured by HIF, may not be suitable for the clinical youth sample used in this study.

It is known that the psychometric properties of self-report measures may differ between clinical and nonclinical samples (Page et al., 2007). One of the defining features of our study’s clinical sample is that a large number of participants were diagnosed with ADHD. Many individuals with ADHD struggle with regulating emotions, which can lead to clinically meaningful functional impairments (Shaw et al., 2014). The factor structure found in the current study, which is different from previous studies with nonclinical samples, may largely be attributable to characteristics unique to ADHD.

The three-factor Negative Emotion structure may also be partially explained by how children differentiate emotions. Many children struggle to understand that more than one emotion can be felt at one time and do not report experiencing mixed emotions concurrently, especially with negative emotions (Nook et al., 2018). This concept may support the separation of negative emotions in our clinical sample. Furthermore, previous studies suggest that anger is more strongly associated with externalizing symptoms than other negative emotions (Kim et al., 2007). Hence, the high number of externalizing symptoms related to ADHD and the large number of children with this diagnosis in our study may help explain the particular structure of negative emotions such that “mad” is distinct from “sad” and “scared.”

It is also important to note that the demographic composition of our sample consisted of primarily White male children living in affluent, two-parent households. It is possible that the factor structure could be altered by these sample characteristics, and an important focus for future research on the psychometric properties of the HIF will be determining if the same five-factor structure is found in a more diverse clinical sample of children. Additionally, whereas prior studies used participants drawn from multiple clinics, the generalizability of our results is impacted by our reliance on a single clinic for recruitment. The stability and robustness of our results may also be impacted by the smaller sample size as compared to prior studies. Furthermore, given the generally low to moderate correspondence of child and caregiver ratings of psychological functioning (De Los Reyes et al., 2015), future work should examine the factor structure of caregiver ratings of child ER (e.g., BRIEF-2 Emotional Regulation Index; Gioia et al., 2015) and in particular determine the factor structure of negative emotions in such measures. Finally, given the time lapsed since the first psychometric studies of the HIF and the notable societal changes, (e.g., proliferation of cell phones and social media and the COVID-19 pandemic), an updated examination of this topic in nonclinical samples is warranted. The FF study of ER is an important aspect of child psychopathology, and valid tools designed to assess ER in children are essential. The HIF is a self-report measure of ER that assesses the controllability, frequency, and intensity of both positive and negative emotions. Our findings suggest that the factor structure of the HIF is substantially different when used to assess ER in children with psychological diagnoses as compared to neurotypical children. Therefore, the HIF may not be an optimal means for assessing the intensity and frequency of negative emotions in clinical samples.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Brendan A. Rich

References

Adrian

Zeman

Veits

(2011). Methodological implications of the affect revolution: A 35-year review of emotion regulation assessment in children. Journal of Experimental Child Psychology, 110(2), 171–197. https://doi.org/10.1016/J.JECP.2011.03.009

Bunford

Evans

S. W.

Langberg

J. M.

(2018). Emotion dysregulation is associated with social impairment among young adolescents with ADHD. Journal of Attention Disorders, 22(1), 66–82. https://doi.org/10.1177/1087054714527793

Ciucci

Baroncelli

Grazzani

Ornaghi

Caprin

(2016). Emotional arousal and regulation: Further evidence of the validity of the “how I Feel” questionnaire for use with school-age children. Journal of School Health, 86(3), 195–203. https://doi.org/10.1111/josh.12370

Cole

P. M.

Martin

S. E.

Dennis

T. A.

(2004). Emotion regulation as a scientific construct: Methodological challenges and directions for child development research. Child Development, 75(2), 317–333. https://doi.org/10.1111/j.1467-8624.2004.00673.x

Compas

B. E.

Jaser

S. S.

Bettis

A. H.

Watson

K. H.

Gruhn

M. A.

Dunbar

J. P.

Williams

Thigpen

J. C.

(2017). Coping, emotion regulation, and psychopathology in childhood and adolescence: A meta-analysis and narrative review. Psychological Bulletin, 143(9), 939–991. https://doi.org/10.1037/bul0000110

De Los Reyes

Augenstein

T. M.

Wang

Thomas

S. A.

Drabick

D. A. G.

Burgers

D. E.

Rabinowitz

(2015). The validity of the multi-informant approach to assessing child and adolescent mental health. Psychological Bulletin, 141(4), 858–900. https://doi.org/10.1037/a0038498

Eisenberg

Valiente

Fabes

R. A.

Smith

C. L.

Reiser

Shepard

S. A.

Losoya

S. H.

Guthrie

I. K.

Murphy

B. C.

Cumberland

A. J.

(2003). The relations of effortful control and ego control to children’s resiliency and social functioning. Developmental Psychology, 39(4), 761–776. https://doi.org/10.1037/0012-1649.39.4.761

Gioia

G. A.

Isquith

P. K.

Guy

S. C.

Kenworthy

(2015). Behavior rating inventory of executive Function^®, 2nd ed. PAR Inc.

Ivcevic

Brackett

(2014). Predicting school success: Comparing conscientiousness, grit, and emotion regulation ability. Journal of Research in Personality, 52, 29–36. https://doi.org/10.1016/J.JRP.2014.06.005

10.

Kim

Walden

Harris

Karrass

Catron

(2007). Positive emotion, negative emotion, and emotion control in the externalizing problems of school-aged children. Child Psychiatry and Human Development, 37(3), 221–239. https://doi.org/10.1007/s10578-006-0031-8

11.

Lai

(2018). Estimating standardized SEM parameters given nonnormal data and incorrect model: Methods and comparison. Structural Equation Modeling: A Multidisciplinary Journal, 25(4), 600–620. https://doi.org/10.1080/10705511.2017.1392248

12.

Nook

E. C.

Sasse

S. F.

Lambert

H. K.

McLaughlin

K. A.

Somerville

L. H.

(2018). The nonlinear development of emotion differentiation: Granular emotional experience is low in adolescence. Psychological Science, 29(8), 1346–1357. https://doi.org/10.1177/0956797618773357

13.

Page

A. C.

Hooke

G. R.

Morrison

D. L.

(2007). Psychometric properties of the Depression Anxiety Stress Scales (DASS) in depressed clinical samples. British Journal of Clinical Psychology, 46(Pt 3), 283–297. https://doi.org/10.1348/014466506X158996

14.

Rosseel

. (2012). lavaan: An R Package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02

15.

Solomon

R. C.

(2002). Back to basics: On the very idea of “basic emotions. Journal for the Theory of Social Behaviour, 32(2), 115–144. https://doi.org/10.1111/1468-5914.00180

16.

Walden

T. A.

Harris

V. S.

Catron

T. F.

(2003). How I Feel: A self-report measure of emotional arousal and regulation for children. Psychological Assessment, 15(3), 399–412. https://doi.org/10.1037/1040-3590.15.3.399

17.

Zeman

Klimes-Dougan

Cassano

Adrian

(2007). Measurement issues in emotion research with children and adolescents. Clinical Psychology: Science and Practice, 14(4), 377–401. https://doi.org/10.1111/j.1468-2850.2007.00098.x

18.

Zygmont

Smith

M. R.

(2014). Robust factor analysis in the presence of normality violations, missing data, and outliers: Empirical questions and possible solutions. The Quantitative Methods for Psychology, 10(1), 40–55. https://doi.org/10.20982/tqmp.10.1.p040