Multidimensional Assessment of Beliefs About Emotion

Lay Theories

Adherence to judgments about emotions is consistent with the idea that people hold “lay theories” or “implicit theories” about psychological constructs. Here the term implicit refers to assumptions about how constructs and people operate, which are typically poorly articulated (Dweck, 1986) and tend to operate outside conscious awareness. However, unlike in the realm of implicit associations (Schnabel, Asendorpf, & Greenwald, 2008), implicit or lay beliefs can be consciously accessible.

For example, Dweck’s work on implicit beliefs about intelligence has demonstrated that people tend to view intelligence as fixed or unchangeable (“entity” theorists) or as malleable and capable of growth with effort (“incremental” theorists; Dweck, 1986; Dweck & Leggett, 1988). Those who hold incremental beliefs tend to pursue learning goals and respond to failure with increased effort, supporting the notion that challenges can be overcome and intelligence can grow, whereas those who hold entity beliefs tend to pursue performance goals and quit tasks when the task becomes difficult (Dweck & Leggett, 1988). Beyond lay theories of intelligence, implicit beliefs have been investigated in a variety of domains, including shyness (Beer, 2002), personality (Chiu, Hong, & Dweck, 1997), morality (Chiu, Dweck, Tong, & Fu, 1997), interpersonal relationships (Knee, 1998), and self-regulation (Job, Dweck, & Walton, 2010). Importantly, implicit beliefs have been shown to be domain-specific, where it is feasible for one person to hold entity beliefs about intelligence but incremental beliefs about morality; implicit beliefs only influence outcomes in their associated domains.

Measurement of Beliefs About Emotion

Together, research and clinical wisdom suggest there is utility in identifying emotional beliefs that may predict emotion regulation processes and distress (e.g., negative affect, symptoms of depression, or anxiety). Tamir et al.’s (2007) brief malleability of emotion measure, while useful, does not fully capture the breadth of beliefs about emotion that people may hold. For example, experts suggest that beliefs that emotions are all-or-nothing phenomena and beliefs that emotions dictate behavior when present, may be judgments that contribute to deficits in emotion management (Leahy et al., 2011; Linehan, 1993a; Spradlin, 2003). In addition, the four-item measure used by Tamir et al. (2007) assesses a unidimensional construct. Their findings were consistent with those of Dweck and colleagues, where entity and incremental beliefs lie at each end of a single dimension of malleability beliefs (Dweck, 1999; Job et al., 2010). However, emotion researchers have suggested that humans may hold simultaneous beliefs that emotions are associated with both a loss of control and beliefs that emotions can be regulated (Gross & Muñoz, 1995; John & Gross, 2004). Additional research on emotion beliefs is necessary to determine if these beliefs are truly mutually exclusive, and whether additional specific beliefs about emotion likewise predict important emotion regulation outcomes.

Although a host of self-report measures assess constructs related to emotional beliefs, the extant measures of emotion regulation (Catanzaro & Mearns, 1990; Gratz & Roemer, 2004; Gross & John, 2003), emotional intelligence (Salovey, Mayer, Goldman, Turvey, & Palfai, 1995), and responses to emotions (Baer, Smith, Hopkins, Krieteymeyer, & Toney, 2006; Gámez, Chmielewski, Kotov, Ruggero, & Watson, 2011; Harmon-Jones, Harmon-Jones, Amodio, & Gable, 2011; Simons & Gaher, 2005; Taylor et al., 2007) assess self-efficacy in dealing with emotions or self-reported tendencies in emotional situations, but they do not assess erroneous beliefs about the emotions themselves. Essentially, the extant measures address one’s own perceived emotional control (Hofmann, 2005; White, Brown, Somers, & Barlow, 2006) rather than beliefs about the construct of emotion. For example, one item on the Difficulties with Emotion Regulation Scale (DERS; Gratz & Roemer, 2004) is “I experience my emotions as overwhelming and out of control.” Endorsement of this item suggests that an individual perceives his or her emotions to be uncontrollable, but does not assess whether this individual believes emotions are generally overwhelming for all people (Tamir & Mauss, 2011). Even items on the Beliefs About Emotions Scale (BAES; Rimes & Chalder, 2010), which theoretically measures beliefs about emotions, includes items such as “I should be able to cope with difficulties on my own without turning to others for support,” and “It would be a sign of weakness to show my emotions in public.” These items likewise address an individual’s perception of personal control over emotion rather than beliefs about the emotions themselves. A recent study addressed this distinction and found that both general (i.e., about people) and personal (i.e., beliefs about the self) malleability beliefs uniquely predicted lower use of cognitive reappraisal and greater levels of psychological distress (De Castella et al., 2013), although personal malleability beliefs may be more predictive of behavior than general malleability beliefs. Despite the evidence that personal beliefs and general beliefs are different constructs, assessment thus far has focused on malleability beliefs only, necessitating a more extensive assessment of general emotional beliefs than currently exists.

Implications for Mental Health

The majority of the small corpus of work beliefs of emotion malleability has addressed the effect of entity beliefs on emotion regulation outcomes and negative affect cross-sectionally (De Castella et al., 2013), longitudinally (Tamir et al., 2007) and in the lab (Kappes & Schikowski, 2013). Despite the clear implications of emotion beliefs on mental health, only one published study to date has evaluated the effect of emotion beliefs on psychiatric symptoms. Researchers found that beliefs that emotions are not malleable are associated with higher levels of depression and stress (De Castella et al., 2013). These findings are consistent with clinical advice to address “myths” about emotion in psychotherapy (Leahy et al., 2011), and consistent with anxiety research suggesting that perceived emotional control decreases symptoms of anxiety (Hofmann, 2005) and avoidance behaviors (White et al., 2006). Theoretically, individuals who hold negative beliefs about emotion should be less likely to engage in adaptive and flexible emotion regulation strategies, thus increasing symptoms and diagnoses of psychological disorders (De Castella et al., 2013).

Overview

The Emotion and Regulation Beliefs Scale (ERBS) was developed to assess beliefs about emotion more comprehensively than measured in prior studies (Tamir et al., 2007). Namely, we wished to broaden the assessment of implicit theories of emotion to explore beliefs beyond emotion malleability, with the intention of constructing a measure with enhanced psychometric characteristics. The scale construction and validation strategy roughly followed guidelines described by Clark and Watson (1995) and Reise, Waller, and Comrey (2000). The initial phase involved generation of an overinclusive item pool and administration to a diverse sample of community participants (N = 255) with the goal of establishing an initial factor structure. The second phase involved development of new items and administration to a separate community and college sample to finalize factor structure and provide initial validity evidence.

Study 1 Method

Study 1 Results

Prior to analyses, items were coded so that high scores represented a more adaptive view (e.g., emotions can be controlled). Item characteristics were examined and seven items with negative item–total correlations were removed. Factor structure of the remaining 46 items was examined via exploratory factor analysis (principal axis factoring [PAF]). Oblique (promax) rotation was used as we had no reason to believe that the factors would be uncorrelated, and the Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy was .88, suggesting the data were appropriate and sample size sufficient for factor analysis.

The initial solution produced 11 factors with eigenvalues greater than 1, explaining 46.6% of the variance; however, the scree plot indicated a two-factor solution, which was further validated by the finding that Factors 3 through 11 were all one- or two-item factors. After dropping cross-loading items and items that did not load highly (>.40), a PAF constrained to two factors resulted in an interpretable solution accounting for 40.71% of the variance. All 27 items loaded at .40 or higher on only one of the two factors. The first factor (α = .93) included items consistent with incremental emotion beliefs, whereas the second factor (α = .82) consisted of items assessing entity beliefs.

Preliminary scales assessing each factor were calculated using the mean score on the set of items for each factor. The two scales evidenced a significant, albeit weak, negative correlation (r = −.18, p < .05).

Study 1 Discussion

Although prior studies of implicit beliefs in emotion used a simple unidimensional measure (Kappes & Schikowski, 2013; Tamir et al., 2007), these initial scale analyses revealed two separate factors, only mildly correlated. However, the Factor 1 scale was highly negatively skewed, with a median of 4.19. This overendorsement is similarly reflected in Dweck’s measurement of implicit beliefs in intelligence (Hong et al., 1999) and is stated as the rationale for including only entity items when measuring implicit beliefs. We also wished to bolster the integrity of the scales by developing more items consistent with those remaining after initial factor analysis.

Study 2 Method

The second phase of data collection had four functions: (a) refinement of the item pool, (b) inclusion of a college student sample in addition to a separate community sample, (c) detailed examination of factor structure, and (d) further testing of convergent, discriminant, and construct validity.

Additional items were generated that were consistent with the 27 items from Phase 1. Rather than returning to the entire set of possible emotion beliefs, we explicitly added items that theoretically corresponded with the items remaining after initial factor analyses. Newly added items addressed specific emotions (e.g., “When a person feels really angry, it’s virtually impossible to not take the anger out on people or objects nearby”), opposite action (e.g., “It’s virtually impossible for people to act opposite to the way they feel”), and the notion that emotions “rule” people (e.g., “People are slaves to their emotions”). We intentionally included both incremental and entity-phrased items, as we did not want to assume that control and changeability beliefs are mutually exclusive (John & Gross, 2004). Instead, in an attempt to decrease overadherence to incremental items, we added several strongly worded items with absolute phrasing (e.g., “It is possible, with effort, to alter strong feelings in any situation”), such as are included on social desirability measures (e.g., Crowne & Marlow, 1960).

Study 2 Results

Measure Evaluation

After removing nonloading items in Study 1 and developing new items, Study 2 evaluated the revised 47-item measure in three stages. First, items were evaluated for overendorsement and redundancies. Seven items were removed for having means greater than 4.25 or less than 1.75. Six of these items were items representing incremental beliefs and one was an entity item. Intercorrelations among items were calculated with the intention of removing items that correlated at an absolute value of .60 with other items, but no items were removed based on this criteria.

Second, factor structure was evaluated using exploratory factor analysis (principal axis factoring using oblique rotation) in the student sample. KMO in the initial solution was .80, again supporting the sample size for factor analysis. Similar to Study 1, the initial solution produced 11 factors with eigenvalues more than 1, but a scree plot strongly suggested three factors (Floyd & Widaman, 1995). To be thorough, two-, three-, and four-factor solutions were investigated. The four-factor solutions had only two items in the fourth subscale, and the three-factor solution had a better fit than the two-factor solution, so the three-factor solution was retained. Sixteen items that did not load highly (>.40) on any of the three factors were removed (no items had cross-loadings >.4 on two scales). Together, the three factors accounted for 36.00% of the variability in scores and were highly interpretable.

Next, an initial confirmatory factor analysis [CFA] was conducted on approximately 50% of the mTurk sample, N = 309, using maximum likelihood estimation in Amos 20.0. Half of the sample was used so that factor structure could be revised following initial fit. This strategy thus represents an exploratory use of the CFA procedures. The initial model was not a good fit, χ² = 646.93, df = 249, χ²/df = 2.60, comparative fit index (CFI) = .83, root mean square error of approximation (RMSEA) = .07 (90% confidence interval [CI] = .065-.079). Residuals and modification indices were evaluated and three items were removed because of high covariance with the nonprimary factor. Moreover, error variances between similarly worded items were allowed to covary. These modifications resulted in adequate model fit, χ² = 357.00, df = 182, χ²/df = 1.96, CFI = .91, RMSEA = .06 (90% CI = .047-.064).

Finally, the final three-factor CFA model was evaluated for replicability using the other random half of the online sample (N = 267) which had acceptable model fit, χ² = 296.92 df = 182, χ²/df = 1.63, CFI = .93, RMSEA = .05 (90% CI = .04-.06). See Table 2 for the final scale items and regression weights from the final CFA model (using the second half of the online sample).

Factor 1 was labeled Emotional Constraint, representing the beliefs that emotions are forces that constrain or narrow an individual’s choices in an emotional situation (see Table 1 for items and factor loadings). Factor 2, labeled Regulation Worth, represents the belief that emotion regulation is both possible and worthwhile, and Factor 3, labeled Hijack, represents the belief that emotions hijack or usurp peoples’ self-control. At first blush, the Emotion Constraint beliefs and the Hijack beliefs appear similar, as both appear be composed of entity items. However, Hijack beliefs are associated with strong emotions and loss of self-control, whereas Emotion Constraint beliefs are associated with longevity of emotions and behavioral constraint consistent with the experienced emotion.

OPEN IN VIEWER

Table 1.

Final Scale Items, Item Descriptives and Regression Weights From Final Confirmatory Factor Analysis Model (N = 267).

Number	Item	Mean	SD	Factor 1 weight	Factor 2 weight	Factor 3 weight
11.	When people acknowledge their emotions, the emotions will completely take them over.	2.03	1.03	.70
14.	When a person feels really angry, it’s virtually impossible to not take the anger out on people or objects nearby.	1.98	1.13	.66
7.	When people are feeling down, they have to wait for a better mood to arrive before they can be productive.	2.44	1.19	.63
6.	When a person has a strong emotional reaction to another person, they will always feel that way about that other person.	2.06	1.08	.62
18.	When feelings of sadness take over, a person can’t really do anything but wallow in the misery.	1.82	0.96	.60
20.	It’s virtually impossible for people to act opposite to the way they feel.	2.03	0.99	.60
2.	Emotions can either be expressed entirely or hidden from others—it isn’t possible to share only part of an emotional response.	2.44	1.21	.59
10.	When an emotion comes along, it will continue unless there is a change in the environment.	2.61	1.07	.59
1.	Emotions operate like a floodgate that is either open or closed. In other words, emotions are either “on” or “off.”	2.59	1.14	.56
12.	Learning how to alter strong emotions is a worthwhile pursuit.	4.02	0.82		.76
19.	People benefit from learning how to regulate their feelings.	4.24	0.73		.71
8.	People would be better off if they took time to figure out where their emotions come from.	4.16	0.71		.65
5.	Putting forth effort to alter emotional experience is valuable.	4.05	0.71		.62
16.	People would be better off if they spent more time learning how to control their emotions.	3.91	0.88		.61
13.	It is possible, with effort, to alter strong feelings in any situation.	3.88	0.85		.51
3.	People can learn to control/regulate their emotions.	4.18	0.71		.43
9.	When strong emotions are present, they dictate what a person says or does.	3.07	1.14			.67
15.	People are slaves to their emotions.	2.12	1.06			.64
4.	People are ruled by their emotions.	2.85	1.02			.63
17.	Strong emotions will make people do things they wouldn’t normally do.	3.86	1.04			.41
21.	Emotions make people lose control.	3.46	1.13			.41

Correlations between each subscale and means for the subscales in both the online and college samples are reported in Table 2. Although the magnitudes of the correlation coefficients are slightly different for each sample, the general scope of relations is the same. Emotion Constraint beliefs were positively associated with Hijack beliefs, and Emotion Constraint beliefs were negatively associated with Regulation Worth beliefs. There was no significant association between Regulation Worth beliefs and Hijack beliefs. When the subscales were compared by sample, we found that the online sample held lower Emotion Constraint and Hijack beliefs, and higher beliefs in Regulation Worth compared with the college sample.

OPEN IN VIEWER

Table 2.

Correlations Among Subscales of the ERBS.

					M (SD)
		1.	2.	3.	College sample	Online Sample	t test
1. Emotion Constraint		—	−.29**	.51**	2.34 (0.69)	2.20 (0.70)	−2.74*
2. Regulation Worth		−.15*	—	−.05	3.94 (0.56)	4.06 (0.56)	2.90**
3. Hijack		.42**	.03	—	3.22 (.069)	3.05 (0.76)	−3.17*

Note. ERBS = Emotion and Regulation Beliefs Scale. College student correlations below diagonal, mTurk above diagonal.

*

p < .05. **p < .01.

Reliability

The internal consistency, evaluated using Cronbach’s alpha and AICs (average interitem correlations) of the three subscales can be found in Table 3, presented separately for the student and community sample, along with means and standard deviations for subscale scores. The alphas are all acceptable (>.7), slightly lower for the five-item Hijack subscale compared with the longer subscales, and AICs range from .30 to 34.

OPEN IN VIEWER

Table 3.

Internal Consistency of ERBS by Sample.

	α		AIC
	College sample (N = 281)	Online sample (N = 576)	College sample	Online sample
Emotion Constraint	.79	.83	.30	.36
Regulation Worth	.78	.82	.32	.40
Hijack	.70	.74	.30	.36

Note. ERBS = Emotion and Regulation Beliefs Scale; AIC = average interitem correlation.

Validity

Convergent and Discriminant Validity

Associations between the ERBS subscales, other measures of implicit beliefs and measures of emotion regulation tendencies are displayed in Table 4. We expected to see positive relationships between Regulation Worth, incremental beliefs in emotion (ITES), cognitive reappraisal, and facets of emotional intelligence as measured by the TMMS. As expected, the ERBS scales significantly correlated with the ITES, where strongest correlation was with the Regulation Worth Scale (r = .52). We also expected significant domain specificity, and as predicted, we saw low correlations (<.15 in magnitude) between the three ERBS scales and implicit beliefs in intelligence. Further demonstrating domain specificity, none of the ERBS scales nor the ITES correlated significantly with the BAES, suggesting that these constructs are indeed distinct.

OPEN IN VIEWER

Table 4.

Bivariate Correlations of ERBS Subscales and Measures Assessing Convergent and Discriminant Validity.

Scale	1.	2.	3.	4.	5.	6.	7.	8.	9.	10.	11.
1. Emotion Constraint	.82
2. Regulation Worth	−.25**	.80
3. Hijack	.49**	−.03	.73
4. ITES	−.35**	.51**	−.23**	.78
5. BAES a	.06	.10	.07	−.00	.88
6. Implicit Intelligence	−.12**	.08*	−.07*	.19**	−.13*	.93
7. ERQ: Reappraisal	−.16**	.38**	−.08*	.38**	−.16*	.11*	.87
8. ERQ: Suppression	.26**	−.03	.07	−.06	.33**	−.11*	−.03	.74
9. TMMS: Attention	−.24**	.09*	.01	.07	−.22**	.12*	.15*	−.46**	.83
10. TMMS: Clarity	−.32**	.21**	−.20**	.29**	−.31**	.11*	.33**	−.26**	.32**	.86
11. TMMS: Repair	−.21**	.23**	−.16**	.33**	−.21**	.15**	.56**	−.19**	.25**	.45**	.88

Note. Cronbach’s alpha reported in italics along the diagonal. ERQ = Emotion Regulation Questionnaire; ITES = Implicit Theories of Emotion Scale; TMMS = Trait Meta-Mood Scale; BAES = Beliefs About Emotion Scale; ERBS = Emotion and Regulation Beliefs Scale.

a.

Data from student sample only (N = 281).

*

p < .05. **p < .01.

Of note, only the Emotion Constraint beliefs subscale predicted use of expressive suppression as an emotion regulation strategy, whereas all three subscales were differentially associated with reappraisal, with higher Regulation Worth beliefs particularly associated with greater use of cognitive reappraisal. Emotion Constraint beliefs were associated with lower attention to emotions, lower clarity of emotions, and lower self-efficacy in emotional repair.

We also adopted a hierarchical regression approach in testing the incremental validity of the ERBS over the four-item ITES by placing the ITES in Step 1 and the three ERBS subscales in Step 2 in predicting emotion regulation outcomes (ERQ and TMMS subscales). This strategy assessed the unique predictive power of each ERBS subscale after controlling for variability in implicit beliefs of emotion malleability. Results are presented in Table 5. The ITES predicted both cognitive reappraisal and suppression, as well as emotional clarity and mood repair. Importantly, the ERBS subscales predicted all five emotion regulation outcomes above and beyond the ITES. Regulation Worth significantly predicted cognitive reappraisal and emotion repair, consistent with the idea that people who value emotion regulation are more likely to engage in adaptive regulation processes. Emotion Constraint beliefs predicted greater use of expressive suppression and lower emotional clarity. Hijack and Emotion Constraint beliefs differentially predicted attention to emotion such that higher Hijack beliefs were associated with greater attention to emotions, whereas higher Emotion Constraint beliefs were associated with less attention to emotions. These results indicate that despite moderately high correlations, the two entity subscales are differentially associated with salient emotional outcomes.

OPEN IN VIEWER

Table 5.

Hierarchical Regressions With ERBS Predicting Emotion Regulation Outcomes After Controlling for the Four-Item ITES.

	Step 1		Step 2
	R 2	Four-item β	ΔR2	Emotion Constraint β	Regulation Worth β	Hijack β
ERQ: Reappraisal	.14**	.38**	.05**	.01	.26**	−.02
ERQ: Suppression	.01	−.06	.07**	.31**	.05	−.09*
TMMS: Attention	.00	.07	.07**	−.31**	.03	.14**
TMMS: Clarity	.08**	.29**	.06**	−.22**	.06	−.06
TMMS: Repair	.11**	.33**	.02**	−.07	.08*	−.06

Note. ERBS = Emotion and Regulation Beliefs Scale; ITES = Implicit Theories of Emotion Scale; TMMS = Trait Meta-Mood Scale.

*

p < .05. **p < .01.

Clinical Utility

We wished to investigate the potential clinical utility of the ERBS by assessing the association between the ERBS scales and important outcomes known to be associated with emotion regulation (depression, anxiety, binge eating, and alcohol use). Due to the high prevalence of these outcomes in college students as well as the regularity in which they are assessed in college student populations, the overall sample was divided into college students (N = 270, including the entire subject pool sample and the college student–identified Mechanical Turk participants) and noncollege students (N = 461). Independent t tests confirmed no differences in binge eating between college students (M = 8.69, SD = 7.74) and noncollege participants (M = 8.00, SD = 8.23), t(729) = 2.42, p < .05. Depression scores were marginally higher in the college students (M = 6.79, SD = 7.44) compared with noncollege participants (M = 5.80, SD = 7.78), t(729) = 1.94, p = .05. There were no differences in anxiety scores (t < 1). After excluding individuals who indicated they were nondrinkers (e.g., had not consumed alcohol in the past year), college students also reported higher hazardous drinking (M = 7.09, SD = 5.06) than noncollege participants (M = 4.72, SD = 4.55), t(563) = 3.79, p < .001.

Correlations between ERBS subscales and clinically relevant outcomes are presented separately for college students and noncollege participants (see Table 6). Emotion Constraint and Hijack beliefs were associated with increased depression, anxiety, and binge eating in both samples. In the noncollege participants, higher Regulation Worth was associated with less depression, and Hijack beliefs were associated with greater alcohol use.

OPEN IN VIEWER

Table 6.

Correlations Among ERBS Subscales and Clinical Outcome Variables of Depression, Anxiety, Binge Eating and Alcohol Use Among College Students (N = 270) and Noncollege Students (N = 461).

	Emotion Constraint		Regulation Worth		Hijack
Scale	College	Noncollege	College	Noncollege	College	Noncollege
Depression	.16*	.28**	−.04	−.13**	.22**	.18**
Anxiety	.15**	.15**	.01	−.09	.24**	.19**
Binge Eating	.11 †	.12*	−.02	−.06	.14*	.21*
Alcohol a	.07	.05	−.08	.06	.08	.14**

Note. ERBS = Emotion and Regulation Beliefs Scale.

a.

Results restricted to individuals who reported drinking within the past year; college students (N = 306) and noncollege students (N = 351).

†

p < .07. *p < .05. **p < .01.

In addition to correlations, we also performed a series of multiple regression analyses regressing each of the four clinically relevant outcomes on the ERBS subscales after controlling for college student status (Step 1) and ITES scores (Step 2). Results indicated that after controlling for college student status, ITES predicted increase alcohol use ΔR² = .01, p < .05, and the ERBS scales did not account for additional variance, ΔR² = .01, ns. However, higher incremental beliefs on the ITES were associated with less binge eating, (ΔR² = .02, β = −.13, p < .001). In addition, the ERBS scales predicted an additional 3% of the variance in binge eating, where only Hijack beliefs were uniquely associated with greater binge eating (β = .15, p < .001).

Higher incremental beliefs on the ITES predicted lower depression, ΔR² = .03, β = −.18, p < .001, and the set of ERBS subscales predicted an additional 4% of the variability in depressive symptoms. Specifically, higher Emotion Constraint beliefs (β = .14, p < .001) and higher Hijack beliefs (β = .11, p < .05) also predicted increased depression, although Regulation Worth did not. A similar pattern emerged for anxiety symptoms, with the ITES predicting 6% of the variability in anxiety (β = −.24, p < .001), and the ERBS accounting for an additional 3% after controlling for incremental beliefs. Hijack was the only significant unique ERBS predictor of anxiety (β = .15, p < .001).

Study 2 Discussion

Correlations of the ERBS with emotion regulation and clinically relevant outcomes revealed that Emotion Constraint beliefs were associated with the tendency to engage in expressive suppression, as well as the tendency to pay less attention to emotions and engage in less emotional repair efforts. These findings are notable because Tamir et al. (2007) did not find a relationship between incremental beliefs and suppression; it appears that more extensive assessment of emotional beliefs adds to the prediction of emotion regulation strategies. Specifically, beliefs that emotions constrain or restrict behavior are associated with the experience of facial restriction. Future research may investigate the mechanisms and direction of the relationship, as it may be that use of expressive suppression leads to increased Emotional Constraint beliefs, as expressive suppression is a way of constraining emotional responses.

On the other hand, Regulation Worth beliefs were associated with use of cognitive reappraisal, consistent with past research (Tamir et al., 2007), as well as increased clarity of emotions and attention to mood repair. Beliefs that emotions can usurp or hijack self-control appear to be important in predicting symptoms of psychopathology, especially binge eating and anxiety. Moreover, it is notable that the correlations between emotion beliefs and outcomes were not identical for college and noncollege participants. For example, Regulation Worth beliefs were not predictive of symptoms for college students, and Hijack beliefs predicted alcohol use only for noncollege participants. Further research may wish to examine the mechanisms by which these beliefs predict behavior, as well as how the beliefs may differ across the lifespan.

We also found that the three ERBS subscales are significantly related to each other, but are conceptually distinct. The strongest relationship among the three scales was between the Emotion Constraint scale and the Hijack acale, which is logical as both are broadly representative of more entity beliefs in emotion. However, we found unique predictions for each scale, such that when in the same model predicting attention to emotions, the Hijack acale positively predicted attention to emotions, whereas the Emotion Constraint scale negatively predicted attention to emotions. These results suggest that perhaps Hijack beliefs are more associated with overwhelming emotions which are difficult to ignore, whereas Emotion Constraint beliefs are associating with emotional distancing. These predictions may be worth examining in future studies using the ERBS.

Moreover, the ERBS is not redundant with the four-item ITES; the ERBS significantly predicted emotion regulation tendencies and symptoms of psychopathology after controlling for the ITES. In addition, the ERBS was not significantly related to one included measure of personal beliefs about emotion (BAES). It would be useful to compare the ERBS with additional constructs measuring personal beliefs about emotional management (i.e., perceived emotional control), such as difficulties with emotion regulation (Gratz & Roemer, 2004), or distress tolerance (Simons & Gaher, 2005). Moreover, although the ERBS scales have clinical implications, it remains unclear if the ERBS is useful in discriminating clinical from nonclinical samples. These questions are addressed in Study 3.

Study 3 Method

We wished to compare the ERBS scales with other measures theoretically related to emotion beliefs, in a continued attempt to establish discriminant validity. We also hoped to establish further clinical utility of the measure by demonstrating that the ERBS scales predict clinical samples above and beyond extant measures of emotion regulation. Research suggests that individuals who suffer from depression and eating disorders evidence difficulties with emotion regulation (Aldao, Nolen-Hoeksema, & Schweizer, 2010). Thus, we focused on members of these two groups during the present investigation.

Measures

Emotion and Regulation Beliefs Scale

Same as in Study 2. See Table 7 for alpha values

OPEN IN VIEWER

Table 7.

Bivariate Correlations of ERBS Subscales and Emotion Regulation Measures.

Scale	1.	2.	3.	4.	5.	6.
1. Emotion Constraint	.84
2. Regulation Worth	−.22**	.81
3. Hijack	.64**	−.06	.75
4. ITES	−.39**	.51**	−.29**	.84
5. DERS	.15**	−.06	.10*	−.10*	.96
6. DTS	−.37**	.07	−.30**	.18**	–.09*	.94

Note. Cronbach’s alpha reported in italics along the diagonal. ERBS = Emotion and Regulation Beliefs Scale; ITES = Implicit Theories of Emotion Scale; DERS = Difficulties with Emotion Regulation Scale; DTS = Distress Tolerance Scale.

*

p < .05. **p < .01.

Implicit Theories of Emotion Scale

Same as in Study 2.

Difficulties with Emotion Regulation Scale

The Difficulties with Emotion Regulation Scale (DERS; Gratz & Roemer, 2004) is a 36-item self-report measure assessing dimensions associated with emotion dysregulation, including lack of acceptance of emotional responses, difficulties engaging in goal-directed behavior, limited access to emotion regulation strategies, lack of emotional awareness, impulsivity, and lack of emotional clarity. Items are given on a 5-point Likert-type scale from 1 (almost never) to 5 (almost always). Responses are scored such that higher ratings indicate increased problems with emotion regulation. In the current study, total DERS score were used.

Distress Tolerance Scale

The Distress Tolerance Scale (DTS; Simons & Gaher, 2005) is a self-report measure assessing self-perceived distress tolerance or the ability to withstand negative emotional states. The measure is given on a 1 (strongly agree) to 5 (strongly disagree) Likert-type scale where high scores indicate a greater tolerance of distressing affective states. The DTS is one of the most popular measures of affect tolerance (Leyro, Zvolensky, & Bernstein, 2010), and has excellent reliability and validity (Simons & Gaher, 2005).

Depression, Anxiety and Stress Scales–21-Item Version

Same as in Study 2. Cutoff points in published norms (Lovibond & Lovibond, 1995), suggestion individuals with a score of 22 of above correspond to “severe” levels of depression.

Eating Disorder Diagnostic Scale

The Eating Disorder Diagnostic Scale (EDDS; Stice, Telch, & Rizvi, 2000) is a 22-item self-report measure assessing specific eating disorder diagnoses at both threshold and subthreshold levels, according to DSM-IV diagnostic criteria, including provisional diagnoses of binge eating disorder. Items assess body image and weight perceptions, binge eating, compensatory (i.e., purging) behaviors, loss of self-control related to eating, and restriction of food intake, as well as body mass index. The scale has shown excellent validity in discriminating among eating disorder groups and adequate temporal reliability (Stice et al., 2000).

Study 3 Results

First, zero-order correlations were conducted between the ERBS and the emotion regulation variables (Table 7). As in Study 2, Emotion Constraint and Hijack were positively associated, Regulation Worth and Emotion Constraint were negatively associated, and there was no significant relationship between Hijack and Regulation Worth. The pattern of relationships between the ERBS and the ITES was also essentially the same as in Study 2. We also found that the DERS was significantly, albeit weakly, associated with Emotion Constraint and Hijack, suggesting that individuals who believe that emotions constrain behavior and are associated with a loss of self-control have greater emotion regulation difficulties. Finally, the ERBS scales were associated with distress tolerance whereby higher Emotion Constraint and Hijack beliefs corresponded with lower self-perceived abilities to tolerate negative states.

Second, approximations of clinical samples were calculated from the DASS and the EDDS. Participants were classified as depressed (N = 61) or nondepressed (N = 493) based on a cutoff score of 22 (“severe”) on the DASS (Lovibond & Lovibond, 1995). Participants were likewise classified as “eating disordered” if they met at least subthreshold criteria for any eating disorder on the EDDS (N = 125). These were primarily individuals meeting criteria for bulimia nervosa (n = 86), with a smaller sample meeting criteria for binge eating disorder (n = 37), and only a few subthreshold for anorexia (n = 2).

We compared the emotion belief and regulation measures between clinical and nonclinical samples using independent samples t tests (Table 8). The depressed groups was different from the nondepressed group on all variables except Regulation Worth; depressed people had higher Emotion Constraint and Hijack beliefs and greater problems with emotion regulation, lower incremental beliefs, and lower distress tolerance. A similar pattern emerged with the eating disorder groups.

OPEN IN VIEWER

Table 8.

Independent Samples t Tests Examining Emotion Regulation and Emotion Beliefs Scores by Clinical Group.

	Depressed	Nondepressed	t test	Eating disordered	Noneating disordered	t test
Emotion Constraint	2.97 (.66)	2.28 (.74)	6.99**	2.63 (.78)	2.27 (.74)	4.72**
Regulation Worth	3.79 (.67)	3.92 (.62)	−1.47*	3.91 (.59)	3.90 (.64)	.09
Hijack	3.47 (.64)	3.03 (.78)	4.22**	3.30 (.74)	3.01 (.78)	3.68**
ITES	3.36 (.80)	3.66 (.80)	−2.75**	3.47 (.90)	3.66 (.78)	−2.40*
DERS	111.20 (21.53)	78.52 (25.58)	9.56*	94.18 (26.71)	78.60 (26.29)	5.81**
DTS	2.80 (.97)	3.32 (.94)	4.03**	3.08 (.91)	3.31 (.97)	−2.36*

Note. ITES = Implicit Theories of Emotion Scale; DERS = Difficulties with Emotion Regulation Scale; DTS = Distress Tolerance Scale.

†

p < .07. *p < .05. **p < .01.

Finally, as a fairly stringent test of incremental validity, we performed two multinomial logistic regressions predicting disorder classification from the set of belief and regulation variables; all three of the ERBS scales, the ITES, the DERS, and the DTS were entered as predictors. The model predicting depression classification fit the data well, R² = .16 (Cox & Snell), χ²(6) = 93.58, p < .001. With all the variables in the model, only the DERS—B = .05, SE = 0.1, Wald χ²(1) = 36.41, p < .001—and Emotion Constraint—B = .91, SE = 0.28, Wald χ²(1) = 10.46, p < .01—predicted depression status. A similar result emerged for predicting eating disorder status, where the DERS—B = .02, SE = 0.1, Wald χ²(1) = 13.28, p < .001—and Emotion Constraint—B = .37, SE = 0.19, Wald χ²(1) = 3.70, p = .05—predicted eating disorder diagnostic status.

Study 3 Discussion

The pattern of zero-order correlations was consistent with our earlier findings (Study 2) and we also demonstrated low-to-moderate relationships between the ERBS scales and the indices of emotion dysregulation, further validating the notion that the ERBS scales represent distinct constructs. We also found that negative beliefs about emotion (e.g., Emotion Constraint and Hijack) are higher in individuals who would meet criteria for an eating disorder and likely would meet criteria for depression. Finally, in a strict evaluation of the incremental validity of the ERBS, we found that Emotion Constraint predicts clinical groups uniquely, in addition to emotion regulation difficulties, even when the ITES and distress tolerance were also in the model. Further work will be needed to identify the domains in which Regulation Worth and Hijack may likewise contribute to psychopathology and emotion regulation difficulties.

Overall Discussion

The development of the ERBS contributes to the literature on implicit self-theories (Dweck, 1986) as well as the growing literature on emotion regulation. Our results provide preliminary evidence for three separate beliefs about emotion: (a) the belief that emotions can “hijack” self-control, (b) the belief that emotions can constrain behavior, and (c) the belief that emotion regulation, even if effortful, is a worthwhile pursuit. The findings suggest the importance of measuring multiple dimensions of emotion beliefs rather than focusing only on implicit beliefs of emotion malleability (Kappes & Schikowski, 2013; Tamir et al., 2007), and are consistent with other domains that have identified several implicit beliefs (e.g., Marziano, Ward, Beech, & Pattison, 2006) as well as with clinical wisdom regarding the utility of challenging “myths” about emotion during treatment (Leahy et al., 2011; Linehan, 1993b; Spradlin, 2003).

In their review of social cognitive factors in emotion regulation, Tamir and Mauss (2011) title the section about malleability of emotion as “Can emotions be controlled and can I control my emotions?” This heading suggests that these are truly two separate questions, the controllability of an attribute versus personal abilities to control an attribute (i.e., self-efficacy). Thus far, the majority of research on emotion malleability, controllability, and regulation has fallen into the personal attribute camp. The convergent and discriminant validity results from the ERBS suggest that beliefs about emotion are distinguishable from beliefs about self-efficacy in emotion regulation, including beliefs about the unacceptability of emotion and emotion expression (i.e., the BAES) and self-perceived difficulties in emotion regulation (the DERS). These results indicate that beliefs about the construct of emotion are distinct from beliefs about personal experiences with emotion and the value of perceived emotional control.

The current study, with three separate scales, counters the unidimensional ITES scale by supporting the notion of multiple and potentially contradictory emotional beliefs (John & Gross, 2004; Kennedy-Moore & Watson, 1999). Indeed, recent evidence suggests that individuals use multiple emotion regulation strategies (Aldao & Nolen-Hoeksema, 2012) rather than assuming that people have sole “go-to” regulation processes, as is suggested by trait self-report measures (Gross & John, 2003). In this work, we have found that the set of beliefs assessed by the ERBS allows people to hold multiple beliefs. Moreover, although the Emotion Constraint and Hijack scales are conceptually similar, both relatively consistent with an entity perspective, they are not redundant, as they have exhibited opposite relationships to some constructs (e.g., attention to emotions) and differentially predict outcomes. Our conjecture is that Emotion Constraint beliefs are associated with fear of emotions, whereas Hijack beliefs are used as justifications for engaging in risk behaviors when emotional (e.g., emotional urgency). It may be that these beliefs differentiate those with emotional undercontrol (e.g., borderline personality disorder, who may be likely to endorse Hijack beliefs), from those with emotional overcontrol (e.g., obsessive–compulsive personality disorder), although these predictions will require empirical examination before any definitive assertions can be made.

It may be that situational characteristics activate different beliefs and contribute to use of varying emotion regulation strategies across situations, consistent with if . . . then models of behavior (Cervone, 2004; Mischel & Shoda, 1995). For example, it may be that situations that evoke strong feelings of anger (e.g., jealousy situations) activate Hijack beliefs, which contribute to lack of inhibition (e.g., punching the wall). Future research will need to test the idea that these beliefs can be differentially activated based on contextual factors. Similarly, future work will also want to address the malleability of the beliefs themselves. In the work on implicit beliefs in intelligence, priming incremental beliefs resulted in greater effort on challenging tasks, just as priming entity beliefs resulted in the tendency to quit sooner (Dweck & Leggett, 1988). No published work to date has examined whether emotion beliefs shift in response to treatment; we would expect to see lower Emotion Constraint and Hijack beliefs following a successful course of treatment. It may also be worth examining the Regulation Worth subscale as an indicator of treatment pursuit or adherence; people who do not believe that emotion regulation is worth the effort may be less likely to seek treatment or commit to treatment compared with those with high Regulation Worth beliefs. These are questions ripe for future investigation.

Interpretation of the current work requires attention to methodological limitations. The results are self-report, and it is thus unknown how the ERBS beliefs predict momentary emotional responses or dynamic regulation processes. People vary in their accessibility of emotions (Gratz & Romer, 2004) and thus it will be important to validate the effect of emotion beliefs on in vivo emotional responding in future studies. Also, although the initial scale attempted to broadly address implicit beliefs in emotion, we do not claim that the beliefs here fully address the common judgments people hold about emotion. For example, some people also believe that emotions should be “released” (e.g., that holding on to emotions is damaging; Kennedy-Moore & Watson, 1999). Indeed, the myth that anger, in particular, must be expressed, has been identified (and debunked) as one of the 50 Great Myths of Popular Psychology (Lilienfeld, Lynn, Ruscio, & Beyerstein, 2010). Others hold the opposite myth, that expressions of emotion are signs of weakness, and nonemotionality indicates strength (Kennedy-Moore & Watson, 1999). Additional beliefs about emotion may include further assessment of the transience of emotions (e.g., emotions are fleeting or long-lasting), the viability of mixed emotions (e.g., if it is possible to experience both positive and negative emotions at the same time). Items representing these additional beliefs were included in our initial item pool, but they were discarded following factor analysis, suggesting that there were insufficient items to fully capture these alternative beliefs.

Additionally, the studies presented here used both community and college student samples; however, the participants were still predominantly White women. Moreover, even though prior research has demonstrated the viability of Mechanical Turk as a data collection strategy and that the data collected using Mechanical Turk workers is at least of equal quality than typical college student participants (Buhrmester et al., 2011; Shapiro, Chandler, & Mueller, 2013), the extent to which these beliefs generalize beyond these samples are heretofore unknown. Replication of the results with other samples is necessary to ensure utility of the ERBS, including a variety of diverse nonclinical and clinical populations, particularly other groups prone to emotion dysregulation difficulties (e.g., substance abusers, personality disorders).

Strengths of the current study include sampling of both community and college student participants, inclusion of clinical subgroups, and a carefully stepped process of scale development and examination of factor structure. Future research may also wish to examine the relationship of the ERBS to other relevant measures, such as experiential avoidance (Gámez et al., 2011; Kashdan et al., 2006), or anxiety sensitivity (Taylor et al., 2007). As discussed above, whether the ERBS predicts momentary emotional fluctuation or in vivo emotion regulation processes also remains to be tested. Similarly, for the ERBS to be a clinically useful instrument, the ERBS should be examined as a predictor of treatment attrition and treatment outcomes. As an initial attempt to measure multiple myths and beliefs about emotion, the development of the ERBS paves the way for future research on implicit beliefs about emotion from a multidimensional perspective. It appears that in addition to beliefs about emotion malleability, people hold beliefs about other aspects of emotion that preliminarily show promise as predictors of emotion regulation processes and a range of symptom presentations.