Multidimensional Assessment of Beliefs About Emotion

Abstract

Recent work has extended the idea of implicit self-theories to the realm of emotion to assess beliefs in the malleability of emotions. The current article expanded on prior measurement of emotion beliefs in a scale development project. Items were tested and revised over rounds of data collection with both students and nonstudent adult online participants. Exploratory and confirmatory factor analyses revealed a three-factor structure. The resulting scale, the Emotion and Regulation Beliefs Scale, assesses beliefs that emotions can hijack self-control, beliefs that emotion regulation is a worthwhile pursuit, and beliefs that emotions can constrain behavior. Preliminary findings suggest that the Emotion and Regulation Beliefs Scale has good internal consistency, is conceptually distinct from measures assessing individuals’ beliefs in their management of emotions and facets of emotional intelligence, and predicts clinically relevant outcomes even after controlling for an existing short measure of beliefs in emotion controllability.

Keywords

emotion emotion regulation beliefs

Emotional disturbance is at the heart of many forms of psychopathology (Dillon, Deveney, & Pizzagali, 2011; Kring, 2008; Mennin, Heimberg, Turk, & Fresco, 2005), such that deficits in emotional processing and emotion regulation cut across diagnostic categories (Gross & Muñoz, 1995; Kashdan, Barrios, Forsyth, & Steger, 2006). Treatments with a central emphasis on emotion have becoming increasingly popular over the past decade, including Dialectical Behavior Therapy (Linehan, 1993a), and the Unified Protocol (Barlow et al., 2010; Ellard, Fairholme, Boisseau, Farchoine, & Barlow, 2010). Many of these treatments include psychoeducation about emotion, including teaching clients to question judgments about emotions because they tend to perpetuate maladaptive behaviors such as emotional avoidance (Leahy, Tirch, & Napolitano, 2011; Linehan, 1993b; Spradlin, 2003).

Lay Theories

Adherence to judgments about emotions is consistent with the idea that people hold “lay theories” or “implicit theories” about psychological constructs. Here the term implicit refers to assumptions about how constructs and people operate, which are typically poorly articulated (Dweck, 1986) and tend to operate outside conscious awareness. However, unlike in the realm of implicit associations (Schnabel, Asendorpf, & Greenwald, 2008), implicit or lay beliefs can be consciously accessible.

For example, Dweck’s work on implicit beliefs about intelligence has demonstrated that people tend to view intelligence as fixed or unchangeable (“entity” theorists) or as malleable and capable of growth with effort (“incremental” theorists; Dweck, 1986; Dweck & Leggett, 1988). Those who hold incremental beliefs tend to pursue learning goals and respond to failure with increased effort, supporting the notion that challenges can be overcome and intelligence can grow, whereas those who hold entity beliefs tend to pursue performance goals and quit tasks when the task becomes difficult (Dweck & Leggett, 1988). Beyond lay theories of intelligence, implicit beliefs have been investigated in a variety of domains, including shyness (Beer, 2002), personality (Chiu, Hong, & Dweck, 1997), morality (Chiu, Dweck, Tong, & Fu, 1997), interpersonal relationships (Knee, 1998), and self-regulation (Job, Dweck, & Walton, 2010). Importantly, implicit beliefs have been shown to be domain-specific, where it is feasible for one person to hold entity beliefs about intelligence but incremental beliefs about morality; implicit beliefs only influence outcomes in their associated domains.

Implicit Theories of Emotion

The concept of implicit beliefs has recently been extended to beliefs about emotion. Initial research suggests that beliefs about the malleability of emotion influence emotion regulation and the experience of positive and negative affect (De Castella et al., 2013; Kappes & Schikowski, 2013; Labroo & Mukhopadhyay, 2009; Tamir, John, Srivastava, & Gross, 2007). Modeled after Dweck’s research (Dweck, 1999), Tamir et al. (2007) used a brief, four-item unidimensional measure to assess beliefs in the malleable nature of emotion such that higher scores represented stronger beliefs that emotions can be controlled (i.e., incremental beliefs). They found that higher incremental beliefs were associated with greater habitual use of cognitive reappraisal and greater self-efficacy in emotion regulation. These findings were replicated in a study that linked higher incremental beliefs with cognitive reappraisal, self-esteem, and life satisfaction, as well as lower levels of depression and stress (De Castella et al., 2013). Longitudinally, those who held incremental beliefs experienced more positive and less negative affect over the course of an academic school year compared with people with entity beliefs (Tamir et al., 2007). Finally, entity beliefs were associated with greater negative affect in response to viewing negative film clips in a laboratory, as well as greater self-reported avoidance of affective components of the film (Kappes & Schikowski, 2013).

Measurement of Beliefs About Emotion

Together, research and clinical wisdom suggest there is utility in identifying emotional beliefs that may predict emotion regulation processes and distress (e.g., negative affect, symptoms of depression, or anxiety). Tamir et al.’s (2007) brief malleability of emotion measure, while useful, does not fully capture the breadth of beliefs about emotion that people may hold. For example, experts suggest that beliefs that emotions are all-or-nothing phenomena and beliefs that emotions dictate behavior when present, may be judgments that contribute to deficits in emotion management (Leahy et al., 2011; Linehan, 1993a; Spradlin, 2003). In addition, the four-item measure used by Tamir et al. (2007) assesses a unidimensional construct. Their findings were consistent with those of Dweck and colleagues, where entity and incremental beliefs lie at each end of a single dimension of malleability beliefs (Dweck, 1999; Job et al., 2010). However, emotion researchers have suggested that humans may hold simultaneous beliefs that emotions are associated with both a loss of control and beliefs that emotions can be regulated (Gross & Muñoz, 1995; John & Gross, 2004). Additional research on emotion beliefs is necessary to determine if these beliefs are truly mutually exclusive, and whether additional specific beliefs about emotion likewise predict important emotion regulation outcomes.

Although a host of self-report measures assess constructs related to emotional beliefs, the extant measures of emotion regulation (Catanzaro & Mearns, 1990; Gratz & Roemer, 2004; Gross & John, 2003), emotional intelligence (Salovey, Mayer, Goldman, Turvey, & Palfai, 1995), and responses to emotions (Baer, Smith, Hopkins, Krieteymeyer, & Toney, 2006; Gámez, Chmielewski, Kotov, Ruggero, & Watson, 2011; Harmon-Jones, Harmon-Jones, Amodio, & Gable, 2011; Simons & Gaher, 2005; Taylor et al., 2007) assess self-efficacy in dealing with emotions or self-reported tendencies in emotional situations, but they do not assess erroneous beliefs about the emotions themselves. Essentially, the extant measures address one’s own perceived emotional control (Hofmann, 2005; White, Brown, Somers, & Barlow, 2006) rather than beliefs about the construct of emotion. For example, one item on the Difficulties with Emotion Regulation Scale (DERS; Gratz & Roemer, 2004) is “I experience my emotions as overwhelming and out of control.” Endorsement of this item suggests that an individual perceives his or her emotions to be uncontrollable, but does not assess whether this individual believes emotions are generally overwhelming for all people (Tamir & Mauss, 2011). Even items on the Beliefs About Emotions Scale (BAES; Rimes & Chalder, 2010), which theoretically measures beliefs about emotions, includes items such as “I should be able to cope with difficulties on my own without turning to others for support,” and “It would be a sign of weakness to show my emotions in public.” These items likewise address an individual’s perception of personal control over emotion rather than beliefs about the emotions themselves. A recent study addressed this distinction and found that both general (i.e., about people) and personal (i.e., beliefs about the self) malleability beliefs uniquely predicted lower use of cognitive reappraisal and greater levels of psychological distress (De Castella et al., 2013), although personal malleability beliefs may be more predictive of behavior than general malleability beliefs. Despite the evidence that personal beliefs and general beliefs are different constructs, assessment thus far has focused on malleability beliefs only, necessitating a more extensive assessment of general emotional beliefs than currently exists.

Implications for Mental Health

The majority of the small corpus of work beliefs of emotion malleability has addressed the effect of entity beliefs on emotion regulation outcomes and negative affect cross-sectionally (De Castella et al., 2013), longitudinally (Tamir et al., 2007) and in the lab (Kappes & Schikowski, 2013). Despite the clear implications of emotion beliefs on mental health, only one published study to date has evaluated the effect of emotion beliefs on psychiatric symptoms. Researchers found that beliefs that emotions are not malleable are associated with higher levels of depression and stress (De Castella et al., 2013). These findings are consistent with clinical advice to address “myths” about emotion in psychotherapy (Leahy et al., 2011), and consistent with anxiety research suggesting that perceived emotional control decreases symptoms of anxiety (Hofmann, 2005) and avoidance behaviors (White et al., 2006). Theoretically, individuals who hold negative beliefs about emotion should be less likely to engage in adaptive and flexible emotion regulation strategies, thus increasing symptoms and diagnoses of psychological disorders (De Castella et al., 2013).

Overview

The Emotion and Regulation Beliefs Scale (ERBS) was developed to assess beliefs about emotion more comprehensively than measured in prior studies (Tamir et al., 2007). Namely, we wished to broaden the assessment of implicit theories of emotion to explore beliefs beyond emotion malleability, with the intention of constructing a measure with enhanced psychometric characteristics. The scale construction and validation strategy roughly followed guidelines described by Clark and Watson (1995) and Reise, Waller, and Comrey (2000). The initial phase involved generation of an overinclusive item pool and administration to a diverse sample of community participants (N = 255) with the goal of establishing an initial factor structure. The second phase involved development of new items and administration to a separate community and college sample to finalize factor structure and provide initial validity evidence.

Study 1 Method

Preliminary Item Pool

We expanded on the malleability-focused item pool (Tamir et al., 2007) by including items that suggest myths about emotion (Leahy et al., 2011; Linehan, 1993b, Spradlin, 2003), such as items related to emotional “contagion” (e.g., emotions can “infect” others; Savani, Kumar, Naidu, & Dweck, 2011), the sense that emotions are all-or-nothing phenomena, the belief that emotions may dictate behavior, and the belief that people cannot help the way they experience particular emotions. We also included items addressing broader beliefs in controllability and flexibility of emotions, such that people can display partial emotions (e.g., the “half smile,” Spradlin, 2003) and mixed emotions (Leahy et al., 2011). Items were generated by the first two authors after consulting with treatment manuals and texts that refer to emotion beliefs, with the remaining authors vetting and revising the items for clarity and readability. Approximately half of the items were worded positively, suggesting beliefs in emotion flexibility and controllability, and the other half worded to suggest adherence to negative judgments about emotion, including beliefs that emotions are forces that reduce self-control. With the intention of aiming for an overinclusive item set (Clark & Watson, 1995), we generated 53 preliminary items. All items were presented on a scale from 1 (strongly agree) to 5 (strongly disagree). Items were worded with agree at the low end of the scale to maintain consistency with other studies of implicit self-theories (Hong, Chiu, Dweck, Lin, & Wan, 1999; Job et al., 2010).

Participants and Procedure

Participants were recruited via Amazon Mechanical Turk, a web-based service that allows “workers” to complete online tasks for small amounts of money. Mechanical Turk has been increasingly used as a recruitment tool in the behavioral sciences, and has been shown to provide samples of at least equal quality to traditional college student (i.e., subject pool) samples (Buhrmester, Kwang, & Gosling, 2011), and has also demonstrated validity in recruiting clinical populations (Shapiro, Chandler, & Mueller, 2013).

The current study restricted workers to individuals from the United States. Participants were paid $0.40 to complete the measure. Eleven participants were deleted for incomplete data (e.g., participants who did not finish the first measure of the questionnaire set), and 9 were excluded for low effort (e.g., participants with no variability and those who completed the entire set in less than 5 minutes), leaving a sample of 255 participants. The percentage of excluded participants (7.8%) was similar to other studies that used completion time to exclude participants for effort (e.g., Winer, Veilleux, & Ginger, 2014). The sample was just more than half female (54.3%), primarily Caucasian (73.3%), with a mean age of 34.05 years (SD = 12.46) and represented geographic diversity across the United States in terms of region and city size. Only 18.8% of the sample reported current college student status, 39% were married, 47.6% reported at least a bachelor’s degree, and 45.9% were employed at least 20 hours per week.

Study 1 Results

Prior to analyses, items were coded so that high scores represented a more adaptive view (e.g., emotions can be controlled). Item characteristics were examined and seven items with negative item–total correlations were removed. Factor structure of the remaining 46 items was examined via exploratory factor analysis (principal axis factoring [PAF]). Oblique (promax) rotation was used as we had no reason to believe that the factors would be uncorrelated, and the Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy was .88, suggesting the data were appropriate and sample size sufficient for factor analysis.

The initial solution produced 11 factors with eigenvalues greater than 1, explaining 46.6% of the variance; however, the scree plot indicated a two-factor solution, which was further validated by the finding that Factors 3 through 11 were all one- or two-item factors. After dropping cross-loading items and items that did not load highly (>.40), a PAF constrained to two factors resulted in an interpretable solution accounting for 40.71% of the variance. All 27 items loaded at .40 or higher on only one of the two factors. The first factor (α = .93) included items consistent with incremental emotion beliefs, whereas the second factor (α = .82) consisted of items assessing entity beliefs.

Preliminary scales assessing each factor were calculated using the mean score on the set of items for each factor. The two scales evidenced a significant, albeit weak, negative correlation (r = −.18, p < .05).

Study 1 Discussion

Although prior studies of implicit beliefs in emotion used a simple unidimensional measure (Kappes & Schikowski, 2013; Tamir et al., 2007), these initial scale analyses revealed two separate factors, only mildly correlated. However, the Factor 1 scale was highly negatively skewed, with a median of 4.19. This overendorsement is similarly reflected in Dweck’s measurement of implicit beliefs in intelligence (Hong et al., 1999) and is stated as the rationale for including only entity items when measuring implicit beliefs. We also wished to bolster the integrity of the scales by developing more items consistent with those remaining after initial factor analysis.

Study 2 Method

The second phase of data collection had four functions: (a) refinement of the item pool, (b) inclusion of a college student sample in addition to a separate community sample, (c) detailed examination of factor structure, and (d) further testing of convergent, discriminant, and construct validity.

Additional items were generated that were consistent with the 27 items from Phase 1. Rather than returning to the entire set of possible emotion beliefs, we explicitly added items that theoretically corresponded with the items remaining after initial factor analyses. Newly added items addressed specific emotions (e.g., “When a person feels really angry, it’s virtually impossible to not take the anger out on people or objects nearby”), opposite action (e.g., “It’s virtually impossible for people to act opposite to the way they feel”), and the notion that emotions “rule” people (e.g., “People are slaves to their emotions”). We intentionally included both incremental and entity-phrased items, as we did not want to assume that control and changeability beliefs are mutually exclusive (John & Gross, 2004). Instead, in an attempt to decrease overadherence to incremental items, we added several strongly worded items with absolute phrasing (e.g., “It is possible, with effort, to alter strong feelings in any situation”), such as are included on social desirability measures (e.g., Crowne & Marlow, 1960).

Participants

A student sample (N = 333) was recruited from a psychology subject pool at a large mid-south university. Students received credit toward an introductory psychology course research requirement for participation in an online study, also administered via Qualtrics (Provo, UT). The sample was predominantly female (67.9%) and Caucasian (85.4%), with an average age of about 19 years (M = 19.4 years, SD = 2.47).

A second online sample (N = 624) was recruited from Amazon Mechanical Turk, just as in Study 1. Participants were restricted to individuals from the United States. Participants were paid $0.75 to complete a 20-minute survey. This sample was similar to that of Study 1 in all respects; participants were predominantly female (60.1%), Caucasian (79.7%), with an average age in the mid-thirties (M = 34.54 years, SD = 13.09). The sample was also similar with regard to geographic diversity, percentage of college students (19.8%), percentage with at least a bachelor’s degree (46.2%) and percentage employed at least 20 hours per week (49.7%).

Participants were excluded for completing measures outside the average completion time or for no variability on the target measure. The student sample was additionally asked to self-report whether or not they put forth effort after the full set of surveys and participants who admitted no effort were excluded. After these exclusions, 281 remained in the student sample and 576 in the online sample.

Measures

Emotion and Regulation Beliefs Scale

The revised set of 47 items were given on a 1 (strongly disagree) to 5 (strongly agree) scale. Of note, this scale was oppositely anchored to the items given in Phase 1, as we wanted higher scores on each subscale to reflect greater adherence to those beliefs.

Implicit Theories of Emotion Scale

In the Implicit Theories of Emotion Scale (ITES, Tamir et al., 2007), 4 items assessed implicit beliefs in the malleability of emotion, two incremental items (e.g., “If they want to, people can change the emotions that they have”), and two entity items (e.g., “The truth is, people have very little control over their emotions”) on a scale from 1 (strongly disagree) to 5 (strongly agree). Tamir et al. (2007) found the scale to assess a unidimensonal construct with adequate reliability. Alphas for all Study 2 measures can be found in Table 4.

Implicit Intelligence

Three items were used to assess implicit theories of intelligence (Dweck & Henderson, 1988), taken from studies assessing implicit beliefs (e.g., Hong et al., 1999). These items all assessed entity beliefs (e.g., “you have a certain amount of intelligence and you really can’t do much to change it”) and scored from 1 (strongly agree) to 6 (strongly disagree) where higher scores represent more incremental beliefs. Brief scales such as this one have demonstrated adequate reliability and validity (Hong et al., 1999; Tamir et al., 2007).

Emotion Regulation Questionnaire

The Emotional Regulation Questionnaire (ERQ; Gross & John, 2003) is a 10-item measure that assesses two facets of emotion regulation: Cognitive Reappraisal, or the use of altering thinking about emotions to change emotional experience, and Suppression, attempts to suppress emotional expression. The ERQ was included in the current study to verify that incremental beliefs are associated with higher reappraisal (Tamir et al., 2007). The subscales of the ERQ demonstrate good internal consistency (Gross & John, 2003) are often-used measures of emotion regulation strategies (Kashdan et al., 2006; Tamir et al., 2007).

Trait Meta-Mood Scale

The Trait Meta-Mood Scale (TMMS; Salovey et al., 1995) is a 30-item measure of emotional intelligence. Items are given on a 1 (strongly disagree) to 5 (strongly agree) Likert-type scale. The TMMS has three subscales, attention to emotions, clarity of feelings, and mood repair. The scale has demonstrated excellent reliability and validity and has been used often to assess emotional intelligence and general approaches to emotion regulation (Gratz & Roemer, 2004; Gross & John, 2003; Salovey et al., 1995).

Beliefs About Emotions Scale

The BAES (Rimes & Chalder, 2010) is a 12-item measure assessing the unacceptability of experiencing and expressing negative emotions. Items are measured on a 7-point Likert-type scale from 0 (totally disagree) to 6 (totally agree). The scale has demonstrated adequate reliability and validity, and was associated with heightened anxiety and depression in a sample of individuals with chronic fatigue syndrome (Rimes & Chalder, 2010) and general distress in people with early-stage multiple sclerosis (Dennison, Moss-Morris, Silber, Galea, & Chalder, 2010). Data on this measure were collected for the student sample only.

Alcohol Use Disorders Identification Test

The Alcohol Use Disorders Identificaton Test (AUDIT; Saunders, Aasland, Babor, de la Fuente, & Grant, 1993) is a 10-item measure assessing problem drinking behaviors. Participants are provided with information about standardized drinks and asked to provide information relating to frequency and quantity of alcohol consumed as well as problematic consequences associated with alcohol use. The AUDIT is commonly used as a screener for alcohol problems, where scores greater than 8 suggest hazardous drinking and scores about 16 are indicative of possible alcohol dependence (Saunders et al., 1993).

Binge Eating Scale

The Binge Eating Scale (BES; Gormally, Black, Daston, & Rardin, 1982) is a 16-item measure assessing the tendency to lose control over eating. Each item has three or four possible responses as a Guttman-type scale, where higher scores are given to behaviors associated with more binge eating. For example, a 0-point item is “I don’t think much about trying to control unwanted eating urges” and the associated 3-point item is “It seems to me that most of my waking hours are preoccupied by thoughts about eating or not eating. I feel like I’m constantly struggling not to eat.” Items are summed to form the overall binge eating score. This scale has excellent reliability and validity (Gormally et al., 1982) and is one of the most frequently used measures of binge eating behavior.

Depression, Anxiety, and Stress Scales

The 21-item Depression, Anxiety, and Stress Scales (DASS-21; Henry & Crawford, 2005) assesses symptoms of stress (e.g., “I found it hard to wind down”), anxiety (e.g., “I felt scared without any good reason”), and depression (e.g., “I felt that life wasn’t worthwhile”) within the past week. The scale has demonstrated adequate reliability and validity in both clinical (Brown, Chorpita, Korotitsch, & Barlow, 1997) and nonclinical (Henry & Crawford, 2005) samples.

Study 2 Results

Measure Evaluation

After removing nonloading items in Study 1 and developing new items, Study 2 evaluated the revised 47-item measure in three stages. First, items were evaluated for overendorsement and redundancies. Seven items were removed for having means greater than 4.25 or less than 1.75. Six of these items were items representing incremental beliefs and one was an entity item. Intercorrelations among items were calculated with the intention of removing items that correlated at an absolute value of .60 with other items, but no items were removed based on this criteria.

Second, factor structure was evaluated using exploratory factor analysis (principal axis factoring using oblique rotation) in the student sample. KMO in the initial solution was .80, again supporting the sample size for factor analysis. Similar to Study 1, the initial solution produced 11 factors with eigenvalues more than 1, but a scree plot strongly suggested three factors (Floyd & Widaman, 1995). To be thorough, two-, three-, and four-factor solutions were investigated. The four-factor solutions had only two items in the fourth subscale, and the three-factor solution had a better fit than the two-factor solution, so the three-factor solution was retained. Sixteen items that did not load highly (>.40) on any of the three factors were removed (no items had cross-loadings >.4 on two scales). Together, the three factors accounted for 36.00% of the variability in scores and were highly interpretable.

Next, an initial confirmatory factor analysis [CFA] was conducted on approximately 50% of the mTurk sample, N = 309, using maximum likelihood estimation in Amos 20.0. Half of the sample was used so that factor structure could be revised following initial fit. This strategy thus represents an exploratory use of the CFA procedures. The initial model was not a good fit, χ² = 646.93, df = 249, χ²/df = 2.60, comparative fit index (CFI) = .83, root mean square error of approximation (RMSEA) = .07 (90% confidence interval [CI] = .065-.079). Residuals and modification indices were evaluated and three items were removed because of high covariance with the nonprimary factor. Moreover, error variances between similarly worded items were allowed to covary. These modifications resulted in adequate model fit, χ² = 357.00, df = 182, χ²/df = 1.96, CFI = .91, RMSEA = .06 (90% CI = .047-.064).

Finally, the final three-factor CFA model was evaluated for replicability using the other random half of the online sample (N = 267) which had acceptable model fit, χ² = 296.92 df = 182, χ²/df = 1.63, CFI = .93, RMSEA = .05 (90% CI = .04-.06). See Table 2 for the final scale items and regression weights from the final CFA model (using the second half of the online sample).

Factor 1 was labeled Emotional Constraint, representing the beliefs that emotions are forces that constrain or narrow an individual’s choices in an emotional situation (see Table 1 for items and factor loadings). Factor 2, labeled Regulation Worth, represents the belief that emotion regulation is both possible and worthwhile, and Factor 3, labeled Hijack, represents the belief that emotions hijack or usurp peoples’ self-control. At first blush, the Emotion Constraint beliefs and the Hijack beliefs appear similar, as both appear be composed of entity items. However, Hijack beliefs are associated with strong emotions and loss of self-control, whereas Emotion Constraint beliefs are associated with longevity of emotions and behavioral constraint consistent with the experienced emotion.

Table 1.

Final Scale Items, Item Descriptives and Regression Weights From Final Confirmatory Factor Analysis Model (N = 267).

Number	Item	Mean	SD	Factor 1 weight	Factor 2 weight	Factor 3 weight
11.	When people acknowledge their emotions, the emotions will completely take them over.	2.03	1.03	.70
14.	When a person feels really angry, it’s virtually impossible to not take the anger out on people or objects nearby.	1.98	1.13	.66
7.	When people are feeling down, they have to wait for a better mood to arrive before they can be productive.	2.44	1.19	.63
6.	When a person has a strong emotional reaction to another person, they will always feel that way about that other person.	2.06	1.08	.62
18.	When feelings of sadness take over, a person can’t really do anything but wallow in the misery.	1.82	0.96	.60
20.	It’s virtually impossible for people to act opposite to the way they feel.	2.03	0.99	.60
2.	Emotions can either be expressed entirely or hidden from others—it isn’t possible to share only part of an emotional response.	2.44	1.21	.59
10.	When an emotion comes along, it will continue unless there is a change in the environment.	2.61	1.07	.59
1.	Emotions operate like a floodgate that is either open or closed. In other words, emotions are either “on” or “off.”	2.59	1.14	.56
12.	Learning how to alter strong emotions is a worthwhile pursuit.	4.02	0.82		.76
19.	People benefit from learning how to regulate their feelings.	4.24	0.73		.71
8.	People would be better off if they took time to figure out where their emotions come from.	4.16	0.71		.65
5.	Putting forth effort to alter emotional experience is valuable.	4.05	0.71		.62
16.	People would be better off if they spent more time learning how to control their emotions.	3.91	0.88		.61
13.	It is possible, with effort, to alter strong feelings in any situation.	3.88	0.85		.51
3.	People can learn to control/regulate their emotions.	4.18	0.71		.43
9.	When strong emotions are present, they dictate what a person says or does.	3.07	1.14			.67
15.	People are slaves to their emotions.	2.12	1.06			.64
4.	People are ruled by their emotions.	2.85	1.02			.63
17.	Strong emotions will make people do things they wouldn’t normally do.	3.86	1.04			.41
21.	Emotions make people lose control.	3.46	1.13			.41

Correlations between each subscale and means for the subscales in both the online and college samples are reported in Table 2. Although the magnitudes of the correlation coefficients are slightly different for each sample, the general scope of relations is the same. Emotion Constraint beliefs were positively associated with Hijack beliefs, and Emotion Constraint beliefs were negatively associated with Regulation Worth beliefs. There was no significant association between Regulation Worth beliefs and Hijack beliefs. When the subscales were compared by sample, we found that the online sample held lower Emotion Constraint and Hijack beliefs, and higher beliefs in Regulation Worth compared with the college sample.

Table 2.

Correlations Among Subscales of the ERBS.

				M (SD)
	1.	2.	3.	College sample	Online Sample	t test
1. Emotion Constraint	—	−.29**	.51**	2.34 (0.69)	2.20 (0.70)	−2.74*
2. Regulation Worth	−.15*	—	−.05	3.94 (0.56)	4.06 (0.56)	2.90**
3. Hijack	.42**	.03	—	3.22 (.069)	3.05 (0.76)	−3.17*

Note. ERBS = Emotion and Regulation Beliefs Scale. College student correlations below diagonal, mTurk above diagonal.

p < .05. **p < .01.

Reliability

The internal consistency, evaluated using Cronbach’s alpha and AICs (average interitem correlations) of the three subscales can be found in Table 3, presented separately for the student and community sample, along with means and standard deviations for subscale scores. The alphas are all acceptable (>.7), slightly lower for the five-item Hijack subscale compared with the longer subscales, and AICs range from .30 to 34.

Table 3.

Internal Consistency of ERBS by Sample.

	α		AIC
	College sample (N = 281)	Online sample (N = 576)	College sample	Online sample
Emotion Constraint	.79	.83	.30	.36
Regulation Worth	.78	.82	.32	.40
Hijack	.70	.74	.30	.36

Note. ERBS = Emotion and Regulation Beliefs Scale; AIC = average interitem correlation.

Validity

Convergent and Discriminant Validity

Associations between the ERBS subscales, other measures of implicit beliefs and measures of emotion regulation tendencies are displayed in Table 4. We expected to see positive relationships between Regulation Worth, incremental beliefs in emotion (ITES), cognitive reappraisal, and facets of emotional intelligence as measured by the TMMS. As expected, the ERBS scales significantly correlated with the ITES, where strongest correlation was with the Regulation Worth Scale (r = .52). We also expected significant domain specificity, and as predicted, we saw low correlations (<.15 in magnitude) between the three ERBS scales and implicit beliefs in intelligence. Further demonstrating domain specificity, none of the ERBS scales nor the ITES correlated significantly with the BAES, suggesting that these constructs are indeed distinct.

Table 4.

Bivariate Correlations of ERBS Subscales and Measures Assessing Convergent and Discriminant Validity.

Scale	1.	2.	3.	4.	5.	6.	7.	8.	9.	10.	11.
1. Emotion Constraint	.82
2. Regulation Worth	−.25**	.80
3. Hijack	.49**	−.03	.73
4. ITES	−.35**	.51**	−.23**	.78
5. BAES^a	.06	.10	.07	−.00	.88
6. Implicit Intelligence	−.12**	.08*	−.07*	.19**	−.13*	.93
7. ERQ: Reappraisal	−.16**	.38**	−.08*	.38**	−.16*	.11*	.87
8. ERQ: Suppression	.26**	−.03	.07	−.06	.33**	−.11*	−.03	.74
9. TMMS: Attention	−.24**	.09*	.01	.07	−.22**	.12*	.15*	−.46**	.83
10. TMMS: Clarity	−.32**	.21**	−.20**	.29**	−.31**	.11*	.33**	−.26**	.32**	.86
11. TMMS: Repair	−.21**	.23**	−.16**	.33**	−.21**	.15**	.56**	−.19**	.25**	.45**	.88

Note. Cronbach’s alpha reported in italics along the diagonal. ERQ = Emotion Regulation Questionnaire; ITES = Implicit Theories of Emotion Scale; TMMS = Trait Meta-Mood Scale; BAES = Beliefs About Emotion Scale; ERBS = Emotion and Regulation Beliefs Scale.

Data from student sample only (N = 281).

p < .05. **p < .01.

Of note, only the Emotion Constraint beliefs subscale predicted use of expressive suppression as an emotion regulation strategy, whereas all three subscales were differentially associated with reappraisal, with higher Regulation Worth beliefs particularly associated with greater use of cognitive reappraisal. Emotion Constraint beliefs were associated with lower attention to emotions, lower clarity of emotions, and lower self-efficacy in emotional repair.

We also adopted a hierarchical regression approach in testing the incremental validity of the ERBS over the four-item ITES by placing the ITES in Step 1 and the three ERBS subscales in Step 2 in predicting emotion regulation outcomes (ERQ and TMMS subscales). This strategy assessed the unique predictive power of each ERBS subscale after controlling for variability in implicit beliefs of emotion malleability. Results are presented in Table 5. The ITES predicted both cognitive reappraisal and suppression, as well as emotional clarity and mood repair. Importantly, the ERBS subscales predicted all five emotion regulation outcomes above and beyond the ITES. Regulation Worth significantly predicted cognitive reappraisal and emotion repair, consistent with the idea that people who value emotion regulation are more likely to engage in adaptive regulation processes. Emotion Constraint beliefs predicted greater use of expressive suppression and lower emotional clarity. Hijack and Emotion Constraint beliefs differentially predicted attention to emotion such that higher Hijack beliefs were associated with greater attention to emotions, whereas higher Emotion Constraint beliefs were associated with less attention to emotions. These results indicate that despite moderately high correlations, the two entity subscales are differentially associated with salient emotional outcomes.

Table 5.

Hierarchical Regressions With ERBS Predicting Emotion Regulation Outcomes After Controlling for the Four-Item ITES.

	Step 1		Step 2
	R ²	Four-item β	ΔR²	Emotion Constraint β	Regulation Worth β	Hijack β
ERQ: Reappraisal	.14**	.38**	.05**	.01	.26**	−.02
ERQ: Suppression	.01	−.06	.07**	.31**	.05	−.09*
TMMS: Attention	.00	.07	.07**	−.31**	.03	.14**
TMMS: Clarity	.08**	.29**	.06**	−.22**	.06	−.06
TMMS: Repair	.11**	.33**	.02**	−.07	.08*	−.06

Note. ERBS = Emotion and Regulation Beliefs Scale; ITES = Implicit Theories of Emotion Scale; TMMS = Trait Meta-Mood Scale.

p < .05. **p < .01.

Clinical Utility

We wished to investigate the potential clinical utility of the ERBS by assessing the association between the ERBS scales and important outcomes known to be associated with emotion regulation (depression, anxiety, binge eating, and alcohol use). Due to the high prevalence of these outcomes in college students as well as the regularity in which they are assessed in college student populations, the overall sample was divided into college students (N = 270, including the entire subject pool sample and the college student–identified Mechanical Turk participants) and noncollege students (N = 461). Independent t tests confirmed no differences in binge eating between college students (M = 8.69, SD = 7.74) and noncollege participants (M = 8.00, SD = 8.23), t(729) = 2.42, p < .05. Depression scores were marginally higher in the college students (M = 6.79, SD = 7.44) compared with noncollege participants (M = 5.80, SD = 7.78), t(729) = 1.94, p = .05. There were no differences in anxiety scores (t < 1). After excluding individuals who indicated they were nondrinkers (e.g., had not consumed alcohol in the past year), college students also reported higher hazardous drinking (M = 7.09, SD = 5.06) than noncollege participants (M = 4.72, SD = 4.55), t(563) = 3.79, p < .001.

Correlations between ERBS subscales and clinically relevant outcomes are presented separately for college students and noncollege participants (see Table 6). Emotion Constraint and Hijack beliefs were associated with increased depression, anxiety, and binge eating in both samples. In the noncollege participants, higher Regulation Worth was associated with less depression, and Hijack beliefs were associated with greater alcohol use.

Table 6.

Correlations Among ERBS Subscales and Clinical Outcome Variables of Depression, Anxiety, Binge Eating and Alcohol Use Among College Students (N = 270) and Noncollege Students (N = 461).

	Emotion Constraint		Regulation Worth		Hijack
Scale	College	Noncollege	College	Noncollege	College	Noncollege
Depression	.16*	.28**	−.04	−.13**	.22**	.18**
Anxiety	.15**	.15**	.01	−.09	.24**	.19**
Binge Eating	.11^†	.12*	−.02	−.06	.14*	.21*
Alcohol^a	.07	.05	−.08	.06	.08	.14**

Note. ERBS = Emotion and Regulation Beliefs Scale.

Results restricted to individuals who reported drinking within the past year; college students (N = 306) and noncollege students (N = 351).

†

p < .07. *p < .05. **p < .01.

In addition to correlations, we also performed a series of multiple regression analyses regressing each of the four clinically relevant outcomes on the ERBS subscales after controlling for college student status (Step 1) and ITES scores (Step 2). Results indicated that after controlling for college student status, ITES predicted increase alcohol use ΔR² = .01, p < .05, and the ERBS scales did not account for additional variance, ΔR² = .01, ns. However, higher incremental beliefs on the ITES were associated with less binge eating, (ΔR² = .02, β = −.13, p < .001). In addition, the ERBS scales predicted an additional 3% of the variance in binge eating, where only Hijack beliefs were uniquely associated with greater binge eating (β = .15, p < .001).

Higher incremental beliefs on the ITES predicted lower depression, ΔR² = .03, β = −.18, p < .001, and the set of ERBS subscales predicted an additional 4% of the variability in depressive symptoms. Specifically, higher Emotion Constraint beliefs (β = .14, p < .001) and higher Hijack beliefs (β = .11, p < .05) also predicted increased depression, although Regulation Worth did not. A similar pattern emerged for anxiety symptoms, with the ITES predicting 6% of the variability in anxiety (β = −.24, p < .001), and the ERBS accounting for an additional 3% after controlling for incremental beliefs. Hijack was the only significant unique ERBS predictor of anxiety (β = .15, p < .001).

Study 2 Discussion

Correlations of the ERBS with emotion regulation and clinically relevant outcomes revealed that Emotion Constraint beliefs were associated with the tendency to engage in expressive suppression, as well as the tendency to pay less attention to emotions and engage in less emotional repair efforts. These findings are notable because Tamir et al. (2007) did not find a relationship between incremental beliefs and suppression; it appears that more extensive assessment of emotional beliefs adds to the prediction of emotion regulation strategies. Specifically, beliefs that emotions constrain or restrict behavior are associated with the experience of facial restriction. Future research may investigate the mechanisms and direction of the relationship, as it may be that use of expressive suppression leads to increased Emotional Constraint beliefs, as expressive suppression is a way of constraining emotional responses.

On the other hand, Regulation Worth beliefs were associated with use of cognitive reappraisal, consistent with past research (Tamir et al., 2007), as well as increased clarity of emotions and attention to mood repair. Beliefs that emotions can usurp or hijack self-control appear to be important in predicting symptoms of psychopathology, especially binge eating and anxiety. Moreover, it is notable that the correlations between emotion beliefs and outcomes were not identical for college and noncollege participants. For example, Regulation Worth beliefs were not predictive of symptoms for college students, and Hijack beliefs predicted alcohol use only for noncollege participants. Further research may wish to examine the mechanisms by which these beliefs predict behavior, as well as how the beliefs may differ across the lifespan.

We also found that the three ERBS subscales are significantly related to each other, but are conceptually distinct. The strongest relationship among the three scales was between the Emotion Constraint scale and the Hijack acale, which is logical as both are broadly representative of more entity beliefs in emotion. However, we found unique predictions for each scale, such that when in the same model predicting attention to emotions, the Hijack acale positively predicted attention to emotions, whereas the Emotion Constraint scale negatively predicted attention to emotions. These results suggest that perhaps Hijack beliefs are more associated with overwhelming emotions which are difficult to ignore, whereas Emotion Constraint beliefs are associating with emotional distancing. These predictions may be worth examining in future studies using the ERBS.

Moreover, the ERBS is not redundant with the four-item ITES; the ERBS significantly predicted emotion regulation tendencies and symptoms of psychopathology after controlling for the ITES. In addition, the ERBS was not significantly related to one included measure of personal beliefs about emotion (BAES). It would be useful to compare the ERBS with additional constructs measuring personal beliefs about emotional management (i.e., perceived emotional control), such as difficulties with emotion regulation (Gratz & Roemer, 2004), or distress tolerance (Simons & Gaher, 2005). Moreover, although the ERBS scales have clinical implications, it remains unclear if the ERBS is useful in discriminating clinical from nonclinical samples. These questions are addressed in Study 3.

Study 3 Method

We wished to compare the ERBS scales with other measures theoretically related to emotion beliefs, in a continued attempt to establish discriminant validity. We also hoped to establish further clinical utility of the measure by demonstrating that the ERBS scales predict clinical samples above and beyond extant measures of emotion regulation. Research suggests that individuals who suffer from depression and eating disorders evidence difficulties with emotion regulation (Aldao, Nolen-Hoeksema, & Schweizer, 2010). Thus, we focused on members of these two groups during the present investigation.

Participants

A third online sample (N = 554) was recruited from Amazon Mechanical Turk. Participants were restricted to individuals from the United States, and were paid $3 to complete a 1-hour long survey. Participants were also predominantly female (54.8%) and Caucasian (79.2%), with an average age in the mid-thirties (M = 34.58 years, SD = 12.16). The sample was similar to earlier samples in terms of geographic diversity, percentage of college students (13.8%), percentage with at least a bachelor’s degree (44.8%), and percentage employed at least 20 hours per week (57.8%).

Measures

Emotion and Regulation Beliefs Scale

Same as in Study 2. See Table 7 for alpha values

Table 7.

Bivariate Correlations of ERBS Subscales and Emotion Regulation Measures.

Scale	1.	2.	3.	4.	5.	6.
1. Emotion Constraint	.84
2. Regulation Worth	−.22**	.81
3. Hijack	.64**	−.06	.75
4. ITES	−.39**	.51**	−.29**	.84
5. DERS	.15**	−.06	.10*	−.10*	.96
6. DTS	−.37**	.07	−.30**	.18**	–.09*	.94

Note. Cronbach’s alpha reported in italics along the diagonal. ERBS = Emotion and Regulation Beliefs Scale; ITES = Implicit Theories of Emotion Scale; DERS = Difficulties with Emotion Regulation Scale; DTS = Distress Tolerance Scale.

p < .05. **p < .01.

Implicit Theories of Emotion Scale

Same as in Study 2.

Difficulties with Emotion Regulation Scale

The Difficulties with Emotion Regulation Scale (DERS; Gratz & Roemer, 2004) is a 36-item self-report measure assessing dimensions associated with emotion dysregulation, including lack of acceptance of emotional responses, difficulties engaging in goal-directed behavior, limited access to emotion regulation strategies, lack of emotional awareness, impulsivity, and lack of emotional clarity. Items are given on a 5-point Likert-type scale from 1 (almost never) to 5 (almost always). Responses are scored such that higher ratings indicate increased problems with emotion regulation. In the current study, total DERS score were used.

Distress Tolerance Scale

The Distress Tolerance Scale (DTS; Simons & Gaher, 2005) is a self-report measure assessing self-perceived distress tolerance or the ability to withstand negative emotional states. The measure is given on a 1 (strongly agree) to 5 (strongly disagree) Likert-type scale where high scores indicate a greater tolerance of distressing affective states. The DTS is one of the most popular measures of affect tolerance (Leyro, Zvolensky, & Bernstein, 2010), and has excellent reliability and validity (Simons & Gaher, 2005).

Depression, Anxiety and Stress Scales–21-Item Version

Same as in Study 2. Cutoff points in published norms (Lovibond & Lovibond, 1995), suggestion individuals with a score of 22 of above correspond to “severe” levels of depression.

Eating Disorder Diagnostic Scale

The Eating Disorder Diagnostic Scale (EDDS; Stice, Telch, & Rizvi, 2000) is a 22-item self-report measure assessing specific eating disorder diagnoses at both threshold and subthreshold levels, according to DSM-IV diagnostic criteria, including provisional diagnoses of binge eating disorder. Items assess body image and weight perceptions, binge eating, compensatory (i.e., purging) behaviors, loss of self-control related to eating, and restriction of food intake, as well as body mass index. The scale has shown excellent validity in discriminating among eating disorder groups and adequate temporal reliability (Stice et al., 2000).

Study 3 Results

First, zero-order correlations were conducted between the ERBS and the emotion regulation variables (Table 7). As in Study 2, Emotion Constraint and Hijack were positively associated, Regulation Worth and Emotion Constraint were negatively associated, and there was no significant relationship between Hijack and Regulation Worth. The pattern of relationships between the ERBS and the ITES was also essentially the same as in Study 2. We also found that the DERS was significantly, albeit weakly, associated with Emotion Constraint and Hijack, suggesting that individuals who believe that emotions constrain behavior and are associated with a loss of self-control have greater emotion regulation difficulties. Finally, the ERBS scales were associated with distress tolerance whereby higher Emotion Constraint and Hijack beliefs corresponded with lower self-perceived abilities to tolerate negative states.

Second, approximations of clinical samples were calculated from the DASS and the EDDS. Participants were classified as depressed (N = 61) or nondepressed (N = 493) based on a cutoff score of 22 (“severe”) on the DASS (Lovibond & Lovibond, 1995). Participants were likewise classified as “eating disordered” if they met at least subthreshold criteria for any eating disorder on the EDDS (N = 125). These were primarily individuals meeting criteria for bulimia nervosa (n = 86), with a smaller sample meeting criteria for binge eating disorder (n = 37), and only a few subthreshold for anorexia (n = 2).

We compared the emotion belief and regulation measures between clinical and nonclinical samples using independent samples t tests (Table 8). The depressed groups was different from the nondepressed group on all variables except Regulation Worth; depressed people had higher Emotion Constraint and Hijack beliefs and greater problems with emotion regulation, lower incremental beliefs, and lower distress tolerance. A similar pattern emerged with the eating disorder groups.

Table 8.

Independent Samples t Tests Examining Emotion Regulation and Emotion Beliefs Scores by Clinical Group.

	Depressed	Nondepressed	t test	Eating disordered	Noneating disordered	t test
Emotion Constraint	2.97 (.66)	2.28 (.74)	6.99**	2.63 (.78)	2.27 (.74)	4.72**
Regulation Worth	3.79 (.67)	3.92 (.62)	−1.47*	3.91 (.59)	3.90 (.64)	.09
Hijack	3.47 (.64)	3.03 (.78)	4.22**	3.30 (.74)	3.01 (.78)	3.68**
ITES	3.36 (.80)	3.66 (.80)	−2.75**	3.47 (.90)	3.66 (.78)	−2.40*
DERS	111.20 (21.53)	78.52 (25.58)	9.56*	94.18 (26.71)	78.60 (26.29)	5.81**
DTS	2.80 (.97)	3.32 (.94)	4.03**	3.08 (.91)	3.31 (.97)	−2.36*

Note. ITES = Implicit Theories of Emotion Scale; DERS = Difficulties with Emotion Regulation Scale; DTS = Distress Tolerance Scale.

†

p < .07. *p < .05. **p < .01.

Finally, as a fairly stringent test of incremental validity, we performed two multinomial logistic regressions predicting disorder classification from the set of belief and regulation variables; all three of the ERBS scales, the ITES, the DERS, and the DTS were entered as predictors. The model predicting depression classification fit the data well, R² = .16 (Cox & Snell), χ²(6) = 93.58, p < .001. With all the variables in the model, only the DERS—B = .05, SE = 0.1, Wald χ²(1) = 36.41, p < .001—and Emotion Constraint—B = .91, SE = 0.28, Wald χ²(1) = 10.46, p < .01—predicted depression status. A similar result emerged for predicting eating disorder status, where the DERS—B = .02, SE = 0.1, Wald χ²(1) = 13.28, p < .001—and Emotion Constraint—B = .37, SE = 0.19, Wald χ²(1) = 3.70, p = .05—predicted eating disorder diagnostic status.

Study 3 Discussion

The pattern of zero-order correlations was consistent with our earlier findings (Study 2) and we also demonstrated low-to-moderate relationships between the ERBS scales and the indices of emotion dysregulation, further validating the notion that the ERBS scales represent distinct constructs. We also found that negative beliefs about emotion (e.g., Emotion Constraint and Hijack) are higher in individuals who would meet criteria for an eating disorder and likely would meet criteria for depression. Finally, in a strict evaluation of the incremental validity of the ERBS, we found that Emotion Constraint predicts clinical groups uniquely, in addition to emotion regulation difficulties, even when the ITES and distress tolerance were also in the model. Further work will be needed to identify the domains in which Regulation Worth and Hijack may likewise contribute to psychopathology and emotion regulation difficulties.

Overall Discussion

The development of the ERBS contributes to the literature on implicit self-theories (Dweck, 1986) as well as the growing literature on emotion regulation. Our results provide preliminary evidence for three separate beliefs about emotion: (a) the belief that emotions can “hijack” self-control, (b) the belief that emotions can constrain behavior, and (c) the belief that emotion regulation, even if effortful, is a worthwhile pursuit. The findings suggest the importance of measuring multiple dimensions of emotion beliefs rather than focusing only on implicit beliefs of emotion malleability (Kappes & Schikowski, 2013; Tamir et al., 2007), and are consistent with other domains that have identified several implicit beliefs (e.g., Marziano, Ward, Beech, & Pattison, 2006) as well as with clinical wisdom regarding the utility of challenging “myths” about emotion during treatment (Leahy et al., 2011; Linehan, 1993b; Spradlin, 2003).

In their review of social cognitive factors in emotion regulation, Tamir and Mauss (2011) title the section about malleability of emotion as “Can emotions be controlled and can I control my emotions?” This heading suggests that these are truly two separate questions, the controllability of an attribute versus personal abilities to control an attribute (i.e., self-efficacy). Thus far, the majority of research on emotion malleability, controllability, and regulation has fallen into the personal attribute camp. The convergent and discriminant validity results from the ERBS suggest that beliefs about emotion are distinguishable from beliefs about self-efficacy in emotion regulation, including beliefs about the unacceptability of emotion and emotion expression (i.e., the BAES) and self-perceived difficulties in emotion regulation (the DERS). These results indicate that beliefs about the construct of emotion are distinct from beliefs about personal experiences with emotion and the value of perceived emotional control.

The current study, with three separate scales, counters the unidimensional ITES scale by supporting the notion of multiple and potentially contradictory emotional beliefs (John & Gross, 2004; Kennedy-Moore & Watson, 1999). Indeed, recent evidence suggests that individuals use multiple emotion regulation strategies (Aldao & Nolen-Hoeksema, 2012) rather than assuming that people have sole “go-to” regulation processes, as is suggested by trait self-report measures (Gross & John, 2003). In this work, we have found that the set of beliefs assessed by the ERBS allows people to hold multiple beliefs. Moreover, although the Emotion Constraint and Hijack scales are conceptually similar, both relatively consistent with an entity perspective, they are not redundant, as they have exhibited opposite relationships to some constructs (e.g., attention to emotions) and differentially predict outcomes. Our conjecture is that Emotion Constraint beliefs are associated with fear of emotions, whereas Hijack beliefs are used as justifications for engaging in risk behaviors when emotional (e.g., emotional urgency). It may be that these beliefs differentiate those with emotional undercontrol (e.g., borderline personality disorder, who may be likely to endorse Hijack beliefs), from those with emotional overcontrol (e.g., obsessive–compulsive personality disorder), although these predictions will require empirical examination before any definitive assertions can be made.

It may be that situational characteristics activate different beliefs and contribute to use of varying emotion regulation strategies across situations, consistent with if . . . then models of behavior (Cervone, 2004; Mischel & Shoda, 1995). For example, it may be that situations that evoke strong feelings of anger (e.g., jealousy situations) activate Hijack beliefs, which contribute to lack of inhibition (e.g., punching the wall). Future research will need to test the idea that these beliefs can be differentially activated based on contextual factors. Similarly, future work will also want to address the malleability of the beliefs themselves. In the work on implicit beliefs in intelligence, priming incremental beliefs resulted in greater effort on challenging tasks, just as priming entity beliefs resulted in the tendency to quit sooner (Dweck & Leggett, 1988). No published work to date has examined whether emotion beliefs shift in response to treatment; we would expect to see lower Emotion Constraint and Hijack beliefs following a successful course of treatment. It may also be worth examining the Regulation Worth subscale as an indicator of treatment pursuit or adherence; people who do not believe that emotion regulation is worth the effort may be less likely to seek treatment or commit to treatment compared with those with high Regulation Worth beliefs. These are questions ripe for future investigation.

Interpretation of the current work requires attention to methodological limitations. The results are self-report, and it is thus unknown how the ERBS beliefs predict momentary emotional responses or dynamic regulation processes. People vary in their accessibility of emotions (Gratz & Romer, 2004) and thus it will be important to validate the effect of emotion beliefs on in vivo emotional responding in future studies. Also, although the initial scale attempted to broadly address implicit beliefs in emotion, we do not claim that the beliefs here fully address the common judgments people hold about emotion. For example, some people also believe that emotions should be “released” (e.g., that holding on to emotions is damaging; Kennedy-Moore & Watson, 1999). Indeed, the myth that anger, in particular, must be expressed, has been identified (and debunked) as one of the 50 Great Myths of Popular Psychology (Lilienfeld, Lynn, Ruscio, & Beyerstein, 2010). Others hold the opposite myth, that expressions of emotion are signs of weakness, and nonemotionality indicates strength (Kennedy-Moore & Watson, 1999). Additional beliefs about emotion may include further assessment of the transience of emotions (e.g., emotions are fleeting or long-lasting), the viability of mixed emotions (e.g., if it is possible to experience both positive and negative emotions at the same time). Items representing these additional beliefs were included in our initial item pool, but they were discarded following factor analysis, suggesting that there were insufficient items to fully capture these alternative beliefs.

Additionally, the studies presented here used both community and college student samples; however, the participants were still predominantly White women. Moreover, even though prior research has demonstrated the viability of Mechanical Turk as a data collection strategy and that the data collected using Mechanical Turk workers is at least of equal quality than typical college student participants (Buhrmester et al., 2011; Shapiro, Chandler, & Mueller, 2013), the extent to which these beliefs generalize beyond these samples are heretofore unknown. Replication of the results with other samples is necessary to ensure utility of the ERBS, including a variety of diverse nonclinical and clinical populations, particularly other groups prone to emotion dysregulation difficulties (e.g., substance abusers, personality disorders).

Strengths of the current study include sampling of both community and college student participants, inclusion of clinical subgroups, and a carefully stepped process of scale development and examination of factor structure. Future research may also wish to examine the relationship of the ERBS to other relevant measures, such as experiential avoidance (Gámez et al., 2011; Kashdan et al., 2006), or anxiety sensitivity (Taylor et al., 2007). As discussed above, whether the ERBS predicts momentary emotional fluctuation or in vivo emotion regulation processes also remains to be tested. Similarly, for the ERBS to be a clinically useful instrument, the ERBS should be examined as a predictor of treatment attrition and treatment outcomes. As an initial attempt to measure multiple myths and beliefs about emotion, the development of the ERBS paves the way for future research on implicit beliefs about emotion from a multidimensional perspective. It appears that in addition to beliefs about emotion malleability, people hold beliefs about other aspects of emotion that preliminarily show promise as predictors of emotion regulation processes and a range of symptom presentations.

Footnotes

Acknowledgements

The authors would like to thank the Social Psychology group at the University of Arkansas for guidance and feedback on this research.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Aldao

Nolen-Hoeksema

(2012). One versus many: Capturing the use of multiple emotion regulation strategies in response to an emotion-eliciting stimulus. Cognition & Emotion, 27, 753-760. doi:10.1080/02699931.2012.739998

Aldao

Nolen-Hoeksema

Schweizer

(2010). Emotion-regulation strategies across psychopathology: A meta-analytic review. Clinical Psychology Review, 30, 217-237. doi:10.1016/j.cpr.2009.11.004

Baer

R. A.

Smith

G. T.

Hopkins

Krietemeyer

Toney

(2006). Using self-report assessment methods to explore facets of mindfulness. Assessment, 13, 27-45.

Barlow

D. H.

Fairchoine

T. J.

Fairholme

C. P.

Ellard

K. K.

Boisseau

C. L.

Allen

L. B.

Ehrenreich-May

J. T.

(2010). Unified protocol for transdiagnostic treatment of emotional disorders: Therapist guide. New York, NY: Oxford University Press.

Beer

J. S.

(2002). Implicit self-theories of shyness. Journal of Personality and Social Psychology, 83, 1009-1024.

Brown

T. A.

Chorpita

B. F.

Korotitsch

Barlow

(1997). Psychometric properties of the Depression Anxiety Stress Scales (DASS) in clinical samples. Behaviour Research and Therapy, 35, 79-89.

Buhrmester

Kwang

Gosling

S. D.

(2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3-5. doi:10.1177/1745691610393980

Catanzaro

S. J.

Mearns

(1990). Measuring generalized expectancies for negative mood regulation: Initial scale development and implications. Journal of Personality Assessment, 54, 546-563.

Cervone

(2004). The architecture of personality. Psychological Review, 111, 183-204. doi:10.1037/0033-295X.111.1.183

10.

Chiu

C. Y.

Dweck

C. S.

Tong

J. Y. Y.

J. H. Y.

(1997). Implicit theories and conceptions of morality. Journal of Personality and Social Psychology, 73, 923-940.

11.

Chiu

C. Y.

Hong

Y. Y.

Dweck

C. S.

(1997). Lay dispositionism and implicit theories of personality. Journal of Personality and Social Psychology, 73, 19-30.

12.

Clark

L. A.

Watson

(1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309-319. doi:10.1037//1040-3590.7.3.309

13.

Crowne

D. P.

Marlow

(1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349-354.

14.

De Castella

Goldin

Jazaieri

Ziv

Dweck

C. S.

Gross

J. J.

(2013). Beliefs about emotion: Links to emotion regulation, well-being, and psychological distress. Basic and Applied Social Psychology, 35, 497-505. doi:10.1080/0197-3533.2013.840632.

15.

Dennison

Moss-Morris

Silber

Galea

Chalder

(2010). Cognitive and behavioral correlates of different domains of psychological adjustment in early-stage multiple sclerosis. Journal of Psychosomatic Research, 69, 353-361. doi:10.1016/j.jpsychores.2010.04.009

16.

Dillon

D. G.

Deveney

C. M.

Pizzagalli

D. A.

(2011). From basic processes to real-world problems: How research on emotion and emotion regulation can inform understanding of psychopathology, and vice versa. Emotion Review, 3, 74-82. doi:10.1177/1754073910380973

17.

Dweck

C. S.

(1986). Motivational processes affecting learning. American Psychologist, 41, 1040-1048.

18.

Dweck

C. S.

(1999). Self-theories: Their role in motivation, personality, and development. New York, NY: Psychology Press.

19.

Dweck

C. S.

Henderson

V. L.

(1988). Theories of intelligence: Background and measures. Unpublished manuscript.

20.

Dweck

C. S.

Leggett

E. L.

(1988). A social-cognitive approach to motivation and personality. Psychological Review, 95, 256-273. doi:10.1037//0033-295X.95.2.256

21.

Ellard

K. K.

Fairholme

C. P.

Boisseau

C. L.

Farchoine

T. J.

Barlow

D. H.

(2010). Unified protocol for the transdiagnostic treatment of emotional disorders: Protocol development and initial outcome data. Cognitive and Behavioral Practice, 17, 88-101.

22.

Floyd

F. J.

Widaman

K. F.

(1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, 286-299.

23.

Gámez

Chmielewski

Kotov

Ruggero

Watson

(2011). Development of a measure of experiential avoidance: The Multidimensional Experiential Avoidance Questionnaire. Psychological Assessment, 23, 692-713. doi:10.1037/a0023242

24.

Gormally

Black

Daston

Rardin

(1982). The assessment of binge eating severity among obese persons. Addictive Behaviors, 7, 47-55.

25.

Gratz

K. L.

Roemer

(2004). Multidimensional assessment of emotion regulation and dysregulation: Development, factor structure, and initial validation of the Difficulties in Emotion Regulation Scale. Journal of Psychopathology and Behavioral Assessment, 26, 41-54. doi:10.1023/B:JOBA.0000007455.08539.94

26.

Gross

J. J.

John

O. P.

(2003). Individual differences in two emotion regulation processes: Implications for affect, relationships, and well-being. Journal of Personality and Social Psychology, 85, 348-362. doi:10.1037/0022-3514.85.2.348

27.

Gross

J. J.

Muñoz

(1995). Emotion regulation and mental health. Clinical Psychology: Science and Practice, 2, 151-164.

28.

Harmon-Jones

Amodio

D. M.

Gable

P. A.

(2011). Attitudes toward emotions. Journal of Personality and Social Psychology, 101, 1332-1350. doi:10.1037/a0024951

29.

Henry

J. D.

Crawford

J. R.

(2005). The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. British Journal of Clinical Psychology, 44, 227-239. doi:10.1348/014466505X29657

30.

Hofmann

S. G.

(2005). Perception of control over anxiety mediates the relation between catastrophic thinking and social anxiety in social phobia. Behaviour Research and Therapy, 43, 885-895. doi:10.1016/j.brat.2004.07.002

31.

Hong

Chiu

Dweck

C. S.

Lin

D. M.-S.

Wan

(1999). Implicit theories, attributions, and coping: A meaning system approach. Journal of Personality and Social Psychology, 77, 588-599. doi:10.1037/0022-3514.77.3.588

32.

Job

Dweck

C. S.

Walton

G. M.

(2010). Ego depletion—Is it all in your head? Implicit theories about willpower affect self-regulation. Psychological Science, 21, 1686-1693. doi:10.1177/0956797610384745

33.

John

O. P.

Gross

J. J.

(2004). Healthy and unhealthy emotion regulation: Personality processes, individual differences, and life span development. Journal of Personality, 72, 1301-1333. doi:10.1111/j.1467-6494.2004.00298.x

34.

Kappes

Schikowski

(2013). Implicit theories of emotion shape regulation of negative affect. Cognition & Emotion, 27, 952-960. doi:10.1080/02699931.2012.753415

35.

Kashdan

T. B.

Barrios

Forsyth

J. P.

Steger

M. F.

(2006). Experiential avoidance as a generalized psychological vulnerability: Comparisons with coping and emotion regulation strategies. Behaviour Research and Therapy, 44, 1301-1320. doi:10.1016/j.brat.2005.10.003

36.

Kennedy-Moore

Watson

J. C.

(1999). Expressing emotion: Myths, realities and therapeutic strategies. New York, NY: Guilford Press.

37.

Knee

C. R.

(1998). Implicit theories of relationships: Assessment and prediction of romantic relationship initiation, coping, and longevity. Journal of Personality and Social Psychology, 74, 360-370.

38.

Kring

A. M.

(2008). Emotion disturbances as transdiagnostic processes in psychopathology. In Lewis

Haviland-Jones

J. M.

Barrett

L. F.

(Eds.), Handbook of emotions (3rd ed., pp. 691-705). New York, NY: Guilford Press.

39.

Labroo

A. A.

Mukhopadhyay

(2009). Lay theories of emotion transience and the search for happiness: A fresh perspective on affect regulation. Journal of Consumer Research, 36, 242-254. doi:10.1086/597159

40.

Leahy

R. L.

Tirch

Napolitano

L. A.

(2011). Emotion regulation in psychotherapy: A practitioner’s guide. New York, NY: Guilford Press.

41.

Leyro

T. M.

Zvolensky

M. J.

Bernstein

(2010). Distress tolerance and psychopathological symptoms and disorders: A review of the empirical literature among adults. Psychological Bulletin, 136, 576-600. doi:10.1037/a0019712

42.

Lilienfeld

S. O.

Lynn

S. J.

Ruscio

Beyerstein

B. L.

(2010). 50 Great myths of popular psychology: Shattering widespread misconceptions about human behavior. Malden, MA: Wiley-Blackwell.

43.

Linehan

M. M.

(1993a). Cognitive-behavioral treatment of borderline personality disorder. New York, NY: Guilford Press.

44.

Linehan

M. M.

(1993b). Skills training manual for treating borderline personality disorder. New York, NY: Guilford Press.

45.

Lovibond

S. H.

Lovibond

P. F.

(1995). Manual for the Depression Anxiety Stress Scales (2nd ed.). Sydney, New South Wales, Australia: Psychology Foundation.

46.

Marziano

Ward

Beech

A. R.

Pattison

(2006). Identification of five fundamental implicit theories underlying cognitive distortions in child abusers: A preliminary study. Psychology, Crime & Law, 12, 97-105. doi: 10.1080/10683160500056887

47.

Mennin

D. S.

Heimberg

R. G.

Turk

C. L.

Fresco

D. M.

(2005). Preliminary evidence for an emotion dysregulation model of generalized anxiety disorder. Behaviour Research and Therapy, 43, 1281-1310. doi:10.1016/j.brat.2004.08.008

48.

Mischel

Shoda

(1995). A cognitive-affective system theory of personality: Reconceptualizing situations, dispositions, dynamics, and invariance in personality structure. Psychological Review, 102, 246-268.

49.

Reise

S. P.

Waller

N. G.

Comrey

A. L.

(2000). Factor analysis and scale revision. Psychological Assessment, 12, 287-297. doi:10.1037//1040-3590.12.3.287

50.

Rimes

K. A.

Chalder

(2010). The Beliefs about Emotions Scale: Validity, reliability and sensitivity to change. Journal of Psychosomatic Research, 68, 285-292.

51.

Salovey

Mayer

J. D.

Goldman

S. L.

Turvey

Palfai

T. P.

(1995). Emotional attention, clarity, and repair: Exploring emotional intelligence using the Trait Meta-Mood Scale. In Pennebaker

J. W.

(Ed.), Emotion, disclosure and health (pp. 125-154). Washington, DC: American Psychological Association.

52.

Saunders

J. B.

Aasland

O. G.

Babor

T. F.

de la Fuente

J. R.

Grant

(1993). Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO collaborate project on early detection of persons with harmful alcohol consumption II. Addiction, 88, 791-804.

53.

Savani

Kumar

Naidu

N. V. R.

Dweck

C. S.

(2011). Beliefs about emotional residue: The idea that emotions leave a trace in the physical environment. Journal of Personality and Social Psychology, 101, 684-701. doi:10.1037/a0024102

54.

Schnabel

Asendorpf

J. B.

Greenwald

A. G.

(2008). Assessment of individual differences in implicit cognition: A review of IAT measures. European Journal of Psychological Assessment, 24, 210-217. doi:10.1027/1015-5759.24.4.210

55.

Shapiro

D. N.

Chandler

Mueller

P. A.

(2013). Using Mechanical Turk to study clinical populations. Clinical Psychological Science, 1, 213-220. doi:10.1177/216770-2612469015

56.

Simons

J. S.

Gaher

R. M.

(2005). The Distress Tolerance Scale: Development and validation of a self-report measure. Motivation and Emotion, 29, 83-102. doi:10.1007/s11031-005-7955-3

57.

Spradlin

S. E.

(2003). Don’t let emotions run your life: How dialectical behavior therapy can put you in control. Oakland, CA: New Harbinger.

58.

Stice

Telch

C. F.

Rizvi

S. L.

(2000). Development and validation of the Eating Disorder Diagnostic Scale: A brief self-report measure of anorexia, bulimia, and binge-eating disorder. Psychological Assessment, 12, 123-131.

59.

Tamir

John

O. P.

Srivastava

Gross

J. J.

(2007). Implicit theories of emotion: Affective and social outcomes across a major life transition. Journal of Personality and Social Psychology, 92, 731-744. doi:10.1037/0022-3514.92.4.731

60.

Tamir

Mauss

I. B.

(2011). Social cognitive factors in emotion regulation: Implications for well-being. In Nyklicek

VingerHoets

Zeelenberg

(Eds.), Emotion regulation and well being (pp. 31-47). New York, NY: Springer.

61.

Taylor

Zvolensky

M. J.

Cox

B. J.

Deacon

Heimberg

R. G.

Ledley

D. R.

. . .Cardenas

S. J.

(2007). Robust dimensions of anxiety sensitivity: Development and initial validation of the Anxiety Sensitivity Index-3. Psychological Assessment, 19, 176-188. doi:10.1037/1040-3590.19.2.176

62.

White

K. S.

Brown

T. A.

Somers

T. J.

Barlow

D. H.

(2006). Avoidance behavior in panic disorder: The moderating influence of perceived control. Behaviour Research and Therapy, 44, 147-157. doi:10.1016/j.brat.2005.07.009

63.

Winer

E. S.

Veilleux

J. C.

Ginger

E. J.

(2014). Development and validation of the Specific Loss of Interest and Pleasure Scale. Journal of Affective Disorders, 152-154, 193-201.