Incremental Validity of the Subscales of the Emotional Regulation Related to Testing Scale for Predicting Test Anxiety

Abstract

The Emotional Regulation Related to Testing Scale (ERT Scale) assesses strategies students use to regulate emotion related to academic testing. It has four dimensions: Cognitive Appraising Processes (CAP), Emotion-Focusing Processes (EFP), Task-Focusing Processes (TFP), and Regaining Task-Focusing Processes (RTFP). The study examined the factor structure of each dimension and the incremental validity of the ERT subscales (N = 213 undergraduates). Confirmatory factor analysis indicated good fit for CAP and EFP dimensions. TFP and RTFP models had poor fit. The TFP dimension appears to involve a two-factor structure, which may account for why it has been observed to have low internal consistency reliability across studies. Subscales of CAP and EFP dimensions resulted in statistically significant increments in R² after accounting for self-efficacy for learning and metacognitive self-regulation of learning. The TFP dimension and RTFP subscales did not exhibit incremental validity. The importance of assessing self-regulation of emotion and learning is emphasized.

Keywords

confirmatory factor analysis emotion regulation ERT Scale incremental validity test anxiety

Most college students experience test anxiety to some degree (Lowe, 2015; Nelson, Lindstrom, & Foels, 2014). Some may benefit from manageable levels of test anxiety, yet others may experience excessive levels, which could result in reduced academic performance or academic failure, psychological distress, illness, and institutional departure (Zeidner, 2007). For students to gain the most benefits of the college experience, managing test anxiety is of great importance.

The goals of the present study include the following: (a) examine the psychometric properties and factor structure of an instrument designed to assess how students perceive academic exams and how they regulate emotions related to academic testing, and (b) examine incremental validity of subscales of the instrument after accounting for self-efficacy for learning and an important component of self-regulation of learning, metacognitive self-regulation, the extent to which students perceive their ability to regulate their learning and make appropriate adjustments to improve it.

Emotion Regulation (ER) and Academic Testing

ER is part of a broader concept of affect regulation that is one of four constructs, which include psychological defenses, mood regulation, coping, and ER (Gross & Thompson, 2007). “Emotion Regulation requires the activation of a goal to up-or-down-regulate either the magnitude or duration of the emotional response” (Gross, 2013, p. 399). According to one model, ER consists of five processes, which include situation selection, situation modification, attentional deployment, cognitive change, and response modulation (DeSteno, Gross, & Kubzansky, 2013; Gross & Thompson, 2007).

Situation selection is when a person chooses a situation that will arouse positive emotion or avoids a situation that will arouse negative emotion. Students who suffer from test anxiety may avoid classes that have difficult exams. Situation modification is when a person makes an effort to directly modify a situation to minimize its emotional impact. Instructors may place easier exam questions at the beginning of an exam to allow students to experience a sense of mastery early in the test situation, and students may in turn search for the easiest test items for the same reason. Attentional deployment involves redirecting attention within a situation (DeSteno et al., 2013). Redirecting attention to attempt comprehension of test questions and then rereading to ensure comprehension keeps attention focused on the task rather than on anxiety. Cognitive change refers to changing one’s perception of a situation to influence its emotional impact. This can be done “either by changing how we think about the situation or about our capacity to manage the demands it poses” (Gross & Thompson, 2007, p. 14). Students may reappraise the benefits and/or limitations of tests for helping them evaluate their progress in attaining an academic goal. Response modulation refers to attempts to regulate the physiological or emotional response to a situation. Awareness of an accelerated heart rate or shallow breathing may alert students to increased anxiety, which in turn results in an attempt to relax.

Efforts to identify students who are at risk for test anxiety include criterion-related validity research in which measures of emotional intelligence (EI; for example, awareness of emotion), personality (e.g., neuroticism, trait anxiety), and state anxiety have been used as predictors. Although EI is negatively correlated with test anxiety (Austin, Saklofske, & Mastoras, 2010), EI may be limited in its ability to predict constructs that potentially affect academic performance for two reasons. First, content of global measures may not overlap with situationally specific content (test anxiety), and global measures of EI may not predict narrow criteria (Harms & Credé, 2010). Incremental validity of EI is more likely to be observed when trait-level EI measures are used rather than global measures (Petrides, Pérez-González, & Furnham, 2007). In a similar vein, use of broad personality measures may underestimate criterion validity, and prediction may be improved when personality facets are used rather than broad domain scores (O’Connor & Paunonen, 2007; Paunonen & Ashton, 2001). Efforts to establish incremental criterion validity should include use of facets rather than global measures (Smith, Fischer, & Fister, 2003).

Measurement of ER Related to Academic Testing

A measure of perceived ability to regulate emotion during testing, the Emotional Regulation Related to Testing Scale (ERT Scale), provides trait-based assessment that is situationally specific. The ERT Scale is designed to assess appraisals and strategies students use for ER related to testing. Its development was based on a model of ER (Gross & Thompson, 2007) and appraisal theory (Folkman & Lazarus, 1985). The ERT Scale includes four dimensions: Cognitive Appraising Processes (CAPs), Emotion-Focusing Processes (EFPs), Task-Focusing Processes (TFPs), and Regaining Task-Focusing Processes (RTFPs; Schutz, Benson, & DeCuir-Gunby, 2008).

CAPs involve cognitive change. For example, beliefs that one has personal control over test performance and that exams are congruent with achieving academic goals. EFPs involve attentional deployment in which attention is focused on emotions associated with testing (e.g., anxiety, anger, guilt). For example, students may experience anger during an exam when they realize that their preparation for the exam was inadequate. Such attentional deployment is disadvantageous because attentional resources are allocated to emotional processing rather than to the test itself. In contrast, TFPs involve attentional deployment that is advantageous, with attentional resources allocated to the test itself. For example, students may reword difficult test questions to better understand what is asked. RTFPs involve cognitive change and response modification. For example, students may reappraise the relative importance of a test for determining course grades (cognitive change) and/or attempt to relax (response modulation).

Self-Regulation of Learning and Academic Testing

Pintrich (2004) conceptualized self-regulation of learning as involving cognitive, affective/motivational, behavioral, and contextual factors. Students have numerous opportunities to engage in appropriate activities prior to a test situation. For example, they read objectives for an upcoming exam, apply an appropriate study strategy, read relevant material, and they monitor comprehension and retrieval while using the study strategy. Such extensive preparation may be beneficial for increasing judgment of learning and confidence in anticipation of an upcoming exam, which in turn may reduce emotional arousal prior to and during the exam. The main advantage of this model is the identification of multiple domains of self-regulation that include cognitive, affective/motivational, behavioral, and contextual variables.

A more recent metamemory model (Bjork, Dunlosky, & Kornell, 2013) identifies numerous decisions about type of processing students use when selecting a study strategy, deciding when to terminate studying, and testing retrieval in an effort to monitor what has been learned. Emotions could affect such decisions, for example, students may possibly avoid the process altogether until just before the exam. Identifying test-appropriate study strategies and monitoring effectiveness of selected study strategies are consistent with facets of metacognitive self-regulation. McCabe (2011) observed that metacognitive scores were correlated with knowledge of evidence-based study strategies. She concluded that many undergraduate students are not well-informed about the learning benefits of empirically supported approaches to learning without direct instruction. In addition, metacognitive self-regulation has been observed to be related to academic performance (Credé & Phillips, 2011; Richardson, Abraham, & Bond, 2012) and to retention (Robbins et al., 2004). This line of thinking is supported by evidence that test-anxious students may have deficits in their approaches to learning (Brown & Nelson, 1983; Covington & Omelich, 1987; Naveh-Benjamin, McKeachie, & Lin, 1987), that is, test anxiety may be the result of more than dispositional factors such as trait anxiety. A related construct, self-efficacy for learning and academic performance, concerns confidence that one can experience successful outcomes when engaged in academic situations. Self-efficacy for academic learning is moderately correlated with academic performance (i.e., college grade point average; Richardson et al., 2012; Robbins, Allen, Casillas, Peterson, & Le, 2006; Robbins et al., 2004) and retention (Robbins et al., 2006; Robbins et al., 2004).

The purpose of the present study was to expand the framework for investigating incremental validity of the ERT subscales by including measures of academic self-efficacy and metacognitive self-regulation related to learning. We specifically focused on demonstrating incremental criterion validity of the ERT subscales after accounting for metacognitive self-regulation and self-efficacy for learning. The study also investigated relationships between each of the ERT subscales, test anxiety, metacognitive self-regulation, and self-efficacy for learning. In addition, we attempted to follow the work of Schutz and colleagues (Schutz et al., 2008; Schutz, DiStefano, Benson, & Davis, 2004) by examining the factor structure of each of the ERT dimensions.

Method

Participants

Participants included 213 undergraduate college students (77% women, 79% first year) who were recruited from psychology classes (e.g., primarily introductory psychology and developmental psychology, but also biological, health, and abnormal psychology) at a 4-year university in the Midwest. The selected classes have representation of several majors (primarily education, nursing, psychology, and social work). Age ranged from 17 to 48 years (M = 20.5, SD = 4.4). Percentages of Caucasians, Hispanics, African Americans, and Asian Americans were 88.7%, 5.2%, 4.7%, and 0.5%, respectively.

Instruments and Procedure

The following instruments were administered following completion of the informed-consent procedure and after students had completed at least one exam in the course.

Motivated Strategies for Learning Questionnaire (MSLQ)

The MSLQ (Pintrich, Smith, Garcia, & McKeachie, 1991) includes a 5-item measure to test anxiety (awareness of heart rate and thoughts about test performance and consequences of test performance related to tests), an 8-item measure of self-efficacy about academic learning and performance, and a 12-item measure of metacognitive self-regulation in academic learning (planning, monitoring, and regulation). The MSLQ has a Likert-type scale ranging from 1 (not at all true of me) to 7 (very true of me). Cronbach’s coefficient alpha values for the Test Anxiety, Self-Efficacy for Learning and Performance, and the Metacognitive Self-Regulation subscales are .80, .93, and .79, respectively (Pintrich, Smith, Garcia, & McKeachie, 1993). The decision to use the MSLQ Test Anxiety subscale was based on its brevity, breadth, and degree of specificity that closely matches that of the ERT Scale. The statements for the three subscales were modified to include perceptions related to “a course” rather than “this course.” Such global assessment is consistent with ERT Scale assessment in that the ERT Scale does not focus on a specific test or course.

ERT Scale (Schutz et al., 2008)

The latest version of this scale has 34 items with a Likert-type scale ranging from 1 (almost never) to 5 (almost always). Cronbach’s coefficient alpha (Schutz et al., 2008) is displayed within parentheses. The CAP dimension includes facet subscales of Goal Congruence (.72), Agency (.79), and Testing Problem-Efficacy (.75). The EFP dimension includes facet subscales of Self-Blame (.84) and Wishful Thinking (.79). The RTFP dimension includes facet subscales of Importance Reappraisal (.68) and Tension Reduction (.74). Each subscale consists of four items. The TFP dimension includes six items (.64).

Results

Two surveys were omitted because of incomplete reporting (i.e., an entire section of items without a response); thus, 213 surveys included usable data.

Confirmatory Factor Analysis

Maximum likelihood estimation was used in light of evidence that assumptions for such estimation were tenable and that the sample size was sufficient for maximum likelihood estimation. Model fit was evaluated based on the following well-performing fit indices: comparative fit index (CFI), root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR). Good fit is indicated by CFI ≥ .95, RMSEA ≤ .05, and SRMR ≤ .08; acceptable fit is indicated by CFI ≥ .90 but < .95, and RMSEA > .05 but < .08 (Browne & Cudeck, 1993; Hu & Bentler, 1999). We elected to examine each dimension separately as opposed to a higher-order model to be consistent with the approach used by Schutz et al. (2008). In addition, researchers may elect to use some subscales and not others for purposes of identifying students at risk for academic failure. For example, Davis, DiStefano, and Schutz (2008) observed several subscales to differentiate clusters of first-year students, primarily subscales of the cognitive-appraising and emotion-focusing dimensions.

Results are displayed in Table 1. Chi-square goodness-of-fit test values are also reported. For each dimension, a one-dimensional model (Model 1) was compared with a model based on Schutz et al. (2008). For all dimensions, Model 2 fit better than Model 1. Fit for CAP and EFP dimensions was good based on at least two fit indices. For the CAP dimension, factor loadings ranged from .27 to .89, from .57 to .94, and from .56 to .80, for Agency, Goal Congruence, and Testing Problem-Efficacy, respectively. For the EFP dimension, factor loadings ranged from .63 to .81 and from .64 to .79 for Wishful Thinking and Self-Blame, respectively.

Table 1.

Confirmatory Factor Analysis of Dimensions of the Emotional Regulation Related to Testing Scale.

Dimension	Model	χ²	df	CFI	RMSEA	SRMR
Cognitive-appraising processes
Cognitive-appraising processes	1	417.99	54	.61	.18 [.16, .19]	.13
	2	139.27	51	.97	.09 [.07, .11]	.11
Emotion-focusing processes
	1	119.88	20	.86	.15 [.13, .18]	.07
	2	54.61	19	.95	.09 [.07, .12]	.05
TFP
		34.54	9	.67	.12 [.08, .16]	.08
Regaining TFP
	1	224.77	20	.61	.22 [.19, .25]	.12
	2	176.68	19	.70	.20 [.17, .23]	.09
	3	49.77	18	.94	.09 [.06, .12]	.05

Note. N = 213. Model 1 is a unidimensional model with original items. Model 2 is based on Schutz, Benson, and DeCuir-Gunby (2008). Model 3 (Regaining TFP) included the addition of an estimate of the correlated error for two items of the Tension Reduction subscale. TFP is unidimensional. CFI = comparative fit index; RMSEA = root mean square error of approximation (with 90% confidence interval); SRMR = standardized root mean square error residual. TFP = Task-Focusing Processes.

In contrast, fit of the TFP dimension did not meet criteria for acceptable fit, especially for CFI and RMSEA indices. Factor loadings ranged from .25 to .64. This could be unique to this sample, however, Schutz et al.’s (2008) results did not indicate a good fit, with the values of fit indices suggesting mediocre to acceptable fit.

Low values of Cronbach’s coefficient alpha could be due to lack of unidimensionality, item heterogeneity, and/or respondent interpretation of specific items based on personal experience (McCrae, Kurtz, Yamagata, & Terracciano, 2011). For example, students who have had extensive experience with multiple-choice exams may interpret some of the items differently. Cronbach’s coefficient alpha for the present study was only .51, which was slightly below estimates reported in previous studies. The average inter-item correlation coefficient provides additional evidence of internal consistency. If the TFP dimension represents a broad construct, an average inter-item correlation of .15 may be acceptable (Clark & Watson, 1995), however the observed range of inter-item correlations, from −.05 to .33, was beyond an accepted standard of .05 to .50, suggested by Clark and Watson (1995).

Given evidence of poor fit, we used exploratory factor analysis (EFA) to examine TFP’s factor structure. A principal axis factors solution with direct oblimin rotation (Δ= 0) was used. Preliminary analysis supported the factorability of the correlation matrix (Kaiser–Meyer–Olkin measure of sampling adequacy = .59; Bartlett’s test of sphericity, p < .001). A two-factor solution was indicated with three items per factor. Pattern coefficients ranged from .32 to .54 for one factor, which focused on comprehension of questions. For the second factor, which appeared to involve searching for answers to questions, pattern coefficients ranged from .31 to .71. Eigenvalues for the second and third factors were 1.27 and 0.88, respectively. A parallel analysis was conducted using a recommended procedure in which our eigenvalues were compared with the 95th percentiles of first, second, and so on eigenvalues of 50 randomly generated samples (Hayton, Allen, & Scarpello, 2004; Thompson & Daniel, 1996). Results also supported a two-factor structure.

The RTFP dimension also appears to have questionable fit, although subscales have adequate reliability. Schutz et al. (2008) claim that RTFP was originally included with the EFP dimension, however, model fit was poor. Factor loadings ranged from .31 to .69 and from .55 to .79 for Importance Reappraisal and Tension Reduction, respectively. One area of strain based on a standardized residual covariance value of 5.93 and a modification index value of 100.6 indicated that estimating a parameter for a correlated error for two items of Tension Reduction (“I take a deep breath” and “I take a minute to relax”) would improve model fit. Estimation of the parameter improved quality of fit based on CFI, RMSEA, and SRMR values. Given that correlated errors may indicate the possibility of item redundancy, unexplained variance due to another latent variable, or are unique in terms of how participants of a particular sample interpret specific items, estimating the parameter to improve model fit was not justifiable in this case.

Relationships Among Variables

Means, standard deviations, Cronbach’s coefficient alpha values, and zero-order correlation coefficients involving relationships among variables are displayed in Table 2. The effect size or magnitude of each correlation coefficient was interpreted based on Cohen (1992): r = .15 (small effect), r = .30 (medium effect), r = .50 (large effect).

Table 2.

Descriptive Statistics, Zero-Order Correlation Coefficients, and Reliability Coefficients.^a

Variable	1	2	3	4	5	6	7	8	9	10	11
Test Anxiety (1)	.85^a
Metacognitive Self-Regulation (2)	−.35***	.78^a
Self-Efficacy for Learning (3)	−.55***	.55***	.93^a
Agency (4)	−.28***	.23***	.35***	.75^a
Goal Congruence (5)	−.66***	.38***	.65***	.29***	.78^a
Testing Problem-Efficacy (6)	−.52***	.51***	.57***	.35***	.56***	.76^a
Task-Focusing Processes (7)	−.25***	.54***	.40***	.19**	.29***	.51***	.51^a
Wishful Thinking (8)	.67***	−.41***	−.48***	−.27***	−.70***	−.50***	−.21**	.80^a
Self-Blame (9)	.69***	−.32***	−.53***	−.18**	−.69***	−.51***	−.18**	.63***	.82^a
Importance Reappraisal (10)	−.05^†	.23***	.13*	.00^†	−.05^†	.28***	.23***	.08^†	−.15*	.66^a
Tension Reduction (11)	−.13*	.26***	.20**	.16**	.09^†	.30***	.36***	−.07^†	−.19**	.41***	.77^a
M	20.86	49.19	37.83	4.21	3.09	3.22	3.84	3.43	3.26	2.92	3.71
SD	7.03	10.05	8.69	0.65	0.98	0.76	0.54	0.99	0.99	0.78	0.90

Cronbach’s coefficient alpha.

†

p > .05. *p ≤ .05. **p ≤ .01. ***p ≤ .001 (df = 211, two-tailed).

Relationships with test anxiety were all statistically significant with the exception of Importance Reappraisal. Both subscales of EFP were positively correlated with test anxiety, with large effect sizes observed. Negative relationships with large effect sizes included the Goal Congruence and Testing Problem-Efficacy subscales of the CAP dimension. Agency, TFP, and Importance Reappraisal had relationships that were small to medium in effect size. These observations indicate that students who have favorable cognitions regarding their ability to solve problems that arise during exams and that tests are congruent with their academic goals report less test anxiety. In contrast, students who report higher levels of EFP report higher levels of test anxiety.

Students who reported higher levels of self-efficacy for learning also reported higher levels of Testing Problem-Efficacy, Goal Congruence, TFP, and lower EFP and test anxiety. In addition, students who reported higher levels of metacognitive self-regulation reported higher TFP, Testing Problem-Efficacy, and self-efficacy for learning. These relationships were lower in magnitude relative to those involving self-efficacy for learning, with the exception of the relationship between metacognitive self-regulation and TFP. These relationships affirm the importance of self-regulation of learning for helping students to engage in self-regulated processing during exams. One other relationship that should be noted is the one between Tension Reduction and TFP (r = .36), which is similar in direction and magnitude to the zero-order coefficients reported in three previous studies (Davis et al., 2008; DeCuir-Gunby, Aultman, & Schutz, 2009; Schutz et al., 2004). The meaning of this relationship is unclear; however, it may reflect processing efficiency in efforts to remain calm and then redirect attention to test processing.

Analysis of intercorrelations indicated that all relationships were statistically significant with the exception of Importance Reappraisal. Subscales of the CAP dimension and MSLQ and the TFP dimension showed negative relationships. Perception that one can perform well academically, that tests are congruent with attaining academic goals, that one is in control of the testing situation, and that one can solve any testing problem that arises were associated with decreases in test anxiety. Perceptions of control over the test situation, task focusing, and efforts to reduce test anxiety were negatively correlated with test anxiety, although the relationships were weak in terms of effect size (Cohen, 1992). Wishful Thinking and Self-Blame were associated with increases in test anxiety.

Relationships involving ERT subscales and metacognitive self-regulation were all positive and statistically significant with the exception of Self-Blame and Wishful Thinking. Two subscales with large effect sizes included testing problem-efficacy, goal congruence, and the TFP dimension. Students who are reflective of study strategy selection and who monitor comprehension view tests as consistent with their goals and believe that they can solve problems that arise during exams, and they are less likely to engage in Self-Blame and Wishful Thinking. This supports the notion that assessment of metacognitive self-regulation is relevant to understanding ER related to testing.

Relationships involving the ERT subscales and self-efficacy for learning were similar to those involving metacognitive self-regulation, although of slightly greater magnitude for the CAP and EFP subscales. Noteworthy are relationships with large effect sizes indicating that high levels of self-efficacy for learning are associated with favorable cognitions related to testing. Students with high levels of self-efficacy for learning are less likely to engage in Wishful Thinking and Self-Blame related to testing.

Hierarchical Multiple Regression

Preliminary analysis indicated that age and gender did not predict test anxiety (N = 211; two cases failed to report either age or gender); thus, age and gender were not included in subsequent regression analyses. Predictors for the regression analysis were entered in five steps (a) metacognitive self-regulation, (b) self-efficacy for learning, (c) CAP (Goal Congruence, Agency, Testing Problem-Efficacy), (d) TFP, and (e) EFP (Self-Blame and Wishful Thinking) and regaining task-focus processing (Tension Reduction and Importance Reappraisal). The criterion variable was test anxiety.

The rationale for the order of entry of predictors was as follows. In preparation for an exam, the progression is from self-regulatory mechanisms that are likely to be used in advance of an exam (metacognitive self-regulation and self-efficacy for learning), cognitions about test performance and the testing situation, task focusing, and finally, efforts to regulate emotion and regaining task focus. This also reflects a continuum of opportunities to identify at what phase of preparing for and taking an exam self-regulation is compromised. Such strategic deployment of strategies may be beneficial in helping students judge whether preparation for an upcoming exam is adequate, therefore, reducing levels of anxiety. For the analysis, skew and kurtosis coefficients were within acceptable limits. There was no evidence of multicollinearity (tolerance values ranged from .31 to .82, M = 0.54; variance inflation factor (VIF) values ranged from 1.22 to 3.24, M = 2.03).

The results of the regression analysis are displayed in Table 3. Results indicated statistically significant increases in R² with entry of the following predictors: metacognitive self-regulation (p < .001), self-efficacy for learning (p < .001), two facets of CAP (Goal Congruence and Testing Problem-Efficacy, p < .001 and p < .05, respectively), and both facets of EFP (Wishful Thinking and Self-Blame, both p < .001). Agency, TFP, Importance Reappraisal, and Tension Reduction did not have statistically significant beta values (p > .05). The final model resulted in R² = .60, F(10, 212) = 30.28, p < .001. Regression diagnostics did not indicate violations of assumptions or the presence of outliers. ERT facet-level prediction of test anxiety yielded an equation that predicts 60% of the variance of test anxiety.

Table 3.

Hierarchical Multiple Regression Analysis to Predict Test Anxiety.

Step	Predictors	R ²	ΔR²	β
1	Metacognitive self-regulation	.13***	—	−.35***
2	Self-efficacy for learning	.31***	.18***	−.51***
3	CAPs	.48***	.17***
	Agency			−.03^†
	Goal Congruence			−.47***
	Testing Problem-Efficacy			−.17*
4	TFP	.49***	.01^†	.05^†
5	Emotion-Focusing Processes and Regaining TFP	.60***	.11***
	Wishful Thinking			.29***
	Self-Blame			.33***
	Importance Reappraisal			.00^†
	Tension Reduction			.01^†

Note. TFP = Task-Focusing Processes; CAP = Cognitive Appraising Process.

†

p > .05. *p ≤ .05. **p ≤ .01. ***p ≤ .001.

Discussion

Results of the present study indicate that two dimensions of the ERT have an acceptable structure with subscales that demonstrate incremental criterion validity for predicting test anxiety. After accounting for self-efficacy for learning and metacognitive self-regulation, the addition of two subscales of the CAP dimension, Goal Congruence and Testing Problem-Efficacy, resulted in a significant increment in R². A significant increment in R² was also observed with the addition of both subscales of the EFP dimension. At present, the ERT subscales of the EFP and CAP dimensions hold most promise for investigating ER related to testing. Although the subtest of Agency had a relatively high mean rating, it was not predictive of test anxiety. Given cognitions about whether tests impede progress in attaining academic goals and whether problems that arise during exams can be handled (efficacy), sense of personal responsibility for test performance may not predict well due to some degree of redundancy.

Our results are similar to those of DeCuir-Gunby et al. (2009) in terms of relationships between test anxiety and the ERT subscales. Zero-order relationships involving subscales of the CAP and EFP dimensions showed small differences (ranged from .01 to .10 for three subscales), with the exception of Agency (.22) and Self-Blame (.13). Zero-order relationships involving Importance Reappraisal, Tension Reduction, and the TFP dimension observed by DeCuir-Gunby et al. (2009) were −.01, −.07, and −.17, respectively, in contrast to our −.05, −.13, and −.25, respectively. In both studies, the addition of the CAP and EFP subscales resulted in substantial increments in R², and the magnitudes of the beta values were relatively small for the RTFP subscales and the TFP dimension. In structural modeling, Schutz et al. (2008) observed much larger path coefficients for the CAP and EFP dimensions for direct effects on unpleasant emotions than those of the TFP and RTFP dimensions. Results of the Schutz et al.’s (2008) CFA for the TFP and RTFP dimensions indicated only mediocre fit. Given evidence of poor to mediocre fit, weak relationships with test anxiety, and small beta values in the regression model, the utility of these dimensions may be questionable.

The TFP dimension has been modified from a five- to a six-item subscale (Schutz et al., 2008; Schutz et al., 2004), and four TFP items were used in the Davis et al. (2008) study. It has the lowest alpha coefficient among the four dimensions, which range from .51 to .64 across three samples. In addition to poor fit, our CFA factor loadings ranged from .25 to .64 (median = .35); thus, it is unclear as to the nature of the latent variable. Davis et al. (2008) suggested that “. . . task-focusing strategies refer not only to the deployment of attention but also to qualitatively different ways of processing information and solving problems” (p. 959). Some items of the dimension suggest effort to understand questions, and other items imply task focusing within a multiple-choice testing context (i.e., search for answers in other questions). Schutz et al. (2008) suggested that the TFP dimension requires additional development, and they indicated that task-focusing may depend on specific processing demands of a test. Given our observation that the range of inter-item correlations was less than optimal, and a possible two-factor structure, revision is warranted. Given an expanded framework that includes study strategies and metacognitive self-regulation, TFP may serve an important role in understanding the balance between self-regulation of task processing and ER.

Similarly, the RTFP subscales show minimal relationship with test anxiety. Item content of Importance Reappraisal reflects reappraising the significance of academic testing for determining course grades. We suspect that Importance Reappraisal may be less effective because students are unlikely to see exams as less important than other components of a course. Reappraising the importance of tests relative to the importance of other course components may be unlikely. Students will continue to see test performance as an important factor that influences course performance. However, a recent meta-analysis of ER experimental research indicated that reappraisal is one of the most effective ER strategies (Webb, Miles, & Sheeran, 2012). Effect sizes were greatest for reappraisal relative to those of other interventions (e.g., attentional deployment), and reappraisals that involved taking the perspective of a detached observer or reinterpreting the emotional stimulus or situation were most effective. Rather than reappraising the importance or significance of tests, reinterpreting the testing context in a more positive way may represent a more effective strategy for ER within the context of academic testing. If reappraisal is to remain within the RTFP dimension, then it should reflect evidence-based reappraisal, and item content would need to be revised accordingly.

Our results indicate the importance of including metacognitive and self-efficacy measures in regression models when predicting test anxiety. Given this expanded framework, the TFP dimension could be valuable for helping researchers to understand the relationship between test anxiety and task requirements of different course content and types of exams. Likewise, metacognitive self-regulation and self-efficacy for learning measures could be revised to be more situationally specific. For example, metacognitive self-regulation and self-efficacy measures could be revised to reflect task processing related to essay or multiple-choice exams, or reflect different content (e.g., math, biology, social science).

Implications for research include use of several constructs that fit well with the ER process model (Gross & Thompson, 2007). Given the vast empirical base from which to understand student experiences related to academic testing, the ERT Scale continues to have the potential for identifying effective strategies for regulating emotion related to testing. The addition of learning-strategy and metacognitive constructs should expand our understanding of how students are capable of maintaining attentional demands while regulating emotion. For example, if task-focusing functions to direct attention away from anxiety-related intrusions during an exam, then the effectiveness of task focusing should be evaluated in relation to possible mediating variables such as emotional working memory capacity (Shi, Gao, & Zhou, 2014) and central executive processes of inhibition (Derakshan & Eysenck, 2009; Miyake, Friedman, Emerson, Witzki, & Howerter, 2000). Future research is needed to determine the extent to which relationships involving test anxiety and self-regulation of emotion and learning are mediated by central executive processes, in particular, processes that allow students to remain task-focused when faced with possible threat-related distractions. The inhibitory function of the central executive is likely to play a prominent role (Derakshan & Eysenck, 2009).

Implications for counseling include the importance of assessing more than test anxiety and emotional regulation related to testing. For students who experience test anxiety, academic skill intervention combined with emotional regulation intervention may improve academic performance by improving both emotional and motivational self-regulation. Although study skills intervention may be effective for improving academic performance, interventions that are broader in scope, such as self-management interventions, may increase academic performance as well as retention (Robbins, Oh, Le, & Button, 2009). Such interventions are designed to help students to improve self-regulation of motivation, emotion, and learning (Robbins et al., 2009). Assessment of study strategy use, metacognitive self-regulation of study strategies, and ER strategies related to testing would allow academic-support personnel to identify whether test anxiety is related to motivation and effort, deficient use of self-regulation of study strategies for test preparation, or deficient use of ER. Such assessment provides personnel with insight as to the best possible intervention for a particular student.

Two limitations include the limited generalizability of our findings. Because our data were collected at a private Midwestern university, with a high percentage of women and first-year students, replication with representative diverse samples is needed to ensure generalizability. However, given that the sample was similar to that of DeCuir-Gunby et al. (2009) and Schutz et al. (2008), our results are similar in terms of direction and magnitude of correlational relationships, and beta and R² values of the regression model. Second, common method variance may have inflated the magnitude of observed relationships. However, some relationships were weak; thus, if inflation occurred, it was not consistent across all measures. In addition, differences in anchors and number of anchors differ between the ERT Scale and MSLQ, thus minimizing two possible sources of common method variance when data are collected from the same raters during one session. Future efforts should separate data collection in which the ERT Scale, measures of self-regulated learning, and test anxiety are administered on different occasions.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Ronald Feldt, PhD, is a professor of psychology at Mount Mercy University. Current research interests include career indecision, emotion regulation, self-regulation of learning, college adjustment, and stress in college students.

Kyla Lindley, BS, majored in psychology at Mount Mercy University. Her research experience includes emotion regulation and performance anxiety. She plans to enter graduate school to study industrial/organizational psychology.

Rebecca Louison is currently completing requirements for a BS degree with a major in psychology at Mount Mercy University. Her research experience includes emotion regulation, test anxiety, causal attributions regarding exam performance, and big five personality facets and career development.

Allison Roe, BS, majored in biology at Mount Mercy University. She is currently in medical school at Des Moines University in Iowa. Her research experience includes emotion regulation, test anxiety, and biomedical research.

Megan Timm, BS, majored in biology at Mount Mercy University. She is currently in dental school at the University of Iowa. Her research experience includes emotion regulation and test anxiety.

Nikola Utinkova is currently studying English philology/journalism at Palacký University, Olomouc, Czech Republic. Her research interests include bilingualism, emotion regulation, test anxiety, and big five personality facets and career development.

References

Austin

E. J.

Saklofske

D. H.

Mastoras

S. M.

(2010). Emotional intelligence, coping and exam-related stress in Canadian undergraduate students. Australian Journal of Psychology, 62, 42-50. doi:10.1080/00049530903312899

Bjork

R. A.

Dunlosky

Kornell

(2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64, 417-444. doi:10.1146/annurev-psych-113011-143823

Brown

S. D.

Nelson

T. L.

(1983). Beyond the uniformity myth: A comparison of academically successful and unsuccessful test-anxious college students. Journal of Counseling Psychology, 30, 367-374. doi:10.1037/0022-0167.30.3.367

Browne

M. W.

Cudeck

(1993). Alternative ways of assessing model fit. In Bollen

K. A.

Long

J. S.

(Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: SAGE.

Clark

Watson

(1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309-319. doi:10.1037/1040-3590.7.3.309

Cohen

(1992). A power primer. Psychological Bulletin, 112, 155-159. doi:10.1037/0033-2909.112.1.155

Covington

M. V.

Omelich

C. L.

(1987). “I knew it cold before the exam”: A test of the anxiety-blockage hypothesis. Journal of Educational Psychology, 79, 393-400. doi:10.1037/0022-0663.79.4.393

Credé

Phillips

(2011). A meta-analytic review of the Motivated Strategies for Learning Questionnaire. Learning and Individual Differences, 21, 337-346. doi:10.1016/j.lindif.2011.03.002

Davis

H. A.

DiStefano

Schutz

P. A.

(2008). Identifying patterns of appraising tests in first-year college students: Implications for anxiety and emotion regulation during test taking. Journal of Educational Psychology, 100, 942-960. doi:10.1037/a0013096

10.

DeCuir-Gunby

J. T.

Aultman

Schutz

P. A.

(2009). Investigating transactions among motives, emotional regulation related to testing, and test emotions. Journal of Experimental Education, 77, 409-436. doi:10.3200/JEXE.77.4.409-438

11.

Derakshan

Eysenck

M. W.

(2009). Anxiety, processing efficiency, and cognitive performance: New developments from attentional control theory. European Psychologist, 14, 168-176. doi:10.1027/1016-9040.14.2.168

12.

DeSteno

Gross

J. J.

Kubzansky

(2013). Affective science and health: The importance of emotion and emotion regulation. Health Psychology, 32, 474-486. doi:10.1037/a0030259

13.

Folkman

Lazarus

R. S.

(1985). If it changes it must be a process: Study of emotion and coping during three stages of a college examination. Journal of Personality and Social Psychology, 48, 150-170. doi:10.1037/0022-3514.48.1.150

14.

Gross

J. J.

(2013). Emotion regulation: Taking stock and moving forward. Emotion, 13, 359-365. doi:10.1037/a0032135

15.

Gross

J. J.

Thompson

R. A.

(2007). Emotion regulation: Conceptual foundations. In Gross

J. J.

(Ed.), Handbook of emotion regulation (pp. 3-24). New York, NY: Guilford.

16.

Harms

P. D.

Credé

(2010). Remaining issues in emotional intelligence research: Construct overlap, method artifacts, and lack of incremental validity. Industrial and Organizational Psychology: Perspectives on Science and Practice, 3, 154-158. doi:10.1111/j.1754-9434.2010.01217.x

17.

Hayton

J. C.

Allen

D. G.

Scarpello

(2004). Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational Research Methods, 7, 191-205. doi:10.1177/1094428104263675

18.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. doi:10.1080/10705519909540118

19.

Lowe

P. A.

(2015). Should test anxiety be measured differently for males and females? Examination of measurement bias across gender on measures of test anxiety for middle and high school, and college students. Journal of Psychoeducational Assessment, 0734282914549428, first published on September 17, 2014.

20.

McCabe

(2011). Metacognitive awareness of learning strategies in undergraduates. Memory & Cognition, 39, 462-476. doi:10.3758/s13421-010-0035-2

21.

McCrae

R. R.

Kurtz

J. E.

Yamagata

Terracciano

(2011). Internal consistency, retest reliability, and their implications for personality scale validity. Personality and Social Psychology Review, 15, 28-50. doi:10.1177/1088868310366253

22.

Miyake

Friedman

N. P.

Emerson

M. J.

Witzki

A. H.

Howerter

(2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41, 49-100. doi:10.1006/cogp.1999.0734

23.

Naveh-Benjamin

McKeachie

W. J.

Lin

(1987). Two types of test-anxious students: Support for an information processing model. Journal of Educational Psychology, 79, 131-136. doi:10.1037/0022-0663.79.2.131

24.

Nelson

J. M.

Lindstrom

Foels

P. A.

(2014). Test anxiety and college students with attention deficit hyperactivity disorder. Journal of Psychoeducational Assessment, 32, 548-557. doi:10.1177/0734282914521978

25.

O’Connor

M. C.

Paunonen

S. V.

(2007). Big five personality predictors of post-secondary academic performance. Personality and Individual Differences, 43, 971-990. doi:10.1016/j.paid.2007.03.017

26.

Paunonen

S. V.

Ashton

M. C.

(2001). Big five factors and facets and the prediction of behavior. Journal of Personality and Social Psychology, 81, 524-539. doi:10.1037/0022-3514.81.3.524

27.

Petrides

K. V.

Pérez-González

Furnham

(2007). On the criterion and incremental validity of trait emotional intelligence. Cognition and Emotion, 21, 26-55. doi:10.1080/02699930601038912

28.

Pintrich

P. R.

(2004). A conceptual framework for assessing motivation and self-regulated learning in college students. Educational Psychology Review, 16, 385-407.

29.

Pintrich

P. R.

Smith

D. A.

Garcia

McKeachie

W. J.

(1991). The Motivated Strategies for Learning Questionnaire. Ann Arbor: University of Michigan.

30.

Pintrich

P. R.

Smith

D. A.

Garcia

McKeachie

W. J.

(1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53, 801-813. doi:10.1177/0013164493053003024

31.

Richardson

Abraham

Bond

(2012). Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138, 353-387. doi:10.1037/a0026838

32.

Robbins

S. B.

Allen

Casillas

Peterson

(2006). Unraveling the differential effects of motivational and skills, social, and self-management measures from traditional predictors of college outcomes. Journal of Educational Psychology, 98, 598-616. doi:10.1037/0022-0663.98.3.598

33.

Robbins

S. B.

Lauver

Davis

Langley

Carlstrom

(2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130, 261-288. doi:10.1037/0033-2909.130.2.261

34.

Robbins

S. B.

Button

(2009). Intervention effects on college performance and retention as mediated by motivational, emotional, and social control factors: Integrated meta-analytic path analyses. Journal of Applied Psychology, 94, 1163-1184. doi:10.1037/a0015738

35.

Schutz

P. A.

Benson

DeCuir-Gunby

J. T.

(2008). Approach/avoidance motives, test emotions, and emotional regulation related to testing. Anxiety, Stress, & Coping: An International Journal, 21, 263-281. doi:10.1080/10615800701787672

36.

Schutz

P. A.

DiStefano

Benson

Davis

H. A.

(2004). The emotional regulation during test-taking scale. Anxiety, Stress, & Coping: An International Journal, 17, 253-269. doi:10.1080/1061580041000171086

37.

Shi

Gao

Zhou

(2014). Emotional working memory capacity in test anxiety. Learning and Individual Differences, 32, 178-183. doi:10.1016/j.lindif.2014.03.011

38.

Smith

G. T.

Fischer

Fister

S. M.

(2003). Incremental validity principles in test construction. Psychological Assessment, 15, 467-477. doi:10.1037/1040-3590.15.4.467

39.

Thompson

Daniel

L. G.

(1996). Factor analytic evidence for the construct validity of scores: A historical overview and some guidelines. Educational and Psychological Measurement, 56, 197-208. doi:10.1177/0013164496056002001

40.

Webb

T. L.

Miles

Sheeran

(2012). Dealing with feeling: A meta-analysis of the effectiveness of strategies derived from the process model of emotion regulation. Psychological Bulletin, 138, 775-808. doi:10.1037/a0027600

41.

Zeidner

(2007). Test anxiety in educational contexts: Concepts, findings, and future directions. In Schutz

P. A.

Pekrun

(Eds.), Emotion in education (pp. 165-184). San Diego, CA: Elsevier Academic Press.