Measuring Test Anxiety With an Invariant Measure Across Genders: The Case of the German Test Anxiety Inventory

Abstract

Since test performance is increasingly relevant in educational and occupational circles, the assessment of test anxiety—the phenomenological, physiological, and behavioral responses to the negative consequences that often emerge in evaluative situations—has become increasingly important to scholars and practitioners. One of the most widely employed scales to measure test anxiety in adolescents is the German Test Anxiety Inventory (in German: Prufungsangstfragebogen, PAF). The current study investigated the psychometric properties of the PAF when administered to Italian students. Our research found evidence of validity, supported the five-factor structure, and demonstrated the test’s good internal consistency. Moreover, the invariance of the dimensional structure across genders was examined. Overall, this study provides evidence for the reliability and validity of the PAF among Italian students.

Keywords

Test anxiety PAF gender invariance reliability validity

Introduction

Test anxiety, which is also known as evaluative anxiety or performance anxiety, “refers to the set of phenomenological, physiological and behavioural responses that accompany concern about possible negative consequences or failure of an exam or a similar evaluative situation” (Sieber, O’Neil, & Tobias, 1977, p. 23). Although test anxiety has been treated as one of the specifiers of a generalized anxiety disorder in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 2013), several researchers (Blöte, Kint, Miers, & Westenberg, 2009; Hofmann, Heinrichs, & Moscovitch, 2004; Hook & Valentiner, 2002) have claimed that it is qualitatively distinct from other types of anxiety disorders. Indeed, this kind of anxiety seems to have a phobic quality, greater than the features of general anxiety or other types of social anxiety disorders.

Test anxiety is a situation-specific anxiogenic response that derives from trait anxiety and affects evaluative situations (Spielberger & Vagg, 1995; Zeidner, 2007). Test anxious people perceive performance evaluations as threatening in nature, which in turn leads them to respond with heightened state anxiety (Ringeisen, Raufelder, Schnell, & Rohrmann, 2016). Considering that tests and other performance indicators are frequently used in both educational and occupational contexts (Roediger & Karpicke, 2006), test anxiety is seen as a serious issue among scholars and practitioners alike. Indeed, test anxiety is a relatively widespread condition, as research has found that between 14% and 50% of college students (Cizek & Burg, 2006), and more than 33% of school-age children and adolescents, experience some degree of test anxiety (Methia, 2004).

Gender difference is an important factor to consider, as female students across all age groups report higher levels of test anxiety than male students (Chapell et al., 2005; Goetz, Bieg, Lüdtke, Pekrun, & Hall, 2013; Hodapp, Rohrmann, & Ringeisen, 2011; A. S. McDonald, 2001; Ringeisen et al., 2016; Schnell, Tibubos, Rohrmann, & Hodapp, 2013; Wren & Benson, 2004; Zeidner, 2007). For instance, in their work on noncognitive predictors of grades among freshman students, Fischer, Schult, and Hell (2013) found that females showed significantly higher test anxiety than males, and this difference displayed an effect size of .36. Chin, Williams, Taylor, and Harvey (2017) similarly found that female high school students showed higher levels of test anxiety than boys. Finally, even in a sample involving younger children (average age of 10 years), girls reported higher levels of test anxiety than boys (Lohbeck, Nitkowski, & Petermann, 2016); this difference had an effect size of .32.

Test anxiety can negatively affect education, as test-anxious students “do not approach a task such as a test with a positive outlook or expectation of success, but with dread regarding the potential for negative evaluation or failure” (Cizek & Burg, 2006, p. 17), and often feel tense and worried in evaluative situations (Gierl & Rogers, 1996). For this reason, it is very likely that students with high levels of test anxiety exhibit diminished or poor performance (Cassady & Johnson, 2002; Chapell et al., 2005; Segool, Carlson, Goforth, von der Embse, & Barterian, 2013) and, in some cases, school attrition (Cizek & Burg, 2006). Several models have focused on the impact of test anxiety in performance evaluations. For instance, according to supporters of control-value theory (Pekrun, 2006; Pekrun, Frenzel, Goetz, & Perry, 2007), low dispositional control beliefs in performance situations and high personal relevance for academic success increase the anticipated risk of failure via test anxiety. Meanwhile, proponents of self-regulation theory (self-regulation theory (SRT); Schwarzer, 2001) postulate that low levels of self-efficacy can negatively affect performance through goal setting and goal significance, initiation and perseverance of learning, and test anxiety. Furthermore, research based on the tripartite model of emotions (TME; Chin et al., 2017) suggests that a general tendency to experience negative affect and physiological hyperarousal often predicts higher levels of test anxiety in adolescents, while positive affect did not influence test anxiety. Moreover, TME shows that negative emotions have a detrimental effect on educational outcomes through test anxiety. Thus, test anxiety occupies a central role in mediating the interplay between dispositional characteristics and performance (Schnell, Ringeisen, Raufelder, & Rohrmann, 2015).

Obviously, in order to more precisely understand the learning and performance capacities of students and screen youths with high levels of test anxiety, it is necessary to have access to valid and reliable measures. For this reason, different measures have been developed to assess test anxiety among high school and college students. In line with a two-dimensional conceptualization of test anxiety into a cognitive and an emotional declination (Liebert & Morris, 1967; Schwarzer, 1984; Wacker, Jaunzeme, & Jaksztat, 2008; Ware, Galassi, & Dew, 1990), Spielberger (1980) developed the Test Anxiety Inventory (TAI), which regards test anxiety as a construct defined by a cognitive component (worry—thoughts of failure regarding performance) and an affective component (emotionality—autonomous excitation and emotional reaction toward the evaluative moment). Subsequently, Sarason (1984) developed a new version of the TAI—the Reactions to Test Scale—composed of four dimensions that split the emotionality component into test and bodily symptoms and the cognitive component into worry and test-irrelevant thinking. In order to better investigate the different facets of test anxiety, Hodapp (1991) then revised this measure, creating a new German version of the scale—the TAI-G—for secondary school and college students. This scale was composed of 30 items and four subscales: Worry, which assesses concerns about individual performance and the consequences of failure; Emotionality, which examines emotional and physical tension; Interference, which measures distraction from the task via irrelevant thoughts; and Lack of Confidence, which evaluates the impact of confidence levels on mastering academic challenges. This new version was eventually shortened and revised, resulting in an easy-to-administer measure—the German Test Anxiety Inventory (in German: Prufungsangstfragebogen, PAF; Hodapp et al., 2011). It consists of 20 items, with five items per subscale. This test has been used predominantly in German studies that investigate test anxiety among adolescents (Hodapp et al., 2011; Ringeisen et al., 2016; Schnell et al., 2013) and young adults (Fischer et al., 2013; Reiss et al., 2017). Several other studies (Ringeisen et al., 2016; Schnell et al., 2013) have also found it to be a reliable and valid measure, and an English version for 11- to 16-year-old students has recently been published (PAF-E; Hoferichter, Raufelder, Ringeisen, Rohrmann, & Bukoski, 2016).

Internal consistency of the PAF has been found to be satisfactory, with Cronbach’s alphas that range from .71 to .85 (Hoferichter et al., 2016; Schnell et al., 2013). In terms of construct validity, an exploratory factor analysis and a confirmatory factor analysis (CFA) supported the four-factor model: Worry explained 26.51% of the total variance, Emotionality 17.51%, Interference 7.63%, and Lack of Confidence 5.60%. Regarding criterion validity, Hoferichter et al. (2016) found that the four scores associated with the PAF subscales were all significantly and positively correlated with educational self-efficacy, with the highest coefficient value found in the Lack of Confidence score. Again, scores pertaining to the Worry and Lack of Confidence subscales were significantly and negatively correlated with performance motivation, and significant and negative correlations between the PAF subscales and grades were found, particularly math grades. Consistent with these findings, Schnell et al.’s (2013) research on high school students found that the Interference and Lack of Confidence subscale scores were significantly negatively correlated with both math and language performance. Additionally, they found that all of the PAF subscale scores, with the exception of Worry, were significantly positively related to math anxiety. PAF subscores have been also found to be correlated with personality variables, including conscientiousness, neuroticism, and agreeableness. As for convergent validity, the four subscores associated with the PAF positively correlated with the inhibitory test anxiety measure (Hoferichter et al., 2016), with correlation coefficients of .59, .51, .30, and .18 with Worry, Emotionality, Interference, and Lack of Confidence subscale scores, respectively. Finally, as support for incremental validity, the total score of the PAF was a significant predictor, together with gender, of overall school performance in a regression model that also included gender and math anxiety (Schnell et al., 2013).

Despite its usefulness, few psychometric studies of the PAF have been published (Hoferichter et al., 2016; Schnell et al., 2013), and psychometric data remain underinvestigated. For instance, CFA of the factorial structure has been tested in only one study (Hoferichter et al., 2016), and measurement invariance—the ability of a test to uniformly measure a specific construct among different groups of respondents—has not been assessed. Measurement invariance represents a fundamental property of a test; if a test does not uniformly measure a construct among different groups of respondents, the comparison of test scores between different groups obviously suffers (Waiyavutti, Johnson, & Deary, 2012). By contrast, invariance testing ensures the fairness and thus the validity of a scale (Kane, 2013). Since gender-related differences in test anxiety have been found (Fischer et al., 2013; Hodapp et al., 2011; Ringeisen et al., 2016; Schnell et al., 2013), and the PAF has been used to examine gender invariance between various test anxiety predictors among adolescents (Ringeisen et al., 2016), it becomes necessary to verify measurement invariance across genders. Also, since test anxiety has been found to be an obstacle to successful performance among both preadolescents and adolescents (Chin, Williams, Taylor, & Harvey, 2017; Lohbeck et al., 2016), we deemed it necessary to analyze the psychometric properties of the PAF among youth who were between 11 and 16 years of age, which is in line with the English validation study (Hoferichter et al., 2016).

Following these premises, this study analyzed the psychometric properties of the PAF in an Italian context and tested the adequacy of the four-factor model (Hodapp, Rohrmann, & Ringeisen, 2011; Hoferichter et al., 2016). However, since Hoferichter et al. (2016) have shown that the covariance between Worry and Lack of Confidence (r = .02) was not significant, and the coefficient value of the covariation between Worry and Interference was .08, we hypothesized that both Worry and Lack of Confidence and Worry and Interference were unrelated dimensions. We were also interested in studying gender invariance via a multigroup confirmatory analysis in order to determine whether the detected factors were replicable in boys and girls.

We also examined the internal consistency of the subscales by calculating Cronbach’s alpha (α) coefficients (Cronbach & Shavelson, 2004), all of which have been found to be satisfactory in previous studies (Hoferichter et al., 2016; Schnell et al., 2013). Additionally, we analyzed the internal consistency by calculating R. P. McDonald’s (1999) omega (Ω), which corresponds to the ratio of true-score variance and total variance, calculated using the item’s coefficient on each factor.

We evaluated the criterion validity of the PAF by examining the correlation between test anxiety and math anxiety and by analyzing the relationship between the total and subscale scores at the Abbreviated Math Anxiety Scale (AMAS, Hopko, Mahadevan, Bare, & Hunt, 2003; Italian version: Primi, Busdraghi, Tomasetto, Morsanyi, & Chiesi, 2014). Test and math anxiety are increasingly recognized as two different, yet partially overlapping, constructs (Schnell et al., 2013). In fact, previous studies have found moderate to strong correlations between test anxiety and math anxiety among high school and college students (Ashcraft, 2002; Dew, Galassi, & Galassi, 1984; Morsanyi, Busdraghi, & Primi, 2014; Primi et al., 2014; Schnell et al., 2013). Additionally, we analyzed the relationship between math performance and test anxiety in order to find a significant negative association with Interference and Lack of Confidence (Schnell et al., 2013). To assess math performance, we used the AC-MT 11–14 standardized mathematics test (Cornoldi & Cazzola, 2004), which is designed for Italian middle school students and measures calculation procedures through written multidigit calculations. Finally, we investigated gender differences in the PAF by emulating previous studies that have found substantially higher levels of test anxiety among female students than male students (Chapell et al., 2005; Chin et al., 2017; Fischer et al., 2013; Goetz et al., 2013; Hodapp et al., 2011; Lohbeck et al., 2016; A. S. McDonald, 2001; Ringeisen et al., 2016; Schnell et al., 2013; Wren & Benson, 2004; Zeidner, 2007).

Methods

Participants

Three hundred and twenty-six youths participated in our study (mean age: 13.36 years, standard deviation (SD): 1.34, range: 11–16 years; 146 males: mean age: 13.34 years, SD: 1.37, range: 11–16 years; 180 females: mean age: 13.38 years, SD: 0.32, range: 11–16 years). All participants attended public secondary schools that were located in both the North-Centre (43%) and South (57%) of Italy. Four schools (two middle schools and two high schools) were randomly selected from each region. The principals were then contacted, and the researchers explained the goals of the study. Once the schools agreed to participate, a detailed study protocol, designed in accordance with the criteria of the Declaration of Helsinki, was approved by the institutional review boards at each school. Two schools—one middle school in the North-Centre of Italy and one high school in the South of Italy—declined to participate because they were already involved in other projects. Written informed consent was requested from the students’ parents, and they were assured that the data would be handled confidentially. The research was conducted during school hours and all students who were invited to participate agreed to do so, which means that sample bias that might have emerged during the recruitment stage was avoided.

Measures

The PAF (Hodapp et al., 2011) measures self-reported anxious emotions and thoughts during test situations. It consists of 20 items with a four-point response scale that ranges from 1 (hardly ever) to 4 (almost always). The four subscales are organized as follows: Lack of Confidence (L) consists of five reversed items (e.g., I trust in my performance), Worry (W) is composed of five items (e.g., I think about how important the test is to me), Emotionality (E) is assessed through five items (e.g., I feel anxious), and Interference (I) includes five items (e.g., Suddenly thoughts cross my mind which inhibit me). The Italian version of the PAF was created by adapting the English version of each item. A forward-translation method was used, whereby two nonprofessional translators worked independently and then compared their translations. A group of five people then read a provisional version of the new scale, revised it, and settled on a final version. In line with previous studies (Reiss et al., 2017; Schnell et al., 2013; Tempel & Neumann, 2016), also a total PAF score was calculated as a measure of test anxiety.

The AMAS (Hopko et al., 2003; Italian version: Primi et al., 2014) is a two-factor measure of math anxiety that assesses Learning Math Anxiety (LMA; five items) and Math Evaluation Anxiety (EMA; four items). A five-point response scale is used, ranging from 1 (low anxiety) to 5 (high anxiety). A sample item is Thinking about an upcoming math test one day before. High scores indicate high levels of math anxiety. In line with previous studies (Cipora, Szczygiel, Willmes, & Nuerk, 2015; Morsanyi et al., 2014; Primi et al., 2014), the two subscale scores and a single composite score were calculated based on how participants rated each statement. Previous studies have found support for its validity and reliability. Concurrent validity was supported by showing good associations between the AMAS subscales and total score, test anxiety, and math attitude (Primi et al., 2014). Convergent and divergent validity were provided in the original study of the AMAS by stressing the strong associations with the Math Anxiety Rating Scale-Revised (Plake & Parker, 1982) and moderate associations with other anxiety measures, including general anxiety, state and trait anxiety, and test anxiety (Hopko et al., 2003). Good internal consistency values were found in previous studies (Hopko et al., 2003; Primi et al., 2014), and the test–retest reliability was found to be excellent (Hopko et al., 2003). Cronbach’s alpha coefficients in the present study were .76 for the LMA subscale, .75 for the EMA subscale, and .79 for the total scale.

In order to assess math performance, the AC-MT 11–14 standardized mathematics test (Cornoldi & Cazzola, 2004) was used. This test assesses calculation procedures and number comprehension by means of a set of paper-and-pencil tasks that can be grouped into two areas: “written calculation” and “number knowledge.” Our research used calculation tasks whereby participants were asked to solve eight written multidigit calculations, including two additions, two subtractions, two multiplications, and two divisions. Total scores were obtained by adding up correct responses. Good reliability of this measure was found in previous studies both in terms of test–retest stability (Caviola, Primi, Chiesi, & Mammarella, 2017) and internal consistency (Cornoldi & Cazzola, 2004). Moreover, adequate concurrent validity (Cornoldi & Cazzola, 2004) and discriminant validity (Passolunghi, Caviola, De Agostini, Perin, & Mammarella, 2016) were confirmed.

Procedure

The measures were administered individually during class time. All participants completed the PAF. In order to analyze criterion validity, a subsample of 138 students (68 males and 70 females, mean age: 13.06 years, SD: 0.62) also completed the AMAS and the AC-MT. The order of administration of the scales was randomized. Participants were provided with a brief introduction to the study and instructions on how to proceed. Answers were collected in a paper-and-pencil format. Data collection was completed in approximately 20 minutes for the PAF, and 40 minutes for the PAF, AMAS, and AC-MT combined.

Statistical analyses

A missing value analysis was performed during the early stages of the data analysis phase. Missing data were not allowed to exceed 10% of the total cases in the sample (Kline, 1998). Indeed, when missing data exceeded 10%, we decided to exclude the case. In total, two cases were excluded, which means that the data of 324 participants were ultimately analyzed. When missing data were deemed acceptable, the score of each omitted item was treated as being equal to the mean of its subscale.

Univariate distributions of PAF items were then examined in order to assess normality, placing special emphasis on skewness and kurtosis indices. Subsequently, the four-factor structure was tested by a CFA that employed the maximum likelihood estimator (AMOS software; Arbuckle, 2003). To verify the model’s fit, the following indices were taken into account: The ratio of chi-square to its degrees of freedom (χ²/df), the comparative fit index (CFI; Bentler, 1990), the Tucker–Lewis index (TLI; Tucker & Lewis, 1973), and the root mean square error of approximation (RMSEA; Steiger & Lind, 1980). In the case of χ²/df, values below or equal to 2 are considered good, while values between 2 and 3 are considered acceptable (Schermelleh-Engel, Moosbrugger, & Müller, 2003). For the TLI and CFI, values above .90 are indicative of acceptable fit, while values above .95 are indicative of excellent fit (Hu & Bentler, 1999). The RMSEA value is considered acceptable when it is below .08 and good when it is below .05 (Kline, 2010).

Gender invariance analyses were conducted by performing hierarchically nested CFAs (see Byrne, 2004, for testing multigroup invariance with AMOS). Invariance was evaluated using not only Δχ², which is sensitive to sample size, but also ΔCFI, which has been found to be the most sensitive index to detect a lack of invariance (Meade, Johnson, & Braddy, 2008). An absolute value of ΔCFI of less than .01 was used (Cheung & Rensvold, 2002; Dimitrov, 2010).

In order to assess the reliability of the PAF, Cronbach’s alphas and McDonald’s omega values were obtained for the various subscales. To examine validity, correlation coefficients (Pearson’s r) were calculated with the AMAS subscale/total score and the AC-MT score. After testing for gender invariance, we compared the PAF subscale scores of male and female students through an independent samples t test, adopting a hypothesis stipulating that gender difference was evidence of discriminant validity.

Results

Dimensionality

We looked at the PAF items to assess normality. Skewness and kurtosis indices were between −1 and +1 for all but three items, which were slightly outside the range of normality (Table 1). However, this type of deviation is often considered to be negligible (Ghasemi & Zahediasl, 2012).

Table 1.

Means, SDs, skewness, and kurtosis of the 20 items of the PAF.

Item	M	SD	Skewness	Kurtosis
1	2.34	0.83	−0.08	−0.70
2	2.94	0.86	−0.40	−0.58
3	1.92	0.79	0.60	−0.05
4	2.30	0.86	0.15	−0.65
5	1.90	0.85	0.80	0.16
6	1.65	0.81	1.15	0.74
7	3.23	0.80	−0.77	−0.07
8	2.12	1.05	0.44	−1.07
9	2.95	0.90	−0.43	−0.71
10	2.51	1.03	0.14	−1.15
11	1.68	0.91	1.31	0.84
12	2.30	0.94	0.16	−0.88
13	2.50	1.01	0.03	−1.08
14	1.61	0.93	1.40	0.85
15	1.73	0.78	0.89	0.32
16	1.59	0.84	1.45	1.42
17	2.12	0.88	0.37	−0.62
18	2.68	1.02	−0.17	−1.12
19	1.70	0.88	1.12	0.40
20	2.39	0.84	0.14	−0.61

Note: The PAF uses the following Likert-type scale: 1 = hardly ever, 2 = sometimes, 3 = often, and 4 = nearly always. n = 324. SD: standard deviation.

A CFA was carried out by testing the PAF’s four-factor model. The results showed that the fit indices were not acceptable. Modification indices suggested adding error covariance between items 1 and 4 of the Lack of Confidence subscale, items 7 and 13 of the Worry subscale, items 8 and 10 of the Emotionality subscale, and items 3 and 15 of the Interference subscale. These links were theoretically justified because linked items reported similar contents (Byrne, 2004). The modified model showed a good fit (χ² = 304.47, df = 162, p < .001, χ²/df = 1.88, CFI = .93, RMSEA = .05, 90% confidence interval (CI) = .04–.06). Each item loaded strongly and significantly on its hypothesized factor, and the hypothesized correlations between the factors were all significant ranging from .32 to .50 (Figure 1).

Figure 1.

The four-factor model of the PAF in the global sample (n = 324). Standardized parameters were all significant at .001. L: lack of confidence; W: worry; E: emotionality; I = interference; PAF: German Test Anxiety Inventory.

Gender invariance

In order to test gender invariance according to recommended standards (Dimitrov, 2010; Little, 1997; Vandenberg & Lance, 2000), the independence model was fitted (χ² = 2493.73, df = 380, p < .001). As Table 2 illustrates, weak or metric factorial invariance and configural invariance was supported, thus confirming that the factor loadings were equal across genders. The factors’ variances and covariances were also found to be invariant. Finally, the equality of the items’ variances and covariances was also tested. We obtained a nonsignificant Δχ², and the differences in CFI values were less than .01. Thus, the results supported gender invariance.

Table 2.

Goodness-of-fit statistics for each level of structural and measurement invariance across genders.

Model	χ² (df)	χ²/df	p	CFI	RMSEA (90% CI)	Model comparison	Δχ²	Δdf	p	ΔCFI
1. Invariance of model configuration	631.40 (324)	1.95	<.001	.855	.05 (.05–.06)	—	—	—	—	—
2. Invariance of factor loadings	648.99 (340)	1.91	<.001	.854	.05 (.05–.06)	Model 1–Model 2	17.59	16	.349	.001
3. Invariance of structural variances/covariances	658.77 (348)	1.89	<.001	.853	.05 (.05–.06)	Model 2–Model 3	9.79	8	.280	.001
5. Invariance of measurement error variances/covariances	689.66 (368)	1.87	<.001	.848	.05 (.05–.06)	Model 3–Model 4	30.89	20	.057	.005

Note: χ²: chi-square test; df: degrees of freedom; CFI: robust comparative fit index; RMSEA: robust root mean square error of approximation; Δχ²: Satorra–Bentler scaled difference; Δdf: difference in degrees of freedom between nested models; p: probability value of Δχ² test; ΔCFI: difference between robust CFIs of nested models. n = 324.

Reliability

Cronbach’s alphas showed acceptable internal consistency, producing values equal to .83 (95% CI = .80–.86) for the Lack of Confidence subscale, .72 (95% CI = .69–.77) for the Worry subscale, .79 (95% CI [.76, .83]) for the Emotionality subscale, and .81 (95% CI = .77–.84) for the Interference subscale. These values did not increase when an item was deleted, and all item-corrected total correlations were above .44. Following the standards proposed by the European Federation of Psychologists’ Association (EFPA; Evers et al., 2013), the internal consistency value was good for the Worry and Emotionality subscales and excellent for the Lack of Confidence and Interference subscales.

McDonald’s Ω scores were found to be .86 for the Lack of Confidence subscale, .75 for the Worry subscale, .85 for the Emotionality subscale, and .80 for the Interference subscale. According to standards proposed by the EFPA (Evers et al., 2013), the internal consistency value was good for the Lack of Confidence, Emotionality, and Interference subscales and adequate for the Worry subscale.

Validity

To test the validity, the relationship between the PAF subscales and total score, the AMAS subscales and total score, and the AC-MT total score was assessed (Table 3). The results showed that all of the PAF subscales and the PAF total score were significantly and positively correlated to both AMAS subscales and its total score. Additionally, significant negative relationships were found between math performance, the PAF total score, and the Lack of Confidence and Interference subscale scores. The values of the correlations were either adequate (ranging from .20 to .34), good (ranging from .35 to .49), or excellent (≥.50), all of which support the criterion validity of the PAF, according to EFPA standards (Evers et al., 2013).

Table 3.

Pearson correlations between the PAF subscale scores, the AMAS subscale scores and total score, and the AC-MT score.

PAF	LMA subscale score	EMA subscale score	AMAS total score	AC-MT score	M (SD)
L subscale score	.29***	.27***	.32***	−.20*	12.12 (3.45)
W subscale score	.24**	.40***	.37***	−.04	13.42 (3.35)
E subscale score	.28***	.52***	.47***	−.10	8.85 (3.57)
I subscale score	.32***	.25**	.32***	−.31***	8.71 (3.01)
PAF total score	.43***	.55***	.56***	−.24**	43.09 (8.89)
M (SD)	7.25 (3.05)	12.22 (3.94)	19.46 (6.19)	4.87 (2.51)

Note: L: lack of confidence; W: worry; E: emotionality; I: interference; PAF: German Test Anxiety Inventory; LMA: Learning Math Anxiety; EMA: Math Evaluation Anxiety; AMAS: Abbreviated Math Anxiety Scale; AC-MT: AC-MT 11–14 standardized mathematics test. n = 138; SD: standard deviation.

**p < .01; ***p < .001.

Gender differences

After having verified the measurement equivalence of the scale among both male and female respondents, a comparison by gender was conducted on the PAF subscale scores. No significant differences were found with respect to the Lack of Confidence (t(322) = −.06, p = .950, Cohen’s d = .01) and the Interference (t(322) = 1.57, p = .118, Cohen’s d = .18) subscales. Gender differences were found in the Emotionality subscale (t(322) = −4.14, p < .001), as female students (M: 10.25; SD: 3.67) displayed higher levels of anxiety than male students (M: 8.66; SD: 3.13). This difference had a moderate effect size (Cohen’s d = .47). Additionally, a small (Cohen’s d = .23) difference was found in the Worry dimension (t(322) = −2.10, p = .036), as girls (M: 14.62; SD: 3.12) displayed significantly higher scores than boys (M: 13.88; SD: 3.20).

Discussion

Performance anxiety has been recognized as a variable that can negatively affect not only isolated performance but also broader educational processes (Cassady & Johnson, 2002; Chapell et al., 2005; Cizek & Burg, 2006; Segool et al., 2013). Thus, measuring test anxiety in students should be seen as an important issue. For this reason, this study investigated the psychometric properties of the Italian translation of the PAF (Hodapp et al., 2011), a self-report scale that measures anxious emotions and thoughts during test situations, treating lack of confidence, worry, emotionality, and interference as key factors.

Our research suggests that the Italian version of the PAF displays characteristics comparable to those reported in previous studies involving adolescents (Hoferichter et al., 2016; Schnell et al., 2013). Our factor analysis, in particular, supported the four-factor structure of the scale, and good internal consistency was found for the various subscales. Whereas previous studies have not taken into account measurement invariance, this study found that the four-factor structure was invariant across genders. This finding legitimates the use of the PAF to compare male and female students and analyze gender-related differences in test anxiety and relative predictors.

We also found evidence of the validity of the PAF in the association between test anxiety, math anxiety (Ashcraft, 2002; Dew et al., 1984; Morsanyi et al., 2014; Primi et al., 2014; Schnell et al., 2013), and math performance (Hoferichter et al., 2016; Schnell et al., 2013). Additionally, we found support for gender-related differences in test anxiety, as our data suggest that female students were more anxious toward evaluative situations than were male students, a trend that was especially noticeable in relation to the Emotionality and Worry dimensions.

The PAF can help practitioners identify at-risk male and female adolescents who suffer from high levels of test anxiety and plan appropriate educational interventions and/or therapeutic treatments (see Ergene, 2003, for a meta-analysis on effective interventions to reduce test anxiety). Using previous research by Reiss et al. (2017) as a guide, we concluded that different strategies (i.e., relaxation techniques and imagery rescripting) should be employed in order to reduce test anxiety in students. For instance, reinforcing learning and motivation, investigating dysfunctional beliefs about performance and restructuring them into adaptive thoughts, teaching functional coping strategies, and providing training on how to identify and overcome blackout situations during testing situations could be done with the aim of reducing test anxiety. These activities could be conducted not only in a therapeutic setting but also in a classroom setting by psychologists and counselors. Another potential use of the PAF can be in the field of experimental studies in which there is the necessity of measuring test anxiety to select high-anxious individuals (Tempel & Neumann, 2016). Additionally, assessment of test anxiety is necessary to measure the effects of different type of didactic materials on test anxiety (e.g., Rockinson-Szapkiw, Wendt, & Lunde, 2013) and to evaluate the consequences of test anxiety on learning under different kinds of social conditions (e.g., Cassady, 2004). Finally, the PAF should be used as a self-report measure to control for test anxiety in studies that aim to better understand the effects of math anxiety on math performance and reasoning (Morsanyi et al., 2014).

Some limitations characterize this study. For instance, the participants were all secondary school students, representing a relatively homogeneous population in terms of age. Future studies with a more heterogeneous sample could be useful in detecting measurement invariance. Indeed, it might be useful to compare participants from different age groups who are at different stages of the educational process. Further analyses are also needed to investigate cross-cultural invariance of the instrument. Another limitation is that we used only math anxiety and math performance as criterion validity measures. A broader range of measures are needed in future investigations of validity, including constructs that are correlated with test anxiety—most notably performance motivation, self-efficacy, school grades, and personality dimensions such as conscientiousness, neuroticism, and agreeableness (Hoferichter et al., 2016). Despite these limitations, the PAF maintains good psychometric properties when used with Italian students, which means that it could be a useful instrument for measuring test anxiety in adolescents.

Footnotes

Appendix 1: Italian translation of the PAF scale

Article Notes

References

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.

Arbuckle

J. L.

(2003). Amos 5.0 [Computer software]. Chicago, IL: Smallwaters.

Ashcraft

M. H.

(2002). Math anxiety: Personal, educational and cognitive consequences. Current Direction in Psychological Science, 11, 181–185. doi:10.1111/1467-8721.00196

Bentler

P. M.

(1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246.

Blöte

A. W.

Kint

M. J.

Miers

A. C.

Westenberg

P. M.

(2009). The relation between public speaking anxiety and social anxiety: A review. Journal of Anxiety Disorders, 23(3), 305–313. doi:10.1016/j.janxdis.2008.11.007

Byrne

B. M.

(2004). Testing for multigroup invariance using AMOS graphics: A road less traveled. Structural Equation Modeling, 11(2), 272–300. doi:10.1207/s15328007sem1102_8

Cassady

J. C.

(2004). The influence of cognitive test anxiety across the learning-testing cycle. Learning and Instruction, 14(6), 569–592. doi:10.1016/j.learninstruc.2004.09.002

Cassady

J. C.

Johnson

R. E.

(2002). Cognitive test anxiety and academic performance. Contemporary Educational Psychology, 27(2), 270–295. doi:10.1006/ceps.2001.1094

Caviola

Primi

Chiesi

Mammarella

I. C.

(2017). Psychometric properties of the Abbreviated Math Anxiety Scale (AMAS) in Italian primary school children. Learning and Individual Differences, 55, 174–182. doi:10.1016/j.lindif.2017.03.006

10.

Chapell

M. S.

Blanding

Z. B.

Silverstein

M. E.

Takahashi

Newman

Gubi

McCann

(2005). Test anxiety and academic performance in undergraduate and graduate students. Journal Educational Psychology, 97(2), 268–274.

11.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. doi:10.1207/S15328007SEM0902_5

12.

Chin

E. C.

Williams

M. W.

Taylor

J. E.

Harvey

S. T.

(2017). The influence of negative affect on test anxiety and academic performance: An examination of the tripartite model of emotions. Learning and Individual Differences, 54, 1–8. doi:10.1016/j.lindif.2017.01.002

13.

Cipora

Szczygieł

Willmes

Nuerk

H. C.

(2015). Math anxiety assessment with the Abbreviated Math Anxiety Scale: Applicability and usefulness: Insights from the polish adaptation. Frontiers in Psychology, 6, 1–16. doi:10.3389/fpsyg.2015.01833

14.

Cizek

G. J.

Burg

S. S.

(2006). Addressing test anxiety in a high-stakes environment: Strategies for classrooms and schools. Thousand Oaks, CA: Corwin Press.

15.

Cornoldi

Cazzola

(2004). Test AC-MT 11-14 - Test di Valutazione delle Abilità di Calcolo e Problem Solving [AC-MT 11-14: Test for assessing calculation and problem solving skills]. Trento, Italy: Erickson.

16.

Cronbach

L. J.

Shavelson

R. J.

(2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64(3), 391–418. doi:10.1177/0013164404266386

17.

Dew

K. H.

Galassi

J. P.

Galassi

M. D.

(1984). Math anxiety: Relation with situational test anxiety, performance, physiological arousal, and math avoidance behavior. Journal of Counseling Psychology, 31(4), 580–583.

18.

Dimitrov

D. M.

(2010). Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 43(2), 121–149. doi:10.1177/0748175610373459

19.

Ergene

(2003). Effective interventions on test anxiety reduction: A meta-analysis. School Psychology International, 24(3), 313–328. doi:10.1177/01430343030243004

20.

Evers

Hagemeister

Høstmælingen

Lindley

Muñiz

Sjöberg

(2013, July). EFPA Review Model for the description and evaluation of psychological and educational tests. Test review form and notes for reviewers (Version 4.2.6). Retrieved from http://www.efpa.eu/download/650d0d4ecd407a51139ca44ee704fda4

21.

Fischer

Schult

Hell

(2013). Sex differences in secondary school success: Why female students perform better. European Journal of Psychology of Education, 28(2), 529–543. doi:10.1007/s10212-012-0127-4

22.

Ghasemi

Zahediasl

(2012). Normality tests for statistical analysis: A guide for non-statisticians. International Journal of Endocrinology and Metabolism, 10(2), 486–489. doi:10.5812/ijem.3505

23.

Gierl

M. J.

Rogers

W. T.

(1996). A confirmatory factor analysis of the Test Anxiety Inventory using Canadian high school students. Educational and Psychological Measurement, 56(2), 315–324. doi:10.1177/0013164496056002012

24.

Goetz

Bieg

Lüdtke

Pekrun

Hall

N. C.

(2013). Do girls really experience more anxiety in mathematics? Psychological Science, 24(10), 2079–2087. doi:10.1177/0956797613486989

25.

Hodapp

(1991). Das Prüfungsängstlichkeitsinventar TAI-G: Eine erweiterte und modifizierte Version mit vier Komponenten [The Test Anxiety Inventory: An expanded and modified version with four components]. Zeitschrift Für Pädagogische Psychologie, 5, 121–130.

26.

Hodapp

Rohrmann

Ringeisen

(2011). Prüfungsangstfragebogen (PAF)[Test-Anxiety Questionnaire]. Göttingen, the Netherlands: Hogrefe.

27.

Hoferichter

Raufelder

Ringeisen

Rohrmann

Bukowski

W. M.

(2016). Assessing the multi-faceted nature of test anxiety among secondary school students: An English version of the German Test Anxiety Questionnaire: PAF-E. The Journal of Psychology, 150(4), 450–468. doi:10.1080/00223980.2015.1087374

28.

Hofmann

S. G.

Heinrichs

Moscovitch

D. A.

(2004). The nature and expression of social phobia: Toward a new classification. Clinical Psychology Review, 24(7), 769–797. doi:10.1016/j.cpr.2004.07.004

29.

Hook

J. N.

Valentiner

D. P.

(2002). Are specific and generalized social phobias qualitatively distinct? Clinical Psychology: Science and Practice, 9(4), 379–395. doi:10.1093/clipsy.9.4.379

30.

Hopko

D. R.

Mahadevan

Bare

R. L.

Hunt

M. K.

(2003). The Abbreviated Math Anxiety Scale (AMAS) construction, validity, and reliability. Assessment, 10(2), 178–182. doi:10.1177/1073191103010002008

31.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. doi:10.1080/10705519909540118

32.

Kane

M. T.

(2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. doi:10.1111/jedm.12000

33.

Kline

R. B.

(1998). Software review: Software programs for structural equation modeling: Amos, EQS, and LISREL. Journal of Psychoeducational Assessment, 16(4), 343–364.

34.

Kline

R. B.

(2010). Principles and practice of structural equation modeling (3rd ed.). New York, NY: Guilford Press.

35.

Liebert

R. M.

Morris

L. W.

(1967). Cognitive and emotional components of test anxiety: A distinction and some initial data. Psychological Reports, 20(3), 975–978.

36.

Little

T. D.

(1997). Mean and covariance structures (MACS) analyses of cross-cultural data: Practical and theoretical issues. Multivariate Behavioral Research, 32(1), 53–76.

37.

Lohbeck, A., Nitkowski, D., & Petermann, F. (2016, December). A control-value theory approach: relationships between academic self-concept, interest, and test anxiety in elementary school children. In Child & Youth Care Forum (Vol. 45, No. 6, pp. 887–904). Springer US.

38.

McDonald

A. S.

(2001). The prevalence and effects of test anxiety in school children. Educational Psychology, 21(1), 89–101. doi:10.1080/01443410020019867

39.

McDonald

R. P.

(1999). Test theory: A unified treatment. Mahwah, NJ. Erlbaum.

40.

Meade

A. W.

Johnson

E. C.

Braddy

P. W.

(2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93(3), 568–592. doi:10.1037/0021-9010.93.3.568

41.

Methia

(2004). Help your child overcome test anxiety & achieve higher test scores. College Station, TX: Virtual Bookworm.Com Pub Incorporated.

42.

Morsanyi

Busdraghi

Primi

(2014). Mathematical anxiety is linked to reduced cognitive reflection: A potential road from discomfort in the mathematics classroom to susceptibility to biases. Behavioral and Brain Functions, 10(1), 31. doi:10.1186/1744-9081-10-31

43.

Passolunghi

M. C.

Caviola

, De Agostini

Perin

Mammarella

I. C.

(2016). Mathematics anxiety, working memory, and mathematics performance in secondary-school children. Frontiers in Psychology, 7, 42. doi:10.3389/fpsyg.2016.00042

44.

Pekrun

(2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4), 315–341. doi:10.1007/s10648-006-9029-9

45.

Pekrun

Frenzel

A. C.

Goetz

Perry

R. P.

(2007). The control-value theory of achievement emotions: An integrative approach to emotions in education. In Emotion in education (pp. 13–36).

46.

Primi

Busdraghi

Tomasetto

Morsanyi

Chiesi

(2014). Measuring math anxiety in Italian college and high school students: Validity, reliability and gender invariance of the Abbreviated Math Anxiety Scale (AMAS). Learning and Individual Differences, 34, 51–56. doi:10.1016/j.lindif.2014.05.012

47.

Plake, B. S., & Parker, C. S. (1982). The development and validation of a revised version of the Mathematics Anxiety Rating Scale. Educational and Psychological Measurement, 42(2), 551–557.

48.

Reiss

Warnecke

Tolgou

Krampen

Luka-Krausgrill

Rohrmann

(2017). Effects of cognitive behavioral therapy with relaxation vs. imagery rescripting on test anxiety: A randomized controlled trial. Journal of Affective Disorders, 208, 483–489. doi:10.1016/j.jad.2016.10.039

49.

Ringeisen

Raufelder

Schnell

Rohrmann

(2016). Validating the proposed structure of the relationships among test anxiety and its predictors based on control-value theory: Evidence for gender-specific patterns. Educational Psychology, 36(10), 1826–1844. doi:10.1080/01443410.2015.1072134

50.

Rockinson-Szapkiw

A. J.

Wendt

Lunde

(2013). Electronic versus print textbooks: The influence of textbook format on university students’ self-regulated learning strategies, motivation, and test anxiety. American Journal of Distance Education, 27(3), 179–188. doi:10.1080/08923647.2013.796230

51.

Roediger

H. L.

III. Karpicke

J. D.

(2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255. doi:10.1111/j.1467-9280.2006.01693.x

52.

Sarason

I. G.

(1984). Stress, anxiety, and cognitive interference: Reactions to tests. Journal of Personality and Social Psychology, 46(4), 929–938.

53.

Schermelleh-Engel

Moosbrugger

Müller

(2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23–74.

54.

Schnell

Ringeisen

Raufelder

Rohrmann

(2015). The impact of adolescents’ self-efficacy and self-regulated goal attainment processes on school performance—Do gender and test anxiety matter? Learning and Individual Differences, 38, 90–98. doi:10.1016/j.lindif.2014.12.008

55.

Schnell

Tibubos

A. N.

Rohrmann

Hodapp

(2013). Test and math anxiety: A validation of the German Test Anxiety Questionnaire. Polish Psychological Bulletin, 44(2), 193–200. doi:10.2478/ppb-2013-0022[Mismatch]

56.

Schwarzer

(1984). Worry and emotionality as separate components in test anxiety. Applied Psychology, 33(2), 205–220.

57.

Schwarzer

(2001). Social-cognitive factors in changing health-related behaviors. Current Directions in Psychological Science, 10(2), 47–51. doi:10.1111/1467-8721.00112

58.

Segool

N. K.

Carlson

J. S.

Goforth

A. N.

von der Embse

Barterian

J. A.

(2013). Heightened test anxiety among young children: Elementary school students’ anxious responses to high‐stakes testing. Psychology in the Schools, 50(5), 489–499. doi:10.1002/pits.21689

59.

Sieber

J. E.

O’Neil

H. F.

Jr. Tobias

(1977). Anxiety, learning, and instruction. New York, NY: Wiley.

60.

Spielberger

C. D.

(1980). Test Anxiety Inventory (“Test Attitude Inventory”). Preliminary professional manual. Palo Alto, CA: Consulting Psychologists Press.

61.

Spielberger

C. D.

Vagg

P. R.

(1995). Test anxiety: A transactional process model. Washington, DC: Taylor & Francis.

62.

Steiger

J. H.

Lind

J. C.

(1980, May). Statistically based tests for the number of common factors. In Annual meeting of the psychometric society, Iowa City, IA (Vol. 758, pp. 424–453).

63.

Tempel

Neumann

(2016). Taming test anxiety: The activation of failure-related concepts enhances cognitive test performance of test-anxious students. The Journal of Experimental Education, 84(4), 702–722. doi:10.1080/00220973.2015.1094649

64.

Tucker

L. R.

Lewis

(1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1–10.

65.

Vandenberg

R. J.

Lance

C. E.

(2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. doi:10.1177/109442810031002

66.

Wacker

Jaunzeme

Jaksztat

(2008). Eine Kurzform des Prüfungsängstlichkeitsinventars TAI-G. Zeitschrift Für Pädagogische Psychologie Test Anxiety Inventory TAI-G]. Journal of Pedagogical Psychology, 22(1), 73–81.

67.

Waiyavutti

Johnson

Deary

I. J.

(2012). Do personality scale items function differently in people with high and low IQ? Psychological Assessment, 24, 545–555. doi:10.1037/a0026266

68.

Ware

W. B.

Galassi

J. P.

Dew

K. M. H.

(1990). The test anxiety inventory: A confirmatory factor analysis. Anxiety Research, 3(3), 205–212. doi:10.1080/08917779008248753

69.

Wren

D. G.

Benson

(2004). Measuring test anxiety in children: Scale development and internal construct validation. Anxiety, Stress & Coping, 17(3), 227–240. doi:10.1080/10615800412331292606

70.

Zeidner

(2007). Test anxiety in educational contexts: Concepts, ﬁndings, and future directions. In Schutz

P. A.

Pekrun

(Eds.), Emotion in education (pp. 165–184). San Diego, CA: Academic Press.