Construction of the Examination Stress Scale for Adolescent Students

Abstract

The tools used for measuring examination stress have three main limitations: sample selected, sample sizes, and measurement contents. In this study, we constructed the Examination Stress Scale (ExamSS), and 4,717 high school students participated in this research. The results indicate that ExamSS has satisfactory reliability, construct validity, and measurement-invariance validity.

Keywords

examination stress Examination Stress Framework adolescents construct validity measurement invariance

Examination stress is a common and prevalent phenomenon among teenagers (Ang et al., 2009; Beidel, Turner, & Taylor-Ferreira, 1999). The examination stress phenomenon is not only prevalent but also likely to have a negative impact. In the early 20th century, Yerkes and Dodson (1908) indicated that the appropriate kind of stress enhances performance, that is, when stress was gradually increased, performance gradually improved; but once maximal performance had been reached, any further gradual increase in stress brought about a gradual decline in performance. Experimental studies have also indicated that under stressful evaluation situations, students’ scores may be reduced because of anxiety (Worthy, Markman, & Maddox, 2009). Meanwhile, past studies have shown that test anxiety and poor test performance are significantly correlated (Hembree, 1988; Seipp, 1991).

Examination stress is recognized as an important issue, and measuring the level of stress is the first step toward investigating topics related to examination stress. However, there are three major drawbacks when measuring the degree of examination stress for students with current measurement tools. First, the development of most of the current test-anxiety scales involved college students as participants, yet in many countries with high school entrance examinations or college entrance examinations, such as Australia, China, the United Kingdom, Hong Kong, and Korea, adolescents in high school actually represent a substantial population of people dealing with test anxiety.

Second, studies using small samples are insufficient. Small samples neither provide sufficient evidence of scale validity nor allow for the establishment of norms for testing examination stress. Consequently, the currently available tools are neither sufficient to understand the relativity of stress intensity nor to provide more detailed information for guidance consultations dealing with examination stress.

Third, the definition of examination stress is not sufficiently comprehensive for the examination stress experienced by adolescents. More stressors and reactions to anxiety should be considered when assessing adolescents’ examination stress. For example, the stress attributable to social expectations and social comparisons is powerful and long term, yet it has thus far been neglected in research.

The aims of this study were threefold: (a) to construct an Examination Stress Framework (ESF) that comprehensively describes the stressors and appraisal of, and responses to examination stress faced by high school students; (b) to construct an Examination Stress Scale (ExamSS) based on the ESF to provide a tool for the measurement of examination stress among high school students; and (c) to determine the reliability and validity of the ExamSS with sufficiently large and representative samples and to establish the norms of the ExamSS for assisting the practice of learning guidance and counseling.

Literature Review

Stress and Anxiety

Stress is a process in which people get aware of, evaluate, and respond to the threats in the environment. Several scholars consider that there are three components to stress that interact between the individual and their environment: the stressor, appraisal, and the stress response (Grant, Behling, Gipson, & Ford, 2005; Lazarus & Folkman, 1986; Zeidner, 1998). A stressor is a threat within the environment to an individual’s life, property, or psychological state, such as fire, daily work, or a long-term economic crisis. Appraisals refer to whether an individual perceives those threats and whether they can determine what kind of responses one should have when coping with such stressors (Lazarus & Folkman, 1986). Stress responses indicate the physiological and psychological changes that are experienced when facing stress, such as bodily arousal and worry (Liebert & Morris, 1967).

Anxiety is a basic human emotion and is the most common and crucial response to stressful events (Sarason & Sarason, 1990). The most typical anxiety responses are physical arousal, such as an accelerated heart rate and sweating and emotional responses, such as worry or fear. Since anxiety is the most frequent emotional response experienced by people when facing stressful events, psychological studies have focused on how anxiety affects the results of these events, including cognitive performance, social performance, subjective perceptions, and physical health. Anxiety is defined by many as an aspect of the stress response.

Existent Examination Stress Measurement Tools

Based on the concepts of stress or anxiety mentioned above, several tools have been developed for measuring examination stress or test anxiety (Table 1). The major limitations of the existing stress measurement tools are threefold. First, the applicability of the existing scales is limited. The development of current test-anxiety scales involved mostly college students as the participants; few studies have tested their scales on elementary school students and teenagers (Wren & Benson, 2004). Moreover, none of the previous scales have used high school students—and more particularly senior high school students—as research targets during their construction. Second, the constructs of examination stress have not been covered comprehensively for the stressors of and responses to the regular and high-stakes examinations that adolescents experience. Finally, small sample sizes restricted the evidence for validity scores and norms. The sample size for model estimation is an important issue; an insufficient size will restrict statistical power and lead to imprecise model estimation (Kline, 2010). Jackson (2003) estimated that the ideal sample size relative to the number of parameters in structural equation modeling is 20:1. None of the previous scales meet this standard (Table 1). Meanwhile, the small samples in these studies made it hard to establish the norms of stress scale scores to enable counselors to understand the level of examination stress students withstand.

Table 1.

Summary of Previous Studies of Examination Stress.

Scale	Author/s (Year)	Subjects (N)	Dimension	Reliability
Test Anxiety Questionnaire	Mandler and Sarason (1952)	Freshmen (154)	Test anxiety	Not reported
Test Anxiety Scale	Sarason (1961)	Freshmen (838)	Test anxiety	Not reported
Worry-Emotional Questionnaire	Liebert and Morris (1967)	College students (54)	Emotion and worry	Not reported
Suinn Test Anxiety Behavior Scale	Suinn (1969)	College students (233)	Test anxiety behavior	.74 for sample 1
Suinn Test Anxiety Behavior Scale	Suinn (1969)	College students (233)	Test anxiety behavior	.80 for sample 2
Test Anxiety Inventory	Spielberger et al. (1980)	—	Emotion and worry	.93 for 20-items
Test Anxiety Inventory	Taylor and Deane (2002)	College students (333)	Emotion and worry	.84 for 4-items
Reaction to Tests Scale	Sarason (1984)	Study 1: Freshmen (390); Study 2: Freshmen (385)	Tension, worry, test-irrelevant thinking, and bodily reaction	Not reported
German Test Anxiety Inventory	Hodapp and Benson (1997)	College and graduate students (436)	Emotionality, worry, cognitive interference, and lack of confidence	.89, .85, .92, and .87
	Keith, Hodapp, Schermelleh-Engel,and Moosbrugger (2003)	College, night school, and nursing school students (302)
	Stöber (2004)	College students (162)
Cognitive Test Anxiety Scale	Cassady and Johnson (2002)	College students (167)	Cognitive reactions to evaluative situations	.91
Children’s Test Anxiety Scale	Wren and Benson (2004)	Primary school students (218)	Off-task behavior, autonomic reactions, and thoughts	.76, .82, and .89
Test Emotions Questionnaire	Pekrun et al. (2004)	College students (26 for pilot study, 186 for formal study)	Affective, cognitive, physiological, and motivational	Not reported; reliabilities corresponded not to the four dimensions, but the eight emotions

The Examination Stress Framework for Teenagers

According to stress theory (Grant et al., 2005; Lazarus & Folkman, 1986), examination stress comprises three parts: examination stressors, appraisal, and examination stress responses. Based on these three parts, herein an ESF is proposed that is adaptive to high school students (Figure 1).

Figure 1.

The Examination Stress Framework (ESF) for teenagers.

Examination Stressors

There are two examination stressors for high school students in the ESF: situational and societal. Situational stressors refer to the general test situations experienced at school, such as daily quizzes and mid- or final-term examinations (Stöber, 2004; Stöber & Pekrun, 2004). Situational stressors comprise three major processes: preparing for examinations, taking examinations, and coping with the examination results. Most previous research into test anxiety has focused on situational stressors. However, the present study considered “societal stressors”—which involve the atmosphere around examinations that are created by society—as an additional category of stressors for students.

The social atmosphere may comprise three layers. First, examinations are seen as an element of culture in a society. In both Eastern and Western societies, examinations are an important mechanism for competing for educational resources, forming bureaucracies, and for enabling social mobility (Collins, 2006). For example, China’s imperial examination (605 AD–1905 AD) gave commoners the opportunity to move into a higher stratum within society through public examinations. This system also greatly influenced the civil service selection systems of many other Asian countries, such as Vietnam, Korea, and Japan (Creel, 1964). In a culture that regards examinations as a major mechanism for social mobility, people naturally show caution and respect toward them.

Second, examinations are considered as both a link to the social system and as a method of responding to social demands. Many countries have established entrance examinations because of the demands of educational tracking (Heyneman, 2009), such as the Scholastic Assessment Tests in the United States and the Basic Competence Test (Wang, Sung, Sung, Lu, & Li, 2008) in Taiwan. Meanwhile, more and more states in the United States hold senior high school exit examinations before the end of compulsory education (Reardon, Arshan, Atteberry, & Kurlaender, 2010). Most of these high-stakes examinations induce great pressure and uneasiness in those undertaking them (Putwain, 2008; Richman, Brown, & Clark, 1987).

The third layer is the social expectation and social comparison that result from examinations. “Social expectation,” which is derived from the core value of a society and shaped by social systems, refers to inherent social norms within individuals, organizations, or the whole country, which explicitly or implicitly influence both individual and collective behaviors (Hasegawa, Shinohara, & Broadbent, 2007). With respect to the culture and systems related to examinations, social expectations at all times remind students that obtaining good grades on examinations (e.g., entrance examinations) equates with being admitted to good schools, which will subsequently lead to a better future and a higher social status (Bae & Lee, 1988). Such social expectation naturally creates social behaviors that intensify examination stress, encouraging (or forcing) students to go to cram schools, to join in mock examinations, and to improve their test-taking skills. Such collective behavior forms a cycle that further increases the emphasis on examinations as a result of social expectations.

People compare themselves with each other because of these expectations. Social comparison theory (Zell & Alicke, 2009) proposes that students compare their examination performance with that of their peers, with the result becoming their academic self-concept or motivation for learning. In addition, parents are likely to compare their children’s performance with those of other parents (Ang et al., 2009). The comparison among students, parents, and schools regarding enrollment rates may either directly or indirectly cause stress to students.

Therefore, it is clear that societal stressors cannot be ignored—rather than focusing entirely on situational stressors—when investigating the sources of examination stress.

Examination Stress Appraisal

Appraisal refers to an individual’s process of evaluating the stressor, his or her responses to it, and his or her subsequent coping strategy (Lazarus, & Folkman, 1984). Lazarus and Folkman (1986) indicated that there are two phases of appraisal: primary and secondary. Primary appraisal refers to the individual’s perception of whether the stressor represents a threat. If it does, the person will evaluate what the most appropriate responses are and what coping resources are available; this part of the process is called secondary appraisal. After appraising the stress, one responds and implements coping strategies (Grant, Behling, Gipson, & Ford, 2005).

Examination Stress Responses

Zeidner (1998) suggested that stress responses comprise cognitive, behavioral, physiological, and emotional components. Most of the relevant research has thus far considered responses to examination stress as anxiety responses (Hodapp & Benson, 1997; Liebert & Morris, 1967; Mandler & Sarason, 1952; Sarason, 1984). However, given the different stressors, the proposed ESF of the present study classifies high school students’ examination stress responses into three categories: physiological anxiety response, cognitive and behavioral response, and perceived social expectation and social comparison.

Physiological anxiety response

Anxiety responses are emotional responses when facing examinations. Mandler and Sarason (1952) defined test anxiety as the anxiety response to examinations or assessment situations; such anxiety responses are strongly related to physiological responses, as mentioned above. Liebert and Morris (1967) categorized test anxiety into emotionality and worry. In the 1980s, a more detailed definition of the dimension of worry was proposed, such as worry and fear of failure (Covington, 1985), irrelevant thinking and worry (Sarason, 1984), and distraction and low self-confidence (Hodapp & Benson, 1997). It may be concluded from past research that the concept of anxiety has evolved into a multilevel concept with different dimensions of responses, including physiological, thinking, behavioral and emotional components, as suggested by Zeidner (1998). Based on these deductions, this study concurs with those of Schwarzer and Buchwald (2003) by including anxiety as a major component of examination stress, and by focusing on physiological arousal, as proposed by Mandler and Sarason (1952).

Cognitive and behavioral responses

Current test-anxiety scales usually center around two dimensions, emotionality and worry. In the dimension of worry, the main concepts are associated with one’s own negative evaluation. Various researchers have proposed different subdimensional definitions of worry (Hodapp & Benson, 1997; Keith et al., 2003; Pekrun et al., 2004; Sarason, 1984; Spielberger et al., 1980; Stöber, 2004). The degree of threat that an individual perceives from examinations, as well as the resulting behavior exhibited when dealing with those examinations can be indicators of the degree of stress examinees feel (Suinn, 1969). Therefore, studies of examination stress must consider whether students regard examinations as being serious threats or whether their behavior on dealing with examinations can represent the intensity of examination stress.

Cognitively, students probably view examinations as serious threats; they believe that examinations are closely related to their future success. It is important whether or not students show certain behaviors because of examination stress. For example, when preparing for examinations, students force themselves to study and keep practicing even if they have already understood the subject content. The more students think or react, as mentioned above, the greater the examination stress they suffer.

Perceived social expectation and social comparison

Perceived social expectations with respect to examinations is a major source of stress for high school students. In Hong Kong and Singapore, students worry about their academic performance, along with pressure from the expectations held by their parents/family and teachers, both of which are reported to be a source of stress for them (Ang et al., 2009). With respect to oriental education, social expectation results in academic-, test-, and study-related stress among students. Some researchers even call this phenomenon “examination hell” (Lee & Larson, 2000). Thus, the intensity of perceived social expectations appears to be an indicator of examination stress.

On the other hand, perceived social comparison should also be regarded as an indicator of examination stress. Erdogan, Kesici, and Sahin (2011) showed that social comparisons among students could significantly increase the degree of predicted anxiety toward mathematics. Therefore, a student will experience greater stress when he or she perceives stronger social comparisons and more negative attitudes toward that perception.

Based on the above reviews, this present study intends to construct a more comprehensive examination stress scale for high school students based on the stressors and stress responses in the ESF and examine the psychometric properties of this scale by using abundant participants.

Method

Construction of the ExamSS

According to the ESF, the three dimensions of the responses to examination stress are physiological anxiety responses, cognitive and behavioral responses, and perceived expectations and comparisons. After determining the dimensions, items were constructed for each. The item descriptions were required to fit the definition of the dimension content.

The pilot-test version of the ExamSS consisted of 43 items: 14 for physiological anxiety responses, 15 for cognitive and behavioral responses, and 14 for perceived social expectation and social comparison. Among these 43 items, 22 were related to regular school examination situations and 21 were related to high-stakes examinations, such as entrance examinations.

Responses to the ExamSS are given on a 5-point Likert-type scale, with scores of 0, 1, 2, 3, and 4 denoting totally disagree, slightly disagree, partially agree, agree, and totally agree, respectively. For both the subscales and the whole scale, higher scores indicate a greater level of examination stress. After the pilot test, the ExamSS was revised according to the results of the item analyses and exploratory factor analysis, and the final version of ExamSS comprised 27 items, as presented in Table 2. The details of the data processing are given in the Results section.

Table 2.

Dimensions and Items of the Examination Stress Scale (ExamSS).

Dimension	Definition	Items	Item Discrimination Index	Structural Factor Loading (Standardized Coefficient)	Subscale Reliability	Scale Reliability
Anxiety responses	Responses such as physical illness or discomfort, sleep disorders, and emotional distress	I experience stomach discomfort before or during examinations	.53	.55	.89	.92
		I always feel uneasy before important examinations	.70	.73
		I am too nervous to remember what I have learned during examinations	.59	.63
		I tremble when waiting for examinations to begin	.57	.60
		I feel ill-at-ease before examinations	.70	.74
		My muscles are tense and sore before examinations	.62	.64
		Preparing for examinations often makes me physically uncomfortable	.66	.69
		I experience sleep disorder when I think about entrance examinations	.66	.72
		I experience shoulder stiffness and headaches when facing important examinations	.68	.72
		As the entrance examination approaches, I feel anxious and distressed	.65	.71
Cognitive and behavioral responses	Thoughts and behaviors that are triggered by examination stress, such as concerns for examination scores and forcing themselves to study intensely	I will be very upset if my performance in examinations does not match the efforts invested	.61	.69	.85
		I often force myself to study intensively for examinations	.64	.69
		I care about my scores for every examination	.60	.65
		My entrance examination results are closely related to my future career	.58	.62
		Even if I feel tired, I do not dare to stop studying when I think about the approaching entrance examination	.65	.72
		The entrance examination is similar to a battle that I cannot afford to lose	.70	.74
		If I get poor scores in the entrance examination, all my efforts are wasted	.63	.67
		If my entrance examination score is not good enough, it is worth taking one more year to prepare for and take the examination again	.38	.40
Perceived social expectation and social comparison	The intensity of perceived significant others’ expectation and the perceived comparison made based on the performance of examinations. The attitudes toward the perceived social expectation and social comparisons	My parents’ expectations of my examinations scores disturb me	.56	.59	.88
		I do not like my parents comparing my examination scores with those of my siblings	.50	.52
		My teachers’ expectations of high scores in examinations burdened me	.59	.65
		Comparing my examination scores with my classmates makes me restless	.61	.64
		I do not like to be ranked after the school examinations	.70	.72
		I feel stressed because my parents often remind me that I should get into an elite high school (or college)	.68	.71
		The higher the expectations others have about my entrance examination performance, the more I worry	.63	.70
		I feel jealous when hearing about other students that have been admitted to elite schools	.72	.76
		If my entrance examination score is not as high as expected, I do not dare to face my relatives or neighbors	.56	.61

Participants

In Taiwan’s educational system, all junior high school graduate students (i.e., Grades 7–9) must sit the Basic Competence Test (Sung, Tseng, Kuo, Chang, & Chiou, 2014; Wang et al., 2008), which is a national entrance examination for admission to senior or vocational high schools. Furthermore, all senior high school graduates (i.e., Grades 10–12) must sit the General Scholastic Ability Test (Chang & Cheng, 2008), which is a national entrance examination for admission to colleges or universities. Most students feel much stressed during the 6 years of preparing for regular classroom examinations and these entrance examinations, which makes them optimal candidates for studies of examination stress.

In the present study, the scale evaluation process was divided into a pilot-test stage and a formal-test stage. In the pilot-test stage, data were collected from 922 ninth-grade students in junior high schools, of which 463 were male (50.22%) and 459 were female (49.78%). In the formal-test stage, 3,795 students were recruited, of which 1,907 were males (50.3%) and 1,869 were females (49.2%). The students comprised 2,291 (60.4%) from the ninth grade and 1,504 (39.6%) from the tenth and eleventh grades.

Stratified random sampling was used to recruit junior high school students from northern, central, southern, and eastern Taiwan at a ratio of 3:2:2:1. The distribution of the research participants covered schools in every region, and students with various learning achievements throughout Taiwan. Since students in Taiwan are stratified during the transition from junior to senior high school, the abilities of senior high school students are segmented between different schools. Therefore, purposive sampling was adopted to select five schools in northern Taiwan, five schools in central Taiwan, and four schools in southern Taiwan, according to school-ability levels. The distribution of living areas and abilities of the research sample was identical to the actual distribution of junior- and senior high school students in Taiwan, meaning that the sample was representative of such students in this country. All the students participated in the study under the agreement of participation in the pilot test of the National Entrance Examination Research project.

Procedure

The pilot-test data were collected between late February 2011 and mid-March 2011, while the formal-test data were collected between late May 2011 and early June 2011. The purpose of the scale and the method of completing it were explained to the participating students before it was administered to them. After they completed the scale, the students received a gift in appreciation of their participation. The time taken to complete ExamSS was typically 20 to 25 minutes.

Results

Pilot-Test Data Analyses

First, the discrimination index of the items was analyzed using the corrected item-total correlation, which indicates whether points obtained for a single item are correlated to the total score. Items are generally considered to be of high quality if the correlation coefficient exceeds .40 (Ebel & Frisbie, 1991).

Exploratory factor analysis was subsequently conducted with the principal axis factor and the oblique rotation method. After combining the results of item analyses and factor analyses, items with a discrimination index lower than .4 and a factor loading lower than .4 were removed. Overall, 16 items were removed and the remaining 27 items could explain 50.68% of the variance for factors in the three dimensions. The factor loadings of the three dimensions were .54 to .76 for anxiety responses, .45 to .77 for cognitive and behavioral responses, and .51 to .78 for perceived expectations and comparisons. The final scale comprised 27 items, with 10 items for anxiety responses, 8 for cognitive and behavioral responses, and 9 for perceived expectations and comparisons (see Table 2).

Formal-Test Data Analyses

Reliability Analyses

Cronbach’s alpha coefficient was used to assess the internal consistency reliability of the ExamSS. Cronbach’s alpha coefficients for the physiological anxiety response, cognitive and behavioral response, and perceived social expectations and social comparisons subscales were .89, .85, and .88, respectively, while that for the whole scale was .92 (Table 2). These statistics indicate that the ExamSS has satisfactory reliability.

Model-Fit Analyses

Confirmatory factor analysis (CFA) was performed to determine the construct validity of the ExamSS, to verify that the items fit well to the model with three dimensions. The statistical software Mplus 6.0 (Muthén & Muthén, 2010) that could create appropriate standard errors for stratified samples was employed, along with the maximum likelihood estimation method. The one- and three-dimensional models of examination stress were analyzed to determine whether the examination stress responses measured corresponded to the three proposed dimensions or whether they could be seen as a single dimension of examination stress (Figure 2). The chi-square values for both measurement models exceeded the expected value, such that the exact-fit hypotheses were rejected (Table 3). An excessive sample size or a stronger correlation between variables may have been reasons for the increased chi-square values (Kline, 2010). Approximate fit indices were then assessed—including the comparative fit index (CFI) and the Tucker–Lewis index (TLI)—as alternative indices for the criterion of model fit (Byrne, 2001): their values for the three-dimensional model were .87 and .86, respectively; both these values are not higher than the value of .9 suggested by Byrne (2001). The upper limit for the confidence interval of the root mean square error of approximation (RMSEA) and the standardized root mean square residual (SRMR) of the three-dimensional model did not exceed .08, which indicated an acceptable fit (Byrne, 2001). The indices presented in Table 3 show that the three-dimensional model of examination stress proposed in this study, although it only reaches a marginal fit, is more appropriate than considering all items under a single dimension.

Figure 2.

Confirmatory factor analysis of the three examination stress dimensions.

Table 3.

Comparison of the Examination Stress Scale (ExamSS) Models.

	One-Dimensional Measurement Model	Three-Dimensional Measurement Model
Number of parameters	81	84
Degrees of freedom	324	321
Chi-square	11956.64	6359.21
Chi-square test	<.001	<.001
Comparative fit index	.65	.87
Tucker–Lewis index	.63	.86
Root mean square error of approximation	.098~.101	.059~.062
Standardized root mean square residual	.091	.053

Validity Analyses of the Measurement Model

Two structural indices of the three-dimensional model—the factor rho coefficient (Raykov, 2004) and the average variance extracted (AVE)—were calculated using the estimated parameter of each item. The factor rho coefficient refers to the ratio of variance that is explainable by the parameters of the measurement indices to the total variance; AVE represents the average variance explained for latent variables by each measurement index.

The factor rho coefficients for the three dimensions of physiological anxiety responses, cognitive and behavioral responses, and perceived expectation and comparison were .89, .85, and .88, respectively. These coefficients for the three latent variables all exceeded .7, indicating that this measurement model was satisfactory (Raykov, 2004). AVE values of .46, .43, and .43 were obtained for these three dimensions, which are all higher than the minimum criteria of .25 (squared .5 of standardized factor loading) proposed by Hair, Black, Babin, and Anderson (2010), indicating that the three-dimensional model of ExamSS is an acceptable model. The results of the CFA show that the ExamSS has satisfactory construct validity.

Measurement-Invariance Analyses

Measurement-invariance analysis was conducted to determine if the measurement model of the ExamSS was identical for students of different genders and education levels. Measurement invariance means that the measurement construct is robust and that the items do not present bias for groups. The measurement-invariance hypothesis was verified by comparing restricted models, in which factor loading, covariance and variance matrices, and residual terms in factor structure were constrained, with the unrestricted model. Since the chi-square difference test has the same problem as the chi-square test in that it is sensitive to sample size (Chen, 2007; Kline, 2010), some researchers have proposed that differences between certain indices of model fit, such as differences between the CFI and RMSEA values, can be used to verify measurement invariance (Chen, 2007; Cheung & Rensvold, 2002). Table 4 indicates that all the chi-square differences were statistically significant. However, the values of ΔCFI and ΔRMSEA for the gender variable did not exceed −0.01 and 0.015, respectively, indicating that there was invariance among the genders (Chen, 2007). In addition, for junior and senior high school students (Table 5), ΔCFI and ΔRMSEA were lower than the same criteria proposed by Chen (2007). These analyses indicate that groups of different genders and education levels can be considered to have the same restricted models with factor loading, covariance and variance, and residual terms; the measurement construct of the ExamSS was thus consistent between different groups.

Table 4.

Measurement-Invariance Analyses of Assumption Models for Students of Different Genders.

Model	Explanation	χ² (df)	Δχ² (df)	CFI	ΔCFI	RMSEA	ΔRMSEA
Model 1	Configural invariance	6359.209 (321)	—	.869	—	.070	—
Model 2	Construct-level metric invariance	6736.654 (666)	377.445*** (345)	.865	−.004	.069	−.001
Model 3	Intercept/scalar /strong invariance	7161.801 (690)	425.147*** (24)	.856	−.009	.070	.001

Note. CFI = comparative fit index; RMSEA = root mean square error of approximation.

***

p < .001.

Table 5.

Measurement-Invariance Analyses of Assumption Models for Junior and Senior High School Students.

Model	Explanation	χ² (df)	Δχ² (df)	CFI	ΔCFI	RMSEA	ΔRMSEA
Model 1	Configural invariance	6359.209 (321)	—	.869	—	.070	—
Model 2	Construct-level metric invariance	6894.886 (666)	535.677*** (345)	.864	.005	.070	<.001
Model 3	Intercept/scalar /strong invariance	7310.478 (690)	415.592*** (24)	.855	.009	.071	.001

Note. CFI = comparative fit index; RMSEA = root mean square error of approximation.

***

p < .001.

Developing the Norms for the ExamSS

To help practitioners of guidance and counseling to better comprehend the relative levels of stress among students, their ExamSS scores are presented in the form of norms with a percentile rank; for the reason of space, please see the norms of ExamSS at http://140.122.96.188/norms.

Discussion

Construction of the ExamSS

The ExamSS constructed in the present study differs markedly from previous test-anxiety scales. The first significant differential characteristic of the ExamSS is that representative samples of junior and senior high school students were enrolled as research participants. Most previous studies selected university students as participants, thereby neglecting that high school students may be one of the biggest populations to suffer from examination stress. From this point of view, the ExamSS expands on the practicality of previously developed scales designed to evaluate test anxiety and stress.

The second significant characteristic is that the ExamSS considers various stressors and personal responses related to examination stress. The ESF was proposed herein as a framework to describe the relationships among examination stressors, appraisal, and responses. The items of the ExamSS were constructed according to the three dimensions of responses to examination stress, the physiological anxiety response, cognitive and behavioral response, and perceived expectation and comparison dimensions. Compared with most previous test-anxiety scales, which cover only the dimensions of physiological anxiety, the ExamSS not only broadens the construct of test anxiety or stress, but also is a closer fit to the recent construct regarding test anxiety proposed by Zeidner (1998) and Busari (2011), in which the construct of test anxiety comprises physiological, cognitive, behavioral, and affective responses. Similar to the Student Academic Stress Scale (Busari, 2011), certain behaviors and thoughts exhibited by students could reflect the level of stress that they experience. Previous test-anxiety scales were developed primarily based on the two dimensions of “emotionality” and “worry,” as proposed by Liebert and Morris (1967), although they have been expanded to four dimensions (Hodapp & Benson, 1997; Keith et al., 2003; Pekrun et al., 2004; Sarason, 1984; Stöber, 2004). However, in addition to emotional anxiety and worry, ExamSS included students’ thoughts and behavioral responses to examination stressors and can thus reflect levels of examination stress more comprehensively. Another prominent dimension of examination stress proposed in the present study was the stress derived from societal stressors, such as the examination culture, the social system of high-stakes testing such as entrance examinations and credentials, and the social expectations and social comparisons related to the whole process of preparing for undertaking high-stakes testing. The perception and attitude of students toward perceived expectations and comparisons were used as indicators of examination stress. The correlation coefficients of perceived expectations and comparisons with physiological anxiety responses and cognitive and behavioral responses were .59 and .57, respectively. The moderate correlation between these dimensions indicates that the measured perceived expectations and comparisons share common elements with physiological anxiety and cognitive and behavioral responses, such that although perceived expectations and comparisons can be viewed as an independent dimension of examination stress. It also elicits a certain degree of anxiety and may induce cognitive and behavioral reactions. From these findings it can be concluded that the ExamSS can measure examination stress in a more integrated and complete way.

Reliability and Validity of the ExamSS

Empirical evaluation revealed the ExamSS to be psychometrically sound. The internal consistency reliability of Cronbach’s alpha coefficient for the whole scale was .92, and those for the three subscales exceeded .85, indicating that the ExamSS also possessed satisfactory reliability.

Evidence about the construct validity, such as the findings of the measurement model fit or measurement-invariance tests, were not provided with previous test-anxiety scales, in part because of the small number of participants used to construct those scales (e.g., Wren & Benson, 2004). Furthermore, of those that did provide the results of model testing (e.g., Hodapp & Benson, 1997; Pekrun et al., 2004), their sample sizes did not fit the ideal sample size for solid parameter estimation (Jackson, 2003). The collection of data from 3,795 high school participants in the present study enabled the ExamSS to be subjected to more rigorous evaluation of the SEM-based psychometric properties. Through CFA, it showed that the data fit the three-dimensional construct better than its one-dimensional counterpart. This indicates that although all three dimensions were used to measure examination stress, differing dimensions were discriminated as theoretically proposed, presenting evidence of discriminant validity. In addition, according to the factor rho coefficient (Raykov, 2004) and AVE, the present findings show that the items reflect their corresponding dimensions excellently, providing strong evidence of convergent validity for the subscales.

The measurement-invariance analyses revealed that the construct of the ExamSS was identical for the students, regardless of their gender or education level. The measurement invariance of the ExamSS thus indicates that this stress-measurement tool can be applied appropriately to all high school students regardless of their gender or education level. Before confirming the invariance properties of a developed scale, any differences between groups should not be evaluated because the scale may measure different constructs in different groups. The evidence of measurement invariance of the ExamSS provided herein expands its usefulness for practitioners who are concerned with the counseling or guidance of students regarding examination stress, and for research comparing examination stress among students of different groups and learning stages.

Conclusions and Implications

Neglecting the adolescents as a research target is a missing link in examination stress studies. Aiming to compensate this missing link, this study has several important features. By including under “examination stress,” the dimensions of anxiety, cognitive and behavior responses, and perceived social expectations and comparisons, this study proposed the ESF framework and expanded on the scope of traditional test-anxiety scales and may remind researchers or practitioners that including dimensions of cognitive/|behavior responses and social factors may be important for designing measurement tools and learning guidance for adolescent students. Second, the ExamSS—evaluated through representative samples of high school students—has been demonstrated to have satisfactory reliability and validity. Third, by recruiting high school students as participants for evaluating the psychometric properties of the ExamSS and building norms, this new scale can fill the current lack of a solid tool for measuring examination stress among this population, who are particularly prone to examination stress.

ExamSS may benefit practitioners in at least in two ways. The first is using the ExamSS as a tool for monitoring the stress level of students who are facing entrance examinations. By comparing the strength of entrance examination stress measured by ExamSS, the government may have solid evidence for the change of stress for different cohorts of students in different years and adjust their policy based on that evidence for further students. For example, in Taiwan, the change of junior high school students’ ExamSS scores in different years has been used as an indicator of the success of a policy that aims to relieve students from the stress from severe examination competition. The second aspect is using ExamSS as a tool for school counseling and guidance. As there are norms for the scores of ExamSS, school counselors may use the norms to evaluate if a certain student has an extraordinary level of stress while preparing for their testings. School counselors can also use ExamSS to measure students’ stress at both the beginning and end of each school semester. Based on profiles of students’ ExamSS percentile ranks, school counselors will be able to obtain examination-stress scores in three components. Referring to the function of STABS (Suinn, 1969), researchers may also consider developing some counseling plans for reducing students’ examination stress and use ExamSS as a repeat indicator to monitor whether personalized counseling plans are effective.

Future studies may address the issue of measuring examination stress in several directions. Regarding the dimensions of examination stress, future measurement tools may incorporate the process of appraisals as an important dimension of stress responses. Furthermore, future studies may investigate the relationships among examination stress and some other important variables, such as academic achievement and coping strategies, which are closely related to examination stress and could mediate the responses to it. By investigating the relationship between stress and achievement, we can find out if this relationship confirms the Yerkes–Dodson Law or if stress only negatively affects performance among high school students. By doing so, we can provide more evidence-based policy decisions for lessening students’ examination stress.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The author appreciate the generous funding supports from the Taiwan Ministry of Education’s “Aim for Top University Project” and the National Science Council, Taiwan (103-2911-I-003 -301; 102-2511-S-003 -001 -MY3; 101-2511-S-003 -058 -MY3; 101-2511-S-003 -047 -MY3; 99-2631-S-003-003).

Author Biographies

Yao-Ting Sung is a research chair professor in the Department of Educational Psychology and Counseling, and the director of Research Center for Psychology and Educational Testing at National Taiwan Normal University.

Tzu-Yang Chao is a Ph.D. student in the Department of Educational Psychology and Counseling, and a research fellow of Research Center for Psychology and Educational Testing at National Taiwan Normal University.

References

Ang

R. P.

Klassen

R. M.

Chong

W. H.

Huan

V. S.

Wong

I. Y. F.

Leo

L. S.

Krawchuk

L. L.

(2009). Cross-cultural invariance of the Academic Expectations Stress Inventory: Adolescent samples from Canada and Singapore. Journal of Adolescence, 32, 1225–1237.

Bae

J. K.

Lee

M. N.

(1988). Substance of Korean education: How Korean people think education? Seoul, South Korea: Kyoyouk KwaHak.

Beidel

D. C.

Turner

S. M.

Taylor-Ferreira

J. C.

(1999). Teaching study skills and test-taking strategies to elementary school students. Behavior Modification, 123, 630–692.

Busari

(2011). Validation of Student Academic Stress Scale (SASS). European Journal of Social Sciences, 21, 94–105.

Byrne

B. M.

(2001). Structural equation modeling with Amos: Basic concepts, applications, and programming. Mahwah, NJ: Lawrence Erlbaum.

Cassady

J. C.

Johnson

R. E.

(2002). Cognitive test anxiety and academic performance. Contemporary Educational Psychology, 27, 270–295.

Chang

C. Y.

Cheng

W. Y.

(2008). Science achievement and students’ self-confidence and interest in science: A Taiwanese representative sample study. International Journal of Science Education, 30, 1183–1120.

Chen

F. F.

(2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, 464–504.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233–255.

10.

Covington

M. V.

(1985). Test anxiety: Causes and effects over time. In van der Ploeg

H. M.

Schwarzer

Spielberger

C. D.

(Ed.), Advances in test anxiety research (Vol. 4, pp. 55–68). Lisse, Netherlands: Swets & Zeitlinger.

11.

Collins

(2006). Comparative and historical patterns of education. In Hallinan

M. T.

(Ed.), Handbook of the sociology of education (pp. 213–240). New York, NY: Springer.

12.

Creel

H. G.

(1964). The beginning of bureaucracy in China: The origin of the Hsien. Journal of Asian Studies, 23, 155–183.

13.

Ebel

R. L.

Frisbie

D. A.

(1991). Essentials of educational measurement (5th ed.). Englewood Cliffs, NJ: Prentice-Hall.

14.

Erdogan

Kesici

Sahin

İ.

(2011). Prediction of high school students’ mathematics anxiety by their achievement motivation and social comparison. Elementary Education Online, 10(2), 646–652.

15.

Grant

K. E.

Behling

Gipson

P. Y.

Ford

R. E.

(2005). Adolescent stress: The relationship between stress and mental health problems. The Preventive Researcher, 12, 3–6.

16.

Hair

J. F.

Black

W. C.

Babin

B. J.

Anderson

R. E.

(2010). Multivariate data analysis: A global perspective (7th ed.). Upper Saddle River, NJ: Pearson Education.

17.

Hasegawa

Shinohara

Broadbent

J. P.

(2007). The effects of social expectation on the development of civil society in Japan. Journal of Civil Society, 3, 179–203.

18.

Hembree

(1988). Correlates, causes, effects, and treatment of test anxiety. Review of Educational Research, 58, 47–77.

19.

Heyneman

S. P.

(2009). Introduction: The importance of external examinations in education. In Vlaardingerbroek

Taylor

(Eds.), Secondary school external examination systems. Amherst, NY: Cambria Press.

20.

Hodapp

Benson

(1997). The multidimensionality of test anxiety: A test of different models. Anxiety, Stress, & Coping, 10, 219–244.

21.

Jackson

D. L.

(2003). Revisiting sample size and number of parameter estimates: Some support for the N:q hypothesis. Structural Equation Modeling, 10, 128–141.

22.

Keith

Hodapp

Schermelleh-Engel

Moosbrugger

(2003). Cross-sectional and longitudinal confirmatory factor models for the German Test Anxiety Inventory: A construct validation. Anxiety, Stress, & Coping, 16, 251–270.

23.

Kline

R. B.

(2010). Principles and practice of structural equation modeling (2nd ed.). New York, NY: Guilford Press.

24.

Liebert

R. M.

Morris

L. W.

(1967). Cognitive and emotional components of test anxiety: A distinction and some initial data. Psychological Reports, 20, 975–978.

25.

Lazarus

R. S.

Folkman

(1984). Stress, appraisal, and coping. New York, NY: Springer.

26.

Lazarus

R. S.

Folkman

(1986). Cognitive theories of stress and the issue of circularity. In Appley

Trumbull

(Eds.), Dynamics of stress (pp. 63–80). New York, NY: Plenum Press.

27.

Lee

Larson

(2000). The Korean examination hell: Long hours of studying, distress, and depression. Journal of Youth and Adolescence, 29, 249–272.

28.

Mandler

Sarason

S. B.

(1952). A study of anxiety and learning. Journal of Abnormal and Social Psychology, 47, 166–173.

29.

Muthén

L. K.

Muthén

B. O.

(2010). Mplus user’s guide (6th ed.). Los Angeles, CA: Muthén & Muthén.

30.

Pekrun

Goetz

Perry

R. P.

Kramer

Hochstadt

Molfenter

(2004). Beyond test anxiety: Development and validation of the Test Emotions Questionnaire (TEQ). Anxiety, Stress, & Coping, 17, 287–316.

31.

Putwain

D. W.

(2008). Deconstructing test anxiety. Emotional and Behavioural Difficulties, 13, 141–155.

32.

Raykov

(2004). Behavioral scale reliability and measurement invariance evaluation using latent variable modeling. Behavior Therapy, 35, 299–331.

33.

Reardon

S. F.

Arshan

Atteberry

Kurlaender

(2010). Effects of failing a high school exit exam on course taking, achievement, persistence and graduation. Educational Evaluation and Policy Analysis, 34, 498–520.

34.

Richman

C. L.

Brown

Clark

(1987). Personality changes as a function of minimum competency test success or failure. Contemporary Educational Psychology, 12, 7–16.

35.

Sarason

I. G.

(1961). Test anxiety and the intellectual performance of college students. Journal of Educational Psychology, 52, 201–206.

36.

Sarason

I. G.

(1984). Stress, anxiety, and cognitive interference: Reactions to tests. Journal of Personality and Social Psychology, 46, 929–938.

37.

Sarason

I. G.

Sarason

B. R.

(1990). Test anxiety. In Leitenberg

(Ed.), Handbook of social and evaluation anxiety (pp. 475–496). New York, NY: Plenum Press.

38.

Schwarzer

Buchwald

(2003). Examination stress: Measurement and coping. Anxiety, Stress, & Coping, 16, 247–249.

39.

Seipp

(1991). Anxiety and academic performance: A meta-analysis of findings. Anxiety Research, 4, 27–41.

40.

Spielberger

C. D.

Gonzalez

H. P.

Taylor

C. J.

Anton

E. D.

Algaze

Ross

G. R.

Westberry

L. G.

(1980). Manual for the Test Anxiety Inventory (Test Attitude Inventory). Redwood City, CA: Consulting Psychologists Press.

41.

Suinn

R. M.

(1969). The STABS, a measure of test anxiety for behavior therapy: Normative data. Behaviour Research and Therapy, 7, 335–339.

42.

Stöber

(2004). Dimensions of test anxiety: Relations to ways of coping with per-exam anxiety and uncertainty. Anxiety, Stress, & Coping, 17, 213–226.

43.

Stöber

Pekrun

(2004). Advances in test anxiety research. Anxiety, Stress, & Coping, 17, 205–211.

44.

Sung

Y. T.

Tseng

F. L.

Kuo

N. P.

Chang

T. Y.

Chiou

J. M.

(2014). Evaluating the effects of programs for reducing achievement gaps: A case study in Taiwan. Asia-Pacific Education Review, 15, 99–113. doi:10.1007/s-12564-013-9304-7

45.

Taylor

Deane

F. P.

(2002). Development of a short form of the Test Anxiety Inventory (TAI). Journal of General Psychology, 129, 127–136.

46.

Wang

W. L.

Sung

Y. T.

Sung

F. C.

T. H.

C. Y.

(2008). Low birth weight, prematurity, and paternal social status: Impact on the basic competence test in Taiwanese adolescents. Journal of Pediatrics, 153, 333–338.

47.

Worthy

D. A.

Markman

A. B.

Maddox

W. T.

(2009). Choking and excelling under pressure in experienced classifiers. Attention, Perception, & Psychophysics, 71, 924–935.

48.

Wren

D. G.

Benson

(2004). Measuring test anxiety in children: Scale development and internal construct validation. Anxiety, Stress, & Coping, 17, 227–240.

49.

Yerkes

Dodson

(1908). The relation of strength of stimulus to rapidity of habit formation. Journal of Comparative Neurology and Psychology, 18, 459–482.

50.

Zeidner

(1998). Test anxiety: The state of the art. New York, NY: Plenum Press.

51.

Zell

Alicke

M. D.

(2009). Self-evaluative effects of temporal and social comparison. Journal of Experimental Social Psychology, 45, 223–227.