Psychometric Properties of the Turkish Adaptation of Teacher Emotions Scale

Abstract

The current research investigated the psychometric characteristics of the Turkish adaptation of the Teacher Emotions Scale (TES). Study I included a sample of 164 Turkish middle school teachers, and Study II included a sample of 222 mathematics teachers. The three-emotion factor model was confirmed in both studies with satisfactory reliability estimates. This model was invariant across gender. In addition, the latent mean structure analyses revealed no significant differences in teacher emotions by gender. Regarding convergent validity, teacher emotion scores showed a significant relationship with teacher self-efficacy. Thus, the Turkish version of the TES demonstrated sound psychometric characteristics for measuring teachers’ academic emotions in educational contexts.

Keywords

teacher emotions adaptation validation Turkish measurement invariance

Emotion is a complex and controversial construct, to comprehensively explain which several scientific and constitutive definitions have been proposed in the literature. For example, Rosenberg (1998) defined emotions as “acute, intense, and typically brief psychophysiological changes that result from a response to a meaningful situation in one’s environment” (p. 250). Hochschild (1990) proposed a more extended characterization of emotion as “an awareness of four elements that we usually experience at the same time: a) an appraisal of a situation, b) changes in bodily sensations, c) the free or inhibited display of expressive gesture, and d) a cultural label applied to specific constellations of the first three elements” (p. 118–119). More recently, Feldman Barrett (2012) described emotions as people’s socially constructed and subjective interpretations of experiences and the expression of these experiences via different channels. Depending on the valence of emotions, these experiences may induce various physiological, cognitive, and behavioral changes in people. For example, positive emotions may promote empathetic thinking, problem-solving ability, and acceptance of responsibility (Isen, 2008). On the other hand, negative emotions might lead to a decline in motivation and employment of rigid or superficial learning strategies rather than productive use of cognitive resources (Pekrun, 2006; Pekrun & Linnenbrick-Garcia, 2014). Whatever their effect, the critical role of emotions in people’s lives cannot be denied.

From this perspective, teaching is an emotion-laden job, particularly with regard to teacher–student, teacher–teacher, teacher–parent, and teacher–administrator interactions, and teachers experience various emotional states during their professional careers. Bahia et al. (2013) identified 160 teacher emotions experienced by teachers in Portuguese classrooms, ranging from joy and love to sadness, fear, anger, as well as surprise. In a study of teachers’ journal writing, O’Toole et al. (2010) found 113 emotions within a period of 5 days of entries, most expressed by tertiary teachers. The differential nature of teacher emotions was also explored by Prosen et al. (2014). Although primary school teachers experienced both positive and negative emotions, the latter seemed to outweigh the former, especially anger, which was highest on the list. Anttila et al.’s (2016) examination of teacher candidates’ emotions corroborated previous findings of many distinct emotions, including enthusiasm, satisfaction, and interest, but also disappointment and inadequacy. The experience of varying emotions in academic settings is found to be unavoidable for both pre-service and in-service teachers. However, the intensity and the valence of these emotions may differ across several conditions. Accordingly, teachers enjoy positive teacher-student relationships; feel gratified by high levels of student motivation, engagement, and success (Becker et al., 2015; Hagenauer et al., 2015; Hosotani & Imai-Matsumura, 2011; Khajavy et al., 2018; Sutton & Wheatley, 2003); take pride in students’ accomplishments, whether expected or unexpected; and be encouraged by students’ positive feedback (Buechner et al., 2016; Khajavy et al., 2018). On the other hand, teachers may experience anxiety due to inadequate teaching preparation, feel ineffective when unable to respond to students’ questions, or lack confidence as novices in the profession (Khajavy et al., 2018; Sutton & Wheatley, 2003). Teachers may also feel anger because of classroom management and discipline problems, student disengagement, and colleagues’ unsupportive behaviors (Becker et al., 2015; Chang, 2009; Hagenauer et al., 2015; Hargreaves, 2000; Hosotani & Imai-Matsumura, 2011; Khajavy et al., 2018; Sutton & Wheatley, 2003); sadness about students’ failures (Hosotani & Imai-Matsumura, 2011); shame due to feeling responsible for their students’ failures (Khajavy et al., 2018); and disengagement due to students’ lack of interest and motivation.

In addition to these potential sources, teacher emotions may be related to a wide range of cognitive and psychological constructs such as teachers’ pedagogical content-knowledge formation (Brigido et al., 2013), their self-efficacy (Brigido et al., 2013; Borrachero et al., 2013; Buric & Macuka, 2017; Chen, 2018; De Mauro & Jennings, 2016; Stephanou et al., 2013), their teacher identity (Bair et al., 2010), their teaching quality (Chen, 2019; Frenzel et al., 2015; 2016; Hagenauer & Volet, 2014; Klassen et al., 2011; Sutton & Wheatley, 2003; Taxer & Frenzel, 2015), and their interactions with students (Becker et al., 2014; Chen, 2019; Hagenauer et al., 2015; Yan et al., 2011).

It is important to investigate both potential sources and consequences of teachers’ academic emotions. Toward this end, several scales have been developed to measure teachers’ emotions, many of which are adapted to different languages and cultures. For example, Chen (2016) developed the Teacher Emotion Inventory (TEI) and validated it in Asian contexts. The TEI included the emotions of sadness, anger, fear, and love. Exploratory and confirmatory factor analyses resulted in five emotional dimensions for primary school teachers. Positive emotions were mainly related to interactions with students and colleagues. In contrast, negative emotions were linked to educational policy changes and teachers’ difficulty preserving a balance in their lives. Therefore, the TEI seems less comprehensive and may provide a limited measure of teachers’ emotions while interacting with students. Frenzel et al. (2016) adopted a discrete trait-based approach in developing the Teacher Emotions Scale (TES), taking into account the frequency and saliency of real-life emotions. Accordingly, the instrument consists of 24 items with a four-point Likert scale in two sections, addressing teacher enjoyment, anger, and anxiety in general and with regard to a specific group of students. To assess construct validity, Frenzel et al. (2016) tested the TES with a single-emotion factor model, a two-factor (positive vs. negative affect) model, and a three-emotion factor (enjoyment, anger, and anxiety) model with different groups of participants. The factor analyses supported the three-emotion factor model, which yielded satisfactory internal consistency estimates for each dimension. Hong et al. (2016) revised the TES by adding pride and frustration and validated the revised form with Japanese and Korean teachers. However, frustration items cross-loaded with anger dimension and displayed low reliability.

Prevalent research on discrete emotion categories (e.g., Henschel & Roick, 2018; Skiffington et al., 1998) stresses the complex structure of emotions, so there is a need to explore the structure of teacher emotions with accurate measures for different cultures. As Frenzel et al.’s (2016) scale takes into account the frequency and saliency of teachers’ real-life emotions only in interactions with students, the TES seems quite specific. Accordingly, the present research aimed to adapt the TES to the Turkish language and obtain evidence concerning its psychometric characteristics in Turkish educational contexts.

Methods

Translation and Adaptation Process

Merenda (2006) described three basic steps to satisfy item and test equivalence in the scale adaptation process. Following those steps, the TES items and response scales were first reviewed from emic and etic perspectives before translation. The TES was then translated into the Turkish language by the researcher and two bilingual experts. The “decentering” method rather than a literal translation of each item was employed. The Turkish translation of the scale was then back-translated to English by three experts who had a good command of English. The original and back-translated versions were compared to ascertain that the latter corresponded to the original scale. Problems were found with the equivalency of the translated versions for the phrases “feeling annoyed” (item 3), “being frustrated” (item 5), and “having much fun” (item 6). Merenda emphasized that some items may not “be directly transferrable” (p. 310) and may need modification or replacement. Thus, three experts in Guidance and Psychological Counseling were asked to clarify the meanings of those words to make a well-informed recommendation for their translation. The experts agreed on the best expressions to match the original scale. In addition, before finalizing the instrument, cognitive interviews were carried out with two former mathematics teachers, in which the interviewees were asked to identify unclear words, phrases, or sentences in the scale. They also commented on the format and the design of the instrument. They found no problematic items and considered the scale’s length and appearance appropriate. Therefore, no changes were made to the final version after the interviews.

Participants

Cluster sampling, in which intact groups are randomly selected (Frankel et al., 2019), was employed in recruiting a different sample of middle school teachers. Two independent samples were selected: the first sample was used for Study I—pilot study, whereas the second sample was used for Study II—main study.

Sample for Study I

The Teacher Emotions Scale (TES) was administered to 164 subject areas (Turkish, math, science, and social studies) teachers (67.7% female) from 16 public middle schools in four central districts of a metropolitan city in Turkiye. Eighty-two (50%) had ten or fewer years of teaching experience, 49 (29.9%) had 11–20 years, and 25 (15.2%) had 21 or more years of teaching experience. Eight teachers did not mention their years of teaching experience. Regarding the faculties from which they graduated, 121 (73.8%) were graduates of Education Faculties, and 33 (20.1%) were Faculty of Arts and Science or Open University graduates with a Pedagogical Formation Certificate for teaching. 10 teachers did not mention their graduation faculty. Besides, 140 teachers had a bachelor’s degree (85.4%), while 16 teachers completed a graduate-level program and had a master’s or doctoral-level degree (9.7%). Eight teachers did not specify their level of education.

Sample for Study II

Because teachers in different subject areas might experience different emotions, the second study sample was restricted to one subject area, mathematics, to control for such variation. It was assumed that mathematics teachers might experience particular issues because of the abstract nature of the discipline, so the scales were administered to 222 middle school mathematics teachers (66.7% female) from 53 public middle schools in 11 central districts of a metropolitan city in Turkiye. The majority of the teachers had bachelor’s degrees (n = 195, 87.8%), 26 (11.7%) had master’s degrees, and one (0.50%) had an Associate degree. Most had graduated from a Faculty of Education (n = 142, 64%), while some had graduated from a Faculty of Arts and Science (n = 55, 24.8%) and other faculties (n = 3, 1.4%). The remaining 22 teachers (9.9%) did not provide information about their faculty of graduation. Participants had 11 years of teaching experience on average (SD = 7.76), ranging from 1 month to 41 years.

Measures

Demographics information

In both study samples, demographic information included the teachers’ gender, age, latest education degree, faculty from which they graduated, and years of teaching experience.

Teacher Emotions Scale

Teacher Emotions Scale (Frenzel et al., 2016) is a multidimensional self-report instrument measuring three emotions: enjoyment (4 items, e.g., “I teach these students with enthusiasm”), anger (4 items, e.g., “I often have reason to be angry while I teach these students”), and anxiety (4 items, e.g., “I feel uneasy when I think about teaching these students”). Teachers respond using a 4-point Likert scale ranging from “strongly disagree” (1) to “strongly agree” (4). The questionnaire has two sections, 12 items addressing teaching in general and 12 addressing teaching a specific student group, to allow for differentiation between overall emotions about teaching and emotions while teaching a specific group. The instructions for responding in the second section direct teachers to think about the students in one of the classes they are currently teaching. The items are the same in both sections except for the addition of the phrase “these students” at the end of each item in the second section. In the present study, only the second section was used.

Frenzel et al. (2016) constructed a three-emotion factor structure based on the discrete emotion approach. For the English version, the three-emotion factor model of the general scale has the following goodness-of-fit indices: Comparative Fit Index (CFI) =.943, Standard Root Mean Square Residual (SRMR) = .044, and Root Mean Square Error of Approximation (RMSEA) = .068. The English version of the student-group specific scale has the following goodness-of-fit indices: CFI = .976, SRMR = .030, and RMSEA = .052. Thus the three-emotion factor model has satisfactory goodness of fit. Also, Cronbach’s alpha coefficients for the general emotions sub-scale were .73 for enjoyment, .80 for anger, and .81 for anxiety, and for the student-group specific emotions sub-scale, they were .80 for enjoyment, .87 for anger, and .87 for anxiety, all above the .70 threshold.

Teachers’ Sense of Efficacy Scale

Teachers’ Sense of Efficacy Scale was developed by Tschannen-Moran and Woolfolk Hoy (2001) to measure in-service and pre-service teacher self-efficacy beliefs. TSES was designed to have a nine-point rating scale, with responses varying from 1 (nothing) to 9 (a great deal). The scale has long (24 items) and short (12 items) versions, including both of which are composed of three dimensions: self-efficacy for student engagement (SE; 8 items, e.g., “How much can you do to help your students value learning?”), self-efficacy for instructional strategies (IS; 8 items, e.g., “How well can you implement alternative strategies in your classroom?”), and self-efficacy for classroom management (CM; 8 items, e.g., “How much can you do to get children to follow classroom rules? ”) In study II, the Turkish adaptation of the long version of TSES (Çapa et al., 2005) was used. The Cronbach’s alpha estimates were .85 for SE, .89 for IS, and .93 for CM.

Procedure

The permissions for the study were granted by the Human Subjects Ethics Committee (HSEC) of the host university and the Ministry of National Education in Turkiye. Study I was carried out in the Fall semester, and study II in the Spring semester of the 2018–19 academic year. Informed consent to participate was obtained from the teachers, who were informed of the purposes and procedures and their rights as participants before data collection. They were requested to fill out the scales with one of their classes in mind.

Data Analysis

Confirmatory factor analyses (CFA) were performed through Mplus 8.1 (Muthen & Muthen, 2017) to validate the factorial structure of TES in Turkish educational contexts. Consistent with the original scale (Frenzel et al., 2016), three models were tested in both studies. Accordingly, one-emotion factor model (Model I), a two-factor model (positive vs. negative affect, Model II), and the three-emotion factor models (enjoyment, anger, and anxiety, Model III) were tested (Figure 1). To evaluate the model fit, Kline (2016) recommends giving estimates of model Chi-Square test value, Steiger-Lind Root Mean Square Error of Estimation (RMSEA), Bentler Comparative Fit Index (CFI), and Standardized Root Mean Square Residual (SRMR) values. In the present study, chi-square values (χ²) and SRMR as absolute fit indices, CFI and NNFI values as relative fit indices, and RMSEA values as evidence of non-centrality fit indices were utilized (Kline, 2016). To adjust for non-normality, the Satorra–Bentler corrected χ2 values (Satorra & Bentler, 1994) were calculated. Reliability analyses were conducted using SPSS 22 (IBM SPSS Corp., 2013).

Figure 1.

CFA Models for Teacher Academic Emotions. Upper part: Model I (one-emotion factor model), Middle part: Model II (two-factor model), Lower part: Model III (three-emotions factor model). Enj represents enjoyment, anx=anxiety, ang=anger.

In addition, multigroup CFA was conducted to examine measurement invariance of the scale with female and male teachers using the sample of 222 middle school mathematics teachers in Study II. This analysis involved several steps starting with testing the model for male and female groups separately (Byrne, 2013). The following steps included models ordered in an increasingly restrictive manner and these nested models were compared using chi-square difference test. A non-significant difference was an indicator of invariance across groups. Difference in the CFI (ΔCFI) was also used, where difference less than <.010 showed invariance (Chen, 2007). In Model 1 (configural), no equality constraints are imposed, and this model serves as a reference model. Model 2 (metric invariance) imposes invariance of factor loadings across gender. In Model 3 (scalar invariance), both factor loadings and intercepts are treated as invariant across gender. Model 4 (strict invariance) adds the constraint of invariant residuals in addition to factor loadings and intercepts. Meredith and Teresi (2006) described last three steps as weak, strong, and strict tests of invariance, respectively. Testing strict invariance is considered unnecessary in practice (Wang & Wang, 2019). Therefore, strict invariance was not tested in the present study. Finding the metric and scalar invariance model tenable allows for researcher to compare group mean differences on the latent constructs (Meredith & Teresi, 2006). Next, equivalence of latent means across gender was tested via Mplus 8.1 to provide construct validity evidence for multigroup differences.

A canonical correlation was performed using the data collected in Study II to provide convergent validity evidence through SPSS 22. In light of the proposed relationship in the literature between emotion and self-efficacy, the relationship between the dimensions of TSES (i.e., SE, IS, and CM) and TES (i.e., enjoyment, anger, and anxiety) was investigated. Also, meaningful canonical variate(s), canonical loadings, and the redundancy indices were inspected for further validity.

Results

Factorial Validity and Reliability

In Study I, CFA with Satorra–Bentler corrected chi-square test revealed significant results for three models. For a good model fit, CFI and NNFI should be above .95, and SRMR should be below .08 (Hu & Bentler, 1999). For RMSEA, the values should be less than .05 for a good model fit, while those within the range of .05 and .08 reflect a mediocre fit (Browne & Cudeck, 1993). As presented in Table 1, Model II (two-factor model or positive vs. negative affect model) and Model III (three-emotion factor model) fit better than Model I (one-emotion factor model). The standardized estimates for Model II and Model III were found to be above .40 in both models (Appendix 1).

Table 1.

Confirmatory Factor Analysis Results for Study I and II.

	Models	χ²	df	CFI	NNFI	RMSEA	SRMR
Study I	Model I	470.189	55	.554	.464	.235	.930
	Model II	87.226	53	.963	.954	.069	.045
	Model III	83.697	51	.965	.955	.068	.046
Study II	Model I	470.189	55	.554	.464	.235	.930
	Model II	87.226	53	.963	.954	.069	.045
	Model III	98.896	51	.969	.959	.065	.053

In Study II, CFA with Satorra–Bentler corrected chi-square test yielded significant results for three models. As displayed in Table 1, Model II and Model III fit the data better than Model I, corroborating the Study I results. However, Model III fit slightly better when CFI, NNFI, and RMSEA values were compared. Therefore, Model III was chosen, and the standardized estimates are presented in Appendix 2. All of the loadings were .40 and above.

Cronbach’s alpha coefficients were examined for reliability. As Model III fit better than the other models, internal consistency estimates were computed for each emotion dimension. Cronbach’s alphas were .92 and .90 for enjoyment, .75 and .75 for anger, and .87 and .87 for anxiety in study I and II, respectively.

Measurement Invariance

The measurement invariance of the TES according to gender was tested through multigroup CFA. As previously noted, configural, metric, and scalar invariances were tested in a nested manner, thus allowing for comparisons in chi-square and CFI differences. Fit indices for factor invariance testing of the three-factor model across gender and the difference in chi-square and CFI values between the models are presented in Table 2. As suggested by Byrne (2013), analysis began with testing the model for male and female groups separately. The overall fit of the model for male groups [χ² (51) = 66.81, p = .068, CFI = .954, NNFI = .940, RMSEA = .065] and female groups [χ² (51) = 101.04, p = .00, CFI = .953, NNFI = .939, RMSEA = .081] were satisfactory. The configural model also fit the data well. Furthermore, findings provided an evidence of model invariance across gender as revealed by non-significant chi-square difference and less than .01 CFI change: metric invariance [Δχ² (9) = 15.69, p = .073, ΔCFI = .004] and scalar invariance [Δχ² (12) = 9.17, p = .689, ΔCFI = .004].

Table 2.

Measurement Invariance of the TES across Gender.

Models	χ²	df	∆χ²	∆df	CFI	∆CFI
Male teachers	66.81	51	-	-	.954	-
Female teachers	101.04	51	-	-	.954	-
Model 1 (configural)	214.35	102	-	-	.936	-
Model 2 (metric invariance)	230.03	111	15.69	9	.932	.004
Model 3 (scalar invariance)	239.20	123	9.17	12	.934	.002

Note. None of the ∆χ² was significant at 0.05. CFI = Comparative Fit Index.

To compare latent mean differences by gender, the strong invariance model was examined by setting the means of the three factors as 0 for men and free for women. Analysis showed that men and women did not differ on enjoyment (z = 1.01, p = .31), anger (z = 0.16, p = .87), and anxiety (z = −0.19, p = .85).

Convergent Validity

For convergent validity evidence, a canonical correlation was also performed with the sample of study II to determine the potential relationships between the dimensions of TES and TSES. The relationships between sets of multiple dependent and independent variables were examined through canonical correlation analysis (Hair et al., 2019). The first set consisted of three-emotion dimensions (i.e., enjoyment, anxiety, anger), and the second set included self-efficacy for student engagement (SE), self-efficacy for instructional strategies (IS), and self-efficacy for classroom management (CM). In accordance with the literature, a positive relationship was expected between enjoyment and TSES dimensions, and a negative relationship between either anxiety or anger and TSES dimensions.

As displayed in Table 3, the first canonical correlation was .37 (14% overlapping variance), χ² (9) =33.28, p<.001. As the first canonical pair was above .30 and accounts for more than 14% of the overlapping variance, the first solution was considered for interpretation. Accordingly, enjoyment was positively correlated with the first canonical variate based on the cut-off value of .30 (Tabachnick & Fidell, 2019). In other words, the higher the level of enjoyment, the higher the self-efficacy for instructional strategies, classroom management, and student engagement. On the other hand, anxiety and anger were negatively related to teacher self-efficacy. An increase in anxiety and anger yielded a decline in self-efficacy for instructional strategies, classroom management, and student engagement.

Table 3.

Canonical Correlation Results.

	First Canonical Variate
	Canonical Loadings	Standardized Coefficients
Teacher emotions
Enjoyment	.98	.79
Anxiety	−.80	−.30
Anger	−.68	.02
Percentage of variance	.69
Redundancy	.09
Teacher self-efficacy
Self-efficacy for instructional strategies	.53	.44
Self-efficacy for classroom management	.78	.57
Self-efficacy for student engagement	.91	.86
Percentage of variance	.57
Redundancy	.08
Canonical correlation	.37

According to the variances within and between sets for the first canonical variate, 69% of the variance was explained by the teacher emotions set. In comparison, the teacher self-efficacy set accounted for 57%. Based on the redundancy results, while the teacher emotions set explained 9% variance in the teacher self-efficacy set, the teacher self-efficacy set explained 8% variance in the teacher emotions set.

Discussion

For this research, the TES was translated into the Turkish language. Confirmatory factor analyses and reliability analyses were performed to determine the scale’s psychometric properties in two studies. In line with the original scale, three models were tested, among which Model II (the two-factor model or positive vs. negative affect model) and Model III (the three-emotion factor model) fit the data better than Model I (one-emotion factor model) in both studies. Frenzel et al. (2016) reported similar findings, particularly the high fit indices for Model III. Although the fit indices seemed to be at the edge of criteria for both German and Canadian samples, Model III results best fit the data. Also, in the current research, Model III seemed to fit the data better when indices were compared in both studies. The reliability coefficients were also above the cut-off criterion (above .70) for each emotion dimension (Nunnally, 1978).

Although significant differences in teachers’ academic emotions regarding gender have been found in some studies (e.g., Borrachero et al., 2014; Kimura, 2010; Lee et al., 2016), many have indicated no difference (e.g., Hargreaves, 1998; Mattern & Bauer, 2014; Sutton, 2004; Yin, 2015). Given these mixed findings, no conclusions can be drawn in the literature about gender differences in teachers’ academic emotions. Strong measurement invariance across gender was observed in the present study that indicated the TES was assessing the same latent constructs in men and women. Therefore, mean values can be effectively compared across genders. No significant difference was found based on latent mean structure analysis, adding additional support for the use of the TES.

Also, convergent validity was examined by looking at the relationship between teacher emotions and teacher self-efficacy. According to canonical correlation results, enjoyment had a positive relationship with teacher self-efficacy dimensions, whereas anxiety and anger had a negative relationship. This finding was in line with the literature, confirming that self-efficacy was positively associated with positive emotions and negatively associated with negative emotions (e.g., Brigido et al., 2013; Borrachero et al., 2013; Buric & Macuka, 2017; Chen, 2018; De Mauro & Jennings, 2016; Stephanou et al., 2013).

Overall, the Turkish version of the TES displayed appropriate psychometric characteristics for measuring teachers’ academic emotions in Turkish educational contexts. Validity and reliability findings supported the assumption that the scale reflected the three dimensions of the original scale: enjoyment, anger, and anxiety (Frenzel et al., 2016). As the first instrument to systematically measure teachers’ academic emotions, TES assesses teachers’ trait-based retrospective judgments about their emotional experiences in non-obtrusive, economical, and easy ways (Frenzel et al., 2016). Although there is still a debate on differentiating trait- and state-based emotions (Nett et al., 2017), the trait-based scale in this study could also be converted into state-based assessments for future studies. However, the limited number of emotions (i.e., anger, anxiety, enjoyment) in the scale might cause confusion by focusing on the valence of emotions as positive or negative rather than highlighting their distinctive nature. That possibility might help explain the higher index values in Model II. Indeed, a bulk of the research in the literature stressed the differential nature of teacher emotions (e.g., Frenzel et al., 2015; Hagenauer et al., 2015; Šarić, 2015; Sutton & Wheatley, 2003). To explore this direction further, the number of emotions might be increased in future studies.

In the present study, the adapted version of the TES was administered in Turkish educational contexts. Thus, the participants had different socio-cultural characteristics from those of the teachers who participated in administering the original scale. The results were similar in the two settings, providing some evidence about the universal nature of emotions. The availability of the scale in different languages might advance research in this field to better understand emotions and to examine how teachers regulate and cope with their emotions, which, in turn, might suggest solutions to the problems related to emotional exhaustion frequently experienced by teachers (Karakelle & Canpolat, 2008; Yıldız Kırılmaz et al., 2003; Seferoğlu et al., 2014). Furthermore, the scale might be used to measure teacher emotions associated with a specific group of students to avoid overgeneralization of their emotions. As emotions are unstable and highly specific for each person and condition, general emotion scales might not reveal teachers’ actual emotional states in particular settings and situations. In this regard, the TES-student group-specific scale can enable researchers to detect teachers’ academic emotions with a maximum level of specificity. Moreover, the scale might be administered to teachers in different subject areas and at different grade levels if the instructions are adapted accordingly.

Emotion, especially teacher emotion, is a nascent field. Further development requires more research in this area to fully explore possible antecedents and consequences of teachers’ emotions and their relations with the emotions of other stakeholders in education (i.e., students, administrators, and parents). In future studies, in addition to convergent validity, the divergent validity of TES should be examined to provide sufficient evidence for the external validity of the instrument. Also, test–retest reliability evidence could be provided to confirm the stability of the findings.

Footnotes

Author Note

This study was just a part of the Doctoral Dissertation of the corresponding author under the supervision of the second author at Middle East Technical University.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the Middle East Technical University-Scientific Research Projects Coordination Unit (BAP) [METU-BAP-YLT-502-2018-2766] and Turkish National Science Foundation (TUBITAK) 2214-A International Research Fellowship Programme.

ORCID iD

Basak Calik

Appendix I

Standardized Estimates of Items across Model II and Model III for Study 1

Model II				Model III
Dimension	Items	Standardized Estimates	Dimension	Items	Standardized Estimates
Positive	Item 11	.89	Enjoyment	Item 11	.89
	Item 6	.89		Item 6	.89
	Item 7	.84		Item 7	.84
	Item 1	.81		Item 1	.81
Negative	Item 12	.87	Anxiety	Item 12	.88
	Item 5	.83		Item 10	.80
	Item 10	.81		Item 8	.65
	Item 9	.79		Item 2	.43
	Item 4	.78	Anger	Item 5	.84
	Item 3	.76		Item 4	.80
	Item 8	.68		Item 9	.79
	Item 2	.43		Item 3	.77

Appendix II

Standardized Estimates of Items across Model III for Study 2

Dimension	Items	Standardized Estimates
Enjoyment	Item 11	.91
	Item 6	.86
	Item 1	.83
	Item 7	.76
Anxiety	Item 12	.84
	Item 10	.69
	Item 8	.59
	Item 2	.48
Anger	Item 5	.87
	Item 4	.83
	Item 3	.75
	Item 9	.74

Appendix III

References

Anttila

Phyhaltö

Soins

Pietariren

(2016). How does it feel to become a teachers? Emotions in teacher education. Social Psychology of Education, 19(3), 451–473. https://doi.org/10.1007/s11218-016-9335-0

Bahia

Freire

Amaral

Estrela

M. T.

(2013). The emotional dimension of teaching in a group of Portuguese teachers. Teachers and Teaching, 19(3), 275–292. https://doi.org/10.1080/13540602.2012.754160

Bair

M. A.

Bair

D. E.

Mader

C. E.

Hipp

Hakim

(2010). Faculty emotions: A self-study of teacher educators. Studying Teacher Education, 6(1), 95–111. https://doi.org/10.1080/17425961003669490

Becker

E. J.

Goetz

Morger

Ranelluci

(2014). The importance of teachers’ emotions and instructional behaviour for their students’ emotions- an experience sampling analysis. Teaching and Teacher Education, 43, 15–26. https://doi.org/10.1016/j.tate.2014.05.002

Becker

E. S.

Keller

M. M.

Goetz

Frenzel

A. C.

Taxer

J. L.

(2015). Antecedents of teachers’ emotions in the classroom: An intraindividual approach. Frontiers in Psychology, 6(635), 1–12. https://doi.org/10.3389/fpsyg.2015.00635

Borrachero

A. B.

Brıgido

Costillo

Bermejo

M. L.

Mellado

(2013). Relationship between self-efficacy beliefs and emotions of future teachers of Physics in secondary education. Asia-Pacific Forum on Science Learning and Teaching, 14(2), 1–11.

Borrachero

A. B.

Brígido

Mellado

Costillo

Mellado

(2014). Emotions in prospective secondary teachers when teaching science content, distinguishing by gender. Research in Science & Technological Education, 32(2), 182–215. https://doi.org/10.1080/02635143.2014.909800

Brígido

Borroachero

A. J.

Bermejo

M.L.

Mellado

(2013). Prospective primary teachers’ self-efficacy and emotions in science teaching. European Journal of Teacher Education, 36(2), 200–217. https://doi.org/10.1080/02619768.2012.686993

Browne

M. W.

Cudeck

(1993). Alternative ways of assessing model fit. In Bollen

K. A.

Long

J. S.

(Eds), Testing structural equation models (pp. 136–162). Sage.

10.

Buechner

V. L.

Pekrun

Lichtenfeld

(2016). The achievement pride scales (APS). European Journal of Psychological Assessment, 34(3), 181–192. https://dx-doi-org.web.bisu.edu.cn/10.1027/1015-5759/a000325

11.

Buric

Macuka

(2017). Self-efficacy, emotions and work engagement among teachers: A two wave cross-lagged analysis. Journal of Happiness Studies, 19(7), 1917–1933. https://doi.org/10.1007/s10902-017-9903-9

12.

Byrne

B. M.

(2013). Structural equation modeling with Mplus. Basic concepts, applications and programming. Routledge, Taylor & Francis Group.

13.

Çapa

Çakıroğlu

Sarıkaya

(2005). The development and validation of a Turkish version of teachers’ sense of efficacy scale. Education and Science, 30(137), 74–81.

14.

Chang

M. L.

(2009). An appraisal perspective of teacher burnout: Examining the emotional work of teachers. Educational Psychology Review, 21(3), 193–218. https://doi.org/10.1007/s10648-009-9106-y

15.

Chen

F. F.

(2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14, 464–504. https://doi.org/10.1080/10705510701301834

16.

Chen

(2016). Understanding teacher emotions: The development of a teacher emotion inventory. Teaching and Teacher Education, 55, 68–77. https://doi.org/10.1016/j.tate.2016.01.001

17.

Chen

(2018). Efficacious and positive teachers achieve more: Examining the relationship between teacher efficacy, emotions, and their practicum performance. The Asia-Pacific Education Researcher, 28, 327–339. https://doi.org/10.1007/s40299-018-0427-9

18.

Chen

(2019). Exploring the impact of teacher emotions on their approaches to teaching: A structural equation modelling approach. British Journal of Educational Psychology, 89, 57–74. https://doi.org/10.1111/bjep.12220

19.

De Mauro

A. A.

Jennings

P. A.

(2016). Pre-service teachers’ efficacy beliefs and emotional states. Emotional and Behavioral Difficulties, 21(1), 119–132. https://doi.org/10.1080/13632752.2015.1120057

20.

Feldman Barrett

(2012). Emotions are real. Emotion, 12(3), 413–429. https://doi.org/10.1037/a0027555

21.

Frankel

J. R.

Wallen

N. E.

Hyun

H. H.

(2019). How to design and evaluate research in education (10th ed.). McGraw-Hill.

22.

Frenzel

A. C.

Becker-Kurz

Pekrun

Goetz

(2015). Teaching this class drives me nuts!-examining the person and context-specificity of teacher emotions. Plos One, 10(6), 1–15. https://doi.org/10.1371/journal.pone.0129630

23.

Frenzel

A. C.

Pekrun

Goetz

Daniels

L. M.

Durksen

T. L.

Becker-Kurz

Klassen

R. M.

(2016). Measuring teachers’ enjoyment, anger, and anxiety: The teacher emotions scale (TES). Contemporary Educational Psychology, 46, 148–163. https://doi.org/10.1016/j.cedpsych.2016.05.003

24.

Hagenauer

Hascher

Volet

S. E.

(2015). Teacher emotions in the classroom: Associations with students’ engagement, classroom discipline and the interpersonal teacher-student relationship. European Journal of Psychology of Education, 30(4), 385–403. https://doi.org/10.1007/s10212-015-0250-0

25.

Hagenauer

Volet

S. E.

(2014). I don’t think, I could, you know, just teach without any emotion”: Exploring the nature and origin of university teachers’ emotions. Research Papers in Emotion, 29(2), 240–262. https://doi.org/10.1080/02671522.2012.754929

26.

Hair

J. F.

Anderson

R. E.

Tatham

R. L.

Black

W. C.

(2019). Multivariate data analysis (8th ed.). Prentice-Hall.

27.

Hargreaves

(1998). The emotional practice of teaching. Teaching and Teacher Education, 14(8), 835–854. https://doi.org/10.1016/s0742-051x(98)00025-0

28.

Hargreaves

(2000). Mixed emotions: Teachers' perceptions of their interactions with students. Teaching and Teacher Education, 16, 811–826. https://doi.org/10.1016/S0742-051X(00)00028-7

29.

Henschel

Roick

(2018). The multidimensional structure of math anxiety revisited. European Journal of Psychological Assessment, 36(1), 123–135. https://doi.org/10.1027/1015-5759/a000477

30.

Hochschild

A. R.

(1990). Ideology and emotion management. A perspective and path for future research. In Kemper

T. D.

(Ed), Research agendas in the sociology of emotions (pp. 117–134). State University of New York Press.

31.

Hong

Heddy

Ruan

You

Kambara

(2016). Revising and validating achievement emotions questionnaire -teachers (AEQ-T). International Journal of Educational Psychology, 5(1), 80–107. https://doi.org/10.17583/ijep.2016.1395

32.

Hosotani

Imai-Matsumura

(2011). Emotional experience, expression, and regulation of high quality Japanese elementary school teachers. Teaching and Teacher Education, 27(6), 1039–1048. https://doi.org/10.1016/j.tate.2011.03.010

33.

L. H.

Bentler

P. M.

(1999). Cut off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–15. https://doi.org/10.1080/10705519909540118

34.

IBM SPSS Corp (2013). IBM SPSS Statistics for Windows, Version 22.0 [Computer Software]: IBM Corp.

35.

Isen

A. M.

Lewis

Haviland-Jones

J. M.

Feldman Barrett

(2008). Some ways in which positive affect influences decision making and problem solving. In: Handbook of emotions (pp. 548–573). The Guilford Press.

36.

Karakelle

Canpolat

(2008). Analyzing the student relationship styles of the primary school teachers with high burnout level. Education and Science, 33(147), 106–120. https://www.proquest.com/openview/6b1693f1e03ee84ff063d5418ea0e1b3/1?pq-origsite=gscholar&cbl=1056401

37.

Khajavy

G. H.

Ghonsooly

Fatemi

A. H.

Frenzel

A. C.

(2018). Antecedents of pleasant and unpleasant emotions of EFL teachers using an appraisal-theoretical framework. Iranian Journal of Language Teaching Research, 6(2), 39–55. https://files.eric.ed.gov/fulltext/EJ1183768.pdf

38.

Kimura

(2010). Expressing emotions in teaching: Inducement, suppression, and disclosure as caring profession. Educational Studies in Japan: International Yearbook, 5, 63–78.

39.

Klassen

R. M.

Tze

V. M. C.

Betts

S. M.

Gordon

K. A.

(2011). Teacher efficacy research 1998-2009: Signs of progress or unfulfilled promise? Educational Psychology Review, 23(1), 21–43. https://doi.org/10.1007/s10648-010-9141-8

40.

Kline

R. B.

(2016). Principles and practice of structural equation modeling (4th ed.). The Guilford Press.

41.

Lee

Pekrun

Taxer

J. L.

Schutz

P. A.

Vogl

Xie

(2016). Teachers' emotions and emotion management: Integrating emotion regulation theory with emotional labor research. Social Psychology of Education, 19(4), 843–863. https://doi.org/10.1007/s11218-016-9359-5

42.

Mattern

Bauer

(2014). Does teachers' cognitive self-regulation increase their occupational well-being? The structure and role of self-regulation in the teaching context. Teaching and Teacher Education, 43, 58–68. https://doi.org/10.1016/j.tate.2014.05.004

43.

Meredith

Teresi

J. A.

(2006). An essay on measurement and factorial invariance. Medical Care, 44, 69–77.

44.

Merenda

P. F.

(2006). An overview of adapting educational and psychological assessment instruments: Past and present. Psychological Reports, 99, 307–314. https://doi.org/10.2466/pr0.99.2.307-314

45.

Muthen

L. K.

Muthen

B. O.

(2017). MPlus8.1 [Computer Software]: Muthen &Muthen.

46.

Nett

U. E.

Bieg

Keller

M. M.

(2017). How much trait variance is captured by measures of academic state emotions? A latent state-trait analysis. European Journal of Psychological Assessment, 33(4), 239–255. https://doi.org/10.1027/1015-5759/a000416

47.

Nunnally

J. C.

(1978). Psychometric theory (2nd ed.). McGraw-Hill.

48.

O’Toole

Ogier-Price

Hucks

(2010). Diary study of tertiary teachers’ emotions. Journal of Adult Learning, 38(1), 23–43. https://ir.canterbury.ac.nz/handle/10092/15077

49.

Pekrun

(2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4), 315–341. https://doi.org/10.1007/s10648-006-9029-9

50.

Pekrun

Linnenbrick-Garcia

(2014). Introduction to emotions in education. In Pekrun

Linnenbrick-Garcia

(Eds), International handbook of emotions in education (pp. 1–109). Routledge.

51.

Prosen

Smrtnik Vitulic

Poljsak Skraban

(2014). Teachers’ emotional expression in the classroom. Hacettepe University Journal of Education, 29(1), 226–237. http://www.efdergi.hacettepe.edu.tr/yonetim/icerik/makaleler/114-published.pdf

52.

Rosenberg

E. L.

(1998). Levels of analysis and the organization of affect. Review of General Psychology, 2(3), 247–270. https://psycnet.apa.org/doi/10.1037/1089-2680.2.3.247

53.

Šarić

(2015). Teachers’ emotions: A research review from a psychological perspective. Journal of Contemporary Educational Studies, 4, 10–26. https://www.sodobna-pedagogika.net/en/archive/load-article/?id=1133

54.

Satorra

Bentler

P. M.

(1994). Corrections to test statistics and standard errors in covariance structure analysis. In A. von Eye & C. C. Clogg (Eds.), Latent variables analysis: Applications for developmental research (pp. 399–419). Sage Publications, Inc.

55.

Seferoğlu

S. S.

Yıldız

Avcı Yüksel

Ü.

(2014). Teachers’ burnout: Indicators of burnout and investigation of the indicators in terms of different variables. Education and Science, 174, 348–364. https://doi.org/10.15390/EB.2014.2515

56.

Skiffington

Fernandez

McFarland

(1998). Towards a validation of multiple features in the assessment of emotions. European Journal of Psychological Assessment, 14(3), 202–210. https://psycnet.apa.org/doi/10.1027/1015-5759.14.3.202

57.

Stephanou

Gkavras

Doulkeridou

(2013). The role of teachers’ self-and collective-efficacy beliefs on their job satisfaction and experienced emotions in school. Psychology, 4(3A), 268–278. https://psycnet.apa.org/doi/10.4236/psych.2013.43A040

58.

Sutton

R. E.

(2004). Emotional regulation goals and strategies of teachers. Social Psychology of Education, 7(4), 379–398. https://doi.org/10.1007/s11218-004-4229-y

59.

Sutton

R. E.

Wheatley

K. F.

(2003). Teachers' emotions and teaching: A review of the literature and directions for future research. Educational Psychology Review, 15(4), 327–358. https://doi.org/10.1023/A:1026131715856

60.

Tabachnick

B. G.

Fidell

L. S.

(2019). Using multivariate statistics (7th ed.). Allyn & Bacon.

61.

Taxer

J. L.

Frenzel

A. C.

(2015). Facets of teachers’ emotional lives: A quantitative investigation of teachers’ genuine, faked, and hidden emotions. Teaching and Teacher Education, 49, 78–98. https://doi.org/10.1016/j.tate.2015.03.003

62.

Tschannen-Moran

Woolfolk Hoy

A.W.

(2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17, 783–805. https://doi.org/10.1016/S0742-051X(01)00036-1

63.

Wang

(2019). Structural equation modeling. Application using Mplus. Wiley.

64.

Yan

E. M.

Evans

I. M.

Harvey

S. T.

(2011). Observing emotional interactions between teachers and students in elementary school classrooms. Journal of Research in Childhood Education, 25(1), 82–97. https://doi.org/10.1080/02568543.2011.533115

65.

Yıldız Kırılmaz

Çelen

Ü.

Sarp

(2003). A study on teacher burnout with a group of primary school teachers. Elementary Education Online, 2(1), 2–9. http://www.ilkogretim-online.org/fulltext/218-1596598261.pdf?1652727256

66.

Yin

(2015). The effect of teachers' emotional labour on teaching satisfaction: Moderation of emotional intelligence. Teachers and Teaching: Theory and Practice, 21(7), 789–810. https://doi.org/10.1080/13540602.2014.995482