Development and validation of the Principal Emotion Inventory: A mixed-methods approach

Abstract

The current study aimed to develop and validate a Principal Emotion Inventory (PEI). Equipped with the theoretical-empirical strategy of test construction, this development and validation procedure consists of one expert survey and five sequential empirical studies with sufficient samples of participating principals based on existing emotion theories and empirical studies. The reliability, substantive validity, structural validity and external validity were tested using exploratory factor analysis, confirmative factor analysis, invariance analysis and structural equation modelling. As a result, a multidimensional model on principal emotions with 25 items was identified with a good model fit. The PEI consists of five factors: enjoyment, pride, frustration, anxiety and hopelessness. This paper is the first attempt to develop and validate a sound instrument that enables measuring school principals’ salient emotions in relation to their working experiences. The PEI would provide a validated instrument for future research on discovering the nature of principal emotions as an independent construct. Measurement is critically important to the replicability and robustness of the research. The findings on principal emotions are also informative for principal preparation and evaluation while taking emotions into consideration.

Keywords

Principal mixed method emotion instrument development

Introduction

Historically, scholars have emphasized the technical and rational dimension but left behind the emotive side of the leadership role (Crawford, 2011). Over 40 years ago Bridges (1979) pointed out the disjunction between the rational perspectives on leadership adopted by scholars and the emotionally laden tasks of leading in organizational settings. As Hargreaves (1997: 108–109) warned, ‘if educational reformers ignore the emotional dimensions of educational change, emotions and feelings will only re-enter the change process by the back door’. In a similar vein, James and Connolly (2000: 146) assert that ‘responses to change may be influenced by very powerful non-rational emotional forces’ and ‘any set of leadership principles must be founded on an understanding that the emotional dimension of leadership is crucial’. These observations suggest that the emotional aspect of leadership warrants further investigation (Crawford, 2018). However, research on principal emotions has been underdeveloped, which leads to limited understanding of this aspect (Berkovich and Eyal, 2017a).

Nowadays, school principals are increasingly held accountable for implementing changes that have been adopted by policymakers at higher levels of the education system (Hallinger, 2018). Leading change has been described as an emotionally complex facet of leadership behaviour (Fullan, 2015). These ‘emotionally hot climates’ (Leithwood and Beatty, 2009: 91) have the effect of decreasing decision latitude (Lingard et al., 2013) and autonomy (Fink and Brayman, 2006), which leads to scrutiny stress (Lasalvia, 2011; Maxwell and Riley, 2017). This negatively impacts the psychosocial and physiological health of principals (Dewa et al., 2009), reciprocally affecting job performance and satisfaction (De Nobile and McCormick, 2010), and personal engagement with their staff (Crawford, 2018). When school principals’ wellbeing declines, their ability to significantly impact school functioning and whole-school wellbeing also declines (Maxwell and Riley, 2017). Recent empirical research has begun to elaborate on how principal emotions, as one of the professional capitals in addition to the other three (human, social, and decisional) capitals, add to the leadership repertoire of school principals (Berkovich and Eyal, 2017a; Cliffe, 2018; Zembylas, 2018). Furthermore, it is found that principals enabled to knowingly or subconsciously make intelligent use of their own emotions and those of others when facing the challenge of leadership achieve the positions that they strive for (Cliffe, 2011). Therefore, principal emotions increasingly become one of the key aspects to take a school forward (Arar, 2017). In a recent review on school leaders and emotions, Berkovich and Eyal (2015) found that qualitative methods (60%) have dominated. Among quantitative and mixed-approach efforts, the majority of the studies focused on emotional intelligence and emotional labour of school leaders. Lack of quantitative instruments of principal ‘emotions’ may account for this phenomenon. To fill in these gaps, the current article conducted a first attempt, aiming at developing and validating a Principal Emotion Inventory (PEI) using sequential empirical studies based on current emotion theories.

Theoretical framework

Understanding emotion and principal emotion

To date, no consensus has been reached by scholars on an absolute definition of emotion. A psychological viewpoint influenced by psychological-based research, and often adopted by early emotion researchers, had a focus on the structure and function of emotion (Berkovich and Eyal, 2015). These days, however, the general consensus is that emotion has a multi-componential nature. The most cited definition of emotion by Scherer (2013) considers it to consist of several collections of components. These are more or less unordered, and jointly stimulated by how an incident is determined and its component tendencies. Varying terminologies are used according to the underlying theoretical perspective. However, most scientists tend to refer to similar or the same components. While referencing related components, clinical psychologist Izard (2010), for example, used the terms antecedent cognitive appraisal, cognitive interpretation, expressive behaviour, feelings or a feeling state, neural systems and response systems. Izard (2010) highlighted social effects and contended that elements can be seen as being socially constructed as a contrast to being merely individual and psychological. Emotion has been interpreted by Schutz et al. (2006) as being constructed socially and they contend that, from judgements, whether conscious and/or unconscious, an individual personally executes ways of being, and will afterwards emerge with perceived successes, and achievement of personal goals or sustainment of beliefs or standards within social-historical contexts. In order to have a broad-based appreciation and understanding of emotion, Corcoran and Tormey (2012) have claimed it is important for individuals to understand how such an accumulation of emotional components interacts with the social sphere.

In educational settings, the role of principal emotions has been well recognized; however, few studies have explicitly defined the term (Berkovich and Eyal, 2015). Scholars tend to borrow the definition of emotions from the psychological field. The ideology of this definition (Schutz et al., 2006) in education is broadly used as it is a basis for merging the changes that take place in a human’s internal emotions and the interactions of external emotional experiences. In this project, the emotional experiences of principals not only occur within their psychological undertakings, but also include the emotional feelings of stakeholders including school staff, students, supervisors, and other associated affiliations surrounding him/her, as well as the exchanges that occur in an individual’s personal, professional and social environment.

Studies of principal emotions

Although limited efforts have been put into investigation of principal emotions, the existing literature has identified the antecedents and consequences of principal emotions. Some studies have claimed the antecedents of principal emotions could be classified into three aspects, that is, contextual, organizational and individual (Berkovich and Eyal, 2015). First, they found that some contextual factors shape principal emotions. For example, sociocultural power relations (Blackmore, 2010), social justice (Zembylas, 2016), hegemonic occupational culture (Crawford, 2007) and hierarchical supervision style from supervisors to school principals (Schermuly et al., 2011). Second, a few studies identified organizational antecedents such as organizational climate (Brennan and Mac Ruairc, 2011) and crisis management (Ackerman and Maslin-Ostrowski, 2004). In particular, the results show that a supportive organizational climate, manifested in staff collaboration and recognition, promotes principals’ positive emotions, whereas a non-supportive climate is associated with their negative emotions (Berkovich and Eyal, 2015). Furthermore, students’ and teachers’ performance affects principal emotions (Arar and Oplatka, 2018; Kelly et al., 2007). Findings demonstrate that highly positive emotions have been voiced by principals towards their students and teachers who are high-performing in their studies and work respectively. Third, principals’ individual factors such as gender are identified to be influential.

With respect to the consequences of principal emotions, it seems that existing literature tends to investigate how principals’ emotional capacities, rather than, principal emotions directly, have impacts on other related aspects. For example, emotional labour strategies (Maxwell and Riley, 2017), emotional intelligence (Arar and Oplatka, 2018; Chen and Guo, 2020; Cliffe, 2018; Oplatka, 2017), and emotional regulation (Arar, 2017). Most of these research studies are qualitative efforts focusing on exploring the relationships of these constructs with other relevant constructs such as leadership, intention to leave, teacher wellbeing and teaching strategy. This phenomenon might be because there is no quantitative tool for measuring principal emotions.

Rational of test construction of the PEI

When developing and validating an instrument, previous researchers suggest that a combination of theory and empirical investigation be implemented in the design process (Frenzel et al., 2016; Hong et al., 2016). The advantage of employing both theoretical and empirical strategies is to ensure the instrument develops more authentically to cover relevant items and content domains and covers a greater content validity than those solely generated from existing literature or target respondents (Burić et al., 2018). Particularly, theories could be used to identify relevant domains and items of each domain. While empirical studies can be utilized to generate the validation (for example, substantive validity, structural validity, external validity and cross-validity) of an instrument, they can also make up missing domain and/or items and erase irrelevant domains and/or items in the situation with samples of target respondents. Scholars warn that using theories alone to validate an instrument might result in the danger of unproven and potentially untrustworthy measurement since the procedure leaves out the target respondents of the reality and heavily relies on creators’ subjective opinions (Hong et al., 2016). On the other hand, sole use of empirical evidence might lead to a lack of theoretical foundation so it would be difficult to achieve a stable reliability and generalization due to potential sample bias (for example, idiosyncrasies of samples, Butcher, 2000). Therefore, it is strongly recommended that theoretical and empirical considerations need to be leveraged to the development and validation of an instrument, principal emotions in this case, in a procedure known as the rational-empirical strategy of test construction (Frenzel et al., 2016).

Taken together, the aim of this study was to develop and validate a psychometrically grounded self-report, namely, the PEI, by adopting a theoretical-empirical approach. The following research questions are outlined:

What are the perceptions of emotions from the principal’s perspective?

Is the PEI reliable?

Is the PEI substantively valid?

Is the PEI structurally valid?

Is the PEI externally valid?

The present research

The PEI will measure a variety of theoretically and empirically relevant school principals’ salient emotions in relation to their working experiences. This project followed the basic principles for scale development and validation recommended by the test development guidelines offered in the Standards for Educational and Psychological Testing (Joint Committee on Standards for Educational and Psychological Testing, 2014) and other modern guidelines (DeVellis, 2003; International Test Commission, 2017).

This project employed the three recommended sequential validities to develop the PEI through five empirical studies. First, establishing substantive validity evidence typically consists of two steps, namely, determining the nature and scope of the measure and creating an item domain and pool. Second, structural validity is built through testing the measure on a target sample and respectively evaluating the item distributions, latent structure, internal consistency and construct boundaries using descriptive, factor analytic, reliability and measurement invariance. Third, if structural validity evidence is obtained, then external validity evidence is installed by testing the generalizability of the measure’s psychometric properties with diverse samples and its utility (e.g. convergent, discriminant or predictive relationships) in applied contexts.

Substantive validity

The aim of this section is to establish the substantive validity of the PEI. In the first step of instrument development, the nature and scope of the measure was outlined. We intended to develop a concise, multidimensional instrument that could be used for various purposes (for example, understanding the nature of principal emotions and the relationships with other constructs) in future studies. The PEI was expected to comprise three to six domains with four to five items for each. With this in mind, the second step was to create item dimensions and pool relating to principal emotions based on relevant literature and two empirical studies (Study 1 and Study 2).

Study 1

Before conducting Study 1, the research team firstly defined content domains from previous literature relating to principal emotions (Arar and Oplatka, 2018; Berkovich and Eyal, 2015; Brennan and Mac Ruairc, 2011; Crawford, 2007, 2018; Leithwood and Beatty, 2009; Oplatka, 2017; Zembylas, 2016, 2018) and then generated new items for each domain. The following two aspects were especially examined. One was the categories of principal emotions. Based on the multiple-component point of view of emotions (Scherer, 2013), the PEI tended to cover multiple categories of emotions that school principals experience with relation to their work. Second, we tried to identify the influential drivers of those emotions and challenges that principals experienced so that we could determine the trigger sources of those emotions. This would support constructing the description of the items in the PEI. After a comprehensive review, five categories of emotion with 14 items, namely, anxiety (4), frustration (3), hopelessness (3), enjoyment (2) and pride (2), were identified. Furthermore, five types of sources of emotions were recognized, which consisted of contextual level, school level, classroom level, job nature and emotional capacity. For example, principals from Canada reported feeling frustrated with insufficient autonomy or support (Leithwood and Beatty, 2009). Moreover, principals from Ireland experienced negative emotions (for example, anxiety) due to conflicts from their staff (Brennan and Mac Ruairc, 2011). By contrast, Yariv (2009) found that principals felt pride in high-performing staff in Israel.

Sample

Study 1 then turned to generating new domains and new items for each domain though interviewing 25 principals from 13 secondary schools and 12 primary schools in China. There were 16 male and nine female principals with an average of 15 years’ working experience. Their age ranged from 36 to 55 years with an average age of 46. We chose principals who had at least two years of principalship experience, which ensures they were familiar with the practice and the role challenges of being a principal and had great potential for experiencing sufficient work-related emotions of significant intensity.

Procedure

A semi-structured interview was conducted with 25 participants. They were asked to (1) share two scenarios in which they experienced most salient emotions relating to their work; (2) share their views of the influential drivers and/or situations to trigger those emotions; (3) tell four to seven types of emotions which they thought most influential during their work and provide examples to express how and why those emotions were salient. By following guidelines of thematic analysis proposed by Braun and Clarke (2006), transcriptions were coded by the perspectives aforementioned (categories, influential drivers and challenges).

Results

The participants described a range of emotions experienced from aforementioned perspectives (e.g. contextual level, school level, classroom level, job nature and emotional capacity). In general, the results from interviews were consistent with those in the literature. Principals reported a great source of negative emotions such as anxiety, frustration, hopelessness, fatigue and exhaustion as the most important and frequent in a descending sequence. It is noted that feelings described as psychological, emotional and physical fatigue, tiredness, numbness or exhaustion appeared also salient and frequent in the qualitative analysis. By contrast, these principals reported fewer positive emotions. The more-frequent positive emotions consisted of happiness and pride in relation to professional lives. Only a few participants mentioned love, affection, care and enthusiasm.

Bearing in mind the two principles mentioned above (multiple-component definition of emotional experience and different influential drivers and challenges from five perspectives), a PEI with 42 items was generated to enable evaluation of each content domain relating to principal emotions. The PEI comprised five domains of emotions: enjoyment (11), pride (7), frustration (9), anxiety (8) and hopelessness (7). These served as an empirical guide for the selection and operationalization of principal emotions in subsequent studies with regard to the various investigations of the PEI validity.

Study 2

Study 2 aimed at examining the substantive validity of the PEI using an expert panel (Krabbe, 2017).

Sample, procedure and results

Three experts in the fields of leadership and emotion consisted of two females and one male with an average of 12 years of research experience. Using a paper-and-pencil survey, a quantitative sorting process was utilized to examine whether the 42 items fitted into the instrument evaluating principal emotions and whether the items were in line with the five corresponding domains. As a result, three items were discarded because of unsatisfactory confidence ratings, one item was added, four items were modified, and three items were moved from one to another domain. The result was that 40 items classified into the five original dimensions were kept in the PEI, that is, enjoyment (7), pride (8), frustration (9), anxiety (9) and hopelessness (7).

Structural validity

This section tested the structural validity of the PEI using a sample of principals in two sequential empirical studies (Study 3 and Study 4).

Study 3

Study 3 aimed to demonstrate preliminary evidence on the factor structure, reduce the items which did not fit well, and unveil the latent dimensions of the PEI.

Sample

A sample of 213 principals in a principal training workshops in China were approached with a 72% response rate. Participants consisted of 115 (54%) secondary principals and 98 (46%) primary principals. Among these participants, 153 (72%) were male and 60 (28%) female with an average age of 42 years old and 18 years of working experience. More than half (55%, n = 117) of these principals had less than five years of working experience of being a principal, 38% (n = 81) had six to 15 years, and 7% (n = 15) had greater than 15 years. About 75% (n = 160) of them held a master’s degree, 18% (n = 38) held a bachelor’s degree, and 7% (n = 15) held a doctoral degree.

Procedure and results

Besides demographic questions, the participants were asked to indicate their agreement with each item of the 40-item PEI. The study employed a symmetric agreement Likert scale with 6 points without a neutral point. Exploratory factor analysis (EFA) was employed to generate the model using SPSS 21. In the EFA procedure, the following criteria were employed: (1) all items with regression loadings greater than 0.50 on their intended conceptual factors; (2) three or more conceptual aligned items in one factor; (3) all items’ cross loadings fewer than 0.30; (4) all items with no negative error variance (Costello and Osborne, 2005). Results from the EFA procedure yielded an original five-factor solution with eigenvalues > 1, accounting for approximately 65.69% of the variance characterized by an adequate sample size (Kaiser-Meyer-Olkin = 0.85; Bartlett’s test of sphericity χ² = 731.28, df = 29, p < 0.001). During this process, 11 items were dropped and a 29-item PEI with the same five factors remained, namely, enjoyment (7), pride (6), frustration (6), anxiety (5) and hopelessness (5).

Study 4

Study 4 aimed to confirm the latent structure and test the psychometric features of the PEI with a different sample of respondents.

Sample

A different sample of 223 principals in the same principal training workshops in China were approached with a better response rate of 88%. Participants consisted of 117 (52%) secondary principals and 106 (48%) primary principals. Among these participants, 164 (74%) were male and 59 (26%) female with an average age of 43 years and 18 years of working experience. Less than half (45%, n = 100) of these principals had less than five years of working experience of being a principal, 36% (n = 80) had six to 15 years, and 19% (n = 42) had greater than 15 years. The majority of them (72%, n = 161) held a master’s degree, 21% (n = 47) of them held a bachelor’s degree, and 7% (n = 15) held a doctoral degree.

Procedure

The exploratory model identified in Study 4 was then evaluated in confirmative factor analysis (CFA) which allowed better detection of items with weak loadings and items with strong covariance with other factors which is a violation of simple structure (i.e. items belong to only one factor). A multi-criteria approach for acceptable model fit was adapted (Marsh et al., 2004). All analyses were carried out in Mplus 7.

Results

This section reported the CFA model, reliability and internal construct validity.

First, following the criteria of the EFA procedure and CFA model fit index, and the original scope and principles of developing the PEI, the CFA procedure discarded four items. The 25-item trimmed model was given the same analysis and demonstrated a good model fit (χ² = 885.07, df = 460, p < 0.001; CFI = 0.92, TLI = 0.93, Grammar hat = 0.91, RMSEA = 0.051, 90% C.I. = 0.048–0.054, SRMR = 0.042). The original five factors remained, namely, enjoyment (5), pride (5), frustration (5), anxiety (5) and hopelessness (5).

Second, internal reliability of each scale in the PEI was tested. All item loadings were greater than 0.50, thus indicating that the items were associated with each other as detached factors in the PEI. These are shown in Table 1. Descriptive statistics also showed that the mean scores of two positive emotions were high (pride: M = 5.68, SD = 0.41 on a 6-point scale; enjoyment: M = 5.12, SD = 0.59). By contrast, the mean scores of three negative emotions were relatively lower (anxiety: M = 3.78, SD = 0.84; hopelessness: M = 3.42, SD = 0.74; frustration: M = 3.01, SD = 0.79). The conducted internal consistency analysis identified the values of Cronbach’s alphas had a range from 0.81 to 0.91 with an average value of 0.84, indicating that each scale was sufficiently reliable such that the model could be meaningfully used in further analysis (Table 2).

Table 1.

PEI factors, items and factor loadings.

Emotion scale and items	Factor loading
F1. Enjoyment
1. I enjoy my work as a principal.	0.59
2. I am happy to see my staff use classroom data to make improvements.	0.63
3. I am happy to empower my colleagues.	0.78
4. I am happy if our school goals have been achieved.	0.56
5. I am happy when I manage my school well.	0.77
F2. Pride
6. I am proud that I am capable of doing my job.	0.71
7. I am proud when I am trusted.	0.75
8. I feel like a winner when my students and/or colleagues succeed.	0.68
9. I am proud when my colleagues work together for a common goal.	0.82
10. I am proud when my school outperforms others.	0.58
F3. Frustration
11. I feel frustrated when I have self-doubt.	0.81
12. I feel frustrated if I am not recognized.	0.90
13. I feel frustrated when my colleagues are not motivated or collaborative.	0.62
14. I feel frustrated when school climate is resistant to change.	0.71
15. I am frustrated when I am not sufficiently empowered.	0.59
F4. Anxiety
16. I feel exhausted at the end of the working day.	0.81
17. I am struggling with the isolation of my job.	0.79
18. I am worried when my school performance is not good.	0.64
19. I am worried whether a sustainable school plan could be achieved.	0.62
20. I am worried about the unhealthy competition between schools.	0.77
F5. Hopelessness
21. I feel hopeless when I have to implement unreasonable work in my school.	0.68
22. I feel I cannot do anything more about the hierarchical supervision style.	0.82
23. I feel I cannot do anything more about the increasing accountability.	0.77
24. I feel hopeless about irrational external expectations.	0.74
25. I feel hopeless in the face of unreasonable public blame.	0.67

Table 2.

Descriptive statistics, Cronbach α and factor inter-correlations.

Emotion scale	M¹ (SD)	Cronbach (α)	Factor inter-correlations (r)
Emotion scale	M¹ (SD)	Cronbach (α)	F2	F3	F4	F5
F1. Enjoyment	5.12 (0.59)	0.91	0.67*	0.01	-0.15	0.08
F2. Pride	5.68 (0.41)	0.81	—	-0.14	-0.13	0.09
F3. Frustration	3.01 (0.79)	0.83		—	0.66*	0.45*
F4. Anxiety	3.78 (0.84)	0.87			—	0.46*
F5. Hopelessness	3.42 (0.74)	0.78				—

Note. 1= maximum agreement is 6.00; * indicates significantly different.

Third, internal construct validity was also examined using correlation coefficient. Table 2 demonstrated that the inter-correlations between the five factors ranged from |0.01| to |0.67| with an average value of 0.28. Latent inter-correlations between five scales also yielded a logical trend. The same types of emotions (i.e. positive emotions including enjoyment and pride; negative emotions including anxiety, frustration and hopelessness) were highly correlated with each other and positive, while correlations between emotions of the opposite valence approached weak to zero. On one hand, such findings demonstrated a good convergent validity of the five PEI scales. The correlations between the common types of emotion scales in the same valence were moderate to high in magnitude. However, the correlations between different types of emotion scales in a different valence were weak. These evidences revealed that separate emotion scales could be differentiated from each other even if they were highly or moderately correlated and shared the same valence. In conclusion, the initial test of internal validity of the PEI indicated a sufficient degree of convergent and divergent validity of its scales.

External validity

This section tested the external validity including convergent, divergent and predictive validity between PEI scales and the other three measures. Convergent validity refers to how closely the new scale is related to other variables and other measures of the same construct. Conversely, divergent validity shows that two measures that are not supposed to be related are indeed unrelated. Predictive validity is the extent to which performance on a test (measure) is related to later performance that the test (measure) was designed to predict (Krabbe, 2017).

Study 5

Study 5 aimed at establishing the external validity of PEI using three additional measures: emotional exhaustion, job satisfaction and job performance. These three measures were distributed to 223 principals in Study 5 as it was hard to recruit the principal participants and there is only a difference of four items with the 25-item model in Study 4.

Procedure

Anticipating that the structural validity analyses would yield promising findings, further analyses were conducted to investigate the external validity of the PEI with a target sample of principals. Specifically, the distributions and inter-correlations and internal consistency of the PEI scales were re-evaluated. Convergent validity and divergent validity were examined using correlation coefficient with the measures of emotional exhaustion and job satisfaction. Predictive validity was tested using structural equation modelling (SEM) with the measure of job performance. As with the CFA procedure, the same criteria and model fit indices were applied in the SEM procedure performed in Mplus 7.

Measures

The Maslach Burnout Inventory (MBI on Emotional exhaustion, Enzmann and Kleiber, 1989) was used to test the PEI convergent and discriminate validities. The MBI consisted of a 4-point scale with four items and a range from 1 = strongly disagree to 4 = strongly agree. The internal reliability was 0.81. The job satisfaction scale by Xu and Zhao (2012) consisted of 10 items with a 5-point response agreement scale. The internal consistency reliability was 0.90. The job performance scale (Motowidlo and Van Scotter, 1994) was adopted to test predictive validity. The 14-item instrument assessed the self-rated job performance on a 6-point agreement Likert scale. The internal reliability was 0.94.

Results

This section reported the external validity with relation to three additional measures. It is expected that principals’ unpleasant emotions are positively connected to the MBI items. By contrast, their pleasant emotions are either negatively related or positively but weakly related to the MBI items (Blackmore, 2010). Likewise, it is expected pleasant emotions are positively related to the job satisfaction items. On the contrary, unpleasant emotions are negatively or positively but weakly associated with the job satisfaction items (Hafsa, 2015). Table 3 revealed that most evidence from the three measures was expected except for two exceptions. They were the correlation between frustration and job satisfaction (r = 0.09) and the correlation between hopelessness and job performance (r = 0.01). To conclude, the patterns of correlations obtained should establish good convergent and discriminant validities of the PEI.

Table 3.

Correlations of the PEI scales to the three additional measures in Study 5.

Emotion scale	Emotional exhaustion	Job satisfaction
F1. Enjoyment	-0.52**	0.47**
F2. Pride	-0.41**	0.50**
F3. Frustration	0.61**	0.09
F4. Anxiety	0.55**	-0.31**
F5. Hopelessness	0.51**	-0.58**

Note. n = 223; *p < 0.05; **p < 0.001.

The predictive validity was tested using SEM. It is expected that principals’ pleasant emotions positively predict job performance, while their unpleasant emotions negatively relate to, or positively but weakly relate to, job performance (Judge and Bono, 2001). A SEM model was established (χ² = 1508.80, df = 501; p < 0.001; TLI = 0.91; CFI = 0.91; gamma hat = 0.94; SRMR = 0.070; and RMSEA = 0.050, 90% C.I. = 0.048–0.052). A significantly predictive relation between the PEI and the job performance scale was established as hypothesized based on the previous literature (enjoyment β = 0.53; pride β = 0.39; frustration β = -0.41; anxiety β = -0.30; hopelessness β = -0.71). Hence, while there are reasonably substantial and meaningful relations between espoused principal emotions and their job performance, these relations explain at best a moderate amount of variance at the levels of endorsement of their job performance.

Taken together, findings from these external validity analyses, accompanied by those from the structural validity analyses and substantive validity procedures described above, are considered to provide ample validity evidence in favour of the PMI as a promising, dependable measure developed based on the theoretical and empirical evidence.

Discussion

The Principal Emotion Inventory

The PEI encompasses five dimensions capturing school principals’ salient and discrete emotions including enjoyment, pride, frustration, anxiety and hopelessness. The sources of these emotions are associated with contextual level, school level, classroom level, job nature and emotional capacity. In particular, the causes of negative emotions are more related to policy, school culture, professional attitudes and nature of the principalship (Leithwood and Beatty, 2009; Yariv, 2009). The PEI also reflects a broad perspective on the individual practice of school principals, both within and outside the school, and can be perceived in the particular context of professional emotions. This entails an extended view of principalship, characterized by an emotional perspective that embraces the broader context of education, culture and school policy, but also the relevance of individual emotional capacity and the specific role of a school principal (Berkovich and Eyal, 2015).

From the literature, it seems that principals reported more negative emotions with relation to emotional exhaustion, such as burnout (Brennan and Mac Ruairc, 2011) and fatigue (Schermuly et al., 2011). Hence, the research team in this project focused more on this kind of negative emotion during principal interviews in Study 1. Notably, anger belongs to one basic emotion out of six in Parrott’s (2001) tree emotion model. Interestingly, Sutton (2007) made a theoretical distinction between frustration and anger. According to Sutton (2004), anger is experienced when people make a primary appraisal that a situation is incongruent with their goals and a secondary appraisal that an individual is to blame. Frustration is similar to anger in that there exists an initial primary appraisal that an event is relevant and incongruent with one’s goals. However, frustration differs from anger with regard to the secondary appraisal. This distinction is echoed by Sutton’s (2007) following study which reported that teachers experienced frustration and anger simultaneously. However, the participants in this project tended to put the anger items into frustration and anxiety. Eventually, anger was not included in the PEI in this project. This indicates that theoretical rationales cannot hold their truth and validity without empirical evidence (Hong et al., 2016).

Furthermore, the participants in this project reported strong negative emotions such as fear, sadness and hopelessness. Only hopelessness was kept by participants’ choice in the PEI. The items tend to focus on the structural and management issues which lead to hopelessness. This aligns with those in the existing literature. For example, Ackerman and Maslin-Ostrowski (2004) found that principals in America consider experiences of a crisis nature (or what have also been described as ‘wounding’ experiences) can contribute to being fearful, helpless and lonely. Furthermore, when crises occur, specific and different types of fears that characterize leadership experiences are exacerbated. They might fear failure, change or stagnation, losing professional identity, and being criticized and dismissed.

Comparatively, two pleasant emotions, enjoyment and pride, were identified. The items are related to positive experience with school management, a collaborative climate and good performance of students, staff and school. With respect to frequencies of emotions in human life in general, enjoyment (or happiness) outranks all other discrete emotions, as demonstrated by a quasi-representative survey reported by Scherer (2013). According to Oatley and Johnson-Laird’s (1987) research working towards a cognitive theory of emotions, basic emotions, which are recognized and perceived similarly across cultural contexts, include enjoyment. Staff recognition, support and collaboration have been identified by leaders as stimulating their positive emotions (Mac Ruairc, 2011). Particularly, leaders who have positive emotions including joy have received validation and positive feedback from their subordinates (Leithwood and Beatty, 2009).

Pride has been recognized as a universal and distinctive emotion observed in various cultures and environments. In particular, pride also functions to increase self-esteem and adaptive behaviours, and promotes an individual’s social status and group acceptance (Hart and Matsuba, 2007). From the empirical aspect, Yariv (2009) found that principals from elementary schools in Israel felt high pride in high-performing teachers. Thus, pride is an important emotion especially for school leaders who are in a critical stage of developing a sense of professional identity and belonging to the professional community (Hong et al., 2016). It is noted that only a few participants in the current study mentioned love, affection, care and enthusiasm. However, these emotions (e.g. love and care) were identified in other contexts such as Israel (Arar and Oplatka, 2018) and United Kingdom (Cliff, 2011; Crawford, 2018). Social and cultural differences may lead to this discrepancy but future investigations are needed in this regard.

Validation of the PEI

The PEI shows strong psychometric properties in terms of reliability, substantive validity, structural validity and external validity with diverse samples. The PEI in this project achieved a good substantive validity procedure. Five domains with concise items on principal emotions were identified and validated. As mentioned above, the advantage of employing both theoretical and empirical strategies was to ensure the PEI tended to develop more authentically to cover relevant items and content domains and to cover a greater content validity than those solely generated from existing literature or target respondents (Burić et al., 2018).

The PEI also demonstrated a good structural validity procedure. All item loadings in the PEI were greater than 0.50 and all factor reliabilities were acceptable to excellent ranging from 0.78 to 0.91. In addition, the CFA analyses revealed more detailed information on the structural validity of the different factors of the PEI. Specifically, our discrete emotion approach proved to be valid in the five-factor model included separate latent variables of enjoyment, pride, anger, frustration, anxiety and hopelessness. This trimmed model was given the same analysis and demonstrated a good model fit. As such, the PEI is clearly suitable for assessing five kinds of principal emotions as distinct constructs. Such findings demonstrated a convergent validity of the five PEI scales.

The external validation provided evidence that the PEI shows consistent external validity with three related constructs including emotional exhaustion, job satisfaction and job performance. As expected, principals’ unpleasant emotions positively connected with the MBI items. By contrast, their pleasant emotions negatively related or positively but weakly related to the MBI items. Likewise, pleasant emotions were positively related to the job satisfaction items, while unpleasant emotions were negatively or positively but weakly associated with the job satisfaction items. As for predictive validity, principals’ pleasant emotions positively predicted job performance, while their unpleasant emotions were negatively related or positively but weakly related to job performance. Generally, these relationships were consistent in their direction, yet small enough in size to warrant clear conceptual separation of discrete emotions as experienced by principals and those related constructs although with a couple of exceptions.

Implications

The findings of this project have a few favourable implications for theory, methodology and practice.

Theoretically, the project will contribute to the advancement of knowledge production of principal emotions. Scholars have emphasized the technical and rational dimension but left behind the emotive side of school leadership role (Bridges, 2012; Crawford, 2018). It seems that knowledge base of principal emotions is at the ‘emerging’ stage (Berkovich and Eyal, 2017). The scientific efforts on principal emotions are even fewer in China. This project is particularly critical and timely because principals’ emotions are proving to be highly important not only for their own wellbeing but also for the functioning of a school (Bush, 2018; James et al., 2019; Maxwell and Riley, 2017). This project will not only enrich our understanding of the nature of school principals’ salient emotions, but also their possible drivers and consequences since these will be explored in the first interview study. This will contribute to our conceptually understanding of antecedents, nature and consequences of principal emotions as a whole, which will provide a theoretical framework for future research. Therefore, this project will promote more efforts on principal emotions, which will largely mature the knowledge production.

Instrumentally, this project is the first attempt to generate a multidimensional instrument that enables the measurement of school principals’ salient emotions. It is noted that emotions of an individual are believed to be conceptually elusive (LeDoux, 1995) and difficult to be measured (Šarić, 2015). However, the quantitative measures of student and teacher emotions and have been already developed (for example, Authors, 2016; Burić et al., 2018; Frenzel et al., 2016; Pekrun et al., 2005), empirical evidences have showed that individual’s emotions could be quantitatively measured. However, it should acknowledge that this instrument is static in its nature, so it could only capture the emotions at one time point. The PEI will provide a validated instrument for future research, aiming at discovering the nature of principal emotions as an independent construct and the relations to other related constructs. The PEI could be used to establish test–retest reliability in future studies to test responsiveness to change over time. Furthermore, it has been identified that most quantitative studies have measured the emotional capacity of school principals rather than their emotions and the relations with other constructs (Berkovich and Eyal, 2017b). Hence, the emergence of the PEI will contribute to balancing the various methodological approaches as Berkovich and Eyal (2017a) have claimed that that qualitative methods have been dominant and there is a lack of quantitative and mixed-approach efforts in their recent review school leaders’ emotions.

Practically, this project will be informative for principal preparation at the induction stage and evaluation at the in-service stage. Principal preparation programs and courses in universities and continuous training programs might consider including the element of principals’ salient emotions extracted from empirical evidences in the proposed project. Particularly, professional development programs can be developed to promote the emotional capacity especially emotional resilience of future and current principals. The incorporation of emotional literacy should be regarded as an ongoing element of principal professional development and principal selection. Moreover, it is expected that this project will make policymakers, researchers and school principals aware of the importance of principals’ emotional literacy in their evaluation. The standards for educational administration might include the emotional element as one of professional capitals for developing principal capacity and evaluating principal effectiveness as a function of changes in behaviours and performance of their schools (Oplatka and Arar, 2019; Zembylas 2018).

This project will offer specific recommendations for developing interventions on principal emotions to support future and current principals in ‘emotionally hot climates’ during change (Leithwood and Beatty, 2009). Measurement is critically important to the replicability and robustness of research findings, particularly regarding the rigorous evaluation of intervention programs (Thomas et al., 2015). The PEI to be developed in this project will be a measurement strategy for making meaningful and comparable assessments of different forms of principal emotions in different situations over change. This is both important for extending the scope of hypotheses that can be tested to achieve generalizability, and for the evaluation and tailoring of intervention programs to target subscale profiles (Thomas et al., 2019). Therefore, this robust PEI will not only promote the development of interventions on principal emotions, but also ensure a high quality of interventions.

Limitations and conclusion

Although the current paper demonstrates the stable psychometric quality of the PEI, some limitations of the paper need to be addressed. First, given that all of the measures in the present project were self-reported, the concurrent validity findings in the present project may be biased by common-method variance (for example, the variance attributed to the measurement method rather than to the constructs represented by the measures) (Podsakoff et al., 2003). The potential for this bias is most likely implicated in the convergent validity correlations conducted with a general sample of principals. To mitigate this potential confound, we recommend expanding the repertoire of future validation measures to utilize 360-degree assessment surveys from different related stakeholders such as the principal, the principal’s supervisors, and the teachers in the principal’s school (Goldring et al., 2015). Such multi-source feedback typically entails a more robust self-evaluation of the principals as well as parallel evaluations from subordinates, peers and/or superiors. Second, although self-report is a widely accepted standard of measurement, it is acknowledged that by assessing emotions in this way, the construct being measured was the subjective experiences of principal emotions (Thomas et al., 2019). Such retrospective judgements may be influenced not only by the ‘true scores’ of principals’ emotional experiences, but also by their beliefs about their emotions in various situations (Frenzel et al., 2016). Third, it is noted that the intended scope of the current project was to develop a concise, multidimensional instrument that could be used for various purposes. Therefore, the PEI was only concerned with certain theoretically critical and empirically salient emotions but skipped emotions that might be also important in principal professional life (e.g. sadness and fear) (Burić et al., 2018; Keller et al., 2014). Future studies may consider involving other salient emotions of principals. It is worth mentioning that future research may also want to address a broader range of discrete emotions of principals. Furthermore, as mentioned above, it should acknowledge that this instrument is static in its nature, so it could only capture the emotions at one time point.

In closing, from the beginning, our intention was never to create an exhaustive measure – an instrument assessing all possible principal emotions – but rather to develop a brief, multidimensional, parsimonious measure that assessed several salient emotions of principals at work. Within this scope, we produced a theoretically and empirically validated measure that could be used to assess principal emotions in various situations. This in turn facilitates more robust means for gauging comprehensive principal emotions. It is very clear that much more research is on call. Hence, we hope that future research will pick up where this project leaves off, further validating the PEI as both a basic research instrument and means of promoting principal wellbeing and functioning of a school.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Junjun Chen

References

Ackerman

Maslin-Ostrowski

(2004) The wounded leader and emotional learning in the schoolhouse. School Leadership & Management 24(3): 311–328.

Arar

(2017) Emotional expression at different managerial career stages: Female principals in Arab schools in Israel. Educational Management Administration & Leadership 45(6): 929–943.

Arar

Oplatka

(2018) Emotion display and suppression among Arab and Jewish assistant principals in Israel: The key role of culture, gender and ethnicity. Journal of Professional Capital and Community 3(3): 173–191.

Authors (2018) [details removed for peer review].

Berkovich

Eyal

(2015) Educational leaders and emotions: An international review of empirical evidence 1992–2012. Review of Educational Research 85(1): 129–167.

Berkovich

Eyal

(2017a) Methodological review of studies on educational leaders and emotions (1992-2012): Insights into the meaning of an emerging research field in educational administration. Journal of Educational Administration 55(5): 469–491.

Berkovich

Eyal

(2017b) The mediating role of principals’ transformational leadership behaviours in promoting teachers’ emotional wellness at work: A study in Israeli primary schools. Educational Management Administration & Leadership 45(2): 316–335.

Blackmore

(2010) Preparing leaders to work with emotions in culturally diverse educational communities. Journal of Educational Administration 48(5): 642–658.

Braun

Clarke

(2006) Using thematic analysis in psychology. Qualitative Research in Psychology 3(2): 77–101.

10.

Brennan

Mac Ruairc

(2011) Taking it personally: Examining patterns of emotional practice in leading primary schools in the republic of Ireland. International Journal of Leadership in Education 14(2): 129–150.

11.

Bridges

(1979) The Principal in Metropolitan Areas. Berkeley: McCutchan.

12.

Bridges

(2012) Administrator preparation: Looking backwards and forwards. Journal of Educational Administration 50(4): 402–419.

13.

Burić

Slišković

Macuka

(2018) A mixed-method approach to the assessment of teachers’ emotion: Development and validation of the teacher emotion questionnaire. Educational Psychology 38(3): 325–349.

14.

Bush

(2018) Preparation and induction for school principals: Global perspectives. Management in Education 32(2): 66–71.

15.

Butcher

(2000) Revising psychological tests: Lessons learned from the revision of the MMPI. Psychological Assessment 12(3): 263–271.

16.

Cliff

(2011) Emotional intelligence: A study of female secondary school headteachers. Educational Management Administration & Leadership 39(2): 205–218.

17.

Cliff

(2018) Emotional intelligence testing for headteachers: Globalization or lost in translation. Journal of Professional Capital and Community 3(3): 223–240.

18.

Corcoran

Tormey

(2012) Developing Emotionally Competent Teachers: Emotional Intelligence and Pre-service Teacher Education. Oxford: Peter Lang.

19.

Costello

Osborne

(2005) Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation 10(7): 1–9.

20.

Crawford

(2007) Rationality and emotion in primary school leadership: An exploration of key themes. Educational Review 59(1): 87–98.

21.

Crawford

(2011) Rationality and emotion in education leadership-enhancing our understanding. In: Day

Lee

(eds) New Understandings of Teacher’s Work. New York: Springer, 207–217.

22.

Crawford

(2018) Personal engagement, emotion and the educational workplace. Journal of Professional Capital and Community 3(3): 223–240.

23.

De Nobile

McCormick

(2010) Occupational stress of Catholic primary school staff: A study of biographical differences. International Journal of Educational Management 24(6): 492–506.

24.

DeVellis

(2003) Scale Development: Theory and Applications. 2nd ed. Thousand Oaks: Sage.

25.

Dewa

Dermer

Chau

Lowrey

Mawson

Bell

(2009) Examination of factors associated with the mental health status of principals. Work 33(4): 439–448.

26.

Enzmann

Kleiber

(1989) Helfer-Leiden: Stress und Burnout in Psychosozialen Berufen [Helper Ordeals: Stress and Burnout in the Human Services]. Heidelberg: Asanger-Verlag.

27.

Fink

Brayman

(2006) School leadership succession and the challenges of change. Educational Administration Quarterly 42(1): 62–89.

28.

Frenzel

Pekrun

Gostz

Daniels

Durksen

Becker-Kurz

Klassen

(2016) Measuring teachers’ enjoyment, anger, and anxiety: the teacher emotions scales (TES). Contemporary Educational Psychology 46: 148–163.

29.

Fullan

(2015) Leadership From the Middle: A System Strategy. Toronto: Canadian Education Association: Education Ontario, Canada.

30.

Goldring

Xiu

Murphy

Porter

Elliott

(2015) The convergent and divergent validity of the Vanderbilt assessment of leadership in education wang 361 (VAL-ED): Instructional leadership and emotional intelligence. Journal of Educational Administration 53(2): 177–196.

31.

Hafsa

(2015) Emotional intelligence and job satisfaction among university teachers. Journal of Research in Educational Science 7(8): 5–11.

32.

Hallinger

(2018) Bringing context out of the shadows of leadership. Educational Management Administration & Leadership 46(1): 5–24.

33.

Hargreaves

(1997) Rethinking Educational Change With Heart and Mind. Alexandria: The Association for Supervision and Curriculum Development Yearbook(ASCD).

34.

Hart

Matsuba

(2007) The development of pride and moral life. In: Tracy

Robins

Tangney

(eds) The Selfconscious Emotions: Theory and Research. New York: Guilford, 114–133.

35.

Hong

Heddy

Ruan

You

Kambara

Nie

(2016) Revising and validating achievement emotions questionnaire-teachers (AEQ-t). IJEP-International Journal of Educational Psychology 5(1): 80–107.

36.

International Test Commission (2017) The ITC guidelines for translating and adapting tests (Second edition). Available at: https://www.intestcom.org/files/guideline_test_adaptation_2ed.pdf.

37.

Izard

(2010) The many meanings/aspects of emotion: Definitions, functions, activation, and regulation. Emotion Review 2(4): 363–370.

38.

James

Connoly

(2000) Effective Change in Schools. London: Routledge Falmer.

39.

James

Crawford

Oplatka

(2019) An affective paradigm for educational leadership theory and practice: connecting affect, actions, power and influence. International Journal of Leadership in Education 22(5): 617–628.

40.

Joint Committee on Standards for Educational and Psychological Testing (2014) Standards for Educational and Psychological Testing. Washington: American Educational Research Association, American Psychological Association, and National Council on Measurement in Education.

41.

Judge

Bono

(2001) Relationship of core self-evaluations traits – self-esteem, generalized self-efficacy, locus of control, and emotional stability – with job satisfaction and job performance: A meta-analysis. Journal of Applied Psychology 86(1): 80–92.

42.

Keller

Frenzel

Goetz

Pekrun

Hensley

(2014) Exploring teacher emotions: A literature review and an experience sampling study. In: Richardson

Karabenick

Watt

HMG

(eds) Teacher Motivation: Theory and Practice. New York: Routledge, 69–82.

43.

Kelly

Carey

McCarthy

Coyle

(2007) Challenging behaviour: Principals’ experience of stress and perception of the effects of challenging behaviour on staff in special schools in Ireland. European Journal of Special Needs Education 22(2): 161–168.

44.

Krabbe

PFM

(2017) The Measurement of Health Status: Concepts, Methods and Applications From a Multidisciplinary Perspective. London: Elsevier.

45.

Lasalvia

(2011) Occupational stress, professional burnout and job satisfaction among psychiatrists. In: Langan-Fox

Cooper

(eds) Handbook of Stress in the Occupations. Cheltenham: Edward Elgar, 49–68.

46.

LeDoux

(1995) Seting ‘stress’ into motion. In: Friedman

Charney

Deutch

(eds) Neurobiological and Clinical Consequences of Stress: From Normal Adaptation to PTSD. New York: Raven Press, 125–134.

47.

Leithwood

Beatty

(2009) Leadership for emotionally hot climates. International Studies in Educational Administration 37(1): 91–103.

48.

Lingard

Martino

Rezai-Rashti

(2013) Testing regimes, accountabilities and education policy: Commensurate global and national developments. Journal of Education Policy 28(5): 539–556.

49.

Mac Ruairc

(2011) Destination zero – deconstructing inclusion. Contemporary Management Quarterly 2: 23–35.

50.

Marsh

Hau

K-T

Wen

(2004) In search of golden rules: Comment on hypothesis testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing hu and Bentler’s (1999) findings. Structural Equation Modelling 11(3): 320–341.

51.

Maxwell

Riley

(2017) Emotional demands, emotional labour and occupational outcomes in school principals: Modelling the relationships. Educational Management Administration & Leadership 45(3): 484–502.

52.

Motowidlo

Van Scotter

(1994) Evidence that task performance should be distinguished from contextual performance. Journal of Applied Psychology 79(4): 475–480.

53.

Oatley

Johnson-Laird

(1987) Towards a cognitive theory of emotions. Cognition and Emotion 1(1): 29–50.

54.

Oplatka

(2017) Empathy regulation among Israeli school principals: Expression and suppression of major emotions in educational leadership. Journal of School Leadership 27(1): 94–118.

55.

Oplatka

Arar

(2019) Introduction. In: Oplatka

Arar

(eds) Emotion Management and Feelings in Teaching and Educational Leadership (Studies in Educational Administration). Bingley, UK: Emerald Publishing Limited, 1–6.

56.

Parrott

(2001) Emotions in Social Psychology. Philadelphia: Psychology Press.

57.

Pekrun

Goetz

Frenzel

(2005) Achievement Emotions Questionnaire-Mathematics (AEQ-M). User’s Manual. Munich: University of Munich, Department of Psychology.

58.

Podsakoff

MacKenzie

Lee

Podsakoff

(2003) Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology 88(5): 879–903.

59.

Šarić

(2015) Teachers’ emotions: A research review from a psychological perspective. Journal of Contemporary Educational Studies 66(4): 10–26.

60.

Scherer

(2013) Measuring the meaning of emotion words: A domain-specific componential approach. In: Fontaine

JJR

Scherer

Soriano

(eds) Components of Emotional Meaning: A Sourcebook. New York: Oxford University Press, 7–30.

61.

Schermuly

Meyer

(2011) Effects of vice-principals’ psychological empowerment on job satisfaction and burnout. International Journal of Educational Management 25(3): 252–264.

62.

Schutz

Hong

Cross

Osbon

(2006) Reflections on investigating emotion in educational activity settings. Educational Psychology Review 18(4): 343–360.

63.

Sutton

(2007) Teachers’ anger, frustration, and self-regulation. In: Schutz

Pekrun

(eds) Emotions in Education. San Diego: Elsevier, 259–274.

64.

Sutton

(2004) Emotional regulation goals and strategies of teachers. Social Psychology of Education 7(4): 379–398.

65.

Thomas

Connor

Scott

(2015) Integrating traditional bullying and cyberbullying: Challenges of definition and measurement in adolescents – a review. Educational Psychology Review 27(1): 135–152.

66.

Thomas

Scott

Coates

Connor

(2019) Development and validation of the bullying and cyberbullying scale for adolescents: A multi- dimensional measurement model. British Journal of Educational Psychology 89(1): 75–94.

67.

Zhao

(2012) An empirical study of job satisfaction of Beijing primary school teachers. Teacher Education Research 24(1): 85–92.

68.

Yariv

(2009) The appraisal of teachers’ performance and its impact on the mutuality of principal-teacher emotions. School Leadership & Management 29(5): 445–461.

69.

Zembylas

(2016) The therapization of social justice as an emotional regime: Towards a politics of vulnerability in critical education. Journal of Professional Capital and Community 1(4): 286–301.

70.

Zembylas

(2018) Professional standards for teachers and school leaders: Interrogating the entanglement of affect and biopower in standardizing processes. Journal of Professional Capital and Community 3(3): 223–240.