What Aspects of Principal Leadership Are Most Highly Correlated With School Outcomes in China?

Abstract

Purpose: The purpose of this study is to build a broader framework for Chinese principal leadership and to determine what aspects of principal leadership correlate most highly with school outcomes from the perspectives of both principals and teachers. Method: The data come from a 2013 national student achievement assessment in China comprising 37,749 students in Grade 8 and 9,165 teachers in 613 secondary schools. Adopting Grissom and Loeb’s measurement framework, we use both exploratory and confirmatory factor analyses to examine the structure of principal leadership. Then, a hierarchical linear model is employed to analyze the relationship between principal leadership and five school outcomes, controlling for basic student and teacher demographics and certain school context variables. Findings: We identify many differences and some similarities between China and the United States. We obtain five leadership factors from the principals’ self-rating data (Visibility and Direct Participation, Instruction Organization, Internal Environment Organization, Planning and Personnel, and External Relations) and three leadership factors from the teachers’ rating data (Organization and Management, Instruction and Curriculum, and Visibility and Direct Participation). Regarding student reading achievement and learning efficacy, from both the principals’ and teachers’ perspectives, the most highly correlated aspect is Instruction Organization. For teachers’ occupational stress, job burnout, and teaching efficacy, teachers’ ratings of principal leadership exhibit more significant relationships than does principals’ self-rated leadership. Implications: We explore a broader framework of principal leadership in China. We identify the benefits and analyze alternative views of the indirect effects of principal leadership on students.

Keywords

principal leadership structure school outcomes effectiveness teachers’ perspective principals’ perspective

Purpose of the Study

Principal leadership is widely regarded as playing a significant role in school effectiveness and improvement. However, although consensus has been reached regarding its importance, precisely why it is important remains a controversial and open-ended question. In addition to the methodological problems in most educational studies that have greatly impeded efforts to identify causal effects through a true experimental design, other obstacles have hindered researchers from identifying the relevant qualities. For example, because principals engage in numerous daily practices, what approach should be used to identify the critical behaviors that demonstrate leadership effectiveness? How does one measure a principal’s leadership, or indeed, do survey data comprising principals’ self-reports exhibit the same trends as data obtained from their teachers? What types of dependent variables are plausible? The difficulty lies not only in the complexity involved in principals’ routine jobs but also in the diversity of the contexts in which they work. Dozens of studies have observed the effects of principal leadership on school improvement and student outcomes in different environments. Using a national sample of Chinese students in Grades 4 and 8, Li, Sun, Chen, Zheng, and Zhang (2016) applied the propensity score approach to estimate the effects of principals’ instructional leadership on student performance in math and science. In sum, the study found that principals’ traditional instructional leadership had no significant positive effects on student achievement. The researchers determined that the key to this result was that unduly focal instructional behaviors were too “narrow” for contemporary Chinese principals. This finding led us to consider a broader framework for Chinese principals’ leadership.

Although researchers have developed various models to understand the relationship between leadership and student outcomes (Bossert, Dwyer, Rowan, & Lee, 1982; Hallinger & Heck, 1996, 1998; Leithwood & Levin, 2005; Pounder, Ogawa, & Adams, 1995), it is consistently difficult to ensure systematic empirical validation of the models, whether through direct causal links or mediated effects. Recently, researchers have attempted to use a more integrated model that focuses on broader leadership activities than those covered by the specific models applied in earlier research, with studies increasingly using stronger empirical methodologies, larger sample sizes, and a wide range of related measures to detect theoretical evidence and to provide practical guidance on becoming an effective school leader (Bruggencate, Luyten, Scheerens, & Sleegers, 2012; Grissom & Loeb, 2011; Grissom, Loeb, & Master, 2013; Hallinger, 2003; Leithwood & Levin, 2005; Robinson, Lloyd, & Rowe, 2008; Thoonen, Sleegers, Oort, Peetsma, & Geijsel, 2011). Grissom and Loeb (2011) used combined data from surveys of principals, assistant principals, teachers, and parents and rich administrative data to determine which principal skills were most highly correlated with school outcomes. A factor analysis of a 42-item task inventory identified five skill categories (Instruction Management, Internal Relations, Organization Management, Administration, and External Relations), but only one, principals’ organization management skills, consistently predicted student achievement and other success measures in the U.S. context. This result argues for a broad view of principals’ leadership effectiveness that includes organizational management skills as a key complement to supporting curricula and instruction. It also serves as an example of effective principal leadership from the perspectives of principals, assistant principals, and teachers.

Regarding the selection of dependent variables, most early empirical studies used only student academic achievement as the dependent variable. However, Murphy (1988) concluded that doing so caused two main problems: First, schools’ numerous goals other than achievement were generally overlooked in determining the effects of leadership, and second, this approach reduced researchers’ confidence in drawing conclusions regarding the effects of leadership in schools. Thus, to identify actual evidence of stronger leadership effectiveness, studies have increasingly used multiple outcome measures based on both student and teacher outcomes.

In this study, we apply Grissom and Loeb’s (2011) broad framework of principal leadership effectiveness to a national sample of Chinese secondary school (Grade 8) students to determine the different relationships between principal leadership and school outcomes—not only students’ reading achievement and Chinese learning efficacy but also teachers’ occupational stress and job burnout and Chinese teachers’ teaching efficacy—from the perspectives of both principals and their teachers.

The Structure of Principal Leadership Effectiveness

Determining a principal leadership effectiveness structure first depends on what the principal’s role in the school is. Early in 1984, Glasman (1984) identified two broadly conceived value positions concerning a principal’s proper role, as shown in essays published in the United States beginning in early 1950: One was educational, and the other was administrative. The former highlighted the principal’s role as educational, including instructional, political, middleman, and change agent dimensions. The second position identified dimensions involving administrative authority, planning and evaluation, and management (Glasman, 1984). In reality, however, these two stances were reflected not only in the essays published in the United States but also in the vivid portrayal of the principal’s role and function in the evolutionary history of schooling education. During disparate historical periods in different contexts, schools have assumed specific functions that determined principals’ particular primary roles. Consequently, although a variety of conceptual models have been employed in principal leadership studies, two major approaches have predominated in this field: instructional leadership and transformational leadership.

The term instructional leadership originated during the effective schools movement of the 1970s (e.g., Edmonds, 1979; Leithwood & Montgomery, 1982; Rowan, Bossert, & Dwyer, 1983; Rutter, Maughan, Mortimore, Ouston, & Smith, 1979). After the 1980s, studies moved beyond personal characteristics to focus on the practices of the principals of effective schools, and more recently, studies have made great strides in capturing the typical behaviors of principals with high instructional leadership effectiveness. Joseph Murphy, Ronald H. Heck, and Philip Hallinger are the most active and influential researchers in this field. We are able to develop a concise summary of the development of this field from these authors’ important studies and reviews (e.g., Hallinger, 2005; Hallinger, Bickman, & Davis, 1996; Hallinger & Murphy, 1985; Heck, 1992; Heck, Larsen, & Marcoulides, 1990; Murphy, 1988). For example, after controlling for certain contextual factors (i.e., socioeconomic status [SES] and students’ language backgrounds), Heck (1992) predicted achievement outcomes based on teachers’ and principals’ perceptions of instructional leadership activities in both high schools and elementary schools. His research results indicate that there are three dominant instructional leadership predictors: spending time directly observing classroom practices, promoting discussion on instructional issues, and emphasizing the use of test results for program improvement. It is thus possible that this framework for principal leadership effectiveness has become the most common and accepted understanding of principals’ instructional leadership. One of the most well-known models in this field is that proposed by Hallinger and Murphy (1985), who developed the Principal Instructional Management Rating Scale (PIMRS), a research tool that has been used in over 200 empirical studies conducted in 26 different countries (Hallinger, 2010). This model proposes three dimensions for a principal’s instructional leadership role: defining the school’s mission, managing the instructional program, and promoting a positive school learning climate. These three dimensions are further delineated into 10 instructional leadership functions (Hallinger, Wang, & Chen, 2013). However, as Murphy indicated in 1988, certain activities that principals can perform in a relatively brief period of time, for example, establishing policies that increase the academic rigor of student course loads, may have more important consequences for student learning than other, more labor-intensive activities, for example, classroom observations. This could mean that principals rely primarily on informal, indirect, and facilitative strategies as means of instructional leadership. Murphy (1988) concluded that the more we continue to define instructional leadership in solely behavioral terms, we will increasingly continue to underestimate the amount of administrative instructional leadership in schools. He therefore stressed the danger of specifying instructional leadership behaviors independent of administrators’ management styles.

The other reason for changing the instructional leadership model was the rise of transformational leadership. School restructuring movements in North America arose in the 1990s, creating new expectations for school leaders while not adequately encapsulating those expectations within the context of instructional leadership. In contrast to instructional leadership’s specific focus on the direct coordination, control, and supervision of curricula and instruction, transformational leadership seeks to build the organization’s capacity to determine its purposes and to support changes in teaching and learning practices. Transformational leadership may be viewed as focusing on developing a shared vision and commitment to school change (Hallinger, 2003). After years of empirical research on transformational forms of leadership in schools in response to a variety of restructuring initiatives, Leithwood and Jantzi summarized the evidence on transformational leadership practices and behaviors in schools by including their effects on a variety of school and teacher variables (Leithwood, 1994; Leithwood & Jantzi, 2000, 2006). In sum, their results indicated significant effects of transformational leadership on teachers’ classroom practices but not on student achievement. In the famous Seven Strong Claims About Successful School Leadership, Leithwood, Harris, and Hopkins (2008) developed four broader categories of basic leadership practices for transformational leadership: building a vision and setting directions, understanding and developing people, redesigning the organization, and managing the teaching and learning program. Jan Stewart (2006) presented a comprehensive review of the conceptual and empirical development of transformational leadership. The author argued that transformational leadership will continue to evolve to adequately respond to schools’ changing needs in the context of educational accountability and school reform.

In our view, the evolution of the principal leadership concept and structure is also a process of using a more integrated model to measure the effectiveness of principal leadership. Just as coding principals’ routine duties is not the same as capturing the core practices of their leadership effectiveness, focusing only on certain typical behaviors is not sufficient to form a framework for principal effectiveness. Waters, Marzano, and McNulty (2003) also found this to be the case in their research on balanced leadership. They presented a synthesis of more than 70 research studies conducted from the 1970s through the 2000s and found 21 leadership practices that were correlated with improved student achievement. These practices included establishing a set of standard operating procedures and routines; being directly involved in designing and implementing the curricula, instruction, and assessment practices; providing teachers with materials and professional development; being knowledgeable about current curricula, instruction, and assessment practices; and monitoring the effectiveness of school practices and their effect on student learning.¹ Robinson’s systematic review produced 27 published studies that quantified the relationship between types of school leadership and a range of social and academic student outcomes. Inspecting the 199 survey items used to measure school leadership revealed five sets of leadership dimensions: establishing goals and expectations; resourcing strategically; planning, coordinating, and evaluating teaching and curricula; promoting and participating in teacher learning and development (with strong average effects); and ensuring an orderly and supportive environment (Robinson, 2007; Robinson et al., 2008). With the Leadership Improvement for Student Achievement project, a comprehensive Pashiardis–Brauckmann holistic leadership framework was developed by Pashiardis and his team, which explored how school leadership along with other context variables directly or indirectly affect student achievement in secondary school. In this framework, they used an integrated leadership that entailed a second-order factor model of school leadership—the leadership radius, which consisted of five first-order factors reflecting the domains or styles that principals employed in their work: instructional style, structuring style, participative style, entrepreneurial style, and personnel development style. This framework was validated in some quantitative studies across eight European countries, like the United Kingdom, Netherlands, Germany, Cyprus, and so on (Brauckmann & Pashiardis, 2011; Pashiardis, 2014; Savvides & Pashiardis, 2016). All of these findings are in line with the direction of integrated leadership that incorporates the development of shared instructional leadership combined with the qualities of transformational leadership, which is the best predictor of the quality of student work (Bruggencate et al., 2012; Marks & Printy, 2003; Neumerski, 2013; Robinson et al., 2008). This perspective can also be observed in articles about preparation for principals, which indicate the importance of an integrated model of principal capacities (Brazer & Bauer, 2013; Ylimaki, 2012). Recently, studies have increasingly employed larger scale administrative data and new measures of principal leadership behavior to determine whether these effects are really of the same magnitude, as reported by early studies (Bruggencate et al., 2012; Grissom et al., 2013; Grissom & Loeb, 2011; Horng, Klasik, & Loeb, 2010; May, Huff, & Goldring, 2012; Orphanos & Orr, 2014; Williams, 2015).

In this study, we would structure the leadership effectiveness in an integrated model by applying Grissom and Loeb’s (2011) broader framework. From this perspective, a set of principals’ day-to-day tasks (not narrowly focused on the traditional leadership of teaching and learning) were incorporated into the leadership model, for example, hiring staff, supervising school administration system, communicating with parents, and managing school budgets and resources. Furthermore, our instrument, being composed of a list of these tasks, focused on the effectiveness of principal leadership—that is, how good the principal is at accomplishing a task rather than the frequency or extent of a particular behavior.

The Measurement Perspectives of Principal Leadership

Considering the perspective from which principals and/or their teachers evaluate the effectiveness of a principal’s leadership should be a basic approach to measurements in this field. Surprisingly, however, in our reading of the corresponding literature, we found very few published journal articles that focused on this topic. It appears that most of the evidence is from teachers’ perspective, and we suspect that there may be two reasons for this phenomenon.

First, data from teachers’ perspective are more readily available than data from principals’ perspective, particularly in large-scale inquiries. A significant portion of the empirical evidence appears to be from teachers’ perspective, including the evidence from early studies on effective schooling that indicated the principal’s crucial role in students’ academic performance (e.g., Andrews & Soder, 1987), newer studies that have employed sophisticated methodologies to examine the relationships between student learning and principal leadership (e.g., Supovitz, Sirinides, & May, 2009), and even a national U.S. survey exploring how leadership affected student achievement (Louis, Dretzke, & Wahlstrom, 2010). We obtained additional evidence from the aforementioned meta-analysis by Robinson et al. (2008) that addressed the paradoxical differences between the qualitative and quantitative evidence on leadership effects. Their search yielded 27 studies, published between 1978 and 2006, that provided evidence of the links between leadership and student outcomes. We examined the original research methods of all 27 studies and found that 20 of the 27 used leadership data from teachers’ perspective, 6 considered both teachers’ and principals’ perspectives, and only 1 considered solely the principals’ perspective. In addition to large-scale inquiries, the data from principals’ perspective could be obtained only by case study: For example, Pashiardis (2001) and Pashiardis, Costa, Mendes, and Ventura (2005) compared the principals’ perceptions of leadership styles with the teachers’ perceptions in a high school of Cyprus and in a Portuguese public school, respectively. The perceptions of the teachers were collected by questionnaires and the perceptions of the principals were obtained by semistructured interview. Restricted by the sample size, the results would be limited to generalize.

The second reason for the focus on teachers’ perspective relates to the trend of studies increasingly adopting the “indirect effects” model to detect the influence of principal leadership on school outcomes. For example, Mulford and Silins (2003) conducted a large-sample, longitudinal research project (Leadership for Organizational Learning and Student Outcomes) and concluded that leadership contributes to organizational learning, which in turn influences outcomes in schools’ core activities, teaching, and learning. They argued that leadership influences how teachers organize and conduct their instruction and educational interactions with students and the challenges and expectations they place on their students. Finally, the authors noted that students’ positive perceptions of teachers’ work directly promote their participation and engagement in school and their academic self-concepts. Based on this effectiveness model, we expect that studies will increasingly consider primarily the perspective of teachers, or at least that of both principals and their teachers. However, in a meta-analysis of studies of PIMRS reliability, Hallinger (2010) found that researchers consistently reported significant differences between teachers’ and principals’ perceptions of the principals’ instructional leadership. Moreover, principals’ self-reported scores tended to be substantially higher than scores obtained from teachers, and these “role set” differences in PIMRS ratings extend to contexts other than the United States. In addition, despite differences in the magnitude of ratings obtained by the two groups, researchers have often reported a similar pattern in their ratings on the various subscales that comprise the PIMRS (Hallinger et al., 2013).

In Grissom and Loeb’s (2011) research that triangulated principals’ effectiveness, assistant principals’ ratings of principals’ effectiveness were lower and more varied than the principals’ self-ratings, although the patterns across the items were quite similar. Again, however, as with the trend found in the principals’ data, after controlling for school characteristics, organization management was positively and statistically significantly related to school accountability, teacher satisfaction, and parent survey assessments of school performance.

Thus, having established the existence of a consensus on the use a more integrated model to measure principal leadership effectiveness, we might ask what model is best suited for modern-day China and how the candidate models relate to or differ from one another depending on the principals’ and teachers’ perspectives. Will all models predict the same pattern of relationships between principal leadership and school outcomes? Answering these questions will undoubtedly benefit not only the leadership theory establishment but also training and recruitment practices for principals in China.

Method

Research Design

In this study, we first explore what constitutes principal leadership effectiveness and then compare the differences between principals’ and teachers’ perspectives. Finally, we examine principal leadership effectiveness using empirical data from both sides. The basic research questions are as follows:

Research Question1: What are the dimensions of principal leadership as perceived by principals and their teachers, and what is the relationship between these two perspectives?

Research Question 2: What are the differences between principals’ and teachers’ ratings of leadership effectiveness, and do the two groups agree on the relationship between principal leadership and student achievement?

Research Question 3: What are the key factors in principal leadership that correlate most highly with school outcomes from both perspectives?

Data Sources

The data for this study came from a 2013 Chinese national student achievement assessment that was conducted by the National Assessment Center for Education Quality (NAEQ).² A three-stage sampling procedure with unequal probabilities was adopted for this assessment. In the first stage, using indicators such as location, the ratio of urban to rural students, and the level of education and economic development in the particular district, 104 districts/counties were selected from 32 Chinese provinces (31 provinces and the Xinjiang Production and Construction Corp). In the second stage, all schools within each district/county were divided into different hierarchies based on school education quality (good, medium, poor) and location (city, county, rural). A total of 613 secondary schools were selected from these 104 districts and counties. In the final stage, 37,749 Grade 8 students from the 613 secondary schools were randomly selected, with approximately 64 students per school. Based on this data set, our original sample thus consisted of 613 secondary school principals, 9,165 teachers, and 37,749 students.

In this assessment, each student was given a survey and a randomly assigned booklet containing a Chinese reading achievement test. This test was based on a matrix sampling design, and student achievement was represented as a linear transformation of the ability parameter, estimated by item response theory.³ Linked to the student achievement and questionnaire data, a principal survey, which included basic background, administrative data, and a leadership scale, was completed by the principal of each participating school. Simultaneously, a teacher survey, which included demographic information, measures of occupational stress and job burnout, and the same leadership scale as in the principals’ survey, was completed by the head teachers and subject teachers, including teachers of Chinese, music, sports, and art in Grade 8. Furthermore, teaching efficacy was also included in the Chinese teachers’ survey.

Regarding the process of missing data, because of the nested data construction, the deletion method, for example, listwise deletion, at the school level would have caused a large data loss at the student and teacher levels. Thus, instead of deletion, the imputation method, for example, expectation maximization and multiple imputation (MI), was preferred in the current scenario. Furthermore, Little’s missing completely at random tests were used to detect the missing pattern in the totality of principal cases. The results showed that χ² = 13,838.271 (degrees of freedom = 13,467), significant at the .05 level, which indicated that the data were not missing completely at random. Again, the imputation method was more suitable than the deletion method in this context. Among the various imputation methods, we chose the MI method, which is commonly used by researchers. We defined a case as invalid if it had more than 10 items missing (11% missing responses), and based on this rule, the proportion of valid principal questionnaires was 95.1% (583 final valid cases). Then, MI was conducted on the valid school-level missing data using Statistical Analysis System. The number of samples imputed was five. For the student and teacher data, we used the listwise method, which is the default setting in most statistical software, to process the missing data. As a result, the proportion of valid student questionnaires was 77.1% (29,104 final valid cases), the proportion of all valid teacher questionnaires was 75% (6,861 final valid cases), and the proportion of valid Chinese teachers’ questionnaires was 97% (2,522 final valid cases). Table 1 shows the descriptive statistics of all variables used in the analyses and the principals’ demographic variables.

Table 1.

Descriptive Statistics.

	N	%	M	SD
Student-level variables
Gender: male	37,160	51.3	—	—
Family structure: complete	37,182	90.8	—	—
Socioeconomic status	29,889	—	0.00	1.00
Teacher-level variables
Gender: male	9,015	44.3	—	—
Age	9,092	—	36.78	7.16
Teaching experience	7,860	—	15.41	8.32
School-level variables
Principal characteristics
Gender: male	583	89.4	—	—
Age	583	—	46.18	5.6
No. of years in current job	583	—	4.53	3.89
No. of years as a middle-level cadre	583	—	7.43	4.1
Ratio of still currently teaching	583	47.3	—	—
School characteristics
School size	583	—	1385.4	1180.97
School location: city	583	28.6	—	—
School location: county	583	18.7	—	—
School location: rural	583	52.7	—	—
Percentage of girls in the school	583	—	47.46	6.19
Student/teacher ratio	583	—	11.88	5.44
Chinese teachers’ sufficiency	583	—	3.17	0.79
General teaching resources	583	—	3.02	0.55
Chinese teaching resources	583	—	2.62	0.65

Note. The descriptive statistics of the school-level variables were averaged using five random imputation samples.

Measurements

Independent variable: Principal leadership

To measure principal leadership effectiveness, we adopted a revised version of Grissom and Loeb’s (2011) measurement framework, which contained 40 items. To better align the items’ expressions and contents with Chinese principals’ behaviors and activities, we conducted extensive in-depth, one-on-one interviews with principals across China. The principals were from different representative regions, including city, county, and rural areas. The interview protocol consisted of three questions: First, “Could you please judge whether there was anything inappropriate or omitted concerning the five dimensions of principal leadership (Instruction Management, Internal Relations, Organization Management, Administration, and External Relations)?” Second, “Please estimate the fitness of the 40 translated leadership items—that is, whether these 40 tasks are common in your daily work? Third, “Please list any tasks that are common in your daily work but that are not included among these 40 items.” Each interview was recorded and transcribed. The results showed that this five-dimensional framework basically fit Chinese principals’ leadership behavior, but some of the terms overlapped. We deleted or changed the expression of some items that were not correlated with the tasks that most of the principals we interviewed undertook. As a result, our principal survey comprised 36 revised and indigenous tasks rated on a 5-point response scale (never do, did but was ineffective, did but was only a little effective, did and was effective, and did and was very effective). Specific examples of the interview results are presented in the Discussion section.

To obtain different leadership assessment perspectives, we simultaneously conducted a similar survey of the teachers at each participating school. The teacher survey included the same 36 principal tasks, and the teachers were asked to assess how effective their principals were at accomplishing each task.

Both the principals’ leadership self-ratings and the teachers’ principal leadership ratings were computed into factor scores in the factor analysis process.

Dependent variables

All of our school outcome variables can be divided into two categories: student outcomes and teacher outcomes. Student outcomes include students’ Chinese reading achievement and Chinese learning efficacy. Teacher outcomes include occupational stress, job burnout, and teaching efficacy; these teacher outcomes have been the major concerns in educational research. Occupational stress and job burnout, which are treated as a psychological syndrome entailing emotional exhaustion, depersonalization, and reduced personal accomplishment, have effects on the lives of teachers, administrators, students, and all of society (Davis & Wilson, 2000; Mazur & Lynch, 1989). Teaching efficacy, referring to beliefs in one’s capabilities to address the learning needs of students, can influence how well students learn (Brouwers & Tomic, 2000; Wahlstrom & Louis, 2008). The school outcomes are described in detail as follows.

Students’ Chinese reading achievement

The students’ Chinese reading achievement measures came from a mixed model: The Rasch model was fitted for multiple-choice items, and a partial credit model was fitted for subjective items. All responses from the six booklets were simultaneously estimated by the expectation maximization algorithm. The students’ Chinese reading achievements were then transferred to have mean 500 and standard deviation 100.

Students’ Chinese learning efficacy

The students’ Chinese learning efficacy was measured by one item, as in Zimmerman, Bandura, and Martinez-Pons (1992), on the student survey: “Do you think you can learn Chinese well?” A 4-point response scale (I can completely, I can basically, it is a little difficult for me, and I can never) was used.

Teachers’ occupational stress

The teachers’ occupational stress was measured using a composite score that averaged the 11 teacher survey scale items. We constructed this scale and periodically revised it based on the situations of the Chinese teachers. Cronbach’s alpha coefficient was .84. Using a 4-point response scale (no pressure, low pressure, pressure, and high pressure), the occupational stress measure asks the respondents to select the degree of pressure they feel with regard to the following:

Students’ grade assessments between classes

Heavy teaching burden

Student safety

Formality of school activities and inspections

Students’ discipline violations

Relationships with colleagues

Personal physical condition

Requirements from society, the principal, or students’ parents

Personal professional development

Relationships with leaders

Curriculum reform.

Teachers’ job burnout

The teachers’ job burnout scale was adapted from Maslach’s Burnout Inventory (Schaufeli, Leiter, & Maslach, 2009), which is also a composite score averaged from a nine-item teacher survey. Cronbach’s alpha coefficient was .60. Using a 4-point response scale (never, occasionally, sometimes, and frequently), the job burnout scale asks respondents to select the frequency of the following phenomena:

Feel full of energy when I get up

Feel exhausted when working

Think that if I work hard, I can get good results

Am emotionally drained by work

Feel that I work like a machine without any effect

Easily feel down when working

Often feel sleepless and have headaches caused by work

Like to work with people

Feel exhausted after work; don’t want to do anything.

Chinese teachers’ teaching efficacy

Seven items measuring Chinese teachers’ teaching efficacy were adapted from the self-efficacy scale in the Organisation for Economic Co-operation and Development’s (OECD) 2008 Teaching and Learning International Survey (TALIS; OECD, 2010). Here, teaching efficacy is measured based on the Chinese teachers’ perceptions of their confidence in their teaching activities, such as being able to: (a) answer students’ questions related to Chinese, (b) demonstrate various types of reading skills, (c) arrange challenging activities and exercises for the top students, (d) adjust the teaching method to arouse students’ learning interest, (e) help students understand the value of language, (e) write using the blackboard, and (f) write compositions. Cronbach’s alpha coefficient was .77. Using a 3-point response scale (no confidence, some confidence, and full confidence), the above items were averaged into the Chinese teachers’ teaching efficacy score.

Covariate variables

At the student level, we controlled for the basic demographic characteristics of student gender, family structure, and SES, which are highly related to student outcomes. At the teacher level, we controlled for demographic variables—including teachers’ gender, age, and teaching experience—which are strongly related to their occupational stress, job burnout, and teaching efficacy. At the school level, we controlled for school demographics—including school size and location, the percentage of girls in the school, Chinese teachers’ sufficiency, and the student/teacher ratio—along with Chinese teaching resources and general teaching resources. All of these variables have been found to have a significant effect on school outcomes (Lee & Smith, 1997; Miles & Darling-Hammond, 1998; Orlitzky & Benjamin, 2003; U.S. Department of Education, National Center for Education Statistics, 1996; Wenglinsky, 1997). The descriptive statistics on all covariates are shown in Table 1.

Student gender, family structure, and SES

Student gender is a binary variable: 0 for male and 1 for female. Student family structure is also a binary variable: 0 for a complete family with married parents and 1 for an incomplete family with divorced parents or a parent who has passed away. SES, adapted from the index of economic, social, and cultural status in the Programme for International Student Assessment (OECD, 2012), is a standardized composite score, aggregated by a two-order confirmatory factor analysis (CFA) model. In this model, SES is extracted by three first-order factors: both parents’ education level, the family’s material resources (whether the family has a telephone, television, computer, or car), and both parents’ occupations. Table 1 shows that the percentage of male students is 51.3% and that the percentage of students in complete families is 90.8%.

Teacher gender, age, and teaching experience

Teacher gender is a binary variable—0 for male and 1 for female—and teacher age is a continuous variable. Teacher experience is a continuous variable that indicates the number of years that a teacher has been teaching. As shown in Table 1, the percentage of male teachers is 44.3%, the average age of secondary school teachers is 37, and their average teaching experience is 15 years.

School size, school location, percentage of girls in school, student/teacher ratio

School size is a continuous indicator of the number of students enrolled in the school. The three-category school location variable was dummy coded into two binary variables—School Location 1 is a city, and School Location 2 is a county—with rural as the reference group. The percentage of girls in the school and the student/teacher ratio are both continuous variables: The latter is the total number of students divided by the total number of teachers in the school. Table 1 shows that the average size of a secondary school in China is 1,385 students, and the school location distributions are 28.6% in a city, 18.7% in a county, and 52.7% in a rural area. The average percentage of girls in a school is 47.5, and the student/teacher ratio is 11.9.

Chinese teachers’ sufficiency

Whether there are a sufficient number of Chinese teachers was measured by asking the principals, “Are there enough Chinese teachers in your school to meet your teaching needs?” A 4-point response scale (serious shortage, not quite enough, basically sufficient, and fully sufficient) was used. The average score was 3.17, indicating that there is essentially a sufficient number of Chinese teachers in secondary schools.

General teaching resources and Chinese teaching resources

General teaching resources and Chinese teaching resources were both composite scores that were averaged using, respectively, nine and four items from the principals’ survey, and both were scored on a 4-point scale (none, have but cannot meet the teaching needs, have and can meet the basic teaching needs, and have and can completely meet the teaching needs). For general teaching resources, the nine items were (a) instructional materials (e.g., textbooks, teaching references); (b) office supplies; (c) school buildings and grounds; (d) heating, cooling, and lighting systems; (e) classroom size and facilities; (f) computers for teachers’ use; (g) computers for students’ use; (h) multimedia devices (including interactive electronic whiteboard); and (i) network resources (Internet access). For the Chinese teaching resources, the four items were the following: (a) books and materials related to Chinese teaching, (b) books in the library or file room, (c) electronic curriculum resources and image data (including software) related to Chinese teaching, and (d) material objects, models, and teaching aids. The average scores were 3.02 and 2.62 for general teaching resources and Chinese teaching resources, respectively, indicating that the general secondary school teaching resources can essentially meet the teaching needs, but the Chinese teaching resources are somewhat insufficient.

Analytic Methods

Factor analysis

Our first set of questions determined the structure of principal leadership according to the responses of both principals and teachers and the relationship between the two. First, data from one-half of the principal sample were randomly selected in SPSS 20.0 and used to conduct an exploratory factor analysis (EFA) using the principal component extraction with eigenvalues of greater than 1.0 and varimax rotation.⁴ One task from the original 36 was deleted due to low factor loading across all leadership factors (see the Results section). Second, a CFA was performed in Mplus Version 7 using the data from the second half of the principal sample to statistically test the structure using the remaining 35 items. Model fit was evaluated using chi-square goodness of fit, the root mean squared error approximation (RMSEA; Steiger, 1990), the Tucker–Lewis index (TLI; Tucker & Lewis, 1973), and the comparative fit index (CFI; Bentler, 1990).

The same process was followed to explore principal leadership structure from the teachers’ perspective. To incorporate the same leadership tasks in both perspectives, we used principal component extraction for the same 35-item tasks as the principals’ EFA. Factors were retained if they had an eigenvalue greater than 1.0 and rotated using varimax rotation. Then, a CFA was conducted to verify this structure (see the Results section).

Finally, we calculated the correlation coefficients between the two perspectives’ leadership factor scores.

Descriptive analysis

Our second set of questions compared principals’ leadership effectiveness from the principals’ and teachers’ perspectives and examined the relationship between the two perspectives and student achievement. The questions were descriptive. Thus, we simply calculated and compared the principals’ and teachers’ ratings on the 35 leadership tasks—in terms of means, standard deviations, proportions of Responses 4 and 5 (did and was effective and did and was very effective) on the ratings scale, and the correlation coefficient with students’ Chinese reading achievements in SPSS 20.0.

Hierarchical linear model

Our third and final question related to the different effects of principal leadership on school outcomes from the two perspectives under study. We used a hierarchical linear model (HLM) to detect differences in the effectiveness of principal leadership. Focusing on school-level leadership, we aggregated teachers’ principal leadership ratings to the school level. This analysis was conducted using HLM Version 7 software.

First, we ran a model with no predictors, the unconditional model (see Equation 1), to calculate the intraclass correlation (ICC). This result can be interpreted as an indicator of group homogeneity: As the Level 2 units become more heterogeneous, the ICC increases. Because traditional ordinary least squares assumes that there is no clustering in variance, positive ICCs can thus potentially bias statistical inferences. Then, the HLM is tailored to situations in which the ICC is high (Steenbergen & Jones, 2002).

Level 1 : A_{i j} = β_{0 j} + ε_{i j},

Level 2 : β_{0 j} = γ_{00} + μ_{0 j},

We then chose the random intercept model (see Equation 2) to determine the relationship between principal leadership and different school outcomes:

Level 1 : A_{i j} = β_{0 j} + β_{1 j} X_{i j} + ε_{i j},

\begin{array}{l} Level 2 : β_{0 j} = γ_{00} + γ_{01} W_{j} + μ_{0 j}, \\ β_{1 j} = γ_{10}, \end{array}

where Level 1 is the student level and the teacher level, Level 2 is the school level, and A_ij is the Level 1 dependent variable for student i or teacher i (= 1, . . ., N_j) nested in a school j (=1, . . ., J), including students’ reading achievement and Chinese learning efficacy and teachers’ occupational stress, job burnout, and teaching efficacy. X_ij is the Level 1 predictor, which is the vector of either student-level covariates, including students’ gender, family structure and SES, or teacher-level covariates, including teachers’ gender, age, and teaching experience, and W_j is the Level 2 predictor, which is the vector of principal leadership factor scores and school-level covariates, such as school size, school location, percentage of girls in the school, and the student/teacher ratio. β Denotes regression parameters, which are not fixed but, instead, vary across Level 2 units (as indicated by the j subscripts on the regression parameters), γ denotes the fixed Level 2 parameters, and ϵ_ij and µ_0j are the Level 1 and Level 2 disturbance terms, respectively.

Results

The Factor Analysis of Principal Leadership Effectiveness

Principals’ perspective

Our first research question is to explore the structure of principal leadership from the perspectives of principals and teachers. First, we consider the principals’ perspective. As described above, in the first stage of EFA using one-half of the principal sample, one item—“arranging teaching jobs by the teachers’ performance”—was dropped because of the low loadings across all factors. We then conducted an EFA among the remaining 35 items using the principal component extraction and varimax rotation. Five factors with eigenvalues greater than 1.0 were retained (see Figure 1). Bartlett’s test (χ² = 5574.16, p ≤ .00) and the Kaiser–Meyer–Olkin measure of sampling adequacy (.95) supported the appropriateness of the factor analysis. Then, a CFA was performed using the data from the second half of the random split of the principal group. The results indicated that this five-factor model suggested by the EFA produced an acceptable fit, χ²/(550) = 1.72, p ≤ .00; RMSEA = .06, CFI = .87, TLI = .86; the factor loadings ranged from .43 to .87.⁵ We then labeled these five dimensions of principal leadership as Visibility and Direct Participation, Instruction Organization, Internal Environment Organization, Planning and Personnel, and External Relations. Cronbach’s alpha coefficients of these five dimensions were .87, .80, .87, .77, and .83, respectively. Three of these—Instruction Organization, Internal Environment Organization, and Planning and Personnel—strongly reflected the principals’ organizational abilities but in terms of different aspects of the job.

Figure 1.

Comparison of principals’ and teachers’ ratings of task effectiveness grouped by principals’ perspective factors.

Visibility and direct participation

The first dimension, labeled Visibility and Direct Participation, represents the principals’ task of personally participating in school activities. Nine items loaded highly on this factor. The highest four items are solving students’ practical problems (.64), counseling staff about the contradictions with other staff (.64), attending school activities (.62), and informally talking to teachers about students (.58).

Instruction organization

The second dimension comprises tasks related to principals’ role in the development of teaching and learning, particularly in guiding, evaluating, and monitoring teaching, and is labeled Instruction Organization. Eight items belong to this factor. The four items that loaded most highly are using students’ school management and teaching evaluations to measure teaching effort (.71), using students’ evaluation results to evaluate teachers’ teaching (.66), learning and grasping new education ideas and new teaching research results (.59), and using different aspects of school data to inform instruction (.59).

Internal environment organization

The third dimension represents the principals’ task of creating a positive environment or climate within the school. We labeled this dimension Internal Environment Organization. Seven items belong to this factor. The four items with the highest loadings are the tasks related to creating a positive campus environment and strong relationships with staff: interacting socially with staff (.77); creating a safe, easy school atmosphere (.73); cooperating with school middle-level personnel (.66); and establishing campus order (.63).

Planning and personnel

The fourth dimension of principals’ task effectiveness is labeled Planning and Personnel. These sets of tasks concern establishing long-term goals and selecting and managing personnel for the school’s development. Seven items belong to this dimension. The four highest loading items are evaluating curricula (.63), formulating long-term school plans and guaranteeing their implementation (.59), hiring staff (.59), and implementing teachers’ professional development plans (.57).

External relations

The final dimension of principal leadership is External Relations. Only four items regarding working with communities and organizations outside of the school belong to this dimension: networking with regional management departments to obtain resources for the school (.81), interacting with social organizations or individuals to obtain resources for the school (.80), fundraising (.71), and working with the local community (.70).

Teachers’ perspective

After confirming the structure of principal leadership from the principals’ perspective, we wanted to know whether teachers perceived the same principal leadership structure as principals did. We thus engaged in the same process. First, in conducting the EFA using the principal component extraction method with eigenvalues greater than 1.0 and varimax rotation, three latent dimensions emerged from the teachers’ rating data (see Figure 1). The Bartlett’s test (χ² = 124895.30, p ≤ .00) and the Kaiser–Meyer–Olkin measure of sampling adequacy (.98) also supported the appropriateness of factor analysis. In addition, a CFA was performed to evaluate the model using the data from the second half of the random split of the teacher group. The results indicated that the three-factor model suggested by the EFA produced an acceptable fit, χ²/(557) = 24.78, p ≤ .00; RMSEA = .08, CFI = .90, TLI = .89; the factor loadings ranged from .65 to .85. We labeled these three dimensions of principal leadership Organization and Management, Instruction and Curriculum, and Visibility and Direct Participation. Cronbach’s alpha coefficients of these three dimensions were .96, .95, and .94, respectively.

Organization and management

The first dimension of teachers’ rating of principal leadership is tasks related to addressing administrative affairs within and outside the school. We labeled these principal leadership tasks as Organization and Management, and there are 14 items in this dimension. Eight of these tasks overlap with the Internal Environment Organization and External Relations facets of the principals’ leadership ratings; the other six items included three from Planning and Personnel, two from Visibility and Direct Participation, and one from Instruction Organization in the principals’ leadership ratings (see Figure 1).

Instruction and curriculum

The second dimension comprises tasks that primarily focus on school teaching activities, which we labeled Instruction and Curriculum. Twelve items loaded highly on this dimension: seven from the principals’ categories of Instruction Organization, three from Planning and Personnel, and two from Visibility and Direct Participation.

Visibility and direct participation

The last of the teachers’ leadership rating dimensions captures the principals’ tasks related to their direct participation in various school activities. We labeled this dimension Visibility and Direct Participation, which was also a dimension on the principals’ rating scale. Nine items belong to this dimension: five of them overlap with the principals’ Visibility and Direct Participation, three with Internal Environment Organization, and one with Planning and Personnel.

Relationship between these two perspectives

Principals appear to rate their leadership effectiveness differently from their teachers. The five leadership effectiveness dimensions from the principals’ perspective appear to correlate more with the principals’ specific tasks, whereas the three dimensions from their teachers’ perspectives are more extensive and obscure. This could be because teachers are not aware of their principals’ actual daily tasks. Table 2 shows the correlation coefficients between these two perspectives on leadership effectiveness. Three of the principals’ rating factors, Instruction Organization, Internal Environment Organization, and Planning and Personnel, have somewhat positive significant correlations with the teachers’ rating factors. As mentioned above, these three factors best reflect principals’ organizational skills.

Table 2.

Correlation Matrix for the Two Perspectives of Principals’ Leadership Effectiveness.

Teacher^R	Principal^R
Teacher^R	Visibility and Direct Participation	Instruction Organization	Internal Environment Organization	Planning and Personnel	External Relations
Organization and Management	.054	.104*	.114**	.079	−.008
Instruction and Curriculum	.027	.015	.015	.138**	−.009
Visibility and Direct Participation	.045	.059	.112*	.003	.006

Note. Principal^R represents principals’ leadership ratings. Teacher^R represents teachers’ leadership ratings.

p < .05. **p < .01.

Comparison of Principals’ Leadership Effectiveness From the Two Perspectives

Our second research question is to compare principals’ leadership effectiveness from the perspectives of both principals and their teachers. We compared the means, standard deviations, and proportions of Responses 4 and 5 (did and was effective and did and was very effective) on the ratings scale for all 35 items. Figure 1 illustrates these comparisons by sorting each item within the five dimensions from the principals’ perspective by the item mean. First, the means of the principals’ self-ratings are higher than those of their teachers’ ratings on most tasks, but on the four External Relations items, the situation is reversed. We attribute this finding to teachers’ difficulties in precisely perceiving their principals’ work with the community and organizations outside of the school. Second, the standard deviations of the principals’ ratings on the 35 items are always lower than those of the teachers, except for “hiring staff,” which suggests little variation across principals on each item. Third, the trend of the proportions of choosing Responses 4 and 5 (did and was effective and did and was very effective) on the ratings scale are similar to the means: 83% of principals chose 4 or 5 for most tasks, whereas only 71% of the teachers did so.

We also compared these two correlation coefficients with students’ Chinese reading achievement (see Figure 1). For the teachers’ leadership ratings, each of the 35 items has a positive and significant correlation with students’ Chinese reading achievement, with the correlation coefficients ranging from .21 to .45. For the principals’ leadership self-ratings, fewer than half of the items have somewhat positively significant correlations with the students’ Chinese reading achievement, mostly in Instruction Organization and Planning and Personnel, with the significant correlation coefficients ranging from .09 to .23.

The HLM of Principal Leadership on Particular School Outcomes From the Principals’ and Teachers’ Perspectives

Our final research question concerns the key principal leadership factors that most highly correlated with school outcomes from both principals’ and their teachers’ perspectives. Choosing five school outcomes—students’ Chinese reading achievement scores, students’ Chinese learning efficacy, teachers’ occupational stress, teachers’ job burnout, and Chinese teachers’ teaching efficacy—we used HLM to detect the different relationships. As described in the Analytic Methods section, we first ran an unconditional model to calculate the ICCs of the five school outcomes. Table 3 shows the results. The ICC (ρ) represents the variance explained by the school level in the total variance. As the ICC increases, so do the differences among schools. The ICCs of our five school outcomes were .33, .07, .12, .15, and .10, all of which are significant. Therefore, it is fair to assume that using HLM is suitable for our analysis.

Table 3.

Parameter Estimates of the Unconditional Model.

Fixed Effect Coefficient (SE)	Students’ Reading Achievement	Students’ Chinese Learning Efficacy	Teachers’ Occupational Stress	Teachers’ Job Burnout	Chinese Teachers’ Teaching Efficacy
γ₀₀	501.30 (2.35)	1.91 (0.01)	2.81 (0.01)	2.73 (0.01)	2.53 (0.01)
Random effect
Intraclass correlation, ρ	.33***	.07***	.12***	.15***	.10***
Level 1 variance, σ²	6401.11	0.37	0.25	0.35	0.12
Level 2 variance, τ²	3083.26	0.03	0.04	0.06	0.01

Note. Coefficients are reported. Standard errors are in parentheses.

***

p < .001.

Next, we ran a random intercept model to determine the association between principal leadership and the five school outcomes from both principals’ and their teachers’ perspectives. For student outcomes in Chinese reading achievement and Chinese learning efficacy, we controlled for students’ gender, family structure, and SES at the student level; at the school level, we controlled for school size, school location, percentage of girls in the school, Chinese teachers’ sufficiency, and Chinese teaching resources. For teacher outcomes regarding occupational stress, job burnout, and Chinese teachers’ teaching efficacy, we controlled for teachers’ gender, age, and teaching experience at the teacher level and for school size, school location, general teaching resources, and the student/teacher ratio at the school level. Table 4 and Table 5 show the results for the principals and teachers, respectively. The results indicate that regarding principals’ self-ratings of leadership (see Table 4), Instruction Organization is positively related to students’ Chinese reading achievement (γ = 6.16, SE = 2.45) and Chinese learning efficacy (γ = −0.03, SE = 0.01). Visibility and Direct Participation is negatively but only slightly related to teachers’ occupational stress (γ = 0.02, SE = 0.01). Meanwhile, External Relations is also negatively related to both teachers’ occupational stress and job burnout, with coefficients of 0.04 (SE = 0.01) and 0.03 (SE = 0.01), respectively. The other leadership factors are nonsignificant. For the teachers’ leadership ratings (see Table 5), Instruction and Curriculum is positively related to students’ Chinese reading achievement and Chinese learning efficacy, with coefficients of 24.61 (SE = 3.98) and −0.10 (SE = 0.01), respectively. However, with regard to teacher outcomes, the teachers’ leadership ratings show more significant relationships than the principals’ self-ratings. Visibility and Direct Participation is positively related to teacher occupational stress (γ = −0.10, SE = 0.02), and all three factors in teachers’ leadership ratings show positive relationships with teacher job burnout, with coefficients of −0.06 (SE = 0.03), −0.07 (SE = 0.03), and −0.16 (SE = 0.03). Instruction and Curriculum and Visibility and Direct Participation are positively associated with Chinese teachers’ teaching efficacy, with coefficients of 0.11 (SE = 0.02) and 0.06 (SE = 0.02), respectively.

Table 4.

Hierarchical Linear Modeling Results From the Principals’ Perspective.

Fixed Effects Coefficient (SE)	Students’ Reading Achievement	Students’ Chinese Learning Efficacy^a	Teachers’ Occupational Stress^a	Teachers’ Job Burnout^a	Chinese Teachers’ Teaching Efficacy
Student predictors
Gender	32.74 (1.19)***	−0.19 (0.01)***	—	—	—
Family structure	−3.31 (2.07)	0.04 (0.02)*	—	—	—
Socioeconomic status	14.00 (0.96)***	−0.08 (0.01)***	—	—	—
Teacher predictors
Gender	—	—	0.02 (0.01)	0.09 (0.02)***	−0.06 (0.02)***
Age	—	—	0.01 (0.00)***	0.00 (0.00)	−0.00 (0.00)
Teaching experience	—	—	−0.00 (0.00)*	0.00 (0.00)	0.00 (0.00)
School predictors
School size	0.01 (0.00)**	−0.00 (0.00)*	0.00 (0.00)	−0.00 (0.00)	0.00 (0.00)
School Location 1: city-rural	60.16 (5.79)***	−0.17 (0.02)***	−0.02 (0.03)	0.08 (0.04)*	0.08 (0.02)**
School Location 2: county-rural	34.21 (6.47)***	−0.11 (0.02)***	0.00 (0.03)	0.03 (0.04)	0.08 (0.02)**
% girls in the school	0.82 (0.36)*	−0.00 (0.00)	—	—	—
Chinese teachers’ sufficiency	0.13 (2.57)	−0.00 (0.01)	—	—	—
Chinese teaching resources	16.09 (3.77)***	−0.03 (0.01)*	—	—	—
General teaching resources	—	—	−0.04 (0.02)	−0.01 (0.03)	−0.00 (0.02)
Student/teacher ratio	—	—	0.01 (0.00)***	0.01 (0.00)*	−0.00 (0.00)
Leadership factor
Visibility and Direct Participation	−1.66 (2.18)	0.00 (0.01)	0.02 (0.01)*	0.02 (0.01)	0.00 (0.01)
Instruction Organization	6.16 (2.45)*	−0.03 (0.01)**	−0.01 (0.01)	−0.00 (0.01)	0.01 (0.01)
Internal Environment Organization	−0.60 (2.42)	−0.00 (0.01)	−0.01 (0.01)	−0.00 (0.01)	−0.01 (0.01)
Planning and Personnel	1.33 (2.29)	−0.02 (0.01)	−0.01 (0.01)	0.00 (0.01)	0.01 (0.01)
External Relations	−2.35 (2.08)	0.00 (0.01)	0.04 (0.01)**	0.03 (0.01)*	0.01 (0.01)
Random effect
Level 1 variance, σ²	6048.32	0.36	0.25	0.35	0.11
Level 2 variance, τ²	1733.57	0.02	0.03	0.06	0.01

Note. Coefficients are reported. Standard errors are in parentheses.

Represents a reverse-scored item.

p < .05. **p < .01. ***p < .001.

Table 5.

Hierarchical Linear Modeling Results From the Teachers’ Perspective.

Fixed Effects Coefficient (SE)	Students’ Reading Achievement	Students’ Chinese Learning Efficacy^a	Teachers’ Occupational Stress^a	Teachers’ Job Burnout^a	Chinese Teachers’ Teaching Efficacy
Student predictors
Gender	32.74 (1.19)***	−0.19 (0.01)***	—	—	—
Family structure	−3.29 (2.07)	0.04 (0.02)*	—	—	—
Socioeconomic status	14.00 (0.96)***	−0.08 (0.01)***	—	—	—
Teacher predictors
Gender	—	—	0.02 (0.01)	0.09 (0.02)***	−0.07 (0.02)***
Age	—	—	0.01 (0.00)***	0.00 (0.00)	−0.00 (0.00)
Teaching experience	—	—	−0.00 (0.00)	0.00 (0.00)	0.00 (0.00)
School predictors
School size	0.01 (0.00)**	−0.00 (0.00)*	−0.00 (0.00)	−0.00 (0.00)	0.00 (0.00)
School Location 1: city-rural	52.18 (5.74)***	−0.14 (0.02)***	−0.01 (0.03)	0.10 (0.03)**	0.05 (0.02)*
School Location 2: county-rural	29.40 (6.37)***	−0.09 (0.02)***	0.02 (0.03)	0.07 (0.04)	0.05 (0.02)*
% girls in the school	0.97 (0.33)**	−0.00 (0.00)*	—	—	—
Chinese teachers’ sufficiency	1.05 (2.53)	−0.01 (0.01)	—	—	—
Chinese teaching resources	12.41 (3.62)**	−0.02 (0.01)	—	—	—
General teaching resources	—	—	−0.02 (0.02)	0.02 (0.03)	−0.02 (0.02)
Student/teacher ratio	—	—	0.01 (0.00)***	0.01 (0.00)*	−0.00 (0.00)
Leadership factor
Organization and Management	4.10 (4.08)	−0.02 (0.02)	−0.03 (0.02)	−0.06 (0.03)*	−0.01 (0.02)
Instruction and Curriculum	24.61 (3.98)***	−0.10 (0.01)***	−0.03 (0.02)	−0.07 (0.03)**	0.11 (0.02)***
Visibility and Direct Participation	2.43 (3.90)	−0.02 (0.01)	−0.10 (0.02)***	−0.16 (0.03)***	0.06 (0.02)**
Random effect
Level 1 variance, σ²	6048.33	0.36	0.25	0.35	0.11
Level 2 variance, τ²	1607.84	0.02	0.03	0.05	0.01

Note. Coefficients are reported. Standard errors are in parentheses.

Represents a reverse-scored item.

p < .05. **p < .01. ***p < .001.

Discussion

To explore the nature of the effectiveness of principal leadership in China, we surveyed principals and the teachers at their schools using the same items adopted from the broader principals’ leadership framework in Grissom and Loeb’s (2011) study, using a national school sample of Grade 8 students. We used both EFA and CFA to determine the latent structure of principals’ leadership effectiveness from both the principals’ and the teachers’ perspectives, and we analyzed their differences and relationships. We used the students’ Chinese reading achievement scores and Chinese learning efficacy, the teachers’ occupational stress and job burnout, and Chinese teachers’ teaching efficacy as the predictors of school outcomes to detect what aspects of principal leadership are most highly correlated with school outcomes from both principals’ and their teachers’ perspectives. Here, we will discuss the major findings and implications.

Comparison of the Leadership Structures in China and the United States

First, we distinguished five leadership skill categories from the principals’ perspective, suggesting five areas or aspects through which principals can exert influence: Instruction Organization, Internal Environment Organization, Planning and Personnel, Visibility and Direct Participation, and External Relations. These five factors are similar to the five factors identified by Grissom and Loeb (2011) among principals in the United States: Instruction Management, Internal Relations, Organization Management, Administration, and External Relations. However, there are differences between the two countries that emerged early in the process of developing the survey items. When we translated the U.S. principals’ tasks into Chinese and interviewed our principals, we clearly realized that we would have to abandon some tasks and add others, even within the same framework, to make it indigenous. For example, the task “fulfilling compliance requirements and paperwork,” which was used in the United States, seemed to not be correlated with principal leadership for most of the principals we interviewed in China. Their response was, “This kind of thing does not reflect our functions at school at all.” In addition, the task “maintaining campus facilities” appeared strange to the Chinese principals, and we had to change the task “managing student discipline” to “establishing campus order” among other changes. Furthermore, we added tasks such as “maintaining the initiative in school management” and “formulating long-term school plans and guaranteeing their implementation” based on the Chinese principals’ opinions. The data analysis from the pilot study on this subject further confirmed this trend.

Clearly, there are vast cultural differences between the Chinese and U.S. schooling systems, and in disparate cultural contexts, school principals will have correspondingly diverse roles and functions. After careful review and intensive rewording, we used 36 revised items reflecting different day-to-day tasks of Chinese principal in the final survey. Comparing the leadership structure between China and the United States, we first found that the United States’ Administration domain “disappeared,” not only because some of the original tasks belonging to it had to be deleted in the process of indigenization but also because both the new tasks and the remaining tasks seemed as other leadership skills in Chinese context. These changes suggest that Chinese principals establish their effectiveness much less through routine administrative tasks than do U.S. principals.

The changes show that except for External Relations, for which the tasks retained the same content and affiliations, the other four skill categories changed their original connotations. The most obvious difference is that organization skills no longer seem to be independent of Instruction or Internal Relations management, which suggests that for Chinese principals, organization skills are something of a basis for their effectiveness. Some tasks, such as “classroom observation” and “informally coaching teachers,” which fell under the American Instruction Management skills, belonged to the Visibility and Direct Participation skills in China, whereas the tasks in the current Chinese Instruction Organization skills appeared to reflect more integrated organization skills that focus on the core school functions—teaching, learning, and curriculum. Although the principals themselves are not experts in all of the subjects taught at school, particularly in middle schools compared with primary schools, they determine the effectiveness of the instruction and curricula based on their higher level organizational skills: formulating criteria, monitoring, evaluating, and guiding.

Moreover, the original United States’ Organization Management skills tasks respectively belonged to Internal Environment Organization, Instruction Organization, and Planning and Personnel skills in China. Each appears to stress a different area: The first is focused on maintaining a positive internal atmosphere or environment; the second focuses on teaching and learning; and the third concerns long-term development. Ultimately, Chinese principal leadership effectiveness follows a trend similar to that of their U.S. counterparts, but our results suggest that Chinese principals exhibit more integrated organizational skills, both overall and in certain core areas.

From the teachers’ perspective, we distinguished three latent principal leadership skill categories—Organization and Management, Instruction and Curriculum, and Visibility and Direct Participation—and these closely resemble the three areas in which assistant principals in the United States distinguished their principals’ effectiveness: Organization Management, Instruction Management, and Internal Relations. These findings indicate that teachers in China, similar to assistant principals in the United States, did not distinguish Organization and Management from more routine administrative tasks, and they did not identify a separate External Relations dimension from Organization and Management, perhaps because they do not commonly observe principals performing these tasks.

In fact, when we attempted to conduct an EFA for the teachers’ rating data similar to that used for the principals’ self-rating data, we found that although three factors were identified, the initial eigenvalue of the first factor was much higher than those of the other two (20.213 > 1.699 > 1.133). This finding suggests that teachers might regard the leadership effectiveness of their principals as one general skill. It is true that both Chinese teachers and U.S. assistant principals on average rated their principals’ leadership effectiveness lower and with more variation than did the principals themselves, although the patterns across the items are similar. This trend has already been observed in many other studies (e.g., Blank, 1987; Hallinger, 2003, 2010).

However, unlike Grissom and Loeb’s (2011) results in U.S. schools, which showed low correlations between principals’ and assistant principals’ ratings, our data showed that the three teachers’ principal leadership effectiveness factor ratings are highly correlated with the principals’ factor ratings related to organization skills: Instruction Organization, Internal Environment Organization, and Planning and Personnel. This result is likely due to the differences between the two countries regarding the principals’ perspective.

What Leadership Factors Are Associated With School Outcomes in China?

We chose multiple outcome measures to examine leadership effectiveness, both student outcomes (students’ reading achievement and Chinese learning efficacy) and teacher outcomes (teachers’ occupational stress and job burnout and Chinese teachers’ teaching efficacy), from principals’ and their teachers’ perspectives. Different school outcomes showed different relationships with the principal leadership factors.

According to both principals’ and teachers’ perspectives, our data show that, of the leadership factors, Instruction Organization is most highly correlated with student outcomes. There are 8 items in the Instruction Organization factor from the principals’ perspective and 12 items in the Instruction and Curriculum factor from the teachers’ perspective, with 6 overlapping items. Among the others, three items from the teachers’ perspective are in the Planning and Personnel factor and the other three are in the Visibility and Direct Participation factor from the principals’ perspective. These findings suggest that principals’ organization skills, particularly regarding instruction and the curriculum, most reflect the influence of principals’ leadership effectiveness on students.

For the teacher outcomes, the results are somewhat more complicated. The principals’ leadership ratings for Visibility and Direct Participation and External Relations are negatively and significantly correlated with teachers’ occupational stress and job burnout, which may suggest a predictive relationship but not a true causal link. Teachers’ perceptions of their principals’ effectiveness in Visibility and Direct Participation are positively and significantly correlated with their occupational stress; three of their perceived principal effectiveness factors are all positively and significantly correlated with their job burnout; and Instruction and Curriculum and Visibility and Direct Participation are positively and significantly correlated with Chinese teachers’ teaching efficacy. These findings suggest that principals’ leadership tasks or practices have a greater effect on teacher outcomes than on student outcomes, which also supports the alternative view of the “indirect model” of principals’ leadership effects on students. Teachers’ perceptions are sensitive to their principals’ leadership effectiveness, which not only affects their own instruction and curriculum practices but, combined with the principals’ leadership effectiveness in instruction organization, also affects students.

Conclusion and Future Research

The aim of the present study is to develop a broader framework for principals’ leadership in China and to detect its relationships with certain school outcomes from both principals’ and their teachers’ perspectives. Here, we summarize our conclusions.

In relation to the first research question regarding the dimensions of Chinese principal leadership, five leadership factors are obtained from the principals’ perspective that particularly correlate with principals’ specific tasks. However, the teachers’ perspective is more extensive and obscure, resulting in three leadership factors. To explore the relationship between these two perspectives, the data show that three of the principals’ rating factors (most reflecting the principals’ organization skills) have a somewhat positive and significant correlation with the teachers’ rating factors—that is, Instruction Organization, Internal Environment Organization, and Planning and Personnel.

The second research question concerns the comparison between principals’ and teachers’ ratings of leadership effectiveness. First, the principals’ self-rating means on all 35 items are higher than the teachers’ ratings on most tasks, except for the four External Relations items, which were not precisely perceived by the teachers. Second, the principals’ self-rating standard deviations on all 35 items are lower than the teachers’ ratings, except for the one item—“hiring staff.” Third, all of the 35 teacher rating items have a positive and significant correlation with their students’ Chinese reading achievement; however, for the principal self-ratings, fewer than half of the items have positive and significant correlations with students’ Chinese reading achievement—most are in the Instruction Organization and Planning and Personnel factors.

The conclusions regarding the third research question emphasize the aspects of principal leadership that are most highly correlated with school outcomes. For student outcomes, Instruction Organization is the most important factor in predicting students’ reading achievement and Chinese learning efficacy. Relative to the student outcomes, principals’ leadership tasks or practices have a stronger association with teacher outcomes, that is, teachers’ occupational stress, teachers’ job burnout, and Chinese teachers’ teaching efficacy.

In this study, we can answer only questions concerning the relationship (not the causal effects) between principal leadership and school outcomes due to limitations of the research design. Based on the logic of causal inference, a randomized controlled experiment is the most powerful design for estimating causal effects when correctly implemented. However, in some educational research contexts, either because of costs or for ethical reasons, researchers may need to approximate randomized experiments with observational data (Schneider, Carnoy, Kilpatrick, Schmidt, & Shavelson, 2007).

In the present study, without longitudinal data, we could not further identify any causal links between principal leadership and school outcomes. Furthermore, we did not include student achievement in other subjects, as a student’s reading achievement in his or her native language typically has a different relationship with school education than do other subjects such as mathematics or science. In addition, we did not include more nonacademic student outcome variables. Due to the essential deficiencies in the research design, we could not more deeply investigate how principals’ integrated organization skills and effectiveness affected the teachers and hence the students.

With a further focus on integrated organization skills, particularly instruction organization, based on longitudinal proof or an intervention program, we could establish the effects of principals’ integrated organization skills. This is the next step and direction in constructing a theory of Chinese principals’ leadership and providing a practical policy analysis.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Notes

Author Biographies

Qiao Zheng is a doctoral candidate at the Collaborative Innovation Center of Assessment Toward Basic Education Quality of Beijing Normal University. Her research interests include school leadership, school self-diagnosis, and school improvement.

Lingyan Li is an associate professor at the Collaborative Innovation Center of Assessment Toward Basic Education Quality of Beijing Normal University and assistant director of National Assessment of Education Quality. Her research interests include school leadership, school self-diagnosis, and policy analysis.

Huijuan Chen is a doctoral candidate at the Collaborative Innovation Center of Assessment Toward Basic Education Quality of Beijing Normal University. Her research interests include school self-diagnosis and school improvement.

Susanna Loeb is the Barnett Family Professor of Education at Stanford University, faculty director of the Center for Education Policy Analysis, and a codirector of Policy Analysis for California Education. Her research addresses teacher policy, school leadership, and school finance.

References

Andrews

R. L.

Soder

(1987). Principal leadership and student achievement. Educational Leadership, 44(6), 9-11.

Bentler

P. M.

(1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238-246.

Blank

R. K.

(1987). The role of principal as leader: Analysis of variation in leadership of urban high schools. Journal of Educational Research, 81(2), 69-80.

Bossert

S. T.

Dwyer

D. C.

Rowan

Lee

G. V.

(1982). The instructional management role of the principal. Educational Administration Quarterly, 18, 34-64.

Brauckmann

Pashiardis

(2011). A validation study of the leadership styles of a holistic leadership theoretical framework. International Journal of Educational Management, 25, 11-32.

Brazer

S. D.

Bauer

S. C.

(2013). Preparing instructional leaders: A model. Educational Administration Quarterly, 49, 645-684.

Brouwers

Tomic

(2000). A longitudinal study of teacher burnout and perceived self-efficacy in classroom management. Teaching and Teacher Education, 16, 239-253.

Bruggencate

G. T.

Luyten

Scheerens

Sleegers

(2012). Modeling the influence of school leaders on student achievement: How can school leaders make a difference? Educational Administration Quarterly, 48, 699-732.

Davis

Wilson

S. M.

(2000). Principals’ efforts to empower teachers: Effects on teacher motivation and job satisfaction and stress. The Clearing House, 73, 349-353.

10.

Edmonds

(1979). Effective schools for the urban poor. Educational Leadership, 37(1), 15-24.

11.

Fabrigar

L. R.

Wegener

D. T.

MacCallum

R. C.

Strahan

E. J.

(1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272-299.

12.

Glasman

N. S.

(1984). Student achievement and the school principal. Educational Evaluation and Policy Analysis, 6, 283-296.

13.

Grissom

J. A.

Loeb

(2011). Triangulating principal effectiveness: How perspectives of parents, teachers, and assistant principals identify the central importance of managerial skills. American Educational Research Journal, 48, 1091-1123.

14.

Grissom

J. A.

Loeb

Master

(2013). Effective instructional time use for school leaders: Longitudinal evidence from observations of principals. Educational Researcher, 42, 433-444.

15.

Hallinger

(2003). Leading educational change: Reflections on the practice of instructional and transformational leadership. Cambridge Journal of Education, 33, 329-352.

16.

Hallinger

(2005). Instructional leadership and the school principal: A passing fancy that refuses to fade away. Leadership and Policy in Schools, 4, 221-239.

17.

Hallinger

(2010). A review of three decades of doctoral studies using the Principal Instructional Management Rating Scale: A lens on methodological progress in educational leadership. Educational Administration Quarterly, 47, 271-306.

18.

Hallinger

Bickman

Davis

(1996). School context, principal leadership, and student reading achievement. Elementary School Journal, 96, 527-549.

19.

Hallinger

Heck

R. H.

(1996). Reassessing the principal’s role in school effectiveness: A review of empirical research, 1980-1995. Educational Administration Quarterly, 32, 5-44.

20.

Hallinger

Heck

R. H.

(1998). Exploring the principal’s contribution to school effectiveness: 1980-1995. School Effectiveness and School Improvement, 9, 157-191.

21.

Hallinger

Murphy

(1985). Assessing the instructional management behavior of principals. Elementary School Journal, 86, 217-247.

22.

Hallinger

Wang

W. C.

Chen

C. W.

(2013). Assessing the measurement properties of the Principal Instructional Management Rating Scale: A meta-analysis of reliability studies. Educational Administration Quarterly, 49, 272-309.

23.

Heck

R. H.

(1992). Principals’ instructional leadership and school performance: Implications for policy development. Educational Evaluation and Policy Analysis, 14, 21-34.

24.

Heck

R. H.

Larsen

T. J.

Marcoulides

G. A.

(1990). Instructional leadership and school achievement: Validation of a causal model. Educational Administration Quarterly, 26, 94-125.

25.

Horng

E. L.

Klasik

Loeb

(2010). Principal’s time use and school effectiveness. American Journal of Education, 116, 491-523.

26.

Lee

V. E.

Smith

J. B.

(1997). High school size: Which works best and for whom? Educational Evaluation and Policy Analysis, 19, 205-227.

27.

Leithwood

(1994). Leadership for school restructuring. Educational Administration Quarterly, 30, 498-518.

28.

Leithwood

Harris

Hopkins

(2008). Seven strong claims about successful school leadership. School Leadership & Management, 28, 27-42.

29.

Leithwood

Jantzi

(2000). The effects of transformational leadership on organizational conditions and student engagement with school. Journal of Educational Administration, 38, 112-129.

30.

Leithwood

Jantzi

(2006). Transformational school leadership for large-scale reform: Effects on students, teachers, and their classroom practices. School Effectiveness and School Improvement, 17, 201-227.

31.

Leithwood

K. A.

Levin

(2005). Assessing school leader and leadership programme effects on pupil learning (Report No. RR662). London, England: Department for Education and Skills.

32.

Leithwood

K. A.

Montgomery

D. J.

(1982). The role of the elementary school principal in program improvement. Review of Educational Research, 52, 309-339.

33.

L. Y.

Sun

Chen

H. J.

Zheng

Zhang

D. H

. (2016). Does instructional leadership matter in China: A propensity score approach. Manuscript submitted for publication.

34.

Louis

K. S.

Dretzke

Wahlstrom

(2010). How does leadership affect student achievement? Results from a national US survey. School Effectiveness and School Improvement, 21, 315-336.

35.

Marks

H. M.

Printy

S. M.

(2003). Principal leadership and school performance: An integration of transformational and instructional leadership. Educational Administration Quarterly, 39, 370-397.

36.

May

Huff

Goldring

(2012). A longitudinal study of principals’ activities and student performance. School Effectiveness and School Improvement, 23, 417-439.

37.

Mazur

P. J.

Lynch

M. D.

(1989). Differential impact of administrative, organizational, and personality factors on teacher burnout. Teaching and Teacher Education, 5, 337-353.

38.

Miles

K. H.

Darling-Hammond

(1998). Rethinking the allocation of teaching resources: Some lessons from high-performing schools. Educational Evaluation and Policy Analysis, 20, 9-29.

39.

Mulford

Silins

(2003). Leadership for organisational learning and improved student outcomes: What do we know? Cambridge Journal of Education, 33, 175-195.

40.

Murphy

(1988). Methodological, measurement, and conceptual problems in the study of instructional leadership. Educational Evaluation and Policy Analysis, 10, 117-139.

41.

Neumerski

C. M.

(2013). Rethinking instructional leadership, a review: What do we know about principal, teacher, and coach instructional leadership, and where should we go from here? Educational Administration Quarterly, 49, 310-347.

42.

Organisation for Economic Co-operation and Development. (2010). TALIS 2008 technical report. Paris, France: Author.

43.

Organisation for Economic Co-operation and Development. (2012). PISA 2009 technical report. Paris, France: Author.

44.

Orlitzky

Benjamin

J. D.

(2003). The effects of sex composition on small-group performance in a business school case competition. Academy of Management Learning & Education, 2, 128-138.

45.

Orphanos

Orr

M. T.

(2014). Learning leadership matters: The influence of innovative school leadership preparation on teachers’ experiences and outcomes. Educational Management Administration & Leadership, 42, 680-700.

46.

Pashiardis

(2001). Secondary principals in Cyprus: The views of the principal versus the views of the teachers-a case study. International Studies in Educational Administration, 29, 11-27.

47.

Pashiardis

(2014). (Ed.). Modeling school leadership across Europe: In search of new frontiers. New York, NY: Springer.

48.

Pashiardis

Costa

Mendes

Ventura

(2005). The perceptions of the principal versus the perceptions of the teachers: A case study from Portugal. International Journal of Educational Management, 19, 587-604.

49.

Pounder

D. G.

Ogawa

R. T.

Adams

E. A.

(1995). Leadership as an organization-wide phenomena: Its impact on school performance. Educational Administration Quarterly, 31, 564-588.

50.

Robinson

V. M.

(2007). School leadership and student outcomes: Identifying what works and why. ACEL Monograph Series, 41, 5-28.

51.

Robinson

V. M.

Lloyd

C. A.

Rowe

K. J.

(2008). The impact of leadership on student outcomes: An analysis of the differential effects of leadership types. Educational Administration Quarterly, 44, 635-674.

52.

Rowan

Bossert

S. T.

Dwyer

D. C.

(1983). Research on effective schools: A cautionary note. Educational Researcher, 12(4), 24-31.

53.

Rutter

Maughan

Mortimore

Ouston

Smith

(1979). Fifteen thousand hours: Secondary schools and their effects on children. Cambridge, MA: Harvard University Press.

54.

Savvides

Pashiardis

(2016). An exploration of relationships between leadership and student citizenship outcomes in Cyprus middle schools. Educational Administration Quarterly, 52, 497-526.

55.

Schaufeli

W. B.

Leiter

M. P.

Maslach

(2009). Burnout: 35 Years of research and practice. Career Development International, 14, 204-220.

56.

Schneider

Carnoy

Kilpatrick

Schmidt

W. H.

Shavelson

R. J.

(2007). Estimating causal effects: Using experimental and observational designs. Washington, DC: American Educational Research Association.

57.

Steenbergen

M. R.

Jones

B. S.

(2002). Modeling multilevel data structures. American Journal of Political Science, 46, 218-237.

58.

Steiger

J. H.

(1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 173-180.

59.

Stewart

(2006). Transformational leadership: An evolving concept examined through the works of Burns, Bass, Avolio, and Leithwood. Canadian Journal of Educational Administration and Policy, 54, 1-29.

60.

Supovitz

Sirinides

May

. (2009). How principals and peers influence teaching and learning. Educational Administration Quarterly, 46, 31-56.

61.

Thoonen

E. E.

Sleegers

P. J.

Oort

F. J.

Peetsma

T. T.

Geijsel

F. P.

(2011). How to improve teaching practices the role of teacher motivation, organizational factors, and leadership practices. Educational Administration Quarterly, 47, 496-536.

62.

Tucker

L. R.

Lewis

(1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1-10.

63.

U.S. Department of Education, National Center for Education Statistics. (1996). Location, poverty and schools. Washington, DC: Government Printing Office.

64.

Wahlstrom

K. L.

Louis

K. S.

(2008). How teachers experience principal leadership: The roles of professional community, trust, efficacy, and shared responsibility. Educational Administration Quarterly, 44, 458-495.

65.

Waters

J. T.

Marzano

R. J.

McNulty

B. A.

(2003). Balanced leadership: What 30 years of research tells us about the effect of leadership on student achievement. Aurora, CO: Mid-Continent Research for Education and Learning.

66.

Wenglinsky

(1997). How money matters: The effect of school district spending on academic achievement. Sociology of Education, 70, 221-237.

67.

Williams

M. M.

(2015). Effects of collegial principal leadership and trust on collaboration and teacher role stress (Unpublished doctoral dissertation). University of Alabama, Tuscaloosa. Retrieved from http://acumen.lib.ua.edu/content/u0015/0000001/0001861/u0015_0000001_0001861.pdf

68.

Ylimaki

R. M.

(2012). Curriculum leadership in a conservative era. Educational Administration Quarterly, 48, 304-346.

69.

Zimmerman

B. J.

Bandura

Martinez-Pons

(1992). Self-motivation for academic attainment: The role of self-efficacy beliefs and personal goal setting. American Educational Research Journal, 29, 663-676.