Principal Perceptions and Student Achievement in Reading in Korea,Mexico,and the United States

Abstract

This study compared PISA 2009 student reading literacy scores with principal perceptions across three countries with varying levels of student performance: Korea, Mexico, and the United States. Seventy-five countries participated in PISA 2009, which measured 15-year-old children’s reading achievement and principal perceptions. The study explored the relationship of principals’ perceived levels of leadership, school autonomy, and use of test results with student attainment of reading literacy. School variables were treated as covariates when each effect of principal leadership was interpreted. All variables were included in a multilevel model and analyzed simultaneously. The means and standard deviations of outcome variables and the explanatory and control variables for the model of the study were calculated by taking into account sampling weights, as well as plausible values for reading literacy scores. SAS PROC MIXED was used to fit hierarchical linear models for the study. There was a positive relationship between student achievement in reading literacy and testing to improve instruction in all three countries—Korea, Mexico, and the United States—and there was a negative relationship between student achievement and lack of resources. Social, economic, and cultural status showed a positive relationship with reading literacy. To be specific, testing to improve instruction can be beneficial in all three countries when it is defined as using tests to group students for instructional purposes and to identify aspects of instruction or the curriculum that could be improved. Results also indicate that students are likely to achieve better if principals perceive that there are no shortages of personnel and equipment.

Keywords

principals reading achievement leadership international PISA 2009

Student achievement is a national and global concern as reflected in recent large-scale standardized assessments including the results from the Trends in International Mathematics and Science Study (TIMSS), the U.S. National Assessment of Educational Progress (NAEP), and the Organization for Economic Cooperation and Development (OECD) Program for International Student Assessment (PISA). Throughout the multiple cycles of administering these assessments over the past decade, student performance in subject areas being tested has been monitored and reported to the public. Furthermore, there have been many studies exploring important factors attributed to student achievement in the content domains. The findings of such studies can inspire or alert educational stakeholders to change existing school policies or launch new educational initiatives for improving student achievement in schools. However, it is often difficult to find direct connections between principal leadership and student achievement across countries.

Shin and Slater (2010) analyzed TIMSS data and found little consistent evidence for connections between principals’ perceived leadership and student achievement in mathematics across eight countries. This study was not unique in finding difficulty establishing relationships between principal leadership and student achievement. Most large-scale studies that have examined direct connections between leadership and student achievement have not had significant results (Backhoff, Andrade, Bouzas, Santos del Real, & Santibáñez, 2009). It is more likely to find small effects through examination of intervening variables (Hallinger & Heck, 1997). Another promising approach is to look at schools over time and examine changes in student achievement related to leadership (Leithwood & Day, 2008).

The International Successful School Principals Project (ISSPP) has used a methodology to identify schools that have a multi-year record of success and followed up with a qualitative approach to examine what is going on inside the school from the perspectives of different constituencies. This narrative approach is being undertaken on an international level and includes investigation of themes of entrepreneurship, instructional improvement, organizational strategy, and social justice (Slater, 2011).

The ISSPP has now produced more than 100 case studies of successful leadership. A key assumption has been that stability of leadership is necessary for success. Day, Leithwood, and Sammons (2008) examined headteachers by level of experience. More change was reported in schools with headteachers who had more years of experience. The ISSPP studies are beginning to suggest who the successful principals are and what we want them to know and be able to do.

Key areas in principal leadership have been identified by the ISSPP studies as well as other researchers (Day et al., 2008; Leithwood, Harris, & Hopkins, 2008; Pont, Nusche, & Moorman, 2008; Yukl, 1989). Three variables stand out: instructional leadership, school autonomy, and use of test results.

Quantitative studies of instructional leadership have indicated that principals have an indirect effect on student achievement primarily through motivation and guidance of teachers (Hallinger & Heck, 1997; Leithwood & Day, 2008). Marzano, Waters, and McNulty (2005) completed a meta-analysis of 69 studies reported as articles published in the United States or dissertations at U.S. universities. The studies involved 2,802 schools. They examined studies that used teacher ratings of principal leadership compared with student achievement and found a correlation of .25.

Pont et al. (2008) conducted a study of leadership in 21 countries that were members of the OECD. Policy makers in each country completed an extensive background report on the status of educational leadership. They identified several policy levers. First, school leadership should be redefined to grant higher levels of autonomy with support to improve student learning. This effort requires teamwork, professional development, teacher evaluation, goal setting, assessment, strategic financial and human resource management, and collaboration with other schools. Second, school leadership should be distributed across the organization. Spillane (2006) has developed an extensive line of research to examine informal leadership roles of educators throughout the school.

Ouchi and Segal’s (2003) work supported Pont et al. (2008) to suggest that schools with more budget autonomy had higher levels of student achievement. Areas of school financial autonomy under principal leadership may include selecting teachers, dismissing teachers, establishing starting salaries, determining salary increases, formulating a school budget, and deciding on budget allocations within the school. Autonomy in teaching and learning, on the other hand, may consist of establishing student assessment policies, approving students for admission, choosing textbooks, determining course content, and deciding what courses are offered. However, the effects of principal leadership and school autonomy do not seem to remain the same from country to country.

Backhoff et al. (2009) analyzed Mexican data from the Teaching and Learning International Study (TALIS; 2008) and the Evaluacion Nacional de Logro Academico en Centros Escolares (ENLACE). The ENLACE is a national student achievement test in Mexico that has been administered annually since 2005. They did not find any variable that related achievement with teacher professional development, principal leadership, school autonomy, or material and human resources. Thus, more research needs to be done to examine how these school characteristics affect student achievement in schools from different global regions where the levels of student performance on international assessments are quite different and educational policies and school internal and external culture vary considerably.

The use of test results to improve instruction has received extensive attention in the United States since the enactment of the No Child Left Behind Act (NCLB; 2001). Schools, local educational agencies, and states have been held accountable for improving the academic achievement of all students, and state-mandated academic assessments have been used as a major component of accountability systems across states (Colorado Department of Education, 2009; Delaware Department of Education, 2010; Linn, 2006; Texas Educational Agency, 2010). Consequently, the use of data is expected to lead to higher student achievement (Schmoker, 1999).

PISA in Korea, Mexico, and the United States

The PISA offered an opportunity to explore the variables of instructional leadership, school autonomy, and use of test results in relation to reading achievement. The PISA study focuses on students who, at the time of application, were in the age range from 15 years 3 months to 16 years 2 months (OECD, 2009b). They must also have been enrolled in an educational institution since the seventh grade (UNESCO Institute for Statistics, 2012).

Three countries were chosen for study that varied by level of achievement, history, culture, level of wealth, and recent policy reforms. The countries under study included Korea, which was the highest performing OECD country in reading literacy; the United States, which scored in the middle; and Mexico, which scored in the lowest level (OECD, 2010). These countries were selected to provide a snapshot of differences and similarities in the effects of background characteristics in and out of school over the continuum of the international performance distribution in reading literacy. The three countries should not be considered as a random sample of all possible participating countries. In the past few decades, Korea has developed a strong economy and is recognized in a variety of technical fields. It has employed relatively traditional management techniques in public schools. The United States has a more mature economy that experienced signs of weakness that could be related to lack of educational attainment of the workforce. Mexico has struggled to expand its economy as a developing country and is weathering the recent recession.

The structure and purpose of the educational systems in the three countries are similar. They consist of three levels: basic elementary education, middle and secondary education, and higher education. PISA tests 15-year-old students who attend 9th and 10th grades in each country. In the United States, these grades are in high school, whereas they are in the third year of middle school and the first year of high school in Korea. In Mexico, 15-year-old students are split between the third year of the escuela secundaria (9th grade) and the first year of el bachillerato (10th grade).

Each country varies in history, culture, and economic status. However, they share a common purpose to prepare students to pursue higher education or to enter the workforce. Education is provided free in government schools and is also carried out in private institutions. In the following section, we briefly discuss background factors in each country.

Korea

Korea had 3,341,168 elementary age children and 4,107,194 secondary age children in 2009 (UNESCO Institute for Statistics, 2012). The literacy rate is close to 100%. Koreans have traditionally attached great importance to education, and this attitude continues to be expressed in a highly competitive educational system.

The economy of Korea has grown to become one of the largest in the world. Until relatively recently, it was a traditional society with centralized control of political power. Schools were managed hierarchically with principals appointed by the central government. Reforms have included school management committees adopted in 1998 (Lo & Gu, 2008).

The Korean Teachers and Education Workers Union was established in 1989. It is a national union with a mission to increase teacher power in schools. It is strongly opposed to appointment of principals by the central government (Lo & Gu, 2008). The principal had played a highly autocratic role in running the school, but the teachers union and the school management committees now require negotiation and collaboration.

In Korea, there have been increasing calls for talented and capable teachers who can teach as many students as possible to high educational standards and contribute to closing the achievement gap between high performing and low performing students (Lee, 2011). Since the government launched a teacher compensation policy in 2001, teachers have been awarded monetary incentives corresponding to their performance. Not only teachers but schools are also evaluated on their performance and awarded collective bonuses at the school level (Korean Ministry of Education, Science, and Technology, 2010). Part of the index measuring school effectiveness is the school average of student performance in a government-mandated exam.

Mexico

In Mexico, between 1970 and 2000, total enrollment went from 11.5 to almost 29.5 million, the average years of schooling increased from 3.4 grades to 7.6. For the year 2010, the country’s total enrollment rose to 33.2 million and the average schooling rose to 8.6 (Partida, 1999).

Two major changes in the Mexican educational system have taken place in recent years. First, in 1992 Mexico decentralized the educational system to extend coverage of educational services and improve quality and equity, particularly in elementary education. The decentralization of public education was a response to excessive centralization and the need for local authorities to act with more autonomy.

Second, Mexico has paid a lot of attention to assessment. In 2002, the National Institute for Educational Evaluation was created to provide indicators for achievement and student information from preschool to high school levels. In 2005, the Ministry of Education implemented an annual census and testing system of elementary school students. The results are then used to pay a monetary bonus to teachers according to their students’ performance. Finally, since 2000 Mexico has increased its participation on international assessment surveys, such as PISA and the TALIS, putting a lot of attention on country and state results.

School principals are teachers who have obtained the scores needed from various factors, such as knowledge, discipline, punctuality, and experience. However, the tasks of principals are very different from those of teachers. Before 2001, when the first national course for principals was created, their managerial skills had been taken for granted on the basis of the training they received as teachers.

The National Program for Education included a proposed program to prepare school principals. This program included knowledge of philosophical and legal bases; analysis of the school conditions, the influence of principals on educational outcomes; the influence of management in the role of schools; and the acquisition of knowledge and skills for the exercise of leadership. One of the goals for 2006 was for all principals of basic education to take accredited courses to upgrade their skills as managers. Unfortunately, very few principals have yet been able to achieve this type of preparation (García Garduño, Slater, & López-Gorosave, 2011).

United States

Enrollment in U.S. Schools has been increasing since 1985. The total enrollment in kindergarten through Grade 12 was 54.0 million in 2008, including 5.7 million in private schools. The figures include 34.3 million at the K-8 level and 15.0 million at the Grade 9-12 level (U.S. Census Bureau, 2012). The number of African American, Asian, and Hispanic students has been steadily increasing so that California, the most diverse state, is now a “minority majority” state and will soon be followed by the country as a whole.

The high school completion rate in 2010 was 87% (U.S. Census Bureau, 2012), but the rate varies by race and income level. The achievement gap between White students and minority students has been a policy concern at least since the beginning of the Effective Schools Movement. Most recently, the NCLB Act (2001) has required schools to test students at all grade levels and report results based on a variety of categories including race, income, and special education status. The results are then used to rank schools according to performance.

The use of data is at the heart of educational accountability process in many nations (Teddlie & Reynolds, 2000). In the United States, the NCLB Act of 2001 requires states to set academic standards for elementary and secondary school students in their state and to develop high-quality academic assessments and accountability systems, which are aligned with the challenging state academic standards (U.S. Department of Education, 2002).

Teachers are required to have a bachelor’s degree and certification, and there are also certification programs for principals that are typically run by universities and often include a master’s degree. School principals often have a master’s degree in educational administration and certification from the university.

Purpose

The current study compared PISA 2009 student reading literacy scores across three countries with varying levels of student performance: Korea, Mexico, and the United States. Specifically, the study was aimed at exploring the relationship of principals’ perceived levels of leadership, school autonomy, and use of test results with student attainment of reading literacy.

PISA conceptualizes three types of leadership. The first is instructional leadership (LEADINS), which has to do with the principal’s use of student performance results to develop the school’s educational goals and to help make decisions regarding curriculum development as well as the responsibility for assuring the coordination of curriculum. Hallinger, Bickman, and Davis (1996) focused on practices most closely related to this PISA category. They included observing classes, reviewing test scores with faculty, facilitating teacher collaboration around the instructional program, securing resources, and maintaining visibility.

The second PISA category is leadership of teachers (LEADTCH), which has to do with actions to supervise directly teachers’ instruction and learning outcomes. Hallinger and Heck (1997) and Leithwood and Day (2008) found motivation and guidance of teachers to be a critical variable.

The third is leadership for professional development (LEADPD), which includes making sure that professional development activities of teachers are in accordance with the teaching goals of the school. Principals inform teachers about possibilities for updating their knowledge and skills.

PISA defined two types of autonomy. The first is financial autonomy (AUTOFNCE), in which the principal has considerable responsibility for hiring and firing teachers, determining salaries, and formulating the school budget. Ouchi and Segal’s (2003) work suggested that schools with more budget autonomy had higher levels of student achievement.

The second is autonomy in the area of teaching and learning (AUTOTCH), in which the principal has considerable responsibility for establishing student admission, assessment, and discipline as well as determining course content and textbooks. Pont et al. (2008) called this area a policy lever to produce change.

There are three ways that test results are used. The first is testing for accountability (TSTSCHACC). Achievement data are used in evaluation of the principal’s performance and in decisions about instructional resource allocation to the school. The second is testing for comparison (TSTCOMP), in which principals use test data to make decisions about student retention or promotion, compare the school to district or national performance results, make judgments about teacher effectiveness, and compare the school with other schools.

The third is testing for instruction (TSTINS), which is the use of data to group students for instructional purposes and identify aspects of instruction or curriculum that can be improved. Heck, Larsen, and Marcoulides (1990) emphasized practices that were most like the PISA category of using test results to improve instruction (TSTINS). They mentioned practices such as overseeing school governance, organizing the school around instruction, and developing activities to foster an instructionally oriented school climate.

There were three main questions in the study. First, what was the relationship of principals’ perceived levels of leadership with student attainment of reading literacy? PISA examined leadership in three areas: management, supervision of instruction, and leadership for professional development. Management was defined as job functions associated with leadership in effective schools. Supervision of instruction was taking actions to directly supervise teachers’ instruction and learning outcomes. Principal leadership for professional development addressed three aspects of school organizational capacity: teachers’ knowledge, skills, and dispositions; professional community; and program coherence.

The second question was: what was the relationship of principals’ perception of school autonomy with student attainment of reading literacy? School autonomy refers to funding and human resources. Schools may be autonomous to varying degrees regarding these aspects.

The third question was: what was the relationship of principals’ perception of the use of test results with student attainment of reading literacy? The use of test data is a tool for schools to improve instruction and turn the act of data analysis into a process that improves the organization, function, and climate of schools.

Other background characteristics at school and student levels may also have correlations with student achievement. For instance, it is well documented that school or student socioeconomic status (SES) has a negative relationship with student academic achievement when the variable is measured by computing the percentage of students in school who came from economically disadvantaged homes (Shin & Slater, 2010). Another example is a shortage of school resources such as qualified teachers in subject areas, support personnel, laboratory equipment, library materials, instructional materials, computer and Internet access, and audiovisual resources. There is considerable debate about whether additional resources improve achievement in developed countries, but in developing countries, UNESCO (2000) provided evidence that additional resources could improve student achievement for schools that fell below certain minimal levels of expenditure. The lack of school resources is known to be negatively related to student performance in literature in Korea (Ban & Shin, 2011) and in Mexico (Backhoff, Bouzas, Contreas, Hernández, & García, 2007). The community in which a school is located can also make a difference in student performance.

Background characteristics at the school and student levels were used as control variables and associated with reading literacy performance. The variables of the study and the covariates are defined in Appendix A. The two at the school level were hindrance in school capacity to provide instruction due to the lack of human and material resources (CAPTYHIND) and community size where the school was located (COMMTY). Last, the student variables considered in this study were related to student attendance in test language enhancement lessons out of school time (OUTSCH) and their economic, social, and cultural status (ESCS). Appendix A contains the mapping of research and control variables of the study with relevant questions in the actual PISA 2009 survey questionnaires. Note that PISA does not give the information of the constructs being measured at the item level.

Research Method

Data Sources

In PISA 2009, the assessment focused on measuring 15-year-old children’s ability to apply knowledge and skills in reading and to analyze, reason, and communicate effectively as they pose, interpret, and solve problems in a variety of real-life situations (OECD, 2010). A total of 34 OECD member countries and 41 partner countries and educational jurisdictions participated in the PISA 2009 administration. The international database is accessible on the official PISA website (www.pisa.oecd.org). PISA 2009 provided not only student scores in subject areas but also supplied contextual information surrounding students and schools, respectively, through the additional collection of student and school principal responses to survey questionnaires. Parent questionnaires were also administered to the parents of the tested students in some countries (OECD, 2009b). For this study, the student scores in reading literacy along with student- and school-level survey responses from Korea, Mexico, and the United States were analyzed simultaneously.

Data Analysis

PISA 2009 was administered to a sample drawn from the population of 15-year-old children in each participating country. The outcomes resulting from the sample were used to estimate characteristics of the whole population in the country. PISA is known to use a two-stage sample design: a sample of schools is selected from a complete list of schools containing the student population of interest and then students are randomly selected within the selected schools. The estimation of population characteristics based on samples typically involves some extent of uncertainty or error. A measure of this uncertainty is a sampling variance. PISA employs a particular replication method for estimating sampling variances, which is named as the Fay’s variant of the Balanced Repeated Replication method (OECD, 2009b). It is important to note that all statistical analyses or procedures concerning the PISA data should be weighted and that unweighted analyses will yield biased estimates of the standard errors for population parameter estimates, thus resulting in an incorrect conclusion of hypothesis testing.

Another unique feature relating to the PISA data is that PISA reports student performance through five plausible values in each subject area. That is, the plausible values are random draws from the marginal posterior distribution of the latent trait for each student in reading literacy (OECD, 2009b). Hence, a population characteristic should be estimated in such a way that each plausible value is used separately first, and then the final estimate of the population characteristic is the average of the statistics over five plausible values. The standard error of the parameter estimate should also be adjusted accordingly. The detailed adjustment will be described later in this article.

Furthermore, as shown in the sampling strategy, the data structure of PISA 2009 was multilevel with students at the lower level and schools at the upper level for each country. Student scores were not independent within the school. Student and school background characteristics existed at student and school levels, respectively. When a conventional multiple regression is applied to multilevel data, the Type I error rate associated with a significance test is known to be inflated (Bryk & Raudenbush, 1992; Littell, Milliken, Stroup, & Wolfinger, 1996; Mislevy, Beaton, Kaplan, & Sheehan, 1992; Singer, 1998; Verbeke & Molenberghs, 2000). That is, significant results for the regression coefficients are more likely spurious than authentic. For each country, the regression coefficients of student and school predictor variables can be analyzed simultaneously through a two-level hierarchical linear model (HLM), handling related observations properly. Furthermore, since the study aimed to compare three specific countries intentionally selected from roughly low to high reading literacy levels, a variable with three country categories had to be incorporated into the model for the sake of the direct comparison. For this study, the data from Korea, Mexico, and the United States were analyzed through two-level hierarchical modeling simultaneously. The method can be called a multiple group analysis using a two-level HLM. In the HLM, the five plausible values for each student were used as the criterion variable, and student and school characteristics were predictor variables whose associations with the criterion variable were investigated.

The set of the predictors of the model was identical across countries. The student and school background characteristics addressed in the section introducing research questions earlier and the original survey questions, which comprised the variables, are presented in Appendix A.

According to the multilevel model formulation convention, an HLM was formulated without predictors first. Then, the predictors were added to the model at student and school levels simultaneously. In multilevel modeling, the former is called the unconditional model whereas the latter is the conditional model. The amount of variance at each level accounted for by the set of predictors at the level indicates the practical importance of the predictors. The larger the amount, the more associated they are with the outcome variable. The unconditional model for the study was formulated with each plausible value at a time for a student as follows:

Y_{i j} = π_{0 j} + e_{i j},

where Y_ij was a plausible value for student i in school j, and the expected score of the plausible values across students for school j was denoted by π_0j; e_ij indicated a random error associated with each student. The random error was assumed to be normally distributed with mean 0 and variance σ². Subsequently, the Level 2 model was specified as follows:

π_{0 j} = β_{00} + d_{0 j},

where β₀₀ was the expected value of the measures across the three countries and d_0j denoted a random school deviation from the expected value. The deviation was also assumed to be normally distributed with a mean of 0 and variance τ². Then, β₀₀ was further decomposed into three components such as

β_{00} = γ_{001} {KOR}_{j} + γ_{002} {MEX}_{j} + γ_{003} {USA}_{j},

where the dummy variables KOR, MEX, and USA took on a value of either 1 or 0 to reflect the nationality of the school. As a result, the unconditional HLM for the multiple group analysis of the current study was

Y_{i j} = γ_{001} {KOR}_{j} + γ_{002} {MEX}_{j} + γ_{003} {USA}_{j} + {d_{0 j} + e_{i j}} .

The model had three fixed effects along with the two random effects within the curly bracket. The fixed effects stood for the expected values of the measures for three countries. The variances of the random effects were σ² and τ², respectively, and the random effects were independent of each other. The extension of the HLM into the conditional HLM was straightforward. Twelve predictor variables were added to the model. First, the Level 1 model had the following predictors:

Y_{i j} = π_{0 j} + π_{1 j} {OUTSCH}_{i j} + π_{2 j} {ESCS}_{i j} + e_{i j} .

With 10 school background variables resulting from principals’ survey responses, the Level 2 model was specified as follows:

\begin{array}{l} π_{0 j} = β_{00} + β_{01} {LEADINS}_{j} + β_{02} {LEADTCH}_{j} + β_{03} {LEADPD}_{j} + \\ β_{04} {AUTOFNCE}_{j} + β_{05} {AUTOTCH}_{j} + β_{06} {TSTSCHACC}_{j} + \\ β_{07} {TSTCOMP}_{j} + β_{08} {TSTINS}_{j} + β_{09} {CAPTYHIND}_{j} + \\ β_{0 (10)} {COMMTY}_{j} + d_{0 j}, \\ π_{1 j} = β_{10} + d_{1 j}, \\ π_{2 j} = β_{20} + d_{2 j}, \end{array}

where the school background variables were included as predictors for intercept π_0j only. The effects of enrichment lessons (OUTSCH) and economic, social, and cultural status (ESCS) were assumed to vary across schools. Then, each beta was further decomposed into three national indicators as shown below.

\begin{array}{l} β_{00} = γ_{001} {KOR}_{j} + γ_{002} {MEX}_{j} + γ_{003} {USA}_{j} \\ β_{01} = γ_{011} {KOR}_{j} + γ_{012} {MEX}_{j} + γ_{013} {USA}_{j} \\ : : \\ β_{09} = γ_{091} {KOR}_{j} + γ_{092} {MEX}_{j} + γ_{093} {USA}_{j} \\ β_{0 (10)} = γ_{0 (10) 1} {KOR}_{j} + γ_{0 (10) 2} {MEX}_{j} + γ_{0 (10) 3} {USA}_{j} \\ β_{10} = γ_{101} {KOR}_{j} + γ_{102} {MEX}_{j} + γ_{103} {USA}_{j} \\ β_{20} = γ_{201} {KOR}_{j} + γ_{202} {MEX}_{j} + γ_{203} {USA}_{j} \end{array}

The school deviations from the national averages of the intercept and the effects of OUTSCH and ESCS—d_0j, d_1j, and d_2j—were assumed to be normally distributed with a mean of 0 and a variance of τ₀², τ₁², and τ₂² each. The random error e_ij was also assumed to follow a normal distribution with mean 0 and variance σ². All the variance components were assumed to be independent of each other in this model.

As shown in the model formulation, the country did not constitute Level 3 above the school level because the three countries were not randomly drawn for this study. They were specifically selected with the intention for direct comparison. Accordingly, the country variable with three categories was used as part of the covariates to reflect school background characteristics in interaction terms at the school level. Thus, the combined conditional HLM was

\begin{array}{l} Y_{i j} = {γ_{001} {KOR}_{j} + γ_{002} {MEX}_{j} + γ_{003} {USA}_{j} + γ_{011} {(KOR * LEADINS)}_{j} + \\ γ_{012} {(MEX * LEADINS)}_{j} + γ_{013} {(USA * LEADINS)}_{j} + \\ γ_{021} {(KOR * LEADTCH)}_{j} + γ_{022} {(MEX * LEADTCH)}_{j} + γ_{023} {(USA * LEADTCH)}_{j} \\ + γ_{031} {(KOR * LEADPD)}_{j} + \\ γ_{032} {(MEX * LEADPD)}_{j} + γ_{033} {(USA * LEADPD)}_{j} + \\ γ_{041} {(KOR * AUTOFNCE)}_{j} + γ_{042} {(MEX * AUTOFNCE)}_{j} + \\ γ_{043} {(USA * AUTOFNCE)}_{j} + γ_{051} {(KOR * AUTOTCH)}_{j} + \\ γ_{052} {(MEX * AUTOTCH)}_{j} + γ_{053} {(USA * AUTOTCH)}_{j} + \\ γ_{061} {(KOR * TSTSCHACC)}_{j} + γ_{062} {(MEX * TSTSCHACC)}_{j} + \\ γ_{063} {(USA * TSTSCHACC)}_{j} + γ_{071} {(KOR * TSTCOMP)}_{j} + \\ γ_{072} {(MEX * TSTCOMP)}_{j} + γ_{073} {(USA * TSTCOMP)}_{j} + \\ γ_{081} {(KOR * TSTINS)}_{j} + γ_{082} {(MEX * TSTINS)}_{j} + γ_{083} {(USA * TSTINS)}_{j} + \\ γ_{091} {(KOR * CAPTYHIND)}_{j} + γ_{092} {(MEX * CAPTYHIND)}_{j} + \\ γ_{093} {(USA * CAPTYHIND)}_{j} + γ_{0 (10) 1} {(KOR * COMMTY)}_{j} + \\ γ_{0 (10) 2} {(MEX * COMMTY)}_{j} + γ_{0 (10) 3} {(USA * COMMTY)}_{j} + \\ γ_{101} {(KOR * OUTSCH)}_{i j} + γ_{102} {(MEX * OUTSCH)}_{i j} + \\ γ_{103} {(USA * OUTSCH)}_{i j} + γ_{201} {(KOR * ESCS)}_{i j} + γ_{202} {(MEX * ESCS)}_{i j} + \\ γ_{203} {(USA * ESCS)}_{i j}} + {d_{0 j} + d_{1 j} OUTSCH + d_{2 j} {ESCS}_{i j} + e_{i j}}, \end{array}

where the first and second curly brackets contain the fixed and random effects of the conditional HLM, respectively. The components of the total variance were σ² at the student level for τ₀², τ₁², and τ₂² at the school level. The random effects were independent of each other. The regression coefficients, γ₀₀₁, γ₀₀₂, and γ₀₀₃ denoted each country’s expected value of the measures when all the interaction terms of the background information by the country were 0. Hence, the meaning of the intercept has changed in the conditional model accordingly.

PISA suggests that the final estimates of statistics and their standard errors should be computed using five plausible values with the total and replicate samples to avoid biased results. This recommendation was not any exception for HLMs. There were one total weight and 80 replicate sampling weights at the student level in the PISA 2009 data. Therefore, the unconditional and conditional HLMs of the study were analyzed with 81 samples for each of five plausible values, and hence a total of 405 runs were performed for each model. The final estimates of the population parameters of the fixed and random effects of the model alongside their standard errors were obtained following the instruction shown in PISA Data Analysis Manual SAS® Second Edition (2009a). The detailed steps are as follows:

Obtain new total and replicate sample weights for each country. For the rth replicate sample, the new weight is

w_{r i_{n e w}} = (n_{k} * w_{r i}) / \sum_{i = 1}^{n_{k}} w_{0 i}

where n_k represents the number of complete cases for country k and w_0i and w_ri are the total and the rth weights for student i, respectively.

Compute the five estimates of the population parameter of interest (e.g., a regression coefficient), ${\hat{γ}}_{1}$ , ${\hat{γ}}_{2}$ , ${\hat{γ}}_{3}$ , ${\hat{γ}}_{4}$ , and ${\hat{γ}}_{5}$ by running the model five times with each of the five plausible values as the dependent variable in conjunction with the new total weight.

Compute the five sampling variance estimates of the estimated regression coefficients. For instance, the sampling variance estimate for the jth regression coefficient estimate is

σ_{{\hat{γ}}_{j}}^{2} = \frac{1}{G {(1 - d)}^{2}} \sum_{r = 1}^{G} {({\hat{γ}}_{r j} - {\hat{γ}}_{j})}_{i},

where G denotes the number of replicate samples and d indicates the Fay coefficient. In the PISA data, G = 80 and d = 0.5, respectively.

Compute the final estimate of the regression coefficient which is equal to

\hat{γ} = \frac{{\hat{γ}}_{1} + {\hat{γ}}_{2} + {\hat{γ}}_{3} + {\hat{γ}}_{4} + {\hat{γ}}_{5}}{5}

Compute the final sampling variance estimate which is equal to

σ_{\hat{γ}}^{2} = \frac{1}{5} (σ_{{\hat{γ}}_{1}}^{2} + σ_{{\hat{γ}}_{2}}^{2} + σ_{{\hat{γ}}_{3}}^{2} + σ_{{\hat{γ}}_{4}}^{2} + σ_{{\hat{γ}}_{5}}^{2})

Compute the imputation variance related to five plausible values which is equal to

σ_{i m p}^{2} = \frac{1}{4} \sum_{j = 1}^{5} {({\hat{γ}}_{j} - \hat{γ})}^{2}

Compute the final error variance which is equal to

σ_{e r r}^{2} = σ_{\hat{γ}}^{2} + (1.2 σ_{i m p}^{2})

Compute the final standard error for the regression coefficient by taking the square root of the final error variance as

s . e . = \sqrt{σ_{e r r}^{2}}

For a significance test, take the ratio of the final estimate of the parameter of interest to the final standard error, which is assumed to follow a standard normal distribution asymptotically. The ratio values beyond the ±1.96 range are flagged as significant at the .05 level of significance.

Results

The current study only used public schools for data analysis because the vast majority of the schools were public schools in the sampled data for each country: 93% of schools belonged to public schooling in the United States, 87% in Mexico, and 63% in Korea. Mexico had the largest sample size among the countries of interest (1,332 schools and 34,138 students), followed by the United States (154 schools and 4,888 students), whereas Korea showed the smallest sample size (99 schools and 3,091 students). The large sample in Mexico was due to the country’s desire to have information on a regional level; however, the other two countries just looked for information at the national level. Korea had more boys than girls, whereas more girls were found in Mexico’s database. The United States showed an almost equal gender ratio. The detailed information of these demographic characteristics is shown in Table 1.

Table 1.

Sample Size.

	Country
Unit	Korea (%)	Mexico (%)	United States (%)
Schools	99	1,332	154
Students	3,091	34,138	4,888
Females	1,390 (45%)	17,788 (52%)	2,427 (50%)
Males	1,701 (55%)	163,501 (48%)	2,461 (50%)

The means and standard deviations of the outcome variables and the explanatory and control variables for the model of the study are found in Table 2. The statistics were calculated by taking into account sampling weights—as well as plausible values for reading literacy scores—to avoid estimation bias. As seen in this table, Korea showed the highest average score in reading literacy (533) followed by the United States (494) and Mexico (420). These results show that the reading abilities for the three countries are very different from each other. The mean of the scale was set at 500 and the SD at 100 when the PISA literacy scale was established (OECD, 2009b). The United States is behind Korea by about a half SD, and Mexico is more than a half SD lower than the United States and more than one SD lower than Korea in the average literacy score.

Table 2.

Descriptive Statistics of Explanatory and Outcome Variables.

	Country
	Korea		Mexico		United States
Variable	M	SD	M	SD	M	SD
Reading literacy	533	81	420	84	494	95
LEADINS	2.6733	0.5961	2.9865	0.5642	3.4359	0.5311
LEADTCH	3.0573	0.6100	3.4451	0.5611	3.7404	0.5238
LEADPD	2.9296	0.5925	3.3241	0.5093	3.6229	0.4767
AUTOFNCE	0.9129	0.8737	1.3746	1.3133	3.2969	1.0445
AUTOTCH	3.6241	1.6560	1.2960	1.2049	4.2247	1.6184
TSTSCHACC	0.6226	0.7043	0.7201	0.7456	1.3986	0.6924
TSTCOMP	2.2347	1.1785	3.1570	1.0548	3.1165	0.9386
TSTINS	1.6485	0.5793	1.5772	0.5925	1.6816	0.4852
CAPTYHIND	1.8818	0.5139	2.5147	0.6429	1.5613	0.5017
COMMTY	4.1223	0.8693	3.0314	1.3410	2.9985	1.1777
OUTSCH^a	26%	n/a	26%	n/a	10%	n/a
ESCS	−0.16	0.81	−1.38	1.22	0.09	0.90

Percentage of “yes” responses is shown for this dichotomous variable.

Besides the mean, it is important to analyze the dispersion of the student scores; the scores varied more in the United States because it had the biggest SD (95) of the three countries. Mexico appeared to have a slightly larger variability in student scores as compared with Korea (84 and 81, respectively).

In relation to the explanatory variables, Table 2 shows that the United States had the highest result in principals’ perception of their leadership oriented to: instruction (LEADINS), school educational goals (LEADTCH), and professional development (LEADPD). Also, the United States had the highest scores in school financial autonomy (AUTOFNCE) and teaching autonomy (AUTOTCH), as well as in the use of student assessment results for accountability (TSTSCHACC) and instructional improvement (TSTINS), although in the last variable the results from Korea were quite similar. Even though Mexico obtained the highest score in test use for school comparison (TSTCOMP), the results of the United States were also very high.

Finally, in Table 2 it can be seen that the results for the three countries related to context or control variables. Mexico had the highest score for the shortage of qualified teachers and school resources that hinder the school capacity to provide instruction (CAPTYHIND), whereas Korea had the major proportion of its schools located in large cities, compared with Mexico and the United States (COMMTY). Mexico and Korea had a larger percentage (26%) than the United States (10%) of students attending enrichment lessons in the test language (OUTSCH); and Mexican students presented the lowest SES and the United States the highest (ESCS). About 5% or less of the data were missing across countries. The missing data occurred in predictor variables alone. Any cases missing any of the values for predictors were discarded from the analysis.

For checking multicollearity, 81 correlation coefficients were computed for each of 66 pairs of the 12 predictor variables in total using 1 total and 80 replicate sampling weights. The resulting coefficients are presented in Table 3. Overall, the predictors showed little to small correlations across the countries except the three leadership-related variables. The correlations among these variables ranged from .37 to .80. Korea showed highest correlations followed by the United States in these variables. A general rule of cutoffs for strong linear associations between variables is known to be ±.8 to ±.9 (Mason & Perreault, 1991). It is well documented that the presence of strong correlations indicates collinearity and that collinearity may lead to inflated variances of some of the coefficients in the linear model and the structural equation model (Grewal, Cote, & Baumgartner, 2004; Sengupta & Bhimasankaram, 1997). The variance inflation is frequently measured by the variance inflation factor (VIF). If the VIF is 10 or larger, the level of collinearity is believed to be problematic (Belsley, Kuh, & Welsch, 1980). However, few statistical software programs provide the VIF in multilevel regression analysis. The authors could not find any option in SAS PROC MIXED that produced this index. Furthermore, the true effects of the VIF to hypothesis testing should be considered alongside other factors such as sample size, coefficient of determination, and so on (O’Brien, 2007), as opposed to blindly deleting any variable associated with VIF 10 or larger. Given that the primary research questions of the study were about principals’ leadership, all three leadership predictors were kept in the analysis with caution.

Table 3.

Correlation Coefficients for Predictors.

Variable	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)	(11)
(1) LEADINS
(2) LEADTCH	.61
	.37
	.46
(3) LEADPD	.74	.80
	.47	.56
	.65	.63
(4) AUTOFNCE	.37	.36	.29
	.03	.12	.14
	.10	.13	.10
(5) AUTOTCH	.32	.37	.19	.54
	.09	.07	.06	.29
	−.02	−.08	−.02	.39
(6) TSTSCHACC	.11	.06	−.02	.18	.09
	.25	.10	.15	.10	.03
	.18	.08	.15	.07	−.02
(7) TSTCOMP	−.10	−.15	−.06	−.09	−.11	.01
	.19	.07	.18	.01	−.03	.22
	.21	.04	.17	.16	.06	.37
(8) TSTINS	.13	.01	.12	−.06	−.04	.21	.42
	.28	.22	.23	.04	.02	.17	.33
	.17	−.01	.05	.04	.06	.20	.31
(9) CAPTYHIND	−.10	−.26	−.31	.15	.04	.01	−.06	.10
	−.19	−.30	−.25	−.20	−.02	−.09	−.03	−.16
	−.05	−.08	−.02	.10	−.00	.14	.10	.00
(10) COMMTY	−.10	−.03	−.03	.00	.03	−.12	.19	−.06	−.09
	.12	.17	.20	.25	−.03	.12	.05	.04	−.40
	.09	−.03	−.01	.11	−.16	.20	.20	.25	−.00
(11) OUTSCH	.08	−.01	.04	.08	.02	.13	.04	.04	−.01	−.03
	−.01	−.04	−.05	−.09	−.00	−.03	.00	−.01	.10	−.13
	.04	.01	.03	.01	.00	.04	.03	.01	.02	.07
(12) ESCS	−.03	−.04	.01	.01	.01	−.07	.11	.07	−.02	.14	.09
	.08	.12	.16	.16	−.02	.05	.06	.04	−.26	.43	−.12
	−.05	−.02	−.00	−.04	−.05	−.05	−.05	.04	−.16	−.07	−.04

Note. Correlations are presented in the order of KOR, MEX, and U.S. in each cell.

SAS PROC MIXED (Version 9.2) was used to fit HLMs for the study. The model statistics were obtained by running each model 405 times—with the 1 total weight and 80 replicate sample weights for each of the 5 plausible values—and the adjusted standard errors for significance tests were computed following the guidelines as addressed in detail earlier. Missing data were deleted listwise, meaning that complete cases were included in the analysis. The percentage of incomplete cases ranged from 5% (Korea and Mexico) to 3% (the United States).

The estimates of the fixed effects, namely, dummy-coded country indicator variables and the variance components of the unconditional model are shown in Table 4. The expected reading scores (γ₀₀₁, γ₀₀₂, γ₀₀₃) were 419.09 (Mexico), 495.51 (the United States), and 535.78 (Korea), respectively. These scores were similar to the countries’ mean scores found in Table 2. The two independent variance components at the two levels of the model were estimated to capture the variability existing between students (σ²) and between schools (τ²). The between-students variance, for example, reflected the extent to which the students’ scores varied randomly from the school mean score, whereas the between-schools variance expressed the degree to which the schools’ average scores deviated randomly from the nation’s mean score in the PISA 2009 test. It is worth noting that these unbiased estimates resulted from 405 runs of the unconditional model. The proportion of the between-schools variance in the total variance that consisted of the between-students variance plus the between-schools variance is called intraclass correlation (ρ) in the literature (Bryk & Raudenbush, 1992). As the intraclass correlation increases, the difference among schools becomes larger. The intraclass correlation was around .40 on average across the three countries. Overall, students’ scores seemed correlated within schools in a moderate degree. Hence, it appeared that using HLMs was justified. The detailed results of the unconditional model are shown in Table 4.

Table 4.

Parameter Estimates of the Unconditional Model.

	Country
Fixed Effect Coefficient (SE)	Korea	Mexico	United States
Country, γ₀₀₁, γ₀₀₂, γ₀₀₃	535.7827* (171.4558)	419.0886* (171.4549)	495.5094* (171.4554)
Random effect
Intraclass correlation, ρ	.3960*
Level 1 variance, σ²	4078.3183
Level 2 variance, τ²	2673.9080

p < .05.

When the conditional model was fit, the percentage of the between-students variance (σ²) explained by the two Level 1 predictors (OUTSCH and ESCS) was relatively small: 11% for the three countries overall. In other words, the between-students variance in the unconditional model was 4,078 and 3,642 in the conditional model. The difference between the two values was 436. The variance at this level was reduced by 436 (11%) from 4,078 when the two predictors were added to the model. On the other hand, the amount of reduction in the variance for the intercept attributed to the predictors as a whole at the school level was 1,460 (55%) from 2,673. Figure 1 depicts the percentages of the variance components of the two models for the study. The results of the significance tests for the individual predictors can be found in Table 5. The study employed the statistical significance level of .05.

Figure 1.

Decomposition of total variance.

Table 5.

Parameter Estimates of the Conditional Model.

	Country
Fixed Effect Coefficient (SE)	Korea	Mexico	United States
Country, γ₀₀₁, γ₀₀₂, γ₀₀₃	494.4308* (8.7556)	404.0327* (3.8340)	575.6578* (7.0417)
LEADINS, γ₀₁₁, γ₀₁₂, γ₀₁₃	13.2105* (2.4240)	−4.0253* (0.7999)	−8.6285* (1.5971)
LEADTCH, γ₀₂₁, γ₀₂₂, γ₀₂₃	−5.9264* (3.0098)	3.5443* (0.8380)	−4.6384* (2.2725)
LEADPD, γ₀₃₁, γ₀₃₂, γ₀₃₃	−9.2450* (2.8410)	4.7398* (1.1537)	4.6255* (1.9907)
AUTOFNCE, γ₀₄₁, γ₀₄₂, γ₀₄₃	−3.8528* (1.3474)	8.1695* (0.2547)	1.0636 (0.6123)
AUTOTCH, γ₀₅₁, γ₀₅₂, γ₀₅₃	2.9335* (0.4839)	−0.7737* (0.2967)	−2.2773* (0.7201)
TSTSCHACC, γ₀₆₁, γ₀₆₂, γ₀₆₃	−9.5945* (1.3125)	0.6715 (0.4952)	−3.5402* (1.1449)
TSTCOMP, γ₀₇₁, γ₀₇₂, γ₀₇₃	6.1351* (1.0840)	−2.2485* (0.3769)	−0.7300 (1.1389)
TSTINS, γ₀₈₁, γ₀₈₂, γ₀₈₃	13.6500* (1.5794)	2.1055* (0.7581)	9.5794* (1.9101)
CAPTYHIND, γ₀₉₁, γ₀₉₂, γ₀₉₃	−9.1838* (1.9895)	−11.5180* (0.5436)	−23.9032* (1.7612)
COMMTY, γ₀₍₁₀₎₁, γ₀₍₁₀₎₂, γ₀₍₁₀₎₃	7.3679* (1.0382)	13.7375* (0.3811)	−4.2471* (0.8833)
OUTSCH, γ₁₀₁, γ₁₀₂, γ₁₀₃	6.4113* (3.2015)	−37.3968* (0.8412)	−57.0450* (3.6950)
ESCS, γ₂₀₁, γ₂₀₂, γ₂₀₃	14.0899* (1.6933)	6.3921* (0.3500)	27.9513* (1.4830)
Random effect (reduction %)
Level 1 variance, σ²	3641.7143 (11%)
Level 2 variance, τ₀²	1213.6115 (55%)
Level 2 OUTSCH, τ₁²	226.1608
Level 2 ESCS τ₂²	56.8929

p < .05.

Principals’ Perceptions of Leadership

The first research question of the study was about principals’ perceptions of their instructional leadership. The three variables related with the principals’ perceptions of their leadership showed inconsistent results with student learning. None of the three variables had the same effect (negative or positive) in the three countries. For example, principals’ perceptions of their instructional leadership (LEADINS) were positively related to the PISA scores in Korea but negatively in Mexico and the United States; likewise, principals’ perceptions of their leadership of teachers (LEADTCH) showed a positive impact in Mexico but negative in the other two countries; and principals’ perceptions of their leadership for professional development (LEADPD) were negatively correlated in Korea but positively in Mexico and the United States.

School Autonomy

The degree of reported school financial autonomy (AUTOFNCE) was related positively with student reading literacy in Mexico (γ₀₄₂), whereas the effect was negative in Korea (γ₀₄₁). This variable was not significant in either direction in the United States. On the other hand, as the degree of school autonomy in teaching and learning (AUTOTCH) increased, students’ reading literacy tended to increase in Korea (γ₀₅₁), whereas the tendency decreased in the other two countries (γ₀₅₂, γ₀₅₃).

Use of Test Data

As schools used achievement data in more accountability procedures (TSTSCHACC), students scored lower in reading literacy in Korea (γ₀₆₁) and the United States (γ₀₆₃), whereas there was no significant impact in Mexico (γ₀₆₂). As schools used assessments more for administrative and comparative purposes (TSTCOMP), such as making decisions about promotion to next grade, comparing between schools, and evaluating teachers’ effectiveness, students tended to perform better in the reading literacy test in Korea (γ₀₇₁), whereas the students from Mexico tended to score lower on the test (γ₀₇₂). As assessments of students were used more for instructional purposes (TSTINS), students from all three countries performed better (γ₀₈₁, γ₀₈₂, γ₀₈₃).

School Hindrance for Instruction and Community Size

Concerning the covariates of the model for the study, as school’s capacity to provide instruction hindered by shortages of qualified teachers or school resources (CAPTYHIND) increased, students appeared to perform worse in all three countries (γ₀₉₁, γ₀₉₂, γ₀₉₃). As schools were located in a more populous community (COMMTY), students showed a better performance in the reading literacy test in Korea (γ₀₍₁₀₎₁) and Mexico (γ₀₍₁₀₎₂), but they performed more poorly in the United States (γ₀₍₁₀₎₃). The influences of these covariates were statistically controlled for when the effects of the variables of primary interest were interpreted earlier.

Enrichment Lesson and Socioeconomic Status

As was addressed earlier, only 26% or less of the students reported having enrichment lessons in the test language outside of normal school hours (OUTSCH) across countries. As students attended such lessons, they appeared to perform better in the reading literacy test in Korea (γ₁₀₁) and worse in both Mexico (γ₁₀₂) and the United States (γ₁₀₃). The SES of students (ESCS) was, however, positively associated with their reading literacy. The higher the SES of students, the higher students scored on the test in all three countries (γ₂₀₁, γ₂₀₂, γ₂₀₃). The influence of students’ SES like the other control variables was statistically taken into account when the effects of principal leadership, school autonomy, and use of test data were interpreted.

Discussion

Comparisons of Korea, Mexico, and the United States should be made carefully in light of their policy agendas, economic histories, and varying cultural traditions. The politics of leadership, preparation, and practice are quite different. The countries vary in the level of public financing for schools, the extent of formal preparation programs for administrators, the role of nonpublic schools, and the emphasis on accountability, standards, and preparation.

Table 6 shows the results for each of the variables in Korea, Mexico, and the United States alongside the questionnaire items that defined the variable. A plus symbol is listed for a positive correlation, a minus for negative, a blank for no correlation with student reading literacy.

Table 6.

Summary of Predicting and Control Variables.

Variable		Country
Name	Description	Korea	Mexico	United States
Predicting variables
LEADINS	Instructional leadership	+	−	−
LEADTCH	Leadership of teachers	−	+	−
LEADPD	Leadership for professional development	−	+	+
AUTOFNCE,	School financial autonomy	−	+
AUTOTCH	School autonomy in teaching and learning	+	−	−
TSTSCHACC	Testing for accountability	−		−
TSTCOMP	Testing to compare	+	−
TSTINS	Testing to improve instruction	+	+	+
Control variables
CAPTYHIND	School instruction hindered by shortage of human and material resources	−	−	−
COMMTY	Community size where school is located	+	+	−
OUTSCH	Student attend enrichment lessons in test language	+	−	−
ESCS	Student socioeconomic status	+	+	+

Consistent Results Across Countries

The first conclusion from Table 6 is that there was consistency across countries on three variables. First, there was a positive relationship between student achievement in reading literacy and testing to improve instruction in all three countries: Korea, Mexico, and the United States. Testing to improve instruction was defined as grouping students for instructional purposes to identify aspects of instruction or the curriculum that could be improved. It is important to distinguish between this testing variable and two others. The results for testing for accountability and testing for comparative purposes were not uniformly positive across countries.

Second, there was a positive relationship between student achievement in reading literacy and SES of the students. The third area of consistency was a negative relationship between student achievement and capacity hindrance, which referred to shortages in personnel and equipment. In other words, students achieved less in schools that were hindered by lack of resources.

Variations Across Countries

The countries varied across other independent variables studied in the following ways:

With one exception (TESTINS), the direction of the relationships of the seven research variables with reading literacy was the opposite between Korea and Mexico. In other words, the same variables worked in different ways in the two countries.

In the case of the United States, with one exception (TESTINS), four out of five variables had a negative correlation with literacy.

The four control variables all had an impact on reading results. The community size (COMMTY) and the status of having enrichment lessons in test language (OUTSCH) did not show the same relationship for the three countries.

To understand these country to country variations, each country will be discussed individually.

Korea

In Korea, principals’ leadership of instruction (LEADINS) turned out to have a positive impact on student literacy as contrasted with the negative effects of their leadership of teachers work (LEADTCH) or professional development (LEADPD) activities for school educational goals. These findings may imply that direct pressures imposed on teachers from the school leader might not be helpful for student achievement in reading literacy in this country. The negative relationship between school financial autonomy (AUTOFNCE) and reading achievement appeared to be modest overall in Korea, considering the small mean and SD of this variable in Table 2. Those schools that had more responsibility for making decisions in finance-related matters such as teacher salaries, budget allocations, and firing of teachers tended to be poorer in student reading literacy.

As schools were more responsible for establishing student disciplinary or assessment policies, and determining course content or courses to be offered (AUTOTCH), the students were more likely to perform better in reading literacy. Promoting this type of school autonomy relating directly to teaching and learning may result in effective schooling in Korea.

Except for budget allocations to schools or evaluations of principals, a variety of uses of assessments seemed positively associated with student literacy in this country. As school tests are used for more high-stakes purposes (TSTCOMP), students were likely to perform better on the test on average. This finding needs to be cautiously interpreted in guiding educational administrators for school policies. Good performance on the test per se is not an educational goal for children. Instead, it should be considered a byproduct of successful learning acquisition. Thus, high-stake tests cannot be implemented at school simply to improve student performance on tests.

School instruction that was hindered by lack of human and material resources (CAPTYNHIND) had a negative impact, whereas students’ economic, social, and cultural status had a positive impact on student reading literacy in Korea. This finding appeared in line with some previous studies. Shortages of human or material resources at school or student SES affected student academic achievement significantly in other studies (Ban & Shin, 2011; Harris & Sass, 2007; Shin & Slater, 2010). Meanwhile the size of the community where a school was located was positively related with student reading literacy. This finding was also consistent with the result of a previous study (Kang, 2010), which reported that schools located in cities performed better than their counterparts in small towns or villages consistently over time in Korea. There was a significant difference between the two comparison groups with regard to having enrichment lessons in the Korean language. The students who reported attending enrichment lessons in Korean tended to score higher. This tendency was opposite to the findings of Mexico and the United States where the variable showed a negative relationship with the test score. This may indicate that high achievers are more likely to seek enrichment courses in Korea, whereas such courses are taken by students who need remedial treatment in test language most of time in Mexico and the United States. The overall degree of parental pressure for children’s academic performance at K-12 grade levels may be considerably different across the three countries. This difference may be attributable to the inconsistent findings among the countries with regard to this variable.

Mexico

Within Mexico there were large positive relationships between the two predictor variables and reading literacy: autonomy in the use of school finance (AUTOFNCE) and community size (COMMTY). There were also smaller but significant relationships between achievement and three variables of principal perception: principal leadership of teachers (LEADTCH), principal leadership of professional development (LEADPD), and Principal leadership for instruction (TSTINS).

These results are what might be expected from the literature. Mexico has been moving slowly from a strong centralized educational system to one with more local autonomy. Also, Mexico has a significant rural population that is less educated than those in urban centers. One would expect that the larger the community, the higher the achievement. An examination of conditions in small communities that affect student achievement indicates that small communities schools may harbor inherent disadvantages (González-Romo, Estrella-Chulím, & Ramírez-Valverde, 2006). They may suffer from poor educational conditions, because sparse population bases often result in geographic and cultural isolation, limited economic development, and restricted educational opportunities (McCombs & Bansberg, 1997). Rural schools typically lack the facilities, physical plants, course materials, and educational programs that typify larger, more resource-rich districts. Also, rural teachers generally have less professional preparation (Stern, 1994).

Mexico also fulfills expectations that there is a connection between leadership and student achievement. Principals who instruct teachers to work according to the school’s educational goals (LEADTCH) have students with higher achievement. Principals who lead teachers and provide professional development also have higher achievement.

On the other hand, Mexico students’ reading literacy was related negatively with three predicting school variables: instructional leadership (LEADINS), school autonomy in teaching and learning (AUTOTCH), and testing to compare (TSTCOMP); and no relationship was found with testing for accountability (TSTSCHACC).

A shortage of human and material resources (CAPTYHIND) and student attendance at enrichment lessons in the test language (OUTSCH) were negatively related with reading abilities. Meanwhile, community size where a school is located (COMMTY) and student SES (ESCS) were positively related with PISA scores. It is not difficult to understand that students who need lessons to improve their Spanish abilities and who live in small communities in this country are those with indigenous or rural background, a known disadvantaged for learning success.

United States

In the United States, there were positive relationships between reading achievement and leadership for professional development (LEADPD) and testing to improve instruction (TSTINS). Professional development is necessary for successful implementation of new programs, which must then be monitored by regular examination of test data. McKinsey and Company (2007) studied nations around the world and found a commonality of professional development and use of data to improve instruction among successful schools.

There were negative relationships between reading achievement and a number of variables in the United States: accountability (TSTSCHACC), leadership for instruction (LEADINS), leadership of teachers (LEADTCH), school autonomy in teaching and learning (AUTOTCH), shortage of resources (CAPTYHIND), student attendance at enrichment lessons in the test language (OUTSCH), and community size (COMMTY).

The negative correlation between achievement and accountability (TSTSCHACC) raises questions about the efficacy of large-scale accountability projects in the United States. Principals perceived that accountability and use of test data were relatively widespread, but they were not correlated with reading achievement. The negative correlations with leadership for instruction (LEADINS), leadership of teachers (LEADTCH), and school autonomy in teaching and learning (AUTOTCH) are not easily explained.

The literature cited earlier would generally suggest the opposite result (Hallinger et al., 1996; Hallinger & Heck, 1997; Leithwood & Day, 2008; Pont et al., 2008).

Whether shortage of resources (CAPTYHIND) affects achievement is a continuing debate. The U.S. principals in this sample tended to rate resources as a problem if they were in lower achieving schools. Student attendance at enrichment lessons in the test language (OUTSCH) in the United States is largely for students whose families have recently immigrated to the country and tend to be poorer than the rest of the population.

The results for community size may at first seem puzzling. Why would community size correlate positively with achievement in Korea and Mexico but negatively in the United States? The answer is likely that people who live in big cities in Korea and Mexico are better educated than their fellow citizens in rural areas. However, in the United States there is a preponderance of low-SES students with poor achievement in big cities like Chicago, Philadelphia, Houston, Los Angeles, and New York (Berger & Robinson, 1982; NAEP, 2011). Research on small schools (which included a large majority of rural schools) revealed that small school size can mitigate the influence of poverty (Howley, Strange, & Bickel, 2000). The resource limitations rural schools often experience can be compensated for by the supportive ethos found in smaller communities and their generally smaller schools (Stern, 1994). Many rural schools feature low student–teacher ratios, individualized instruction and attention, cooperative learning opportunities, close relationships and ties to the community, and strong staff commitment (DeYoung, 1987; Mid-Continent Regional Educational Laboratory, Rural Institute, 1990). According to the Schools and Staffing Survey, rural schools tend to be a better place for learning than their urban or suburban counterparts in terms of teacher and student absenteeism, safe learning environment, student misbehavior, and alcohol and drug use (Stern, 1994).

Conclusion

Before examining the implications of this study, it is important to ask why we did not find consistent results between countries for almost all the research questions. There are at least three reasons: the peculiarities of the educational systems in each country, the distance between principal perceptions and students’ learning, and the PISA questionnaire limitations to measure the same educational construct across countries.

Each country has a particular educational system and their principals, teachers, and students behave differently from each other in certain circumstances. For example, we pointed out that in Korea and Mexico community size positively affected student achievement; in the United States, the effect was the opposite. The reason for this difference was likely that in Mexico and Korea the best schools were located in big cities, whereas the contrary could be happening in the United States, where the majority of immigrants and poor people are concentrated in the big cities.

Second, there is a gap between principals’ perceptions of their type of leadership and student learning, which is mediated by school teachers and classroom activities. Of course, principal attitudes and behaviors have an important influence on teacher professional activities, but in the end, teacher pedagogical activities make the most difference in student achievement. In other words, even though a school may have an excellent principal, it may not have teachers who are able to operationalize the principals’ school policies.

Third, it is complicated to design an international questionnaire that uniformly and validly measures characteristics across different countries, with their own idiosyncrasies and cultural differences (Diaz-Loving, 1999). The same translated questions could have different meanings for each group of principals, depending on their cultural background. Solano-Flores, Backhoff, and Contreras-Nino (2009) have shown that item translation could easily be biased for at least 60 reasons. Cultural issues make it difficult to construct international questionnaire items that behave in the same way, without response variance.

The findings of this study are based on 1 year of data; hence, implications are limited to the investigated year and years close to it. To examine steady differences or similarities between the countries over time, the significant effects found in this study need to be replicated by analyzing years of data tracked over extensive time. Also, this study focused on reading results leaving math and science literacy for future analysis.

Furthermore, the analysis results of the study came from complete cases, excluding about 3% to 5% data from the analysis. PROC MIXED is known to handle missing data where the dropout process is random (MAR) correctly (Verbeke & Molenberghs, 2000), but lead to incorrect estimates if the data are completely random (MCAR; Larsen, 2011). The missing data in this study were assumed to be MCAR because cases were thrown out as far as the data on the predictors was missing. Although the missing rates of the study are quite small, there is still a chance that the missing responses could lead to different results if the incompleteness is handled differently. In future research, the impact of varying degrees of incomplete data in the context of HLM needs to be explored in depth.

Collinearity is known to inflate the Type II error rate. A Type II error happens when failing to reject the null hypothesis although the alternative hypothesis is true. In the current study, all three predictor variables relating to school leadership that showed substantially higher bilateral correlations than the other predictors turned out significant. This might increase the likelihood that the correlations of these variables with reading literacy are true in the population. For a deeper understanding of the effects of the VIF, further research is needed that manipulates the degree of collinearity along with other influential factors in the context of HLM.

The most important implication of this study is that testing to improve instruction was beneficial in all three countries when it was defined as using tests to group students for instructional purposes and to identify aspects of instruction so that the curriculum can be improved. However, there was no uniform support for testing for accountability or testing for comparative purposes. These results add to evidence that efforts to use data to improve instruction can be highly beneficial in these three countries. However, emphasis on external comparison may not be helpful and may actually hinder school improvement efforts.

Howard (2010) recently completed a study of successful schools across the United States. He expected to find teachers’ effective practices, intensive academic intervention, explicit acknowledgement of race, and engagement of parents and community, but he was surprised by the level of importance of visionary leadership.

Each of the principals at these schools set a tone and vision of high achievement, worked tirelessly to ensure academic success for all students, and was relentless in his or her efforts to get all stakeholders to buy into the vision for student learning and overall school success. While visionary leadership can be hard to define explicitly, it is an important variable to study further to determine how it varies from culture to culture and how it relates to system policies related to school autonomy and testing for school improvement. (p. 134)

Results of the current study indicated that students are likely to achieve better if principals perceive that there are no shortages of personnel and equipment. There is an international debate about whether more funding will improve education (McKinsey & Company, 2007), and the key is to find the best way to invest resources to make a difference in achievement.

Finally, this study found, as many others have, that there was a positive relationship between student achievement in reading literacy and SES of the students. Ensuring the best possible education for all children means overcoming this. Teachers are the key to improving instruction and principals are the likely source of guidance and leadership.

Footnotes

Appendix

Appendix A

Predictors and Corresponding Survey Questions

Name of Predictor (Score Composition)	Level of Predictor	Survey Question
LEADINS (mean of item scores)	2	Indicate the frequency of the following activities and behaviors in your school during the last school year.
		I use student performance results to develop the school’s educational goals.
		I take exam results into account in decisions regarding curriculum development.
		I ensure that there is clarity concerning the responsibility for coordinating the curriculum.
		1—Never
		4—Very often
LEADTCH	2	Indicate the frequency of the following activities and behaviors in your school during the last school year.
		I ensure that teachers work according to the school’s educational goals.
		1—Never
		4—Very often
LEADPD (mean of item scores)	2	Indicate the frequency of the following activities and behaviors in your school during the last school year.
		I make sure that the professional development activities of teachers are in accordance with the teaching goals of the school.
		I inform teachers about possibilities for updating their knowledge and skills.
		1—Never
		4—Very often
AUTOFNCE (sum of item scores)	2	Regarding your school, who has a considerable responsibility for the following tasks?
		Selecting teachers for hire.
		Firing teachers.
		Establishing teachers’ starting salaries.
		Determining teachers’ salaries increases.
		Formulating the school budget.
		Deciding on budget allocations within the school.
		0—Others
		1—Principals
AUTOTCH (sum of item scores)	2	Regarding your school, who has a considerable responsibility for the following tasks?
		Establishing student disciplinary policies.
		Establishing student assessment policies.
		Approving students for admission to the school.
		Choosing which textbooks are used.
		Determining course content.
		Deciding which courses are offered.
		0—Others
		1—Principals
TSTSCHACC (sum of item scores)	2	In your school, are achievement data used in any of the following <accountability procedures>?
		Achievement data are used in evaluation of the principal’s performance.
		Achievement data are used in decisions about instructional resource allocation to the school.
		0—No
		1—Yes
TSTCOMP (sum of item scores)	2	In your school, are assessments of students in <national modal grade for 15-year-olds> used for any of the following purposes?
		To make decisions about students’ retention or promotion.
		To compare the school to <district or national> performance.
		To make judgments about teachers’ effectiveness.
		To compare the school with other schools.
		0—No
		1—Yes
TSTINS (sum of item scores)	2	In your school, are assessments of students in <national modal grade for 15-year-olds> used for any of the following purposes?
		To group students for instructional purposes.
		To identify aspects of instruction or the curriculum that
		could be improved.
		0—No
		1—Yes
CAPTYHIND (mean of item scores)	2	Is your school’s capacity to provide instruction hindered by any of the following issues?
		A lack of qualified science teachers.
		A lack of qualified mathematics teachers.
		A lack of qualified <test language> teachers.
		A lack of qualified teachers of other subjects.
		A lack of library staff.
		A lack of other support personnel.
		Shortage or inadequacy of science laboratory equipment.
		Shortage or inadequacy of instructional materials (e.g., textbooks).
		Shortage or inadequacy of computers for instruction.
		Lack or inadequacy of Internet connectivity.
		Shortage or inadequacy of computer software for instruction.
		Shortage or inadequacy of library materials.
		Shortage or inadequacy of audiovisual resources.
		1—Not at all
		4—A lot
COMMTY	2	Which of the following definitions best describes the community in which your school is located?
		1—A village, hamlet, or rural area (fewer than 3,000 people)
		5—A large city (with more than 1,000,000 people)
OUTSCH	1	Do you attend enrichment lessons in test language currently?
		0—No
		1—Yes
ESCS	1	A composite score which is provided by PISA represents student socioeconomic status.

Note. Dichotomous item scores within a variable were summed for a composite score, whereas polytomous item scores were averaged.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Seon-Hi Shin, PhD, currently serves as a principal research scientist and chair of the Division of Measurement and Statistical Research at the TEPS Center in Seoul National University, Seoul, Korea. Her research interest focuses on exploring theories and applications in standardized testing.

Charles L. Slater, PhD, is professor of educational leadership at California State University Long Beach. He previously served as professor at Texas State University, San Marcos, and was superintendent of schools in Texas and Massachusetts. His research has focused on what is needed for successful leadership in the United States, Mexico, and Costa Rica, with special attention to educational administration preparation.

Eduardo Backhoff is an educational researcher at the Institute of Educational Research and Development, Autonomous University of Baja California, Mexico. He has worked in the field of large scale assessment, computer assisted assessment, and the problems of international test translation. He was the Scientific Editor of the Online Journal of Educational Research (REDIE) and is a member of the Technical Advisory Group of OECD for the Teaching and Learning International Survey (TALIS) project.

References

Backhoff

Andrade

Bouzas

Santos del Real

Santibáñez

(2009). Teaching and Learning International Survey (TALIS). Mexico City, Mexico: Secretaria de Educacion Publica.

Backhoff

Bouzas

Contreras

Hernández

Gracia

(2007). Factores escolares y aprendizaje en México: El caso de la educación básica [School factors and learning in Mexico: The case of elementary education]. Del Benito Juárez, México: Instituto Nacional para la Evaluación de la Educación.

Ban

Shin

(2011). Trend of the effects of multilevel contextual variables on student achievement: 2003-2009 NAEA data analysis. Korean Journal of Educational Evaluation, 24, 685-712.

Berger

Robinson

A. H.

(Eds.). (1982). Secondary school reading: What research reveals for classroom-practice. Urbana, IL: National Council of Teachers of English.

Belsley

D. A.

Kuh

Welsch

R. E.

(1980). Regression diagnostics: Identifying influential data and sources of collinearity. New York, NY: Wiley.

Bryk

A. S.

Raudenbush

S. W.

(1992). Hierarchical linear models. Newbury Park, CA: Sage.

Colorado Department of Education. (2009). The Colorado growth model. Retrieved from http://www.schoolview.org/ColoradoGrowthModel.asp

Day

Leithwood

Sammons

(2008). What we have learned, what we need to know more about. School Leadership & Management, 28(1), 83-96.

Delaware Department of Education. (2010). School accountability in Delaware. Retrieved from http://www.doe.k12.de.us/

10.

DeYoung

A. J.

(1987). The status of American rural education research: An integrated review and commentary. Review of Educational Research, 57, 123-148.

11.

Diaz-Loving

(1999). The indigenisation of psychology: Birth of a new science or rekindling of an old one? Applied Psychology: An International Review, 48, 433-449.

12.

García Garduño

J. M.

Slater

C. L.

López-Gorosave

(2011). El director escolar novel: Estado de la investigación y enfoques teóricos [The new principal: Status of research and theoretical approaches]. Revista Iberoamericana sobre Calidad, Eficacia y Cambio en Educación, 9(3), 3-50.

13.

González-Romo

Estrella-Chulím

Ramírez-Valverde

(Eds.). (2006). Enfoques de la pobreza rural en México [Approaches to rural poverty in Mexico]. Puebla, Mexico: Colegio de Postgraduados.

14.

Grewal

Cote

J. A.

Baumgartner

(2004). Multicollinearity and measurement error in structural equation models: Implications for theory testing. Marketing Science, 23, 519-529.

15.

Hallinger

Bickman

Davis

(1996). School context, principal leadership, and student reading achievement. Elementary School Journal, 96, 527-549.

16.

Hallinger

Heck

(1997). Exploring the principal’s contribution to school effectiveness. School Effectiveness and School Improvement, 8(4), 1-35.

17.

Heck

R. H.

Larsen

T. J.

Marcoulides

G. A.

(1990). Instructional leadership and school achievement: Validation of a causal model. Educational Administration Quarterly, 26, 94-125.

18.

Howard

T. C.

(2010). Why race and culture matter in schools: Closing the achievement gap in America’s classrooms. New York, NY: Teachers College Press.

19.

Howley

Strange

Bickel

(2000). Research about school size and school performance in impoverished communities. Charleston, WV: ERIC Clearinghouse on Rural Education and Small Schools.

20.

Kang

(2010). The analysis of the trend of educational gaps between high schools after 531 Educational Reform: Data analysis of 1995-2010 CSAT assessment (Research Report). Seoul, South Korea: Korean Ministry of Education, Science and Technology.

21.

Korean Ministry of Education, Science, and Technology. (2010). Instruction on 2010 school-based performance incentive program for educational public service employees. Seoul, South Korea: Author.

22.

Larsen

(2011). Missing data imputation versus full information maximum likelihood with second-level dependencies. Structural Equation Modeling, 18, 649-662.

23.

Lee

(2011). A study on teachers’ perception of school-based performance incentive program (Unpublished doctoral dissertation). Hong-Ik University, Korea.

24.

Leithwood

Day

(2008). The impact of school leadership on pupil outcomes. School Leadership & Management, 28(1), 1-4.

25.

Leithwood

Harris

Hopkins

(2008). Seven strong claims about successful school leadership. School Leadership & Management, 28(1), 27-42.

26.

Linn

R. L.

(2006). Educational accountability systems (CSE Tech. Rep. No. 687). Boulder: CRESST/University of Colorado at Boulder.

27.

Littell

R. C.

Milliken

G. A.

Stroup

W. W.

Wolfinger

R. D.

(1996). SAS system for mixed models. Cary, NC: SAS Institute.

28.

Mason

C. H.

Perreault

W. D.

(1991). Collinearity, power, and interpretation of multiple regression analysis. Journal of Marketing, 28, 268-280.

29.

Marzano

R. J.

Waters

McNulty

B. A.

(2005). School leadership that works. Alexandria, VA: Association for Supervision and Curriculum Development.

30.

McKinsey & Company. (2007). How the world’s best performing school systems come out on top. New York, NY: Author. Retrieved from http://mckinseyonsociety.com/how-the-worlds-best-performing-schools-come-out-on-top/

31.

McCombs

B. L.

Bansberg

(1997). Meeting student diversity needs in poor, rural schools: Ideal practices and political realities. In Wang

M. C.

Wong

K. K.

(Eds.), Implementing school reform: Practice and policy imperatives (pp. 162-91). Philadelphia, PA: Temple University Center for Research in Human Development and Education.

32.

Mid-Continent Regional Educational Laboratory, Rural Institute. (1990). Rural schools and education reform: They may be closer than you think (Rural Report). Aurora, CO: Author.

33.

Mislevy

R. J.

Beaton

A. E.

Kaplan

Sheehan

K. M.

(1992). Estimating population characteristics from sparse matrix samples of item responses. Journal of Educational Measurement, 29, 133-161.

34.

National Assessment of Educational Progress. (2011). District reports (IES National Center for Educational Statistics). Retrieved from http://nces.ed.gov/nationsreportcard/districts/

35.

O’Brien

R. M.

(2007). A caution regarding rules of thumb for variance inflation factors. Quality & Quantity, 41, 673-690.

36.

OECD (2009a). PISA data analysis manual (SAS® 2nd ed.). Retrieved from http://www.pisa.oecd.org/

37.

OECD. (2009b). PISA 2006 Technical report. Retrieved from http://www.pisa.oecd.org/

38.

OECD. (2010). PISA 2009 results: Executive summary. Retrieved from http://www.pisa.oecd.org/

39.

Ouchi

W. G.

Segal

L. G.

(2003). Making schools work: A revolutionary plan to get your children the education they need. New York, NY: Simon & Schuster.

40.

Partida

(1999). Proyecciones de la población económicamente activa de la matrícula educativa de los hogares y las viviendas y de la población por tamaño de la localidad [Projections of the economically active population for educational enrollment of houses and apartments by size of locality]. México City, Mexico: Consejo Nacional de Población.

41.

Pont

Nusche

Moorman

(2008). Improving school leadership. Vol. 1: Policy and practice. Paris, France: OECD.

42.

Schmoker

M. J.

(1999). Results: The key to continuous school improvement. Alexandria, VA: Association for Supervision and Curriculum Development.

43.

Sengupta

Bhimasankaram

(1997). On the roles of observations in collinearity in the linear model. Journal of the American Statistical Association, 92, 1024-1032.

44.

Shin

Slater

C. L.

(2010). Principal leadership and mathematics achievement: An international comparative study. School Leadership & Management, 30, 317-334.

45.

Slater

C. L.

(2011). Understanding principal leadership: An international perspective and a narrative approach. Educational Management Administration & Leadership, 39, 219-227.

46.

Singer

J. D.

(1998). Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. Journal of Educational and Behavioral Statistics, 24, 323-355.

47.

Solano-Flores

Backhoff

Contreras-Nino

L. A.

(2009). Theory of test translation error. International Journal of Testing, 9(2), 78-91.

48.

Spillane

J. P.

(2006). Distributed leadership. San Francisco, CA: Jossey-Bass.

49.

Stern

J. D.

(Ed.). (1994). The condition of education in rural schools. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement. (ERIC Document Reproduction Service No. ED 371 935)

50.

Teddlie

Reynolds

(Eds.). (2000). International handbook of school effectiveness research. London, England: Falmer.

51.

Texas Educational Agency. (2010). Calculating projections with the TPM. Retrieved from http://www.tea.state.tx.us/student.assessment/taks/tpm/index.html

52.

UNESCO. (2000). Education for all. Paris, France: Author.

53.

UNESCO Institute for Statistics. (2012). Education: Demographic and economic data. Retrieved from http://stats.uis.unesco.org/unesco/

54.

U.S. Census Bureau. (2012). School enrollment. Retrieved from http://www.census.gov/compendia/statab/

55.

U.S. Department of Education. (2002). Public law 107-110. Retrieved from http://find.ed.gov/

56.

Verbeke

Molenberghs

(2000). Springer series in statistics: Linear mixed models for longitudinal data. New York, NY: Springer-Verlag.

57.

W. Y. W.

J. O.

(2008). Reforming school governance in Taiwan and South Korea: Empowerment and autonomization in school-based management. International Journal of Educational Management, 22, 506-526.

58.

Yukl

(1989). Leadership in organizations. Englewood Cliffs, NJ: Prentice Hall.