Abstract
This study seeks to explore the effect of tools of school governance advanced by the global reform agenda (1) on student performance and (2) on reducing the effect of sociodemographic variables on this performance. To do so, we compare two Nordic welfare states with an egalitarian tradition: Sweden and Finland. The Swedish school system has undergone comprehensive market-based and accountability-driven reforms beginning in the early 1990s. The Finnish school system is still based on the professional trusteeship of teachers, but has moderately implemented reforms of decentralization, enhancing school autonomy and opportunities of school or curriculum choice beginning in the late 1990s. Our analysis comprises of two steps. In the first step, a literature review of the reforms and their effects in Sweden and Finland is carried out. In the second step, regression analyses are conducted using data from the OECD’s Programme for International Student Assessment (PISA) of 2000, 2009, and 2015. The results show an increasing influence of sociodemographic factors and growing inequality and no effect of governance tools as recommended by the global reform agenda in both countries over time, but more so and earlier in Sweden than in Finland.
Keywords
Introduction
Education reforms aiming at improving performance and closing achievement gaps have been on the agenda worldwide for a good three decades. One major trend of these reforms is extending opportunities of school choice, extending school autonomy, establishing educational markets and tightening control of schools through assessments and tests. The aim is the “self-improving school-led system” (Ball, 2012: 122; Greany and Higham, 2018; Münch, 2020). Sahlberg (2016) has coined the term “Global Educational Reform Movement” (GERM) for describing this worldwide trend. In his view “competition and choice,” “standardization,” “increased emphasis on reading literacy, mathematics, and science,” “borrowing of change models from the corporate world,” and “test-based accountability policies” are the main features of GERM (Sahlberg, 2016: 133–136). A major part of GERM is enhanced school autonomy along with the establishment of school markets and competition between schools on this market for the enrollment of students. Ranking lists recording graduation rates and test scores of the schools serve as measure for comparing schools when parents decide where to send their children. The aim is the establishment of portfolio school districts where parents have an open choice between different types of schools. A major device for establishing such portfolio districts is the replacement of “low performing” traditional public schools by publicly funded but privately run independent schools (Sweden), charter schools (U.S.), or academies (UK) (Gunter and McGinity, 2014; Münch, 2020: 63–77; Rayner et al., 2018; Salokangas and Ainscow, 2017). Kauko et al. (2018) along with other authors emphasize the widespread implementation of systems of quality assurance, Verger et al. (2019) the significance of National Large-Scale Assessments (NLSAs) in the implementation of the movement’s agenda in national education policies. Courtney and Gunter (2015) as well as Gunter et al. (2018) have turned attention to the emphasis on strong school leaders as part of educational managerialism making teachers recipients of their orders.
The guidelines of GERM are set by New Public Management (NPM) (Verger and Parcerisa, 2017). As teaching is the core of any education at school, most important is how accountability measures exert an impact on the professional work of teachers (Münch, 2020: 15–34). NPM replaces professional accountability by the implementation of external pressure to succeed in competition measured by the proliferation of test scores, enacted by the combination of school autonomy and high stakes assessments. Enhanced school autonomy implies, therefore, shrinking teacher autonomy. How far there remains teacher autonomy, how it is interpreted and carried out by teachers depends, on the one hand, on the participation of teachers in defining the external accountability criteria and, on the other hand, on how well established there is a tradition of teacher professionalism in a country (Salokangas and Wermke, 2020; Salokangas et al., 2020). Comparing Finland and Singapore, Hwa (2019) shows that teacher accountability measures improve student performance in terms of PISA scores, if they are consistent with the socio-cultural context. A major part of diminishing teacher professionalism is increased emphasis on principals as school managers which is the other side of the same coin (Münch, 2020: 147–151).
The claim of GERM is improving education and closing achievement gaps by applying the recommended tools of school governance. In this study, we submit this claim to a test at two egalitarian Nordic welfare states—Sweden and Finland—which have implemented such reforms on their developmental path, Sweden more radically since the early 1990s, Finland more moderately since the late 1990s (cf. Samuelsson and Lindblad, 2015). After deriving two hypotheses from our theoretical foundation, we first look at the current state of research with a review of the literature covering the years from 1990 to 2020 and secondly conduct a statistical analysis with data from the OECD’s Programme for International Student Assessment (PISA) for the years from 2000 to 2015, thus using data from a central agent of GERM.
Theoretical foundations
The theoretical foundation of this study starts with two opposing assumptions which are condensed in two opposing hypotheses. The first assumption claims that specific governance tools as recommended by GERM help to improve the performance of schools and students and to reduce the influence of family background on this performance. In this perspective, the school remedies social inequalities based on family background. The opposing assumption contradicts this claim arguing that the effectiveness of these tools is counteracted by the unequal ability of parents and students to benefit from them according to their socioeconomic status. It is assumed that this counteracting effect works against improving average performance and against reducing achievement gaps according to family background. In this perspective, the school just reproduces social inequality according to family background.
As regards the theoretical assumption and agenda of GERM, the OECD’s Programme for International Student Assessment (PISA) plays a central role in shaping and disseminating them. Global players of the education industry like McKinsey and Pearson make widely use of PISA data in backing up their recommendations for improving school systems, for example, in McKinsey’s publications on how schools improve and achieve (Barber and Mourshed, 2007; Mourshed et al., 2010) and Pearson’s (2012) The Learning Curve. The OECD itself makes recommendations for improving school systems, for example, for the United States or Sweden (OECD, 2011, 2015). Pearson has been part of the PISA consortium in charge of constructing the PISA test since 2015. With regard to the search for most effective school governance, we see three phases in the OECD’s focus. In the first phase, the focus was on school autonomy, free school choice, competition between schools, and regular central testing. This was complemented by focusing on teacher quality and school leadership in the second phase and school climate in the third phase (OECD, 2016: 81–103). The PISA reports have realized that school autonomy in itself is ambivalent in its effect on test scores. Accordingly, the argument has become more important that autonomy does only work if there is a strong focus on performance for which managerial school leadership and strict accountability of principals and teachers in terms of student test scores is assumed to be the right device. This is exactly the claim made by the OECD (2016: 114): “After all, several studies find that to reap the full benefits of school autonomy, education systems need to have effective accountability systems to discourage opportunistic behaviour by school staff, and highly qualified teachers and strong school leaders to design and implement rigorous internal evaluations and curricula.” (OECD, 2016: 114)
For supporting this claim, the OECD refers to Hanushek et al. (2013) who argue, that school autonomy needs complementation by effective instruments of school leadership and accountability in order to bring about the desired improvement of educational performance and closing of achievement gaps. Hanushek and Woessmann (2010, 2015) are key advisors of the OECD and the European Community (EC) in the agenda claiming that improving PISA scores with the most effective educational policies will bring about higher economic growth rates for every country following this advice (Komatsu and Rappleye, 2020; OECD, 2010). The evidence for this claim is, however, heavily questioned, for example, by Komatsu and Rappleye (2017) who show that there is no significant association between PISA scores and economic growth rates if one correctly accounts for a time lag between PISA performance and later economic growth.
The theoretical background of the OECD’s accountability-hypothesis based on Hanushek, Link, and Woessmann is principal/agent theory which expects better performance of agents (principals, teachers), if the principal (government, parents) counts on the local and practical knowledge of agents and grants them autonomy in their activity. However, in order to avoid “moral hazard” and “shirking” by agents, instruments of guaranteeing accountability of agents have to be applied. Hanushek, Link, and Woessmann prove this statement by demonstrating that school autonomy is associated with higher performance in highly developed countries, but not in underdeveloped ones. In their view, highly developed countries stand for effective instruments of securing accountability, but not underdeveloped countries.
Different to Hanushek et al. (2013), our focus is not on the country level, but on the school and individual levels in two highly developed countries representing the social democratic Welfare regime in its changing character following the global reform agenda since the 1990s. Hypothesis H1 represents the global reform agenda adjusted to the availability of PISA data:
H1: School systems and schools improve performance and minimize
performance gaps through reducing the impact of family background by applying governance tools. These tools include (1) enlarged autonomy and self-government of schools, complemented by (2) strong school leadership and (3) regular monitoring of teaching and student performance through tests and assessments.
Family background is first of all the socioeconomic status of individual students and the average socioeconomic status of a school’s students, secondly migrant background and test language spoken at home.
For testing the effectiveness of the global reform agenda with PISA data, we have to choose variables of the PISA survey that can be taken as operationalization of governance tools recommended by this agenda. For testing their effectiveness over a longer period of time, we need variables for which there are data provided by several test years spanning from 2000 to 2018. These requirements are fulfilled by the test years 2000, 2009, and 2015. This is why we take indices of (1) school autonomy, (2) school leadership, and (3) assessments as variables, each one consisting of a number of indicators that stand for governance tools of the global reform agenda. School autonomy stands indirectly for the establishment of educational markets on which autonomous schools compete for students and funds directly related to the number of enrolled students. Indices allow to depict numerical variations of the variables so that we get more nuanced insights. We think that the three indices cover the global reform agenda in its broader sense in three central aspects. More is not possible with the available PISA data.
What we do not include in hypothesis H1, is “highly qualified teachers,” because PISA only tells us about the average ISCED level of teachers at a school, for example, ISCED 5 or 6 for highly qualified teachers. This variable very likely just indicates that there is a higher-ranking school attended by highly qualified students of higher socioeconomic status, and such a school employs highly qualified teachers. Therefore, the effect is rather the other way round than assumed in the above-mentioned statement of the OECD. The variables of strong leadership and accountability might, however, be taken as proxies for highly qualified teachers in the reformers’ sense if strong school leaders and accountability measures imply strong emphasis on teacher quality in terms of their selective recruitment and regular training. This is, however, only a so far unproven assumption.
As regards the theoretical assumption, which is opposed to the theoretical assumption of GERM, we take Bourdieu and Passeron’s theory of the school mode of reproducing inequality (Bourdieu, 1984, 1998; Bourdieu and Passeron, 1990). We may argue with Bourdieu and Passeron that governance tools as spelled out by hypothesis H1 cannot significantly remedy disadvantages resulting from family background, if acquired habitus and inherited economic, social, and cultural capital determine how much students benefit from them. This may be the case with instruments that emphasize school autonomy, strong school leadership, highly qualified teachers, and accountability. In Bourdieu and Passeron’s perspective, they do not effectively compensate for socioeconomic inequalities, if they require a high degree of initiative on the part of parents and students, which in turn is easier to do the more economic, social and cultural capital is available. Better-off students are thus better prepared to do well on more frequent tests, learn more from better qualified and supervised teachers, choose higher-performing schools based on free school choice and the publication of school performance data, and can more easily meet the demands of high school discipline under strict school governance. And this advantage is further enhanced by attending schools at a higher average level of students’ economic, social, and cultural capital (Van Zanten, 2005). We take Bourdieu and Passeron’s class theory of education because habitus and available capital can be conceived as moderators of the effect of governance tools.
Goldthorpe (2007a) has criticized Bourdieu’s theory of the reproduction of social inequality, arguing that the expansion of education since the 1960s has proven the opposite. Following Boudon’s (1974) distinction between primary and secondary effects of family socialization, he sees compensatory education through schooling as a means of correcting primary socialization disadvantages and informing parents about the low cost of schooling relative to the achievable returns as a means of correcting secondary socialization disadvantages. However, Goldthorpe (2007b: 166) also finds that better-off parents have and perceive more opportunities than worse-off parents to ensure the educational achievement of their children (see Breen and Jonsson, 2005; Erikson and Goldthorpe, 1992; Hertel and Groh-Samberg, 2019). Accordingly, we set up hypothesis H2:
H2: As School systems and schools apply governance tools like (1) enlarged autonomy and self-government of schools, complemented by (2) strong school leadership and (3) regular monitoring of teaching and student performance through tests and assessments, these tools are more effectively used the higher the socioeconomic status of students so that they do not improve performance and minimize
performance gaps through reducing the impact of family background.
For testing hypotheses H1 and H2, we proceed in two steps. In the first step we review evidence provided by studies using country-specific data, in the second step we conduct a multilevel regression analysis using PISA data. We look at two countries with a common egalitarian tradition, Sweden and Finland, with Sweden having strongly implemented reforms along GERM since the early 1990s and Finland having implemented such reforms more moderately since the late 1990s.
Market-driven reforms on an egalitarian developmental path in Sweden
In the 1990s, Sweden underwent a series of educational reforms in the course of general decentralization and deregulation of the public sector with the aim of strengthening democracy. They together led to a comparatively rapid and radical restructuring of the Swedish education system, making it one of the most decentralized in the OECD (Beach, 2018: 101–131; Björklund et al., 2004; Dahlstedt and Fejes, 2019; Fjellman et al., 2019; Grek, 2017, 2020; OECD, 2015: 48).
With the first reform in 1991, formal and financial responsibility for compulsory education, secondary education and adult education (but not for higher education) was transferred from the central government level to the municipal level (see Björklund et al., 2004:10; Varjo et al., 2018). This reform was followed in 1992 by the introduction of free school choice. Nevertheless, the principle of residence is still considered the primary distribution principle (Björklund et al., 2004: 10; Böhlmark and Lindahl, 2007: 6). The introduction of free choice of school was directly related to another reform: Starting in 1992, “independent” or “charter” schools have increasingly gained in importance. These are schools that are run by private institutions but are funded by the state and are therefore not allowed to charge school fees or apply special selection procedures in admitting pupils (Björklund et al., 2004: 11; Skolverket, 2012: 6). As a result of this reform, the number of charter schools has steadily increased. In 2017, 15% of all compulsory school students and 27% of all upper secondary school students attended charter schools (Sweden, 2020). In the end, school markets have emerged, and it can be said that marketization has become a hallmark of Swedish education policy. Referring to our theoretical assumptions, explanatory factor (1) enlarged autonomy and self-government of schools of hypothesis H1 and along with this measure educational markets have been introduced.
In 1994, a reform of the national curriculum was implemented. Here, too, decentralization has taken place in that the central government continues to have responsibility for the national curriculum and educational goals but leaves teachers greater freedom of decision with regard to content and forms of teaching (Björklund et al., 2005: 2) While the evaluation of schools had previously been the responsibility of the central government, the emphasis was now on the self-evaluation of schools (Björklund et al., 2004: 11; Hanberger et al., 2016; Skolverket, 2012: 6). In 1996, the payment of teachers was reformed. Whereas previously the salaries of teachers and school administrators were negotiated at central government level and applied nationwide, since that reform, teachers’ salaries have been negotiated in a more decentralized and individualized manner. As a rule, salaries are set by school administrators in negotiations with the municipality (Björklund et al., 2004: 10). We assume that these reforms implied increased emphasis on (2) strong school leadership and (3) regular monitoring of teaching and student performance through tests and assessments according to hypothesis H1.
What is the evidence? Has average performance of students improved and have achievement gaps according to family background been reduced according to hypothesis H1 or has no improvement occurred as postulated by hypothesis H2? To begin with, Hennerdal et al. (2020) have shown that there is no improvement when individual level background factors and differences in the peer composition of schools are controlled. Böhlmark and Lindahl (2007: 41) conclude, based on longitudinal data from 1988 to 2003, that school choice has induced greater segregation of students according to parental education and second-generation immigrant status. In a more recent study on students who graduated from compulsory school (at grade 9) between 1988 and 2009, Böhlmark et al. (2016) have found that segregation between immigrant and native students is first of all based on neighborhood segregation, on the one hand, but has increased most where opportunities of choice are greatest, on the other hand. Nevertheless, they note that the increase in segregation has been relatively small since the introduction of school choice. A study by Andersson et al. (2021) on school leavers from 1991 to 2012 demonstrates increasing school and residential segregation of students according to migrant background, and, from 2008 on, according to income, employment status and social allowance reception (see also Lundahl and Lindblad, 2018).
In a study covering the period from 1988 to 2014, Gustafsson and Yang Hansen (2018) have found a clear and increasing correlation between the educational status of parents and the school performance of pupils. Further studies show that free choice of school has increased the variance in student performance between schools (OECD, 2015; Skolverket, 2009). For example, Östh et al. (2013) note that the variation in student achievement between schools in Sweden has increased rapidly since 2000 and that this increase must be attributed mainly to the reform of free school choice and decentralization measures in the school system. Hansen and Gustafsson (2019) have found in a comprehensive study that socioeconomic inequality in final grades of 9-year compulsory schooling, which is left at around the age of 16, increased sharply between 1998 and 2014. This applies equally to the groups of pupils with and without migration background. There is further evidence for this finding in international comparison (Hansen and Strietholt, 2018; Löfstedt, 2019; Põder et al., 2017). For the period from 1998 to 2011, Hansen and Gustafsson (2016) observe a growing inequality of performance between schools, mostly in metropolitan regions, followed by large cities, less so in small towns and rural areas. This is due to the growing segregation between the schools in which migrant pupils are concentrated and the schools attended by native pupils (Behtoui et al., 2019).
The increasing separation into native and migrant schools supports the observed fact of a rising performance inequality between schools according to the socioeconomic status of parents. Hansen and Gustafsson (2016: 16) see the interplay of residential segregation and free choice of school as the cause of the growing segregation of schools in their performance levels. Brandén et al. (2019), in turn, have found in a study that the ethnic composition of a school does not change the influence of social background on school grades, but pupils with a migrant background have slightly better chances of qualifying for higher secondary school the more fellow pupils they have from their own ethnic group.
Competition changes the schools’ view of the pupils, differentiating between “right” pupils and those who are an obstacle to the school’s success because they have special needs (Beach and Sernhede, 2011; Kallstenius, 2010). Although independent schools, like their public counterparts, must not apply selection criteria other than the residence and first-come-first-serve principles, there are ways of influencing the pool of applicants in terms of cream skimming. This can be done, for example, by opening independent schools in residential areas where better performing pupils are expected to live, or by targeting advertising and information at this group (Forsberg, 2015). And independent schools do not always follow the rules and avoid enrolling underperforming pupils (Böhlmark et al., 2016: 1162).
With the introduction of school choice, the parents’ choice strategies have become most important for their children’s achievement (Kallstenius, 2010: 234). Trumberg and Urban (2021) have found that socioeconomically better off parents make more use of free choice of school than socioeconomically worse off parents, but that in the long run their children achieve worse than to be expected based on their status. However, the educational performance of pupils with a migrant background and/or low socioeconomic status whose parents choose a school other than the one closest to their home improves, particularly, the longer their parents live in Sweden (Smith et al., 2019). In general, however, the effect of free school choice is schools being increasingly segregated according to their performance level. In the end, evidence from studies using country-specific data is in favor of hypothesis H2 and against hypothesis H1.
Decentralization reforms on an egalitarian developmental path in Finland
In its egalitarian tradition shared with Sweden, the Finnish comprehensive school is focused on providing all pupils with the best possible learning conditions. The school is expected to promote and include, not select and exclude. What makes the Finnish approach in this endeavor distinct, is the emphasis on the role of teacher professionalism. The teachers shall be well trained, and it shall be their responsibility to organize teaching. Reciprocally they shall enjoy highest respect and occupy a high social position (Andere, 2020; Li and Dervin, 2018; Sahlberg, 2015; Tirri, 2014). However, we cannot ignore the fact that the authority Finnish teachers enjoy in society and the respect their students have for them is fostered by a cultural tradition that is not, or no longer, so pronounced in other Western countries. With regard to teacher authority, Simola (2005: 457–458) attested Finland an authoritarian, obedient and collectivist mentality which he sees rooted in the country’s history. In their 2017 analysis of the Finnish school system, Simola et al. (2017) still argue that teaching combines a “strong Finnish paternalistic pedagogical tradition with student-centered progressivism,” (p. 111) but acknowledge that teacher authority will need new foundations in late modernity (p. 120).
However, Finnish school policy has by no means remained unaffected by the global reform movement aiming at improving school quality and offering opportunities of choice for parents regarding schools and curricula (Kalalahti and Varjo, 2020). Nevertheless, the professionalism of teaching has remained at the heart of the reforms that have been implemented in Finland in the context of increasing international benchmarking since the late 1990s. Decision-making powers have been decentralized and brought down to the level of municipalities. However, reforms did not involve the turn to privatization, marketization, and managerialism as in Sweden (cf. Dovemark et al., 2018). With decentralization, there is greater diversity between municipalities and schools. This change is, however, intended to be kept within limits within which universal access to best possible education is still maintained. Now regular municipal evaluation of schools is expected to guarantee universalism. This new device of maintaining universalism is the Finnish interpretation of the global trend toward extended evaluation of schools (Kalalahti and Varjo, 2020: 33–35). Evaluation data are for administrative use and developmental purpose only, there is no national testing and there are no school ranking lists (Simola et al., 2013: 621–624). Thus, there is greater autonomy of schools, more emphasis on leadership and accountability according to hypothesis H1, but in the Finnish tradition of teacher professionalism and in a more moderate way than in Sweden. Schools are not directly sent into a competition for resources and high-performing pupils. Rather, the schools, which are still expected to be equally well equipped, are given more freedom in the implementation of nationally defined learning goals and in the organization of everyday school life and teaching. Even more has been invested in the training and regular further training of teachers (Paulsrud and Wermke, 2020; Salokangas et al., 2020).
Nevertheless, as a number of studies have revealed, schooling has moved away from the egalitarian ideal of comprehensive schooling for every child independent of his or her family background in Finland too (Ouakrim-Soivio et al., 2018). In the 1990s, school choice was increasingly considered a device of improving school performance, of providing for parents more opportunities to find the best school and the best curriculum for their child (Kivirauma et al., 2003; Seppänen et al., 2015; Varjo and Kalalahti, 2019; Varjo et al., 2014, 2018). With more autonomy, schools were enabled to supply special curricula and special subjects for special demand by parents. In this process, differentiation was enhanced according to the more or less demanding character of curricula. While in the early years of improved opportunities of choice, parents chose the school allocated to their child in their neighborhood (Poikolainen, 2012), they afterward have increasingly made a choice for another school or at least for a more demanding curriculum (Kosunen et al., 2020: 1467–1469). For example, choosing a curriculum with an additional language is a common strategy of better off parents to enroll their children in a school or at least in a class within a school with a differentiated curriculum which achieves a higher level of socioeconomic status and a higher level of performance. On the part of schools, offering special curricula focused on language, music, math or sports allows them to select students according to aptitude tests. Thus, in a more moderate way than Sweden, Finland has implemented reforms which at least partly meet the global reform agenda as expressed in our hypothesis H1: There is (1) enlarged autonomy and self-government of schools, but it is complemented in only a very limited way by (2) strong school leadership and (3) regular monitoring of teaching and student performance through tests and assessments.
What is the evidence provided by studies using country-specific data regarding hypothesis H1 and its opposite, hypothesis H2? In the wake of decentralization and increased school autonomy processes of selection and segregation have taken place in urban areas. There is increased segregation between schools and between classes within schools according to socioeconomic background and ability. Cultural capital is necessary for supporting students to succeed in more demanding curricula (Kosunen, 2014; Kosunen and Carrasco, 2016; Kosunen and Seppänen, 2015). In this way the egalitarian character of the comprehensive school system has undergone a transformation in the direction of more socioeconomic segregation, between or within school ability tracking and inequality in educational outcomes.
Bernelius (2013) shows for Helsinki that sociodemographic differentiation is accentuated by school choice insofar as better off areas perform better and worse off areas perform worse. Berisha and Seppänen (2017) observe at the city of Espoo that differences in performance between students occurs also between classes, namely between general classes and selective classes focused on certain more demanding subjects (see also Bernelius and Vaattovaara, 2016). This finding is further confirmed by Kosunen et al. (2020), also for Espoo, focusing on the choice of classes offering rare languages by parents who want to improve the educational profile of their children. Choosing a class with an additional language is a focused strategy of distinction of better off parents and contributes to the reproduction of socioeconomic inequality. Bernelius and Vilkama (2019) investigated the two-way relationship between urban residential mobility and catchment area segregation among the majority population in Helsinki. Their result is a systematic association between socio-spatial segregation and the differentiation of catchment areas. Disadvantaged areas fall increasingly behind the better off areas. Additionally, there is a striking association between school performance and house prizes (Harjunen et al., 2018). Furthermore, in the past, a very small immigrant population did not draw public attention to underachievement of immigrant students, but with the increased influx of immigrant people this neglect of educational inequality between native and immigrant students has changed in the meantime (Harju-Luukkainen and McElvany, 2018). Though school policy still aims at providing high quality schools for all children, residential choices of families strengthen socio-spatial inequality and subsequently also educational inequality. Because in Helsinki socio-spatial segregation is still lower than in metropolitan areas of other countries, for example the UK and Sweden, the effect is less strong here, but nevertheless visible (Burger, 2019; Harju-Luukkainen et al., 2020). Facing declining performance in international benchmarking and increasing inequality have led to intensified search for reasons and remedies (cf. Ouakrim-Soivio et al., 2018; Sahlberg, 2015; Sahlgren, 2015; Salmela-Aro and Chmielewski, 2019). In the end, the evidence provided by country-specific studies speaks more for hypothesis H2 than for hypothesis H1.
Improving performance and equity through school governance according to the global reform agenda? Multilevel linear regression analyses with PISA data
The studies so far reviewed prove that there is increasing inequality in student performance coming along with marketization reforms in Sweden and decentralization reforms in Finland. These studies do, however, not directly tell us whether the use of governance tools according to GERM as spelled out in hypothesis H1 and contradicted by hypothesis H2 makes a difference between schools. This is what we do in this second step of our empirical test of hypotheses H1 and H2. We look at the performance of both countries in the PISA tests of 2000, 2009, and 2015 (OECD, 2020). According to hypothesis H1, we should observe improvements in performance and in reducing achievement gaps over these years and differences between students according to the use of governance tools by their school indicating (1) enlarged autonomy and self-government of schools, complemented by (2) strong school leadership and (3) regular monitoring of teaching and student performance through tests and assessments. If we do not see such improvements, the opposite hypothesis H2 is confirmed.
Data and methods
For testing hypotheses H1 and H2 and find out differences between the countries at the three points in time, cross-sectional multilevel linear regression models are calculated for Sweden and Finland for the years 2000, 2009, and 2015 separately. The dependent variable is the PISA science score of students. Making use of PISA data, we have to address reliability questions (Blasius and Thiessen, 2015; Jerrim et al., 2017; Rutkowski and Rutkowski, 2016). For doing so, we conduct a number of tests and make use of data standardization. Our multi-level regression models feature two levels. The first addresses variables on student level, the second on school level. Snijders and Bosker’s (1994) R² and the Intraclass Correlation Coefficient (ICC) (Bartko, 1966) are applied to calculate the goodness of fit of our models. Snijders and Bosker’s R² allows us to decompose the variance explained by each model into variance on student level (Level 1) and school level (level 2). The ICC indicates the similarity of students at school level regarding their PISA science scores. The higher the values (ranging from 0 to 1), the more similar the students are within schools and the less similar between schools regarding their PISA science scores. Therefore, the ICC can be interpreted as a measure of social stratification of schools. In statistical terms, higher ICC values prove the adequacy of using multilevel mixed effects models. We apply z-standardization to improve the comparability of effect sizes. Additionally, we calculate student weights provided by the OECD to compensate for missing observations on student level. We conduct Shapiro-Francia W’ tests to account for the normality of the error terms assumption and the variance inflation factor (VIF) using Stata (version 14) to test for heteroscedasticity of our models. For descriptive statistics see Supplemental Appendix A, for tests Supplemental Appendix B.
We construct the models stepwise and start with a null model (ICC) showing the ICC, including the student- and school-specific intercepts of the Pisa science score only. Models one to three scrutinize the association between socioeconomic background and PISA science scores. Model one (student ESCS) includes the students’ socioeconomic status. PISA provides an index of economic, social and cultural status (ESCS). The “ESCS is a composite score built by indicators of parental education, highest parental occupation, and home possessions including books” (OECD, 2017: 339–340). For conducting a robustness check, we took “father’s occupational status” instead of ESCS as indicator of socioeconomic status in a separate regression analysis included in Supplemental Appendix D (cf. Avvisati, 2020; Jerrim and Micklewright, 2014; Rutkowski and Rutkowski, 2013). The second model (school ESCS) addresses the average of the students’ ESCS per school to investigate the link between socioeconomic status on school level and PISA science scores. The third model (student and school ESCS) comprises the measures of socioeconomic status on student and school level. The fourth model (migration background) looks at the impact of students’ migration background on the PISA science score. It entails the students’ migration status (second-generation versus no migration background), share of students with second generation migration background at the respective school and if the language spoken at home is the same as in the PISA test (“yes” as reference). The fifth model (disciplinary climate) investigates the association between school climate and PISA science scores. It contains variables such as the students’ rating of the disciplinary climate in science classes, whether student learning is not hindered at all by teacher or student absenteeism. Disciplinary climate on school level is based on items that measure whether students listen to the teacher or not, noisiness and disorderliness in class, the time it takes the students to be quiet and to participate in the lessons and the students’ working conditions (OECD, 2017: 314). The sixth model (governance tools) addresses a possible link between school governance and students’ PISA science scores. It involves indices of (1) school autonomy, (2) school leadership, and (3) school accountability, the three variables which are crucial for improving performance and reducing achievement gaps based on family background according to hypothesis H1. For their construction see Supplemental Appendix A. The seventh model (interaction effects) turns to variables related to socioeconomic status, migration background and school climate and adds interaction effects as control variables measuring interactions between the disciplinary climate in science classes and the students’ ECSC, the disciplinary climate and the school mean ECSC, as well as the ESCS on student level and ESCS on school level. The eight model (ESCS robustness under governance effects) aims at testing the robustness of the variables of student and school ESCS after including governance variables. The aim of the ninth model (full model of all variables) is testing the robustness of all variables hypothetically associated with PISA science scores, particularly the variables of student and school ESCS. The results for 2000, 2009, and 2015 are depicted in Tables 1 and 2. For the regression analyses see Supplemental Appendix C.
Sweden, results of multilevel regression.
Standardized beta coefficients +p < 0.10. *p < 0.05. **p < 0.01. ***p < 0.001; variables: first column shows beta coefficient of models focused on variable group, second column shows beta coefficient of full model.
Finland, results of multilevel regression.
Standardized beta coefficients +p < 0.10. *p < 0.05. **p < 0.01. ***p < 0.001; variables: first column shows beta coefficient of models focused on variable group, second column shows beta coefficient of full model.
Results
Looking at the results for Sweden, in this country, a major change toward increased inequality according to social background occurs between 2000 and 2009. This change takes place earlier than in Finland and with a stronger effect of student and school ESCS in 2000 to reach a much higher level in 2009, which remains stable in 2015 when Finland comes closer to Sweden with a much stronger effect of student and school ESCS than in 2009. The positive effect of the interaction between student and school ESCS in 2015 is too weak and caused by outliers to be interpretable. Student and school ESCS explain nearly all variance in the PISA science score. Adding the variables of migration, disciplinary climate and school governance in the full model increases explained variance only slightly, that is from 59.9% to 61.5% in 2009 and 51.8% to 61.7% in 2015. The difference between the two test years is the remarkably increased effect strength, explained variance and robustness of the school disciplinary climate. We observe that Sweden set out earlier on the way to increasing inequity than Finland. For explaining this difference, we may refer to earlier and more radical market-based reforms but also to higher rates of socioeconomically disadvantaged immigrant youth in Sweden. According to OECD statistics, in 2000, the percentage of students with a migrant background was 1.0% in Finland and 5.9% in Sweden. In 2009, it was 2.6% in Finland and 11.7% in Sweden, in 2015, 4.0% in Finland and 17.4% in Sweden (OECD, 2003: Table 6.8; 2010: Table II.4.1; 2016: Table I.7.1). Of the migration variables, however, only test language spoken at home shows a consistent and significant effect in all three test years. It is weak, though, and positive. In 2015, there is a negative R2 of the model including the three migration variables on school level, indicating that, together, they do not explain any variance in the data in this year. A negative R² indicates a wrong signed slope of the predicted regression line. One possible reason for this result is the variance decomposition in student and school level when differing slopes for each school are assumed but the data follows a trend that is uniform. The significant negative effects of disciplinary climate on student and school level in 2000 are too weak and are influenced by outliers in the data to be interpretable. In 2009, there is no significant effect of these variables. The strongly significant positive effect of the disciplinary climate on school level accompanied by a weakly significant positive effect of disciplinary climate on student level in 2015 tells that there is considerably increased segregation according to socioeconomic status and disciplinary climate in Swedish schools. Though there is a similar effect of these two factors in Finland in 2015, it is much stronger in Sweden. This difference between Sweden and Finland is confirmed by an increasing ICC in Sweden over time, again with the greatest growth from 2000 to 2009, while it remains on about the same low level in Finland throughout all years. That means, schools are more segregated according to performance in Sweden than in Finland. Most important, there is no significant positive effect of the three variables representing the global reform agenda of school governance worth mentioning throughout all years in Sweden as spelled out in hypothesis H1, that is school autonomy, strong leadership, and accountability measures. And they do not reduce the effect of student and school ESCS in any way as assumed by this hypothesis. In 2015, we even see a small significant negative and robust effect of strong leadership. This may speak for a reverse effect so that underperforming schools call for stronger leadership which does, however, not help to change the situation. Therefore, hypothesis H1 is refuted, while hypothesis H2 is confirmed. As postulated by hypothesis H2, socioeconomically advantaged students are in a better position to benefit from reforms focused on marketization, leadership, and accountability according to GERM than socioeconomically disadvantaged students.
Looking at the results for Finland, in this country, the effects of the student and school ESCS are smaller in terms of effect strength, explained variance and robustness in 2009 compared to 2000. In 2015, however, we observe remarkably increased effects of student and school ESCS in terms of effect strength, explained variance according to the R²-values and robustness. And the effect of the school ESCS has gained remarkably in all three measures to come close to the student ESCS, indicating an increased segregation between schools. With the student and school ESCS alone we almost achieve the same explained variance as in the full model including a whole set of further variables., that is 38.8% as compared to 41.4%. Thus, in the years from 2000 to 2009, Finland is the country with only a small significant and robust effect of the student ESCS and no robust effect of the school ESCS. In 2015, however, we observe a significant change in the direction of a remarkably stronger significant and robust effect of the student as well as the school ESCS. Finland has significantly lost in equity along with decreasing average scores in the PISA test. Nevertheless, there is no additional positive effect on PISA scores for students on a higher ESCS attending schools on a higher ESCS. Together with a low ICC, this speaks for still higher equity compared to Sweden. There is either a very small effect only or not any significant effect of other variables. The significant negative effect of school climate on student level in 2000 and the significant negative effect of the interaction between student and school ESCS in 2000 and 2009 are too weak, explain only small portions of the data according to the R²-values and are influenced by outliers in the data to be interpretable. Worth mentioning might be an increasing significant positive and robust effect of the test language spoken at home and of the disciplinary climate on the student level and a small significant negative and robust effect of a second-generation migrant background in 2015. However, as the negative R2 of the models on migration and disciplinary climate in 2015 shows, these variables do not explain any variance in the data. Most important, the three variables representing the global reform agenda of school governance according to hypothesis H1—school autonomy, leadership, and accountability—do not show a systematic and visible effect, and they reduce nothing of the overwhelming effects of the student and school ESCS. Apparently socioeconomically advantaged students benefit more than socioeconomically disadvantaged students. Hypothesis H1 is refuted, hypothesis H2 is confirmed.
Discussion and conclusion
Our literature review and our regression analyses of PISA data have not generated any evidence that the democratization and decentralization reforms entailing elements of marketization in Sweden and the decentralization reforms in Finland have led to any visible achievements. The market-driven and decentralization reform strategies do not change the overriding and even increasing effect of family background in the school system. Contrary to what is postulated by hypothesis H1, the GERM governance tools have not proved to be effective, neither in Sweden nor in Finland. The socioeconomic status of the school attended and of the student are crucial. We see mainly the reproduction and even growth of social inequality according to hypothesis H2 and additionally the disciplinary climate at work. Obviously, the influence of family background on educational achievement has increased again after the educational expansion has come to an end. Against the background of greater social inequality and greater differentiation through competition in the school system, this is more and earlier the case in Sweden than in Finland. But Finland has also moved in this direction. Better off families are in a better position to benefit from greater opportunities of school and curriculum choice and to support their children to achieve in the competition at school which has been intensified by the global reform agenda. Decreased equity in Finland might be due to a coincidence of three changes: extended use of school or curriculum choice by better off parents, increased heterogeneity of the population through immigration and decreased subjection to teacher authority. In line with the current state of research (see Chmielewski, 2014, 2019; Chmielewski and Reardon, 2016; Volante et al., 2019), our results offer further evidence for the continued or even increased influence of the socioeconomic status of the family on the educational achievement of children. And we eventually see that the turn away from professional autonomy and toward external accountability in terms of competition between schools, stronger school leadership and large-scale assessments does neither result in any improvement of educational outcomes nor in any reduction of the determination of educational performance through family background. The latter has even increased along with the implementation of governance tools according to GERM, in Sweden and Finland as well. Going beyond existing evidence, our analysis reveals that the application of GERM governance tools does not make a difference between schools.
The extent to which our results are robust must be shown by further studies that first work with other data, such as TIMSS and PIRLS, and secondly include other countries and other PISA test years as well as other structural, cultural, and governance-specific factors in the analysis. For further testing of hypothesis H2, strategies of parents and students to benefit from student-centered governance tools could be included, such as parents’ choice of school based on achievement data and their contacting of teachers, as well as students taking advantage of cream skimming by schools.
As there is no evidence left in PISA supporting the global reform movement, unintended effects become even more important for assessing the viability of this program (cf. Meyer and Benavot, 2013; Sellar and Lingard, 2014; Sjøberg, 2019). First, there is the unintended effect that the increased use of PISA data for educational benchmarking fuels competition between schools for achievement in standardized tests as major instrument for improving education and reducing achievement gaps. However, there is enough evidence that this agenda has so far failed. Evidence even tells that intensified competition between schools increases educational inequality instead of reducing it as it is claimed. Educational systems are caught in a vicious circle of growing economic inequality which leads to increasing educational inequality which in turn augments economic inequality. Secondly, the standardizing effect of standardized testing promoted by PISA speaks even more against this program. It produces standardized minds which are not the right device for enhancing critical thinking, creativity, and innovation. This effect calls for an open debate on the usefulness of standardized international educational benchmarking. Thus, rethinking the global reform agenda is our central policy advice.
Supplemental Material
sj-docx-1-eer-10.1177_14749041211069240 – Supplemental material for Equity lost: Sweden and Finland in the struggle for PISA scores
Supplemental material, sj-docx-1-eer-10.1177_14749041211069240 for Equity lost: Sweden and Finland in the struggle for PISA scores by Richard Muench, Oliver Wieczorek and Julian Dressler in European Educational Research Journal
Supplemental Material
sj-docx-2-eer-10.1177_14749041211069240 – Supplemental material for Equity lost: Sweden and Finland in the struggle for PISA scores
Supplemental material, sj-docx-2-eer-10.1177_14749041211069240 for Equity lost: Sweden and Finland in the struggle for PISA scores by Richard Muench, Oliver Wieczorek and Julian Dressler in European Educational Research Journal
Supplemental Material
sj-docx-3-eer-10.1177_14749041211069240 – Supplemental material for Equity lost: Sweden and Finland in the struggle for PISA scores
Supplemental material, sj-docx-3-eer-10.1177_14749041211069240 for Equity lost: Sweden and Finland in the struggle for PISA scores by Richard Muench, Oliver Wieczorek and Julian Dressler in European Educational Research Journal
Supplemental Material
sj-docx-4-eer-10.1177_14749041211069240 – Supplemental material for Equity lost: Sweden and Finland in the struggle for PISA scores
Supplemental material, sj-docx-4-eer-10.1177_14749041211069240 for Equity lost: Sweden and Finland in the struggle for PISA scores by Richard Muench, Oliver Wieczorek and Julian Dressler in European Educational Research Journal
Supplemental Material
sj-pdf-5-eer-10.1177_14749041211069240 – Supplemental material for Equity lost: Sweden and Finland in the struggle for PISA scores
Supplemental material, sj-pdf-5-eer-10.1177_14749041211069240 for Equity lost: Sweden and Finland in the struggle for PISA scores by Richard Muench, Oliver Wieczorek and Julian Dressler in European Educational Research Journal
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors received funding for the research from the Deutsche Forschungsgemeinschaft (German Research Foundation) under Grant MU 608/29-1.
Supplemental material
Supplemental material for this article is available online.
Author biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
