Abstract
The aim of this study was to characterize those schools in the Autonomous Community of the Basque Country (Spain) with high numbers of immigrant students in accordance with their effectiveness level (high or low). Three effectiveness criteria (scores, residuals, and times) were used to select the schools, resulting in three models: a ceiling or floor effect model, a contextualized cross-sectional model, and a contextualized longitudinal model. Of the 524 initial schools analyzed, 7 were found to have high or low effectiveness levels and higher-than-average immigration levels. Following the selection, the discourses of 40 education stakeholders were analyzed through semi-structured interviews and discussion groups. The results highlight the strong sense of vulnerability felt in some schools, as well as the general conviction that greater support, commitment, and stable, collaborative leadership by the management team and inspectorate would contribute to improving effectiveness.
Introduction
Equality, quality, effectiveness
Both the new Sustainable Development Goals established for 2030 (United Nations, 2015) and the Education World Forum (UNESCO, 2015) reiterate the need to guarantee inclusive and equitable quality education for all. Similarly, different reforms throughout the world have established, as their primary focus, the need reduce the achievement gap between students (Clark, 2014).
There is a large body of literature attesting to the fact that when education systems are less compressive and distribute students across different pathways, both individual and academic inequalities are increased (Benito et al., 2014; Liu et al., 2015). One thing that all substantially more egalitarian education systems have in common is that they strive to tackle social factors (housing, nutrition, healthcare) along with educational ones (Darling-Hammond, 2010, 2014). International research has shown that schools’ academic achievement is linked to quality, teacher qualification, available facilities and resources, type of school, and area in which they are located (rural, urban/east/west) (Clark, 2014).
Based on PISA data from 2015, the Organization for Economic Cooperation and Development (OECD, 2016) identified a close relationship between students’ outcomes, their individual ESCS (economic, social, and cultural status) and the ESCS of their school. In another report (OECD, 2018a), this same organization pointed out that, over the last decade, 48% of disadvantaged students living in OECD countries have attended disadvantaged schools (i.e. schools that are in the bottom 25% of the school-level ESCS index, which is calculated as the average ESCS index among students in a school), thereby effectively doubling their level of disadvantage. In more than a third of the countries participating in the PISA 2015 study, teachers at more disadvantaged schools were less qualified and had less experience than those teaching at more advantaged schools and that these latter schools were also better equipped (OECD, 2018b). This may be due to the fact that teaching conditions tend to be more challenging in disadvantaged schools, meaning that they find it hard to attract and retain the best teachers, giving rise to a higher teacher turnover rate, the recruitment of less-experienced teachers, and greater uncertainty in teacher assignments to schools (Bénabou et al., 2009). More experienced teachers choose more advantaged schools for a number of reasons, including more satisfactory working conditions, better climate for instructive work, and a stronger culture of collaboration (OECD, 2018a). Although most countries provide extra financial support for disadvantaged schools this does not seem to compensate for the lack of initial human resources (OECD, 2018a).
School segregation occurs on the basis of students’ socioeconomic and ethnic characteristics and mostly affects schools located in areas with limited resources (Granvik Saminathen et al., 2018). These schools suffer from the ‘white flight’ (Bonal et al., 2019) of students in more favorable socioeconomic and psychological situations, a circumstance that has been observed also in research carried out in California, Scandinavia, and Spain (Cascio & Lewis, 2012; Farré et al., 2015; Gerdes, 2013; Rangvid, 2010). The OECD (2019) has warned that the relationship between social and academic segregation and school-choice policies is a complex one, since school segregation is impacted by (among other factors) academic selection by the school, different pathways, grade retention rate, and residential segregation. In Barcelona, Bonal et al. (2019) observed how school segregation varied in neighborhoods with similar indexes of residential segregation and enrolment in public and private schools, concluding that it is also necessary to analyze the influence of local education markets. These same authors (2019) also warned that both educational factors (e.g. the presence of private schools) and non-educational ones (e.g. the socioeconomic characteristics of the neighborhood or residential segregation) may significantly affect how school choice possibilities impact families’ decisions, and how this in turn ultimately affects school segregation. When there is free school choice (i.e. when parents can decide which school to send their children to), natives tend to make more use of it than immigrants. In the Autonomous Community of the Basque Country (ACBC), it has been found that immigrant families tend to send their children to the school closest to them (Intxausti et al., 2010), whereas native and middle-class families take more advantage of free school choice by employing social distinction strategies (Bonal et al., 2019). One of the reasons for this in a context similar to ours is that middle-class families have the financial means to send their children to semi-private schools. Moreover, it is usually natives with higher education levels who tend to opt for private schools due to the increase in the number of immigrants in the public school system (Farre et al., 2018). Some studies have found that this in itself has resulted in a higher degree of school segregation (Bonal et al., 2019; Brunello & De-Paola, 2017).
In light of this situation, what can be done to ensure that schools with different socioeconomic and cultural profiles become equally effective? What are the keys to creating the right conditions for educational improvement? The school effectiveness and improvement movement aims precisely to respond to these questions.
Schools’ contribution to improving effectiveness
The school effectiveness and improvement movement has provided insight into how students’ learning outcomes interact with institutional, school, and classroom contexts (Reynolds et al., 2014; Scheerens, 2016). Within the movement, different explanatory frameworks have been suggested, encompassing different levels of analysis, such as: classroom, school (Creemers, 1994; Kyriakides et al., 2015; Muijs et al., 2014); district and education system (Creemers & Kyriakides, 2008; Stringfield & Slavin, 1992); as well as approaches integrating the perspective of change and improvement (Slater & Teddlie, 1992); and studies on school ineffectiveness (Abrantes et al., 2013; Hernández-Castilla et al., 2013).
In the Autonomous Community of the Basque Country (ACBC) (Spain), the school effectiveness movement has identified a set of best practices associated with school effectiveness within the fields of teacher training (Azpillaga et al., 2020; Lizasoain Hernández et al., 2016), attention to diversity (Intxausti et al., 2017), educational leadership (Intxausti et al., 2016), involvement of families (Azpillaga et al., 2014), and teaching method (Bartau Rojas et al., 2017).
The literature shows that most methodological progress has been achieved through the application of value-added measures (Ferrão & Couto, 2014; Ray et al., 2009). Value-added models refer to a set of complex statistical techniques that use test score data from students in various different academic years to estimate the effect of individual schools. The models attempt to isolate the contribution of the school itself to students’ learning development. Various different versions of the models have been developed (Gaviria & Castro, 2009). The multilevel modeling approach using linear hierarchical models respects the nested structure of the data and enables the effects of the variables to be studied jointly at each level (Gaviria & Castro, 2005; Raudenbush & Bryk, 2002; Snijders & Bosker, 2011). From a longitudinal perspective, the aim of mixed effect models with repeated measures is to assess added value (Bryk et al., 2010; Singer & Willett, 2003). Furthermore, the progress made in the field of change analysis (Collins & Sayer, 2001) and dynamic models (Kyriakides et al., 2010) has helped advance the movement as a whole. Over recent years, an increasing number of studies have focused on equity in educational outcomes (Nachbauer & Kyriakides, 2020; Scherer & Nilsen, 2019).
Diversity in schools and improving effectiveness
Among those studies focusing on highly-effective schools with different social, economic, and cultural profiles, of particular interest is the one by Bryk et al. (2010) on the results of the decentralizing reforms carried out in Chicago. The authors highlight the fact that school community factors influence the development and maintenance of key support elements, which in turn affect the likelihood of significant improvements being achieved in student outcomes. They refer specifically to one primary school and highlight the school’s exceptional leadership, close relations with organizations both inside and outside the community, the trust established between different stakeholders within the school community, the existence of a consistent, improvement-oriented program, broad-ranging support for professional development, and the concerted effort made to ensure that faculty are motivated to teach at the school.
Sampson (2011) carried out three longitudinal case studies in three low-ESCS schools from one region in the state of Texas (USA) that reported sustained improvement in student achievement. Aspects common to the three schools included a priority focus on students; regular, consistent formal and informal communications among educational stakeholders; and emphasis on hiring well-qualified teaching staff and ensuring they are strongly supported by the management team. This support takes the form of a firm commitment to obtaining financial resources to fund programs, recognition of the work carried out by teachers, a positive working relationship between the board of education and the district superintendent, and a commitment from both to regularly monitor and acknowledge efforts to improve student outcomes.
Few studies have attempted to link school effectiveness with high numbers of immigrant students. Granvik Saminathen et al. (2018) found that in schools with high numbers of immigrant students, and particularly in more segregated schools, teachers perceive a lower level of effectiveness (leadership, cooperation among teachers, school ethos).
Other qualitative studies report that, in schools with high numbers of immigrant students, it is important to focus on promoting different cultures, ensuring a good relationship with families and fostering student empowerment (Hajisoteriou et al., 2018; Simón-Rueda & Barrios-Fernández, 2019; Zirkel, 2008). Reynolds et al. (2014) highlight the importance of turning schools into communities of practice, along with the need, in certain areas, to find support from outside the school, such as additional resources for fostering innovation and change.
The research context: The Basque education system
The Autonomous Community of the Basque Country (ACBC) is a bilingual region located in Spain. Almost the entire system is supported by state funds, although there are two main types of school: public (approximately 53%) and semi-private (approximately 47%). Semi-private schools are private schools which were not set up by the government but which are nevertheless mainly funded by the state (Decree 293/1987).
Over recent years, public policies aimed at fostering attention to diversity have broadened the concept of diversity and generated a discourse in favor of educational inclusion (Basque Government [BG], 2012, 2016, 2019). As regards students from immigrant families, the most recent data indicate that 16.4% of primary students are non-natives (i.e. were either born abroad or are of foreign descent, with at least one foreign parent). If both place of birth and parents’ origin are taken into consideration, immigrant students account for 22.8% of all students in the public school system, whereas in the semi-private system, this percentage is just 9.6%. However, it should be noted that, together, 3.3% of schools in the semi-private system and 7.6% of those in the public system account for over 50% of foreign-origin students (BG, 2018). In these schools, the percentage of immigrant students does not always coincide with the general immigration rate registered for the local area. Based on PISA 2015 data, and taking only first-generation immigrant students into account, Murillo et al. (2017) found a high rate of school segregation by origin in the Autonomous Community of the Basque Country (0.54 on the Gorard index).
The Popular Legislative Initiative (PLI) for improving cohesion and equity in the education system, entitled Eskola Inklusiboa (Inclusive School) (Popular Legislative Initiative [PLI], 2018), which was presented in 2018 and rejected by the Basque Parliament, proposed measures for guaranteeing equity and the fair distribution of students across the different systems. The latest report on the schooling of immigrant students (BG, 2018) highlighted some of the progress made in this field. From a legislative perspective, important milestones include the new Student Admissions Decree (Decree 1/2018), which adapts the admissions criteria and the management of mid-term enrollments, and the new resolution outlining the composition, tasks, organizational rules, and functioning of the Schooling Commissions, in an attempt to unify criteria and ensure better monitoring. Various intercultural promotion programs and initiatives such as Bidelaguna (Traveling companion) and Hamaika Esku (Many hands), facilitate the assignment of additional staff and resources for improving conditions in disadvantaged areas and schools.
The education policies enacted over recent years fail to satisfy many sectors of the educational community, as evident in the presentation of the aforementioned PLI. This study aims to explore the discourses of the principal educational stakeholders (inspectorate, management teams, and teaching staff) working at schools in the ACBC selected for their high and low levels of effectiveness and high number of immigrant students. The results may shed some light on which school-related factors may help improve student outcomes in diverse contexts.
Aims
The general aim of the study was to explore the perceptions of educational stakeholders regarding the elements that may explain the outcomes obtained in various different Diagnostic Assessments (DA) by students at schools with high and low effectiveness levels and high percentages of immigrant students. The specific aims were:
To identify and characterize schools with high and low effectiveness levels in the ACBC, in order to then identify, within this group, those with high percentages of immigrant students.
To identify key problem areas which explain, in the opinion of the participants in the study, the outcomes obtained.
Methods
The study followed a descriptive-exploratory mixed-method research design (quantitative and qualitative).
Quantitative study
The aims were: (1) to identify primary schools with high and low effectiveness levels in the ACBC, based on three effectiveness criteria; and (2) to select those schools with a high percentage of immigrant students.
Selected schools
First of all, 23 of the 524 primary schools in the ACBC were pre-selected on the basis of the effectiveness criteria outlined below. Of these 23 pre-selected schools, 7 were selected for the final study, due to having a percentile of over 60 in relation to immigrant student numbers. Percentiles are numbers which divide a series of ordered data into 100 equal parts.
Instruments and data collection: Schools were identified on the basis of the scores obtained by students in grade 4 of primary school in the DAs carried out in 2009, 2010, 2011, 2013, and 2015 for three basic instrumental competencies: language communication in Basque, language communication in Spanish and mathematics. The results of the DAs were provided by the Basque Institute for Research and Evaluation in Education (ISEI-IVEI). A code system was used to guarantee confidentiality, and all data were translated into percentiles in order to facilitate understanding and enable comparisons between schools. To improve the accuracy of the estimates made, schools with classrooms containing fewer than 12 students were not included in the analysis (Nachbauer & Kyriakides, 2020).
Data analysis
The definition of effectiveness used in this study is based on two elements. The first is the excellence criterion, which refers to the need to provide all students with a customized education by ensuring the educational resources necessary to guarantee their full development (Aizpurua et al., 2016).
The second element is linked to the fact that previous studies have shown that educational activity depends, to a large extent, on socioeconomic and cultural context. Consequently, the present study also takes into account an analytical approach to effectiveness that enables a more accurate and equitable determination of the impact of individual schools (Murillo, 2005; Townsend & Avalos, 2007). Within this approach, effectiveness is analyzed by statistically controlling for the effect of covariates in order to isolate the impact of the school itself, which is considered an indicator of effectiveness. This enables a more equitable selection of schools. In statistical terms, the resulting score is called a residual or differential (the difference between the expected score and the one actually obtained by the school). This approach encompasses two models: the contextualized cross-sectional model and the contextualized longitudinal model, both of which are described below.
Three criteria were therefore used in this study to determine schools’ effectiveness levels: scores, residuals, and times, which together resulted in the generation of three models: (1) a ceiling or floor effect model; (2) a contextualized cross-sectional model; and (3) a contextualized longitudinal model. These three models were chosen because they take into account both equity and excellence, based on the assumption that in a high-quality education system, these two aspects should not be opposing categories, but rather can and should be enhanced simultaneously through processes of continuous improvement (Lizasoain Hernández, 2020).
Criterion 1 (ceiling or floor effect model): This model takes the gross mean scores from each school as an indicator of its effectiveness, taking into account those who could do no better (ceiling effect) or no worse (floor effect) in terms of their mean scores in all three competencies across all 5 years. It indicates that all students obtained scores around a high (or low) value.
Criterion 2 (contextualized cross-sectional model): A school is deemed to have a high (or low) level of effectiveness if the mean of its differential is very high (or very low). Before calculating the residual, to avoid possible errors or collinearity effects between the independent variables (L1 and L2), we performed a new regression analysis eliminating from the model, one by one, those variables not found to be statistically significant, starting with the variable with the highest significance score (p) and continuing on to that with the lowest significance score. The resulting model therefore only contained those variables with a significance score of less than .01 (p < .01).
The scores obtained by students (L1) in the study variables were used, along with those obtained by the schools (L2) to which said students belonged. The strictly contextual covariates at the student level were as follows: gender, previous performance, family’s ESCS (economic, social, and cultural status), being an immigrant (or not), native language, and language model. L2 included the system to which the school belonged (public or semi-private) and the aggregated L1 variables.
Thus, the multilevel multiple regression model pertaining to any of the dependent variables and for each level was as indicates in Table 1:
Regression model.
The different elements were:
Yij: score obtained independently (i.e. not in a multivariate manner) in each of the different competencies by student i from school j;
β0j: the mean score obtained by each school in the corresponding competency;
βqj: the linear influence of the student covariate Xq;
Xqij: the score obtained by student i from school j in the covariate Xq;
rij: the differential or residual for student i from school j (i.e. when each student is separated from what is expected or estimated on the basis of the equation in their school).
The level 2 model included factors which systematically affected the school. These were labeled Ws, with s = {1. . .S}.
Here, β0j is the mean performance of each school j;
γ00 is the effect common to all schools;
γ0s is the linear effect of the covariate Ws on the mean performance of the schools;
Wsj is the value taken by school j in the covariate of school Ws.
The term u0j represents the differential of each school after controlling for all the individual and school factors included in the model. It constitutes the basis for one of the statistical selection criteria, in the sense that a school whose mean residual (across all three competencies and all five DAs) is very high is considered highly-effective, since its mean performance is better than what could statistically be expected once the effect of all contextual variables has been controlled for.
Criterion 3 (contextualized longitudinal model): A school is deemed to have a high (or low) level of effectiveness if its differential indicates a marked upwards (or downwards) trend across the five measures.
This is a model which focuses on change. It is obviously not a longitudinal study of students in the strict sense of the term, since the DAs were administered to different cohorts, but it is possible to adopt a time-based approach in relation to schools, since outcomes from five different time points are available. Direct time comparisons are possible thanks to the fact that all DAs used the same metric (a mean of 250 and a standard deviation of 50) and the tests have anchor items that enable comparability. A similar approach was used in a study by Spoth et al. (2004), albeit in the field of clinical psychology.
The contextualized longitudinal model, as shown in Table 2, could only be calculated for schools, since the DAs provide cross-sectional results for cohorts from year 4 primary. To this end, the multilevel model encompassed times at level 1 and schools at level 2. Hierarchical regression models were also established, in which level 1 represented the year of the assessment (t: YEAR, with t ∈ {2009, 2010, 2011, 2013, 2015}), and level 2 the school (j ∈ J: generic school, with J = {schools}). The dependent variable P was either the percentile corresponding to the mean raw score (PS) or the percentile corresponding to the residual or differential score (PR) of the schools in the different competencies, since it was necessary to equate the mean scores obtained by the schools in each DA and each competency assessed. It was calculated using two parameters: π0j and π1j. The notation here sometimes varies and the letter pi is used, but these parameters are the same as those used in the previous models. Thus:
B00 is the main mean (intersection) of all schools in terms of raw or residual/differential scores at the start of the study period (year 0);
B10 is the mean slope for all schools;
U0j is the residual for each school relative to the intersection;
U1j is the annual increase in slope for each school. In other words, it is the amount by which each school’s slope (π1j) differs from the main slope (i.e. the mean slope of all the schools: B10). It is therefore an indicator of effectiveness, similarly to U0j in the case of the second model (extreme differentials). In sum:
Longitudinal models.
A well-ordered list of schools was established, thereby enabling the identification of those with very high and very low levels of effectiveness in all five DAs and across all three competencies, in accordance with three criteria explained above.
Qualitative study
The second phase of the study followed an exploratory and qualitative-interpretative research design.
Participants
The term ‘stakeholders’ is used here to refer to all professionals who are involved, either directly or indirectly, in the teaching-learning process. The following stakeholders participated in the present study: key inspectors at the selected schools (external support staff who advice schools), management teams, and a voluntary group of teachers working in the educational stage that was the object of the analysis. As mentioned earlier, the final sample comprised seven schools, meaning that participants in this phase were 7 inspectors 1 , 7 management teams, and 26 teachers.
Techniques and instruments
Two techniques were used for the data collection: semi-structured interviews (with inspectors and members of the management teams) and discussion groups (with teachers). Instruments specially adapted to each group of participants were designed with the aim of collecting data about nine different areas.
Data collection
A common protocol was established for the interviews. Informants were sent an introductory letter beforehand and once they had given their consent and authorization, interviews were held and audio recorded during 2016 and 2017. When setting up the discussion groups, priority was given to the voluntary nature of the participation and the position held by potential informants. The measures adopted were the same in all cases and the sessions were held during 2018.
Data analysis
This study forms part of a broader research project analyzing a wide range of aspects. This paper focuses only on the data pertaining to schools with the highest numbers of immigrant students. All interviews and discussion groups were transcribed and entered into the NVivo 10 computer program in accordance with informant type (inspectors, management, and teachers) and node. In order to fulfill the aims of the study, the analysis followed an inductive procedure, using an emerging system of categories. For the content analysis, new nodes were established in NVivo, based on the comments made by participants regarding different aspects linked to the research aims and which, in the analyst’s opinion, corresponded to a specific type of content. The definitive result comprised eight categories, outlined in Table 3. The contents included in each category were then drafted and the information reorganized in accordance with the study aims. Subsequently, a coding matrix query was run to obtain the total number or categorization frequency of the emerging nodes. A coding comparison query was run also to verify the inter-rater agreement index (k = 0.96).
Aim 1: To identify and characterize schools with high and low effectiveness levels in the ACBC, in order to then identify, within this group, those with high percentages of immigrant students.
Categories and descriptions.
Seven primary schools were identified in the Autonomous Community of the Basque Country with very high or very low levels of effectiveness and a percentage of immigrant students 2 over the 60th percentile (33%; 26%; 17%; 15%; 7%; 6%; 5%). As shown in Figure 1, the mean differential of these schools (shown as red triangles) ranged from positive (12.47; 8.13; 1.19) to negative (−1.15; −1.25; −8.13; −8.03).

Schools selected (shown as triangles) on the basis of their mean differential in the three competencies and percentage of immigrant students.
Next, from a longitudinal perspective, a more detailed approximation of the real situation in these schools was carried out, in order to better understand the trends observed in their differentials. All data were converted to percentiles in order to make them easier to interpret and to show the similarities and differences more clearly.
Criterion 1 (ceiling or floor effect model)
Very high raw score
No schools were identified with both very high raw scores and high immigration.
Very low raw score
One school was identified in this category. School 30052 was a small/medium-sized public school (mean percentile 39). The percentage of immigrant students (93.6) was very high and the family ESCS was very low (0.8).
As shown in Table 4, the raw scores obtained by school 30052 remained stable and very low across all five DAs, and its differentials increased in 2010, 2011, 2015. This indicates that it scored extremely low and well beneath that which could reasonably be expected given its context.
Scores and differentials of schools with very low raw scores (in percentile terms).
Criterion 2 (contextualized cross-sectional model)
Very high differential
Two semi-private schools were identified in this category. School 33000 was a small school (8.4) with a high percentage of immigrant students (97.2) and a low family ESCS (20); 30316 was a medium-sized school (60.8) with a moderate percentage of immigrant students (56) and a medium family ESCS (51.4).
As shown in Table 5, both schools obtained high differentials in the means for the three competencies measured (Basque, Spanish, and Mathematics) in the five DAs. School 33000 obtained very high differentials from the third year onwards (between the 100th and 93rd percentile); whereas the differentials obtained by school 30316 were more stable, oscillating between the 98th and 76th percentiles. This indicates that both schools scored very highly, above what could be expected in light of their contexts.
Scores and differentials of schools with very high differentials (in percentile terms).
Very low differential
No schools were identified with both very low differentials and high immigration.
Criterion 3 (contextualized longitudinal model)
Differential with an upwards trend
Two schools were identified in this category. School 25080 was a small public school (mean percentile 21.8) with a high percentage of immigrant students (81.6) and a very low family ESCS (7). For its part, school 27698 was a medium-size semi-private school (65.6) with a medium-high percentage of immigrant students (61.2) and a medium family ESCS (51.8).
Table 6 shows the upwards trend in the differentials of schools 25080 and 27698. This indicates that both schools managed to improve their performance/situation, since in the DAs, from 2011 (school 27698) and 2013 onwards (school 25080), they scored similar to or higher than that which was to be expected.
Scores and differentials of schools with an upwards trend in their differentials (in percentile terms).
Differential with a downwards trend
Two schools were identified in this criterion. School 29942 was a medium-sized semi-private school (mean percentile 50) with a medium-high percentage of immigrant students (57.4) and low family ESCS (24.8). School 37224 was medium-large public school (69.2), with high rates of immigrant students (82.4) and a medium-low family ESCS (39.8).
Table 7 shows the downwards trend in the differentials of schools 29942 and 37224. This indicates that the performance/situation of both schools worsened over time, since in the DAs, from 2010 (school 37224) and 2011 onwards (school 29942), they scored similar to or lower than that which was to be expected.
Aim 2: To identify key problem elements which explain, in the opinion of participating informants, the outcomes obtained
Scores and differentials of schools with an downwards trend in their differentials (in percentile terms).
The following eight sections outline the results linked to the second study aim.
Identity(ies)
In schools with very high differentials and those with an upwards trend, participants’ discourse on school identity was consistent and associated with their religious or cultural identity. A high level of engagement and commitment by teachers at these schools was observed, although a certain reluctance to assume responsibility was also evident.
We are all members of staff, and there’s a feeling of ‘all for one’, even when we criticize. . . For me, this unity in all matters is vital (. . .) I believe we have a common vision and this helps a lot. (Management team, School 27698)
The analysis also revealed other, less cohesive situations. These schools form part of the Hamaika Esku program, which provides help and resources to disadvantaged schools, and are therefore engaged in an effort to unite staff around the school’s shared project. The vision shared by inspectors and management staff was seen as a lever for initiating the improvement process. Management teams said they felt protected and supported in their attempts to bring about change.
The fact that all the factors come together, that the administration and all of us are pulling in the same direction (. . .) seems to me to be a golden opportunity which should not be wasted. (Management team, School 30052)
In public schools, the constant turnover of staff results in many adaption and learning periods, thereby hampering the development of a shared vision. This situation takes its toll on both the management team and permanent members of staff, and has a negative effect on students and their families also, who feel insecure in the face of teachers’ constant comings and goings. In those schools participating in the Hamaika Esku program, measures linked to the secondment of staff members were welcomed because they enabled a greater degree of stability.
Teachers’ attitudes and the readjustment of their expectations
The core idea of educational equity was present in the majority of the schools analyzed. Informants were clear in their rejection of social prejudices against immigrant students. As regards the readjustment of expectations in relation to new students, in schools with very high differentials, a well-developed, consistent, positive discourse was identified regarding what teachers expected from their students.
We are aware that our students have to be competent. That’s the first thing. And of course, when completing a diagnostic test, they should feel that they can do it, not that they can’t. (Management team, School 33000)
Schools with a downwards trend in their differentials were in a process of transition, learning to accept and adapt to the new situation, and to modify their expectations accordingly. New family situations were mentioned as an added difficulty requiring a concerted effort and often resulting in a sense of discouragement and low expectations.
The results reveal that it is possible for professionals to change their view of their own performance, and this may help them change the way they see their students and their families also. Furthermore, informants highlighted the importance of the support provided by inspectors in the effort to accept this new situation.
Uniting people around a school-wide project
At the time of the study, two schools were trying to unite teachers around a project led by the management team and Education Inspectorate. One of the main challenges to be overcome in this effort was the wide range of different ways teachers had of understanding educational goals, and the diversity of the factors motivating them. These differences disrupt the working dynamics and have, on occasions, halted, or at least hampered, attempts to bring about improvement.
So there’s no school-wide vision. There’s no common goal; just a lot of individual outlooks. It all depends on the teacher and the group; we will achieve some results this year, but you can’t build a school overnight. (Inspector, School 30052)
In schools whose differentials revealed a downwards trend, a consistent discourse was identified in relation to the difficulties faced. Informants attributed the poor results obtained to the characteristics of students and their families, as well as to a lack of effective policies and resources.
Well-aligned leadership styles
Another factor valued by informants was the help and support received by teachers from the management team, and said team’s proactive attitude to resolving problems.
It is important to have a management team that keeps its doors open and is willing to listen to your needs, because, as a teacher, you face some very serious situations in your everyday dealings with the kids. (Teacher, School 30052)
In two of the schools participating in the Hamaika Esku program, plans had been made to ensure a stable management team in the short-medium term, in order to avoid high levels of staff turnover in management positions, which is a clear impediment to change. In these schools, the collaborative network established among different stakeholders made it easier to agree on certain intervention elements, which in turn enabled better monitoring of and support for the schools’ actions, along with a change in the way the schools viewed inspectors – going from seeing them as mere controllers to valuing them as allies.
I had always thought of the inspectorate as an entity that never did what it was supposed to do (. . .) right now, the feeling I have is that we are all pulling in the same direction, we all want the same thing, and this really sets my mind at rest; it gives me a sense of security. (Management team, School 30052)
One determining factor in this sense, according to members of several management teams, is the management function itself, which is perceived as complex and exhausting, involving the handling of many daily occurrences requiring both commitment and determination. The management team and staff at one school with a downtrend in its differentials complained about feeling vulnerable and unsupported. They considered available resources and the support provided to teachers to be scarce and insufficient.
We’re completely alone; we have the feeling that no one takes any notice of us. They come and ask what we need, and we beg - please, SOS, help us - but we never get anything. (Teacher, School 37224)
Resolution-oriented management and organization
In the opinion of the management teams of schools implementing a quality system, this is a factor which explains the improvement observed in the results. A quality system helps define a framework for the functioning of the school, enabling a better systematization and organization of tasks and roles. It also fosters a more distributed leadership style. On the other hand, however, it substantially increases teachers’ workloads. These schools demonstrated a good degree of management maturity and had proven themselves capable of adapting existing instruments to their specific set of needs, including the management of training processes.
Most of the other schools had incorporated effective changes in relation to basic coordination, such as, for instance, coordination in accordance with knowledge field, as well as between different educational stages. However, informants also said they thought that the content and systematization of coordination meetings should be improved.
Conflict resolution and relations with families: priority areas
Informants acknowledged that ensuring effective conflict resolution was a priority aim. The measures implemented were all designed to resolve conflicts at different levels: teachers and staff, families, and students. Examples included training in conflict resolution and group therapy, the drafting of a common action protocol for the entire educational community, and working with students to help prevent conflicts, as well as to establish corrective measures based on consensus.
In two schools involved in the Hamaika Esku program, interventions with families were a priority, since there was a growing awareness of the importance of involving families in the school. Although there was already a small group of very involved and engaged families, they believed it was necessary to reach out to more of them and therefore diversified their initiatives to include more informal, relaxed meetings, and encouraged greater family participation in the classroom, particularly in preschool.
Conflicts were a recurring problem in schools with a downwards trend in their differentials. Even though they had adopted certain conflict resolution measures, such as group dynamics during tutorials, their results remained unsatisfactory. Teachers referred to a lack of support and protection and mainly attributed the low level of family participation to cultural and language differences, as well as to continuous changes of address. They said that training and translation resources were ineffective.
In schools with very high differentials, the commitment to learning about and understanding families’ needs and demands was evident, as were their efforts to satisfy them. The desire to be there for families was evident among the professionals working at these schools, and this was fostered in turn by the small size of the schools and their family atmosphere. Furthermore, an awareness of the fact that the school’s actions have a positive impact on family engagement leads to increasingly favorable attitudes.
Families? Well, mainly you look out for those that may have problems; you try to be there for them (. . .) I think that all teachers here are clear about that - about the fact that being there for families is more important than following a specific math method, for example. (Management team, School 30316)
How to organize effective resources and support?
The majority of the schools analyzed recognized the need for flexibility and ongoing revision, meaning that resources were constantly reorganized. Informants talked about how difficult it was to provide effective inclusive support to such a diverse body of students, many of whom have very different academic levels.
A concerted effort was being made to coordinate human and functional resources within the schools, but informants said it was not easy due to difficulties overcoming traditional roles. One strategy mentioned was placing internal resources at the service of those who most need them, such as, for example, including recreational activities in the curriculum or opening the library outside school hours. Another key element was informants’ feeling of vulnerability, or in other words, their perceived lack of security when dealing with difficult situations arising from the disruptive behavior of certain students, and their request for specialist therapeutic aid.
If dance or choir singing, for instance, were offered as extracurricular activities, no one would come. So the management decided to include them in the curriculum. (Management team, School 33000)
Schools also looked for external resources in the local community, working with volunteers (retired teachers, associations, families, etc.), coordinating with different stakeholders and participating in community networks. Nevertheless, informants said that organizing this and channeling the good intentions of those willing to contribute something to the school involved a lot of hard work. They were somewhat skeptical about the psychological support provided by social services, saying that it was not particularly effective and had a limited, short-term outlook.
Some informants claimed to feel hurt when talking about their relationship with adjacent municipalities/districts, saying that a very negative, even discriminatory image of their schools had grown up and taken root in their local areas, and that it was very difficult for them to redress this situation.
Teaching and learning processes: Overcoming reticence and consolidating new methods
Cooperative learning and project-based methods were common practice in the majority of the schools analyzed, although they were still in the design or experimentation phase, with their use not having yet been consolidated. Competence-based assessment had yet to be implemented and was viewed as being costly.
Teachers from schools involved in the Hamaika Esku improvement project had internalized the importance of improving student outcomes and were working to strengthen methodological interventions with the help of the Education Inspectorate, with special focus on preschool. Informants also highlighted new training initiatives led by the Berritzegune councilor, aimed at encouraging teachers to think about their own practice. Some participants also pointed out that the Diagnostic Assessments had triggered a phase of strategic reflection, and had encouraged them to start transforming their teaching-learning processes. However, they were also critical of the highly decontextualized way in which the DAs are carried out, claiming that they provide no relevant data that cannot be gathered through internal assessments. Nor did they positively rate the improvement plans in place, which they viewed as ineffective.
Discussion and conclusions
The results presented here have several clear implications for the study of school effectiveness and improvement. The first is linked to the three models used for selecting schools in accordance with their effectiveness. The longitudinal model is particularly relevant for school improvement studies, since it helps identify the factors which may impact changes in school outcomes, both for better and for worse. The longitudinal model made it possible to identify not only whether the schools in question had high or low differentials (contextualized cross-sectional model) or ceiling or floor effect outcomes, but also whether these differentials decreased or increased over time. This in turn ensured the inclusion in the study of those schools with initially low differentials which were nevertheless improving, thereby guaranteeing that it was the effort being made that was taken into account, regardless of the starting point. This perspective helps add a greater degree of nuance to the findings reported in some previous studies (Betts & Fairlie, 2003; Conlon & Kimenyi, 1991), which argue that schools with high numbers of immigrant students tend to be low-quality teaching centers.
In two of the schools selected, one for the increase in its differentials and the other due to the floor effect (very low raw scores), more systematic interventions were identified as one outcome of their participation in the Hamaika Esku program, which is led by the Basque Government and is targeted at schools in disadvantaged circumstances. The discourses of informants from these schools focused mainly on the changes that had been introduced. Both were fairly divided, with a low level of cohesion. It has been found that cohesion and consistency around a single discourse linked to the educational project, coupled with high levels of commitment and engagement, pave the way for the establishment of different types of improvements (Creemers & Reezigt, 1999; Fullan & Quinn, 2016).
It is important to note that external aid in the form of support and advice (from the Education Inspectorate and consultancy services) may help create conditions that are more conducive to improvement. Some schools in the sample were reluctant to accept all the responsibility for their results in the DAs, claiming that the Education Authorities should also be held accountable and should be involved in improvement processes, since they have an obligation to provide better support and stability, something which Elmore (2000, p. 21) described as ‘reciprocity of accountability and capacity’. In other words, the Education Authorities have a responsibility to try and ensure greater teacher retention, as indeed has been pointed out by other authors (Loeb et al., 2012; Simon, 2015). Informants’ discourses also highlighted the importance of collaborative leadership, with the management team and inspectors working together and sharing responsibility (Bryk et al., 2010); they also referred to the advantage of these efforts being framed within programs such as Hamaika Esku, which support schools and provide them with resources and advice, and are designed in accordance with a medium-to-long-term vision. This is one way to begin building professional learning communities (Ainscow, 2005; Fullan, 2019; Hargreaves & Fullan, 2014) and start consolidating the three components of professional capital: human, social, and decisional capital. But leadership must be ‘nested and ecological’ (Bolivar, 2014, p. 32), and it is therefore necessary to generate professional capital among teachers, schools and district superintendents, as well as among national and international organizations (Hargreaves & Fullan, 2014). The results reported here are consistent with the proposal made by Fullan (2019) regarding the importance of consensus-based, well-aligned and collaborative interventions involving schools, municipalities, and the administration, all working to improve consistency at each level in order to overcome the feeling of vulnerability expressed by some schools, and to improve and optimize professional and school-wide contexts and conditions.
The results of our study also point to another kind of school in our sample – semi-private schools with a high degree of stability among the teaching staff – which explains the outcomes achieved in terms of the organizational models employed; in other words, the results are due to the quality management models implemented and the school’s strong focus on being there for and helping families. These schools also have a more mature discourse regarding what expectations they have of their students. The results reveal that, as indeed reported by previous studies (Wadhwa, 2015; Zirkel, 2008), it is possible for professionals to change their view of their own performance, and this may help them change the way they see the students and families with whom they work. These schools managed to generate trust with families (Bryk & Schneider, 2002) by being there for and paying attention to them, as well as through specific measures. No feelings of vulnerability were detected in the discourses of informants from these schools, nor any demand for greater support from the Education Authorities.
Another factor valued by informants from all schools was the help and support received by teachers from the management team, and said team’s proactive attitude to resolving problems. Collaboration with the wider community is also becoming increasingly common (Hajisoteriou et al., 2018). The negative image generated about them in neighboring municipalities and districts affected almost all the schools studied, which suggests that one area of improvement may be to focus on developing multi-sector policies in collaboration with other socio-educational and health teams and associations, aimed at avoiding school segregation.
Limitations and future research
When selecting the schools included in this study, only data pertaining to the Diagnostic Assessments were taken into consideration, specifically those relating to core competencies (mathematics, language communication in Spanish, and language communication in Basque). The approach is therefore limited and future studies may wish to include other learning outcomes also (Nachbauer & Kyriakides, 2020). Furthermore, effectiveness was measured here in terms of mean scores in all three competencies, and future studies may wish to analyze each competency individually. Finally, participation in the teacher discussion groups was voluntary, and not all schools were represented.
The present study is a starting point for a more in-depth exploration of the equity dimension of school effectiveness and improvement. The three models used to select the schools enabled us to obtain a sample of schools with diverse socioeconomic and cultural characteristics. Future studies may wish to use these models to analyze equity from the gender perspective.
Footnotes
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was conducted within the framework of the R&D Project entitled ‘Longitudinal, contextualized study of schools with very high and very low levels of effectiveness. Design of school improvement actions’ funded by the Spanish Secretary of State for Research, Development and Innovation at the Ministry of the Economy and Competitiveness, as part of the State Program for Fostering Excellence in Scientific and Technical Research, with reference EDU2014-53511-P. Principal researcher: Luis Lizasoain Hernández.
