School improvement plans and student achievement: Preliminary evidence from the Quality and Merit Project in Italy

Abstract

This study provides preliminary evidence from an Italian in-service training program addressed to lower secondary school teachers which supports school improvement plans (SIPs). It aims at exploring the association between characteristics/contents of SIPs and student improvement in math achievement. Pre-post standardized tests and text analysis of SIPs were performed regarding a sample of 248 schools (13,816 students) participating in the project. Results show differences in planning strategies: better school improvements are associated with the ability to carry out a careful analysis of the context, prioritize elements in the diagnostic phase of the process and detect specific improvement goals.

Keywords

Improvement plan school improvement student achievement

Introduction

This article provides preliminary evidence from the Quality and Merit Project (PQM PON), an Italian in-service training program addressed to lower secondary school teachers which supports school improvement planning.

Many national and local projects focus on the improvement of student achievement, based on the capacity of schools to transform themselves into supportive environments for teacher learning and change. In this regard, high-performing school systems have shown three core competences (Curtis & City, 2009): a deep understanding of the core business of facilitating learning; a theory of action for improving instruction, through a concrete vision and an effective line-up of resources; the strategies to stimulate self-assessment in key areas of competence and to build capacity at different levels and stages of development. Indeed, site-based interventions to improve student learning outcomes, focused on needs assessments and school improvement plans, often called ‘comprehensive school reform’ have been hallmarks of school reform in several countries (Fernandez, 2011; Lockheed, Harris, & Jayasundera, 2010). The purpose of these interventions is to provide financial incentives for schools to develop comprehensive school reforms, based upon scientifically based research and effective practices which include an emphasis on basic academics and parental involvement so that all children can meet challenging state academic content and academic achievement standards.

In line with the dynamic model of the educational effectiveness, schools which are able to recognize their weaknesses and take actions to improve their policy on aspects associated with teaching and their School Learning Environment (SLE) can improve their effectiveness status (Creemers & Kyriakides, 2010, 2012). Indeed, research has shown that effective school improvement requires school-level processes (Reezigt & Creemers, 2005), and teachers are considered an essential lever of change. At the school level, the research in the Effective School Improvement (ESI) Project (Reezigt, 2001) identifies three key elements: improvement culture, processes and outcomes. The cycle of improvement processes expects five factors/stages: assessment of improvement needs, diagnosis of improvement needs and setting of detailed goals, planning of improvement activities, implementation/evaluation and reflection. In this sense, schools can play a substantial role in supporting also teacher learning by creating continuous learning opportunities, promoting inquiry and dialogue, encouraging collaboration and team learning, and establishing systems to capture and share learning, in order to promote change as a result of this learning (Opfer, Pedder, & Lavicza, 2011). Participative decision-making, teaming, teacher collaboration, an open and trustful climate, cultures which value shared responsibilities, values and tasks, and transformational leadership practices can foster teachers’ professional learning in schools (Thoonen, Sleegers, Oort, & Peetsma, 2012).

Research shows that school improvement planning may help guide leaders make effective and efficient policy decisions (Hummel-Rossi & Ashdown, 2002). Indeed, it can improve the quality of the educational setting by increasing the efficiency of service delivery through various management techniques (Wong, 2003), because efficiency gains reduce costs, freeing up resources to increase services and performance. In addition, school improvement planning can promote an ongoing organizational learning because it aims at reflecting on current conditions and past practices (Huber, 1991; Levitt & March, 1988). The entire school community is a learning organization (Hayes, Christie, Mills, & Lingard, 2004) and the educational staff must learn to learn so that organizations can adapt to an ever changing environment and to deal with new challenges (Schechter, Sykes, & Rosenfeld, 2004). Schools can be thus seen as organizations capable of responding to internal and external stimuli and learning new educational and managerial techniques (Kruse, 2001). In this sense, planning can be used to help guide school improvement reforms efforts and adapt to new challenges (Beach & Lindahl, 2004).

With regard to the effectiveness of school improvement planning, there is scarce evidence that such ‘new public management’ strategies really work (Fernandez, 2011) and only few studies show a statistically significant impact (Armstrong, 1982). Indeed, a general weaknesses of prior studies on the effectiveness of planning is detected, mainly concerning a small or modest sample size which produce insufficient statistical power to explore the complex relationship between planning and performance (Phillips & Moutinho, 2000; Ramanujam, Venkatraman, & Camillus, 1986). In addition, some argue that constantly reinventing management or strategic plans can waste valuable time and resources (Bobic & Davis, 2003) and that ‘planning requirements often have the unintended effect of overloading teachers and administrators’ (Levine & Leibert, 1987, p. 398). In particular, Bell (2002) states that frequently organizational strategies or activities are not a rational response to the school’s environment and that ‘in most circumstances there are only a very limited number of options available to staff in schools’ (p. 416).

The School Improvement Plan (SIP)

Today schools are asked to become much more data-driven and results-oriented (Kelly & Lazotte, 2003), moreover the lowest performing ones. Strategic planning can assist in this endeavor and has been considered a key management tool to help bring together the different actors within an organization to assess problems and articulate and achieve goals (Preedy, Glatter, & Levaçic, 1997).

The development of a School Improvement Plan (from now on, SIP) is an integral part of every successful ongoing individual school improvement effort (Doud, 1995). It is considered as the accepted best practice for school-wide improvement, and the perceptions of the purpose and value of the process significantly affect its success (Dunaway, Kim, & Szad, 2012). The purpose of strategic planning is to design a plan to help determine where an organization wants to go, what is needed to get there, and how to know if it got there (McNamara, 2003). Since the ultimate objective of school improvement planning is to improve the level of student achievement, the person who has the greatest impact on students during the school time, the teacher, plays a critical role in the school improvement planning process. In this sense teachers should actively participate and assume leadership roles in establishing priorities, setting goals, and formulating implementation strategies for the plan, developing strategies for improving the level of student achievement, supporting the evaluation of the plan and pursuing professional development focused on the goals and strategies identified in the plan (Education Improvement Commission, 2000).

There is no one definition or model for a SIP. Each state provides SIP guides and templates to assist schools in preparing school improvement plans and many state have support teams or specialists to help give advice to schools on the planning process. Although the guides and templates vary greatly across states, SIPs share several commonalities and are frequently described in a similar fashion where staff analyze problems, identify underlying causes, establish measurable goals, incorporate strategies and adopt policies that directly address the problems, and monitor implementation (US Department of Education, 2006). For instance, the SIP should directly address the problems which caused the school to be identified for school improvement, incorporate improvement strategies based on scientific research, establish specific and measurable objectives for progress and improvement, identify who is responsible for implementation of strategies and include strategies to promote professional development and parental involvement (Fernandez, 2011).

Indeed, as suggested by the Education Improvement Commission (2000), student performance improves when teachers use curriculum-delivery strategies which specifically address the needs of their students, when the school environment is positive, and when parents are involved in their children’s education. In planning improvements, therefore, schools should establish one priority in each of these three areas (curriculum delivery, school environment, and parental involvement). In sum, strategic planning involves scanning the environment and conditions that schools face, formulating goals and targets, developing an action to achieve the goals, and designing a method of monitoring and controlling implementation (Robinson & Pearce, 1983).

The Quality and Merit Project (PQM) in Italy

PQM PON¹ (Italian acronym for National Plan for Quality and Merit) is an Italian in-service training program which aims at providing lower secondary school teachers some innovative teaching materials in order to enhance student achievement in math. It is a joint endeavor of the Ministry of Education, the National Institute of Documentation, Innovation and Education Research (INDIRE) and the National Institute for the Evaluation of the Educational System of Instruction and Training (INVALSI). It is addressed to the teachers of lower secondary schools in the Southern Italian regions having access to the European Union funds for low income EU regions (Campania, Sicily, Calabria and Apulia).

The rationale for intervening stems from the fact that schools in these areas are characterized by markedly lower student performance in the various dimensions of learning if compared to schools in the rest of the country. In this regard, systematic evidence from international surveys (i.e. IEA-PIRLS, IEA TIMMS and OECD PISA) has identified the gap between the Italian school system and that of other OECD countries. This figure conceals a good deal of variability across regions, with Northern areas performing in line with other European countries and Southern areas performing markedly below. In addition, it is demonstrated that the North/South divide increases at the end of the primary school and grows even larger in middle schools (INVALSI, 2010a). In particular, four regions located in Southern Italy (Campania, Sicily, Calabria and Apulia) are eligible to benefit from EU Regional Development Funds and from the European Social Fund, for the period 2007–2013, to improve teaching and learning processes in middle and high schools. Indeed, there is evidence that regions that have eventually employed EU funds have sensibly improved their performance at PISA tests between 2006 and 2009, in particular in Apulia and Sicily (INVALSI, 2010b). For example, according to the distribution of test scores in mathematics in 2006, Apulia was ranked 17th amongst the 20 Italian regions, and 11th in 2009. The same figures for reading are 16th in 2006 and 12th in 2009, and for sciences are 17th in 2006 and 13th in 2009. One of the actions taken with these EU funds is the implementation of the PQM project which aims at improving the quality of educational system, school accountability and student achievement by teacher professional development, according to the principles of school autonomy law in curricular, organizational and financial (ability of diversifying sources of funding) terms. Participation of schools is not compulsory. Applicant schools are ranked according to a series of performance indicators (i.e. percentage of repeating and failing students and dropout rates), and only those schools at the bottom end of the performance distribution are eventually enrolled.

The PQM project adopts an ‘input-process-output’ model arising from school effectiveness research (Scheerens, 2000) which deals with the impact of inputs (i.e. teacher experience, per-pupil expenditure, parent support) on measured outputs (i.e. student achievement) and the influence of processes (i.e. educational leadership, quality of school curricula, classroom climate) intervening in this causal relationship. According to this model, the context can be seen as a generator of inputs, as a level that determines or co-determines the definition of desired outcomes that should be generated, and as a level that judges quality and provides feedback. In more practical terms the context-dimension gives room for situational adaptation to local conditions.

PQM is not intended to be a traditional content-focused training program, but it provides teachers with polyvalent training, offering diagnosis instruments, didactic planning skills and didactic materials. The teachers participating in the project are part of a network of schools coordinated by a tutor, who gives them both formal and online training, all through the school year. The training has two main goals: 1) help teachers to set up a SIP, based on student results in standardized test prepared by INVALSI and administered at the beginning of school year; 2) provide teachers with alternative solutions for teaching the usual curricular contents by using elements such as didactic material, teamwork and lab activities.

The drafting of the SIP is the most important aspect of the training, because it determines the number and the type of remedial activities on which the teachers will be then trained. By setting up the SIP teachers should thus identify the skills they would need to acquire both in didactic planning and teaching. The structure of SIP is organized in three sections:

1) Analysis of the educational context, which is a fundamental step to plan effective and specific improvement interventions. It should be addressed at two levels. At school level, it should describe teaching organization and parental involvement in both the project and student learning more generally. At PQM class level, it should inspect classroom climate and student motivation with specific regard to math.

2) Diagnosis of student needs, which are detected by the INVALSI assessment of math achievement deficits. The diagnosis should be integrated also with information on class background and ordinary teaching. In detail, it should identify both weak and strong points related to student cognitive processes and learning subject areas.

3) Detection of improvement goals for planning specific and detailed activities. The main improvement goals deal with: remedy/empowerment of student education, teacher professional development and parental involving in school activities.

The activities that teachers can implement mainly fall in three categories:

- remedial and extra education outside regular school time (15 hours each) with small groups of students (didactic units based on the main subject areas);

- producing new didactic materials;

- opportunities to share innovative teaching materials with other colleagues in the school in a sort of professional community (teacher peer-to-peer laboratory sessions).

At the end of the school year, students are tested a second time and the results are used as a check of the activities of the current year and as a starting point for the drafting of the SIP for the following year.

Aims of the study and theoretical framework

In line with what suggested by the dynamic model (Creemers & Kyriakides, 2008), PQM supports a whole school approach and school self-evaluation mechanisms for decision-making about improvement of policies and actions at both school and class level. In this frame, data collection, monitoring procedures, periodic evaluations and status updates become part of the normal school routine, and this is a relevant factor related to improving education (Education Improvement Commission, 2000).

The PQM project also gives opportunity for teachers to engage in continuous and sustained learning about their practice in the settings in which they actually work and to confront similar problems with colleagues and other schools. This is an essential principle of a theory of action which provides a through line to the instructional core, which are the vital activities that need to happen to improve teaching and learning (City, Elmore, Fiarman, & Teitel, 2009). To think in terms of action means thinking of planning practice as situated; that is, always embedded in specific, historically conditioned places and surrounded by the objects which constitute those places (Beauregard, 2012). Silins and Mulford (2002) depict school as a place where all participants – teachers, principals, parents and students – should be engaged in a process of continuous and ongoing learning and teaching. Viewed from this perspective, school’s efforts for a new planning process and an alternate planning model are right measures in the direction of becoming a learning organization continually in search of new and expansive knowledge (More, 2011). In this sense, PQM supports change knowledge (Fullan, 2005) as it shows some key-elements of theory of action, such as focus on motivation, capacity-building with a focus on results, learning in context, changing context, a bias for reflective action, tri-level engagement persistence and flexibility in staying the course.

This article aims at exploring the main features of PQM SIPs in relation to student achievement, given the theoretical relevance of them for an effective school practice. Thus, our research question concerns two specific aims:

1) Evaluate the improvement in student math achievement from 2009/2010 to 2010/2011 school year in order to provide a preliminary assessment of the effectiveness of PQM project.

2) Explore the association between characteristics/contents of SIPs and some illustrative variables of schools: geographical area, improvement level in student math achievement, socio-economic status (SES).

Methods and data sources

Participants

In this article we focus on the schools of the four regions of Southern Italy (Calabria, Campania, Apulia and Sicily) which started the PQM project in 2009/2010 school year (with sixth-grade classes) and continued it in 2010/2011. Unfortunately, the reliability of the measures related to the entry test in 2009 was very low and only provided us with information on the classes involved in the program (and not on the students), so we excluded it. We can use pre- and post-treatment measures for the second year of implementation. Thus we exclude both schools which participated in the program only in 2009/2010 or in 2010/2011. In more detail, we use pre- and post-results of the standardized test by INVALSI only for the students (n = 13,816) participating in PQM activities in both school years; they belong to 504 classes coming from 248 schools.

Data sources

Data at the school level are provided by the Italian Ministry of Education through INVALSI. Data at the student level are collected directly by INVALSI, through standardized tests in mathematics at sixth (at the end of 2009/2010 school year) and seventh grade (at the end of 2010/2011 school year), the former being the pre-treatment and the latter the post-treatment outcome. The test measures knowledge of the mathematics contents and logical and cognitive processes used in the mathematical reasoning, according to the theoretical framework for Mathematics curriculum based on the recommendations of the National Commission for Math Instruction and Trends in International Mathematics and Science Study (IEA-TIMSS). This theoretical framework addresses two main dimensions for test item development consistent with most of the international studies on student achievement assessment:

Math contents belonging to four different thematic areas: Numbers and algorithms, Geometry, Relations and functions, Data handling.

Cognitive processes regarding the student ability to know and master some elements, such as: specific math contents, algorithms and procedures, forms of math representation, problem solving, measurement tools, math thinking and quantitative information.

The SIPs and data of the activities by schools and classes are provided by INDIRE. For each student, student questionnaires were also administered and provide us with data of the students’ individual and socio-economic characteristics.²

Analysis procedures

Given the nature of the research questions, we address the issue by adopting a mixed-methodology approach, a research paradigm which utilizes and assigns an equivalent status to both qualitative and quantitative components (Tashakkori & Teddlie, 1998).

In order to assess student improvement in math achievement we calculate math test score simply as a percentage of correct answers out the total number of questions and which hence varies between 0 and 100. For this purpose, we use T-tests to compare pre-post results (based on school average math score from PQM classes) in the two school years, controlling for regions and socio-economic status (SES). We calculate also the correlation (r coefficient) between school data on PQM intervention (number of didactic activities, school and class size, percentage of PQM students and classes out of the total number of the school) and average math scores in order to better understand participation levels and treatment intensity.

Since this article provides only a preliminary assessment of the effectiveness of the PQM project, we will repeat the analyses on twin classes³ (selected in PQM schools) not participating in the PQM program in order to compare them with PQM classes, also by using anchored scores of pre- and post-math tests which are not yet available.

We analyze SIPs written by schools with text analysis software (Lexico3 and T-Lab) focusing on each section (analysis of the context, diagnosis of student needs, detection of goals and activities). Besides, we explore the relationship between textual data of school SIP and some illustrative variables at school level (in our case, region, student improvement in math and SES). Given that illustrative variables need to be categorical, we split the distributions of both math improvement⁴ and SES scores into five divisions at the 10th, 25th, 75th, 90th percentiles so to determine different levels for each variable (very high, high, medium, low, very low).

In more detail, we calculate some lexicometric indexes of SIPs in order to gather quantitative and qualitative information from the formal aspects of the texts, such as:

Corpus dimension (N) in terms of total number of occurrences or word-tokens.⁵

Vocabulary dimension (V) in terms of total number of different graphic forms or word-types.

Indexes of lexical richness, such as the Average Word Frequency (the occurrence of each word-type in the whole corpus) and the Type-Token Ratio (the number of type-words out of the total number of token-words).

Indexes of lexical specificity and density, derived from the number of Hapaxes (word-types which occur only one time in the whole text) divided by the corpus (Lexical Variety) or the vocabulary (Hapax Percentage) dimension.

Computer-aided thematic analysis is also carried out to deepen the specific contents dealt with. This is to detect the main thematic repertoires (cluster analysis) and latent dimensions (multiple correspondences analysis) of SIPs texts. Indeed, thematic analysis allows us to explore a representation of textual corpus contents through a few and significant thematic clusters, related to different semantic nuclei (Lancia, 2004). Each cluster consists of a set of elementary contexts (i.e. sentences) characterized by the same patterns of key-words and can be described through the lexical units (words or lemmas) and the most characteristic variables of the context units from which it is composed. Chi-square tests allow the test of the significance of a word recurrence within each cluster.

Then, correspondence analysis enables us to explore the relationship between clusters in bi-dimensional spaces, so to detect the latent factors which organize the main semantic oppositions in the textual corpus. In geometrical terms, each factor sets up a spatial dimension – which can be represented as an axis line – whose center (or barycenter) is the value ‘0’, and which develops in a bipolar way towards the negative (-) and positive (+) end, so that the objects put on opposite poles are the most different, almost like the ‘left’ wing and the ‘right’ wing on the political axes.

The relationship between the detected factors and illustrative variables is evaluated through Test Value, a statistical measure with a threshold value (2), corresponding to the statistical significance more commonly used (p 0.05) and a sign (-/+) which helps in understanding the poles of factors detected through the correspondence analysis.

In sum, the T-LAB tool we used for the analysis was the ‘Thematic analysis of elementary context’ which transforms the textual corpus in a digital ‘presence-absence’ matrix. To do that, each sentence was considered as a segment of the corpus (namely, an elementary context unit) and represented a row of the matrix, while all the words present in the corpus represented the columns of the matrix. The analysis procedure consists of the following steps:

a) construction of a data table context units x lexical units (up to 150,000 rows x 3000 columns), with presence/absence values;

b) normalization and scaling of row vectors to unit length (Euclidean norm);

c) clustering of the context units (measure: cosine coefficient; method: bisecting K-means);

d) filing of the obtained partitions and for each of them;

e) construction of a contingency table lexical units x clusters (n x k);

f) Chi square test applied to all the intersections of the contingency table;

g) correspondence analysis of the contingency table lexical units x clusters.

This procedure therefore performs a type of co-occurrence analysis (steps a–c) and, subsequently, a type of comparative analysis (steps e–g). In particular, comparative analysis uses the categories of the ‘new variable’ derived from the co-occurrence analysis (categories of the new variable = thematic clusters) to form the contingency table columns.

Results and discussion

Student improvement analysis

Concerning the first research question, preliminary analyses limited to PQM classes have already provided some results. Pre–post analysis reveals an increase in PQM student math scores (p < 0.01). On average, students get 4 points percentage in correct answers from 2009/2010 to 2010/2011 school year. This difference remains significant also when considering each region. In particular, Apulia has the highest improvement (almost 7 percentage points), while Calabria shows the minimum one (close to 0 percentage points) (Table 1).

Table 1.

Pre–post measures of math test score (School Average Score).

		N	Mean	S.D.	Standard error of the Mean
	Post	248	55.72*	16.23	1.03
	Pre	248	51.39*	9.90	0.63
Region:
Calabria	Post	32	51.14*	14.76	2.61
Calabria	Pre	32	51.39*	10.76	1.90
Campania	Post	82	54.63*	16.05	1.77
Campania	Pre	82	52.05*	9.78	1.08
Apulia	Post	69	59.70*	15.82	1.90
Apulia	Pre	69	52.73*	7.98	0.96
Sicily	Post	65	55.15*	17.01	2.11
Sicily	Pre	65	49.12*	11.22	1.39

Pre–post difference (2011–2010) is statistically significant (p < 0.01).

Participation and treatment intensity (number of didactic units, school and class size, percentage of PQM students and classes out the total of the school) has no relation with achievement. We find a correlation between class average SES and each math score for 2009/2010 and 2010/2011 (p < 0.05) but not with the gap scores (Table 2).

Table 2.

Correlations between SES and math test scores (2011 School Average; 2010 School Average; 2011–2010 difference).

		SES
2011 Math Score	Pearson correlation	.141
	Sig.	.027
	N	248
2010 Math Score	Pearson correlation	.159
	Sig.	.036
	N	248
2011-2010 Math Score difference	Pearson correlation	.020
	Sig.	.750
	N	248

In this regard, results confirm the association between SES and student math achievement, when considering a single school year, in line with national (INVALSI, 2009, 2010c) and international data (OECD, 2010), since socio-economic background is widely recognized as an important contributor to student and school achievement (Coleman et al., 1996; Sirin, 2005). Instead, SES doesn’t seem to affect student improvement in math achievement when considering the gap score (between 2010/2011 and 2009/2010 school years), although an ‘incremental effect’ of SES on student improvement may probably need a longer time range, and not just one school year.

Lexicometric analysis of SIPs

Our general corpus is composed of 248 texts and includes a total of 494,538 word-tokens (N) and 51,442 word-types (V).

Some lexicometric indexes are calculated to gather quantitative and qualitative information from the formal aspects of the texts with specific regard to the three SIP sections. The analysis of the context is addressed to both school and class level and refers to teaching organization, parental involvement, classroom climate, student achievement and learning motivation. The diagnosis of student needs relies on the INVALSI assessment of math achievement, as well as on information derived from class background and ordinary teaching and aims at identifying both weak and strong points related to student cognitive processes and learning subject areas. Then, the detection of improvement goals refers to the planning of specific and detailed activities (i.e. remedy/empowerment of student education, teacher professional development and parental involving in school activities) which can overcome the critical problems identified in the diagnostic phase.

Looking at each SIP text section (Table 3), the Type-Token Ratio is less than 20 percent and the Hapax percentage is less than 50 percent, hence it is possible to state the consistency of a statistical approach (Bolasco, 1999). The comparison of the different corpora shows that, overall, Analysis of the context is longer and also characterized by higher lexical richness and variety, differently from Diagnosis of student needs which uses a repetitive (although detailed) vocabulary, and from Detection of improvement goals whose lexicon is sufficiently rich but too generic (Table 3).

Table 3.

Lexicometric indexes of SIP sections.

SIP section	Word-tokens (N)	Word-types (V)	Hapaxes (V₁)	Average word frequency (N/V)	Hapax percentage (V1/V)*100	Lexical variety (V1/N)*100	Type/token ratio (V/N)*100
Analysis of the context	173,413	11,814	5342	14.68	45.22	3.08	6.81
Diagnosis of student needs	159,560	6245	2488	25.55	39.84	1.56	3.91
Detection of improvement goals	161,565	8776	2488	18.41	28.35	1.54	5.43

Using illustrative variables as text partition keys, we can compare SIPs’ lexicometric indexes among different geographic areas (regions), SES and achievement improvement levels⁶ (Table 4). In sum, Calabria is the region whose SIPs are generally characterized by greater richness and detail, whilst Campania’s SIPs tend to be slightly more stereotyped.⁷ SES is not significantly associated to any measure. Then, the SIPs of schools with very high student achievement improvement provide a more accurate analysis of the context and a greater originality and specificity of improvement goals, whilst the SIPs of schools with very low student achievement improvement show a more precise and articulated diagnosis of student needs.

Table 4.

Lexicometric indexes of SIPs partitions by illustrative variables.

Illustrative variables	Word-tokens (N)	Word-types (V)	Hapaxes (V₁)	Average word frequency (N/V)	Hapax percentage (V₁/V)*100	Lexical variety (V₁/N)*100	Type/token ratio (V/N)*100
Analysis of the context
Geographic area (region)
Calabria	20,436	3663	1857	5.58	50.70	9.09	17.92
Campania	61,699	7064	3385	8.73	47.92	5.49	11.45
Apulia	54,238	6508	3200	8.33	49.17	5.90	12.00
Sicily	37,040	5192	2583	7.13	49.75	6.97	14.02
Socio-economic status (SES)
Very high	20,872	3920	2070	5.32	52.81	9.92	18.78
High	22,269	3983	2086	5.59	52.37	9.37	17.89
Medium	85,308	8318	3912	10.26	47.03	4.59	9.75
Low	26,448	4439	2348	5.96	52.89	8.88	16.78
Very low	18,516	3635	1940	5.09	53.37	10.48	19.63
Achievement improvement
Very high	14,659	3248	1829	4.51	56.31	12.48	22.16
High	24,008	4095	2119	5.86	51.75	8.83	17.06
Medium	89,692	8665	4087	10.35	47.17	4.56	9.66
Low	27,961	4476	2292	6.25	51.21	8.20	16.01
Very low	17,093	3566	1941	4.79	54.43	11.36	20.86
Diagnosis of student needs
Geographic area (region)
Calabria	15,314	1951	704	7.85	36.08	4.60	12.74
Campania	54,716	3603	1432	15.19	39.74	2.62	6.58
Apulia	60,582	3710	1446	16.33	38.98	2.39	6.12
Sicily	28,948	2633	1005	10.99	38.17	3.47	9.10
Socio-economic status (SES)
Very high	16,362	2046	883	8.00	43.16	5.40	12.50
High	20,314	2496	1139	8.14	45.63	5.61	12.29
Medium	83,986	4264	1604	19.70	37.62	1.91	5.08
Low	25,134	2482	967	10.13	38.96	3.85	9.88
Very low	13,764	1959	821	7.03	41.91	5.96	14.23
Achievement improvement
Very high	15,857	1699	713	9.33	41.97	4.50	10.71
High	24,885	2362	885	10.54	37.47	3.56	9.49
Medium	77,727	4553	1875	17.07	41.18	2.41	5.86
Low	26,954	2628	1058	10.26	40.26	3.93	9.75
Very low	15,137	1948	839	7.77	43.07	5.54	12.87
Detection of improvement goals
Geographic area (region)
Calabria	25,176	3401	1451	7.40	42.66	5.76	13.51
Campania	62,965	5547	2153	11.35	38.81	3.42	8.81
Apulia	39,454	4385	1907	9.00	43.49	4.83	11.11
Sicily	33,970	3785	1408	8.97	37.20	4.14	11.14
Socio-economic status (SES)
Very high	16,042	2987	1409	5.37	47.17	8.78	18.62
High	26,113	3968	1886	6.58	47.53	7.22	15.20
Medium	81,868	6306	2509	12.98	39.79	3.06	7.70
Low	23,528	3170	1324	7.42	41.77	5.63	13.47
Very low	14,014	2557	1186	5.48	46.38	8.46	18.25
Achievement improvement
Very high	10,046	2119	1051	4.74	49.60	10.46	21.09
High	23,321	3534	1609	6.60	45.53	6.90	15.15
Medium	80,522	6185	2385	13.02	38.56	2.96	7.68
Low	29,913	3873	1626	7.72	41.98	5.44	12.95
Very low	17,763	3234	1559	5.49	48.21	8.78	18.21

Thematic analysis of SIPs

As follows, we present both thematic clusters and latent factors which are detected for each textual section of SIPs. Clusters are labeled by the researcher based on the typical vocabulary (lemmas) and elementary context units (sentences) of which they are composed. In the same way, the meaning of each factor is inferred by the researcher from the different contents attributable to the main clusters associated to the factorial poles.

The interpretative process takes into account the semantic context which characterizes all the co-occurring lemmas in each cluster, more than a single word taken in isolation. Then, an in-depth qualitative analysis of the text segments derived from the SIPs (i.e. the elementary context units) grouped in each cluster is also used in order to check the robustness of this inferential process. Because a word or a phrase has more than one meaning depending on context and usage, the function of the co-occurrence of words in a text is hypothesized to reduce the association of meanings attributable to each word (i.e. polysemy), thus allowing a thematic domain to be progressively constructed.⁸

Analysis of the context

The analysis detects four thematic clusters of which we report both some of the most characteristic lemmas (Table 5) and examples of elementary context units, indicating their percentage out of the total (Table 6).

Table 5.

Analysis of the context. The most characteristic lemmas in each cluster (Chi-square).

Cluster 1		Cluster 2		Cluster 3		Cluster 4
Lemma	Chi-square	Lemma	Chi-square	Lemma	Chi-square	Lemma	Chi-square
Difficulty	162.90	Laboratory	138.13	Territory	387.58	Population	126.44
Study	60.20	Teacher	98.20	Social	194.50	Family	82.31
Mathematics	52.12	Training	71.69	Cultural	166.08	Immigrant	78.62
Ability	48.94	Multimedial	67.68	Context	111.33	Poverty	24.50
Remedy	44.58	Tool	64.98	Offer	107.21	Social class	16.33
Lacking	41.99	Assessment	63.31	Users	101.14	Unemployment	15.55
Knowledge	41.01	Project	62.56	Parish	67.70	Service industry	15.12
Learning	40.73	Information Technology	39.32	Resource	65.06	Economic	13.58
Improvement	36.79	Experimentation	18.31	Educational	62.02	Community	11.43
Competence	20.30	Workshop	18.24	Agency	59.03	Ethnic group	10.25

Note: Textual data were translated into English only for the purposes of the article.

Table 6.

Analysis of the context. Examples of Elementary Context Units of each cluster (%).

Cluster 1 (31.84%)

Participating in the project may be an opportunity to consolidate and develop logical mathematical knowledge and skills for some students, to remedy basic abilities for other ones instead.

Cluster 2 (18.03%)

PQM project tutor in the school department has provided teacher training on ICT, with specific regard to the use of didactic software. The school offers several laboratories: information technology, scientific, linguistic and musical.

Cluster 3 (18.44%)

The school is set in a densely populated suburb, recently characterized by indiscriminate council housing and lack of suitable community services. This territory offers few meeting places for youth. The only educational agencies are family, school and parish.

Cluster 4 (31.68%)

The working population is heterogeneous: professionals, office workers, shopkeepers, artisans, casual workers and unemployed; it follows that the socio-economic conditions of children attending the school are different and the level of education of their families is lower-middle. There are also poor pupils who live in residential children’s homes, and some without families.

Note: Textual data were translated into English only for the purposes of the article.

Cluster 1: Student ability and performance

It deals with student characteristics regarding study method and basic abilities. In more detail, it shows a specific focus on general learning difficulties of the class and the need for improving student performance.

Cluster 2: School resources

It highlights school educational and organizational resources, also in terms of didactic tools (availability of school equipment, laboratories, ICT, etc.). This cluster is associated with a tendency to invest on student assessment, teaching innovation and experimentation, and to create a teacher professional community inside the school.

Cluster 3: External socio-educational agencies

It focuses on the relationship between school and external socio-educational agencies (parish, youth associations, social services). Since youth education is seen as a shared responsibility with territorial partners outside the school, the context is given a central role in supporting school efficacy.

Cluster 4: Family background and social context

This cluster mainly relies on the description of student family background and origin. It also refers to wider social, cultural and economic context and some critical issues (immigration, youth problems, unemployment, poverty) which are likely to affect student education and development.

Correspondence analysis enables us to explore the relationship between the four thematic clusters detected in a bi-dimensional space (Figure 1). Thus it allows the analysis of the latent factors which organize the main semantic oppositions in the textual corpus, from the different position of clusters on the first two factorial axes – as indicated by Test Values (Table 7) – which explain about 86 percent of total inertia.

Figure 1.

Analysis of the context. Factorial space.

Table 7.

Analysis of the context. Relation between clusters and factors (Test Value).

	Factor 1	Factor 2
CLU 1	−50.33	3.11
CLU 2	−33.98	−26.53
CLU 3	60.86	−28.70
CLU 4	29.12	41.55

In more detail, the first latent factor (horizontal axis) seems to refer to school responsibility for student education (61.11% of the variance). On the negative pole, the analysis of the context focuses on the role of the school in improving both student achievement (Cluster 1) and teaching methods (Cluster 2). Instead, on the positive one, there is little internal commitment because schools tend to delegate their institutional function to other external socio-educational agencies (Cluster 3) or to families and the wider social context (Cluster 4).

The second latent factor (vertical axis) expresses school self-efficacy (23.44% of the variance): the negative pole is associated to the school perception of internal (Cluster 2) and external (Cluster 3) resources which can support its educational aims and efficacy; on the contrary, the positive pole is mainly represented by the perception of powerlessness in front of unchangeable characteristics of student environment (Cluster 4).

Analyses also show some associations between latent factors and illustrative variables: schools with low achievement improvement level mainly attribute responsibility for student education to the social context (Test Value = 2.18); whilst schools with high SES perceive themselves effective in educational improving (Test Value = -2.21).

Diagnosis of student needs

The analysis detects five thematic clusters of which we report both some of the most characteristic lemmas (Table 8) and examples of elementary context units, indicating their percentage out of the total (Table 9).

Table 8.

Diagnosis of student needs. The most characteristic lemmas in each cluster (Chi-square).

Cluster 1		Cluster 2		Cluster 3		Cluster 4		Cluster 5
Lemma	Chi-square	Lemma	Chi-square	Lemma	Chi-square	Lemma	Chi-square	Lemma	Chi-square
School	651.12	Mathematics	221.68	Activity	633.64	Answer	201.76	Master	404.66
Score	365.76	Information	189.39	Remedy	442.52	Correct	97.49	Representation	233.85
Region	324.28	Use	178.68	Improvement	358.84	Question	72.54	Algorithm	169.74
Result	256.56	Tool	125.52	Planning	313.54	Level	63.35	Procedure	166.09
Achievement	206.83	Scientific	121.00	Proposal	61.59	Item	45.20	Thinking	164.87
South	201.17	Description	119.45	Qualification	49.74	Percentage	44.78	Symbolic	120.19
Test	189.43	Solve	116.49	Didactics	41.13	Gap	30.11	Argument	119.51
National	180.37	Detect	116.49	Subject	32.73	Weakness	27.43	Problem	117.59
Average	157.46	Explain	116.26	Curriculum	\17.65	Output	16.45	Arithmetic	93.65
Comparison	84.61	Quantitative	114.50	Performance	6.68	Error	5.83	Process	56.39

Note: Textual data were translated into English only for the purposes of the article.

Table 9.

Diagnosis of student needs. Examples of Elementary Context Units of each cluster (%).

Cluster 1 (9.93%)

In order to formulate the diagnosis for the class, we analyze results from the test administered in May 2010. The class scored 51.9% of correct answers, in line with our school, PQM schools and national average, and higher than the regional and Southern Italy one.

Cluster 2 (23.10%)

Use learned math to analyze information through quantitative data in scientific, technological, economic and social fields (describe a phenomenon in quantitative terms, interpret it with statistical tools or functions, use math models).

Cluster 3 (15.04%)

I propose to plan some remedial didactic units concerning Arithmetic and Geometry and some empowering units on Relations and Functions already used last year.

Cluster 4 (34.42%)

There are low percentages of correct answers to the questions n. 14 and n. 19, which concern the subject area Data handling. Another negative element is the high rate of missing answers to the question n. 25 dealing with Arithmetic.

Cluster 5 (17.51%)

The class seems to have greater difficulties in two subject areas, Arithmetic and Data Handling, and in the following processes: problem-solving by math tools; know and master various forms of representation; progressively acquire typical forms of math thinking.

Note: Textual data were translated into English only for the purposes of the article.

Cluster 1: School achievement

This cluster is associated to the use of results from INVALSI standardized test, mainly focusing on school ranking at regional and national level in the wider context. General information is derived on math score at school or class level, without further reference to math subject areas or in-depth analysis on students.

Cluster 2: Student ability in using math

This cluster is completely focused on one of the cognitive processes measured by INVALSI standardized test, which deals with the student ability in using math for analyzing quantitative information and interpreting reality. In this sense, it seems to emphasize a competence-based approach for student assessment rather than a knowledge-based one.⁹

Cluster 3: Curriculum-based information

This cluster highlights the importance of curriculum-based information to diagnose student needs. In this regard, teacher experience and continuity in education are some key elements to plan effective goals, consistently with the usual teaching practice.

Cluster 4: Detailed analysis of math test

This cluster refers to the detailed analysis of student results deriving from the INVALSI standardized test. The external assessment is accurately used as the main source to detect student deficits, from a multi-focused and analytical view.

Cluster 5: Cognitive processes and subject areas

In this cluster student needs are detected from test results concerning both cognitive processes and subject areas measured in math tests. In this sense, the use of external assessment allows us to identify priorities and specific learning skills to address in planning next activities.

The analysis detects two factorial axes (Figure 2) which overall explain 80.53 percent of the total inertia and are differently associated to the five thematic clusters (Table 10).

Figure 2.

Diagnosis of student needs. Factorial space.

Table 10.

Diagnosis of student needs. Relation between clusters and factors (Test Value).

	Factor 1	Factor 2
CLU 1	−72.46	46.53
CLU 2	36.31	−5.50
CLU 3	−72.95	−51.96
CLU 4	−7.38	13.63
CLU 5	75.72	−0.88

The first latent factor refers to the specific utilization of standardized test (61.97% of the variance) for the diagnosis of student needs. On the negative pole, the diagnosis relies on a multi-focused approach which is mainly curriculum-based (Cluster 3) and fails to use test results correctly, because the analysis of the test is more school- than student-centered (Cluster 1) or is over-detailed but without identifying key elements to improve (Cluster 4). Instead, on the positive pole, more attention is paid to detect specific student skills (Cluster 2) or learning processes (Cluster 5) as priorities to enhance.

The second latent factor focuses on the degree of integration between internal and external assessment (18.55% of the variance). It mainly opposes a self-assessment approach – negative pole – giving greater importance to the curriculum and teaching processes inside the school (Cluster 3) to an external assessment – positive pole – which provides objective data and allows the comparison of school achievement with other schools in the wider context (Cluster 1).

The results on the association between factors and illustrative variables show that the schools with low (Test Value = -2.71) or very low (Test Value = -3) SES tend to highlight the centrality of self-assessment processes and teaching practice inside the school in order to detect student needs, rather than using external assessment. It is also true for the schools with low (Test Value = -2.23) or very low (Test Value = -2.96) student achievement improvement which, in addition, mainly rely on a multi-focused diagnosis and fail to use test results correctly (Test Value = -4.69 for low level; Test Value = -3.81 for very low level).

Detection of improvement goals

The analysis detects five thematic clusters of which we report both some of the most characteristic lemmas (Table 11) and examples of elementary context units, indicating their percentage out of the total (Table 12).

Table 11.

Detection of improvement goals. The most characteristic lemmas in each cluster (Chi-square).

Cluster 1		Cluster 2		Cluster 3		Cluster 4		Cluster 5
Lemma	Chi-square	Lemma	Chi-square	Lemma	Chi-square	Lemma	Chi-square	Lemma	Chi-square
Work	269.71	Figure	454.69	Strategy	169.39	Represent	187.18	Class	318.56
Practice	160.95	Geometry	165.72	Instrument	159.76	Interpret	178.25	Student	260.73
Tutor	119.14	Space	78.96	Identify	134.93	Master	158.40	Remedial	206.35
Laboratory	115.08	Didactic Unit	76.53	Scheme	122.23	Know	112.35	Empowering	172.39
Activity	84.78	Drawing	61.41	Link	113.39	Describe	104.97	Improvement	134.13
Time	84.58	Calculate	36.48	Usefulness	90.40	Meaning	66.97	Involving	92.05
Cooperative	81.25	Play	20.30	Resolving	80.19	Learn	46.20	Goal	43.90
Method	81.18	Handle	18.71	Measure	74.26	Codify	46.13	Responsibility	34.33
Meeting	58.11	Dynamic	10.62	Operational	57.64	Understand	31.49	Self-evaluation	30.40
Innovative	42.98	Writing	9.86	Explore	35.14	Cognitive	10.48	Skill	30.14

Note: Textual data were translated into English only for the purposes of the article.

Table 12.

Detection of improvement goals. Examples of Elementary Context Units of each cluster (%).

Cluster 1 (8.55%)

Laboratory practice promotes exploration, modeling and operational deduction. Cooperative methodology (small working group) activity enhances argumentation about experiences and procedures used. Along the different steps of the work, after individual production it is possible to exchange views and analyze the process.

Cluster 2 (10.09%)

The training activity will be centered on Geometry, in particular on the didactic unity ‘Equi-decomposability of elementary figures: the Tangram’ that will allow to deal with the concepts of equi-extension and equivalence of plane figures in order to explain simple literal formulas.

Cluster 3 (22.73%)

Problem-solving by using math tools (i.e. detect and connect relevant information, compare solution strategies, identify solution schemes such as calculation sequences, explain the solution process).

Cluster 4 (29.11%)

Know and master specific contents of mathematics (mathematical objects, properties, structures). Know and master algorithms and procedures (in arithmetic and geometry). Know and master various forms of representation (verbal, written, symbolic, graphical) and know how to switch between them.

Cluster 5 (29.52%)

On the basis of this diagnosis, it is appropriate to plan an empowering training activity, addressed to ten students, and two remedial activities concerning Arithmetic and Relations and Functions, each addressed to another group of 15 students.

Note: Textual data were translated into English only for the purposes of the article.

Cluster 1: Training methods

This cluster mainly focuses on training methods and procedures used for the goals implementation plan. In detail, it refers to a teaching approach which is based on laboratory and group working in order to enhance cooperative learning among students, consistently with PQM philosophy on school improvement.

Cluster 2: Didactic units

This cluster deals with the selection of specific didactic units related to math contents which need to be improved. It thus refers to teaching materials and activities as concrete and practical dimensions of the experimentation, in close relation to the curriculum.

Cluster 3: Problem-solving

The focus is on a specific cognitive process of student learning which is a key element of math teaching experimentation. In detail, it concerns problem-solving seen as the ability of using math tools to solve real-life problems in everyday situations.

Cluster 4: Cognitive processes

This cluster includes several student cognitive processes to which PQM intervention is addressed. These processes are associated to cross-curricular sub-competences which students are asked to know, use and reflect on in relation to mathematics.

Cluster 5: Detailed description of activities

In this cluster more attention is paid to the specific context at class and student level. The main goal of SIP is declared in terms of remedy or empowering education, and some differences are detected among students – based on previous assessment – in order to diversify PQM classroom activities.

The analysis detects two factorial axes (Figure 3) which overall explain 79.12 percent of the total inertia and are differently associated to the five thematic clusters (Table 13).

Figure 3.

Detection of improvement goals. Factorial space.

Table 13.

Detection of improvement goals. Relation between clusters and factors (Test Value).

	Factor 1	Factor 2
CLU 1	−47.42	−47.42
CLU 2	13.14	41.82
CLU 3	13.34	−49.17
CLU 4	81.56	3.63
CLU 5	−78.72	40.73

The first latent factor seems to rely on the degree of specificity of planned goals (49.87% of the variance). It mainly opposes a specific to a general goal orientation. On the negative pole, there is a greater tendency to explicit educational aims and detail activities (Cluster 5), and also focus on training methods and materials (Cluster 1). Instead, on the positive one, general suggestions are given about student cognitive processes which need to be improved, but without indications on how to do it (Cluster 4).

Then, the second latent factor highlights the degree of originality in SIP elaboration (29.25% of the variance). On the negative pole, the focus is on the wider PQM framework concerning a teaching approach which is oriented to collaborative learning (Cluster 1) and problem-solving (Cluster 3). SIPs seem thus to address general key-elements of PQM activities, which are reported in a very stereotypical way¹⁰ without further elaboration. On the contrary, the positive pole is characterized by higher originality and autonomy in setting up SIPs because various strategies are accounted for in order to select didactic units (Cluster 2) and to diversify classrooms activities (Cluster 5).

The results on the association between factors and illustrative variables show that the schools with very high student achievement improvement tend to report more specific activities and better explicit improvement goals (Value Test = -5.34); whilst the schools with very low improvement are characterized by a stereotypical tendency in SIP elaboration (Value Test = -3.63). Apulia is the region which shows the greatest degree of specificity of planned goals (Value Test = -6.75).

Conclusions

Regardless of some limitations (need for anchoring of the tests, need for repeating the analyses on control group, unavailability of more waves of data), some evidence exists on the fact that PQM students are improving, as indicated by pre–post analysis on math scores.

Moreover, the SIPs associated with better school improvements are those in which the schools have been able to carry out a more careful analysis in terms of context and reflection on aims and have been able to prioritize the various elements already in the diagnostic part of the process. Thus, it is possible that these schools have then been able to identify and implement more effective improvement activities than what happened in schools that were not able to focus on specific needs and tried instead to address simultaneously a multitude of aspects, as suggested by both lexicometric and thematic analysis of SIPs. This result is consistent with a previous study in Nevada (Fernandez, 2011), which revealed that the schools which faced a multitude of problems in their SIPs were less likely to be able to produce a clear improvement plan which could effectively address those problems. With regard to the main components differentiating the quality of SIPs highlighted in current research (Reeves, 2006a, 2006b) – which are assessment, goals and implementation – our study finds a substantial agreement. Indeed, both assessment (in terms of analysis of the context and diagnosis of students’ needs) and goals (in terms of planning detailed and specific improvement activities) emerge as relevant factors which are associated with better student achievement. On the contrary, there is not a specific effect deriving from the level of participation and treatment intensity (school and class size, percentage of PQM students and classes out the total of the school, number of didactic units), thus suggesting the need for further exploration of the variables affecting implementation.

In regard to improvement planning strategies at school level, the key factors which seem to promote better student achievements concern the schools’ capabilities of taking account of the educational context and, above all, detecting specific and detailed improvement goals, as shown by results in Apulia, which is also the region with highest student improvement. Indeed, as stated by Bell (2002), the purpose of strategic planning should be to scan the environment in which the school operates.

On the contrary, the main obstacles to school improvement seem to refer to: the tendency to attribute responsibility for student education to external socio-educational agencies, the exclusive focus on school self-assessment to detect student needs, a diagnosis of student learning deficits not based on cognitive processes or specific subject areas to enhance, and poor autonomy and originality in setting up improvement activities. Indeed, the literature regarding the effectiveness of SIPs in boosting student learning suggests that if such plans are not owned by the school they will show little impact on student achievement (Borman, Hewes, Overman, & Brown, 2004). Despite participation in the PQM program is not mandatory for schools, the lack of originality and internal responsibility of schools are two notable pitfalls pointed out by our analyses. This seems to suggest the importance of stimulating interest and direct involvement of schools in the planning process. Indeed, mandatory planning imposed on public schools tends to induce an excess of formalization (which impedes learning what works in practice) and lack of enthusiasm (which is failing in generating original strategies), which, in turn, do not produce large improvements in schools (Bell, 2002; Mintzberg, 1994). In this sense, we can conclude that to be useful, school improvement planning should rely on strategic thinking, concerning synthesis, intuition and creativity rather than formal steps and automatic implementation, in order to avoid inflexible and myopic practices (Bryson & Roering, 1987; Halachmi, 1986; Mintzberg, 1994). The internal responsibility of schools is supposed to determine, guarantee and safeguard their quality and improve the teaching–learning process and school performance (Hofman, Dijkstra, & Hofman, 2005, 2009). In this regard, previous research demonstrated that the quality of the planning process and the performance outcomes are based on internal agency leaders’ perceptions and beliefs rather than on external objective assessment (Berry & Wechsler, 1995; Phillips & Moutinho, 2000). Additionally, socio-economic background does not seem to be specifically associated with student improvement, consistently with other studies (Fernandez, 2011), but can influence school planning strategies: high-SES schools are likely to perceive themselves with more self-efficacy in determining student outcomes; whilst low-SES schools seem to rely on internal evaluation mechanisms to take decisions on how to improve school functioning, without perceiving benefits from external assessment.

Some considerations can be made with regard to the potential added-value of our research study in the international literature. Current research on SIPs shows a limited number of published studies which specifically address the effects of SIPs. In addition, these studies frequently use a qualitative methodology to better understand school improvement planning (Doud, 1995; Levine & Leibert, 1987; Mintrop & McLellan, 2002; Schutz, 1980); however, they cannot be generalized to larger populations and are limited in their ability to observe patterns across a large number of schools. Instead, our research study adopts a mixed-methodology on a sizeable sample of schools so that wider implications may be considered. The potential of this approach relates to combine both quantitative and qualitative data, thus allowing a focus on locally based processes, which are inherently fuzzy and linked to the specific context, as well as the generalization of this evidence across schools.

Another strong point of our study deals with the focus on the actual implementation of improvement practices by the schools, not just on the planning process. In this regard, some researchers have been critical about the fact that many organizations attempt to engage in formal planning but are not successful in producing effective plans or disconnect between planning and the actual execution of a plan (Kaplan & Norton, 2005).

A limitation of this study concerns the lack of anchorage measures of math tests and of a control group. As mentioned before, we intend thus to check the robustness of our findings and better estimate the size of improvement by using anchorage measures and non-PQM classes as controls. Such very preliminary conclusions certainly require further research and reflection; yet, they feed into a stream of discussion which is currently growing in Italy and related to the ability of schools and school staff to use strategically information and autonomy of action for initiating school improvement processes.

Footnotes

Notes

References

Armstrong

J. S.

(1982). The value of formal planning for strategic decisions: Review of empirical research. Strategic Management Journal, 3, 197–211.

Beach

R. H.

Lindahl

(2004). Identifying the knowledge base for school improvement. Planning and Changing, 35(1–2), 2–32.

Beauregard

R. A.

(2012). Planning with things. Journal of Planning Education and Research, 32(2), 182–190.

Bell

(2002). Strategic planning and school management: Full of sound and fury, signifying nothing? Journal of Educational Administration, 40(5), 407–424.

Berry

F. S.

Wechsler

(1995). State agencies’ experience with strategic planning: Findings from a national survey. Public Administration Review, 55(2), 159–168.

Bobic

M. P.

Davis

W. E.

(2003). A kind word for theory X: Or why so many newfangled management techniques quickly fail. Journal of Public Administration Research and Theory, 13(3), 239–264.

Bolasco

(1999). L’analisi multidimensionale dei dati [Multidimensional data analysis]. Roma: Carocci.

Borman

G. D.

Hewes

G. M.

Overman

L. T.

Brown

(2004). Comprehensive school reform and achievement: A meta-analysis. In Cross

C. T.

(Ed.), Putting the pieces together: Lessons from comprehensive school reform research (pp. 53–108). Washington, DC: National Clearinghouse for Comprehensive School Reform.

Bryson

Roering

W. D.

(1987). Applying private-sector strategic planning in the public sector. Journal of the American Planning Association, 53, 9–22.

10.

City

Elmore

R. F.

Fiarman

S. E.

Teitel

(2009). Instructional rounds in education: A network approach to improving teaching and learning. Cambridge, MA: Harvard Education Press.

11.

Coleman

J. S.

Campbell

E. Q.

Hobson

C. J.

McParland

Mood

Weinfeld

F. D.

York

R. L.

(1996). Equality of educational opportunity. Washington, DC: US Government Printing Office.

12.

Creemers

Kyriakides

(2008). The dynamics of educational effectiveness: A contribution to policy, practice and theory in contemporary schools. London: Routledge.

13.

Creemers

Kyriakides

(2010). Explaining stability and changes in school effectiveness by looking at changes in the functioning of school factors. School Effectiveness and School Improvement: An International Journal of Research, Policy and Practice, 21(4), 409–427.

14.

Creemers

Kyriakides

(2012). Improving quality in education: Dynamic approaches to school improvement. London: Routledge.

15.

Curtis

City

(2009). Strategy in action: How school systems can support powerful learning and teaching. Cambridge, MA: Harvard Education Press.

16.

Doud

J. L.

(1995). Planning for school improvement: A curriculum model for school based evaluation. Peabody Journal of Education, 70(3), 175–187.

17.

Dunaway

D. M.

Kim

Szad

E. R.

(2012). Perceptions of the purpose and value of the school improvement plan process. The Educational Forum, 76(2), 158–173.

18.

Education Improvement Commission. (2000). School improvement planning: A handbook for principals, teachers and school councils. Toronto: Education Improvement Commission.

19.

Fernandez

K. E.

(2011). Evaluating school improvement plans and their affect on academic performance. Educational Policy, 25(2), 338–367.

20.

Fullan

(2005). Leadership and sustainability: System thinkers in action. Thousand Oaks, CA: Corwin; Toronto, Canada: Ontario Principals Council.

21.

Halachmi

(1986). Strategic planning and management: Not necessarily. Public Productivity Review, 40, 35–50.

22.

Hayes

Christie

Mills

Lingard

(2004). Productive leaders and productive leadership: Schools as learning organisations. Journal of Educational Administration, 42(5), 520–538.

23.

Hofman

R. H.

Dijkstra

N. J.

Hofman

W. H. A.

(2005). School self-evaluation instruments: An assessment framework. International Journal for Leadership in Education, 8(3), 253–272.

24.

Hofman

R. H.

Dijkstra

N. J.

Hofman

W. H. A.

(2009). School self-evaluation and student achievement. School Effectiveness and School Improvement: An International Journal of Research, Policy and Practice, 20(1), 47–68.

25.

Huber

G. P.

(1991). Organizational learning: The contributing processes and the literatures. Organization Science, 2(1), 88–115.

26.

Hummel-Rossi

Ashdown

(2002). The state of cost-benefit and cost-effectiveness analyses in education. Review of Educational Research, 72(1), 1–30.

27.

Istituto nazionale per la valutazione del sistema educativo di istruzione e di formazione (INVALSI). (2009). Rilevazioni degli apprendimenti. Scuola primaria. Prime analisi [Surveys on learning. Primary school. First analyses]. Rome: INVALSI.

28.

Istituto nazionale per la valutazione del sistema educativo di istruzione e di formazione (INVALSI). (2010a). Servizio nazionale di valutazione 2010–11: Le rilevazioni degli apprendimenti a.s. 2010–11 [National Evaluation Service 2010–2011: Technical report on student achievement 2010–2011 school year]. Rome: INVALSI.

29.

Istituto nazionale per la valutazione del sistema educativo di istruzione e di formazione (INVALSI). (2010b). Le competenze in lettura, matematica e scienze degli studenti quindicenni italiani: Rapporto Nazionale PISA 2009 [Reading, math and science literacy of 15-years-old Italian students: PISA National technical report 2009]. Rome: INVALSI.

30.

Istituto nazionale per la valutazione del sistema educativo di istruzione e di formazione (INVALSI). (2010c). Le prove del Servizio Nazionale di Valutazione 2008–2009 – Analisi Tecnica [Tests of the National Evaluation Service 2008–2009 – Technical Analysis]. Rome: INVALSI.

31.

Kaplan

R. S.

Norton

D. P.

(2005). The office of strategy management. Harvard Business Review, 83(10), 72–80.

32.

Kelly

L. K.

Lezotte

L. W.

(2003). Developing leadership through the school improvement process. Journal of School Improvement, 4(1). Retrieved 12 January 2013, from http://www.ncacasi.org/jsi/2003v4il/develop_leadership.

33.

Kruse

S. D.

(2001). Creating communities of reform: Continuous improvement planning teams. Journal of Educational Administration, 39(4), 359–383.

34.

Lancia

(2004). Strumenti per l’analisi dei testi. Introduzione all’uso di T-LAB [Tools for text analysis. Introduction to the use of T-LAB]. Milan: FrancoAngeli.

35.

Levine

D. U.

Leibert

R. E.

(1987). Improving school improvement plans. The Elementary School Journal, 87(4), 397–412.

36.

Levitt

March

J. G.

(1988). Organizational learning. Annual Review of Sociology, 14, 319–340.

37.

Lockheed

Harris

Jayasundera

(2010). School improvement plans and student learning in Jamaica. International Journal of Educational Development, 30(1), 54–66.

38.

McNamara

(2003). Field guide to nonprofit strategic planning and facilitation. Minneapolis, MN: Authenticity Consulting.

39.

Mintrop

McLellan

A. M.

(2002). School improvement plans in elementary and middle schools on probation. The Elementary School Journal, 102(4), 275–300.

40.

Mintzberg

(1994). The fall and rise of strategic planning. Harvard Business Review, 72(1), 107–114.

41.

(2011). The effect of Appreciative Inquiry as organizational development intervention on organizational planning and service quality improvement in St. Francis School (ICSE). Revista de cercetare si interventie sociala, 33, 27–43.

42.

Opfer

Pedder

D. J.

Lavicza

(2011). The influence of school orientation to learning on teachers’ professional learning change. School Effectiveness and School Improvement: An International Journal of Research, Policy and Practice, 22(2), 193–214.

43.

Organization for Economic Co-operation and Development. (OECD). (2010). Education at a glance. Paris: OECD.

44.

Phillips

P. A.

Moutinho

(2000). The strategic planning index: A tool for measuring strategic planning effectiveness. Journal of Travel Research, 38, 369–379.

45.

Preedy

Glatter

Levaçic

(Eds.) (1997). Educational management: Strategy, quality, and resources. Buckingham: Open University Press.

46.

Ramanujam

Venkatraman

Camillus

J. C.

(1986). Multi-objective assessment of effectiveness of strategic planning: A discriminant analysis approach. The Academy of Management Journal, 29(2), 347–372.

47.

Reeves

D. B.

(2006a). Achievement, equity, and leadership: The Clark County School District commitment to success. Englewood, CO: The Leadership and Learning Center.

48.

Reeves

D. B.

(2006b). The learning leader: How to focus school improvement for better results. Alexandria, VA: Association for Supervision and Curriculum Development.

49.

Reezigt

(2001). A framework for effective school improvement. Final report of the ESI project. Groningen, The Netherlands: GION, Groningen Institute for Educational Research, University of Groningen.

50.

Reezigt

Creemers

(2005). A comprehensive framework for effective school improvement. School Effectiveness and School Improvement: An International Journal of Research, Policy and Practice, 16(4), 407–424.

51.

Robinson

R. B.

Pearce

J. A.

(1983). The impact of formalized strategic planning on financial performance in small organizations. Strategic Management Journal, 4(3), 197–207.

52.

Schechter

Sykes

Rosenfeld

(2004). Learning from success: A leverage for transforming schools into learning communities. Planning and Changing, 35(3/4), 154–168.

53.

Scheerens

(2000). Improving school effectiveness. Paris: UNESCO International Institute for Educational Planning.

54.

Schutz

R. E.

(1980). The design of school improvement efforts. Educational Evaluation and Policy Analysis, 2(6), 31–36.

55.

Silins

Mulford

(2002). Schools as learning organisations: The case for system, teacher and student learning. The Journal of Educational Administration, 40(5), 425–446.

56.

Sirin

S. R.

(2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417–453.

57.

Tashakkori

Teddlie

(1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: SAGE.

58.

Thoonen

E. E. J.

Sleegers

P. J. C.

Oort

F. J.

Peetsma

T. T. D.

(2012). Building school-wide capacity for improvement: The role of leadership, school organizational conditions, and teacher factors. School Effectiveness and School Improvement: An International Journal of Research, Policy and Practice, 1(20), 1–20.

59.

US Department of Education. (2006). LEA and school improvement non-regulatory guidance. Office of Elementary and Secondary Education. Washington, DC: US Department of Education.

60.

Wong

K. K.

(2003). The new politics of urban schools. In Pelissero

J. P.

(Ed.), Cities, politics and policy (pp. 283–311). Washington, DC: CQ Press.