The curriculum in school external evaluation frameworks in Portugal and England

Abstract

The curriculum has been target of social and political demands due to its central role in school education and to the changes that occurred in education over the 20th century. The changes include more autonomy assigned to schools and teachers and the establishment of educational standards. These raised concerns that led European bodies to recommend the implementation of quality assurance systems. As a response, many countries created school external evaluation (SEE) processes in an attempt to improve school education and the curricula. Portugal and England are two countries that followed this tendency. This paper analyses SEE frameworks used in both countries to answer the following question: (1) How is the curriculum being addressed in the SEE in Portugal’s and England’s frameworks? The main conclusions drawn are as follows: (1) the curriculum is conceived as an open project; (2) the SEE frameworks from both countries address the curriculum by focusing on teachers’ role as curriculum developers and their curriculum development processes; and (3) the frameworks from Portugal and England reveal a concern with assessing the actual classroom environment and functioning.

Keywords

Curriculum school external evaluation evaluation frameworks

Introduction

This paper relies on three basic premises: (1) education is an essential part of modern societies and, for this reason, it is a target for quality assurance demands; (2) the curriculum in all of its range is central to the quality of the educational service provided; (3) school evaluation constitutes one of the primary quality assurance procedures.

Education is a key part of every society. It is the process par excellence through which young children grow into citizens, developing the necessary social and personal competencies to actively participate in society (European Union Council, 2009). Through school education, children also acquire knowledge and skills to integrate into the labour market as specialized professionals. Education is therefore essential in sustaining Europe’s social and economic development (European Commission/EACEA/Eurydice, 2015). The curriculum is a very important part of this process and is, therefore, at the centre of concerns and demands. The main concern is related to education quality, as a consequence of the various changes made in European educational systems over the years. Changes such as the decentralization of power and consequent autonomy granted to schools – in the 1990s in Europe (Eurydice, 2007) – led to schools being held accountable and targeted with different modes of regulation, which in a more radical circumstance can be constricting (Afonso, 2014); the phenomenon of globalization that brought to education economic and market principles, such as efficacy and efficiency, pressured schools to perform according to the global agenda, despite the efforts to act locally – autonomously – and contextualize school education.

This scenario led to the development of measures aiming to promote the quality of educational systems, all across Europe, leading many European countries to develop and implement quality assurance systems. School external evaluation (SEE) processes, recommended by European bodies and national and international agencies, were the most common ones (Faubert, 2009).

SEE processes focus mainly on the procedures concerning the verification of to what extent the schools’ work is in compliance with regulations; on the outcomes concerning the results achieved by schools in general and students in particular; or on both the procedures and outcomes, concerning all aspects of school work, from management to the provision of educational service (Faubert, 2009). School evaluation processes that analyse processes and outcomes are more likely to address the curriculum. Countries such as Portugal and England are good examples of this.

These two countries are particularly interesting, since they present a similar journey in educational transformations – school autonomy and decentralization during the final years of the 1980s and 1990s – having engaged in quality assurance systems with the same basic characteristics. In addition, their SEE models are based on the one from Scotland.¹ Investigating the SEE models in Portugal and England can help one to understand how the same European recommendations towards quality assurance through school evaluation are put into practice. It can also provide some clues on the relation between one of the most crucial parts of school education, the curriculum, and the processes of quality assurance. Regarding the curriculum, this can provide some insights on how it is perceived and what is valued in terms of curriculum management and development. This, in turn provides clues on the path to follow in improving the curricula, having the potential to enhance school work and teachers’ practices, if the information is well used.

Based on this idea, this paper analyses the SEE frameworks used in Portugal and England by their inspectorates,² focusing on how the curriculum is addressed. It aims to answer to the following question: (1) How is the curriculum being addressed in the SEE process in Portugal and England’s frameworks? Specifically, the objective is to understand: (a) what conception of curriculum is expressed in SEE processes; (b) how SEE address the curriculum and what it means in curricular development and management.

This paper starts by presenting some considerations of the relation between the curriculum and the SEE processes. Then, it presents the analysis of the SEE frameworks used in both countries, identifying similarities and differences, as well as the key indicators used to assess the curriculum, and interpreting them in the light of the curriculum conception they point to.

The relationship between school external evaluation and the curriculum

In the European context there is a continuous discussion on what should be the goals, functions and achievements of school education, and on the need to ensure its quality, efficiency and efficacy, resulting in recommendations, guidelines, reports and studies developed and published by the Organization for Economic Cooperation and Development (OECD), European Union and Eurydice, among others. European bodies, and researchers, discuss what should be addressed and how it should be developed in the teaching and learning processes, in order to achieve the European goals and standards and, consequently, how to assess the extent to which educational institutions and professionals are delivering the process. According to the Strategic framework – Education & Training 2020 – these are the current four main goals for education in Europe: ‘Making lifelong learning and mobility a reality; Improving the quality and efficiency of education and training; Promoting equity, social cohesion, and active citizenship; Enhancing creativity and innovation, including entrepreneurship, at all levels of education and training’ (European Council, 2009). These goals point in the direction of concerns with the curriculum and the evaluation.

These two processes are central in school education and the relationship between them is quite obvious. It is not possible to evaluate a school’s work without addressing the main processes beneath it, the curricular processes; likewise, it is not possible to ensure curriculum quality and its development without a thorough evaluation of the dynamics involved, especially in a context of constant social change and pressure. But what do these issues mean?

The concept of curriculum has evolved from a traditional perspective that conceives it as a neutral object and a strictly structured set of contents into a conception that sees it as a process, opened to the characteristics of the context and the people involved (Biesta et al., 2015; Doyle, 1992; Fernandes, 2011; Leite, 2002; Leite and Fernandes, 2014; Miller and Seller, 1985; Pacheco, 1996). As Doyle (1992: 487) states, there is ‘a formal curriculum that defines the core substance of schooling and an experienced curriculum that is taught and learned in classrooms’. This point of view contemplates room for the adaptation and contextualization of the curricular contents regarding contextual factors (Fernandes et al., 2012). In summary, the curriculum entails both the disciplinary contents to be learned and the processes through which they are taught. As such, we believe the most appropriate term to use is ‘curricular processes’.³ The official discourse from European bodies concerning curriculum management and development seems to be in line with this conception.

The concept of evaluation, similar to the curriculum, has evolved from being a simple process of performance judgement (Scriven, 1967; Tyler, 1950) to a process for providing information (Stufflebeam et al., 1971), becoming a tool for the diagnosis, analysis and assessment of subjects, situations, contexts and institutions, towards improvement and accountability (Ehren and Swanborn, 2012; Figueroa, 2008; Glasman and Nevo, 1988; Grek et al., 2009; Guba and Lincoln, 1989; Leite et al., 2006; West et al., 2009). It has acquired new features, a new place on the social and educational fabric and is currently seen as an indicator to define improvement as well as a potential promoter of educational quality and efficiency (Toner, 2011). The process of evaluation can be understood as a process for gathering information to analyse and assess the quality, efficiency and adequacy of a situation, a service, an individual or institution’s performance and to identify its strengths, weaknesses and needs, aiming to provide opportunities for improvement.

As in the case of the curriculum, the discourse from European documents presents a similar approach to evaluation (Faubert, 2009). The social, economic and political context, in Europe, in the last two decades, has brought about a number of changes in educational planning and management. It is possible to identify three main trends in European education that clearly exemplify this: (1) the tension between socioeconomic and cultural development and school education; (2) the concerns about the quality and efficiency of the educational service; and (3) the movements of autonomy and accountability issues in school education. These changes have reinforced the relation between curricular processes and school evaluation.

Tension between socioeconomic and cultural development and school education

School education is responsible for preparing students with the conditions to develop a set of skills and competencies to contribute to society (Commission of the European Communities, 2007; European Council and Representatives of the Governments of the Member States, 2009; European Parliament and Council, 2001, 2006). However, what is meant by the ‘necessary skills, knowledge and competencies’ changes over time. The phenomenon of globalization has made it essential to ensure that education is able to respond to the demands of industry, science, research and technology (The Economic and Social Committee, 2000). At the same time, the labour market demands workers with ‘… “functional flexibility” in that their greater stock of knowledge increases the rate at which they learn and develop higher order problem solving skills’ (Toner, 2011: 32), taking into consideration that schools promote students’ development as active citizens (Coleman, 2011) and that it is required for the curriculum to be balanced (Karseth and Sivesind, 2010). However, it seems that school education has not been able to keep up with what is required (European Commission, 2012a; Marchesi, 2002). This raises the need to make sure that the knowledge being taught in school corresponds to what is needed and that it is up to date, making use of processes that are able to shed some light on the matter and provide insights on how this situation can be solved. School evaluation appears a valid option, at this point, since it may provide information on this (Campbell and Levin, 2009; Coe, 2009; Ehren and Swanborn, 2012; Figueroa, 2008; Grek et al., 2009; Plowright, 2007; Reezigt and Creemers, 2005; Sun et al., 2007; West et al., 2009) and show possible alternatives and solutions (Coe, 2009; Hayman and Napier, 1979; Reezigt and Creemers, 2005). Some European discourses also emphasize the usefulness of SEE in this matter, providing similar arguments (European Parliament and Council, 2001; Faubert, 2009). It is our belief that school evaluation can be useful to promote the type of education set for European nations. However, some problems may arise from these processes, as well. School evaluation can be used to enforce some specific conception of education that is not context adequate (nationally). The ideal of a common European educational system, although economically competitive, may fall into radical standardization. Although European documents stress that educational ideals must be adapted, instead of merely adopted, the competitiveness between nations may be a source of pressure and may lead to using school evaluation as a regulatory process (Afonso, 2014).

Concerns with the quality and efficiency of the educational service

The changes mentioned above place school education in the front line of demands for higher quality and for methods of ensuring the quality and effectiveness of the educational service (European Union Council, 2009; European Commission/EACEA/Eurydice, 2015). Consequently, measures were undertaken, such as setting educational quality standards, to be fulfilled by European nations (Alexiadou, 2007; Working Committee on Quality Indicators, 2000).

Also, due to the competitive and demanding features of a globalized world (Dias Sobrinho, 2012), schools are faced with national and international comparisons. As Alexiadou (2007: 107) states, ‘the use of benchmarks and performance indicators has been part of social (and education) policy making in many member states, and also in international organisations’. The comparison focuses on ensuring school quality, at all educational levels, regardless of the differences between nations (European Parliament and Council, 2001). This creates pressure for schools to improve their practices and, consequently, their outcomes in terms of students’ academic results.

In Europe, the pursuit for quality of educational systems and schools has led to the creation and implementation of quality assurance systems. Evaluation processes are the centrepiece of quality assurance systems and it is believed that even though ‘… school evaluation systems vary in their characteristics, they share a common global purpose of improving teaching and learning’ (Faubert, 2009: 6). There is no point in developing such systems if their goal is not to improve the quality of school work and students’ learning outcomes.

Through SEE it is possible to understand how this level of quality is being ensured (Faubert, 2009), especially regarding the quality of classroom work, which is directly linked to students’ outcomes. European recommendations stress that nations ought to use evaluation to assess how educational quality is being ensured (Faubert, 2009) and if educational systems are effective in terms of what they deliver, as well as schools’ management and use of resources (Commission of the European Communities, 2001).

In terms of curricular processes quality, European bodies took turns in producing recommendations, arguing for a learning process in which contents are taught in a meaningful way so that all students can learn at their own pace (European Commission, 2012b; European Council, 2013; European Union Council, 2011; OECD, 2010). This calls for curricular contextualization by using diversified pedagogical approaches that meet students’ needs and provide a significant learning environment (European Commission/EACEA/Eurydice, 2012; Fernandes et al., 2012; Kärkkäinen, 2012). Even countries with a mandatory national curriculum, such as Portugal and England, should take these recommendations into account.

School evaluation processes can be of aid in ensuring the curricular processes are in line with these guidelines, since it is important to recognize that providing ‘… effective links to classroom practice is a key policy challenge in the design of evaluation and assessment framework’ (OECD, 2010: 2), such as the one analysed in this paper, concerning the curriculum. These recommendations serve as a parameter for assessing the quality of school education, therefore being part of school evaluation frameworks in some countries. Despite what has been said about educational quality throughout this section, it seems that this ideal has been replaced by a results-oriented conception. Comparison initiatives based on students’ outcomes, such as PISA, TIMMS, PIRLS and TALIS, disseminated the idea of results as the sole indicator of educational quality. The danger of such an approach is that schools may be driven by performativity towards products, instead of by the aim of global education of their students (Ehren and Swanborn, 2012; Paro, 2011).

Movements of autonomy and accountability in school education

Some of the most important changes in educational management date back to the end of the 20th century, particularly to the late 1980s. Since then, the European context has experienced a transfer of power and responsibilities from the state to public organisms, such as schools. Many countries went from a centralized governing of schools to a decentralization of power and the assignment of more autonomy to schools (Eurydice, 2007). Greater school autonomy is seen as a potential ‘… tool to be used to improve the quality of education (…) Greater attention is paid to pedagogic autonomy which seems more closely linked to raising achievement at school’ (Eurydice, 2007: 12). The main arguments justifying this statement claim that a certain amount of pedagogical autonomy – within a national standardized curriculum – would most likely be able to meet quality goals as it enables one to address specific issues in a more appropriate and effective way (Coe, 2009; Devos and Verhoeven, 2003; Eurydice, 2007; Figueroa, 2008; Gorad, 2010; Grek et al., 2009; Reezigt and Creemers, 2005; Scheerens, 2003; Schildkamp et al., 2012). The role of school evaluation, on this matter, is to provide information on how schools are making use of their autonomy to effectively meet the needs and characteristics of their students and environments, as they are expected to do (Campbell and Levin, 2009; Coe, 2009; Hofman et al., 2009; Plowright, 2007; Reezigt and Creemers, 2005; Sun et al., 2007), guaranteeing a solid foundation for intervention and action aimed at improvement (Campbell and Levin, 2009; Coe, 2009; Hadji, 1994; Hayman and Napier, 1979; Marchesi, 2002; Reezigt and Creemers, 2005).

A note should be made concerning the balance between autonomy and evaluation. More autonomy should mean additional decision-making power for schools in terms of management and in terms of planning and developing the teaching and learning process and the curriculum (Faubert, 2009). However, this is not always, if ever, the case. The autonomy appears to be constricted only to administrative aspects of schools’ functioning in almost all countries, Portugal and England included (Faubert, 2009).

Whereas school autonomy is associated with more context-adequate school and curricular management, what it actually means to schools is many times the opposite. The autonomy of schools gained a heavy counterpart: accountability. Schools are held accountable for their work and for the good use of the autonomy granted to them, before the State and the overall population (Looney, 2009), hence justifying actions and initiatives for assessment (Leite et al., 2014). That is, the ‘increased autonomy has been balanced by the strengthening of accountability through the setting of outcome assessments and national standards that all schools should meet’ (Faubert, 2009: 7). It seems safe to assume that the pressure for higher quality, the setting of standards and the apparent loss of governmental regulation power in school education has led to new levels of control over schools. In this sense, the autonomy that was discursively aimed at providing schools with a more context approach to decision-making power, gave place to a masqueraded autonomy under the scope of verification and assessment.

Again, school evaluation processes may play a part in this. This paper assumes that school evaluation is a positive process and a tool at the service of educational quality and improvement. However, frequently, school evaluation is used as a tool for control and to exert pressure on schools to act according to a checklist of what is commonly considered to be valid schooling (Ball, 2003, 2004). In such circumstances, school evaluation contradicts the premises of autonomy.

Despite the arguments defending the implementation of SEE processes to assess the quality of educational systems, it seems that European nations face some problems in doing so, particularly in what concerns the curriculum (Commission of the European Communities, 2007; Kärkkäinen, 2012). Nonetheless, Portugal and England seem to be two countries that attempt to overcome the lack of evaluation concerning the curriculum. In fact, in an OECD working paper on the influence of accountability measures – within which SEE can be found – in curriculum decision making, it is said that Portugal and England’s SEE processes collect data on the curriculum. Still, the level of influence of the results of this collection is generally non-existent, in Portugal, or low, in England (Kärkkäinen, 2012: 30).

It is based on all these aspects that the study presented in this paper is justified, as it attempts to understand to what extent the curriculum is present in SEE referents used in Portugal and England.

The cases of Portugal and England

Portuguese educational policies have been following European trends, since 1986, when Portugal joined the European Economic Community, and later with globalization (Lawn and Lingard, 2002). During the following years, educational policies in general, and curricular processes in particular, suffered changes, one of the most popular ones being the curricular reform of the late 1990s – the Flexible Curriculum Management project, which aimed for local flexibility of the national curriculum and the curricular reorganization from 2001 (Leite, 2002). Up to today, the Portuguese educational system has travelled from centralized to decentralized and, again, a recentralized rationale (Fernandes, 2008; Teodoro and Estrela, 2010). These changes are also related to what is expected from different national educational systems, within an international and global agenda (Teodoro and Estrela, 2010). What this means in terms of the country context is a certain abandonment of the national identity in order to adopt values and modes of organization that are globally, European, accepted and desired (Ball, 2001; Dale, 2004; Nóvoa and Lawn, 2002; Robertson, 2009; Robertson and Dale, 2001). The strategies used to regulate school education follow the same trend, and school evaluation is another example of this. In Portugal, school evaluation is first referred to in Law 46/86, the Basic Law of the Portuguese Educational System, and has appeared in several laws since then. Since the late 1990s, initiatives have been undertaken concerning the evaluation of schools, but it was not until 2002 that it became a mandatory process (Law 31/2002). This Law was published right after the 2001 recommendation from the European Union on the European cooperation in quality evaluation in school education, which may, again, be a sign of international influences. Finally, in 2006, a SEE was created and implemented, and has been conducted ever since. This process has been developed by the General Inspectorate of Education and Science (IGEC). A team is responsible for gathering and analysing different types of data following an evaluation framework with the domains and subdomains to be assessed, which, again, are in line with the European recommendation of 2001.

In England, educational policies have also travelled from centralized to decentralized governing. Since the 1990s, Local Educational Authorities (LEAs) and schools have been granted more educational decision-making power and control. However, this was more of an administrative autonomy. In terms of the curriculum, it remained centralized, through the national curriculum was implemented in the late 1980s (Education Reform Act of 1988). This reveals a certain resistance to adopt the European trend of local curriculum development and management, which as Oates (2011) argues, might mean that England may be falling short on the European educational scheme and missing valuable lessons and alternatives from only looking inside. Nonetheless, as in the Portuguese case, England is following the European tendency in terms of school autonomy, with new modes of State regulation (Higham and Earley, 2013). Since the 1990s, school autonomy has been associated with school inspection – SEE – in the English landscape. This process was first implemented in 1992, with the Education (Schools) Act 1992, and has been developed by the Office for Standards in Education, Children’s Services and Skills (OFSTED) ever since. However, as in Portugal, before 1992, school inspection was already carried out by the LEAs. The processes of inspection have been changing ever since, particularly in terms of the framework to be followed, which acquires different features almost every year. A major change in the process took place with the Education Act 2005, which ascribed relevance to the school’s self-evaluation processes. This also confirms a trend in Europe (SICI, 2007). Currently, OFSTED’s approach relies on documental analysis, visits to schools and lesson observation, all of it guided by a framework that sets the domains and indicators to be considered.⁴

The Portuguese and English scenarios show that the different national educational changes are, at a certain level, influenced by European trends and demands. It also shows that the same recommendations and trends, such as the autonomy or evaluation of schools, may lead to different local realities according to the specific national contexts.

Methodological approach

This paper, as it was already mentioned, is focused on SEE processes from Portugal and England and is based on documental analysis. Three main reasons justify this choice: (1) both are European countries and, being so, are influenced by the political discourses and social demands in the European context; (2) both have a tradition of school evaluation and their processes present some clear similarities; (3) there are clear differences in the national political and social contexts, which could unveil key aspects to understand the role of school evaluation in promoting educational quality.

Bearing in mind the aim of understanding how the curriculum is addressed in schools’ external evaluation in Portugal and England, the first methodological option fell upon comparative document analysis (Bowen, 2009; Prior, 2008; Schensul, 2008). Although the document analysis on its own may not fully reveal how SEE processes contemplate the curriculum, they can provide clues on the matter. Through the comparative document analysis, it is possible to organize information and identify trends, approaches, concerns and conceptions that characterize the contexts and situations being studied (Bowen, 2009; Prior, 2008; Schensul, 2008). A first step in this process was the selection and gathering of key documents framing the SEE processes in Portugal, by the IGEC, and in England, by OFSTED. A search of these agencies’ websites revealed the external evaluation frameworks (IGEC, 2012; OFSTED, 2013) used by each country. The documents were then analysed through a process of content analysis (Krippendorf, 2003), which started with a first look upon the frameworks to understand their organization in terms of how the curriculum is addressed (see Figure 1). Aiming to identify any reference to the curriculum in the external evaluation frameworks, the analysis contemplated all of the indicators related to the curriculum, in both frameworks. These are indicators assessing curricular offerings as well as indicators assessing teachers’ practices, which are the realization of curriculum management and development processes.

Figure 1.

The curriculum in the frameworks.

The preliminary analysis revealed the extent to which Portugal and England address the curriculum in SEE/inspection and allowed some conclusions to be drawn on what kind of curriculum is considered in the processes.

The curriculum in school external evaluation frameworks from Portugal and England

Before presenting the results of the final analysis, it should be noted that some of the items presented may not be explicit in what concerns curriculum, meaning that the word ‘curriculum’ may not be mentioned. However, as was explained earlier, this paper assumes an open conception of the curriculum, which includes both contents and processes. Therefore, the analysis focused on any aspect that could be related with curriculum planning, development, management and assessment. This analysis is also presented in sections in order to emphasize the most relevant aspects within both frameworks.

The structure of the SEE frameworks

In the Portuguese case, the framework is much more detailed when considering the provision of educational service, in comparison to the English one. As shown in Figure 1, there are three subdomains in which the curricular processes are addressed. Each one of these domains has a number of indicators used to assess teachers’ pedagogical approaches. The level of detail in this framework reveals a concern with three phases of curricular processes: how schools and teachers organize and plan the curriculum; how it is developed and managed; and how it is monitored and assessed. This structure also seems to indicate that IGEC is concerned with specific practices considered positive. Despite the comprehensiveness of the indicators, a question arises as to whether the process itself is conducted to match the deepness of the framework.

OFSTED guidelines have a more general approach to curricular practices, based mostly on one domain in which curricular processes are contemplated. The full scope of curricular issues is concentrated in one domain and a handful of indicators. Although this is not suggestive of a minor concern with the curriculum, it does raise some questions on the depth of the evaluation process concerning the curriculum. Also, the framework seems to address more the intentions and philosophy of teaching and other general concerns.

Focusing on the indicators, it becomes possible to draw on more detailed conclusions.

How is the curriculum conceived in SEE frameworks?

It seems that both IGEC and OFSTED’s processes contemplate the curriculum as an open project containing the contents as well as the management and development practices, which is very much in line with the current conception of the curriculum.

Despite the fact that the contents are nationally prescribed in both countries, in what concerns school evaluation, the important aspects are the ones related with how this national curriculum is developed into a more meaningful curriculum and how learning is enhanced.

The framework used in Portugal presents clear examples of this concern, particularly on the subdomains Planning and articulation and Teaching practices. For instance, in the first subdomain, attention is drawn to Curriculum articulated management, which is related to how the curricular contents in the various subjects or in different levels of the same subject are managed in an articulated meaningful and coherent way; and Curriculum contextualization and flexibility to include the context, which is related to how the national curriculum is adapted to meet students’ needs, to become closer to students’ lives and experiences and, consequently, more meaningful to them. A possible explanation may rest on the belief that by adapting the abstract knowledge that constitutes the curriculum, by contextualizing it, the learning process is facilitated and better results can be achieved (Fernandes et al., 2012). The same can be seen in the remaining indicators in these subdomains, namely Use of information on students’ school course, concerning how the classroom environment and the learning process is enriched by students’ previous knowledge and experiences and, once again, adapt the curriculum based on these factors; Teachers’ collaborative work, related with how teachers work together in ways to improve their teaching practices and promote students’ learning and how they work with the curriculum; Adequacy of educational activities taking into consideration students’ capacities and learning rhythms; Adequacy of educational responses to children and students with educational needs, which clearly points towards how the prescribed curriculum is translated in teachers’ planning of learning activities, and how these activities respect and acknowledge students; Active and experimental teaching and learning activities; Profitability of educational resources and time dedicated to teaching; Monitoring and supervision of the classroom practice, which can be related to how teachers use their time and resources, curriculum included, to promote the teaching and learning process. Each of these items, apart from the fact that curriculum can be worked in a myriad of different ways, enrich and develop through different approaches and resourcing to several aspects and tools. They also seem to value teachers’ use of educational autonomy, in the sense that what is under scrutiny is how they transform the national curriculum. These indicators are also pointing towards the teachers’ role as curriculum developers and co-constructors (Leite, 2002; Mouraz et al., 2013), since the process of contextualization demands and can only be achieved through their autonomy and intervention. These are also in line with European recommendations stressing that such practices are a synonym of curriculum quality (European Commission/EACEA/Eurydice, 2012; Fernandes et al., 2012; Kärkkäinen, 2012).

OFSTED’s framework addresses the domain related to the provision of educational service by focusing on several general indicators covering mostly teachers’ dedication and posture, and some general consideration on teaching strategies to meet all students equally, meaning curriculum management practices. For example, OFSTED inspectors should pay attention to Teaching engagement and how all pupils can be included with work that is challenging enough and meets their needs as identified by teachers; How teachers monitor pupils’ responses in lessons and adapt their approach accordingly, also whether they monitor pupils’ progress over time and use the information well to adapt their planning; and If information at transition points between schools is used effectively so that teachers plan to meet pupils’ needs in all lessons from the outset. These aspects are, to some extent, similar to IGEC’s guidelines related with curriculum adaptation and development, as they assess how teachers plan and develop their lessons and how the curriculum is translated in order to reach all students, acknowledging and respecting their individual characteristics. Related to the teaching strategies, that is, to ways of working the curricular contents, we can find: If teachers assign homework which is in line with the school’s policy and challenges all pupils, especially the most able; and If assessment is frequent and accurate and is used to set challenging work that builds on prior knowledge, understanding and skills.

What is the actual focus of IGEC and OFSTED’s frameworks?

A final conclusion from the analysis regards the actual focus of the school evaluation processes in each country.

As already mentioned the Portuguese framework is set to assess how curriculum development and management is delivered. The following indicators are clear examples of the relevance attributed to teachers’ practices: Use of information on students’ school course; ‘Teachers’ collaborative work; Adequacy of educational activities taking into consideration students’ capacities and learning rhythms; Adequacy of educational responses to children and students with educational needs; and Active and experimental teaching and learning activities.

IGEC also focuses on internal assurance of the quality of teaching and learning, specifically in terms of curriculum development, by focusing on Internal monitoring of curriculum development and Monitoring and supervision of the classroom practice, which are dedicated to assess how schools ensure the quality of classroom work.

The focus of IGEC’s SEE framework on the processes of curriculum development may be representative of an attempt to link evaluation with the actual classroom environment and functioning (OECD, 2010).

OFSTED’s guidelines also focus on teachers’ practices, as expressed in the following indicators: How teachers monitor pupils’ responses in lessons and adapt their approach accordingly, also whether they monitor pupils’ progress over time and use the information well to adapt their planning; If information at transition points between schools is used effectively so that teachers plan to meet pupils’ needs in all lessons from the outset; If teachers assign homework which is in line with the school’s policy and challenges all pupils, especially the most able; and If assessment is frequent and accurate and is used to set challenging work that builds on prior knowledge, understanding and skills.

A great deal of attention is paid to teachers’ dedication to the teaching process and their students, by assessing the level of interest and the expectations teachers show towards their students, as shown below: Teaching engagement and how all pupils can be included with work that is challenging enough and meets their needs as identified by teachers; ‘Teachers have high expectations of all pupils, as well as their position in searching for the best teaching method. OFSTED also takes into consideration the following: If teachers seek to assess the effectiveness of their own teaching and adapt accordingly and If teaching helps to develop a culture and ethos of Scholastic excellence. Regarding teachers’ dedication, there is a need to state that, in what concerns the quality of school education and its adequacy to the requirements, it is a very important factor. According to some researches, teachers’ approach to curricula and their position in the classroom impacts on the learning process (Bustos-Orosa, 2008; Kalbach and Forester, 2006; Paliwal and Subramaniam, 2006). Therefore, this is also an indicator of the quality of teaching and learning and, consequently, schools’ work.

In general terms, the analysis of IGEC and OFSTED’s external evaluation frameworks revealed that IGEC’s framework covers a wide range of key points to be assessed that focus mostly on curriculum development initiatives; OFSTED’s also focus on teachers’ dedication and on the learning environment. All of these items, both in the Portuguese and the English frameworks, reveal an undeniable concern in assessing the curriculum through a set of indicators that are, in turn, closely related to the European orientations for the curriculum (European Commission, 2012b; European Council, 2013; European Union Council, 2011; OECD, 2010).

Despite this difference, since the evaluation/inspection process takes into consideration curriculum management and development or teacher’s position, concerning the students and their characteristics, it can provide insights on the quality of such measures and indicates directions to be followed in order to improve it. SEE, then, reveals if the teaching and learning process, and schools’ work, correspond to the standards. If so, schools should keep up the good work and reinforce their practices. If not, and if the information provided is complete and useful – and not just a list of wrong actions – SEE can give some clues and present options to help change practices and overcome difficulties and flaws. This can culminate in better quality in schools and, particularly, in the curriculum.

Again, the question remains as to whether, or not, the processes are carried out to meet the discourse and guidelines present in the frameworks or if the pressure of competition and to perform according to the European standards and goals leads to deflection towards less a comprehensive and humanistic approach. The fact is that, as many researchers (Ball, 2003, 2004; Ehren and Swanborn, 2012; Paro, 2011; Richards, 2004) have already stated, school quality and academic results – numbers – are now seen as synonyms, which is resulting in evaluation processes and conclusions based solely on students’ achievement rates and grades in national decontextualized examinations.

Also, it is not possible to ignore the question of whether the information collected and the knowledge produced from it is ensuring and improving the quality of the curriculum. It should be recalled here that, as concluded in an OECD working paper (Kärkkäinen, 2012), the impacts of SEE on the curriculum are scarce or yet to be seen. This is a question to be explored in the future.

Final remarks

Throughout this paper two arguments were made evident: on the one hand, the curriculum is central to the educational process and needs to be improved in order to meet social and political demands and reach students. On the other hand, it is necessary to ensure this curricular adequacy and its quality, which can be done through SEE.

As argued in this paper, the processes of curriculum development and management exist in a dichotomy between respecting the contents defined at a ministerial level and reaching out to the diversity of classrooms and students, through curricular autonomy procedures that enable the contextualization of the curriculum prescribed nationally (Cortesão and Stoer, 1996, 1997; Karseth and Sivesind, 2010; Leite, 2002).

In terms of the SEE process, this could mean that two different things need to be assessed: (1) the fulfilment of the curriculum length; and (2) the processes of curriculum contextualization.

The analysis of the evaluation/inspection frameworks seems to reveal that school evaluation processes are more concerned with the second aspect. Most of the indicators used to assess the curriculum are related to teachers’ practices and posture towards the adaptation of contents, the teaching strategies used to provide a more meaningful teaching and learning environment and curriculum. Both countries value how teachers use information on their students, their needs and characteristics, and also their surrounding social and cultural context, to improve students’ learning.

In summary, the analysis showed that the domain dedicated to the curriculum SEE/inspection frameworks in Portugal and England considers the curriculum as a project to be developed in context and emphasizes the use of the autonomy granted to schools; hence, teachers’ active role as curriculum developers. This perspective seems to be in line with what is considered to be, in the literature as well as in political discourses, a means to enhance the quality of the curriculum and school education. If SEE processes are successful at assessing it, and if the information resulting from SEE is used to improve the curriculum and teaching practices, this could make a real impact in schools. In consequence, SEE might achieve its potential in providing educational systems with the conditions to become better.

However, the documental analysis cannot attest to whether, or not, the processes of evaluation are conducted in a way that corresponds to the aims and focuses of the frameworks. Still, it does raise some questions on the depth of the evaluation process concerning the curriculum.

Even though the remaining SEE domains were not a target in this analysis, it is possible to assume that the issue of curriculum fulfilment is contemplated in the ‘Results’ domain. This assumption is based on the knowledge that the referred domain addresses students’ results, including their external examination (national exams). These exams assess students’ knowledge of the entire length and contents of the curriculum.

The understanding of the impacts of SEE in school curricula could be achieved through a research focused on the following questions: (1) What are the actual impacts of SEE in the curriculum development and management processes? (2) Does the process of SEE correspond to the frameworks or is there a deviation between the conceptualization of SEE and its enactment? and (3) What are the consequences of social pressure for students’ higher academic results in national exams in curriculum management?

Answering these questions could help one to understand how SEE processes are being developed and their impact in curriculum management.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is supported by the European Social Fund, under the Human Potential Operational Programme (POPH) from National Strategic Reference Framework, and by the Portuguese Foundation for Science and Technology, through the FCT PhD Programmes [grant number SFRH/BD/86125/2012].

Notes

Author biographies

Carla Figueiredo is a PhD grant student in a doctoral program at the Faculty of Psychology and Educational Sciences at Porto. She has a master’s degree in Educational Sciences. She participated, as a fellow or as a researcher, in several research projects on the subject of curriculum and evaluation. Her main interests focus on educational and curricular policies, curriculum and evaluation and assessment. She is co-author of papers and book chapters on curriculum and evaluation.

Carlinda Leite is Full Professor in the Faculty of Psychology and Educational Sciences at Porto University Portugal. Her main interests and research focus are curriculum studies, educational and curricular policies, teaching and curriculum in higher education and training teachers. She has coordinated several projects in curriculum studies and is the author of several books and articles. She is part of several associations focused on education and on the curriculum.

Preciosa Fernandes is a Professor of Education and Curriculum in the Faculty of Psychology and Educational Sciences at Porto University, Portugal. Her main interests focus on education, curriculum theory, educational and curricular policies and higher education. She is the author and co-author of several scientific papers and books on curriculum and education.

References

Afonso

(2014) Questões, objetos e perspetivas em evaliação. Avaliação 19(2): 487–507.

Alexiadou

(2007) The Europeanisation of Education Policy: Researching changing governance and ‘new’ modes of coordination. Research in Comparative and International Education 2: 102–116.

Ball

(2001) Diretrizes Políticas Globais e Relações Políticas Locais em Educação. Currículo sem Fronteiras 1(2): 99–116.

Ball

(2003) The teacher’s soul and the terrors of performativity. Journal of Education Policy 18(2): 215–228.

Ball

(2004) Performatividade, Privatização e o Pós-Estado do Bem-Estar. Educação e Sociedade 25: 1105–1126.

Biesta

Priestley

Robinson

(2015) The role of beliefs in teacher agency. Teachers and Teaching: Theory and Practice 21: 624–640.

Bowen

(2009) Document analysis as a qualitative research method. Qualitative Research Journal 9(2): 27–40.

Bustos-Orosa

(2008) Inquiring into Filipino teachers’ conceptions of good teaching: A qualitative research study. The Asia-Pacific Education Researcher 17: 173–189.

Campbell

Levin

(2009) Using data to support educational improvement. Educational Assessment Evaluation and Accountability 21: 47–65.

10.

Coe

(2009) School improvement: reality and illusion. British Journal of Educational Studies 57: 363–379.

11.

Coleman

(2011) O conceito de igualdade de oportunidades educacionais. Educação, Sociedade & Culturas 34: 137–155.

12.

Commission of the European Communities (2001) Report from the commission on the concrete future objectives of education systems. Brussels. Available at: http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52001DC0059&from=EN (accessed 15 January 2015).

13.

Commission of the European Communities (2007) Schools for the 21^st century – Commission staff working paper. Brussels. Available at: http://ec.europa.eu/danmark/documents/alle_emner/information/070712_skole-hoering.pdf (accessed 15 January 2015).

14.

Cortesão

Stoer

(1996) A interculturalidade e a educação escolar: dispositivos pedagógicos e a construção da ponte entre culturas. Inovação 9: 35–51.

15.

Cortesão

Stoer

(1997) Investigação-acção e a produção de conhecimento no âmbito de uma formação de professores para a educação inter/multicultural. Educação, Sociedade & Culturas 7: 7–28.

16.

Dale

(2004) Globalização e educação: demonstrando a existência de uma “cultura educacional mundial comum” ou localizando uma “agenda globalmente estruturada para a educação”? Education and Society, Campinas 25(87): 423–460.

17.

Devos

Verhoeven

(2003) School self-evaluation-conditions and caveats: The case of secondary schools. Educational Management, Administration & Leadership 31: 403–420.

18.

Dias Sobrinho

(2012) Políticas y conceptos de calidad: dilemas y retos. Avaliação 17: 601–618.

19.

Doyle

(1992) Curriculum and pedagogy. In: Jackson

(ed.) Handbook of Research on Curriculum: A Project of the American Educational Research. New York: Macmillan.

20.

Ehren

Swanborn

(2012) Strategic data use of schools in accountability systems. School Effectiveness and School Improvement 23: 257–280.

21.

European Commission (2012a) Rethinking education: Investing in skills for better socio-economic outcomes – Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions. Available at: http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52012DC0669&from=EN (accessed 16 January 2015).

22.

European Commission (2012b) Annual Growth Survey 2013. Available at: http://ec.europa.eu/europe2020/pdf/ags2013_en.pdf (accessed 30 March 2015).

23.

European Commission/EACEA/Eurydice (2012) Developing Key Competences at School in Europe: Challenges and Opportunities for Policy. Eurydice Report. Luxembourg: Publications Office of the European Union.

24.

European Commission/EACEA/Eurydice (2015) Assuring Quality in Education: Policies and Approaches to School Evaluation in Europe. Eurydice Report. Luxembourg: Publications Office of the European Union (accessed 30 May 2015).

25.

European Council (2013) Council conclusions on investing in education and training: A response to ‘Rethinking Education: Investing in skills for better socio-economic outcomes’ and the ‘2013 Annual Growth Survey’. Available at: http://www.consilium.europa.eu/uedocs/cms_data/docs/pressdata/en/educ/135467.pdf (accessed 30 March 2015).

26.

European Council and Representatives of the Governments of the Member States (2009). Conclusions of the Council and of the Representatives of the Governments of the Member States, meeting within the Council, of 26 November 2009 on developing the role of education in a fully- functioning knowledge triangle. Available at: http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:42009X1212(01)&from=EN (accesssed 13 February 2015).

27.

European Parliament and Council (2001) Recommendation of the European Parliament and of the Council of 12 February 2001 on European cooperation in quality evaluation in school education. Official Journal of the European Communities L60. Available at: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2001:060:0051:0053:EN:PDF (accessed 15 February 2015).

28.

European Parliament and Council (2006) Recommendation of the European Parliament and of the Council of 18 December 2006 on key competences for lifelong learning. Official Journal of the European Communities L394. Available at: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:394:0010:0018:en:PDF (accessed 15 January 2015).

29.

European Union Council (2009) Council conclusions of 12 May 2009 on a strategic framework for European cooperation in education and training (‘ET 2020’). Official Journal of the European Union 119/2. Available at: http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52009XG0528%2801%29&from=EN (accessed 15 January 2015).

30.

European Union Council (2011) Council conclusions on the role of education and training in the implementation of the ‘Europe 2020’ strategy. Available at: http://www.consilium.europa.eu/uedocs/cms_data/docs/pressdata/en/educ/119282.pdf (accessed 15 January 2015).

31.

Eurydice (2007) School autonomy in Europe. Policies and measures. Available at: http://www.eurydice.org.

32.

Faubert

(2009) School evaluation: Current practices in OECD countries and a literature review, OECD Education Working Papers, 42, OECD Publishing. Available at: https://dx-doi-org.web.bisu.edu.cn/10.1787/218816547156 (accessed 15 January 2015).

33.

Fernandes

(2008) Perspectivas curriculares de suporte ao ensino básico no Sistema Educativo Português (1989–2005). In: Lopes

Leite

(orgs) Políticas educativas e dinâmicas curriculares em Portugal e no Brasil. Porto: LivPsic.

34.

Fernandes

(2011) O currículo do ensino básico em Portugal: políticas, perspectivas e desafios. Porto: Porto Editora.

35.

Fernandes

. (2012) Curricular contextualization: tracking the meanings of a concept. The Asia-Pacific Education Researcher 22: 417–425.

36.

Figueroa

(2008) Modelo de autoevaluación institucional global con apoyo externo en un centro educativo particular de Lima. Un estudio de caso. Educación 17(33): 63–80.

37.

Glasman

Nevo

(1988) Evaluation in Decision Making: The Case of School Administration. Boston, MA: Kluwer.

38.

Gorard

(2010) Serious doubts about school effectiveness. British Educational Research Journal 36: 745–766.

39.

Grek

. North by northwest: Quality assurance and evaluation processes in European education. Journal of Education Policy 24: 121–133.

40.

Guba

Lincoln

(1989) Fourth Generation Evaluation. Newbury Park, California: Sage Publications.

41.

Hadji

(1994) A avaliação, regras do jogo: das intenções aos instrumentos. Porto: Porto Editora.

42.

Hayman

Napier

(1979) Avaliação nas escolas: Um processo humano para a renovação. Coimbra: Livraria Almedina.

43.

Higham

Earley

(2013) School autonomy and government control: School leader’s views on changing policy landscape in England. Educational Management Administration Leadership 41(6): 701-717.

44.

Hofman

Dijkstra

Hofman

(2009) School self-evaluation and student achievement. School Effectiveness and School Improvement 20: 47–68.

45.

IGEC (2012) Quadro de referência para a avaliação externa das escolas. Lisboa.

46.

Kalbach

Forester

(2006) The world and the world: A lesson in critical literacy and its impact on student achievement and self-esteem. Curriculum and Teaching Dialogue 8: 69-82.

47.

Kärkkäinen

(2012) Bringing about curriculum innovations: Implicit approaches in the OECD area, OECD Education Working Papers; 82, OECD Publishing. Available at: https://dx-doi-org.web.bisu.edu.cn/10.1787/5k95qw8xzl8s-en (accessed 16 January 2015).

48.

Karseth

Sivesind

(2010) Conceptualising curriculum knowledge within and beyond the national context. European Journal of Education 45: 103–120.

49.

Krippendorf

(2003) Content Analysis: An Introduction to its Methodology. Beverly Hills, CA: Sage Publications.

50.

Lawn

Lingard

(2002) Constructing a European policy space in educational governance: The role of transnational policy actors. European Educational Research Journal 1(2): 290–307.

51.

Leite

(2002) O currículo e o Multiculturalismo no Sistema Educativo Português. Lisbon: Fundação Calouste Gulbenkian.

52.

Leite

Fernandes

(2014) Avaliação, qualidade e equidade. Avaliação 19: 421–438.

53.

Leite

Morgado

Seabra

(2014) External school evaluation in Portugal: A glance at the impacts on curricular and pedagogical practices. European Journal of Curriculum Studies 1: 33–43.

54.

Leite

Rodrigues

Fernandes

(2006) A auto-avaliação das escolas e a melhorias da qualidade da educação: Um olhar reflexivo a partir de uma situação. Revista de Estudos Curriculares 4: 21–45.

55.

Looney

(2009) Assessment and innovation in education, OECD Education Working Papers; 24, OECD Publishing. Available at: https://dx-doi-org.web.bisu.edu.cn/10.1787/222814543073 (accessed 16 January 2015).

56.

Marchesi

(2002) Mudanças educativas e avaliação das escolas. In: Azevedo

(org.) Avaliação das escolas: Consensos e divergências. Porto: Edições ASA.

57.

Miller

Seller

(1985) Curriculum: Perspectives and Practices. New York: Longman.

58.

Mouraz

Leite

Fernandes

(2013) Teachers’ role in curriculum design in Portuguese schools. Teachers and Teaching: Theory and Practice 19(5): 478–491.

59.

Nóvoa

Lawn

(eds) (2002) Fabricating Europe – The Formation of an Education Space. Dordrecht/Boston, MA/London: Kluwer Academic Publishers.

60.

Oates

(2011) Could do better: Using international comparisons to refine the National Curriculum in England. The Curriculum Journal 22(2): 121–150.

61.

OECD (2010) Ministerial report on the OECD Innovation Strategy Innovation to strengthen growth and address global and social challenges: Key findings. Available at: http://www.oecd.org/sti/45326349.pdf (accessed 15 January 2015).

62.

OFSTED (2013) The framework for school inspection. London.

63.

Pacheco

(1996) Currículo: Teoria e praxis. Porto: Porto Editora.

64.

Paliwal

Subramaniam

(2006) Contextualising the curriculum. Contemporary Education Dialogue 4: 25–51.

65.

Paro

(2011) Progressão continuada, supervisão escolar e avaliação externa: implicações para a qualidade do ensino. Revista Brasileira de Eduação 16(48): 695–716.

66.

Plowright

(2007) Self-evaluation and OFSTED Inspection: Developing an integrative model of school improvement. Educational, Management, Administration and Leadership 35: 373–393.

67.

Prior

(2008) Document analysis. In: Given

(ed.) The Sage Encyclopedia of Qualitative Research Methods. Vol 1-2. London: Sage, pp.230–232.

68.

Reezigt

Creemers

(2005) A comprehensive framework for effective school improvement. School Effectiveness and School Improvement 16: 407–424.

69.

Richards

(2004) Towards respectful inspection – A critique of proposed arrangements for school inspection and self-evaluation. Education Review 18(1): 42–48.

70.

Robertson

(2009) Europe, competitiveness and higher education. In: Dale

Robertson

(ed.) Globalisation and Europeanisation in Education. Oxford: Symposium Books, pp.65–84.

71.

Robertson

Dale

(2001) Regulação e risco na governação da educação: Gestão dos problemas de legitimação e coesão social em educação nos estados competitivos. Educação, Sociedade & Culturas 15: 117–147.

72.

Scriven

(1967) The methodology of evaluation. In: Tyler

Gagné

Scriven

(eds) Perspectives of curriculum evaluation, pp. 39–83.

73.

Scheerens

(2003) Melhorar a Eficácia das Escolas. Porto: Edições ASA.

74.

Schensul

(2008) Documents. In: Given

(ed.) The Sage Encyclopedia of Qualitative Research Methods. Vol 1-2. London: Sage Publications, p.232.

75.

Schildkamp

. (2012) The use of school self-evaluation results in the Netherlands and Flanders. British Educational Research Journal 38: 125–152.

76.

SICI – The Standing International Conference if Inspectorates (2007) Self-evaluation in Thirteen Countries. Brussels: Directora-General for Education and Culture.

77.

Stufflebeam

. (1971) Educational Evaluation and Decision Making. Itasca, IL: Peacock.

78.

Sun

Creemers

Jong

(2007) Contextual factors and effective school improvement. School Effectiveness and School Improvement 18: 93–122.

79.

Teodoro

Estrela

(2010) Curriculum policy in Portugal (1995–2007): Global agendas and regional and national reconfigurations. Journal Curriculum Studies 42(5): 621–647.

80.

The Economic and Social Committee (2000) Opinion of the Economic and Social Committee on the ‘Proposal for a Recommendation of the European Parliament and of the Council on European cooperation in quality evaluation in school education’. Official Journal of the European Communities C 168/30.

81.

Toner

(2011) Workforce skills and innovation: An overview of major themes in the literature. Available at: https://search.oecd.org/innovation/inno/46970941.pdf (accessed 13 February 2015).

82.

Tyler

(1950) Basic principle of curriculum and instruction. Chicago: Chicago University.

83.

West

Mattei

Roberts

(2009) Accountability and sanctions in English schools. British Journal of Educational Studies 59(1): 41–62.

84.

Working Committee on Quality Indicators (2000) Quality of school education: Sixteen quality indicators.