PISA’s colonialism: Success,money,and the eclipse of education

Abstract

This article provides a critical analysis of the OECD’s Programme for International Student Assessment (PISA) based on an examination of the OECD’s public documents, including publications, webpages, and videos. Based on this analysis, I argue that PISA is not an assessment tool but rather an all-encompassing framework that intends to govern education and schooling worldwide. PISA, de facto, allows a monopoly on the right to establish who is well prepared for life, who is well prepared for society, and who can achieve success. Far from being only an assessment tool, PISA is a life brand. In a strong (although hidden) chain, the OECD identifies education with learning, learning with assessment, and assessment with PISA’s test. Thus, PISA, in the OECD’s opinion, signifies education. The OECD – at the very heart of its colonialist nature – expropriates culture and knowledge from subjects, denying their legitimacy and imposing the OECD’s own univocal logic. Under the aegis of objectivity, PISA manifests a clear ideology and situates education in a well-defined value square: money, success, evidence, and competition. This situation raises substantial doubts concerning a tool that claims to be “a mirror” of education.

Keywords

Pisa's Colonialism education assessment Foucault

Introduction

The OECD (Organisation for Economic Co-operation and Development) was established in 1961 as an enlargement of the OEEC (Organisation for European Economic Cooperation). The declared intent of the OECD is to promote policies that encourage the growth of member economies and the development of trade and the world economy. Both the OEEC and the OECD developed the goals established by the 1944 Bretton Woods agreement, which defined the post-war global order of international economic relations. The establishment of a new global order after the Second World War led to the birth of several powerful organisations, such as the International Monetary Fund (IMF), the World Trade Organization, and the World Bank. The roles of these organisations have become increasingly important since their formation.

According to Carroll and Kellow (2011) and Eccleston (2011), the OECD underwent important transformations from the previous organisation (OEEC) and became an important link in transgovernmental politics; indeed, it was expected to exert pressure upon member countries. The professed intention to foster cooperation and dialogue among member states has increasingly disappeared and has been replaced by the will to exert power upon member states (Woodward, 2009).

With the collapse of the Soviet bloc and the end of the Cold War, the OECD enlarged and reinforced its influence. The OECD reconstituted itself as a centre of policy expertise and comparative international data based on programmes of measurement, comparison, and analysis (Jakobi and Martens, 2010). The OECD also exercises its influence on non-member states (Sellar and Lingard, 2013), enhancing its relationships with “BRIC” nations (Brazil, Russia, India, China). Throughout this period, the role of the OECD has increasingly moved toward the economic dimension. According to Sellar and Lingard, “[t]he slippage between the use of ‘nations’ and ‘economies’ in OECD discourse is symptomatic of its largely economic focus and the ways in which other policy domains, including education, are framed in this way” (Sellar and Lingard, 2013: 17). In addition to the increasing influence of the OECD on member and non-member states, we must consider the sudden growth of education as a focus of interest within the OECD. From the 1990s to the present, furnishing data on the performance of educational systems has become a “core business” of the OECD.

In 2000, the OECD launched PISA (Programme for International Student Assessment) as a triennial assessment programme, and in 2002 education was established as an autonomous directorate. Since then, the importance of PISA has grown consistently (Henry et al., 2001; Lawn and Lingard, 2002). Under the direction of Angel Gurrìa, the OECD’s Secretary-General since June 2006, the OECD has reinforced PISA’s role as not merely an assessment tool with a specific goal, but something very different.

In the OECD’s own words:

The Programme for International Student Assessment (PISA) is a triennial international survey which aims to evaluate education systems worldwide by testing the skills and knowledge of 15-year-old students. To date, students representing more than 70 economies have participated in the assessment. PISA is unique because it develops tests which are not directly linked to the school curriculum. The tests are designed to assess to what extent students at the end of compulsory education, can apply their knowledge to real-life situations and be equipped for full participation in society. (OECD, 2014a, emphasis added)

In this article, I intend to provide a critical analysis of PISA’s stance and hidden ideology. To this end, two elements of the statement above should be considered: the use of the term “economies” in place of “countries” and the testing of 15-year-old students’ skills “for full participation in society”.

In the OECD’s words, PISA has not only become “the world’s premier yardstick for evaluating the quality, equity and efficiency of school systems” (OECD, 2014f: 2), but also an all-encompassing framework that intends to govern educational processes worldwide (OECD, 2004, 2010a, 2014a). In the OECD’s words, PISA “shows countries … how effectively they educate their children” (OECD, 2014a). Furthermore, “PISA tests provide a mirror to all countries and demonstrate what is possible” (OECD, 2014b). PISA tests show whether and to what extent “boys and girls [are] prepared for life” (OECD, 2014b), “measur[ing] whether 15-year-olds around the world are well-prepared to participate in society” (OECD, 2014a). This objective involves several risks.

PISA is anything but neutral. Under the aegis of objectivity, PISA manifests a clear ideology, situating education in a well-defined value square of money, success, evidence, and competition. PISA conceives education – and life – mainly in terms of individual economic success (OECD, 2014f; Pearson Foundation and OECD, 2014). Thus, in PISA’s philosophy, education is education for success (OECD, 2014a, 2014b), and success, in a global economy, is measured in monetary terms. Moreover, this ideology is hidden. In the OECD’s authoritative words, PISA produces evidence (OECD, 2014b, 2014f: 2). By providing “evidence” regarding educational systems worldwide, PISA is in the position to say who is right and wrong, who should have the resources to work, and who is “well prepared for life”, including governments, teachers, and families. In this way, PISA seems to be more of a life brand than an assessment tool, and one which makes expansive claims. In a clear chain, I will argue how the OECD identifies education with learning, learning with assessment, and assessment with PISA. Thus, PISA signifies education. In doing so, the OECD tends to expropriate subjects and communities of their knowledge and culture, denying their legitimacy. This is a colonialist stance.

I make these points by analysing some of the main OECD public documents, including publications, webpages, and videos. I argue that PISA’s declared ideology presents several contradictions. The contradictions can be divided into two types: (a) contradictions between what PISA claims to produce and what it actually produces, and (b) contradictions within PISA, or contradictions between different PISA statements. This situation raises substantial doubts concerning the influence of a tool that claims to be “a mirror” of education worldwide (OECD, 2014b).

PISA’s square: money, success, evidence, and competition

More and more countries are looking beyond their own borders for evidence of the most successful and efficient policies and practices. Indeed, in a global economy, success is no longer measured against national standards alone, but against the best-performing and most rapidly improving education systems. Over the past decade, the OECD Programme for International Student Assessment, PISA, has become the world’s premier yardstick for evaluating the quality, equity and efficiency of school systems. However, the evidence base that PISA has produced goes well beyond statistical benchmarking. By identifying the characteristics of high-performing education systems PISA allows governments and educators to identify effective policies that they can then adapt to their local contexts. (OECD, 2014f: 2)

This is Gurrìa’s well-known opening statement from “PISA Results in Focus: What 15-year-olds know and what they can do with what they know”. If the OECD Secretary-General’s statement on the first page of a main PISA document has importance, then this statement is a significant example of the “essence” of PISA. Several assumptions are included in this statement; some are explicit, and others are implicit.

The statement itself is anything but neutral and innocent. A powerful direction situates education in a well-defined value square of money, success, evidence, and competition. Schematising my reasoning, three elements are significant. First, according to PISA, education – and life – is conceived and enacted in terms of economics, competition, and success. Education is for success, and success, in a global economy, is measured in monetary terms. Second, PISA is neither only an international survey nor an assessment tool. In Gurrìa’s words, PISA produces evidence or indisputable facts. Who has given PISA this licence (Gorur, 2011)? Finally, there is no trace of any of this. It is simply predetermined by assumption that success, money, and competition are the aims of education, and that education is a function of the global economy (Au, 2011; Wolf, 2002). PISA clearly exhibits a strong nexus with the neo-liberal globalisation “narrative”. Furthermore, it is assumed that these features are fully measurable and that PISA is the best tool for this measurement. The questionable reasoning in Gurrìa’s statement is neither questionable nor reasoning; it is a matter of “evidence”. Here, we encounter PISA’s basic feature, namely, its colonialist stance.

I argue that PISA’s colonialism is not related to arguments, such as immigrant students’ learning outcomes or intercultural analyses, although these are significant questions that PISA raises (Pereira et al., 2011; Shohamy, 2004). Colonialism is the very nature of PISA for reasons that are completely internal to PISA’s philosophy. PISA, in Gurrìa’s authoritative words, is the unique model of assessment. In a strong – although hidden – chain, PISA identifies education with learning, learning with assessment, and assessment with its test. Thus, PISA signifies education. Of course, PISA states that every human being – “not just those who come from wealthier or more intellectual or more culturally sophisticated families” (OECD, 2014c) – can yearn for excellence and achieve a high PISA score. However, this achievement is only possible within the confines of PISA. Furthermore, no one can know PISA’s individual outcomes – not students, teachers, or schools. Their role is strictly limited to taking the test. I make these points by analysing four documents: three directly produced by the OECD and one research study on PISA.

The first document is an OECD video that presents the PISA test, with the title “PISA – Measuring student success around the world” (OECD, 2014a). I quote some significant passages and then analyse them:

The OECD brings together 34 countries with the aim of developing better policies for better life. In the late 1990s countries that are members of OECD came up with the idea to measure whether 15-year-olds around the world are well-prepared to participate in society … PISA is less interested in knowing whether students can repeat – like parrots – what they have been taught in class. Rather, the survey is designed to find out whether, for example, students can use the reading skills they have learned at school to make sense of the information they find in a book, a newspaper, on a government form or in an instruction manual … It also helps government, educators and parents track their country’s progress to a more successful education system … PISA shows countries … how effectively they educate their children.

In this statement, at least two related elements in the OECD’s words are significant. First, the OECD develops policies “for [a] better life”. Second, PISA does not assess students’ competence regarding mathematics or problem-solving. PISA establishes whether students are “well-prepared to participate in society”, “show[ing] countries … how effectively they educate their children”. Thus, PISA is no mere tool. PISA can decide if a student can gain a place in the world because the OECD has the key to a “better life”. PISA can also determine whether a student is fit to live with other human beings or participate in society. The idea of “measur[ing] whether 15-year-olds around the world are well-prepared to participate in society” entails several risks.

Participating in society is a complex matter. It is not predetermined that this matter is translatable into a unique set of competences and that this set is assessable. Is it possible to establish a definition of what it means to be “well-prepared to participate in society”? Who or what should be the authority that gives this licence to citizens worldwide? What is the idea of society against which PISA judges this preparation? Focusing only on the West, there are very different ideas about what is considered a “good society” and what it means to be “a good citizen” (Biesta, 2007; Giroux, 1981, 1989; Levinson, 2011; Onosko, 2011; Torres, 1998). A response to these issues from the OECD is needed.

The second document of my analysis is PISA’s homepage, which is even more explicit about PISA’s goal. Because of space limitations, I quote only two statements that show how PISA intends to be an all-encompassing framework for education – and life – worldwide: “These PISA results reveal what is possible in education” and “[a]re boys and girls equally prepared for life?” (OECD, 2014b). It is not difficult to identify the underlying assertions that PISA tests a student’s preparation for life and that PISA demonstrates what education is.

The third document I consider is the homepage of the Pearson Foundation, which works in close collaboration with the OECD. Here, we can read the following statement:

Starting from very different levels, a number of countries and regions have succeeded over the last few years in raising their students’ performance substantially. They display some important common features. Their politicians and social leaders share with parents, teachers and students a strong belief in the value of education. Resources are channelled to the areas where they will provide the greatest results. […] By showing what they achieve, the PISA tests provide a mirror to all countries and demonstrate what is possible. (Pearson Foundation and OECD, 2014)

In this statement, several concepts are dangerous, such as “PISA tests provide a mirror to all countries and demonstrate what is possible”. This statement is both fascist and inadequate. It is fascist because the nature of fascism lies in the certainty that there is one truth and only one truth. The violence of fascism is, ultimately, a consequence of this violence of thought. Each of us expresses opinions, albeit scientifically argued. However, individual expression is not the case with PISA, which is “a mirror” that “demonstrate[s] what is possible”. Furthermore, the statement is inadequate because thinking that one has “a mirror” of reality is, in the best case, a medieval epistemological stance. Another statement – “[r]esources are channelled to the areas where they will provide the greatest results” – is likely less evident in its consequences but is nonetheless dangerous. Who judges the areas of greatest results? PISA does, of course. Assuming that PISA has a method to establish, with no room for error, the areas that provide “the greatest results”, what happens to the other areas? What is their destiny? The message is clear: if you do not improve according to PISA, do not expect to receive any resources.

The fourth document of my analysis is a book that “summarises central outcomes of the PISA Research Conference 2009” (Prenzel, 2013: v). Here, we find an overview of the studies concerning PISA, including “research for PISA …, research with PISA … [and] research on PISA” (p. xvi). There are no references to critical or challenging studies regarding PISA or to studies that go beyond the limits of PISA or raise concerns regarding it. The following passage is significant:

Limitations due to the design of studies are natural for researchers. In the context of PISA, such limitations have to be communicated to policymakers and the public. As PISA may identify severe problem areas in educational systems, researchers may feel motivated to do more and specific research that helps to go beyond such boundaries of the usual PISA design. (Prenzel, 2013: 14--15)

Two ideas must be emphasised here. First, the “natural limitations” that are intrinsic to any research effort are already contained in PISA’s boundaries. Second, “research that helps to go beyond such boundaries of the usual PISA design” is “specific”. It is assumed that research relies on the ongoing debate between different paradigms and models. If you only proceed according to one model, it is difficult to see its limitations from within. It is also presumed that every model is situated within a perceptual field in which some things are visible and some things are not. The entire movement of twentieth-century epistemology from Dewey (1929a, 1929b, 1938) to Foucault (1973 [1969], 2002 [1969]) has shown that a scientific model is, above all, defined by the phenomena at stake. The second concept is perhaps more subtle but equally important: you can go beyond PISA’s boundaries, but only within PISA’s boundaries. Further studies, rather than challenging PISA and broadening its limits, must be “specific”, moving within the space PISA permits. Furthermore, if you do not play “for” PISA and “with” PISA, you do not play at all. This issue is even more important given PISA’s claim to represent neutral, objective, independent research, and its intention to reflect education worldwide. In a certain sense, PISA epitomises the inner violence of the Enlightenment project (Horkheimer and Adorno, 2002 [1944]; Rancière, 2003 [1983]). This violence is closely related to PISA’s imposition of its own values as unique and unavoidable, and with its simultaneous concealment of the ethical dimension.

Money and success: What else?

At the very heart of PISA is the desire to subsume every educational aim, possibility, or value into its own model. PISA does this in a silent way because we find no trace of the identification of PISA’s values with “the” values. It is predetermined, a matter of “evidence”. What are PISA’s values, given that PISA’s model is framed according to those ends? PISA clearly defines these values: success and money. The concept of success, conceived in economic terms, is the focus of PISA. The OECD states this concept in many ways. In the following paragraphs, I examine three documents: the homepage of PISA, a video of Gurrìa produced by the OECD and the Pearson Foundation, and an OECD 2011 publication.

The homepage of PISA starts with the simple question, “Do students have the drive to succeed?” It immediately continues: “How many times have you heard successful people, in all walks of life, credit their triumphs to hard work and perseverance? Now, PISA adds to the chorus with some hard evidence” (OECD, 2014b). Of course, I am not opposed to success, and everyone most likely strives for it, but there are several problems with the homepage’s statement. What is success? In what terms can I regard it? Moreover, is it appropriate for an educational programme to sing its praises? Is it correct to speak about success as a value in itself, with no further clarification? Can success really be the primary index of a good education? Good education is an undefined – and most likely indefinable – concept (Biesta, 2012; Dewey, 1938; Peters, 1966). However, success most likely is even more ambiguous. Who are “successful people”? Were Stalin or Francisco Franco successful men? A con man, too, can be very successful in his activity; to be a good con man you must be sly and acquire several competences. How does education relate to success? I have no provocative intention, and I am committed to every good objective, just like PISA’s supporters, but by framing education and learning in terms of success, PISA commits a basic mistake in foregrounding a dimension that is not related to ethics. Admittedly, ethics is an uncertain dimension, but we have little choice. Either we live with ethics as our common ground, or we proceed with very uncomfortable paradoxes.

Here, we arrive at an ordinary argument that is underestimated by the OECD. The foundation of PISA is predetermined, not only in its method but also in its ethical choice. PISA, in its plea for success, makes a clear value choice. There is nothing necessarily wrong here; in education, everyone makes choices. However, the first ethical and scientific rule is to be aware of our choices. To the contrary, PISA “appears to have become a modern day Delphic Oracle … Appearing unbiased and neutral, it speaks in a detached manner in terms of ‘facts and figures’, giving policy makers information about their own countries from its lofty vantage point” (Gorur, 2011: 77). In addition, PISA is grounded in a value choice. PISA is completely ignorant of – or intentionally omitting – this grounding. PISA’s values are clearly stated as follows: an economic-based concept of life; competition; effectiveness, which is also a value; and the belief that this is the only possible world and that PISA is the only effective tool to assess and improve education. The fundamental weakness of PISA is its lack of awareness; its very certainty represents a universal standard that reveals a colonialist stance.

The second document that I analyse is a video of Gurrìa produced by the Pearson Foundation and the OECD (OECD, 2014e; Pearson Foundation and OECD, 2014). The video is a meaningful discourse on PISA’s goals. I quote the main passages as follows:

In the global economy, work … can be done everywhere, by whoever does it best, whoever provides best value for money – and value comes increasingly from knowledge … Citizens are to be able to create, absorb, adapt and use knowledge. To do so, they need the right skills. Investing in education to give … the skills [young people] will need in tomorrow’s economy is essential for development and growth. But in a competitive and globalised world, achievement is relative; success can no longer be measured just by low goals … or national standards … Countries are to measure themselves against the best performing education systems internationally … The students are tested not just on what they have been taught but on how creative they can be in finding solutions to unexpected problems. PISA tests the readiness for an active role in today’s high-tech society; it tests how they think and how they work … PISA shows what achievements are possible in education.

The phrase “the students are tested” is an objectification of human beings. If your philosophy is “to test people”, we all know what your position towards people can be. This philosophy is even more significant if the statement is not contextualised. If you say, for example, “I am testing people for this work” or “I am testing people regarding this task”, the intention is quite different; you are testing people about something. This is not the case with PISA, which tests people on life, with instruments which can tell if you are well prepared to assume a position in society and the world; in this sense PISA seems to be a life brand. What are the consequences of this attitude for a 15-year-old? What attitudes does PISA promote beyond success and competition? With PISA, there is no trace of other values. Of course, Gurrìa mentions equity, but it is always the same: countries must pursue equity only according to PISA. According to PISA, everyone can perform, including disadvantaged people. A comparison between the results of PISA 2012 and the table indicating countries’ Gross Domestic Product (GDP) per capita of the IMF 2011–2012 shows a disturbing connection between prosperity and PISA scores. However, this connection is not the central point. The central point is that you can perform only on the condition that you adjust to PISA’s philosophy. Everyone can play, but only by PISA’s rules.

Gurrìa’s statement that PISA should show “[h]ow [students] think and how they work” also demonstrates inconsistency. Significantly, these dimensions are completely beyond PISA. In what way can I understand how students work and how students think with PISA? The meta-cognitive dimension and qualitative appreciation are beyond the boundaries of PISA (Bonderup Dohn, 2007).

Another example of PISA’s stance is in a 2011 OECD publication, where the following passage can be found:

Globalisation and modernisation are rapidly posing new and demanding challenges to individuals and societies alike. Increasingly diverse and interconnected populations, rapid technological change in the workplace and in everyday life, and the instantaneous availability of vast amounts of information are just a few of the factors contributing to these new demands. In this globalised world, people compete for jobs not just locally but internationally … The competition among countries now revolves around human capital and the comparative advantage in knowledge. (OECD, 2011: 14)

Once again, we find the same implicit postulates. For example, the only possible option is that education follows and adjusts, which is why education is all-consumed by learning. Existence is framed by success, money, and competition, and there is no space for alternative values. Thus, knowledge is defined and consumed by its monetary value, and human beings are conceived and understood as human capital – an approach that is both characteristic of and functional to neoliberalism. Finally, there is no competition between different perspectives; the competition is within the system, not between different systems. The report, moreover, is clearly intended to understand and explain what, how, and who has a “comparative advantage” in knowledge (OECD, 2011: 13), in policies and reforms (p. 75), and in economics (p. 162). What, how, and who can aim for success? How can “educational success” (pp. 13–15) be gained? How can “successful schools” and “successful education systems” (p. 15) be built? How can “[w]hat makes a school successful” (p. 20) be understood? Finally, what is “the range of factors that lead to success?” (p. 21). In the first 24 pages of the document, the term “success” occurs 25 times, and this number excludes similar terms, such as “top” or “excellence”. The title, too, is conceived in terms of success, and the table of contents shows that several chapters and sections are framed in terms of “success” (e.g. “A changing yardstick for educational success”, “Canadian success in education”, “Singapore’s success in education”).

Regarding the relationship between success and money, PISA’s approach is explained in a box titled “The approach of industrial benchmarking”:

The aim of the American firms was to learn enough from their competitors to beat them at their own game. To do this, they identified their most successful competitors. But, they also identified the companies that led the league tables in each of their major business process areas (e.g., accounting, sales, and inventory). They collected all the information they could possibly find concerning their direct competitors and the companies that led the league tables in the relevant business processes … When this research was complete, they would analyse all the information and research they had gathered. Their aim was … to build a better mousetrap than any they had seen anywhere by combining the best they had seen in one place with the best they had seen in another. (OECD, 2011: 22)

The OECD’s opinion regarding education is sufficiently clear. Considering how to build a “better mousetrap”, the idea that education is a matter of exchanging pieces implies that education is, in the best case, a Harlequin, if not a Frankenstein. Ironically, building a good mousetrap is a matter of intelligence and creativity, not a matter of simply taking pieces and deciding where to “put and remove” them.

Competition, competition, competition

The choice of benchmarking and the politics of “whoever provides the best value for money” (OECD, 2014e) clearly value competition and individualism. Competition is a basic fact of life; through competition, I can improve myself. Everyone in competition is stimulated to excel. However, in PISA there are two problems: it seems to promote competition as the only value, and it operates as an archetypal zero-sum game in which I gain if my competitor loses.

PISA’s idea of education, ultimately, is exclusive rather than inclusive. PISA clearly fosters competition not only between (Pons, 2011) but also within countries. According to its philosophy, every region in a country, every district in a region, and every school in a district intends to overtake the others (Alexander, 2011; Au, 2008; Dorn, 2007). Moreover, the corresponding and pervasive use of tools, such as PISA covering the entire school system from primary to high school, and the publication, if not the advertisement, of PISA scores move the entire educational process towards continuous competition to be the best according to the test (Biesta, 2012; Mansell, 2007). This is true at systemic levels (countries, regions, schools) and at individual levels. No collaboration between students is allowed during the test, so everyone is urged to do better than everyone else. Of course, not every individual form of assessment entails collaboration between students, but teachers who are aware of this fact can adopt different strategies to encourage cooperation and dialogue (e.g. cooperative learning, group work)¹. However, if the only legitimate assessment is “you and the test” and the goal is to consolidate and increase your “comparative advantage”, the people involved want to either improve their performance or hope that others do not achieve a good score. These desires hold true among classrooms in a school, schools in a region, regions in a country, and countries against countries (Grek, 2007). The structure of PISA as a zero-sum game is significant since what matters, just as in a race, is not merely that I improve my performance. The goal is to improve my position against others regardless of whether those others are my schoolmates or countries. Of course, to run a race and to train for a race are not sins; competition is a part of life and a tool for self-improvement. However, in the educational process, competition should be a component and a tool. In PISA, competition is the framework and the entire goal.

Regarding a new tool, the PISA-based Test for Schools, the webpages state the following:

It is expected that the PISA-Based Test for Schools will provide … the opportunity to share good practices to help identify ‘what works’ to improve learning and build better skills for better lives … In addition to assessing students’ ability to apply the knowledge they have acquired, the PISA-based Test for Schools also benchmarks how students … are prepared to become members of an increasingly global society … and the PISA-based Test for Schools is voluntary for schools that are interested in international benchmarking and improving their student outcomes. It is not to be used for mandated or accountability purposes. The assessment … is a learning experience for teachers and students that can prompt discussions on the types of knowledge, skills and competencies that are relevant in a quickly changing world. (OECD, 2014h, emphasis in original)

Once again, the same central concepts are evident. PISA’s framework is all encompassing: it develops “better skills for better lives”; it tests “how well [students] are prepared to become members of an increasingly global society”; and it also “benchmarks how students are prepared to become members of an increasingly global society”. Formally, the test is voluntary, but often, there is no clear line between what is voluntary and what is compulsory. For example, a university education is not compulsory, but sometimes students who want to improve their employment prospects or students who come from families that regard a university education as unavoidable feel that it is compulsory. Of course, to the simple question “is a university education compulsory?”, everyone will answer, “no, it is not”. However, reality says something quite different.

Very soon, this tool could be a line of demarcation between good schools and bad schools. Moreover, when PISA states, “The PISA-based Test for Schools and its results are not meant to be interpreted or used as school rankings or for ‘league tables’” (OECD, 2014h), it contradicts its philosophy. PISA’s goal is to show where you “stand” in relation to others (OECD, 2014c). Therefore, it is hypocritical to say that:

[s]chools with performance results that place them at the very top in comparison with other schools in England and in other countries should not see in the tool a means to ‘validate’ their excellence for publicity; they should see it as a means to strive for even higher levels of performance for all students. (OECD, 2014a)

According to pragmatism, one criterion for evaluating an act is its consequences. It is reasonable, then, to wonder whether a tool that assesses who is better and who is worse provides a means to validate who is better and who is worse. What is the difference between “[m]easuring student success” and “validat[ing] … excellence for publicity”? The same statement appears several lines later: “The assessment should be considered a tool for school improvement, not a tool for developing ranking or league tables” (OECD, 2014a). What will happen in concrete terms through the use of this tool? Although PISA underestimates how education is framed by ends, the OECD states that its intention – that is, how “[t]he assessment should be considered” – is not to develop rankings but to foster improvement. However, we should not consider the intentions but rather the consequences. In the OECD’s own words, PISA demonstrates several contradictions.

PISA’s inconsistency

Reading further on this webpage, we find three statements that are in contradiction with some basic rules of research:

In the future, the PISA-based Test for Schools can provide important peer-to-peer learning opportunities – locally, nationally and internationally – as well as the opportunity to share good practices to help identify ‘what works’. [The test] is a learning experience for teachers and students that can prompt discussions on the types of knowledge, skills and competencies that are relevant in a quickly changing world … [and] schools that decide to implement the assessment will need to work with accredited service providers to make sure that quality standards and procedures are followed in preparation for administering the test and on the day of testing.

Regarding this statement, there are two issues at stake. First, sharing and discussions require a broad space. If you decide who is able to apply your protocol and how the preparation of people for it must be conducted, you cannot claim that you are providing “the opportunity to share good practices” or “prompt discussions”; you are only testing the efficacy of your protocol. This detail might be very important, but it is not related to sharing and discussions. In addition, if you are testing your protocol, you need a term of comparison that cannot be the same as your protocol. Therefore, to what extent does what PISA says it does actually work?

This is one example of PISA’s inconsistency. In several documents, we find various contradictions. For clarity of explanation, these contradictions can be divided into two types: contradictions between what PISA claims to produce and what it actually produces, and contradictions within PISA – that is, contradictions between different statements, even in the same document. Because of space limitations, I can only discuss the point regarding the main reason for the inconsistency: the contradiction between what PISA establishes as important in learning and what it actually promotes.

The section “Do students have the drive to succeed?” on PISA’s homepage states the following:

Teachers’ use of cognitive-activation strategies, such as giving students problems that require them to think for an extended time, presenting problems for which there is no immediately obvious way of arriving at a solution, and helping students to learn from their mistakes, is associated with students’ drive. (OECD, 2014f)

The problem with this statement is that “to think for an extended time” is exactly the opposite of PISA’s philosophy. To think through a problem for “an extended time” is the best way to fail the PISA test; the approach that is encouraged and rewarded is to optimise time and to eliminate everything effectively that does not strictly concern the test questions. With multiple-choice questions, it is counterproductive to waste time thinking autonomously to arrive at the best possible answer; it is far more effective to look at the pre-set answers and choose the correct one. In addition, if I must respond in a few lines and in a few minutes, I do not need “to think for an extended time”. The goal of PISA, as stated several lines later, is “to become proficient in a skill” (OECD, 2014f). The OECD should be aware that “to become proficient in a skill” and to give students “problems that require them to think” are often two contradictory goals. Of course, life requires both; thus, schools must simultaneously train students to perform both skills. However, this aim can be achieved and assessed through different methods and tools. To confront a problem head-on, to understand how this problem concerns us, to pose the problem correctly, and, finally, to solve the problem are different skills that require different approaches to encourage.

It may be useful to read another statement from this document:

Raw potential and talent are only a small part of what it takes to become proficient in a skill. Students’ success depends on the material and intangible resources that are invested by families, schools and education systems to develop each and every student’s potential. (OECD, 2014f)

This statement, speaking in favour of “intangible resources that are invested by families”, is wise. It is important to recognise the intangibleness not only of resources but also of the major part of the educational process. Here, PISA touches on a serious and substantial argument, which is the crux of educational research: the problem of how and what matters to education and the resulting problem of how and what concerns educators’ evaluation. The analysis of arguments such as these is both fundamental and continuous, and these issues cannot be solved quickly or definitively. Moreover, it is difficult to understand how PISA can help students “to learn from their mistakes” when students do not receive individual feedback on the test.

Furthermore, PISA seems to be unaware of the fact that students make sense of their own learning and the gap between what is taught and what is learned – that is, at the same time, the limits and the possibilities of education (Biesta, 2005, 2012). The problem of this situation lies in the fact that PISA claims to pursue “what 15-year-olds can do with what they know”. In fostering “skills for life”, PISA aims to prepare students for life. Granting this – and there are several good reasons to state the opposite (Bonderup Dohn, 2007; Hopman and Brinek, 2007) – the problem is that life is not a univocal concept. What is this life to which the OECD refers? Is life, according to PISA, something in which everyone is able to contribute equally to the development of democracy and society and have a voice? The answer, most likely, is no. Society, in PISA’s words, is framed not by citizens but rather by who has “the right skills” (OECD, 2014e). Is it allowed or simply convenient to share knowledge in PISA’s conception of life? I do not think so. If you are studying with your classmate and he is on the wrong track, it is better not to make him aware, because your absolute value is not important, according to PISA’s logic; the only thing that is important is your position in the rankings.

With PISA, we can find other examples of this lack of awareness:

To attract the best graduates to the teaching profession, these systems need to transform the work organisation in their schools to an environment in which professional norms of control replace bureaucratic and administrative forms of control. Equally important, more professional discretion accorded to teachers allows them greater latitude in developing student creativity and critical thinking skills that are important to knowledge based economies; such skills are harder to develop in highly prescriptive learning environments. (OECD, 2014f)

The question is simple: does PISA treat teachers as autonomous persons who foster autonomous thinking or as stupid and ancient soldiers who educate “parrots”, as in one of its popular videos (OECD, 2014c)? Of course, the objection would be that this is only a caricature that shows one wrong way to teach. Unfortunately, the opposite appears to the case. Regarding the relevance and the concepts expressed, the video is a meaningful document that explains PISA’s aims, opinions, and methods. Notwithstanding our opinion on PISA, this rough division appears to be dangerous and unfair. It is faith in a unique tool – and in a unique society – that reveals a colonialist mind-set.

This is not the place – and I am not the scholar – to discuss the relationship between the real nature of today’s economy and the fairy tale version of it, but the opportunity to create new processes in the globalised world is intentionally overstated. That globalisation has led to a wider space for creativity in work is all too apparent. The weight of international associations, such as the OECD, with their strong influence on countries (Bracey, 2008; Grek et al., 2009), and also increasing economic inequality, point in the opposite direction. The real decisions about education are most likely made by fewer people than in the recent past. The OECD, for example, hardly creates a space for sharing knowledge and experience about education because such a space starts with teachers, educators, and key local actors in the field of schooling and education. Rather, the OECD aims to govern the educational process through pervasive penetration into schools, educational departments, government agencies dedicated to education, and families². This totalitarian stance is a mistake, and not merely from a humanistic and old-fashioned perspective. PISA and the OECD should be aware that complexity, long-term ideas, and apparent stalemates are essential, not only to scientific progress but also to the preservation of life, which depends on variety.

Final thoughts

Throughout this article, I have argued that PISA’s nature has roots in colonialism. Through PISA, the OECD intends to expropriate subjects of their culture and knowledge, denying the subjects’ legitimacy and imposing the OECD’s own univocal logic. This colonialism should not be understood either as an expropriation of resources or as only an economic and ideological penetration of countries. To understand PISA’s colonialism, the Foucauldian category of “governmentality” may be useful. This idea concerns “the way in which the behavior of a set of individuals became involved, more and more markedly, in the exercise of sovereign power” (Foucault, 1997a [1954–1984]: 78). In a Foucauldian understanding, governmentality is exercised through “rational forms” and “technical procedures, instrumentations […] and strategic games” (Foucault, 1997b [1954–1984]: 202). To grasp the meaning of this issue, we must bear in mind how Foucault, throughout his life, challenged the idea that power and knowledge are separate entities. For him, power and knowledge came into the world together, and, in a sense, they “make” the world. The subject is always situated in a power/knowledge domain (Foucault, 1995 [1975]). This, of course, does not imply that our relationship with knowledge and power is blocked. On the contrary, we can challenge a given relationship and a given domain by another understanding – and another acting – of the power/knowledge domain.

Thus, we cannot see the prominence of one entity over the other. Power exists and expresses itself in a knowledge domain, and knowledge exists and establishes itself in a power domain. At the same time, these embedded domains need and produce institutions and concepts, objects, and languages. In a sense, they also produce the “kind” of subjects we are (Foucault, 1973 [1969]). The person who aims to exercise power must establish knowledge strictly functional to his or her power’s position and a “field of existence” of the phenomena at stake. According to Foucault, by establishing this knowledge, the person who exercises power then establishes what is visible, what is relevant, and what is not. PISA corresponds fully to the Foucauldian criterion of ‘governmentality’: it is a “rational form”, a “technical procedure” and an “instrumentation” through which the OECD penetrates countries, schools, and families. PISA’s well-defined idea of education exposes its goals to the actors involved in education – students, teachers, policy makers, and headteachers. Furthermore, through this process, the OECD aims to establish what a society, education, and students should be.

Considering this, PISA can be seen as a tool of the “narrative” of neo-liberal globalisation. This narrative does not imply a pre-eminence of one country over another; the decline of the major shareholder of neoliberalism, the United States, is significant regarding this issue. Rather, this narrative implies the elimination of any other way of conceiving the world and a way of life that is not coherent with its logic. In Gurrìa’s words, this narrative is the logic of “whoever provides best value for money” (OECD, 2014e). PISA’s birth and success – and that of the national assessment tools based on PISA’s model – are clearly linked to the neo-liberal globalisation “narrative” (Alexander, 2011; Au, 2011; Bourdieu, 2003; Nichols and Berliner, 2007; Rizvi and Lingard, 2010). Further, it may be considered that the OECD, under the aegis of “evidence” and “objectivity”, is turning education not only into a tool that is strictly functional for the goals of the new Taylorism but into a Weltanschauung whose founding values, in the OECD myth, are “knowledge”, “equity”, “quality”, and education. After all, who would speak against these ideas?

Education is eclipsed by PISA. Education is the place where our life and the life of society take form (Biesta, 2007; Dewey, 1930, 1938; Gramsci, 1992 [1948--1951]; Torres, 1998). When we conceive of education from a pre-established stance, we conceive of living before living can show its own possibilities. Society is the same. In fostering only one concept of education, we freeze, so to speak, the current form of society and the current power relationship. Thus, education, to the extent that it is concerned with freedom and justice, must be also concerned with otherness and possibilities, namely, with the space that is not-yet-thought.

Being in education requires thinking and acting in existential frames, and this engagement with human destiny entails responsibility for our direction. In education, we are called to act on something and by someone. Being in education – in research and in practice – requires us to take a position and to choose a way to act. Thus, we are faced with a paradox: on one hand, we should be able to remain in the presence of otherness without the constraints of a previous definition, and to respect the otherness of others (Biesta, 2007; Gur-Ze’ev, 2003); on the other hand, we must decide in which direction we are called for a concrete process, or which direction to show to the subjects we are educating. This paradox is anything but new; it is one of the cruxes of educational research, from Dewey to Gramsci to Gur-Ze’ev. The paradox has its roots in the difficulty – or perhaps, the impossibility – of defining the question of otherness in satisfying terms. Insofar as otherness is radically outside oneself, one can never grasp it without betraying it. Further, this paradox is of more than philosophical significance. As Foucault has shown, this paradox involves a choice regarding who has the right and who has the power to speak and to act, which is the founding concern of democracy. However, to the extent that the meaning of education lies in transformation, and education is concerned with justice, then education must be concerned with the ungraspable openness of our thought. Education must demand the work to go beyond what we and society currently are. As educators, we must continually ask whether our stance towards education is large enough and good enough to match and manage this challenge. Without this engagement, without challenging given “forms of life”, education is at risk of becoming the means to facilitate the progressive impoverishment of the living, and the perpetuation of injustice and inequality into the future.

Footnotes

Notes

References

Alexander R (2011) Evidence, rhetoric and collateral damage: The problematic pursuit of ‘world class’ standards. Cambridge Journal of Education 41(3): 265--286.

(2008) Devising inequality: A Bernsteinian analysis of high-stakes testing and social reproduction in Education. British Journal of Sociology of Education 29(6): 639–651.

(2011) Teaching under the new Taylorism: High-stakes testing and the standardization of the 21st century curriculum. Journal of Curriculum Studies 43(1): 25–45.

Biesta

GJJ

(2005) Against learning. Reclaiming a language for education in an age of learning. Nordisk Pedagogik 25: 54–66.

Biesta

GJJ

(2007) Education and the democratic person: Towards a political understanding of democratic education. Teachers College Record 109(3): 740–769.

Biesta

GJJ

(2012) Good education in an age of measurement: On the need to reconnect with the question of purpose in education. Educational Philosophy and Theory 44(6): 581–593.

Bonderup Dohn

(2007) Knowledge and skills for PISA—Assessing the assessment. Journal of Philosophy of Education 41(1): 1–16.

Bourdieu P (2003) Firing Back Against the Tyranny of the Market. London: Verso.

Bracey

(2008) The leaning (toppling?) tower of PISA? Principal Leadership 9(2): 49–51.

10.

Carroll

Kellow

(2011) The OECD: A Study of Organisational Adaptation, Cheltenham: Edward Elgar.

11.

Dewey

(1929a) Experience and Nature, London: George Allen & Unwin.

12.

Dewey

(1929b) The Quest for Certainty: A Study of the Relation between Knowledge and Action, New York: Minton, Balch & Company.

13.

Dewey

(1930) Democracy and Education. An Introduction to the Philosophy of Education, New York: The MacMillan Company.

14.

Dewey

(1938) Experience and Education. In: Boydston

(ed.) The Later Works, 1925--1953 Vol. 13. Carbondale: Southern Illinois University Press, pp. 1–62.

15.

Dorn

(2007) Accountability Frankenstein: Understanding and Taming the Monster, Charlotte, NC: Information Age Publishing.

16.

Eccleston

(2011) The OECD and global economic governance. Australian Journal of International Affairs 65(2): 243–255.

17.

Foucault M (1973) [1969] The Order of Things. An Archaeology of Human Sciences. New York: Vintage/Random House.

18.

Foucault

(1995) [1975]) Discipline and Punish, New York: Vintage.

19.

Foucault

(1997a) [1954–1984]) Security, territory and population. In: Rabinow

(ed.) Ethics Subjectivity and Truth. The Essential Works of Michel Foucault, 1954--1984 Volume I, New York: The New Press, pp. 223–252.

20.

Foucault

(1997b) [1954–1984]) Preface to the history of sexuality, Volume II. In: Rabinow

(ed.) Ethics Subjectivity and Truth. The Essential Works of Michel Foucault 1954--1984 Volume I, New York: The New Press, pp. 199–206.

21.

Foucault

(2002) [1969]) Archaeology of Knowledge, London: Routledge.

22.

Giroux

(1981) Ideology Culture and the Process of Schooling, Philadelphia: Temple University Press.

23.

Giroux

(1989) Schooling for Democracy. Critical Pedagogy in the Modern Age, London: Routledge.

24.

Gorur

(2011) ANT on the PISA trail: Following the statistical pursuit of certainty. Educational Philosophy and Theory 43(s1): 76–93.

25.

Gramsci

(1992) [1948–1951]) Prison Notebooks, New York: Columbia University Press.

26.

Grek S (2007) ‘And the Winner is … ’: PISA and the construction of the European education space. Paper presented at the ‘Advancing the European Education Agenda’, European Education Policy Network Conference.

27.

Grek S, Lawn M and Ozga J (2009) Production of OECD’s ‘Programme for International Student Assessment (PISA)’. Edinburgh: Centre for Educational Sociology, University of Edinburgh.

28.

Gur-Ze’ev

(2003) Destroying the Other’s Collective Memory, New York: Peter Lang.

29.

Henry M, Lingard B, Rizvi F, et al. (2001) The OECD Globalisation and Education Policy. Amsterdam: Pergamon.

30.

Hopman S and Brinek TG (2007) PISA according to PISA — Does PISA keep what it promises? In: Hopman S, Brinek TG and Retzl M (eds) PISA According to PISA — Does PISA Keep What It Promises? Berlin: Lit Verlag Dr.W. Hopf.

31.

Horkheimer

Adorno

(2002) [1944]) Dialectic of Enlightenment. Philosophical Fragments, Stanford: Stanford University Press.

32.

Jakobi

Martens

(2010) Introduction: The OECD as an actor in international politics. In: Martens

Jakobi

(eds) Mechanisms of OECD Governance: International Incentives for National Policy-making?, Oxford: Oxford University Press, pp. 1–25.

33.

Lawn

Lingard

(2002) Constructing a European policy space in educational governance: The role of transnational policy actors. European Educational Research Journal 1(2): 290–307.

34.

Levinson

(2011) Democracy, accountability and education. Theory and Research in Education 9(2): 125–144.

35.

Mansell

(2007) Education by Numbers: The Tyranny of Testing, London: Politico’s Publishing.

36.

Nichols

Berliner

(2007) Collateral Damage: How High-stakes Testing Corrupts America’s Schools, Cambridge, MA: Harvard Education Press.

37.

OECD (2004) Learning for Tomorrow’s World. First Results from PISA 2003, Paris: OECD Publications.

38.

OECD (2010a) PISA 2009 Results: What Students Know and Can Do – Student Performance in Reading Mathematics and Science (Volume I). Available at: https://dx-doi-org.web.bisu.edu.cn/10.1787/9789264091450-en.

39.

OECD (2010b) The OECD Innovation Strategy: Getting a Head Start on Tomorrow, Paris: OECD Publishing.

40.

OECD (2011) Lessons from PISA for the United States Strong Performers and Successful Reformers in Education. Paris: OECD Publishing. Available at: https://dx-doi-org.web.bisu.edu.cn/10.1787/9789264096660-en. http://www.oecd.org/pisa/46623978.pdf.

41.

OECD (2014a) About PISA. Available at: http://www.oecd.org/pisa/aboutpisa.

42.

OECD (2014b) Home-page. Available at: http://www.oecd.org/pisa/home.

43.

OECD (2014c) OECD’s video PISA – Measuring student success around the world. Available at: http://www.oecd.org/pisa/aboutpisa.

44.

OECD (2014d) OECD test for schools. Available at: http://www.oecd.org/pisa/aboutpisa/oecdtestforschoolsintheus.htm.

45.

OECD (2014e) OECD’s video Strong performers and successful reformers in education. A video series profiling policies and practices of education systems that demonstrate high or improving performance in the PISA tests. A message from Angel Gurrìa. Available at: http://www.pearsonfoundation.org/oecd.

46.

OECD (2014f) PISA 2012 Results in Focus. What 15-year-olds know and what they can do with what they know. Available at: know. http://www.oecd.org/pisa/keyfindings/pisa-2012-results-overview.pdf.

47.

OECD (2014g) PISA 2015 Draft Framework. Available at: http://www.oecd.org/pisa/pisaproducts/pisa2015draftframeworks.htm.

48.

OECD (2014h) Test for Schools (based on PISA): Frequently Asked Questions (and answers). Available at: http://www.oecd.org/pisa/aboutpisa/pisa-basedtestforschools.htm.

49.

Onosko J (2011) Race to the top leaves children and future citizens behind: The devastating effects of centralization, standardization, and high-stakes accountability. Democracy & Education 19(2): article 1.

50.

Pearson Foundation and OECD (2014) Strong performers and successful reformers in education. Available at: http://www.pearsonfoundation.org/oecd.

51.

Peters

(1966) Ethics and Education, London: Allen & Unwin.

52.

Pereira MA, Kotthoff HG and Cowen R (eds) (2011) PISA Under Examination. Changing Knowledge Changing Texts and Changing Schools. Rotterdam, Boston, Taipei: Senser Publisher.

53.

Pons

(2011) What do we really learn from PISA? The sociology of its reception in three European countries (2001–2008). European Journal of Education 46(4): 540–548.

54.

Prenzel M (2013) Introduction: Research on PISA, with PISA, and for PISA. In: Prenzel M, Kobarg M, Schöps K and Rönnebeck S (eds) Research on PISA. Research Outcomes of the PISA Research Conference 2009. Dordrecht, Heidelberg, New York, London: Springer, pp. XII--XXI.

55.

Rancière

(2003 [1983]) The Philosopher and His Poor, Durham, North Carolina: Duke University Press.

56.

Rizvi F and Lingard B (2010) Globalizing Education Policy. London, Routledge.

57.

Sellar S and Lingard B (2013) The OECD and the expansion of PISA: New global modes of governance in education. British Educational Research Journal. Epub ahead of print. DOI: 10.1002/berj.3120.

58.

Shohamy

(2004) Assessment in multicultural societies: Applying democratic principles and practices to language testing. In: Norton

(ed.) Critical Pedagogies and Language Learning, Cambridge: Cambridge University Press, pp. 72–93.

59.

Torres

(1998) Democracy Education and Multiculturalism. Dilemmas of Citizenship in a Global World, Boston: Rowman and Littlefield.

60.

Wolf

(2002) Does Education Matter? Myths about Education and Economic Growth, London: Penguin.

61.

Woodward

(2009) The Organisation for Economic Cooperation and Development (OECD), Abingdon: Routledge.