Improving education through research? From effectiveness,causality and technology to purpose,complexity and culture

Introduction

Ever since the establishment of the first professorship in education at the University of Halle in Germany in 1779, educators and educationalists have raised questions about the potential contribution of research to the improvement of educational practice. Ernst Christian Trapp, the first holder of this chair, not only devoted his inaugural lecture to what has become known as the theory-practice problem in education (see Trapp, 1779) but also contributed to the discussion via other publications, including one with the rather contemporary sounding title On the Promotion of Effective Knowledge (Von der Beförderung der wirksamen Erkentniss; Trapp, 1778).

The suggestion that one of the key tasks of educational research lies in increasing the effectiveness of educational action has been a recurring theme in discussions about the role of research in education. Recent manifestations of this line of thinking can be found in the school effectiveness movement (see, for example, Townsend, 2007) and in the suggestion that educational research should focus on generating evidence about ‘what works’ (see, for example, Thomas and Pring, 2004; Biesta, 2007). A concrete recent attempt to bridge the gap between research and practice can be found in TLRP's 10 Principles for Effective Pedagogy (James and Pollard, 2012a), ¹ which were formulated on the basis of research ² conducted in the UK's Teaching and Learning Research Programme, ‘the largest programme of educational research on teaching and learning that the UK has ever seen’ (James and Pollard, 2012b: 269).

In this paper I wish to raise a number of questions about the idea of educational improvement and about the role research can and should play in it. While a detailed discussion of TLRP's 10 principles lies beyond the scope of this paper, I will use the principles to illustrate what I see as some of the common problems in discussions about educational research and educational improvement. The principles thus function as an exemplary ‘case’ within this paper. I will start with a brief presentation of the principles and will then focus on three issues. The first has to do with the idea that educational improvement entails increasing the effectiveness of educational processes and practices. Here I will argue that any discussion about effectiveness always needs to be connected to wider considerations about the aims and purposes of education. The second point has to do with underlying assumptions about the dynamics of education – that is, with ideas about how education ‘works’ and how it can be made to ‘work’. I show that much of the discussion about educational improvement relies on a quasi-causal conception of education, where the task of research is seen as that of identifying key variables so that the overall process and, more specifically, the relationship between ‘inputs’ and ‘outcomes’ can be controlled. In response to this I will suggest looking at the way in which education works and can be made to work through the lens of complexity theory. This is related to the third point, which has to do with assumptions about the kind of knowledge research can and should generate in order to be of practical use for education. Here I will make a distinction between technical and cultural knowledge, and hence between a technical and a cultural role for research in relation to practice, and will point at the limitations of technical knowledge and a technical role for research in the improvement of education.

TLRP's 10 principles for effective pedagogy

As mentioned, my aim with this paper is not to provide a critical analysis of TLRP's 10 principles as such, but to use them as an example of current thinking about the relationships between research and practice in education, specifically with regard to the idea of educational improvement. The principles themselves were developed over a number of years. They have their background in what James and Pollard (2012b: 277) refer to as ‘the conceptual map that TLRP has developed to represent the scope of its interests with reference to teaching and learning’ (for the actual map see James and Pollard (2012b: 278)) and work that was conducted within the programme aimed at synthesising findings from individual projects (see James and Pollard (2012b: 278)). A first iteration of the principles was published in 2006 (James and Pollard, 2006) when the principles were presented as ten ‘evidence-informed principles for effective teaching and learning’ (James and Pollard, 2012b: 279). Eventually, it was decided to group the principles under four headings: (1) educational values and purposes; (2) curriculum, pedagogy and assessment; (3) personal and social processes and relationships; and (4) teachers and policies (see James and Pollard, 2012b: 279).

While in a number of iterations the principles were presented as principles ‘of effective teaching and learning’ (James and Pollard, 2012b: 279), in one of the most recent (and perhaps final) presentations they are articulated as ‘principles for effective pedagogy’ (see James and Pollard, 2012b: 279). James and Pollard provide four reasons for the shift from ‘teaching and learning’ to ‘pedagogy’. They claim that the term ‘pedagogy’ is less ‘academic jargon’ than other phrases; that it is ‘now more widely used by UK practitioners and policy makers’ than at the start of TLRP; that ‘“pedagogy” expresses the contingent relationship between teaching and learning … and does not treat teaching as something that can be considered separately from an understanding of how learners learn’; and that most of the TLRP projects actually focused more on ‘implications for teaching of what we know about learning, than it did on developing new knowledge about learning per se’ (James and Pollard, 2012b: 280; emph. in original). ³ While the use of the term ‘pedagogy’ thus seems to indicate a shift away from the language of ‘teaching and learning’, James and Pollard nonetheless quote Alexander's definition of pedagogy as ‘the act of teaching together with its attendant discourse’ (Alexander, 2004: 11, quoted in James and Pollard 2012b: 280) and add that this definition ‘fits well with the way TLRP came to understand pedagogy’ thus giving the impression that ‘pedagogy’ first and foremost refers to teaching. In this sense the concept of ‘pedagogy’ used here still (see Simon, 1981; Alexander, 2004) differs significantly from its Continental counterpart (in German: ‘Pädagogik’; see Biesta, 2011).

Inspiration for formulating the outcomes of the review of TLRP projects in terms of principles apparently came from the way in which the UK Assessment Reform Group had used this format to summarise ‘evidence on effective “assessment for learning”’ (James and Pollard, 2012a: 279). ⁴ The most recent formulation of the ten principles – listed on the TLRP website under the heading of ‘TLRP's evidence-informed pedagogic principles’ and in James and Pollard (2012b) as ‘TLRP's 10 principles for effective pedagogy’ is as follows.

Effective pedagogy equips learners for life in its broadest sense. Learning should aim to help individuals and groups to develop the intellectual, personal and social resources that will enable them to participate as active citizens, contribute to economic development and flourish as individuals in a diverse and changing society. This means adopting a broad conception of worthwhile learning outcomes and taking seriously issues of equity and social justice for all.

Educational improvement: Effectiveness or change for the better?

The idea of ‘effective pedagogy’ suggests that educational improvement is a matter of increasing the effectiveness of educational processes and practices. Here we can already find a very common but also very fundamental problem in the discourse about educational improvement. This problem has to do with the fact that ‘effectiveness’ is a process value, that is, a value that says something about the ability of certain processes to ‘produce’ certain ‘outcomes’, but that, in itself, has nothing to say about the desirability of those outcomes. The crude way to make the point here is to say that there is both ineffective and effective torturing and, while one may wish to work hard to increase the effectiveness of the torturing methods one deploys, it does not make the outcome any better. Ineffective torturing is, in this regard, as morally reprehensible as effective torturing.

While in a general sense it may be desirable, therefore, to have more effective rather than less effective ways of working in education ⁵ – albeit that the multidimensional nature of educational purpose makes this matter a bit more complicated; see below – just to argue that an increase in effectiveness constitutes educational improvement, is a rather empty statement if we do not specify what it is that the activity aims to achieve. Given that ‘effectiveness’ is a process value, the key question to ask with regard to educational improvement should therefore be: ‘Effective for what?’ (see Bogotch et al., 2007) – and given that what may be effective for one individual or group may not necessarily be effective for another individual or group, one may wish to add a second question: ‘Effective for whom?’ (see Bogotch et al., 2007; see also Peterson (1979) for a short publication that shows that the point is an old but perhaps forgotten one). In most of the ten principles this connection with the question of purpose seems to be absent. Let us take another look at principles 2 to 10.

 2. Effective pedagogy engages with valued forms of knowledge.

What is remarkable here is that general and decontextualized claims are being made about how education ought to proceed, without providing any pragmatic ‘framing’, that is, without providing any insights in what it is that effective pedagogy is supposed to bring about. For example, while in some cases and for some purposes it might indeed be a good idea to engage with valued forms of knowledge, there are other cases when this is precisely not how education should proceed. Sometimes because the orientation is not on knowledge – think, for example, of the education of very specific skills – and sometimes because the point is precisely for students to understand that there is no such thing as ‘most valued’ knowledge in the abstract, since what is valued by some is not necessarily valued by others and the challenge of a democratic world is precisely to accommodate such radical plurality, rather than to cover it up. Similarly, when sometimes it can be a good idea to recognise the importance of prior experience and learning by taking account of what the learner knows already and by building upon prior learning, there are other cases when a radical break with such learning is called for, for example, when students have misconceptions that block them from making progress in understanding – a common issue in the teaching of mathematics and physics – or because students have learned and internalised something about themselves, for example, that they are not suited for any participation in formal education – a common theme when the question of learning is looked at from a lifecourse perspective (on this particular issue see Biesta et al., 2011; Goodson et al., 2010).

The more general point I am trying to make here, is that whether any of the principles mentioned in the list of ten are indeed examples of effective pedagogy that can only be judged if there is clarity about the aims that one seeks to achieve through education. In this regard it matters crucially that education is a teleological practice, that is, a practice that is not only framed but is actually constituted by its purpose (in Greek: telos). Without a purpose – or without purposes – there may be learning, but not education in the sense of a person intentionally trying to influence someone else's cognitions, beliefs, values, attitudes, habits, ways of doing and being and so on. Educational situations, in other words, always raise the question of what the situation is for, what it aims to achieve, etc. This does not mean that such aims should be 100% determined – an educator can, after all, also decide to initiate processes that are entirely open with regard to their outcomes. It also does not mean that only teachers or parents should be ‘allowed’ to formulate what the aims of the educational activity ought to be – although it can be claimed that they have a responsibility in educational processes and relationships that is fundamentally different from the responsibility of students and children.

One could say that the ten principles actually address this issue through what is stated in the first principle, namely the suggestion that ‘effective pedagogy equips learners for life in its broadest sense’. Leaving aside the rather odd formulation in terms of ‘effective pedagogy’, one could say that there is little that is problematic in the assumption that school education should equip students for life in its broadest sense, and one could even say that this statement is so broad that it actually gives very little, if any, direction to educational processes and practices. The way in which this general statement is operationalised already raises further problems, particularly because James and Pollard seem to have opted for a rather functional, if not functionalistic, view of what education is for, in that they approvingly identify ‘three major strands of philosophical and political thinking on educational purposes’ (James and Pollard, 2012b: 276), namely ‘economic productivity’, ‘social cohesion’ and ‘personal development’ (James and Pollard, 2012b: 276, emph. in original). Moreover, these three strands of thinking on educational purposes are rather specific and are connected to very particular educational and political ideologies, more those of the homo economicus than, say, the homo democraticus. One could, after all, ask why it is that ‘effective pedagogy’ should focus on economic productivity and not on sustainable and respectful engagement with limited natural and social resources; why it should focus on social cohesion and not on peaceful democratic co-existence; or why it should focus on personal development, fulfilment and expression and not on compassion, altruism and ethical engagement.

Yet even with this particular formulation of the educational values and purposes that are supposed to give direction to our educational endeavours, there are still a number of further problems. One is that the other nine principles remain disconnected from what is stated in principle 1. If I have read the documentation correctly, no attempt is made, for example, to show that scaffolding is in all cases the more effective way to educate for individual flourishing or economic productivity than, for example, self-directed study or discovery learning. Second, the values and purposes listed in the first principle lack justification and in this regard raise the deeper question of whether school education should be seen as a function of society – or even more specifically, a function of the state or the current government – or whether education should also be informed and guided by purposes that lie outside of this scope. This question – which is the question of the autonomy of the educational ‘sphere’ – has been an ongoing topic of discussion in the educational literature at least since Rousseau and can be formulated as the question whether school education should indeed be a function of and functional to society or the state, or whether education always also implies a ‘duty to resist’ as the French educationalist Philippe Meirieu formulates it (see Meirieu, 2008). Third, the TLRP principles seem to narrow down the question of educational purpose to a formulation of ‘learning outcomes’ (see James and Pollard, 2012b: 282), forgetting that any formulation of such ‘outcomes’ as the desired results of education is contingent upon a formulation of educational purposes rather than that such outcomes can replace such purposes.

While the question of educational purpose is therefore indispensable in any discussion about educational improvement and effectiveness, I do not think that the discussion about the purpose or purposes of education should be thought of exclusively in terms of the formulation of differing ideological positions. As I have argued elsewhere in more detail (Biesta, 2010a) it is possible to create greater clarity in such discussions about what good (rather than effective) education is, by acknowledging that education never functions in relation to only one purpose or set of purposes, but that educational activity always operates in relation to a number of different domains of educational purpose. One way to understand what is at stake here is to start by looking at the different areas in which school education actually has an impact. With regard to this I have suggested that it is possible to make a distinction between three areas in which education tends to function. One is the area of qualification. This has to do with the ways in which, through education, students acquire knowledge, skills and dispositions that qualify them to do certain things – and the doing can either be very precise and confined such as, for example, the ability to perform a mathematical operation or execute a practical skill, or it can be very broad, such as the ability to navigate a complex multicultural society successfully. A second area is that of socialisation. This has to do with the ways in which, through education, students become part of existing traditions, cultures and ways of being and doing. Here we can think, for example, of the ways in which education – either deliberately or in more ‘hidden’ ways – reproduces particular societal and cultural configurations and identities, sometimes because this is what education aims to do and sometimes because of the ways in which education actually operates. But professional socialisation, that is, the deliberate attempt to make students competent members of particular professional communities, is another example of this – as are particular forms of religious and moral education. In addition to these two areas in which education can function, I have suggested that education always also impacts on the person, either to make students more dependent on existing structures and practices or with the intention of making them more independent from such structures and practices – a line of thought that relates to education as a process of enlightenment and emancipation.

While qualification, socialisation and what I have suggested to call ‘subjectification’ – that is, the process of becoming- subject – can be seen as three distinct functions of education, i.e. three ways in which most if not all education functions, I have further suggested that we can see qualification, socialisation and subjectification as three potential purposes of education or, in order to acknowledge that there are important choices to be made with regard to each of them, as three domains of educational purpose. To think of the question of purposes in education in this way at least allows for more precision in the discussion. But for the particular focus of this paper there is another important advantage in looking at the question of educational purpose in this way. This advantage stems from the fact that the three domains of educational purpose pull education in different directions. In other words, there is no complete synergy between the three domains, but rather potential for tension and conflict. While, for example, competition might be a desirable ‘driver’ in the domain of qualification – one could argue, for example, that students achieve more in a competitive environment – it is not necessarily also a desirable value in the domain of socialisation if, for example, one wishes to promote a collaborative attitude. It is also not automatically a desirable orientation in the domain of subjectification, that is if one wishes to promote the emergence of a homo democraticus rather than, say, a home economicus.

I am inclined to believe that the multidimensional character of educational purpose – that is, the fact that education always operates in relation to a number of different ‘domains’ – is something that is specific to education as a practice and endeavour. Unlike other human practices, which are often constituted and framed by a single orientation, for example, medicine's orientation on (the promotion of) health, or the legal system's orientation on (the promotion of) justice, the multidimensionality of educational purpose means that there is always a need for a judgement about how to balance the different ‘interests’ that education seeks to promote. In many cases this will be a question of ‘trade-offs’, that is, how much one is willing to ‘give in’ in relation to one domain in order to promote a particular achievement in relation to another. My main point here is that this creates an additional and very specific problem for the question of ‘effective pedagogy’ and the question of educational effectiveness more generally, because it means that what may be effective in furthering progress in relation to one domain of purpose may not be effective in relation to another. The question of educational effectiveness is, therefore, not only about connecting particular educational strategies and ways of working to what it is one aims to achieve, which is why ‘effective pedagogy’ remains a meaningless idea until one specifies what it is that the pedagogy ought to be effective for, that is, what it ought to effect. The additional difficulty is that education always operates in relation to (at least) three different domains of educational purpose, so that what is effective in relation to one domain may be ineffective in relation to another. This indicates that there is not only judgement needed with respect to the relationship between means and ends but also with regard to potential tensions and conflicts between those ends.

Some try to address this particular problem by arguing that school education should actually only focus on qualification – and that matters of socialisation and subjectification belong to the remit of parents and not teachers. We can see this tendency at the level of policy, particularly when very narrow qualification agendas are being put forward as the only thing that counts and ought to count in education (think for example of the definition of good education in PISA). Whether such an artificial reduction of the function of schooling is de facto possible remains to be seen. One could argue that even schools that are entirely constructed in terms of qualification still perform a socialising and subjectivising role, as they socialise their students into the idea that the only things that count in life are knowledge and skills and that the only valuable way of being human is through qualification – in a sense this is what the literature on the hidden curriculum makes visible. But even policy makers who put schools under pressure to focus on the qualification dimension will always blame schools for failing to contribute to socialisation and subjectification when things go wrong in society, for example, when young people participate in protests or riots.

My observations about the ten principles and the idea of ‘effective pedagogy’ can be summarised by saying that ‘effectiveness’ is never an educational good in itself and only becomes a meaningful idea in relation to views about the purpose or purposes of education. In this sense I wish to argue that, in the field of education, no one approach or way of working is desirable in itself, but everything depends on what it is one aims to achieve. Whether education should be flexible, personalised, student-led and motivating, or whether it should be strict, structured, general, teacher-led and difficult, entirely depends on what it is one aims to achieve. In this regard, we could say that all judgements about educational processes and practices are entirely pragmatic; they are never about the desirability of such processes and practices in themselves, but always in relation to what they are supposed to bring about. ⁶ While increasing the effectiveness of education may count as a limited and rather technical definition of educational improvement, any judgements about whether change counts as improvement ultimately has to be made in relation to the wider purposes that orientate the activity. Given the multidimensional character of educational purpose, an additional difficulty is that what is effective in relation to one domain of purpose is not necessarily effective in relation to another. This makes it extremely difficult, if not impossible, to speak about increasing the effectiveness of education in a general sense. Anything one can say here ultimately has to be said in relation to specific domains of educational purpose.

All this suggests that the very idea of formulating principles of effective pedagogy in a general and abstract sense is fundamentally flawed, because any claims about what is effective and desirable can only be made with reference to what it is one aims to achieve, while acknowledging that what is effective in relation to one domain of purpose may be counterproductive with respect to another. Does that imply that it is actually impossible to formulate any principles for educational practice? I do not believe that this is the case, but what is required is that one approaches such a task in a pragmatic way, that is, in the form of ‘if … then’ statements, such as ‘if one wishes to increase students’ exam scores, then it is advisable to train students as much as possible on the specific tasks in which they will be examined’, or ‘if one wishes to teach for understanding, then it is advisable to provide students with many opportunities for application, reflection and discussion’. Such an approach to the formulation of research-informed principles for educational improvement goes back to the work of Lawrence Stenhouse and is captured in the idea of ‘principles of procedure’ (see Stenhouse, 1975). Principles of procedure are driven by educational aims and purpose – that is, by statements about what is considered to be desirable – and then try to articulate the approaches and ways of working that are likely to make a positive contribution to achieving this. A concrete example of this particular approach, which is significantly different from TLRP's 10 principles, can be found in James and Biesta (2007: 143–160).

Making education work: causality or complexity?

If the foregoing section gives an idea of the kind of questions that need to be raised in order to engage with the issue of educational improvement as different from the issue of just increasing the effectiveness of the educational ‘operation’, I now wish to turn to the question of the role or roles that research has to play in this. I will do this in two steps. In this section I will look at assumptions about the dynamics of educational processes and practices. In the following section I will ask what kind or kinds of knowledge we might expect from research in relation to this.

The assumption that seems to inform many discussions about education and its improvement is that education works in some kind of quasi-causal way, where there are, at one end, ‘input’ variables such as teaching, curriculum, assessment and perhaps also such variables as student ability, material resources and wider policies, and at the other a more or less wide range of learning outcomes. While many researchers and policy makers would concede that there is still much we do not know about the possible connections between all the variables and, while many would also concede that there is a complicated web of possible connections between ‘inputs’ and ‘outcomes’, the general tendency seems to be that this is just a practical matter, i.e. that if we just conduct more research we will, eventually, be able to identify the factors that determine the production of certain learning outcomes ⁷ – or, put in terms of another popular discourse, we will eventually be able to determine ‘what works’. That the assumptions are (quasi-)causal can already be seen from the language that is being used, that is, a language of ‘inputs’ and ‘outcomes’, of ‘working’ and of ‘production’. But it can also be seen from recent research that tries to identify those factors that ‘make a difference’ (see, for example, Hattie, 2008).

TLRP's ten principles also seem to be informed by quasi-causal assumptions. This can not only be glanced from the central role of the idea of ‘effectiveness’, but also from the ‘conceptual map’, which basically consists of a number of input variables, a process space where we find curriculum, pedagogy and assessment, and all this feeding into ‘learning outcomes’ (see James and Pollard, 2012b: 278). While James and Pollard stress that the TLRP research projects have not managed ‘to make unequivocal claims about findings in terms of categorical knowledge or cause-effect relationships’ (see James and Pollard, 2012b: 277), and while they also apologise for the fact that since most projects were conducted in authentic settings that made it ‘impossible to control all the variables operating at any one time’ (James and Pollard, 2012b: 277), these statements show precisely that they are concerned with the dynamics of education in the quasi-causal way outlined above.

One problem with causal assumptions about education is that they have difficulty with giving a place to the agency of the actors involved in education and, more specifically, with the fact of their reflexive agency – that is, with the fact that teachers and students can think and can act on the basis of their thoughts, judgements and decisions. One could, of course, suggest that these are precisely the aspects that need to be brought under control if education is ever able to work properly, and there is indeed a tendency within research on school effectiveness that considers the reflexivity of teachers and students as a problem rather than as part of the reality of education. But one could also argue that the fact that education ‘works’ as a result of the acts of reflexive agents implies that we need to think differently about its operation, not as a quasi-causal mechanical reality that happens behind the backs of those involved, but rather as a (complex) social reality constituted by the conscious acts of reflexive agents. In one sense we touch here upon a rather old and pervasive discussion in the field of education and social research more generally, which has to do with the ontological question of whether the object of such research can be likened to that of the physical sciences, or whether it has its own distinctive nature. This is, of course, an important discussion – one that, in my view, is all too often replaced by the far less interesting methodological discussion about so-called ‘qualitative’ and ‘quantitative’ research – but slightly too big for the purposes of this paper. A helpful way to make some progress here is through the language of systems theory. ⁸

Systems theory helps us to see that the idea of education as a production process – a process of input, throughput and output – and thus as a quasi-causal and, ultimately, a perfectly causal process only makes sense under very specific conditions. Such conditions can only be found in closed deterministic systems, i.e. systems that do not suffer from any interference from the outside and where, internally, the elements that make up the system work in a strictly deterministic way. While some (but actually not that many) systems in physical reality may operate in this way, educational systems do not operate like this, which already indicates why it is problematic to have assumptions about (perfect) causality with regard to education. The reason for this lies in the fact that education systems are open, semiotic and recursive systems. They are open in that the boundaries with the environment are never completely closed – children, for example, go home after school and are therefore subjected to many more ‘variables’ than just those that can be controlled by the school environment. They are semiotic in that the interactions between teachers and students are not based on physical push and pull, but on meaning and interpretation – students trying to make sense of what teachers are saying and doing and, through this, learning from their teachers. They are recursive in that the actions of the ‘elements’ in the system (i.e. teachers and students) feed back into the system and alter the direction in which the system will develop. The main reason for this lies in the fact that the ‘elements’ in the system are not stimulus-response machines but thinking and feeling beings who, based on their perceptions and interpretations, can choose to act in a range of different ways.

Looking at education in this way may raise the question of how the educational system manages to work at all as there are so many openings, slippages and uncertainties. Given that research is able to find correlations between ‘inputs’ and ‘outcomes’, the argument is also sometimes made that to depict education systems in the way I have just done must be mistaken, because research shows that education actually can work in quasi-causal ways. The response to this is threefold. The first point to bear in mind is that whereas it is possible to find correlations between variables of many open, semiotic and recursive systems, what this particular way of understanding the ‘working’ of educational systems shows is how such correlations are actually ‘achieved’, i.e. that they are the result of people trying to make sense, trying to communicate, trying to teach and trying to be taught, rather than that they happen behind the backs of those involved. (Stating it in this way can also be an argument for saying that if this is how such correlations are ‘achieved’ then it actually makes sense to involve those who are doing the ‘work’ rather than just treat them as objects of interventions.)

The second point that follows from depicting education in the way I have chosen, is that it helps to understand much better what actually needs to be done (and is done) in order to make open, semiotic, recursive systems operate in more predictable and ultimately more deterministic ways. The key principle here is that of complexity reduction, which is about reducing the number of ‘options’ for action for the ‘elements’ in the system. ⁹ Complexity reduction can be undertaken in three ways: (1) by reducing the openness of the system; (2) by reducing the semiosis that goes on within the system; and (3) by reducing the recursivity of the system, either by blocking feedback loops or by stopping the ‘elements’ from acting in reflexive ways. When we look at education we can actually see concrete examples of complexity reduction in order to make education ‘work’. School buildings, curricula, time-tables and also interventions in the home context are all attempts at reducing the openness of the education system by reducing the potential impact of influences from the outside. Assessment is one key way in which education reduces and tries to control ‘meaning making’ by making selections from all the possible interpretations that students generate and identifying some as ‘correct’, ‘right’ or ‘true’ and others as ‘wrong’, ‘mistaken’ or ‘false’. Furthermore, many operational procedures – such as teachers’ meetings, complaints procedures, but also professional development – are all measures to reduce and control feedback loops.

In one sense there is nothing wrong with the reduction of the complexity of the education system as it is precisely through such measures that the system can work. But when we look at the ways in which education can be made to work in this way we can begin to imagine that there is a tipping point where complexity reduction turns into unjustifiable control. This is the point where, for example, all communication with the outside world is blocked; where students are instructed what they should do, say and think; and where teacher reflection and student reflection are taken out of the system by labelling them as ‘disruptive interventions’, for example. The point here is that it is to a large extent possible to turn open social systems, such as education, into closed deterministic systems, but that this always comes at a price. Therefore, the key question is not whether we should try to reduce the complexity of the education system, but to what extent we should be doing this, for what reasons and what the price is that we should be willing to pay for this.

One critical ‘tipping point’, as I wish to suggest, concerns the moment where students can no longer appear as subjects of initiative and responsibility and are turned into objects of educational intervention. For any education that takes its orientation towards the becoming-subject of students seriously, this obviously poses a clear limit, one where education turns into indoctrination (which is not to suggest that it is always easy to identify exactly where this limit lies or where it is encountered).

What do these observations mean for educational improvement and the role of research in it? I wish to suggest that if we approach the task of educational improvement in terms of a quasi-causal understanding of the dynamics of educational processes and practices, we are in a sense operating with a ‘black box’ concept of education where we assume some kind of (mysterious) connection between ‘inputs’ and ‘outcomes’, but have little idea how such connections are actually achieved. This is why correlational research between ‘inputs’, ‘outcomes’ and ‘mediating variables’ without a proper theorisation of the underlying dynamics, is actually of limited use as it neither gives us an understanding of how such correlations are ‘achieved’, nor provides us with an understanding of the potential drivers for educational change towards improvement. ¹⁰ While one can debate, of course, what would count as a ‘proper’ theorisation, I have suggested that systems theory, particularly interpreted in terms of complexity, provides a more helpful theorisation, as it is able to account for the fact that the agents operating in the educational system are reflexive agents rather than stimulus-response machines. This not only provides a more accurate ‘social ontology’ of education but also a more educational approach, on the assumption that education is ultimately interested in promoting the reflexivity of those involved – both students and teachers. As I have already indicated above, much of this is lacking in the way in which TLRP's ten principles have been formulated and conceived, as there the modelling of education itself is done in a quasi-causal black box approach.

The practical roles of research

The foregoing observations also have important implications for what we expect from research. In the discussion about educational improvement in terms of effectiveness and ‘what works’, one could say that the main if not only expectation there is about research is that it generates technical knowledge, that is, knowledge about possible relationships between variables – and in education the focus is perhaps first and foremost on the relationship between those variables that can be controlled by the teacher, which include pedagogy (in the narrow sense), curriculum and assessment. Technical knowledge, however, tends to rely on quasi-causal assumptions about the dynamics of education. While, as I have suggested, open, semiotic, recursive systems such as education can be pushed towards greater predictability by reducing the complexity of its operation, thinking of them in quasi-causal terms assumes a ‘black box’ approach that does not generate an understanding of the actual dynamics at work. In this regard, the quest for technical knowledge in education is problematic, perhaps first and foremost because such knowledge is in itself unable to provide insights into why things are actually working as they work. If we think of such knowledge and the research that generates such knowledge as part of a much wider attempt to reduce and ultimately control the complexity of the educational system, we can begin to see something of the politics of complexity reduction that operate through such a research-policy-practice ‘complex’ (for a further discussion of this particular and rather recent phenomenon, see also Biesta, 2012).

One could see this as an argument for the need for a different kind of technical knowledge and, hence, a different kind of research, one that actually probes deeper into the dynamics of educational systems. While in one sense this is indeed what follows from the line I have been pursuing in this paper, to suggest that this is still a kind of technical knowledge misses an important point about the social ontology of education I have tried to articulate here, which is the fact that this particular social ontology assumes that education works as a result of the intentional activities of reflexive agents. This means that an important avenue for educational change towards improvement is to be found in the ways in which the agents in the situation make sense of it and the activities going on inside it. Just as knowledge about possible connections between actions and consequences might inform the perceptions, judgements and actions of educational agents, so do different conceptions and interpretations of what might be going on. It does, after all, make all the difference whether one sees a classroom in terms of behavioural objectives, learning difficulties, inclusion, legitimate peripheral participation, critical race theory or teaching as a gift – to name but a few different ways in which educational processes and practices can be made meaningful. The kind of knowledge that research can offer in relation to this – and I do take ‘research’ in the broad sense of including empirical and theoretical scholarship – can, according to De Vries (1990), be called cultural knowledge, as it is knowledge that provides us with different interpretations of educational phenomena (including the important but difficult task as to what it means to ‘name’ a phenomenon as an educational phenomenon in the first place; see Biesta, 2011).

De Vries utilises the distinction between these two ‘modes’ of knowledge to identify two different ways in which research can actually be of practical use: one which he refers to as the technical role of research and the other which he refers to as the cultural role. When research performs a technical role vis-à-vis educational practice, it provides technical knowledge, that is, knowledge about possible relationships between ‘inputs’ and ‘outcomes’ and, most importantly, about those ‘inputs’ that can in some way be controlled by teachers and other educational agents. When research performs a cultural role vis-à-vis educational practice it provides different interpretations, different ways of ‘meaning making’ and different ways of ‘sense making’. The point De Vries makes – and this is in line with what I have been trying to argue in this paper – is that along both lines research can contribute to the improvement of educational practices, which means that research can do more than just what is assumed in discussions about effectiveness and ‘what works‘. What I have tried to argue in addition to this, is that the technical role is actually rather limited and that a more complex understanding of the dynamics of educational processes and practices actually hints at the need for different knowledge and understanding – particularly knowledge and understanding that take the role of reflexive intentional agents into account. The irony here is that whereas much educational research assumes the existence of such reflexive intentional agents – otherwise, why would researchers bother to formulate principles for the improvement of educational practice – when such recommendations come in a technical form, they imply at the very same time that education works behind the backs of such reflexive intentional agents.

Although James and Pollard (2012b) concede that the TLRP projects have actually generated only a limited amount of technical knowledge, it is therefore striking that the ten principles are mainly formulated on the basis of a quasi-causal understanding of education and on the idea that what the improvement of education requires first and foremost is technical knowledge about how to make the overall operation more effective.

Conclusions

In this paper I have tried to raise a number of critical questions about educational improvement and the role research can play in it. I have used TLRP's ten principles for effective pedagogy as a ‘case’ for exploring these questions in more detail. I have argued that educational improvement cannot be understood as just an increase in the effectiveness of the educational operation, but always needs to engage with the question of what education should be effective for, that is, with the question of educational purpose, as it is only in relation to this that a distinction between educational change and educational improvement can be made. I have suggested that the multidimensional nature of educational purpose puts a further limit on effectiveness thinking, in that what might be an effective approach or strategy with regard to one (domain of) purpose may not be effective in relation to another. This is why educational judgement is always required and research can never be translated into abstract and general principles for effective pedagogy. In addition, I have shown that much talk about educational improvement relies on a quasi-causal concept of education, which basically refrains from theorising the dynamics of education, but rather relies on a black box account that looks for correlations between ‘inputs’ and ‘outcomes’. I have used insights from complexity and systems theory to open up this black box, particularly in order to acknowledge that education always involves reflexive intentional agents. I have shown how such a ‘social ontology’ of education opens up new avenues for educational improvement – avenues that always need to engage with normative questions about the price we are willing to pay for making educational systems more predictable. I have also suggested that this puts further limits on the possibility of generating technical knowledge about education and at the very same time provides us with a better understanding of the importance of what I have termed ‘cultural knowledge’ in relation to the improvement of education.

Do any lessons follow from this in terms of where educational research might go if it is committed to the improvement of education? I think it does, but perhaps the main lesson to learn from it has to do with the urgent need to gain a better understanding of education as a teleological practice, so that any questions about improvement are always dealt with pragmatically, that is, in relation to what it is one aims to achieve through education. There is also an urgent need to gain a better understanding of the dynamics of education, for which I have suggested the value of a complexity informed understanding. This partly points at the need for more and better educational theory. And it points at the need to move beyond futile discussions about ‘qualitative’ versus ‘quantitative’ towards discussions that take their starting point in questions about the particular nature of educational processes and practices. The case of TLRP's ten principles for effective pedagogy shows that much of this is still lacking in ‘the largest programme of educational research on teaching and learning that the UK has ever seen’.