A critical analysis of the International Baccalaureate’s Middle Years Programme assessment design with particular focus on feedback

Abstract

The International Baccalaureate’s Middle Years Programme (IBMYP) is designed to support the development of creativity, critical thinking, international-mindedness and values. However, close inspection of the programme’s assessment structure suggests that many of the competence-related and dispositional elements of the programme’s construct are to be assessed using performance rather than competence models. The article discusses and problematises this tension. Furthermore, the salient formative assessment strategy of feedback, widely researched and accepted as a teaching technique with considerable gains for learning, is not sufficiently explained or developed in the IBMYP assessment guide to allow for deep understanding of its potential to move learning forward. The article argues for a more competence-related assessment design in the IBMYP with greater detail around the strategy of feedback for learning.

Keywords

Assessment competence feedback IBMYP performance

Introduction

This article is intended for policy makers, researchers in education and international education, teachers, school leaders and administrators with knowledge and experience in various branches of Middle School education and/or the International Baccalaureate programmes and, most specifically, the Middle Years Programme (MYP). I have two aims: first to problematise the assessment design of the MYP in general, using Bernstein’s performance/competence model as a framework to stimulate reflection on the fundamental purposes of assessment and, second, with particular focus on feedback – arguably the most pertinent component of assessment with a formative purpose – to investigate how in the case of the MYP, and by implication also other curriculum frameworks, the classroom strategy of feedback can be best presented in teacher support material.

One and a-half decades into the 21st century, enough research has been conducted for us to know which variables drive learning. Out-of-school factors such as employment, ‘family income, parental styles, parental educational level, geography, and resources in the home such as the number of books available to children’ (Milner, 2013: 5) play a crucial determining role, whereas the most powerful inside-of-school predictors of learning are teachers and teaching (Friedman and Mandelbaum, 2011; McKinsey & Company, 2007; Wiliam, 2011).

Multiple reports such as those generated by the McKinsey group, United Nations Educational, Scientific and Cultural Organization (UNESCO) or the World Bank, peer-reviewed research and research-backed publications point to a handful of in-school strategies that have the potential to raise learning considerably. These include teacher recruitment, teacher collaboration (planning together, peer observation), educationally minded and focussed school leadership, relevant, ongoing teacher professional development and specific classroom strategies such as mastery learning, building on prior knowledge, peer-learning, questioning and feedback.

In fact, according to some studies (Hattie and Timperley, 2007; Wiliam, 2011), feedback is among the most important drivers, if not the most important driver, for student learning. It would make sense, therefore, to evaluate the role feedback plays in educational systems, particularly those that are reputed for excellence. Feedback is a term related to engineering and while it evokes accurately the idea of a loop that allows a system to regulate, it can be misleading when understood this way as the system that is being regulated is not a mechanical, static object but rather the complex, developmental phenomenon of learning. Winnie and Butler’s (1994) definition is that ‘feedback is information with which a learner can confirm, add to, overwrite, tune, or restructure information in memory, whether that information is domain knowledge, meta-cognitive knowledge, beliefs about self and tasks, or cognitive tactics and strategies’ (p. 5740). Feedback has been defined as ‘any of the numerous procedures that are used to tell a learner if an instructional response is right or wrong’ (Kulhavy, 1977: 211) and ‘information about the gap between the actual level and the reference level of a system parameter which is used to alter the gap in some way’ (Ramaprasad, 1983: 4).

This article provides a critical analysis of the clarity, detail and usefulness of the suggestions made about feedback in the International Baccalaureate’s Middle Years Programme (IBMYP), one of the suite of IB programmes which also includes the Primary Years Programme (PYP: ages 4–11), Diploma (ages 16–19) and Career-related Certificate (ages 16–19) offered by the so-called IB World Schools. The MYP, designed for students aged 11–16 years with a focus on interdisciplinary, concepts-focussed learning through criterion-related assessment, is a curriculum framework for international education. It ‘encourages students to become creative, critical and reflective thinkers […], emphasizes intellectual challenge, […] and fosters the development of skills for communication, intercultural understanding and global engagement’ (IB, 2014b). Furthermore, the International Baccalaureate’s published research asserts that ‘MYP improve[s] teacher pedagogy by encouraging collaboration and teaching beyond tested material’ (Kobylinski-Fehrman, 2013) and that ‘the majority of MYP teachers indicated that they are better teachers as a result of teaching in the MYP’ (IB, 2013a).

Out of the various IBMYP publications covering curriculum, rules, standards and regulations, the document that discusses feedback for learning is the 2014 Middle Years Programme, MYP: From principles into practice (hereafter abridged as MYP principles into practice).This guide applies ‘to all teachers in all IB World Schools offering the MYP’ and ‘provides a guide to teaching and learning in the context of the [MYP]’ (IB, 2014a: 1).

My ontological premise is that the strategy of feedback on and for learning is fundamental whereas the epistemological basis of this article’s criticism is constructivist: as such points will be built up iteratively and syllogistically, using researched evidence as premises and logical, demonstrative argument as method. This qualitative epistemological methodology suits the content of the article in that it discusses the quality of feedback in assessment and will recommend that increased descriptive, analytical and expository argument for feedback should feature in the MYP principles into practice.

Having outlined the MYP’s model of assessment, I will pay particular attention to the theme of feedback, discussing what is put forward in the guide. In doing this, I will suggest that feedback is an area that particularly suits the MYP philosophy of learning. I will then elaborate on the broader importance of feedback, citing examples from the research literature on how it should be used to enhance student learning. This will allow me to return to the MYP with suggestions for more detail, clarity and concrete examples concerning the area of feedback as discussed in the MYP principles into practice. My conclusions on feedback are made within a wider discussion of the MYP’s assessment design.

The MYP’s model of assessment

The overarching construct that the MYP aims to assess is articulated in the IB Learner Profile (IB, 2013b), a published statement of desired cognitive, affective, conative and social competences and attitudes that IB schools should nurture in students: ‘IB Learners […] strive to be inquirers […], knowledgeable […], thinkers […], communicators […], principled […], open-minded […], caring […], risk-takers […], balanced [and] reflective’ (IB, 2013b). All IB programmes ‘focus attention on the processes and the outcomes of internationally minded learning described in the IB learner profile’ (p. 9). More specifically,

From its beginning, the MYP was guided by three principles that have had special currency for learners aged 11–16, inspired by the IB mission: holistic learning, intercultural awareness and communication. These fundamental concepts of the programme provide a strong foundation for teaching and learning in the MYP. (IB, 2014a: 4)

The MYP and competence or performance assessment models

From its inception in 1994, the MYP has focussed on the process of learning, whether it be through the programme’s ‘Approaches to Learning’, advocating ‘process-focused disciplinary and interdisciplinary teaching and learning’ (p. 20), an emphasis on reflection that should involve ‘process journals, portfolios and self-assessment’ (p. 31) or learning through service and action. The reader is told that ‘for students, IB World Schools support lifelong learning when they emphasize “learning how to learn”, helping students interact effectively with the learning environments they encounter in school and beyond’ (p. 10).

The idea that learning is a process that should take individual learners’ needs into account is paramount to MYP educational philosophy. This is not surprising given the ambitious construct of the Learner Profile towards which the MYP is aimed, many of the facets of which require a wide base of personalised, experiential and metacognitive learning in order to be assessed successfully (‘caring’, ‘principled’, ‘balanced’, ‘reflective’). To use Bernstein’s 2000 dichotomy of pedagogic practice (performance/competence) as evoked by Cambridge (2010), the MYP clearly lends itself to an integrated competence model of assessment which ‘reveals […] competence development […], personalised forms [of control] with focus upon the intentions, dispositions, relations and reflexivity of the acquirer’ (Bernstein, 2000: 47–48). This progressive, learner-centred approach to education (Fitz et al., 2006: 7) permeates the MYP: students end the 5 years of the MYP with a personal project whereby they should ‘consolidate prior and subject-specific learning and develop an area of personal interest. Personal projects revolve around a challenge that motivates and interests individual students’ (IB, 2014a: 6).

Interestingly, recent developments in the MYP (termed the ‘next chapter’), whereby ‘rigorous, criterion-related internal assessment (coursework) for all subject groups, as well as an optional range of externally marked or moderated eAssessments [examinations]’ (IB, 2013c: 10) seem to point to Bernstein’s (2000) performance mode, which ‘places the emphasis upon a specific output of the acquirer upon a particular text the acquirer is expected to construct and upon the specialised skills necessary to the production of this specific output, text or product’ (p. 44) which is very much ‘the dominant, established model with […] evaluation focused on […] explicit and specific criteria’ (Fitz et al., 2006: 6). The ‘next chapter’ involves more centrally designed assessment criteria, whereas earlier models of the MYP allowed greater degrees of teacher and student freedom in developing assessment rubrics. One might argue that the reason for this development is to align assessment protocols in the IB with dominant and cost-effective models involving high-stakes testing of how well students can perform under pressurised conditions, rather than what they can do over time.

The extent to which the MYP assessment construct is viewed as a performance or competence model is important for the scope of this article as it will influence the style, content, purpose and design of assessment for formative purposes, in which feedback plays a central role. This is something that will ultimately be left to individual schools to decide upon – since externally assessed examinations are optional. This much said, given the potential that feedback has to move learning forward, whatever the specifics of the learning destination (performance-related or competence-related) as a formative assessment strategy, it is surely worthy of investigation. I will return to some of the implications of the ‘next chapter’ in the article’s conclusion.

The range of assessment instruments in the MYP

The MYP principles into practice guide states that

In IB programmes, assessment is ongoing, varied and integral to the curriculum. Assessment may be formal or informal, formative or summative, internal or external; students benefit from assessing their own work and the work of others. IB students demonstrate their learning through a variety of assessments and consolidations of learning, culminating in the MYP with the community project or the personal project. (IB, 2014a: 13)

A host of assessment strategies are listed in order to provide teachers with examples of some of the ways they might use assessments for formative or summative purposes. These include observation, selected response, open-ended tasks, performance, process journals, and portfolio assessments (pp. 85–86). The guide also includes examples of assessment tasks (‘Compositions – musical, physical, artistic; Creations of solutions or products in response to problems; Essays; Examinations; Questionnaires; Investigations; Research; Performances; Presentations – verbal [oral or written], graphic – through various media’ [pp. 86–87]). The assessment chapter explains how teachers can plan assessments, generate final assessments and record assessment data.

This is clearly a strong point in the MYP principles into practice guide, since the ambitious project of nurturing the Learner Profile in students suggests a broad range of assessment techniques, and the guide covers a wide repertoire. The guide presents a balance of progressive assessment strategies that are aligned with competence models (observation, process journal, portfolio assessment, investigation, research), and those typical of performance models (selected response, performance, essays, examinations, questionnaires). However, closer inspection of the language used to describe assessment in this section of the guide reveals an overwhelming predilection for terms strongly associated with classical performance models where, according to Bernstein (2000), ‘positional control’ (p. 47) is held within the confines of the assessment structure rather than the learner. As such, the various assessment strategies in the MYP principles into practice guide explain how students are assessed ‘to show clearly what they can achieve’ (IB, 2014a: 85), observation is of ‘behaviours and skills’, students should be able to ‘demonstrate predetermined learning objectives’ and this is ‘based on the theory that understanding is not something we have – like a set of facts we possess – but rather is something we can do’. Furthermore, tasks should be ‘used to develop suitable and appropriate performances of understanding’ and students are assessed on their ability to ‘meet MYP objectives’ (p. 87).

One might wonder what the value is behind students performing to be ‘principled’, ‘reflective’ or interculturally aware and to what extent high-stakes assessments such as the examinations proposed in the ‘next chapter’ might conflict with the aspirational values of the Learner Profile. The outcomes-based, behaviouristic and performative thrust of many of the assessment strategies suggested in the MYP principles into practice guide is somewhat misaligned with the competence-driven construct of the Learner Profile. To what extent, therefore, these assessment strategies are fit-for-purpose is a salient question: It could be said that there is potential misalignment of the three vertices of the assessment triangle as described by Pellegrino et al. (2001) (construct, observation instrument, interpretation).

Furthermore, the research on the MYP, much of it commissioned by the International Baccalaureate, uses performance models for comparison: ‘final (optional) external assessments for MYP students are internationally benchmarked, balancing valid measurement with reliable results’ (IB, 2014: 13). To demonstrate that the anticipated effects of the MYP assessment model are that it creates gains for learning, a 2011 study was undertaken by the Australian Council for Educational Research (ACER) where ‘50,714 international students participated in ISA assessments, of which 68% were IB students, and 32% were non-IB students’ (Tan and Yan, 2011: vi). Results showed that for Grade 9 and 10 students (therefore those studying the MYP), International Schools’ Assessment (ISA) means were ‘all significantly higher than the PISA means in both Mathematical Literacy and Reading’ (p. 50). There was, however, ‘insufficient evidence to suggest that IB schools authorized for a longer period produce better student outcomes’ (2011, Abstract).

Feedback in the MYP

The MYP principles into practice guide gives details specifically on assessment for formative purposes in two sections called ‘approaches to teaching’ (IB, 2014a: 72–76) and ‘assessment for learning’ (pp. 78–96) but the overarching ideas of constructivist pedagogy, communication and stimulating various approaches to inquiry-based and concepts-focussed learning run through the entire document. The guide makes a number of salient statements about learning that imply the centrality of feedback: ‘schools need to ensure that the relationships students establish with each other and with teachers, which are of central importance to development and learning, will flourish’ (p. 74). The guide also states that ‘schools must regularly report student progress towards the MYP objectives using the prescribed subject-group assessment criteria’ (p. 80).

Feedback is specifically mentioned in the MYP principles into practice guide. School assessment policies must include ‘a process for gathering feedback’ (p. 36) as part of a design to support students in their learning: ‘students will experience varying levels of support in their units, since peer conferencing, formative assessment with feedback from the teacher, editing and correcting are all essential learning tools’ (p. 84).

The document ends with a list of ‘research perspectives’ that have influenced the MYP (p. 116), over 35 ‘international curriculum benchmarks’ (pp. 117–118) and over eight pages of ‘bibliography and suggested further reading’ (pp. 119–127), quoting authorities such as Lynn Erickson, Carol Dweck, David Perkins, Grant Wiggins, Howard Gardner, Art Costa, Michael Fullan and classic references such as Benjamin Bloom, Lev Vygotsky and Jerome Bruner. Although one reference from John Hattie is mentioned (Hattie et al., 1996) and there are some references to well-known studies of assessment for formative purposes such as Working Inside the Black Box: Assessment for Learning in the Classroom (Black et al., 2002) and the studies of Brookhart (2010) and Brookhart et al. (2009), John Hattie and Helen Timperley, whose seminal 2007 study is the most comprehensive single study of feedback, do not feature in the list – nor do any of the established references on feedback (Hattie, 1999; Hattie and Timperley, 2007; Kluger and DeNisi, 1996; Kulhavy, 1977; Locke and Latham, 1984; Sadler, 1989; Winnie and Butler, 1994).

Overall, feedback is mentioned 17 times. These instances can be divided into three broad groups: timely feedback, meaningful feedback and the purposes of feedback. I will discuss these themes in this section of the article and then again when I turn to what the research says about feedback.

Timely feedback

The cadence of feedback is evoked in the MYP principles into practice, by stating that it should be ‘timely’ (IB, 2014a: 79), ‘regular’ (‘teachers should provide students with regular […] feedback’ [p. 20]), ‘consistent’ (p. 79) and ‘continuous’ (p. 91). Furthermore, teachers are encouraged to reflect on the question of cadence. For summative assessments, for instance, teachers should ask the question ‘how and when will students receive feedback?’ (p. 64). In outlining one specific area of assessment (selected response), the guide mentions instant feedback: ‘this strategy is particularly useful during the course of a unit, in formative assessment, as it is usually quick and straightforward to administer and can provide instant feedback for students and teachers’ (p. 85).

Some questions that might arise from these points would include exactly what ‘timely’ means, whether feedback can be delayed or should be instant, and why it is important for feedback to be ‘continuous’, ‘regular’ and ‘consistent’.

Meaningful feedback

Many of the mentions of feedback in the MYP principles into practice make it clear that it should be meaningful: giving and receiving ‘meaningful feedback’ (pp. 79, 98–99) is evoked in various areas of learning whilst ‘to understand what students have learned, and to monitor their progress, teachers use a range of assessment strategies that provide meaningful feedback’ (p. 13). When discussing action and service, the reader is told that

good practices developed by schools with successful MYP service programmes include […] guided practice in critical reflection, including models and strategies that help students create meaning from their experience in service activities, as well as meaningful feedback from peers, teachers and other adults. (p. 25)

Of course, what ‘meaningful’ means is a crucial question, the answer to which will determine the style and content of feedback. ‘Meaningful’ could be synonymous with ‘comprehensible’, indicating that students must understand exactly what is being fed back to them. It could, on the other hand, be synonymous with ‘salient’, suggesting that the substance of the feedback should be relevant to the learning process – in other words feedback should be on drivers of learning and not peripheral or insignificant elements of learning. ‘Meaningful’ is also contingent on context and learning objectives – so ‘meaningful’ feedback on a test will not necessarily look the same as it would on a reflective portfolio. Finally, ‘meaningful’ suggests personal relevance, leaving teachers with the task of making feedback resonate with the student to motivate him or her to improve.

The purposes of feedback

What exactly feedback is for is made clear in the MYP principles into practice guide: ‘the development of ATL [Approaches to Learning] skills’ (p. 20), programme requirements (‘teachers in IB World Schools develop, administer and provide feedback on assessment tasks that meet the programme requirements’ [p. 43]) and learning (‘assessment in the MYP aims to: support and encourage student learning by providing feedback on the learning process’ [p. 78]). In the context of peer and teacher conferencing, the purpose of feedback is described as something that can be used ‘to gain insight into the task, topic, concepts and skills at hand’ (pp. 114–115).

The idea that feedback should be given towards performance goals is made clear: ‘to help students improve performance through […] feedback’ (p. 79) while ‘all units include summative tasks that are assessed according to one or more MYP criteria to ensure continuous assessment and feedback of students’ performance against the MYP objectives’ (p. 91). The reader is informed that ‘the MYP places an emphasis on assessment processes that involve the gathering and analysis of information about student performance and that provide timely feedback to students on their performance’ (p. 79).

These descriptions are helpful but still leave the reader with many unanswered questions, for if the purpose of feedback is to advance learning against criteria, then which facet of the learning process should it target in order to be most effective? Should feedback be about the task, the process, the student’s metacognition or the student’s general disposition? When and how should feedback target any one of these specific areas?

A sentence that sums up the elements of timeliness, meaningfulness and purposes is the following: ‘Internal summative and formative assessments are closely linked, and teachers must use their knowledge of IB assessment expectations and practices to help students improve performance through consistent, timely and meaningful feedback’ (p. 79). This sentence also evokes questions around depth and specificity: are IB assessment expectations and practices always aligned (see my arguments about competence and performance), how timely should the feedback be, and when (is ‘timely’ the same across different constructs?), and why, and meaningful in which ways?

It seems clear, however, from reading the MYP principles into practice guide, that feedback is identified as an important part of formative assessment. This makes sense as it is very much in keeping with the constructivist pedagogical stance that the IB advocates with inquiry-based learning at the centre; teachers are to help ‘students to become confident, independent, self-managed learners for life’ (p. 65).

Feedback in the MYP classroom

In a study on the MYP conducted between 2011 and 2012, published by the National Foundation for Educational Research, Sizmur and Cunningham (2012) found that 191 students out of 309 respondents (62%), when asked what helped them learn, cited ‘feedback from assessment’ (p. 65), putting it top of the list. ‘Despite the general positivity about the usefulness and availability of feedback, students in some schools felt that the quality of feedback varies by teacher’ (p. 66). Interestingly, while ‘nearly nine out of ten teachers (87%) felt that the MYP encourages the use of feedback in lessons […], teachers were less positive about specific impacts in the classroom’ (p. 35).

When teachers and students were asked how frequently students ‘make corrections to tests/projects using feedback from the teacher’, 8% of the teachers and 3% of the students replied ‘never’, 18% of the students and 9% of the teachers replied ‘rarely’, with 34% of the students and 30% of the teachers replying ‘sometimes’ (p. 75). In all, 58 parents were interviewed and ‘a number of parents raised concerns about teaching methods and curriculum in particular subject areas, for example, Science and Maths’ (p. 50). The study quotes a parent saying that ‘it’s up to the skill of the individual teachers to communicate [the assessment criteria] effectively. Some do a better job than others’ (p. 47).

The findings of this research suggest that whilst the MYP is clear about the value of feedback, there is less clarity and agreement among teachers and students on how well feedback is administered. This might be due to the unanswered questions around the nature of feedback I have evoked, which were no less explicit in the previous, 2008, version of the MYP principles into practice. As was the case with the 2008 version, whilst the 2014 guide mentions feedback extensively, it does not break it down in the way in which educational researchers have in the studies I shall describe next.

Educational research on feedback

Edward Thorndike’s 1931 publication Human Learning showed how important feedback was for learning by conducting an experiment whereby subjects drew lines repetitively without any feedback. The result of this ‘blind practice’ was that there was no real change or progress. In this experiment, it was shown that ‘practice without feedback produces little learning’ (Pellegrino et al., 2001: 87). Elawar and Corno’s (1985) study, meanwhile, clearly showed how specific feedback on how to improve ameliorated primary school students’ performances in mathematics. Butler’s famous 1987 and 1988 studies on the effects of written comments alongside grades gave the surprising results that grades alone had little effect on student performance, that written comments indicating how to improve caused significant gains, particularly concerning ‘ego involvement’ (Butler, 1987) but that grades would tend to wash out the potential effect of comments when the two were communicated to the student side-by-side, as they tend to be in most conventional reporting systems.

Sadler’s (1989) paper ‘Formative assessment and the design of instructional systems’ problematised feedback by explaining how the reception of and active response to feedback is what is critical, the point being to ensure that communicated feedback leads to self-regulation. Sadler pointed out that ‘feedback is commonly defined in terms of information given to the student about the quality of performance’ (p. 142) but that ‘it is insufficient for students to rely upon evaluative judgments made by the teacher’ (p. 143). In a five-stage feedback cycle developed by Bangert-Drowns et al. (1991), learners interact with the social environment as they adjust their cognition to new information in moving towards performance expectations. This model involves searching, responding, evaluating and adjusting and is not unlike inquiry models developed by Kolb (1984) and Short and Burke (1991).

Kluger and DeNisi (1996) offered a feedback intervention theory, explaining that feedback can be given at the levels of meta-task processes, task-motivation processes or task-learning processes. Essentially, they argued that when feedback is focussed on the latter, it is more productive, whereas feedback directed at meta-task processes is the least effective, with that focussed on task-motivation midway between the two.

Pellegrino et al.’s (2001) seminal Knowing What Students Know: The Science and Design of Educational Assessment makes it clear that ‘one of the most important roles for assessment is the provision of timely and informative feedback to students during instruction and learning so that their practice of a skill and its subsequent acquisition will be effective and efficient’ (p. 4). Citing Black and Wiliam (1998), they stipulate that ‘assessments that inform teachers about the nature of student learning can help them provide better feedback to students, which in turn can enhance learning’ (p. 31). They point out that ‘students learn more when they receive feedback about particular qualities of their work, along with advice on what they can do to improve’ (p. 38). An entire section of this study is dedicated to ‘Practice and Feedback’ (pp. 84–87). They point out, most saliently, that ‘one of the persistent dilemmas in education is that students often spend time practicing incorrect skills with little or no feedback. Furthermore, the feedback they ultimately receive is often neither timely nor informative’ (p. 87). The study goes on to discuss feedback as part of the learning process, arguing points for intelligent tutor system and teacher-generated feedback (pp. 232, 234, 257, 273). The main thrust of the discussion around feedback is that it should be timely and informative (p. 91). The study draws not only from the work of Black and Wiliam but also from Butler (1988), Kluger and DeNisi (1996) and Kintsch et al. (2000).

John Hattie’s 1999 inaugural address at the University of Auckland substantiates the value of feedback as a tool for improvement in learning by drawing on evidence from ‘337 meta-analyses, 200,000 effect-sizes from 180,000 studies, representing approximately 50+ million students’ (p. 5). From that pool of data, feedback (and remediation) features as one of the top drivers of student learning with an average effect size of .65 derived from 146 different effects (p. 8). Hattie states that ‘the most powerful single moderator that enhances achievement is feedback’ (p. 9). In a 2003 paper (‘Building Teacher Quality’), Hattie puts feedback top of the list of strategies for learning with an extremely large effect size of 1.13 (p. 5). Hattie gives greater granularity to the issue of feedback by dividing it into elements such as reinforcement, corrective feedback, remediation, diagnostic feedback and mastery learning (these are listed as useful types of feedback) as opposed to less effective strategies such as extrinsic rewards, immediate feedback and punishment (p. 10).

Hattie, and Hattie and Timperley, work off effect sizes in their argumentation but some caution needs to be taken in reading these standard deviations:

The magnitude of commonly made quantitative claims for the efficacy of formative assessment is suspect, to say the least. The most frequently cited effect-size claim of .4 – 7 standard deviations is neither meaningful as a representation of the impact of a single well-defined class of treatments, nor readily traceable to any inspectible, empirical source. (Bennet, 2011: 20)

In his 1999 paper, Hattie begins to elaborate the important distinction between task, and person-related feedback (initiated by Kluger and DeNisi), suggesting that the former is preferable. He also points out that feedback needs to be used to set goals, and explains, like Sadler, that what is crucial is the response to the feedback, how it is internalised and whether the receiver of the feedback is prepared to accept it or not (p. 15).

Hattie and Timperley go into more detail still in their 2007 paper, ‘The Power of Feedback’ – a useful and detailed overview of the question of feedback in educational research, what type of feedback works best and how feedback can be used in the classroom. In their ‘model of feedback to enhance learning’ (p. 87), three essential questions are asked (Where am I going? How am I going? Where to next?), while each question can operate at any of four levels (task, process, self-regulation and self).

Numerous rich, detailed claims are made about feedback (that for processes and tasks it is more effective in response to correct rather than incorrect responses [p. 85]; that rewards are not effective strategies [p. 84]; that personal or ‘self’ related feedback is rarely effective [p. 96] and that the timing of feedback should be contingent on the level at which it is given). Hattie and Timperley’s paper draws on work by eight pages worth of references and offers the reader an extremely composite view.

Dylan Wiliam’s (2011) Embedded Formative Feedback dedicates a chapter to feedback (pp. 107–132). Detailed points for consideration are developed, which include criticism of too much praise as an ineffectual feedback strategy (Brophy, 1981; Good and Grouws, 1975); the importance of timing (Bangert-Drowns et al., 1991) and the value of scaffolded responses as opposed to full answers (Day and Cordon, 1993), how feedback yields better results when the designated goals suggest improved performance (Kluger and DeNisi, 1996) and the all-important variable of self-perception, especially as regards unhelpful deterministic approaches to ability (Dweck, 2000, 2006).

Wiliam also discusses the relationship between grading and learning, suggesting that assessments with a formative purpose should be qualitative more than quantitative. He concludes by stating that ‘feedback should cause thinking. It should be focussed; it should relate to the learning goals that have been shared with the students; and it should be more work for the recipient than the donor’ (Wiliam, 2011: 132). Dylan’s book has the considerable benefit of giving numerous examples from across the world of what feedback can look like in different subject domains.

What one can take away from the research on feedback in education is that it has the potential to move learning forward dramatically but that, like so many questions in education, it is a subtle issue that must be investigated in detail to be fully appreciated. Feedback needs to be given in a timely, carefully scaffolded manner and many variables are contingent on its success. A major point of convergence in the literature is that feedback should be specific, that is to say goal-related. This is an issue that can be broken down into Hattie’s levels of feedback.

Above all, it is clear that feedback is a sensitive mechanism that can have adverse effects if it is not administered correctly. Much of this has to do with the response to the feedback, a question that, in turn, is relative to the recipient’s motivation, the context and the goal at hand. Wiliam warns readers that ‘when we give students feedback, there are eight things that can happen, and six of them are bad’ (p. 131). Clearly, teachers should treat the question of feedback with care.

Revisiting the MYP principles into practice in the light of what the research has to say

To return to the discussion around assessment in MYP principles into practice, and more specifically the area of feedback, we can start to answer some of the questions that teachers might have about timely feedback, meaningful feedback and the purposes of feedback.

Timely feedback

The importance of the timing of feedback is emphasised by Pellegrino et al. (2001: 4, 87, 91, 103, 272) and Wiggins (2012). The timeliness of feedback is something that Hattie (2003) emphasises too (Hattie and Timperley, 2007).

As we have seen, ‘timely’ is not the only temporal marker used in the MYP principles into practice guide; the reader is also told that feedback should be ‘ongoing’. This coheres with the larger concept of concurrency in the literature on learning. Pellegrino et al. (2001) state that ‘ongoing classroom assessment [is] an integral part of teaching practice’ (pp. 96, 225–226) and that learning is an ongoing process (p. 228) that should incur ongoing reflection (p. 293). Hattie (2003) also mentions how expert teachers tend to ‘monitor [students’] ongoing solution attempts’ (pp. 7–8) and Hattie and Timperley (2007) describe learning as an ‘ongoing’ process (p. 88), whereas Wiggins (2012) claims that ‘the more feedback I can receive in real time, the better my ultimate performance will be’.

One area that could be debated is when ‘instant feedback’ (IB, 2014a: 85) is described as a potential benefit to be associated with selected response assessment strategies such as tests and quizzes. In their 2007 paper, Hattie and Timperley nuance the question of the timing of feedback considerably: ‘immediate error correction during task acquisition […] can result in faster rates of acquisition, whereas immediate error correction during fluency building can detract from the learning of automaticity and the associated strategies of learning’ (p. 98). They go on to substantiate the view that immediate feedback is useful for process rather than task and point out that, when the learning experience involves more difficulty, some delay is beneficial as opposed to the situation with easier solutions which can be fed back on immediately.

Furthermore, from reading the MYP principles into practice guide, it is clear that teachers can make mistakes: for example, giving immediate feedback on a difficult, task-related learning experience and delayed feedback on a much more straightforward, process-related issue. The research cited in this article helps us to understand this.

Meaningful feedback

If Wiliam places emphasis on the relationship between mindfulness, thought and feedback to state the importance of the feedback process involving careful reflection, then this overarching principle is reflected in the MYP document where teachers are encouraged to ask themselves questions such as ‘how and when will students receive feedback?’ (IB, 2014a: 64).

From Sadler through Dweck, including Wiliam, Hattie, and Hattie and Timperley, one sees that a considerable challenge facing feedback is knowing how students will respond emotionally to feedback, being able to work around the motivational elements that are essential features of a positive or negative response to feedback. Therefore, one of the first steps that can be taken to ensure that feedback is meaningful in so much as it resonates positively with the student is to understand that it is not one-way traffic and must involve dialogue between the student and the teacher with much listening, empathy and caring.

From reading MYP principles into practice, one wonders what there is to prevent teachers from giving ineffectual feedback (e.g. praise, punishments, grades without comments) or feedback that is lost on deaf ears because it has not been designed sensitively. The research gives us some ideas on how to take these important interpersonal dimensions into account.

The purposes of feedback

While at the broadest level feedback is for learning, it is less than clear in much of the MYP principles into practice which precise element of learning is being fed back upon and how; therefore, feedback should be given in each of those instances. This is an important area to develop since research shows how feedback, when administered in the wrong context, can have adverse effects.

A particularly useful model, building on those developed earlier by Bangert-Drowns et al. and Kluger and DeNisi, is Hattie and Timperley’s model that addresses levels of feedback (task, process, self-regulation and self) and guiding questions (Where am I going? How am I going? Where to next?). However, a model is not enough as it needs to be understood in the context of when and where feedback works, and where it does not.

One area of feedback in MYP principles into practice that corresponds closely with the literature is goal-setting. The guide mentions learning goals 12 times and refers to ‘challenging goals’ (IB, 2014a: 9), a strategy to which Hattie attributes noticeable effect size. However, challenging should still be achievable (Vygotsky’s zone of proximal development). It needs to be made clear to teachers that feedback ‘is most powerful when it addresses faulty interpretations, not a total lack of understanding. Under the latter circumstance, it may even be threatening to a student’ (Hattie and Timperley, 2007: 82).

I have argued that learning in a competence-driven model is not quite the same thing as learning in a performance-driven model, and there is a tension in the MYP as parts of the programme look to process and individual needs (competence) whereas others, particularly those evoked in the ‘next chapter’, require achievement against established criteria (performance). Teachers should be mindful of Butler’s 1987 and 1988 work on the relationship between comments and grades. Indeed, since the MYP insists on schools reporting either grades or achievement levels (IB, 2014a: 91), MYP schools will be faced with the wash-out effects that levels or grades have on comments. If the purpose of the feedback is to move the learning forward then, according to the research, it should come before or even instead of grades.

Ways forward

The MYP principles into practice guide rightly situates feedback as an important assessment for learning strategy and coheres with the research in the broadest sense. However, it does not break down the types of feedback that are possible, the possible effects, the different levels of feedback and the guiding questions that should take it forward. Indeed, all the reader is left with is the idea that feedback should be timely, ongoing, formative and appropriate.

There is something of an assumption in the MYP guide that teachers will know how to administer the right type of feedback in the right set of circumstances, and yet we know that ‘classroom assessments, which have the potential to enhance instruction and learning, are not being used to their fullest potential’ (Pellegrino et al., 2001: 1).

Fundamentally, the MYP assessment guide’s discussion of feedback is not nearly as detailed as other educational frameworks such as the Assessment for Learning website developed by the education departments of the States, Territories and Commonwealth of Australia (Assessment for Learning, 2013) or the Higher Education Academy (2013).

I would suggest three ways in which the MYP principles into practice sections on assessment can enhance and make more purposeful the discussion around feedback. First there is the issue of a definition. The guide would do well to explain what is meant by feedback, perhaps with reference to either Winnie and Butler (1994), Kulhavy (1977) or Ramaprasad (1983). Second, more detail and nuance should be given to the frequently evoked questions of timing, meaning and purpose. This would allow the guide to elaborate upon all-important facets such as the response to feedback, the relationship between grades and comments in reporting cycles, and the general style and nature of feedback. Finally, arguments about feedback should be demonstrative, offering a plethora of real-life examples of how feedback can be useful. Wiliam’s (2011) work gives the reader numerous examples from the classroom. These could be listed or tabulated in the way that assessment strategies and tools are in the MYP principles into practice guide. Without concrete examples, there is little guarantee against faulty interpretation and a lack of understanding of how feedback operates in the larger context of the assessment triangle stipulated by Pellegrino et al. (cognition, observation and interpretation).

Conclusion

The assessment design of the MYP draws on research, constructivism and student-centred learning; it is a forward-looking, dynamic vision of quality and relevance. However, the ‘next chapter’, by reverting – albeit in part – to high-stakes examinations, puts the overall value and purpose of the programme into question. The guide states that ‘MYP eAssessment offers students opportunities to demonstrate disciplinary and interdisciplinary understanding, international-mindedness, critical and creative thinking, problem-solving skills and the ability to apply knowledge in unfamiliar situations’ (IB, 2014a: 95). Why examinations are needed, and how fit-for-purpose they are as observation instruments on troublesome constructs such as creative thinking and international-mindedness, remains to be seen. This performance-driven assessment strategy will be a necessary path for ‘official IB recognition of achievement […], only available for students who participate in and successfully complete the required eAssessments’.

This puts the question of feedback into context: feedback for what exactly? If students are working towards narrow performance indicators such as those that naturally and inevitably arise from item-response examinations, this will surely encourage feedback on performance rather than feedback on competence, leading to discussions on how students should do, execute, perform, respond and react rather than how they might engage, try, learn about learning, position themselves and choose. The IB should remain a leader in international education by resisting any urge to conform to the national education world of high-stakes testing. This is not to say that the competence-related tasks that characterise the MYP have disappeared, since the personal project remains a pillar of the programme, and portfolios will be taken into account from 2014, but by awarding official IB recognition to students only once they have successfully completed ‘five on-screen examinations’ (p. 95) a new direction has been taken, towards performance rather than competence.

Within this changing landscape of the MYP, the issue has been evoked of how clear and effective the MYP’s advice on feedback is. Curriculum guides must ensure that short-cuts are not taken when representing the world of learning, especially when it comes to elements of learning that are established, researched predictors for substantive educational gains such as feedback.

I have argued that while feedback features prominently and in a relatively coherent manner in the MYP curriculum guide, it is not sufficiently broken down for there to be clear, useful understanding of what it entails and how it can be used to enhance learning. With a definition of exactly what we mean by feedback, more detail into the ways that it can function and concrete examples, the guide would take a step towards aligning the research literature with classroom practice most felicitously.

If an MYP education is for the 21st century – an age of volatility, uncertainty, complexity and ambiguity – then students should be encouraged to reflect on process, learn about learning and know how to re-position themselves on a shifting landscape. By keeping the development of competences at the centre of its assessed model of cognition and by increasing the quality of information around feedback for learning towards that model, the MYP will continue to aspire towards what Alvin Toffler (1970) stated so poignantly:

The new education must teach the individual how to classify and reclassify information, how to evaluate its veracity, how to change categories when necessary, how to move from the concrete to the abstract and back, how to look at problems from a new direction – how to teach himself. (p. 367)

Footnotes

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Author biography

Conrad Hughes is Director of Education at the International School of Geneva, where he also teaches IB Diploma Theory of Knowledge. He was educated in France, Swaziland and South Africa, holds a PhD in English literature from the University of the Witwatersrand, and has worked in schools in France, the Netherlands and India. He has published articles in academic and educational journals as well as an English textbook, published by Pearson, for students pursuing the International Baccalaureate Diploma.

References

Assessment for Learning (2013) http://www.assessmentforlearning.edu.au/default.asp?id=912

Bangert-Drowns

Kulik

C-L

. (1991) The instructional effect of feedback in test-like events. Review of Educational Research 61(2): 213–238.

Bennet

(2011) Formative assessment: a critical review. Assessment in Education: Principles, Policy & Practice 18(1): 5–25.

Bernstein

(2000) Pedagogy, Symbolic Control, and Identity. Lanham, MD: Rowman & Littlefield.

Black

Wiliam

(1998) Assessment and classroom learning, Assessment in Education: Principles, Policy & Practice 5(1): 7–74.

Black

Harrison

Lee

. (2002) Working inside the Black Box: Assessment for Learning in the Classroom. London: GL Assessment.

Brookhart

(2010) How to Assess Higher-Order Thinking Skills in Your Classroom. Alexandria, VA: ASCD Publishing.

Brookhart

Moss

Long

(2009) Formative assessment that empowers. In: Scherer

(ed.) Challenging the Whole Child: Reflections on Best Practices in Learning, Teaching and Leadership. Alexandria, VA: ASCD Publications.

Brophy

(1981) Teacher praise: a functional analysis. Review of Educational Research 51(1): 5–32.

10.

Butler

(1987) Task-involving and ego-involving properties of evaluation: effects of different feedback conditions on motivational perceptions, interest and performance. Journal of Educational Psychology 79(4): 474–482.

11.

Butler

(1988) Enhancing and undermining intrinsic motivation: the effects of task-involving and ego-involving feedback evaluation on interest and performance. British Journal of Educational Psychology 58(1): 1–14.

12.

Cambridge

(2010) The International Baccalaureate Diploma Programme and the construction of pedagogic identity: a preliminary study. Journal of Research in International Education 9(3): 199–213.

13.

Crisp

(2013) Criteria, comparison and past experiences: how do teachers make judgements when marking coursework? Assessment in Education: Principles, Policy & Practice 20(1): 127–144.

14.

Day

Cordon

(1993) Static and dynamic measures of ability: an experimental comparison. Journal of Educational Psychology 85(1): 76–82.

15.

Dweck

(2000) Self-Theories: Their Role in Motivation, Personality and Development. Philadelphia, PA: Psychology Press.

16.

Dweck

(2006) Mindset: The New Psychology of Success. New York: Random House.

17.

Elawar

Corno

(1985) A factorial experiment in teachers’ written feedback on student homework: changing teacher behaviour a little rather than a lot. Journal of Educational Psychology 77(2): 162–173.

18.

Fitz

Davies

Evans

(2006) Education Policy and Social Reproduction: Class Inscription & Symbolic Control. Oxford: Routledge.

19.

Friedman

Mandelbaum

(2011) That Used to Be Us: How America Fell behind in the World It Invented and How We Can Come Back. New York: Picador.

20.

Good

Grouws

(1975) Process-Product Relationships in Fourth Grade Mathematics Classrooms (Grant no. NEG-00-3-0123). Columbia, MO: University of Missouri.

21.

Hattie

(1999) Influences on student learning. Inaugural lecture, The University of Auckland, Auckland, 2 August. Available at: https://cdn.auckland.ac.nz/assets/education/hattie/docs/influences-on-student-learning.pdf

22.

Hattie

(2003) Teachers make a difference: what is the research evidence? In: Australian Council for Educational Research annual conference on building teacher quality. Available at: http://www.cdn.auckland.ac.nz/assets/education/hattie/docs/teachers-make-a-difference-ACER-(2003).pdf

23.

Hattie

Timperley

(2007) The power of feedback. Review of Educational Research 77(1): 81–112.

24.

Hattie

Biggs

Purdie

(1996) Effects of learning skills interventions on student learning: a meta-analysis. Review of Educational Research 66(2): 99–136.

25.

Heritage

(2008) Learning Progressions: Supporting Instruction and Formative Assessment. Washington, DC: Council of Chief State School Officers.

26.

Higher Education Academy (2013) http://www.heacademy.ac.uk

27.

International Baccalaureate (IB) (2013a) http://www.ibo.org/announcements/2013/mypdevelops.cfm

28.

International Baccalaureate (IB) (2013b) IB learner profile. Available at: https://www.ibo.org/myib/digitaltoolkit/files/pdfs/learner-profile-en.pdf

29.

International Baccalaureate (IB) (2013c) MYP: next chapter. Development report, Online Curriculum Centre, International Baccalaureate, August. Available at: http://share.nanjing-school.com/dailybulletin/files/2013/08/Next-Chapter-Aug-2013-17dueyc.pdf

30.

International Baccalaureate (IB) (2014a) Middle Years Programme, MYP: From Principles into Practice. Cardiff: IB.

31.

International Baccalaureate (IB) (2014b) The IB Middle Years Programme. Available at: http://www.ibo.org/myp/

32.

Kintsch

Steinhart

Stahl

. (2000) Developing summarization skills through the use of LSA-based feedback. Interactive Learning Environments 8(2): 87–109.

33.

Kluger

DeNisi

(1996) The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin 119: 254–284.

34.

Kobylinski-Fehrman

(2013) The International Baccalaureate Middle Years Programme and its effect on students in poverty. Georgia State University. Available at: http://digitalarchive.gsu.edu/eps_diss/104/

35.

Kolb

(1984) Experiential Learning: Experience as the Source of Learning and Development. Englewood Cliffs, NJ: Prentice Hall.

36.

Kulhavy

(1977) Feedback in written instruction. Review of Educational Research 47(1): 211–232

37.

Locke

Latham

(1984) Goal setting: A motivational technique that works. Englewood Cliffs, NJ: Prentice Hall.

38.

McKinsey & Company (2007) How the world’s best performing school systems come out on top. Available at: http://mckinseyonsociety.com/how-the-worlds-best-performing-schools-come-out-on-top/

39.

Milner

(2013) Analyzing poverty, learning and teaching through a Critical Race Theory Lens. Review of Research in Education 37(1): 1–53

40.

Pellegrino

Chudowsky

Glaser

(2001) Knowing What Students Know: The Science and Design of Educational Assessment. Washington, DC: National Academy Press.

41.

Ramaprasad

(1983) On the definition of feedback. Behavioral Science 28(1): 4–13.

42.

Sadler

(1989) Formative assessment and the design of instructional systems. Instructional Science 18(2): 119–144.

43.

Short

Burke

(1991) Creating Curriculum: Teachers and Students as a Community of Learners. Portsmouth, NH: Heinemann.

44.

Sizmur

Cunningham

(2012) International Baccalaureate Middle Years Programme (MYP) in the UK. Slough: NFER.

45.

Tan

Yan

(2011) Performance Comparison between IB School Students and Non-IB School Students on the International Schools’ Assessment (ISA) and on the Social and Emotional Wellbeing Questionnaire. ACER. Available at: http://www.ibo.org/research/policy/programmevalidation/pyp/documents/IB_ISA_report_Nov2011_Final.pdf

46.

Thorndike

(1931) Human Learning. New York: Century.

47.

Toffler

(1970) Future Shock. New York: Random House.

48.

Wiggins

(2012) Seven keys to effective feedback. Educational Leadership: Feedback for Learning 70(1): 10–16. Available at: http://www.ascd.org/publications/educational-leadership/sept12/vol70/num01/Seven-Keys-to-Effective-Feedback.aspx

49.

Wiliam

(2011) Embedded Formative Assessment. Bloomington, IN: Solution Tree.

50.

Winnie

Butler

(1994) Student cognition in learning from teaching. In: Husen

Postlethwaite

(eds) International Encyclopaedia of Education, 2nd ed. Oxford: Pergamon Press, pp. 5738–5745.