Abstract
This edited volume provides a substantive review of learning-oriented assessment (LOA) research as it has been conceptualized in the literature and is currently playing out across diverse language teaching contexts. Although many definitions of LOA are discussed, in general they accord with Fulcher’s (Chapter 3) observation that LOA “is defined by the tasks that learners are asked to do, learner involvement in the process of doing and assessing the tasks, and the feedback provided to the learner on task performance” (p. 34).
Throughout, the contributors acknowledge influential antecedents of LOA. They draw attention to the Assessment Reform Group (ARG), which reported on the negative consequences of large-scale assessment, argued for increased trust in teachers’ assessment, and reported that such formative assessment—undertaken on an ongoing basis by teachers and students for learning purposes—significantly improved overall school performance (cf. Black & Wiliam, 1998). Also from the general education literature, the contributors highlight Carless’ (2007) LOA framework and concomitant assessment principles and Pellegrino et al.’s (2001) vision of alignment as a “comprehensive, coherent, and continuous” (p. 9) system, seamlessly linking [macro-level] policy, curriculum, and large-scale external tests with [micro-level] classroom-based assessment through a collectively shared model of student learning. However, as several contributors note, a shared model of learning (which is required to maintain such alignment) has proved elusive. Within language assessment research, two other LOA frameworks are prominently featured: Jones and Saville’s (2016) systemic LOA Cycle, which extended Pellegrino et al.’s vision of alignment (see Saville, Chapter 2), and Turner and Purpura’s (2016) Working framework for LOA, which identified seven “interrelated dimensions” that, taken together, account for LOA’s “complex” and “multifaceted” nature (p. 262). Research reported in the volume is recurrently informed by these frameworks.
Overview
Gebril (Chapter 1) introduces the volume by highlighting the ongoing “tension between [the] accountability and improvement functions of assessment” (p. 1). He observes that test-intensive, examination-driven policies have resulted in “teaching to the test . . . at the expense of real learning, mounting pressure on teachers, and narrowing down of the curriculum” (p. 1). At the heart of this tension are fundamentally divergent views relating to LOA, which are illuminated in Part 1 of the book. In Part 2, examples of LOA research in practice are provided. Overall, the collection addresses key, recurring questions about LOA from a range of differing perspectives. Some of these questions and the contributors’ responses are briefly summarized below.
What role should large-scale, external tests and generic standards play in LOA?
Saville (Chapter 2) argues for systemic linkages among what he refers to as components of the ecosystem of learning. Drawing on Pellegrino et al.’s (2001) conceptualization of alignment and extending the Jones and Saville (2016) framework, he argues that “large-scale and school-based assessments need to be brought into alignment with an orientation towards learning” (p. 16). Such alignment would view all assessment evidence as “complementary,” and by systematic “elicitation and appropriate use of the evidence for specified teaching and learning purposes,” the aim of “measurably better language learning” (p. 30) would be achieved. Saville acknowledges that such alignment “has hitherto proved challenging” (p. 14) and takes an equally understated view of large-scale testing’s impact on teaching and learning, noting that “Some commentators argue that the balance has tilted too far away from learners and their learning goals and towards policy-related processes and the summative use of results for accountability purposes” (p. 16; emphasis added). A more critical view of this imbalance is found in Fulcher (Chapter 3), who warns against the “attempts by the examination industry to annex LOA as a tool for aligning language teaching to policy standards documents such as the CEFR” (p. 35) and obscuring the profound theoretical and functional differences between high-stakes testing and classroom-based LOA. He argues that powerful, at-a-distance efforts “seek to blur . . . [such] distinctions and recreate local classroom assessment and teaching in the image of prevailing institutional models” (p. 35).
Evidence supporting Fulcher’s warning is found in Figueras (Chapter 5), who relates LOA’s increasing importance to the interest that “international testing organizations have shown . . . in how their standardized tests, albeit externally produced, may influence learning” (p. 69). Figueras focuses on systematically closing gaps between clearly articulated performance descriptors or standards (e.g., CEFR) and classroom-based assessment practices. Like Saville (Chapter 2), she argues that Pellegrino et al.’s (2001) notion of instructionally relevant assessments will advance LOA through the systematic articulation of “a unified approach to learning, teaching, and assessment” (p. 76) and highlights the need to streamline LOA best practices, so that teachers will more readily recognize their merits and more effectively implement them in their teaching practice.
The claim that increasing interest in LOA has been, in part, driven by the large-scale testing industry is substantiated to a degree by contributions to the present volume. Much of the research reported in the book was funded or conducted by researchers affiliated with Cambridge Assessment English (CAE)—a major player in the examination and testing industry. On the one hand, this gives credence to Fulcher’s warning that industrial interests (e.g., in external tests, performance standards) are active in their attempts to appropriate LOA. On the other hand, Salamoura and Morgan (Chapter 11), who are senior CAE researchers, view this involvement as evidence of CAE’s “commitment to supporting teacher development” (p. 187).
Should the validity requirements of external, large-scale testing be applied to LOA?
In Part 1, some contributors suggest that traditional approaches to validity should be applied to considerations of LOA, while others argue forcefully against this. For example, Fulcher explains that LOA assessment violates large-scale testing’s requirement for maximal control of variability (e.g., across contexts, test administrations, test versions, raters) in order to support validity arguments for test use and score interpretability. Furthermore, the purpose of such testing is to report on achievement, proficiency, performance, and so on at a given moment in time, and thus allow for ranking and quality comparisons. Conversely, evidence drawn from classroom-based, learning-oriented assessment is by its nature variable: mediated by teachers; context-specific; embedded within diverse social, cultural, and historical contexts; and fluid. Its purpose is to account for “growth and development” over time. Fulcher argues that for LOA, “change is the most fundamental validity concern” (p. 35).
Which research approaches are best suited to study LOA, given its focus on change, growth, and development?
Banerjee (Chapter 4) reports a general trend in the use of qualitative methods in LOA research (evident throughout the present volume). Based on a selective survey of second language (L2) LOA research, she explains that qualitative methods are best suited to researching practices and processes, although quantification of results is helpful in explaining findings. For example, Amer (Chapter 8) and Kahn and Hassan (Chapter 9) apply qualitative case study approaches to investigate the impact of LOA educational reform in the centralized, “overarching examination-oriented culture[s]” (p. 142) of Egypt and Malaysia, respectively (see also Gebril, Chapter 7, for an overview of the Egyptian educational reform). Salamoura and Morgan (Chapter 11) provide a refreshing review of learning-oriented, teacher-initiated Action Research (AR) projects. AR recognizes that “teachers are uniquely placed to notice, investigate, and address issues and challenges in the classroom” (p. 187). Ironically, AR has been actively impeded in many jurisdictions. However, evidence is clear that when teachers are trusted to undertake classroom-based research on learning, overall educational quality improves. Finally, Voss (Chapter 12) offers a comprehensive view of the role of technology in classroom-based research and teaching, highlighting its potential to “enable new forms of learning interaction and [capture] new forms of evidence for learning” (p. 207).
How can teachers raise their students’ effective use of feedback, promote students’ responsibility for their learning, and actively engage them in peer- and self-assessment?
The provision and interactive use of feedback by teachers and students is a foundational principle of LOA, with deep roots in the educational research literature. In their respective chapters, Lam (Chapter 6) and Amer (Chapter 8) highlight research on feedback and the challenges of developing feedback literacy (Carless & Boud, 2018). The key question for feedback is how a student uses it, that is, acts as an outcome of it. As Lam points out, it is of critical importance “for learners to ultimately identify themselves as agents for their own improvement and develop a proactive attitude toward it” (pp. 102–103).
In this regard, Baker, Polikar, and Homayounzadeh’s (Chapter 10) contribution is noteworthy and of particular importance to the volume as a whole. First, they report on the “live” responses of learners to an LOA task assigned as part of ongoing language class activity. While it is laudable that teachers are well represented in other chapters in the volume, given that learners are the key stakeholders in LOA, learner voices are significantly missing elsewhere (which may signal a critical gap in current LOA research). Second, they assume a teacher-researcher stance, drawing on their design and use of LOA tasks within their own L2 classrooms, whereas the authors in the other chapters assume a researcher-stance (i.e., they are positioned external to and maintain a distance from the language learning settings they consider). Third, Baker et al. demonstrate how a rubric developed for a high-stakes writing test can be unpacked by students themselves through a systematically designed peer- and self-assessment classroom assignment (cf. Lam, Chapter 6), thus providing research evidence of teacher-orchestrated learners’ active engagement.
There is much more to consider in this rich and stimulating collection of perspectives on LOA. It will be of interest to all those who are concerned about language teaching and assessment in support of learning.
