Abstract
This book, which is part of the New Perspectives on Language and Education series, is divided into four main parts, with three chapters in each, plus an introduction and overview chapter. Part 1 focuses on Intercultural Competence, Part 2 on Autonomy, Part 3 on Literature, and Part 4 on Language and Content.
The book starts with Paran advocating going beyond language, to ‘more than language … through a focus on additional aspects of language testing, either the content of what is being tested, or additional skills that are related to language learning’ (p. 5), and criticizing the ‘narrowing vision of language tests’, which ‘circumscribe the content that is taught in the language classroom’ (p. 3). Paran is also concerned about what he believes is ‘the continuing tension between assessment of learning and assessment for learning’ (p. 8), and it is these tensions that lie at the heart of the book and connect many of the parts and the chapters together.
As reflected in the title of Chapter 2, co-editor Sercu acknowledges that in relation to assessing intercultural competence, there are ‘more questions than answers’ (p. 17). Indeed, later he notes that ‘as yet, such a holistic measure [of intercultural competence] is not available’ (p. 25). This is one of the recurring themes of the book: while most of the contributors argue the importance of testing such language-related skills and competencies, they also point out the difficulties associated with developing, administering and marking such tests, and few of them are able to give details of any tests. Hence, despite the title, the book is more of a ‘call to arms’ than a blueprint for how these skills might be tested.
In Chapter 3, Korhonen reports on the results of an intercultural training course with a group of Finnish Bachelor of Engineering students taking compulsory English studies. Korhonen used a series of pre- and post-questionnaires to assess the change in the students’ intercultural sensitivity as a result of the course, concluding that ‘[t]he intercultural training process … is slow, gradual and difficult to assess’ (p. 49).
In the final chapter in Part 1, Liddicoat and Scarino present two case studies focusing on the elicitation of intercultural behaviours in language teaching and learning situations. They conclude by recommending the use of portfolios for assessment as these provide ‘multiple exemplars of various aspects of the intercultural’ which ‘capture diverse instantiations of the intercultural over time’ (p. 71).
Part 2 of the book, on Autonomy, starts with a chapter by Benson, who proposes ‘three poles of attraction in regard to control over learning: student control, other control or no control’ (pp. 79–80). In an interesting extension of some of his earlier work on learner autonomy (see, e.g., Benson, 2001 and 2007), Benson claims that the three poles constitute ‘a possible framework for the measurement of autonomy based on observable behaviours’ (p. 82). However, it is not clear how such a model would translate into a measurement tool, and Benson also expresses serious reservations about the whole idea of measuring autonomy, as a result of difficulties that he describes as ‘intractable’ (p. 95).
In Chapter 6, Lamb argues for formative assessment of autonomy, and compares and contrasts Assessment for Learning with Assessment for Autonomy (pp. 99–103). The main focus of the chapter is on the relationships between autonomy, motivation and metacognitive knowledge, based on which Lamb concludes that ‘the framework of metacognitive knowledge and beliefs about learning and the tool of the FGC [Focused Group Conversations] offer a way forward in facilitating assessment for autonomy’ (p. 109).
In relation to the title of the book, Dam and Legenhausen start their chapter by stating ‘It is clearly a paradox to claim that one can test the untestable’ (p. 120). The basis of their argument lies in the distinction between testing and evaluation: they note that, when it comes to assessing learner autonomy ‘the obvious and only answer for teachers to get a real insight into the learning process is constant and recurring evaluation where the learners’ voices are heard and taken seriously’ (p. 137). They also add that ‘The history of testing in language pedagogy has shown that testing objectives that seem most worthwhile turn out to be recalcitrant to testing’ (p. 121). These kinds of statements may surprise or even frustrate some language testers, and although such claims are not as well-supported as the certainty of their position appears to indicate, this book should nevertheless be of interest to language testers, if only so they might respond to, or even dispute, such claims.
One of the chapters that does attempt to give some concrete details of how these ‘untestable’ language skills and competencies might be assessed, if not tested, is Chapter 8, by Paran, which is the first chapter of Part 3, on Literature. Reminiscent of Bailey’s Learning About Language Assessment: Dilemmas, Decisions and Directions (1998), Paran first presents six dilemmas for testing literature in foreign language testing, including, inter alia, Testing language or testing literature? and Testing knowledge or testing skills?
Paran’s suggestions for the testing and assessment of literature in the language classroom include: using a variety of tasks, giving the test takers choices, providing texts, using portfolio assessment, and making the criteria transparent. These are all useful suggestions, although many language testers would claim that they have already been incorporated into many language tests. Paran also suggests minimizing the weighting of language, although he does accept that might look an odd principle in a discussion of language teaching.
In Chapter 9, Spiro presents a concise and thoughtful consideration of creativity and language learning, in relation to a Language through Literature course co-taught with a colleague, and using data from approximately 100 students in two cohorts. Spiro discusses the value of rubrics in establishing transparency, and the importance of ‘clarity of criteria, congruence and appropriacy of the feedback’ (p. 176). Also, as Spiro explains in her introduction, the chapter ‘explores the process of arriving at an assessment of creativity’ (p. 165), so the focus of this chapter, like that of many chapters in this collection, is not so much on the testing of specific language skills, but more on the broader assessment of language-related competencies.
In the last chapter in Part 3, Lin describes how he developed ‘a language awareness test’ (p. 192), which he clarifies by explaining ‘As this test was aimed at assessing language awareness, not testing grammatical knowledge or language proficiency, students were free to respond to the test items in either Chinese or English as they preferred’ (p. 197). Lin draws on Bachman (1990) and Bachman and Palmer (1996) in the creation of his test, and makes the only attempt in the book to describe a test in some detail. However, he also notes that ‘a number of test methods have gone unexplored in this project’ and there is a ‘need to include more distinct elicitation procedures (e.g. fill-in-the-blanks, matching, sentence completion, multiple-choice, short essay) that can tap … the learner’s capacity to handle responses … to literary texts’ (p. 208).
In the first chapter of Part 4 of the book, Language and Content, Mohan, Leung and Slater discuss what they refer to as ‘the integrated assessment of language and content (IALC), with particular reference to second language learning and use’ (p. 217). In relation to ‘testing the untestable’, the authors state clearly and concisely another one of the recurring themes in the book ‘We are talking about assessment in a broad sense, which includes situated classroom assessment processes; we are not talking about “tests” ’ (p. 218). Although Mohan, Leung and Slater’s directness is commendable, this recurring theme does question the validity of the book’s title.
In the next chapter, Low presents an account of ‘classroom assessment practices through teacher talk’ (p. 241). Her study is a discourse analysis of teachers’ discussions, drawing on data gathered from an international college in Canada and from an elementary school in American Samoa, both of which are linguistically interesting and complex language teaching and learning contexts. While the extracts from the teachers’ discourse do shed some light on teacher decision-making processes, and the dilemmas they feel they face, especially when they are evaluating written texts, it is not clear how the discussion contributes to a consideration of ‘testing the untestable in language education’.
The last chapter (13), by Mohan, and Slater, presents arguments in favour of a ‘systematic and sustained formative assessment of causal explanations in oral interactions’ (p. 256). This brief concluding chapter balances Low’s by focusing on the discourse of students, who are trying to describe cause and effect relations orally. The chapter focuses on the students’ abilities or inabilities to use grammatically metaphorical constructions, and presents a ‘general path for development of oral causal meanings’ (p. 268), which are important aspects of first- and second-language development. However, the claim that ‘this model has important implications for judging validity in standardized written tests as well as academic oral proficiency interviews’ (p. 268) would need to be more solidly supported.
To sum up, most language testers would probably not consider this to be a book on language testing, in the traditional sense, or even a book on testing given the acknowledgement by most authors of the impossibility of using formal testing methods. It is, nonetheless, an interesting and potentially useful collection of international writings ‘about assessment in a broad sense’, as Mohan, Leung and Slater put it. While it appears as though classroom-based language assessment may have been confused with large-scale language testing, the book should still be of value to language testers who are interested in the assessment of language-related knowledge, skills and competencies, such as intercultural competence, learner autonomy, understanding literature, and content-based instruction.