A forum on assessment,accountability and the humanities

It’s one of those words that doesn’t travel particularly well. ‘Assessment’ means different things to different people in different places. In the English-speaking world it is another example of the common language that divides us in the scholarship of teaching and learning community. And finding mutual understanding is even more complicated, when discussions around the term are particularly volatile, as they are right now.

In some contexts, ‘assessment’ refers to the way student learning is judged in the classroom: the assignments and exams that make student learning visible, the grading and feedback students receive on their work, and the criteria by which their performance is evaluated. This is part and parcel of pedagogy, and while it can be done well or poorly, and as a joy or a burden, it is seldom seen as a threat to instructors’ professional standing or the disciplinary integrity of their teaching. Indeed, just the reverse. Designing good assignments or examinations, providing helpful feedback, and using appropriate criteria for judgment are widely accepted as key aspects of professional academic work. In most, if not all, higher education settings, instructors have the freedom – and obligation – of performing these tasks on their own, subject only to institutional norms of pedagogical propriety and their own discipline’s traditions of grading/assessment.

However, when an external accountability movement captures ‘assessment’, both the term and the activities are in danger of losing whatever innocence they may once or otherwise have enjoyed. No longer concerned with the work of individual students, but with the ‘learning outcomes’ of larger groups, the coupling of ‘assessment’ to instruction is loosened. When the assessment process used for purposes of accountability is designed by specialists elsewhere, it is not likely to call on instructors’ professional knowledge, judgment, and care. Further, if the measures focus only on a narrow set of generic learning outcomes, assessment won’t capture the kinds of disciplinary learning that instructors care most deeply about.

These opposing portraits of what we might call, rather irreverently, ‘white hat’ and ‘black hat’ assessment are exaggerations, of course. Few academics are as intentional and purposeful about classroom assessment as they could (or should) be; few accountability systems in higher education are as distanced from disciplinary instruction as faculty members fear they might become (sometimes looking at what’s happened in primary and secondary education). Yet the opposition is not entirely imaginary, and, as accountability movements in many countries gain steam, concern among academics has risen sharply. In some quarters a bunker mentality has taken hold. Elsewhere, academics are mounting initiatives to mind (and manage) the gap: domesticating assessment for accountability within the disciplines, and moving assessment for improvement up from the classroom to the ‘course’ (UK) or ‘program’ (North America and Australasia) level.

The arts and humanities have a special stake in the success of such efforts, because they are the fields where traditional cultures of classroom assessment differ most dramatically from the kinds of assessment most often proposed as suitable to meeting accountability aims. Of course, the various arts and humanities fields differ among themselves in this regard. But, for many humanists, accountability’s emphasis on explicit, measurable – often generic – learning outcomes strikes a highly discordant note, missing certain kinds and qualities of knowledge that are hard to define but highly valued, nonetheless (Heiland and Rosenthal, 2011). Yet humanists also realize that they cannot just sit on the sidelines and wait the accountability movement out. With college costs rising in many countries, students (with encouragement from parents) are opting to focus their studies on fields that promise more immediate placement in the workforce than the arts and humanities typically do, leaving these fields particularly vulnerable in the new political economy of academe. Recent budgetary developments in the United Kingdom around the funding of arts and humanities courses/programs speak directly and menacingly to the consequences of such trends (Parker, 2011).

So, how is this predicament playing out? What are humanists doing about it? What could they do? Are there lessons for other disciplines in the arts and sciences? The three essays in this mini-forum provide a window on debates and efforts in three countries with different accountability regimes: the United States, Australia, and the United Kingdom (UK). They are not parallel accounts of policy stimulus and humanist response, but rather three quite different ‘takes’ on what is – for all the differences between them – a shared dilemma: how to bridge the divide between (if you will) ‘white hat’ and ‘black hat’ assessment.

We lead with ‘More than limited learning: The case for focusing on the disciplines’, by Donna Heiland and Laura J. Rosenthal, who look askance at the tendency in US accountability circles to focus ‘learning outcomes assessment’ almost exclusively on so-called generic, or crosscutting, skills – including writing, critical thinking, and problem-solving. US higher education may be particularly tempted in this direction, because virtually all four-year undergraduate programs in traditional colleges and universities include a liberal (sometimes called ‘general’) education component in addition to – and often before – students declaring their disciplinary majors. Yet it is widely recognized that crosscutting skills have particular disciplinary inflections and that generic assessments may underestimate what students have learned in common. Thus, ‘grounding assessment work in the disciplines’, as Heiland and Rosenthal emphasize, is a recommendation that resonates both within the humanities and beyond. Indeed, once embraced, as it has been through a variety of exploratory projects outlined by the authors, it’s possible that even humanists will rally to the invitation to bring ‘to the surface the metacognitive frameworks for learning’ their fields employ.

A particularly interesting initiative to take charge of disciplinary assessment is offered in the second essay in our set, ‘Learning outcomes assessment and History: TEQSA, the After-Standards Project, and the QA/QI challenge in Australia’. The authors – Sean Brawley, Jennifer Clark, Chris Dixon, Lisa Ford, Erik Nielsen, Shawn Ross, and Stuart Upton – describe the on-going work of Australian historians through a nationally funded project, to follow the lead of professional bodies (like those for engineers and/or psychologists) to accredit university history programs, using assessed student learning outcomes as part of the accreditation process. In an audacious move a discipline in the arts and humanities is seeking to secure some control over its own destiny in an emerging regulatory environment where audit and compliance will become drivers of engagement between disciplines, programs, institutions, and regulators.

The third essay, from the UK, speaks to the dilemma of assessment that has been identified by the likes of Trudy W. Banta. Banta (2007) notes that institutions have become fixated with (what we call) ‘black hat’ assessment for accountability at the expense of ‘white hat’ assessment for student learning and improvement, and encourages consideration of the intersections. In ‘Understanding arts and humanities students’ experiences of assessment and feedback’, Joelle Adams and Nicole McNab present results from a national multi-disciplinary study funded by the British Higher Education Academy and led by the redoubtable Graham Gibbs on ‘Transforming the Experience of Students through Assessment’ (TESTA). Using data from an audit of disciplinary assessment practices, a student questionnaire, and focus groups with students, the authors point to strengths and shortcomings in the assessment experience of students in humanities majors and courses/programs. Adams and McNab look to ‘best practice’ literature to suggest ways for humanists to improve their usual assessment routines. In the process, of course, these authors usefully redirect our attention to what really matters. All the time and attention now devoted to assessment for accountability will count for nothing if our everyday assessment practices do not adequately support the actual learning environments experienced by students.

And so in these three multi-authored contributions from three nations we see the identification of the problem of assessment for accountability, a possible way for disciplines to engage with regulatory change on their own terms, and an important reminder about why we are here in the first place.

‘White hat’ assessment must inform and direct ‘black hat’ assessment because it is in ‘white hat’ assessment that we as university teachers have the greatest influence and the greatest opportunity to demonstrate and nurture our disciplinary distinctiveness. If we do not we are simply creating a lot more work for ourselves by having to meet audit/compliance requirements that are not implicitly connected back to our classrooms.