Abstract
Assessment is acknowledged as a central motivator for learning, as being perhaps the most difficult and arduous task for tutors, and also, a defining component of institutional quality, curriculum, courses and degrees. Therefore, given this, surely our understanding of terms, processes and their relationships, which reveal our knowledge of theories, practices and research, would be expected to be coherent and critically defensible. Yet, this study supports other literature that demonstrates that this is not the case. What to do about resolving theoretical and practical issues in assessment is perhaps a key challenge for education and educationalists. One problem is that it is often perceived as being the realm of specialists and for specialist journals, when the reality is that understanding assessment is central to everyone in education.
Views on assessment
Given the ubiquitous nature of assessment in all aspects of education, research that can inform us about our processes, products and protocols is of great value (Broadfoot, 2008; Stobart, 2008). Educational frameworks are dominated by our perceptions and prejudices, in particular in relation to assessment as can be witnessed by both tutor and student discourses (Hargreaves, 2005; Maclellan, 2001). These beliefs extend to institutions, government agencies and national education priorities (Broadfoot and Black, 2004; Yorke, in press).
Our views about assessment, learning and teaching have changed radically in the past 50 years (Broadfoot, 2007; Stobart, 2008). During this time, our basic philosophies and understandings of these three processes have continually developed: learning moved from Pavlov and Skinner (Woollard, 2010) to Piaget and Vygotsky (cited in Smith et al., 1997) with Dweck’s (1999) work highlighting the combined importance of physical and cognitive practice and therefore to some extent removing the dichotomy of choice.
Learning and teaching has benefitted from learner and learning-centred discourses and practices and much data and discussion has fuelled these changes, in part consolidated by Dearing (1997). One of the most challenging and difficult developments has been to make assessment part of the learner and learning-centred discourses and practices. Making assessment inclusive of students is still the most important hurdle that needs to be faced in higher education (HE), despite an increasing body of literature devoted to this (e.g. Berry and Adamson, 2011; Cowan, 2006; Havnes and McDowell, 2008). Furthermore, many of the issues in unravelling our understandings of formative assessment (FA), summative assessment (SA) and peer- and self-assessments originate from including learners in assessment (Black and Wiliam, 2009; Sadler, 2010).
The distinctions between SA and FA have been perceived and used to separate the functions of assessment that focus on accreditation and validation on one hand, and the support of teaching and learning on the other (Black and Wiliam, 2009; Sadler, 1989). However, owing to both differences in developments within and across sectors and to a separation of focus between processes of assessment and functions of assessment, these distinctions, including within and between different discourses, have become unaligned and misunderstood (Taras, 2009).
Functions and processes of assessment
Very little homogeneity exists across theoretical discourses of assessment in educational sectors except in the use of Sadler (1989) and Scriven (1967), which are perhaps the most widely-cited works. As concerns Sadler, his theory of FA is perhaps the most coherent and cogent. The theory delineates how we can use feedback, criteria and standards to integrate learners and tutors into a dialogic process that sustains learning.
Scriven (1967) originally made the distinction between SA and FA. Much of the discourse in his seminal article focuses on the processes of assessment, which he states should be kept distinct and separate from the functions. This article has been the object of much controversy because it has been interpreted so diversely. The issue hinges on whether the processes or the functions of assessment are the defining features of SA and FA. Taras (2009, 2010) argues for definitions to be based on process, citing Scriven (1967) as supporting evidence, whereas Black and Wiliam (2009) and Wiliam (2007, 2009) follow Bloom et al. (1971) by basing the definitions on functions, also citing Scriven (1967).
The controversy has split the academic community and has confused the utilities and developments of both SA and FA. Understanding how they relate to each other and how they relate to process and functions is essential for having a coherent theoretical framework that can support and help to improve practice. Having fragmented discourses that are mutually contradictory and exclusive within the educational community will impact negatively on practitioners by resulting in fragmented understandings (Hargreaves, 2005; Maclellan, 2001). Very little discourse focuses on the explicit relationship between summative and formative processes of assessment apart from Scriven (1967) and Taras (2009), making it difficult for the academic community to engage with and think about this critical issue.
One of the tensions that has been at the forefront of assessment discourse issues is that between assessment for validation and accreditation (sometimes known as ‘assessment of learning’) and assessment to support learning and understanding (sometimes known as ‘assessment for learning’; Gardner, 2006; Harlen, 2006; Stobart, 2008). This seemingly clear divide becomes complex and confusing when additional terminologies of SA and FA are introduced, especially when these four terms appear to change over time and also to be different within and across sectors. Although the arguments and the discourses appear similar in different contexts, the meanings and roles of the terms seem increasingly disparate (Black and Wiliam, 2009; Taras, 2008a, 2009; Wiliam, 2009).
In the compulsory sector, there are added tensions and complications among functions of assessment because of the dual assessment processes caused by external accreditation of examinations and qualifications (Broadfoot, 2007, 2008; Stobart, 2008). Many of these discourses are not of direct relevance in HE, and yet because of the increasing cross-sector use of research literature, there is importing of concepts that do not sit easily outside the original context. It can be argued that this problem is particularly salient in education departments since they are at the cross-roads of such differences; however, it can also be argued that with internationalisation of discourse, this is perhaps a general problem. How these discourses and research examples influence and impact at grass-roots level in the classroom will be reflected to some degree in the understandings of lecturers, whatever their subject.
The tensions between ‘assessment of learning’ and ‘assessment for learning’, often differentiated by summative and formative functions of assessment (Hargreaves, 2005; Harlen, 2006), are due, in no small part, to social injustices linked to the uses made of assessments and the fall-out on personal representations and beliefs that become attached to the individual as opposed to the assessment (Broadfoot, 2008; Stobart, 2008). The negative impact of assessment on future prospects and the links to perceived personal worth has led many academics to condemn the negative influence of summative functions of assessments (Broadfoot, 2007, 2008; Stobart, 2008). However, it is not the SA function per se that is problematic. Functions are many and can be combined into multiple uses, and basing assessments on functions is not efficient or productive (Wiliam, 2009). It is more important to ensure that the processes of assessment are ethical, transparent and communicated (Scriven, 1967; Taras, 2009). This article explores how the controversies and ambiguities described above have impacted on lecturers’ understandings.
Just how much can, should or must learners be involved, implicated in or part of assessment decisions, procedures and discussions is still the million dollar question in assessment. Attempts to answer this are implicit and explicit in the discourses around FA and in the separation of SA and FA.
Despite the extensive research on assessment over the last two decades, the balance of work across theory, practice and empirical work is heavily skewed towards the last two aspects. The limited range of work on theory, and in particular, linking it to practice (Hargreaves, 2005; Maclellan, 2001) has, in part, stimulated this work.
The central focus here is on university lecturers’ understanding of the functions, processes and products of assessment, how feedback and student involvement are understood and how all these relate to understandings of SA and FA. As such, this research sets out to give clarity to the gaps in understandings and perceptions, clarification of which goes some way to allowing us to remediate and eliminate the gaps.
Research method
A questionnaire of 26 questions was distributed to 50 lecturers in a variety of health and life science-related academic teams in a science faculty at an English university in 2010 (Appendix 1). It was not possible to distribute the questionnaire to all participants at the same time and so it was issued on an opportunistic basis over a period of 8 months on occasions when each academic team met to discuss business. All academics within these teams were sampled, except those absent on the day of the meetings. Lecturers were told that the questionnaire was to be completed anonymously and were asked not to confer but told that they could ask for clarification of any question. Six lecturers were observed to scan the entire questionnaire before completing it. One question asked, ‘Is theory important to us as teachers?’, and two participants wondered whether the word ‘theory’ referred to educational theory or within-discipline theory. The former was indicated. Questionnaires were collected immediately on completion; no participant took more than 15 minutes to complete the questionnaire. This questionnaire had been validated during an earlier study (Taras, 2008b).
It was not possible to compare results between academic teams because the sample sizes were too small; in any case, academic teams were not strictly differentiated by discipline and there was long-term movement of academic staff between teams.
Results
The results are considered question-by-question in the order in which they were asked (Appendix 1). They are grouped according to the aspects of understandings that were explored, beginning with student self-assessments, as indicated in the following sections. The fractions quoted are based on the number of responses to each question. Since the number of respondents was not always the same, the denominator may vary.
Information on student self-assessment (questions 1–4)
In total, 56% (28/50) of lecturers use student self-assessment with their students, 46% (23/48) believe that self-assessment is related to FA; 20% (10/48) state that they present self-assessment as SA, whereas 72% (36/48) state that they do not. However, 34% (17/43) believe that self-assessment is both SA and FA (Table 1).
Information on SSA
SSA: student self-assessment; SA: summative assessment; FA: formative assessment.
One responded ‘not applicable’, which was classified as a nil response.
Is theory important to us as teachers (question 5)?
All 50 respondents answered this question and 90% agreed that theory is important but interestingly 10% did not.
SA and FA can be used for end- or mid-course grades (questions 6–9)
Questions 6–9 (Appendix 1) focused on the functions of assessment – 82% (41/49) agreed that SA can be used for end-course grades but interestingly 16% (8/49) did not, 70% (35/49) stated that SA can be used for mid-course grades but again a considerable proportion (26%, 13/49) did not, 40% (20/49) agreed that FA can be used for end-course grades, and 54% (27/49) stated that FA can be used for mid-course grades. But again considerable proportions thought that FA could not be used for mid- and end-course grades (44% (22/49) and 54% (27/49), respectively) (Table 2).
SA and FA can be used for end- or mid-course grades
SA: summative assessment; FA: formative assessment.
SA and FA assess product and/or process (questions 10–13)
In all, 76% (38/48) of lecturers saw SA as assessing a product, while 62% (31/47) saw SA as assessing a process. For FA, there was a slightly stronger but opposite response: 80% (40/49) see it as assessing process and 66% (33/48) as assessing product. Most (52%) indicated that SA assesses both product and process and for FA this value was 56%. A little under half (42%) thought that both SA and FA assess both product and process (Table 3).
SA and FA assess product and/or process
SA: summative assessment; FA: formative assessment.
SA and FA assess for validation (grading) or for learning (questions 14–17)
Questions 14–17 also focused on the functions of assessment − 62% (31/48) indicated that SA can be used for grading, 52% (26/47) stated that FA can be used for grading, and 40% (20/47) did not think that FA is for grading. FA is predominant as the assessment mode for learning (90% (45/49) agreed), although a learning function for SA was also indicated by a considerable majority (78%, 39/48) (Table 4).
SA and FA assess for validation (grading) or for learning
SA: summative assessment; FA: formative assessment.
One responded ‘not applicable’, which was classified as a nil response.
SA and FA provide useful feedback (questions 18 and 19)
Most respondents thought that both SA and FA provided useful feedback, but a greater proportion (88%, 44/48) regarded FA as having this role than SA (78%, 39/49). Interestingly, 8% (4/48) did not associate FA with feedback to students (Table 5).
SA and FA provide useful feedback
SA: summative assessment; FA: formative assessment.
SA and FA are different or similar processes (questions 20 and 21)
In total, 50% (25/49) recognised SA and FA as different processes, while 64% (32/49) regarded them as similar. Clearly some, in addition to those respondents who indicated ‘both’ in response to these questions (4% (2/49) each), indicated both that the processes are different and similar (Table 6).
SA and FA are different or similar processes
SA: summative assessment; FA: formative assessment.
Sure/unsure how SA and FA relate (question 22)
In total, 46% (23/34) stated that they were sure how SA and FA relate to each other, and 22% (11/34) were not sure. This question was the one most often left blank; 32% failed to respond, which perhaps indicates ‘not sure’.
Students understand/focus on SA and FA (questions 23–26)
In all, 72% (36/50) thought students understood SA, while only 50% (25/50) thought students understood FA; 76% (38/50) thought students focused on SA, though only 28% (14/49) thought students focused on FA and 18% of lecturers thought that students focused on both SA and FA (Table 7).
Students understand/focus on SA and FA
SA: summative assessment; FA: formative assessment.
Discussion
Although an overwhelming majority of our sample of science lecturers agreed that theory was important, the answers to many of the other questions indicate that the importance is not consistently translated into an understanding of theory or indeed into practical use. While recognising the importance of theory, they might regard it as unimportant to their activities as lecturers and that it is something that does not concern them, perhaps the preserve of education subject specialists.
About half of the science lecturers use self-assessment and believe it is related to FA, though fewer present self-assessment as SA and believe it can be both SA and FA. The association of self-assessment with SA or FA is a debate in progress (Taras, 2010). Generally, from the theoretical frameworks of Sadler (1989, 1998, 2010) and Scriven (1967), self-assessment can be considered as the student equivalent of SA, requiring the explicit step of using feedback for it to become FA. The literature that defines SA and FA according to functions (Black et al., 2003; Black and Wiliam, 2009; Wiliam, 2007, 2009) systematically presents self-assessment, although more recently, FA also requires the explicit step of using feedback for it to become FA (Black and Wiliam, 2009; Wiliam, 2007, 2009).
A substantial minority of our science lecturers did not agree that SA can be used in grading, a stance that seems counter-intuitive, since a summation at any point is possible (Scriven, 1967), and grading is a logical extension or integral part of making a summative judgement. About half thought that FA could be used for grading, roughly consistent with their reporting that they grade formative work. Thus there seems a discontinuity in the understanding of the relationship between FA and grading, which leads to confusion, and it is tempting to suggest that the results recorded may reflect a general lack of understanding of the terms used. Likewise, there was a lack of consistency on whether assessments assess product or process even though any assessment (SA, FA, peer- or self-assessment) can be either of product or process or both.
The science lecturers see both SA and FA as vehicles for learning and as important for grading. One interpretation (advanced by Taras, 2008b) is that those who did not regard FA as assessing for grading (40%) indicated by implication that during this process feedback to students on their performance does not occur. But this implication may or may not have been realised by respondents. It is becoming evident that these science lecturers do not have a clear notion of the relationship between, or functions of, FA and SA, terms that they may have to deal with in, for example, institutional assessment strategies and materials related to ensuring academic standards and quality.
Over a quarter of lecturers thought that students did not understand SA, and this rose to almost half for FA. Staff think students are undergoing processes of assessment that they do not understand. One might argue that staff could easily educate students so that they do understand assessment, but this view is compromised if staff themselves do not understand it, which emerges as an overwhelming theme in this study. Most HE students have graduated from a culture where grades have determined their current social and perhaps financial position, and thus they focus on those assessments that have associated grading (e.g. Black et al., 2003).
In any research involving questionnaires, one has to take the results at face value; in other words, we assume that honest answers are provided. However, the results show so many contradictions that it is natural to question the balance point between genuine confusion and that brought about by participants wanting to give the ‘correct’ answer, even though they knew the questionnaire was anonymous. Furthermore, some of the variation in answers might occur because questionnaires were not all completed at the same time and place. This then raises the general point that not all research based on interrogation may yield data that are robust enough to warrant analysis and from which to draw sound conclusions. This is a small-scale study and therefore generalisations are naturally limited, which invites further research. A further limitation is that this study did not examine differences relating to gender or experience, for example, as measured by length of service.
Given the confusion shown by lecturers in the discipline of education (Taras, 2008b), it is perhaps not surprising that these science lecturers also had problems coordinating coherent understandings of functions and processes of assessment. Most institutions have at their disposal internal staff development programmes, although their uptake is optional. Since these programmes are often delivered by lecturers in the discipline of education, perhaps the confusion of non-education subject specialists is understandable. In addition, differential understandings of theoretical frameworks of assessment further muddy the waters. This is a pity especially since so much time, effort and money is spent by the HE sector on assessment. Over the last few decades, there has been much activity in developing and disseminating new forms of assessment, and yet it seems that there is confusion at a fundamental level.
Using terms that are often used in a HE setting, we set out to understand whether lecturers’ understandings of assessment were clear, cogent, coherent and shared. Our discovery of largely negative answers to these questions leads us to question whether staff, particularly in this discipline, think about assessment and its consequences in sufficient depth. Perhaps they find debate on assessment difficult and maybe irrelevant, in part because of the confusion of terms and lack of an accepted wisdom. This may lead lecturers to concentrate on their own understanding of assessment, which may not be explicitly linked to theory. Focusing on practicalities of assessment processes may well be equated with a perception of a job well done. But is this sufficient? Is it possible that HE lecturers are making varied decisions in assessing students because of their differences in understanding about assessment? Since the impact of the consequences of assessment, particularly grading, are so important to all involved, these are of great importance to the academic community and represent an area for future research.
There is a real danger in compartmentalisation of the various facets that together make up the broad topic that is assessment: only when the various strands are integrated can we hope for a comprehensive understanding of assessment as a whole. Taking such a holistic view of assessment will help to clarify how we can really improve learning and teaching, rather than having disjointed and isolated practices. This research represents a contribution to that goal.
We are not in a position to offer remedial advice but question how staff can be expected to understand even the simple tenets of assessment theories when there seems to be a general confusion of terms. There seems to be a lack of impetus to produce coherent and workable theories that relate to and support practice. Furthermore, assessment is perceived as being the realm of specialists and specialist journals, when the reality is that understanding assessment is central to everyone in education.
If the pattern we discovered is repeated in other disciplines and institutions, then such accumulation of evidence will warrant much further investigation. Discovering the generality of pattern is an obvious priority for further work. In addition, other approaches include examining more basic understandings and definitions of terms such as ‘assessment’, ‘theory’ and ‘learning’, and examining the perspectives offered by students. The work described and explored in this article is, however, a good starting point for highlighting the importance of discussing fundamental aspects of assessment and ensuring a shared understanding.
Footnotes
Appendix 1
Funding
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.