More than limited learning: The case for focusing on the disciplines

Abstract

In their 2011 study Academically Adrift, Richard Arum and Josipa Roksa present considerable evidence that undergraduates in US colleges and universities make surprisingly little progress in their first two years of college. While acknowledging the force of this argument, the authors ask whether Arum and Roksa’s evidence – about students’ development of general skills including critical thinking, analytical reasoning, writing and problem solving – fully accounts for students’ learning in college. On the basis of their experiences as a faculty member who teaches literature and oversees humanities assessment at her university (Rosenthal) and as a former staff member of a foundation that supports learning outcomes assessment at colleges and universities (Heiland), the authors argue that a fuller understanding of what students learn in college would account for the contributions of the disciplines. Disciplinary learning contributes both to the development of general skills like those listed above, and to the development of knowledge and skills particular to given fields. We in the humanities must articulate and assess the contributions made by our own fields, lest they be represented reductively, or, worse, not at all.

Keywords

appreciation assessment disciplinary learning outcomes

In their study Academically Adrift, Richard Arum and Josipa Roksa (2011) present considerable evidence that undergraduates in US colleges and universities make surprisingly little progress in their first two years. Based on results from the Collegiate Learning Assessment (CLA – a test we describe below), Arum and Roksa show that 45 percent of students advance only minimally in critical thinking, complex reasoning, and writing. This study has received a great deal of attention in academic journals and in the popular press (see the Social Science Research Council’s project website http://highered.ssrc.org/?page_id=13 for a full list), most of it preoccupied with the fact that the news is so bleak. And it is, though not entirely. For those of us in the liberal arts, the finding that students who undertake this kind of study make greater gains on the CLA provides some justification for our efforts. Still, Arum and Roksa also offer substantial evidence that there is a lot of room for improvement in all areas, and a follow-up study on how much students learn in their final two years of college shows that this improvement does not take place then either. The new results are more or less consistent with what the authors found regarding the first two years, with 36 percent of students failing to show ‘any significant improvement in learning, as measured by CLA performance’ (Arum et al., n.d.: 4).

One typical response to Arum and Roksa’s gloomy findings has been to challenge the validity of the CLA itself. Designed to assess institutional rather than individual performance, the CLA gets at this elusive indicator by evaluating students’ demonstration of skills that virtually every college and university in the US says it wants its students to acquire: ‘critical thinking, analytical reasoning, problem solving and writing’ (Arum and Roksa, 2011: 21). Students demonstrate these abilities either through ‘analytical writing tasks’ that ask them ‘to make an argument and to break an argument’, or through a ‘performance task’ that asks them ‘to respond to a writing prompt that is associated with a set of background documents’ (Arum and Roksa, 2011: 21). For example, one task asks students ‘to generate a memo advising an employer about the desirability of purchasing a type of airplane that has recently crashed’, and students are given a range of documents – emails, articles about the plane, about the accident and related accidents, and more – on which to base their recommendation (summarized in Arum and Roksa, 2011: 21–22). As Arum and Roksa note, the performance task is the ‘most well-developed and sophisticated part’ of the CLA, and it is on data generated by the performance task that they base their analysis of overall student performance (2011: 21).

Objectors to Arum and Roksa’s study will point to the unreliability of standardized tests, to the possibility that the subjects under scrutiny do not take them seriously as they are not part of regular coursework, and to the hazards of encouraging faculty members to ‘teach to the test’. The CLA, however, seems like a pretty good test to us, as it avoids the traditional multiple-choice format and in the performance tasks asks students to propose ideas for solving problems. These tasks require the student to take different pieces of information into account and weigh the value, reliability, and significance of each. While there is always room for improvement in such instruments, one could do worse than teach to this test (if we take that phrase to mean orienting program outcome goals to the kinds of skills that CLA probes). Further, the results recorded by Arum and Roksa have been reproduced in another major national study of undergraduate learning (Pascarella et al., 2011), suggesting that, whatever their method, the findings were not idiosyncratic. Neither of us is a social scientist, but we are willing to accept, more or less, the accuracy of Arum and Roksa’s findings. We are also concerned, like so many others who have read their book and seen the results of other studies, that students make disturbingly limited progress in the higher-order skills documented in their study. We are grateful for this wake-up call to pay more attention to student learning and less to climbing walls, fancy dorms, and football teams.

While we fully acknowledge the important points made by this study, and just as fully support Arum and Roksa’s call for more research of this kind, along with a greater commitment to improvement of student performance, we would nevertheless like to complicate the way this project yokes low CLA scores to ‘limited learning’. We are mindful of the fact that the CLA is designed to offer a picture of overall institutional performance, and can see the logic of gauging that performance through students’ acquisition of general skills. Institution-level assessments of undergraduate achievement in the US often focus on what Americans call ‘general education’ outcomes (what others now call ‘generic attributes’), by which is meant the kinds of general competencies measured by the CLA (though the list may vary some). But do such general competencies alone add up to ‘learning’? And how does the picture of ‘limited learning on college campuses’ change when we consider the contributions of disciplinary learning to undergraduate education? Pat Hutchings (2011) has identified skills such as writing, critical thinking, and problem solving – the very skills measured by the CLA – as ‘cross-cutting’ outcomes that are cultivated by faculty across a wide range of disciplines, and such skills are arguably cultivated in different ways by different disciplines (Brooks, 2011). We want to argue, then, that even our understanding of general competencies such as writing, critical thinking, and problem solving – the skills the CLA measures – would be more sophisticated if we considered the specific forms they take in different disciplines. We are glad that research on this issue is ongoing (Brooks, 2011).

Beyond acknowledging the power of disciplinary learning in shaping general education outcomes, however, we would also like to consider what we stand to gain from greater attention to disciplinary learning as it develops skills and abilities that are specific to a given field of study. We feel that there would be three major advantages to this turn. First, attention to disciplinary learning will do a better job in engaging faculty members in ‘learning outcomes assessment’ projects (understood here, in an American context, as distinct from the usual practices of grading papers and tests)¹ and the general goal of improving student learning on their campuses. Second, greater attention to learning in the disciplines will offer a more accurate picture of what is really happening in higher education. And third, while we greatly admire Arum and Roksa’s courage in squarely confronting inadequacies in higher education from within, we also feel that a more balanced perspective could offer the public a fuller and more complex picture of learning at the college and university level. We thus want to suggest that one strong response to Arum and Roksa’s data would be to work through the disciplines, trying to achieve a more sophisticated understanding of what they offer to students and whether they might in fact lead to more than the ‘limited learning’ that Arum and Roksa document.

From experience to evidence: Working in the field

While administrators in a wide range of colleges and universities have taken notice of Academically Adrift, we wonder what kind of impact its findings are having on ‘on the ground’. In her academic setting, one of us (Rosenthal) has frequently heard it dismissed, if mentioned at all, as yet another attack on higher education in what often feels like an unrelenting stream, even though the authors present not a polemic but a sophisticated sociological study. Even more careful readers, we fear, will not take it to heart – or if they do, may not have much sense of what to do about it – because the learning outcomes on which it focuses seem so abstracted from the daily life of teaching. Because of this gap between general skills and disciplinary learning, each of us, while fascinated by Arum and Roksa’s powerful research, has for different reasons found the results in Academically Adrift somewhat at odds with her own experiences. We feel that it is important to take this dissonance seriously, and would like to suggest that it arises from differences in where we look for student learning. Where Arum and Roksa focused on students’ demonstration of generic skills in solving ‘real world’ problems, most faculty members have more interest in student achievement in the disciplines. The former may indeed draw on some of what is learned in the latter, but it surely does not capture all of what students learn in their English major or their History minor. We believe that there is much to gain by making more visible the kinds of learning that takes place within the disciplines.

We begin with our own – admittedly limited and at times even anecdotal – account of where student learning seems to thrive. One of us (Heiland) worked at a foundation that supports higher education and has funded a range of learning outcomes assessment projects focused on undergraduate learning (including – full disclosure – Arum and Roksa’s research).² Watching the assessment projects unfold, Heiland noticed that faculty participants often seek evidence of general learning outcomes by way of specific disciplinary learning. Thus a project at Seattle University engaged a range of departments in assessing and developing students’ capacity for producing ‘expert insider prose’ in their disciplines (Bean, 2011),³ while a similar effort at Middlebury College has traced students’ progress in writing as they transitioned from high school into their college years, moved into their majors, and started to produce writing that demonstrated their ‘acquisition of the language and forms of a discipline’, so that ‘Psychology majors sounded like Psychology majors and very different from History majors’, and ‘English majors sounded like English majors’ (Middlebury College, 2011). The importance of learning in the majors was again emphasized when Beloit, Knox, Monmouth and Ripon colleges collaborated on a project to understand ‘how learning in the major supports all-college goals’, including ‘civic engagement, quantitative reasoning and critical thinking’ (Beloit College et al., 2006–2008). And the key difference a discipline can make became the focus of yet another study: a Kalamazoo College assessment of student learning that initially focused on students’ gains in general skills (with the CLA as its main data source) but developed a distinctly disciplinary resonance when faculty researchers disaggregated CLA results by academic division, and learned that student performance on the test did indeed differ along divisional lines, or, as the researchers put it, ‘the trajectories students take through that education seem to affect the degree to which those abilities develop’ (Sotherland et al., 2007: 23).

The value of grounding learning outcomes assessment work in the disciplines is so powerful that the Association of American Colleges and Universities (AAC&U) – which represents roughly 1,200 institutions of higher education in the United States – deliberately ‘tapped into faculty passion and interest’ in their disciplines when it developed (with Teagle support) its ‘Engaging Departments Institute’. Focused on ‘the academic major as the place where students should achieve their highest levels of cognitive sophistication, integrative learning and demonstrable accomplishment in both disciplinary and liberal education outcomes’, the Institute brought together teams of faculty and administrators to develop ‘improved programs for student learning, verified through multiple, informative assessments’ (AAC&U, 2008). All of these projects attest to what scholars in the field have argued – that generic learning outcomes take different forms in different disciplines – and all attest to how interested faculty can become in the project of figuring out whether or not students in their programs achieve such growth.

The other of us (Rosenthal) has been teaching at the university level for more than 20 years, is currently a faculty member at the University of Maryland in the English department, and has had good opportunity to observe student learning as it develops within the English major, and in ways specific to that major. While she has not taught undergraduate courses outside of the English major, she has taught the ‘gateway’ course to the major, sophomore and junior level survey courses, and advanced undergraduate courses. In her experience and perhaps running counter to the findings in Academically Adrift, most students seem to have become more reflective and sophisticated by the time they become seniors: they write better, they understand the basics of argumentation, they are able to move beyond their gut reactions to a text, and they connect the works they are reading with ideas, texts, and forms of historical context that they have learned in other classes. They bring insights to class from other courses. They have roughly put together a timeline of history, so they can think about how Shakespeare influenced the Gothic novel, or how Jane Austen differs from Daniel Defoe. They seem generally able to take more responsibility for independent projects and have become comfortable sharing their ideas with the class. Not all students, of course, have developed these capacities, but overall a difference seems visible. Could this be an illusion? Given this perceived progress in sensitivity, sophistication, and awareness, it is difficult to accept the proposition that only ‘limited learning’ – as Arum and Roksa’s subtitle argues – has taken place. This is not at all to suggest that gaps in students’ knowledge and abilities are not also visible and that the best efforts of a dedicated faculty do not sometimes fail. Nevertheless, most faculty members assume that they can rely on a certain foundation of knowledge and sophistication (albeit not as robust as desired) in preparing to teach an upper-level course.

Rosenthal also oversees disciplinary assessment for the Arts and Humanities College at her institution and over the course of several years has seen many reports from individual programs on the expectations for and evidence of student learning. In the college, various students learn to dance, play the oboe, critique a literary text, analyze a primary historical document, think about ethnic bias in communication, reflect on social media, contemplate the intersections of race and gender, compare Greek to Latin expressions of sexual desire, and speak French. Of course, they learn these abilities at various levels. Nevertheless, unless all of the faculty members involved in the assessment process are fabricating their data, it seems safe to say that there is, actually, quite a bit of learning going on. She has seen similar energetic and productive learning outcomes assessment on other campuses she has visited. We realize that these ‘on-the-ground’ assessments do not constitute academic research; they are what Peter Ewell calls ‘craft-based’ (cited in Suskie, 2009: 13) or ‘action research’ undertaken for the purposes of improving local programs (Suskie, 2009: 13). We realize also that they could be more granular: department-level assessment projects often fail to investigate deeply enough to get at the places with room for improvement. Busy faculty members on a department learning outcomes assessment committee may only spend a few minutes thinking about each sample of student work. Finally, we realize that everyone around the table may have a stake in a positive result. Nevertheless, in spite of their inexact nature, these kinds of assessments commonly bear witness to learning that is more than ‘limited’.

Lessons from the field

Pondering this combination of findings from both a range of funded assessment projects across the US and individual experience embedding assessment work on a single campus, we wonder whether this disconnection between the globally ‘limited learning’ discussed by Arum and Roksa, and the experience of developing cultures of expertise among disciplinary majors, might account for (or create) some faculty resistance to the findings in Academically Adrift. Most faculty instructors see their programs as cumulative, although the knowledge and skills accumulated of course vary across disciplines, and it is probably fair to say that – in spite of the findings in Arum and Roksa’s study – most faculty members have a sense that the majority of students change significantly over the course of an American four-year degree program. While we do on occasion overestimate the progress of our students (Bean, 2011), and while we do also on occasion posit capacities that students do not actually possess (Bean, 2011), most programs are grounded in the experientially based observation that students come to upper-level courses in their majors better prepared than they do to introductory courses. Local, department-level assessment practices have been perhaps the most useful in helping to pinpoint when this is not happening and at least in theory use contrary information to improve the program.

It would be helpful, then, to have better data on what exactly accounts for student success in disciplinary majors, and to understand which aspects of that success are captured by a test like the CLA and which are not. If we can understand more about how learning is working in the disciplines, and how this learning might be aligned with the more general goals of higher education, we might find ourselves with a somewhat less bleak overall picture, and also with a truer sense of what is to be gained by a college education.

There are a few places where this kind of work has begun. One exemplary effort is taking place at Indiana University, where the History Learning Project has been under way for some years now. Built on an earlier undertaking called ‘Decoding the Disciplines’, which in turn builds on the work of Lee Shulman, who in the 1980s focused attention on the importance of the disciplinary context for learning (Middendorf and Pace, 2004: 2) and subsequently developed the influential notion of ‘signature pedagogies’ for the professions (Shulman, 2005), the History Learning Project’s genealogy maps one path by which disciplinary learning has made its way into the scholarship of teaching and learning. Further, the project itself has yielded a number of lessons that are helpful for thinking about how to develop – and use – disciplinary assessment as a basis for improving student learning. At its heart, the project seeks to understand what it takes for students to succeed in undergraduate history courses. The first step of the faculty who designed the project was to identify ‘bottlenecks’ to student learning in the discipline, and interviews with 17 faculty members in Indiana’s history department pinpointed seven: ‘misunderstanding the role of facts’, ‘interpreting primary sources’, ‘maintaining appropriate emotional distance’, ‘understanding the limits of knowledge of historical actors’, ‘identifying with people in another time/place’, ‘constructing and evaluating arguments’, and ‘linking specific details to a broader context’. As the project leaders note, some of these sticking points ‘have parallels to problems encountered by students in other disciplines’ while ‘others are specific to history’ (Diaz et al., 2008; Middendorf et al., 2007; History Learning Project website: http://www.iub.edu/~hlp/index.html). With the bottlenecks identified, the next step was (and is) to develop ways of teaching that help students get past them and assess what students have learned. (A grant from the Spencer and Teagle Foundations has funded exploration of ‘strategies to get past two significant bottlenecks: the ways that undergraduates learn to analyze primary sources … and the creation of original and persuasive written arguments’ – see the ‘History of the HLP’ on the History Learning Project website.)

One thing that the History Learning Project tells us is that the implicit ‘frameworks’ within which a discipline does its work need to be brought to the surface if students are to understand the discipline and be able to succeed fully in it (Middendorf et al., 2007). This insight is consonant with what research on learning tells us about the importance of bringing to the surface the metacognitive frameworks for learning in any given context (National Research Council, 2000). Beyond that, though, the History Learning Project is heartening in its attention to the very specific nature of learning in history. It is also important to note that a project like this might be imaginable for many faculty members in many disciplines. While we speculate that few faculty would be able to design and implement projects to assess – with the ultimate aim of learning how to improve – generic ‘critical thinking’ skills, we know with some certainty that they can do so if they work within their disciplinary frameworks. The previously mentioned project conducted by faculty at Beloit, Knox, Monmouth, and Ripon Colleges, was designed to do just that. A larger and still ongoing research project – funded by the Teagle Foundation and led by Rachelle Brooks – is studying the development of ‘general learning outcomes’, including critical thinking, in the disciplines of Classics and Political Science, with the aim of understanding precisely what difference disciplinary training makes in the development of those skills. Importantly, the assessment instruments for this project were developed through intensive work with faculty in both disciplines, and are being administered to students at multiple colleges in their first or second years of study, and then again in their fourth year (Brooks, 2011). Testing will be completed soon and analysis of the data will follow (Brooks, 2012, personal communication). This study should tell us quite a bit about how students develop their higher cognitive abilities in specific disciplines, as well as about the degree to which those skills transfer to other contexts.

The work ahead

That said, we are certain that we would all benefit from still further research about the extent to which a high level of mastery in a discipline leads to the general capacities measured by CLA, and – especially – about the degree to which disciplinary learning is not captured by measures such as the CLA. More specific studies of how to improve learning in the majors are likely to have more appeal to faculty, may ring closer to the truth for faculty, may demonstrate more fully the benefits of higher education as well as disciplinary training, and may even push back a little at federally based tendencies to restrict academic goals to those most obviously useful in the workplace. We take heart from the fact that Arum and Roksa themselves invite further research on disciplinary learning, noting

that faculty engage in distinct practices across fields. Faculty members in different disciplines value specific domains of knowledge and forms of interaction. Consequently, they structure courses, interact with students, and emphasize and reward distinct interests, abilities, and competencies. They also differentially encourage specific educational practices such as faculty-student contact or engagement in active learning, and they are more or less likely to communicate high expectations to students. In other words, faculty in different fields create distinct socializing environments which foster development of specific skills, attitudes, and values. (Arum and Roksa, 2011: 107–108)

We realize, however, that the assessment of disciplinary learning for the purposes of complicating conclusions about limited learning in higher education is more easily advocated than executed. The History Learning Project is particularly interesting along these lines because of the way it takes a systematic and analytical approach to learning in a particular discipline. Nevertheless, this project – with its focus on local learning outcomes assessment work done on the ground, class by class and department by department – cannot compete for impact with a broad study like Academically Adrift. In order to balance that study in ways that we think would offer a richer picture, we need to continue to cultivate the local, but also figure out ways to go beyond it as well. This would require coming to a more fully articulated sense of what students learn in various majors. Certain professionally oriented majors have already done this through their disciplinary associations. On the one hand, these could thus provide models for the liberal arts. On the other hand, we would need to use them critically and carefully, since (as mentioned above) students in those programs tend to make more modest gains on the CLA than students in the liberal arts, which so far have less standardization. We would not argue for that lack of standardization as the reason why liberal arts majors score higher, and in fact we agree in general with Gerald Graff and Cathy Birkenstein (2011) that our own field (English literature) and probably many others are more standardized than most people in them would care to admit. Still, we think articulating learning outcomes for humanities fields in particular holds special challenges and risks that we would not want to diminish.

The knowledge and capacities that students gain in a humanities major and that exceed the limits of what is captured by the CLA (or any measure of general learning outcomes) are notoriously difficult to define. Nevertheless, we think that, until we make an attempt to do so, learning outcomes assessment will not gain significant traction in those fields. Indeed, that belief was one motivation for our development of the essay collection Literary Study, Measurement, and the Sublime (Heiland, Rosenthal and Ching, 2011), which was in part an attempt to think through how we might use the tools of literary study to identify some of those especially elusive outcomes, and we discussed the point elsewhere (Heiland and Rosenthal, 2011). Taking that effort one step further, and speaking from admittedly unsystematic personal experience in teaching literature, Rosenthal observes that students in her upper-level classes come to the material with a more open mind. Or, to put it more crudely, they complain less. They are less likely to blurt out that the material is ‘boring’ and more likely to give it a chance, even if it seems distant from and alien to their life concerns. They continue to have trouble with many aspects of coursework, but in general seem to be more sophisticated in the way they think about literature. Now, it is possible that they have not made significant gains but instead have learned how to appear engaged, since their professors seem to like this better and might as a result give them a better grade. Possibly, they have figured out that disparaging the material does not win them favor. While that is most likely the case for a few (and wouldn’t good behavior be a positive learning outcome as well?), surely some, if not most of them, display a genuine change.

This kind of change, if real, would not show up on the CLA, and would be very difficult to measure. We might call it something like ‘appreciation’. In the eighteenth century, Joseph Addison and Richard Steele called it ‘the pleasures of the imagination’, which, they argued, opened up new possibilities of enjoyment for those of limited means:

A Man of a Polite Imagination, is let into a great many Pleasures that the Vulgar are not capable of receiving. He can converse with a Picture, and find an agreeable Companion in a Statue. He meets with a secret Refreshment in a Description, and often feels a greater Satisfaction in the Prospect of Fields and Meadows, than another does in the Possession. (Mackie, 1998: 388)

While Addison and Steele distinguish this man from the unrefined (the ‘Vulgar’, with obvious class connotations, but not entirely limited to class designation), they also distinguish him from the man who locates the main source of potential pleasure in acquisition. Thus you don’t need to own the statue in order to enjoy it. The appreciation of art, literature, and nature becomes in The Spectator ways of extending the kinds of cultivation long available to the elite classes to a wider spectrum of the population (including women, in spite of the gendered example). Certainly, there were severe limits to the democratizing range of the vision expressed in this periodical. Addison and Steele’s aesthetic vision, however, is connected to the possibility of an expanded participation in the public sphere.

This is only one example of an outcome specific to the study of literature that would not be captured by the CLA, but would be considered significant for, we think, many instructors. There are undoubtedly other such outcomes specific to the study of literature, and still others specific to the full range of academic disciplines. We need to know more than we do about the skills and capacities developed by the different disciplines – about the ways in which disciplines help students develop general skills and abilities, such as writing and critical thinking, and about the discipline-specific skills and abilities that result from the pursuit of a specific major. The more we understand the ways that disciplinary learning plays into the complex outcomes of higher education, the better we will be able to serve our students, and – not incidentally – make the case for the value of such learning. And if we do not develop this understanding, if the capacities measured by the CLA become the standard for all majors, then there is a danger – not often discussed, but we think quite real – that the unique contributions of the disciplines in developing not just content knowledge but capacities will be lost as well.

We end, then, with a paradox and an exhortation. On the one hand, as suggested above, the outcome goals of humanities fields are genuinely difficult to define. The achievements of humanities students reside in ‘minute particulars’, to borrow a phrase from William Blake. They are in some ways diminished by generalizations (which, as Virginia Woolf claimed, are ‘always also Generals’ ideas’). Most learning outcomes goals in the humanities seem reductive even to the most seasoned assessors. On the other hand, if humanities faculty leave the generalizations to others, we will most likely be left out of them entirely. We need to find ways to identify, assess, and communicate the learning that we all know really goes on. We need to take seriously what we find and map out ways to improve. We need to figure out how to do this not just at the local level, but in a systematic way with results that can be shared in public discussions. If we don’t, the model of limited learning will dominate these discussions for years to come.

Footnotes

Notes

Donna Heiland assumed the position of Vice President and Special Assistant to the President of Emerson College (Boston, MA) in July 2012. Her prior experience includes nearly eight years as Vice President of the New York City-based Teagle Foundation, which is best known for supporting evidence-based efforts to improve student learning. She has also served as Director of Fellowship Programs at the American Council of Learned Societies, and earned tenure in the English Department at Vassar College, where she taught for nearly 12 years. Her writing on literature includes Gothic and Gender: An Introduction (Blackwell, 2004), and her most recent publication is Literary Study, Measurement, and the Sublime: Disciplinary Assessment, which she co-edited with Laura Rosenthal and the assistance of Cheryl Ching (Teagle Foundation, 2011). She also contributed an essay to that collection, entitled ‘Approaching the ineffable: Flow, sublimity, and student learning’.

Laura J Rosenthal is Professor of English at the University of Maryland, College Park, where she teaches Restoration and eighteenth-century British literature and has been involved in various learning outcomes assessment projects, including chairing the Arts and Humanities Committee on Student Learning. In the field of eighteenth-century studies, she is the author, most recently, of Infamous Commerce: Prostitution in Eighteenth-Century Literature and Culture (Cornell, 2006) and editor of Nightwalkers: Prostitute Narratives from the Eighteenth Century (Broadview, 2008). Her work on assessment includes Literary Study, Measurement, and the Sublime: Disciplinary Assessment, which she co-edited with Donna Heiland and the assistance of Cheryl Ching (Teagle Foundation, 2011). Her essay in that collection is entitled ‘Assessment, literary study, and disciplinary futures’.

References

Arum

Roksa

(2011) Academically Adrift: Limited Learning on College Campuses, Chicago, IL: University of Chicago Press.

Arum R, Roksa J and Cho E (n.d.) Improving Undergraduate Learning: Findings and Policy Recommendations from the SSRC-CLA Longitudinal Project. Social Science Research Council.

AAC&U (2008) Proposal to the Teagle Foundation for ‘The engaged department: Building cross-departmental faculty leadership for liberal education outcomes’, Washington, DC: Association of American Colleges and Universities.

Bean

(2011) Backward design: Towards an effective model of staff development in writing in the disciplines. In: Deane

O’Neill

(eds) Writing in the Disciplines: Universities into the 21st Century, Houndmills: Palgrave Macmillan.

Beloit College, Knox College, Monmouth College and Ripon College (2006–2008) Assessing the value added to liberal education by academic majors. Available at: bioquest.org/teagle/index.php (accessed March 2012).

Brooks

(2011) Making the case for discipline-based assessment. In: Heiland

Rosenthal

(eds) with the assistance of Cheryl Ching, Literary Study, Measurement, and the Sublime: Disciplinary Assessment, New York: Teagle Foundation.

Diaz

Middendorf

Pace

Shopkow

(2008) The History Learning Project: A department ‘decodes’ its students. Journal of American History ((March): 1211–1224.

Graff

Birkenstein

(2011) A progressive case for educational standardization: How not to respond to calls for common standards. In: Heiland

Rosenthal

(eds) with the assistance of Cheryl Ching, Literary Study, Measurement, and the Sublime: Disciplinary Assessment, New York: Teagle Foundation.

Heiland D and Rosenthal LJ (2011) Let's close the gap between teaching and learning. Chronicle of Higher Education. February 24. Available at: http://chronicle.com/article/Lets-Close-the-Gap-Between/126499/ (accessed 10 September 2012).

10.

Heiland D and Rosenthal LJ, with the assistance of Ching C (2011) Literary Study, Measurement, and the Sublime: Disciplinary Assessment. New York: Teagle Foundation. Available at http://www.teaglefoundation.org/Resources/Teagle-Book (accessed 10 September 2012).

11.

Hutchings

(2011) From departmental to disciplinary assessment: Deepening faculty engagement. Change 43(5): 36–43.

12.

Mackie

(1998) The Commerce of Everyday Life: Selections from The Tatler and The Spectator, Boston, MA: Bedford/St. Martin’s.

13.

Middendorf

Pace

(2004) Decoding the disciplines: A model for helping students learn disciplinary ways of thinking. New Directions for Teaching and Learning 98: 1–12.

14.

Middendorf J, Pace D, Shopkow L and Diaz A (2007) Making thinking explicit: Decoding history teaching. The National Teaching and Learning Forum 16(2). Available at: http://www.ntlf.com/html/ti/thinking.htm (accessed 13 February 2012).

15.

Middlebury College (2011) Assessing student progression: Writing in the disciplines. Year III Report to the Teagle Foundation.

16.

National Research Council (2000) How People Learn: Brain, Mind, Experience, and School: Expanded Edition, Washington, DC: The National Academies Press.

17.

Pascarella

Blaich

Martin

Hanson

(2011) How robust are the findings of Academically Adrift? Change 43(3): 20–24.

18.

Shulman LS (2005) Signature pedagogies in the professions. Daedalus (Summer): 52–59. Available at: http://gse.buffalo.edu/gsefiles/documents/about/Signature-pedagogies-in-the-professions.pdf (accessed 29 March 2012).

19.

Sotherland

Dueweke

Cunningham

Grossman

(2007) Multiple drafts of a college’s narrative. Peer Review. (Spring): 20–23.

20.

Suskie

(2009) Assessing Student Learning: A Common Sense Guide, San Francisco, CA: Jossey-Bass.

21.

Teagle Working Groups (n.d.) The disciplines and liberal education. Available at: http://www.teagle.org/grantmaking/grantees/disciplines.aspx (accessed 4 March 2012).