Abstract
American Educational Research Association presidents’ presidential addresses have only intermittently considered relevance as a criterion of quality for education research. A few, though, argued that education research could only distinguish itself from research in the disciplines through attention to improving educational outcomes. David Krathwohl (1969) called for a coherent community encompassing practitioners, policymakers, and researchers. Philip Jackson (1990) urged that local research knowledge, rather than research-based dictates, be communicated. Alan Schoenfeld (1999) challenged education researchers simultaneously to build knowledge and offer solutions to practical problems. He anticipated current research-practice partnerships and shifts toward practice-embedded research supports. But only if the larger education research enterprise supports these efforts will the shift be radical and robust enough to persist and survive.
Reviewing 99 years of American Educational Research Association (AERA) presidential addresses occasions a certain level of humility about the capacity of any organization, however large and however well led, to significantly influence a field as large and complex as education. Yet the field of education research has developed enormously, in ways reflected in the presidential addresses and perhaps also at times influenced by them. The evident differences in the primary concerns of AERA presidents of different eras mark seismic shifts in the nature and aspirations of education research. The burning question for me is the direction in which those shifts in the concerns of education researchers take the field. Do those concerns represent a trend toward greater rigor, toward more urgent attempts to convince the world that education research is as methodologically sound and as concerned with important issues as research in the biological and physical sciences? Is there an increasing focus on the need for research to inform education policy? Or is there a trend toward greater concern with relevance to education practice, with making education research useful and usable in the world of schooling? These are, of course, not mutually exclusive tendencies. We would hardly wish to influence either policy or practice with research that is methodologically unsound or trivial in its concerns. But the justification for education research (and for an association of education researchers), as distinct from research in history, anthropology, economics, psychology, sociology, political science, and the other disciplines relevant to education, must reside in the relevance of research to policy or practice. I have chosen to review a subset of AERA presidential addresses in order to explore how the leaders of AERA were thinking about those issues at various points in the history of the organization, with a particular focus on links from research to practice.
I learned that the expectation that education researchers concern themselves with educational practice has been far from universal among the leaders of AERA. The AERA presidents during the early years of the Association defined the task of the presidential address as providing an inventory of the previous year’s accumulated education research, as represented in fewer than a dozen journals (see, for example, Woody, 1924) or, alternately, a mention of the most outstanding few publications within each of a set of defined areas (e.g., Thiessen, 1923). That this task was even considered feasible signals how small the field was, and how limited the number of publications available. It is also notable the degree to which the research being conducted focused on assessment and on the statistical methods associated with assessment, no doubt in response to the emergence during the First World War and the period immediately following of multiple efforts to assess IQ.
AERA presidents also selected themes that responded to the times in which they lived. Charters (1931) defended funding for education research threatened by straitened economic circumstances, and in 1958 Herrick presented stark evidence about the paucity of funding for education and behavioral research in comparison to that available for the physical and life sciences, a theme taken up again, but with greater optimism associated with the better funding prospects of the sixties, by Bloom in 1966 and Krathwohl in 1969. McConnell (1942) defended the place of education research in institutions of higher education during a period when “hard science” was emerging as the growth stock of universities. As funding for the biological sciences expanded, Campbell (1970) issued a plea for commensurate expansion in support for educationally relevant research. AERA presidents also worried a lot in their presidential addresses about methodological approaches, arguing, during the postwar period of expanded research funding and activity, variously for universal training in statistics (Walker, 1950), for the value of experiments (Stanley, 1967), for meta-analysis (Glass, 1976), and for theory building (Suppes, 1974).
One theme of high current interest, though, is touched upon only intermittently and glancingly: the relevance of education research to education improvement. In 1977, Fred Kerlinger even argued that there was little evidence of a close connection between research and practice in education and, in fact, that education researchers would do more good in the world by pursuing basic research and efforts at theory building. Kerlinger’s position perhaps reflected a more widespread pessimism generated by the failure of many applied research efforts that were originally embraced with enthusiasm, from programmed learning to the Physical Sciences Study Committee Physics to foreign language teaching using contrastive analysis. Indeed, he went on to document the difficulty of doing applied research, noting issues that in 2015 would be referred to as problems of implementation, contextualization, and subgroup differences.
On the other hand, Kerlinger’s predecessors and successors in the AERA presidency embraced the need for relevance and utility of education research. In 1948, Douglas Scates issued a heartfelt plea for a “science of man” that would rival the “science of things” that had so recently alarmed the world by producing an atomic bomb. Scates elaborated the tasks for a science of man as building and understanding social machines (mentioning schools, school districts, administration processes, and teaching as examples) and “ascertaining and distilling values” (Scates, 1948, p. 122). Aside from his brief exemplification of social machines, though, and one paragraph briefly summarizing his own work in studying them, Scates’s presidential address made remarkably little reference to education. He explicitly noted that developing the science of man would require inputs from multiple social sciences: I would not imply that our own number, working alone, is equal to the full task. But we, working together with all those everywhere who are endeavoring to clarify the problems of man, bear the onus and the opportunity of producing the basic materials and the well-knit principles, on which practical answers depend; out of which value judgments emerge; and from which philosophers, administrators, and all of us as citizens can obtain a clearer perspective on the course which man and his society should pursue. (Scates, 1948, p. 123)
In 1966, Benjamin Bloom exploited AERA’s 50th anniversary as an occasion for some stock taking. He celebrated the recent increase in funding, in quantity of research activity, in quantity and quality of doctoral students, and in the ability of the field to attract “new breeds” of workers from other disciplines. Nonetheless, he stated explicitly, “The case for educational research is yet to be demonstrated” (Bloom, 1966, p. 212) and asked somewhat plaintively, “Have the schools been changed in any respect by our work over the past 25 years?” (Bloom, 1966, p. 212).
Implicitly answering his own question, Bloom noted an emerging consensus that different instructional approaches are roughly equivalent in their impact on students’ factual knowledge or simple skills, and he averred a less completely convincing but emerging suggestion that dialogic methods greatly outperform didactic methods in influencing “higher mental processes.” He noted progress as well in understanding a variety of phenomena, ranging from individual development to individual differences, environmental effects to principles of learning, but concluded that none of these topics seemed likely to change schools directly. Bloom deplored the tiny percentage of published studies in education that were “crucial and significant,” a term I take to mean relevant to the improvement of education. Bloom estimated that only one out of a thousand published studies met the criterion—a ratio that current systematic reviews of the literature would probably not massively adjust. It is hard, of course, to have an impact on practice with trivial and unimportant studies and perhaps not even desirable!
Relevance of education research to education practice did receive serious attention, though, in a handful of the presidential addresses reviewed. The emergence of a concern for relevance between the 1960s and the 2010s is worth tracing in some detail, with an eye to projecting the future of this concern to AERA presidents and members. That is the task I will undertake in this article.
David Krathwohl’s 1969 Concerns
In his 1969 presidential address, David Krathwohl documented a number of successes for education research: increased attention from special commissions and from Congress; growth in funding that was on the verge of paying off in findings; the emergence of centers, laboratories, and companies focused on educational problems; and improved methods for asking substantive questions. The expanded capabilities of education research, he argued, were raising demands for critical self-examination, in particular in light of the widespread worries about the quality of education research. He went on: Educational research is often viewed as unrelated to practical purposes, too fragmentary, and poorly related to an overall framework of education. Let us examine these charges in relation to a three-way formulation of the problem and then consider a proposal. First we shall discuss the charge of irrelevancy as it relates to the extent to which researchers and practitioners have a sense of community, a relationship to which we shall refer as vertical integration. Second, we shall consider whether we have appropriate horizontal integration across the research community. Third, we shall examine the charge of fragmentation in the work itself. (Krathwohl, 1969, p. 5)
Acknowledging that charges of irrelevance were coming from state department personnel, superintendents, supervisors, principals, and teachers as well as university students, Krathwohl identified as a challenge the lack of a community that linked these groups to researchers. AERA had recently severed ties with the National Education Association, no doubt exacerbating the division between practitioners and researchers. Krathwohl is explicit about the costs this move incurred: But state department personnel, superintendent, principal, and teacher, all feel they have something to contribute to the researcher. Most often this is in terms of the definition of problems that would help to make research more relevant. Sometimes they have ideas for problem solutions. (Krathwohl, 1969, p. 6)
Krathwohl explains that a sense of community among these stakeholders, with parent-citizens added in, is a mechanism for creating readiness for adoption or installation of researcher- or developer-produced processes and products and that its absence can lead to suspicion about change and resistance to it. This notion, which has become part of the canon of some approaches to research presented as 21st-century innovations (see Snow, 2015, for a brief review), is then elaborated by Krathwohl with a suggestion for an approach that has since been tried recurrently and alas shown quite definitively not to work in changing practice: establishing a “translational” journal that presents technical research reports in popularized and accessible form.
Krathwohl’s prescience is further displayed in his complaint that school personnel have been insufficiently appreciative of the potential of research and development and that researchers have been insufficiently attentive to dissemination and installation. In other words, he was calling in 1969 for serious attention to the issues of implementation that are only now becoming a central focus of rigorous research and helpful theorizing. He also noted the theoretical fragmentation of education research as a challenge and proposed “one way of advancing theory building is to note that theory often grows out of observing the successful work of the skilled practitioner” (Krathwohl, 1969, p. 12), again an anticipation of the kind of practice-embedded research that many are calling for in 2015.
In 1969, Krathwohl identified a problem that persists in 2015—that many people, including many of those responsible for making education decisions and funding education research, are skeptical of the capacity of education researchers to deliver the goods. His proposed solution was the creation of a National Institute of Education (NIE), which would house and/or support problem-oriented education research. That solution was indeed implemented in 1972, and during its first 8 years (while administered as part of the U.S. Department of Health, Education, and Welfare [HEW]), NIE shaped an education research agenda while also supporting field-initiated research. NIE’s funding was relatively strong and stable until 1980. When HEW was split and a cabinet-level Department of Education established, NIE was absorbed into the Office of Educational Research and Improvement, and its research leadership role declined until it was eliminated in 1985. The Institute of Education Sciences (IES) was established in 2002, with an explicit focus on rigor rather than relevance. Unfortunately, although IES improved the funding picture and increased the quantity of research produced, the existence of a federal entity supporting and overseeing education research was not the ultimate solution to the problem of relevance. Education research remained largely tangential to efforts at educational improvement.
Philip Jackson’s 1990 Musings
Philip Jackson devoted his 1990 presidential address directly to the nature and value of education research. He presented and then dissected the distinction, originally made popular by Cronbach and Suppes (1969), between decision-oriented and conclusion-oriented research. Although decision-oriented research might seem to be more applied, more practical, more relevant to education practice, Jackson argued for the value of conclusion-oriented research, research that derived from the scientist’s questions rather than an external problem, because of its capacity to change default thinking and presupposed ways of knowing and asking.
Jackson went on, though, to characterize two kinds of qualitative, descriptive research (which I characterize as ethnographic and historical, although he did not use those terms) as a move toward relevance even as traditional applied questions are abandoned. He wrote, That dream of finding out once and for all how teaching works or how schools ought to be administered no longer animates nearly as many of us as it once did. In its place we have substituted the much more modest goal of trying to figure out what’s happening here and now or what went on there and then. . . . A focus on local knowledge (to borrow yet another term from Geertz) can be just as oriented toward solving the day-to-day problems of practitioners as can the search for invariant principles of practice. Nonetheless, the move toward more descriptive strategies, with their heavy dependence upon narration, naturalistic observation, and all of that, does signal a more relaxed attitude toward the relationship between the research community on the one hand and the world of practice on the other than has been true in the past. (Jackson, 1990, p. 7)
In other words, Jackson was defining the role of the researcher as communicator and of research knowledge as deeply connected to the experiencer/observer rather than an external body of agreed-upon facts. Although this approach puts the practitioner into an equal relationship to the researcher, it falls far short of ensuring that the research activity is relevant to the practitioner’s interests or needs.
Alan Schoenfeld’s 1999 Ambitions
Schoenfeld echoed none of the contemplative hesitancies of Jackson or Krathwohl in defining a role for education research. Rather, he argued for the possibility of an orderly accrual of knowledge in response to the identification of urgent questions, a process that had been successful in mathematics after David Hilbert’s (1900) challenge to the field. Schoenfeld (1999) presupposed the relevance of education research to education practice, writing, The two main points I wish to make are as follows: It is possible to conceptualize educational (and other) research in such a way that “pure” and “applied” work are not in conflict, but so that contributions to basic knowledge and contributions to practice can be seen as compatible and potentially synergistic dimensions of our work. Educational research has evolved to the point where it is possible, much of the time, to conduct research in contexts that are of practical import, working on problems whose solutions help make things better and contribute to theoretical understanding. Finding and working on such problems is a high-leverage strategy for making a difference in the years to come. (p. 5)
Schoenfeld then went on to identify six key problem areas. Under the first, “unifying the cognitive and the social,” he raised two topics described as having a particularly strong likelihood of bridging to practice:
Theories and models of competence in various content areas, which he argued provide direct input to curricular design, to pedagogical approaches, and to instructional goals.
Theories and models of acting in context, which he argued were crucial to understanding performances like teaching and thus to providing appropriate supports.
Schoenfeld reverted to Stokes’s (1997) two-dimensional conceptualization of scholarly undertakings for a resolution of the applied/basic distinction and argued that education had achieved a level of maturity that allowed for lots of work in Pasteur’s quadrant. He provided two salient examples from his own work: the development of a course to teach mathematical problem solving that emerged originally from theory but was elaborated and refined in crucial ways over 10 years of being simultaneously implemented and studied, and a study of teaching that married theories about good teaching with careful observations of real-world teaching.
Schoenfeld went on to identify six domains of practice that could, if studied carefully, contribute to theory: curriculum, assessment, reform efforts, research infrastructure, teachers’ professional lives, and pursuing opportunity. His strong endorsement of the synergy between research and practice has, in fact, been played out in his own research agenda in the years after his presidential address, as he joined the Strategic Education Research Partnership work in San Francisco and contributed to the development of math learning tools for teachers and students that responded to urgent problems of practice, embodied research knowledge and theory, and were tested and improved in partnership with practitioners (see, for example, http://math.serpmedia.org/gateway/).
The Future of Relevance
Education researchers have become increasingly concerned about the irrelevance of much education research to the improvement of education and the contribution of that irrelevance to the low esteem in which education research is held. Alternate approaches, referred to as practice-embedded educational research (Snow, 2015) or design-based implementation research (http://learndbir.org/), have been developed specifically to put the issues of practice at the center of researchers’ concerns. The Strategic Education Research Partnership (serpinstitute.org) has formulated a set of procedures and principles for prioritizing practice issues in conceptualizing and implementing research agendas. Central in SERP’s informal manifesto is the formation of partnership relations with districts or groups of schools, so that efforts found to be successful at the classroom level are less likely to be derailed by organizational change (Donovan, Snow, & Daro, 2013). Other research organizations (the Carnegie Foundation for the Advancement of Teaching, the Consortium for Chicago School Research, the Baltimore Education Research Consortium, and others) have embraced these principles, making it a default that questions of concern to education researchers should derive directly from practice rather than from theory or minor inconsistencies among prior studies.
Fully reviewing the contributions these various research groups have managed as a result of their modified approach to formulating research questions goes beyond the scope of this brief review. But in addition to Schoenfeld’s work, cited above, SERP has worked with Boston, San Francisco, and Baltimore in developing, evaluating, and modifying a discussion-based curriculum focused on improving literacy and critical thinking skills (Word Generation; wordgen.serpmedia.org); Cobb, Smith, and colleagues have worked since 2007 in partnership with districts to improve middle school math outcomes (http://peabody.vanderbilt.edu/departments/tl/teaching_and_learning_research/mist/); the Carnegie Foundation for the Advancement of Teaching has partnered with community colleges to improve pass rates for gatekeeper math courses (http://www.carnegiefoundation.org/in-action/pathways-improvement-communities/); and research-practice consortia have been established in Baltimore (http://baltimore-berc.org/) and New York City (http://steinhardt.nyu.edu/research_alliance/), all to some extent inspired by the example of the Chicago Consortium on School Research (https://consortium.uchicago.edu/). See Snow (2015) for more information about these and other such encouraging examples.
Of course, starting from urgent problems of practice and working in partnership with practitioners, one encounters and generates many obstacles. It is worth articulating a few of these, in the hope that anticipating them might reduce their capacity to derail a noble effort at relevance. First, neither researchers nor practitioners are currently receiving professional training that supports them to work in collaboration with one another, in what Krathwohl would have called “vertically integrated” ways. The particular challenges of being responsive to practitioner concerns are difficult for any researcher but may be particularly impractical for doctoral students eager to finish a dissertation within the prescribed time frame. Furthermore, such efforts are risky—subject to the volatility of school, district, and state education leadership. Similar issues arise for junior faculty members, for whom traditional research modes may constitute safer routes to the funding and publication record required for tenure. Thus the responsibility to take the risks associated with practice-embedded research falls to secure senior scholars, those least likely to modify their ways. And although we could cite many cases in which doctoral students and junior scholars have collaborated productively on research undertaken in partnership settings, it is still not the case that training in how to do such work is a normal part of the methods courses they take.
Funding priorities and criteria also need to be revised if practice-embedded research is to thrive. It is quite understandable that funders want clear plans and anticipated outcomes before investing scarce research dollars. Clear plans do emerge from researcher-practitioner partnerships but only after the investment of time and energy in establishing the partnership. That crucial establishment phase of the work needs to be funded as well, in a way that fully recognizes the complexity of creating partnerships and defining questions worth taking on in a collaborative way. The recently established IES Partnership funding mechanism is one effort to address this issue, as is the Spencer Foundation’s support for the continuation of existing partnerships, but far more funding is available for traditional than for practice-embedded research.
Decades apart, Scates, Krathwohl, Jackson, and Schoenfeld all called for increased relevance of education research. At times their voices were resonant with the concerns of policymakers, as was the case with the short-lived NIE and when IES extended funding for the Regional Education Laboratories and Centers. Although the latter have survived the congressional axe, the financial and contractual constraints under which they operate ensure their limited influence on the vast field of education research. If current calls for relevance are to meet with greater success, they must reach further into the education research enterprise, shifting funding patterns more substantially while changing publishing standards, criteria for promotion and tenure, and opportunities for both education researchers and practitioners to develop the expertise required for productive collaboration. In the absence of those changes, the current interest in relevance will be just one more data point for scholars in 2116 looking back to track the impact of education research on practice.