Abstract
This special section is directed to a continuing conversation as to what counts as quality research in education. For any field of science and scholarship, serious reflection on the elements of transparent and well-warranted research merits ongoing attention. In this section, that dialogue is extended to further exploring what constitutes “high-quality” research. The American Educational Research Association (AERA) formally addressed some of these issues in issuing “Standards for Reporting on Empirical Social Sciences Research in AERA Publications” in 2006, followed by issuing a second set of standards in 2009 focused on humanities-oriented research. Continuing to engage with such questions reflects the editorial team’s premise: that high-quality research should be fundamental to the improvement of educational policy and practice.
The Search for Criteria of Quality in Research
The commentaries in this special section respond to the continuing need to define within our various academic and research communities the criteria of rigor so that we can be better positioned to share these criteria with other communities, including policymakers and educators. They also point to the importance of understanding how policy decisions shape research that is conducted in education. Assessments of what constitutes rigorous research shape what policymakers choose to support, what educational researchers hold as valuable, and what educational practitioners choose to implement.
The possibilities for sound research and inherent tensions in responding to both the formulation of research and theory and the significance of practice are reflected, in part, in the recent (August 2013) report titled “Common Guidelines for Education Research and Development,” jointly released by the Institute of Education Sciences (IES; 2013a) and the National Science Foundation.
Likewise, the most recent call for proposals (May 2013) from IES (2013b) raises some of the issues:
Education has always produced new ideas, new innovations, and new approaches, but only appropriate empirical evaluation can identify those that are in fact improvements. Taken together, work across the various goals [of the call for proposals] should not only yield information about the practical benefits and the effects of specific interventions on education outcomes but also contribute to the bigger picture of scientific knowledge and theory on learning, instruction, and education systems. (IES, 2013b, p. 11)
Special Section Content
For this special section, we have invited five scholars to offer their views on this call for “scientific knowledge” in education and how scientific knowledge is understood and perhaps misunderstood. The scholars come from a variety of research fields and draw upon a range of research orientations, including those who focus on history, philosophy, learning, and the needs of traditionally marginalized learners as well as a researcher from the natural sciences who is also known for his work in physics teaching and learning.
The conversation begins with D. C. Phillips’ commentary, “Research in the Hard Sciences and in Very Hard ‘Softer’ Domains.” Phillips, a philosopher of education and social science, argues that all “competent inquiries” share basic features. He also draws our attention to the problematic nature of the term scientific and argues for a greater need for “ecological validity” in our work.
Carl E. Wieman, in his commentary titled “The Similarities Between Research in Education and Research in the Hard Sciences,” extends this comparison, writing that “the nature of research in the two areas is far more similar than researchers in either community recognize” (p. 12). Wieman, a physicist, suggests that cutting-edge science is often “much messier, complicated, and less precise” (p. 12) than is commonly acknowledged—and in this way, it bears remarkable similarities to education research. Wieman argues that “the predictive power, and corresponding new insights, that a research study will provide is a more meaningful measure of its rigor and value than what particular research design it uses” (p. 13).
This emphasis on understanding the nature of scientific inquiry continues with John L. Rudolph’s commentary, “Why Understanding Science Matters: The IES Research Guidelines as a Case in Point.” Rudolph, who focuses on the history of science education in his research, describes the problematic nature of what is often termed the “experimental model of science.” He suggests that this standard risks casting other research traditions as deficit. Rudolph also notes that “methods of inquiry are highly contextual, contingent, and emergent over time” and that many may “fall outside the narrow band of those recognized as experimental. This fact, however, makes them no less scientific” (p. 16).
Kris D. Gutiérrez and William R. Penuel continue in the vein established by Rudolph in their essay titled “Relevance to Practice as a Criterion of Rigor.” Drawing on their expertise from literacy education, learning sciences, and educational psychology, Gutiérrez and Penuel argue that if education research is to be meaningful—that is, if it is to allow us to understand substantive and equitable change in education so that we can better organize conditions for learning—relevance to practice must be an explicit criterion of quality research. In their contribution to the conversation, these authors consider what this new criterion, “relevance to practice,” means for the conceptualization, design, and conduct of programs of education research.