Abstract
In 2002, the National Research Council (NRC) released a report that articulated its vision about education research (Eisenhart & Towne, 2003; Feuer et al., 2002), which focused on scientifically based research methods. The release of the report was followed by an extensive debate and is broadly understood as part of the long-standing “paradigm war” 1 in the field (Fischman & Tefera, 2014; Munoz-Najar Galvez et al., 2019).
Some of the scholars defending the criteria advanced by the report had somewhat traditionally dismissive views of education research “as something of a stepchild, reluctantly tolerated at the margins of academe and rarely trusted by policy makers, practitioners, or members of the public at large” (Lagemann, 2000, p. x). A similar perspective was advanced by Grover Whitehurst, the influential director of the Institute of Education Sciences from 2002 to 2008, who claimed that the world of education, unlike defense, heath care, or industrial production, does not rest on a strong research base. In no other field are personal experience and ideology so frequently relied on to make policy choices, and in no other field is the research base so inadequate and little used. (Whitehurst, 2007, quoted in Hess, 2008, p. 9)
On the opposite side were those who argued that by defining scientifically based research as primarily based on quantitative approaches, the report devalued some long-standing ways of generating knowledge in education (Erickson & Gutierrez, 2002; St. Pierre, 2006; Willinsky, 2001). For the skeptics, the report was seen as another manifestation of a long-standing pejorative assessment of education scholars as using outdated, ineffective, and narrowly constructed research methods, and slow to adapt to new developments. Criticizing federal policies for narrowly defining scientific research in education and arguing for a more inclusive approach to education research, Lather (2004) noted that “in spite of its [NRC’s] oft-repeated intentions of balance across multiple methods, objectivity is enshrined and prediction, explanation, and verification override description, interpretation, and discovery” (p. 762).
It is hard to assess to what extent the paradigm war in education research is over (Munoz-Najar Galvez et al., 2019), but we are convinced it is necessary to move beyond it in the service of answering urgent and compelling questions for the field. This move is important because the landscapes where education research occurs have become even more complicated due in part—but not limited to—the emergence of novel knowledge infrastructures, new developments in information and communication technologies, as well as political, scientific, and cultural changes.
We want to highlight two trends. First, the predictions of both sides in the paradigm wars about the devastating effects of what each perceived as the poor research practices of their opponents (Biesta, 2020; Lather, 2004) have not destroyed the field. Indeed, based on the number of scholarly publications, attendance at scientific conferences, and enrollments in graduate programs, it appears evident that education research continues to grow as a field (Levine & Hill, 2015; McCarty, et al., 2017). Second, increased power in data-processing and technological capacities has generated new conceptual and technical challenges for education research.
These two trends have created opportunities for education researchers to (a) reconceptualize existing approaches, (b) adopt and advance methods associated with other fields, and (c) leverage new technologies to expand the capabilities of existing tools. 2 This volume highlights how education research is a sophisticated and robust field, and the diversity and complexity of the ever-shifting contexts for education research, while also featuring some novel ways of analyzing those contexts.
Our goal as editors was to create a volume that will serve as a resource for both novice and experienced education researchers for understanding how innovative methodological approaches might provide more comprehensive explanations into enduring questions, challenge existing theoretical frameworks and paradigms, or address novel challenges in the field of education. The chapters in this volume demonstrate how emerging approaches continue to address the complexities of education research—ethics, politics, diversity, and its interdisciplinary nature, to name a few—all of which make education research “the hardest-to-do science of them all” (Berliner, 2002, p. 18). Some of these approaches can also be regarded as responses to changes in the context of research that showcases how education scholars adapt to new demands and opportunities.
It is probably not surprising that most of the chapters in this volume position emergent methodologies as responses to challenges educational systems face locally, nationally, and internationally. Arguably the most prominent of these challenges 3 is to address unjust gaps in educational access and opportunities. Inspired by Ladson-Billings’s (2015) call to “move from justice as theory to justice as praxis,” the 2017 volume of Review of Research in Education was devoted to how inequities in education research and practice can be disrupted. In this introduction, we show how the chapters in the present volume extend that call and propose new methodological ways to further educational equity and promote social justice.
Technological Innovations that Change the Scale and Scope of Education Research
A number of chapters in this volume address the way that technological advances have changed the scale of education research. Consider, for example, how new technologies have changed some aspects of teaching and learning. The adoption of assessment and instructional technologies (Halverson & Smith, 2009; Straub, 2009) and online learning at scale have enabled researchers to collect large amounts of data that monitor and record learning activities in online and technology-rich environments. These developments have also fostered the development of new methods of inquiry aimed at understanding these changes and their consequences.
Researchers are developing new methods to adapt to the increased scale of the research context and utilize the opportunity provided by big as well as fine-grained data, and advances in computer-based systems and software packages. Several of our authors describe the emergent methods that respond to these demands. The geographical mapping of educational data and spatial methods of analysis have allowed researchers to identify, document, and assess incidences and patterns of inequality and injustice (Cobb, Chapter 4; Butler & Sinclair, Chapter 3). Using geospatial methods, education scholars are able to examine the distribution of opportunities and access to educational and other resources. Take, for instance, spatial justice—the distribution of opportunity across geographies at multiple scales. As charter schools, vouchers, and inter- or intradistrict choice options have expanded, researchers have used Geographic Information System (GIS) mapping (Cobb, Chapter 4) and other geospatial methods (Butler & Sinclair, Chapter 3) to document how spatial location, locale, and sense of place influence the contexts and outcomes of schooling. These methods have helped us better understand the factors, such as distance between students’ residences and schools, that influence the choices students and families make, and to demonstrate the segregating effects of choice.
Similarly, in Chapter 5, Fischer et al. address the challenges of the partial relocation of traditional knowledge infrastructures online by mapping the landscape of big data and their increasing use in education research. Connecting administrative and learning process data using new techniques of data mining allows researchers to understand the mechanisms of education policy effects, reveals the nuances of educational inequities, and provides insights for policymakers.
The availability of large databases, the globalization of networks, and new search engines enabled by high computing power have led to a reconceptualization of how research syntheses are conducted. As Hammond et al. point out in Chapter 1 of this volume, this traditional staple of education research should be regarded not only as a formal methodology but rather also as a process of engagement with “the ecologies of policies, practices, norms, resources, social structures, technologies, and methodologies through which knowledge is produced” (p. 2).
Analyses of big data integrated with natural language processing and network analysis complemented with qualitative analytical methods, is yet another example of how education researchers expand the boundaries of traditional methods. Goren et al. (Chapter 2) demonstrate how a novel way to review and compare large bodies of literatures across disciplines facilitates the identification of interconnections, common trends, and divergences. Yet, alongside this increased scale of data and the technological ability to condense large amounts of information, Philip and Gupta (Chapter 7) and Green et al. (Chapter 6) highlight the continued importance of attending to smaller scale processes occurring within classrooms.
Expanding the Boundaries of Education Research by Borrowing from Other Disciplines
Relatedly, new methodological approaches enter education research via interdisciplinary channels as tools and methods prevalent in other fields are adopted and adapted by education researchers. For example, the spatial sciences contributed to understanding how location might perpetuate educational inequality (Cobb, Chapter 4; Butler & Sinclair, Chapter 3), and advances in social network analysis allow researchers to reconceptualize the relationship within and between educational actors and networks (Froehlich et al., Chapter 9). The multidisciplinary nature of geospatial methods and social network analysis provides opportunities to integrate the insights and efforts from other fields to raise new questions and identify otherwise undetected patterns and phenomena. Similarly, Miller et al. (Chapter 10) highlight how education researchers used counternarrative drawn from critical race theory, which has its origins in legal analysis, to highlight how racial inequality has been deeply institutionalized within educational systems and practices.
Another example of the integration and adaptation of methods is the use of quasi-experimental research designs based on naturally occurring experiments to produce causal claims. In the wake of the “credibility revolution” in economics (Angrist & Pischke, 2010) and the strong emphasis on an evidence-based approach to policy and interventions by the government (No Child Left Behind, 2002), an increased number of education researchers have utilized quasi-experimental research designs (Angrist, 2004). In Chapter 8, Gopalan et al. describe the methods and standards for causal inference in education research formalized by the What Works Clearinghouse (WWC). The influence of the WWC standards for evaluation of education interventions in education research is also evident in the analysis of the WWC as one of the major research synthesis infrastructures (Hammond et al., Chapter 1).
Yet another example of merging approaches within the discipline is what Green et al. describe in Chapter 6 as a microethnographic-discourse approach. This logic of inquiry leverages the knowledge and experiences of researchers around the globe in their respective contexts to explore potential common understandings of learning in classrooms. Similar to many chapters in this volume, the guiding principle of the telling case studies analyzed by Green et al. in this chapter is to understand and support the needs of diverse learners in different educational contexts, and to ensure their access to educational opportunities. They also highlight how disciplinary knowledge is an important resource for researchers as they come to understand how students learn in different settings.
Continuing the conversation about learning, Philip and Gupta (Chapter 7) analyze the vertical and horizontal interactions between learners and their environments in math and science settings. Presenting the evidence bases on interaction and microgenetic analyses within the critical social theory framework, Philip and Gupta question the notion that students’ engagements with macro-level identities and ideologies in spaces of learning are stable, and underscore the need for explicit analytical attention to intersectionality.
Rethinking What We Know So Far
New approaches to research are often developed by merging existing methodologies between as well as within different types of methods and reconceptualizing traditional ways of asking and answering questions. In education research, as in other social sciences, “the social and cultural contexts of the phenomenon studied are crucial for understanding the operation of causal mechanisms” (Maxwell, 2004, p. 6). This requires approaches that combine a variety of methods and perspectives that allow researchers to examine phenomena from different angles.
Yurkofsky et al. in Chapter 14 utilize the tools of social network analysis to demonstrate the relationships between different continuous improvement methods. In that chapter, authors also rethink the directionality of the connection between practice and research—one of the challenges intrinsic to education research. They point out that the failure of both experimental and implementation paradigms to find policy solutions occurred because these solutions were developed separately from the context-specific problems facing the educators they are meant to serve. The authors argue that researchers should allow practitioners to identify problems and participate in knowledge generation to develop solutions to those problems of practice.
Building appropriate infrastructures for meaningful interactions between researchers and practitioners will foster research that is useful to practice. That said, the success of the infrastructure depends on the willingness of teachers and other practitioners to participate in the research process. Understanding teachers, their teaching, and their thinking is critical if researchers want to engage educational professionals in a more substantive way. Chapter 13 by Schachter et al. introduces a solution to the conceptual and methodological challenge of documenting the relationship between what teachers think and how they act, or between their private and public identities. These identities, or teachers’ public and private worlds, can be captured methodologically by visual and language data. Researchers have repeatedly identified the disconnect between the two worlds, what the authors characterize as bifurcation. In the presence of these bifurcating worlds, researchers should balance their use of visual and language data, but this is rare in reality—few studies draw conclusions from equally weighted visual and language data. This raises both methodological and theoretical concerns and suggest the need for more integrative research procedures. The reconceptualization of conventional approaches is also the focus of Froehlich et al. (Chapter 9), who show how traditionally quantitative social network analysis benefits from incorporating mixed-methods and qualitative approaches to research. They also highlight some of the challenges of mixed-methods social network analysis.
Another example of methodological craftsmanship in combining existing methods to uncover new perspectives and illuminate new approaches is the use of counter-narrative (Chapter 10, Miller et al.). As a tool developed to foster education equity, critical counter-narrative borrows from narrative inquiry, autoethnography, and life history and is based on the premises of critical race theory. Critical counter-narrative is a timely and absolutely necessary approach for a society that is becoming increasingly and more openly diverse because it gives power and voice to marginalized communities beyond storytelling. Miller et al. also point to the ways that this rigorous scientific approach serves as a pedagogical tool with the potential to transform teaching practice and how it might be expanded to more effectively promote educational equity.
As showcased by all the chapters in this volume, the interdisciplinary nature of education research requires synthesizing frameworks and methods from different perspectives and suggests that education research in part depends on advancements in other disciplines (NRC, 2002). For example, the review of qualitative comparative analysis (QCA) by Cilesiz and Greckhamer (Chapter 12) spans political science, psychology, sociology, management, and the organizational sciences to show how education researchers can use QCA to reconceptualize causal relationships in complex settings characterized by conjunction, equifinality, and asymmetry. While still not completely embraced by scholars in education, the logic of Boolean algebra is a promising approach to help us better address social inequality by highlighting the complex relationships between the factors associated with disadvantage and privilege through the lens of intersectionality. Cilesiz and Greckhamer show how other fields and disciplines have approached similar questions and encourage education researchers to take advantage of QCA, especially in the area of policy analysis.
The issues we highlight above have also created new challenges for research synthesis as well as ethical issues that researchers will need to consider as they move forward. The instability of research databases such as ProQuest due to frequent updates of search algorithms creates challenges with the replicability of research syntheses (Schachter et al., Chapter 13). Hammond et al. (Chapter 1) emphasized the need for a new approach to the standards of research synthesis that could vary between full flexibility on one end and common standards on the other end. Such an approach requires understanding the mechanisms for quality control among the existing synthesis infrastructures and the accessibility they provide to researchers. Considerations of accessibility versus privacy is one of the central issues in using big data and specifically administrative data. The trade-off researchers face is on the one hand limiting the range of questions to be addressed and on the other hand protecting the subjects of research and their private information (Fischer et al., Chapter 5).
The expanded use of administrative data has also raised questions about the extent to which it allows researchers to adequately address the ways race shapes educational phenomena. As Viano and Baker (Chapter 11) observe, there is a tension between the static racial categories often used in administrative data and the comparatively fluid nature of racial identities. While administrative data could provide researchers with possibilities to analyze the fluidity of racial/ethnic identities and capture the dynamics in how identities are defined, they also highlight a need for a new framework that could address the complexity of the data themselves (Viano & Baker, Chapter 11).
Spatial analysis and the associated use of geographical methods revealed the need to develop new theoretical perspectives as many of the reviewed studies that employ GIS methods are undertheorized in that they use a technique without being accompanied by a clear theory. To address this challenge, Butler and Sinclair (Chapter 3) propose merging GIS and other spatial methods with critical spatial research paradigms and perspectives to explore the experiences of communities that are most often affected by inequity and spatial injustice. Relatedly, in Chapter 13, Schachter et al. note a lack of explicit theorization of the connection between visual and language data and how these two types of information allow researchers to understand the link between teachers’ private lives and the public process of teaching.
Conclusion
What I am going to argue is that the critical mind, if it is to renew itself and be relevant again, is to be found in the cultivation of a stubbornly realist attitude. (Latour, 2004, p. 231)
Our main motivation and goal when we conceptualized this volume for Review of Research in Education was to contribute to “the cultivation of a stubbornly realist attitude.” Integral to our realist perspective is the belief that methods and methodological approaches have a peculiar place in the inquiry process—they are often overshadowed by the significance of the research question and the consequences of the answer and solutions to it. Unlike theories and conceptual frameworks or paradigms that set down the motivation, intent, and expectations for systematic inquiry, methods provide systems and principles for carrying out that inquiry. In this volume, the authors brought research methods and methodologies to their well-deserved spotlight.
The chapters in this volume describe approaches that cannot be easily summarized by the traditional distinction of quantitative versus qualitative methods. They highlight how moving beyond the paradigm wars and engaging in key concerns within the field can foster the innovation needed to adapt to the changing environment for education research and advance the field. The different contributions also suggest how we might move beyond the easy temptation of defending sides in the paradigm war to engage with matters of central concern to the field, and this is how innovations and adaptations to the changing environments for modern education research are generated.
The contributions to this volume also foster the dialogue between conceptual perspectives and technical and methodological procedures. There are no perfect methods that will provide effective tools for all educational problems, and as the authors of this volume explicitly showed, testing theoretical ideas require new methods, and new methods call for revising existing conceptualizations and theorizations.
The methods, models, and processes discussed in this volume demonstrate that education researchers are quite adept at adapting and using state-of-the art methods such as natural language processing, geographical methods, and network analysis. This is in sharp contrast with what is often the dismissive perspective that education researchers do not keep up with other fields in how they develop rigorous procedures and adapt new methods. 4 Education researchers not only adapt methods but also develop creative interdisciplinary approaches such as combining video and language data. The interdisciplinary orientation of the field is evident in the multitude of methods used to study educational phenomena, and also in the combinations of these methods. As the authors in this volume demonstrate, these include but are not limited to geospatial data and methods used to study justice, critical social theory combined with the methodological approaches of interaction analysis and microgenetic analysis to understand the dynamics of power and learning, or the use of administrative data or traditional qualitative methods to study race and identity.
Overall, the methods discussed in this volume provide evidence of the strengthening of the interdisciplinary and intersectional perspectives in the field (Tefera et al., 2018). We are also hopeful that our readers will agree that the chapters in this volume reflect Latour’s (2004) particular perspective of being critical: “That is generating more ideas than we have received, inheriting from a prestigious critical tradition but not letting it die away, or ‘dropping into quiescence’ like a piano no longer struck” (p. 248).