Evaluating Rigor in Qualitative Methodology and Research Dissemination

Abstract

Despite previous and successful attempts to outline general criteria for rigor, researchers in special education have debated the application of rigor criteria, the significance or importance of small n research, the purpose of interpretivist approaches, and the generalizability of qualitative empirical results. Adding to these complications, the breadth of qualitative research methods makes a single set of universally applicable criteria difficult to identify and use. Based on input from qualitative researchers across the social sciences, we augment and expand the oft-cited criteria for rigor established in special education, thus broadening the potential application and contribution of qualitative research in the areas of disability and education. We identify exemplars from special education research and use these to illustrate ways qualitative research can push the field to strengthen the theoretical foundations of empirical work, as well as to acknowledge more forthrightly the roles of the researcher in the research endeavor.

Keywords

qualitative research peer reviewing evidence base scientific education research

Dissemination of research is a key step in the research process. In a peer-review culture, reviewers act as crucial players in dissemination efforts; they are charged with evaluating the rigor of the design the validity of the results, the presentation of the work, and its overall importance and contribution to the field (Graue, 2006). But, as implied by the term culture, peer reviewers are not simply individuals making decisions based on personal caches of expertise. The culture of a given field shapes members’ decisions about what is valuable, mediating practice and reinforcing institutional hierarchies (Bourdieu, 1990). A discussion of rigor in qualitative special education research requires contextualizing the field’s broad definition of quality and recognizing that it is both generative of and generated by values and beliefs about the nature of valuable knowledge relative to disability, teaching, and learning.

Special Education and the Conceptualization of Rigor

Education research has often been critiqued for not being sufficiently scientific (Condliffe Lagemann, 2000; Labaree, 2004). One response to this critique has been the promotion of scientific education research produced through randomized experiments and verifiable through objectively generated evidence (Eisenhart, 2005; Eisenhart & Towne, 2003; National Research Council, 2002, 2005; Slavin, 2002, 2003). Special education has followed this pattern. In 2005, a group of scholars identified criteria for rigorous scientific methods in a special issue of Exceptional Children (Odom et al., 2005). In 2009, another special issue of the same journal included studies using these criteria to evaluate the evidence base of reading, writing, math, and behavior interventions (Cook, Tankersley, & Landrum, 2009).

The introduction to the 2005 special issue promotes experimental designs as the gold standard for evaluating the effectiveness of a teaching practice or intervention (Odom et al., 2005), aligning its criteria with the National Research Council’s (2002) Scientific Research in Education report and the Institute of Educational Sciences’ (IES) What Works Clearinghouse. However, the authors argue that the complicated nature of special education requires multiple methods of knowledge production. Quality indicators were identified for group experimental and quasi-experimental (Gersten et al., 2005), single subject (Horner et al., 2005), correlational (Thompson, Diamond, McWilliam, Snyder, & Snyder, 2005), and qualitative research designs (Brantlinger, Jiminez, Klingner, Pugach, & Richardson, 2005). Indicators of quality were communicated as questions (e.g., “Is a power analysis provided to describe the adequacy of the minimum call size?” Gersten et al., 2005, p. 151), declarative benchmarks (e.g., “The process of selecting participants is described with replicable precision”; Horner et al., 2005, p. 174), and rubrics (e.g., “Description of dependent variable,” ranging from none to operationalized; Chard, Ketterlin-Geller, Baker, Doabler, & Apichatabutra, 2009). Through discussion in these special issues, special education scholars began to codify a culture of research quality, positing standards focused on alignment of purpose and design and defining the parameters of design rigor.

Much like the National Research Council and IES, the field of special education operationalized the value and rigor of research hierarchically, prioritizing evidence resulting from a range of experimental designs (Odom et al., 2005). In this model, qualitative research is relegated to exploration of processes in the natural setting where experimentation is not possible, development of new teaching and learning practices, and verification of promising practices via social validity studies. Whatever the purpose, in this view, qualitative methods are tools used to describe and/or to explain processes (Brantlinger, Jiminez, et al., 2005), and quality criteria for qualitative special education studies have been introduced in this context.

Framing Quality in Special Education Qualitative Research

Although rigor in qualitative research has received considerable attention in social science research (Trainor & Graue, 2012), few articles on this topic have appeared in scholarly journals in special education. One exception is Brantlinger, Jiminez, and colleagues’ (2005) identification of quality indicators published in the aforementioned special issue of Exceptional Children, which has a focused discussion of rigor around trustworthiness, data collection, and analysis. This discussion of rigor is conceptually linked to Lincoln and Guba’s (1986) foundational work and emphasized the truth-value of the data collected as tethered to researchers’ interpretations. Brantlinger, Jiminez, et al. (2005), provide a set of general, interpretive criteria for qualitative research across methodological approaches to aid in assessing the credibility of the work, as well as a number of quality indicators that address the implementation of qualitative methods specific to interviewing, observing, and analyzing documents.

The justification for the use of qualitative methodologies in special education research is inextricably linked to the types of questions qualitative research best answers. Research questions that attempt to answer how or why a process or phenomenon occurs within complex contexts, where variables are difficult to control and measure, are particularly well-suited to qualitative methods of investigation. Qualitative methods are also useful in the examination of processes and phenomena where the perspectives of multiple stakeholders are, or are thought to be, central to our understanding and practice. In addition, because disability is situated in diverse sociocultural, historical, and political contexts, interpretation is an essential component of knowledge base (Pugach, 2001). While ascertaining quality and presenting a rationale for the importance of research are not synonymous, Brantlinger, Klingner, and Richardson (2005) are not alone in linking the two concepts and couching them in the broader debate about knowledge production, evidence-based practices, and the science of education (e.g., Cook et al., 2009; Eisenhart & Towne, 2003; Giangreco & Taylor, 2003; National Research Council, 2002, 2005; Odom et al., 2005). The discussion of scientific evidence in support of educational interventions, however, has been narrow in scope and has largely discounted evidence generated through interpretive, constructive, and post-modern research approaches (Lather, 2004; Maxwell, 2004; Schwandt, 2005). Concerns about the narrowing valuation of research approaches have prompted calls to expand the underlying paradigms of research for the purpose of maximizing our capacity to use appropriate and varied tools in our attempt to solve educational problems (Guba & Lincoln, 2005; Lather, 2006; Schwandt, 2012; Trainor & Bal, 2014). Furthermore, the paradigmatic foundation of research approaches drive conceptualizations of rigor (Denzin, 2005; Lincoln, 1995; Smith & Deemer, 2000; Smith & Hodkinson, 2005).

The Need to Expand Quality Indicators for Qualitative Research in Special Education

Professional cultures are shaped by and shape practices that are both obvious and explicit as well as those that function obliquely and implicitly (Bourdieu, 1990). Disseminating research is one such practice in education research fields. Publication decisions in special education journals predominantly rely on evaluations of quality and rigor that reflect conceptualizations of qualitative research that are primarily suited to answer descriptive questions, focusing on an identifiable reality, albeit one from multiple perspectives. Research that modifies the subject/object dichotomy of positivism and allows for a critical perspective of reality is known as post-positivism (Guba & Lincoln, 2005). Post-positive qualitative research is characterized by its focus on the truth-value of findings that can be substantiated by objectively established evidence tantamount to reality. For example, in Trainor’s (2005) study of adolescent males and self-determination, the racial/ethnic identities of participants were treated as distinct, observable categories as was participants’ classification as students with learning disabilities. Furthermore, the interpretation of participants’ views of disability and the transition to adulthood were presented as a discoverable truth that the researcher was able to distill from interview data. By post-positivistic standards, this study could be considered rigorous in that Trainor (2005) provides many examples of data that support her thematic analysis, as well as provides extensive data on the methodological decisions made through the course of the study.

Contrastingly, Snelgrove’s (2005) study of stakeholders’ perspectives of inclusive schools critiques the exclusion of adolescents and youth with moderate and severe intellectual disabilities and their construction by researchers and professionals without disabilities as being unreliable informants, unable to contribute their opinions and ideas on inclusive education. Snelgrove’s (2005) work is an example of research that employs constructivism and critical theory, dialoguing with readers about her multiple interpretations of the construction of disability in the context of inclusive education. Furthermore, she defends the rigor of her study by explaining the co-construction of inclusive education with her participants who are considered to have intellectual disabilities and in some cases are nonverbal. The purpose of juxtaposing these two examples is to demonstrate that a multitude of paradigmatic foundations exist and potentially contribute to our knowledge base. Accordingly, criteria for rigor vary across methodologies. Unfortunately, narrow applications of criteria for rigor best suited to a single or predominant paradigm limit the breadth and scope of research, and thus the breadth and scope of knowledge.

Further illustrations from the field and extant literature underscore the need to expand paradigmatic foundations and related quality indicators. First, in the funding of research, IES considers qualitative designs to be appropriate for exploratory and descriptive projects, known also as Goal 1 projects (IES, 2013). Positioning qualitative designs as descriptive relegates qualitative research as supplemental to quantitative designs (e.g., assessing the social validity of an intervention). Second, recent literature reviews examining the evidence base across special education interventions have excluded qualitative studies from consideration, bracketing entire lines of research and their results (see Griffin’s, 2011, review of diverse students’ participation in individual education planning meetings). Third, in disseminating implications of empirical research to practitioners whose primary responsibility it is to consume rather than to conduct research, qualitative research is positioned as being of limited use. For example, in an article targeting practitioners, McDuffie and Scruggs (2008) explain, “They [qualitative studies] are generally intended to describe or explain phenomena and thus logically should not be used to make statements about whether a practice or intervention is evidence based” (p. 92).

These factors likely limit the number of published qualitative studies in certain scholarly areas (Ceglowski, Bacigalupa, & Peck, 2011), including those in the field of special education research. Two reviews of special education publications, Calder, Justen, and Smith (1990) and Mastropieri et al. (2009), show that qualitative research comprises 6% of articles in major special education journals and has not exceeded this amount in the past several decades. And yet, qualitative methods are important tools in special education research and their contribution to the knowledge base has also been recognized (Brantlinger, 1999; Brantlinger, Jiminez, et al., 2005; Brantlinger, Klingner, & Richardson, 2005; Harry, Sturges, & Klingner, 2005; Kliewer, Biklen, & Kasa-Hendrickson, 2006; Pugach, 2001). More broadly, across disciplines, methodological diversity is important to the vitality, relevance, and responsiveness of academic research in solving real-world problems (Eisenhart, 2005, 2006; Lather, 2006; Schwandt, 2012). Such diversity of methods, however, requires a cadre of researchers with adequate breadth of methodological expertise to make evaluative decisions about funding and publication.

While the Brantlinger paper detailing quality indicators for qualitative research (Brantlinger, Jiminez, et al., 2005) has been most instructive, it represents only one perspective on evidence in educational research. In contrast to the homogeneous perspectives on rigor common in the current culture of special education research, qualitative scholars across the social sciences have divergent views of rigor, ranging from the reluctance to set any standards to the acceptance of criteria for rigor that focus on creativity, complexity, and variation (Trainor & Graue, 2012). Similarly, outside of special education, but within the broader field of education research and other social sciences, the purpose of qualitative research extends beyond description to include engaging the voices of those outside academia (e.g., family and youth perspectives), critiquing inequity (e.g., examining racism, ableism, and sexism in educational contexts), and using innovative methods to produce new knowledge about persistent educational problems (e.g., action research studies wherein researchers and stakeholders collaborate to generate and study a research question relevant to their shared goals). Developing expanded criteria for rigor aligns with the breadth and scope of qualitative methodologies and their epistemological foundations and has the potential to address both quality and contribution to the body of special education research.

Expanded Criteria for Quality

Broader criteria for evaluating manuscripts leads us to suggest quality indicators that can be used in the review process. In our recent work, Reviewing Qualitative Research in the Social Sciences (Trainor & Graue, 2012), we asked experts to identify specific indicators of quality within their methodological community. We started by identifying a set of method(ologies) in qualitative research (e.g., action research, case study, discourse analysis, ethnography, interview, mixed methods, narrative), and we invited scholars known for their work in that method to contribute chapters on rigor and quality specific to publication. We asked authors to follow a chapter framework that would simultaneously highlight basic tenets about quality while demonstrating the variation across qualitative methods. This format included key terms with definitions, acknowledgement of controversies and tensions related to diversity in methodological approaches, the rationale for a set of quality indicators, key readings, counterpoint perspectives acknowledged in the field, and a reflection on perspectives on research and on quality evaluations.

In this article, we distill this book-length publication to several broad points that we believe will aid researchers in thoughtfully reviewing qualitative manuscripts submitted for publication, maximizing the contribution of these works to the field while maintaining integrity with the field’s demand for rigor. We acknowledge that these broad points do not represent exhaustive criteria for rigor across qualitative methods. We argue, though, that the three have an elemental and constitutive relationship to paradigm. And, as we argued above, we see paradigm both as foundational to criteria for rigor and, at times, hidden from plain sight. We introduce the following criteria as a way to expand the work of Brantlinger, Jiminez, and colleagues (2005) and to strengthen its connection to quality criteria in the broader field of qualitative research across the social sciences.

The Role of Theory

Researchers should be forthcoming and clear about the roles theory has played in design, implementation, and writing. Regardless of explicit acknowledgment, theory plays a variety of roles in research. Following tradition, theory gleaned from the extant literature informs research questions and purpose as well as design and analysis. In this genre, theory connects research findings to earlier scholarship, resulting in explanations verifiable by examining empirical evidence (Anfara & Mertz, 2006). Others, such as grounded theorists aligned with the early works of Glaser and Strauss (1967), focus on field-based theoretical induction that begins with data collection rather than a close examination of the literature (LeCompte & Schensul, 1999). Others focus on interpretivist, constructivist, and post-modern epistemologies and methodologies that embrace intuition, creativity, and flexibility through diverse relations to theory and patterned empirical data. The approaches often focus on anomalies, outliers, and challenges, often in the context of patterned data (Alvesson & Karreman, 2011). So, while theory exerts a range of roles that influence the design and implementation of research, the key to quality is to note these roles explicitly.

Related to an intentional discussion of theory and its role in a study, researchers should demonstrate internal consistency in the application of theory in design and implementation. As Charmaz (2006) demonstrates through her work on constructivist grounded theory, she departed from traditional grounded theorists’ vision of pure induction by designing and implementing research that explicitly acknowledges how theoretical underpinnings inform all stages of research. In this way, she explicitly acknowledges deduction as an accompaniment to induction. For a traditional grounded theorist, this departure might seem heretical. The task of the reviewer, however, is not to measure practices against her own but to ascertain whether the researcher delivers both an explanation of the role of theory and consistency in its application. A longitudinal study of educational outcomes for two people with significant disabilities (Ryndak, Ward, Alper, Montgomery, & Storch, 2010) illustrates both a purposeful discussion of theory and an alignment of theory to design and implementation that meets our expectation of rigorous qualitative research. In this study, Ryndak and colleagues explain the relationship between inclusive education and longitudinal outcomes, they link theory to methodological approaches for studying cases over time, and they provide a description of data collection and analysis that aligns with both the theoretical positions of their content (e.g., indicators of inclusivity and post-school outcomes such as social networks and employment) and the theoretical positions of their methodology (e.g., multiple data sources and participant perspectives over time). Being clear about the role of theory allows reviewers to see the decisions that researchers made thereby contributing to the overall of the study.

Transparency

The notion of how we know is important in any form of research. In fact, the American Educational Research Association’s (AERA; 2006) recent statement of standards for reporting empirical research highlights the role of knowing in research. Given what might be considered the nonstandard methods used by qualitative researchers, there has been considerable attention paid to the ways that researchers document and share their process (Freeman, DeMarrais, Preissle, Routston, & St. Pierre, 2007). Data collection or analysis strategies can be internally evaluated in terms of whether the procedures make sense or align with the research questions or methodological tradition. But it is the transparency of the researcher’s account that makes or breaks a study.

In many ways we see transparency as an enactment of epistemology—the strategies we use to document and communicate the process of our knowing. Here we suggest three kinds of transparency important to evaluations of quality. The first element, methodological transparency, provides information about the technical—what did the researcher do in terms of design, sample, data collection, and analysis? This is important given methodological diversity in qualitative research but also because it gives the reader a schematic sense of the inquiry. The question of interest about methodological transparency is not Did you do it the way I would? but Does understanding how you did your research provide a foundation for my understanding of the world and your interpretation of it? Some researchers address methodological transparency by providing lists of all codes used in a study. While the lists might provide a small window on the analysis process, it does very little to inform the reader about the analytic rigor or integrity of a study. We think of lists of codes like lists of adjectives in a story—one must understand how they are used to understanding their meaning. Fully detailed descriptions of the analytic process with examples to illustrate the researcher’s thinking are more informative. Knowing what someone did in a research project is critical—however, it is not enough for assessing the quality of research.

The second element, interpretive transparency, is related to the question how did you know? The basic interpretive nature of qualitative research, in which the researcher is a key instrument in the inquiry, makes the mechanisms or process of analysis a foundational part of judging quality. In qualitative research, the question of how you know is as important as what you know. The AERA (2006) standards argue that reports of empirical research should provide information on the logic of inquiry and the path from identification of the general topic, problem definition, design, collection, and analysis of data, to the assertions made as a result of the inquiry. All of these are important so readers can understand the mechanisms that produced the research and open it to public scrutiny.

The final element is narrative transparency, or how does the text of the research create a world? In linear research designs, writing happens toward the end of the study and its focus is reporting the outcomes of the study (Graue, 2006). In qualitative research, the process of writing is a critical part of the analysis process; some argue that writing is a method of inquiry (Richardson & St. Pierre, 2005). Narrative transparency addresses the degree to which the writing of an academic paper illuminates the process of inquiry itself and also the voices and agendas of the participants. Critical to the notion of narrative transparency is the issue of who is speaking. For example, qualitative research texts must be evaluated in terms of whose voice is used and how that voice reflects theoretical, methodological, and epistemological assumptions embedded in a study. If a researcher presents an analysis of the perspectives of fathers of children with severe disabilities living in poverty, the text should provide some direct access to these experiences rather than being filtered through the voice of an omniscient narrator.

Transparency is an interlocking construct that helps readers evaluate the process a researcher uses in inquiry. It illuminates what the researcher did, how he or she knows, and the textual strategies used to tell a researchers story. Harry and colleagues’ (2005) article detailing their grounded theory study of disproportionality in one school district provides a rich example of both methodological and interpretational transparency. In this piece they provide a data analysis map, examples from their coding schemes, and a compelling discussion of the complexities of interpreting and analyzing data from multiple perspectives. Goff, Martin, and Thomas (2007) render an exemplar of narrative transparency in their phenomenological study of African American middle school students and the risks of academic failure. To achieve this, the authors included ample quotes from participants and thick descriptions that, taken as a whole, helped to explain the phenomenon of “the burden of acting white” (p. 134). When richly addressed, transparency can set the stage for better understandings of positionality and reflexivity.

Positionality and Reflexivity

Across paradigms and methods, researchers choose to design and implement studies informed by their personal and professional experiences and identities. Who we are, our identities, contribute to our positions and the vantage points from which we view a research problem. Furthermore, the extent to which a researcher who is a former teacher views teacher interview data sympathetically, or as an insider, is related to the extent to which her teacher-identity functions centrally and in concert with her other identities. Subedi (2006), an American researcher born in Nepal, articulates his multiple identities and how these influences all stages of his research projects both at home and abroad. Yet, positionality is complicated. As Subedi (2006) explains, managing multiple identities during the course of research can be complex and may be intentional or a matter of habit. For example, a bilingual researcher may be conscious or not of which language he or she uses with participants. The use of a second language may be necessary for shared understanding or it may be used to communicate a deeper understanding of a shared culture. In addition, positionality and reflexivity are central to both study implementation and dissemination, related to methodological and interpretive transparency. One criterion for quality is that researchers maintain an appropriate balance when acknowledging their positionality, articulating sufficient detail to reveal relevant personal and professional identities and their significance while avoiding overshadowing the participants’ voices and related data (Fine, Weis, Weseen, & Wong, 2003).

Researchers feature predominantly in qualitative data collection and analysis. Research activities often require relationships among researchers and participants and maintaining distance and objectivity is neither effective nor desirable. Identities are fluid and multiple, complicating the ways in which researchers’ positions are linked to ways in which they represent participants (Guba & Lincoln, 2005). Reflexivity is judicious consideration for one’s positions, as informed by one’s identities, and the effects of position on the process of doing research (Pillow, 2003). Reviewers should first ensure reflexivity is addressed, and then again look for congruency between an authors’ reflexivity and his or her stated methodology. For example, when employing a critical methodology such as feminist theory to research design and implementation, a reviewer should expect the researcher to articulate central issues of power in relation to his or her own identity while addressing the ethics of research. Trainor and Bouchard (2012) do this by addressing the ethics of power in school- and family-based research in special education through an examination of their positions as former teachers, articulating specific junctures where both analyses and dissemination were shaped by reflexive research practices, explicitly connecting their work to feminist theory. Such reflexivity increases or substantiates the validity of qualitative research, for which quantitative tests of validity are not applicable (Lather, 1986; Maxwell, 1992; Pillow, 2003).

To address the tension between subjectivity and validity, Brantlinger, Jiminez, and colleagues (2005) closely adhered to qualitative methods texts that presented measures of validity, allowing researchers’ views to be visible in design and implementation. Audit trails, as well as member checking and data triangulation, are seen as ways to increase the verification, and therefore the credibility of qualitative research from a post-positivistic view (Creswell & Miller, 2000). Reviewers should understand, however, that research conducted within constructivist and other paradigms may focus on reflexivity and relational aspects of the research activities (e.g., peer debriefing, action, and collaboration with participants) to establish a work’s credibility, one definition of validity in the context of qualitative research (Creswell, 2013; Lather, 1986; Maxwell, 1992). The range of paradigmatic vantage points represented in qualitative research demands a variety of strategies for ensuing quality and rigor, and these link to positionality and reflexivity. Reviewers need to be aware of multiple paradigms and methods and judge the quality of the research using criteria that aligns with the paradigmatic foundation of the work. Hui-Michael and Garcia’s (2009) study of teachers’ perceptions of Asian American students in the context of special education referrals provides an example of reflexivity that contributes to the rigor of the overall research. The authors articulate their positionality and biases, connecting these to both the methods they used to collect and analyze data, as well as acknowledging ways in which this impacted the credibility of their work. Their articulation of reflexivity also demonstrates balance and does not overshadow their participants’ voices.

Is Quality Relative?

We have argued that understanding quality in qualitative research requires researchers and reviewers alike to employ criteria for rigor, and to understand that differing paradigms may invoke variation in these criteria. Some readers may wonder, “Is quality relative?” and the question itself may cause uneasiness. Others may be more aligned with Smith and Hodkinson (2005), and might agree with the acknowledgment that we are in “an era of relativism” (p. 915). Smith and Hodkinson (2005) recognize such a claim is unpalatable to those who see relativity as a threat to order and an endorsement to unending argumentation that results in an “anything goes” approach to knowledge generation through research. While we are not pure relativists, we agree criteria for “good research” are socially constructed and are not free from political implications.

Judging the quality of research requires reviewers and consumers alike to understand how researchers justify their methodological decision-making and the steps they pursue as they implement research and disseminate the results. We agree with Schwandt (1996, 2012) and Lather (2006) that multiple paradigms and their offspring methodologies and methods are necessary for making progress toward solving complex social science problems. Such pluralism in the generation of knowledge requires communication about those paradigmatic differences for the purpose of valuing the multiplicity and complexity of social science knowledge. Expanding the criteria for rigor is one way to prudently expand the potential contributions of qualitative research particularly in a field that grapples with enduring problems and affronts to effective and equitable education.

Special education has a long history of being multidisciplinary; embracing this multiplicity and complexity requires adaptation of criteria for rigor common in other social sciences. The expanded criteria we present—acknowledging the roles of theory, attending to transparencies, and pursuing reflexivity—complement the criteria for rigor already established in the field of special education. We hope the criteria we have suggested present specific instances for reviewers to consider as they evaluate the rigor of research. With this consideration, we are confident special education research using qualitative methods will play a more prominent role in providing answers to our field’s most challenging questions.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Alvesson

Karreman

(2011). Qualitative research and theory development. Los Angeles, CA: SAGE.

American Educational Research Association. (2006). Standards for reporting on empirical social science research in AERA publications. Educational Researcher, 35(6), 33–40. doi: 10.3102/0013189X035006033

Anfara

V. A.

Jr. Mertz

N. T.

(2006). Theoretical frameworks in qualitative research. Thousand Oaks, CA: SAGE.

Bourdieu

(1990). The logic of practice ( Nice

Trans.). Stanford, CA: Stanford University Press.

Brantlinger

E. A.

(1999). Inward gaze and activism as moral next steps in inquiry. Anthropology & Education Quarterly, 30, 413–429. doi:10.1525/aeq.1999.30.4.413

Brantlinger

E. A.

Jiminez

R. T.

Klingner

Pugach

Richardson

(2005). Qualitative studies in special education. Exceptional Children, 71, 195–207.

Brantlinger

E. A.

Klingner

J. K.

Richardson

(2005). Importance of experimental as well as empirical qualitative studies in special education. Mental Retardation, 43, 92–119. doi:10.1357/0047-6765

Calder

J. E.

Justen

J. E.

Smith

D. L.

(1990). Qualitative and quantitative research trends in special education journals. Teacher Education and Special Education, 13, 172–174. doi:10.1177/088840649001300306

Ceglowski

Bacigalupa

Peck

(2011). Aced out: Censorship of qualitative research in the age of “scientifically based research.” Qualitative Inquiry, 17, 679–686. doi:10.1177/1077800411415497

10.

Chard

D. J.

Ketterlin-Geller

L. R.

Baker

S. K.

Doabler

Apichatabutra

(2009). Repeated reading interventions for students with learning disabilities: Status of the evidence. Exceptional Children, 75, 263–281.

11.

Charmaz

(2006). Constructing grounded theory: A practical guide through qualitative analysis. Thousand Oaks, CA: SAGE.

12.

Condliffe Lagemann

(2000). An elusive science: The troubling history of education research. Chicago, IL: University of Chicago Press.

13.

Cook

B. G.

Tankersley

Landrum

T. J.

(2009). Determining evidence-based practices in special education. Exceptional Children, 75, 365–383.

14.

Creswell

J. W.

(2013). Qualitative inquiry and research design: Choosing among five traditions (3rd ed.). Thousand Oaks, CA: SAGE.

15.

Creswell

J. W.

Miller

D. L.

(2000). Determining validity in qualitative inquiry. Theory Into Practice, 39, 124–130. doi:10.1207/s15430421tip3903_2

16.

Denzin

N. K.

(2005). Emancipatory discourses and the ethics and politics of interpretation. In Denzin

N. K.

Lincoln

Y. S.

(Eds.), The SAGE handbook of qualitative research (3rd ed., pp. 933–958). Thousand Oaks, CA: SAGE.

17.

Eisenhart

(2005). Hammers and saws for the improvement of educational research. Educational Theory, 55, 245–261.

18.

Eisenhart

(2006). Qualitative science in experimental time. International Journal of Qualitative Studies in Education, 19, 697–707. doi:10.1080/09518390600975826

19.

Eisenhart

Towne

(2003). Contestation and change in national policy on “scientifically-based” education research. Educational Researcher, 32(7), 31–38. doi:10.3102/0013189X032007031

20.

Fine

Weis

Weseen

Wong

(2003). For whom? Qualitative research, representations, and social responsibilities. In Denzin

N. K.

Lincoln

Y. S.

(Eds.), The landscape of qualitative research (pp. 167–207). Thousand Oaks, CA: SAGE.

21.

Freeman

DeMarrais

Preissle

Roulston

St. Pierre

E. A.

(2007). Standards of evidence in qualitative research: An incitement to discourse. Educational Researcher, 36(1), 25–32. doi:10.3102/0013189X06298009

22.

Gersten

Fuchs

L. S.

Compton

Coyne

Greenwood

Innocenti

M. S.

(2005). Quality indicators for group experimental and quasi-experimental research in special education. Exceptional Children, 71, 149–164.

23.

Giangreco

M. F.

Taylor

S. J.

(2003). “Scientifically based research” and qualitative inquiry. Research and Practice for Persons with Severe Disabilities, 28, 133–137.

24.

Glaser

Strauss

A. L.

(1967). The discovery of grounded theory. Chicago, IL: Aldine.

25.

Goff

Martin

J. E.

Thomas

M. K.

(2007). The burden of acting White: Implications for transitions. Career Development and Exceptional Individuals, 30, 134–146.

26.

Graue

(2006). Writing in education research. In Conrad

C. F.

Serlin

R. C.

(Eds.), The SAGE handbook for research in education: Engaging ideas and enriching inquiry (pp. 515–528). Thousand Oaks, CA: SAGE.

27.

Griffin

M. M.

(2011). Promoting IEP participation: Effects of interventions, considerations for CLD students. Career Development and Exceptional Individuals, 34, 153–164. doi:10.1177/0885728811410561

28.

Guba

E. G.

Lincoln

Y. S.

(2005). Paradigmatic controversies, contradictions, and emerging confluences. In Denzin

N. K.

Lincoln

Y. S.

(Eds.), The SAGE handbook of qualitative research (3rd ed., pp. 191–215). Thousand Oaks, CA: SAGE. doi:10.1002/ev.1427

29.

Harry

Sturges

K. M.

Klingner

J. K.

(2005). Mapping the process: An exemplar of process and challenge in grounded theory analysis. Educational Researcher, 34(2), 3–13. doi:10.3102/0013189X034002003

30.

Horner

R. H.

Carr

E. G.

Halle

J. W.

McGee

Odom

S. L.

Wolery

(2005). The use of single-subject research to identify evidence-based practices in special education. Exceptional Children, 71, 165–179.

31.

Hui-Michael

Garcia

S. B.

(2009). General educators’ perceptions and attributions about Asian American students: Implications for special education referral. Multiple Voices for Ethnically Diverse Exceptional Learners, 12, 21–37.

32.

Kliewer

Biklen

Kasa-Hendrickson

(2006). Who may be literate? Disability and resistance to the cultural denial of competence. American Educational Research Journal, 43, 163–192. doi:10.3102/00028312043002163

33.

Labaree

D. F.

(2004). The trouble with ed schools. New Haven, CT: Yale University Press.

34.

Lather

(1986). Issues of validity in openly ideological research: Between a rock and a soft place. Interchange, 17(4), 63–84. doi:10.1007/bf01807017

35.

Lather

(2004). Scientific research in education: A critical perspective. British Educational Research Journal, 30, 760–773. doi:10.1080/0141192042000279486

36.

Lather

(2006). Paradigm proliferation as a good thing to think with: Teaching research in education as a wild profusion. International Journal of Qualitative Studies in Education, 19, 35–57. doi:10.1080/09518390500450144

37.

LeCompte

M. D.

Schensul

J. J.

(1999). Analyzing & interpreting ethnographic data. Walnut Creek, CA: AltaMira Press.

38.

Lincoln

Y. S.

(1995). Emerging criteria for quality in qualitative interpretive research. Qualitative Inquiry, 1, 275–289.

39.

Lincoln

Y. S.

Guba

E. G.

(1986). But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation. New Directions for Program Evaluation, 30, 73–84.

40.

Mastropieri

M. A.

Berkeley

McDuffie

K. A.

Graff

Marshak

Conners

N. A.

. . .Cuenca-Sanchez

(2009). What is published in the field of special education? An analysis of 11 prominent journals. Exceptional Children, 76, 95–109.

41.

Maxwell

J. A.

(1992). Understanding validity in qualitative research. Harvard Educational Review, 62, 279–300.

42.

Maxwell

J. A.

(2004). Causal explanation, qualitative research, and scientific inquiry in education. Educational Researcher, 33(2), 3–11. doi:10.3102/0013189X033002003

43.

McDuffie

K. A.

Scruggs

T. E.

(2008). The contributions of qualitative research to discussions of evidence-based practice in special education. Intervention in School and Clinic, 44, 91–97. doi:10.1177/1053451208321564

44.

National Research Council. (2002). Scientific research in education. Washington, DC: National Academy Press.

45.

National Research Council. (2005). Advancing scientific research in education. Washington, DC: Author.

46.

Odom

S. L.

Brantlinger

Gersten

Horner

R. H.

Thompson

Harris

(2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71, 137–148.

47.

Pillow

(2003). Confession, catharsis, or cure? Rethinking the uses of reflexivity as methodological power in qualitative research. International Journal of Qualitative Studies in Education, 16, 175–196. doi:10.1080/0951839032000060635

48.

Pugach

M. C.

(2001). The stories we choose to tell: Fulfilling the promise of qualitative research for special education. Exceptional Children, 67, 439–453.

49.

Richardson

St. Pierre

E. A.

(2005). Writing: A method of inquiry. In Denzin

N. K.

Lincoln

Y. S.

(Eds.), The SAGE Handbook of qualitative research (3rd ed., pp. 959–978). Thousand Oaks, CA: SAGE.

50.

Ryndak

D. L.

Ward

Alper

Montgomery

J. W.

Storch

J. F.

(2010). Long-term outcomes of services for two persons with significant disabilities with differing educational experiences: A qualitative consideration of the impact of educational experiences. Focus on Autism and Other Developmental Disabilities, 45, 323–338.

51.

Schwandt

T. A.

(1996). Farewell to criteriology. Qualitative Inquiry, 2, 58–72. doi:10.1177/107780049600200109

52.

Schwandt

T. A.

(2005). A diagnostic reading of scientifically based research for education. Educational Theory, 55, 284–305. doi:10.1111/j.1741-5446.2005.00004.x

53.

Schwandt

T. A.

(2012). Valuing methodological diversity. Qualitative Social Work, 11, 125–129. doi:10.1177/1473325011433928a

54.

Slavin

R. E.

(2002). Evidence-based education policies: Transforming educational practice and research. Educational Researcher, 31(7), 15–21. doi:10.3102/0013189X031007015

55.

Slavin

R. E.

(2003). A reader’s guide to scientifically based research. Educational Leadership, 60(5), 12–16.

56.

Smith

J. K.

Deemer

D. K.

(2000). The problem of criteria in the age of relativism. In Denzin

N. K.

Lincoln

Y. S.

(Eds.), Handbook of qualitative research (2nd ed., pp. 877–896). Thousand Oaks, CA: SAGE.

57.

Smith

J. K.

Hodkinson

(2005). Relativism, criteria, and politics. In Denzin

N. K.

Lincoln

Y. S.

(Eds.), Handbook of qualitative research (3rd ed., pp. 915–932). Thousand Oaks, CA: SAGE.

58.

Snelgrove

(2005). Bad, mad, and sad: Developing a methodology of inclusion and a pedagogy for researching students with intellectual disabilities. International Journal of Disability Development and Education, 9, 313–319.

59.

Subedi

(2006). Theorizing a “halfie” researcher’s identity in transnational fieldwork. International Journal of Qualitative Studies in Education, 19, 573–593. doi:10.1080/09518390600886353

60.

Thompson

Diamond

K. E.

McWilliam

Snyder

S. W.

(2005). Evaluating the quality of evidence from correlational research for evidence-based practice. Exceptional Children, 71, 181–194.

61.

Trainor

A. A.

(2005). Self-determination perceptions and behaviors of diverse students with LD during the transition planning process. Journal of Learning Disabilities, 38, 233–249.

62.

Trainor

A. A.

Bal

(2014). Development and preliminary analysis of a rubric for culturally responsive research. The Journal of Special Education, 47, 203–216. doi:10.1177/0022466912436397

63.

Trainor

A. A.

Bouchard

K. A.

(2012). Exploring and developing reciprocity in research design. International Journal of Qualitative Studies in Education, 26, 986–1003. doi:10.1080/09518398.2012.724467

64.

Trainor

A. A.

Graue

(2012). Reviewing qualitative research in the social sciences. New York, NY: Routledge.