Deeper than Wordplay: A Systematic Review of Critical Quantitative Approaches in Education Research (2007

Abstract

Although the critical research cannon is often associated with qualitative scholars, there is a growing number of critical scholars who are refusing positivist-informed quantitative analyses. However, as a growing number of education scholars engaged in critical approaches to quantitative inquiry, instances of conflation began to surface. We understood this conflation as the interchangeable use of the terms quantitative criticalism, QuantCrit, and critical quantitative throughout the literature and even within the same chapter or article. The purpose of our systematic literature review is twofold: (a) to understand how critical approaches to quantitative inquiry emerged as a new paradigm within quantitative methods and (b) whether there is any distinction between quantitative criticalism, QuantCrit, and critical quantitative inquiries or simply interchangeable wordplay. We share how critical quantitative approaches are definite shifts within the quantitative research paradigm, highlight relevant assumptions, and share strategies and future directions for applied practice in this emergent field.

Keywords

critical theory race social justice multicultural education mathematics education equity research methodology statistics critical quantitative CritQuant QuantCrit quantitative criticalism systematic literature review

Francis Galton, Karl Pearson, and Ronald Fisher were eugenicists. They developed statistics as a scientific discipline in the modern social sciences to discover universal laws about human differences that they believed would be as indisputable as scientific laws such as Newton’s laws of motion (Clayton, 2020; Louca, 2009; Zuberi, 2001; Zuberi & Bonilla-Silva, 2008). Aligned with positivism, these white European men drew inspiration and techniques from the natural sciences (i.e., astronomy and earth sciences) and favored quantitative analysis to repeatedly observe, test, verify, and corroborate their theories that many revered as facts (Zuberi & Bonilla-Silva, 2008). However, statistics are not facts; they are human-generated estimations based on uncertainty (Zuberi, 2001). Those who engage in critical approaches to quantitative inquiry understand that “there is not a set logic within the methods themselves” (Zuberi & Bonilla-Silva, 2008, p. 8). Still, statistics reflect the logic; theories; accepted practice; and consensus of society, practitioners, and researchers (Tabron & Thomas, in press). That is, statistics reflect theories of society and “these men adhered to the unquestioned white supremacy of their time” (Omi & Winant, 2014, p. 5). Although these founders of modern social statistics espoused to be theory free (positivism), their racial theories were value-laden attempts to advance white supremacy and prove that human differences reflect the natural order (Toldson, 2019; Zuberi, 2001; Zuberi & Bonilla-Silva, 2008).

Although the critical research canon is often associated with qualitative scholars, there has always been and continues to be a growing number of critical scholars who are refusing positivist-informed quantitative analyses (Garcia et al., 2018; Gillborn et al., 2018; Omi & Winant, 2014; Smith, 1999; Stage, 2007a; Stage & Wells, 2014; Walter & Andersen, 2016; Wells & Stage, 2015; Wilson, 2008; Zuberi, 2001; Zuberi & Bonilla-Silva, 2008). Although critical approaches to quantitative inquiry for social justice builds on the freedom struggle and foundational intellectual contributions of Black scholars and other scholars of Color (Omi & Winant, 2014; Smith, 1999; Zuberi, 2001; Zuberi & Bonilla-Silva, 2008) that predates Stage’s (2007a) editorial, their editorial introduced the terms critical quantitative, quantitative criticalist, and quantitative criticalism as frames of reference for education scholars who challenge quantitative research framed within the positivist tradition.

Critical Quantitative and Quantitative Criticalism

Stage’s foregrounding of criticality began with the Frankfurt school of critical philosophy and extended toward Kincheloe and Peter McLaren’s (1994) work on rethinking critical theory and qualitative inquiry. Stage (2007a) based their definition of critical on Kincheloe and McLaren’s (1994) assumptions about critical researchers that included:

(a) Thought is mediated by socially and historically created power relations; (b) facts cannot be isolated from values; (c) the relationship between concept and object is never fixed and is often socially mediated; (d) language is central to the formation of subjectivity; (e) certain groups in society hold privilege over others that is maintained if subordinates accept their status as natural; (f) oppression has many faces that must be examined simultaneously; and (g) mainstream research practices reproduce class, race, and gender oppression. (p. 7)

For Stage (2007a), these seven assumptions did not prevent the use of quantitative methods to critically interrogate issues in education. Likewise, in the first special issue on critical approaches to quantitative inquiry (Stage, 2007a), the contributing authors united around three methodological commitments with a heavy focus on establishing the legitimacy of and encouraging more scholars to engage in critical epistemologies within quantitative inquiry (Baez, 2007; Carter & Hurtado, 2007; Kinzie, 2007; Perna, 2007; St. John, 2007; Teranishi, 2007). These two methodological commitments (“tasks”) were:

1) Use data to represent educational processes and outcomes on a large scale to reveal inequities and to identify social or institutional perpetuation of systematic inequities in such processes and outcomes. (Stage, 2007a, p. 10)

2) Question the models, measures, and analytic processes of quantitative research in order to offer competing models, measures, and analytic practices that better describe the experiences of those who have not been adequately represented. (Stage, 2007a, p. 10)

Stage and Wells (2014) offered a third task in the second special issue, which was to “conduct culturally relevant research by studying institutions and people in context” (p. 3).

Within the second special issue (Stage and Wells, 2014), there was a growing number of higher education scholars “who embraced the term quantitative criticalist” (p. 3), and the goal was to continue to raise awareness of past “critical quantitative work” and encourage future research in this area (Alcantar, 2014; Conway, 2014; John & Stage, 2014; Metcalf, 2014; Oseguera & Hwang, 2014; Rios-Aguilar, 2014; Williams, 2014). At the end of the second special issue, Cecilia Rios-Aguilar (2014) synthesized the scholarship presented in Stage and Wells (2014) by challenging quantitative criticalists to engage in methodological self-reflection as they are designing and conducting research. These tasks included:

1) Ask relevant questions (about equity and power).

2) Choose relevant data.

3) Apply appropriate, rigorous, and sophisticated analyses.

4) Disaggregate analysis on gender, race/ethnicity, language proficiency, socioeconomic status, and conduct research on several groups of marginalized students (e.g., students who have had contact with the juvenile justice system, indigenous students, and students with disabilities).

5) Know how to interpret results (i.e., pay attention to statistical and educational significance).

6) Employ challenging and enriching theories in multiple disciplines.

7) Inform and challenge existing institutional practices and decisions.

8) Inform and challenge existing educational policies.

In the third special issue (Wells & Stage, 2015), Hernández (2015) offered three more challenges to “shift toward a focus on grappling with paradigmatic concerns that require attention in order to ensure rigor and appropriate training for the application of quantitative criticalism” (p. 94). These three challenges are:

1) Quantitative criticalism challenges normative assumptions and research practices in “quantitative research.”

2) Quantitative criticalism requires a high level of expertise in both statistical analyses and critical theory.

3) Quantitative criticalism requires the use of a set of critical theoretical tenets to ensure legitimacy and rigor.

Contributing authors in Stage (2007a), Stage and Wells (2014), and Wells and Stage (2015) special issues identified as quantitative criticalists who engaged in critical quantitative inquiry (CritQuant). They understood critical quantitative inquiry as grounding their diverse critical theoretical perspectives in these shared methodological commitments (tasks). The authors collectively aimed to provide the theoretical backing that justified critical quantitative research as definite “paradigm shift” and legitimate approach in quantitative inquiry (Stage & Wells, 2014, p. 1), a legitimacy exclusively attached to positivist and post-positivist research paradigms.

Emergent Conflation

However, as a growing number of education scholars engaged in critical approaches to quantitative inquiry, instances of conflation began to surface. We understood this conflation as the interchangeable wordplay of quantitative criticalism, QuantCrit, and critical quantitative within the same chapter or article (Garvey et al., 2017; Rios-Aguilar, 2014; Rockenbach & Mayhew, 2013; Wells & Stage, 2015) and wondered to what extent this is simply interchangeable wordplay or distinct perspectives and techniques. Given this interchangeable use of the words quantitative criticalism, QuantCrit, and critical quantitative, we decided to conduct a rigorous systematic review of the existing scholarship to highlight the “distinction[s] between the different levels of analysis” connected to each approach and illustrate how understanding these distinctions can promote the “adequate location of phenomena” for future scholarship (Alexander, 2020; Kabatek, 2015, p. 213). The purpose of our systematic literature review is twofold: (a) to understand how critical approaches to quantitative inquiry emerged as a new paradigm within quantitative methods and (b) whether there is any distinction between quantitative criticalism, QuantCrit, and critical quantitative inquiries or simply interchangeable wordplay. We share how critical quantitative approaches are definite shifts within the quantitative research paradigm, highlight relevant assumptions, and share strategies and future directions for applied practice in this emergent field.

Positionality

It is vital to share how we locate ourselves within this work. As Black women engaged in quantitative research, the historical antecedents of our criticality predated Stage (2007a), the Frankfurt School, and Kincheloe and McLaren’s (1994) scholarship on criticality in qualitative research (Tabron et al., 2021). Although these scholars provided the starting point for Stage’s (2007a, 2007b) definition of critical quantitative research, it was not possible for us to do the same. The Frankfurt school’s criticality is rooted in economic determinism and tracing historical materialism for liberal class consciousness among the white working class in Europe and then America (Mills, 2003; Palmer, 1993).

We located ourselves within a centuries-long resistance to the eugenic origins of quantitative research and racialized statistics on Black, Indigenous, and people of Color (BIPOC) (Zuberi & Bonilla-Silva, 2008; Omi & Winant, 2014). Critical approaches to quantitative inquiry are not new to our communities who have always been distrustful of statistical research trying to “prove” the intellectual and cultural inferiority of communities of color and obscure racism (Tabron, 2019). Our journey of locating ourselves within this work is based on our inherited role as scholars who push back against quantitative research that reflects and protects white supremacy (Crawford, 2019; Stage, 2014; Tabron, 2019; Tabron et al., 2021; Zuberi, 2001; Zuberi & Bonilla-Silva, 2008, Wells & Stage, 2015). Our positionality is present in every phase of this review to advance social equity and racial justice in quantitative research.

Method

Systematic Literature Search

We conducted this systematic literature review to synthesize educational research on critical approaches to quantitative inquiry between 2007–2021. In Figure 1, we present our flow chart that reflects our screening process. We began phase one with clearly defined questions to guide the review of literature, inclusion and exclusion criteria, and search terms that returned our first pool of articles screened. During phase two we completed our title and abstract screening and inter-rater reliability check, and in phase three we discussed our full-text coding procedures.

Figure 1.

Systematic review flow chart. From: PRISMA 2020 flow diagram template for systematic reviews (Page et al., 2021; adapted from flow diagrams proposed by Boers and Mayo-Wilson et al. and Stovold et al.).

Phase One: Begin with Clearly Defined Research Questions

Phase one of our systematic review of literature began with two questions that guided our review to unpack conflation between critical quantitative, quantitative criticalism, and QuantCrit educational research:

Research Question 1: How did critical quantitative inquiry emerge as a new paradigm within quantitative methods?

Research Question 2: What is the difference between critical quantitative inquiry (CritQuant), quantitative criticalism, and QuantCrit?

We formulated both research questions to understand whether these terms were interchangeable or conceptually different and to understand trends within this body of scholarship. In the next section, we share our inclusion and exclusion criteria and show replicable parameters for future reviews on critical approaches to quantitative inquiry.

Search Criteria and Terms

We narrowed our scope to books, dissertations, and peer-reviewed articles published between 2007–2021. We selected the 2007 publication year to correspond to Frances Stage’s (2007a) first special issue on critical approaches to quantitative inquiry. We included conceptual and empirical studies with an education focus along the P-20 pipeline whose authors applied critical perspectives to quantitative research. Next, we used the search terms “critical quantitative” OR “quantitative criticalism” OR CritQuant OR QuantCrit OR “critical quantitative inquiry” OR “critical race theory of statistics” OR “quantitative criticalist” OR “crit* and quant* OR Crit* race theory” and Statistics within Academic Search Complete (n = 50), Social Science Premium (n = 128), ERIC (n = 17), and Education Collection (n = 70). These searches returned a total of 265 studies that we stored within the RefWorks data management software (ProQuest, 2020). After this first pull, we cross-checked all databases for duplicates (n = 132). This left 133 articles ready for phase two abstract screening.

Phase Two: Abstract Screening

We independently screened 133 initial studies using a predetermined coding scheme stored within our shared RefWorks database. We carefully read through each study’s title and abstract and coded “yes” (forward to full-text screen) or “no” (does not meet inclusion and exclusion criteria). If we marked “no” for the study, we added a numerical code explaining why we excluded it. We coded studies “no-1” if it was a duplicate not caught in the database crosscheck, coded “no-2” if the study did not have an education focus, coded “no-3” if the study did not have a focus on critical quantitative inquiry, and coded “no-4” if the study was not an empirical or conceptual study. Whenever a coding structure was difficult to figure out by the abstract, we scanned the full text to decide the appropriate code. After we completed abstract coding, we calculated inter-rater reliability estimates to measure the consistency of our coding structures. Out of the 133 articles, author one coded (n = 99) articles “no” and author two coded (n = 109) articles “no,” yielding a 91% IRR agreement on the articles that we did not include for full-text review.

Upon this result, we discussed our coding application process. We discovered that the first author coded “yes” if the article met the inclusion criteria and the researchers asked critical research questions or used critical theoretical perspectives to guide their quantitative research designs and challenged majoritarian narratives and assumptions. The second author qualified studies focused on challenging traditional quantitative assumptions, methods, and measures. We eventually decided to include all studies coded “yes” from either author. We excluded 97 articles from our abstract screening pool as follows: (n = 3) previously undetected duplicates, (n = 58) noneducation focus, (n = 23) studies where researchers did not apply critical approaches to quantitative inquiry, and (n = 13) nonempirical studies. This left (n = 36) articles for full-text screening.

Phase Three: Full Text Screen

We carefully read all 36 articles independently, conducted open coding, and aggregated our codes to form emergent themes from the data using NVivo v.12 (QSR International, 2020). Through our forward and backward reference search we found 27 articles not captured in our first search, bringing our final pool to the 63 full-text articles for review. During each check-in, we engaged in a dialogic decision-making process (Kinloch & San Pedro, 2014) about our codes and emergent trends within our data and kept a research journal to capture our learnings shared within our discussion section.

Phase Four: Included Study Characteristics

Our sample included 63 articles (35 peer-reviewed manuscripts, 24 book chapters, and 4 dissertations). There were 19 conceptual studies and 44 empirical studies. Most of the articles (76%, n = 48) reviewed had a higher education focus. At the time of this review, there were 12 articles (19%) that had a P–12 focus, which shows that QuantCrit and CritQuant approaches are slowly beginning to appear in P–12 education. There were three articles (5%) that had a general education focus but not a K–12 or higher education distinction. As seen in Figure 2, our review included articles published between 2007–2021. Most of the articles (60% or n = 38) were published between 2015–2021. The years with the highest frequency correspond with years where there were special issues on QuantCrit and CritQuant approaches.

Figure 2.

Frequency count of QuantCrit and CritQuant articles published between 2007–2021.

Special Issues

Almost half of articles in our sample (n = 29, 46%) were by invitation through special issues. Frances Stage edited and published the first special issue (n = 8), Using Quantitative Data to Answer Critical Questions, in New Directions for Institutional Research in 2007. In this first issue, Stage (2007a) provided examples of how scholars can conduct critical research using quantitative methods. Seven years later, Frances Stage and Ryan Wells (2014) published the second special issue (n = 8), New Scholarship in Critical Quantitative Research-Part 1: Studying Institutions and People in Context, in New Directions for Institutional Research. This issue addressed critiques of critical quantitative research “provided an expanded conceptualization of the tasks” and highlighted “the need to conduct culturally relevant research by studying institutions and people in context” (Stage & Wells, 2014, p. 1). In 2015, Ryan Wells and Frances Stage edited a third special issue (n = 7), New Scholarship in Critical Quantitative Research-Part 2: New Populations, Approaches, and Challenges, in New Directions for Institutional Research. This issue provided a current pulse of developments and areas of expansion in critical approaches to quantitative inquiry.

The fourth special issue (n = 6), QuantCrit: Rectifying Quantitative Methods through Critical Race Theory, was edited by Nichole Garcia, Nancy López, and Verónica N. Vélez in 2018 in the journal Race Ethnicity and Education. Here, scholars applied critical race theory (CRT) as a theoretical or methodological framework to disrupt structural racism in quantitative research. The year 2019 marked the highest number of studies published outside of a special issue (n = 10). When we examined these 10 studies, half of the researchers (n = 5 studies) referenced Garcia et al.’s (2018) special issue and grounded their quantitative inquiry in CRT. These four special issues were among the first to publish critical approaches to quantitative inquiry. Outside of these four special issues, the number of publications dropped dramatically when viewed annually. This could be a signal of potential challenges, and pushback scholars might face getting their work accepted in journals without a critical focus (Hernández, 2015). The lower publication numbers year to year outside of special issues made us curious about the potential gatekeeping mechanisms that might be slowing the dissemination and momentum of critical approaches to quantitative research.

Journals and Journal Focus

In Figure 3, most articles (23 articles or 35%) were from the journal New Directions for Institutional Research. We reviewed seven articles, or 11%, from the journal of Race, Ethnicity and Education. Research in Higher Education, and Journal of College Student Development published four, or 6%, of the articles reviewed in this study. Based on their impact factor, 29 (or 45%) of the manuscripts we reviewed were published in (Q1) top tier journals, 3 articles (5%) were published in mid-tier journals (Q2), 1 article (2%) was published in a tier-three journal (Q3). Twenty-seven (42%) of the manuscripts were published in outlets that did not have a published impact factor, and the remaining four manuscripts were dissertations.

Figure 3.

Journals for manuscripts included in sample.

In our review, 27 journals covered general education topics (i.e., American Educational Research Journal [AERJ]), 10 journals had a higher education focus, 1 journal had a middle school focus, 1 journal focused on science, and another emphasized gifted education. Four journals focused on race (Latinx, n = 5; Black, n = 2), intersectionality (n = 1), ethnic studies (n = 7), or urban education (n = 2). Most scholars found homes for their critical quantitative work through special issues or in journals known for publishing critical work with an emphasis on historically marginalized communities.

CritQuant, Quantitative Criticalism, QuantCrit Author Classifications

Although all the manuscripts had critical approaches to quantitative inquiry, there were inconsistencies in how authors positioned their work. As seen in the Supplementary Appendix (available in the online version of this article), Tabular Listing of Data Sources Analyzed in Systematic Review, the authors in 40 studies (63%) labeled their work as CritQuant, 11 (17%) labeled their work as QuantCrit, 2 (3%) labeled their work as quantitative criticalism, 8 labeled their work (13%) as something other than these terms (i.e., trans QuantCrit), and the authors of 2 studies (3%) did not label their work although it aligned with the assumptions of either QuantCrit or CritQuant.

In our initial search of the literature, most scholars began their exploration with one or more of the following works: (a) Frances Stage and Ryan Wells’s three-part series in New Direction for Institutional Research on critical quantitative research (Stage, 2007a; Stage & Wells, 2014); (b) Tukufu Zuberi’s (2001) book, Thicker than Blood: How Racial Statistics Lie; (c) Tukufu Zuberi and Eduardo Bonilla-Silva’s (2008) edited book, White Logic, White Methods Racism and Methodology; (d) Alejandro Covarrubias (2011) and Alejandro Covarrubias and Verónica Vélez’s (2013) work on critical race quantitative intersectionality; (e) Nichole M. Garcia, Nancy López, and Verónica N. Vélez’s (2018) QuantCrit special issue in Race, Ethnicity and Education. Most of the authors (n = 44, 70%) used Stage (2007a, 2007b), Stage & Wells (2014), and Wells and Stage’s (2015) definitional framing of critical approaches to quantitative inquiry as their starting point. Other scholars cited Zuberi (2001) and Zuberi and Bonilla-Silva (2008) (n = 16, 25%); Covarrubias (2011) or Covarrubias and Vélez’s (2013) (n = 15, 24%); Garcia, López, and Vélez’s (2018) (n = 8, 13%); Gillborn et al.’s (2018) (n = 13, 20%); Du Bois (1899) (n = 3, 5%); or Kimberle Crenshaw’s (2018, 1991) intersectionality theory (n = 5, 8%) as a frame of reference for their work. The remaining articles in our sample did not reference these foundational texts in their studies.

Results

In this section, we present the results that appeared from our codified data and thematic analyses—theme 1: paradigm shift and theme 2: critical quantitative (CritQuant) and quantitative critical race (QuantCrit) are deeper than wordplay.

Paradigm Shift

Critical quantitative inquiry denotes the first stages of a “paradigm shift” (Stage, 2014, p. 1) away from positivism and post-positivism in quantitative inquiry. Our first theme, paradigm shift, emerged from the subthemes: resistance to normal science, the racialization of educational crises, problem solving, and critical approaches to quantitative research are viable ways of knowing.

Resistance to Normal Science

Normal science is a shared understanding among scholars about the thought patterns (logic), theories, methods, and standards for what constitutes legitimate contributions to a field (Kuhn, 1962). Scholars who practice normal science answer quantitative research questions in ways that reify the positivist paradigm (Cook & Campbell, 1979; Kuhn, 1962). Historically, what counted as quantitative research was based on positivist assumptions (objectivity, universal truth, value free) to test theories to reveal fundamental truths about people (Clayton, 2020). Kuhn (1962) defined a paradigm shift as resistance to normal science, crisis identification, problem solving, and emergence of an alternative paradigm for crisis resolution. Critical approaches to quantitative inquiry are a resistance to normal science grounded in the positivist and post-positivist paradigms.

The Racialization of Educational Crises

Critical approaches to quantitative inquiry shed light on the longstanding racialization of social issues within quantitative research and the perpetuation of the biological, cultural, and intellectual superiority of one group over others (Garcia et al., 2018; Zuberi, 2001; Zuberi & Bonilla-Silva, 2008). Scholars engaged in critical approaches to quantitative inquiry tracked the different crises in educational leadership and policies brought on by the flawed application and interpretation of quantitative research, underused and misused datasets, faulty statistical interpretations, gap gazing and exclusion of diversity themes. They called out federal and state agencies using critical policy questions (Perna, 2007; St. John, 2007) to speak useful truths to power about the mishandling of crises faced by students within the P–20 pipeline based on specific agendas (Alcantar, 2014; Allen & Wolniak, 2019; Campbell-Montalvo, 2020; Covarrubias, 2011; Garcia et al., 2018; Malcolm-Piqueux, 2015; Solórzano & Yosso, 2001; Tabron & Ramlackhan, 2019; Teranishi, 2007; Wells, 2010). Several authors highlighted the tendency for users of national and state surveys to produce misleading social categorizations and “carelessly” use data sets (Wells, 2010, p. 1688).

Positivist researchers often racialized educational crises using data from local, state, and federal agencies to model and test deficit theories and produce deficit interpretations (i.e., fixed capabilities, low performers, below average) about students of color. Those engaged in critical approaches to quantitative inquiry have named this as gap-gazing research (Gutiérrez, 2008; Young et al., 2018a, 2018b) that is used to frame the discourse through a deficit lens and undervalue the educational aspirations and attainment among underrepresented and marginalized groups, particularly in science, technology, engineering, and math (STEM) (Van Dusen & Nissen, 2020; Metcalf, 2011; Young & Young, 2018; Young et al., 2018b). Scholars engaged in critical approaches to quantitative inquiry redirected the racialization of crises toward the racist policies, practices, and discourses that reproduce inequities (Gillborn et al., 2018; Kendi, 2019; Zuberi, 2001).

For example, scholars in our sample found that many of the earlier quantitative analyses did not portray the complex yet nuanced factors shaping the experiences of Black, Indigenous, people of Color (BIPOC) communities. Campbell-Montalvo (2020) illuminated the practice of “racial reformation” among school personnel in data reporting. This “racial reformation included school personnel changing the race/ethnicity reported on teacher-created grade-level cards and presenting racial/ethnic categories inconsistently across different school measures. Campbell-Montalvo (2019) argued how this “racial reformation” contributes to the erasure of Indigenous students in data reporting and prevents the inclusion of diverse race(s) of Latinx youth in school, local, state, and federal reporting, which could have grave political, financial, and legal implications for this historically marginalized populations (p. 181).

Teranishi (2007) illuminated the exclusion and misrepresentation of Asian Americans in educational research and policy through collapsing subgroups within the population. Teranishi (2007) critiqued between-group comparisons that masks differences within ethnic groups and “often promote and reinforce racialized assumptions that are held about Asian Americans based on studies that aggregate the population into a single racial category so they can be compared to other racial groups” (Teranishi, 2007, p. 47). Rios-Aguilar (2014) critiqued dominant perspectives that Latina/o students were not civically engaged by critiquing traditional and normative models and definitions of engagement. These models often do not consider engagement within sociohistorical and policy contexts. She emphasizes social civic behaviors such as translating for the community, tutoring, and mentoring youth and other forms of engagement connected to their ethnic and/or immigrant identities (Rios-Aguilar, 2014, p. 3). Similarly, Faircloth and colleagues (2015) critiqued the construction of large-scale data sets that do not capture the full linguistic and cultural diversity of American Indian and Alaskan Native students, lack culturally relevant variables, combine distinct cultural groups (American Indian and Alaska Native) and group them as “other” due to small sample sizes, or erase them completely from the data. Young and colleagues (2018b) critiqued the gap gazing that masks the high achievement of Black girls in math and creates missed opportunities to “promote growth by citing and affirming strengths as a means to build capacity and promote success” (p. 170).

Taken together, researchers engaged in critical approaches to quantitative inquiry centered and contextualized the experiences and perspectives of BIPOC students. They expanded the discourse by sharing asset-based narratives of how historically marginalized students excel and redefine success and engagement (Bielby et al., 2014; Gonzalez Canche & Rios-Aguilar, 2015; Covarrubias et al., 2018; Curley, 2019; Faircloth et al., 2015; Garcia et al., 2018; Garvey, 2019; Kilgo et al., 2019; Rios-Aguilar, 2015; Tabron & Ramlackhan, 2019; Vaccaro et al., 2015; Wells, 2010).

Problem-Solving

Problem-solving activities within paradigms are the “universally recognized scientific achievements which provide model problems and solutions for a community of practitioners” (Kuhn, 1977, p. viii). Through methodological innovation researchers engaged in critical quantitative approaches solved some of the weak points connected to positivist research (Thomas, 2022). Alternative, intersectional, inclusive, and participatory frameworks addressed the problematic framing of research questions and hypotheses (Williams, 2014). Scholars engaged in critical approaches to quantitative inquiry amended statistical formulae to address the intersectional, additive, and multiplicative effects of racialized and marginalized students’ identities (Mayhew & Simonoff, 2015a, 2015b; Wells, 2010). The frequency in which researchers linked race to disparate academic outcomes led scholars to stop the application of race as a fixed explanatory variable (Mayhew & Simonoff, 2015a, 2015b; Wells, 2010; Zuberi & Bonilla-Silva, 2008).

Researchers used efficient data management techniques to troubleshoot the perpetuation of deficit racial formation in big data (Campbell-Montalvo, 2020; Oseguera & Hwang, 2014). This helped stymie the lumping of students into monolithic racial categories (Covarrubias et al., 2018; Teranishi, 2007). The intersectionality of several marginalized identities became visible in aggregate achievement data (Covarrubias, 2011; Covarrubias et al., 2018; Rockenbach et al., 2015; Vaccaro et al., 2015). Classical quantitative methods such as logistic regression (Garcia & Mayorga, 2018), multiple regression (Jang, 2018; Saunders, 2015; Wells, 2010), fixed effects modeling (Allen & Wolniak, 2019), and history analyses (Zerquera & Gross, 2017), as well as education opportunity models (Lopez et al., 2018), scale development, and validation procedures were also updated (Knowles & Hawkman, 2019).

Critical Approaches to Quantitative Inquiry are Viable Ways of Knowing

Critical approaches to quantitative inquiry are defined by their own: axiologies, ontologies, epistemologies, and methodologies for conducting research (Stage, 2014). Within our sample, for more than two decades (2007–2021) critical researchers have moved beyond criticality and disruption and toward alternative ways of knowing and doing quantitative methods. We created Table 1 for tracing the paradigmatic issues related to critical approaches to quantitative inquiry in educational research. Lincoln, Lynham, and Guba’s (2018) work on the paradigmatic differences between qualitative and quantitative research was instrumental for developing our understanding of critical approaches to quantitative inquiry as a paradigm shift within the quantitative research. In the first column we list axiology, epistemology, ontology, and methodology as the knowledge-producing systems used to define research paradigms. In the second and third columns we specify the tenets that classify positivism and post-positivism as normalized science. In the last column we describe the critical quantitative researchers’ worldviews that guided critical approaches to quantitative inquiry. This table provides evidence supportive of critical approaches to quantitative inquiry as a viable paradigm shift for social justice–oriented quantitative researchers.

Table 1

Showing three philosophical orientations for producing knowledge using quantitative research.

Knowledge-producing system	Philosophical worldviews
Knowledge-producing system	Positivism	Post-positivism	Critical approaches to quantitative inquiry
Axiology: The values and ethical principles informing research practices.	My personal experiences and values do not influence my use of objectively created and interpreted quantitative methods. Data speaks for itself. Researchers drive decisions about participants’ consent and participation in research. Researchers derive primary benefit from research.	My personal experiences and values influence what I study and the methods I use to study it. Participants have the right to make their own decisions about consent and participation in research. Participants should benefit from research; however, they have no influence over the distribution of these benefits.	My personal experiences and values can be theorized, parameterized, and generalized. My research is grounded in my values of equity, freedom, and social justice. Participants have the right to make their own decisions about consent and participation in research. The primary beneficiaries of the research should be the participants/community.
Epistemology: How is knowledge known? The relationship between the researcher and those being researched.	There is a universal truth discoverable through the scientific method. I position myself as the knower and my participants are the subjects of careful and systematic investigations. I distance myself from subjects.	There is a degree of uncertainty that I try to carefully control in modeling; the information I produce about participants is a probable truth.	Participants are knowers/experts and knowledge is co-constructed with them directly and/or indirectly but controlled through systems of power. I am connected to my participants because we often share positionalities and lived experiences.
Ontology: The nature of reality in research. What are the underlying dynamics of human existence that I need to explain in my research?	Reality is knowable through ongoing testing of theory, through the scientific process, and by eliminating rival hypotheses. Social reality can be reliably and validly measured once you control for systematic error.	Reality is knowable based on the probabilities of occurrence. Reality is observed through the background knowledge of researchers. Reality should be contextualized. Social reality can be reliably and validly measured once you control for systematic error.	The tensions, contradictions and inconsistencies of deeply embedded inequality and racial and other hegemonic hierarchies shape the realities and lived experiences of marginalized and racially minoritized groups.
Methodology: The rules explaining use of the methods and research procedures. What are the most useful rules guiding my selection and use of research methods?	I use quantitative methods to generate evidence to support or refute my hypotheses. Results are ahistorical and often decontextualized.	The measurement of social reality is inherently fallible. I should triangulate different measures to get the best representation of reality.	Researchers should innovate new measures to capture the intersectional and multiplicative social realities imposed on marginalized and racially minoritized groups. Researchers should engage interdisciplinary perspectives, approaches, and methodologies. Results must be contextualized.

Critical Quantitative (CritQuant) and QuantCrit are Deeper than Wordplay

Within theme two, critical quantitative (CritQuant) and quantitative critical race theory (QuantCrit) are deeper than wordplay, we share the results of our comparative analysis of important similarities and differences between CritQuant and QuantCrit. This theme is based on the supportive evidence within our three subthemes: analogical reasoning, clearing up conflation, and applied practice-strategies used.

Analogical Reasoning

As we analyzed and coded the studies in our sample by their critical theoretical perspectives, we found uses of the terms critical quantitative, quantitative criticalism, and QuantCrit. Many scholars (n = 44 or 70%) used Stage (2007a, 2007b), Stage and Wells (2014), or Wells and Stage (2015) as anchor texts for the framing of their critical approaches to quantitative inquiry. This group of authors (quantitative criticalists) classified their work as critical quantitative or quantitative criticalism because of their foregrounding of diverse critical theoretical epistemologies. Some of the theories included critical race theory (Hernández, 2015; Sullivan et al., 2010; Teranishi, 2007); multicontextual model for diverse learning environments (Alcantar, 2014); college access framework (Oseguera & Hwang, 2014); QueerQuant (Garvey et al., 2019); critical queer theory (Metcalf, 2011), trans QuantCrit (Curley, 2019), and socialization of achievement (Young et al., 2018b). For these researchers, critical quantitative and quantitative criticalism was used interchangeably to indicate that critical epistemological, ontological, and methodological beliefs guided their inquiry.

Quantitative critical race theory (QuantCrit)

Critical race theory appeared as the most recurring theoretical framework in our sample (Covarrubias et al., 2018; Crawford, 2019; Garcia et al., 2018; Gillborn et al., 2018; Knowles & Hawkman, 2019; Sablan, 2019). Certainly, the critical analysis of race in quantitative research predates our bounded time parameters, 2007–2021 (see Du Bois, 1899; Zuberi, 2001; Solórzano et al., 2005). However, in our sample, we traced the use of CRT in quantitative studies to the published work Teranishi (2007) (disaggregating achievement and demographic data for Asian American students in U.S. higher education institutions). Additionally, there were thirteen studies (20%) that referenced Gillborn et al. (2018) as an anchor text for the framing of their study. For these thirteen studies, QuantCrit was not shorthand for quantitative criticalism but reflected a parallel movement of scholars whose epistemological genealogy is rooted in critical race theory that originated in critical legal studies (Covarrubias et al., 2018; Crawford, 2019; Garcia et al., 2018; Gillborn et al., 2018; Knowles & Hawkman, 2019; Pérez Huber et al., 2018; Sablan, 2019). These scholars’ definition of critical branch off from the line of critical theory that originated in the German Frankfurt school ((Stage, 2007a, 2007b; Sablan, 2019). For these scholars, the tenets of critical race theory guide their inquiry as they make more explicit connections to race and its intersections in their quantitative inquiries (Garcia et al., 2018; Gillborn et al., 2018; Sablan, 2019).

Gillborn et al. (2018) distinguishes this practice by adopting the label QuantCrit (the combining of CRT and quantitative methods) as shorthand for the approach and framing it through five early principles to guide researchers’ future use and analysis. The following five principles guide quantitative critical race theory, which are dynamic and in their infancy:

1) “The centrality of racism” (Gillborn et al., 2018, p. 170) means that the pervasiveness of racism in the United States and Britain is central to analyses and interpretations. QuantCrit researchers understand that quantitative analysis is not independent of race and racism.

2) “Numbers are not neutral” (Gillborn et al., 2018, p. 170) is a rejection of the objectivity of numbers. Researchers have used statistics to protect white supremacy ideologies. When researchers present the numbers as neutral, it implies that researchers are independent from the data. It also ignores the extent researchers use and construct deficit theories that normalize racial inequities as “true or legitimate” reflections of the natural order.

3) “Categories/groups are neither ‘natural’ nor given” (Gillborn et al., 2018, p. 170) means that socially constructed categories are reflective of unequal power dynamics within social relationships. These categories are not naturally forming or fixed variables.

4) “Data cannot ‘speak for itself’” (Gillborn et al., 2018, p. 170) means that quantitative data is interpretative like all other research and thus open to conflicting interpretations.

5) Apply a “social justice/equity orientation” (Gillborn et al., 2018, p. 170) to use statistical research for liberation and equality.

Gillborn and colleagues (2018) do not consider these principles exhaustive but a starting point for researchers who want to resist perpetuating racist ideologies and its intersections.

Clearing Up Conflation

Critical approaches to quantitative inquiry are a direct challenge to quantitative research framed within the positivist paradigm (Garcia et al., 2018; Gillborn et al., 2018; Stage, 2007a, 2007b; Stage & Wells, 2014; Wells & Stage, 2015). However, these terms are not interchangeable. The term quantitative critical race theory (QuantCrit) first appears in the special issue QuantCrit: Rectifying Quantitative Methods through Critical Race Theory, edited by Nichole Garcia, Nancy López, and Verónica N. Vélez in 2018 in the journal Race Ethnicity and Education. The abbreviation QuantCrit refers to scholarship guided by the tenets of critical race theory (Gillborn et al., 2018). The terms Critical Quantitative and Quantitative Criticalism emerged from Stage’s (2007a) first special issue. In all three issues, scholars interchangeably used the terminology critical quantitative inquiry and quantitative criticalism to refer to the ways in which critical epistemologies, axiologies, and ontologies guided their research process. Quantitative criticalist refers to scholars who engaged in critical quantitative work (Stage, 2007a, 2007b; Wells & Stage, 2015).

To be clear, before there was QuantCrit (Garcia et al., 2018; Gillborn et al., 2018), some of the contributing scholars in Stage’s (2007a, 2007b) special issue classified their work as CritQuant or quantitative criticalism and used critical race theory to guide their inquiry (Hernández, 2015; Teranishi, 2007). Because the scholars in our sample have diverse critical perspectives, by name alone, conflation of what is meant by QuantCrit and CritQuant emerged within our review. To that end, we concur with Hernández (2015), in the necessity that “scholars clearly define what ‘critical’ means to them, what they are critical about, and how their definition has informed their work” (p. 99). That is, it is essential that scholars critically interrogate how they have come to understand the notion of criticality and explicitly describe how critical epistemologies guide their inquiry.

Applied Praxis: Strategies Used

Whether a researcher classified their work as critical quantitative or QuantCrit, there were eleven common strategies used to reveal and disrupt the systemic inequities experienced by historically marginalized groups, which we discuss next.

For us by us (FUBU)

One of the first patterns that emerged from our coding was that many critical scholars engaged in this work identify as members of historically marginalized communities who are reclaiming and constructing new narratives of themselves and their communities (Covarrubias et al., 2018; Pérez Huber et al., 2018; Rios-Aguilar, 2014; Tabron, 2019; Tabron et al., 2020, 2021; Teranishi, 2007; Young et al., 2018a, 2018b; Zuberi, 2001). Many researchers who have felt excluded, misrepresented, or dehumanized as a statistic positioned themselves within the critical quantitative paradigm. This lineage of “for us, by us” (Du Bois, 1899) evolved into “nothing about us without us” in the Disability Rights movement (Inckle, 2015) and later the trans community (Curley, 2019; Nicolazzo, 2017). Voices that have been historically situated in the margins are centered. The continuum of quantitative criticality, particularly for scholars of color, is a form of resistance to what has been considered the property rights of white scholars (Tabron et al., 2020).

Present strength narratives of historically marginalized communities

Within this critical quantitative research, scholars co-construct strengths-based counternarratives to resist white supremacy in quantitative inquiry (Garcia & Mayorga, 2018; Pérez Huber et al., 2018; Jang, 2018; Williams, 2014; Young et al., 2018b; Young & Young, 2018). They are focused on the achievement of Black girls (Young et al., 2018b; Young & Young, 2018), refuting homogeneity in Latinx scholarship by contributing as Chica@ scholars (Pérez Huber et al., 2018) and including narratives from and for the LGBTQ+ communities (Kilgo et al., 2019). Alcantar (2014) challenged traditional measures of civic engagement for Latina/o college students and shared how undocumented and resident Latina/os engaged in affirming behaviors that are not captured in the national data sets. Ryan Wells (2010) reframed the educational success of children of immigrants from assimilation using evidence on the absence of institutional mechanisms that impede students’ educational goals and aspirations. Similarly, Zerquera and Gross (2017) disputed the deficit narrative that Latina/o access and success is limited due to the individual’s background rather than policies, practices, and programs (i.e., cost of attendance, lack of a critical mass of faculty of color, or financial aid allocation) that do not meet this community’s needs. Presenting the strength narratives of historically marginalized communities is a prominent strategy used to refute racialized statistics, disrupt deficit narratives, and racial generalizations.

Ground truthing the data

Ground truthing is an interdisciplinary method most used in atmospheric science (meteorology) to confirm information at the site from data gathered remotely through machines. Ground truthing the data has translated in educational research as the practice of ensuring that:

each step of the research process is driven by community expertise, particularly when the research is attempting to understanding [sic] phenomena connected to race and racism. Numbers can offer vital insights, highlight patterns, and convey particular analyses, but when they are decontextualized, ahistorical, and disconnected from the everyday lives of People of Color, they are hypothetical at best. The numbers must be groundtruthed in experiential knowledge. (Pérez Huber, Vélez, and Solórzano, 2018, p. 212)

Scholars who use groundtruthing are building on the intellectual legacy of W. E. B. Du Bois’s groundbreaking conceptualization and visualizations of the socially constructed color line that paved the way for geographic information system (GIS) mapping and big data analytics (Battle-Baptiste & Rusert, 2018; Du Bois, 1889; Toldson, 2019). For example, scholars such as Daniel Solórzano and Veronica Vélez (2015) were among the first to advance this work through their critical race spatial analysis to critically analyze sociospatial mechanisms “that reflect and operate as tools of white supremacy” (p. 426). Overall, education researchers within our sample used groundtruthing to co-construct counternarratives that uplift communities’ strengths (Solórzano & Vélez, 2015). Groundtruthing is a strategy in which the expertise of the communities informs the entire research process and gives the data its meaning (Du Bois, 1899; Pérez Huber et al., 2018; Solórzano & Vélez, 2015).

Critique social categorizations

Another strategy is critiquing the causal interpretations of social categories and outcomes in surveys (Garcia & Mayorga, 2018; Jang, 2018; Zuberi, 2001). Scholars within our sample have critiqued the construction of assumed and fixed categories that reproduce hegemonic privilege and marginalization (i.e., sexual and gender identities) (Garvey, 2019; Metcalf, 2011). The scholarship in this area has resulted in the new and more inclusive definitions for measures (i.e., student engagement, civic engagement, gender identity) and were not treated as standalone entities but intersectional experiences of BIPOC students (Castellanos and Cole, 2015; Covarrubias et al., 2018; Garvey et al., 2019; Hernandez et al., 2013).

Reject the oversimplification of aggregate data

Single group sampling and analysis to avoid homogenizing racial groups was another strategy (Conway, 2014; Teranishi, 2007; Vaccaro et al., 2015; Zerquera & Gross, 2017). Some between-group comparisons perpetuate gap gazing by situating white students as the comparison group for all other groups. This deflects deeper inquiry from the policies and practices that perpetuate deficit stereotypes (Young et al., 2018b). Robert Teranishi (2007) wrote about this issue with Asian American students in educational policy. Jemimah Young and colleagues (2018b) tracked these dynamics and its relation to math achievement and Black girls. Cindy Kilgo and colleagues (2019) and Garvey (2019) wrote about the masking of LQBTQ+ communities in aggregate data. Katherine Conway (2014) highlighted the importance of not aggregating children of immigrants in with the broader race/ethnic group.

Typically, single group effects in education research are decomposed using t-tests and analysis of variance (ANOVA). Both tests are dependent on the extent group differences affect individual differences. This is implicated in producing inaccurate generalizations about marginalized groups and whole communities. Instead, the single within-group analytic techniques frames variation based on students’ lived experiences and not between-group analyses, which often portrays BIPOC students as falling behind white students as the standard of measurement.

Build multidimensional statistical models

Researchers designed new, multidimensional models to replace widely accepted statistical and economic models that perpetuate inaccurate results, for historically marginalized groups (Stage, 2007a, 2007b; Perna, 2007; Wells, 2010). Complex constructs warrant the use of dynamic and multidimensional statistical models and “finer analyses for various subpopulations” (Wells, 2010, p. 1688) that better describe students’ experiences not adequately represented in statistical tests and results.

Critically interrogate measure validation

Enacting cultural and equity-minded approaches to validating surveys and not norming or validating their instrument among predominantly white male student populations was another strategy used (Alcantar, 2014; Hernandez et al., 2013; Knowles & Hawkman, 2019; Park et al., 2017; Kilgo et al., 2019; Teranishi, 2007; Malcolm-Piqueux, 2015; Covarrubias et al., 2018). Data sets that reinforced deficit and gap-dependent perspectives (Alcantar, 2014, p. 31) should be reinterpreted and reanalyzed using updated statistical approaches. These are sources of measurement error often overlooked in quantitative research. Alcantar offered recommendations that researchers engaged in critical approaches to quantitative inquiry should: (a) critically examine past models and studies in higher education and question the inclusive or exclusive nature of the models, research questions, and measures; (b) expand currently existing measures to capture nontraditional student activities; and (c) develop new models that include racial/ethnic identities and sociohistorical and policy contexts.

Ending the checkbox culture in survey design and administration

Researchers also tracked changes to federal and state data-gathering policies over time and how racially marginalized communities are characterized within large data sets (Allen & Wolniak, 2019; Wells, 2010). Scholars developed strategies for overcoming the checkbox culture of survey design that subsumed students’ identities under monolithic categories (Stewart, 2013; Jang, 2018) and excluded LGBTQIA+ (Curley, 2019; Kilgo et al., 2019; Garvey et al., 2019) and Latinx and students with immigrant backgrounds (Wells, 2010).

Presenting a numerically broad picture

These scholars also showed that quantitative researchers should use a variety of statistical analyses to produce multiple lines of evidence for estimating the accuracy of statistical outcomes and interpreted results for diverse groups (Allen & Wolniak, 2019; Wells, 2010; Kilgo et al., 2019; Mayhew & Simonoff, 2015a, 2015b; Ro & Bergom, 2020; Van Dusen & Nissen, 2020). Van Dusen ands Nissen (2020) showed the extent to which using different measures of the same construct can produce different yet conflicting outcomes.

Advanced heterogeneity of variance

Critical quantitative researchers accept heterogeneity of variance as an important assumption when measuring differences between groups (Alcantar, 2014; Allen & Wolniak, 2019; Wells, 2010). These researchers took a stand against homogeneity as a longstanding assumption of traditional quantitative research by addressing the “tendency to view heterogeneity of variance as a methodological nuisance rather than a source of important information” (Bryk & Raudenbush, 1987, p. 402). Diversity is inherently heterogeneous. Therefore, students do not need to have the same qualities for comparability. These researchers developed novel ways of modeling the contextual, structural, situational, and historical variability connected to students’ lived experiences.

Mayhew and Simonoff (2015a, 2015b) showed mastery of the theoretical linkages between one-way ANOVA and multiple group regression while challenging the within-group homogeneity assumption. Mayhew and Simonoff (2015a, 2015b) applied effect coding to model group membership based upon multiple racial identities rather than modeling each racial identity separately. This study illustrated the power of critical quantitative approaches for framing race as a multifaceted categorical variable by merging algebra and statistical analyses with the logic of CRT. Ro and Bergom (2020) updated Mayhew and Simonoff’s work to show how researchers can avoid using dummy coding that positions white students as a normative reference group for racially minoritized students. Effect coding proved useful for comparing minoritized students’ perceived social distance between themselves and white students.

Model specification and estimation

Regression analyses were often used to build and test models that deconstruct individual and interaction effects to estimate BIPOC student outcomes. The scholars in our study addressed the interaction effects of social categorizations based upon sex and gender (Van Dusen & Nissen, 2020; Garvey et al., 2019; Lopez et al., 2018; Mayhew & Simonoff, 2015a, 2015b; Metcalf, 2011; Park et al., 2017; Wells, 2010). Race/ethnicity appeared as a key area of consideration for Jang (2018), particularly for modeling the “multiple marginalized social categorizations” (p. 1270) of Cambodian, Vietnamese, Hmong, Thai, and Laotian Southeast Asian female students. Zerquera and Gross (2017) modeled the sociohistorical and policy context of Latinx students’ immigrant backgrounds using event history analysis to predict the timing and occurrence of enrollment.

Wells (2010) and Van Dusen and Nissen (2020) used hierarchical linear modeling (HLM) to deconstruct the multilevel effects of racialized inequity. Wells (2010) applied HLM to estimate the effects of school composition on immigrant students’ expectations. Cultural relatedness variables were included in the model to deconstruct Latinx students’ educational expectations. Van Dusen and Nissen (2020) included sexism instead of gender and racism instead of race to their HLM model to capture the marginalized experiences of female and BIPOC students. Van Dusen and Nissen (2020) operationalized academic achievement using the Equity of Individuality and the equality of learning measures and modeled the individual and interaction effects of relevant independent variables. the equity of individuality measure predicted larger learning gains for BIPOC students whereas student achievement gaps persisted within the results produced by the equality of learning measure. Based on these activities, researchers have creatively countered some of the most long-standing critiques directed toward traditional model building and estimation. They critically examined the misinterpretation of regression coefficients as one of “the most frequent error(s) of interpretation” (Judd et al., 2017, p. 151) and highlighted the predictor variables that should be included when modeling factors relevant to BIPOC students’ lived experiences (Williams, 2014).

As presented in our results, some of the scholars foreground critical perspectives and other methods in their work. Scholars who foreground diverse critical theoretical epistemologies had the strong theoretical grounding and understanding of critical theories (Hernández, 2015) to introduce new critical quantitative research questions and designs to achieve larger equity goals. Scholars who foregrounded methods had the advanced quantitative training needed to flip their positivist methodological training toward critical, deconstructionist, and transformative praxis. This led to different forms of methodological innovation and alternative pathways for conducting quantitative analyses (Best & Byrd, 2014; Sablan, 2019). To continue advancing this work, we concur with Hernández (2015) and Sablan (2019), who argued that scholars engaged in critical approaches to quantitative inquiry need both. Scholars engaged in critical approaches to quantitative inquiry should have advanced training and expertise in both critical theoretical perspectives and statistics to continue to push this emerging field forward. This work can begin by critically interrogating and more deeply understanding how instructors currently train and socialize graduate students in graduate-level statistics courses in education (Thomas, 2022).

Discussion

In this review, we shared why critical approaches to quantitative inquiry is a paradigm shift within quantitative research and a consolidation of a “centuries-long conflict between white domination and resistance by people of color” (Omi & Winant, 2014, p. 3). We highlighted the contributions of scholars who have engaged in CritQuant and QuantCrit between 2007 to 2021 to show areas for further development of the field. We identified and discussed distinctions between quantitative criticalism, QuantCrit, and critical quantitative, which are not interchangeable wordplay, and we presented several recommendations for applied praxis within critical education research. A continuum of criticality undergirded the entire research process of critical approaches to quantitative research. All the scholars engaged in critical approaches to quantitative inquiry in our sample consider theory and method inextricably linked to every phase of the research design process. All scholars engaged in this work focused on critical epistemological, ontological, axiological, and methodological orientations and methods in quantitative inquiry.

Embracing Heterogenous Orientations and Methodological Approaches

As scholars are locating themselves within critical approaches to quantitative inquiry, the epistemological roots of the researchers’ criticality differ and lend them to different theoretical perspectives that shapes their entire research process. This has resulted in multiple parallel yet complementary movements such as scholars whose criticality is rooted in the tenets of critical race theory (QuantCrit) (Garcia et al., 2018; Gillborn et al., 2018). There are also scholars engaged in quantitative inquiry informed by critical queer theory (Garvey et al., 2019; Kilgo et al., 2019). There is also an emerging scholarship that centers the trans community through Trans Quant Crit (Curley, 2019). Indigenous scholars have made valuable contributions to critical approaches to quantitative inquiry that have provided new insights into what it means to decolonize research methodologies (Kukutai & Taylor, 2016; Smith, 1999; Walter & Andersen, 2013; Walter & Suina, 2019; Wilson, 2008). Scholars engaged in critical approaches to quantitative inquiry are united in their resistance to positivist-informed quantitative inquiry that centers Western colonial thought as superior, but they may differ in how they engage in critical work based on how they have authentically located themselves within the work by understanding the roots of their criticality. Embracing multiple ways of knowing, being, doing, and competing perspectives aligns with critical approaches to quantitative inquiry and is another form of resistance to the positivist tradition that seeks to uncover probable and universal truth claims (Tabron & Thomas, in press).

Common Practices

There were several common practices throughout the studies in our review that we recommend in future studies. First, all scholarship began with asking critical questions and linked those questions to a critical theoretical perspective. Second, scholars explicitly and implicitly engaged in critical self-reflexivity of their positionalities and added critical and racial consciousness to their research. Third, scholars revealed how systemic racism and other forms of oppression remain undetected and perpetuated in statistical inquiry. Fourth, scholars critiqued and changed reference categories, deracialized statistics, and focused on context. Fifth, they introduced convergent historicism (Tabron et al., 2021), which focused on distinctiveness rather than generality by adding meaning, capturing complexity, and situating the data within its historical, cultural, and political contexts. Sixth, the scholarship led to creation of new and more inclusive variables and covariates in the statistical models. Seventh, all the studies in our sample centered the experiences of historically marginalized communities through strengths-based lenses.

What Are Not Critical Approaches to Quantitative Inquiry?

Although it is important to name the synergies within critical approaches to quantitative research, it is also critical to clarify what would not be considered QuantCrit or CritQuant based on the themes that appeared from the data. QuantCrit or CritQuant is not a label or buzzword for performative allyship. It is not QuantCrit or CritQuant if the eugenic history of statistics is not acknowledged and disrupted (Covarrubias et al., 2018; Gillborn et al., 2018; Tabron et al., 2020; Zuberi, 2001; Zuberi & Bonilla-Silva, 2008). Focusing on a racial or historically marginalized group while white supremacist logic is still perfectly intact is not QuantCrit or CritQuant (Tabron et al., 2020; Zuberi & Bonilla-Silva, 2008). Thoughtlessly accepting a secondary data set or big data without critical interrogation is not QuantCrit or CritQuant (Faircloth et al., 2015; Garcia & Mayorga, 2018; Rios-Aguilar, 2015). Conducting research about historically marginalized groups without being a member of the community or engaging their expertise through the literature or collaboration is not QuantCrit or CritQuant (Pérez Huber et al., 2018; Toldson, 2019). QuantCrit and CritQuant is not about finding or creating diverse ways to prove the superiority or inferiority of a group; it is about honoring the rich cultural histories and alternative ways of knowing, being, and doing. It is building upon those gifts to improve the conditions of historically marginalized communities (Teranishi, 2007; Toldson, 2019).

Conclusion

Critical approaches to quantitative inquiry are transformative because they are radically reflexive and relationally transparent (Anthym & Tuitt, 2019). It reflects a more inclusive spectrum of approaches that departs from positivist praxis based on natural selection or the survival of the fittest doctrines that informed early statistical practice. At times, resistant structures, groups, and people may push back against the transformative praxis of critical change agents (Anthym & Tuitt, 2019; Hernández, 2015). However, Baez (2007) invites us to ask, “How research in higher education is or can be critically transformative—that is to what extent educational research can offer critiques of our world that allow us to transform it” (p. 18). An essential charge for scholars who want to engage in critical approaches to quantitative inquiry to deeply reflect upon their positionality and how it shapes their research design and statistical interpretations (Covarubbias et al., 2018; Pérez Huber et al., 2018). They also challenge notions of researcher objectivity and statistics as numeric truth (Crawford, 2019; Garcia & Mayorga, 2018; Jang, 2018).

Despite framing some aspects of our emergent findings using Thomas Kuhn’s structure of scientific revolutions (1962), we do not view paradigmatic research as the end-all and be-all of quantitative knowledge. We learned that to effectively engage in quantitative inquiry from a critical quantitative perspective, there must be active work to develop a cogent understanding of ourselves outside of whiteness and its systems and structures. We needed to disrupt our tendencies to gaze at quantitative research through the established status quo. We also needed to question the extent our activities reify who should be centered and who should be at the margins while conducting quantitative research. As more scholars take part in critical approaches to quantitative research, they will expand on our work and create more approaches for strengthening this paradigm shift within quantitative research.

Finally, this systematic review ties in with our earlier scholarship using critical quantitative approaches to actualize an emancipatory praxis within introductory graduate statistics courses (Tabron et al., 2021; Thomas, 2022). Completing this review was helpful in showing us how much our own work needs to evolve. We infuse hooks’ (1994) statement as our definitional frame of reference for our future scholarship and praxis:

The academy is not paradise. But learning [quantitative methods] is a place where paradise can be created [for faculty, practitioners, and students]. The [statistics] classroom, with all its limitations, remains a location of possibility. In that field of possibility, we have the opportunity to labor for freedom, to demand of ourselves and our comrades, an openness of mind and heart that allows us to face reality even as we collectively imagine ways to [engage quantitative methods] to move beyond boundaries, to transgress. This is [quantitative methods] as the practice of freedom. (p. 207)

We hope that our work serves as a catalyst for deeper acknowledgement of critical quantitative research approaches as legitimate pathways for thinking about and conducting quantitative research.

Footnotes

Acknowledgements

We would like to thank the anonymous reviewers for taking the time to review our manuscript. We appreciate their valuable comments and suggestions, which strengthened our manuscript.

ORCID iD

Lolita A. Tabron

Authors

LOLITA A. TABRON is an associate professor in the Department of Educational Leadership and Policy Studies at the University of Denver, 1999 East Evans Avenue, Katherine A. Ruffatto Hall 361, Denver, CO 80208; e-mail: lolita.tabron@du.edu. She critically examines how public policies, leadership practice, and statistical data can positively transform educational systems or reproduce systemic racism and educational inequalities.

AMANDA K. THOMAS is a research assistant professor at the Department of Research and Innovation at the University of the Southern Caribbean, P.O. Box 175, Port of Spain, Trinidad, West Indies]; e-mail: thomasa@usc.edu.tt. Her research interests include transformational educational praxis and theoretical and applied qualitative, quantitative, mixed, and evaluation research methods.

References

Alcantar

C. M.

(2014). Civic engagement measures for Latina/o college students. New Directions for Institutional Research, 158, 23–35. https://doi.org/10.1002/ir.20043

Alexander

P. A.

(2020). Methodological guidance paper: The art and science of quality systematic reviews. Review of Educational Research, 90(1), 6–23. https://doi.org/10.3102/0034654319854352

Allen

Wolniak

G. C.

(2019). Exploring the effects of tuition increase on racial/ethnic diversity at public colleges and universities. Research in Higher Education, 60, 18–43. https://doi.org/10.1007/s11162-018-9502-6

Anthym

Tuitt

(2019). When the levees break: The cost of vicarious trauma, microaggressions and emotional labor for Black administrators and faculty engaging in race work at traditionally White institutions. International Journal of Qualitative Studies in Education, 32(9), 1072–1093. https://doi.org/10.1080/09518398.2019.1645907

Baez

(2007). Thinking critically about the “critical”: Quantitative research as social critique. New Directions for Institutional Research, 2007(133), 17–23.

Battle-Baptiste

Rusert

(Eds.). (2018). WEB Du Bois’s data portraits: Visualizing black America. Chronicle Books.

Best

L. E.

Byrd

W. C.

(2014). Methods beyond methods: A model for Africana graduate methods training. The Journal for Pan African Studies, 7(1), 61–73. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5915346/

Bielby

Posselt

J. R.

Jaquette

Bastedo

M. N.

(2014). Why are women underrepresented in elite colleges and universities? A non-linear decomposition analysis. Research in Higher Education, 55(8), 735–760.

Bryk

A. S.

Raudenbush

S. W.

(1987). Application of hierarchical linear models to assessing change. Psychological bulletin, 101(1), 147.

10.

Campbell-Montalvo

R. A.

(2020). Being quantcritical of U.S. K-12 demographic data: Using and reporting race/ethnicity in Florida heartland schools. Race Ethnicity and Education, 23(2), 180–199. https://doi.org/10.1080/13613324.2019.1679748

11.

Carter

D. F.

Hurtado

(2007). Bridging key research dilemmas: Quantitative research using a critical eye. New Directions for Institutional Research, 133, 25–35. https://doi.org/10.1002/ir.202

12.

Castellanos

Cole

(2015). Disentangling the impact of diversity courses: Examining the influence of diversity course content on students’ civic engagement. Journal of College Student Development, 56(8), 794–811.

13.

Clayton

(2020, October 28). How eugenics shaped statistics: Exposing the damned lies of three science pioneers. Nautilus. https://nautil.us/issue/92/frontiers/how-eugenics-shaped-statistics

14.

Conway

K. M.

(2014). Critical quantitative study of immigrant students. New Directions for Institutional Research, 158, 51–63. https://doi.org/10.1002/ir.20045

15.

Cook

T. D.

Campbell

D. T.

(1979). Quasi-experimentation: Design and analysis issues for field settings. Houghton Mifflin Company.

16.

Covarrubias

(2011). Quantitative intersectionality: A critical race analysis of the Chicana/o educational pipeline. Journal of Latinos and Education, 10(2), 86–105. https://doi.org/10.1080/15348431.2011.556519

17.

Covarrubias

Nava

P. E.

Lara

Burciaga

Vélez

V. N.

Solórzano

D. G.

(2018). Critical race quantitative intersections: A testimonio analysis. Race, Ethnicity, and Education, 21(2), 253–273. https://doi.org/10.1080/13613324.2017.1377412

18.

Covarrubias

Vélez

(2013). Critical race quantitative intersectionality: An anti-racist research paradigm that refuses to “let the numbers speak for themselves”. In Lynn

Dixson

(Eds). Handbook of critical race theory in education (pp. 290–306). Routledge.

19.

Crawford

C. E.

(2019). The one-in-ten: Quantitative critical race theory and the education of the ‘new (white) oppressed’. Journal of Educational Policy, 34(3), 423–444. https://doi.org/10.1080/02680939.2018.1531314

20.

Crenshaw

. (2018). Demarginalizing the intersection of race and sex: A Black feminist critique of antidiscrimination doctrine, feminist theory, and antiracist politics [1989]. In Feminist legal theory (pp. 57–80). Routledge.

21.

Crenshaw

(1991). Mapping the margins: Intersectionality, identity politics, and violence against women of color. Stanford Law Review, 43, 1241–1252.

22.

Curley

(2019). Debunking the false dichotomy: Developing and applying trans quantcrit at the intersection of trans/non-binary identities and religious, secular, and spiritual engagement in college. Doctoral dissertation, Eastern Michigan University. https://commons.emich.edu/theses/986

23.

Du Bois

W. E. B

. (1899). Philadelphia Negro. Cosimo, Inc.

24.

Faircloth

S. C.

Alcantar

C. M.

Stage

F. K.

(2015). Use of large-scale data sets to study educational pathways of American Indian and Alaska Native students. New Directions for Institutional Research, 163, 5–24. https://doi.org/10.1002/ir.20083

25.

Garcia

N. M.

Lopez

Vélez

V. N.

(2018). QuantCrit: Rectifying quantitative methods through critical race theory. Race Ethnicity and Education, 21(2), 149–157. https://doi.org/10.1080/13613324.2017.1377675

26.

Garcia

N. M.

Mayorga

O. J.

(2018). The threat of unexamined secondary data: A critical race transformative convergent mixed methods. Race Ethnicity and Education, 21(2), 231–252. https://doi.org/10.1080/13613324.2017.1377415

27.

Garvey

J. C.

(2019). Queer quantitative query: Sexual orientation in higher education surveys. Journal of College Student Development, 60(4), 495–501. http://doi.org/10.1353/csd.2019.0042

28.

Garvey

J. C.

Hart

Metcalfe

A. S.

Fellabaum-Toston

(2019). Methodological troubles with gender and sex in higher education survey research. The Review of Higher Education, 43(1), 1–24. http://doi.org/10.1353/rhe.2019.0088

29.

Garvey

J. C.

Sanders

L. A.

Flint

M. A.

(2017). Generational perception of campus climate among LGBTQ undergraduates. Journal of College Student Development, 58(6), 795–817. doi:10.1353/csd.2017.0065

30.

Gillborn

Warmington

Demack

(2018). QuantCrit: Education, policy, ‘big data’ and principles for a critical race theory of statistics. Race Ethnicity and Education, 21(2), 158–179. https://doi.org/10.1080/13613324.2017.1377417

31.

Gonzalez Canche

M. S.

Rios-Aguilar

. (2015). Critical social network analysis in community colleges: Peer effects and credit attainment. New Directions for Institutional Research, 163, 75–91. https://doi.org/10.1002/ir.20087

32.

Gutiérrez

(2008). Research commentary: A gap-gazing fetish in mathematics education? Problematizing research on the achievement gap. Journal for Research in Mathematics Education, 39(4), 357–364.

33.

Hernández

(2015). What is ‘good’ research? Revealing the paradigmatic tension in quantitative criticalist work. New Directions for Institutional Research, 163, 93–101. https://doi.org/10.1002/ir.20088

34.

Hernandez

Mobley

Coryell

Martinez

(2013). Examining the cultural validity of a college student engagement survey for Latinos. Journal of Hispanic Higher Education, 12(2), 153–173. https://doi.org/10.1177%2F1538192713475453

35.

hooks

(1994). Teaching to transgress: Education as the practice of freedom. Routledge.

36.

Inckle

(2015). Debilitating times: Compulsory ablebodiedness and White privilege in theory and practice. Feminist Review, 111(1), 42–58. https://doi.org/10.1057/fr.2015.38

37.

Jang

S. T.

(2018). The implication of intersectionality on Southeast Asian female students’ educational outcomes in the United States: A critical quantitative intersectionality analysis. American Educational Research Journal, 55(6), 1268–1306. https://doi.org/10.3102%2F0002831218777225

38.

John

Stage

F. K.

(2014). Minority-serving institutions and the education of the U.S. underrepresented students. New Directions for Institutional Research, 158, 65–76. https://doi.org/10.1002/ir.20046

39.

Judd

C. M.

McClelland

G. H.

Ryan

C. S.

(2017). Data analysis: A model comparison approach to regression, ANOVA, and beyond (3rd ed.). Routledge. https://doi.org/10.4324/9781315744131

40.

Kendi

I. X.

(2019). How to be an antiracist. One World.

41.

Kilgo

C. A.

Linley

J. L.

Renn

K. A.

Woodford

M. R.

(2019). High impact for whom? The influence of environment and identity on lesbian, gay, bisexual, and queer college students’ participation in high-impact practices. Journal of College Student Development, 60(4), 421–436. doi:10.1353/csd.2019.0038.

42.

Kincheloe

J. L.

McLaren

(1994). Rethinking critical theory and qualitative research. In Denzin

N. K.

Lincoln

Y. S.

(Eds)., Handbook of qualitative research (pp. 138–157). Sage.

43.

Kinloch

San Pedro

(2014). The space between listening and storying: Foundations for projects in humanization. In Paris

Winn

M. T.

(Eds.), Humanizing research: Decolonizing qualitative inquiry with youth and communities (pp. 21–42). Sage.

44.

Kinzie

(2007). Women’s paths in science: A critical feminist analysis. New Directions for Institutional Research, 133, 81–93. https://doi.org/10.1002/ir.206

45.

Knowles

R. T.

Hawkman

A. M.

(2019). Anti-racist quantitative research: Developing, validating and implementing racialized teaching efficacy and racial fragility scales. The Urban Review, 52, 238–262. https://doi.org/10.1007/s11256-019-00526-1

46.

Kuhn

T. S.

(1962). The structure of scientific revolutions. University of Chicago Press.

47.

Kuhn

T. S.

(1977). The essential tension: Selected studies in scientific tradition and change. University of Chicago Press.

48.

Kukutai

Taylor

(2016). Indigenous data sovereignty: Toward an agenda. ANU Press.

49.

Lincoln

Y. S.

Lynham

S. A.

Guba

E. G.

(2018). Paradigmatic controversies, contradictions and emerging confluences revisited. In Denzin

N. K.

Lincoln

Y. S.

(Eds.), The Sage handbook of qualitative research (5th ed., pp. 108–150). Sage.

50.

Lopez

Erwin

Binder

Chavez

M. J.

(2018). Making the invisible visible: Advancing quantitative methods in higher education using critical race theory and intersectionality. Race Ethnicity and Education, 21(2), 180–207. https://doi.org/10.1080/13613324.2017.1375185

51.

Louca

(2009). Emancipation through interaction. How eugenics and statistics converged and diverged. Journal of the History of Biology, 42(4), 649–684.

52.

Malcolm-Piqueux

(2015). Application of person-centered approaches to critical quantitative research: Exploring inequities in college financing strategies. New Directions for Institutional Research, 163, 59–73. https://doi.org/10.1002/ir.20086

53.

Mayhew

M. J.

Simonoff

J. S.

(2015a). Effect coding as a mechanism for improving the accuracy of measuring students who self-identify with more than one race. Research in Higher Education, 56, 595–600. https://doi.org/10.1007/s11162-015-9364-0

54.

Mayhew

Simonoff

J. S.

(2015b). Non-white, no more: Effect coding as an alternative to dummy coding with implications for higher education researchers. Journal of College Student Development, 56(2) 170–175. http://doi.org/10.1353/csd.2015.0019

55.

Metcalf

(2011). Formation and representation: Critical analysis of identity, supply, an demand in science, technology, engineering, and mathematics. Doctoral dissertation, University of Arizona. https://repository.arizona.edu/handle/10150/145744

56.

Metcalf

H. E.

(2014). Disrupting the pipeline: Critical analyses of student pathways through postsecondary STEM education: Disrupting the pipeline. New Directions for Institutional Research, 158, 77–93. https://doi.org/10.1002/ir.20047

57.

Mills

(2003). From class to race: Essays in White Marxism and Black radicalism. Rowman & Littlefield.

58.

Nicolazzo

(2017). Imagining a trans* epistemology: What liberation thinks like in postsecondary education. Urban Education, 56(3), 511–536. https://doi.org/10.1177/0042085917697203

59.

Omi

Winant

(2014). Racial formation in the United States. Routledge.

60.

Oseguera

Hwang

(2014). Using large data sets to study college education trajectories. New Directions for Institutional Research, 158, 37–50. https://doi.org/10.1002/ir.20044

61.

Page

M.J.

McKenzie

J.E.

Bossuyt

P.M.

Boutron

Hoffman

T.C.

Mulrow

C.D.

Shamseer

Tetzlaff

J.M.

Akl

E.A.

Brennan

S.E.

Chou

Glanville

Grimshaw

J.M.

Hrobjartsson

Lalu

Loder

E.W.

Mayo-Wilson

McDonald

McGuinness

L.A.

Stewart

L.A.

Thomas

Tricoo

Welch

V.A.

Whiting

Moher

. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Systematic Reviews, 372 (71). https://doi.org/10.1136/bmj.n71

62.

Palmer

(1993). Critical theory, historical materialism, and the ostensible end of Marxism: The poverty of theory revisited. International Review of Social History, 38(2), 133–162.

63.

Park

J. J.

Buckley

J. B.

Koo

. (2017). How Do I Quantify This?: Students reflect on survey items related to diversity and cross-racial interaction. Journal of College Student Development, 58(5), 733–751.

64.

Pérez Huber

Vélez

V. N.

Solórzano

. (2018). More than ‘papelitos’: A quantcrit counterstory to critique Latina/o degree value and occupational prestige. Race Ethnicity and Education, 21(2), 208–230. https://doi.org/10.1080/13613324.2017.1377416

65.

Perna

L. W.

(2007). The sources of racial-ethnic group differences in college enrollment: A critical examination. New Directions for Institutional Research, 133, 51–66. https://doi.org/10.1002/ir.204

66.

QSR International. (2020). NVivo (released in March 2020), https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home

67.

Rios-Aguilar

(2014). The changing context of critical quantitative inquiry. New Directions for Institutional Research, 158, 95–107. https://doi.org/10.1002/ir.20048

68.

Rios-Aguilar

(2015). Using big (and critical) data to unmask inequities in community colleges. New Directions for Institutional Research, 163, 43–57. https://doi.org/10.1002/ir.2008

69.

H. K.

Bergom

(2020). Expanding our methodological toolkit: Effect coding in critical quantitative studies. New Directions for Student Services, 169, 87–97. https://doi.org/10.1002/ss.20347

70.

Rockenbach

A. B.

Mayhew

M. J

. (2013). How the collegiate religious and spiritual climate shapes students’ ecumenical orientation. Research in Higher Education, 54(4), 461–479.

71.

Rockenbach

A. B.

Riggers-Piehl

T. A.

Garvey

J. C.

M. A.

Mayhew

M. J.

(2015). The influence of campus climate and interfaith engagement on self-authored worldview commitment and pluralism orientation across sexual and gender identities. Research in Higher Education, 57, 497–517. https://link.springer.com/content/pdf/10.1007/s11162-015-9395-6.pdf

72.

Sablan

J. R.

(2019). Can you really measure that? Combining critical race theory and quantitative methods. American Educational Research Journal, 56(1), 178–203.

73.

Saunders

(2015). A critical quantitative analysis of the effects of budgeting models in creating equity for high school students in a large urban school district. Doctoral dissertation, Howard University. https://www.proquest.com/openview/67db1ce999fcadca7430f0044292f61d/1?pq-origsite=gscholar&cbl=18750&diss=y

74.

Smith

L. T.

(1999). Decolonizing methodologies: Research and indigenous peoples. Bloomsbury Publishing.

75.

Solórzano

D. G.

Vélez

V. N.

(2015). Using critical race spatial analysis to examine the Du Boisian color-line along the Alameda Corridor in Southern California. Whittier Law Review, 37, 423–437.

76.

Solórzano

D. G.

Villalpando

Oseguera

(2005). Educational inequities and Latina/o undergraduate students in the United States: A critical race analysis of their educational progress. Journal of Hispanic higher education, 4(3), 272–294.

77.

Solórzano

D. G.

Yosso

T. J.

(2001). Critical race and LatCrit theory and method: Counter-storytelling. International Journal of Qualitative Studies in Education, 14(4), 471–495.

78.

St. John

E. P

. (2007). Finding social justice in education policy: Rethinking theory and approaches in policy research. New Directions for Institutional Research, 133, 67–80. https://deepblue.lib.umich.edu/bitstream/handle/2027.42/56005/205_ftp.pdf;sequence=1

79.

Stage

F. K.

(2007a). Answering critical questions using quantitative data. New Directions for Institutional Research, 133, 5–16. https://dx-doi-org.web.bisu.edu.cn/10.1002/ir.200

80.

Stage

F. K.

(2007b). Moving from probabilities to possibilities: Tasks for quantitative criticisms. New Directions for Institutional Research, 133, 95–100. https://doi.org/10.1002/ir.207

81.

Stage

F. K.

Wells

R. S.

(2014). Critical quantitative inquiry in context. New Directions for Institutional Research, 158, 1–7. https://doi.org/10.1002/ir.20041

82.

Stewart

D-L.

(2013). Racially minoritized students at U.S. four-year institutions. Journal of Negro Education, 82(2), 184–197. https://doi.org/10.7709/jnegroeducation.82.2.0184

83.

Sullivan

Larke

P. J.

Webb-Hasan

(2010). Using critical policy and critical race theory to examine Texas’ school disciplinary policies. Race, Gender & Class, 17(1–2), 72–87.

84.

Tabron

Ramlackhan

(2019). Hypocrisy, state policy, and African American students with disabilities: The guise of access. Educational Policy, 33(1), 181–204. https://doi.org/10.1177%2F0895904818807308

85.

Tabron

L. A.

(2019). I pulled up a seat at the table: My journey engaging in critical quantitative inquiry. In Strunk

Locke

(Eds). Research methods for social justice and equity in education (pp. 275–282). Palgrave Macmillan.

86.

Tabron

L. A.

Hunt-Khabir

Thomas

A. K.

(2021). Disrupting whiteness in introductory statistics course design: Implications for educational leadership. In Mullen

(Ed.), Handbook of social justice interventions in education. Springer.

87.

Tabron

L. A.

Thomas

(in press). (Un)Learning White supremacy ideologies to advance critical approaches to quantitative inquiry. In Diem

Young

(Eds.), Handbook of critical education research: Qualitative, quantitative, and emerging approaches. Routledge.

88.

Ternashi

R. T.

(2007). Race, ethnicity, and higher education policy: The use of critical quantitative research. New Directions for Institutional Research, 133, 37–49. https://doi.org/10.1002/ir.203

89.

Thomas

A. K.

(2022). A grounded theory inquiry into the pedagogical socialization of graduate students within graduate quantitative methods courses. Unpublished doctoral dissertation. University of Denver. http://iaseweb.org/documents/dissertations/21.AmandaKayThomas.Dissertation.pdf

90.

Toldson

I. A.

(2019). No BS (Bad Stats) (pp. 3–16). Brill Sense.

91.

Vaccaro

Kimball

Wells

R. S.

Ostiguy-Finneran

B. J.

(2015). Researching students with disabilities: The importance of critical perspectives. New Directions for Institutional Research, 163, 25–41. https://doi.org/10.1002/ir.20084

92.

Van Dusen

Nissen

. (2020). Equity in college physics student learning: A critical quantitative intersectionality investigation. Journal of Research in Science Teaching, 57(1), 33–57. https://doi.org/10.1002/tea.21584

93.

Walter

Andersen

(2016). Indigenous statistics: A quantitative research methodology. Routledge.

94.

Walter

Suina

(2019). Indigenous data, indigenous methodologies and indigenous data sovereignty. International Journal of Social Research Methodology, 22(3), 233–243.

95.

Wells

R. S.

(2010). Children of immigrants and educational expectations: The roles of school composition. Teachers College Record, 112(6), 1679–1704. https://www.tcrecord.org/Home.asp

96.

Wells

R. S.

Stage

F. K.

(2015). Past, present, and future of critical quantitative research in higher education. New Directions for Institutional Research, 163, 103–112. https://doi.org/10.1002/ir.20089

97.

Williams

K. L.

(2014). Strains, strengths, and intervention outcomes: A critical examination of intervention efficacy for underrepresented groups. New Directions for Institutional Research, 158, 9–22. https://doi.org/10.1002/ir.20042

98.

Wilson

(2008). Research is ceremony: Indigenous research methods. Fernwood Publishing.

99.

Young

J. L.

Foster

M. D.

Druery

D. M.

(2018a). A critical exploratory analysis of Black girls’ achievement in 8th grade U.S. history. Middle Grades Review, 4(3), 1–15. https://scholarworks.uvm.edu/mgreview/vol4/iss3/2/

100.

Young

J. L.

Young

J. R.

(2018). The structural relationship between out-of-school time enrichment and black student participation in advanced science. Journal for the Education of Gifted, 41(1), 43–59. https://doi.org/10.1177%2F0162353217745381

101.

Young

J. L.

Young

J. R.

Capraro

R. M.

(2018b). Gazing past the gaps: A growth-based assessment of the mathematics achievement of Black girls. The Urban Review, 50, 156–176. https://doi.org/10.1007/s11256-017-0434-9

102.

Zerquera

D. D.

Gross

J. P. K.

(2017). Context matters: A critical consideration of Latina/o student success outcomes within different institutional contexts. Journal of Hispanic Higher Education, 16(3), 209–231. https://doi.org/10.1177%2F1538192715612915

103.

Zuberi

(2001). Thicker than blood: How racial statistics lie. University of Minnesota Press.

104.

Zuberi

Bonilla-Silva

(Eds.). (2008). White logic, White methods: Racism and methodology. Rowman & Littlefield Publishers.

Deeper than Wordplay: A Systematic Review of Critical Quantitative Approaches in Education Research (2007–2021)

Abstract

Keywords

Critical Quantitative and Quantitative Criticalism

Emergent Conflation

Positionality

Method

Systematic Literature Search

Phase One: Begin with Clearly Defined Research Questions

Search Criteria and Terms

Phase Two: Abstract Screening

Phase Three: Full Text Screen

Phase Four: Included Study Characteristics

Special Issues

Journals and Journal Focus

CritQuant, Quantitative Criticalism, QuantCrit Author Classifications

Results

Paradigm Shift

Resistance to Normal Science

The Racialization of Educational Crises

Problem-Solving

Critical Approaches to Quantitative Inquiry are Viable Ways of Knowing

Critical Quantitative (CritQuant) and QuantCrit are Deeper than Wordplay

Analogical Reasoning

Quantitative critical race theory (QuantCrit)

Clearing Up Conflation

Applied Praxis: Strategies Used

For us by us (FUBU)

Present strength narratives of historically marginalized communities

Ground truthing the data

Critique social categorizations

Reject the oversimplification of aggregate data

Build multidimensional statistical models

Critically interrogate measure validation

Ending the checkbox culture in survey design and administration

Presenting a numerically broad picture

Advanced heterogeneity of variance

Model specification and estimation

Discussion

Embracing Heterogenous Orientations and Methodological Approaches

Common Practices

What Are Not Critical Approaches to Quantitative Inquiry?

Conclusion

Footnotes

Acknowledgements

ORCID iD

Authors

References