Abstract
Although the critical research cannon is often associated with qualitative scholars, there is a growing number of critical scholars who are refusing positivist-informed quantitative analyses. However, as a growing number of education scholars engaged in critical approaches to quantitative inquiry, instances of conflation began to surface. We understood this conflation as the interchangeable use of the terms quantitative criticalism, QuantCrit, and critical quantitative throughout the literature and even within the same chapter or article. The purpose of our systematic literature review is twofold: (a) to understand how critical approaches to quantitative inquiry emerged as a new paradigm within quantitative methods and (b) whether there is any distinction between quantitative criticalism, QuantCrit, and critical quantitative inquiries or simply interchangeable wordplay. We share how critical quantitative approaches are definite shifts within the quantitative research paradigm, highlight relevant assumptions, and share strategies and future directions for applied practice in this emergent field.
Keywords
Francis Galton, Karl Pearson, and Ronald Fisher were eugenicists. They developed statistics as a scientific discipline in the modern social sciences to discover universal laws about human differences that they believed would be as indisputable as scientific laws such as Newton’s laws of motion (Clayton, 2020; Louca, 2009; Zuberi, 2001; Zuberi & Bonilla-Silva, 2008). Aligned with positivism, these white European men drew inspiration and techniques from the natural sciences (i.e., astronomy and earth sciences) and favored quantitative analysis to repeatedly observe, test, verify, and corroborate their theories that many revered as facts (Zuberi & Bonilla-Silva, 2008). However, statistics are not facts; they are human-generated estimations based on uncertainty (Zuberi, 2001). Those who engage in critical approaches to quantitative inquiry understand that “there is not a set logic within the methods themselves” (Zuberi & Bonilla-Silva, 2008, p. 8). Still, statistics reflect the logic; theories; accepted practice; and consensus of society, practitioners, and researchers (Tabron & Thomas, in press). That is, statistics reflect theories of society and “these men adhered to the unquestioned white supremacy of their time” (Omi & Winant, 2014, p. 5). Although these founders of modern social statistics espoused to be theory free (positivism), their racial theories were value-laden attempts to advance white supremacy and prove that human differences reflect the natural order (Toldson, 2019; Zuberi, 2001; Zuberi & Bonilla-Silva, 2008).
Although the critical research canon is often associated with qualitative scholars, there has always been and continues to be a growing number of critical scholars who are refusing positivist-informed quantitative analyses (Garcia et al., 2018; Gillborn et al., 2018; Omi & Winant, 2014; Smith, 1999; Stage, 2007a; Stage & Wells, 2014; Walter & Andersen, 2016; Wells & Stage, 2015; Wilson, 2008; Zuberi, 2001; Zuberi & Bonilla-Silva, 2008). Although critical approaches to quantitative inquiry for social justice builds on the freedom struggle and foundational intellectual contributions of Black scholars and other scholars of Color (Omi & Winant, 2014; Smith, 1999; Zuberi, 2001; Zuberi & Bonilla-Silva, 2008) that predates Stage’s (2007a) editorial, their editorial introduced the terms critical quantitative, quantitative criticalist, and quantitative criticalism as frames of reference for education scholars who challenge quantitative research framed within the positivist tradition.
Critical Quantitative and Quantitative Criticalism
Stage’s foregrounding of criticality began with the Frankfurt school of critical philosophy and extended toward Kincheloe and Peter McLaren’s (1994) work on rethinking critical theory and qualitative inquiry. Stage (2007a) based their definition of critical on Kincheloe and McLaren’s (1994) assumptions about critical researchers that included: (a) Thought is mediated by socially and historically created power relations; (b) facts cannot be isolated from values; (c) the relationship between concept and object is never fixed and is often socially mediated; (d) language is central to the formation of subjectivity; (e) certain groups in society hold privilege over others that is maintained if subordinates accept their status as natural; (f) oppression has many faces that must be examined simultaneously; and (g) mainstream research practices reproduce class, race, and gender oppression. (p. 7)
For Stage (2007a), these seven assumptions did not prevent the use of quantitative methods to critically interrogate issues in education. Likewise, in the first special issue on critical approaches to quantitative inquiry (Stage, 2007a), the contributing authors united around three methodological commitments with a heavy focus on establishing the legitimacy of and encouraging more scholars to engage in critical epistemologies within quantitative inquiry (Baez, 2007; Carter & Hurtado, 2007; Kinzie, 2007; Perna, 2007; St. John, 2007; Teranishi, 2007). These two methodological commitments (“tasks”) were:
1) Use data to represent educational processes and outcomes on a large scale to reveal inequities and to identify social or institutional perpetuation of systematic inequities in such processes and outcomes. (Stage, 2007a, p. 10)
2) Question the models, measures, and analytic processes of quantitative research in order to offer competing models, measures, and analytic practices that better describe the experiences of those who have not been adequately represented. (Stage, 2007a, p. 10)
Stage and Wells (2014) offered a third task in the second special issue, which was to “conduct culturally relevant research by studying institutions and people in context” (p. 3).
Within the second special issue (Stage and Wells, 2014), there was a growing number of higher education scholars “who embraced the term quantitative criticalist” (p. 3), and the goal was to continue to raise awareness of past “critical quantitative work” and encourage future research in this area (Alcantar, 2014; Conway, 2014; John & Stage, 2014; Metcalf, 2014; Oseguera & Hwang, 2014; Rios-Aguilar, 2014; Williams, 2014). At the end of the second special issue, Cecilia Rios-Aguilar (2014) synthesized the scholarship presented in Stage and Wells (2014) by challenging quantitative criticalists to engage in methodological self-reflection as they are designing and conducting research. These tasks included:
1) Ask relevant questions (about equity and power).
2) Choose relevant data.
3) Apply appropriate, rigorous, and sophisticated analyses.
4) Disaggregate analysis on gender, race/ethnicity, language proficiency, socioeconomic status, and conduct research on several groups of marginalized students (e.g., students who have had contact with the juvenile justice system, indigenous students, and students with disabilities).
5) Know how to interpret results (i.e., pay attention to statistical and educational significance).
6) Employ challenging and enriching theories in multiple disciplines.
7) Inform and challenge existing institutional practices and decisions.
8) Inform and challenge existing educational policies.
In the third special issue (Wells & Stage, 2015), Hernández (2015) offered three more challenges to “shift toward a focus on grappling with paradigmatic concerns that require attention in order to ensure rigor and appropriate training for the application of quantitative criticalism” (p. 94). These three challenges are:
1) Quantitative criticalism challenges normative assumptions and research practices in “quantitative research.”
2) Quantitative criticalism requires a high level of expertise in both statistical analyses and critical theory.
3) Quantitative criticalism requires the use of a set of critical theoretical tenets to ensure legitimacy and rigor.
Contributing authors in Stage (2007a), Stage and Wells (2014), and Wells and Stage (2015) special issues identified as quantitative criticalists who engaged in critical quantitative inquiry (CritQuant). They understood critical quantitative inquiry as grounding their diverse critical theoretical perspectives in these shared methodological commitments (tasks). The authors collectively aimed to provide the theoretical backing that justified critical quantitative research as definite “paradigm shift” and legitimate approach in quantitative inquiry (Stage & Wells, 2014, p. 1), a legitimacy exclusively attached to positivist and post-positivist research paradigms.
Emergent Conflation
However, as a growing number of education scholars engaged in critical approaches to quantitative inquiry, instances of conflation began to surface. We understood this conflation as the interchangeable wordplay of quantitative criticalism, QuantCrit, and critical quantitative within the same chapter or article (Garvey et al., 2017; Rios-Aguilar, 2014; Rockenbach & Mayhew, 2013; Wells & Stage, 2015) and wondered to what extent this is simply interchangeable wordplay or distinct perspectives and techniques. Given this interchangeable use of the words quantitative criticalism, QuantCrit, and critical quantitative, we decided to conduct a rigorous systematic review of the existing scholarship to highlight the “distinction[s] between the different levels of analysis” connected to each approach and illustrate how understanding these distinctions can promote the “adequate location of phenomena” for future scholarship (Alexander, 2020; Kabatek, 2015, p. 213). The purpose of our systematic literature review is twofold: (a) to understand how critical approaches to quantitative inquiry emerged as a new paradigm within quantitative methods and (b) whether there is any distinction between quantitative criticalism, QuantCrit, and critical quantitative inquiries or simply interchangeable wordplay. We share how critical quantitative approaches are definite shifts within the quantitative research paradigm, highlight relevant assumptions, and share strategies and future directions for applied practice in this emergent field.
Positionality
It is vital to share how we locate ourselves within this work. As Black women engaged in quantitative research, the historical antecedents of our criticality predated Stage (2007a), the Frankfurt School, and Kincheloe and McLaren’s (1994) scholarship on criticality in qualitative research (Tabron et al., 2021). Although these scholars provided the starting point for Stage’s (2007a, 2007b) definition of critical quantitative research, it was not possible for us to do the same. The Frankfurt school’s criticality is rooted in economic determinism and tracing historical materialism for liberal class consciousness among the white working class in Europe and then America (Mills, 2003; Palmer, 1993).
We located ourselves within a centuries-long resistance to the eugenic origins of quantitative research and racialized statistics on Black, Indigenous, and people of Color (BIPOC) (Zuberi & Bonilla-Silva, 2008; Omi & Winant, 2014). Critical approaches to quantitative inquiry are not new to our communities who have always been distrustful of statistical research trying to “prove” the intellectual and cultural inferiority of communities of color and obscure racism (Tabron, 2019). Our journey of locating ourselves within this work is based on our inherited role as scholars who push back against quantitative research that reflects and protects white supremacy (Crawford, 2019; Stage, 2014; Tabron, 2019; Tabron et al., 2021; Zuberi, 2001; Zuberi & Bonilla-Silva, 2008, Wells & Stage, 2015). Our positionality is present in every phase of this review to advance social equity and racial justice in quantitative research.
Method
Systematic Literature Search
We conducted this systematic literature review to synthesize educational research on critical approaches to quantitative inquiry between 2007–2021. In Figure 1, we present our flow chart that reflects our screening process. We began phase one with clearly defined questions to guide the review of literature, inclusion and exclusion criteria, and search terms that returned our first pool of articles screened. During phase two we completed our title and abstract screening and inter-rater reliability check, and in phase three we discussed our full-text coding procedures.
Systematic review flow chart. From: PRISMA 2020 flow diagram template for systematic reviews (Page et al., 2021; adapted from flow diagrams proposed by Boers and Mayo-Wilson et al. and Stovold et al.).
Phase One: Begin with Clearly Defined Research Questions
Phase one of our systematic review of literature began with two questions that guided our review to unpack conflation between critical quantitative, quantitative criticalism, and QuantCrit educational research:
We formulated both research questions to understand whether these terms were interchangeable or conceptually different and to understand trends within this body of scholarship. In the next section, we share our inclusion and exclusion criteria and show replicable parameters for future reviews on critical approaches to quantitative inquiry.
Search Criteria and Terms
We narrowed our scope to books, dissertations, and peer-reviewed articles published between 2007–2021. We selected the 2007 publication year to correspond to Frances Stage’s (2007a) first special issue on critical approaches to quantitative inquiry. We included conceptual and empirical studies with an education focus along the P-20 pipeline whose authors applied critical perspectives to quantitative research. Next, we used the search terms “critical quantitative” OR “quantitative criticalism” OR CritQuant OR QuantCrit OR “critical quantitative inquiry” OR “critical race theory of statistics” OR “quantitative criticalist” OR “crit* and quant* OR Crit* race theory” and Statistics within Academic Search Complete (n = 50), Social Science Premium (n = 128), ERIC (n = 17), and Education Collection (n = 70). These searches returned a total of 265 studies that we stored within the RefWorks data management software (ProQuest, 2020). After this first pull, we cross-checked all databases for duplicates (n = 132). This left 133 articles ready for phase two abstract screening.
Phase Two: Abstract Screening
We independently screened 133 initial studies using a predetermined coding scheme stored within our shared RefWorks database. We carefully read through each study’s title and abstract and coded “yes” (forward to full-text screen) or “no” (does not meet inclusion and exclusion criteria). If we marked “no” for the study, we added a numerical code explaining why we excluded it. We coded studies “no-1” if it was a duplicate not caught in the database crosscheck, coded “no-2” if the study did not have an education focus, coded “no-3” if the study did not have a focus on critical quantitative inquiry, and coded “no-4” if the study was not an empirical or conceptual study. Whenever a coding structure was difficult to figure out by the abstract, we scanned the full text to decide the appropriate code. After we completed abstract coding, we calculated inter-rater reliability estimates to measure the consistency of our coding structures. Out of the 133 articles, author one coded (n = 99) articles “no” and author two coded (n = 109) articles “no,” yielding a 91% IRR agreement on the articles that we did not include for full-text review.
Upon this result, we discussed our coding application process. We discovered that the first author coded “yes” if the article met the inclusion criteria and the researchers asked critical research questions or used critical theoretical perspectives to guide their quantitative research designs and challenged majoritarian narratives and assumptions. The second author qualified studies focused on challenging traditional quantitative assumptions, methods, and measures. We eventually decided to include all studies coded “yes” from either author. We excluded 97 articles from our abstract screening pool as follows: (n = 3) previously undetected duplicates, (n = 58) noneducation focus, (n = 23) studies where researchers did not apply critical approaches to quantitative inquiry, and (n = 13) nonempirical studies. This left (n = 36) articles for full-text screening.
Phase Three: Full Text Screen
We carefully read all 36 articles independently, conducted open coding, and aggregated our codes to form emergent themes from the data using NVivo v.12 (QSR International, 2020). Through our forward and backward reference search we found 27 articles not captured in our first search, bringing our final pool to the 63 full-text articles for review. During each check-in, we engaged in a dialogic decision-making process (Kinloch & San Pedro, 2014) about our codes and emergent trends within our data and kept a research journal to capture our learnings shared within our discussion section.
Phase Four: Included Study Characteristics
Our sample included 63 articles (35 peer-reviewed manuscripts, 24 book chapters, and 4 dissertations). There were 19 conceptual studies and 44 empirical studies. Most of the articles (76%, n = 48) reviewed had a higher education focus. At the time of this review, there were 12 articles (19%) that had a P–12 focus, which shows that QuantCrit and CritQuant approaches are slowly beginning to appear in P–12 education. There were three articles (5%) that had a general education focus but not a K–12 or higher education distinction. As seen in Figure 2, our review included articles published between 2007–2021. Most of the articles (60% or n = 38) were published between 2015–2021. The years with the highest frequency correspond with years where there were special issues on QuantCrit and CritQuant approaches.
Frequency count of QuantCrit and CritQuant articles published between 2007–2021.
Special Issues
Almost half of articles in our sample (n = 29, 46%) were by invitation through special issues. Frances Stage edited and published the first special issue (n = 8), Using Quantitative Data to Answer Critical Questions, in New Directions for Institutional Research in 2007. In this first issue, Stage (2007a) provided examples of how scholars can conduct critical research using quantitative methods. Seven years later, Frances Stage and Ryan Wells (2014) published the second special issue (n = 8), New Scholarship in Critical Quantitative Research-Part 1: Studying Institutions and People in Context, in New Directions for Institutional Research. This issue addressed critiques of critical quantitative research “provided an expanded conceptualization of the tasks” and highlighted “the need to conduct culturally relevant research by studying institutions and people in context” (Stage & Wells, 2014, p. 1). In 2015, Ryan Wells and Frances Stage edited a third special issue (n = 7), New Scholarship in Critical Quantitative Research-Part 2: New Populations, Approaches, and Challenges, in New Directions for Institutional Research. This issue provided a current pulse of developments and areas of expansion in critical approaches to quantitative inquiry.
The fourth special issue (n = 6), QuantCrit: Rectifying Quantitative Methods through Critical Race Theory, was edited by Nichole Garcia, Nancy López, and Verónica N. Vélez in 2018 in the journal Race Ethnicity and Education. Here, scholars applied critical race theory (CRT) as a theoretical or methodological framework to disrupt structural racism in quantitative research. The year 2019 marked the highest number of studies published outside of a special issue (n = 10). When we examined these 10 studies, half of the researchers (n = 5 studies) referenced Garcia et al.’s (2018) special issue and grounded their quantitative inquiry in CRT. These four special issues were among the first to publish critical approaches to quantitative inquiry. Outside of these four special issues, the number of publications dropped dramatically when viewed annually. This could be a signal of potential challenges, and pushback scholars might face getting their work accepted in journals without a critical focus (Hernández, 2015). The lower publication numbers year to year outside of special issues made us curious about the potential gatekeeping mechanisms that might be slowing the dissemination and momentum of critical approaches to quantitative research.
Journals and Journal Focus
In Figure 3, most articles (23 articles or 35%) were from the journal New Directions for Institutional Research. We reviewed seven articles, or 11%, from the journal of Race, Ethnicity and Education. Research in Higher Education, and Journal of College Student Development published four, or 6%, of the articles reviewed in this study. Based on their impact factor, 29 (or 45%) of the manuscripts we reviewed were published in (Q1) top tier journals, 3 articles (5%) were published in mid-tier journals (Q2), 1 article (2%) was published in a tier-three journal (Q3). Twenty-seven (42%) of the manuscripts were published in outlets that did not have a published impact factor, and the remaining four manuscripts were dissertations.
Journals for manuscripts included in sample.
In our review, 27 journals covered general education topics (i.e., American Educational Research Journal [AERJ]), 10 journals had a higher education focus, 1 journal had a middle school focus, 1 journal focused on science, and another emphasized gifted education. Four journals focused on race (Latinx, n = 5; Black, n = 2), intersectionality (n = 1), ethnic studies (n = 7), or urban education (n = 2). Most scholars found homes for their critical quantitative work through special issues or in journals known for publishing critical work with an emphasis on historically marginalized communities.
CritQuant, Quantitative Criticalism, QuantCrit Author Classifications
Although all the manuscripts had critical approaches to quantitative inquiry, there were inconsistencies in how authors positioned their work. As seen in the Supplementary Appendix (available in the online version of this article), Tabular Listing of Data Sources Analyzed in Systematic Review, the authors in 40 studies (63%) labeled their work as CritQuant, 11 (17%) labeled their work as QuantCrit, 2 (3%) labeled their work as quantitative criticalism, 8 labeled their work (13%) as something other than these terms (i.e., trans QuantCrit), and the authors of 2 studies (3%) did not label their work although it aligned with the assumptions of either QuantCrit or CritQuant.
In our initial search of the literature, most scholars began their exploration with one or more of the following works: (a) Frances Stage and Ryan Wells’s three-part series in New Direction for Institutional Research on critical quantitative research (Stage, 2007a; Stage & Wells, 2014); (b) Tukufu Zuberi’s (2001) book, Thicker than Blood: How Racial Statistics Lie; (c) Tukufu Zuberi and Eduardo Bonilla-Silva’s (2008) edited book, White Logic, White Methods Racism and Methodology; (d) Alejandro Covarrubias (2011) and Alejandro Covarrubias and Verónica Vélez’s (2013) work on critical race quantitative intersectionality; (e) Nichole M. Garcia, Nancy López, and Verónica N. Vélez’s (2018) QuantCrit special issue in Race, Ethnicity and Education. Most of the authors (n = 44, 70%) used Stage (2007a, 2007b), Stage & Wells (2014), and Wells and Stage’s (2015) definitional framing of critical approaches to quantitative inquiry as their starting point. Other scholars cited Zuberi (2001) and Zuberi and Bonilla-Silva (2008) (n = 16, 25%); Covarrubias (2011) or Covarrubias and Vélez’s (2013) (n = 15, 24%); Garcia, López, and Vélez’s (2018) (n = 8, 13%); Gillborn et al.’s (2018) (n = 13, 20%); Du Bois (1899) (n = 3, 5%); or Kimberle Crenshaw’s (2018, 1991) intersectionality theory (n = 5, 8%) as a frame of reference for their work. The remaining articles in our sample did not reference these foundational texts in their studies.
Results
In this section, we present the results that appeared from our codified data and thematic analyses—theme 1: paradigm shift and theme 2: critical quantitative (CritQuant) and quantitative critical race (QuantCrit) are deeper than wordplay.
Paradigm Shift
Critical quantitative inquiry denotes the first stages of a “paradigm shift” (Stage, 2014, p. 1) away from positivism and post-positivism in quantitative inquiry. Our first theme, paradigm shift, emerged from the subthemes: resistance to normal science, the racialization of educational crises, problem solving, and critical approaches to quantitative research are viable ways of knowing.
Resistance to Normal Science
Normal science is a shared understanding among scholars about the thought patterns (logic), theories, methods, and standards for what constitutes legitimate contributions to a field (Kuhn, 1962). Scholars who practice normal science answer quantitative research questions in ways that reify the positivist paradigm (Cook & Campbell, 1979; Kuhn, 1962). Historically, what counted as quantitative research was based on positivist assumptions (objectivity, universal truth, value free) to test theories to reveal fundamental truths about people (Clayton, 2020). Kuhn (1962) defined a paradigm shift as resistance to normal science, crisis identification, problem solving, and emergence of an alternative paradigm for crisis resolution. Critical approaches to quantitative inquiry are a resistance to normal science grounded in the positivist and post-positivist paradigms.
The Racialization of Educational Crises
Critical approaches to quantitative inquiry shed light on the longstanding racialization of social issues within quantitative research and the perpetuation of the biological, cultural, and intellectual superiority of one group over others (Garcia et al., 2018; Zuberi, 2001; Zuberi & Bonilla-Silva, 2008). Scholars engaged in critical approaches to quantitative inquiry tracked the different crises in educational leadership and policies brought on by the flawed application and interpretation of quantitative research, underused and misused datasets, faulty statistical interpretations, gap gazing and exclusion of diversity themes. They called out federal and state agencies using critical policy questions (Perna, 2007; St. John, 2007) to speak useful truths to power about the mishandling of crises faced by students within the P–20 pipeline based on specific agendas (Alcantar, 2014; Allen & Wolniak, 2019; Campbell-Montalvo, 2020; Covarrubias, 2011; Garcia et al., 2018; Malcolm-Piqueux, 2015; Solórzano & Yosso, 2001; Tabron & Ramlackhan, 2019; Teranishi, 2007; Wells, 2010). Several authors highlighted the tendency for users of national and state surveys to produce misleading social categorizations and “carelessly” use data sets (Wells, 2010, p. 1688).
Positivist researchers often racialized educational crises using data from local, state, and federal agencies to model and test deficit theories and produce deficit interpretations (i.e., fixed capabilities, low performers, below average) about students of color. Those engaged in critical approaches to quantitative inquiry have named this as gap-gazing research (Gutiérrez, 2008; Young et al., 2018a, 2018b) that is used to frame the discourse through a deficit lens and undervalue the educational aspirations and attainment among underrepresented and marginalized groups, particularly in science, technology, engineering, and math (STEM) (Van Dusen & Nissen, 2020; Metcalf, 2011; Young & Young, 2018; Young et al., 2018b). Scholars engaged in critical approaches to quantitative inquiry redirected the racialization of crises toward the racist policies, practices, and discourses that reproduce inequities (Gillborn et al., 2018; Kendi, 2019; Zuberi, 2001).
For example, scholars in our sample found that many of the earlier quantitative analyses did not portray the complex yet nuanced factors shaping the experiences of Black, Indigenous, people of Color (BIPOC) communities. Campbell-Montalvo (2020) illuminated the practice of “racial reformation” among school personnel in data reporting. This “racial reformation included school personnel changing the race/ethnicity reported on teacher-created grade-level cards and presenting racial/ethnic categories inconsistently across different school measures. Campbell-Montalvo (2019) argued how this “racial reformation” contributes to the erasure of Indigenous students in data reporting and prevents the inclusion of diverse race(s) of Latinx youth in school, local, state, and federal reporting, which could have grave political, financial, and legal implications for this historically marginalized populations (p. 181).
Teranishi (2007) illuminated the exclusion and misrepresentation of Asian Americans in educational research and policy through collapsing subgroups within the population. Teranishi (2007) critiqued between-group comparisons that masks differences within ethnic groups and “often promote and reinforce racialized assumptions that are held about Asian Americans based on studies that aggregate the population into a single racial category so they can be compared to other racial groups” (Teranishi, 2007, p. 47). Rios-Aguilar (2014) critiqued dominant perspectives that Latina/o students were not civically engaged by critiquing traditional and normative models and definitions of engagement. These models often do not consider engagement within sociohistorical and policy contexts. She emphasizes social civic behaviors such as translating for the community, tutoring, and mentoring youth and other forms of engagement connected to their ethnic and/or immigrant identities (Rios-Aguilar, 2014, p. 3). Similarly, Faircloth and colleagues (2015) critiqued the construction of large-scale data sets that do not capture the full linguistic and cultural diversity of American Indian and Alaskan Native students, lack culturally relevant variables, combine distinct cultural groups (American Indian and Alaska Native) and group them as “other” due to small sample sizes, or erase them completely from the data. Young and colleagues (2018b) critiqued the gap gazing that masks the high achievement of Black girls in math and creates missed opportunities to “promote growth by citing and affirming strengths as a means to build capacity and promote success” (p. 170).
Taken together, researchers engaged in critical approaches to quantitative inquiry centered and contextualized the experiences and perspectives of BIPOC students. They expanded the discourse by sharing asset-based narratives of how historically marginalized students excel and redefine success and engagement (Bielby et al., 2014; Gonzalez Canche & Rios-Aguilar, 2015; Covarrubias et al., 2018; Curley, 2019; Faircloth et al., 2015; Garcia et al., 2018; Garvey, 2019; Kilgo et al., 2019; Rios-Aguilar, 2015; Tabron & Ramlackhan, 2019; Vaccaro et al., 2015; Wells, 2010).
Problem-Solving
Problem-solving activities within paradigms are the “universally recognized scientific achievements which provide model problems and solutions for a community of practitioners” (Kuhn, 1977, p. viii). Through methodological innovation researchers engaged in critical quantitative approaches solved some of the weak points connected to positivist research (Thomas, 2022). Alternative, intersectional, inclusive, and participatory frameworks addressed the problematic framing of research questions and hypotheses (Williams, 2014). Scholars engaged in critical approaches to quantitative inquiry amended statistical formulae to address the intersectional, additive, and multiplicative effects of racialized and marginalized students’ identities (Mayhew & Simonoff, 2015a, 2015b; Wells, 2010). The frequency in which researchers linked race to disparate academic outcomes led scholars to stop the application of race as a fixed explanatory variable (Mayhew & Simonoff, 2015a, 2015b; Wells, 2010; Zuberi & Bonilla-Silva, 2008).
Researchers used efficient data management techniques to troubleshoot the perpetuation of deficit racial formation in big data (Campbell-Montalvo, 2020; Oseguera & Hwang, 2014). This helped stymie the lumping of students into monolithic racial categories (Covarrubias et al., 2018; Teranishi, 2007). The intersectionality of several marginalized identities became visible in aggregate achievement data (Covarrubias, 2011; Covarrubias et al., 2018; Rockenbach et al., 2015; Vaccaro et al., 2015). Classical quantitative methods such as logistic regression (Garcia & Mayorga, 2018), multiple regression (Jang, 2018; Saunders, 2015; Wells, 2010), fixed effects modeling (Allen & Wolniak, 2019), and history analyses (Zerquera & Gross, 2017), as well as education opportunity models (Lopez et al., 2018), scale development, and validation procedures were also updated (Knowles & Hawkman, 2019).
Critical Approaches to Quantitative Inquiry are Viable Ways of Knowing
Critical approaches to quantitative inquiry are defined by their own: axiologies, ontologies, epistemologies, and methodologies for conducting research (Stage, 2014). Within our sample, for more than two decades (2007–2021) critical researchers have moved beyond criticality and disruption and toward alternative ways of knowing and doing quantitative methods. We created Table 1 for tracing the paradigmatic issues related to critical approaches to quantitative inquiry in educational research. Lincoln, Lynham, and Guba’s (2018) work on the paradigmatic differences between qualitative and quantitative research was instrumental for developing our understanding of critical approaches to quantitative inquiry as a paradigm shift within the quantitative research. In the first column we list axiology, epistemology, ontology, and methodology as the knowledge-producing systems used to define research paradigms. In the second and third columns we specify the tenets that classify positivism and post-positivism as normalized science. In the last column we describe the critical quantitative researchers’ worldviews that guided critical approaches to quantitative inquiry. This table provides evidence supportive of critical approaches to quantitative inquiry as a viable paradigm shift for social justice–oriented quantitative researchers.
Showing three philosophical orientations for producing knowledge using quantitative research.
Critical Quantitative (CritQuant) and QuantCrit are Deeper than Wordplay
Within theme two, critical quantitative (CritQuant) and quantitative critical race theory (QuantCrit) are deeper than wordplay, we share the results of our comparative analysis of important similarities and differences between CritQuant and QuantCrit. This theme is based on the supportive evidence within our three subthemes: analogical reasoning, clearing up conflation, and applied practice-strategies used.
Analogical Reasoning
As we analyzed and coded the studies in our sample by their critical theoretical perspectives, we found uses of the terms critical quantitative, quantitative criticalism, and QuantCrit. Many scholars (n = 44 or 70%) used Stage (2007a, 2007b), Stage and Wells (2014), or Wells and Stage (2015) as anchor texts for the framing of their critical approaches to quantitative inquiry. This group of authors (quantitative criticalists) classified their work as critical quantitative or quantitative criticalism because of their foregrounding of diverse critical theoretical epistemologies. Some of the theories included critical race theory (Hernández, 2015; Sullivan et al., 2010; Teranishi, 2007); multicontextual model for diverse learning environments (Alcantar, 2014); college access framework (Oseguera & Hwang, 2014); QueerQuant (Garvey et al., 2019); critical queer theory (Metcalf, 2011), trans QuantCrit (Curley, 2019), and socialization of achievement (Young et al., 2018b). For these researchers, critical quantitative and quantitative criticalism was used interchangeably to indicate that critical epistemological, ontological, and methodological beliefs guided their inquiry.
Quantitative critical race theory (QuantCrit)
Critical race theory appeared as the most recurring theoretical framework in our sample (Covarrubias et al., 2018; Crawford, 2019; Garcia et al., 2018; Gillborn et al., 2018; Knowles & Hawkman, 2019; Sablan, 2019). Certainly, the critical analysis of race in quantitative research predates our bounded time parameters, 2007–2021 (see Du Bois, 1899; Zuberi, 2001; Solórzano et al., 2005). However, in our sample, we traced the use of CRT in quantitative studies to the published work Teranishi (2007) (disaggregating achievement and demographic data for Asian American students in U.S. higher education institutions). Additionally, there were thirteen studies (20%) that referenced Gillborn et al. (2018) as an anchor text for the framing of their study. For these thirteen studies, QuantCrit was not shorthand for quantitative criticalism but reflected a parallel movement of scholars whose epistemological genealogy is rooted in critical race theory that originated in critical legal studies (Covarrubias et al., 2018; Crawford, 2019; Garcia et al., 2018; Gillborn et al., 2018; Knowles & Hawkman, 2019; Pérez Huber et al., 2018; Sablan, 2019). These scholars’ definition of critical branch off from the line of critical theory that originated in the German Frankfurt school ((Stage, 2007a, 2007b; Sablan, 2019). For these scholars, the tenets of critical race theory guide their inquiry as they make more explicit connections to race and its intersections in their quantitative inquiries (Garcia et al., 2018; Gillborn et al., 2018; Sablan, 2019).
Gillborn et al. (2018) distinguishes this practice by adopting the label QuantCrit (the combining of CRT and quantitative methods) as shorthand for the approach and framing it through five early principles to guide researchers’ future use and analysis. The following five principles guide quantitative critical race theory, which are dynamic and in their infancy:
1)
2)
3)
4)
5)
Gillborn and colleagues (2018) do not consider these principles exhaustive but a starting point for researchers who want to resist perpetuating racist ideologies and its intersections.
Clearing Up Conflation
Critical approaches to quantitative inquiry are a direct challenge to quantitative research framed within the positivist paradigm (Garcia et al., 2018; Gillborn et al., 2018; Stage, 2007a, 2007b; Stage & Wells, 2014; Wells & Stage, 2015). However, these terms are not interchangeable. The term quantitative critical race theory (QuantCrit) first appears in the special issue QuantCrit: Rectifying Quantitative Methods through Critical Race Theory, edited by Nichole Garcia, Nancy López, and Verónica N. Vélez in 2018 in the journal Race Ethnicity and Education. The abbreviation QuantCrit refers to scholarship guided by the tenets of critical race theory (Gillborn et al., 2018). The terms Critical Quantitative and Quantitative Criticalism emerged from Stage’s (2007a) first special issue. In all three issues, scholars interchangeably used the terminology critical quantitative inquiry and quantitative criticalism to refer to the ways in which critical epistemologies, axiologies, and ontologies guided their research process. Quantitative criticalist refers to scholars who engaged in critical quantitative work (Stage, 2007a, 2007b; Wells & Stage, 2015).
To be clear, before there was QuantCrit (Garcia et al., 2018; Gillborn et al., 2018), some of the contributing scholars in Stage’s (2007a, 2007b) special issue classified their work as CritQuant or quantitative criticalism and used critical race theory to guide their inquiry (Hernández, 2015; Teranishi, 2007). Because the scholars in our sample have diverse critical perspectives, by name alone, conflation of what is meant by QuantCrit and CritQuant emerged within our review. To that end, we concur with Hernández (2015), in the necessity that “scholars clearly define what ‘critical’ means to them, what they are critical about, and how their definition has informed their work” (p. 99). That is, it is essential that scholars critically interrogate how they have come to understand the notion of criticality and explicitly describe how critical epistemologies guide their inquiry.
Applied Praxis: Strategies Used
Whether a researcher classified their work as critical quantitative or QuantCrit, there were eleven common strategies used to reveal and disrupt the systemic inequities experienced by historically marginalized groups, which we discuss next.
For us by us (FUBU)
One of the first patterns that emerged from our coding was that many critical scholars engaged in this work identify as members of historically marginalized communities who are reclaiming and constructing new narratives of themselves and their communities (Covarrubias et al., 2018; Pérez Huber et al., 2018; Rios-Aguilar, 2014; Tabron, 2019; Tabron et al., 2020, 2021; Teranishi, 2007; Young et al., 2018a, 2018b; Zuberi, 2001). Many researchers who have felt excluded, misrepresented, or dehumanized as a statistic positioned themselves within the critical quantitative paradigm. This lineage of “for us, by us” (Du Bois, 1899) evolved into “nothing about us without us” in the Disability Rights movement (Inckle, 2015) and later the trans community (Curley, 2019; Nicolazzo, 2017). Voices that have been historically situated in the margins are centered. The continuum of quantitative criticality, particularly for scholars of color, is a form of resistance to what has been considered the property rights of white scholars (Tabron et al., 2020).
Present strength narratives of historically marginalized communities
Within this critical quantitative research, scholars co-construct strengths-based counternarratives to resist white supremacy in quantitative inquiry (Garcia & Mayorga, 2018; Pérez Huber et al., 2018; Jang, 2018; Williams, 2014; Young et al., 2018b; Young & Young, 2018). They are focused on the achievement of Black girls (Young et al., 2018b; Young & Young, 2018), refuting homogeneity in Latinx scholarship by contributing as Chica@ scholars (Pérez Huber et al., 2018) and including narratives from and for the LGBTQ+ communities (Kilgo et al., 2019). Alcantar (2014) challenged traditional measures of civic engagement for Latina/o college students and shared how undocumented and resident Latina/os engaged in affirming behaviors that are not captured in the national data sets. Ryan Wells (2010) reframed the educational success of children of immigrants from assimilation using evidence on the absence of institutional mechanisms that impede students’ educational goals and aspirations. Similarly, Zerquera and Gross (2017) disputed the deficit narrative that Latina/o access and success is limited due to the individual’s background rather than policies, practices, and programs (i.e., cost of attendance, lack of a critical mass of faculty of color, or financial aid allocation) that do not meet this community’s needs. Presenting the strength narratives of historically marginalized communities is a prominent strategy used to refute racialized statistics, disrupt deficit narratives, and racial generalizations.
Ground truthing the data
Ground truthing is an interdisciplinary method most used in atmospheric science (meteorology) to confirm information at the site from data gathered remotely through machines. Ground truthing the data has translated in educational research as the practice of ensuring that: each step of the research process is driven by community expertise, particularly when the research is attempting to understanding [sic] phenomena connected to race and racism. Numbers can offer vital insights, highlight patterns, and convey particular analyses, but when they are decontextualized, ahistorical, and disconnected from the everyday lives of People of Color, they are hypothetical at best. The numbers must be groundtruthed in experiential knowledge. (Pérez Huber, Vélez, and Solórzano, 2018, p. 212)
Scholars who use groundtruthing are building on the intellectual legacy of W. E. B. Du Bois’s groundbreaking conceptualization and visualizations of the socially constructed color line that paved the way for geographic information system (GIS) mapping and big data analytics (Battle-Baptiste & Rusert, 2018; Du Bois, 1889; Toldson, 2019). For example, scholars such as Daniel Solórzano and Veronica Vélez (2015) were among the first to advance this work through their critical race spatial analysis to critically analyze sociospatial mechanisms “that reflect and operate as tools of white supremacy” (p. 426). Overall, education researchers within our sample used groundtruthing to co-construct counternarratives that uplift communities’ strengths (Solórzano & Vélez, 2015). Groundtruthing is a strategy in which the expertise of the communities informs the entire research process and gives the data its meaning (Du Bois, 1899; Pérez Huber et al., 2018; Solórzano & Vélez, 2015).
Critique social categorizations
Another strategy is critiquing the causal interpretations of social categories and outcomes in surveys (Garcia & Mayorga, 2018; Jang, 2018; Zuberi, 2001). Scholars within our sample have critiqued the construction of assumed and fixed categories that reproduce hegemonic privilege and marginalization (i.e., sexual and gender identities) (Garvey, 2019; Metcalf, 2011). The scholarship in this area has resulted in the new and more inclusive definitions for measures (i.e., student engagement, civic engagement, gender identity) and were not treated as standalone entities but intersectional experiences of BIPOC students (Castellanos and Cole, 2015; Covarrubias et al., 2018; Garvey et al., 2019; Hernandez et al., 2013).
Reject the oversimplification of aggregate data
Single group sampling and analysis to avoid homogenizing racial groups was another strategy (Conway, 2014; Teranishi, 2007; Vaccaro et al., 2015; Zerquera & Gross, 2017). Some between-group comparisons perpetuate gap gazing by situating white students as the comparison group for all other groups. This deflects deeper inquiry from the policies and practices that perpetuate deficit stereotypes (Young et al., 2018b). Robert Teranishi (2007) wrote about this issue with Asian American students in educational policy. Jemimah Young and colleagues (2018b) tracked these dynamics and its relation to math achievement and Black girls. Cindy Kilgo and colleagues (2019) and Garvey (2019) wrote about the masking of LQBTQ+ communities in aggregate data. Katherine Conway (2014) highlighted the importance of not aggregating children of immigrants in with the broader race/ethnic group.
Typically, single group effects in education research are decomposed using t-tests and analysis of variance (ANOVA). Both tests are dependent on the extent group differences affect individual differences. This is implicated in producing inaccurate generalizations about marginalized groups and whole communities. Instead, the single within-group analytic techniques frames variation based on students’ lived experiences and not between-group analyses, which often portrays BIPOC students as falling behind white students as the standard of measurement.
Build multidimensional statistical models
Researchers designed new, multidimensional models to replace widely accepted statistical and economic models that perpetuate inaccurate results, for historically marginalized groups (Stage, 2007a, 2007b; Perna, 2007; Wells, 2010). Complex constructs warrant the use of dynamic and multidimensional statistical models and “finer analyses for various subpopulations” (Wells, 2010, p. 1688) that better describe students’ experiences not adequately represented in statistical tests and results.
Critically interrogate measure validation
Enacting cultural and equity-minded approaches to validating surveys and not norming or validating their instrument among predominantly white male student populations was another strategy used (Alcantar, 2014; Hernandez et al., 2013; Knowles & Hawkman, 2019; Park et al., 2017; Kilgo et al., 2019; Teranishi, 2007; Malcolm-Piqueux, 2015; Covarrubias et al., 2018). Data sets that reinforced deficit and gap-dependent perspectives (Alcantar, 2014, p. 31) should be reinterpreted and reanalyzed using updated statistical approaches. These are sources of measurement error often overlooked in quantitative research. Alcantar offered recommendations that researchers engaged in critical approaches to quantitative inquiry should: (a) critically examine past models and studies in higher education and question the inclusive or exclusive nature of the models, research questions, and measures; (b) expand currently existing measures to capture nontraditional student activities; and (c) develop new models that include racial/ethnic identities and sociohistorical and policy contexts.
Ending the checkbox culture in survey design and administration
Researchers also tracked changes to federal and state data-gathering policies over time and how racially marginalized communities are characterized within large data sets (Allen & Wolniak, 2019; Wells, 2010). Scholars developed strategies for overcoming the checkbox culture of survey design that subsumed students’ identities under monolithic categories (Stewart, 2013; Jang, 2018) and excluded LGBTQIA+ (Curley, 2019; Kilgo et al., 2019; Garvey et al., 2019) and Latinx and students with immigrant backgrounds (Wells, 2010).
Presenting a numerically broad picture
These scholars also showed that quantitative researchers should use a variety of statistical analyses to produce multiple lines of evidence for estimating the accuracy of statistical outcomes and interpreted results for diverse groups (Allen & Wolniak, 2019; Wells, 2010; Kilgo et al., 2019; Mayhew & Simonoff, 2015a, 2015b; Ro & Bergom, 2020; Van Dusen & Nissen, 2020). Van Dusen ands Nissen (2020) showed the extent to which using different measures of the same construct can produce different yet conflicting outcomes.
Advanced heterogeneity of variance
Critical quantitative researchers accept heterogeneity of variance as an important assumption when measuring differences between groups (Alcantar, 2014; Allen & Wolniak, 2019; Wells, 2010). These researchers took a stand against homogeneity as a longstanding assumption of traditional quantitative research by addressing the “tendency to view heterogeneity of variance as a methodological nuisance rather than a source of important information” (Bryk & Raudenbush, 1987, p. 402). Diversity is inherently heterogeneous. Therefore, students do not need to have the same qualities for comparability. These researchers developed novel ways of modeling the contextual, structural, situational, and historical variability connected to students’ lived experiences.
Mayhew and Simonoff (2015a, 2015b) showed mastery of the theoretical linkages between one-way ANOVA and multiple group regression while challenging the within-group homogeneity assumption. Mayhew and Simonoff (2015a, 2015b) applied effect coding to model group membership based upon multiple racial identities rather than modeling each racial identity separately. This study illustrated the power of critical quantitative approaches for framing race as a multifaceted categorical variable by merging algebra and statistical analyses with the logic of CRT. Ro and Bergom (2020) updated Mayhew and Simonoff’s work to show how researchers can avoid using dummy coding that positions white students as a normative reference group for racially minoritized students. Effect coding proved useful for comparing minoritized students’ perceived social distance between themselves and white students.
Model specification and estimation
Regression analyses were often used to build and test models that deconstruct individual and interaction effects to estimate BIPOC student outcomes. The scholars in our study addressed the interaction effects of social categorizations based upon sex and gender (Van Dusen & Nissen, 2020; Garvey et al., 2019; Lopez et al., 2018; Mayhew & Simonoff, 2015a, 2015b; Metcalf, 2011; Park et al., 2017; Wells, 2010). Race/ethnicity appeared as a key area of consideration for Jang (2018), particularly for modeling the “multiple marginalized social categorizations” (p. 1270) of Cambodian, Vietnamese, Hmong, Thai, and Laotian Southeast Asian female students. Zerquera and Gross (2017) modeled the sociohistorical and policy context of Latinx students’ immigrant backgrounds using event history analysis to predict the timing and occurrence of enrollment.
Wells (2010) and Van Dusen and Nissen (2020) used hierarchical linear modeling (HLM) to deconstruct the multilevel effects of racialized inequity. Wells (2010) applied HLM to estimate the effects of school composition on immigrant students’ expectations. Cultural relatedness variables were included in the model to deconstruct Latinx students’ educational expectations. Van Dusen and Nissen (2020) included sexism instead of gender and racism instead of race to their HLM model to capture the marginalized experiences of female and BIPOC students. Van Dusen and Nissen (2020) operationalized academic achievement using the Equity of Individuality and the equality of learning measures and modeled the individual and interaction effects of relevant independent variables. the equity of individuality measure predicted larger learning gains for BIPOC students whereas student achievement gaps persisted within the results produced by the equality of learning measure. Based on these activities, researchers have creatively countered some of the most long-standing critiques directed toward traditional model building and estimation. They critically examined the misinterpretation of regression coefficients as one of “the most frequent error(s) of interpretation” (Judd et al., 2017, p. 151) and highlighted the predictor variables that should be included when modeling factors relevant to BIPOC students’ lived experiences (Williams, 2014).
As presented in our results, some of the scholars foreground critical perspectives and other methods in their work. Scholars who foreground diverse critical theoretical epistemologies had the strong theoretical grounding and understanding of critical theories (Hernández, 2015) to introduce new critical quantitative research questions and designs to achieve larger equity goals. Scholars who foregrounded methods had the advanced quantitative training needed to flip their positivist methodological training toward critical, deconstructionist, and transformative praxis. This led to different forms of methodological innovation and alternative pathways for conducting quantitative analyses (Best & Byrd, 2014; Sablan, 2019). To continue advancing this work, we concur with Hernández (2015) and Sablan (2019), who argued that scholars engaged in critical approaches to quantitative inquiry need both. Scholars engaged in critical approaches to quantitative inquiry should have advanced training and expertise in both critical theoretical perspectives and statistics to continue to push this emerging field forward. This work can begin by critically interrogating and more deeply understanding how instructors currently train and socialize graduate students in graduate-level statistics courses in education (Thomas, 2022).
Discussion
In this review, we shared why critical approaches to quantitative inquiry is a paradigm shift within quantitative research and a consolidation of a “centuries-long conflict between white domination and resistance by people of color” (Omi & Winant, 2014, p. 3). We highlighted the contributions of scholars who have engaged in CritQuant and QuantCrit between 2007 to 2021 to show areas for further development of the field. We identified and discussed distinctions between quantitative criticalism, QuantCrit, and critical quantitative, which are not interchangeable wordplay, and we presented several recommendations for applied praxis within critical education research. A continuum of criticality undergirded the entire research process of critical approaches to quantitative research. All the scholars engaged in critical approaches to quantitative inquiry in our sample consider theory and method inextricably linked to every phase of the research design process. All scholars engaged in this work focused on critical epistemological, ontological, axiological, and methodological orientations and methods in quantitative inquiry.
Embracing Heterogenous Orientations and Methodological Approaches
As scholars are locating themselves within critical approaches to quantitative inquiry, the epistemological roots of the researchers’ criticality differ and lend them to different theoretical perspectives that shapes their entire research process. This has resulted in multiple parallel yet complementary movements such as scholars whose criticality is rooted in the tenets of critical race theory (QuantCrit) (Garcia et al., 2018; Gillborn et al., 2018). There are also scholars engaged in quantitative inquiry informed by critical queer theory (Garvey et al., 2019; Kilgo et al., 2019). There is also an emerging scholarship that centers the trans community through Trans Quant Crit (Curley, 2019). Indigenous scholars have made valuable contributions to critical approaches to quantitative inquiry that have provided new insights into what it means to decolonize research methodologies (Kukutai & Taylor, 2016; Smith, 1999; Walter & Andersen, 2013; Walter & Suina, 2019; Wilson, 2008). Scholars engaged in critical approaches to quantitative inquiry are united in their resistance to positivist-informed quantitative inquiry that centers Western colonial thought as superior, but they may differ in how they engage in critical work based on how they have authentically located themselves within the work by understanding the roots of their criticality. Embracing multiple ways of knowing, being, doing, and competing perspectives aligns with critical approaches to quantitative inquiry and is another form of resistance to the positivist tradition that seeks to uncover probable and universal truth claims (Tabron & Thomas, in press).
Common Practices
There were several common practices throughout the studies in our review that we recommend in future studies. First, all scholarship began with asking critical questions and linked those questions to a critical theoretical perspective. Second, scholars explicitly and implicitly engaged in critical self-reflexivity of their positionalities and added critical and racial consciousness to their research. Third, scholars revealed how systemic racism and other forms of oppression remain undetected and perpetuated in statistical inquiry. Fourth, scholars critiqued and changed reference categories, deracialized statistics, and focused on context. Fifth, they introduced convergent historicism (Tabron et al., 2021), which focused on distinctiveness rather than generality by adding meaning, capturing complexity, and situating the data within its historical, cultural, and political contexts. Sixth, the scholarship led to creation of new and more inclusive variables and covariates in the statistical models. Seventh, all the studies in our sample centered the experiences of historically marginalized communities through strengths-based lenses.
What Are Not Critical Approaches to Quantitative Inquiry?
Although it is important to name the synergies within critical approaches to quantitative research, it is also critical to clarify what would not be considered QuantCrit or CritQuant based on the themes that appeared from the data. QuantCrit or CritQuant is not a label or buzzword for performative allyship. It is not QuantCrit or CritQuant if the eugenic history of statistics is not acknowledged and disrupted (Covarrubias et al., 2018; Gillborn et al., 2018; Tabron et al., 2020; Zuberi, 2001; Zuberi & Bonilla-Silva, 2008). Focusing on a racial or historically marginalized group while white supremacist logic is still perfectly intact is not QuantCrit or CritQuant (Tabron et al., 2020; Zuberi & Bonilla-Silva, 2008). Thoughtlessly accepting a secondary data set or big data without critical interrogation is not QuantCrit or CritQuant (Faircloth et al., 2015; Garcia & Mayorga, 2018; Rios-Aguilar, 2015). Conducting research about historically marginalized groups without being a member of the community or engaging their expertise through the literature or collaboration is not QuantCrit or CritQuant (Pérez Huber et al., 2018; Toldson, 2019). QuantCrit and CritQuant is not about finding or creating diverse ways to prove the superiority or inferiority of a group; it is about honoring the rich cultural histories and alternative ways of knowing, being, and doing. It is building upon those gifts to improve the conditions of historically marginalized communities (Teranishi, 2007; Toldson, 2019).
Conclusion
Critical approaches to quantitative inquiry are transformative because they are radically reflexive and relationally transparent (Anthym & Tuitt, 2019). It reflects a more inclusive spectrum of approaches that departs from positivist praxis based on natural selection or the survival of the fittest doctrines that informed early statistical practice. At times, resistant structures, groups, and people may push back against the transformative praxis of critical change agents (Anthym & Tuitt, 2019; Hernández, 2015). However, Baez (2007) invites us to ask, “How research in higher education is or can be critically transformative—that is to what extent educational research can offer critiques of our world that allow us to transform it” (p. 18). An essential charge for scholars who want to engage in critical approaches to quantitative inquiry to deeply reflect upon their positionality and how it shapes their research design and statistical interpretations (Covarubbias et al., 2018; Pérez Huber et al., 2018). They also challenge notions of researcher objectivity and statistics as numeric truth (Crawford, 2019; Garcia & Mayorga, 2018; Jang, 2018).
Despite framing some aspects of our emergent findings using Thomas Kuhn’s structure of scientific revolutions (1962), we do not view paradigmatic research as the end-all and be-all of quantitative knowledge. We learned that to effectively engage in quantitative inquiry from a critical quantitative perspective, there must be active work to develop a cogent understanding of ourselves outside of whiteness and its systems and structures. We needed to disrupt our tendencies to gaze at quantitative research through the established status quo. We also needed to question the extent our activities reify who should be centered and who should be at the margins while conducting quantitative research. As more scholars take part in critical approaches to quantitative research, they will expand on our work and create more approaches for strengthening this paradigm shift within quantitative research.
Finally, this systematic review ties in with our earlier scholarship using critical quantitative approaches to actualize an emancipatory praxis within introductory graduate statistics courses (Tabron et al., 2021; Thomas, 2022). Completing this review was helpful in showing us how much our own work needs to evolve. We infuse hooks’ (1994) statement as our definitional frame of reference for our future scholarship and praxis: The academy is not paradise. But learning [quantitative methods] is a place where paradise can be created [for faculty, practitioners, and students]. The [statistics] classroom, with all its limitations, remains a location of possibility. In that field of possibility, we have the opportunity to labor for freedom, to demand of ourselves and our comrades, an openness of mind and heart that allows us to face reality even as we collectively imagine ways to [engage quantitative methods] to move beyond boundaries, to transgress. This is [quantitative methods] as the practice of freedom. (p. 207)
We hope that our work serves as a catalyst for deeper acknowledgement of critical quantitative research approaches as legitimate pathways for thinking about and conducting quantitative research.
Footnotes
Acknowledgements
We would like to thank the anonymous reviewers for taking the time to review our manuscript. We appreciate their valuable comments and suggestions, which strengthened our manuscript.
Authors
LOLITA A. TABRON is an associate professor in the Department of Educational Leadership and Policy Studies at the University of Denver, 1999 East Evans Avenue, Katherine A. Ruffatto Hall 361, Denver, CO 80208; e-mail:
AMANDA K. THOMAS is a research assistant professor at the Department of Research and Innovation at the University of the Southern Caribbean, P.O. Box 175, Port of Spain, Trinidad, West Indies]; e-mail: