Abstract
Critical race theory (CRT) has been used in educational literature to emphasize the influence of racism on educational opportunity and the assets of students of color. Quantitative methods appear antithetical to CRT tenets according to some, but this article endeavors to show why this is not the case, based on both historical and contemporary notions. To build this argument, the author presents results from an empirical study that used data from a survey of undergraduates and measurement theory to quantify students’ community cultural wealth, a CRT framework that describes the cultural assets of communities of color. The author concludes with recommendations for incorporating quantitative methods into future CRT studies.
Quantitative data can drive public consumption of education policy research (Covarrubias & Velez, 2013). Given the saliency of racial inequality in education, the use of quantitative methods (QMs) to study and address problems of access and equity for students of color seems like a natural choice. However, in critical race theory (CRT) training, statistics and quantitative reasoning can be thought of as not bias-free endeavors but related to White supremacist origins (Bonilla-Silva & Zuberi, 2008). For the bulk of research that takes this critical lens, the use of strong statistical analyses is not only absent but, according to some, almost entirely antithetical (Carbado & Roithmayr, 2014).
A range of quantitative studies in education have examined issues of racial disparities. Contemporary research in the past few years has examined issues ranging from teacher-student racial mismatch to student discipline disparities and college affordability, finding the importance of race and racial inequality in explaining outcomes and experiences. The naive educational researcher may wonder why a “critical-quantitative” approach is necessary or conceptually distinct if issues of race are controlled for in most standard education models or ameliorating racial disparities is addressed in the policy implications section of most education research. A CRT student may wonder how the seemingly rigid methods of statistical and causal inference could possibly speak to the complexities of structural racism at the heart of critical race perspectives. These examples may oversimplify perspectives, but they highlight a central tension between CRT and quantitative methodology.
The purpose of this article is twofold. First, I outline how CRT, which emphasizes race and racism in inequality analyses, can be an appropriate framework for quantitative studies in the field of education research. To do this, I show that QMs have been an important, foundational component of critical legal studies and are thus a key part of educational studies informed by CRT. However, the extant literature suggests that current critical race quantitative work has not reached its full potential in its combination of theory and method.
The possibilities of critical QMs (Covarrubias & Velez, 2013; Teranishi, 2007) are demonstrated by way of a current empirical example. The second purpose is to present the initial quantitative measurement of community cultural wealth (CCW), an oft-cited CRT framework that emphasizes students’ assets.
I use this empirical example not as a paragon free of limitations but as a demonstrative case. I first review the tenets of CRT, including its foundations in legal scholarship. I then briefly review empirical work in the educational literature that has used CRT and posit some potential reasons for a perceived underrepresentation of quantitative work as well as examples of work that does exist. A quantitative case study of CCW is then presented, including methods and results. I conclude with observations of quantitative methodology and CRT for future research.
Critical Race Theory: A Review
The roots of CRT come from critical legal studies (CLS). This line of work acknowledged that attention to race was missing in legal scholarship, including the intersection of race with gender in discrimination law and the role of White supremacy and structural racism in legal progress (Crenshaw, 1996; Delgado & Stefancic, 1993). CRT is a method of legal analysis wherein implementation of the law is a tool to subordinate racial groups (Brown, 2003; Crenshaw, 1996). For example, while Brown v. Board of Education (1954) is seen as a win for desegregation, a CRT analysis discusses how the case is an alignment between White interests and Black goals (Bell, 2005). Thus, legal and civil rights progress is intimately tied to the establishment of White supremacy (Brown, 2003). A framework that emphasizes how White supremacy and racism structure policy and procedures could be extended from legal studies to studies in education (Ladson-Billings & Tate, 1995).
In the educational field specifically, CRT was adopted as a framework that emphasizes the centrality of race, racism, and White supremacy in describing educational structures and social practice (Ladson-Billings & Tate, 1995; Yosso, 2005). Previous theories of gender and class are insufficient for studying inequities, and instead, race should be a theoretically developed focus (Tate, 1997). For example, dominant education discourses may state that students of color are deficient in their achievement (Ladson-Billings, 1998), as opposed to recognizing the structures in education and the assets communities of color bring to school even when faced with oppression (Solorzano & Bernal, 2001; Solorzano, Villalpando, & Oseguera, 2005).
CRT’s role in education research typically adheres to foundational tenets, briefly reproduced here (Solorzano, 1997, 1998; Solorzano & Bernal, 2001; Yosso, Parker, Solórzano, & Lynn, 2004, pp. 3–4):
Racism, race, and its intersections (with gender, class, etc.) are an endemic part of society.
CRT challenges dominant frameworks and ideologies that are White centered or White supremacist in origin.
Scholarship works toward social justice, including the empowerment of oppressed groups and elimination of racism and poverty.
The experiential knowledge of people of color is a legitimate way of understanding the world, such as through storytelling.
CRT is inter- or transdisciplinary.
These tenets help frame an understanding of how CRT may function as both a theory and a methodology as well as how empirical research in education has used CRT to date.
Theory is defined as a representation of knowledge based on a systemized framework of concepts (Kezar, 2006). While the T in “CRT” implies that the above tenets are principles that guide inquiry, there is also acknowledgment of CRT as a methodology (McCoy & Rodricks, 2015). That is, the centering of the needs and experiences of people of color is not just a framework but a tool to collect and analyze data as well. Critical race methodology (CRM) is defined as a theoretically grounded approach that foregrounds race, racism, and intersectionality; challenges traditional research paradigms and texts; offers liberatory frameworks for subordination; and focuses on the experiences of students of color and interdisciplinary perspectives (Solorzano & Yosso, 2002).These tenets of critical race methodology are not explicitly qualitative, yet others perceive a preponderance of qualitative CRT research in the educational literature (McCoy & Rodricks, 2015). Part of this may be rooted in the role of stories and counterstories in the CRT tradition. Stories, in the form of parables, chronicles, poetry, fiction, and revisionist histories, are a primary tool for CRT scholars (Ladson-Billings, 1998). Stories are also a central component of indigenous communities and may reflect a shift in power away from the strict positivistic emphasis on qualitative and quantitative data (Brayboy, 2006).
While extant education literature and theory appear “natural,” the White-centered and majoritarian nature of much existing research diminishes the voices and stories of groups of color (Bernal, 2002; Solorzano & Yosso, 2002). Thus, a counterstory is defined as a method of telling stories for those people whose experiences are not often told in a way that challenges these “master narratives” of White privilege (Solorzano & Yosso, 2002, p. 32). Understanding how CRT operates as a theory and as a method helps frame its historical role in education research.
Critical Race Theory and Education Research
After early works suggested the transferability of legal theory to the study of education (Ladson-Billings, 1998; Ladson-Billings & Tate, 1995), empirical works sought to use this frame to study a range of educational problems. Qualitative research has used counterstories and other methods to discuss educational issues such as college choice, racial campus climate, microaggressions, challenges to affirmative action, and the experiences of students and faculty of color in schools and universities (DeCuir & Dixson, 2004; Solorzano, 1998; Solorzano, Ceja, & Yosso, 2000; Yosso et al., 2004).
Social science research on the whole is regarded by some as not fully equipped to reflect oppressed communities, including indigenous and colonized populations (Smith, 2012). CRT and QMs may seem incompatible in some respects, despite their importance in documenting discrimination. The following reviews some potential reasons why qualitative methods are often used for enacting CRT tenets. These reasons include the notion of objectivity and paradigms, historical uses of methods, how statistics are used and interpreted, and statistical education and training. I review these proposed reasons for the incompatibility of QMs and CRT and respond to each in turn.
One assumption about quantitative methodology is that it is a bias-free endeavor, which contradicts tenets that take a definitive stance on the role of race and racism (Carbado & Roithmayr, 2014; D. F. Carter & Hurtado, 2007). Since quantitative methodology tends to use positivistic paradigms, qualitative methods may be better suited for critical paradigms and alternative epistemologies (DeCuir-Gunby & Walker-DeVose, 2013). This interpretation, however, is simplistic and limiting because it equates methods with paradigms, which can and should be distinct (Creswell, 2007). While there may be a perceived relationship between the two, critiquing the limits of post-positivism does not negate the potential use of QMs.
Some may argue that CRT and empirical social science methodology are entirely mismatched when taking a stricter legal approach (Carbado & Roithmayr, 2014) and that qualitative methods are more readily available for unearthing counterstories, as in education research using CRT (Bernal, 2002). Since counterstories and QMs focus on individuals and quantitative methodology tends to emphasize group and summary statistics, QMs may be less appropriate (DeCuir-Gunby & Walker-DeVose, 2013). In an attempt to test models and hypotheses, QMs may oversimplify the relationships between variables (Labaree, 2003). CRT thrives on the recognition of the complexity of race relations. Just as the equating of critical paradigms and qualitative methods is limiting, so too is the equating of counterstory and qualitative inquiry. While narrative data are a component of counterstories, the response to majoritarian narratives is an equally important function of the counterstory. Responding to these dominant narratives can be accomplished through quantitative studies.
One possible response to this criticism is that there is often little difference between the methods used in post-positivistic approaches and critical approaches; rather, the researcher’s motivation is distinct (Stage, 2007). In this way, intent and intentionality guide methodological decisions in models and measures when taking a critical race approach, just as they do for the CRT researcher implementing qualitative counterstories.
QMs and social statistics also have a historical origin in the methods used to justify the biological inferiority of people of color and the eugenics movement (Zuberi, 2001). This historical aspect of QMs can contradict CRT (DeCuir-Gunby & Walker-DeVose, 2013) and make the distinction between motivation and method unconvincing. The work on the history of statistical methods is essential; however, this history does not beget contemporary methods. For example, this critique reflects an entire social science enterprise: Qualitative methods are not immune to a critique grounded in perspectives of racism and colonialism, such as the use of ethnography in stereotyping Indigenous and Native communities (Hau’ofa, 1975).
As causal inference is a key part of contemporary quantitative research in education (Murnane & Willett, 2011), problems also arise when trying to attribute causes to the socially constructed notion of race (Holland, 2008). One perspective on “policy-relevant” quantitative research is that such research identifies variables that can be isolated from selection bias and manipulated by policy change (Schneider, Carnoy, Kilpatrick, Schmidt, & Shavelson, 2007). CRT’s emphasis on the clearly nonrandom condition of race and the endemic nature of racism appears counter to such methods. However, there may be some value to studies that unearth the context of how policies and practices differentially affect groups.
Finally, strong teaching and graduate school preparation are essential, yet lacking, for quality use of QMs across educational fields (Henson, Hull, & Williams, 2010; Murtonen & Lehtinen, 2010), and possibly by extension also in CRT research. While little research is available regarding the experiences of students of color for QMs training in education research, graduate school mentorship in general for researchers of color, who are more likely to employ CRT frames, may be lacking (Davidson & Foster-Johnson, 2001; Gildersleeve, Croom, & Vasquez, 2011). Furthermore, racial differences in educational attainment and experiences are often attributed to explanations other than race/racism (Harper, 2012), suggesting a need for greater theorizing on race. Thus, an untested assumption may be that the intersection of QMs and CRT professionalization and training could be a hidden barrier to QM usage. Improving both QM training and CRT training for professional researchers is one response to this critique, rather than the dismissal of either of these approaches in research literature broadly.
Concerns regarding objectivity, historical origins, statistical interpretations, and training can limit the use of QMs in critical race research. At the same time, there are also calls for expanded mixed and quantitative methodologies in CRT work (DeCuir-Gunby & Walker-DeVose, 2013; McCoy & Rodricks, 2015) and increasing concern regarding the lack of attention to race and racism in education policy literature (Harper, 2012). Social science methods and empirical work in general are regarded as important drivers of CRT development (Carbado & Roithmayr, 2014), so ensuring the full range of methodological options is key. The following section reviews how researchers have used QMs in CRT scholarship and how this use can be improved. The case quantitative study presented later also demonstrates how to put these arguments into research practice.
The State of Quantitative Methods and CRT in Education
Researchers have recently explored QMs from a critical race perspective (Covarrubias, 2011; Covarrubias & Velez, 2013; Teranishi, 2007). In a CRT with QMs, descriptive and inferential statistics can be used to demonstrate CRT assumptions and document racial inequity (Milkman, Akinola, & Chugh, 2015). Before reviewing these sorts of studies, some attention should be paid to the origins of QMs and CRT.
Quantitative Criticalism and Critical Race Realism
A quantitative criticalist (Baez, 2007; Stage, 2007) has been defined as a researcher who uses QMs “to represent educational processes and outcomes to reveal inequities . . . to identify perpetuation of those that were systemic . . . [and to] question models, measures, and analytical practices, in order to ensure equity” (Stage & Wells, 2014, p. 1). This definition emerged from the line of critical theory that originated in the German Frankfurt school (Stage, 2007), whereas CRT as described above (a) tends to acknowledge CLS as its origin following the civil rights movement and (b) makes more explicit connections to the centrality of race and its intersections.
Critical race realism has also been described as a central part of CRT (Parker & Castro, 2013). It sees social science research as a foundation for making legal arguments about race-based discrimination and uses QMs in particular to complement the qualitative counterstorytelling of CRT (Parker & Castro, 2013). These notions are one starting point for considering QMs and CRT studies.
Empirical CRT Studies and “QuantCrit”
While the examples of quantitative research in education that have examined racial differences or controlled for race are surely many, few studies have taken a more critical approach. For the purposes of this article, I define a critical approach to QMs as one that emphasizes the assets of students of color rather than deficits and/or speaks to the overarching structure of racism and racial inequity (vs. individualistic determination) in framing, interpretation, and approach. A more explicit approach to CRT and QMs has been taken by studies that advanced a specific method self-identified by the authors under labels such as “QuantCrit,” “critical quantitative intersectionality,” and others (Covarrubias & Velez, 2013; Garcia, López, & Vélez, 2018).
In these works, topics such as student disparities have been explored, as well as microaggressions and racial battle fatigue (Covarrubias & Velez, 2013; Franklin, Smith, & Hung, 2014; Teranishi, 2007). By disaggregating the Asian and Native Hawaiian/Pacific Islander (NHPI) population by ethnicity, social class, and immigration status, Teranishi (2007), for example, uses a critical framework for analyzing achievement and demographic data.
Covarrubias (2011) argued for the use of “quantitative intersectionality” by examining how Chicana/o students progress through the educational pipeline. Using U.S. Census data on Chicana/os disaggregated by gender, class, and citizenship status, the report’s presentation of push-out rates illustrates how these students are failed by their educational institutions (p. 93).
This sort of empirical work influences what Covarrubias and Velez (2013) describe as a critical race–QMs intersectionality framework. This framework suggests that numbers are contextualized and do not “speak for themselves”; that quantitative analysis is grounded in experiential knowledge and standpoints; that research, like the tenets of CRT, is designed to advance social justice; and that transdisciplinary approaches are necessary (Covarrubias & Velez, 2013).
The transdisciplinary observation also reflects how critical-quantitative studies of race exist in disciplines beyond educational research, such as the study of stratification economics, which focuses on how intergroup disparities in economics can be studied with sociological and social-psychological concepts of racial bias (Darity, Hamilton, & Stewart, 2015). Rather than focus merely on human capital deficiencies, stratification economics recognizes the differences in structural resources that account for economic inequality (Darity, 2008). Psychological literature has also heavily influenced quantitative work on microaggressions and racial battle fatigue (Smith, Hung, & Franklin, 2011): Structural equation modeling aided researchers in testing how exposures to racial microaggressions contribute to psychological, physiological, and behavioral stress in students of color, using data from surveys including one using a Racial Battle Fatigue Scale (Franklin et al., 2014; Smith et al., 2011).
Current Limitations of CRT and Quantitative Research
One purpose of this article is to make the case that the use of QMs in CRT studies in education is important but leaves much to be desired. I earlier reviewed the important reasons advanced for why CRT and QMs are potentially incompatible and the shortcomings of these arguments. More recent work in the critical-quantitative realm has similarly discussed how QMs and CRT can be combined (Garcia et al., 2018). However, the establishment of compatibility does not fully develop the field. Limitations do exist in the methods of current research.
Much of critical-quantitative work is concerned with the premise that numbers are not neutral, statistics are not color-blind, and descriptive descriptions of educational statistics can unearth counterstories of people of color and their trajectories through education (Garcia et al., 2018; Gillborn, Warmington, & Demack, 2018). These premises align with CRT tenets, but more methodological guidance may be necessary for those seeking to move these premises into research practice.
For example, a substantial limitation of critical race QMs is that not all areas of quantitative methodology are fully used in CRT scholarship. A dominant focus has been descriptive and demographic statistics, with a few exceptions (López, Erwin, Binder, & Chavez, 2018). While demographic statistics, such as census data, are rightly criticized for not adequately responding to the needs of communities of color (Walter & Anderson, 2013; Zuberi, 2001), other forms of quantitative methodology can also adapt to culturally responsive methods.
Descriptive statistics highlight important outcome differences between groups, but they may do little to establish underlying causes or motivations that can guide policy change or the implementation of interventions. Experimental designs, for example, have documented racial discrimination in professors’ perceptions of prospective graduate students (Milkman et al., 2015). Similar studies with CRT frameworks could be potentially conceived. While there is some concern that causal modeling may misinterpret racial issues (Holland, 2008), predictive, experimental/quasi-experimental, and evaluative modeling is underdeveloped with regard to CRT.
Causal evidence for culturally relevant interventions is thought to be sparse (Dee & Penner, 2016), for example, though the theoretical development and empirical qualitative evidence are bountiful. This is not to say that one form of evidence is less important than the other. Rather, if we accept the premise, as argued in previous works, that using QMs in CRT research is a worthwhile endeavor (e.g., Covarrubias & Velez, 2013), then it is conceptually appropriate to envision a slate of studies that use a full range of QMs, beyond descriptive statistics.
Measurement theory is another such area that can be used to investigate critical race theories, one that will be explored more in this article. While the literature has considered culturally relevant measures for various racial and ethnic groups (Padilla, 2004), the explicit connection between these measures and CRT tenets and CRT as a theory could be more prominent. This sort of work can complement the work discussed above regarding the validity of psychological models of racism and stress.
Predictive regression models, causal inference via quasi-experimental studies, and exploratory and confirmatory theory building (e.g., exploratory factor analysis [EFA], structural equation modeling) are all areas that are yet to be fully developed in terms of using a critical race lens in education literature. For this article, I consider the area of measurement theory and its application to CRT for the following objectives.
Some may wonder if complex subjects such as CCW are “measurable.” Others may point to the origins of methods such as psychometrics as directly contradictory to critical tenets (Bonilla-Silva & Zuberi, 2008; Covarrubias & Velez, 2013). I will argue that when taken with an appropriate lens (Padilla, 2004), measurement theory, including survey methodology and scale development, can adequately contribute to critical race dialogues. This is due to the possibility that counterstories can be incorporated into scale development, and validation techniques can refine asset-based theories.
Self-report surveys, while limited (Bowman, 2010), are also an effective means of ascertaining the cultural and affective perspectives of students (Gonyea, 2005). In this way, individual self-report of CRT constructs combined with an appropriate theoretical framework and interpretive lens can also operate within the counterstory framework of CRT. I will demonstrate the potential of a quantitative counterstory by reviewing CCW as a response to dominant cultural capital.
Exploring the Possibilities: Community Cultural Wealth
For the purposes of this article, I turn to one specific approach in CRT scholarship, namely CCW. In this article, I emphasize the methodological development (vs. the results and theoretical implications) of the scale to bolster an argument for the use of QMs, measurement, and CRT. CCW is an appropriate case to examine the possibility of combining quantitative methodology and CRT given that (a) it is grounded in CRT frameworks (Yosso, 2005; Yosso & Solorzano, 2006) and (b) it is a well-cited yet empirically underdeveloped form of QM (Jayakumar, Vue, & Allen, 2013; Liou, Antrop-Gonzalez, & Cooper, 2009).
CCW refers to the assets students of color bring to schooling. The forms of CCW considered in this article are as follows (Yosso, 2005):
Aspirational capital: the ability to maintain hopes and dreams for the future
Familial capital: connections to and knowledge of family and kinship networks
Navigational capital: the ability to navigate through schooling institutions that were not designed with communities of color in mind
Resistant capital: the knowledge of and motivation to transform oppressive structures
CCW counters dominant notions of cultural capital. Bourdieu’s (1986) cultural capital theory specifies a system of social reproduction whereby the tastes and habits of the dominant class (or cultural capital) are most rewarded by schools. However, this application of cultural capital to marginalized students is typically thought of as deficit minded, where students’ culture (or lack thereof) is to blame for failure (P. L. Carter, 2005; Dixon-Roman, 2014; Ladson-Billings, 1998; Yosso, 2005). Dominant school settings may advance a working identity whereby marginalized groups such as students of color may feel that they need to do “extra” work to adhere to the standards of the dominant class (Carbado & Gulati, 2000).
While cultural capital is typically measured in education as characteristics of the dominant upper class, nondominant cultural capital (NDCC) acknowledges the assets that students bring from their home communities that may aid in academic achievement (P. L. Carter, 2003; P. L. Carter, 2005). CCW also relates to broader issues in CRT/CLS, such as racial capitalism: While universities may exploit the perceived value of a racially/ethnically diverse student body (Leong, 2013), there is often little emphasis on school-based strategies to engage students’ CCW (P. L. Carter, 2005; Deyhle, 1995; Yosso, 2005). As higher education is deeply rooted in the reproduction of White knowledge (Davis, 2016), understanding CCW from an empirical perspective can further shed light on not only how a CRT operates in practice but also how it can be better harnessed for liberation (Parker & Castro, 2013).
Quantitative operationalizations of cultural capital are also routinely criticized for oversimplifying Bourdieu’s (1986) theory or too heavily emphasizing high-status cultural participation (Winkle-Wagner, 2010). Thus, quantifying CCW may be one way to counter and reconsider the applicability of this theory to communities of color and move beyond reductionist notions of what counts and does not count as valuable cultural capital (Sablan & Tierney, 2014). CCW, therefore, is an appropriate theoretical example of using CRT and quantitative methodology as its application to the experiences of students of color can help elucidate instances of school-based discrimination and inequity, a central purpose of CLS and CRT.
The forms of CCW in this study are aspirational capital, familial capital, navigational capital, and resistant capital. These forms of capital, like dominant cultural capital, have an exchange value, but the ways in which schools value them and how students use them for further postsecondary benefits is up for future empirical study.
Specifically, I will operationalize CCW in four separate scales, following Yosso’s (2005) definitions listed above (for aspirational capital, familial capital, navigational capital, resistant capital). In the following section, I present an empirical example of a quantitative study of CCW to demonstrate the possibility of connecting QMs, specifically measurement theory and EFA, to critical race theories. This empirical example is provided as a tool for researchers to consider the previous arguments about the role of quantitative methodology in CRT by seeing it in practice.
A Quantitative Case Study of Community Cultural Wealth
The following outlines the data collection, sample, and survey development of a quantitative operationalization of CCW. The survey data reviewed below come from a larger project assessing CCW among NHPI and Asian students.
Data Collection and Sample
The following data come from an online survey of undergraduates in two open-access AANAPISIs (Asian American Native American Pacific Islander–serving institutions; N = 772) in the U.S. Pacific. The majority of these students were Pacific Islander or Asian American: 38.40% of the respondents were Filipino; 28.97%, Chamorro; 8.58%, Micronesian (Chuukese, Kosrean, Yapese, Pohnpeian, Palauan, or Marshallese); 3.93%, other Asian or Pacific Islander; 1.41%, White; and 18.71%, multiethnic/multiracial. The majority of the students were female, with approximately a quarter being male.
Over 60% of the students did not have a parent with a college degree, and the majority reported financial need and low family incomes. Thus, this sample is unique not only because of the large number of Native Pacific Islander and Asian American students but also by its representation of these students from the lower-socioeconomic strata. Much of the research on Asian or NHPI students assumes that this population is economically and educationally successful (Teranishi, 2010); the few critical race QM studies that exist encourage this sort of disaggregation to unearth information about populations typically neglected in education research (Teranishi, 2007). Because these students come from cultures with values corresponding to Yosso’s (2005) model (Buenavista, Jayakumar, & Misa-Escalante, 2009; Vakalahi, 2009), CCW is an apt framework for the sample surveyed.
Nondominant Cultural Capital Scales
A major component of this survey was a series of NDCC scales developed by me. I discuss the development of the scales and how they conform to the principles of measurement theory, namely reliability and validity, while also aligning with CRT. I then discuss the specific statistical analyses undertaken to further demonstrate the appropriate operationalization of CCW.
Development of the Scales
To create this scale, I operationalized Yosso’s (2005) CCW in four separate scales: (1) aspirational capital, (2) familial capital, (3) navigational capital, and (4) resistant capital. Each scale had seven to eight items, which asked students to agree on a 6-point scale with how a statement described them (ranging from 1 = Not at all like me to 6 = Exactly like me). The items were meant to capture aspects of that particular form of NDCC (e.g., “I know how to find resources at my college”—navigational capital). The following practices helped establish validity: content validity testing, expert reviews, pilot testing, and cognitive interviewing.
To address content validity (Allen & Yen, 2002), I conducted a review of the literature including the concept of NDCC and CCW. This work was also informed by the literature on Native Pacific Islander and Asian American higher education (Museus & Chang, 2009; Underwood, 1987; Wright & Balutski, 2013). After determining a set of definitions and content parameters, I generated items that reflected these components (Table 1). The items for the scale were also reviewed by expert reviewers, including academics with expertise in cultural capital theory and community leaders with expertise in Pacific Islander culture. This type of review by cultural community leaders is argued to be essential for culturally responsive quantitative research on multicultural populations (Padilla, 2004). The reviewers assessed the items for the content and coverage and provided feedback on ways to revise items or add content areas. While the final decision on the items and wording was at the discretion of the researcher (DeVellis, 2012), comments were incorporated as appropriate.
Reliability, Construct Definition, and Representative Items of Nondominant Cultural Capital Scales
A small group of Asian American or NHPI students at a different institution but not in the final study population were asked to complete a pilot survey in order to assess the initial survey and NDCC items. This pilot was intended to test the dependability of the online survey design as well as the extent to which students may fatigue in the survey. All of the pilot students completed the online survey; all the students agreed with the statement that the survey questions were understandable, and the majority felt that the survey was of an appropriate length for undergraduates. Some feedback given from the pilot regarding the layout and content of the questions was also incorporated into the final survey design.
Individuals were also asked to complete cognitive interviews. Cognitive interviews are a process by which respondents from a similar population as that of the intended survey sample are asked questions related to their interpretation of the survey questions (Desimone & Le Floch, 2004; DeVellis, 2012). This session is in effect a “think aloud” where respondents are asked to take the survey in person and to respond to questions about the survey questions. Cognitive interview participants explain what they think the question means and how they would arrive at the answer. They also suggest ways in which the questions can be improved by identifying words or phrases that are confusing or by offering interpretations of questions that are found to be not aligned with the theoretical content intended by the researcher. The insights from the completed cognitive interviews were used to assess the degree to which the items were reflective of the theoretical components they were attempting to operationalize. Questions that were unclear were reworded or deleted, provided that enough questions remained to adequately measure the concept. While a full-fledged qualitative study was not conducted prior to developing the items, these various techniques—expert reviews, pilot survey, and cognitive interviews—helped determine the final sets of items for the NDCC scales as well as the overall survey length and content. In addition, the qualitative studies of CCW cited above also informed scale development, along with consultation with community cultural experts.
Instrument Analysis: Classical Test Theory, Reliability, and Factor Analysis
I used the assumptions of classical test theory to analyze the latent constructs of CCW. This assumes that the observed score is composed of the true score plus error. The term latent construct refers to traits that are by their nature unobservable and lack a discrete measurement. The assumptions of this measurement theory are used to justify how a student’s responses to a set of items can generate a score that approximates the student’s NDCC. Using these assumptions provides guidance on how to assess the suitability of items. Reliability tests and EFA were conducted.
I have already discussed how I addressed content validity through the survey development process. Another way to establish validity empirically is through factor validity, or through the use of factor analysis (Allen & Yen, 2002). For this study, I use EFA to examine the NDCC scales. The goal of factor analysis is to determine how underlying factors predict the variance in the scale items. Items (xn) that are being factored can be represented by the following equation (Russell, 2002):
where F denotes the common factors that underlie the item analyzed, U denotes the factors unique to the item, w is the loading of each item on the factor, and e is the random measurement error of each item.
To assess how the items comprise underlying factors, I examine the structure coefficients, or factor loadings, as a measure of how related the items are to each other (Acock, 2010). In this study, I use a threshold of structure coefficients greater than .40 (Acock, 2010) to assess the suitability of the items. EFA is highly discretionary, with extrastatistical decisions being made by the researcher (Henson & Roberts, 2006; Schmitt, 2011; Worthington & Whittaker, 2006). These decisions include extraction method, rotation, and items per factor, all considerations I made in conducting the study.
This empirical example is not without its limitations. Validating a scale for use across samples rarely occurs with one study (Allen & Yen, 2002; DeVellis, 2012), and even validated scales may require analysis with diverse populations to assess their applicability to different contexts (Kim, Atkinson, & Yang, 1999; Okazaki & Sue, 1995; Worthington & Whittaker, 2006). Despite these limitations, the sampling procedure for this study was determined to be effective for carrying out the design, sufficient for making inferences regarding measurement and associations, but limited in its generalizability to other contexts in future research.
These limitations make it clear that more work is needed to investigate fully a nascent measurement of a concept such as CCW. Still, this article maintains through its conclusions that operationalizing CCW in a quantitative approach is a worthy endeavor for studying communities of color from an asset-based perspective. This approach can substantively improve on previous quantitative studies that rarely incorporate these theoretical models and may rely on theories bounded in rationality or dominant culture that are not as responsive to communities of color.
Results of the CCW Study
I demonstrate that a novel scale of NDCC, operationalized through CCW, has preliminary reliability and validity evidence. The following are the coefficients α for the separate seven- to eight-item NDCC scales (Table 2): aspirational capital: α = .79, familial capital: α = .87, navigational capital: α = .83, and resistant capital: α = .78.
Exploratory Factor Analysis of Nondominant Cultural Capital Scales
Coefficient α is presented as a fraction measured on a 0 to 1 scale, where a value of 1 is considered perfectly reliable and a value of 0, perfectly unreliable (Kline, 2005). Coefficients greater than .70 are used as a common threshold for reliability measures in social science methods (Acock, 2010). Given the preliminary and exploratory nature of this study, I consider the range of α coefficients (.78 to .87) as sufficient evidence of scale reliability.
For each subscale, I conducted an EFA using principal component factor analysis followed by promax (or oblique) rotation in order to (1) extract a factor that best represents the variance of the items and (2) account for the potentially correlated nature of the multiple factors that may emerge. Kurtosis values for each item were examined, and none exceeded the recommended thresholds. These analyses and the following results imply that the process of producing content-valid items could result in internally consistent scales; these items, through further validation work, can be used to answer research questions or assess issues of culturally relevant campus climates in future critically minded studies (Museus, 2014).
EFA Results for Navigational and Familial Capital
Navigational capital and familial capital were measured without any modifications needed. In other words, there is satisfactory evidence that these items measure an underlying factor, and using CCW as a theoretical guide could justify navigational or familial capital as the respective potential underlying factor. As explained in Table 2, eigenvalues, screeplots, and factor loadings were all assessed to make this determination. For familial capital, all the items proposed for the survey loaded onto one common factor (eigenvalue = 4.21). The first factor explained 52.58% of the variance in the items. Structure coefficients ranged from .67 to .79, which are considered high loadings.
The EFA for navigational capital also showed that all the items proposed for the survey loaded onto one common factor (eigenvalue = 3.56). The subsequent factor would have an eigenvalue of .81, indicating the initial factor was appropriate to retain. This factor explained 50.84% of the variance. Structure coefficients ranged from .61 to .77.
The loadings for familial capital and navigational capital, along with the eigenvalues and screeplots, suggest that the items are all well predicted by the underlying factor, which explains a large percentage of the variance in the items. Coupled with the high reliability coefficients, the evidence suggests that the proposed items hold together well as a measure of an underlying factor (i.e., familial capital or navigational capital). This suggests empirical support for a theoretically driven measure of CCW that could be used in further empirical study. However, the empirical results indicated that some items for aspirational capital and resistant capital may need modification to best capture a measurement of these factors.
EFA Results for Aspirational Capital
Aspirational and resistant capital contained items that did not sufficiently load onto a factor, and a decision was made on the best structure and items to include (Bandalos & Finney, 2010). This presents the researcher with a decision to delete items from a scale to retain the items that the empirical results suggest better represent the underlying factor structure. If this process is done, EFA and reliability coefficients should be rerun on the retained items to confirm a better fit.
The initial principal component factor analysis for all the items of aspirational capital revealed two factors with eigenvalues greater than 1 (eigenvalue1 = 3.42, eigenvalue2 = 1.45). The first factor accounts for 42.93% of the variance in the items. Structure coefficients ranged from .42 to .90. Items for this factor solution cross-loaded, or could be a part of more than one factor, and were subsequently examined in the rotation. Following the oblique rotation, items that did not load onto the single factor were removed, and the scale was reexamined. A final total of four items was thought to comprise one factor (eigenvalue = 2.44). The new screeplot also showed a one-factor structure. Structure coefficients ranged from .63 to .85. This factor explained 60.94% of the variance in the aspirational items. The three items removed concerned aspirations from or related to the family, including the aspiration to surpass parents’ educational and occupational success. This suggests that although they are part of the definition, an empirical fit may be better achieved through the retained items. More research or consideration may be needed regarding the applicability of the definition of aspirational capital and how best to measure it.
EFA Results for Resistant Capital
For resistant capital, a distinct two-factor structure emerged. This two-factor solution explained 41% of the variance by the first factor and 18% of the variance by the second factor. The eigenvalues for the factors were 3.24 and 1.42, respectively. In the rotated solution, the items were shown not to cross-load but instead to cluster around the two factors. The factor loadings for each of the items on their respective factors ranged from .72 to .90.
Another principle of factor analysis involves examining question content to explain the empirical patterns in the retained factors. After reviewing the results and reconsulting with the literature, the two factors of resistant capital were labeled as follows: (1) identification of oppression in society and (2) motivation to transform oppressive structures. These labels reflect how resistant capital is described in the CRT literature (Solorzano & Bernal, 2001), given that various resistant strategies are available to youth of color. Eigenvalues, screeplots, and factor loadings were again assessed to confirm this interpretation.
This factor structure may be supported by the wider CRT literature, which has discussed various types of resistance available to communities of color that vary in their approach to transformative social justice. For example, students may display reactionary behavior, self-defeating resistance, conformist resistance, or transformational resistance (Solorzano & Bernal, 2001; Yosso, Smith, Ceja, & Solorzano, 2009). That resistant capital comes in varied forms lends theoretical support to the notion that students display varied expressions of oppositional behavior, which may include, and distinguish between, both knowledge of and motivation to act against social injustices.
Measuring Community Cultural Wealth
This empirical example demonstrates the possibility of using the directives of measurement theory in analyzing a CRT issue of CCW among marginalized communities of color, such as NHPIs and Asian Americans. Satisfactory factors can be scored and then used in appropriate multivariate models, such as regressions, that address the specific content of the research questions.
The results reviewed above advance the second purpose of this article, which was to demonstrate an empirical example of CRT and QMs. While they are not reviewed here, additional multivariate analyses with the retained factor scores were conducted in a full analysis of college access and CCW, a process that can help advance critical quantitative methodology goals in documenting and addressing campus inequity issues.
To review, an EFA of an operationalization of CCW demonstrated that items assessing aspirational capital, familial capital, navigational capital, and resistant capital have preliminary reliability and validity evidence. Some alterations may improve the empirical fit, particularly as it relates to resistant capital and its alignment with the CRT literature. This study illustrates how to design and implement projects using critical racial frames and measurement theory.
Conclusion: Revisiting CRT and QMs
Qualitative methods are not the only way to present counternarratives. This analysis combines the tenets of asset-based critical race theories with the methodological considerations of QMs, specifically measurement theory. The conclusion revisits the discussion on the relationship between CRT and QMs, demonstrates the potential for measurement of CRT concepts, and briefly reviews the implications for policy and practice. As one tenet of CRT in education is the advancement of social justice and as QMs can inform policy discourse, making practical connections between theory and methods should also be modeled in critical-quantitative studies.
Conventional quantitative studies in education and education policy research may fail to adequately engage issues of race. Cultural assets are not operationalized, leaving way for deficit interpretations. Institutionalized racism is absent from theoretical frameworks and interpretative explanations. However, critical QMs may fail to engage the full potential of QMs, instead relying on the establishment of nonneutrality and methods-theory compatibility. The following discussion expands on this tension.
Unresolved Tension?
The first purpose of this article was to discuss these issues regarding QMs and CRT. Contrary to the notion that QMs and CRT are incongruent, there are possibilities for their being compatible. Some previous examples in the educational literature have concerned intersectional approaches to educational attainment among racial and ethnic groups or the effects of racism on students’ well-being. However, much of this potential is unrealized. Many works to date that explicitly claim a critical race quantitative frame are descriptive in nature. Combining conventional practices in QMs, such as the reliability and validity techniques of measurement theory, can produce theoretically and methodologically driven studies. Additional dialogues can more fully consider other robust quantitative techniques and their applicability to critical race framing.
The second purpose of this discussion was to illustrate an example of CRT as a framework for quantitative studies. CCW was highlighted in this particular piece because, as a framework, it can in many ways illustrate the tensions of the initial dialogue on QMs and CRT. The work on CCW is theoretically robust and experientially rich. I presented this study to contribute to the empirical base, but I am cognizant of the tensions involved in doing so. If CCW can be measured, is CCW an appropriate “variable” for statistical modeling, and do correlational analyses oversimplify the framework’s complexity? These are understandable concerns.
While students’ aspirations for college and their connections to their culture and family have been explored in other studies (Rios-Aguilar, 2010), there is still a potentially worthwhile discussion on measuring and analyzing CCW among various groups of students and empirically investigating— through multiple methods—how CCW is an appropriate framework for educational notions such as college access and readiness. To address the potential concerns raised above, CCW can be credibly measured through additional research, explored in multivariate analyses, and still be confined to the limitations of any and all statistical analyses. Future research studies on quantitative analysis of CCW could explore the scales in various populations, examine how factors hold up across intersectional identities (e.g., ethnicity, gender, class), and consider confirmatory factor approaches based on how qualitative studies have suggested CCW is manifest among communities of color.
Another tension or question reflects the polar perspectives presented at the onset of this article: Researchers may wonder what makes QMs critical. Part of the challenge of critical QMs is that students yearning for direction may be unclear on how to put quantitative analysis into practice. On the one hand, the actual practice of statistical analysis—that is, the running of a regression model or structural equation model, for example—may appear similar across “critical” and “noncritical” studies. For example, factor loading thresholds and rotation methods may look the same, raising the question whether an EFA is truly critical if there are few implemented methodological differences. On the other hand, these conventional practices have long been interpreted in ways that run counter to CRT tenets.
One response from Stage (2007) reviewed above is that intent is the driver rather than methods. CRT-informed studies of CCW, for example, are driven by the notion that students bring cultural assets to schooling that, while important to home communities, are not only not measured but also not valued by the dominant school culture. This intent contrasts with measures meant to examine constructs that are not explicitly critical and in practice counter to social justice.
At the same time, providing too few methodological guidelines can leave an empirical gap, where CRT-intending scholars have few quantitative cases, with little methods diversity, to turn to for exemplars. There may likely be methodological practices beyond what is conventionally taught in education and social science methods courses that attend to issues of race and racism better than others. Rather than the doing of critical-quantitative work across theoretical frameworks, methods, and topics, critical-quantitative scholars have exerted much effort in establishing the compatibility of methods and theory by emphasizing the nonneutrality or even racialized dangers of numbers. This article’s discussion calls for more methodological doing and teaching while also highlighting the important practice implications that arise from studies that are critically informed and intended.
Beyond the implications for research, this exercise could also potentially aid practitioners in understanding how institutions can be designed with students’ cultures in mind (Museus, 2014). Critical measurement issues have implications for policy as well. Recent work on noncognitive—or nonacademic—indicators reveals that policymakers are paying attention to alternative measures to high-stakes decisions in K–12 accountability or college admissions decisions. Yet critical scholars have critiqued the potential for deficit interpretations of such constructs at the same time as measurement scholars have urged caution regarding accurate psychometric properties (Almeida, 2016). CRT studies are ripe for such translation to policy and social change by using well-designed quantitative work to challenge such policy discourse.
My hope is that critical QMs can move beyond simple descriptive statistics of racial difference, to inferential, measurement, and theoretical modeling, and that QMs in education and education policy research recognize the limits of analyzing race without counterstory contexts. These models would not just document or describe relationships in data but also test theories from a critical race perspective and operate from assumptions that emphasize students’ cultural assets.
Footnotes
Notes
J