What Have They Done? An Analysis of Data Generation Methods Funded by the Institute of Education Sciences

Abstract

On December 2, 2020, the Institute of Education Sciences (IES) published a summary report from their technical working group titled “Increasing Diversity and Representation of IES-Funded Education Researchers.” A major barrier to diversifying funded researchers described was that many scholars did not believe the institute funded researchers “like them.” In this article, we sought to better understand this perception by analyzing funding patterns of IES between the years of 2008 and 2022. Specifically, we analyzed the data generation methods (e.g., quantitative, qualitative, or mixed) funded to better understand IES priorities accordingly. Interpreting our results through a lens of critical race theory, we show that knowledge generated through methods that include stories and narratives are marginalized by IES funding priorities.

Keywords

content analysis critical theory diversity race textual analysis

It is important to reinforce that the concept of an epistemology is more than a “way of knowing.” An epistemology is a “system of knowing” that has both an internal logic and external validity.

—Ladson-Billings (2000, p. 257)

As Ladson-Billings (2000) explained, epistemologies should be understood as systems of knowledge rather than individual ways of knowing. In other words, the relationship between knowledge and truth should be considered as a process that supports a certain type of “internal logic,” which is reified through external validations. In relation to this article, we present an analysis of the ways the Institute of Education Sciences (IES) influences and reifies knowledge creation beyond its own walls by monetarily incentivizing certain forms of data generation methods. Specifically, we were interested in answering the following research question:

Research Question: What data generation methods were described within awarded grant projects funded by IES over a 15-year period (2008–2022)?

Interpreting the results of our analysis using critical race theory (CRT), we show how the internal logic of IES perpetuates an external invalidation of education researchers who understand story, narrative, and experiences as “valid” forms of knowledge creation. In short, our analysis confirms what Milner (2008) argued is an “epistemological issue as much as a conceptual one” (p. 332): IES funders seem to appreciate specific forms of data—those aimed to be used for replication—rather than those aimed at uncovering nuanced experiences within social settings (e.g., schools). Moreover, results of our analysis validate recommendations made by members of a technical working group (TWG) organized within IES that sought to, among other things, increase participation from “diverse groups” in IES-funded research projects.

Defining Truth by Generating Certain Types of Knowledge

On December 2, 2020, IES published a summary report from their TWG titled “Increasing Diversity and Representation of IES-Funded Education Researchers” (Chhin & Stapleton, 2020). Described as a product of the National Center for Education Research (NCER), the report was compiled based on presentation slides and personal notes taken during the TWG, and the views expressed in the document “reflect[ed] individual and collective opinions and judgments of the presenters and participants at the meeting and are not necessarily those of IES or the U.S. Department of Education” (Chhin & Stapleton, 2020, title page). The TWG focused on four topics: (a) reducing barriers to participation in IES research, (b) expanding outreach, (c) increasing outreach, and (d) increasing research capacity. As it relates to the first topic, TWG members described a perception that a major barrier to increasing the diversity of applicants and therefore grantees to IES-funded programs is that many scholars do not believe the institute funds researchers “like them” (Chhin & Stapleton, 2020, p. 8).

Specifically, TWG members explained, “There is a perception that the focus of their research, their research methods, their institutional affiliation, and/or their demographic backgrounds are at odds with IES’s priorities” (Chhin & Stapleton, 2020, p. 8). To change this perception, TWG members offered numerous recommendations, including but not limited to changing IES communication strategies that at minimum would include updating competition announcement instructions and scoring criteria, targeting recruitment toward researchers and institutions that have not historically received IES funds, implementing needs assessments with historically marginalized groups, and developing explicit topics or competitions around “dismantling systemic inequality in education” (Chhin & Stapleton, 2020, p. 8). As the TWG members explained, these recommendations were to be understood not for their individual merit but as a collective project with each recommendation presupposing the other in terms of importance. Moreover, the TWG members recommended “revisiting what knowledge is in educational research and what types of backgrounds and experiences are important for researchers (e.g., emphasizing the role of qualitative research in educational research)” (Chhin & Stapleton, 2020, p. 8). As members of the TWG explained, qualitative data hold unique value in raising up specific forms of knowledge and expertise that lie outside of “academic credentials, such as understanding community values, norms, and needs” (Chhin & Stapleton, 2020, p. 9). If there is a perception that qualitative data are not appreciated by IES funders, the knowledge that these data generate should be further discussed.

The Core Mission of IES

“At its core, scientific inquiry is the same in all fields . . . a continual process of rigorous reasoning. . . . It builds understanding in the form of models or theories that can be tested (U.S Department of Education, Institute of Educational Sciences & National Science Foundation, 2013, p. 7). This quote, taken from the IES Common Guidelines for Education Research and Development, describes the sort of research IES was created to undertake. Like institutes in other fields of science, both natural (e.g., biology, physics, medicine) and social, IES was created to produce knowledge through education research that could be applied, replicated, and tested across contexts to understand “what works for whom and under what circumstances/conditions” (Schneider, 2018). To carry out this goal, researchers supported by IES funding must ensure their activities are “objective, secular, neutral, nonideological and are free of partisan political influence and racial, cultural, gender, or regional bias” (U.S. Congress, 2002, p. 6). What the members of the TWG seemed to understand, however, is that knowledge generated by IES funding is never “neutral” but represents the goals and objectives of individuals within the institute—what Ladson-Billings (2000) described as an “internal logic.”

Although IES is by no means the only funding option for education researchers in the United States, the Institute is the representative body of the federal government and therefore has unique monetary power to validate and spread the internal logic of the Institute through external funding. To do so, IES is comprised of multiple offices and institutes that work in tandem to determine how federal funding for education research will be awarded. Housed within the U.S. Department of Education (USDOE), IES is headed by a presidentially appointed and senate-confirmed director and two deputy directors. The National Board of Education Sciences oversees the Institute and is composed of 15 presidentially appointed and senate-confirmed experts. According to the IES website, these experts “advise the Director on activities to be supported by the Institute, including the general areas of research to be carried out by the National Center for Education Research (NCER)” (IES, 2023). NCER is one of four centers supporting researchers through IES funding, the others being the National Center for Education Statistics (NCES), National Center for Educational Evaluation and Regional Assistance (NCEERA), and National Center for Special Education Research (NCSER). NCER, however, houses most of the funded research projects out of IES.

Who Applies for IES Funding?

A few months before the TWG report, IES published a “diversity statement” that began with the following reminder:

IES was established by the Education Sciences Reform Act of 2002 (ESRA – P.L. 107-279), in part to improve academic achievement and access to educational opportunities for all students. In carrying out this mission, IES takes steps to ensure that our work is carried out in a manner that is free of racial, cultural, gender, or regional bias. (IES, 2020)

The writers of the statement went on to present how the work of IES benefits from “diverse perspectives” that cannot be achieved without the Institute’s ability to “attract, train, partner with, and support talented researchers, statisticians, and evaluators from all backgrounds” (IES, 2020). Such ability was testament to IES leadership’s “commitment to the many facets of human diversity that shape our lives, including age, gender, gender identity, sexual orientation, culture, race, ethnicity, religion, disability, and socioeconomic status” (IES, 2020). Moreover, writers of the statement described specific ways this sort of commitment was translated into action.

For instance, IES established Pathways to Education Sciences to award minority-serving institutions with grants that create educational-research training programs. The Institute also includes “diversity recruitment requirements” for all Research Training Programs (IES, 2020). Moreover, IES funds research to

address the challenges faced by students in underserved groups. Recently funded studies address disproportionality in school discipline, professional development to train educators on implicit bias and strategies to increase equity in school discipline, and the use of culturally and contextually relevant academic and behavioral practices. (IES, 2020)

Data presented by the TWG reveal that the intentions of IES to increase diversity in its grant funding is a slow-moving process.

Table 1, adapted from the TWG report (Chhin & Stapleton, 2020), shows the racial/ethnic breakdown of people awarded grants through Pathways to Education Sciences—touted in the diversity statement as testament to IES’s commitment to diversity and inclusion. These are the most recently available data according to the program representative from IES (personal communication, 2024).

Table 1

Race/Ethnicity of Pathway Fellowship Applicants

Race/Ethnicity	Pathways to Education Sciences (2016–2020), N = 251
American Indian/Alaskan Native	4%
Asian	3%
Black/African American	45%
Native Hawaiian/Pacific Islander	0%
Multiple/multiracial	21%
White	14%
Not reported	13%
Hispanic	37%

Source. Adapted from Chhin and Stapleton (2020).

Table 1 shows that since 2016, applicants from minoritized backgrounds have been awarded the most Pathways fellowships. Although this is indeed positive news, analyzing the Pathways program with a critical eye reveals some interesting facts. First, the Pathways program is much shorter than other predoctoral fellowships. Although Pathways funds fellows anywhere between 8 weeks and 1 year, the Predoctoral Interdisciplinary Research Training Program awards 2- to 4-year grants. Moreover, the Interdisciplinary Research Training program has existed since 2004 and therefore has funded many more fellows (N = 1,008 as of 2020). Of those funded, 75% (n = 767) were reported to identify as White (Chhin & Stapleton, 2020, p. 6). To increase applications from racially marginalized backgrounds, the TWG recommended IES funders target application outreach to minority-serving institutions (MSIs).

Elizabeth Albro, the commissioner of NCER explained, “Annually, NCER manages a grant portfolio of approximately $156 million, and NCSER manages a grant portfolio of approximately $56.5 million” (Chhin & Stapleton, 2020, p. 7). According to Chhin and Stapleton (2020), between 2013 and 2020, 4% of applications to NCER and fewer than 1% of applications to NCSER were from MSIs. Table 2 disaggregates these data by showing the specific sorts of MSIs funded by NCER.

Table 2

Disaggregated NCER Instututional Funding

Minority-Serving Institutions	Number of Institutions Funded
Hispanic-Serving Institutions	16
Asian American and Native American Pacific Islander-Serving Institutions	9
Historically Black College or University	1
Predominately Black Institution	1

Source. Adapted from Chhin and Stapleton (2020).

It should also be noted that no MSIs received funding from NCSER between 2013 and 2020 (Chhin & Stapleton, 2020).

Tables 1 and 2 present examples of how members of the TWG found a lack of diversity in applications and therefore who gets funded by IES. Based on this, the group recommended numerous initiatives IES could undertake to change. The following were recommended changes to the structure of IES grant processes that align with the goals of our article:

Consider the value of qualitative research methods in conducting education research.

Recognize that conducting qualitative research often requires expertise other than academic credentials, such as understanding community values, norms, and needs.

Require all grant applicants to demonstrate how their research is relevant to and meets the needs of the communities involved in the research, including key stakeholders and the individuals or groups that the research is meant to address. (Chhin & Stapleton, 2020, p. 9)

With these recommendations, the TWG identified both epistemological and ontological differences in how decision makers at IES currently understand and value education research. Specifically, the report highlighted the lack of appreciation for the unique experiences many members of communities who have been historically marginalized hold. With the passing of the Congressional Act of 2002, IES was established to generate specific sorts of knowledge based on a specific internal logic—one that purports the only sort of worthwhile knowledge is that which can be decontextualized (i.e., free of regional bias) and de-personified (i.e., free of racial, cultural, and gender bias). What the recommendations from the TWG allude to is the way this logic, in turn, devalues specific sorts of knowledge and truths created by the unique experiences of those who are marginalized based on the contextual nature of oppression within the United States (Ladson-Billings, 2000).

In the following section, we present the work of seminal critical race scholars, such as Derrick Bell, Cheryl Harris, Richard Delgado, Gloria Ladson-Billings, and others, to anchor our interpretation of outcomes related to our analysis of IES funding patterns from 2008 to 2022. By applying the work of these scholars, we illustrate how efforts to produce “objective” research through IES funding perpetuate Eurocentric majority-normative ways of knowing/being. Based on the fact IES was established to generate this sort of “objective” knowledge, the justification for the statement made by TWG members that “their research methods, their institutional affiliation, and/or their demographic backgrounds are at odds with IES’s priorities” (Chhin & Stapleton, 2020, p. 8) becomes more concrete.

Critical Race Theory

CRT is a theoretical framework developed out of critical legal studies in the late 20th century that aims to center experiences of people of color in the United States to show the ways “neutral” laws disproportionately affect communities of color (Bell, 1995; Harris, 1994). We quote Bell (1995) in reference to Professor Charles Lawrence, who Bell explained spoke for many CRT scholars when he argued that

critical race theorists disagree with the notion that laws are or can be written from a neutral perspective. Lawrence asserts that such a neutral perspective does not, and cannot, exist – that we all speak from a particular point of view from what he calls a “positioned perspective.” (p. 901)

Such an argument against the fallacy of neutrality can be extended beyond law and into any field in which humans interpret and experience outcomes of policies. As this false sense of neutrality relates to the field of education research in particular, understanding the ways “neutral” policies disproportionately affect the educational experiences of students of color is essential for developing research projects aimed at supporting individuals within such groups (Souto-Manning & Price-Dennis, 2010).

Using this understanding of false neutrality as a guide, it becomes clear that because IES was founded based on an internal logic that prioritizes only “neutral” and decontextualized research projects, the research funded through IES grants will not be effective in supporting students from marginalized racial groups. Again, to quote Bell (1995), “[critical race scholars] insist . . . that abstraction, put forth as ‘rational’ or ‘objective’ truth, smuggles the privileged choice of the privileged to depersonify their claims and then pass them off as the universal authority and the universal good” (p. 901).

Moreover, as Solórzano and Yosso (2002) explained, “Although social scientists tell stories under the guise of ‘objective’ research, these stories actually uphold deficit, racialized notions about people of color” (p. 23). To reconceptualize the ways knowledge is developed and appreciated in education research, scholars like the ones cited here and others (e.g., Brayboy, 2005; Bernal, 2002) have argued for a needed shift toward understanding experience as a valid form of empirical data. In short, these scholars argue for a change in epistemological and ontological orientations within the field of education research.

To push back against knowledge and truth falsely presented as “objective,” CRT scholars hold stories and narratives as essential. In describing the need for the development of TribalCrit, an offshoot of CRT, Brayboy (2005) explained,

Theory is not simply an abstract thought or idea that explains overarching structures of societies and communities; theories, through stories and other media, are roadmaps for our communities and reminders of our individual responsibilities to the survival of our communities. These notions of theory, however, conflict with what many in the “academy” consider “good theory.” At the heart of this conflict are different epistemologies and ontologies. (p. 427)

For CRT scholars and specifically those who instantiate the tenets of TribalCrit, theory and story are synonymous products of personal experiences within specific communities. To shift ontological and epistemological orientations toward an appreciation for these sorts of knowledge and truth, counterstories of marginalized experiences play a central role. As it relates to this article, qualitative data are fundamental in developing this knowledge base.

Bell (1995) explained, “Critical race theory writing and lecturing is characterized by frequent use of the first person, storytelling, narrative, allegory . . . and the unapologetic use of creativity” (p. 899). The reason for the use of story in CRT was described by Alemán and Alemán (2010) as “build[ing] consensus, solidarity and community amongst members of disenfranchised groups . . . to nurture community cultural wealth, add to collective memory, and strengthen resources for resistance and survival” (p. 7). If IES leadership takes pride in their stated appreciation and “commitment to the many facets of human diversity that shape our lives” (IES, 2023), funding research projects that include the narration, counterstories, and first-person experiences of students, teachers, and other members of those groups is of the upmost importance. What our analysis of IES funding patterns presented in the following sections shows, however, is that IES guidelines have been operationalized to perpetuate the “privilege smuggling” Bell highlighted under the guise of an internal logic centered on “objective” research.

Data Generation

To understand patterns in funding related to inclusion of story/narrative, the first and second authors analyzed the previous 15 years of IES funding using the link on the IES website titled “Search Funded Research Grants and Projects” (IES, 2024). We then searched every project within each of the funding years under analysis, resulting in 2,135 projects. Provided descriptions included sections such as but not limited to project “Purpose,” “Activities,” “Products,” “Setting,” “Population,” “Intervention,” “Research Design,” “Control Conditions,” “Key Measures,” and “Data Analytic Strateg[ies].” It should be made clear that these sections offer much more information than an abstract would alone, allowing us to search each project more comprehensively.

Within each funded grant description, we used both visual and computer function searches for the following terms: “qualitative,” “interview,” “focus group,” and “observation.” Our reasoning for using these terms was based on the contention that any possible use of story in IES-funded projects would be found in relation to these qualitative data generation methods. We also searched for qualitative analytical methods (e.g., “coding,” “assertions,” “bricolage,” “ad hoc,” “discourse analysis,” “document analysis”). If the search for these terms yielded results, we read through the full description independently to understand if the project goal was to gather solely qualitative data, regarding both data generation and analysis (coded with a 1), or was a mixed methods study (coded with a 3). If none of the search terms were found, we coded the project as generating solely quantitative data (coded with a 2). The first and second authors then met virtually for interrater reliability, finding 100% agreement after these conversations. Although the use of critical theories (e.g., QuantCrit) to interpret both quantitative and mixed data generation methods could provide nuanced interpretations of such data, we were more interested in the inclusion of marginalized perspectives as valid data rather than interpretations of normative data using critical frames. Therefore, we did not include such frames as part of our codes.

For our analysis of funding year 2022, we expanded our search terms to include specific references to “story,” “narrative,” and/or “experience.” Our decision to expand for year 2022 alone was in relation to the TWG having been published in December 2020. Because this date is so close to the end of the year, we believed changes to funding patterns might not be reflected in funding year 2021. Therefore, we chose 2022 for our expanded analysis. Moreover, to better understand the data generation methods used in funded projects for 2022, the first author emailed each listed principal investigator (PI) on the IES grants website to ask if “story or personal narratives” were going to be used as part of their data (Appendix A [available on the journal website]). Institutional Review Board (IRB) approval was sought for this aspect of our work; however, the IRB at the first author’s institution concluded the project did not “qualify as research per 45 CFR §46.102(l)” and therefore deemed approval unnecessary.

Of the 117 PIs contacted, 47 replied (40%). Some of these PIs asked us to further define “story” or “narrative.” For these inquiries, the first author replied by describing our goal of understanding if any funded research projects were using stories to center the experiences of marginalized individuals that could juxtapose majority ways of being/knowing. Of these, 10 (21%) consented to have information from their responses shared in this article. Selected portions of these email responses are presented in the results section.

One note regarding the previously described coding: Some projects included observational videotaping for fidelity of implementation measures alongside quantitative procedures and were coded as mixed methods, which could be interpreted to skew the number of these projects. Some projects were not applicable to our method of analysis because they were funded trainings or development projects that did not describe the use of qualitative or quantitative data generation methods specifically. Although an in-depth analysis of these funding categories (e.g., predoctoral and postdoctoral fellowship trainings) lies outside the scope of this article, it should be noted that no training (e.g., predoctoral and postdoctoral) or technological innovation (e.g., small business innovation) funded by IES included any of our qualitative search terms. Other projects were unclear based on a lack of specifics regarding generation methods. These projects (n = 445) were coded with an “n/a” or “unclear” and removed from our overall number of projects (N = 2,135), resulting in 1,690 projects analyzed. For a complete list of programs categories funded by IES, see Appendix B (available on the journal website). For a detailed copy of our coding procedures and justifications, please contact the first author.

Results

Overall Trends: 2008–2022

Over a 15-year period (2008–2022), 99.3% of IES funding was awarded to researchers who described generating either solely quantitative data (40.7%) or mixed methods (58.6%) in their proposals (Figure 1). Based on the TWG members’ critique that IES does not appreciate community-based research, a closer look at researcher-practitioner partnerships (RPPs) is also warranted. We consider RPPs more open to including community needs based on their inherent use of localized knowledge being an essential part of the research designs (Coburn & Penuel, 2016). It should be noted that at the time of this writing, RPPs were no longer offered as a research program because “partnerships are now encouraged in all [IES] grant competitions” (IES support, personal communication, 2023). Even so, a deeper analysis of these programs is warranted to understand if IES funding patterns were repeated even within designs that lend themselves to including unique and localized perspectives through qualitative data generation methods.

Figure 1.

Awarded applicant described data generation methods (2008-2022).

Research Practitioner Partnerships

The majority of RPPs funded by IES are administered within Regional Education Laboratories (RELs). RELs aim to “bridge the gap between research and practice” (Scher et al., 2018, p. 5). To do so, IES contracts with outside organizations to operate 10 RELs across the United States. Each is tasked with supporting some aspect of applied research based on defined needs within specific geographic locations. As such, RELs and RPPs are well suited to include community-based knowledge. To administer RPPs, IES funds data generation in similar patterns to those previously presented in the other research programs.

RPPs were first offered to researchers as a possible program classification in 2013. Since then, out of the 1,103 projects¹ funded, 63 projects were RPPs (5.7%; Figure 2). Of these 63, only one (1.6%) used solely qualitative data generation methods, 28 (44%) generated solely quantitative data, and 33 (52.3%) employed a mixed-methods approach (Figure 3).

Figure 2.

Funded RPPs (2013-2022).

Figure 3.

Funded data generation methods within RPPs (2013-2022).

As Figure 3 shows, like other IES programs, mixed data generation methods are the majority of those funded through RPPs. As is also clearly shown in Figures 1 through 3, however, research generating solely qualitative data are sorely missing from IES-funded projects. Two years after the TWG published their recommendations, those trends continued.

Funding Years Since the TWG Report

During the funding years 2008 to 2020, six funded projects reported generating solely qualitative data. Based on the recommendations of the TWG members published at the end of 2020, our analysis reveals IES did not shift its funding patterns related to qualitative data. No projects funded² by IES in 2021 or 2022 (n = 224) generated solely qualitative data. The majority (55.8%; n = 125) reported mixed methods, with 44% (n = 99) reporting generating solely quantitative data. Within funding year 2022 specifically, of those who responded, seven PIs (7%) described the use of story or narrative in their funded projects. Responses varied in relation to how “story” or “narrative” was defined. No PI defined story in relation to highlighting marginalized ways of knowing/being.

For instance, one PI reported there “could be some site-specific or personal stories recorded” in interviews that will be used to understand fidelity of implementation around their intervention (personal communication, April 13, 2023). Another PI explained that they do plan on including “learning stories” in a professional development they are designing; however, they had not begun generating data because of delays related to COVID19. Moreover, the PI explained even if they plan on generating qualitative data, “the restriction of page/word lengths [in academic journals] often force just a quantitative reporting” (personal communication, April 5, 2023). Of the seven PIs reporting using stories/narrative, two described doing so with the goal of highlighting the experiences of certain marginalized groups.

For example, one PI explained they would be asking “Black and Latino caregivers about the characteristics they noticed first” relating to children being diagnosed with autism (personal communication, April 20, 2023). The other PI explained they would ask individuals in their project, of whom are majority students of color and economically disadvantaged, “to draw on their personal experiences” about using text messages to improve kindergarten readiness (personal communication, April 24, 2023). Neither, however, explained any goal to juxtapose traditional ways of being/knowing after responding to the first author’s second email providing more clarity around conceptualizations of “story” and “narrative.”

If value can be measured by the amount of money invested into certain projects, our results reveal TWG members were accurate in their belief that IES funders do not appreciate the sort of knowledge generated through solely qualitative data. Moreover, of the PIs who described using mixed data generation methods in 2022, only two described the use of story as a central component to their design, and neither defined a goal of using story to push back against traditional ways of knowing. In the following sections, we discuss these results through the lens of CRT. Through this framework, possible implications for the lack of story in IES-funded research as it relates to outcomes for historically minoritized and resilient students and teachers in schools across the United States will be presented, along with providing possible context for the current push to shutter the USDOE.

Discussion

Since the publication of the TWG report in 2020, the data show that IES has continued to fund research that utilizes solely quantitative or mixed-methods data. As stated at the beginning of this article, our interest in undertaking this project was to better understand the data generation methods funded by IES over a 15-year period to provide possible insights relating to why barriers to participation in IES-funded research persist, as highlighted by members of the TWG. Our analysis reveals the Institute is centered on an internal logic that interprets the goals of scientific inquiry as being the same in all fields of science. This internal logic is pushed outside of the walls of IES through external validation of funding researchers who align with the IES mission of supporting “objective” and decontextualized projects. Our analysis also shows that both prior to and after the TWG report, such a goal manifests in a lack of funded research methods that generate solely qualitative data, of which localized and context-dependent knowledge are paramount.

Although we do not know the genesis of the lack of qualitative data generation methods being funded (i.e., if scholars utilizing such data apply and are rejected, are goaded away from applying, or both), our analysis confirms the critiques of TWG members who described “a perception that the focus of their research, their research methods, their institutional affiliation, and/or their demographic backgrounds are at odds with IES’s priorities” (Chhin & Stapleton, 2020, p. 8). Moreover, the lack of change since 2020 reveals IES has not begun funding qualitative data generation methods. Interpreting this outcome through CRT provides further explanation about barriers to participation in IES-funded research by researchers working with and for historically marginalized and resilient communities. In other words, the central problem with the disparities in research funding presented here is that they further marginalize and perpetuate normative and oppressive forms of knowing. Our results show the way individuals who hold these valuable ways of knowing can be epistemologically disincentivized from applying for IES funding. Although the codes we used to analyze IES funding offered a useful framework for our analysis, we acknowledge the methodologies that influence choices of research methods should be considered on more of a spectrum. Nonetheless, as Solórzano and Yosso (2002) argued, the knowledge created by IES-funded work under the “guise of objectivity” is more likely to perpetuate “deficit, racialized notions about people of color” (p. 23). This is because “objective” research “smuggles the privileged choice of the privileged to depersonify their claims and then pass them off as the universal authority and the universal good” (Bell, 1995, p. 901).

To put the person into the research, stories, narratives, and the knowledge generated through marginalized and resilient experiences must be included. If a “system of knowledge” is a more apt way to understand ways of knowing (i.e., epistemologies; Ladson-Billings, 2000), researchers who seek to remove “racial, cultural, gender, or regional bias” will never generate useful knowledge for individuals who live through the systemic oppressions created by racial, cultural, gender, or regional biases. Rather, such decisions perpetuate dominant forms of knowing under the guise of objectivity and normality. To change the participation rates for historically marginalized individuals related to IES applications, we echo the TWG members who argued it is imperative for decision makers at IES to begin appreciating the sort of knowledge created through qualitative data. That knowledge is not disconnected from race, culture, gender, or region but represents the embodiments of these powerful influences within social realities—of which, education is one. As we write this recommendation, however, the current federal administration is doing everything in its power (and beyond it at times) to push back on such goals. CRT once again offers a useful framework to understand such actions.

Along with exposing the fallacy of neutrality, Bell (1995) argued that efforts to increase opportunities for minoritized populations of color within the United States will never sustain unless those opportunities simultaneously benefit the majority. As a response to the erosion of legislative victories seen in the 1960s, Bell explained,

Even those herculean efforts we hail as successful will produce no more than temporary “peaks of progress,” short-lived victories that slide into irrelevance as racial patterns adapt in ways that maintain white dominance. This is a hard-to-accept fact that all history verifies. We must acknowledge it, not as a sign of submission, but as an act of ultimate defiance. (p. 12)

Efforts such as the organizing of the TWG and the targeting of minoritized institutions for outreach could be interpreted as one of these “peaks of progress.” Nonetheless, our recommendations do not change. Although the IES mission purports scientific inquiry is the same in all fields, the ways to find answers to those inquiries may not be. As Flyvbjerg (2001) explained, “Praxis has always been contingent on context-dependent judgment, on situational ethics” (p. 136). Praxis here is conceptualized as ways of knowing that lead to and inform specific actions (Freire, 1968). Possibly nowhere are situational ethics and judgments more prevalent than in specific classrooms nested in specific schools nested in specific districts nested in specific communities nested within specific states. To increase participation of marginalized members of these systems, the knowledge generated from their experiences therein must be understood as powerful forms of empirical data. Recommendations to appreciate the value of qualitative data generation methods, however, are not limited to the context of the United States.

Scholars in countries around the world have stressed the importance of qualitative data in providing nuanced and experiential knowledge regarding a variety of topics in social science. For instance, in relation to studying sexuality across the continent of Africa, Tamale (2011) explained the bulk of prior research regarding AIDS/HIV “were quantitative and epidemiological, largely ignoring the qualitative socio-economic impacts of the epidemic” (p. 25). In New Zealand, Smith (2005, 2016) advocated for the use of qualitative data to forefront the “richness and complexity” of groups such as the Māori to pushback against researchers who utilize data that seek to problematically simplify nuanced realities. Such calls reverberate similar arguments made by scholars such as Bell (1995) and other critical race theorists. To increase racial and other forms of diverse identities in the IES application pool as recommended by the TWG, a similar appreciation for contextual nuance must be adopted.

Whether or not IES sustains under the current federal administration in the United States (which it very well may not), funders at the federal, state, and/or local levels should move beyond targeted outreach of minoritized communities and consider the sort of “internal logic” they might be representing through “external validations” of grant funding. By investing in researchers who generate solely qualitative data, funding agencies could, in turn, signal their appreciation for the value of these data generation methods in providing space for members of historically marginalized communities to share their experiences, develop counterstories, and push back on normative systems of knowing. If there is a desire to support underserved communities through education research, a more accurate understanding of how valuable knowledge is conceptualized, generated, and funded should presuppose any targeted outreach effort.

Supplemental Material

sj-pdf-1-edr-10.3102_0013189X251355601 – Supplemental material for What Have They Done? An Analysis of Data Generation Methods Funded by the Institute of Education Sciences

Supplemental material, sj-pdf-1-edr-10.3102_0013189X251355601 for What Have They Done? An Analysis of Data Generation Methods Funded by the Institute of Education Sciences by Jacob S. Bennett, Colleen Fitzpatrick and Stanley Trent in Educational Researcher

Footnotes

ORCID iDs

Jacob S. Bennett

Colleen Fitzpatrick

Stanley Trent

Notes

Authors

JACOB S. BENNETT, PhD, is an independent researcher and faculty lecturer at Vanderbilt University, 231 Appleton Pl., Nashville, TN 37212; Jacob.s.bennett@vanderbilt.edu . Grounded in emergence and the reflexive and adaptive traditions of social science, his research informs praxis-oriented strategies within classrooms to sustain lasting reforms across schools, districts, and their surrounding communities.

COLLEEN FITZPATRICK, PhD, is the Daso-Herb Endowed Chair for International Experiences and assistant professor at the University of Toledo, 2801 West Bancroft Street, Mail Stop 924, Toledo, OH 43606-3390; colleen.fitzpatrick@utoledo.edu . Her research explores the complex and interactional role the context of a classroom, school, district, or state plays in how teachers and students experience teaching and learning history.

STANLEY C. TRENT, PhD, is an associate professor in the School of Education and Human Development at the University of Virginia, 405 Emmet St, Charlottesville, VA 22904; sct3m@virginia.edu . His research focuses on the disproportional placement of African American students in special education and culturally responsive teacher education.

References

Alemán

Jr Alemán

S. M.

(2010). ‘Do Latin@ interests always have to “converge” with White interests?’:(Re) claiming racial realism and interest-convergence in critical race theory praxis. Race Ethnicity and Education, 13(1), 1–21.

Bell

D. A.

(1995). Who’s afraid of critical race theory. University of Illinois Law Review, 1995(4), 893–910.

Bernal

D. D.

(2002). Critical race theory, Latino critical theory, and critical raced-gendered epistemologies: Recognizing students of color as holders and creators of knowledge. Qualitative Inquiry, 8(1), 105–126.

Brayboy

B. M. J.

(2005). Toward a tribal critical race theory in education. The Urban Review, 37, 425–446.

Chhin

Stapleton

(2020). Increasing diversity and representation of IES-funded education researchers (Technical Working Group summary). National Center for Education Research. https://ies.ed.gov/ies/2025/01/increasing-diversity-and-representation-ies-funded-education-researchers

Coburn

C. E.

Penuel

W. R.

(2016). Research–practice partnerships in education: Outcomes, dynamics, and open questions. Educational Researcher, 45(1), 48–54.

Flyvbjerg

(2001). Making social science matter: Why social inquiry fails and how it can succeed again. Cambridge University Press.

Freire

(1968). Pedagogy of the oppressed ( Ramos

M. B.

, Trans.). Continuum.

Harris

A. P.

(1994). The jurisprudence of reconstruction. California Law Review, 82(4), 741–785.

10.

Institute of Educational Sciences. (2020). About IES: IES diversity statement. https://archive.ph/m4C6c

11.

Institute of Educational Sciences. (2023). National board for education sciences. https://ies.ed.gov/director/board/

12.

Institute of Educational Sciences. (2024). Funding opportunities: Educational research grant programs. https://ies.ed.gov/funding/grantsearch/index.asp?mode=1&sort=1&order=1&searchvals=&SearchType=or&checktitle=on&checkaffiliation=on&checkprincipal=on&checkquestion=on&checkprogram=on&checkawardnumber=on&slctAffiliation=0&slctPrincipal=0&slctYear=0&slctProgram=0&slctGoal=0&slctCenter=0&FundType=1&FundType=2

13.

Ladson-Billings

(2000). Racialized discourses and ethnic epistemologies. In Denzin

Lincoln

(Eds.), Handbook of qualitative research (2nd ed., pp. 257–277). Sage.

14.

Milner

H. R.

IV . (2008). Critical race theory and interest convergence as analytic tools in teacher education policies and practices. Journal of Teacher Education, 59(4), 332–346.

15.

Scher

McCowan

Castaldo-Walsh

(2018). Regional educational laboratory researcher-practitioner partnerships: Documenting the research alliance experience. U.S. Department of Education. https://files.eric.ed.gov/fulltext/ED581137.pdf

16.

Schneider

(2018). Changes are coming to research competitions. Institute of Education Sciences. https://ies.ed.gov/director/remarks/researchcomp2018.asp

17.

Solórzano

D. G.

Yosso

T. J.

(2002). Critical race methodology: Counter-storytelling as an analytical framework for education research. Qualitative Inquiry, 8(1), 23–44.

18.

Souto-Manning

Price-Dennis

(2010). Teacher as researcher: Teacher research as a political act. Childhood Education, 86(4), 273–275.

19.

Smith

L. T.

(2005). On tricky ground: Researching the Native in the age of uncertainty. In Denzin

N. K.

Lincoln

Y. S.

(Eds.), The Sage handbook of qualitative research (3rd ed., pp. 85–107). Sage Publications Ltd.

20.

Smith

L. T.

(2016). Choosing the margins: The role of research in indigenous struggles for social justice. In Denzin

N. K.

Giardina

M. D.

(Eds.), Qualitative inquiry and the conservative challenge (pp. 151–173). Left Coast Press.

21.

Tamale

(2011). Researching and theorising sexualities in Africa. African Sexualities: A Reader, 14(3), 11–36.

22.

U.S. Congress. (2002). Education Science Reform Act of 2002. H. R. 3801-8. https://www2.ed.gov/policy/rschstat/leg/PL107-279.pdf

23.

U.S Department of Education, Institute of Educational Sciences & National Science Foundation. (2013). Common guidelines for educational research and development: A report from the Institute of Educational Sciences, U.S. Department of Education and the National Science Foundation. http://www.nsf.gov/pubs/2013/nsf13126/nsf13126.pdf

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.09 MB