Research Use and Politics in the Federal Higher Education Rulemaking Process

Abstract

This case study examined how research has been used in the federal higher education rulemaking process, through which the U.S. Department of Education implements programs under the Higher Education Act. Findings indicate that in this high-conflict policymaking process, politics infuses various forms of research use to create several overlapping categories of use (political-instrumental, political-conceptual, and political-imposed) that existing typologies of research use omit. This study demonstrates how previous conceptualizations of forms of research use should be expanded to account for these politically infused forms of research use. This study also uncovered consequences of widespread political research use in rulemaking, including a general distrust of research presented in the process and political actors using the same or similar data to reach different conclusions.

Keywords

educational policy higher education policy federal policy higher education research use rulemaking

The use of evidence and research in policymaking, including educational policymaking, dates back decades (Ness, 2010; Weiss, 2001). Yet, it has been widely acknowledged that research is often used for purposes other than creating policy that is responsive to that evidence (Weiss, 1979, 2001). Indeed, a “rational” approach to research use—where research is objectively conducted in an attempt to address a previously identified problem—has been viewed as unrealistic in the conflict-laden context of policymaking (Greenberg & Mandel, 1991, p. 635). Scholars of research use in policymaking have long recognized that research has been used, and sometimes misused, in distinct and observable ways that have little to do with conducting research in direct response to an identifiable problem (Alkin & King, 2017; Henig, 2008; Nutley et al., 2007; Weiss, 1979).

One area in which research may be used in policymaking involves the development of federal regulatory policy. Regulations are policies created by federal agencies to implement and administer authorizing statutes in a process known as rulemaking, such as when the U.S. Department of Education creates regulations to carry out provisions of the Higher Education Act (Kerwin & Furlong, 2011; Lubbers, 2014; Natow, 2017; Pelesh, 1994). The rulemaking process is governed by federal law that requires agencies to produce regulations that are grounded in reason rather than arbitrariness (Kochan, 2019; Shapiro & Murphy, 2013). Federal agencies are also required to conduct regulatory impact analyses, cost–benefit analyses, and other analyses for many regulations (Belfield et al., 2018; Shapiro & Murphy, 2013). Well-designed studies can provide agencies with rigorous assessments and help agencies to argue that their regulations are grounded in reason.

Rulemaking is also a political process rife with conflict and opposing interests (Shapiro & Murphy, 2013; Yackee, 2019). In such a context, research is likely to be used as a political device rather than as an instrument for effective policy design—a practice known as the political use of research (Ness, 2010; Weiss, 1979). As a result, the rulemaking process is susceptible to politics overtaking evidence as a basis for regulations despite the need for federal regulations to possess a “reasoned explanation” (Shapiro & Murphy, 2013, p. 1461). Highly political rulemaking processes therefore provide an opportunity to examine the political use of research in depth to better understand the complexities and consequences of this form of research use. Previous studies have adeptly observed and described political uses of research in policymaking (e.g., Ness, 2010; Weiss, 1979). However, existing conceptualizations of the typology of research use have paid scant attention to the ways that political use overlaps with other forms of research use in highly politicized contexts. These conceptualizations are therefore limited, and as such may lead to limited understandings and analyses of practice.

The study presented here investigated the manner in which research has been used in the higher education rulemaking process through which the U.S. Department of Education implements federal programs under the Higher Education Act. This is a political process influenced by interest groups and government leaders (Natow, 2017; Pelesh, 1994), and like all federal agencies, the department is required to base its regulations on reason (Kochan, 2019; Shapiro & Murphy, 2013). Yet, no previous study has provided an analysis of how research is used in this process. This study used qualitative case study methods, including key informant interviews and document analysis, to examine how research has been used in higher education rulemaking. The case of higher education rulemaking allowed for a detailed examination of how the political use of research works in practice and how politics interacts with other forms of research use in policy actors’ attempts to influence final rules. This study’s findings show how research has been used in higher education rulemaking and how previous conceptualizations of the forms of research use (instrumental, conceptual, political, and imposed) should be modified to account for synthesized forms of research use in highly political environments.

The Rulemaking Process for Federal Higher Education Regulatory Policy

Rulemaking refers to the process for creating regulations within government agencies. Lengthy as they may be, federal authorizing statutes such as the Higher Education Act often leave out a lot of details about how their policies and programs should be administered. It is therefore up to federal agencies to develop regulations that provide details for policy implementation (Kerwin & Furlong, 2011; Lubbers, 1998; Natow, 2017; Office of the Federal Register, 2011). Because this process occurs after federal law authorizes an agency to regulate, rulemaking is an aspect of policy implementation; however, because federal regulations often involve more than purely technical details and in fact often result in new substantive policy, rulemaking is also a policymaking process in its own right (Anderson, 2015).

The U.S. Department of Education’s Office of Postsecondary Education is responsible for creating regulations that implement the Higher Education Act. This rulemaking process for higher education follows a procedure that is governed by the Administrative Procedure Act, the Higher Education Act, certain executive orders, and other federal laws. This process (as described more fully in Kerwin & Furlong, 2015; Natow, 2017, 2019; Office of the Federal Register, 2011 and Pelesh, 1994) generally proceeds in the following manner: First, the Department of Education decides to create a new regulation. Unless a limited exception applies, the department must post a Notice of Proposed Rulemaking containing the details and language of the proposed regulation in the Federal Register. If the regulation involves a program authorized under Title IV of the Higher Education Act (which includes federal student financial aid programs such as Pell Grants, Direct Loans, and Federal Work Study), then the department is required to convene a negotiated rulemaking panel as part of the development of the proposed rule.¹ Negotiated rulemaking is a process for gathering representatives of stakeholders who are likely to be affected by a new regulation together with agency personnel to debate and develop the language of the proposed rule. Negotiated rulemaking teams meet several times over the course of several months, and if all participants consent to the proposed rule’s language, then the agency agrees to use that language in the Notice of Proposed Rulemaking. But if even one negotiated rulemaking participant (even the one representing the agency) does not consent to the language, then the agency drafts the Notice of Proposed Rulemaking on its own.

If the proposed rule is considered “significant”—meaning that the regulation is likely to have a large or adverse effect on the economy or cause major policy changes or inconsistencies—then the Office of Management and Budget (OMB) must review it before it appears in the Federal Register. Once the Notice of Proposed Rulemaking is published, members of the public have a certain period of time (typically 30 days) in which to provide written comments to the Department of Education regarding their perspectives on the proposed rule. The department must read the comments and decide whether to make any changes to the regulatory language based on them. Once again, the OMB must also review significant rules at this stage. Significant rules must also contain a regulatory impact assessment and undergo a cost–benefit analysis, per federal law (Belfield et al., 2018). When publishing the final rule, the department provides summaries of the comments received regarding the proposed rule and responds to those comments, either identifying changes from the proposed language that are being made in response to comments or, more frequently, explaining why the department is not changing the proposed language. Final rules go into effect at least 30 days after their publication in the Federal Register, or at least 60 days for rules that are deemed “major” or “significant” (Kerwin & Furlong, 2011; Lubbers, 1998; Natow, 2017; Office of the Federal Register, 2011). Even after the publication of a final rule, the rulemaking process continues. Rules that are influential or “major” enough to be subject to the Congressional Review Act can be overturned via congressional resolution (Kerwin & Furlong, 2011; Stephan, 2017). A new administration could step in and use the rulemaking process to undo the prior administration’s regulatory policies with which the new administration disagrees, as has happened previously in the higher education space (Natow, 2017). Final rules may also be subject to legal challenges (Mettler, 2014; Natow, 2017).

The higher education policy arena is no stranger to interest group politics and other political battles (Cook, 1998; Parsons, 1997), and the higher education rulemaking process has been the subject of great interest group pressure (Mettler, 2014; Natow, 2017; Pelesh, 1994). At the same time, certain regulatory and cost–benefit assessments must be conducted for significant rules (Belfield et al., 2018). The political nature of higher education rulemaking could be at odds with the expectation for government to conduct objective assessments when creating new rules, which makes higher education rulemaking an interesting case study for how politics influences research use in educational policymaking.

Forms of Research Use in Policymaking

Decision makers have long used research in policymaking for a number of purposes. Policymakers have used research to justify the need for a policy or program, researchers have promoted the use of their own research, and intermediaries have brought research to policymakers’ attention (Greenberg & Mandel, 1991; Lubienski et al., 2014; Ness, 2010; Nutley et al., 2007). Research use is far more complex than simply identifying a problem, researching the problem and potential solutions, and determining which solution to implement (Ness, 2010; Nutley et al., 2007; Weiss, 1979; Weiss et al., 2005). In her landmark work on forms of research use, Carol Weiss (1979) identified several ways that research has been used in decision-making, ranging from highly linear and utilitarian models, to pragmatic and political models, to a largely subconscious “enlightenment” model (p. 429). Over time, Weiss and others have conceptualized research use by decision makers into several distinct categories: instrumental use, conceptual use, imposed use, and political use (Beyer, 1997; Ness, 2010; Trostle et al., 1999; Tseng, 2012; Weiss, 2001; Weiss et al., 2005).²

Instrumental research use involves identifying and using research findings in direct response to a particular policy problem (Beyer, 1997; Farley-Ripple, 2012; Ness, 2010; Nutley et al., 2007; Tseng, 2012; Weiss et al., 2005). Instrumental research use has also been described as “rational” (Trostle et al., 1999, p. 104) and “problem-solving research” (Klemperer et al., 2001, p. 200). When decision makers seek research to determine how to address a problem or otherwise base their decisions directly on research findings, instrumental use has occurred (Ness, 2010; Trostle et al., 1999; Weiss, 1979). Examples of this use of research are when decision makers discontinue programs because research has shown them to be ineffective or adopt new policies because research has demonstrated they are likely to achieve desired goals (Weiss et al., 2005). Scholars have noted that true instances of instrumental research use are rare because policymakers are influenced by numerous factors, including economics and the preferences of important constituents, which might be inconsistent with a directly research-driven decision (Nutley et al., 2007; Sunesson & Nilsson, 1988; Weiss et al., 2005).

Conceptual research use refers to the indirect and long-term role of research in decision-making (Beyer, 1997; Farley-Ripple, 2012; Ness, 2010; Nutley et al., 2007; Tseng, 2012; Weiss et al., 2005). Sometimes called the “enlightenment model” of research use, conceptual use occurs when research findings “percolat[e]” in decision makers’ minds for some time, and may alter decision makers’ conceptions about issues before decisions are made (Weiss, 1979, p. 429). Policymakers may, for example, learn of research findings at conferences, in the news, or through conversations with people in the field. Sometime later, recollections of that research find their way into conversations or arguments about policy reform, even if the exact sources of those findings do not come immediately to mind (Weiss, 1979, 2001) With conceptual use, research on a policy matter may change policy actors’ viewpoints over time and may help to determine policymakers’ agendas (Weiss, 1979). Whereas instrumental research use involves direct connections between research and decision makers’ actions, conceptual use is less direct and instead may provide context and definitions to better understand a policy matter (Klemperer et al., 2001). Therefore, evidence of conceptual research use is often nonspecific, reflecting changes about which policymakers may not be directly aware (Farley-Ripple, 2012).

Imposed research use occurs when research-based practices are mandated or encouraged by government or other funders (Tseng, 2012; Weiss et al., 2005; Zane & Welsh, 2018). Via legal mandates or financial incentives, government and other funding organizations can require that research be used in particular circumstances. Examples of imposed research use include requirements that funded programs adopt or at least consider practices that have been shown by research to be effective (Weiss et al., 2005; Zane & Welsh, 2018). Although the literature regards imposed use as a distinct form of research use, it might also be viewed as an attempt to compel the instrumental use of research (Zane & Welsh, 2018).

Finally, political research use involves presenting research to support a political position that one already holds (Beyer, 1997; Farley-Ripple, 2012; Henig, 2008; Lubienski et al., 2014; Ness, 2010; Nutley et al., 2007; Tseng, 2012; Weiss, 1979; Weiss et al., 2005). For example, political research use occurs when a policymaker conducts a literature review to locate research whose findings justify the position the policymaker already supported (Weiss et al., 2005), or when decision makers assert that very act of conducting research is evidence of an issue’s importance (Weiss, 1979). The fact that a particular report is identified as research can itself be political because this bestows a level of respect and power that can influence understandings of policy problems and perceived policy solutions (Henig, 2012; Nutley et al., 2007). Weiss (1979) wrote that the using research in a political manner is legitimate unless the findings have been misrepresented to achieve a political end. The political use of research is particularly prominent in policymaking contexts (Henig, 2008, 2012; Ness, 2010).

Although this typology of research use has been documented extensively, there is considerably less literature on blended or overlapping forms of research use. In fact, observations of how different forms of research use work in concert with each other have been quite rare. One of the few studies that did recognize overlapping forms of research use was Sunesson and Nilsson’s (1988) analysis of research use in local government welfare offices in Sweden. Their study found that participants used multiple forms of research use concurrently. Also, Klemperer et al.’s (2001) study of higher education policymaking in the Netherlands urged that conceptualizations of research use “should allow for a classification of mixtures of different types of uses” (pp. 216, 218). Nutley et al. (2007) wrote about the need to more extensively analyze how different forms of research use work together and to move beyond the more “static typologies” (p. 45).

Policymaking, including higher education policymaking, is a political process (Cook, 1998; Head, 2015; Parsons, 1997). The use of research and data is also inherently political (Henig, 2008, 2012; Nutley et al., 2007). Yet, few studies have focused on how politics interacts with other forms of research use in policymaking to create separate, politically infused forms of research use that are not adequately captured by existing typologies. Research demonstrating how categories of research use in the traditional typology overlap and combine with each other would contribute to a more complete theoretical model and an enhanced understanding of how research is used in policy and practice.

Research Method

Research Approach

This study employed a qualitative case study approach (Creswell & Poth, 2018; Merriam & Tisdell, 2016). As described below, the study made use of multiple data sources and collection methods: interviews with a diverse range of rulemaking participants and observers as well as documentary data regarding different kinds of higher education rules. The use of multiple data collection methods is a common practice in case study research (Creswell & Poth, 2018) and serves to corroborate information from different data sources (Merriam & Tisdell, 2016).

Sampling and Data Collection

Interviews

A total of 34 interviewees participated in this study. Interviewees were recruited via a two-part purposeful sampling strategy involving criterion sampling and maximum-variation sampling. Criterion sampling refers to recruiting study participants who meet a preestablished criterion (Creswell & Poth, 2018; Suri, 2011), which in this case was having specialized knowledge about the higher education rulemaking process from participation in and/or close observation of the process. The second part of the sampling strategy involved maximum-variation sampling to recruit participants with a diverse range of characteristics (Creswell & Poth, 2018; Suri, 2011). As indicated in Table 1, this study’s interviewees represented a range of professional positions both inside and outside the federal government. The sample was also diverse in that participants had knowledge of several different higher education rules: all interviewees were familiar with at least one of the regulations for which this study’s documentary data were gathered (see below for those regulations), and in most cases were familiar with at least two of those regulations as well as others.

Table 1.

Sample of Interview Participants.

Categories of policy actors^a	n ^b
Federal government
U.S. Department of Education Personnel	7
Other federal government personnel	5
State government (employee or association)	2
Higher Education Institutions/Associations
Private not-for-profit institutions	6
Public 4-year institutions	3
Public 2-year institutions	3
Private for-profit institutions	2
Student financial aid officers	5
Other
Student/Consumer Advocacy Groups (includes youth advocates and student-veteran advocates)	6
Think tanks	4
Student lending industry groups (includes guarantors)	3
Accreditors/Accreditor Associations	2

These represent the positions held prior to or at the time of the respondent’s interview. ^bA total of 34 interviewees participated in this study. The sum of this column is greater than 34 because several interviewees held multiple positions in which they were involved in the higher education rulemaking process.

I conducted semi-structured interviews with all participants by telephone or (in one case) video conference. Table 2 presents an outline of sections of the interview protocol relevant to the topic of forms of research use in higher education rulemaking. When answering interview questions, respondents often provided examples from their own experiences with or observations about particular rulemaking proceedings.

Table 2.

Overview of Interview Protocol Sections and Examples of Questions Relevant to Forms of Research Use in Higher Education Rulemaking.

1. Introduction

Briefly explain the purpose of the study and the areas to be covered in the interview

Ask if interviewee has any questions

Interviewee’s background in higher education rulemaking

2. How research has been used in higher education rulemaking

Have you/your organization introduced research in higher education rulemaking?

Which rule(s)?

Kind(s) of research used?

During what phase(s) of the rulemaking process?

For what purpose was the research introduced?

To your knowledge, did the Department of Education rely on this research? How so?

Have you observed other policy actors introduce research in higher education rulemaking?

[Same follow-up questions as immediately above]

What other types of evidence (besides research) have been introduced in higher education rulemaking?

3. Factors that hinder or facilitate the use of research in higher education rulemaking

What circumstances make it less likely that research will be used in higher education rulemaking?

What circumstances make it more likely that research will be used in higher education rulemaking?

4. Is there anything else about research use in the higher education rulemaking process that we have not yet discussed, but that you think would be important for me to know?

Documents

Documentary data included the texts of final regulations, proposed regulations, and negotiated rulemaking records (negotiated rulemaking agendas, minutes, transcripts, issue papers, and other handouts used during negotiated rulemaking) for five higher education rules issued by the U.S. Department of Education: the Gainful Employment and Borrower Defense rules developed during the Obama administration (issued in 2014 and 2016, respectively); the Revised Pay As You Earn (REPAYE) rule (issued in 2015); the Teacher Preparation rule (issued in 2016); and the rescission of the Gainful Employment rule (issued in 2019). These documents were obtained from the electronic Federal Register and the Department of Education’s website. I also analyzed laws and Federal Register entries relevant to higher education rulemaking. Table 3 identifies the categories of documents reviewed in this analysis.

Table 3.

Table of Documentary Data.

Laws/Federal Register entries	Rulemaking records	Other documents
Final regulations	Negotiated rulemaking agendas	Research reports cited in regulations or negotiated rulemaking records
Notices of proposed rulemaking	Negotiated rulemaking lists of participants	Department of Education’s website
Notices of negotiated rulemaking	Negotiated rulemaking session summaries
Statutes	Negotiated rulemaking transcripts
Executive orders	Negotiated rulemaking issue papers
Case law	Regional meeting transcripts
Agency guidance	Regional meeting lists of participants
	Public comments

The sampling strategy to identify the rules for which documentary data were gathered involved a combination of typical case and maximum variation sampling (Creswell & Poth, 2018; Suri, 2011). The selected rules were all typical cases of higher education rulemaking: All were developed through the typical higher education rulemaking procedure, including negotiated rulemaking and a notice-and-comment phase. This was important because such rules represent a large component of the rulemaking that takes place under the Higher Education Act. The sampling strategy also aimed for maximum variation in that the reviewed regulations were different from each other in key ways. Although four of the negotiated rulemaking committees did not reach consensus, one (the REPAYE committee) did. The Gainful Employment and Borrower Defense rules received more attention and public comments than did REPAYE or Teacher Preparation. One of the rules (the Gainful Employment rescission) was issued during a Republican presidential administration, while the others were issued during a Democratic administration. Finally, the case study rules involved variation with regard to rule content. While all five regulations involved Title IV student financial aid programs in some way, the substantive details of these rules were different: The 2014 Gainful Employment rule involved accountability and reporting requirements for career-focused higher education programs; the 2019 Gainful Employment rule repealed the 2014 version; the Borrower Defense rule involved potential remedies for student debtors; the REPAYE rule regulated an income-based loan repayment plan; and the Teacher Preparation rule involved regulation of Title II of the Higher Education Act regarding teacher education programs (Program Integrity, 2014, 2019; Student Assistance General Provisions, 2015, 2016; Teacher Preparation Issues, 2016). This variation in the sample allowed for an examination of how research use varied or was similar for regulations that were different with regard to prominence, subject matter, or issuing presidential administrations.

Data Analysis

Data analysis involved multiple cycles of reviewing and coding interview transcripts and documentary data (Gibbs, 2018; Miles et al., 2019). First, I reviewed all interview transcripts and documents in their entirety and assigned codes to segments of each based on a standardized coding scheme. The initial coding scheme was based on concepts from the literature regarding how research has been used in policymaking. During the first cycle of data analysis, I added new codes based on concepts and themes that emerged from this study’s data set. Relevant sections of the final coding scheme appear in Table 4.

Table 4.

Relevant Coding Scheme Sections for Forms of Research Use.

Specific discussions of particular rules	Form of research use (code all that apply)
Borrower Defense Rule 2016	Instrumental
Gainful Employment Rule 2014	Conceptual/Enlightenment
Gainful Employment Rescission 2019	Political/Symbolic/Tactical
REPAYE Rule 2015	Imposed
Teacher Preparation Rule 2016	Process
	Other form of research use
Procedure of research use
Stage(s) in the rulemaking process	Identity of research user(s)
How research is brought into the process
Source of research used
Other aspect of research use procedure

In the next cycle, I created a data analysis matrix (see Miles et al., 2019) by exporting all segments of interview transcripts coded as identifying specific forms of research use into a spreadsheet for further analysis, to identify categories and clusters of concepts. Segments of documentary data that evidenced forms of research use were also included in this spreadsheet. I further analyzed each category and cluster of concepts to identify broader themes and detailed descriptions of each form of research use identified in the data. I wrote analytic memos and took notes throughout the process for purposes of both data analysis and maintaining a record of coding decisions (Gibbs, 2018; Merriam & Tisdell, 2016; Miles et al., 2019).

Results

Traditional Categories of Research Use in Higher Education Rulemaking

Research use in the federal higher education rulemaking process displays many aspects of the traditional typology of research use in policymaking, which includes instrumental, conceptual, imposed, and political forms of research use. The political use of research was by far the most prominent. But as demonstrated below, the case of higher education rulemaking illustrates how this traditional conceptual model should be expanded to account for the ways that political use infuses other forms of research use in a high-conflict policymaking situation.

Instrumental use

The instrumental use of research was observed in the higher education rulemaking process, with nine interview respondents describing this form of research use. An example of instrumental research use in higher education rulemaking was observed when negotiated rulemaking participants debated a policy issue and concluded that they needed more information before a decision about that issue could be made. This type of problem-solving was sometimes done by forming subgroups for the purpose of gathering more information. A lending industry representative who participated in negotiated rulemaking discussed this example:

It happens all the time where they’ll do subgroups where somebody will say, “What about this?” . . . And they form a subgroup . . . depending on the topic and what the collective expertise of the group [is,] it has been where they’ll take something and they’ll say, “Who wants to be on this work group? You guys are going to do it offline and then come back and report back to the group.” (Interviewee 7)

Instrumental research use was also observed when policy actors were faced with a question and turned to statistical data to determine the answer. In an example from the 2016 Borrower Defense rulemaking, negotiators were faced with the issue of how to calculate partial borrower relief. To figure this out, according to an institutional representative familiar with that rulemaking, negotiators looked to data from the Bureau of Labor Statistics (BLS) to “benchmark different careers and what they make” (Interviewee 1). Instrumental research use in the form of seeking statistical data to solve a policy problem also took place when conducting regulatory impact assessments during higher education rulemaking. A federal staffer familiar with these assessments described the thought processes of individuals tasked with conducting these assessments as follows: “Okay, we need to estimate the number of schools that do this, we need to estimate the number of staff hours on this.” The respondent indicated that BLS data sets on wages were “sort of a go-to” for individuals conducting these assessments (Interviewee 28). Documentary data reflected instances of the instrumental research use as well. For example, a summary of a Borrower Defense negotiated rulemaking meeting indicated that some nonfederal negotiators were tasked with conducting research and gathering information in between meetings and reporting that information to negotiators at the next meeting (Document-Negotiated Rulemaking Summary, Borrower Defense, January 2016).

Sometimes negotiated rulemaking participants would identify a problem and seek data from the Department of Education that they believed would be helpful at solving the problem. Data requests sometimes went unfulfilled due to such reasons as lack of capacity for department staff to respond to the requests and a statutory ban on the collection of student unit record data.³ One nonfederal negotiated rulemaking participant observed that “there were a number of data requests and research requests made by the negotiators to the federal officials,” and that in many cases, “there was either no response or there was not a response that was timely enough to actually inform the conversation” (Interviewee 31). However, sometimes the department did provide negotiators with the information they requested. As an interviewee who had worked for the Department of Education explained:

We would have questions in the negotiated rulemaking committee and we were like, you know what? There’s a factual answer to that. We can find out the answer. You know, like how many dollars floated these schools? How much is Pell? How much is loans? Those kinds of things the department would get asked to do data runs for, and depending on capacity issues, we would try to answer people’s questions. (Interviewee 29)

Each of these examples illustrates a pattern familiar to instrumental research use: identifying an issue and seeking out data to understand and address the issue in a more-or-less direct manner.

Conceptual use

Conceptual research use was also observed in the higher education rulemaking process, with seven respondents describing this form of research use. For example, a federal official explained that a Department of Education Inspector General’s report about “waste, fraud, and abuse in distance education” was influential on policymakers’ thinking about distance learning programs and how they might be regulated. With regard to the Teacher Preparation rule, the same respondent said there was

a lot of research at the time about teacher effectiveness and . . . the influence of teachers on student achievement, and some of that shaped the desire to view the federal vehicle to improve student achievement by improving teacher quality. (Interviewee 28)

In another example of conceptual research use, a representative of institutions recounted how a negotiated rulemaking participant’s vague recollection of research on the value of a 4-year postsecondary degree led the participant to locate the research and inform other negotiators about it. The negotiator then located a study showing this information and discussed it at the next negotiated rulemaking session (Interviewee 17). This example illustrates how conceptual use can lead to research being considered by policymakers when a participant learns about the research and, at some point later, remembers that research and becomes motivated to locate and discuss it when the research becomes relevant to a policy issue.

Imposed use

While two interviewees described specific instances of imposed research use in higher education rulemaking, documentary data and legal mandates clearly show that imposed use has occurred in this process. Regulatory policymakers have been required to use research, data analysis, and evidence-based practices in the higher education rulemaking process. By long-standing executive order, all proposed and new regulations that are deemed to be significant are required to undergo regulatory impact assessments and, in some cases (i.e., when economically significant), cost–benefit analyses (Executive Order 12866, 1993). Another executive order requires that federal regulations “be based on the best available science” (Executive Order 13563, 2011, p. 215). Moreover, in order not to be deemed “arbitrary and capricious,” federal courts have said that regulations should have a “reasoned explanation,” as a federal court held in Association of Private Sector Colleges and Universities v. Duncan (APSCU) v. Duncan (2012), the case that overturned much of the Obama administration’s first Gainful Employment rule (pp. 1, 22–23, 31, 38; see also Shapiro & Murphy, 2013). Each of these legal mandates imposes the use of research or other evidence in the rulemaking process.

Another form of imposed use in rulemaking occurred when the Department of Education encouraged the use of research as part of its final regulations. An example of this was when the department wrote in the final Teacher Preparation rule, “We encourage States and teacher preparation programs to adopt research-based practices of effective teacher preparation for all aspects of their program accountability systems” (Teacher Preparation Issues, 2016, p. 75505). Thus, although the Department of Education has been subjected to imposed use by the elected and judicial branches of government, it has also mandated imposed use by encouraging states and institutions to use research-based practices in implementing federal programs.

Political use

In higher education rulemaking, political uses of research were by far the most common of the forms identified by the traditional research use typology. Documents provided to negotiated rulemaking participants, transcripts of regional meetings that preceded negotiated rulemaking, and records of negotiated rulemaking proceedings were replete with examples of policy actors citing research that supported their point of view or their stakeholder group’s interests. For example, a transcript of negotiated rulemaking proceedings for what would become the Gainful Employment rescission contains numerous instances of parties citing research that support their policy positions and identifying flaws in research that does not (U.S. Department of Education, 2017). Similarly, the texts of final regulations—which contain summaries of comments received about proposed regulations and the Department of Education’s response to those comments as well as the department’s reasoning behind the final rules—also include citations to research that supported research users’ positions (e.g., Program Integrity, 2014, 2019).

In addition, the widespread political use of research was perceived by 25 out of 34 of this study’s interviewees. As one representative of institutions said,

People have interests, then those interests will deploy facts and data and findings and research the way attorneys deploy them in a courtroom, which is to say they will highlight the stuff that happens to support the case, and they will actively seek to suppress the countervailing argument . . . If [research] happens to support my case, you will see me citing it as the best thing since sliced bread. And if I decide that you’re totally opposed to where I’m coming from, I’m probably going to dismiss it. That’s just the nature of this gig. (Interviewee 27)

The political use of research in higher education rulemaking has produced some interesting consequences. First, policy actors holding different viewpoints about higher education policy have used research based on similar data, and in some cases the exact same data, to argue for vastly different types of policies. Perhaps the most striking example of this was when the Obama and Trump administrations cited the same research to support very different policies regarding debt-to-earnings metrics in each administration’s Gainful Employment rule. The research—Baum and Schwartz’s (2006) report for the College Board entitled, How Much Debt is Too Much?—explained the so-called “8 percent rule,” which posits that individuals “should not devote more than 8 percent of their gross income to the repayment of student loans” based on a standard that is widely used in the mortgage industry (p. 2). Baum and Schwartz discussed some criticisms of this standard but also concluded it was not “unreasonable” in the context of student loans (p. 3). In its 2014 Gainful Employment rule, Obama’s Department of Education cited Baum and Schwartz (2006) as support for setting 8% of student loan debtors’ income as the “passing threshold” for debt-earnings standards of career-focused higher education programs (Program Integrity, 2014, p. 65096). Five years later, the Trump administration issued a regulation rescinding the Gainful Employment rule. In doing so, the department cited the same Baum and Schwartz (2006) report in support of the repeal, arguing that the report called into question the justification of the 8% standard because the authors had noted that the standard had “no particular merit or justification” for use in the student loan context (Baum & Schwartz, 2006, as quoted in Program Integrity, 2019, p. 31426). One former federal staffer who was familiar with both Gainful Employment rulemakings described the rule’s rescission as not a “research-driven change. It’s just a difference in their preferences, and perhaps also . . . a different interpretation of similar evidence” (Interviewee 32).

After the Gainful Employment rescission’s Notice of Proposed Rulemaking was issued, the lead author of Baum and Schwartz (2006) released a statement indicating that she did not support her research being used for this purpose (Kreighbaum, 2018). In the final rule, the department recognized the researcher “does not wish her paper to be used in support of the Department’s decision to rescind the [Gainful Employment] regulations” but nonetheless relied on the report to justify the rescission (Program Integrity, 2019, p. 31407). This highlights another implication of political research use: that research may be used to justify a policy with which the researcher disagrees.

In another example of the same or similar data being used for different political purposes, one negotiated rulemaking participant who represented institutions observed that in the federal higher education policy arena, actors on different sides of the political spectrum would use data “from the same statistical place, whether it’s IPEDS or NPSAS⁴ or whatever,” but the research conducted with that data would reach different conclusions that “essentially fit the narrative of what they’re trying to say.” The same interviewee reflected that,

the old adage that we’re all entitled to our opinions, but we all have to use the same facts, or whatever the quote is, it kind of flies out the window a little bit. Because each side is, in some ways, using the same facts, but the folks who are doing the research and cutting the data and writing the policy ideas to fit a narrative often come to starkly different conclusions. (Interviewee 17)

Another consequence of the widespread political use of research in higher education rulemaking is that it has led participants to mistrust research that is presented during this process. Many participants were aware that policy actors use research to justify their own previously held viewpoints and policy preferences, and these participants expressed skepticism about research introduced during the rulemaking process. As one institutional representative and negotiator explained,

It’s really difficult sitting at the table if you’re getting data from these different data points to sort through and be able to pull out what is the reality and what maybe is, they’re only sharing the part of the data that made their point. (Interviewee 9)

A different negotiated rulemaking participant representing institutions said the following:

Even the best quality research is highly susceptible to manipulation and gaming by people who have other interests . . . I get very annoyed when people talk about, “Oh, a recent study by [an ideological think tank], blah, blah blah.” And I have to tell you, just because somebody collected some data and maybe even wrote a regression equation does not make a study, because it may be a propaganda piece that happens to, again, deploy data that support the case and may have actually, willfully ignored opposite facts. (Interviewee 27)

Other interviewees made similar comments, which evidence an important implication of the widespread political use of research in higher education rulemaking: a general distrust of research presented in the process.

Politically Infused Forms of Research Use in Higher Education Rulemaking

Some forms of research use in higher education rulemaking do not fit neatly under any of the forms included in the traditional typology. Instead, these forms involve combinations of political use with other forms of research use. The political influence should be expected in a policymaking situation, and perhaps especially one that has experienced as much conflict and political dueling in recent years as the higher education rulemaking process (Mettler, 2014; Natow, 2017). In such contexts, political considerations are likely to infuse many decisions, particularly those made by political actors. As a respondent of this study who had worked for the Department of Education succinctly noted, “This is all politically driven. It’s all political agenda. You’re serving constituencies. You want to make sure that your interest groups know that you’re doing things in rulemaking that are going to serve them” (Interviewee 14). This study identified three types of politically infused forms of research use in higher education rulemaking: political-instrumental, political-conceptual, and political-imposed.

Political-instrumental use

Eight of this study’s respondents described instances of political-instrumental research use in higher education rulemaking. Research was used in a political-instrumental way when participants, in promoting their policy ideas, were faced with a question they were not able to answer or a problem about which they needed more information. Participants gathered data to address those issues, and then returned to arguing in favor of their policy positions with the new information in hand. These actions resemble instrumental use in that they involve policy actors faced with a problem engaging in research to address the problem. But this form of use is also political because the problem itself arose while the political actor was advocating for a political position, and the actor returned to advocating that position after collecting the new information.

An example of political-instrumental research use occurred when negotiated rulemaking participants received assistance from outside associations who represented the negotiators’ stakeholder groups (e.g., when an association representing community colleges or public institutions provided support to a negotiator representing those types of colleges). One negotiated rulemaking participant representing institutions described:

We were getting information to help us comment on areas that were from our—like a support team. You go in there with folks that you know. If you write to them and say, “How do you do X, Y, Z?” they can give you information . . . You don’t necessarily know absolutely everything, even if you’ve worked in that field for many, many years, so having that kind of support, people that can send you information, keep you updated on the research that affects what you’re talking about, will be really helpful. (Interviewee 23)

A lending industry interviewee provided the following example from the Borrower Defense negotiated rulemaking, where one nonfederal negotiator did some quick research to respond to a question from another negotiator who had questioned the first negotiator’s argument:

[A non-federal negotiator] was saying that there needed to be an appeal process for closed school discharge. And at the table, [another negotiator] asked why he thought that was so. Because as opposed to other discharges that can be somewhat subjective, closed school discharge isn’t subjective. Either the borrower attended at the time within 90 days of the school closing or they didn’t. And he came back to the table and said, he had one borrower, one time that had been denied inappropriately. (Interviewee 2)

As these examples indicate, political-instrumental use involves using research to address questions and problems that arise within a larger context of using research to advocate for the research user’s political viewpoint.

Political-conceptual use

Political-conceptual research use involves actors widely distributing research that favors their policy preferences with the goal that the research will, among other things, change the perspectives of key decision makers on relevant policy issues. This is a conceptual use of research in that it is largely indirect, and the “enlightenment” goal may take some time to achieve (Weiss, 1979, p. 429). But it is also political because the research is disseminated for the purpose of influencing people’s viewpoints on policy. This form of research use was discussed by seven of this study’s interviewees.

A respondent representing an advocacy group that regularly releases research on higher education matters described how such research can have a political-conceptual effect:

With respect to changing someone’s mind, yeah, I think it is possible. I think it is rare that a single paper will take people going full speed in one direction and lead them to go full speed in another direction. I think it is more common that a paper will either elevate an issue or strategy that was previously a lower priority, or that a paper will be part of a body of research that, over time, changes people’s opinions. (Interviewee 33)

As this quotation suggests, the goal of changing policy actors’ long-term perceptions on political issues need not be the only goal of publicizing the research, or even one of the goals. But research can have an enlightenment effect over time on decision makers, and the effect may be to change their perspectives on policy issues in a direction favorable to the policy actor that originally posted the research. For example, a federal official interviewed for this study indicated that politically oriented research can have a conceptual-use effect by which it, in the words of Weiss (1979), “redefines the policy agenda” (p. 430). This respondent said,

In shaping people’s perception of what we need to address, you know, what problems are out there that we need to address, that I think is where research papers and think tank papers, honestly, are sort of most influential in defining: Is it a debt crisis? Is it not a debt crisis? Can students afford college or not? That kind of stuff. So I think the research world and the think tank world end up shaping the view of what policy problems we need to address through rulemaking. (Interviewee 28)

Another interviewee who worked for the Department of Education explained how one ideological think tank went about putting research out in a manner that was likely to catch policymakers’ attention. This respondent said that the think tank

put out their report and they made sure it got enough press coverage so that we couldn’t miss it . . . And there were a bunch of us who would, every time we saw an interesting article, send it around to everyone who might be interested. So if that came up in a Washington Post article or something, we almost certainly would have come across it and sent it around to the people who were working on [a particular rulemaking issue]. (Interviewee 24)

This example illustrates how think tanks and advocacy groups may circulate and advertise research that supports their political positions, and then that research gets noticed by policy actors who may later use it when setting policy agendas or defining the scope of policy issues. Even if the advocacy organization’s immediate motive in disseminating the research was more akin to classic political use (i.e., providing research to decision makers to help them justify shared policy goals), over time, these reports can have a political-conceptual influence.

Political-imposed use

Political-imposed use occurs when research use is required by law or funding mandate (the “imposed” part of this form of research use [e.g., Weiss et al., 2005]), and then, the research that is used as a result of that requirement justifies the policy preference of the research user (i.e., research is used in a political way). In other words, political-imposed use occurs when research use is required by an actor in a position of authority, and the actor on whom the research mandate is imposed in turn uses research that promotes the subordinate actor’s own political position. This form of research use was described by three interviewees who worked for the federal government on higher education rulemaking issues.

As explained above, in the higher education rulemaking process, various federal agencies (such as the Department of Education and the Office of Management and Budget) are required to conduct impact and cost–benefit analyses for many proposed and final rules (Belfield et al., 2018). These requirements are imposed by congressional and presidential actions on the agencies to whom Congress and the president have delegated authority to create regulations (Kerwin & Furlong, 2011). But when conducting these required analyses in higher education rulemaking, federal agencies have used research that is expected to help promote the success of the rule rather than use research in the more instrumental sense of figuring out the rule’s potential impact and possibly adjusting regulatory language as a result. One federal official explained that at the point of the Teacher Preparation rulemaking when it was time to conduct the required analyses,

it’s at a point where the rule was baked, but the idea was, OK, we don’t want to miss anything in the literature and we also want to kind of put the best face on it for what we’re trying to do. (Interviewee 26)

Another example of political-imposed research use in higher education rulemaking occurred with respect to the 2014 Gainful Employment rule. A lawsuit filed by representatives of for-profit higher education following the adoption of the first Gainful Employment rule (issued in 2011) resulted in some aspects of the rule being nullified for not having a “reasoned explanation for a core element of its central regulation” (APSCU v. Duncan, 2012, p. 38). After that court decision, according to some of this study’s respondents, the Department of Education stepped up efforts to ensure that new rules possessed a “reasoned explanation” for their key provisions, including by citing more research in the rules than had been done in the past. But politics was still influential in this form of imposed research use. One federal official familiar with recent higher education rulemakings said of these later stages of rulemaking:

I think once you’re at the point where you’re just looking to justify what you’re doing, [the source of the research] doesn’t make a huge difference . . . The bar is so low in terms of like, from a legal standpoint, I think that’s our impression anyway, that really any kind of research will do. (Interviewee 26)

Similarly, a respondent who had worked for the Department of Education for several years described how the department’s mandate to have a reasoned explanation for its rules mixed with politics when creating new regulations:

The number one factor in the Obama administration was creating a rule that would err on the side of providing borrower benefits. The second main consideration was crafting a rule that would survive legal challenge. The third main consideration was connected to the second, which was having a strong empirical grounding for that decision. But you’ll notice that the first one came before the other two. When there was a question or a tie breaker, it would be resolved on the side of, we aren’t sure, so let’s err on the side of giving the borrower the benefit. (Interviewee 14)

These statements indicate that even when policymakers are required to use research in the higher education rulemaking process, the research use takes on a political nature.

Discussion and Conclusion

This study addressed a gap in previous literature by documenting the forms of research use that have occurred in the higher education rulemaking process—a consequential educational policymaking process that requires some use of evidence (e.g., Belfield et al., 2018), but for which no previous studies have analyzed how research has been used in the process. This process is rather unique in that it involves open deliberation among policymakers and stakeholders via negotiated rulemaking (Natow, 2019), and many of the participants represent universities and other organizations that conduct research. As such, higher education rulemaking presents an optimal case for studying how research use and politics interact in a policymaking environment.

In addition to the widely recognized instrumental, conceptual, imposed, and political forms of research use (e.g., Weiss, 1979; Weiss et al., 2005), this study has identified political-instrumental, political-conceptual, and political-imposed forms of research use at work in the federal higher education rulemaking process. Traditional conceptualizations of research use in policymaking have given insufficient attention to the manner in which different forms of research use—be they instrumental, conceptual, imposed, or political—intersect with and influence each other (Nutley et al., 2007). A theoretical model that captures these combinations may lead to more accurate understandings of how research is used in policy and practice. In a policymaking process, it is unsurprising that the political use of research would occur commonly and more frequently than purely instrumental, conceptual, or imposed forms of research use. But this study has shown that in such a context, politics and the political use of research are so dominant that they fuse with other forms of research use to produce new categories of use that do not fit neatly into the traditional typology.

This study contributes to the literature on research use in policymaking by demonstrating how traditional conceptualizations of forms of research use should be expanded. In high-conflict contexts such as the higher education rulemaking process, political use dominates and influences other forms of research use such that they take on a political character. This study has also shed light on some interesting consequences of common political uses of research. First, when research is used in a political manner, policy actors espousing different viewpoints have used the same or similar research to arrive at very different conclusions. Related to that point is another implication of widespread political research use: policy actors may be skeptical about any research presented in the process. Skepticism about the use of research in policymaking has been observed in other educational policy contexts (e.g., Henig, 2008). This study found that political research use resulted in distrust of research in the higher education rulemaking process as well.

Moreover, researchers of issues relevant to federal higher education policy should be mindful that their findings may be used as a political tool, and not always to support a position with which the researcher agrees. When researchers believe their findings are being misinterpreted, they can release a public statement clarifying those findings. As this study has shown, one piece of research can serve as a force for multiple sides of a political debate, and researchers’ clarifying statements in such circumstances can obtain visibility in specialized media (e.g., Kreighbaum, 2018).

Using the expanded conceptual model developed here, future researchers may examine how politically infused forms of research use appear in other contexts, such as at the state or local level, in educational practice, and in policy areas other than education. Future researchers may also explore how nonpolitical forms of research use overlap with one another. The findings of this study have implications for both researchers and policymakers: Policy actors, aware that research is used for political purposes, may be skeptical of research considered in policymaking. Proponents of evidence-based policy and researchers hoping their work will influence policy should be aware of this skepticism and consider how to cultivate policy actors’ confidence in research, such as by educating policymakers on research methods or advocating for more funding to enhance the quality of research (Henig, 2008). Without confidence in the validity of research on the part of the policy community, the value of evidence-based policy will continue to be questioned, and the use of research in policymaking will face an uncertain future.

Footnotes

Acknowledgements

The author thanks Jeffrey Henig, Monica Reid Kerrigan, and two anonymous reviewers for comments provided on earlier drafts of this article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research received financial support from the Hofstra College of Liberal Arts and Sciences Faculty Research and Development Grant.

ORCID iD

Rebecca S. Natow

Notes

Author Biography

Rebecca S. Natow is an Assistant Professor of Educational Leadership and Policy at Hofstra University. Dr. Natow’s research and teaching focus on higher education policy, higher education leadership, and qualitative research methods.

References

Alkin

M. C.

King

J. A.

(2017). Definitions of evaluation use and misuse, evaluation influence, and factors affecting use. American Journal of Evaluation, 38(3), 434–450.

Anderson

J. E.

(2015). Public policymaking (8th ed.). Cengage.

Association of Private Sector Colleges and Universities v. Duncan . Memorandum Opinion. Civil Action 11-1314 (RC) (D.D.C. 2012).

Baum

Schwartz

(2006). How much debt is too much? Defining benchmarks for manageable student debt. The College Board.

Belfield

C. R.

Bowden

A. B.

Rodgriguez

(2018). Evaluating regulatory impact assessments in education policy. American Journal of Evaluation, 40, 335–353. https://doi.org/10.1177/1098214018785463

Beyer

J. M.

(1997). Research utilization: Bridging a cultural gap between communities. Journal of Management Inquiry, 6(1), 17–22.

Blake

S. C.

Ottoson

J. M.

(2009). Knowledge utilization: Implications for evaluation. In Ottoson

J. M.

Hawe

(Eds.), Knowledge utilization, diffusion, implementation, transfer, and translation: Implications for evaluation. New Directions for evaluation (Vol. 124, pp. 21–34). Wiley.

Cook

C. E.

(1998). Lobbying for higher education: How colleges and universities influence federal policy. Vanderbilt University Press.

Creswell

J. W.

Poth

C. N.

(2018). Qualitative inquiry and research design (4th ed.). SAGE.

10.

Executive Order No. 12866, Title 3 C.F.R. 638 (1993 comp.).

11.

Executive Order No. 13563, Title 3 C.F.R. 215 (2011 comp.).

12.

Farley-Ripple

E. N.

(2012). Research use in school district central office decision making: A case study. Educational Management Administration & Leadership, 40(6), 786–806.

13.

Gibbs

(2018). Analyzing qualitative data. SAGE.

14.

Greenberg

D. H.

Mandel

M. B.

(1991). Research utilization in policymaking: A tale of two series (of social experiments). Journal of Policy Analysis and Management, 10(4), 633–656.

15.

Harris

(2017, May 25). Here’s how a student “unit record” system could change higher ed. The Chronicle of Higher Education. https://www.chronicle.com/article/Here-s-How-a-Student-Unit/240165

16.

Head

B. W.

(2015). Toward more “evidence-informed” policymaking? Public Administration Review, 76(3), 472–484.

17.

Henig

J. R.

(2008). Spin cycle: How research is used in policy debates: The case of charter schools. Russell Sage.

18.

Henig

J. R.

(2012). The politics of data use. Teachers College Record, 114(11), 1–32.

19.

Kerwin

C. M.

Furlong

S. R.

(2011). Rulemaking: How government agencies write law and make policy (4th ed.). CQ Press.

20.

Klemperer

Theisens

Kaiser

(2001). Dancing in the dark: The relationship between policy research and policy making in Dutch higher education. Comparative Education Review, 45(2), 197–219.

21.

Kochan

D. J.

(2019). Reason-giving, rulemaking, and the rule of law. University of Missouri Kansas City Law Review, 87, 525–548.

22.

Kreighbaum

(2018, August 23). Questioning the evidence. Inside Higher Education. https://www.insidehighered.com/news/2018/08/23/researcher-says-devos-misrepresented-her-work-justify-repeal-student-loan-rule?width=775&h

23.

Lubbers

J. S.

(1998). A guide to federal agency rulemaking (3rd ed.). American Bar Association.

24.

Lubbers

J. S.

(2014, December 5). Enhancing the use of negotiated rulemaking by the U.S. Department of Education. https://doi.org/10.2139/ssrn.2764954

25.

Lubienski

Scott

DeBray

(2014). The politics of research production, promotion, and utilization in educational policy. Educational Policy, 28(2), 131–144.

26.

Merriam

S. B.

Tisdell

E. J.

(2016). Qualitative research: A guide to design and implementation. Jossey-Bass.

27.

Mettler

(2014). Degrees of inequality: How the politics of higher education sabotaged the American dream. Basic Books.

28.

Miles

M. B.

Huberman

A. M.

Saldaña

(2019). Qualitative data analysis: A methods sourcebook (4th ed.). SAGE.

29.

Natow

R. S.

(2017). Higher education rulemaking: The politics of creating regulatory policy. Johns Hopkins University Press.

30.

Natow

R. S.

(2019). Negotiated rulemaking for U.S. higher education regulatory policy: A process of deliberative democracy? The Journal of Public Deliberation, 15(1), Article 9.

31.

Ness

E. C.

(2010). The role of information in the policy process: Implications for the examination of research utilization in higher education policy. In Smart

J. C.

(Ed.), Higher education: Handbook of theory and research (Vol. 25, pp. 1–49). Springer.

32.

Nutley

S. M.

Walter

Davies

H. T. O.

(2007). Using evidence: How research can inform public services. The Policy Press.

33.

Office of the Federal Register. (2011, January). A guide to the rulemaking process. https://www.federalregister.gov/uploads/2011/01/the_rulemaking_process.pdf

34.

Parsons

M. D.

(1997). Power and politics: Federal higher education policymaking in the 1990s. State University of New York Press.

35.

Pelesh

M. L.

(1994). Regulations under the Higher Education Amendments of 1992: A case study in negotiated rulemaking. Law & Contemporary Problems, 57(4), 151–170.

36.

Program Integrity: Gainful Employment, Final Regulations. 79 Fed. Reg. 64890 (2014) (to be codified at 34 C.F.R. pts. 600 and 668).

37.

Program Integrity: Gainful Employment, Final Regulations. 84 Fed. Reg. 31392 (2019) (to be codified at 34 C.F.R. pts. 600 and 668).

38.

Shapiro

Murphy

(2013). Public participation without a public: The challenge for administrative policymaking. Missouri Law Review, 78(2), 489–510.

39.

Stephan

(2017, July 12). Congress rejects teacher preparation reporting rule. The Regulatory Review. https://www.theregreview.org/2017/07/12/stephan-teacher-preparation-programs/

40.

Student Assistance General Provisions, Federal Family Education Loan Program, and William D. Ford Federal Direct Loan Program: Final regulations, 80 Fed. Reg. 67204 (2015) (to be codified at 34 C.F.R. pts. 668, 682, & 685).

41.

Student Assistance General Provisions, Federal Perkins Loan Program, Federal Family Education Loan Program, William D. Ford Federal Direct Loan Program, and Teacher Education Assistance for College and Higher Education Grant Program: Final regulations, 81 Fed. Reg. 75926 (2016) (to be codified at 34 C.F.R. pts. 30, 668, 674, 682, 685, & 686).

42.

Sunesson

Nilsson

(1988). Explaining research utilization: Beyond “functions.” Knowledge: Creation, Diffusion, Utilization, 10(2), 140–155.

43.

Suri

(2011). Purposeful sampling in qualitative research synthesis. Qualitative Research Journal, 11(2), 63–75.

44.

Teacher Preparation Issues: Final regulations, 81 Fed. Reg. 75494 (2016) (to be codified at 34 C.F.R. pts. 612 & 686).

45.

Trostle

Bronfman

Langer

(1999). How do researchers influence decision-makers? Case studies of Mexican policies. Health & Policy Planning, 14(2), 103–114.

46.

Tseng

(2012). The uses of research in policy and practice. Society for Research in Child Development.

47.

U.S. Department of Education. (2017, December 6). Gainful Employment negotiated rulemaking committee 2017-2018. https://www2.ed.gov/policy/highered/reg/hearulemaking/2017/wednesdaydecemeber6transcript.doc

48.

Weiss

C. H.

(1979). The many meanings of research utilization. Public Administration Review, 39(5), 426–431.

49.

Weiss

C. H.

(2001, July). What kind of evidence in evidence-based policy [Conference session]. Third International Inter-disciplinary Evidence-Based Policies and Indicator Systems Conference, CEM Centre, Durham, UK.

50.

Weiss

C. H.

Murphy-Graham

Birkeland

(2005). An alternate route to policy influence: How evaluations affect D.A.R.E. American Journal of Evaluation, 26(1), 12–30.

51.

Yackee

S. W.

(2019). The politics of rulemaking in the United States. Annual Review of Political Science, 22, 37–55.

52.

Zane

S. N.

Welsh

B. C.

(2018). Toward an “age of imposed use”? Evidence-based crime policy in a law and social science context. Criminal Justice Policy Review, 29(3), 280–300.