Does Contracting Achieve Better Performance for Democratic-Constitutional,Procedural Tasks? Evidence From the EEO Discrimination Complaint Process

Abstract

Contracting out has been considered one of the main performance management strategies to reduce costs and bring more expertise to government agencies. However, there is a lack of research assessing the performance of contractors compared with that of in-house agency employees, when both contractors and public employees deliver complex services. This study examines whether or not contracting achieves better performance in democratic-constitutional, procedural (DCP) tasks compared with in-house delivery, by analyzing contracting use in the Equal Employment Opportunity (EEO) discrimination complaint process. Using agency-level panel data from the Federal EEO Statistical Report of Discrimination Complaints, combined with data from the Federal Procurement Data System and the Federal Employee Viewpoint Survey, the study offers evidence resolving the competing logics for a relationship between contracting use and performance in DCP tasks. The findings show that an increase in contracting is associated with a decrease in timely completion of case investigations, which is a key measure of DCP task performance.

Keywords

contracting performance democratic-constitutional procedural tasks mission-extrinsic public values

Introduction

To enhance government performance, public agencies implement several performance management strategies as part of result-oriented public management systems. Under result-oriented management systems, there is widespread use of contracting out as one form of privatization, to “deliver a service or perform a function for the government” (Fernandez & Smith, 2005, p. 357). As the demands for services have become extensive and complicated, it is more likely that governments contract out to access private-sector expertise, lower costs, and reduce regulations to meet their policy goals. However, contracting theories such as principal–agent theory and incentive theories argue that in-house delivery is more appropriate for complex services, because there is more possibility of failing when performance is difficult to observe. Despite this warning, what happens if public agencies finally decide to contract these difficult-to-measure services out, instead of or in addition to in-house delivery? This is an important point, because in practice a government cannot always provide in-house delivery of complex services, and the contracting areas of government programs and services have been expanded, due to fiscal restraint and personnel flexibility (Brown et al., 2018; DeHoog, 1985) or even political factors (Fernandez & Brudney, 2008).

Although prior work deepens our understanding of complex contracting focused on management techniques and structures for contracts (Brown et al., 2018; Girth, 2017; Girth & Lopez, 2019; Malatesta & Smith, 2014; Petersen et al., 2019), measuring and assessing contracting performance of complex services has received limited scholarly attention. Previous studies on complex contracting explore managerial, social, and technical factors that influence contracting performance (Brown et al., 2016; Davis et al., 2016). However, there is a lack of research assessing the performance of contractors compared with that of in-house agency employees when both contractors and public employees deliver complex services (except Jung et al., 2018). Performance measurement practices play a crucial role, regarding an agency’s capacity not only to effectively manage contracts but also to accurately evaluate contractors’ performance (Amirkhanyan, 2011). As a result, along with “make or buy” or “how to manage complex contracting” questions, it is also important for scholars and practitioners to gauge performance outcomes when public agencies finally decide to contract these difficult-to-measure services out, instead of or in addition to in-house delivery.

Among the contracting areas of complex services, contracting for complex services in pursuit of democratic-constitutional, procedural (DCP) values such as equity, transparency, and individual rights deserves our attention. In particular, we know relatively little about how contractors perform in complex, non-traditional DCP tasks—for instance, contracting use in the Equal Employee Opportunity (EEO) discrimination complaint process. Scholars have warned that performance measurement processes will pay less attention to those DCP values often identified as non-mission-based or mission-extrinsic (Piotrowski & Rosenbloom, 2002; Radin, 2006). Moreover, public employees raise concerns about using contractors for DCP tasks that can be considered as “inherently government functions” (Piotrowski, 2007). As the outcomes of complex services for DCP tasks are often non-mission-based and cannot be easily measured, governments need to be more careful when contractors take part in DCP tasks.

Given this, the purpose of this study is to examine the relationship between contracting use and performance with regard to DCP tasks, by looking at contracting use in the EEO discrimination complaint process. It aims to answer the following research question:

Research Question: Does contracting achieve better performance in complex human resource management services for DCP values compared with in-house delivery?

To answer this question, the study proceeds as follows. First, this article defines DCP values and tasks, and places an emphasis on the importance of such values in the public administration domain. Second, it describes the context of the EEO discrimination complaint procedure as an example of DCP tasks. Third, this article develops a hypothesis about the relationship between contracting out and performance of DCP tasks. To do this, prior studies of contracting are reviewed and integrated into the literature on performance measurement of DCP tasks. Last, after providing a description of the data and methods, the results of the study are discussed. In particular, this study offers empirical findings about how contracting use affects performance of DCP tasks that are measured with regard to timeliness of completing procedural milestones. This study also provides the theoretical and practical implications of the findings.

DCP Values and Tasks in Public Administration

Before reviewing the current literature on contracting out and performance on DCP tasks, it is worth first clarifying what DCP values and tasks are, and discussing why we need to pay attention to these values in the public administration domain. Table 1 summarizes the definitions and main examples of DCP values and tasks in detail. To begin with, DCP values can be broadly defined as the values that an agency pursues according to legislation and the Constitution, including equity, transparency, individual rights, and due process, above and beyond specific program requirements (Baehler et al., 2014; Piotrowski et al., 2018). Piotrowski and Rosenbloom (2002) also describe democratic-constitutional values as non-mission-based or mission-extrinsic values that are “neither mission based nor part of a results-oriented calculus” (p. 643).

Table 1.

Definitions and Examples of DCP Values and Tasks.

Constructs	Definitions	Examples
DCP values	The values that an agency pursues according to legislation and the Constitution Often known as non-mission-based or mission-extrinsic values (Baehler et al., 2014)	Equity Fairness Transparency Non-discrimination Due process
DCP tasks	Non-mission-based tasks that ultimately pursue the DCP values	Responding to FOIA requests Dealing with EEO discrimination complaints

Note. DCP = democratic-constitutional, procedural; FOIA = Freedom of Information Act; EEO = Equal Employment Opportunity.

In line with this, this study defines DCP tasks as “non-mission-based tasks” that ultimately pursue DCP values that are based on the Constitution or legislation. Responding to Freedom of Information Act (FOIA) requests can be one example of a DCP task, as these tasks pursue transparency, the value an agency must pursue according to legislation. As another example, dealing with EEO discrimination complaints is one type of non-mission-based task, pursuing equity, fairness, and non-discrimination (see Table 1).

Unlike mission-based values and tasks that are mostly integral to agencies’ core missions and performance targets, DCP values and tasks are not directly related to their core missions. DCP values are “often imposed across all agencies in one-size-fits-all fashion that is not strategically tailored to individual mission” (Rosenbloom, 2017, p. 43). For example, transparency can be treated as one of the non-mission-based, DCP values for most agencies, with the exception of the National Archives and Records Administration, in that transparency is embodied in FOIA but is not necessarily reflected in agencies’ performance plans. In this case, compliance with FOIA requests can be considered as a DCP task that ultimately pursues transparency.

In public administration, it is essential to maintain a balance between managerial values (i.e., administrative efficiency and effectiveness) and DCP values, by bringing democratic-constitutional values into a performance management strategy (Radin, 2006; Stivers, 2008). Waldo (1948) strongly argues that democratic values must coexist with administrative values such as efficiency and effectiveness, with the strong belief that public administration does not differ from political theories and principles. In his book The Administrative State, he states that efficiency is “a major objective of public administration, but it must be ‘socially and humanly interpreted’” (Waldo, 1948, p. 197). Therefore, it is not surprising that “social equity should be the third pillar for the theory and practice of public administration” (Frederickson, 1990, p. 235). This is also well articulated in Frederickson’s normative idea of New Public Administration, which integrates social equity and political systems in public administration (Frederickson, 2016). Also, scholars have argued that democratic-constitutional governments count on DCP values based on legislation and the Constitution, which need the attention of agencies (Piotrowski et al., 2018).

With respect to performance management and measurement, there is existing literature on DCP values, especially focusing on equity, fairness, and transparency (see Radin, 2000; Rosenbloom, 2007; Wood & Lewis, 2017). However, “there is no universal protocol in the literature on strategic planning and performance measurement with regard to the treatment of non-mission-based, democratic-constitutional values” (Piotrowski & Rosenbloom, 2002, p. 651). Relying on performance measurement may lead agencies to pay attention to measurable indictors and to neglect other outcomes that are important but difficult to quantify (Hatry, 2007). Scholars have argued that democratic-constitutional values such as equity are less focused in result-oriented management systems, and are often ignored in performance measurement discussion, in spite of their importance in public programs (Hatry, 2007; Moynihan et al., 2011; Piotrowski et al., 2018). The main problem is that although some may argue that democratic values need to be integrated into agency missions, “their direct connection to specific performance outputs and outcomes is difficult or impossible to measure” (Piotrowski & Rosenbloom, 2002, p. 651). This inattention to non-mission-based or mission-extrinsic public values, which can be classified as a form of goal displacement, results in the neglect of important performance of DCP tasks under a result-oriented public management system.

The EEO Discrimination Complaint Process: An Example of DCP Tasks

This study looks particularly at the EEO discrimination complaint process as a main example of DCP tasks. The EEO discrimination complaint procedure is an appropriate context to look at contracting performance on DCP tasks, for two reasons. First, this complaint process can be considered as a series of DCP tasks, as the ultimate goal of the process is to promote DCP values—equity, fairness, and non-discrimination—in the federal workforce. Second, both (in-house) agency EEO employees and EEO contractors serve in the discrimination complaint process. Thus, it is relevant to examine contractors’ performance in DCP tasks compared with agency employees’ performance.

This discrimination complaint process is legally based on Title VII of the Civil Rights Act of 1964 (Title VII). According to this law, it is illegal to “discriminate against someone on the basis of race, color, religion, national origin, or sex” (U.S. Equal Employment Opportunity Commission [EEOC], 2017). From 1972 onward, the EEOC has been in charge of overseeing the enforcement of non-discrimination laws. In this process, complainants explain when and why they feel they have been treated inappropriately by the law. To deal with the discrimination complaints, EEO staff need to counsel complainants on their rights and, if the complainant requests a full review, investigate the claim to see whether discrimination actually occurred.

There are several procedural milestones that must be met within a specific time frame, when a federal employee believes that he or she has experienced discrimination in the workplace. First, once the federal employee believes that there is an incident of discrimination, they are required by regulation to contact an agency EEO counselor “within 45 days from the date discrimination is alleged to have occurred” (Lipnic, 2017, p. 4). At this stage, the EEO counselor will advise the employee to choose either EEO counseling or an alternative dispute resolution (ADR) for mediation. If the claim is not settled through this informal complaint process, the agency EEO office will issue a notice of the right to file a complaint, and the employee can file a formal discrimination complaint “within 15 days from the date notice from an EEO counselor about how to file a complaint is received” (Lipnic, 2017, p. 4). From the date the discrimination complaint is formally filed, regulations require the agency to complete the investigation within 180 days, with a few exceptions.

In every stage of the informal and formal EEO complaint process, (in-house) EEO agency employees and/or EEO contractors facilitate review of a discrimination case by serving in one of three roles: counselor, investigator, or consolidated counselor/investigator. Within this procedure, EEO contractors play a part in the process by counseling or investigating the discrimination complaints. They can be involved in procedural activities to achieve democratic-constitutional values (i.e., non-discrimination and fairness), even though they do not make the final decision on behalf of the agency.

Government Contracting for Mission-Extrinsic, DCP Tasks

Public administration scholarship has explored the effects of contracting out and the conditions in which contracting out becomes effective in public organizations (Brown & Potoski, 2003; Fernandez, 2004; Jang & Eger, 2019; Lee et al., 2019; O’Toole & Meier, 2004). Advocates for contracting argue that it is an effective strategy to enhance government performance. The majority of prior research focuses on contracting for traditional government services and products (e.g., refuse collection) that can be easily measured and is mission-based (Boitani et al., 2011; Kavanagh & Parker, 2000; Lannier & Porcher, 2014; Thompson, 2011). There is less evidence of whether that argument is still valid for non-traditional performance outcomes of DCP tasks that are relatively hard to measure and observe and are mostly mission-extrinsic. In other words, the following question still remains: Is there any evidence that contracting out is as beneficial to complex DCP tasks as it is to conventional service delivery?

Prior studies have mainly responded to this question in two ways. First, some argue that DCP values are at odds with the conventional values that contracting out mostly pursues, such as efficiency. Thus, the tension between these values may cause problems, particularly when governments contract to private firms, which are strongly “motivated by profit and consequently they may focus more on innovation and efficiency” (Brown et al., 2006, p. 327; Hart et al., 1997). Thus, contracting vendors may pursue reducing costs and enhancing efficiency at the expense of other public values, such as DCP values and service quality, if both cannot be achieved at the same time. For example, Brown et al. (2006) note,

When private contractors are forced to choose between taking steps to maintain or upgrade service quality and keeping costs low, public managers should be alert. Vendors may favor reducing expenses, particularly if this means pursuing their own (profit) goals at the expense of the government’s objectives. (p. 327)

Second, previous studies look at contracting for complex DCP tasks, focusing on the fact that these performance outcomes are difficult to measure. Proponents of privatization and small governments usually believe that contracting is still beneficial to the programs whose outcomes are difficult to observe and measure (Bennett & Johnson, 1981; Savas, 1982). However, according to contracting theories such as principal–agent theory and incentive theory, there is a great challenge for public managers when supervising contracts for difficult-to-measure services, as it is not easy to set clear standards and measures for tracking contractors’ performance (Kelman, 2002). When performance can be easily measured and assessed in government contracting, it is easier for governments to hold agents accountable for what they perform (Van Slyke, 2003; Van Slyke & Roch, 2004). Conversely, when performance outcomes are difficult to measure and observe, this is likely to create a higher risk of principal–agent problems, which may lead to contracting failure. As a result, in principle, it is often suggested that in-house delivery is more suitable for certain services and products, acknowledging that these kinds of public services can make the principal–agent problems much more complicated (Brown et al., 2006; Hefetz & Warner, 2004; Hirsch & Osborne, 2000). All in all, effective contracting for DCP tasks may depend on the degree to which performance is easy to measure, so that the contract enables principals to assess the outputs or outcomes of the service.

Measuring Performance in DCP Tasks

Due to the difficulty of measuring and assessing non-traditional performance in DCP tasks, scholars attempt to find alternative ways to measure program performance for DCP tasks. Hatry (2007) gives a useful insight by categorizing the programs with difficult-to-measure outcomes and suggesting alternative measures for each case. For example, for the programs that might take a long time to produce outcomes, he suggests that “agencies can track early intermediate outcomes such as the percent of time that reports and plans were provided on schedule” (Hatry, 2007, p. 76). His suggestions are worthwhile in that most outcomes of the programs for DCP values may take many years to be achieved and sometimes might not appear frequently. Other studies also suggest ways in which agencies can regularly track program outcomes using balanced scorecards (Rosenbloom, 2007), or assess outputs and outcomes for DCP tasks by considering the activities as “proxies for desired outcomes” (Brown et al., 2006, p. 326), if monitoring the activities is undemanding. This implies that having those proxy measures would be helpful in reducing the difficulties of measuring performance in DCP endeavors.

In contracting relationships, even though outcomes are difficult to measure, “service performance can still be assessed if it is relatively straightforward to monitor the activities of the vendor, and these activities are reasonable proxies for desired outcomes” (Brown et al., 2006, p. 326). Thus, for performance of DCP tasks which are not easy to measure, public agencies can utilize proxies, and this mitigates the possible drawbacks that occur in contracting out with difficult-to-measure services. As a result, it is possible to assess contracting performance of DCP tasks by focusing on the milestones that must be met in the procedure of DCP tasks.

This article utilizes timeliness of completing procedural milestones as the performance measure for DCP tasks (see Table 2). Hatry (2007) notes that, for the programs whose outcomes do not occur very often (such as emergency response programs and federal or state litigation programs), “indicators might include response times and the number of people who are served over the periods” (p. 76). In line with this, Wood and Lewis (2017) specifically measure agency performance of non-mission tasks in the FOIA process, which pursues transparency. Because performance of DCP tasks (the tasks in the FOIA process) is hard to measure and observe directly, they instead used four proxy measures—time to confirm, time to respond, the number of exemptions, and whether or not the agency charged fees. In Piotrowski’s (2007) study, the speed of processing tasks was taken as a proxy measure to assess contractors’ performance in FOIA requests, which is hard to measure directly. In other words, they focus on timeliness measures and/or specific activities that must be carried out during the process as alternative measures of performance for non-mission-based, DCP tasks (see Table 2).

Table 2.

Performance Measurement of DCP Tasks.

Study	Main DCP tasks	DCP values	Legally based on	Performance measures on DCP tasks
Piotrowski (2007)	Responding to FOIA requests	Transparency	FOIA	The speed of processing tasks
Wood & Lewis (2017)	Responding to FOIA requests	Transparency	FOIA	Time to confirm Time to respond The number of exemptions Whether or not the agency charged fees
This study	Dealing with discrimination complaint process	Fairness and non-discrimination	Title VII	The percentage of timely completed cases in stages (i.e., timeliness)

Note. DCP = democratic-constitutional, procedural; FOIA = Freedom of Information Act.

From a practitioner’s point of view, taking timeliness as a performance measure of DCP tasks is also imperative. As mentioned earlier, the DCP tasks are basically about achieving DCP values that are based on the Constitution and legislation. Accordingly, the provision of the DCP tasks is often implemented under federal laws and regulations that set specific time frames or time limits (e.g., FOIA, and Title VII of the Civil Rights Act of 1964). For example, once a FOIA request is received, agencies have 20 working days to respond to the request unless it is in unusual circumstances. Also, in the federal EEO discrimination complaint process, regulations require that an agency complete the investigation within 180 days from the date of the formally filed discrimination complaint. If tasks are being delayed, it affects the recipients of the services. Thus, it is emphasized that “delays in processing federal EEO complaints have been long-standing concerns of the EEOC, other federal agencies, and Congress” (U.S. Government Accountability Office, 2009, p. 1), such that practitioners seek to “measure agencies’ progress [in EEO programs] as the timeliness of investigations” (U.S. Government Accountability Office, 2009, p. 1).

In short, this study suggests that the performance of complex services with DCP tasks can be assessed using a proxy measure—timeliness of completing the milestones that must be carried out during the process. Acknowledging the difficulty of directly assessing performance of DCP tasks, having those proxies might be useful for principals when tracking and monitoring outputs of procedural activities, which is one way of increasing contract management capacity.

Contracting Performance on Mission-Extrinsic, DCP Tasks

In light of the literature on contracting and performance measurement for DCP tasks, the relationship between contracting use and its performance with regard to DCP tasks is interesting to examine. There is a small but growing number of empirical studies looking into contracting performance for complex services; studies have shown mixed results in terms of the impact of contracting out on performance, with regard to non-traditional outcomes of DCP tasks. On one hand, a few studies find that contracting out for these tasks has a positive influence on performance, in spite of difficult-to-measure outcomes, which is consistent with what proponents of contracting out have maintained. In the context of FOIA, for example, “the use of federal FOIA contractors is not widespread, but if used judiciously to expedite old FOIA requests, they may improve the speed of the release process” (Piotrowski, 2007, p. 80). In addition, some evidence suggests that contracting out for social service programs is more effective when competition, trained state contract managers, and government managerial capacity exist (Romzek & Johnston, 2002; Van Slyke, 2003).

On the contrary, some findings show either a negative or a not positive relationship between contracting out and performance in DCP tasks. Contracting out can fail to increase performance of social service programs when there is still monopoly power in the private sector (Van Slyke, 2003). In addition, prisons managed by governments do a better job in some dimensions of quality that are relatively difficult to measure (e.g., prisoners’ safety), whereas privately managed prisons have better performance in easy-to-measure dimensions of quality (e.g., prisoners’ purposeful activities) (Alonso & Andrews, 2016). As a more normative view, opponents of contracting out may still argue that contractors, who often value their profits and self-interest more, are not the right people to deliver public goods and services for DCP values, because of the conflict between these values. Thus, government employees, not contractors, should play a role in releasing government information (Piotrowski, 2007). Privatization generally reduces transparency, because private contractors are not legally responsible for regulations (Rosenbloom, 2007). Combined, there are concerns that private contractors will not honor DCP values, but they are generally responsive to measured program outcomes. Given the conflicting arguments and empirical results in the existing literature, this study proposes a non-directional hypothesis:

Hypothesis: Contracting use in DCP tasks will be associated with agency performance of the DCP tasks measured in timeliness measure.

In addition to evaluating the effects of contracting on the performance of DCP tasks, the existing literature suggests that staff expertise and task complexity are likely to affect performance and are thus included as control variables. First, in terms of staff expertise, previous studies have argued that public agencies are easily captured by contractors possessing high skill sets, when the agencies do not have employees with sufficient knowledge and skills to structure detailed contracts, and monitor and assess contractors’ performance (Van Slyke & Hammonds, 2003). Second, task complexity can be referred to as “the degree of difficulty in specifying and monitoring the terms and conditions of a transaction” (Globerman & Vining, 1996, p. 579). High task complexity means tasks consist of a series of rules or legal requirements, such that it is more difficult to draft specific terms and conditions. Accordingly, when highly complicated tasks are contracted out, it makes it extremely difficult for public managers to specify contract details (Fernandez, 2004).

Data and Measures

To test the hypothesis, this study mainly uses the 2013–2016 Federal EEO Statistical Report of Discrimination Complaints (EEOC Form 462 reports) data which are filed by federal agencies, combined with data from the Federal Procurement Data System and the Federal Employee Viewpoint Survey. The sample includes 356 observations at agency level (89 federal agencies for each of 4 years). Agencies excluded from the sample include the following: all National Guard agency units, the United States Postal Service (USPS), four intelligence-related agencies due to incomplete data, and 11 other federal agencies that are missing one or more years of Form 462 reports.

Dependent Variable: Performance in DCP Tasks

As discussed earlier, this study employs a proxy measure—timeliness of completing milestones that should be met—for measuring performance in DCP tasks. Timeliness is calculated by the following items below, measured in a proportion ranging from 0 to 1:

Timeliness = A consolidated measure calculating the average of the proportion of timely completed cases of counseling (excluding remands), the proportion of timely completed cases of investigations, and the proportion of timely completed cases of merit final agency decisions (FADs) without administrative judge (AJ) decision (see details in Table 3).

Table 3.

Summary of Variables, Survey Items/Calculations, and Sources.

Variable	Survey item(s)/calculation(s)	Source
Dependent variable
Performance of DCP tasks	Timeliness: The average of the three following proportions: The proportion of timely completed/ended counseling (excluding remands) The proportion of timely completed investigations The proportion of timely merit FADs without AJ decision	EEOC Form 462 reports
Independent variable
Contracting use	Model I (continuous variable) The percentage of EEO contractors = 100 × (The number of EEO contractors / A total number of EEO workforce) Model II (four dummy variables based on the quartiles) Low contracting use = from the min to Q1 (base group) Low/medium contracting use = from Q1 to Q2 Medium/high contracting use = from Q2 to Q3 High contracting use = from Q3 to max	EEOC Form 462 reports
Control variables
Staff expertise	The weighted average of the EEO staff receiving training hours (in six points) = {1 × (The number of new staff receiving no training) + 2 × (The number of new staff receiving more than 8 hr and less than 32 hr) + 3 × (The number of new staff receiving more than or equal to 32 hr) + 4 × (The number of experienced staff receiving no training) + 5 × (The number of experienced staff receiving more than 8 hr and less than 32 hr) + 6 × (The number of experienced staff receiving more than or equal to 32 hr)} / Total number of EEO staff	EEOC Form 462 reports
Task complexity	Proxy metric: The proportion of unresolved cases in a pre-complaint process (i.e., ADR process) = 1 – (The number of completed ADR cases with settlements with benefits or no formal complaint filed / The number of accepted into ADR program)	EEOC Form 462 reports
Independence of EEO director	“Does the EEO director report to the agency head?” (Yes = 1, No = 0)	EEOC Form 462 reports
Types of agency mission	10 dummy variables (financial regulation, science and research, energy and environment, public health, economic development, oversight, law enforcement, social support, national security, and all other agencies)	EEOC Form 462 reports
Diversity climate	The sum of respondents’ weighted answers: “Policies and programs promote diversity in the workplace (for example, recruiting minorities and women, training and awareness of diversity issues, mentoring).” (Q. 34) “Supervisors/team leaders in my work unit are committed to a workforce representing all segments of society.” (Q. 45) “Managers/supervisors/team leaders work well with employees from different backgrounds.” (Q. 55) (5 = strongly agree to 1 = strongly disagree)	FEVS
Contracting vendors	Dummy variables for 46 contracting vendors participating the EEO discrimination complaint process	FPDS
Workforce (ln)	log (The number of employees of the agency)	EEOC Form 462 reports
Year	Year dummy variables from 2013 to 2016 (base year = 2013)	EEOC Form 462 reports

Note. DCP = democratic-constitutional, procedural; FADs = final agency decisions; AJ = administrative judge; EEOC = U.S. Equal Employment Opportunity Commission; EEO = Equal Employment Opportunity; ADR = alternative dispute resolution; FEVS = Federal Employee Viewpoint Survey; FPDS = Federal Procurement Data System.

Independent Variable: Contracting Use

This study operationalizes contracting use with the percentage of EEO contractors out of the total EEO workforce. The percentage of EEO contractors is calculated by taking the number of EEO contractors divided by the total EEO workforce (i.e., all counselors, investigators, and counselors/investigators) and then it is multiplied by 100. If an agency only has in-house agency EEO employees in the EEO workforce, Contracting Use is indicated as 0 for the agency. However, if an agency only uses EEO contractors for dealing with the agency’s EEO discrimination complaint cases, Contracting Use is indicated as 100. Thus, depending on how much an agency uses EEO contractors out of the total EEO workforce, Contracting Use is measured in a percentage ranging from 0 to 100.

Control Variables

As mentioned earlier, Staff Expertise and Task Complexity are controlled as the factors that might also account for contracting performance. To measure staff expertise, this study develops a score based on the number of hours of training completed by experienced and new staff each year. This training score is then weighted more heavily for longer tenured staff, compared with newly hired staff, to account for differences in on-the-job experience. The detailed formula is presented in Table 3.

To measure task complexity, this study develops a proxy measure using the proportion of unresolved cases in a pre-complaint process. When the discrimination complaint case is complex and it is hard to find the cause, it might not be easily resolved in the pre-complaint process that is the ADR process. Thus, the rate of unresolved cases in the ADR process (i.e., the ADR resolution rate subtracted from one) can be used as a proxy when measuring task complexity. The detailed formula is shown in Table 3.

Other control variables account for agency characteristics or year dummy variables. First, this study includes the Independence of the EEO Director variable, which indicates whether or not an agency EEO director reports to the agency head. The EEOC (2014) describes why independence of agency EEO directors is crucial, stating that

[b]y placing the EEO director in a direct reporting relationship to the agency head, the agency underscores the importance of the EEO to the agency’s mission and ensures that the EEO director is able to act with the greatest degree of independence. (I-1)

Second, the type of agency categorized by the mission is controlled by including Types of Agency Mission variables. Agency mission types can be potentially related to the EEO process in a way, so it might be helpful to control such differences for the analysis. Following Rubin et al.’s (2017) work, this article adopts 10 categories based on agency mission areas, including financial regulation, science and research, energy and environment, public health, economic development, oversight, law enforcement, social support, national security, and all other agencies. Third, Diversity Climate is controlled in this model to consider the agency’s environment with regard to workforce diversity (Choi, 2017). Fourth, Contracting Vendors are included as categorical variables in the model to control the potential influence of vendor units on performance of DCP tasks. Finally, the model controls for the size of the agencies by taking the natural log of the number of employees, and Year dummy variables (base year is 2013) are added into the model to control for the effect of the unchanging characteristics of the year across the units.

Model and Methodology

Deeply grounded in the theories and literature, this research suggests a model specification to test a hypothesis as follows:

\begin{array}{l} {PERFORMANCE}_{-} {TIMELINESS}_{i t} = β_{0} + β_{1} {CONTRACT}_{i t} + \\ C + δ_{t} + υ_{i t} \end{array}

where ${PERFORMANCE}_{-} {TIMELINESS}_{i t}$ is the proportion of timely completed tasks, ${CONTRACT}_{i t}$ is the percentage of having EEO contractors out of total EEO staff (a continuous variable for Model I or a categorical variable for Model II), C includes control variables, $δ_{t}$ represents year fixed effect, $υ_{i t}$ is an error term, β₀ and β₁ are estimable parameters, and i is the agency and t is the year.

This study uses fractional response generalized linear models (GLMs) and then ordinary least squares (OLS) regression for robustness checks. The reason for using fractional response GLM instead of OLS regression is that the dependent variable (i.e., timeliness) is measured in a proportion. It is possible that OLS assumptions might not be met when the dependent variable is indicated as a proportion. Accordingly, OLS regression models might not be appropriate for predicting timeliness operationalized in the proportion that has an interval from 0 to 1. In this case, a fractional response GLM is more suitable for predicting values, which is “quasi-maximum likelihood estimation with a fractional logit technique” (Papke & Wooldridge, 1996; Smith & Fernandez, 2010, p. 92).

In particular, this study runs two separate models for the dependent variable—timeliness—to provide robust regression results as well as a deeper understanding of the results. Model I tests the effect of contracting on agency performance in DCP tasks by utilizing contracting use as a continuous variable. Model II aims to test the difference of agency performance in DCP tasks between high contracting use agencies and low contracting use agencies. To look at extreme cases of contracting use, the contracting variable is recoded into a categorical variable with four categories based on the quartile of contracting use. The first category represents Low Contracting Use if the percentage of contracting use is equal to or below the first quartile, which is the bottom 25%. The second category is indicated as Low/Medium Contracting Use, if the percentage of contracting is between the first and the second quartile and third category is Medium/High Contracting Use, where the percentage of contracting is between the second and the third quartile. The last category is High Contracting Use, where the percentage of contracting is equal to or above the third quartile, which is the top 25%. For the purpose of this study, the focus will be on a comparison between the top 25% group (i.e., High Contracting Use) and the bottom 25% group (i.e., Low Contracting Use) in Model II.

Results

Prior to estimating the models, the descriptive statistics of the study are presented in Table 4. Overall, the percentage of contracting use in EEO workforce (i.e., the percentage of EEO contractors out of total EEO staff) is roughly 50%. This shows that the EEO workforce is quite balanced between (in-house) agency EEO employees and EEO contractors in federal agencies. However, when we look at the percentages in detail, it is interesting to see a big difference between the percentage of EEO contractors who are counselors or investigators. In terms of counselors, the percentage of contracting use is about 15%. When it comes to investigators, the percentage of EEO contractors is about 89%. This tells us that federal agencies tend to use contracting out, especially for conducting investigations that need a much higher skill set.

Table 4.

Descriptive Statistics.

Variables	N	M	SD	Minimum	Maximum
Timeliness	356	0.664	0.178	0.232	1
Contracting use	356	49.985	33.890	0	100
Staff expertise	356	4.862	0.362	2.5	6
Task complexity	356	0.520	0.227	0	1
Independence of EEO director	356	0.593	0.492	0	1
Workforce (ln)	356	8.808	1.284	6.333	12.528
Diversity climate	356	10.585	2.350	0	12.588

Note. EEO = Equal Employment Opportunity.

The correlation matrix offers preliminary snapshots of the association between contracting use and performance measured in timeliness (see Table 5). Contracting use is significantly and negatively correlated with timeliness when completing milestones in DCP tasks. In addition, there is a positive correlation between staff expertise and timeliness in completing tasks, whereas there is a negative correlation between workforce and timeliness. The results of the fractional response GLM estimations presented in Table 6 indicate that agencies using more contracting in DCP tasks experience lower performance measured in timeliness of completing DCP tasks. When contracting use is indicated as a continuous variable in Model I, there is a significant and negative relationship between contracting use and timeliness. If a percentage of contracting use for DCP tasks increases, the proportion of timely completed DCP tasks decreases. In Model II, where the high percentage of contracting use group (i.e., High Contracting Use) is compared with the low percentage of contracting use group (i.e., Low Contracting Use), the results are also consistent with the previous findings of Model I. The findings reveal that High Contracting Use agencies are more likely to have untimely completed DCP tasks compared with Low Contracting Use agencies (see Table 6).

Table 5.

Correlation Matrix.

Variables	(1)	(2)	(3)	(4)	(5)	(6)
1. Timeliness	1
2. Contracting use	−.1866*	1
3. Staff expertise	.1051*	.1002	1
4. Task complexity	.0745	.1420*	.0303	1
5. Workforce (ln)	−.1172*	−.3290*	−.0081	−.1560*	1
6. Diversity climate	−.0451	−.1058*	−.0028	−.0055	.0711	1

p < .05.

Table 6.

Fraction Response Generalized Linear Regression Results (Dependent Variable: Timeliness).

Variables	(1)	(2)
Variables	Model I	Model II
Contracting use (continuous)	−0.009**	—
	(0.002)	—
Low/medium contracting use	—	−0.383**
	—	(0.129)
Medium/high contracting use	—	−0.713**
	—	(0.140)
High contracting use	—	−0.615**
	—	(0.164)
Staff expertise	0.239*	0.233*
	(0.116)	(0.113)
Task complexity	0.104	0.058
	(0.203)	(0.204)
Independence of EEO director	−0.156	−0.138
	(0.098)	(0.098)
Workforce (ln)	−0.104*	−0.085^†
	(0.038)	(0.048)
Diversity climate	0.001	7.86e–05
	(0.022)	(0.022)
Constant	0.555	0.444
	(0.702)	(0.701)

Observations	356	356
Agency FE	Agency mission	Agency mission
Year FE	Yes	Yes
Log pseudolikelihood	−150.5	−150.6

Note. Unstandardized coefficients are reported. Robust standard errors in parentheses. The reference group for Contracting Use variable is Category 0: Low. Agency missions, contracting vendors, and years are controlled in the regression model but not reported. EEO = Equal Employment Opportunity; FE = Fixed Effects.

†

p < .1. *p < .05. **p < .01.

Table 7 reports the marginal effects from the fractional response GLM estimations, which offer more substantial findings. The marginal effects indicate that a one-unit increase of contracting use (i.e., one percentage point) decreases 0.19 percentage points in timeliness of completing DCP tasks at the mean values of all predictors (see Table 7). In other words, a 10-percentage point increase in contracting use will result in about a two-percentage point decrease in timely completed DCP tasks. Focusing on the extreme cases, the marginal effects from Model II indicate that agencies that use contractors at the highest rates decrease the probability of having timely completed tasks (i.e., timeliness) by approximately 13 percentage points, compared with the reference group agencies that use contractors at the lowest rates.

Table 7.

Marginal Effects From the Fraction Response Generalized Linear Regression Models.

Variables	Timeliness
	Model I		Model II
	d_y/d_x	SE	d_y/d_x	SE
Contracting use (continuous)	−0.0019	0.0004	—	—
Low/medium contracting use	—	—	−0.0767	0.0254
Medium/high contracting use	—	—	−0.1518	0.0286
High contracting use	—	—	−0.1289	0.0341
Staff expertise	0.0526	0.0256	0.0511	0.0248
Task complexity	0.0228	0.0445	0.0127	0.0449
Independence of EEO director	−0.0342	0.0215	−0.0304	0.0215
Workforce (ln)	−0.0230	0.0104	−0.0186	0.0106

Note. d_y/d_x for factor levels is the discrete change from the base level. EEO = Equal Employment Opportunity.

Table 8 presents the results of the OLS regression analysis for robustness checks. While it is not the primary goal of using OLS for timeliness measure, it is worth mentioning that the OLS regression results are consistent with the previous findings of the fractional logit estimation using a GLM: a significant and negative association between timeliness and contracting use. In Model II, the findings show that High Contracting Use agencies are less likely to complete DCP tasks in a timely manner compared with Low Contracting Use agencies (see Table 8).

Table 8.

Robustness Checks: OLS Regression Results.

Variables	Timeliness
	(1)	(2)
	Model I	Model II
Contracting use	−0.002**	—
(Continuous)	(0.0003)	—
Low/medium contracting use	—	−0.078**
	—	(0.028)
Medium/high contracting use	—	−0.150**
	—	(0.032)
High contracting use	—	−0.122**
	—	(0.036)
Staff expertise	0.052^†	0.050^†
	(0.028)	(0.028)
Task complexity	0.020	0.011
	(0.046)	(0.047)
Independence of EEO	−0.034	−0.029
Director	(0.023)	(0.023)
Workforce (ln)	−0.020^†	−0.016
	(0.012)	(0.011)
Diversity climate	−0.0005	−0.0004
	(0.005)	(0.005)
Constant	0.617**	0.597**
	(0.164)	(0.165)

Observations	356	356
Adjusted R²	.283	.277

Note. Unstandardized coefficients are reported. Robust standard errors in parentheses. The reference group for Contracting Use variable is Category 0: Low. Agency missions, contracting vendors, and years are controlled in the regression model but not reported. OLS = ordinary least squares; EEO = Equal Employment Opportunity.

†

p < .1. *p < .05. **p < .01.

Discussion and Conclusion

The goal of this study is to examine the influence of contracting out on agency performance in DCP tasks that are mission-extrinsic and difficult to measure. This study offers intriguing findings on the effects of contracting use on agency performance in DCP tasks measured in a proxy metric: timeliness of completing DCP tasks. The findings show that there is a negative association between contracting use and timeliness as performance measures of DCP tasks. Furthermore, the difference in performance of DCP tasks between high contracting use agencies and low contracting ones further illustrates the findings, showing the robustness of the results. This study is important because existing studies on contract performance lack examples of contracting performance for complex human resource management services pursuing DCP values.

The findings convey two meaningful implications and contributions to the literature on contracting performance for complex services. First, this study tests competing logics and provides empirical evidence on the relationship between contracting use and performance in DCP tasks that are mission-extrinsic and not easily measured. The findings are consistent with some concerns that previous studies have raised, confirming that managing contracting performance of DCP tasks is demanding. This implies that contractors might not thoroughly consider completing their case or job in a timely manner, although the regulations require them to do so. Also, compared with in-house agency EEO employees, it is possible that contractors are less likely to value equity and non-discrimination in the workforce. Thus, the negative relationship between contracting use and performance of DCP tasks reaffirms the argument that contracting out for complex tasks may not be effective due to a higher risk of principal–agent problems (Brown et al., 2006). However, this does not necessarily mean that agencies should not hire more contractors for DCP tasks. Rather, the study suggests that it is important to have effective management strategies that hold contractors accountable for performance in DCP tasks, which is consistent with the previous contracting studies (Romzek & Johnston, 2002; Yang et al., 2009).

Second, this study contributes to the literature on contracting performance for complex services by integrating the literature on performance measurement in DCP tasks into the contracting studies. While most previous studies have examined contracting cases for traditional services and products that are relatively easily measured and mission-based, they have not fully explored other cases of contracting: contracting for services that are mission-extrinsic and difficult to assess. In this sense, it is worthwhile to analyze contracting performance in the EEO discrimination complaint procedure, which is non-mission-based, and DCP tasks that promote equity and fairness. More importantly, this study sheds light on the measurement issues that scholars have raised, offering the use of proxy or alternative metrics to assess performance in DCP tasks that are difficult to directly measure. As many scholars such as Piotrowski and Rosenbloom have argued, DCP values and tasks should be measured to make agencies pay more attention to those values in an era of performance regimes. The results of the study offer evidence on this point, measuring contracting performance as timeliness of completing specific milestones that must be met to accomplish mission-extrinsic and DCP values.

This study provides empirical evidence of the impact of contracting out on performance of DCP tasks using a quantitative analysis. That said, it is limited to revealing the performance mechanism of the EEO discrimination complaint procedure by considering a more nuanced context. Also, the analysis is still not causal because there might be some selection bias issues around why an agency chooses to contract out, selects specific contractors, and allocates complaint cases between contractors and agency employees if an agency has both. Therefore, future research could add more qualitative components to elucidate contextual explanations on how federal agencies select contractors, and how cases are being selected or assigned to the EEO employees and the EEO contractors. The findings of the study might be more robust if it could analyze sufficient qualitative data through more contract documents or interviews with EEO contractors and agency EEO employees. For example, future studies could contain interviews with EEO contractors and agency employees, asking which factors might hinder better performance in the discrimination complaint process. These qualitative analyses would be useful to show the dynamics of contracting out performance in depth.

In conclusion, the results deepen our understanding of the effect of contracting use on performance in DCP tasks. Given the paucity of empirical evidence, it is worthwhile to examine whether the use of contracting out affects performance of DCP tasks that are mission-extrinsic and not easily measured. Testing a non-directional hypothesis in the context of the EEO discrimination complaint procedure, we find that contracting use in DCP tasks is negatively associated with agency performance of DCP tasks. The analysis of the study somewhat challenges what the proponents of contracting believe. But the findings further the discussion, suggesting that contracting needs more sophisticated contract designs to hold contractors more accountable for the performance of DCP tasks, considering the alignment of competing values and the difficulty of measuring the outcomes.

Footnotes

Acknowledgements

The author is grateful to Ellen Rubin, Don Moynihan, Edmund Stazyk, and Shinwoo Lee for the helpful comments and feedback and is deeply thankful to the blind reviewers for their feedback and support. Any errors found are author’s responsibility.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Iseul Choi

Author Biography

Iseul Choi is a doctoral candidate in Public Administration at University at Albany, State University of New York and joining the faculty in the School of Public Administration at the University of New Mexico as of Fall 2020. Her research focuses on performance management, organizational behavior, and human resource management.

References

Alonso

J. M.

Andrews

(2016). How privatization affects public service quality: An empirical analysis of prisons in England and Wales, 1998–2012. International Public Management Journal, 19(2), 235–263. https://doi.org/10.1080/10967494.2015.1048913

Amirkhanyan

A. A.

(2011). What is the effect of performance measurement on perceived accountability effectiveness in state and local government contracts? Public Performance & Management Review, 35(2), 303–339. https://doi.org/10.2753/PMR1530-9576350204

Baehler

Liu

A. C.

Rosenbloom

D. H.

(2014). Mission-extrinsic public values as an extension of regime values: Examples from the United States and the People’s Republic of China. Administration & Society, 46(2), 199–219. https://doi.org/10.1177/0095399713515873

Bennett

J. T.

Johnson

M. H.

(1981). Better government at half the price: Private production of public services. Caroline House.

Boitani

Nicolini

Scarpa

(2011). Do competition and ownership matter? Evidence from local public transport in Europe. Applied Economics, 45(11), 1419–1434. https://doi.org/10.1080/00036846.2011.617702

Brown

T. L.

Potoski

(2003). Contract-management capacity in municipal and county governments. Public Administration Review, 63(2), 153–164. https://www-jstor-org.web.bisu.edu.cn/stable/977587

Brown

T. L.

Potoski

Van Slyke

D. M.

(2006). Managing public service contracts: Aligning values, institutions, and markets. Public Administration Review, 66(3), 323–331. https://www-jstor-org.web.bisu.edu.cn/stable/3843914

Brown

T. L.

Potoski

Van Slyke

D. M.

(2016). Managing complex contracts: A theoretical approach. Journal of Public Administration Research and Theory, 26(2), 294–308. https://doi.org/10.1093/jopart/muv004

Brown

T. L.

Potoski

Van Slyke

D. M.

(2018). Complex contracting: Management challenges and solutions. Public Administration Review, 78(5), 739–747. https://doi.org/10.1111/puar.12959

10.

Choi

(2017). Workforce diversity and job satisfaction of the majority and the minority: Analyzing the asymmetrical effects of relational demography on whites and racial/ethnic minorities. Review of Public Personnel Administration, 37(1), 84–107. https://doi.org/10.1177/0734371X15623617

11.

Davis

R. S.

Girth

A. M.

Stazyk

E. C.

(2016). Social and technical determinants of perceived contract performance: Rules, autonomy, and ethics. Public Performance and Management Review, 39(3), 728–755. https://doi.org/10.1080/15309576.2015.1137773

12.

DeHoog

R. H.

(1985). Human services contracting: Environmental, behavioral, and organizational conditions. Administration & Society, 16(4), 427–454.

13.

Fernandez

(2004). Explaining contracting effectiveness: An empirical analysis of contracting for services among local governments. The University of Georgia.

14.

Fernandez

Brudney

J. L.

(2008). Exploring variations in contracting for services among American local governments: Do politics still matter? The American Review of Public Administration, 38(4), 439–462.

15.

Fernandez

Smith

C. R.

(2005). Looking for evidence of public employee opposition to privatization: An empirical study with implications for practice. Review of Public Personnel Administration, 26(4), 356–381. https://doi.org/10.1177/0734371X05281629

16.

Frederickson

H. G.

(1990). Public administration and social equity. Public Administration Review, 50(2), 228–237. https://www-jstor-org.web.bisu.edu.cn/stable/976870

17.

Frederickson

H. G.

(2016). Toward a new public administration. In Shafritz

J. M.

Hyde

A. C.

(Eds.), Classics of public administration (8th ed., pp. 282–294). Cengage Learning.

18.

Girth

A. M.

(2017). Incentives in third-party governance: Management practices and accountability implications. Public Administration Review, 77(3), 433–444. https://doi.org/10.1111/puar.12645

19.

Girth

A. M.

Lopez

L. E.

(2019). Contract design, complexity, and incentives: Evidence from U.S. federal agencies. The American Review of Public Administration, 49(3), 325–337. https://doi.org/10.1177/0275074018787558

20.

Globerman

Vining

A. R.

(1996). A framework for evaluating the government contracting-out decision with an application to information technology. Public Administration Review, 56(6), 577–586.

21.

Hart

Shleifer

Vishny

R. W.

(1997). The proper scope of government: Theory and an application to prisons. Quarterly Journal of Economics, 112(4), 1126–1161. https://doi.org/10.1628/0932456012974558

22.

Hatry

H. P.

(2007). Performance measurement. Rowman & Littlefield.

23.

Hefetz

Warner

(2004). Privatization and its reverse: Explaining the dynamics of the government contracting process. Journal of Public Administration Research and Theory, 14(2), 171–190. https://doi.org/10.1093/jopart/muh012

24.

Hirsch

W. Z.

Osborne

(2000). Privatization of government services: Pressure-group resistance and service transparency. Journal of Labor Research, 21(2), 315–326. https://link-springer-com-443.web.bisu.edu.cn/content/pdf/10.1007/s12122-000-1050-z.pdf

25.

Jang

Eger

R. J.

(2019). The effects of state delinquent tax collection outsourcing on administrative effectiveness, efficiency, and procedural fairness. The American Review of Public Administration, 49(2), 236–251. https://doi.org/10.1177/0275074018759435

26.

Jung

Malatesta

LaLonde

R. J.

(2018). Should government go it alone or with a partner? A comparison of outcomes from a work release program using different policy tools. Public Administration Review, 78(4), 545–555. https://doi.org/10.1111/puar.12948

27.

Kavanagh

Parker

(2000). Managing the contract: A transaction cost analysis of externalisation. Local Government Studies, 26(4), 1–22.

28.

Kelman

S. J.

(2002). Contracting. In Salamon

L. M.

(Ed.), The tools of government: A guide to the new governance (pp. 282–318). Oxford University Press.

29.

Lannier

A. L.

Porcher

(2014). Efficiency in the public and private French water utilities: Prospects for benchmarking. Applied Economics, 46(5), 556–572. https://doi.org/10.1080/00036846.2013.857002

30.

Lee

G. R.

Lee

Malatesta

Fernandez

(2019). Outsourcing and organizational performance: The employee perspective. The American Review of Public Administration, 49(8), 973–986. https://doi.org/10.1177/0275074019855469

31.

Lipnic

V. A.

(2017). Testimony before Committee on House Administration “Preventing Sexual Harassment in the Congressional Workplace: Examining Reforms to the Congressional Accountability Act.” https://cha.house.gov/hearing/preventing-sexual-harassment-congressional-workplace-examining-reforms-congressional

32.

Malatesta

Smith

(2014). Designing contracts for complex services. Public Administration, 92(3), 531–548. https://doi.org/10.1111/padm.12004

33.

Moynihan

D. P.

Fernandez

Kim

Leroux

K. M.

Piotrowski

S. J.

Wright

B. E.

Yang

(2011). Performance regimes amidst governance complexity. Journal of Public Administration Research and Theory, 21(Suppl. 1), i141–i155. https://doi.org/10.1093/jopart/muq059

34.

O’Toole

L. J.

Meier

K. J.

(2004). Parkinson’s law and the new public management? Contracting determinants and service-quality consequences in public education. Public Administration Review, 64(3), 342–352. https://www-jstor-org.web.bisu.edu.cn/stable/3542599

35.

Papke

L. E.

Wooldridge

J. M.

(1996). Econometric methods for fractional response variables with an application to 401(k) plan participation rates. Journal of Applied Econometrics, 11, 619–632.

36.

Petersen

O. H.

Baekkeskov

Potoski

Brown

T. L.

(2019). Measuring and managing ex ante transaction costs in public sector contracting. Public Administration Review, 79(5), 641–650. https://doi.org/10.1111/puar.13048

37.

Piotrowski

(2007). The impact of contracting. In Governmental transparency in the path of administration reform (pp. 75–88). State University of New York Press.

38.

Piotrowski

Rosenbloom

D. H.

(2002). Nonmission-based values in results-oriented public management: The case of freedom of information. Public Administration Review, 62(6), 643–657. https://doi.org/10.1111/1540-6210.00247

39.

Piotrowski

Rosenbloom

D. H.

Kang

Ingrams

(2018). Levels of value integration in federal agencies’ mission and value statements: Is open government a performance target of U.S. Federal Agencies? Public Administration Review, 78(5), 705–716. https://doi.org/10.1111/puar.12937

40.

Radin

B. A.

(2000). The Government Performance and Results Act and the tradition of federal management reform: Square pegs in round holes? Journal of Public Administration Research and Theory, 10(1), 111–135. https://doi.org/10.1093/oxfordjournals.jpart.a024260

41.

Radin

B. A.

(2006). Challenging the performance movement: Accountability, complexity, and democratic values. Georgetown University Press.

42.

Romzek

B. S.

Johnston

J. M.

(2002). Effective contract implementation and management: A preliminary model. Journal of Public Administration Research and Theory, 12(3), 423–453. https://www-jstor-org.web.bisu.edu.cn/stable/3525732

43.

Rosenbloom

D. H.

(2007). Reinventing administrative prescriptions: The case for democratic-constitutional impact statements and scorecards. Public Administration Review, 67(1), 28–39.

44.

Rosenbloom

D. H.

(2017). Beyond efficiency: Value frameworks for public administration. Chinese Public Administration Review, 8(1), 37–46. https://doi.org/10.22140/cpar.v8i1.143

45.

Rubin

E. V.

Park

Y. J.

Alteri

(2017). Understanding the link between retaliation claims and agency EEO resources in the federal government.

46.

Savas

E. S.

(1982). Privatizing the public sector: How to shrink governments. Chatham House.

47.

Smith

C. R.

Fernandez

(2010). Equity in federal contracting: Examining the link between minority representation and federal procurement decisions. Public Administration Review, 70(1), 87–96. https://www-jstor-org.web.bisu.edu.cn/stable/40469113

48.

Stivers

(2008). The significance of the administrative state. Public Administration Review, 68(1), 53–56.

49.

Thompson

(2011). The estimated cost impact of privatizing student transportation in Minnesota school districts. Public Choice, 146(3), 319–339. https://doi.org/10.1007/s11127-010-9592-y

50.

U.S. Equal Employment Opportunity Commission. (2014). Annual report on the federal work force part I: EEO complaints processing.

51.

U.S. Equal Employment Opportunity Commission. (2017). Laws enforced by EEOC. https://www.eeoc.gov/laws/statutes/

52.

U.S. Government Accountability Office. (2009). EQUAL EMPLOYMENT OPPORTUNITY: Pilot projects could help test solutions to long-standing concerns with the EEO complaint process. https://www.gao.gov/new.items/d09712.pdf

53.

Van Slyke

D. M

. (2003). The mythology of privatization in contracting for social services. Public Administration Review, 63(3), 296–315. https://www-jstor-org.web.bisu.edu.cn/stable/977489

54.

Van Slyke

D. M.

Hammonds

C. A

. (2003). The privatization decision: Do public managers make a difference? American Review of Public Administration, 33(2), 146–163. https://doi.org/10.1177/0275074003251374

55.

Van Slyke

D. M.

Roch

C. H

. (2004). What do they know, and whom do they hold accountable? Citizens in the government-nonprofit contracting relationship. Journal of Public Administration Research and Theory, 14(2), 191–209. https://doi.org/10.1093/jopart/muh013

56.

Waldo

(1948). The administrative state: A study of the political theory of American public administration. Routledge.

57.

Wood

A. K.

Lewis

D. E.

(2017). Agency performance challenges and agency politicization. Journal of Public Administration Research and Theory, 27(4), 581–595. https://doi.org/10.1093/jopart/mux014

58.

Yang

Hsieh

J. Y.

T. S.

(2009). Contracting capacity and perceived contracting performance: Nonlinear effects and the role of time. Public Administration Review, 69(4), 681–696. https://www-jstor-org.web.bisu.edu.cn/stable/27697914