The Best Disparity,or Lack Thereof,That Money Can Buy

Abstract

This research examines the complex interactions underlying disparity studies. While communities do frequently encounter disparity in public procurement, the commissioning of a disparity study may not ultimately solve the pressing challenges that hinder access to public contracts. Instead, disparity studies promise a politically palatable “quick fix” for a societal problem. In doing so, disparity studies may satisfy a legal basis, but their content and recommendations may fail to achieve their intended rationale. This failure ultimately raises serious questions about the legal merit of these studies and their methods, the growth of a cottage industry, and the benefit of the study for underserved groups. The greatest failure is that disparity studies may ultimately exacerbate, rather than resolve, a government’s ability to reduce chronic, and often historical, challenges to inclusion.

Keywords

public procurement disparity study affirmative action public policy judicial tests

Introduction

The United States has a long tradition of supporting diverse businesses through affirmative action programs in public contracting. Inclusion in contracting opportunities to rectify past discrimination was, and often still is, a discrimination remedy that forms a compelling governmental interest (U.S. Commission on Civil Rights, 2006). For more than 50 years, debates have raged about the nature of this involvement—how to define the problem of exclusion, use the tools of government largesse to ameliorate discrimination and unfair contracting environments, and know when the problem as defined has been sufficiently addressed.

Public policy questions surrounding diversity and inclusion in government purchasing programs are complex (Blount & Hill, 2015). In City of Richmond v. J. A. Croson Company (488 U.S. 469 (1989)), the U.S. Supreme Court ruled that simply contending that past discrimination had occurred would not justify establishing a system for including minority businesses on public contracts without proof of injury or denial of property rights. The majority opinion required that specific discrimination be documented, as set forward in Justice O’Connor’s formulation, following the “strong basis in evidence” previously required in Wygant v. Jackson Bd. of Ed., 476 U.S. 267:

“Where there is a significant statistical disparity between the number of qualified minority contractors willing and able to perform a particular service and the number of such contractors actually engaged by the locality or the locality’s prime contractors, an inference of discriminatory exclusion could arise . . . In the extreme case, some form of narrowly tailored racial preference might be necessary to break down patterns of deliberate exclusion” (488 U.S. 469 (1989)).

Without requiring proof of the government’s compelling interest in resolving past discriminatory practices, the Supreme Court ruled that governments could not proactively enact race-conscious programs for general conditions; substantial proof of specific instances of past discrimination was required. Courts have subsequently taken up the question of the type, extent, and manner of proof, notably in Western States Paving v. Washington State, 407 F.3d 983 (2005), where the court found that the form of proof, a disparity study, “should rigorously determine the effects of factors other than discrimination that may account for statistical disparities between DBE¹ availability and participation. This is likely to require multivariate/regression analysis.”

The Court gave rise to the disparity (or diversity) study, and courts have refined this direction in subsequent years. The primary purpose of a disparity study is to document past discrimination in public contracting and, if it is proven that a government discriminated, the government could then establish race-conscious measures in public contracting (see, for example, Rice & Mongkuo, 1998). However, the Court in the Croson decision neither required nor recommended any specific statistical approach to prove discrimination. Instead, the legal justification for conducting disparity studies is defined by the requirements of Croson and related case law since Croson. The commissioning of a disparity study, in addition to the substantial financial cost (LaNoue, 2009), ultimately produces a report that must satisfy the strict scrutiny test. According to the Croson decision, if state and local programs establish race-conscious programs to “set aside” a portion of public contracting exclusively to minority-owned businesses, those programs must pass the “strict scrutiny” standard of judicial review.

For many stakeholders, the disparity study is exactly what is needed. However, the decision to commission a study and the assumption of the validity of the programmatic recommendations that are given in these studies deserve further examination and exploration. The purpose of this research is to explore the legitimacy and merit in current approaches to conducting disparity studies. The existence of a problem and supposed solution for the problem may force governments to respond without much thought or consideration for costs and legitimacy. However, there is potential for misunderstanding the procurement, acceptance, and implementation of the recommendations from disparity studies. This misunderstanding is often due to the legal nature of the study’s basic elements, but the problem of understanding and serving stakeholder needs, often wrapped up in the idea of a disparity study, may be little served by such a venture. These studies promise a great deal, at a significant cost, but might deliver relatively little over the long term to the very businesses that they are ostensibly commissioned to serve. Although the short-term political value of these studies is evident, the tangible value to most businesses served by an eventual program might be less obvious.

This research aims to explore disparity studies and related policy phenomena, including questions of access to government opportunity, outcomes of resultant programs, and the nature of the studies themselves. Although some literature exists on the topic of disparity studies, this material is often dated even as the disparity study business continues to respond to a present and oft-voiced need. Understanding the differences between responding to legal requirements and addressing the needs of a community with regard to contracting opportunity access and alleviation, if not remediation, of past discriminatory behavior, is essential. It is especially essential if governments at all levels are to truly open up their procurement processes for all and achieve the sort of inclusionary goals that are frequently stated as desirable, if not essential, to the maintenance and growth of small business communities, regardless of owner race and gender.

This article is organized as follows: A review of the literature on disparity studies in the context of public procurement is provided. A section on the studies themselves, focusing on an increasing commonality among the products of the industry, follows, showing attention to legal requirements and community needs for identification of problems and quick, and perhaps overly simplified, solutions. The point is made that these studies do not necessarily resolve the problem of discrimination or exclusion in the public marketplace. Instead, the result is often a crude solution that may satisfy some stakeholders, but ultimately may not address individual and institutional concerns regarding accessible and equitable contracting opportunities. It is suggested that, in aiding short-run political ends, investing in a disparity study might ultimately serve a purpose in legal requirement, but fails to encourage ongoing engagement in discussions to address injustice, unfairness, and barriers to involvement that affect citizens and businesses in government contracting and beyond.

Disparity Studies: Legal Basis, Cottage Industry and Product, and Symbol

Public procurement programs have often sought to remedy the persistent effects of past discrimination, real or perceived, by establishing initiatives intended to increase the level of contracting with minority- and women-owned businesses (Qiao et al., 2009). The thought behind such programs is that past discrimination has limited diverse businesses from fully participating in public procurement and attaining contracting success, and that positive reinforcement of diverse firms would help in increasing their productive capacity, while also assisting communities through the potential for increased economic development (Martin et al., 2007). Despite the large and increasing number of minority business enterprises, the proportion of work associated with their involvement in public contracting continues to be relatively meager (Dayanim, 2011). The use of race-conscious remedies, like goal setting in public procurement, is closely aligned with the documentation base provided by a disparity study—a report that “evaluates whether a particular jurisdiction has a documentable pattern of underutilizing minority contractors, and if so, what level of effort is justified to remedy the imbalance” (Martin et al., 2007, p. 514). Ideally, if a level playing field existed in public contracting, the availability of firms representing diverse groups across industries would equal their utilization on government contracts. If this is not the case, invidious disparity may exist.

The literature on disparity studies is weighted toward a compliance perspective. Given the landmark cases of the U.S. Supreme Court in the 1990s, governments and the legal community had to change their views on the use of race-conscious measures in governmental purchasing. Previously acceptable practices were constrained significantly by legal requirements; programs favoring minority- and women-owned business enterprises were challenged with the burden to prove the existence of disparity. This need created a substantial business opportunity for economic consultants to document outcomes and practices before making recommendations for the proper use of race considerations in procurement. Governments were desirous of approaches that would allow them to maintain race-conscious procurement programs, even if it meant reconfiguration of these programs or imposition of costly requirements for documenting the need. Stakeholders, especially those in a community that had been adversely impacted by active or passive discrimination, were anxious to have the situation remedied with the hope of more actively responding to government solicitations and winning contracts (Solis, 2001).

The disparity studies themselves are boundary objects - “plastic enough to adapt to local needs, and the constraints of the several parties employing them, yet robust enough to maintain a common identity” (Star & Griesemer, 1989, p. 393). How disparity studies have been created, their variance, and their sufficiency for documentation in favor of race-conscious contracting programs are pertinent topics not only to government agencies that institute such measures, but also to the wider contracting community in terms of determination of opportunity for public contracting dollars and value for money. These programs ostensibly seek to rectify discrimination, but the quality of programs created in response to findings is directly dependent on the quality of the study document. Inaccurate studies may lead to wasted efforts, misdirected public resources, or a combination of both, resulting in poorly constructed programs with mediocre results. Work conducted by disparity study consultants, contracted by public entities on behalf of their constituencies, is often taken for granted and accepted without much enquiry, but the output and outcomes of these efforts deserve a closer examination. Most of these studies are hundreds of pages long, resulting in the possibility that the government officials commissioning the studies may not even completely read them. Regardless, these studies serve as more than a conveying of simple empirical results; they represent a third-party response indirectly accountable to legal constraint. They are a product and cottage industry for a regular customer base, while also carrying considerable symbolic value as boundary objects to a variety of stakeholder groups and official public actors. This literature review considers disparity studies from three perspectives: as a legal basis for instituting race-conscious measures in the conduct of government procurement and acquisitions processes, as industry and product, and, finally, as symbol.

Disparity Studies in Their Legal Context

Public procurement programs supporting minority business enterprises are not new. House-Soremekun (2007) traced set-aside programs in the United States to the 1930s, with the New Deal, and subsequently through various executive orders at the federal level. Of note are programs like the 8(a) Business Development Program, which was a federal business development initiative that unified small businesses and race/ethnic minority-favoring contracting programs, originating in 1967 and given a place in statute in 1978 (Dilger, 2018). State and local programs for minority business inclusion in the 1980s increasingly included a social-disadvantage component, allowing for participation by minority groups as a matter of the basic construction of the law, on a basis that assumed disadvantage (House-Soremekun, 2007). Established overall goals were often politically determined and, in some cases, arbitrary, using quotas with only cursory documentation or statistical analysis to support the level established.

Major cases in the U.S. Supreme Court, such as City of Richmond v. Croson (488 U.S. 469 (1989)) and Adarand Constructors, Inc. v. Peña (515 U.S. 200 (1995)), reined in what were previously very loose requirements for the imposition of race-based contracting programs. These decisions echoed through all levels of government. In the Croson decision, the Court rejected Richmond’s program, which had assigned 30% of contracting opportunities to minority business enterprises. This program was found to be counter to the Equal Protection clause of the Fourteenth Amendment to the U.S. Constitution because it afforded certain groups access to opportunity solely based on race, without regard to evidence of specific instances of discrimination. General discrimination in society could not be used as a reason for enacting a program because it is inherently unmeasurable (see Regents of the University of California v. Bakke (438 U.S. 265 (1978))). The standard of strict scrutiny was imposed; this highest level of scrutiny required “the state [to] establish that it has a compelling interest that justifies and necessitates the law in question” (Garner, 2014, p. 1648). Compelling interest meant a consideration of whether the government interest in the law or policy was strong enough to balance against the potential lessening of individual rights caused by the law (Garner, 2014). Richmond’s program failed to meet this standard.

The Adarand decision further constrained race-based contracting programs in the Court’s additional requirement for narrow tailoring: “being only as broad as is reasonably necessary to promote a substantial governmental interest that would be achieved less effectively without the restriction; no broader than absolutely necessary” (Garner, 2014, p. 1183). The time remedies that are in place must be of limited length and not inflict excessive burden on other groups (Chan, 2010).

More recently, the case of Western States Paving Co., Inc. v. United States of America; U.S. Department of Transportation; Federal Highway Administration, in the Ninth Circuit Court of Appeals (03-35783, 2005), addressed implementation of race-conscious measures by a federal standard-bearer for these programs, the U.S. Department of Transportation, under the Transportation Equity Act for the 21st Century (PL 105-178), implemented under Title 49 CFR 26. In this case, while the court found that the program was valid, the State of Washington did not satisfy the requirement to document specific cases of discrimination (meeting the strict scrutiny standard), rather than relying upon a general finding of discrimination on a national basis. The appeals court found that intended goals had to be adjusted “upward or downward to reflect the proven capacity of DBEs to perform work (as measured by the volume of work allocated to DBEs in recent years) and evidence of discrimination against DBEs obtained from statistical disparity studies.” The court found the calculation of disparity that had been employed by the State of Washington to be oversimplified and required more documentation, mentioning, for example, anecdotal evidence, which can be helpful in determining “whether observed statistical disparities are due to discrimination and not to some other nondiscriminatory cause or causes” (National Academies of Sciences, Engineering, and Medicine, 2010, p. 7). In doing so, it found in favor of the plaintiff, Western States Paving, because the State of Washington did not proffer “any evidence of discrimination within its own contracting market and has thus failed to meet its burden of demonstrating that its DBE program is narrowly tailored to further Congress’s compelling remedial interest” (03-35783, VI). This case further cemented the idea that a disparity study met the standard suggested by the courts of strict scrutiny and narrow tailoring, in the imposition of race-conscious procurement programs, if that standard was not exceeded (U.S. Commission on Civil Rights, 2006). It also meant that recipients of federal grant dollars, in seeking to impose federal requirements for the DBE program, might need to document more carefully, through a disparity study, the specific discrimination and need in their own markets.

Essentially, disparity studies rely on two modes of thought on inclusion. The first mode is showing that, if not for discrimination, minority- and women-owned businesses would have performed better in the market and thus been awarded more contracts directly and through subcontracting. The second mode is amassing enough examples of discrimination, as anecdotal evidence, that the balance test is met and governmental action to remedy the problem is more valuable than any detriment to individual rights (specifically, to those not assisted by the program; LaNoue, 1995). In recent years, disparity studies have regularly included both a statistical analysis of past procurement (a utilization analysis to accompany an availability analysis of the marketplace) and anecdotal evidence of disparity; anecdotal evidence alone has not proven sufficient (Brown, 2004).

For the statistical part of a disparity study, the emphasis is on showing that

the utilization proportions differ significantly from the availability proportions for each race/gender group . . . [with the] disparity ratio . . . defined as the ratio of proportional utilization to proportional availability, with a ratio less than 1.00 indicating underutilization and a ratio less than 0.80 indicating statistical support for an inference of . . . exclusion. (Celec et al., 2000, p. 135)

The statistical assessment is likely to include multivariate/regression analysis (LaNoue, 2008; Rice & Mongkuo, 1998).

Utilization is derived from procurement data of the governmental entity, but availability analysis has presented a variety of problems, including quality or granularity of the data in the U.S. Census Bureau Economic Census, time differences and mismatches between the data for availability and utilization, differences in how minority business enterprises are defined and counted, and determining the field of qualified, able, and willing firms, among other concerns that undermine analysts’ ability to provide an analysis that conforms to the requirements of the court. There is no definitive method (Bangs et al., 2007). Because governments do not usually prequalify vendors, information of high quality and reliability on availability is absent (Celec et al., 2000). It has been suggested that “the most subjective variable in any disparity ratio and the key to its validity is how availability is measured” (LaNoue, 2011, p. 36) and, consistent with this, disparity study consultants use a variety of approaches in determining availability (LaNoue, 2011). The distinctions between business categories in the analysis, formerly known as standard industry classification codes (LaNoue, 2000) and into the present day with NIGP and NAICS codes for business classification, serve less to provide clarity and more to contribute to flawed methodology through the introduction of ineffectual distinctions. Ratios in disparity studies may be skewed, failing to account for important differences in firm capacity (U.S. Commission on Civil Rights, 2006). A failure to account for firm size and capacity in analysis has also been seen as a fatal flaw (Brown, 2004) from a legal perspective on challenging these studies.

An entity that institutes a preference program puts itself in a potentially precarious place if it lacks sufficient documentation of need. Kasarda and Luther III (2009) suggested that a governmental entity instituting a preference policy in procurement without the benefit of a disparity study documenting need and support for the policy may be liable for the constitutional tort. From an implementation perspective, meeting small business goals might be seen as more important than other aspects of the procurement process—such as a solicitation with full and open competition, yielding the best price for goods and services—because the goals are often the most public aspect of a disparity study–driven program. In an effort to solve the problem of discrimination through public procurement, it may not even be clear who is discriminating, if anyone, and whether that discrimination is active on the part of the entity itself, or if the entity is passively discriminating by simply participating in an economy that evidences discrimination. The courts have held that a general determination of discrimination in a marketplace, documented in a disparity study, is insufficient reason to institute a race-conscious program because it does not prove inequity carried out by the government (Alphran, 2003). However, in an effort to please stakeholders, government officials may find themselves pressed into resolving problems for which the governmental entity itself is not entirely responsible; as the public face of a societal problem, though, a government response that allows a path to provide for additional opportunity may seem a simple and appropriate solution.

When race-conscious programs are challenged, disparity study consultants may be asked to testify in court about the study, consistent with Rule 702: Testimony by Expert Witnesses of the Federal Rules of Evidence.² The expert provides information based upon facts and data, and they may draw an inference based upon their analysis. There is some allowance even for expertise differences between experts in the natural and social sciences; this is important because the disparity study consultant as expert may be challenged by other experts, including those in only marginally related fields (Herling et al., 2009). It is important to note that the disparity study consultant, as a third party to the community, is separate and apart from either the stakeholders or the government entity. The consultant is commissioned to write the study, and possibly speak about it in court, but is not directly accountable for the findings, the recommendations, or the success or failure of the resultant program, except through the contractual relationship. This observation is relevant because an outcome of the study is a product that includes public policy recommendations, which may be adopted wholesale and made law. Attention is now paid to the study as both a cottage industry and a product to purchase.

The Disparity Study as Cottage Industry and Product

The legal requirements for documentation of discrimination in public contracting created an industry. A number of consulting groups have amassed considerable credibility and fortune through the selling of the disparity study as a product; although the market is small, competition is fierce. In addition to providing the study document itself, these consultants have variously offered to assist in the defense of the programs upon which their studies rely. Given the interest that communities, counties, states, and even the federal government may have in maintaining race-conscious measures within procurement programs, the disparity study has become a stock product to satisfy court requirements, fend off potential challenges, achieve needed societal goals whether real or perceived, and satisfy constituents and interest groups. There were plenty of potential customers: In 1995, 234 governmental entities had race-conscious programs (LaNoue, 1995). All of them implicitly required documentary defense to continue operating these programs, per the Croson decision.

The disparity study business has seen a limited group of competitive consulting groups. A part of the reason is the emphasis on experience: government entities want a consulting firm with experience in this type of study and perceive a benefit to hiring a firm that has issued disparity studies, particularly where the resultant program has withstood challenges, or still stands. Communities may hire a specific study consultant when a study is completed in a nearby area (Gallo, 2018). These industry traits limit the potential for new consultants to be competitive, leading to an insular community of usual bidders, and commonality among the products, driven by both legal factors and “best practice.” The study, as a product and the brand-name status of established disparity study contractors, is apparent in other ways: Media coverage of disparity study news often includes details of which of the well-known national consultants in the market conducted any particular study, and references to the content of the study are perfunctory. Instead, the study in popular media is an object, very nearly a consumer product, purchased from a well-known consultant, which allows for a desired program to be instituted. The study is a means to an end. The reputation of these studies precedes the work; most studies find the discrimination that led to the commission (Feagin, 2001), so an entity is essentially buying a finding of discrimination that will allow for a program of some sort, as well as a study. A finding of no discrimination at all is possible but relatively rare.

The price of a disparity study is typically high in the hundreds of thousands of dollars, even if it is driven by the size of the governmental entity and its procurement operation. The price is high enough, in public resources and dollars, such that some entities that might otherwise engage in commissioning a study find the institution of a race-neutral, small business program, not requiring a disparity study, to be a preferable alternative. The opportunity cost of a disparity study is not easily overcome.

It is generally understood that governments seeking to commission a disparity study can save money if the data available on contracting are “clean,” consistently collected and recorded, and readily available to the consultant to conduct the analysis (National Academies of Sciences, Engineering, and Medicine, 2010). However, even with the cleanest data, many government entities, with procurement data in various states of disrepair or in multiple databases, are simply not ready to conduct a disparity study. Failings in the data available can cause the cost of a study to increase dramatically through remedial work to bring the entity’s data into alignment with the model of the analysis conducted by the study consultant. A study alone does not necessarily mean that the resulting program will make sense or achieve intended results; there are instances where a study has been conducted and a program instituted, but inclusive participation has decreased, or where the problems identified remain even after institution of a program solution (Martin et al., 2007). There is reason to question the connection between the commissioning of a disparity study, the enactment and implementation of a program to overcome the identified discrimination, and whether the programs ever work to the level necessary to alleviate disparity as identified. This disconnection can become a serious statistical challenge, with the study as a snapshot, and the community economy and government contracts continuing to evolve after the study’s completion. Programs keyed to previous disparity may realistically have little chance of keeping pace with these market changes and new challenges to inclusion.

Symbol in the Study of Disparity

In addition to studying the disparity study from a legal perspective and as a cottage industry and product, disparity studies may also have a striking symbolic aspect, being a boundary object that means different things to different stakeholders. The existence of a diversity program may, on its own, satisfy political need and have great symbolic value even if the program makes little difference to firm growth, capacity increase, or inclusion. Commissioning and receiving a disparity study does not resolve any of the knotty complexities that prevent program success, where administration, time, and resources are what are needed. A tendency may be for officials to receive a study, institute a program, and in effect declare victory for the action, when the real work of engaging businesses and increasing capacity had only just begun. Businesses that are being excluded still may not know what they are doing wrong and why they are failing to receive contracts (Ballou, 2018). These businesses may also choose not to bid for public contracts, even if they are aware of the contract opportunities, because they may not have the necessary information, or feel that the costs for preparing a response are not justified (Bangs et al., 2007). Imposition of goals does little to bridge the gulf between prime contractors and DBE subcontractors who are often unfairly marginalized despite their considerable expertise and ability (Orndoff et al., 2011).

This process and remedy are perhaps not what is primarily desired by the government. Government procurement has a primary motivation to provide the best product or service for the best cost. Other qualitative factors, and the inclusion of socioeconomic programs, may be important but are less geared to traditional procurement systems. While the recommendation exists for procurement professionals to become more aware of how to work successfully within such systems, this recommendation does not mean that the goal is accomplished (Qiao et al., 2009).

Minority- and women-owned business programs exist in a difficult operating environment, questioned within and without by large contractors, contracting organizations, and clients internal to government that see the programs as causing delay and increased cost. Vendors see government processes as lowering prices for products and services to abnormally low levels, thereby reducing the potential for small businesses to work contracts at a reasonable level of return (MacManus, 1991). The programs are sometimes seen as being counter to tenets of competition, with supplies and service provided being of inferior quality, and subject to abuse with firms not meeting certification requirements (National Association of State Procurement Officials, 2019). What is more, the potential value of these programs is undermined by political realities. For example, administrators responsible for the implementation of the U.S. Department of Transportation’s DBE program are aware that program violations occur, but a lack of enforcement prevents the program from being taken seriously by contractors and powerful contracting interest groups. Enforcement against well-connected interests is obviously unpopular. With that noted, the lack of sanctions against firms that transgress program rules is a serious matter (LaNoue, 2008). Even with a disparity study and the setting of goals, weak implementation may encourage anti-program behavior. The official acts of establishing proportional goals for business inclusion and evaluating responses to solicitations for program adherence have a symbolic value in that they appear as dynamic activity leading to desired outcomes. It may be possible for organizations to focus on the goals established, rather than participation achieved, resulting in programs that do not achieve goals but are nevertheless held in high regard by client/targeted businesses. The tendency to enforce program requirements may be informed by political and business inclinations, including those of well-connected business owners acting in an advisory capacity that may stand to gain from program institution and, organizationally, punishment may not follow a failure to implement minority business enterprise program requirements (Terman, 2017). This hesitancy speaks volumes about the potential for differences between the intent in the program as written and the program in practice.

For their part, disparity studies have seen isomorphism that reduces the potential of the document to serve as analysis of a particular community’s effort to understand and address discrimination in public procurement. While the studies are not necessarily “paint-by-number” exercises, there is some tendency on the part of study consultants to perfect and utilize a well-worn formula that has proved successful in the past and use the “boilerplate” in a variety of communities. The courts have contributed to this, with consistent expectations of general types of content, even if the statistical analysis of studies is beyond many directly involved in program operation. Some study tropes have become commonplace, such as the existence of a “good old boy” network as an anecdotal point (Alphran, 2003) or a need for contract unbundling as a recommendation (Rouan, 2018). When study recommendations look mostly the same among study providers, although the challenges facing communities can be quite different, there is perhaps reason to question the isomorphism and imposition of supposed best practice, when attention to community history and context may better help resolve problems.

Isomorphism is a problem because resolution of a societal problem requires attention to the very specific and different needs of citizens and communities. Despite the potential that some best practices may exist, default to such approaches echoes what is possibly a limited repertoire of government policies in responding to rights issues (Skrentny, 2002). Furthermore, the recommendations themselves do not necessarily represent a new paradigm, so much as an appeasement; there is little interest in angering business interests from large firms, for example, with considerable political influence of their own. Even where the pronouncement of a disparity study and its program change is emphasized, the benefit that may extend to firms not already actively involved in government procurement may be limited. It does not help that it can be quite difficult to discern the specific benefits that have resulted due to program intervention, rather than other variables, such as the overall health of the market (Skrentny, 2002). It is also worth noting that the minority and women-owned business groups reviewed in the course of disparity studies themselves have different barriers to inclusion and blanket recommendations do little to attend to these distinctions. The reliance on precedent possibly prevents meaningful change from occurring; governments enact programs safe from litigation as a result of studies and, having done that, may fail to engage serious problems of access and opportunity beyond the study and establishing goals for inclusion.

Disparity studies stand as boundary objects between competing interests and a variety of stakeholder groups; each interest or entity see in the study something that is germane to its prevailing narrative—either something to fight for or against. The study, ostensibly an objective exploration of procurement outcomes and documentation of fairness or discrimination, is nevertheless inescapably subjective, carrying with it the expectations of its commissioning parties. The study commissioning process and the eventual document carry with them high hopes for programs that arise out of a disparity study. In Boston, for example, Hussein Dayib, an organizational diversity director, suggested, in Klocek et al. (2014), as follows:

We have to change the way we do business for the benefit of our community. The mantra for supplier diversity has changed over the past decade from “it is the right thing to do” to “it makes good business sense.” (p. 35)

In the same article, the move toward diversity is described as a “journey” and a “natural step” (Klocek, in Klocek et al., 2014, p. 36). The disparity study is presented as the key to move forward in developing minority-inclusion goals and instituting policy and administration necessary to attain those goals. As alluded to previously in this article, there is an expectation, potentially injurious to the validity of the analysis because of bias, that one looking for discrimination and disparity will assuredly find it, especially if this thinking is implied in the study purchase.

Rhetoric surrounding disparity studies is often impassioned, reformist, and hopeful; for example, the word “prayerful” was used in the hope that a disparity study would soon be forthcoming in New Jersey, so that goals could be quickly instituted (Schaffler, 2018). Mathews and Rice (1999) referred to San Francisco’s use of a disparity study as a shield. Elsewhere, the disparity study has been described as a sort of “insurance policy,” so that a program designed according to the study’s recommendations will not be disputed or at least not successfully challenged (Alphran, 2003). Politicians have run for office on the need to commission a disparity study (e.g., Bernard, 2018; Oakland Post, 2018). The commissioning of a disparity study is an official action and event that shows intent to include, meaning that it has great rhetorical value to not only stakeholders but also to the government in mollifying concerned parties. The passage of a business opportunity program when the disparity study is received is seen as historic by stakeholders. With unanimous approval, programs are described as landmarks and “proud moments,” where “the message is, it’s not going to be business as usual anymore . . . we . . . [take] equity seriously” (Vercher, in Garrison, 2019), that it is a “new day” for working with diverse business owners (Kebede, 2018), or that a study will create a “new climate” and a “new chapter” (Talamo, 2017) for communities.

Symbol also intrudes into the programs created under the auspices of a disparity study. Meeting goals in a race-conscious program can become the primary order of business to address findings of discrimination in the disparity study. However, from an inclusion standpoint, the counting of a few minority firms receiving the bulk of available contracting opportunities would count the same as many minority firms receiving a share of contracts. The availability of real opportunity, where a firm interested in government work can approach the procurement process and be successful in it, may have nothing to do with it, from a metric standpoint, when issuing contracts to a group of already successful minority businesses may be sufficient to meet the goals. Interest in change at the neighborhood level is frustrated by data and analysis on businesses that do not extend past city or metropolitan area consideration. Setting goals does not address barriers to inclusion, like a lack of capital or bonding, inconsistencies in access to information and business networks that act as supportive structures, and spatial/location related impediments (Dayanim, 2011). Firms may become certified as eligible in new inclusion programs, only to receive no work; the commissioning of a disparity study may be a regular occurrence, even when the minority businesses themselves see no appreciable change in the contracting climate or in their ability to gain public contracts (Davis, 2018). Necessary outreach and supportive programs are given less attention.

Prime contractors and consultants, upon whom the task of creating teams of diverse businesses lies, may resist efforts to include new businesses because there are immediate time and resource costs associated with the change even if it ends up being fruitful. There is a desire to continue with existing subcontracting relationships. Furthermore, minority businesses that are already successful may strenuously support commissioning of disparity studies and race-conscious programs, for the benefits of smaller market and less competition that these programs bring, but be opposed to establishment of eligibility standards that portend additional competition, even if the primary intent of the program is to increase opportunity and grow capacity, or to support economic development generally. Meeting the goal might be important, but the impact on actual businesses could be more important, especially when the metrics may not tell a complete story. Even when goals are exceeded, problems may still exist, which result in something other than a “level playing field” for minority contractors (Nixon, 2019).

A major component of programs that incorporate race-based preferences is a “race-neutral” component, typically favoring support of small businesses generally. Race-neutral programs may involve provisions to reduce vendors’ barriers to contacting, providing for loan programs, bonding and insurance, timely payments, and addressing the impact of advertisement timing on competitive responses. Programs can be created to target specific areas of a community that have been historically underserved, so that businesses located there receive some preference in public procurement. Programs can be instituted to guide some subcontracting business to targeted vendors, and recordkeeping requirements may be established to ensure more widely dispersed opportunity and transparency. When targeted vendors are rejected from opportunities, for example, there may be a requirement to provide a rationale in writing. Such program features can assist targeted vendors by reducing their barriers to involvement, without the need for invoking race or gender and additional program scrutiny. However, like general isomorphism in studies, programs, and recommendations, race-neutral measures, even if effective in some way, may be employed when targeted measures are what is needed to address specific types of inequality and discrimination, and the result can be that the problem is prolonged, changed, or worsened (Myers & Ha, 2018).

Small business is a positive construct imbued with deep, often emotion-laden, meaning. From the perspective of a public program construct, however, supporting small businesses may not be all that it seems. Programs define small businesses in a variety of ways, but following the approach taken by the U.S. Small Business Administration, firms are determined to be “small” based upon the gross receipts they take in, or the number of people employed, based differentiated by major industry. In practice, most businesses are small, meeting the generous size standards, and are therefore eligible for small business programs. This may appear as if the program is providing great opportunity, but the number of projects available for bidding and award is not necessarily larger. Very small firms may not be competitive against the responses of larger firms that are nevertheless counted as small under these programs. The result is that the most competitive firms, with both capacities to do the work and awareness of how government solicitations work, are the firms that often receive the awards. In this respect, the race-neutral component may not balance the race-conscious aspect but instead prove a distraction, operating for most eligible firms purely in the realm of the symbolic.

Disparity Studies: Change and Challenges

As suggested earlier in this research, the cottage industry of disparity study preparation grew out of a need created by the U.S. Supreme Court in the form of the City of Richmond v. J. A. Croson Company decision, 488 U.S. 469 (1989). The Court found fault with a “generalized assertion that there has been past discrimination in an entire industry,” and with no specific ground for governmental action to redress discrimination to a certain well-defined extent. The resultant system, with its quota goal, did not “in any realistic sense” tie “to any injury suffered by anyone.” What was needed as a result was documentation and statistical “proof” of harm: “a significant statistical disparity between the number of qualified minority contractors willing and able to perform a particular service and the number of such contractors actually hired, might provide an inference of discriminatory exclusion” (Kasarda, 2015).

Nothing present in the Richmond program provided a “strong basis in evidence for its conclusion that remedial action was necessary,” as required previously (Wygant v. Jackson Board of Education, 476 U.S. 267, 106 S.Ct. 1842, 90 L.Ed.2d 260 (1986)). The discrimination from that point had to be specifically identified; justification for the intervention had to be provided. Essentially, the government must offer a plan showing where it had failed and where it hoped to address the problem and provide solutions; only then would it be known when success was achieved. Within a few years of the Croson decision, the first generation of disparity studies began to arrive, to provide the level of factual predicate and proof necessary to sustain a government’s decision to intervene and enact affirmative programs on behalf of excluded groups.

LaNoue’s review (1994) of early disparity studies indicated a general grasping at what the Court had intended. Utilization analyses were present but subject to distortions in the counting of firms and the sources of information. Anecdotal evidence was included but not in ways that could be considered systematic or objective. In the years since these first studies, the art of this industry has become much more involved, complex, and voluminous. We propose that changes made to study protocols and expectations over the years are worth review in themselves and offer a typology for discussion along with the invitation for additional research and refinement of this approach. Table 1 highlights some of the differences between the generations of disparity studies; an entity that has commissioned and received three studies is said to be on its “third-generation” study, but these studies are further reflective of the consulting practice of the era.

Table 1.

General Comparison of Disparity Studies.

Generation	Period; description
First	1990 to 1999; studies attempted to respond to Croson requirements, including discussion of anecdotal evidence. Very few included regression analyses. A study was valid for a period of 10 years.
Second	2000 to 2007; Regression analysis included. Evidentiary requirements are strict. Expectation became that a study was valid for a maximum of 5 years.
Third+	2008 to present; Current studies are expected to include more information and analysis. As an example, one discussion of a fourth-generation study suggests that it “goes beyond comparing the availability and utilization of Small, Minority, Women, and Disadvantaged Business Enterprises (S/M/W/DBE). It will examine whether there are statistically significant disparities in contract award amounts, bid success rates, prequalification rates, and whether there are specific market barriers to full participation in the entire range of procurement and contracting activities,” and “detail the performance of race-neutral programs and examine whether the policies and practices of race-conscious programs result in an undue burden on firms that are not Disadvantaged Business Enterprises.” (Roy Wilkins Center, 2019)

For purposes of improving general understanding about these study documents, Table 2 reflects current general sections of an archetypal disparity study (National Academies of Sciences, Engineering, and Medicine, 2010). Some sections, such as the legal review and analysis, are commonplace and can be quite lengthy, but because the law is the law, references to Croson and Adarand will appear in practically every study. Review of procurement law and policy varies by locale. Although the requirement that a market area be established for study analysis is generally accepted, even in 2012 there was considerable disagreement within one consultant’s study about how to approach this question (MGT of America, 2012a). The utilization, availability, and disparity analyses, and presence of discussion of anecdotal evidence are perhaps enhanced in some respect, but all have their lineage in the original direction given in the Croson decision. The regression analysis is likely the source of some variation, not only in method but also in means and materials as coding of firms as minority or not, certified or not, and the like remains a concern for study consultants even to the present day. Recommendations round out the document.

Table 2.

Major Sections of Disparity Studies and Likely Subsections.

Major sections	Subsections
Legal review/analysis	Standard of review; burden of proof; Croson-level requirements (including that the government entity must have been a participant—active or passive—in discrimination; that evidence of discrimination is systemic; and instances of behaviors with negative impacts on minority vendors).
Review of procurement law/policy	Varies. May include sections for federal, state, and local laws. Even if it is a local program, state requirements may supersede, such as for professional services contracts.
Establishment of market area	Methodology, analysis of prime contracting and subcontracting, and other spending (MGT of America, 2012a and 2012b).
Utilization analysis	Prime contractor analysis; subcontractor analysis (by industry/NAICS code).
Availability analysis	Prime contractor/subcontractor availability; capacity to perform; ready/willing/able
Disparity analysis	Overutilization/ underutilization; prime contracting/subcontracting by industry
Anecdotal analysis	Business owner responses; Review of judicial records
Regression analysis	Regression analyses of business ownership, earnings, and loan approvals to find out whether discrimination in a market area impacts ability for firms to access contracting opportunities
Recommendations	Recommendations might include administrative roles/responsibilities and programmatic enhancements, as well as suggestions for legal changes to code/ordinance/program language (Mason Tillman Associates, 2015; NERA Economic Consulting, 2015).

As a product, the disparity study is an increasingly complex document but written according to a certain formula based on supposedly successful documents before it. As questions arise in court cases, study authors have modified their approach to address those questions and plan for future needs of judicial review. Some of the big names in the disparity study business (e.g., MGT of America, NERA, and Mason-Tillman) were present for the first-generation studies and remain in the field. It is an insular field, despite the fact that the requirements for disparity studies are more or less well-known at this point, because the drafting of a disparity study format and engaging in the calculations without previous experience could be daunting and time-consuming to a would-be competitor.

Government entities within the United States, especially larger local governments, are aware of one another’s experiences and developments regarding shared demands of populations and changes to and challenges for policy to serve diverse groups. The institution of minority- and women-owned business programs and the call to be responsive to needs and to document discrimination therefore speaks to a certain policy isomorphism. The brand-making of study consultants is notable; their ability to navigate what would otherwise be unsettled waters with not only a product (the study itself) but also a series of legislative/policy recommendations that have been shown in other contexts to be acceptable to diverse communities provide major selling points to concerned legislative bodies and administrators.

As suggested above, the isomorphism is notable in the disparity study consultant community and ticks off many of the markers noted by DiMaggio and Powell (1983) for such phenomena. Communities are aware of successful minority and women business enterprise (MWBE) programs and highlight these programs as exemplars prior to engaging in their own programs; programs regularly participate in benchmarking, and this activity is often expected as part of the budgeting process. Communities want successful and defensible programs because the existence of these programs is highly symbolic of an open, diverse business community, even if the availability or openness of opportunity is less evident in practice. There is a tendency toward mimicry when one community has a study, and the expectation rises that nearby communities will soon engage in studies as well. Because communities often have a high level of uncertainty, made worse by a litigious environment both for and against MWBE programs, disparity studies are a means to this end and serve as an avenue for achieving program goals with the hope of silencing critics. Study consultants are dependent on one source of approval for their work (the courts). There are few models for studies; there is little innovation and most studies include a great majority of the material in Table 2 because it is expected. What is worse, discrimination where it exists is perhaps not appropriately or fully addressed because studies only point to evidence of discrimination—they do not root it out. A study does not engage or begin to eliminate bias inherent in institutions, and the presence of a study may seem to some as a shield from accusations that bias or discrimination persists (U.S. Commission on Civil Rights, 2006).

A sensible question may be how much the recommendations follow from the analyses in the document, or if they are largely consistent from study to study, within consultant’s output and across the outputs of consultants as a group; this may provide further evidence of isomorphism within this industry.

Because the goal of commissioning a disparity study is arguably not a search to identify parity, but rather to find disparity (and so, looking for it, it will be found), a research bias exists. When the cost of a study is often hundreds of thousands of dollars, the countering view—that a disparity study is a pure social science maneuver—is laughable. The study is commissioned precisely because a finding of disparity is presupposed and a program will result. Otherwise, there would be little reason to commission a study in the first place.

Conclusion

Previous research has not examined disparity studies as a group of work products to gain greater understanding about their contents and statistical methods and results. Disparity studies are imposing documents; in a political and social context where attention, time, and financial resources are sometimes scarce, a disparity study may be accepted as valid on its face without much challenge or question. The placement of disparity study consultants into an environment where they not only produce these documents but also provide a suite of services in support of eventually enacted policy may give pause to the involvement of unelected disparity practitioners in the offering of solutions, which may be enacted without much thought for other alternatives and may ultimately fail to achieve stated goals. Because disparity studies themselves are costly and time-consuming endeavors, it is increasingly essential that the public not only gets best value for its money, but also understands, given the multifarious pressures on the decision to engage in the process, what it is that is being purchased.

This research is important to not only a greater understanding of pressures upon the policy process, including the contracting environment that has resulted from a Supreme Court case decided vaguely and left largely unchanged (with the exception of the Adarand decision and Western States case) since 1989, but also to governments and their procurement operations. Governments may seek to provide for inclusion, and quite rightly so, but engage in a choice that essentially allows for a quick elimination of the problem from a political perspective. Commissioning a study appears as responsive, and the receipt of a disparity study and acceptance of a range of policy options may be perceived as progress, but the problem of resolving discrimination and providing for inclusion is much knottier than the answer provided for by this simple and expensive answer.

While the discussion of inclusion and resolving discrimination is beyond the scope of this article, it would be worth knowing more about the disparity study as a causative to adopting goals and to more fully understand the relationship between disparity studies and goal utilization. Is there a potential that race-conscious programs actually harm the interests of eligible vendors? As not every entity engages in disparity studies, it might be worthwhile to do a macro-study of first-time study commissions to identify the prevalence of disparity in places where a study commission was not a priority. Given the attention to best practices in study preparation, do the guidelines make a difference for study quality? Absent race-conscious goals, do race-neutral programs and measures work equitably?

Inherent in discussions of disparity in government procurement is the idea of racism, on an institutional and even individual basis, at the level of official actors engaged in aspects of acquisitions and contract awards. There can be a sense of differentiation and inequality among certain firms based on race or gender classifications or at least the perception that this is the case. The pattern and practice of a procurement system that results in outcomes that favor majority-owned vendors may raise allegations of discrimination or exclusion. As to why individual actions in given instances are at issue, institutional discrimination results from “established institutional policies, routines, norms, and functions, as well as those individuals with power and influence who control institutions . . . called gatekeepers” (Kamali, 2009, pp. 5–6). Even if the discrimination is not intentional, the results and outcomes are often well-entrenched in the activity of an organization, and painful to those experiencing exclusion, particularly in the public context, where expectations are that processes and results are fair, or at least seek toward fairness.

Through preferential affirmative action, it is thought that resultant policy and programs will “involve the establishment of goals or other strategies favoring the selection of qualified minorities or women for positions which they are significantly underrepresented based on a reasonable estimation of what equitable representation would look like” (Kellough, 2006, p. 146). This approach considers past harm and the intent to set right various institutional, if not societal, wrongs. In procurement, marginalized vendors would be brought back in, through such a system, through goals that calculate what a fair share to each group would constitute and how to achieve it, given the products and services purchased by the entity. In practice, because of the complexity of acquisitions and the nature of competition, equally placed vendors on the basis of knowledge, skills, and abilities may have quite different outcomes through the procurement process, given the size of business, capacity, and ability to submit low bids, which often win on that basis alone. Quality of bid and social priorities must balance against price, especially when the enacted regulatory system strongly encourages compliance with some factors, like minority- and women-owned business goals, but demands compliance with pricing factors.

Rather than dealing with this complexity, disparity studies offer, for a price, the ability to identify a culprit in institutional discrimination and a way forward to fix the problem through well-worn recommendations. In doing so, legal requirements are met, along with offers to help defend the entity against legal trouble for instituting the recommendations. More importantly however, the entity appears responsive in purchasing this product. As long as the entity agrees to the recommendations and takes its rhetorical punishment in the court of public opinion, it can begin the hard road back to inclusion and legitimacy. It is worth asking, “Are disparity problems then resolved? If so, why are entities commissioning third- and fourth-generation disparity studies, which find discrimination?” “Is the goal really to solve the problem? Or is this theater—where the dynamics of responding to the problem and the announcement of having arrived at a solution are indeed ‘good enough?’”

In a public sector environment characterized by increasingly scarce resources, entities serving the public interest cannot afford to waste money on a charade of accountability, when necessary institutional improvements subsequently do not occur. From a legal basis, additional attention should be paid to these studies as products; the lack of literature in this area and the reliance of institutions and judicial processes on these expensive research projects highlight a failure in the academic literature to evaluate and enlighten. Even given the vague requirements of the Supreme Court on the topic of disparity, and their requirements for sufficiency in documentation, disparity studies should be rooted in and for the communities of study. The tendency toward isomorphism in recommendations suggests a reaffirmation of what has worked before, rather than a serious effort to respond to the challenges of vendors, institutions, and whole communities. In the law context, as cottage industry and product, and from a symbolic standpoint, surely more can be done beyond disparity studies to remedy discrimination, assuming that discrimination indeed exists.

Footnotes

Acknowledgements

The authors are grateful for the comments of anonymous peer reviewers, which have been incorporated into our revision of this article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Christopher L. Atkinson

Notes

References

Alphran

D. M.

(2003). Proving discrimination after Croson and Adarand: “If it walks like a duck.” University of San Francisco Law Review, 37(4), 887–969.

Ballou

(2018, February 8). Minority firms seldom hired by Miramar, study shows. Sun-Sentinel. https://www.sun-sentinel.com/local/broward/miramar/fl-sb-miramar-disparity-study-20180207-story.html

Bangs

R. L.

Murrell

Constant-Huggins

(2007). Minority business bidding for local government contracts: The complexity of availability. Social Work in Public Health, 23(2/3), 247–262.

Bernard

(2018, October 16). Point of view: Disparity study clearly shows unfairness in county government contracting. Palm Beach Post. https://www.palmbeachpost.com/opinion/20181015/point-of-view-disparity-study-clearly-shows-unfairness-in-county-government-contracting

Blount

I. Y.

Hill

J. A.

(2015). Supplier diversification by executive order: Examining the effect reporting compliance, education and training, outreach, and proximity to leadership have on government procurement behavior with minority business enterprises. Journal of Purchasing & Supply Management, 21, 251–258.

Brown

T. L.

(2004). Deceived by disparity studies: Why the tenth circuit failed to apply Croson’s strict scrutiny standard in concrete works of Colorado. Denver University Law Review, 81(3), 573–595.

Celec

S. E.

Voich

Jr. Nosari

E. J.

Stith

M. T.

Sr. (2000). Measuring disparity in government procurement: Problems with using Census data in estimating availability. Public Administration Review, 60(2), 134–142.

Chan

(2010). Asian American struggles for civil, political, economic, and social rights. In Wu

J. Y. S.

Chen

(Eds.), Asian American Studies Now: A critical reader (pp. 213–238). Rutgers.

Davis

(2018, December 11). Minority businesses want action as city considers new disparity study. KXAN News. https://www.kxan.com/news/local/austin/minority-businesses-want-action-as-city-considers-new-disparity-study/amp/

10.

Dayanim

S. L.

(2011). Do minority-owned businesses face a spatial barrier? Measuring neighborhood-level economic activity differences in Philadelphia. Growth and Change, 42(3), 397–419.

11.

Dilger

R. J.

(2018). SBA’s “8(a) program”: Overview, history, and current issues. Congressional Research Service. https://fas.org/sgp/crs/misc/R44844.pdf

12.

DiMaggio

P. J.

Powell

W. W.

(1983). The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 48(2), 147–160.

13.

Feagin

J. R.

(2001). Racist America: Roots, current realities, and future reparations. Routledge.

14.

Gallo

(2018, July 16). Baton Rouge to use same firm as New Orleans city hall for disparity study. The Advocate. https://www.theadvocate.com/baton_rouge/news/article_16091b3e-8932-11e8-8dcc-6fba7fa2fd12.html

15.

B. A.

Garner

(Ed.). (2014). Black’s law dictionary (10th ed.). Thomson Reuters.

16.

Garrison

(2019, January 4). Nashville council passes “historic” equal business opportunity ordinance for contractors. The Tennessean. https://www.tennessean.com/story/news/2019/01/04/nashville-council-equal-business-opportunity-ordinance-minority-contractors/2485743002/

17.

Herling

D. J.

Williams

G. M.

Wise

J. T.

(2009, Fall). In Daniel J. Herling,. Gene M. Williams & Jeffery T. Wise (Eds.), Non-Traditional Uses of Daubert: A Review of Recent Case Law, 60(1), 69.

18.

House-Soremekun

(2007). The impact of minority set-aside programs on Black business success in Cleveland, Ohio: Implications for public policy. Western New England Law Review, 30(1), 19–38.

19.

Kamali

(2009). Racial discrimination: Institutional patterns and politics. Routledge.

20.

Kasarda

(2015, December 1). Disparity studies get no respect. Pacific Legal Foundation. https://pacificlegal.org/disparity-studies-get-no-respect/

21.

Kasarda

R. W.

Luther

III . (2009). Why courts must subject municipalities to constitutional tort liability. Civil Rights Law Journal, 19(2), 371–404.

22.

Kebede

L. F.

(2018, June 29). Memphis school leaders pledge “a new day” in contracting with diverse business owners. Chalkbeat. https://chalkbeat.org/posts/tn/2018/06/29/memphis-school-leaders-pledge-a-new-day-in-contracting-with-diverse-business-owners/

23.

Kellough

J. E.

(2006). Understanding affirmative action: Politics, discrimination, and the search for justice. Georgetown University Press.

24.

Klocek

Dayib

Mukherjee

Crespo

A. E.

(2014). Supplier diversity: From “checking the box” to developing a trusted and valued partner. Journal of the American Water Works Association, 106(9), 30–36.

25.

LaNoue

G. R.

(1994). Standards for the second generation of Croson-inspired disparity studies. The Urban Lawyer, 26(3), 485–540.

26.

LaNoue

G. R.

(1995). Selective perception: The role of history in the disparity study industry. The Public Historian, 17(2), 13–20.

27.

LaNoue

G. R.

(2000). To the “disadvantaged” go the spoils? The Public Interest, 138, 91–98.

28.

LaNoue

G. R.

(2008). Follow the money: Who benefits from the Federal Aviation Administration’s DBE Program? American Review of Public Administration, 38(4), 480–500.

29.

LaNoue

G. R.

(2009). Policies to ensure group equality in public contracting in four countries. International Journal of Diversity in Organisations, Communities & Nations, 8(6), 81–95.

30.

LaNoue

G. R.

(2011). Western states light: Restructuring the federal transportation disadvantaged business enterprise program. Civil Rights Law Journal, 22(1), 1–60.

31.

MacManus

S. A.

(1991). Why businesses are reluctant to sell to governments. Public Administration Review, 51(4), 328–344.

32.

Martin

Berner

Bluestein

(2007). Documenting disparity in minority contracting: Legal requirements and recommendations for policy makers. Public Administration Review, 67(3), 511–520.

33.

Mason Tillman Associates. (2015). Broward County Public Schools disparity study final report. https://web.archive.org/web/20180216135256/http://www.broward.k12.fl.us/supply/sdop/docs/disparity/SBBC%20Disparity%20Study%20Final%20Report%2010-4-15%20(Posted%2003.2016).pdf

34.

Mathews

A. L.

Rice

M. F.

(1999). Adarand v. Peña: Turning challenges into opportunities. International Journal of Public Administration, 22(7), 1017–1058.

35.

MGT of America. (2012a). Comprehensive disparity study for the city of Pensacola: Final report. https://web.archive.org/web/20190318023123/https://www.cityofpensacola.com/DocumentCenter/View/131/Comprehensive-Disparity-Study-PDF

36.

MGT of America. (2012b). Disparity study for the minority/women business enterprise program (City of Greensboro, NC). https://web.archive.org/web/20190318022619/https://www.greensboro-nc.gov/home/showdocument?id=15419

37.

Myers

S. J.

Jr. Ha

(2018). Race neutrality: Rationalizing remedies to racial inequality. Lexington Books.

38.

National Academies of Sciences, Engineering, and Medicine. (2010). Guidelines for conducting a disparity and availability study for the federal DBE program. National Academies Press. https://doi.org/10.17226/14346

39.

National Association of State Procurement Officials. (2019). State and local government procurement: A practical guide (3rd ed). J. Ross.

40.

NERA Economic Consulting. (2015). Business disparities in the San Antonio, Texas market area. https://web.archive.org/web/20190318022407/https://www.sanantonio.gov/Portals/0/Files/EDD/NERA_COSA_Disparity_FINAL_151112.pdf

41.

Nixon

(2019, January 1). Economic disparity: Is it hurting New Orleans business? Big Easy Magazine. https://www.bigeasymagazine.com/2019/01/01/nola-business-and-economic-disparity/

42.

Oakland Post. (2018, December 27). Support Rebecca Kaplan for city council president. Oakland Post. https://www.postnewsgroup.com/support-rebecca-kaplan-city-council-president/

43.

Orndoff

Papkov

Behney

Lubart

(2011). Defining the problem: Disadvantaged business enterprise program impediments identified by survey of program administrators. Public Works Management & Policy, 16(2), 132–156.

44.

Qiao

Thai

K. V.

Cummings

(2009). State and local procurement preferences: A survey. Journal of Public Procurement, 9(3/4), 371–410.

45.

Rice

M. F.

Mongkuo

(1998). Did Adarand kill minority set-asides? Public Administration Review, 58(1), 82–86.

46.

Rouan

(2018, December 17). Columbus: Minority-, women-owned businesses got more city contracts last year. The Columbus Dispatch. https://www.dispatch.com/news/20181217/columbus-minority—women-owned-businesses-got-more-city-contracts-last-year

47.

Roy Wilkins Center. (2019). Disparity studies. https://www.hhh.umn.edu/technical-assistance/disparity-studies

48.

Schaffler

(2018, December 27). NJ African-American chamber of commerce helps minority-owned businesses compete. NJTV News. https://www.njtvonline.org/news/video/nj-african-american-chamber-of-commerce-helps-level-the-playing-field/

49.

Skrentny

J. D.

(2002). The minority rights revolution. Belknap Press of Harvard University Press.

50.

Solis

(2001, August 19). Minority-owned businesses struggle to win public contracts in Dallas. The Dallas Morning News. Business Insights: Essentials, ITEM01231004.

51.

Star

S. L.

Griesemer

(1989). Institutional ecology, “translations,” and coherence: Amateurs and professionals in Berkeley’s museum of vertebrate zoology, 1907-1939. Social Studies of Science, 19(3), 387–420.

52.

Talamo

(2017, February 4). What disparity studies have shown. Shreveport Times. https://www.shreveporttimes.com/story/news/2017/02/04/what-disparity-studies-have-shown/97426044/

53.

Terman

J. N.

(2017). What happens when rules stay the same? Examining changes in implementation intent over time. International Journal of Public Administration, 40(1), 36–53.

54.

U. S. Commission on Civil Rights. (2006). Disparity studies as evidence of discrimination in federal contracting (Briefing Report). https://namc-oregon.org/uploads/qpbV3ylA/Disparity-Studies-As-Evidence-of-Discrimination-in-Federal-Contracting-2006.pdf