Disparity Studies: Isomorphic Discrimination?

Introduction

Minority-based set-aside programs have been a well-established method to advance social and economic equity of public procurement in the United States (Rice & Mongkuo, 1998; U. S. Commission on Civil Rights, 2006). Public procurement set-aside programs aim to remedy the reported consequences of past and current discrimination by establishing programs that increase minority-owned businesses, women-owned businesses, and disabled-owned business enterprises in government contracting (“M/WBE/DBE”) (Patrucco et al., 2017; Qiao et al., 2009). However, how governments justify these set-aside programs changed in 1989. As a result of the City of Richmond v. J. A. Croson Co., 488 U.S. 469 (1989) decision, the U.S. Supreme Court imposed several legal requirements on governments, including requirements to document current or past discriminatory practices in public contracting prior to establishing minority-based set-aside programs. According to the majority opinion, “racial quotas for awarding government contracts are not justified by general statistical evidence of inequality. The city [Richmond, Virginia] did not investigate any race-neutral methods to correct the imbalance, nor did its 30% goal correspond to any measured injury. Strict scrutiny is warranted, a test which Richmond's law fails.” ¹

Supreme Court Justice O’Connor created a loophole by launching the disparity study industry with the rationale established in Croson. Justice O’Connor stated, “where there is a significant statistical disparity between the number of qualified minority contractors willing and able to perform a particular service and the number of such contractors engaged by the locality or the locality's prime contractors, an inference of discriminatory exclusion could arise” (Croson, 488 U.S. at 518; see also Hanson, 2003). Therefore, state or local governments establishing specific policies to increase minority participation in government contracting must conduct a “disparity” study to provide a “strong basis on the evidence” (Atkinson et al., 2020; Wainwright & Holt, 2010).

Croson prompted utilizing statistically driven disparity studies by governments looking to implement preferential set-aside programs. Since Croson, federal, state, and local governments have monetized this quest by manufacturing evidence of discrimination using statistical disparity studies (Kasarda, 2015). According to La Noue (2021, para 14) over the last 30 years, “the field became dominated by a handful of for-profit consulting firms which have completed over 600 disparity studies at a cost to taxpayers of roughly $300 million.” La Noue (2021) further posits that these firms have come to dominate the field since most governments who wish to commission a disparity study use the request for proposal (RFP) or request for qualifications procurement methods to select the winning firm (Goel, 2016). These procurement methods do not rely solely on evaluating the most responsible and responsive lowest price bid. For most RFPs, the price is not negotiated until after the top firms are selected based on the knowledge, skills, and past and current experience of the submitted proposals (Qiao & Cummings, 2003).

Wainwright and Holt (2010) summarized guidelines for departments of transportation on disparity-study-related matters, including the determination of whether a study is needed, a prospective scope of services, and detailed study design elements. Published under the auspices of the National Academies of Sciences, Engineering, and Medicine, the detailed directions in the guidelines likely led to further specifications of expected patterns for acceptable disparity studies. In a later review, Atkinson et al. (2020) confirmed that most contemporary disparity studies contain the following elements: (1) legal review/analysis; (2) review of procurement law/policy; (3) establishment of market area; (4) utilization analysis; (5) availability analysis; (6) disparity analysis; (7) anecdotal analysis; (8) regression analysis; and (9) recommendations, very much in keeping with the Wainwright and Holt (2010) prescription. Atkinson et al. (2020) further connote that the study outcomes are primarily based on statistical evidence since the Supreme Court noted that anecdotal evidence, if supported by appropriate statistical proof, provides additional support that discrimination has occurred in the procurement process (Hanson, 2003).

The isomorphic nature of disparity studies can present several problems addressed herein. It can reduce the ability of arduous and costly studies to present a full assessment of community standing within procurement practices (McCue & Pitzer, 2000). We define community standing as the full representation of how the disparity studies represent community dynamics, this includes the social and economic status as well as the positionality of the city in question. These disparity studies can serve as a “boilerplate,” utilizing a tried and tested formula to satisfy court requirements irrespective of what reality demonstrates (Matthews, 2005). The importance of assessing these disparity studies creates a pressure point for policy advancement because they can be perceived as a cosmetic solution to significant contextual community issues (Wainwright & Holt, 2010). Finally, even though governments may aim to increase diversity and inclusion initiatives through their procurement of goods and services, the commissioning of these “boilerplate” disparity studies acts as a swift political dismissal/response to a considerable and ongoing issue. The studies may look responsive and progressive, but the larger issue of tackling inequity and discrimination requires context and effective conceptualizations.

This research examines the legitimacy of current approaches to conducting disparity studies. “Legitimacy” and “veracity” are used interchangeably to assess the methods used in disparity studies. Legitimacy is defined here as a “generalized perception or assumption that the actions of an entity are desirable, proper, or appropriate within some socially constructed system of norms, values, beliefs, and definitions” (Suchman, 1995, p. 574). In commissioning a disparity study, the goal is not to identify parity, but to find disparity. Objectively, this goal potentially creates a research bias. In practice, firms are contracted to find evidence of disparity and there is institutional pressure on these firms to find it—they will have failed if they do not find it, or if they find the “wrong” disparity (with the wrong group). These are not academically neutral studies. Moreover, previous research has not fully examined the collective nature of disparity study work products or their community effects (see Bates, 2001, 2015; Celec et al., 2000; Freilich et al., 1994; Rice, 1992; Rice, 1995; Rice & Mongkuo, 1998; Snider et al., 2013).

By examining 54 disparity studies at the city level conducted by nine firms, this research discusses their history, content, goals, statistical methods, and related policy phenomena to examine the impact on disparity study outcomes. It is hypothesized that by relying mainly on statistical standardization to show “evidence” of discrimination, studies fail to capture sufficient contextual evidence. These failures include historical circumstances of discrimination within certain communities, process challenges associated with public procurement, and the contextual understanding of the communities affected by discrimination. The focus of this study will be on the quantitative and qualitative strategies adopted within these studies. Quantitative strategies have become more recognized for their direct answers to questions of disparity, while qualitative measures that provide the full context for evidence of disparity are given lesser focus not only in these studies but in courts of law. However, this study is not a meta-analysis of effects, instead, this analysis considers methodology typologies without consideration of how conclusions differ with respect to whether there are consequential disparities based on statistical significance.

This article begins with a review of the evolution of disparity studies, including their legal basis. We address the evidentiary standards in disparity studies set out in the Croson decision (1989). We include a section examining the city disparity studies by the requisite firms and consider whether there are commonalities or differences in approaches. We then expound upon the contextual deficiencies of the studies and identify potential policy implications that can be valuable to state and local governments looking to institutionalize minority-based set-aside policies. This research provides some guidance, understanding, and areas of concern regarding disparity studies for cities to consider before paying the high financial and possible political price for these studies.

Disparity Studies and Legal Context

The legal history behind disparity studies established the necessary standard of review for questions predicated on equitable procurement practices. In the 1990s there was an adaptation to legal precedent in how disputes to public procurement were instituted relating to set-asides. There are two types of set-asides: (1) there are pure set-asides where a percentage of total government contracts are allotted to minority-owned businesses and (2) subcontractor set-asides requiring some of a “prime contractor's fee to be spent with a minority-owned contractor” (Wilson, 1987, p. 337; see also Rice, 1991). Programs favoring diverse businesses were challenged with the burden to prove the existence of a disparity and not a mere supposition (Atkinson et al., 2020). Furthermore, political context based on the public contract awards mattered. Despite the elimination of overt partisan patronage, there was still a push to distribute contracts based on race, ethnicity, gender, and special needs in the 1990s (Atkinson et al., 2020).

Disparity studies are legally justified, and their development has expanded since Croson and its progeny. The contemporary disparity study litigation originated with the U.S. Supreme Court case, City of Richmond v. J. A. Croson Company. In Richmond, Virginia, the city was at least 50% black yet only 0.67% of prime contractors were awarded to black-owned businesses. The state sought to remedy this through set-aside programs that required prime contractors to allocate 30% of their subcontracting dollars to firms with at least a 51% minority stake.

The case was adjudicated based on constitutional rights under the Equal Protection Clause (EPC) in the 14th Amendment. While it was evident that racial disparity existed because the prime contracts were not a representation of the city demographics, the Court's question rested on whether the present disparity was a result of systematic discrimination and whether the set-aside program was unconstitutional (Rice, 1992). Ultimately, six of the nine justices reasoned that absent explicit discrimination within a procurement process, governments should not be able to remedy harm that was not outwardly demonstrated. The justices held that the outcome was counter to the 14th Amendment's EPC because it afforded distinct groups opportunities that were not narrowly achieved to specific instances of discrimination.

The standard of review adopted was strict scrutiny. Richmond had to demonstrate that through the 30% race requirement, their action was necessary to achieve a compelling government interest and that the methods used to achieve this government interest were narrowly tailored (Croson, 488 U.S. at 493-94; Nay & Jones, 1989). Richmond's set-aside policy did not meet this standard. The U.S. Supreme Court ruled that evidence of past discrimination was not sufficient enough to establish the set-aside program. There needed to be a clear showing of proof of injury or denial of property rights to be valid, that is, “a compelling interest” to encourage creating a set-aside program.

Establishing procurement set-aside programs to remedy inequality is not novel. House-Soremekun (2007) tracked set-aside plans in the United States dating back to the 1930s, with the New Deal, and subsequent federal orders. State and local programs to expand the presence of diverse businesses in the 1980s comprised of a social-disadvantage component, to include minorities based on the assumption of disadvantage (Atkinson et al., 2020). While the issue of remedying racial disparities was ripe for analysis in Croson, the questions surrounding challenges to that holding did not end in 1989. The outcome of public procurement raises many questions, namely that the courts have wrestled with the evidence necessary to demonstrate when discrimination has occurred. There is a juxtaposition between the methods and evidence necessary for a successful demonstration of discriminatory practices, where the Courts have wrestled with which evidence to give strength to, and what is the most compelling method to adhere to the strict scrutiny requirement of Croson (Rice, 1991).

In 1995 (para.10), in Adarand Constructors, Inc v. Pena, the Court “extended the strict scrutiny standard to federal race preferential contracting programs” and continued to reign in previously loose requirements for imposing race-based contracts (515 U.S. 200). The issue was whether the presumption of disadvantage (predicated on race) resulting in preferential treatment was a discriminatory practice that violated equal protection under the Due Process Clause of the Fifth Amendment. The court held that “all race-based government statutes or policies must be evaluated under a strict scrutiny analysis, even if they intended to remedy racial discrimination” (515 U.S. 200, para 12).

Adarand was a landmark decision with the Court prohibiting government flexibility in adopting racially classified legislation to remedy previous discrimination. Strict scrutiny was still the adoptive standard. The court's approach to this analysis was focused on three concepts: skepticism, consistency, and congruence. Thus, regardless of litigation level or intent behind the race-based classifications, strict scrutiny will always be the operative standard. There was a further constraint for narrowly tailored set-aside programs, such that programs are only as broad as reasonably necessary to accomplish government objectives (Garner, 2014).

Historically, there has been extensive litigation based on the programmatic results of procurement disparity studies. In Contractors Ass’n of E. Pa. v. City of Philadelphia, 893 F. Supp. 419 (E.D. Pa. 1995), the Contractors Association aimed to prohibit Philadelphia's preferential procurement program that was adopted due to a previous disparity study. In that case, there was a facial constitutional challenge to provisions of Chapter 17-500 of the Philadelphia Code (Chapter 17-500). The provisions created preferences for contracts with diverse businesses. They contended that Chapter 17-500's goal to increase participation of those communities by 15% did not satisfy strict scrutiny under the Equal Protection Clause of the Fourteenth Amendment, and the court agreed. Additionally, the federal court found that the disparity study was deficient in its measurement of qualified, willing, and able firms and thus insufficient. The study focused on just the disparity index as a centerpiece.

The results of the disparity study showed a substantial statistical disparity in the participation of diverse businesses in Philadelphia Public Works contracts for fiscal years 1979 to 1981, as compared to their readiness in the market. The disparity index was facial evidence of racial discrimination. However, rising to the level of strict scrutiny in this study was deficient based on the sole use of the disparity index. The court held the disparity study was deficient because there was not a sufficient measure of businesses that were available, qualified, and able to contract (the three pillars of Croson) (893 F. Supp. 419 (E.D. Pa. 1995)). The court focused on the delivery of the quantitative methods used as a metric to assess the viability of the claims. Moving forward disparity studies started to be more conscious of the three pillars of Croson (Celec et al., 2000).

Further challenges were presented in Associated General Contractors of America v. City of Columbus, 936 F. Supp. 1363 (S.D. Ohio 1996). The city had imposed 21% minority-owned business and 10% women-owned business subcontractor goals. Disparity studies were conducted by two major firms; however, the district court judge did not accept their findings as valid regarding the disparity studies (Associated General Contractors v. Cal. Dep’t of Transp., No. 11–16228 (9th Cir. 2013)). The disparity studies did not support a finding of discrimination against diverse businesses because, despite the disparity studies’ evaluation of the business ownership rates, that was only a mere assumption of discrimination. The court also refuted the use of Census data to produce the study's conclusion. Therefore, the broad presence of diverse businesses within the community did not mean they would be economically represented in the same capacity (Associated Gen. Contractors of America v City of Columbus, 1996).

The same year in Engineering Contractors Association of South Florida v. Metropolitan Dade County, a federal district court held the county's use of anecdotal evidence was not persuasive in the disparity study as a central indicator of discrimination. Decision-makers were concerned with interviewer bias and response bias in the study. Individuals had a vested interest in an outcome and consequently, their anecdotal recollection of evidence demonstrated that bias (Creswell & Poth, 2016). An analysis of discrimination needs a balanced approach to make comparable conclusions. In this case, the court held that absent the ability to corroborate allegations with quantitative indices then evidentiary findings were inadequate.

In Western States Paving v. Washington State Department of Transportation, Washington state found a showing of discrimination since 11.17% of the firms were diverse, whereas only 9% of the contract funds were awarded to them (407 F.3d 983, 1000–01 (9th Cir. 2005)). In this instance, the issue of relative capacity to contextual issues was contended by the Ninth Circuit. The oversimplification of discrimination without addressing contextual issues that affect the capacity of DBEs to contract was inadequate. These cases demonstrate an initial finding of disparity is the catalyst for discrimination inquiries on whether discrimination is the causal factor and not an outcome. Thus, courts are consistent with the requirement that type, extent, and manner of proof are rigorously required as effectual factors other than a claim of discrimination.

Evidentiary Standards for Disparity Studies

The analysis of the adjudication and rationale behind disparity study challenges can be compared to the debates on the two forms of evidence in courts of law (Mayanja, 2017). The courts have established no greater weight assigned to direct evidence over circumstantial evidence yet they both independently serve as sufficient to establish fact (Mayanja, 2017). Direct evidence establishes a fact absent the need to make inferences as a prerequisite (Blum, 2019). When evidence is established as true then it is considered a fact. Circumstantial evidence requires the use of inference to connect evidence to fact (Blum, 2019). There is no direct connection between evidence and fact through circumstantial evidence, thus inferences create that connection (Heller, 2006). Given that there is no greater weight attributed to direct evidence over circumstantial evidence, either is acceptable to establish a verdict (Blum, 2019; Mayanja, 2017).

Within the context of litigation of disparity studies, there is some variance in the strength given to the methods, but overall, the methods when considered in tandem hold equal weight. The quantitative methods used to show discrimination often operate reputationally like direct evidence. Concrete determinations of fact are represented and/or misrepresented through statistical analysis and indices in disparity studies. Firms collect and assess quantifiable contracting data to verify the incidence of disparity. They establish indices to quantify economic inequality. The use of those indices has become a norm in the field, and most consulting firms follow this model. The firms recognize the strength that the courts have placed on these methods, and they follow suit irrespective of the fact that quantitative methods, while appearing more rigorous, may be an inaccurate holistic evidentiary form.

Qualitative evidence operates similarly to circumstantial evidence. Unlike quantitative methods, these methods mean that inferences can be made from the data collected through qualitative methods, to establish facts about the representative cities. Courts have not accepted qualitative methods (within disparity studies) as solely acceptable evidence despite their contextual strength in the same way courts view circumstantial evidence (see generally Blum, 2019; Wigmore, 1904). This approach means that the value attributed to circumstantial evidence rests in its cumulative force. Its operation as a singular source of evidence does not realistically provide sufficient probative value (Mayanja, 2017). While some circumstantial evidence may have inherent cogent value, it becomes useful when combined with quantitative approaches in disparity studies.

Judicial factfinders are routinely fused to assess the reliability and veracity of evidence giving adequate weight (without prejudice) to both direct and circumstantial evidence (Zamir et al., 2016). However, in some areas of the law like criminal law, direct evidence is the preliminary choice to establish facts and verdicts, whereas circumstantial evidence serves as supplementary support. The perception is that circumstantial evidence is inferior evidence with less persuasiveness or probative value than direct evidence (Mayanja, 2017).

The evidentiary distinction and cumulative nature of direct and circumstantial evidence lend themselves to the probative nature of evidence. Thus, a ruling need not rely on a “priori reason to classify circumstantial evidence as probatively inferior, or to suspect that it leads to less accurate outcomes than direct evidence” (Blum, 2019, p. 64). In the same way, courts connote the standard of combined value that qualitative and quantitative methods offer is most probative for assessing the nature of disparity studies.

Similarly, the main consulting firms that conduct disparity studies use direct quantitative methods to establish discrimination inherent in the representative cities’ government procurement processes, whilst using the qualitative evidence as “circumstantial” support to reinforce quantitative outcomes, in identifying specific examples of discrimination, or where barriers to contracting exist. Qualitative means may be the only way a study has to tell the story of vendors’ interactions with procurement, in experiencing active discrimination on the part of official city processes and actors, and yet, qualitative means are secondary, if present at all. In some instances, firms do not use any qualitative methods. The qualitative evidence provides the underlying context to better understand behavioral characteristics in a city with disparities that data, in quantitative analysis and disparity indices, cannot capture.

Jurisprudence establishes structure, consistency, and fairness. However, with the need to exercise justice under the law, the current state of disparity studies provides little resolution. Studies are becoming more commonplace and indistinguishable because they are focused solely on the existing working model that courts use to determine which studies are adequate. The studies focus on quantitative methods paying little attention to the qualitative methods that provide context to issues of discrimination.

While the Supreme Court in Croson did not deliver recommendations or requirements of specific statistical approaches to establish discrimination, studies from the most prominent legal firms ² have adopted similarly consistent practices. These practices, although defined by the requirements in Croson, offer minor distinctions based on which firm has conducted the study. The degree to which these differences present, is an issue for determining whether classic procurement goals are achieved and extended to account for contextual bases. Attention must be paid to the isomorphism present within these practices such that history, context, reform, and development become central themes inherent in analyzing public procurement discrimination (Atkinson et al., 2020).

Methods

This research represents a line of investigation into disparity studies, constitutional applications, and diversity contextual analysis. Disparity studies were analyzed to assess the veracity of current approaches to conducting disparity studies by methodologically drawing on 54 cities and nine firm analyses, with over 900 firms assessed within those studies. Purposeful sampling was operationalized to gather available information from the necessary studies and conduct an in-depth assessment.

The criteria for selecting firms consisted of using the largest consulting firms that had conducted the most disparity studies between 2010 and 2021. Experience is emphasized when government entities allow these large consulting firms to conduct these studies (Gallo, 2018). The most sought-after firms to deliver these disparity studies were those that effectively completed studies where resultant programs withstood challenges in courts (Atkinson et al., 2020). From that pool, assessments were split by state, city, and county and found a greater sample of city-wide disparity studies by the requisite nine firms. This resulted in a purposeful sample of 54 city studies conducted by the nine most prominent consulting firms in the industry.

A four-step analytical process was used for categorical coding which included (1) performing a comparison of significant methods used within the given quantitative and qualitative categories to facilitate inductive categorical coding; (2) being able to refine distinct categories; (3) exploring relationships and patterns across firm studies; finally (4) integrating data to conceptualize the findings and assess whether contextual deficiencies exist within the studies (Lungu, 2022; Saldaña, 2021).

Perceptual data was collected using document reviews and represented using cross-study comparisons. A within-case and cross-case was used (Gibbs, 2007; Miles et al., 2014; Shkedi, 2005) to create visual summaries of the data acquired from the disparity studies. These visual summaries included the range of observations within the studies. As we accounted for the attributes of each study, we recognized the prevalence of each research method (quantitative and qualitative) and provided a summary of total tallies for each firm and method before thematically evaluating the data (Creswell & Poth, 2016; Patton, 2014). We collected and analyzed the programmatic data to assess the merit, worth, effectiveness, and quality of research methods applied by each firm. This collation showed which firms used certain quantitative or qualitative methods and to what extent.

Overall, the symbolism and comparison of these studies was imperative. For some firms, disparity studies consisted of study-specific data generated to estimate the parameters of regression models and sensibly capture the effects of race/gender/ethnicity on outcomes directly related to public procurement success/failure. These include contract/subcontracting awards, as well as those indirectly related such as commercial bank loan denials and building permits. In addition, firms estimated the parameters of regression models in which capacity/ability are controlled for, allowing for firm classification dummy variables to capture the effects of being a minority firm on outcomes of interest. Additionally, some disparity studies captured the contemporary role of exclusive informal networks and perceived discrimination, and submission behaviors; all of which inform barriers to inclusive procurement, the social standing/positionality of nonwhites, and the contemporary context. Although these structures exist, the samples established here show that this systematic process is not commonplace by industry standards.

The nine firms have constructed their reputation as respected firms such that the production of a study is forefront to the perfunctory content of the study. Consequently, governmental entities are essentially purchasing findings of discriminatory practices within a state/city/county, so that they may implement a narrowly tailored race-conscious public policy program to address the study findings. A secondary aspect of the involvement of a disparity study consultant is that the consultant would reasonably be expected to make themselves available to defend the study in court, should the study (and the resultant program) be challenged on a legal basis. Disparity studies cost hundreds of thousands of dollars (Kettunen & Kwak, 2018), and inconsistencies in contract data can increase the cost of conducting studies.

Governments and study consultants alike prefer the use of clean data to conduct these studies, which explains the structured isomorphic methodologies. Consequently, an over-reliance on structured isomorphic quantitative methodologies is common, but can be detrimental to community standing and the effectiveness of these disparity studies. Finally, qualitative methods should augment quantitative methods by providing a supplemental focus on conceptual issues at hand. The nexus of the two methods show strong alignment between both will be necessary to create balance and utility.

Context Deficiencies and Policy Implications

The traditional procurement approach is highly centralized and regulated. Within this centralized paradigm, public procurement goals should espouse efficiency, economy, equity, and integrity (Klein, 1991; McCue & Pitzer, 2000). Given the reformation of procurement practices, societal exhorts are predicated on improved systems administration. Thus, the traditional procurement model has been a targeted source for reform (Coggburn, 2003). Traditional structures should exemplify open competition goals as a trademark of public procurement thus promoting equitable economic advancement and universal access (McCue & Pitzer, 2000).

With particular attention to procurement network limitations, Rosa (2022) presents “best next steps” and appeals to mechanism design, auction theory, and attention to statistical discrimination to develop augmented procurement bidding schemes. Disadvantaged firms may have limited access to networks that extend their access to contracts giving rise to highly inequitable equilibria. This introduces “history” as a source of disparity because disadvantaged firms do not have networks in which primes can deem them qualified. Therefore, government interventions may fail to remedy disparity and may have the unintended consequence of discouraging investment (Rosa, 2022). When government agencies use broad statistical categories to evaluate distributional objectives, they may fail to capture all forms of discrimination and ultimately undermine equitable practices. Rosa's (2022) findings suggest that governments need to augment their procurement design bidding schemes to incentivize discrimination-free procurement outcomes, paying close attention to historical network biases against disadvantaged firms.

The use of disparity studies has been a developmental answer to assessing cities with set-aside programs (Blount & Hill, 2015). These studies verify that city set-aside programs that increase diverse business contracts will withstand scrutiny in a court of law. These studies “assess, quantify, and evaluate the prevalence, significance (degree and weight), and scope of discrimination in the marketplace against diverse businesses” (Capital Metropolitan Transportation Authority, 2021, p. 68).

The narratives surrounding disparity studies have been described as “impassioned, reformist, and hopeful” (Atkinson et al., 2020, p.12). They serve as an insurance policy for instituted programs such that there will be little dispute regarding the methods or recommendations within the studies (Alphran, 2003). Unfortunately, the isomorphism present in disparity studies reduces the ability of the analysis to result in added initiatives that answer questions of historical discrimination both within the community and in the domain of public procurement.

When considering the quantitative methods adopted by the consulting firms herein, there is a specific protocol that these firms follow when delivering their studies, considered boilerplate. The standardized quantitative formula has become commonplace and a successful anchor for these firms to use due to familiarity with these methods in courts. While the litigation process values structure and predictability, these firms operate consistently with the expectations set out for them in terms of methodology. Quantitative methods look the same, but in reality, communities face varying differences. For this reason, it is important to question the isomorphism present and how to better operate using contextual analysis and history to make quantifiable and immediate improvements to communities. Isomorphism is problematic as a reflection of societal issues that would be best remedied through cumulative tools addressing contextual and historical contexts.

Regarding qualitative methods, Croson and its progeny did not set up recommendations or requirements to measure discrimination, the courts subjectively adopted standards that firms have utilized. Given that qualitative methods are often used in an ancillary manner, some firms have opted not to use them at all or to use them minimally, focusing their efforts on quantitative practices, presumably because quantifiable outcomes give the appearance of validity and reliability. Despite some overlap within the firms in using familiar qualitative methods, these methods still do not deliver a new paradigm of knowledge about how discrimination is a lived and harmful experience that must be wholly contextualized (Hemmatian et al., 2021; Skrentny, 2002). When considering diverse businesses, they typically face vastly different barriers to establishment and growth such that the blanket recommendations developed from these disparity studies do little to account for the distinct issues that these businesses face.

Conclusion

Evaluations of disparity studies show a deficiency in the contextual grounding. The decision to commission a valid study and present recommendations to a city to improve procurement practices justifiably requires close examination (Schapper et al., 2006). While these studies propose reforms, in the long term, they risk becoming inadequate, inefficient, and expensive props commissioned by governments.

Inequities in government services, programs, and contracting have been systemically linked to regulations, policies, and laws (Snider & Rendon, 2008; see generally Lungu & Lungu, 2021). Failure to account for the unique barriers faced by systemically marginalized communities has resulted in a history of litigation over contracted disparity studies. Resultantly, the normative recognizance of inclusive practices should be a sustained strategy for equality within public procurement.

Analyzing the city-level disparity studies shows generalized ratios frequently unaccompanied by qualitative methods to support findings. Thus, absent more contextual and immediate findings, it becomes difficult to establish a narrowly tailored remedy for requisite cities. Establishing strategic parity within public procurement practices will remain an important consideration, particularly where there is a history of discrimination (Guarnieri & Gomes, 2019).

The novelty of this research is based on an analysis of 54 disparity studies at the city level to provide some resolve and understanding of the methods used both quantitatively and qualitatively to accomplish these studies. Our question rests on whether the current state of these studies provides enough contextual basis to improve societal needs and development. Despite some examples of firms seeking to understand the wider context of discrimination issues, these approaches are limited in scope despite the costs to commission these studies continually increasing.

This research serves to inform public officials that these very expensive studies rarely identify the specific source of discrimination or recommend sufficient and specific remedies to those who are most impacted. If they do identify the source of discrimination or provide specific remedies, there is minimal attention given to the “best next step.” This boilerplate approach means firms are not considering contextual factors in each city, or, if they are, it is at a minimum. This observation might mean that it is difficult to show whether disparity studies are the most effective tools for change. Surely more can be done for such an expensive price tag to identify and resolve disparity.