Abstract
Free Clinics (FCs) in the United States contribute to the healthcare safety net by providing care to a variety of populations including the uninsured. Data Envelopment Analysis (DEA) is used to evaluate FC performance by examining clinic funding sources and the number of visits and prescriptions provided on an annual basis. Cross-sectional data including 48 Virginia FCs in 2010 are analyzed to distinguish efficient and inefficient FCs. In all, 30 clinics (62.5%) were found to be top performers and defined the efficiency frontier, whereas 18 clinics (37.5%) were evaluated as inefficient. On average, to obtain efficiency, inefficient clinics would annually need to increase the number of provided general medical visits by 2,183, specialty visits by 1,969, other visits by 1,495, and dispensed prescriptions by 7,305. These findings have implications for healthcare policy and FC management.
Introduction
The healthcare safety net in the United States is defined by the Institute of Medicine as “providers that organize and deliver a significant level of health care and other related services to uninsured, Medicaid, and other vulnerable patients” (Lewin & Altman, 2000, p. 3). This includes, but is not limited to, organizational types such as Federally Qualified Health Centers (FQHCs), public health departments, academic medical centers, migrant health centers, and free clinics (FCs). FCs are an important contributor to the healthcare safety net, yet they remain an understudied organizational form. With the passage of the Patient Protection and Affordable Care Act (ACA) and subsequent Supreme Court ruling in June 2012, researchers and policy makers have focused attention on organizations that contribute to the healthcare safety net. It is estimated that approximately 20 million individuals will remain uninsured following full implementation of ACA legislation (Chazin, Friedenzohn, Martinez-Vidal, & Somers, 2010) including undocumented populations (Jerome-D’Emilia & Suplee, 2012) and people who are non-compliant with the individual mandate (Hall, 2011). These individuals will continue to seek care in safety net healthcare organizations, including FCs.
FCs in the United States annually provide medical care to approximately 1.8 million poor and uninsured patients (Darnell, 2010). Individual FCs have developed and evolved over time to meet the needs of their communities, and often fill a healthcare gap identified by medical professionals or community members (Weiss, 2006). Although FCs provide a variety of services that reflect distinct community needs paired with a community-specific approach, a few common themes among FCs include their dependence on volunteers (Geller, Taylor, & Scott, 2004), their non-profit organizational form (Darnell, 2010), and their mission to serve a majority of uninsured patients (Darnell, 2010).
FCs have been the unit of analysis for a number of past studies, including those that focus on patient satisfaction (Gertz, Frank, & Blixen, 2011), quality of care (Ryskina, Meah, & Thomas, 2009; Tennant & Day, 1974), and adherence to clinical guidelines (Zucker, Gillen, Ackrivo, Schroeder, & Keller, 2011). However, to the authors’ knowledge, overall FC operational performance has not been documented in the literature. This article aims to fill this gap by utilizing Data Envelopment Analysis (DEA) techniques to evaluate FC performance. The main objective of this article is to identify top FC performers and inform FCs of potential performance improvement measures. Due to data limitations, this article should be considered a first step in defining, identifying, and evaluating FC performance.
Method
DEA methodology utilizes a linear combination of outputs over a linear combination of inputs to determine unit performance. This technique was first developed in 1978 by Charnes, Cooper, and Rhodes, and has been widely used in the health care sector, including analyses of addiction treatment facilities (Corredoira, Chilingerian, & Kimberly, 2011), community-based youth service organizations (Yeh, White, & Ozcan, 1997), mental health organizations (Brandeau, Sainfort, & Pierskalla, 2004), agencies on aging (Ozcan & Cotter, 1994), community mental health centers (Tyler, Ozcan, & Wogen, 1995), and ambulatory surgery centers (Iyengar & Ozcan, 2009). In addition, over 79 articles have been published utilizing DEA methods examining hospital performance in the United States and Europe (O’Neill, Rauner, Heidenberger, & Kraus, 2008).
DEA methodology examines a group of decision-making units (DMUs)—the name assigned to individual units of analysis—and identifies top performers that define the efficiency frontier based on the simultaneous evaluation of all defined unit inputs and outputs. This analysis will refer to DMUs as FCs. Rather than comparing each FC with the mean value, as is essentially the case when using linear or logistic regression methods, DEA compares inefficient FCs with top performers by identifying an efficiency frontier and assigning them a performance index value of 1. Hence, each clinic is evaluated based on the comparison with their top performing peers on this efficiency frontier.
Figure 1 provides a conceptualization of an efficiency frontier defined by DEA methods and is not reflective of the data utilized in this article as study variables cannot be portrayed in a two-dimensional figure. For simplicity, assume that the analysis examines only two outputs, general medical visits and specialty medical visits, and one input, government funding, for 21 FCs. DEA analysis considers the output to input ratio for each of these outputs. The clinics with the best combination of outputs to inputs characterize the efficiency frontier and are considered the top performers (Clinics 6, 12, and 20). All other clinics need to improve their output to input ratio to reach the frontier. The distance between the FC and the efficiency frontier is defined as the efficiency target for that FC. Therefore, DEA does not define one organization as the unique model for efficiency; rather, clinics with maximized outputs for given inputs, as a group, will define top performance.

Conceptualization of efficiency frontier for an output-oriented DEA model.
Figure 2 illustrates this study’s conceptualization of FC inputs and outputs in which FC performance is measured according to the number of medical, specialty, and other patient visits provided by FCs given their various funding levels, including government, private, and miscellaneous funding sources. Consequently, FCs that annually provide care and prescriptions to more patients utilizing fewer financial resources would be identified as higher performers and are better able to maximize their funding dollar. For descriptive purposes, imagine two FCs: Clinics A and B. Both clinics would be seen as equally effective in our model if Clinic A received US$10,000 total dollars annually and provided 125 patient visits and Clinic B received US$200,000 total dollars annually and provided 2,500 annual visits (example adaptation; Corredoira et al., 2011). This measurement is independent of other resources used such as volunteers, paid staff, clinic space, patient case mix, and quality of care. This limitation will be discussed below.

Inputs and outputs for free clinic analysis.
DEA analysis results in individual efficiency scores and identifies which outcomes need to be increased to obtain efficiency (Ozcan, 2008). These results for individual FCs have the potential to demonstrate to policy makers and FC administrators how time and resources can be allocated to maximize individual clinic performance. In addition, DEA utilizes a number of inputs and outputs simultaneously, and does not rely on an a priori functional form linking inputs and outputs (Corredoira et al., 2011).
Prior to analysis, DEA methodology requires the definition of two specific criteria. First, DEA estimates can be obtained by utilizing a constant returns to scale (CRS) or variable returns to scale (VRS; Ozcan, 2008). This article adopts a VRS model because an increase in funding may or may not lead to a linear proportional increase in the number of visits provided in any given FC. Second, DEA methodology also requires that the user specify an input-oriented model, an output-oriented model, or a non-oriented model (slack based model). This article adopts an output-oriented model based on the assumption that FC managers have more control over outputs (i.e., the number of annual visits per service type the clinic provides; Ozcan, 2008), in addition to the fact that FCs do not aim to reduce funding, rather increase the number of annual visits with the limited or given funding available. Efficiency scores for output-oriented models may range from 1 to infinity; however, these scores will be converted to a 0 to 1 scale by calculating the reciprocal of the output-oriented efficiency score for ease of interpretation.
A diverse group of organizations define themselves as FCs, despite differences in services provided. DEA analysis requires that units be suitable for comparison. For example, a number of FCs provide only mental health services while others provide support services to other clinics (i.e., prescription services). To standardize the types of services provided by FCs for this analysis, inclusion criteria required that each clinic provide general medical visits and specialty visits.
A detailed description of DEA methods and the calculations behind DEA have been described at length in the literature (Cooper, Seiford, & Tone, 2006; Corredoira et al., 2011; Ozcan, 2008). We provide a brief explanation for the calculation of DEA efficiency scores here using mathematical notations adapted from Ozcan (2008, pp. 24-56). The efficiency scores (θ o ) for a group of peer clinics (j = 1, . . . , n) are computed for the selected outputs (yrj, r = 1, . . . , s) and inputs (xij, i = 1, . . . , m) using the following fractional programming formula:
In this formulation, the weights for the outputs and inputs, respectively, are ur and vi, and “o” signifies a focal clinic (to obtain efficiency scores, each clinic, in turn, becomes a focal clinic when its efficiency score is being computed relative to others). Note that the input and output values, as well as all weights, are assumed by the formulation to be greater than zero. The weights ur and vi for each DMU are determined entirely from the output and input data of all clinics in the peer group of data. Therefore, the weights used for each clinic are those that maximize the focal clinic’s efficiency score. To solve the fractional program described above, it needs to be converted to a linear programming formulation for easier solution. As the focus of this article is not on the mathematical aspects of DEA, an interested reader is referred to Ozcan (2008) for more detail on how the above equations are algebraically converted to a linear programming formulation. Other DEA books may also be consulted for an in-depth exposure (Cooper et al., 2006).
In summary, the DEA identifies a group of optimally performing clinics that are defined as efficient and assigns them a score of one. These efficient clinics are then used to create an “efficiency frontier” or “data envelope” against which all other clinics are compared.
Data
Data were obtained through the Virginia Association of Free and Charitable Clinics (VAFCC). The VAFCC included 60 member organizations in 2010. These 60 clinics and clinical support organizations provide a variety of care, including general medical visits, specialty visits, dental visits, health education visits, mental health visits, and prescriptions to over 82,000 unduplicated patients annually, with an estimated value of nearly 119 million dollars (VAFCC, 2011). In all, 48 of the member clinics in the dataset met the inclusion criteria. The study’s sample size meets the suggested rule that the number of units should be at least three times the total number of inputs in DEA analysis (Cooper et al., 2006).
Results
In 2010, the 48 evaluated FCs received a total of US$6,701,208 from government sources, US$19,510,414 from private sources, and US$1,737,861 from miscellaneous sources. This funding provided a total of 157,979 general medical visits, 99,379 specialty visits, 104,873 other visits, and 671,624 dispensed prescriptions. Table 1 provides additional descriptive statistics.
Descriptive Statistics for Study Variables (n = 48).
The VRS output-oriented DEA model resulted in 30 efficient FCs (with an efficiency score of 1) and 18 inefficient FCs (with an efficiency score below 1), resulting in 62.5% of Virginia FCs achieving efficiency in 2010. By definition, inefficient clinics need to increase their outputs while holding inputs constant to obtain efficiency. Inefficient clinics, on average, need to provide an additional 2,183 general medical visits, 1,969 specialty visits, 1,495 other visits, and dispense an additional 7,305 prescriptions to be deemed efficient. Table 2 displays the average calculated number of additional services needed by inefficient FCs to achieve efficiency according to DEA results.
Efficiency Targets for Inefficient Free Clinics (n = 18).
Discussion
Overall, 62.5% of Virginia FCs in 2010 were found to be efficient, but to put the 18 inefficient clinics (37.5%) into perspective, consider the actual results of two example clinics: Clinic X and Clinic Y. With 2010 reported data, Clinic X achieved an efficiency score of 0.822, or 82.2% efficient. This score categorizes Clinic X as inefficient because it is not equal to 1. In general, Clinic X needs to increase outputs by 17.8%. During 2010, Clinic X collected US$82,655 from government sources, US$284,394 from private sources, and US$10,962 from miscellaneous sources. During the year, Clinic X provided 2,991 general medical visits, 576 specialty visits, and 969 other visits, and dispensed 7,691 prescriptions. Based on the DEA results, Clinic X should have been able to provide an additional 726 general medical visits, 787 specialty visits, 209 other visits, and 1,661 prescriptions.
Clinic Y achieved an efficiency score of 0.453, or 45.3% efficient, meaning it needs to make much more improvement than Clinic X to obtain efficiency (i.e., increase outputs by 54.7%). During 2010, Clinic Y collected US$41,333 from government sources, US$141,500 from private sources, and US$8,571 from miscellaneous sources. With this funding, Clinic Y provided 880 general medical visits, 142 specialty visits, and 350 other visits, and dispensed 5,246 prescriptions. According to the DEA results, with given resources, Clinic Y should have been able to provide an additional 1,213 general medical visits, 338 specialty visits, 771 other visits, and 6,315 prescriptions. Table 3 displays the differences between actual and target outputs for Clinics X and Y.
Two Examples of Inefficient Free Clinics.
Note. DEA = Data Envelopment Analysis.
The above description of Clinics X and Y can guide clinic administrators regarding approaches to achieve efficiency. Clinic X is much closer to efficient (as demonstrated by an efficiency score closer to 1) and therefore has fewer improvements to make. Although Clinic X is more efficient than Clinic Y, both clinics need to focus on providing additional services to maximize efficiency. Management at Clinic X may want to focus their attention on increasing the amount of other medical visits, whereas Clinic Y may want to focus on providing additional other visits or prescriptions because these are the two areas where they could maximize efficiency with the lowest percentage increase in services.
These decisions should be made on an individual clinic level because clinical aspects of each of these outputs may vary across organizations. For example, based on individual clinic strengths, some administrators may focus on recruiting new volunteers whereas other clinic administrators may focus on scheduling more patients per volunteer hour. Administrators at each of the inefficient clinics are suggested to discuss the results with clinic staff and volunteers to understand how to augment the various service types to best meet the needs of their patient base.
This study has further implications for efficient clinics. From a managerial perspective, these clinics need to maintain their position on the efficiency frontier. FCs typically approach funding on an annual basis, so every year, efficient clinics need to provide the maximum amount of services across service types given limited funding resources. In addition, funding sources and amounts change annually. If funding increases, the number of visits also needs to increase to maintain efficiency by maximizing the output to input ratio. Future studies should utilize longitudinal FC data utilizing Malmquist DEA techniques to evaluate changes in clinical performance in time.
A number of policy implications can be drawn from these results. One of the many roles of the VAFCC is to dispense annual funding allocated to FCs by the Commonwealth of Virginia. This insight into FC performance could serve as a guide for new funding criteria or may provide direct support to inefficient clinics to improve performance.
Face validity
Prior to this research, FCs had not yet been evaluated using DEA techniques. Results were discussed with the Executive Director of the VAFCC to gain clarity regarding face validity. In general, the distribution of efficient and inefficient clinics was consistent with FC knowledge. A few FC efficiency scores were questioned, but after considering additional factors that were not included in this analysis (including the limitations discussed above), the results were considered to have face validity.
Limitations and future research
This initial attempt to understand FC efficiency has a number of limitations. First, the data included FCs located in the Commonwealth of Virginia, introducing generalizability concerns. The authors acknowledge that Virginia FCs do not represent all FCs in the United States, although Virginia FCs do share a number of characteristics with national FC statistics. Patients receiving care in VAFCC member clinics in 2010 reflect national FC patient demographics (Darnell, 2010). The majority of patients at Virginia clinics were female, between 18 and 64 years of age, Caucasian, and earned less than 100% of the Federal Poverty Level (FPL; VAFCC, 2011). The top three diseases among Virginia FC patients in 2010 were diabetes, hypertension, and depression/anxiety (VAFCC, 2011). Although a nationally representative understanding of FC patient diseases is, to the authors’ knowledge, unknown, these ailments are similar to studies that have identified common free patient disease characteristics (Nadkarni & Philbrick, 2003; Notaro et al., 2012).
A second limitation to the study is that this evaluation assessed a partial production system in these clinics and is valid for the portions evaluated. This partial production system is defined as the amount of funding available to provide specific services. Results would differ if alternative inputs and outputs were defined for the analysis. For example, additional inputs of interest include volunteer and employee hours whereas outputs potentially include student-training hours. These additional input measures could potentially help to target future volunteers (Komp, Van Tilburg, & van Groenou, 2012) and employees. The analysis presented here aimed to provide a practical evaluation of FC inputs and outputs that could easily be applied to FC performance. For example, when defining FC inputs, funding sources were not combined into a “total funding” variable, as the various funding streams demand different fundraising techniques and attention. Likewise, outputs require varying action by FC volunteers and staff. Of particular interest is the output evaluating prescriptions. In the FC environment, dispensed prescriptions represent the number of prescriptions filled by the organization, rather than written prescriptions (filled/unfilled, on/off site). As this is an output for the DEA performance evaluation, inefficient providers need to augment prescriptions to reach the frontier while holding funding constant. In traditional settings, this augmentation may encourage inappropriate overprescribing behavior; however, in FCs, the incentive is to appropriately treat as many patients as possible given funding constraints. In addition, as DEA techniques compare clinic performance with the top performers in the data set, it is possible that all clinics are generally very efficient or performing poorly. This is consistent with scholarship that argues non-profit effectiveness is a matter of comparison (Herman & Renz, 1999). The DEA analysis presented cannot evaluate the general performance of FCs as a whole; rather, it assumes that some clinics outperform others. Finally, it would be of great interest to include clinical-based outcome measures in the evaluation. Many clinics are in the process of collecting such measures to demonstrate quality of care to funders and patients (North Carolina Association of Free Clinics, 2011), but such statistics were not uniformly available for Virginia FCs. Although outputs were defined as the number of visits (and prescriptions) provided on an annual basis, the quality of each of these visits may vary by clinic. Patient demographics are also not included in this study, yet it is clear that each clinic provides care to a unique population that likely affects clinic performance.
Future studies should examine the above issues in addition to utilizing the results to better understand FCs. One approach to gain further understanding of FC performance while addressing current limitations is to evaluate factors that may be associated with clinic performance. These potential factors include patient, volunteer, staff, and community characteristics. Data regarding health care quality outcomes, once uniformly collected, would also improve our understanding of FC performance. In addition, it is suggested that researchers use DEA results as a guide to qualitatively explore efficient and inefficient FCs to understand why clinics are high or low performers and how they obtain this status. A few potential research interests include clinic policies and procedures, the use of teams at FCs, volunteer perspectives on efficiency, administrative approaches to manage clinic staff and volunteers, and the role of executive directors and board members in obtaining funding sources (Brown, Hillman, & Okun, 2012). This information could be used to provide tailored guidance to the needs of inefficient FCs.
Finally, the data used for this study are reported on an annual basis to the VAFCC. Although similar data have been collected since 2007, these reports may not accurately reflect funding or visits and prescriptions provided by the clinic. Member clinics utilize a variety of software to manage data and some clinics do not yet have access to such tools. To minimize data errors, the VAFCC provides definitions for reported data of interest and actively examines reported data to correspond with clinics to resolve potential discrepancies.
DEA methodology also has two inherent limitations. First, the frontier is very sensitive to outliers (Corredoira et al., 2011). Second, performance is a relative measurement, in that it is defined by other performers. DEA cannot compare FC performance with best practices or anticipated changes in technology that may affect input or output variables (Corredoira et al., 2011).
Conclusion
DEA analysis was used to assess the partial production efficiency of FCs in Virginia. Of the 48 Virginia FCs in 2010, 30 FCs were found to be efficient (62.5%), whereas 18 clinics (37.5%) lacked additional outputs needed to maximize performance. Information on individual clinic efficiency could be used in the future to improve the performance of this valuable component of the health care safety net. Furthermore, additional data are needed to fully utilize DEA methodology to understand FC efficiency. When these data become available to researchers, more complete results may be drawn that will describe this important contributor to the healthcare safety net. As healthcare reform implementation progresses, all safety net organizations including FCs are encouraged to examine how to maximize access and quality of services while utilizing fewer resources to provide health care to the remaining uninsured.
Footnotes
Acknowledgements
Thank you to Josh Jurrens for providing support in conceptualization and design of figures, and to Patrick Shay, Trinity University, for providing helpful comments and suggestions on the manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
