The Effects of Benchmarking of Public Works Performance Measures on the Cost-Effectiveness of Service Delivery

Abstract

This research is an empirical analysis of the effects of the benchmarking of Efficiency Performance Measures on total costs for six Public Works service delivery areas based on data over the period 1998 to 2011 from municipalities participating in the North Carolina Performance Measurement Project, a well-established and nationally recognized performance measurement and benchmarking system. The results of this research are significant in providing empirical evidence that the benchmarking of Efficiency Performance Measures for the six Public Works service delivery areas had modestly positive impacts on total cost-effectiveness in a significant percentage of these Public Works service delivery areas. The results reflect that management decision-making utilizing performance measures in a benchmarking context, especially in Public Works service delivery areas, can be effective and that performance measurement and benchmarking can achieve identifiable efficiencies and modest improvements in cost-effective outcomes in Public Works service delivery areas over time.

Keywords

performance benchmarking performance measurement public works performance measures Efficiency Performance Measures cost-effectiveness of public works service delivery

Introduction

Research about the impact of performance measurement on budgeting and management processes in local governments (Rivenbark & Kelly, 2006; Wang, 2002a, 2002b, 2008; Wang & Gianakis, 1999) has traditionally relied on survey research results based on the perceptions of local government officials (mostly city managers or finance/budget directors) about the use of performance information. Most research has found that performance measures have a somewhat more limited use in decision-making (especially by elected officials) than might be expected from the widespread use of performance measurement in local government (Rivenbark & Kelly, 2004, 2006). Wang (2002a) found that only about one third of respondents indicated they thought performance measures influenced budget allocation decisions. Proponents of performance measurement discount the limited use of performance measures by rationalizing that performance measures inform resource allocation decision-making because such information is available and can be used (Kelly & Rivenbark, 2011; Rivenbark, 2004, 2008; Rivenbark & Kelly, 2006). This general view of performance measurement is that performance measurement systems provide the basis for public managers to improve service delivery and cost-effectiveness of services, and elected officials are free to use performance information or their own political criteria in making judgments on resource allocation in the budgeting process.

There is relatively little empirical research examining the effect of performance information on resource allocation and budget change. Ho (2011) has found indications from a somewhat limited case assessment of budget changes over a 3-year period where performance measurement approaches were used in Indianapolis that performance measurement appears to be positively related to sub-departmental budget changes. Research findings have not been clear about the extent to which departments or programs have been successful in using performance measurement information and performance benchmarking effectively in managing service delivery and achieving increased cost-effective outcomes or in using performance benchmarking. Public Works service delivery areas associated with high, capital infrastructure costs and delivery of critical essential services are clearly areas where it would be important to know more about using performance measurement information more effectively.

This research is an empirical analysis of the relationship between performance measures and service delivery costs in six Public Works service delivery areas for the North Carolina municipalities involved in the North Carolina Local Government Performance Measurement Project (NCLGPMP). The NCLGPMP, also known as the North Carolina Benchmarking Project, is a nationally recognized benchmarking performance measurement project initiated in 1995 through the University of North Carolina (UNC) at Chapel Hill’s Institute of Government. The NCLGPMP allows local governments to assess service delivery and costs by compiling service and cost information, calculating selected performance measures, and comparing the results with other units and with their own operations over time. Over half of the service delivery areas included in the NCLGPMP are Public Works service delivery areas, including Residential Refuse Collection, Household Recycling, Yard Waste/Leaf Collection, Asphalt Maintenance & Repair, Fleet Maintenance, and Water Services. Because of the North Carolina Benchmarking Project’s longevity and reported successes (Ammons & Rivenbark, 2008; Rivenbark, 2008), it is a good population of municipalities from which to assess the impact of Efficiency Performance Measures and benchmarking of them over time associated with the NCLGPMP on the cost-effectiveness for Public Works service delivery areas as indicated by changes in total costs.

This study explores changes in performance measures and service delivery costs in the case of a well-established performance measurement and benchmarking system for a consortium of municipalities. The underlying premise for this study is that management improvements in service delivery areas as a result of applying performance measurement should result in cost savings that would be represented in changes in total costs. It empirically tests the effects Public Works Efficiency Performance Measures in a benchmarking context have had over an extended time period on total costs in Public Works service delivery areas. It evaluates the extent to which Public Works areas can achieve increased cost-effectiveness through the use of performance measurement and benchmarking.

Relevant Research on Local Government Performance Measurement and Its Use in Decision-Making

Performance measurement has become a steadily increasing trend at all levels of government since its resurgence in the 1990s. The premise of the performance movement is that performance-based approaches can be used to improve government performance and accountability to citizens. Performance measurement depends on the development of good measures in adopting a performance measurement system and then utilizing the resulting performance measures to improve the efficiency and effectiveness of service delivery. Performance management goes beyond adopting a performance measurement system to the utilization by managers of performance information derived from the performance measurement system to improve how actual decisions are made.

Even before 2000, substantial numbers of local governments were beginning to collect performance indicators (Cope, 1987; Poister & Streib, 1999). Although progress on performance measurement was being made by municipalities at this point, improvement was needed in development and reporting of measures and in using them effectively to improve management and decision-making (Poister & Streib, 1999). Our knowledge about performance measurement in local governments tends to be limited by availability of comprehensive and reliable data; thus, how much has been accomplished remains unclear with respect to the progress that has been made (Sanger, 2008). In spite of data issues, there is evidence that local governments in the last 15 years have wholeheartedly embraced performance measurement approaches to a substantial degree, but there have been conflicting conclusions about how widespread the use of performance measures is.

In trying to examine public officials’ attitudes toward, and motives for using, performance measurement, Wang (1999) found that the majority of respondents from among local government officials he surveyed believed that management demand for efficiency and effectiveness was the primary reason for using performance measurement rather than pressure to use performance measurement from elected officials. The lack of external demands from elected officials for the use of performance measurement activities meant that their use was often limited to performance reporting on service activities (Wang, 1999).

More recent evidence finds public managers use performance information because of their public service motivation, not just because they are pressured to do so. The use of performance information by public managers is also influenced by factors such as leadership/political support, information availability, goal-oriented organizational culture, administrative flexibility, and citizen support and involvement in the performance process (Moynihan & Panday, 2010). Wang (2000) found that public managers use performance measurement to better identify managerial and operational problems and to better develop solutions to these problems. Thus, the way public managers indicate they use performance measurement implies it has a substantial impact on internal management.

Performance measurement and benchmarking approaches have long been advocated in a range of municipal functions. Ammons (2012) believes performance measurement and monitoring systems are important for evaluation and benchmarking efforts because they offer objective information that can identify where adjustments are needed. He also sees them as important for supporting planning and budgeting systems and for performance reporting to insure internal and external accountability. While workload or output measures (indicating the amount of work performed or services provided) and productivity measures (combining dimensions of efficiency and effectiveness) are possible types of performance measures, Ammons generally advocates for efficiency (reflecting the relationship between work performed and the resources required to perform it) and effectiveness or outcome measures (indicating quality of performance or extent to which objectives have been met). When these types of measures are used in a benchmarking approach, the performance record is put in the context of comparison with standards or the performance achieved by others. Ammons describes three forms of benchmarking: comparison of performance statistics either of the municipality’s own performance to standards or statistics of other service providers; visioning benchmarks related to social indicators or desired community conditions as targets; and best practices benchmarking where cities compare themselves with an outstanding performer for a particular process and try to adapt superior practices for their own use. Ammons has provided benchmark data for a comprehensive range of municipal services focusing on benchmarking involving comparison of performance statistics with professional standards and the performance of respected municipalities on a range of efficiency and effectiveness measures.

Public Works service delivery areas have been the focus of benchmarking initiatives since the 1990s. Meszaeros and Owen (1997) outlined types of benchmarking approaches similar to those Ammons (2012) describes. They described a seven-step benchmarking model that provided a framework for developing a process benchmarking program for assessing municipal utility services and examined the application of competitive benchmarking in water and wastewater utility operations in the City of Fort Wayne, Indiana. The steps in their model involve identifying what to benchmark, determining what to measure, identifying who to benchmark, collecting data, analyzing the data and determining the gap on the benchmarked measure, setting goals and developing an action plan to close the gap, and monitoring the process toward achievement of that end result. Meszaeros and Owen describe competitive benchmarking as comparison with the best of direct competitors and focused on benchmarking against both private sector contractors, who contracted for privatization of services and a database of the performance by a range of other cities. Based on following the steps in the framework for a competitive assessment, the city found that there was a gap and that they could improve their operations and maintenance productivity levels, which would give flexibility to reduce costs, meet increasing service demands, improve customer satisfaction, and compete against private sector providers. The Fort Wayne case demonstrated that even limited comparison benchmarking can be effective in providing meaningful assessment of competitive position and in identifying performance gaps for improvement.

Frevert, Newfarmer, and Paul (1998) described another early initiative to use benchmarking in Public Works functions, but in this case, in a wider array of Public Works program areas by the Public Works Department of Kansas City, Missouri. They initiated a three-step approach involving developing performance measures and then conducting strategic and process benchmarking. After developing their own performance measures, the department compared them with data from other agencies. Then teams did site visits to benchmarking partner cities and identified work process improvements for inclusion in their department action plan to improve their own operations.

These examples show that performance measurement and benchmarking have been around for a long time and used innovatively in some instances for Public Works functions. Since the late 1990s, the use of performance measurement has continued to expand in a wide range of service delivery areas in local government. While there is evidence that performance measures are being collected and reported by many local governments, evidence indicating that performance measures are being used by public managers is much more limited. Nonetheless, evidence is beginning to develop that performance measures are utilized in some instances for internal management purposes (Ho, 2011), even if there is less evidence of their broader impacts from a macro perspective in performance-based budgeting. In spite of the seemingly widespread use of performance measurement by local governments, there has been relatively little evidence that the measures are actually used to influence decisions. An important assumption implied about the influence of performance measures on decisions (i.e., budgeted total costs) is that evidence of cost efficiencies in cost decisions can be construed to indicate that performance information is being used in whole or in part to make those cost decisions.

Conflicting views remain about the degree to which performance measurement systems produce performance measures useful in management of service delivery. It is unclear whether performance measures have any effect in steering service delivery toward improved efficiency and helping management to create cost savings. Even more important from a budgetary perspective is the lack of evidence that performance measurement creates cost savings that affect resource allocation and budgetary outcomes in functional budgets related to service delivery.

The NCLGPMP is an example of an effort that provides evidence that there are instances in which performance measures are being used in decision-making. In the NCLGPMP, comparative performance measurement is being utilized by public managers to foster improvement in decisions about service delivery (Ammons & Rivenbark, 2008). Ammons and Rivenbark (2008) reported that performance data from the cities in the project is being successfully incorporated into key management systems. Project officials in these cities indicated that the collection of higher order efficiency measures rather than simply workload or output measures made them more likely to use the performance data for operational decisions. They reported that the use of performance data was also influenced by the willingness of officials to utilize benchmarking comparison with other governments, a feature highlighted by the project (Ammons & Rivenbark, 2008).

This research evaluates how changes in Efficiency Performance Measures for Public Works service delivery areas resulting from using the benchmarking approach over time in the NCLGPMP, a well-established performance measurement system, are linked to changes in total costs and improving cost-effectiveness for Public Works service delivery areas included in this study. Cost-effectiveness is defined in terms of this linkage as economical in terms of service provided for dollars spent. This is the result of managerial decisions that improve Efficiency Performance Measures resulting in declining total costs. Therefore, decreasing total costs are appropriate as a measure of cost-effectiveness because declines in total costs are indicative of the cost savings resulting from improvements in Efficiency Performance Measures. The underlying premise for this study is that management improvements in service delivery areas as a result of applying comparative benchmarking of Efficiency Performance Measures in the NCLGPMP result in cost savings that are represented in declines in total costs.

Data and Research Methodology

The source of data on performance measures and total costs for Public Works service delivery areas included in the study is the annual report data from the NCLGPMP and the separate Final Reports on City Services for Fiscal Years 1998 to 2011. Performance and cost data provide the basis for evaluating performance measures and total cost patterns for six Public Works service delivery areas, including Residential Refuse Collection, Household Recycling, Yard Waste/Leaf Collection, Asphalt Maintenance & Repair, Fleet Maintenance, and Water Services.

NCLGPMP

The 17 cities in North Carolina included in the study are part of the nationally recognized benchmarking performance measurement project developed through UNC Chapel Hill’s Institute of Government. The NCLGPMP was initiated in September 1995, providing a comparative basis for local governments to assess service delivery and costs. It allows municipalities to compare themselves with other units and with their own operations over time. The benchmarking process includes compiling service and cost information, calculating selected performance measures, and comparing the results. The project uses a methodology that involves developing service profiles, performance measures, a cost accounting approach for capturing the full cost of service delivery, and explanations for results. This provides a basis for a comprehensive source of information that allows participants to compare service delivery and costs between localities in performance measurement and benchmarking. Participating municipalities use the performance data collected for service improvement. The project has been successful because it has developed consensus on specific service definitions and measurement statistics among participating municipalities. This process requires significant involvement from government officials in participating municipalities and extensive consultation, training, and technical assistance from the UNC School of Government. The project has produced accurate, reliable, and comparable performance and cost data, which are used for service improvement (North Carolina Benchmarking Project Website, http://www.sog.unc.edu/node/909).

Rivenbark (2008) cites the success of some of these North Carolina municipalities in moving to performance-based budgets. Ammons and Rivenbark (2008) reported that evidence from the NCLGPMP indicates performance data from the cities in the project is being successfully incorporated into key management systems, whereas efforts by other local governments nationally often show meager use of performance measures to improve service delivery. Consequently, the North Carolina municipalities involved in the project are a useful population of municipalities from which to assess whether performance measures are having an impact on service delivery cost-effectiveness because their performance measurement and benchmarking system has been well developed in terms of performance measures for more than 14 years. Consequently, if performance measurement has an impact on managerial and budgetary decision-making and cost savings, it would be evident in these cities.

Municipalities in North Carolina and service areas included in the NCLGPMP are shown in Tables 1 and 2, respectively.

Table 1.

List of Cities Included in the North Carolina Benchmarking Project and City Population (2005).

Cities	Population
Asheville	73,189
Burlington	47,295
Carrboro	17,797
Cary	115,967
Charlotte	640,270
Concord	63,429
Durham	209,123
Gastonia	70,243
Greensboro	237,316
Hickory	39,018
High Point	92,491
Matthews	25,442
Raleigh	338,357
Salisbury	29,058
Wilmington	97,135
Wilson	47,815
Winston-Salem	198,593

Source. North Carolina Benchmarking Project Website.

Table 2.

List of Service Areas Included in Project.

Residential Refuse Collection

Household Recycling

Yard Waste/Leaf Collection

Police Services

Emergency Communications

Asphalt Maintenance & Repair

Fire Services

Building Inspections

Fleet Maintenance

Human Services

Water Services

Source. North Carolina Benchmarking Project Website.

Although 17 North Carolina municipalities currently participate, municipalities participating in the project have changed over the course of the project. Five municipalities, which are not in the list above, participated briefly for several years and were included in the data analyzed. Some of the 17 cities listed did not participate in the project on a continuous basis during the entire period included in this research. In addition, reporting of service areas varies in both when performance and cost data collection was initiated and in participation by individual municipalities in the list. In this research, the results of the project’s performance measurement system over 14 years (1998-2011) is the basis for testing the effects of the selected Efficiency Performance Measures on total costs (in constant dollars) over the period in Public Works service delivery areas included in the study.

Service area codes for Public Works service delivery areas are as follows: Residential Refuse Collection (RR), Household Recycling (RS), Yard Waste/Leaf Collection (YW), Asphalt Maintenance & Repair (AM), Fleet Maintenance (FM), and Water Services (WS). Total cost codes (in constant dollars) are as follows: RR_TC, RS_TC, YW_TC, AM_TC, FM_TC, and WS_TC. Efficiency performance measures by Public Works service delivery area include the following: Service area codes for Public Works service delivery areas and number codes for specific individual Efficiency Performance Measures in constant dollars, except for specific Efficiency Performance Measures not in dollars (RR7, RS8, YW7, FM8, WS7, and WS8).

Research Methodology

The initial analysis in this research focuses on evaluating trend patterns in total costs for the six Public Works service delivery areas included in the project and the associated Efficiency Performance Measures for each service area as indicated in Figure 1. Patterns are evaluated using descriptive analysis based on patterns of mean annual total cost values for each of the six Public Works service delivery areas and patterns of the average annual values for each of the 18 associated Efficiency Performance Measures. Analyzing the patterns of average total costs and average efficiency measures allows the assessment of how well continued benchmarking activities over time for Efficiency Performance Measures achieved changes in total costs, indicating their usefulness in having a positive effect in steering Public Works service delivery areas toward improved efficiency and helping management to create cost savings.

Figure 1.

Efficiency Performance Measures by Public Works Service Delivery Areas.

The analysis of aggregate trends in total cost and Efficiency Performance Measures provides a basis for further examination of total costs and Efficiency Performance Measures. The principal focus is to evaluate the effect that Efficiency Performance Measures have had on cost-effectiveness for the service delivery areas, as evidenced by patterns of total costs for Public Works service delivery areas included. To accomplish this research objective, pooled cross-sectional time-series regression analysis is used in this study. Pooled cross-sectional time-series captures variations across time and units. In using panel data in this way, the number of available observations is increased. The available number of observations for 17 municipalities over 14 years results in a potential total of 238 observations. Of course, the actual available observations for individual service areas in the project falls short of that total due to non-continuous and inconsistent participation and varying years for initiating data collection in the project. Several models of pooled regression are available, and the choice of the appropriate model is contingent on the results from analyzing the data-error term. For this study, a fixed effect model was used to control both unit-specific (municipality) and time-specific (year) effects in error terms by using a fixed effects estimator. Since total costs by service area and municipality may be affected by unobservable city-specific and time-specific events, the model is expected to capture these variations. The fixed effect model of the Xtreg (Longitudinal-Data/Panel-Data) procedure of the Stata program was used to conduct the statistical analysis. The dependent variable for analysis is total costs in constant dollars for each Public Works service delivery area over the period and indicates changes in cost-effectiveness over time in that service delivery area for each municipality. The independent variables for each service delivery area are the Efficiency Performance Measures (in constant dollars where appropriate) associated with each Public Works service delivery area in the project. The pooled cross-sectional time-series models developed predict the effects of Efficiency Performance Measures for each Public Works service delivery area on total costs for that service delivery area. Pooled cross-sectional time-series models allow for the effect of predictor variables on the dependent variable to be determined using variables in the model. The fixed effect models used in this research are useful in taking confounding factors into account, especially ones that are time constrained. The current research is an exploratory study trying to determine whether benchmarking of Efficiency Performance Measures, which results in changes in those performance measures, has an effect in terms of predicting total costs. Consequently, a model including just Efficiency Performance Measures serves to indicate whether there is a relationship between benchmarking of Efficiency Performance Measures and total costs.

Findings

Table 3 provides descriptive statistics for the variables (in constant dollars except for RR7, RS8, YW7, FM8, WS7, and WS8) included in the study. City Identification Number (CITYID) indicates that the 17 municipalities over 14 years plus additional observations included for five other municipalities that participated for several years, but still had only limited participation, produce 308 observations. As can be seen from the number of observations for total cost variables and individual Efficiency Performance Measures by service delivery area, there are significant variations in the level of participation across years for municipalities and service delivery areas. In addition, the standard deviations and minimum and maximum indicate fairly wide variations in the values for the dependent and independent variables across municipalities.

Table 3.

Summary Descriptive Statistics for Variables Utilized (in Constant Dollars Except for RR7, RS8, YW7, FM8, WS7, and WS8).

Variable	Observations	M	SD	Minimum	Maximum
CITYID	308	11.48701	6.374049	0	22
RR_TC	202	1,743,960	1,488,907	104,138.4	6,697,983
RR5	203	48.29961	16.88073	18.95	117.65
RR6	203	41.96251	14.22464	11.26	99.75
RR7	185	1,329.892	829.5442	312	3,946
RS_TC	192	819,190.7	758,229.7	51,182.64	3,386,188
RS6	193	108.5725	44.93778	36.53	302.75
RS7	193	19.2086	8.37027	4.89	79.14
RS8	86	626.0349	501.7685	145	2,848
YW_TC	199	921,186.1	780,914.1	54,809.12	3,948,063
YW5	201	25.71458	9.251604	9	57.62
YW6	201	64.25552	22.3971	18.4	144.79
YW7	197	623.3858	222.0423	263	1,767
AM_TC	176	2,123,673	3,257,496	43,322.67	1.65e+07
AM6	168	920.7444	1,042.72	45	11,651.61
AM7	146	24,260.46	11,477.35	4,869.37	82,427.45
AM8	143	38.58448	16.88326	2.22	92.68
FM_TC	152	1,777,855	1,851,918	172,583.4	1.24e+07
FM6	151	187.1607	60.70465	73.31	343.53
FM7	114	447.7521	110.9024	34.24	666.53
FM8	114	0.7117544	0.1125508	0.44	0.95
WS_TC	61	1.03e+07	9,087,440	285,974	3.28e+07
WS6	61	1.579508	1.688918	0.56	14.12
WS7	61	57.50328	15.99594	12.1	91
WS8	60	0.8338333	0.0885302	0.62	0.98

The descriptive analysis of total costs by Public Works service delivery areas in constant dollars utilizes mean values for total costs in constant dollars by year as depicted in Table 4. The table shows that average total costs were not computed for all years for two of the six Public Works service delivery areas because data collection by the benchmarking project for these service delivery areas began at a later point in the time period covered. Table 4 indicates trends in the values of average total costs in constant dollars for individual Public Works service delivery areas and includes overall change and percentage change values for each of the total cost service delivery areas. Table 4 reflects the evaluation of the trends for average total costs in constant dollars for individual Public Works service delivery areas in the columns on the far right of the table. A positive trend for the total cost variables would indicate declining values for total costs for a service delivery area representing cost savings for the service delivery area as reflected in decreased total costs or budgetary outcomes. At the least, total costs should not reflect an increasing trend. By these standards, half of the Public Works service delivery areas reflect generally positive decreasing trends in average total costs while the others are either inconsistent, or reflect an increasing trend. Average total costs for Residential Refuse Collection reflects a positive overall decreasing trend, and the trends for two other average total cost variables, Fleet Maintenance and Water Service, are inconsistent but generally positive in that they are overall decreasing. Average total costs for Household Recycling and Asphalt Maintenance & Repair are inconsistent and increasing or decreasing in different years over the period. The average total cost trend for Yard Waste/Leaf Collection clearly indicates an overall increasing trend in average total costs, which should be evaluated as a negative trend in terms of reflecting an undesirable performance outcome.

Table 4.

Average Total Costs in Constant Dollars by Public Works Service Delivery Area and Year.

TC VARs	1998	1999	2000	2001	2002	2003	2004	2005	2006	2007	2008	2009	2010	2011	Overall change	Percent change	Evaluation of trends:
RRTCCD	1,788,383	1,655,965	1,844,826	1,811,729	1,980,324	1,955,733	1,894,641	1,676,038	1,674,047	1,626,420.969	1,701,067.275	1,749,102.376	1,657,586.321	1,490,626	−297,757	−16.65%	Overall decreasing +
RSTCCD	665,443	765,545	850,228	831,695	850,515	792,299	775,411	828,180	871,571	844,214.8118	899,379.572	930,555.452	818,012.7713	678,256	12,813.22	1.93%	inconsistent ±
YWTCCD	736,490	811,400	860,353	893,841	992,143	954,872	927,907	851,952	904,877	920,780.1887	959,450.3154	1,039,902 965	978,777 8631	987,936	251,446	34.14%	Overall increasing −
AMTCCD	1,715,292	675,209	2,116,995	2,845,538	3,164,564	2,300,948	2,038,706	1,836,021	1,771,355	1,960,941.765	2,040,173.645	1,933,498.942	2,409,264.822	1,958,878	243,586	14.20%	Inconsistent ±
FMTCCD					1,713,862	1,508,814	2,485,684	1,609,282	1,648,029	1,728,584.203	1,709,848.911	1,883,080.664	1,824,879.028	1,707,122	−6,739.84	−0.39%	Inconsistent, overall decreasing ± +
WSTCCD										10,259,903.93	10,440,334.36	10,722,487.28	11,675,257.06	8,527,136	−1,732,768	−16.89%	Inconsistent, overall decreasing ± +

Note. TC = total cost; VARs = Variables; RRTCCD = Residential Refuse total costs in constant dollars; RSTCCD = Household Recycling total costs in constant dollars; YWTCCD = Yard Waste/Leaf Collection total costs in constant dollars; AMTCCD = Asphalt Maintenance & Repair Total Costs in constant dollars; FMTCCD = Fleet Maintenance total costs in constant dollars; WSTCCD = Water Services total costs in constant dollars.

The descriptive analysis of 18 Average Efficiency Performance Measures for Public Works service delivery areas by year (in constant dollars where appropriate) utilizes mean values for individual Efficiency Performance Measures as depicted in Table 5. The benchmarking project has collected data on three Efficiency Performance Measures for each of the Public Works service delivery areas included in the study as indicated in the listing in the previous section. Again Efficiency Performance Measures were not included in Table 5 for all measures in all years, reflecting non-continuous data collection. A positive trend for Efficiency Performance Measures is not as easily determined as it is for average total costs for a service delivery area. Whether the trend for an individual Efficiency Performance Measures represents a positive trend depends on the nature of the individual Efficiency Measure. A majority of the Efficiency Performance Measures (12 out of 18) are unit cost measures in which a positive trend would indicate a declining unit cost as reflecting improving efficiency and cost savings for the service delivery area. The other six have to be interpreted individually with respect to their potential impact on efficiency or improving service delivery, but increasing trends for each of them reflects a positive trend for the measure. Table 5 includes overall change and percentage change values for each Efficiency Performance Measure over the period. Table 5 reflects the evaluation of the trends for Average Efficiency Performance Measure values (in constant dollars except for RR7, RS8, YW7, FM8, WS7, and WS8, which are not in dollars) for individual Public Works service delivery areas in the column on the far right of the table. As the evaluation of trends in Table 5 indicates, five Public Works Efficiency Performance Measure trends out of 18 are clearly positive: three out of three measures for Residential Refuse Collection (RR5CD, RR6CD, RR7), one of three measures for Household Recycling (RS7CD), and one of three measures for Yard Waste/Leaf Collection (YW7). Seven Efficiency Performance Measures have inconsistent trends that are nonetheless positive in their overall tendencies: one measure in Household Recycling (RS6CD), one in Yard Waste/Leaf Collection (YW6CD), one in Asphalt Maintenance & Repair (AM6CD), one in Fleet Maintenance (FM8), and three in Water Service (WS6CD, WS7, and WS8). Five measures have trends that are inconsistent but overall generally negative in their trends: one out of three in Recycling (RS8); one out of three in Yard Waste/Leaf Collection (YW5CD); two out of three in Asphalt Maintenance (AM7CD and AM8CD); and one out of three in Fleet Maintenance (FM7CD). Only one Efficiency Performance Measure is clearly negative in its trend: one of three in Fleet Maintenance (FM6CD). Overall 5 out of 18 Efficiency Performance Measures in Public Works service delivery areas are clearly positive, while an additional seven measures are more positive than negative in reflecting efficiency improvements or impacts on cost savings. While nearly two-thirds of all Efficiency Performance Measures (12 out of 18) are inconsistent in their trends, only five were more negative than positive in their overall trends, and only one measure was clearly negative. These trends tend to reflect well on the Public Works service delivery areas in their successful use of Efficiency Performance Measures from the NCLGPMP in achieving desirable improvements in unit cost and other efficiency measures, especially in comparison with other service delivery areas included in the benchmarking project.

Table 5.

Average Efficiency Performance Measures by Public Works Service Delivery Area and Year.

Year	1998	1999	2000	2001	2002	2003	2004	2005	2006	2007	2008	2009	2010	2011	Overall change	% Change	Trend evaluation
RR5CD	54.53988	60.85511	54.21437	55.41663	52.42229	46.66149	49.05594	47.39503	44.45685	43.20795	41.68544	45.54267	43.20646	43.75789	−10.782	−19.77	Decreasing
RR6CD	47.30061	49.035	47.95089	47.91482	45.53812	42.04193	42.84454	42.75474	38.69048	37.44878	34.86366	40.49761	37.89983	37.56574	−9.73487	−20.58	Decreasing
RR7	979.5	1,155.667	1,329.692	1,217.923	1,333.25	1,437.692	1,305.385	1,304.214	1,364.714	1,466.8	1,439	1,408.625	1,324.846	1,406.333	426.8333	43.58	Increasing
RS6CD	113.681	90.03601	100.2226	102.1551	101.2101	104.0761	98.60596	104.2499	114.8148	123.2866	112.5546	124.8036	109.5721	112.7925	−0.8885	−0.78	Inconsistent ±
RS7CD	24.47853	18.7575	19.26055	18.49238	18.21525	17.85714	18.10482	19.09438	19.39782	19.10672	19.2795	20.62465	18.63578	18.77493	−5.7036	−23.30	Decreasing
RS8	735.5	847.5	631.6667	588.6667	552.6667	576.3333	580.3333	527.8333	580.3333	546.7143	584.7143	495.25	681.625	990.8333	255.3333	34.72	Inconsistent ±
YW5CD	21.53374	24.60984	25.63464	26.86134	25.65528	28.06677	25.33969	25.95366	25.23562	23.81331	24.4423	28.78977	26.43489	26.13407	4.600327	21.36	Inconsistent ±
YW6CD	72.51534	58.92357	62.51037	67.95999	69.61132	63.19876	58.40833	63.71608	60.29886	62.36677	70.94653	67.64225	60.0108	63.09647	−9.41887	−12.99	Inconsistent ±
YW7	497.9	572.6364	638.2143	568.4286	548.2308	706.7143	662.7143	610.5333	641.9333	624.625	547.0625	665.3529	653.8571	738.4286	240.5286	48.31	Increasing
AM6CD		1,979.892	964.8664	1,752.917	1,214.614	783.7862	720.7314	728.9981	727.1495	852.185	793.1264	803.5619	844.3113	937.9686	−1,041.92	−52.63	Inconsistent/decreasing ±
AM7CD		44,311.52	22,445.03	19,562.63	19,001.21	20,475.64	21,138.25	23,329.17	25,023.28	24,037.16	26,503.21	28,950.75	32,377		−11,934.5	−26.93	Inconsistent/decreasing ±
AM8CD		37.81513	28.3585	34.44382	31.79544	30.43478	32.86571	40.52374	44.82001	43.51784	42.98057	43.34917	46.4101	40.58878	2.773656	7.33	Inconsistent/overall increasing ±
FM6CD					165.727	154.5807	160.2802	176.0113	182.9427	191.925	197.6285	222.0377	199.7521	211.0096	45.2826	27.32	Increasing
FM7CD					35.57532	34.23913	225.5161	456.5412	449.2808	461.9535	470.4126	472.7006	452.3085	448.6931	413.1178	1,161.25	Inconsistent/steady ±
FM8					0.688182	0.712727	0.726	0.705833	0.7175	0.691667	0.681667	0.681667	0.759	0.761667	0.073485	10.68	Inconsistent/overall increasing ±
WS6CD										1.268436	2.302301	1.431347	1.396222	1.44	0.17	13.38	Inconsistent/steady ±
WS7										59.675	56.48462	53.33846	58.91818	59.65	−0.03	−0.04	Inconsistent ±
WS8										0.8425	0.835385	0.837692	0.819091	0.832727	−0.01	−1.16	Overall decreasing slightly ±

To determine the specific effects of Efficiency Performance Measures for each Public Works service delivery area (in constant dollars, except for RR7, RS8, YW7, FM8, WS7, and WS8, which are not in dollars) on the total costs in constant dollars in their service areas, and to determine models predicting total costs in constant dollars for each of the six Public Works service delivery areas included in this study, pooled cross-sectional time-series regression analysis was undertaken. The results of the pooled cross-sectional, time-series regressions of each service area’s Efficiency Performance Measures (in constant dollars, except for RR7, RS8, YW7, FM8, WS7, and WS8, which are not in dollars) on the total costs in constant dollars in the respective service areas is shown in the results in Table 6.

Table 6.

Pooled Cross-Sectional Time Series Regressions of Total Costs in Public Works Service Delivery Areas by Efficiency Performance Measures.

	Standardized Xtreg regression coefficients
	RR_TC	RS_TC	YW_TC	AM_TC	FM_TC	WS_TC
Total cost variables by Public Works service delivery area	n = 183	n = 86	n = 194	n = 132	n = 85	n = 60
Efficiency performance measures by Public Works service delivery area:
Residential Refuse Collection cost per ton collected (RR5)	1,855.83
Residential Refuse Collection cost per collection point (RR6)	19,313.72**
Refuse tons collected per municipal collection FTE (RR7)	−194.4**
Constant.	1,208,625
F(3, 162)	22.21***
R²	.2915
Recycling Services cost per ton collected (RS6)		3,631.3
Recycling cost per collection point (RS7)		32,707.03**
Tons collected curbside per municipal collection FTE (RS8)		50.97
Constant.		85,113.55
F(3, 74)		25.72***
R²		.5104
Yard Waste and Leaf Collection cost per collection point (YW5)			14,703.32***
Yard Waste and Leaf Collection cost per ton collected (YW6)			2,057.7
Yard Waste and leaf tons collected per collection FTE (YW7)			126.8
Constant.			352,303.4
F(3, 172)			10.26***
R²			.1518
Cost of maintenance per lane mile maintained (AM6)				789.49***
Cost per lane mile resurfaced (AM7)				−11.2
Cost per ton for contract resurfacing (AM8)				6,911.3
Constant.				1,670,030
F(3, 112)				73.19***
R²				.6622
Fleet Maintenance cost per work order (FM6)					8,996.98
Fleet Maintenance cost per vehicle equivalent unit (FM7)					−7,245.67***
Hours billed as a percentage of total hours (FM8)					−4,281.6
Constant.					3,600,367
F(3, 67)					5.27**
R²					.1909
Total cost per thousand gallons of billed water (WS6)						−25,751
Millions gallons of billed water per water services FTEs (WS7)						−57,123.6
Billed water as a percentage of finished water (WS8)						−5.06E+07***
Cons.						5.60E+07
F(3, 43)						5.72**
R²						.2852

Note. Variables are in constant dollars except for RR7, RS8, YW7, FM8, WS7, and WS8, which are not in dollars.

p > F or (t) significant at .05. **p > F or (t) significant at .01. ***p > F or (t) significant at .001.

Of the six pooled cross-sectional time-series regressions for the six Public Works service delivery areas, Table 6 indicates that the resulting equations for Residential Refuse Collection (RR), Household Recycling (RS), Yard Waste/Leaf Collection (YW), Asphalt Maintenance & Repair (AM) have statistically significant F values with p > T = .000, which indicates that the overall models predicting the effects of Performance Efficiency Measures on total costs in constant dollars in these Public Works service delivery areas are statistically significant. In addition, the resulting equations for the other two Public Works service delivery areas are not statistically significant at p > F = .000, but they have p values for their F Statistics, which indicate that the overall models are still statistically significant at p > F for Fleet Maintenance (FM) equal to .0025 and Water Services (WS) equal to .0022. For the six Public Works service delivery areas, the F values for the equations indicate that the equations are statistically significant and that the models overall are statistically significant in terms of the amount of variance explained.

In the Residential Refuse (RR) model, predicting RR total costs in constant dollars (RRTCCD), two Efficiency Performance Measures, collection cost per collection point (RR6CD) and refuse tons collected per municipal collection Full-Time Employee (FTE) (RR7), have coefficients which are significant in the equation at the .01 level. Both RR7 and RR5CD (not significant) have a negative sign. The negative sign for the coefficient for RR7 indicates that as RR7 is increasing overall, RRTCCD decrease. The coefficient for RR6CD has a positive sign indicating a positive effect on RRTCCD and suggesting that a dollar increase in collection cost per collection point results in an average increase in RRTCCD of US$19,314. For Xtreg models using the fixed effects estimator, the within R² is interpretable as an ordinary R². The RRTCCD model has a within R² of .2915 and thus, the Efficiency Performance Measures can be interpreted to explain 29.2% of variance in total costs for RRTCCD.

Only recycling cost per collection point in constant dollars (RS7CD) has a significant coefficient at the .01 level in the model predicting recycling total costs in constant dollars (RSTCCD). Other efficiency measures variables do not have statistically significant coefficients at the .05 level. The coefficient for RS7CD has a positive sign indicating a positive effect on RSTCCD and suggesting that a dollar increase in RS7CD results in an average increase in RSTCCD of US$32,707. The within R² for the RSTCCD model is .5104 and thus, primarily RS7CD helps to explain 51% of the variance in RSTCCD.

For the Yard Waste/Leaf Collection (YW) model, only Yard Waste and Leaf Collection cost per collection point (YW5CD) has a significant coefficient in the model predicting YW total costs in constant dollars (YWTCCD) at p < .001. The coefficient for YW5CD indicates a positive effect on YWTCCD and suggests that an increase in YW5CD results in an average increase of US$14,703 in YWTCCD. The within R² for the YWTCCD model is .1518, which indicates that the YW Efficiency Performance Measures explain 15% of the variance in YWTCCD.

In the model predicting Asphalt Maintenance (AM) total costs in constant dollars, only cost of maintenance per lane mile maintained in constant dollars (AM6CD) is statistically significant in the equation at p < .001. The coefficient of AM6CD is positive and indicates an increase in AM6CD results in an average increase of US$789 in AM Total Costs in constant dollars (AMTCCD). The within R² for the model is .6622 and the Efficiency Performance Measures, primarily AM6CD, explains 66% of the variance in AMTCCD.

For the model predicting Fleet Maintenance (FM) total costs in constant dollars (FMTCCD), Fleet Maintenance cost per vehicle equivalent unit (FM7CD) is statistically significant in the equation at p < .001. The coefficient for FM7CD has a negative sign and implies a US$1 increase in maintenance cost per vehicle equivalent unit results in a US$7,246 decrease in FM total costs in constant dollars. The within R² for the model is .1909, indicating that it explains 19% of the variance in FMTCCD.

The overall results of the pooled cross-sectional time-series analysis for the Public Works service delivery areas included in the study indicates that the Public Works Efficiency Performance Measures are significant in predicting total costs in all six Public Works service delivery areas. Each Public Works service delivery area regression model has at least one significant Efficiency Performance Measure predicting total costs in that service delivery area. In the six Public Works Service delivery areas, there are two fairly strongly predictive models for Asphalt Maintenance and Recycling, two moderately predictive models for Residential Refuse Collection and Water Services, and two very modestly predictive models for Yard Waste and Leaf Collection and Fleet Maintenance. Thus, the Efficiency Performance Measures in the Public Works service areas examined are able to predict total costs to varying degrees depending on the Public Works service delivery area. Most of the models are dependent on one primary Efficiency Measure, which is often a unit cost measure. The results indicate that Public Works Efficiency Performance Measures have a significant effect on total costs in most Public Works service delivery areas.

Summary and Conclusion

This research has examined the relationship between performance measures and service delivery costs for selected municipalities in North Carolina that participated in the North Carolina Performance Measurement Project, a well-established performance measurement system, which encourages the use of benchmarking for a range of different types of measures, including Efficiency Performance Measures, which were the focus of this analysis. This study investigated whether changes in performance measures for selected municipal service areas in these municipalities resulted in changes in service delivery costs in the selected municipalities. There have long been conflicting views about the degree to which performance measurement systems produce performance measures useful in management of service delivery and whether performance measures have any effect in steering service delivery toward improved efficiency and helping management to create cost savings affecting resource allocation and budgetary outcomes in functional areas of the budget. The underlying premise for this study is that management improvements in service delivery areas as a result of applying performance measurement should result in cost savings that would be represented in changes in total costs.

The study examined the total costs and Efficiency Performance Measures from an overall aggregate perspective using data from annual performance and cost reporting for the project. Descriptive analysis was initially used to examine patterns in average total costs (in constant dollars) for the six Public Works service areas and the associated Efficiency Performance Measures (in constant dollars except for RR7, RS8, YW7, FM8, WS7, and WS8) included in the project’s performance measurement system to determine how patterns compared with those expected for improving efficiency and achieving cost savings. The results from the descriptive analysis of average total cost and average Efficiency Performance Measures (all in constant dollars) reflect cost savings in total costs for half of the Public Works service delivery areas and clear or substantial efficiency improvements in their associated Efficiency Performance Measures in up to 12 out of 18 Efficiency Performance Measures for Public Works service delivery areas, which is a good reflection of improved cost-effectiveness for these service delivery areas.

To determine the specific effects of Efficiency Performance Measures for each Public Works service delivery area (in constant dollars except for RR7, RS8, YW7, FM8, WS7, and WS8) on the total costs in constant dollars in their service areas, and to determine models predicting total costs in constant dollars for each of the six Public Works service delivery areas included in this study, pooled cross-sectional time-series regression analysis was undertaken. The overall results of the pooled cross-sectional time-series analysis indicated that Efficiency Performance Measures (in constant dollars except for RR7, RS8, YW7, FM8, WS7, and WS8) are significant in predicting total costs (in constant dollars) in the six Public Works service delivery areas included in the study, but the amount of explained variance differs by Public Works service delivery area. Efficiency Performance Measures fairly strongly predicted total costs in Recycling and Asphalt Maintenance, moderately predicted total costs in Residential Refuse Collection and Water Services, but only modestly predicted total costs in Yard Waste and Fleet Maintenance. Thus, the results of the pooled cross-sectional time-series analysis indicated that Efficiency Performance Measures had a significant effect on total costs in about two-thirds of the Public Works service delivery areas.

This research examined the effect of the benchmarking of Efficiency Performance Measures on total costs for the Public Works service delivery areas analyzed. It does not rely on survey research, but rather analyzed aggregate reporting over an extended time period on Efficiency Performance Measures and total costs from the NCLGPMP. While the results reflect the impacts in the case of the North Carolina municipalities participating in the NCLGPMP, the results are not necessarily generalizable to other states and municipalities. Nonetheless, the results are promising in indicating that a comparative benchmarking approach to the use of Efficiency Performance Measures for management improvement of Public Works service delivery areas can have modest but significant impacts in cost savings in total costs. The results of this research are significant in providing empirical evidence of the overall impact that benchmarking of Efficiency Performance Measures had on total costs for some Public Works functional service areas in a well-established performance measurement and benchmarking system. The results reflect that the benchmarking of Efficiency Performance Measures for each of six Public Works service delivery areas had modestly positive impacts on total cost-effectiveness in a significant percentage of these Public Works service delivery areas. The results reflect that management decision-making utilizing performance measures in a benchmarking context, especially in Public Works service delivery areas, can be effective. In addition, the results indicate that performance measurement in a well-established performance measurement and benchmarking system has the potential to be integrated into management systems in a way that can achieve identifiable efficiencies and modest improvements in cost-effectiveness outcomes in Public Works service delivery areas over time.

The results of this research indicate that comparative benchmarking of performance measures can have positive effects on the cost-effectiveness of Public Works service delivery areas. It requires the development of the accurate, reliable, and comparable performance and cost data on Public Works service delivery areas similar to that discussed in the case of the NCLGPMP. The implications of the results of this research are that such performance information can be used for service improvements producing cost savings when it is used in a comparative benchmarking framework like the NCLGPMP. The development of the North Carolina’s benchmarking project required time-consuming performance measurement and cost accounting, which has its own costs and trade-offs. The evidence from the NCLGPMP’s comparative benchmarking approach indicates that the use of Efficiency Performance Measures in managerial decision-making to achieve improvements in Public Works service delivery areas can have positive impacts on cost-effectiveness in terms of total costs. More widespread use of such performance measurement approaches in Public Works service delivery areas are needed to further assess the potential of using performance benchmarking in improving service delivery and achieving cost-effectiveness.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biography

Kenneth A. Klase is an associate professor in the Department of Political Science at the University of North Carolina at Greensboro. His recent research focuses on the effects on state and local budgeting of performance measurement and cutback management.

References

Ammons

D. N.

(2012). Municipal benchmarks: Assessing local performance and establishing community standards (3rd ed.). Armonk, NY: M.E. Sharpe.

Ammons

D. N.

Rivenbark

W. C.

(2008). Factors influencing the use of performance data to improve municipal services: Evidence from the North Carolina benchmarking project. Public Administration Review, 68, 304-318.

Cope

G. H.

(1987). Local government budgeting and productivity: Friends or foes? Public Productivity Review, 19(3), 45-57.

Frevert

Newfarmer

Paul

A. C.

(1998). Kansas City’s public works benchmarking experience. Public Management, 80(8), 12-16.

A. T.

(2011). PBB in American local governments: It’s more than a management tool. Public Administration Review, 71, 391-401.

Kelly

Rivenbark

W. C.

(2011). Performance budgeting for state and local government (2nd ed.). Armonk, NY: M.E. Sharpe.

Meszaeros

Owen

C. J.

(1997). Improving through benchmarking: A comparative assessment of municipal utility services. Public Works Management & Policy, 2, 11-23.

Moynihan

D. P.

Panday

S. K.

(2010). The big question for performance management: Why do managers use performance information? Journal of Public Administration Research and Theory, 20, 849-866.

Poister

Streib

(1999). Performance measurement in municipal government: Assessing the state of the practice. Public Administration Review, 59, 325-335.

10.

Rivenbark

W. C.

(2004). Defining performance budgeting for local government. Popular Government, 69, 27-36.

11.

Rivenbark

W. C.

(2008). Performance budgeting in local government. In Redburn

Stevens

R. J. S.

Buss

T. F.

(Eds.), Performance management and budgeting: How governments can learn from experience (pp. 135-148). Armonk, NY: M.E. Sharpe.

12.

Rivenbark

W. C.

Kelly

(2004). Performance budgeting in federal, state & local government. Journal of Government Financial Management, 53, 50-58.

13.

Rivenbark

W. C.

Kelly

(2006). Performance budgeting in municipal government. Public Performance & Management Review, 30, 35-46.

14.

Sanger

M. B.

(2008). From measurement to management: Breaking through barriers to state and local performance [Special issue]. Public Administration Review, 68, S70-S85.

15.

Wang

(1999). Conditions to implement outcome-oriented performance budgeting: Some empirical evidence. Journal of Public Budgeting, Accounting & Financial Management, 11, 533-552.

16.

Wang

(2000). Performance measurement in budgeting: A study of county governments. Public Budgeting & Finance, 20, 102-118.

17.

Wang

(2002a). Assessing performance measurement impact: A study of U.S. local governments. Public Performance & Management Review, 26, 26-43.

18.

Wang

(2002b). Perception and reality in developing an outcome performance measurement system. International Journal of Public Administration, 25, 805-829.

19.

Wang

(2008). Convincing legislators with performance measures. International Journal of Public Administration, 31, 654-667.

20.

Wang

Gianakis

G. A.

(1999). Public officials’ attitudes toward subjective performance measures. Public Productivity & Management Review, 22, 537-553.