Abstract
This research assesses how state and local factors combine to influence the adoption of performance measurement for local service contracting nested in state contexts over time. Using a multilevel linear growth model to analyze local survey and official data, the findings demonstrate that local adoption of performance measurements nested in state contexts changed significantly; local contract management capability are significantly linked to the adoption of performance measurement. State contexts have a large impact on that adoption as well. The findings, which may have been unobserved by previous studies, suggest that local practices are indeed embedded in state governance across time.
Keywords
Local governments have adopted management reform to promote their development (e.g., such as adoption of strategic plan, adoption of performance measurement; Kwon, Berry, & Feiock, 2009). Local management reform has always attracted the attention of scholars (Blair, 1998; Liou, 1998; Poister & Streib, 2005). A number of important gaps, however, exist in these studies. First, past research has been limited to a cross-sectional process at the local level (Poister & Harris, 2000; Schneider, 2007). This may demonstrate an incapability to establish the cause–effect relationship of management reform. In addition, the characteristics that lead to a local government’s decision to adopt management reform may not be the same as the factors that shape its evolution. To capture the reality of local practices, local management reform should be therefore modeled as a processive path rather than as a series of outcomes 1 (Kwon et al., 2009; Rogers, 1995). In addition, these studies only look at public management’s impact on a particular jurisdiction. However, this ignores the fact that higher hierarchical level (e.g., state contexts) is a key intervening variable in the local management reform equation. If we do not emphasize hierarchical governance, some incorrect conclusions are likely to be made. Given this gap in the literature, the primary goal of the present study is to consider state–local relationships which may more accurately account for variations in the adoption of local management reform than when the local level alone was considered. In addition, previous studies tended to focus on the adoption of management reform based on overall rather than on specific programs (Berman & Wang, 2000; Poister & Streib, 2005; Rivenbark & Kelly, 2003; Wang, 2002). This may obscure the understanding of the adoption of a particular management reform in a certain service. This research therefore discusses local service delivery contracting programs to evaluate local management reform. According to the International City/County Management Association (ICMA), contracting for service delivery remained the most widely used alternative local service delivery approach to local management reform (Brown & Brudney, 1998). Another impetus for federal agencies, state and local governments to become more measurement oriented for contracting-out services is government regulations, such as OMB Circular A-76, “Performance of Commercial Activities,” the Federal Acquisition Regulation (FAR), and the Government Performance and Results Act (GPRA) of 1993. The implication is that performance measurement issues can undermine service contracting. This research also examines know-how local contract management capability links to adoption of the performance measures; previous studies have emphasized this possibility 2 (Hendrick, 2003; Hou, Moynihan, & Ingraham, 2003).
This study builds on the findings of past studies while addressing some of their limitations, such as being unable to measure the effects of time effects and state contexts. This study uses national representative samples and multilevel linear growth modeling (i.e., hierarchical linear model [HLM]): (a) to find out whether there is a significant variance in within-local, between-local, and between-state samples; (b) to investigate the extent to which there is a relationship between local contract management capacity and adoption of performance measurement across time; and (c) to examine the impact of state contexts on local adoption of performance measurement across time.
Theoretical Contexts
The Affiliated State and Local Hierarchical Relationship
Norms and values might influence the perceived legitimacy of the innovation (reform) and might, therefore, facilitate its adoption and diffusion (Tolbert & Zucker, 1983). That is, state-level governance shapes the institutional rules that affect local governments, as well as include provisions that create local-level governments (McCabe & Feiock, 2005). In addition, local programs are made within the decision space established by the boundaries of state rules. Due to decentralization, the federal government has shifted much of its policy and administrative and management load to the states. It is the states, then, that bear considerable responsibility for the public programs (Nice & Fredericksen, 1995), and often, it is the local government’s service delivery that most directly affects citizens. The decentralization of the responsibility from state to local levels means that the success of some government policies and programs depends on the state’s influence on local governments (Bowling & Wright, 1998; Coggburn & Schneider, 2003; Honadle, 2003; Rodriguez, 2007), especially with regard to money, programs, grants, services, political parties, and the activities of interest groups (Berman, 2006). As indicated previously, the adoption of local management reform relates to expected outcomes within not only local provisions but also opportunities and constraints created by state-level rules. Thus, it may be more accurate to study local management reform within state contexts when considering state–local relationships.
Adoption of Local Performance Measurement Across Time
The beginning of contemporary performance measurement, from the New York Bureau of Municipal Research in the early 1900s, was part of the Progressive movement. The application of scientific management to public managerial decisions promised to take the politics out of government (Williams, 2004). The current state and the diffusion and usefulness of performance measurement closely follow the path of other productivity improvement strategies. In the 1980s, the private sector experimented with a number of productivity initiatives defined by the total quality management movement, as the performance measurement field expanded to consider service quality, customer satisfaction, and managing by results (Kopczynski & Lombardo, 1999). In the 1990s, for federal government practices, the GPRA of 1993 required federal agencies to develop strategic plans tied to proposed budgets and performance measures by way of legislations and mandates. For example, 47 of 50 states adopted some types of performance-based budgeting during the 1990s; 16 of those states had legislation that linked their agency strategic planning process to the development of performance measures based on the agency’s missions and goals (Melkers & Willoughby, 1998). In 1998, only one third of U.S. counties with populations more than 50,000 were using some type of performance measurement (Berman & Wang, 2000). In a 2000 study performed with the Government Accounting Standards Board, Melkers and Willoughby (1998) found that performance measurements were used by 47.8% of local governments in the United States. Poister and Streib (2005) also found that among municipalities with populations more than 25,000, about 56% of respondents reported that their jurisdictions used performance measures to track the implementation of projects. A recent study showed that only 17% of the responding U.S. Midwest cities with populations between 10,000 and 200,000 have involved citizens in the development and selection of performance measures (Ho, 2006). Such empirical studies confirmed that adoption of local performance measurement was not a fad, but a body of slowly diffusing knowledge and management practices (Berman, 2006), which was expected to contribute to local development.
Local Service Delivery Contracting and Performance Measurement
Service delivery contracting out is not a fad in government reform. Since the 1970s, the federal government, state governments, and local governments began to experiment with the idea of “service delivery contracting.” A series of the administrative regulations require strengthening the linkage between service delivery contracting and performance. For examples, OMB Circular A-76, “Performance of Commercial Activities,” emphasized that private sector for goods and services are the government’s policy “to rely on competitive private enterprise.” Federal officials need to try to ascertain the potential contractors could conduct cost-effective commercial activities. The FAR also requires a prospective contractor to comply with the required or proposed delivery or performance schedule, and have a satisfactory performance record. Then after contracting out services, the agencies need contract management capacity to management and evaluate the performance of the contractor.
The emphasis on performance in the contracting relationship means that the primary responsibility of contracting agencies and contractors is to produce desired results for citizens (Behn & Kant, 1999). Portz, Reidy, and Rochefort (1999) identified the new tasks that confront managers when they contemplate contracting out: Performance specifications must be written, and a system of monitoring and evaluation must be put in place. Many governments have developed sophisticated and systematic methods to measure output, cost-efficiency, intermediate outcomes, benchmarking programs, outcomes, and impact of contracting services. The evidence has demonstrated that local governments have still used performance measurements to monitor their programs, budgets, and effects (Melkers & Katherine, 2005). Although there may have been many attempts to evaluate service delivery, the ICMA survey which is conducted every 5 years uses four nominal scales to measure aspects of service delivery: citizen satisfaction, cost, compliance with delivery standards specified, and other. As the ICMA survey results indicated, local governments used the first three the most to evaluate local service delivery, contracting such as the 1997 ICMA survey: citizen satisfaction (58.5%), cost (83.6%), compliance with delivery standards specified (80.4%), and other (4.9%). The 2002-2003 ICMA survey: citizen satisfaction (69.0%), cost (86.8%), compliance with delivery standards specified (84.0%), and other (3.5%). The following section will discuss the frequently used performance measures of citizen satisfaction, cost, and compliance with delivery standards.
Citizen satisfaction
Public managers may be required to establish formal systems for tracking and monitoring citizens’ complaints or to learn about public concerns through citizen surveys about service delivery contracting (Brown, Potoski, & Van Slyke, 2006). By developing and selecting performance measures of citizen satisfaction, public managers and elected officials track and explore whether the public services satisfy the needs of the citizens; then they can decide to terminate or modify their quality and quantity (Kelly, 2003).
Cost
Numerous studies have cited monetary and cost-efficiency considerations as key factors in service delivery contracting decisions (Kettl, 1993; Ni & Bretschneider, 2007; Savas, 1999; Seidenstat, 1999). Cost or expenditures associated with innovation adoption is considered to be a critical component of the efficiency dimension of performance (Downs & Mohr, 1976). Private contractors operating in competitive markets are under constant pressure to keep costs down often through innovative service delivery (Kettl, 1993; Savas, 1999). The government, through the cost indicator, can know whether the contracted service arrives at the needs of cost-efficiency.
Compliance with delivery standards
A compliance with delivery standards measure requires governments to define their required outcomes in specific, measurable, assignable, realistic, and time-related terms (Proeller, 2007). It saves costs and improves services by allowing and encouraging competition based on cost, qualifications, and committed quality. A compliance with delivery standards measure increases the likelihood that the working relationship between the government and the contractor will be better than in traditional contractual arrangements and that the quality of service will be higher as a result of the improved relationship (ICMA, 2001).
Theoretical Framework
Local Contract Management Capability and Performance Measurement
There has been increasing interest in the relationship between management and performance. Some scholars of public management assume that management and performance are connected (Dilulio, 1989; Donahue, Selden, & Ingraham, 2000); others require more rigorous investigation of management links to performance (Heinrich & Lynn, 2000; Lynn, Heinrich, & Hill, 2000; Rainey & Steinbauer, 1999). In other words, they view management capacity as a “necessary antecedent to performance” (Donahue et al., 2000, p. 385). 3 For example, good management can decrease costs and improve results; poor management can bring about the reverse (Coggburn & Schneider, 2003). As Brown and Potoski (2003a) argued, the success or failure of any alternative service-delivery arrangement likely depends on how well governments can manage the entire contract process, from assessing the feasibility of contracting through implementation to monitoring and evaluation activities (here termed contract management capacity). 3 Feasibility assessment capacity, implementation capacity, and evaluation capacity were often seen as those of program evaluation standards, which will enhance the quality and fairness of professional practice (Sanders & The Joint Committee on Standards for Educational Evaluation, 1994), for example, local service delivery contracting.
Feasibility assessment capacity
Feasibility assessment capacity is intended to ensure that a program will meet practical procedures, political viability, and cost-effectiveness. Public managers should take those actions to collect and use information to judge the worth or merit of a program (Sanders & The Joint Committee on Standards for Educational Evaluation, 1994). Feasibility assessment capacity is used to determine whether to make or buy the goods or services, including the hiring of staff trained in market analysis, legislative study groups to assess whether a service or function is appropriate for contracting (Brown & Potoski, 2003b), or the availability of time, budget, staff, and contractors to implement procedures that minimize disruption. This capacity is expected to link to adoption of performance measurements to management service delivery contracting practices. We can now make the following hypothesis:
Hypothesis (H1): Local feasibility assessment capacity will be positively related to local adopting performance measurements, controlling for state-level variance over time.
Implementation capacity
The implementation process normally runs through several stages, beginning with passage of the basic statute, followed by the policy outputs of the implementation agencies; and the compliance of target groups with those decisions (Mazmanian & Sabatier, 1983). Implementation capacity in service delivery contracting is to bid on the contract, to select a provider(s), and to negotiate a contract such as hiring staff to negotiate tenders and to create management systems for contracting (Brown & Potoski, 2003a; Kettl, 1993). Successful implementation entails the adoption of performance measures to track the implementation of bidding, letting, negotiating, and enforcing contracts. Therefore, this research expects to find a positive relationship between implementation capacity and adoption of performance measurements. The following hypothesis can be made:
Hypothesis (H2): Local implementation capacity will be positively related to local adopting performance measurements, holding state-level variance over time.
Evaluation capacity
Evaluation capacity is the ability to evaluate the contractor’s performance; it includes procedures for collecting performance information and staff to conduct project audits (Brown & Potoski, 2003a). In the absence of the capacity to monitor and audit contracts, governments may be unable to determine whether the contractor has delivered the service according to contract specifications (Milward, 1996).
In the public sectors, organizational success is also evaluated in terms of the satisfaction of stakeholders. External stakeholders (such as citizens and clients) and internal stakeholders (other organizational staff), judge the quality and performance of the program based on their own criteria, which may not be the professional standards that organizational staff uses in assessing their performance (Berry, 2007). Therefore, local government must identify internal and external stakeholders to assess their relative importance, interest, expectations, and assessments of the service delivery contracting. This research expects evaluation capacity–inside and evaluation capacity–outside stakeholders to require the adoption of performance measurements to management contracted service. Thus, the following hypotheses are presented:
Hypothesis (H3): Local evaluation capacity–inside stakeholder will be positively related to local adopting performance measurements, controlling for state-level variance over time.
Hypothesis (H4): Local evaluation capacity–outside stakeholder will be positively related to local adopting performance measurements, controlling for state-level variance over time.
State Contexts and Local Performance Measurement
As discussed previously, the hierarchical relationship between the state and local levels influences local practices. Therefore, state contexts are expected to influence the adoption of performance measurements in local service delivery contracting.
State politics
State politics may influence local governments because voters elect state officials and legislatures. State governments and legislatures need to respond to voters. The empirical results in Ni and Bretschneider’s research (2007) suggested that the political environment is strongly present in state-level contracting decisions. Moon and deLeon (2001) also found a positive relationship between a leader’s liberal ideology and the adoption of reinvention. In Schneider’s study (2007), ideological alignment has the strongest effect on the adoption of administrative innovative practices in local governments. Newly elected governors and their appointed agency heads are positively associated with the adoption of management reform practices (Berry, 1994).
To assess the consequences of state politics on adoption of performance measurements, we consider levels of partisan conflicts in state governments. Partisan discord among the branches of state government can promote partisan conflict, measured here with a dummy variable indicating divided government, where the legislature is unified and the governor is of the opposing party (Kelleher & Wolak, 2007). Divided government intensifies legislative and administrative micromanagement of the federal bureaucracy, as well as in state and local governments (Durant & Wilson, 1993). Therefore, the following hypotheses can be inferred:
Hypothesis (H5): A state with divided government is more likely than a state with unified government to positively influence the adoption of performance measurement, holding local-level variance over time.
State reinventions
Unlike the earlier reforms that were frequently a product of a major reform commission, study group, or similar enterprise, state reinventions may have resulted from a governor’s initiative or may have been undertaken by an agency director (Brudney, Hebert, & Wright, 1999; Brudney & Wright, 2002). The trend of state reinventing government is a hallmark of results-oriented management, which emphasizes the results rather than the process in a program or in a policy (Brudney et al., 1999). Although no empirical evidence has indicated that state reinventions influence adoption of local performance measurement, we expect that high levels of state reinventions will prompt local governments to adopt performance measurement because of hierarchical state–local relationship. Then the following hypothesis can be formulated:
Hypothesis (H6): State reinventions will be positively related to local adopting performance measurements, controlling for local-level variance over time.
State legislation
Under state legislation, local governments must work together to prepare regional plans that address the interrelated goals of controlling growth, combating environmental problems, and providing infrastructure. State mandates often require localities to adopt new programs or meet higher performance standards, and thereby create unfunded costs for local governments (Berman, 2006). For many states, performance-based budgeting is made through targeted legislation by incorporating accountability, strategic planning, reinvention, and budget reform, as well as performance requirements in their appropriation bill (Melkers & Willoughby, 1998). Then the state performance-based legislation always urges their affiliated local governments to adopt performance measurements as the result of hierarchical state and local relationship. This leads to the following hypothesis:
Hypothesis (H7): A state adopting performance-based legislation should be more likely than a state that has not adopted such legislation to positively influence the adoption of performance measures, holding local-level variance over time.
State fiscal health
The presence of resource slack may offer public officials the opportunity to expand programs or to pursue service quality through contracts (Ni & Bretschneider, 2007). State financial aid to local governments consists of grants and shared taxes. Grants are usually funneled to specific programs in areas such as education or transportation (Berman, 2006). However, the empirical evidence on the relationship between fiscal stress and contracting is mixed. In their study of contracting in school districts, O’Toole and Meier (2004) found that an abundance of local resources was positively related to the amount of contracting. Recognizing that some governments may adopt contracting to improve service quality, Boyne (1998) suggested reconsidering the theoretical relationship between fiscal stress and contracting out. Then we can expect the state’s fiscal health to influence local administrative practices because it represents resource amounts that support local practices. This leads to the following hypothesis:
Hypothesis (H8): State fiscal health will be positively related to local adopting performance measurements, controlling for local-level variance over time.
Sample Characteristics
The samples are representative of municipalities and counties along basic criteria such as population, geographic location, and metropolitan status. In all, 230 valid repeated samples made up of 185 cities with populations 10,000 and above, and 45 counties with populations 25,000 and above were taken up for this study. All of these are nested in 42 states in two cohorts of 5 years from 1992 to 1997 and from 1997 to 2002. That is, the number of repeated measurements across two 5-year time span represents the time level of our model (within local), the number of local characteristics is our local level (between local), and our samples at state level (between state) is state characteristics. As seen in the demographic information shown in Table 1,10.87% of the sampling localities are located in Northeast states in comparison with 22.61% of those in North Central areas, 41.74% of those in Southern areas, and 24.78% of those in Western areas. Although the Northeast samples are less compared with other areas, they are not unusual. Most of the related research reported the similar situation.
Demographic Information and Representative Sample Test for Sample Localities.
Note: MSA = metropolitan statistical area.
As Table 1 indicates, there is no significant difference between the sample and the population in terms of the locations (t = 0.03, p > .01). 4 The central cities contain 33.48% of the sample governments, and 47.83% of those are from the suburbs located in a metropolitan statistical area (MSA), while 18.70% of the remaining are independent cities/counties. The representative sample test shown in Table 1 also reveals that there is no significant difference between the sample and the population in terms of MSA. The small size and homogeneity of our sample may limit the generalizability of our findings. Such homogeneity, however, can be viewed as advantageous for detecting statistical relationships as it restricts many sources of superfluous variation.
Measurement
The outcome variable, explanatory variables, and control variables supply much of the information with a desire to estimate state and local variations on the adoption of performance measurement in this research. The ways in which ICMA survey items compose local dependent and independent variables are shown in Appendix A, which also indicates ratio of each survey item from 1992, 1997, and 2002, and Cronbach’s α for the composed variables. Appendix B presents their conceptual definitions, operational definition, and data sources for local and state variables. They come from objective and subjective data sources which rule out common-source bias.
Outcome Variable
The outcome variable, adoption of performance measurement, indicates that local governments adopt the techniques to evaluate their private and nonprofit service delivery. The ICMA survey asks, “Does your local government use any techniques to systematically evaluate its private service delivery?” The ICMA survey defines private service delivery as for-profit firms, nonprofit organizations, and private industries. The ICMA asked the respondents to evaluate three aspects of service delivery: citizen satisfaction, cost, and compliance with delivery standards specified. For our research data, some local governments simultaneously adopt two or three performance measures to support service delivery contract programs. In 2002, 99 out of 230 local service delivery contracting employed both the measures of cost and performance contract. In 1997, 69 out of 230 local samples adopted the three performance measures. In addition, because the unit of analysis (i.e., local service delivery program) cannot be separated from the overall performance measurement practices used to manage service contracting, we summarize the scores of three performance measures. Therefore, an index variable of adoption of performance measurement combines three aspects with a high internal consistency of Cronbach’s α, .84, 5 as shown in Appendix A.
Explanatory Variables
Within-local predictor
To analyze growth rates of adoption of performance measurement, this research uses multilevel linear growth modeling to estimate local management reform nested in the context of state governments. As a result, the rate of change is expressed in terms of parameters: time (Fitzmaurice, Laird, & Ware, 2004). Time was coded as the number of years which local governments that had entered at each time of measurement, beginning with Time 0 (1992), Time 1 (1997), and Time 2 (2002).
Between-local predictors
Between-local predictors which were assessed at all three times included five summary variables to aggregate information on several related items as shown in Appendix A: feasibility assessment capacity, evaluation capacity–internal stakeholders, evaluation capacity–external stakeholders, and implementation capacity.
Feasibility assessment capacity
This index variable is based on seven response item parcels, 6 and has a Cronbach’s α of .71, as shown in Appendix A. This variable is one in which “a local government responded to study the feasibility of adopting private delivery alternatives within the last 5 years because of external fiscal pressures, including restrictions placed on raising taxes (e.g., proposition),” in which ICMA survey operated its survey item.
Evaluation capacity—Internal stakeholders
The explanatory variable is defined by the ICMA survey item that asked the following question: “Who inside your local government was involved in evaluating the feasibility of private service delivery?” This index variable, summarizing nine items, has a Cronbach’s α of .80, as shown in Appendix A.
Evaluation capacity—External stakeholders
The variable of evaluation capacity–external stakeholders is operationalized with the ICMA survey item that asked “Who outside your local government organization was involved in evaluating the feasibility of private service delivery?” This variable, which consists of six items, has a Cronbach’s α of .70, as shown in Appendix A.
Implementation capacity
The variable of implementation capacity is measured with an ICMA survey question, “Has your local government undertaken any activities to ensure success in implementing private service delivery?” This variable, made up of 12 items, has a Cronbach’s α of .75, as shown in Appendix A.
Between-state predictors
Between-state predictors were to measure the influences of state contexts on adopting of performance measurements that include state reinvention, state politics-divided government, state fiscal health, and state performance legislation.
State reinvention
The variable of state reinvention is conceptualized as the level at which the state reinvents its agencies by empowering employees and providing customer services, contract-like relationships, competition, performance incentives, and results management (Brudney et al., 1999). This variable combined with two data sources: the American State Administrator’s Project (ASAP) of University of North Carolina–Chapel Hill in 1994 7 ; and the Government Performance Project (GPP) of Maxwell School of Citizenship and Public Affairs at Syracuse University in 1999, and in 2002. 8 The GPP, which is an indicator of management capacity, may not be a direct measure of state reinvention. However, combining the data of ASAP and GPP is not unusual, as previous research has seen ASAP and GPP as the equivalent indicators to measure state reinvention (Brudney et al., 1999; Brudney & Wright, 2002; Burke & Wright, 2002).
Because the GPP assigns only letter grades, we have operationalized management capacity using a standard letter-grade point-conversion scheme. In combinations with GPP and ASAP, 9 we assign scores to state grades with a range from A- = 7 to C- = 1. 10
State politics-divided government
This variable indicates that when the governor is controlled by one party with its own ideas, preferences, and policy positions, and the state legislatures (including the House and Senate) are controlled by another party with other competing ideas, preferences, and policy positions (Coleman, 1999). We include a dummy variable, scored 1, if the state government is a divided one; otherwise, a unified state government is scored as 0.
State fiscal health
The variable of state financial health is constructed as the average percentage of state actual annual general fund revenue minus general fund expenditures (Ni & Bretschneider, 2007). A negative value indicates decreasing fiscal health or greater fiscal stress. A positive number suggests surplus and slack resources.
State legislation
The variable of state legislation is operationalized as state incorporated themes of accountability, strategic planning, reinvention, and budget reforms into performance requirement through targeted legislation and appropriation bills (Melkers & Willoughby, 1998). We create a dummy variable, scored 1, if the state government takes key performance legislation; otherwise, nonadoption of key performance legislation is scored as 0.
Control Variables
Researchers have taken different approaches to examining the characteristics of a jurisdiction that would explain the adoption of a particular management reform. Considering the local and state contextual factors, a positive relationship exists between the economic health, the form of government, population size, growth and density of local communities, and the tendency for their local governments to adopt management reforms (Berry, 1994; Berry & Wechsler, 1995; Feiock, Jeong, & Kim, 2003; Rivenbark & Kelly, 2003).
Between-local controls
Council-manager government. Governments with more professional management structures—in other words, those with council-manager forms of government—should be more likely to adopt management reforms to monitor contract (Brown & Potoski, 2003b). The previous research has also found that the influences of management reform (e.g., strategic planning, performance measurements) on local economic development policy were evident in the form of governments (Feiock et al., 2003; Rivenbark & Kelly, 2003). Then we include a dummy variable, scored 1, if the government is a council-manager form of government; a mayor-council form is scored as 0.
Metropolitan statistical area
Local governments located in densely populated metropolitan areas are more likely to have opportunities to employ service delivery contracting than local governments in sparsely populated rural communities (Brown & Potoski, 2003b; Stein, 1990). Then this control is used to measure the effects of local context on the adoption of performance measurements in service contracting. The local government is, therefore, scored as 1 if it located within a U.S. Census Bureau standard MSA; otherwise, it is scored as 0.
Local population
The size of local population is an important source of information for local administration and a signal for local economic development (Tiebout, 1956). Previous research has used this variable to control the potential false relationship between explanatory and outcome variables (Ho, 2002; Warner & Hefetz, 2007). This control is included to gauge the effect of local contexts, which was measured with the numbers of the residents who live in the same geographical area.
Between-state controls
The Republican governor. To control state administration to influence adoption of performance measurement, we consider party affiliation of the governor. We measure this with a nominal variable indicating a Republican governor or non-Republican (Democratic or Independent) governor. If the state governor was affiliated with the Republican Party at the time, the locality in that state was coded 1; a non-Republican governor was coded as 0.
State population
We include the number of residents living within the state government’s jurisdiction as reported in the U.S. Census Bureau. Previous research has shown that government contracting is likely to increase as population increases from low to medium and then to decrease as population increases from medium to large (Stein, 1990). Thus, we include state populations to control our models.
Multilevel Growth Model (MGM) as an Analytical Lens
Because our data consist of local management reform clustered within state contexts that change over time (Brudney & Selden, 1995; Holt, 2008; Singleton & Straits, 2005; Raudenbush & Bryk, 2002), we used a three-level MGM. This procedure accounted for correlation of data within local governments, thus reducing the probability of a type 1 error that could be introduced if this correlation were ignored. 11 A series of MGM analyses were carried out using the Stata 11.0 xtmixed models program (Rabe-Hesketh & Skribdal, 2008). A four-step multivariate model was used to examine time, local, and state characteristics associated with both the mean level and linear growth rate in local adoption of performance measurement. The first model included only the outcome—local adoption of performance measurement (null model). The second model included only mean linear growth late (year). The third model included within-local predictor (year), and between-local predictors including local contract management capacity (feasibility assessment, implementation, evaluation-inside stakeholder, evaluation-outside stakeholder), and local controls (council-manager, local population, and MSAs). The final model incorporated the between-state measures (divided government, state reinventions, state legislation, state fiscal health, state population, and Republican governor) into fully specified intercept and slope model. In addition, deviance statistics assessed the significance of the difference in fit between successive models in which one model was nested within the next. MGM using these procedures enables the testing of three research questions: (a) On average, is there a change in adoptions of performance-measurement over time? (b) Do local and state characteristics impact performance measurements over time? (c) How much variance do time, local and state characteristics resemble each other across time?
Findings
Table 2 summarizes the results of analyses testing Hypotheses 1 to 8, using the HLM standard procedure from null model, adding time predictor model, local predictors model, and state predictors model.
Results of Multilevel Linear Growth Analyses Predicting Adoption of Performance Measurements.
Note: ICC = intraclass correlation coefficient.
*p < .1. **p < .05. ***p < .01. ****p < .001.
Results of HLM Null Model
Null models were run for the individual-level dependent variables of interest. Resulting intraclass correlation coefficient (ICC) values and associated chi-square tests revealed that 93% of the variance in adoption of performance measurement resided within local level (random variance = 1.539, p < .001). Accordingly, we used HLM to predict adoption of performance measurement (Hypotheses 1-8).
For each model, our strategy was to compare the model deviance statistics or goodness of fit against the deviance of a competing model. The difference in deviances is distributed as a chi-square statistic. This allows us to use a likelihood ratio test to make decisions about the best fitting model. If the test reveals that the difference in deviances is nonsignificant, we retain the more parsimonious model. In this research, the final model—adding state predictors—is significantly better than adding local predictors’ model (df = 18 – 14 = 4, deviance = 2076.883 – 1995.797 = 81.086, p < .001). The addition of a state predictors’ model will be discussed as follows.
Within-Local Level Analyses
The within-local level indicates an intralocal correlation coefficient of 21.60% (1.171 / [1.171 + 2.06 + 2.19]). This coefficient indicates that a small significant variation exists in the intercept within locals. The growth trajectories of adopting performance measurement increased because the coefficient of time was statistically positive. This result suggests that adopting performance measurements for the local service delivery contracting has continued to increase, perhaps due to the existence of management trends or practical needs to rely on them.
Between-Local Level Analyses
Table 2 summarizes the results of HLM analyses testing Hypotheses 1 to 4. We tested these hypotheses by entering the local controls (council-manager form, local population, and MSAs) and the four explanatory variables (feasibility assessment, implementation, evaluation inside stakeholder, evaluation outside stakeholder) as local-level predictors. Results supported three hypotheses: local governments with high levels of feasibility assessment, implementation, and evaluation inside stakeholder were significantly more likely to adopt performance measurements than local governments with low feasibility assessment (coefficient = .072, p < .05), implementation (coefficient = .174, p < .001), and evaluation-inside stakeholder (coefficient = .129, p < .001). No local control significantly influences the adoption of performance measurement. Contract management capacity therefore positively influences the adoption of performance measurement. One possible reason is that local governments depend on their management capacity to evaluate contract performances. As mentioned previously, management capacity is expected to be closely linked to performance in government practices. Our research confirmed this argument in local management reform.
Between-State Level Analyses
We entered two state control variables (state population and Republican governor), and four explanatory variables (divided government, state reinventions, state legislation, and state fiscal health) as state-level predictors. Results supported that local government situated in the divided state government was significantly more likely to adopt performance measurement than those located at the unified state government (coefficient = .211, p < .01; Table 2). This indicates that the intermonitoring between state administration and state legislature stimulated local governments’ adoption of performance measurement. Next, Hypothesis 6 predicts that the state reinvention is a positive significant predictor on adoption of performance measurement. State reinventions have significant negative impacts on adoption of performance measurement (coefficient = −.066, p < .001; see Table 2). Hypothesis 4 was thus not supported. Beyond expectation, high levels of state reinventions have adverse effects on adoption of performance measurements. Although most state governments take a positive attitude toward government reinvention, the state practices of reinvention do not strongly influence the adoption of local performance measurement.
As a block in the model of adding local predictors, the local predictors explained 76.01% of the available within-local variance, and within-state variance in adoption of performance measurement (R2 = 76.01%). When state factors were taken account, they explained 21.9% of the available within-local variance, within-state variance, and between-state variance in adoption of performance measurement (R2 = 21.9%; i.e., [2.59 − 2.19] / 2.59 = 0.219 × 100% = 21.9%).
How much does time factor, state factors, or local factors influence adoption of local performance measurement? The degree to which low-level outcome within the high level context is similar is measured through the ICC (Raudenbush & Bryk, 2002). As a measure of similarity or homogeneity, the ICC reflects the correlation between pairs of low level randomly selected from within the same random high level. In the final model—adding state predictors, for the adoption of performance measurements—the ICC of the within-local variance is estimated as 21.60%, suggesting that about 21.60% of the variance in adopting performance measurements occurs within the same local governments nested in the same state. 12 The ICC of the within-state variance is estimated as 38.80%, indicating that about 38.80% of the variance in adopting performance measurement occurs within the different local governments nested in the same state. The ICC of the between-state variance is estimated as 40.60%, demonstrating that about 40.60% of the variance in adopting performance measurement occurs within the different states. All of this evidence reveals that local adoption of performance measurement can be decomposed into time-level variance, local-level variance, and state-level variance, which is one advantage of HLM measurement.
Discussion
The statistical results run counter to theoretical expectations that led to predict the state and local factors impact adopting performance measurement across time. This research identifies the importance of the statistical significance of key variables in this model when we introduce measures of time variable, and state-level variables, particularly those that either were not previously studied or were poorly measured in other studies. First, the previous research focused on the local level to the exclusion of the multilevel influence. These studies may have neglected to consider important information, such as state laws, and state rules, causing the analysis to yield untrustworthy results. Lynn et al. (2000) urged scholars to recognize the hierarchical nature of governance. They asserted that phenomena at the policy or program level are embedded in a multilevel governance arrangement and are endogenous to the broader context of governance (Heinrich & Lynn, 2000). Some state predictors (e.g., state-divided government, state reinvention practices) did significantly influence the adoption of performance measures whereas others had only a weak influence (e.g., state fiscal health, state legislation, state population) because of state–local hierarchical relationships. These findings have been underestimated or overlooked by previous research (e.g., Brown & Potoski, 2003a, 2003b, 2003c). State-divided government has a positive impact on the adoption of performance measurement. It may become more difficult for the executive branch to accomplish its goals when one party controls the House and the other controls the Senate. However, the result is surprising; perhaps when the governor faces a divided legislature, citizens see sharper partisan discord and delay and hold the executive branch accountable (Kelleher & Wolak, 2007); this positively influences local performance practices.
Our findings also indicated that state reinvention practices have negative influences on the adoption of local performance measurements. One possible explanation is that high-level state management practices did not necessarily diffuse to local performance practices. In addition, local performance practices have been integrated into their governance mechanisms rather than from the influences of state reinvention. Another plausible reason for this negative statistical significance may lie in the small size of local samples. In the present study, only 230 local samples participated in this research. Moreover, high levels of state management practices that implied the centralization on state management may not direct local management practice; on the contrary, they depend on local self-management. The evidence presented here also shows that the tendency to adopt performance measurement did significantly grow over time. Although cross-sectional research also presented similar findings (Berman & Wang, 2000; Melkers & Willoughby, 1998; Poister & Streib, 2005), this research employs panel study which offers possibilities for examining the causal relationships between contract management capability, state factors, and adoption of performance measurement due to time change.
As expected, the contract management capacity is closely linked to the adoption of performance measurement in the multilevel linear growth model. In other words, contract management capacity depends on the adoption of performance measurements. This confirms previous findings that argued the potential link between management and performance (Hou et al., 2003). The empirical findings also show that local or state controls present no heterogeneity on the adoption of performance measurement. This indicates that local and state demographics that were not being studied were held constant so as not to influence the outcome variable.
Implications for Public Management Research
In sum, the present study increases our knowledge of the relationship between contract management capacity and adoption of performance measurements and addresses the key empirical and theoretical limitations of past research. Empirically, we use recent large, nationally representative samples of local governments nested in states across time, applying the multilevel model designed to examine intralocal, interlocal, and interstate variations simultaneously. The research also found that local service delivery contracting had adopted performance measurements across time. Adoption of performance measurement to monitor local practices may become a management trend among local governments. This may remind us that we need to accumulate more panel studies to advance our knowledge of public management research (Moynihan, 2010). These findings also indicated that local and state behaviors did significantly result in the adoption of performance measurement in localities nested in state levels. State–local relationships were deeply influenced by an arrangement of activities within the states, especially given this hierarchical relationship in the American administrative system. If we ignore this calculation of the state role in estimating local practices, any spurious findings that result may adversely affect local practices. Nevertheless, this research cannot conclude that the previous research produced untrustworthy results because of their research limitations, measurement errors, or research design.
Limitations and Future Research
A few caveats are in order. First, much of the performance movement techniques historically from the Progressive Era and the municipal reform movement to currently reinvention either start at the local levels or combine simultaneous state and local reform that are difficult to sort out empirically to make the kind of argument that this study does (Shippan & Volden, 2006). Second, this research only identifies the three performance measures as indicators of adoption of performance measurement. However, each 5-year cohort of the ICMA survey uses these three survey items to obtain performance information since 1992. Future research may introduce other types of performance measures to test their adopting trends and antecedent factors. In addition, the statistical evidence supplies information for time effects, making a significant impact on the adoption of performance measurement. We need to acknowledge that time order may be a problem because performance measurement may have been established as “techniques to evaluate private service delivery” prior to some of the explanatory variables. Although the adopting growth rates are significant, we are reluctant to leap to the conclusion that the diffusions of local performance measurement will succeed. One of the reasons is that the current data set may not have enough time points that satisfy the reliable and valid inferences. In addition, the research samples do not seem to sufficiently represent total localities to support the successive phenomenon for the diffusion of adopting performance measurement.
Several questions in our research remain unanswered. For example, additional research is needed to understand which techniques are used to evaluate the above aspects of service delivery to acquire performance information (e.g., conducting citizen surveys, monitoring citizen complaints, conducting field observations, and analyzing data/records). Because of data limitations, we may have underestimated certain factors that influence the adoption of local performance management, such as local politics, local fiscal health, and the internal characteristics of organizations (e.g., local employees). Inclusion of additional measures would provide insight into possible reasons for the change pattern. In addition, a useful extension of this analysis would be to investigate whether this research framework also exhibit in other areas to increment its external validity. Despite these limitations and questions, this exploratory analysis constructs the contents of local contract management capacity and adoption of performance management nested in state levels over time, relating to local service delivery contracting, which advance public management research in both method and practice.
Footnotes
Appendix
Conceptual Definition, Operational Definition, and Data Sources for Local and State Variables
| Local variables | Conceptual definition | Operational definition | Data sources |
|---|---|---|---|
| Adoption of performance measurement | Your local government uses any techniques to systematically evaluate its private service delivery. | Summated continuous variables | 1992, 1997, and 2003 local government service delivery choices of ICMA survey |
| Feasibility assessment capacity | Has your local government studied the feasibility of adopting private service delivery within the past 5 years? | Summated continuous variables | |
| Evaluation capacity–internal stakeholders | Who inside your local government was involved in evaluating the feasibility of private service delivery? | Summated continuous variables | |
| Evaluation capacity–external stakeholders | Who outside your local government organization was involved in evaluating the feasibility of private service delivery? | Summated continuous variables | |
| Implementation capacity | Has your local government undertaken any activities to ensure success in implementing private service delivery | Summated continuous variables | |
| Metropolitan statistical area | This indicates whether municipality is located within an MSA (Metropolitan Statistical Area) as defined/designated by the U.S. Office of Management and Budget [OMB]). | 1. MSA (city = core city in an MSA; central counties are these in which a central city is located; suburban = city/county located in MSA). 0. Non-MSA = Explanatory (city/county not located in MSA). | |
| Council-manager government | This indicates whether the form of government is council-manager government or not. | 1 = Council-manager (CM, city), Council-administrator (CM, county); 0 = Noncouncil-manager (Mayor-council, Commission, Town meeting, Representative town meeting, commission, council-elected executive). | |
| Local populations | The number of residents living within the local government’s jurisdiction as reported in the Census Bureau. | High numbers indicate more populations live in the locality. | The Census Bureau |
| State variables | |||
| State reinvention | This is conceptualized as the level of how much state reinvents its agencies through empowering to employees, customer services, contract-like relationship, competition, performance incentives, results management and so on (Kettl, 1993; Brudney, Hebert, & Wright, 1999). | We assign scores to state grades with a range from A- = 7 to C- = 1 | 1. Government Performance Project (GPP) of Maxwell School of Citizenship and Public Affairs at Syracuse University in 1999, and in 2002. |
| 2. American State Administrator’s Project (ASAP) of University of North Carolina–Chapel Hill in 1994. | |||
| State-divided government | This indicates that when the governor is controlled by one party with its own ideas, preferences and policy positions and the state legislatures (including State House, State Senate) is controlled by another party with some competing ideas, preferences and policy positions (Coleman, 1999). | A state-divided government is scored as 1; otherwise, an unified state government is scored as 0 | 1.National Conference of State Legislatures. |
| 2.National Governors’ Association. | |||
| State fiscal health | The average percentage of state actual annual general fund revenue minus general fund expenditures (Ni & Bretschneider, 2007). | The average percentage of state actual annual general fund revenue minus general fund expenditures. | The Census Bureau |
| Key state legislation | The state has performance or performance legislation (Mekers & Willoughby, 1998). | A dummy variable, scored 1, if the government takes a key performance legislation; otherwise, nonadoption of key performance legislation is scored as 0. | Mekers and Willoughby (1998) |
| Republican governor | The party affiliation of the governor. | If the political ideology of the state governor was affiliated to the Republican Party at that time, the locality located in that state was coded 1; otherwise, non-Republican governor was coded 0. | National Governors’ Association. |
| State populations | The number of residents living within the state government’s jurisdiction as reported in the Census Bureau. | High numbers indicate more populations live in the state. | The Census Bureau |
Note: ICMA = International City/County Management Association; MSA = metropolitan statistical area.
Acknowledgements
I am most grateful to Frances Stokes Berry, Kaifeng Yang, and Richard Feiock for their precious opinions on this research.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
