Does Technologically Enabled Citizen Participation Lead to Budget Adjustments? An Investigation of Boston,MA,and San Francisco,CA

Abstract

Citizen participation in government can provide a broad range of benefits to governments and citizens alike. Advances in information technologies have enabled new types of citizen participation with governments. However, we currently lack an understanding of how these new types of participation, particularly those that generate information on community needs, influence resource allocations. This article focuses on one of these new technologies, 311 systems, and how citizen requests might influence departmental budget allocations. We track budget allocation in the cities of Boston and San Francisco for 106 departments or subunits from FY2005 to FY2013. Our findings indicate that there is no significant resource benefit for departments using 311 versus those that do not. While departments using 311 do have larger budget allocations than those that do not, those departments had larger budget allocations prior to the implementation of 311. And while data generated in the 311-enabled citizen participation are increasingly used to measure departmental performance, the findings of this study show that this information has little to no effect on the allocated share of the budget for departments.

Keywords

local government budget allocation coproduction 311 systems

Introduction

Government officials have traditionally used “citizen involvement primarily as a way to educate the public” and not as a way to receive citizen input (Berner, 2001, p. 23). Others have noted that “[s]cholars have long advocated the important roles of citizens in the budget process, arguing that it is necessary to ascertain the wants, needs, and desires of the polis in a democracy in order to accurately represent them” (Franklin, Ho, & Ebdon, 2009, p. 52). At a time when state and local budgets are under tight fiscal constraints, citizen input may be a way to get around statutory and political constraints that limit budgetary resources and facilitate improvements in governmental efficiency (Thomas, 2012) without sacrificing quality and equity. This article explores how a constant stream of technologically enabled citizen participation in local governments may be influencing budget allocations.

The types of technologically enabled citizen participation we focus on are 311 systems (nonemergency versions of 911), companion service request websites, and smartphone service request applications. All three of these platforms (phone/web/mobile), which we will jointly refer to as 311 systems throughout this article for the sake of simplicity, cut through municipal government red tape by providing a simpler and streamlined pathway for a resident to follow a problem to resolution. These systems, as implemented by our two case cities (Boston and San Francisco), also provide ways for citizens to get feedback on their service requests, examine overall trends of service delivery, and demand for services across these cities.

The type of citizen participation that 311 provides is a more indirect means of citizen participation in the budget process than is seen in the traditional participatory budgeting literature in places like Porto Alegre, Brazil (de Sousa Santos, 1998). Franklin et al. (2009) examine many different ways that citizens have traditionally participated in the budget process and how elected officials view these forms of participation. The form of citizen participation under investigation in this article is not specifically addressed by Franklin and coauthors, nor does 311 citizen participation fit cleanly into one of their broad categories of citizen participation mechanisms. For example, the individual service requests made by citizens represented in a 311 system would likely be considered a micro-level mechanism of citizen engagement in the budget process (Franklin et al., 2009). However, the aggregation of all data generated by the 311 systems, represented as a citywide demand for services map for example (see Figure 1), would be seen more as a macro-level mechanism for combining the opinions of the community rather than as an individual interaction. The simple heat map in Figure 1, which shows the density of requests for one type of service (sidewalk cleaning) in 1 month provides administrators and citizens alike a way to jointly assess where particular problems are cropping up within the city, and when repeated, can demonstrate the changing nature of problems facing the city. Thus, the goals behind 311 generally do not clearly satisfy one specific objective underlying Franklin et al.’s spectrum of citizen participation, but rather it offers an outlet to influence decision making, enhance trust, and provide for two-way communications. When 311 is implemented, as it has been in both Boston and San Francisco, it can also provide means to providing a broader message of need on a neighborhood by neighborhood or street by street level, and show overall citywide demand—again as seen in Figure 1. The collective picture that develops from 311 systems is one of an active citizens, not a mere customer of government services (Thomas, 2012). While 311 represents a demand for services (i.e., What can government do for me?), it is also a way in which citizens are engaging in the process of information provision, thus actively producing services for their governments (Thomas, 2012). As citizens provide real-time and accurate information, they participate and coproduce information for their governments as partners (Thomas, 2012).

Figure 1.

Heat map (request density) of sidewalk cleaning requests made in January 2013 in San Francisco, CA.

Guo and Neshkova (2013, p. 1) found that citizen participation in the information sharing stage of the budgeting process demonstrates the “greatest positive effect on organizational performance”. With 311 systems, cities are able to monitor reported problems across a wide range of issues and across the entire geography of the jurisdiction without having to first send city representatives to identify the problem. Using citizens as sensors of problems, while not error free (Meyer, 2013), provides public officials vital information about their cities, which can potentially both reduce the costs of providing services and improve government efficiency (Thomas, 2012). Increasingly, cities are incorporating the 311-generated information into their performance and budget reports (City and County of San Francisco, 2015; City of Boston Performance Management System, 2013; City of Chicago, 2013; City of Knoxville, 2013; City of Pittsburgh, 2012; The Government of the District of Columbia, 2013; NYC Mayor’s Office of Operations, 2013; Office of the Controller/City and County of San Francisco, 2012a, 2012b, 2013). The use of 311 related information in performance measurement and reporting in budget documents would make it conceivable that 311 systems do provide an indirect form of citizen participation. This would potentially mean that these data can influence the allocation of resources in the budget process by bettering matching citizen demand for services with the level of supply being offered by the government (Thomas, 2012). And unlike many other types of public participation in the bureaucratic decision-making process that require citizens to “show up” to make their voices heard, using 311 metrics as a means to interject public participation in the budgeting process has the potential benefit of increasing participation of underrepresented and disinterested groups.

The goal of this article is to specifically study whether the introduction of 311 systems in two large U.S. metropolitan areas, Boston¹ and San Francisco, allows saving tax dollars by the responsible departments. As more cities adopt 311 systems and use the data generated from these systems in decision making, it will become more important to understand how this process can and does influence resource allocation.

Earlier works studying Boston’s (Clark, Brudney, & Jang, 2013) and San Francisco’s (Clark & Brudney, 2016a) 311 systems have demonstrated that socioeconomic status plays little to no role on the demand and supply sides of 311 requests. In addition, the use of 311 has been shown to improve overall satisfaction with government performance (Clark & Shurik, 2016) and can provide an effective mechanism for the joint production of information and its transmission (Clark, Zingale, Logan, & Brudney, 2016). These earlier works have not investigated, however, how citizen participation via 311 impacts resource allocation within a government. The findings from this study not only add to the citizen participation literature but also contribute to our understanding of performance measurement and management. While it is clear that Boston and San Francisco, and the many other cities with 311 systems, do use the information generated through citizen participation for management purposes (i.e., performance management and monitoring), the results of this article show that these systems have little to no measurable effect on the allocation of resources.

The remainder of this article is structured as follows. First, we provide an overview of the case cities and their 311 systems. Then, we place 311 systems in the citizen participation and performance measurement literature. Next, we offer a discussion of the data and modeling techniques followed by the statistical analysis of these two case cities and associated data. Finally, we will discuss the policy and managerial implications of citizen participation in local governments as a way to resource allocation.

Background

Background on Case Cities

The cities studied in this article, Boston, MA, and San Francisco, CA, are both leading civic innovators in all realms of 311 (phone/web/mobile). And, while these two cities may not yet be representative of all large American cities, an increasingly higher number of other cities divert their attention to Boston and San Francisco as models for 311, web reporting, and smartphone applications’ implementation and usage. We recognize the limitations of using only two cities in a study of this sort. Clearly, such a small sample of cities may limit the inferences that can be made. However, the study of leading cases can provide learning opportunities for other adopters of 311, even if Boston and San Francisco may be considered atypical in some ways (McDavid, Huse, & Hawthorn, 2012).

The Boston Mayor’s Hotline, introduced in calendar year 2008, is the city’s all-in-one call center. The telephone service has been complimented by a web portal that allows residents to request services through the Internet (www.cityofboston.gov/online_services) or via the city’s smartphone application, CitizensConnect (for iPhone and Android).

The City and County of San Francisco implemented their traditional 311 system in calendar year 2007. Like Boston, San Francisco has a website (www.sf311.org) that allows for reporting problems, as well as a smartphone application that interfaces with their system. San Francisco has broader coverage across their city in terms of the number of departments that have requests logged at any point of the system’s existence.

Overall, 62 departments in both cities, or about 58%, have never used 311 (29% of San Francisco’s departments and 87% of Boston’s), whereas 44 departments have used the system (71% of San Francisco’s departments and 13% of Boston’s). Boston has a much narrower implementation within its government compared with San Francisco, as only about 13% of units participate. However, it should be noted that in terms of the frequency of the departmental use of 311, Boston and San Francisco are essentially identical. The departments that ever used the 311 system in both cities in general made a very small number of requests (less than 100 requests to these departments over 4 or 5 years). Table 1 shows the number of departments or subunits in both cities that have used 311 since it was introduced. For our purposes, a use of 311 by a department is indicated when a service request gets logged into the database. Some calls to 311 are for information only and do not get logged into the system. Tables A1 and A2 of the appendix list a range of the types of services that are called and logged into the 311 databases, submitted online, or submitted via the smartphone applications.

Table 1.

Use/Nonuse of 311 by City (in 2013).

	Boston	San Francisco	Total
Department has not used 311	47	15	62
Department has not used 311	87%	29%	58%
Department has used 311	7	37	44
Department has used 311	13%	71%	42%
Total	54	52	106
Total	100%	100%	100%

Both Boston and San Francisco are traditional municipal governments in many other perspectives, making it possible to infer from their cases. Both cities follow traditional municipal budgeting processes. In Boston, the budgets of all departments (except the School District) are included in the General Fund; consequently, schools are not included in this analysis. Boston’s General Fund budget is prepared under the directions of the Mayor and the City Council, and this is the only fund that is legally adopted. The budget is submitted by the Mayor for legislative considerations and is approved by the City Council (City of Boston, 2012b). Democrat Thomas Menino was mayor of Boston from 1993 until the end of 2013—He did not seek reelection after 20 years in office. The Boston City Council is made up of four at-large representatives and nine district representatives (City of Boston, 2013). Although all Boston councilpersons are technically nonpartisan, nearly all are affiliated with the Democratic Party.

The city and the county of San Francisco are a consolidated government that issues one consolidated budget. Departmental budgets are first submitted to the Controller. Once approved by the Controller, the consolidated budget is submitted to the Mayor and then to the city’s legislative body, the Board of Supervisors, for adoption (City and County of San Francisco, 2012). In 2009, the voters of San Francisco adopted a biennial budget cycle; the first biennial budget was submitted in 2012. Democrat Edwin Lee became mayor in 2012, succeeding Gavin Newsom (who became California’s Lieutenant Governor), also a Democrat, who had held this post since 2004. There are 11 members of the city’s Board of Supervisors, each representing one geographic district (City and County of San Francisco, 2002). All members of the board have been affiliated with the Democratic Party or the Green Party during the entire study period.

311 System Overview

Dialing 311 in a community with such as system connects a resident directly to a city agent to assist on a nonemergency problem or information request. As a companion service to 911, 311 is also frequently compared with 411 for government services. By calling 3-1-1, residents may, among a multitude of other things, request to remove a skunk from the road or graffiti from municipal property, report a pothole, or request information about recycling services or a missed trash pickup. Government, therefore, involves citizens as reporters or sensors of problems. Such citizen–government interactions could potentially reduce the demands for street-level bureaucrats to see all the problems of the city themselves. Or, it could simply result in the “squeaky wheels” getting more grease at the expense of all other government functions. However, the studies of San Francisco’s 311 users have shown that these users in general, and high-frequency users, specifically, are very representative (Clark & Brudney, 2016a, 2016b), and general participation in 311 improves satisfaction in government (Clark & Shurik, 2016). In an examination of Boston’s system, Clark et al. (2013) found no evidence of systematic bias in 311 systems directing services to particular neighborhoods or socioeconomic groups.

The first 311 system was rolled out in 1996 in the city of Baltimore, MD, and is currently available in more than 70 cities across the United States and Canada. Similar systems also operate in Europe, Asia, South America, and Australia.²

Underpinning all 311 system is the citizen (or constituent) relationship management (CRM) system. This database system tracks requests and routes them to the appropriate departments for resolution. Citizens reporting issues are given a tracking number that they can use to follow up on their request to assure their issue is addressed. The ability to track an issue provides citizens with an accountability measure to assure their problem does not fall through the cracks—though not all cities make effective use of this technology (Clark & Shurik, 2016; Clark et al., 2016; Meyer, 2013).

311 Systems, Citizen Participation, and Performance Measurement

Empirical research has often focused on the demands and the costs of government services and programs. Frequently this literature emphasizes economies of scale or reducing costs through increasing production. However, in terms of providing services for a larger population, the results have been mixed and shown the presence of economies and diseconomies of scale, constant returns to scale, or even U-shaped cost functions for the same public services. The variations in results may be caused by many factors such as resource prices and differences in output (Duncombe, 1996).

As governments become more citizen-centered, accountable, and transparent entities because of 311 systems, we might expect reforms associated with performance budgeting to follow (Andrews & Shah, 2005). Performance budgeting is evidence of the pervasiveness of the more recent citizen-centered approach to public-sector reform. This engages citizens and elicits citizen action and direct participation. Citizen participation motivates government accountability. Citizen input allows the government to connect with and respond to the needs and preferences of citizens. Citizen participation also breaks down the attitudes of distrust of government that have become prominent (Shah & Shen, 2007). Citizen participation in the budget process specifically has been seen as a way to “educate participants on the budget, enhance two-way communication, inform decision making, gain support for budget proposals, create a sense of community, and enhance trust” (Franklin et al., 2009, p. 55).

Performance measurement of local governments has been in place since the turn of the 20th century (Ho, 2007b). These early efforts, and most efforts throughout history, have focused on measurement of performance by professionals, not citizens (Ho, 2007b). Furthermore, scholars such as Robert Behn (2003) have asserted that the measurement of performance in government should be influencing and informing the budgeting process. Numerous federal efforts, including the Government Performance and Results Act (GPRA) and the Program Assessment Rating Tool (PART), have formalized the role of performance measurement in the budgeting process at the federal level.

While there are examples of a limited influence of performance measures on the budget process (Jordan & Hackbart, 2005; Melkers & Willoughby, 2005), when the information is derived from citizens’ needs, “elected officials may give [it] greater political credibility and may pay closer attention to the data in the decision-making process” (Ho, 2007b, p. 1160). Franklin et al. (2009) found that “elected officials perceive higher value for [participatory] mechanisms that have closer interaction with citizens and mechanisms that are more effective in addressing micro-level or individual issues” (p. 69). This would imply that under the right circumstances and design, concerned, knowledgeable citizens have the capability to influence the budget process. Engaged citizens may act as the representative body of specific interest groups, the community at large, or just themselves. They are size-based, and when they present themselves as serious and committed to long-term goals, elected officials listen (Ho, 2007b). Citizen participation has the potential to change the inefficient, command-and-control culture of government. A shift in focus toward a participatory structure that engages and empowers citizens may also lead to greater efficacy in the provision of government services.

What Andrews and Shah (2005) refer to as the “new citizen-centered framework to guide reform” was born from the “lessons learned from past reform failures” (p. 159). This framework focuses on designing the internal framework of government to fully actualize “results-oriented reform” (Andrews & Shah, 2005, p. 166). Results-oriented reform creates a bottom-up evaluation of governance and administrative measures. This approach demands government accountability and increases the reliability of performance evaluation, as citizens are directly involved as firsthand users of government services. The “citizen-centered framework to guide reform” puts citizen input at the center of government decisions, ultimately demanding that their government produces better, more equitable, and efficient results (Andrews & Shah, 2005).

The motivations behind citizen participation in government service provision in our two case cities are varied (both cities in this article have at least 70 different types of 311 service requests), but it is often driven by individualistic actions (Alford, 2002). A list of types of service requests can be found in Tables A1 and A2 of the appendix. For example, if residents want the trash cleaned up or a pothole fixed so that they individually do not have to suffer from these problems, they become a participant in the process of information discovery and generation for the city—or the micro-level participants that Franklin et al (2009) describe. But the motivations are not always individualistic. Advocates of citizen participation believe in its ability to promote and foster a community-based spirit of citizenship. This broader perception of involvement, something Franklin et al. describe as a macro-level mechanism, is possible with 311 when citizens and leaders are able to distill out citizen demands on a citywide basis from demand maps or other means of distilling aggregate demand (see Figure 1 for an example). The “community has to develop a supporting structure for experimenting policy’” (Roberts, 2008, p. 89). Furthermore, if “the use of citizens in service delivery is treated as a marginal activity by public agencies, then we should not expect” for it to be an effective means to improve community outcomes (Roberts, 2008, p. 89). For this reason, we expect that governments, and more specifically the departments within a government, will take heed of the citizen interaction that is the result of the 311. This is because the intents of 311 are to ease the burden of communication between citizens and government and to better engage with residents. In our analysis, we measure the citizen participation in the budget process through the input they provide via 311 system requests to each department—and look to determine how this information might influence the budgetary allocations in subsequent years to those departments.

Alfred Ho (2007a) notes that citizens may be discouraged from participating in performance measurement because “concrete results from their input” are not seen “until years later” (p. 112). However, in the case of 311, response time to citizen input can often be measured in hours or days, not years. The means by which the citizen participation is taking place via 311 is much more public facing, open and transparent—Many cities state transparency as an explicit goal of 311.³ These efforts to improve services to citizens through a system that is more transparent than the traditional bureaucratic mechanisms allow us to posit that changes in resource allocation could accompany a change in service demands through citizen participation. The success or failure of the government, at least in the case of Boston’s and San Francisco’s 311 system, is open and viewable to the public via their ability to track requests online in an open forum. Citizens are able to view live statistics and statuses of service requests, and thus are potentially much more likely to view this form of participation as viable, useful, and with more political power; it is all occurring in a transparent forum online, rather than within the black box of government. Ho (2007a) warns that the use of the Internet may limit participation because of concerns over access and the digital divide. However, three studies, Clark et al. (2013) examining Boston and Clark and Brudney (2016a, 2016b) examining San Francisco, found that the advanced communication technologies (smartphone applications in particular) have the potential to at least partially bridge the digital divide and that access via these systems is not systematically biased against disadvantaged groups.

It is clear that department managers in the City of Boston take the data derived from their 311 system seriously. Managers from “Inspectional Services, Parks, Property Management, Public Works and Transportation meet regularly to discuss the recent requests for City services recorded in the City’s Constituent Relationship Management (CRM) system” (City of Boston, 2012a, p. 176). These managers use the sessions to benchmark their actual performance, as measured in their 311/CRM data (City of Boston, 2012a).

The City and County of San Francisco also regularly use their 311 data to benchmark their performance, and improve performance for departments using 311. For example, the city’s graffiti abatement program has found that the faster they respond to reports of graffiti, the lower the levels of new graffiti would be, resulting in an overall reduction in the total amount of graffiti seen in the city. This information is largely derived from 311 system data (Office of the Controller/City and County of San Francisco, 2013).

Based on the inferences from the citizen participation literature, and studying the 311 implementation and usage cases in Boston and San Francisco, we predict that the interaction between citizens and government significantly altered not only how governments manage and benchmark their performance in two cities (City of Boston, 2012a; Office of the Controller/City and County of San Francisco, 2013) but also how they reallocated the resources within the government units. Consequently, we propose the following two hypotheses:

Hypothesis 1: Government functions directly tied to the 311 system show significantly larger increases in budget share than those not tied to 311 systems.

Hypothesis 2: Increasing citizen participation in service delivery, as measured by 311 requests, increases a department’s budget allocation.

Data and Statistical Modeling

Dependent variable

The dependent variable in all models is the share of budget allocated to each department in the total budget in a given fiscal year. In some instances, a department has been broken into separate functions if those functions were identifiable in the budget documents and matched with our 311 data. The years span from FY2005 to FY2013. These data were drawn from the budget documents for both study cities: Boston, MA (City of Boston, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012b) and San Francisco, CA (City and County of San Francisco, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012). The dependent variable’s structure in its raw format is not normally distributed and, as a consequence, skewness in these data may result in biased estimations. We transformed the raw budget share into the natural logarithm of the budget allocation for each unit of observations.⁴

Primary independent variables

To investigate the explicit influence of citizen participation, we use data derived from Boston’s and San Francisco’s 311/CRM databases. The Boston data were given directly to the authors by the city of Boston (a slightly more limited data set is freely available online), whereas San Francisco’s data are readily available online via the city’s data portal (http://data.sfgov.org). To better match with the departmental budget decision-making process, all 311 request counts are lagged. According to the cities’ budget documents, department heads submit their budget request to the Mayor’s office in early January for the next fiscal year. For example, all 311 requests logged in the calendar year 2009 are used to measure their influence on the FY2011 budget, which was passed in calendar year 2010. This results in what amounts to a 6-month lag between when the data collection process ends and when a new fiscal year begins, though the data are current for decision-making purposes.⁵

In Models 1 to 12⁶ (Table 3), we use a set of dichotomous variables and their interactions for our difference-in-differences (DID) modeling. The first of these is a variable that indicates years in which 311 was in place government-wide. It takes the value of 1 in the fiscal year in which the 311 data would first inform the budgeting process, and in all subsequent years, it is coded zero for all years prior to implementation. The second variable is a variable that indicates that a specific department has received service requests at any point in time via 311—taking on the value of 1 if during the study period they have been the recipients of any 311 requests, and a value of zero if they have not received any requests. The third dichotomous variable we use is the interaction between these first two indicator variables, the years in which 311 was in place and departments that used 311. This variable is a so-called difference-in-differences estimator. In Models 13 to 15⁷ that cover all years of the data in the study period, we include the total count of 311 requests made to a department in a given year, measured in thousands of requests for interpretive purposes.

Control variables

We control for the length of time a 311 system has been in place. A priori it is unknown if the longer the 311 system is in place, the stronger or weaker the effect of 311 on budget allocations might be. On one hand, the longer administrators have to manage 311 data into the budget process, the more effective they will be at using the information to argue for more funds. On the other hand, one could see the adoption of 311 as the punctuating event for budget allocation, followed by years of stasis—having little to no long-term effect. The length of time since adoption variable is coded such that the variable takes on the value of 1 in the first full year after implementation for the department, and adds 1 for each subsequent year. The year of implementation varies by department, as not all departments implemented 311 in the same year. In addition, we include an interaction between the length of time of 311 usage and the indicator variable for having used 311. The direction of the effect of this interaction term can be twofold. The departments that have used the 311 system longer may have become more efficient and are, therefore, able to provide services at lower cost. On the other side, the length of usage might be positively correlated with the amount of budget allocations, because as more residents become familiar with the system, the number of service requests may go up resulting in higher need for additional resources.

Clearly, the type of citizen participation we measure in the models is just one of the many influences on budget allocations. Consequently, we have included other variables in our models (such as per capita gross domestic product [GDP] and the size of department workforce) to control for a variety of other known determinants of budget allocations.

Setting budget goals and projections requires a review of the local economic conditions. If the economy or housing market is sputtering, then local government revenue generation is not far behind. The economic condition will likely have an impact on expenditures if reserves (rainy day funds, fund balance, etc.) are not sufficient to cover the decline in revenue. If the national economy is sluggish, or there is a cut-oriented political climate in Washington, there may also be cuts in federal grants-in-aid to states and local governments; it is clear that this impact on the financial environment must be evaluated (Dworak, 1980). Of course if an economy is booming, the pressure on the expenditures may be stressed by price inflation. We use a per capita real GDP to control for the financial condition of the city’s residents, and how it changes during the study period—a period of significant economic upheaval because of the Great Recession. These data are for the metro area, not just the central cities of Boston and San Francisco. The data represent midyear estimates for each of the years. These statistics were collected and calculated by the Bureau of Economic Analysis (2013). These data are in thousands of constant 2005 dollars.

For local governments, the largest budget driver is the size of their workforce. The full-time equivalent (FTE) figures were obtained from both cities’ budget documents over the course of the study period.

Modeling techniques

In this article, we study what effect the introduction of 311 systems in two large U.S. cities, Boston and San Francisco, had on government budget allocations. Specifically, we are interested in learning whether departments using 311 receive larger budget allocations than those that do not, as well as whether a broader and more expansive use of 311 increased budget allocations.

In Models 1 to 12 we test the effect of 311 use versus nonuse on budget allocations by means of a difference-in-differences estimator, where introduction of the 311 system in Boston and San Francisco is treated as a natural experiment:

y = β_{0} + α_{0} d 2 + β_{1} d T + α_{1} d 2 d T + other factors,

where y is the natural logarithm of the budget allocation share of a particular department in the overall city budget; d2 is a year dummy, d2 = 1 for the period after 311 system was introduced; dT is a dummy variable that stands for the treatment group (those that have used 311), dT = 1 if department is in a treatment group, T, that is, the department uses 311 system in the postperiod. The departments that did not use the 311 system are in the comparison group, C.

Without other factors in the regression, ${\overset{\land}{α}}_{1}$ is the difference-in-differences estimator and the variable of interest for us:

{\overset{\land}{α}}_{1} = ({\bar{y}}_{2, T} - {\bar{y}}_{2, C}) - ({\bar{y}}_{1, T} - {\bar{y}}_{1, C}),

where the bar denotes average, the first subscript denotes the year, and the second subscript denotes the group. In other words, ${\overset{\land}{α}}_{1}$ shows the effect of 311 usage on the departmental share of budget allocation after the introduction of 311.

Unfortunately, the limited number of observations does not allow us to study cities separately in difference-in-differences models. Both Boston and San Francisco, however, exhibited similar trends in the budget allocation between their departments, and, therefore, can be studied together (see Figure 2 for the smoothed values of the share of budget allocations between the departments that use and do not use 311 over time for two cities). The vertical dashed lines in each of the four quadrants of Figure 2 represent the year when most departments implemented a 311 system. In Boston, 55 departments (over 90%) implemented the system in 2008; thus, the vertical line in the two Boston quadrants is set at 2008. In San Francisco, 50 departments (about 91% of total) implemented 311 in 2007; thus, the vertical line is set at 2007. The two quadrants on the left side of Figure 2 display the departments that did not use 311 and show that these departments have a relatively constant share in budget allocations before and after 311 implementation. The two quadrants on the right-hand side of Figure 2 present the share of the budget allocations for the departments that use 311, and indicate a flat to modestly decreasing slope over time. It should be noted that in the case of Boston’s departments that use 311 (upper right quadrant), the slope of this line is negative before and after the implementation, but appears steeper after the 311 implementation.

Figure 2.

Lowess smoothers of budget allocations by city.

The underlying structure of our model is based on citizen demand for services. As such, we are trying to model the demand for services as they relate to the changes in budget allocations to departments. 311 service request data provide us with the unique opportunity to measure a type of direct citizen participation across a municipal government. Admittedly, it only captures one side of the demand for services equation—demands to fix problems. We, for example, have no way of measuring demands or use of parks within the city, though we do have service request to fix problems at the park facilities. We do not know about all of the services that are used problem-free each day. What we are measuring is how the “squeaky wheel” of service requests influences the budget allocation process.

In Models 13 to 15 (Table 4), we use year fixed effect linear regressions. We conducted the Hausman postestimation test to evaluate whether the fixed or random effects model would be the most appropriate approach. The estimators were found to be systematically different; thus, the fixed-effect model was deemed to be the most appropriate. For Models 13 to 15, we also tested for serial autocorrelation using Woolridge’s (2002) method. The test results indicate no evidence of serial autocorrelation in the models. These fixed-effect panel models incorporate all years of the study period rather than the approach taken in the difference-in-differences modeling, seen in the first 12 models, that considers only two given years, one before and one after the policy change.

In our data, there are a total of 866 year-department observations, drawn from 109 departments. All of these observations are used in Model 13, which captures observations from both cities in all years, and in all governmental subunits. Models 14 and 15 capture a subset of the total sample limited by city.

Of the total 866 observations, 417 are from Boston and 449 are from San Francisco. The number of observations in Models 1 to 12 ranges from 189 to 195. The variation in the number of department observations is due to changes in government structures (creation, elimination, or merging of departments) across the study time period. Postestimation tests for all 15 models reveal the presence of heteroscedasticity across all models. Consequently, robust standard errors are used.

Table 2 provides more information on the variables used in this analysis. The mean values, standard deviations, and number of observations are reported for each variable. To provide greater insight into differences between the two cities, we have provided descriptive statistics by city, in addition to the statistics for both cities combined.

Table 2.

Descriptive Statistics.

Variable	M	SD	No. of observations
Boston
Share of Budget Allocation	1.722	4.539	417
Log of Share of Budget Allocation	−1.198	1.767	417
Dummy Variable for Years After 311 Implementation	0.446	0.498	417
Total Count of 311 Requests to the Department (in 1,000s)	0.450	3.909	417
Dummy Variable for Departments That Use 311	0.134	0.341	417
Length of Use in Years	3.060	0.545	417
Department’s Share of FTEs	0.016	0.045	417
GDP per Capita	61.309	1.492	417
San Francisco
Share of Budget Allocation	1.996	3.800	449
Log of Share of Budget Allocation	−0.894	2.0	449
Dummy Variable for Years After 311 Implementation	0.548	0.5	449
Total Count of 311 Requests to the Department (in 1,000s)	130.272	1,239.577	449
Dummy Variable for Departments That Use 311	0.693	0.462	449
Length of Use in Years	3.935	0.324	449
Department’s Share of FTEs	0.020	0.041	449
GDP per Capita	68.697	2.306	449
Total
Share of Budget Allocation	1.864	4.172	866
Log of Share of Budget Allocation	−1.040	1.9	866
Dummy Variable for Years After 311 Implementation	0.499	0.5	866
Total Count of 311 Requests to the Department (in 1,000s)	67.759	894.444	866
Dummy Variable for Departments That Use 311	0.424	0.494	866
Length of Use in Years	3.514	0.624	866
Department’s Share of FTEs	0.018	0.043	866
GDP per Capita	65.139	4.180	866

Note. FTEs = full-time equivalents; GDP = gross domestic product.

Results

In Hypothesis 1, we proposed that government functions directly tied to the 311 system will show significantly larger increases in budget share than those not tied to 311 systems. Looking at Figure 3, which shows the median share of budget allocations by city and use of 311, it appears that there are clearly differences between the departments that use the 311 systems and those that do not. It is not clear from looking at this figure, however, if the implementation of 311 had a statistically significant influence on the budget share. The black vertical line in this figure represents the year in which the majority of departments implemented 311 in both cities. While it seems as though there may have been an initial upward trend for departments using 311 in San Francisco, the trend quickly returns to positions similar to prior to implementation—and any changes upward or downward are very small in nature in real terms.

Figure 3.

Median share of budget allocation by city and use of 311.

This visual exploration is further explored in the 12 difference-in-differences models’ estimates in Table 3. The results indicate that the departments that use 311 do have between 3% and nearly 7% higher budget allocations—seen in the variable Dummy Variable for Departments That Use 311, which is statistically significant in four of the 12 models and is positively signed in all 12. However, the results of these 12 models reveal no statistically significant differences. This result indicates that differences in budget share existed prior to 311 implementation, not as a result of the implementation of 311—This result is found by examining the Interaction of Years With 311 in Use and Departments That Use 311 variable, which is not significant in any model.

Table 3.

Difference in Difference Models—Dependent Variable for All Models Is Log of Budget Allocations per Capita (Years Below Model Numbers Indicate Fiscal Years Being Compared in the Model).

	(1) 2005 \| 2011	(2) 2005 \| 2012	(3) 2005 \| 2013	(4) 2006 \| 2011	(5) 2006 \| 2012	(6) 2006 \| 2013	(7) 2007 \| 2011	(8) 2007 \| 2012	(9) 2007 \| 2013	(10) 2008 \| 2011	(11) 2008 \| 2012	(12) 2008 \| 2013
Dummy Variable for Fiscal Years With 311	0.00	0.24	1.70	−0.18	−0.06	0.14	−0.26	−0.17	−0.07	−0.32	−0.22	−0.06
Dummy Variable for Fiscal Years With 311	(0.55)	(0.49)	(1.29)	(0.33)	(0.34)	(0.61)	(0.27)	(0.28)	(0.43)	(0.25)	(0.25)	(0.34)
Dummy Variable for Departments That Use 311	5.37***	6.78***	6.38***	3.80***	4.37***	4.25***	3.73***	4.26***	4.18***	5.37***	6.91***	6.85***
Dummy Variable for Departments That Use 311	(1.22)	(1.18)	(1.16)	(1.28)	(1.24)	(1.25)	(1.18)	(0.83)	(0.84)	(1.19)	(1.22)	(1.21)
Interaction of Years With 311 in Use and Departments That Use 311	0.07	−0.32	−0.29	0.07	−0.20	−0.16	0.14	−0.13	−0.09	0.07	−0.29	−0.27
	(0.51)	(0.51)	(0.45)	(0.53)	(0.54)	(0.53)	(0.52)	(0.53)	(0.53)	(0.50)	(0.52)	(0.52)
Length of Use in Years	0.96***	0.87**	0.99***	0.00	0.04	0.05	−0.02	0.00	0.01	0.99***	0.85**	0.87**
Length of Use in Years	(0.27)	(0.35)	(0.34)	(0.36)	(0.37)	(0.37)	(0.21)	(0.22)	(0.22)	(0.28)	(0.36)	(0.36)
Interaction Between Length of 311 Use and Departments That Use 311	−1.02***	−1.40***	−1.29***	−0.58	−0.74**	−0.70*	−0.58*	−0.72***	−0.70***	−1.03***	−1.44***	−1.42***
	(0.38)	(0.37)	(0.38)	(0.38)	(0.36)	(0.36)	(0.35)	(0.24)	(0.25)	(0.38)	(0.38)	(0.38)
Department’s Share of FTEs	24.94***	25.14***	25.07***	24.92***	24.91***	24.90***	24.77***	24.76***	24.75***	24.84***	25.05***	25.06***
Department’s Share of FTEs	(3.87)	(3.88)	(3.76)	(3.92)	(3.89)	(3.80)	(3.80)	(3.77)	(3.68)	(3.85)	(3.86)	(3.77)
GDP per Capita	−0.10	−0.14	−0.41	−0.04	−0.06	−0.09	−0.02	−0.04	−0.06	−0.05	−0.07	−0.12
GDP per Capita	(0.16)	(0.12)	(0.26)	(0.10)	(0.08)	(0.15)	(0.09)	(0.08)	(0.13)	(0.09)	(0.08)	(0.14)
San Francisco Control Variable	−0.80	−0.38	1.44	−0.36	−0.13	0.15	−0.43	−0.16	−0.03	−1.06	−0.57	−0.14
San Francisco Control Variable	(0.92)	(0.57)	(1.59)	(0.77)	(0.57)	(1.09)	(0.70)	(0.55)	(1.00)	(0.65)	(0.52)	(1.06)
Constant	1.38	3.70	19.48	0.45	1.70	3.57	−0.60	0.75	1.62	−1.81	−0.01	2.95
Constant	(9.60)	(6.89)	(15.26)	(6.19)	(5.11)	(8.86)	(5.43)	(4.63)	(7.63)	(5.83)	(4.95)	(8.59)
R ²	.549	.527	.524	.529	.508	.503	.536	.514	.508	.554	.530	.525
Number of observations	189	191	191	191	193	193	192	194	194	193	195	195

Note. Robust standard errors in parentheses. FTEs = full-time equivalents; GDP = gross domestic product.

p < .1. **p < .05. ***p < .01.

We also looked at the length of time that 311 had been utilized by a given department and length interacted with whether or not 311 was being used. Looking at both the length of time variable and its interaction with use shows a general trend of a diminishing return—particularly when the departmental use of 311 variable is examined. Again, we can see this trend by examining Figure 3, which seems to show that any advantage that 311 might have produced, particularly in the case of San Francisco, seems to diminish in a few years. While there are differences seen in Figure 3, from a statistical standpoint these differences substantiated—as the control for the San Francisco is not statistically significant.

If departmental performance as measured by 311 does not produce statistically significant effects, what then is the largest predictor of budget share? The results indicate that the size of a department’s workforce, as measured by the share of FTEs in each department, is the most substantial predictor of budget share. GDP per capita is not significantly associated with increasing budget allocations in any of the 12 models. The control variable for San Francisco is not significantly associated with budget share in any of the 12 models.

The overall fit of these models, as measured by R², ranges between .503 and .549 in all 12 of the models. Thus, a sizable portion of the variation in budget allocation to each department is explained by the proposed set of independent variables.

We find no support of Hypothesis 2 (that as the number of requests increased, so too would the department’s budget allocation) in the fixed effect panel regressions (Models 13-15, Table 4). The Boston model (Model 14) is positively signed, while the combined model (Model 13) and the San Francisco Model (Model 15) are both negatively signed. None of the number of requests coefficient estimates are statistically or substantively significant. Thus, we see no evidence that the volume of 311 requests affects budget allocation decisions.⁸ As in DID Models 1 to 12, the panel data analysis also indicates that the departments using 311 have a larger budget share than those that do not use 311. As before, this effect is mediated to some extent when we look at how long a department has been using 311. As in Models 1 to 12, the budgetary advantage of 311 user departments diminishes, though it does not disappear.

Table 4.

Year Fixed Effect Linear Regressions—Dependent Variable for All Models Is Log of Budget Share.

	(13)Both cities	(14)Boston	(15)San Francisco
Total Count of 311 Requests to the Department (in 1,000s)	−0.00***	0.02***	−0.00***
Total Count of 311 Requests to the Department (in 1,000s)	(0.00)	(0.02)	(0.00)
Dummy Variable for Departments That Use 311	4.64***	3.39***	7.52***
Dummy Variable for Departments That Use 311	(0.63)	(0.70)	(1.69)
Length of 311 use (in years)	0.16	−0.30**	1.26***
Length of 311 use (in years)	(0.12)	(0.14)	(0.28)
Interaction Between Length of Use and Departments That Use 311	−0.83***	−0.42*	−1.60***
	(0.17)	(0.22)	(0.43)
Department’s Share of FTEs	25.54***	24.19***	27.67***
Department’s Share of FTEs	(1.11)	(1.27)	(1.89)
San Francisco Control Variable	−0.60***
San Francisco Control Variable	(0.14)
Constant	−2.34***	−0.93**	−7.24***
Constant	(0.39)	(0.42)	(1.11)
R ²	.516	.585	.495
Number of observations	866	417	449

Note. Robust standard errors in parentheses. FTEs = full-time equivalents.

p < .1. **p < .05. ***p < .01.

Again the most substantial predictor of budget share is the size of a department’s FTE workforce. The control for the share of FTEs is again statistically significant and positively related to the budget allocation.

In Model 13, we utilize a San Francisco control variable, which shows a statistically significant negative difference between that city and Boston. This indicates that the share of budget allocated to the individual departments is substantively smaller for San Francisco compared with Boston, but it does not say anything specific about policy decisions more broadly.

For all three fixed-effect panel models, we find that we have R² figures ranging from .49 to .58. This again demonstrates that a sizable portion of our dependent variable’s variation is explained by our chosen set of independent variables.

Conclusions and Policy Implications

In an increasingly wired society, government leaders need to adapt to technologies and means of communication that are prevalent and preferred by citizens. The use of 311 systems and affiliated technologies (smartphone apps, websites, and utilization of social media) is without a doubt going to play more prominent roles in the citizen–government interface—particularly given the digital native status of younger generations. If citizen–government interactions are to be used as a supplement to the citizen participation process, we need to understand the extent to which a relationship between the budget allocations and this type of citizen participation exists.

This article’s initial look at the influence of a new kind of citizen participation in government has shed some light on the limitations of citizen influence on the budget process. We are able to provide evidence that indirect citizen participation in the budget process is not influencing budget allocation on a widespread manner. Apparently the “squeaky wheel” citizens are not quite squeaky enough yet to influence budget allocations—even if they are impacting the livability of their cities (Clark & Shurik, 2016). We do recognize that not all types of service requests are going to take the same amount of resources to address, but undertaking a cost accounting study to determine average cost per request would be an extraordinary burden and is clearly a limitation of our findings.

The results demonstrate what many were already aware of: Performance metrics, in whatever form they are generated, do not seem to drive budget policy, even if they can be useful for management. Data derived from 311 are increasingly showing up in performance management and budget documents in Boston (City of Boston, 2012b; City of Boston Performance Management System, 2013) and San Francisco (Office of the Controller/City and County of San Francisco, 2012a, 2013), but they are also seen in other cities across the nation, including New York City (NYC Mayor’s Office of Operations, 2013); Chicago (City of Chicago, 2013); Washington, DC (The Government of the District of Columbia, 2013); Knoxville, TN (City of Knoxville, 2013); and Pittsburgh, PA (City of Pittsburgh, 2012). While it is apparent that cities recognize the value of the information they collect from 311 in measuring performance, there is no support to suggest that this form of citizen participation has a role in altering how funds are allocated to departments.

Clearly, governments do not rely solely upon 311 requests as a means to allocate resources. People contact their governments to report a problem—not to report the stellar service they just received. This provides an inherent bias in the information source. Nonetheless, local government managers and leaders need to heed this information fully and carefully. Failing to address problems reported by hundreds or perhaps thousands of residents would be foolish; but only addressing these issues would fail to address all of the needs of a city and its residents. However, earlier studies (Clark & Brudney, 2016a; Clark et al., 2013) have demonstrated that the production function that underpins 311 service requests is not systematically biased, that is, delivering more services to particular groups or neighborhoods within the city. Clark et al. (2013) found that citizen-generated service requests were complementing those service requests being generated internally by government employees. As 311 is used more and more, perhaps we will see a day in which resources can be freed up in one area of the government and be shifted to another because more citizens are providing information needed to identify and solve problems for the government—but so far the volume of requests and the evidence presented here does not suggest we are there yet.

Robert Behn (2003) describes eight purposes for performance management in government, one of which is budgeting. He states that performance measures can help managers to decide to which “Programs, People, or Projects” to allocate resources in the budget (Behn, 2003). The National Academy of Public Administration stated in 1991 that “performance monitoring should be an essential part of program administration and the budget process” (Wholey & Hatry, 1992, p. 607). At the federal level, the implementation of GPRA and PART puts performance measurement front and center in the budget process. It is clear, however, that performance information is scarcely used by public officials in the allocation of resources (Joyce, 2007; Moynihan, 2008). This federal-level experience is echoed at the state and local level, where it is apparent that the effects of performance information on budget allocations were limited in nature (Jordan & Hackbart, 2005; Melkers & Willoughby, 2005; O’Toole & Stipak, 2002; Poister & Streib, 1999; Wang, 2002; Willoughby & Melkers, 2000), even if managers appreciate and use the information to manage their programs.

A number of next steps would be prudent based on the findings of this article. First, we may not be looking “in the right places” (Joyce, 2007, p. 21) to see the link between citizen participation and the budget allocations. Behn (2003) and Joyce (2007) both contend that we need to take a more micro approach to find the appropriate connection between the performance measurement and budget allocation. This would mean looking more closely at the intradepartmental budget allocation process, rather than interdepartmental shifts in a budget, to better understand how performance measurement is shaping the budget process. This would lead us to believe that the focus of future research should be on allocations within the department, rather than within a government more broadly. It is conceivable that the increase in requests of a particular type is driving resource allocations within one department for a particular type of service as a result of 311 requests, something we cannot see when looking at the governments as a whole. It is entirely possible that individual departments might not expect a substantial across the board budget increase, and thus are finding ways to move funds internally to accommodate the increased citizen-generated data that drive workloads toward some services and away from others.

Next, we fully recognize that utilizing a sample of only two cities has its limitations, including some challenges in inferring beyond Boston and San Francisco. However, both San Francisco and Boston can still provide illustrative examples as leading cases (McDavid et al., 2012). The rapid rate of technological adoption, particularly mobile applications, indicates that other cities will begin to look more and more like these two cities technologically, rather than less (Clark & Brudney, 2016a; Clark et al., 2013; Thomas, 2012); thus, these leading cases will become increasingly representative.

And finally, further research will have to establish whether the use of this new measure of citizen participation in the budget process closes the gap between what citizens expect from their government and their willingness to pay. A more open and responsive government, a primary goal of most 311 systems, should in theory lead citizens to have more trust in their government and more faith that their government is spending their money wisely. However, that too is a question for further research.

Footnotes

Appendix

Acknowledgements

We would like to thank Adrienne Crawford for her research assistance on the initial draft of the article and Maria Shurik for her assistance in collecting data for the final version of the article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Author Biographies

Benjamin Y. Clark joined the faculty of the Department of Planning, Public Policy and Management at the University of Oregon in 2016 after six years at the Levin College of Urban Affairs at Cleveland State University. His research focuses on local government management broadly, and more specifically on 311 systems, coproduction, public sector crowdsourcing, and budgetary/financial management.

Tatyana S. Guzman is an assistant professor at the Levin College of Urban Affairs at Cleveland State University. Her research interests are in municipal finance, taxation, and education finance.

References

Alford

(2002). Why do public-sector clients coproduce? Administration & Society, 34, 32-56. doi:10.1177/0095399702034001004

Andrews

Shah

(2005). Citizen-centered governance: A new approach to public sector reform. In Shah

(Ed.), Public expenditure analysis (pp. 153-216). Washington, DC: The International Bank for Reconstruction and Development.

Behn

R. D.

(2003). Why measure performance? Different purposes require different measures. Public Administration Review, 63, 586-606.

Berner

(2001). Citizen participation in local government budgeting. Popular Government, 66(3), 23-30.

Bureau of Economic Analysis. (2013, September 17). Per capita real GDP by metropolitan area (chained 2005 dollars). Retrieved from http://www.bea.gov/iTable/iTable.cfm?reqid=70&step=1&isuri=1&acrdn=2#reqid=70&step=10&isuri=1&7007=-1&7036=-1&7090=70&7035=-1&7006=1446041860&7001=21000&7002=2&7003=1000&7004=NAICS&7005=1&7093=Levels

City and County of San Francisco. (2002). Composition and salary, charter of the city and county of San Francisco § Section 2.101. Retrieved from http://www.amlegal.com/nxt/gateway.dll/California/charter_sf/articleiilegislativebranch?f=templates$fn=default.htm$3.0$vid=amlegal:sanfrancisco_ca$anc=JD_ 2.100

City and County of San Francisco. (2004). Consolidated budget and annual appropriation ordinance: Fiscal year ending June 30, 2005 (No. File No. 040725, ordinance 197-04). San Francisco, CA. Retrieved from http://www.sfcontroller.org/Modules/ShowDocument.aspx?documentid=82

City and County of San Francisco. (2005). Consolidated budget and annual appropriation ordinance: Fiscal year ending June 30, 2006 (No. File No. 051130, ordinance 203-05). San Francisco, CA. Retrieved from http://www.sfcontroller.org/Modules/ShowDocument.aspx?documentid=83

City and County of San Francisco. (2006). Consolidated budget and annual appropriation ordinance: Fiscal year ending June 30, 2007 (No. File No. 060729, ordinance 202-06). San Francisco, CA. Retrieved from http://www.sfcontroller.org/Modules/ShowDocument.aspx?documentid=84

10.

City and County of San Francisco. (2007). Consolidated budget and annual appropriation ordinance: Fiscal year ending June 30, 2008 (No. File No. 070773, ordinance 174-07). San Francisco, CA. Retrieved from http://www.sfcontroller.org/Modules/ShowDocument.aspx?documentid=70

11.

City and County of San Francisco. (2008). Consolidated budget and annual appropriation ordinance: Fiscal year ending June 30, 2009 (No. File No. 080725, ordinance 140-08). San Francisco, CA. Retrieved from http://www.sfcontroller.org/Modules/ShowDocument.aspx?documentid=71

12.

City and County of San Francisco. (2009). Consolidated budget and annual appropriation ordinance: Fiscal year ending June 30, 2010 (No. File No. 090779, ordinance 184-09). San Francisco, CA. Retrieved from http://www.sfcontroller.org/Modules/ShowDocument.aspx?documentid=72

13.

City and County of San Francisco. (2010). Consolidated budget and annual appropriation ordinance: Fiscal year ending June 30, 2011 (No. File No. 100701, ordinance 190-10). San Francisco, CA. Retrieved from http://www.sfcontroller.org/Modules/ShowDocument.aspx?documentid=991

14.

City and County of San Francisco. (2011). Consolidated budget and annual appropriation ordinance: Fiscal year ending June 30, 2012 (No. File No. 440387, ordinance 147-11). San Francisco, CA. Retrieved from http://www.sfcontroller.org/Modules/ShowDocument.aspx?documentid=2390

15.

City and County of San Francisco. (2012). Consolidated budget and annual appropriation ordinance: Fiscal year ending June 30, 2013 and fiscal year ending June 30, 2014 (No. File No. 120591, ordinance 164-12). San Francisco: CA. Retrieved from http://sfcontroller.org/Modules/ShowDocument.aspx?documentid=3358

16.

City and County of San Francisco. (2015). 2015 San Francisco city survey. San Francisco, CA. Retrieved from http://sfcontroller.org/Modules/ShowDocument.aspx?documentid=6652

17.

City of Boston. (2004, June). The FY05 adopted budget. Boston, MA: Author.

18.

City of Boston. (2005, June 21). The FY06 adopted budget. Boston, MA: Author. Retrieved from http://www.cityofboston.gov/budget/fy06.asp

19.

City of Boston. (2006, June 20). The FY07 adopted budget. Boston, MA: Author. Retrieved from http://www.cityofboston.gov/budget/fy07.asp

20.

City of Boston. (2007, June 19). The FY08 adopted budget. Boston, MA: Author. Retrieved from http://www.cityofboston.gov/budget/fy08.asp

21.

City of Boston. (2008, June 18). The FY09 adopted budget. Boston, MA: Author. Retrieved from http://www.cityofboston.gov/budget/fy09.asp

22.

City of Boston. (2009, June 19). The FY10 adopted budget. Boston, MA: Author. Retrieved from http://www.cityofboston.gov/budget/fy11.asp

23.

City of Boston. (2010, June 22). The FY11 adopted budget. Boston, MA: Author. Retrieved from http://www.cityofboston.gov/budget/fy11.asp

24.

City of Boston. (2011, June 21). The FY12 adopted budget. Boston, MA: Author. Retrieved from http://www.cityofboston.gov/budget/fy12.asp

25.

City of Boston. (2012a, June 18). Boston about results—Performance goals. Retrieved from http://www.cityofboston.gov/images_documents/06%20Budget%20and%20Performance%20Goals_final_tcm3-37457.pdf

26.

City of Boston. (2012b, June 18). The FY13 adopted budget. Boston, MA: Author. Retrieved from http://www.cityofboston.gov/budget/fy13.asp

27.

City of Boston. (2013, December 16). Boston City Council members. Retrieved from http://www.cityofboston.gov/citycouncil/councillors/

28.

City of Boston Performance Management System. (2013, September). Boston about results: Performance management—Public works achievements. Retrieved from http://www.cityofboston.gov/bar/scorecard/details.html?contextId=112823395596547&isExternal=_external

29.

City of Chicago. (2013, December 1). Service delivery metrics. Retrieved from http://www.cityofchicago.org/city/en/narr/foia/key_performance_indicators0/city_management_benchmarks.html

30.

City of Knoxville. (2013). FY 2013/14 adopted operating budget. Retrieved from http://www.cityofknoxville.org/budget/2013/finalbudget.pdf

31.

City of Pittsburgh. (2012, December). Managing for results reports: 311 customer service. Retrieved from http://apps.pittsburghpa.gov/mayor/311_PittMAPS_December.xls

32.

Clark

B. Y.

Brudney

J. L.

(2016a). Representation and beyond: Coproduction and the new information technology (SSRN Scholarly Paper No. ID 2473543). Rochester, NY: Social Science Research Network. Retrieved from http://papers.ssrn.com/abstract=2473543

33.

Clark

B. Y.

Brudney

J. L.

(2016b, February). Too much of a good thing? Frequent flyers and the implications for the crowdsourcing & coproduction of public service delivery. Presented at the Computer-Supported Cooperative Work and Social Computing, San Francisco, CA.

34.

Clark

B. Y.

Brudney

J. L.

Jang

S.-G.

(2013). Coproduction of government services and the new information technology: Investigating the distributional biases. Public Administration Review, 73, 687-701. doi:10.1111/puar.12092

35.

Clark

B. Y.

Shurik

(2016). Do 311 systems shape citizen satisfaction with local governments? In Gibson

Julnes

(Eds.), Innovations in the public and nonprofit sectors: A public solutions handbook (pp. 147-166). Armonk, NY: M.E. Sharpe.

36.

Clark

B. Y.

Zingale

Logan

Brudney

J. L.

(2016). A framework for using crowdsourcing in government. International Journal of Public Administration in the Digital Age. Retrieved from http://papers.ssrn.com/abstract=1868283

37.

de Sousa Santos

(1998). Participatory budgeting in Porto Alegre: Toward a redistributive democracy. Politics & Society, 26, 461-510.

38.

Duncombe

(1996). Public expenditure research: What have we learned? Public Budgeting & Finance, 16(2), 26-58.

39.

Dworak

R. J.

(1980). Taxpayers, taxes, and government spending: Perspectives on the taxpayer revolt. New York, NY: Praeger.

40.

Franklin

A. L.

A. T.

Ebdon

(2009). Participatory budgeting in Midwestern states: Democratic connection or citizen disconnection? Public Budgeting & Finance, 29(3), 52-73. doi:10.1111/j.1540-5850.2009.00936.x

41.

The Government of the District of Columbia. (2013, August 8). FY 2014 proposed budget and financial plan, Volume 2 agency budget chapters—Part I: Governmental direction and support, economic development and regulation, and public safety and justice. Retrieved from http://cfo.dc.gov/sites/default/files/dc/sites/ocfo/publication/attachments/DCOCFO_Volume_2_Final.pdf

42.

Guo

H. (David)

Neshkova

M. I.

(2013). Citizen input in the budget process: When does it matter most? The American Review of Public Administration, 43, 331-346. doi:10.1177/0275074012443588

43.

A. T.

(2007a). Citizen participation in performance measurement. In Box

R. C.

(Ed.), Democracy and public administration (pp. 107-118). Armonk, NY: M.E. Sharpe.

44.

A. T.

(2007b). Exploring the roles of citizens in performance measurement. International Journal of Public Administration, 30, 1157-1177. doi:10.1080/01900690701225341

45.

Jordan

M. M.

Hackbart

(2005). The goals and implementation success of state performance-based budgeting. Journal of Public Budgeting, Accounting & Financial Management, 17(4). Retrieved from http://search.ebscohost.com/login.aspx?direct=true&profile=ehost&scope=site&authtype=crawler&jrnl=10963367&AN=19384466&h=pqkptLfilhqfO2SoT0I7b%2Bn5RE8xgRywAriEj2CbJjGBSdga%2Fwpxb82Jrn%2FuSu0nDeZGFd5EVbB8xG2bVQXE9Q%3D%3D&crl=c

46.

Joyce

P. G.

(2007). Linking performance and budgeting: Opportunities in the federal budget process. In Breul

J. D.

Moravitz

(Eds.), Integrating performance and budgets: The budget office of tomorrow (pp. 19-70). Lanham, MD: Rowman & Littlefield.

47.

McDavid

J. C.

Huse

Hawthorn

L. R. L.

(2012). Program evaluation and performance measurement: An introduction to practice (2nd ed.). Los Angeles, CA: Sage.

48.

Melkers

Willoughby

(2005). Models of performance-measurement use in local governments: Understanding budgeting, communication, and lasting effects. Public Administration Review, 65, 180-190.

49.

Meyer

(2013, November 15). INVESTIGATOR: 311 calls for help a mystery. WKYC. Retrieved from http://www.wkyc.com/story/news/investigations/2013/11/13/311–calls-help-problems/3508331/

50.

Moynihan

D. P.

(2008). The dynamics of performance management: Constructing information and reform. Washington, DC: Georgetown University Press.

51.

NYC Mayor’s Office of Operations. (2013, September). Mayor’s management report: Department of Sanitation. Retrieved from http://www.nyc.gov/html/ops/downloads/pdf/mmr2013/dsny.pdf

52.

Office of the Controller/City and County of San Francisco. (2012a, October 30). ANNUAL YEAR-END PERFORMANCE MEASURE REPORT v1.01. Retrieved from http://sfcontroller.org/Modules/ShowDocument.aspx?documentid=3669

53.

Office of the Controller/City and County of San Francisco. (2012b, November 1). Citywide performance measures data: FY 11-12 dataset. Retrieved from http://www.sfcontroller.org/modules/showdocument.aspx?documentid=3671

54.

Office of the Controller/City and County of San Francisco. (2013, February 14). GOVERNMENT BAROMETER: Quarter 2, Fiscal Year 2013. Retrieved from http://www.sfcontroller.org/Modules/ShowDocument.aspx?documentid=4007

55.

O’Toole

D. E.

Stipak

(2002). Productivity trends in local government budgeting. Public Performance & Management Review, 26, 190-203. doi:10.2307/3381277

56.

Poister

T. H.

Streib

(1999). Performance measurement in municipal government: Assessing the state of the practice. Public Administration Review, 59(4), 325-335.

57.

Roberts

(2008). The age of direct citizen participation. Armonk, NY: M.E. Sharpe.

58.

Shah

Shen

(2007). Citizen-centric performance budgeting at the local level. In Shah

(Ed.), Local budgeting (pp. 151-178). Washington, DC: The International Bank for Reconstruction and Development.

59.

Thomas

J. C.

(2012). Citizen, customer, partner: Engaging the public in public management. Armonk, NY: M.E. Sharpe.

60.

Wang

(2002). Assessing performance measurement impact: A study of U.S. local governments. Public Performance & Management Review, 26, 26-43. doi:10.2307/3381296

61.

Wholey

J. S.

Hatry

H. P.

(1992). The case for performance monitoring. Public Administration Review, 52, 604-610. doi:10.2307/977173

62.

Willoughby

K. G.

Melkers

J. E.

(2000). Implementing PBB: Conflicting views of success. Public Budgeting & Finance, 20(1), 85-120. doi:10.1111/0275-1100.00006

63.

Woolridge

J. M.

(2002). Econometric analysis of cross section and panel data. Cambridge, MA: MIT Press.