The many outcomes from contracting out: The voice of public managers

Abstract

This paper argues that adopting a stakeholder approach to the study of contracting outcomes produces more rich and rounded representations of the realities of the contracting out of public services. We revisit the research on contracting outcomes, highlighting the public manager perspective as key for gaining deeper, more detailed insights. The public manager perspective is explored in an inductive analysis of answers to open-ended survey questions collected from public managers with contracting experience within the context of municipal park management in Scandinavia. The emerging managerial perspective is summarized in a best-case, worst-case and complex-case scenario highlighting the mix, complexities and trade-offs in a composite set of contracting outcomes. The nature of contracting outcomes as complex and composite rather than unidimensional and clear-cut is one key finding. Furthermore, the importance of some specific outcomes (e.g. learning) complements existing research themes. Our findings sustain the initial argument, demonstrating how the stakeholder approach can produce new insights. A key implication is that future research can benefit from assessing contracting outcomes by providing voice to multiple stakeholders.

Keywords

Contracting out marketization outcomes public management qualitative analysis stakeholder

Introduction

Since the 1970s–1980s, contracting out has become a well-established strategy for reforming and organizing public service delivery in most developed economies as part of New Public Management-oriented reforms (Hansen and Lindholst, 2016; Pollitt and Bouckaert, 2011). In a policy evaluation perspective, long-standing key questions for the implementation of contracting out, as a particular strategy for privatization and marketization in the public sector, are whether this strategy delivers on its ‘promises’ as well as what the ‘broader’ outcomes are (Lundqvist, 1989).

The literature on contracting outcomes reveals that the current stock of knowledge is predominantly based on quantitative approaches and evaluation criteria driven by economic perspectives concerned with public expenditures, cost savings and to some degree quality effects – a concern tightly aligned with official objectives – or the promises – in most public policies and reforms promoting contracting out (Alonso et al., 2017; Bel et al., 2010; Boyne, 1998; Domberger and Jensen, 1997; Donahue, 1989; Hansen, 2010b; Hodge, 2000; Petersen et al., 2017).

However, the predominant approach to the study of contracting outcomes emerges as somewhat ‘narrow’ when contrasted with (a) the broader range of approaches and criteria of potential relevance identified in the general literature on public management and performance (Andersen et al., 2016; Boyne, 2002; Hansen, 2017; Pollitt and Bouckaert, 2011; Walker and Andrews, 2015); (b) the few studies that have approached contracting outcomes from alternative perspectives, such as citizens’ perceptions (Andrews and Van de Welle, 2013), consequences for staff (Vrangbæk et al., 2015), accountability issues (Hodge and Coghill, 2007; Mulgan, 2006), challenges for management (Kettl, 2010; Lindholst and Bogetoft, 2011) and the distribution of power (Hansen, 2010a); (c) research emphasizing the multidimensional and composite characteristics of contracting outcomes (Amirkhanyan et al., 2010; Fernandez, 2009; Lindholst, 2017) or (d) qualitative studies enabling broader narrative accounts (Jones, 2000; Romzek and Johnston, 2002). Furthermore, several authors have highlighted the importance of specifying the stakeholder perspective in performance evaluations, as different stakeholder groups represent substantial differences in interests, values, insights and worldviews (Davis et al., 2016; Radin, 2006).

The counterpositions above make clear that the predominant approach leaves us with partial and potentially misrepresentative accounts of the range of contracting outcomes of relevance for involved key stakeholders (e.g. citizens/tax payers, service users, politicians, public managers, staff, private businesses). We therefore argue that the characteristics of and criteria for evaluating contracting outcomes must be revisited and conferred beyond the narrow remnants in the predominant research perspective in order to advance more ‘rounded’, representative evaluations of contracting outcomes.

In order to gauge the relevance of more rounded approaches, this paper provides an exploration of the significant dimensions and characteristics of contracting outcomes from the perspective of the public managers, who play a pivotal role in the realization and delivery of reform and policy objectives by implementing contracting out as a particular way of managing and organizing public service provision. Given their pivotal role in the implementation of contracting out as key internal stakeholders (Brown and Potoski, 2003; Davis et al., 2016; Kettl, 2010; Lindholst and Bogetoft, 2011), public managers must be expected to hold profound experience and detailed insights (albeit still partial) into the contracting outcomes. By adding insights on contracting outcomes based on the voices of public managers (as yet another research perspective), the paper suggests that research can deliver inputs to evaluations of contracting outcomes.

The paper studies public managers’ views on contracting outcomes within the context of the provision of maintenance services in municipal parks and green spaces – a traditional context for the use of contracting out as well as a ‘most likely setting’ for its successful implementation due to well-suited key transactional characteristics in terms of relatively low demands for ‘specialized investments’ and a relative ease of ‘measurability’ (Brown and Potoski, 2005; Hefetz and Warner, 2012). The study relies on a qualitative data set generated in 2013 on municipal park and green space managers’ open assessments of negative and positive outcomes from the contracting out of maintenance services in municipal parks and green spaces in the two Scandinavian countries of Denmark and Sweden – a context where contracting out has been promoted and become widespread since the 1990s (Bretzner et al., 2016; Lindholst et al., 2016). The data set allows us to explore more openly and evaluate the range of contracting outcomes from the viewpoint of public managers who are responsible for managing service provision by contracts. In order to advance our overall knowledge of contracting outcomes in the public sector, our research is guided by the following research question:

How does the range of contracting outcomes perceived as important by public managers compare to the range of outcomes highlighted in the literature?

The paper is outlined as follows: First, theories and arguments concerning contracting outcomes in the public sector are reviewed. Second, our data and research methods are described, including a short description of the study's context. Third, key findings from our study are presented. Fourth, the findings are discussed and compared to current knowledge. Finally, a brief conclusion summarizes our findings and puts them into perspective.

Public managers and contracting outcomes revisited

A reading of the key literature on contracting out highlights how several conceptualizations of contracting outcomes are available. Overall, research interests in contracting outcomes in the private delivery of public services have ranged from a narrow, unidimensional focus on economic performance (e.g. Boyne, 1998; Hodge, 2000; Petersen et al., 2017) to broader multidimensional performance foci, such as various forms of ‘contract accountability’ and different types of outcomes and impacts (e.g. Amirkhanyan et al., 2007; Andrews and Van de Welle, 2013; Fernandez, 2009; Lindholst, 2017; Romzek and Johnston, 2005). While some scholars (e.g. Hodge, 2000) have provided summaries and discussions of different types of outcomes, no authoritative lists of contracting outcomes are readily available. Over the years, most of the studies and systematic reviews have explored contracting outcomes, with a predominantly policy-driven interest in economic outcomes related to expenditure levels and cost savings, and to some degree addressed quality issues (Alonso et al., 2017; Bel et al., 2010; Boyne, 1998; Domberger and Jensen, 1997; Donahue, 1989; Hodge, 2000; Petersen et al., 2017). A few studies have studied and discussed problems with the quality of privately produced services more independently through the concept of ‘accountability’ (e.g. Hodge and Coghill, 2007; Lamothe and Lamothe, 2010; Mulgan, 2006). In the predominant economic perspective, however, other outcomes are often discussed as moderating effects or externalities rather than unique outcomes of inherent interest and value. This is the case in discussions of ‘quality-shading’ effects (e.g. Hart et al., 1997) or ‘transaction costs’ as moderating effects on direct cost savings (e.g. Alonso et al., 2017; Hodge, 2000). However, from a managerial perspective, as highlighted by Lindholst and Bogetoft (2011) and O'Flynn and Alford (2008), it also becomes clear that activities associated with the transaction costs of contracting out (preparing, procuring and monitoring contracts) and the required organizational changes represent a fundamentally different way of managing public service delivery – imbued with a different set of challenges, advantages and disadvantages – compared to bureaucratic modes of service delivery based on internal organizational routines and professional expertise. The implementation of contracting out is intrinsically associated with substantial changes in the organizations wherein public managers operate, entailing change in organizational structures, governing values and cultural outlook through shifts from bureaucratic modes of service provision to market-based modes (Hood, 1991, 1998). Key features of organizational change include the disaggregation of former ‘monolithic’ hierarchies into clearly demarcated and separate ‘purchaser’/‘provider’ roles, each with greater autonomy, greater focus on organizational output and efficiency rather than common goal settings together with a shift in cultural outlook from a collectivistic mind-set towards an individualistic mind-set. Likewise, Patterson and Pinch (1995) observed that the radical British approach to contracting out in the 1980s and 1990s entailed a shift towards ‘strategic centralization’ and ‘operational decentralization’ together with the emergence of a ‘business’ culture in public organizations. Organizational changes have impacted the conditions and responsibilities for public management, and research has identified some managerial outcomes. O'Flynn and Alford (2008), for example, highlighted a dilemma between the requirements to separate organizational responsibilities and those for accessing sufficient knowledge for well-working contract management as one key outcome for public management. Likewise, employees in public-service providing organizations have been found to be impacted significantly by the implementation of contracting out by, for example, adverse changes in employment rates and terms (Vrangbæk et al., 2015). The impact of contracting has also been discussed from the viewpoint of broader governance perspectives. Rhodes (1994) discussed whether the shift towards contracts in public service delivery would limit the scope for political and democratic decision making and suggested a ‘hollowing out’ hypothesis. Congruently, Hodge (2000) provided short overviews and discussions on the ‘democratic’, ‘legal’ and ‘political’ performance of privatization (including contracting out) in the public sector. In brief, the major types of contracting outcomes highlighted in the literature revisited above can be subsumed under headings for economic, service, managerial, organizational, employment and governance outcomes. However, the predominant approach and most research have entailed a narrow, unidimensional interest in economic outcomes.

Beyond the literature on contracting out, a general literature on performance in relation to public management and the public sector has developed more systematic, general approaches to conceptualizations of performance and identifying evaluation criteria of potential interest (e.g. Andersen et al., 2016; Boyne, 2002; Hansen, 2017; Walker and Andrews, 2015). Working with the 3E (economy, efficiency and effectiveness) and input–output–outcome (IOO) models, Boyne (2002) developed a set of five main ‘performance domains’ related to ‘outputs’, ‘efficiency’, ‘service outcomes’, ‘responsiveness’ and ‘democratic outcomes’, which were further divided into 15 ‘subdomains’. Similarly, Hansen (2017) identified a general list of six commonly used (‘institutionalized’) performance criteria – ‘economy’, ‘efficiency’, ‘effectiveness’, ‘equity and fairness’, ‘responsiveness’ and ‘quality’ – which must be evaluated from particular ‘points of references’. Scholars have emphasized that outcomes are multidimensional, can (or must) be viewed from the perspective of multiple stakeholders, are difficult to measure due to inherent measurement errors in both objective and subjective measures, and involve long and somehow uncertain causal chains (Boyne et al., 2006; Pollitt and Bouckaert, 2011). Given that reliance on ‘objective’ performance measures have become challenged and reliance on ‘narrow’ performance measurements have become ‘risky’, shifts have been warranted towards approaches acknowledging the inherent subjectivity in performance measures and openings towards the perspectives of multiple stakeholders (Brewer, 2006; Cuganesan et al., 2014; Radin, 2006; Schachter, 2010). Indeed, concepts like ‘performance’, ‘efficiency’ and ‘effectiveness’ are value-laden and their relevance and content vary across various stakeholders. In order to provide a systematic conceptual approach beyond lists of performance criteria used for evaluations in various settings, Andersen et al. (2016) proposed a framework for capturing conceptualizations of performance based on distinctions between questions related to ‘stakeholders’, ‘formality’, ‘subjectivity’, ‘type of processes’, ‘type of products’ and ‘unit of analysis’. Clearly, any list of relevant criteria is seldom complete or fully comprehensive. Consequently, it is stressed that the relevance of any list of criteria adopted for performance evaluations must be justified against context and research interests.

What is left from the perspective of the research on contracting out is to identify and further specify the range of outcomes (e.g. domains, institutionalized criteria, distinctions) by varying interests. One key question in this endeavour is to acknowledge the stakeholder perspective and explore what can be gained from including this. Public managers responsible for service provision contracted out to private service providers are key stakeholders and play pivotal roles when implementing contracting out, as they are routinely involved in the practicalities of contracting decisions (Brown and Potoski, 2003; Davis et al., 2016; Lindholst and Bogetoft, 2011; Yang et al., 2009). They are involved in developing and preparing tender and contract materials, carrying out procurement processes, evaluating tenders, overseeing contracts and monitoring private contractor service delivery. Public managers are also responsible for ensuring that policy and financial objectives are achieved and can be involved in the development of contracting policies and decisions (Nuppenau, 2009). The introduction of contracting out in the public sector has required public managers to learn and assume new roles. Kettl (1993) highlighted the need for public managers to become ‘smart buyers’ by learning to address the most fundamental contracting questions (e.g. ‘what to buy’, ‘from whom to buy’, ‘what has been bought’). Similarly, theory has emphasized how organizing economic exchange through contracts entails needs for developing competitive markets (public choice theory) and investing in and implementing sturdy management instruments in terms of information systems (agency theory) and ‘safeguards’ (transaction cost theory) in order to ensure contractual compliance and mitigate contractual hazards and risks. More generally, these functions have been considered part of a set of contracting capabilities encompassing the assessment of feasibility, implementation and evaluation of contracting out (Brown and Potoski, 2003).

Due to their pivotal role(s) in the implementation of contracting out, public managers can be anticipated to hold deep, detailed insight into the positive and negative impacts of implementation. This insight can be anticipated to cover the predominant focus on finance and service quality, go beyond such focus and encompass impacts on management itself and the organizations wherein management operates on a daily basis as well as diverging from the perspective of other key stakeholders (policymakers, service staff, citizens).

Materials and methods

Context of study

The study is couched within the context of the provision of maintenance services in municipal parks and green spaces in Denmark and Sweden. Within an urban environment, such spaces are integrated parts of a larger green infrastructure, which provides a range of urban functions of social, economic and environmental value (Millennium Ecosystem Assessment, 2005). In this context, proper maintenance service management and delivery are essential for the upkeep and development of the value and multi-functionality of urban parks and green spaces (Dempsey and Burton, 2012). Financially, however, spending on park services represents a minor fraction of total public expenditures in the two countries. According to national statistic bureaus, less than 1% of total local government service expenditures in each country in 2015 was spent on public parks, green spaces and other outdoor recreational areas (author's own calculations).

Compared to other local government services, parks maintenance can be regarded as a ‘most likely setting’ for the implementation of contracting out due to well-suited transactional characteristics in terms of relatively low demands for ‘specialized investments’ and the ease of ‘measurability’, also expressed as low levels of ‘contracting difficulties’ (Brown and Potoski, 2005; Hefetz and Warner, 2012). The two Scandinavian countries of Denmark and Sweden represent ‘almost similar’ institutional contexts, where contracting out at an early stage has been promoted and developed by a pragmatic policy push within a decentralized institutional structure (Bryntse and Greve, 2002). Consequently, contracting out has become a common arrangement implemented by the mid-2010s by most local governments in both countries for providing park and green space maintenance services. Bretzner et al. (2016) reported that 54% of Swedish municipalities used private park maintenance to some degree, while Lindholst et al. (2016) reported that 81% of Danish municipalities used some measure of private services. In both countries, the most common arrangement for service provision was found to be a private/public mix. Given the respective histories with the contracting out of park maintenance in Sweden and Denmark, both countries should provide ample experiences with contracting out.

Data collection

Our analysis relies on a qualitative data set generated by an open-ended question in a self-administered web-based survey distributed in 2013 to all of the municipal departments responsible for park and green space management in Denmark and Sweden. The survey was designed and distributed to explore the range of outcomes from contracting out municipal park and green space maintenance as perceived by the responsible public managers. Key respondents in both countries were identified from membership lists for professional associations as well as information on municipal websites. The professional associations in Denmark and Sweden were ‘Municipal Park and Nature Managers’ [Kommunale Park-og Naturforvaltere] and the ‘Association for Swedish City Gardeners’ [Föreningen Sveriges Stadsträdgårdsmästare], respectively. Key selection criteria for respondents were a position as mid-level manager within the overall municipal hierarchy combined with a key position in a department responsible for parks and green space services – characteristics commonly entailing responsibilities for and insights into strategic and operational domains of service delivery. Within the municipal hierarchy, these are often mid-level managers overseeing one or two layers of lower-level managers and with direct reference to top-level managers on the municipal board. They are usually also responsible for representing the administrative level in relations with the political level.

Within municipal organizations, the strategic and operational management of parks and green spaces is integrated to some degree with road services. Given the municipal profile and organization, strategic and/or operational responsibilities for parks are possibly integrated within a ‘road department’ or joint ‘road and park’ department. Typical job titles, somewhat difficult to translate, were ‘chief park officer’ [ parkchef ], ‘chief road officer’ [vejchef ] or ‘head city gardener’ [stadsgartner/stadsträdgårdsmästar] – the historical title for the highest ranked administrative officer within park administrations in Denmark and Sweden.

The survey consisted of a few questions (nine items in total) related to whether the municipality contracted out green space maintenance, the use of legal frameworks, the historical, current and expected contracting levels as well as an open-ended question inviting the respondents to evaluate, in their own words, respectively, any positive and/or negative outcomes from contracting out green space maintenance. The differentiation of evaluations into positive and negative outcomes corresponds to the conceptualization of contracting performance proposed by Amirkhanyan et al. (2007). The question on outcomes was (translated from Danish and Swedish): ‘What do you think is positive and negative about using public procurement for maintenance services within outdoor area management?’ ‘Outdoor area’ is often used by practitioners in the municipal park sector to denote various types of spaces (parks, playgrounds, sports fields, wooded areas). The respondents could write their answers in two separate boxes: one for negative outcomes and one for positive outcomes. While open-ended questions are not usually regarded as the most optimal data source from surveys, they sometimes have advantages over closed questions. In particular, they can provide unexpected answers, reveal the ‘real’ views of respondents more closely and enable respondents to answer questions in their own words (Fowler, 2009).

An initial screening of the final data provided rich material ranging from entries with detailed accounts of various contracting outcomes to entries with lists of key outcomes. The length of accounts and topics varied. The total length of the data material available for analysis encompassed 3800 words. Usable entries from a total of 62 municipalities with contracting out experience, including 26 Swedish and 36 Danish municipalities, were available for further analysis. Based on self-reported data for contracting levels, the 62 municipalities comprise almost even distributions of municipalities for each country, with small-scale contracting (≤20%), medium-scale contracting (20–70%) and large-scale contracting out (≥70%). However, these 62 municipalities tend to represent larger municipalities within the two countries, particularly for Sweden (see the appendix for further details).

Data analysis

We combined qualitative and simple quantitative techniques for the data analysis using the Nvivo software package for qualitative analysis. A grounded theory-inspired inductive approach (Glaser and Strauss, 2006) combined with template analysis (Brooks et al., 2015; King, 2004) was adopted for data analysis, which generated a substantive account in the sense that it corresponds closely to the realities of managers working with contracting out (and be understandable by them).

An initial template for analysis was set up with the categories (codes) for outcomes derived from the reviewed contracting literature, including the domains in the final template related to ‘economic’, ‘service’, ‘managerial’ and ‘organizational’ outcomes. Outcomes related to broader political and democratic perspectives were not recorded in the data set (and therefore excluded from the final template). The structure of the collected data allowed us to differentiate the reported outcomes according to country context, scale of contracting, size of municipality and positive/negative outcomes. The four main codes in the final template could be used immediately for data organization and interpretation. When developing the template, a range of sub-codes for each main code emerged. Here, the survey data were recursively coded in conjunction with revisions of the template. Table 1 reports our final template, including background codes, the main code and sub-codes (including text examples). The main and sub-codes in the final template covered almost all of the text in the data. Some instances could also be interpreted as relevant for more than one outcome. For example, activities related to a perceived increase in transactions costs could be interpreted as both an economic outcome (e.g. increasing monitoring activities has economic costs) and a managerial outcome in terms of a change in the content of management work. These instances are discussed in the result section below.

Table 1.

Final template for text analysis.

Background codes	Sub-codes
Country	• Denmark • Sweden
Direction of evaluation	• Positive outcome • Negative outcome
Main codes	Sub-codes	Examples of coded text
Economic outcomes	• Price/cost levels • Transaction costs • Accountability • Resource use	• ‘Costs have decreased compared to in-house provision’ • ‘Control often requires more than in-house provision’ • ‘You can compare our own provision with private contractors’ • ‘Contractors can use machines and expertise more efficiently’
Service outcomes	• Quality of maintenance services • Citizen feedback • Development • Social services	• ‘Cost savings from procurement often imply permanent reductions in quality’ • ‘More citizen complaints’ • ‘Development of areas is lost’ • ‘Fewer opportunities to include youth work and vocational training’
Managerial outcomes	• Managerial effectiveness • Management information • Managerial focus	• ‘We get the maintenance we order’ • ‘Improved land use registration’ • ‘Inspections instead of visions’
Organizational outcomes	• Staff • Knowledge • Motivations • Quality of relationships	• ‘Frustrations and insecurity among transferred employees’ • ‘Knowledge about areas disappears due to increased staff turnover’ • ‘An in-house provider has more sense of ownership and does that little “extra”’ • ‘Very negative that contractors can't stay for very long’

Note: The template provides an overview over background codes, main codes, sub-codes and examples of coded text.

In order to gauge the importance of the various outcomes among the managers in our data set, we undertook a simple quantitative content analysis inspired by Bryman (2012) by summarizing the frequencies of the instances of the coded outcomes (see Table 2 in the result section). Some outcomes were clearly reported more frequently than others, and we interpreted the presence of a higher number of instances as reflecting the saliency of an outcome in the corresponding group of cases. Conversely, we regarded outcomes of less/no salience if they were briefly reported only once or a few times. Table 2 reveals how, at the main-code level, all main outcomes can be regarded as salient to some degree as defined above except for ‘positive service outcomes’.

Table 2.

Frequencies for coded contracting outcomes.

	Instances of outcomes
	Positive	Negative	Total
Economic outcomes	57 (in 43 cases)	35 (28)	92
Price/cost levels	16	8	24
Transaction costs	0	21	21
Accountability (for costs)	22	6	28
Resource use	19	1	20
Service outcomes	4 (in 4 cases)	20 (17)	24
Maintenance quality	3	13	16
Citizen feedback	0	4	4
Development of areas	1	2	3
Social services	0	1	1
Managerial outcomes	35 (in 25 cases)	45 (34)	80
Effectiveness	15	27	42
Focus	8	12	20
Information	12	6	18
Organizational outcomes	23 (in 19 cases)	39 (27)	62
Staff	0	5	5
Knowledge	18	14	32
Motivation	1	2	3
Quality of relations	4	18	22

Note: The table shows the frequencies a particular outcome were coded (instances) in the data set at the level of main codes and sub-codes. A case (i.e. a municipality) typically reported several outcomes across the various types of main outcomes.

Findings

Table 2 provides an overview of the frequencies for all outcomes (at the main-codes and sub-code levels) found in the data set. Most instances are recorded for economic outcomes (92 instances), while the fewest instances (24) are recorded for service outcomes. The reported outcomes are explored and discussed in further detail in separate sections below. The content and characteristics of the outcomes are illuminated by (translated) quotes from the managers. The country context (DK and S), scale of contracting (small, medium or large, as defined above) and municipality size (nearest 10,000) are indicated for each quote.

An assessment (not shown) of the frequencies of reported outcomes against country context (DK and S) and scale of contracting (small, medium and large) found no major skewnesses. For example, around 30% of the municipalities in each country reported on service outcomes, while some 80% reported on economic outcomes The only notable difference between context and reported outcomes was for scale of contracting and the frequency of reported economic outcomes, where the group with small-scale contracting had a higher frequency of reported economic outcomes (approximately 85%) compared to the group with large-scale contracting (approximately 70%).

Economic outcomes

Table 2 provides an overview of reported economic outcomes in the data set. ‘Price/cost levels’ relates to the impact of cost levels on contracted-out services (24 instances). ‘Transaction costs’ (21) relates to ex-ante and ex-post activities and organizational resources spent on managing contracting processes. These activities can also be seen as impacting management, particularly in terms of changes in management focus (see section on Managerial outcomes below). In our analysis, ‘accountability’ (28 instances) – in this case economic – relates to whether transparency and information levels on the cost and price of maintenance services have been affected by contracting out. In particular, managers must prove (to their principals, i.e. politicians) that they have arranged service delivery systems in economically efficient ways. Benchmarks from competitive market pricing (i.e. contestability) have become one standard for documenting this. ‘Resource use’ (21) relates to whether contracting out affects the use of machinery and expertise.

Overall, positive economic outcomes (57 instances) primarily relate to reduced price/cost levels (16), economic accountability primarily due to improved transparency on price/cost levels (22) and, finally, access to and more effective use of resources in the form of specialized and expensive machinery and expertise (19).

The many positive outcomes for price/cost levels were stated in terms such as, ‘maintenance costs have fallen compared to in-house operations’ (S, medium scale, 120,000), ‘a lot of money has been saved’ (DK, large scale, 50,000), ‘reduced costs – it's more cost-effective’ (S, medium scale, 140,000), ‘it costs less’ (S, large scale, 130,000), ‘overall reduction in costs’ (DK, medium scale, 30,000) or plainly ‘savings’ (DK, large scale, 50,000). Moreover, the positive outcomes regarding resource use can be seen as a special case of how service costs are reduced. This type of outcome was described in various ways, such as ‘We get to purchase expertise and machinery we don't have ourselves and from which an external provider can get a better economy’ (S, small scale, 80,000); ‘You can contract out those tasks where the machine park is not effective enough’ (DK, medium scale, 60,000) or ‘They [external providers] have greater expertise with special tasks, such as forestry, so they do the job quicker and cheaper than when organized in-house’ (DK, small scale, 40,000). In-house service provision requires specialized expertise and/or machinery and will be excessive costly if only needed occasionally. Improved economic accountability was another major outcome with several characteristics. Examples related to economic accountability ranged from improved financial management due to increased transparency on cost levels, such as, ‘It has become easier to keep the budgets as everything is calculated and we got fixed prices on all types of jobs’ (S, large scale, 70,000) or ‘Citizens and politicians cannot criticize the administration for having (too) high costs for the provision of maintenance, as they had been exposed to public tender and it has been established that they cost what they cost’ (DK, medium scale, 30,000) to examples related to comparisons between internal and external provision of services, such as, ‘To have [maintenance of] areas contracted out alongside other areas with in-house maintenance gives a good opportunity to compare cost and quality’ (DK, small scale, 100,000). Overall, the data support the interpretation that ‘competition’ or ‘benchmarking’ against private contractors was used to show political peers that the overall service delivery arrangement was economically sound.

Moreover, some negative outcomes were related to a lack of accountability about whether services were provided as agreed (six instances). These concerns were formulated in statements like ‘It isn't very transparent whether we get the service we order’ (DK, small scale, 70,000), ‘Some [external providers] strive to deliver less than agreed’ (DK, small scale, 40,000), or ‘We're suspicious about fraud’ (S, large scale, 150,000). The reports on failing accountability can also support a negative interpretation of the impact from contracting out on service outcomes (see below).

Further negative economic outcomes relate to increased cost levels for some types of services (eight instances) and increases in administrative activities and responsibilities due to the management of contractors, i.e. transaction costs (21). The prospects for increased costs were stated in terms such as, ‘Money becomes “king” and it's really costly even if we only want a “little extra” done’ (S, medium scale, 110,000) or ‘Not everything can be described in advance and, thus, ad hoc works become expensive’ (DK, large scale, 110,000). The increase in administrative activities owed to paperwork, the elaboration of service and work specifications, and monitoring functions. These ‘cost-bearing’ activities could range from routine management activities like ‘It's necessary to carry out more control – and money must be allocated for this’ (DK, small scale, 70,000), ‘Preparing a tender is a big job’ (S, small scale, 90,000), ‘[Contracting out] requires extensive tender materials, and it costs a lot of money and time to put services out on tender’ (DK, small scale, 50,000) to more ‘demanding’ or ‘failing’ cases, such as ‘A poorly performing contractor is incredibly time-demanding and expensive to deal with’ (S, small scale, 80,000) or ‘Terminating a contract also costs a lot’ (DK, medium scale, 50,000). Some also questioned whether lower service provision costs were offset by increased transaction costs: ‘[It's a] big job to contract out, but does the administration – including procurement, service specification and quality control – not overshadow eventual cost savings on the provided work?’ (DK, medium scale, 60,000).

According to our findings, a trade-off in economic accountability may exist between transparent price/cost levels and decreased service quality transparency. Similarly, a trade-off in cost/price levels for different types of maintenance services could be identified. This trade-off becomes clear in the comment, ‘A good work description ensures work well done for a fixed low price, but not all work can be described in advance and ad hoc works become expensive’ (DK, large scale, 110,000).

Service outcomes

An overview of reported service outcomes is provided in Table 2. ‘Quality of maintenance’ (16 instances) relates to the effect on quality in provided maintenance; ‘Citizen feedback’ (four) relates to the effects on the feedback from citizens to management; ‘Development of areas’ (three) relates to reported effects on area development; and ‘Social services’ (one) relates to public service provision going beyond the green space maintenance itself.

There were only four instances of positive service outcomes. The few positive service outcomes related to higher, more consistent service quality and situations where savings are re-invested in area development. Far more instances of service outcomes are negative (20 instances), including problems with low quality (13) and increased citizen complaints due to low quality (4). Quality issues were stated in common terms (e.g. ‘failing quality in provided work’, ‘poorer quality’). Quality levels for services provided by private contractors were also compared directly to the in-house service quality: ‘An in-house provider is more flexible and may feel very responsible for the areas, meaning you get “that little extra” external contractors wouldn't do because they have very different profit requirements’ (S, medium scale, 40,000). Other instances reported in the dataset further explain what ‘that little extra’ is about: ‘The contractors lack a “holistic view” and pass on work that obviously must be done – and don't report it’ (S, medium scale, 20,000), ‘Only the most necessary work is done’ (S, large scale, 140,000) or ‘Contractors lack sensitivity towards citizens’ (DK, medium scale, 60,000).

Furthermore, declining quality was sometimes associated with low contract pricing; e.g. ‘The price “determines”, meaning that the best for the job rarely wins the contract’ (S, small scale, 90,000) or ‘Procurements often coincide with savings on the areas, meaning a permanent quality reduction (DK, large scale, 200,000)’. This indicates a trade-off between contract pricing and quality. Overall, the relatively low frequency of reported service outcomes suggests that effects on service levels are not experienced as salient to the same extent as, for example, economic or management effects. Please note, however, that most instances are negative evaluations that are related to negative effects on service quality. An overall negative slant when interpreting service outcomes is sustained by reports of decreased transparency regarding the quality of services provided by private contractors.

Managerial outcomes

Table 2 provides an overview of reported managerial outcomes in the data set (85 instances). ‘Managerial effectiveness’ (42) relates to whether managers can effectively manage their maintenance responsibilities through contracts; ‘Managerial focus’ (20) relates to whether the attention in managerial tasks changes due to contracting out; and ‘Management information’ (18) relates to whether information required for managing maintenance effectively changes due to the use of contracting. Furthermore, as mentioned above, the outcomes related to economic accountability can be interpreted as improvements or impediments of a particular kind of information available to managers. The instances below are ‘merely’ related to changes in formal planning systems (e.g. improved area registration).

Most notable is the relatively high frequency of instances for outcomes related to ‘managerial effectiveness,’ particularly the negative outcomes (27 instances). First, on the positive side, improved effectiveness is reported in various ways. In particular, the traditional managerial task of carrying out activities as planned is noted as becoming easier: ‘You get what you order’ (DK, small scale, 90,000); ‘You can get rid of a provider who doesn't deliver and it's easier to impose fines’ (S, medium scale, 140,000); and management tasks related to setting priorities and defining what should be done, such as ‘When you manage by contracts you can decide 100% on maintenance levels in contrast to the management of in-house provisions, where decisions on resource use and quality levels are to some degree delegated to the personnel in the field’ (DK, small scale, 100,000). On the negative side, effectiveness was impeded by perceived rigidity in contracts, as expressed in statements such as, ‘We can't just adjust the level of work or service levels’ (DK, medium scale, 50,000); ‘Reaction time is slower’ (DK, medium scale, 60,000); or ‘stiff procedures for collaboration’ (DK, large scale, 70,000); as well as impeded by inadequate internal managerial resources for appropriate contract management, expressed in phrases/statements like ‘Insufficient resources for control and documentation’ (S, large scale, 150,000) or ‘The public sector often doesn't have sufficient resources to carry out control and documentation’ (DK, small scale, 40,000).

The outcomes of the overall ‘managerial focus’ were also ambiguous. On the negative side (12 instances), management was ‘burdened,’ especially with a focus on monitoring as a negative activity. For example: ‘A lot of follow-ups to ensure the required maintenance’ (DK, small scale, 80,000); ‘Continuous follow-up and quality control is needed’ (S, large scale, 70,000); ‘Control instead of visions’ (S, small scale, 80,000). Moreover, a loss of holistic perspective was associated with private-contractor maintenance operations: ‘They [private contractors] pass on work that obviously must be done and don't report it’ (S, medium scale, 20,000). On the positive side (eight instances), managerial focus was improved by a clearer focus on strategic and operational planning and less focus on issues related to the management of personnel and investments in machinery: ‘Problems with machinery or staff are not relevant for the purchaser [us]’ (DK, large scale, 110,000); ‘It's simpler to be a purchaser to a private contractor – you don't have field staff that takes up a lot of time’ (S, large scale, 70,000).

Information required for management was improved in various ways (12 instances). In particular, information levels were improved by introducing and updating standards for maintenance operations and quality levels, area registrations and the financial implications of various management decisions and the overview over operations. For example: ‘Procurement processes give oversight over areas, work and quality’ (DK, medium scale, 60,000); ‘As the client-side, we're continuously required to have our registrations, maps etc. updated’ (DK, small scale, 40,000); ‘You're required to reflect upon and specify which standards you want’ (S, large scale, 50,000). Most instances of positive outcomes on information levels seem associated with improved formal knowledge systems stored in systems for area registrations, geographical information systems (GIS) and service standards. On the negative side (six instances), a loss of important information for effective management was stated repeatedly. These outcomes relate to less contact with key stakeholders (operational staff, citizens): ‘As managers, we don't work as closely to the field personnel and have to go through a work leader as contact’ (S, large scale, 70,000); ‘Less contact between the municipality and citizens in the daily work’ (S, large scale, 150,000). While the positive outcomes on information levels seem associated with formal information, the negative outcomes on information levels seem to be associated with a loss in informal, ad hoc information given through day-to-day contact with operational personnel or citizens.

Organizational outcomes

Table 2 provides an overview of reported organizational outcomes in the data set. ‘Staff’ (five instances) relates to outcomes for staff within the public organization related to contracting out; ‘knowledge’ (32) relates to the outcome of the maintenance and development of the organizational knowledgebase due to contracting out; ‘motivation’ (three) relates to changes in incentives and ‘quality of relations’ (22) relates to the outcomes from the client–contractor relationships.

The majority of outcomes related to ‘knowledge’ were positive (18 instances). In several instances, contracting out was associated with learning opportunities in which the knowledge of or ideas for improved routines, methods, processes and/or how to undertake specialized tasks were transferred from private contractors to public clients or became available through contractual relations. This outcome was expressed in numerous statements and phrases: ‘New ideas and mind-sets’ (S, small scale, 90,000); ‘New methods and new perspectives on how to do the work’ (DK, small scale, 80,000); ‘Easier introduction to new routines and ideas’ (S, large scale, 150,000); ‘Adding knowledge to our organisation’ (DK, small scale, 40,000); ‘Contractors bring fresh eyes and experiences from other municipalities’ (DK, medium scale, 30,000). Fear of losing knowledge is another major negative outcome (14 outcomes), especially related to staff losing knowledge about parks and green spaces (as purchaser and provider alike): ‘Knowledge of maintenance operations disappears due to huge staff turnover’ (DK, medium scale, 30,000); ‘The contractor doesn't have the same detailed knowledge about the areas as in-house staff’ (S, medium scale, 100,000). Another major negative outcome concerns the ‘quality of relations’, indicating that, eventually, well-functioning, continuous relationships between management and those carrying out maintenance operations are somehow lost when contracting out and more difficult to develop within time-limited contractual relations. The negative outcome for the quality of relations was expressed in statements such as, ‘Disagreement about what is contained in the contract’ (DK, medium scale, 40,000); ‘It takes time to build up well-functioning collaborative relations with a contractor, which can be difficult with a short-term contract’ (S, medium scale, 100,000); ‘poor continuity’ (S, medium scale, 100,000); ‘typically opposing interests’ (DK, medium scale, 90,000). Note also that the data set contains very few instances related to outcomes for ‘staff’ and ‘motivations’. These instances related to negative outcomes, like general insecurity and deteriorating employment terms stated in phrases such as, ‘The in-house staff is always threatened that they can be contracted out. I wouldn't like it myself’ (DK, small scale, 90,000) or ‘There are problems with ensuring the terms of the contract, e.g., wage levels and employment conditions’ (DK, medium scale, 30,000). Positive outcomes for staff were not recorded at all in the data.

Contracting outcomes between two extremes

Based on the reported outcomes, our findings can be summarized in three different scenarios: A best-case scenario (one end of a continuum) summarizes all of the positive outcomes and captures the best of the promises of contracting out (as one extreme), whereas a worst-case scenario summarizes all of the negative outcomes and captures the pitfalls and drawbacks of contracting out (the other end of the continuum). Not representing any specific, real case, these two scenarios instead summarize the lists of positive and negative outcomes recorded in our dataset. Between these two extremes, a complex-case scenario represents a set of numerous trade-offs and mixes of contracting outcomes that are likely to accrue when contracting out is implemented across a group of local governments. Table 3 provides an overview of the three scenarios.

Table 3.

Summary of findings: Contracting outcomes between two extremes.

	Best-case scenario	Complex-case scenario	Worst-case scenario
Economic outcomes	Cost savings, improved economic accountability, more efficient use of specialized expertise and machinery.	Costs for standard operations reduced while costs for non-standard operations increased. Costs reduced but at expense of service quality levels. Transaction costs offset gains from cost savings on service provision.	Increased overall costs. Expensive adjustments. Increased monitoring costs.
Service outcomes	Consistent service quality, improvements through reinvestment.	Accountability on service quality is reduced while accountability on price/cost levels is improved.	Declining or failing service quality. Lack of reinvestment.
Managerial outcomes	More effective management. Improved strategic and operational management.	Improved effectiveness of contract management is limited by lack of internal resources.	Inflexible service provision. Lack of internal resources for contract management.
Organizational outcomes	Access to specialized expertise. Innovation and improved organizational routines and work methods.	Loss of operational knowledge organized in-house, but introduction of new knowledge through collaboration with private contractors. Collaboration framed and challenged by time-limited contractual relations.	Loss of key knowledge for planning and management about maintenance operations. Ongoing, burdensome disputes with contractors, poor contractual relations.

The best-case scenario is characterized by cost savings and improved technical efficiency and economic accountability, access to externally provided specialized expertise, improved strategic and operational management as well as a more effective management, innovation and improved organizational routines and work methods. The worst-case scenario is characterized by declining/failing quality in service provisions, increased overall costs, inflexible service provision, overly expensive costs for adjustments, increased and costly efforts for monitoring, loss of key knowledge about services for planning and management, and ongoing and burdensome disputes with contractors, uncertainty for in-house staff as well as poor collaborative relations. Between the two extremes is a scenario describing contracting outcomes as a mix of negative and positive outcomes as well as a set of trade-offs.

The illumination of the complexities related to outcomes at the aggregate sector level is a key finding in our analysis. Accountability for price/cost levels might be enhanced while accountability on service quality is simultaneously reduced; costs of standard operations are reduced while non-standard operations become more expensive; contract management becomes more effective but limited by a lack of internal resources or knowledge within an organization on maintenance operations is lost while collaborating with private contractors produces new knowledge.

Discussions

Our study generally highlights the importance of integrating more multidimensional understandings of the outcomes from contracting out than implied by the (albeit important) focus of the dominant economic focus on costs and quality. Our study thus restates the importance of earlier research emphasizing the multidimensional characteristics of contracting outcomes (e.g. Fernandez, 2009; Lindholst, 2017). Our study finds important positive outcomes worth highlighting, including particular economic outcomes in terms of improved economic accountability, cost levels and resource use, some managerial and organizational outcomes in terms of improved information levels, learning and knowledge, as well as managerial effectiveness. Given the balance in the recorded economic outcomes, our study particularly tends to support reported findings on cost reduction for technical services (e.g. Hodge, 2000). Several negative aspects are also highlighted. In particular, quality issues, transaction costs, burdensome contractual relations, impeded managerial effectiveness and lack of flexibility are among the key drawbacks. Given the balance in the reported service outcomes, our study tends to support findings in earlier studies highlighting a range of accountability (quality) issues when private contractors provide services (e.g. Lamothe and Lamothe, 2010; Romzek and Johnson, 2005). It should be noted that the frequencies of instances of various outcomes recorded in the data set are only indicative of the overall balance, as the data set does not evaluate the relative importance (weight) of the various outcomes (e.g. prospective cost savings may still render contracting out more favourable for local authorities than alternative modes of service provision despite managerial and organizational drawbacks).

Importantly, our study highlights outcomes related to knowledge, such as learning opportunities (e.g. ‘fresh eyes’, ‘new thinking’), as an important yet somehow overlooked outcome in the contracting literature. Learning is a key strategic driver in inter-organizational co-operations, which contrasts with strategic drivers rooted in cost concerns (Child et al., 2005). Conversely, outcomes related to democratic and legal performance apparently matter little to our sample of managers. This is in contrast to the early critique of contracting out for ‘hollowing out’ the functions of government (Rhodes, 1994) and the lack of deliberation in New Public Management-oriented approaches to contracting out (Vincent-Jones, 2007). Our study thus sustains the more general point that the choice of stakeholder perspective is important for identifying and evaluating contracting outcomes. Further along these lines, public managers are found to be fully occupied with a range of outcomes that can be expected to be of less interest to other stakeholders. The highlighted intricacies of dealing with daily management would most certainly be of little or no interest to service users and citizens. Service users would certainly be affected negatively in the long run if management becomes immersed in struggles with contractors over operational matters rather than being occupied with ensuring that services are fit for purpose. However, the analysis finds that, more positively, contracting out can contribute to improved management in terms of, for example, the more effective prioritization of resources, stronger managerial controls and developing routines and methods. Similarly, it is worth noting that contracting out is highlighted as a way of serving political needs to demonstrate the sound use of economic resources through ‘market-prices’ and ‘benchmarks’, whereby service budgets are protected from politically motivated searches for cost savings. Our study also provides evidence of the potentially conflicting evaluation of key outcomes. All told, while contracting out has been associated with negative outcomes for employees (Vrangbæk et al., 2015), several public managers indicated that being released from internal staff management responsibilities was a positive outcome and improved the managerial focus (e.g. on more user-oriented core tasks).

In a comparative perspective, services related to green space maintenance can be argued as relatively straightforward and ‘simple’ to contract out based on evaluations of their transactional characteristics (Brown and Potoski, 2005; Hefetz and Warner, 2012). Nevertheless, our study finds that contracting outcomes can be a ‘mixed bag’ for public managers with implementation responsibilities, even for ‘simple’ services. Implementing the contracting out of simple services may still be a daunting task for public managers in local governments, and the characteristics of parks and green space maintenance might not neatly fit the requirements for the effective use of contracting out, as some studies also have noted (Jang, 2006; Lindholst and Bogetoft, 2011). It can also be hypothesized that for services with more complex characteristics (e.g. those increasingly difficult to specify or demanding high investments), it would be relatively harder to achieve positive outcomes, and negative outcomes would be even more pronounced. In sum, for more complex, ‘high transaction cost’ services, it would be plausible to expect a relatively more challenging task to strike a favourable balance in the negative/positive outcomes mix.

Compared to extant research that reports on contracting outcomes through sharp, theoretical or policy-oriented lenses, our study provides a more open and vivid portrait from the public manager perspective. More particularly, our study provides important nuances for understanding contracting outcomes in terms of variations, complexities and trade-offs. Furthermore, the complex-case scenario does not fit well with typical ‘pro’ and ‘con’ or ‘proponents’ versus the ‘critics’ positions used to motivate much of the research on contracting outcomes. One of our key findings is that upsides come with downsides of different kinds across a service sector. In this respect, the findings are more closely aligned with earlier qualitative, case-based studies emphasizing multiple performance dimensions (e.g. Romzek and Johnson, 2005) than quantitative assessments of a single dimension (e.g. Alonso et al., 2017). Our findings will hopefully inspire future research to include a broader focus on the many outcomes of contacting out. For example, as economic returns from continued exposure to competition diminish (e.g. Petersen et al., 2017) – when in-house or private delivery of services is provided at cost-efficient levels – evaluations of contracting outcomes in terms of cost savings might simply be the wrong focus vis-à-vis the underlying realities. The economic accountability of the overall service delivery system, service quality and/or organizational development might be more relevant outcomes to evaluate in such scenarios. Some earlier research supports these arguments, which indicates the necessity to see contracting as a strategic practice designed and undertaken by local authorities to serve different purposes over time (Brown et al., 2006; Lindholst and Bogetoft, 2011; Warner and Hefetz, 2012) as well as research (e.g. Bel et al., 2007, 2010), which indicates the necessity to take contextual contingencies further into account when evaluating contracting outcomes.

Conclusion

Our study provides an explorative, ‘grounded’ portrait of how public managers assess outcomes from the contracting out of green space maintenance within the municipal park sector in Denmark and Sweden. The portrait adds important nuances to the current understanding of outcomes and supports the use of multidimensional conceptualizations of contracting outcomes while also indicating that insights can be gained from accommodating evaluations from the stakeholder viewpoint. Our findings can be summarized in three scenarios that stylize certain characteristics of contracting outcomes within a most-likely context for effective use of contracting out. A best-case scenario captures the promises of contracting out by summarizing all of the positive outcomes. Key outcomes in the best-case scenario are cost reductions and the improved use of resources, improved economic accountability, better information for decision making and more effective maintenance operations management, together with organizational innovation and development. Conversely, a worst-case scenario captures the pitfalls and drawbacks of contracting by summarizing all of the negative outcomes. Key outcomes in the worst-case scenario are failing quality in service provision, increased overall costs, inflexible service provision, expensive costs for adjustments, increased monitoring costs, loss of knowledge about services for planning and management, ongoing and burdensome disputes with contractors and poor collaborative relations. The best-case and worst-case scenarios are mere abstractions based on summaries of, respectively, the positive and negative outcomes. They form the endpoints in a continuum between attractive and unattractive outcomes. Between the two extremes, we highlight a complex-case scenario that captures contracting outcomes as a complex set of trade-offs and interrelated negative and positive outcomes. We believe that highlighting the complex and composite nature of contracting outcomes best captures the realities of contracting out in the public sector. The implications for further research on contracting outcomes are obvious.

Footnotes

Acknowledgements

We are thankful for the support from Hedeselskabet and the Swedish University of Agricultural Sciences, The Movium Platform, who generously co-financed the research. We are also grateful for the time devoted by the public managers in Scandinavia who provided data for the research.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Andrej Christian Lindholst has over the years done research within the fields of urban planning and public administration. His research in planning has primarily focused on management and evaluation of open space values in urban environments. His research in public administration has primarily focused on marketization in local governments. His recent research on marketization has involved a comparative perspective on service sectors and countries. He works as associate professor at the Department of Political Science, Aalborg University, Denmark and is associated with Center for Organization, Management and Administration.

Morten Balle Hansen has over the years carried out research in public organization and management, and combines research in managerial behaviour in public organizations with research into policy processes. His research has had particular emphasis on evaluation, performance management and innovation. His research has been published in well-established journals within the field of Public Administration. He works as professor at the Department of Political Science, Aalborg University, Denmark and is associated with Center for Organization, Management and Administration.

Thomas Barfoed Randrup holds extensive research expertise in planning and management of urban open spaces, including strategy and policy, function and uses of green space, as well as organization of maintenance related to urban open spaces. His research has over the years been published within the fields of urban planning and urban forestry. He is professor and research coordinator at the Department of Landscape Architecture, Planning and Management, as well as leading the Department’s Landscape Governance and Management theme group. He also holds a professorship in Urban and Peri-urban Green Space Management at the University of Life Sciences, in Norway.

Bengt Persson works in the border zone between research and practice within open space and landscape area management. He has been active within landscape management since the 1980s and co-author of a series of maintenance manuals which have become the Swedish standard for maintenance terms and definitions of the maintenance products. Another area of expertise is professional and organizational learning. He is a senior lecturer at the Department of Landscape Architecture, Planning and Management, and part of the Department's Landscape Governance and Management theme group.

Anders Kristoffersson holds expertise in greenspace construction and management. He works with research and teaching regarding procurement of green space maintenance in different research projects and courses. He is senior lecturer at the Department of Landscape Architecture, Planning and Management, and part of the Department's Landscape Governance and Management theme group.

Appendix

References

Alonso

Clifton

Díaz-Fuentes

(2017) The impact of government outsourcing on public spending: Evidence from European Union countries. Journal of Policy Modeling 39(2): 333–348.

Amirkhanyan

Kim

Lambright

(2007) Putting the pieces together: A comprehensive framework for understanding the decision to contract out and contractor performance. International Journal of Public Administration 30(6): 699–725.

Amirkhanyan

Kim

Lambright

(2010) Do relationships matter? Assessing the association between relationship design and contractor performance. Public Performance and Management Review 34(2): 189–220.

Andersen

Boesen

Pedersen

(2016) Performance in public organizations: Clarifying the conceptual space. Public Administration Review 76(6): 852–862.

Andrews

Van De Welle

(2013) New public management and citizens' perceptions of local service efficiency, responsiveness, equity and effectiveness. Public Management Review 15(5): 762–783.

Bel

Fageda

Warner

(2010) Is private production of public services cheaper than public production? A meta-regression analysis of solid waste and water services. Journal of Policy Analysis and Management 29(3): 553–577.

Bel

Hebdon

Warner

(2007) Local government reform: Privatisation and its alternatives. Local Government Studies 33(4): 507–515.

Boyne

(1998) Public Choice and Local Government: A Comparative Analysis of the UK and the USA, Basingstoke: Macmillan.

Boyne

(2002) Concepts and indicators of local authority performance: An evaluation of the statutory frameworks in England and Wales. Public Money and Management 22(2): 17–24.

10.

Boyne

Meier

O'Toole

Jr et al. (2006) Public management and organizational performance: An agenda for research. In: Boyne

Meier

O'Toole

Jr et al. (eds) Public Service Performance: Perspectives on Measurement and Management, New York: Cambridge University Press, pp. 295–311.

11.

Bretzner

Randrup

Persson

(2016) Do public procurement cost efficiency depend on market types? A critical test in park and road sectors in Sweden. International Journal of Public Sector Management 29(5): 488–501.

12.

Brewer

(2006) All measures of performance are subjective: More evidence on US federal agencies in public service performance. In: Boyne

Meier

O'Toole

Jr. et al. (eds) Public Service Performance: Perspectives on Measurement and Management, New York: Cambridge University Press, pp. 35–54.

13.

Brooks

McCluskey

Turley

et al. (2015) The utility of template analysis in qualitative psychology research. Qualitative Research in Psychology 12(2): 202–222.

14.

Brown

Potoski

(2003) Contract-management capacity in municipal and county governments. Public Administration Review 63(2): 153–164.

15.

Brown

Potoski

(2005) Transaction costs and contracting: The practitioner perspective. Public Performance and Management Review 38(3): 326–351.

16.

Brown

Potoski

Van Slyke

(2006) Managing public service contracts: Aligning values, institutions, and markets. Public Administration Review 66(3): 323–331.

17.

Bryman

(2012) Social Research Methods, New York: Oxford University Press.

18.

Bryntse

Greve

(2002) Competitive contracting for public services: A comparison of policies and implementation in Denmark and Sweden. International Public Management Review 3(1): 1–21.

19.

Child

Faulkner

Tallman

(2005) Cooperative Strategy: Managing Alliances, Networks and Joint Ventures, Oxford: Oxford University Press.

20.

Cuganesan

Guthrie

Vranic

(2014) The riskiness of public sector performance measurement: A review and research agenda. Financial Accountability and Management 30(3): 279–302.

21.

Davis

Girth

Stazyk

(2016) Social and technical determinants of perceived contract performance: Rules, autonomy, and ethics. Public Performance & Management Review 39(3): 728–755.

22.

Dempsey

Burton

(2012) Defining place-keeping: The long-term management of public spaces. Urban Forestry & Urban Greening 11(1): 11–20.

23.

Domberger

Jensen

(1997) Contracting out by the public sector: Theory, evidence, prospects. Oxford Review of Economic Policy 13(4): 67–78.

24.

Donahue

(1989) The Privatization Decision: Public Ends, Private Means, New York: Basic Books.

25.

Fernandez

(2009) Understanding contracting performance: An empirical analysis. Administration and Society 41(1): 67–100.

26.

Fowler

FJJ

(2009) Survey Research Methods, 4th. London: Sage.

27.

Glaser

Strauss

(2006) The Discovery of Grounded Theory: Strategies for Qualitative Research, London: Transaction.

28.

Hansen

(2010a) Privatisering, demokrati og magt i de danske kommuner. Politica 42(2): 183–201.

29.

Hansen

(2010b) Marketization and economic performance. Public Management Review 12(2): 255–274.

30.

Hansen

(2017) Performance management and evaluation. In: Greve

(ed.) Handbook of Social Policy Evaluation, Cheltenham: Edgar Elwin, pp. 223–241.

31.

Hansen

Lindholst

(2016) Marketization revisited. International Journal of Public Sector Management 29(5): 398–408.

32.

Hart

Shleifer

Vishny

(1997) The proper scope of government: Theory and an application to prisons. Quarterly Journal of Economics 112(4): 1127–1161.

33.

Hefetz

Warner

(2012) Contracting or public delivery? The importance of service, market, and management characteristics. Journal of Public Administration Research and Theory 22(2): 289–317.

34.

Hodge

(2000) Privatization: An International Review of Performance, Oxford: Boulder Westview.

35.

Hodge

Coghill

(2007) Accountability in the privatized state. Governance 20(4): 675–702.

36.

Hood

(1991) A public management for all seasons?. Public Administration 69(1): 3–19.

37.

Hood

(1998) The Art of the State: Culture, Rhetoric, and Public Management, Oxford: Oxford University Press.

38.

Jang

(2006) Contracting out parks and recreation services: Correcting for selection bias using a Heckman selection model. International Journal of Public Administration 29(10/11): 799–818.

39.

Jones

(2000) Managing the green spaces: Problems of maintaining quality in a local government service department. Managing Service Quality 10(1): 19–31.

40.

Kettl

(1993) Sharing Power: Public Governance and Private Markets, Washington: The Brookings Institution.

41.

Kettl

(2010) Governance, contract management and public management. In: Osborne

(ed.) New Public Governance?, Oxon: Routledge, pp. 239–254.

42.

King

(2004) Using templates in the thematic analysis of text. In: Cassell

Symon

(eds) Essential Guide to Qualitative Methods in Organizational Research, London: Sage, pp. 256–287.

43.

Lamothe

(2010) Competing for what?. The American Review of Public Administration 40(3): 326–350.

44.

Lindholst

(2017) A review of the outcomes from contracting out urban green space maintenance: What we know, don't know and should know. Urban Forestry & Urban Greening 27(October): 50–58.

45.

Lindholst

Bogetoft

(2011) Managerial challenges in public service contracting: Lessons in green-space management. Public Administration 89(3): 1036–1062.

46.

Lindholst

Hansen

Petersen

(2016) Marketization trajectories in the Danish road and park sectors: A story of incremental institutional change. International Journal of Public Sector Management 29(5): 457–473.

47.

Lundqvist

(1989) The literature on privatization. Scandinavian Political Studies 12(3): 271–277.

48.

Millennium Ecosystem Assessment (2005) Ecosystems and Human Well-being: Synthesis, Washington DC: Island Press.

49.

Mulgan

(2006) Government accountability for outsourced services. Australian Journal of Public Administration 65(2): 48–58.

50.

Nuppenau

(2009) Marketizing Municipal Park Management Organisations in Denmark: A Study of Policymaking and Organisational Change in the Period 1985–2005, Frederiksberg: University of Copenhagen.

51.

O'Flynn

Alford

(2008) The separation/specification dilemma in contracting: The local government experience in Victoria. Public Administration 86(1): 205–224.

52.

Patterson

Pinch

(1995) Hollowing out the local state: Compulsory competitive tendering and the restructuring of British public-sector services. Environment and Planning 27(9): 1437–1461.

53.

Petersen

Hjelmar

Houlberg

(2017) Is contracting out of public services still the great panacea? A systematic review of studies on economic and quality effects from 2000 to 2014. Social Policy and Administration. DOI: 10.1111/spol.12297.

54.

Pollitt

Bouckaert

(2011) Public Management Reform: Comparative Analysis of New Public Management, Governance and the Neo-Weberian State, Oxford: Oxford University Press.

55.

Radin

(2006) Challenging the Performance Movement: Accountability, Complexity, and Democratic Values, Washington: Georgetown University Press.

56.

Randrup

Persson

(2009) Public green spaces in the Nordic countries: Development of a new strategic management regime. Urban Forestry and Urban Greening 8(1): 31–40.

57.

Rhodes

RAW

(1994) The hollowing out of the state: The changing nature of the public-service in Britain. Political Quarterly 65(2): 138–151.

58.

Romzek

Johnston

(2002) Effective contract implementation and management: A preliminary model. Journal of Public Administration Research and Theory 12(3): 423–453.

59.

Romzek

Johnston

(2005) State social services contracting: Exploring the determinants of effective contract accountability. Public Administration Review 65(4): 436–449.

60.

Schachter

(2010) Objective and subjective performance measures: A note on terminology. Administration and Society 42(5): 550–567.

61.

Vincent-Jones

(2007) The new public contracting: Public versus private ordering?. Indiana Journal of Global Legal Studies 14(2): 259–278.

62.

Vrangbæk

Petersen

Hjelmar

(2015) Is contracting out good or bad for employees? A systematic review of international experience. Review of Public Personnel Administration 35(1): 3–23.

63.

Walker

Andrews

(2015) Local government management and performance: A review of evidence. Journal of Public Administration Research and Theory 25(1): 101–133.

64.

Warner

Hefetz

(2012) Insourcing and outsourcing. Journal of the American Planning Association 78(3): 313–327.

65.

Yang

Hsieh

(2009) Contracting capacity and perceived contracting performance: Nonlinear effects and the role of time. Public Administration Review 69(4): 681–696.