Abstract
This paper reviews the New Public Management literature on the effects of the creation and ongoing operations of agencies across European public sectors. It finds that the bulk of evidence concerns internal effects on processes/activities of agency creation and management and little evidence on outputs and outcomes. The article identifies a number of patterns across the sample and finds positive effects on improved processes and an orientation toward results and service users’ needs. Similarly, it finds improvements in transparency and accountability across various countries, but evidence in these areas is less clear. It finds that 46% of the studies included in the sample identified concerns about fragmentation, coordination or organizational stability and shows that unintended consequences are an important part of the evaluation of the effects of agencification. The paper argues that the picture of the effects of agencies is nuanced and discusses possible factors that can tip the balance for or against the success of the agency model.
Introduction
The creation of agencies has been at the core of the New Public Management (NPM). Much has been written on and commented about the suitability, advantages, and problems of hiving off public tasks. It might therefore be expected that the effects of the creation and ongoing operations of agencies are well understood. However, whether agencification “works,” and the circumstances under which it does, are still unfinished tasks (Dan et al., 2012; Manning and Matsuda, 2000; Verhoest et al., 2012). This article reviews the NPM literature on agencies across European public sectors to identify possible patterns in the success or failure of the agency model. It does so by analysing “internal” effects on the processes or activities of agencies as well as effects on outputs and outcomes.
To ask what the effects of agencification have been may seem like a straightforward question, but actually it is anything but straightforward. There are two principal groups of reasons why it is hard to answer: the first have to do with meaning and the second with evidence. The terms “agency” and “effects” have both been used with a range of meanings, so one needs to be careful to specify what it is one is talking about at any particular point.
On the question of meanings, it has long been recognized that there is no standardized international view of what is an “agency” (Pollitt and Talbot, 2004; Pollitt et al., 2004; Verhoest et al., 2012: 18–21). There is no widely shared legal or constitutional category that ensures that (say) the British, the Dutch, the French, and the Germans are all talking about the same thing. Neither is the cultural and political meaning of agencification similar in all countries or periods. For example, a detailed study of key official documents on agencification in Australia, the Netherlands, and Sweden came to the conclusion that debates about why agencification was necessary and what it meant were quite different in each case (Smullen, 2010). Or again, during the 1990s in the UK, agencification was widely seen as giving blocks of operational activity more autonomy from ministries, while in the Netherlands a view was that agencification was attractive because it placed such activities under closer ministerial supervision than would putting them into the then popular ZBO format (Van Thiel and Pollitt, 2007). A study of agencies operating in the same sectors in Finland, the Netherlands, Sweden, and the UK showed that even where agencies had similar performance indicator systems, the status of these systems and the ways in which they were used varied considerably (Pollitt et al., 2004). This means that international lesson drawing is difficult, because similar-looking organizational structures or performance management systems may actually be operated in different ways.
Theoretical background
Scholars have repeatedly stressed that there was not one single coherent theory that fed into the NPM (Kettl, 1997). Nevertheless, economic theories, typically neo-classical and rational choice, are often considered to form the core of the theoretical basis of NPM. This includes public choice theory (Downs, 1967; Niskanen, 1968, 1971; see also Boyne et al., 2003 for an application to public services in the UK), principal-agent theory (Jensen and Meckling, 1976; see also Buchanan, 1988; Verhoest, 2005), managerialism (Pollitt, 1993) or property rights theory (for example, Preker and Harding, 2003). This has been contrasted with general management theory which tends to take a more heroic view of managers as leaders who can “transform” their organizations and change the organizational culture. So for this reason more autonomy—freedom for managers to manage—is seen as a good idea. Rational choice has very little to do with (say) cultural change, leadership, and all the things which concern generic management theory. It takes a more narrowly self-interested view of motivation. The key expectation of these theories is that autonomous organizations are uniquely positioned to deliver performance—if only they were equipped with the proper incentives. The proponents of agencification were sometimes quite vague about exactly what agencification was supposed to achieve (for example, Van Thiel, 2001). On other occasions, however, a variety of positive claims were made. It was assumed that executive agencies need to enjoy a certain degree of discretion so that decisions can be made on the basis of specialized expertise and customer preferences. Therefore, allocative efficiency was presumed to improve following specialization and disaggregation of monolithic entities. Technical efficiency of operations was likewise believed to increase. A flexible service delivery organization is expected to be in a better position to innovate constantly and find cost-effective and creative solutions to improve quality, address financial problems, and adapt to changing patterns of demand. As a result, public accountability and user/customer satisfaction are expected to get better. If management is empowered with the adequate decision-making capacity, it can motivate personnel by designing tailored incentives, which in turn are expected to lead to performance improvements. An expanded choice of services and maintained equity are two additional expected benefits. The overall expectation was that performance would improve without a significant decrease in equity or access to services. Other issues such as fragmentation, coordination, and their implications for systemic performance and policy coherence were not part of the original model. Weight was placed mainly on the performance of operations at unit level measured by the production of outputs (Schick, 1996).
In this article, I adopt what has become a fairly standard definition of an agency (Van Thiel, 2012: 18–20). This definition treats an agency as a public sector organization which:
is structurally disaggregated from government ministries; and operates under more business-like conditions than the core government bureaucracy.
Such organizations operate at arms’ length from the core of government and undertake public tasks such as service delivery, regulation, and policy implementation. This article distinguishes between three types of effects:
Activities/processes (for example, budgeting, making accountability statements; organizing to produce services; and training staff); Outputs (for example, treatments/lessons/inspection/information delivered to service users; grants; and loans issued); and Outcomes (for example, improved health status for citizens; better economic growth; educational attainment for students; and increased citizen satisfaction with the quality of services).
An organization or program is conceived as a set of activities or processes. These include organizational arrangements like the division of responsibility, the allocation of authority, the standard operating procedures, and so on. These procedures enable the organization to deploy and redeploy its resources (staff, money, buildings, and so forth) which are collectively termed inputs. From these activities and processes, the organization or program then produces a set of outputs, which could, for example, be lessons (in a school), licenses (from a licensing agency), medical treatments (from a hospital), and so on. These outputs are, in a sense, what the agency “gives” to the outside world—to citizens, to civil society associations, and to business firms. They are like messages, passing across the membrane that separates (on the one hand) the state from (on the other) the market sector and civil society. Outputs are invariably intended to produce desirable outcomes, beyond the organization or program—so school lessons are supposed to produce educated students and hospital-provided medical treatments are supposed to produce the cure or the alleviation of ill-health. An outcome is something that happens in the world outside the organization and the program: it is an effect “out there in the real world.”
The performance of organizations and programs (the value of their activities) is usually thought of in terms of certain relationships between these inputs, outputs, and outcomes. Thus, the ratio between inputs and outputs is a measure of efficiency (or technical efficiency or X-efficiency, or productivity). Effectiveness, however, is a different concept, which is usually conceived as the degree to which the outcomes match the original goals or objectives set for the organization or program. As many writers have remarked, if goals are multiple, conflicting, or ambiguous then it will be difficult to determine effectiveness, which will, in effect, become a “contested concept.” Unfortunately, policy goals frequently are multiple, conflicting, or ambiguous, not least because that is what politicians may need to get sufficient agreement to launch the policy in the first place. This certainly includes public management reforms which are often claimed to be all things to all people—to save money, raise service quality, increase effectiveness, and so forth.
Another point to be borne in mind is that, increasingly, official reports and evaluations, as well as academic studies, make use of complex indices of performance, which combine two or more elements (see for example, Audit Commission, 2009; Pollitt, 2011; Van de Walle et al., 2008). These aggregated indices can be very useful to busy decision makers or to non-specialists and citizens, but they may also (deliberately or inadvertently) conceal underlying trades-off between two or more desirable values (for instance equity versus efficiency). They can also give spurious precision to judgments which are more correctly seen as hedged about by quite wide brackets of uncertainty (Jacobs and Goddard, 2007).
When assessing effects, there is also frequently a problem of attribution. That is to say, there may be evidence of, say, an increase in efficiency, but what exactly was the cause? If the creation of an agency structure was the only thing that happened during the relevant time period, then the efficiency gain might be plausibly attributed to that new structure. But if there were other developments at the same time (for example, technological changes; a new management; a government-wide efficiency drive) then it can be very hard to assess what share of the efficiency gain (if any) should be attributed to agencification. This problem of multiple possible causes is actually very common in public management reform. And it tends to get worse the further one moves from changes in processes to changes in outputs to changes in outcomes.
Methods and data
The studies of the effects of the creation and operation of agencies reviewed in this article are derived from a database of 519 studies of NPM reforms across Europe. The database was created as part of an international comparative project involving 10 research teams in nine European countries (Pollitt and Dan, 2011; Pollitt and Dan, 2013).
1
The process of identifying and selecting the studies followed the following steps (see also Pollitt and Dan, 2011):
Reviewed titles and keywords in articles published since 1980 (or later in case the first number of a journal appeared after 1980) in the following public administration and management journals: Journal of Public Administration Research and Theory, Public Administration, Governance, Public Administration Review, International Review of Administrative Sciences, International Public Management Journal, Public Management Review, Public Policy and Administration, and Evaluation. The following keywords were used to guide the search: New Public Management (NPM), managerialism, performance, public sector reform, and public management reform; Read the abstract of those articles that satisfied step (a); Submitted the articles that met step (b) to a detailed review and made a decision for inclusion in the database on the basis of meeting cumulatively the following criteria: The study includes an evaluation of NPM reforms that have already been launched; The NPM reforms have been partially or completely implemented and a formal ongoing or final evaluation has been conducted; and The study makes claims about the effects of NPM, and it provides evidence to support those claims.
Countries covered
Percentage of the total 72 studies of agencies.
Type of studies
Percentage of the total 72 studies of agencies.
Studies of agencies by type of effects
Examples of effects of agencification in Europe
PIs: performance indicators.
Findings
A first main finding is that the literature on the effects of agencification in Europe has been much more concerned with evaluating changes in processes/activities than with changes in outputs and outcomes. Table 3 shows that 36% of the studies (26 studies) found evidence on outputs or outcomes and of these only 7% on outcomes. As a result, much more is known about the internal machinery of establishing and managing agencies than about the effects that agencies and their services have on service users and citizens. Compared to what the citizens “get” as a result of agencification and how they are affected by it—outcomes—we know relatively more about what services agencies provide and how they provide them—outputs. This is reflected in Table 4 which shows specific examples of the effects of agencies. This is a first clear pattern to which the data point.
Mixed evidence on outputs and outcomes of agencification
Although most of the convincing evidence only refers to changes in processes, a number of studies in our sample discuss outputs and outcomes, most notably in Austria, Belgium, France, and the UK. Column “What were the effects?” in Table 4 summarize these effects. A first notable example is the Austrian experience with agencification. External independent and academic evaluations—both of a decade ago and more recent—have found evidence on improved efficiency (Hammerschmid et al., 2008; Rossmann, 2001; Rossmann and Leitsmüller, 2010). These improvements are attributed to two main factors: (i) improvements in management processes following an increase in autonomy and (ii) a decrease in inputs consisting mostly of staff reductions. These findings are extremely interesting, but there is limited evidence of technical or X-efficiency reflecting improvements in input–output ratios. With no or limited evidence on corresponding output levels, these efficiency improvements can best be described as savings or economies. Technical efficiency may have improved, but there is insufficient evidence to show that output levels improved, remained unchanged, or decreased less than the level of inputs. However, overall, taking into account the evidence on processes, Austria is among the most successful cases of the agency model—despite a number of unintended consequences as discussed in detail below. Another salient example refers to the creation of semi-autonomous regional hospital agencies in France. This was found to lead to improvements in transparency and accountability to various stakeholders including the public. The main factor explaining these positive effects is improved communication between agencies and stakeholders (Fargeon et al., 2002). However, no clear positive effects were found on cost containment, access to services, or quality of healthcare (Minvielle, 2006). A case study evaluation concluded that the French regional healthcare agency model granted limited agency autonomy in the attempt to maintain central policy and service coordination and control. This decreased the negotiation power of regional agencies and managerial flexibility but improved overall cohesion (Dumond and Jourdain, 2006). The UK studies of agencies evaluating the Next Steps Programme provide some evidence on changes in outputs and outcomes although the bulk of existing evidence refers to the processes of setting up and managing agencies. This includes improved accountability and transparency as a result of framework agreements and better reporting as well as changes in customer and result orientation and overall positive perceptions of performance (James, 2003; Pollitt, 2006; Pollitt et al., 2004). These findings point to certain improvements in outputs and outcomes in a number of countries, but the picture is far more nuanced with evidence of trade-offs and consequences in other areas. It is true, however, that these other considerations were not part of the original model. This is a pattern that is not exclusive to outputs and outcomes alone but is also common to changes in processes or activities.
Positive effects of agencification on processes/activities
There is considerable evidence on various effects of the creation of agencies on processes/activities, as shown in Table 4. Compared to what we know about the outputs and outcomes of agencification, the evidence on processes is more convincing and more readily attributable to agencification. The data point to certain improvements in various countries in the management and organizational culture of government bodies following an increase in agency autonomy compared to agencies with tighter subordinate relations. These improvements have taken various forms, including an orientation toward the needs and demands of service users as evidenced in the case of Austrian, Dutch, and UK agencies (Hammerschmid et al., 2008; James, 2003; Kraak and van Oosteroom, 2002; Pollitt, 2006; Pollitt et al., 2004; Rossmann, 2001). The management by results model promoted in Swedish agencies improved orientation toward results (Sundström, 2006). Similarly, improvements in the form of greater innovative activity were found in Flemish and Dutch agencies in various policy areas (Kraak and van Oosteroom, 2002; Verhoest et al., 2007). However, evidence in this particular area is mixed with another study finding no significant improvements in innovation in Belgium and Norway (Lægreid et al., 2011). These findings show that in some Western European countries (specifically Austria, Belgium, the Netherlands, and the UK), the creation of agencies has intensified and improved management processes as a result of greater managerial flexibility, and some evidence exists of cultural change with a greater emphasis on identifying service users’ needs and achieving results. This has not been the case, however, in those countries where agencies were granted limited autonomy. The French experimentation with setting up regional healthcare agencies is a case in point where tight central control left room for only limited managerial flexibility (Fargeon et al., 2002). The data seem to indicate that agency managers need to enjoy a high-enough degree of autonomy to “give them space” to make decisions and incentivize them to think tactically and strategically to find ways to innovate and assume responsibility for performance.
Despite this positive pattern the data show that the effects of agencification have been more complex and nuanced, and not uniformly positive. A number of unintended consequences—ignored by the more optimistic proponents of agencification—are emphasized by the authors of the studies. This is a third pattern for which this review finds evidence.
Unintended consequences: Mixed and nuanced effects of agencification
The main unintended consequences, identified by 13 studies (46% of the sample), consist of lowered organizational stability, diminished central coordination, and increased fragmentation. Issues of national coordination in a context of agencification have been found in Austria (for example, Greiling, 2011; Hammerschmid et al., 2008; Rossmann and Leitsmüller, 2010), Belgium (Bouckaert et al., 2010; Spanhove and Verhoest, 2008), Hungary (Hajnal, 2010), the Netherlands (Kraak and van Oosteroom, 2002), Norway (for example, Christensen and Lægreid, 2009), and the UK (James, 2003; Pollitt et al., 2004). Agency autonomy—depending on its degree—has created organizational incentives to improve performance, but it has raised challenges to coordination and performance of the whole system. Although agencification has not “invented” the need for systemic coordination, processes of autonomization have reinforced it. The sample, however, points to limited empirical evidence documenting the effects of coordination problems. It readily describes the need for coordination, but efforts to improve coordination need to be based on evidence on what negative effects these coordination problems have reinforced. This is an area where further research is needed especially in the current context where “joining-up” policies are promoted in various European countries.
A second unintended consequence concerns transparency and accountability. Seven studies in the sample (25%) have found evidence of either a decrease or—interestingly—an increase in accountability and/or transparency. For example, in Austria, Greiling (2011) found that agency activity and leadership became less transparent due to the use of shadow budgets coupled with rising costs of top executives. Increased fragmentation, due to a large number of organizational forms in the agency landscape in Flanders, diminished transparency (Spanhove and Verhoest, 2008). In other countries, however, performance management and agencification enhanced accountability and transparency depending on the clarity and comprehensiveness of reporting guidelines, frameworks, and performance measures developed by ministries (for example, Sundström, 2006 in the case of Sweden or see Pollitt et al., 2004 and Pollitt, 2006 for evidence in various sectors in Finland, the Netherlands, Sweden, and the UK). We see how these factors affect the “success” of agencification in improving accountability and transparency. Reporting requirements of ministries on agency activity and performance need to be clear and understandable to improve agency-ministry communication. Poor information exchange, by contrast, can reinforce agencies to look “inwards” and pursue their own targets leading to a loss in central steering and coordination.
Conclusions
Agencification has been seen as a route to economies (input minimization) to the professionalization of management (better processes), to greater efficiency (an improved input/output ratio), and ultimately to improvements in outcomes. The evidence, however, becomes thinner the further one moves toward outcomes—we know much more about how the creation of agencies affected the internal relations within ministries than we do about how they actually changed service quality. A count of the type of effects evaluated in the sample indicates that relatively little is known about outcomes: that is effects outside the administrative system and the so-called “democratic effects” —effects on legitimacy and accountability toward society (Van Thiel, 2012: 432–435). Of the sample reviewed in this study, 36% includes some type of evidence on outputs, efficiency or outcomes. However, only 7% of these discuss outcomes.
The empirical research across Western Europe reviewed here found evidence of improvements in the organization and management processes of agencies following an increase in managerial autonomy. Likewise, evidence exists of cultural change with greater interest and emphasis on achieving results and focusing on service users’ needs and demands. These two types of evidence seem consistent across policy areas and countries, and is one main pattern to which this review points. They show that agencification can and has led to or reinforced certain positive effects in the organization, management, and delivery of public services. The issue is that little is known about what outcomes these positive developments have led to and whether they have much—or anything to do—with the creation, autonomy, and management of agencies.
A second main pattern shows that the creation of agencies has led to nuanced and mixed effects with improvements in some areas and insignificant changes or even deteriorations in other areas. A total of 46% of the sample found evidence of problems with organizational stability, fragmentation, or coordination. While this pattern is common across different European countries, these challenges varied depending on the intensity of agencification and the effectiveness of coordination mechanisms and instruments put in place to address fragmentation and coordination problems. This type of problems has by far been the most frequently identified in the reviewed sample. A second unintended consequence refers to concerns with accountability and transparency, but evidence is mixed with only two studies finding deteriorations in Austria and Belgium, while six studies found evidence of improved accountability and transparency particularly in Sweden and the UK. Key factors that can facilitate the accountability and transparency of agencies include improved communication with stakeholders, both political and the public, clear and comprehensive exchange of information, guidelines, and reporting requirements coupled with understandable contractual frameworks and performance measures.
This paper has looked at the effects of agencification and reviewed a sample of studies of agencies derived from the NPM literature. There are good reasons to do this. Agencification in a context of performance frameworks has been a key pillar of the NPM and has spawned great interest for academics and practitioners alike. This is reflected in the number of studies of agencies—both single-country and comparative—that have been conducted in the past two decades. It needs to be acknowledged, however, that agencies have been created for other reasons and strands of literature other than NPM may well include evidence on the impacts of agencies. The goal here was to look at the NPM literature on agencies, and the sample is based on a large database of studies of NPM. In this sense, it can be argued first that the net was cast wide enough by including in the sample a variety of studies across Europe. Second, by carefully reviewing 28 studies selected based on the strength of method and evidence, the article sought to identify nuances and possible patterns based on empirical literature that provided more convincing evidence of effects.
The priority for future research into agencies should be the direct study of changes in outputs and, especially, outcomes. For the public, these are the real “results” of reform, but they have been far less often studied than changes in organizational structures and processes. That, of course, is partly because they are more difficult to study practically and politically. Once created, agencies have undergone change in their organization and management as well as in the agency landscape. This has taken place in a context of cost cutting and need for savings which translated into reductions in the number of agencies, recentralization, and central steering and coordination efforts. A relevant second line of research into agencies is to document whether these changes to the “autonomizing” agency model proved effective. Compared to the previous state of affairs, did they bring about the anticipated savings, improvements in quality, and access to services? Did they improve outcomes or has there been a drop in outcomes, and if so, can they be convincingly attributed to changes in the original agency model? These questions form a second line of needed research into agencies.
In conclusion, the picture of the effects of agencification is more nuanced than the proponents of agencification had originally envisaged. Unintended consequences received little consideration in the models that saw agencification as a panacea for various public sector ills. In this sense, the evidence reviewed here challenges the original model of agencification proposed years ago by enthusiastic public choice theorists in the Anglophone world. However, this is only part of the story. As this review clearly shows, the agency model has led to a series of positive effects – mostly in processes or activities – across European public sectors. In the current context of fiscal stringency with a growing interest in “joining-up” initiatives and reductions in the number of agencies in various European countries, the “autonomizing” model—the more disaggregation and the greater autonomy, the better—has already come under fire. This seems fair, but the challenge is not to transform, let us call it, “deagencification” in another yet know-it-all model. There are inherent trade-offs, paradoxes, and unintended consequences, some of which are clear in this article, between granting organizational-level autonomy and centrally consolidating control. It can be expected that stronger central coordination at the expense of organizational autonomy may lead to disincentives to innovate and overall assume responsibility for decisions and results. There may well be a price that needs to be paid. The question, however, is whether reforms of the agency model aimed at greater central control, coordination, or steering bring about savings and greater “systemic” performance in the public sector compared to a disaggregated and autonomous agency model.
Footnotes
Acknowledgements
This article is based on a co-authored report prepared for the Dutch Senate into the effects of agencification and privatization on the relationship between citizens and the national government led by Sandra van Thiel. Christopher Pollitt helpfully reviewed a previous draft. Two anonymous reviewers provided additional helpful comments.
Funding
The research leading to these results has received funding from the European Union's Seventh Framework Programme under grant agreement No. 266887 (Project COCOPS), Socio-economic Sciences & Humanities.