Blame avoidance,scapegoats and spin: Why Dutch politicians won’t evaluate ZBO-outcomes

Abstract

Despite high expectations about the results of agencification and a legal obligation to evaluate executive agencies, ministers and MPs seem not very interested in evaluating agencies’ results. Hood’s theory on blame avoidance is used to explain the lack of evaluation in the case of the Dutch ZBOs. Only one in seven ZBOs is evaluated as frequently as mandated. Findings show that ZBO evaluations are more an administrative than a political process. Reports do not offer hard evidence and are seldom used in parliamentary debates. There are no clear patterns as to which ZBOs are evaluated more, or less, often.

Keywords

Blame avoidance blame games executive agencies evaluations

Introduction

Agencification came with high expectations. Compared to government bureaucracy, executive agencies were expected to operate more business-like, provide more value and quality of public services for less money, operate at a distance from politics and closer to citizens, ensure more transparency and accountability, and offer a more motivating work environment for employees (Overman, 2016). Little is known however about the actual results, mostly due to a generic lack of formal evaluation studies on the results of these types of reforms (Pollitt and Dan, 2013).

A lack of formal evaluations of policy outcomes is not uncommon (McCubbins and Schwartz, 1984). Different explanations are offered for the lack of interest among politicians – members of parliament and ministers alike – for evaluating policy implementation. McCubbins and Schwartz (1984) posit that politicians prefer incidental (‘fire alarm oversight’) and informal evaluations over formal and systematic evaluations (‘police patrol oversight’) as this is a more efficient strategy, fitting with opportunity costs, available technology and limited human cognitive abilities. Busuioc and Lodge (2017) point to reputation effects as explanation; evaluation is only undertaken when it contributes to the reputation of either the accountholder or the account-giver. Hood (2007, 2011) and Weaver (1986) mention the inclination of public office holders to avoid blame as the main reason for not wanting to evaluate the outcomes of their decisions, while other authors show that even if evaluation information is present many politicians are not interested or cognitively unable to use that information to properly evaluate their policies and decisions regarding policy execution (Askim, 2007; Dubnick and Frederickson, 2010; Nielsen and Baekgaard, 2015; Schillemans and Busuioc, 2015; Ter Bogt, 2004).

This article will delve into the lack of evaluations in the case of Dutch ZBOs, a specific type of executive agency (more explanation below). Despite the high expectations and a legal obligation to evaluate these agencies every five years, only 14.4% is evaluated as often as they should be. The central research question is how this lack of evaluation, and the apparent disinterest of politicians in agency evaluation, can be explained.

Because one cannot study evaluations that did not happen, I will use 102 reports that have been published so far, to examine which ZBOs have been evaluated compared to the overall population. This will offer insight into which types of ZBOs have been evaluated, but also which ones not. Furthermore, I will look into why, how and what is evaluated, and how evaluation reports are used in political decision-making, as this may also shed light on the question when politicians are interested in evaluations, and by implication when not.

The outline of this article is straightforward. First the theoretical framework is presented. I will use Hood’s theory as an integrative framework for a number of explanations and deduce six testable hypotheses. The method section is next, describing the data, mostly from public sources, and the methods used to test these hypotheses. Based on the results, conclusions are drawn and theoretical implications are discussed.

Blame games

Multiple authors have observed a lack of evaluation of policy implementation. McCubbins and Schwartz (1984) were one of the first to discuss the lack of oversight by Congress. They concluded that this is not to be lamented, but rather understood as a very logical and efficient choice. Instead of spending a lot of time and energy on systematic forms of formal evaluation (‘police patrol’) politicians focus on incidental cases that attract their attention (‘fire alarms’) or are of direct concern to them and their voters. Following a comparable line of reasoning, Busuioc and Lodge (2017) posit that accountholders like MPs and ministers will only call for information if it will contribute to their reputation – or damage the reputation of opponents. Saliency and reputation are also important elements in Hood’s (2011) theory on blame avoidance. This theory departs from the observation that all public officeholders (politicians, managers, professionals and front-line bureaucrats) face risks of personal blame while carrying out their tasks. Examples are easy to find, for example, when ministers are held to account in parliament for certain incidents or when media report on public organizations’ managerial, financial or performance problems. Hood explains that evaluations are not undertaken, to avoid blame for poor performance. Besides arguments of saliency and reputation, other authors also point to the limited cognitive ability of politicians to process vast amounts of expert information (Askim, 2007; Dubnick and Frederickson, 2010; Nielsen and Baekgaard, 2015; Ter Bogt, 2004) or their limited amount of time to spent on such a task.

In sum, there are various explanations for a lack of evaluation of policy implementation and the executive agencies in charge of policy implementation. In this article, I will put these explanations to the test in the case of Dutch ZBOs. To that end I will have to make the explanations more concrete and pinpoint exact conditions under which politicians will be more interested in evaluating ZBOs, and when not. As Hood’s theory offers a more comprehensive explanation, with multiple factors to explain a lack of evaluation interest, I will use this as the overarching framework while integrating the other explanations as mentioned before. This way I can pay attention to both variables related to politicians and ZBOs, as well as characteristics of evaluations and the evaluation process.

Agency strategy

Hood (2011, 2007) posits that public officeholders can use three strategies to reduce or even avoid the risk of being blamed for certain policy outcomes and prevent reputational damage (cf. Busuioc and Lodge, 2017): presentational, agency and policy strategies (cf. also Weaver, 1986).

Presentational strategies focus on limiting the blame by making excuses or giving justifying decisions, to change the perception of what went wrong. Policy strategies, sometimes also referred to as operational strategies, aim to minimize risks by introducing new routines and protocols to limit discretion and non-compliance with rules and procedures during implementation. Agency strategies consist of the delegation of tasks and hence redistribution of the blame to others, for example to semi-autonomous agencies. See Table 1 for some examples of each of these strategies.

Table 1.

Examples of blame avoidance strategies by public officeholders.

Strategy	Examples
Presentational	Keep a low profile (‘no comment’); win the argument (justification); change the subject (divert attention); draw a line (excuses)
Agency	Delegation; defensive reorganization and staff rotation; partnership structures (share the blame); blame the market (external attribution)
Policy/ Operational	Abstinence (closing down services); individualization (impose ethical standards); protocolization (formal rules); herding (collective blame)

Source: based on Hood, 2011.

Agency strategies are directly linked to the object of study in this paper: ZBOs, which are a Dutch type of semi-autonomous agency, in charge of policy implementation, regulation or public service delivery. (More explanation about ZBOs will be given in the method section below.) By delegating tasks to agencies, politicians can shift or avoid blame when things go wrong (Cohn, 1997; Hood, 2002). This was however not mentioned by politicians as an official motive (nor admitted playing a role) when agencies were created in large numbers in western countries from the 1980s on (Pollitt et al., 2004; Van Thiel, 2001). Instead, two other motives were given. First, it was expected – along the lines of New Public Management ideology – that agencies would operate more business-like and hence more efficiently than the traditional government bureaucracy. Second, politicians can use agencies to assure voters that public tasks will be carried out by impartial and expert officeholders, forfeiting the possibility to interfere with task execution for political or personal reasons. Moreover, delegation of a task to an impartial and expert agency will also ensure the continuity of task execution even after elections and changing rulers. Majone (2001) labels this the logic of political credibility. In light of Hood’s theory on blame (2011) this motive could also be construed as an example of ‘credit claiming’; politicians can claim the success of expert, impartial and longstanding public services. However, Hood upholds a smaller definition of credit claiming; he argues that a decision to delegate will render credit only if the agency is performing well. The decision to create an agency as such is not sufficient to claim credit.

Given the expected benefits of agencification, the question can be raised why politicians would ever choose direct control over a public service rather than delegate them all (cf. Hood, 2002). Hood (2011: 72–29) suggests that in some cases politicians may have a personal preference for direct control, while in other situations not taking control is simply not an option and seen as failure, leading to blame. Blame cannot always be avoided (Novak, 2013; Weaver, 1986). However, there are different types of delegation with varying degrees of control and discretion – ranging from soft delegation with limited autonomy to hard delegation with high degrees of autonomy – which may allow for a combination of direct control and delegation, offering multiple opportunities to claim credit and avoid blame. In general, however, Hood’s theory predicts that tasks with higher risks of blame will be delegated to agencies more often than tasks with lower risks of blame. And tasks with higher chances of claiming credit will be less often delegated to agencies than tasks with lower chances of credit.

If these predictions are correct, it is clear why politicians would not want to know how agencies are performing: the chances of credit claiming are small, while the risk of blame is high, which is why the agency was established in the first place. This effect is even stronger as an agency has been granted more autonomy (i.e. in case of hard delegation). This leads to the following hypothesis:

Hypothesis 1: ZBOs with higher degrees of autonomy (hard delegation) are evaluated less often than ZBOs with less degrees of autonomy (soft delegation).

Hood’s predictions attest to the effect of negativity bias. This refers to the tendency of people in general, and public officeholders in particular, that it is more important to avoid negative information that to gain positive information or claim success (Weaver, 1986). Hood (2011: 10–11) relates this attitude to the high degree of risk aversion in the public domain, and the increase in the use of risk management techniques. Negativity bias creates a tendency among public officeholders to avoid information, even if it is positive information. Or, as Hood posits:

whether or not privatization, agencification, and outsourcing of public service provisions really do cut costs, improve quality, or produce all the other effects that are so confidently and earnestly claimed for them (on the basis of so little hard evidence), what those arrangements can offer is the apparent prospect of shifting blame away from politicians and central bureaucrats to private or independent operators. (Hood, 2011: 68, original emphasis)

When we translate Hood’s ideas to the evaluation of ZBOs, we would expect a generic aversion of politicians to evaluate them for two reasons. First, the agency is put at arm’s length to avoid blame, for example, because the agency is charged with a task that is unpopular with voters (cf. Cohn, 1997), or its establishment is intended to deal with poor performance or incidents (Hood, 2011). The risk of negative information coming out of an evaluation would then be too high (negativity bias). Second, politicians might be interested in an evaluation, but only if it can be expected that the evaluation will show a positive result – enabling credit claiming. However, performance measurement is not an easy task when dealing with agency performance (Overman, 2016). Most agencies are monopolists and therefore lack a benchmark. Moreover, public organizations generally have multiple and ambiguous goals, which are difficult to put into indicators. Therefore, the chances of ‘hard’ evidence coming out of an evaluation are scarce. The meta-analysis by Pollitt and Dan (2013) corroborates this as they found that most evaluations focus on processes rather than outputs, often based on qualitative rather than quantitative analysis. This leads to the following hypothesis:

Hypothesis 2: ZBO evaluations will use more qualitative (soft) than quantitative (hard) information.

At the same time, research into agencification has shown that politicians cannot always avoid blame (Flinders, 2008; cf. Hood, 2002; Novak, 2013; Pollitt, 2005; Verhoest et al., 2012b). Politicians are frequently held accountable for agencies and their performance, regardless of the formal delegation of such responsibilities. Following Hood (2002), this is most likely to occur in case of fierce political competition (when parties try to lay blame on each other, cf. Busuioc and Lodge, 2017), or in case of incidents in policy implementation by agencies that affects voters directly – as their dissatisfaction will lead them to vote for another person or party in the next elections, something which politicians will want to prevent (McCubbins and Schwartz, 1984). In such situations blame is by default attributed to the delegator (Hood, 2002: 22), which in most western countries is the minister of the parent ministry,¹ who will then have to resort to other strategies to mitigate this “blame reversion” or ‘”blame boomerang” (Hood, 2002: 27–29). Evaluations could in fact be used as a blame avoidance strategy under such conditions, for example to divert attention from the incident (a presentational strategy), or to collect information for a decision on reorganization (an agency strategy) or to close down the agency all together (a policy strategy). Commissioning an evaluation can thus also be used to avoid blame (Schillemans and Busuioc, 2015). Therefore, it is of interest who commissions an evaluation. This article concerns ‘politicians’ in a broad sense, encompassing both members of parliament and/or members of political parties, either ruling parties or opposition, as well as ministers in charge of ministries and agencies at arm’s length. Weaver (1986) states that incumbents will always be more interested in blame avoidance than credit claiming compared to for example opposition parties and interest groups, who will want to lay blame on the incumbents to suit their own purposes. Opposition and interest groups would therefore be expected to call for evaluations more often:

Hypothesis 3: ZBO evaluations are carried out more often by order of opposition parties (or parliament) than ruling parties (represented by the minister).

Agencies themselves might be interested in evaluations for another reason, namely, to claim success or offer justifications for poor performance (a presentational strategy), or to protect their reputation (Busuioc and Lodge, 2017) particularly in cases where there have been incidents. Incidents are most likely to become exposed when tasks are highly salient to voters, as these attract more attention from the media, the public and parliament (Dudley, 1994; Weaver, 1986). For politicians, ZBOs with tasks that are salient to voters are of great interest, and hence more likely to be evaluated (as a ‘fire alarm’, McCubbins and Schwartz, 1984). On the other hand, evaluations of ZBOs with salient tasks are also a greater risk to the own reputation and a potential source of blame (Busuioc and Lodge, 2017; Hood, 2011) and can hence expected to be carried out less often. This implies two contradictory hypotheses:

Hypothesis 4a: ZBOs in charge of tasks salient to voters are evaluated more often than ZBOs with non-salient tasks.

Hypothesis 4b: ZBOs in charge of tasks salient to voters are evaluated less often than ZBOs with non-salient tasks.

Incidents are also expected to occur more often in case of younger organizations because they lack experience both at operations as well as the political game with the parent ministry (Yesilkagit, 2014). Therefore, I would expect politicians to be keener to evaluate ‘younger’ ZBOs more often:

Hypothesis 5: ‘Younger’ ZBOs are evaluated more often than ‘older’ ones.

It has been pointed out that politicians can use evaluation reports to their benefit, either to claim credit in case of good results or to attribute blame to others (a presentational strategy, Hood (2011)). There is a third aspect that is of interest here. In order to deflect blame politicians can try to manipulate the results, for example by determining who is put in charge of an evaluation. Politicians can prefer to do this themselves (hoping to be able to frame the report to their benefit, a presentational strategy) or, for example, charge an independent investigator (to show the voter that the incident is taken seriously and done by an impartial expert, a policy strategy). It is difficult to deduce a clear prediction here, so I have not formulated a hypothesis but an open question: by whom are ZBO evaluations carried out?

Similarly, the question is relevant what will happen with the results of the report. Above it was argued that reports will be of use to politicians only if they can claim credit, but credit claiming is very difficult in the case of policy implementation. This would suggest that reports are not often used. Other indications that point to the same result would be twofold. First, if an evaluation is only used to divert attention (a presentational strategy, Hood (2011)) or, second, if it is just part of an administrative process (Busuioc and Lodge, 2017). In both instances I would not expect to see much use of a report by politicians:

Hypothesis 6: ZBO evaluation reports are not used by politicians.

Data and method

Dutch ZBOs (acronym of ‘ z elfstandige b estuurs o rganen’) are public bodies that have been granted a relatively high degree of managerial autonomy (Van Thiel, 2012). In Hood’s concept they would fall between soft and hard delegation. ZBOs operate at arm’s length of the government; they are structurally disaggregated and sometimes even financially independent (cf. Pollitt et al., 2004; Talbot, 2004). They are comparable to for example the British Non-Departmental Public Bodies, the French public establishments, and the New Zealand Crown Entities (OECD, 2002; Verhoest et al., 2012a). ZBOs can be found in all policy areas, ranging from welfare (e.g., paying benefits), to public transport (e.g., railway infrastructure), finances (e.g., regulating markets) and agriculture (e.g., food quality). ZBOs exist only at the national level of government, although they may have multiple local offices. In 2015, there were 118 ZBOs according to the official register (online at the website of the Home Office). However, when we include all the local offices, the total number of agencies amounts to over 730 (Van Thiel and Verheij, 2017).

In 2007, the Charter Law on ZBOs became effective after a ten-year long debate in parliament. Article 39 of the Charter Law stipulates that every ZBO has to be evaluated every five years, to assess its effectiveness and efficiency (article 39). The minister of the parent department of a ZBO has to send the report of this evaluation to parliament, which will make it a public document. Before the Charter Law came into effect in 2007, there were official guidelines on ZBOs in place (from 1996 onwards) that also included a formal obligation to evaluate each ZBO every five year. There is no format prescribed for how to carry out this evaluation, and there are no further instructions besides the two criteria (efficiency and effectiveness), nor is there an authority in charge of checking whether parent ministries comply with this article of the Charter Law. More recently, reports have been included in the official ZBO register (online), although not all.

In this study, evaluation reports were included that were published until and including 2015. Most reports were found in the official government document repositories (www.rijksoverheid.nl), some were found online for example on ZBO-websites. Following the list of ZBOs in 2015, we tracked for each ZBO which reports were published, identified as a formal evaluation, and made public. Reports until 2011 were collected as part of a parliamentary inquiry for the Dutch Senate (POC, 2012), by an information specialist from the Dutch parliament.² Older reports were digitalized as they were only available on paper (in the parliamentary library). Reports from the period 2012-2015 were collected as part of five master student projects (supervised by the author); the students first compiled a list of ZBOs and then searched for reports in the online government repository, and in some cases from the websites of the ZBOs themselves for easier access. Appendix 1 lists all reports that were included in the analysis (and which are available upon request to the author).

To enable the comparison with population data, information on for example the task and date of establishment of ZBOs was taken from earlier surveys by the author (Van Thiel and Yesilkagit, 2014), although it has to be noted that data from years before 2008 were not always available or complete (then left missing). All data were analysed using SPSS 25, using simple statistical techniques (e.g., descriptives and cross tabulations) because of the nominal/ordinal nature of most variables.

The analysis of the usage of reports in parliamentary debate was carried out as part of the POC study, and findings have only been summarized and translated for this article. 18 reports were selected for 10 ZBOs, varying in the type of report (by a consultancy or the ministry) and over different policy sectors. The minutes of parliamentary debates in which these reports were put on the agenda were analysed (content analysis), searching for references to the evaluation report. When such references were found, the text was further analysed to determine what was being said and by whom (POC, 2012). In addition, four interviews with parliamentary clerks were held to determine how evaluation reports are dealt with (see POC (2012) for more details).

Results

Based on the guidelines and Charter Law, it is legally obligatory to evaluate each ZBO every five years. Figure 1 shows the increase in the total number of reports (N = 102) that have been published until 2016, in five-year intervals. The first reports we found date back to the early 1990s, so even before it became legally obligatory. The number has since then indeed increased strongly. As most reports (62.7%) have been published in or after 2007, we can assume that the Charter Law has spurred this growth. However, although the number of ZBOs that have been evaluated has increased, most of them have been evaluated only once (31, 53.4%) and not every five years as is legally mandatory. The 102 evaluation reports cover 58 (out of 118) individual ZBOs; 15 of those (25.9%) have been evaluated twice, 9 (15.5%) have been evaluated thrice, and 3 (5.1%) have been evaluated four or five times. That implies that non-compliance with the Charter Law is quite substantial: 51% of the ZBOs has never been evaluated, and of the 58 ZBOs that have been evaluated only 17 (14.4%) have been evaluated multiple times, every five years or even more frequently.

Figure 1.

Number of ZBOs and ZBO-evaluation reports 1995–2015.

Which ZBOs are evaluated?

When we compare the ZBOs that have been evaluated to the ones that have not been evaluated, we find some noteworthy differences. Public law based ZBOs are evaluated more frequently than private law based, even when taking into account that there are more public law based than private law based ones: 69% of the evaluated ZBOs have a public law basis (against 56.3% in the population), 23.7% of the evaluated ZBOs have a private law basis (against 43.7% in the population). If we assume that public law based ZBOs are more similar to what Hood would call ‘soft’ delegation and private law based ZBOs are more exemplary of ‘hard’ delegation, this finding contradicts the theoretical expectation (H1) formulated above.

Task effects

There are some task differences. ZBOs in charge of financial transfers (e.g. paying benefits), regulation and research/information related tasks have been evaluated more often than would be expected based on the proportion of ZBOs with such tasks in the population, see Table 2. ZBOs in charge of certification, licensing and quality assessment (e.g. as part of accreditations) are evaluated less often.

Table 2.

Percentage of ZBOs being evaluated, per parent ministry.

Parent Ministry	Population	Evaluated ZBOs	Difference
Home Office	8.7%	5.5%	–
Education, Culture and Science	4.9%	20.0%	+
Economic Affairs & Agriculture	26.2%	21.8%	–
Health, Welfare and Sports	20.4%	10.9%	–
Security & Justice	13.6%	18.2%	+
Infrastructure & Environment	17.5%	10.9%	–
Finances	3.9%	9.1%	+
Social Affairs and Employment	2.9%	3.6%	0

Numbers may not always add up to 100% due to missing data in individual cases.

It was expected (H4a) that tasks that are more salient to voters would be evaluated more often. It is hard to say whether or which of these tasks is more salient than the others, but it could be argued that financial transfers most directly affect voters – which would make them highly salient. Moreover, their output might be easier to measure and hence suitable for evaluation. ZBOs with regulatory tasks are also evaluated more often than would be expected based on their proportion in the population. Regulatory tasks are often put at arm’s length to ensure impartiality, in line with Majone’s (2001) argument about political credibility. The higher frequency of evaluation could then be seen as part of an agency strategy (Hood, 2011) and of credit claiming. Finally, ZBOs in charge of research and knowledge related tasks could be more open to evaluation as that fits with their own purpose, but that is pure speculation.

Certification and quality assessment could be considered as a type of task for which specific expertise is crucial, creating higher risks of blame (or lower chances of credit claiming) and hence less interest among politicians to call for an evaluation (H4b) – but again that is speculation and it calls into question whether these tasks have an inherent quality that makes them more suitable for agencification than other tasks (cf. Pollitt et al., 2004, on the task specific path dependency theory). Agencies in general, as well as ZBOs more specifically, have been charged with a large array of tasks, of which certification and quality assessment are important categories but not the most frequent ones (Van Thiel and Yesilkagit, 2014). An alternative explanation for the lack of attention for ZBOs with these tasks could be that their output is more difficult to measure, but that does not seem plausible: certificates and accreditations are quite tangible results. More research is therefore necessary to help explain these findings better.

Sector differences might shed some light on this matter. The two parent ministries (Economics and Health) with the largest numbers of ZBOs both have lower numbers of evaluations, see Table 3. This could in part be attributed to the fact that many of their ZBOs are private law based (see above on H1). Other ministries, like Education, Justice and Finance have proportionally more ZBO evaluations than expected, which could be linked to the tasks of the majority of ZBOs in their policy sectors: education ZBOs do research, financial ZBOs are often regulators. However, there is no clear explanation for ZBOs in the field of Justice, so that will require more in-depth case study research.

Table 3.

Percentage of ZBOs being evaluated per task.

Task	Population	Evaluated ZBOs	Difference
Regulator	18.4%	29.1%	+
Quality assessment	10.7%	9.1%	0
Financial transfers	18.4%	23.6%	+
Certifying and licensing	19.4%	7.3%	–
Registration	6.8%	1.8%	–
Tribunal	2.9%	3.6%	0
Research, education, knowledge	9.8%	16.4%	+

Numbers may not always add up to 100% due to missing data in individual cases.

All in all, task may matter but it is not straightforwardly clear how. A more case by case analysis would be necessary to get more clarity (on H4a and H4b).

Age effects

There is no statistically significant correlation between age (i.e. the year of a ZBO’s establishment) and being evaluated, refuting the theoretical expectation that younger ZBOs are evaluated more frequently (H5).

Motives for evaluation

Table 4 shows the motives that were mentioned in the reports for doing the evaluation. Although 80% of the ZBOs that have been evaluated fall under the Charter Law, the legal obligation to evaluate is mentioned in only 17 cases. This makes sense when one considers that the Charter Law became effective in 2007 and some reports predate that, but even in the 64 reports that have been published since 2007 this motive is mentioned in a minority (15 out of 64, 23.4%)³ of the cases. Changes in the legal basis or task of a ZBO are the most frequent motive (23.5%), while incidents (10.8%) and parliamentary requests (7.8%) are mentioned much less often, contrary to what was expected (H3). ZBOs also take the initiative themselves to be evaluated, sometimes as part of a peer review procedure.⁴ This fits with the idea that account-givers will actively seek accountability to improve their reputation (Busuioc and Lodge, 2017).

Table 4.

Motives for carrying out an evaluation (N = 102).

	Number of reports
Legally mandatory	17 (16.7%)
Request in parliamentary debate	8 (7.8%)
Incident	11 (10.8%)
Own initiative of ZBO	17 (16.7%)
Changes in legal basis or task	24 (23.5%)

Note: multiple motives could be mentioned.

Commissioners and evaluators

Table 5 shows who commissioned the evaluation and who carried it out. In most cases (68.6%) the ministry i.e. civil servants are the commissioner and the evaluation is carried out by a consultancy firm (50%). This pattern holds regardless of the motive mentioned for an evaluation. In the case of a peer review evaluation, the ZBO is usually the commissioner itself. ‘Other’ commissioners are for example an audit office or the board of supervisors of the ZBO. Independent committees that perform an evaluation have very mixed compositions; they may for example include civil servants (from the parent ministry or the Senior Civil Service), academic experts, or former/retired politicians. All in all, the findings in Tables 4 and 5 suggest that evaluating ZBOs is more an administrative process than a political one (Busuioc and Lodge, 2017), refuting H3. ZBO-evaluations do not have a high priority for politicians.

Table 5.

Who has commissioned and who has carried out a ZBO-evaluation? (N = 102).

Commissioner		Evaluator
Minister/ministry	70 (68.6%)	Consultancy	51 (50%)
ZBO	14 (13.5%)	Peer review	14 (13.7%)
Other	9 (9.8%)	Audit unit parent ministry	8 (7.8%)
		ZBO	6 (5.9%)
		Audit office	3 (2.9%)
		Academics	4 (3.9%)
		Independent committee	11 (10.8%)

Topics of evaluation

The legal obligation to evaluate ZBOs specifies that the evaluation should focus on efficiency and effectiveness over the past five years. Table 6 shows which topics are covered in the reports (note that multiple topics can be covered in one report).

Table 6.

Topics mentioned most frequently in ZBO evaluations (N = 102).

Topic	Mentioned in reports
Efficiency	79 (77.5%)
Relationships with stakeholders	72 (70.6%)
Effectiveness	71 (69.6%)
Task execution	70 (68.6%)
Results	64 (62.7%)
Quality	55 (53.9%)
Governance	51 (50.0%)
Finances	49 (48.0%)
Legal status	23 (22.5%)

As mentioned in the Charter Law, efficiency and effectiveness are the most important topics (although not in all). However, the relationship with stakeholders scores equally high. To me, as a researcher who has studied ZBOs for over 20 years this is not surprising as the relationship between ZBOs and their parent ministries is complicated and often dysfunctional (Van Thiel, 2019). However in most reports relationships with stakeholders are not mentioned in the central question nor as a motive for the evaluation; in my experience, it comes up during the evaluation process as something that deserves attention and therefore is reported on – mostly in a negative tone. The attention for governance in half of the reports is connected to this.

Several other performance related topics such as task execution, results and quality are reported on, fitting with the focus on efficiency and effectiveness (but not mentioned in the Charter Law). And even though the legal status is mentioned in the explanatory notes of the Charter Law as a topic for evaluation, it is discussed in only a minority (22.5%) of the reports. From the perspective of blame avoidance (Hood, 2011) this could be interpreted as fitting with the agency strategy to put ZBOs at arm’s length and leave them there.

Methodology

The lower coverage of financial matters (see above) fits with the qualitative nature of most reports. Most evaluators use a combination of (a) interviews (81.4%) with respondents from the ZBO, its parent ministry and stakeholders, and (b) content analysis of documents (85.3%), usually in a qualitative way. About 40% of the reports displays figures for example on personnel, budgets and results, but with limited analysis thereof. A qualitative approach can be explained by measurement problems with policy results, but it could also be interpreted as avoidance of ‘hard’ results – to avoid blame (in line with H2).

Use of evaluation reports

Finally, we turn to the use that politicians make, or do not make, of the evaluation reports. Most reports (73, 71.6%) are sent to parliament, with an accompanying letter in which the parent minister expresses agreement (62.2%) or disagreement with the conclusions. Most reports are however not discussed in parliament at all. Table 7 offers a few examples, taken from the POC (2012) study (note that this includes reports until 2012 only). These examples show how 18 reports on 10 selected ZBOs were dealt with in the parliamentary debates.

Table 7.

Discussion of ten ZBO evaluation reports in Dutch parliament.

ZBO	Short summary of parliamentary debate
AFM (Financial regulator)	Report from 2012, written by Ministry of Finance (parent ministry). Contents of the report are not discussed. Minister mentions the report during a discussion on a new proposal regarding oversight on financial regulators.
Bureau Architectenregister (Register of architects)	Report from 2001 by consulting agency is not sent to parliament. Minister mentions the report when discussing the revision of the law on the right to use the title of architect and changes in the ZBO statutes.
Centraal Fonds Volkshuisvesting (Housing Fund)	Report is written by the accountancy unit of parent ministry (VROM) in 2003. Report is included in the discussion about the financial oversight on housing associations (the main task of the Housing Fund). Recommendations from the report are quoted during the parliamentary debate by members of parliament, and they ask questions about the results of the study.
Commissariaat voor de media (Regulator media)	Report from 2008, written by consultancy agency, is sent to parliament and put on the agenda for the meeting on the budget of the parent ministry (Education Culture and Science). However, there is no reference to the report in the parliamentary debate.
Nederlands Zorginstituut (CVZ) (Advise on health insurance policy)	Only the second evaluation (by a consultancy agency in 2009) is sent to parliament. The report is mentioned but not discussed substantively during the debate on the new health insurance legislation.
Kamers van Koophandel (Chambers of commerce)	Report was written by consultancy agency in 2010. The cabinet response to the report is mentioned but not discussed when the cabinet presents new plans for a merger and change in the statutes of the ZBO to parliament.
KCB (Quality inspection of fruit and vegetables)	A project group has carried out the evaluation in 2004. The report is included in the list of documentation for a discussion on the organizational structure of all four quality inspection agencies in this sector (three other ZBOs next to KCB). The report is however not discussed substantively.
OPTA (now ACM) (Regulator telecommunications)	Three reports (200,120,052,009), all by consultancy agencies, are all sent to parliament. Only the second report is discussed substantively, as evident from references and quotes in the minutes. The third report is included in the list of documentation in preparation of a debate on the future of the ZBO, but not discussed.
LSOP (Police Academy)	Three reports (1996, 1999, 2007) are all sent to parliament. Authors of the reports are an accountancy agency, the inspectorate and an unknown organization (not mentioned). All reports are discussed during the debates on the financial and managerial problems that the ZBO is going through. The reports are discussed substantively and methodologically. Members of parliament request follow-up research, which they want to include in the decision-making on the future and structure of the ZBO.
Staatsbosbeheer (Forestry agency)	Three reports (2004, 2007, 2009) are produced by two committees and one taskforce. All reports are sent to parliament and discussed in parliament during debates on the steering and oversight on the ZBO by the minister. Particularly, the position of the board of overseers and the legal status as a ZBO are debated during multiple debates. There are many and even literal references to the results of the evaluation. A fourth report from a peer review committee (2006) is not included in the discussion.

Source: based on POC, 2012 (my translation, SvT).

The analysis shows that reports are not always sent to parliament, and even when they are, most reports are not discussed in a substantive way – only in a minority of the cases (8 out of the 18 reports in Table 7) and on an even smaller number of ZBOs (4 out of 10 ZBOs, marked in grey). Reports are generally part of a long list of documentation that members of parliament are given to read in preparation of a debate on new plans from the cabinet. This was confirmed by the interviews with four clerks of parliamentary committees (POC, 2012). The initiative to use evaluation reports thus lies in most cases with the parent minister, although their usage should not be over-exaggerated as most reports are only mentioned in passing and not discussed substantively.

Moreover, substantive discussion of reports does not imply that the evaluation studies were carried out for the topic of the debate in which the discussion takes place; reports are usually included in discussions of new legislative proposals (made by the minister) or how to deal with certain problems or incidents (like in the cases of the policy academy and the forestry agency). The reports have often been published before these debates occurred, before the proposals were developed, or before incidents arose. Most of the reports in Table 7 have been commissioned by the parent ministry and carried out by a consulting agency – like the full sample. Only in the case of the police academy did political parties explicitly request further investigation, although not through an official evaluation.

Based on these findings we can conclude that ZBOs are not only evaluated less frequently than they legally should be, but also that when they have been evaluated the reports are seldom used for further political decision-making and political debate (confirming H6).

Discussion and conclusion

Despite the high expectations underlying the creation of ZBOs (Van Thiel, 2001) and legal obligations to evaluate their efficiency and effectiveness every five years, less than half of the ZBOs has been evaluated once and only one in seven is evaluated as frequently as legally required. This article set out to find an explanation for this lack of evaluating. By looking into reports that were published, we can also establish in which cases evaluations have not been carried out. The findings show that ZBO-evaluations are mostly an administrative process, commissioned by civil servants from the parent ministry and carried out by independent consultants. Political requests for evaluations are rare (H3-), and reports are generally not used for political debate (H6+). Most reports do not present hard evidence (H2+), which probably makes it difficult to pass hard judgements. There are no clear patterns which types of ZBOs are evaluated more or less often. Most hypotheses were not corroborated: ‘younger’ ZBOs are not evaluated more often than older ones (H5-), task differences are found but not very clear (H4?), and private law based ZBOs are not evaluated more often than public law based ones (H1-).

The lack of evaluations and the low interest of politicians was expected, based on among others the theories on the lack of oversight by McCubbins and Schwartz (1984) and the inclination of politicians to avoid blame (Hood, 2011). However, it was also expected that under certain conditions politicians would be interested, for example in the case of incidents (‘fire alarms’ in the terminology of McCubbins and Schwartz, 1984), hard delegation (Hood, 2011) or when tasks were at stake that are very salient to voters (McCubbins and Schwartz, 1984), but this was not found to be true. This shows that the lack of evaluating in the case of executive agencies is very persistent and perhaps even generic, despite the legal obligations. This is in line with the agency strategy, as defined by Hood (2007). The lack of reports can be explained better by the desire of politicians to avoid blame, rather than the fire alarm theory: incidents only seldom lead to evaluations being undertaken – and not for a lack of incidents, as there were multiple incidents during the research period. However, when an incident occurs, existing reports are included in the parliamentary debate (see the example of the Police Academy in Table 7), whereas most reports usually remain undiscussed.

The implementation of the Charter Law did lead to an increase in the number of evaluation studies (see Figure 1). Reports are also made public more often, but they are not used very often in political debates. This lack of interest is in part due to the administrative processual character (as predicted by Busuioc and Lodge, 2017) but could also be contributed to the methodological quality of the reports. The predominantly qualitative research makes it difficult to draw ‘hard’ conclusions – although this could also be done on purpose, to avoid blame or reputation damage (or scapegoat the researchers). The latter interpretation would fit with the negativity bias that Hood (2011) mentions. But we should not forget that measurement of policy outcomes is a difficult exercise, and chances of credit claiming are small (Hood, 2007, 2011), which could be seen as a more neutral explanation. The lack of a format or clear instructions on how to carry out a ZBO-evaluation does not help to overcome such measurement problems – which again could be done on purpose. And similarly, the absence of a regulatory authority to oversee ZBO-evaluations being carried out makes it easy to not comply with the legal obligation. For example, in the United Kingdom, a dedicated unit has been established within the Cabinet Office to oversee policies regarding the Non-Departmental Public Bodies (comparable to ZBOs) and the triannual reviews that are carried out quite diligently.

All in all, ZBO-evaluations have a low priority for Dutch politicians (Busuioc and Lodge, 2017; Dudley, 1994). The ZBOs themselves attach more importance to evaluating their performance, as reflected in the establishment of a peer review procedure. This could be interpreted as an attempt to increase the ZBOs’ reputation (Busuioc and Lodge, 2017; Schillemans and Busuioc, 2015) or ‘spin’ the debate (a presentational strategy; Hood, 2011). Reports that have come about through peer review are however not presented to parliament by the parent minister as a formal evaluation; they are made publicly available by the ZBOs and have thus only on occasion reached parliament (POC, 2012).

Finally, this study has a number of limitations that need to be mentioned. First of all, the findings are restricted to the Dutch case of ZBOs. Whether they also apply to other countries can only be found out by replicating this study. The Dutch politico-administrative context is characterized by consensus, with political parties cooperating in changing coalitions over time. Not only would this reduce the motivation to seek out political conflict, for example over incidents related to ZBOs and policy outcomes (cf. Novak, 2013), it also creates a time-lag in instigating or reporting ZBO-evaluations; the average cabinet lasts four years while ZBO-evaluations cover a period of five years. Replication of this research in countries with a majoritarian system could lead to different results (McCubbins and Schwartz, 1984; Weaver, 1986). Second, the data did not allow for advanced statistical analysis. A more in-depth interpretation of the data, through qualitative research like interviews and case studies, could give more insight. For example, additional data could be used like media reporting to determine whether incidents involving a particular ZBO have played a role in setting up an evaluation or in the report thereof.

Supplemental Material

sj-pdf-1-ppa-10.1177_09520767211022490 - Supplemental material for Blame avoidance, scapegoats and spin: Why Dutch politicians won’t evaluate ZBO-outcomes

Supplemental material, sj-pdf-1-ppa-10.1177_09520767211022490 for Blame avoidance, scapegoats and spin: Why Dutch politicians won’t evaluate ZBO-outcomes by Sandra van Thiel in Public Policy and Administration

Footnotes

Acknowledgement

The author would like to thank Joep van der Spoel, Peter van Goch, Martijn Dresmé, Stefan Soetekouw, Niels van Beurden and Harm Bassa for their assistance in the collection and coding of the data.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Sandra van Thiel

Notes

References

Askim

(2007) How do politicians use performance information? An analysis of the Norwegian local government experience. International Review of Administrative Sciences 73(3): 453–472.

Busuioc

Lodge

(2017) Reputation and accountability relationships: Managing accountability expectations through reputation. Public Administration Review 77(1): 91–100.

Cohn

(1997) Creating crises and avoiding blame: The politics of public service reform and the new public management in Great Britain and the United States. Administration & Society 29(5): 584–616.

Dubnick

Frederickson

(2010) Accountable agents: Federal performance measurement and third-party government. Journal of Public Administration Research and Theory 20(Supplement 1): i143–i159.

Dudley

(1994) The next steps agencies, political salience and the arm's‐length principle: Barbara Castle at the ministry of transport 1965‐68. Public Administration 72(2): 219–240.

Flinders

(2008) Delegated Governance and the British State: Walking without Order. Oxford: Oxford University Press.

Hood

(2002) The risk game and the blame game. Government and Opposition 37(1): 15–37.

Hood

(2007) What happens when transparency meets blame avoidance? Public Management Review 9(2): 191–210.

Hood

(2011) The Blame Game: Spin , Bureaucracy, and Self-Preservation in Government. Princeton: Princeton University Press.

10.

Kickert

WJM

(2001) Public management of hybrid organizations: Governance of quasi-autonomous quangos. International Public Management Journal 4(2): 135–150.

11.

Majone

(2001) Two logics of delegation: Agency and fiduciary relations in EU governance. European Union Politics 2(1): 103–122.

12.

McCubbins

Schwartz

(1984) Congressional oversight overlooked: Police patrols versus fire alarms. American Journal of Political Science 28(1): 165–179.

13.

Nielsen PA and Baekgaard M (2015) Performance information, blame avoidance, and politicians’ attitudes to spending and reform: Evidence from an experiment. Journal of Public Administration Research and Theory 25(2): 545–569.

14.

Novak

(2013) The silence of ministers. JCMS: Journal of Common Market Studies 51(6): 1091–1107.

15.

OECD (2002). Distributed Public Governance: Agencies, Authorities and Other Government Bodies. Paris, OECD.

16.

Overman

(2016) Great expectations of public service delegation: A systematic review. Public Management Review 18(8): 1238–1262.

17.

POC: Parlementaire Onderzoekscommissie (2012) Verbinding verbroken? Onderzoek naar de parlementaire besluitvorming over de privatisering en verzelfstandiging van overheidsdiensten. Netherlands: Eerste Kamer, vergaderjaar 2012–2013. B & C.

18.

Pollitt

(2005) Ministries and agencies: Steering, meddling, neglect and dependency. In: Painter

Pierre

(eds) Challenges to State Capacity: Global Trends and Comparative Perspectives. Basingstoke: Palgrave MacMillan, pp.112–136.

19.

Pollitt

Dan

(2013) Searching for impacts in performance-oriented management reform: A review of the European literature. Public Performance & Management Review 37(1): 7–32.

20.

Pollitt

Caulfield

Smullen

, et al. (2004) How Governments Do Things through Semi-Autonomous Organizations. Basingstoke: Palgrave MacMillan.

21.

Schillemans

(2008) Accountability in the shadow of hierarchy: The horizontal accountability of agencies. Public Organization Review 8(2): 175–194.

22.

Schillemans

Busuioc

(2015) Predicting public sector accountability: From agency drift to forum drift. Journal of Public Administration Research and Theory 25(1): 191–215.

23.

Talbot

(2004) The agency idea: Sometimes old, sometimes new, sometimes borrowed, sometimes untrue. In: Pollitt

C and

Talbot

(eds) Unbundled Government: A Critical Analysis of the Global Trend to Agencies, Quangos and Contractualisation. London: Routledge, pp.3–21.

24.

Ter Bogt

(2004) Politicians in search of performance information? Survey research on Dutch aldermen’s use of performance information. Financial Accountability and Management 20(3): 221–252.

25.

‘t Hart

Wille

(2006) Ministers and top officials in the Dutch core executive: Living together, growing apart? Public Administration 84(1): 121–146.

26.

Tonkiss

van Thiel

Verhoest

, et al. (2015). How are arm’s length relationships instrumentalised in Europe: A three-country comparison. In: Paper presented at MPSA conference, Chicago, USA, 16–19 April 2015.

27.

Van Thiel

(2001) Quangos: Trends, Causes and Consequences. Aldershot: Ashgate Publishing

28.

Van Thiel

(2012) Comparing agencies across countries. In: Verhoest

Van Thiel

Bouckaert

, et al. (eds) Government Agencies: Practices and Lessons from 30 Countries. Basingstoke, Palgrave MacMillan, pp.18–26.

29.

Van Thiel

(2019) The politics of quangocracy. The British Journal of Politics and International Relations 21(3): 522–529.

30.

Van Thiel

Verheij

(2017) Het aantal zelfstandige bestuursorganen in Nederland 1993–2013. [The number of ZBOs in The Netherlands, 1993–2013]. Beleid en Maatschappij 44(1): 27–41.

31.

Van Thiel

Yesilkagit

(2014) Does task matter? The effect of task on the establishment, autonomy and functioning of semi-autonomous agencies. International Review of Administrative Sciences 80(2): 318–341.

32.

Verhoest

Van Thiel

Bouckaert

, et al. (eds) (2012a) Government Agencies: Practices and Lessons from 30 Countries. Basingstoke: Palgrave MacMillan.

33.

Verhoest

Van Thiel

Hajnal

, et al. (eds) (2012b) The politics of agencies. Special Issue of Nispacee Journal of Public Administration and Policy, V(1): 11–14.

34.

Weaver

(1986) The politics of blame avoidance. Journal of Public Policy 6(4): 371–398.

35.

Yesilkagit

(2014) There’s more than life and death: A competing risks approach to agency duration. Paper prepared for the 2014 SOC conference ‘what makes agencies tick? Hebrew University of Jerusalem, 5–7 January 2014.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB