Rethinking democratic evaluation for a polarised and mediatised society

Abstract

This article discusses how democratic evaluation can manage threats to democracy, democratic renewal, and the mediatisation of public policy and governance. It considers the readiness of five democratic evaluation orientations to deal with current threats and discusses how to develop them. It demonstrates that democratic evaluation is poorly prepared to manage current threats to democracy or the mediatisation of public policy. Progressive evaluation is the only approach offering some new keys to addressing certain current threats and challenges. The other orientations have some capacity to manage threats to democracy and support democratic renewal, but need further development. The article suggests that democratic evaluation could be a constructive tool for maintaining and developing democracy in an increasingly polarised and mediatised society if evaluators gain knowledge of threats to democracy, democratic transition, and democratic renewal and, informed by mediatisation and democracy research, develop the necessary awareness and competence to deal with these challenges.

Keywords

democratic evaluation democratic renewal democratic threats governance mediatisation polarization

Introduction

There is growing worry about increasing polarisation and threats to democracy, but few have discussed how such threats can be dealt with in evaluation. Picciotto (2015) is an exception, having suggested that progressive evaluation can deal with the new evolving context of evaluation. Since he presented his approach it is claimed that the situation has become worse in many respects. It is an open question as to whether we are experiencing a time of democratic decay or renewal. Obviously, the current conditions for democratic evaluation are radically different from when MacDonald (1976, 1978) first developed the democratic evaluation concept in the 1970s. Today democracy is under pressure, and the status and credibility of scientific knowledge and public media have declined among some citizens, organisations, and politicians. Supposedly, this goes for evaluation as well. Some observers have claimed that the ‘open society’ and established democratic institutions have weakened (Freedom House, 2018; Keane, 2008). Others have discussed the potential for democratic renewal and point to new forms of political engagement, rising youth participation, social media, and deliberative polls and ‘mini-publics’ (e.g. Ceron and Negri, 2016; Friess and Eilders, 2015; Liston et al., 2013; Schmitter, 2015). This article discusses how progressive evaluation and four other democratic evaluation orientations can help manage current threats and challenges, helping maintain and develop democracy.

The article also draws attention to the mediatisation of public policy and governance, as this phenomenon has created new challenges for democracy and evaluation. Mediatisation refers to the process by which institutions and organisations (Pallas et al., 2014) as well as public policy and governance (Helms, 2012; Peters, 2016) adapt to the demands and logics that media apply when scrutinising the political world (Asp, 1986; Peters, 2016; Schulz, 2004). It is recognised that both old and new media as well as mass communication through the internet have affected politicians and citizens and changed the conditions for public policy and democracy (Keane, 2008; Knox, 2013; Strömbäck, 2008). It is disputed, however, whether new channels and forums and the mediatisation of politics and governance have undermined democracy or provided opportunities for democratic renewal (Friess and Eilders, 2015; Keane, 2008; Knox, 2013; Liston et al., 2013). We must consider how to address this development; for example, how to conceive and evaluate a mediatised policy and how to grasp what mediatisation implies for democracy and democratic accountability (Behn, 2001).

Furthermore, there is little discussion and a lack of knowledge and practical wisdom in dealing with the media in evaluation (House, 2015). In practice, government agencies and organisations conducting evaluation have some kind of media strategy, but what do they entail and what do they imply for democracy? How evaluators and commissioners respond when evaluations are used as political ammunition is not a new question but one that merits further discussion when evaluation is used in various media to increase polarisation, for example.

This article discusses challenges to democratic evaluation against the backdrop of threats to democracy, ongoing democratic renewal, and the mediatisation of public policy and governance. It seeks answers to three research questions:

how does and can democratic evaluation help manage threats to democracy and support democratic renewal?

how does and can democratic evaluation deal with the mediatisation of public policy and governance? and

does democratic evaluation have media strategies? If so, what do they entail?

It is recognized that the conditions for democratic evaluation varies and changes; evaluation is conducted in different political systems and contexts, with varying social trust and trust in established institutions and governments. The article provides a framework, based on democratic theories, with which to consider democratic evaluation in view of current challenges in Sweden and other democratic countries that perceive threats to established institutions and support, or perceive a need for, democratic renewal. Besides progressive evaluation, it looks into the readiness of four other democratic evaluation orientations to help manage these challenges and into how these orientations can be developed. It is beyond the scope of this article to discuss in detail how these orientations can be developed; it only points at ways to develop them. I use the term ‘orientation’ to indicate that more than one democratic evaluation approach can exist in an orientation. The article’s findings can be used by evaluators, commissioners and others concerned in deliberating on how evaluation can be developed to support democracy today, addressing the mediatisation of policy and what this means for democracy, and developing media strategies.

Next, the article presents a vignette that illustrates how threats to democracy can arise and the mediatisation of politics can occur today. After that, it examines two ways to conceive recent changes in democracy. The theoretical section summarises research into the mediatisation of politics and governance and presents key features of democratic theory and five democratic evaluation orientations that are further discussed in the article. The following section compares and discusses the readiness of these orientations to help manage the identified challenges. Finally, the article draws conclusions and discusses how to develop democratic evaluation in light of the findings and identified challenges.

Vignette

This vignette illustrates the growing polarisation and mediatisation of politics and evaluation, in this case the use of official crime reports. Here are some key events in the story:

Mr Åkesson, party chair of the Sweden Democrats, and Mr Karlsson, the party’s group leader in Parliament, wrote an article in The Wall Street Journal, 22 February 2017, referring to an official crime monitoring report from 1996 and a newer report from Brå, the Swedish National Council for Crime Prevention. The article was headline news and prompted international and national reactions. They argued that the government, the Council, and the traditional media were concealing certain facts, including that immigrants were perpetrators more often than ethnic Swedes. Here is an extract from the article:

When President Trump last week raised Sweden’s problematic experience with open-door immigration, sceptics were quick to dismiss his claims. Two days later an immigrant suburb of Stockholm was racked by another riot. No one was seriously injured, though the crowd burned cars and hurled stones at police officers.

Mr Trump did not exaggerate Sweden’s current problems. If anything, he understated them. Sweden took in about 275,000 asylum-seekers from 2014–16 – more per capita than any other European country. Eighty per cent of those who came in 2015 lacked passports and identification, but a majority come from Muslim nations. Islam has become Sweden’s second-largest religion. In Malmö, our third-largest city, Mohamed is the most common name for baby boys.

Riots and social unrest have become a part of everyday life. Police officers, firefighters and ambulance personnel are regularly attacked. Serious riots in 2013, involving many suburbs with large immigrant populations, lasted for almost a week. Gang violence is booming.

The number of sex crimes nearly doubled from 2014 to 2015, according to surveys by the Swedish government body for crime statistics [i.e. Brå]. One-third of Swedish women report that they no longer feel secure in their own neighbourhoods, and 12% say they don’t feel safe going out alone after dark. A 1996 report from the same government body found that immigrant men were far likelier to commit rape than Swedish men. Last year our party asked the minister of justice to conduct a new report on crime and immigration, and he replied: ‘In light of previous studies, I do not see that a further report on recorded crime and individuals’ origins would add knowledge with the potential to improve the Swedish society.’

The article triggered polarised discussion of what is happening in Sweden, changes in crime prevalence and incidence, and who is committing crime, sex crimes in particular. Swedish public television and radio, key newspapers, and other media – and the minister of justice all questioned the situation that the Sweden Democrats described using selected crime statistics. The Brå reports include discussion of the difficulties of describing trends based on questionnaires and statistics and of how to interpret measured changes (e.g. related to sources of error and willingness to report sexual harassment), but these factors were not considered by the Sweden Democrats. The authors of the Brå reports were interviewed by various media representatives and explained that they had not analysed ethnic background or whether immigrant men were overrepresented as perpetrators because they had not been commissioned to do that. However, nothing had stopped them from undertaking such analysis as part of their general commission. Successively, most political parties have admitted some of the problems addressed in the article, but framed the problems differently without blaming the immigrants.

The vignette illustrates how the polarisation and mediatisation of politics and evaluation can occur and interact, and how a party with undemocratic roots can use official reports to advance a political agenda indicating a democracy in decline. However, this is not the only way to portray today’s democracy.

Democracy in decline or transition?

Whether democracy is described as in decline or in transition affects how we conceive current threats and challenges to democratic evaluation. The dominant narrative of democracy in the media today is that of democracy in decline. Growing polarisation is a response to an uneven distribution of wealth, jobs, housing, and education and to a feeling of alienation in some groups. As the gap between the people and the elite grows, people’s trust in the political system and established institutions diminishes. When power is turned over to international and transnational institutions and multinational enterprises, national democracy and sovereignty weaken (Behn, 2001; Schmitter, 2015). Picciotto (2015) has described what is causing the emergence of this new context of evaluation: internationalisation and powerful vested interests are threats that explain current conditions. Freedom House (2018) recognises continuing democratic setbacks in a number of western countries, the United States in particular. This decline is reflected by the growing attraction of ‘modern authoritarianism’, defined by Freedom House as, for example, ‘an illusion of pluralism that masks state control over key political institutions’, ‘state or oligarchic control over information on certain political subjects and key sectors of the media, which are otherwise pluralistic’, and ‘politically obedient courts’ (Freedom House, 2017: 6). However, some observers question the narrative of decline and suggest that we should instead look to what current crises of democracy have enabled.

Diamond (2015: 142), for example, has questioned Freedom House’s description of decline, suggesting that the halt in expansion of freedom and democracy in the world since 2006 can be interpreted as marking a period of equilibrium. Schmitter (2015: 35) went one step further, claiming that Freedom House’s annual reports have misleadingly portrayed a general decline in democracy over the last decade: ‘democracy is not in decline but it is in crisis and in the process of transition from one type to another – although it is not at all clear what the new type (or types) will be or whether any new type will be an improvement over existing practices’. He used the term ‘post-liberal’ to indicate that democracy is in transition to something qualitatively different from what we now know. Schmitter (2015) and others (some cited above) pay more attention to democratic renewal and new channels of communication and citizen influence. Whether democracy is perceived as in decline or transition is also a question of how we measure democracy, a matter on which there is no consensus (see Diamond, 2015: 142–3).

Keane (2008) has provided a different narrative of democracy in transition. He described how various power-scrutinising institutions and mechanisms have created a fundamental change in representative democracy since the Second World War. These institutions and mechanisms constitute a new form of democracy, monitory democracy, which is the most institutionally complex form of democracy so far and is characterised ‘by the multiplication and dispersal of many different power monitoring and power-contesting mechanisms, both within the “domestic” fields of government and civil society and beyond’ (Keane, 2008: 3). Some scrutinising institutions focus on ‘citizens’ inputs to government or civil society; others are preoccupied with monitoring and contesting policy throughputs; still others concentrate on scrutinising policy outputs produced by governmental or non-governmental organisations’ (Keane, 2008: 11). Evaluation is one such scrutinising institution, and although its purpose can be to support democracy, it may, following Keane, help undermine representative democracy. Kean also recognizes that well-functioning monitory institutions are needed, but today they are targets for some democratic elected leaders who have begun dismantling them.

When rethinking how democratic evaluation can respond to current challenges, the evaluator should consider which narrative of changing democracy to assume. Viewed from a liberal or elite-democratic evaluation perspective (Hanberger, 2006), the notion of decline fits well and the challenge is then to protect current democratic institutions. In contrast, from a participatory or deliberative democratic evaluation perspective (Friess and Eilders, 2015; Hanberger, 2006; Schmitter, 2015), democracy in transition is more suitable. The challenge then expands to one of supporting democratic renewal, for example, through deliberation in new digital settings.

Theoretical considerations

Mediatisation of public policy and governance

Asp (1986: 359) defines a mediatised political system as one that is ‘strongly influenced by and adapted to the requirements of the mass media in their coverage of the political world’ (translated from Swedish). The concept should be distinguished from when public policy is mediated (Strömbäck, 2008); that is, when people get information about public policy and governance through the media instead of from inter-personnel communication or their own experience. Strömbäck (2008) developed a process-oriented conceptualisation of the mediatisation of politics comprising four phases of mediatisation. The first phase is when politics has become mediated and the media constitute the dominant source of information and channel of communication. This phase is a prerequisite for the following phases. In the second, third, and fourth phases, the media are increasingly important as the dominant source and channel of communication, and the media and politics are increasingly governed according to a media logic. Strömbäck discusses the mediatisation of politics as encompassing all political actors, whereas this article focuses on the mediatisation of public policy and governance.

Mediatisation research recognises the various ways the media influence politics and governance, noting, for example, that the media determine and/or influence what issues parties and people think are most important and that governance has adapted to media demands, but most theories do not consider the interaction and mutual dependencies between media and politics (Schulz, 2004; Strömbäck, 2008). The above vignette illustrates what theory fails to reflect. Mediatisation research has grown as new media open up new ways for politicians and citizens to communicate and for citizens to influence decision makers. Knox (2013) claimed that the creation of new channels to raise issues can increase political legitimacy, but he also recognised problems in accommodating large flows of feedback. Social media have changed the power relationship between old news media and decision makers, providing the latter with opportunities to circumvent the gatekeeping role of old media and communicate directly with citizens (Djerf-Pierre and Pierre, 2016: 59). Before internet-based communication existed, the mediatisation of governance took the form of adaptation to traditional mass media, whereas today it also includes adaptation to the demands of new internet-based mass media and social media.

The mediatisation of national governance has so far been the main focus of research; the mediatisation of local governance is less developed but is gaining ground (Larsson, 2013). A Swedish survey of social media activity among local decision makers, 1989–2010, demonstrated that local political and administrative leaders make use of social media, but not extensively; it also demonstrated that they use social media when subjected to social media scrutiny and that social media have not replaced the old news media (Djerf-Pierre and Pierre, 2016). The authors noted that local decision makers active in social media also tend to be active in traditional media, thus using both old and new channels of communication. This study did not reveal whether local governance has adapted to the media logic.

Keane (2008: 26) has recognised that governments adapt to the media and to other scrutinising organisations and that internet-based mass communication shapes today’s democracy: monitory democracy ‘is tied closely to the growth of media-saturated societies – societies in which all institutions operate within fields of media’. Moreover,

[w]ithin the age of communicative abundance, it is as if all mediated opinions are flung constantly into a swirling cyclotron, a container filled with high-density messages. Whether a message makes its way through the cyclotron to its intended receivers, and whether they accept its intended meaning, without contradiction by others’ messages and counter-messages, is never entirely predictable. Even the most powerful groups and individuals have to accept the contingency of outcomes. The price to be paid for a media message is not just money and influence. It is acceptance of the fact that the world of mediated communication is multi-semantic and semantically slippery, full of overlapping, sometimes colliding, messages whose meaning is always in the last instance unpredictably determined by the receivers and interpreters of messages. Thanks to the overlap of communicative abundance and power monitoring institutions, the powerful become permanently vulnerable to the power of the powerless. (Keane, 2008: 33–4)

The vignette illustrates that messages from government body reports (evaluations) can be used as political ammunition, for example, being disseminated by the media to support a party’s political agenda, and that the messages are ‘slippery’ and can eventually change the political discourse. It also illustrates the need for evaluators to be better prepared to respond to actors who use evaluation to increase polarisation and promote particular political agendas. With reference to Keane (2008), one may ask what the value is of disseminating messages from evaluations when they can end up in a ‘swirling cyclotron’. The cited research implies that we must recognise traditional news media, new social media, and internet-based mass communication as central components of governance and that commissioners and evaluators must consider this when demanding, conducting, and disseminating evaluations.

Mediatisation of evaluation

The cited literature presents no discussion of, or research into, the mediatisation of evaluation. To facilitate discussion of this phenomenon, I suggest defining the mediatisation of evaluation as the process and outcome of an evaluation’s adaptation to demands that the media or mediatised policies or programmes have applied, or are perceived to have applied, to evaluation.

This definition reflects that adaptation to the media logic can be either voluntary or imposed. For example, an evaluator could adopt an evaluation to meet media demands so that the media will publish an article about the evaluation findings, or a commissioner could ask the evaluator to emphasise programme success for dissemination on its website and in social media, to legitimise the programme. The message thus mediatised may not reflect the evaluation’s intended message – perhaps the evaluation arrived at a multifaceted assessment of the programme, a message not easily communicated.

Democratic theories

As democratic evaluation is intended to support existing democracy or develop it in new directions, there is a need to clarify what notion of democracy it serves. When rethinking democratic evaluation for a polarised and mediatised society, many democratic theories should be considered (Birch, 2001; Cunningham, 2002; Diamond, 2015; Dryzek, 2000; Fishkin, 2014; Gutmann and Thompson, 2004; Habermas, 1996; Schmitter, 2015; Tomasi, 2012). Here, attention is paid to four democratic theories on which evaluation can be based. I have discussed three democratic theories and related democratic evaluation orientations in earlier work (Hanberger, 2001, 2006); for the purpose of this article, they are briefly described below.

According to the first theory, that of elitist democracy, political elites compete for power in open societies (Schumpeter, 1942). This theory, sometimes referred to as the liberal or Lockean view (Habermas, 1996), assumes that citizens can control their government by choosing among competing elites. Ordinary citizens are encouraged to participate every three or four years in elections. Citizens are not given a direct role in the policy process, and democratisation implies improving the elite’s representation of the people. Decision making is a task for those in power. The core idea here is that of representative democracy, i.e. a democracy for the people.¹

The participatory theory of democracy assumes that people’s participation is the most important quality of a democracy and that people exercise their power when they participate. Accordingly, apathy and non-participation are seen as major threats to democracy. Moreover, participation is assumed to foster democratic citizens. Participation is assumed to help in creating identity, to encourage a desire to participate further in common affairs, to develop responsibility, and so on. According to this view, it is only through participation that the idea of democracy can be realised (Pateman, 1970). There are various notions of participatory democracy; the participatory notion discussed here is a form of democracy by the people, associated with a self-governing society in which citizens empower themselves or are delegated freedom of choice to decide what is feasible for them.

The discursive theory of democracy refers to democracy with the people. According to this theory, democracy can only be realised through discussions among free and equal citizens. This type of democracy, sometimes called ‘deliberative democracy’ (Dryzek, 2000; Gutmann and Thompson, 2004), is based on a common commitment to a mode of reasoning about matters of public policy. The discourse is open to those affected by collective decisions and/or to their representatives. According to Gutmann and Thompson (2004: 7), it is ‘a form of government in which free and equal citizens (and their representatives) justify decisions in a process in which they give one another reasons that are mutually acceptable and generally accessible, with the aim of reaching conclusions that are binding in the present on all citizens but open to challenge in the future’. Democracy is realised in situations in which policy makers, citizens, and other stakeholders have access to basic information, and are given enough time and trust to participate in practical reasoning to resolve social problems.

Tomasi’s (2012) market democracy, the fourth theory acknowledged, stresses the significance of private economic liberty in democracy. Market democracy combines ideas from classical liberals (Hayek) and ‘high liberals’ (Rawls) and is based on three theses:

1) economic liberty is among the basic constitutional rights and liberties held by liberal citizens, 2) social justice, on some interpretation to be specified, is the ultimate standard of social evaluation, and 3) as a matter of ideal theoretic analysis, the institutions of market democracy realize the conception of social justice specified in thesis 2) even while affirming the priority of economic liberty set out in 1). (Tomasi, 2012: 52)

Although there is a potential conflict between economic liberty and social justice, this is not how Tomasi (2012: 50) sees it: ‘The distributional requirements of social justice are to be pursued mainly through the forces of spontaneous economic order.’ In Tomasi’s market democracy, the benefits produced by established institutions should be enjoyed by all citizens, including the least fortunate. Market democracy is a democracy for and by the people.

Real democracies can incorporate features of all four types, but can resemble one more than another; the same holds true for democratic evaluation; that is, it can be based on one or several democratic theories.

Democratic evaluation orientations

The democratic evaluation approach developed 40 years ago by MacDonald (1976) has long shaped the notion of democratic evaluation in the evaluation society. His pluralistic notion of democracy is not clearly aligned with any one of the above democratic theories, though it has most in common with discursive democracy. Since MacDonald’s time, more democratic evaluation orientations have been developed by evaluation researchers (see below) and researchers in other fields (Fishkin, 2014; Gastil et al., 2013; Mathur and Skelcher, 2007). This article discusses democratic evaluation approaches presented in the evaluation literature. Table 1 summarises five evaluation orientations and their bases in democratic theory, evaluators’ roles, assumed notions of the citizen, and support for democratic institutions.

Table 1.

Characteristics of five democratic evaluation orientations.

	Elitist	Participatory	Discursive	Progressive	Market
Democratic theory	For the people (Schumpeter)	By the people (Pateman)	With the people (Habermas)	With the people (not specified)	For and by the people (Tomasi)
Roles	Expert	Advocate, facilitator, coach	Advocate, counsellor, mediator	Expert, advocate	Expert, advocate
Notion of the citizen	Voter, consumer	Active citizen	Active citizen	Active citizen	Active citizen, consumer
Supported democratic institutions	Established	Established and new	Established and new	Established and new	Established

Adapted from Hanberger (2006), Picciotto (2015), and Tomasi (2012). Active citizens act as voters, consumers, and actors in policy making or implementation.

Elitist democratic evaluation (EDE), the first orientation, is based on elitist democratic theory (Schumpeter, 1942). The main role of the evaluator in this orientation is to act as an expert to serve the decision makers’ information and knowledge needs and provide feedback to enhance their learning and accountability. On behalf of the citizens, the decision makers (i.e. the elite) want to know whether or not an intervention works and whether its goals have been achieved. Different elites can be supported, that is, a political and/or administrative elite can be informed by and gain or lose support as a result of an EDE (see Farazmand, 1999).

The participatory democratic evaluation (PDE) is based on participatory democratic theory (Pateman, 1970) and assumptions of self-governance. The main functions of PDE, as understood here, are self-learning, empowerment, and self-determination. Evaluations focus primarily on whether participating and affected citizens or clients are included and empowered by the policy or programme, as well as by the evaluation process. The evaluation is designed by the clients/citizens with assistance from an evaluator, whose role is to facilitate and coach self-reflection, empowerment, and, ultimately, self-determination. The evaluation pays attention to preconditions and incremental progress made in the programme or the client’s own agenda.

The discursive democratic evaluation orientation (DDE) is based on deliberative or discursive democratic theory (Dryzek, 2000; Gutmann and Thompson, 2004; Habermas, 1996). There are different approaches within this orientation (Abma et al., 2016; House and Howe, 1999; MacDonald, 1976). DDE is first of all intended for collective learning, i.e. to meet the main stakeholders’ practical knowledge needs, to justify collective action and facilitate public debates that enhance collective learning. DDE seeks to include all legitimate stakeholders in the evaluation process and to assess the programme according to criteria considered relevant by all legitimate stakeholders. The evaluator acts as a mediator and counsellor by bringing arguments and analysis into the learning processes. The general public benefits from this type of evaluation through accessible information and insights that help them become better informed about the performance of policies and programmes, and by facilitated citizen participation in public debates. The citizen is viewed as an active agent who votes and participates in policy making and implementation (Hanberger, 2001).

Picciotto (2015) has described progressive evaluation (PROGE) as an update of MacDonald’s (1976, 1978) and House and Howe’s (1999) democratic evaluation approaches without reference to democratic theory:

Progressive evaluation would be engaged in the policy-making process as the traditional MacDonald model prescribes and it would draw on the procedural neutrality, value driven principles and ethical canons of House/Howe’s deliberative democratic evaluation model. But it would break free of the neutral brokerage role associated with both of these models. Instead it would assert professional autonomy and make full use of social networking and the new information technologies to amplify the voices of the poor and involve them in the evaluative process. It would seek independent funding and rely on external sponsors to avoid evaluation capture. (Picciotto, 2015: 163)

The PROGE approach is intended to protect the evaluator’s professional autonomy and independence, speak truth to power, and form alliances with advocacy groups concerned with social justice to protect the weak and disadvantaged. It also seeks inspiration from social-justice–oriented evaluations. The evaluator combines acting as an independent evaluator and acting as an advocate promoting social justice. The citizen is conceived as an active agent, particularly in advocacy groups.

The market democratic evaluation (MDE), which I have formulated from Tomasi’s (2012) democratic theory, is intended to support market democracy and protect citizens’ economic rights and liberties. It claims social justice to be the basic standard of social evaluation, though it is unclear what notion of social justice Tomasi (2012) has in mind (see Tomasi, 2012: 77). Tomasi does not discuss what roles the evaluator should have, but I assume that the evaluator acts as an expert to provide accounts that can be used to maintain a well-functioning market democracy, ensure social justice, and facilitate consumer power. Tomasi assumes that active citizens, described as ‘self-authors’ of their lives, are empowered and capable people disposed to governing themselves and wielding their consumer and voting power.

Method

This article applies the cited research in analysing these five democratic evaluation orientations in the following way:

First, it examines whether the orientation incorporates any commitment to protect democracy and manage polarisation and, if so, what this entails. It also interprets how the orientation generally helps maintain and develop democracy. The analysis is based on the democratic evaluation orientations as presented here.

Second, it examines whether the orientation has guiding principles, strategies, or policies for dealing with the mediatisation of public policy and governance and, if so, what characterises these. It also interprets whether and how the approach copes with mediatisation. The analysis applies the concept of mediatisation and the four-dimensional approach to mediatisation developed by Strömbäck (2008).

Third, it analyses whether the orientation has a media strategy or policy for evaluation and, if so, what it comprises, for example, for how to communicate findings with the media. It applies the definition of the mediatisation of evaluation presented here, and considers whether the evaluation orientation is inclined to present findings according to a media logic.

Managing mediatisation and threats to democracy

First, the EDE evaluator assumes general trust and stability in democratic institutions and is thus ill prepared to manage threats to democracy, for example, as described in the vignette. Still, the EDE can help maintain established democratic institutions by providing an independent evaluation that gives decision makers the knowledge needed in order to improve democratic governance. If needed, the EDE evaluator informs the public about political pressure on the evaluation, for example, whether demands are being made for changes in conclusions or whether powerful actors are trying to influence the evaluation. The responsibility also includes correcting any misrepresentation of the evaluation in the interest of the general public. However, an EDE evaluator is not well prepared to discharge these responsibilities in a polarised society. For example, how would she act if the evaluation were used by a government or pressure group to justify restricting freedom of speech or other democratic rights? Second, the EDE evaluator is not inclined to account for the mediatisation of a public policy under scrutiny unless she is commissioned to do so. Third, this orientation lacks any media strategy that can guide the evaluator. However, if the media misrepresent the evaluation, the EDE evaluator should be inclined to correct this on behalf of the citizens. When the evaluator presents the evaluation to the commissioner, there is no indication of the mediatisation of the evaluation unless the commissioner demands adjustments.

First, the PDE evaluator is somewhat prepared to manage polarisation and threats to democracy while protecting the evaluands, often weak groups, from external pressure. MacTaggert (1991: 20) stressed the need for democratic evaluators to ‘protect people from unwarranted exposure’. Acting as a guard and coach is the way the PDE evaluator can prevent unwanted exposure and empower the evaluands. There is, however, a risk that she may become a target for organisations questioning, for example, why a vulnerable group, such as unaccompanied minors, is empowered instead of retired ethnic Swedes/Germans. Like the EDE evaluator, the PDE is poorly prepared to perform the assigned roles in a polarised society. For example, how can the evaluator respond if subject to pressure and criticised for serving a disadvantaged group? The evaluator is prepared to support democratic renewal intended to increase participation and self-governance. Second, this orientation does not incorporate preparedness to deal with the mediatisation of a policy or programme under scrutiny. At the time when MacTaggert discussed this approach, the mediatisation of governance was not an issue. Third, this orientation also lacks a media strategy, but the evaluator would be inclined to protect the evaluand from unwanted media exposure if needed.

The DDE evaluator is somewhat prepared to help manage threats to democracy and polarisation as part of her general responsibilities. The DDE evaluator will maintain and support established democratic institutions in a way similar to that of the EDE evaluator and will also support democratic renewal, for example, by evaluating new democratic institutions for deliberation. The DDE evaluator integrates decision makers’ evaluation questions together with key stakeholders’ questions, in contrast to the EDE evaluator who serves the elite’s needs and the PDE evaluator who supports only one group at time. The DDE evaluator acts mainly as a mediator and counsellor to promote collective learning. Polarisation could possibly be dampened while the DDE contributes to better-informed discussion by mediating between different arguments and stakes. However, this is never an easy task and may not work if conflicting interests participate in the evaluation. Second, this orientation is not prepared to address the mediatisation of a policy or programme except if the commission specifically includes this task. Third, a media strategy remains to be developed for DDE-evaluation; in this, the evaluator’s negotiation and communication skills can be useful in managing different media.

A PROGE evaluator will manage threats to liberal democracy in a way somewhat similar to that of the DDE evaluator. However, the PROGE evaluator can act from a stronger position as she is not dependent on the commissioner’s demands and funding. When a PROGE evaluator acts as an independent evaluator and forms alliances with advocacy groups to promote social justice, this supports participatory or discursive democracy in civil society. According to Picciotto (2015: 161) there is a need for ‘speaking truth to power in situations where democracy is absent, social inequities are rampant and/or governance has been captured’. If the evaluator builds alliances with one advocacy group, other groups may question her objectivity, which could undermine the evaluator’s professional autonomy and authority. It is difficult to see how the evaluator can dampen polarisation through alliances unless the advocacy group has extensive public support and even then such dampening would not be guaranteed. Second, there are no indications as to how this approach will deal with the mediatisation of a policy being evaluated. Third, Picciotto (2015) suggests that this approach should extensively use new information technologies and social networking and involve certain advocacy groups in the evaluation process, but he gives no clues as to how the PROGE evaluator should manage the media.

In the interest of the ordinary citizen, the MDE evaluator will be prepared to reveal whether a policy or programme harms the market economy and undermine the basic economic rights and liberties of citizens. Tomasi (2012) does not discuss how to manage the threats to democracy and the growing polarisation addressed here. The evaluator’s responsibility is mainly to expose dysfunctions in the market economy and redistribution chain, and offer solutions to resolve them. Second, Tomasi does not discuss market democracy in light of the mediatisation of public policy and governance. Third, the mediatisation of evaluation is overlooked in Tomasi’s discussion of social evaluation.

The EDE, PDE, and DDE orientations were developed to support democracy before today’s challenges were known, which may explain why they are not well prepared to help manage them. The PROGE approach was developed to update democratic evaluation to take on powerful interests and confront threats to democracy. However, it remains to be seen how well this approach works in practice, whether it can mobilise independent funding, build alliances with advocacy groups, and support democracy as intended. The MDE orientation could have confronted these challenges, as Tomasi (2012) developed the theory when these threats were already known, but he did not. None of the orientations reflects awareness of the mediatisation of public policy and governance, or of evaluation. PROGE is the only approach that considers and suggests how new channels of communication can be used, but it lacks media strategies for evaluation. Hence, democratic evaluation needs further development to become a constructive tool in today’s polarised and mediatised society.

Conclusions

First, democratic evaluation is ill prepared to help manage current growing polarisation and threats to democracy. Progressive evaluation is the only approach that addresses threats to democracy and offers some new keys to deal with some of them. However, it remains to be seen whether the approach will work as intended. The other orientations seem somewhat ready to address threats to democracy and support democratic renewal, in line with the evaluators’ assumed responsibilities. The democratic theories outlined here guide the evaluator in conceiving and managing the challenges somewhat differently: the EDE and the MDE evaluators will mainly support established democratic institutions, whereas the PDE, DDE, and PROGE evaluators can do this but also support democratic renewal that enhances participation and deliberation in both traditional and new ways.

Second, the approaches are poorly prepared to deal with the mediatisation of public policy and governance, and possess incomplete media strategies for democratic evaluation. The different orientations are somewhat ready to deal with the mediatisation of public policy and to include the management of various media as part of the evaluators’ assigned responsibilities. The PDE evaluator, for example, is somewhat prepared to protect participants from undesirable media exposure. The PROGE approach advocates the full use of new information technologies and social networking but without discussing the risks involved in doing so. Hence, comprehensive media strategies for all types of democratic evaluation remain to be developed.

Third, democratic evaluation can be a constructive tool with which to maintain and develop democracy in a polarised and mediatised society if evaluators gain knowledge of the threats to democracy, democratic transition, and renewal, and if they develop the awareness and competence required to manage these challenges, informed by research into mediatisation and democracy. The orientations can be constructive but in different ways. Democratic evaluation has, and always will, head in different directions as they are based on different democratic theories, theories that are constantly contested. Thus, which way is the most constructive is a question that cannot be answered.

Discussion

Some organisations and parties question established democratic institutions and knowledge if they do not support their agenda, which undermines democracy. The attraction of populist movements reflects the fact that democracy and its institutions are not currently working well in all respects, indicating a need for renewal. Democratic evaluators should not serve undemocratic organisations and should take account of criticism when supporting established democratic institutions and democratic renewal. A more difficult question is whether democratic evaluators should accept commissions from governments consisting of parties with anti-democratic roots. The vignette illustrates how the Sweden Democrats, with Nazi roots, act in a way that erodes trust in public policy and democracy as well as in evaluation. Democratic evaluators can undertake evaluations commissioned by national or local governments as long as they do not violate democratic and human rights principles, but should refuse to evaluate programmes that treat different residents in different ways or seek evidence blaming immigrants for committing crimes more often than the majority population, for example. However, they should not be afraid to explore controversial questions if necessary, in order to arrive at a multifaceted assessment of a policy or programme, providing they are aware of the background to these questions.

The perspective applied here is that democracy and evaluation are contested concepts, with meanings that are constantly under contention and constantly evolving. How current problems and challenges are described indicates that problems (e.g. democracy in decline or transition) and solutions are co-constructed (Hanberger, 2003; Harmon and Mayer, 1986). A productive discussion of threats to democracy and of how to maintain and develop democratic evaluation should be informed by democratic theory. Most evaluators and commissioners commit themselves to democratic ideals and view their work as part of a democratic project without reference to democratic theory, but the ways evaluations are set up and conducted inevitably reflect one or more such theories. The chosen evaluation approach and assumptions should be explicitly described so stakeholders can judge the value and limitations of the evaluation.

Generally, a democratic evaluator should be aware of how a policy or programme and its evaluation can support or threaten democracy. The evaluator can support established democratic institutions, for example, by not accepting evaluation commissions that could undermine democracy or restrict the terms of evaluation, and by excluding participants who violate democratic principles and human rights. Democratic evaluation should also have a critical function in democratic governance, whether supporting or renewing established institutions (Hanberger, 2012). There are many ways evaluation can help develop democracy, for example, by developing criteria and using them to evaluate programmes of democratic experimentation, deliberation, and civic engagement (Fishkin, 2014; Gastil et al., 2013). Whether evaluations of, for example, online deliberation (Friess and Eilders, 2015) will help in renewing democracy depends on whether these modes function well.

Ways to develop different orientations

The elite-democratic evaluation would benefit from developing competence in helping manage threats to democracy and resisting increased political and media pressure. The EDE evaluator may overlook changes that undermine democracy if she uncritically accepts decision makers’ evaluation questions. The evaluation could, contrary to its intention, contribute to dismantling democracy from within. She should therefore be prepared to act if the commissioner or those in power use the evaluation to restrict democracy or misuse the evaluation in other ways, potentially leaving the evaluation and informing the public as to why. The evaluator’s responsibility should include that of correcting misrepresentations, for example, if the evaluation is used as demonstrated in the vignette. An EDE evaluator ensures that the contract allows her to act as an independent expert, that the report will be public accessible and follows up that the evaluation is reflected in a fair way in different channels and media.

In democracies violating democratic principles or moving toward authoritarian leadership (Freedom House, 2017), progressive evaluation could help reclaim democracy. It would be useful to know more about how to get independent funding, how to ensure professional autonomy, and how to choose organisations with which to build alliances – and under what conditions, without losing independence and credibility. The PROGE would benefit from some kind of organisation or institutional arrangement so the evaluator does not stand alone when ‘speaking truth to power’. It is not clear which democratic theory guides this approach. Evaluations developed together with civil society organisations could be informed by both discursive and participatory democracy. An evaluation developed together with voluntary organizations to improve reception of refugees or the sharing economy in civil society could promote participatory democracy whereas an evaluation developed to provide a multifaceted account of a contested governmental integration programme could support collective learning and discursive democracy, for example.

A participatory democratic evaluator could consider building a stronger wall around the evaluation and take measures to prevent increased political and media pressure and to protect participants from it. MacTaggert (1991) recognised long ago that the evaluator needs to protect weak groups from political pressure. Today, the evaluator would need to offer protection from pressure exerted by organisations outside the political system, as well as from political insiders using the evaluation in a detrimental way. The PDE evaluator also needs to develop a strategy for using social media. The evaluation participants could, for example, agree to use an internal network or private group to share experiences during the evaluation process and decide what to share and with whom outside the network.

A discursive democratic evaluator should exclude participants who impede deliberation, for example, by using the evaluation as a platform for a political agenda. Criteria for inclusion should be developed, clarified, and maintained to achieve sound deliberation and collective learning. A comprehensive media strategy for DDE should explain why DDE-informed evaluations must provide multifaceted representations of programmes and policies and that these evaluations may not be aligned with the media logic. Without a multi-criteria-based evaluation, stakeholders and citizens cannot use the evaluation for collective learning, programme development, and democratic renewal and accountability (Behn, 2001). Developing DDE would benefit from strengthening an existing evaluation institution, or build one, founded on a discursive democratic platform with a developed media and communication policy. A ‘DDE-institution’ develops stakeholder evaluations by integrating knowledge and experiences of evaluators, media, democracy and area experts and key stakeholders (including citizens). The conducted evaluations support discursive democracy in different settings including new digital settings. This could e.g. entail creating and coordinating a digital evaluation forum that provides electronic access to evaluation reports and relevant documents. Evaluations of e.g. online deliberation could include assessment of social trust, respect and other factors claimed important for democracy in this context. The director and evaluators can communicate evaluations internally and externally, promote collective learning, manage various media and tendencies to polarization among participants or in the wider accountability environment. The institution builds and sustains credibility, authority and legitimacy by developing high-quality evaluations and engage in evaluation discourses.

Awareness of mediatisation

Democratic evaluators can learn from policy and media research about the mediatisation of politics and governance and about what new media and internet-based mass communication imply for democratic governance and evaluation. Discussion of monitory democracy (Keane, 2008) illustrates how the mediatisation of public policy and evaluation has changed the conditions not only for representative democracy but for public scrutiny as well. This implies that a democratic evaluator needs to respond if other scrutinising institutions are using the evaluation or are drawing different conclusions regarding the same policy or programme, and must speak out about and defend the evaluation in the media.

In line with Keane (2008), the philosopher Han (2017) has paid attention to what social media do with us as human beings and citizens and how these media have changed the conditions for political action and democracy. These new modes of digital communication have created new opportunities but are also responsible for the disintegration of community and public space and are slowly eroding any possibility for real political action and meaningful political discourse. The fact that anyone can post their opinions anonymously affects fundamental values such as respect, trust, and responsibility (Han, 2017). Keane (2008) noted that the era of mass communication has gone hand in hand with the growth of all kinds of scrutinising organisations, making ‘the powerful’ constantly vulnerable to these organisations and ‘the power of the powerless’. Other researchers (cited above), including Picciotto (2015), have a more optimistic view of this evolution and foresee new options for deliberative democracy. Democratic evaluators should be aware of the opportunities, risks, and implications of new modes of communication for democracy and learn to communicate and act in this new context.

When evaluating a policy or programme, the evaluator should account for whether it has been mediatised (Strömbäck, 2008) and, if so, how this may have changed it. A mediatised policy can be a moving target that distorts the policy discourse and complicates democratic accountability (Behn, 2001). Evaluation could also be asked to or voluntarily adapt to the media logic in order to become visible in a crowded information society. Whether democratic evaluation can safely be mediatised and what challenges and risks are involved in this merit further discussion.

A media strategy for evaluation should guide the evaluator in managing old and new media, addressing, for example, how to communicate findings and correct misrepresentations of an evaluation if needed. The media strategy should go along with guidelines for quality in evaluation. The programme evaluation standards (Yarbrough et al., 2011) include partial guidelines for helping manage the challenges discussed here (e.g. the utility standard 8, the feasibility standard 3, the propriety standards 3–6, and the accuracy standard 8), but they do not address how to relate to new media and other scrutinising institutions.

Evaluation is but one democratic tool that needs to work in a democracy. Decision makers, officials, professionals, evaluators, various media, and citizens all have a shared responsibility to demand, commission, conduct, and apply evaluations to maintain and develop democracy. It is hoped that this article inspires further discussion of these challenges and of how democratic evaluation can become a productive tool with which to maintain and develop democracy in an increasingly polarised and mediatised society.

Footnotes

Declaration of interest

The author reports no declaration of interest

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Notes

Anders Hanberger is Professor of Evaluation at Umeå University, Sweden. He has published articles on democratic evaluation, evaluation of public policies and programs, evaluation systems, evaluation methodology and policy analysis.

References

Abma

Leyerzapf

Landeweer

(2016) Responsive evaluation in the interference zone between system and lifeworld. American Journal of Evaluation 38(4): 507–20.

Asp

(1986) Mäktiga massmedier: Studier i politisk opinionsbildning [Powerful Mass Media: Studies in the Creation of Political Opinions]. Stockholm: Akademilitteratur.

Behn

(2001) Rethinking Democratic Accountability. Washington, DC: Brookings Institution Press.

Birch

(2001) The Concepts and Theories of Modern Democracy. London: Routledge.

Ceron

Negri

(2016) The ‘social side’ of public policy: Monitoring online public opinion and its mobilization during the policy cycle. Policy & Internet 8(2): 131–47.

Cunningham

(2002) Theories of Democracy: A Critical Introduction. London: Routledge.

Diamond

(2015) Facing up to the democratic recession. Journal of Democracy 26(1): 141–55.

Djerf-Pierre

Pierre

(2016) Mediatised local government: Social media activity and media strategies among local government officials 1989–2010. Policy & Politics 44(1): 59–77.

Dryzek

(2000) Deliberative Democracy and Beyond: Liberals, Critics, Contestations. Oxford: Oxford University Press.

10.

Farazmand

(1999) The elite question: Toward a normative elite theory of organization. Administration & Society 31(3): 321–60.

11.

Fishkin

(2014) Deliberative democracy in context: Reflections on theory and practice. In: Grönlund

Bächtiger

Setälä

(eds) Deliberative Mini-publics: Involving Citizens in the Democratic Process. Colchester: ECPR Press, 27–39.

12.

Freedom House (2017) Breaking Down Democracy: Goals, Strategies, and Methods of Modern Authoritarians. Available at: https://freedomhouse.org/report/modern-authoritarianism-origins-anatomy-outlook (accessed 9 February 2018).

13.

Freedom House (2018) Freedom in the World 2018. Available at: https://freedomhouse.org/report/freedom-world/freedom-world-2018 (accessed 6 February 2018).

14.

Friess

Eilders

(2015) A systematic review of online deliberation research. Policy & Internet 7(3): 319–39.

15.

Gastil

Knobloch

Kelly

(2013) Evaluating deliberative public events and projects. In: Nabatchi

Gastil

Leighninger

, et al. (eds) Democracy in Motion: Evaluating the Practice and Impact of Deliberative Civic Engagement. Oxford: Oxford University Press, 205–32.

16.

Gutmann

Thompson

(2004) Why Deliberative Democracy? Princeton, NJ: Princeton University Press.

17.

Habermas

(1996) Three normative models of democracy. In: Benhabib

(ed.) Democracy and Difference: Contesting the Boundaries of the Political. Princeton, NJ: Princeton University Press, 21–30.

18.

Han

B-C

(2017) In the Swarm: Digital Prospects. Cambridge, MA: MIT Press.

19.

Hanberger

(2001) Policy and program evaluation: Civil society and democracy. American Journal of Evaluation 22(2): 211–28.

20.

Hanberger

(2003) Democratic implications of public organizations. Public Organization Review 3(1): 29–54.

21.

Hanberger

(2006) Evaluation of and for democracy. Evaluation 12(1): 17–37.

22.

Hanberger

(2012) Framework for exploring the interplay of governance and evaluation. Scandinavian Journal of Public Administration 16(3): 9–28.

23.

Harmon

Mayer

(1986) Organization Theory for Public Administration. Boston: Little, Brown and Company.

24.

Helms

(2012) Democratic political leadership in the new media age: A farewell to excellence? British Journal of Politics and International Relations 14(4): 651–70.

25.

House

Howe

(1999) Values in Evaluation and Social Research. Thousand Oaks, CA: SAGE.

26.

House

(2015) Evaluating: Values, Biases, and Practical Wisdom. Charlotte, NC: Information Age.

27.

Keane

(2008) Monitory democracy. Paper prepared for the ESRC Seminar Series, ‘Emergent Publics’, Milton Keynes: The Open University, 13–14 March 2008.

28.

Knox

(2013) Public administrators’ use of social media platforms: Overcoming the legitimacy dilemma? Administration and Society 48(4): 477–96.

29.

Larsson

(2013) Bringing it all back home? Social media practices by Swedish municipalities. European Journal of Communication 28(6): 681–95.

30.

Liston

Harris

O’Toole

(2013) Bridging normative democratic theory and internet technologies: A proposal for scaling citizen policy deliberations. Policy & Internet 5(4): 462–85.

31.

MacDonald

(1976) Evaluation and the control of education. In: Tawney

(ed.) Curriculum Evaluation Today: Trends and Implications. Schools Council Research Studies. London: Macmillan, 125–36.

32.

MacDonald

(1978) Democracy and evaluation. Public address at the University of Alberta Faculty of Education, Edmonton, AB, Canada, 17 October, 1978.

33.

MacTaggert

(1991) When democratic evaluation doesn’t seem democratic. Evaluation Practice 12(1): 9–21.

34.

Mathur

Skelcher

(2007) Evaluating democratic performance: Methodologies for assessing the relationship between network governance and citizens. Public Administration Review 67(2): 228–37.

35.

Pallas

Strannegard

Jonsson

(2014) Organizations and the Media: Organizing in a Mediatized World. London: Routledge.

36.

Pateman

(1970) Participation and Democratic Theory. Cambridge: Cambridge University Press.

37.

Peters

(2016) Governance and the media: Exploring the linkages. Policy & Politics 44(1): 9–22.

38.

Picciotto

(2015) Democratic evaluation for the 21st century. Evaluation 21(2): 150–66.

39.

Schmitter

(2015) Crisis and transition, but not decline. Journal of Democracy 26(1): 32–44.

40.

Schulz

(2004) Reconstructing mediatization as an analytical concept. European Journal of Communication 19(1): 87–101.

41.

Schumpeter

(1942) Capitalism, Socialism and Democracy. New York: Harper.

42.

Strömbäck

(2008) Four phases of mediatization: An analysis of the mediatization of politics. The International Journal of Press/Politics 13(3): 228–46.

43.

Tomasi

(2012) Democratic legitimacy and economic liberty. Social Philosophy and Policy 29(1): 50–80.

44.

Yarbrough

Shulha

Hopson

et al . (2011) Program Evaluation Standards, 3rd ed. Thousand Oaks, CA: SAGE.