Abstract
This systematic literature review examines the impact of open government data on democratic societies. The paper uses a multidimensional operationalization of democracy, concentrating on monitorial, deliberative and participatory processes. It presents expected and found impact of open government data on democratic processes, an overview of the indicators used to measure impact and challenges of realizing impact. The results show that so far limited empirical studies have been conducted regarding the impact of open data on democratic processes. Furthermore, the empirical studies show mixed results and use a broad variety of indicators to measure the impact of OGD. Challenges identified are related to data skills, representation and diversity. More research is necessary especially regarding the impact of open data on deliberative processes and the impact of open data for citizens.
Introduction
At the start of his term, President Obama signed the Open Government Directive [1]. The Directive states that in order to increase accountability agencies shall expand the access to government information by making it available online in open formats. In 2011 eight governments, including the USA, launched the Open Government Partnership (OGP). Currently, more than 70 countries are participating in the OGP [2]. The OGP and other international developments led to open government data initiatives around the world [3]. There are two main societal benefits that prompt governments to develop open data initiatives: the spirit of democracy and economics [4, 5]. Open data are expected to spur the creation of more public value [6], to improve democratic governance and to enhance political participation [4]. For example, during the OGP meeting in New York City on September 24, 2014, President Obama mentioned in his speech: “We’re going to improve transparency with our upgraded website, USAspending.gov, to make it easier for Americans to access and understand how the federal government spends their tax dollars”. Furthermore, open data platforms are also aimed at stimulating innovation, economic growth and improved service delivery [3, 7, 8, 9, 10]. Kroes, Vice-President of the European Commission Responsible for the Digital Agenda, mentioned on December 12, 2011: “(..), my message today is that data is gold. We have a huge goldmine in public administration.”
This study does not focus on the economic value of open government data (OGD) but on the impact of OGD for democracy. Democracy is not a one-dimensional concept [11]. Since the democratic concept is notoriously broad and thus prone to ‘conceptual stretching’ [12], we use a (simplified) conception of democratic interrelated and complimentary processes: monitorial, deliberative and participatory [13]. The basic idea of monitorial democracy [14, 15] is that government obtains a mandate from the people to rule but the way this mandate is used is monitored and can be revoked if abused. The role of government is to disclose information; either proactively or reactively based on a freedom of information request [16] so that citizens can monitor their representatives, be watchful and hold government accountable [14, 15]. Citizens are aware that they need to keep an eye on politics but allow intermediary institutions such as the media, to play an important role [15]. Deliberative democracy highlights that an open debate is needed to find collective solutions to societal problems [17]. Government’s role is to facilitate channels or ways in which citizens (with or without governance actors) can deliberate [11]. Citizens exchange ideas, perspectives and come to an informed consent [15]. They can engage directly in decision-making through discussion. Furthermore, it has the potential to include diverse citizens’ voices in the policy process [18]. The basic idea of a participatory democracy is that citizens actively engage and collaborate directly in the solution of societal problems [11, 19]. Government’s role is to facilitate collaboration [13]. Collaboration brings individuals with expertise together with government decision-makers to create solutions. OGD can have different impacts in these different democratic contexts. OGD can respectively provide insights into government processes, can fuel open debates and can be regarded as a resource that can be used to develop new solutions for collective problems.
The opportunities of OGD for democracy are promising, but the question arises at the end of the second term of the Obama presidency, what impacts and results have been found of open government data initiatives on democratic societies. So far no systematic overview exists of the actual impact of OGD. OGD research is still in its infancy and more research in this area is necessary [20, 21]. Furthermore, several scholars [4, 5] point out that there might be several challenges to OGD initiatives that could prevent OGD initiatives from reaching their full potential.
This leads us to the following research questions:
What impact does open government data have on monitorial, deliberative and participatory processes in representative democracies and what are possible challenges in realizing impact?
The purpose of this study is to systematically review and summarize the literature published that measures the impact of OGD on democratic societies as of 2009, when OGD programs took flight [4, 22]. The review contributes to the literature in three ways. First, it provides an overview of the impact of OGD on democratic processes. The impacts of OGD are understudied and this study aims to fill a gap by providing an overview of what we know about democratic impacts. The review will describe expected impacts based on theoretical and empirical studies and found impact based on empirical studies. Second, it provides an overview of the indicators used in the literature to measure the impact. These indicators can be used for future empirical research. The indicators can also be useful to guide practitioners in making decisions about their current and future OGD initiatives. Third, the systematic review provides an overview of challenges pointed out by different studies that can contribute to a future research agenda [23].
A systematic literature review was conducted in order to obtain an overview of the current state of knowledge regarding the impact of OGD on democratic processes. Our research strategy is based on the PRISMA-method Preferred Reporting Items for Systematic reviews and Meta-Analyses [24]. While developed for the clinical medical field, the PRISMA method also proves a suitable method for a systematic literature review within the field of public administration [24, 25] with the advantage of being replicable and transparent [25].
Study & report eligibility criteria
Both theoretical and empirical studies regarding the impact of OGD on democracy were included in this review. Furthermore, only peer-reviewed articles in English were included which is common in systematic reviews [25]. Books, chapters or conference papers were not included. Finally, the scope of our search included articles published between 2009 and 2016.
Search strategy
Three search rounds were conducted in the summer and fall of 2016. We searched for articles in two databases, Scopus and Web of Science, and in three journals; Government Information Quarterly, Information Polity and Transforming Government: People, Process and Policy. The following terms were used to search the databases: Government open data AND democracy, Government data portal AND democracy, Government public portal AND democracy, Government open data publishing AND democracy, Open government platform evaluation, Evaluating AND open government AND democracy, ((transparency AND democracy) AND impact), ((online deliberation AND democracy) AND impact), ((citizen online participation AND democracy) AND impact), ((collaboration AND democracy) AND impact), ((democracy AND e-governance) AND impact).
Record selection
In line with the PRISMA method two rounds of assessing articles were conducted. In the first round, articles were assessed on the basis of their title and abstract. Articles that mentioned open government data or open government or democratic processes were included. Studies that focused exclusively on online or offline media, such as newspapers without a reference to government or democratic processes were excluded in this round. In the second round, the full-text of the articles were assessed. Articles that did not mention open government data in the fully assessed text or used open data as a dependent variable were not included in the text. In total, 43 articles were included in the meta-analysis. An overview of the review process is given in the form of the PRISMA flow diagram (Fig. 1). ‘Additional records’ in Fig. 1 are the sum of the three journals we included.
Overview of articles selected following the PRISMA method.
Theoretical and empirical studies
The articles included in the review consist of both empirical and theoretical articles. Twenty-four of the 43 articles are based on empirical studies. The initial studies were mainly theoretical but a strong increase in empirical studies can be observed in the last three years (see Fig. 2). Empirical studies in this review include quantitative and qualitative studies, but also historical analyses and meta-analyses.
Number of empirical and theoretical articles included in the meta-analysis, per year.
Regarding the democratic dimensions, it can be observed (see Fig. 3) that the included studies pay most attention to monitorial and participatory processes and least to deliberative processes.
Overview of democratic dimensions covered in the meta-analysis, per year.
In the articles analyzed, 28 percent (12 articles) of the authors explicitly referred to monitorial, deliberative or participatory democratic processes. As noticed earlier, democracy is a concept that is very broadly applied. In order to also be able to interpret the results of studies that are not explicit about the democratic processes, the articles were coded based on the conceptualization of monitorial, deliberative and participatory processes of democracy used by (Ruijer et al. [13]). The intercoder-reliability: Cohen’s Kappa was 0.9, which reflects a high level of agreement between the two coders.
In order answer the research question regarding the impact of OGD on the democratic dimensions an explicit distinction was made between expected and found impact. The expected impact and indicators or strategies proposed are based on both theoretical and empirical studies. The found impact is solely based on the articles that empirically research the impact of OGD. Furthermore, scholars take different levels of analysis in the assessed articles. Some focus on the role of institutions and the workings of government, whereas others focus on the role of citizens, information or technology, or take a more holistic approach. These different levels or perspectives are described in the findings below. Finally, challenges in realizing impact of OGD as pointed out in the different studies are described.
Monitorial democratic processes
Expected impact
In order to have a well-functioning democratic society citizens should be able to monitor government processes [4]. By providing access to relevant information, OGD can provide insight into government processes and mechanisms [27, 28, 20] and increase transparency of these processes [29, 30, 31, 27, 32, 33, 34, 35, 4, 36]. Indirectly, OGD is expected to contribute to a more accountable government [37, 18, 30, 27, 38, 39] reduce corruption [40, 4] and enhance public trust [41]. Furthermore, Meijer [39] argues that information access should be combined with enabling environments for collective action in order to generate impact on accountability. Finally, some authors [42, 34] are more critical towards the impact of OGD on democratic societies. Birchall [34] argues that the publication of government information will put certain responsibilities in the hands of citizens: they are asked to be auditors, analysts, translators and programmers. She expects that ‘ordinary’ citizens cannot fulfill these responsibilities. Moreover, if the data are open, the question rises whether it is then considered the fault of the monitorial citizen if anomalies, abuse or corruption are not noticed [34].
Indicators for impact
Scholars use a broad variety of indicators (see Table 1), to measure the impact of OGD on monitorial processes. From a government perspective, Zuiderwijk and Janssen [33] develop a framework for assessing open government data policies. They make a distinction between performance indicators (the usages of publicized data, risks and benefits of publicizing data) and impact in terms of the realization of public values, thereby referring to the seven types of impacts identified by Harrison et al. [18]: financial, political, social, strategic, quality of life, ideological and stewardship. Other scholars [41, 35] focus on the role of government and how opening of data or information could have an impact on daily work processes or rule-making procedures of government agencies. Worthy [21] makes a distinction between direct and indirect effects of OGD. The direct accountability effects refer to professionalization of the system, improved information flows and a greater understanding internally. Indirect effects refer to accountability effects that involve network and feedback effects of attention and decay regarding OGD.
Overview OGD impact and indicators monitorial dimension
Overview OGD impact and indicators monitorial dimension
Other studies focus on the role of citizens [29, 4] or intermediaries (journalists) [30, 43]. OGD should be relevant for citizens, allowing them to scrutinize and reuse the data and monitor policies and government initiatives [4]. For citizens the robustness of access in the form of physical access, intellectual access and social access, are a strong determinant of the levels of freedom given in a society [29].
Several authors focus on information. Fung [30] distinguishes four principles that articulate the conditions under which information can produce consequences desirable for a democratic society such as transparency: availability, proportionality, accessibility and finally information or data should catalyze action. Other transparency information-based indicators mentioned are stewardship and usefulness [41], but also timeliness, the presence of Freedom of Information rules, relevance of location, primariness and machine readability [30, 40].
The authors who empirically research the impact of OGD on monitorial democratic processes find varying impacts. In terms of government related impact, some scholars argue that changes in transparency of government processes are connected and interrelated with societal and political changes [39, 44]. Bates [42] from a critical point of view finds that the UK Open Government Data agenda is not only being used as democratization project to increase transparency and citizens engagement but also to fuel a range of broader and more controversial policies, aimed at a neoliberal form of state. Other scholars find a positive impact. Jaakola et al. [35] for instance find that open government data can lead to improved internal government processes in terms of producing, unlocking, sharing and using data. Parycek et al. [27] find that OGD is perceived as contributing to employee motivation due to more interaction with stakeholders. They also find that OGD can contribute to more insights in activities of government, more information for voting, diversity of opinion and to an improved public image of government agencies. Other scholars [33, 21] find limited impact on government processes. Dawes et al. [5] take an ecosystem approach to open government data initiatives and find that in one of the two cases in their study that progress was made toward political and strategic goals of transparency, public service and good management along with improvements in stewardship and agency mission accomplishment. Regarding accountability, Worthy [21] examines the democratic impact of the publication of the UK’s Transparency agenda and finds that the local government spending data have driven only some accountability and information transmission [21].
Taking intermediaries as point of departure, Felle [43] finds that the accountability role of journalists as watchdogs is strengthened by the use of data reporting methodologies and digital tools to investigate and to tell stories in the public interest. Finally, Murillo [40] finds limited impact in his international comparative study in Latin America. He finds a moderate release of data relevant to areas where corruption generally takes place but its contribution to providing meaningful information to the citizenry is in fact minimal.
Challenges
In order to realize the monitorial democratic impact of government data scholars point out several challenges. First of all, the impact of OGD is more complex and political than advocates proclaim [21]. Political support for OGD might be limited with only a few OGD enthusiasts [21]. Janssen and van den Hoven [36] point out the complexity of balancing the two democratic values, transparency and privacy. Second, several authors point out the gaps in society in terms of access and skills for using technology [29, 34, 40, 27, 37]. Finally, Birchall [34] explores the implications of what she calls the info-capitalist-democracy and argues that citizens may not have the skills or time to realize the social value of OGD and will therefore outsource to entrepreneurs. Consequently, “the rationality of the market extends to the democratic contract between representatives and the represented itself. We become reliant upon the market to close the circle of democratic representation and accountability upon which it is based” [34, p. 191].
Deliberative dimension
Expected impact
Moss and Coleman [49, p. 411] define deliberation broadly as communication that induces reflection on preferences and perspectives in a non-coercive way with the aim of achieving inclusive, informed and negotiated policy formation and political decision-making. In this dimension diversity of opinions, and debate are perceived as societal benefits [27, 45]. OGD can contribute to informed citizenry [46, 47], who are able to make informed arguments based on the analyses of OGD [48], enabling deeper contributions to deliberative debate on policy issues, leading to increased feedback and engagement from citizens on governments workings [35, 4, 49, 47]. Indirectly, OGD can lead to better and informed decision making, as well as facilitating social stability by developing a sense of community [37, 50].
Indicators for impact
A broad variety of indicators are used to measure the impact of OGD on deliberative processes (see Table 2). Roy [47] identifies in his study three inter-related dimensions that influence deliberative processes: 1) individuals and the way they make decisions, 2) information itself and 3) institutional processes that refer to the structures and cultures of the political arena and the division of roles and responsibilities of government. At the government level, several scholars [51, 49, 48] focus on engagement by indicating whether citizens were involved in selecting policy options and whether feedback options were provided.
OGD impact and indicators deliberative dimension
OGD impact and indicators deliberative dimension
From the perspective of citizens, Jaakola et al. [35] analyze whether it is possible for citizens to track a particular issue, make comparisons, post comments and annotations and read the contributions made by others. Furthermore, an increase in citizens’ feedback on public policy is indicated by an increasing use of data downloads and exchanges of comments on that data [35]. Similarly, Felle [43] uses better story telling by journalists, in the form of visualizations, interactive maps and applications as an indicator to analyze whether this allows the audience to tailor their stories and influences their engagement. In addition, he analyzed the diversity of the audience, whether it concerned a new technical elite or a more diverse audience that read data based stories.
From a technological perspective, Sivarajah et al. [46] study how OGD and e-participation as part of community learning can be leveraged for evidence-based policy decision-making in local government. A tool should take into account the public’s view when setting priorities, help decision makers evaluate impact of different decisions for a scenario and enable users to view different priorities and the impact of policy decisions by means of visualizations.
The empirical studies find mixed results of OGD on the role of government aimed at facilitating citizen deliberation. Some scholars find limited [49] to some impact [47]. Moss and Coleman [49] find that online deliberative initiatives have only been adopted sporadically and argue that micro deliberative practices that engage publics around specific policy issues could be much more widely established. By contrast, focusing on citizens, Jaakola et al. [35] do find impact. They provide examples of outcomes of OGD usage in terms of citizens being able to provide feedback and follow the decision-making of the city in real-time backed up by fresh information. Felle [43] finds that digital data journalism is being widely practiced across all media platforms thereby making stories available to a wider demographic than before. Furthermore, he indicates that visualization of data makes stories easier to read and understand for citizens and allow readers to engage.
From a technology perspective, several authors [46, 45] point out in their empirical studies that tools and OGD have the potential to increase citizen participation in political and public sector processes. Sivarajah et al. [46] find that OGD via tools can potentially “foster communication and interaction between politicians and civil-society, simplifying decision-making processes, demystifying legislative texts, and allowing to effectively visualize arguments and impacts of proposed decisions” [46].
Challenges
Several challenges are pointed out. First of all, there is the challenge of diversity and representativeness [45]. This according to some scholars [45, 49] should be an explicit aim of the deliberation process; ways need to be found to recruit and consult the broadest possible range of participants. Second, several authors refer to the digital divide [47, 49, 45] and plead for a transformation of e-democracy towards equity and social justice [52]. There is a need to bridge the gap between data providers and consumers by using data intermediaries [4].
Participatory dimension
Expected impact
The participatory dimension assumes a horizontal, non-hierarchical, two-way relationship between government and citizens in more collaborative and networked systems [45, 53]. New technologies enable citizens to engage in democratic processes and enable citizens to organize their own forms of public value production, resulting in new forms of cooperation between government and citizens [54, 19]. OGD platforms are expected to facilitate partnership leading to collaborative information production and policy making [55, 56, 19, 53, 39, 48, 57, 45, 44]. Collaboration in this sense has a formal framework thereby distinguishing it from informal networks and interest groups [44]. Citizens can become co-producers of information, knowledge and platforms; they can enhance data quality through feedback [56, 4, 58, 50, 5, 59]. Indirectly, OGD is expected to contribute to empowerment of citizens, influence and participation in policy formulation and collective action [55, 56, 47, 32, 60, 61].
OGD impact and indicators participatory dimension
OGD impact and indicators participatory dimension
Nam [56] propose that citizen participation can be evaluated in terms of a process and as an outcome being the ‘products’ of a participatory process (see Table 3). In this line, Lakka et al. [57] analyze participation as an end product of E-Government maturity. Other scholars focus on processes of collaboration e.g. Citizen-to-Government, Citizen-to-Citizen, Government-to-Citizen [60, 45, 44]. The new roles and responsibilities of government [55, 60, 59] within these processes can be considered as indicators. Similarly, Sieber and Johnson [59] distinguish four roles of government in data provision: 1) a status quo data over the wall form of government data publishing, 2) a form of code exchange with government acting as an open data activist, 3) open data as a civic issue tracker where government accepts direct feedback from citizens on issues, and participatory open data, 4) where government-citizens co-produce data. Especially, the latter forms explicitly promote transparency, rights and democratic objectives [59]. They require from the public sector a role of facilitating and orchestrating, providing tools, managing assets (including data), ensuring public value and providing rules [55, 60]. It requires from citizens and communities to take on more responsibility [55, 59]. Among stakeholders, the degree of formal and direct and the degree of power may vary thereby affecting the outcome [31].
Janssen and Helbig [48] focus on the role of citizens in the policy process. The indicators for impact in their study are linked to the policy process where citizens identify problems based on OGD, put issues on the political agenda, are involved in policy development, collaborative implement policies and finally monitor the issue in terms of policy enforcement.
From an information perspective, Taylor [62] takes a holistic approach, which he describes as the information polity. The information polity consists of two intertwined trajectories: information intensifying government and attempts to strengthen representative democracy through the use of ICT’s as direct forms of democracy.
Finally, some scholars take a more general holistic [38] or ecosystem approach to OGD [5, 51] consisting of interdependent components: data producers, context that includes policy, strategy, organizational culture and legal frameworks, data publication, portals and use, feedback and communication, benefit generation and advocacy and interaction among stakeholders (both inside and outside government).
Found impact
From a government perspective, mixed impact is found. Several studies indicate that technology and OGD can be an enabler of e-government maturity [57, 53] resulting in a more open and participatory government [53]. Janssen and Helbig [48] find that new forms of collaboration have led to new roles for policy makers: 1) orchestration to ensure consistency among tasks and to oversee whether the various stakeholders work in concert to contribute to meaningful engagement; 2) assuring quality of engagement, legitimacy of the process and the usability of the data and information; 3) aggregating and reporting the data collected through new forms of connection and communication [48]. In addition, Baka [50] finds that technology was perceived as creating opportunities to influence the political agenda. She finds a two-fold impact through participatory democratic practices: affecting both society and governance institutions. Furthermore, Ohemeng and Ofosu-Adarkwa [61] find an impact on the supply side regarding the Ghana open data initiative. By contrast, De Blasio and Selva [44] analyze the implementation of open government policies of four EU nations and find that scarce attention is paid to participation and collaboration.
From a citizens’ perspective, scholars [21, 31, 61] find limited impact. For instance, Gonzalez-Zapata and Heeks [31] find in their case study on the Chilean Open Data initiative that the voice of citizens is largely absent. They conclude that the political perspective aimed at democratic change, public participation and empowerment is so far largely aspirational. Similarly, Ohemeng and Ofosu-Adarkwa [61] find the level of civil societal involvement to be abysmal in the Ghana open data ecosystem and conclude that the Ghana ecosystem does not bode well for effective participation. Finally from a holistic perspective, Taylor [62] predicts based on his analysis of past and present developments in the UK, that no major impact will be realized.
Challenges
Several challenges are pointed out. First of all, new forms of collaboration could result in empowerment of citizens or in elite domination [19, 53]. Citizens are expected to make informed arguments but Janssen and Helbig [48] point out that citizens often have limited time and expertise. Therefore it is likely that infomediaries provide services to help citizens. In this light, several authors point to a gap in representativeness; participation might be driven by those who are already engaged [56, 21, 55]. Second in terms of processes, collaboration involves a transfer of decision rights to external parties, which may cause reluctance from public sector actors towards such initiatives [53, 60]. It might even require government to move outside the strongly regulated and entrenched procedures [59]. Finally, there is the quality argument related to citizens’ generated data [58]. However, Craglia and Shanley [58] argue that crowdsourcing can have excellent results but that the outcome depends on the process of designing the project and the active participation of citizens. A bigger challenge they argue is to sustain the commitment of the participants throughout the project.
Discussion
Our findings provide several insights. First of all, our findings show that OGD have the potential to enhance monitorial, deliberative and participatory democratic processes. For each of the democratic processes some studies found impact of OGD. Regarding monitorial democratic processes the impact found concerned the incorporation of OGD in daily work processes of government, improved reputation of government and insight in government mechanisms by citizens and intermediaries. Regarding deliberative processes the impact found mainly concerned the perspective of citizens in relation to diversity, the option of feedback and of story telling. And finally regarding participatory processes, impact was found concerning e-government maturity and collaboration between government and citizens.
Second, our findings also show that about half of the studies assessed found some or no impact. This could simply mean that OGD does not live to its current promise to enhance democracy [4]. However there are also other explanations for this finding. One explanation is that research regarding the impact of OGD is still in its infancy [33, 21]. Even though the number of empirical studies has been increasing recently, the number of empirical studies conducted is so far limited. Many open data initiatives have only recently been developed. They may not yet have generated any impact. Another possible explanation relates to how impact is measured in the different studies. When analyzing the indicators, different levels of analysis can be identified, varying from a government, citizen or information/technology perspective. Consequently, the operationalization of the indicators is very diffuse. The operationalization varies from the number of data downloads and visualizations made to the realization of public values. The studies that took information as point of departure found no impact. In this line, Dawes et al. [5] point out the tendency of some authors to oversimplify the belief in the benefits of OGD, that a direct connection exists between the amount of information made public and enhancement in democracy. Therefore, several authors [21, 5, 51] emphasize the complexity of OGD implementation. These authors suggest a more holistic or eco-system approach that includes a broad range of interdependent components that could be further researched empirically. Finally, it should be pointed out that a few studies included [34, 42] in this review use a critical perspective regarding the impact of OGD. These authors argue that OGD is not only used to increase transparency and accountability but also for market-oriented policies, which can also explain the limited impact on democratic processes. This is confirmed by a historical analysis conducted by Yu and Robinson [63] who show that open government policies have lost its precision over time and currently no longer refer to those that only promote accountability but also to new efficiencies in government services, economy and innovation. Consequently, the current type of datasets released can be used to e.g. improve the quality of life and service delivery but may in fact have little impact on political accountability [63].
Third, most studies seem to focus on the direct impact of OGD and less on the indirect impact of OGD. In this line, Nam [56] refers to impact in terms of whether the ideas and proposals generated by citizens actually have an effect on the work of government. In her study on online deliberation and policy-making, Deligiaouri [64] does not focus on OGD specifically but analyzes whether the suggestions made by citizens were included in proposals of the final law. This indirect effect could be relevant for the impact of OGD as well; in terms of what suggestions based on OGD in monitorial, deliberative and participatory processes are actually included in government policies, plans or proposals. However, indirect effects are especially complicated to measure and can only be measured over time [33].
Fourth, several challenges were found that influence the impact of OGD, such as the political complexity of the open data process, a lack of focus on the demand side and data literacy. Unless open data initiatives are supported by public education, data literacy, skill improvement, driven by a multi-stakeholder body and laws that promulgate agencies to make high quality and valuable data available, the likelihood is that these initiatives will not have an impact [61, 58]. At the same time, research showed that OGD can strengthen the role of intermediaries of e.g. journalists [43]. Monitorial democratic processes rely partly on intermediaries but deliberative and participatory processes ask for direct interaction and collaboration with diverse groups of citizens. These processes are based on the assumption that citizens will monitor, deliberate and participate in democratic processes. However this also raises two other important questions: who are the citizens and how do they participate [65]. Citizens may participate differently due to variation in skills, willingness and interest [66, 65] and due to the degree of openness adopted by government in different societies and political cultures [66]. Furthermore, specific policy issues might require a specific form of expertise. This could result in a select and unrepresentative group of experts who are able to exercise a significant impact on decision-making [56, 60]. The question then becomes who benefits from OGD and one can wonder if it will lead to too much reliance on e.g. entrepreneurs [34]. Further research is therefore necessary regarding how OGD can have an impact on social equity and how communities can and should be engaged in OGD initiatives [52]. As several scholars [67, 68, 62] point out optimism must be tempered regarding participation. Citizens will not simply come to an open source tool and participate. More is necessary, such as infrastructural intermediaries who generate and coordinate online mobilization [68].
Overview OGD impact and indicators
Overview OGD impact and indicators
Fifth, more research is necessary to measure the impact of open data initiatives [4, 19], specifically regarding the open data impact on deliberative processes. Furthermore, additional research is necessary to identify commonalities in these studies and the degree and conditions under which impact is generated [38]. This requires a contextually embedded approach. Consequently, evaluations of OGD programs (should) differ from ex-ante evaluations into the ’costs and benefits’ of an e-government program (see for an ex-ante evaluations [69]). Costs and benefits analysis give less attention to whether the expectations of government programs are met. Evaluations of OGD programs would, in our perspective therefore benefit from an evaluation of costs and benefits, complemented with a normative evaluation of the impact. This could include for example perspectives from multiple stakeholders, but also within different contexts and a more holistic point of view, see for example [18, 21, 51, 50, 31, 54].
Based on our findings an integrative overview (see Table 4) can be presented of the indicators used in the literature to measure the impact that include both the different levels of analysis regarding the role of government, citizens and information for the three democratic processes: monitorial, deliberative and participatory. The elements or components of the integrative overview; the context of the democratic processes, government as provider of information, citizen as user (and provider), journalist as intermediary, open data policies, rules, feedback and diversity, are in line with the holistic contextual approaches [5, 51] identified in this review.
Finally, we would like to point out limitations of our study. One of them is the conceptual debate surrounding democracy. What exactly entails a democratic society is an ongoing discussion. Including such a scope in this article makes it vulnerable, because the full depth of the debate about democracy and its different aspects is not reflected in this article. Furthermore, it should be noted that the three democratic processes described in this article are interlinked in practice. Several scholars refer to different democratic processes in one article and are therefore not easily categorized within one dimension. The different processes build on each other. As Nam [56, p. 17] puts it: “Transparency should enhance citizens’ participation and collaboration”.
Several scholars assume that OGD initiatives will have an impact on monitorial, deliberative and participatory processes in representative democracies. Open data have the potential to enhance insight into government processes, facilitate informed deliberation and informed collaboration. In this systematic literature review it was analyzed whether empirical findings support the claim that OGD can enhance democratic processes. It was found that especially studies that take a monitorial government perspective find positive impact in terms of incorporating OGD in work processes, government reputation and some accountability. A few studies find impact for citizens across the three dimensions in terms of insight in government mechanism, diversity, feedback and story telling. However, the number of empirical studies conducted and evidence found is so far limited. This systematic literature review has also provided insight in the diffuse and varied indicators used to measure the of open government data. An overview was provided that can be helpful for further research into the impact of open government data. Scholars also pointed out several challenges in realizing impact of OGD, whereby the challenge of OGD in relation to social equity issues was pointed out across the different democratic processes.
For researchers, this review provides input for a future research agenda regarding the impact of OGD. More research is needed in terms of the impact of OGD on deliberative and participatory processes, whereby a contextual holistic approach seems especially promising. Future studies may aim to concentrate on further developing empirical measures of the impact of open government data on democratic societies. For practitioners, the strategies pursued by scholars may serve as an inspiration for reviewing, evaluating and assessing the impact of open government programs. The overview of strategies and measures this review reports can help practitioners to look beyond specific communities of users involved in specific projects, and evaluate the impact of open government projects for democracy.
Footnotes
Acknowledgments
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 645860. The authors would like to thank their ROUTE-TO-PA colleagues in particular professor Albert Meijer, for their valuable contributions.