Open data policy-making: A review of the state-of-the-art and an emerging research agenda

Abstract

This section presents a selection of papers on open data policy-making from the 19th Annual International Conference on Digital Government Research 2018 (dg.o 2018). To position the research discussed in this section meaningfully, our introductory article aims to create an overview of the state-of-the-art of open data policy-making research and to derive an emerging research agenda from this overview. We found that much research has been done in the field of open data in the past few decades. However, the number of conducted systematic literature reviews concerning open data research is limited and literature reviews that have included aspects related to open data policy-making are even more rare. Our analysis of eight systematic literature review articles that include the topic of open data policy-making shows that these articles contain four main content-related elements: open data policy-making, a discussion of open data policy-making research, theory used in open data policy-making research and suggested areas for future research about open data policy-making. We discuss each of these areas and we outline emerging research directions categorized by three main topics: 1) open data policy-making and theory development, 2) open data policy-making effects, and 3) open data policy-making from a multi-actor perspective. Finally, we provide an overview of the papers included in this section on open data policy-making.

Keywords

Open data policy policy-making literature review state of the art research agenda

1. Introduction

This section presents a selection of papers on open data policy-making from the 19th Annual International Conference on Digital Government Research 2018 (dg.o 2018). The central theme of this conference was ‘governance in the data age’. Governments increasingly share their data and collaborate with other public and with private agencies to create value (Attard et al., 2016; Janssen, 2011; Mellouli et al., 2014). In recent years, significantly more attention has been focused on how government data is used (Erete et al., 2016; Okamoto, 2016) and how data use may be facilitated and supported (Roman et al., 2018; Weerakkody et al., 2017). While the motivations spurring this increased attention to open data vary, there seems to be an underlying assumption that more open data serves as an important catalyst for societal advancement and transformation (Attard et al., 2016; Janssen, 2011; Jetzek, 2016).

At the same time, most Open Government Data (OGD) studies still focus on the release of OGD without empirically testing the various ways that the data is used and the subsequent results of such practices (Safarov et al., 2017). A great deal of resources are being invested in the creation of open data policies and on the release of government data. However, convincing evidence with respect to the impact and value created by OGD initiatives is lacking (Davies, 2013; Jetzek, 2015). Little is known about the extent to which the objectives of OGD initiatives are actually achieved (Zuiderwijk et al., 2018). This is problematic since researchers, practitioners, and the public need to know whether public value is being created by the development and implementation of open data policies and whether these policies need to be further tailored to become (more) impactful.

In this introductory article we focus on selected research that contributes to the theme of open data policy-making. Open data policies aim to ensure long-term availability to data and to contribute to citizens’ rights of public access to government information (Charalabidis et al., 2018). Thus, open data policies are essential in ensuring access to open data and information on the long term (Jaeger & Bertot, 2010; Zuiderwijk & Janssen, 2014). Open data policies are created through a so-called policy-making cycle (see Charalabidis et al., 2018; Zuiderwijk & Janssen, 2014) that includes five phases:

1.
policy environment and context; the regulatory, social and political context in which the policy is created;
2.
the policy content; the input, including the policy strategy, the policy principles and practical aspects of opening data;
3.
performance indicators; the output, including indicators to evaluate the progress of the open data policy (e.g. the number of datasets downloaded by users);
4.
public values; the impact created through the open data policy, including political, social, economic, operational and technical value;
5.
open data policy change or termination; areas for improvement of the existing open data policy (Charalabidis et al., 2018; Zuiderwijk & Janssen, 2014).

In this introductory article we adopt a broad definition of open data policy-making. We do not only look at documents and directives containing aims for open data policies produced by governments, but also at implementation. For example, the actual provision and use of open data through open data portals and the evaluation of existing open data policies. This is important, since all these aspects are interdependent in the creation of impactful open data policies.

Open data policy-making has already been analyzed at many different levels. For instance, Nordell (2015) examined open data policy-making at the city level of Uppsala, Västerås and Stockholm in Sweden. He concluded that these policies and the motivations to create them vary considerably per city, much depending on the political aims of the municipalities. Nugroho (2013) investigated open data policy-making at the national level of the United Kingdom, the United States, The Netherlands, Kenya and Indonesia (Nugroho, 2013) and Saxena (2018) performed a similar analysis at the national level of Iran, Lebanon and Jordan. Both authors conclude that many challenges still exist. For instance, Saxena (2018) states that the countries she examined merely have “‘aggregated data’ over their web interfaces which are incomplete, outdated and do not permit analytics” (Saxena, 2018, p. 47). Furthermore, Saxena (2017) studied open data policy-making at the international level, namely those of the Gulf Cooperation Council (GCC) member states (Bahrain, Kuwait, Qatar, Oman, Saudi Arabia and the United Arab Emirates) and concluded that these policies are still at an early stage.

Although various studies have already been conducted in the area of open data policy-making, a high-level overview of open data policy-making research is lacking. To position the open data policy-making research discussed at dg.o 2018 meaningfully, this introductory article aims to create an overview of the state-of-the-art of open data policy-making research and to derive an emerging research agenda from this overview.

This paper is constructed as follows. In the next section we present our research approach, followed by an analysis of research articles in the domain of open data policy-making. Then we discuss the state-of-the-art in open data policy-making research and outline emerging research directions. Finally, we provide an overview of the papers included in this section on open data policy-making.
2. Research approach

Systematic literature reviews can be conducted to summarize existing evidence about a particular topic, to identify gaps in current research, to suggest areas for future research, and to create a framework to position new research activities (Kitchenham, 2004). A review of existing literature is essential for academic research (Levy & Ellis, 2006; Webster & Watson, 2002). To determine the state of the art of open data policy-making research we created an overview of existing literature reviews that contain any information about open data policy-making. The focus of our literature review was on generating an overview of existing open data policy-making research and to identify gaps in order to propose areas for further investigation. We included peer-reviewed conference and journal articles from three databases: Scopus (including many other databases), Google Scholar and ACM Digital Library. We searched for existing literature reviews in the field of open data policy-making using the following keywords: “(open data)” AND (policy OR policies)”, “open data literature review” and “open data state of the art” in the article title, abstract and keywords.

Not surprisingly, the searches produced thousands of publications. Therefore, an in-depth assessment was carried out to refine the corpus of publications that were primarily focused on aspects of “open data” and more specifically “open data policy-making”. For each of the selected databases and for each of the three key word combinations we looked at the first 50 results for the period 2014–2019 (thus reviewing 450 results – although there was overlap), sorted by relevance to our key words, yet we found that many of the reviewed papers were irrelevant to the context of our study. As proposed by Webster and Watson (2002), the citations in the identified articles were also examined to find additional relevant literature and to enrich the literature base. We evaluated the identified articles by first determining their applicability. We assessed the titles, abstract and content. Articles were selected when they contained a review of literature in the field of open data, and at least certain statements were made about open data policy-making. An article’s literature review had to be systematic, the review approach had to be carefully described and explained, and the full article had to be accessible (not just an abstract).

Although the assessment yielded 13 articles that contained a review of open data literature, five did not focus on open data policy-making and, therefore, were eliminated from further examination (Corrales-Garay et al., 2019; Crusoe & Melin, 2018; Hassan & Twinomurinzi, 2018; Lowry, 2015; Virkar & Pereira, 2018). We conclude that much research has been done in the field of open data in the past few decades, yet the number of conducted literature reviews concerning open data research is limited. Moreover, literature reviews have included aspects related to open data policy-making are even more rare.

In the output phase of a literature review the analysis is conducted and synthesis takes place by identifying associations between the relevant parts of the selected individual studies (Danyer & Tranfield, 2009). More specifically, in the context of this study we carefully read again each of the eight selected articles and compared them with regard to their perspectives and conclusions regarding open data policy-making. The results of this analysis and synthesis are discussed in the next section.

3. Open data policy-making research: State of the art

Eventually we selected eight studies for our analysis (see Table 1). In these eight articles, open data policy-making was usually only one of the aspects discussed, in addition to other aspects of open data research. Literature review articles that include the topic of open data policy-making contain four main content-related elements:

1.
open data policy-making;
2.
open data policy-making research;
3.
theory used in open data policy-making research;
4.
suggested areas for future research about open data policy-making.

We discuss each of these elements below.

Table 1
Literaure reviews conducted in the area of open data policy-making

# Author(s) Objective of the literature review Main conclusion(s)

1 Martin (2014) To obtain an overview of implementation barriers, and barriers to use that currently prevent the OGD agenda from completing a breakthrough into the mainstream Open data policy-making:
•
Identify learning effects for OGD policy-making
•
Adopt a strategic approach to creating and supporting niches for OGD use across society
•
Provide greater opportunities for social learning to explore and understand the more contested and contentious aspects of OGD agenda (p. 237)

2 Zuiderwijk et al. (2014) To derive the state-of-the-art with respect to understanding the context of open data innovation, developments, challenges and barriers, to present an overview of open data research and to outline emerging research directions Open data policy-making:
•
Countries without an open data policy may be able to learn from other countries and examine how they can progress quickly
•
Focus more on stimulating open data use

Theory used in open data policy-making research:
•
Institutional and organizational theories have been used to study the topics of policy development, changing systems, and changing organizational cultures and structures (e.g. the theory of institutional/organizational isomorphism) in the open data literature

Suggested areas for future research about open data policy-making:
•
Give insight in which aspects open data policies should contain in different cultural and legal contexts
•
Reveal how open data policies can be implemented

3 Attard et al. (2015) To analyze existing open government data initiatives, tools, and approaches for publishing and consuming open government data Open data policy-making:
•
There is a lack of open government data policies, yet many OGD initiatives belong to existing legal frameworks

4 Susha et al. (2015) Organizational measures to facilitate the use of OGD Open data policy-making:
•
Open data policies on their own are not sufficient to facilitate the exploitation of open data
•
The open data publication process needs to become more problem-driven (aiming to solve societal problems) in open data policy-making
•
Stakeholders need to become more involved in open data policy-making

5 Nugroho et al. (2015) To develop a framework for comparing open data policies Open data policy-making:
•
“Cross-national comparisons of open data policies offer possibilities for systematic cross-national lesson-drawing” (p. 286)
•
Three waves of open data policy making exist: 1) focused on stimulating the release of data, 2) aimed at stimulating open data use, 3) focused on realizing added value from utilizing open data. Existing policies are in wave 1 or 2, but should aim for wave 3
•
Recommendations to improve open data policies:

(a)
Create a law that regulates the continuous release of data from the government ministries and agencies (for countries that lack the legislation for freedom of information);
(b)
Open data policies should provide the guidance that is needed to prepare the data according to what is needed by users;
(c)
Create a relationship between data publishers and users to stimulate the provision of data and to increase the involvement of data users;

Table 1, continued

# Author(s) Objective of the literature review Main conclusion(s)

4.
Create a designated agency of taskforce in-charge for the nation’s open data processes and an ICT infrastructure capable of supporting open data;
5.
Create initiatives and incentives stimulating the demand for data.

Suggested areas for future research about open data policy-making:
•
Systematic comparisons of open data policies are needed

6 Hossain et al. (2016) To ascertain the current state of research on open data, and to present an extensive exploration for 11 types of analyses: contexts, perspectives, level of analysis, research methods, the drivers, benefits, barriers, theory/model development, the most productive journals, authors, and institutions Open data policy-making:
•
A mandatory policy may eventually lead to a voluntary data-sharing culture
•
“Few studies mentioned that open data policies have to be treated as a regular activity of a department or firm (rather than just releasing some data as part of the commitment) – which IS studies call as routinization.” (p. 33)

7 Kvamsdal (2017) To systematically analyze the foci, methodological underpinnings and theoretical perspectives of 43 open data articles from the e-government and public administration field Open data policy-making research:
•
Out of 43 analyzed open data articles, most focus on policies ( $n=$ 11) and only few focus on adoption ( $n=$ 3) and value ( $n=$ 5)
•
Most papers aim at comparing and evaluating policies to propose possible directions for development (e.g. by studying the role of individual policies or using comparative frameworks to evaluate them
•
Areas that are barely studied concern the actual impact of policies and how they can support value creation and address privacy concerns

Open data policy-making:
•
There is a strong call in the literature to update and integrate policies to support better use and re-use

Theory used in open data policy-making research:
•
The majority of the analyzed studies are empirical and theoretical generalization is not a common topic in the open data literature

8 Safarov et al. (2017) To systematically analyze the different types of utilization, the effects of utilization, the key conditions, and the different users of 101 academic studies about OGD Open data policy-making research:
•
Out of the 101 analyzed studies concerning OGD utilization, 26 discuss legislation and policy as a condition for the use and reuse of OGD resources
•
There are strong implications that the different types of utilization, the effects of utilization, the key conditions, and the different users affect the success of OGD initiatives considerably, yet more research is needed

Open data policy-making:
•
“A clear and harmonized legal framework is needed to regulate the relationship and eliminate ambiguities between copyright, privacy, personal data and data openness to achieve the full potential of OGD” (p. 11)
•
OGD policies should address privacy issues upfront to ensure the compliance of confidentiality and privacy guidelines
•
“Open data developers perform significant role to encourage the adoption of OGD policies” (p. 13)

3.1 Open data policy-making

#	Author(s)	Objective of the literature review	Main conclusion(s)
1	Martin (2014)	To obtain an overview of implementation barriers, and barriers to use that currently prevent the OGD agenda from completing a breakthrough into the mainstream	Open data policy-making: • Identify learning effects for OGD policy-making • Adopt a strategic approach to creating and supporting niches for OGD use across society • Provide greater opportunities for social learning to explore and understand the more contested and contentious aspects of OGD agenda (p. 237)
2	Zuiderwijk et al. (2014)	To derive the state-of-the-art with respect to understanding the context of open data innovation, developments, challenges and barriers, to present an overview of open data research and to outline emerging research directions	Open data policy-making: • Countries without an open data policy may be able to learn from other countries and examine how they can progress quickly • Focus more on stimulating open data use Theory used in open data policy-making research: • Institutional and organizational theories have been used to study the topics of policy development, changing systems, and changing organizational cultures and structures (e.g. the theory of institutional/organizational isomorphism) in the open data literature Suggested areas for future research about open data policy-making: • Give insight in which aspects open data policies should contain in different cultural and legal contexts • Reveal how open data policies can be implemented
3	Attard et al. (2015)	To analyze existing open government data initiatives, tools, and approaches for publishing and consuming open government data	Open data policy-making: • There is a lack of open government data policies, yet many OGD initiatives belong to existing legal frameworks
4	Susha et al. (2015)	Organizational measures to facilitate the use of OGD	Open data policy-making: • Open data policies on their own are not sufficient to facilitate the exploitation of open data • The open data publication process needs to become more problem-driven (aiming to solve societal problems) in open data policy-making • Stakeholders need to become more involved in open data policy-making
5	Nugroho et al. (2015)	To develop a framework for comparing open data policies	Open data policy-making: • “Cross-national comparisons of open data policies offer possibilities for systematic cross-national lesson-drawing” (p. 286) • Three waves of open data policy making exist: 1) focused on stimulating the release of data, 2) aimed at stimulating open data use, 3) focused on realizing added value from utilizing open data. Existing policies are in wave 1 or 2, but should aim for wave 3 • Recommendations to improve open data policies: (a) Create a law that regulates the continuous release of data from the government ministries and agencies (for countries that lack the legislation for freedom of information); (b) Open data policies should provide the guidance that is needed to prepare the data according to what is needed by users; (c) Create a relationship between data publishers and users to stimulate the provision of data and to increase the involvement of data users;

Table 1, continued
			4. Create a designated agency of taskforce in-charge for the nation’s open data processes and an ICT infrastructure capable of supporting open data; 5. Create initiatives and incentives stimulating the demand for data. Suggested areas for future research about open data policy-making: • Systematic comparisons of open data policies are needed
6	Hossain et al. (2016)	To ascertain the current state of research on open data, and to present an extensive exploration for 11 types of analyses: contexts, perspectives, level of analysis, research methods, the drivers, benefits, barriers, theory/model development, the most productive journals, authors, and institutions	Open data policy-making: • A mandatory policy may eventually lead to a voluntary data-sharing culture • “Few studies mentioned that open data policies have to be treated as a regular activity of a department or firm (rather than just releasing some data as part of the commitment) – which IS studies call as routinization.” (p. 33)
7	Kvamsdal (2017)	To systematically analyze the foci, methodological underpinnings and theoretical perspectives of 43 open data articles from the e-government and public administration field	Open data policy-making research: • Out of 43 analyzed open data articles, most focus on policies ( $n=$ 11) and only few focus on adoption ( $n=$ 3) and value ( $n=$ 5) • Most papers aim at comparing and evaluating policies to propose possible directions for development (e.g. by studying the role of individual policies or using comparative frameworks to evaluate them • Areas that are barely studied concern the actual impact of policies and how they can support value creation and address privacy concerns Open data policy-making: • There is a strong call in the literature to update and integrate policies to support better use and re-use Theory used in open data policy-making research: • The majority of the analyzed studies are empirical and theoretical generalization is not a common topic in the open data literature
8	Safarov et al. (2017)	To systematically analyze the different types of utilization, the effects of utilization, the key conditions, and the different users of 101 academic studies about OGD	Open data policy-making research: • Out of the 101 analyzed studies concerning OGD utilization, 26 discuss legislation and policy as a condition for the use and reuse of OGD resources • There are strong implications that the different types of utilization, the effects of utilization, the key conditions, and the different users affect the success of OGD initiatives considerably, yet more research is needed Open data policy-making: • “A clear and harmonized legal framework is needed to regulate the relationship and eliminate ambiguities between copyright, privacy, personal data and data openness to achieve the full potential of OGD” (p. 11) • OGD policies should address privacy issues upfront to ensure the compliance of confidentiality and privacy guidelines • “Open data developers perform significant role to encourage the adoption of OGD policies” (p. 13)

All of the selected articles contain information about open data. Based on an analysis of studies in the field of open data, four of these studies urge that open data policies should become more use-oriented and problem-oriented since many existing studies focus on the supply-side of open data policies (Kvamsdal, 2017; Nugroho et al., 2015; Susha et al., 2015; Zuiderwijk et al., 2014). Several scholars also emphasize the importance of a clear and harmonized legal framework “to regulate the relationship and eliminate ambiguities between copyright, privacy, personal data and data openness to achieve the full potential of OGD” (Safarov et al., 2017, p. 11). Since many OGD initiatives already belong to existing legal frameworks (Attard et al., 2015), such a legal framework is especially important for countries that lack freedom of information legislation (Nugroho et al., 2015). Two studies mention that open data policies should ensure the continuous release of data as a regular activity (Hossain et al., 2016; Nugroho et al., 2015) and two show the potential for generating learning effects, i.e. the activity of improving an existing open data policy through learning (Martin, 2014) or the activity of a country without an open data policy to learn from other countries with such a policy (Nugroho et al., 2015).

3.2 Open data policy-making research

Two open data policy-making literature reviews include the topic of open data policy-making research. Kvamsdal (2017) and Safarov et al. (2017) analyzed the research done in the area of open data policy-making. Kvamsdal (2017) analysed 43 open data articles and concludes that most of these articles focus on policies ( $n=$ 11) and only few focused on adoption ( $n=$ 3) and value ( $n=$ 5). She states that most papers aim at comparing and evaluating policies to propose possible directions for development (e.g. by studying the role of individual policies or using comparative frameworks to evaluate them). Areas that are barely studied concern the actual impact of policies and how they can support value creation and address privacy concerns (Kvamsdal, 2017). Safarov et al. (2017) analyzed 101 articles concerning OGD utilization and state that 26 of them discuss legislation and policy as a condition for the use and reuse of OGD resources. They conclude that there are strong implications that the different types of open data utilization, the effects of open data utilization, the key conditions, and the different users affect the success of OGD initiatives considerably, yet more research is needed to make stronger claims.

3.3 Theory used in open data policy-making research

Two of the analyzed literature reviews concern theory used in open data policy-making research. They reveal that that the majority of the open data studies are empirical and theoretical generalization is not a common topic in the open data literature (Kvamsdal, 2017; Zuiderwijk et al., 2014). Theories are barely used, applied or extended in the area of open data policy-making. This is confirmed by other studies in the area of open data (not focusing on policy specifically). For instance Hassan and Twinomurinzi (2018) state that “OGD is an emergent research area with no existing theoretical frameworks” (p. 299). Zuiderwijk et al. (2014) state that at the moment they conducted their review of open data literature, institutional and organizational theories were used to study the topics of policy development, changing systems, and changing organizational cultures and structures (e.g. the theory of institutional / organizational isomorphism), while other theories may also provide opportunities for learning.

3.4 Suggested areas for future research about open data policy-making

Two of the eight analyzed studies suggest areas for future research about open data policy-making. Zuiderwijk et al. (2014) recommend future open data policy-making research to give insight regarding which characteristics open data policies should possess in different cultural and legal contexts. Furthermore, they state that future research should reveal best practices for how open data policies can be implemented. Similarly, Nugroho et al. (2015) state that systematic comparisons of open data policies are needed and future research should provide those.

3.5 Other topics covered

Finally, we should note that one article went beyond content-related aspects. Hossain et al. (2016) additionally produced findings related to the most productive open data journals, authors and institutions, although these findings concern open data research in general and are not related to open data policy-making in particular, so we did not include this in our analysis.

4. Research agenda for open data policy-making

From our analysis of eight literature reviews in the area of open data policy-making, we derived a number of topics that buttress an emerging open data policy-making research agenda. This research agenda is categorized by three main topics – 1) open data policy-making and theory development, 2) open data policy-making effects, and 3) open data policy-making from a multi-actor perspective (see Fig. 1). While these topics may not be completely exhaustive in regards future research, they do clearly identify omissions in the knowledge regarding open data policy-making. Furthermore, it is important to realize that these research areas are not necessarily independent of each other within the research space we discuss here. In fact, they are in many ways interdependent and necessary components of a field that is energetically creating new and useful knowledge.

Figure 1.

Proposed directions for open data policy-making research.

4.1 The need for theory

Our analysis in Section 3 highlights that existing theories are barely used, applied or extended in the area of open data policy-making. Note that Zuiderwijk et al. (2014) already made a similar point several years ago. Later studies (e.g., Hassan & Twinomurinzi, 2018; Kvamsdal, 2017) reached similar conclusions. We speculate that the absence of clear empirically tested theories of work in open data policy-making may be the result of both the applied and practical nature of the field. Additionally, an initial focus on technical approaches to achieving open data have often given less attention to related social issues or the theories which might assist in their understanding. However, as Lewin (1943) and others (e.g., Van de Ven, 1989) have opined, “nothing is so practical as a good theory”. Often theories emerge within a research field after initial practice-based attempts have yielded unexpected or undesired results. Over time, varied results of open data and open data policy-making (as in other research fields) will necessitate a better overall understanding of practices (i.e. interventions) and results within different socio-technical environments. Ultimately, the very practical evaluation of early practices, frameworks, and prescriptions that emerge in the early phases of any new endeavour or research field, such as open data, may yield more refined empirically tested theories that assist in a more careful development of the research field. More careful theory construction and testing, should yield opportunities to extend the knowledge regarding how open data policies may be applied within different socio-technical contexts.

4.2 Effects of open data policy-making

Our analysis in Section 3 showed that much research in the area of open data policy focuses on the supply-side. However, having a policy in place does not necessarily mean that public objectives that the policy is supposed to address are actually met (Zuiderwijk et al., 2018). As a practical and theoretical necessity, future investigations should examine the direct and indirect effects of various implementations of open data policies. Such evaluations should focus on the nature of open data policy design, formation and implementation, as well as both near and long-term impacts. Future studies should not only highlight the impact of specific policies on public organizations but also on specific actors within the broader social environment. The role of open data and open data policy-making in the creation of public value is of ultimate importance since real or perceived value creation is, ultimately, the primary outcome measure for the ODP policies and programs. However, public value creation is not always easy to assess since it is inherently a multidimensional and often dynamic concept within any policy area (Bozeman, 2007). One researcher may consider a relatively straightforward financial assessment of inputs and outputs, while others may consider a more complex function that includes both direct, indirect, or endogenous effects among key social and technical factors. Evaluation factors are value-laden and, therefore, may also be difficult to measure and analyze. As a result, outcomes regarding policy-making may have multiple interpretations (Hanberger, 2001).

4.3 A multi-actor perspective

In our search for literature reviews related to open data policy-making, we only found articles that discussed open data policies of public organizations. One of them compared open data policies of different countries (Nugroho et al., 2015). In addition to governments, other actors, such as business representatives, research institutes, universities, and public interest groups are also working on open data policies. Moreover, each of these types of organizations have a number of stakeholders that can be directly or indirectly involved in open data policy formation or be impacted by the resulting open data policies. For example, large multinational corporations such as Google, Amazon, and Facebook often directly participate in the formation of information and other regulatory policies by various governments. Such organizations are also impacted by what data or data policies governments adopt (Haucap & Heimeshoff, 2014). In some cases, such corporations directly benefit from making government data available because it facilitates or directly enables services that they provide their customers. Additionally, as with any policy area, a particular stakeholder, or stakeholder group, may fall within a highly interrelated and overlapping network of open data policies adopted and implemented by various organizations (Hanf & O’Toole, Jr., 1992). In essence, open data policy-making has potential to have multiple impacts on a variety of stakeholders. While taking into account the different environmental, cultural and legal contexts of such actors, comparisons of various policies may also offer opportunities for knowledge development (O’Toole, Jr., 2000).

5. Selected research that contributes to the theme of open data policy-making: Paper overview

In the introduction we stated that we adopt a broad definition of open data policy-making, consisting of policy environment and context, policy content, performance indicators, public values and policy change or termination (Charalabidis et al., 2018; Zuiderwijk & Janssen, 2014). It is important that all the different phases of the open data policy-making cycle are being investigated, because all these stages eventually contribute to the creation of impactful open data policies. If research would only address one or a few of these stages it may be difficult to understand why certain open data policies are not effective whereas others are.

This section contains papers concerning open data policy-making that reflect our broad open data policy-making definition. Some of them address multiple stages of the open data policy-making cycle, others address a single stage, but all together cover an interesting variety of aspects related to open data policy-making, including open data policy-making in multiple countries and from different perspectives.

5.1 Yoon, Joo and Kwon – How to Guarantee the Right to Use PSI in the Age of Open Data: Lessons from the Data Policy of South Korea

This paper by Yoon, Joo and Kwon covers the first three phases of the open data policy-making cycle and looks into the actual management, construction, operation, and utilization of data. It looks specifically at the elements of the South-Korean data policy, including governance principle for data collection, processing, opening, and reuse, and what lessons can be learned from this for future directions of such policy. Based on a case study in South-Korea, the authors state that this policy is mainly aimed at creating the necessary conditions for economic development, unlike general open government data policies aiming to promote administrative transparency. The key elements of the South-Korean open data policy are data management and communication among stakeholders. The main elements of data management will be the quality of data (accuracy and reliability), standardization of data forms (compatibility and availability) and data security (safety).

5.2 Kurtz, Santos and Rover – Open data via websites of Brazilian Superior Courts of Justice: changes between 2013 and 2017

This paper by Kurtz, Santos and Rover focuses on the third phase of the open data policy-making cycle, namely on the actual implementation and performance indicators of open data policy. These authors investigated whether websites containing open data from the Brazilian judicial sector comply with the open data guidelines of the Brazilian Access to Information Act (LAI). Using a structured form they evaluated the availability and quality of data about superior courts’ activities, management, finances and requests for information between 2013 and 2017. The authors found that the institutions under analysis are falling short on their objective of data disclosure and in the compliance with the LAI. Moreover, access to the data varies over time and part of the data is not being updated.

5.3 Luna-Reyes and Najafabadi – The US Open Data Initiative: The Road Ahead

This paper covers the third and fourth stage of open data policy-making, namely the stages of performance indicators and public value creation. Luna-Reyes and Najafabadi carried out five case studies at the U.S. federal government to answer the question ‘what are the main conditions to promote open government data programs?’ The authors identify four core components for the success of open data programs, including:

•
policy, effective policy also provides governance and leadership to make decisions and execute the plan;
•
community; the cases suggest that it is most useful to think on domain-specific groups with interests and needs to produce innovations and solutions to their main problems;
•
technology, technological platforms and standards are basic infrastructures that enable the OGD program; and
•
data quality, which is at the heart of any OGD application, and it is the main concern of every community.

The authors state that current developments in regulation, policy and strategy development at the US Federal Government are promising and have the potential of revitalizing the OGD program, however, these developments are still in a very early stage, and constitute more of a promise than a reality. The authors define a number of challenges that need to be overcome to create more value with open data in the cases that they studied.
6. Conclusions

This section presents a selection of papers on open data policy-making from the 19th Annual International Conference on Digital Government Research (dg.o 2018). To position the research discussed in this section meaningfully, our introductory article aims to create an overview of the state of the art of open data policy-making research and to derive an emerging research agenda from this overview. We found that much research has been done in the field of open data in the past few decades. However, the number of systematic literature reviews concerning open data research is limited. Moreover, literature reviews that have included aspects related to open data policy-making are even more rare.

We selected and analyzed eight systematic literature reviews. The articles selected are necessarily limited given our criteria for inclusion. Therefore, we do not claim this to be an exhaustive review of the relevant literature. The eight selected literature review articles that include the topic of open data policy-making contain the following main content-related elements:

Open data policy-making (addressed by all eight selected studies). Four studies call for more use-oriented and problem-oriented open data policy-making (Kvamsdal, 2017; Nugroho et al., 2015; Susha et al., 2015; Zuiderwijk et al., 2014), two studies refer to open data policy-making’s potential for generating learning effects (Martin, 2014; Nugroho et al., 2015) and some scholars emphasize the importance of a clear and harmonized legal framework (Attard et al., 2015; Nugroho et al., 2015; Safarov et al., 2017), especially for countries that lack freedom of information legislation (Nugroho et al., 2015).

Open data policy-making research (two studies). It is stated that the actual impact of policies and how they can support value creation is barely studied (Kvamsdal, 2017).

Theory used in open data policy-making research (two studies). These articles show that most open data studies are empirical and theoretical generalization is not a common topic in the open data literature (Kvamsdal, 2017; Zuiderwijk et al., 2014). Theories are barely used, applied or extended in the area of open data policy-making.

Suggested areas for future research about open data policy-making (two studies). It is argued that future research in the area of open data policy-making should give insight in which aspects open data policies should contain in different cultural and legal contexts, that it should reveal best practices for how open data policies can be implemented (Zuiderwijk et al., 2014) and that it should provide systematic comparisons of open data policies at different levels (Nugroho et al., 2015).

From our analysis of eight literature reviews in the area of open data policy-making, we derived a number of topics that lead to an emerging open data policy-making research agenda. This research agenda has been categorized by three main topics:

Open data policy-making and theory development. Ultimately, the very practical evaluation of early practices, frameworks, and prescriptions that emerge in the early phases of any new endeavour or research field, such as open data, may yield more refined and empirically tested theories that assist in more careful development of the research field. More carefully tested and applied theories, should provide opportunities to extend the knowledge regarding how open data policies may be developed and implemented within different socio-technical contexts.

Open data policy-making effects. As a practical and theoretical necessity, future investigations should examine the direct and indirect effects of various implementations of open data policies. Such evaluations should focus on the nature of open data policy formation and implementation, as well as both near and long-term impacts.

Open data policy-making from a multi-actor perspective. In essence, open data policy-making has potential to have multiple impacts on a variety of stakeholders. While taking into account the different environmental, cultural and legal contexts of such actors, comparisons of various policies may also offer opportunities for knowledge development.

While these topics may not be completely exhaustive in regards future research, they do clearly identify omissions in the knowledge regarding open data policy-making. Furthermore, it is important to realize that these research areas are in many ways interdependent and necessary components of a field that is energetically creating new and useful knowledge.

Finally, we provide an overview of the papers included in this section. This section contains papers concerning open data policy-making that reflect our broad open data policy-making definition. Some of them address multiple stages of the open data policy-making cycle, others address a single stage, but all together cover an interesting variety of aspects related to open data policy-making, including open data policy making in multiple countries and from different perspectives.

References

Attard,

Orlandi,

& Auer,

(2016). Data Driven Governments: Creating Value Through Open Government Data ( Hameurlain

Küng

Wagner

Anjomshoaa

Hung

Kalisch

. & Sobolevsky

(Eds.): Springer-Verlag Berlin Heidelberg.

Attard,

Orlandi,

Scerri,

& Auer,

(2015). A systematic review of open government data initiatives. Government Information Quarterly, 32(4), 399-418.

Bozeman,

. (2007). Public Values and Public Interest. Counterbalancing Economic Individualism. Washington, D.C.: Georgetown University Press.

Charalabidis,

Zuiderwijk,

Alexopoulos,

Janssen,

Lampoltshammer,

& Ferro,

(2018). The World of Open Data: Concepts, Methods, Tools and Cham: Springer.

Corrales-Garay,

Mora-Valentín,

E-M.

& Ortiz-de-Urbina-Criado,

(2019). Open Data for Open Innovation: An Analysis of Literature Characteristics. Future Internet, 11(3), 77.

Crusoe,

& Melin,

(2018). Investigating open government data barriers. A Literature Review and Conceptualization. Paper presented at the International Conference on Electronic Government.

Danyer,

& Tranfield,

(2009). Producing a systematic review. In Buchanan

D. A.

& Bryman

(Eds.), The Sage handbook of organizational research methods (pp. 671-689). London: Sage Publications Ltd.

Davies,

. (2013). Open Data Barometer: 2013 Global Report. Retrieved from http//www.opendataresearch.org/dl/odb2013/Open-Data-Barometer-2013-Global-Report.pdf.

Erete,

Ryou,

Smith,

Fassett,

K. M.

& Duda,

(2016). Storytelling with data: examining the use of data by non-profit organizations. Paper presented at the 19th ACM conference on Computer-Supported cooperative work & social computing.

10.

Hanberger,

. (2001). What is the policy problem? Methodological challenges in policy evaluation. Evaluation, 7(1), 45-62.

11.

Hanf,

& O’Toole

L. J.

Jr. (1992). Revisiting old friends: networks, implementation structures and the management of inter-organizational relations. European Journal of Political Research,21(1-2), 163-180.

12.

Hassan,

M. I. A.

& Twinomurinzi,

(2018). A Systematic Literature Review of Open Government Data Research: Challenges, Opportunities and Gaps. Paper presented at the 2018 Open Innovations Conference.

13.

Haucap,

& Heimeshoff,

(2014). Google, Facebook, Amazon, eBay: Is the Internet driving competition or market monopolization? International Economics and Economic Policy, 11(1-2), 49-61.

14.

Hossain,

M. A.

Dwivedi,

Y. K.

& Rana,

N. P.

(2016). State-of-the-art in open data research: Insights from existing literature and a research agenda. Journal of Organizational Computing and Electronic Commerce, 26(1-2), 14-40.

15.

Jaeger,

P. T.

& Bertot,

J. C.

(2010). Transparency and technological change: Ensuring equal and sustained public access to government information. Government Information Quarterly, 27(4), 371-376.

16.

Janssen,

. (2011). The influence of the PSI directive on open government data: an overview of recent developments. Government Information Quarterly, 28(4), 446-456.

17.

Jetzek,

. (2015). The sustainable value of open government data. Uncovering the generative mechanisms of open data through a mixed methods approach. Copenhagen: Copenhagen Business School. aaa(000) Jetzek, T. (2016). Managing Complexity Across Multiple Dimensions of Liquid Open Data: The case of the Danish Basic Data Program. Government Information Quarterly, 33(1), 89-104. aaa(000) Kitchenham, B. (2004). Procedures for performing systematic reviews. Keele, UK, Keele University, 33(2004), 1-26. aaa(000) Kvamsdal, P. (2017). Open government data - A literature review and a research agenda. Retrieved from https://www.resear chgate.net/profile/Pernille_Kvamsdal. aaa(000) Levy, Y. & Ellis, T. J. (2006). A systems approach to conduct an effective literature review in support of information systems research. Informing Science: International Journal of an Emerging Transdiscipline, 9(1), 181-212. aaa(000) Lewin, K. (1943). Psychology and the process of group living. Journal of Social Psychology, 17, 113-131. aaa(000) Lowry, J. (2015). Open Government Data Literature Review Project. Retrieved from https//interparestrust.org/assets/public/ dissemination/EU02_20151210_OpenGovernmentDataLiteratureReview_FinalReport.pdf.

18.

Martin,

. (2014). Barriers to the open government data agenda: taking a multi-level perspective. Policy & Internet, 6(3), 217-240. doi10.1002/1944-2866.POI367:

19.

Mellouli,

Luna-Reyes,

L. F.

& Zhang,

(2014). Smart government, citizen participation and open data. Information Polity, 19(1, 2), 1-4.

20.

Nordell,

. (2015). Open Government and Open Data: A qualitative study of three Swedish municipalities. (Master thesis), Mid Sweden University, Sundsvall, Sweden.

21.

Nugroho,

R. P.

(2013). Acomparison of open data policies in different countries. Lessons learned for an open data policy in Indonesia. Master thesis, Delft University of Technology, The Netherlands, Delft.

22.

Nugroho,

R. P.

Zuiderwijk,

Janssen

& de Jong,

(2015). A comparison of national open data policies: lessons learned. Transforming Government: People, Process and Policy, 9(3), 286-308.

23.

O’Toole

L. J.

Jr. (2000). Research on policy implementation: Assessment and prospects. Journal of Public Administration Research and Theory, 10(2), 263-288.

24.

Okamoto,

. (2016). What is being done with open government data? An exploratory analysis of public uses of New York City open data. Webology, 13(1), 1-12.

25.

Roman,

Nikolov,

Putlier,

Sukhobok,

Elvesæter,

Berre,

, et al. (2018). DataGraft: One-stop-shop for open data management. Semantic Web, 9(4), 393-411.

26.

Safarov,

Meijer,

& Grimmelikhuijsen,

(2017). Utilization of open government data: A systematic literature review of types, conditions, effects and users. Information Polity, 22(1), 1-24. doi:10.3233/IP-160014.

27.

Saxena,

. (2017). Significance of open government data in the GCC countries. Digital Policy, Regulation and Governance, 19(3), 251-263.

28.

Saxena,

. (2018). Open Government Data (OGD) in Iran, Lebanon and Jordan: a comparative approach. VINE Journal of Information and Knowledge Management Systems, 48(1), 47-61.

29.

Susha,

Grönlund,

Å.

& Janssen,

(2015). Organizational measures to stimulate user engagement with open data. Transforming Government People, Process and Policy, 9(2), 181-206.

30.

Van de Ven,

A. H.

(1989). Nothing is quite so practical as a good theory. Academy of Management Review, 14(4), 486-489.

31.

Virkar,

, & Pereira,

G. V.

(2018). Exploring Open Data State-of-the-Art: A Review of the Social, Economic and Political Impacts. Paper presented at the International Conference on Electronic Government.

32.

Webster,

, & Watson,

R. T.

(2002). Analyzing the past to prepare for the future: writing a literature review. MIS Quarterly, 26(2), xiii-xxiii.

33.

Weerakkody,

Irani,

Kapoor,

Sivarajah,

& Dwivedi,

Y. K.

(2017). Open data and its usability: an empirical view from the Citizen’s perspective. Information Systems Frontiers, 19(2), 285-300.

34.

Zuiderwijk,

Helbig,

Gil-García,

J. R.

& Janssen,

(2014). Innovation through open data: A review of the state-of-the-art and an emerging research agenda. Journal of Theoretical and Applied Electronic Commerce Research, 9(2), I-XIII. doi:10.4067/S0718-18762014000200001.

35.

Zuiderwijk,

& Janssen,

(2014). Open data policies, their implementation and impact: A comparison framework. Government Information Quarterly, 31(1), 17-29. doi:10.1016/j.giq.2013.04.003.

36.

Zuiderwijk,

Shinde,

& Janssen,

(2018). Investigating the attainment of open government data objectives – Is there a mismatch between objectives and results?International Review of Administrative Sciences, 86(1). doi:10.1177/0020852317739115.