Towards a Framework for Digital Governance Curricula: What Skills and Knowledge do Governance Students Need?

Abstract

Digital technologies are rapidly becoming deeply entrenched in society. These technologies provide challenges and opportunities for democratic and organizational processes, and social interactions between governments and citizens. In this study, we focus on the skills governance students need to obtain to fully grasp the opportunities and challenges raised by digitalization in the public sector and society at large. Using a design approach, we first identify two distinct categories of skills: operational and reflective digital skills, based on the literature, followed by a round of interviews to explore these skills in governance curricula. This resulted in a framework for Digital Governance Curricula based on skills and (future) roles of students as a) engaged citizens, b) professionals, and c) academics. Following, we test and refine our framework based on an assessment and development of three digital governance programmes. Based on our study, we derive implications for digital governance literature and lessons for digital governance course designers.

Keywords

digital governance digital transformation digital skills big data algorithms AI

Key Points for Practitioners

In our study, we developed a heuristic framework for Digital Governance Curricula. Educators and professionals can use this framework for assessing their current curriculum and for developing new curricula aimed at digital governance.

Based on the literature we identify two categories of knowledge and skills relevant for governance students. This implies for academic teachers and practitioners, that they need to consider developing curricula in which attention is paid to both categories.

In addition, we identify three roles of students as academics, professionals and citizens. Our study highlights that attention should be paid to all roles when educators and professionals are developing curricula.

We discuss three different case studies and discuss how the two categories of skills, and the different roles, relate to these.

1. Introduction

The use of digital technologies such as Artificial Intelligence (AI), and Big (Open) Data, brings changes to the public sector and public governance (e.g., Schäfer, 2024). These technologies not only impact the inner workings of public organizations but also government-business relationships, democratic processes, and social interactions between governments and citizens (e.g., Duberry, 2022; Luna-Reyes, 2017; O'Neil, 2016; Ravšelj et al., 2022). While digital technologies are increasingly used in different domains, society is yet to fully grasp the ethical and economic implications on human life (Dwivedi et al., 2021). The capability of effectively using these technologies offers opportunities across various policy domains. Technologies can contribute to poverty alleviation, better health and well-being, and offer opportunities for sustainable development, education, employment, and entrepreneurship (Cordella et al., 2024). It enables public organizations to allocate scarce resources to tasks at which human workers perform better than machines, such as problem-solving activities that require empathy, creativity, and innovation. At the same time, these technologies pose challenges such as algorithmic bias, algorithmic opacity, misinformation, fake news, and filter bubbles (Aïmeur et al., 2023; Peeters & Widlak, 2023; Ruijer et al., 2023). Digital technologies also come with organizational challenges, such as insufficient knowledge and expertise of these technologies, moral dilemmas, and a lack of strategy relating to the implications of these technologies (Dwivedi et al., 2021; Ruijer et al., 2023).

In this study, we connect these two views of techno-optimism and pessimism (Vydra & Klievink, 2019) for the development of academic digital governance curricula and the skills students need. We distinguish two broad categories of skills: operational and reflective skills (van Deursen & van Dijk, 2009; van Deursen & Helsper, 2020). Operational skills include a practical understanding of the relevant developments in digital technology and the possibilities they provide. Reflective skills include the capacity to (critically) reflect on and have knowledge of the implications of digital technologies. Civil servants’ digital skills and knowledge are prerequisites for digitalization in the public sector. It requires that public professionals possess digital skills to use digital technologies, and that they can (critically) reflect on the usage and impact of these technologies. There has been attention to digital skills in general (e.g., Cordella et al., 2024; van Deursen & van Dijk, 2009; van Laar et al., 2017), but the research on digital skills is fragmented, and an understanding is lacking of what “digital skills are made of and how they can be used in everyday life” (Audrin & Audrin, 2022). Furthermore, Kausch-Zong and Schenk (2022) demonstrate that there is a clear digital skills gap between governance education and the duties required by professionals in public organizations. They conclude that digital skills need to play a key role in digital governance curricula to prepare future public professionals.

The technological developments and the digital skills gap make it necessary to rethink digital governance curricula to deliver the knowledge and skills needed for participation in society, and for employment (European Commission, 2019). Digital governance refers to the effects of information technologies on the systems of rules, norms, institutions, and practices, and is especially used in the context of public-private networks of collaboration in public administration and government-citizen interactions (Luna-Reyes, 2017; Idzi and Gomes, 2022). In this study, we use a broad definition of digital governance that encompasses both structural governance (institutional structures and processes) and normative governance (value-related features of structural governance) (Bannister and Connolly 2012). Hence, by digital governance curricula, we mean curricula in higher education that focus on both structural and normative digital governance. So far, limited attention has been paid to which digital skills need to be incorporated in digital governance curricula and how. In this study, we aim to address this gap by examining what skills students need to fully grasp the opportunities and challenges raised by digitalization in the public sector and society at large. We focus on general skills and knowledge that students need to develop in bachelor, master, and professional Digital Governance programs.

By using a design approach, we aim to contribute to both academic knowledge and knowledge for education practitioners by developing a framework for teaching digital governance curricula that identifies a variety of digital skills (Meijer, 2025). Design approaches aim to generate concrete solutions for practice (Romme & Meijer, 2020). This approach fits well with our study that aims to identify what digital skills need to be incorporated in digital governance curricula, so that digital governance students are prepared for their role as professionals in practice (Kausch-Zongo and Schenk, 2022).

The structure of the paper follows our design approach (Brown, 2019; Meijer, 2025; Romme & Meijer, 2020; van Hout et al., 2024). First, we analyze the academic literature, followed by interviews among stakeholders and experts. Based on the literature and interviews, we develop a framework for digital governance curricula. Our framework identifies operational and reflective skills based on (future) roles of students as a) engaged citizens, b) professionals, and c) academics. Following, we test and further refine our framework based on three illustrative cases of digital governance curricula. Our framework can provide input for public sector scholars and practitioners who would like to develop new digital governance curricula or would like to assess and redesign existing curricula.

2. Design Approach

In our study, we use a design approach (Brown, 2019; Hermus et al., 2020) to develop and test a Framework for Digital Governance Curricula that is embedded in both theory and practice (see Table 1). Design approaches aim to create concrete solutions to existing complex problems in practice, thereby using abductive sensemaking (Dorst, 2011; Van Hout et al., 2024). Our design approach, consists of four steps in which we pass through the inspiration space which concerns the exploration of the problem, the space of ideation which refers to the development of ideas, and the implementation space in which the designed solution is applied in the educational context (Bason & Austin, 2022, Brown, 2019; Meijer, 2025; van Hout et al., 2024).

Table 1.
Design Approach Steps and Phases of Inspiration, Ideation, and Implementation (Van Hout et al., 2024) and Brown (2019).

STEP Inspiration phase Ideation phase Implementation phase

1: Literature review X

2: Analysis of education practice by conducting expert interviews among different university departments and external stakeholders X

3: Designing a framework of skills and knowledge for digital governance curricula X

4. Testing and refining the framework in digital governance practices X

STEP	Inspiration phase	Ideation phase	Implementation phase
1: Literature review	X
2: Analysis of education practice by conducting expert interviews among different university departments and external stakeholders	X
3: Designing a framework of skills and knowledge for digital governance curricula		X
4. Testing and refining the framework in digital governance practices			X

First, we explore the problem that motivates the search for solutions (Brown, 2019; Bason and Austin, 2022) by analyzing the academic literature. Second, to explore the problem in practice, we analyze a digital governance practice in a Dutch University by conducting 19 expert interviews. Third, based on the previous two steps, we generate ideas for the design of a framework that identifies skills and knowledge students need in the digital society, which can provide input for the development of digital governance curricula. Fourth, we test the framework based on three cases (Meijer, 2025): the implementation of a bachelor and master student program, the assessment of an existing program for public professionals, and the development of a program for professionals (implementation). Based on this step, we refine our framework by providing concrete examples of digital skills and knowledge that practitioners can use to design or improve their digital governance curricula (ideation and implementation). Our design approach also enhances our academic understanding of digital skills and knowledge needed in the digital era by demonstrating how two views in the digital governance literature regarding techno-optimism and pessimism (Vydra & Klievink, 2019) can be aligned in digital governance curricula.

3. Step 1: Digital Skills Literature

Students are citizens of the Digital Society and are expected to engage in online public services and democratic processes, which increasingly require digital skills and knowledge. In the 1990s the European Commission already stressed the importance of digital skills in education and for EU citizens, with an emphasis on computer skills (Leahy & Dohlan, 2010). In light of the new technological developments, the European Commission developed the Digital Competence Framework (DigComp 2.2) that focuses on twenty-first-century skills, such as the confident, critical, and creative use of technology to achieve goals related to work, learning, innovation, leisure, and participation in society (Marinič & Válek, 2024; Vuorikari et al., 2022b). A recent systematic literature review (Zhao et al., 2021) on digital skills in higher education concluded that the five dimensions of this DigComp framework were most used in higher education. The DigComp framework contains five dimensions that refer to both practical and critical problem-solving skills: 1) information and data literacy (e.g., searching and filtering data but also evaluating data and information content), 2) communication and collaboration through digital technology (e.g., interacting through digital technologies, collaboratively creating knowledge through digital technologies), 3) digital content creation (e.g., programming, but also creating new information and knowledge), 4) safety (protecting personal data and privacy) and 5) problem solving (e.g., understanding conceptual problems and problem situations related to digital environments) (Vuorikari et al., 2022b). These competencies are in themselves not new; however, the importance of their role seems to have changed in the digital society. Scholars also highlight the digital divide: the differential possession of digital skills among citizens (van Deursen & van Dijk, 2009). The problem of being short of digital skills seems quite pervasive; for example, without the ability to critically reflect on digital content, citizens may be vulnerable to digital misinformation. Digital misinformation may have a weakening effect on both the rule of law and democracy (e.g., O'Neil, 2016; Redden, 2018).

Remarkably, only recently have scholars focused on digital skills required by professionals in the public sector (Bilan et al., 2023; Cordella et al., 2024, Dingelstad et al., 2022; Edelmann et al., 2023; Lopes et al., 2023). These studies often build on the five DigComp dimensions mentioned above (cf. Bilan et al., 2023; Vuorikari et al., 2022a). Dingelstad et al. (2022) derive digital skills based on an empirical study among public servants and identify eight competencies: data literacy, data analytical skills, critical thinking, teamwork, domain expertise, engaging stakeholders, innovativeness, and political astuteness. What this study adds to the existing literature is that the digital society requires hybrid competencies, that is, some competencies concern traditional public service competencies (teamwork, engaging stakeholders, domain expertise, and political astuteness), while other identified competencies concern new competencies (data literacy, data analytical skills, innovativeness). Furthermore, Dingelstad et al. (2022) stress that, in fact, every public professional must possess at least a basic level of data literacy skills, critical thinking, and teamwork. The other competencies (engaging stakeholders, domain expertise, political astuteness, data analytical skills, and innovativeness) can be represented at the team level. Another recent study by Edelmann et al. (2023) focuses on the competencies internal and external stakeholders need for the digital transformation in the public sector. They take a holistic meso-level perspective and do not focus on what individual professionals need.

Thus, these studies focus on what skills public organizations and their professionals need, but not on what governance students as future professionals need. Recently, Kausch-Zongo and Schenk (2022) conducted a survey of public administration students to assess the digital skills of students. However, their survey focused on digital skills related to the opportunities of technology, such as technical, informational skills, and social (media) skills, but not on the challenges of technologies that require critical thinking and political astuteness, as pointed out by Dingelstad et al. (2022). Even so, Kausch-Zongo and Schenk (2022) conclude that there is a digital skills gap between. In governance education and the skills required by professionals in public organizations need to be addressed. This study aims to fill this gap.

Furthermore, it can be observed that there is an overlap in the types of skills identified in the literature. Some studies focus on functional or operational skills, such as technical skills, data literacy, and information management, while other studies also name reflective skills, consisting of critical thinking and ethical awareness. van Deursen and Helsper (2020) summarize the diverse types of skills in two broad categories of digital skills: functional skills and critical or reflective skills. The authors indicate that having functional skills makes developing reflective skills possible. Similarly, van Laar et al. (2017) make a distinction between core and context skills. Finally, in its most recent Skills Outlook report, the OECD stresses operational skills and the increasing importance of metacognitive or critical thinking skills (OECD 2023, p. 31). In this study, we build on these studies and distinguish operational and reflective skills as two broad categories for teaching digital governance.

Operational skills imply that governance students have a practical understanding of the relevant developments in digital technology and their opportunities for the internal workings of the public sector, but also for government-citizens interactions and public-private collaborations (van Deursen & Helsper, 2020; van Laar et al., 2017). This includes, for example, basic programming skills, making infographics, interacting through technology, and creating knowledge out of (Big) datasets. Operational skills also include searching for information, being able to search, find, and use information for a specific goal (van Deursen & Helsper, 2020; Cordella et al., 2024). It includes using technologies for collaboration and communication with citizens and other stakeholders, such as transmitting information via social networks, exchanging messages online, and making an online profile (Dingelstad et al. 2022; van Deursen & Helsper, 2020). Operational skills imply using technologies for innovation, problem solving, and creativity, such as making content with technologies (Dingelstad et al., 2022; van Deursen & Helsper, 2020; van Laar et al., 2017). Finally, operational skills are strongly embedded in recent developments of the literature on statistical methods. It is about the recognition of the types of data used and the understanding of Large Language Models based on data from websites (Oberski et al., 2024). In addition, there are many methodological issues that students must be aware of. Despite the new statistical developments, major classical issues remain that students need to be aware of, such as internal and external validity (Schram, 2005), selectivity (Tam & Kim, 2018), missing data (Graham, 2012), measurement error (Ree and Carretta, 2006), and correlation versus causation (Angrist & Pischke, 2009).

Reflective skills include that governance students can (critically) reflect on and have knowledge of the challenges and effects of digital technologies on the internal workings of the public sector, government-citizen interactions, public-private collaborations, and society at large (e.g., European Commission, 2019). It includes that students have a grasp of the legal and ethical justifiability of using certain techniques and knowledge of the risks that are associated with various technologies, such as bias, racial profiling in algorithms, or sharing and tagging content or pictures (Dingelstad et al., 2022; van Deursen & Helsper, 2020; van Laar et al., 2017). It implies being aware of and dealing with values that might be under pressure, and the related ethical dilemmas associated with the usage of technologies in the public sector. Reflective skills also include awareness and informed judgments on how technologies such as AI, LLMs, and algorithms create information and content and how algorithms influence search results (van Deursen & Helsper, 2020; van Laar et al., 2017). Reflection also includes lifelong learning skills aimed at continuously improving one's capabilities (Van Laar et al., 2017). These reflective skills are more than solely analytical skills, but include creativity as well (Donoghue, Voytek & Ellis, 2021). Finally, reflective skills are important because university students need to be aware that Faculty ethical boards require them to safeguard against unethical research designs and their impact on investigation subjects. Data handling in the European context should also be GDPR- proof. A data management plan (DMP) needs to be written before the analysis, which includes a description of how to store the data for replication purposes (Burnette et al., 2016). Furthermore, data must be FAIR–findable, accessible, interoperable, and reusable (Shamoo & Resnik, 2009; Abiteboul & Stoyanovich, 2019). In addition, it may be useful to have preregistration to document the research plan before the study starts in a read-only public repository (Nosek et al., 2018).

In conclusion, the literature on digital skills is fragmented and identifies a broad range of skills for professionals and citizens, but remarkably not on the skills that digital governance students need. Based on the literature, we distinguish two broad categories that are relevant for digital governance curricula: operational and reflective skills that, if incorporated, both in Digital Governance Curricula, align with the opportunities and challenges of digital technology practices in the public sector.

4. Step 2: Assessing a Digital Governance Practice

To further explore the design problem at hand in practice, we conducted 19 semi-structured expert interviews with faculty members and stakeholders within and outside the Governance¹, Law, and Economics Faculty of a Dutch university. This allowed us to analyze what digital skills are currently part of the curriculum and which skills and knowledge need to be developed. The Dutch context is relevant to our study because Dutch universities have jointly expressed their aspiration to create a new international profile for the Netherlands in the Research Agenda: “The Digital Society” (VSNU, 2017). Dutch universities aim to be leading in the field of digital technology oriented towards people and society. Within this context, we interviewed innovation external experts, department heads, and education directors about their vision on the Digital Society and what it implies for the curriculum (see Appendix A). We selected them as leading figures regarding governance, educational curricula, or because of their knowledge of digital governance aspects. More specifically, we asked them whether the current bachelor's and master's digital governance curriculum of the Faculty sufficiently prepares their students for their future positions in a Digital Society and what digital skills and knowledge students need. See Appendix B. The interviews were transcribed, analyzed, and coded using a thematic analysis. The answers were coded concerning active and passive skills, the importance of the skill, and whether it was applied in the current Digital Governance curricula. In addition, we conducted a document analysis of syllabi of the department to assess whether there was attention to digital governance and what skills students learned in these courses. In the interviews and document analysis, we interpreted digital governance broadly in line with our definition. We focused on both normative aspects of digital governance (Bannister & Connolly, 2012) in terms of whether knowledge and skills related to values, norms, and ethics about digital technologies and structural aspects of digital governance (Bannister & Connolly, 2012) such as regulation and the use of technology, or the use of digital technologies in public-private partnership, and the actual use of technology by students. Furthermore, we examined the programs at different levels. We assessed whether programs at the bachelor and master level had a course focused on digital governance-related topics or if digital governance was addressed within general bachelor and master courses, such as “public administration, policy and organization” or developments in Dutch society.”

4.1 Types of Skills

Based on the interviews and document analysis, we found that no structural attention was paid to digitalization within the Governance department. The Law department focused on the development of reflective skills, while the Economics department focused on operational skills. However, the respondents indicated that all students need to have both reflective and operational skills regarding digitalization. Reflective skills are needed to enable students to critically reflect on the impact that digitalization has on their field of study. Operational skills are needed for having a basic level understanding of how algorithms work, how to find the information they need, and offer them insights that are needed to be able to critically reflect on the use of these technologies and the impact they may have. Hence, developing operational skills among students will also reinforce their reflective skills.

At the same time, respondents stressed the importance of a discipline-specific approach. At the Governance department, reflective skills were especially considered important. Respondents mentioned that governance students must be able to critically reflect on the consequences of technology at various levels. At the macro (societal) level, they mentioned knowledge and skills related to e-democracy, Smart Cities, and Cybersecurity. At the meso (organizational) level, e-government, evidence-based policy making, digital public services (using a customer journey approach, chatbots), information management, and ethical dilemmas were mentioned. Finally, at the micro (individual) level, respondents mentioned topics such as what consequences technology has for interactions and communication between public professionals and citizens, but also what the consequences might be of using a chat robot in public service instead of a person. Regarding operational skills, respondents mentioned media literacy, dealing with information overload, being able to assess the quality of data, data management, and making infographics.

For law students, reflective skills allow them to consider the impact that digital technologies can have on the laws and their underlying values. Most interviewees highlight that such reflective skills require at least a passive understanding of how digital technologies work. For instance, one interviewee mentioned the importance of a basic understanding of how algorithms work to understand how the use of algorithms in decision-making processes may lead to discriminatory practices in the public sector. Only if students can identify the chances and risks of the use of technology, they are able to reflect on the appropriateness of the laws in place and the need for adaptations. One interviewee also stressed the importance of being able to assess and validate statistics, graphs, and other representations of data, as these play increasingly prevalent roles in legal research and practice.

For economics, respondents indicated that there should be more attention to specific analytical data-oriented skills, e.g., the software packages R and Python. Respondents indicated that there is an enormous variety of different formats of data, and it is important that students can assess the implications of the quality of these data for the conclusions that can be drawn from an analysis. For instance, students should understand how the process of data selection affects the quality of the statistics that are produced by organizations. In addition, it was emphasized that students should be able to reflect on the consequences of the digital society and gain knowledge about the ethical issues of digitalization.

4.2 Roles

The assessment demonstrated that even though several initiatives were developed at the faculty, the current curriculum required improvement. The respondents thereby distinguished different roles of students as citizens, professionals, and academics. Regarding the impact on the role of students as citizens, respondents stressed the importance of paying attention to the issue of fake news and misinformation. Digitalization also has a strong impact on how people are connected; through social media, people are connected 24/7, worldwide. In other words, it is impossible to be digitally disconnected. Some respondents mentioned that digital communication may reduce interpersonal communication. Additionally, interviewees indicated that students know their way around the front end of digital technology but lack an understanding of processes on the back end. Put simply, they would know which buttons to click, but not why this has the effect that it does. Some of the interviewees followed a so-called “Bildung” argument. This describes the idea that students should be trained by the university to become responsible citizens, capable of reflecting upon life in a broader context than their academic career.

Respondents agreed that digitalization has consequences for students as future professionals. Governments and other organizations are increasingly using various techniques that require both operational and reflective skills. For instance, the issue was raised that it is possible to collect detailed information about individual citizens in public places by the government. The increased digitalization was a major reason for the installation of the General Data Protection Regulation (GDPR) (European Union, 2016). Respondents indicated that students should be trained to think critically about the various forms of data they might encounter in their future professional careers. They should have some general knowledge of cybersecurity, GDPR, ethics, and data management, as well as knowledge about the possibilities that the data offers. Respondents mentioned that the use of algorithms has implications for judgments in the courtroom. In addition, it was mentioned that algorithms contribute to the formation and development of digital platforms, the sharing economy, and new forms of innovation.

About the role of students as governance academics, respondents indicated that digitalization changes the way research is done in terms of data analysis. Most of the interviewees emphasized the importance of understanding the quality of information. It was referred to as “E-literacy”. Students should have some general knowledge about how to process data and have knowledge of issues such as selectivity. Some stakeholders even argued that students should have some elementary knowledge of the principle of algorithms and coding. Furthermore, the digital society brings new societal and academic questions, such as claims of liability. Finally, it was mentioned that students at the three departments do not need to be fully informed about how artificial intelligence works, but they should have some understanding about the essentials of artificial intelligence and digital technologies to better understand the impact of it on their discipline.

Overall, the assessment of the curriculum of the faculty confirms our findings in the literature that both reflective and operational digital skills are important for digital governance curricula, but also demonstrates that in practice, both reflective and operational skills need to be strengthened, since they have not yet been sufficiently incorporated into the programs. In addition, the experts indicated that operational skills were seen as essential for reflective skills. Finally, what this assessment adds to the previous step is the identification of different roles of students as future citizens, professionals, and academics in relation to reflective and operational skills.

5. Step 3: Designing a Framework of Skills and Knowledge for Digital Governance Curricula

Based on the outcomes of the previous two steps and within the design logic, we now move from the inspiration to the ideation space (Brown, 2019). In this section, we develop a framework for digital governance curricula. In the academic literature, we identified reflective and operational skills as two broad categories of digital skills and knowledge. Furthermore, we highlighted the importance of aligning reflective and operational skills, so that students can deal with the opportunities and challenges of digital technologies that public organizations face. Next, through the assessment of digital governance curricula in practice, we identified different roles for students as citizens, future public professionals, and academics. These roles are important for addressing the gap in current curricula and what is required from students in practice (Kausch-Zongo and Schenk, 2022). Combining these outcomes allows us to design a first framework of skills and roles needed for Digital Governance Curricula (Table 2). Each of the six cells of Table 2 gives an integration of the skills and roles students need that can provide input for digital governance curricula. For each of the cells, we included examples that we came across in our assessment in Step 2.

Table 2.
Framework for Digital Governance Curricula Based on Skills.

SKILLS ROLES Operational skills Reflective skills

Citizen Skills and knowledge needed to operate online (public) services, select and find information for a particular goal, and use technologies for social communication, collaboration, citizenship, and creativity. Skills and knowledge for assessing and understanding search results, reliability of information, the impact on human interaction, and created content by technologies, and an understanding of privacy and cyber security risks.

Professional Skills and knowledge needed to operate technologies for specific tasks (e.g., monitoring and evaluation, data driven decision making, evidence-based policy making) in the public service workplace, search, select and use information, having a basic understanding of the workings of digital technologies and using technologies for collaboration with stakeholders, problem solving and innovativeness. Skills and knowledge for assessing what normative digital governance entails (at the macro, meso and micro level): the ethical, legal and human implications and risks of the usage of technologies in the public sector (e.g., in light of data-driven decision making, monitoring evidence-based policy, using chatbots in public services), implications for society, and awareness that technologies are designed by humans.

Academic Skills and knowledge needed to understand challenges and limitations of quantitative analyses in research on governance, with a particular focus on types of data, empirical methods, and methodological issues (e.g., selectivity, missing data, randomization, internal and external validity, measurement error, causality). Skills and knowledge for understanding normative digital governance (macro, meso and micro level): ethical and legal issues around data-infrastructures, such as FAIR, data ownership, GDPR and data management plan, preregistration of empirical research, methodological pitfalls in applied work, the importance of plausibility checks of empirical outcomes.

SKILLS ROLES	Operational skills	Reflective skills
Citizen	Skills and knowledge needed to operate online (public) services, select and find information for a particular goal, and use technologies for social communication, collaboration, citizenship, and creativity.	Skills and knowledge for assessing and understanding search results, reliability of information, the impact on human interaction, and created content by technologies, and an understanding of privacy and cyber security risks.
Professional	Skills and knowledge needed to operate technologies for specific tasks (e.g., monitoring and evaluation, data driven decision making, evidence-based policy making) in the public service workplace, search, select and use information, having a basic understanding of the workings of digital technologies and using technologies for collaboration with stakeholders, problem solving and innovativeness.	Skills and knowledge for assessing what normative digital governance entails (at the macro, meso and micro level): the ethical, legal and human implications and risks of the usage of technologies in the public sector (e.g., in light of data-driven decision making, monitoring evidence-based policy, using chatbots in public services), implications for society, and awareness that technologies are designed by humans.
Academic	Skills and knowledge needed to understand challenges and limitations of quantitative analyses in research on governance, with a particular focus on types of data, empirical methods, and methodological issues (e.g., selectivity, missing data, randomization, internal and external validity, measurement error, causality).	Skills and knowledge for understanding normative digital governance (macro, meso and micro level): ethical and legal issues around data-infrastructures, such as FAIR, data ownership, GDPR and data management plan, preregistration of empirical research, methodological pitfalls in applied work, the importance of plausibility checks of empirical outcomes.

6. Step 4: Testing and Refining the Framework

Following the design logic of our methodology (Brown, 2019; Van Hout et al., 2024), we now move from the ideation to the implementation space in which we test the framework in different digital governance education programs. We provide examples of three cases at a University in the Netherlands. The selected cases were obtained as a convenience sample (Yin, 2013). To investigate how digital governance curricula can be better tailored to practice, addressing the gap between academic programs and practice, we selected programs for both future professionals and current professionals. The first case concerns the development of a multidisciplinary Skills Academy for bachelor's and master's students. The second case concerns the assessment of a data science program for public professionals, and the third case is about the development of a micromaster's Applied Data Science for professionals. The cases are based on internal reports such as student evaluations, course syllabi, course descriptions, and academic literature.

6.1 Testing the Framework

6.1.1 Case 1: Development of Multidisciplinary Skills Academy

Based on the outcomes of the interviews in step 2, the faculty of Governance, Law, and Economics of the Dutch University under study felt an urgency to change the curriculum. This contributed to the establishment of the multidisciplinary “Skills Academy”. The Skills Academy aims to allow students to develop a broader set of (digital) skills in addition to discipline-specific skills. The Skills Academy is a platform for developing a more specific individual profile. “Skills play an important role in the connection between academic knowledge and society” (Skills Academy policy document 2024). The multidisciplinary Skills Academy consists of short elective courses open to both bachelor's and master's students of the three departments.

The Skills Academy was designed based on the future roles of students as professionals, academics, and engaged citizens. In the first year, two initial courses were offered: one focused on operational skills and another on reflective skills. Concerning operational skills, students obtained elementary knowledge about how digitization works in the context of readily available big data. By using large data sets, they were confronted with methods of data visualization, algorithms, and machine learning that they can use both in a professional and academic role. Reflective skills focused on good digital governance. Students worked in small groups on an advisory assignment for a municipality (c.f. Rosenshine, 2012). The municipality gave a guest lecture on good digital governance and asked the students to critically reflect on a concrete case: algorithmic proactive service delivery. Students were asked to take the role of a public professional and to reflect on the case based on the academic literature that considers the impact of the technology at micro, meso and macro level, and on a public values framework used by government organizations that include values such as inclusion, transparency, autonomy, procedural fairness and non-discrimination (Meijer and Ruijer, 2021). The Skills Acadmacy currently offers 18 short courses that address a variety of topics for students in their future roles as academics, professionals, and a few focus on citizenship.

The Skills Academy was designed to stimulate students to participate in courses that complement their discipline-specific program. Even though this aim is largely met, we also observed a challenge in practice. Students might not take all classes and therefore learn some digital skills, and not the full variety of skills identified in the framework. This case study highlights that short additional elective courses on specific digital governance topics can address a wide range of both reflective and operational skills related to different roles, but that structural embedding of these digital skills remains essential as well.

6.1.2 Case 2: Assessment of the Implementation of the Data Science Trainee Program

This case study examines a data science trainee program for young professionals entering the Dutch government as civil servants. A revised curriculum was implemented in 2024, building on the 2019 design. The two-year program consists of several components, including national government operations, personal development, and data science. The data science component includes ten full training days for approximately 30 participants, with a new cohort starting annually. While all participants have completed (applied) university programs related to data science, their academic backgrounds vary. While the program incorporates professional, academic, and citizen perspectives, the professional role provides the most relevant lens for examining the relationship between operational and reflective skills. The following sections analyze the program's documentation and assess whether these skills can be distinguished based on evaluations.

To design the program, domain-specific reference frameworks were identified and aligned with the target group. The EDISON data science framework (Demchenko et al., 2019; Schmitt et al., 2023) served as a basis, incorporating elements from the data science analyst, data science engineer, and data science manager profiles. The framework encompasses skills in data analytics (e.g., machine learning, data visualization), data engineering (e.g., big data infrastructure, cloud computing), and data management (e.g., governance, privacy, ethics). Given the diverse academic backgrounds of trainees, an optional onboarding phase was introduced. The program development also emphasized the integration of theory and practice, drawing from translational science and transdisciplinary education perspectives (e.g., Chute et al., 2013; Daneshpour & Kwegyir-Afful, 2021).

In studying the curriculum, an analysis of program descriptions and presentations reveals the program's objective: “to find, nurture, and inspire top talent to become professionals and/or managers in applied data science, making a positive organizational and societal impact.” The program is structured into three phases: onboarding to address skill gaps, a nurturing phase based on the three selected EDISON profiles, and an inspiring phase featuring advanced courses on broader societal and organizational impact. The onboarding and nurturing phases primarily emphasize operational skills, while the inspiring phase allows for greater engagement with reflective skills.

The course descriptions show a predominantly operational focus, emphasizing technical skills and their applications. However, three sessions specifically incorporate reflective elements, such as discussions on data science's responsibilities across disciplines, the knowledge discovery process, and the gap between theoretical best practices and government practice. Topics on fairness and ethics further cultivate reflective skills by encouraging debates on public values, identifying ethical pitfalls in algorithms, and fostering ethical awareness.

Participant evaluations overwhelmingly highlight the need for more operational skills training and improved applicability, with no explicit mention of reflective skills. Among the numerous feedback snippets analyzed, a unanimous call for enhanced operational training emerged. Teacher evaluations corroborate this finding, as instructors seek to strengthen connections to practice and directly applicable skills but make no reference to reflective skills. The program heavily emphasizes operational skills, with reflective skills explicitly addressed in only three out of ten training days. Interestingly, an analysis of past program designs indicates a stronger presence of reflective skills in earlier versions. However, the co-creation process with government officials and trainees has gradually shifted the program's focus toward operational skills.

This case study highlights considerations for teaching digital governance. While the program aligns well with the domain framework, emphasizing operational competencies alongside some reflective skills such as communication, collaboration, and ethical reasoning, expert and participant feedback indicate a growing shift toward immediately applicable technical skills. Given the inherently interdisciplinary nature and societal impact of data science, reflective skills—including critical thinking, ethical deliberation, and strategic communication—should be strengthened rather than diminished in such programs. To ensure a balanced and future-proof curriculum, program designers could more effectively articulate the necessity of these reflective skills and demonstrate their enduring relevance, particularly as they remain foundational and are less susceptible to rapid technological changes influencing operational skills.

6.1.3 Case 3: Development Micromaster Applied Data Science

This case study focuses on the development of a micromaster's on Applied Data Science (ADS). Micromasters are small educational programs that consist of a selection of graduate-level courses. The micromaster ADS focuses on practitioners, specifically on managers who would like either to apply insights from ADS in their daily work or to learn more about the potential use of ADS. The target group of professionals –those working in the governmental sector – leads to specific issues for the setup of the curriculum. There is an emphasis on the professional role of the participants. The role of citizens remains untouched.

Concerning the operational skills, the courses focus on the application of statistical methods and techniques. Professionals should be able to recognize the pitfalls of empirical research (Table 2). Specific for professionals is that they need to be able to learn to translate the outcomes of the analyses to the professional organization (and vice versa), in which they can report on potential flaws of the analyses. In addition, the professionals will learn about big data research, coding, cybersecurity, data management and liability, and AI and Large Language Models. Reflective skills will also be included in the micromaster. Courses provide a broader perspective on ADS, such as ethical and legal issues of the use of data and algorithms. This second type of knowledge can be applied by participants who are employed in a line function because they are focused on creating connections between departments within organizations.

The design of this case study highlights finding the right balance and order between operational and reflective skills in curricula for professionals. Based on the previous steps of our research, it was decided to start with operational skills in the curriculum. This will lead to a common language among the participants with a broad range of backgrounds. Students need to recognize the pitfalls so they can assess the quality of empirical analyses. In addition, as professionals, they need to be able to take care of these issues so that they can improve the methodological quality of empirical analyses. It can be observed here that there is an overlap between the operational and reflective skills that are needed for academic and professional roles.

6.2 Refining the Framework

Following the design logic (Van Hout et al., 2024), we now refine our framework based on our illustrative cases and the academic literature. These refinements can guide practitioners and academic educators responsible for developing digital governance curricula to make choices that contribute to bridging the gap between digital governance curricula and the work practice of public professionals. First, the findings demonstrate that our framework allows room for different disciplines to accentuate specific skills within the two categories of operational and reflective skills, while also providing input for the implementation (case 1), assessment (case 2), and development (case 3) of discipline-specific and multidisciplinary programs in practice. At the same time, in all three cases, we also observe an overlap between the professional role and the academic role for operational and reflective skills. Skills such as coding, understanding of algorithms, big data analysis, and their implications are important for both roles. Hence, our framework should mainly be considered as a heuristic framework that practitioners can use to design, implement, and assess their programs.

Second, our findings demonstrate that programs tend to focus on either reflective or operational skills, depending on the discipline. We argue that academic teachers and practitioners need to consider developing curricula in which attention is paid to both sets of skills. However, combining both skills in one program can be challenging. After all, not all governance students want or must be programmers, yet a minimum level of both types of skills needs to be specified. Furthermore, we need to point out that the field of data science is quite young. The field is only now starting to formalize and consider reflective skills. Also, operational skills are vulnerable to technological changes, including the rise of large language models and generative AI (cf. Cronholm, 2021; Chiu, 2023), while the centrality of systematic reflection is a new pedagogical function of universities (Barnett, 2011).

Third, we distinguished three different future roles of governance students as engaged citizens, professionals, and academics. The cases provide examples of collaborations between universities and practice that allow for bridging the gap between academic programs and the requirements in the workplace. At the same time, these cases highlight the roles of students as future professionals and academics, while (digital) citizenship receives less attention. This leads to the fundamental question of the role of higher education and whether they should also educate students to be “democratic, creative, caring, constructive citizens of a democratic society” (Harkavy, 2006 p. 6) at a time in which democratic societies are challenged by online misinformation and “fake news”. The intensification of social conflicts and political polarization is fertile ground for misleading information that can, among others, be countered with digital literacy and awareness (Saurwein & Spencer-Smith, 2020; Blaj-Ward and Winter, 2019). For teaching digital governance, this requires a reflection on the curriculum on how and whether these three roles should be embedded in the program, and what this concretely entails in line with the mission and vision of universities.

7. Discussion and Conclusion

Digital technologies such as Artificial Intelligence (AI), including Large Language models (LLMs) and the increasing availability of Big (Open) Data, provide opportunities in terms of efficiency and effectiveness for the inner workings of public organizations and democratic processes. At the same time, these technologies pose challenges for public organizations and society at large. According to Vydra and Klievink (2019), a rejoinder between techno-optimism and pessimism requires a balancing act. Therefore, public professionals need to possess digital skills that allow them to take advantage of the possibilities digital technologies offer while also being able to (critically) reflect on the implications of these technologies. At the same time, there is currently a clear gap between governance education programs and the duties required from professionals in public organizations (Kausch-Zongo and Schenk, 2022; Edelmann et al., 2023). This study has tried to fill this gap by following a design approach to develop an academically grounded, yet practice-oriented framework for Digital Governance Curricula.

Following our design approach, we first explored the problem–the gap between digital governance curricula and the skills needed in the workplace – via academic literature and interviews in practice (inspiration). This led us to take steps towards a framework for Digital Governance Curricula (ideation). Our heuristic framework is built on two categories of skills: operational and reflective skills, and on three future roles of students: as engaged citizens in society, as professionals in the public or private sector, and as governance academics. Following, we validated and refined the framework by assessing it in government practices (implementation). Based on our findings, we conclude that the two categories of skills and the three future roles form an overarching stable foundation for digital governance curricula. Course designers could use our framework, and the examples provide to assess whether the different categories of skills and roles are addressed in the course objectives, study materials and assignments of their digital governance curricula. Furthermore, in line with the aptitude of students, the rapidly evolving technological landscape and the continuous development of tech regulations (see for example the EU AI act) requires that course designers continuously reflect upon, update and modify their courses.

Our design study contributes to existing frameworks for curricula, such as the Digital Competence Framework (Vuorikari et al., 2022b) and the EDISON (Schmitt et al., 2023) framework, by demonstrating the importance of reflective skills next to operational skills. In addition, we have shown that these frameworks could also be extended by taking into consideration the different roles future students may have. Furthermore, our framework highlights that skills and roles should not be considered separately. Barnett (2011) discusses how these different roles overlap in the context of lifelong learning, and that integrating different roles and different contexts of learning is important. Institutes of higher education and research should help to develop such ideas into their existing paradigms of what higher education means (e.g., Barnett, 2011, p. 12; Biesta, 2015), and how value is created for academics, professionals, and citizens in the Digital Society. Such value seems evident for employability, though we also found that in co-creating programs, this does not follow naturally. Integrating roles and skills should be a deliberate effort, enabling learners to deal with new situations and technologies in a complex and changing world.

Finally, we point out some limitations of our study. First, this study was conducted in a university in a Dutch context, which may have limited the outcomes of the application of our heuristic framework. Future research could apply the framework for Teaching Digital Governance in other countries across the globe. This may result in different concrete examples of operational and reflective skills and the different roles. Second, we tested our framework in three cases, but we did not, for pragmatic reasons, conduct a longitudinal study of how students apply the acquired skills in their future roles and practices. A follow-up study is needed to validate the framework by assessing the impact of the programs on the development of digital skills of governance students over time. Third, our study focused on Digital Governance students with a strong focus on the public sector. Yet, students might also become professionals in industry and consultancy companies. This context might also require an adaptation of the skills and knowledge needed from students in these specific contexts. Finally, our study touches upon many different literatures, from the role of the university and pedagogy to the development of (adaptive) expertise and professional learning. These literatures provide valuable lenses to look at our case studies from different fields and angles. Though we have lightly touched upon a selection of these fields, a more in-depth approach is left open for future research.

Footnotes

Acknowledgements

Erna Ruijer received funding from the European Union (grant agreement ID: 101079227). For more information please see:

Notes

Author Biography

Erna Ruijer is an associate professor at the Utrecht University School of Governance. Her research interests include government transparency, open government data, and social equity and inclusion in the digital society.

Matthieu Brinkhuis is an associate professor at the Department of Information and Computing Sciences, at Utrecht University. He is interested in education and educational scholarship, including developing data science programs for public professionals. His research interests include psychometrics and measurement in computer science applications.

Wolter Hassink is professor of Applied Econometrics at the Utrecht University School of Economics at Utrecht University. His applied work is on labor and housing economics in particular, but he is also interested in the development of big data infrastructures.

Appendix A: List of respondents

School of Governance	- Chair Public Administration - Chair Public Management - Chair Public Innovation - Chair Society Oriented Higher Education - Professor Old and New Media in an Ageing society - Director Undergraduate program - Director Master program
School of Law	- Chair Fundamental Rights - Chair Copyright Law and Media and Communications Law - Director Undergraduate program - Director Master program
School of Economics	- Director Undergraduate program - Chair Marketing and Business development - Project manager research and IT
Applied Data Science	- Chair statistics
Education department	- Manager education innovation programs - Head of Educational Affairs
External stakeholders	- Associate Director Company - Manager at Company

Appendix B – Structured questions to the respondents

1. This advice concerns which facets of the digital society should be included in education. What do you think of when you think of the digital society?

2. What does the digital society require of students? Can you explain this?

3. Which skills are important in this regard?

4. What does a digital society require of the final objectives of education? Can you explain this?

5. Which social themes are important in the context of the digital society that should be included in the educational offer?

6. Is the current educational curriculum prepared for the digital society? Why/why not. What is (still) needed? (new subjects? Such as… why this?)

7. What is already being done within the department in the field of digital society or skills? (programs, subjects, etc.).

8. Do the current teachers have the expertise to respond to the digital society? Why/why not? Which competencies does this require of employees? Do they receive training in this? Are some teachers already working on this? What? What is the relationship with the research and the expertise of the teachers? What steps should be taken to work on the expertise of lecturers in the field of digital technologies?

9. What facilities are currently available in this context? Is this sufficient? What is needed?

10. Has research been conducted within the department among potential employers such as law firms/companies/governments? If so, what was the outcome; what do they expect from students in this area?

11. Any other comments/suggestions on this theme?

References

Abiteboul

Stoyanovich

(2019). Transparency, fairness, data protection, neutrality: Data management challenges in the face of new regulation. Journal of Data and Information Quality (JDIQ), 11(3), 1–9. https://doi.org/10.1145/3310231

Aïmeur

Amri

Brassard

(2023). Fake news, disinformation and misinformation in social media: A review. Social Network Analysis and Mining, 13(1), 30. https://doi.org/10.1007/s13278-023-01028-5

Angrist

J. D.

Pischke

J. S.

(2009). Mostly harmless econometrics: An empiricist's companion. Princeton University Press.

Audrin

(2022). Key factors in digital literacy in learning and education: A systematic literature review using text mining. Education and Information Technologies, 27(6), 7395–7419. https://doi.org/10.1007/s10639-021-10832-5

Bannister

Connolly

(2012). Defining e-governance. E-Service Journal: A Journal of Electronic Services in the Public and Private Sectors, 8(2), 3–25.

Barnett

(2011). Lifewide education: A new and transformative concept for higher education. Learning for a Complex World: A Lifewide Concept of Learning, Education and Personal Development, 22–38. http://lifewidelearningconference.pbworks.com

Bason

Austin

R. D.

(2022). Design in the public sector: Toward a human-centred model of public governance. Public Management Review, 24(11), 1727–1757. https://doi.org/10.1080/14719037.2021.1919186

Biesta

(2015). What is education for? On good education, teacher judgement, and educational professionalism. European Journal of Education, 50(1), 75–87. https://doi.org/10.1111/ejed.12109

Bilan

Mishchuk

Samoliuk

(2023). Digital skills of civil servants: Assessing readiness for successful interaction in e-society. Acta Polytechnica Hungarica, 20(3), 155–174. https://doi.org/10.12700/APH.20.3.2023.3.10

10.

Blaj-Ward

Winter

(2019). Engaging students as digital citizens. Higher Education Research & Development, 38(5), 879–892. https://doi.org/10.1080/07294360.2019.1607829

11.

Brown

(2019). Change by Design. How design thinking transforms organizations and inspires innovation. Harper Collins Publishers.

12.

Burnette

M. H.

Williams

S. C.

Imker

H. J.

(2016). From action plan: Successful data management plan implementation in a multidisciplinary project. Journal of eScience librarianship, 5(1), 1–12. https://doi.org/10.7191/jeslib.2016.1101

13.

Chiu

T. K.

(2023). The impact of generative AI (GenAI) on practices, policies, and research direction in education: A case of ChatGPT and midjourney. Interactive Learning Environments, 32(10), 6187–6203.

14.

Chute

C. G.

Ullman-Cullere

Wood

G. M.

Lin

S. M.

Pathak

(2013). Some experiences and opportunities for big data in translational research. Genetics in Medicine, 15(10), 802–809. https://doi.org/10.1038/gim.2013.121

15.

Cordella

Gualdi

van de Laar

(2024). Digital skills within the public sector: A missing link to achieve the sustainable development goals (SDGs). Information Polity, 29(1), 13–33.

16.

Cronholm

(2021). Lifelong learning: Principles for designing university education. Journal of Information Technology Education: Research, 20, 35–60. https://doi.org/10.28945/4686

17.

Daneshpour

Kwegyir-Afful

(2021). Analysing transdisciplinary education: A scoping review. Science & Education, 31(4), 1047–1074. https://doi.org/10.1007/s11191-021-00277-0

18.

Demchenko

Wiktorski

Cuadrado Gallego

Brewer

(2019, September). EDISON data science framework (EDSF) extension to address transversal skills required by emerging industry 4.0 transformation. In 2019 15^th international conference on eScience (eScience) (pp. 553–559). https://doi.org/10.1109/escience.2019.00076

19.

Dingelstad

Borst

R. T.

Meijer

(2022). Hybrid data competencies for municipal civil servants: An empirical analysis of the required competencies for data-driven decision-making. Public Personnel Management, 51(4), 458–490. https://doi.org/10.1177/00910260221111744

20.

Donoghue

Voytek

Ellis

S. E.

(2021). Teaching creative and practical data science at scale. Journal of Statistics and Data Science Education, 29(sup1), S27–S39. https://doi.org/10.1080/10691898.2020.1860725

21.

Dorst

(2011). The core of ‘design thinking’and its application. Design Studies, 32(6), 521–532. https://doi.org/10.1016/j.destud.2011.07.006

22.

Duberry

(2022). Artificial intelligence and democracy: Risks and promises of AI-mediated citizen-government relations, edward elgar: Cheltenham. Information Polity, 28(3), 435–438. https://doi.org/10.3233/ip-239006

23.

Dwivedi

Y. K.

Hughes

Ismagilova

Aarts

Coombs

Crick

Duan

Dwivedi

Edwards

Eirug

Galanos

Ilavarasan

P. V.

Janssen

Jones

Kar

A. K.

Kizgin

Kronemann

Lal

Lucini

, et al. (2021). Artificial intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. International Journal of Information Management, 57, 101994. https://doi.org/10.1016/j.ijinfomgt.2019.08.002

24.

Edelmann

Mergel

Lampoltshammer

(2023). Competences that foster digital transformation of public administrations: An Austrian case study. Administrative Sciences, 13(2), 44. https://doi.org/10.3390/admsci13020044

25.

European Commission Directorate-General for Education Youth, Sports, & Culture. (2019). Key competences for lifelong learning. Publications Office. https://doi.org/doi/10.2766/569540

26.

European Union (2016). General Data Protection Regulation (GDPR, Regulation (EU) 2016/679) of the European Parliament and of the Council of 27 April 2016. European Union.

27.

Graham

J. W.

(2012). Missing data: Analysis and design. Springer Science & Business Media.

28.

Harkavy

(2006). The role of universities in advancing citizenship and social justice in the 21st century. Education, Citizenship and Social Justice, 1(1), 5–37. https://doi.org/10.1177/1746197906060711

29.

Hermus

van Buuren

Bekkers

(2020). Applying design in public administration: A literature review to explore the state of the art. Policy & Politics, 48, 21–48. https://doi.org/10.1332/030557319X15579230420126

30.

Idzi

F. M.

Gomes

R. C.

(2022). Digital governance: Government strategies that impact public services. Global Public Policy and Governance, 2(4), 427–452. https://doi.org/10.1007/s43508-022-00055-w

31.

Kausch-Zongo

Schenk

(2022). General technological competency and usage in public administration education: An evaluation study considering on-the-job trainings and home studies. Smart Cities and Regional Development (SCRD) Journal, 6(1), 55–65.

32.

Leahy

Dolan

(2010, August). History of the European computer driving licence. In IFIP International conference on the history of computing (pp. 134–145). Springer Berlin Heidelberg.

33.

Lopes

A. S.

Sargento

Farto

(2023). Training in digital skills—the perspective of workers in public sector. Sustainability, 15(13), 10577. https://doi.org/10.3390/su151310577

34.

Luna-Reyes

L. F.

(2017). Opportunities and challenges for digital governance in a world of digital participation. Information Polity, 22(2-3), 197–205. https://doi.org/10.3233/IP-170408

35.

Marinič

Válek

(2024). Developing financial literacy by using digital competences. In INTED2024 Proceedings (pp. 7622–7631). IATED.

36.

Meijer

(2025). Design science in public administration: Producing both situational interventions and generic knowledge. Perspectives on Public Management and Governance, 8(1), 1–11. https://doi.org/10.3233/IP-230015

37.

Meijer

Ruijer

(2021). De Code Goed Digitaal Openbaar Bestuur. Retrieved from https://www.rijksoverheid.nl/documenten/rapporten/2021/04/30/code-goed-digitaal-openbaar-bestuur

38.

Nosek

B. A.

Ebersole

C. R.

DeHaven

A. C.

Mellor

D. T.

(2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114

39.

Oberski

D. I.

Domenech

García-Bernardo

Gerbrands

Hassink

Mohammadi

Mustica

Schilpzand

van Witteloostuijn

(2024). Measuring Firm Characteristics with Language Models. mimeo.

40.

OECD. (2023). OECD Skills Outlook 2023: Skills for a Resilient Green and Digital Transition. OECD Publishing. https://doi.org/10.1787/27452f29-en

41.

O’Neil

(2016). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown.

42.

Peeters

Widlak

A. C.

(2023). Administrative exclusion in the infrastructure-level bureaucracy: The case of the Dutch daycare benefit scandal. Public Administration Review, 83(4), 863–877. https://doi.org/10.1111/puar.13615

43.

Ravšelj

Umek

Todorovski

Aristovnik

(2022). A review of digital era governance research in the first two decades: A bibliometric study. Future Internet, 14(5), 126. https://doi.org/10.3390/fi14050126

44.

Redden

(2018). Democratic governance in an age of datafication: Lessons from mapping government discourses and practices. Big Data & Society, 5(2), 205395171880914. https://doi.org/10.1177/2053951718809145

45.

Ree

M. J.

Carretta

T. R.

(2006). The role of measurement error in familiar statistics. Organizational Research Methods, 9(1), 99–112. https://doi.org/10.1177/1094428105283192

46.

Romme

A. G. L.

Meijer

(2020). Applying design science in public policy and administration research. Policy & Politics, 48(1), 149–165.

47.

Rosenshine

(2012). Principles of instruction: Research-based strategies that all teachers should know. American Educator, 36(1), 12.

48.

Ruijer

Porumbescu

Porter

Piotrowski

(2023). Social equity in the data era: A systematic literature review of data-driven public service research. Public Administration Review, 83(2), 316–332. https://doi.org/10.1111/puar.13585

49.

Saurwein

Spencer-Smith

(2020). Combating disinformation on social media: Multilevel governance and distributed accountability in Europe. Digital Journalism, 8(6), 820–841. https://doi.org/10.1080/21670811.2020.1765401

50.

Schäfer

M. T.

(2024). Collaborative Research in the Datafied Society (1 ed.) Schaefer

van Es

Lauriault

(Eds.). Amsterdam University Press. https://doi.org/10.5117/9789463727679

51.

Schmitt

K. R.

Clark

Kinnaird

K. M.

Wertz

R. E.

Sandstede

(2023). Evaluation of EDISON’s data science competency framework through a comparative literature analysis. Foundations of Data Science, 5(2), 177–198. https://doi.org/10.3934/fods.2021031

52.

Schram

(2005). Artificiality: The tension between internal and external validity in economic experiments. Journal of Economic Methodology, 12(2), 225–237. https://doi.org/10.1080/13501780500086081

53.

Shamoo

A. E.

Resnik

D. B.

(2009). Responsible conduct of research. Oxford University Press.

54.

Tam

S. M.

Kim

J. K.

(2018). Big data ethics and selection-bias: An official statistician’s perspective. Statistical Journal of the IAOS, 34(4), 577–588. https://doi.org/10.3233/SJI-170395

55.

van Deursen

A. J. A. M.

Helsper

E. J.

(2020). Digitale vaardigheden: een onderzoeks- en beleidsagenda. Centrum voor digitale inclusie, Universiteit Twente.

56.

van Deursen

A. J. A. M.

van Dijk

J. A.

(2009). Improving digital skills for the use of online public information and services. Government Information Quarterly, 26(2), 333–340. https://doi.org/10.1016/j.giq.2008.11.002

57.

van Hout

M. A.

Braams

R. B.

Meijer

A. J.

(2024). Designing an instrument for scaling public sector innovations. Science and Public Policy, 51(4), 654–668. https://doi.org/10.1093/scipol/scae007

58.

van Laar

van Deursen

A. J.

van Dijk

J. A.

De Haan

(2017). The relation between 21st-century skills and digital skills: A systematic literature review. Computers in Human Behavior, 72, 577–588. https://doi.org/10.1016/j.chb.2017.03.010

59.

VSNU Association of Universities of the Netherlands (2017). Digital society research agenda. Leading the way through cooperation in a digital society. https://www.thedigitalsociety.info/wpcontent/uploads/2018/04/VSNU-Digital-Society-Research-Agenda.pdf

60.

Vuorikari

Jerzak

Karpinski

Pokropek

Tudek

(2022a). Measuring Digital Skills Across the EU: Digital Skills Indicator 2.0.

61.

Vuorikari

Kluzer

Punie

(2022b). DigComp 2.2: The Digital Competence Framework for Citizens - With new examples of knowledge, skills and attitudes. Publications Office of the European Union. ISBN 978-92-76-48882-8. https://doi.org/10.2760/115376

62.

Vydra

Klievink

(2019). Techno-optimism and policy-pessimism in the public sector big data debate. Government Information Quarterly, 36(4), 101383. https://doi.org/10.1016/j.giq.2019.05.010

63.

Yin

R. K.

(2013). Validity and generalization in future case study evaluations. Evaluation, 19(3), 321–332. https://doi.org/10.1177/1356389013497081

64.

Zhao

Llorente

A. M. P.

Gómez

M. C. S.

(2021). Digital competence in higher education research: A systematic literature review. Computers & Education, 168, 104212. https://doi.org/10.1016/j.compedu.2021.104212