Determinants of 21st-Century Skills and 21st-Century Digital Skills for Workers: A Systematic Literature Review

Abstract

This study brings attention to the determinants of 21st-century skills and 21st-century digital skills. The following skills are investigated: technical, information, communication, collaboration, critical thinking, creativity, and problem-solving skills. To understand differences in the level of these skills among workers, we need to know the factors that determine an individual’s skill level. A systematic literature review was conducted to provide a comprehensive overview of empirical studies measuring skill determinants. The results show that there is strong need for research on determinants of communication and collaboration skills. In a digital context, determinants for creativity and critical thinking are hardly studied. Furthermore, the identified determinants of 21st-century skills studies are limited to personality and psychological determinants, neglecting, for example, social determinants such as social support. Although digital skills studies show more variety, they mostly cover demographic and socioeconomic determinants.

Keywords

21st-century skills digital skills determinants workforce systematic literature review

Introduction

Society has shifted from an economy based on commodities and manual labor to an economy based on knowledge and highly qualified human capital (e.g., Dede, 2010; Jara et al., 2015; Levy & Murnane, 2004). Employees need to be prepared to shift jobs and to be flexible in acquiring skills. Information and communication technology (ICT) is pervasive in the workplace and there is a high demand for ICT-proficient employees. To study differences in digital skills and to develop interventions for skill improvements, in the past years several skill frameworks and definitions have been introduced (e.g., 21st-century skills, digital skills, digital competence, digital literacy, e-skills, internet skills). The approach to the definition of digital skills has shifted from a technical orientation toward a wider perspective that considers content-related or higher-order skills (Claro et al., 2012). A recent systematic literature review of academic literature proposed seven core skills with digital components. The identified 21st-century digital skills are technical, information, communication, collaboration, creativity, critical thinking and problem-solving (Van Laar et al., 2017).

While the importance of these skills to fulfill the demands for workers in the 21st century has been well established, research has identified that comprehensive knowledge about skill assessment is lacking (Voogt & Roblin, 2012). Although various components of digital skills have been described in theory (e.g., Claro et al., 2012; Jara et al., 2015; Siddiq et al., 2017; Van Deursen et al., 2016), it remains unclear which of these skills are influenced by what variables. Moreover, the majority of articles on 21st-century and digital skills describe the skills on conceptual level with little evidence of corresponding data (Siddiq et al., 2016). As such, it is useful to synthesize existing knowledge concerning the factors that cause differences in the level of 21st-century digital skills among workers. We know relatively little about how the range of different skills may vary due to different individual background variables (Helsper & Eynon, 2013). The aim of this study is to provide a state-of-the-art overview of empirical studies on determinants relevant to each type of skill.

A systematic literature review is conducted to synthesize the academic English-language literature concerned with determinants of 21st-century skills and 21st-century digital skills. We expect that determinants of 21st-century skills also play a role in understanding 21st-century digital skills. The review also shows what methods are currently used to measure skills. The overview of determinants and skills indicates relevant factors that encourage or hinder skill development, it can contribute to the development of a parsimonious model to explain differences in mastering these skills, and it identifies the research areas that gained little attention. The results are also useful for designing interventions or justifying skill development policies. Furthermore, the overview will help educational experts who need to equip students with skills that meet the demands of the workforce and employers who are responsible for the development and consolidation of employees’ skill levels. Two research questions are addressed:

Which are significant determinants of 21st-century (digital) skills?

What are the nonsignificant determinants of 21st-century (digital) skills?

Theoretical Background

21st-Century Skills

The skills needed for education and the workplace in the current economy have been labeled 21st-century skills. To define and systemize these skills, a number of initiatives have outlined frameworks. The Partnership for 21st Century Skills (P21, 2007) is a joint government–corporate organization which lists three types of skills: learning skills (creativity and innovation; critical thinking and problem-solving; communication and collaboration), literacy skills (information literacy; media literacy; ICT literacy), and life skills (flexibility and adaptability; initiative and self-direction; social and cross-cultural skills; productivity and accountability; leadership and responsibility). Another initiative is the international research project Assessment and Teaching of 21st Century Skills (ATC21S). The ATC21S project resulted in 10 skills grouped into four categories: ways of thinking (creativity and innovation; critical thinking, problem-solving, and decision-making; learning to learn and metacognition), ways of working (communication; collaboration), tools for working (information literacy; ICT literacy), and living in the world (citizenship; life and career skills; personal and social responsibility) (Binkley et al., 2012). Other public organizations have proposed similar skills. The Organisation for Economic Co-operation and Development (OECD), for example, has categorized 21st-century skills as information, communication, and ethics and social impact (Ananiadou & Claro, 2009). As the exact definition, amount, and subset of incorporated skills differ, efforts have been made to point out the commonalities in the conceptualization of 21st-century skills. Voogt and Roblin (2012) showed that all frameworks include ICT-related skills, collaboration, communication, and social and cultural competences. Besides, most acknowledge creativity, critical thinking, and problem-solving. However, most 21st-century skills frameworks do not go beyond the stage of conceptual definition. Ferrari (2012) mentioned that a plethora of concepts and frameworks have been introduced to highlight the need to handle technology in the digital age.

In the digital skills literature, a number of instruments have been used to measure digital skills (e.g., Hargittai & Hsieh, 2012; Spitzberg, 2006; Van Deursen et al., 2016). Digital skills research acknowledges that both basic skills necessary to use the internet and skills required to comprehend and use online content should be accounted for (Bawden, 2008; Brandtweiner et al., 2010; Eshet-Alkalai & Amichai-Hamburger, 2004; Ferrari, 2012; Gui & Argentin, 2011; Mossberger et al., 2003; Spitzberg, 2006; Van Deursen et al., 2016). From this point of departure, several authors have suggested specific skills, mostly related to information searching. Although this is a valuable addition to the concept, the focus is often on the technicalities of internet use as opposed to a broad range of skills. Only a few approaches provide an integration of digital and 21st-century skills, and therefore, we did not consider the digital part as a precondition for identifying potential determinants of 21st-century digital skills.

On the one hand, 21st-century skills literature emphasizes a broad spectrum of skills, yet do not explicitly integrate digital aspects. The digital skills literature, on the other hand, often does not cover the broad spectrum of skills posed by 21st-century skills studies. Van Laar et al. (2017) conducted a systematic literature to synthesize the relevant academic literature concerned with 21st-century skills and digital skills. Their review resulted in seven core and five contextual skills. As ICT is pervasive in the workplace, the digital component can be integrated into 21st-century skills. This study elaborates on the seven core skills supported by the use of ICT: technical, information management, communication, collaboration, creativity, critical thinking, and problem-solving. These skills are fundamental for performing tasks in a broad range of occupations. Here, we deliberately distinguish between 21st-century skills and digital skills as they are often considered separately. Therefore, we first discuss the core 21st-century skills, and in the next paragraph we systematically add the digital component.

Technical skills

To maintain competitive advantage, employees must be fluent in the skills and languages of ever-changing technologies (Lemke, 2002). For increasing productivity, new technology is developed, and as a consequence, technology is increasingly replacing manual labor and being integrated into most aspects of work (Fuchs, 2010). Workforces need to be capable of continuously adapting to shifting job requirements related to new skill-intensive technologies (Levy & Murnane, 2004). As workplaces have become more complex and supported by ICT, more jobs require technical skills.

Information skills

The abundance of information and data implies that employees in nearly all sectors of the economy must be able to search, evaluate, and organize information, often coming from multiple sources (Silva, 2009). The quick access to a wide range of information sources means that people need to recognize when information is needed and to evaluate the reliability and relative value of information (Marchionini & White, 2007; Starkey, 2011).

Communication skills

Communication skills are vital in the growing service sector and concern the ability to transmit information, ensuring that the meanings are effectively expressed by taking into account the audience and medium (Ananiadou & Claro, 2009; Katz, 2007). One must be able to effectively regulate one’s needs and goals with those of the larger society to successfully navigate in the current social world (Voogt et al., 2013). Because of the interconnectedness of our global economy, employers demand people with communication skills (Levy & Murnane, 2004).

Collaboration skills

Work is becoming more knowledge-based, interdisciplinary, and specialized. The complexity of tasks requires employees to collaborate, as individuals cannot possess all knowledge and skills (Wang, 2010). As a consequence, work is increasingly performed by teams of people with complementary expertise and roles (Dede, 2010; Fraser & Hvolby, 2010). Employees are often dependent on others to accomplish their tasks (Bronstein, 2003). To function interdependently, they need a clear understanding of their own roles and those of their collaborating partners.

Critical thinking skills

Critical thinking broadly refers to making informed choices about obtained information and communication by using sufficient reflection and reasoning. It concerns the ability to think reflectively and judge skillfully, so as to decide what information or communication is relevant in a given context (Gut, 2011). The ability to filter the amount of incoming data to formulate your own point of view is a key 21st-century skill (Dede, 2010). To think critically, employees need knowledge that is central to the particular domain to formulate an independent, well-grounded perspective or opinion (Van de Oudeweetering & Voogt, 2018).

Creativity skills

In addition to being able to process and transmit information, it is necessary to transform information into new knowledge. Previous research has often reasoned that complex problems necessitate creative solutions (Kaufman, 2013). Creativity is related to the production of new and useful ideas on products, services, or processes that are both novel and potentially useful (e.g., Amabile, 1988; Oldham & Cummings, 1996). Because employee creativity is presented as an imperative for long-term organizational success (DiLiello & Houghton, 2008), it arises as a critical skill for organizations to lead or adapt to change.

Problem-solving skills

As the workforce is increasingly confronted with challenging and nonrecurrent problems (Autor et al., 2003), employees need the skills to solve domain-specific problems. Situations that are complex and uncertain and that have no precedent require problem-solving skills (Keane et al., 2016). Problem-solving is often conceptualized as the knowledge and skills that are required to deal effectively with complex nonroutine situations (Funke et al., 2018). Although domain-specific knowledge plays an important role, it is not just prior knowledge. An employee must identify necessary actions, possible gaps, and steps to obtain this information (Rausch & Wuttke, 2016).

Adding the Digital Component: 21st-Century Digital Skills

Technical skills are similar as proposed in the notion of 21st-century skills. These are the skills that workers need to use software or operate a digital device. They are dynamic, involving a continual effort to keep up with new technologies and practices.

Information digital skills

The information abundance caused by ICT requires skills for searching, evaluating, and organizing information in digital environments (Catts & Lau, 2008). Information management includes the ability to (a) clearly define information needs, (b) identify digital information, and (c) select digital information in an effective and efficient way (Ananiadou & Claro, 2009). Once the information has been found, workers need the skills to evaluate how valuable the source and its contents are for the task. Moreover, workers need the skills to store and organize the digital information for easy retrieval. As today’s workers often use multiple digital devices, they need the skills to distribute and maintain information across their digital devices (Song & Ling, 2011).

Communication digital skills

ICT has made it easier to reach a wide audience and communicate at a distance, faster and more ubiquitously. Individuals are able to express themselves, establish relationships, and interact with others at any distance in time and space (Yu et al., 2010). ICT-based communication is regarded as a means of generating social interactions and strengthening social relationships (Hwang, 2011). It is imperative that workers understand how to appropriately and effectively communicate using email, social networking sites, and instant messaging services (Lewin & McNicol, 2015; Wang et al., 2012). People are encouraged to share ideas and opinions within organizations and online forum communities (Lu & Lee, 2012). Workers need the skills to contact other members, maintain those contacts, and share online content and media with their contacts. Online content-sharing activities range from sharing status updates, posts, photos, and videos to writing comments and blogs (Brandtzæg et al., 2010).

Collaboration digital skills

Collaboration processes—managing interdependencies across time to achieve a common goal—are increasingly supported by ICT. ICT is especially useful when teams must share information and make decisions across business and national boundaries (Wang, 2010). With the use of collaboration software as chats (e.g., Skype or WhatsApp), colleagues can instantaneously interchange ideas, information, and experiences. Workers therefore need the skills to connect and collaborate with others beyond a constrained physical environment (Starkey, 2011). Moreover, with the help of content management systems, it is possible to work on the same document at the same time. As such, workers need the skills to work together on shared documents and projects beyond the restrictions of time and place (Lewin & McNicol, 2015). In today’s knowledge society, given the emergence of online collaborative platforms, it is even more important to understand and manage the sharing of information across the organization (Bălău & Utz, 2017).

Critical thinking digital skills

Critical thinking has been identified as being particularly important because in a global online environment people participate and resources are created with various intentions and competences (Starkey, 2011). Online contents must be critically assessed in this age of disinformation and fake news. It is crucial that people understand its nature and source. The focus is on the quality of messages in relation to performance in argumentation. It is crucial for workers to rapidly filter incoming online information and communication and to extract valuable information (Dede, 2010). They must be able to induce critical reflection upon the points that are being discussed online and give sustained arguments that steer the online discussion.

Creative digital skills

ICT can support creativity in multiple ways, including developing ideas and creating or realizing ideas (Loveless, 2007). Digital environments allow workers to assess various design concepts, experiences, and ideas. Furthermore, Web 2.0 technology enables workers to produce and share content in new ways. Online content creation is the use of online spaces to create content including weblogging and photo and video sharing (Brake, 2014). User-generated content creation becomes a common creative practice (Lai & Yang, 2014; Lessig, 2008) in which creativity determines whether the online-generated content is successfully received by the audience.

Problem-solving digital skills

In an information-abundant society, problems can be defined differently, and multiple solutions can be found online. The disadvantage is that the knowledge to solve specific problems can be available online but possibly remains unnoticed because of a lack of an integrated view (David & Foray, 2002). As such, workers need online problem-solving skills to either formulate the problem or find strategies to determine the best solution for a problem. They need the skills to find multiple solutions, solve unfamiliar problems, and transfer knowledge to new situations (Barak, 2018). ICT has become an important medium for accessing and connecting information and, thereby, solving problems.

Determinants of 21st-Century (Digital) Skills

There is widespread consensus among researchers that to use the internet in meaningful ways, users must develop sufficient digital skills (Jenkins et al., 2009; Mossberger et al., 2003). However, regarding how users could develop these skills, different answers are provided. Most initial investigations of the digital divide tended to look at basic demographic and socioeconomic predictors of mere access such as gender, age, education, income, and employment status (DiMaggio et al., 2004). The digital divide approach based on inequalities in internet access has evolved into a divide that includes differences in skills to use the internet (Fuchs, 2009; Selwyn, 2004; Van Dijk, 2005). Several studies have demonstrated that once access to technology is equal, the differences in how effectively it is used relate to economic, cultural, and social variables (Jara et al., 2015).

Most of the literature reviews related to skills research attempted to structure and synthesize conceptualizations instead of evaluating skills assessments in empirical studies (Siddiq et al., 2016). Moreover, existing reviews of digital skills–related assessments mainly focused on unidimensional aspects such as basic internet skills (Litt, 2013). Van Deursen and Van Dijk (2010) showed that similar determinants of internet access and use determine internet skills; however, the relative influence of these determinants depends on the type of skill measured. Given the controversies of definition that are apparent, an extended perspective on assessments of digital skills as a broader concept is missing. The main goal of this systematic literature review is to develop a comprehensive description of state-of-the-art 21st-century (digital) skills assessments by identifying the variety of empirical studies that aim to measure determinants of these skills. This study furthermore establishes an empirical base to indicate the determinants’ impact on these skills and to highlight potential interventions. To present the findings of the review, we categorized the identified determinants adapted from the resources and appropriation theory (De Haan, 2004; Van Dijk, 2005). This theory relates the differences in people’s digital skills to a distribution of resources (temporal, material, mental/motivational, social, and cultural) that, in turn, are explained by personal categories and positions in society. Here, we divided personal and positional categorical inequalities into demographic, socioeconomic, and personality/psychological determinants. Demographic determinants cover concepts such as age, gender, and race/ethnicity, whereas the personality and psychological determinants refer to a person’s traits and intelligence. Socioeconomic determinants include positional categories such as education, income, and labor position. Temporal determinants mean having the time to use digital media. Material determinants concern a person’s possessions. Mental and motivational determinants refer to a person’s learning style, motivation, and skills (as they can also be a determinant of other skills). Social determinants concern having a social network to assist in using digital media. Finally, cultural determinants cover variables such as religion, language, and attitude toward other cultures.

Method

Systematic Literature Review

A systematic literature review attempts to collate all relevant evidence that fits prespecified eligibility criteria to answer a specific research question (Shamseer et al., 2015). It uses an explicit, reproducible methodology to minimize bias in the identification, selection, and summary of studies. This method fits our research purpose because it helps synthesize all academic articles that measure determinants of 21st-century skills and 21st-century digital skills. The review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) approach (Moher et al., 2015). This approach intends to guide the documentation of systematic literature reviews by creating a guideline to improve the transparency, accuracy, and completeness of publications.

Search Terms

The search action was conducted using the Scopus, Web of Science, and PsycINFO databases, which together cover an inclusive range of social science journals. The first search stream included core skills dimensions in agreement with several keywords for determinants. The keywords had to be in the title of the article to specify the search stream. In addition, the keywords skills, competence, and literacy were added. As a result, the first Boolean search action was conducted: (“technical” OR “information literac*” OR “communication competenc*” OR “collaborativ*” OR “teamwork*” OR “creativ*” OR “critical thinking” OR “problem solving”) AND (“associat*” OR “antecedent*” OR “contribut*” OR “determin*” OR “factor*” OR “influenc*” OR “predict*” OR “related” OR “relation*” OR “moderat*”) AND (“skills” OR “competenc*” OR “literac*”)

As the skills mentioned above depart from the multitude of existing concepts, a second search stream included 21st-century skills and digital skills–related terms and keywords for determinants. The keywords had to be in the abstract, title, or keywords of the article. As a result, the second Boolean search action was conducted: (“21st-century skills” OR “twenty-first century skills” OR “e-skills” OR “digital skills” OR “digital competenc*” OR “digital literac*” OR “internet skills” OR “ICT skills” OR “ICT competenc*” OR “ICT literac*”) AND (“associat*” OR “antecedent*” OR “contribut*” OR “determin*” OR “factor*” OR “influenc*” OR “moderat*” OR “predict*” OR “related” OR “relation*”)

Selection Criteria

A number of criteria were specified to select relevant English-language articles. The searches were refined by specifying the following six selection criteria:

Contain skills (technical, information management, communication, collaboration, creativity, critical thinking, and problem-solving) as the dependent variable because the focus is on the factors that influence an individual’s skill level.

Include the impact of determinants of 21st-century (digital) skills at the level of the individual worker. Organizational determinants (e.g., organizational culture and leadership style) fall outside the scope.

Present original quantitative empirical data rather than qualitative data or an overview of previously reported data because the aim is to examine significant determinants. The mixed method is included in our analysis when it provides us with quantitative data on the determinants of 21st-century (digital) skills.

Report directional significant effects (p < .05) rather than correlation effect sizes to provide the strongest empirical support for the determinants.

Involve participants from secondary school age and older because this group represents preparation for working life and the workforce.

Be published in a peer-reviewed journal because such journals are considered the most reliable source of scientific information.

Study Selection

The study selection was performed in three steps. First, the titles of all retrieved articles were screened for eligibility based on the abovementioned inclusion criteria. Second, the abstracts of all initially relevant articles were screened by applying the same six uniform criteria. Third, the full text of all remaining publications was checked for inclusion. For each article deemed relevant, information from the full-text article was extracted. Each potential article was coded in terms of the following: names of the authors, date published, journal, aims, method, dependent variables and their operationalization, independent variables, results, and conclusion. The coding of the articles was performed to ensure that all relevant articles were selected.

PRISMA Flowchart

Given the restrictions of the document type and language, 4,266 articles were identified from the Boolean search actions. Of the 4,266 articles, 1,706 were duplicates, which means that 2,560 different articles were screened. After the title and abstract screening, 339 were read in full text, of which 154 articles met all six inclusion criteria. Figure 1 presents the flowchart for selection. Additional records were not identified through other sources because the references of the included articles did not contribute to the received information. There were seven reasons for excluding a full-text screening: (a) no skills as dependent variable, (b) no full-text available, (c) no independent variable, (d) no quantitative research, (e) no directional effect, (f) only organizational determinants, and (g) no participants in the age category of secondary school or older.

Figure 1.

PRIMA flowchart of the literature selection process.

Selection Bias

To verify that the selected articles met the selection criteria, 10% of the articles derived from both search actions in Scopus (n = 209) were independently coded by a second coder. Publication bias in a systematic literature review occurs mostly during the selection process, and a transparent selection process is necessary to minimize such bias. A second coder performed both search actions and followed the study selection steps of title, abstract, and full-text evaluation according to a predefined instruction. The interrater reliability was .90, which shows a good agreement between the two coders. Any differences of opinion about whether or not to include an article were resolved through discussion until a consensus was reached.

Results

Categorization of Selected Studies

The number of studies measuring determinants of 21st-century skills (Table 1) and 21st-century digital skills (Table 2) were categorized by type of skill and method. The categorizations of skills were based on the operationalization used. If a study measured multiple skills combined as the dependent variable, we placed the determinants into all corresponding skills categories. It is important to note that technical, information, and communication were frequently combined as a dependent 21st-century digital skill. For 21st-century skills studies, creativity (33.7%), critical thinking (22.9%), and problem-solving (18.1%) were the most investigated skills. Technical (2.4%) and information skills (2.4%) were underrepresented. For 21st-century digital skills studies, technical (38.3%) and information skills (29.7%) were the most investigated skills, whereas critical thinking (3.9%) and creativity (2.3%) were underrepresented. Both measured determinants of problem-solving relatively frequently, whereas communication and collaboration were underreported. Furthermore, surveys were the most commonly employed method. In addition, for 21st-century skills, creativity was relatively often measured in experiments and problem-solving in performance tests. For 21st-century digital skills, technical, information, and problem-solving skills were relatively often measured in performance tests.

Table 1.

The 21st-Century Skills Studies Categorized by Type of Skill and Method.

Skills	Method
Skills	Survey	Performance test	Experiment	Mixed method	Total
Technical	2 (3.4%)	0 (0%)	0 (0%)	0 (0%)	2 (2.4%)
Information	2 (3.4%)	0 (0%)	0 (0%)	0 (0%)	2 (2.4%)
Communication	9 (15.3%)	0 (0%)	0 (0%)	1 (33.3%)	10 (12.0%)
Collaboration	6 (10.2%)	1 (11.1%)	0 (0%)	0 (0%)	7 (8.4%)
Critical thinking	15 (25.4%)	1 (11.1%)	2 (16.7%)	1 (33.3%)	19 (22.9%)
Creativity	19 (32.2%)	2 (22.2%)	7 (58.3%)	0 (0%)	28 (33.7%)
Problem-solving	6 (10.2%)	5 (55.6%)	3 (25.0%)	1 (33.3%)	15 (18.1%)
Total	59	9	12	3	83

Table 2.

The 21st-Century Digital Skills Studies Categorized by Type of Skill and Method.

Skills	Method
Skills	Survey	Performance test	Experiment	Mixed method	Total
Technical	35 (44.3%)	10 (25.6%)	1 (20.0%)	3 (60.0%)	49 (38.3%)
Information	24 (30.4%)	10 (25.6%)	3 (60.0%)	1 (20.0%)	38 (29.7%)
Communication	9 (11.4%)	3 (7.7%)	0 (0%)	1 (20.0%)	13 10.2%)
Collaboration	1 (1.3%)	1 (2.6%)	0 (0%)	0 (0%)	2 (1.6%)
Critical thinking	1 (1.3%)	3 (7.7%)	1 (20.0%)	0 (0%)	5 (3.9%)
Creativity	2 (2.5%)	1 (2.6%)	0 (0%)	0 (0%)	3 (2.3%)
Problem-solving	7 (8.9%)	11 (28.2%)	0 (0%)	0 (0%)	18 (14.1%)
Total	79	39	5	5	128

Significant Determinants of 21st-Century Skills and 21st-Century Digital Skills

The results of the review are presented in schemes matching the seven core skills and eight groups of determinants. An overview of all significant determinants is displayed in Supplemental Appendix A. Table 3 shows the number of significant determinants for 21st-century skills and Table 4 the number of significant determinants for 21st-century digital skills. A list of all determinants per skill is displayed in Supplemental Appendix B. For 21st-century skills studies, creativity (n = 82), critical thinking (n = 38), and problem-solving (n = 30) reported a large number of determinants. They clearly show the determinants in a particular direction. Personality and psychological determinants were mainly covered in studies that examined problem-solving (66.7%), critical thinking (57.9%), and creativity (50.0%). Examples were the Big Five dimensions of personality (Openness to Experience, Extraversion, Conscientiousness, Agreeableness, and Neuroticism). Moreover, mental and motivational determinants were well represented for studies measuring creativity (22.0%). Two studies measuring ethical decision-making and time management styles were mainly responsible for this number. Although studies examining communication and collaboration skills were underreported, they show the determinants in a particular direction. Communication mainly addressed social (25.0%), mental/motivational (25.0%), and cultural determinants (20.8%). One study measuring sensitivity to the partner while communicating was mainly responsible for the social determinants. Examples of cultural determinants were cultural capital and intercultural sensitivity. Collaboration focused on personality and psychological determinants (63.6%) such as personality traits, emotional intelligence, and thinking styles.

Table 3.

Significant Determinants of 21st-Century Skills.

Determinants	Skills
Determinants	Technical	Information	Communication	Collaboration	Critical thinking	Creativity	Problem-solving	Total
Demographic	0 (0%)	0 (0%)	1 (4.2%)	0 (0%)	6 (15.8%)	5 (6.1%)	0 (0%)	12 (5.7%)
Socioeconomic	1 (16.7%)	1 (12.5%)	3 (12.5%)	1 (4.5%)	5 (13.2%)	5 (6.1%)	2 (6.7%)	18 (8.6%)
Personality/psychological	1 (16.7%)	1 (12.5%)	2 (8.3%)	14 (63.6%)	22 (57.9%)	41 (50.0%)	20 (66.7%)	101 (48.1%)
Temporal	0 (0%)	0 (0%)	1 (4.2%)	0 (0%)	0 (0%)	1 (1.2%)	0 (0%)	2 (1.0%)
Material	0 (0%)	0 (0%)	0 (0%)	0 (0%)	1 (2.6%)	7 (8.5%)	2 (6.7%)	10 (4.8%)
Mental/motivational	4 (66.7%)	4 (50.0%)	6 (25.0%)	6 (27.3%)	0 (0%)	18 (22.0%)	5 (16.7%)	43 (20.5%)
Social	0 (0 %)	0 (0%)	6 (25.0%)	1 (4.5%)	3 (7.9%)	3 (3.7%)	1 (3.3%)	14 (6.7%)
Cultural	0 (0%)	2 (25.0%)	5 (20.8%)	0 (0.0%)	1 (2.6%)	2 (2.4%)	0 (0%)	10 (4.8%)
Total	6	8	24	22	38	82	30	210

Table 4.

Significant Determinants of 21st-Century Digital Skills.

Determinants	Skills
Determinants	Technical	Information	Communication	Collaboration	Critical thinking	Creativity	Problem-solving	Total
Demographic	44 (22.3%)	21 (15.1%)	7 (10.6%)	0 (0%)	6 (37.5%)	3 (33.3%)	8 (11.9%)	89 (17.6%)
Socioeconomic	46 (23.4%)	23 (16.5%)	10 (15.2%)	3 (25.0%)	2 (12.5%)	1 (11.1%)	15 (22.4%)	100 (19.8%)
Personality/psychological	9 (4.6%)	17 (12.2%)	11 (16.7%)	2 (16.7%)	1 (6.3%)	1 (11.1%)	10 (14.9%)	51 (10.1%)
Temporal	28 (14.2%)	22 (15.8%)	12 (18.2%)	0 (0%)	0 (0%)	0 (0%)	4 (6.0%)	66 (13.0%)
Material	11 (5.6%)	7 (5.0%)	3 (4.5%)	1 (8.3%)	1 (6.3%)	0 (0%)	6 (9.0%)	29 (5.7%)
Mental/motivational	44 (22.3%)	24 (17.3%)	6 (9.1%)	4 (33.3%)	2 (12.5%)	2 (22.2%)	10 (14.9%)	92 (18.2%)
Social	14 (7.1%)	9 (6.5%)	6 (9.1%)	2 (16.7%)	3 (18.8%)	2 (22.2%)	2 (3.0%)	38 (7.5%)
Cultural	1 (0.5%)	16 (11.5%)	11 (16.7%)	0 (0%)	1 (6.3%)	0 (0%)	12 (17.9%)	41 (8.1%)
Total	197	139	66	12	16	9	67	506

For 21st-century digital skills studies, technical (n = 197), information (n = 139), problem-solving (n = 67), and communication (n = 66) reported a large number of determinants. Demographic and socioeconomic determinants were well represented in studies measuring these skills. Age, gender, and educational level were frequently reported as significant. Personality and psychological determinants were well represented for communication (16.7%), problem-solving (14.9%), and information skills (12.2%). Examples of these determinants include ICT self-efficacy and academic achievements. Temporal determinants such as ICT use and ICT experience accounted for the largest share in studies examining information (15.8%) and technical skills (14.2%). Material determinants such as ICT access were mainly covered in studies examining problem-solving (9.0%), technical (5.6%), and information skills (5.0%). Mental and motivational determinants were again the most prevalent in studies that examined technical (22.3%), information (17.3%), and problem-solving skills (14.9%). ICT training was frequently a significant mental/motivational determinant. Social determinants were often reported as significant for technical (7.1%) and information skills (6.5%). Social support was frequently a significant social determinant. Cultural determinants were often reported as significant for problem-solving (17.9%), communication (16.7%), and information skills (11.5%). However, it is important to note that one author measured cultural determinants in multiple studies and therefore is primarily responsible for this number.

Nonsignificant Determinants of 21st-Century Skills and 21st-Century Digital Skills

Tables 5 and 6 show the nonsignificant determinants. A low number of significant determinants have to do with either a lack of studies examining this determinant or the fact that they turned out to be nonsignificant. For 21st-century skills studies, collaboration (81.8%), creativity (52.9%), critical thinking (51.9%), and problem-solving (19.1%) reported a large number of nonsignificant personality and psychological determinants. However, except for collaboration skills, personality and psychological determinants turned out more frequently to be significant. For creativity studies, material determinants such as available resources appeared as nonsignificant.

Table 5.

Nonsignificant Determinants of 21st-Century Skills.

Determinants	Skills
Determinants	Technical	Information	Communication	Collaboration	Critical Thinking	Creativity	Problem-solving	Total
Demographic	1 (14.3%)	2 (40.0%)	3 (42.9%)	1 (4.5%)	7 (25.9%)	3 (8.3%)	4 (18.2%)	21 (16.7%)
Socioeconomic	4 (57.1%)	2 (40.0%)	3 (42.9%)	2 (9.1%)	2 (7.4%)	4 (11.1%)	4 (18.2%)	21 (16.7%)
Personality/psychological	0 (0%)	0 (0%)	0 (0%)	18 (81.8%)	14 (51.9%)	19 (52.8%)	13 (19.1%)	64 (50.8%)
Temporal	0 (0%)	0 (0%)	0 (0%)	0 (0%)	0 (0%)	0 (0%)	0 (0%)	0 (0%)
Material	0 (0%)	0 (0%)	0 (0%)	0 (0%)	0 (0%)	5 (13.9%)	0 (0%)	5 (4.0%)
Mental/motivational	2 (28.6%)	1 (20.0%)	0 (0%)	1 (4.5%)	3 (11.1%)	3 (8.3%)	1 (4.5%)	11 (8.7%)
Social	0 (0%)	0 (0%)	0 (0%)	0 (0%)	0 (0%)	2 (5.6%)	0 (0%)	2 (1.6%)
Cultural	0 (0%)	0 (0%)	1 (14.3%)	0 (0%)	1 (3.7%)	0 (0%)	0 (0%)	2 (1.6%)
Total	7	5	7	22	27	36	22	126

Table 6.

Nonsignificant Determinants of 21st-Century Digital Skills.

Determinants	Skills
Determinants	Technical	Information	Communication	Collaboration	Critical thinking	Creativity	Problem-solving	Total
Demographic	16 (16.3%)	16 (21.1%)	4 (25.0%)	1 (14.3%)	1 (7.7%)	0 (0%)	11 (26.2%)	49 (19.3%)
Socioeconomic	25 (25.5%)	10 (13.2%)	3 (18.8%)	2 (28.6%)	2 (15.4%)	0 (0%)	5 (11.9%)	47 (18.5%)
Personality/psychological	5 (5.1%)	4 (5.3%)	3 (18.8%)	1 (14.3%)	2 (15.4%)	0 (0%)	3 (7.1%)	18 (7.1%)
Temporal	9 (9.2%)	16 (21.1%)	1 (6.3%)	0 (0%)	1 (7.7%)	0 (0%)	8 (19.0%)	35 (13.8%)
Material	14 (14.3%)	10 (13.2%)	1 (6.3%)	0 (0%)	1 (7.7%)	0 (0%)	5 (11.9%)	31 (12.2%)
Mental/motivational	14 (14.3%)	13 (17.1%)	1 (6.3%)	2 (28.6%)	2 (15.4%)	1 (50.0%)	4 (9.5%)	37 (14.6%)
Social	15 (15.3%)	7 (9.2%)	3 (18.8%)	1 (14.3%)	4 (30.8%)	1 (50.0%)	5 (11.9%)	36 (14.2%)
Cultural	0 (0%)	0 (0%)	0 (0%)	0 (0%)	0 (0%)	0 (0%)	1 (2.4%)	1 (0.4%)
Total	98	76	16	7	13	2	42	254

For 21st-century digital skills studies, although demographic, socioeconomic, temporal, material, and mental/motivational determinants show a large number of nonsignificant determinants, they appeared more frequently as significant for technical and information skills. By contrast, technical and information skills studies were inconclusive about the effect of social determinants (n = 14 compared with n = 15 and n = 9 compared with n = 7). For problem-solving skills, demographic determinants (e.g., gender and age) and temporal determinants (e.g., frequency of ICT use) appeared more frequently as nonsignificant.

Discussion

This systematic literature review provides a state-of-the art overview of empirical studies on determinants of 21st-century (digital) skills. The results reveal which skills and type of determinant are relevant for future research. Revealing the research gaps can contribute to a continued focus on developing and monitoring the variety of 21st-century skills people should attain in the digital context.

Main Findings

A first conclusion is that the determinants for creativity and critical thinking are less studied in a digital context. One conceivable explanation for why some skills are frequently digitally considered is that creativity and critical thinking can more easily be separated from digital contexts in comparison with technical and information management skills. Nevertheless, academic thinking shows each 21st-century skill has a digital variant. Moreover, 21st-century skills and 21st-century digital skills studies measured the determinants of problem-solving skills relatively frequently, whereas collaboration and communication skills studies were underreported. Similarly, Siddiq and colleagues (2016) showed that a large majority of existing tests assess students’ digital information and technical skills, whereas other aspects of ICT literacy are not equally covered. It is therefore necessary to understand how to measure, for example, problem-solving, communication, and collaboration with ICT. Although communication and collaboration are viewed as essential (Ahonen & Kinnunen, 2015), they are scarcely covered in academic research. One possible explanation of why some skills are studied more frequently than others is that technology and society are mutually shaping (MacKenzie & Wajcman, 1985). While technologically deterministic viewpoints consider that society is shaped by technology, this technical viewpoint is avoided by social construction theorists. From their point of view, social aspects are more important than technological characteristics for determining how a technology is used. In this respect, instead of only focusing on technical skills, the so-called content-related skills (such as communication and collaboration) become more important, as they strongly influence the outcomes of how the internet is used and thus the outcomes of work performance. As a consequence of the mutual shaping of technology and society, most studies concentrate on technical skills first. Another possible reason for the lack of attention is that content-related skills are more difficult to observe, quantify, or measure (Cobo, 2013; Silva, 2009). Furthermore, the 21st-century digital skills concept is broad, making it difficult to develop one test that covers all (Aesaert & Van Braak, 2015).

A second conclusion is that for 21st-century skills studies, the most frequently reported significant determinants are personality and psychological factors. Personality and psychological determinants are often reported as significant in studies that examined creativity, critical thinking, problem-solving, and collaboration skills. The determinants of 21st-century skills show less variety than those of 21st-century digital skills. The determinant groups are better represented in studies measuring the digital aspect of technical, information, communication, and problem-solving skills. In particular, demographic, socioeconomic, temporal, and mental/motivational determinants are frequently reported as significant. These results show that digital skills studies take into account a variety of determinants. Learning styles and sources of help are examples of determinants that were part of 21st-century digital skills studies, in contrast to 21st-century skills studies. In addition, digital-related determinants such as ICT experience, ICT use, and ICT training are mostly covered in 21st-century digital skills studies. Remarkably, there are many studies around personality in relation to 21st-century skills in contrast to digital skills literature where sociological explanations are more prominent. The digital divide generally implies differences in access based on socioeconomic divisions (Van Deursen & Van Dijk, 2015).

A third conclusion is that, except for collaboration skills, personality and psychological determinants more frequently turn out to be significant than nonsignificant for creativity, critical thinking, and problem-solving skills. Personal determinants are proven to be important for these 21st-century skills. For 21st-century digital skills studies, the same holds true for demographic, socioeconomic, temporal, and mental/motivational determinants of technical, information, and communication skills. By contrast, for problem-solving skills, demographic and temporal determinants appear more frequently as nonsignificant. Determinants such as gender, age, ICT experience, and ICT use more frequently turn out to be nonsignificant than significant. Because of the number of different determinants and the scattered overview that they provide, it is difficult to note the factors that can possibly be ignored by research for specific skills. Overall, factors such as age, gender, socioeconomic status, personality traits, and intelligence are often investigated but are difficult to account for in skill policies—these are more permanent and belong to an individual’s position in society. This is in line with a previous systematic literature review which showed that digital skills studies are largely limited to demographic and socioeconomic determinants (Hargittai, 2010; Scheerder et al., 2017). To conclude, surveys are the most commonly employed method to measure skill determinants. Technical, information, and problem-solving skills are relatively frequently measured in performance tests.

Limitations

Although peer-reviewed journal articles are considered to be validated knowledge likely to have high scientific impact (Keupp et al., 2012), this review might have excluded other relevant work as we did not consider books or conference papers. Besides, this review was limited by the choices that were made in the search streams. Within the first search stream, the keywords had to be in the title. Although this choice was needed to specify the search results, it means that potential articles mentioning the terms only in the abstract or full text were excluded. In addition, the term skills, competence, or literacy was inserted to specify the search stream. Consequently, studies measuring the determinants of skills without mentioning these keywords in the title or abstract were excluded. These design choices were based on the balance between sensitivity, finding as many articles as possible that may be relevant, and specificity, ensuring that those articles are relevant. Because of the heterogeneity of the data and study designs reviewed, we did not conduct a meta-analysis but aimed to present an overview of past empirical evidence concerning skill determinants. Furthermore, we made the decision to focus on the core skills. As a result, for example, ethics and responsibility were excluded even if they could certainly be valuable. Moreover, we decided to include articles that measured multiple skills combined. It must be observed that technical, information, and communication skills were often part of this combined dependent variable. Furthermore, we had to perform a categorization to make the number of determinants manageable. Although a categorization is arbitrary, we made it transparent by providing an overview of all significant determinants. Another limitation of our review is that the conditions within the organization were not considered. Determinants of skills at the level of individual workers are more often linked to a person, and therefore, a separate search stream would be necessary to synthesize the organizational determinants. Finally, we used significance as an indicator to select relevant publications. Critics call for a broader approach because p-values are commonly misused and misinterpreted (Wasserstein & Lazar, 2016). The validity of scientific conclusions, including their reproducibility, depends on more than statistical methods. Nevertheless, as p-values are widely used and easily recognized in papers, we decided, though aware of these objections, to use them.

Future Research Agenda

First, we can conclude that the research on the whole range of 21st-century digital skills requires a thorough investigation to define policies for the development of these important skills; in particular, studies that focus on determinants of creativity, critical thinking, collaboration, and communication are underreported. To do so, it would be interesting to look at the 21st-century skills research because here, except for communication skills, determinants are more frequently measured. Moreover, identifying the relevant factors influencing the differences in digital skills can be considered the background knowledge for explaining these differences. To understand these differences, it is necessary to build an explanatory model. Such a model needs to be parsimoniousness, which requires not only identifying but also selecting relevant aspects based on theoretical insights that contribute to the consistency of the model and to the specification of the relationships between these aspects.

Furthermore, although concerns about the lack of performance tests are increasingly addressed for technical, information, and problem-solving skills, surveys are still the most commonly employed method. Although self-report questionnaires have advantages such as the ability to present a large number of questions on a wide range of skills in a short period of time, the method has problems of validity (Hargittai, 2005). Many of the existing studies gather data based on people’s own perceptions or estimations of their skills. It is likely that people overrate their own skill levels because they link the concept of 21st-century digital skills to basic technical skills instead of the content-related skills (Talja, 2005). To gain insight into an individual’s actual skill level, there is a strong need for a performance-based measurement for each type of skill.

Although numerous studies have been conducted to identify determinants, the main emphasis is on positional determinants, which an individual cannot manage. Future research could focus on determinants that can be influenced by the users of the technologies themselves as well as policy makers, educators, and managers in organizations. The research concerning material, temporal, mental/motivational, social, and cultural determinants might identify factors that can be altered. A stronger focus on these determinants in future investigations might help define better focused policies on how to improve individuals’ skill levels. A variety of studies have highlighted the importance of participation in guided ICT training and informal social networks (e.g., Brandtweiner et al., 2010; Helsper & Eynon, 2013). In addition, it could be important to look into the qualitative aspects of support and training. Future research could measure a person’s satisfaction level after asking for help and the reasons for not attending ICT training.

Finally, future research could focus on the consequences of the differences in people’s skill levels. Several scholars have argued that digital divides should be approached more comprehensively, in which not only internet access, skills, and use are addressed but also the consequences of internet skills (e.g., Fuchs, 2009; Scheerder et al., 2017; Selwyn, 2004). In the labor market context, it would be interesting to know to what extent skills contribute to the quality of work performance, higher incomes, and chances of employment. In addition, skills are also assumed to be important for its contribution to people’s emancipation, empowerment, and self-fulfillment (Punie, 2007).

Supplemental Material

Supplementary_3a_Sage – Supplemental material for Determinants of 21st-Century Skills and 21st-Century Digital Skills for Workers: A Systematic Literature Review

Supplemental material, Supplementary_3a_Sage for Determinants of 21st-Century Skills and 21st-Century Digital Skills for Workers: A Systematic Literature Review by Ester van Laar, Alexander J. A. M. van Deursen, Jan A. G. M. van Dijk and Jos de Haan in SAGE Open

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by NWO, the national research council of the Netherlands (grant number: 409-15-214).

ORCID iD

Alexander J. A. M. van Deursen

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Ester van Laar is a postdoctoral researcher in the Department of Communication Science at the University of Twente. In her research she combines various quantitative and qualitative research methods to identify and measure digital skills youth and working professionals need to learn in the 21st-century.

Alexander J. A. M. van Deursen is a professor at the University of Twente and chair of the Department of Communication Science. His research focuses on digital inequality in contemporary society. He has published widely on the topic.

Jan A. G. M. van Dijk is an emeritus professor of the University of Twente on a Chair of the Sociology of the Information Society. He has an international reputation since the 1980s about research on the network society, the digital divide and digital democracy.

Jos de Haan works as a senior researcher at the Netherlands Institute for Social Research (SCP). His research focuses on media use and the diffusion, use and consequences of ICTs.

References

Aesaert

Van Braak

(2015). Gender and socioeconomic related differences in performance based ICT competences. Computers & Education, 84, 8–25. https://doi.org/10.1016/j.compedu.2014.12.017

Ahonen

A. K.

Kinnunen

(2015). How do students value the importance of twenty-first century skills? Scandinavian Journal of Educational Research, 59(4), 395–412. https://doi.org/10.1080/00313831.2014.904423

Amabile

T. M.

(1988). A model of creativity and innovation in organizations. Research in Organizational Behavior, 10(1), 123–167.

Ananiadou

Claro

(2009). 21st century skills and competences for new millennium learners in OECD countries (OECD Education Working Papers No. 41). Paris, France: OECD Publishing. https://doi.org/10.1787/218525261154

Autor

D. H.

Levy

Murnane

R. J.

(2003). The skill content of recent technological change: An empirical exploration. The Quarterly Journal of Economics, 118(4), 1279–1333. https://doi.org/10.1162/003355303322552801

Bălău

Utz

(2017). Information sharing as strategic behaviour: The role of information display, social motivation and time pressure. Behaviour & Information Technology, 36(6), 589–605. https://doi.org/10.1080/0144929X.2016.1267263

Barak

(2018). Are digital natives open to change? Examining flexible thinking and resistance to change. Computers & Education, 121, 115–123. https://doi.org/10.1016/j.compedu.2018.01.016

Bawden

(2008). Origins and concepts of digital literacy. In Lankshear

Knobel

(Eds.), Digital literacies: Concepts, policies and practices (pp. 15–32). New York, NY: Peter Lang Publishing.

Binkley

Erstad

Herman

Raizen

Ripley

Miller-Ricci

Rumble

(2012). Defining twenty-first century skills. In Griffin

Care

(Eds.), Assessment and teaching of 21st century skills: Methods and approach (pp. 17–66). Dordrecht, the Netherlands: Springer.

10.

Brake

D. R.

(2014). Are we all online content creators now? Web 2.0 and digital divides. Journal of Computer-Mediated Communication, 19(3), 591–609. https://doi.org/10.1111/jcc4.12042

11.

Brandtweiner

Donat

Kerschbaum

(2010). How to become a sophisticated user: A two-dimensional approach to e-literacy. New Media & Society, 12(5), 813–833. https://doi.org/10.1177/1461444809349577

12.

Brandtzæg

P. B.

Lüders

Skjetne

J. H.

(2010). Too many Facebook “friends”? Content sharing and sociability versus the need for privacy in social network sites. International Journal of Human-Computer Interaction, 26(11–12), 1006–1030. https://doi.org/10.1080/10447318.2010.516719

13.

Bronstein

L. R.

(2003). Index of interdisciplinary collaboration. Social Work Research, 48(3), 297–306. https://doi.org/10.1093/sw/48.3.297

14.

Catts

Lau

(2008). Towards information literacy indicators. Paris, France: UNESCO Publishing.

15.

Claro

Preiss

D. D.

San Martín

Jara

Hinostroza

J. E.

Valenzuela

Cortes

Nussbaum

(2012). Assessment of 21st century ICT skills in Chile: Test design and results from high school level students. Computers & Education, 59(3), 1042–1053. https://doi.org/10.1016/j.compedu.2012.04.004

16.

Cobo

(2013). Mechanisms to identify and study the demand for innovation skills in world-renowned organizations. On the Horizon, 21(2), 96–106. https://doi.org/10.1108/10748121311322996

17.

David

P. A.

Foray

(2002). An introduction to the economy of the knowledge society. International Social Science Journal, 54(171), 9–23. https://doi.org/10.1111/1468-2451.00355

18.

Dede

(2010). Comparing frameworks for 21st century skills. In Bellanca

Brandt

(Eds.), 21st century skills (pp. 51–76). Bloomington, IN: Solution Tree Press.

19.

De Haan

(2004). A multifaceted dynamic model of the digital divide. IT & Society, 1(7), 66–88.

20.

DiLiello

T. C.

Houghton

J. D.

(2008). Creative potential and practiced creativity: Identifying untapped creativity in organizations. Creativity and Innovation Management, 17(1), 37–46. https://doi.org/10.1111/j.1467-8691.2007.00464.x

21.

DiMaggio

Hargittai

Celeste

Shafer

(2004). From unequal access to differentiated use: A literature review and agenda for research on digital inequality. In Neckerman

(Ed.), Social inequality (pp. 355–400). New York, NY: Russell Sage Foundation.

22.

Eshet-Alkalai

Y. E.

Amichai-Hamburger

(2004). Experiments in digital literacy. CyberPsychology & Behavior, 7(4), 421–429. https://doi.org/10.1089/cpb.2004.7.421

23.

Ferrari

(2012). Digital competence in practice: An analysis of frameworks. Seville, Spain: Joint Research Centre, Institute for Prospective Technological Studies. https://doi.org/10.2791/82116

24.

Fraser

Hvolby

(2010). Effective teamworking: Can functional flexibility act as an enhancing factor? An Australian case study. Team Performance Management: An International Journal, 16(1–2), 74–94. https://doi.org/10.1108/13527591011028933

25.

Fuchs

(2009). The role of income inequality in a multivariate cross-national analysis of the digital divide. Social Science Computer Review, 27(1), 41–58. https://doi.org/10.1177/0894439308321628

26.

Fuchs

(2010). Grounding critical communication studies: An inquiry into the communication theory of Karl Marx. Journal of Communication Inquiry, 34(1), 15–41. https://doi.org/10.1177/0196859909338409

27.

Funke

Fischer

Holt

D. V.

(2018). Competencies for complexity: Problem solving in the 21st century. In Care

Griffin

Wilson

(Eds.), Assessment and teaching of 21st century skills: Research and applications (pp. 41–53). Dordrecht, the Netherlands: Springer.

28.

Gui

Argentin

(2011). Digital skills of Internet natives: Different forms of digital literacy in a random sample of northern Italian high school students. New Media & Society, 13(6), 963–980. https://doi.org/10.1177/1461444810389751

29.

Gut

D. M.

(2011). Integrating 21st century skills into the curriculum. In Wan

Gut

D. M.

(Eds.), Bringing schools into the 21st century (pp. 137–157). Dordrecht, the Netherlands: Springer.

30.

Hargittai

(2005). Survey measures of web-oriented digital literacy. Social Science Computer Review, 23(3), 371–379. https://doi.org/10.1177/0894439305275911

31.

Hargittai

(2010). Digital na(t)ives? Variation in internet skills and uses among members of the “net generation.” Sociological Inquiry, 80(1), 92–113. https://doi.org/10.1111/j.1475-682X.2009.00317.x

32.

Hargittai

Hsieh

Y. P.

(2012). Succinct survey measures of web-use skills. Social Science Computer Review, 30(1), 95–107. https://doi.org/10.1177/0894439310397146

33.

Helsper

E. J.

Eynon

(2013). Distinct skill pathways to digital engagement. European Journal of Communication, 28(6), 696–713. https://doi.org/10.1177/0267323113499113

34.

Hwang

(2011). Is communication competence still good for interpersonal media? Mobile phone and instant messenger. Computers in Human Behavior, 27(2), 924–934. https://doi.org/10.1016/j.chb.2010.11.018

35.

Jara

Claro

Hinostroza

J. E.

San Martín

Rodríguez

Cabello

Ibieta

Labbé

(2015). Understanding factors related to Chilean students’ digital skills: A mixed methods analysis. Computers & Education, 88, 387–398. https://doi.org/10.1016/j.compedu.2015.07.016

36.

Jenkins

Purushotma

Weigel

Clinton

Robison

A. J.

(2009). Confronting the challenges of participatory culture: Media education for the 21st century. Cambridge, MA: MIT Press.

37.

Katz

I. R.

(2007). Testing information literacy in digital environments: ETS’s iSkills assessment. Information Technology and Libraries, 26(3), 3–12. https://doi.org/10.6017/ital.v26i3.3271

38.

Kaufman

K. J.

(2013). 21 ways to 21st century skills: Why students need them and ideas for practical implementation. Kappa Delta Pi Record, 49(2), 78–83. https://doi.org/10.1080/00228958.2013.786594

39.

Keane

W. F.

Blicblau

A. S.

(2016). Beyond traditional literacy: Learning and transformative practices using ICT. Education and Information Technologies, 21(4), 769–781. https://doi.org/10.1007/s10639-014-9353-5

40.

Keupp

M. M.

Palmié

Gassmann

(2012). The strategic management of innovation: A systematic review and paths for future research. International Journal of Management Reviews, 14(4), 367–390. https://doi.org/10.1111/j.1468-2370.2011.00321.x

41.

Lai

C. Y.

Yang

H. L.

(2014). The reasons why people continue editing Wikipedia content-task value confirmation perspective. Behaviour & Information Technology, 33(12), 1371–1382. https://doi.org/10.1080/0144929X.2014.929744

42.

Lemke

(2002). enGauge 21st century skills: Digital literacies for a digital age. Naperville, IL: North Central Regional Educational Laboratory (NCREL).

43.

Lessig

(2008). Remix: Making art and commerce thrive in the hybrid economy. New York, NY: Penguin Press.

44.

Levy

Murnane

R. J.

(2004). The new division of labor: How computers are creating the next job market. Princetown, NJ: Princetown University Press.

45.

Lewin

McNicol

(2015). Supporting the development of 21st century skills through ICT. In Brinda

Reynolds

Romeike

Schwill

(Eds.), KEYCIT 2014: Key competencies in informatics and ICT (pp. 98–181). Potsdam, Germany: Universitätsverlag Potsdam.

46.

Litt

(2013). Measuring users’ internet skills: A review of past assessments and a look toward the future. New Media & Society, 15(4), 612–630. https://doi.org/10.1177/1461444813475424

47.

Loveless

(2007). Creativity, new technologies and learning: A review of recent literature [An update]. Bristol, UK: Futurelab.

48.

H. P.

Lee

M. R.

(2012). Experience differences and continuance intention of blog sharing. Behaviour & Information Technology, 31(11), 1081–1095. https://doi.org/10.1080/0144929X.2011.611822

49.

MacKenzie

Wajcman

(1985). The social shaping of technology. Buckingham, UK: Open University Press.

50.

Marchionini

White

(2007). Find what you need, understand what you find. International Journal of Human-Computer Interaction, 23(3), 205–237. https://doi.org/10.1080/10447310701702352

51.

Moher

Shamseer

Clarke

Ghersi

Liberati

Petticrew

Shelleke

Stewart

L. A.

(2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4(1), 1–9. https://doi.org/10.1186/2046-4053-4-1

52.

Mossberger

Tolbert

C. J.

Stansbury

(2003). Virtual inequality: Beyond the digital divide. Washington, DC: Georgetown University Press.

53.

Oldham

G. R.

Cummings

(1996). Employee creativity: Personal and contextual factors at work. Academy of Management Journal, 39(3), 607–634. https://doi.org/10.2307/256657

54.

Partnership for 21st Century Skills. (2007). Framework for 21st-century learning. http://www.p21.org/documents/P21_Framework_Definitions.pdf

55.

Punie

(2007). Learning Spaces: An ICT-enabled model of future learning in the knowledge-based society. European Journal of Education, 42(2), 185–199. https://doi.org/10.1111/j.1465-3435.2007.00302.x

56.

Rausch

Wuttke

(2016). Development of a multi-faceted model of domain-specific problem-solving competence and its acceptance by different stakeholders in the business domain. Unterrichtswissenschaft, 44(2), 164–189.

57.

Scheerder

A. J.

Van Deursen

A. J. A. M.

Van Dijk

J. A. G. M.

(2017). Determinants of Internet skills, uses and outcomes. A systematic review of the second-and third-level digital divide. Telematics and Informatics, 34(8), 1607–1624. https://doi.org/10.1016/j.tele.2017.07.007

58.

Selwyn

(2004). Reconsidering political and popular understandings of the digital divide. New Media & Society, 6(3), 341–362. https://doi.org/10.1177/1461444804042519

59.

Shamseer

Moher

Clarke

Ghersi

Liberati

Petticrew

Shekelle

Stewart

L. A.

(2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: Elaboration and explanation. British Medical Journal, 349, Article g7647. https://doi.org/10.1136/bmj.g7647

60.

Siddiq

Gochyyev

Wilson

(2017). Learning in digital networks: ICT literacy: A novel assessment of students’ 21st century skills. Computers & Education, 109, 11–37. https://doi.org/10.1016/j.compedu.2017.01.014

61.

Siddiq

Hatlevik

O. E.

Olsen

R. V.

Throndsen

Scherer

(2016). Taking a future perspective by learning from the past: A systematic review of assessment instruments that aim to measure primary and secondary school students’ ICT literacy. Educational Research Review, 19, 58–84. https://doi.org/10.1016/j.edurev.2016.05.002

62.

Silva

(2009). Measuring skills for 21st-century learning. Phi Delta Kappan, 90(9), 630–634.

63.

Song

Ling

(2011). Users’ attitude and strategies in information management with multiple computers. International Journal of Human-Computer Interaction, 27(8), 762–792. https://doi.org/10.1080/10447318.2011.555307

64.

Spitzberg

B. H.

(2006). Preliminary development of a model and measure of computer-mediated communication competence. Journal of Computer-Mediated Communication, 11(2), 629–666. https://doi.org/10.1111/j.1083-6101.2006.00030.x

65.

Starkey

(2011). Evaluating learning in the 21st century: A digital age learning matrix. Technology, Pedagogy and Education, 20(1), 19–39. https://doi.org/10.1080/1475939X.2011.554021

66.

Talja

(2005). The social and discursive construction of computing skills. Journal of the American Society for Information Science and Technology, 56(1), 13–22. https://doi.org/10.1002/asi.20091

67.

Van de Oudeweetering

Voogt

(2018). Teachers’ conceptualization and enactment of twenty-first century competences: Exploring dimensions for new curricula. The Curriculum Journal, 29(1), 116–133. https://doi.org/10.1080/09585176.2017.1369136

68.

Van Deursen

A. J. A. M.

Helsper

E. J.

Eynon

(2016). Development and validation of the Internet Skills Scale (ISS). Information, Communication & Society, 19(6), 804–823. https://doi.org/10.1080/1369118X.2015.1078834

69.

Van Deursen

A. J. A. M.

Van Dijk

J. A. G. M

. (2010). Measuring Internet skills. International Journal of Human-Computer Interaction, 26(10), 891–916. https://doi.org/10.1080/10447318.2010.496338

70.

Van Deursen

A. J. A. M.

Van Dijk

J. A. G. M

. (2015). Internet skill levels increase, but gaps widen: A longitudinal cross-sectional analysis (2010-2013) among the Dutch population. Information, Communication & Society, 18(7), 782–797. https://doi.org/10.1080/1369118X.2014.994544

71.

Van Dijk

J. A. G. M

. (2005). The deepening divide: Inequality in the information society. London, UK: Sage Publications.

72.

Van Laar

Van Deursen

A. J. A. M.

Van Dijk

J. A. G. M.

De Haan

. (2017). The relation between 21st-century skills and digital skills: A systematic literature review. Computers in Human Behavior, 72, 577–588. https://doi.org/10.1016/j.chb.2017.03.010

73.

Voogt

Erstad

Dede

Mishra

(2013). Challenges to learning and schooling in the digital networked world of the 21st century. Journal of Computer Assisted Learning, 29(5), 403–413. https://doi.org/10.1111/jcal.12029

74.

Voogt

Roblin

N. P.

(2012). A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies. Journal of Curriculum Studies, 44(3), 299–321. https://doi.org/10.1080/00220272.2012.668938

75.

Wang

(2010). Using online shared workspaces to support group collaborative learning. Computers & Education, 55(3), 1270–1276. https://doi.org/10.1016/j.compedu.2010.05.023

76.

Wang

Hsieh

J. P. A.

Song

(2012). Understanding user satisfaction with instant messaging: An empirical survey study. International Journal of Human-Computer Interaction, 28(3), 153–162. https://doi.org/10.1080/10447318.2011.568893

77.

Wasserstein

R. L.

Lazar

N. A.

(2016). The ASA’s statement on p-values: Context, process and purpose. The American Statistician, 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108

78.

A. Y.

Tian

S. W.

Vogel

Kwok

R. C. W.

(2010). Can learning be virtually boosted? An investigation of online social networking impacts. Computers & Education, 55(4), 1494–1503. https://doi.org/10.1016/j.compedu.2010.06.015

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.09 MB