A Quasi-Experimental Analysis of the Adult Learning Effect on Problem-Solving Skills

Abstract

This study builds an empirical research model that explores societal and individual antecedents of adult learning and examines the adult learning effect on problem-solving skills in the Organization for Economic Cooperation and Development (OECD) countries. Considering national differences, it uses multilevel data sources available from the Programme for the International Assessment of Adult Competencies (PIAAC) 2008-2013 and the OECD data lab. The results from this study suggest that young adults (25- to 34-year-olds) with more favorable backgrounds benefit from their social origin and occupational environment in terms of nonformal learning participation. It was also found that the participants outperform nonparticipants in the PIAAC problem-solving skill assessment. The key findings of this study advocate for the importance of policy interventions to combat the cumulative effects of multiple disadvantages in the educational trajectory from initial education to adult learning as well as to reduce the problem-solving skill gaps of young disadvantaged adults.

Keywords

adult learning and education problem-solving skills quasi-experimental analysis cross-national assessment data

Introduction

Continued economic, social, and technological developments require acquiring new skills through education and training. As a new skill necessary for the 21st century, well-documented research has noted the significance of problem-solving skills (Jacobs & Castek, 2018; Rosen & Vanek, 2017; van Laar, van Deursen, van Dijk, & de Haan, 2017). Problem-solving skills in technology-rich environments refer to how well people deal with specific types of problems using information and communication technology (ICT) tools. It is a competency domain at the intersection of computer literacy skills (i.e., the capacity to use ICT tools and applications) and the cognitive skills required to solve problems (Organization for Economic Cooperation and Development [OECD], 2013a). The ability to use these tools to access, process, and evaluate information intelligently is essential for the maintenance and upgrading of workforce skills. Moreover, the production, distribution, and use of new knowledge and information have created high-paying jobs.

As a result, high proficiency in these information-managing skills provides adults with more access to rewarding occupations and to participation in further learning and training over their career span. In contrast, low-skilled adults are likely to be left behind, which may increase inequality in the various socioeconomic outcome dimensions such as the distribution of income, nutrition, civic engagements, citizenship, and social trust. Consequently, academic and policy discussions stress the importance of digital readiness and the need for adults to continue learning throughout their entire lives (Horrigan, 2016; OECD, 2015; UNESCO, 2016). Learning occurs during every life stage in different contexts such as in the home, school, work, and community. From a lifelong-learning perspective, all citizens as self-directed individuals must have open, flexible, and personally relevant opportunities to develop the knowledge and competences necessary at all stages of their lives. Through a broader lens, a key skill is seen not just as a basic skill such as reading and writing but as a set of competencies applied to tasks in technology-rich environments (Hickling-Hudson, 2007; Wilson, Scalise, & Gochyyev, 2015).

This study aims to explore the nonformal learning effect on problem-solving skill acquisition in technology-rich environments focusing on the young adult population (25- to 34-year-old adults) in OECD countries. As young adults do develop their skills through ongoing learning activities, this study adopts the conceptual model that explores societal and individual antecedents of adult learning and examines the consequences of adult learning on problem-solving skill acquisition in technology-rich environments (see Figure 1). Among the countries at comparable levels of economic development, variation is noticeable in terms of national institutions. As Blossfeld, Kilpi-Jakonen, de Vilhena, and Buchholz (2014) have acknowledged the importance of national institutions, adult learning and skills at the individual level might vary across countries. However, relatively few studies have explored such a cross-national difference in terms of structural dimensions and education systems. Considering the lack of international evidence on the impact of the national antecedent, this study attempts to identify the specific features of national institutions affecting the relationship between adult learning and skills. One particular focus of this study is in examining the influence of national institutions on the relationship between nonformal learning and adult problem-solving skill acquisition.

Figure 1.

Adult learning and the role of various national institutions.

Participation in Adult Learning at the Individual and Societal Levels

As a way of promoting the transition from school to work, adult learning in the workplace may improve the adaptability of workers to technological and structural changes in the economy. Indeed, the benefits of adult learning are expected to offset disadvantages from the initial schooling stage (Rubenson, 2006a, 2006b; Tsatsaroni & Evans, 2014; UNESCO, 2013). From a lifelong-learning perspective, adults can enhance key skills via informal means in addition to formal learning in school or nonformal learning outside the formal education system (Tsatsaroni & Evans, 2014; UNESCO, 2013). As an alternative to traditional modes of learning, ICT plays an important role. The rapid growth in access to mobile phones and computers has enabled illiterate adults to use text messages that can enhance their literacy (Molnár & Benedek, 2014; UNESCO, 2016). Specifically, the technology helps adults acquire literacy skills in two primary ways (UNESCO, 2016; Wagner & Kozma, 2005). First, ICT-based instruction can develop the cognitive processes and basic skills related to literacy (Greene, Seung, & Copeland, 2014; Leu, Kinzer, Coiro, Castek, & Henry, 2017). Second, the development of literacy skills can be facilitated by technology that has created new opportunities for learning at a distance (Mohammadyari & Singh, 2015). Acquiring the ability to learn increases the interaction between education, jobs, and other contexts in adult life. The life-wide and lifelong perspective defines learning as a continuous activity independent of age. Adults continue to acquire and develop skills before, during, and after compulsory schooling, in and out of school, and through formal, nonformal, and informal learning (UNESCO, 2006, 2013).

However, participation in adult education and training is dominated by more educated populations (Desjardins & Rubenson, 2013; Fouarge, Schils, & De Grip, 2013; UNESCO, 2013). Marginalized groups risk exclusion from the further learning opportunities offered by social media and ICTs. In terms of inequality, Tuijnman and Boudard (2001) pointed out that adults with little initial education might have a lower probability of continuing to learn than their counterparts with high skills and a long period of initial education do. Indeed, the social theory of lifelong learning suggests that an individual’s capacity to obtain learning opportunities depends on his or her social environment. A theoretical lens of social interaction (Strauss, 1962) explains a working-class male’s experience of initial education and his subsequent learning and training differs systematically from that of middle-class females. Their access to learning opportunities is affected by their social status. Indeed, emergence of life course paradigm has illuminated cumulative process of socioeconomic status into the later years of adult (Elder & Giele, 2009). Regarding this issue, Colley, James, Diment, and Tedder (2003) developed a concept of “vocational habitus” to explain how learners are oriented to a particular set of dispositions and are prepared to enter occupations. They suggest that as a process of becoming, learning is socially constructed in terms of classed relations of power within the educational field and society at large.

The relevant literature notes that adult learning participation has been conditioned by national institutions (Blossfeld et al., 2014; Rubenson & Desjardins, 2009; UNESCO, 2016). At the societal level, educational systems and social policies have been defined as influencing factors for adult learning. Within the rigid system, tracking in upper secondary schooling might lead cumulative advantage or disadvantage of education over the life course (Blossfeld, Buchholz, Skopek, & Triventi, 2016; Blossfeld et al., 2014;). That is because such an inflexible system places people on a specific track within the educational hierarchy. For example, individual adults with a general education are somewhat more likely to have advanced education compared with those with a vocational education (Hanushek, Schwerdt, Woessmann, & Zhang, 2017). Those who are left behind during formal initial schooling may overcome their disadvantage through nonformal learning (Roosmaa & Saar, 2012; UNESCO, 2013; Werquin, 2010). As a result, the acknowledgement of a learning continuum between formal and nonformal education is an important issue at the system level (UNESCO, 2013). In terms of a linkage between formal and nonformal education, the relative degree of openness of formal education institutions to nontraditional students indicates a development of the adult education system in different countries (Desjardins & Rubenson, 2013). This suggests that countries that establish a system for recognizing nonformal learning outcomes may have an increase in participation rates in adult learning and skills.

Comparative evidence suggests that the social welfare policy is related to variations in adult education in different countries (Desjardins, 2013; Desjardins & Rubenson, 2013; Rubenson & Desjardins, 2009). For example, Nordic countries with a social democratic regime focus on dealing with structural barriers, whereas Anglo-Saxon countries follow a liberal welfare-state regime underscoring means-tested assistance, modest universal transfers, and social insurance plans. With the recognition of the public good aspects of adult education, the Nordic countries spend more money on adult education targeting disadvantaged groups compared with the Anglo-Saxon countries.

Since the global financial crisis in 2008 to 2009, however, most OECD countries have had increased demands on social protection systems providing institutional support for adult learning at the societal level. Emphasizing the importance of an effective labor supply, socioeconomically developed countries have activated their welfare states to increase employability (OECD, 2013b). As a way to improve labor market performance, OECD policy makers encourage member countries to implement activation reforms such as Active Labor Market Programs that provide vocational training and hiring subsides to support disadvantaged workers. These active labor market policies have indicated that the level of social policy development is important (Powell & Barrientos, 2004). As a type of active labor market program, public support for adult learning and training is essential to boost adults’ motivation to participate in further learning and to enhance their employability.

Method

Data Sources

In considering institutional differences, this study uses the data sources available from the recent international assessment, the Programme for the International Assessment of Adult Competencies (PIAAC) survey data from 2008 to 2013, and the OECD data lab. The PIAAC assessment concentrates on cognitive and workplace skills necessary for successful participation in 21st-century society and the global economy. To rule out the possibility that aging is closely associated with the level of performance in problem-solving skills and adult learning, this study targets young adults aged from 25 to 34 years for two reasons. First, a life span perspective defines the age of young adults as those individuals up to their mid-30s who have started taking on key responsibilities such as establishing a family or circle of friends, and getting a good job (Armstrong, 2007). Second, in this research, adult education refers to learning activities after the initial schooling stage, thus excluding those still in their first formal cycle of schooling. As a result, the final sample for this study consists of 23,615 adults from 19 countries including Austria, Belgium (Flanders), Canada, the Czech Republic, Denmark, Estonia, Finland, Germany, Ireland, Japan, Korea, the Netherlands, Norway, Poland, the Russian Federation, the Slovak Republic, Sweden, the United Kingdom (England and Northern Ireland), and the United States.

Measurements

This study uses an outcome variable measured by the PIAAC assessment framework that defines key adult skills in three domains (literacy, numeracy, and problem-solving skills) in technology-rich environments. By reflecting the changing nature of information, the assessment instruments used in PIAAC examine literacy in digital environments, the ability to access, use, and communicate mathematical information and ideas, and the ability to analyze computer-based simulation tasks, to define goals, and to monitor their progress (OECD, 2013a). Distinguished from previous adult assessments, the PIAAC underscores these adult competencies to access, manage, integrate, and construct information using the technologies. In what follows, operationalizing problem-solving skills in technology-rich environments is described as a dependent variable.

The problem-solving assessment mainly examines the knowledge about how to handle digital tools and to structure the problem, to set goals, to measure progress toward those goals, and to practice metacognition (Levy, 2010). As a new skill, problem solving is distinguished from the other two cognitive domains—literacy and numeracy—in several particular tasks, focusing on the processes of problem solving in various environments, using pragmatic evaluation sources and the integration of information across sources. The PIAAC assessment estimates plausible values (PVs) from multiple imputations which combine item response theory scaling of the cognitive assessments with an unobservable latent regression model using information available in the background questionnaire in a population (OECD, 2013a). This study uses all 10 PVs as a dependent variable granted for an equivalent estimate and analyzed using an R package for complex surveys including PVs (svyPVpack).

Key factors affecting problem-solving skills come from the individual and country levels. In line with the systematic overview suggested by Blossfeld et al. (2014), this study takes account of characteristics of participation in adult learning at the micro and macro levels (see Figure 2). The individual-level determinants are measured by background questionnaire items from the PIAAC survey. The PIAAC collected data on the characteristics and backgrounds of respondents in five main areas: demographics, educational attainment and participation, labor-force status and employment, social outcomes, and literacy and numeracy practices and the use of skills. Using the PIAAC background questionnaire, the independent variables were operationalized as follows.

Figure 2.

Characteristics of participation in adult learning at the micro and macro levels.

This study includes nonformal adult learning participation as a second set of dependent variables as well as an independent variable. As discussed in the relevant literature, nonformal education is categorized into two types: compensatory and complementary (Desjardins & Rubenson, 2013). Compensatory types pertain to basic education, literacy programs, and second chances related to formal qualifications, whereas complementary types are on-the-job training, continuing vocational or professional training, and adult higher education. Given that a key dependent variable—problem-solving skills in technology-rich environments—is closely associated with the complementary types, “participation in nonformal education for job-related reasons in the 12 months preceding the survey on nonformal learning” is defined as a measure of nonformal adult learning.

To examine the adult learning and training effect on the problem-solving skill, educational attainment initially needs to be considered as substantially discussed by the relevant research (Hanushek, Schwerdt, Wiederhold, & Woessmann, 2013; Lam & Warriner, 2012; UNESCO, 2013). As an indicator of initial educational attainment, the highest level of education completed is used. At the individual level, the other key variables are measured as follows. First, gender is measured as a dummy variable having the value 1 for the male group. Bearing in mind that cognitive decline depends on biological age, how old the respondent is was controlled for empirically as a continuous variable. In terms of social class, two indicators are measured: the highest level of education that either the mother or father guardian achieved and a derived variable of the yearly income percentile rank from the lowest group (less than 10%) to the highest group (90% and above).

The important covariates are variables that are also related to workplace environments (OECD, 2013a; Tuijnman & Boudard, 2001). To clarify the nature of the job’s and organization’s characteristics, three measurements are used to consider the relationship between the adult’s skills profiles in terms of occupational status, learning at work, and the use of ICT skills. First, occupational status is considered via occupation and status in employment coded by the International Standard Classification of Occupations. Also, a derived variable is employed from three items measuring the frequency of learning opportunities from supervisors or coworkers, learning by doing, and keeping up to date with new products or services. To examine the response to new technology in the workplace, an index is employed indicating how ICT skills at work are used, which is available from the PIAAC survey data.

Focusing on the importance of societal antecedents, the education system and labor market policies are considered in the research model. First, this study defines participation rates in adult formal education as a key country-level variable related to the education system since countries with well-developed adult education systems show high-participation rates for nonformal learning. Thus, the relative degree of openness of formal educational institutions to nontraditional students is measured as an indicator of highly advanced adult learning systems (Desjardins, 2013; Desjardins & Rubenson, 2013).

As a measure of the “supply-side” policies of national governments, an indicator of the labor market policy is included. To help unemployed adults back to work, OECD countries have implemented policies including job-placement services, benefit administration, and labor market programs such as training and job creation (OECD, 2013b). To cover this aspect, a scaled variable is used that measures the protection of permanent workers against individual dismissal. To avoid a multicollinearity issue among the independent variables, a correlation matrix was screened prior to research modeling (see the appendix).

Analytical Strategies

In this study, four specific analytical techniques are used. First, a preliminary analysis builds a specific picture of national variations in problem-solving skill gaps through the lens of nonformal adult learning across 19 countries. For this, the R package (instvy) is used to estimate an average difference in the problem-solving skill assessment in the PIAAC between participants and nonparticipants of nonformal adult learning. It calculates a correct estimate of the means and associated standard errors of problem-solving skill-achievement variables measured by 10 PVs.

Second, a multilevel analysis considers the institutional difference in the participation in nonformal learning, assuming individual adults are nested within countries. As a common way to analyze data from nested samples, the multilevel approach takes account of variations at two levels and decreases aggregation bias (Raudenbush & Bryk, 2002). The multilevel model analysis predicts an adult’s propensity for post-initial learning activities. Through a propensity score analysis, this study aims to examine unbiased estimates of the nonformal adult learning effects. By replacing the confounding covariates with one scalar function of these covariates, it reduces unobserved heterogeneity. As a primary data source, the PIAAC provides a variety of variables used as covariates to characterize adults and their ecological backgrounds such as households and workplaces. To adjust for such differences in the background variables, the propensity score is estimated as a balancing score.¹

This propensity score needs to include unobserved covariates in multilevel structured populations (Arpino & Mealli, 2011; Hong & Raudenbush, 2005; Kim & Seltzer, 2007). Since the average probability of participating in adult learning programs may vary across countries, a multilevel technique specifies the propensity score for two different levels, assuming that the variation may depend on measured or unmeasured country-level characteristics. Without considering the national contexts, it might be problematic to compare the educational trajectory of adults among the different countries (Arpino & Mealli, 2011). In considering such institutional variations, this study uses a multilevel logistic regression model to estimate the propensity score that predicts adult learning and training. With the covariates used in initial generalized linear models, this study examines the treatment effects after controlling not only for individual-level characteristics but also for country-level variables.

Propensities derived from the multilevel model are used to match participants and nonparticipants in nonformal adult learning. To find the best match for the participation in nonformal learning, the covariate balancing propensity score (CBPS) methodology is implemented, resulting in an optimized balance (Imai & Ratkovic, 2014) using the CBPS package in R. A propensity score weight analysis is also carried out for the complex sample design of PIAAC via the toolkit for weighting and analysis of nonequivalent groups (twang) package and a package for complex surveys including PVs (svyPV) in R. This tailored approach is essential when using complex population-based sample survey data such as that found in PIAAC.

Last, this study employs multiple imputation methods to handle missing values. When observations are missing, the available options are to delete the data or to replace the missing value with an imputed value. Given that listwise deletion reduces the analytic sample size, which can be problematic if missing observations occur for many subjects (Kline, 1998), this study adopts multiple imputations using the Markov chain Monte Carlo algorithm from the mice package in R. Using multiple imputation analysis, the final sample consists of 23,615 young adults from 19 countries for which individual backgrounds and national characteristics were collected.

Results

Countries vary in terms of adult education participation and have different patterns for the relationship between nonformal learning participation and the problem-solving skills of young adults. The purpose of this study is to examine variations across countries and to determine whether individual and societal antecedents are related to participation in nonformal adult education. The ultimate goal is to estimate the effect of nonformal education on problem-solving skills in technology-rich environments across and within countries.

National Variations in the Nonformal Learning Effect on Problem-Solving Skills

A preliminary analysis of international assessment data finds an explicit tendency for problem-solving skill gaps linked to participation in nonformal adult learning among different OECD countries. It shows cross-country variations in the nonformal learning effect (see Table 1 and Figure 3). When controlling the covariate variables, three countries demonstrate a significant association between nonformal learning and problem-solving skills (Canada, the Russian Federation, the Slovak Republic). This covariate analysis reveals a prominent pattern of educational attainment that is strongly associated with problem-solving skills in all OECD countries. Considering the close linkage between two factors—educational attainment and adult learning participation (OECD, 2015; UNESCO, 2016)—, a further examination is required to determine which factors predict nonformal education participation in addition to educational attainment. Consequently, the association between individual and societal antecedents and participation in nonformal adult education is estimated with a series of fixed effects multilevel models. The results are summarized in the following section.

Table 1.

National Variations in Problem-Solving Skill Gaps.

OLS regression slopes	NFE		EDU
OLS regression slopes	B	SE	B	SE
Austria	3.53	2.55	4.61***	1.07
Belgium	4.98	3.31	4.74***	1.01
Canada	9.86***	2.56	5.6***	0.8
Czech Republic	0.88	3.84	5.66***	1.28
Denmark	1.02	3.11	4.68***	0.95
Estonia	−0.59	2.44	4.49***	0.86
Finland	−0.42	2.91	4.59***	1.24
Germany	0.8	3.19	7.9***	1.02
Ireland	1.8	2.54	5.48***	0.99
Japan	0.15	0.03	4***	1.29
Netherlands	2.67	3.67	6.99***	1.25
Norway	1.43	2.74	4.08***	0.94
Poland	1.93	3.83	2.78***	0.96
Russian Federation	10.06**	4.36	4.55***	1.43
Slovak Republic	13.14***	3.82	3.2**	1.08
South Korea	1.44	2.03	5.4***	1.02
Sweden	1.96	2.86	4.08***	1.14
The United Kingdom	5.77	3.68	1.66**	0.65
The United States	4.46	3.13	6.75***	1.31

Note. OLS = ordinary least squares; NFE = nonformal education; EDU = educational attainment; B = coefficient; SE = standard error.

***

p < .001. **p < .01.

Figure 3.

National variations in problem-solving skill gaps.

Individual and Societal Antecedents of Nonformal Learning Participation

Using multilevel logistic regression sets, a young adult’s propensity score to receive nonformal education is predicted with important covariates including both their individual-level background and national context. The first model includes individual-level variables that are fixed so that they have the same effect for countries. Table 2 shows that participation in nonformal education is significantly associated with a variety of individual characteristics. First, the gender effect shows that males are 31% more likely to participate in nonformal education relative to females. Among 25- to 34-year-olds, age has statistical significance, but this is not practically meaningful due to the limit of sample size. The statistically significant positive estimated effect of social class holds true. The odds of taking nonformal education among adults with highly educated parents are 1.13 times higher than those with low-educated parents. Education has a noticeable effect. Adults with a higher level of educational attainment are more likely to take nonformal education, with an increase in odds of 14% per standard deviation increase in the level of educational attainment. The odds of participating in nonformal education among workers with a higher yearly salary are 1.21 times higher than for low-paid workers.

Table 2.

Multilevel Propensity Models for Nonformal Educational Participation: Fixed Effects.

Fixed effects	Model 1	Model 2	Model 3
Fixed effects	Exp. (%)	Exp. (%)	Exp. (%)
National characteristics
Labor market policy (γ₀₁)		1.09	1.08
Adult education system (γ₀₂)		1.05*	1.09***
Individual characteristics
Gender (γ₁₀)	1.31***	1.31***	1.31***
Age (γ₂₀)	1.01**	1.01**	1.01***
Parental education level (γ₃₀)	1.13***	1.13***	1.13***
Educational attainment (γ₄₀)	1.14***	1.14***	1.22***
Yearly income (γ₅₀)	1.21***	1.21***	1.21***
Workplace characteristics
Occupational status (γ₆₀)	1.43***	1.43***	1.43***
Learning at work (γ₇₀)	1.27***	1.27***	1.27***
ICT at work (γ₈₀)	1.16***	1.16***	1.16***
Interaction effects
Educational attainment × adult education system (γ₁₁)			0.99**
Model fit statistics
Deviance (chi-square test)	26425	26420^†	26412**
AIC	26445	26444	26438

Note. ICT = information and communication technology; AIC = Akaike information criterion.

***

p < .001. **p < .01. *p < .05. ^†p <.10.

After controlling for individual and family background characteristics, workplace characteristics are consistently found to predict the likelihood of being a participant in nonformal education. The coefficients indicate that the more highly skilled occupational groups are 43% more likely to undertake nonformal education than the less-skilled occupational groups are. Learning opportunities at work have a positive effect as well. The coefficients indicate that adults who more proactively learn at work are 27% more likely to participate in nonformal education. A positive effect of ICT skills is prominent in the workplace. Adults with proficient ICT skills at work have 1.16 times the odds of those with limited ICT skills of participating in nonformal education.

By adding a variety of country-level predictors in Model 2, the probability of participating in nonformal education is predicted by national characteristics. After controlling for individual background variables, the adult education system at the country level is significantly related to nonformal learning participation. Specifically, the odds of taking nonformal education among adults in a country with a more flexible education system are 1.05 times higher than for those in a country providing limited educational access for nontraditional learners. In addition, adults in a country with a labor market policy that protects permanent workers against dismissal are more likely to participate in nonformal learning, with an increase in odds of 9%, but this is not statistically significant.

In Model 3, which adds interaction terms between country-level variables and individual variables, a net effect of the national context on nonformal learning participation is found. The multilevel logistic regression detects a significant cross-level interaction effect between institutional and individual characteristics. First, nonformal learning participation increases with relatively low educated adults in countries with well-developed adult learning systems. The odds of young adults undertaking nonformal education with a lower level of educational attainment in countries with rigid educational systems are 0.99 lower than for those in countries with more flexible and open educational systems. Although this odds difference implies a well-developed education system might mitigate against the impact of initial education inequality on adult learning participation, it cannot be practically meaningful due to the sample size.

To test differences in the overall fit of the sets of nested models, the model fit statistics were examined. The overall goodness of fit indicates that the association between predictors and the outcome varies across countries, and individual and societal antecedents condition participation in nonformal learning. From between the two types of multilevel models, the Model 3 multilevel model was selected as it takes the country-level variation as arising from the interaction between institutional and individual characteristics based on the overall deviance test. Using Model 3, which predicts the probability of nonformal learning participation, a propensity score was obtained for each individual adult. The sections below specify how the multilevel propensity score model was run and analyze the causal effect of nonformal education on problem-solving skills in technology-rich environments.

Multilevel Propensity Score for Nonformal Learning Participation

Multilevel propensity score modeling was used to estimate the effect of nonformal learning participation on problem-solving skills in technology-rich environments. Three analytical packages (CBPS, twang, and svyPV) were used for this purpose. This allows for the maximization of the resulting covariate balance and the prediction of treatment assignments using the PIAAC data with a complex survey design and PVs.

The propensity score predicting adult education participation includes a number of significant covariates at both the individual and country levels. Propensity score weighting was applied with multilevel data to estimate the average treatment effect of nonformal learning participation on problem-solving skills. Table 3 below summarizes the covariate balance evaluation to ensure an equivalence in the distribution of each covariate for the treated (participation) and control (nonparticipation) adults. The results of the statistical test in the table (p values) demonstrate that two groups are balanced in terms of propensity scores and that selection bias has been removed.

Table 3.

Covariate Balance.

Variables	tx.mn	tx.sd	ct.mn	ct.sd	stat	p
Gender	0.48	0.50	0.47	0.50	1.12	0.26
Age	29.50	2.93	29.56	2.93	−1.28	0.20
Parental education level	2.24	0.68	2.23	0.69	1.01	0.31
Educational attainment	4.56	1.76	4.57	1.80	−0.33	0.74
Occupational status	3.13	0.98	3.16	0.94	−1.33	0.18
Yearly income	3.48	1.38	3.49	1.39	−0.43	0.67
Learning at work	2.12	0.95	2.10	1.01	1.60	0.11
ICT at work	1.97	1.00	1.95	1.05	1.23	0.22
Labor market policy	1.84	0.69	1.82	0.70	1.27	0.20
Adult education system	10.23	3.81	10.17	3.81	0.95	0.34

Note. ICT = information and communication technology; tx.mn/ct.mn = treatment (participation) means and the control (nonparticipation) means; tx.sd/ct.sd = the propensity score weighted treatment (participation) and the control (nonparticipation) group’s standard deviations; stat, p = depending on the types of variables (i.e., continuous or categorical), stat is a t statistic or chi-square statistic and p is the associated p value. Insignificance results in covariate balance, statistically rejecting a significant difference in backgrounds between treated and controlled groups.

Estimated Effects of Adult Learning on Problem-Solving Skills

Table 4 displays the estimated effects of nonformal learning on problem-solving skills in technology-rich environments. The participants generally outscore nonparticipants for OECD countries. The average achievement of the participant group (296.66) is significantly higher than that of the nonparticipant group (292.02). This result suggests that nonformal learning might make a difference in problem-solving skill acquisition after accounting for individual and country characteristics on average across OECD countries.

Table 4.

Nonformal Learning Effect on Problem-Solving Skills Within the Strata.

	Participant	Nonparticipant	Mean difference
Mean score for problem-solving skill assessment	296.66	292.02	4.64***
Number of cases	9733	11899
Sum of weights	21006.19	21537.5

Note. Average treatment effect: 4.64*** (t = 7.19, p = .000).

Discussions

The key findings of this study have called attention to three important issues. First, the influence of adult learning on skill acquisition provokes a concern due to the close relationship between education and skills. A number of intergovernmental reports have emphasized the importance of learning and training as an investment in human resources (Guile, 2006). To enhance adult skills, developed countries have a wide variety of approaches and provisions for vocational training in different types of institutions (Dolton, 2004). In terms of the provision of further education, for example, the United Kingdom shows a strict dichotomy between work-based training and academic training, while many other countries have a much more united integration of the work-based training system. Despite marked variations in adult learning systems among different countries, it is commonly known, however, that access to learning opportunities substantially depends on one’s family background, especially for young adults.

From a sociological perspective, adult skills are learned through interaction with a sociocultural environment and skill acquisition should be understood as a learning metric (Steinberg, 1990; UNESCO, 2015b). In this context, skill gaps relate to an unequal chance of learning, and it is crucial to promote lifelong learning for all to increase sustainable development. Regarding social reproduction, this study addresses the question of unequal access to educational experiences from a life-course perspective. Adults from low SES families are less likely to have good levels of initial education or to obtain more access to further learning opportunities. Those from educationally disadvantaged backgrounds might find it hard to get decent jobs that could provide more opportunities for them to improve their professional development via workplace learning. This might trigger a skill gap where the low-skilled jobs are overrepresented by a low-educated workforce. A vulnerable adult with a low education level might encounter the double jeopardy scenario of a lack of work-based learning and skill development. In a lifelong-learning society, skill can beget skill when mediated by workforce learning, which might repeat a vicious cycle. Herein, the hypothesis that high-skilled adults are more likely to participate in further learning is not examined, although further examination is needed of the other causes linked to skills and adult learning in the near future.

In general, more systematic and structured approaches are necessary for the disadvantaged groups. To meet their different learning needs and demands, there is strong support for developing a system for recognizing nonformal learning outcomes (UNESCO, 2013; Werquin, 2010). Consequently, some countries now employ two routes toward achieving this goal: law and negotiation for social consensus (Werquin, 2010). As for the state-led model, the law and policies are globally observed in the legislative framework of recognition, validation, and accreditation of nonformal and informal learning (UNESCO, 2015a, 2015b). In Finland, for example, the national policy has legally validated learning outside the formal system in the various sectors of education from comprehensive schooling to adult vocational education through the Competence-Based Qualification system (Damesin, Fayolle, Fleury, Malaquin, & Rode, 2014). The recognition of nonformal learning has proceeded with agreement among the social partners as well. In particular, the Nordic countries and Germany have demonstrated a well-developed institutional system driven by shared responsibility and a social partnership among the stakeholders such as the government, employees, employer organizations, and trade unions based on the coordinated market system (Rubenson, 2006a, 2006b; UNESCO, 2015a).

Since inequality in access to adult learning cannot be reduced without institutional and public policy frameworks, nation-specific institutions should be considered (Desjardins & Rubenson, 2013; Saar, Ure, & Desjardins, 2013). This study also identifies those who are excluded from access to post-initial learning and the most at risk of demonstrating poor skills in terms of social equity. To promote more equitable access to participation in adult learning and education, combating the cumulative effects of multiple disadvantages is of particular importance. The results of this study suggest that women, those from low SES backgrounds, low-educated adults, and adults with less experience of ICT in low-skilled jobs are likely to be marginalized in adult learning. To implement effective educational strategies for the disadvantaged groups, the system needs to respond to identifiable groups entering into trajectories of multiple disadvantages, in particular during the initial education stage. Therefore, systematic support should target the disadvantaged groups via all educational policies and interventions.

Last, the influence of societal antecedents found in this study sheds light on the importance of structural differences in the national systems of adult learning and skill formation. As a demand-side factor at the macro level, the level of participation in nonformal learning is associated with the occupational structure and investment in innovative activities (Roosmaa & Saar, 2012). In market-centered societies, a higher proportion of low-skilled workers in the occupational structure may decrease participation in nonformal learning. Innovation characteristics such as employment in high-tech services and manufacturing might have a positive effect on participation rates as well. Likewise, adult skill formation is related to the different types of political economies (Estevez-Abe, Iversen, & Soskice, 2001). In a coordinated market economy, firms place an emphasis on industry-specific skills and the promise of employment and unemployment security. The Nordic countries including Sweden, Norway, Finland, and Denmark encourage firms to invest in training that is based on long-term worker loyalty, supporting the social democratic welfare state. By contrast, the liberal market economy provides economic actors with an opportunity to acquire general skills to deploy their resources for higher returns. Lacking in employment protection and adequate wages, these liberal welfare-state countries hardly create any incentives for firms to invest in industry-specific skills. Differences in the social policies may indicate how training opportunities are distributed for skills development (Roosmaa & Saar, 2012). At the societal level, however, the empirical results of this study do not strongly prove that countries with stronger employment protection show higher participation rates in nonformal learning related to jobs, as has been suggested by the relevant research (O’Connell & Bryne, 2012).

Indeed, a limitation of this study is not to ensure that a flexible educational system at the country level can increase participation in adult learning at the individual level. In spite of statistical significance on a relationship between educational system and adult learning participation, that observation is not practically effective because of sample size issue. Such a small magnitude of effect restricts to verify a mean difference in problem-solving skill as well. The other constraint of data analysis is to use a dichotomous variable of adult learning participation, which does not consider various range of learning activities and specify different impact of adult learning. To enrich the theoretical foundations relating to adult learning, follow-up research needs to capture a wider range of adult learning participatory modes beyond the dichotomous classification linked to nonformal learning.

Footnotes

Appendix

Correlation Matrix.

	1	2	3	4	5	6	7	8	9	10	11
1. Gender	—
2. Age	0.016	—
3. Parental education level	−0.049	0.104	—
4. Educational attainment	0.042	0.025	−0.078	—
5. Adult education system	0.007	−0.001	0.001	0.402	—
6. Occupational status	0.207	−0.05	−0.091	−0.149	−0.006	—
7. Yearly income	−0.184	−0.113	0.001	−0.016	0.005	−0.054	—
8. Learning at work	−0.008	0.055	0.012	0.002	−0.01	−0.068	−0.016	—
9. ICT at work	−0.097	−0.038	−0.037	−0.038	0.006	−0.235	−0.127	−0.091	—
10. Labor market policy	−0.005	0.019	0.007	−0.011	0.154	−0.008	0.011	0.014	0.002	—
11. Educational attainment × adult education system	−0.006	−0.028	−0.008	−0.924	−0.433	0.015	−0.023	0.002	−0.001	0.011	—

Note. ICT = information and communication technology.

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Financial assistance provided by National Research Foundation of Korea Grant (NRF-2016S1A3A2924944) is gratefully acknowledged.

Notes

ORCID iD

Suehye Kim

Author Biography

Suehye Kim is a research professor at the Center for Social Cohesion Education in Korea University. She worked as a Programme Specialist at the UNESCO Institute for Lifelong Learning (UIL) in Hamburg, Germany from 2015 to 2017 after attaining a PhD from the State University of New York in Albany. Her research has mainly addressed the interaction between micro- and macro-level processes in the dynamics of learning outcomes and the social environment from a comparative and international perspective.

References

Armstrong

(2007). The human odyssey: Navigating the twelve stages of life. New York, NY: Sterling.

Arpino

Mealli

(2011). The specification of the propensity score in multilevel observational studies. Computational Statistics & Data Analysis, 55, 1770-1780.

Blossfeld

H. P.

Buchholz

Skopek

Triventi

(Eds.). (2016). Models of secondary education and social inequality: An international comparison. Northhampton, MA: Edward Elgar.

Blossfeld

H. P.

Kilpi-Jakonen

de Vilhena

D. V.

Buchholz

(2014). Adult learning in modern societies: An international comparison from a life-course perspective. Northhampton, MA: Edward Elgar.

Colley

James

Diment

Tedder

(2003). Learning as becoming in vocational education and training: Class, gender and the role of vocational habitus. Journal of Vocational Education & Training, 55, 471-498.

Damesin

Fayolle

Fleury

Malaquin

Rode

(2014). Challenges, actors and practices of non-formal and informal learning and its validation in Europe. Brussels, Belgium: European Trade Union Institute.

Desjardins

(2013). The economics of adult education. New Directions for Adult and Continuing Education, 138, 81-90.

Desjardins

Rubenson

(2013). Participation patterns in adult education: The role of institutions and public policy frameworks in resolving coordination problems. European Journal of Education, 48, 262-280.

Dolton

(2004). What do policy makers need to know about the skills of young people and the school to work transition. Retrieved from http://www.oecd.org/employment/emp/34474626.pdf

10.

Elder

G. H.

Giele

J. Z.

(Eds.). (2009). The craft of life course research. New York, NY: Guilford Press.

11.

Estevez-Abe

Iversen

Soskice

(2001). Social protection and the formation of skills: A reinterpretation of the welfare state. Retrieved from https://pdfs.semanticscholar.org/6c7a/294833375219875442e4a9365932a49a7184.pdf

12.

Fouarge

Schils

De Grip

(2013). Why do low-educated workers invest less in further training? Applied Economics, 45, 2587-2601.

13.

Greene

J. A.

Seung

B. Y.

Copeland

D. Z.

(2014). Measuring critical components of digital literacy and their relationships with learning. Computers & Education, 76, 55-69.

14.

Guile

(2006). Access, learning and development in the creative and cultural sectors: From “creative apprenticeship”to “being apprenticed.” Journal of Education and Work, 19, 433-453.

15.

Hanushek

E. A.

Schwerdt

Wiederhold

Woessmann

(2013). Returns to skills around the world: Evidence from PIAAC. Cambridge, MA: National Bureau of Economic Research.

16.

Hanushek

E. A.

Schwerdt

Woessmann

Zhang

(2017). General education, vocational education, and labor-market outcomes over the lifecycle. Journal of Human Resources, 52, 48-87.

17.

Hickling-Hudson

A. R.

(2007). Beyond schooling: The role of adult and community education in postcolonial change. In Arnove

Torres

C. A.

(Eds.), Comparative education: The dialectic of the global and the local (pp. 197-216). Lanham, MD: Rowman & Littlefield.

18.

Hong

Raudenbush

S. W.

(2005). Effects of kindergarten retention policy on children’s cognitive growth in reading and mathematics. Educational Evaluation and Policy Analysis, 27, 205-224.

19.

Horrigan

J. B.

(2016). Digital readiness gaps. Washington, DC: Pew Research Center. Retrieved from http://www.pewinternet.org/2016/09/20/digital-readiness-gaps/

20.

Imai

Ratkovic

(2014). Covariate balancing propensity score. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76, 243-263.

21.

Jacobs

G. E.

Castek

(2018). Digital problem solving: The literacies of navigating life in the digital age. Journal of Adolescent & Adult Literacy, 61, 681-685.

22.

Kim

Seltzer

(2007). Causal inference in multilevel settings in which selection processes vary across schools (CSE technical report 708). Los Angeles, CA: National Center for Research on Evaluation, Standards, and Student Testing.

23.

Kline

R. B.

(1998). Principles and practice of structural equation modeling. New York, NY: Guilford Press.

24.

Lam

W. S. E.

Warriner

D. S.

(2012). Transnationalism and literacy: Investigating the mobility of people, languages, texts, and practices in contexts of migration. Reading Research Quarterly, 47, 191-215.

25.

Leu

D. J.

Kinzer

C. K.

Coiro

Castek

Henry

L. A.

(2017). New literacies: A dual-level theory of the changing nature of literacy, instruction, and assessment. Journal of Education, 197(2), 1-18.

26.

Levy

(2010). How technology changes demands for human skills. Paris, France: OECD.

27.

Mohammadyari

Singh

(2015). Understanding the effect of e-learning on individual performance: The role of digital literacy. Computers & Education, 82, 11-25.

28.

Molnár

Benedek

(2014, June). ICT in education: A new paradigm and old obstacle. Paper presented at the ICCGI 2014, The Ninth International Multi-Conference on Computing in the Global Information Technology, Seville, Spain.

29.

O’Connell

P. J.

Byrne

(2012). The determinants and effects of training at work: Bringing the workplace back in. European Sociological Review, 28(3), 283-300.

30.

Organization for Economic Cooperation and Development. (2013a). OECD skills outlook 2013: First results from the survey of adult skills. Paris, France: Author.

31.

Organization for Economic Cooperation and Development. (2013b). Public expenditure on active labour market policies 2013/1. Paris, France: Author. Retrieved from https://www.oecd-ilibrary.org/employment/public-expenditure-on-active-labour-market-policies-2013-1_lmpxp-table-2013-1-en

32.

Organization for Economic Cooperation and Development. (2015). Adults, computers and problem solving: What’s the problem? Paris, France: Author.

33.

Powell

Barrientos

(2004). Welfare regimes and the welfare mix. European Journal of Political Research, 43, 83-105.

34.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). Thousand Oaks, CA: Sage.

35.

Roosmaa

E. L.

Saar

(2012). Participation in non-formal learning in EU-15 and EU-8 countries: Demand and supply side factors. International Journal of Lifelong Education, 31, 477-501.

36.

Rosen

D. J.

Vanek

J. B.

(2017). Technology for innovation and change in adult basic skills education. New Directions for Adult and Continuing Education, 155, 51-60.

37.

Rubenson

(2006a). Adult education and cohesion. In Lauder

Brown

Dillabough

J.-A.

Halsey

A. H.

(Eds.), Education, globalization, and social change (pp. 936-948). Oxford, England: Oxford University Press.

38.

Rubenson

(2006b). The Nordic model of lifelong learning. Compare, 36, 327-341.

39.

Rubenson

Desjardins

(2009). The impact of welfare state regimes on barriers to participation in adult education a bounded agency model. Adult Education Quarterly, 59, 187-207.

40.

Saar

Ure

O. B.

Desjardins

(2013). The role of diverse institutions in framing adult learning systems. European Journal of Education, 48, 213-232.

41.

Steinberg

R. J.

(1990). Social construction of skill gender, power, and comparable worth. Work and occupations, 17, 449-482.

42.

Strauss

(1962) Transformations of identity. In Rose

(Ed.), Human behaviour and social processes: An interactionist approach (pp. 63-85). London, England: Routledge.

43.

Tsatsaroni

Evans

(2014). Adult numeracy and the totally pedagogised society: PIAAC and other international surveys in the context of global educational policy on lifelong learning. Educational Studies in Mathematics, 87, 167-186.

44.

Tuijnman

Boudard

(2001). Adult education participation in North America: International perspectives. Ottawa, Ontario, Canada: Statistics Canada.

45.

UNESCO. (2006). Education for all literacy for life, UNESCO education for all global monitoring report. Paris, France: Author.

46.

UNESCO. (2013). 2nd Global report on adult learning and educaton: Rethinking literacy. Hamburg, Germany: UNESCO Institute for Lifelong learning.

47.

UNESCO. (2015a). Global perspectives on recognising non-formal and informal learning: Why recognition matters. Hamburg, Germany: UNESCO Institute for Lifelong Learning.

48.

UNESCO. (2015b). Levelling and recognizing learning outcomes: The use of level descriptors in the twenty-first century. Paris, France: Author.

49.

UNESCO. (2016). Third global report on adult learning and education (GRALE 3). Hamburg, Germany: UNESCO Institute for Lifelong Learning.

50.

van Laar

van Deursen

A. J.

van Dijk

J. A.

de Haan

(2017). The relation between 21st-century skills and digital skills: A systematic literature review. Computers in Human Behavior, 72, 577-588.

51.

Wagner

D. A.

Kozma

(2005). New technologies for literacy and adult education. Paris, France: UNESCO.

52.

Werquin

(2010). Recognition of non-formal and informal learning: Outcomes, policies, and practices. Retrieved from http://www.eucen.eu/sites/default/files/OECD_RNFIFL2010_Werquin.pdf

53.

Wilson

Scalise

Gochyyev

(2015). Rethinking ICT literacy: From computer skills to social network settings. Thinking Skills and Creativity, 18, 65-80.