Development and Initial Validation of the Nonparticipation in Nonformal Education Questionnaire

Abstract

Nonformal adult education (NFE) provides adults with the opportunity to obtain competences needed to adapt to today’s changing job market. Despite the well-documented positive effects of NFE, there remains a lack of valid research instruments to assess factors of nonparticipation. Specifically, defining the perspectives of social groups with the highest incidence of barriers and the lowest level of participation in NFE has not been a primary research goal. This study describes the development and initial validation of a novel research tool entitled the Nonparticipation in Nonformal Education Questionnaire (NP-NFE-Q). The analysis covers the use of the NP-NFE-Q on a representative sample of the adult population (n = 878) as well as three additional cohorts: low-educated workers (n = 227), persons caring for children under the age of three (n = 227), and retired persons (n = 232). The results consistently support the validation of a correlated five-factor model, which includes two situational, one institutional and two dispositional factors.

Keywords

nonformal education lifelong learning barriers to participation in adult education factor analysis

Development and Initial Validation of the Nonparticipation in Nonformal Education Questionnaire

Nonformal adult education (NFE) has been described in various policy documents (Organization for Economic Co-operation and Development [OECD], 2018, 2019; United Nations Educational, Scientific and Cultural Organization, 2019, 2020) and research studies (Boeren, 2016; Rees, 2013; Regmi, 2015) as a key component of lifelong learning. NFE can be defined as any further adult education that occurs outside formal education settings and involves structured activities that do not result in official certification according to the International Standard Classification of Education (ISCED, 2011), which typically include courses, distance learning, private tuition, and guided on-the-job training (European Statistical Office, 2016). Although these educational activities may not be directly job-oriented, they generally occur in the work environment, which has become the main domain of NFE in recent years (Desjardins, 2017; Rubenson, 2018).

In most countries, two to three times more adults over the age of 24 years are engaged in NFE every year as compared to formal adult education. NFE has been described as a primary source of dynamism in today’s knowledge economies (Iversen & Soskice, 2019) as well as an efficacious tool to increase the competitiveness of individuals in the labor market and enhance their adaptability to social change (Boeren, 2016; Busemeyer & Trampusch, 2012; OECD, 2018, 2019). It has also been identified as an effective means of promoting health, civic and cultural engagement and thus contributing to overall sociopolitical trust (Iñiguez-Berrozpe et al., 2020). NFE is particularly relevant today as societies are facing difficult challenges associated with institutional responses to the COVID-19 pandemic (Boeren et al., 2020; Rambla & Milana, 2020; Waller et al., 2020).

Despite the positive outcomes of NFE programs, a number of social groups remain significantly underrepresented (Desjardins, 2017; Kalenda et al., 2020; Kyndt et al., 2011; Van Nieuwenhove & De Wever, 2021). Three populations in particular have shown low participation: adults with low education levels, the elderly, and persons taking care of young children. Each of these groups is distinguished by a specific set of features that makes its members vulnerable, potentially decreasing their desire and opportunity to become involved in education and training. Several authors (Dämmrich et al., 2015; Desjardins & Ioannidou, 2020; Iñiguez-Berrozpe et al., 2020) have described the crucial importance of reducing these negative effects.

Most low-educated adults are employed in areas with a high risk of automation (e.g., manufacturing, accommodation, food services, wholesale, and retail). According to the latest McKinsey report on the future of work in Europe (Smit et al., 2020), 51 million workers are at risk for at least the partial automation of their jobs. Low-educated adults are most threatened by changes in the labor market associated with the obsolescence of skills, the precarization of work, and the risk of unemployment (Berger & Frey, 2016; Brynjolfsson & McAfee, 2014). Moreover, these adults also have generally had a lack of positive experience with previous education (Rubenson, 2011, 2018). Employers are generally less willing to invest in enhancing the skills of this population as compared to higher-educated workers (Kyndt et al., 2011).

The elderly are challenged by the fast pace of social change represented by digitalization. This population often has not developed (to a sufficient degree) the digital literacy necessary to master the everyday skills needed to fully participate in contemporary society (United Nations Educational, Scientific and Cultural Organization, 2019, 2020). Moreover, due to the dominant tendency of NFE towards job-oriented learning, the number of opportunities for participation sharply declines after individuals have entered into retirement (Rubenson, 2018). These factors often make the situation of older adults quite precarious, decreasing their level of engagement with the wider society. Greater inclusivity for this population would be made possible through their participation in NFE.

Persons taking care of young children face several issues. The responsibilities of childcare bring many situational constraints into the lives of these individuals, many of which are female. Moreover, when they do (re)enter the workplace, especially in the professions and other knowledge-intensive careers, these individuals may find it difficult to keep up with the continuing education demanded in certain jobs (Dämmrich et al., 2015).

To further complicate matters, gender discrimination can exacerbate risk factors in all the groups mentioned above. As indicated, many women have fewer opportunities for NFE due to their commitment to family-related responsibilities. Women have also been shown to face lower support from employers (e.g., due to perceptions in many cultures regarding the traditional social roles of women). These factors have been found to negatively affect the motivation of women towards participation in NFE. Finally, the NFE programs that women participate in are generally less work-related than those that men engage in (Albert et al., 2010; Boeren, 2011; Vaculíková et al., 2020). In exploring barriers to participation in NFE, increased emphasis should be placed on factors related to gender as well as how these factors intersect.

Barriers to Participation and Related Theoretical Factor Structures

In her ground-breaking work Adults as Learners. Increasing Participation and Facilitating Learning (1981), K. Patricia Cross theorized three independent types of barriers to adult learning: (1) dispositional, (2) situational, and (3) institutional. This framework has become the basis for numerous empirical studies in the 40 years since the book’s publication.

Dispositional barriers are usually understood as personal constraints connected with attitudes towards learning and self-perceptions which influence an individual’s willingness to participate in NFE. In comparison, situational and institutional barriers represent structural obstacles. Situational barriers arise directly from life situations (e.g., employment, economic resources, family obligations, health, and disposable time for learning). Institutional obstacles are typically connected to the provision of NFE, mainly in terms of practices that can prevent adults from participating (e.g., lack of accessibility and lack of information).

Cross’s typology was partially corroborated by Darkenwald and Valentine (1985; see also Valentine & Darkenwald, 1990), whose theoretical conception, although never fully validated, was used in a number of empirical studies (see Courtney, 1992; Hovdhaugen & Opheim, 2018; Roosmaa & Saar, 2017; Rubenson & Desjardins, 2009; Van Nieuwenhove & De Wever, 2021). This conception remains the basis for the classification of barriers in international surveys such as the Programme for the International Assessment of Adult Competencies (PIAAC) and the Adult Education Survey (AES).

A number of scholars (e.g., Boeren, 2016; Desjardins, 2017; Kyndt et al., 2013; Rubenson, 2011; Rubenson & Desjardins, 2009) agree that nonparticipation in NFE is multi-causally influenced, with both cognitive (dispositional) and non-cognitive (situational and institutional) factors playing a role. In addition to established typologies, more recently some researchers (Desjardins, 2017; Rubenson, 2011, 2018) have described how the accessibility of NFE as well as quality of information about NFE may also represent significant independent factors with regard to participation. The basis for this view lies in the argument that information concerning organized learning activities represents a crucial precondition for decision-making about involvement in NFE. According to this model, institutional barriers related to information regarding NFE may influence adults more strongly than situational or dispositional obstacles (Baert et al., 2006; Boeren, 2016).

In addition, other relatively recent studies (Hovdhaugen & Opheim, 2018; Roosmaa & Saar, 2017) have accentuated the view that economic reasons for nonparticipation should not be classified as situational factors—the view taken by Cross (1981)—but as institutional factors. According to these findings, economic factors depend more on policies and programs supporting adult education (Rubenson & Desjardins, 2009; Rubenson, 2011) rather than strictly on the financial resources available to adults in a given situation. Other authors (Darkenwald & Valentine, 1985; Larson & Milana, 2006; Valentine & Darkenwald, 1990) argue for the classification of financial and economic constraints as independent barriers that more profoundly affect the most vulnerable social groups.

Despite the intensive theoretical discussion on the factors influencing nonparticipation in NFE and adult education in general, many key issues require further study (Merriam & Baumgartner, 2020). The primary sources of knowledge about NFE remain the surveys PIAAC and AES. Several issues can be identified in these instruments: (1) they contain only a few items measuring barriers to participation; (2) a number of certain factors are not measured at all, for example, dispositional factors (Van Nieuwenhove & De Wever, 2021); (3) these research tools focus on constraints that lower the involvement of adults in NFE, but they do not assess obstacles that prevent participation altogether (Hovdhaugen & Opheim, 2018; Rubenson, 2011). Besides the PIAAC and AES, few other research tools that can be used for this purpose have been validated (Darkenwald & Valentine, 1985; Valentine & Darkenwald, 1990). Furthermore, these instruments target the entire adult population, failing to consider differences among social groups, especially populations who face the highest number of barriers. Some tools focus solely on participation factors and not at all on constraints against participation (Blunt & Yang, 2002; Boeren & Holford, 2016; Boeren et al., 2012; Mulenga & Liang, 2008).

Study Aim

The main objective of this study is to describe the development of a novel research tool entitled the Nonparticipation in Nonformal Education Questionnaire (NP-NFE-Q) and to present an evaluation of the face validity and construct validity of the instrument. Within this context, we seek to empirically verify whether nonparticipation in NFE can be described in a basic three-factor structure, as indicated in previous literature (e.g., Cross, 1981), or whether other factors should also be identified and taken into account, as has been suggested by more recent studies (Rubenson, 2011, 2018).

Additionally, the study aims to (a) evaluate the construct validity of the questionnaire for three typical groups of non-participants in NFE: low-educated workers, persons caring for children under the age of three, and retired persons over 60 years of age; (b) evaluate the measurement invariance of the tool across gender; and (c) evaluate the overall reliability of the instrument.

Regarding the structure of this study, first we describe the development of the NP-NFE-Q including information about the research samples and analyses. The results of our own application of the questionnaire are then presented, followed by a discussion of certain limitations of the present research as well as potential directions for future investigations.

Method

The development phase of the research project consisted of a set of 56 semi-structured interviews conducted with low-educated workers (n = 21), persons caring for children under the age of three (n = 17) and retired persons over 60 years old (n = 18). In these interviews we sought to deepen our knowledge regarding non-participants in NFE by focusing on patterns of respondent interpretations, emotions, biographies and life conditions associated with NFE. Detailed results from this phase of the project are included in a manuscript currently submitted for publication (Karger et al., 2021).

With this qualitative research we were able to determine a number of categories and codes from which the questionnaire items were created. Items reflecting non-participation barriers applicable to all groups of non-participants were compared and in some cases aligned with already existing items taken from the AES. Following this process, we added other questionnaire items in line with the theory-driven situational, institutional and dispositional factors described above.

The research team carefully evaluated and connected 16 well-fitted items aligned with the AES to our 13 newly created items based on the interviews. A draft questionnaire with a total of 29 items was drawn up, with all items based on a 6-point Likert scale ranging from 1 (strongly disagree) to 6 (strongly agree). An overview of the items in the draft of the NP-NFE-Q is presented in Supplementary Table 1 and Supplementary Figure 1. The table includes sources (AES or our semi-structured interviews) for each item and full descriptions of the items. The figure contains the theoretical structure, confirmed structure, basic factor descriptions as well as the items saturating each factor.

The questionnaire was distributed to a representative sample of the target population (see section: Research sample) to obtain the data for initial validation. Face validity of the NP-NFE-Q was repeatedly assessed by the research team by focusing on the content and logical cohesion of the questionnaire. Great care was taken to ensure that the integrity of the original three-factor structure (e.g., Cross, 1981) was preserved from the initial drafts to the final version of the instrument. Based on more recent research (Rubenson, 2011, 2018), a breakdown of these three factors into a number of more specific factors was considered acceptable. The newly created factors were assessed in terms of internal composition, mutual cohesion as well as cohesion with the overall focus of the questionnaire.

Construct validity of the NP-NFE-Q was evaluated using typical psychometric analytical procedures for scale development and validation: exploratory and confirmatory factor analysis (EFA, CFA), correlations, and reliability estimates (DeVellis, 2017; Furr, 2018). These procedures are described below in the section Analysis. The final model of the questionnaire was evaluated with regard to the three specific groups of respondents, gender invariance as well as overall reliability (see Section Results).

Research Sample

Data collection was conducted in the Czech Republic during June 2020 by a specialized agency. Trained interviewers across the country addressed selected persons based on the Computer Assisted Personal Interviewing (CAPI) method. When respondents fell into the target group and agreed to participate in the research, they were verbally asked questions, with their responses recorded on a digital device by the interviewers.

The agency collected a representative stratified random sample of non-participants in NFE (n = 878) in which the gender, age and education ratio of the overall population was reflected (Czech Statistical Office, 2014). For the purposes of analysis, this sample was randomly divided into two separate sample groups. Sample 1 (n = 439) was used for conducting EFA, Sample 2 (n = 439) for CFA.

Quota samples from the three specific groups were also collected. Sample 3 (n = 227) contains low-educated workers with primary or secondary school education without a secondary school leaving exam (up to ISCED 3c). Sample 4 (n = 227) comprises persons caring for children under the age of three. Sample 5 (n = 232) contains retired persons over 60 years old.

In all phases of the survey process, emphasis was placed on the ethical principles of research, especially anonymity respecting the ICC/ESOMAR International Code (ESOMAR, 2016). The socio-demographic distribution of the five samples can be found in Table 1.

Table 1.

Socio-Demographic Characteristics Of The Five Research Samples.

Variables	Representative Stratified Samples				Quota Samples
	Sample 1 (n = 439)		Sample 2 (n = 439)		Sample 3 (n = 227)		Sample 4 (n = 227)		Sample 5 (n = 232)
	n	%	n	%	N	%	N	%	n	%
Gender
Male	206	46.9	216	49.2	114	50.2	44	19.4	98	42.2
Female	233	53.1	223	50.8	113	49.8	183	80.6	134	57.8
Age
18–24	44	10.0	39	8.9	13	5.7	40	17.6	0	0.0
25–34	66	15.0	63	14.4	41	18.1	89	39.2	0	0.0
35–44	83	18.9	70	15.9	59	26.0	88	38.8	0	0.0
45–54	65	14.8	69	15.7	59	26.0	9	4.0	0	0.0
55–64	72	16.4	92	21.0	51	22.5	1	0.4	24	10.3
65 and more	109	24.8	106	24.1	4	1.8	0	0.0	208	89.7
Educational attainment level
ISCED2	75	17.1	87	19.8	30	13.2	21	9.3	11	4.7
ISCED3c	172	39.2	163	37.1	197	86.8	69	30.4	73	31.5
ISCED3a,b	146	33.3	134	30.5	0	0.0	83	36.6	87	37.5
ISCED5,6	46	10.5	55	12.5	0	0.0	54	23.8	61	26.3
Main employment status
Full time	214	48.7	190	43.3	183	80.6	44	19.4	0	0.0
Part time	11	2.5	16	3.6	26	11.5	5	2.2	0	0.0
Self-employed person, freelancer	21	4.8	44	10.0	18	7.9	4	1.8	0	0.0
Performing domestic tasks, family caregiver	5	1.1	1	0.2	0	0.0	1	0.4	0	0.0
Unemployed	12	2.7	14	3.2	0	0.0	2	0.9	0	0.0
Pupil, student, further training, unpaid work experience	14	3.2	11	2.5	0	0.0	0	0.0	0	0.0
Retired over age 60	143	32.6	148	33.7	0	0.0	0	0.0	232	100.0
Parental leave	18	4.1	15	3.4	0	0.0	169	74.4	0	0.0
Prefer not to answer	1	0.2	0	0.0	0	0.0	2	0.9	0	0.0

Note. ISCED = International Standard Classification of Education.

Data Analysis

All analytical procedures were applied along with judgmental criteria which took into account the theoretical framework as well as practical usefulness of the tool. The analysis was carried out in IBM SPSS 27.0 with IBM SPSS AMOS 27.0. We also used JASP 0.14.1.0 to conduct parallel analysis as well as the calculation of McDonald’s ω and Gutmann’s λ⁶.

In the first step, descriptive statistics including the means, standard deviations, skewness and kurtosis of each item were calculated. EFA using the principal axis factoring (PAF) extraction method was performed on the item correlation matrix of Sample 1. As the correlated factor structure was assumed, oblique (intercorrelated) promax rotation was applied (Whitney et al., 2019). To retain satisfactory variables, the .40–.30–.20 rule was adopted (Howard, 2016). Furthermore, Cattell’s scree plot (Cattell, 1966) and parallel analysis (Horn, 1965) were used to determine the number of factors to retain. Pearson’s correlation with a Bonferroni correction was used to control for multiple testing. To evaluate reliability, Cronbach’s α, McDonald’s ω, and Gutmann’s λ⁶ were compared.

Next, a series of CFAs of the factor structure of the NP-NFE-Q as obtained in EFA was performed on Sample 2. The final model was verified on Samples 3, 4 and 5. Measurement invariance across gender was evaluated as well as the reliability of the final solution.

Results

Our results are presented in terms of the aims of this study as described above. The main objective is reflected in the following three sections, which present an exploration of the NP-NFE-Q factor structure, an evaluation of the structure, and an examination of interrelationships among the factors. Following these sections, we focus on our three additional aims: (1) construct validity of the NP-NFE-Q in terms of the three specific groups; (2) measurement invariance across gender, and (3) reliability of the final instrument.

Exploring the Factor Structure of the NF-NPE-Q

We began with descriptive statistics to check the data before applying the other analytical procedures. The means of all 29 items ranged from 2.61 to 4.54 (on a scale 1–6), with a mean score of 3.60. The values of the standard deviations (SD) of all items ranged from 1.28 to 1.78, with a mean of 1.51. The skewness and kurtosis of all the items did not exceed the value of ± 2 (Trochim & Donnelly, 2006), suggesting acceptable data dispersion.

Before applying EFA, the suitability of the data was checked. The Kaiser–Meyer–Olkin measure (KMO) of sampling adequacy was .89, exceeding the commonly recommended minimal cut-off value of .60. Bartlett’s test of sphericity (x²₍₄₀₆₎ = 5888.72, p < .001) supported the factorability of the correlation matrix.

Based on oblique promax rotation, the examination of the scree plot suggested a five-factor structure, as did the parallel analysis. After comparing these findings with the theory, the five-factor structure was adopted. The initial three-factor (dispositional, situational, institutional) structure was split into five factors while retaining the original framework of the tool, as described in detail below.

A total of 23 items were chosen to remain based on the results of EFA. Out of these items, five scales were created which accounted for 48.75% of the variance, with eigenvalues of 8.49, 3.14, 2.07, 1.59, and 1.33, respectively. The factor loadings and descriptive statistics of each item as well as reliability coefficients are presented in Supplementary Table 2.

The first subscale consisted of 10 items assessing factors of nonparticipation in the form of personal Needs. These included the general lack of pleasant feelings toward learning as well as indications by the respondent of no need to be educated. The second subscale labeled Offer included three items reflecting the low quality of courses, the lack of suitable content, and the lack of information about courses. The third subscale comprised three items related to Work (the respondent’s job position, no need for further education at the respondent’s place of employment, no expectation of further education). The fourth subscale assessed three items connected to Time (no time for further education based on commitments to work, family, hobbies). The fifth subscale labeled Worries contained four items representing fear of failure and other negative self-perceptions with regard to education.

Evaluating the Factor Structure of the NP-NFE-Q

Based on the results of EFA conducted on Sample 1, CFA was carried out on Sample 2 to determine how well the a priori model retained from EFA fit the sample data. Based on common recommendations (Hooper et al., 2008), the comparative fit index (CFI), Tucker–Lewis index (TLI), and root mean square error of approximation (RMSEA) were applied to determine the goodness of fit of the model. The cut-offs for the CFI and TLI indices were set at a value higher than .90 (Hooper et al., 2008). The RMSEA cut-off point was set to an upper limit of .070 (Steiger, 2007). Finally, the values of the Bayesian information criterion (BIC) were used to compare the models, with the lowest value indicating the best model fit (Kass & Raftery, 1995).

Confirmatory factor analysis was first performed on the correlated five-factor model (Model 1) with 23 items retained from EFA, however an acceptable fit was not shown (CFI = .859, TLI = .838, RMSEA = .083, BIC = 1218.245). The next steps of modeling were based on the statistical criteria (modification indices, model fit) as well as judgmental criteria (a comparison of each item’s content with the other items along with an overall conceptualization of the tool) of the model purification (Wieland et al., 2018). Based on previous discussions on correlated errors in structural equation modeling (Gerbing & Anderson, 1984; Hermida, 2015), a purified model containing only the strongest items with no covariations among their error terms was preferred. The practical benefits of this method were brevity and clear interpretability along with higher applicability potential to specific groups without the need to adapt the model. As part of the purification of the tool, we focused on item reductions. Before the elimination of each particular item, it was carefully assessed with regard to the content of the questionnaire and the factors associated with it.

The largest number of item reductions had to be made in the Needs factor. Additional inspection showed the presence of a highly correlated subgroup of items (DF1, DF4, DF9, and DF11) which formed a specific sub-construct within the factor. Due to the low logical cohesion of this subgroup, we decided to remove the items from the model. In the next step, the three statistically as well as judgmentally weakest items IF6, DF6, and DF12 were dropped, reducing the first subscale Needs to the three strongest items: DF7, DF8 and DF10. In addition, the item SF4 in the Worries subscale showed low factor loading (below .40). Reflecting the overall content of this factor, the item was evaluated as too specific and was removed. The revised correlated five-factor model (Model 2) which retained a reduced number of 15 items was tested. This model fulfilled the statistical (CFI = .961, TLI = .949, RMSEA = .051, BIC = 413.396) as well as judgmental criteria described above.

Examination of the Interrelationships Among the NP-NFE-Q factors

The internal division of items into factors preserved the theoretically intended structure: dispositional (Needs and Worries), institutional (Offer) and situational (Work and Time). As part of the main objective of this study, our intention was to empirically verify whether this three-factor structure (see Figure 1) could serve as the basis for the NP-NFE-Q. A correlation analysis (see Supplementary Table 3), however, did not fully support a three-factor structure. Using Cohen (1988) conventions to interpret effect sizes, the correlation between Needs and Worries was moderate (r = .47, p < .001), while the correlation between Work and Time was small (r = .13, p < .001). Thus, the correlation analysis did not fully support the connection of the Work and Time subscales into one situational factor. On the contrary, the correlations of the five dimensions with the overall construct were large (r = .50 to .74, p < .001), which suggested the possibility of connecting all the subscales into one overall construct. In addition to the current five-factor model (Model 2), two other variants were composed and tested by CFA: one featuring a connection of five factors into three (Model 3), the other connecting five factors into one second-order factor (Model 4). A basic graphical presentation of the three potential structures of interrelationships among the factors of the NP-NFE-Q can be found in Figure 1.

Figure 1.

Potential structures of the interrelationships among the factors of the NP-NFE-Q. Note. A = Revised correlated five-factor model (Model 2); B = Second-order three-factor model (Model 3); C = Second-order one-factor model (Model 4).

The five-factor structure has been presented above (Model 2). As the next step, an attempt was made to compose and test a second-order three-factor structure (Model 3). The factors Needs and Worries were connected into one second-order dispositional factor (DF); Offer represented the institutional factor (IF); Needs and Worries were connected into one second-order situational factor (SF). As the model fit indices of this version proved to be worse (CFI = .947, TLI = .933, RMSEA = .058, BIC = 431.828), the three-factor solution was rejected. As a final step, we attempted to construct and test a second-order one-factor model (Model 4) connecting the five factors into one broader overall factor: Barriers. The quality of this model also proved worse in comparison to Model 2 (CFI = .943; TLI = .930; RMSEA = .060; BIC = 429.981), so it was also rejected. All of these results of CFA regarding model fit are displayed in Table 2.

Table 2.

Goodness of Fit of CFAs for Sample 2 (n = 439).

Model	No	x ²	df	CFI	TLI	RMSEA	BIC
1. Correlated five-factor model	23	877.513	220	.859	.838	.083	1218.245
2. Revised correlated five-factor model	15	170.016	80	.961	.949	.051	413.396
3. Second-order three-factor model	15	206.702	83	.947	.933	.058	431.828
4. Second-order one-factor model	15	217.023	85	.943	.930	.060	429.981

Note. No. = number of items; χ2 = Chi-Square statistic; df = degrees of freedom; p < .001.

Although Models 3 and 4 produced a good fit, Model 2 yielded the best fit (see Table 2), so Model 2 was accepted based on the model’s fit indices as well as its simplicity and wide intelligibility. A graphical representation of all models with the estimated parameters of each can be found in Supplementary Figure 2.

Evaluating Construct Validity for the Three Specific Groups

To fulfill the first additional aim of this study, Model 2 was tested separately for each of the three specific groups of non-participants in NFE: (a) low-educated workers (Sample 3), (b) persons caring for children under the age of three (Sample 4), and (c) retired persons over 60 years old (Sample 5). The results showed stability across all the samples (CFI > .900; TLI > .900; RMSEA < .070). This indicates a qualitatively invariant measurement pattern of factors for Model 2, thus the questionnaire can be applied both to the general adult population and to the three specific groups. These results can be found in Table 3.

Table 3.

Goodness of Fit of the Model T2 for Samples 3 (n = 227), 4 (n = 227), and 5 (n = 232).

Sample	x ²	df	CFI	TLI	RMSEA
Sample 3: Low-educated workers	149.943	80	.952	.936	.062
Sample 4: Persons caring for children	156.019	80	.936	.916	.065
Sample 5: Retired persons	156.404	80	.946	.929	.064

Note. p < .001.

Measurement Invariance of the NP-NFE-Q Across Gender

To fulfill the second additional aim of this study, we used Sample 2 to evaluate the NP-NFE-Q in terms of measurement invariance across gender. Three levels of invariance were evaluated (Xu & Tracey, 2017): configural (a qualitatively invariant measurement pattern of factors), metric (a quantitatively invariant measurement model of factors) and scalar (the invariant mean levels of item intercepts). These results are presented in Table 4.

Table 4.

Goodness of Fit of Invariance Tests of the Model 2 for the Sample 2 (n = 439).

Invariance Test	x ²	df	CFI	ΔCFI	TLI	RMSEA
Configural	254.259	160	.960	-	.948	.052
Metric	278.755	170	.954	.006	.943	.054
Scalar	288.892	180	.954	.006	.946	.052

Note. p < .001.

In line with Cheung and Rensvold (2002), the differences among the values of CFI were used to interpret the results (with values lower than 0.01 considered as a sign of invariance). The biggest difference was shown between the configural and metric levels (ΔCFI = 0.006), with no difference found between the metric and scalar levels. Thus we may conclude that Model 2 of the NP-NFE-Q is invariant across gender up to the scalar level. At this level of measurement invariance, the factor means can be compared.

Reliability of the NP-NFE-Q

In addition to presenting the final model, Supplementary Table 4 summarizes the reliability coefficients for samples 2, 3, 4, and 5. Although reliability is sample size sensitive, and the model does contain a relatively small number of scale items, the subscales were expected to be reliable with coefficients exceeding the value .70 (DeVellis, 2017). Although the values we obtained were not always well above the recommended acceptable range, we consider the obtained values satisfactory in terms of how they reflect expert discussions regarding the usefulness of reliability coefficients with scales containing a low number of items (Sijtsma, 2009; Taber, 2018).

Discussion

With this research study, we aimed to create and initially validate a questionnaire to measure factors that prevent the adult population from participating in NFE. In the construction of this novel questionnaire—the NP-NFE-Q—we sought to address the lack of an instrument to measure educational barriers from an assessed perspective (Boeren, 2018, 2019). In line with current research (Roosmaa & Saar, 2017; Rubenson, 2011, 2018; Rubenson & Desjardins, 2009; Van Nieuwenhove & De Wever, 2021), a correlated five-factor solution was finally identified as most suitable. The incorporation of non-cognitive factors (Offer, Work, and Time) with cognitive factors (Needs and Worries) is consistent with arguments regarding the multiple individual and structural factors that influence participation in NFE (Boeren, 2016; Desjardins, 2017; Desjardins & Ioannidou, 2020).

The importance of the pioneering work of Cross (1981) in terms of introducing questions related to why adults participate (or not) in educational programs cannot be denied. In particular, her identification of three independent types of barriers to adult learning serve as a useful starting-off point for further research. Nevertheless, our results indicate that the structure of barriers can be described in a more complex way. Barrier construction and perception, for example, has been significantly connected to the context of employment (see the factor Work above). The increasing prevalence of job-related learning in contemporary NFE has influenced recent theory linking work conditions to adult nonformal education (Desjardins & Ioannidou, 2020; Rubenson, 2018).

Our results do not empirically confirm the strict use of Cross’s three-factor model as it was originally presented in the 1980s. The five-factor solution we have put forward might be integrated into a more general model which complements the three original categories proposed by Cross (see Figure 1). In this context, our factor of Offer represents institutional barriers to NFE; Work and Time can be related to situational obstacles; and Needs and Worries correspond to various aspects of dispositional barriers.

Whereas some authors (Darkenwald & Valentine, 1985; Larson & Milana, 2006; Valentine & Darkenwald, 1990) have proposed the influence of the independent factor of financial barriers, our results indicate that this factor does not play a significant role. One possible explanation for this lies in the high financial support of NFE by employers, which has been previously demonstrated in other research as specific to the Czech Republic (Kalenda & Kočvarová, 2020). As a consequence, the lack of financial resources does not seem to form a substantial barrier to participation, at least not in the Czech context. Some previous studies (Desjardins, 2017; Rubenson, 2011, 2018) have pointed out that information availability about NFE should represent another independent factor. Our data support connecting availability with information about NFE and its quality into the one factor Offer.

Another important finding of our study is the suitability of the NP-NFE-Q for the three specific groups of non-participants in NFE we identified. In addition, measurement invariance was found across gender and the instrument was evaluated as reliable. One more advantage of this tool is that it employs a concise and clear factor structure (i.e., results can be quickly and easily interpreted in practical usage). The questionnaire can be administered in diverse lifelong settings and across various samples by researchers and adult education specialists, for example, within companies by HR professionals, among managers and lectors as well as by employers seeking to invest in the skills enhancement of their employees (Kyndt et al., 2011). The tool is also applicable for counseling purposes within the framework of social services for low-educated adults, the elderly as well as individuals caring for children, who may face unemployment, social exclusion, etc.

A rapid evolution is taking place in the types and causes of barriers to adult NFE. Although the relatively simple three-factor structure of barriers created by Cross (1981) once worked well, the issues involved are becoming more complex, as new dimensions of barriers are being identified and the specifics of different groups are changing. These include questions regarding the inclusivity of potentially marginalized populations such as workers facing unemployment in the digital economy (Desjardins, 2017; Roosmaa & Saar, 2017; Rubenson, 2018; Hovdhaugen & Opheim, 2018). We hope that our novel, empirically verified research tool along with the knowledge presented here in our study can be used not only in the Czech Republic but also internationally. The NP-NFE-Q can be further modified or extended to further identify, describe and verify barriers towards adult participation in NFE. The NP-NFE-Q might be used to supplement data from other surveys or as an alternative measurement tool.

Limitations and Directions for Future Research

First, by no means does the current five-factor version of the NP-NFE-Q represent full and comprehensive coverage of all the variables and factors underlying lifelong learning or even nonparticipation in NFE. Several groups of researchers have provided evidence that other factors may become prominent in the future, especially in the years to come as many economies continue to adapt to Industry 4.0 (fourth industrial revolution in the sense of digitalization, automation and related changes in the labor market). As an example of these evolving perspectives, even within the verification process of our questionnaire the financial situation was originally reflected among the situational (SF4) as well as institutional (IF4) barriers. Finally neither of the associated items was retained in the final Model 2 due to their specificity. Other specific barriers such as health issues (represented by item SF3) and the distance of NFE from the place of residence (SF7) also did not fit into the factor structure. As indicated, it is clear that the factor structure of the current NP-NFE-Q may be supplemented or modified in the future on the basis of further reconstruction and re-validation of the tool. It would also be possible to develop specialized surveys reflecting particular target populations and/or specific sets of barriers to participation in NFE.

Another limitation to our study is its basis on research findings from the perspective of one country. Though we generated a representative sample, the research in this respect lacked cultural diversity, which may influence the generalizability of the results. It is anticipated that further utilization of the NP-NFE-Q in various contexts across diverse cultural samples will yield reliable and repeatable results.

In our study, the use of the term “initial validation” accentuates the fact that this work does not represent a comprehensive validation. The full validation of the tool will entail a gradual, long-term process through many research projects based on various judgmental as well as statistical criteria. Concurrent validity of the NP-NFE-Q is not evaluated here, as no related standardized instruments were applied. Nor was the question of predictive or other kinds of validity addressed in this study. All of these research targets may be included in the further development and validation of the NP-NFE-Q.

Despite the research limitations listed above, the authors hope that the development of the current version of the NP-NFE-Q will serve as a useful step toward addressing a significant gap in the literature by facilitating more specifically targeted research into nonparticipation in NFE.

Supplemental Material

sj-pdf-1-jpa-10.1177_07342829211060571 – Supplemental Material for Development and Initial Validation of the Nonparticipation in Nonformal Education Questionnaire

Supplemental Material, sj-pdf-1-jpa-10.1177_07342829211060571 for Development and Initial Validation of the Nonparticipation in Nonformal Education Questionnaire by Ilona Kočvarová, Jitka Vaculíková, and Jan Kalenda in Journal of Psychoeducational Assessment

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Czech Science Foundation (GACR) through the project Blind Spots of Nonformal Education in the Czech Republic: Non-participants and their Social Worlds, GA_19-00987S.

ORCID iD

Ilona Kočvarová

Supplementary Material

Supplementary material for this article is available online.

References

Albert

García-serrano

Hernanz

(2010). On-the-job training in Europe: Determinants and wage returns. International Labour Review, 149(3), 315–341. https://doi.org/10.1111/j.1564-913X.2010.00089.x

Baert

De Rick

Van Valckenborgh

(2006). Towards the conceptualisation of learning climate. In Vieira de Castro

Sancho

A. V.

Guimaraes

(Eds.), Adult education: New routes new landscapes (pp. 87–111). University de Minho.

Berger

Frey

(2016) Structural transformation in the OECD: Digitalisation, deindustrialisation and the future of work. Oxford Martin School. https://www.oxfordmartin.ox.ac.uk/publications/structural-transformation-in-the-oecd-digitalisation-deindustrialisation-and-the-future-of-work/

Blunt

Yang

(2002). Factor structure of the adult attitudes toward adult and continuing education scale and its capacity to predict participation behaviour: Evidence for adoption of a revised scale. Adult Education Quarterly, 52(4), 299–314. https://doi.org/10.1177/074171302400448627

Boeren

(2011). Gender differences in formal, non-formal and informal adult learning. Studies in Continuing Education, 33(3), 333–346. https://doi.org/10.1080/0158037X.2011.610301

Boeren

(2016). Lifelong learning participation in a changing policy context. An interdisciplinary theory. Palgrave Macmillan. https://doi.org/10.1057/9781137441836

Boeren

(2018). The methodological underdog: A review of quantitative research in the key adult education journals. Adult Education Quarterly, 68(1), 63–79. https://doi.org/10.1177/0741713617739347

Boeren

(2019). Quantitative research in research on the education and learning of adults. In Fejes

Nylander

(Eds.), Mapping out the resarch field of adult education and learning (pp. 139–155). Springer. https://doi.org/10.1007/978-3-030-10946-2_8

Boeren

Holford

(2016). Vocationalism varies (a lot): A 12-country multivariate analysis of participation in formal adult learning. Adult Education Quarterly, 66(2), 120–142. https://doi.org/10.1177/0741713615624207

10.

Boeren

Holford

Nicaise

Baert

(2012). Why do adults learn? Developing motivational typology across twelve European countries. Globalisation, Societies and Education, 10(2), 247–269. https://doi.org/10.1080/14767724.2012.678764

11.

Boeren

Roumell

E. A.

Roessger

K. M.

(2020). COVID-19 and the future of adult education: An editorial [Editorial]. Adult Education Quarterly, 70(3), 1–4. https://doi.org/10.1177/0741713620925029

12.

Brynjolfsson

McAfee

(2014). The second machine age: Work, progress, and prosperity in a time of brilliant technologies. W W Norton & Co.

13.

Busemeyer

Trampusch

Ch.

(Eds.). (2012). The political economy of collective skill formation. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199599431.001.0001

14.

Cattell

R. B

(1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 2, 245–276. https://doi.org/10.1207/s15327906mbr0102_10

15.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5

16.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Routledge. https://doi.org/10.4324/9780203771587

17.

Courtney

(1992). Why do adults learn? Towards a theory of participation in adult education. Routledge.

18.

Cross

K. P.

(1981). Adults as learners. Increasing participation and facilitating learning. Jossey-Bass.

19.

Czech Statistical Office . (2014). Czech republic in figures - 2013. https://www.czso.cz/documents/10180/20556813/32020414.pdf/2bed0885-54d8-44d1-8529-6ac3466da01d?version=1.0

20.

Dämmrich

Kosyakova

Blossfeld

H.-P.

(2015). Gender and job-related non-formal training: A comparison of 20 countries. International Journal of Comparative Sociology, 56(6), 433–459. https://doi.org/10.1177/0020715215626769

21.

Darkenwald

G. G.

Valentine

(1985). Factor of deterrents to public participation in adult education. Adult Education Quarterly, 35(4), 177–193. https://doi.org/10.1177/0001848185035004001

22.

Desjardins

(2017). Political economy of adult learning systems. Comparative study of strategies, policies, and constraints. Bloomsbury.

23.

Desjardins

Ioannidou

(2020). The political economy of adult learning systems: some institutional features that promote adult learning participation. Zeitschrift für Weiterbildungsforschung, 43, 2, 143–168. https://doi.org/10.1007/s40955-020-00159-y

24.

DeVellis

R. F.

(2017). Scale development: Theory and applications (4th ed.). Sage.

25.

European Statistical Office . (2016). Classification of learning activities (CLA) manual (2016th ed.). Publications Office of the European Union. https://doi.org/10.2785/874604

26.

European Statistical Office (2021). Adult education survey. Reference metadata in Euro SDMX metadata structure (ESMS). https://ec.europa.eu/eurostat/cache/metadata/en/trng_aes_12m0_esms.htm

27.

Furr

R. M.

(2018). Psychometrics: An introduction (3rd ed.). Sage.

28.

Gerbing

D. W.

Anderson

J. C.

(1984). On the meaning of within-factor correlated measurement errors. Journal of Consumer Research, 11(1), 572–580. https://doi.org/10.1086/208993

29.

Hermida

(2015). The problem of allowing correlated errors in structural equation modeling: Concerns and considerations. Computational Methods in Social Sciences, 3(1), 1–17.

30.

Hooper

Coughlan

Mullen

(2008). Structural equation modelling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6(1), 53–60.

31.

Horn

J. L.

(1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179–185. https://doi.org/10.1007/BF02289447

32.

Hovdhaugen

Opheim

(2018). Participation in adult education and training in countries with high and low participation rates: Demand and barriers. International Journal of Lifelong Education, 37(5), 560–577. https://doi.org/10.1080/02601370.2018.1554717

33.

Howard

M. C.

(2016). A review of exploratory factor analysis decisions and overview of current practices: What we are doing and how can we improve? International Journal of Human-Computer Interaction, 32(1), 51–62. https://doi.org/10.1080/10447318.2015.1087664

34.

Iñiguez-Berrozpe

Elboj-Saso

Flecha

Marcaletti

(2020). Benefits of adult education participation for low-educated women. Adult Education Quarterly, 70(1), 64–88. https://doi.org/10.1177/0741713619870793

35.

International Chamber of Commerce/ESOMAR . (2016). ICC/ESOMAR international code on market, opinion and social research and data analytics. ESOMAR. https://www.esomar.org/uploads/pdf/professional-standards/ICCESOMAR_Code_English_.pdf

36.

International Standard Classification of Education . (2011). International standard classification of education (ISCED) 2011. UNESCO Institute for Statistics. http://uis.unesco.org/sites/default/files/documents/international-standard-classification-of-education-isced-2011-en.pdf

37.

Iversen

Soskice

(2019). Democracy and prosperity. Reinventing capitalism through turbulent century. Princeton University Press.

38.

Kalenda

Kočvarová

(2020). Participation in non-formal education in risk society. International Journal of lifelong education. Advance online publication. https://doi.org/10.1080/02601370.2020.1808102

39.

Kalenda

Kočvarová

Vaculíková

(2020).Determinants of participation in nonformal education in the Czech Republic. Adult Education Quarterly, 70(2), 99–118. https://doi.org/10.1177/0741713619878391

40.

Karger

Kalenda

Kroutilová Nováková

(2021). How Disadvantaged Groups Legitimize Non-participation in Adult Education and Training: The Situational Logic of Decision-Making. [Manuscript submitted for publication]. Zlín: Research Centre of FHS, Tomas Bata University.

41.

Kass

R. E.

Raftery

A. E.

(1995). Bayes factors. Journal of the American Statistical Association, 90(430), 773–795. https://doi.org/10.2307/2291091

42.

Kyndt

Dochy

Onghena

Baert

(2013). The learning intentions of low-qualified employees: A multilevel approach. Adult Education Quarterly, 63(2), 165–189. https://doi.org/10.1177/0741713612454324

43.

Kyndt

Govaerts

Dochy

Baert

(2011). The learning intention of low-skilled employees: A key for participation in lifelong learning and continuous training. Vocations and Learning, 4(3), 211–229. https://doi.org/10.1007/s12186-011-9058-5

44.

Larson

Milana

(2006) Barriers towards participation in adult education and training [Paper presentation]. European Conference on Educational Research, University of Geneva.

45.

Merriam

S. B.

Baumgartner

L. M.

(2020). Learning in adulthood. A comprehensive guide. Josssey-Bass.

46.

Mulenga

Liang

J. S.

(2008). Motivations for older adults’ participation in distance education: A study at the National Open University of Taiwan. International Journal of Lifelong Education, 27(3), 289–314. https://doi.org/10.1080/02601370802047791

47.

Organisation for Economic Co-operation and Development . (2018). The future of education and skills. Education 2030. https://www.oecd.org/education/2030/E2030%20Position%20Paper%20(05.04.2018)

48.

Organisation for Economic Co-operation and Development . (2019) Getting skills right: Future-ready adult learning systems. https://www.oecd.org/fr/publications/getting-skills-right-future-ready-adult-learning-systems-9789264311756-en.htm

49.

Rambla

Milana

(2020). The stepping stones of lifelong learning policies: Politics, regions and labor market. International Journal of Lifelong Education, 39(1), 1–4. https://doi.org/10.1080/02601370.2020.1747589

50.

Rees

(2013). Comparing adult learning systems: An emerging political economy. European Journal of Education, 48(2), 200–212. https://doi.org/10.1111/ejed.12025

51.

Regmi

K. D.

(2015). Lifelong learning: Foundational models, underlying assumptions and critiques. International Review of Education, 61(1), 133–151. https://doi.org/10.1007/s11159-015-9480-2

52.

Roosmaa

E.-L.

Saar

(2017). Adults who do not want to participate in learning: A cross-national European analysis of their barriers. International Journal of Lifelong Education, 36(3), 254–277. https://doi.org/10.1080/02601370.2016.1246485

53.

Rubenson

(2011). Barriers to participation in adult education. In Rubenson

(Ed.), Adult learning and education (pp. 216–224). Elsevier.

54.

Rubenson

(2018). Conceptualising participation in adult learning and education. Equity issues. In Milana

(Ed.), The Palgrave international handbook on adult and lifelong education and learning (pp. 337–357). Palgrave. https://doi.org/10.1057/978-1-137-55783-4_18

55.

Rubenson

Desjardins

(2009). The impact of welfare state requirements on barriers to participation in adult education: A bounded agency model. Adult Education Quarterly, 59(3), 187–207. https://doi.org/10.1177/0741713609331548

56.

Sijtsma

(2009). On the use, the misuse, and the very limited usefulness of cronbach’s alpha. Psychometrika, 74(1), 107–120. https://doi.org/10.1007/s11336-008-9101-0

57.

Smit

Tacke

Lund

Manyika

Thiel

(2020). The future of work in Europe. Automation, workforce transitions, and shifting geography of employment. Mckinsey Global Institute. https://www.mckinsey.com/∼/media/mckinsey/featured%20insights/future%20of%20organizations/the%20future%20of%20work%20in%20europe/mgi-the-future-of-work-in-europe-discussion-paper.pdf

58.

Steiger

J. H.

(2007). Understanding the limitations of global fit assessment in structural equation modelling. Personality and Individual Differences, 42(5), 893–898. https://doi.org/10.1016/j.paid.2006.09.017

59.

Taber

K. S.

(2018). The use of Cronbach’s Alpha when developing and reporting research instruments in science education. Research in Science Education, 48, 6, 1273–1296. https://doi.org/10.1007/s11165-016-9602-2

60.

Trochim

W. M.

Donnelly

J. P.

(2006). The research methods knowledge base (3rd ed.). Atomic Dog.

61.

United Nations Educational Scientific and Cultural Organization . (2019). 4th global report on adult learning and education. Leave no one behind: Participation, equity and inclusion. https://reliefweb.int/report/world/4th-global-report-adult-learning-and-education-leave-no-one-behind-participation-equity

62.

United Nations Educational, Scientific and Cultural Organization . (2020). Embracing a culture of lifelong learning. Contribution to the futures of education initiative. https://unesdoc.unesco.org/ark:/48223/pf0000374112

63.

Vaculíková

Kalenda

Kočvarová

(2020). Hidden gender differences in formal and non-formal adult education. Studies in Continuing Education, 43(1), 1–15. https://doi.org/10.1080/0158037X.2020.1732334

64.

Valentine

Darkenwald

G. G.

(1990). Deterrents to participation in adult education: Profiles of potential learners. Adult Education Quarterly, 41(1), 29–42. https://doi.org/10.1177/0001848190041001003

65.

Van Nieuwenhove

De Wever

(2021). Why are low-educated adults underrepresented in adult education? Studying the role of educational background in expressing learning needs and barriers. Studies in Continuing Education. Advance online publication. https://doi.org/10.1080/0158037X.2020.1865299

66.

Waller

Hodge

Holford

Milana

Webb

(2020). Lifelong education, social inequality and the COVID-19 health pandemic. International Journal of Lifelong Education, 39(3), 243–246. https://doi.org/10.1080/02601370.2020.1790267

67.

Whitney

B. M.

Cheng

Brodersen

A. S.

Hong

M. R.

(2019). The scale of student engagement in statistics: Development and initial validation. Journal of Psychoeducational Assessment, 37(5), 553–565. https://doi.org/10.1177/0734282918769983

68.

Wieland

Kock

Josiassen

(2018). Scale purification: State-of-the-art review and guidelines. International Journal of Contemporary Hospitality Management, 30(11), 3346–3362. https://doi.org/10.1108/IJCHM-11-2017-0740

69.

Tracey

T. J. G.

(2017). Use of multi-group confirmatory factor analysis in examining measurement invariance in counseling psychology research. The European Journal of Counselling Psychology, 6(1). 75–82. https://doi.org/10.5964/ejcop.v6i1.120

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.52 MB