The Methodological Underdog: A Review of Quantitative Research in the Key Adult Education Journals

Abstract

An examination of articles published in leading adult education journals demonstrates that qualitative research dominates. To better understand this situation, a review of journal articles reporting on quantitative research has been undertaken by the author of this article. Differences in methodological strengths and weaknesses between quantitative and qualitative research are discussed, followed by a data mining exercise on 1,089 journal articles published in Adult Education Quarterly, Studies in Continuing Education, and International Journal of Lifelong Learning. A categorization of quantitative adult education research is presented, as well as a critical discussion on why quantitative adult education does not seem to be widespread in the key adult education journals.

Keywords

research paradigms prevalence of quantitative research quantitative scales secondary data analysis quantitative research in adult education

Introduction

This article aims to explore the nature of quantitative research in adult education. There is limited presence of quantitative research published in the leading journals in the field, such as within Adult Education Quarterly, Studies in Continuing Education, and International Journal of Lifelong Education. Exploring research methodologies and methods is important to understand the leading frameworks in which adult education research is currently being conducted and the ways in which new insights are added to the knowledge base. It is not new that empirical studies in the field, as published in leading journals, tend to be dominated by qualitative research approaches (for a discussion, see Fejes & Nylander, 2015). As a scholar active in engaging in quantitative research, I aim to provide a synthesis and review of research tools available for adult education researchers from a quantitative perspective. This article briefly discusses different research paradigms, methodologies, and methods as discussed in the academic methodological literature in order to locate quantitative research’s place in the “methodological jungle,” but I start with discussing a range of hypotheses on why quantitative research does seems to be underrepresented in the leading adult education journals.

Hypotheses on the Limited Presence of Quantitative Research

Before turning to the overview of what quantitative research has to offer to the field of adult education, I start by discussing a range of hypotheses on why quantitative research is clearly less present in the leading journals in the field. Fejes and Nylander (2015) undertook a bibliometric analysis on the top cited articles in Adult Education Quarterly, International Journal of Lifelong Education, and Studies in Continuing Education and concluded that “qualitative approaches have near total dominance.” They included 57 articles in their analysis and found that 7 of these 57 articles included a quantitative component. The empirical aspects of these seven articles were either purely quantitative or part of a multistrategy design, combining qualitative and quantitative methods (Robson, 2011). Apart from providing data on methodological approaches, the authors also discussed potential explanations for the lack of research using quantitative methods. In general, I tend to agree with these hypotheses, although I would like to offer some comments as well. For example, Fejes and Nylander (2015) discuss the intake of doctoral candidates in the field who are often coming from a practical background, therefore, likely more interested in capturing the experiences of adult learners, more likely to result in the choice to adopt qualitative methods. Although quantitative research is also able to ask about experiences, it is more likely to provide an overview of “what” they are feeling, instead of “why” they are experiencing these feelings, because of the different nature of questions to be answered when using quantitative research approaches, generally more focusing on static objective data instead of subjective meanings (see Robson, 2011). Researching “experiences” might thus profit most from qualitative approaches, although it is also possible to combine it with existing quantitative scales, as will be clear from my discussion of research instruments below. Also, professors currently supervising these doctoral students, explain Fejes and Nylander (2015), were most likely trained within an era where qualitative methods gained popularity as a reaction to quantitative positivist ideas, perceiving truth as something that can be objectively verified. Based on my personal experience of visiting conferences and discussing work with scholars in the field, I have indeed noticed the dominant qualitative expertise of colleagues, and it is, therefore, thus not surprising that this mirrors the research output published in leading journals. As will be discussed later, I will also confirm Fejes and Nylander’s findings that quantitative research published in the leading journals is mainly undertaken by scholars in the United States and that this seems to limit the presence of quantitative studies published in the International Journal of Lifelong Education and Studies in Continuing Education, edited in Europe and Australia. Furthermore, argue Fejes and Nylander (2015), exploring the specific aims of the journals, they all make specific reference to the “relation between theory and practice.” While I believe certain types of quantitative research, for example, experiments or surveys drawing on psychometric scales have the potential to inform practice too, it is important to further discuss the opportunities of doing so and to raise awareness among scholars of what quantitative research can and cannot inform about. Finally, the authors of the review (Fejes & Nylander, 2015) point out that in a difficult funding climate, it is hard to obtain large pots of money to conduct extensive quantitative studies, for example, longitudinal studies. This might be one of the reasons why quantitative research is less present, but as I will argue later, there is a wide range of opportunities to work with available secondary data sets free of charge, although it is needed to work with these survey data with a critical approach, as will be discussed in the section on secondary data analysis below.

Before discussing a range of quantitative tools available for researchers based on studies published in the leading journals in the field, I provide a brief overview on historical discussions between the role of qualitative versus quantitative research.

Research Paradigms

Thomas (2009, p. 72) defines paradigms as “the technical word used to describe the ways we think about and research the world.” He goes on that, following his reading of the methodological literature, the “leading” paradigms in social sciences are “positivism” and “interpretivism.” It should be noted that other authors discuss the “paradigm landscape” in a more sophisticated way, for example, Denzin and Lincoln (2003) who distinguish between “positivism and postpositivism,” “interpretivism, constructivism, and hermeneutics,” “feminism,” “radicalized discourses,” “critical theory and Marxist models,” “cultural studies” and “queer theory,” going beyond the binary divide between “positivism” and “interpretivism” as discussed by Thomas (2009). Space is limited here to go into detail on all of these separate paradigms, but for a detailed overview on these paradigms aforementioned, Denzin and Lincoln (2003) can be consulted.

Focusing on the core messages, what Thomas (2009), as well as other authors discussing paradigms, want to bring out is that the way we “think about and research the world” is affecting the way we decide on our research approaches. Positivists, for example, as discussed by Thomas (2009), start from the assumption that knowledge can be obtained in an objective and value-free way, based on facts and figures. Assumptions and methods are therefore generally borrowed from exact sciences and tend to be concentrated around the use of quantitative methods in order to test or reject a set of hypotheses. It is, thus, the deductive hypothesis testing type of research interested in presenting “objective” facts and figures that is perceived as being suitable for quantitative research focusing on questions such as how many, what percentage, and so on. Interpretivists, on the other hand, start from the perspective of individuals constructing and interpreting the world. Interpretivists’ work is concerned with how people are making sense of the world and thus not with the believed objective realities as featured in positivism. Their work is therefore more likely to draw on qualitative methodologies.

In practice, research often combines elements of quantitative and qualitative approaches instead of strictly separating them. The “paradigm war,” involving academic arguments between those proquantitative and antiquantitative positivist approaches, has been widely discussed in the methodological literature, especially during the 1970s and 1980s of the previous century (see e.g., Gage, 1989; Robson, 2011). These discussions have also led to the discussion of “mixed-methods research,” which could be perceived as an additional paradigm according to Cohen, Manion, and Morrison, (2011), and which has been labelled as “the third methodological movement” in the work of Johnson, Onwuegbuzie, and Turner (2007) and Teddlie and Tashakkori (2009) or as the “pragmatic” approach (Robson, 2011). Nowadays, it is believed that there is a tendency for researches to adopt the research approaches best suited to answer their research questions and to avoid polarizing between quantitative versus qualitative approaches, but to focus on their complementarities, or to use mixed methods to answer different research questions relating to the same phenomena (Ercikan & Roth, 2006; Teddlie & Tashakkori, 2009).

Going back to the field of adult education, we know publications in leading journals are dominated by qualitative research approaches and that quantitative research is the “methodological underdog” (see e.g., Fejes & Nylander, 2015). In order to deepen knowledge on the use of quantitative research in adult education, it is important to undertake a review of existing quantitative work with the aim to better understand its use in the field.

Review Procedure Distinguishing Between Quantitative and Qualitative Approaches

Before going more into detail about the nature of specific examples of quantitative research in the field of adult education, I outline the procedures I followed to undertake this review analysis. First of all, I had to decide which keywords would fall under the methodological group of quantitative research and what would count as qualitative research. In order to make a decision informed by the methodological literature, despite categorizations always being artificial to a certain extent, I decided to focus on the distinction made by Creswell (2003). Quantitative research methods therefore refer to data that are gathered using “predetermined” instruments such as questionnaires, although data can also be obtained through, for example, experiments. Quantitative research methods are characterized by the fact that these data are being subjected to statistical analyses. On the other hand, qualitative research methods start from questions which tend to be more “open” and additional ideas for data collection can emerge during the data collection phase. Data can be gathered using a range of methods, including interviews, focus groups, or observations. Analysis of these data tends to be text-based. Mixed-methods research approaches combine elements of both quantitative and qualitative methods.

Based on 1,089 journal articles, all published between 2000 and 2014 in the leading adult education journals, the keywords qualitative, quantitative, interview, focus group, participant observation, questionnaire, regression, correlation, analysis of variance (examples of common statistical analyses), and (quasi-)experimental design were searched for to find out which methodological words—based on Creswell (2003)—were mostly used in the texts. An additional search was included for the term “mixed methods.” All original articles published in Adult Education Quarterly, Studies in Continuing Education, and International Journal of Lifelong Education in the past 15 years—from 2000 till 2014—have been included in the analysis (N = 1,089), including more than 6 million words of text. These journals were included in order to keep the selection similar to previous research undertaken by Fejes and Nylander (2015), as such, building further on their finding that quantitative research is underrepresented in the leading academic lifelong learning journals.

The data were subjected to a context and text mining analysis undertaken with the help of software packages QDA Miner and WordStat, products developed by Provalis Research. QDA Miner is able to code, analyze, and manage big data—in this case, all articles from the three leading journals between 2000 and 2014—and can be linked to Wordstat, which is able to undertake further analyses on the data, such as exploring co-occurences between keywords, for example, through cluster analyses presented in dendograms—building taxonomies of keywords—or through proximity plots that map the co-occurrence of specific keywords with chosen target keywords. In short, the program has done a search on all sentences in all articles that contain the different keywords. Afterward, I have explored articles that mention specific data collection methods to distinguish whether these were used as part of the literature review or discussion, or whether the article reflected on empirical research using these methods. A straightforward example of this is searching for the word “percentage,” which is largely used in, for example, contextual and background section of an article, without, therefore, being an article drawing on quantitative methods. This is also the way in which data are being reported in the section below.

As explained before, the three journals are chosen because of their longstanding contribution to the field and to keep the selection of journals parallel to the review undertaken by Fejes and Nylander (2015). Additionally, the journals represent editorial responsibility in three different continents. Adult Education Quarterly is an American journal, International Journal of Lifelong Education is edited in the Europe, and Studies in Continuing Education in Australia.

Results

General Patterns

This results section discussing the prevalence of quantitative research in the three leading adult education journals starts by demonstrating the underrepresentation of articles mentioning the use of quantitative research approaches (see Table 1).

Table 1.

Patterns of Qualitative Versus Quantitative Research Approaches.

Qualitative (n = 498)	n	Quantitative (n = 191)	n
Interview	429	Questionnaire	131
Focus group	68	Regression	38
Participant observation	32	Correlation	26
Mixed methods	8	Analysis of variance	17
		Quasi-experimental design	3

Source. Own analysis.

The numbers reported in this table represent the number of cases (journal articles) in which one of these words has appeared, with an additional scrutinizing exercise for the keywords reflecting on specific data collection methods. It does not reflect how many times this word has been mentioned in the 1,089 articles, but reflects on the number of articles that use these methods. Although this is a keyword search only, it does provide us with a first impression that research reporting on qualitative research is significantly more common than on quantitative research, which is a confirmation of what Fejes and Nylander (2015) found as well, although their analyses were based on top cited articles only. Furthermore, it is also clear that a high volume of articles does not contain any of these keywords at all, as 1,089 articles were taken into account. Publications in adult education journals are therefore not automatically empirical in nature, but can also take the form of, for example, policy or theoretical reviews.

This new analysis thus includes all articles of the past 15 years in the same journals, but the conclusion about the dominance of qualitative research approaches remains valid. Especially more specialized quantitative terms such as “regression” only appeared in 38 journal articles (3.5% of the entire database), a keyword one would expect to see in a range of quantitative studies. Correlation was mentioned in 26 articles, of which 11 also mentioned regression analyses. In general, it seems that the majority of qualitative projects is based on interview studies, the majority of quantitative projects on questionnaire studies without engaging in advanced statistical analyses of the data. Experimental designs have been searched for but seem to be mostly absent from the adult education literature as published in the leading journals. Only three articles mentioned they were the result of quasi-experimental research and it is also interesting to see how “mixed-methods” studies are not that strongly represented in the leading adult education journals.

Although quantitative research is thus not well represented in adult education research, it is important to understand what we can learn from existing research to improve the quality of our own research. In short, there are two ways in which scholars can deal with quantitative data: (a) based on primary data collected by researchers themselves or (b) secondary data collected by others, usually international agencies, on which researchers can work further. While primary data are newly collected data, it is not uncommon that specific questions in the questionnaire are being borrowed from existing questionnaires used by others before. More information about the use of primary data in quantitative research and tools available to borrow from the adult education literature have been reviewed below. Afterward, a similar discussion will be presented in relation to secondary data analysis.

Primary Data in Quantitative Research

As stated by Robson (2011), fixed research designs often draw on quantitative measurements, either through experiments or surveys. Based on a review of the adult education literature in three leading journals, it became clear that most quantitative research is based on questionnaire studies, not on experiments. Collecting facts with the aim to observe trends and quantify these trends is commonly labelled as survey research and one of the major aims of quantitative research (Andres, 2014; Bryman, 2012). In setting up a survey, the researcher will have to make decisions on not only how to sample but also on how to formulate the specific questions that will be asked, which is extremely important as these questions cannot be changed anymore once data collection has started. Cohen et al. (2011), drawing on work by Sellitz, Wrightsman, and Cook (1976), discuss the need to make clear decisions not only on the content of the questions but also the way in which these questions are worded. Questions can be open, leaving room to the respondent to formulate his or her own answer, but quite often, specific answering options will be formulated, for example, through checklists, Likert-type scales, drop down lists, or rating exercises. Last but not least, the sequence of the different questions in the overall survey is also extremely important, grouping questions that are similar in content. The formulation of questions will also depend on the choice of survey methodology (Fink, 1995). Asking respondents to complete the survey online or through postal service is different from conducting a telephone or face-to-face interview where additional explanations can be given on key terms, although no further questions are supposed to be asked, as surveys are usually entirely structured and fixed (Brinkmann & Kvale, 2014).

In starting a new survey questionnaire, existing survey questionnaires can be explored. Borrowing questions that have been used before will increase the validity and reliability of your results. Another layer of validity and reliability can be added if measurement instruments have gone through a pilot phase. So what information and tools to use in our own adult education research can we find in the leading journals in the field? While researchers have produced too many questionnaires to discuss in detail here, it is important to review existing standardized scales as these are helpful research tools for a variety of reasons. These scales can be used in new settings not explored before, can further increase the validity and reliability of these measurement instruments and can be used to refine theory based on them. A search for the keyword “scale” within the text mining exercise demonstrates that the word had been used in 334 articles, although often not specifically in the context of research methods. Therefore, an additional screening was undertaken to filter out the specific measurement scales used by adult education scholars in the past 15 years. Despite the limited presence of quantitative research in these journals, a number of scales were found, most of them based on a range of items measured through typical Likert-type scales (e.g., 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree; Likert, 1929). Having explored and reviewed measurement scales’ content, I decided to group them into four categories: (a) participation scales, (b) experiences scales, (c) psychometric scales, and (d) learning styles scales. Results of the review are being discussed using these four categories (Table 2).

Table 2.

Overview of Quantitative Scales as Found in the Leading Adult Education Journals.

Participation scales	Experiences scales	Psychometric scales	Learning style scales
Education Participation Scale (Boshier, Huang, Song, & Song, 2006; Isaac, Guy, & Valentine, 2001)Reasons for Participation Scale (Mulenga & Liang, 2008)Adult attitudes toward Adult and Continuing Education Scale (Blunt & Yang, 2002)	Noel-Levitz Adult Student Priorities Survey (Giancola, Munz, & Trares, 2008)Power and Influence Tactics Scale; Problem Solving Inventory (Hendricks, 2001)	Motivated Strategy for Learning Questionnaire (Justice & Dornan, 2001)Abbreviated Math Anxiety Scale; Mathematics Self-Efficacy Scale; Self-Description Questionnaire III–Math Subscale (Jameson & Fusco, 2014)Beck Anxiety Inventory (Carney-Crompton & Tan, 2002); Academic Self-Efficacy Scale Parental Self-Efficacy Scale; Work–Family Balance Scale; Extended Satisfaction with Life Scale (van Rhijn & Lero, 2014)General Self-Efficacy Scale (Bath & Smith, 2009)Self-Concept and Perceived Problem-Solving Skills Scales (Porras-Hernandez & Salinas-Amescua, 2012)Borg CR-10 scale (Piirainen & Viitanen, 2010)	Personal Responsibility Orientation to Self-Direction in Learning Scale (Stockdale & Brockett, 2011)Oddi Continuing Learning Inventory (Harvey, Rothman, & Frecker, 2006)Student Engagement Questionnaire (Lee, 2014)Approaches to Supervision Scale; Supervision Practices Scale; Supervision Outcome Scale (Lizzio, Stokes, & Wilson, 2005)

Participation scales

First of all, and probably the most well-known scales in adult education research related to participation in adult education. The following scales were found based on the analysis in QDA Miner. Boshier (1973) developed the “Education Participation Scale” as a further empirical testing and validation of Houle’s (1961) typology of adult learners, distinguishing between goal-oriented, activity-oriented, and content-oriented learners. In the past 15 years, the scale has been used to discover the motivations of African American adult learners in church-based education (Isaac et al., 2001). Boshier was also involved in a project measuring the motivation of adult learners in Shanghai, measured through his Education Participation Scale (Boshier et al., 2006). While Mulenga and Liang (2008) refer to Boshier’s scale, they used the “Reasons for Participation Scale” developed by Steele (1984) to measure participation of adults studying at the Open University in Taiwan. Factors discussed were “keeping up and fulfillment,” “intellectual stimulation,” “escape and social contact,” and “adjustment.” Another scale developed to specifically predict participation behavior in adult education is the “Adult Attitudes towards Adult and Continuing Education Scale” (Blunt & Yang, 2002). Their scale consists of nine items relating to three factors: “enjoyment of learning,” “importance of adult education,” and “intrinsic value.” Drawing on attitudinal work undertaken by Fishbein and Ajzen (1975) to explain planned and intended behavior, Blunt and Yang (2002) expand on the importance of positive attitudes toward learning in relation to adult education participation.

Scales measuring learning experiences

A second group of scales found in the leading journals relates to the experiences of adult learners, mainly in relation to their participation in a specific setting. While “experiences” are often perceived as ideally measured through qualitative research (e.g., Thomas, 2009), quantitative scales equally attempt to capture feelings and experiences, although the presentation of the analysis will be more static and numerical, answering “what” or “how” people feel, instead of “why” they feel a certain way. The following scales were identified.

Giancola et al. (2008) used the “Noel-Levitz Adult Student Priorities Survey,” which consists of a scale with 50 items, divided into eight subscales on “academic advising,” “academic services,” “admissions and financial aid effectiveness,” “campus climate,” “instructor effectiveness,” “registration effectiveness,” “safety and security,” and “service excellence” in order to study the differences between priorities of adult versus first-generation students. Experiences in relation to program planning in adult education, from the perspectives of both students and staff members were measured through the “Power and Influence Tactics Scale” (POINTS) and the “Problem Solving Inventory” in the work of Hendricks (2001). The authors argue for a further testing of the POINTS instrument in order to enhance the reliability of the scale and to test the construct of power and influence in a wider range of settings with diverse samples. To date, no other research using POINTS has been published in one of the three leading adult education journals.

Psychometric scales

Scales are often used in psychological—psychometric—research and it is thus not surprising to see that, based on the analysis, a group of measurement instruments relate to concepts like anxiety and self-efficacy and these type of scales can be identified as a third type. The “Motivated Strategy for Learning Questionnaire” was used by Justice and Dornan (2001) to explore metacognitive differences between traditional and nontraditional students and focuses on factors like test anxiety, self-efficacy, and self-regulation. Anxiety in relation to mathematics courses was assessed by Jameson and Fusco (2014) using items from the “Abbreviated Math Anxiety Scale” as well as the “Mathematics Self-Efficacy Scale,” and the “Self-Description Questionnaire III-Math Subscale.” Anxiety has also been a central feature of the work conducted by Carney-Crompton and Tan (2002) on the performance of functioning of female nontraditional students in Canada. They used the “Beck Anxiety Inventory,” which consists of 21 anxiety items and which has, according to previous research, a strong internal consistency. Self-efficacy has also been the main variable in research conducted by van Rhijn and Lero (2014) with Canadian student parents. They used the “Academic Self-Efficacy Scale” as well as the “Parental Self-Efficacy Scale.” Also the “Work–Family Balance Scale” was included in their measures. The project revealed that parent students’ self-efficacy matches their satisfaction in relation to being a student and a family member, with satisfaction measured through use of the “Extended Satisfaction with Life Scale.” Apart from the academic and parental scales, there is also a “General Self-Efficacy Scale,” which had been used by Bath and Smith (2009) to analyze propensities of lifelong learners. In understanding the nonparticipation of adults, Porras-Hernandez and Salinas-Amescua (2012) worked with the “Self-Concept and Perceived Problem-Solving Skills Scales” and found that nonparticipation of poorly educated women cannot solely explained by their dispositional characteristics. A scale that is different from the previous ones, but which probably best fits in the category on psychometrics, is the “Borg CR-10 scale” used by Piirainen and Viitanen (2010) in a project on community development based on individual expertise.

Scales measuring learning styles

A fourth group of scales as found in the leading journals relates to learning styles, some of them specifically focusing on self-directed learning. The following scales were found. Stockdale and Brockett (2011) reviewed the literature on self-directed learning and developed a new “Personal Responsibility Orientation to Self-Direction in Learning Scale,” providing the scholarly community with an improved measurement instrument replacing the “Self-Directed Learning Readiness Scale” (Guglielmino, 1977). Another instrument to study self-directed learning, the “Oddi Continuing Learning Inventory” was used by Harvey et al. (2006), proposing a four-factor structure based on “learning with others,” “learner motivation/self-efficacy/autonomy,” “ability to be self-regulating,” and “reading avidity.” The development and learning of students has also been studied using a modified version of the “Student Engagement Questionnaire” by Lee (2014), which consists of a range of items related to “critical thinking,” “self-managed learning,” “adaptability,” “problem solving,” “communication skills,” “interpersonal skills and group work,” “computer literacy,” “active learning,” “teaching for understanding,” “feedback to assist learning,” “assessment,” “teacher–student relationship,” and “student–student relationship.” Within the specific context of supervision for practicing psychologists, Lizzio et al. (2005) constructed the “Approaches to Supervision Scale” to analyze supervisees perceptions of teaching and management approaches used during the supervisory process, one in relation to themselves and one in relation to the approaches used by their supervisor. These scales were conducted together with a “Supervision Practices Scale” and a “Supervision Outcome Scale” to measure the use of supervision techniques and the effectiveness of supervision.

Secondary Data in Quantitative Research

For researchers interested in undertaking quantitative research, there is also an option to use existing data sets. Technically, every use of an existing data set can be labelled as “secondary data analysis,” although generally speaking, one is inclined to think about the major data sets as collected by leading international organizations, for example, the OECD, the Organisation for Economic Co-operation and Development (Smith, 2008). Smith (2008, p. 37) argues that “secondary data analysis remains a relatively underused methodological technique in in the social sciences” and also focuses on the limited use of quantitative research in education generally. The lack of quantitative research is thus not only present in adult education research but also within the broader field of education. One of the reasons Smith (2008) puts forward why scholars might feel skeptical about the use of secondary data might relate to the quality of data, for example, the level of missing values and measurement errors. Furthermore, she says, scholars might not like the fact that these data are “socially constructed,” reducing the complexity of life into a range of digits. The trust in statistics is generally not very high because of its manipulative power. However, as Smith (2008) goes on, the pitfalls need to be judged against the strengths of working with secondary data. First of all, existing data sets can be used multiple times and explored from different angles, being used to advance both theoretical insights and methodological approaches. Data sets are often available to scholars at a low or no price, which is certainly true for the adult education field. Nowadays, these data are also used for evidence-based policy making, for example, through working with benchmarks and indicators as means of putting peer pressure on a wide range of countries, to strengthen education policy making (Holford & Mohorcic-Spolar, 2012). Journals’ aims of reflecting on practice, as pointed out by Fejes and Nylander (2015) might thus also include working with these quantitative data. However, it remains important to understand that secondary data sources were initially produced for another purpose than the own research to be undertaken. Surveys constructed by, for example, the OECD or Eurostat are being designed to serve a specific policy agenda, such as understanding the role of education and skills in relation to economic prosperity.

Currently, one of the major data sets of interest to adult education scholars is based on data from PIAAC’s (Program for the International Assessment of Adult Skills) survey of adult skills, organized by the OECD. While it is too early to make an overview of articles drawing on data from PIAAC, it is possible to explore how widely researchers in the field have published analyses using data from other large scale surveys, an analysis I will further explore based on the use of the International Adult Literacy Survey (IALS) in the three leading journals.

The IALS was also organized by the OECD and was conducted in three waves between 1994 and 1998 (Desjardins, Rubenson, & Milana, 2006). Given this time span and given the time needed to make data sets ready for use in research projects, it is expected that analyses of these data have been published in the early 2000s. While other surveys exist, Desjardins et al. (2006) mention that IALS “is one of the most complete of all surveys undertaken.” Other OECD sources mentioned by Desjardins et al. (2006, p. 28-29) are as follows:

International Adult Literacy Survey (IALS)

Adult Literacy and Lifeskills Survey (ALL)

Thematic Review on Adult Learning (TRAL)

Program for the International Assessment for Adult Competencies (PIAAC)

Those working in the European context might also be interested in working with surveys conducting within European Union countries that measure specific adult and lifelong learning aspects. These include the following:

European Labour Force Survey (LFS)

Adult Education Survey (AES)

Continuing Vocational Training Survey (CVTS)

European Survey on Working Conditions (ESWC)

Eurobarometer on lifelong learning

While I do not have the space to go into detail exploring each individual data set, all of them are relevant for adult education research as the questionnaires of these surveys explicitly measure participation in learning and training activities. Currently, PIAAC and the European surveys are being updated by new waves of data collection.

Nowadays, most of these data are available free of charge, for example, PIAAC data can be downloaded for free from the OECD website. All data sets are backed up by extensive guides, such as codebooks, reports focusing on sampling procedures, survey methods, and quality of data. These are also downloadable for free.

Going back to the data mining exercise, results indicate that International Journal of Lifelong Education had nine hits for the key term “IALS,” but has in fact only one research article that draws on data from the survey in an aggregated form (Bathmaker, 2007). Studies in Continuing Education has four hits for IALS, but none of the articles can be classified as an example of secondary data analysis using data from IALS. The term has thus been used within another section such as within the literature review. Adult Education Quarterly even only shows two hits for IALS, none of them analyzing data from IALS. The article from Rubenson and Desjardins (2009) exploring the bounded agency model refers to IALS, but draws on data from the Eurobarometer 2003. Searching for the full key term “International Adult Literacy Survey” instead of the acronym IALS does not increase the number of articles that can be classified as secondary data analysis articles.

What about another data set then? The specific adult education data set provided by the European Commission is based on the Eurostat Adult Education Survey. Adult Education Quarterly does not have any articles drawing on secondary data from these data sets. In Studies in Continuing Education, I found one article (Boeren, 2011a). In International Journal of Lifelong Education, I found two articles that draw on aggregated data from adult education survey. One by Broek and Hake (2012) in relation to adults’ participation in higher education and one by Roosmaa and Saar (2012) on nonformal education in the old European Union member states.

The limited availability of research drawing on secondary data analyses in our field might indicate the limited interest or lack of skills in working with these data.

Limitations, Discussion, and Conclusions

The use of different methods and methodologies in a field of research can enhance the quality of research through exploring similar topics from different angles, employing different empirical approaches, for example, through the combination of collecting data on facts and figures by means of quantitative research and deepening out the further understanding of why certain facts exist (Robson, 2011). Based on previous research by Fejes and Nylander (2015), but also confirmed in an additional review undertaken by myself, there is no doubt that the leading academic journals in the field of adult education feature more qualitative than quantitative studies. One of the limitations of both studies (my own and the one by Fejes and Nylander) is that they exclusively focused on the leading generic adult education journals, not focusing on other types of social sciences journals. However, in case of more quantitative research being available in other outlets, the question then remains why it does not end up in the three leading journals? Why the underrepresentation of quantitative research does not necessarily needs to be problematic, it might be worthwhile to end with some critical suggestions to raise awareness among adult education scholars on opportunities available to them to engage with quantitative research.

First of all, returning to the hypotheses mentioned earlier in this article based on work by Fejes and Nylander (2015), I want to elaborate on the most likely existence of a skills deficit in the field and that new researchers and PhD students are most unlikely to undertake quantitative research if their supervisors or mentors are also not working within numerical data. However, in times where our field—and in fact not only our field—is dominated by a focus on “big data” and the use of benchmarks and indicators, both by the European Commission, the OECD and UNESCO (United Nations Educational, Scientific and Cultural Organization), it would be a pity if our field would miss this boat and not publish more high-quality articles in our leading journals based on data from, for example, the Eurostat Adult Education Survey, the Labour Force Survey, PIAAC’s Survey of Adult Skills, or indeed a range of high-quality data sets available at the country level. It would be interesting to undertake research on whether adult education researchers feel reluctant in working with these large scale survey data because of their specific nature, for example, dominated by the economic focus of the OECD and the European Commission, or whether scholars feel not confident in working with quantitative data because of the absence of quantitative skills training available to them. Without this type of research, claims about the limited presence of quantitative research remains largely hypothetical and thus needs to be dealt with carefully. An example of an initiative open to scholars worldwide to increase quantitative skills is the Essex Summer School in Social Science Data Analysis (see http://www.essex.ac.uk/summerschool/). Researchers can take stand-alone courses or combine them toward a master’s qualification. Courses are offered at introductory, intermediate, and advanced level. It is also interesting to know that the European Commission (2015) organizes data user conferences for researchers who use data from, for example, the Labour Force Survey or the Adult Education Survey. Attending these events might increase scholars’ understanding of how colleagues work with large scale data and for those working with these data, it might be an opportunity to put adult education research more into the picture. However, it might be that adult education scholars have no interest in participating in these events or that they do not have the time or resources to attend.

Second, it might be needed to produce more methodological guides specifically focusing on adult education research. In recent years, a number of high-quality books on research methods for education and social sciences have appeared (e.g., Cohen et al., 2011; Robson 2011), but as with many general books on education, examples are often taken from research on compulsory schooling, not adult education. It is recommended to have a stronger exchange about research methodologies at research conferences through, for example, organizing symposia on methodological aspects of adult education research exploring the strengths of what quantitative methods in the field can offer, instead of solely focusing on content specific aspects of the field, including theoretical and policy-oriented contributions As a researcher engaging in quantitative research, I would hope that these debates and an increased level of information about methodological opportunities in the field would encourage more researchers to explore quantitative research and to lower the barriers for researchers who might fear that their research might not fit in the dominant discourses in the field and that their work will be evaluated in a skeptical way because of the unfamiliarity of many fellow researchers about the specific methods they have used. As explored above, research approaches are ideally chosen based on the specific research questions we want to answer and there is no doubt that the field can still answer a lot of interesting questions that would profit from being investigated using quantitative methods, as long as researchers are aware of a range of existing validated scales, appropriate statistical techniques and the types of questions requiring a quantitative approach.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biography

Ellen Boeren is lecturer at the Moray House School of Education, University of Edinburgh, United Kingdom. She is the author of “Lifelong Learning in a Changing Policy Context: An Interdisciplinary Theory.” She leads the Edinburgh team in the Horizon 2020 project “Encouraging Lifelong Learning for an Inclusive and Vibrant Europe.”

References

Andres

(2014). Designing and doing survey research. London, England: Sage.

Bath

Smith

(2009). The relationship between epistemological beliefs and the propensity for lifelong learning. Studies in Continuing Education, 31, 173-189.

Bathmaker

A. M.

(2007). The impact of Skills for Life on adult basic skills in England: How should we interpret trends in participation and achievement? International Journal of Lifelong Education, 26, 295-313.

Blunt

Yang

(2002). Factor structure of the Adult Attitudes toward Adult and Continuing Education Scale and its capacity to predict participation behavior: Evidence for adoption of a revised scale. Adult Education Quarterly, 52, 299-314.

Boeren

(2011a). Gender differences in formal, non-formal and informal adult learning. Studies in Continuing Education, 33, 333-346.

Boshier

R. W.

(1973). Educational participation and dropout: A theoretical model. Adult Education Journal, 23, 255-282.

Boshier

Huang

Song

(2006). Market socialism meets the lost generation: Motivational orientations of adult learners in Shanghai. Adult Education Quarterly, 56, 201-222.

Brinkmann

Kvale

(2014). Interviews: Learning the craft of qualitative research interviewing. London, England: Sage.

Broek

Hake

(2012). Increasing participation of adults in higher education: Factors for successful policies. International Journal of Lifelong Education, 31, 397-417.

10.

Bryman

(2012). Social research methods. Oxford, England: Oxford University Press.

11.

Carney-Crompton

Tan

(2002). Support systems, psychological functioning, and academic performance of nontraditional female students. Adult Education Quarterly, 52, 140-154.

12.

Cohen

Manion

Morrison

(2011). Research methods in education. London, England: Routledge.

13.

Creswell

(2003). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage.

14.

Denzin

Lincoln

(2003). Handbook of qualitative research. London, England: Sage.

15.

Desjardins

Rubenson

Milana

(2006). Unequal chances to participate in adult learning: International perspectives. Paris, France: UNESCO.

16.

Ercikan

Roth

(2006). What good is polarizing research into qualitative and quantitative? Educational Researcher, 35(5), 14-23.

17.

European Commission. (2015). Call for papers: Microdata from Eurostat (Fourth European user conference). Brussels, Belgium: Author.

18.

Fejes

Nylander

(2015). How pluralistic is the research field on adult education? Dominating bibliometrical trends, 2005-2012. European Journal for Research on the Education and Learning of Adults, 6, 103-123.

19.

Fink

(1995). How to ask survey questions: The survey kit. London, England: Sage.

20.

Fishbein

Ajzen

(1975). Belief, attitude, intention, and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.

21.

Gage

(1989). The paradigm wars and their aftermath. Teachers College Record, 91, 135-150.

22.

Giancola

Munz

Trares

(2008). First- versus continuing-generation adult students on college perceptions: Are differences actually because of demographic variance? Adult Education Quarterly, 58, 214-228.

23.

Guglielmino

(1977). Development of the self-directed learning readiness scale (Doctoral dissertation). University of Georgia, Athens, Greece.

24.

Harvey

Rothman

Frecker

(2006). A confirmatory factor analysis of the Oddi Continuing Learning Inventory (OCLI). Adult Education Quarterly, 56, 188-200.

25.

Hendricks

(2001). Contextual and individual factors and the use of influencing tactics in adult education program planning. Adult Education Quarterly, 51, 219-235.

26.

Holford

Mohorcic-Spolar

(2012). Neoliberal and inclusive themes in European lifelong learning policy. In Riddell

Markowitsch

Weedon

(Eds.), Lifelong learning in Europe: Equity and efficiency in the balance (pp. 39-62). Bristol, England: Policy Press.

27.

Houle

(1961). The inquiring mind. Madison: University of Wisconsin.

28.

Isaac

Guy

Valentine

(2001). Understanding African American learners’ motivations to learn in church-based adult education. Adult Education Quarterly, 52, 23-38.

29.

Jameson

Fusco

(2014). Math anxiety, math self-concept, and math self-efficacy in adult learners compared to traditional undergraduate students. Adult Education Quarterly, 64, 306-322.

30.

Johnson

Onwuegbuzie

Turner

(2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1, 122-133.

31.

Justice

Dornan

(2001). Metacognitive differences between traditional-age and nontraditional-age college students. Adult Education Quarterly, 51, 236-249.

32.

Lee

(2014). Opening up a road to somewhere: Development of associate degree students in Hong Kong. International Journal of Lifelong Education, 33, 607-624.

33.

Likert

(1929). A technique for the measurement of attitudes. New York, NY: Columbia University.

34.

Lizzio

Stokes

Wilson

(2005). Approaches to learning in professional supervision: Supervisee perceptions of processes and outcome. Studies in Continuing Education, 27, 239-256.

35.

Mulenga

Liang

Jr. (2008). Motivations for older adults’ participation in distance education: A study at the National Open University of Taiwan. International Journal of Lifelong Education, 27, 289-314.

36.

Piirainen

Viitanen

(2010). Transforming expertise from individual to regional community expertise: A four-year study of an education intervention. International Journal of Lifelong Education, 29, 581-596.

37.

Porras-Hernandez

Salinas-Amescua

(2012). Nonparticipation in adult education: From self-perceptions to alternative explanations. Adult Education Quarterly, 62, 311-331.

38.

Robson

(2011). Real world research. Chichester, England: Wiley.

39.

Roosmaa

E. L.

Saar

(2012). Participation in non-formal learning in EU-15 and EU-8 countries: Demand and supply side factors. International Journal of Lifelong Education, 31, 477-501.

40.

Rubenson

Desjardins

(2009). The impact of welfare state regimes on constraints to participation in adult education: A bounded agency model. Adult Education Quarterly, 59, 187-207.

41.

Sellitz

Wrightsman

Cook

(1976). Research methods in social relations. New York, NY: Holt, Rinehart & Winston.

42.

Smith

(2008). Using secondary data in educational and social research. New York, NY: McGraw-Hill.

43.

Steele

(1984). The motivational orientations of persisting older learners in the university setting (Unpublished doctoral dissertation). University of Arkansas, Fayetteville.

44.

Stockdale

Brockett

(2011). Development of the PRO-SDLS: A measurement of self-direction in learning based on the personal responsibility orientation model. Adult Education Quarterly, 61, 161-180.

45.

Teddlie

Tashakkori

(2009). Foundations of mixed methods research. Thousand Oaks, CA: Sage.

46.

Thomas

(2009). How to do your research project. London, England: Sage.

47.

van Rhijn

Lero

(2014). The influence of self-efficacy beliefs for student parents attending university. International Journal of Lifelong Education, 33, 541-555.