Abstract
The U.S. workforce is increasingly comprised of older adults, women, and minorities who lack basic skills and are unable to acquire these skills through traditional educational and training programs. New approaches are needed to provide effective training to the adult learner and flexible support for nontraditional students who must balance work-life demands with limited educational opportunities. Contextualized teaching and learning (CTL) is a form of experiential learning that blends both basic skills and occupational training together in environments that allow students to relate subject matter to real-world situations. Virtual CTL environments can be created to better engage students, provide immediate performance feedback, reduce training time, and improve accessibility. To determine the effectiveness of CTL, the Society of Manufacturing Engineers Tooling U curriculum and learning management system (LMS) was studied. This intervention consisted of online modules with basic skills remediation blended with interactive labs and virtual reality exercises. A nonrandom population of 342 participants was chosen for study, including 75 exposed to the CTL intervention (experimental group) and 267 not exposed to the CTL intervention (control group). Learning outcomes such as test scores, completed credit hours, course completions, and earned credentials, were compared between CTL and non-CTL groups and between demographic subsets within the CTL group. Underrepresented groups, including older adults, women, and minorities, were 2 to 3 times greater in the CTL group compared with the non-CTL group. Overall, students exposed to CTL achieved higher rates of credentialing (55%) when compared with students not exposed to contextualized instruction (20%).
“
Introduction
In 2012, a US$6 million U.S. Department of Labor (DOL) grant was awarded to an educational consortium of five community colleges and a Midwestern University to increase and expand access to industry-recognized credentials for recession-displaced and underemployed workers in manufacturing industries. The purpose of this project, called Innovations Moving People to Achieve Certified Training (IMPACT), was to expand and improve the ability of this consortium to deliver technology-enabled education and career training programs to displaced workers and underrepresented groups such as women, minorities, veterans, and workers who face unemployment and underemployment.
As part of Project IMPACT, the goal of this research was to determine whether contextualized teaching and learning (CTL) improved learning outcomes, particularly among older adults, women, and minorities. Specific objectives included the following:
Comparing the effectiveness of contextualized instruction between an experimental group exposed to contextualized instruction and a control group not exposed to contextualized instruction.
Comparing the effectiveness of contextualized instruction among demographic groups within an experimental group exposed to contextualized instruction.
Background
Contextualized teaching and learning (CTL) using digital technology is being used to build recruitment networks, improve learning outcomes, and assess performance. These evidence-based strategies include both an online, modular curricula, and learning management systems (LMS) that incorporate the use of 4D gaming, simulation, virtual instruction, digital tutors, and real-time collaboration between learners and instructors. Immersive, occupational environments are created using augmented and mixed reality, and instruction is contextualized using authentic engineering and support tasks in core areas of design, process, and production. For traditional “iGen” students, this digital world is an indistinguishable part of their identity and has proven to be the most effective medium to attract and develop skills sets among secondary and postsecondary students, particularly when blended with more traditional classroom and lab instruction. LMS has been shown to improve instruction by detecting learning behaviors of students and their interaction with digital CTL applications (Liu & Hwang, 2010).
Digital instruction has also proven to be effective for nontraditional students such as women, minorities, and adult learners, who must often balance basic skills deficiencies and competing demands of work-life with limited educational opportunities (Epper & Baker, 2009). For these students, the ability to relate new skills to past, real-world experiences and to contextualize relationships between concepts and practical applications is key (Ruben, 1999). Similarly, digital instruction supports distance education, which reduces time and cost while improving accessibility for both traditional and nontraditional students. CTL supports the notion that the workplace experience generates a demand for learning (de Jong, Specht, & Koper, 2008). Online, modular curricula can further be stacked or “latticed” toward industry-recognized credentials while articulating toward diplomas and degrees, accelerating skills attainment while providing flexible “on-” and “off-ramps” between continuing education and work (Imel, 2000; Killingsworth & Grosskopf, 2012; Rademacher, 2002).
CTL is a strategy that engages adult learners and promotes improved skills development by helping instructors relate subject matter to real-world situations and experiences (Berns, 2001). Throughout the literature (Ambrose, Davis, Zeigler, & Kirvan, 2013), various terminology is used for the practice of systematically connecting basic skills instruction to specific content that is meaningful and useful to students; contextual teaching and learning, contextualized instruction, content area literacy, embedded instruction, integrative curriculum, situated cognition, problem-based learning, theme-based instruction, anchored instruction, curriculum integration, academic–occupation integration, work-based learning, and functional context education (Perin, 2011; Specht, 2008; Wenger & Lave, 1991). Each of these descriptions has been used to express or define CTL (Carrigan, 2008). CTL has been developed and proven effective for nontraditional students, including older adult learners who have not been in a structured learning environment or used high-school equivalency skills in several years, if at all (Baker, Hope, & Karandjeff, 2009; Fernandes, Leite, Mouraz, & Figueiredo, 2013). Many have difficulty understanding basic academic concepts needed for vocational training. Too often, remedial skills are taught independently of, or, as a prerequisite to occupational training, resulting in adult learner discouragement and high training program attrition.
Methods
As part of Project IMPACT, the consortium developed an entry-level Diversified Manufacturing Technology (DMT) certificate providing 12 credit hours of training in basic safety, quality control, manufacturing, and maintenance. The DMT certificate was aligned with the nationally recognized Manufacturing Skill Standards Council (MSSC) and Certified Production Technician (CPT) credential and was designed to articulate toward other certificates, diplomas, and degrees within the program.
Among 1,021 Project IMPACT participants from 2012 to 2016, 342 participants were chosen for further study, including 75 exposed to a CTL intervention (experimental group) and 267 not exposed to a CTL intervention (control group). The Society of Manufacturing Engineers (SME) Tooling U curriculum and LMS was chosen for the CTL intervention. The SME Tooling U intervention consisted of 414 Internet-accessible modules with interactive, graphic-oriented exercises, labs, and simulations. Tooling U modules were designed to provide foundational skills in 22 manufacturing-related areas at three (3) competency levels (100-introductory, 200-intermediate, and 300-advanced) in a sharable content object reference model (SCORM) format.
For the DMT certificate, 82 contextualized Tooling U modules were selected in the core areas of safety, quality control, manufacturing, and maintenance including 53 with basic skills remediation, 28 with interactive labs, and 14 with virtual reality (VR) exercises. Test scores, course completions, credentials, and other learning outcomes were compared among experimental and control groups and between demographic subsets within the experimental group to determine the effectiveness of the Tooling U CTL intervention among older adults, women, and ethnic minorities. The study design was considered quasi-experimental as study participants and training interventions were not randomly selected.
Upon enrollment, Project IMPACT participants were assigned a confidential project code to protect their identity. Participant data were then entered into a student intake and tracking database and updated quarterly. Participant intake and tracking data included student demographics, learning outcomes, and posttraining employment status (Table 1).
Participant Intake and Tracking Data
Note. DMT = Diversified Manufacturing Technology; IMPACT = Innovations Moving People to Achieve Certified Training.
With participant intake and tracking data, a matrix was first developed to compare demographics and learning outcomes between experimental and control groups. This initial comparison was designed using simple frequency distributions and descriptive statistics to quantify differences in demographics between experimental and control groups as well as differences in learning outcomes such as credit hours and courses completed and certifications, diplomas, and degrees awarded. Descriptive analyses were also used to quantify differences in learning outcomes such as Tooling U pretest and posttest scores and modules completed among subsets of the experimental group.
Inferential statistics were then used to determine whether changes in learning outcomes were caused by the Tooling U CTL intervention. To observe the association between learning outcomes and the CTL intervention, Fisher’s exact test was used. Fisher’s exact test is more appropriate for smaller categorical data sets than other more common measures of this type (e.g., chi-square test) and provides an exact calculation of statistical significance (p ≤ .05) rather than an approximation. Finally, a logistic regression model was developed to determine whether changes in Tooling U CTL performance was associated with participant demographics such as age, gender, ethnicity, enrollment status, employment status, veteran status, disability, and financial need.
Results
Of 1,021 Project IMPACT participants from 2012 to 2016, roughly one-third where women or ethnic minorities. The majority were nontraditional students (average age of 26 years) employed full-time (66%) upon enrollment. By 2016, 904 (89%) students enrolled in the IMPACT project had completed grant-funded coursework (Table 2). Nearly 60% of enrolled students had either completed a grant-funded program of study or were in the process of completing a program of study. More than 300 associates degrees and 700 credentials were conferred to these students, with several earning multiple certificates and diplomas.
Participant Learning Outcomes and Employment Status
Of 342 IMPACT participants chosen for further study, 17% were female compared with 7% for all Project IMPACT participants. The average age of study participants was 39 years compared to 26 years of age for IMPACT participants. Ethnic minorities composed 40% of students in the study group compared with 22% for IMPACT project participants. Veterans comprised 12% of study group students compared with 9% for all Project IMPACT participants.
Comparison of Learning Outcomes Between CTL and Non-CTL Groups
Within the 342-student study population, the CTL group was significantly more representative of the nontraditional student when compared with the non-CTL group (Figures 1 and 2). Overall, students exposed to CTL achieved higher rates of credentialing (55%) when compared to students not exposed to contextualized instruction (20%). The vast majority of credentials earned by CTL participants, however, were program and industry certificates. The majority of credentials earned by non-CTL participants were 2-year associate degrees (Table 3).
Comparison of CTL and non-CTL age.
Comparison of CTL and non-CTL ethnicity.
Comparison of CTL and Non-CTL Learning Outcomes
Note. CTL = contextualized teaching and learning; FT = full-time; PT = part-time.
As shown, 48% of male participants in the CTL group received some form of credential compared with 19% of males in the non-CTL group. Among females, 85% of participants in the CTL group received some form of credential compared with 22% of females in the non-CTL group. Among ethnic groups, 61% of White participants in the CTL group received some form of credential compared with 20% of White participants in the non-CTL group. Comparatively, 42% of Hispanic participants and 46% of Black participants in the CTL group received some form of credential compared with 24% of Hispanic participants and 10% of Black participants in the non-CTL group.
For most age groups, participants in the CTL group achieved higher rates of credentialing compared with participants in the non-CTL group. Among veterans, 73% of participants in the CTL group received some form of credential compared with 29% of participants in the non-CTL group. Two-thirds of disabled participants in the CTL group received some form of credential compared with 11% of participants in the non-CTL group. Among participants receiving financial aid, 75% in the CTL group received some form of credential compared with 19% of participants in the non-CTL group.
Of the participants, 72% enrolled full-time in the CTL group received some form of credential compared with 29% of full-time participants in the non-CTL group. Among part-time participants, 54% of students in the CTL group received some form of credential compared with 11% of students in the non-CTL group. Overall, participants enrolled full-time achieved a higher rate of credentialing when compared with part-time participants in both CTL and non-CTL groups. Of note, females achieved a higher rate of credentialing when compared with all other demographic groups in both the CTL and non-CTL groups, despite more than two-thirds of female participants being enrolled only part-time. For both CTL and non-CTL groups, employed participants were more likely to be enrolled part-time.
Comparison of Learning Outcomes Between CTL Demographic Groups
In addition to earned credentials, CTL-specific learning outcomes such as Tooling U module completions and pretest and posttest scores were compared among demographic subsets of the CTL (experimental) group to determine the effectiveness of contextualized training among women, ethnic minorities, veterans, older adults, disabled persons, and participants requiring financial assistance. Overall, a total of 4,326 Tooling U modules were completed within the CTL group. Of these, 2,945 modules (68%) were completed by participants scoring 60% or above on module pretests, suggesting that prior training and experience may have been a factor in successfully completing course modules. Similarly, 3,231 modules (75%) were completed by participants scoring above 80% on module posttests. Approximately half (53%) of CTL participants had pretest scores of 60% to 80% with significant improvements between total average pretests (60%) and total average posttests (83%).
Average pretest scores among male participants (61%) were slightly higher on average than female pretest scores (58%) although CTL posttest scores were almost the same on average for both groups (83%). As a result, female participants achieved greater improvement between pretest and posttest scores compared with males. Furthermore, female participants achieved significantly higher rates of module completion (99) on average than males (70). Average pretest scores among White participants (61%) were slightly higher on average than Hispanic (60%), Black (56%), and other ethnic minority (59%) pretest scores. Average posttest scores among White participants (84%) were also slightly higher on average than Hispanic (82%), Black (83%), and all other ethnic minority (79%) posttest scores. Black participants, however, achieved the highest average improvement between pretest and posttest scores (27%) compared with all other ethnicities. Hispanic participants achieved higher rates of module completion (85) on average than White (81), Black (72), and other ethnic minority (35) participants.
Participants older than 45 scored higher on pretests (61%) and posttests (86%) and achieved higher rates of module completion (90) than all other age groups. Participants enrolled full-time achieved slightly higher pretest scores (61%) on average when compared with part-time participants (59%). Part-time participants, however, achieved higher rates of module completion (92) on average than full-time students (59). Employed participants scored lower on pretest and posttests, but achieved significantly higher rates of module completion (99) when compared with unemployed participants (65). Participants employed part-time achieved higher pretest scores (63%) on average when compared with full-time participants (56%). Part-time participants also achieved higher posttest scores (85%) compared with full-time participants (84%) as well as higher rates of module completion (81-78, respectively). Among veterans, pretest (67%) and posttest scores (88%) were higher on average than all other demographic groups. Disabled participants had pretest scores more than 10 points higher than the average student and achieved higher rates of module completion (87) compared with nondisabled participants (74). Students who had financial need (e.g., Pell Grant eligible) had posttest scores nearly 6 points lower than the average student.
Participants achieving a 20% or greater improvement between CTL pretest and posttest scores were more than twice as likely to complete credit hours toward a credential when compared with participants achieving a 10% or less improvement (r2 = .77). Overall, female participants in the CTL group achieved a higher rate of credentialing (85%) when compared with male participants (49%) in the CTL group, including higher rates of industry certificates and Associate of Applied Science (AAS) degrees. White participants in the CTL group achieved a higher rate of credentialing (61%) when compared with Black participants (46%) and Hispanic participants (42%). Participants older than 45 years achieved the highest rate of credentialing (68%) compared to all other age groups. Participants younger than 25 years, however, achieved significantly lower rates of credentialing (25%) when compared to all other age groups in the CTL group.
To determine whether correlations between CTL demographics and learning outcomes were significant, logistic regression was used.
Summary and Discussion
Results indicate that contextualized training blended with traditional online instruction and improved learning outcomes for most participants, especially among older adults, ethnic minorities, and women. Overall, students exposed to CTL achieved higher rates of credentialing when compared with students not exposed to contextualized instruction. Within the CTL group, significant improvements between total average pretests and total average posttests occurred. Participants achieving a 20% or greater improvement between CTL pretest and posttest scores were more than twice as likely to complete credit hours toward a DMT credential when compared to participants achieving a 10% or less improvement. Female participants achieved greater improvement between pretest and posttest scores than males and were more likely to complete courses. Black participants achieved greater improvement between pretest and posttest scores than all other ethnic groups, while Hispanics achieved higher rates of module completion. Participants older than 45 years had higher test scores and achieved higher rates of module completion than all other age groups. Other significant findings comparing the effectiveness of contextualized instruction between CTL demographic subgroups include the following:
Female participants achieved significantly higher rates of Tooling U module completion, DMT course completion, and credentialing on average than males.
Participants enrolled part-time achieved higher CTL posttest scores and higher rates of CTL module completion compared with full-time participants.
Employed participants achieved significantly higher rates of CTL module completion when compared with unemployed participants.
Participants older than 45 achieved the highest rate of credentialing among all age groups.
Participants with prior military service scored higher on CTL pretest and posttests than all other demographic groups.
Disabled participants achieved higher rates of CTL module completion compared with nondisabled participants.
Participants who are Pell Grant eligible had CTL posttest scores nearly 6 points lower than the average student.
Footnotes
Authors’ Note
This study has been done as a part of the U.S. Department of Labor, Training Adjustment Assistance Community College, and Career Training grants program.
Conflict of Interest
The author(s) declared no potential conflicts of interest with respect to the authorship and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Author Biographies
Arefeh Mohammadi, PhD, is a full-time assistant professor of Manufacturing Technology at Clark State Community College. Her area of expertise is workforce development and training for STEM industries using experiential learning techniques. She graduated from the University of Nebraska–Lincoln with PhD in engineering and a minor in educational administration. She is a professional member of American Society of Engineering Education, Society of Women Engineers, and American Association of University Women in which she has advocated for success of women in STEM education.
Kevin Grosskopf, PhD, is a professor and associate director of the Durham School of Architectural Engineering and Construction (DSAEC) at the University of Nebraska–Lincoln. He is the recipient of more than US$6.5 million in externally funded research grants and endowments and has authored (or co-authored) more than 135 peer-reviewed papers, technical reports, and books. He has more than 15 years of prior industry experience.
John Killingsworth, PhD, is an assistant professor at Colorado State University. He has worked on numerous state and federal workforce development grants. His research includes a specific focus on modular or manufactured construction methods and the development of contextualized training programs.