Abstract
Introduction
Young adults with higher levels of education have higher median earnings and lower unemployment rates (United States Department of Education [USDOE], 2015), a finding which has pushed college and career readiness (CCR) to the forefront of current policy initiatives (National Governors Association & Council of Chief State School Officers [NGA & CCSSO], 2010). This movement has led to the near nation-wide adoption of the Common Core State Standards (Association for Supervision and Curriculum Development, 2012), that focus on concepts in mathematics and English Language Arts and are meant “to ensure that all students graduate from high school with the skills and knowledge necessary to succeed in college, career, and life” (NGA & CCSSO, 2010).
Despite higher expectations for CCR and success for all students, students with disabilities (SWD) have lower employment rates than their peers without disabilities (The National Collaborative on Workforce & Disability for Youth, 2014). Additionally, SWDs tend to be unemployed more often, and when employed, earn lower wages (Sanford et al., 2011). Moreover, in postsecondary education, SWDs have lower retention rates and lower degree completion rates (Horn et al., 1999; Murray et al., 2000), and take longer to complete degrees (Wessel et al., 2009). These employment and education rates indicate that traditional transition services for individuals with disabilities may not be adequate and may not be keeping pace with the demands of the knowledge economy in the 21st century.
College and career readiness: The Role of 21st Century Skills
Several groups have referred to the skills necessary for success in this “new” economy as 21st Century skills. They are often described in tandem with CCR and specifically include technology and critical thinking skills (Greenstein, 2012) that are required to be “career ready” in the future job market. Given the ubiquitous nature of technology in our society, the majority of today’s workforce needs to be computer-literate (Izzo et al., 2010). This need for computer literacy to participate in the workforce will only continue to increase (Ezziane, 2007).
It will also be important for employees to move beyond rudimentary understanding of computer programs (computer literacy) to an understanding of how to leverage these technology tools to manage, analyze, and present information (information technology literacy, or sometimes referred to as information and communication technology literacy). In this way, information technology (IT) literacy, which considers information, media, and technology skills as crucial components of effective citizenship and employment (Izzo et al., 2010; Partnership for 21st Century Learning, 2009), is the ultimate embodiment of 21st Century skills. In other words, IT literacy is essentially computer skills, or digital literacy, combining reading with efficient and effective searching skills (Coiro, 2012). Further, among adolescents with disabilities, IT literacy has shown to be an effective method of instructional delivery of CCR, 21st Century skills, and transition content (Izzo et al., 2010).
Twenty-first Century skills include a variety of competencies in the following areas: a) life and career; b) information, media, and technology literacy; c) content-area specializations; and d) learning and innovation skills, such as communication, creativity, collaboration, and critical thinking (Partnership for 21st Century Learning, 2016). Functional literacy in the 21st Century is especially salient given that most of the reading that students do now is online (Drew, 2012; Kaiser Family Foundation, 2010), which demands additional reading comprehension strategies (Coiro, 2003; Drew, 2012; Leu et al., 2004) to navigate the Internet, comprehend higher level text, discern between different types of online tools and media, and evaluate information credibility. This complexity in information processing lead Hutchison and Woodward (2014, p. 458) to state “digital tools and environments alter what it means to be literate” and that technology in schools should be used to promote literacy skill development – a finding documented in the literature (see Barone & Wright, 2008; Hutchison & Woodward, 2014; Northrop & Killeen, 2013; Vasinda & McLeod, 2011).
Despite some promising, preliminary findings on the benefits of IT literacy learning for SWDs (Izzo et al., 2010), very little research has been conducted in this area. Related research has focused on computer-assisted instruction (Ferretti et al., 2001; Hall et al., 2000; Leu et al., 2015) or placing video anchors within texts to aid understanding (Fitzgerald et al., 2008). While these approaches are valuable, they only use technology in a very basic way and have merely replaced the instructor with the computer. In other words, these approaches are not as self-directed as IT literacy. IT literacy inherently provides opportunities for students to quickly read and locate information as well as to be able to use that information for learning and reading (Henry, 2006).
Leveraging information technology literacy to promote transition services
Though most experts agree that teaching 21st Century skills is essential to functioning in modern society and should be a focus of classroom instruction, there are currently no systematic processes or procedures for educators to teach these skills to students (Voogt et al., 2013). Concrete guidance to teachers on how to integrate these skills into curriculum development and classroom practice is scant (Binkley et al. 2012; Voogt et al., 2013). In fact, most people develop these skills through their own personal experiences and hands-on learning (Lombardi et al., 2016). Most sets of academic standards, including the CCSS, and most 21st Century frameworks describe what should be taught as educational maxims or goals, but they do not describe the how, such as the pedagogy or content to support proficiency in 21st Century skills (Voogt et al., 2013).
As a 21st Century skill that is essential for college and careers, IT literacy is critical for all students to be proficient in by the time of graduation. For SWDs, IT literacy may better prepare them for immediate employment or postsecondary education coursework. Despite this potential, IT literacy has been largely absent from special education transition services. Thus, we know very little about the relationship between IT literacy learning, transition services, and student outcomes. The purpose of this study was to examine the effectiveness of a transition curriculum focused on CCR and IT literacy. We compared secondary students who took a dedicated course with EnvisionIT as the curriculum with a comparison group who received business-as-usual transition services without a dedicated curriculum or course. Primarily, we were interested in IT literacy as an indicator of improvement, which are skills that map onto CCR and 21st Century skills. As such, our primary study objective was to examine the change in IT literacy skills of students in the intervention and comparison groups before and after the implementation of EnvisionIT.
Method
EnvisionIT (EIT) is an online curriculum intended to facilitate the transition planning process, and it is infused with IT literacy and Common Core State Standards in English Language Arts, both of which are connected to CCR and 21st century skills. The current study was conducted using a quasi-experimental group design to test the promise of the EIT curriculum to increase IT literacy skills for secondary students with disabilities.
Participants
The sample included secondary students with disabilities (n = 108) in grades 10–12 across six high schools in two states. Nine teachers from six secondary schools in Connecticut and Ohio participated in this study. In the intervention group, students were enrolled in a dedicated career or vocational elective course, some of which were inclusive general education settings and others were exclusively special education courses. In the comparison group, students received business-as-usual transition services that were not delivered in a dedicated course. Instead, comparison group students received transition services during resource room time. Although scheduled during a dedicated class period, the resource room is a time when students receive supplemental instruction in various content areas (e.g., mathematics, English Language Arts), as well as receive transition services. Thus, a major difference between intervention and comparison group students was the dedicated class period and EIT curriculum. Students enrolled or assigned to the participating teachers’ courses or caseloads were automatically selected for participation in the study. Enrolled students were asked to consent to participate with parent notification or a consent letter, depending on their age (age 18 or older were provided with a consent letter, students under age 18 were given a parental notification letter).
The Intervention
EIT is a 12-unit curriculum that is intended to facilitate the transition planning process, including the IEP measurable postsecondary goals (Izzo et al., 2010). The content is centered on career readiness and exploration as well as IT literacy, reading, and financial literacy. Since 2003, through experimental research, the EIT curriculum has been piloted in general and special education classrooms throughout Ohio and, more recently, in Connecticut in order to scale-up and sustain EIT. Izzo et al. (2010) described the evaluation of EIT effectiveness in 2007-2008 through a pretest-posttest experimental design with randomization. In this study, 15 Ohio high schools were stratified by socioeconomic status and randomly assigned to the EIT experimental condition or a control condition in which students received traditional instruction. Pretest-posttest data for 287 students, including 119 SWDs, were analyzed using hierarchical linear modeling (HLM). Findings indicated that students in the experimental group made significantly greater gains in information technology (IT) literacy than students in the control group. Students in the experimental group also showed greater gains in several transition skills, including goal setting, knowledge of how to find jobs, and information about college. These results lend credence to EIT as an evidence-based curriculum that teaches the 21st Century skills students need for success in today’s increasingly technological world.
EIT is delivered over one academic school year through an online learning management system called Schoology. While it is an online curriculum, the lessons are teacher-directed in a blended learning environment; that is, students are not encouraged to self-pace through the 12 units. Another feature of EIT is the alignment to Common Core State Standards in English Language Arts, in particular the reading for informational text, writing, and speaking and listening standards as well as technical subjects. By the end of the 12 units, students complete a Transition Portfolio designed to assist with the transition to postsecondary education, training, and/or employment after high school. Portfolio products include, to name a few, age-appropriate transition self-assessments, a 5-paragraph career narrative, a resume, a cover letter, and measurable postsecondary goals. Table 1 shows a complete list of products in the Transition Portfolio, all of which are consistent with previous research on vocational skills competencies for secondary students (Sarkees-Wircenski & Wircenski, 1994). The 12 units can be taught over the course of one semester or one school year, depending on how often the EIT lessons are taught (e.g., once per week, three times per week, etc.).
EnvisionIT transition portfolio
EnvisionIT transition portfolio
EnvisionIT literacy scale
The Envision Information Technology Literacy (EITL) scale was used as a pretest-posttest measure in the current study (Lombardi et al., 2016). The EITL scale is an untimed 20-item multiple-choice test in which students must select the correct fact-based answer. The EITL scale is a curriculum-based measure that maps onto the three domains covered in EIT: (a) tools/mechanics of the Internet (e.g., how to access and use the Internet, how the Internet is organized); (b) research processes (e.g., how to properly cite sources, how to most effectively search the Internet for information); and (c) application to career research (e.g., how to search career databases). Lombardi and colleagues (2016) utilized classical test theory, exploratory factor analysis, and item response theory approaches to validate the EITL scale on pre and post responses of students with and without disabilities. Results showed promising psychometric properties and validity evidence for the EITL scale, including preferable model fit indices (Comparative Fit Index = 0.960; Tucker Lewis Index = 0.955, Root Mean Square Error of Approximation = 0.061) and reliability (alpha estimated at 0.915).
EnvisionIT student satisfaction survey
The satisfaction survey was developed for the intervention study to measure student perspectives of the usefulness of the curriculum content after they completed the course. The survey has 19 items with a Likert response scale that ranges from 1 (none) to 4 (a lot), whereby students self-rate their skills before and after the curriculum in several key areas: (a) using the Internet, (b) reading, test-taking, and writing strategies, (c) careers of interest, (d) postsecondary education and training options, (e) setting goals and making plans, and (f) preparing for a job (e.g., resume, job interview, soft skills).
Procedures
As part of statewide scaling-up initiatives in Ohio and Connecticut, EnvisionIT was implemented in select schools. A recruitment flier was emailed from state-level transition leaders to district-level transition coordinators and special education directors. The flier described the EnvisionIT curriculum and advertised that select schools could participate. Interested schools responded and were asked to take a brief online survey that consisted of items regarding the technology capabilities in their school buildings. Because EnvisionIT is an online curriculum, it was critical for participating schools to have enough computers available for each student to work individually during class, and for all computers to be equipped with reliable Internet connections and Microsoft Office products. School partners were identified in a collaborative effort by the research team and state-level transition practitioners. After schools responded to the survey, the research team selected participants that had the best match with the project specifications.
Prior to implementing EnvisionIT, intervention teachers participated in a one-day six-hour Professional Development (PD) training. During the PD day, teachers were provided with an overview of the curriculum and practiced sample lessons in small groups. They were given some additional time to plan their academic year and consider how the EnvisionIT lessons would be embedded within their existing courses. They were also given the opportunity to set up their courses in Schoology in order to get acquainted with navigating the platform and reduce learning curve during the school year.
To collect pretest and posttest and student satisfaction data, trained members of the research team visited intervention and comparison classrooms. The EITL scale was administered as an online survey via Qualtrics. The satisfaction survey was paper-based and administered on the last day of the curriculum. All data were entered and analyzed in SPSS, version 20.0. These protocols were approved by the Institutional Review Boards to ensure the protection of human subjects.
Data analysis
To evaluate the effectiveness of EIT on student IT literacy skills, a hierarchical regression model was constructed. The outcome variable was the difference score (posttest minus pretest scores) of the EITL scale. Due to the fact that students were nested within teachers, the analysis method of choice was the multilevel linear model (MLM; Snijders & Bosker, 1999). To determine whether or not MLM is tenable, the intraclass correlation coefficient (ICC) can be calculated, which is interpreted as the proportion of variance found between level two units (e.g., teachers). If the ICC is small, an argument can be formed for not analyzing data within the MLM framework due to the fact that most of the variance is found at level one (e.g., students). As such, ordinary least square (OLS) estimates will not be biased (Snijders, T. & Bosker, 1999). The ICC for the EITL difference score was.012, indicating about 1% of the variance was attributed to level two (e.g., teachers). Because this ICC is considered low, a decision was made to carry out the hierarchical regression analysis without MLM. Finally, items frequencies and means were examined on the student satisfaction survey. These data were collected at the conclusion of the intervention to examine the social validity of the curriculum.
Results
EIT was implemented across the 2014-2015 academic school year. Within the intervention and comparison sites, there were some rates of attrition from the beginning to end of the school year. The initial number of participants (n = 150) decreased, as 42 students who were pretested were not available at the time of the posttest. Thus, the sample decreased by 28% by the end of the intervention, and the final sample size for the data analysis was 108 (intervention group = 64; comparison group = 44). The demographic characteristics of participating students are shown in Table 2. In the final sample, 62% of the participants were male, 25% were Hispanic, and 44% were eligible for free or reduced priced lunch service. The majority of the sample (79%) consisted of students with learning disabilities (LD), Attention Deficit Hyperactivity Disorder (ADHD), and Autism Spectrum Disorder (ASD). The remaining 21% of the sample represented the disability categories Emotional Disturbance, Intellectual Disability, and Speech and Language Disorder, with 10% of the sample having no documented disability.
Demographic characteristics by intervention and comparison groups
Demographic characteristics by intervention and comparison groups
aMeans and standard deviations are shown instead of N and percent.
A hierarchical regression model was constructed to evaluate the main effects of the intervention on student EITL scores. Student demographic characteristics were grouped at step 1 and group assignment (intervention or comparison group) was entered at step 2. The purpose of this model structure was to isolate the effects of the intervention while simultaneously controlling for student demographic characteristics and disability type. The combination of predictors significantly explained the variance in EITL scores; approximately 38% of the variance in IT Literacy scores was explained, R2 = 0.383, F(7, 101) = 4.91, p < 0.001. After controlling for demographic characteristics and disability type, the intervention accounted for approximately 12% of variance in EITL scores. Autism (β= 0.268, p < 0.05) contributed unique variance to the equation. Table 3 shows the regression results as well as mean difference scores by intervention and comparison groups and further disaggregated by demographic characteristics. Notably, all intervention group students across all demographic categories had positive mean difference scores; whereas, multiple comparison group students showed negative mean difference scores (e.g., students with LD, students with ADHD, students who qualify for free or reduced priced lunch).
Comparison of mean difference and standardized beta weights in IT literacy scores
Comparison of mean difference and standardized beta weights in IT literacy scores
The social validity of EIT was examined with the EnvisionIT Student Satisfaction Survey. At the conclusion of the curriculum, the satisfaction survey was given to students in order to self-rate their skills before and after the curriculum. Table 4 shows the student self-ratings across the content areas covered by EIT. Notably, students felt they improved across all areas, and percentage increases between before-and-after item pairings ranged from 9 (using the Internet) to 31 percent (preparing for a job).
The change in “A Lot” responses before and after EnvisionIT – student ratings
The change in “A Lot” responses before and after EnvisionIT – student ratings
The purpose of this study was to examine the effectiveness of a transition curriculum focused on CCR and IT literacy. In doing so, we focused on the IT literacy skills of two groups of students with and without disabilities in 10th through 12th grade: (a) those who received targeted transition services via an online curriculum that included IT literacy content, and (b) those who received business-as-usual transition services without a specific curriculum. Significant IT literacy gains were made for the students in the intervention group. Although the comparison group students also made gains, these were not as meaningful. In contrast, the comparison group students received transition services during a resource period but did not have the opportunity to work on career readiness skills in a structured classroom environment.
Overall, students who received the intervention showed notable gains in IT literacy skills, whereas comparison group students did not make the same improvements. In fact, the difference scores (shown in Table 3) show that some comparison group students decreased in EITL scores over the duration of the school year (e.g., students with LD, students with ADHD, students who qualify for free or reduced priced lunch). These findings contribute to and extend the literature base because they imply students may benefit from receiving targeted instruction in IT literacy and computer skills, including online searching and evaluating source materials (Coiro, 2012; Leu et al., 2015). Also, the current findings contribute to the notion that IT literacy could be used to supplement teacher-directed instruction (Hall et al., 2000) potentially with video anchors that situate learning (Fitzgerald et al., 2008) and provide students with more opportunities to practice fundamental reading skills (Hall et al., 2000) right up until the end of 12th grade. Although EIT is teacher-directed, it is online, delivered through a learning management system, and includes embedded multimedia content throughout text. These findings contribute to the secondary special education literature because 21st Century skills broadly and IT literacy specifically have been largely absent from intervention studies in this area. Our findings suggest that SWDs may benefit from targeted instruction in IT literacy, resulting in better preparation for college and careers.
Additionally, following completion of the intervention, students self-rated their growth (Table 4) as favorable in using the Internet, reading, test-taking and writing strategies, exploration of careers of interest, postsecondary education and/or training opportunities, setting goals and making plans, and preparing for a job. This social validity evidence further confirms the usefulness of the curriculum for secondary students especially with regard to preparation for life after high school. In other words, students who completed EIT felt they saw a change in their own growth in IT literacy and transition skills and valued the Portfolio products. Most importantly, students expressed their intention to use the Portfolio products after graduation, which is the purpose of transition services all together. Ultimately, the EIT curriculum emphasizes 21st Century skills through IT literacy and career readiness. Findings from the current study highlight the potential to link transition planning and assessment with 21st Century skills, IT literacy, and CCR. As declared in IDEA, transition services are meant to be a “coordinated effort.” We suggest, as illustrated through this study, that coordinated transition efforts could be facilitated with a specific curriculum aligned with CCR and other aspects of 21st Century learning, such as technology skills, especially for students in their final year of high school. Potentially, a targeted transition curriculum will help students link the material they learn in class with the measurable, postsecondary goals specified on their IEPs, and ultimately better prepare them for adult life.
Limitations
Although this study shows promising findings and support for a curriculum dedicated to transition planning with embedded technology features, there are several important limitations to consider in the interpretation of the findings. First, the study was underpowered and included only six high schools in two states. Scaling-up efforts will be crucial to test the effectiveness of the curriculum on a larger scale in future studies. These efforts must also include investigation of fidelity of implementation of EIT to a broad range of high school teachers of special and general education courses. Second, outcome measures were limited to project-based measures and were not linked to the school-based assessment (e.g., grade point average, course grades). Collection of these data were outside of the scope of the current project but should be considered as another indicator in future studies. In particular, school-based measures can help validate the project-based measures. Third, the study design was quasi-experimental and random assignment was not utilized. Fourth, the range of disability categories is not representative of all categories covered under the IDEA. The highest representation from the study sample was from LD, ADHD, and ASD. While LD continues to be the most prevalent disability category in K-12 environments (USDOE, 2014), there are other categories that were not well represented in this study (e.g., Emotional/Behavioral Disorders, Speech/Language Disorders, Hearing Impairments, Intellectual Disability).
Implications for research and practice
Despite these limitations, the findings from this study are promising with regard to promoting CCR, linking IEP goal statements with targeted instruction and activities, and improving the IT literacy skills of secondary students with IEPs. IEP teams should consider more efforts around linking transition goals statements, specifically those linked to a transition portfolio, to course content. Specifically, embedding a transition curriculum such as EIT has several advantages: (a) course flexibility, as it can be embedded into a general education English Language Arts course or a career elective; (b) targeted instruction on career-related skills, such as resume writing, job interviewing, and computer skills, all of which will have meaningful impact on adult life immediately after high school; and (c) none of the curriculum is intended only for SWDs, suggesting students without disabilities may just as easily benefit from EIT. Thus, because both students with and without disabilities could benefit from the curriculum, implementation in inclusive, general education settings is preferred and suggested.
Ultimately, part of promoting CCR is to equip imminent high school graduates with flexible tools, products, and resources they will use immediately after high school that will be applicable to a wide variety of purposes (e.g., job applications, postsecondary school or employment training). IT literacy is a 21st Century skill used in workplace and postsecondary education settings. Resumes, cover letters, and interviewing skills are also used in workplace and postsecondary education settings. Providing teachers with the tools and resources to explicitly teach these skills in a variety of settings within and outside of special education should be prioritized. This study shows an example of a coordinated effort of transition services that enables teachers to systematically teach skills that are included in transition goal statements on IEPs. Giving teachers a dedicated class period, rather than expect them to provide transition skills during resource room time along with other areas of remediation, is one example of providing teachers with adequate time and resources to better prepare students for adult life.
Conflict of interest
The authors have no conflict of interest to report.
Footnotes
Acknowledgments
The EnvisionIT curriculum and related products were produced under the U.S. Department of Education (USDOE), Office of Special Education Programs Grants H327A020037, H327A050103, and H327S120022. The views expressed herein do not necessarily represent the positions or polices of the USDOE. No official endorsement by the USDOE of any product, commodity, service, or enterprise mentioned in this document is intended or should be inferred.
