Abstract
Content Acquisition Podcasts for Teachers (CAP-T) are a form of multimedia-based instruction that are supported by an empirical record of effectiveness and are grounded in Mayer’s cognitive theory of multimedia learning and accompanying instructional design principles. In this study, 162 students enrolled in an introductory course in special education were assigned to watch CAP-Ts for a variety of topics throughout the semester. Students tracked their viewings per CAP-T and self-reported the number of views during the midterm and final examinations. Researchers included clusters of questions coded to each CAP-T on the exams. Linear regression was used to predict student performance on each cluster of questions based on the number of CAP-T viewings. Results show a positive, predictive relationship between CAP-T views and performance on midterm and final exam questions overall, but some unexpected findings emerged when looking at individual subcomponents of the assessments.
It is well documented that most students with disabilities spend the majority of the school day in inclusive, general education settings (U.S. Department of Education, 2012). Most teacher preparation programs, however, offer a single survey course to introduce the basics of special education to future general educators, including the 14 categories of disability covered by the Individuals with Disabilities Education Act (Reschly, Holdheide, Behrstock, & Weber 2009). Although this content is important, all preservice teachers require strong content and pedagogical knowledge on which to build their skills (Brownell et al., 2009; Grossman & McDonald, 2008). This requires an extension of content knowledge that races far beyond what is generally offered in an introductory course (Leko, Brownell, Sindelar, & Kiely, 2015). Therefore, the “disability of the week approach” (usually delivered via a combination of lecture and activities) used by many instructors during introductory courses may leave enrollees without meaningful exposure to key content needed for successful teaching. This is not necessarily the fault of the instructor—the curriculum is the curriculum—but it remains our collective problem as a field to try and address this omnipresent issue (Lignugaris/Kraft & Harris, 2014; Maheady, Smith, & Jabot, 2014).
A seemingly obvious solution is to require (or at least strongly suggest) all future teachers take more than one course preparing them to teach students with disabilities; however, programs are already overloaded with coursework, field placements, and other experiences, so adding new requirements is unlikely to become the norm (Blanton, McLeskey, & Hernandez-Taylor, 2014; Grossman & McDonald, 2008; Leko et al., 2015). This situation constitutes a wicked problem faced by most in the field of special education and teacher preparation. We know most students with disabilities are being taught at least some of the day by teachers not at all prepared to address their individualized needs (Brownell, Griffin, Leko, & Stephens, 2011; Pugach, Blanton, & Boveda, 2014), yet we as faculty in teacher preparation programs are somewhat hamstrung in terms of options for addressing this issue. New and creative thinking is needed.
Stakeholders in the field of teacher preparation can trace novice teachers’ instructional deficiencies across K-12 placements to knowledge and skills they failed to acquire and/or master during preparatory programs (Lignugaris/Kraft, Sindelar, McCray, & Kimerling, 2014). Of significant concern are methods and dosage of instruction used to provide preservice teachers with exposure to key content and practices needed to teach students with disabilities (Brownell, Sindelar, Kiely, & Danielson, 2010; Lignugaris/Kraft et al., 2014). As an example, although few would get into an airplane helmed by a pilot who learned to land during a semester primarily comprised of lectures and discussions, we as teacher educators routinely sit back and tacitly allow students with disabilities to be served by general education teachers who acquired knowledge and skills in this fashion. Teacher educators do this despite full awareness that this type of learning is unlikely to transfer to effective practice (Bransford & Schwartz, 1999).
There are calls from leaders in the field (e.g., Brownell et al., 2010; Leko et al., 2015) for preparation programs to move away from teaching about practice, and instead refocus on a series of carefully structured practical experiences where preservice teachers apply specific instructional practices in increasingly diverse and realistic scenarios. We agree this is essential, but simultaneously maintain preservice teachers must acquire knowledge before it can be reconstituted as pedagogical skill (Shulman, 1987). If teacher educators can streamline methods for providing preservice teachers with essential content knowledge, it may be possible to free up time, even within the confines of the one introductory course, to refocus attention on the learning and implementation of evidence-based practices.
The purpose of this article is to extend the conversation and empirical literature around the need to improve instructional options for teacher preparation coursework offered to general education candidates in the domain of special education. Although there are numerous possible pedagogical avenues to pursue in this charge, multimedia currently enjoys a prominent role in the field’s psyche (Sayeski, Hamilton-Jones, & Oh, 2015; Smith & Kennedy, 2014). In the following section, we present empirical evidence from a naturalistic study of a time-saving, multimedia-based instructional approach called Content Acquisition Podcasts for Teachers (CAP-T). Our thesis, grounded in empirical research, is CAP-Ts can provide general and special education preservice teachers with essential content knowledge contained within introductory coursework in an efficient and effective manner. If effective, instructors could confidently move toward Leko et al.’s (2015) call to make teacher preparation more focused on deliberate and scaffolded practice.
The Theory and Instruction of CAP-Ts
There is strong empirical evidence that CAP-Ts are an effective tool for supporting preservice teachers’ learning about various topics within the introductory course in special education (Dieker et al., 2014). CAP-Ts are not really podcasts in the way that many would think about them, but are relatively short (approximately 5-15 minutes), multimedia-based vignettes. The term podcast is used in this work because of many users’ familiarity with and acceptance of podcasting as a valued instructional tool in higher education (Evans, 2008). CAP-Ts can be created for teacher candidates and in-service teachers (Kennedy, Alves, & Rodgers, 2015), whereas CAP-Ss are used for students with and without disabilities (see Kennedy, Deshler, & Lloyd, 2015). A sample CAP-T can be seen at https://vimeo.com/143387419
CAP-Ts are not intended to fully replace lectures or assigned readings. Instead, they offer learners a supplement to existing instruction and a way to receive high-quality instruction beyond the boundaries of the traditional classroom. In practice, instructors assign CAP-Ts to accompany weekly readings and recommend students use the video either as an advance organizer prior to reading the textbook or as a review afterward.
Theory of CAP-Ts
CAP-Ts are set apart from generic audio-only, enhanced, or video podcasts, because they use Mayer’s cognitive theory of multimedia learning (2009) and accompanying 12 instructional design principles (2008) to guide the design and delivery of instruction. Mayer’s design principles provide a roadmap that helps instructors make design decisions that are considerate of how people learn. In the following section, we introduce seven critical principles and discuss how they influence CAP-Ts. The five not explicitly discussed (multimedia, modality, temporal contiguity, image, and voice principles) are automatically addressed within the CAP-T model.
First, the coherence principle (Mayer & Jackson, 2005) holds that a learner will retain more of her available cognition to attend to new information if instruction is constrained to only the most important content. All CAP-Ts only contain the most critical information relevant to the topic being taught. The signaling principle (Stull & Mayer, 2007) states that explicitly highlighting key information within instruction will result in better learning outcomes. Each CAP-T has explicit cues (text and audio) throughout that orient the viewer’s attention to critical points. The redundancy principle helps instructors recognize that multimedia-based instruction that asks a learner to use his or her visual and auditory inputs to acquire the same content (e.g., think of a PowerPoint slide with only words while the presenter also reads the slides but also elaborates into other content) is not a good match for maximizing limited cognitive resources. Instead, slides with a clear image and a short phrase of key content will result in better learning (Mayer & Johnson, 2008). Students in Kennedy’s courses are told that when text appears within a CAP-T, it is important, and likely the answer to a future test item. Otherwise, text is not used in the videos.
Other design ideas include Mayer’s spatial contiguity, pretraining, segmenting, and personalization principles. The spatial contiguity principle (Moreno & Mayer, 1999) states core content should be strategically placed on a screen to limit eye movement. All CAP-Ts have one core image, placed in the middle of the screen, and when text is used, it also appears right above or below the image. The pretraining principle (Mayer, Mathias, & Wetzell, 2002) reminds instructors to provide an advance organizer or define key terms/concepts that appear within instruction. CAP-Ts begin with a specific statement introducing the key content and defining new concepts. The segmenting principle (Mayer, Dow, & Mayer, 2003) holds that instruction broken into learner-sized chunks will help scaffold learning success. CAP-Ts contain embedded pause points where the instructors can insert questions. Finally, the personalization principle (Moreno & Mayer, 2004) states that multimedia recorded by students’ instructor for a specific purpose in a conversational voice (not generic or “off the rack” technology) results in augmented learning. Kennedy and his team created CAP-Ts for the introductory audience in his special education courses.
The use of Mayer’s model to create CAP-Ts sets this intervention apart from other multimedia-based instructional tools (Kennedy, Alves, & Rodgers, 2015). For example, generic, audio-only podcasts do not have any specific learning theory that underwrites their capacity for learning (Heilesen, 2010). Kennedy, Hart, and Kellems (2011) demonstrated that teacher candidates who learn using CAP-Ts learn and retain significantly more content than peers who listened to an audio-only podcast. Other well-known products, such as the IRIS modules (iris.peabody.vanderbilt.edu), are largely driven by text, and also have a broader purpose than CAP-Ts. Sayeski, Hamilton-Jones, et al. (2015) demonstrated teacher candidates made significant learning gains across various content areas when learning using this resource. A worthwhile research question would be a comparison in teacher candidate learning when using CAP-Ts compared with using the IRIS modules. It would also make sense to embed CAP-Ts within, or use alongside the IRIS modules to further bolster the utility of that outstanding resource.
Research Supporting CAP-Ts
To date, there are 16 published studies demonstrating CAP-Ts’ positive impact on learning of teacher candidates. The first group of studies (n = 11) were simple comparisons of learning when teacher candidates watched a CAP-T or read a practitioner-oriented chapter or article on the same topic (Carlisle, Thomas, & McCathren, 2016; Driver, Pullen, Kennedy, Williams, & Ely, 2014; Ely, Pullen, Kennedy, Hirsch, & Williams, 2014; Hart & More, 2013; Kennedy, Aronin, et al., 2014; Kennedy, Driver, Pullen, Ely, & Cole, 2013; Kennedy, Ely, et al., 2012; Kennedy et al., 2011; Kennedy & Thomas, 2012; Kennedy, Thomas, Aronin, Newton, & Lloyd, 2014; Sayeski et al., 2015). Following that group of studies, researchers began comparing learning using CAP-Ts to live lectures (Hirsch, Kennedy, Haines, Thomas, & Alves, 2015), and including measures that provide additional insights into why this tool is powerful, such as motivation (Kennedy, Wagner, et al., 2016), and perceived cognitive load (Kennedy, Hirsch, et al., 2016). Finally, researchers have used CAP-Ts in conjunction with modeling videos (CAP-TV) to support teacher candidates’ learning and implementation of evidence-based practices (Ely, Kennedy, Pullen, Williams, & Hirsch, 2014; Ely, Pullen, Kennedy, & Williams, 2015).
Research supporting CAP-Ts is primarily experimental (n = 16 published studies), but is also somewhat limited with respect to generalizability. To illustrate, the CAP-T line of research has followed standards for rigor of quantitative research in special education (Odom et al., 2005), randomly assigning participants, and exerting careful experimental control over the instructional conditions for the independent variable and comparisons. During all but one of the published studies of CAP-T effectiveness, participants were randomly assigned to either watch a CAP-T, read a practitioner-oriented version of the same content, or receive a lecture in a laboratory setting. Participants in the CAP-T groups spread out within a computer lab or classroom, watched the video(s) at individual laptop or computer terminals, and wore headphones. Although researchers in each experiment instructed students to pause videos and take notes as usual, it is doubtful that this distraction-free setting is congruent to how most university students complete assigned CAP-Ts in preparation for class. Instead, when used as a weekly learning tool, instructors recommend students watch videos at their convenience and use methods that work well for them.
The dependent measure for the previous CAP-T studies is a test of knowledge given the same day as the CAP-T viewing, or, in newer studies (e.g., Hirsch et al., 2015), 1 week later. Although CAP-T studies separate independent and dependent variables with at least an hour-long distractor (e.g., lecture on a different topic), it remains a test of short-term learning. Although CAP-Ts have consistently demonstrated positive and significant learning gains, an unanswered research question is the extent to which naturalistic viewings of CAP-Ts over different periods of time (e.g., half semester, full semester) affect performance on items presented during a course’s midterm and final exam. Such a test would begin to reveal the durability of learning. This is the impetus for the present study.
The Present Study
In preliminary studies of a novel intervention, researcher control is desirable to provide proof of concept and evidence for efficacy (Cook & Campbell, 1979). As efficacy evidence accumulates, there is a corresponding obligation on the researcher to reduce experimental control and test the intervention in contexts that more closely resemble conditions that would arise given widespread adoption. The present study builds on previous research on the efficacy of CAP-Ts in important ways.
First, researchers asked students to record and report how many times they watched each video in preparation for class and leading up to exams. These factors permit an investigation of the relation of CAP-T views to midterm and end of course knowledge outcomes, and preliminary data on the ways in which students utilize CAP-Ts given a naturalistic design and according to individual learning needs. Also, learning was checked at two distal time points compared with the short-term learning typically measured in previous CAP-T studies. Thus, two research questions guided the study:
Method
Participants
Participants in this study were 162 students enrolled in one section of a course titled “The Exceptional Learner,” an introductory course in special education at a large, prestigious public research intensive university on America’s east coast. Although exempted by the institution’s human participants review board, all students were given opportunity without penalty to refuse personal contribution (demographic data, number of CAP-T views, and midterm and final exam scores) to the data set. The study was introduced to students via a faculty member not affiliated with the course or research. Researchers did not have access to the participant’s reported CAP-T views until the conclusion of the semester. One hundred percent participation was achieved, with no students opting out and no attrition. For the purpose of analyses, names of individual students and their reported CAP-T views were linked in a database with their corresponding responses from the midterm and final exams, checked for accuracy by two researchers, and the data were then de-identified using numbers as codes. No breaches of confidential student data (e.g., grades) were reported.
Participants included undergraduate (25.3%) and graduate students (74.7%). There are no study-relevant differences in the assignments or structure of the course for undergraduate or graduate students. The course is required for teacher licensure in the elementary (43.5% of the participants) and secondary (35.1%) education programs, and is also taken by students in the communication disorders program (11.5%), other majors (e.g., kinesiology) within the school of education (3.4%), and non-education students seeking to learn more about the topic (6.5%). Unlike previous CAP studies, the majority of the participants were graduate students. Like many courses within teacher preparation courses at large, flagship universities in the United States, 13.5% of the participants in this course (and study) were male, and 86.5% were female. Similarly, 78.3% of the participants were Caucasian, 12.2% were Black, and 5.2% were Hispanic; the remaining students, 4.3%, reported other races.
Procedures
Students were introduced to CAP-Ts during the first session of the course as part of the syllabus review. CAP-Ts for each week’s session were included as links embedded within the syllabus. The instructor of the course explained the purpose and structure of CAP-Ts, and recommended strategies for their use (e.g., watching the CAP-T before assigned readings as a Pre-CAP, or after reading as a Re-CAP). CAP-Ts were assigned as part of the required preparation for class along with reading the course textbook and other activities (e.g., reviewing websites). Research findings from previous studies (e.g., Kennedy, Hirsch, et al., 2016) were presented with intent of convincing students that CAP-Ts are an effective and efficient tool for learning, and that other students had benefited from accessing CAP-Ts during the course.
Faculty members and doctoral students at the researcher’s university from 2011 to 2015 created the CAP-Ts used in this study. All CAP-Ts (from this study and beyond) are available for free at the Introduction to Special Education site on Vimeo at www.SPEDIntro.com. The technical process for creating a CAP-T is described in detail elsewhere (see Kennedy, Kellems, Thomas, & Newton, 2015), and a two-part CAP-T on how to create this tool is included on the aforementioned Vimeo site. A few words about the processes relevant to this study are warranted here. To keep the length of CAP-Ts to a manageable time and adhere to Mayer’s (2008) coherence principle, we often split videos into Parts 1 and 2. To illustrate, for most of the CAP-Ts providing content on specific disability categories (e.g., specific learning disabilities), Part 1 covers the characteristics, causes, prevalence, and related information. Part 2 contains evidence-based practices and prominent accommodations and modifications. In some cases, specific practices such as video modeling and curriculum-based measurement are captured as separate individual CAP-T to provide learners with more detail on each topic.
In addition, for this study, the instructional team used CAP-T to “flip the classroom” (Bull, Ferster, & Kjellstrom, 2012), meaning students were assigned to watch CAP-T outside class, much like an assigned reading, to develop background knowledge on important topics so that face-to-face instructional time would not be dominated and consumed providing lectures covering the same information contained within the required readings and CAP-Ts. Rather, class time could be spent working on case studies, listening to guest speakers (e.g., practitioners and other experts), engaging students in elaborated discussions, watching and participating in modeling of new practices, and watching clips of videos relevant to the topic. The instructor only provided in-class content from videos when students asked questions or other data such as questions, missed quiz, or test items indicated a review was necessary.
A researcher unaffiliated with the research team introduced the study. They instructed students to keep careful track of how many times they watched each CAP-T during the semester. Although instructors from this research team now use www.EdPuzzle.com as a way to capture individual student viewings, at the time of this study the best option we had available was the self-report approach. An excel spreadsheet was provided to students as a suggested way to record data. Any time the student watched three quarters or more of the CAP-T it counted as a viewing. Students were informed that a significant group of questions on the midterm and final exams would come exclusively from the CAP-Ts, and thus, it would behoove them to access videos on a regular basis for preliminary exposure to course content, and during test review processes. During the midterm exam, in class, students were instructed to use their own device and login to report the number of viewings per CAP-T into an online database, which was not accessible to the instructional team. This database was kept separate from the exam itself, which was accessed by its own individual link and exam data were accessible to the instructor team for course-grading purposes. The process was repeated during the final exam, with students now asked to report only the number of CAP-T views since the midterm.
Measures
The measures in this study are questions written by the researchers (who are also the instructor and teaching assistants of the course) for the midterm and final exams. This same group created all of the CAP-Ts for the study and attended all class sessions, so the content is well known to the team. Researchers broke into small groups to write questions based on CAP-Ts and brought them back to the full team. Each proposed question was examined to ensure it is fully answerable using videos, and for adherence to overall best practices for item construction and psychometric testing (e.g., American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999). Total questions (n = 110) are a blend of multiple choice (n = 88), true and false (n = 8), and open-ended (n = 14) items. There were 47 questions on the midterm, which covered content from the first half of the semester, and 63 questions on the final exam.
Both the midterm and final exams contained items specifically drawn from individual CAP-Ts as well as items drawn from other course resources such as lecture, case study, readings, websites, and the course textbook. The final exam was cumulative. Of the 47 items on the midterm, nine were not from CAP-T content, and on the final, 14 of the items were not from CAP-Ts. Multiple-choice and true/false items were scored automatically (1 = correct, 0 = incorrect) by our university’s course management system. Open-ended items were scored by the instructor and a teaching assistant using a calibrated rubric. For the purpose of the study, reliability checks were conducted on 20% of students’ exams to evaluate interscorer reliability. Scores were initially calibrated at 96%, and discrepancies were resolved through discussion.
Psychometric properties of instruments
Researchers completed several statistical tests to establish the psychometric properties of the instruments used in this study. Results demonstrate the midterm and final demonstrated adequate psychometric properties and demonstrated few differences between items drawn from CAP and non-CAP content. To illustrate, the mean facility index across all items was 78% (SD = 21.2), which suggests that the items are fairly easy; however, they are sufficiently difficult to introduce separation among examinees. The facility index did not differ between CAP and non-CAP items (77% [SD = 21.2] and 79% [20.2], respectively), suggesting the items functioned similarly. This is further bolstered by data from the subtest-level discrimination index, demonstrating the CAP items (27% [SD = 19.9]) did not function differently from the non-CAP items (25% [SD = 22.7]). Results demonstrate there was modest skewness for the full instrument (−.70) and slight excess kurtosis (.68). Cronbach’s alpha for the full instrument was .87 at midterm and .89 on the final.
The effective weight represents an index of the contributions of specific items to the total score. Across all items, the mean effective weight was comparable with the intended weight. Ten items demonstrated negative co-variances with the total score, suggesting an unexpected relation with the outcome of interest. Thus, analyses were conducted with and without these items to determine whether these items were exerting undue influence on the total exam score. Results across the analyses were similar. In addition, the effective weights of CAP items and non-CAP items were evaluated. Both sets of items demonstrated mean effective weights, which were comparable with their intended weight (CAP items: effective weight = 1.16, intended weight = 1.16; non-CAP items: effective weight = 8.33, intended weight = 8.33).
Design
Students’ self-reported CAP-T views were used in a regression model to predict performance on specific questions contained within the midterm and final exams. In addition, researchers explored the relationships between individual CAP-T topics, views, and performance on the matching items on midterm and final exams.
Results
Primary Analysis: The Relation of CAP Views to Knowledge Outcomes
The primary research question addressed whether the total number of CAP-T views predicted student performance on specific items contained within the midterm and final exam. This research question was answered with two linear regression models, in which exam performance was regressed on the total number of CAP-T views completed prior to the midterm and prior to the final. To facilitate interpretation, both independent variables (midterm and final CAP-T views) were centered at the sample mean. Thus, the intercept represents the sample mean and the beta coefficient represents the adjustment in predicted exam score for each additional CAP-T view above or below the sample mean number of views.
Results indicate that the number of CAP-T views completed at midterm was a significant predictor of participants’ midterm exam score, F(1, 160) = 40.26, p < .001, R2 = .20. For each CAP-T view above the sample mean number of views, participants predicted performance on the midterm exam increased by .31 points. Conversely, for each CAP-T view less than the sample mean number of views, participants’ predicted performance on the midterm exam decreased by .31 points. A student who viewed the mean number of CAP-T prior to the midterm (M = 25.53, SD = 9.00) would be predicted to receive a score equal to the sample mean midterm exam performance (M = 70.64, SD = 6.21).
Results indicate that the number of CAP-T views completed prior to the final exam was a significant predictor of participants’ final exam score, F(1, 160) = 22.87, p < .001, R2 = .13. For each CAP view above the sample mean number of views, participants’ predicted performance on the final exam increased by .13 points. Conversely, for each CAP-T view less than the sample mean number of views, participants’ predicted performance on the final exam decreased by .13 points. A student who viewed the mean number of CAP-T prior to the final (M = 33.45, SD = 13.06) would be predicted to receive a score equal to the sample mean final exam performance (M = 58.94, SD = 6.21).
Exploratory Analysis: Differences in CAP-T Utilization and Outcomes by Topic, Difficulty, or Time
The second research question addressed whether, in naturalistic settings, student utilization of specific CAP-Ts differed by topic. This exploratory question was addressed through an analysis of descriptive statistics, including rank orders by topic to evaluate the extent to which CAP-Ts are utilized differently and whether future study is warranted.
Table 1 provides descriptive statistics on the number of CAP-T views by topic for the midterm and final exam. The mean number of CAP views at midterm ranged from 1.7 (SD = 0.9; Related Services) to 3.9 (SD = 1.9; Individualized Education Programs [IEPs]). The median number of views at midterm was 2.15. For the final exam, the mean number of views ranged from 1.6 (SD = 1.2; Co-Teaching) to 4.4 (SD = 1.4; Characteristics of Students with Autism Spectrum Disorders). On the final exam, the median number of views was 2.9.
Descriptive Statistics for CAP Views and Exam Performance by Topic and Sorted by Percent Correct for Discrete Topics.
Note. CAP = Content Acquisition Podcasts; IDEA = Individuals With Disabilities Education Act.
In addition, Table 1 notes the percent of items correct by topic on the midterm and final exam. Within the table, topics are sorted from the highest percent correct to lowest percent correct for the midterm/final and topic to facilitate the identification of patterns between CAP-T views and percent correct by topic. The mean percent correct by topic for the midterm ranged from 73.0 to 95.5. The median percent correct was 86.0. For the final exam, the percent correct ranged from 57.7 to 93.7 and median percent correct was 80.1. Ceiling effects were minimal as few students scored perfectly on sets of questions.
To evaluate whether there may be patterns that would suggest a relation between different utilization of specific CAPs and the percent correct on that topic, we evaluated a list by the number of views and percent correct for specific topics on the midterm and final. The analyses provide interesting insights that suggest in some instances, CAP-Ts viewed fewer times on the midterm resulted in higher scores for matching items. In fact, four of the five CAP-T topics where students scored highest on the midterm had the lowest average views. These topics included, working with families, introduction to related service providers, universal design for learning, and using accommodations and modifications (the fifth where views were higher was the CAP-T on response to intervention). On the final exam, four of the six top scores for items came from CAP-Ts viewed more often (e.g., transition, characteristics of students with learning disabilities, autism spectrum disorders, and emotional/behavioral disorders). We hypothesize reasons for these disparate findings in the discussion.
Discussion
The call for sustained programs of research that explore multiple facets of an intervention or instructional approach is often repeated in the field of teacher preparation (see Lignugaris/Kraft et al., 2014). Specifically, the field of teacher preparation is looking for instructional methods and tools that can help instructors provide future educators with essential knowledge in efficient, but powerful, ways so that the more important issue of practice implementation can be addressed (Leko et al., 2015). This need is not fully addressed by the body of literature amassed over the years—indeed, in many ways, we have more questions than answers (Sindelar, Brownell, & Billingsley, 2010). Significant room remains for scholars to innovate and rigorously test their ideas in light of specific learning needs within course and fieldwork. The program of research described in this article is part of an attempt to address the need for high-quality research in this field (Dieker et al., 2014).
As noted, researchers from multiple universities have published 16 articles that report empirical results demonstrating capacity of CAP-Ts to improve teacher candidate learning across a range of content. Before this project, the effectiveness of this tool was limited to laboratory-style experiments. Thus, demonstrating the effectiveness of CAP-T in a naturalistic setting, and over long periods of time, is an important advance of the CAP-T line of research. These results, although preliminary, provide instructors a measure of confidence that this tool can be used throughout an introductory course to help ensure teacher candidates learn essential content.
It is acknowledged that teachers need to demonstrate learning in more sophisticated ways than answering multiple-choice items on a midterm and final exam. That said, the predecessor to implementation of evidence-based practices is the type of learning delivered using CAP-Ts and measured within this study during an introductory course with 162 students. As teacher candidates progress beyond introductory coursework, they become engaged in practicum experiences and methods courses where they are required to apply their learning. A logical next step after this study would be to follow participants into their methods courses to evaluate the extent to which they are implementing evidence-based practices with fidelity. It is also important to consider the use of measures beyond midterm and final exams with multiple-choice and open-ended questions, even in introductory coursework to evaluate student learning following use of CAP-Ts. Research conducted by Ely and colleagues (2015) is an example of the type of applied learning teacher candidates are expected to demonstrate following use of CAP-Ts and CAP-TVs. Ely and colleagues were answering a different research question than addressed here, but it does have a valuable lesson for future planning and research.
Exploring Results and Implications for Practice
The main finding of this study, that more CAP-T views predict higher performance on exam questions, provides corroboration for the previously published findings using this tool in laboratory settings. This finding also expands on previous research given the longer space of time between CAP-T views and the evaluation on the midterm and final exams. Due to the nature of our naturalistic research question, we cannot be precise regarding the length of time between CAP-T views and the two data collection points; however, it is certainly longer than the previous studies. It is logical to expect (and was our hypothesis) that more CAP-T views would result in a boost in performance for teacher candidates on assessments that counted for a substantial portion of their course grade. Prior to this study, there was not empirical evidence to support that assertion. Therefore, the findings of this study are important, and can be used to help future teacher candidates understand the value in using this learning tool as part of their learning during introductory coursework.
For the second exploratory research question, results raise interesting questions. Given the aggregate effects, it was expected that on topics for which students viewed more CAPs, they would perform better. The pattern of results for some topics suggests the opposite may be true. This is possibly explained by the content and questions of some CAP-Ts being easier than others. In other words, it is possible that CAP-Ts with easier content led students to view the video less times because it was mastered quickly. In addition, variability in the number of views can be attributed at least in part to the emphasis placed on each topic by the instructor during class (e.g., more time and emphasis were placed on IEPs than related service providers). It logically follows that questions on those topics were also easier, and thus correctly answered at a higher rate than items from more difficult content.
Data presented in the “Method” section on the technical qualities of the instrument offer some clarity to this point. For example, a modest amount of skewness and kurtosis was observed for the full instrument, and the facility index statistics demonstrate most students found the overall exams to be relatively easy. When looking at individual question level data for facility index for midterm questions stemming from the CAP-Ts on working with families, introduction to related service providers, universal design for learning, and using accommodations and modifications, results support the hypothesis that these items were easier than the exam on the whole. Although we are still unable to be conclusive that students found this content easier than other topics and adjusted their CAP-T viewing patterns accordingly, this combination of evidence is persuasive and should be explored further in follow-up studies.
Results on the final exam more closely fit our expectations given the overall result. One reason why could be that the lion’s share of the course’s content fell after the midterm this particular semester due to the University’s unbalanced scheduling and a snow day that canceled a meeting before. In addition, the meatier content, such as the characteristics and evidence-based practices for teaching students from the various disability categories all fall after the midterm. Thus, in the view of the course instructors, the more important and challenging content led to more CAP-T views and solid performance on the final exam. It is also likely that as the semester progressed, the students in the course simply became better consumers of CAP-Ts. In other words, as cumulative number of views went up, so did comfort with and ability to learn from CAP-Ts. Although we do not have specific data to back this claim, it is a logical hypothesis to explore in future research. Data from the facility index demonstrate the final exam was more difficult than the midterm, and the distribution of scores was less skewed. This provides evidence supporting the main finding of the study, that more CAP-T views led to stronger performance on the cumulative final exam for the course.
In summary, instructors of introductory courses in teacher preparation courses can rely on CAP-Ts as a tool that can arm future teachers with the content they need to be successful. Although the type of knowledge learned from CAP-Ts does not automatically convert into improved practice, it is a solid grounding for advanced implementation and sustainability of evidence-based practices (Shulman, 1987).
Limitations
There are several important limitations to this study. First, students self-reported their number of CAP-T views. Although they were provided with an Excel spreadsheet and periodic reminders to track views, provided data are ultimately subject to errors. Researchers such as Salvucci, Walter, Conley, Fink, and Saba (1997) and Cook and Campbell (1979) offered caution to those collecting and interpreting self-reported data as they can be over-reported or underreported and subject to bias. Others such as Chan (2009) had more faith that self-reported data can be credible and useful. Chan (2009) explained that participants in research studies who were explicitly cued to remember specific information and were provided with a specific method to do so can be counted on to provide reliable recollections. Despite erring on the side of caution and acknowledging this is a significant limitation, we are cautiously optimistic that the data are reasonably accurate as there would be little or no reason for students to provide false data. It cannot be ruled out that students over-reported CAP-T views in an attempt to please the instructor/researchers and thus help confirm the unstated, but obvious research hypothesis that more views may result in higher achievement.
Relatedly, we have no way of knowing when, or the extent to which participants were watching or listening to CAP-Ts and what competing demands for their attention were present. In previous studies, researchers carefully controlled CAP-T access using headphones and individual laptop terminals. To a large extent, distractions were eliminated and attention was focused on the content of the CAP-Ts. Although the purpose of this study was to explore the impact of CAP-Ts when those experimental controls were not in place, it therefore doubles as the study’s purpose and one of the important limitations.
Next, all dependent measures were created by the researchers/instructors. Although items are used every semester on midterm and final exams (and been revised to eliminate confusing wording and items that are too easy), data presented herein demonstrate that some are easier and harder than others. The variability in ease/difficulty in items is a major contributor to the exploratory finding that some CAP-Ts with less views yielded higher scores on the exam questions. Findings would be strengthened by using items that had standardized psychometric properties. Even with items subjected to additional pilot and field testing, some content in the introductory course in special education is simply easier for students to learn than others (see above). Previous coursework, common sense, and life experiences are all contributors to this variability. Relatedly, important data that contributed to the students’ exam scores such as instructor quality and individual study habits and achievement levels were not measured or included in analyses. Although it would be difficult, if not impossible, to objectively measure instructor quality and include it in a statistical model, this study did only have the one instructor. Future studies might use student Grade Point Average(GPA) or Scholastic Achieveme Test (SAT)/American College Test(AP) scores as covariates to help explain findings.
Researchers did not collect data from participants that would explain why they watched some CAP-Ts more or less than others. Although it can be implied fewer views stem from easier content and more views relate to more difficult content, we cannot be certain. Students in a subsequent section of the course with highly similar demographics and characteristics were asked via survey why they watch CAP-Ts more or less than others, and almost all (n = 134) strongly agreed (m = 4.87/5, SD = .35) that “I watch CAP-Ts more often when content is harder to learn.” In addition, students were not queried with respect to their opinions of the CAP-Ts. Although useful, this information is not of paramount concern given the headline finding that more CAP-T views result in higher scores on the midterm and final exams.
Conclusion
Given the strong empirical record of CAP-Ts to improve teacher candidate learning of important topics in introductory coursework in special education, we are encouraged by the results of this naturalistic study. As more researchers and instructors use or create their own CAP-Ts, we look forward to more replications of highly experimental, naturalistic, and qualitative studies that help provide additional information on the utility of this tool.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.