Evaluating the Relationship Between Naturalistic Content Acquisition Podcast Views and Course Performance

Abstract

Content Acquisition Podcasts for Teachers (CAP-T) are a form of multimedia-based instruction that are supported by an empirical record of effectiveness and are grounded in Mayer’s cognitive theory of multimedia learning and accompanying instructional design principles. In this study, 162 students enrolled in an introductory course in special education were assigned to watch CAP-Ts for a variety of topics throughout the semester. Students tracked their viewings per CAP-T and self-reported the number of views during the midterm and final examinations. Researchers included clusters of questions coded to each CAP-T on the exams. Linear regression was used to predict student performance on each cluster of questions based on the number of CAP-T viewings. Results show a positive, predictive relationship between CAP-T views and performance on midterm and final exam questions overall, but some unexpected findings emerged when looking at individual subcomponents of the assessments.

Keywords

multimedia podcasts teacher education intervention higher education

It is well documented that most students with disabilities spend the majority of the school day in inclusive, general education settings (U.S. Department of Education, 2012). Most teacher preparation programs, however, offer a single survey course to introduce the basics of special education to future general educators, including the 14 categories of disability covered by the Individuals with Disabilities Education Act (Reschly, Holdheide, Behrstock, & Weber 2009). Although this content is important, all preservice teachers require strong content and pedagogical knowledge on which to build their skills (Brownell et al., 2009; Grossman & McDonald, 2008). This requires an extension of content knowledge that races far beyond what is generally offered in an introductory course (Leko, Brownell, Sindelar, & Kiely, 2015). Therefore, the “disability of the week approach” (usually delivered via a combination of lecture and activities) used by many instructors during introductory courses may leave enrollees without meaningful exposure to key content needed for successful teaching. This is not necessarily the fault of the instructor—the curriculum is the curriculum—but it remains our collective problem as a field to try and address this omnipresent issue (Lignugaris/Kraft & Harris, 2014; Maheady, Smith, & Jabot, 2014).

A seemingly obvious solution is to require (or at least strongly suggest) all future teachers take more than one course preparing them to teach students with disabilities; however, programs are already overloaded with coursework, field placements, and other experiences, so adding new requirements is unlikely to become the norm (Blanton, McLeskey, & Hernandez-Taylor, 2014; Grossman & McDonald, 2008; Leko et al., 2015). This situation constitutes a wicked problem faced by most in the field of special education and teacher preparation. We know most students with disabilities are being taught at least some of the day by teachers not at all prepared to address their individualized needs (Brownell, Griffin, Leko, & Stephens, 2011; Pugach, Blanton, & Boveda, 2014), yet we as faculty in teacher preparation programs are somewhat hamstrung in terms of options for addressing this issue. New and creative thinking is needed.

Stakeholders in the field of teacher preparation can trace novice teachers’ instructional deficiencies across K-12 placements to knowledge and skills they failed to acquire and/or master during preparatory programs (Lignugaris/Kraft, Sindelar, McCray, & Kimerling, 2014). Of significant concern are methods and dosage of instruction used to provide preservice teachers with exposure to key content and practices needed to teach students with disabilities (Brownell, Sindelar, Kiely, & Danielson, 2010; Lignugaris/Kraft et al., 2014). As an example, although few would get into an airplane helmed by a pilot who learned to land during a semester primarily comprised of lectures and discussions, we as teacher educators routinely sit back and tacitly allow students with disabilities to be served by general education teachers who acquired knowledge and skills in this fashion. Teacher educators do this despite full awareness that this type of learning is unlikely to transfer to effective practice (Bransford & Schwartz, 1999).

There are calls from leaders in the field (e.g., Brownell et al., 2010; Leko et al., 2015) for preparation programs to move away from teaching about practice, and instead refocus on a series of carefully structured practical experiences where preservice teachers apply specific instructional practices in increasingly diverse and realistic scenarios. We agree this is essential, but simultaneously maintain preservice teachers must acquire knowledge before it can be reconstituted as pedagogical skill (Shulman, 1987). If teacher educators can streamline methods for providing preservice teachers with essential content knowledge, it may be possible to free up time, even within the confines of the one introductory course, to refocus attention on the learning and implementation of evidence-based practices.

The purpose of this article is to extend the conversation and empirical literature around the need to improve instructional options for teacher preparation coursework offered to general education candidates in the domain of special education. Although there are numerous possible pedagogical avenues to pursue in this charge, multimedia currently enjoys a prominent role in the field’s psyche (Sayeski, Hamilton-Jones, & Oh, 2015; Smith & Kennedy, 2014). In the following section, we present empirical evidence from a naturalistic study of a time-saving, multimedia-based instructional approach called Content Acquisition Podcasts for Teachers (CAP-T). Our thesis, grounded in empirical research, is CAP-Ts can provide general and special education preservice teachers with essential content knowledge contained within introductory coursework in an efficient and effective manner. If effective, instructors could confidently move toward Leko et al.’s (2015) call to make teacher preparation more focused on deliberate and scaffolded practice.

The Theory and Instruction of CAP-Ts

There is strong empirical evidence that CAP-Ts are an effective tool for supporting preservice teachers’ learning about various topics within the introductory course in special education (Dieker et al., 2014). CAP-Ts are not really podcasts in the way that many would think about them, but are relatively short (approximately 5-15 minutes), multimedia-based vignettes. The term podcast is used in this work because of many users’ familiarity with and acceptance of podcasting as a valued instructional tool in higher education (Evans, 2008). CAP-Ts can be created for teacher candidates and in-service teachers (Kennedy, Alves, & Rodgers, 2015), whereas CAP-Ss are used for students with and without disabilities (see Kennedy, Deshler, & Lloyd, 2015). A sample CAP-T can be seen at https://vimeo.com/143387419

CAP-Ts are not intended to fully replace lectures or assigned readings. Instead, they offer learners a supplement to existing instruction and a way to receive high-quality instruction beyond the boundaries of the traditional classroom. In practice, instructors assign CAP-Ts to accompany weekly readings and recommend students use the video either as an advance organizer prior to reading the textbook or as a review afterward.

Theory of CAP-Ts

CAP-Ts are set apart from generic audio-only, enhanced, or video podcasts, because they use Mayer’s cognitive theory of multimedia learning (2009) and accompanying 12 instructional design principles (2008) to guide the design and delivery of instruction. Mayer’s design principles provide a roadmap that helps instructors make design decisions that are considerate of how people learn. In the following section, we introduce seven critical principles and discuss how they influence CAP-Ts. The five not explicitly discussed (multimedia, modality, temporal contiguity, image, and voice principles) are automatically addressed within the CAP-T model.

First, the coherence principle (Mayer & Jackson, 2005) holds that a learner will retain more of her available cognition to attend to new information if instruction is constrained to only the most important content. All CAP-Ts only contain the most critical information relevant to the topic being taught. The signaling principle (Stull & Mayer, 2007) states that explicitly highlighting key information within instruction will result in better learning outcomes. Each CAP-T has explicit cues (text and audio) throughout that orient the viewer’s attention to critical points. The redundancy principle helps instructors recognize that multimedia-based instruction that asks a learner to use his or her visual and auditory inputs to acquire the same content (e.g., think of a PowerPoint slide with only words while the presenter also reads the slides but also elaborates into other content) is not a good match for maximizing limited cognitive resources. Instead, slides with a clear image and a short phrase of key content will result in better learning (Mayer & Johnson, 2008). Students in Kennedy’s courses are told that when text appears within a CAP-T, it is important, and likely the answer to a future test item. Otherwise, text is not used in the videos.

Other design ideas include Mayer’s spatial contiguity, pretraining, segmenting, and personalization principles. The spatial contiguity principle (Moreno & Mayer, 1999) states core content should be strategically placed on a screen to limit eye movement. All CAP-Ts have one core image, placed in the middle of the screen, and when text is used, it also appears right above or below the image. The pretraining principle (Mayer, Mathias, & Wetzell, 2002) reminds instructors to provide an advance organizer or define key terms/concepts that appear within instruction. CAP-Ts begin with a specific statement introducing the key content and defining new concepts. The segmenting principle (Mayer, Dow, & Mayer, 2003) holds that instruction broken into learner-sized chunks will help scaffold learning success. CAP-Ts contain embedded pause points where the instructors can insert questions. Finally, the personalization principle (Moreno & Mayer, 2004) states that multimedia recorded by students’ instructor for a specific purpose in a conversational voice (not generic or “off the rack” technology) results in augmented learning. Kennedy and his team created CAP-Ts for the introductory audience in his special education courses.

The use of Mayer’s model to create CAP-Ts sets this intervention apart from other multimedia-based instructional tools (Kennedy, Alves, & Rodgers, 2015). For example, generic, audio-only podcasts do not have any specific learning theory that underwrites their capacity for learning (Heilesen, 2010). Kennedy, Hart, and Kellems (2011) demonstrated that teacher candidates who learn using CAP-Ts learn and retain significantly more content than peers who listened to an audio-only podcast. Other well-known products, such as the IRIS modules (iris.peabody.vanderbilt.edu), are largely driven by text, and also have a broader purpose than CAP-Ts. Sayeski, Hamilton-Jones, et al. (2015) demonstrated teacher candidates made significant learning gains across various content areas when learning using this resource. A worthwhile research question would be a comparison in teacher candidate learning when using CAP-Ts compared with using the IRIS modules. It would also make sense to embed CAP-Ts within, or use alongside the IRIS modules to further bolster the utility of that outstanding resource.

Research Supporting CAP-Ts

To date, there are 16 published studies demonstrating CAP-Ts’ positive impact on learning of teacher candidates. The first group of studies (n = 11) were simple comparisons of learning when teacher candidates watched a CAP-T or read a practitioner-oriented chapter or article on the same topic (Carlisle, Thomas, & McCathren, 2016; Driver, Pullen, Kennedy, Williams, & Ely, 2014; Ely, Pullen, Kennedy, Hirsch, & Williams, 2014; Hart & More, 2013; Kennedy, Aronin, et al., 2014; Kennedy, Driver, Pullen, Ely, & Cole, 2013; Kennedy, Ely, et al., 2012; Kennedy et al., 2011; Kennedy & Thomas, 2012; Kennedy, Thomas, Aronin, Newton, & Lloyd, 2014; Sayeski et al., 2015). Following that group of studies, researchers began comparing learning using CAP-Ts to live lectures (Hirsch, Kennedy, Haines, Thomas, & Alves, 2015), and including measures that provide additional insights into why this tool is powerful, such as motivation (Kennedy, Wagner, et al., 2016), and perceived cognitive load (Kennedy, Hirsch, et al., 2016). Finally, researchers have used CAP-Ts in conjunction with modeling videos (CAP-TV) to support teacher candidates’ learning and implementation of evidence-based practices (Ely, Kennedy, Pullen, Williams, & Hirsch, 2014; Ely, Pullen, Kennedy, & Williams, 2015).

Research supporting CAP-Ts is primarily experimental (n = 16 published studies), but is also somewhat limited with respect to generalizability. To illustrate, the CAP-T line of research has followed standards for rigor of quantitative research in special education (Odom et al., 2005), randomly assigning participants, and exerting careful experimental control over the instructional conditions for the independent variable and comparisons. During all but one of the published studies of CAP-T effectiveness, participants were randomly assigned to either watch a CAP-T, read a practitioner-oriented version of the same content, or receive a lecture in a laboratory setting. Participants in the CAP-T groups spread out within a computer lab or classroom, watched the video(s) at individual laptop or computer terminals, and wore headphones. Although researchers in each experiment instructed students to pause videos and take notes as usual, it is doubtful that this distraction-free setting is congruent to how most university students complete assigned CAP-Ts in preparation for class. Instead, when used as a weekly learning tool, instructors recommend students watch videos at their convenience and use methods that work well for them.

The dependent measure for the previous CAP-T studies is a test of knowledge given the same day as the CAP-T viewing, or, in newer studies (e.g., Hirsch et al., 2015), 1 week later. Although CAP-T studies separate independent and dependent variables with at least an hour-long distractor (e.g., lecture on a different topic), it remains a test of short-term learning. Although CAP-Ts have consistently demonstrated positive and significant learning gains, an unanswered research question is the extent to which naturalistic viewings of CAP-Ts over different periods of time (e.g., half semester, full semester) affect performance on items presented during a course’s midterm and final exam. Such a test would begin to reveal the durability of learning. This is the impetus for the present study.

The Present Study

In preliminary studies of a novel intervention, researcher control is desirable to provide proof of concept and evidence for efficacy (Cook & Campbell, 1979). As efficacy evidence accumulates, there is a corresponding obligation on the researcher to reduce experimental control and test the intervention in contexts that more closely resemble conditions that would arise given widespread adoption. The present study builds on previous research on the efficacy of CAP-Ts in important ways.

First, researchers asked students to record and report how many times they watched each video in preparation for class and leading up to exams. These factors permit an investigation of the relation of CAP-T views to midterm and end of course knowledge outcomes, and preliminary data on the ways in which students utilize CAP-Ts given a naturalistic design and according to individual learning needs. Also, learning was checked at two distal time points compared with the short-term learning typically measured in previous CAP-T studies. Thus, two research questions guided the study:

Research Question 1: To what extent does the number of self-reported CAP-T views predict performance on midterm and final assessments?

Research Question 2: To what extent does CAP-T utilization differ by topic?

Method

Participants

Participants in this study were 162 students enrolled in one section of a course titled “The Exceptional Learner,” an introductory course in special education at a large, prestigious public research intensive university on America’s east coast. Although exempted by the institution’s human participants review board, all students were given opportunity without penalty to refuse personal contribution (demographic data, number of CAP-T views, and midterm and final exam scores) to the data set. The study was introduced to students via a faculty member not affiliated with the course or research. Researchers did not have access to the participant’s reported CAP-T views until the conclusion of the semester. One hundred percent participation was achieved, with no students opting out and no attrition. For the purpose of analyses, names of individual students and their reported CAP-T views were linked in a database with their corresponding responses from the midterm and final exams, checked for accuracy by two researchers, and the data were then de-identified using numbers as codes. No breaches of confidential student data (e.g., grades) were reported.

Participants included undergraduate (25.3%) and graduate students (74.7%). There are no study-relevant differences in the assignments or structure of the course for undergraduate or graduate students. The course is required for teacher licensure in the elementary (43.5% of the participants) and secondary (35.1%) education programs, and is also taken by students in the communication disorders program (11.5%), other majors (e.g., kinesiology) within the school of education (3.4%), and non-education students seeking to learn more about the topic (6.5%). Unlike previous CAP studies, the majority of the participants were graduate students. Like many courses within teacher preparation courses at large, flagship universities in the United States, 13.5% of the participants in this course (and study) were male, and 86.5% were female. Similarly, 78.3% of the participants were Caucasian, 12.2% were Black, and 5.2% were Hispanic; the remaining students, 4.3%, reported other races.

Procedures

Students were introduced to CAP-Ts during the first session of the course as part of the syllabus review. CAP-Ts for each week’s session were included as links embedded within the syllabus. The instructor of the course explained the purpose and structure of CAP-Ts, and recommended strategies for their use (e.g., watching the CAP-T before assigned readings as a Pre-CAP, or after reading as a Re-CAP). CAP-Ts were assigned as part of the required preparation for class along with reading the course textbook and other activities (e.g., reviewing websites). Research findings from previous studies (e.g., Kennedy, Hirsch, et al., 2016) were presented with intent of convincing students that CAP-Ts are an effective and efficient tool for learning, and that other students had benefited from accessing CAP-Ts during the course.

Faculty members and doctoral students at the researcher’s university from 2011 to 2015 created the CAP-Ts used in this study. All CAP-Ts (from this study and beyond) are available for free at the Introduction to Special Education site on Vimeo at www.SPEDIntro.com. The technical process for creating a CAP-T is described in detail elsewhere (see Kennedy, Kellems, Thomas, & Newton, 2015), and a two-part CAP-T on how to create this tool is included on the aforementioned Vimeo site. A few words about the processes relevant to this study are warranted here. To keep the length of CAP-Ts to a manageable time and adhere to Mayer’s (2008) coherence principle, we often split videos into Parts 1 and 2. To illustrate, for most of the CAP-Ts providing content on specific disability categories (e.g., specific learning disabilities), Part 1 covers the characteristics, causes, prevalence, and related information. Part 2 contains evidence-based practices and prominent accommodations and modifications. In some cases, specific practices such as video modeling and curriculum-based measurement are captured as separate individual CAP-T to provide learners with more detail on each topic.

In addition, for this study, the instructional team used CAP-T to “flip the classroom” (Bull, Ferster, & Kjellstrom, 2012), meaning students were assigned to watch CAP-T outside class, much like an assigned reading, to develop background knowledge on important topics so that face-to-face instructional time would not be dominated and consumed providing lectures covering the same information contained within the required readings and CAP-Ts. Rather, class time could be spent working on case studies, listening to guest speakers (e.g., practitioners and other experts), engaging students in elaborated discussions, watching and participating in modeling of new practices, and watching clips of videos relevant to the topic. The instructor only provided in-class content from videos when students asked questions or other data such as questions, missed quiz, or test items indicated a review was necessary.

A researcher unaffiliated with the research team introduced the study. They instructed students to keep careful track of how many times they watched each CAP-T during the semester. Although instructors from this research team now use www.EdPuzzle.com as a way to capture individual student viewings, at the time of this study the best option we had available was the self-report approach. An excel spreadsheet was provided to students as a suggested way to record data. Any time the student watched three quarters or more of the CAP-T it counted as a viewing. Students were informed that a significant group of questions on the midterm and final exams would come exclusively from the CAP-Ts, and thus, it would behoove them to access videos on a regular basis for preliminary exposure to course content, and during test review processes. During the midterm exam, in class, students were instructed to use their own device and login to report the number of viewings per CAP-T into an online database, which was not accessible to the instructional team. This database was kept separate from the exam itself, which was accessed by its own individual link and exam data were accessible to the instructor team for course-grading purposes. The process was repeated during the final exam, with students now asked to report only the number of CAP-T views since the midterm.

Measures

The measures in this study are questions written by the researchers (who are also the instructor and teaching assistants of the course) for the midterm and final exams. This same group created all of the CAP-Ts for the study and attended all class sessions, so the content is well known to the team. Researchers broke into small groups to write questions based on CAP-Ts and brought them back to the full team. Each proposed question was examined to ensure it is fully answerable using videos, and for adherence to overall best practices for item construction and psychometric testing (e.g., American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999). Total questions (n = 110) are a blend of multiple choice (n = 88), true and false (n = 8), and open-ended (n = 14) items. There were 47 questions on the midterm, which covered content from the first half of the semester, and 63 questions on the final exam.

Both the midterm and final exams contained items specifically drawn from individual CAP-Ts as well as items drawn from other course resources such as lecture, case study, readings, websites, and the course textbook. The final exam was cumulative. Of the 47 items on the midterm, nine were not from CAP-T content, and on the final, 14 of the items were not from CAP-Ts. Multiple-choice and true/false items were scored automatically (1 = correct, 0 = incorrect) by our university’s course management system. Open-ended items were scored by the instructor and a teaching assistant using a calibrated rubric. For the purpose of the study, reliability checks were conducted on 20% of students’ exams to evaluate interscorer reliability. Scores were initially calibrated at 96%, and discrepancies were resolved through discussion.

Psychometric properties of instruments

Researchers completed several statistical tests to establish the psychometric properties of the instruments used in this study. Results demonstrate the midterm and final demonstrated adequate psychometric properties and demonstrated few differences between items drawn from CAP and non-CAP content. To illustrate, the mean facility index across all items was 78% (SD = 21.2), which suggests that the items are fairly easy; however, they are sufficiently difficult to introduce separation among examinees. The facility index did not differ between CAP and non-CAP items (77% [SD = 21.2] and 79% [20.2], respectively), suggesting the items functioned similarly. This is further bolstered by data from the subtest-level discrimination index, demonstrating the CAP items (27% [SD = 19.9]) did not function differently from the non-CAP items (25% [SD = 22.7]). Results demonstrate there was modest skewness for the full instrument (−.70) and slight excess kurtosis (.68). Cronbach’s alpha for the full instrument was .87 at midterm and .89 on the final.

The effective weight represents an index of the contributions of specific items to the total score. Across all items, the mean effective weight was comparable with the intended weight. Ten items demonstrated negative co-variances with the total score, suggesting an unexpected relation with the outcome of interest. Thus, analyses were conducted with and without these items to determine whether these items were exerting undue influence on the total exam score. Results across the analyses were similar. In addition, the effective weights of CAP items and non-CAP items were evaluated. Both sets of items demonstrated mean effective weights, which were comparable with their intended weight (CAP items: effective weight = 1.16, intended weight = 1.16; non-CAP items: effective weight = 8.33, intended weight = 8.33).

Design

Students’ self-reported CAP-T views were used in a regression model to predict performance on specific questions contained within the midterm and final exams. In addition, researchers explored the relationships between individual CAP-T topics, views, and performance on the matching items on midterm and final exams.

Results

Primary Analysis: The Relation of CAP Views to Knowledge Outcomes

The primary research question addressed whether the total number of CAP-T views predicted student performance on specific items contained within the midterm and final exam. This research question was answered with two linear regression models, in which exam performance was regressed on the total number of CAP-T views completed prior to the midterm and prior to the final. To facilitate interpretation, both independent variables (midterm and final CAP-T views) were centered at the sample mean. Thus, the intercept represents the sample mean and the beta coefficient represents the adjustment in predicted exam score for each additional CAP-T view above or below the sample mean number of views.

Results indicate that the number of CAP-T views completed at midterm was a significant predictor of participants’ midterm exam score, F(1, 160) = 40.26, p < .001, R² = .20. For each CAP-T view above the sample mean number of views, participants predicted performance on the midterm exam increased by .31 points. Conversely, for each CAP-T view less than the sample mean number of views, participants’ predicted performance on the midterm exam decreased by .31 points. A student who viewed the mean number of CAP-T prior to the midterm (M = 25.53, SD = 9.00) would be predicted to receive a score equal to the sample mean midterm exam performance (M = 70.64, SD = 6.21).

Results indicate that the number of CAP-T views completed prior to the final exam was a significant predictor of participants’ final exam score, F(1, 160) = 22.87, p < .001, R² = .13. For each CAP view above the sample mean number of views, participants’ predicted performance on the final exam increased by .13 points. Conversely, for each CAP-T view less than the sample mean number of views, participants’ predicted performance on the final exam decreased by .13 points. A student who viewed the mean number of CAP-T prior to the final (M = 33.45, SD = 13.06) would be predicted to receive a score equal to the sample mean final exam performance (M = 58.94, SD = 6.21).

Exploratory Analysis: Differences in CAP-T Utilization and Outcomes by Topic, Difficulty, or Time

The second research question addressed whether, in naturalistic settings, student utilization of specific CAP-Ts differed by topic. This exploratory question was addressed through an analysis of descriptive statistics, including rank orders by topic to evaluate the extent to which CAP-Ts are utilized differently and whether future study is warranted.

Table 1 provides descriptive statistics on the number of CAP-T views by topic for the midterm and final exam. The mean number of CAP views at midterm ranged from 1.7 (SD = 0.9; Related Services) to 3.9 (SD = 1.9; Individualized Education Programs [IEPs]). The median number of views at midterm was 2.15. For the final exam, the mean number of views ranged from 1.6 (SD = 1.2; Co-Teaching) to 4.4 (SD = 1.4; Characteristics of Students with Autism Spectrum Disorders). On the final exam, the median number of views was 2.9.

Table 1.

Descriptive Statistics for CAP Views and Exam Performance by Topic and Sorted by Percent Correct for Discrete Topics.

Topic			Exam performance
	CAP views		Raw score		% correct
	M (SD)	Range	M (SD)	Range	M (SD)
Midterm exam
Working with families	1.6 (0.9)	0-5	6.7 (0.9)	1-7	95.5 (13.2)
Designing accommodations	1.9 (0.9)	0-5	2.8 (0.4)	0-3	94.8 (14.7)
Related services	1.7 (0.9)	0-5	5.4 (1.1)	0-6	90.3 (18.4)
Response to intervention	3.1 (1.5)	0-7	9.7 (1.3)	4-11	87.8 (11.6)
Universal design for learning	1.8 (1.0)	0-5	6.1 (1.1)	1-7	87.7 (15.1)
Individualized Education Programs	3.9 (1.9)	0-14	13.8 (1.6)	8-16	86.1 (10.0)
Learning disabilities	3.8 (1.5)	0-10	6.4 (1.2)	2-8	79.9 (14.8)
Attention deficit/hyperactivity disorder	2.4 (1.1)	0-7	3.9 (0.7)	2-5	78.8 (14.9)
IDEA 2004	2.3 (1.6)	0-10	5.3 (1.3)	1-7	75.7 (19.2)
Transition	1.9 (0.8)	0-5	2.9 (0.7)	1-4	73.0 (16.5)
No CAP			7.6 (1.4)	3-9	84.2 (15.6)
Midterm total	2.3 (1.2)	8-61	85.1% (1.1)	42-80 (/83)	84.9 (14.9)
Final exam
Transition	3.9 (1.1)	0-6	5.6 (0.6)	3-6	93.7 (10.0)
Learning disabilities	4.3 (2.4)	0-12	10.3 (1.1)	5-11	93.2 (10.0)
Related services	1.8 (1.1)	0-5	2.8 (0.5)	0-3	93.2 (16.3)
IDEA 2004	2.4 (1.2)	1-6	3.4 (0.7)	1-4	84.9 (18.6)
Autism spectrum disorders	4.4 (3.1)	0-20	5.1 (1.3)	1-7	84.7 (21.3)
Emotional/behavioral disorders	4.2 (2.9)	0-18	4.8 (1.2)	1-6	80.1 (19.0)
Individualized Education Programs	3.1 (2.5)	0-11	3.1 (0.6)	2-5	77.2 (15.4)
Co-teaching	1.6 (1.2)	0-7	3.0 (0.7)	1-4	75.2 (17.7)
Response to intervention	2.9 (1.9)	0-9	2.2 (0.8)	0-3	72.8 (26.8)
Intellectual disabilities	2.8 (1.4)	2-8	5.8 (1.4)	2-8	72.7 (17.8)
Working with families	1.8 (1.1)	0-6	1.2 (0.7)	0-2	57.7 (37.2)
No CAP			11.8 (1.5)	7-14	90.4 (11.6)
Final exam total	3.0 (1.8)	6-78	81.0% (1.0)	35-70 (/73)	88.7 (20.2)

Note. CAP = Content Acquisition Podcasts; IDEA = Individuals With Disabilities Education Act.

In addition, Table 1 notes the percent of items correct by topic on the midterm and final exam. Within the table, topics are sorted from the highest percent correct to lowest percent correct for the midterm/final and topic to facilitate the identification of patterns between CAP-T views and percent correct by topic. The mean percent correct by topic for the midterm ranged from 73.0 to 95.5. The median percent correct was 86.0. For the final exam, the percent correct ranged from 57.7 to 93.7 and median percent correct was 80.1. Ceiling effects were minimal as few students scored perfectly on sets of questions.

To evaluate whether there may be patterns that would suggest a relation between different utilization of specific CAPs and the percent correct on that topic, we evaluated a list by the number of views and percent correct for specific topics on the midterm and final. The analyses provide interesting insights that suggest in some instances, CAP-Ts viewed fewer times on the midterm resulted in higher scores for matching items. In fact, four of the five CAP-T topics where students scored highest on the midterm had the lowest average views. These topics included, working with families, introduction to related service providers, universal design for learning, and using accommodations and modifications (the fifth where views were higher was the CAP-T on response to intervention). On the final exam, four of the six top scores for items came from CAP-Ts viewed more often (e.g., transition, characteristics of students with learning disabilities, autism spectrum disorders, and emotional/behavioral disorders). We hypothesize reasons for these disparate findings in the discussion.

Discussion

The call for sustained programs of research that explore multiple facets of an intervention or instructional approach is often repeated in the field of teacher preparation (see Lignugaris/Kraft et al., 2014). Specifically, the field of teacher preparation is looking for instructional methods and tools that can help instructors provide future educators with essential knowledge in efficient, but powerful, ways so that the more important issue of practice implementation can be addressed (Leko et al., 2015). This need is not fully addressed by the body of literature amassed over the years—indeed, in many ways, we have more questions than answers (Sindelar, Brownell, & Billingsley, 2010). Significant room remains for scholars to innovate and rigorously test their ideas in light of specific learning needs within course and fieldwork. The program of research described in this article is part of an attempt to address the need for high-quality research in this field (Dieker et al., 2014).

As noted, researchers from multiple universities have published 16 articles that report empirical results demonstrating capacity of CAP-Ts to improve teacher candidate learning across a range of content. Before this project, the effectiveness of this tool was limited to laboratory-style experiments. Thus, demonstrating the effectiveness of CAP-T in a naturalistic setting, and over long periods of time, is an important advance of the CAP-T line of research. These results, although preliminary, provide instructors a measure of confidence that this tool can be used throughout an introductory course to help ensure teacher candidates learn essential content.

It is acknowledged that teachers need to demonstrate learning in more sophisticated ways than answering multiple-choice items on a midterm and final exam. That said, the predecessor to implementation of evidence-based practices is the type of learning delivered using CAP-Ts and measured within this study during an introductory course with 162 students. As teacher candidates progress beyond introductory coursework, they become engaged in practicum experiences and methods courses where they are required to apply their learning. A logical next step after this study would be to follow participants into their methods courses to evaluate the extent to which they are implementing evidence-based practices with fidelity. It is also important to consider the use of measures beyond midterm and final exams with multiple-choice and open-ended questions, even in introductory coursework to evaluate student learning following use of CAP-Ts. Research conducted by Ely and colleagues (2015) is an example of the type of applied learning teacher candidates are expected to demonstrate following use of CAP-Ts and CAP-TVs. Ely and colleagues were answering a different research question than addressed here, but it does have a valuable lesson for future planning and research.

Exploring Results and Implications for Practice

The main finding of this study, that more CAP-T views predict higher performance on exam questions, provides corroboration for the previously published findings using this tool in laboratory settings. This finding also expands on previous research given the longer space of time between CAP-T views and the evaluation on the midterm and final exams. Due to the nature of our naturalistic research question, we cannot be precise regarding the length of time between CAP-T views and the two data collection points; however, it is certainly longer than the previous studies. It is logical to expect (and was our hypothesis) that more CAP-T views would result in a boost in performance for teacher candidates on assessments that counted for a substantial portion of their course grade. Prior to this study, there was not empirical evidence to support that assertion. Therefore, the findings of this study are important, and can be used to help future teacher candidates understand the value in using this learning tool as part of their learning during introductory coursework.

For the second exploratory research question, results raise interesting questions. Given the aggregate effects, it was expected that on topics for which students viewed more CAPs, they would perform better. The pattern of results for some topics suggests the opposite may be true. This is possibly explained by the content and questions of some CAP-Ts being easier than others. In other words, it is possible that CAP-Ts with easier content led students to view the video less times because it was mastered quickly. In addition, variability in the number of views can be attributed at least in part to the emphasis placed on each topic by the instructor during class (e.g., more time and emphasis were placed on IEPs than related service providers). It logically follows that questions on those topics were also easier, and thus correctly answered at a higher rate than items from more difficult content.

Data presented in the “Method” section on the technical qualities of the instrument offer some clarity to this point. For example, a modest amount of skewness and kurtosis was observed for the full instrument, and the facility index statistics demonstrate most students found the overall exams to be relatively easy. When looking at individual question level data for facility index for midterm questions stemming from the CAP-Ts on working with families, introduction to related service providers, universal design for learning, and using accommodations and modifications, results support the hypothesis that these items were easier than the exam on the whole. Although we are still unable to be conclusive that students found this content easier than other topics and adjusted their CAP-T viewing patterns accordingly, this combination of evidence is persuasive and should be explored further in follow-up studies.

Results on the final exam more closely fit our expectations given the overall result. One reason why could be that the lion’s share of the course’s content fell after the midterm this particular semester due to the University’s unbalanced scheduling and a snow day that canceled a meeting before. In addition, the meatier content, such as the characteristics and evidence-based practices for teaching students from the various disability categories all fall after the midterm. Thus, in the view of the course instructors, the more important and challenging content led to more CAP-T views and solid performance on the final exam. It is also likely that as the semester progressed, the students in the course simply became better consumers of CAP-Ts. In other words, as cumulative number of views went up, so did comfort with and ability to learn from CAP-Ts. Although we do not have specific data to back this claim, it is a logical hypothesis to explore in future research. Data from the facility index demonstrate the final exam was more difficult than the midterm, and the distribution of scores was less skewed. This provides evidence supporting the main finding of the study, that more CAP-T views led to stronger performance on the cumulative final exam for the course.

In summary, instructors of introductory courses in teacher preparation courses can rely on CAP-Ts as a tool that can arm future teachers with the content they need to be successful. Although the type of knowledge learned from CAP-Ts does not automatically convert into improved practice, it is a solid grounding for advanced implementation and sustainability of evidence-based practices (Shulman, 1987).

Limitations

There are several important limitations to this study. First, students self-reported their number of CAP-T views. Although they were provided with an Excel spreadsheet and periodic reminders to track views, provided data are ultimately subject to errors. Researchers such as Salvucci, Walter, Conley, Fink, and Saba (1997) and Cook and Campbell (1979) offered caution to those collecting and interpreting self-reported data as they can be over-reported or underreported and subject to bias. Others such as Chan (2009) had more faith that self-reported data can be credible and useful. Chan (2009) explained that participants in research studies who were explicitly cued to remember specific information and were provided with a specific method to do so can be counted on to provide reliable recollections. Despite erring on the side of caution and acknowledging this is a significant limitation, we are cautiously optimistic that the data are reasonably accurate as there would be little or no reason for students to provide false data. It cannot be ruled out that students over-reported CAP-T views in an attempt to please the instructor/researchers and thus help confirm the unstated, but obvious research hypothesis that more views may result in higher achievement.

Relatedly, we have no way of knowing when, or the extent to which participants were watching or listening to CAP-Ts and what competing demands for their attention were present. In previous studies, researchers carefully controlled CAP-T access using headphones and individual laptop terminals. To a large extent, distractions were eliminated and attention was focused on the content of the CAP-Ts. Although the purpose of this study was to explore the impact of CAP-Ts when those experimental controls were not in place, it therefore doubles as the study’s purpose and one of the important limitations.

Next, all dependent measures were created by the researchers/instructors. Although items are used every semester on midterm and final exams (and been revised to eliminate confusing wording and items that are too easy), data presented herein demonstrate that some are easier and harder than others. The variability in ease/difficulty in items is a major contributor to the exploratory finding that some CAP-Ts with less views yielded higher scores on the exam questions. Findings would be strengthened by using items that had standardized psychometric properties. Even with items subjected to additional pilot and field testing, some content in the introductory course in special education is simply easier for students to learn than others (see above). Previous coursework, common sense, and life experiences are all contributors to this variability. Relatedly, important data that contributed to the students’ exam scores such as instructor quality and individual study habits and achievement levels were not measured or included in analyses. Although it would be difficult, if not impossible, to objectively measure instructor quality and include it in a statistical model, this study did only have the one instructor. Future studies might use student Grade Point Average(GPA) or Scholastic Achieveme Test (SAT)/American College Test(AP) scores as covariates to help explain findings.

Researchers did not collect data from participants that would explain why they watched some CAP-Ts more or less than others. Although it can be implied fewer views stem from easier content and more views relate to more difficult content, we cannot be certain. Students in a subsequent section of the course with highly similar demographics and characteristics were asked via survey why they watch CAP-Ts more or less than others, and almost all (n = 134) strongly agreed (m = 4.87/5, SD = .35) that “I watch CAP-Ts more often when content is harder to learn.” In addition, students were not queried with respect to their opinions of the CAP-Ts. Although useful, this information is not of paramount concern given the headline finding that more CAP-T views result in higher scores on the midterm and final exams.

Conclusion

Given the strong empirical record of CAP-Ts to improve teacher candidate learning of important topics in introductory coursework in special education, we are encouraged by the results of this naturalistic study. As more researchers and instructors use or create their own CAP-Ts, we look forward to more replications of highly experimental, naturalistic, and qualitative studies that help provide additional information on the utility of this tool.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Michael J. Kennedy, Ph.D. is an Assistant Professor in the Curry School of Education at the University of Virginia. He is an interventionist dedicated to the experimental testing of multimedia’s impact on teacher and student learning.

Kat D. Alves, Ph.D. is an Assistant Professor at Longwood University. Her primary program of research focuses on training preservice and inservice teachers to use evidence-based practices for vocabulary and reading comprehension in order to improve literacy outcomes for students.

Jeremy Miciak, Ph.D. is an assistant research professor in the Department of Psychology and the Texas Institute for Measurement, Evaluation, and Statistics (TIMES) at the University of Houston. His research interests include the identification and treatment of learning disabilities and learning difficulties, particularly among students from diverse cultural and linguistic backgrounds.

John Romig, M.Ed. is a Doctoral Student at the University of Virginia. His research interests include teacher preparation, writing instruction, and curriculum-based measurement.

Hannah Morris Mathews, M.Ed. is a Doctoral Student at the University of Virginia. Her research interests include teacher preparation, induction, and mentoring of novice special education teachers.

Cathy Newman Thomas, Ph.D. is an Associate Professor of Special Education at the University of Missouri-Columbia. Her research interests focus on intervention studies of technology in teacher education and access to the general curriculum for adolescents with and at risk for disabilities.

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Blanton

L. P.

McLeskey

Hernandez-Taylor

(2014). Examining indicators of teacher education program quality: Intersections between general and special education. In Sindelar

P. T.

McCray

E. D.

Brownell

M. T.

Lignugaris/Kraft

(Eds.), Handbook of research on special education teacher preparation (pp. 129-142). New York, NY: Routledge.

Bransford

J. D.

Schwartz

D. L.

(1999). Rethinking transfer: A simple proposal with multiple implications. Review of Research in Education, 24, 61-100.

Brownell

M. T.

Bishop

A. G.

Klingner

J. K.

Penfield

R. D.

Dimino

Haager

. . . Sindelar

P. T.

(2009). The role of domain expertise in beginning special education teacher quality. Exceptional Children, 75, 391-411.

Brownell

M. T.

Griffin

Leko

M. M.

Stephens

(2011). Improving collaborative teacher education research: Creating tighter linkages. Teacher Education and Special Education, 34, 235-249. doi:10.1177/0888406411404570

Brownell

M. T.

Sindelar

P. T.

Kiely

M. T.

Danielson

L. C.

(2010). Special education teacher quality and preparation: Exposing foundations, constructing a new model. Exceptional Children, 76, 357-377.

Bull

Ferster

Kjellstrom

(2012). Inventing the flipped classroom. Learning & Leading With Technology, 40(1), 10-11.

Carlisle

A. A.

Thomas

C. N.

McCathren

R. B.

(2016). The effectiveness of using a content acquisition podcast to teach phonological awareness, phonemic awareness, and phonics to preservice special education teachers. Journal of Special Education Technology, 31, 1-12. doi:10.1177/0162643416651723

Chan

(2009). So why ask me? Are self report data really that bad? In Lance

C. E.

Vandenberg

R. J.

(Eds.), Statistical and methodological myths and urban legends: Doctrine, verity and fable in organizational and social sciences (pp. 309-335). New York, NY: Routledge.

10.

Cook

T. D.

Campbell

D. T.

(1979). Quasi-experimentation: Design and analysis issues. Boston, MA: Houghton Mifflin.

11.

Dieker

L. A.

Kennedy

M. J.

Smith

Vasquez

III Rock

Thomas

C. N.

(2014). Use of technology in the preparation of pre-service teachers (Document No. IC-11). University of Florida, Collaboration for Effective Educator, Development, Accountability, and Reform Center. Retrieved from http://ceedar.education.ufl.edu/tools/innovation-configurations/

12.

Driver

M. K.

Pullen

P. C.

Kennedy

M. J.

Williams

M. C.

Ely

(2014). Using instructional technology to improve preservice teachers’ knowledge of phonological awareness. Teacher Education and Special Education, 37, 309-329. doi:10.1177/0888406414537902

13.

Ely

Kennedy

M. J.

Pullen

Williams

M. C.

Hirsch

S. E.

(2014). Improving instruction of future teachers: A multimedia approach that supports implementation of evidence-based vocabulary practices. Teaching and Teacher Education, 44, 35-43. doi:10.1016/j.tate.2014.07.012

14.

Ely

Pullen

Kennedy

M. J.

Hirsch

S. E.

Williams

M. C.

(2014). Use of instructional technology to improve preservice teacher knowledge of vocabulary instruction. Computers & Education, 75, 44-52. doi:10.1016/j.compedu.2014.01.013

15.

Ely

Pullen

Kennedy

M. J.

Williams

M. C.

(2015). Using multimedia to improve evidence-based vocabulary instruction in elementary classrooms. Journal of Special Education Technology, 30, 59-72.

16.

Evans

(2008). The effectiveness of m-learning in the form of podcast revision lectures in higher education. Computers & Education, 50, 491-498. doi:10.1016/j.compedu.2007.09.016

17.

Grossman

McDonald

(2008). Back to the future: Directions for research in teaching and teacher education. American Educational Research Journal, 45, 184-205.

18.

Hart

J. E.

C. M.

(2013). Investigating the impact of technology on pre-service teacher knowledge of autism spectrum disorder. Education and Training in Autism and Developmental Disabilities, 48, 504-513.

19.

Heilesen

S. B.

(2010). What is the academic efficacy of podcasting? Computers & Education, 55, 1063-1068. doi: 10.1016/j.compedu.2010.05.002

20.

Hirsch

S. E.

Kennedy

M. J.

Haines

Thomas

C. N.

Alves

K. D.

(2015). Improving teacher candidate knowledge and application of functional behavioral assessments using multimedia. Behavioral Disorders, 41, 38-50. doi:10.17988/0198-7429-41.1.38

21.

Kennedy

M. J.

Alves

K. D.

Rodgers

W. J.

(2015). Innovations in the delivery of content knowledge in special education teacher preparation. Intervention in School and Clinic, 51, 73-81. doi: 10.1177/1053451215579268

22.

Kennedy

M. J.

Deshler

D. D.

Lloyd

J. W.

(2015). Effects of multimedia vocabulary instruction on adolescents with learning disabilities. Journal of Learning Disabilities, 48, 22-38. doi:10.1177/0022219413487406

23.

Kennedy

M. J.

Driver

M. K.

Pullen

P. C.

Ely

Cole

M. T.

(2013). Improving teacher candidates’ knowledge of phonological awareness: A multimedia approach. Computers & Education, 64, 42-51. doi:10.1016/j.compedu.2013.01.010

24.

Kennedy

M. J.

Ely

Thomas

C. N.

Pullen

P. C.

Newton

J. R.

Ashworth

. . . Lovelace

S. P.

(2012). Using multimedia tools to support teacher candidates’ learning. Teacher Education and Special Education, 35, 243-257. doi:10.1177/0888406412451158

25.

Kennedy

M. J.

Hart

J. E.

Kellems

R. O.

(2011). Using enhanced podcasts to augment limited instructional time in teacher preparation. Teacher Education and Special Education, 34, 87-105. doi:10.1177/0888406410376203

26.

Kennedy

M. J.

Hirsch

S. E.

Dillon

S. E.

Rabideau

Alves

K. D.

Driver

M. K.

(2016). Using multimedia to increase university students’ knowledge and to reduce perceived cognitive load. Teaching of Psychology, 43, 153-158. doi: 10.1177/0098628316636295

27.

Kennedy

M. J.

Kellems

R. O.

Thomas

C. N.

Newton

J. R.

(2015). Using content acquisition podcasts to deliver core content to preservice teacher candidates. Intervention in School and Clinic, 50, 163-168. doi:10.1177/1053451214542046

28.

Kennedy

M. J.

Thomas

C. N.

(2012). Effects of content acquisition podcasts to develop preservice teachers’ knowledge of positive behavioral interventions and supports. Exceptionality, 20, 1-19. doi:10.1080/09362835.2011.611088

29.

Kennedy

M. J.

Aronin

O’Neal

Newton

J. R.

Thomas

C. N.

(2014). Creating multimedia-based vignettes with embedded evidence-based practices: A tool for supporting struggling learners. Journal of Special Education Technology, 29(4), 15-30.

30.

Kennedy

M. J.

Thomas

C. S.

Aronin

Newton

J. R.

Lloyd

J. W.

(2014). Improving teacher candidate knowledge using content acquisition podcasts. Computers & Education, 70, 116-127.

31.

Kennedy

M. J.

Wagner

Stegall

Lembke

Miciak

Alves

K. D.

. . . Hirsch

S. E.

(2016). Using content acquisition podcasts to improve teacher candidate knowledge and application of curriculum-based measurement. Exceptional Children, 82, 303-320. doi:10.1177/0014402915615885

32.

Leko

M. M.

Brownell

M. T.

Sindelar

P. T.

Kiely

M. T.

(2015). Envisioning the future of special education personnel preparation in a standards-based era. Exceptional Children, 82, 25-43. doi:10.1177/0014402915598782

33.

Lignugaris/Kraft

Harris

(2014). Teacher preparation: Principles of effective pedagogy. In Sindelar

P. T.

McCray

E. D.

Brownell

M. T.

Lignugaris/Kraft

(Eds.), Handbook of research on special education teacher preparation (pp. 233-254). New York, NY: Routledge.

34.

Lignugaris/Kraft

Sindelar

P. T.

McCray

E. D.

Kimerling

(2014). The “wicked question” of teacher education effects and what to do about it. In Sindelar

P. T.

McCray

E. D.

Brownell

M. T.

Lignugaris/Kraft

(Eds.), Handbook of research on special education teacher preparation (pp. 461-471). New York, NY: Routledge.

35.

Maheady

Smith

Jabot

(2014). Field experiences and instructional pedagogies in teacher education: Don’t know, and must learn soon. In Sindelar

P. T.

McCray

E. D.

Brownell

M. T.

Lignugaris/Kraft

(Eds.), Handbook of research on special education teacher preparation (pp. 161-177). New York, NY: Routledge.

36.

Mayer

R. E.

(2008). Applying the science of learning: Evidence-based principles for the design of multimedia instruction. American Psychologist, 63, 760-769. doi:10.1037/0003-066X.63.8.760

37.

Mayer

R. E.

(2009). Multimedia learning (2nd ed.). New York, NY: Cambridge University Press.

38.

Mayer

R. E.

Dow

Mayer

(2003). Multimedia learning in an interactive self-explaining environment: What works in the design of agent-based microworlds? Journal of Educational Psychology, 95, 806-813.

39.

Mayer

R. E.

Jackson

(2005). The case for coherence in scientific explanations: Quantitative details can hurt qualitative understanding. Journal of Experimental Psychology: Applied, 11, 13-18.

40.

Mayer

R. E.

Johnson

C. I.

(2008). Revising the redundancy principle in multimedia learning. Journal of Educational Psychology, 100, 380-386. doi:10.1037/0022-0663.100.2.380

41.

Mayer

R. E.

Mathias

Wetzell

(2002). Fostering understanding of multimedia messages through pretraining: Evidence for a two-stage theory of mental model construction. Journal of Experimental Psychology: Applied, 8, 147-154.

42.

Moreno

Mayer

R. E.

(1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91, 358-368.

43.

Moreno

Mayer

R. E.

(2004). Personalized messages that promote science learning in virtual environments. Journal of Educational Psychology, 96, 165-173.

44.

Odom

S. L.

Brantlinger

Gersten

Horner

R. H.

Thompson

Harris

K. R.

(2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71, 137-148.

45.

Pugach

M. C.

Blanton

L. P.

Boveda

(2014). Working together: Research on the preparation of general education and special education teachers for inclusion and collaboration. In Sindelar

P. T.

McCray

E. D.

Brownell

M. T.

Lignugaris/Kraft

(Eds.), Handbook of research on special education teacher preparation (pp. 143-160). New York, NY: Routledge.

46.

Reschly

D. J.

Holdheide

L. R.

Behrstock

Weber

(2009). Enhancing teacher preparation, development and support. In Goe

(Ed.), America’s opportunity: Teacher effectiveness and equity in K-12 classrooms (pp. 41-64). Washington, DC: National Comprehensive Center for Teacher Quality.

47.

Salvucci

Walter

Conley

Fink

Saba

(1997). Measurement error studies at the National Center for Education Statistics. Washington, DC: U.S. Department of Education.

48.

Sayeski

K. L.

Hamilton-Jones

(2015). The efficacy of IRIS STAR legacy modules under different instructional conditions. Teacher Education and Special Education, 38, 291-305. doi:10.1177/0888406415600770

49.

Sayeski

K. L.

Kennedy

M. J.

de Irala

Clinton

Hamel

Thomas

(2015). The efficacy of multimedia modules for teaching basic literacy-related concepts. Exceptionality, 23, 237-257. doi:10.1080/09362835.2015.1064414

50.

Shulman

L. S.

(1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57, 1-21.

51.

Sindelar

P. T.

Brownell

M. T.

Billingsley

(2010). Special education teacher education research: Current status and future directions. Teacher Education and Special Education, 33, 8-24. doi:10.1177/0888406409358593

52.

Smith

S. J.

Kennedy

M. J.

(2014). Technology and teacher education. In Sindelar

P. T.

McCray

E. D.

Brownell

M. T.

Lignugaris/Kraft

(Eds.), Handbook of research on special education teacher preparation (pp. 178-193). New York, NY: Routledge.

53.

Stull

Mayer

R. E.

(2007). Learning by doing versus learning by viewing: Three experimental comparisons of learner-generated versus author-provided graphic organizers. Journal of Educational Psychology, 99, 808-820.

54.

U. S. Department of Education, National Center for Education Statistics (2012). Digest of education statistics [NCES 2012-001]. Retrieved from http://nces.ed.gov/fastfacts/display.asp?id=64