An Exploratory Study of Universal Design for Teaching Chemistry to Students With and Without Disabilities

Abstract

In this exploratory study, students in four co-taught high school chemistry classes were randomly assigned to a Universal Design for Learning (UDL) treatment or a comparison condition. Each co-teaching team taught one comparison and treatment class. UDL principles were operationalized for treatment: (a) a self-management strategy (using a mnemonic, IDEAS) for the multi-step mole conversion process; (b) multi-media lessons with narration, visuals, and animations; (c) procedural facilitators with IDEAS for conversion support; and (d) student workbooks mirroring video content and containing scaffolded practice problems. All students completed a pre-test, post-test, and a 4-week delayed post-test. There were no significant differences between conditions; however, there was an interaction effect between students with and without disabilities for post-tests. Social validity indicated students found IDEAS helpful. Implications for future research include continued focus on disaggregated learning outcomes for students with and without disabilities for UDL interventions, and refinements for UDL interventions that benefit students with and without disabilities.

Keywords

Universal Design for Learning multi-media self-management

According to the 2011 High School Transcript Study (Nord et al., 2011), 45% of students with disabilities did not complete the standard curriculum for graduation. Of those students with disabilities, the same percentage (45%) needed only science credits. A similar high percentage of students without disabilities (39%) did not earn sufficient science credits in biology, chemistry, and physics. Consequently, a significant number of students with and without disabilities in science classes could benefit from pedagogies other than what they are receiving. Even so, few studies have occurred in high school science classes where students with and without disabilities receive instruction (Brigham, Scruggs, & Mastropieri, 2011; Therrien, Taylor, Hosp, Kaldenberg, & Gorsh, 2011). Several researchers note the potential for pedagogies derived from the Universal Design for Learning (UDL) framework to benefit students with and without disabilities in science (Basham & Marino, 2013; Curry, Cohen, & Lightbody, 2006; Goeke & Ciotoli, 2014; Kortering, McClannon, & Braziel, 2008; Kurtts, Matthews, & Smallwood, 2009; Marino, 2009; Marino & Beecher, 2010).

UDL is a framework for designing instructional techniques that minimizes, reduces, or eliminates learning barriers for content so that students with diverse learning needs, including students with high-incidence disabilities (HID), can access content (National Center on Universal Design for Learning [NCUDL], 2010; Rose, Hasselbring, Stahl, & Zabala, 2005; Rose & Meyer, 2002). In federal legislation, the Higher Education Opportunity Act of 2008 defines UDL as “a scientifically valid framework for guiding educational practice that:

provides flexibility in the ways information is presented, in the ways students respond or demonstrate knowledge and skills, and in the ways students are engaged; and

reduces barriers in instruction, provides appropriate accommodations, supports, and challenges, and maintains high achievement expectations for all students, including students with disabilities and students who are limited English proficient.”

The UDL framework widely referred to in UDL literature (e.g., Marino, 2009; McMahon & Walker, 2014; Pisha & Coyne, 2001; Reid, Strnadova, & Cumming, 2013) is described by the NCUDL (http://www.udlcenter.org/), as comprised of three principles derived from research in cognitive neuroscience (Center for Applied Special Technology, 2011; Rose & Gravel, 2010). Each principle is further operationalized by nine guidelines, which are further identified by two to five “checkpoints.” Three of the most often cited principles (NCUDL, 2010), along with sample guidelines and checkpoints are as follows:

The principle of “multiple means of representation” refers to multiple means of perception, language, expression, symbols, and comprehension. One guideline is providing options for comprehension; a checkpoint example is highlighting patterns.

The principle of “multiple means of expression and action” include guidelines for multiple means of physical action, expression and communication, and executive function. One guideline is providing students options for communication and expression; a checkpoint example is gradually removing supports as students acquire fluency.

The principle of “multiple means of engagement” refers to multiple means of recruiting interest, sustaining effort and persistence, and self-regulation. One guideline is providing options for recruiting interest, and a checkpoint is to minimize distractions.

UDL is also featured in some states’ regulations (e.g., Maryland) as a framework for differentiation. Even so, basic and applied research supporting UDL’s efficacy and use with diverse populations, including students with HID (e.g., learning disabilities [LD], emotional or behavioral disorders, other health impaired) is scant (Edyburn, 2010; McGuire, Scott, & Shaw, 2006; Rappolt-Schlichtmann, Daley, & Rose, 2012), albeit emerging (Hall, Meyer, & Rose, 2012; Kennedy, Thomas, Meyer, Alves, & Lloyd, 2014; Marino et al., 2014). Nonetheless, many authors have called for educators’ use of UDL techniques so that students with diverse learning needs are taught in educational environments designed to proactively design and deliver instruction responsive to students’ variable needs (cf. Jimenez, Graf, & Rose, 2007; King-Sears, 2001, 2009; McPherson, 2009; Pisha & Coyne, 2001; Stanford & Reeves, 2009; Zhang, 2005). Admittedly, there is widespread intuitive appeal that UDL techniques can promote learning for students with HID, particularly for techniques that can potentially increase students’ access to, and subsequent performance in, the general education curriculum. However, contrary to how evidence-based practices emerge, a strong empirical foundation has not yet been established specific to how well students with HID learn within UDL-enhanced content instruction.

UDL Research

In a recent review of UDL research, 13 studies featuring UDL interventions were identified (Rao, Ok, & Bryant, 2014). Two studies examined lesson plans after UDL instruction, others focused on instructors’ or students’ engagement or satisfaction during or after UDL implementation, but few focused on student learning after UDL instruction. In some studies, the UDL interventions’ descriptions lacked detail that made it clear exactly which UDL principles and corresponding guidelines were being operationalized. Notably, Marino (2009) explicitly connected the technology-based science research to specific UDL principles, conducted the research within the context of general education science classes, and reported learning outcomes disaggregated for students with reading difficulties compared with peers without reading problems. His findings are instructive because the anticipated positive learning impact for students who had reading difficulties did not occur, although more proficient readers did benefit from the UDL intervention. Absent examining differential impacts on students, universal impacts are not known.

Also included in Rao et al.’s (2014) review is a case study research design in which UDL techniques were incorporated into high school general education science classes that included students with a range of disabilities (Dymond et al., 2006). The educators indicated changed roles when UDL was in place, including that the science teacher expressed more ownership of all students, and the special educators noted they became more active in teaching the content versus developing adaptations. From the teachers’ perspectives, students enjoyed the UDL-enhanced techniques, such as graphic organizers, step-by-step activity directions, demonstrations for how to complete lab activities, and visual models of projects at different stages of completion. Although techniques were well-described, the authors did not specify which UDL principles were being operationalized, nor did they report on student learning outcomes.

Rao et al. (2014) concluded with three recommendations. First, the authors called for uniformity when describing UDL interventions by identifying explicit connections of UDL principles and guidelines to corresponding component(s) of independent variables. Second, Rao et al. noted more detail about the participants in UDL research was needed, including disability category and achievement information, so that practitioners and researchers would be able to determine generalizability for their students. In addition, just as states’ assessment scores are disaggregated for specific subgroups, researchers should similarly disaggregate when reporting learning outcomes for students receiving UDL treatments. By doing so, researchers would be able to provide evidence of UDL’s differential effects on students with diverse learning needs. Finally, although acknowledging potential definition and operationalization issues for UDL may be at an evolutionary stage, Rao et al. noted the need for research that provides definitive information about how UDL-derived techniques affect learning.

Several UDL research studies have been published since research included in Rao et al.’s (2014) review. Kennedy et al. (2014) developed “content acquisition podcasts,” or CAPs, to determine the differential effects of learning world history definitions for high school students with and without disabilities. Retention of terms’ definitions by students with and without disabilities who learned via CAPs were compared with students who learned using more traditional techniques, such as using the textbook’s glossary, copying the terms and definitions in their notebooks, and engaging in review activities. The study spanned 2 units across an 8-week period, and the teacher used CAPs at natural points during instruction to teach each term using the CAP. The amount of time spent per each of 81 terms was 1 to 3 min, once when the term was introduced, and another time prior to the test. Students with disabilities reviewed 2 more times during their study period, once after the term was shown via CAPs when the teacher taught it, and at another time prior to the test. For students with and without disabilities, their learning was significantly higher in the CAPs treatment. Disaggregating scores per condition (treatment and comparison) for students with or without disabilities provided Kennedy et al. evidence that CAPs benefitted both groups of students similarly.

In another study where UDL implementation occurred in elementary science classes, the researchers found that students who used a UDL-enhanced notebook (e.g., text-to-speech features, terms with illustrations, prompts and scaffolds during activities, multi-media response options) scored higher on post-tests than their peers who used traditional notebooks (Rappolt-Schlichtmann et al., 2013). Students used either the UDL-enhanced or traditional notebook over an 8- to 10-week period; the amount of time the notebooks were used during this period was not identified. However, the researchers found students at different reading and writing levels, and with variable motivation for science learning, benefitted similarly to the intervention.

Conversely, Marino et al. (2014) conducted a study in which video games and alternative print-based texts were used to supplement middle school science instruction, students with LD did not perform as well as anticipated. The average instructional intervention time was 800 min across 14 days, with approximately 100 min of that time spent with students playing the video games. Contrary to expectations, students with disabilities, regardless of whether they were in the UDL treatment or comparison condition, performed at about the same level. Although Marino et al. posited reasons why treatment students did not outperform their peers receiving more traditional instruction (e.g., comparison condition students received explicit review of content the day before the post-test), the researchers’ results exemplify why empirical support is needed to more fully understand how UDL affects learning outcomes for students with disabilities. The researchers focusing on UDL in secondary science classes (Dymond et al., 2006; Marino, 2009; Marino et al., 2014) are of particular interest because students with disabilities need science credits to graduate with a regular high school diploma (McIntosh, 2011). Among the science areas that students with and without disabilities need to successfully complete is chemistry. Two chemistry studies examined the performance of students with and without disabilities in general education classes, and each found favorable results for students with and without disabilities in treatment conditions (Lynch et al., 2007; Mastropieri, Scruggs, & Graetz, 2005). However, no UDL studies have occurred in chemistry.

The Current Study

Given the need for research on both UDL and chemistry for secondary students with HID, we developed a multi-component module of lessons focusing on molar conversions that integrated principles, guidelines, and checkpoints from universal design and incorporates prior research about how students with and without disabilities learn. Our research questions were as follows:

Research Question 1: Are students with and without HID taught using a UDL treatment better able to solve one- and two-step mole conversion problems than students taught using comparison instruction (i.e., business as usual)?

Research Question 2: Do these students maintain performance after a 4-week delay?

Method

Participants

Students and teachers from four co-taught chemistry classes participated in the study. They were recruited from two high schools in the South Atlantic region of the United States. Consistent with procedures from both the university and school system research boards, informed consents were required from all participants. Two teams of co-teachers each taught two similar chemistry classes at their respective high schools. For each team, random assignment occurred to determine which classes would be UDL or comparison conditions. The participants included general education students (GED) without disabilities and students with HID. To be eligible as research participants, students needed to be present for entire sessions on all days of the study. In the UDL condition, there were 17 GED students and 7 students with HID. Four of the students with HID had LD, 1 had autism, 1 had “other health impairment” (OHI), and 1 had speech/language disabilities. In the comparison condition, there were 24 GED students and 12 students with HID. Six of the students with HID had LD, 4 had OHI, 1 was labeled with serious emotional disturbance, and 1 was identified with having autism. Refer to Table 1 for more detail on the participants’ characteristics, including ethnicity, English language learner status, socioeconomic status, and individualized assessment scores that were available for all students with HID.

Table 1.

Participant Characteristics.

	Comparison GED	Comparison HID	UDL GED	UDL HID
Characteristic	(n = 24)	(n = 12)	(n = 17)	(n = 7)
Gender
Male	8	3	10	4
Female	16	9	7	3
Race/ethnicity
White (not Hispanic)	13	5	5	4
African American (not Hispanic)	3	1	3	0
Hispanic	1	5	4	1
Asian or Pacific islander	6	0	4	1
Multiracial	1	1	1	1
Primary disability category
Specific learning disability		6		4
Other health impairment		4		1
Serious emotional disturbance		1		0
Autism		1		1
Speech or language impairment		0		1
English as second language
Yes	5	3	9	2
No	19	9	8	5
Free/reduced-price lunch
Yes	1	3	3	2
No	23	9	14	5
Grade
10	24	7	13	7
11	0	4	4	0
12	0	1	0	0
Highest math-level course as determined by completion of state assessment
Algebra 1	7	9	7	4^a
Geometry	17	3	10	2
	Students with HID only
Intelligence quotient average, (n), range
Wechsler Intelligence Scale for Children		98, (8), 84–116		108.5, (2), 103–114
Wechsler Adult Intelligence Scale		0		89, (1)
Reynolds Intellectual Assessment Scales		96.5, (2), 89–104		0
Stanford–Binet		0		110, (1)
Data not available		2		3
Reading achievement, (n), range
Woodcock Johnson Battery 3		81, (3), 76–87		0
Kaufman Test of Educational Achievement		99, (6), 83–112		99.67, (3), 90–107
Woodcock Reading Mastery		110, (1)		0
Kaufman Test of Educational Achievement–II		0		87, (1)
Data not available		2		3
Mathematics achievement, (n), Range
Woodcock Johnson Battery 3		88, (3), 81–93		0
Kaufman Test of Educational Achievement		97, (6), 90–105		100.33, (3), 87–118
KeyMath		103, (1)		0
Kaufman Test of Educational Achievement–II		0		97, (1)
Data not available		2		3
Writing achievement, (n), range
Woodcock Johnson Battery 3		84.5, (4), 79–93		0
Kaufman Test of Educational Achievement		102.8, (5), 93–119		98.67, (3), 88–111
Test of Written Language		103, (1)		0
Test of Written Language–3		0		82, (1)
Data not available		2		3

Note. GED = general education student; HID = student with high-incidence disability; UDL = universal design for learning.

One student did not complete this state assessment.

Although all classes were co-taught, the special education co-teachers took the lead role for instruction in both the UDL Mole Module and comparison classes, and the general education teachers were either observers or assisted the students. The special educators (one male, one female) were both Caucasian with 9 years of teaching experience and an average of 5 years teaching chemistry (range = 2–8 years). Both held master’s degree in special education, with certification in special education. One was also certified in middle school mathematics and chemistry. Both general education co-teachers were female (one Caucasian, one African American) with an average of 10.5 years of teaching experience in chemistry (range = 5–16 years). Both were certified in chemistry, and one had an undergraduate degree in chemistry.

Instruments

Mole conversion tests

Three equivalent versions of Calculating Mole Conversions tests were developed by the researchers. A different version was used as the pre-test, post-test, and delayed post-test. Five mole conversion problems were on each test, with each problem-per-test constructed to encompass the same problem type and rigor taught in the intervention. To ensure equivalent forms, a checklist was used to mark off each problem type and to determine that similar complexity was evident with chemistry compounds used per assessment for balance on the three tests. In addition, wording for the problems varied similarly across each test. Two examples are as follows: (a) What is the mass of 2.3 moles of Ca₃(PO₄)₂? (b) Convert 71 L of CO₂ gas to moles.

Summing points for accuracy of each step in the problem calculated an overall score. Tests were scored using a rubric designed to assign weighted point values in the conversion process as well as the accurate answer. Similar to how students are required to show their work for long division calculations, students in this study needed to show all their work for each conversion problem. The weighted point system was based on the relative importance of individual steps to the overall calculation of the final answer, along with the cognitive complexity required to negotiate each step. For example, calculating the molar mass of CO₂ was less complex than for Al₂(SO₄)₃, so the weight for each differed. Each one- and two-step mole conversion problem consisted of varying point values, from 25 to 38 points, with a total of 153 points for the pre-test, post-test, and delayed post-test. The rubric was validated by two university science education faculty members and three high school chemistry teachers (one was also a science department chair, and another developed items for the state’s chemistry assessment). Inter-rater reliability was calculated for all pre-tests, post-tests, and delayed post-tests, resulting in 99%, 98%, and 99% agreement, respectively.

Social Validity Questionnaire

Students in the UDL treatment were asked to complete social validity questionnaires after the delayed post-test. Students indicated their response to statements about the processes and materials used in the study using a 4-point Likert-type scale (1 = strongly disagree, 2 = disagree, 3 = agree, and 4 = strongly agree). Students were also asked to respond to open-ended questions, describing their feedback about specific parts of the UDL Mole Module.

Materials

Types of materials

Only students in the UDL treatment condition were provided materials from the researchers. There were four types of materials comprising the UDL Mole Module. First, 10 video clips were developed and narrated, using Camtasia™ software, with video clips varying in length from 3 min to 16 min. The sequence and content for the video clips are shown in Table 2 and further described in the next section. The first three video clips overviewed the UDL Mole Module materials, described the scientist Amedeo Avogadro, and demonstrated how to calculate molar mass of elements and compounds. The fourth video clip demonstrated the IDEAS self-management strategy and how to solve two problems, with the fifth and sixth video clips (5a and 5b) focusing on solving one-step mole conversions (refer to http://www.youtube.com/watch?v=y-soVWF0vDQ for Video Clip 5a). The next three video clips (6a, 6b, and 6c) demonstrated how to solve two-step mole conversions. The last video clip (7) prepared students for completing a set of problems with both one- and two-step conversions (referred to as “mixed problems”).

Table 2.

UDL Mole Module: Day of Treatment and Video Clip Number and Title.

Day	Video clip no.	Title
1	1	Overview of calculating mole conversions and student materials
	2	What is a mole? And who is Avogadro?
	3	Molar mass demonstration
	4	Identify and describe the IDEAS Self-Management Strategy (first)
	5a	Demonstrate second 1-step problem with IDEAS and pattern
	5b	Demonstrate third 1-step problem with IDEAS and pattern
2	6a	Demonstrate first 2-step problem with IDEAS and pattern
	6b	Demonstrate second 2-step problem with IDEAS and pattern
	6c	Demonstrate third 2-step problem with IDEAS and pattern
3	7	Demonstrate mixed problems with IDEAS and pattern

Note. UDL = Universal Design for Learning.

The IDEAS self-management strategy (see Figure 1) consisted of a sequential process focusing students on how and when to make decisions to solve conversion problems, including whether to use the one-step or two-step process. In addition to IDEAS as the self-regulation strategy, patterned boxes depicting where to write numbers and corresponding units (e.g., 25 grams) using IDEAS were developed as a graphic procedural facilitator to “guide students in the simultaneous processes of utilizing learning skills and learning content” (Scanlon, Cass, Amtzis, & Sideridis, 2009, p. 293). The video clips were uploaded to YouTube or a drop box so teachers could access them online. The Camtasia™ video clips were developed from a PowerPoint™ presentation in which animation was used to draw attention to specific features of problems as they were solved. In addition, the Camtasia™ software included highlighting features so that when a static problem was depicted on the screen (“How many moles are in 2.86 × 10²⁵ atoms of Sb?”), the parts of the problem being narrated could be spotlighted. Animation was used, such as arrows moving to specific parts of the patterned boxes or underlining to indicate what part of the problem was being described. As each step in the IDEAS self-management strategy was modeled and described by verbal narration, corresponding visuals were used. In this manner, the I or D or E or A or S were demonstrated step-by-step, explaining how decisions were made and showing where content appeared in the Procedural Facilitator (patterned boxes). How and when hard copy materials should be used by students as prompts or reminders was visually and verbally presented initially in the video clips and also throughout the Mole Student Workbook (MSW). Directions and materials were organized to facilitate students’ procedural memory.

Figure 1.

IDEAS self-management strategy and pattern for two-step mole conversions in Mole Student Workbook.

Second, each student’s MSW contained slides from each video clip as well as practice problems. Descriptions of some key features within the MSW follow. The IDEAS self-management strategy was described along with rationale for using the strategy. Within specific video clips, demonstrations of three 1-step problems using IDEAS occurred, and each of those slides was depicted on pages in the MSW. This provided students with the option to access content from the video clip after it was shown, and they could refer to worked examples later when practicing. After each of three 1-step problems demonstrated in the MSW were 12 practice problems (practice problems were not in video clips):

Problems 1 through 3 showed all IDEAS content (refer to Figure 1 for an example).

Problems 4 through 6 showed the first word in IDEAS in checklist format.

Problems 7 through 9 showed the first letter in IDEAS in checklist format.

Problems 10 through 12 showed only the practice problems.

The gradual fading of IDEAS was used to provide maximum support initially for students to become accustomed to using the strategy. Students could continue to use IDEAS strategy on their own for the last practice problems. The final three practice problems included no supports, and resembled what students would see on their state’s chemistry assessment. Similar to how the pre-, post-, and delayed post-tests contained problems with different wording and a mix of compounds’ complexity, the same occurred across all practice problems. Within the MSW, the same sequence was used for two-step and then mixed (one-step or two-step) problems. That is, all demonstration problems were in the MSW followed by 12 two-step practice problems with scaffolding, then mixed practice followed by 12 practice problems with scaffolding.

Third, each student had a laminated strategy sheet that consolidated key elements of information needed for mole conversions. The periodic table of elements was shown on one side of the laminated strategy sheet; the other side contained the IDEAS self-management strategy, a visual of one element from the periodic table cueing students for how to locate which number represented the molar mass, and three mole equalities from which to choose when calculating conversions. The three mole equalities were contained within a gray rectangular box representing the Mole Equality Organizer; students referred to the organizer as the “gray box of mole equalities.” For the equality representing mass, a box contained “Mm,” to mean Molar mass, which cued students to refer to the squares on the periodic table for the needed values.

Fourth, multiple copies of answer keys for the practice problems in the MSW were developed. Students who worked at different paces could check their work after completing the practice problems in the MSW.

UDL framework operationalized for this study

Table 3 presents UDL principles and how guidelines and checkpoints (in parentheses) were operationalized for this study. Several UDL Mole Module features overlapped between and among principles. For example, the IDEAS self-management strategy was initially used for representation to provide options for comprehension and language as well as mathematical expressions. IDEAS was also used as an engagement tool to have students focus on the critical steps (minimize distractions) and have students self-regulate their completion of each practice conversion problem. Finally, for expression, the IDEAS mnemonic could be used by the students to scaffold their levels of support so that they referred to the strategy as needed until they could convert accurately independently.

Table 3.

UDL Mole Module Aligned to UDL Framework.

UDL principle	UDL mole module feature, guideline, and checkpoint
Representation principle: Options for multiple ways to represent new content	IDEAS Self-Management Strategy
	Options for mathematical expressions (support decoding of mathematical notations)
	Strategy Sheet and Mole Equality Organizer
	Options for perception (offer alternatives for auditory and visual information), and comprehension (highlight big ideas)
	Multi-Media Mole Video Clips and Scaffolded Practice
	Options for language, mathematical expressions, symbols (illustrate through multiple media, clarify syntax and structure, and support decoding of text, mathematical notation, and symbols), and comprehension (highlight patterns, critical features, and relationships, and guide information processing and visualization)
Engagement principle: Options for multiple ways students can engage in practice of new content	IDEAS Self-Management Strategy
	Options for expression, communication (build fluency with graduated levels of support for practice and performance), and executive functions (support planning, strategy development)
	Strategy Sheet, Procedural Facilitator, and Mole Equality Organizer
	Options for interest (minimize distractions)
Expression principle: Options for multiple ways students can express what they know	IDEAS Self-Management Strategy
	Options for interest (minimize distractions), and self-regulation (facilitate use of strategy)
	Strategy Sheet and Mole Equality Organizer
	Options for executive function (facilitate managing information and resources)
	Scaffolded Practice
	Options for expression (build fluency with graduated levels of support for practice and performance)
	Procedural Facilitator
	Options for communication and expression (i.e., use multiple tools for construction)

Note. UDL = Universal Design for Learning.

Procedure

Prior to intervention, all students completed the pre-test. Then, students in the treatment group commenced with the UDL instruction, while students in the comparison group were taught how to calculate mole conversions with the teachers using the techniques and materials that they typically used. Each teacher had different procedures for teaching mole conversions to students in the comparison groups. One teacher relied heavily on a visual support called the “mole Y map,” and the other teacher used a five-step process. The duration and focus of instruction for both treatment and control classes was the same, although the instructional methods and materials varied. The duration of the instruction was determined by the quantity of time prescribed by the school system for instructing on mole conversions. Because block scheduling occurred for each school, the treatment and comparison sessions occurred every-other day across a 2-week time period for a total of 3 sessions, with 90 min per session.

For the UDL treatment, teachers were provided an 11-step script describing the UDL Mole Module’s implementation on each of the 3 days of instruction. In-person and phone meetings occurred to describe the UDL treatment. Demonstration of the IDEAS self-management strategy occurred, along with modeling how to guide students during practice to use IDEAS. All materials were described in terms of how each related to the intervention as well as when and how the materials were to be used. The 11-step script alerted teachers about content to be practiced in the MSW within or at the end of specific video clips, as well as when students should access content on their laminated sheets. Other directions for teachers were within the video clips themselves, such as alerting teachers to pause and have students calculate molar mass in their MSW. Teachers viewed video clips on their own, MSW, and other materials to learn IDEAS as well as the sequence of the intervention. Researchers met with teachers approximately 1 week prior to the intervention’s start date to ensure they were satisfactorily familiar with IDEAS and other intervention components. Teachers were asked to play the video clips in the classroom where intervention would be occurring to determine that the Internet was accessible, the video clips were playing well, and audio was satisfactory. We videoed the teachers’ instruction, and the videos were examined to determine fidelity of treatment.

For students in the treatment groups, MSW were distributed, and Video Clip 1 was played, which was an overview of the students’ materials and general content (e.g., the unit organizer, the laminated sheet, the IDEAS strategy). Teachers reviewed concepts from the video clip that had just ended prior to starting the next video clip. Next, Video Clip 2 was played. The script indicated when teachers should expect student practice to occur within a movie so they could pause the video clip at the appropriate points for students to engage in practice. When students practiced, they could choose to partner with each other, work in small groups, or work independently.

On the last day of intervention, if students had not had a chance to complete the problems from a previous day of intervention, they were directed to complete all practice problems in the MSW prior to completing the mixed problem set. During the next class session after the last day of intervention, students completed the post-test. Four weeks after that, all students completed the delayed post-test.

Fidelity of Treatment

We randomly selected two video clip segments across two of three 90-min teaching sessions for analysis. Teacher implementation was observed to determine fidelity, such as whether directions within and between the video clips were followed (e.g., pause video, check that students completed specific MSW pages, help students check their work, and resume video).

Both teachers experienced technical difficulties on the first day of intervention. Although each teacher had tested Internet access and audio prior to Day 1, each experienced initial problems accessing, then playing, video clips. Once the video clips were accessed, audio transmission in the classroom environment became sporadic with clarity and volume. Consequently, each teacher discontinued use of video clips and used content from the video clips from the MSW pages, to deliver instruction.

On Day 2, two parts of the day’s instruction were randomly selected for fidelity analysis. In School 1, the teacher’s instruction and use of materials yielded 100% fidelity. In School 2, the teacher’s instruction and use of materials yielded 57% fidelity. Among the issues contributing to the lower fidelity was re-teaching a video clip instead of having students begin practice problems and playing the video clips out of sequence.

Results

Means, standard deviations, and ESs for student performance data are presented in Table 4. To determine whether there were differences between conditions prior to intervention, pre-test data were analyzed in a two-condition (UDL vs. Comparison) × two-population (GED vs. HID) ANOVA. Results showed no significant differences for condition, F(1, 56) = .41, p = .53; population, F(1,56) = .68, p = .41; nor between condition and population, F(1, 56) = .19, p = .67.

Table 4.

Scores on Calculating Mole Conversions Pre-test, Post-test, and Delayed Post-test.

	UDL condition	Comparison condition
Stage and group	M (SD)	M (SD)	ES
Pre-test	10.38 (20.82)	7.06 (26.09)
GED	11.18 (23.87)	9.76 (31.09)
HID	8.43 (11.62)	0.91 (1.58)
Post-test	111.25 (42.53)	114.22 (49.48)	−.06
GED	110.35 (44.34)	131.60 (35.45)	−.53
HID	113.43 (41.02)	74.73 (55.53)	.80
Delayed post-test	111.22 (44.21)	113.13 (44.32)	−.04
GED	113.94 (47.80)	130.75 (20.84)	−.49
HID	105.00 (37.26)	60.25 (54.87)	.97

Note. UDL = Universal Design for Learning; GED = general education student; HID = student with high-incidence disability.

After receiving the intervention, post-test data were also analyzed in a two-condition (UDL vs. Comparison) × two-population (GED vs. HID) ANOVA. There were no significant differences observed for condition, F(1, 56) = .50, p = .48. However, results did show a significant difference for population, F(1, 56) = 4.75, p = .03, and a significant interaction effect between condition and population, F(1, 56) = 5.9, p = .02 (see Figure 2).

Figure 2.

Interaction effect between condition and population.

To determine whether students maintained performance 4 weeks after the intervention, the Calculating Mole Conversions delayed post-test data were analyzed. There were not significant differences observed for condition, F(1, 51) = 1.45, p = .23. However, results again showed a significant difference for population, F(1, 51) = 11.74, p = .001, and an interaction effect between condition and population, F(1, 51) = 7.05, p = .01.

Social Validity

Students in the UDL treatment rated statements using a 4-point Likert-type scale (strongly disagree to strongly agree). All students, whether GED or with HID, agreed or strongly agreed that the laminated strategy sheet and the Mole Equality Organizer were helpful, and a majority of students indicated that the MSW was helpful (87% GED; 100% HID). A majority of students also agreed or strongly agreed that (a) IDEAS improved their learning (80% GED; 86% HID) and (b) the procedural facilitators (i.e., patterned boxes) helped them organize for conversions (73% GED; 83% HID). Students also indicated that they liked IDEAS (73% GED; 100% HID), would recommend IDEAS to other students (87% GED; 71% HID), and that they would like to learn other strategies like IDEAS (67% GED; 100% HID). Just more than half of the students agreed or strongly agreed that the video clips (53% GED; 57% HID) were helpful, or that their grades improved due to IDEAS (67% GED; 57% HID).

In students’ responses to open-ended questions, almost all comments about the IDEAS strategy, the procedural facilitator, and the MSW were positive. Contrary to their ratings for the laminated strategy sheet, a mix of positive comments emerged about students’ use of the strategy sheet, with some students indicating they did not use the sheet. Students’ comments on the video clips indicated the need for improvement in how well they played, particularly for the audio and pace.

Discussion

Group Averages Per Condition

When focusing on the group means for all students in the treatment versus comparison conditions, the UDL Mole Module treatment was not more effective compared with typical instruction. Instruction in both comparison and treatment conditions warrant further examination and instructional enhancements to yield higher mean scores. Although we did see some aspects of effective instruction occurring in the comparison conditions, those aspects were not equally distributed across each comparison group. In addition, further refinements of the UDL treatment remain focused on not just whether students learn more, but how close the students’ learning is to the desired outcome of 100% on the post-test.

Disaggregated Averages Per Condition

Differential effects for the subgroups, GED and HID, were more clearly evident when findings were disaggregated by subgroup in each condition. Disaggregated findings suggest that students with HID in the UDL condition scored substantially higher on the post-test than students with HID in the comparison condition, as evidenced by the large effect sizes of .80 at post-test and .97 at delayed post-test. In fact, post-test scores of students with HID were comparable with post-test scores of GED students. Scores were lower for students with HID in both conditions at delayed post-testing. However, although students with HID in both conditions retained less knowledge at delayed-post testing, students in the UDL treatment condition still continued to outperform students with HID who received typical instruction. Because one intent for UDL-derived instruction is to be responsive to varying needs of diverse learners, unless data from those diverse learners are isolated (in this study, students with HID), there is meager evidence at best when relying on the group’s mean to “tell the whole story.” If we only calculate the groups’ mean and deduce that traditional instruction is slightly more effective, then we have not yet uncovered whether UDL has similarly affected learning for subgroups within the larger group. Indeed, it is only when we disaggregate the data for comparison students that we realize at post-test, GED students’ mean in percentage is 86%, and the mean for students with HID is 49%. GED students’ mean for the delayed post-test was fairly steady, at 85%, whereas the mean for students with HID fell to 39%.

Although there is some evidence that the UDL treatment may be potentially positive for students with HID, outcomes were not as favorable for GED students who performed more poorly in the UDL condition than GED students who received typical instruction. Because comparable gains were not evident for GED students (per negative ESs = −.53 for post-test and −.49 for delayed post-test), this raises questions about the utility of the UDL treatment for typically achieving students. Therefore, future research should proceed cautiously and with full consideration of how to either achieve or expand the flexibility characteristics of universally designed treatments to address the learning needs of all students—including typically developing students in general education settings.

Flexibility and Fidelity

Further studies are needed to ascertain what flexibility is needed to create an environment in which students in GED and students with HID are similarly responsive to UDL instruction. Although UDL is characterized by flexibility and variability, identifying critical intervention components vary dependent on learners’ needs. That is, other students may not require what may be critical agents for some students. Rappolt-Schlichtmann et al. (2013) noted the importance of UDL’s capacity to be responsive to and appropriately challenging for students with the range of learning needs: “A student can choose to access or ignore a given support to use any of the various means of responding to a prompt, and to watch or pass by a video that provides additional information” (p. 1221). With flexibility and variability as key features of UDL-derived interventions, fidelity of treatment becomes more difficult to monitor and measure when different students access different aspects of the intervention, for different lengths of time, and for different purposes. Ways to differentiate while still adhering to fidelity measures are also a focus for future studies.

Study Limitations

There are several limitations for this exploratory study. First, the size of the groups, both for students with and without HID, were limited. Second, fidelity of treatment was affected when the technology did not work as intended on the first day of intervention. Although teachers could provide the treatment in lieu of the video clips, and the MSW contained corresponding slides and practice opportunities, it was a disruption when audio was not clear and Internet access was not consistent. In addition, for the second day of instruction, one UDL treatment class had low fidelity. Third, although the amount of time for the study was consistent with the school system’s pacing guide, an intervention of longer duration is desirable. Finally, for both the tests and scoring procedures, the focus was on students’ skills for calculating one- and two-step mole conversions. Although as dependent variables, the measures were consistent with assessing student learning, whether students’ background knowledge or prerequisite skills varied in ways that influenced their learning was not explored in this study.

Future Research

Any exploratory treatments, whether derived from UDL or otherwise, that result in student learning that is comparable with “business as usual” raise further questions about the treatment and what is going on in the comparison instruction. Because of the limited number of teachers and classrooms within the current study, and because of fidelity issues with technology malfunctions and implementation, findings are interpreted cautiously. Future research that compares UDL-designed interventions with one technological and the other not can help focus researchers and practitioners on advantages and disadvantages of each format with learning outcomes of students with and without disabilities retained as a requirement.

Additional research is needed to determine whether results for the UDL intervention’s effectiveness might differ with greater fidelity, both for the technology’s operation and for teachers’ implementation. Several researchers acknowledge technological and Internet access difficulties in some school environments that do not have the infrastructure to accommodate the level of access needed for interventions (Coy, Marino, & Serianni, 2014; Rappolt-Schlichtmann et al., 2013). As such, when there is capacity on the researchers’ end to build stand-alone interventions that are not reliant on Internet access or transcending systems’ technological capacities, difficulties encountered in this study may be reduced or eliminated entirely. Moreover, in schools that do not have technological resources, there could be options for teacher-delivered versus technology-delivered instruction on the IDEAS self-management strategy.

As found by Marino (2009) and Marino et al. (2014), even when fidelity for UDL interventions is high, data analyses are key for determining impact on different students and different groups of students. Did the UDL treatment, which benefitted students with HID, slow the learning pace for students without disabilities? Even with provisions for varied pacing (i.e., students could work ahead; students could review) built into the UDL treatment, students without disabilities may have needed more flexibility with pacing, including both more varied ways to practice the problems and practice that focused on proficiency and accuracy (engagement). As noted by Kennedy et al. (2014) in their use of CAPs as a multi-media UDL intervention for vocabulary definitions in world history, students with disabilities were able to view CAPs at least 2 more times prior to the post-test. Future research should provide for that flexibility, yet within parameters stipulated by fidelity of treatment measures.

As is characteristic in design-based research, feedback from students and teachers inform the next iteration of an intervention. Based on feedback we received for this study, more engagement opportunities either within or instead of video clips are desired. Similar to results from Marino et al. (2014), who found that students with LD in the UDL treatment group thought they learned much better when game playing than when more traditional instruction occurred, students with and without disabilities in our study provided positive feedback about the varied UDL components. As such, although the students’ feedback provides encouragement from a social validity perspective from the UDL intervention, it is critical that corresponding learning outcomes are attained. To that end, continued disaggregation of scores for students with and without HID is necessary for determining UDL’s differential effects, as is evident in this research when examining the classes’ mean scores. Interventions intended as UDL are not evidenced for students with and without HID until performance is examined. Correspondingly, teachers who believe their typical instruction is reaching students with HID should consider similar disaggregation of teacher-made tests to determine whether students are progressing.

As noted by Rao et al. (2014), researchers should also clearly identify how their intervention aligns to the UDL framework, and describe those connections with sufficient detail that explicitly notes how UDL is operationalized. In addition, there is research that has not evolved from the UDL framework but seems aligned with some aspects. There is merit to examining the empirical work derived from other researchers who have more extensively examined effective techniques with strong theoretical underpinnings (e.g., cognitive load, multi-media-principles, explicit instruction, scaffolding, strategy instruction) and determine the extent to which the UDL framework can be informed (Carlson, Chandler, & Sweller, 2003; Hughes, 2011; Kirschner, Sweller, & Clark, 2006; Liu & Bera, 2005; Moreno & Park, 2010; Plass, Kalyuga, & Leutner, 2010; Rosenzweig, Krawec, & Montague, 2011; Swanson & Hoskyn, 2001; Sweller, 2010).

Similarly, there are other researchers who did not use the UDL framework to derive their interventions, but whose interventions might be considered applications of UDL. Exemplars from special education research that could ostensibly be matched to the UDL framework are providing structure via using explicit instruction, scaffolding during learning, and facilitating metacognitive supports (Hughes, 2011; Montague, 2007; Scheuermann, Deshler, & Schumaker, 2009; Witzel, Mercer, & Miller, 2003). But are interventions that match principles, guidelines, or checkpoints of UDL exemplars of UDL? This is a question that provokes further discussion for educators in general, and special educators in particular who are working with secondary students with HID in general education settings. If the potential of UDL is that the design of instruction in general education content classes could be enhanced such that students with HID might be more successful, then operationalization of what UDL is and how much UDL is needed for whom and under what conditions must expand to conducting exploratory and experimental research studies.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Basham

J. D.

Marino

M. T.

(2013). Introduction to the topical issue: Shaping STEM education for ALL students. Journal of Special Education Technology, 25(3), 1–2.

Brigham

F. J.

Scruggs

T. E.

Mastropieri

M. A.

(2011). Science education and students with learning disabilities. Learning Disabilities Research & Practice, 26, 223–232.

Carlson

Chandler

Sweller

(2003). Learning and understanding science instructional material. Journal of Educational Psychology, 95, 629–640. doi:10.1037/0022-0663.95.3.629

Center for Applied Special Technology. (2011). Universal design for learning guidelines version 2.0. Wakefield, MA: Author.

Coy

Marino

M. T.

Serianni

(2014). Using universal design for learning in synchronous online instruction. Journal of Special Education Technology, 29, 63–74.

Curry

Cohen

Lightbody

(2006). Universal design in science learning. Science Teacher, 73, 32–37.

Dymond

S. K.

Renzaglia

Rosenstein

Eul Jung

Banks

R. A.

Niswander

Gibson

C. L.

(2006). Using a participatory action research approach to create a universally designed inclusive high school science course: A case study. Research & Practice for Persons With Severe Disabilities, 31, 293–308.

Edyburn

(2010). Would you recognize universal design for learning if you saw it? Ten propositions for new directions for the second decade of UDL. Learning Disability Quarterly, 33, 33–41.

Goeke

J. L.

Ciotoli

(2014). Inclusive STEM: Making integrative curriculum accessible to all students. Children’s Technology and Engineering, 18, 18–22.

10.

Hall

T. E.

Meyer

Rose

D. H.

(2012). An introduction to universal design for learning. In Hall

T. E.

Meyer

Rose

D. H.

(Eds.), Universal design for learning in the classroom (pp. 1–8). New York, NY: Guilford Press.

11.

Higher Education Opportunity Act. (2008). Retrieved from http://www2.ed.gov/policy/highered/leg/hea08/index.html

12.

Hughes

(2011). Effective instructional design and delivery for teaching task-specific learning strategies to students with LD. Focus on Exceptional Children, 44, 1–16.

13.

Jimenez

T. C.

Graf

V. L.

Rose

(2007). Gaining access to general education: The promise of universal design for learning. Issues in Teacher Education, 16, 41–54.

14.

Kennedy

M. K.

Thomas

C. N.

Meyer

J. P.

Alves

K. A.

Lloyd

J. L.

(2014). Using evidence-based multimedia to improve vocabulary performance of adolescents with LD: A UDL approach. Learning Disability Quarterly, 37, 71–86. doi:10.1177/0731948713507262.

15.

King-Sears

M. E.

(2001). Three steps for gaining access to general education curriculum for learners with disabilities. Intervention in School and Clinic, 37, 67–76.

16.

King-Sears

M. E.

(2009). Universal design for learning: Technology and pedagogy. Learning Disability Quarterly, 32, 199–201.

17.

Kirschner

P. A.

Sweller

Clark

R. E.

(2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41, 75–86. doi:10.1207/s15326985ep4102_1

18.

Kortering

L. J.

McClannon

T. W.

Braziel

P. M.

(2008). Universal design for learning: A look at what algebra and biology students with and without high incidence conditions are saying. Remedial and Special Education, 29, 352–363.

19.

Kurtts

S. A.

Matthews

C. E.

Smallwood

(2009). (Dis)solving the differences: A physical science lesson using universal design. Intervention in School and Clinic, 44, 151–159.

20.

Liu

Bera

(2005). An analysis of cognitive tool use patterns in a hypermedia learning environment. Educational Technology Research & Development, 53, 5–21.

21.

Lynch

Taymans

Watson

W. A.

Ochsendorf

R. J.

Pyke

Szesze

M. J.

(2007). Effectiveness of a highly rated science curriculum unit for students with disabilities in general education classrooms. Exceptional Children, 73, 202–223.

22.

Marino

M. T.

(2009). Understanding how adolescents with reading difficulties utilize technology-based tools. Exceptionality, 17, 88–102. doi:10.1080/09362830902805848

23.

Marino

M. T.

Beecher

C. C.

(2010). Conceptualization RTI in 21st-century secondary science classrooms: Video games’ potential to provide tiered support and progress monitoring for students with learning disabilities. Learning Disability Quarterly, 33, 257–272.

24.

Marino

M. T.

Gotch

Israel

Vasquez

III Basham

J. D.

Becht

(2014). UDL in the middle school science classroom: Can video games and alternative text heighten engagement and learning for students with learning disabilities? Learning Disability Quarterly, 37, 87–99.

25.

Mastropieri

M. A.

Scruggs

T. E.

Graetz

(2005). Cognition and learning in inclusive high school chemistry classes. In Scruggs

T. E.

Mastropieri

M. A.

(Eds.), Cognition and learning in diverse settings: Advances in learning and behavioral disabilities (Vol. 18, pp. 107–118). Oxford, UK: Elsevier Science.

26.

McGuire

J. M.

Scott

S. S.

Shaw

S. F.

(2006). Universal design and its application in educational environments. Remedial and Special Education, 27, 166–175.

27.

McIntosh

(2011). State high school tests: Changes in state policies and the impact of the college and career readiness movement. Washington, DC: Center on Education Policy.

28.

McMahon

Walker

(2014). Universal design for learning features and tools on iPads and other iOS devices. Journal of Special Education Technology, 29, 39–49.

29.

McPherson

(2009). A dance with the butterflies: A metamorphosis of teaching and learning through technology. Early Childhood Education Journal, 37, 229–236. doi:10.1007/s10643-009-0338-8

30.

Montague

(2007). Self-regulation and mathematics instruction. Learning Disabilities Research & Practice, 22, 75–83.

31.

Moreno

Park

(2010). Cognitive load theory: Historical development and relation to other theories. In Plass

J. L.

Moreno

Brunken

(Eds.), Cognitive load theory (pp. 9–28). Cambridge, MA: Cambridge University Press.

32.

National Center on Universal Design for Learning. (2010). UDL guidelines. Retrieved from http://www.udlcenter.org/aboutudl/udlguidelines

33.

Nord

Roey

Perkins

Lyons

Lemanski

Brown

Schuknecht

(2011). The nation’s report card: America’s high school graduates (NCES 2011-462, Report prepared for the U.S. Department of Education, National Center for Education Statistics). Washington, DC: U.S. Government Printing Office.

34.

Pisha

Coyne

(2001). Smart from the start. Remedial and Special Education, 22, 197–203.

35.

Plass

J. L.

Kalyuga

Leutner

(2010). Individual differences and cognitive load theory. In Plass

J. L.

Moreno

Brunken

(Eds.), Cognitive load theory (pp. 65–87). New York, NY: Cambridge University Press.

36.

Rao

Bryant

B. R.

(2014). A review of research on universal design educational models. Remedial and Special Education, 35, 153–166. doi:10.1177/0741932513518980

37.

Rappolt-Schlichtmann

Daley

S. G.

Lim

Lapinski

Robinson

K. H.

Johnson

(2013). Universal design for learning and elementary school science: Exploring the efficacy, use, and perceptions of a web-based science notebook. Journal of Educational Psychology, 104, 1210–1225. doi:10.1037/a00332171210

38.

Rappolt-Schlichtmann

Daley

S. G.

Rose.

L. T.

(Eds.). (2012). A research reader in universal design for learning. Cambridge, MA: Harvard Education Press.

39.

Reid

Strnadova

Cumming

(2013). Expanding horizons for students with dyslexia in the 21st century: Universal design and mobile technology. The Journal of Research in Special Educational Needs, 13, 175–181.

40.

Rose

D. H.

Gravel

J. W.

(2010). Universal design for learning. In Peterson

Baker

McGraw

(Eds.), International encyclopedia of education (pp. 119–124). Oxford, UK: Elsevier.

41.

Rose

D. H.

Hasselbring

T. S.

Stahl

Zabala

(2005). Assistive technology and universal design for learning: Two sides of the same coin. In Edyburn

Higgins

Boone

(Eds.), Handbook of special education technology research and practice (pp. 507–518). Whitefish Bay, WI: Knowledge by Design.

42.

Rose

D. H.

Meyer

(2002). Teaching every student in the digital age: Universal design for learning. Alexandria, VA: Association for Supervision and Curriculum Development.

43.

Rosenzweig

Krawec

Montague

(2011). Metacognitive strategy use of eighth-grade students with and without learning disabilities during mathematical problem solving: A think-aloud analysis. Journal of Learning Disabilities, 44, 508–520. doi:10.1177/0022219410378445

44.

Scanlon

Cass

Amtzis

Sideridis

(2009). Procedural facilitation of propositional knowledge in the content areas. Reading & Writing Quarterly, 25, 290–310. doi:10.1080/10573560903120854

45.

Scheuermann

A. M.

Deshler

D. D.

Schumaker

J. B.

(2009). The effects of the explicit inquiry routine on the performance of students with learning disabilities on one-variable equations. Learning Disability Quarterly, 32, 103–120.

46.

Stanford

Reeves

(2009). Making it happen: Using differentiated instruction, retrofit framework, and universal design for learning. Teaching Exceptional Children, 5, 1–9.

47.

Swanson

H. L.

Hoskyn

(2001). Instructing adolescents with learning disabilities: A component and composite analysis. Learning Disabilities Research & Practice, 16, 109–119.

48.

Sweller

(2010). Cognitive load theory: Recent theoretical advances. In Plass

J. L.

Moreno

Brunken

(Eds.), Cognitive load theory (pp. 29–47). New York, NY: Cambridge University Press.

49.

Therrien

W. J.

Taylor

J. C.

Hosp

J. L.

Kaldenberg

E. R.

Gorsh

(2011). Science instruction for students with learning disabilities: A meta-analysis. Learning Disabilities Research & Practice, 26, 188–203.

50.

Witzel

B. S.

Mercer

C. D.

Miller

M. D.

(2003). Teaching algebra to students with learning difficulties: An investigation of an explicit instruction model. Learning Disabilities Research & Practice, 18, 121–131. doi:10.1111/1540-5826.00068

51.

Zhang

(2005). Collaborative professional development model: Focusing on universal design for technology utilization. ERS Spectrum, 23, 31–38.