Developing Preservice Teachers’ Instructional Design Skills Through Case-Based Instruction: Examining the Impact of Discussion Format

Abstract

For more than 100 years, case-based instruction (CBI) has been an effective instructional method for building problem-solving skills in learners. While class discussion is often included as part of the CBI learning process, the impact on learning is unclear. Furthermore, little research has focused on how specific facilitation strategies influence the development of learners’ problem-solving skills. This study examined the impact of case discussion facilitation strategies on the development of preservice teachers’ problem-solving skills. Specifically, two discussion formats were compared: instructor-facilitated (class discussions guided by instructor-crafted prompts and an active facilitator) and instructor-supported (discussions guided by instructor-crafted prompts only). Results indicated that while preservice teachers’ problem-solving skills improved in both sections of the course, individuals in the instructor-facilitated section demonstrated significantly higher scores on course activities and designed instructional activities at higher cognitive levels compared with preservice teachers who participated in the instructor-supported discussions. Results underscore the importance of an active facilitator in CBI.

Keywords

case-based instruction discussion facilitator role teacher education instructional design

Background

Observations of expert teachers indicate they are cognizant of learners’ needs as they work toward established goals, observe and interpret environmental and learner cues, adapt plans to meet changing demands and constraints, and reflect on implemented efforts (Lachner, Jarodzka, & Nückles, 2016; Soslau, 2012). Given that many unknown components exist when designing instruction and many different solutions can potentially meet the needs of learners within the given constraints, the instructional design process represents a complex, ill-structured problem for educators. In short, expert teachers must be effective problem solvers (Goeze, Zottmann, Vogel, Fischer, & Schrader, 2014). For the purposes of this research, instructional design is defined as a problem-solving process (Ertmer & Stepich, 2005), and therefore, the term instructional design will be used to represent the problem solving required of teachers as they design and develop instructional solutions.

Literature Review

For many years, case-based instruction (CBI) has been considered an effective method for helping preservice teachers bridge the gap between theory and practice (Goeze et al., 2014). As an instructional method, CBI comprises realistic complex situations, presented in narrative form, in which learners are expected to analyze and solve problems through the application of principles, discussion of case components, and reflection on the entire process (P. L. Smith & Ragan, 2005; Stepich, Ertmer, & Lane, 2001). In teacher education, CBI has been used to address numerous learning goals (e.g., diversity and multicultural awareness; Andrews, 2002), in various formats (e.g., video cases; Goeze et al., 2014), and incorporating different peer and instructor interactions (e.g., cases to support thinking; Kim & Hannafin, 2011; small group discussions; Gravett, de Beer, Odendaal-Kroon, & Merseth, 2017).

Review of the literature on CBI, in general, and discussion facilitation, more specifically, reveals the potential of CBI to support the needs of preservice teachers (Goeze et al., 2014; Gravett et al., 2017). First, CBI affords preservice teachers the opportunity to participate in experiences comparable with the ones they will experience in their profession (e.g., developing a professional persona, transitioning into the profession, managing student relationships, and handling curricular matters; Gravett et al., 2017). Second, while participating in CBI, learners interact with complex problems in a nonthreatening environment, thus building professionally relevant experiences (Goeze et al., 2014). Third, as preservice teachers’ problem-solving strategies are often limited to their own learning experiences (Feiman-Nemser, 2001), CBI provides opportunities to share diverse perspectives as a means to build collaborative knowledge (Ertmer & Koehler, 2014). Fourth, participation in CBI requires self-directed learning behaviors (Savery, 2006); because full-time teaching professionals interact with, analyze, and solve ill-structured problems on a daily basis, CBI affords preservice teachers opportunities to develop these relevant problem-solving skills.

As the means to support learners’ development of problem-solving skills, CBI is typically facilitated through three phases: First, learners participate in a preparatory stage, which includes familiarization with the case through multiple readings, preparing an analysis, and completing guiding questions (Ertmer & Koehler, 2014). Second, learners interact with multiple viewpoints through discussions that may include a variety of instructional activities (e.g., role-playing; Stepich et al., 2001). Third, learners reflect on the entire case analysis process (Tawfik & Jonassen, 2013). While variation among facilitation practices exists, CBI is centered on two main components: (a) real-world problems that need solving and (b) discussions that facilitate the problem-solving process.

Although potentially helpful for developing learners’ instructional design skills, CBI can be challenging for both instructors and students. Specifically, CBI can be difficult as students experience uncertainty regarding how to manage the limited amount of instructional guidance provided by instructors (Goeze et al., 2014). Furthermore, given preservice teachers’ limited experiences with classroom realities, their exploration of a problematic classroom situation is often constrained (Goeze et al., 2014).

Because discussions are considered a central component of CBI (Ertmer & Koehler, 2014, 2015), structuring and designing them in ways that enable students to navigate the problem context is critical (Ertmer & Koehler, 2014). Given the challenging nature of CBI, learners could benefit from a variety of supports during this process. Commonly, support during CBI has been provided through at least two methods: instructor-facilitated strategies (e.g., debates, role-plays; Stepich et al., 2001) and self-guided methods (e.g., prompts, guidelines used during analysis or discussion; (Ertmer et al., 2008; Goeze et al., 2014).

A variety of facilitation strategies are described in the literature (Ertmer & Koehler, 2014; Goeze et al., 2014; Gravett et al., 2017). For example, Levin (1995) implemented CBI with 24 practicing and preservice teachers. Participants were divided into two groups. Individuals assigned to the experimental group wrote a case analysis, discussed the case, and wrote a second analysis. Individuals in the control group wrote a case analysis and, after a few days, wrote a second analysis without completing a case discussion. Both groups completed a second case analysis focused on similar topics to see if changes in thinking were retained. After analyzing case analyses and transcribed discussions, Levin concluded that using discussions during case facilitation improved the “quality, form, and content of thinking” (p. 72) and led to improved understanding in learners who participated in discussions compared with learners who did not.

Stepich et al. (2001) investigated how different facilitation methods (e.g., structured discussion, debate, reflective practitioner, role-play, discussion chain) influenced changes in students’ (n = 37) instructional design methods. Student responses were compared across five processes (e.g., conceptualization of the case, informational searches, attention to the relationship among factors, level of commitment to solutions, and consideration of implications) and labeled as expert-like or novice-like for each process. The authors concluded that the manner in which the instructors facilitated the discussion (e.g., the timing, type, and amount of questions being asked) had a strong influence on the way students approached their analyses of the issues in the cases.

Finally, just as discussion facilitation strategies can influence the learning that occurs while considering complex problems, a lack of facilitation can also affect learning (Ng & Tan, 2006). Ng and Tan investigated the ill-structured problem-solving processes of preservice teachers (n = 21) participating in an unmoderated, asynchronous online discussion. In the discussion, participants were asked to share a problem they had faced while completing a field experience and to collaboratively suggest problem-solving strategies. Based on an analysis of the discussion threads, results indicated that most students failed to fully understand the central problems before proposing solutions and suggested strategies that primarily focused on situations with which they were familiar. Without guidance, only a surface-level analysis was completed.

Although discussion plays a critical role in developing learners’ problem-solving skills in CBI, facilitating the direction of the conversation is challenging for instructors (Adejumo, Fakude, & Linda, 2014). Based on a meta-analysis of 94 studies focused on problem-centered methodologies, Leary, Walker, Shelton, and Fitt (2013) found that facilitator skill (i.e., ability to engage learners in content) was more impactful on student learning than facilitator content expertise. Given the importance of facilitation as a scaffolding tool, educators have used different methods (Tawfik & Kolodner, 2016). For example, Ertmer and Stepich (2002) and Stepich et al. (2001) suggested specific guidelines to direct the behaviors of novice facilitators (e.g., “Begin the discussion with a structure, but avoid rigid adherence to that structure” p. 63).

As an alternative approach to overcoming challenges associated with instructor facilitation, some educators have used self-guided prompts (e.g., prompts or questions used to guide students’ self-directed problem solving) to promote the development of problem-solving skills. When investigating the problem-finding abilities of novice instructional designers, Ertmer et al. (2008) found that novices, who were given analysis guidelines when examining a case, performed more like expert instructional designers than those who did not receive guidelines. Goeze et al. (2014) compared the impact of different discussion instructional supports (e.g., hyperlinks to multiple perspectives and conceptual knowledge) on preservice teachers’ learning processes and outcomes. Findings revealed that the instructional supports had a strong positive effect on the quality of preservice teachers’ perspective taking.

Research indicates that predetermined prompts can provide the guidance learners need to navigate complex problems (Ge, Planas, & Er, 2010). As such, these types of prompts potentially can be used to facilitate case discussions to achieve the same results as instructor-facilitated discussions. This is beneficial to educators, as using self-directed prompts can readily be implemented without the time and effort required by instructor-facilitated methods. However, using such an approach shifts responsibility from the instructor to the student, which could lead to challenges for novice problem solvers. Overall, while both instructor-facilitated and self-guided methods have been used to support learning during CBI, limited empirical research has compared these methods and the instructional equality of these methods is unclear.

Purpose

Although discussions are often used during case facilitation, little research has focused on how their structure affects student performance, particularly the development of instructional design skills (Ertmer & Koehler, 2014; Levin, 1995). Although some researchers suggest that interacting with a complex problem is enough to engage learners and promote mastery (Wirkala & Kuhn, 2011), others insist that the discussion is a significant factor in promoting student learning (Ertmer & Koehler, 2014, 2015; Goeze et al., 2014; Gravett et al., 2017). In other words, best practices for facilitating CBI discussions have yet to be established, the role these conversations play is unclear, and a determination of which methods induce the most learning is lacking. Therefore, the following questions guided this research:

Research Question 1: What is the impact of discussion format on the development of students’ instructional design skills in a case-based learning environment?

Research Question 2: How do students’ perceptions of lessons learned, from four narrative case studies, compare across discussion formats?

Specifically in this research, two different discussion formats were compared: instructor-facilitated (discussions guided by instructor-crafted prompts and an active facilitator) and instructor-supported (discussions guided by instructor-crafted prompts).

Method

Research Design

We used a mixed-methods research design in which both quantitative and qualitative data were integrated to more thoroughly answer our research questions (Creswell & Plano Clark, 2011). More specifically, a convergent parallel mixed-methods design was implemented. In this type of approach, both quantitative and qualitative data sources are collected independently, analyzed separately, and merged for understanding (Creswell & Plano Clark, 2011). Triangulation is a strength of this type of design, as consideration of “different but complementary data on the same topic” is promoted (Morse, 1991, p. 122). While variation exists in the implementation of convergent parallel mixed methods, this research followed a “parallel-databases” approach (cf. Feldon & Kafai, 2008) “where two parallel strands are conducted independently and are only brought together during interpretation” (Creswell & Plano Clark, 2011, p. 80). For the purposes of this study, students’ instructional design skills were measured through individual performances (pre/post-activity scores), group performances (case lesson plan scores), group instructional design techniques (content of case lesson plans), and student perspectives (lessons learned blog reflections). Interpretations of these data sources were merged during the discussion of results.

Through an analysis of these diverse data sources, differences in student performance—as they relate to different discussion facilitation strategies—can be compared. By collecting, analyzing, and interpreting multiple complementary data sources, triangulation of results and a deeper insight into the phenomenon were possible. Although the processes of instructional design and the development of problem-solving skills are complex (Stepich et al., 2001), this research design allowed for a more complete understanding of these processes in a teacher education context.

Measuring CBI Learning Outcomes

Measuring CBI learning outcomes is a difficult process (Yew & Yong, 2014). To overcome limitations of previous CBI research that focused primarily on student perceptions and satisfaction (Yadav & Barry, 2009), this research focused on measuring the instructional design processes adopted by participants. When assessing problem-solving performance, Jonassen (2011; based on Elliott’s, 1995, work) suggested including three elements in the evaluation process: (a) “students must construct a response or a product” (p. 364), (b) “assessment consists of direct observation or assessment of student behavior on tasks or on the product that they produced” (p. 364), and (c) the quality of the solution is considered through the use of a rubric. Heeding this advice, instructional design measures used in this study were based on students’ individual or group instructional design efforts and their reflections on the instructional design process. Scores were calculated using rubrics.

For the purposes of this study, instructional design skills were conceptualized as involving two major processes: problem representation (i.e., problem finding) and solution generation (e.g., problem solving; Ertmer & Stepich, 2005; Eseryel, Ge, Ifenthaler, & Law, 2011). Ertmer and Stepich (2005) conceptualized problem finding as “being able to articulate a clear and concise representation of the problem(s) in a particular situation” (p. 39). Ertmer and Stepich defined problem solving or generating solutions as “developing a clear and relevant solution plan that explicitly describes how the proposed solutions address the issues that have been identified in the previous step” (p. 41). Therefore, in this study, when problem solving was measured, the two components of problem finding and generating solutions were used to conceptualize the process and to guide the development of rubrics.

Setting

Data for this study were collected from an introductory educational technology course for preservice teachers at a large Midwestern university. This course focused on the effective integration of technology in educational settings, while emphasizing strong instructional design principles. Students enrolled in the course were required to meet twice a week: (a) on Mondays, for a large-group meeting (approximately 65 students per section) lasting for 1 hr, and (b) on Wednesdays, for a lab session (approximately, 15-20 students per lab) lasting for 2 hr. The Monday portion of the course was divided into two sections, and eight lab sections were offered during the semester the study was conducted. Four lab sections were assigned to each Monday meeting. The course used a “flipped” classroom approach (Herreid & Schiller, 2013) in which students were required to review content and instructional videos online via a course management system, prior to coming to class. Students were also asked to complete weekly review questions prior to the Monday class meetings to encourage viewing of the instructional materials. As many of the activities during the Monday meetings required students to search for additional information and to develop instructional materials, students were instructed to bring their own electronic devices (e.g., computers, tablets, smartphones) to use while completing case activities. Access to technology appeared equitable across sections, with each group having at least one device available at all times.

Students sat at round tables, designed for collaborative activities. At each table, six students could be seated. The same instructor taught both sections of the Monday portion of the course, and six teaching assistants (TAs) taught the eight lab sections. Two TAs taught two lab sections. The instructor was an advanced graduate student, who had previously taught the lab portion of the course several times. The TAs were all graduate students in Learning Design and Technology and had been in the program from 1 to 5 years, with varying levels of experience teaching the lab sections (1 to 5+ times). To maintain consistency across all lab sections, all TAs followed the same schedule during the 16-week course, and all students completed the same assignments. In addition, all TAs were trained prior to the start of the semester; more details are included in the discussion of reliability and validity.

Participants

Participants included 125 undergraduate students enrolled during the spring of 2014. Demographic information is included in Table 1. Comparing student demographics across course sections reveals similar compositions: Both sections had a high percentage of freshman and sophomore students, and in both sections, there were many more female than male students. In addition, the composition of students’ majors was similar across course sections.

Table 1.

Participant Demographic Information.

Characteristic	Instructor-facilitated(Section 1)	Instructor-supported(Section 2)	Total
Level
Freshman	33	40	73
Sophomore	15	13	28
Junior	9	8	17
Senior	2	5	7
Gender
Female	49	43	92
Male	10	23	33
Major
Elementary/early childhood education	25	21	46
Special education	4	4	8
Secondary education/content specific	26	36	62
Noneducation	4	5	9
Median age	19	19

Case Analysis Process

As part of the Monday portion of the course, students were required to complete four case studies throughout the semester. Each case was implemented using the same 3-week process: During Week 1, students participated in a discussion. Depending on course section, the discussion format varied. This variation is discussed in more detail later. During Week 2, students had the entire class period to work with group members to design and develop case lesson plans. During Week 3, students participated in closure activities and a final case debriefing. Following the case wrap-up, students posted a final reflection to their blogs. This process started during Week 4 of the semester and was repeated every 3 weeks.

Data Sources: Collection and Analysis

To address our research questions, data were collected from several sources: pre-activity/post-activity scores, case lesson plans, and lessons learned blog reflections. By using multiple data sources, the research questions were examined from multiple angles. Each data source is discussed in detail, including data analysis methods for interpreting data from each source.

Pre-activity/post-activity

During the third and 15th weeks of the semester, students were asked to complete a case analysis activity. Participation in the activity was voluntary; students were awarded bonus points only if they completed both the pre- and post-activity. In the activity, students were presented an instructional design scenario. Students were told to assume the role of a sixth-grade social studies teacher who was required to create a lesson focused on improving writing skills to address the principal’s school improvement plan. To assist students’ decision making, the case included environmental characteristics (e.g., available technology), student characteristics (e.g., past standardized test scores), and content requirements (e.g., sixth-grade social studies standards). After reading the scenario, students were asked to answer 12 questions, focused on the process they would follow to design and develop instruction that met the requirements of the writing initiative. Students were given a printed copy of the scenario and entered responses via an online data entry system. The same activity was used for both the pre- and post-activity.

In total, there were 111 usable submissions. Student responses were analyzed using a rubric. To strengthen the scoring process, reviewers were blinded from knowing whether they were examining a response from a pre- or post-activity. Based on the rubric, each student received a problem-finding score and a generating-solutions score for his or her pre- and post-activity work. Problem-finding scores were derived from student descriptions of the following: (a) stakeholders affected by the proposed instruction, (b) steps they would follow during the design and development process, (c) characteristics of the target audience, (d) characteristics of the learning environment, (e) instructional goal(s), and (f) informational sources that would be helpful during the design of their lessons. Generating-solutions scores were calculated using a similar method. Student descriptions were scored on different aspects of the proposed instruction: (a) alignment with student characteristics, (b) alignment with environmental characteristics, (c) alignment with stakeholder needs, (d) consideration of social studies standards and writing initiative goals, (e) proposed technology integration, (f) proposed evaluation methods, and (g) motivational value of the instruction. For each item on the rubric, students received 1 point for an appropriate response and 0 points for missing or inappropriate responses.

Problem-finding and generating-solutions scores, from both the pre- and post-activity, were used to make multiple comparisons. First, individual growth from pre-activity to post-activity was considered. Second, comparisons between sections (impact of discussion style) were made.

Case discussions

As students analyzed the case narratives, they participated in discussions with their group members to design and develop lesson plans. Two groups of two to three students were seated at each round table during the Monday portion of the course. Therefore, the class discussions were at a table level versus being confined to a group level. For each of the four cases, the discussion focused on a different aspect of the instructional design process: During Case 1, learners were encouraged to focus their analyses on understanding their learners and instructional contexts. Case 2 prompted learners to think about issues related to design and development, while Case 3 pushed students to consider implementation issues and methods. Finally, for Case 4, the discussion topics centered on evaluating student learning. Although the discussion focus and prompts were the same across sections of the course, discussion implementation methods differed across sections.

Instructor-facilitated section

In Section 1 of the course, students participated in an instructor-facilitated (IF) discussion. In this section, during the first 50-min class meeting in each case assignment cycle, students were asked to complete specific activities related to the discussion prompts. These activities included role-playing, synthesizing, and sharing. For example, during Case 3, students were asked to design training materials for teachers. As part of the discussion in their small groups, they were prompted to consider what was important to note about the realities of teachers’ jobs and schedules. After having a chance to discuss with their group, the facilitator asked students to share highlights from their discussions with the entire course section.¹ During the second course meeting during a case cycle, students had the opportunity to work on their case assignments with their group members. The instructor and TAs would interact with students and answer any questions during this time.

Instructor-supported section

In Section 2 of the course, students completed instructor-supported (IS) discussions, using handouts with the discussion prompts as they completed their analyses. Although the instructor did not facilitate the use of the prompts, she encouraged students to consider the prompts during their self-guided discussions when introducing each case. The discussion prompts used in both sections focused on the same topics (e.g., science, technology, engineering, and math [STEM] activities for at-risk students, teaching English language learners). During the first two course meetings of a case cycle, students in this course section used class time to work with their group members in whatever way they deemed useful (see Note 1).

In both sections, when students worked on case assignments during class time, the instructor and TAs walked around the room to monitor student work and answer questions. In most cases, this interaction was initiated by the students. However, in both course sections, the instructor made a point to talk with each group during each class period.

Case lesson plans

Case studies were presented in narrative format and focused on complex problems that are representative of the teaching profession. For each case, students were assigned to work with one or two peers from their lab sections. Group composition changed for each case. The case assignments required students to develop a solution in lesson plan format. Each case focused on a different topic:

Case 1: Students worked with their group members to develop a lesson for Colombian middle school students learning English.

Case 2: Students developed an afterschool STEM activity for at-risk students. Students could select the age group for whom they designed.

Case 3: Students developed training on cyber security for 275 teachers in a small school corporation.

Case 4: Students were required to incorporate an educational game into a personal finance lesson for high school students.

In each case assignment, students were asked to include specific components in their lesson plans: overview of lesson, description of learners and intended learning outcomes, lesson goal and learning objectives, teaching standards, required materials, procedures, assessment, and references and reference materials. Although students were not required to use a specific organizational or formatting method for presenting these elements, each element was expected to be fully described so that someone else could easily and successfully implement the lesson.

Using Klein’s (1991) work, a rubric was developed to score each case. The rubric detailed nine aspects of the required lesson plan: topic of lesson, intended learning outcomes, target audience, instructional objectives, instructional procedures, content standards, motivational strategies, evaluation methods, and uses of technology. Lesson plans received two scores: (a) problem finding (topic of lesson, intended learning outcomes, target audience, instructional objectives, and standards) and (b) generating solutions (instructional procedures, motivational strategies, evaluation methods, and uses of technology). Each case received a problem-finding score (0-9 points) and a generating-solutions score (0-15 points).

In addition to case scores, the content of the lessons was examined. Specifically, the proposed instructional activities were analyzed using Bloom’s Taxonomy to determine the cognitive levels at which members of the targeted audience were expected to perform. Bloom’s Taxonomy “is a framework for classifying statements of what we expect or intend students to learn as a result of instruction” (Krathwohl, 2002, p. 212). In its revised form, Bloom’s Taxonomy is a hierarchy of six levels of cognitive processing, representing differences in instructional complexity: remembering (“Retrieving relevant knowledge from long-term memory”), understanding (“Determining the meaning of instructional messages, including oral, written, and graphic communication”), applying (“Carrying out or using a procedure in a given situation”), analyzing (“Breaking material into its constituent parts and detecting how the parts relate to one another and to an overall structure or purpose”), evaluating (“Making judgments based on criteria and standards”), and creating (“Putting elements together to form a novel, coherent whole or make an original product”; Krathwohl, 2002, p. 215).

Bloom’s Taxonomy has been used as an analysis tool to consider critical thinking at many different instructional levels and in many different instructional contexts. A few examples include program evaluation (McNeil, 2011), assessment item development (Dagostino, Carifio, Bauer, Zhao, & Hashim, 2014), and lesson plan evaluation (Sultana & Klecker, 1999). For the purposes of this investigation, the activities described in each lesson plan were analyzed to determine the highest level of Bloom’s Taxonomy addressed by the lesson. Using this approach provided an opportunity to examine the instructional solutions designed by the groups to meet the needs of learners and requirements of the case.

Lessons learned blog reflections

As a course activity, students were required to create website portfolios to showcase their work. One page of the portfolio was dedicated to a blog space where students were required to add posts for various assignments throughout the semester. At the end of every case, students completed reflections of their case analysis experiences in which they described the biggest takeaway gained from completing the case. Ideas in the reflections blog were initially coded as being an example of problem finding, generating solutions, both, or neither. Categories of lessons learned were created for each of these areas to capture students’ understanding. Finally, to compare differences across course sections, frequencies for each category of lessons learned were calculated.

Reliability and Validity

Many methods were used to reduce threats to validity and reliability. First, to increase internal validity, the case materials, discussion and activity prompts, and scoring rubrics were developed based on previous research and reviewed by a content expert (Creswell & Plano Clark, 2011). For the qualitative analysis, deductive methods were used that informed the development of the coding protocol. During the semester prior to implementation, the procedures and instruments were piloted. Following these activities, the methods used in this study were revised and strengthened to overcome weaknesses identified during the pilot implementation.

During the semester, six TAs (three per course section) were assigned to teach the eight labs offered. Primarily, the TAs’ responsibilities were to teach their labs. During lab time, students focused on the design and development of digital instructional materials—assignments and projects separate from the casework required of the Monday meetings. Therefore, although nothing prevented students from discussing case elements during lab time, the occurrence of such activities would have been limited as the labs covered different learning goals and objectives than the Monday portion of the course. In addition, TAs monitored and facilitated aspects of the Monday course meetings. For instance, when students had class time to discuss or create case lesson plans, TAs would monitor student collaboration and answer questions. Prior to the start of the semester, all TAs were trained on how to assist students as they completed their case analyses. TAs were instructed to help students think through case issues using a “reflective toss” approach rather than to just provide answers. In general, a reflective toss strategy takes the form of a specific question that is used to probe a student’s initial response, which then allows facilitators to build on students’ current thinking about the issue (Hmelo-Silver & Barrows, 2006). For instance, during a typical discussion, a student might ask how to complete part of the instructional design process (e.g., How would you assess students in an afterschool program?). Instead of answering the student’s question by providing potential solutions, the instructor would instead ask follow-up questions (e.g., How is student learning typically assessed? What makes the afterschool format different for assessing learning from a traditional learning environment?). Finally, weekly TA meetings were held to clarify any confusion about the facilitation process and to address any concerns.

Reliability measures were also taken to confirm accuracy of the methods. All case lesson plans were scored by the lead researcher. For each case, 10 lesson plans were randomly selected and scored by a second coder. Prior to scoring, the second coder was trained on the case lesson plan scoring process. A two-way mixed, consistency average-measures intra-class correlation (ICC) was used to confirm the consistency of the scorers’ techniques (Hallgren, 2012). Interrater reliability was strong for both the problem-finding scores (ICC = .97) and generating-solutions scores (ICC = .94; Cicchetti, 1994). A similar process was used for the pre- and post-activity scores. The lead researcher scored all pre- and post-activity submissions, and a second coder was trained for reliability purposes. For both the pre- and the post-activity, 10 submissions were randomly selected and scored by the second coder. Again, reliability results indicated excellent agreement between raters for pre-activity problem-finding scores (ICC = .97) and generating-solutions scores (ICC = .94) and post-activity problem-finding scores (ICC = .99) and generating-solutions scores (ICC = .98). Results from both case lesson plan scores and pre- and post-activity scores indicate that coders exhibited a high degree of agreement in scoring student responses, and the lead researcher was consistent in her coding methods (Hallgren, 2012).

Finally, findings were validated through the use of multiple data sources, allowing for rich descriptions of the CBI experiences and instructional design processes used by the preservice teachers (Creswell & Plano Clark, 2011). By using multiple reliability and validity procedures, a more rigorous investigation was possible.

Findings

To explore differences in the development of instructional design skills in preservice teachers participating in IF and IS discussions during CBI, case scores and pre-activity/post-activity scores were analyzed. For each case, a group problem-finding and a generating-solutions score were calculated. Similarly, for the pre- and post-activity, students’ individual problem-finding scores and generating-solutions scores were compared. While it was not possible to randomly assign students to a course section, it is assumed that the course sections were comparable as they had similar course compositions (i.e., large percentage of underclassmen and female students, median age of 19). In addition, comparing pre-activity scores between course sections revealed no statistically significant differences between problem-finding scores, t(109) = 1.67, p = .098, or generating-solutions scores, t(109) = –.122, p = .903. This suggests that students did not differ on their initial instructional design skills across course sections.

Comparing Pre-Activity and Post-Activity Scores Across Course Sections

To answer our first research question regarding the impact of discussion format on the development of students’ instructional design skills, we compared students’ responses, across course sections, on a pre- and post-activity. In addition, we compared group case scores and lesson plan content across course sections. The results from these different comparisons are included in the next several sections.

During the third and 15th weeks of the semester, students completed the pre-activity and post-activity, respectively. Students were asked to individually work through an instructional scenario and answer several questions. Students’ responses were scored using a rubric. Each pre-activity/post-activity was broken into two scores: (a) problem finding and (b) generating solutions. In total, 111 students completed both the pre-activity and post-activity (n_IF = 52 and n_IS = 59). Descriptive statistics for the test scores are included in Table 2. Although the IF and IS data have similar ranges, for both problem-finding and generating-solutions scores, students’ post-activity scores in the IF group were higher.

Table 2.

Frequencies, Means, and Standard Deviations for Pre-Activity/Post-Activity Scores Across Course Sections.

Variable	n	M	SD	Minimum	Maximum	df	t	p	R
Instructor-facilitated
Pre-activity PF	52	10.29	3.10	1	18	51	−6.58	<.001*	.46
Post-activity PF	52	13.48	3.24	5	23	51	−6.58	<.001*	.46
Pre-activity GS	52	10.83	3.10	6	18	51	−4.58	<.001*	.29
Post-activity GS	52	13.52	3.50	4	20	51	−4.58	<.001*	.29
Instructor-supported
Pre-activity PF	59	9.42	2.36	5	15	58	−4.83	<.001*	.29
Post-activity PF	59	11.46	3.20	5	19	58	−4.83	<.001*	.29
Pre-activity GS	59	10.90	3.10	5	19	58	−1.98	.052	.06
Post-activity GS	59	11.92	3.68	6	19	58	−1.98	.052	.06

Note. PF = problem finding; GS = generating solutions.

Indicates significant differences between pre- and post-activity scores.

Using pre-activity scores as a covariate to control for differences in students’ initial instructional design skills, two ANCOVAs were completed to investigate differences in (a) problem-finding scores and (b) generating-solutions scores between the two sections of the course. When controlling for problem-finding pre-activity scores, a main effect of course section, F(1, 108) = 8.101, p = .005, $η_{p}^{2}$ = .070, was observed. When controlling for generation-solutions pre-activity scores, a main effect of course section, F(1, 108) = 6.004, p = .016, $η_{p}^{2}$ = .053, was also observed. These results indicate that, for problem-finding scores, 13.7% of the variance was attributed to pre-activity scores, while 7.0% of the variance related to discussion facilitation style. In terms of generating-solutions scores, results indicate that 6.7% of the variance was attributed to pre-activity scores, while 5.3% of the variance was attributed to facilitation style. Furthermore, students participating in the IF section of the course demonstrated significantly higher problem-finding and generating-solutions scores on the post-activity than students participating in the IS section.

To further explore the difference between the two facilitation styles, the individual student improvements in each section were compared using paired-samples t tests. In the IF section, significant differences were found between students’ pre- and post-activity problem-finding scores and between students’ pre- and post-activity generating-solutions scores. In the IS section, a significant increase was observed in the problem-finding scores only. See Table 2 for full results.

Group Case Scores

For each case, group lesson plans received two different scores: (a) problem finding (possible score range = 0-9) and (b) generating solutions (possible score range = 0-15). To explore differences in scores resulting from the facilitation methods, problem-finding and generating-solutions scores were compared between course sections for each case. For Cases 1 and 2, 46 groups created lesson plans (n_IF = 23, n_IS = 23). As some students dropped the course or disappeared as the semester continued, the number of groups decreased for Cases 3 and 4 (n_IF = 20, n_IS = 23). Note that n indicates the number of groups, as group members all received the same scores. When comparing case scores, nonparametric methods were selected to overcome normality concerns with the data set. Specifically, Mann–Whitney U tests were selected, as the comparisons made for each case included a dependent variable (i.e., course section) and an independent variable (i.e., case scores). A p value of .05 was selected as a standard level for determining significance. Results from each case comparison are included in Table 3.

Table 3.

Frequencies, Means, Standard Deviations, and Comparison Results for Group Lesson Plan Scores.

	n	M	Mdn	SD	U	p	r
Case 1
IF problem finding	23	6.22	6	1.731	142.5	.006*	−.40
IS problem finding	23	4.78	5	1.242	142.5	.006*	−.40
IF generating solutions	23	10.43	11	2.233	192.5	.109	—
IS generating solutions	23	9.3	9	2.305	192.5	.109	—
Case 2
IF problem finding	23	6.3	7	1.717	204	.177	—
IS problem finding	23	5.65	6	1.849	204	.177	—
IF generating solutions	23	10.52	11	2.064	182.5	.069	—
IS generating solutions	23	9.13	9	2.651	182.5	.069	—
Case 3
IF problem finding	20	5.05	5	1.504	118	.005*	−.42
IS problem finding	23	3.70	4	1.608	118	.005*	−.42
IF generating solutions	20	9.50	9	2.090	146	.039*	−.31
IS generating solutions	23	8	7	2.594	146	.039*	−.31
Case 4
IF problem finding	20	7.15	7.5	1.725	89	.001*	−.53
IS problem finding	23	4.96	5	1.770	89	.001*	−.53
IF generating solutions	20	12.60	12.5	1.501	71.5	<.001*	−.60
IS generating solutions	23	9.96	10	2.056	71.5	<.001*	−.60

Note. IF = instructor-facilitated; IS = instructor-supported.

Indicates significant differences between course sections.

For all four cases, median and average IF group scores for problem finding and generating solutions were higher than median and average IS group scores. The biggest differences between group scores were in students’ performances on Case 4. In Cases 1, 3, and 4, students in the IF section scored significantly higher on problem finding than students participating in the IS section. In addition, on Cases 3 and 4, students in the IF section scored significantly higher on generating solutions than students participating in the IS section. There were no significant differences between other scores. Results indicate moderate effect sizes for Cases 1 and 3 scores. A large effect size was observed for Case 4, with over 25% of the variance between group scores explained.

Case 2 was the only case in which no significant differences existed between either problem-finding or generating-solutions scores across course sections. In general, Case 2 focused on environments, learners, and contexts that were most similar to students’ previous learning experiences, perhaps allowing them to problem solve more easily. Looking at case scores across the semester, we did not observe a steady improvement across the cases. For instance, although students made small gains from Case 1 to Case 2 in problem-finding and generating-solutions scores, the average case scores, in both course sections, dropped considerably for Case 3. During Case 3, students were asked to design instructional activities for teachers, possibly the least familiar context for which they were asked to design. Effect sizes were calculated for all significant comparisons made between case scores (Field, 2005; Rosenthal, 1991) and are included in Table 3. Moderate effect sizes were present for Case 1 problem-finding scores (r = –.40) and Case 3 problem-finding (r = –.42) and generating-solutions (r = –.31) scores. Finally, large effect sizes were present for Case 4 problem-finding (r = –.53) and generating-solutions (r = –.60) scores.

Lesson Plan Content

The instructional activities described in the lesson plans developed by the preservice teachers for each case were compared with Bloom’s Taxonomy. Based on the described activities in a lesson, we recorded the highest level of Bloom’s Taxonomy addressed. Although Bloom’s Taxonomy is composed of six levels, these were grouped to simplify the analysis. That is, the first two levels of Bloom’s Taxonomy were grouped (remembering and understanding); the middle two levels (applying and analyzing); and the last two levels (evaluating and creating). Table 4 provides counts for the Bloom’s levels observed in lesson plans across cases and course sections. With the exception of Case 3, more groups in the IF section of the course designed lessons that included more complex cognitive activities. In contrast, across all four cases, in the IS section of the course, the lowest levels (1-2) of Bloom’s Taxonomy were most frequently addressed by lesson plan activities, whereas in the IF group, the highest levels (5-6) were most frequently addressed by activities.²

Table 4.

Counts of Lesson Activities at Each Level of Bloom’s Taxonomy by Course Section.

	Observed Bloom’s taxonomy levels
	Levels 1 and 2	Levels 3 and 4	Levels 5 and 6
Case 1
Instructor-facilitated	9	8	6
Instructor-supported	16	6	1
Case 2
Instructor-facilitated	4	7	12
Instructor-supported	14	3	6
Case 3
Instructor-facilitated	7	6	7
Instructor-supported	10	10	3
Case 4
Instructor-facilitated	1	8	11
Instructor-supported	8	10	5
Total instructor-facilitated groups by level	21	29	36
Total instructor-supported groups by level	48	29	15
All groups by level	69	58	51

For Cases 1, 2, and 4, there was a significant association between course section and the level of Bloom’s Taxonomy observed in the lesson activities. Chi-square analyses results are shown in Table 5. For Cases 1, 2, and 4, this suggests that groups assigned to the IF section of the course were more likely to develop lessons that included activities at the higher levels of Bloom’s Taxonomy than were groups assigned to the IS section of the course. To consider the effect size of the association between course section and Bloom’s Taxonomy levels, Cramer’s V values were calculated for each case (Field, 2005). Moderate effect sizes were present for Cases 1, 2, and 4.

Table 5.

Results of Chi-Square Comparison of Case Activities.

	χ²	df	n	p	Ø c
Case 1	6.24	2	46	.044*	.36
Case 2	9.57	2	46	.008*	.45
Case 3	2.98	2	43	.225	.26
Case 4	8.52	2	43	.014*	.42

indicates significant association between course section and level of Bloom’s Taxonomy observed

Lessons Learned Blog Reflections

To address our second research question regarding differences in students’ perceptions of lessons learned from the four case studies, we compared students’ blog postings across course sections. In their blog postings, students shared lessons learned after completing each case assignment. These reflections were classified based on whether they related to problem finding, generating solutions, or other topics. Table 6 provides an overview of the number of students sharing lessons learned related to each category by course section. In some instances, students shared lessons learned in multiple categories. Therefore, the number of lessons learned surpasses the total number of students in some instances.

Table 6.

Overview of Lessons Learned Content.

	Case 1		Case 2		Case 3		Case 4		Total
	IF	IS	IF	IS	IF	IS	IF	IS	IF	IS
n	50	53	53	56	54	55	55	62	—	—
PF	11	8	6	2	8	5	1	1	26	16
GS	19	14	21	18	22	23	19	24	81	79
Other	18	21	19	23	18	25	14	26	69	95

Note. IF = instructor-facilitated; IS = instructor-supported; PF = problem finding; GS = generating solutions.

Problem-finding lessons learned

In some instances, students shared lessons learned related to problem finding. While this was the least common classification of lessons learned, in three of the four cases, more students in the IF section of the course mentioned problem-finding lessons learned. Table 7 provides counts for problem-finding themes across course sections. Not reflected in Table 7 is the fact that in the IF section of the course, three students shared two lessons learned during Case 1, and four students shared two lessons learned during Case 3. Students discussed seven topics as they shared their problem-finding lessons learned. The most commonly discussed problem-finding lesson learned related to students’ realization of the importance of understanding the characteristics and needs of learners. Students also discussed the importance of recognizing environmental, content (e.g., standards), and stakeholder characteristics/needs when planning their lessons. Finally, a few students considered the usefulness of personal research, the implications of constraints, and the importance of understanding their future roles as teachers.

Table 7.

Counts for Problem-Finding Topics by Case and Course Section.

	Case 1		Case 2		Case 3		Case 4		Totals
	IF	IS	IF	IS	IF	IS	IF	IS	IF	IS
Student characteristics	6	6	2	1	6	4	1	1	15	12
Environmental characteristics	2	1	2	0	0	0	0	0	4	1
Content characteristics	1	1	1	1	0	1	0	0	2	3
Usefulness of personal research	2	0	0	0	1	0	0	0	3	0
Recognition of constraints	2	1	0	0	3	0	0	0	5	1
Teacher role	1	0	0	0	2	0	0	0	3	0
Stakeholder needs	0	0	1	0	0	0	0	0	1	0

Note. IF = instructor-facilitated; IS = instructor-supported.

Comparing the problem-finding lessons learned across the two groups reveals that themes were similar.³ However, while both groups consistently discussed central problem-finding topics (i.e., learner, environmental, and content characteristics), more students in the IF section discussed diverse problem-finding topics (i.e., recognizing constraints, usefulness of personal research).

Generating-solutions lessons learned

In their blog reflections, students also discussed lessons learned related to generating solutions. These ideas related to the realization that lesson designs needed to meet the needs of the learners, environment, and content while working within the constraints of the case. In some reflections, these were directly related to the case they completed, whereas in other instances students considered how they would approach lesson planning in the future. For example, during Case 3, students were asked to create lessons for their colleagues. In their reflections, students shared how this affected their design approach:

I think the biggest takeaway I gained from completing this case would be that if I were ever placed into this situation where I had to teach other teachers, I would really have to communicate and collaborate with colleagues in order to make the lesson more interactive and productive.

The biggest “takeaway” that I gained from this case would probably be the fact that by making lesson plans, you do not only want to teach about things that you yourself already know. You need to do further research to create good lesson plans. You as a teacher also should learn through them. At first, I was confused on how I was to make a lesson plan for other teachers who know just as much about plagiarism as myself, but once I did some research and dug up more information I was surprised on the things that I did not know.

Across sections, the generating-solutions reflections were very similar, with comparable numbers and ideas.

Other lessons learned

The final category of lessons learned related to ideas that went beyond the specific instructional design process. Most commonly, students discussed learning something about how to use technology (e.g., “The biggest takeaway for me would be how to use Lino!”) or how their knowledge of instructional technology had increased (e.g., “The biggest takeaway I got from this case was that there are some really great games that can be very beneficial in the classroom.”). In addition, many students shared experiencing an increase in content knowledge related to the case topics. For instance, many students referenced learning more about poverty and financial management as their biggest takeaways for Case 4 and developing a greater understanding of cyber topics from Case 3. In several reflections, students expressed a new appreciation for the demands of teaching (e.g., “The best takeaway from this project is the appreciation for teachers and how hard they really have it in their classrooms. I never thought coming up with ideas and executing them would be so difficult” and “This case helped demonstrate the necessity for teachers to continue to learn. Even after teachers have been in the classroom, there is still a need for their education to continue, especially on technological and social topics, like the Internet.”). Students also acknowledged the challenges of lesson planning (e.g., “The most important thing that I took away from this case is that a lot more goes into a quality lesson plan than I previously thought. I didn’t realize how important it was to do background research in order to develop a useful and truly educational lesson plan, so I will definitely keep that in mind going forward.”).

Finally, the case analysis and debriefing process also yielded some important lessons for students. During the Case 2 debriefing, groups had to exchange their lesson plans with another group. Through this activity, students were required to teach a lesson plan they had never seen before. Not surprisingly, many students shared that their biggest takeaway from Case 2 was the importance of writing clear lesson plans:

The biggest “takeaway” of the case was that we have to be extremely specific in order for another person to use our lesson plan if we just give it to them. My group looked at another group’s lesson plan and had a hard time trying to teach it because [their] instructions lacked.

In other instances, many individuals reflected that they benefited from collaborating with their peers and developing strong group-work skills (e.g., “Sometimes the best ideas are not your own, or sometimes some of the suggestions would have been ideas I would have never thought of.”) and described how completing the lesson planning process gave them confidence (e.g., “The biggest takeaway that I gained by completing this case was knowing that I could create a lesson plan. As crazy as this sounds, the task seemed very daunting to me when I first started my educational classes here.”). Interestingly, compared with students in the IF section, students in the IS section reported many more lessons learned related to topics beyond the instructional design process.

Students in both sections of the course appeared to find the case process beneficial, as they shared many meaningful takeaways. Interestingly, more students in the IF section shared lessons specifically related to the instructional design process, while more students in the IS section shared lessons learned related to more general topics. As students discussed their lessons learned, they shared how the case had influenced them in many ways beyond instructional design. This underscores how powerful cases can be in promoting many forms of learning. Furthermore, comparing differences among lessons learned suggests that the facilitated discussions served an impactful role in helping students focus on important aspects of the instructional design process.

Summary of Findings

When considering the first research question, examination of several data sources allowed us to compare instructional design skill development in preservice teachers participating in instructor-facilitated and instructor-supported CBI, resulting in many key findings. First, at the individual level, students who participated in the IF discussions developed significantly stronger instructional design skills. Second, when developing lesson plans as a group, IF students earned significantly higher problem-finding scores on three of the four cases and also earned higher generating-solutions scores in two of the four cases. Third, examining the actual content of groups’ lesson plans revealed that students in the IF section developed more lesson activities that incorporated higher levels of Bloom’s taxonomy. Finally, students in both sections of the course reported learning many important lessons from participating in the case analysis and discussion process. Categorizing these lessons indicated that students in the IF section typically discussed more lessons learned about the instructional design process, while students in the IS section focused more on broader topics.

Discussion and Implications

Results indicate that using problem-centered methods, like CBI, to prepare preservice teachers for the teaching profession can be worthwhile. Most participants in the study increased their instructional design skills, as evidenced by improvements in scores from the pre- to post-activity. Particularly encouraging, students in both sections of the course demonstrated statistically significant increases in problem-finding scores. Beginning teachers experience several challenges as they design instruction for the first time, and many of these obstacles are the result of weak or missing problem-finding skills: writing unclear learning objectives (Jones, Jones, & Vermette, 2011); relying heavily on preconceptions of teaching, learning, and students (Hammerness et al., 2005); and struggling with identifying students’ current knowledge levels (Rusznyak & Walton, 2011). In addition, other common instructional design difficulties (e.g., assessment mismatched with objectives, knowing where to begin a lesson, oversimplification of teaching and learning; Jones et al., 2011; Rusznyak & Walton, 2011) may also stem from faulty or missing problem-finding skills.

If preservice teachers can develop stronger problem-finding skills while they are learning how to design instruction, then, possibly, some of these issues can be minimized or eliminated altogether. In general, part of being an effective problem solver involves developing a full understanding of the problem before attempting to design a solution (Ng & Tan, 2006; Stepich & Ertmer, 2009). Strengthening problem-finding skills can lead to stronger solutions. Results from this investigation demonstrated a positive relationship between a student’s problem-finding skills and his or her ability to generate solutions. As such, if students can improve their problem-finding skills through the use of CBI methods, they may also improve their abilities to develop effective instructional solutions.

Another promising result from this study is the observed evolution in preservice teachers’ thinking about instructional design problems. As the semester progressed, the preservice teachers began to consider connections among case elements, solutions, and constraints more deeply and reported many important lessons learned from completing the case analyses. Developing mental processes that mirror expert thinking is important for developing expertise (Ertmer & Stepich, 2005). By using CBI to promote this type of thinking in preservice teachers, students might be expected to index these experiences and then draw upon them at a later time (Stepich & Ertmer, 2009). Not only can CBI help preservice teachers develop expert thinking but it can also enable them “to put what they know into action” (Hammerness et al., 2005, p. 359) by developing plausible case solutions (Stepich & Ertmer, 2009). Moreover, using a CBI approach, the learners in this study had the opportunity to explore and struggle with many real complexities involved with teaching. Using authentic approaches, such as the one in this study, can help avoid the oversimplification of many teacher preparation efforts (Hammerness et al., 2005; Rusznyak & Walton, 2011).

Clearly, the participants in this study did not become instructional design experts by the end of the course. As exhibited by fluctuating case scores, participants showed moments of developing expertise instead of a steady progression of skills. This is similar to Stepich et al.’s (2001) finding when working with novice instructional designers: “In general, students showed both novice-like and expert-like responses throughout the semester” (p. 57). Whereas scores improved from Case 1 to Case 2, Case 3 scores dropped when students were asked to develop instruction for teachers, learners with whom the participants likely were less familiar. Learning to solve complex problems is a challenging process that takes time to develop (Stepich et al., 2001). Although the preservice teachers in this study were not experts after participating in the four CBI learning experiences, they did strengthen their instructional design skills, nudging forward along the novice–expert continuum. These findings are promising for shaping future preservice teacher preparation experiences.

Although there is some uncertainty about the role that discussions and peer interaction play in problem-centered methodologies (Wirkala & Kuhn, 2011), the results of this study support other research findings that underscore the role of discussions in CBI (Ertmer & Koehler, 2014; Goeze et al., 2014; Gravett et al., 2017; Levin, 1995). Specifically, in this study, we compared students’ experiences in two sections of an introductory educational technology course. Although both groups of students participated in CBI, one group participated in instructor-facilitated discussions, while the other group directed their own discussions in an instructor-supported environment. Results suggest that the discussions played an important role in the learning that occurred during CBI. In addition, the structure of the discussions appeared to affect the overall effectiveness of the method: Instructor-facilitated discussions were more impactful in promoting student learning during CBI than student-directed discussions, which were guided by prompts only.

In this study, students completed a pre- and post-activity, which asked them to work through an instructional problem. Although the average score increased for students in both sections of the course, average increases in problem-finding and generating-solutions scores were significantly higher for students participating in the IF section of the course. Similarly, mean and median problem-finding and generating-solutions scores were higher for groups participating in the IF section. As discussion structure was the only noticeable difference between the two sections, these findings suggest that the way in which a CBI discussion is facilitated can affect the learning that results from the experience. Although Wirkala and Kuhn (2011) concluded that social collaboration is not necessary for learning in a problem-centered approach, the findings from this study indicate that collaboration during CBI significantly affected learning when it was structured and guided in a meaningful way.

To further emphasize this point, it is helpful to compare the lesson activities developed by the preservice teachers in the two sections. That is, participants in the IF section consistently designed lessons that incorporated activities at higher levels of Bloom’s Taxonomy than participants in the IS section. These lessons moved away from asking learners to recall information and instead made attempts to ask learners to apply, analyze, evaluate, and create for understanding. While there is certainly a place for instruction aimed at all cognitive levels (Ertmer & Newby, 1993, 2013), seeing the clear difference between the types of lesson activities designed by the different groups indicates differences in the ways preservice teachers thought about and approached the instructional design process.

One potential explanation for the difference between activity levels included in students’ lessons is that, with the guidance of their instructor, students in the IF section were able to frame and more fully understand problems presented in the cases and develop a more solid understanding of case components (Svihla & Reeve, 2016). For instance, students enrolled in the IF section of the course were required to discuss many problem-finding elements before moving to solutions. Furthermore, when sharing their ideas with the class, students received formative feedback from the instructor. During Case 2, students were asked to develop instructional solutions for an afterschool program. When students in the IF section were discussing solutions, one group shared its plans for using a worksheet as the main activity for its target audience. In response, the instructor asked students to consider the specific audience and environmental characteristics previously identified during earlier discussions and whether using a worksheet best served these needs. With an active facilitator, all students in the IF section could benefit from hearing their peers’ ideas and the instructor’s response. Although students in the IS section of the course had access to the same prompts as the IF students, the extent to which they used such guides is unclear, and student–instructor interaction was not shared at a macro level for the benefit of all.

As a result, students in the IF section who heard peers’ ideas received focused formative feedback, more fully considered the connections between problem elements and solutions, and thus, more readily designed instructional experiences that were motivational and meaningful for their learners. As the trend toward promoting 21st-century skills in K-12 students continues (J. J. Smith & Dobson, 2011), observing preservice teachers design lessons that addressed higher cognitive levels is a promising result. However, this is not to say that the lessons designed by the preservice teachers in the IF section were flawless or that there were no preservice teachers in the IS section of the course who designed engaging, cognitively provocative lessons. Rather, knowing that facilitated discussions in CBI can potentially prompt preservice teachers to create more engaging, meaningful lessons more consistently is promising and gives hope that these individuals will design similarly stimulating types of activities for their future students.

In general, preservice teachers have little experience designing instruction and often have difficulties initiating and completing the process (Jones et al., 2011). This is likely due, at least in part, to an incomplete understanding of the instructional problem at hand (Ertmer & Koehler, 2015). For instance, Ng and Tan (2006) found that preservice teachers participating in an unfacilitated online discussion, focused primarily on solving classroom management problems, discussed only surface aspects of the issues, relied primarily on personal experiences to analyze the problem, and dedicated the majority of their efforts toward developing solutions. These types of behaviors are not uncommon for novice instructional designers, as research shows that individuals new to the process often begin constructing solutions before having a solid understanding of problem components (Hmelo-Silver, Nagarajan, & Day, 2002). Comparing these realities with the findings of this study indicate the importance of providing appropriate support to preservice teachers as they begin working through instructional problems and the important role that discussion plays in CBI. As such, the structure of the discussion, as well as the manner in which it is facilitated, can greatly affect not only the discussion itself, but also the overall learning that occurs among participants (Ertmer & Koehler, 2014, 2015).

Although similar prompts were used in both sections of the course, results were not the same. While the reason for the ineffectiveness of the IS facilitation prompts is not totally clear, there are some likely explanations for these results. Possibly, students in the IS section were constrained by their limited knowledge of instructional strategies, contexts, and processes, and their frustration with the uncertainty of the process may have prevented them from fully benefiting from a discussion that was guided only by predetermined prompts. As other research reveals, novice teachers have difficulties getting started designing appropriate instruction to meet the needs of learners (Jones et al., 2011). Although the prompts were designed to alleviate some of this uncertainty, alone, they did not appear to give students the clarity or direction they needed. As many preservice teachers come to the teaching profession with preconceptions (Hammerness et al., 2005; Lortie, 1975), possibly, even if they used the prompts, their preconceived ideas, accurate or inaccurate, were perpetuated. In the IF section of the course, if a misconception was shared, the facilitator would prompt learners to think further on the topic. In contrast, in the IS section, these ideas were likely not considered or questioned in the same ways.

As prompts alone did not seem to offer enough direction for the case discussion, this underscores the role of the facilitator in CBI. Like other research, the results of this study indicate that during CBI, a facilitator has great responsibility for directing the discourse in meaningful ways (Hmelo-Silver & Barrows, 2006). In the IF section of the course, students had the opportunity to share ideas on a larger scale (i.e., with the whole class), and the instructor prompted students to think through the pros and cons of their ideas. In the IS section, although students had the same opportunity, these discussions were conducted at a micro level (i.e., with their small groups) and ideas were not shared in such a way that all students could benefit.

A final note worth considering is the multiple ways in which CBI affected the students’ learning and understanding during the semester. Although the case narratives used in this study were designed to develop instructional design skills in preservice teachers, the impact of the cases went beyond this goal. Participants reported many other meaningful lessons learned from completing the cases: increased knowledge related to case topics, newfound respect for the teaching profession, awareness about the availability of technology, refined skills for working with peers, and a greater understanding of future work realities. These lessons learned speak to how impactful CBI can be. As instructors design or choose cases for students to use, they should be mindful of case topics and be sure to select those that prompt students to consider worthwhile subjects. This can be especially helpful in addressing the preconceptions that preservice teachers bring with them to their future profession (Feiman-Nemser, 2001). Although students in both sections of the course shared a wide variety of specific and nonspecific instructional design lessons learned, students enrolled in the IF section of the course reported more lessons learned focused on the instructional design process (e.g., problem finding and generating solutions), while students in the IS section of the course typically focused on broader topics (e.g., becoming better acquainted with the realities of their future profession). While both types of lessons learned can certainly support the development of preservice teachers’ problem-solving skills, the ability to understand components of both instructional design and the realities in which instructional design strategies are used will serve preservice teachers more completely. Therefore, in addition to actively guiding preservice teachers’ discussion of instructional design during CBI, instructors should also develop prompts and use questioning techniques to facilitate the consideration of broader topics. In addition, using more focused prompts following a case analysis to guide students’ reflections can be helpful in directing attention toward important areas.

In conclusion, based on the results of this study, facilitated discussions appear to play an important role in the learning that takes place during CBI. That is, CBI discussions provide a space for active problem solving: students engage with real-world problems, interact with diverse perspectives through collaborations with peers, and reflect on and receive clarification on important case topics. However, many considerations must be made regarding the structure and facilitation of the discussion during CBI. Findings from this investigation indicate that CBI can be an effective instructional strategy to use when developing instructional design skills in preservice teachers. However, for best results, CBI should be implemented in a way that meets learners where they are and builds understanding through an active facilitation process. The significant differences between the students’ experiences and learning outcomes observed in this study provide important information for educators who are utilizing problem-centered methods: how CBI is facilitated matters. While these findings highlight many advantages of instructor-facilitated discussions over instructor-supported discussions during CBI, many different methods are used as part of the facilitation process. Additional research is needed to completely understand and define effective facilitation.

Supplemental Material

755701 – Supplemental material for Developing Preservice Teachers’ Instructional Design Skills Through Case-Based Instruction: Examining the Impact of Discussion Format

Supplemental material, 755701 for Developing Preservice Teachers’ Instructional Design Skills Through Case-Based Instruction: Examining the Impact of Discussion Format by Adrie A. Koehler, Peggy A. Ertmer and Timothy J. Newby in Developing Preservice Teachers’ Instructional Design Skills Through Case-Based Instruction: Examining the Impact of Discussion Format

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Author Biographies

Adrie A. Koehler is a visiting assistant professor in the Learning Design and Technology program at Purdue University. Her research interests include considering instructional methods that utilize emerging technologies and facilitating case-based instruction.

Peggy A. Ertmer is professor emerita of Learning Design and Technology at Purdue University. Her research interests relate to technology integration, teacher beliefs, and helping students become expert instructional designers, specifically through the use of case- and problem-based learning methods.

Timothy J. Newby is a professor in the Learning Design and Technology program area in the Department of Curriculum and Instruction at Purdue University. His current research focuses on the impact of open digital badges on learning and motivation.

References

Adejumo

Fakude

Linda

(2014). Revisiting innovative approaches to teaching and learning in nursing programmes: Educators’ experiences with the use of a case-based teaching approach at a nursing school. South African Journal of Higher Education, 28(6), 1694-1707.

Andrews

(2002). Preparing general education pre-service teachers for inclusion: Web-enhanced case-based instruction. Journal of Special Education Technology, 17(3), 27-35.

Cicchetti

D. V.

(1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Part of a Special Section on: Normative Assessment, 6, 284-290. doi:10.1037/1040-3590.6.4.284

Creswell

J. W.

Plano Clark

V. L.

(2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: SAGE.

Dagostino

Carifio

Bauer

J. D. C.

Zhao

Hashim

N. H.

(2014). Assessment of a reading comprehension instrument as it relates to cognitive abilities as defined by Bloom’s revised taxonomy. Current Issues in Education, 17(1), 1-15.

Elliott

S. N.

(1995). Creating meaningful performance assessments. http://files.eric.ed.gov/fulltext/ED375566.pdf

Ertmer

P. A.

Koehler

A. A.

(2014). Online case-based discussions: Examining coverage of the afforded problem space. Educational Technology Research & Development, 62(5), 617-636.

Ertmer

P. A.

Koehler

A. A.

(2015). Facilitated versus non-facilitated online case discussions: Comparing differences in problem space coverage. Journal of Computing in Higher Education, 27, 69-93.

Ertmer

P. A.

Newby

T. J.

(1993). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly, 6(4), 50-72. doi:10.1111/j.1937-8327.1993.tb00605.x

10.

Ertmer

P. A.

Newby

T. J.

(2013). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly, 26(2), 43-71. doi:10.1002/piq.21143

11.

Ertmer

P. A.

Stepich

D. A.

(2002). Initiating and maintaining meaningful case discussions: Maximizing potential of case-based instruction. Journal on Excellence in College Teaching, 13(2/3), 5-18.

12.

Ertmer

P. A.

Stepich

D. A.

(2005). Instructional design expertise: How will we know it when we see it? Educational Technology, 45(6), 38-43.

13.

Ertmer

P. A.

Stepich

D. A.

Flanagan

Kocaman-Karoglu

Reiner

Reyes

. . . Ushigusa

(2008). Impact of guidance on the problem-solving efforts of instructional design novices. Performance Improvement Quarterly, 21(4), 117-132. doi:10.1002/piq.20041

14.

Eseryel

Ifenthaler

Law

(2011). Dynamic modeling as a cognitive regulation scaffold for developing complex problem-solving skills in an educational massively multiplayer online game environment. Journal of Educational Computing Research, 45(3), 265-286.

15.

Feiman-Nemser

(2001). From preparation to practice: Designing a continuum to strengthen and sustain teaching. Teachers College Record, 103(6), 1013-1055.

16.

Feldon

D. F.

Kafai

Y. B.

(2008). Mixed methods for mixed reality: Understanding users’ avatar activities in virtual worlds. Educational Technology Research & Development, 56(5-6), 575-593.

17.

Field

(2005). Discovering statistics using SPSS (2nd ed.). Thousand Oaks, CA: SAGE.

18.

Planas

L. G.

(2010). A cognitive support system to scaffold students’ problem-based learning in a web-based learning environment. Interdisciplinary Journal of Problem-Based Learning, 4(1), 30-56.

19.

Goeze

Zottmann

J. M.

Vogel

Fischer

Schrader

(2014). Getting immersed in teacher and student perspectives: Facilitating analytical competence using video cases in teacher education. Instructional Science, 42(1), 91-114.

20.

Gravett

de Beer

Odendaal-Kroon

Merseth

K. K.

(2017). The affordances of case-based teaching for the professional learning of student-teachers. Journal of Curriculum Studies, 49(3), 369-390.

21.

Hallgren

K. A.

(2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23-34.

22.

Hammerness

Darling-Hammond

Bransford

Berliner

Cochran-Smith

McDonald

Zeichner

(2005). How teachers learn and develop. In Darling-Hammond

L. B.

(Ed.), Preparing teachers for a changing world: What teachers should learn and be able to do (pp. 358-389). San Francisco, CA: Jossey-Bass.

23.

Herreid

Schiller

(2013). Case studies and the flipped classroom. Journal of College Science Teaching, 42(5), 62-66.

24.

Hmelo-Silver

C. E.

Barrows

H. S.

(2006). Goals and strategies of a problem-based learning facilitator. Interdisciplinary Journal of Problem-Based Learning, 1(1), 21-39.

25.

Hmelo-Silver

C. E.

Nagarajan

Day

R. S.

(2002). “It’s harder than we thought it would be”: A comparative case study of expert-novice experimentation strategies. Science Education, 86, 219-243.

26.

Jonassen

D. H.

(2011). Learning to solve problems: A handbook for designing problem-solving learning environments. New York, NY: Routledge.

27.

Jones

K. A.

Jones

Vermette

P. J.

(2011). Six common lesson planning pitfalls: Recommendations for novice educators. Education, 131(4), 845-864.

28.

Kim

Hannafin

M. J.

(2011). Developing situated knowledge about teaching with technology via Web-enhanced case-based activity. Computers & Education, 57(1), 1378-1388. doi:10.1016/j.compedu.2011.01.008

29.

Klein

J. D.

(1991). Preservice teacher use of learning and instructional design principles. Educational Technology Research & Development, 39(3), 83-89.

30.

Krathwohl

D. R.

(2002). A revision of Bloom’s taxonomy: An overview. Theory Into Practice, 41(4), 212-218.

31.

Lachner

Jarodzka

Nückles

(2016). What makes an expert teacher? Investigating teachers’ professional vision and discourse abilities. Instructional Science, 44(3), 197-203.

32.

Leary

Walker

Shelton

B. E.

Fitt

(2013). Exploring the relationships between tutor background, tutor training, and student learning: A problem-based learning meta-analysis. Interdisciplinary Journal of Problem-Based Learning, 7(1), 40-66.

33.

Levin

B. B.

(1995). Using the case method in teacher education: The role of discussion and experience in teachers’ thinking about cases. Teaching and Teacher Education, 11(1), 63-79.

34.

Lortie

D. C.

(1975). Schoolteacher: A sociological study. Chicago, IL: University of Chicago Press.

35.

McNeil

R. C.

(2011). A program evaluation model: Using Bloom’s Taxonomy to identify outcome indicators in outcomes-based program evaluations. Journal of Adult Education, 40(2), 24-29.

36.

Morse

J. M.

(1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40(2), 120-123.

37.

C. S. L.

Tan

(2006). Investigating Singapore pre-service teachers’ ill-structured problem-solving processes in an asynchronous online environment: Implications for reflective thinking. New Horizons in Education, 54, 1-15.

38.

Rosenthal

(1991). Meta-analytic procedures for social research (Rev. ed.). Newbury Park, CA: SAGE.

39.

Rusznyak

L. L. R.

Walton

(2011). Lesson planning guidelines for student teachers: A scaffold for the development of pedagogical content knowledge. Education as Change, 15(2), 271-285.

40.

Savery

J. R.

(2006). Overview of problem-based learning: Definitions and distinctions. Interdisciplinary Journal of Problem-Based Learning, 1(1), 8-20.

41.

Smith

J. J.

Dobson

(2011). Beyond the book: Using Web 2.0 tools to develop 21st century literacies. Computers in the Schools, 28(4), 316-327.

42.

Smith

P. L.

Ragan

T. J.

(2005). Instructional design (3rd ed.). Hoboken, NJ: John Wiley.

43.

Soslau

(2012). Opportunities to develop adaptive teaching expertise during supervisory conferences. Teaching and Teacher Education, 28(5), 768-779. doi:10.1016/j.tate.2012.02.009

44.

Stepich

D. A.

Ertmer

P. A.

(2009). “Teaching” instructional design expertise: Strategies to support students’ problem-finding skills. Technology, Instruction, Cognition and Learning, 7(2), 147-170.

45.

Stepich

D. A.

Ertmer

P. A.

Lane

M. M.

(2001). Problem-solving in a case-based course: Strategies for facilitating coached expertise. Educational Technology Research & Development, 49(3), 53-69.

46.

Sultana

Klecker

B. M.

(1999, November). Evaluation of first-year teachers’ lesson objectives by Bloom’s Taxonomy. Paper presented at the meeting of the Twenty-Eighth Annual Conference of the Mid-South Educational Research Association, Point Clear, AL.

47.

Svihla

Reeve

(2016). Facilitating problem framing in project-based learning. Interdisciplinary Journal of Problem-Based Learning, 10(2), available at: https://docs.lib.purdue.edu/ijpbl/vol10/iss2/10/

48.

Tawfik

A. A.

Jonassen

(2013). The effects of successful versus failure-based cases on argumentation while solving decision-making problems. Educational Technology Research & Development, 61(3), 385-406. doi:10.1007/s11423-013-9294-5

49.

Tawfik

A. A.

Kolodner

J. L.

(2016). Systematizing scaffolding for problem-based learning: A view from case-based reasoning. Interdisciplinary Journal of Problem-Based Learning, 10(1), available at https://docs.lib.purdue.edu/ijpbl/vol10/iss1/6/

50.

Wirkala

Kuhn

(2011). Problem-based learning in K-12 education: Is it effective and how does it achieve its effects? American Educational Research Journal, 48(5), 1157-1186.

51.

Yadav

Barry

B. E.

(2009). Using case-based instruction to increase ethical understanding in engineering: What do we know? What do we need? International Journal of Engineering Education, 25, 138-143.

52.

Yew

E. H. J.

Yong

J. J. Y.

(2014). Student perceptions of facilitators’ social congruence, use of expertise, and cognitive congruence in problem-based learning. Instructional Science, 42(5), 795-815

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB

0.23 MB