Instructional Design,Active Learning,and Student Performance

Abstract

This research used a quasi-experimental design with two conditions to test the impact of active learning in the context of integrated instructional design. The control condition was a traditional approach to teaching an undergraduate strategy capstone class. The intervention condition was an undergraduate strategy capstone class that was designed based on Fink’s integrated instructional design principles and that incorporated an active learning element to teach students financial analysis. As part of the instructional design, the intervention representing active learning content was a set of financial trading room assignments. Though there were no differences in standardized test scores measuring total business knowledge, results showed a difference in student performance associated with financial knowledge between students who experienced the active learning intervention condition and students in the traditional control condition. Even when controlling for gender, major, and grade point average, the integrative instructional design using the trading room had an impact on student learning outcomes associated with financial knowledge.

Keywords

strategy capstone class instructional design trading room finance undergraduate

Introduction

Leading advocates for teaching and learning in higher education (e.g., Fink, 2003; Whetten, 2007) promise that careful attention to integrated course design may lead to significant learning experiences. An entire special issue in the journal New Directions for Teaching and Learning in the Fall of 2009 was devoted to testimonial articles by university educators who described their successes with L. Dee Fink’s integrated instructional design model for learning outcomes. Despite the intuitive appeal of Fink’s taxonomy of significant learning and the intuitive logic of Fink’s instructional design model, we have found almost no empirical work to support the claims of improved learning outcomes using Fink’s approach other than professorial testimonials and student satisfaction surveys (e.g., Persky, 2008).

Although the ideas presented by Fink and others (e.g., Barr & Tagg, 1995) are exciting on their face, we believe that it is critical to document the impact of new instructional design practices for several reasons. First, the current traditional student population, the millennials, presents challenging issues for educators. Second, instructors face broader variance in student talent than in the past. Third, the institutional environment for higher education is evolving rapidly and facing competitive pressures from alternative educational models. Finally, some portion of faculty will always require strong evidence to overcome inertia associated with course redesign. To this end, our research question was simply: Will students in course sections incorporating Fink’s integrated instructional design perform better on a standardized comprehensive assessment exam than those in a traditionally designed course? The results of our pedagogical intervention suggested support for the hypothesis that emphasis on integrated instructional design may have a positive effect on student learning outcomes.

Background

Current Challenges in the Undergraduate Strategy Classroom

Both research and anecdotal evidence suggested that the millennial generation (those born in 1980 through 2000) create challenges for higher education institutions and employers (Alsop, 2008). The millennials were described as sophisticated technologically and sexually but naïve in terms of their own abilities and roles (Stewart & Bernhardt, 2010). They had high self-esteem but relied on parents to protect and defend them. They had inexhaustible amounts of information at their fingertips, but they required specific instructions and feedback to sustain their performance. They were young and inexperienced but felt entitled to perks and benefits that do not exist for more seasoned and experienced employees (Alsop, 2008; Westerman, Bergman, Bergman, & Daly, 2012). They were described as socially networked but without good social skills (Kattner, 2009). Bauerlein (2009) explained why excessive texting and social networking have reduced this generation’s skill at reading nonverbal cues, leaving them vulnerable to misunderstanding interpersonal situations that older generations believed were perfectly clear.

Twenge and Campbell (2008) compared millennials with earlier generations on many classic personality, attitude, psychopathology, and behavior scales using meta-analytic techniques. They concluded that millennials have less need for social approval than earlier generations, translating into less formality such as more casual dress at work and a more casual sense of time. Twenge and Campbell (2008) concurred with Alsop (2008) and Westerman et al. (2012) that millennials demonstrated much higher self-esteem that manifested in the need for positive feedback and praise. Millennials also exhibited more anxiety and depression, requiring careful attention to their state of mind in any interaction with them. They reported that millennial women were more assertive than earlier generations. Finally, many millennials had an external locus of control, meaning they took less responsibility for their performance, good or bad. For example, Baumeister et al. (2005) suggested that when students’ exams are returned, they tended to blame poor scores on external factors, for example, “the test was too hard,” or “the prof didn’t explain that,” or “the questions are too picky/tricky.”

Two studies of millennials that were more classroom-oriented made some telling recommendations. Feiertag and Berge (2008) reported that millennials were not independent workers. They required “more structure, guidance and regular feedback” (p. 457). They found that this cohort lacked the ability to make critical extensions of ideas, and therefore, they did better with information that was tailored to them individually rather than general lecture format. Milliron (2008), in a study that was not surprising but was disturbing, reported that millennials preferred to enroll in accounting courses that had lower workloads and had less emphasis on analytical and computational assignments, despite the importance of these functions for learning and for future work skills.

Although millennials present a particular challenge to faculty, other changes in the higher education environment are also emerging. Higher education as institutions that cater primarily to academically talented students are no longer the norm. Data from the National Center for Education Statistics (2009) suggested that 66% of high school graduates now expect to complete a college degree compared with approximately 20% two decades ago. Federal and state governments have substantially increased their expectations for workforce development as part of the mission of higher education (Parker, 2002). Finally, increased accountability from both government and accrediting bodies such as the Association to Advance Collegiate Schools of Business (AACSB) have focused on classroom learning to an extent unimagined a generation ago.

The role of the faculty in the classroom has also been challenged by the availability of information. Faculty are no longer the gatekeepers and unique transmission mechanisms for relevant information. Concepts and theories are readily available via the Internet at almost no cost. The role of the faculty has evolved such that effective teaching is less about introducing concepts and more about helping students integrate, synthesize, and analyze the information to which they have access.

Faculty teaching undergraduate capstone strategy courses particularly feel the impact of these demographic and institutional changes. The strategy capstone course has always been a point of integration requiring critical thinking, analysis, and synthesis. There certainly is a domain of unique concepts and theories that are taught in the area of strategy. However, the position of strategy as a capstone in many undergraduate curriculums adds to the expectations that a fundamental part of the course includes critical thinking and application of a wide variety of core business concepts. These institutional expectations, combined with the changes in the background, learning preferences, attitudes, and academic preparation of the millennial student, have increased the challenges felt by many undergraduate strategy faculty and are driving many faculty to seek new instructional designs.

The undergraduate strategy capstone course seems to be facing a perfect storm. Although institutions highlight the need for critical thinking, especially in a world of information overload, millennials seem to be dispositionally less prepared for this complexity. Faculty have typically responded to environmental conditions by expanding the number of concepts and topics covered, by incorporating more case analysis, and by adding more time-intensive experiential techniques such as simulations and service learning to their classrooms. Though these experiential techniques are highly touted, there are mixed results associated with assessment of student learning. Some faculty seem to be able to use these techniques and enhance student learning, whereas other faculty are frustrated with the additional effort and time investment and do not see improvements in student learning that seem worth the additional work and coordination costs (Seaton & Boyd, 2008).

We believe that one important element that distinguishes experiential techniques with greater learning impact from those without is the extent to which a new teaching paradigm is embraced and implemented. This new paradigm shifts the focus of academic courses from covering the material to achieving specific learning experiences (Diamond, 1998). To execute this shift, faculty must give purposeful attention to integrated instructional design.

Theory and Hypothesis Development

Integrated Instructional Design for Significant Learning Experiences

Taxonomy of significant learning experiences

One key principle of integrated instructional design is similar to a core concept stated by Collins and Porras (2000) in the popular management book Built to Last. Simply stated, one must “start with the end in mind.” In this case, the “end in mind” is creating significant learning experiences for students and creating them consistently over time and across offered sections. For the purposes of our research, we focused on Fink’s (2003) model that describes how significant learning experiences are created through integrated instructional design. This model of instructional design has three interrelated components: (a) learning goals, (b) teaching and learning activities, and (c) feedback and assessment. These integrated instructional design elements are in service to a taxonomy of types of significant learning experiences (Fink, 2003).

The taxonomy includes six interdependent components: foundational knowledge, application, integration, human dimension, caring, and learning how to learn. Integrated instructional design provides the impetus and context for the emergence of one or more of these six types of significant learning. Briefly, foundational knowledge refers to understanding and remembering the basic information of the course. Application moves into skill acquisition where the student engages in critical creative or practical thinking so that the basic information is put to some use. Integration learning connects different information and ideas. Human dimension learning explores the human significance of the course content. Caring develops new feelings, interests, and values in the learner. And finally, learning how to learn builds skills for becoming a better student, that is, inquiry skills and becoming self-directed learners.

Integrated instructional design, learning goals

A key element of integrated instructional design that creates these six significant learning experiences is the articulation of clear learning goals. As stated earlier, the institutional expectations of the capstone strategy course demand a greater attention to student learning than more concept-driven courses. Furthermore, the strategy capstone course has an additional goals associated with transitioning college students to working professionals. Fink (2003) offers specific suggestions for how to articulate learning goals. Keys to goal setting are to identify a few reasonable goals rather than a long laundry list of unrealistic goals and to use verbs to begin goal statements. A typical goal for a capstone strategy course might be “determine if a firm is creating or losing value in its marketplace.”

Integrated instructional design, feedback and assessment

Fink (2003) uses a model of feedback and assessment that discourages the traditional retrospective process that simply tests acquired knowledge and assigns a grade. Instead, he proposes a more interactive model where (a) students have clear criteria and standards for each assignment, (b) students participate in self-assessment, and (c) assessments are forward-looking so that both students and faculty can engage in continuous improvement throughout the semester. The feedback and assessment dimension emphasizes iterative processes, rather than linear designs, to enhance learning experiences.

Integrated instructional design, teaching and learning activities

Fink advocates active learning exercises over passive learning modes. Passive modes (lectures, PowerPoint slides) fail to engage students in the learning process. Active learning, on the other hand, has three components: (a) information and ideas, (b) experiences of students doing things or observing others doing things, and then, importantly, (c) student reflection associated with the activity so that stronger cognitive models of new learning can be built.

Fink (2003) would argue that teaching activities within a course would have limited impact without embedding those activities, whatever they are, within the specific learning goals, context, feedback mechanisms, assessment tools, and reflective elements described in his model. Instructional design makes the level of analysis the course and includes not only instructional content but also issues of sequencing, delivery, type and style of teaching activities, resources necessary, critical evaluation of the design plan, and continuous monitoring of the operation of the course so that changes can be incorporated as needed. Table 1 summarizes what Fink (2003) identifies as considerations in integrated instructional design.

Table 1.

Elements of Fink’s Instructional Design

1. Ambitious learning goals

2. Wide range of learning activities used

3. Powerful learning and doing (active) experiences

4. Multiple opportunities for in-depth reflection on the learning

5. Alternative ways to introduce students to the content elements in the course

6. A coherent and meaningful course structure

7. A dynamic instructional strategy

Note: Adapted from Fink (2003, p. 151).

Active learning and instructional design

The application of integrated instructional design means that undergraduate strategy capstone classes should incorporate elements of active learning but within a larger design framework. Active learning seeks to achieve some of the same types of learning as experiential learning (Kolb, 1984). Experiential learning theory (Kolb, 1984, . 27) posits that “learning is a continuous process grounded in experience,” or more explicitly, a “process in which internalized reflection follows concrete experience, resulting in an adaptation revealed in further experience” (Quay, 2003, p. 108). Experiential learning via service learning or internships has traditionally been used to create significant learning experiences for students in many types of programs. Experiential learning, though, tends to occur outside the classroom and often outside the control of the instructor.

Although there is widespread belief that experiential learning is valuable for developing skills and mastering techniques that are taught conceptually in the classroom, these activities, particularly at the undergraduate level, have not become widespread or fully integrated into business school curricula. Many instructors incur a problem of scalability when incorporating ongoing service projects and other cocurricular experiential activities into their courses. Coordination costs for such programs are high, and maintaining the same quality of experience for each student is difficult. Instructors have little control over “real life.” Thus many faculty have a dilemma. They may endorse the real value that comes from an experiential approach but may not have resources to continuously support that aspect of a course.

Active learning using elements of integrated instructional design may be one way to incorporate consistent, hands-on experiences and maximize their impact on undergraduate student learning. “The single biggest improvement most teachers can make is to give students more “doing” and “observing” experiences related to the subject of the course. Direct experiences are the most powerful” (Fink, 2003, p. 123).

In the capstone strategy class, the “doing” behaviors can include collecting information directly, through using computer applications, search engines, library resources, and oral history techniques. Active learning can include using complex simulations in classrooms to model concepts and interactions (Seaton & Boyd, 2008). However, participation alone in these active experiences may not be sufficient for learning. Reflection is a key dimension that must be built into the instructional design. After the student has acquired the information and had an opportunity to do something with it, the student needs an opportunity for active reflection. This follow-up loop transitions the information and behaviors of doing and observing into useful knowledge through a sense-making process (Weick, 1995). The reflection can be stimulated with individual thought questions, class debriefing discussions, projects or other probing assignments. The reflection should connect the active learning to the course itself and the learning process as well.

Hypothesis Development

We believe that active learning, supported by the proper instructional design can lead to significant learning experiences. Fink (2003) and Diamond (1998) made strong pragmatic arguments for why integrated instructional design adds value in the classroom. Fink (2003) presented survey evidence suggesting that student respondents report improved learning experiences. Seaton and Boyd (2008) argue for the use of active learning, specifically simulations in business courses, as design elements that would create significant learning experiences. Unfortunately, they did not present empirical evidence for the learning impact of courses that use simulations over courses without simulations. Saiia, Granger, and Boyd (2008) reported that a majority of their students responded favorably to an active learning role play exercise in a strategy course, but they did not formally assess the learning outcomes of this activity within the context of the instructional design.

Despite the intuitive appeal of using active learning activities and integrated instructional design concepts, pedagogical research that explicitly examined instructional design and learning outcomes reported mixed findings. Priluck (2004), for instance, compared two business marketing courses, one typical in class format and one designed with a hybrid lecture and web-assisted format. Arguably, the second format added out-of-class learning opportunities that might have followed some of Fink’s (2003) active learning prescriptions. In fairness, however, Priluck’s experiment was not designed to test Fink’s proposals but was aimed at understanding the benefits of web-based educational delivery. Priluck reported that students in the pure face-to-face format were more satisfied, and they believed that their skills and knowledge improved to a greater extent than the students in the hybrid face-to-face/web-delivery format. Although there was a significant difference in the students’ perceptions, the students’ actual performance on the end of semester exams did not differ. Therefore, there was no difference in performance.

Persky (2008) followed Fink’s (2003) prescriptions for integrated course design for basic and clinical pharmacokinetics courses. He added three group-based games that represented active learning, as well as immediate feedback assessment and reflective writing assignments. In addition, there was an out of class multimedia-enriched learning module that was very hands-on. Despite these course design changes, average examination scores did not improve. He reports that student satisfaction did increase. It is important to note that Persky (2008) suggested that the types of learning that the significant learning experiences facilitate (critical thinking, deep learning) may be best measured over the long term and in practical experiences rather than in examination formats. In both the Priluck and Persky examples, changes in perceptions of learning, satisfaction, and skill improvement were reported with the design changes; however, measurable student learning was unchanged. One conclusion could be that active learning, absent integrated instructional design, may not have the desired effect on student learning outcomes.

These mixed findings suggest that there is need for more empirical investigation of the relationship between integrated instructional design, active learning activities, and measures of learning outcomes. It is possible that Priluck’s research failed to find measureable results because it did not encompass the full range of Fink’s instructional design. In Persky’s case, as he suggested, his research may have failed to find measureable differences in outcomes because both classes had very high performance, so there was little variance.

The research above, when applied to the undergraduate strategy capstone course, suggests the following: undergraduate capstone strategy courses may suffer from a lack of integrated instructional design, thus limiting the extent of significant learning that could occur. Greater attention to integrated instructional design and application of Fink’s approach may have a positive impact on the measurable learning outcomes from this course. Therefore, undergraduate strategy courses may demonstrate improvement in measurable learning outcomes in circumstances where Fink’s integrated instructional design model is applied in concert with active learning.

In the authors’ experiences, successful active learning techniques in the strategy capstone class tend to be consistent with teaching activities associated with the integrated instructional design approach articulated by Fink (2003). We also believe that Fink’s approach addresses the issues of teaching strategic concepts to millennials, given their demonstrated lack of engagement when passive pedagogy is used. Therefore, we believe that undergraduate strategy courses should be able to demonstrate improvement in measurable learning outcomes in circumstances where Fink’s instructional design model is applied to active learning events within the classroom. This line of reasoning leads to the following hypothesis:

Hypothesis: Students in strategy courses based on integrated instructional design principles and using active learning techniques will demonstrate higher performance on end-of-semester assessments of foundational knowledge than students in courses that do not use these techniques.

Research Method

The research methodology used a quasi-experimental design with two conditions. A quasi-experimental design is similar to a traditional experimental design except that assignment to the experimental and control conditions are not randomized (Cochran, 1983). According to Aussems, Boomsma, and Snijders (2011),

A study is labeled quasi-experimental if two conditions hold. The first is that a researcher is interested in the effect of an intervention on one or more responses. . . . The second condition is that an intervention effect is investigated by comparing groups, which means that control or referent groups are used for the assessment of a treatment effect. The treatment and control groups are formed or identified before the treatment is imposed, and it is assumed that external influences are affecting the groups to the same extent during the experimental period. Furthermore, no reference is made to the use of randomization in allocating subjects to treatment conditions. (p. 22)

In this research, the control condition was a traditional pedagogical approach to teach an undergraduate strategy capstone class. The intervention condition was an undergraduate strategy capstone class that was redesigned based on Fink’s (2003) integrated instructional design principles and that incorporated an integrated active learning element to teach students financial analysis.

Sample

To test the hypothesis and to examine the impact of instructional design and active learning further, we used a nonrandomized convenience sample of 114 undergraduate strategy students across four sections of an undergraduate capstone class. All students enrolled in the four course sections were included in our study. Of the 114 students, 112 provided complete data for the study, and these 112 students were the subjects used in the study. The strategy course was the capstone class for all majors in a School of Business and Economics at an AACSB-accredited institution in the southeastern United States. The strategy classes used in the study were offered in the same semester (spring), and the 112 students were spread across four course sections. The three faculty members teaching these sections have terminal degrees in the area of strategic management awarded from research-intensive institutions. The three faculty members each have at least 20 years of teaching experience in AACSB-accredited institutions and experience in teaching undergraduate, graduate, and Ph.D. courses.

Two instructors taught one section, and a third professor taught two sections of the strategy capstone course. Two professors (responsible for three sections) revised the capstone strategy course based on integrative instructional design principles described by Fink (2003). Although they did not team-teach, the two faculty designers coordinated classroom activities and content coverage. The third faculty member taught one section of the capstone course using traditional teaching methodology and coordinated content coverage with the other two sections.

Frequency distributions and an ANOVA of Gender, Major, and Grade Point Average (GPA) by course section (and/or confounded factors) are presented in Table 2. The results seem to show no statistically significant differences among across course sections. Nonetheless, because of our inability to randomly assign students to course sections, the possibility exists that true differences among sections were masked by other, confounded factors. Generally though, these sections were homogeneous with respect to type of student. All students in all sections are traditional, full-time students of similar ethnic background.

Table 2.

Number of Students by Attribute, Classroom, Instructor, and Teaching Methodology

	Teaching Methodology
	Traditional Design		Integrated Instructional Design
	A		B		C		D
Class Section	1		2		2		3
Instructor	n	%	n	%	n	%	n	%
Gender
Male	15	60	19	58	13	48	15	56
Female	10	40	14	42	14	52	12	44
Major
Management	9	36	13	39	7	26	5	19
Marketing	4	16	9	27	4	15	4	15
Finance	1	4	2	6	3	11	4	15
Accounting	10	40	9	27	9	33	8	30
Economics	1	4	0	0	2	7	0	0
MIS	0	0	0	0	0	0	4	15
Transportation	0	0	0	0	2	7	1	4
Education	0	0	0	0	0	0	1	4
GPA
2.50-2.74	2	8	5	15	5	19	2	7
2.75-2.99	10	40	16	48	9	33	13	48
3.00-3.24	6	24	7	21	7	26	7	26
3.25-3.49	3	12	3	9	3	11	0	0
3.50-3.74	4	16	1	3	1	4	4	15
3.75-4.00	0	0	1	3	2	7	1	4

ANOVA
		Sum of Squares	df	Mean Square	F	Significance

Gender
Between groups		.136	1	.132	.540	.463
Within groups		28.062	111	.251
Total		28.198	112
Major
Between groups		.096	1	.096	.025	.876
Within groups		421.975	111	3.827
Total		422.071	112
GPA
Between groups		.231	1	.222	1.035	.317
Within groups		24.575	111	.219
Total		24.806	112

Note: GPA = grade point average; ANOVA = analysis of variance.

The Instructional Design Intervention

The learning goals for the intervention sections were (a) to improve retention of foundational knowledge in business and (b) to improve application and integration of the foundational knowledge. Although many strategy courses include emphasis on foundational knowledge, application, and integration, the intervention sections of the course were framed around specific “direct” active learning events and “forward looking” assessment (Fink, 2005). Criteria and standards were explicit and evaluated via rubrics directly associated with classroom analysis assignments. Feedback on assignments was “frequent, immediate, discriminating, and empathically delivered” (Fink, 2005, p.4). Reflection on individual and collective section performance was incorporated with the assessment process. Peer assessment was also a consistent component of the intervention course sections.

As part of the integrated instructional design, the revised capstone course purposely integrated strategy content with active learning activities using a new financial trading room that was opened for undergraduate student use. The trading room was opened initially in the previous semester (fall). The trading room is a dedicated technology-supported classroom featuring 32 student workstations with dual monitors, two large screen display boards, a two-line electronic stock ticker with 20-minute delayed data feed and an instructor workstation. This interactive technology-based facility engages students in a “learning by doing” environment. The trading room uses the software “Factset.” FactSet offers access to data and analytics for analysts, portfolio managers, and investment bankers at global financial institutions (www.factset.com). FactSet integrates several hundred databases from multiple vendors. The applications include company analysis, multicompany comparisons, industry analysis, company screening, portfolio analysis, predictive risk measurements, alpha testing, portfolio optimization and simulation, real-time news and quotes, and tools to value and analyze fixed income securities and portfolios (Factset Research Systems Inc., 2012).

The opening of the trading room in the previous semester provided the impetus for the revision of the capstone strategy class and the opportunity for a study to test its impact on student learning. Students, in general, had no experience with the trading room software prior to their experience in the strategy capstone course. For the intervention sections, activities in the trading room were incorporated throughout the semester and linked to other assignments associated with strategic analysis.

Following the instructional design recommendations of Fink (2003), the revised capstone course greatly enlarged the number and type of learning activities, particularly those associated with the trading room, and created multiple doing and observing (active learning) experiences. These active learning elements created the context for several types of significant learning experiences (foundational, application, integration), provided opportunity for individual student reflection, and built an instructional strategy that purposefully sequenced several learning activities to integrate financial data acquisition and analysis with strategic company analysis (Fink, 2003).

Student work was assessed repeatedly during the semester with significant feedback given on student submissions. All case analyses in these sections incorporated guided trading room activities to encourage students to focus on interpretation of financial data and application of financial data for purposes of evaluating company strategy. Students spent at least 6 structured contact hours of 48 total semester contact hours in guided, hands-on active learning within the trading room. The trading room was also available at other times for student use outside of the structured course.

The control condition in this research was a section of strategic management that was taught in a more traditional style using didactic lectures, midterm examinations, and team presentations of written case analyses, which were then evaluated by the instructor. Course content, in terms of chapters covered and number of cases assigned, was similar across all four sections. The greatest content difference between the control section and the intervention sections was the control section’s (a) lack of participation in the trading room, (b) the lack of course restructuring based on the integrated instructional design approach with an emphasis on active learning, and (c) lack of forward looking assessment.

Data Collection

Data were collected from all sections at the end of the semester as part of the standard assessment activities of the School of Business and Economics. Data on GPA, Major, and student demographics were collected. In addition to testing students on general business knowledge, students were also surveyed regarding their perception of the value of learning from the class as well as their perceptions regarding the quality of teaching in their section.

Dependent Variable: Student Learning Outcomes

The goal of this research was to examine the impact of instructional design principles and active learning on student learning outcomes in an undergraduate strategy course. One goal of an undergraduate capstone course is to encourage students to gain a deeper foundational understanding and apply knowledge from a variety of business courses. Students access their business knowledge from many disciplinary areas and use it within strategy frameworks to practice analysis and decision-making skills. Thus, although concepts specific to strategy are indicative of learning, the course is also where wide variety of foundational knowledge domains in business are reinforced.

In this research, the dependent variable was operationalized as performance on the Educational Testing Service (ETS) Major Field Test in Business. ETS Major Field Tests are comprehensive assessments designed to measure the core knowledge and understanding by students in a major field of study. The undergraduate MFT in Business is a standardized test used by many accredited schools of business in the United States for program assessment. The Major Field Test (MFT) is an objective multiple-choice test covering the core areas of business knowledge. Scores are reported for each individual. Scores are reported as an overall score, and specific scores for nine disciplinary areas are provided. The MFT was administered at the very end of the semester and was not directly related to any of previous active learning elements of the course. The MFT was done as part of the ongoing assessment activities of the School done at the end of the capstone course. Consistent with best practices, this assurance of learning activity was embedded in the course and was a component of the total student grade.

The dependent variable was measured in two ways. Dependent Variable 1 (DV1) was calculated as the percentage of correct answers on the MFT for each student. This measure was used to represent the broad domain of topics covered in the context of the course. The second dependent variable (DV2) was measured by calculating the scores (percentage of correct answers) for the finance and accounting areas of the MFT. DV2 is considered to be a more fine-grained dependent variable. These two disciplinary areas were selected for separate analysis because the active learning events in the course were so closely tailored to the reinforcement of foundational knowledge and application in these two disciplinary areas.

Experiential Variable

The intervention was operationalized by whether a student was a member of the control section or a member of one of the strategy sections with the integrated instructional design intervention. Assignment to the intervention was indicated by a categorical variable where 0 indicated the control group and 1 indicated the integrated instructional design condition.

Control Variables

Two control dummy variables and one continuous control variable were used in this research. Gender was included to account for the fact that males and females often perform differently on mathematical tasks and can also perform differently in classroom environments. This variable was coded 1 for males and 0 for females. School major was included to control for possible specialized knowledge accessible to finance and accounting majors who may have had more experience with the content associated with the trading room. The dummy variable was coded 0 if the student was not a finance or accounting major and 1 if they were. GPA was used to control for the academic ability and academic success of students across all sections. We collected the current cumulative GPA of each student recorded for the semester when the research occurred.

Analysis and Results

Check for Bias Due to Section Differences

A manipulation check for bias was done to determine if the students in the four sections perceived differences in the value of learning in the course or quality of teaching in their section. To reduce the probability that results were due simply to the teaching styles of the faculty, we looked for differences in student perceptions by instructor. An ANOVA of Teaching Quality by Section (and/or confounded factors) was presented in Table 3. The results showed no statistically significant differences across sections. Nonetheless, because of our inability to assign students to sections randomly, the possibility exists that true differences across sections were masked by other, confounding factors. Also in Table 3 we reported a similar ANOVA result, which must be similarly qualified, for response variable Value of Learning. On its face, the result of the ANOVA suggests that students across sections were similarly satisfied with their learning and classroom experiences regardless of which professor taught their course.

Table 3.

Students’ Perceptions of Teaching and Learning Across Sections

	Teaching Methodology
	Traditional Design		Integrated Instructional Design
	A		B		C		D
Class Section	1		2		2		3
Instructor	n	Mean	n	Mean	n	Mean	n	Mean
Value of learning	25	4.71	33	4.81	27	4.37	27	4.66
Quality of teaching	25	4.73	33	4.48	27	4.59	27	4.68

ANOVA
		Sum of Squares	df	Mean Square	F	Significance

Value of learning
Between groups		.068	1	.063	.158	.724
Within groups		55.702	111	.498
Total		55.770	112
Quality of teaching
Between groups		.794	1	.794	1.469	.223
Within groups		55.696	111	.513
Total		56.490	112

Note: GPA = grade point average; ANOVA = analysis of variance.

Test of the Hypothesis: Overall Knowledge (DV1)

Table 4 was a test of our hypothesis using DV1 as the measure of the dependent variable construct. Our hypothesis suggested that the emphasis on integrated instructional design will have an impact on student learning. We argued that emphasis on the design elements would improve overall performance on the MFT. To test this hypothesis, we applied a hierarchical multiple regression analysis and entered the control variables separately in the model prior to entering the dummy variable representing the intervention. The impact of the control variables was significant for gender and GPA. Student major did not have a significant impact on overall student learning. The results for gender suggested that the male students performed better on the MFT than the female students. As expected, GPA also was an important predictor of performance on the major field test. Higher GPAs were associated with better performance on the MFT. After control variables were entered in the model, the impact of the course design intervention demonstrated that performance overall on the MFT did not differ across the sections. There was no significant change in the adjusted R². This result suggested that when examining overall business knowledge, there seemed to be a lack of support for the hypothesis that integrated instructional design improved student learning outcomes across all business knowledge domains.

Table 4.

Hierarchical Multiple Regression Analysis: DV1 = Percent Correct All Knowledge Areas

	B	B	B	B
Constant	.401***	.388***	.184***	.167**
Gender	.057**	.060***	.053**	.052**
Major		.003	−.001	−.001
GPA			.079***	.081***
Integrated instructional design				.017
R ²	0.321	0.328	0.513	0.518
Adjusted R²	0.095	0.091	0.242	0.241
Model F statistic	12.628**	6.555**	12.825***	9.820***

Note: GPA = grade point average; DV1 = Dependent Variable 1.

***

p < .001. **p < .01. *p < .05.

Test of the Hypothesis: Finance and Accounting Scores (DV2)

Part of the reason for including the trading room as the primary active learning component of the integrative instructional design was to enhance the students’ use and analysis of financial and accounting information. We decided to examine the impact of this active learning component more closely. Because the active learning emphasis was on financial and accounting application in the trading room, we expected that the intervention might improve performance in this specific knowledge domain. A subset of questions on the MFT that were associated with accounting and financial topics was used to examine the impact of the active learning component of the course (the trading room) on foundational knowledge in finance and accounting.

As shown in Table 5, the hierarchical multiple regression model showed an increase in the adjusted R² when the dummy variable representing the intervention was included in the model. Even when controlling for Gender, Major, and GPA, the variable representing the intervention was associated with an increase in financial learning outcomes. An examination of the unstandardized beta coefficient showed that the intervention was associated with an average increase of 5.6 points in DV2. This result provided support for the idea that an active learning approach may benefit student learning outcomes, particularly in the population of traditional students of the millennial generation. However, although the intervention seemed to have an impact on student learning associated with financial knowledge, because of the limitations of our quasi-experimental design, we were unable to unequivocally claim that the change in student learning was a statistically significant result based on the integrated instructional design intervention alone.

Table 5.

Hierarchical Multiple Regression Analysis: DV2 = Percent Correct Financial Questions

	B	B	B	B
Constant	.421***	.370***	.118	.059
Gender	.055*	.064**	.056*	.052*
Major		.006	.007	.006
GPA			.097***	.105***
Integrated instructional design				.056*
R ²	0.210	0.274	0.432	0.467
Adjusted R²	0.035	0.058	0.164	0.189
Model F statistic	5.072*	4.429**	8.274***	7.468***

Note: GPA = grade point average; DV2 = Dependent Variable 2.

***

p < .001. **p < .01. *p < .05.

Discussion and Conclusions

The results of this research were mixed. On one hand, the emphasis on integrated instructional design may not have had the measurable impact on broader student learning that Fink (2003) describes. On the other hand, when we examined the component of the instructional design that was most closely aligned with the hands-on, active learning component, a noticeable measurable impact was found. It was difficult in this study to tease out the impact of the design from the impact of the active learning activity itself. What we do know is that student learning in knowledge domains that did NOT have the active learning component in the intervention condition showed no difference from the traditional capstone course.

The results led the authors to conclude that a key element in the integrated instructional design process may be the type of teaching activities that are chosen. If this is a valid conclusion, there are significant implications for teaching the strategy capstone course to millennial students. Unlike studies that found no relationship between active learning and student outcomes, this study, which embedded active learning in a larger instructional design context, seemed to find an effect on student learning outcomes. However, these outcomes seemed to be most closely related to the knowledge domain of the active learning exercise.

These results suggest that greater emphasis on incorporating targeted active learning may substantially add to student learning. The capstone strategy class has always been considered a transition point for business students. However, it must be stressed that students were not just introduced to the trading room and left to their own devices. The trading room experience was carefully organized and linked to other elements in the strategy course. Students were assigned specific exercises with reflective elements. Students were given feedback on the quality of their analysis based on the activities in the trading room. Instructional strategies were devised and implemented, and the sequencing of the course was such that the trading room featured prominently in more than one element of the class. In short, the professors created a purposeful experiential component and coordinated their implementation both within and across the three sections of the course where the trading room was used.

The most exciting result from this purposeful design was the additional impact that this activity had over and above the impact of GPA. In 21st century higher education there is greater emphasis than ever before on teaching students of all academic backgrounds and levels of talent. Traditional pedagogy, which may have been well suited to academically talented students, may not work as well for the average student who now enrolls in college. More active learning and more attention to the nuts and bolts of design elements in the classroom may be the way that faculty must respond to this challenge. For too long, we have relied on the quality of the student to compensate for lack of faculty attention to and emphasis on good instructional design. With academically talented students, learning often occurs regardless of faculty effort. With a much wider variance of academic talent within many higher education institutions, greater emphasis on design may be a necessity for student success.

Though greater attention to what goes on in the classroom is important, we must resist the desire to implement course revisions without a rigorous examination of their potential impact. Scholarship of teaching and learning, which tests theories of design, such as Fink (2003), is critical to the process of evidence-based improvement in higher education instruction. As business faculty, models of continuous process improvement and performance measurement provide us with the tools we need to take a more systematic and rigorous approach to testing instructional design principles.

Many universities in the United States have built dedicated trading rooms. However, even universities that do not have trading rooms can create these types of purposeful experiential components by more closely incorporating existing resources into instructional design of strategy courses. Large, multiyear financial databases in Excel format are directly downloadable from online subscriptions to Standard and Poor’s. Many university libraries have this kind of online access. In addition, free websites such as http://finance.yahoo.com or www.money.msn.com provide basic types of company financial data over multiple years. Though the data may not be as easily accessible or comprehensive as FactSet, we believe that the student learning outcome is not because of the source of the data but rather the purposeful and integrated way that acquisition and use of data was incorporated in the analysis activities in class. Developing assignments that are completed collectively in class or computer labs with students working together to acquire data and apply it to business problems in real time is the basis for improved student learning.

The environment in higher education is a challenging one. The characteristics of the millennial student and the expectations of society regarding the role and responsibility of higher education demand a response from faculty. Testing new techniques in undergraduate programs can be a significant contribution to the growing body of scholarship associated with the teaching and learning role of the Academy.

Limitations

Although results of this research seemed promising, there are significant limitations. In this study we examined both integrated instructional design and active learning. As these were both elements within the same course, it was difficult to parse out the impact of each. Would the result have been the same if we had incorporated the trading room activity without any revised course design?

Also, this research was opportunistic in that the opening of the trading room provided a window of opportunity to examine its impact on the students. Now that the resource has been available for more than a year and used in other courses, it would be difficult to tease out the impact of the active learning exercises in the capstone course.

Another limitation of this research is that both the faculty and the students were learning the elements and capabilities of the trading room simultaneously. In addition, this was the faculty’s first attempt at purposefully incorporating Fink’s (2003) instructional design principles. Thus, the impact of the design could become stronger as faculty become more experienced at designing and implementing this approach to developing the course.

Finally, there were limitations with using the MFT to measure learning in the capstone course. Significant learning experiences described by Fink (2003) go beyond cognitive learning. The emphasis on critical thinking and knowledge synthesis was not necessarily captured on an objective, standardized examination.

Conclusion

This research attempted to examine the value of Fink’s (2003) integrated instructional design principles. Although some indications from this work showed promise, it was not completely clear that attention to design alone improved student performance. It did seem, though, that active learning, used in the context of integrated instructional design, was a very important element, particularly for the millennial generation of students. The more that schools of business can purposefully use instructional design to integrate new tools needed for the 21st century workforce, the more likely it is that undergraduate business programs will remain relevant and important to the corporate community.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Alsop

(2008, October 20). The “Trophy Kids” go to work. Wall Street Journal, p. D1.

Aussems

Boomsma

Snijders

(2011). The use of quasi-experiments in the social sciences: A content analysis. Quality & Quantity, 45(1), 21-42. doi:10.1007/s11135-009-9281-4

Barr

R. B.

Tagg

(1995). From teaching to learning: A new paradigm for undergraduate education. Change, 27, 13-25.

Bauerlein

(2009, September 4). Why gen-Y Johnny can’t read nonverbal cues. Wall Street Journal. Retrieved from http://online.wsj.com/article/SB10001424052970203863204574348493483201758-email.html

Baumeister

R. F.

Campbell

J. D.

Krueger

J. I.

Vohs

K. D.

(2005). Exploding the self-esteem myth. Scientific American Mind, 16(4), 50-57.

Cochran

W. G.

(1983). Planning and analysis of observational studies ( Moses

L. E.

Mosteller

, Eds.). New York, NY: John Wiley.

Collins

J. C.

Porras

J. I.

(2000). Built to last: Successful habits of visionary companies. London, England: Random House.

Diamond

R. M.

(1998). Designing and assessing courses and curricula: A practical guide. San Francisco, CA: Jossey-Bass.

Factset Research Systems Inc. (2012). Company report 2009. Retrieved from http://moneycentral.msn.com/companyreport?symbol=FDS

10.

Feiertag

Berge

Z. L.

(2008). Training generation N: How educators should approach the net generation. Education & Training, 50, 457-464.

11.

Fink

L. D.

(2003). Creating significant learning experiences. San Francisco, CA: Jossey-Bass.

12.

Fink

L. D.

(2005). Integrated course design (Idea Paper No. 42). Retrieved from http://www.theideacenter.org/sites/default/files/Idea_Paper42.pdf

13.

Kattner

(2009). Best practices for working with millennial students. Student Affairs Leader, 37(16), 5.

14.

Kolb

D. A.

(1984). Experiential Learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall.

15.

Kolb

A. Y.

Kolb

D. A.

(2005). Learning styles and learning spaces: Enhancing experiential learning in higher education. Academy of Management Learning and Education, 4, 193-212.

16.

Milliron

(2008). Exploring millennial student values and societal trends: Accounting course selection preferences. Issues in Accounting Education, 23, 405-419.

17.

National Center for Education Statistics. (2009). The condition of education 2009 (NCES 2009-081). Washington, DC: U.S. Department of Education.

18.

Parker

L. D.

(2002). It’s been a pleasure doing business with you: A strategic analysis and critique of university change management. Critical Perspectives on Accounting, 13, 603-619.

19.

Persky

A. M.

(2008). Multi-faceted approach to improve learning in pharmacokinetics. American Journal of Pharmaceutical Education, 72(2), 1-7.

20.

Priluck

(2004). Web-assisted courses for business education: An examination of two sections of principles of marketing. Journal of Marketing Education, 26, 161-173.

21.

Quay

(2003). Experience and participation: Relating theories of learning. Journal of Experiential Education, 26, 105-112.

22.

Saiia

Granger

Boyd

M. P.

(2008). Meaningful learning in management: Recombining strands of knowledge DNA through engaged dialog and generative conflict. Organization Management Journal, 5, 167-179.

23.

Seaton

L. J.

Boyd

(2008). The effective use of simulations in business courses. Academy of Educational Leadership Journal, 12, 107-118.

24.

Stewart

K. D.

Bernhardt

P. C.

(2010). Comparing millennials to pre-1987 students and with one another. North American Journal of Psychology, 12, 579-602.

25.

Twenge

J. M.

Campbell

S. M.

(2008). Generational differences in psychological traits and their impact on the workplace. Journal of Managerial Psychology, 23, 862-877.

26.

Whetten

D. A.

(2007). Principles of effective course design: What I wish I had known about learning-centered teaching 30 years ago. Journal of Management Education, 31, 339-357.

27.

Weick

K. E.

(1995). Sensemaking in organizations. Thousand Oaks, CA: Sage.

28.

Westerman

J. W.

Bergman

J. Z.

Bergman

S. M.

Daly

J. P.

(2012). Are universities creating millennial narcissistic employees? An empirical examination of narcissism in business students and its implications. Journal of Management Education, 36(1), 5-32. doi:10.1177/1052562911408097