Teaching Real-World Applications of Business Statistics Using Communication to Scaffold Learning

Background

At Seattle University, all business majors are required to complete a two-quarter business statistics sequence, taught by economics department faculty. The introductory business statistics course covers descriptive statistics, probability, sampling, and hypothesis testing. The intermediate business statistics course covers analysis of variance, regression analysis, model building, time series analysis, and forecasting. The courses share the objective of developing students’ abilities to interpret statistical findings accurately and communicate them effectively. Students are encouraged to take the two-quarter series early in their program so they will be able to use the statistical tools in upper-level courses. Most students in the statistics sequence are sophomores or juniors who have already completed business calculus and a business communication course, as well as most of the university’s general education requirements.

An Early Failed Experiment With a Required Research Paper

Because our assessment projects left us unsatisfied with students’ ability to communicate statistical concepts, we turned to research on uses of writing in business disciplines. We found a variety of studies indicating that writing can improve student understanding in a variety of business disciplines, including statistics (Becker, 1998;Hulsizer & Woolf, 2009;Smith, 1998;Stromberg & Ramanathan, 1996), accounting (Riordan, Riordan, & Sullivan, 2000), economics (Crowe & Youga, 1986;Palmini, 1996), and finance (Carrithers & Bean, 2008). To improve communication, we decided to introduce a research paper in the intermediate statistics course with specified formal requirements (length, format, citation style). Students were instructed to find a real-world business issue, define it, and analyze it qualitatively and quantitatively using statistical tools developed in the course. Students had 4 weeks to complete the assignment, which was worth 20% of the course grade.

Unfortunately, this assignment did not achieve our intended goals. When faced with a large project, students struggled to define specific business issues that could be answered with available data. Although students learned useful information about their topics, most did not effectively apply the statistical tools developed in the course or communicate their findings professionally. Many of the completed projects were poorly structured with long introductions and extensive background information followed by a weak analysis and a jumbled write-up of the results. Others presented stronger analyses, but their papers were filled with statistical jargon that would be unintelligible to a typical business audience. These were exactly the type of results our advisory board had complained about—meager or even faulty analysis and poor communication. Additionally, students’ choice of different topics combined with their frequent failure to demonstrate appropriate statistical modeling and analysis made the papers onerous and time-consuming to grade. Overall, the research paper assignment failed to promote better understanding and communication of statistics concepts and frustrated both students and instructors.

Learning Theory, Scaffolded Assignments, and Backward Design

In hindsight, the failure of the major paper assignment can be explained as a failure to recognize how novice learners develop disciplinary expertise. To improve our pedagogy, we decided to study recent developments in learning theory and research. Learning theorists have shown that novices’ difficulty in “thinking like an expert” comes from their inadequate storage in long-term memory of the discipline’s domain-specific conceptual knowledge as well as its procedural knowledge or schemata that enables disciplinary professionals to address new problems (Bransford, Brown, Cocking, & National Research Council Committee on Developments in the Science of Learning, 2000;Kellogg & Whiteford, 2009;Willingham, 2009). These procedural schemata, developed through extensive and repeated practice, eventually become stored as meaningful constructs in the neural networks of long-term memory (Kirschner, Sweller, & Clark, 2006). Pioneering studies with chess players show that chess experts draw on a retrievable storehouse of game situations and associated strategies built into long-term memory through many hours of practice. Similarly, novice business students need many hours of practice at recognizing recurring patterns within real-world problems and applying both logic and experimentation to determine the models and tools that best match each pattern.

To develop this recognition of patterns, students need guided practice on applying algorithmic knowledge to real-world problems that gradually increase in difficulty. Much of the literature on learning builds onVygotsky’s (1978)concept of thezone of proximal development—that is, a learning zone that is just beyond students’ current skill or knowledge level. If a task is too simple or too much within their comfort zone, students get bored. If it is too difficult, they flounder or give up. A problem that is “just right” creates significant cognitive struggle for learners but within an environment where the instructor can provide progressively more difficult assignments and intervene at the right moments to provide expert guidance. The scaffolding metaphor derives from Vygotsky’s instructional method of providing a sequence of assignments that gradually increase in difficulty, while allowing instructors to gradually withdraw help (Puntambekar & Hübscher, 2005;Reiser, 2004).

Insights from learning theory helped the business statistics faculty realize that the initial lengthy research paper assignment was too difficult for novice statisticians in that it violatedVygotsky’s (1978)zone of proximal development. Students needed intermediate help before they could succeed at top-of-the-wall assignments like the research paper. In designing our new sequence of assignments, we applied the principle ofbackward design, which asks instructors to focus initially on the end goals of the course and design the last assignment first (Bean, 2011a;Wiggins & McTighe, 2005). Instructors could then develop earlier scaffolding assignments to help students build the skills needed for the final assignment.

The Assignment Sequence

We now turn to a more detailed description of the assignment sequence designed to move students from algorithmic competence to successful analysis of real-world problems (seeFigure 1).

OPEN IN VIEWER

Figure 1.

Assignment sequence by stages.

AsFigure 1shows, the sequence begins with well-structured algorithmic problems and concludes with an ill-structured real-world problem that requires students to communicate results to specified audiences within standard business genres. The key to the sequence is the intermediate scaffolding assignments that mix well-structured and ill-structured elements to bridge the gap between algorithmic and real-world problems. The scaffolding problems help students understand the relevance of the statistical methods and tools they have learned through doing algorithmic calculations and also include low-stakes writing tasks that ask students to interpret results in nontechnical language for a lay audience. The sample assignments that follow were designed for an intermediate business statistics course that focuses on regression analysis. We have designed similar assignments for introductory business statistics courses and think that similar assignments could be designed for courses in other analytic business fields such as accounting, economics, and finance.

The Scaffolding: Mixed-Structure Problems That Blend Algorithmic With Real-World Elements

Mixed-structure problems comprise an essential element of our sequence of scaffolded assignments. We call them mixed-structure because they blend well-structured algorithmic elements with ill-structured real-world elements that require effective communication. We treat these mixed-structure problems as mostly low-stakes assignments that promote learning. Their function is to help students recognize recurring patterns in real-world problems, select and use the appropriate tools to match the patterns, determine relevant and plausible data for calculations, and communicate to a targeted audience the meaning of the results. Regular use of mixed-structure problems, both as homework and in-class discussion assignments, enables students to build up the statistical reasoning and communication abilities required for our “top-of-the-wall” real-world assignments. To develop mixed-structure problems, we draw on current business issues involving statistical applications, thereby making them interesting and relevant.Figure 2presents an example of a mixed-structure problem.

OPEN IN VIEWER

Figure 2.

Mixed-structured problem.

Collectively, the questions in this mixed-structure problem lead students to appreciate the way statistical tools can be used to help make business decisions. These questions engage four key elements needed for real-world problem solving:

Basic understanding of the research process, establishing a set question and outlining concerns and preconceptions (Part A)

Basic calculations in which students have to select the variables and analyze them using different model formulations (Parts B and C)

Understanding and interpretation of statistical tools in technical terms (Part D)

Interpretation, evaluation, and communication of results in lay terms (Part E)

We have developed a sequence of similar mixed-structure problems to use throughout the term. A typical weekly homework assignment includes algorithmic problems from the text along with one or two mixed-structure problems. Students are asked to come to class ready to share their calculations and written responses. In class, we use small groups or whole class discussions to deepen students’ thinking and help clear up misunderstandings about calculations and interpretations. Students often propose alternative approaches to a mixed-structure problem, creating teachable moments where the instructor can give examples of what would have to happen for each case to apply (for further discussion of the value of alternative cases in promoting deep learning, seeMarzano, Pickering, & Pollock, 2001).

Top of the Wall: Communication Assignments Addressing Ill-Structured Problems

Having climbed part way up the wall by using mixed-structure assignments as scaffolding, students are ready to ascend to the top. We have developed two kinds of top-of-the-wall assignments, both of which seem to work equally well. The first is a recommendation memo using an instructor-provided “messy” data set. The recommendation memo format was suggested by members of our advisory board and job recruiters as a common genre in business writing. The second kind—a statistical consulting project—asks students to gather their own data set, perform an analysis, and write a short report responding to a question posed by a client. This genre—the business report—is also common in business settings. Each assignment achieves the goal of having students apply their statistical knowledge to a real-world problem and communicate their conclusions to a targeted audience in a business genre.

Option 1: Recommendation Memo

The recommendation memo assignment is more complex than the mixed-structure problems but is built on the kinds of thinking students have already practiced. In contrast to the original end-of-term research paper, which overwhelmed students with too much freedom, this assignment is less open-ended. It provides students with an already collected, but messy, data set and poses to all students the same ill-structured problem while specifying an audience, purpose, and genre. The recommendation memo challenges students to determine the research question, select the appropriate variables, develop the best model, apply relevant statistical tools, interpret the statistical results, write a technical appendix, recommend a course of action, and summarize the analysis in plain English. An example of a recommendation memo assignment is found inFigure 3.

OPEN IN VIEWER

Figure 3.

Recommendation memo assignment.

Option 2: Statistical Consulting Project

An alternative top-of-the-wall assignment is the statistical consulting project. Because this project requires more steps than the recommendation memo, it includes its own stage-by-stage scaffolding in accord withVygotsky’s (1978)“zone of proximal development.” The consulting project assignment asks students to produce a consulting report with an introduction, data section, results section, and conclusion, as well as a letter to a client summarizing the implications of the results in a nontechnical way. The structure of the consulting report is modeled on the sample paper in “Sample Paper in Econometrics” (Dvorak, 2007) available to students on the course website. The handout given to students is reproduced inFigure 4.

OPEN IN VIEWER

Figue 4.

Assignment directions handout for consulting project.

As shown in the “Deadlines” section, in the real estate pricing assignment, we ask students to develop their consulting report in incremental stages that focus first on the data, then on the technical results, and finally on the nontechnical letter to prospective house buyers. In contrast to the recommendation memo, the consulting project has scaffolding built into the project with incremental deadlines and feedback as sections of the project are completed. This incremental scaffolding is crucial. As we explained earlier, our first research paper assignments were ineffective at promoting deep understanding of statistical tools because they moved students too quickly beyond their zone of proximal development. In the original research paper, many students turned in projects with wordy introductions and background sections, minimal or inaccurate statistical analysis, and lengthy, wordy conclusions. These unsuccessful papers had an “hourglass” shape: a wide introduction full of generalities, a narrow body where the students presented their statistical analysis and results in a thin, undeveloped way, and a wide conclusion where students attempted to fill out their assignments with wordy and sometimes irrelevant and unsupported conclusions. By scaffolding the project into stages, we helped students write an “egg-shaped” paper: wide and substantive in the middle, where the data, methodology, and results are presented. We have found that this process encourages deeper learning of business statistics, increases student engagement in the task, and enables most students to achieve success.

Improvements in Student Learning

Our observations of students’ performance, as well as departmental assessment projects in upper-level courses, suggest that scaffolding assignments are successful at promoting deeper learning. By linking algorithmic procedures to a variety of real-world contexts, the assignments help students develop expert schemata for addressing real-world problems. For example, one of the early mixed-structure homework problems led students to a surprising revelation—that professionals use the practical and statistical significance of regression coefficients to make recommendations about business decisions. Although students could calculate coefficients, they understood them only as quantifying a relationship between independent and dependent variables. It was an “aha” moment for students to see that the statistical results of the coefficient could have real-world meaning. For example, in the age discrimination case (presented above as “Option 1: Recommendation Memo”), students found that education had a more practical and statistically significant effect on hiring than did age. Through a comparison of coefficients, they could say that although age was relevant to the litigation case, the defense could argue that education had a bigger influence—highlighting the challenges the lawyer might encounter if he or she were to accept the case. This new understanding illustrates nicely the difference between surface learning and deep learning. At a surface level, students had internalized the procedures for calculating the coefficient, but they did not understand what the coefficient meant. The additional steps of interpreting results and making recommendations required students to stretch their understanding further.

Surface learning becomes deep learning when this memorized algorithmic procedure gets linked to meaningful knowledge already existing in the learner’s long-term memory (Entwistle, 2009). In the case discussed above, a student’s awareness of factors that might lead to firing and rehiring is stored in the brain as part of the student’s overall memory of life experiences. But the learner’s memory is not linked to the newly encountered conceptcoefficient. Recognizing that the coefficient can measure something meaningful about firing and rehiring reorganizes the way coefficient is stored in the student’s long-term memory. As brain scientists would put it, the neural network associated with “coefficient” has become more richly elaborated, dense, and interconnected with other neural networks (Kirschner et al., 2006). Other business statistics concepts or routine calculations that have been made meaningful through the scaffolding assignments also include these examples:

These examples are based on in-class discussions following the completion of mixed-structure scaffolding problems. Prior to our use of these assignments, class time was mostly devoted to lecture, working problems on the board, and student questions. The discussion of mixed-structure problems opened up new dimensions of student thinking.

Another illustration of the effectiveness of the scaffolding assignments in promoting deep learning is found in students’ salutary struggle to select a relevant set of variables for use in regression assignments. Algorithmic problem sets typically provide tidy, unambiguous data sets. In contrast, real-world problems require business professionals to identify information that is relevant to the issue based on feasible approaches and identifying potentially available data. Often, in fact, extensive research may be needed to gather the relevant data—a process simulated in many of the mixed-structure homework problems and required in the “top-of-the-wall” assignments. Students’ difficulty determining appropriate variables was particularly revealed in “Option 2: The Statistical Consulting Project.” Students were asked to develop a model that predicted a home’s price using a subset of a home’s characteristics such as square footage, number of bedrooms, number of bathrooms, lot size, and so on. Because there is typically a high degree of correlation between a home’s square footage, number of bedrooms, and number of bathrooms, the inclusion of all of these variables in the model led to counterintuitive results. For example, in some cases the results indicated that an additional bedroom reduced the predicted price of a home, holding other variables constant. As students tried to make sense of these results, they begin to develop an understanding of the impact of multicollinearity on regression results. Students were forced to make choices about which variables to include in their final model, choices that do not have unambiguous, right solutions and therefore require justification. Students were surprised to arrive in class with different solutions to the same problem, some of which were equally justifiable. The communication portion of the assignment helped students to better understand the appropriateness of the different approaches, highlighting the importance of communicating the results persuasively. Students’ struggles to select variables and make sense of their results indicate how hard it is to move from well-structured algorithmic problems to real-world problems where more than one solution may be valid.

Another positive result of the communication assignments is students’ increased engagement with analytical methods for solving problems. Because the “mixed-structure” homework assignments and the “top-of-the-wall” writing assignments all position the student as professionals in training working on meaningful assignments, the assignments produced high-energy class sessions with vigorous disagreements and lots of hands in the air. In fact, increased engagement is one of the characteristics of deep learning identified by learning researchers (Entwistle, 2009;Nelson Laird & Garver, 2010). These studies have shown how surface learning (memorization of unconnected facts, definitions, and algorithmic procedures) is motivated primarily by students’ desire for good grades, whereas deep learning is motivated by students’ intrinsic interest in the subject matter. Rather than being passive memorizers of information, our students began seeing themselves as agents with the ability to solve meaningful problems.

Although many contemporary statistics textbooks have evolved to include application examples similar to the mixed-homework problems, they are not designed explicitly to scaffold students’ movement from well-structured to ill-structured problems. It is the sequencing of assignments that makes the difference. For example, the consulting project requires students to draft the results section of the paper 2 weeks prior to the paper’s final due date. Students typically arrive (breathless) to class on the date that the results section is due with more questions than answers about what their many pages of regression results are trying to tell them. It is truly rewarding to hold a class discussion at this stage: Most students have come to class highly engaged in the project but with much uncertainly remaining about how to choose one model over another, how to deal with a counterintuitive result, or how to interpret their findings. A great deal of learning takes place by allowing students to discuss their questions in small groups and/or as a whole class. The key to the success is that the scaffolding assignments give students repeated practice at wrestling with contextual ambiguity, determining their own solutions to problems, and making sustained arguments that depend on a deep understanding of quantitative tools.

Faculty also found that they could improve engagement by applying new statistical tools to problems covered in previous mixed-structure homework assignments. For example, the local car dealer mixed-structure problem discussed earlier could also be used to introduce analysis of variance or forecasting in a later class. Returning to problems they were familiar with helped students appreciate the function and value of different tools. Seeing different tools applied to different dimensions of the same real-world problem helped students develop a more elaborate and integrated understanding of tools and to appreciate why different situations require different techniques.

Another indicator of increased learning is the types of questions we are now able to ask on exams. Prior to our scaffolding approach, exams consisted of algorithmic problems with varying degrees of difficulty. We now construct exams that supplement algorithmic problems with analytic and interpretive questions applied to business settings. Recent test scores show no decline in students’ performance on the algorithmic problems, suggesting that the increased focus on interpretation and communication has not lessened students’ mathematical competence. Interestingly, although the test means and medians have not declined, the distribution of grades has flattened with more As and Fs and fewer Cs. We attribute this change to some students’ moving out of the C range into the A or B range because the scaffolded assignments created more engagement and deeper learning. Conversely, some C students went the other direction because the more difficult exams tested knowledge that required more than plugging numbers into a formula.

Finally, students themselves testify to the value of communication assignments. In their course evaluations, many identified the sequence of scaffolding assignments as the best portion of the course. Students noted that applying analytical tools to business or policy issues helped them better understand what professionals do and why quantitative analysis is relevant to decision making. Anecdotal evidence suggests that students retained the skills developed in completing the assignments. For example, former students reported to us that later on, in interviews for jobs and internships, they were able to describe their analytical writing projects when asked for examples of their proficiency in business statistics and communication. Professors teaching subsequent courses also found students more prepared to understand empirical arguments, to generate statistical analysis, and to communicate their results in an audience-appropriate manner.

Practicalities of Adopting Scaffolding Assignments

The major claim of this article is that scaffolded communication assignments can enhance students’ learning of analytical tools while also improving their skills at audience-appropriate communication. Unfortunately, there are costs of including the communication assignments. One cost involves the time needed to create the assignments. One possibility is for instructors to use data provided in textbook problems but to embed these data in a richer story and change the questions to require discernment regarding statistical techniques and audience-targeted exposition of results. The end-of-chapter data and case studies provided in some textbooks can also serve as a starting point for building scaffolding assignments. Another possibility is to create scaffolding assignments that require students to gather their own background information or data. For example, one problem we use to teach a statistical technique asks students to research the prices of books currently required in undergraduate business courses and then compare the prices of textbooks sold at the university bookstore and two online vendors. Though the data are relatively easy to find, students still have to make and justify decisions on which data they selected and why, making the task a mixed-structure problem without significant instructor effort.

Another potential cost is the increased workload for students. The fact that we did not reduce coverage of material or the number of exams to include the new communication assignments indicates that our revised course placed more burdens on students. However, based on recent findings fromArum and Roksa’s (2011)well-publicizedAcademically Adrift: Limited Learning on College Campusesthat students report studying only 12 to 14 hours per week for all of their courses, the faculty did not feel that increasing the student workload was a bad thing.

The most significant cost to faculty was the burden of grading the final writing assignment. However, the redesigned assignments were significantly less onerous to grade and more enjoyable to read than the initial failed research assignment. Because all the students were writing to the same assignment, it was easier to distinguish among strong, middling, and weak papers. Faculty adopted the use of a task-specific grading rubrics (Appendix A) and check sheets (Appendix B), which reduced grading time significantly and increased grading consistency as well (for a discussion of “task-specific” rubrics, seeBean, 2011b, pp. 267-289). The rubric and check sheet made providing feedback easier by identifying criteria specified in the rubric. Faculty also instructed students to use the rubrics and check sheets with partners to provide feedback to one another on their projects based on the criteria provided. Though grading time did increase, faculty felt it was a worthwhile trade-off to promote deeper learning.

Conclusion

In this article we explain how we have applied ideas from pedagogical research on transfer and assignment design to the problem of deep learning in a core business statistics course. We argue that embedding quantitative problems within business communication contexts—where solutions have meaning for real-world audiences—promotes enhanced understanding of algorithmic processes. We use scaffolding to move students from a novice cognitive framework to a more expert framework. Based on our experience, we believe that the deep learning benefits of using scaffolded communication assignments outweigh the costs. We have argued in this article that instructors of quantitative subjects such as accounting, business statistics, economics, and finance can draw on learning theory to give students guided practice in analytical techniques—practice that transforms course knowledge from memorized definitions or formulae into expert schemata for problem solving.

We began with a goal of teaching students to address real-world problems in a professional manner. We then designed assignments to build their abilities within their zone of proximal development to prepare them for our top-of-the-wall assignments. Mixing algorithmic and communication elements in the scaffolding assignments serves as the bridge to develop deep learning and understanding of the quantitative methods in business statistics. The scaffolded communication assignments promote a richer, more elaborated understanding of analytical techniques without compromising students’ algorithmic skills. The assignments create a positive feedback loop that teaches students to communicate the results of their analysis effectively while at the same time deepening their understanding of analytical techniques. Moreover, we believe that this deeper understanding of analytical methods developed through scaffolded communication assignments is more likely to be retained meaningfully in long-term memory and more likely to be transferred into students’ upper-level courses and professional careers.