Qualitative Oral-Presentation Feedback: Comparisons from Business Professionals,Instructors,and Student Peers

Abstract

Previous studies compare quantitative feedback ratings of student peers and instructors, but new presentation-feedback technologies enable qualitative-feedback comparison. This study extends previous research by comparing qualitative feedback and business professionals’ feedback. Compared to the professionals, the instructors provided similar feedback types and sentiment; students, however, de-emphasized message delivery and made fewer suggestions for improvement. The results suggest that students may need additional practice in critiquing message delivery and in suggesting needed improvements in their peers’ oral presentations. The study also provides a methodology using the new technologies for instructors to calibrate their own and their students’ feedback with professionals’ feedback.

Keywords

oral presentations peer feedback instructor feedback assessment

The use of peer feedback could offer opportunities for instructors to enhance the assessment of oral presentations in business communication courses. In a study comparing peer and instructor feedback on oral presentations, Campbell et al. (2001) observed that, in comparison to feedback on written documents, business communication courses afford students few opportunities to receive feedback on oral presentations because the instructors do not have enough time to assess numerous presentations. In a recent survey, additional evidence for this observation is provided by Moshiri and Cardon (2020), who reported the greater number of written assignments than oral assignments in business communication courses. Campbell et al. (2001) investigated the usefulness of peer feedback on oral presentations as a way to provide students with more oral presentation feedback without adding to instructors’ assessment burden. For peer feedback to be useful, Campbell et al. argued that it would give “detailed, individualized instruction” for improvement and provide presenters with “a realistic view of their performance quality . . . to predict how the instructor will evaluate them” (p. 24). Campbell et al. and additional studies (L. De Grez et al., 2012; Freeman, 1995) found that averaged quantitative peer ratings tend to be equivalent to or lower than averaged quantitative instructor ratings, and that these ratings correlate positively, but in a low to moderate way. Although the ratings in these studies were not highly reliable, the results in their study prompted Campbell et al. to remark that “peer evaluations offer considerable promise” for offering additional feedback that would help the presenters become more prepared to present for an instructor (p. 57).

Although these studies suggest potential for achieving consistent quantitative ratings between peers and instructors, few studies have investigated qualitative feedback, a type of feedback that emerging oral-presentation feedback software affords (Lee, 2020). Further, studies have not triangulated their findings with professionals’ feedback, which could be helpful for students who are seeking to predict not only their instructor’s evaluation but also their employer’s evaluation (Campbell et al., 2001). Insight into how the qualitative feedback of business professionals, instructors, and peers compares could be helpful for numerous reasons, including to help instructors and students calibrate their presentation assessment according to professionals’ perspectives, which is the focus of this study. Therefore, the present study seeks to extend previous studies on peer and instructor quantitative feedback by comparing the qualitative feedback of business professionals, instructors, and peers. Ultimately, such insight could help instructors know whether they can rely on students to provide qualitative feedback that could foster improvement and prepare their peers for instructor and professional evaluation.

Literature Review

Oral presentations play an integral role in the success of many professionals, such as business executives, entrepreneurs, government officials, engineers, and attorneys. Not surprisingly, employers associate oral presentations with communication skills (Coffelt, Grauman, & Smith, 2019) and report the importance of presentation skills in new hires (Coffelt, Baker, & Corey, 2016; Gray & Murray, 2011; Ortiz, Region-Sebest, & MacDermott, 2016). Given the importance of oral presentation skills in the workplace, business communication instructors consistently report the inclusion of oral-presentation instruction and oral-presentation assignments in their courses (Moshiri & Cardon, 2014, 2020; Russ, 2009). However, students appear to be receiving far more feedback on their written assignments than on their oral assignments. In their recent survey, Moshiri and Cardon (2020) found that over 90% of responding business communication instructors assign four or more written assignments, in comparison to 18% of instructors who assign four or more oral assignments. To aid student learning through these oral-presentation assignments, instructors provide their students with feedback, a topic frequently discussed in business communication research. This literature review will highlight this research as it relates to oral-presentation feedback sources and technologies.

Feedback Sources

Feedback is “information provided by an agent (e.g., teacher, peer. . .) regarding aspects of one’s performance or understanding” (Hattie & Timperley, 2007, p. 81). Feedback plays an important role both in communication and in learning as individuals incorporate feedback to close the gap between their performance and a given standard (Weiner, 1954). In professional situations, such as pitch presentations for entrepreneurs, feedback can take the form of dialogue with market stakeholders as entrepreneurs develop high-stakes presentations (Spinuzzi et al., 2014; Spartz & Weber, 2015). In the classroom environment, such dialogic feedback can also take place when audience members, such as instructors and student peers, critique student presentations (Campbell et al., 2001). For example, in one study, business communication instructors assigned students to write self-reflections to critique their performance; peers were also assigned to write replies that assessed the accuracy of the self-reflections and provided overall feedback on the presenter’s performance, including the presenter’s delivery (Smith, Schieber, & Austin, 2020). After four rounds of presentations and feedback, the results indicated that the self-reflections and peers’ comments focused most frequently on delivery as opposed to content, the self-reflection comments were more constructive than positive at first but became more positive after multiple rounds, and the majority of peers’ comments were always positive but became more positive over time. The researchers observed that over time, the comments became more “relevant and helpful,” advocating for giving students multiple opportunities to practice evaluating their own and their peers’ oral presentations throughout the semester (p. 118).

Outside of the business communication classroom in particular, studies across academic disciplines have found additional benefits of giving students opportunities to engage in peer feedback, including that students appreciate their peers’ feedback on their presentations (D. De Grez, 2010; Dollisso & Koundinya, 2011). Other studies have found that students improve their oral-presentation performance from peer feedback (Planas Lladó et al., 2014; Van Ginkel et al., 2017). Students also feel motivated to learn in the presence of peer feedback on oral presentations (Planas Lladó et al., 2014). Further, critiquing—an evaluative act underlying the giving of feedback—is considered a higher-order cognitive and learning process, according to Bloom’s revised taxonomy (Anderson et al., 2001). When critiquing, “a student notes the positive and negative features” and “makes a judgment at least partly on those features” (p. 84). Learning researchers have observed that “critiquing lies at the core of what has been called critical thinking” (Anderson et al., 2001, p. 84; see also Dyrud & Worley, 1998), so it is not surprising that learning occurs both with the students receiving the feedback and with the peers who watch the presentations and provide the feedback (Lee, 2020). However, in one study, students reported not trusting peer feedback, doubting the accuracy and validity of the feedback because they know it might not be consistent with that of the instructor (Planas Lladó et al., 2014).

To mitigate the mistrust, researchers have compared business communication student peers’ feedback with that of their instructors. Freeman (1995) compared 41 business student teams’ assessments of their peer teams’ oral presentations with the instructors’ assessments. Although he found no significant difference in the average scores assigned by teams and instructors, he did find a significant difference in the standard deviations of the two groups, with the peers’ standard deviations being half the value of the instructors’, meaning that instructors’ feedback varied more widely than that of student peers. Freeman also found the positive correlation between peer and instructor score to be moderate. Campbell et al. (2001) compared the self-, peer, and instructor feedback on team business presentations. In their study, peers and the instructor reviewed live presentations using holistic and analytic rubrics focusing on both content and delivery. The presenters themselves rated the recorded presentations using the same rubrics. All the evaluators were trained on the rubrics. Using the averaged quantitative feedback, the researchers found a positive, but moderate, correlation between the peers’ feedback and their instructors’ feedback. The researchers concluded that, based on their results and given adequate training, peer evaluations do have potential to help students prepare for instructors’ evaluations.

In a related study, L. De Grez et al. (2012) analyzed self-, peer, and instructor feedback on individual video-recorded student presentations given in a business administration course. Individual instructors were randomly assigned to evaluate the presentations. Business administration peers originated from a different course focused on communication. The researchers compared three summed scores, one each provided by the individual presenter, the assigned six peers, and the assigned individual instructor. The reviewers assessed the presentations using a quantitative rubric focused on content, delivery, and overall criteria. On average, the peers’ feedback scores were significantly higher than those of the instructors, but the researchers found an “acceptable but low reliability level” between the instructors and peers (p. 136). These studies suggest that averaged quantitative peer feedback tends to be equivalent to or higher than averaged instructor feedback, and that the two types of feedback correlate positively in a low to moderate way.

Although it is appropriate for business communication instructors to compare student peers’ quality assessments of oral presentations with their own (because instructors ultimately set the grading criteria for their students), Campbell et al. (2001) suggest that it could be of value to have assessments made “by a variety of business professionals” (p. 29). Although Campbell et al. do not elaborate exactly on what the value of business professionals’ assessments would be, their study’s focus on assessment suggests that at least part of the value would be to help instructors better prepare students for giving presentations in the workplace. Specifically, instructors could compare their own feedback with the professionals’ feedback, ensuring that as students aim to perform well for their instructors’ assessment, the students are concomitantly preparing to perform well in the workplace. Without such calibration, students could perform well for an instructor, yet not perform well in the workplace.

Despite the potential calibrating benefits, research on oral presentation assessments of business professionals is almost nonexistent. Only two such studies were discovered in our related literature search—both from entrepreneurship research. The first was conducted by Mason and Harrison (2003) and illustrates the need for strong content in presentations. The study involved 30 business angels who watched one videotaped pitch for funds by a member of a management team. The angels wrote their real-time reactions on cards, and then after watching the presentation, they indicated their investment interest and the reasons for their interest. The researchers categorized the resulting real-time impressions into categories, with the greatest number of comments falling into a category relating to presentational issues (e.g., style, content, and structure), and the rest falling into nonpresentational categories (e.g., market, product, deal structure). Of the comments focusing on presentational issues, over two-thirds focused on what could be considered content-related issues (“general content,” “technical content,” “focus/structure,” and “illustrations”), and about one-third focused on what could be considered noncontent- or delivery-related issues (“presentation style” and “presentation”) (p. 35). The vast majority of the statements were negative, so not surprisingly, most of the investors indicated no interest in investing. The majority of the reasons given for not investing focused on the failure of the presenter to provide adequate or clear information. Although presentation delivery was noticed and evaluated by these investors, presentation issues related to content seem to be the main reason for their investment decisions. Further, many non-presentational issues related to the need for additional information about the market or product, further indicating that the content presented influenced the investors’ decisions.

The second study was conducted by Clark (2008) and provides a contrasting view about the importance of delivery. That study involved a group of 24 business angels who watched the pitches of three videotaped entrepreneurs. After the pitches, the angels quantitatively rated (on a 7-point scale) 32 presentational (i.e., structure, style, and delivery) and non-presentational (i.e., company, marketing, producing, funding) issues. The study synthesized the quantitative ratings with the angels’ qualitative reasons for investing or not investing. The resulting data showed the angels’ level of investment interest was significantly related to their overall mean score of the presentational and nonpresentational ratings―the higher the overall score, the higher their investment interest. Presentational factors, such as style of delivery, tended to have the greatest impact on the overall scores, although the angels’ qualitatively stated that their investment decisions focused on non-presentational substance (content) factors. Similar to the Mason and Harrison (2003) study, qualitative comments that focused on presentational factors were negative, although the pitch that garnered the most investment interest did receive more positive than negative comments.

The Mason and Harrison (2003) and Clark (2008) studies provide a valuable glimpse into how some professionals qualitatively react to and evaluate oral presentations. Further, Clark’s study provides valuable insight into a potential discrepancy between professionals’ qualitative and quantitative evaluations. However, similar to the Mason and Harrison study, the Clark study blurred the categorization of content issues—the organization of overall content and the completeness of overall content were considered presentational issues, whereas the completeness of specific content (e.g., information related to market, product, and funding) was considered a nonpresentational issue. The categorization of what content is and how the content is delivered is surely a difficult methodological task, but a clearer delineation of content versus non-content factors could shed additional insight into professionals’ feedback. In addition, although the Mason and Harrison and Clark studies valuably gathered feedback from professionals, the focus of their studies did not involve the comparison of professionals’ feedback with instructors’ and student peers’ evaluations. For instructors and peers to be able to calibrate their feedback with professionals’ feedback, this comparative element is critical. Furthermore, the presentations the professionals evaluated in the Mason and Harrison and Clark studies differed, implying that the feedback of the professionals would similarly differ. Unless the presentations evaluated are held constant or the professionals’ feedback itself is held as the constant to which instructors’ and student peers’ feedback is compared, then little information is gleaned from professionals’ feedback alone for calibrating instructors’ and peers’ feedback to that of the professionals. Therefore, this additional comparative element is important for yielding insight to aid business communication instructors and students in their assessment processes.

Although few studies exist that examine business professionals’ assessments of oral presentations, researchers and instructors employ myriad other methods to prepare students for anticipating audiences’ responses to their business and professional communication. Some research seeks to better understand the oral communication skills professionals seek in new graduates through surveys (Coffelt, Baker, & Corey, 2016; Gray, 2010; Ortiz et al., 2016), interviews (Coffelt & Smith, 2020; Coffelt, Grauman, & Smith, 2019), and integrative literature reviews (Keyton et al., 2013). Other instructors employ classroom methods such as the case method (Robles & Baker, 2019; Rozumalski & Graves, 1995), client projects (Pope-Ruark, 2011), usability testing (Jameson, 2013), or think-aloud protocols (Schriver, 1992; Stratman, 1991).

For example, in one study focused on audiences’ responses to written documents as opposed to oral presentations, Schriver (1992) reported how instructors taught students in control classrooms to anticipate readers’ needs through audience-analysis activities, including peer critique and peer review. In the experimental classrooms, students were asked to review written documents and predict and diagnose readers’ potential problems with the texts. The students in the experimental classrooms were then provided written transcripts of think-aloud protocols generated by audiences who read texts and described their experience understanding them. The students in the experimental classrooms then compared their predictions with the readers’ actual responses. Pretests and posttests were administered to both the control and experimental groups to assess who could detect and diagnose the readers’ problems. The results indicated that students in the experimental classrooms improved significantly more than students in the control classrooms in their ability to detect and diagnose accurately the readers’ problems. Notably, the students in the study improved the most in their ability to anticipate problems focused on readers as opposed to problems focused on the text, suggesting an improved awareness of the audience specifically (Schriver, 1992, p. 198).

Such studies are helpful for instructors to help students develop the competency of audience awareness and for instructors to anticipate the oral communication skills students will need in the workplace, yet additional studies focusing on oral presentations specifically and on business professionals’ feedback on oral presentations could yield insight more particularly focused on the oral presentation genre. Importantly, recent advances in feedback technology (Lee, 2020) could make the process of gathering professionals’ feedback on oral presentations easier.

Feedback Technologies

Feedback technologies can provide either instantaneous or postponed feedback. For example, feedback is “delayed” when readers provide feedback on a person’s written word, but feedback is “immediate” when it originates from audience members attending a live oral business presentation (Suchan & Dulek, 1988, p. 30). Because of the asynchronous nature of writing, audience feedback is given after the writer disseminates the words. In contrast, an oral presentation is a synchronous communication, with a speaking presenter and a listening/viewing audience whose body language and comments can immediately convey their reaction (Suchan & Dulek, 1988, p. 30). This synchronous aspect is eliminated, however, for audiences who provide feedback through written rubrics or through other delayed means after watching digital video recordings of presentations, practices commonly found in business communication courses today (Campbell et al., 2001; Leeds, Ravel, & Brawley, 2007; Kenkel, 2011; Lee, 2020; Lucas & Rawlins, 2015). In recorded-video environments specifically, the live nature of the presentation can be simulated, but audience members’ delayed feedback based on recorded presentations becomes more like the delayed feedback of readers of the written word.

From a pedagogical perspective, delayed feedback does not afford the same dynamic presence of a live, embodied audience, but new technologies are substituting the immediacy of the live feedback with simulated live feedback or highly contextualized delayed feedback. Lee (2020) described PitchVantage software that enables a student to practice presenting in front of a digital, simulated audience that provides real-time feedback on ten presentation delivery features. The “automated” feedback features include “pitch variability, pace variability, volume variability, verbal distractors, pauses, pace, long pauses, engagement, volume and eye contact” (PitchVantage, n.d.). In contrast to PitchVantage’s automated feedback, other technology, such as GoReact, enables real (i.e., nonautomated) audiences watching a live or prerecorded presentation to give “time-coded comments” to presenters (Lee, 2020, p. 56). Instructors and peers can provide the time-coded feedback during the presentation (Lee, 2020), and the presenters can view the feedback after the presentation when they review their recorded presentations (see Figure 1). The feedback comments can be given as text, video, or audio (GoReact, n.d.). Lee (2020) observes that this time-coded feedback enables student presenters to “learn faster” as they view their feedback “at exactly the right moment” in the presentation (p. 56). Although these technologies do not exactly substitute for the embodied response of a live audience, their innovative approaches enable students to receive immediate, but automated, responses from simulated audiences (as in the case of PitchVantage) or delayed, but highly contextual, responses from real audiences (as in the case of GoReact). In addition, because these technologies are web based, professionals could be recruited to respond to student presentations without the inconvenience of being away from their offices or other workspaces. Unfortunately, little research exists on qualitative, contextual responses, particularly on the content and sentiment of the textual comments provided by the audiences, whether those audiences be professionals, instructors, or students.

Figure 1.

Example of GoReact interface with time-stamped feedback.

Before such technologies existed, business communication researchers proposed other methods for assessing oral presentations, including rubrics—either holistic or analytic. Holistic rubrics contain a single scale, with all the evaluation factors considered as one whole. Analytic rubrics usually break out the various grading factors for segmented grading into content (message) factors―such as substance, structure, and organization―and noncontent (messenger, delivery, form) factors―such as self-confidence, eye contact, gestures, and voice qualities (Campbell et al., 2001; Baker & Thompson, 2004; Clark, 2008; L. De Grez et al., 2012; van Ginkel et al., 2017).

Further, assessment can function either for formative or for summative purposes. For formative purposes, assessment typically entails detailed feedback that can be used by presenters to improve their subsequent performance, and such feedback often is given for early work in process before a final assignment is due. In contrast, assessment for summative purposes often provides only an overall grade or score, and typically occurs when providing a grade on the final assignment, without providing detail to justify the grade or score. Such purposes may blur when instructors provide more detailed feedback on a final assignment in hopes that students can use it in future applications, such as for scaffolding between assignments (LeFebvre, 2016), encouraging revision and resubmission (Anderson et al., 2001), or transferring knowledge to the workplace (Murdock, 2017). Campbell et al. (2001) conclude that formative assessment is better when it is analytic and contains more specific suggestions. It gives presenters more information about what improvement is needed, which is especially helpful if the assignments are part of a scaffolded course where the feedback from one assignment is useful in preparing for the next assignment. Further, Moreno (2004) suggests that the lack of specificity in summative, holistic grading leads students to perceive the general feedback as useless. However, Campbell et al. (2001) state there is little agreement on what aspects must be analyzed to help presenters improve. Moreover, Miller (2003) found that larger numbers of items on an evaluation checklist increase the amount of variance among raters’ scores. Compared to analytic assessment used for giving formative feedback, holistic assessment is more acceptable for summative evaluation, because it is easier and more efficient for the evaluators (instructors). In the everyday workplace environment, communication rubrics generally are not used, except, perhaps, by firms that specialize in oral communication training (Pittenger et al., 2004). Nevertheless, even though typical professional audiences might not have an internalized structured rubric to guide their evaluation of presentations, they still have some seemingly innate basis for evaluating what they hear, see, and feel during presentations.

To attempt to mimic this professional evaluation process and to extend previous business communication research on oral presentation feedback, the current study used an unstructured approach to assess the effectiveness of oral presentations; that is, the evaluators used no rubric, nor were they coached in any way as part of the study, but rather evaluated based on their own mental and emotional perceptions throughout the presentation. Specifically, the study contributed to business communication research by analyzing the qualitative feedback provided by three audiences (professionals, instructors, and student peers) as they reviewed business presentations, as opposed to the summative quantitative feedback from instructors and students analyzed in many previous studies.

To better understand the content of the qualitative feedback, thereby extending previous business communication research focused primarily on quantitative ratings, the study posed its first research question:

Research question 1: What analytic factors do business professionals, instructors, and student peers comment on in their qualitative feedback on oral business presentations?

To better understand how the qualitative comments made by business professionals compare to those made by instructors and student peers, thereby extending previous business communication research focused on instructors’ and student peers’ evaluations, the study posed its second research question:

Research question 2: How do the analytic factors commented on compare for business professionals, instructors, and student peers in their qualitative feedback on oral business presentations?

To better understand how the sentiment of the qualitative comments of business professionals compare to that of instructors and student peers, thereby extending previous business communication research focused on professionals’ and student peers’ sentiment, the study posed its third research question:

Research question 3: How do the evaluations of what went well (compliments) in the presentations and what could be improved (suggestions) compare for business professionals, instructors, and student peers?

Methods

To answer these questions, we conducted an exploratory study to collect and content analyze qualitative feedback on two business presentations given by student teams. We received IRB approval (IRB: X14355) for the study to recruit participants, to secure permission to use the recorded video, and to analyze the data.

Participants

A convenience sample of participants was recruited from a pool of adjunct business communication instructors at a university in the western United States and from a pool of students who had taken a business communication course from one of the researchers. We minimized the influence of peer familiarity with the presenters by recruiting students who had taken the business communication course two years after the presentations were recorded. Business professionals were recruited from the professional network of one of the researchers. Although convenience sampling reduces the generalizability of the study’s results, convenience sampling has been used by other business communication researchers investigating oral-presentation feedback (Campbell et al., 2001; Freeman, 1995) and can be appropriate for exploratory studies such as this one (Wrench et al., 2013). In total, 10 business professionals, 10 business communication instructors, and 10 student peers were recruited to participate in this study.

Instruments and Data Collection

Similar to Campbell et al.’s (2001) methodology, participants provided feedback on two video-recorded business presentations given by student teams. Each team included four presenters, with the first team’s presentation lasting 13 minutes 43 seconds and the second team’s presentation lasting 13 minutes 55 seconds (excluding question-and-answer time that was not evaluated in this study). The teams had conducted research on online customer complaints logged against an existing large corporation chosen by the teams (each team chose a different corporation), and they presented their methods of analysis, findings, and recommendations based on their analysis. All student presenters consented to their team’s video being used for research purposes.

The 30 participants providing feedback on the two presentations were asked to respond to each presentation using GoReact software by typing real-time, time-coded text comments as they watched each presentation (Lee, 2020; GoReact, n.d.). This process provided the qualitative comments for this study’s analysis. The participants were invited to give suggestions for improvement as well as compliments. Each of the feedback providers watched the videos alone in a quiet office setting, thus minimizing outside influences on their comments. The presentation automatically paused as the feedback providers typed their comments, ensuring that participants did not miss any presentation content while they typed. The data generated by this research consisted of 1,067 short, impromptu typed comments. To prepare the data, we systematically unitized the comments based on thematic units (Budd, Thorp, & Donohew, 1967, p. 34). For example, this comment provided by Teacher 9 (T9) was unitized into two units (designated by open <u> and close </u> tags), such as “<u1>good eye contact</u1> <u2>and use of floor space from last presenter</u2>.” Unitizing resulted in 2,241 analyzable units.

Codebook Development and Data Analysis

Because the qualitative nature of the comments did not allow us to understand the comments’ topics or intent without reading each comment, we analyzed the data manually using content analysis (Neuendorf, 2017) instead of using an automated method. To analyze the units, we engaged in two coding processes: one for the analytic factors commented on that enabled us to answer research questions 1 and 2 and one for the evaluation of those factors that enabled us to answer research question 3. We followed procedures common to content analysis, including codebook development, achieving acceptable pilot reliability, and establishing acceptable reliability on a second coding sample on at least 10% of the data (Neuendorf, 2017).

In the first coding process for analytic factors, we developed the codebook’s categories through a deductive or “directed approach” (Hsieh & Shannon, 2005, p. 1281), forming categories based on the 3M evaluation method (Baker & Thompson, 2004). The presenters, student feedback providers, and instructor feedback providers were familiar with this method because it formed the basis for the oral presentation instruction in the university’s business communication textbook used in the university (Baker, 2013). The business professionals who provided feedback were not trained in the 3M evaluation method, enabling us to gather professional feedback that was not influenced by the university’s classroom instruction. The 3M method consists of evaluating the analytic factors of message, media, and messenger, although only message and messenger were included in this study because the presenting teams’ slides (i.e., media) could not be seen well in the recorded video. Message-related factors consisted of the participants’ reactions to the content of the presentations, and messenger-related factors consisted of the participants’ reactions to the delivery of the presentations. In our view, the two factors more clearly delineate the difference between content and how content is delivered than do the presentational and nonpresentational categories used in the Mason and Harrison (2003) and Clark (2008) studies. Further, the two factors are similar to the content and delivery factors that Campbell et al. (2001) analyzed in their quantitative comparison of student presenter, student peer, and instructor ratings. The final codebook can be found in Table 1.

Table 1.

Analytic Factors Final Codebook.

Code	Definition
Message	An audience reaction to content, including ideas presented, words used, and the content’s organization, structural parts, transitions and coherence, clarity, conciseness, accuracy, persuasiveness, relevance, emphasis, word choice, interest level, pacing, and effect on the audience
Messenger	An audience reaction to how the message is presented, including the presenter’s confidence, preparation, presence, professionalism, personality, nervousness, behavior, facial expressions, tone of voice, voice volume, voice pacing, enunciation, filler words, gestures, movement, body stance, dress and appearance, eye contact, and effect on the audience
Either	An audience reaction that is broad, general, and/or applicable to both message and messenger
Neither	An audience reaction that is not applicable to either the message or messenger

To assess the reliability of the first codebook, we conducted a pilot assessment on 11 percent of the data with an independent, undergraduate intercoder. One of the authors served as the second coder. Two coding rounds in the pilot assessment resulted in a Cohen’s kappa reliability statistic of .92. Next, following recommended practices for content analysis, we conducted a secondary, final assessment of reliability (Neuendorf, 2017). In the final assessment, an additional reliability check resulted in a Cohen’s kappa of .89.¹ After we established the codebook’s reliability with the independent coder, the independent coder proceeded to code the remaining data. After final coding was completed, we analyzed the data by summing each code for business professionals, instructors, and student peers to compare differences and similarities.

In the second coding process for the evaluation of the analytic factors, we developed the codebook’s categories through an inductive or “conventional approach” (Hsieh & Shannon, 2005, p. 1279). Following Hsieh and Shannon’s (2005) guidelines, we read the comments, discussed commonalities in the comments, and categorized the commonalities into codes. Two codes emerged that reflected the feedback providers’ evaluation of the presentations: what went well (compliments) and what could be improved (suggestions). The final codebook can be found in Table 2.

Table 2.

Evaluation Final Codebook.

Code	Definition
What Went Well	An audience reaction that provides feedback to the presenters on what went well about the presentation, such as compliments, positive evaluations or observations, agreement, and examples or elaborations of what went well
What Could Be Improved	An audience reaction that provides feedback to the presenters on what could be improved about the presentation, such as directives, negative evaluations or observations, questions for clarity or consideration, disagreement, suggestions, and examples or elaborations of what could be improved

To assess the reliability of the second codebook, we conducted a pilot assessment on 11 percent of the data with an independent undergraduate intercoder. One of the authors served as the second coder. One coding round in the pilot assessment resulted in a Cohen’s kappa reliability statistic of .94. Following recommended practices for content analysis, we conducted a secondary, final assessment of reliability (Neuendorf, 2017). In the final assessment, an additional reliability check resulted in a Cohen’s kappa of .93.² After we established the codebook’s reliability with the independent coder, the same independent coder proceeded to code the remaining data. After final coding was completed, we compared the differences and similarities in the summed data.

Results

This section will present the results for the study’s three research questions. Specifically, it will discuss the types of feedback provided, the frequency of the types of feedback, and the participants’ evaluations communicated in those types of feedback.

Research Question 1: Feedback Types

The qualitative feedback of business professionals, instructors, and student peers coalesced primarily into two types of analytic factors: message related and messenger related. Message-related feedback was given in response to the actual content of the presentation. For example, business professional 2 (B2) reacted to the words of the content, directing the presenters to “avoid ‘cute’ remarks such as ‘we just received a text’ or ‘our battery is almost dead’ as these potentially take away from the credibility of the substance.” Instructor 2 (I2) questioned the content presented: “Have you checked whether or not this double-checking is already standard procedure?” Student 7 (S7) responded to one of the presentations by writing, “This is an interesting point.”

In contrast to message-related feedback given in response to the actual presentation content, messenger-related feedback was given in response to how the presenters presented the content. For example, business professional 5 (B5) critiqued a team member who was not presenting by observing that “the student rocking back and forth in the background is distracting.” Instructor 1 (I1) responded positively to the way one presenter spoke, noting, “Great voice variation.” Student 8 (S8) shared perceptions about one presenter’s persuasiveness: “I don’t feel the presenter is very convinced of his own message.” Table 3 provides additional examples.

Table 3.

Qualitative Feedback Examples.

Code	Examples
Message	• B4: Good point—to have another person check the prescription. • B2: Avoid “cute” remarks such as “we just received a text” or “our battery is almost dead” as these potentially take away from the credibility of the substance. • I2: Good details about solution options. • I3: Have you checked whether or not this double-checking is already standard procedure? • S7: This is an interesting point. • S4: We think that’s important or we know that’s important?
Messenger	• B5: The student rocking back and forth in the background is distracting. • B2: Knows material well. • I1: Great voice variation. • I10: Hand gestures are bothersome. • S8: I don’t feel the presenter is very convinced of his own message. • S5: Excellent ability to tell the story from a meaningful perspective and quote verbatim!
Either	• B1: Overall an effective presentation. • B7: Good preparation. • I4: Would like to see much stronger close. • I9: Maybe this is okay. • S9: That works. • S6: That was a good way to regain our attention.
Neither	• B8: Just a thought. • B8: Are they aware they are being filmed? • I6: A fairly effective use of the "clicker" as a pointer. • I4: Blank out the screen when you tell a story. • S1: Good use of PowerPoint. • S10: Video froze for several seconds.

Research Question 2: Frequency of Feedback Types

Table 4 provides an overall view of the frequency of analytic factors commented on by business professionals, instructors, and student peers. Specifically, business professionals provided 574 feedback units (M = 57.4, SD = 28.77), with 203 (M = 20.3, SD = 15.54, % = 35.37) being message-related feedback units and 344 (M = 34.4, SD = 27.26, % = 59.93) being messenger-related feedback. Of their 1,218 feedback units (M = 121.8, SD = 59.95), instructors provided 446 (M = 44.6, SD = 27.4, % = 36.62) message-related feedback units and 740 (M = 74, SD = 56.98, % = 60.76) messenger-related feedback units. Of their 449 feedback units (M = 44.9, SD = 14.73), student peers provided 196 (M = 19.6, SD = 13.65, % = 43.65) message-related feedback units and 200 (M = 20.0, SD = 7.97, % = 44.54) messenger-related feedback units. Table 5 presents similar percentage ranges of the analytic factors broken out by Presentation 1 and Presentation 2.

Table 4.

Factor Frequency by Evaluation Group.

Factor	Evaluation Group
Factor	Professionals, n (%)	Instructors, n (%)	Students, n (%)
Message	203 (35.37)	446 (36.62)	196 (43.65)
Messenger	344 (59.93)	740 (60.76)	200 (44.54)
Both	24 (4.18)	29 (2.38)	23 (5.12)
Neither	3 (0.52)	3 (0.25)	30 (6.68)
Total	574 (100)	1,218 (100)	449 (100)

Table 5.

Factor Frequency by Presentation and Evaluation Group.

Factor	Evaluation Group
Factor	Professionals, n (%)	Instructors, n (%)	Students, n (%)
Presentation 1
Message	109 (37.46)	219 (33.74)	93 (47.69)
Messenger	168 (57.73)	412 (63.48)	91 (46.67)
Both	11 (3.78)	16 (2.47)	7 (3.59)
Neither	3 (1.03)	2 (0.31)	4 (2.05)
Subtotal	291 (100)	649 (100)	195 (100)
Presentation 2
Message	94 (33.22)	227 (39.89)	103 (40.55)
Messenger	176 (62.19)	328 (57.64)	109 (42.91)
Both	13 (4.59)	13 (2.28)	16 (6.30)
Neither	0 (0.00)	1 (0.18)	26 (10.24)
Subtotal	283 (100)	569 (100)	254 (100)

Although all three groups commented on message- and messenger-related analytic factors, overall business professionals provided proportionally more feedback units related to messenger factors than to message factors. Likewise, instructors provided proportionally more feedback units related to messenger factors than to message factors. In contrast, student peers provided almost equal numbers of message-related factors and messenger-related factors (see Figure 2). Across the two different presentations, these proportions stayed generally consistent for all three groups (see Figure 3). Therefore, in comparison to the business professionals, instructors placed overall similar emphasis on both message- and messenger-related factors, and peers placed considerably less emphasis on messenger-related factors.

Figure 2.

Message and messenger factor percentages overall.

Figure 3.

Message and messenger factor percentages by presentation.

Research Question 3: Evaluation of Feedback Types

Whereas the foregoing results focused on the analytic factors of the three groups’ feedback, the additional analysis here provides insight into whether the three groups were complimenting or suggesting improvement when they commented on the analytic factors (Table 6 provides qualitative examples of both compliments and suggestions). Of business professionals’ 574 feedback units, 176 (M = 17.6, SD = 8.11, % = 30.66) focused on what went well, and 848 (M = 39.8, SD = 22.12, % = 69.34) focused on what could be improved. Of instructors’ 1,218 feedback units, 370 (M = 37.0, SD = 19.77, % = 30.38) focused on what went well, and 848 (M = 84.8, SD = 55.21, % = 69.62) focused on what could be improved. Of student peers’ 449 feedback units, 215 (M = 21.5, SD = 7.99, % = 47.88) focused on what went well, and 234 (M = 23.4, SD = 12.14, % = 52.12) focused on what could be improved (see Table 7). Tables 8 and 9 present these same numbers broken out across presentations and across analytic factors.

Table 6.

Examples of Compliments and Suggestions for Improvement.

Code	Examples
What Went Well	• B6: You focused on something very important to the board.• B7: Nice tie of solution to individual complaint category.• I7: They all look very professional.• I9: Good rhetorical questions regarding the company’s ethics.• S7: Bringing back the values was effective.• S6: Does well at engaging the entire audience.
What Could Be Improved	• B3: Button suit.• B1: How do you know additional costs will be offset?• I2: Avoid cliches and be precise.• I4: Vocal tone is too steady for the seriousness of the story you are telling.• S9: I think that a more focused description of the problems would help keep the issues close to mind.• S7: Pronounced “autonomy” wrong.

Table 7.

Compliment and Suggestion for Improvement Frequency by Evaluation Group.

Code	Evaluation Group
Code	Professionals, n (%)	Instructors, n (%)	Students, n (%)
What Went Well	176 (30.66)	370 (30.38)	215 (47.88)
What Could Be Improved	398 (69.34)	848 (69.92)	234 (52.12)
Total	574 (100.00)	1,218 (100.00)	449 (100.00)

Table 8.

Compliment and Suggestion for Improvement Frequency by Presentation and Evaluation Group.

Code	Evaluation Group
Code	Professionals, n (%)	Instructors, n (%)	Students, n (%)
Presentation 1
What Went Well	71 (24.40)	155 (23.88)	85 (43.59)
What Could Be Improved	220 (75.60)	494 (76.12)	110 (56.41)
Subtotal	291 (100.00)	649 (100.00)	195 (100.00)
Presentation 2
What Went Well	105 (37.10)	215 (37.79)	130 (51.18)
What Could Be Improved	178 (62.90)	354 (62.21)	124 (48.82)
Subtotal	283 (49.30)	569 (46.72)	254 (56.57)
Total	574 (100.00)	1,218 (100.00)	449 (100.00)

Table 9.

Compliment and Suggestion for Improvement Frequency by Presentation, Factor, and Evaluation Group.

Code	Evaluation Group
Code	Professionals, n (%)	Instructors, n (%)	Students, n (%)
Presentation 1: Message
What Went Well	29 (26.61)	64 (29.22)	46 (49.46)
What Could Be Improved	80 (73.39)	155 (70.78)	47 (50.54)
Subtotal	109 (100.00)	219 (100.00)	93 (100.00)
Presentation 1: Messenger
What Went Well	36 (21.43)	80 (19.42)	33 (36.26)
What Could Be Improved	132 (78.57)	332 (80.58)	58 (63.74)
Subtotal	168 (100.00)	412 (100.00)	91 (100.00)
Presentation 2: Message
What Went Well	34 (36.17)	86 (37.89)	47 (45.63)
What Could Be Improved	60 (63.83)	141 (62.11)	56 (54.37)
Subtotal	94 (100.00)	227 (100.00)	103 (100.00)
Presentation 2: Messenger
What Went Well	60 (34.09)	121 (36.89)	54 (49.54)
What Could Be Improved	116 (65.91)	207 (63.11)	55 (50.46)
Subtotal	176 (100.00)	328 (100.00)	109 (100.00)

Overall, business professionals and instructors provided more than twice as many suggestions as compliments. In contrast, students provided suggestions and compliments in almost equal amounts, with slightly more suggestions than compliments. Therefore, when the feedback is compared overall, the sentiment of the instructors closely mirrored the professionals’ sentiment whereas the students’ sentiment was far more complimentary than the professionals’ was and did not reflect the professionals’ sentiment (see Figure 4).

Figure 4.

Compliment and suggestion for improvement percentages overall.

When the two presentations were compared (see Table 8 and Figure 5), the business professionals complimented in Presentation 1 about one-fourth of the time; in Presentation 2, they complimented about one-third of the time. This upward shift in compliments indicates that they evaluated the second presentation more positively than the first. Instructors’ sentiment shifted almost identically to the professionals’ sentiment, suggesting that instructors also evaluated the second presentation more positively than the first. Students similarly viewed the second presentation more positively, as indicated by their shift toward more positive compliments in the second presentation. Therefore, when the feedback is compared across the two presentations, the sentiment of the instructors again reflected the professionals’ sentiment very closely. Although not to the same extent, the students’ sentiment shifted in the same direction as that of professionals, providing some evidence that the students sensed a similar quality shift between the two presentations.

Figure 5.

Compliment and suggestion for improvement percentages by presentation.

The findings can be considered in greater detail when the sentiment and analytic factors were compared across presentations (see Table 9, Figures 6 and 7). In Presentation 1, the business professionals complimented the message factor about one-fourth of the time and the messenger factor about one-fifth of the time. Both percentages increased to over one-third in the second presentation, indicating a perceived upward shift in the quality in both message and messenger factors between Presentation 1 and Presentation 2. Instructors’ percentage changes indicate a similarly perceived upward shift in quality in both factors. Students complimented the message factor about half the time and the messenger about one-third of the time in Presentation 1. In Presentation 2, students complimented the message factor less than half the time and the messenger factor about half the time. Students, therefore, perceived a slight downward shift in quality in the message factor and an upward shift in quality in the messenger factor. Across analytic factors and across presentations then, instructors perceived the quality of the presentations similar to the way professionals did. Student peers perceived the quality of the message less favorably than did instructors, but the student peers’ perceptions of messenger quality reflected a similar shift to that of professionals.

Figure 6.

Message compliment and suggestion for improvement percentages by presentation.

Figure 7.

Messenger compliment and suggestion for improvement percentages by presentation.

Discussion

Time constraints and emphasis on written genres over oral ones complicate instructors’ efforts to provide students with adequate feedback on their presentations. Peer feedback on oral presentations provides one way for business communication students to receive more feedback on their oral presentations. Peer feedback may differ from instructors’ feedback, though, so presenters receiving peer feedback may not trust that their peers’ feedback will prepare them for their instructor’s assessment. Ideally, whether feedback comes from instructors or peers, it should prepare student presenters to succeed in their workplace presentations. With that aim, the purpose of this article was to compare instructors’ and peers’ qualitative feedback with that of business professionals. The comparison of professionals’ qualitative feedback uniquely contributes to business communication literature, which has not yet calibrated peer and instructor feedback with professional feedback and which has nearly all focused on quantitative ratings. Calibrating with professionals’ qualitative feedback can benefit students who are seeking to better predict how professionals will evaluate presentations in the workplace without pedagogical rubrics, and it can also benefit instructors who are seeking to give feedback that prepares their students for professional evaluation.

Overall, the results show that the business professionals, instructors, and student peers in this study all commented on message and messenger analytic factors but that on average the instructors aligned much more closely with the professionals than the students did in the emphasis and sentiment of their comments. The instructors and professionals emphasized messenger factors more than the students did, and the students were more complimentary than the instructors and professionals were. The following discussion of these results will focus on the conclusions and implications of the feedback provided by each participant group: business professionals, instructors, and student peers.

Although the business professionals in this study placed more emphasis on messenger-related factors than on message-related factors and more emphasis on suggestions than on compliments, these findings alone should not dictate changes in instructors’ pedagogical emphasis. Although Clark’s (2008) study corroborates the emphasis of messenger over message, Mason and Harrison’s (2003) study contradicts those findings. Both Clark’s (2008) and Mason and Harrison’s (2003) studies corroborate the emphasis of suggestions over compliments. Although the consistent negative sentiment results provide a pattern across the present study and the other two studies, the contrasting emphasis on message or messenger indicates a need for additional research, as well as a potential methodological concern when drawing implications based only on professionals’ evaluations. The concern is that each presentation is unique and different. In some presentations, the message may be stronger or weaker, whereas in other presentations the delivery may be stronger or weaker. Unless the quality of the presentations being evaluated is controlled for across studies or the professional feedback itself is considered the control and instructor and student peer comparisons are made to the professional feedback, then pedagogical implications are limited. In the present study, we therefore avoid inferring conclusions based on professionals’ feedback alone and instead will focus on conclusions and implications based on comparisons of the instructors’ and student peers’ feedback to the professionals’ feedback.

A comparison of instructor feedback to student peer feedback may lead to student presenters being better able to predict their instructors’ feedback (Campbell et al., 2001), but such a comparison will help students predict professionals’ feedback only when the instructors’ feedback reflects the type of feedback that the professionals would give. For the measures and the sample of instructors and professionals in this study, we conclude that the instructors’ qualitative feedback reflected that of the professionals in this study remarkably well. Even across two different presentations, the instructors’ emphasis on messenger-related analytic factors and on suggestion-focused sentiment generally mirrored that of the professionals. Further, as the professionals’ sentiment shifted to more positively evaluate the second presentation’s message- and messenger-related analytic factors, the instructors’ sentiment shifted in a similar direction. For the students of these instructors then, we conclude that the instructors’ qualitative feedback would prepare the student presenters to better predict these professionals’ feedback types and sentiment.

Despite the association between the instructors’ and professionals’ qualitative feedback in this study, we acknowledge that a different sample of instructors and professionals could yield different results. Larger sample sizes could yield more generalizable results, although the time and effort needed to analyze the qualitative comments in such large-scale studies might be prohibitive. For individual instructors who are seeking to prepare their students for certain professions, an approach might be to duplicate the methodology employed in this study for their own curricula, students, and employers. Given the online nature of the data-gathering tools (Lee, 2020), instructors and students can evaluate the presentations in either face-to-face or online classroom settings. Assuming that students give their consent for their presentations to be evaluated by employers, the employers can then remotely provide qualitative comments on the presentations without leaving their offices or other workspaces. In this way, program directors and instructors can better understand local employers’ qualitative feedback, thereby calibrating their formative feedback with the professionals’ feedback to better prepare students to give oral presentations in the workplaces where they will most likely be employed.

For students to improve, they may need more feedback on their oral presentations than that which a single instructor has the time to provide. Peer feedback provides one option, but unfortunately this study’s qualitative peer feedback did not associate as well with the professionals’ feedback as the instructors’ feedback did. Compared to the instructors, the students tended to place relatively less emphasis on messenger-related analytic factors and less emphasis on suggestions. These findings contrast with those of Smith et al. (2020), who found that the students’ self-reflections and peer comments in their study tended to emphasize delivery over content. For these data in their study, Smith et al. did not distinguish between comments given by the presenter and the comments given by peers, so parsing the two might yield results more similar to those found in the present study. The prompt provided to students in that study also specifically asked students to comment on their peers’ delivery, which could have led to more comments on messenger-related factors. Furthermore, each presentation is unique and different, so the presentations critiqued in that study could have lent themselves to an emphasis one way or another. Importantly, the present study mitigates the confounding variance of the underlying presentations as the students’ feedback is compared to the professionals’ feedback.

In comparison to the professionals in the present study, the students also tended to overemphasize compliments and underemphasize suggestions. With the caveat that each presentation is different, these findings corroborate those of Smith et al. (2020) whose data indicated that the majority of peers’ comments in four rounds of presentations was positive instead of constructive, with the percentage of positive feedback generally increasing over time. Taken together with the negative or suggestion-focused sentiment of the feedback from the professionals in the present study and in the Mason and Harrison (2003) and Clark (2008) studies, it appears that peers’ qualitative feedback may need to become more constructive or suggestion oriented in order to better reflect professionals’ feedback. Pedagogically, evaluation is a higher-order learning activity according to Bloom’s revised taxonomy (Anderson et al., 2001), and yet the students in the present study seem to have little trouble evaluating the positive aspects of presentations. Evaluative critique, however, involves students judging both “positive and negative features” (Anderson et al., 2001, p. 84) so instructors may need to employ classroom activities that specifically involve students critiquing not only what went well in oral presentations but also what could be improved.

Although these findings indicate that the sentiment of the peers’ feedback deviated from that of the professionals, across the two presentations the students’ feedback did shift in overall sentiment in the same direction as the professionals’ feedback, suggesting that the students and the professionals evaluated the second presentation more positively than the first. Given that a similar trend existed when comparing the students’ feedback with the instructors’ feedback, these findings seem to corroborate other studies that have found a moderate correlation between instructor and peer quantitative evaluations (Campbell et al., 2001; L. De Grez et al., 2012; Freeman, 1995). However, the students’ overemphasis on compliments over suggestions seems to corroborate De Grez et al.’s (2012) study that students’ average quantitative evaluations are higher than those of instructors. In addition, the results indicate that the peers’ overall sentiment of message-related qualitative evaluations shifted in the opposite direction of professionals’ overall sentiment between the two presentations. Therefore, although the peers’ overall evaluations and specific evaluations of messenger-related analytic factors appear to show some promise, the students appear to need additional help calibrating their evaluation of message-related analytic factors in particular.

One classroom activity that could help foster students’ ability to better anticipate professionals’ feedback builds on Schriver’s (1992) pedagogical experiment using think-aloud protocols for improving students’ abilities to anticipate reader concerns on written documents. Extending the results of Schriver’s study from a written to oral mode, instructors could ask business professionals to comment on oral presentations online using new technologies that enable real-time comments (Lee, 2020). Instructors could then ask their students to comment on the same presentations and try to predict the problems that professionals might have with the presentations’ content and delivery. Students could then review the professionals’ comments alongside their own, gaining a better awareness of professionals’ concerns. Ultimately, the goals of such activities are not only to help students better predict professionals’ (and instructors’) concerns but also to help the students then critically evaluate their peers’ presentations so their peers can be better prepared for instructors’ and professionals’ evaluations. The Smith et al. (2020) study indicates that students’ qualitative oral feedback can change and improve over time through repeated practice, so students likely need multiple opportunities to engage with professionals’ evaluations. Through repeated practice, students also receive more feedback on oral presentations from their peers, without relying solely on the feedback of time-constrained instructors.

Conclusion

These findings and implications should be applied with the following limitations, which also provide opportunities for future research. First, like other studies on this topic (Campbell et al., 2001 Clark, 2008; L. De Grez et al., 2012; Freeman, 1995; Mason & Harrison, 2003; Smith et al. 2020), the data were gathered from a targeted but limited participant pool. As noted previously, instructors could expand the reach of the data gathering and assess whether the findings remain reliable across universities and geographic areas. Second, the data were gathered in a controlled setting. Additional studies could replicate aspects of this study in a live environment to assess whether variables present in live settings influence the results presented here. Third, the data gathered were qualitative comments from business professionals, instructors, and student peers. Future studies could gather both qualitative feedback and quantitative feedback on analytic factors and gather additional self-assessment feedback from the presenters themselves to triangulate and enrich the results, a study that Smith et al. (2020) have also called for. Finally, the researchers gathered little demographic data on the participants outside of their roles as business professionals, instructors, and student peers. Future research could control for other demographic variables to assess whether they influence the types or evaluative aspects of the feedback provided.

Oral presentation assessment can be difficult to fit into business communication classrooms that are already packed with instruction and assessment of written communication, and personal time constraints might make it impossible for instructors to provide all the oral presentation feedback they might hope to provide. One opportunity for student presenters to receive more feedback might be for student peers to provide qualitative feedback using new feedback technologies. However, because the student peers in this study deviated from professionals’ feedback in their deemphasis of messenger-related analytic factors and suggestions, these findings suggest that student peers may need additional calibration with professionals in those areas so their qualitative feedback helps their peers predict professionals’ feedback. This professional feedback can be obtained quite easily with new feedback technologies.

Footnotes

Acknowledgements

The authors wish to thank the editors and reviewers for their time and helpful feedback, as well as Emily Garrett and Catherine Niesporek who served as undergraduate research assistants and helped with data coding.

Declaration of Conflicting Interests

William H. Baker was the originator of the video-feedback technology used in this study, but it was sold to GoReact in 2011. Baker is now a minor shareholder in the GoReact company but has no active participation in its operations.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Matthew J. Baker

Notes

Author Biographies

Matthew J. Baker is an assistant professor in the linguistics department at Brigham Young University where he teaches courses in editing and publishing.

William H. Baker is an emeritus professor in the Marriott School of Business at Brigham Young University where he taught courses in management communication.

References

Anderson

L. W.

Krathwohl

D. R.

Airasian

P. W.

Cruikshank

K. A.

Mayer

R. E.

Pintrich

P. R.

Raths

Wittrock

M. C.

(2001). A taxonomy for learning, teaching, and assessing. New York, NY: Longman.

Baker

W. H.

(2013). Writing & speaking for business. Provo, UT: BYU Academic Publishing.

Baker

W. H.

Thompson

M. P.

(2004). Teaching presentation skills. Business Communication Quarterly, 67(2), 216-219. doi: 10.1177/1080569904672009

Budd

R. W.

Thorp

R. K.

Donohew

(1967). Content analysis of communications. New York, NY: Macmillan.

Campbell

K. S.

Mothersbaugh

D. L.

Brammer

Taylor

(2001). Peer versus self assessment of oral business presentation performance. Business Communication Quarterly, 64(3), 23-42. doi: 10.1177/108056990106400303

Clark

(2008). The impact of entrepreneurs’ oral “pitch” presentation skills on business angels’ initial screening investment decisions. Venture Capital, 10(3), 257-279. doi: 10.1080/13691060802151945

Coffelt

T. A.

Baker

M. J.

Corey

R. C.

(2016). Business communication practices from employers’ perspectives. Business and Professional Communication Quarterly, 79(3), 300-316. doi: 10.1177/2329490616644014

Coffelt

T. A.

Grauman

Smith

F. L.

(2019). Employers’ perspectives on workplace communication skills: The meaning of communication skills. Business and Professional Communication Quarterly, 82(4), 418-439. doi: 10.1177/2329490619851119

Coffelt

T. A.

Smith

F. L.

(2020). Exemplary and unacceptable workplace communication skills. Business and Professional Communication Quarterly, 83(4), 365-384.

10.

Cohen

(1960). A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement, 20(1): 37-46. doi: 10.1177/001316446002000104.

11.

De Grez

. (2010). Peer assessment of oral presentation skills. Procedia—Social and Behavioral Sciences, 2(2), 1776-1780.

12.

De Grez

Valcke

Roozen

. (2012). How effective are self- and peer assessment of oral presentation skills compared with teachers’ assessments? Active Learning in Higher Education, 13(2), 129-142. doi: 10.1177/1469787412441284

13.

Dollisso

Koundinya

(2011). An integrated framework for assessing oral presentations using peer, self, and instructor assessment strategies. NACTA Journal, 55, 39.

14.

Dyrud

M. A.

Worley

R. B.

(1998). Critical thinking. Business Communication Quarterly, 61(3), 62-64.

15.

Freeman

(1995). Peer assessment by groups of group work. Assessment & Evaluation in Higher Education, 20(3), 289-300. doi: 10.1080/0260293950200305

16.

GoReact (n.d.). Feedback. https://help.goreact.com/hc/en-us/articles/360003066092

17.

Gray

F. E.

(2010). Specific oral communication skills desired in new accountancy graduates. Business Communication Quarterly, 73(1), 40-67. doi: 10.1177/1080569909356350.

18.

Gray

F. E.

Murray

(2011). “A distinguishing factor”: Oral communication skills in new accountancy graduates. Accounting Education, 20(3), 275-294. doi: 10.1080/09639284.2011.560763

19.

Hattie

Timperley

(2007). The power of feedback. Review of Educational Research, 77(1), 81-112. doi: 10.3102/003465430298487

20.

Hsieh

H. F.

Shannon

S. E.

(2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277-1288. doi: 10.1177/1049732305276687

21.

Jameson

D. A.

(2013). New options for usability testing projects in business communication courses. Business Communication Quarterly, 76(4), 397-411.

22.

Kenkel

C. S.

(2011). Teaching presentation skills in online business communication courses. MERLOT Journal of Online Learning and Teaching, 7(3), 412-418.

23.

Keyton

Caputo

J. M.

Ford

E. A.

Leibowitz

S. A.

Liu

Polasik

S. S.

Ghosh

(2013). Investigating verbal workplace communication behaviors. Journal of Business Communication, 2(50), 152–169. doi:10.1177/0021943612474990

24.

Pittenger

K. K. S.

Miller

M. C.

Mott

(2004). Using real-world standards to enhance students’ presentation skills. Business Communication Quarterly, 67(5), 327-336.

25.

Landis

J. R.

Koch

G. G.

(1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174. doi: 10.2307/2529310

26.

Lee

Y. S.

(2020). Instructional innovations with business communication in classrooms: Practice-based presentation topics and online communication tools. Business Communication Research and Practice, 3(1), 53-59. doi: 10.22682/bcrp.2020.3.1.53

27.

Leeds

E. M.

Raven

Brawley

(2007). Primary traits of oral business presentation: Translatable use for assessment in a virtual learning environment. College Teaching Methods & Styles Journal (CTMS), 3(4), 21-34. doi: 10.19030/ctms.v3i4.5583

28.

LeFebvre

(2016). Team-based learning for the basic communication course: A transformative pedagogical approach. Review of Communication, 16(2-3), 192-212. doi: 10.1080/15358593.2016.1187454

29.

Lucas

Rawlins

J. D.

(2015). The competency pivot: Introducing a revised approach to the business communication curriculum. Business and Professional Communication Quarterly, 78(2), 167-193. doi: 10.1177/2329490615576071

30.

Mason

C. M.

Harrison

R. T.

(2003). “Auditioning for money”: what do technology investors look for at the initial screening stage? The Journal of Private Equity, 6(2), 29-42. doi: 10.3905/jpe.2003.320037

31.

Miller

P. J.

(2003). The effect of scoring criteria specificity on peer and self-assessment. Assessment & Evaluation in Higher Education, 28(4), 383-394. doi: 10.1080/0260293032000066218

32.

Moreno

(2004). Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discovery-based multimedia. Instructional Science, 32(1), 99-113. doi: 10.1023/b:truc.0000021811.66966.1d

33.

Moshiri

Cardon

(2014). The state of business communication classes: A national survey. Business and Professional Communication Quarterly, 77(3), 312-329. doi: 10.1177/2329490614538489

34.

Moshiri

Cardon

P. W.

(2020). An updated and expanded nationwide study of business communication courses. Business and Professional Communication Quarterly, 83(4), 469-488. doi: 10.1177/2329490620934043

35.

Murdock

R. C.

(2017). An instrument for assessing the public communication of scientists (Publication no. 10602331) [Doctoral dissertation, Iowa State University]. ProQuest Dissertations Publishing.

36.

Neuendorf

K. A.

(2017). The content analysis guidebook. Los Angeles, CA: Sage.

37.

Ortiz

L. A.

Region-Sebest

MacDermott

(2016). Employer perceptions of oral communication competencies most valued in new hires as a factor in company success. Business and Professional Communication Quarterly, 79(3), 317-330. doi: 10.1177/2329490615624108

38.

Planas Lladó

Soley

L. F.

Fraguell Sansbelló

R. M.

Pujolras

G. A.

Planella

J. P.

Roura- Pascual

Martínze

J. S.

Moreno

L. M

. (2014). Student perceptions of peer assessment: An interdisciplinary study. Assessment & Evaluation in Higher Education, 39(5), 592-610. doi: 10.1080/02602938.2013.860077

39.

PitchVantage (n.d.). Universities. https://pitchvantage.com/how-it-works-universities/

40.

Pope-Ruark

(2011). Know thy audience: Helping students engage a threshold concept using audience-based pedagogy. International Journal for the Scholarship of Teaching and Learning, 5(1), 1-16.

41.

Robles

V. D.

Baker

M. J.

(2019). Using case-method pedagogy to facilitate audience awareness. IEEE Transactions on Professional Communication, 62(2), 192-207.

42.

Rozumalski

L. P.

Graves

M. F.

(1995). Effects of case and traditional writing assignments on writing products and processes. Journal of Business and Technical Communication, 9(1), 77-102.

43.

Russ

T. L.

(2009). The status of the business communication course at US colleges and universities. Business Communication Quarterly, 72(4), 395-413. doi: 10.1177/1080569909349524

44.

Schriver

K. A.

(1992). Teaching writers to anticipate readers’ needs: A classroom-evaluated pedagogy. Written Communication, 9(2), 179-208.

45.

Smith

A. B.

Schieber

D. L.

Austin

T. L.

(2020). Avoiding "Great job!": Self- and peer feedback as formative assessment for oral communication. Journal of the Academy of Business Education, 21.

46.

Spartz

J. M.

Weber

R. P.

(2015). Writing entrepreneurs: A survey of attitudes, habits, skills, and genres. Journal of Business and Technical Communication, 29(4), 428-455. doi: 10.1177/1050651915588145

47.

Spinuzzi

Nelson

Thomson

K. S.

Lorenzini

French

R. A.

Pogue

Burback

S. D.

Momberger

(2014). Making the pitch: Examining dialogue and revisions in entrepreneurs’ pitch decks. IEEE Transactions on Professional Communication, 57(3), 158-181. doi: 10.1109/TPC.2014.2342354

48.

Stratman

J. F.

(1991). Teaching lawyers to revise for the real world: A role for reader protocols. The Journal of the Legal Writing Institute, 1, 35-78.

49.

Suchan

Dulek

(1988). Toward a better understanding of reader analysis. Journal of Business Communication, 25(2), 29-45. doi: 10.1177/002194368802500204

50.

Van Ginkel

Gulikers

Biemans

Mulder

. (2017). The impact of the feedback source on developing oral presentation competence. Studies in Higher Education, 42(9), 1671-1685. doi: 10.1080/03075079.2015.1117064

51.

Weiner

(1954). The human use of human beings: Cybernetics and society. Boston, MA: Houghton Mifflin.

52.

Wrench

J. S.

Thomas-Maddox

Richmond

V. P.

McCroskey

J. C.

(2013). Quantitative research methods for communication: A hands-on approach (2nd ed.). Oxford: Oxford University Press.