Abstract
Business, like many other fields in higher education, continues to rely largely on conventional testing methods for assessing student learning. In the current article, another evaluation approach—the oral exam—is examined as a means for building and evaluating the professional communication and oral dialogue skills needed and utilized by business graduates. Prior studies of oral exams in higher education are reviewed, along with the empirical findings from an exploratory investigation of an oral exam in an undergraduate human resource course. Implications for future research and the use of oral exams in business education are also presented.
Despite the fact that many jobs in business require oral dialogue and face-to-face communication (Armstrong, 2003;Gray, 2010;Sapp & Zhang, 2009), written exams continue to dominate student evaluation in U.S. business courses (undergraduate and masters) due to their notable ease of administration and efficiency of processing (Walker & Thompson, 2001). Not surprisingly then, one evaluation method that has received scant focused study in business education, despite the high degree of oral communication skills ultimately used by business graduates, is the oral exam. In this paper, I contend that oral exams may help build and assess the types of skills most utilized by graduates in the world of work.
One could argue that business communication skills are the primary skills that applied disciplines (e.g., business schools) should be developing to ensure students’ “workplace readiness” (Barrett, 2002;Gray, 2010;McLester & McIntire, 2006;Zinser, 2003). Bolstering and assessing students’ professional communication abilities are increasingly relevant in undergraduate business education (Gray, 2010;Sapp & Zhang, 2009). Interestingly, oral exams are perceived by students as vocationally relevant (Oakley & Hencken, 2005;Sibbald, 1998), which could increase their openness to this form of evaluation. Oral exams can also reduce the odds of academic misconduct (i.e., cheating), which we know is particularly problematic among U.S. business students (Brown & McInerney, 2008;McCabe & Trevino, 1995).
Oral exams are one of many tools (e.g., oral presentations, research papers, and written exams) available to instructors for assessing students’ understanding of course content and their ability to demonstrate relevant skills, and each method possesses its own pros and cons. In this article, I examine the documented use of oral exams across various disciplines in higher education, as well as the pros and cons of oral exams, and then present the results of an initial empirical investigation of their use in an undergraduate human resource (HR) management course. Although the current article is not arguing that traditional written exams be replaced, a goal of this study is to help readers understand how oral exams can be used in undergraduate business education and how students perform on these exams.
Review of Relevant Literature
Generally speaking, a written exam format is used by college instructors to evaluate student learning because of its ease and efficiency of grading (Parmenter, 2009;Walker & Thompson, 2001). However, the traditional written exam—particularly when utilizing only closed-ended questions—may provide little information other than which questions were answered “correctly” (Walker & Thompson, 2001). How students interpret the questions, how they understand nuances of factors that might affect their answers, or how they might apply the concepts to their personal learning typically remain untapped (Walker & Thompson, 2001). In contrast, oral exams have been advanced as appropriate tests of students’ understanding, mental agility, depth of understanding, and their ability to organize ideas and use language to express ideas (Evans, Ingersoll, & Smith, 1966;Sibbald, 1998). Another attribute of oral exams is that any ambiguity in the testing setting (e.g., vague directions) can be cleared up quickly by the examiner. Of particular value to the present author, oral exams can minimize student cheating, given that the student is sitting a few feet away during the exam from the instructor and making eye contact. Cheating is a documented problem in business education and one that many instructors would like to address (Becker, Connolly, Lentz, & Morrison, 2006;Brown & McInerney, 2008;McCabe & Trevino, 1995). Last, oral exams have been viewed as career relevant by students in applied disciplines, for example, pharmaceutical education (Sibbald, 1998).
While some researchers have explored group oral exams (Davis, 2009;Ockey, 2009), the current literature review focuses on oral exams conducted with individual learners. As an example, one school that has used oral exams for all graduates before they earn their diploma is St. John’s, a 4-year liberal arts college (Winslow, 2009). Donning full academic cap and gown, a fourth-year student enters an ornate, formal meeting room with three “tutors” (faculty) and must respond to 1 hour of questioning about his or her capstone essay (e.g., about a classic book). Anecdotal evidence, as reported byWinslow (2009), suggested that students view this capstone oral exam as important for building their confidence, developing their critical thinking skills, and increasing the ability to “think on their feet.”
Empirical studies of oral exams have been reported in the research literature in fields such as foreign language, pharmacology, medicine, nutrition, and sports education, as well as business education. In medical education, for example,Evans et al. (1966)conducted a classic empirical investigation of oral exams. In their review of the medical education literature to that point, they discussed a study byCarter (1962)that reported findings of positive correlations of oral exam scores with written scores (r= .45), indicating that both written and oral exams tap course knowledge to some extent. In Evans and colleagues’ own study of 25 internal medicine students, oral exam grades did not statistically correlate with written exam grades (r= .314), but oral scores were statistically significantly correlated with students’ total Medical College Admission Test scores (r= .335) and verbal Medical College Admission Test subgroup scores (r= .362). Also, there was a high degree of reliability within the three examining teams (rs ranged from .77 to .85).Evans et al. (1966)also found that students’ oral exam scores correlated negatively with the examiners’ speaking time and number of words used, but correlated positively with the students’ number of words and speaking time. These findings indicated that examiners view student performance more negatively if examiners have to talk, explain, probe, or clarify more frequently during the oral test.
In two pharmacology courses,Sibbald (1998)similarly studied skills developed by oral clinical skills exams in a sample of second and third year undergraduates. Students indicated that their preparation for the exam and their experience in the oral exam itself would improve their information-gathering skills, counseling skills, and improve the ultimate pharmaceutical care provided for patients. Participants in this study also reported oral exams as a useful tool for assessing their communication skills and their knowledge of nonprescription medications, and they even requested the use of oral exams in subsequent program coursework.
In four undergraduate sports degree programs (n= 157 students),Oakley and Hencken (2005)reported on their pilot test of an oral examination. Evidence was presented of a moderate correlation between the oral examination scores and students’ mean overall end-of-year performance (r= .53,p< .05). Three of the four student cohorts (with mixed reactions in the fourth cohort) also reported an overall acceptance of the value of the oral exam approach, and most students in the study recognized the close vocational links of the oral assessment with their career trajectory (e.g., sports medicine). Oakley and Hencken concluded that the oral exam motivated many of the students to prepare harder than they did for traditional written exams.
Simper (2010)studied the use of an oral exam in a final-year nutrition module using 37 undergraduate students. The performance on the oral exam was compared with the students’ previous year’s written paper, and no statistical difference was found in average scores. Students reported that the oral test was more anxiety provoking than the written exams they had taken in their program; however, students also qualitatively reported the experience as helpful for employment search in their field because of its “real-world nature.”
Few published studies of oral exams in business education exist. In a study of a peer-to-peer learning approach in introductory programming classes at two large state schools,Gill and Hu (2006)discussed their use of oral exams to “validate” student learning in course assignments. Because there were no traditional exams in the course, the authors wanted to ensure individual student learning. If, during the oral exam, a student was unable to convince the instructor or teaching assistant that he or she understood the submitted assignment, the student was placed in a “not yet passed” status with no assignment credit and was allowed to retake the oral exam. Students in the study agreed that oral exams provided a fair assessment of their knowledge across the assignments. Gill and Hu anecdotally reported other benefits of using oral exams, including students’ perception of enhanced communication skills along with an increase of peer-to-peer coaching (as students tutored teammates who had to retake an oral exam).
As pressures mount for larger class sections in higher education (Guder, Malliaris, & Jalilvand, 2009;Thanopoulos, 2004), conducting oral exams with each student is more challenging, especially if competent teaching assistants are unavailable. Administering oral exams in the online educational environment might also prove unworkable, although possible, via Skype or other technologies. Furthermore,Roberts, Sarangi, Southgate, Wakeford, and Wass (2000)suggested that examiners be carefully selected, trained, and monitored, or else scores unrelated to student competence may occur.Evans et al. (1966)pointed out the possibility of rater errors, student fatigue, and the tendency of some examiners to overuse questions in their area of interest. Arguably, these problems can occur in the use and scoring of any type of exam (written or oral). For example, rater bias can certainly occur when examiners know a test-taker’s identity, students can fatigue during traditional written exams, and faculty can overemphasize test questions of personal interest on written exams. Means of addressing these issues in oral exams will be discussed in the implications section.
Summary of Literature Review
We can conclude the following synthesized points from this literature review on using oral examinations in higher education:
Oral exams are claimed to be good tests of students’ depth of understanding, ability to organize ideas, and ability to use language to express ideas.
Oral exams can minimize students’ ability to cheat or engage in academic misconduct.
Documented benefits of oral exams include perceived fairness by students and perceived link to students’ vocational interests and relevant skill development (e.g., oral communication and one-on-one interviewing or counseling skills).
Oral exam scores tend to be correlated with traditional and standardized exam scores.
Oral exam processes tend to demonstrate high interrater reliability.
Cons of oral exams include potential rater bias and demands on instructor time.
Few empirical studies in business education focusing exclusively on oral exams exist.
Therefore, asTable 1summarizes, oral exams appear to vary from more traditional testing formats in opportunities to cheat, skills developed, perceptions of vocational relevance by students, rater time, and rater bias. Given the paucity of published research on using oral exams in business, the current study explores an oral quiz approach to evaluating student learning in an undergraduate HR course over multiple semesters. Implications for instructors in the design and execution of oral exams are then presented.
Summary of Oral Versus Written Exam Attributes.
For a discussion of using essay versus multiple-choice question formats to assess student learning, seeParmenter (2009).
Method
Sample
The present author, in a medium-sized, public southeastern U.S. university and Association to Advance Collegiate Schools of Business–accredited business school, used an oral quiz across three sections of an undergraduate HR course over two semesters. The archival grade data were analyzed in aggregate form for the present study. The total sample size of data gathered was 64 juniors and seniors, all of whom were either general management or HR majors in the college of business. In all, 50% were female and 18.75% were non-White (of whom 66% were African American), with English being all the participants’ first language. No participants were international students. The course was Government Regulation of Human Resource Management taught during one fall and one spring semester, with a mean section size of 21.33 students (median = 26).
Procedures
The oral exam was designed to simulate (in part) a one-on-one professional exchange between an employee and manager, in which the student was asked to orally showcase his/her knowledge (seeVerderber & Serey, 1996, regarding the “instructor as manager” approach, p. 36). To increase scoring reliability, the same amount of time was given to each student, predetermined questions were used and stated the same to each student, and the same preestablished evaluation criteria and grading rubric were used for all students. The operational procedures used in this study to administer the oral exam are detailed and summarized inTable 2, including length and location of the oral exam, weight toward final course grade, timing during the semester, question selection (as based on course learning goals), and the scoring process used.
Operational Procedures.
Measures
At the outset of the semester, students were given the specific evaluation criteria against which their performance on the oral quiz would be scored to help clarify expectations and minimize ambiguity. The evaluation criteria, applied to each student, included the following components: (a) demonstration of an accurate grasp of relevant course concepts; (b) provision of a complete answer covering the main points; (c) provision of a clear, articulate answer; (d) ability to converse comfortably by avoiding awkward pauses and nonverbal behaviors (e.g., fidgeting in one’s seat); and (e) a behaviorally exhibited confidence in stated answers (e.g., use of eye contact). These elements, representing performance criteria of value in the U.S. workplace (i.e., verbal and nonverbal communication skills, demonstrating expertise;McLester & McIntire, 2006;Verderber & Serey, 1996;Zinser, 2003), were rated on a scale of excellent (A), good (B), satisfactory (C), or poor (D) to determine a total score. 1
Student performance was evaluated immediately after the student exited the instructor’s office to avoid recall error. Ratings across all five behavioral criteria were generated and then aggregated to arrive at one final numerical score. After submission of final grades in all courses, archival scores were entered into an Excel spreadsheet for research purposes, along with the student’s sex (male or female) and demographic subgroup. Students’ sex was coded as 1 = female, 2 = male, and race was coded as 1 = non-White or 2 = White. Also entered in the data set, for comparison purposes of student performance across various graded opportunities in the course, were the student’s written exam scores and final course grade.
Findings
Correlational analyses were initially performed on the data (totaln= 64, Whitesn= 52, non-Whitesn= 12, femalen= 32, malen= 32). In the overall sample, there was a statistically significant positive correlation between students’ oral quiz scores and written exam scores (r= .66). These findings are consistent with prior studies and indicate that students’ oral quiz performance tracked their written exam scores, with each similarly tapping course learning.
In the aggregate sample, students’ oral quiz scores (M= 88.09) were significantly higher than their average written exam scores (M= 82.89;t= −5.39,p< .00,df= 63). Thus, students in this sample (all of whom were either general management or HR majors) performed better overall in an oral format than a traditional written format. Given the evaluation criteria utilized, an average sample performance of 88.09% reveals that students in the class responded to the oral exam questions with sufficiently complete answers, demonstrated a “good” grasp of the course topics and main learning points, and exhibited sufficient (although not excellent) confidence and comfort in an oral exam setting.
No statistical differences were found between females’ oral quiz scores (M= 88.61) and males’ oral quiz scores (M= 87.58), showing that females and males performed comparably in the oral exams. Too few minorities in this sample precluded a reliable analysis of possible ethnic differences.
More subgroup analyses were then performed using pairedttests. All subgroups performed relatively better on the oral quiz compared with written exams. Specifically, significant differences were found between Whites’ written exams (M= 84.03) and their oral quiz scores (M= 89.26;t= −5.70,p< .00,df= 51); significant differences were found between males’ written exams (M= 82.3) and their oral quiz scores (M= 87.58;t= −4.112,p< .00,df= 31); and significant differences were found between females’ written exams (M= 83.51) and their oral quiz scores (M= 88.61;t= −3.487,p< .00,df= 31). Minorities’ oral exam scores (M= 83.04) and their written scores (M= 78.05), while practically different, were not statistically significant. Again, this result may be attributable to the small cell size for minorities (n= 12).
Discussion and Implications of the Study
Certainly, the fields of pharmacy, medicine, nutrition, foreign language, and sports education should not be the sole users of vocationally relevant assessments, such as oral exams. Given the professional communication skill sets used by managers in the workplace, business education is a relevant venue for considering additional forms of learning evaluation, such as oral exams. The present study provides readers with insights about the pros and cons of using oral examination formats, summarizes empirical findings from prior studies of utilizing oral exams across various fields, and presents results from an initial investigation into the performance of different demographic groups of students on an oral quiz in multiple sections of an HR course.
Specifically, the results of this empirical study of oral exams in management education indicated that (a) student performance on oral measures was consistent with their performance on traditional written exams, (b) students performed better on oral measures than written measures, and (c) females performed comparably to males on oral exams. Given the small number of minorities in this study, testing for possible ethnic differences is recommended using larger cell sizes. Taken to their logical end, although the present study’s findings do not prove oral exams are “better than” written exams, they suggest oral exams are a viable, relevant assessment tool for business courses.
That all students in the current sample of management and HR majors scored relatively high in an oral format may indicate a possible skill strength or innate tendency of this group. (Of course, it may also highlight a relative weakness in their writing skills). In one study that tested a model of factors influencing business students’ choice of a specific major, the authors found in a sample of 112 business students that students will select a major that fits their skill strengths (Strasser, Ozgur, & Schroeder, 2002). Specifically, students in their sample believed that management (and marketing) careers lead to greater usage of interpersonal skills. Because oral exams are grounded in a one-on-one format, it is logical to suggest that management students may thrive in an orally administered testing format and not surprisingly perceive it as vocationally relevant.
Limitations of the study, which minimize the generalizability of the findings, include the one-university setting for study and the potential bias of the “researcher-teacher” approach (Fraenkel & Wallen, 2006). Another study limitation is the use of one rater. Intrarater reliability (i.e., test-retest) could have been calculated in the present study had the oral quiz been administered to students multiple times, but such an endeavor would likely be seen by students as an inefficient use of their time. Nonetheless, this study has limitations that should be considered when interpreting the findings, particularly in other cultures or universities with different attributes.
Implications for Instructors
For those business instructors enticed by oral exams’ assessment of business communication skills, decreased opportunities for student cheating, and enhanced student perceptions of vocational relevance, what issues should they consider? The extant literature and results of this study offer guidance. In the classroom, instructors should explain to students why oral exam formats are being used in the course and why they are relevant to business students’ professional skill development, especially if this format is rarely used in the instructor’s college or department. Based on the author’s experience, oral exams will be readily perceived by business students as relevant to their field of study.
Good testing design and implementation procedures should be followed with oral exams, just as with written exams (Evans et al., 1966). As implemented in the current study, the process should be standardized, meaning that the same amount of time should be given to each student, predetermined questions should be stated exactly the same to each student, and the same preestablished evaluation criteria and grading rubric should be used for all students (Heneman & Judge, 2010;Rubin, 2006;Whetten, 2007)—all in an effort to increase scoring reliability. In other words, rather than a friendly social exchange, which could produce inflated ratings, the rater’s approach should be skilled and methodical. Raters should be selected, trained, and monitored carefully, particularly on typical biases, and they should be using a grading key in a standardized manner (e.g.,Magin & Helmore, 2001). If more than one competent and trained rater is available, it will allow for interrater reliability calculations, which may minimize concerns of validity in the measurement; however, instructors should realize that multiple raters may also increase student anxiety.
Similar to the challenges managers face in real-world performance appraisals and interviewing—and even for those instructors grading identifiable written exams—rater biases need to be managed when scoring performance. There are several suggestions for minimizing rater bias in any forum. As in the current study, questions should be predetermined such that examiners do not, in an ad hoc fashion, pursue questions only in their personal field of special interest (Evans et al., 1966). Also, a halo effect may be more likely if the rater has evaluated the ratee in prior classes. (Fortunately, in the current study, the probability of having taught a student in prior semesters was very low, because this course was the first in a series of higher numbered electives the author taught.) To deal with this, in theRaman-Wilms (2001)study, examiners who had previously assessed a student in any capacity (e.g., a prior class) were not assigned to be an examiner for that student; thus, trained teaching assistants are an option. Moreover, as in the present study, raters should be provided an explicit, written grading key (with specific measureable components) to evaluate student responses. This approach can also help minimize leniency bias and the contrast effect by yoking scores to a standardized template and is similar to the “grading rubric” approach advocated to more objectively assess written assignments (Rubin, 2006;Whetten, 2007). Using a second trained rater may also help in a practical manner; one examiner could ask the questions and the other could write down responses. After the exam, each tester would complete an evaluation form separately, and then the pair would arrive at a consensus to decide the assessment rating for how well the student performed (Raman-Wilms, 2001).
Similar to other forms of assessment (e.g., written exams), high stress levels may distort students’ performance, potentially misrepresenting their grasp of course content and ability to apply it (Burns, 2004;Phillips, 1992). As discussed earlier, ways to minimize uncertainty and stress include communicating to students the evaluation criteria and presenting a “bank” of questions beforehand that students may be asked (as done in the present study). Some instructors may also benefit from a selective use of silence (seeLi-Shih, 2005) during the exam, which provides space for the student to participate (or not), to identify students who are legitimately unprepared versus those who are emotionally overwhelmed yet cognitively prepared. Last, another factor that will likely affect the level of student anxiety in oral exams is the weight assigned to the oral exam grade. Instructors will want the oral exam to count enough to garner sufficient student attention and effort but not too high to produce perceptions of unfairness, although the literature does not address best practice in this area.
Barrett (2002)suggested that there should be “constant assessment” of students’ professional communication skills (p. 93). While the literature provides no specific guidance on the appropriate length of oral exams, the shortest oral exam in the studies reviewed for this article was 10 minutes inSibbald (1998)to mimic the typical length of a pharmacist-patient consultation), and the longest oral exam was a 2-hour oral comp for third- and fourth-year pharmacy students (Raman-Wilms, 2001). Instructors should consider their situation and the career prospects of their graduates to devise a customized approach. University-mandated weekly office hours remain underutilized by students (Nadler & Nadler, 2000), making the administration of oral exams during these already-scheduled office times quite workable. Nonetheless, individually administered oral exams may be more practical in higher level, specialized courses with smaller enrollments, such as electives within a business major that demand expanded professional skill sets from upper level students.
There may be other uses for oral exams in business education as well. In quantitative business courses, students could be given a cost-benefit problem, and then the instructor could administer an oral exam where students are asked to explain their recommendations and justify them, just as they would to their boss in an organizational setting. A more minor role for oral exams could be as the makeup exam format for those who miss in-class tests; this approach would also support instructors who want to first “pilot” oral exams on a smaller scale. Finally, in terms of advance instructor preparation, it would be prudent to first talk with other experienced professors who have used oral exams, explore the academic institution’s receptivity to novel examination approaches, and anticipate situations that might arise. Instructors, for example, should be aware that a student could bring in “cheat sheets” and indiscreetly place them in a location to consult during the exam (i.e., asking students to place their book bag across the room is a potential remedy, along with watching for “wondering eyes” during the exam).
Conclusion
Despite the nature of communication skills actually utilized in professions that business students enter, written examinations remain the norm in higher education in U.S. professional management education. Interestingly, multiple measures are strongly advised when conducting academic research (Judd, Smith, & Kidder, 1991) but for some reason are neglected in the evaluation of student learning, at least in some disciplines. In courses that aim to build and develop specific skills, such as communication, it makes pedagogical sense to actually evaluate students’ demonstration of those skills as part of a larger evaluation strategy, instead of solely tapping cognitive comprehension. In short, when an instructional goal is to build skills, then instructors should attempt to assess skills.
Additional research is needed to outline the conditions for when learning is appropriately evaluated via an oral exam. Future studies could replicate the present findings across other business schools, especially in an international context, and could use larger sample sizes. It would also be useful to study, for example, what country-specific differences exist in using oral exams, since national culture affects individuals’ norms for nonverbal communication. Other proposed benefits of oral exams could also be empirically tested. For example, in a senior capstone oral exam,Winslow (2009)claimed that the approach helped build students’ confidence, develop their critical thinking skills, and increase their ability to “think on their feet.” It would be useful to study whether oral exams could actually increase such skills, and whether such assessment approaches could even be used to strategically differentiate a program of study in the market. Developing business students who are work-world ready is a necessary and worthy goal (McLester & McIntire, 2006;Zinser, 2003), and identifying interventions that most effectively enhance such skills certainly remains understudied. Last, researchers could investigate whether oral exams can ameliorate, or not, the more recently investigated psychological phenomenon of “stereotype threat” (seeNguyen & Ryan, 2008). This concept refers to an individual’s fear of being confirmed as possessing a negative characteristic or stereotype associated with a group.
Finally, for those instructors who desire a more impenetrable method for assessing student learning, in which students have a difficult time engaging in unethical behaviors, oral exams offer a promising option. The desire for high scores, with minimal effort, is all too prevalent across business students, and research has documented students’ willingness to engage in such unethical behavior (Brown & McInerney, 2008;McCabe & Trevino, 1995). Thus, faculty may be enticed to explore the use of oral exams, not only to bolster business communication skills but also to minimize cheating. In summary, I suggest that oral exams can be an important component of business classes to build and assess workplace readiness, demonstrate vocational relevance, and curtail academic misconduct.
Footnotes
Author’s Note
This study was approved by the Institutional Review Board at The University of Tennessee at Chattanooga.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Notes
Author Biography
