Abstract
In the present article, we describe a course exercise in which students were administered four course evaluation forms throughout the semester, on which they provided their overall impressions of the class as well as their desire to change certain aspects of the course. Critically, during the semester, a total of three changes were made to the structure of the course as voted on by the students. Compared to the previous semester where students completed only end-of-semester evaluations, improvements in exam performance as well as instructor ratings were observed. Furthermore, students indicated that the changes made throughout the semester improved the course, and they hoped that other classes would adopt a similar classroom developmental strategy. This supports a growing body of evidence, suggesting that midsemester feedback is crucial for optimizing the learning environment for the student, particularly when concrete changes are made after the administration of course feedback.
It is a common practice in academics for students to assess the effectiveness of an instructor and course through evaluation forms (Newport, 1996). These ratings are important for summative purposes, such as quantitatively comparing teaching performance across various instructors within a department in order to make staffing, tenure, and promotion decisions (Ketteridge, Marshall, & Fry, 2002). Feedback in the form of student ratings also serves the purpose of improving instructor performance (Cohen, 1980). Generally, standardized evaluation forms are given once to students at the end of the semester, where they rate various aspects of the course. Unfortunately, however, this method of feedback does not allow for adjustments in the course until the next time it is taught (Hunt, 2003). End-of-semester evaluations also do not allow instructors to clarify comments written on evaluation forms from current students. As a result, there is a great deal of disagreement regarding the value of end-of-semester evaluations (Benton & Cashin, 2014; Gray & Bergmann, 2003). Issues of validity, reliability, and practicality are also called into question when it comes to student feedback (Langbein, 2005; Moore, 2009). Although Benton and Cashin (2014) debunk a number of misconceptions related to student evaluations (e.g., students are incapable of making consistent judgments of a teacher’s behavior, students are not competent nor qualified to rate an instructor, and students give higher evaluations in easy courses), biases in ratings are still present (Braskamp & Ory, 1994; Feldman, 1978; Marsh & Roche, 1997). As such, many educators have begun utilizing midsemester feedback (MSF) along with the standard end-of-semester ratings to improve teacher effectiveness and the overall learning experience of the student. Here, we had students complete multiple evaluation forms throughout the semester which allowed us to determine how this impacted instructor and course ratings relative to a previous semester when no such forms were administered. Furthermore, it was of interest to determine how ratings and exam performance were influenced when students had the opportunity to change certain aspects of the course throughout the semester.
One important function of end-of-semester evaluations is to improve teaching performance. After receiving feedback, it is the responsibility of the instructor to reflect on the comments and implement changes in the taught course. Surprisingly, however, 77% of instructors fail to change any aspect of the course after receiving student feedback in end-of-semester evaluations (Spencer & Flyr, 1992). Furthermore, of the 23% of instructors who do alter the course, the vast majority of these changes were minor and pertained to only a specific aspect of the course (i.e., changing the pace of lecture). Although it is widely acknowledged that student evaluation forms can be effective (i.e., Cohen, 1981; Koon & Murray, 1995), many instructors perceive the ratings as unreliable and unfair (for a review, see Wachtel, 1998). Unsurprisingly then, it has been shown that the introduction of end-of-semester feedback had no significant influence on the quality of teaching in a department after a 3- or 4-year period (Kember, Leung, & Kwan, 2002). As changes are rarely implemented after an evaluation period, students may also form distaste toward evaluations, feeling that their feedback will be ignored.
Due to these challenges and lack of agreement regarding the value of end-of-semester evaluations, an alternative method of feedback is needed for instructors and students alike. Implementing MSF in a course appears to be beneficial for both the instructor and students as such evaluations are designed to improve rather than judge the course. Interestingly, instructors who utilize MSF receive higher ratings (one third of a standard deviation [SD]) on their end-of-semester evaluations compared to instructors who do not utilize such feedback (Cohen, 1980). Students also rate MSF as more positive compared to traditional end-of-semester evaluations (Abbott, Wulff, Nyquist, Ropp, & Hess, 1990). As Keutzer (1992) highlights, MSF allows instructors to make changes in a course throughout a semester, adjusting and adapting to the needs of the current class. Furthermore, MSF allows a degree of empowerment for the student, where their voice shapes the content and structure of the course. Keutzer also suggests that MSF allows for tailored individual feedback for the instructor rather than the global standardized feedback found in end-of-semester evaluations. Not only is this sort of feedback more practical, but it is also less worrisome for the instructor, as these ratings will not influence administrative decisions such as tenure or promotion. MSF ratings also have the potential to improve the competence, and thus intrinsic motivation of the instructor (Bess, 1977). Instructors feel more competent in guiding students through a course when they receive MSF, since they know which practices are and are not beneficial for students (Cohen, 1980).
MSF also allows the potential for students to indicate certain aspects of the course they would like changed in order to create the best possible learning environment. Past research demonstrates that humans value the freedom and flexibility that choice allows, which strongly influences an individual’s level of motivation on a task (e.g., Brehm, 1956; Brown, Reed, & Summers, 2003). Furthermore, the ability to choose results in more enjoyment when completing a task as well as enhanced performance and creativity (e.g., Amabile & Gitomer, 1984; Langer & Rodin, 1976). For example, Cordova and Lepper (1996) demonstrated the positive consequences of choice even when the choice itself had little influence on the primary task. Elementary-aged school children were taught arithmetic and problem-solving skills while playing a computer video game. When children were given the chance to choose the icon that would represent them in the game as well as name various icons throughout the board, they had increased motivation to play the game, were more engaged, and actually learned more relative to children not given this incidental option of choice. In a college-aged sample, Perlmuter and Monty (1977) investigated the effect of choice utilizing a paired-association task. In the choice condition, participants were able to choose which of five words to associate with a target word. In the no-choice condition, the experimenter chose the word the participant would associate with the target word. Participants in the choice condition demonstrated faster learning and better performance on the pair-association task compared to those in the no-choice condition, once again demonstrating the power of choice in the process of learning.
Some allowance of choice seems crucial then to optimize academic performance for students, where instructors encourage feedback and changes are implemented appropriately (Biggs, 1999; Higgins, Hartley, & Skelton, 2001). To encourage two-way communication between students and teachers, the present students received four course feedback opportunities throughout the semester. Traditionally, students only complete end-of-semester evaluations or end-of-semester evaluations coupled with MSF. In this class, during the second week of the semester, students completed a course feedback form (CFF) where they provided their overall impressions of the class and indicated certain aspects of the course they would like to see changed. Students completed a nearly identical form during 6th and 11th weeks of class, and then a final course report form along with a formal end-of-semester evaluation form during the 16th week of class. This allowed us to determine whether course ratings and exam performance remained stable across the semester as well as if ratings and performance improved compared to the previous semester where students were only presented with end-of-semester feedback. Furthermore, we were able to see whether the opportunity to vote and change three aspects of the course throughout the semester resulted in improved satisfaction in the course as well as enhanced performance on exams compared to the previous semester when no such feedback was requested from students. Critically, students were not changing the overall course structure, but instead minor aspects of the course (e.g., including sample multiple-choice questions at the end of class). As such, it was of interest to determine whether this degree of control had a positive influence on students.
Method
Participants
Seventy-three undergraduate students from a large Midwestern University enrolled in the spring semester section of perception had the opportunity to complete the CFFs. Perception is a three-credit upper level large lecture course, where we analyze and compare approaches to the study of current problems in human perception and information processing, including psychophysical judgment, signal detection theory, perception of form and space, and the role of imagery in perception. Only those students attending class during the days the CFFs were administered completed the survey. As a result, 55 students completed CFF 1, 56 students completed CFF 2, 53 students completed CFF 3, 49 students completed CFF 4, and 55 students completed the end-of-semester evaluation form. We also looked at performance data on the three exams across the spring semester, where 73 students completed each of the exams. These data were compared to the fall semester perception course that I also taught, where 58 students completed the end-of-semester evaluation form and 74 students completed the three exams across the semester. 1
CFF
CFF 1–4 were presented sequentially to students throughout the semester. CFF 1–3 were similar in format, consisting of two sections: (1) overall impressions of the instructor/course and (2) preference on changing certain aspects of the course (see Appendix for an example CFF). In Section 1, across 9 items students indicated on a 1–7 scale—with 7 being the best—their impressions regarding various aspects of the course (e.g., “The lectures or other class presentations are clear and well-organized”; “The course engages you in active thinking about the subject or its application”; see Appendix for a complete listing of questions). Items were an abbreviated version of the end-of-semester evaluation form routinely used by the Department of Psychology (reliability and validity data are unknown). In Section 2, students indicated with a “yes” or “no” response whether or not they agreed with the following changes to implement in the class. The possible changes (brainstormed together as a class in the first week of the semester) included the following: administering more quizzes, covering the textbook supplementary website in class, engaging in more partner discussion in class, engaging in more group discussion in class, including fill in the blanks in PowerPoint during lecture, including more examples of real-world application, including sample multiple-choice questions at the end of each lecture, and watching a supplementary media clip during each class period. There was also room for students to write other changes they would like to see made in the course (usually these suggested changes reiterated the already proposed changes in the CFF and thus no new proposed changes were added to the form across the various parts) and provide any other feedback related to the course. Finally, in CFF 2–4, students indicated with a “yes” or “no” response whether or not the implemented change made in the course improved their overall satisfaction with the course (CFF 1 did not have this question as no changes were implemented yet; only one change was made after each CFF).
In CFF 4, the first nine questions again pertained to the overall impressions of the instructor and course. In Section 2, however, participants answered questions regarding the perceived effectiveness of the various changes made throughout the course on a 1–7 scale, with 7 being the best (strongly agree; the scale is illustrated in Appendix), including “The course evaluation forms/changes I made throughout the semester improved the course,” “The changes I made throughout the semester were implemented and noticeable changes were observed,” and “I would like it if other classes would implement evaluation forms and changes throughout the semester.” Further, students rank ordered the changes from best to worst.
End-of-Semester Course Evaluations
We report here the most critical questions from our department’s end-of-semester evaluation form (“Compared to other instructors you’ve had, how good was this instructor?”; “Compared to other courses you’ve taken, how good was this course?”; “Compared with courses at a similar level, the amount I learned in this course was”). The remaining 16 items on the department’s evaluation form pertain to specific aspects of the instructor and course (e.g., “The course content is appropriate to the course title and description”), do not pertain to learning, show little variation across semesters and courses, and thus were excluded from the analysis.
Procedure
On the first day of class, I informed students that they could voluntarily take part in a course development exercise throughout the semester where they would anonymously evaluate the class 4 times (not including the end-of-semester evaluation form) as well as indicate changes that they would like to see made in the course. Students completed CFF 1 during the second week of class. Afterward, CFF 2 was completed during the 6th week of class, CFF 3 during the 11th week, and CFF 4 was completed during the 16th week of class. Students were instructed to be honest and straightforward when completing the form; however, in contrast to the typical end-of-semester evaluation forms, I informed students that I would be the only one seeing these evaluations and that they will be used to enhance the class. Three changes were made throughout the course as voted on by the students, each of which started immediately after the completion of CFF 1–3. Specifically, students voted to have more supplementary videos shown in class (CFF 1), cover sample multiple-choice questions at the end of each lecture (CFF 2), and include more real-world examples during lecture (CFF 3). The following day after students completed CFF 1–3, I presented the mean ratings of the first nine questions in Section 1 to the class. I then summarized to the students the strengths of the course that the students highlighted as well as areas of weakness and how I planned to address these areas. When summarizing the results of Section 2, I would present to students the percentage of the class that stated “yes” to each of the proposed change. The proposed change that received the highest percentage “yes” vote in class was implemented.
Results
Summary Statistics of Changes Made
The first aspect of the course I changed was showing a supplementary movie clip in class during each lecture period, as 89% of the students indicated “yes” to wanting to see more videos in CFF 1. In CFF 2, at the end of each lecture, I began including two sample multiple-choice questions at the end of class (93% of students voted “yes” for this change), and in CFF 3, I included more real-world examples in lecture (one slide during each lecture was specifically devoted to highlighting the practical importance of the material discussed in class; 91% of students voted “yes”). In CFF 4, students rank ordered, on a 1–3 scale, the changes from best to worst. Students indicated that the change they liked the most was the sample multiple-choice questions (44% of the class), followed by the inclusion of in class videos (29% of the class) and then having more real-world examples in class (27% of the class; note that all changes made were viewed favorably).
Effectiveness of Changes
When examining the effectiveness of the changes made throughout the semester, 95% of the class indicated that the movie clips improved the overall experience of the class, 94% of the class indicated that the sample multiple-choice questions improved the class, and 98% of the students believed that having more real-world examples in the final 5 weeks of class improved the course. Further, in CFF 4, overall on a 1–7 scale, with 7 being the best, students believed the CFF improved the course (M = 6.08, SD = 1.10), that the changes made throughout the semester were implemented and noticeable changes were observed (M = 6.18, SD = 1.05), and students believed other classes should implement evaluation forms and changes throughout the semester to improve the course (M = 6.27, SD = 1.06).
Comparison to Previous Semester Ratings
As students were satisfied with the changes made throughout the semester and believed these changes improved the overall quality of the course, we also looked at whether the class and instructor ratings differed in the spring semester relative to the previous semester when CFFs were not administered. The average student ratings from the fall and spring semester are found in Table 1. Although we are comparing two discrete classes, the format of the class did not change across the fall and spring semester with the only major difference being the use of CFFs in the spring semester. Demonstrating the effectiveness of the CFF, students rated the instructor higher in the spring semester compared to the fall, t(110) = 3.99, p < .001, d = .76. Although trending where ratings were higher in the spring compared to the fall, there was no significant effect across semesters for the general class rating, t(110) = 1.65, p = .10, d = .31, and students’ perceived amount of information learned, t(110) = 1.39, p = .17, d = .26.
Mean Student Ratings.
Note. Mean rating across CFF 1–4 and the end-of-semester evaluation form from the fall and spring semester. Students indicated on a 1–7 scale, with 7 being the best, their impressions regarding various aspects of the course. CFF = course feedback form.
Coupled with the usage of the CFFs, it is possible that simple experience with teaching perception an additional semester could have contributed to the increase in ratings. However, this seems unlikely due to two factors. First, ratings of the instructor were higher in CFF 1 compared to the end-of-semester evaluations during the fall semester, t(111) = 3.29, p = .001, d = .62. As CFF 1 was administered only 2 weeks into the spring semester, little experience was gained in teaching, and therefore this boost in instructor ratings may be attributed to the CFF. Second, if experience were associated with improvements in teacher ratings, one would expect a linear trend across the semester, where ratings would gradually increase across CFF 1–4 (for means and SD across CFF 1–4, see Table 1). During the spring semester, student ratings marginally decreased when comparing CFF 1 to CFF 2, t(109) = 1.76, p = .08, d = .33. There was no significant difference when comparing CFF 2 to CFF 3, t(107) = .16, p = .87, d = .03, and a significant increase when comparing CFF 3 to CFF 4, t(100) = 3.16, p = .002, d = .62.
CFF and Performance
Although students had an objectively positive opinion of the CFF, it is also worthwhile to consider whether students in the spring semester had improved exam scores relative to students in the fall semester where no CFFs were administered. In the fall semester, 74 students completed three exams while in the spring semester, 73 students completed the same three exams. Each exam consisted of 50 multiple-choice questions surveying topics covered in lecture. The first change that was made during the spring semester was including a video clip in each class period starting the second week of class. For Exam #1, which took place during the sixth week of class, students performed better in the spring semester (M = 76.44, SD = 13.42) compared to the fall semester (M = 71.41, SD = 12.62), t(145) = 2.34, p = .02, d = .39. This same pattern held true for Exam #2, t(145) = 2.18, p = .03, d = .36, in the (spring (M = 80.49, SD = 12.66) and in the fall (M = 76.00, SD = 12.31) during which sample multiple-choice questions were included at the end of each lecture. However, with regard to Exam #3, there was no difference in performance, t(145) = .10, p = .92, d = .02, in the spring (M = 76.85, SD = 10.69) and fall semester (M = 77.03, SD = 10.22). In the final 5 weeks of class in the spring semester, more emphasis was placed on real-world applications.
General Discussion
In the present article, we have demonstrated how multiple feedback forms throughout the semester—which afforded students the ability to change certain aspects of the course—influenced overall class ratings and exam performance. Across CFF 1–3, students voted to have more supplementary videos shown in class, cover sample multiple-choice questions at the end of each lecture, and include more real-world examples during lecture. Critically, the vast majority of students believed each of these changes improved the overall quality of the class. Unsurprisingly then, students also indicated that they wished other classes would implement evaluations forms like the CFF. Compared to the previous semester teaching perception when I did not administer CFFs, students in the spring semester rated the instructor significantly higher than students in the fall semester. Furthermore, students in the spring semester outperformed those in the fall semester on two of the three exams.
Improved scores for Exam #1 and Exam #2 in the spring could be the result of feedback improving the quality of teaching from the instructor (Cohen, 1980; Overall & Marsh, 1979). Instructors feel more competent teaching a course when they are aware of what practices are and are not beneficial for students (Bess, 1977). The CFFs allowed me the opportunity to measure student interest and understanding of the material as the course was being taught, which could also result in improved instructor performance. However, this does not explain why there were no performance differences for Exam #3. While covering the material for Exam #1, I presented to students’ various videos supplementing the difficult anatomical features of the eye and visual cortex as well as videos depicting the physiology behind how individuals encode light during the different stages of the visual process (videos I did not show during the fall semester). By showing these videos, students had another way in which to learn the material apart from lecture and reading the textbook. For Exam #2, by covering sample multiple-choice questions at the end of lecture, students had more opportunities to gauge their knowledge of the material as well as gain more experience with the nature of actual exam questions. Furthermore, testing of information has been shown to improve long-term retention of the material (Roediger & Karpicke, 2006). As this was not afforded to students in the fall semester when covering the material for the exam, this change in particular seems likely to have contributed to the boost in exam performance. Including at least one real-world application in lecture, though inherently interesting and important, may not directly result in improvements in exam performance to the same degree as showing videos during lecture and including sample multiple-choice questions. Although all but one student perceived the real-world examples as improving the overall course, these examples may not result in better retention and understanding of the material.
Interestingly, all three implemented changes were perceived as effective even though this is not reflected based solely on exam performance. Although adding videos to lecture and providing students with sample multiple-choice questions appear to help students learn difficult material, these changes were minor compared to the overall scope of the class. Our observations suggest that this ability to control certain aspects of the course can increase student satisfaction with a course and instructor, which may in turn result in greater motivation to attend class, learn the material, and perform well. Although it may seem like a great deal of additional effort, finding videos to show in class as well as including sample multiple-choice questions and real-world examples were quite easy to implement and also serve to benefit the instructor given that they may learn new things about the material themselves. In fact, these were teaching tools I have contemplated utilizing in my other classes before, and therefore I was eager to obtain an indication of their effectiveness. Educators considering incorporating different teaching methods into class may want to provide students the opportunity to vote and choose the methods to incorporate, as the present exercise highlighted the beneficial power of student control over course structure. Even allowing students the opportunity to voice their opinion on matters related to other minor aspects of the course such as when assignments are due may be enough to elicit some degree of control over course structure.
What we observe here is consistent with previous research suggesting the importance of choice in learning (i.e., Cordova & Lepper, 1996; Perlmuter & Monty, 1977). By providing students with multiple feedback opportunities and the ability to alter certain aspects of the course, this would seem to result in enhanced intrinsic motivation for students. Intrinsic motivation has been shown to improve student performance (Adelman, 1978; Gottfried, 1985), where students who are intrinsically motivated academically spend longer amounts of time on task (Brophy, 1983), continue working on a task even when it is difficult (Gottfried, 1983), and are more interested in learning (Gottfried, 1983). Intrinsic motivation would appear to be of particular importance than for upper level psychology courses such as research methods and design, statistics, and perception, where the content is often more difficult as compared to introductory courses. The CFF along with other MSF variations could be a useful method to increase students’ appreciation and willingness to learn difficult material. Relatedly, it is possible that the implementation of the various CFFs conveyed to students that the instructor cared about student learning and satisfaction. In an educational setting, a caring relationship between students and the instructor not only involve the instructor inherently caring for the well-being of the students but the students also feeling cared for (Philipp & Thanheiser, 2010). Therefore, students must sense that their instructor is enthusiastic about the class content and genuinely wants the students to perform well and learn the material. Philipp and Thanheiser argue that one approach to develop this caring relationship is for instructors to view the class as composed of individuals rather than one large group. Although speculative, it is reasonable to assume that administering CFFs throughout the semester can help foster this caring relationship, contributing to the positive effects of these forms. Even in a large lecture courses, administering forms like the CFF may result in students feeling like valued individuals in the classroom, and not simply a roster name.
Whether it is the CFF or another variation of MSF, in order for any evaluation form to be effective, instructors must encourage feedback and implement the appropriate changes. The changes made in the reported class were obvious as they were implemented immediately after a CFF was administered. Unfortunately, however, students are oftentimes confused about the purpose of evaluation forms (Ahmadi, Helms, & Raiszadeh, 2001). Adding to the confusion, students are rarely given a summary about what was stated on these forms (Dunegan & Hrivnak, 2003). It appears then that we as educators need to change the culture behind evaluation forms. Our observations suggest that CFFs administered during the semester with the purpose of improving rather than judging the course contributes to improved student satisfaction with the course, where students understand the purpose of the forms and were confident that changes were going to be made. This is further highlighted by the number of positive comments received from students in the spring semester on the end-of-semester evaluation form, where students wrote comments like “I appreciated the instructor’s willingness to collect feedback from students and actually make changes in the course. This was a fun exercise that other classes should adopt.”
Although students rated the instructor higher in the spring semester compared to the fall semester when no CFFs were administered, it is important to acknowledge that additional factors may have influenced the change in scores. Most critically, as we are comparing across semesters and therefore comparing two discrete classes, it is possible that students in the spring semester were simply “better” compared to their counterparts in the fall. It is important to note, however, that the class was taught in as identical a manner as possible across both semesters. For instance, I covered the same material in the same order with little dissimilarity in slide content across semesters (outside of changes related to the CFF). Furthermore, the grading scheme remained the same, and test and quizzes were administered at similar time points in the respective semesters. Although I was more experienced teaching perception in the spring semester, it seems unlikely that experience teaching is contributing to the improvements in ratings across semesters. Previous research has shown that experience is not associated with improvements in teacher ratings, where after examining evaluations over a 13-year period (including instructors who began with minimal teaching experience), there was no evidence that a relationship between experience and ratings regarding the instructor and course exists (Marsh & Hocevar, 1991). Therefore, in general, there appears to be no correlation between teaching experience and student ratings (Benton & Cashin, 2014; for an alternative account, see Spooren, Brockx, & Mortelmans, 2013), providing evidence that the inclusion of the CFFs was primarily responsible for the boost in ratings found in the spring semester compared to the fall.
Future research will be required to understand the effectiveness of classroom developmental techniques such as the administration of CFFs, given that we only compared course evaluation ratings and exam scores across the fall and spring semester of a single class. Although the course content remained the same across both semesters, there could certainly be factors external to the CFF that could contribute to performance and rating differences. For instance, I had additional experience teaching the course in the spring having taught the same course in the semester prior. Although the idea for this exercise was generated ahead of the fall semester—allowing me to ensure that I maintained everything the same across both semesters with the exception of the CFF—it is possible that additional experience with the material will make an instructor appear more confident or knowledgeable. Nonetheless, student ratings of the effectiveness of the CFFs demonstrate the positive influence that these forms can have on a course. Considering the CFF gathered both formative feedback and had students vote on changing certain aspects of course structure, future research is also needed to examine whether the positive effects derived from this form are due to (a) gathering general formative feedback throughout the course, (b) making changes to the course based on a student vote, or (c) a combination of these two factors. It is our prediction that both aspects of the CFF improved the overall quality of the class, as adjusting and making changes to a course is a pivotal aspect to any type of feedback. Simply soliciting feedback from students without making changes to the course seems counterproductive, especially because students have little confidence that instructors and administrators take the feedback seriously (Spencer & Schmelkin, 2002). Therefore, we believe any valuable midsemester CFF should result in concrete changes to the course. Finally, future research can explore whether multiple CFFs are necessary or whether one midsemester or early semester evaluation would be sufficient to foster student interest and performance. We predict multiple forms have an advantage over the standard MSF as students’ opinions on the class and the methods used to convey information may change from one point in the semester compared to another. Multiple CFFs allow for this continual gauging of student satisfaction, and the opportunity to change certain aspects of the course throughout the semester. It is our hope that more instructors will begin utilizing CFFs and other variations of MSF to improve the learning experience for both students and faculty alike.
Footnotes
Appendix
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.