Comparing student self-assessment to individualized instructor feedback

Abstract

The literature confirms the commonsense belief that feedback promotes learning. However, personalized feedback, especially in an online environment, can be exceedingly time-consuming for the instructor and may not improve student learning. To test this, a non-random sample of students in three sections of an online statistics course received individualized feedback on weekly homework assignments that were graded solely on completion as pass/fail; students in another three sections of the course were responsible to assess their own homework (but not other projects or examinations) via posted answer keys. A total of 47 students voluntarily completed objective questions testing their knowledge of the subject matter at the end of the course. Overall, there was no difference in learning between the two groups, nor were there any differences in student satisfaction of the course or the instructor. Caveats and implications are discussed.

Keywords

answer key feedback grading online learning self-assessment self-directed learning statistics

Importance of comparing instructor feedback to student self-assessment

Several studies have confirmed the commonsense understanding that feedback is important to student learning (Black and Wiliam, 1998; Hattie and Jaeger, 1998). With many colleges and universities moving toward online courses in an effort to accommodate the demanding schedules and varied needs of both traditional and non-traditional students and to increase revenue, creating distance between the instructor and the students raises questions about how to best assist student learning through feedback. This is a particular area of concern in course areas like statistics, which necessitate acquiring highly specialized skills in mathematics, require developing familiarity with related software applications, and have potential to generate high levels of stress and anxiety among some students. Given the statistics requirement in many programs and the demand on instructors’ time with increasing class sizes, understanding effective modes of feedback is important. However, as Carless (2006) points out, “Despite its central impact on learning, feedback is still relatively underexplored” (p. 220). The teaching and learning literature offers suggestions on how to give feedback to students in a variety of classes (see, for example, Nicol and Macfarlane-Dick, 2006), including courses offered online, but relatively few studies examine whether who assesses students’ work—students or instructors—influences student learning outcomes and student satisfaction (but see Huxham, 2007).

There are several arguments for using each feedback method. Self-assessment, for example, may be more engaging for students, encouraging them to take a more active role in their learning and to be self-directed, lifelong learners; illuminating key processes; and increasing their learning potential (see Black and Wiliam, 1998). Students find benefit in model answers as this method gives a concrete example of “how I should have answered,” is useful to study for examinations, may be “easier to read than handwriting,” and the feedback is immediate (Huxham, 2007: 607). Additionally, posting an answer key with general comments is less time-consuming for instructors than to review many assignments to write individualized (but usually similar) comments, which students may or may not review anyway (see Price et al., 2010). Indeed, Krause et al. (2009) argue “when students are not explicitly activated, they often rather passively read than actively self-explain and mindfully process example information” (p. 159). Posting answer keys “activates” students to learn, helping them become lifelong learners, and frees instructor time to focus on specific student misconceptions, concerns, and individual needs (see Black and Wiliam, 1998).

Self-assessment, though, may not be a panacea. The process is confounded by the many complex layers of what Evans (2013: 97) articulates as the feedback landscape. For instance, depending on time, surroundings/setting, and any variety of context-influencing variables, students may not take the time to review their work. If they do, they may not recognize the differences between their work and the answer key (Larreamendy-Joerns et al., 2005), which can be especially problematic when learning statistics. Additionally, students are sometimes frustrated with the idea of having to “teach” themselves, which can impede learning (Kreiner, 2006). Students report a preference for personal feedback because it “helps me to understand where I individually went wrong,” it “allows personal improvement,” and the comments can be encouraging (Huxham, 2007: 607).

Given arguments supporting both modes of feedback, which is more conducive to student learning? We first need to review the theoretical relevance of self-assessment in student learning, followed by a review of the extant literature, focusing on the role of feedback in student learning.

Sadler’s theory of self-assessment for effective learning

The theoretical approach to understanding the relationship between feedback and student learning was developed by Sadler (1989). While he acknowledges that this theory is most relevant to qualitative work rather than assignments with correct/incorrect responses, the theory is applicable here to explain the relationship between feedback and student learning. According to Sadler (1989), there are three necessary conditions for feedback on formative assessments to benefit learners: students must (1) have knowledge of the standard or goal to be achieved, (2) assess their own work against that standard or goal of good performance, and (3) take action to improve their work to meet that standard or goal (i.e. “closure of the gap” between present work and what is expected). In this way, students learn.

Nicol and Macfarlane-Dick (2006) point out some underlying assumptions about student self-regulation, namely, that the student has limited control over the learning process. While students are responsible for self-regulation, they must operate within an environment created by the instructor—that is, instructors are responsible for the learning tasks. Additionally, students do not always need to follow this process in order to learn; “students often learn in implicit or unintentional ways without explicit regulation” (Nicol and Macfarlane-Dick, 2006: 205).

Nevertheless, based on this theory, Sadler (1989, 1998, 2010) concludes that students should be given the experience of evaluating themselves (see also Nicol and Macfarlane-Dick, 2006). Sadler (1989) argued that instructional systems are—or at least should be—designed to move students from being recipients of feedback (external to the student) to having the ability to self-assess (internal to the student). Accordingly, Sadler encourages instructors to help students along this process by showing them how to assess their own work. One way to do that is using exemplars (Sadler, 1989)—examples of ideal assignments, either generated by instructors or exemplary work written previously by students (Handley and Williams, 2011; Huxham, 2007). Exemplars can be applied to a variety of assignments and are particularly appropriate for written assignments such as essays (Huxham, 2007). Handley and Williams (2011) propose offering exemplars with annotated feedback or having students critique the exemplars to develop their own feedback; while this may not necessarily increase assignment grades, students find them useful. Exemplars are distinct from model answers (or “answer keys”), which are better suited for questions or problems with discrete responses that should be identical across students (Huxham, 2007). In particular, model answers would be more appropriate for assignments in courses like statistics.

Huxham (2007: 603) argues that using model answers fits well into Sadler’s (1989) three criteria for effective feedback listed at the outset of this section. By distributing answer keys, students have immediate feedback instead of having to wait for instructors to return personalized comments. Additionally, model answers lack personal comments; commonsense dictates that students dislike feedback they perceive as negative—an attribute of personalized feedback confirmed by research (see, for example, Nesbit and Burton, 2006). Thus, students are less likely to have a negative experience with model answers. Furthermore, students must assess their own work using the answer key provided, in line with Sadler’s (1989) second criterion. Model answers force at least some degree of student engagement. Finally, model answers demonstrate the ideal standards of the learning module.

Research evidence confirms Sadler’s theory that practice does indeed improve students’ ability to assess their own work (see Black and Wiliam, 1998). For example, Boud et al. (2013) compared undergraduate design students’ voluntary self-assessment on several tasks with tutors’ assessments; over time, the assessments converged. However, judgment on one’s own performance varied by level of achievement, with low achievers overestimating their performance and high achievers underestimating their performance (Boud et al., 2013). Also, consistent with Sadler’s (1989) theory, students who could accurately self-assess improved their performance on subsequent tasks (Boud et al., 2013).

Comparing feedback methods

Several studies indicate that student self-evaluation is conducive to learning. For example, using a randomized design, Cook et al. (2005) found that students performed better on tests after modules where students had a set of self-assessment questions at the end than students who completed modules without self-assessment questions, although there was no difference between these groups of students on the final examination. Engaging students to self-regulate their own learning also may affect student satisfaction with the course and the instructor. Magel (1996) placed answer keys to group worksheets on reserve in the library; this technique may have contributed to higher student evaluations and increased student retention in a face-to-face statistics course. But, of course, any type of feedback is better than none at all. Yalcin and Kaw (2011) compared individualized instructor feedback with not grading homework assignments in a mathematical methods course. They found no difference in student learning between the two methods, although some students simply did not complete the work when it was not marked, which adversely affected their performance. Few studies, though, compare student self-assessment with individualized instructor feedback.

One notable exception is a recent evaluation by Huxham (2007), who compared student self-assessment using answer keys with personalized instructor feedback among three groups of students enrolled in an undergraduate biology course. Half of the questions or assignments received personalized feedback, while model answers were distributed for the other half. Survey results (with 58% of students responding) indicated that students prefer to have both model answers along with individualized feedback or personalized feedback only; only 11% of students had a preference for the model answers. However, students performed better on examination items drawn from assignment questions using model answers and poorer on examination items drawn from assignment questions with personalized feedback—and the difference was significant. In short, students learned more from feedback with model answers but preferred personalized feedback.

Indeed, “who” does the assessing (student vs instructor) may be important (Garfield and Gal, 1999), but to date, empirical investigation is minimal (see Evans, 2013). In line with our research question, “Is individualized instructor feedback or student self-assessment more conducive to student learning?” we test three hypotheses. First, we expect that there will be a difference in student learning between students receiving personalized instructor feedback and students required to self-assess using model answers. Additionally, we anticipate a difference in student performance between high- and low-achieving students. Given the literature suggesting student frustration with “teaching themselves” statistics (Johnson et al., 2009) and prior studies indicating students prefer personalized feedback to model answers (Huxham, 2007), we also expect to find differences in student satisfaction with the course and with the instructor between these two groups. Accordingly, our hypotheses are as follows:

H1. Students receiving personalized instructor feedback will score differently on learning assessments than students who evaluate their own work.

H2. High-achieving students will score differently on learning assessments than low-achieving students.

H3. Students receiving personalized instructor feedback will have different levels of satisfaction with the course and the instructor than students who evaluate their own work.

Methods

Course description

About 300 students (70% female; 30% male) enroll in each cohort of this online criminal justice graduate program housed at a small, private university located in Midwestern United States. Almost half (48.3%) of the students are between the ages of 21 and 30 years, another quarter (26.2%) are between 31 and 40 years, 14.8% report they are between 41 and 50 years, and just over 5% are older than 50 years (5.4% did not report their ages). Many students (77.6%) enrolled in this online program work full time in the criminal justice field. Accordingly, the online courses are asynchronous, that is, students do not have to be online at certain times. In the online graduate-level statistics course used in this study, students typically have 1 week to complete each unit, and content ranges from levels of measurement and descriptive statistics through inferential statistics like t-tests, analysis of variance (ANOVA), and chi-square, with a heavy focus on practical applications using SPSS software. In this asynchronous format, students can post questions about the material on the online discussion board, where they can communicate with other students and the instructor. They also can email or telephone the instructor for additional help.

To complete the weekly “homework” assignments, students read from two textbooks. Students also review comprehensive PowerPoint slides (without audio lecture), additional notes and links to supplemental support sites, and participate in content-related discussions. The weekly homework assignments typically consist of about six word problems for students to solve and are not marked for accuracy; instead, students receive full credit for timely completion of the unit assignments. The online format requires that each assignment is submitted typed in one Microsoft Word (or Word-compatible) file.

By the end of the course, students should (1) be able to organize, summarize, and interpret data accurately and professionally; (2) be able to make proper application of selected statistical techniques to criminal justice data and situations and present data and results clearly and professionally; (3) be able to conduct computerized statistical analyses using SPSS as a computational aid; (4) be able to graph data in Microsoft Excel and SPSS; and (5) have advanced their knowledge as consumers of research.

Design

During the spring, summer, and fall semesters of 2009, the first author instructed six sections of the course (two in spring 2009, one in summer 2009, and three in fall 2009). In half of these sections (one spring section, one summer section, and one fall section), the instructor provided personalized feedback on all weekly student homework assignments. This individualized instructor feedback consisted of highlighting incorrect answers in the student’s Word document and showing the correct way to solve the problem, offering additional comments specific to the students’ work and suggesting students post questions on the discussion board. Examples of such personalized feedback can be obtained from the authors. Using this method, the instructor was able to clearly identify for the student his or her mistakes and offer corrections, with an encouraging explanation. That is, students were recipients of feedback (Sadler, 1989). In the other half (one spring section and two fall sections), answer keys were posted online and were available to students at the beginning of the following unit (and available throughout the remainder of the course). Examples of such answer keys can be obtained from the authors. Here, students were tasked with self-assessment on all assignments (except examinations), which were the same across all sections. Unless students asked questions, the instructor offered no guidance (other than the posted answer key) to help students differentiate between their responses and the correct answers; students were expected to critically evaluate their work. In other words, students had to identify every aspect of their work that was different from the posted answer key. While the instructor did not comment on this group’s assignments, she answered questions that students posted on the discussion board.

Students were unaware of the differences in the course design, that is, the instructor did not inform students of the other group. In this quasi-experimental design, two groups (individualized instructor feedback vs student self-assessment) were compared on an objective end-of-semester assessment, their final letter grades, and the end-of-course student evaluation of the instructor. Despite the advantage for providing the best evidence causal influence, relatively few empirical studies of feedback use experimental or quasi-experimental design compared with case studies (Evans, 2013). However, following the precedent set by prior research (e.g. Van Gundy et al., 2006), the quasi-experimental design is appropriate here to assess differences between the two groups, in line with the research question and hypotheses, although purely random group assignment was impossible.

Data

Students who were enrolled in the first author’s online statistics course were asked to voluntarily complete a quantitative summative assessment, consisting of 30 multiple choice questions to survey students about their knowledge of material presented in the course, specifically reflecting the learning outcomes listed above. This assessment was not marked for credit, and completion was optional but encouraged. With ethical approval from the institutional review board determining the study was exempt from review, the assessment was administered during the unit before the final examination (i.e. 14th out of 15 units). Each question was coded “1” for a correct response and “0” for an incorrect response. The number of correct responses was summed to create a total correct score for each student. Because this objective assessment is qualitatively different from the word problems students are asked to complete on both the homework assignments and examinations, students’ final letter grades were compared, as well.

Taking into account research suggesting students may differentially process information and considering that all students in the sample are graduate students who may be highly motivated, we further disaggregated students by achievement level, assuming that differences may be masked by student achievement level. Student achievement level was determined as “high” if students scored at or above the median of 85 points on the first examination and “low” if students scored below 85 points on the first examination.

In addition to these data, student satisfaction with the instructional style was assessed at the end of the course. Instructors receive only aggregate information (i.e. the number of students who select a given response to each question), so end-of-course evaluations cannot be linked to individual students. Accordingly, the instructor’s evaluations from the three sections who received individualized instructor feedback were compared to the three sections of students who were exposed to answer keys. Four questions from the evaluations were compared. First, students often inaccurately assess their own learning (Boud et al., 2013), and the dissonance between their perception of what they know and their actual performance may indirectly tap into their satisfaction with the course, as students who do believe they learned little likely will not be pleased about investing in the course. Accordingly, the first question drawn from the evaluations tapped into student perceptions of their own learning: “Do you feel, as a result of having taken this class, that you better understand the subject matter of this course?” In addition to indirectly assessing course satisfaction, this measure allows us to compare learning subjectively. Two questions were used to measure student satisfaction with the instructor—satisfaction with instructor encouragement of student learning and, given research indicating negative feelings reported by students when they feel they have to “teach themselves” (Johnson et al., 2009), satisfaction with instructor performance. These questions are, respectively, “Did the instructor appear interested in your academic performance?” and “Did the instructor appear prepared to teach this class?” Because students have feedback preferences (Huxham, 2007) that may influence their overall satisfaction with the course depending on the type of feedback used, the final question assessed student satisfaction with the course: “Overall, were you satisfied with the course?” Response options included 1 = not at all, 2 = somewhat, 3 = to a large degree, and 4 = yes. The average response to each question was computed for students in each group.

Results

A total of 97 students were enrolled in six sections of the statistics course under study. Largely due to class size being capped at 25 students and some attrition, class sizes were small, ranging from 13 to 19 students per section. A total of 47 students (48.5%) completed the assessment, while 50 (51.5%) did not. Of the students who completed the voluntary assessment, 23 received individualized instructor feedback and 24 were exposed to the posed answer key. There was no statistically significant difference in the final letter grades of students who completed the assessments and those who did not (χ² = 1.330, df = 2, p = 0.514). An independent samples t-test confirmed this as these groups did not differ in the average number of points earned throughout the course (t = −0.311, df = 95, p = 0.757). So, we are comfortable with the assumption that the sample of 47 students who completed the survey, although not randomly selected, are representative of the performance of all students enrolled in the statistics courses under study.

Are there differences in learning?

Turning to the sample of 47 students who completed the survey, the overall course letter grades of students who received individualized instructor feedback did not differ from students who were exposed to the answer keys (χ² = 2.753, df = 2, p = 0.252; see Figure 1). The “F” grade was collapsed into the “C” grade for the two-way chi-square test because the expected cell count for the “F” grade was <5. Students earning a letter grade of “C” in the course had to repeat it until they passed with an “A” or “B,” so there is little difference between these two grades. Similarly, an independent samples t-test found that the average number of points earned ( ${\bar{x}}_{i n s t r u c t o r} = 449.07$ , s_instructor = 42.66; ${\bar{x}}_{a n s w e r k e y} = 432.90$ , s_{answer key} = 49.50) was not significantly different between groups (t = 1.197, df = 45, p = 0.237).

Figure 1.

Final letter grades by students who received individualized instructor feedback and student self-assessment (n = 47).

Furthermore, there was no significant difference between the two groups on the total number of correct responses on the 30-item assessment (t = −1.339, df = 36.646, p = 0.189). In other words, students who received individualized instructor feedback scored approximately the same number of objective questions correctly ( $\bar{x} = 19.27$ , s = 5.31) as students who were exposed to answer keys only ( $\bar{x} = 21.08$ , s = 3.62). In fact, a comparison of each of the assessment items showed that the groups did not differ on all but three questions. First, a two-way chi-square test found a significant difference between the groups (χ² = 5.839, df = 1, p = 0.016) on a question assessing the definition of a variable: “A trait, characteristic, or attribute of a person, object, or event that can be measured at least at the nominal-scale level is called … (1) a variable; (2) a ratio; (3) an interval; (4) validity; (5) reliability.” Specifically, five students who received individualized instructor feedback missed this question, while all other students answered this question correctly. For this reason, chi-square results should be interpreted with caution: two cells (the “incorrect” responses) had an expected cell count <5; given there are only two response categories (“correct” and “incorrect”), collapsing was not an option. Similarly, students exposed to the answer key had more correct responses than students who received individualized instructor feedback (96% compared with 78%, respectively) on the item asking about the mean:

The mean is … (1) commonly known as the average; (2) an important and useful statistic; (3) what statisticians use to refer to the arithmetic mean as distinguished from other means such as the geographic mean, or the harmonic mean, which also are averages; (4) a measure of central tendency; (5) all of the above

a difference that approached significance (χ² = 3.257, df = 1, p = 0.071). Again, these chi-square results should be interpreted with caution as two cells had an expected count <5 (the expected count was 2.94). Specifically, five students who received individualized instructor feedback answered incorrectly, while only one student exposed to the answer key answered this question incorrectly. Finally, the concept of statistical significance seemed muddier for students receiving individualized instructor feedback (39% answered correctly) than for students who had the answer keys posted (71% answered correctly). The question asked, “Statistical significance is a phrase used to indicate that an observed relationship is … (1) not real; (2) not due to chance; (3) due to chance; (4) not important; (5) important”; differences between groups were statistically significant (χ² = 4.776, df = 1, p = 0.029). Taken together, these results find no support for the first hypothesis.

Does achievement matter?

While there were no differences in learning between students exposed to answer keys and those receiving personalized instructor feedback, there may be differences between students with higher achievement and lower achievement. A two-way ANOVA found no significant differences in the average number of correct responses on the end-of-semester 30-item assessment between the higher and lower achievement groups (F(1, 42) = 19.37, p = 0.171), nor was there a significant difference in the interaction between achievement group and whether students were exposed to the answer key (F(1, 42) = 1.002, p = 0.323; see Table 1). While the differences were not significant, both the high- and low-achieving students performed better, on average, when exposed to the answer key than when receiving personalized instructor feedback. The low-achieving group, especially, in the answer key group, scored closer to their high-achieving counterparts than those in the individualized feedback group, offering no support for the second hypothesis.

Table 1.

Mean score on 30-item assessment by exposure to answer key and by student achievement.

Feedback method	Achievement
	High (n = 21)	Low (n = 25)
Individualized instructor feedback (n = 22)	21.00	17.83
Answer key posted (n = 24)	21.36	20.85

Are there differences in student satisfaction?

While there were no differences in student learning, perhaps students preferred one method. To assess this, we now turn to comparing the end-of-course evaluations. The first question relevant here asked students about their perception of how much they learned: “Do you feel … that you better understand the subject matter of this course?” On a response option scale ranging from 1 to 4, students who received individualized instructor feedback rated their learning, on average, 3.46 (s = 0.78), while students exposed to the answer key rated their learning 3.15 (s = 0.86). This difference approached statistical significance (t = −1.870, df = 97, p < 0.10). Students who had individualized instructor feedback reported that they had more confidence in understanding the subject matter than students who were exposed to answer keys. However, while students receiving individualized instructor feedback rated instructor interest ( $\bar{x} = 3.63$ , s = 0.69) and overall course satisfaction ( $\bar{x} = 3.54$ , s = 0.80) higher than students exposed to answer keys ( $\bar{x} = 3.49$ , s = 0.80, $\bar{x} = 3.36$ , s = 0.87, respectively), students exposed to answer keys rated instructor preparation higher ( $\bar{x} = 3.70$ , s = 0.55) than students receiving individualized instructor feedback ( $\bar{x} = 3.67$ , s = 0.70). None of these differences were significant (t_interest = −0.951, t_course = −1.036, t_prepared = 0.228, all df = 97, all ps > 0.10), and the third hypothesis must be rejected.

Discussion

Online learning poses many challenges for students (and instructors alike!), but some of these challenges can be mitigated by engaging students, encouraging self-regulated learning through self-assessment. Here, we explored this by assessing student learning in an asynchronous online criminal justice statistics graduate course, comparing individualized instructor feedback and student self-assessment. Such research is scant, and this study contributed (see Huxham, 2007) to filling this gap in knowledge. We found no difference in student learning, measured by a 30-item assessment and by students’ final grades. Small differences emerged between the two feedback methods on only three questions. These non-significant findings may be due to the nature of the sample, consisting of graduate students. Graduate students may all be highly motivated to learn, regardless of feedback method. Separating students into those who are more and less motivated to learn, as evidenced by early achievement in the course may help counter this.

However, student achievement had no significant effect on this (lack of) relationship; as one would expect, high-achieving students performed better than low-achieving students in both contexts. While the difference was not significant, both high- and low-achieving students scored better on the 30-item assessment when exposed to the answer key and scored lower when receiving personalized feedback from the instructor. Additionally, no significant differences emerged in student perception of instructor interest, perception of instructor preparation, or satisfaction with the course, although students who received individualized instructor feedback were slightly more confident in their own learning than students responsible for self-assessment (p < 0.10). Recall that Huxham (2007) found that using model answers compared to instructor feedback improved student learning, but students preferred personalized feedback. While our results are non-significant, they are in line with Huxham’s (2007) findings in that “traditional” feedback is not necessarily the most conducive method for student learning. Hence, we conclude that instructors can save time on hand-grading graduate student introductory statistics assignments, refocusing efforts to engage more with individual student misconceptions and helping students, more generally, improve their pedagogic literacy to better benefit from self-regulated learning (see Price et al., 2010).

One concern with making assignment answer keys available for student use is integrity. Indeed, integrity is a concern in all classes. In this particular online statistics course, the instructor was not worried about someone copying an answer key and distributing it to other students. First, students may not know others in the program, communicating only online. That may be a deterrent to sharing answers. Second, the homework assignments are low-stakes; in other words, homework assignments are not worth a large portion of students’ final grades. Third, the answer keys are not downloadable files (at least not in this particular online statistics course), so students would have to copy a webpage to a Word document or some other file format. This would distort the tables and graphs, signaling to the instructor that students did not do their own work. So, evidence of academic dishonesty would be apparent in the homework assignments (unless students who cheated went through the extra effort to change the font and paraphrase the answers, in which case they still are learning—albeit not as much). Most important, all examinations are hand-graded by the instructor; that is, answer keys are not posted for high-stakes assignments like examinations. Accordingly, we suggest instructors may save time and redistribute their energy using answer keys instead of grading individual assignments.

There are some limitations present in this research. First, the sample size is small. Additionally, the sample is not random, so conclusions cannot be generalized beyond this group of students. While it may be an attractive option to administrators, we are in no way suggesting at this stage of our study or analysis that the relevancy of human instructors should be called into question. As previously mentioned, a very high percentage (75%) of students prefers “the human touch” and traditional instructional methods when learning (Johnson et al., 2009). Similarly, this does not take into account faculty members who also prefer the heightened, direct interaction with students, and provide more detailed feedback, even in online courses. That said, and at the risk of near-heresy, with many universities in the United States and elsewhere struggling to survive murky financial climates, it is of use to know when the more timely, individual assessments by instructors are not technically necessary or of measurably different benefit when compared to student self-assessments. As another limitation, readers may notice that the learning objectives focused on application while the objective assessment focused on definitions—a valid discrepancy. However, the objective assessment was preferred to ensure consistency of grading and remove the possibility of unconscious grading bias. That said, the other measures of student learning, final grades, are based on application and are consistent with the learning objectives; there was no difference in final grades between the two groups. A further limitation of the study is that we are working from only one specific course design. It would be beneficial to ask students who have had other online courses if the course design is similar to other online courses, how they would rate the course/instructional design compared to other online courses, and so on. The logic is that something as simple as changing the design of an online class could impact the assessment outcomes (i.e. more robust online courses might warrant less-detailed individual assessments, as they are already instructional resource heavy). We encourage replication in other online courses and other disciplines at both the undergraduate and graduate levels.

Krause et al. (2009) argue “The impact of feedback on learning depends not only on the kind of feedback provided, but also on how the learner deals with feedback information” (p. 160) and identify two factors affecting student learning: feedback design and feedback reception. Here, we tested only the former. An important question that arises out of this body of research is “how do we know students are actually looking at the feedback with either method?” In short, we do not, and the literature is ripe with queries on how to better engage students with feedback (e.g. Handley and Williams, 2011). But empirical evidence suggests that many students do review feedback (Carless, 2006). Furthermore, anecdotal evidence suggests they do in this particular course. Students post their questions on the discussion board asking why their answer for a certain homework response does not exactly match the answer key, similar to instructor-assessed students asking for clarification on instructor feedback. They also must know the information for all examinations, on which they must model their responses after the unit assignment answer keys. Considering that most students pass the course (and the bulk of the course points are concentrated in the examinations), we can assert that some are looking at the answer keys. Also, we would reasonably expect that the proportion of students who completely ignore feedback is approximately the same across classes, considering there were no significant differences in grades in this study population. Nevertheless, perhaps making students aware of the benefits of actively reviewing their assignments with the answer key would have increased their learning (see Black and Wiliam, 1998); this is a question for future research to explore.

Another question that arises is “how does this translate to face-to-face classes?” Instructors can distribute answer keys, post them on online course companions like Blackboard, or display them on an overhead during class. In other words, it is highly likely that some of the self-assessment techniques discussed would be easily transferrable and well suited to face-to-face instruction, as well; however, additional research needs to be conducted in this area, as it is well beyond the scope of this study and sample at this point.

Finally, we were unable to incorporate measures of statistics anxiety, math ability, and variations in student access to the Internet. These are important impediments to learning statistics or any other subject matter and may be possible confounds to these results. Indeed, higher instances of anxiety in courses with higher degrees of difficulty and those requiring the use of applied mathematics are well documented and supported by various studies (see, for example, Auster, 2000; Briggs et al., 2009; Chiesi and Primi, 2010; Davis, 2003; Leming, 1979; Macheski et al., 2008). That said, Van Gundy et al. (2006) documented the benefits of web-based instruction on reducing anxiety in statistics courses. Accordingly, we echo their advice of evaluating the mechanisms of online learning that reduces such anxiety and encourage attention on feedback methods as one way to reduce anxiety and build confidence. Posting answer keys may be one way to alleviate anxiety, as having more resources available to students may allow them to interact more with the content, feeling more comfortable (Larreamendy-Joerns et al., 2005). In this way, it is possible that the lack of differences in learning we found here may be balanced by these other factors rather than feedback method. Future research replicating this study should control for these potential learning impediments.

The answer to our research question “Is individualized instructor feedback or student self-assessment more conducive to student learning?” is there is no difference in the two methods of feedback—at least in this online graduate-level statistics course. Students exposed to both feedback methods had similar final grades and similar scores on an objective assessment. This null relationship remained when students were split into high- and low-achieving groups. Furthermore, students exposed to both feedback methods had similar evaluations of the course and the instructor. Future research may explore nuances affecting the influence of feedback on student learning.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author biographies

Jennifer C Gibbs is an Assistant Professor of criminal justice in the School of Public Affairs at Penn State Harrisburg, where she studies policing, terrorism, violence and victimization, and the scholarship of teaching and learning. Her dissertation, “The Relationship between Legitimacy, Terrorist Attacks and the Police,” completed at the University of Maryland, College Park, won the Homicide Research Working Group 2012 Richard Block Outstanding Dissertation Award. Address: Criminal Justice Program, School of Public Affairs, Penn State Harrisburg, 777 W. Harrisburg Pike, Middletown, PA 17057, USA. [email: jengibbs@psu.edu]

Jim D Taylor is an Associate Professor of Sociology and Criminology at Ohio University Zanesville. He works with dropout prevention programs and has written popular academic books on American gun culture, deviant behavior, and social problems. As a generalist, his research interests are varied and range from experimental research and abstract theory to teaching methods. He is currently completing an extended field study of masculine subcultures with a special focus on stigma management techniques. Address: Department of Sociology, Criminology and Criminal Justice, Ohio University Zanesville, 1425 Newark Road, Zanesville, Ohio 43701 USA. [email: taylorj2@ohio.edu]

References

Auster

(2000) Probability sampling and inferential statistics: An interactive exercise using M&M’s. Teaching Sociology 28(4): 379–85.

Black

Wiliam

(1998) Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice 5(1): 7–74.

Boud

Lawson

Thompson

(2013) Does student engagement in self-assessment calibrate their judgment over time? Assessment & Evaluation in Higher Education 38(8): 941–56.

Briggs

Brown

Gardner

. (2009) D.RA.MA: An extended conceptualization of student anxiety in criminal justice research methods courses. Journal of Criminal Justice Education 20(3): 217–26.

Carless

(2006) Differing perceptions in the feedback process. Studies in Higher Education 31(2): 219–33.

Chiesi

Primi

(2010) Cognitive and non-cognitive factors related to students’ statistics achievement. Statistics Education Research Journal 9(1): 6–26.

Cook

Dupras

Thompson

. (2005) Web-based learning in residents’ continuity clinics: A randomized, controlled trial. Academic Medicine 80(1): 90–7.

Davis

(2003) Statistics anxiety among female African American graduate-level social work students. Journal of Teaching in Social Work 23(3–4): 143–58.

Evans

(2013) Making sense of assessment feedback in higher education. Review of Educational Research 83(1): 70–120.

10.

Garfield

Gal

(1999) Assessment and statistics education: Current challenges and directions. International Statistical Review 67(1): 1–12.

11.

Handley

Williams

(2011) From copying to learning: Using exemplars to engage students with assessment criteria and feedback. Assessment & Evaluation in Higher Education 36(1): 95–108.

12.

Hattie

Jaeger

(1998) Assessment and classroom learning: A deductive approach. Assessment in Education: Principles, Policy & Practice 5(1): 111–22.

13.

Huxham

(2007) Fast and effective feedback: Are model answers the answer? Assessment & Evaluation in Higher Education 32(6): 601–11.

14.

Johnson

Dasgupta

Zhang

. (2009) Internet approach versus lecture and lab-based approach for teaching an introductory statistical methods course: Students’ opinions. Teaching Statistics 31(1): 21–6.

15.

Krause

U-M

Stark

Mandl

(2009) The effects of cooperative learning and feedback on e-learning in statistics. Learning and Instruction 19(2): 158–70.

16.

Kreiner

(2006) A mastery-based approach to teaching statistics online. International Journal of Instructional Media 33(1): 73–80.

17.

Larreamendy-Joerns

Leinhardt

Corredor

(2005) Six online statistics courses: Examination and review. The American Statistician 59(3): 240–51.

18.

Leming

(1979) Research methods: The first class. Teaching Sociology 6(2): 133–7.

19.

Macheski

Lowney

Buhrmann

. (2008) Overcoming student disengagement and anxiety in theory, methods, and statistics courses by building a community of learners. Teaching Sociology 36(1): 42–8.

20.

Magel

(1996) Increasing student participation in large introductory statistics classes. The American Statistician 50(1): 51–6.

21.

Nesbit

Burton

(2006) Student justice perceptions following assignment feedback. Assessment & Evaluation in Higher Education 31(6): 655–70.

22.

Nicol

Macfarlane-Dick

(2006) Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education 31(2): 199–218.

23.

Price

Handley

Millar

. (2010) Feedback: All that effort, but what is the effect? Assessment & Evaluation in Higher Education 35(3): 277–89.

24.

Sadler

(1989) Formative assessment and the design of instructional systems. Instructional Science 18(2): 119–44.

25.

Sadler

(1998) Formative assessment: Revisiting the territory. Assessment in Education 5(1): 77–84.

26.

Sadler

(2010) Beyond feedback: Developing student capability in complex appraisal. Assessment & Evaluation in Higher Education 35(5): 535–50.

27.

Van Gundy

Morton

Liu

. (2006) Effects of a web-based instruction on math anxiety, the sense of mastery, and global self-esteem: A quasi-experimental study of undergraduate statistics students. Teaching Sociology 34(4): 370–88.

28.

Yalcin

Kaw

(2011) Do homework grading policies affect student learning? International Journal of Engineering Education 27(6): 1333–42.