Abstract
We examined how writing cover letters to the instructor influenced final papers in research methods courses. After receiving instructor feedback on drafts of each section of an American Psychological Association style research paper throughout the semester, students in two classes wrote cover letters to the instructor explaining how the instructor feedback was incorporated into their final paper. Students in the two control classes received the same type of feedback throughout the semester but were not required to write cover letters. Students enrolled in the cover letter classes showed more improvement in the quality of their papers than the students in the control classes. Using cover letters to articulate how feedback is being incorporated may help students more efficiently organize and respond to instructor feedback. Future researchers might investigate whether students’ reflective ability is a significant moderator in effective feedback utilization.
According to the American Psychological Association (APA) guidelines for undergraduate psychology majors, one goal is for students to be able to communicate more like scientists (APA, 2007). This includes being able to adhere to the scientific writing conventions of APA style that are often introduced in a research methods course (Perlman & McCann, 1999). In these courses, it is typical for students to design, implement, and write about an empirical study or to write a research proposal. It is also common for instructors to have students write drafts of each major section of this article throughout the semester (Halpern, 2010). Such practices afford instructors opportunities to provide constructive feedback on each section and for students to incorporate revisions into an improved final paper.
When constructing a paper that involves multiple revisions, the writing process includes planning and organizing a paper, expressing thoughts in understandable terms, and evaluating one’s work and revising based on those evaluations. The idea behind feedback is that it should make the revision process more strategic and ultimately improve the final paper (Flower & Hayes, 1981). In fact, several authors have concluded that feedback does indeed improve writing quality and the final paper. For example, Bell (2002) found that students who received feedback from a writing center tended to write papers consistent with course objectives. Likewise, practice writing and peer and instructor feedback (Fallahi, Wood, Austad, & Fallahi, 2006; Johnstone, Ashbaugh, & Warfield, 2002) and designing specific courses intended to address writing conventions that include a substantial amount of feedback (e.g., Goddard, 2003; Johnson, Tuskenis, Howell, & Jaroszewski, 2011) tend to improve papers.
However, not all found feedback beneficial. For example, Covill (2010) had students undergo a peer-review process, self-review process, or no review process (control) for a large paper assignment. She found no evidence for a difference in the quality of writing as a function of condition. More interestingly, the no-review group made more revisions from draft to final paper and had more positive attitudes about the paper than the peer-review and self-review groups. Although Stellmack, Keenan, Sandidge, Sippl, and Konheim-Kalkstein (2012) found that over half of students’ research method papers displayed increases in quality from first to second draft when graded by a third party not associated with the course, 40% of students showed no improved quality or reduced quality from first to second draft. Although some students are effectively incorporating the constructive feedback, there seem to be individual differences among students in their ability to utilize feedback.
One potential reason for the lack of feedback effectiveness in Covill (2010) and individual differences in incorporating feedback in Stellmack et al. (2012) is that some students may lack an organizational system that allows them to process and reflect on the feedback provided. In fact, reflection on feedback is a strong predictor of actual improvement in writing quality. Duijnhouwer, Prins, and Stokking (2012) created a reflection assignment that required students to answer several questions about some of the instructor feedback they received. These questions were related to whether students found the feedback helpful and whether they planned to incorporate some or all of the feedback. In addition, the assignment required students to explain their rationale. They found students who engaged in this process of reflection tended to improve more on the final draft. Furthermore, students who reported more reflection on the assignment tended to show an attitude more consistent with mastery of a skill (e.g., students’ goals were to learn instead of earn a good grade) than those who reported fewer reflections. This effect was even more pronounced when students’ initially had an attitude less consistent with mastery of a skill.
Another method that may support the reflection process in writing is assigning a cover letter that addressed how feedback was incorporated. In academia, when writing a scientific paper for peer review, authors are often required to write a cover letter addressing changes made to the article based on feedback from the reviewers. This cover letter serves many purposes. First, it requires authors to actively process the constructive feedback (i.e., reflection). Second, it requires authors to strategically decide how to implement feedback (i.e., refinement). Finally, it clearly communicates to the reviewer how the paper has been revised. Thus, one method that might help students organize and reflect on feedback is to write a cover letter to the instructor which addresses how instructor-provided feedback was implemented on all sections into the final paper. To determine whether writing a cover letter articulating how feedback was incorporated influences the quality of a final paper, students in two research methods courses wrote cover letters addressing how feedback was incorporated into the final paper assignment. Students in two research methods courses not requiring a cover letter served as a control. Students in all courses received bulleted feedback on drafts of all major sections and were expected to incorporate the draft sections into an APA style final research paper.
Method
Participants
Sixty-eight students enrolled in psychology research methods courses at two midsize universities in the Midwest participated in this study. Although 84 students were enrolled in these classes, 6 students failed to sign the informed consent and 9 failed to complete all draft assignments. One additional student was ineligible because she repeated the course. Thirty-six students participated in the control classes and 32 students participated in the cover letter classes (43 students at Campus 1 and 25 students at Campus 2).
Materials
Paper assignment and feedback
All students wrote an APA style research paper based on a research project or proposal in the research methods courses. All papers were required to adhere to APA style structure, content, and writing conventions. Students completed a single draft of each major section of a scientific paper (i.e., Introduction, Method, Results, and Discussion) at different points during the semester. For each draft, the instructor provided a single round of feedback to the students. At the end of the semester, students submitted a final paper in which they were expected to incorporate changes to their drafts based on instructor feedback to produce a coherent APA style research paper.
Across universities, drafts and the final paper were graded based on overall quality using the five major areas described subsequently: adherence to APA style: included all major sections of a scientific paper, used correct headings and formatting conventions, and correctly cited appropriate literature; inclusion and understanding of appropriate background literature: described an appropriate number of refereed journal articles, included relevant information, and excluded superfluous details; quality of research design: described a well-planned and practical design and hypotheses were well stated and matched design; grammar and formal writing: achieved subject–verb agreement, used the correct tense, used adequate sentence structure, avoided run-on sentences or wordiness, and refrained from colloquial language; and organization: logical flow and transitions throughout the paper.
Students received feedback on these five areas in a bullet point format on the draft of each section. Feedback explicitly noted problems including the rationale and/or suggestions for changes.
Cover letter manipulation
Students from one class at each campus wrote cover letters that explained how they incorporated the instructor feedback into their final paper. The instructor provided instructions on how to construct a cover letter as well as examples of good and bad cover letters. Students were instructed to write a short paragraph that discussed the organization of the cover letter. Then, students were instructed to synthesize instructor feedback into main points and provide detailed explanations of how instructor feedback was incorporated. The example of a good cover letter followed the instructions such that an organizational paragraph appeared at the beginning of the cover letter, main feedback points were synthesized, and detailed and thoughtful explanations appeared below each main point (i.e., explanations walked the instructor through the student’s thought process while revising). In contrast, the example of the bad cover letter contained a superficial synthesis of main points and explanations of how feedback was incorporated were shallow and/or generic (e.g., “I fixed my grammar”).
Procedure
All students received instruction throughout the research methods course on how to write a scientific paper. That is, instructors devoted some lectures to the construction of each section of a scientific paper. Lectures and sample articles covered what should be included and excluded in each of these sections, the purpose of each of these sections, and specifics of APA style. Drafts of each section of the paper were due throughout the semester and returned with instructor feedback on how to improve the section based on the criteria outlined previously. At the end the semester, students submitted the final paper that combined all corrected drafts. In the cover letter courses, students wrote a cover letter outlining how they incorporated the instructor feedback for a separate grade.
Data Organization
Draft score and final paper score
To standardize scores across universities, a percentage score was created for combined drafts and a percentage score was created for the final paper. We summed the points received across all drafts, then divided that number by the total number of possible points for all drafts, and then multiplied by 100 to create a percentage score across drafts. The number of points received on the final paper was divided by the total number of possible points and then multiplied by 100 to create a percentage score for the final paper.
As a quality check of the instructors’ grading, each final paper was graded on a 10-point scale by the instructor from the alternate campus. The alternate instructor was blind to the feedback students received and paper grades. Scores were significantly correlated (all rs > .60; all ps < .001). Thus, there appeared to be agreement among the instructors.
Results
Table 1 shows the means and standard deviations for drafts and final papers as a function of intervention and university. To determine whether the cover letter intervention and university influenced the quality of the final paper, a 2 (intervention: cover letter, control) × 2 (university: Campus 1, Campus 2) between-groups analysis of covariance was computed with final paper percentage score as the dependent measure and draft percentage score as a covariate (to control for individual differences in writing ability). There was a main effect of intervention, F(1, 62) = 10.02, p < .005, η2 = .03, and university, F(1, 62) = 47.75, p < .001, η2 =.14. Students enrolled in cover letter classes tended to have significantly higher improvement scores than students in control classes and Campus 2 tended to have significantly higher improvement scores than Campus 1, respectively. The interaction was not statistically significant, F (1, 62) = 0.94, p = .34, η2 = .003.
Mean Draft Percentage Score and Final Paper Percentage Score by University and Intervention.
Discussion
Students who wrote cover letters articulating how they incorporated feedback from their drafts tended to have higher improvement scores than students who did not write the cover letters. This finding suggests that the cover letters facilitated improvement on the final papers. It is possible that writing a cover letter helps students organize and reflect on changes they should make to their papers, which ultimately helped them utilize feedback. Wade (1995) proposed that writing assignments are a central component for critical thinking instruction because, unlike other classroom activities (e.g., discussions), writing allows students to reflect, revise, and refine ideas. That is, writing allows students to “try again.” However, for these writing benefits to occur, students must actively participate in this revision cycle.
Participating in the reflection aspect of the writing cycle may be the most beneficial for student learning but may often be ignored. Sperling and Freedman (1987) suggested that there is a disconnect between an instructor’s intended outcome and student’s understanding of the intended outcome, which may result in little to no revisions. The cover letter manipulation may reduce disconnection by serving as a catalyst to this cycle by helping students process feedback to make appropriate revisions and refinements. In addition, it is possible that the students who showed no improvement in quality or produced worse quality papers in Stellmack et al. (2012) may have failed to actively process the instructor feedback. Ultimately, interventions that promote active processing and reflective thinking, like the cover letter assignment in the current research and the reflection assignment in Duijnhouwer et al. (2012), may be necessary for courses that require a substantial amount of writing.
The result that cover letters improved quality of final papers is also consistent with prior publications wherein improvements in writing across drafts were due to a variety of interventions (Fallahi et al., 2006; Goddard, 2003; Johnson et al., 2011). These prior authors showed that feedback tends to help students improve their writing. Our study uniquely demonstrates that requiring students to articulate how feedback was used tends to support feedback “uptake” more than providing feedback alone.
The finding that one campus tended to have higher improvement scores than the other campus was unexpected but not surprising. Because we were unable to use random assignment for this between-group variable, it is likely that the natural differences in how instructors grade assignments contributed to this effect. For example, the instructor at Campus 1 appeared to grade drafts more leniently than the final paper. Although, the cover letter class scores seem to remain stable, this actually reflects students incorporating feedback based on the instructor’s grading style. As more evidence of this instructor’s style, in a prior semester in which this instructor taught research methods and did not use the cover letter manipulation, grades were consistent with those reported in the control class (i.e., draft M = 87.8, SD = 10.3; final M = 82.7, SD = 10.7). To minimize these possible differences, future researchers should have a third-party grader score drafts in addition to the final papers. What is important, however, is that the students in cover letter classes tended to have higher improvement scores than students in control classes and this effect did not change as a function of university (i.e., interaction failed to reach statistical significance). In other words, at both universities, papers written by students in cover letter classes improved more than papers written by students in control classes.
It is important to note that Stellmack et al. (2012) found a discrepancy in improvement in student papers when the same papers were graded by the instructor (80% improvement) and a third-party grader (50% improvement). Thus, they suggested that feedback does not improve writing per se but rather improves the ability to write for a particular instructor. Although it is possible that these instructor biases occurred with the current research, they would have been present in both conditions and should be no more likely to occur in one condition versus the other.
It is also important to note the small sample sizes used in each condition across universities. Thus, these results should be interpreted with caution as the probability of inadvertently sampling outliers is higher than if the sample sizes were larger. Future research should replicate these findings. With that being said, it is promising that both universities showed the same effect.
Although we controlled for individual differences in writing ability by using draft percentage score as a covariate, it is possible that some students may be better equipped to utilize feedback. These individual differences may account for some of the differences found in this study. Future research could investigate how/whether students’ reflective skills, for example, see Frederick (2005) for a test to measure cognitive reflection, are related to effective feedback utilization.
Nonetheless, using cover letters to promote the incorporation of feedback into final papers is a relatively easy addition to any course that requires writing. Given the amount of instructor work and time that is involved in providing feedback, this intervention provides a promising technique that appears to promote processing of instructor feedback. This may ultimately make instructor grading of the final paper easier and quicker. Importantly, grading the cover letters does not add an extensive amount of grading time but might make students more accountable for making the appropriate changes in their final paper. This intervention may help students adhere to professional writing conventions, which is one of the goals of the APA guidelines, and may give students practice with the professional convention of writing cover letters. By providing students a framework through which to organize and utilize feedback, it is our hope that students would become more familiar with the revision cycle and ultimately be able to reflect on and revise their writing independently in the future.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Frances Daniel received a summer faculty fellowship from her institution. This fellowship is given to junior faculty and allows them to work on research in the summer instead of teaching for financial support.