Abstract
Background
The COVID-19 pandemic made it difficult to proctor exams after the forced transition to remote teaching and learning.
Objective
We evaluated students’ experiences creating and answering their own exam items in an upper-level psychology course during the transition to remote teaching and learning during the COVID-19 pandemic.
Method
Students in an advanced social psychology course wrote and answered exam items to demonstrate their learning when proctored exams became impossible during the COVID-19 pandemic. The exams were graded for breadth, depth, and accuracy, and compared to other demonstrations of learning, including traditional in-class exams taken prior to the transition to remote teaching and learning.
Results
Students performed well on these exams. Students reported positive perceptions of these exams during the COVID-19 pandemic, but reported that, while these exams reduced test anxiety, they were time-intensive.
Conclusion
Students can demonstrate their learning by writing and completing their own exams. Their higher grades may be related to reduced test anxiety and greater time investments.
Teaching Implications
Instructors should consider having students write and complete their own exams to demonstrate their learning, especially when proctoring exams is difficult or impossible.
One of the more ubiquitous changes to education during the COVID-19 pandemic was the rapid transition of face-to-face (F2F) classes to remote teaching and learning modalities. Many instructors made revisions, often with little training or notice, to their classes so they could continue to teach in reasonable ways (Johnson et al., 2020; Silva de Souza et al., 2020; Sunasee, 2020). Immediately after our university’s decision to move all F2F classes to remote teaching and learning modalities, we reissued our syllabus with an empathy addendum to let students know we understood things were different and challenging, to encourage connection and communication, and to remind them we want them to succeed. As evidence of this, we also shifted the values and parameters of our remaining assignments to be more realistic and appropriate for our students, namely by eliminating some assignments, substituting others, and lowering the stakes of other point opportunities (e.g., grading for completion). But assessing students’ learning via online exams posed one of the greatest logistic challenges for instructors (Clark et al., 2020).
At many universities, including our own, there was no possibility of assessing learning in traditional F2F ways during the COVID-19 pandemic (i.e., exams completed independently in class, without collaboration or resources, while instructors or teaching assistants proctor the exams against cheating and potentially answer questions). Even remote exams had additional concerns to consider, like limited access to technology (Nambiar, 2020), expensive online proctoring services (e.g., Cluskey et al., 2011), academic integrity (Guangul et al., 2020; Moralista & Oducado, 2020), and exacerbated test anxiety that is already commonly experienced under much more normal conditions (Zeidner, 2010). However, less traditional learning assessment methods, like involving students more in exam development, may resolve these issues.
Some ways to involve students in exam development include having students create a question bank for exams that the instructor chooses from (e.g., Ahn & Class, 2011; Jobs et al., 2013) or having students write their own exams. As an example of the latter, Corrigan and Craciun (2013) had students in three undergraduate courses write and answer multiple choice and short essay questions in an open-book, take-home format, limiting the instructor’s role to specifying assignment parameters (e.g., required number of questions). Corrigan and Craciun (2013) argued this approach minimizes academic dishonesty, and they found students reported greater involvement in their learning and less test anxiety compared to instructor-written exams. Accordingly, after the pivot to remote teaching and learning and our one F2F exam (i.e., “Exam 1 In Class”), we replaced our remaining exams with “Exams by You” (i.e., “Exam 2 by You”, “Exam 3 by You”), in our Advanced Social Psychology course. Beyond a traditional essay exam in which instructors provide set topics or questions, Exams by You are consistent with Corrigan and Craciun’s (2013) student-written exam approach in which students wrote and answered their own short essay questions.
We believed Exams by You would be a good alternative to F2F or traditional online exams during the COVID-19 pandemic because they may reduce test anxiety (e.g., Corrigan & Craciun, 2013); the potential for academic dishonesty both by making it more easily identifiable given the uniqueness of each student’s question and answer and allowing students to use their resources (e.g., notes, readings, lectures); and the need to access technology during specified time windows. Given that essay exams are a historically valid method of learning assessment, Exams by You allowed for further creativity and flexibility for both students and instructors during the pandemic. This led us to the following research questions: (a) what were students’ general experiences while writing their own exams? (b) how did these experiences relate to exam grades, if at all? and (c) what was students’ feedback for instructors about writing their own exams?
Method
Participants
Our class included 28 students in an upper-level psychology course at a large Midwestern research intensive university. After our F2F exam (i.e., Exam 1 In Class), we offered extra credit for students who completed surveys after both Exam by You opportunities. All participants were psychology majors. The average age was 21 years (SD = 1.20) for Exam 2 by You survey respondents (N = 19) and 21.13 (SD = 1.20) for Exam 3 by You survey respondents (N = 16). In both surveys, the majority were White (n Exam2 = 17, 89%; n Exam3 = 15, 94%) women (n Exam2 = 17, 89%; n Exam3 = 13, 81%) who were either juniors (n Exam2 = 7, 37%; n Exam3 = 7, 47%) or seniors (n Exam2 = 9, 47%; n Exam3 = 7, 47%). In total, we had 35 survey respondents: 22 were unique respondents (i.e., they completed at least one of the optional follow-up surveys; M age = 20.91, SD = 1.15; White: n = 19, 86%; women: n = 19, 86%; juniors: n = 10, 45%; and seniors: n = 9, 41%) and 13 completed both surveys after each Exam by You (M age = 21.31, SD = 1.25; all White, women: n = 11, 85%; and seniors: n = 7, 54%).
Measures and Materials
Study materials are available in Open Science Framework (OSF; see Saucier et al., 2022). This includes the Exams by You guidelines and rubric as well as the free-response survey items we created.
Exam 1 In Class
Throughout the course of the Spring 2020 semester, students completed three exams. Exam 1 In Class refers to the first and only exam administered before the shift to remote teaching and learning. Consistent with traditional F2F exams, it included four sections: 10 multiple-choice items; 10 matching items; 10 short answer items answered with a sentence or phrase; and five longer answer items answered with a couple sentences, list, or paragraph.
Exams by You
On a 1 (Strongly Disagree) to 9 (Strongly Agree) Likert-type scale, students responded to 10 face-valid items we created regarding their general attitudes toward writing their own exams (e.g., I was happy to write and take my own “Exam by You”), comparisons to standard testing measures (e.g., I preferred writing my own exam over taking an exam written by my instructor), and perceptions of this exam format’s validity (e.g., I feel as though this is an accurate and valid way to assess my learning).
Principal Components Analysis on Exam by You Survey Items.
Note. Factor 1 was named Preference, Factor 2 was named Valid, and Factor 3 was named Successful. ** denotes a reverse-scored item for the composite.
We also created nine free-response items regarding their Exams by You experience (e.g., What was the best part about doing the “Exam by You”? What was the worst part about doing the “Exam by You”? What would you recommend to other students who will be writing “Exams by You” in the future? What is your biggest complaint about the “Exam by You”? see Saucier et al., 2022). The first and second author independently coded these responses and agreed on classifications.
Design and Procedure
This research used a within-subjects design in that after completing each of the two Exams by You, we invited students to take extra credit surveys to provide their feedback and opinions on this exam format. Participants provided informed consent, their unique school ID number to match survey responses to exam grades, basic demographic information, and then completed the survey items. Upon completion, participants were granted extra credit. This process repeated after each Exam by You. When appropriate, we examined students’ grades in relation to their survey responses.
Results and Discussion
General Experiences Completing Exams by You
The Exam 2 by You survey was students’ first opportunity to provide feedback on the revised exam structure. Survey respondents were somewhat neutral in their preference for writing their own exams (M = 5.83, SD = 1.60), but perceived the exam as both a valid measure of their learning (M = 7.09, SD = 1.01) and a successful alternative during the pandemic (M = 7.32, SD = 0.96). Exam 3 by You survey respondents tended to enjoy writing their own exams (M = 6.16, SD = 1.94) and also felt the exam was both a valid (M = 7.02, SD = 1.64) and successful (M = 7.25, SD = 1.46) measure of their learning. Among students who completed both post-Exam by You surveys, there were no significant differences between Exams 2 and 3 by You in terms of Preference, t = 0.86, p = .406, d = .21; Valid, t = 0.30, p = .771, d = .09; or Successful, t < 0.01, p = 1.00, d = .00.
Exams by You Grades and Experiences
Pairwise Correlations for Exam by You Grades and Rubric Components.
Note. Correlations below the diagonal reflect Exam 2 by You whereas correlations above the diagonal reflect Exam 3 by You. Bolded correlations signify p < .05.
Pairwise Correlations Among Exam by You Grades and Survey Composites and Items.
Note. Correlations below the diagonal reflect Exam 2 by You grade and survey responses whereas correlations above the diagonal reflect Exam 3 by You grade and survey responses. The item for 2d was reverse-scored in the Preference composite, but not for the purposes of this correlation table. Bolded correlations signify p < .05.
Students’ Feedback on Exams by You
Exam 2 by You Qualitative Survey Responses (N = 19).
Note. One respondent did not respond to the last question about their biggest complaint.
Exam 3 by You Qualitative Survey Responses (N = 15).
Note. Although 16 students completed this survey, only 15 responded to these qualitative questions.
Conclusion
Limitations and Future Directions
This approach is not without its limitations, most notably Exams by You may be more labor-intensive for students and graders, at least at first. Another limitation is that our grading may have been too lenient, creating a ceiling effect in our analyses; however, based on common feedback, students worked for their grade. Although it is possible students avoided the most difficult material, they would have failed to earn “breadth” points outlined in our rubric. Another possibility is that student-written exams simply tap into a different kind of learning that more closely resembles essay exams (e.g., Bizzell & Singleton, 1988). In any case, grading Exams by You may be less tedious because of the variability and creativity in students’ exams, as Corrigan and Craciun (2013) suggest. Finally, this approach may be too time-consuming for graders in larger classes (Corrigan & Craciun, 2013). However, this approach could easily be adapted to overcome these limitations. For instance, more specific guidelines (e.g., one question on [altruism]) would likely benefit both students and graders. Also, it may be beneficial to consider using a hybrid option for future testing, mixing who writes the question (i.e., instructor or student) and the type of question (i.e., multiple choice, short-answer, longer-answer). In this case, one could also specify student-written questions cover specific topics to make sure that content is represented on the exam and to make the items more predictable for graders. While we have addressed many limitations to the Exam by You format, there are likely others that exist. Most notably, it is difficult to disentangle the potentially confounded relationship between the adoption of this exam format (or any other COVID-19-inspired pedological technique) and the emergence of the COVID-19 pandemic. Nonetheless, our goal was to offer a method of assessment during and, perhaps beyond, the pandemic for instructors to incorporate into their classes at their discretion.
Student-written exams present many benefits for both students and instructors. Most notably, based on students’ feedback, this approach reduced test anxiety that can be experienced with standard testing. Additionally, students took these exams seriously and seemed to enjoy the opportunity to demonstrate their learning in our Exams by You format because it encouraged student creativity and exploration of topics that interested students the most. Moreover, given the increased concern about maintaining academic integrity with the pivot to remote learning (Guangul et al., 2020; Moralista & Oducado, 2020), the Exam by You approach would likely make academic dishonesty more easily identifiable given that every question and answer are idiosyncratic to each student. The flexibility of having students write their own exams can apply to a variety of disciplines and be easily adapted to fit a given course’s objectives and design. Overall, the Exam by You format presented a wonderful alternative to standard F2F testing when teaching and learning remotely that worked to address issues related to student anxiety and possibilities of academic dishonesty. As a result, Exams by You should be incorporated into course design even after the COVID-19 pandemic to provide a flexible, empathetic approach to testing for students and instructors.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
