Abstract
Office hours provide time outside of class for students to consult with instructors about course material, progress, and evaluation. Yet office hours, at times, remain an untapped source of academic support. The current study examined whether office hour attendance in combination with a learning reflection would help students learn material in an introductory statistics course. Psychology students were randomly assigned to meet in person with the course instructor and complete a learning reflection form prior to a test. Students who completed these tasks prior to the second test in the course outperformed other students. Office hour attendance combined with reflection may be a useful way to improve students’ understanding of statistics.
Statistics training is essential in the field of psychology. Learning outcomes recommended for psychology majors by the American Psychological Association (2007) include outcomes centered on understanding statistics. Statistics education forms an important part of psychology degrees and considerable work has been undertaken to explore how to effectively teach statistics. Conners, Mccown, and Roskos-Ewoldsen (1998) suggested instructors of statistics face four key challenges when teaching psychology students: performance extremes, student anxiety, attitudes that the material is uninteresting, and making the learning memorable for students. Whereas students likely find other psychology courses to be of intrinsic interest, this is not always the case with statistics requirements. Students may hold negative attitudes toward statistics and be less motivated to learn statistical material (Evans, 2007; Gal & Ginsburg, 1994; Onwuegbuzie, 2004).
Given these challenges, considerable research has focused on exploring the effectiveness of various pedagogical techniques (e.g., using humor, interesting demonstrations, active learning exercises) to improve the learning of statistics (Christopher & Marek, 2009; Lesser & Pearl, 2008; Neumann, Hood, & Neumann, 2009; Segrist & Pawlow, 2007; Zieffler et al., 2008). Instructors have also considered how to create effective examples and writing assignments to engage students in statistics (Chew, 2007; Schmidt & Dunn, 2007). One yet unexplored possibility is that the interaction between an instructor and student during office hours may also be a powerful pedagogical tool to help students succeed in learning statistics. The literature on office hours has not addressed this question, and on the whole pedagogical research on the learning outcomes of office hour attendance is limited (for two exceptions, see Limberg, 2007 and Skyrme, 2010). Office hour meetings provide additional time for student–faculty contact, which is one of the seven key principles proposed to enhance undergraduate education (Chickering & Gamson, 1999). Such meetings provide time for rapport building and students appreciate efforts made by professors to build rapport (Catt, Miller, & Schallenkamp, 2007; Starcher, 2011). Importantly, instructor–student rapport has been positively linked to both affective and cognitive learning outcomes (Frisby & Martin, 2010). Recently a scale has been developed to measure professor–student rapport given the relationship between this construct and key outcomes such as students’ attitudes toward a course and instructor as well as motivation (Wilson, Ryan, &Pugh, 2010). A shortened version of the scale has been found to be predictive of actual student grades (Wilson & Ryan, 2013).
In addition to attending an office hour meeting, students completed a learning reflection form following the meeting. Previous work has supported the claim that learning reflections have a beneficial effect on student learning. For example, Conrad (2013) found that students who completed reflection papers in a memory course outperformed peers not required to complete such reflections. Fleming (2002) introduced students to learning strategies through the completion of goal and activity forms and found that students completing these forms performed better on three exams compared to a control group. Incorporating reflection activities within a course can encourage students to assess their performance and study habits and to make appropriate changes (Sweidel, 1996).
It was thought that combining these elements (i.e., office hour attendance and a learning reflection) into a learning check-in would benefit student learning. Statistics courses require students to build their understanding of topics throughout the term. That is, understanding material from early in the term is especially important to understanding concepts presented later in the term (e.g., understanding central tendency and variability is important to then understanding hypothesis testing). As such, asking students to reflect on their learning and to meet with a professor about concepts that remain unclear or challenging may have a positive influence on student learning. The purpose of the current study was to determine if a structured office hour meeting followed by reflection on the part of the student would positively influence learning as measured by test performance. This intervention is referred to as a learning check-in.
Method
Participants
Students across two sections of an introductory statistics course for psychology majors were given the opportunity to participate in the study. Seventy-five percent of the students volunteered to participate (33 women, 8 men; M age = 20.43, SD age = 1.16). The study was conducted at a midsized undergraduate university in Western Canada.
Procedure
Students completed three tests and before one of the tests students completed a learning check-in. The learning check-in required sending two questions by e-mail to the professor 12 hr before an in office meeting. Meetings lasted between 20 and 30 min, which provided time to discuss the student’s progress and specific questions about the material. Following the meeting, each student completed a learning reflection form (see Appendix) that required students to (1) assess their learning in the course to date; (2) consider behaviors that help and interfere with their learning; (3) identify three behaviors to adopt, change, continue, or stop to succeed in the course; and (4) develop a study plan for the upcoming test.
To control for the potential of observer bias by the instructor, a research assistant randomly assigned participants to the groups. This ensured that the instructor would not know who was in each group until the learning check-ins began, which occurred up to 10 days before tests. This left little time for expectancy effects to potentially develop and influence participant behavior. Additionally, 65% of the students who completed the first learning check-in did so a week or less before Test 2, which resulted in minimal contact between the instructor and students before the test and little opportunity for the development or operation of expectancy effects. Furthermore, although the instructor knew which students completed the learning check-in prior to the tests she remained unaware of which particular students agreed to participate in the study. Lastly, the tests were graded with student names covered to ensure the instructor remained blind to the condition a test originated from.
Performance on Test 1 was used to check the equivalency of the groups. It was anticipated that participants completing the learning check-in prior to Test 2 would perform better than those not completing it. Performance differences on Test 3 were speculative due to the possibility of carryover effects, but it was important that all students received the learning check-in over the course of the term.
Results
Prior to Test 1, no student completed the learning check-in, which allowed for a check of group equivalency created through random assignment. 1 No significant differences were observed between Group 1 (n = 20), students receiving the learning check-in first (M = 75.93, SD = 11.22, 95% confidence interval, CI [70.67, 81.18]), and Group 2 (n = 20), those receiving it second (M = 72.43, SD = 9.03, 95% CI [68.20, 76.65]), in terms of their performance on Test 1, t(38) = 1.09, p = .28.
A 2 × 2 (Condition [learning check-in, control] × Test [Test 2, 3]) split plot factorial analysis of covariance, controlling for performance differences on Test 1 (p < .001), revealed a significant main effect of the learning check-in, F(1, 37) = 5.34, p = .03, η p 2 = .13. Participants who completed the learning check-in prior to Test 2 (M = 73.82, 95% CI [70.07, 77.56]) outperformed those who did not complete the learning check-in at that time (M = 66.76, 95% CI [63.02, 70.51], p = .01). This pattern continued at Test 3, although was no longer significant (p = .13), possibly indicating a carryover effect (Ms = 60.63 vs. 52.95, 95% CI [53.67, 67.59] and [45.99, 59.92], respectively; see Figure 1). The main effect of test and the interaction effect were not significant (ps > .05).
Adjusted mean performance on tests as a function of when students completed the learning check-in. Error bars represent standard errors.
Discussion
If the learning check-in had a positive effect on student learning, it was expected that students who had received the learning check-in before Test 2 would outperform students who had not completed the learning check-in. This indeed was the case. While controlling for initial differences between the groups, students who completed the learning check-in performed better on the test, observed with a medium effect size. A performance gap remained at Test 3, which presents the possibility of a carryover effect.
Based on the results of this study, it appears that a planned meeting with a professor followed by time spent assessing one’s progress in a statistics course does improve student learning. Skyrme (2010) has noted that such interactions can improve understanding of course material while providing students with a feeling of personal recognition and engagement. Lavooy and Newlin (2008) have reported a positive correlation between online office hour attendance and course performance measures. Further research on the learning outcomes produced by office hours and learning reflections is warranted.
The learning check-in was comprised of three components: e-mailing the professor two questions prior to meeting, meeting for 20–30 min to discuss questions and progress in the course, and submitting a learning reflection form. Although it is quite possible the office meeting provided reflection time for students, the meeting was primarily driven to address the questions students submitted. Considerable research has indicated positive learning outcomes for students asked to generate questions (Berry & Chew, 2008; Foos, 1989). Question generation in combination with discussing the student’s questions during a meeting may result in a powerful learning experience. The learning reflection form was given to provide additional time outside of the meeting for students to reflect on and assess their learning. At this point, it is not known if one component is more important than another, or whether such results might be obtained by shortening or otherwise changing the learning check-in. These would be worthy goals for follow-up studies.
Given that each meeting required 20–30 min of the professor’s time, a reasonable question to ask oneself is whether this type of learning intervention is feasible? The answer in part depends on the number of students one has and one’s access to a teaching assistant (TA). In a small class, setting aside such time for meetings may be reasonable, or including this as a TA duty might also be practical. For instructors with larger class sizes, it would be worthwhile to know if shortening these meetings, conducting them online, or having students come to one’s office in groups would produce similar effects. Additionally, as the majority of participants in the sample were female, it would be beneficial in future research to extend such an investigation to classes with a more equal proportion of men and women and to explore the possibility of gender effects.
Beyond the external validity issue of generalizability across gender, there is a potential internal validity concern in the form of expectancy effects because the instructor was not blind to the purpose of the study. Researchers in the area of teaching and learning are confronted with the task of conducting studies in classroom environments while maintaining high levels of internal validity. In the present study, several controls were put in place to minimize the possibility of expectancy effects (detailed in the Procedure section). Ideally, someone blind to the purpose of the study could meet with students and conduct the learning check-in, but this intervention would likely be less meaningful for students interested in one-to-one engagement with their instructor. The controls used in this study combined with the fact that participants who completed the learning check-in prior to Test 3 did not outperform the comparison group make it unlikely that expectancy effects account for the observed results.
Lastly, it is important to note that participants who received the learning check-in first continued to outperform their peers on Test 3, although not significantly. Statistics is a course that requires students to build upon knowledge of past topics in order to understand newly introduced concepts and perhaps the nature of such a course benefited those who first completed the learning check-in. As an illustration of this point, the textbook used for this particular course (by Aron, Coups, & Aron, 2013) includes tips for success, which oftentimes remind students to assess or review their understanding from previous chapters before reading a new topic. Additionally, both tests included questions about t-tests as well as the logic and steps of hypothesis testing, which could lead to carryover effects. Lastly, the learning check-in could have prompted behavior change and goal setting among students that continued throughout the course. Given this result, instructors should take care to offer this type of intervention to all students early in a course to maximize its benefits while treating all students equally.
Appendix
Learning Check-In Form for Test 2
Now that you have completed your meeting take some time to consider your progress in this course. Reflect on your progress and write a reflection on your learning to date (minimum 125 words). Before writing your reflection, take time to consider the following:
Your performance on Test #1 and your desired performance.
Your comprehension of material during lecture and while reading outside of class.
Your understanding and completion of the assignments.
Your understanding of the major topics we have covered so far: displaying and describing data, central tendency, variability, hypothesis testing, distribution of sample means.
Your ability to explain these concepts to someone else.
Are you satisfied with your progress? Is there room for improvement?
Think about the behaviors you do that help you learn the material in this course and behaviors that interfere with your learning. Create two columns listing such behaviors.
What three behaviors can you adopt, continue to do, change, do less of, or more of to succeed in this course?
Develop a study plan for Test #2 (minimum 125 words). What will you do and when will you do it? Specify strategies and time commitments and study locations. Think about how you study. Do you reread the chapters? Do you test yourself without your notes and textbook? Do you meet with other students? Do you read the material and then ask yourself questions or ponder things not mentioned by the authors? Do you make study notes? Do you have others test you/ask you questions about the material? Do you complete and review all the practice problems? Do you ask questions inside or outside of class? When you do not understand something, what things do you do?
Footnotes
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Nexen Scholars Program offered by the Institute for Scholarship of Teaching and Learning at Mount Royal University.