Abstract
Promoting undergraduate engagement is an important and challenging obstacle at large research-focused universities. Thus, the current study evaluated whether a peer-led program of student-geared events could improve engagement among a diverse group of psychology students early on in their degrees. We randomly assigned interested second-year psychology students to participate in the program or to a wait-list. As compared to students who were randomly assigned to the wait-list, students who participated in the program attended more extracurricular events during the year, had higher grades, and reported greater feelings of engagement and enhanced skill learning. These data provide initial evidence that a peer-led program of student events can improve student life for psychology students attending large research-focused institutions.
Keywords
Student engagement is a critical predictor of academic achievement (Trowler, 2010). Data from the National Survey of Student Engagement (NSSE), an annual survey of 80,000 first-year and senior students, suggest that more engaged undergraduate students report greater satisfaction with their classes, earn higher grades, and have superior employment outcomes upon graduation (Hu & Kuh, 2003; Krause & Coates, 2008; Kuh, Cruce, Shoup, Kinzie, & Gonyea, 2008). Although past research has documented the immediate and long-term benefits of undergraduate student engagement, less research has examined the efficacy of programs designed to promote student engagement at large research-focused institutions.
Many students who attend large research-focused institutions report high levels of disengagement (NSSE, 2011). Students at these institutions often report fewer opportunities to interact with their peers and faculty, fewer opportunities to find out about the events being offered on their campuses, and heightened feelings of anonymity (Krause, 2005). While there are many programs designed to bolster engagement at larger research-focused universities—such as honors degrees and thesis courses—these programs typically target only the top tier of academically strong and motivated students who are interested in pursuing research-focused careers upon graduation (Zhao & Kuh, 2004). In doing so, these programs often fail to engage the majority of students who may stand to benefit most (Chambers & Chiang, 2012).
Psychology is one of the most popular majors on campuses across North America, and the majority of psychology majors do not pursue research-focused, graduate-level education upon completion of their bachelor degrees (Hu & Kuh, 2003). Taken together, these findings underscore the need to design and implement programs that target a diverse group of psychology students. A critical next step for programming within research-focused psychology departments is to create initiatives that connect a large and diverse pool of students with available resources and that help to engage the majority of students in the department (see Savitz-Romer, Jager-Hyman, & Coles, 2009, for a similar discussion). Thus, the goal of this research was to evaluate the efficacy of a cohort-based program designed to causally improve engagement among a diverse group of psychology students early on in their degrees.
To address this need, we designed, implemented, and evaluated the success of a Student Engagement Program at a large research-focused university in Canada. This Student Engagement Program was designed to foster engagement among a diverse group of psychology students early on in their degrees. This program targeted second-year psychology students who were new to the psychology major and connected these students with a small cohort of like-minded peers. These small cohorts of 6–8 students were led by a senior undergraduate student who held monthly in-person group meetings, connected with students outside of group meetings via e-mail and social media, and encouraged students to attend ongoing departmental events and initiatives.
During small group meetings, students discussed various topics ranging from study strategies and stress management techniques to applying for research assistantships. Before the school year started, our team held a daylong training event where we provided student leaders with general topics that they might want to discuss during these small group meetings. At the training session, we provided each student leader with a handbook containing facilitator tools (e.g., icebreakers and games) and general ideas for lesson plans and topics. To facilitate use by other departments, colleges, and universities, we have made our Student Engagement Program training handbook available through the Open Science Framework (https://osf.io/6tna4/).
The departmental events and initiatives that were offered throughout the year were designed to target three educational areas that are critical for promoting student success: skill building, community building, and preparing for the future. These categories were developed with the intention of fostering social connectedness, building important and relevant skills, and improving students’ knowledge about future programs and careers (e.g., Hu & Kuh, 2003).
The Student Engagement Program is distinct from other undergraduate programs that our institution offers in three key ways. First, unlike many programs, which consist of 1 or 2-day workshops, the Student Engagement Program is an ongoing initiative designed to support development throughout the year. Second, unlike many initiatives that target a subgroup of students within a cohort based on demographics (e.g., international or transfer students) or within a discipline based on skills (e.g., honors programs), the Student Engagement Program seeks to broadly connect students within our large department based on the fact that they are all pursuing a psychology degree. Finally, unlike many undergraduate programs offered by our university, which focus on enhancing general knowledge (e.g., career and writing services), the Student Engagement Program is tailored for psychology majors, thereby helping students in our major network and connect with opportunities that are linked directly to their undergraduate training.
Overall, the Student Engagement Program has two key aims. The first aim of this program is to connect students with ongoing events and initiatives in our department that could potentially improve their success at our university (e.g., writing workshops) and beyond (e.g., career workshops). The second aim of this program is to help students apply what they have learned through these events by facilitating conversations between like-minded peers and faculty.
By connecting interested students who might not otherwise know about or utilize available resources with the ongoing initiatives in our department, we predicted that students who participated in our program would experience benefits over the year such as improvements in academic achievement (e.g., self-reported study habits), increased knowledge about career opportunities and graduate school, and heightened feelings of engagement at their university.
To assess this possibility, we randomly assigned interested students to participate in the program or to a wait-list, thereby providing critical causal evidence about the benefits of program participation. Given that most research examining the scholarship of teaching and learning relies extensively on correlational or cross-sectional designs (Chickering & Gamson, 1999), it is important to conduct experimental studies. Indeed, because of the difficulty of ruling out selection effects or other potential confounds, correlational and cross-sectional designs often preclude strong inferences about the impact of undergraduate programs on measures of student engagement. Consequently, we sought to provide causal evidence that a cohort-based peer mentorship program improves engagement for undergraduate psychology students.
Study Overview
The design, implementation, and evaluation of the Student Engagement Program took place during the 3 years. In 2012–2013, we collected information about the efficacy about our department’s ongoing events. We used this feedback to strengthen the quality of our events. In 2013–2014 and 2015–2016, we conducted an experimental study to assess the benefits of the Student Engagement Program. See Table 1 for a list of events offered during the program years and Table 2 for the demographic characteristics of the sample.
Description of the Events Hosted by Our Department Each Month in 2015–2016.
Note. The events listed in this table are from 2015 to 2016 Term 1. Similar Events were offered in 2013–2014 and in 2015–2016 Term 2. This table contains all of the events that were offered by the Student Engagement Program in Term 1 of the 2015–2016 academic year. Because of final exams, we did not offer events in December.
Characteristics of Participants at Study Entry, Full Sample.
Note. N = 189.
aFirst-year students’ final end of year grades (GPA) categories: category 5.00 = 72–75% (B), 6.00 = 68–71% (B-).
We follow the reporting standards proposed by Simmons, Nelson, and Simonsohn (2011): We report all exclusions, every condition that was run and every measure that was given. Our data are publicly available through the Open Science Framework (https://osf.io/6tna4/).
Method
Participants
We recruited second-year psychology students enrolled in our department’s mandatory introduction to research methods course. Students were contacted through an online recruitment e-mail and eligibility was confirmed using an initial online questionnaire. We restricted participation to full-time students. These criteria ensured that we recruited students who would be motivated enough to complete the program over the year. See Online Supplemental Materials for details about recruitment strategies employed across each year of the study.
Procedure
At the beginning of the year, students enrolled in our introduction to research methods course were invited to participate. Each year, approximately 700 students are enrolled in this course, as it is mandatory for psychology majors. Of the students we e-mailed, approximately 300 (43%) completed the initial eligibility survey and 222 (75%) were eligible to participate based on their responses. Overall, 189 students were enrolled and 180 students completed the program in its entirety; 9 students (10%) dropped out of the program due to academic hardship (n = 3), feeling overwhelmed with coursework (n = 3), personal reasons (n = 2), or disinterest (n = 1).
After eligibility was assessed and informed consent was obtained, we randomly assigned students to participate in the program or to the control group. Students who were assigned to participate in the program filled out an initial online questionnaire attended a department orientation session and were asked to attend four events and two academic socials of their choosing and to fill out an exit survey. Participating students were also assigned to a cohort of 6–8 peers led by a senior undergraduate student leader, who ensured students attended required events, and who acted as a mentor to participating students throughout the year. Program participants received a graduation certificate, and they received explicit permission to list participation in the Student Engagement Program on their curriculum vitae (CVs). Program participants did not receive a penalty for failing to complete program events; however, if students did not complete program events they would not receive the advertised benefits of program participation.
Students who were assigned to participate in the wait-list control group also attended the department orientation and completed an identical initial questionnaire and exit survey. These students were able to attend department events, which were widely advertised, but they were not required to do so. Students who were assigned to the control group were not encouraged to attend the events by peer mentors, they were not offered the opportunity to receive a graduation certificate, and they were not offered the opportunity to list event attendance on their CVs.
Measures
Overview
Student engagement is conceptualized as an aggregate of related constructs that includes but is not limited to: active and collaborative learning, student–faculty interactions, enrichment opportunities, teacher approachability, peer relationships, and beyond-class involvement (Krause & Coates, 2008). Given this broad definition, we measured engagement using a variety of questionnaires, including measures from the NSSE, targeting three critical components of student engagement: connection to the department, skill building, and future planning.
Measures of Connection to the Department
Sense of Community Scale (SCS)
The 8-item SCS assesses students’ feelings of belonging in their academic community (Krause & Coates, 2008); αT1 = .79, αT2 = .79.
Student–Faculty Interaction Checklist (SFIC)
The 12-item SFIC assesses the frequency of faculty–student interactions (Kuh & Hu, 2001); αT1 = .75, αT2 = .73.
Student–Faculty Interaction Scale (SFIS)
The 12-item SFIS assesses the quality of students’ relationships with professors and administrators (Kuh, 2001); αT1 = .76, αT2 = .78.
Measures of Skill Building and Future Planning
University engagement exit survey (UEES)
The 14-item UEES asks students to report the extent they believe the knowledge and skills that they will acquire in their psychology degree will be useful in their future work and life (Krause & Coates, 2008); αT1 = .90, αT2 = .91.
Transition Engagement Scale (TES)
The 5-item TES asks students to report how engaged, they are with university life (Krause & Coates, 2008); αT1 = .80, αT2 = .83.
Approach/Avoidant Academic Orientation Scale (AAAS)
The 7-item AAAS assesses whether students cope with academic stress (e.g., studying for exams) with an approach or an avoidance orientation (Eaton & Bean, 1995); αT1 = .52, αT2 = .52.
Measures of Academic Support
Academic Engagement Scale (AES)
The 7-item AES measures students’ ability to manage time and other success strategies (Krause & Coates, 2008); αT1 = .75, αT2 = .77.
Peer Engagement Scale (PES)
The 9-item PES reflects collaborative engagement in both classroom and university activities (Krause & Coates, 2008); αT1 = .90, αT2 = .90.
Event attendance
Across both years, we tracked event attendance at each event, therefore enabling us to attain objective attendance data for program and wait-listed students.
Grades
To provide an objective measure of academic achievement, in 2015–2016, we obtained permission from students to collect information about their grades.
Results
Overview
The t tests were used to compare groups at baseline and to assess intervention effects on students’ grades. To test intervention effects on the engagement measures, we conducted analyses of covariance assessing between-group differences, adjusting for baseline levels.
In text, we report and describe analyses that compare all students who were randomly assigned to the wait-list with all students who were randomly assigned to complete the program (Table 3). We also report analyses excluding participants who dropped out of the program (Table 4). Our results are statistically stronger when we exclude students who did not complete the program. We expand on these findings in more detail in the discussion section.
Analyses Reported in Text, Full Sample (N = 189).
Note. N = 189. The scale descriptions are as follows: SCS = Sense of Community Scale; SFIC = Student–Faculty Interaction Checklist; SFIS = Student–Faculty Interaction Scale; UEES = University Engagement Exit Survey; TES = Transition Engagement Scale; AAAS = Approach Avoidant Academic Orientation Scale; AES = Academic Engagement Scale; PES = Peer Engagement Scale; GPA = students’ final end of year grades; CI = confidence interval.
aBecause not everyone consented to allow us access to their grades, the grade data are based on 101 students overall.
Additional Analyses, Excluding Program Dropouts.
Note. N = 180. The scale descriptions are as follows: SCS = Sense of Community Scale; SFIC = Student–Faculty Interaction Checklist; SFIS = Student–Faculty Interaction Scale; UEES = University Engagement Exit Survey; TES = Transition Engagement Scale; AAAS = Approach Avoidant Academic Orientation Scale; AES = Academic Engagement Scale; PES = Peer Engagement Scale; GPA = students’ final end of year grades; CI = confidence interval.
aBecause not everyone consented to allow us access to their grades, the grade data are based on 97 students overall.
Event Attendance
As expected, students who were randomly assigned to complete the Student Engagement Program attended significantly more departmental events during the academic year (M = 3.93, SD = 2.42) as compared to students who were randomly assigned to the wait-list (M = 1.24, SD = 1.34), F(1, 189) = 64.83, p < .001, d = 1.20.
Measures of Connection to the Department
Students who were randomly assigned to participate in the Student Engagement Program reported significantly higher feelings of belonging in their academic community on the SCS at T2 as compared to students who were assigned to the wait-list. As measured by the SFIS and SFIC, students who were assigned to participate in the program interacted more and had higher quality relationships with students and administrators—although these results did not reach conventional levels of significance.
Measures of Skill Building and Future Planning
Students who were assigned to participate in the Student Engagement Program reported marginally greater agreement with statements that their psychology degree helped them to develop skills necessary for success in academics and/or the workplace, as measured by the TES. As measured by the UEES, students who were assigned to participate in the Student Engagement Program reported learning a greater number of useful skills (e.g., analyzing numerical and statistical information, acquiring job-related knowledge and skills, and learning how to create a CV), as compared to participants assigned to the wait-list. Further, participants who were assigned to the Student Engagement Program reported higher approach orientation toward school work; however, these results did not reach conventional levels of significance—likely due to the low reliability of the AAAS.
Measures of Academic Support
There were no differences between the program and control group regarding the extent to which students felt more academically engaged with their peers as a result of completing the Student Engagement Program.
Grades
Students who were randomly assigned to participate in the Student Engagement Program exhibited higher students’ final end of year grades at the end of the year compared to students who were assigned to the wait-list (see Tables 3 and 4). Because we only obtained permission to collect students’ grades in 2015–2016, and because not every student provided us with permission to do so, there is substantive missing data on this measure, thus these results should be interpreted with caution: Smaller samples increase the likelihood of capitalizing on chance (Fraley & Vazire, 2014).
Exploratory Correlational Analyses
Looking within each condition separately, we examined whether an event attendance was associated with higher scores on the T2 measures, controlling for baseline (Table 5). Upon conducting these analyses, greater event attendance was associated with higher self-reported skill development and peer engagement at T2 only for participants who completed the program. These results suggest that peer mentorship may have played a key role in promoting the benefits of event attendance on student engagement.
Correlation Table of Event Attendance Predicting Key Outcomes at T2, Controlling for T1 Measures (Full Sample).
Note. The scale descriptions are as follows: SCS = Sense of Community Scale; SFIC = Student–Faculty Interaction Checklist; SFIS = Student–Faculty Interaction Scale; UEES = University Engagement Exit Survey; TES = Transition Engagement Scale; AAAS = Approach Avoidant Academic Orientation Scale; AES = Academic Engagement Scale; PES = Peer Engagement Scale; GPA = students’ final end of year grades.
*p ≤ .10. **p ≤ .05. ***p ≤ .01.
Discussion
Despite the academic opportunities and advantages of attending a large research-focused institution, many students who attend such institutions report feeling isolated and disengaged. In addition to designing programs that foster engagement among high achieving students, such as honors programs, it is therefore important to design programs that engage the majority of students in large psychology departments with available resources and events.
To this end, we developed and implemented a program designed to improve student engagement for a broad range of psychology majors early on in their degrees. We found evidence that this program improved students’ feelings of belonging at their university. Perhaps most crucially, this program improved students’ perceived learning of skills essential for finding employment upon graduation: including how to apply and obtain a spot in graduate school, how to write a CV/résumé, and how to acquire career and work-related information. We also found initial evidence that program participation improved students’ academic achievement.
Furthermore, students who were randomly assigned to participate in the program attended significantly more events compared to students who were assigned to the wait-list. This finding is notable, given that all students were interested in learning more about the events being offered by the department and both groups received advertisements about these events. These findings suggest that a key component of engaging a diverse group of psychology students is to provide peer mentorship, to provide event attendance expectations (e.g., to attend four events per year), and to incentivize event completion (such as by offering certification for event attendance; see Roscoe & McMahan, 2014, for further discussion of these points).
Another potential benefit of this program is that it provides students with valuable non-research-related leadership opportunities (see also Veronesi & Gunderman, 2012). Although the empirical examination of the benefits of the program for student leaders was outside the scope of the current project, implementing such programs in the psychology departments of various sizes could also provide students with invaluable opportunities to develop their mentorship skills.
We also found initial evidence that peer-group mentorship appeared to promote the benefits of event attendance among program participants. Building on these results, it would be worthwhile for future research to assess the unique contribution of peer groups on student learning independent of event attendance (see also Whillans et al., 2016, 2017).
Across the various measures examined, we found no evidence that program participation increased students’ skills related to working with other students (e.g., teamwork) or that program participation improved students’ academic engagement with one another. As a result, other programs might want to consider explicitly implementing group work to help students gain more experience working in teams and to encourage academically focused peer engagement.
This project highlights the importance of using causal designs to detect the benefits of extracurricular engagement. Using random assignment, we documented modest effects of program participation as well as several nonsignificant effects—suggesting that more research is needed to better understand the program characteristics that are necessary for benefits to emerge.
Because this was one of the first programs to be systematically evaluated at a large research-focused psychology program in North America, more research is needed to understand when, where, how, and for whom cohort-based programs of student-geared events benefit psychology student engagement. Thus, the current work provides a springboard for research to examine the long-term benefits and boundary conditions of related programs. In doing so, we hope that the current research encourages postsecondary institutions to continue to implement programs that enable psychology undergraduates to empower themselves to succeed at university and beyond.
Supplementary Material
Supplemental_Online_Materials - A Program to Improve Student Engagement at Research-Focused Universities
Supplemental_Online_Materials for A Program to Improve Student Engagement at Research-Focused Universities by Ashley V. Whillans, Sally E. Hope, Lauren J. Wylie, Bob Zhao, and Michael J. Souza in Teaching of Psychology
Footnotes
Acknowledgments
We would like to thank the stellar cast of students who made this program possible: Sara Ahmadian, Alejandra Botia, Hyemin Cho, Camilla DeCesare, Yimiao Gong, Michelle Gyenes, George Kachkovski, Veronica LeTourneau, Hayami Lou, Sean Paredes, Robin Richardson, Dania Salih, Tasia Tsatsanis, Danica Verhoeve, Ryan Villamin, and Alicia Yowartt. We would also like to thank Department Head and Professor Geoffrey Hall, Benjamin Cheung, and graduate student and faculty volunteers for their ongoing support of this program.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by a SSHRC Doctoral Fellowship to A.W. and a Psi Chi Hunt Research Award awarded to A.W. and M.S. This program was also generously supported by our department.
Supplemental Material
Supplementary material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
