Abstract
Research has shown that service-learning can increase student-learning outcomes across various disciplines. Service-learning opportunities have been steadily increasing in college settings, especially in health-related fields. This evaluation of an undergraduate public health course at a mid-size, public university in New England sought to understand the impact of service-learning on material retention, internships, and post-graduation public health careers. A 25-item questionnaire was emailed to students who took the course between the Fall 2010 and Spring 2016 semesters with 75 completed (33.3% completion rate). More than half of the respondents noted that their participation in a service-learning project positively affected their internship experience, their first post-graduation job, and their overall career, while also helping them retain the course material after graduation. Open-ended responses provided insight to the qualitative responses. Respondents noted that in addition to having the understanding and ability to put on programs in their community, they also had more confidence in their abilities. The opportunity to practice public speaking skills as well as work collaboratively in group settings, were both noted as essential skills needed to succeed in the public health workforce. It was concluded the project had a positive impact on student’s learning and retention in planning, implementing, and evaluating an actual health promotion program as well as how they were able to use those skills in their careers.
Introduction
In the United States, undergraduate students at accredited Schools and Programs of Public Health are required to have opportunities that allow for the integration and application of curriculum through cumulative and experiential learning. Council on Education for Public Health (2016) notes that this can occur through the completion of service-learning projects, internships, or field experiences. Service-learning is an educational approach in which students engage in learning activities that also contribute to the community in which they live. While these structured projects allow for increased learning through hands-on work, it also can benefit those who are served by the project (Horney et al., 2016; Jacoby, 1996). While service-learning links theory and practice, it does not have a specific theoretical framework. However, Cashman and Seifer (2008) reviewed numerous previous studies of courses with service-learning components and concluded that having public health students apply academic lessons to a real-world context can be a highly effective mechanism for preparing them for careers in community health.
Research has shown that participation in service-learning experiences has yielded significant increases in student learning outcomes (Astin et al., 1999; Giles & Eyler, 1998; Gottlieb & Robinson, 2006; Tucker, 2010; Ylitalo & Meyer, 2019). In fact, service-learning opportunities in colleges and universities have been increasing since the mid-1980s and are standard in several health-related disciplines, such as physical therapy, nursing, and public health (de Groot et al., 2015; Fralinger et al., 2013; Reising et al., 2008; Tapley & Patel, 2016). Studies of courses utilizing service-learning projects show benefits not only for the agency receiving the service but also for the students, including a better understanding of the course concepts and feeling more prepared for their future (Cashman & Seifer, 2008; Cooke & Kemeny, 2014; Fralinger et al., 2013; Hou, 2009).
Service-Learning in Health Promotion
While the field of health promotion is ripe for service-learning, few studies have looked at the outcomes. One study sought to examine undergraduate students’ feedback of how the health promotion programs that students designed and implemented affected their academic experience and perceived learning of course material (Fralinger et al., 2013). The students self-reported higher levels of perceived skill development, higher levels of learning, and a higher evaluation of classroom experience when compared with traditional, classroom-only methods of learning (Fralinger et al., 2013). A second study (Rooks & Rael, 2013) examined whether service-learning projects facilitated students’ understanding and learning of course concepts. While this study found that students did not believe that service-learning helped them learn course concepts, they did feel that it was rewarding for future public health careers. Additionally, a study focused on nursing students who used service-learning to master health promotion theory found positive benefits in terms of comprehension of course material as well as in terms of developing professional and community responsibility (Reising et al., 2008).
Expanding beyond health promotion to public health undergraduate programs shows additional positive perspectives from students. A qualitative study on the impact of service-learning on personal and professional development among undergraduate students in a nutrition program found that students expressed feelings of purpose or impact and a significant gain of real-world experience (Roofe, 2012). Community-based service-learning projects have been shown to help epidemiology graduate and undergraduate students gain experience outside of the classroom, while benefiting both their learning and their understanding of public health field work (Ylitalo & Meyer, 2019).
To better understand the impact of service-learning, a case was made that longitudinal studies are needed to explore the impacts once the student leaves the university setting (Eyler, 1999; Giles & Eyler, 1998). While this statement was made almost 20 years ago, the literature still shows a dearth of studies where students were surveyed after leaving the university and had time to understand the impact of participating in a service-learning project on their professional development. Previous studies have looked at the impact of integrating service-learning into academic programs in various health-related disciplines, but few have looked at its impact on future internship experience and post-graduate careers in a public health setting. This evaluation would add to the pedagogical literature on the impact of service-learning, post-graduation.
Method
To further understand the continuing impact that undergraduate participation in service-learning has after graduation, an evaluation of an undergraduate health promotion course was conducted.
Service-Learning Project
The Department of Public Health’s undergraduate program is highly intensive, with numerous hands-on classroom activities and internship experiences built into the curriculum, many with a focus on giving back to the community. By taking part in these community-focused experiences, undergraduate students have the chance to apply the classroom concepts to real-world settings, providing invaluable experiences that they can bring to their post-graduate careers. The Program Planning course is one of the many public health courses that allows undergraduate students to apply health promotion and health education theories and concepts to a real-world setting on their college campus. The goal of the course is to ensure students understand how to conduct a needs assessment in order to be able to plan, implement, and evaluate community health programs. Competencies come from the Council on Education for Public Health, and the design of the course is also guided by many of the Certified Health Education Specialist’s Areas of Responsibilities.
Program Planning, a required course limited to public health majors only, incorporates service-learning through a semester-long project that allows students to learn and participate in the program planning process while giving back to the university community. Students work together in subgroups focusing on planning, implementation, or evaluation to conduct two health education events in select first-/second-year residence halls on campus. There are typically two sections of the course offered each semester, with 20 to 30 students per section.
The original design of this Residence Life Service-Learning Project was crafted in collaboration with the Office of Residence Life to create a project in which students would implement health education sessions in residence halls mid-semester. The residence halls housing first- and second-year students were selected for project use for two reasons. First, the buildings are designed as “straight line” traditional residence halls, where two to three students live in one room, with a shared hall bathroom. The lobbies tend to be hubs of activities, providing a space with a significant amount of traffic in the early evening. Second, it was felt that providing information to newer students, many living on their own for the first time, would provide the most benefit to residential students. With course students primarily juniors and seniors, this sets up a strong peer modeling opportunity.
Students were put into groups of approximately 10 and allowed to pick among two to three topics preselected by Residence Life. Prior to the start of each semester, the course instructor met with the Associate Director of Residence Life for Residential Education and Community Development to determine these topics. The goal is to find health topics not being provided by other campus offices in order to fill educational gaps. Typical topics for the service-learning project address stress, healthy eating, navigating health as an underclassman student, skin care, and substance use. With each course section using the same two to three health topics and each group implementing one session in two halls, it allowed for each topic to be implemented in four of the five target buildings each semester within a 4-week window mid-semester.
Residence Life provides a Residence Hall Director to work with the course instructor during the semester, and they are responsible for identifying Resident Advisor (RA) hosts in the targeted buildings. The RA’s work with the students in the Program Planning course to select the dates for implementation, fostering skills for both the course students and the RA’s. Residence Life also provides financial assistance to the students to pay for program materials and food for each event that takes place in the hall lobbies. The 8:00 p.m. implementation time was chosen to account for the most foot traffic in the hall lobbies to invite participants; it also was selected as senior public health students often have night classes that end at 7:30 p.m.
Once course students have been placed into their topical group, they self-select into one of three subgroups: Planning, Implementing, and Evaluating. The Planning Team consists of two to three students; their role is to work with the host RAs to set the program dates, discuss food that the RA will order and provide, create goodie bags that will be given out at the end of the program containing educational materials and other items they find for reduced cost or free, and recruit participants for the night of the event. The Implementation Team consists of three to four students who present the program. The implementation team used a provided fact sheet to create a 30- to 45-minute presentation that includes games mixed in with a PowerPoint presentation using the lobby large-screen TVs. The four-person Evaluation Team creates a pretest–posttest using the same fact sheet. The Evaluation Team gives the pretest prior to the event, observe using an instructor-created process evaluation form, collect posttest data at the end of the event, and then make data tables by the last week of the semester. All subteams submit a portfolio of their work, showing their semester tasks and outcomes. Then, these students write an individual evaluation report using their group’s pretest/posttest data and process evaluation materials as the final exam for the course.
Student Recruitment
Former students were invited to participate in this evaluation if they met all of the following criteria: successfully completed the course between fall 2010 through spring 2016 and graduated with a degree in Public Health by August 2016. This timeline would allow for students to be in the workforce for at least 1.5 years since graduation and possibly implement the skills gained from the course; work experience was not a requirement for data inclusion. Over the nine semesters the course was implemented by the instructor, 422 students met the eligibility criteria (the instructor was on sabbatical in fall 2014 and the project was not implemented that semester). A database was built using each semester’s roster, with students deleted from the list if they did not meet the eligibility criteria. The primary resources for gathering email addresses were the department student list, which had some non-university emails on it; LinkedIn; and Google. For 119 students not currently affiliated with the university as a graduate student or employee, no non-university email could be identified, and so they could not to be contacted.
The questionnaire was sent out in March 2018 via SurveyMonkey, often to multiple emails when available. Seventy-eight emails were invalid. Another 119 email invitations were not opened during the time of the evaluation, so they may also have been invalid or not in current use by the student; therefore, there were 225 verified eligible students. Additionally, no other method, such as mailing hard copies of the survey, was used to gather data from these alumni, as mailing addresses were not accessible to the research team. No incentives were offered, so as not to pressure the alumni to respond and to avoid forcing alumni to identify themselves.
Dillman’s Tailored Method was used to determine the send times for the survey (Dillman, 2000). Ten days after the initial email, a second email was sent from SurveyMonkey thanking those who had completed the questionnaire and reminding those who had not responded to do so. Two weeks thereafter, a final reminder email was sent to those who had not yet responded. The questionnaires were confidential, but not anonymous, and we kept the email tracker function on so that we could identify those who completed the survey for this final email.
Instrument Development
An instrument was developed specifically for the Residence Life Service-Learning Project by the authors. The questionnaire contained a total of 25 questions. All close-ended questions were created as required questions in SurveyMonkey; the open-ended questions were not required to be completed by the respondent. The first section created the student’s unique identification number, which was a combination of their initials and birthdate. The second set of four questions asked when they took the course, when they graduated, if the course was taken at the same time as their internship (typically 5% to 10% of students per semester), and their project subteam. The next section asked about post-graduation activities, including further schooling, whether they had had a job in public health since graduation and, if so, in what area, and if not, in which field they were working. Participants were also asked about the setting of their job, regardless of whether it was in public health. Finally, participants were asked whether they had ever done job tasks that were related to the course, such as putting together a program or conference, evaluating a program, and writing a grant and/or budget.
The next section contained eight questions related to the course and its post-graduation impact. The first five questions, four of which were related to this evaluation, were yes/no questions with a text box for the student to share clarifying thoughts. The remaining questions addressed a desire to learn concepts not covered by the course and, if so, what those concepts were; students also had the opportunity to provide any other feedback about the course. Two demographic questions—gender and age—ended the questionnaire.
During the development stage, the questionnaire was pilot-tested with a group of recent graduates whose graduation date made them ineligible for the evaluation. Ten recent alumni, diverse in gender and race, who were already working in the public health field were invited to participate. These alumni came to the university after work, took the instrument on their laptop, and were then led through a focus group to determine readability of the instrument and whether any questions might be missing. The focus group lasted 40 minutes, and their suggested edits were addressed prior to email distribution. Institutional review board approval for the project was received from the university at two points: for the instrument pilot test phase and for the full distribution of the questionnaire.
Data Analysis
Frequencies were determined for demographic information related to the course: semester of enrollment, semester of graduation, current job focus, graduate school attendance, and project subteam. Frequencies and percentages were used for determining outcomes related to career tasks conducted and the four outcome questions of interest: impact on internship, impact on first post-graduation job, impact on career overall, and retention of course material. For the four outcomes of interest, bivariate frequencies and percentages were calculated compared with semester taken, subteam, and current job focus. Given the small size of the dataset, advanced testing, such as chi-square, could not be undertaken due to small cell sizes.
The four outcomes of interest had spaces after each for the student to provide explanations of their answers, and the qualitative data was reviewed for common ideas and themes. Each question was reviewed individually by the faculty member and research assistant to look for common ideas. Once complete, the information was reviewed and discussed together to determine overlap in common ideas. Each individually reviewed the transcripts to identify illustrative quotes and then came together to make the final decisions on which best represented the idea.
Results
Sample
A total of 75 respondents completed the survey, for a completion rate of 33.3%, which is within normal range for web-based surveys (Dillman et al., 2009; Smyth et al., 2010). Among the alumni who responded, 75.6% were female and 71.7% were in their 20s. The responses were mostly evenly distributed among semesters taken; the most recent semester grouping has one fewer semester due to the project not being implemented while the faculty member was on sabbatical. The respondents were asked which subteam they were on during the Residence Life Service-Learning Project; planners were the smallest group, although that would be expected as their subteams were smaller than the other two (two to three members compared with three to four for implementation and evaluation). An even split was seen among the student’s current job, with a third in health education/promotion, a third in another public health field (primarily health care administration and environmental health/sanitarians), and one third working outside of public health (primarily social work and nursing). Almost half (45%) of respondents had gone on to a Master of Public Health degree program since graduating (Table 1).
Sample Demographics.
Faculty on sabbatical, Fall 2014.
Respondents were asked which activities they did in their current jobs that related to learned course concepts, with the option to select as many on the provided list as were relevant. Nearly half of the respondents planned or implemented programs, either multi- or single-session. Other related activities conducted in their post-graduation careers included putting a community event or forum together, and evaluating a program (Table 2).
Health Promotion Activities, Post-Graduation.
Post-Course Impacts
For each outcome that was evaluated, data indicated that more than half of the respondents felt that the Residence Life Service-Learning Project positively affected their internship experience, their first post-graduation job, their overall career at some point in time, or helped them to retain the course material. When looking just at the impact of the project on respondents’ internships, nearly all respondents in the Evaluation subteam noted the project had had a lasting impact, compared with those who were in the Planning and Implementation subteams. Data indicate that respondents on the Evaluation and Planning subteams noted the most benefit when asked about the project’s impact on first post-graduation job (81.5%, 77.8% vs. 56.0%, respectively). The Residence Life Service-Learning Project especially affected those who ended up working in the health promotion/education field compared with those who did not, with both related questions at 88% for health promotion alumni versus 45% to 69% for the remainder of respondents.
When asked about the Residence Life Service-Learning Project’s impact on retention of course material, those working in health promotion/education or another public health field found the project to be the most impactful. Respondents who were in the Implementation and Evaluation subteams held this view more than those who were in the Planning subteam (67% to 68% vs. 56%). Respondents who were working in either health education/promotion or another public health area noted that the project affected their ability to retain the material post-graduation more than those who were not working in the public health area, which might be expected, as those in public health used the information more regularly (Table 3).
Residence Life Service-Learning Project’s Impact.
Faculty on sabbatical, Fall 2014.
For each question, respondents were given the opportunity to explain their responses. This provided a deeper understanding into the impact of the Residence Life Service-Learning Project for their learning experiences. Respondents stated that their participation in the course project had a significant impact on their internship experience. Many respondents whose internship placements had them planning, implementing, and evaluating health promotion programs noted that the project gave them the experience and tools necessary to successfully run effective programs. One student stated, “. . . it gave me an experience with the steps involved in creating implementing and evaluating a program of my own, which I did in my internship.” Through this project, respondents learned the specific process for running health promotion programs, and many took that knowledge and applied it to their internships. As another student noted, “It was helpful because it taught me how to evaluate the programs for my internship. It also helped because I was responsible for helping plan successful programs and I referred to my notes from class which I still have.”
Frequently mentioned was the fact that the Residence Life Service-Learning Project gave respondents the tools needed to effectively plan and implement programs at their place of employment. Several respondents even stated that they were allowed to take the lead on projects due to the experience they had gained from the project. Other themes that emerged from respondents’ feedback included feeling prepared and confident in their ability to run a successful program even if things do not turn out as originally planned. As one noted, “For my job currently, I create and implement programs frequently. The program in the residence hall helped me learn how to create programs on my own.” Another stated, “Currently have to help plan and implement a program on a daily basis. Helped give me the knowledge and confidence to implement better.” Finally, one student provided a detailed description on the impact of the project: The experience described above provided me with a concrete set of skills/tools with which I could “hit the ground” running in my first post-graduation job. One of the most valuable parts of the Residence Halls project, and the entire [Program Planning] course (and its placement in the health education course sequence at [University Name]), is that respondents obtain practical experience and concrete skills grounded in evidence and theory. Additionally, I used the “notebook” I built during the Residence Hall event planning project as an example of my past work during the interview for the job I obtained post-graduation.
As previously mentioned, respondents noted that their participation in the Residence Life Service-Learning Project had enhanced their overall careers. Not only did the project teach them about the planning, implementing, and evaluating processes, but it also helped them to improve their soft skills in providing health promotion to communities. Several respondents (n = 19) mentioned having improved confidence in their public speaking skills and their ability to conduct presentations as well as efficiently and effectively work in a group setting. As one student who is working in the health care administration field stated, The Residence Halls project taught me how to plan events and work collaboratively with others. In my current job, I often times have to plan lunches and meetings. I also am required to manage contracts and stay within budgets, so [Program Planning] helped me prepare for my responsibilities.
A student working in health promotion noted, “The residence hall project was helpful because it helped me practice working in a team, presenting, and public speaking. Which are all essential skills needed in the work force.”
Several respondents talked about the Residence Life Service-Learning Project’s impact on course material retention after graduation, mentioning that they still have their notes and refer to them in their work. Respondents also mentioned completing their internship soon after finishing the Program Planning course, applying what they had learned to their internship duties, and essentially retaining the course material that way. As one said, Yes, because I was able to put into practice what I learned in class, twice. In addition, I did my internship shortly after taking the class and I was able to practice the course material throughout my internship. This helped me retain the material after graduation and to this very day.
Discussion
Respondents noted that working with service-learning projects outside the classroom helped them better understand the health promotion concepts from the classroom; they also noted that they were more prepared for their professional careers. These results are similar to previously conducted studies on course-based service-learning projects (Cooke & Kemeny, 2014; Fralinger et al., 2013; Hou, 2009). Respondents stated that they retained and used the information from the Residence Life Service-Learning Project within their work settings, both in health promotion and in related public health careers, such as health care administration. The skills used to plan and implement community programs translate well to organizing large meetings for staff in other health settings. Throughout the course, respondents received regular feedback on the program’s development and the project was regularly linked to course goals of understanding programs from planning through evaluation through lectures and classroom discussions. Like other evaluations of courses, linking the project to desired outcomes helped respondents see the benefits of their experiences (Hodges & Videto, 2008; Hou, 2009).
An unexpected finding that became apparent from the respondents’ responses was the project’s impact on the development of soft skills that they were able to apply to their careers. The opportunity to practice their public speaking skills as well as work collaboratively in group settings were both noted as essential skills needed to succeed in the public health workforce. The benefit of developing these soft skills in service-learning projects was also seen in a study on service-learning in a chemical engineering course (West, 2017). While soft skills, such as the ones noted previously, are essential for many types of careers, it is especially important for a career in health promotion. Attainment of these soft skills, combined with the confidence that respondents expressed gaining from their participation in the Residence Life Service-Learning Project, clearly demonstrates the impact that service-learning can have on undergraduate students who plan on entering the health promotion field.
Another theme that emerged from this evaluation was that taking part in the Residence Life Service-Learning Project allowed students to retain course material after graduation. Respondents’ comments on how they were able to retain and apply what they learned post-graduation reinforced similar findings from other studies that showed increases in material retention through participation in service-learning projects in fields such as nursing and web design (Lee et al., 2018; Reising et al., 2008). This evaluation, however, showed how service-learning can yield career benefits in a public health setting.
Limitations
There are limitations to this study that should be noted. It was very difficult to gather emails to contact the students, as the process was limited to what the instructor, academic department, or online database could provide. The university database was often out of date, as those emails were updated by the student prior to graduation. Online sources such as LinkedIn proved challenging as well, as many students had not updated the information or listed their university email on their page. The university often closes email accounts within 6 to 12 months of graduation, so only those students enrolled in graduate programs at the university or working for the university would still have active university emails. These unknown or unverified emails led to a smaller sample size than desired.
It is also possible that students who received the questionnaire did not want to respond, knowing that it came from the course instructor. This may be especially true for those who perceived that they did not have as much positive impact with the Residence Life Service-Learning Project as others. Of those who chose to provide qualitative responses, the comments were either positive or noted that they had no comments to add. Those with negative views may have chosen not to write in an answer, which may show some bias in the responses. Participants were asked to provide information to create an identification number to help reduce the number of follow-up emails sent, and even with reassurance it would not be included in the analysis database, it may have limited responses.
Implications for Course Design
What makes the Residence Life Service-Learning Project successful is the connection between the course instructor and the Residence Life staff. The Office of Residence Life provides support, both in personnel and in funding, that ensures the program can be implemented as designed. The Vice President for Student Affairs at the university is not only aware of the program, but highly supportive, which is critical to implementation. The university has a strong desire to integrate Academic Affairs and Student Affairs to enhance student outcomes, and this atmosphere allows a cross-campus collaboration to flourish; it has also led to other collaborations with faculty, including Living Learning Communities. While intensive for both the instructor to implement and for the respondents to participate in, this service-learning project remains valuable. The respondents noted the project’s impact on their learning and retention in planning, implementing, and evaluating an actual health promotion program as well as on how they were able to use those skills in their careers.
Footnotes
Acknowledgements
The corresponding author thanks the Office of Residence Life for their support in hosting and providing financial assistance for the course project. Additional supports are provided by the university Wellness Center and Fitness Center, all which fall under and are supported in this collaboration by the Vice President of Student Affairs.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported through a university program, Graduate Student Graduate Assistantship, which provided funding for the graduate student coauthor to work on this project for one semester. No other funding was provided.
