Abstract
Master of Public Health (MPH) students must develop skills to work in collaborative, interdisciplinary, and interprofessional teams upon graduation. Teaching pedagogies that involve active learning and collaboration between students, such as team-based learning (TBL), may be helpful in preparing students to work in teams. To our knowledge, there is no literature examining TBL specifically in the context of graduate public health education to date. This study evaluated the implementation of TBL in a Behavioral Science Research Methods course on the following outcomes: (a) engagement with the course material, (b) perceived effectiveness of TBL components, (c) experience working in teams, and (d) perceived mastery of learning outcomes. We conducted a survey of first-year MPH students enrolled in the TBL research methods course (n = 45 respondents) and three focus groups (n = 17 total). Two focus groups were conducted with second-year MPH students who previously took a lecture-based version of the course (n = 10 total). Overall, students in the TBL research methods course felt confident in their mastery of learning outcomes. Students appeared more engaged with the TBL course material than with material from lecture-based courses. They also perceived this instructional method to be effective in facilitating learning. Main themes that emerged around working in teams included the central role of teammates in students’ learning, the logistic and procedural roles students took on within their teams, and team dynamics. The TBL format engages MPH students in course content and has strong potential for preparing students for collaborative work in diverse teams.
Schools and programs of public health must equip graduate students with the knowledge and skills necessary to perform competently in a dynamic and complex public health professional landscape. Entering the public health workforce in any capacity (e.g., researcher, practitioner, consultant, or blended professional roles) demands a fine-tuned ability to apply knowledge and skills within collaborative, interdisciplinary, and interprofessional teams (Fried, 2015; Thibault, 2015). These skills are reflected in the revised accreditation criteria released in October 2016 by the Council on Education for Public Health, which serves as the accrediting agency of public health schools and programs in the United States offering Master of Public Health (MPH) degrees. The list of 22 recommended competencies covers discipline-specific public health core content areas (biostatistics, epidemiology, social and behavioral sciences, health services administration, and environmental health sciences) as well as interdisciplinary/cross-cutting domains that include the areas of leadership, communication, interprofessional practice, and system thinking (Council on Education for Public Health, 2016). While discipline-specific competencies emphasize teaching of foundational public health content, interdisciplinary/cross-cutting competencies seek to instill a set of characteristics and attributes that enhance students’ capacity to function effectively in a multifaceted public health work environment. As such, the latter group of competencies focus specifically on “fostering collaboration,” “guiding decision making,” “empowering others,” “selecting communication strategies,” and “performing effectively on interprofessional teams” (Council on Education for Public Health, 2016). These competencies formally recognize the importance of training a future generation of engaged public health professionals skilled in team processes and effective in creative problem solving. The focus on these professional skills may be particularly important given a trend toward significantly younger matriculation age in schools and programs of public health over the past three decades (Association of Schools and Programs of Public Health, 2017). Although MPH programs historically enrolled working professionals (e.g., physicians, nurses) with substantial experience in interdisciplinary and interprofessional settings, many students today enroll in MPH programs directly after completing their undergraduate degrees (Kennedy & Baker, 2005). Consequently, a significant proportion have little to no work experience that would have offered the opportunity for acquiring teamwork skills.
In recent years, professional degree programs in health science disciplines, such as medicine and nursing, which require professionals to operate effectively as part of complex, interdisciplinary teams, have moved away from traditional didactic pedagogies that rely on passive learning (McLaughlin et al., 2014) toward the implementation of active learning models. The latter are informed by experiential learning theories that encourage the creation of learning environments where students engage in new experiences in ways that encourage abstract conceptualization and application of information to new situations (Kolb, 1984). Team-based learning (TBL) is one such active learning method originally developed by Michaelsen, Knight, and Fink (2004) to help students acquire and apply new information in novel ways within the context of a team.
Overview of Team-Based Learning
The overall scope of TBL is to create structured, active learning experiences that allow students to master all levels of cognitive processing: remembering, understanding, applying, analyzing, evaluating/synthesizing, and creating (Bloom, Engelhart, Furst, & Krathwohl, 1956). The team setting encourages students to individually and collectively develop effective ways to navigate team dynamics through communication and collaboration while also learning necessary discipline-specific content. The successful implementation of TBL rests on four foundational pillars that we will briefly outline here (Michaelsen et al., 2004). Key components of a typical class are italicized. For a complete description of TBL and its implementation, please refer to Michaelsen and Sweet (2008).
Group Formation
Instructors strategically assign students to groups to ensure demographic diversity and adequate distribution of group members’ previous experience with the content area. Students remain with their assigned group for the entire semester, allowing sufficient time to learn how to navigate unique interpersonal dynamics and develop a cohesive team with a high sense of accountability to each other.
Accountability
Students are held accountable for individual and team performance. Individually, students are expected to complete all preclass reading assignments. An individual readiness assurance test (iRAT) consisting of multiple-choice questions is administered at the start of each instructional unit to create accountability for completing the readings and assess whether students have retained and understood key concepts. Immediately following the iRAT, students complete a team readiness assurance test (tRAT) where students must reach consensus and provide their answers to multiple-choice questions on a scratch-off form. The form is designed so that it immediately reveals the correct answer. Students have the opportunity to appeal any question they marked incorrectly by providing a substantive and convincing written request to the instructor substantiating a grade reversal. This further motivates students to engage with the assigned reading material, reinforcing previously learned material and/or highlighting areas of confusion.
Feedback
Students receive frequent and immediate feedback to enhance learning and retention. Subsequent to receiving immediate feedback regarding their individual and team performance on the tests, the instructor offers a brief presentation, akin to a mini lecture, designed to provide further clarification of content material that proved most challenging to students based on their individual and team test performance.
Team Assignments
Instructors develop in-class team assignments that facilitate individual learning and team development. The team assignment requires students to apply course content in order to solve a problem or make a decision. The preclass preparation and readiness processes described above motivate students to engage in meaningful interaction and ongoing collaboration to analyze and evaluate what they have learned, and then generate new ideas and solutions.
Effectiveness of Team-Based Learning
TBL has been used in various disciplines including business (Michaelsen, Peterson, & Sweet, 2009), health sciences (Fatmi, Hartling, Hillier, Campbell, & Oswald, 2013; Sisk, 2011), and to a limited degree in the social sciences (Huggins & Stamatel, 2015; Macke & Tapp, 2012; Travis, Hudson, Henricks-Lepp, Street, & Weidenbenner, 2016). Results from two systematic reviews offer evidence in support of TBL, although discrepant findings are presented across several different outcomes (Fatmi et al., 2013; Sisk, 2011). Participants across 17 studies reviewed by Sisk (2011) included a mix of undergraduate and graduate students representing various disciplines (health sciences, psychology, and economics), as well as one study focusing on physicians in a continuing medical education program. Participants across the 14 studies reviewed by Fatmi et al. (2013) included only professional and allied health professional students. Only three studies overlapped between the two systematic reviews. Both reviews report higher examination scores in TBL courses overall. More recent evidence from a study of over 1,000 undergraduate students in an introductory psychology course found that TBL was associated with higher examination scores than traditional lecture (Travis et al., 2016). However, there is mixed evidence regarding student satisfaction with TBL. Specifically, Sisk (2011) reported overall greater student engagement and satisfaction with TBL; yet among professional and allied health professional students, only one study reported higher satisfaction with TBL (student engagement was not evaluated; Fatmi et al., 2013). The recent study with undergraduate students in an introductory psychology course also found no associations between TBL and student course satisfaction (Travis et al., 2016). Inconsistencies in results may be due in part to a range of target populations and research designs employed in these studies (Sisk, 2011).
Extant research examining TBL suggests this is a promising pedagogical model that should be implemented and evaluated more broadly, particularly in disciplines where teamwork is critical. To our knowledge, there is no literature examining TBL in the context of graduate public health education to date. The purpose of the current study was to evaluate the implementation of TBL in a Behavioral Science Research Methods course on the following outcomes: (a) engagement with the course material, (b) perceived effectiveness of TBL components, (c) experience working in teams, and (d) perceived mastery of learning outcomes.
Method
Behavioral Science Research Methods is a core course taught to first-year MPH students during the first semester of a 2-year academic program at Emory University, Rollins School of Public Helth (Atlanta, Georgia, United States), in the Department of Behavioral Sciences and Health Education. In this study, we assessed students who took the original, lecture-based course in 2014 and those who completed the equivalent course in 2015 using the TBL format.
Behavioral Science Research Methods Course
Lecture Format
In the fall of 2014, three different course sections of roughly equal size were offered to accommodate a total of 86 incoming students. Each of the three sections met weekly for 2 hours and 50 minutes. Two sections were taught by one instructor while another instructor taught the third. Both instructors were assisted by a teaching assistant (TA) who was either a current MPH student who had taken the course previously or a PhD student who had completed an MPH degree. The same course schedule and syllabus were used across all course sections. During the 14-week long semester, there were 11 instructional units in all. These are defined as class sessions addressing new content and exclude exam weeks. During instructional units students received lectures (approximately 60 to 90 minutes long) that explained concepts from assigned reading material, followed by a group activity designed to apply the principles learned in the lecture. For these activities, students would self-select into groups, and they did not have to remain in the same group during the semester. Therefore, students often did not know all of their group members and their working styles from week to week. Group activities were primarily conceived as a method to break up the didactic lecture style and were not intentionally designed as a primary method of learning. Final course grades were based on individual (90%) and group performance (10%; see Table 1).
Comparison of Student Evaluation Criteria in Lecture Versus Team-Based Learning Course Format for a Master’s-Level Behavioral Science Research Methods Course.
Note. TBL = team-based learning; CITI = collaborative institutional training initiative; iRAT = individual readiness assurance test; tRAT = team readiness assurance test.
TBL Format
The same Behavioral Science Research Methods course was redesigned and taught using the TBL format in the fall of 2015. As before, three different sections of the course were taught weekly for 2 hours and 50 minutes and accommodated 76 enrolled students. Two sections were taught by the same instructor while the third section was taught by a second instructor. Similar to 2014, both instructors were assisted by a TA and the course schedule and syllabus were shared across all course sections. The syllabi for the fall 2014 and 2015 courses covered nearly the same instructional units. Differences included the addition of two instructional modules and minor changes to the sequence of modules in 2015 compared to 2014. Of the 14 class sessions, 12 were considered individual instructional units and included the following topics: overview of research process, literature review, ethics, observational research, experimental research, quasi-experimental research, sampling, measurement, qualitative research, mixed methods, community-based participatory research, and program evaluation.
In all three sections, the instructor preassigned students to teams based on a short precourse survey designed to assess students’ previous academic or practical experience with research methods. Students completed the survey prior to the first class meeting and responses were compiled by the TAs. During the first class, students were informed that their responses to the survey served as the basis for assigning them to their team for the duration of the semester. Five students were assigned to each team with the exception of one group with six students.
The TBL components described previously were administered for each of the 12 instructional units following an identical three-phase structure. During Phase 1 (approximately 70 minutes), students completed multiple-choice (four answer choices), closed-book iRATs immediately followed by identically structured tRATs. The instructor and TA graded iRATs during tRAT administration. The Immediate Feedback Assessment Technique was used by each team to grade and receive real-time feedback on their tRAT performance (Michaelsen & Sweet, 2008). Specifically, using scratch-off forms provided by the instructor, teams scratched off the box corresponding to their chosen answer for each tRAT question revealing either a star (correct answer) or a blank (incorrect answer). Partial points were awarded if students scratched off the correct answer on subsequent attempts. After the TA returned the graded iRATs to students, the instructor provided a brief lecture reinforcing key concepts from the assigned reading as well as clarifying areas of confusion raised by students or evident by scanning iRAT and tRAT responses. Following a 10-minute class break, during Phase 2 (approximately 70 minutes) students worked in their teams to complete a team assignment that required application, critical evaluation, and synthesis of content covered during that class period. All teams received the same assignment. The instructor and TA were available to address questions as needed. Last, during Phase 3 (approximately 20 minutes), teams used various forms of simultaneous reporting (e.g., holding up cards indicating teams’ answer choices, simultaneously writing brief answers on boards in the classroom) to present how they had conceptualized the team assignment. Class concluded with a discussion based on students’ synthesis of similarities and difference across responses to the assignment by all teams. After class, each team submitted a written document summarizing the team assignment for grading. Students were allowed to submit written appeals for incorrect answers on the tRATs up to 24 hours after class. Following a mid-semester evaluation required by the department for all new or modified courses, we eliminated the iRATs for the remainder of the semester (see Focus Groups Results below). Final grades were based on individual (50%) and team performance (50%).
Evaluation Design
This mixed-methods evaluation used two data sources: an online quantitative survey and focus groups. The purpose of the online survey was to assess students’ perceived effectiveness of TBL components and perceived mastery of learning outcomes. The survey was administered one time to first-year MPH students who completed the TBL version of the course only. To examine students’ engagement with course material and overall experience with working in teams, focus groups were held with both first- and second-year MPH students who completed the TBL and non-TBL courses, respectively. A graduate student in the doctoral program of the Behavioral Sciences and Health Education Department who had no prior involvement with the courses served as an external evaluator for the project and facilitated data collection. The Emory University Institutional Review Board approved all study procedures.
Student Survey
At the conclusion of the semester, all first-year MPH students enrolled in the TBL version of the Behavioral Science Research Methods course (N = 76) received an e-mail with a link to a short online survey administered through SurveyMonkey. TAs in all sections of the course made announcements about the survey. Additionally, the evaluator assisted in recruiting students for the survey by sending reminder e-mails.
Measure of perceived effectiveness of TBL components
Participants were asked to indicate whether each key component of TBL (i.e., preclass preparation, individual tests, team tests, faculty feedback/lecture, team assignments) was “not helpful at all,” “somewhat helpful,” or “very helpful” in contributing to their learning.
Measure of perceived mastery of learning outcomes
The survey asked students to rate their confidence in mastering eight learning objectives: describe each step in the research process, synthesize previous research findings in order to identify a gap in the literature, write a clearly defined research question, choose an appropriate research design for a research question, choose an appropriate sampling method for a research study, choose appropriate measures for a research study, explain the difference between qualitative and quantitative research, and evaluate the Method section of journal articles. Answer choices were “not confident,” “somewhat confident,” and “very confident.” The survey took approximately 5 minutes to complete. A total of 45 students completed the survey, representing a 59.2% response rate. Basic descriptive statistics of demographic characteristics (gender, race/ethnicity) and the TBL items were examined using SPSS Statistics (Version 22, IBM Corp, Armonk, NY). Chi-square tests were conducted to determine if student responses differed by instructor.
Focus Groups
Five focus groups were conducted between December 2015 and February 2016: two with second-year MPH students who took the course under the traditional lecture-style format, and three with first-year MPH students who received the TBL format. Focus group participants were recruited via e-mail and by announcements made in departmental courses. Out of the 76 first-year MPH students, 17 students volunteered to participate; among the 85 second-year students, a total of 10 students participated.
The external evaluator on the project facilitated all focus groups. Three additional graduate students who had no involvement with the course implementation served as note takers. All focus groups were conducted on the University campus during the business day or early evening. All participants provided informed consent and completed an anonymous demographics questionnaire prior to the focus group. Basic descriptive statistics from the questionnaire were examined using SPSS.
The focus group guides included questions about students’ engagement with course material, the role that the teams or interactions with other peers played in their learning process, and recommendations for the course for future years. The focus group conversations were digitally recorded, transcribed verbatim, checked for accuracy and completeness, and deidentified for analysis. All transcripts were coded and analyzed in MAXQDA software (Version 12, VERBI Software, Berlin, Germany). Two investigators developed the codebook using deductive codes that corresponded to the sections of the focus group guide and inductive codes based on potential emergent themes identified by the focus group facilitator and note takers. All focus group transcripts were independently double coded to ensure reliability. A thematic analysis approach was used for analyzing the transcripts because it is a flexible method for identifying, analyzing, and describing patterns, or themes, within the data (Braun & Clarke, 2006). During and after coding, the researchers wrote memos about key codes and the relationships between these codes. Key codes were grouped together to form themes that represented patterns of responses. These themes were reviewed to ensure that the coded responses included in each were coherent and distinct from other themes (Braun & Clarke, 2006). The investigators focused on the themes related to each instructional component and the emerging themes about working in teams. Where possible, the responses of students who experienced TBL were compared with those of students who took the lecture-based class.
Results
Of the 45 students who participated in the survey, 42.2% were White/Caucasian and 48.9% identified as minority. Participants’ mean age was 24.80 years (SD = 1.48) and the vast majority were women (97.8%). There were no significant differences in student responses by instructor.
The majority of focus group participants were women (94.1% among first years; 80.0% among second years) and were in their mid-20s (mean age = 25.5, SD = 5.8 among first years; mean age = 25.2, SD = 1.8 among second years). Overall, more participants identified as minority (58.9% among first years; 50.0% among second years) than White/Caucasian (35.3% among first years; 50.0% among second years). Demographics of students who participated in either the survey and/or the focus groups largely align with the department’s overall student characteristics; however, slightly more females completed the survey compared to males and more minority students participated in focus groups compared to nonminority students. Findings from both the survey and focus groups are presented below for each outcome of interest and core components of TBL.
Engagement With Course Material and Perceived Effectiveness of TBL Components
Preclass Preparation
Overall, students reported reading more compared to other lecture-based courses (37.8% read a little more; 51.1% consistently read all assigned material). Reading was considered “very helpful” for understanding course material by a majority of participants (66.7%). Moreover, accountability to their team was identified by nearly half of respondents (48.9%) as the most important factor motivating them to complete reading for each class.
Similarly, from the focus groups, students in the TBL class “read a little bit deeper” compared to other classes, took detailed notes, and then reviewed notes before class. Students knew that they had to read in order to do well on the quizzes; they prioritized Research Methods reading over other classes where the reading was not always used in class. Students appreciated knowing that their preclass preparation would be worthwhile. One student said, “I know I’m gonna need to know this and I know I’m gonna need to talk about it and so it’s worth the investment of my time to read it.” In contrast, students in the lecture-based class did not consistently read before class. A couple of students said that reading the book was helpful in preparing for the final exam but that it was not necessary to read on a weekly basis.
Individual Readiness Assurance Tests
Overall, the iRATs increased student accountability, but the students did not think these assessments facilitated their learning. Students in the focus groups mentioned that the tests were stressful and that some of the questions were too detailed or ambiguously worded. Some students would have liked a review of the material or the lecture before taking the test. When the iRATs were discontinued in response to student feedback, most students were more satisfied with the course. Others, though fewer, felt that the iRAT was a valuable part of the learning process and could have been kept if the administration and/or grading was changed (e.g., administering the iRATs every other week, not counting all of them, or using them for extra credit points). One student noted that she/he read less in depth after the iRATs were discontinued. Survey results confirm focus group findings with only 13.3% of students suggesting that the iRATs were very helpful, while 28.9% reported that they were not helpful at all (see Figure 1).
Contribution of team-based learning components to students’ learning in a master’s-level behavioral science research methods course (N = 45).
Team Readiness Assurance Tests
Students found it helpful to apply the preassigned reading material to these assessments and discuss their understanding of the content with their teammates. One student said, “It challenged me to know if I could apply what I just learned or if I could explain it to somebody else, and that proved to me that I learned something from the readings.” Another student described how the tRAT helped him/her understand the questions he/she answered incorrectly on the iRAT. The structure of the tRAT, with the scratch-off form, made teams communicate and find consensus about the answers. Students found value in learning how to be persuasive, defend their positions, and explain concepts to their teammates. A few students also felt that sometimes there was not sufficient time to discuss everyone’s answers. One student described the team process,
It was good to talk through the harder problems . . . I think on every single tRAT there was at least one question where my group was split or just one person was like super confident and everyone else didn’t think it was right. Um, and so we really had to talk it through because yeah, our team was really big on consensus—we really had to agree before we submitted an answer.
The survey results support the focus group findings. Over 75% of students indicated that tRATs (77.8%) were “very helpful” in contributing to their learning (see Figure 1).
Lectures
Survey results indicate that 62.2% of respondents perceived the short lectures in the TBL format to be helpful for understanding course material. One student said, “We’ve all done lectures before . . . so it was like super refreshing to not sit and do the lecture for hours before doing something different in class.” Students also found it beneficial when other students asked questions and there was discussion about the content during the lectures.
Some of the students felt that the lectures served to synthesize the material whereas others thought that this component reviewed only the readings they had completed before class and were thus not particularly helpful. Students especially valued real-world examples, including from the professor’s experience. A couple of students viewed the lecture as an important opportunity to learn from the instructor and, therefore, would have liked more time for lecture. For example, one student said,
I think what I said in my midterm evaluation was like, oh we’re having this much time spent on testing and then we have a little abbreviated lecture and then we have a team activity, like when do I actually get to learn from the professor?
Students who took the lecture-based Research Methods course wanted more meaningful interactive learning activities, such as one student who said, “And more than just like ‘okay turn your chairs around, you’re gonna talk to the person next to you about this one question we have on the board.’ Like more interactive than that I think would be helpful.” Students also mentioned being frustrated with other classes that had long lectures, “It’s like two and half hours most of the time of just like straight lecture, and I just like, I can’t handle this.”
Team Assignments
Students thought that the applied nature of the team assignments was effective. One student said, “It was like the, you know, higher level thinking, ‘cause you’re now taking the ideas and you synthesize it all.” They also found value in considering their teammates’ different perspectives and working together to develop their answers. One student described,
. . . especially when it came to those team assignments where sometimes you were like, “Oh, I wanna just do this idea or do it this way, and I think I have my ideas all set,” but then you have to like, incorporate other people’s ideas or figure out who was—not right, but who . . . was going along the right lines. Um, so I think it was kinda more realistic to what it might feel like if you were doing Methods in a job or in real life.
When approaching the team assignment, several teams adopted a “divide and conquer” approach, where they assigned pieces to each teammate. A couple of teams worked together on the whole assignment. Some teams were efficient, worked together well, and always finished on time. Other teams had trouble finishing the assignments in class, which was frustrating to them. One student said,
But I mean, I definitely saw us change and get more efficient with things, but we were still never really able to get to the point where we were like, breezing through everything and finishing in time. So that was frustrating, um, to deal with . . .
Students recommended having team assignments that can be easily completed within the timeframe of the class.
Most teams used Google Docs when working on the team assignment, which allowed everyone to work on one document and then add to what the other group members had completed. A drawback to this approach was that it essentially required students to have a laptop. One student who did not have a laptop described how she/he could not directly contribute to the document and, therefore, had to take on other roles, such as initiating brainstorming and proofreading the final document.
In the survey, 75% of students felt that their team assignments were “very helpful” for their understanding of the class material (see Figure 1). No respondents indicated that these elements were unhelpful. Additionally, approximately 80% of participants felt that team discussions elicited by either the tRAT or team assignments were very helpful in getting them to think more deeply about the concepts.
Experience Working in Teams
Three major themes emerged related to working in teams: the role of teamwork in students’ learning, the roles team members took on within their teams, and team dynamics.
Role of Teamwork in Learning
Students described several benefits of working in teams, including being able to discuss concepts with their teammates and bounce ideas off each other, teach each other, and hear different perspectives. For example, one student described, “If you weren’t sure about a subject or answer, um, you could feed off of your team. And if you were sure about an answer, it helped you practice, like, explaining it to other people.” Students felt accountable to their teammates, which prompted them to do the readings, prepare for class, and engage in the class to a greater extent compared to non-TBL classes. Teammates counted on each other to come to class prepared. One student said, “I have to do this reading because . . . I don’t want to be that person . . . I don’t want to be the person that’s not contributing.” One drawback of TBL identified in the focus groups was that while students got to know their teammates very well, they did not interact much with the other students in the class.
Team Member Roles
Students discussed two different types of roles that they took on in their teams. The first were concrete roles, such as note taker or the person who scratched off the tRAT answer, that teammates could assume or assign each other in order to complete their work. Some teams would switch the roles around, while members of other teams would keep the same roles. Another type of role was less concrete yet more about leadership, including being a motivator, keeping the team on task, and making decisions. For these roles, some students talked about needing to take on roles that they did not usually assume in teams and that pushed them out of their comfort zone. As one student described,
In my group, it was like 4 out of the 5 of us would want to be the leader, so we kind of figured out that we’d take turns . . . I think it helped me build on my leadership skills, learn how to be a follower, how to be a leader.
Team Dynamics
Some teams “gelled” immediately and worked well together throughout the semester. Other teams had shifting dynamics, where they had difficulties early on but worked together better as the semester progressed. One particular point of tension for some teams involved time management when completing team assignments. As members learned how to work together more efficiently, team dynamics improved. As one student said, “You learn like the flow of how people operate and um, so it was nice to see like your group become a team, like, become a cohesive unit.” Teammates learned a lot about each other over the semester, including their personalities, motivations for learning (i.e., good grades or mastery of content), communication styles, and learning styles. Students talked about needing to trust one another, as well as the process of developing comfort with one another to be able to voice their opinions. This process was easier for some teams than others, depending on the confidence level of the teammates in sharing their thoughts, as well as the blend of personalities and working styles.
TBL allowed students to develop relationships and ways of working together that were different from other classes. Through comparisons of the responses about TBL and other non-TBL classes, it became clear that students view teamwork as distinct from traditional group work. Survey responses suggest that given the choice, 68.9% would prefer to remain with the same team throughout the semester, 20% had no preference, and 11.1% would prefer a different team over the course of the semester. Teamwork involves the processes and relationships described above, whereas group work tends to have a shorter timeframe that does not allow students to get to know each other as well or work together as effectively. As one student described:
There were, group projects in all of the classes I’ve had. Um, but usually in the ones, so the ones that weren’t team-based learning . . . you didn’t really get to know your team as well, you didn’t know how they worked. But with the team-based learning, you knew your team in advance, so you already got the, um, you already became acquaintances and friends and then we, we knew our strengths and weaknesses. So, I think we were better able to structure our time in, uh, completing the assignments.
Students appreciated the opportunity to practice working with a team because, as future public health professionals, they are likely going to be working in teams when they get a job. One student noted,
It felt more like a little bit like real life, ‘cause I think if you were working on a project and you were doing Methods, you wouldn’t be doing them by yourself, and so you kinda have to work with different people’s perspectives, and also learn from each other and help each other.
Students offered several suggestions for future implementation of TBL to support teams as they begin working together. The first suggestion was to have students complete a personality or working style inventory before or at the beginning of the course so that each teammate would be aware of their own style. Teams could discuss the teammates’ styles and how they fit together. Another suggestion was to include an introduction or tips for effectively working in groups.
Perceived Mastery of Learning Outcomes
Survey results suggest that overall, students perceived that they had mastered the course learning objectives (see Table 2). The majority of students reported being “very confident” in seven of the eight learning outcomes. About half (53%) were only “somewhat confident” and 13% were “not confident” in choosing appropriate measures for a research study. In focus groups, students recognized that the TBL course provided them with a foundational understanding of research methods that they could apply to other classes. One student said, “But the class was so helpful, it taught me a lot about methods even though I thought I knew research methods.” They felt confident in using the content covered in the class.
Students’ Perceived Confidence in Mastery of Course Learning Objectives in a Master’s-Level Behavioral Science Research Methods Course That Used Team-Based Learning (N = 45).
Discussion
This study sought to extend current research by conducting a mixed-methods evaluation of TBL to enhance student learning in a graduate public health course. Consistent with prior research, results of this study suggest that students engaged consistently and more deeply with the course material. Specifically, students in the TBL course were motivated to complete readings prior to class knowing that these were directly linked to their ability to demonstrate mastery of the material on weekly tests and team assignments. Students took ownership of their learning rather than remain passive consumers of information delivered through a lecture. This may have influenced students’ perception that the TBL format was effective in conveying principles of behavioral science research methods in a manner that helped students master most of the learning outcomes.
Beyond showing increased engagement in and perceived effectiveness of the TBL course, perhaps the most meaningful finding that augments previous research relates to students’ reports that working in teams within the context of structured TBL activities was perceived as an effective and valuable skill. In fact, accountability to their team was identified by many students as a motivating force to prepare for and engage in the class. TBL students reported that their learning of the course material was enhanced through team discussions and the opportunity to teach each other. To arrive at this point of reciprocal teaching and learning, students had to navigate team dynamics and tensions that arise given various personalities, motivations for learning, and styles of communication and learning. This level of engagement in the team process went beyond what is involved with group work, a typical feature of lecture-based courses.
While it can be argued that these themes are not surprising within the structure of TBL, which by definition creates an environment that compels students to become active learners (Sisk, 2011), one additional theme emerged that is not necessarily prescribed by the TBL structure. As a strategy to navigate team dynamics while staying focused on the shared goals (i.e., getting a good grade and/or mastering the material), students assumed two types of roles within their team: logistics and process. Logistical roles were either assigned to team members for the duration for the semester or switched around from week to week. Such roles included taking notes, submitting the assignment on behalf of the team, or scratching off the tRAT answer form. Process roles included higher level roles that emerged from the team dynamic and included leadership and decision making. In addition to keeping the team motivated, on task, and on time, such roles created an environment that allowed team members to consider other points of view and approaches to problem solving and to practice consensus building. Overwhelmingly, students reported that they preferred to remain with the same team throughout the semester, suggesting that in general whatever challenges and tensions they encountered were offset by gaining meaningful transferrable process skills. These findings on effectiveness in working in teams are encouraging, particularly in light of the heightened focus on preparing public health students to be successful communicators and collaborative team members in varied interdisciplinary and interprofessional settings.
Results of the current study show great promise for the implementation of TBL in graduate public health courses. However, some areas of caution are worth noting based on the current TBL implementation as well as prior assessments of TBL in other disciplines. First, not all public health courses may lend themselves to the structure of TBL. For example, one or two credit hours courses may not allow sufficient time for students to progress through all components of TBL. While these could be broken up over multiple class sessions, discontinuity of context could compromise the learning experience. Alternatively, instructors may wish to make modification to the TBL structure, although such adaptations should be made with caution to avoid detracting from the core components that contribute to an active learning environment. Second, faculty training and accurate delivery of TBL or any new active learning pedagogy are crucial in achieving desirable learning outcomes. Previous research suggests that active learning models may not be effective when their implementation is not grounded in the instructors’ understanding of teaching and learning principles (Andrews, Leonard, Colgrove, & Kalinowski, 2011). Therefore, faculty development is important in creating a cadre of instructors equipped to implement as well as evaluate TBL and other active learning models with maximum impact. Third, to optimize students’ teamwork experience, incorporating some instruction at the beginning of the course on navigating team dynamics, communication styles, and general tips for working effectively in teams may even further enhance students’ team learning experience. This may be particularly important for programs that matriculate younger graduate students with limited work experience to help them become more self-aware as they enter into teamwork (Hartman & McCambridge, 2011). Last, as public health schools and programs begin to expand their pedagogies to include active learning models such as TBL, institutional support of research and evaluations to better understand student experiences with such models are worthwhile. Mixed-methods evaluations, for example, yield much more in-depth, actionable data than typical standard course evaluations. Consequently, public health graduate education could draw from an increasingly larger and more rigorous body of literature to inform the scholarship of teaching and learning similar to that available to our colleagues in the medical education field.
Limitations
Regarding the implementation of TBL, several departures from strict TBL structure proposed by Michaelsen et al. (2004) must be acknowledged. For example, after collecting mid-semester feedback from students, the instructors decided to eliminate iRATs for the second half of the semester while retaining the tRAT. Overwhelmingly, students’ sentiments were that the iRAT was superfluous by causing added stress about the iRAT grade and detracting from having sufficient time to engage in a more meaningful discussion during the tRAT. Additionally, teams did not always convene at the end of class for simultaneous reporting, often due to time constraints.
Several methodological limitations of our evaluation are also worth noting. First, while this study included a well-defined comparison group and also examined team effectiveness in the learning process, components lacking in many previous TBL studies, our study lacked the rigor of an experimental research design. Consequently, our conclusions must be interpreted with caution as we cannot draw causal inferences regarding the direct impact of TBL components on student learning. Second, there was no pretest assessment of learning outcomes as we felt that graduate students in their first semester of a master’s program may not necessarily be sufficiently equipped to judge their level of mastery of learning outcomes in a new content area. Third, the current study lacked an objective educational performance outcome such as grades or other standardized scores. Due to the small sample size combined with the restricted variability in grades that characterizes graduate-level courses (Rojstaczer, 2008), it is unlikely that grade differences would have been detected. Fourth, the conclusions from the current study are attenuated by possible bias due to relatively low response rate to the quantitative survey. Additionally, it is possible that students with favorable views toward TBL self-selected into the focus groups. However, since no identifying information was collected from focus group participants, links to the survey were not possible to examine correlations with quantitative responses. Last, the majority of respondents to both the survey and focus groups identified as female, which may represent another source of possible bias. However, this overrepresentation of females mirrors the overall female-to-male ratio typical of the department of Behavioral Sciences and Health Education at the Rollins School of Public Health.
Conclusion
This implementation and mixed-methods evaluation of TBL suggests that the TBL format engages MPH students in course content, facilitates learning, and has strong potential for preparing students to collaboratively work in diverse teams. TBL may be a useful pedagogical approach for courses that aim to address discipline-specific competencies while also instilling cross-cutting skills related to teamwork. In particular, TBL may help students recognize the value of team processes, develop roles to efficiently manage tasks, and navigate complex team dynamics. Considering that such skills are foundational to the work of any public health professional, TBL may be particularly salient for graduate public health education.
Footnotes
Acknowledgements
We thank all our students for contributing to thoughtful and energetic learning environments as well as for their participation in this study.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: National Institutes of Health/National Institute of General Medical Sciences Institutional Research and Academic Career Development Award (K12 GM00680-05).