Abstract
In line with the complex modern health care system and the increasing importance of interprofessional teams, a powerful strategy to facilitate the acquisition of essential teamwork skills and expose students to complex decision-making processes is learning in teams. The purpose of our study was to obtain empirical evidence of superior decision-making by teams versus individuals in two simulated decision-making exercises conducted 4 months apart. We collected quantitative data from three cohorts of Master of Public Health students to determine if teams make better decisions than individuals (“team effect”) between September and January. Students completed simulated emergency survival exercises requiring them to make correct decisions individually and then as teams. Decision quality was determined by comparison to survival experts’ decisions. We calculated the “team effect” as the gain or loss of mean individual versus group scores across 10 learning teams per cohort for fall and winter exercises. All three cohorts had a consistently small average team effect in September and a much larger team effect in January. Our study showed consistent improvements in decision-making after students had worked in teams for 4 months. Overall, this study demonstrates the potential benefit of incorporating team learning into a public health curriculum and the importance of strategies to teach teamwork in health education. Using simulation in health education and promoting team learning activities can help prepare students for interprofessional collaboration, a part of the demanding public health landscape. These results might help convince students of the benefits of teamwork, facilitate collaborative decision-making, and enhance the learning experience.
Introduction
Several problems in public health are multidimensional and involve decisions made by people with diverse skillsets and backgrounds working in teams. The effectiveness of teamwork has been thoroughly studied (Schmutz et al., 2019). For teams to be effective, members need to communicate with each other and engage in shared decision-making. However, as members from diverse professions work together, challenges related to reaching consensus and sharing power and leadership can potentially emerge and affect team performance (Nancarrow et al., 2013). Teams make decisions, regardless of their context, through interprofessional exchange and assessment of the compulsory information (Halvorsen & Sarangi, 2015). The importance of a balanced decision-making process in healthcare has been well established in the literature, especially when delivering comprehensive care to complex patients (Weinberger et al., 2015). Factors that affect the complexity of decision-making in healthcare include the conflicting goals of health professionals, confusing and unnecessary information, and lack of team coordination (Islam et al., 2015). To achieve effective team decision-making, all members of a team should have the relevant information necessary to inform these decisions.
Team performance can also be impacted by several factors related to team composition, for instance, power relations and imbalances, as well as hierarchy issues (Reeves & Lewin, 2004). Therefore, there are long-term benefits of working in teams and advantages of collaboration for students in the health professions education where achieving effectiveness while working in teams can be difficult to accomplish (Reeves et al., 2010). Team members’ individual characteristics can also affect the quality of teamwork, and cross-cultural interactions and diversity may influence particular dynamics of teamwork (Kumra et al., 2020). Despite the inevitable possibility of ethnocentrism and, at times racism, in culturally diverse healthcare teams, incorporating reflexivity can be a fruitful strategy to interrogate team members assumptions and work toward collective decisions (Cook & Brunton, 2018). In terms of sex/gender inclusion in teams, according to Niler et al. (2020), women representation in STEM teams is essential for their sense of identification, leading to higher chance of team effectiveness. These authors contend that profession-related negative stereotypes also had an impact on teamwork dynamics and that this issue can be mitigated by allowing more women to be present on the team. In addition, enabling minority members to be on teams with other members whom they can identify with, enhances psychological attachment and level of confidence on their team.
There is growing evidence supporting team-focused learning in health professions education, but additional empirical evidence would help establish its significance (Reimschisel et al., 2017). The effectiveness of team-focused learning exercises has commonly been measured using test results or self-reported measures like course evaluations or questionnaires (Reimschisel et al., 2017). Despite the positive effects of team-focused learning on students’ academic performance, assessments that measure team effectiveness must be approached with caution, especially when considering multiple types of classroom environments and other factors that might impact student engagement and participation (Gullo et al., 2015). Perhaps though, using simulated exercises, it is possible to provide tangible, empirical evidence supporting learning in teams. Qualitative studies have shown that simulated decision-making exercises can represent dynamic decision-making in a business education context (Anibaba & Akaighe, 2018). Students in the health professions also need to acquire the fundamental knowledge and skills to effectively work in teams, eventually applying them in real work contexts and clinical practice (Rossler et al., 2017). The use of simulation in health education facilitates the acquisition of valuable knowledge and skills that prepare students for actual clinical practice (Boet et al., 2014). We expect that simulated exercises can mirror the nuanced experiences of a health professional required to coordinate and adapt to challenging scenarios.
There are challenges to quantifying the degree of improvement in the quality of the same decisions by people acting individually versus in groups. One requirement is validly defining decision “quality,” which requires a validated criterion with clearly superior and inferior outcomes. A second requirement is that team members make their individual decisions independently before they come together and collectively reach a consensus decision about the same problem. A third requirement to quantify “team effect” over time is to conduct at least two different decision-making exercises, with enough time between them for team members to learn to work together. If the first exercise is done very soon after the group is formed, it tests the adage that “many heads are better than one” from the start of the team interactions. If a second exercise with a different problem is done after sufficient time has elapsed, an improvement in decision quality on the second exercise suggests the importance of practice to allow a team to learn to communicate and work together. The requirement for different decision-making problems to be used is important to prevent differences from being a simple memory effect. In practical terms, it is difficult to meet these requirements by analyzing decisions made in real-world public health work. Therefore, there is probable value in using simulation exercises to assess the effectiveness of learning in teams in a health professions educational context.
Masters of Public Health students are future health leaders requiring expertise to adapt and coordinate responses during health emergencies, particularly mobilizing their efforts alongside other disciplinary teams. However, there is limited evidence in the literature addressing crisis-responsiveness and simulation learning in public health curriculums. Do teams make better decisions than individuals? If so, how large is this “team effect” and does this team effect grow over time? The purpose of this study was to obtain empirical evidence of superior decision-making by teams versus individuals by analyzing improvements in decision-making in two survival simulations conducted 4 months apart in three cohorts of students.
Methods
Our Master of Public Health (MPH) program offers individuals the knowledge and skills necessary to become professional leaders. This unique 12-month program is hosted by the School of Medicine and Dentistry, supported by seven other faculties on campus and accredited by the Council on Education for Public Health (CEPH). The program places emphasis on group work in learning teams (LTs). LTs of approximately six students, are formed by program faculty and staff in September and remain for the entire year. Students meet up in their LTs everyday (typically between 1 and 4 pm after their morning classes) and collaborate on activities between all their courses. For the entire semester, each team had access to a room with a whiteboard, projector, and internet connectivity to simulate a typical work setting. LTs were deliberately created to balance personal characteristics and be as diverse as possible in terms of gender, country of origin, race/ethnicity, educational/professional background, and years of experience. Demographic details for (1) age, (2) gender, (3) educational background, and (4) citizenship status were recorded for all the graduating cohorts between 2018 and 2020 (Table 1). Ethics approval was obtained from Western Research Ethics Board.
Demographic Details for (1) Age, (2) Gender, (3) Educational Background, and (4) Citizenship Status Was Recorded for All the Graduating Cohorts Between 2018 and 2020.
For three successive cohorts, in early September and again in January, 10 LTs (~6 person) completed two different simulated survival exercises (Human Synergistics, 2014, St. Mary’s, OA, Canada). The simulations were conducted outside of normal class hours, but during weekly lunch seminars. These exercises required students to imagine themselves in an emergency scenario in which they must quickly make decisions they think will improve their chances of survival. Specifically, participants priority-ranked several objects at their disposal in terms of their usefulness for survival. Participants first ranked the objects individually with no discussion. Second, the group discussed the rationale for their individual ranks and reached a consensus on a new set of group ranks. The individual and group ranks were then compared to a “criterion”—ranks determined by people with relevant survival expertise. In September, toward the end of their first week together, students were asked to imagine being with their LT in a cabin in a remote area of the Australian outback with a rapidly approaching bushfire. Participants were told they have access to 12 items in working condition, that the members of their LT are the actual people involved, and that the group has decided to stick together. After reading the scenario, the participants spent about 15 minutes individually ranking the 12 items in terms of their importance for surviving the fire. Next, the team spent about 30 minutes discussing their individual ranks, their survival strategies, and assumptions, and agreed on a new set of ranks.
Decision-making quality was measured by subtracting both the mean of the individuals’ ranks, and the team’s ranks from those provided by survival experts (Human Synergistics, 2014, St. Mary’s, OA, Canada). For the bushfire simulation, the experts were staff members from the state fire authority with over two decades of experience fighting fires in that environment. The comprehensive background of the experts, together with the rationale they provided for their choices, is the basis for assuming their ranks are valid criteria for decision-making quality.
After subtraction from the expert ranks, the 12 absolute individual and team differences (negative signs are ignored) were summed to yield total individual and team scores. A score of zero represented perfect agreement with the experts and the worst possible summary score for individuals or teams was 72: ∑[ 12 − 1 + 11 − 2 + 10 − 3 + . . . |3 − 10| + |2 − 11| + |1 − 12|]. Please see Table 2 for an example of these calculations.
Quantifying Decision-Making: Calculation of a Hypothetical Team Member’s Total Score, a Team’s Total Score, and the Gain/Loss Score.
Notes. Best possible total individual or team score = 0 (perfect agreement with experts); Worst possible total individual or team score, fall exercise = 72; Best possible “Team effect” = 72 − 0 = 72; Worst possible “Team effect” = 0 − 72 = −72.
The experts’ ranks were based on their view that it would be safer to hunker down until the fast-moving fire passed than to try to outrun it. This initial decision affected the ranks of the individual items. For example, cell phones were ranked low (9/12) by the experts because the reception is likely to be poor, the circuits overloaded, and the fire moving too quickly for outside help to arrive in time. As a result, participants ranking the cell phone highest in priority (1/12) would add 9 − 1 = 8 to their individual scores. However, if group discussion led to the decision that a cell phone might be of little use and the team gave it a low consensus rank, the team’s decision-making score would be lower (better) than the average of their individual scores.
To quantify the “team effect” in decision-making, the team score was subtracted from the average of the individuals’ scores to produce a “gain/loss” score. If the team makes better decisions than they did as individuals, the gain/loss is positive, up to a maximum of 72. It is also possible for a team’s collective decision-making to be worse than the average of the individual decisions as indicated by a negative gain/loss score with a maximum of −72 (Table 2).
To prevent a memory effect, a different survival exercise was conducted in January. The emergency scenario was a yacht sinking at sea in the middle of the night. Because there were 15 salvaged objects to rank, the worst possible score is 112. The expert who ranked the usefulness of the objects for the sea survival exercise was an emergency survival and rescue specialist and former director of an emergency survival and rescue center.
To adjust the gain/loss scores for the different maximum gain/loss scores between fall and winter (i.e., 72 vs. 112) and enable a direct comparison between them, we converted gain/loss scores using Cohen’s Effect Size d (Cohen, 1988). We treated the mean individual scores in each team as the base condition, and calculated the standard deviation of those individual scores using the formula for samples with n − 1 in the denominator. Cohen’s d expresses the gain/loss of the team scores in standard deviation units of the individual scores (i.e., the “team effect” was the team score minus the mean individual score divided by the standard deviation of the individual scores). Cohen’s familiar qualitative descriptors for d values of 0.2, 0.5, and 0.8 represent small, medium, and large effects, respectively.
Findings
Table 3 shows mean individual and team survival exercise scores, gain/loss, and d values, for the fall and winter exercises across three cohorts of MPH students. All three cohorts had very similar individual mean scores in the fall bushfire exercise, ranging from 55.4 to 58.6. Similarly, after LT members had known each other for only a few days, all three cohorts showed very small average improvements in team decision-making, with gain/loss scores ranging from −0.19 to 1.1 and d values ranging from −0.03 to 0.26. Interestingly, the 2018 cohort made slightly worse decisions in teams than they did individually as indicated by the negative gain/loss and d values. The average d across all cohorts was 0.14, or “very small” improvement in decision quality in newly formed teams over individuals.
Individual and Team Survival Exercise Scores: Means, Gain/Loss, and Effect Size d, by Semester, by Cohort, Sample: 10 Learning Teams of Six Persons Each.
In the winter exercise, all three cohorts made consistently poor individual decisions, with mean individual scores ranging from 69.5 to 76.4. However, after working together as teams for 4 months, all three cohorts showed large improvements in the quality of their team decisions, with gain/loss scores ranging from 17.6 to 21.1 and d values ranging from 1.61 to 2.3. Table 4 reveals the ranges of performance across each cohort’s 10 LTs. In the fall, five or six LTs had better team than individual scores; by the winter, 9 or 10 LTs were making better team decisions. The range of gain/loss scores shows the size of the differences between individual and team decisions across the 10 LTs. Interestingly, the values of −12.3 and 10.6 in the fall 2018 cohort indicate that one team had a larger loss between individual and team decision making than was experienced by the team with the largest improvement. This partly explains how the class made team decisions that were 19% worse than their individual decisions. The other two cohorts showed larger gains than losses, with mean improvements of 2.16 and 6.5%. In the winter exercise, one LT in the 2019 cohort had a small worsening of team decision-making with a loss score of −0.67. The other two cohorts had substantially more gains than losses and the class had overall improvements in decision quality ranging from 25.1% to 33.2%.
Number of Learning Teams With Better Decision-Making Score Than Mean Individual Scores, Lowest and Highest Team Gain in Scores, and Mean Improvement in Average Score, by Semester, by Cohort. Sample: 10 Learning Teams of Six Persons Each.
Discussion
Decision Making Outcomes
Across 30 newly formed LTs of ~6 MPH students each, a simulated survival exercise consistently showed a very small improvement, and one very small worsening, between team versus individual decision-making. While teams outperformed individuals on average, the overall effect size in newly formed teams was very small at d = 0.14. However, after 4 months of learning in a team, these same LTs showed a substantial improvement in team decision making and large effect sizes of over 1.5 standard deviations of the individuals’ mean decision quality.
This study provides evidence of the potential benefits of incorporating learning through teams and simulation into public health curriculum and the importance of strategies to encourage teamwork in health education. This is congruent with the evidence in the literature that suggests incorporating team learning in the curriculum helps students perform better academically and provides opportunities to acquire teamwork knowledge and skills (Guadagnoli & Miller, 2016). For teams to function, displaying expertise at an individual level is not enough; teamwork competencies need to be developed (Lacerenza et al., 2018). Our statistical effect sizes are compatible with the qualitative study by Burgess et al. (2019), suggesting that team-focused learning encourages cooperative involvement and shared comprehension. In addition, factors pertaining to the environment, such as allocating enough time and space for team building, are also crucial aspects of teamwork (Oandasan et al., 2009).
Long-Term Teamwork Effects
As this study’s results showed, simulation exercises can be used to demonstrate the benefits of collaborative decision making. These exercises also provide feedback and reinforcement to students as they learn to engage in a meaningful decision-making process in a safe place, before being exposed to a more competitive and fast-paced work environment. In addition, simulation is a powerful technique that can help to clarify roles and responsibilities among team members in a safe environment (Weller et al., 2014). However, questions related to the long-term effects of these educational interventions remain. For instance, can students retain and apply teamwork skills and knowledge long-term? What are the benefits when students join the workforce and need to work effectively in teams?
Addressing concerns that might emerge in professional practice can also inform and support early interventions for health profession students working in teams. A study among primary care providers to understand the learning needs of team members indicated several key aspects for effective teamwork, such as respecting other’s roles and professional expertise, regarding everyone as equally important, recognizing the importance of the setting, and communication, deemed as the most important aspect (Sargeant et al., 2008). Addressing challenges that might emerge in real-life contexts can help inform educational interventions and serve as a guide to teach teamwork and improve decision-making processes. Furthermore, taking into consideration other potential challenges, such as team dysfunction and conflict management, can guide the design of simulation exercises in a way that prepares students for the demands of the current health care system and public health landscape.
Our study demonstrated better team versus individual decision-making after the teams had been working together for 4 months; however, more research is needed to determine whether those improvements are specifically linked to the teams they are in and their time working together. Ultimately, we believe the teamwork skills that these students developed in our program will benefit their future team interactions. Anecdotally, we know this to be true based on many conversations with employers as well as student alumni feedback on their learned skills in the workforce. While it is possible that any team-focused activities during health professions education will benefit the interprofessional teams, it seems the dedicated and longitudinal team time spent during our program (i.e., specific teams working together for a while) provided additional and augmented benefit in terms of decision-making and team-based skills. The implication for healthcare practice and education is that to effectively reap the benefits demonstrated by this study, teams may benefit from a focused intervention, for example, when they are first formed.
Limitations
There were some limitations in the study’s design. Because the survival activities are routinely done in each year’s MPH class, it was not possible to include a separate control group. However, because the members of each team remained constant from fall to winter, each team acted as its own control over the 4 months. Although identical exercises were used each fall and winter across the three cohorts, the researchers could not control systematic differences in scenarios between the fall and winter exercises. It is possible that the winter exercise was simply more amenable to a team approach, although the teams had to reach consensus on more item rankings at the same time in the winter than the fall exercise. We were unable to study characteristics of any individuals or teams that might explain why teams occasionally made worse decisions than their individual members, nor were we able to analyze reasons for the range of gain/loss scores across teams or between cohorts. We asked students with prior experience with either survival exercise to not participate. However, it is possible that by chance some students had generalizable survival knowledge that resulted in good individual scores, but these students were not able to convince their teammates. The generalizability may be affected by the fact that our exercises were conducted in person, meaning any virtual training programs may require differential adaptation to supplement learning. Furthermore, simulated survival scenarios are inherently artificial, and assume that experts’ rankings are a valid criterion for decision making quality. The validity of the gain/loss score depends on the experts’ ranks for the items and their supporting rationale. It is possible that an individual or a team foresaw an innovative use for an item not recognized by the experts. We also acknowledge that our study design did not allow us to determine whether the decision-making improvements we reported would translate for students working with other members in a team or different teamwork environments.
Future Directions
Several aspects of team learning, and simulation exercises can be improved in future research, such as qualitative observations of how individual behaviors and leadership styles affect the dynamics of decision-making. However, questions regarding long-term retention of teamwork skills and their application at the workplace remain complex and multidimensional. Despite the importance of promoting the development of teamwork skills early in academic settings being well-established (Bertrand & Slovensky, 2020), predicting long-term effects of simulation exercises, team building, and decision-making is difficult. Therefore, future research should focus on longitudinal studies involving participant follow-up as students enter the workforce.
Team dysfunction is another important aspect that needs to be further explored, although the ethics of the deliberate introduction of dysfunction in teams would need careful consideration. Addressing whether different levels of dysfunction lead to the same result in terms of skills and knowledge acquisition is necessary to understand team dynamics better, as well as planning for future simulation exercises. Allowing a certain degree of conflict can help the team to be successful and can potentially lead to some types of creative performance (Yong et al., 2014); therefore, future research should also address conflict and teamwork. Finally, given the increasing use of technology in team learning interventions (Brault et al., 2015; Lawlor et al., 2018) and the potential benefits of virtual training programs (Gregory et al., 2020; Liaw et al., 2020), future research should explore the impact of technology when completing virtual simulation exercises.
Conclusions
Given that interprofessional collaboration is a powerful means to address public health issues that affect communities and the need for effective teams, the inclusion of team simulation exercises in MPH programs might help convince skeptical learners of the benefits of teamwork, facilitate, and improve collaborative decision-making and enhance the learning experience. An appreciation for other professions and a deeper understanding of their unique contributions are key when working in an interprofessional team (Brewer & Flavell, 2020). Simulated exercises can represent real-world scenarios and offer empirical support for the advantages of incorporating regular team-facilitated learning activities in the curriculum. During these team learning exercises, students interact with others in meaningful and constructive ways as a forward strategy to promote respect, willingness, and openness to work with other professions (Croker et al., 2015). In our MPH program, we deliberately create LTs to be diverse concerning prior academic preparation, while the confluence of team members with diverse professional backgrounds either facilitated or hindered teamwork. This would encourage team members to acknowledge their own unique experiences, potentially leading to better communication and more positive working relationships (O’Rourke et al., 2018). The peer feedback component of team-focused learning would also facilitate the development of positive interpersonal relationships and improve team function (Lerchenfeldt & Eng, 2019). Therefore, this education intervention was critical to helping students gain an accurate understanding of other professions, to navigate collaboration and their professional roles to work effectively in teams (Sexton & Baessler, 2016). The simulated quantification of improved team performance across semesters is consistent with current qualitative literature. Therefore, researchers should consider further investigations with team learning scenarios in MPH cohorts to foster collaboration between aspiring public health leaders and stakeholders.
Footnotes
Acknowledgements
We would like to acknowledge all the students, staff, and faculty at the Master of Public Health Program.
Authors’ Note
Mark Speechley is now affiliated to Department of Epidemiology & Biostatistics, Schulich SMD, and Schulich Interfaculty Program in Public Health, Western University, London, ON, Canada.
Author contributions
The authors meet the four authorship criteria recommended by the International Committee of Medical Journal Editors. All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by Shannon L. Sibbald, Nicole Campbell, and Mark Speechley. The first draft of the manuscript was written by Shannon L. Sibbald. Cecilia Flores-Sandoval contributed to subsequent drafts of the manuscript, and all authors commented on each draft. All authors read, edited, and approved the final manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Ethical approval
All procedures in this study that involved human participants were in accordance with the ethical standards and principles of the Declaration of Helsinki. Ethics approval was obtained from Western Research Ethics Board, at Western University, London, Ontario, Canada.
Informed Consent
No identifiable data was collected for this study, and no written consent was requested from participants.
