The Process of Team Building Among Content Experts and Methodologists: An Example From an Emergency Medical Services Research Investigation Kick-Off Meeting

Abstract

Background:

Community-engaged research engages the community in a collaborative manner to improve the local setting and has been increasingly applied to health services research. These complex studies require collaboration and team building among clinical experts, methodologists, and community stakeholders, yet few examples of this process have been published.

Aim:

Through a study of community emergency medical services, our aim was to understand team building among content experts and methodologists to advance the framework for adaptive community-engaged research.

Methods:

We collected process field notes and transcripts from the project kick-off meetings and analyzed the data using an immersion/crystallization process, which is an exploratory qualitative analysis characterized by continual review of the data collected. All 25 members of the study’s research team, who are both local to Michigan and reside out-of-state, were invited to attend.

Results:

We describe the process of the kick-off event for this community-engaged study and report major themes: collaborating throughout each stage of the kick-off meeting, building trusting relationships by challenging each other’s perspectives and sharing expertise, and gaining shared experiences of learning and understanding of study content and goals.

Conclusion:

Team-building in community engaged health research requires thoughtful effort, and planned kick-off meeting can be a useful strategy to build a shared vision among content experts and methodologists.

Keywords

team building collaboration community-engaged research emergency medicine emergency medical services

Research is increasingly complex as it addresses more nuanced applied topics and requires engaging multiple researchers across disciplines. Understanding complex health systems that involve multiple disciplines, jurisdictions, governmental organizations, and local residents as community members requires complex research methods. However, complex research is harder for a single person or small group of people to conduct thoroughly, and large-scale collaboration is becoming the norm (Leahey, 2016). Researchers across disciplines are needed to provide expertise in both the content and the use of multiple methods that are often required (Poth, 2018). Team science is defined as collaborative and interdependent research conducted by more than one investigator that has arisen to address modern complex research problems (Cooke & Hilton, 2015). When conducting community-engaged research, team building brings additional consideration, as community stakeholders, local residents of the community who may need health services, and multiple researchers bring vital expertise contextualizing and guiding research studies. To ensure a productive collaboration, team building among content researchers, methodologists, and community stakeholders is critical, yet receives relatively little attention in the literature. A shared vision and understanding of the research project among all team members is needed at the start of a project (versus developing it along the research journey) to be most effective (Israel et al., 1998; Lindau et al., 2011; Wallerstein et al., 2018). This shared vision rests upon transparent communication among team members (Wallerstein & Duran, 2006).

Community-Engaged Research

Community-engaged research, such as community-based participatory research (CBPR), is an approach that engages the community in a collaborative manner to improve the local setting (Israel et al., 2013). Several facets of this definition are worth expanding. First, community stakeholders are critical in understanding community health systems and conducting community engaged research (Mendel et al., 2008). In community-engaged research, and, in more comprehensive CBPR approaches, these stakeholders are involved in all aspects of the research, not merely present but as equal partners in the research process. Second, fully engaging community members in research means involvement in all aspects, including the design, conduct, and dissemination of research (Jones & Wells, 2007). Third, community stakeholders may include members of the community, local organizations, community leaders, and practitioners. Community-engaged research principles have been applied within the context of emergency medicine and emergency medical services for community (Sasson et al., 2013) and public health interventions (e.g., addressing resuscitation barriers) to ensure interventions were relevant to the community context (Dalal et al., 2009; Mendel et al., 2011). In the case of community-based emergency medical services, the community includes the residents who may need public services, police, fire, and ambulance services, bystanders, and local health providers.

Team Building

Although community engaged research can improve the quality of findings and disseminations, a barrier is effective team-building, an aspect of community-engaged research that needs further consideration. Specifically, relatively little literature is available to guide how multiple researchers, along with community stakeholders, work collaboratively to address a research problem. Team science rests on the assumption that bringing multiple investigators and disciplines together produces higher quality inquiry. Broadly, how individuals from different disciplines integrate in a research study can occur at three different levels of cross disciplinary research. The levels, in increasing intensities of integration, are multidisciplinary, interdisciplinary, and transdisciplinary (Stokols et al., 2008). Multidisciplinary processes involve team members working independently and in sequence, from their own perspectives to address a common research problem. Interdisciplinary processes consist of researchers working jointly in an interactive manner and draw from—rather than working entirely from—their own disciplinary perspective to address a common research problem. Finally, transdisciplinary processes are fully integrated as researchers operate from a shared conceptual framework to address a common research problem (Rosenfield, 1992). Transdisciplinary team members synthesize across disciplinary theories, concepts, or research methods, and create new concepts and language. In this project, our goal was to promote and enable transdisciplinary collaboration, including across research and public service disciplines (e.g., Fire, EMS, police, leadership at agencies). We also acknowledge the important role of residents who receive care, their family members, and bystanders, who are essential to understanding a community system of care. As detailed further in this article, to promote transdisciplinary collaboration, we provided time for everyone on the team to hear from a survivor and intentionally structured small groups that were heterogenous in terms of discipline.

Promoting Dialogue in Community-Engaged Research

In the community-engaged research process, investigators have used various strategies including meetings, committees, retreats, and designated personnel to facilitate team and relationship building (Jones et al., 2008). Kick-Off Meetings are a strategy to improve the chance of success (Cooke & Hilton, 2015) and promote dialogue among the research team and community (Mendel et al., 2011). We situate dialogue within Jones's concept of group dynamics as applied to community-engaged research. Group dynamics include the individual, structural, and relational dimensions to encourage equitable partnerships (Wallerstein et al., 2008). Individual dynamics arise because team members bring unique motivations and beliefs, which can be clarified through discussion in a kick off-meeting. Structural dynamics refers to the nature of the team, such as trained health professionals in the community, academic content researchers, and methodologists. Recognizing the discipline diversity of team members is a key first step to design and facilitate discussion and interaction. Finally, relational dynamics refers to the discussion and interaction that occurs through the research process. A kick-off meeting attempts to address these dynamics.

When planning a study, kick-off-meetings also serve key functions ensuring a common understanding of goals. Meetings are also critical throughout the research processes such as to engage researchers and study participants and promote shared learning during a study. Finally, during result interpretation and dissemination phases, meetings are a way to help translate study findings and ensure actionable results.

Researchers conducting community-engaged research have increasingly been using a mixed method approach (DeJonckheere et al., 2019), which involves the collection, analysis, and integration of quantitative and qualitative approaches (Creswell & Plano Clark, 2018). Intersecting mixed methods and community-engaged research is uniquely complex because it involves qualitative, quantitative, mixed methods, community-engaged research, and of course, the content research area. Although much has been written about community-engaged research, mixed methods, and their intersection (Creswell & Plano Clark, 2018), less is known about the complexities of how teams of content experts and methodologists work together in a community-engaged mixed methods research study. A single investigator is unlikely to have all of the required expertise, thus, multiple investigators and community partners, must engage in transdisciplinary team science. To date, though articles are published in other disciplines (e.g., Mendel et al., 2011), no known studies have been published that present the process for conducting a kick-off meeting for the personnel of an emergency medicine or emergency medical services research investigation with the purpose of strengthening the community-engaged research initiative. In particular, we were interested in how methodologists and content experts develop dialogue and shared understanding of research goals.

Our focus here is on how to promote dialogue among methodologists and content experts in conducting community-engaged research. Collaboration is needed to ensure the work of these groups is bridged for the most fruitful research that is methodologically rigorous, relevant to the health-related research question, and contextually meaningful to the local community. The overarching research question in this study was: how did team building occur among content experts and methodologists in an emergency medical services (EMS) research investigation? Specifically, we focus on analysis of transcripts and observations from the kick-off meeting that brought all project participants together, in-person, for the first time.

Background: The EPOC Study

Out-of-hospital cardiac arrest is a common, life-threatening event that is associated with poor patient outcomes. Out-of-hospital cardiac arrest outcomes vary significantly across U.S. communities; however, systems of care factors that increase likelihood of survival have not been well elucidated. Accordingly, the Enhancing Pre-Hospital Outcomes for Cardiac Arrest (EPOC) will lay the foundation for future quality improvement efforts in out-of-hospital cardiac arrest by identifying, understanding, and validating “best practices” adopted by top-survival outcome EMS agencies to address these life-threatening events, as well as potential barriers to their implementation. EPOC intends to accomplish this goal through the following three specific aims: (1) Identify variations in out-of-hospital cardiac arrest survival among EMS agencies and communities; (2) Define system of care “best practices” for survival and; (3) Validate system of care factors associated with survival. This study brought together various stakeholders with content expertise, methodological expertise, and community expertise. Because of the importance of developing an early shared vision among stakeholders, we focus on describing the EPOC Kick-Off Meeting, an event held in EPOC study year 1 to engage a multi-disciplinary team of resuscitation and pre-hospital care experts and methodology experts (Figure 1), as the initial stage in an adaptive community-engaged study design to ensure that study deliverables are usable and effectively operationalized. Through the meeting, we anticipated that community members would also benefit and strengthen their commitment by hearing from researchers dedicated to improving their care, and we anticipated that researchers would benefit from hearing from actual survivors, bystanders, and community first responders to contextualize the research.

Figure 1.
EPOC kick-off meeting attendees. EMS: Emergency medical services; MCA: Medical control authority.

Methods

We conducted a qualitative evaluation of the process of community engagement when initiating a community-engaged research study. Ethical approval was provided for the EPOC study by the Institutional Review Board at the University of Michigan (approval for HUM00135840) and the RAND Corporation. Consent was not required as this study phase was determined exempt from IRB oversight. At the start of the EPOC study in August 2017, we developed the goal of promoting dialogue among methodologists and content experts in conducting community-engaged research. Pursuant to this goal, we began planning for a study year 1 Reception Dinner and Kick-Off Meeting using an iterative and deliberate process that was informed by study meetings and EPOC scientific deliverables from Aims 1 and 2. Operations were facilitated by the EPOC co-principal investigator (MA), project coordinator (AK), and two health services research associates (SF, WN). From August 2017 to February 2018, we identified the Kick-Off Meeting objectives, logistics (including location, budget, and timeline), and design that would facilitate the achievement of this goal. In December 2017, our research team began meeting regularly through a series of in-person and phone-based weekly to biweekly methods meetings, quarterly all-team meetings, and resuscitation or pre-hospital expert meetings. Resuscitation and pre-hospital expert meetings with community content experts who were not members of the research team also began during this time. Content and discussions from these meetings informed the meeting planning process. From February to April 2018, we developed the materials to facilitate the process of engagement during the year 1 meeting including a meeting brochure complete with bios and pictures of all study team members, agenda, and presentations of preliminary findings. We determined data collection needs (including notetaking and video recordings) and completed necessary service contracts (i.e. videographer, caterer, facilities manager). From August 2017 to April 2018, we performed data collection and generated preliminary results for EPOC Aims 1 and 2 that would be presented to the research team for feedback during the meeting. These materials were produced in presentation form and as semi-structured discussion questions. In April 2018, members of the research team held two 1-hour meetings to discuss the Kick-Off Meeting planning and solidify day-of operations including: the room layout, videography, photo and video recording permission, signage for the room and building, catering, transportation, an attendee list, meeting materials (including presentations, agenda, brochure), and data collection procedures.

The Reception Dinner and Kick-Off Meeting were held April 19–20, 2018 in Ann Arbor, Michigan. All 25 members of EPOC’s research team (Figure 1), who are both local to Michigan and reside out-of-state were invited to attend. A pre-meeting Reception Dinner was held April 19th at the home of one of the study’s co-principal investigator (P.I.). This dinner was informally structured, and was designed to bring research team members face-to-face for introductions, to be a pre-meeting ice-breaker; to build member relationships, and to prepare the team for more in-depth interactions and discussions related to the research study during the Kick-Off Meeting (Dalal et al., 2009). The Kick-Off Meeting was held April 20th. The meeting was formally structured, using an agenda (Table 1), designed to: bring the entire research team together for introductions and multidisciplinary collaboration; promote engagement of the experts and the EMS community in all stages of the research process; promote shared learning among the research team members throughout the research process; and to use empirical methods to generate action by collecting data on discussions throughout the Kick-Off to inform EPOC methods and deliverables for each specific aim.

Table 1.
EPOC Kick-Off Meeting Agenda and Description of Activities.

Time Event Description

Thursday, April 19th, 2018

7:00 pm EST Reception dinner at the home of the Co-Principal Investigator (Co-PI) An informal dinner for the study team, hosted at the home of the Co-PI, involving discussion and introductions

Friday, April 20th, 2018

8:00–8:40 am EST Breakfast, Welcome & Team Member Introductions Introduced out-of-hospital cardiac arrest, the development of EPOC, and EPOC’s three aims as well as the purpose for the Kick-Off Meeting.

8:40–9:00 am EST A Story of Survival Key EMS community members (i.e. survivor, bystander) played the dispatch tape from the survivor’s cardiac arrest event, followed by informal discussion.

9:00–9:50 am EST Introduction to CARES Data
Aim 1 Preliminary Analysis Results Prepared by Research Team Member (Resuscitation and/or Pre-hospital Expert). Provided an overview of the EPOC data set, followed by informal discussion.
Prepared by Research Team Members (Methods Experts). Presented Aim 1. Methods and Aim 1. Results, followed by informal discussion.

9:50–10:00 am EST Refreshments Informal break session designed to promote socializing among research team members

10:00–10:30 am EST Aim 2 Preliminary Findings EMS Pilot Results/Suggested Blue Print for Aim 2 Site Visits Prepared by Research Team Members (Methods Experts). Presented an overview of Aim 2 pilot site visit results and Aim 3, followed by informal discussion.

10:30–11:30 am EST Panel Prepared by Research Team Members (Resuscitation and/or Pre-hospital Experts: Medical Director for the Michigan Department of Health and Human Services Bureau of EMS, Trauma, and Preparedness, Fire Chief, Medical Director of local Medical Control Authority). Presented system level factors from state/policy and boots on the ground perspectives, followed by informal discussion.

11:30 am–12:00 pm EST Travel to EMS Agency and Community Provided opportunity for discussions among research team members during the commute to a local EMS agency.

12:00–1:00 pm EST EMS Agency Site Visit: Tour & Networking lunch with EMS/MCA leadership Research Team Members (All) and EMS agency administrators participated in a working lunch to discuss agency administrator’s perspectives on cardiac arrest response and related system level factors. EMS agency personnel provided Research Team Members (All) with a tour of the EMS agency and internal dispatch center.

1:00 p–1:30 pm EST Travel from EMS Agency to NCRC Provided opportunity for discussions among research team members during the commute to a local EMS agency.

1:30–2:00 pm EST Refreshments Informal break session designed to promote socializing among research team members.

2:00–3:00 pm EST Small Group Discussions: Site Visit Blue Print & Aim 2 Interview/Focus Group Guide Research Team Members (All) were split into four groups to discuss Aim 2 methods and related-content. Each group was comprised of both methods and resuscitation and/or pre-hospital experts.

3:00–3:10 pm EST Break Informal break session designed to promote socializing among research team members.

3:10–4:00 pm EST Small Group Discussions: Reconvene & Present Ideas Each group was reconvened to discuss proposed changes to methods using an understanding of the study content.

4:00–5:00 pm EST Closing Mixer: Refreshments & Debriefing Session Informal break session designed to promote socializing amongst research team members.

5:00 pm Adjourn

Two activities at the EPOC Kick-Off Meeting were designed to bridge the research and community/service stakeholders: (a) video and live discussion with a couple in which the husband performed dispatch-assisted CPR on his wife (A Story of Survival) and (b) the EMS agency tour. The two activities served several purposes—orienting many of the researchers from different disciplines who were not familiar with EMS systems and processes beforehand, grounding the research in real-world experiences and place, creating different types of dialogue than standard research interaction (e.g., presenting slides and discussion), and leveling the playing field between research and community/service stakeholders by literally placing researchers on community “turf.”

Data Collection

All members of the team took process notes as participant observers throughout the day to capture what was said in addition to the overall process and climate for the day. Fieldnotes were important for capturing detail such as interaction among participants and the tone of the room. Members also captured analytical thoughts in the fieldnotes. Furthermore, to capture the meeting verbatim, we video recorded all sessions, including break out small group meetings of interdisciplinary research and community team members. Recordings were transcribed by professional transcriptionists, de-identified by study staff, and prepared for analysis.

Data Analysis

The analysis included the fieldnotes and the transcripts for a more complete examination that included both what was said and the interaction. Research team members (MA, MF, AK, SF, WN) met to discuss analyses, preliminary findings, and whether the community-engaged research framework was the best fit for the EPOC Kick-Off process until consensus was reached. Dialogue between research team members resulted in the development of three questions that would be used to guide the final analysis including: 1) How did team building occur among content experts and methodologists?; 2) How did content experts react to the presented secondary data set analysis?; and 3) What did we learn from methods and content experts regarding site visits for qualitative work, such as what sites to visit and who to talk to?

Further, dialogue resulted in a proposal for the development of an adaptive community-engaged research framework. For the second stage of the analysis, a member of the research team used Immersion/crystallization, an exploratory qualitative method characterized by continual review of the fieldnotes and transcripts (Crabtree & Miller, 1999). We used matrices to facilitate the analysis process of immersion/crystallization, which involves intense review of the data interspersed with pauses to reflect and identify patterns and themes. After an individual review of the data by one research team member (SF) for emergent themes for each research question, research team members (AK, WN, SF) reflected on and discussed the findings until consensus was reached. Immersion/crystallization was also used to guide the definition of the adapted community-engaged research framework.

Public Involvement

Members of the public (e.g., residents who may receive services) were not involved in setting the research question or the outcome measures, but they were they intimately involved in design and implementation of the Kick-Off meeting. A survivor and spouse were present hearing the presentation of data, and gave context through their personal experiences. In addition, professional first responders from local communities were also involved in the Kick-Off meeting. Once this study has been published, participants will be informed of the results through an email listserv.

Results

Team-building among methods and content experts began before the Kick-Off Meeting, where some EPOC team members met regularly, and others met intermittently on a total of 26 different occasions either in-person or by phone. Methods and content experts collaborated in meetings to conduct study Aim 1 and Aim 2, and to plan for the Kick-Off Meeting. However, the Reception Dinner and the Kick-Off Meeting were the first time many team members were able to meet and interact in-person. The schedule intentionally included time for informal discussion and breaks for networking among team members, many of whom had talked via telephone but never in person. This informal discussion was meant to help to develop relationships further. The intermingling of methods and content experts was an overarching element of the EPOC Kick-Off Meeting process, and was successful in facilitating team building among experts by promoting open dialogue between the two groups, and resulting a shared understanding of study aims, content, and methods among both groups. Three themes emerged from the data analysis including: collaborating throughout each stage of the Kick-Off Meeting, building trusting relationships by challenging each other’s perspectives, and sharing expertise, gaining shared experiences of learning and understanding of EPOC’s goals, content, and research approach (Figure 2).

Figure 2.
Team building among content experts and methodologists.

Collaborating Throughout Each Stage of the Kick-off Meeting

Content and methods experts who attended the Kick-Off Meeting were from diverse disciplines with varied roles in academia and research, pre-hospital care, emergency medicine and other medical specialties, and state government (Figure 2). Among the research team members, 15 methods experts and 6 content experts attended. Among the non-research team members, one content expert, a cardiac arrest bystander, and a cardiac arrest survivor attended. Collaboration by content and methods experts was encouraged to help guide each stage of the research process (e.g., what stakeholders should be included for data collection, and how variation should be captured during sampling and qualitative processes).

Small group discussions in the second half of the Kick-Off meeting seemed to promote more discussion. First, the earlier part of the meeting, including the presentation of data, the survivor story, and the field visit primed individuals with background information. The small groups were a chance to build relationships, shared language, and shared vision (Wallerstein et al., 2008). For example, during the Small Group Discussions, dialogue between experts led to an interpretation of the research that differed from the methods perspective, and resulted in guidance for defining high-and low-performing sites in the first aim of the study.

But the criteria that’s used to define the high performing and the low performing is based on survival and some of these systems, and we are to adjust for, like in rural communities, they can be a highly performing system but put in that low performing category, so…redefining those 2 terms and not using high and low performing, is to do something you know, use different words…I know with working with EMS and fire agencies, they’re very proud of what they do and the work that they’re doing, and they don’t want…they know they can go in everyday and do their best job and that person’s still not going to survive just because of what they’re presented with.—Content Expert/Secondary Data Set Coordinator and Registered Nurse

To facilitate ongoing dialogue and collaboration, following the Kick-Off Meeting, team members were encouraged to stay in contact, and to share interpretations and findings to maintain open dialogue throughout the study.

We know from a mixed methods project, much like out-of-hospital cardiac arrest, that it’s a team process, and so the best mixed method teams actually have lots of dialogue with each other and so it’s been great…I think…if you’re traditionally doing stats you’re thinking about how are we gonna model this differently and this, that, and the other, and you’re trying to come up with the best answer, but just to say, look, we’re doing an iterative process, and at any point in time, if you have a new question, you know, try and bring it back to the group and…we can test hypotheses in the qualitative data and the qualitative data can also generate hypotheses that then can be investigated in the quantitative data as well, um, and so, you know…and if questions are coming up, you know, please share them with the rest of us so that we can think about, well, how can we get some answers for you that might make the modeling easier, or better, or more effective…—Methods Expert/Physician

Building Trusting Relationships by Challenging Each Other’s Perspectives and Sharing Expertise

The value of transdisciplinary teams and research lies in the unique and complementary perspectives each individual brings to the work. We must respectfully engage with differing opinions and facilitate working through disagreements. During the Kick-Off meeting, content and methods experts appeared comfortable enough to challenge each other’s perspectives, to ask for interpretation of findings, and to reach consensus in planning the research process. The small group discussions set up a space for challenging perspectives. Because the goal of the small group discussions was to develop the site visit, interview protocols, and focus group protocols to better address the second aim of the study, we sought to benefit from the transdisciplinary expertise of team members. For example, during the Small Group Discussions, content and methods experts were able to challenge one another, and to reach consensus on which stakeholders should be included in the second aim of the study which involves visiting communities and interviewing stakeholders.

We went back and forth about administrators for interviews. I guess there may be a political imperative to talk to the administrator and may be useful to hear about resources and things of that nature but that there were some mixed feelings about that.—Methods Expert/Political Scientist

We also found team members engaging when discussing and interpreting preliminary findings. For example, during the EPOC Aim 1 Preliminary Findings Discussion, when asked “…any of those numbers look surprising or notable?” and “What are these findings compared to the national registry?,” content experts agreed that preliminary findings were consistent with data and outcomes they have witnessed in practice over the years. Further, in addition to vetting study results, content experts provided their interpretations of findings based on practice.

I think that 70% of arrests occurring in homes is pretty consistent because we talk about strategies we talk about things like…defibrillation, putting defibrillators all over the place…and we’re not putting these in homes or in anyone’s chest…—Content Expert (Non-EPOC Research Team Member)/Medical Director for the State Health Department EMS Bureau and Director of local Medical Control Authority

Team members also raised key distinctions in EMS system stakeholders and research team members’ perspectives. For example, the Small Group Discussions led to consensus on how EMS System stakeholders might conceptualize or operationalize certain process aspects differently than the research team (e.g., quality improvement) for the Aim 2 Interview Guide. Stakeholders helped to refine plans for purposeful sampling of informed stakeholders.

[Discussion on whether to include a “Quality Improvement” person in data collection] So, let’s play this out. So, they say, no, we don’t have anybody who has…we don’t have enough money to pay for a quality improvement for our non-profit ambulances…What are you gonna do?—Content Expert/Emergency Medicine Physician

Gaining Shared Experiences of Learning and Understanding EPOC’s Content and Goals

Dialogue between content and methods experts resulted in a shared understanding of EPOC goals and application to the real-world setting. For example, during EPOC Aim 1 Preliminary Findings, dialogue between content and methods experts resulted in the differentiation between EPOC’s goals and ongoing goals other stakeholders have for analyzing out-of-hospital cardiac arrest data and strategies to improve outcomes. The discussion centered around an explanation of an organization in the state, called SaveMIHeart, for context. SaveMIHeart is a non-profit organization with the mission to double cardiac arrest survival by 2020 by bringing multiple stakeholders together, collaborating with communities and healthcare providers, and using a data registry to monitor and report cardiac arrest information.

So the goal came from SaveMIHeart. I don’t think in EPOC there was a specific goal set, so that’s our statewide program. EPOC is more about finding the “best practices,” but what you should have in mind, and I think what you’re asking is “Are the top-performers in Michigan reaching the best possible outcomes?”–Content Expert/Emergency Medicine Physician

During A Story of Survival, content experts confirmed that EPOC aims and outcomes are important in the practical/operational setting, bridging the gap between research and action.

That’s part of that disparity we see from a system’s standpoint…Where were you calling 911 from? Is it just simply the dispatcher or emergency services…or is it that the time to intervention is zero minutes and its coming right at the time of that call…? And coming from that call you [bystander name removed] were clearly receptive to the [dispatcher name removed] guidance and direction. And I think that from a policy standpoint we try to communicate…importance of dispatch…and citizenry…—Content Expert (Non-EPOC Research Team Member)/Medical Director for the State Health Department EMS Bureau and Director of local Medical Control Authority

Discussion

This research illustrates the collaboration of 19 researchers representing 11 disciplines, a cardiac arrest bystander and survivor, content experts, and 1 member of a local community organization. Our findings indicated that collaborating throughout each stage of a research meeting is feasible and has the potential to foster early team building. Developing a workable schedule to facilitate connection of different stakeholders throughout required intensive planning. However, cultivating trust is a necessary condition to ensure team members feel comfortable sharing their own views and challenging each other when needed. Each member needs to be engaged and a Kick-Off meeting worked well for establishing a culture that disagreement is critical to develop ideas. A schedule that includes clear goals for each session and assigned tasks can potentially promote engagement. The individuals who planned the agenda were instrumental in ensuring the large and small group meetings progressed and remained focus to meet goals. In addition, the kick-off meeting set a culture promoting interaction among the researchers and encouraging questions of each other. Strong leadership can help to set the culture by modeling communication. Small group discussion in particular provided a safer space for sharing ideas, especially when they challenge others. Finally, the Kick-Off meeting was a way for stakeholders to have shared experience in learning about the project’s background and goals in moving toward a shared vision.

Building Trust and Psychological Safety

We learned important lessons from our experience of facilitating a large team meeting of relatively unfamiliar team members. Building trust, often referred to as psychological safety, is critical yet often overlooked (Edmondson, 1999). At a team level, psychological safety is “a shared belief that the team is safe for interpersonal risk taking” (Edmondson, 1999, p. 354), which requires all team members have a similar perception of what safety means. Once trust was established, it was helpful set expectations for both research-to-practice and practice-to-research (i.e., practice informed research) from the onset of the study. Our goal was to build a transdisciplinary team of methodologist and content experts—investigators and community stakeholders. Thus, the entire team needed to understand that research methods, cardiac arrest care, and community context were on equal footing. In brief, we wanted to utilize everyone’s unique talents. Gathering both content and methods experts to discuss objectives and study design in the same setting facilitated identifying potential challenges and opportunities for innovation early in the process. Investigators in large-scale emergency medicine trials have stressed the early efforts to foster collaboration among different stakeholder groups (Guetterman et al., 2015, 2017; Legocki et al., 2015). Our findings here were similar and may be applicable to other fields.

Engaging the Full Team in the Analysis

The review of preliminary findings from the pilot site demonstrated the value of the larger team in interpreting and ensuring actionable findings. A transdisciplinary group of cardiac arrest care specialists, community stakeholders, and methodologists help keep the goals of the research project and how it is conducted grounded in practical realities. We subsequently increased our planned engagement of community stakeholders throughout the project to interpret our findings. We recommend reconvening the team throughout the interpretation and dissemination. The intent of the ongoing collaboration is to ensure that end products—in this case a toolkit of best practices for out-of-hospital cardiac arrest—are usable by practitioners and communities.

Contribution to Literature on Teams in Mixed Methods

Our findings add to the relatively small body of literature on teamwork in mixed methods research, but also likely apply to purely quantitative or qualitative studies. From a mixed methods perspective, the integration of qualitative and quantitative research is the methodology’s defining feature. Mixed methods research is more complicated because of the need for multiple types of methodological expertise, and quantitative or qualitative experts may have little knowledge of other approaches. Thus, integration might be considered in the team dimension (Curry et al., 2012) which involves intentionally drawing members from various backgrounds and orchestrating among research investigators to effectively conduct the study (Archibald, 2016; Creswell et al., 2011; Curry & Nunez-Smith, 2014; Morse & Niehaus, 2009). Convening teams early and throughout the project to discuss and reinforce a shared understanding of the project is a potential strategy to ensure a minimum level of competency across all methods and content as recommended by O’Cathain et al. (2008).

Insights for Team Building in Community-engaged Research

In addition, we gained insight into the challenges of team building in a community-engaged research project. Large teams are inherently complicated. We recommend recognizing complexity and the competing expertise among team members, which involves letting people maintain their own groups while fostering respect and trust (Curry et al., 2012; Poth, 2018). To promote a true transdisciplinary team, it may help to build from existing relationships whenever possible (Wallerstein et al., 2008). Although our large team had not previously worked together, several methodologists, cardiac arrest researchers, and community members had worked together. Engaging new team members early in the process and frequently is a strategy to build relationships and common language about the project through both formal and informal dialogue as advocated by Dalal et al. (2009). We found that having preliminary data available for a transdisciplinary Kick-Off meeting provides a concrete starting point for conversation. Preliminary data stimulated discussion among methodologists and content experts about the actual study. An additional challenge may arise from geographically dispersed teams that would require web meetings. Our project was rather unique in that it was constrained to a single state in the U.S. Although several team members traveled from outside of the state, all data collection and all communities to be visited are within the state of Michigan. Of course, time and resources are needed for the Kick-Off meeting and increase with additional geographical distance.

Limitations of the Study

It is important to review a few limitations of this study. Our results are derived from a single, large scale, community-engaged study in emergency medical services research. However, the outlined approach may be transferrable to other types of community-engaged research, including CBPR that involves multiple stakeholders, content research experts, and methodologists. In addition, in our project, many individuals had worked together previously though not as an entire team. In the unlikely event of research teams comprised of entirely new members, additional considerations like establishing familiarity and networking time are likely important. Another limitation is that our project included only two members of the public—a survivor of cardiac arrest and their spouse. However, at this stage of the research, we were preparing for the next aim which was largely focused on interviewing community responders. Members of community agencies responding were involved and provided insight to plan data collection with their peers.

Conclusion

A Kick-Off meeting is a feasible and potentially effective strategy to build teams and foster collaboration among methodologists and content experts within community-engaged studies. Because community-engaged research often requires a mix of methods, a group of statistical, qualitative, and mixed methods specialists may be needed. As community-engaged research is increasingly part of emergency medical services research and other health services research, team science deserves further consideration to foster transdisciplinary research in which all members are equitable and engaged toward a shared vision.

Time	Event	Description
Thursday, April 19th, 2018
7:00 pm EST	Reception dinner at the home of the Co-Principal Investigator (Co-PI)	An informal dinner for the study team, hosted at the home of the Co-PI, involving discussion and introductions
Friday, April 20th, 2018
8:00–8:40 am EST	Breakfast, Welcome & Team Member Introductions	Introduced out-of-hospital cardiac arrest, the development of EPOC, and EPOC’s three aims as well as the purpose for the Kick-Off Meeting.
8:40–9:00 am EST	A Story of Survival	Key EMS community members (i.e. survivor, bystander) played the dispatch tape from the survivor’s cardiac arrest event, followed by informal discussion.
9:00–9:50 am EST	Introduction to CARES Data Aim 1 Preliminary Analysis Results	Prepared by Research Team Member (Resuscitation and/or Pre-hospital Expert). Provided an overview of the EPOC data set, followed by informal discussion. Prepared by Research Team Members (Methods Experts). Presented Aim 1. Methods and Aim 1. Results, followed by informal discussion.
9:50–10:00 am EST	Refreshments	Informal break session designed to promote socializing among research team members
10:00–10:30 am EST	Aim 2 Preliminary Findings EMS Pilot Results/Suggested Blue Print for Aim 2 Site Visits	Prepared by Research Team Members (Methods Experts). Presented an overview of Aim 2 pilot site visit results and Aim 3, followed by informal discussion.
10:30–11:30 am EST	Panel	Prepared by Research Team Members (Resuscitation and/or Pre-hospital Experts: Medical Director for the Michigan Department of Health and Human Services Bureau of EMS, Trauma, and Preparedness, Fire Chief, Medical Director of local Medical Control Authority). Presented system level factors from state/policy and boots on the ground perspectives, followed by informal discussion.
11:30 am–12:00 pm EST	Travel to EMS Agency and Community	Provided opportunity for discussions among research team members during the commute to a local EMS agency.
12:00–1:00 pm EST	EMS Agency Site Visit: Tour & Networking lunch with EMS/MCA leadership	Research Team Members (All) and EMS agency administrators participated in a working lunch to discuss agency administrator’s perspectives on cardiac arrest response and related system level factors. EMS agency personnel provided Research Team Members (All) with a tour of the EMS agency and internal dispatch center.
1:00 p–1:30 pm EST	Travel from EMS Agency to NCRC	Provided opportunity for discussions among research team members during the commute to a local EMS agency.
1:30–2:00 pm EST	Refreshments	Informal break session designed to promote socializing among research team members.
2:00–3:00 pm EST	Small Group Discussions: Site Visit Blue Print & Aim 2 Interview/Focus Group Guide	Research Team Members (All) were split into four groups to discuss Aim 2 methods and related-content. Each group was comprised of both methods and resuscitation and/or pre-hospital experts.
3:00–3:10 pm EST	Break	Informal break session designed to promote socializing among research team members.
3:10–4:00 pm EST	Small Group Discussions: Reconvene & Present Ideas	Each group was reconvened to discuss proposed changes to methods using an understanding of the study content.
4:00–5:00 pm EST	Closing Mixer: Refreshments & Debriefing Session	Informal break session designed to promote socializing amongst research team members.
5:00 pm	Adjourn

Footnotes

Acknowledgments

We gratefully acknowledge Amanda Kogowski for her instrumental role in collecting data and preparation for the kick-off meeting and Lilly Pritula for her assistance with editing and preparing the manuscript.

Author Contributions

MA, TG and MF contributed to the conception and design of the kick-off meeting. The manuscript was drafted by TG and was revised by MA, SF, MF, WN, BKN, PM, CMS, CN, APH, TS, and the other authors. All authors approved the publication.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics/IRB Approval

Institutional Review Board at the University of Michigan (approval for HUM00135840) and the RAND Corporation.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the National Heart, Lung, and Blood Institute (NHLBI), 1R01HL137964-03.

ORCID iDs

Timothy C. Guetterman

Sydney Fouche

References

Archibald

M. M.

(2016). Investigator triangulation: A collaborative strategy with potential for mixed methods research. Journal of Mixed Methods Research, 10(3), 228–250. https://doi.org/10.1177/1558689815570092

Cooke

N. J.

Hilton

M. L.

(Eds.), (2015). Enhancing the effectiveness of team science. National Academies Press.

Crabtree

B. F.

Miller

W. L.

(Eds.). (1999). Doing qualitative research. Sage.

Creswell

J. W.

Plano Clark

V. L.

(2018). Designing and conducting mixed methods research (3rd ed.). Sage.

Creswell

J. W.

Klassen

A. C.

Plano Clark

V. L.

Smith

K. C.

(2011). Best practices for mixed methods research in the health sciences. National Institutes of Health, 2013, pp. 541–545. https://obssr.od.nih.gov/wp-content/uploads/2016/02/Best_Practices_for_Mixed_Methods_Research.pdf

Curry

L. A.

O’Cathain

Clark

V. L. P.

Aroni

Fetters

Berg

(2012). The role of group dynamics in mixed methods health sciences research teams. Journal of Mixed Methods Research, 6(1), 5–20. https://doi.org/10.1177/1558689811416941

Curry

Nunez-Smith

(2014). Mixed methods in health sciences research: A practical primer (Vol. 1). Sage.

Dalal

Skeete

Yeo

H. L.

Lucas

G. I.

Rosenthal

M. S.

(2009). A physician team’s experiences in community-based participatory research: Insights into effective group collaborations. American Journal of Preventive Medicine, 37(6), S288–S291. https://doi.org/10.1016/j.amepre.2009.08.013

DeJonckheere

Lindquist-Grantz

Toraman

Haddad

Vaughn

L. M.

(2019). Intersection of mixed methods and community-based participatory research: A methodological review. Journal of Mixed Methods Research, 13(4), 481–502. https://doi.org/10.1177%2F1558689818778469

10.

Edmondson

(1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44(2), 350–383. https://doi.org/10.2307%2F2666999

11.

Guetterman

T. C.

Fetters

M. D.

Legocki

L. J.

Mawocha

Barsan

W. G.

Lewis

R. J.

Berry

D. A.

Meurer

W. J.

(2015). Reflections on the adaptive designs accelerating promising trials into treatments (ADAPT-IT) process—Findings from a qualitative study. Clinical Research and Regulatory Affairs, 32(4), 119–128. https://doi.org/10.3109/10601333.2015.1079217

12.

Guetterman

T. C.

Fetters

M. D.

Mawocha

Legocki

L. J.

Barsan

W. G.

Lewis

R. J.

Berry

D. A.

Meurer

W. J.

(2017). The life cycles of six multi-center adaptive clinical trials focused on neurological emergencies developed for the Advancing Regulatory Science initiative of the National Institutes of Health and US Food and Drug Administration: Case studies from the Adaptive Designs Accelerating Promising Treatments into Trials Project. SAGE Open Medicine, 5, 1–16. https://doi.org/10.1177%2F2050312117736228

13.

Israel

B. A.

Eng

Schulz

A. J.

Parker

E. A.

(2013). Methods for community-based participatory research for health. Jossey Bass.

14.

Israel

B. A.

Schulz

A. J.

Parker

E. A.

Becker

A. B.

(1998). Review of community-based research: Assessing partnership approaches to improve public health. Annual Review of Public Health, 19, 173–202. https://doi.org/10.1146/annurev.publhealth.19.1.173

15.

Jones

Koegel

Wells

K. B.

(2008). Bringing experimental design to community-partnered participatory research. In Minkler

Wallerstein

(Eds.), Community-based participatory research for health: From process to outcomes (pp. 67–90). Jossey-Bass.

16.

Jones

Wells

(2007). Strategies for academic and clinician engagement in community-participatory partnered research. JAMA, 297(4), 407–410. https://doi.org/10.1001/jama.297.4.407

17.

Leahey

(2016). From sole investigator to team scientist: Trends in the practice and study of research collaboration. Annual Review of Sociology, 42, 81–100. In Poth

C. N.

(Ed.), Innovation in mixed methods research: A practical guide to integrative thinking with complexity. Sage. 2019.

18.

Legocki

L. J.

Meurer

W. J.

Frederiksen

Lewis

R. J.

Durkalski

V. L.

Berry

D. A.

Barsan

W. G.

Fetters

M. D.

(2015). Clinical trialist perspectives on the ethics of adaptive clinical trials: A mixed-methods analysis. BMC Medical Ethics, 16(1), 27. https://doi.org/10.1186/s12910-015-0022-z

19.

Lindau

S. T.

Makelarski

J. A.

Chin

M. H.

Desautels

Johnson

W. E.

Miller

Peters

Robinson

Schneider

Thicklin

Watson

N. P.

Wolfe

Whitaker

(2011). Building community-engaged health research and discovery infrastructure on the South Side of Chicago: Science in service to community priorities. Preventive Medicine, 52(3–4), 200–207. https://doi.org/10.1016/j.ypmed.2011.01.001

20.

Mendel

Meredith

L. S.

Schoenbaum

Sherbourne

C. D.

Wells

K. B.

(2008). Interventions in organizational and community context: A framework for building evidence on dissemination and implementation in health services research. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 21–37. https://doi.org/10.1007/s10488-007-0144-9

21.

Mendel

Ngo

V. K.

Dixon

Stockdale

Jones

Chung

Jones

Masongsong

Khodyakov

(2011). Partnered evaluation of a community engagement intervention: Use of a “kickoff” conference in a randomized trial for depression care improvement in underserved communities. Ethnicity & Disease, 21(3 Suppl 1), S1–77–88.

22.

Morse

J. M.

Niehaus

(2009). Mixed methods design: Principles and procedures—Volume 4, developing qualitative inquiry. Pennsylvania State University.

23.

O’Cathain

Murphy

Nicholl

J. P.

(2008). Dysfunctional, multidisciplinary or interdisciplinary? Team working in mixed methods research. Qualitative Health Research, 18, 1574–1585. https://doi.org/10.1177%2F1049732308325535

24.

Poth

C. N.

(2018). Innovation in mixed methods research: A practical guide to integrative thinking with complexity. Sage.

25.

Rosenfield

P. L.

(1992). The potential of transdisciplinary research for sustaining and extending linkages between the health and social sciences. Social Science & Medicine, 35(11), 1343–1357. https://doi.org/10.1016/0277-9536(92)90038-R

26.

Sasson

Haukoos

J. S.

Bond

Rabe

Colbert

S. H.

King

Sayre

Heisler

(2013). Barriers and facilitators to learning and performing cardiopulmonary resuscitation in neighborhoods with low bystander cardiopulmonary resuscitation prevalence and high rates of cardiac arrest in Columbus, OH. Circulation: Cardiovascular Quality and Outcomes, 6(5), 550–558. https://doi.org/10.1161/CIRCOUTCOMES.111.000097

27.

Stokols

Hall

K. L.

Taylor

B. K.

Moser

R. P.

(2008). The science of team science: Overview of the field and introduction to the supplement. American Journal of Preventive Medicine, 35(2), S77–S89. https://doi.org/10.1016/j.amepre.2008.05.002

28.

Wallerstein

N. B.

Duran

(2006). Using community-based participatory research to address health disparities. Health Promotion Practice, 7, 312–323. https://doi.org/10.1177%2F1524839906289376

29.

Wallerstein

Duran

Oetzel

J. G.

Minkler

(2018). On community-based participatory research. In Wallerstein

Duran

Oetzel

Minkler

(Eds.), Community-based participatory research for health: Advancing social and health equity (pp. 3–16). Jossey-Bass.

30.

Wallerstein

Oetzel

Duran

Tafoya

Belone

Rae

(2008). What predicts outcomes in CBPR. In Minkler

Wallerstein

(Eds.) Community-based participatory research for health: Process to outcomes (2nd ed., pp. 371–392). Jossey-Bass.