Methodological Insights From a Virtual,Team-Based Rapid Qualitative Method Applied to a Study of Providers’ Perspectives of the COVID-19 Pandemic Impact on Hospital-To-Home Transitions

Abstract

Background

During the COVID-19 pandemic, rapid virtual qualitative methods have gained attention in applied health research to produce timely, actionable results while complying with the pandemic restrictions. However, rigour and analytical depth may be two areas of concern for rapid qualitative methods.

Methods

In this paper, we present an overview of a virtual team-based rapid qualitative method within a study that explored health care providers’ perspectives of how the COVID-19 pandemic has impacted hospital-to-home transitions, lessons learned in applying this method, and recommendations for changes. Using this method, qualitative data were collected and analyzed using the Zoom Healthcare videoconferencing platform and telephone. Visual summary maps were iteratively created from the audio recordings of each interview through virtual analytic meetings with the team. Maps representing similar settings (e.g. hospital providers and community providers) and Sites were combined to form meta-maps representing that group’s experience. The combinations of data that best fit together were used to form the final meta-map through discussion.

Results

This case example is used to provide a description of how to apply a virtual team-based rapid qualitative method. This paper also offers a discussion of the opportunities and challenges of applying this method, in particular how the virtual team-based rapid qualitative method could be modified to produce timely results virtually while attending to rigour and depth.

Conclusions

We contend that the virtual team-based rapid qualitative data collection and analysis method was useful for generating timely, rigorous, and in-depth knowledge about transitional care during the COVID-19 pandemic. The recommended modifications to this method may enhance its utility for researchers to apply to their qualitative research studies.

Keywords

virtual environments qualitative evaluation action research methods in qualitative inquiry narrative

Background

Qualitative research methodologies have been used for centuries in health sciences to gain insights into social phenomena and human experiences (Guest, 2008; Luciani et al., 2019; Morse, 2012; Padgett, 2016; Watkins, 2017; Watkins et al., 2013; Watkins & Gioia, 2015). Some traditional qualitative methodologies can be broadly categorized as “interpretive (e.g. grounded theory, ethnography, phenomenology) or critical (e.g. action research, feminist research)” (Cooper & Endacott, 2007) and apply sets of methods (procedures and tools) to answer research questions. Researchers use qualitative methods to make “an invisible world visible to others” or interpret the phenomenon of interest by engaging in dialogue and/or observation (Luciani et al., 2019). Within the health care field, qualitative research has been used extensively to examine various clinical phenomena, such as patient experiences and health needs (Gisselbaek et al., 2021; Gleeson et al., 2016), design and evaluate health interventions (Hamilton & Finley, 2019; Slattery et al., 2020) and explore health provider service delivery (Brooks, 2019; Tremblay et al., 2022).

Traditional qualitative methods require considerable time and resource investments (Silverio et al., 2020), making these techniques challenging to apply in health care environments where results may be needed urgently to inform solutions to practical and clinical problems (Thorne, 2016). Many qualitative methods are time-consuming due to the time required for transcription, data cleaning, working with qualitative software, team collaboration activities (e.g. training team) and interpreting data at a deep level (Silverio et al., 2020). While these challenges related to time pressures were present before the COVID-19 pandemic (Riley et al., 2013; Silverio et al., 2020; Thorne et al., 2004; Watkins, 2017), they became even more prevalent with pandemic-related contextual changes necessitating a rapid production of results (Tremblay et al., 2021; Vindrola-Padros et al., 2020).

The COVID-19 pandemic has triggered significant and unprecedented changes to health systems (Haldane et al., 2021), including a rapid shift to virtual health care and restricted hospital visitor policies (Munshi et al., 2021; Rawaf et al., 2020; Shreffler et al., 2020; Singh, Under review). Since health systems have traditionally been slow to adapt and respond to change (Fridell et al., 2020; Gale et al., 2019; Institute of Medicine (US) Committee on the Health Professions Education Summit, 2003), some argue that the COVID-19 context can serve as “an accidental catalyst for change,” triggering process improvements within the healthcare system (Jazieh & Kozlakidis, 2020; Tang, 2020). However, given the time requirements associated with qualitative methods noted earlier, qualitative studies run the risk of producing results that are outdated and/or no longer relevant to inform rapid change by the time they are released (Gale et al., 2019; Riley et al., 2013; Silverio et al., 2020).

Qualitative rapid methods

According to Silverio and colleagues, the time-consuming nature of qualitative methods is often under-estimated (Silverio et al., 2020). Attempts have been made to reconceptualize qualitative methods to be more efficient (Vindrola-Padros et al., 2020; Vindrola-Padros & Johnson, 2020). Rapid research has been defined as “research designed to address the need for cost-effective and timely results in rapidly changing situations” (Beebe, 2014). Generally, rapid qualitative methods reduce researchers’ time on data collection, management, and analysis (Taylor et al., 2018). According to a literature review in 2020 by Vindrola-Padros and colleagues, the six primary reasons for the use of rapid qualitative methods include reduced time, reduced cost, increased quantity of data collected, improved efficiency, improved accuracy and generating a closer interpretation of participants’ realities. During the pandemic, there has been a growing interest in using rapid qualitative research approaches to gain insights into pandemic-related responses and experiences to produce timely results for rapid dissemination and application (Anghelescu et al., 2021; Bröer et al., 2021; MacCarthy et al., 2020; McNall & Foster-Fishman, 2007; Roberti et al., 2021; Singleton et al., 2021; Taylor et al., 2018; Tort-Nasarre et al., 2021; Tremblay et al., 2021; Vindrola-Padros et al., 2020). Specifically, during a pandemic, qualitative data can complement epidemiological data by generating insights into the “lived experiences of disease, care, and epidemic response efforts” to inform decisions and practice changes (Vindrola-Padros et al., 2020). However, the concerns of rapid qualitative methods are that the compressed timelines could lack rigour and analytical depth (Burgess-Allen & Owen-Smith, 2010; Gale et al., 2019; McNall & Foster-Fishman, 2007; Silverio et al., 2020).

Qualitative virtual methods

The selection of an appropriate qualitative method is based on multiple factors, including study aims and objectives, resources and contextual factors (Hanley, 2011). Physical distancing restrictions and time constraints during the COVID-19 pandemic have forced qualitative researchers to rethink approaches to both data collection and analysis used within traditional qualitative research (Tremblay et al., 2021); many shifted to virtual qualitative data collection and analysis approaches (Archibald et al., 2019; Gray et al., 2020; Roberts et al., 2021; Santhosh et al., 2021). The concept of “virtual” is broad and can include various modalities, such as phone and videoconferencing software (Crawford & Serhal, 2020; Taylor, 2021; World Health Organization, 2021). With regard to data collection, a knowledge base has emerged on virtual qualitative data collection during the pandemic context (Archibald et al., 2019; Davies et al., 2020; Gray et al., 2020; Santhosh et al., 2021). This literature indicates the need to attend to unique factors with respect to virtual data collection (e.g. talk-based platforms such as Skype), such as rapport building, interaction issues, and logistics that could impact the quality of data collected compared to in-person (Davies et al., 2020). Attending to these factors may minimize the disadvantages of virtual data collection, including reduced depth of data collected (Davies et al., 2020).

Prior studies have presented methodological insights on the use of qualitative data collection and analysis. Giesen and Roeser (2020) presented lessons learned from their team-based coding of qualitative data, which included the need for a “strong and supportive management structure,” breaking down activities related to training and coding, and creating guidance for the team (Giesen & Roeser, 2020). Similarly, Santhosh et al. (2021) and Roberts et al. (2021) shared insights from their virtual qualitative projects, which highlights the importance of thoughtful consideration of specific methodological challenges that could arise in virtual projects, stressing the importance of continuing to expand our knowledge of virtual qualitative research (Roberts et al., 2021). As the use of virtual qualitative methods continues to grow, there is a need to advance current knowledge through experimentation with different formats of virtual data collection and analysis (Lobe et al., 2020). However, compared to virtual data collection, there is a recognized need for more guidance on performing qualitative analysis virtually (Giesen & Roeser, 2020; Roberts et al., 2021; Santhosh et al., 2021).

Rigour, a critical but debated concept in qualitative research (Cypress, 2017), can be defined as “the means by which we demonstrate integrity and competence” (Tobin & Begley, 2004). The importance of rigour is summarized in a quote by Morse and colleagues, who stated that “Without rigor, research is worthless, becomes fiction, and loses its utility” (Morse et al., 2002). Given it’s importance, various checklists, quality indicators and guidelines have been created to enhance rigour in qualitative research (Johnson et al., 2020; Morse et al., 2002). While various terminology, concepts and approaches to rigour and related quality indicators have been proposed (Cypress, 2017; Johnson et al., 2020; Morse et al., 2002), a common approach is the four criteria of rigour in qualitative research: i) truth value, ii) applicability, iii) consistency, and iv) neutrality (Cypress, 2017; Lincoln & Guba, 1985; Sandelowski, 1986). Truth value is the presentation of findings as they are experienced or perceived by participants rather than based on the researcher’s viewpoints. Applicability is whether the results could apply to other contexts, settings, or groups. Consistency refers to whether researchers’ decisions are clear and transparent. Neutrality pertains to whether the findings clearly stem from the data (Sandelowski, 1986).

Although approaches to rapid qualitative research vary, the rapid nature of the analysis is believed to reduce components of rigour and analytical depth or the level of interpretation by preventing sufficient time for reflection and opportunities for interpretation (Burgess-Allen & Owen-Smith, 2010; McNall & Foster-Fishman, 2007). As demonstrated previously, it is possible to enhance rigour and achieve analytical depth within rapid qualitative methods (Gale et al., 2019; Taylor et al., 2018; Watkins, 2017). However, the rigour of virtual rapid qualitative methods is under-explored (Roberts et al., 2021). Applying these rapid qualitative methods in a virtual environment poses additional challenges when attending to rigour, involving distinct considerations compared to in-person approaches (Lobe et al., 2020; Roberts et al., 2021).

Aim

This paper describes a virtual team-based rapid qualitative method applied by our team to understand provider experiences with care transitions during the COVID-19 pandemic. Here we provide an overview of the method we used to produce timely results while attending to rigour and analytical depth, lessons learned in applying this method and recommendations for changes to the method based on our methodological experience. The methodological insights presented in the current paper may apply to other studies requiring timely, rigorous, in-depth qualitative results.

Methods

Setting

This study was conducted in hospital and community organizations in Ontario, Canada during the COVID-19 pandemic (Singh, Under review). The study was conducted between January to April of 2021. During this period in the pandemic, there were significant pressures being exerted on the Ontario health system due to hospital capacity concerns (Crawley, 2020; Wilson, 2021). To manage these pressures and reduce the risk of virus transmission, rapid changes to the healthcare system were implemented that impacted the patient experience, including sheltering in place, restricting in-person research activities and hospital visitations, postponing elective surgeries and procedures and shifting from in-person to virtual care in the community (Crawley, 2020; Munshi et al., 2021; Nielsen, 2020; Wilson, 2021). The research ethics boards of Sinai Health System and Trillium Health Partners provided ethical approval for this study.

Design

A description of the virtual team-based rapid qualitative method used can be found in Table 1. Table 1 describes each stage of the method in terms of who was involved in each collection and analysis activity (e.g. entire research team, two to three analytic research team members) and strategies used to enhance rigour. A key aspect of this virtual team-based rapid method was that it complied with COVID-19 research restrictions (e.g. physical distancing) and intended to generate timely, rigorous and in-depth results to inform practice in a rapidly evolving healthcare context.

Table 1.

Team-based rapid qualitative methodology.

Process of Data Collection and Analysis	Description	Who was Involved?	Strategy to Enhance Rigour
Stage 1: Recruit and screen	Recruitment using snowball sampling	Entire research team	-Maximal variation: Disciplines, settings (i.e. community and hospital), sites
Stage 2: Data collection	Conduct interviews and then generate field notes during and after virtual interviews	HS or JXN (conducted interviews)	-Audit trail
Stage 2: Data collection		HS or JXN (conducted interviews)	-Reflexivity
Stage 3: Data immersion	Listen to each interview, take field notes	Two to three analytical research team members reviewed each of the interviews asynchronously	-Audit trail
Stage 3: Data immersion	Listen to each interview, take field notes		-Reflexivity
Stage 4: Map each individual interview on an individual map	Create a map for each interview (i.e., individual map) (see Figure 1)	Led by one researcher with qualitative and content expertise, asynchronously	-Triangulation (field notes with maps)
Stage 4: Map each individual interview on an individual map			-Peer review
Stage 5: Modify individual maps	Modify and confirm the content of each individual map based on teams' field notes and analytical interpretations (see Figure 2)	Two to three researchers, synchronously or asynchronously	-Audit trail
			-Peer debriefing and review
			-Triangulation (team’s field notes with maps)
Stage 6: Identify patterns among individual maps	Identify which individual maps displayed similar/different findings	Analytical research team members, synchronously	-Audit trail
			-Peer debriefing and review
			-Triangulation (team’s interpretations of the maps)
Stage 7: Combine and collate individual maps to form meta-maps	Combine/collate individual maps in various meta-map combinations to determine the best fit (e.g. site, sector, provider discipline)	Led by one researcher with qualitative and content expertise	-Audit trail
Stage 8: Discuss and debrief about meta-maps	Discuss/debrief about which meta-maps fit the best and were easiest to combine, identify the combination of maps that best work together	Analytical research team members, synchronously	-Peer debriefing and review
Stage 8: Discuss and debrief about meta-maps		Analytical research team members, synchronously	-Triangulation (team’s interpretations and maps)
Stage 9: Finalize and verify the analysis	Review/refine meta-maps to reflect all findings, extract verbatim quotes from audio-recordings, and ensure terminology of themes captures analytical interpretations in final map (see Figure 3)	Led by one researcher with qualitative and content expertise with feedback from the analytical research team members	-Peer debriefing and review
Stage 9: Finalize and verify the analysis			-Triangulation (team’s interpretations of meta-maps)

Stage 1: Recruit and screen

We aimed to recruit providers that delivered health services to older adults with complex care needs and were frequently involved in care transitions. They were recruited using purposeful sampling strategies, such as a recruitment flyer sent via email to hospital and community providers within the networks of the research team. Any individuals interested in participating in this study connected to the research team either using the contact information provided on the recruitment flyer or provided their contact information to the team member’s network contact who then shared the study details with the team. Participants were screened by telephone or email. The 16 hospital and community providers that participated in this study included four hospital physicians, three primary care physicians, three occupational therapists, three care coordinators, a social worker, a physiotherapist, and a discharge planner.

Stage 2: Data collection

Depending on their preference, providers participated in one virtual interview on Zoom Healthcare (a videoconferencing platform) or by telephone. For interviews on Zoom, the researchers provided the participant with a Zoom link and their contact number in case of any technical issues connecting on Zoom and offered flexibility if they did not have access to Zoom during the interview time. Since participants were already familiar with the platform, no training on the use of Zoom was required. We obtained verbal informed consent from all participants prior to data collection. All interviews were conducted during the COVID-19 pandemic between January-April 2021.

Researchers spent a short amount of time (∼5 minutes) engaging in small talk (i.e. light or casual conversation) (Merriam-Webster, 2022) to establish rapport with participants (e.g. ‘How is your day/week going?’). The interviews were facilitated using an interview guide created by five research team members and trialed on Zoom before data collection to evaluate whether modifications were needed (Abdul Majid et al., 2017). At the end of each interview, we asked participants if there was anything else they would like to share that we had not already talked about to ensure we comprehensively captured their thoughts on the topic. Virtual interviews were audio-only or audio and video-recorded with each participant’s consent and facilitated by HS or JXN, both trained in qualitative research methods. Fifteen interviews were audio-recorded (average audio-recorded interview length: 35:08 minutes; range: 17–52 minutes) and one was audio and video-recorded (video-recorded interview length: 25:35 minutes). Concurrent data collection and analysis were conducted as field notes were generated during and after the interviews on Microsoft Word, creating an audit trail and enabling reflexivity. To minimize interruption to the flow of the conversation, brief field notes, which were limited to keywords, phrases and thoughts that came up during the interview, were created during the interview. For transparency and to improve participants’ comfort, the researchers informed participants that they would be taking notes to help make sense of the data after the interview. The researchers muted their audio while taking notes during the interview to avoid noise distractions from typing.

Stage 3: Data immersion

To immerse themselves within the data, HS and two to three ‘analytic research team members’ listened/re-listened to or watched each interview recording to gain content immersion, generate field notes and independently note their analytical interpretations.

Analytic team

A large multidisciplinary research team was closely involved in the virtual data analysis. In total, the analytical research team consisted of seven team members from two different research institutes trained in qualitative research methods. Of note, one team member worked as a front-line clinician at one of the Sites in this study, while the remaining team members worked from home during the study period due to their organization’s work from home mandates. Also, all team members had met one another in person and worked together on other projects prior to the work from home pandemic-related restrictions. See Table 2 for details about the research team members and their contributions to the data analysis.

Table 2.

Analytical research team members.

Team Member Initials	Primary Position	Credentials	Educational discipline(s)	Clinical Background	Research Training	Contributions to Analysis (Methodology Stages Table 1)
HS	Assistant professor	MScOT, PhD	Rehabilitation	Occupational therapy (occupational therapist)	Qualitative rehab/health service research	Led analysis (stages 1–9)
TT	Clinician scientist	MD, MSc	Informatics, medicine	Medicine (physician)	Quantitative, qualitative, health service research	Contributed to and supported analysis (stages 1, 3, 5, 6, 8, 9)
RT	Research coordinator	RN, BScN	Nursing	Nursing (nurse)	Qualitative health service research	Contributed to and supported analysis (stages 1, 3, 5, 6, 8, 9)
AA	Research coordinator	MSc	Anthropology	Not applicable	Qualitative social sciences and health service research	Contributed to and supported analysis (stages 1, 3, 5, 6, 8, 9)
JXN	Research coordinator	BSc, MSc	Kinesiology, biology	Not applicable	Qualitative health service research	Contributed to and supported analysis (stages 1–3, 5, 6, 8, 9)
MLAN	Scientist	MA, PhD	Sociology, rehabilitation	Recreation therapy (recreation therapist)	Qualitative health service research	Finalize and verify (stage 9)
CSG	Scientist	Bkin, MA, PhD	Sociology	Not applicable	Qualitative health service research	Contributed to and supported analysis (stages 1, 3, 5, 6, 8, 9)

Stage 4: Map each individual interview on an individual map

HS created a visual summary map for each interview (i.e. ‘individual maps’; n = 16) on Microsoft PowerPoint; the maps summarized key ideas from each interview (see example in Figure 1). Some maps were created on a graphic design platform; however, it was difficult to modify the maps on this website, so the team shifted to Microsoft PowerPoint. A visual mapping approach was appropriate in our study as we intended to analyze large amounts of audio-recorded qualitative data and share the analysis virtually in a format conducive to Zoom screen sharing with the research team. HS created visual summary maps for each interview, which were reviewed by the team. As depicted in Figure 1, the centre of the individual maps displayed relevant details of each participant, such as their participant code, their role and discipline, work setting and age. Key concepts (e.g. impact of COVID, technology) related to the research questions and aims were depicted as branching off the central idea. Sub-concepts (e.g. discharge processes, caregivers) depicted smaller categories that described important aspects of the key concept. As a single word or phrase was insufficient to describe the sub-concepts adequately, we included bullet points underneath the sub-concepts to explain them. Lines were used to demonstrate the relationships between the central idea, key concepts, and sub-concepts. In cases where HS felt additional explanations of the sub-components should be discussed, but could not fit on the single slide, notes in the PowerPoint slide were included. These notes were used to facilitate discussions with the team during the team meetings within the subsequent stages of analysis.

Figure 1.

Example of individual map.

Stage 5: Modify individual maps

Six individual maps were reviewed synchronously during virtual team meetings, while two or three team members examined the remaining 10 individual maps asynchronously. Two to three analytical research team members reviewed each interview asynchronously, from which analytical interpretations were formed. Synchronous team meetings, lasting between one to 2 hours, were conducted using Zoom to discuss the team’s understanding of the raw data. The individual maps were sent to the team prior to the meeting. During the team meetings, HS provided an overview of the individual maps. Share screen was used to display the individual map during the discussion to ensure each team member could visually analyze the data. For instance, during the team discussions of the individual map for P2, HS reviewed the content within each key concept (represented within the squares in Figure 1) with members of the analytic research team. After this, the other team member(s) compared their own field notes to the content depicted on the individual map. After consulting her field notes, one team member indicated that stress was not captured within the individual map (Figure 1). The team member further explained that she heard a “sense of stress” when listening to P2’s interview recording, and P2 indicated that “colleagues were under more stress and burnout.” HS agreed and added this analytic interpretation to Figure 2. In this way, the individual maps were modified based on team members’ consensus about interpretations to ensure that essential ideas were not missed or overlooked. Regarding the asynchronous review of individual maps, HS sent the individual maps to team members and asked them to review and update them based on their own field notes. Due to several factors, including one team member practicing clinically and others having additional caretaking responsibilities with the shift to virtual schooling, asynchronous options to modify maps were highly valuable during this time as scheduling synchronous meetings to review each individual map became difficult. Different team members were assigned to analyze data by Site: HS, JXN, TT and HS, RT, AA, CSG analyzed data from Site A and Site B, respectively. However, one team member HS, the lead analyst, was involved in analyses for both Sites.

Figure 2.

Example of team revisions to individual map.

Stage 6: Identify patterns among individual maps

After reviewing all individual maps, synchronous team meetings were used to discuss the teams’ interpretations of the maps. We decided to explore patterns among and between maps by combining individual maps to form meta-maps based on provider setting (e.g. hospital providers and community providers), discipline (e.g. physicians) as well as Site (e.g. Site A and B).

Stage 7: Combine and collate individual maps to form meta-maps

HS created various meta-maps, which were used as a tool to facilitate team discussions in stage 8 to determine which combinations of data were the best fit. For example, meta-maps (n = 5) were created for: i) Site A hospital providers, ii) Site A community providers, iii) Site B hospital providers, iv) Site B community providers, and v) physicians to explore whether subgroup perspectives differed from the other providers.

Stage 8: Discuss and debrief about meta-maps

Inter-organizational differences between Site A and B were first analyzed separately by team members assigned to analyze data from each Site. During their review of meta-maps, the team reflected on the similarities and differences between the maps and possible reasons for differences. HS, who was involved in analyses at both Sites was responsible for reviewing the team’s analytic interpretations and had noted similarities among provider experiences at both Sites. This relationship was explored in a large team meeting with team members who were involved in the analyses at Site A and B. The two maps from hospital providers and community providers from Site A and B were then combined after a discussion where team members met to compare findings from Site A and B and formulated an overall interpretation based on similarities and differences between the sites. We did not note significant discipline-specific differences that would necessitate mapping physician data separately.

Stage 9: Finalize and verify the analysis

The last virtual team meeting was held to review and revise the themes and create a final map. The best-fit meta-maps were reviewed during the team meeting and further refined to enhance clarity. For instance, the team revised the wording and components within the themes and subthemes to ensure they reflected the essential components of the data and related to the research questions. Finally, two research team members extracted supporting quotes from the audio who conducted the interviews to illustrate the final themes.

Discussion: Methodological Insights

In this study, we provided an overview of a virtual team-based rapid qualitative method within a study that explored health care providers’ perspectives of how the COVID-19 pandemic has impacted hospital-to-home transitions. We strived to balance timeliness with rigour and analytical depth using the virtual team-based rapid qualitative method to explore healthcare providers’ experiences of transitional care during the COVID-19 pandemic. Below we discuss the lessons learned in applying this method and recommendations for changes that could enhance timeliness and rigour.

Lesson #1: Impact on timeliness

Timeliness was improved with the rapid and virtual aspects of this team-based rapid qualitative method. As it was rapid, we analyzed interview recordings directly, reducing costs, time and resources associated with generating and analyzing transcripts (Burgess-Allen & Owen-Smith, 2010; Coffey, 1996; Northcott, 1996). Timeliness is noteworthy as with traditional qualitative methods, resources dedicated to transcripts can often be substantial. For instance, in one study, 18 interviews that were 50-minutes in length resulted in 324 pages of data and over 300 hours to transcribe (Northcott, 1996). Another benefit of directly analyzing interview recordings is that it can reduce the risk of misinterpreting dialogue as the researchers can hear the participant’s tone, inflection, and pauses (Oliver et al., 2005). Finally, timeliness was enhanced by assigning specific team members to analyze Site A and B data with a team member familiar with both sub-teams (steps 2–6 in Table 1), allowing multiple analysis activities to co-occur.

While timeliness was increased in some regards, some unintended and unforeseen consequences decreased expediency. First, since the research team was trained and accustomed to analyzing transcribed data, research team members reflected that they found themselves transcribing parts of the interviews when listening to recordings rather than taking down their analytical interpretations. While this became easier over time, it decreased timeliness. Second, not transcribing the interview data increased timeliness in terms of turnaround time between data collection and analysis, but it took time to locate supporting quotations when reporting the results. In retrospect, we would recommend extracting key quotes while listening to the transcripts in stage 3 (Table 1) using time stamping so that quotes can be listened to again in context later, rather than during the higher-level analysis in stage 9 (Table 1), as it was time-consuming to locate specific quotes from the recordings based on memory and field notes. Another option to enhance timeliness in future studies is to use the Zoom transcription feature, which automatically transcribes recorded data (McMullin, 2021).

Lesson #2: Impact on rigour

We employed several strategies within this team-based rapid method to enhance the rigour of the results (see Table 3). Truth value and neutrality (Sandelowski, 1986; Tuckett, 2005) were enhanced through the team-based analysis, peer debriefing and review (stages 3, 5, 6, 8, 9 from Table 1), allowing us to assess the fit between participants’ views and analytical interpretations made, and ensure critical points were neither missed nor overemphasized (Janesick; Northcott, 1996; Sandelowski, 1986; Tuckett, 2005). In addition to supporting data analysis, the visual maps (stages 4–9 from Table 1) also generated an audit trail of the research process, helped us identify and record quotes supporting themes, and enhanced consistency and neutrality (Nowell et al., 2017). We also believe that leadership by one research team member who maintained a view of the entire process and led the creation and revision of the maps and meta-maps allowed us to enhance consistency. Notably, the leadership approach was collaborative, which allowed for a free flow of ideas and enhanced analytical depth. Concerns about whether researchers’ interpretations were grounded within the participants’ viewpoints were mitigated through illustrative quotes from participants to support the themes added in stage 9 (Table 1) (Tong et al., 2007). Finally, to strengthen applicability, we used maximal variation sampling (stage 1 from Table 1) at two distinct healthcare organizations and provided details of the settings and participants, as well as the researchers involved in the analysis within the study manuscript (Nowell et al., 2017; Sandelowski, 1986; Tuckett, 2005).

Table 3.

Rigour in the team-based rapid qualitative methodology.

Criteria to Enhance Rigour (Sandelowski, 1986)	Description (Tobin & Begley, 2004; Sandelowski, 1986)	Our Study Approach
Truth value	Fit between participants views and researchers' interpretations	Conducting a team-based analysis (peer debriefing and review), reflexivity and triangulation
Applicability	How applicability findings are to other contexts, settings or groups	Maximum variation sampling, providing details of the participants and settings
Consistency	The research is logical/clearly documented	Generating an audit trail with the visual maps and documenting how maps evolved
Neutrality	Ensuring findings stem from the data	Quotes, audit trail with visual maps, peer debriefing and review

While rigour was increased in some respects, some unintended and unforeseen consequences decreased rigour. The methods employed differed with different participants (e.g. phone vs. video interviews, duration of team meetings, number of reviewers analyzing each interview, asynchronous review of data). While some evolution of methods, particularly when utilizing new approaches, can be expected, this inconsistency and lack of structure may have reduced methodological rigour, namely consistency (Cypress, 2017). Thus, in future studies that employ this method, we recommend establishing roles and responsibilities early in the process and including a lead analyst as we had done (Giesen & Roeser, 2020). This is important as it ensures the timely production of results. Table 2 exemplifies the roles and responsibilities assigned to research team members to enhance rigour concepts of consistency, truth value and neutrality. We also recommend creating a strategy for modifying the maps (e.g. assigning a specific font/colour to each researcher) before stage 5 (Table 1), which we found to be another challenge in this process. Tables may be incorporated into the analysis process, which may help alleviate some of the challenges above and improve the audit trail. Finally, incorporating participants’ feedback on maps in stages 4 and 9 (Table 1) would help to enhance truth value and neutrality further.

Lesson #3: Analytical depth

While we did not note a loss of depth within the virtual interview, findings from a scoping review suggested that participant responses are shorter than face-to-face interviews (Davies et al., 2020). Our interviews ranged from 17 to 52 minutes (Singh, Under review); the shorter interview resulted from the provider being new in her position and unable to compare pandemic-related changes to the pre-pandemic context. Perhaps this may be due to the nature of the content being studied. The review examined studies that examined health and illness experiences (Davies et al., 2020), whereas our study examined provider experiences with care provision. While not encountered within this study, another possible factor to consider that could limit the depth of virtual interviews is the participant’s technical skills. If individuals are unfamiliar with the virtual mode of data collection (e.g. Zoom), it may be helpful to conduct a training session prior to the interview and create a plan in case of technical errors (Santhosh et al., 2021).

In terms of depth within our analysis, we used visual mapping, a technique used in rapid qualitative research. We selected this approach as it can be used to analyze large amounts of qualitative data, and we anticipated it would be suitable for a virtual team-based analysis (Burgess-Allen & Owen-Smith, 2010; Northcott, 1996; Petro, 2010). Petro and colleagues define a visual map as “a two-dimensional or representation or a visual diagram of the critical information and concepts being discussed,” wherein the central idea is located in the middle of the map. At the same time, the branches are topics relevant to the main idea (Petro, 2010). We initially feared that depth might be lost in the analysis due to concerns that team members may have difficulty following the discussion or ‘being on the same page’ during the virtual team meetings. However, the visual maps enhanced our analytic depth. Using visual maps can trigger formative and creative thinking by stimulating both sides of the brain, allowing researchers to present the entirety of the contents on a single page (Petro, 2010). We found visual maps were a valuable tool in our analysis as they allowed our team to i) summarize critical concepts from the recorded interviews on a single page, ii) review pertinent information during team meetings and keep the team focused on the topic, iii) trigger discussion about what was missing, iv) revise the visual maps based on input from multiple team members, and v) generate an audit trail.

Lesson #4: Virtual methods

We used virtual technologies and tools for data collection and team-based data analysis. Virtually conducted data collection was necessary for this project which occurred during the pandemic when public health restrictions required social distancing and restricted hospital visitor policies to minimize transmission of the virus (Herzik & Bethishou, 2021; Tang, 2020). Virtually conducted data collection and analysis allowed us to collect and analyze data while physically distanced. It was also a cost-effective alternative to face-to-face meetings since no travel time, or costs were required (Sah et al., 2020; Sedgwick & Spiers, 2009). Regarding data analysis, using video calls allowed for improved team communication and understanding through non-verbal communication, as well as through sharing information and data using screen share functions. This function allowed the team to co-work on analytic maps together in real-time, which enhanced the speed of analysis as well as the trustworthiness and credibility of interpretations.

The visual mapping supported an effective communication process to support synchronous data analysis (Milford et al., 2017). The visual maps were compatible with virtual team meetings as the maps could be readily displayed using the share screen function on Zoom. Although we expected to do the complete analysis synchronously (Burgess et al., 2021) during the virtual team meetings, this approach was time-consuming and not ideal for team members who worked clinically or may not be comfortable speaking in a virtual group environment. Instead, we integrated a combination of synchronous and asynchronous peer debrief and review activities for feasibility. This approach was timelier than working entirely synchronous as it provided the analytical team time to reflect and contribute thoughtful, analytical interpretations. With traditional qualitative methodologies, only a single research team member (i.e., the interviewer) often listens to the audio recordings while the rest of the team analyzes the resulting transcripts. In our study, we analyzed data by listening to recordings rather than reading transcripts, which allowed each team member to develop a more nuanced understanding of the data. Each was able to appreciate tones, pauses, and inflections. For instance, in one interview, we heard the high degree of stress in one participant’s voice, reflecting the impact of COVID-19 on their workload.

Notably, the method outlined in this paper can be scalable for multi-site studies. Beyond the COVID-19 context, the team-based rapid qualitative methods may be helpful in cross-country or international research as data collection and analysis were able to be completed virtually. Multicountry qualitative research teamwork is receiving increasing recognition in the health field (Milford et al., 2017; Wasser & Bresler, 1996). Teamwork is advantageous, particularly in multicountry qualitative analysis, as it yields more rigorous and relevant findings than individual-level analyses (DeCuir-Gunby et al., 2010; Milford et al., 2017). However, there are limited practical strategies to guide the conduct of multicountry team-based qualitative research (Milford et al., 2017; Wasser & Bresler, 1996). A valuable contribution to this area can be made through the practical approach, but one factor to consider is that the research team members in this study had met in person and had working relationships prior to this entirely virtual analysis. Additional strategies for rapport building among virtual teams that do not have these pre-existing relationships may be required to support strong communication and comfort with providing critical feedback during the analysis required in this method (Gilson et al., 2014).

Some unanticipated outcomes of virtually conducted data collection and analysis and considerations warrant attention. While concerns among the literature that audio interviews can lead to misinterpretations due to a lack of visual cues have been raised (Novick, 2008), we anticipated that these could be mitigated through video interviews (Janghorban et al., 2014). However, nearly all participants in our study (n = 15) preferred audio rather than video interviews, including two participants that started the meeting with video enabled but turned it off for the recording. Although we did not explicitly inquire about participants’ reasoning, this may be due to concerns about confidentiality, feeling uncomfortable with video recordings and/or lack of privacy in their surrounding environment (e.g. being in a shared space with their colleagues at the time of the interview). Researchers should consider participants’ location and offer options for audio-only to accommodate these situations.

In the analysis stage, we experienced confusion about the level of detail to include in the maps to capture all relevant information on a single Powerpoint slide. For clarity, we recommend explicitly discussing the level of detail to fit within the maps and including field notes to supplement or expand on key concepts identified in the maps. We also recommend using virtual tools that offer larger canvases (e.g. Miro boards (Miro, 2022)) so that researchers are not limited to a single page. Finally, since this method was applied to a virtual project, it may not apply to data collected and analyzed within in-person settings.

We have modified the method to include our reflections in Table 4. Two important considerations when interpreting these findings are: i) this study occurred during the pandemic when face-to-face data collection and analysis were restricted, and ii) production of timely knowledge was necessary given the rapidly evolving pandemic context. However, factors such as these may not be relevant in other contexts or situations and should be accounted for in the justification of methodological choice (Hanley, 2011).

Table 4.

Practice recommendations.

Methodology Steps	Practice Recommendations (Description Revised Based on Lessons Learned are Added in Bolded Italics)
Stage 1: Recruit and screen	Recruitment using purposeful and snowball sampling
Stage 2: Data collection	Generate field notes during remote interviews
Stage 3: Data immersion	Listen to each interview, take field notes and extract supporting quotes
Stage 4: Map each individual interviews on an individual map	Explicitly discuss the level of detail to include within the maps, create a map for each interview and ask participants for their feedback on the corresponding map
Stage 5: Modify individual maps	Create a strategy for modifying the maps, modify and confirm the content of each mind map based on teams' field notes and analytical interpretations
Stage 6: Identify patterns among individual maps	Determine which participants had similar/different experiences by identifying the combination of maps that are similar
Stage 7: Combine and collate individual maps to form meta-maps	Combine/collate maps for participants with similar experiences in various ways to determine the best fit (e.g. site, sector, provider discipline)
Stage 8: Discuss and debrief about meta-maps	Discuss/debrief about which maps fit best and were easiest to combine, identify the combination of maps that best work together
Stage 9: Finalize and verify the analysis	Finalize meta-maps to reflect all findings, extract additional verbatim quotes from audio-recordings (if needed), ensure terminology of themes appropriately captures analytical interpretations and ask participants for their feedback on the final map

Conclusions

In conclusion, the turbulent COVID-19 pandemic forced researchers to explore alternative approaches to traditional qualitative research to support rapid and rigorous findings. The lessons learned from using the team-based rapid qualitative method presented here may be helpful for other virtual qualitative studies that aim to generate rigorous and in-depth results in a timely manner.

Figure 3.

Example of a meta-map.

Footnotes

Acknowledgments

We would like to thank the participants for dedicating their time to this study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Canadian Institutes of Health Research, Team Grant: Transitions in Care (FRN 165733) and the March of Dimes Paul J.J. Martin Early Career Professorship to HS. The funder did not have a role in the study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.

ORCID iD

Hardeep Singh

References

Abdul Majid

M. A.

Othman

Mohamad

S. F.

Lim

Yusof

(2017). Piloting for interviews in qualitative research: Operationalization and lessons learnt. International Journal of Academic Research in Business and Social Sciences, 7(4), 1073-1080. https://doi.org/10.6007/IJARBSS/v7-i4/2916

Anghelescu

B. A.

Bruno

Martino

Roach

(2021). Effects of the COVID-19 pandemic on Parkinson's disease: A single-centered qualitative study. Can J Neurol Sci, 49(2), 1-13. https://doi.org/10.1017/cjn.2021.70

Archibald

Ambagtsheer

Casey

Lawless

(2019). Using Zoom videoconferencing for qualitative data collection: Perceptions and experiences of researchers and participants. International Journal of Qualitative Methods, 18, 160940691987459. https://doi.org/10.1177/1609406919874596

Beebe

(2014). Rapid qualitative inquiry: A field guide to team-based assessment. Rowman & Littlefield

Bröer

Veltkamp

Bouw

Vlaar

Borst

de Sauvage Nolting

(2021). From danger to uncertainty: Changing health care practices, everyday experiences, and temporalities in dealing with COVID-19 policies in the Netherlands. Qual Health Res, 31(9), 1751-1763. https://doi.org/10.1177/10497323211005748

Brooks

(2019). Health care provider experiences of and perspectives on medical assistance in dying: A scoping review of qualitative studies. Can J Aging, 38(3), 384-396. https://doi.org/10.1017/s0714980818000600

Burgess

Jongbloed

Vorobyova

Grieve

Lyndon

Wesseling

Salters

Hogg

R. S.

Parashar

Pearce

M. E.

(2021). The “Sticky Notes” method: Adapting interpretive description methodology for team-based qualitative analysis in community-based participatory research. Qual Health Res, 31(7), 1335-1344. https://doi.org/10.1177/10497323211002489

Burgess-Allen

Owen-Smith

(2010). Using mind mapping techniques for rapid qualitative data analysis in public participation processes. Health Expect, 13(4), 406-415. https://doi.org/10.1111/j.1369-7625.2010.00594.x

Coffey

Atkinson

(1996). Making sense of qualitative data: Complementary research strategies. Sage Publications, Inc

10.

Cooper

Endacott

(2007). Generic qualitative research: A design for qualitative research in emergency care? Emerg Med J, 24(12), 816-819. https://doi.org/10.1136/emj.2007.050641

11.

Crawford

Serhal

(2020). Digital health equity and COVID-19: The innovation curve cannot reinforce the social gradient of health. Journal of medical Internet research, 22(6), e19361-e19361. https://doi.org/10.2196/19361

12.

Crawley

(2020). Ontario's 2nd wave of COVID-19 forecast to peak in October. Retrieved from https://www.cbc.ca/news/canada/toronto/ontario-covid-19-second-wave-cases-modelling-projections-1.5739411

13.

Cypress

B. S.

(2017). Rigor or reliability and validity in qualitative research: Perspectives, strategies, reconceptualization, and recommendations. Dimensions of Critical Care Nursing, 36(4)-263. https://doi.org/10.1097/dcc.0000000000000253. Retrieved from https://journals.lww.com/dccnjournal/Fulltext/2017/07000/Rigor_or_Reliability_and_Validity_in_Qualitative.6.aspx

14.

Davies

LeClair

K. L.

Bagley

Blunt

Hinton

Ryan

Ziebland

(2020). Face-to-Face compared with online collected accounts of health and illness experiences: A scoping review. Qualitative Health Research, 30(13), 2092-2102. https://doi.org/10.1177/1049732320935835

15.

DeCuir-Gunby

J. T.

Marshall

P. L.

McCulloch

A. W.

(2010). Developing and using a codebook for the analysis of interview data: An example from a professional development research project. Field Methods, 23(2), 136-155. https://doi.org/10.1177/1525822X10388468

16.

Fridell

Edwin

von Schreeb

Saulnier

D. D.

(2020). Health system resilience: What are we talking about? A scoping review mapping characteristics and keywords. Int J Health Policy Manag, 9(1), 6-16. https://doi.org/10.15171/ijhpm.2019.71

17.

Gale

R. C.

Erhardt

Bounthavong

Reardon

C. M.

Damschroder

L. J.

Midboe

A. M.

(2019). Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implementation Science, 14(1), 11. https://doi.org/10.1186/s13012-019-0853-y

18.

Giesen

Roeser

(2020). Structuring a team-based approach to coding qualitative data. International Journal of Qualitative Methods, 19, 1609406920968700. https://doi.org/10.1177/1609406920968700

19.

Gilson

L. L.

Maynard

M. T.

Jones Young

N. C.

Vartiainen

Hakonen

(2014). Virtual teams research: 10 Years, 10 themes, and 10 opportunities. Journal of Management, 41(5), 1313-1337. https://doi.org/10.1177/0149206314559946

20.

Gisselbaek

Hudelson

Savoldelli

G. L.

(2021). A systematic scoping review of published qualitative research pertaining to the field of perioperative anesthesiology. Can J Anaesth, 68(12), 1811-1821. https://doi.org/10.1007/s12630-021-02106-y

21.

Gleeson

Calderon

Swami

Deighton

Wolpert

Edbrooke-Childs

(2016). Systematic review of approaches to using patient experience data for quality improvement in healthcare settings. BMJ Open, 6(8), e011907. https://doi.org/10.1136/bmjopen-2016-011907

22.

Gray

Wong

Rempel

Cook

(2020). Expanding qualitative research interviewing strategies: Zoom video communications. Qualitative Report, 25. Article 9 https://doi.org/10.46743/2160-3715/2020.4212

23.

Guest

MacQueen

K.M.

(2008). Handbook for team-based qualitative research. AltaMira Press

24.

Haldane

De Foo

Abdalla

S. M.

Jung

A.-S.

Tan

Chua

Verma

Shrestha

Singh

Perez

Bartos

Mabuchi

Bonk

McNab

Werner

G. K.

Panjabi

Nordstrom

Legido-Quigley

(2021). Health systems resilience in managing the COVID-19 pandemic: Lessons from 28 countries. Nature Medicine, 27(6), 964-980. https://doi.org/10.1038/s41591-021-01381-y

25.

Hamilton

A. B.

Finley

E. P.

(2019). Qualitative methods in implementation research: An introduction. Psychiatry Research, 280, 112516-112516. https://doi.org/10.1016/j.psychres.2019.112516

26.

Hanley

(2011). Virtual Data Generation: Qualitative research, computers and counselling psychology. Counselling Psychology Review, 26(4)

27.

Herzik

K. A.

Bethishou

(2021). The impact of COVID-19 on pharmacy transitions of care services. Res Social Adm Pharm, 17(1), 1908-1912. https://doi.org/10.1016/j.sapharm.2020.10.017

28.

Institute of Medicine (US) Committee on the Health Professions Education Summit (2003). Health Professions education: A bridge to quality. In Greiner AC

K. E.

(Ed), Challenges facing the health system and implications for educational reform. National Academies Press

29.

Janesick

V. J.

(1996). Peer debriefing. The blackwell encyclopedia of sociology

30.

Janghorban

Latifnejad Roudsari

Taghipour

(2014). Skype interviewing: The new generation of online synchronous interview in qualitative research. Int J Qual Stud Health Well-Being, 9(1), 24152. https://doi.org/10.3402/qhw.v9.24152

31.

Jazieh

A. R.

Kozlakidis

(2020). Healthcare transformation in the post-coronavirus pandemic era. Front Med (Lausanne), 7, 429. https://doi.org/10.3389/fmed.2020.00429

32.

Johnson

J. L.

Adkins

Chauvin

(2020). A review of the quality indicators of rigor in qualitative research. American Journal of Pharmaceutical Education, 84(1), 7120. https://doi.org/10.5688/ajpe7120

33.

Lincoln

Guba

Pilotta

J. J.

(1985). Naturalistic inquiry. International Journal of Intercultural Relations, 9(4), 438-439. https://doi.org/10.1016/0147-1767(85)90062-8

34.

Lobe

Morgan

Hoffman

K. A.

(2020). Qualitative data collection in an era of social distancing. International Journal of Qualitative Methods, 19, 1609406920937875. https://doi.org/10.1177/1609406920937875

35.

Luciani

Jack

S. M.

Campbell

Orr

Durepos

Strachan

Di Mauro

(2019). An introduction to qualitative health research. Prof Inferm, 72(1), 60-68

36.

MacCarthy

Izenberg

Barreras

J. L.

Brooks

R. A.

Gonzalez

Linnemayr

(2020). Rapid mixed-methods assessment of COVID-19 impact on Latinx sexual minority men and Latinx transgender women. Plos One, 15(12), e0244421. https://doi.org/10.1371/journal.pone.0244421

37.

McMullin

(2021). Transcription and qualitative methods: Implications for third sector research. VOLUNTAS: International journal of voluntary and nonprofit organizations. https://doi.org/10.1007/s11266-021-00400-3

38.

McNall

Foster-Fishman

P. G.

(2007). Methods of rapid evaluation, assessment, and appraisal. American Journal of Evaluation, 28(2), 151-168. https://doi.org/10.1177/1098214007300895

39.

Merriam-Webster (2022). Small talk. Retrieved from https://www.merriam-webster.com/dictionary/small_talk

40.

Milford

Kriel

Njau

Nkole

Gichangi

Cordero

J. P.

Smit

J. A.

Steyn

P. S.

(2017). Teamwork in qualitative research: Descriptions of a multicountry team Approach. International Journal of Qualitative Methods, 16(1), 1609406917727189. https://doi.org/10.1177/1609406917727189

41.

Miro . (2022). Miro. Retrieved from https://miro.com/?utm_source%3Dgoogle%26utm_medium%3Dcpc%26utm_campaign%3DS|GOO|BRN|CA|EN-EN|Brand|Exact%26utm_adgroup=%26utm_custom%3D10897982558%26utm_content%3D458236618554%26utm_term%3Dmiro_board%26matchtype=e%26device=c%26location=9000844&gclid=CjwKCAiAvaGRBhBlEiwAiY-yMJKyTMh65o6tzH65i5r_DMGfiwkKCqad0sfF-agggS3K2Gc7hjDS7hoCKx8QAvD_BwE

42.

Morse

Barrett

Mayan

Olson

Spiers

(2002). Verification strategies for establishing reliability and validity in qualitative research. International Journal of Qualitative Methods, 1(2), 13-22. https://doi.org/10.1177/160940690200100202

43.

Morse

J. M.

(2012). Qualitative health research. Left Coast Press Inc

44.

Munshi

Evans

Razak

(2021). The case for relaxing no-visitor policies in hospitals during the ongoing COVID-19 pandemic. Cmaj, 193(4), E135-e137. https://doi.org/10.1503/cmaj.202636

45.

Nielsen

(2020). A timeline of COVID-19 in Ontario. Retrieved from https://globalnews.ca/news/6859636/ontario-coronavirus-timeline/

46.

Northcott

(1996). Cognitive mapping: An approach to qualitative data analysis. NT Research, 1(6), 456-463. https://doi.org/10.1177/174498719600100610

47.

Novick

(2008). Is there a bias against telephone interviews in qualitative research? Research in Nursing & Health, 31(4), 391-398. https://doi.org/10.1002/nur.20259

48.

Nowell

L. S.

Norris

J. M.

White

D. E.

Moules

N. J.

(2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1), 1609406917733847. https://doi.org/10.1177/1609406917733847

49.

Oliver

D. G.

Serovich

J. M.

Mason

T. L.

(2005). Constraints and opportunities with interview transcription: Towards reflection in qualitative research. Soc Forces, 84(2), 1273-1289. https://doi.org/10.1353/sof.2006.0023

50.

Padgett

D. K.

(2016). Qualitative methods in social work research. SAGE Publications

51.

Petro

N. J.

(2010). Hate taking notes? Try mind mapping. GPSolo, 27(4), 20-23. Retrieved from https://www-jstor-org.web.bisu.edu.cn/stable/23630192

52.

Rawaf

Allen

L. N.

Stigler

F. L.

Kringos

Quezada Yamamoto

van Weel

On behalf of the Global Forum on Universal Health Coverage and Primary Health Care (2020). Lessons on the COVID-19 pandemic, for and by primary care professionals worldwide. Eur J Gen Pract, 26(1), 129-133. doi: https://doi.org/10.1080/13814788.2020.1820479

53.

Riley

W. T.

Glasgow

R. E.

Etheredge

Abernethy

A. P.

(2013). Rapid, responsive, relevant (R3) research: A call for a rapid learning health research enterprise. Clin Transl Med, 2(1), 10. https://doi.org/10.1186/2001-1326-2-10

54.

Roberti

Jorro

Rodríguez

Belizán

Arias

Ratto

M. E.

Reina

Ini

Loudet

García-Elorrio

(2021). Theory-driven, rapid formative research on quality improvement intervention for critical care of patients with COVID-19 in Argentina. Glob Qual Nurs Res, 8, 23333936211015660. https://doi.org/10.1177/23333936211015660

55.

Roberts

J. K.

Pavlakis

A. E.

Richards

M. P.

(2021). It’s more complicated than it seems: Virtual qualitative research in the COVID-19 era. International Journal of Qualitative Methods, 20, 16094069211002959. https://doi.org/10.1177/16094069211002959

56.

Sah

L. K.

Singh

D. R.

Sah

R. K.

(2020). Conducting qualitative interviews using virtual communication tools amid COVID-19 pandemic: A learning opportunity for future research. JNMA J Nepal Med Assoc, 58(232), 1103-1106. https://doi.org/10.31729/jnma.5738

57.

Sandelowski

(1986). The problem of rigor in qualitative research. ANS Adv Nurs Sci, 8(3), 27-37. https://doi.org/10.1097/00012272-198604000-00005

58.

Santhosh

Rojas

Lyons

(2021). Zooming into focus groups: Strategies for qualitative research in the era of social distancing. ATS Scholar, 2, 176-184. ats-scholar.202 https://doi.org/10.34197/ats-scholar.2020-0127PS

59.

Sedgwick

Spiers

(2009). The use of videoconferencing as a medium for the qualitative interview. International Journal of Qualitative Methods, 8(1), 1-11. https://doi.org/10.1177/160940690900800101

60.

Shreffler

Petrey

Huecker

(2020). The impact of COVID-19 on healthcare worker wellness: A scoping review. West J Emerg Med, 21(5), 1059-1066. https://doi.org/10.5811/westjem.2020.7.48684

61.

Silverio

Hall

Sandall

(2020). Time and qualitative research: Principles, pitfalls, and perils.

62.

Singh

Steele Gray

Nelson

MLA.

Nie

JX.

Thombs

Armas

Tang

(Under reviewn.d.). The experiences of hospital and community providers with hospital to home care transitions during the disruptive COVID-19 pandemic. Plos One.

63.

Singleton

Dowrick

Manby

Fillmore

Syverson

Lewis-Jackson

Uddin

Sumray

Bautista-Gonzalez

Johnson

Vindrola-Padros

(2021). UK healthcare workers' experiences of major system change in elective surgery during the COVID-19 pandemic: Reflections on rapid service adaptation. Int J Health Policy Manag. https://doi.org/10.34172/ijhpm.2021.101

64.

Slattery

Saeri

A. K.

Bragge

(2020). Research co-design in health: A rapid overview of reviews. Health Research Policy and Systems, 18(1), 17. https://doi.org/10.1186/s12961-020-0528-9

65.

Tang

Zhou

L. L.

(2020). Covid-19: An accidental catalyst for change in the canadian health care system. British Columbia Medical Journal, 62(7), 242-246

66.

Taylor

(2021). Healthcare technology in context: Lessons for telehealth in the age of COVID-19. Palgrave MacMillan. https://doi.org/10.1007/978-981-16-4075-9

67.

Taylor

Henshall

Kenyon

Litchfield

Greenfield

(2018). Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis. BMJ Open, 8(10), e019993. https://doi.org/10.1136/bmjopen-2017-019993

68.

Thorne

(2016). Interpretive description. Qualitative research for applied practice. Left Coast Press

69.

Thorne

Kirkham

S. R.

O'Flynn-Magee

(2004). The analytic challenge in interpretive description. International Journal of Qualitative Methods, 3(1), 1-11. https://doi.org/10.1177/160940690400300101

70.

Tobin

G. A.

Begley

C. M.

(2004). Methodological rigour within a qualitative framework. Journal of Advanced Nursing, 48(4), 388-396. https://doi.org/10.1111/j.1365-2648.2004.03207.x

71.

Tong

Sainsbury

Craig

(2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. Int J Qual Health Care, 19(6), 349-357. https://doi.org/10.1093/intqhc/mzm042

72.

Tort-Nasarre

Alvarez

Galbany-Estragués

Subías-Miquel

Vázquez-Segura

Marre

Romeu-Labayen

(2021). Front-line nurses' responses to organisational changes during the COVID-19 in Spain: A qualitative rapid appraisal. J Nurs Manag, 29(7), 1983-1991. https://doi.org/10.1111/jonm.13362

73.

Tremblay

Turcotte

Touati

Poder

T. G.

Kilpatrick

Bilodeau

Roy

Richard

P. O.

Lessard

Giordano

É.

(2022). Development and use of research vignettes to collect qualitative data from healthcare professionals: A scoping review. BMJ Open, 12(1), e057095. https://doi.org/10.1136/bmjopen-2021-057095

74.

Tremblay

Castiglione

Audet

L.-A.

Desmarais

Horace

Peláez

(2021). Conducting qualitative research to respond to COVID-19 challenges: Reflections for the present and beyond. International Journal of Qualitative Methods, 20, 16094069211009679. https://doi.org/10.1177/16094069211009679

75.

Tuckett

A. G.

(2005). Part II. Rigour in qualitative research: Complexities and solutions. Nurse Researcher, 13(1), 29-42. https://doi.org/10.7748/nr2005.07.13.1.29.c5998

76.

Vindrola-Padros

Chisnall

Cooper

Dowrick

Djellouli

Symmons

S. M.

Martin

Singleton

Vanderslott

Vera

Johnson

G. A.

(2020). Carrying out rapid qualitative research during a pandemic: Emerging lessons from COVID-19. Qual Health Res, 30(14), 2192-2204. https://doi.org/10.1177/1049732320951526

77.

Vindrola-Padros

Johnson

G. A.

(2020). Rapid techniques in qualitative research: A critical review of the literature. Qualitative Health Research, 30(10), 1596-1604. https://doi.org/10.1177/1049732320921835

78.

Wasser

J. D.

Bresler

(1996). Working in the interpretive zone: Conceptualizing collaboration in qualitative research teams. Educational Researcher, 25(5), 5-15. https://doi.org/10.2307/1176204

79.

Watkins

D. C.

(2017). Rapid and rigorous qualitative data analysis: The “RADaR” technique for applied research. International Journal of Qualitative Methods, 16(1), 1609406917712131. https://doi.org/10.1177/1609406917712131

80.

Watkins

D. C.

Abelson

J. M.

Jefferson

S. O.

(2013). Their depression is something different it would have to Be”: Findings from a qualitative study of black women’s perceptions of depression in black men. American Journal of Men's Health, 7(4 suppl), 45S-57S. https://doi.org/10.1177/1557988313493697

81.

Watkins

D. C.

Gioia

(2015). Mixed methods research. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199747450.001.0001

82.

Wilson

Aguilar

(2021). Ontario issues emergency orders to bolster hospital capacity as cases soar. Retrieved from https://toronto.ctvnews.ca/ontario-issues-emergency-orders-to-bolster-hospital-capacity-as-cases-soar-1.5381845

83.

World Health Organization (2021). Global strategy on digital health 2020-2025. Retrieved from https://www.who.int/docs/default-source/documents/gs4dhdaa2a9f352b0445bafbc79ca799dce4d.pdf