Abstract
Qualitative researchers are under increasing time demands to rapidly collect, analyze, and disseminate the results of their findings. Adaptations to qualitative methods may be required to enable the use of timely and relevant qualitative data across multiple disciplinary settings. The aim of this review is to briefly explore the ways in which data collection and analysis methods have been adapted in qualitative research to deal with short study timeframes. We carried out a two-phased systematic review of the literature and determined there were six primary reasons why rapid techniques were used: (a) reduce time, (b) reduce cost, (c) increase the amount of collected data, (d) improve efficiency, (e) improve accuracy, and (f) obtain a closer approximation to the narrated realities of research participants. In addition, we analyzed the characteristics of the articles, how traditional methods were adapted and evaluated, the benefits and limitations of using rapid techniques, and future recommendations.
Introduction
Many qualitative researchers currently work in environments that require the rapid sharing of findings. This time pressure is particularly relevant for researchers working in health care who are interested in using research findings to inform changes in policy and practice, as the needs and priorities of health care organizations might change throughout the study and the health care landscape might be subjected to external pressures such as budget cuts and shifts in public policies (Skillman et al., 2019; Vindrola-Padros & Vindrola-Padros, 2018). In these situations, the timeliness of the sharing of findings is a critical determinant of their use (McNall & Foster-Fishman, 2007).
Qualitative research has traditionally been represented as an approach requiring long periods of time for data collection and analysis. However, over the past three decades, significant work has been carried out to develop rapid research methods and other techniques to speed up data collection and analysis. Researchers have developed approaches such as rapid appraisals (Beebe, 1995), Rapid Assessment Procedures (RAP; Scrimshaw & Hurtado, 1988), Rapid Ethnographic Assessments (REA; Bentley et al., 1988), the RARE model (Brown et al., 2008), Rapid Qualitative Inquiry (RQI; Beebe, 2014), quick ethnography (Handwerker, 2001) and short-term ethnographies (Pink & Morgan, 2013) among others. Most of these approaches rely on the use of a team of researchers, the combination of multiple methods for data collection and triangulation during analysis, and iterative processes of data collection and analysis to reduce the time required for research (McNall & Foster-Fishman, 2007).
In addition to these approaches, researchers have also developed techniques to reduce the amount of time required for qualitative data collection and analysis. Techniques might entail reducing the amount of time required for the transcription of interviews or combining data analysis methods with data collection to deliver real-time findings. These techniques have been used as part of rapid studies, but are also frequently used in longer term studies to speed up particular aspects of the research. Despite the wide range of rapid techniques available for qualitative research, these have not been synthesized and critically analyzed. The aim of this review is to explore the ways in which data collection and analysis methods have been adapted in qualitative research to deal with short study timeframes. Furthermore, we aim to expose more junior qualitative researchers to novel methods being applied to the field today in the hopes that successful innovations can be built upon and strengthened in future. We present the findings of a systematic review of these techniques, analyze the benefits and limitations of using the techniques in practice, and propose recommendations for their further development. A systematic review approach was chosen to produce a detailed summary of available qualitative research in the field of rapid data collection and analysis techniques.
Materials and Methods
Design
We carried out a phased systematic review of the literature (Tricco et al., 2017). The first phase consisted of a general search for articles describing rapid data collection and analysis methods. Articles were reviewed by both authors to determine if they met the inclusion criteria and to identify any additional terms used to describe rapid data collection and analysis methods. Each article was reviewed in depth, and when new words were identified in the text to explain the rapid tool or technique, these were added to the search strategy for Phase 2. The second phase included a second search using these new terms. Articles from both phases (which met the inclusion criteria) were included in the review. The search strategies for both phases can be found in Online Appendix A. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for the design and reporting of the review.
Research Questions
How are rapid data collection and analysis techniques defined?
How are adaptations of methods (to deal with short study timeframes) defined and justified?
What are the stages of research that normally experience modifications to adapt to rapid timeframes?
What are the benefits and limitations of using rapid techniques?
What recommendations can be made for the future development of rapid techniques?
Search Strategy
Each search phase involved running the search strategies in Online Appendix A through multiple databases: MEDLINE, CINAHL Plus, Web of Science and ProQuest Central. The searches were conducted in January 2018. An additional search was conducted in October 2019 using the same search strategies to update the search and identify any new publications. Results were combined into RefWorks, and duplicates were removed. The reference lists of included articles were screened to identify additional relevant publications.
Selection and Inclusion Criteria
The results from the search were imported into RefWorks to remove duplicates and manage the references through title, abstract, and full-text review. Both authors reviewed the articles independently and discussed any discrepancies until consensus was reached on which articles to include in the full-text review. No exclusions were made based on date or language.
Inclusion criteria
Published in peer-reviewed journal.
Includes an adaptation of data collection or analysis methods (self-identified adaptations).
Involves a rapid study (self-identified as rapid).
Exclusion criteria
Study is not a rapid study (self-identified as rapid).
Methods are not adapted (self-identified as adapted).
Research is not published in a peer-reviewed journal.
Data Extraction, Management, and Synthesis
The included articles were analyzed using a data extraction form developed in RedCap (Online Appendix B). The form was developed after the initial screening of full-text articles. It was then piloted independently by both authors using a random sample of five articles. Disagreements between them were discussed until consensus was reached. The form was revised based on the findings from the pilot screening. Data were exported from RedCap and analyzed in relation to our research questions (see above). A narrative, descriptive synthesis of findings was then carried out by both authors. Risk of bias assessment was not applicable for this review.
Results
Identification of Articles
The search was divided in two phases. Phase 1 was based on an open search strategy exploring various terms related to rapid research (for the full strategy, see Online Appendix A). This initial search yielded 109 published articles (Supplemental Figure 1). These were screened based on the title and type of article, resulting in 13 articles. Screening based on abstracts left 10 articles for full-text review. This phase in screening led to seven articles that met the inclusion criteria. These seven articles were reviewed, and additional keywords were identified. A second search phase was carried out with these additional terms. The second phase yielded 119 published articles. Screening by the title and type of article led to 15 articles. Screening based on abstracts generated eight articles for full-text review. Six articles met the inclusion criteria. Both search phases led to 13 articles. Both search strategies were used again in 2019 to update the search, and an additional five articles were found. A total of 18 articles were included in the review. The bibliographies of all articles were reviewed, but no additional articles were identified for inclusion.
Characteristics of the Included Articles
The dates of publication ranged from 1992 to 2018, with more than half of the articles published from 2010 onwards. Eight articles were developed by authors in the United States, three in Canada, three in the United Kingdom, three in Australia, and one in New Zealand. More than half of the rapid techniques described in the articles were used for data analysis. Three articles described techniques that can be used for data collection, and four articles described techniques that can be used for both collection and analysis (see Table 1).
Summary of Article Characteristics.
Note. RA: rapid analysis.
What are the Reasons Why Rapid Techniques Are Used?
There were six main reasons why the rapid techniques were used: (a) reduce time, (b) reduce cost, (c) increase the amount of collected data (due to the reduction of time required to collect it), (d) improve efficiency, (e) improve accuracy, and (f) obtain a closer approximation to the narrated realities of research participants. The need to reduce the time for research was the most frequently mentioned reason for the use of rapid techniques, but some techniques, such as the analysis of interview recordings, were also focused on reducing the errors and interpretation biases produced by transcription. Mind mapping techniques also sought to improve the accuracy of the research by allowing participants to cross-check interpretations in real time as the mind maps were developed.
It should be noted that some (n = 6) of the articles reviewed discussed the purpose of a rapid study methodology beyond the original study objectives. Among those that did, research findings were additionally used for the following purposes: (a) to demonstrate that rapid methods of data analysis can be used as reliably and systematically as more conventional methods (Gravois et al., 1992; Greenwood et al., 2017; Taylor et al., 2018), (b) to demonstrate a more contemporary method for using technology for data collection, analysis, and dissemination (Markle et al., 2011; Watkins, 2017), and (c) to include the meaningful participation of research participants into the data analysis process (Burgess-Allen & Owen-Smith, 2010).
How Are Conventional Methods Being Adapted?
Despite the diversity of reasons for using rapid data analysis techniques in the articles reviewed, there were two overarching themes for how conventional methods (defined as those that are normally used) were best adapted to suit rapid timeframes: (a) eliminate the creation of transcripts or (b) speed up the process of creating transcripts. The first theme involved by-passing the creation of time-consuming transcripts through simultaneous collection and analysis of data (e.g., mind mapping) or by coding data directly from its audio/visual source—or a combination of both (Burgess-Allen & Owen-Smith 2010; Gravois et al., 1992; Halcomb & Davidson, 2006; Markle et al., 2011; Neal et al., 2015; Petro, 2010; Tattersall & Vernon, 2007). In regard to the latter technique, direct coding of either full or selected segments of audio from interview/discussion sessions was most often described.
About half of the articles reviewed preferred to eliminate the transcript production phase entirely; however, there were four articles which still privileged the creation of verbatim transcripts prior to coding and analyzing text-based data (Anderson, 1998; Johnson, 2011; Park & Zeanah, 2005; Scott et al., 2009). This second group of articles involved the use of specialized equipment (e.g., voice recognition software) or persons with specialized knowledge and equipment (e.g., court reporters or scribes) to speed up the process of creating transcripts prior to beginning data analysis. One article relied on the use of transcripts but proposed a method for data analysis based on the use of tables and spreadsheets to undergo a “data reduction” process (Watkins, 2017). In summary, all articles prioritized either eliminating or speeding up the process of transcribing data to adapt more conventional data collection and analysis methods for a rapid timeframe.
Are These Techniques Evaluated?
Less than half of the articles reviewed (n = 8) evaluated their technique in comparison with more conventional methods of data collection and analysis. We divided the evaluation techniques into studies for which transcripts were not produced and those in which transcripts were produced. For studies in which transcripts were not produced, testing was done on the reliability of codes and/or reliability of identified themes by taking a selection of data (e.g., two focus group discussions) and producing full transcripts which were then coded and analyzed conventionally. Data produced by analyzing full transcripts were then compared with the codes/themes generated without the production of transcripts via more rapid methods (e.g., directly coding audio). The authors from one of these studies concluded that mind mapping presented the same “broad” themes as more conventional methods with the added benefit of allowing participants to be a part of the analysis process (Burgess-Allen & Owen-Smith, 2010). Gravois et al. (1992) provide a detailed discussion of establishing reliability for coding and similarly concluded that the reliability of coded data from transcripts and direct coding from audio fell within the range of “acceptable reliability.” See Gravois et al. (1992) for a detailed discussion on how reliability was assessed for coding themes directly from audiotape versus coding from transcripts.
Two articles used an evaluation technique on transcripts produced via voice recognition software, where direct side-by-side comparisons were made between a selection of data using the listen-and-type method and the software produced transcripts. Johnson (2011) did not find an increase in transcription speed with voice recognition software but indicated that it might ease physical and mental stress normally associated with the listen-and-type method of transcription. Park and Zeanah (2005) also found that voice recognition software led to the creation of transcripts in a comparable time to transcripts developed by a competent typist but highlighted the software might be useful for slow typists or researchers with disabilities. Four recent articles compared the findings obtained from analysis based on direct coding from focus group recordings with coding from transcripts (Greenwood et al., 2017), rapid and conventional or in-depth analysis techniques using the same data set (Gale et al., 2019; Taylor et al., 2018), and scribing (note taking by an external observer) during interviews versus interviews transcribed verbatim (Eaton et al., 2019). These comparative studies found that rapid techniques generated similar findings to conventional techniques but required experienced researchers.
What Are the Benefits and Limitations of Using These Techniques?
Several benefits and limitations of the rapid techniques utilized by authors were described. Specifically, benefits described by the majority of articles—beyond the obvious reduction in time between data collection and analysis—included the ability to collect more data and include more research participants (due to the reduction in time), decrease the cost of research (by reducing the amount of time researchers needed to be hired as well as eliminating high transcription costs), provide greater insight into data beyond what can be learned from reading a transcript/maintaining nonverbal information (e.g., pauses in speech, volume fluctuations, when coding directly from a video or voice recording), potentially eliminate the interpretative bias in creating a transcript (by eliminating intermediaries who might create a transcript without having direct contact with the source of information), limit the opportunity for human error, increase the “authenticity” or “link” with data, create a better match between theory and design (as techniques such as mind mapping are based on iterative and potentially circular forms of interpretation and not linear thinking), increase physical comfort during data analysis (in the case of voice recognition software), and enhance transparency of the data analysis process.
Key limitations described by the articles included reduction in the natural pace of group discussions and/or reduced interaction with the facilitator to allow time for charting data during data collection (i.e., mind mapping), potential for not achieving the same “depth” or “level of interpretation” as conventional data analysis methods, greater possibility of introducing researcher bias (e.g., editing audio/video footage in a way which distorts original intent, misinterpretation of data by the researcher), use of method may still require (selective) use of transcripts and/or use of other methods, additional time required to master new technologies and/or “unlearn” a previous methodology, cost to purchase new technologies, potential loss of data if coding directly from audio or loss of detail if only reliant on notes, reliance on experienced researchers to obtain the same quality of findings as when using conventional analysis techniques, required hiring of a specialty profession that may not be available in rural areas (i.e., court reporter), lack of sophistication of voice recognition software requiring additional time burdens (e.g., proofreading, adding punctuation, which might mean the technique stops becoming rapid), and voice fatigue during dictation.
See Table 2 for a summary of these benefits and limitations as divided according to studies which eliminated the production of transcripts, and those studies which sped up the process of creating transcripts by using specialized software or professions.
Benefits and Limitations of Using Rapid Techniques, Divided by Rapid Technique.
Tessier (2012) is included in both sections of the table, given that this article argues for a combination of methodologies to include both eliminating transcripts (e.g., direct coding) and using new technologies to increase transcript production (e.g., OneNote, SmartPen).
Recommendations for Future Development/Areas of Future Research
Key recommendations for the use of rapid techniques, beyond a more rapid timeframe for data collection and analysis, include its applicability for applied research settings (due to methodological adaptations to the data collection process) and its ability to generate consensus among a diverse group of research participants and/or stakeholders with “divergent” viewpoints. The majority of articles reviewed recommend the use of rapid techniques for the benefits outlined above, however, most caution that rapid techniques do not supplant use of more conventional methodologies. These techniques are not appropriate for all research settings; they will require methodological adaptations that may require (at a minimum) multiple team members (often experienced researchers) to deploy methodological innovations successfully and a willingness on the part of qualitative researchers to engage with new methodologies and technologies. As stated by Markle et al. (2011), “Instead of accepting transcription as the de facto technique for interpretive research, we suggest continually evaluating the technological landscape and considering the emerging possibilities present for improving our research designs” (p. 17). However, such acceptance of methodological adaptations may also create new spaces for ethical uncertainty (e.g., direct coding of video data) which should be considered carefully. Authors recommend that future areas of research into rapid data collection and analysis techniques should receive more “academic scrutiny and assessment” (Burgess-Allen & Owen-Smith, 2010, p. 414) to determine the reliability and quality of data and the potential for increased participant involvement in the research process.
Concluding Thoughts
Research and fieldwork are clearly not what they used to be (Faubion & Marcus, 2009). Qualitative researchers who seek to use their study findings to inform changes in health care policy and practice are constantly grappling with the need to reduce the timescales required for research. Reduction in time often equates to a reduction in costs, an attractive option for students or researchers working with limited budgets. Within the past 30 years, a series of rapid qualitative techniques have been developed to reduce the amount of time required for qualitative research, but these have not been synthesized or critically analyzed. In this review, we sought to critically review a wide range of techniques developed across multiple disciplines to speed up processes of data collection and analysis.
We found that data analysis has been the main stage of research targeted for rapid techniques. Researchers are mainly concerned with how to deal with interview or focus group recordings, analyzing them directly as recordings or speeding up transcription processes. An unexpected theme that emerged from the review were arguments that favored coding recordings, not only to speed up the research but also to guarantee the accuracy and richness of interpretation (e.g., by maintaining nonverbal information of audio files such as pauses in speech, volume fluctuations, and so on). In contrast, the benefits of coding from transcripts were maintained by several of the authors reviewed, and, indeed, the larger body of literature regarding qualitative data analysis techniques heavily favors production of transcripts. We think an important contribution made by reviewed authors who favor the elimination of transcripts is that the production of transcripts themselves is often taken for granted in qualitative research. Several reviewed authors brought this assumption under scrutiny by testing this assumption through use of alternative techniques and finding that the process used to produce transcripts has a direct impact on the nature of data to be analyzed. As Markle et al. (2011) argued, “transcription is never theory-neutral.” One important lesson we can learn from this review is to be transparent and critical about the processes used to transcribe recordings.
Another important finding was the limited number of techniques that has been evaluated, with the few exceptions cited in the review. Additional research is required to compare the reliability of rapid techniques versus conventional, more long-term techniques and also make comparisons between rapid techniques. This involves the evaluation of the use of technological advancements, such as voice recognition software, and the potential impact these could have on the design, implementation, and use of research. Technological advancements should not be seen as an answer to all our problems but should be critically analyzed in relation to the demands they place on researchers (i.e., additional training), the quality of the data produced, and possibilities for interpretation they allow. Many of the rapid techniques described in the review seem promising for qualitative researchers interested in developing timely research. Researchers will need to review the functionality of these techniques for their own research on an individual basis, weighing their benefits and limitations. We conclude that despite evident limitations, several approaches to rapid data collection and analysis, if properly implemented, are able to reduce the amount required for high-quality in-depth qualitative research and potentially facilitate the use of findings in changes in policy and practice.
Supplemental Material
APPENDICES – Supplemental material for Rapid Techniques in Qualitative Research: A Critical Review of the Literature
Supplemental material, APPENDICES for Rapid Techniques in Qualitative Research: A Critical Review of the Literature by Cecilia Vindrola-Padros and Ginger A. Johnson in Qualitative Health Research
Supplemental Material
FIGURES – Supplemental material for Rapid Techniques in Qualitative Research: A Critical Review of the Literature
Supplemental material, FIGURES for Rapid Techniques in Qualitative Research: A Critical Review of the Literature by Cecilia Vindrola-Padros and Ginger A. Johnson in Qualitative Health Research
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Author Biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
