Complex Qualitative Data Analysis: Lessons Learned From the Experiences With the Qualitative Analysis Guide of Leuven

Abstract

In this article, the authors discuss critical factors to be considered when analyzing complex qualitative data. The experiences with the use of the Qualitative Analysis Guide of Leuven (QUAGOL) to analyze qualitative data were used as starting point to develop a deeper understanding of what a good qualitative analysis requires and how to deal with its challenges in less optimal research contexts. A critical analysis and discussion of the strengths and weaknesses of the guide in a broader methodological context supports its potential for use in developing strong qualitative evidence. The analysis highlighted three key strategies of undertaking the analysis of complex narrative data: the case-oriented approach, the method of constant comparison, and the use of data-generated codes. Having a good understanding of the underlying principles and how to implement them are key to conducting methodologically sound analyses.

Keywords

qualitative health care methodology nursing practice guidelines reflexivity discussion paper

Introduction

Data analysis, which is one of the most challenging aspects of the qualitative research process, especially for large amounts of complex qualitative data (Dierckx de Casterlé et al., 2012; Raskind et al., 2019; Savage, 2000), aims to develop strong theoretical insights or constructs grounded in the collected data to enable complex phenomena to be understood. Researchers are confronted with many pages of interview data, with each interview offering a unique story or perspective. Overreliance on qualitative software packages, word overload, coding based on a preconceived framework, retaining the integrity of each respondent’s story, and the inability to fully exploit the data are some of the problems researchers struggle with during the data analysis process (Dierckx de Casterlé et al., 2012; Ward et al., 2013).

There is a clear need for methods that support health care researchers in the analysis of complex qualitative data. Most available guidelines or checklists related to qualitative studies are critical appraisal tools such as the Critical Appraisal Skills Program (CASP; Public Health Resource Unit, 2006) and Consolidated Criteria for Reporting Qualitative (COREQ; Tong et al., 2007) or focus on reporting qualitative research (Flemming et al., 2018; Hannes et al., 2015). However, they do not provide clear instructions on how to analyze, interpret, and summarize qualitative data. There is a consensus about the limited value of practical guides in the development of insights, but there is also agreement about the importance of extensive preparatory work and “intellectual craftmanship” (Mills, 1978/1995, p. 195) in the development of meaning and deepening the understanding of phenomena (Sandelowski, 1995, 1996). Patterns, themes, and categories do not emerge on their own but demand intellectual work.

A few guides or frameworks offer researchers a method that can support them during this challenging intellectual process (e.g., Dierckx de Casterlé et al., 2012; Richards & Hemphill, 2018; Smith & Firth, 2011; Srivastava & Hopwood, 2009; Ward et al., 2013; Watkins, 2017). These guides all employ a structured, rigorous, phasic approach that can help the researcher consider multiple perspectives and engage with the process of continuous meaning-making and progressive focusing. Interconnectivity between the stages, transparency enhancement, and the use of data-generated codes are the main tenets of these approaches. Some guides aim to capitalize on the benefits of teamwork, for example, the Practical Guide to Collaborative Qualitative Data Analysis (Richards & Hemphill, 2018), while other guides emphasize the iterative processes that enable researchers to move back and forth across the data, for example, the Practical Iterative Framework (Srivastava & Hopwood, 2009; Smith & Firth, 2011). The strength of the QUAGOL lies in its multifaceted approach, allowing researchers to take advantage of various strategies. We also found methods that primarily focus on the management of the data in a grounded, holistic, and efficient way such as the RADaR Technique developed by Watkins (2017).

Little is known about the practical use of these guides and their strengths and pitfalls in the analysis of complex qualitative data. The aim of this study is (a) to discuss the potential of the Qualitative Analysis Guide of Leuven (QUAGOL) for use in the analysis of complex qualitative data and (b) to formulate recommendations for dealing with essential requirements in the analysis of large amounts of qualitative data. Having good insight into the underlying principles of the guide and how to prevent potential pitfalls may help researchers better understand how to put into practice key strategies to develop strong qualitative evidence.

Method

The QUAGOL was developed to support researchers in complex data analysis processes, particularly in Grounded Theory designs (Dierckx de Casterlé et al., 2012). The approach is comprehensive and systematic but not rigid. The proposed approach consists of two parts: (a) the preparation for the coding process and (b) the actual coding process; each part consists of five stages. Although these stages are linear, neither the parts nor their stages can be strictly separated. The continuous interaction between data collection and analysis inevitably results in partial overlap and interactions among the parts and stages. The QUAGOL approach is briefly outlined in Figure 1 and described in detail by Dierckx de Casterlé et al. (2012).

Figure 1.

Aim, characteristics, and stages of the Qualitative Analysis Guide of Leuven (QUAGOL).

The guide was selected for several reasons. The integration of various principles or strategies that have been supported in the literature is the major reason for choosing this guide (Dierckx de Casterlé et al., 2012). The case-oriented approach (e.g., Ayres et al., 2003; Sandelowski, 1995, 1996), the use of the method of constant comparison (e.g., Froggatt, 2001; Glaser & Strauss, 1999; Sandelowski, 1995, 1996), the team approach (e.g., Olson et al., 2016; Richards & Hemphill, 2018), and the focus on peopleware rather than software (e.g., Hunter et al., 2002; Jennings, 2007; Sandelowski, 1995) are described as core characteristics of the QUAGOL.

A second reason is related to the wide use of the QUAGOL in a diverse set of countries, settings, designs, and health care disciplines. On July 1, 2019, a literature search in Web of Science was conducted to gain insight into the international use of the QUAGOL. This search returned 140 articles in which the QUAGOL was cited: 115 articles reporting empirical qualitative studies, 11 articles reporting mixed-method studies, 8 systematic reviews, and 6 methodological articles (see supplemental file). Most of the identified articles reported health care studies, but the top 20 Web of Science categories represented by the articles included other disciplines (e.g., management, educational research). The QUAGOL has been used to analyze interview data, particularly in Grounded Theory studies, and also in other research designs such as phenomenological, focus group, and case study designs. In addition, the guide has been used in mixed-method studies and for the analysis of data in reviews of qualitative evidence or argument-based ethics literature. Two of the authors were actively involved in the development of the QUAGOL, and all authors had experience with the use of the guide to analyze complex qualitative data. The great knowledge and experience in the QUAGOL approach also contributed to selecting this guide.

The discussion is based on researchers’ practical experience with the guide in health care research and qualitative research education. In particular, we used our experience and our colleagues’ experience teaching and supervising junior researchers in qualitative data analysis, as reported and discussed in the Network for Qualitative Research Leuven (NQRL) and in national and international doctoral programs. As the QUAGOL was specifically developed for the analysis of interview data, most experience with it has been in the context of the analysis of complex interview data. These experiences were complemented by insights from the published literature involving the implementation of the guide and discussed in light of the international reference literature.

Strengths of the QUAGOL

The strengths of the QUAGOL are derived from its underlying principles and strategies, all of which are supported by the literature (Dierckx de Casterlé et al., 2012). The use of data analysis triangulation (combining a case-oriented narrative and a systematic analytical approach) contributes to an in-depth understanding of the research phenomenon. The case-oriented approach helps researchers discover the uniqueness of each case while considering the context of all other cases. These insights allow the researchers to develop useful coding frameworks based on data-generated insights. Also of crucial importance is the development approach of the guide, which continuously encourages the researcher to be reflexive and to search for confirming and disconfirming cases.

Data Analysis Triangulation

The QUAGOL approach combines two types of analysis to allow for the development of a more complete, contextualized understanding of complex phenomena and thus contributes to the rigor of the analysis (Polit & Beck, 2017). The guide combines a holistic, case-oriented narrative approach with a systematic analytic approach to allow researchers to explore data from different methodological perspectives and to discover alternative ways to interpret the data (Sandelowski, 1996). The process starts with a case-oriented approach that prompts researchers to examine each case as a whole, thus preventing them from becoming lost in the details of the data. This narrative approach provides room for intuition, imagination, and creativity and allows researchers to develop insights or themes in a way that fully utilizes the richness and uniqueness of the cases. Next, a thematic analysis is systematically and consistently applied to all data to identify empirical support. This combination of approaches enables researchers to uncover different layers of understanding that are reflected in the empirical data, which allows for the construction of different versions of the research phenomenon (Coffey & Atkinson, 1996; Leech & Onwuegbuzie, 2007; Savage, 2000).

Understanding “a Particular in the All-Together,” Through a Case-Oriented Approach

One of the most important strengths of the QUAGOL approach is its focus on the uniqueness of each case (by using a case-oriented approach) while considering the broader context formed by other available cases (by using cross-case analysis), which helps researchers better understand and interpret each case (Ayres et al., 2003; Sandelowski, 1995, 1996). The principle behind this strategy is that the peculiarities of one case can help researchers better understand other cases; that which can be identified very explicitly and that which characterizes one case may be only implicitly or indirectly addressed in another case and thus not immediately understood by the researcher. This continual balancing of within-case and cross-case analyses produces contextually grounded findings by retaining the integrity of each case while considering the contexts of the other cases. Using the method of constant comparison, researchers can continuously adapt and refine their conceptual schemes and gradually develop a conceptual understanding of all data based on a deep understanding of each case. Many researchers’ concerns about ignoring the uniqueness of each case can be countered by this approach.

Use of Data-Generated Insights as Coding Frameworks

In the QUAGOL approach, in contrast to many approaches, the actual coding process with the use of qualitative software is explicitly and deliberately postponed. As recommended by Sandelowski (1995), researchers are encouraged to take time to examine the data to understand what they are looking for. This preparatory work will open researchers’ minds to the multiple meanings and perspectives in the qualitative data and help them organize and structure the data in a way that is useful for capturing and understanding the participants’ messages. This approach leads to the development of a coding framework that is analytically and contextually meaningful and that is grounded in the empirical data. As such, the guide offers a compromise between a purely inductive coding approach and a preconceived theoretical framework, which is often imposed on empirical data (Fereday & Muir-Cochrane, 2006).

Continuous Encouragement of Reflexivity With Space for Creativity and Intuition

Throughout the data analysis process, the QUAGOL encourages researchers’ reflexivity. Researchers are systematically and continually encouraged to be attentive to the context of knowledge development and, more specifically, to their own impacts on the collection, analysis, and interpretation of data (Engward & Davis, 2015; Polit & Beck, 2017). The QUAGOL recommends the use of an ongoing process of bracketing and discussing the results within teams (Mertens et al., 2017b). This approach supports researchers in reaching deeper levels of reflection across all stages (Tufford & Newman, 2010). By considering the analysis of qualitative data as a team activity, researchers continuously examine the data from a variety of perspectives (Dixon-Woods et al., 2006). Viewing and interpreting data from several perspectives encourage critical reflection on one’s own opinions, preferences and prejudices and raise awareness regarding the potential bias in the interpretations of the data (Polit & Beck, 2017). The primary aim of teamwork is not to reach consensus but to employ different perspectives and interpretations to better understand the data, which helps researchers concentrate on how and why team members see, hear or understand things differently. Similarly, interdisciplinary team compositions and the organization of one or more peer debriefings are strongly recommended in the QUAGOL (Richards & Hemphill, 2018).

The guide prompts researchers to be reflexive, but the recommended approach also prevents them from becoming so reflexive that creativity is stifled (Attia & Edge, 2017; McGhee et al., 2007; Polit & Beck, 2017). The use of two approaches (a holistic, narrative approach focusing on researchers’ intuition, imagination and creativity and a systematic analytic approach for identifying empirical support) is intended to help researchers strike a balance between creativity and reflexivity (Maher et al., 2018). Furthermore, forward–backward dynamics, teamwork, and the encouragement of reflexivity and creativity all contribute to the saturation process.

Continuous Search for Themes by Confirming and Disconfirming Evidence and Negative Cases

Although summarized in two parts, each consisting of five stages, the analytical method presented in the QUAGOL is characterized by iterative processes of data analysis that involve constant movement between stages. Researchers are encouraged to continuously verify their ideas, themes, hypotheses and concepts against available and newly collected data. This dynamic process allows researchers to more deeply explore the phenomenon under study, reveal several layers of understanding and thus make optimal use of the richness of the qualitative data (Froggatt, 2001; Sandelowski, 1995). Reflection on and adjustments to the common themes and concepts based on these forward–backward dynamics guide further data collection and analysis and provide evidence about the way concepts have been developed (Froggatt, 2001). Peer debriefing is strongly recommended as an additional strategy for supporting researchers in this systematic search for confirming and disconfirming patterns in qualitative data (Richards & Hemphill, 2018).

Pitfalls of the QUAGOL

The potential weaknesses of the guide are mainly related to the rigorous application of the principles and strategies described in the QUAGOL. We observed some practical barriers and misunderstandings related to the postponement of the actual coding process, the distinct nature of the narrative report and conceptual scheme and the application of the forward–backward dynamics in practice.

Deliberately Postponed Actual Coding Process

The guide highlights the importance of first developing a good understanding of a body of interviews as a whole before starting to break down the data and enter codes into a qualitative software package. The guide encourages the researchers to slow the analysis process and engage in meaningful interaction with the data, as is also suggested by Maher et al. (2018). In our discussions with researchers, we observed some misunderstanding related to the postponement of the actual coding process. In general, coding refers to the process of breaking down and conceptualizing data; more concretely, it refers to the process of identifying and assigning labels to words, phrases, sentences, or paragraphs that convey important meanings in response to research question (Polit & Beck, 2010). The “actual coding process” we refer to in Stage 7 of the QUAGOL refers to the process of systematically linking all significant interview passages to one or more codes from the list developed in the previous stages and entering those codes into a qualitative software package. Although data analysis is often equated with this part of the coding process, in the QUAGOL approach, it is assumed that the most important analytical work is performed before using qualitative software (St. Pierre & Jackson, 2014). Hence, it is not the analytical work but the use of software that should be postponed during this process.

The first part of the QUAGOL explicitly aims to guide researchers during the intellectual process of imagining, linking and understanding data within a specific context. Themes in qualitative data do not always immediately emerge but rather need to be gradually discovered by reading, taking a step back, rereading and reflecting (Bowen, 2006). The insights gained through the first stages are used to develop a list of codes, which can be introduced as preliminary codes in the software program. These insights could be considered sensitizing concepts originating from the empirical data that provide researchers with a general sense of guidance and suggest directions from which to approach the data (Charmaz, 2006). Only at Stage 7 are researchers invited to go back to the “ground” and link the insights with the participants’ actual words and stories and enter the codes into qualitative software. In other words, this “actual coding process” is postponed until researchers have developed conceptual insights that can help them develop a contextually and analytically meaningful coding framework.

Narrative Report Versus Conceptual Scheme

A second observation refers to the researchers’ difficulties in understanding the specific aim of the narrative report (Stage 2) and the individual conceptual scheme (Stage 3) as well as the distinction between them. This misunderstanding often results in reports and schemes being of little help in the development of analytical and conceptually relevant insights. Both documents should reflect the key messages of an interviewee’s story in response to the research question and thus focus on the essence of the interview. The major difference lies in the abstraction level.

The narrative report describes the key messages as told by the respondent in response to the research question; the researchers may use the interviewee’s words or examples to clarify these messages. A common mistake is to consider this report a mere summary of the content of the story. There is a tendency among researchers to select too much information rather than to focus on the essence of the messages for fear for leaving out significant information. The ability to distinguish relevant information from less relevant information in relation to the research question is important for developing useful reports.

The descriptions in the narrative report are abstracted in the conceptual scheme. Researchers are invited to filter the most important messages of an interview and cluster them in a web of ideas, themes or hypotheses in response to the research question. In other words, researchers are challenged to describe the same messages described in the narrative report but using fewer words. Many researchers, however, tend to summarize the interview rather than grasping the underlying ideas or messages.

To clarify, the difference between the two documents can be described in terms of how researchers would relate the key messages of an interview if they had different amounts of time to speak: the narrative report would be the format if researchers were allowed to speak for 5 to 7 minutes, while the conceptual scheme would be the format if researchers had only 3 minutes to express the same message. The conceptual scheme could be considered the “identity card” of the interview, helping researchers capture and remember the particularities of each interview during analysis. Conceptual schemes are important instruments of communication in team discussions and are particularly helpful in the development of conceptual insights.

Forward-Backward Movements

The constant movement between the stages and the two parts seems to be another challenge of using the guide. Although the stages are summarized artificially as discrete and linear stages, the importance of constant movement between the stages and the two parts cannot be overstated. (Froggatt, 2001; Sandelowski, 1995; Srivastava & Hopwood, 2009) There is some confusion about how to apply these dynamic processes in practice. Researchers struggle with questions such as when to move to the next stage, when to go back to previous stages and when is the best time to start using qualitative software. However, there is no strict procedure to follow because this process is influenced by numerous factors, such as the richness of the data, the experience of the researcher and the expertise of the research team. Some practical suggestions will be given later.

Practical Obstacles

The most challenging obstacle for many researchers is the time-consuming nature of the method. The case-oriented approach, the continuous forward-backward dynamics, the postponement of the actual coding process and the focus on teamwork are characteristics that slow analysis. A lack of time may incite researchers to skip some stages or neglect some key principles or strategies of the QUAGOL. It may also be practically impossible to collect and analyze data simultaneously and interactively, obliging researchers to start the analysis process after all interviews have been conducted. Not being able to use the method of constant comparison will prevent researchers from fully benefiting from the guide. Some researchers are not supported by teams and thus have to perform analyses on their own. Others have struggled with the question of how to adequately work as part of a team during the analysis process. Some teams have tended to focus on consensus rather than making the best use of the various perspectives to better understand the data, while other teams have had difficulty finding a way to implement peer debriefing or bracketing strategies with the QUAGOL. Here, a one-sided focus on reaching a consensus is a common mistake among researchers. Such scenarios raise questions about the value of the QUAGOL in suboptimal research contexts.

Discussion

The analysis of the strengths the QUAGOL reveal the underlying strategies and principles of the approach, all of which are described in the literature as essential to cope with the great wealth of qualitative data and thus to fully exploit the richness of the data (Dierckx de Casterlé et al., 2012).

The extensive preparation of the coding process (instead of a “line-by-line coding”) stimulates researchers to first “look at” the data to see what they should “look for” in the data (Sandelowski, 1995, p. 371). “Looking at and through each case” is, according to Sandelowski (1996, p. 526), the appropriate initial approach to analyze and interpret qualitative data. By focusing on peopleware instead of software, this approach prevents the researcher from relying too quickly on qualitative software packages, resulting in an overload of meaningless codes and difficulties in understanding of the participant’s story in its fullness. This holistic understanding is of crucial importance as generalizations are, as reported by Ayres et al. (2003), embedded in the contextual richness of personal experiences. This preparatory work allows the researchers to develop coding frameworks grounded in the data and offers ways of organizing data that are helpful to interpret and synthesize the data in answer to the research question (Charmaz, 2006). This practice may prevent some well-known weaknesses in qualitative analysis, such as premature closure, enthusiasm for artificial coherence (fitness addiction), and saturation as a convenient stopping point (Thorne & Darbyshire, 2005). Premature analytic closure typically results from a persistent but often unconscious preconceived view of the phenomenon under investigation (Sandelowski, 1995).

A case-oriented approach is considered as essential in complex qualitative data analysis (Ayres et al., 2003; Sandelowski, 1995, 1996). Each case is unique, differing from the others as regards its content, characteristics, and tone. How to retain the uniqueness of each individual case in the analysis process is a challenge for many qualitative researchers (Bailey & Jackson, 2003). The analytical process, which typically segments and breaks down data, may limit the researchers’ understanding of a participant’s experience in its richness (Bailey & Jackson, 2003). This concern can be countered by focusing on cases rather than lines or sentences. Strategies primarily relying on coding and sorting fragments of texts into units with similar meaning will lose much of the contextual richness of the stories (Ayres et al., 2003). Complementing the case-oriented approach with a systematic analytical approach enables the researcher to look for alternative interpretations of the data and thus offers greater complexity and depth in understanding the phenomenon concerned (Coffey & Atkinson, 1996; Polit & Beck, 2017; Savage, 2000).

Also of crucial importance is the development approach of the guide, which stimulates the researcher to engage into iterative processes of digging deeper, constantly moving between the various stages of the process. Finding a good balance between creativity and reflexivity is key in the analysis of complex qualitative data (Maher et al., 2018). The method offers space that stimulates the researcher’s intuition and creativity as optimal as possible. Both are considered key components in qualitative research because they are essential to developing ideas and themes embedded in the complexity of the participant’s experiences (Ayres et al., 2003; Janesick, 2001; Konecki, 2019). At the same time, the researcher is continuously encouraged to verify developing ideas and themes by iterative dialogue with the interview data. These forward–backward dynamics permit the researcher to go beyond a descriptive account and to reach a deep understanding about meaning in relationship to context and eventually to exploit the full potential of the qualitative data (Froggatt, 2001; Sandelowski, 1995; Srivastava & Hopwood, 2009).

Last but not least, the method gets the researcher out of his or her isolated position because the analytic process is mainly regarded as a team activity that enhances the possibility to gain rich, creative and thoughtful insight in the research phenomenon (Olson et al., 2016; Richards & Hemphill, 2018).

The weaknesses of the guide are mainly related to the requirements of the proposed method that are also essential to qualitative data analysis. Time is essential for developing nuanced insights into complex phenomena and their underlying dynamics (Sandelowski, 1995; Watkins, 2017). Enough time must be taken to immerse oneself into complex qualitative data and to engage in a dynamic iterative analysis process. Using a team approach is also a necessity for developing strong qualitative evidence (Richards & Hemphill, 2018). A lack of these conditions may cause researchers to adapt or simplify the guide or to reduce it to a standard protocol to be implemented linearly step by step. Adaptations of the approach are possible; however, a simplified version of the QUAGOL could limit its strengths or potential to support strong data analysis.

The discursive nature of the article and the authors’ role in relation to the development and use of the QUAGOL are limitations of this study. The analysis of the guide is based on experience, feedback, and the published literature, which allowed us to identify and consider salient issues based on international reference literature. However, more research is needed to better understand the impact of the guide on researchers’ understanding and competencies to develop strong qualitative evidence.

Lessons Learned for Improving the Analysis of Complex Qualitative Data

The analysis of the strengths and pitfalls of the QUAGOL highlights some critical factors to be considered when analyzing great amounts of rich qualitative data. Based on the experiences of using the QUAGOL in complex qualitative data analysis and international reference literature, we formulated some recommendations for dealing with the challenges of analyzing complex narrative qualitative data, particularly in a less optimal research context.

Respect for the essential characteristics

Based on our experience and the literature (Ayres et al., 2003; Charmaz, 2006; Chen & Boore, 2009; Dierckx de Casterlé et al., 2012; Fereday & Muir-Cochrane, 2006; Maher et al., 2018; Sandelowski, 1995, 1996), we identified at least two essential characteristics for a robust analysis of complex qualitative data: a case-oriented approach combined with the method of constant comparison and the use of data-generated codes. A case-oriented approach (instead of a line-by-line approach) is needed to understand “the particular in the all-together” (Ayres et al., 2003; Sandelowski, 1996). Such an approach will help researchers interpret and integrate very large amounts of narrative data without losing the particularity of each case. The strength of this approach lies in the continuous emphasis on the uniqueness of each case as well as the use of the broader contexts of all cases to better understand each individual case. Every case may contribute to a better understanding of the research phenomenon, yet no single case can provide the necessary data to answer the research question. The combination of within- and cross-case analysis techniques makes it possible to use the richness of the data while doing justice to the uniqueness of the respondents’ experiences (Ayres et al., 2003; Dierckx de Casterlé et al., 2012). Clearly, a case-oriented approach cannot be applied without using the method of continuous comparison, which allows researchers to gradually discover different layers of understanding (Charmaz, 2006; Chen & Boore, 2009; Maher et al., 2018).

Second, we need a framework that offers a sensible way of organizing and structuring large amounts of rich data. Thorough and extensive preparation for the coding process explicitly aims to develop analytically and contextually meaningful codes that are grounded in the data and that help address the large amount of data (Charmaz, 2006; Dierckx de Casterlé et al., 2012; Fereday & Muir-Cochrane, 2006; Sandelowski, 1995).

Space for flexibility and adaptation

In suboptimal circumstances, adaptations of the method may be necessary. In these cases, it will be important to determine how to adapt the approach while respecting the essential characteristics of the approach. Most of the concerns relate to time issues, that is, how to reduce the amount of time required in the analytic process without losing quality. The quality of the data and the research team certainly allow researchers to reach saturation more quickly and thereby save time. A lack of experience and possible bias may be offset by teamwork. Experienced researchers with strong analytical competencies may skip some stages or adapt their implementation throughout the analytic process. According to our observations, experienced users of the QUAGOL can use the guide more flexibly, starting with a systematic and comprehensive implementation of the method and gradually shifting to more targeted use (e.g., partial transcription or coding of new cases, selection of the most challenging cases for discussion within the team). The more insights there are into the research phenomenon, the better researchers can focus on conducting a methodologically sound analysis.

The practical barriers to simultaneously collecting and analyzing data are problematic in an interpretive–constructivist approach, but they are not insurmountable. To address these barriers, we advise writing a short memo immediately after conducting an interview to briefly outline the most striking characteristics or messages of the interview that relate to the research question. If the possibilities to conduct collection and analysis interactively are limited, researchers may select just a few interviews to read and analyze and use these insights to collect additional data. A particularly rich or challenging interview may be of great value for the research process. Occasionally, the only option will be to start the analysis process after all interviews have been conducted. In this case, it is important to find the best way to apply the method of constant comparison to allow researchers to more deeply explore the phenomenon. We suggest a gradual phased analysis approach that involves dividing the interviews into groups that are subsequently analyzed. The composition of the groups may be guided by the research question, the characteristics of the interviews, the richness of the interview data or the insights developed through the analysis process. This “theoretical sampling of interviews” allows for a progressive evolution of the analysis process, which might save time and enhance the quality of the analysis. This recommendation is supported by the work of Benoot et al. (2016) who gave evidence for the beneficial effects of purposeful sampling techniques in a qualitative evidence synthesis. In addition, Degrie et al.’s review of qualitative evidence (2017a) illustrated how this kind of selection of cases creates potential to develop a strong conceptual model.

Implementation of forward–backward dynamics

Most authors suggest using a phasic approach to engage with the process of continuous comparison of data and cases (Olson et al., 2016; Richards & Hemphill, 2018; Srivastava & Hopwood, 2009). Although the stages often are summarized as linear stages, the interconnectivity between the stages is clearly underlined. The purpose of the phasing is to stimulate the researcher to first look at the data, trying to understand and narratively describe the underlying key messages before starting with the conceptualization of the themes in each separate interview and later on across all the cases using the method of constant comparison. When and how to start the forward–backward move in this process is, however, less clear. Although there is no strict procedure to follow, some practical tips may help researchers obtain a better idea of how to implement these dynamics in the analysis process.

In general, we recommend starting to develop the narrative reports after a small number of interviews (2 to 4) have been conducted, depending on their quality. As it is easier to obtain the essence of a rich case, we suggest selecting a case that provides the richest information when writing the first narrative report. Next, the researcher can translate the available reports into conceptual schemes and verify the appropriateness of the schemes in dialog with the empirical data. Breaking down the analytical work in these stages explicitly aims to help researchers focus on the particularity of each case rather than on the common features. It is recommended to repeat this work (cf. stages 1–4 of the QUAGOL) until the researchers have a good conceptual understanding of a limited number of rich cases (approximately 6–10). By then, it is indicated to begin forward–backward movement between within-case and across-case analyses using the method of constant comparison. This strategy will help gradually develop conceptual understanding of all the data collected at that time and to develop an overarching web of ideas. These conceptual insights can be used to create the coding list and enter the codes into a qualitative software package. We recommend starting the cross-case analysis of themes using software once the researchers have achieved an initial level of saturation in their conceptual insights developed so far (in general, based on approximately 8–15 cases). For each new case, it is indicated to restart the process from the beginning. This movement back may help optimize the conceptual schemes and eventually the analysis of themes and underlying processes where necessary. Gradually, the researchers will be able to describe and integrate the concepts into a meaningful framework and to reconstruct the story on a conceptual level.

Importance of the quality of the collected data

Researchers need to know that, in any case, the benefits of the method used will increase with the quality of the collected data. Here, we refer to the importance of strong interview skills, well-developed and continuously adjusted interview guides, and meticulously transcribed interviews (Al-Yateem, 2012; Donalek, 2005). We also advise creating a descriptive and methodological report for each case (Mertens et al., 2017b). A careful description of the interviewee and contextual characteristics is useful for understanding data in their specific context. The methodological report includes a critical evaluation of the output or efficiency of the interview and will help researchers enhance the quality of subsequent interviews. Moreover, high-quality interview data will prevent researchers from being overloaded by a large amount of irrelevant data unnecessarily complicating the analysis process.

Conclusion

A critical exploration of the strengths and weaknesses of the QUAGOL in the analysis of complex qualitative data supports the potential of the guide for use in developing strong qualitative evidence. Having a good understanding of its strengths and pitfalls and how to deal with them under suboptimal conditions is crucial to taking full advantage of the guide.

The QUAGOL is particularly useful in qualitative data analysis with an interpretive–constructivist research design, but it also appears to be helpful with other designs, such as reviews of qualitative evidence or argument-based ethics literature.

The analysis highlighted the key principles of undertaking the analysis of complex qualitative data, notably the case-oriented approach, the method of constant comparison and the use of data-generated codes. Provided that the core principles are respected, analytical work will holistically consider the data to make the best use of the richness of each case. However, lack of time or other essential conditions may prevent the application of these principles in practice. The challenge lies in finding out how to adjust the method used to the particular research conditions without losing its greatest assets for a successful data analysis.

In addition to time and teamwork, the researcher’s creativity, courage and mental freedom to engage in iterative, dynamic and recursive analysis remain essential for the analysis of complex qualitative data.

Supplemental Material

sj-pdf-1-qhr-10.1177_1049732320966981 – Supplemental material for Complex Qualitative Data Analysis: Lessons Learned From the Experiences With the Qualitative Analysis Guide of Leuven

Supplemental material, sj-pdf-1-qhr-10.1177_1049732320966981 for Complex Qualitative Data Analysis: Lessons Learned From the Experiences With the Qualitative Analysis Guide of Leuven by Bernadette Dierckx de Casterlé, Kristel De Vliegher, Chris Gastmans and Evelyne Mertens in Qualitative Health Research

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Bernadette Dierckx de Casterlé

Availability of Material

All research material related to this paper can be found in its supplementary information file.

Additional file: International use of the Qualitative Analysis Guide of Leuven (QUAGOL) cf. Web of Science

Supplemental Material

Supplemental Material for this article is available online at . Please enter the article’s DOI, located at the top right hand corner of this article in the search bar, and click on the file folder icon to view.

Author Biographies

Bernadette Dierckx de Casterlé is professor of nursing ethics at KU Leuven, Belgium. She has long standing experience in qualitative research methodologies.

Kristel De Vliegher is head of the nursing department in the White and Yellow Cross, an organization for home nursing in Flanders, Belgium. She provides teaching support in qualitative research methodologies, more specifically data analysis at the KU Leuven, Belgium.

Chris Gastmans is professor of medical ethics at KU Leuven, Belgium. His research focuses on ethical aspects of care for older adults and end-of-life care.

Evelyne Mertens is a research assistant at the University of Leuven, Belgium. She conducts qualitative health care research and also provides teaching support with main focus on data analysis.

References

Al-Yateem

(2012). The effect of interview recording on quality of data obtained: A methodological reflection. Nurse Researcher, 19(4), 31–35. https://doi.org/10.7748/nr2012.07.19.4.31.c9222

Attia

Edge

(2017). Be(com)ing a reflexive researcher: A developmental approach to research methodology. Open Review of Educational Research, 4(1), 33–45. https://doi.org/10.1080/23265507.2017.1300068

Ayres

Kavanaugh

Knafl

K. A.

(2003). Within-case and across-case approaches to qualitative data analysis. Qualitative Health Research, 13(6), 871–883. https://doi.org/10.1177/1049732303013006008

Bailey

D. M.

Jackson

J. M.

(2003). Qualitative data analysis: Challenges and dilemmas related to theory and method. The American Journal of Occupational Therapy, 57(1), 57–65. https://doi.org/10.5014/ajot.57.1.57

Benoot

Hannes

Bilsen

(2016). The use of purposeful sampling in a qualitative evidence synthesis: A worked example on sexual adjustment to a cancer trajectory. BMC Medical Research Methodology, 16(21), 1–12. https://doi.org/10.1186/s12874-016-0114-6

Bowen

G. A.

(2006). Grounded Theory and Sensitizing Concepts. International Journal of Qualitative Methods, 5(3), 12–23. https://doi.org/10.1177/160940690600500304

Charmaz

(2006). Constructing grounded theory. A practical guide through qualitative analysis. SAGE.

Chen

H. Y.

Boore

J. R. P.

(2009). Using a synthesised technique for grounded theory in nursing research. Journal of Clinical Nursing, 18, 2251–2226. https://doi.org/10.1111/j.1365-2702.2008.02684.x

Coffey

Atkinson

(1996). Making sense of qualitative data: Complementary research strategies. SAGE.

10.

Degrie

Gastmans

Mahieu

Dierckx de Casterlé

Denier

(2017). How do ethnic minority patients experience the intercultural care encounter in hospitals? A systematic review of qualitative research. BMC Medical Ethics, 18, Article 2. https://doi.org/10.1186/s12910-016-0163-8

11.

Dierckx de Casterlé

Gastmans

Bryon

Denier

(2012). QUAGOL: A guide for qualitative data analysis. International Journal of Nursing Studies, 49, 360–371. https://doi.org/10.1016/j.ijnurstu.2011.09.012

12.

Dixon-Woods

Cavers

Agarwal

Annandale

Arthur

Harvey

… Sutton

(2006). Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Medical Research Methodology, 6, Article 35. https://doi.org/10.1186/1471-2288-6-35

13.

Donalek

J. G.

(2005). The interview in qualitative research. Urologic Nursing, 25(2), 124–125.

14.

Engward

Davis

(2015). Being reflexive in qualitative grounded theory: Discussion and application of a model of reflexivity. Journal of Advanced Nursing, 71(7), 1530–1538. https://doi.org/10.1111/jan.12653

15.

Fereday

Muir-Cochrane

(2006). Demonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme development. International Journal of Qualitative Methods, 5(1), 80–92. https://doi.org/10.1177/160940690600500107

16.

Flemming

Booth

Hannes

Cargo

Noyes

(2018). Cochrane qualitative and implementation methods group guidance series—paper 6: Methods for question formulation, searching and protocol development for qualitative evidence synthesis. Journal of Clinical Epidemiology, 97, 79–85. https://doi.org/10.1016/j.jclinepi.2017.10.022

17.

Froggatt

K. A.

(2001). The analysis of qualitative data: Processes and pitfalls. Palliative Medicine, 15, 433–438. https://doi.org/10.1191/026921601680419492

18.

Glaser

B. G.

Strauss

A. L.

(1999). The discovery of grounded theory: Strategies for qualitative research. Aldine Transaction.

19.

Hannes

Heyvaert

Slegers

Vandenbrande

Van Nuland

(2015). Exploring the potential for a consolidated standard for reporting guidelines for qualitative research: An argument Delphi approach. International Journal of Qualitative Methods, 14(4), 1–16. https://doi.org/10.1177/1609406915611528

20.

Hunter

Lusardi

Zucker

Jacelon

Chandler

(2002). Making meaning: The creative component in qualitative research. Qualitative Health Research, 12(3), 388–398. https://doi.org/10.1177/104973202129119964

21.

Janesick

V. J.

(2001). Intuition and creativity: Pas de Deux for qualitative researchers. Qualitative Inquiry, 7(5), 531–540. https://doi.org/10.1177/107780040100700501

22.

Jennings

B. M.

(2007). Qualitative analysis: A case of software of “peopleware?” Research in Nursing Health, 30(5), 483–484. https://doi.org/10.1002/nur.20238

23.

Konecki

K. T.

(2019). Creative thinking in qualitative research and analysis. Qualitative Sociology Review, 15(3), 6–25. http://doi.org/10.18778/1733-8077.15.3.01

24.

Leech

N. L.

Onwuegbuzie

A. J.

(2007). An array of qualitative data analysis tools: A call for data analysis triangulation. School Psychology Quarterly, 22(4), 557–584. http://doi.org/10.1037/1045-3830.22.4.557

25.

Maher

Hadfield

Hutchings

de Eyto

(2018). Ensuring rigor in qualitative data analysis: A design research approach to coding combining NVivo with traditional material methods. International Journal of Qualitative Methods, 17, 1–13. https://doi.org/10.1177/1609406918786362

26.

McGhee

Marland

G. R.

Atkinson

(2007). Grounded theory research: Literature reviewing and reﬂexivity. Journal of Advanced Nursing, 60(3), 334–342. https://doi.org/10.1111/j.1365-2648.2007.04436.x

27.

Mertens

Heyligen

Declercq

Hannes

Truyen

Denier

Dierckx de Casterlé

(2017). QualiBuddy: An online tool to improve research skills in qualitative data analysis. Qualitative Research Journal, 17(4), 306–318.

28.

Mills

C.W.

(1995). The sociological imagination. Oxford University Press. (Original work published 1978).

29.

Olson

J. D.

McAllister

Grinnell

L. D.

Walters

K. G.

Appunn

(2016). Applying constant comparative method with multiple investigators and inter-coder reliability. The Qualitative Report, 21(1), 26–42.

30.

Polit

D. F.

Beck

C. T.

(2010). Generalization in quantitative and qualitative research: Myths and strategies. International Journal of Nursing Studies, 47, 1451–1458. https://doi.org/10.1016/j.ijnurstu.2010.06.004

31.

Polit

D. F.

Beck

C. T.

(2017). Nursing research: Generating and assessing evidence for nursing practice (10th ed.). Wolters Kluwer Health. https://doi.org/10.1016/j.iccn.2015.01.005

32.

Public Health Resource Unit. (2006). Critical Appraisal Skills Programme (CASP): Making sense of evidence.

33.

Raskind

I. G.

Shelton

R. C.

Comeau

D. L.

Cooper

H. L. F.

Griffith

D. M.

Kegle

M. C.

(2019). A review of qualitative data analysis practices in health education and health behavior research. Health Education & Behavior, 46(1), 32–39. https://doi.org/10.1177/1090198118795018

34.

Richards

K. A. R.

Hemphill

M. A.

(2018). A practical guide to collaborative qualitative data analysis. Journal of Teaching in Physical Education, 37, 225–231. https://doi.org/10.1123/jtpe.2017-0084

35.

Sandelowski

(1995). Focus on qualitative methods. Qualitative analysis: What it is and how to begin. Research in Nursing & Health, 18, 371–375. https://doi.org/10.1002/nur.4770180411

36.

Sandelowski

(1996). Focus on qualitative methods. One is the liveliest number: The case orientation of qualitative research. Research in Nursing & Health, 19, 525–529. https://doi.org/10.1002/(SICI)1098-240X(199612)19:6<525::AID-NUR8>3.0.CO;2-Q

37.

Savage

(2000). One voice, different tunes: Issues raised by dual analysis of a segment of qualitative data. Journal of Advanced Nursing, 31(6), 1493–1500. https://doi.org/10.1046/j.1365-2648.2000.01432.x

38.

Smith

Firth

(2011). Qualitative data analysis: Application of the framework approach. Nurse Researcher, 18(2), 52–62. https://doi.org/10.7748/nr2011.01.18.2.52.c8284

39.

Srivastava

Hopwood

(2009). A practical iterative framework for qualitative data analysis. International Journal of Qualitative Methods, 8(1), 76–84. https://doi.org/10.1177/160940690900800107

40.

St.Pierre

E. A.

Jackson

A. Y.

(2014). Qualitative data analysis after coding. Qualitative Inquiry, 20(6), 715–719. https://doi.org/10.1177/1077800414532435

41.

Thorne

Darbyshire

(2005). Land mines in the field: A modest proposal for improving the craft of qualitative health research. Qualitative Health Research, 15(8), 1105–1113. https://doi.org/10.1177/1049732305278502

42.

Tong

Sainsbury

Craig

(2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19(6), 349–357. https://doi.org/10.1093/intqhc/mzm042

43.

Tufford

Newman

(2010). Bracketing in qualitative research. Qualitative Social Work, 11(1), 80–96. https://doi.org/10.1177/1473325010368316

44.

Ward

D. J.

Furber

Tierney

Swallow

(2013). Using Framework Analysis in nursing research: A worked example. Journal of Advanced Nursing, 69(11), 2423–2431. https://doi.org/10.1111/jan.12127

45.

Watkins

D. C.

(2017). Rapid and rigorous qualitative data analysis: The “RADaR” technique for applied research. International Journal of Qualitative Methods, 16, 1–9. https://doi.org/10.1177/1609406917712131

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.21 MB