Abstract
Summary
The authors present the field of social work with a methodology specifically aimed at the synthesis of qualitative research informed by existing methods and applications yet tailored to the unique values and goals of the profession of social work.
Findings
Though qualitative research in social work is commonplace, currently, the field lacks a methodology to synthesize these qualitative studies. A synthesis of qualitative studies results in generation of a more in-depth understanding of the phenomena studied that can be used to develop theory and inform practice and policy.
Applications
This methodology enables synergistic understanding of phenomena with richness in diversity of settings, participants, and qualitative traditions. This synergistic understanding can be used to develop theory and inform practice and policy.
Keywords
Introduction
Since social work's inception, the profession has sought to enhance human well-being from a holistic perspective, acknowledging that humans do not live in vacuums – humans are constantly acting and reacting within their social, emotional, and physical environments (International Federation of Social Workers [IFSW], 2000). This holistic perspective is prevalent throughout in both social work practice and social work research. Social work research maintains this attention to the whole as evidenced by the rise in focus on mixed methods, systematic reviews, and meta-analyses over the past 30 years (Littell, Corcoran, & Pillai, 2008) – all of which aim to capture a more holistic view of a phenomenon. Specifically, mixed methods accomplishes this through combining qualitative and quantitative methods in primary data collection; systematic reviews “sum up the best available research on a specific question” (The Campbell Collaboration, n.d.); and meta-analysis is a “statistical procedure that integrates the results of several independent studies considered to be combinable’’ (Egger, Davey, & Phillips, 1997, p. 1533). Each of these exhibits an effort to bring together multiple sources of evidence, specific to a particular phenomenon, into a comprehensive whole that offers insight in terms of social work practice and policy, expanding and deepening the breadth of evidence-based practice. Mixed methods and systematic reviews provide a platform for the integration of qualitative and quantitative evidence on a topic and meta-analysis provides an integration approach for purely quantitative evidence, but, so far, the discipline of social work has not embraced a definitive technique or method for cross-study analysis to synthesize qualitative findings.
An extensive review of the qualitative social work literature revealed that social work lacks a methodology to synthesize qualitative findings; however, Padgett (2004) notes a considerable amount of qualitative research is conducted in the field of social work. Our review unveiled four articles synthesizing qualitative studies among the scholarly social work literature (Forte, 2009; Hodge, 2011; McCalman et al., 2010; Watkins, Walker, & Griffith, 2010), each has used different approaches to the task (e.g. Finfgeld, 2003; Noblit & Hare, 1988) borrowed from nursing, a profession aiming to enhance well-being in terms of health (American Nursing Association, 2004) similar to our goal of enhancing overall well-being.
Since synthesizing qualitative research allows for knowledge gleaned from individual qualitative studies of a particular phenomenon to come together in a broader, in-depth, and more holistic understanding of that phenomenon, having a method uniquely designed for the social work profession's mission is deemed highly desirable. In this paper, we discuss, in generalities, current approaches to synthesis and present a model for synthesizing qualitative research tailored to the field of social work that we have developed and implemented.
How is qualitative research synthesized?
Aggregative versus interpretive
Prior to presenting a social work oriented approach to synthesizing qualitative research, it is important to have a general 1 understanding of existing approaches heretofore used to synthesize studies. Two overarching approaches depict the nature of the methodological procedure. Some researchers operate under the assumption that a qualitative synthesis is an aggregative process while others view qualitative synthesis as an interpretive process. Aggregative syntheses take on a quantitative approach (for example determining an effect size through frequency and intensity) (Sandelowski & Barroso, 2003, 2007). On the other hand, interpretive syntheses focus on theory development (Dixon-Woods et al., 2006) and concept development, and follow a traditional qualitative methodological foundation that does not exhibit quantitative features. Multiple differences exist between these two approaches as well as some similarities; however, we have highlighted differences relevant to our reasoning for selecting our methodological preference. Since we want to present a method that focuses on uncovering the whole of a given phenomenon, we believe the interpretive approach is most congruent with our primary goals of a holistic understanding and theory development. The choices of our colleagues (Forte, 2009; Hodge, 2011; McCalman et al., 2010; Watkins et al., 2010) in conducting their syntheses also reflect a preference for the interpretive approach.
Nomenclature: Varied approaches to cross study analysis
To enhance further understanding, one must be capable of differentiating qualitative synthesis from quantitative meta-analysis, systematic review, literature review, and varying approaches to qualitative synthesis.
Quantitative cross study analysis, systematic review and literature review
The synthesis of quantitative studies is most commonly referred to as meta-analysis, and as stated before can be defined as a “statistical procedure that integrates the results of several independent studies considered to be combinable’’ (Egger et al., 1997, p. 1533). According to Rubin and Babbie (2008) a “meta-analysis simply involves calculating the mean effect size across previously completed research studies on a particular topic” (p. 550). Furthermore, a meta-analysis increases the external validity of findings through a larger sample size and minimizes the sampling error. In turn, increasing the external validity of the research findings adds to the breadth, rigor, and credibility of the results. Both systematic reviews and literature reviews consist of a detailed overview of existing literature. The authors of these reviews summarize findings instead of synthesizing them.
Qualitative cross study analysis
Although use of qualitative cross study analysis in social work research is in its infancy, it has become a widely discussed, theorized and applied concept in the field of nursing. Numerous researchers in nursing have developed methods to conduct qualitative synthesis (Estabrooks, Field, & Morse, 1994; Finfgeld-Connett, 2010; Jensen & Allen, 1996; Sandelowski & Barroso, 2007; Sandelowski, Docherty, & Emden, 1997) and applied these (Barroso & Powell-Cope, 2000; Beck, 2001, 2002; Britten et al., 2002; Campbell et al., 2003; Dixon-Woods et al., 2006; Finfgeld, 1999, 2000; Jensen & Allen, 1994; Kearney, 2001; McCormick, Rodney, & Varcoe, 2003; Nelson, 2002; Paterson, 2001; Paterson, Thorne, & Dewis, 1998; Sandelowski & Barroso, 2003; Thorne & Paterson, 1998). Each of the methods and subsequent applications in the nursing field has generated a wealth of understanding of strengths and limitations of various aspects of qualitative cross study analysis.
As discussed previously, qualitative cross study analysis or synthesis can be approached aggregatively or interpretively. A qualitative metasummary is defined as “a quantitatively oriented aggregation of qualitative findings that are themselves topical or thematic summaries or surveys of data” (Sandelowski & Barroso, 2007, p. 151). In essence, a metasummary incorporates quantitative research methods to express correlations and findings, while a qualitative synthesis lacks any trace of quantitative research methods, maintaining its inherent qualitative identity. Sandelowski and Barroso (2007) have also used the term ‘meta-synthesis’ to describe this method.
There are many interpretive methods 2 for qualitative synthesis; those that seem most commonly used include: meta-ethnography, meta-study, grounded formal theory, and cross-case analysis. In preparation for this paper, we studied these in depth and reviewed worked examples of each, identifying what we wanted to keep from each of these approaches and what we wanted to address in our method in terms of limitations. Before presenting our method, we define these methods briefly along with noting limitations of each.
Meta-ethnography seeks to explicate relationships between and within individual studies through metaphors (Noblit & Hare, 1988; worked examples: Barroso & Powell-Cope, 2000; Beck, 2001, 2002; Jensen & Allen, 1994; Nelson, 2002; Paterson et al., 1998). However, this methodology does not suggest a sampling method or strategies regarding appraisal of individual studies. Grounded Formal Theory is an extension of grounded theory and utilizes the constant comparative method for data collection, analysis, and theory development (Strauss & Corbin, 1998; worked examples: Finfgeld, 2000; Kearney, 2001). The limitation to this method lies in the possibility of theoretical saturation being achieved prior to the inclusion of all of the relevant studies. Furthermore, this methodology does not offer an explanation on how to address this issue. Cross-case Analysis was suggested by Miles and Hubberman (1994), and consists of a technique to identify categories within individual studies, refine, and cross-reference with other studies (worked example: McNaughton, 2000). Again, similar to grounded formal theory and meta-ethnography, this method fails to provide guidance related to sampling or inclusion criteria. Finally, a meta-study is a highly systematic process involving several evaluative phases prior to the actual synthesis: meta-theory, meta-method, and meta-data analysis (worked example: Watkins et al., 2010). These phases are then followed by the actual meta-synthesis. Furthermore, this method provides guidance regarding sampling and appraisal techniques.
Our proposed method: Qualitative interpretive meta-synthesis
From our standpoint, in presenting this model, our goal of an interpretive qualitative meta-synthesis is not to generate a systematic review, a literature review, or quantify qualitative data, but to create a synergy of qualitative findings. However, as in all qualitative research, some aspects of the process will vary as it is emergent and contextual in nature. Throughout the presentation of the method, we note these areas for researchers who choose to conduct qualitative interpretive meta-syntheses. The first step in presenting our model is to define it: considering the three words in “interpretive meta-synthesis'”, “interpretive” meaning that we eschew aggregating findings quantitatively; “meta” “denoting a change of position or condition” and “synthesis” being “the combination of ideas to form a theory or system” (Meta, 2011; Synthesis, 2011). We conceptualize qualitative interpretive meta-synthesis (QIMS) as a means to synthesize a group of studies on a related topic into an enhanced understanding of the topic of study wherein the position of each individual study is changed from an individual pocket of knowledge of a phenomenon into part of a web of knowledge about the topic where a synergy among the studies creates a new, deeper and broader understanding. This can be considered akin to social work's person-in-environment approach to practice. Just as each person a social worker interacts with is not a lone island but rather a part of a system of relationships with other people, organizations, policies, and environments, so too, each individual qualitative study captures only a snapshot of the human experience of a phenomenon. Qualitative studies give in-depth views of a particular phenomenon experienced by a particular group in a particular situation yet the combination of these studies in QIMS allows us to see what is the shared human experience of this phenomenon and what aspects may be divergent.
The development of QIMS
In developing this method, we have drawn from multiple approaches to qualitative cross study analysis in an attempt to operationalize a method specific to the field of social work. As we describe our proposed method for conducting QIMS in social work, we wish to emphasize that this is not a linear process but rather an iterative one as illustrated in Figure 1, depicting the method. Since developing the method, we have implemented it in 22 different cross study analyses across a wide range of social work topics, using these implementations to refine the steps.
3
In addition to describing the proposed methodology, a partial worked example from one of 22 qualitative interpretive meta-syntheses we have conducted (Aguirre & Bolton, forthcoming) to date will be used to enhance understanding of each step. The partial worked example is a QIMS on volunteer motivations in crisis settings (Aguirre & Bolton, forthcoming).
Meta-synthesis path to synergistic understanding.
The steps in QIMS
The first step in beginning a QIMS is to formulate a research question. Once this is accomplished, a sample is selected, followed by steps in analysis (theme extraction, theme synthesis, triangulation, etc.), and credibility reporting.
Sampling
Sampling for QIMS is a combination of purposive and theoretical sampling, and is common practice in qualitative research. Purposive sampling is used to initially select studies followed by theoretical sampling to test, add, and elaborate on the emerging analysis (Dixon-Woods et al., 2006). Sample selection in a QIMS differs slightly from traditional sampling techniques used in literature reviews and systematic reviews. Researchers should cast a broad net including grey literature (i.e., dissertations or unpublished studies), books, and studies from various disciplines. Furthermore, literature searches should be exhaustive in nature to ensure the inclusion of all relevant studies in the synthesis.4,5 Limiting searches to internet databases can limit the overall scope of the synthesis and result in omission of pertinent data. Throughout the search process, we recommend that researchers develop a quorum chart to depict the process of data collection, review, elimination, and inclusion. Quorum charts are commonly found in systematic reviews or meta-analyses and offer a precise and rigorous outline of the sampling process. 6
Things to consider: Traditions, context, temporal relevance and fatal flaws
After exhausting all resources and compiling studies related to the research topic, the next step is to narrow the list of studies which may include consideration of traditions, context, temporal relevance and fatal flaws. In the existing literature on qualitative synthesis, there are varying opinions regarding whether or not to include studies in the sample from varying qualitative traditions. Some researchers are cautious of synthesizing findings from qualitative studies generated using different qualitative traditions out of concern that there would be a misrepresentation of the original research (e.g., Jensen & Allen, 1996). We certainly recognize the differences 7 among traditions and are aware of the concern of comparing “apples to oranges” (Padgett, 2008). However, we, along with others (e.g. Finfgeld, 2003), encourage including studies from various traditions for two major reasons. First, the wide range of philosophical and methodological traditions researchers employ (Padgett, 2008), each expose a different aspect of a phenomenon. For example, ethnographies focus on culture and phenomenologies on the lived experience of a phenomenon – each provides a portion of a richer picture or understanding of the phenomena under study. Including studies of various traditions such as phenomenology and ethnography in a QIMS further advances social work's goal in research to understand a given phenomenon across cultures and situations so as to develop and improve client-centered policy, theory, advocacy, and services.
To illustrate our second reason, we consider the point at which these various traditions and philosophies impact a study. These are employed at every stage of the study to arrive at the essence of the experience under study. However, the resulting essence is reported in units that are easily compared across these traditions and philosophies. As illustrated later in this paper, data gathered from the articles for a QIMS are not to be rewritten, reworked, or analyzed. In fact, it is our goal to maintain the original themes from the articles included in the sample, so as to decrease the opportunity for researcher bias and increase reliability across findings. We conceptualize this similarly to the argument for including studies in a meta-analysis that employ various statistical techniques. In a meta-analysis, the results are compared on the basis of an effect size, a result that each study produces regardless of whether using analysis of variance, multiple regression, etc. This is similar to our procedure in the sense that we take the themes demonstrated in the qualitative studies and synthesize these across the sample selected. We are not by any means altering the methodological approaches nor the impact these had on the processes in the original studies; we are simply focused on the end results presented by the original researchers.
In addition to considering traditions when narrowing the sample, contextual relevance is of concern as well. Studies related to the research topic may differ from the context of the research question and should be discarded accordingly. For example, if conducting a QIMS on motivations to volunteer in crisis situations, articles on volunteer motivation related to non-crisis situations would need to be discarded even though these are related to volunteer motivation. During the process of discarding studies, it is important that the researcher not eliminate a study because it is a “negative case” (or disconfirming case). Patton (2002) emphasizes the importance of the “negative case” because “our understanding of those patterns and trends is increased by considering the instances and cases that do not fit within the pattern” (p. 554). For example, one study may have findings or conclusions different from all other studies in the sample. This “negative case” needs to be acknowledged and incorporated similarly to the way a “negative case” would be in a qualitative study. Including and maintaining the integrity of this “negative case” is imperative to the quality and trustworthiness of the QIMS as a whole.
Similar to the contextual relevance is the temporal relevance. Determining the temporal relevance of each study helps maintain the relevance of the synthesis and increases its transferability (Sandelowski, Barroso, & Voils, 2007). For example, a qualitative interpretive meta-synthesis on the use of new technologies in counseling done in the 21st century would not include previous research on use of the telephone in counseling since the telephone is not a “new” technology.
A final aspect to consider in narrowing the sample is “fatal flaws” (Dixon-Woods et al., 2006) and these should be eliminated from the sample at the researcher's discretion. Fatal flaws can include, but are not limited to, researcher bias, lack of triangulation, or questionable trustworthiness. Furthermore, researchers must differentiate between fatal flaws and poor data presentation. If the text presenting a study is sub-par, it may be included because data within the paper could remain a vital piece to the sample itself. Sub-par studies would include manuscripts that are poorly organized, difficult to read, or fail to present writing at a scholarly level. Even if the presentation of a study is sub-par, the content may be rich with data pertinent to the topic of inquiry. The process of differentiating between fatal flaws and sub-par text is subjective in nature and elimination is at the discretion of the researcher conducting the QIMS. The lack of specific criteria for elimination may be viewed as a limitation but the process of determining whether a study contains a fatal flaw is commonly done in systematic reviews, literature reviews, and meta-analyses as well with the individual researchers” deciding the criteria.
Theme extraction
Example table of studies included in a QIMS.
Example table of theme extraction in a QIMS.
Synthesis of themes
The identification of themes in each study is followed by the actual data synthesis. Once themes are recorded, the studies are translated into one another. This process of translation must maintain the integrity of each individual study while allowing for the synthesis of similar themes. Noblit and Hare state: an adequate translation maintains the central metaphors and/or concept of each account in their relation to other key metaphors or concepts in that account. It also compares both the metaphors and concepts and their interactions in one account with the metaphors or concepts and their interactions in other accounts. (1988, p. 28)
Jensen and Allen (1996) state that a meta-synthesis “is credible when it re-presents such faithful descriptions or interpretations of human experience that the people having that experience would immediately recognize it from those descriptions or interpretations as their own” (p. 556). In other words, failure to maintain the integrity of the original studies results in decreasing the trustworthiness of the meta-synthesis. One way to limit the loss of original integrity is to utilize participant quotes from the original research reports included in the interpretive meta-synthesis.
Example table synthesis of themes in a QIMS.
Triangulation: Synergistic understanding or entropy
Familiarity with triangulation is imperative before engaging in a QIMS. 8 The purpose of triangulation in QIMS is similar to that of triangulation in a qualitative study. Triangulation is a method used to regulate the trustworthiness of qualitative research; specific to QIMS, this is a means of verifying that translation across studies has provided a synergistic understanding rather than a disordered and biased misunderstanding – entropy. There are four types of triangulation: data collection methods, tradition, sources, and analysts (Patton, 2002). All four types of triangulation can and should be utilized in a QIMS. Triangulation of data collection methods, tradition, and sources are inherent in the process with various studies providing diversity in the three areas. For example, 1) a synthesis may include studies where data collection methods included interviews, focus groups, and observation; 2) synthesizing across traditions provides triangulation of traditions; and 3) multiple qualitative studies bring multiple sources of data (e.g., multiple participants' perspectives). Triangulation of analysts is the key in any type of qualitative analysis, with QIMS being no different. This is especially the case for researchers who include their own studies to conduct a qualitative synthesis, as they will need an objective analyst in the triangulation process. This will help prevent the inclusion of information from the original data not included in the final report used in the synthesis.
Description of synergistic understanding
The final steps of a QIMS are description of the phenomena and synergistic understanding. Description of the phenomena is when the researchers begin to develop their written report. In some cases, the researchers may realize their findings are inadequate and further research is required in order to strengthen the breadth of the study. This may involve alteration of the research question to widen the sampling pool through theoretical sampling or adjusting the parameters of the search terms. Regardless, it is important for the researcher to complete the process in the same order (see Figure 2) to maintain the systematic process of the QIMS. This phase is relatively subjective and may vary from researcher to researcher. The result will be synergistic understanding where the researcher is able to generate conclusions, theory, and implications based on the description generated from the synthesis of the included studies.
Cycles to synergy: Data extraction for synergistic understanding.
Limitations
All methods of research encompass limitations associated with data collection, data analysis, and researcher bias – and QIMS is no exception. The main criticism of qualitative research is the subjectivity of the data analysis and the uncontested question of potential researcher bias. All research, both qualitative and quantitative, is subject to possible researcher bias and the role of the researcher is to be aware and address limitations in every research initiative and all levels of research inquiry. Fortunately, qualitative research has established specific procedures for limiting and minimizing researcher bias, especially through the four levels of triangulation previously discussed. A primary criticism of qualitative cross study analysis in other fields is the recommendation we have presented here to synthesize across traditions. As noted previously, this allows for the different traditions to shed light on the different aspects of a synergistic understanding demonstrated in QIMS. Without inclusion of all relevant available studies, regardless of tradition, the understanding would be partial.
Implications for social work
QIMS provides a structured methodology for further, synergistic, understanding of phenomena with richness in diversity of settings, participants, and qualitative traditions. These synergistic understandings would be grounded in studies generated across different qualitative traditions yet focusing on clients who share some similarity depending upon the research question. Synthesis of qualitative studies and the emergence of synergistic understanding increases efficacy in integrating qualitative research into evidence-based practice. It is our hope that the social work community will embrace the technique of QIMS not only to increase the understanding of phenomena but to strengthen the perception of qualitative research as a rigorous component in evidence-based practice.
Footnotes
Notes
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.