Common “Core” Characteristics of Mixed Methods Research

Methodological Eclecticism

The term methodological eclecticism means MMR practitioners select and creatively integrate the most appropriate techniques from a wide variety of QUAL, QUAN, and mixed strategies in order to thoroughly investigate the phenomena of interest (Teddlie & Tashakkori, 2010, 2011). Methodological eclecticism involves using techniques that encompass the entire QUAL and QUAN “toolboxes.” A researcher employing methodological eclecticism is a connoisseur of methods who knowledgeably, and often intuitively, selects the best techniques available to answer research questions that may evolve as a study unfolds.

Methodological eclecticism stems from rejection of the incompatibility of methods thesis, which stated that it is inappropriate to mix QUAN and QUAL methods due to fundamental differences (e.g., epistemological) between the paradigms supposedly underlying those methods. The rejection of the incompatibility of methods thesis (and substitution of the compatibility thesis) is a major point of demarcation between MMR advocates as opposed to “incompatibilists” (e.g., Gage, 1989; Howe, 1988).

Although there is little disagreement among MMR advocates about integrating QUAL and QUAN methods in their research, there are at least three issues related to this core characteristic. The first concerns the skepticism that critics have about researchers developing the skill sets necessary to perform both types of research adequately. For example, Denzin (2008, p. 322) expressed the concern that methodological bilingualism is “superficial, perhaps even unworkable.” QUAL-oriented critics see this as particularly important because they are concerned that researchers will not be trained properly in the ethnographic tradition, nor will they appreciate the resources required to conduct ethnographies. They believe that many MMR projects will result in “QUAL-light” research.

We believe that a combination of integrated coursework and (particularly) extensive field experiences is necessary to begin the journey toward becoming a connoisseur of methods. Whereas issues of mixed methods pedagogy are beyond the scope of this article, there is an active literature developing in this area (e.g., Earley, 2007) that includes details on how MMR courses can evolve over time to accommodate student needs (Christ, 2010).

We believe that mixed methods researchers must be competent in the full spectrum of research methods and approaches to select the best paths for answering their research questions. We can educate our young scholars for such a broad competency only if they are mentored by scholars who are themselves methodologically bilingual and willing to search for innovative combinations of methods. There are numerous examples of dissertations (e.g., Schulenburg, 2007), program evaluations, and published work by single investigators demonstrating that solo investigators can conduct methodologically bilingual research.

Shulha and Wilson (2003) presented an argument for a team approach in which each researcher in a mixed study has a minimum competency level in both methods, plus expertise in one. Bliss (2008) expands this argument further by suggesting that MMR is only possible in teams. Competency and familiarity with both types of research allows team members to communicate effectively because they have a “common” language with a workable number of conceptual similarities. Once again, there are numerous examples of successful MMR team projects conducted throughout the social and behavioral sciences (e.g., Lieber & Weisner, 2010; Sammons, 2010).

Researchers become more competent as they work collaboratively on projects where they see others applying problem-solving skills from a methodological perspective different from their own. For instance, the Jang, McDougall, Pollon, Herbert, and Russell (2008) study of “schools in challenging circumstances” quoted one of the graduate students involved in the study as follows:

My participation in a mixed methods project expanded my horizons from research methodology as a debate between paradigms that dealt with “people versus numbers” and from an understanding that abstract debates between “either/or” actually, and quite compellingly, dialectically resolve into an “and.” (p. 243)

This qualitatively oriented graduate researcher commented that her “rich” understanding of the QUAL data led her to seek a better understanding of the statistical analyses and graphic displays, which she discovered to be “full of life.” This novice researcher is in the beginning stages of becoming a methodological connoisseur.

The second criticism related to methodological eclecticism is that much contemporary MMR involves relatively unimaginative combinations of QUAL and QUAN methods, such as the integration of QUAN questionnaire and QUAL interview techniques. MMR in the future needs to feature more imaginative blends of traditional and emergent methods to reach its true potential in terms of answering complex, interrelated research questions.

Reliance on combinations of basic methods was necessary in the early stages of MMR when their integration was still highly controversial. Part of the reliance on simple methods was also due to the training the students received, since professors teaching rigorous ethnographic methods or complex statistical applications were more likely to emphasize purely QUAL or QUAN applications. The first generation of professors teaching mixed methods will hopefully spawn a second generation that is more likely to be exposed to and adapt a wider variety of research tools as MMR becomes more popular and as new techniques are developed across the methodological spectrum.

Mixed methods researchers should be “shamelessly eclectic” (Rossman & Wilson, 1994), and the future of the field should feature increasingly interesting mixtures of methods (e.g., mixing geographical information systems and qualitative software; Fielding & Cisneros-Puebla, 2009). Several authors have recently described MMR that integrates more advanced techniques from the QUAL and QUAN approaches, inherently mixed techniques (Teddlie & Tashakkori, 2009), and other methods that are uniquely mixed (e.g., Bazeley, 2010; Song, Sandelowski, & Happ, 2010).

A third issue related to methodological eclecticism concerns supporters of the incompatibility thesis, although their position is clearly less popular than it was 25 years ago. For example, Yanchar and Williams (2006) proposed a softer version of the incompatibility thesis that included specific guidelines for using methods in social inquiry. They proposed these guidelines because “advocates of methodological eclecticism . . . fail to take seriously the inescapable assumptions and values that accompany the use of a method and the pursuit of practically useful results” (Yanchar & Williams, 2006, p. 3, italics added). Mixed methods proponents must continue to engage in philosophical discussions with advocates of the incompatibility thesis and other vestiges of the paradigm debates, a point expanded upon in the next section of this article.

Paradigm Pluralism

An important characteristic of MMR is paradigm pluralism, or the belief that a variety of paradigms may serve as the underlying philosophy for the use of mixed methods. This characteristic is based on the rejection of the one-to-one linkage of methods with paradigms (e.g., constructivism with QUAL methods) or the assumptions of those paradigms (e.g., the linkage of epistemological assumptions with specific methods). Proponents of mixed methods understand that the “oppositional component” of paradigms (e.g., either subjectivity or objectivity) is invalid, because there are other alternatives. ³

Contemporary MMR is a kind of “big tent” in that researchers who currently use mixed methods come from a variety of philosophical orientations, including pragmatism, critical realism (Maxwell & Mittapalli, 2010), the dialectical stance (Greene, 2007), the transformative paradigm (Mertens, Bledsoe, Sullivan, & Wilson, 2010), and so forth. We believe that it is both unwise, and unnecessary, at this time to exclude individuals from the MMR community because they subscribe to different conceptual frameworks.

Whereas multiple paradigms have been associated with the use of mixed methods, most MMR advocates work within the philosophical assumptions of one of the aforementioned orientations, the “preferred paradigm” that best fits their “personal history” (Denzin, 2008, p. 322). Employing multiple frameworks or paradigms in the same study is supported by only one of the contemporary MMR positions noted above: the dialectical stance, which involves using multiple assumptive frameworks within the same study (e.g., Greene, 2007; Greene & Hall, 2010).

Whereas there is general agreement among MMR advocates regarding paradigm pluralism, there are at least three important issues regarding this characteristic. As noted in the previous section, disagreements between MMR advocates and incompatibilists will continue, particularly with regard to what Morgan (2007) calls the metaphysical paradigm that connects assumptions regarding epistemology, ontology, axiology, and methodology. ⁴ Biesta (2010) and Greene and Hall (2010) also object to these interconnected assumptions largely because they lead to the “ownership” of methods by paradigms (e.g., postpositivism and QUAN methods).

Whereas the belief in such “ownership” is stated more subtly now, it still exists as exemplified by the following quote: “The mixed-methods movement takes qualitative methods out of their natural home, which is within the critical, interpretive framework” (Denzin & Lincoln, 2005, p. 9). We do not understand what a “natural home” for any research methods is, because multiple paradigms can be associated with any given method. If researchers want to use QUAN or QUAL methods exclusively, then that decision should be based on their research questions, not some link between epistemology and methods.

The incompatibilists’ arguments will continue to resurface as long as there are philosophical debates in social inquiry. MMR advocates must be diligent and persistent in their defense of the compatibility thesis, responding directly to new stances such as the soft incompatibility thesis and its research guidelines (Yanchar & Williams, 2006).

The linking of methodological issues with paradigm considerations also leads to unfortunate and misleading terms such as quantitative paradigm, qualitative paradigm, and mixed methods paradigm, as noted by others (e.g., Gorard, 2010; Gorard & Taylor, 2004). Mixing these terms contributes to conceptual confusion in MMR.

The second issue regarding paradigm pluralism concerns pragmatism, which has been endorsed as a philosophical orientation by many MMR advocates (e.g., Biesta, 2010; Howe, 1988; Johnson & Onwuegbuzie, 2004; Tashakkori & Teddlie, 1998). Critics are now asking for more details on the characteristics of pragmatism that are associated with mixed methods. Yvonna Lincoln recently stated,

Pragmatism is hiding many a positivist these days. . . . The problem, as I see it, is that pragmatism claimed for some mixed-methods theorists rests at the enacted level only. The mixed methods pragmatists tell us nothing about their ontology or epistemology or axiological position. (2010, p. 7)

Fortunately this criticism is being addressed as more nuanced discussions of pragmatism and its relationship to mixed methods emerge. For example, Johnson and Onwuegbuzie (2004) presented 21 characteristics of pragmatism to delineate how the tenets of that philosophy relate to MMR. In recent writing, Burke Johnson (2009, p. 456) defined dialectical pragmatism as a “supportive philosophy for mixed methods research” that combines classical pragmatism with Greene’s (2007) dialectical approach. The contribution of Johnson and his colleagues’ work is that we now have a more clearly defined picture of philosophical pragmatism related to MMR.

In contrast, Gert Biesta (2010, p. 97) contends that “pragmatism should not be understood as a philosophical position among others, but rather as a set of philosophical tools that can be used to address problems.” Biesta reminds us that John Dewey specifically warned against philosophical system building and concludes that Deweyan pragmatism has made a major contribution to the dismantling of the epistemological dualism of objectivity/subjectivity.

More sophisticated theoretical work such as that by Biesta and Johnson is needed to counter criticisms that mixed methods has no philosophical base.

A third issue related to paradigm pluralism concerns differences between pragmatists and advocates of the transformative paradigm with regard to the relative importance of philosophical assumptions. Epistemological issues tend to be emphasized in discussions of pragmatism such as what is knowledge, how is it acquired, and the relationship between the “knower” and the “known” (objectivity-subjectivity). On the other hand, Mertens et al. (2010, p. 195) emphasize the axiological assumption that “takes precedence and serves as a basis for articulating the other three belief systems because the transformative paradigm emerged from the need to be more explicit about how researchers can address issues of social justice.” For transformative scholars, and critical theorists who promote social justice, mixed methods are tools that are used in the service of the value systems that are foremost to their perspectives.

We believe that pragmatists and scholars who emphasize the axiological assumption are a part of the “big tent” to which advocates of MMR belong. It is important to consider, however, the difference between methods (i.e., specific strategies for implementing research) and methodology (i.e., broad approaches to scientific inquiry specifying how research questions should be asked and answered). As matters of methods move toward matters of methodology, inherent tensions may arise between scholars who emphasize axiological considerations as opposed to those who emphasize epistemological considerations.

Iterative, Cyclical Approach to Research

The third characteristic of contemporary MMR listed in Table 1 is an iterative, cyclical approach to research. MMR is characterized by the cycle of research, which includes both deductive and inductive logic in the same study or program of study (e.g., Krathwohl, 2004; Tashakkori & Teddlie, 1998). Research may start at any point in the cycle: Some researchers start from theories or abstract generalizations, whereas others start from observations or other data points. The cycle may be seen as moving from facts or observations (grounded data or results) through inductive logic to general inferences (theory or abstract generalizations), then from those general inferences (or theory) through deductive logic to tentative predictions or hypotheses related to outcomes or results.

We believe that all mixed single studies or programs of inquiry go through the full cycle at least once, regardless of their starting point (e.g., Teddlie & Tashakkori, 2009). This cyclical approach to research has also been conceptualized (e.g., Johnson & Gray, 2010) in terms of the distinction between (a) the context or logic of justification—the process associated with the testing of predictions, theories, and hypotheses; and (2) the context or logic of discovery—the process associated with understanding a phenomenon in more depth, the generation of theories and hypotheses.

This characteristic is not particularly controversial among MMR advocates, but some authors (e.g., Erzberger & Kelle, 2003; Morgan, 2007) have posited abduction as a third type of logic of particular relevance to MMR. Haig (2005, pp. 372–373) defines abductive logic as “reasoning from phenomena, understood as presumed effects, to their theoretical explanation in terms of underlying causal mechanisms.” Abduction entails creatively generating insights and making inferences to the best possible explanation. Morgan (2007, p. 71) concludes that abduction is a “kind of reasoning that moves back and forth between induction and deduction” and then summarizes some alternative uses of abduction within a pragmatic framework, including serving as a point of connection between QUAL and QUAN research.

In our role as coeditors of the second edition of the Handbook of Mixed Methods, we solicited a chapter on abduction and mixed methods, but were unable to find a knowledgeable author who was ready to write about it. An important issue within the field is the development of concepts and applications related to the use of abductive logic within MMR.

Set of Basic “Signature” Research Designs and Analytical Processes

The fourth characteristic of contemporary MMR listed in Table 1 is a set of basic research designs and analytical processes, which are acknowledged and referenced by most MMR scholars. These signature processes include parallel mixed designs, conversion mixed designs, sequential mixed designs, data transformation (quantitizing and qualitizing), and so forth. We call them “signature processes” because they define mixed methods in relation to QUAL or QUAN research; that is, they are unique to MMR and distinguish this approach from the other two.

Although there is general agreement about the existence of these unique MMR processes, there is considerable disagreement about terminology and definitions associated with them. For example, what we refer to as parallel mixed designs (e.g., Teddlie & Tashakkori, 2009) have also been called concurrent, simultaneous, and triangulation designs, but there is much commonality across their definitions. Inconsistencies in basic terminology and definitions have vexed MMR since its emergence as a separate methodological approach as noted by Bryman (2008):

This lack of a language for writing about mixed methods research derives from the fact that we are trying to run before we can walk. There has been little systematic investigation of mixed methods studies out of which general principles regarding practice and prescription can be honed. (p. 88)

An example of this kind of investigation comes from Johnson, Onwuegbuzie, and Turner’s (2007) definition of mixed methods research, which resulted from their examination of 19 alternative meanings. Their constant comparative analysis of these definitions resulted in five themes, which they then incorporated into a “composite” definition. This systematic approach for defining MMR terms with multiple meaning is a valuable one, which could be used in the construction of a dictionary for the field similar to that developed for qualitative inquiry by Schwandt (1997). ⁵ Such a dictionary could summarize the etiology and various meanings associated with MMR terms.

Conclusions

In this review, we introduced some common characteristics of MMR and presented a few contemporary issues that need to be addressed by scholars writing in the field. There are other critical issues in MMR (e.g., the relative importance of research questions as opposed to research purposes), but space limitations preclude us from discussing them.

The first issue we discussed was whether the field should be headed toward greater convergence (e.g., the common characteristics in Table 1) or continued divergence. We believe that divergent thought will always be a part of MMR (see Characteristic #7 in Table 1), but that it is now time for greater convergence on some basic characteristics and principles. Due largely to the emergence of MMR, future research in the social and behavioral sciences will employ a more eclectic blend of methodologies, which may go by a variety of different names depending on the discipline where they originated. Unless mixed methodologists develop a core identity of commonly understood characteristics and principles that cross disciplinary boundaries, it may simply be absorbed into this eclectic blend of research methodologies.

Nothing in such a set of characteristics or principles implies that the results from mixed methods studies would not (or should not) be used to improve the lives of those who are studied, both directly and indirectly (Tashakkori & Teddlie, 2010a). Nor will these principles lead to the exclusion of innovative areas of inquiry, both in terms of discipline and context (e.g., what Mertens [2010, p. 4] calls “unheard voices”).

If the field does move toward greater convergence, several issues that were raised throughout this article need to be addressed. The remainder of this section presents a “call to action” focusing on issues related to language, pedagogy, and philosophy.

Numerous authors (e.g., Bryman, 2008; Johnson et al., 2007; Teddlie & Tashakkori, 2010) have called for greater consistency in the language of MMR, in terms of consistency both in terms of the names and the definitions of common constructs. Language is very important in an emergent field such as MMR because the words we use to define the field ultimately shape how we make sense of it.

We discussed “signature research designs and analytical processes” (e.g., parallel mixed designs) and how they require greater consistency in labeling and definition. Other terms need to be brought into our lexicon in terms of what they mean within the MMR context. For example, Morgan (2007) put forward an organizational framework for understanding his proposed “pragmatic approach to social science methodology” (p. 73). This framework includes key “pragmatic” concepts such as abduction, intersubjectivity, and transferability that supersede the QUAL/QUAN dichotomies of induction/deduction, subjectivity/objectivity, and context/generality. Further development of these pragmatic concepts “creates a range of new opportunities for thinking about classic methodological issues in the social sciences” (Morgan, 2007, p. 72). We need better specification of what abduction, intersubjectivity, transferability, and other important terms related to social inquiry mean within the MMR context.

Next to the language issue, pedagogy may the most important general issue facing MMR advocates. A specific issue concerns how to integrate ethnographic methodologies within the coursework and field experiences of graduate students studying mixed methods. More generally, we need dialogue regarding the curriculum that graduate research programs in the social and behavioral sciences should require to ensure that students are exposed to a wide variety of methodologies. As important, we need dialogue about the types of research experiences that students require to start their journey toward becoming mixed methodologists and how they can acquire that training, either though dissertation work or in team projects or both. The necessity of a diverse and inclusive training and experiences has received considerable attention in the past few years, both within the MMR community (e.g., see Tashakkori & Creswell, 2008) and within the broader discussion of doctoral education in the United States (Richardson, 2006).

With regard to philosophical issues, we must continue to refine our conceptualization of pragmatism within the MMR context. We foresee a continuing evolution of this topical area with specific lines of discussion, including, among others, Is pragmatism a philosophical system or a set of philosophical tools that can be used to address problems? Is there a specific type of pragmatism that is more relevant to MMR than others (e.g., Deweyan pragmatism or some alternative)? and What are the underlying assumptions of pragmatism related to MMR (e.g., epistemological, ontological, axiological)? Philosophically oriented scholars (e.g., Lincoln, 2010) from the QUAL perspective will continue to criticize the MMR community on these issues.

In a more general sense, MMR scholars must be willing to engage in theoretical conversations with incompatibilists (and others who hold oppositional viewpoints). Many mixed methodologists tend to avoid what they consider to be intractable philosophical issues, because the history of MMR involved researchers turning away from unsolvable “either-or” arguments (top-down approach to research; see Morgan, 2007) and toward conducting applied research (bottom-up approach to research; see Tashakkori & Teddlie, 2010a). We must continue to be engaged in these dialogues to convince the “undecided” members of the research community that the mixed methods approach to social inquiry is viable.

We believe that if the mixed methods community pays attention to these issues of language, pedagogy, and philosophy, then MMR will continue its emergence as the third methodological movement