Mixed Methods in the Organizational Sciences

Mixed Methods Research: Toward Formalization of a Distinct Methodological Approach in Organizational Sciences

When considering mixed methods research, it is important to first understand what method really means. Although we often use the term method as a heading in our articles to refer to a study’s description, research design, measures, and processes, some use the term to describe data sources, data analysis, and even as a source of bias (e.g., common methods). With such diverse applications, attention to understanding what a method is will provide us with directions on how we might create and implement mixed methods studies. Fortunately, there appear to be many possible opportunities for designing such studies.

In his influential work on method variance, Spector (2006) notes that initial efforts to define methods have produced highly diverse approaches. For example, Campbell and Fiske (1959) refer to different item formats within a questionnaire as different methods because each can be a source of common methods variance. Since this early work, diverse approaches emerged within organizational research, with Spector observing that “methods vary in large number of ways, and it is not always clear what the critical features of methods might be that can define them” (pp. 227-228). For our purposes, we adopted an inclusive and broad perspective toward defining a method. We considered a method as encompassing data sources, measurement approaches, and analytical techniques.

Social science researchers have integrated qualitative and quantitative methods for many years. However, a coherent conceptualization of “mixed methods” research did not emerge until the 1980s (Bryman, 1988; Greene, Caracelli, & Graham, 1989). Since then, the practice of integrating methodological approaches has developed rapidly, emerging as a research methodology with a recognized name and distinct identity (Denscombe, 2008), especially in education, health sciences, psychology, and sociology. In these fields, mixed method studies have become recognized as a third methodological movement, along with qualitative and quantitative research. Therefore, the practice of integrating quantitative and qualitative methods is not new, as many scholars in other areas, and in our own organizational field, have integrated qualitative and quantitative methods for many years, although they may not have characterized works as “mixed methods” per se. Nonetheless, the knowledge base of the decisions and implications associated with conducting mixed methods studies has lagged behind their application. Our Feature Topic was motivated in part to help narrow the gap between what researchers know about mixed methods research and what they need to know when applying this particular methodological approach.

More specifically, our understanding of the conceptual underpinnings of mixed methods research began around 1980 (Greene, 2015). As Hesse-Biber (2015) pointed out, the mixed methods approach has undergone a dramatic transformation within the past three decades. This contemporary movement toward formalization of mixed methods research is serving to solidify specific theoretical and research design practices. This transformation and formalization of the mixed methods movement, especially in some fields such as education and health sciences, is reflected in the publication of several books and handbooks focused on mixed methods (e.g., Andrew & Halcomb, 2009; Bergman, 2008; Brannen, 1992; Creswell, 2015; Creswell & Plano Clark, 2007, 2011; Curry & Nunez-Smith, 2015; Greene, 2007; Hesse-Biber & Johnson, 2015; Mertens, 2005; Morse & Niehaus, 2009; Niglas, 2004; Plano Clark & Ivankova, 2016; Tashakkori & Teddlie, 2003, 2010; Teddlie & Tashakkori, 2009), the publication of special issues on mixed methods in journals from these fields, the publication since 2007 of a specific interdisciplinary methodological journal focused on mixed methods (Journal of Mixed Methods Research), and the establishment of an international mixed methods association in 2013 (Mixed Methods International Research Association).

However, mixed methods is not so institutionalized and formalized in organizational sciences. Hesse-Biber (2010) points out that the practice of mixing methods was not something readily mentioned in research reports and publications during many years. In fact, most social scientists in the early 1900s up until very recently, even if they used mixed methods, did not explicitly say so. Therefore, much of the early practice has gone under the radar of social research methods history or might have been called something different. This aspect is stronger in organizational research, even in the past few years. Organizational scholars have integrated qualitative and quantitative methods before mixed methods research had been “invented” or legitimized as a recognized approach. Moreover, in the past few years, organizational researchers are also integrating quantitative and qualitative methods without using the “mixed methods” approach to refer to their studies. Furthermore, literature reviews on the use and application of different methods usually do not take into account mixed methods as a separate category, even recently. For example, Scandura and Williams (2000) do not take into account the integration of quantitative and qualitative methods in their content analysis about research methodology in management. As another example, in the seven-part series, “Publishing in AMJ,” published in 2011 and 2012 in which the Journal’s editors offer suggestion for improving the quality of submissions to the Academy of Management Journal, there is not an editorial devoted to mixed methods. Furthermore, in their content analysis of the first decade of Organizational Research Methods, although Aguinis, Pierce, Bosco, and Muslin (2009) take into account mixed methods, they consider this methodological approach as a subcategory within quantitative research, specifically as a quantitative design subcategory. Moreover, in most call for papers in journals in our field, guest editors usually point out that they welcome qualitative and quantitative empirical work, but mixed methods is typically not explicitly indicated.

Therefore, the term mixed methods research has received inconsistent adoption within organizational research although this kind of research is generally welcomed by journals across organizational research. For example, in the strategic management literature, Hitt, Gimeno, and Hoskisson (1998) indicated that research projects may realize the benefits and advantages of both quantitative and qualitative research approaches by integrating them in a single project. Boyd, Gove, and Hitt (2005) pointed out that qualitative research complements quantitative research, and in tandem, quality research of both types can move the strategic management field forward more rapidly. Armstrong and Shimizu (2007) believed that using both qualitative and quantitative methods best contributes to isolating potentially unobservable resources and testing the resource-based view. Calls for the integrated use of quantitative and qualitative research have appeared in other literatures as well (e.g., Coviello & Jones, 2004; Hoang & Antoncic, 2003; Ritchie & Lam, 2006).

Regardless of whether the term mixed methods is used, the integration of research methodologies has a distinguished history in organizational research, dating back to Jick (1979) and Van Maanen (1979). Moreover, over a decade ago, an important emphasis was made not only in the general idea of integrating quantitative and qualitative methods but also in the specific movement of mixed methods in organizational areas. In this regard, in 2006, the Journal of Management Studies published a Point-Counterpoint section about research methods in management research. Two review articles appeared in this section: Echambadi, Campbell, and Agarwal (2006) discussed quantitative research while Shah and Corley (2006) considered qualitative research. Although no specific issue focused on mixed methods per se, these two articles emphasized the importance of integrating quantitative and qualitative methods.

In addition, in 2006, Hurmerinta-Peltomaki and Nummela provided the first systematic review about the prevalence, use, and added value of mixed methods in an organizational area (international business research). Over the past decade, other similar reviews have been published, including the resource-based view (Molina-Azorin, 2007), human resources management (Grimmer & Hanson, 2009), strategic management (Molina-Azorin, 2012), entrepreneurship (Molina-Azorin, López-Gamero, Pereira-Moliner, & Pertusa-Ortega, 2012), information systems (Venkatesh, Brown, & Bala, 2013), project management (Cameron, Sankaran, & Scales, 2015), and environmental management (Molina-Azorin & López-Gamero, 2016).

Regarding these reviews, studies that integrate quantitative and qualitative methods were identified and considered. However, the term mixed methods is not typically used. In this regard, in a prevalence investigation across multiple disciplines, Ivankova and Kawamura (2010) conducted an electronic search intentionally focused only on studies labeled as mixed methods. These authors found only 2 mixed methods studies in business and 13 in management, while they found 325 mixed methods studies in health and medicine and 146 works in education.

In any case, the term mixed methods is not commonly used in organizational research, and there is no apparent institutionalization of mixed methods research within the field. As such, we can learn from other disciplines that are further down the learning curve in terms of their adoption of mixed methods approaches. As such, we seek to elucidate the important benefits and advantages of mixed methods research.

Added Value and Advantages of Mixed Methods Research

A key aspect of mixed methods research involves the benefits gained from integrating the different methodologies into one study. Therefore, researchers must address the integration issue (Bryman, 2006; Fetters & Freshwater, 2015). The key question that arises is: What synergy can be gained by the additional work of integrating both qualitative and quantitative methods? This aspect urges scholars to carefully plan their works with intentional choices that can leverage integration. The necessary and sufficient conditions of applying mixed methods research is to produce insights that exceed the the sum of the individual qualitative and quantitative components.

The central premise of mixed methods studies is that the integration of quantitative and qualitative approaches offers the promise of attaining a better understanding of research problems and complex phenomena than either approach alone (Creswell & Plano Clark 2007), generally through triangulating one set of results with another and thereby enhancing the validity of inferences. In fact, the concept of the triangulation of methods was the intellectual wedge that eventually broke the methodological hegemony of the monomethod purists (Tashakkori & Teddlie, 1998). Early on, Jick (1979) discussed triangulation in terms of the weaknesses of one method being offset by the strengths of another. It is often stressed that different methods have different weaknesses and strengths, and therefore triangulation offers the potential overcome the weaknesses of any single method. Thus, if we use several different methods for investigating the phenomenon of our interest and the results provide mutual confirmation, we can be more sure that our results are valid (Niglas, 2004).

Other purposes, advantages, and rationales for integrating qualitative and quantitative methods exist. Greene et al. (1989) point out additional benefits from triangulation: complementarity (elaboration or clarification of the results from one method with the findings from the other method), development (when the researcher uses the results from one method to help develop the use of the other method), and expansion (seeking to extend the breadth and range of inquiry by using different methods for different inquiry components).

Further, Edmondson and McManus (2007) indicated the conditions under which mixed methods research is helpful. These authors propose that the two methods can be integrated successfully in cases where the goal is to increase validity of new measures through triangulation and/or to generate understanding of the mechanisms underlying quantitative results in at least partially new territory. Moreover, the state of current theory and literature influences when hybrid research strategies are appropriate. Thus, just as quantitative methods are believed to be more appropriate for examining more mature theories while qualitative methods are generally attributed most beneficial for those in the early stages, theories at their more intermediate maturation stages may be particularly well served by a blend of both. The integration of qualitative data to elaborate a phenomenon and quantitative data to provide preliminary tests of relationships can promote both insight and rigor when appropriately applied.

Moreover, we must also take into account that the knowledge about mixed methods research can stimulate a researcher not only to provide a better understanding, analysis, and response to our research questions but also to better define and analyze innovative problems and research questions in organizational research. Mixing methods therefore offers enormous potential for exploring new dimensions. In this regard, two alternative positions about the relationship between questions and methods are indicated in the literature. On one hand, Teddlie and Tashakkori (2010) describe the dictatorship model in which the research question determines the specific methods used within a given study. Thus, a researcher will select the best tools available in their methods toolbox to answer the stated questions. Then, the use of mixed methods research becomes appropriate if, and only if, it is called for by the research questions. This model fits well with a linear process of research where the methodology is the servant of questions. On the other hand, an alternative interactive or systemic model places research questions at the hub of the research process (Maxwell, 2013; Maxwell & Loomis, 2003; Plano Clark & Badiee, 2010). In this model, research questions are directly related to four study components: purposes, theories and beliefs, methods, and validity considerations. While research questions play a central role, this model views them as interacting and integrated with the other interrelated components. When we focus on methods as one of the components, the idea is that research questions inform and are informed by methods, that is, there is a reciprocal relationship between questions and methods. Research questions influence the methods we use, but methods may also influence the research questions we ask. Therefore, research questions shape and are shaped by methods. This reciprocal relationship can be represented as a double-headed arrow between research questions and methods. In any case, both the dictatorship model and the reciprocal approach share the idea that the key point is the fit between questions and methods.

Then, if methods may influence the research questions, an important consequence is that by extending our methodological skills, we can improve the question-asking process. After we complete our graduate studies, we may rely on the methods we learned in our educational careers and may face some inertia to other approaches. Through extending and sharpening our methodological skills to include different methods, we can increase the rigor of our conceptual thinking, see new ways to answer research questions, and even identify questions that would not have occurred to us otherwise (Edwards, 2008). Here mixed methods can play a key role. Because mixed methods research integrates quantitative and qualitative methods, the researcher is motivated to develop a broader set of research skills. Training in mixed methods can overcome the tendency to rely on known methods and play an important role in widening and extending our toolbox and repertoire of methods if the training emphasizes the importance of integrating, comparing, and mixing different methods.

As noted previously, the possible combinations of mixed methods research provides scholars with numerous advantages, including flexible and adaptive tools for studying the idiosyncracies of their topics. And although such approaches may be costly and time intensive, the insights they can provide appear to be rewarded within the citation marketplace. In a survey of 598 Strategic Management Journal (SMJ) articles appearing during the 1990s, a subset of qualitative study, primary data, was collected in just over 20% of these articles, but such studies received higher citation counts than those that used only secondary data (Bergh, Perry, & Hanke, 2006). These primary data studies were not exclusively mixed methods but give some insight into the differentiating value of the data source. Extending these findings, Molina-Azorin’s (2012) assessment of 165 SMJ articles employing mixed methods approaches were positively related to article citation counts, concluding “that on average mixed methods have a higher impact than monomethod studies” (p. 45).

This Feature Topic

In this section, we summarize the manuscripts selected for this Feature Topic. Overall, we received 53 proposals. Of these, the authors of 12 proposals were invited to send full papers. We ultimately accepted 6 manuscripts for publication. The works contained in this special topic forum advance knowledge about mixed methods research along multiple fronts.

Insights about mixed methods research can be gleaned by distilling trends within past mixed methods work. This is the approach taken in Gibson’s article, “Elaboration, Generalization, Triangulation, and Interpretation: Enhancing the Value of Mixed Method Research.” The author takes her own mixed methods approach in examining 69 recently published mixed method studies and explores 4 of them in depth, including interviewing their lead authors. In doing so, Gibson identifies five approaches to mixed methods research, including three that are not yet well understood. Next, she identifies four potential strengths of mixed method research—enhanced capacity for elaboration, generalization, triangulation, and interpretation. Finally, Gibson presents a series of practical suggestions by which researchers can improve their use of mixed methods designs.

As indicated previously, the concept of integration lies at the heart of mixed methods research. In their article, “Strategies of Integration in Mixed Methods Research: Insights Using Relational Algorithms,” Tunarosa and Glynn seek to make progress toward improved integration across intellectual paradigms in order to generate stronger explanations. First, they rely on a linguistic technique called relational algorithms to reveal that connector words—along, near, and within, for example—are stronger channels than the more common word and for bridging quantitative and qualitative methods. The authors also outline five approaches to integration and provide examples from the literature. Management research often includes two of these approaches—conjoined and sequential—but the potential of three others—simultaneous, full-cycle, and mono-logic—remains relatively untapped.

As noted previously, triangulation is a process of validating research conclusions by examining a relationship from different methodological angles. If diverse perspectives lead to the same conclusion, then confidence in that conclusion is heightened. In their article, “Research Design for Mixed Methods: A Triangulation-Based Framework and Roadmap,” Turner, Cardinal, and Burton seek to build a mixed methods approach to research design that is grounded in triangulation and build on this approach to offer tangible suggestions for scholars who wish to perform studies using mixed methods. The outcome of this effort is a four-stage process—decide on the research question, determine theoretical intentions, select the process for triangulation, and determine the mix of methodologies—that can guide researchers in selecting among methods such as archival analysis, case studies, simulations, laboratory experiments, mathematical formulations, interviews, surveys, and others.

Social network analysis is a powerful method, but it has yet to make major inroads into mixed methods research. In their article, “Mixed Method Social Network Analysis: Combining Inductive Concept Development, Content Analysis, and Secondary Data for Quantitative Analysis,” Williams and Shepherd develop a mixed method approach to social network analysis. They do so by combining organizational history development, inductive data structuring, and content analysis. In contrast to other social network methods, the authors’ approach offers the ability to investigate larger networks, the ability to tap into a wider range of actors, reduced bias among informants, and a better capacity to use longitudinal designs. The authors demonstrate their method using data from 143 new ventures. They set the stage for future application of the method by offering suggestions for using it in studies of entrepreneurship, strategic management, and organizational behavior.

Ethnography can be an especially potent element of mixed methods designs because of its ability to uncover new concepts and relationships. Generally, researchers perform their ethnographies first and then test the generated ideas via empirical methods such as surveys and archival analysis. In their article, “Network Ethnography: A Mixed Method Approach for the Study of Practices in Interorganizational Settings,” Berthod, Grothe-Hammer, and Sydow introduce a design called network ethnography wherein qualitative and quantitative data are collected and analyzed concurrently. The authors implement this approach by studying interorganizational networks (IONs) both through social network analysis—a dominant technique for studying IONs—and ethnography. The overall aim is to gain greater understanding of the process elements of networks as well as their dynamics across levels of analysis.

The last article involves qualitative comparative analysis (QCA) and hierarchical linear modeling (HLM), two analytical techniques that have grown in popularity in recent years. In their article, “Integrating QCA and HLM for Multilevel Research on Organizational Configurations,” Meuer and Rupietta take on the challenge of developing a method for integrating qualitative and quantitative approaches across levels of analysis. They view this task as important because organizational research involves a variety of multilevel and cross-level concepts. The authors demonstrate their integrated approach to QCA and HLM using data from 1,201 Swiss firms. By providing this illustration as well as step-by-step guidance on how to implement the technique, the authors aim to provide researchers with a means to develop stronger research designs and heightened confidence in the findings that the designs provide.

Collectively, these studies’ insights identify new research arenas, specify how established research approaches can be integrated in new and novel ways, and create new models for value creation in the research enterprise.

Recommendations, Implications, and Challenges to Moving Mixed Methods Forward

As noted previously, other fields have longer traditions of using and examining mixed methods research. We suggest that organizational scholars can learn from the experiences of scholars in other fields regarding recommendations, implications, and challenges of mixed methods research.

An important recommendation for mixed methods studies is the explicit clarification of several relevant aspects in the written report. A key issue is to determine the core reason or rationale for collecting both forms of data and provide a clear rationale for the interrelationship between the quantitative and qualitative phases (Creswell, Plano Clark, Gutmann, & Hanson, 2003). Hanson, Creswell, Plano Clark, Petska, and Creswell (2005) provided some recommendations for designing, implementing, and reporting a mixed methods study, highlighting that researchers attend closely to design and implementation issues, particularly to how and when data are collected (e.g., concurrently or sequentially). Further, these authors suggest that researchers familiarize themselves with the analysis and integration strategies used in the published mixed methods studies. Clear, well-written purpose statements and research questions that specify the quantitative and qualitative aspects of the study help focus the manuscript. Additionally, these authors recommend that in the introduction, researchers explicitly state a rationale for mixing quantitative and qualitative methods and data (e.g., to triangulate results, to develop or improve one method with the other, to extend the study’s results).

Teddlie and Tashakkori (2006) argue that sometimes mixed methods designs may have an opportunistic nature. Thus, in many cases, a mixed methods research study may have a predetermined research design, but new components of the design may evolve as researchers follow up on leads that develop as data are collected and analyzed. These opportunistic designs may be different from those contained in previously published typologies. The point is for the researcher to adopt a creative and innovative mindset and not be limited by existing designs. Indeed, a new methodological approach may emerge, depending on the conditions and information that is obtained. For example, the well-known Gioia methodology evolved as a hybrid methodological approach through the author’s experiences with inductive research (e.g., Gioia, Corley, & Hamilton, 2013). Thus, a tenet of mixed methods research is that researchers should mindfully create designs that effectively answer their research questions (Johnson & Onwuegbuzie, 2004).

Finally, there is rising interest in the role of replication in the social sciences (Open Science Collaboration, 2015). In general, replication is an important part of the development of a cumulative body of scientific knowledge. Although replication is not considered an issue for most qualitative research, it is for mixed methods research given the ultimate purposes of mixing qualitative and quantitative data. In order to maximize the potential for subsequent research to be able to replicate a mixed methods study, researchers need to be as transparent as possible in reporting their methodological decisions and the rationale behind those choices. When done well, future researchers should be able to understand how both qualitative and quantitative data were collected, analyzed, and integrated such that similar methodological efforts could be recreated in different contexts.

An interesting implication and opportunity for mixed methods research in organizational sciences is its potential to examine and addressing “grand challenges.” Grand challenges are ambitious but achievable objectives that harness science, technology, and innovation to solve important problems. They include issues arising from environmental degradation and climate change, poverty, health, social and economic inequality, and geopolitical instability, among many others, and may call for new theories, concepts, and methods (George, 2014). Researchers who place high value on contributing to changes that are necessary to avoid the negative consequences of environmental degradation, poverty, and inequality believe that the ultimate measure of the quality of their research should be associated with action and change. Grand challenges are complex and contextually bound. Mixed methods approaches can help address these problems because they allow researchers from diverse groups to have a common language to guide their inquiry, participants and stakeholders from vulnerable groups to be included in culturally appropriate and supportive ways, and policymakers in these contexts to be part of the process of problem and solution identification and documentation.

Research from other fields, however, also points to challenges that mixed methods researchers face in balancing rigor and practical relevance. From an academic point of view, rigor (in theory development, research design and inferences, and conclusions) is usually specified as the main indicator of the quality of a study. Yet, scholars must also consider the practical relevance of their work. And undoubtedly, the purpose of trying to solve grand challenges plays a key role within this practical relevance of research. The use of mixed methods research may facilitate and enhance the interpretation of the results obtained in order to emphasize the practical implications of a study. With regard to this practical impact, mixed methods can be used to understand the extent to which a study’s results are significant in practice by including practitioners’ own discourses, with an awareness of inclusion of intended beneficiaries of interventions in culturally respectful ways. Aguinis et al. (2010), with the goal to bridge the science-practice gap, pointed out that to demonstrate a study’s practical significance, there is a need to describe quantitative results in a way that makes sense for practitioners. They suggested that this purpose can be achieved by including practitioners in each research project as part of a qualitative study. Therefore, these authors defend mixed methods research where a quantitative study is completed with a subsequent qualitative part where practitioners and other stakeholders become participants and collaborators. In sum, the analysis of grand challenges requires methodological diversity, and then mixed methods may play a key role as this methodological approach promotes the application of a diversity of methods, combining quantitative and qualitative approaches and using information from several and different stakeholders.

Finally, some barriers and challenges of mixed methods research must also be considered. Mixed methods studies require extensive time, resources, and effort and require that researchers develop a broader set of skills. Creswell and Plano Clark (2007) argued that conducting mixed methods research is not easy. Mixed methods may be perceived as requiring more work and financial resources and will likely require more time (Niglas, 2004). Mixed methods research also demands that researchers develop skills that span quantitative and qualitative designs, which may be contrary to some scholars’ training. One way of overcoming this limitation is to develop teams that can bring together specialists in both kinds of methods (Masterson, Corley, & Schinoff, 2016). Creswell (2015) points out that there is a growing presence of mixed methods teams that consist of individuals with different methodological orientations (quantitative vs. qualitative skills) together with team members who have skills in mixed methods. These mixed methods team members may be a bridge between the quantitative and qualitative members, facilitating the conversation about differences in thinking when they appear. Mixed methods teams have members with a range of expertise, hold respect for diverse methodological orientations, and have a good leader who bridges across the areas of expertise and methodological orientations. The team leader ideally has experience in quantitative, qualitative, and mixed methods research. This leader must also pay attention to team composition, give equal treatment to diverse methodologies, help to shape dialogue and values, and involve all team members in decisions.

Another barrier is the perceived challenges of publishing mixed methods studies. From this perspective, researchers may face two kinds of problems (Bryman, 2007; Plano Clark, 2005). One is the tendency for some journals to be perceived as having a methodological bias toward either quantitative or qualitative research, and as a consequence, researchers may believe that the presence of such a bias limits their ability to publish mixed methods studies. The other is that the need to describe and discuss two sets of data collection, data analysis, and findings may push up against word and page restrictions that journals impose on authors.

In sum, organizational research is eclectic and dynamic. To help understand the ever changing nature of the topics and phenomena within organizations, mixed methods research provides important opportunities for researchers to examine and explain relationships that monomethod designs cannot effectively capture. We hope that our Feature Topic helps excite and encourage researchers to adopt mixed methods approaches by recognizing that they are limited only by their own creativity and ingenuity. The articles in this Feature Topic bear witness to the many mixed methods opportunities awaiting for researchers to adopt and explore. We hope that readers will pause to consider how the insights discussed in the following not only address important research questions but also serve as a platform for others to pursue their own mixed methods investigations.