Strategies of Integration in Mixed Methods Research

Abstract

The challenge of integration, namely, the bridging across different intellectual paradigms to combine empirical insights into a coherent and plausible explanation, is endemic to mixed methods research. In this article, we address this challenge in two ways: first, by drawing attention to the role that theoretical integration plays in mixed methods research as a complement to empirical integration and second, by broadening the repertoire of strategies for enhancing the interplay of theoretical and empirical elements in a mixed methods study. We use the technique of relational algorithms, a linguistic exercise designed to produce “novel relations between pairs of things” by experimenting with different words that can connect theory and empirics. We propose that connector words (e.g., along, near, within) can forge linkages between quantitative and qualitative methods that extend the simple coupling implied by and. We advance five strategies of integration, two that are commonly used in management research—conjoined and sequential—and three high-potential but relatively underused strategies—simultaneous, full-cycle, and mono-logic. We illustrate each of these with examples from the management and organizational literature.

Keywords

mixed methods integration theory

Integration is at the heart of mixed methods research; it is both its greatest advantage and arguably its greatest challenge. As Todd Jick (1979) pointed out more than 35 years ago, a critical step in mixed methods research involves “extensively integrat[ing]” (p. 604) the methods in use to provide a more complete understanding of a complex phenomenon. Although integration is important for both empirics and theory, scholars have tended to focus more on the former, perhaps as a way of disentangling the logistical challenges that result from conducting multiple inquiries within one single study. Yet theoretical integration is core to mixed methods designs: It entails the relational process of “combining two or more sets of logically interrelated propositions into one larger set of interrelated propositions, in order to provide a more comprehensive explanation of a particular phenomenon” (Thornberry, 1989, p. 52). We thus seek to examine theoretical integration in more depth, as a companion to empirical integration, and explore how the interplay between the different modes underpins integration in mixed methods research.

Theoretical integration capitalizes on the rich data that mixed methods research yields to craft the connections among phenomena, a story about why acts, events, structure, and thoughts occur. Theory emphasizes the nature of causal relationships, identifying what comes first as well as the timing of such events … usually laced with a set of convincing and logically interconnected arguments. (Sutton & Staw, 1995, p. 378). And yet, in spite of its criticality, theoretical integration poses a significant challenge for scholars using mixed methods designs, for both poets and quants. Bryman (2006), drawing on his interviews with mixed methods social scientists, succinctly captures scholars’ concerns:

Researchers felt that integrating quantitative and qualitative research at the level of designing the research and developing research instruments did not exercise them greatly. It was bringing together the analysis and interpretation of the quantitative and the qualitative data and writing a narrative that linked the analyses and interpretations that was a cause for concern [italics added]. (p. 9)

The challenge of mixed methods research is this: How does a researcher theorize the explanation for a focal phenomenon connecting data from different domains, different levels of analysis, and different actors’ points of view, generated by different research methods? How do mixed methods researchers meet the challenge of developing “good” theory, that is, theory that explains, predicts, and delights (Weick, 1995), in the face of methodological pluralism? If mixed methods research involves collecting, analyzing, and integrating different types of data or methods within a single study or program (Creswell, Plano Clark, Gutmann, & Hanson, 2003), then how do poets and quants overcome the threat of theoretical compartmentalization to achieve theoretical integration?

To date, researchers have advanced a number of different typologies that enable integration, most of which tend to center on the design of research methods; these include the careful sequencing of the order in which qualitative and quantitative methods are mixed (e.g., Creswell, 2003; Morgan, 1998; Patton, 1990; Steckler, McLeroy, Goodman, Bird, & McCormick, 1992; Tashakkori & Teddlie, 1998) as well as triangulating data sources to achieve convergence (Jick, 1979). However, as useful as these different modes of integration are, they have largely foregrounded empirical strategies of integration to the relative neglect of theoretical strategies:

The key issue is whether in a mixed methods project, the end product is more than the sum of the individual quantitative and qualitative parts. To a significant extent, this issue has been marginalized in much writing on mixed methods research. (Bryman, 2006, p. 8)

Thus, we begin to address the challenge of integration by bringing theory to the fore and expanding the strategies available to construe imaginative yet disciplined connections (Weick, 1989) across insights emerging from distinctly different methodologies. To do this, we introduce the notion of relational algorithms, a linguistic technique for linking one concept to another (Crovitz, 1970) through the use of connector words, such as conjunctions (e.g., and, but, if; Crovitz, 1970) or prepositions (e.g., through, in, after; VanGundy, 1988), and propose it can enable the development of a wider set of integration strategies in mixed methods research. We do this with the understanding that integration is a process that entails more than simply methodological “mix[ing]-and-match[ing] (with strategies liberally selected and combined)” (Morse, 2003, p. 191). Rather, we highlight how integration entails both an empirical and a theoretical dimension that need to be plausibly connected, and we explore a potentially useful tool for doing so, that of relational algorithms.

In general, theorizing involves generating an explanation of the phenomenon under study (Yin, 2014) by following “the chains of interactions, virtual and otherwise, as they evolve in time and space, to connect individuals, resources, and institutions” (Lamont & Swidler, 2014, p. 156). In mixed methods research, theoretical integration is growing in importance, in part because of the blurring of sharp distinctions between qualitative and quantitative methods and the corresponding need to find ways of illuminating how these elements relate to one another. Kaplan (2016) is instructive on this point; she notes that

the popular dichotomy—that quantitative research is deductive and qualitative is inductive or exploratory—is less useful when we consider … [that] qualitative evidence [can be] used to confirm quantitative findings, [or] to explore a phenomenon in order to develop hypotheses for quantitative testing … [or that] quantitative research can be exploratory … and qualitative research can explain why. (p. 431)

Other scholars have similarly noted the attendant problems of drawing a stark contrast between the two methods. Morgan (2007) asserts that qualitative and quantitative approaches are “broad tendencies” and turning them into “absolute, defining characteristics [of two] approaches … den[ies] the possibility of working back and forth between the two” (p. 73). More generally, this can disable integration efforts at iterating between confirming and exploring theory (Creswell et al., 2003), thus losing the variety of designs that falls in between.

Rather than compartmentalizing methods as separate and distinct, it is becoming more commonplace to link individual elements from each type in novel ways (e.g., Anand & Watson, 2004; Brickson, 2005; Rao, Davis, & Ward, 2000; Tilcsik, 2014; Wry, Lounsbury, & Jennings, 2013). For instance, researchers have collected qualitative data using traditional quantitative instruments (e.g., Brickson, 2005; Casciaro, Gino, & Kouchaki, 2014), while qualitative studies have quantified data using frequency “counts” (e.g., Anand & Watson, 2004). Even deductive, multi-method studies are drawing on elements of mixed methods designs by using, for example, qualitative narratives to ground theorizing and maintain closeness to the phenomenon (e.g., Glynn & Abzug, 2002; Navis & Glynn, 2010). Thus, with such changes in mixing methods in research design, there is a more pressing need to articulate possibilities for integrating insights born of empirical analysis into a coherent, succinct theoretical explanation.

We advance the research agenda for mixed methods designs by expanding the available strategies that intertwine theoretical and empirical integration. We view theoretical integration as a complement to, and not a substitute for, empirical integration. We propose that by expanding researchers’ repertoires for mixing and integrating methods, we increase the possibility for discovering novel approaches for understanding and theorizing the focal phenomena under study. We summarize our core argument as follows: We view quantitative and qualitative methods as tools within a broader methodological toolkit and the task of the researcher as that of construing plausible relationships across these tools. Integration thus becomes a generative, relational process aimed at building a logical explanation rather than a logistical process aimed at mixing and matching methods.

We begin by providing an overview of the elements, objectives, and processes of integration in mixed methods research, focusing especially on theoretical approaches. Next, we explore the use of relational algorithms as a specific strategy of integration, initiated by a set of “heterogeneous thought trials” (Weick, 1989, p. 522). We propose that such a strategy can foster understandings of the relationship between theory and methods and especially between qualitative and quantitative perspectives. To illustrate the generative possibilities of this strategy, we draw on relevant examples from the management and organizational literature to highlight the attendant processes and outcomes of integration. Finally, we conclude by discussing the implications of these ideas for future research.

The Role of Integration in Mixed Methods Research

Integration in mixed methods studies involves “the combination of quantitative and qualitative research within a given stage of inquiry” (Creswell et al., 2003, p. 220). And although integration is a central aspect of mixed methods designs, researchers often struggle with how to “move beyond typologies [of integration]” (Bryman, 2006, p. 97) to deepen integration and theorization. There are a number of potential obstacles that can stand in the way of effective integration.

One of these obstacles is perceptual, generated by stylized views of research that tend to cast the two types of methodologies—qualitative and quantitative—as incompatible. Treated this way, the two methods are viewed as nearly orthogonal “conceptual systems” (Rosa, Porac, Runser-Spanjol, & Saxon, 1999, p. 64) that produce incommensurable knowledge (Morgan, 2007) with few points of intersection. Viewed through such a simplified lens, a quantitative study is thought to be one that investigates hypotheses deductively, operationalizes independent and dependent variables with clear measurement scales, pursues statistical sampling, collects data through experiments or surveys, for example, and demonstrates statistical significance; theoretically, the primary aim is considered to be theory testing. By contrast, a qualitative study is thought to be one that is inductive; relies on limited, theoretical sampling; and collects textual (or sometimes visual) data using interviews, ethnographies, (non)/participant observation, case studies, archival analyses, or other modes of field research, with a primary analytical strategy of interpretation; theoretically, the primary aim is considered to be theory building. Such stark—and somewhat erroneous—conceptualizations that sharply contrast quantitative and qualitative methods make integration a heightened challenge.

A related obstacle is one that is epistemological. A common and stylized assumption that is sometimes made is that each method corresponds to a different paradigm, namely, quantitative methods are associated with a functionalist paradigm and qualitative methods with an interpretivist paradigm (Gioia & Pitre, 1990; Guba & Lincoln, 1994; Shah & Corley, 2006). Each method is perceived to have a different and incompatible epistemological stance, or way of knowing, which implicitly draws boundaries around the kinds of questions that are addressed as well as the possibilities for integration.

The stylized views that we have described, drawing sharply contrasting lines between qualitative and quantitative research methods, stand in contradiction to recent thinking that frames the distinction between qualitative (as purely inductive) and quantitative (as purely deductive) in less dramatic terms (Small, 2011). As Creswell and colleagues (2003) observe, “Many inquirers actually go back and forth between confirming and exploring in any given study” (p. 222). This suggests that an iterative commute between methods in the service of integration is not only feasible but desirable. And yet to date, we are lacking in such strategies.

Ironically, what seems to have played into this intellectual chasm is the ascendancy of qualitative methods in the management and organizational literature in the 1980s. The increasing prominence of qualitative methods made visible how different research methods relied on different vocabularies and sets of practices (Guba & Lincoln, 1994) and with it, the significance of integration and the search for tools to do so. The earliest attempts at mixed methods integration in the management and organizational literature tended to create a hierarchy of methods, subordinating one type of method to the other. For instance, in some of this work, qualitative interviews or archival narratives were treated as supplements, or add-ons, to quantitative methods like surveys or laboratory experiments; as such, the role of qualitative methods was seen as an adjunct, simply providing richer context or background but contributing little to theoretical development. This raised an important question:

In a sequential design, … if the quantitative component follows the qualitative one, is there some attempt to embellish the quantitative findings so that the qualitative element is not solely a springboard for hypotheses to be tested using a quantitative approach? (Bryman, 2006, p. 8)

The subjugation of qualitative to quantitative methods was roundly critiqued by a number of different scholars, who argued for change. Jick (1979), for instance, decried the subordination of qualitative methods to quantitative as “the parenthetical, even somewhat patronizing, use of field observations to strengthen statistical results” (p. 603). Similarly, Sutton (1997) noted that researchers who tried to integrate qualitative methods in their quantitative studies were sometimes required to “hide or downplay their qualitative data if they wish[ed] to publish the ideas inspired by such evidence” (p. 99). This turned mixed methods designs into “an exercise in testing findings against each other,” when in reality, integration “is about forging an overall or negotiated account of the findings that brings together both components of the conversation or debate” (Bryman, 2007, p. 21). The result was a less than optimal integration in the mixing of methods and lost opportunities for full integration, either empirically or theoretically. And yet, these roadblocks were not perceived as insurmountable obstacles. Morgan (2007) explains:

There is nothing about the nature of paradigms (in the sense of shared beliefs among the members of a specialty area) that inherently prevents the followers of one such paradigm from understanding the claims of another. Rather, the essential question is how effectively the proponents of the two camps can communicate with each other. (p. 62)

When bridges are effectively construed between different types of methods in a single study, new avenues for knowledge creation emerge, thus increasing the potential theoretical contribution of a study (Hurmerinta-Peltomäki & Nummela, 2006). Nonetheless, the rise of empirical approaches seems to have pushed theorization to the background when paradoxically there is the need for the fuller integration of the two; with the richness of mixed methods research, scholars can potentially explain a complex phenomenon in ways that match its complexity (Glynn, Barr, & Dacin, 2000). As Mason (2006) explains: “Mixing methods helps us to think creatively and ‘outside the box’, to theorize beyond the micro-macro divide, and to enhance and extend the logic of qualitative explanation” (p. 9). Rather than something “parenthetical” (Jick, 1979) or to be “added” (Weick, 1995), theory, we propose, is more foundational to mixed methods research, providing a touchstone for shaping how methods are integrated in the construction of a compelling chain of explanation.

In drawing greater attention to theoretical integration, we do not mean to suggest that empirically oriented strategies have obviated theory. Quite the contrary. We know, for instance, that researchers often start with a qualitative approach to aid in developing hypotheses or generating constructs that can be tested using quantitative tools or that qualitative findings can affirm (or challenge) the validity of quantitative findings, helping to unpack the mechanisms that produce observed patterns in quantitative data (for a review, see Kaplan, 2016). Rather, we acknowledge the significant contributions of both types of strategies, theoretical and empirical, but focus first on theoretical integration and then consider its interplay with empirics in the service of integration in mixed methods research.

Theoretical Integration in Mixed Methods Research

The process of developing theory requires an active dialogue between empiricism and explanation. Alvesson and Kärreman (2007) equate theory development with creating and solving a mystery, using theory as a repertoire of lenses through which to understand the data while using data to problematize theoretical ideas. For Weick (1995), good theory develops from activities like “abstracting, generalizing, relating, selecting, explaining, synthesizing, and idealizing,” which “spin out” of different forms of data or analyses to serve as “placemarkers” in the process of developing a theory (p. 389). That is, these activities interlace the different empirical elements in a study into understanding why we see the particular patterns and associations that we observe in the data.

Abstract concepts are the basic building blocks of theory; in mixed methods, the challenge of theoretical integration is particularly acute because concepts can arise from different types of data or analyses, different levels of analysis, or in different time periods. Theoretical integration is more than a process of simple addition or the stacking of various conceptual building blocks. Rather, theoretical integration involves combining separate, often distinct empirical observations in a sensemaking activity to craft a coherent, logical, plausible, or more complete explanation; ideally, that explanation should tie together different elements into a credible rationale or account of the data. The criteria for the success of theoretical integration is not better predictive power per se but better explanation or understanding of the processes that underlie the focal phenomenon (DiMaggio, 1995; Sutton & Staw, 1995; Weick, 1995).

Nonetheless, in mixed methods research, crafting a plausible and compelling explanation from the seemingly disparate empirical elements is a challenging task. Theoretical integration offers the possibility of leveraging rich and diverse empirical observations, and yet the researcher might not only be concerned with synthesizing different types of data but also with reconciling different types (or even conflicting) theoretical explanations. Following, we present and elaborate a technique for achieving integration, that of relational algorithms.

Integration at the level of theory in mixed methods designs necessitates strategies for comparing constructs and/or data generated by the different methods, a notion captured by the term commensuration. Commensuration, according to Stinchcombe, is essentially “the sociology of standardization, where convergence among participants establishes consensual standards that can then be used to assess similarities and differences in theories” (cited in Glynn & Raffaelli, 2010, p. 363). One technique that Weick (1979) suggests can aid when comparing and contrasting across different theoretical or methodological domains involves the application of relational algorithms, a process initially described by Crovitz (1970) as a way of producing “novel relations between pairs of things.” As a commensuration technique, a relational algorithm forces a researcher to explore a wider range of connections between different domains, which in turn can function as a rich touchstone for the theoretical integration of data and analyses in mixed methods designs.

As many scholars have pointed out, identifying relationships between conceptual domains is a foundational aspect of theory development (e.g., Boxenbaum & Rouleau, 2011; Tsang & Ellsaesser, 2011; Weick, 1989; Whetten, Felin, & King, 2009). For Cornelissen and Durand (2014), for instance, the key forms of reasoning through which researchers develop constructs and hypotheses include analogies and counterfactuals. Both of these concepts refer to a relational activity through which meaning is conveyed by connecting two different domains. It is in discovering and delineating novel relations that a researcher is able to find the sweet spot between “ungrounded theory … [and] atheoretical data” (Weick, 1995, p. 388).

To date, however, mixed methods researchers have emphasized the use of empirically oriented strategies of integration that include ordering the sequence of methods or converting data from one method (e.g., qualitative) into data compatible with the other method (e.g., numerical codes that can be quantitatively analyzed; Tashakkori & Teddlie, 2003). Efforts at commensuration have been focused on making different types of data amiable to each other. The process is extensively documented in the literature. To wit:

After collecting both forms of data, the analysis process might begin by transforming the qualitative data into numerical scores (e.g., themes or codes are counted for frequencies) so that they can be compared with quantitative scores. In another study, the analysis might proceed separately for both quantitative and qualitative data, and then the information might be compared in the interpretation (or discussion) stage of the research. (Creswell et al., 2003, p. 22)

As important as these activities are, focusing on commensuration issues that attend to the empirical diversity to the neglect of theoretical diversity is problematic because one does not necessarily follow from the other. Integration in mixed methods research necessitates strategies for making not only data commensurable but constructs commensurable as well; moreover, these may be distinct strategies. This may involve finding ways to connect pluralistic or even compromising competing lenses, frameworks, and interpretations, building on the heterogeneity afforded by different methods (Glynn et al., 2000). Although the desirability of theoretical pluralism has been debated (see e.g., Pfeffer, 1993; Van Maanen, 1995), it is nonetheless an essential element in theoretical integration in mixed methods research (Glynn & Raffaelli, 2010).

By enabling researchers to map conceptual relationships across insights generated by different methods, relational algorithms can weave together both theory and empirics. In a sense, this technique enables heterogeneous thought experiments, a theorizing strategy that Weick (1989) advocates, which forces the researcher to explore less obvious connections among concepts, theories, and data. Folger and Turillo (1999) also emphasize the usefulness of thought experiments, especially as ways of “zero[ing] in on problematic assumptions and help[ing] theorists to construct imaginary worlds to draw out implications of new assumptions” (p. 745). In essence, different and more specific relational forms better equip researchers to produce what Sutton and Staw (1995) describe as “strong theory,” one that “burrows deeply into microprocesses, laterally into neighboring concepts, or in an upward direction, ties itself to broader social phenomena” (p. 378).

Thus, the advantage of the relational algorithm technique is that in describing novel relations, it introduces new, less obvious, or overlooked solutions. For this reason, it has been used in a variety of fields. For instance, relational algorithms have been used as a way to creatively solve management problems. Carlopio and Andrewartha (2012) offer an illustration that explains how a typical organizational problem (e.g., customer dissatisfaction) might be resolved through creative connections between two broad domains—“customers” and “service.” Table 1 summarizes their example of finding various ways of delivering effective customer service. All of the potential solutions constitute a thought experiment and result from choosing a relational word to connect the two domains. For instance, the relational algorithm “Customers among service” suggests that one way of solving customer dissatisfaction is having customers interact with service personnel. A different relational algorithm, “Customers as service,” suggests relying on customers to deliver service to other customers. Both relational forms lay out linkages of interest, and their generative interpretation may yield underexplored solutions.

Table 1.

Example of the Use of a Relational Algorithm to Solve a Managerial Problem.

Managerial problem: Our customers (C) are dissatisfied with our service (S)
Question: How might the organization connect customers to service to forge new solutions?
Basic unit of relational algorithm: Customers (C) [ relate to ] service (S)
Thought experiments	Potential solution
C [ and ] S	Customers and service personnel work collaboratively to outline standards of service
C [ as] S	Customers deliver service to other customers
C [ for ] S	Customers’ insights collected through focus groups provide the basis for service improvements
S [ when ] C	Service personnel provide timely service when customers want it

Source. Carlopio and Andrewartha (2012, p. 208).

The basic form of a relational algorithm, explains Weick (1979), is the sentence: “Take one thing [relate it to] another thing” (p. 253). Inserting different relational words in the brackets generates a variety of potential connections between the two domains. A set of 42 relational words, including, for example, and, after, before, as, through, with, and so on, was first suggested by Crovitz (1970). Subsequently, VanGundy (1988) expanded the list to include 19 additional relational words such as: above, beyond, into, toward, within, and so on.

Our core idea is that we can broaden our repertoire of strategies for integration by using the technique of relational algorithms to explore potential conceptual linkages across quantitative and qualitative methods in studies that mix these two types. Such linkages across types of data and/or analyses may reveal new ways of making theoretical concepts and propositions commensurable. For clarity and parsimony, to refer to qualitative and quantitative methods, respectively, we use the abbreviated labels Qual and Quant in our examples of relational algorithms, and we draw attention to the connector words by inserting them in brackets.

The benefit of using the relational algorithm technique is that it can enlarge scholars’ conceptual thinking. The predominance of a limited number of relational words to connect qualitative and quantitative methods can lead to a “relational blind spot” (Weick, 1979, p. 253) that stands in the way of novel integration. A researcher who considers novel relational forms, like Qual [opposite/against] Quant, can drive integration by analyzing theoretical counterfactuals, paradoxes, and tensions. Gardner (2012), for instance, uses a quantitative approach to establish a theoretical paradox—that more pressure does not always lead to better team performance—and a qualitative approach to show the behavioral mechanisms that produce process losses in teams and hence worsen performance.

Next, we build on the technique of relational algorithms in order to develop insights in mixed methods research. We first explore the linkages that have predominated in management and organization studies and then suggest how it might be applied to generate novel combinations of theory and empirics using quantitative and qualitative elements. By way of illustration, we use a subset of connector words to develop five different strategies for integration in mixed methods research.

Advancing Five Strategies for Theoretical and Empirical Integration

To illustrate some of the possible applications of relational algorithms in theoretical integration, we offer a variety of examples from the management and organizational literature. Our goal is simply to illustrate the technique in use, not to conduct a comprehensive review of all studies using mixed methods research (for reviews of this nature, see Bryman, 2006; Molina-Azorin, 2011); moreover, we do not mean to suggest that we are singling out the best possible exemplars. Rather, we sought to offer a variety of perspectives by discussing a broad spectrum of studies.

Because our focus is on the management and organization literature, we looked for potential examples in two leading and representative empirical journals in the field: Academy of Management Journal (AMJ) and Administrative Science Quarterly (ASQ). Both are widely regarded as top journals, known for their broad theoretical reach (for a description and rationale, see Glynn & Raffaelli, 2010), and represent different varieties of management scholarship.

Drawing on these examples and applying the technique of relational algorithms, we identify five strategies of integration and present them in Table 2. The first column of the table displays the five strategies. The first two of these—Conjoined Integration and Sequential Integration—have been widely used in mixed methods research in management and for the most part focus on the empirics; the following three—Simultaneous Integration, Full-Cycle Integration, and Mono-Logic Integration—are strategies that we propose as useful extensions because they consider how empirics and theory come together to construct complex explanations. In the table, each strategy of theoretical integration is paired with the particular relational algorithm from which it was generated. The final column in the table illustrates integration in use, with studies from the literature.

Table 2.

Integration via Thought Experiments With Relational Algorithms.

Examples From the Management Literature
Types of Integration	Relational Algorithm	Examples
Conjoined Integration Minimal, less specific theoretical integration; empirical commensuration as focus	Qual [ and ] Quant	Brass (1981). Three phases of research: (1) observation and interviews, (2) questionnaires, (3) additional observation and interviews. Qualitative compensates for some limitations of quantitative, particularly those regarding an organizational network structure. Theoretical integration not specified.
Sequential Integration Theoretical compartmentalization can parallel methodological compartmentalization	Qual [ before ] Quant	Elsbach (1994). Two qualitative studies of the California cattle industry explain how organizational accounts are constructed following legitimacy-threatening events; a lab experiment investigates what elements affect the effectiveness of those accounts.
		Gioia and Thomas (1996). Qualitative case study develops a grounded model of issue interpretation during strategic change; followed by survey to test propositions.
		Grant, Dutton, and Rosso (2008). Interviews identify two mediating mechanisms between employee support programs and employees’ level of commitment; followed by a survey that “provisionally tests” the hypotheses and “lends further grounded support.”
	Qual [ after ] Quant	Sutton and Rafaeli (1988). Quant study first yielded unexpected results, a theoretical counterfactual; qual study explained why the relationship was the reverse of what had been hypothesized.
Simultaneous Integration Continual iteration between theory and empirics in an ongoing, dynamic analysis	Qual [ with/along ] Quant	Brickson (2005). Qualitative is embedded in the survey instrument, which includes open-ended questions (“general perceptions of the organization’s identity”) and scales (identity orientation). Simultaneously, the author introduces the construct of identity orientation and tests its properties and predictors.
		Obstfeldt (2005). Survey and ethnographic data mixed in a narrative that explains the relation among social networks, innovation, and strategic behavioral orientation.
	Qual [ near ] Quant	Jehn (1995). Survey, archival, and evaluation data test hypotheses that relate conflict and performance. Interviews and observations reveal “hidden issues” about intraorganizational conflict.
Full-Cycle Integration Punctuated iterations between theory and empirics mapped over time to establish causal relationships	Qual [ toward ] Quant	Glynn and Abzug (2002). First, analytical narrative of organizational name changes over time pairs quantitative analysis of 1,600 name changes to show how institutional conformity shapes organizational identities, which are then investigated with a survey to assess conferral of legitimacy.
		McDermott, Corredoira, and Kruse (2009). Ethnographic comparative analysis shows how structural embeddedness influences access to knowledge resources. Survey tests the relationship between a firm’s network and its product upgrading.
		Navis and Glynn (2010). Qualitative narrative to understand the emergence and legitimation of a market category. Statistical analyses using data from the industry offer convergent evidence of findings.
	Qual [ beyond ] Quant	Edmondson (1999). Quant and qual data (from surveys and a structured interview instrument) test hypotheses about team shared beliefs, learning behaviors, and performance. Qual investigation (interviews and observations) of seven teams helps to “better understand the relationship between team psychological safety and learning behavior.”
		Vergne (2012). Quantitative approach tests hypotheses about stigmatized categories and firm disapproval in the context of the global arms industry. Qualitative insights from interviews with industry stakeholders help to interpret the econometric results and shed light on the unforeseen effects of stigmatization, namely, that it can turn “secrecy and discreetness into a firm’s assets.”
		Tilcsik (2014). Quant study tests hypotheses about imprinting, and qual insights (based on interviews) explain what the construct “imprint-environment fit” looks like in practice.
	Quant [ beyond ] Qual	O’Mahony and Ferraro (2007). Ethnographic and archival data explore the governance system of an open-source community. Statistical analysis predicts behaviors associated with the community’s conceptualization of leadership.
Mono-Logic Integration Single mode of reasoning (i.e., inductive or deductive), different methods	Qual [ as ] Quant (deductive)	Smith, Mitchell, and Summer (1985). Field study of top-level managers and four laboratory simulations carried out concurrently to test hypotheses about the relation between priorities and life-cycle stages.
	Qual [ within ] Quant (deductive)	Rao, Davis, and Ward (2000). Deductive study of social identity and mobility using data from organizations that migrated from the NASDAQ stock market to the New York Stock Exchange. Media accounts and interviews explain the role of social identity in organizational responses.
	Quant [ as ] Qual (inductive)	Sonenshein, DeCelles, and Dutton (2014). Qualitative inductive study builds theory about self-evaluations related to the support of an issue; followed by quantitative inductive study that “empirically and theoretically deepens [the] core premise.”
	Quant [ within ] Qual (inductive)	Anand and Watson (2004). Triangulation of archival sources with interviews shows how award ceremony rituals influence the evolution of a field. Where possible, the authors “counted the countable” to “underscore the significance of [the] findings.”

Conjoined and Sequential Integration

We start by observing that to date, researchers have invoked a rather limited set of relational words in the service of integration. The conjunction and represents the most basic one in mixing methods, as in Qual [and] Quant methods, one that we label Conjoined Integration (see Table 2). As in the earlier customer-service example (see Table 1), this relational algorithm is the most common and general form of mixed methods research; it suggests treating qualitative and quantitative approaches as separate domains with the researcher trying to make them work collaboratively. As a thought experiment, it suggests a connection but offers little more about how the two approaches might be linked and how different theoretical insights might be used in the process of constructing a fuller explanation. For instance, Brass’s (1981) study of structural relationships uses a qualitative approach to compensate for limitations of the quantitative data collection instrument. The focus is on empirical rather than theoretical commensuration. The potential advantage of Conjoined Integration is that it is a less specific form of theoretical integration and thus can function as an “umbrella” term that can encompass almost any kind of mixed methods research. The potential limitation, however, is that it demands less theoretical integration by design and simply points to potential collaboration between the different methods.

Other common relational algorithms in mixed methods designs rely on the connectors before and after. These refer to the sequence or order in which the different methodologies are conducted and analyzed. In the form of a relational algorithm, the design is Qual [before/after] Quant, which we label as Sequential Integration (see Table 2). Here, the empirical methods are conducted sequentially, each bounded, compartmentalized, and associated with a single form of theorizing—qualitative with theory building, quantitative with theory testing. Theorization unfolds in a logical progression, from theory building to testing or vice versa.

In their studies of sensemaking during strategic change, Gioia and Thomas (1996) use a Qual [before] Quant strategy. In the first stage, they use the qualitative method of case study to generate a grounded theoretical model of top management’s perceptions of the organization’s identity and management’s interpretation of strategic issues. In the second stage, they test the propositions generated qualitatively using survey data. Similarly, Elsbach (1994) and Grant, Dutton, and Rosso (2008) use a Qual [before] Quant approach to first explain qualitatively how a phenomenon unfolds and then test a set of propositions statistically to provide further support.

Reversing the order, namely, placing Qual [after] Quant, is a variant of Sequential Integration. This strategy reverses the stages of theoretical integration, with theory testing preceding theory building, an approach more consistent with theory elaboration (Vaughan, 1992). For example, Sutton and Rafaeli (1988) use a Qual [after] Quant approach to study emotions in the workplace. The initial quantitative study, using data from a sample of 576 convenience stores, yielded unexpected findings about the role of positive emotions in sales. The second study is qualitative and uses case studies, informal interviews, and observations to explain why the relationship was the reverse of what had been hypothesized.

The potential advantage of using Sequential Integration is that this is well-mapped terrain in the literature. There are a number of typologies (e.g., Creswell et al., 2003; Morgan, 1998; Patton, 1990; Tashakkori & Teddlie, 1998) that help researchers “convey a sense of the rigor of the research and provide guidance to others about what [they] intend to do or have done” (Bryman, 2006, p. 98). In addition, the theoretical lines are clear: theory testing is differentiated from theory building, and the sequencing of theoretical explanations parallels empirical sequencing. The potential limitations center on the challenges of overcoming the theoretical and empirical divides of sequencing. Researchers thus face the risk of theoretical compartmentalization, which can be amplified by the use of distinct methods.

Although these two strategies—Conjoined Integration and Sequential Integration—are widely used in the management literature and have been important in advancing mixed methods research, researchers need not limit themselves to these two. We urge researchers to engage in Weickian style thought experiments, using different relational algorithms to identify new forms of theoretical integration beyond these two. Here, using these techniques, we suggest three elaborations: (a) Simultaneous Integration, (b) Full-Cycle Integration, and (c) Mono-Logic Integration.

Simultaneous Integration

The use of different connector words can generate new forms of theoretical integration; we focus on three—with, along, or near. Here, researchers pursue theory building and theory testing at the same time. The relational algorithm Qual [with] Quant suggests that rather than two separate and bounded approaches (as in Sequential Integration), a researcher uses them simultaneously, a strategy we label Simultaneous Integration (see Table 2). This form of integration might use the same data collection technique to capture different types of data (e.g., numbers, text, or visual images) and allow for interpretation and hypothesis testing to unfold simultaneously. For instance, in Brickson’s (2005) study of organizational identity, a qualitative approach is embedded with the quantitative data collection instrument. The survey includes open-ended questions and scales to elicit both people’s perceptions and interpretations of their organization’s identity as well as scale measurements of identity orientation. Embedding a qualitative approach within a quantitative approach enables Brickson to show that organizational identity matters because it shapes how an organization relates to its stakeholders. Theoretical integration is thus grounded in people’s narratives about the organization and theoretical insights about the role of identity orientations. Similarly, Obstfeld (2005) uses survey and ethnographic data simultaneously in a narrative that explains the relation among social networks, innovation, and strategic behavioral orientation. A different form of Simultaneous Integration could be generated by using a Qual [near] Quant approach, relying on qualitative methods to uncover nuanced aspects of the phenomenon that might remain hidden in a quantitative study (for an illustration, see Jehn, 1995).

The potential advantage of Simultaneous Integration is that iteration between theory and data occurs contemporaneously and in an ongoing manner that can be more dynamic, expedient, and interactive. The potential limitations center on the challenges of commensuration in continually mixing data and methods and converting one form of data into another (e.g., text into numerical counts).

Full-Cycle Integration

Other relational algorithms use different sets of words; we found toward and beyond to be generative. With these kinds of connector words, the process of developing theory entails a chain of theory building and theory testing activities with which to reveal causal relationships, including both antecedents and effects, in a strategy we label Full-Cycle Integration. More specifically, the relational algorithm Qual [toward] Quant points to the use of qualitative data and analysis to reveal patterns that explain the phenomenon and its antecedents, then quantitative methods to map effects.

Although this relational algorithm resembles sequencing, the difference is in its theoretical focus on explaining causality. For instance, in their study of satellite radio, Navis and Glynn (2010) investigate how new market categories emerge and are legitimated. An analytical narrative first explains the dynamics and causal antecedents of emergence and market category legitimation over time and directs the theorizing toward a set of hypotheses about the effects of legitimation, which are tested with statistical analyses. By offering a logical chain of explanation, the authors show that once a new market category is legitimated, the focus of market actors’ attention shifts dramatically. McDermott, Corredoira, and Kruse (2009) and Glynn and Abzug (2002) use similar approaches in Full-Cycle Integration, establishing a causal chain of explanation while remaining close to the phenomenon.

Other thought experiments that facilitate Full-Cycle Integration include the relational algorithms Qual [beyond] Quant or Quant [beyond] Qual. In these cases, the connector beyond suggests using one method to interpret the findings of the other method and explain what the construct or causal relationship means in practice (e.g., Edmondson, 1999; O’Mahony & Ferraro, 2007; Tilcsik, 2014; Vergne, 2012). The potential advantage of Full-Cycle Integration is that it enables “researchers to tackle more encompassing phenomena” (Chatman & Flynn, 2005, p. 434), explaining the entirety of the phenomenon from cause to effect over time. The potential limitations center on establishing causality clearly and robustly, especially with the typically rich and often abundant data generated from mixed methods research. Teasing out and bounding causes from effects is a particular challenge.

Mono-Logic Integration

In our final thought experiment using relational algorithms, we relate the two methods—qualitative and quantitative—as tools that converge on a phenomenological explanation but employ only one type of logical reasoning, namely, inductive or deductive. The relational algorithm is suggested by words like as and within, in a strategy we label Mono-Logic Integration. In a Quant [as] Qual relational form, both methods are used inductively with the goal of deepening a theoretical proposition. For instance, Sonenshein, DeCelles, and Dutton’s (2014) study of environmental issue support includes a qualitative investigation based on interviews and a quantitative investigation based on a laboratory experiment; both methods are used inductively. The qualitative approach explains how the perceived challenges of supporting an issue influence the self-evaluations of environmental supporters, and the quantitative approach shows that these self-evaluations play an important role in issue-supportive behaviors. From a statistical perspective, the authors acknowledge that the quantitative observational method raises questions of generalizability due to its “low power/sample size and restricted range on the dependent variable” (p. 29); however, it deepens the implications of self-evaluations for issue-supportive behaviors.

With Mono-Logic Integration, there is not the clear, well-bounded separation of Sequential Integration with the one-to-one association between the method and the logic of theoretical reasoning. Instead, both methods are used to develop theory of the same logical type. The potential advantages of Mono-Logic Integration are in the strength of the theoretical argument because it is buttressed by the convergence of multiple and different methods. The limitations arise in the risk of losing the richness of data collected and analyzed, making it commensurate across methods, or over-reaching the confines of the data in the service of a fuller explanation. Moreover, this is a relatively new approach in mixed methods research; as a result, there are few templates or published pieces from which to draw.

Discussion and Conclusion

The process of integration in mixed methods research is challenging as it involves drawing different empirical strands of meaning from diverse data, sources, and analytics in order to weave a coherent or unified theoretical explanation of a focal phenomenon. Earlier, we discussed how integration necessitates more than simply assembling different types of data or empirics and aggregating them together; rather, it requires stitching together a rich tapestry of explanation and in doing so discovering those theoretical connections that bridge diverse empirical insights.

In highlighting the technique of relational algorithms, we sought to go beyond current strategies of integration in mixed methods research. We were motivated in part by Sutton and Staw’s (1995) argument that “observed patterns like beta weights, factor loadings, or consistent statements by informants rarely constitute causal explanations” (p. 373). Instead, integrating these as elements of a causal explanation resides in the nature of the research and the skills of the researcher.

To enable these skills, we built on Weick’s (1989) admonition to engage in heterogeneous thought experiments and found that relational algorithms enabled this. Relational algorithms leverage how the meanings embedded in a plethora of connector words (e.g., beyond, opposite, to, or within) can suggest a wider range of possibilities for relating theoretical insights generated by mixing quantitative and qualitative modes of inquiry. Using this technique as a tool, we extended the two dominant types of theoretical integration—Conjoined Integration (connected by and) and Sequential Integration (connected by before or after)—to three additional forms: a continuously iterative Simultaneous Integration (connected by with/along or near), a causality-oriented Full-Cycle Integration (connected by toward or beyond), and a deductively or inductively focused Mono-Logic Integration (connected by as or within).

Let us consider a hypothetical example to illustrate these strategies of integration. A researcher interested in investigating the symbolic dimension of entrepreneurship (e.g., Lounsbury & Glynn, 2001; Ravasi, Rindova, & Dalpiaz, 2012) might ask, “How do new ventures infuse products with symbolic value and meanings that transcend utility to resonate with broader cultural sentiments?” and design a mixed methods study. Using relational algorithms to enable integration, the researcher might consider a Qual [toward] Quant approach in which the organization’s symbolic claims about its products could be content analyzed over time with data from the organization’s advertisements or press releases. Coding for claims that reference societal or cultural domains—like sports, music, visual arts, or entertainment—and by time periods, the researcher could generate a narrative about the organization’s cultural entrepreneurship (Lounsbury & Glynn, 2001). The initial qualitative analyses might yield a cogent narrative that moves the researcher toward a quantitative testing of the effects of such symbolic claims; for this, the researcher might examine the resonance with key audiences by conducting lab experiments or surveys.

A different relational algorithm, Quant [within] Qual, would have prompted a different path of theorizing. The direction here might be that of examining the organization’s symbolic claims for their effects on the interpretations of industry actors (e.g., competitors, analysts, or stakeholders), embedding quantitative survey scales within qualitative interviews, asking, for example, the degree to which an organizational claim resonated with broader cultural sentiments. As suggested by this hypothetical illustration and our previous discussion of different modes of integration, we have tried to demonstrate the utility of relational algorithms in mixed methods research. Simply changing the connector words that link empirics and theorization can force the researcher to engage in thought experiments that build in conceptual variance; such engagement will likely expand the repertoire of integration strategies.

In outlining these five strategies, we highlight commensuration as a core activity of mixed methods integration, both for empirical and theoretical reasons. That is, integration necessitates strategies that enable one type of data to be comparable to another (an aspect that has been acknowledged in the management and organizational literature) as well as relational strategies that enable researchers to compare, contrast, and find novel connections across theoretical constructs and between these and empirical insights. Commensuration is critical to integration because, as Espeland and Stevens (1998) point out:

The consequences of commensuration are complex and varied. Commensuration can render some aspects of life invisible or irrelevant … [it] changes the terms of what can be talked about, how we value, and how we treat what we value. It is symbolic, inherently interpretive, deeply political, and too important to be left implicit. (p. 314)

Indeed, in identifying, selecting, and specifying how quantitative and qualitative elements in mixed methods designs relate to each other, researchers can achieve a number of different things: They can bring certain aspects of a phenomenon to the fore while relegating others to the background, they can change the direction of a conversation by illuminating underexplored dimensions of a phenomenon, and ultimately, they can articulate implications for practice that might impact how organizations function day in and day out. We have presented five different possibilities of integration (see Table 2), each of which addresses two of the key challenges of integration in mixed methods research: (a) that the relational linkages across elements constitute the scaffolding on which scholars can build more coherent and fuller accounts of organizational behavior and (b) that the wider the variety of connector words in use, the more likely that poets and quants will be able to effectively craft plausible and nuanced explanations.

Our aim in this article is to advance the tools that enable mixed methods integration and especially to bring into view the role of theoretical integration as a companion to empirical integration. We presented the relational algorithm technique as one of many potential paths to generating less conventional but insightful connections across methods. Certainly, other techniques are possible. Mason (2006), for instance, proposes a different technique, which she describes as “constructing ‘multi-nodal’ dialogic explanations” (p. 20). This technique entails the development of explanations along multiple dimensions of social experience, maintaining “the distinctiveness of different methods” and evidencing the “creative tension” that ensues from this process. More broadly, Bryman (2007) suggests that researchers should avoid “los[ing] sight of the rationale for conducting mixed methods research in the first place” (p. 20), so as to make the most of the potential linkages across elements. Next, we extend our analyses to offer implications for theoretical integration in mixed methods research.

Implications for Mixed Methods Research

Mixed methods designs have evidenced their effectiveness for exploring a multitude of phenomena with different types of studies: when the research question seeks to bridge new and extant theory (Edmondson & McManus, 2007), when historical or temporal dimensions are a critical aspect of the phenomenon (e.g., Navis & Glynn, 2010), when context-specific dynamics call for an in-depth exploration to understand not only what is going on but also why it is going on (e.g., Gardner, 2012), and when understanding individuals’ experience requires mapping broader cultural processes that involve higher level constructs (e.g., Wagner & Gooding, 1987). Organization theory has recently shifted from an emphasis on theory-driven research to that of problem-driven research that attempts to explain complex, significant issues in the world (Davis & Marquis, 2005); the result is to encourage more mixed methods designs because “the benefits of combining qualitative and quantitative methods to form a more complete picture of a phenomenon far outweigh the costs of time and effort” (Shah & Corley, 2006, p. 1832).

This is not to say that mixed methods designs always produce better explanations. As Mason (2006) states: “The value of such approaches must be judged in relation to their theoretical logic, and the kinds of questions about the social world they enable us to ask and answer” (p. 10). Moreover, the methods chosen for a particular study, whether qualitative, quantitative, or mixed, are not an end in themselves (Jick, 1979); rather, the challenge of integration should direct researchers to considering the interrelationship between theorization and empirics. Our argument recognizes that in a dynamic, interconnected, and diverse world, researchers need to be “attentive to a wide range of enabling and constraining dimensions emanating from the spatial, material, and semiotic realms” (Lamont & Swidler, 2014, p. 157), and thus, an expanded toolbox for strategies of integration, such as that which we have suggested, is needed. This necessitates a capacity to work with a wide variety of data collection and analytic tools, for which the mixed methods researcher is potentially well equipped.

Still, a valid question is whether integration leads to stronger mixed methods research—research that offers “detailed and compelling arguments” for developing or testing theories (Sutton & Staw, 1995, p. 373). Our argument and to some extent others’ suggestions to achieve more integrative designs point to the construal of theoretical explanations as a key element of the mixed methods research process. We echo Jick’s (1979) call that quantitative and qualitative methods should become more than “mere window dressing for the other;” ignoring this puts the researcher at risk of studies with an “inadequate or biased” design (p. 609). More generally, we suggest that neither theoretical integration nor empirical integration alone produces strong mixed methods research. Instead, we propose that integration across both elements is most effective. The alternative—a typology of mixing empirical methods—falls short of integration across theory and data, and correspondingly, focusing singularly on theoretical integration—connecting only concepts without empirical grounding—says little about how to collect, analyze, and organize a variety of data to support the argument in a coherent fashion. Both activities—structuring the data and mapping potential conceptual linkages—are crucial in mixed methods research because researchers need disciplining mechanisms when juggling multiple types of data with multiple thought experiments. Only then are we as academics able to discern connections that shed light on the observed data patterns.

To conclude, we have advanced a conceptualization of integration as a generative, relational process aimed at crafting a logical, credible explanation by construing novel associations across quantitative and qualitative elements, woven together through theorization. While quantitative and qualitative tools can together reveal deeper understandings of a phenomenon of interest, theorization can function as an integrating mechanism that affords poets and quants the means to tie empirical observations together. In essence, if the promise of mixed methods research is to deliver more than the sum of its parts (Bryman, 2007), then integration cannot be achieved by solely considering the complementarity or sequencing of data collection or analyses. As a result, “mixed methods” designs may be more accurately described as “integrated methods” designs.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We are grateful for the generous financial support of the Boston College Winston Center for Leadership and Ethics and the Joseph F. Cotter Professorship.

References

Alvesson

Kärreman

(2007). Constructing mystery: Empirical matters in theory development. Academy of Management Review, 32(4), 1265–1281.

Anand

Watson

M. R.

(2004). Tournament rituals in the evolution of fields: The case of the Grammy Awards. Academy of Management Journal, 47(1), 59–80.

Boxenbaum

Rouleau

(2011). New knowledge products as bricolage: Metaphors and scripts in organizational theory. Academy of Management Review, 36(2), 272–296.

Brass

D. J.

(1981). Structural relationships, job characteristics, and worker satisfaction and performance. Administrative Science Quarterly, 26(3), 331–348.

Brickson

S. L.

(2005). Organizational identity orientation: Forging a link between organizational identity and organizations’ relations with stakeholders. Administrative Science Quarterly, 50(4), 576–609.

Bryman

(2006). Integrating quantitative and qualitative research: How is it done? Qualitative Research, 6(1), 97–113.

Bryman

(2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1(1), 8–22.

Carlopio

Andrewartha

(2012). Developing management skills: A comprehensive guide for leaders. New South Wales: Pearson Australia.

Casciaro

Gino

Kouchaki

(2014). The contaminating effects of building instrumental ties how networking can make us feel dirty. Administrative Science Quarterly, 59(4), 705–735.

10.

Chatman

J. A.

Flynn

F. J.

(2005). Full-cycle micro-organizational behavior research. Organization Science, 16(4), 434–447.

11.

Cornelissen

J. P.

Durand

(2014). Moving forward: Developing theoretical contributions in management studies. Journal of Management Studies, 51(6), 995–1022.

12.

Creswell

J. W.

(2003). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage.

13.

Creswell

J. W.

Plano Clark

V. L.

Gutmann

Hanson

(2003). Advanced mixed methods research designs. In Tashakkori

Teddlie

(Eds.), Handbook on mixed methods in the behavioral and social sciences (pp. 209–240). Thousand Oaks, CA: Sage.

14.

Crovitz

H. F.

(1970). Galton’s walk: Methods for the analysis of thinking, intelligence, and creativity. New York, NY: Harper & Row.

15.

Davis

G. F.

Marquis

(2005). Prospects for organization theory in the early twenty-first century: Institutional fields and mechanisms. Organization Science, 16(4), 332–343.

16.

DiMaggio

P. J.

(1995). Comments on “what theory is not.” Administrative Science Quarterly, 40(3), 391–397.

17.

Edmondson

A. C.

(1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44(2), 350–383.

18.

Edmondson

A. C.

McManus

S. E.

(2007). Methodological fit in management field research. Academy of Management Review, 32(4), 1155–1179.

19.

Elsbach

K. D.

(1994). Managing organizational legitimacy in the California cattle industry: The construction and effectiveness of verbal accounts. Administrative Science Quarterly, 39(1), 57–88.

20.

Espeland

W. N.

Stevens

M. L.

(1998). Commensuration as a social process. Annual Review of Sociology, 24, 313–343.

21.

Folger

Turillo

C. J.

(1999). Theorizing as the thickness of thin abstraction. Academy of Management Review, 24(4), 742–758.

22.

Gardner

H. K.

(2012). Performance pressure as a double-edged sword enhancing team motivation but undermining the use of team knowledge. Administrative Science Quarterly, 57(1), 1–46.

23.

Gioia

D. A.

Pitre

(1990). Multiparadigm perspectives on theory building. Academy of Management Review, 15(4), 584–602.

24.

Gioia

D. A.

Thomas

J. B.

(1996). Identity, image, and issue interpretation: Sensemaking during strategic change in academia. Administrative Science Quarterly, 41(3), 370–403.

25.

Glynn

M. A.

Abzug

(2002). Institutionalizing Identity: Symbolic isomorphism and organizational names. Academy of Management Journal, 45(1), 267–280.

26.

Glynn

M. A.

Barr

P. S.

Dacin

M. T.

(2000). Pluralism and the problem of variety. Academy of Management Review, 25(4), 726–734.

27.

Glynn

M. A.

Raffaelli

(2010). Uncovering mechanisms of theory development in an academic field: Lessons from leadership research. Academy of Management Annals, 4(1), 359–401.

28.

Grant

A. M.

Dutton

J. E.

Rosso

B. D.

(2008). Giving commitment: Employee support programs and the prosocial sensemaking process. Academy of Management Journal, 51(5), 898–918.

29.

Guba

E. G.

Lincoln

Y. S.

(1994). Competing paradigms in qualitative research. In Denzin

N. K.

Lincoln

Y. S.

(Eds.), Handbook of qualitative research (pp. 105–117). Thousand Oaks, CA: Sage.

30.

Hurmerinta-Peltomäki

Nummela

(2006). Mixed methods in international business research: A value-added perspective. Management International Review, 46(4), 439–459.

31.

Jehn

K. A.

(1995). A multimethod examination of the benefits and detriments of intragroup conflict. Administrative Science Quarterly, 40(2), 256–282.

32.

Jick

T. D.

(1979). Mixing qualitative and quantitative methods: Triangulation in action. Administrative Science Quarterly, 24(4), 602–611.

33.

Kaplan

(2016). Mixing quantitative and qualitative research. In Elsbach

K. D.

Kramer

(Eds.), Handbook of qualitative organizational research: Innovative pathways and methods (pp. 423–433). New York, NY: Routledge.

34.

Lamont

Swidler

(2014). Methodological pluralism and the possibilities and limits of interviewing. Qualitative Sociology, 37(2), 153–171.

35.

Lounsbury

Glynn

M. A.

(2001). Cultural entrepreneurship: Stories, legitimacy, and the acquisition of resources. Strategic Management Journal, 22(6-7), 545–564.

36.

Mason

(2006). Mixing methods in a qualitatively driven way. Qualitative Research, 6(1), 9–25.

37.

McDermott

G. A.

Corredoira

R. A.

Kruse

(2009). Public-private institutions as catalysts of upgrading in emerging market societies. Academy of Management Journal, 52(6), 1270–1296.

38.

Molina-Azorin

J. F.

(2011). The use and added value of mixed methods in management research. Journal of Mixed Methods Research, 5(1), 7–24.

39.

Morgan

D. L.

(1998). Practical strategies for combining qualitative and quantitative methods: Applications to health research. Qualitative Health Research, 8(3), 362–376.

40.

Morgan

D. L.

(2007). Paradigms lost and pragmatism regained methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48–76.

41.

Morse

J. M.

(2003). Principles of mixed methods and multimethod research design. In Tashakkori

Teddlie

(Eds.), Handbook of mixed methods in social and behavioral research (pp. 189–208). Thousand Oaks, CA: Sage.

42.

Navis

Glynn

M. A.

(2010). How new market categories emerge: Temporal dynamics of legitimacy, identity, and entrepreneurship in satellite radio, 1990-2005. Administrative Science Quarterly, 55(3), 439–471.

43.

O’Mahony

Ferraro

(2007). The emergence of governance in an open source community. Academy of Management Journal, 50(5), 1079–1106.

44.

Obstfeld

(2005). Social networks, the tertius iungens orientation, and involvement in innovation. Administrative Science Quarterly, 50(1), 100–130.

45.

Patton

M. Q.

(1990). Qualitative evaluation and research methods. Thousand Oaks, CA: Sage.

46.

Pfeffer

(1993). Barriers to the advance of organizational science: Paradigm development as a dependent variable. Academy of Management Review, 18(4), 599–620.

47.

Rao

Davis

G. F.

Ward

(2000). Embeddedness, social identity and mobility: Why firms leave the NASDAQ and join the New York Stock Exchange. Administrative Science Quarterly, 45(2), 268–292.

48.

Ravasi

Rindova

Dalpiaz

(2012). The cultural side of value creation. Strategic Organization, 10(3), 231–239.

49.

Rosa

J. A.

Porac

J. F.

Runser-Spanjol

Saxon

M. S.

(1999). Sociocognitive dynamics in a product market. The Journal of Marketing, 63, 64–77.

50.

Shah

S. K.

Corley

K. G.

(2006). Building better theory by bridging the quantitative-qualitative divide. Journal of Management Studies, 43(8), 1821–1835.

51.

Small

M. L.

(2011). How to conduct a mixed methods study: recent trends in a rapidly growing literature. Annual Review of Sociology, 37(1), 57–86.

52.

Smith

K. G.

Mitchell

T. R.

Summer

C. E.

(1985). Top level management priorities in different stages of the organizational life cycle. Academy of Management Journal, 28(4), 799–820.

53.

Sonenshein

DeCelles

K. A.

Dutton

J. E.

(2014). It’s not easy being green: The role of self-evaluations in explaining support of environmental issues. Academy of Management Journal, 57(1), 7–37.

54.

Steckler

McLeroy

K. R.

Goodman

R. M.

Bird

S. T.

McCormick

(1992). Toward integrating qualitative and quantitative methods: an introduction. Health Education Quarterly, 19(1), 1–8.

55.

Sutton

R. I.

(1997). The virtues of closet qualitative research. Organization Science, 8(1), 97–106.

56.

Sutton

R. I.

Rafaeli

(1988). Untangling the relationship between displayed emotions and organizational sales: The case of convenience stores. Academy of Management Journal, 31(3), 461–487.

57.

Sutton

R. I.

Staw

B. M.

(1995). What theory is not. Administrative Science Quarterly, 40(3), 371–384.

58.

Tashakkori

Teddlie

(1998). Introduction to mixed method and mixed model studies in the social and behavioral sciences. In Tashakkori

Teddlie

(Eds.), Mixed methodology: Combining qualitative and quantitative approaches (pp. 1–40). Thousand Oaks, CA: Sage.

59.

Tashakkori

Teddlie

(2003). The past and future of mixed methods research: From data triangulation to mixed model designs. In Tashakkori

Teddlie

(Eds.), Handbook of mixed methods in social and behavioral research (pp. 671–701). Thousand Oaks, CA: Sage.

60.

Thornberry

T. P.

(1989). Reflections on the advantages and disadvantages of theoretical integration. In Messner

S. F.

Krohn

M. D.

Liska

A. E.

(Eds.), Theoretical Integration in the study of deviance and crime: problems and prospects (pp. 51–60). New York, NY: Suny Press.

61.

Tilcsik

(2014). Imprint-environment fit and performance: How organizational munificence at the time of hire affects subsequent job performance. Administrative Science Quarterly, 59(4), 639–668.

62.

Tsang

E. W.

Ellsaesser

(2011). How contrastive explanation facilitates theory building. Academy of Management Review, 36(2), 404–419.

63.

Van Maanen

(1995). Style as theory. Organization Science, 6(1), 133–143.

64.

VanGundy

A. B.

(1988). Techniques of structured problem solving. Englewood Cliffs, NJ: John Wiley & Sons.

65.

Vaughan

(1992). Theory elaboration: The heuristics of case elaboration. In Ragin

C. C.

Becker

H. S.

(Eds.), What is a case? Exploring the foundations of social inquiry (pp. 173–242). Cambridge, UK: Cambridge University Press.

66.

Vergne

J. P.

(2012). Stigmatized categories and public disapproval of organizations: A mixed methods study of the global arms industry (1996-2007). Academy of Management Journal, 55(5), 1027–1052.

67.

Wagner

J. A.

III Gooding

R. Z.

(1987). Effects of societal trends on participation research. Administrative Science Quarterly, 32(2), 241–262.

68.

Weick

K. E.

(1979). The social psychology of organizing (2nd ed.). New York, NY: McGraw Hill.

69.

Weick

K. E.

(1989). Theory construction as disciplined imagination. Academy of Management Review, 14(4), 516–531.

70.

Weick

K. E.

(1995). What theory is not, theorizing is. Administrative Science Quarterly, 40(3), 385–390.

71.

Whetten

D. A.

Felin

King

B. G.

(2009). The practice of theory borrowing in organizational studies: Current issues and future directions. Journal of Management, 35(3), 537–563.

72.

Wry

Lounsbury

Jennings

P. D.

(2013). Hybrid vigor: Securing venture capital by spanning categories in nanotechnology. Academy of Management Journal, 57(5), 1309–1333.

73.

Yin

R. K.

(2014). Case study research: Design and methods (5th ed.). Thousand Oaks, CA: Sage.