Abstract
Application of objectivist methodological assumptions and overreliance on mathematical analysis can cause researchers to oversimplify reality and thereby generate rigorously derived theories and recommendations that lack practitioner relevance. Although mixed-methods approaches to management research have long been heralded, the details remain sparse about how to reconcile apparently disparate approaches. By reflecting on lessons learned over some 15 years of practical experience with a robust mixed-methods approach specifically designed to bridge the academia–practitioner gap, this article demonstrates how inclusion of an interpretive field perspective yields a much more comprehensive picture of the relationship between the organization and its contextual environment. Thus, a key purpose of this article is to stimulate researchers into adopting a more balanced portfolio of research methods that will simultaneously achieve research rigor and generate insightful practitioner-relevant theory.
Introduction
Mixed-methods approaches to management research have long been advocated in response to a belief that using objectivist methodological assumptions and a reliance on objective/quantitative methods such as surveys and mathematical/statistical analysis gives rise to significant discrepancies between theory generated and the “real-world” phenomenon on which the theory is based (Beer, 2001; Scandura & Williams, 2000; Towill, 1999a; Tranfield & Starkey, 1998; Westwood & Clegg, 2003). In short, researchers are criticized for not taking advantage of processes of inquiry that simultaneously achieve research rigor and insightful practitioner relevance (Aram & Salipante, 2003; Conger, 1998; Lalle, 2003; Scandura & Williams, 2000). Perhaps in part because the needed details are absent about how to combine qualitative and quantitative approaches, the literature strongly exhibits either strongly objectivist or strongly positivist philosophical views about the nature of phenomena and what constitutes valid discipline knowledge about those phenomena. By reflecting on some 15 years experience with a well-established mixed-methods approach within the supply chain management discipline, a key purpose of this article is to stimulate researchers into adopting a more balanced portfolio of research methods that will simultaneously achieve research rigor and generate insightful practitioner-relevant theory.
The next section explores the conceptual landscape within which management research, and in particular supply chain management research, is performed. It provides evidence of a dominant research philosophy and tradition in both the management and supply chain management fields and discusses why this is limiting our understanding. This is followed by description and discussion of a well-established mixed-methods approach that was developed and validated for use in the field. Practical advice is offered for sense-making with qualitative data, and the value of the approach for generating original contributions to knowledge is described. Finally, its transferability to the wider management research community is discussed through application of the method to small and medium sized enterprises.
Management Research Context
Over the past 30 years there has been growing concern and debate about the adequacy of research methods in the social sciences. In particular, methods derived from the natural sciences have dominated the field (Azorin & Cameron, 2010; Gummesson, 2006; Morgan & Smircich, 1980). According to Gummesson (2006), quantitative research is appointed superior to qualitative research, and traditional natural sciences are hailed by social scientists as the role model for rigorous and objective research. Although the rigor, objectivity, and relevance of these traditional methods has been questioned by many academics (e.g., Beer, 2001; Scandura & Williams, 2000; Towill, 1999a; Tranfield & Starkey, 1998; Westwood & Clegg, 2003), Gummesson questions their appropriateness for the management discipline altogether and argues that quantitative methods fall short in handling management complexity, fail to place variables into context, and do not incorporate human and social properties. However, the authors of the present study do not favor this strong assertion and propose a combined/mixed-methods approach.
Figure 1 is based on Azorin and Cameron’s (2010) literature review, which focused on the research methodology employed by researchers contributing to two respected journals: the Strategic Management Journal and the Journal of Organizational Behavior (2003-2009). To these, and covering the same time period, has been added the Journal of Leadership & Organizational Studies. The very strong dominance of quantitative approaches in empirical research over the qualitative and mixed (quantitative and qualitative combined) approaches is clear.

Frequency of research approaches in selected journals
According to Näslund (2002), oversimplifying complex “real-world” situations with quantitative analysis might explain why the benefits of research outputs to practitioners often appear to be slight, at least when it comes to managerial/organizational research. Beer (2001) similarly questions the validity and the ability to implement much of the knowledge produced by management scholars and raises the point that most academics have appropriated the model of natural science in ways that have maintained the separation of science in practice.
If normal scientific approaches are inadequate for creating implementable knowledge, what are the alternatives? How might the relative merits of quantitative and qualitative science approaches be integrated to achieve both rigor and relevance? To help answer these questions a mixed-methods research approach that bridges the academia–practitioner gap is described in the next section. This has been deployed for more than 15 years and is anchored within the supply chain management discipline. Arguably supply chain management typifies one of the most significant changes in the paradigm of modern business management; that individual businesses no longer compete as solely autonomous entities but rather as supply chains (Christopher, 1998; Lambert, Cooper, & Pagh, 1998). The supply chain management term was only first coined in the 1980s by Oliver and Webber (1982); thus, its concepts remain poorly understood and there are calls for clearer definitions and meaningful conceptual frameworks (Cooper, Lambert, & Pagh, 1997; Croom, Romano, & Giannakis, 2000; Svensson, 2002; van der Vaart & van Donk, 2008). Näslund (2002) points out that if supply chain academics want to lead, rather than lag behind, the practitioners then their research needs to generate theory that is relevant to practitioners. The best way to achieve relevance is to understand what is going on inside organizations: by being out “in the real world” gathering data first-hand to develop knowledge of logistics in action. According to Aram and Salipante (2003), the same argument holds true for the wider management and social science disciplines.
A Mixed-Methods Approach to Supply Chain Research
In the early 1990s, a procedure known as the quick scan audit methodology (QSAM) originated out of the Logistics Systems Dynamics Group at Cardiff University, to describe and explain the complexities of a “messy” European automotive supply chain environment via the application of multiple, site-centered data collection methods.
Arguably the concept of supply chain management originates from a systems perspective (Christopher, 1998), in which the idea of a system is generally expressed as encompassing interconnected components separated from their environment by a system boundary. This is also in line with Näslund’s description of logistics, who argues that modern logistics is based on holistic, systemic thinking and uses multidisciplinary and cross-functional approaches. Although QSAM continues to evolve, its underlying “systems thinking” approach has remained the same. Fawcett, Ellram, and Ogden (2007) provide the following definition of this approach:
Systems thinking is the holistic process of considering both the immediate local outcomes and the longer-term system-wide ramification of decisions. Whereas traditional functional thinking seeks the local optimum—often at the expense of the overall system’s performance—systems thinking aligns efforts; getting everyone to pull in the same direction. (pp. 74-75)
QSAM uses a structured modeling framework that can be traced back to the original concept of a manufacturing system by Parnaby (1979), which was exploited by Olsmats, Edgehill, and Towill (1988) within a U.K. industrial context. A key characteristic of the framework is that it endeavors to achieve an optimum compromise between qualitative and quantitative methods of management theory research by making maximum use of resources (primarily) in field-based activities in the search for “meaning of evidence” (Eisenhardt, 1989). In practice, it mixes qualitative and quantitative methods when seeking to triangulate information sources (Beach, Muhlemann, Price, Paterson, & Sharp, 2001; Berry, Naim, & Towill, 1995; Jick, 1979).
The QSAM researchers/developers had also realized that supply chain–specific issues need to be combined with related management practices (e.g., marketing, strategic management, human resources, to name a few) as well as industrial norms and environmental settings (Näslund, 2002). This allows researchers to gain a complete “rich” picture of the focal company situation (including its setting in the wider supply chain) and lends a second theoretical underpinning termed contingency theory (Lawrence & Lorsch, 1967; Thompson, 1967). Contingency theory argues that there is no single optimal organizational design because the environment and the industrial sector within which the organization operates will inevitably shape structures, end-to-end processes, and value streams (Scott & Cole, 2000). This suggests that companies should match their structures and processes to the environment in order to maximize performance (Lawrence & Lorsch, 1967).
Theoretical refinement of QSAM required several years of brainstorming, debate, experimentation, and triangulation before the current format emerged. QSAM brings together four key stakeholders (each with their own respective interests): the Host Organization (What’s in it for me?)—for example, the initial group of European automotive sector partners were quite clear from the outset that their transactions costs should be adequately met in return for granting access; the Business Community (What can we learn from them?); the Analytic Auditors (How do we rate this supply chain?); and the Research Community (What new knowledge is revealed?). The four parties are shown in Balanced Scorecard format in Figure 2, which also indicates four feedback loops. It is the ethos and purpose of every QSAM that the complete set of expectations shall be competently satisfied simultaneously.

QSAM balanced scorecard: Bringing together four interested parties
The “Auditor Competence Loop” is critical and requires that the auditors are well trained, focused, observant, and capable of participation as a member of an academic–industry team. In particular, they require an inquisitive mind, good time management skills, should not accept data or opinion at face value, and should aim to achieve good data triangulation via different data sources. The “Value Stream Competitiveness Loop” codifies (and ranks) measures of supply chain performance against external benchmarks. An interesting consequence of accumulating QSAM audit results is that they too provide a rich benchmarking source in their own right. The “Academia Peer Judgment Loop” is where the quality of the final research output is assessed. Finally, there is the “Business Principles Enhancement Loop” whereby new knowledge influences real work practices. At the same time, the need to keep the team anchored into an environment where industrial performance may be audited (and moreover understood) remains paramount.
Our experience of working with host organizations is that there are three major ways in which their transaction costs may be recovered. The first is by the research team identifying “quick hits,” which are relatively straightforward tasks that should result in a rapid performance improvement. The second is the provision of a workable and substantive brief that specifies the necessary activities in any follow-up BPI programs. Third, active participation as part of the audit team provides sound and relevant formal auditing experience for industrialists and academics. New propositions will also be output as interesting (and sometimes counterintuitive) phenomena emerge during the audit. An example is the generation of bullwhip across the supplier/original equipment manufacturer interface (Childerhouse, Disney, & Towill, 2008). Further explanation of the QSAM Balanced Score card can be found in Childerhouse and Towill (2011).
QSAM Process
The research/audit process is typically undertaken by a team of experienced researchers assisted by host organization supply chain “players” in a structured approach designed to fit around the limited time available to busy managers and staff. For example, when the aim of the QSAM is to assess the level of integration of a particular value stream, typically four to five researchers will spend 6 days on site closely assisted by an in-house business champion (Böhme, Childerhouse, Deakins, Potter, & Towill, 2008; Naim, Childerhouse, Disney, & Towill, 2002). Figure 3 illustrates the simplified process flow diagram of such a QSAM audit. Additionally, the researchers will wish to prove and continuously refine the methodology.

Simplified QSAM process flow diagram
Judgments regarding individual supply chains are based on a combination of case study–type metrics and statistically significant data. In seeking to maintain this standard, the researchers aim to exploit knowledge from as many data sources as possible. Figure 3 can only hint at the intensity of the various site-based activities, which are designed to achieve maximum information volume and fidelity. A QSAM is inevitably both time and resource constrained, and although data collection and analysis lies at its heart, front-end and back-end activities help ensure that the host organization receives maximum benefit from the experience and participants “sing from the same hymn sheet.” Continuing with the same example, in addition to taking steps to achieve internal validity through avoiding issues such as groupthink and risky shift within the team (by use of brainstorming, inviting external perspectives, etc.), audit data would be collected from four distinct sources to facilitate methodological triangulation and increase internal validity: process maps, attitudinal and quantitative questionnaires, semistructured interviews, and examination of archival information. Process mapping provides a detailed walkthrough and understanding of the material and information flows for one or more selected value streams. Questionnaires are used to gain overall information of the focal organization, including key customers, suppliers, production volumes, product variants, and company structure. Semistructured interviews are conducted with a cross section of senior and middle managers from each functional division and cover their role, perceived problems, and difficulties according to their respective position in the supply chain process map. The final type of data collection is examination of archival data; this being perceived to be relatively unbiased and able to provide historical factual data from respondents (Flynn, Sakakibara, Schroeder, Bates, & Flynn, 1990). The Ishakawa Diagram illustrated in Figure 4 summarizes the individual QSAM techniques used, grouped by written documentation, people contact, numerical methods, and investigative methods. The goal of the various data collection techniques is to fully understand the phenomenon being studied, and the accumulation of multiple supporting sources of evidence helps ensure that the facts being collected are indeed correct (Meredith, 1998).

Ishakawa diagram of QSAM data sources
Data triangulation also provides stronger substantiation of constructs and hypotheses (Eisenhardt, 1989), and the utilization of multiple investigators at the study site enables the case situation to be viewed from different perspectives, which adds to the richness of the data collected (Eisenhardt, 1989). This helps build confidence in the findings and increases the likelihood of surprise findings. In essence, triangulation improves the researcher’s judgment accuracy by providing several sources of verification (Flynn et al., 1990).
A feature of research to build theory from subjective social realities in this manner is the frequent overlap of data analysis with data collection (Eisenhardt, 1989; Lewis, Naim, Wardle, & Williams, 1998). During the QSAM theory-development process, logic replaces data as the basis for evaluation (Meredith, 1998). The central idea during the theory building process is to constantly compare theory and data—iterating toward a theory that closely fits the data. To build good theory this closeness is important because it takes advantage of new insights gained from the data and yields a valid theory (Eisenhardt, 1989).
When codified the large volume of collected data is used to explain the process, supply, control, and demand sources of uncertainty that indicate the overall integration level of a supply chain (Childerhouse & Towill, 2002). A most critical element of a QSAM audit study concerns how data extracted from the supply chain system is analyzed using systems thinking principles. In essence, cause–effect analysis is used to reveal: (a) the “major pain(s)” the company is feeling (symptoms of the underlying problems), (b) the supply chain/process integration barriers, and (c) the root (initiating) causes of the identified major pain(s). Cause–effect analysis fulfils the integrated/systems thinking perspective requirement of Edwards and Ram (2006) and has two main strengths: First, the cause–effect diagram is developed jointly by the research team members so it does not reflect one person’s opinion (researcher triangulation). Second, by adopting a holistic/systems perspective of the focal company, specific issues are combined to provide a complete “rich picture” of the focal company situation. This is useful for the final feedback session when clarification and endorsement are sought from company executives and employees involved in the study.
Validating the QSAM Approach
Because field research is commonly perceived as being prone to construct error, poor internal and external validation, and questionable generalizability (Meredith, 1998), the same quality criteria need to be applied to subjective research as are applied in objective studies. Table 1 outlines how the key quality criteria—internal validity, external validity, reliability, and objectivity (van der Vorst & Beulens, 2002)—are achieved during a Quick Scan audit. From Table 1, it may be concluded that QSAM is a robust and rigorous field research method, which also continues to evolve as researchers in the field discover further ways to enhance it.
Assessment of the QSAM Against Research Quality Criteria
To date, more than 50 site-based QSAM supply chain assessments have been conducted in a wide range of industrial and commercial sectors, including dairy, timber, health care, engineering, automotive, manufacturing, utility, and retail enterprises. QSAM has also been used on a “zoom and focus” basis, with audits that range from a single focal organization to the examination of three echelons in series. Organization size is no barrier to application and QSAM has been shown to have international application, with investigations to date in Germany, New Zealand, the Netherlands, Thailand, and the United Kingdom. This is significant because meaningful cross-industry/cross-national comparisons of supply chain performance can lead to attempts to “compare apples with oranges” when using quantitative techniques alone due to often questionable external validity.
Disseminating “Soft” Data
To enhance explanatory power and help achieve a meaningful contribution to knowledge it is important to tell a compelling case “story.” This should include descriptions of the organizational context within which the study took place and the personalities involved. An integrated and comprehensive story involves more than the mere plugging in of a few quotes, and it is important that managerial and theoretical implications are stressed; for example, Graebner (2004), Pagell and Zhaohui (2009), and Uzzi (1997) provide good examples of theory building from field studies.
Overall, QSAM audits have yielded a very valuable and varied pool of empirical data, and the understanding gained has manifestly enabled the development of new management theory and the validation, and more often further refinement, of research ideas (Childerhouse & Towill, 2003, 2004). For example, QSAM has provided the accurate data needed for statistical analysis of the levels of simplified material flow, supply chain integration, and presence of complexity symptoms (Childerhouse & Towill, 2003) and has clearly highlighted the advantages of designing and operating simple supply chains (Towill, 1999b). QSAM also facilitated in-depth study of 32 value streams with the explicit objective of benchmarking the overall level of supply chain integration (Childerhouse & Towill, 2004). This research demonstrated that only 10% of supply chains were approaching a seamless supply chain state. However, more encouraging was identification of a well-beaten path to advancement that clearly showed that organizations must get their own house in order before attempting to integrate with their suppliers followed in turn by their customers.
Transferring QSAM to Other Management Disciplines
Initially the Quick Scan was developed to diagnose the state of large organizations; hence, the research team typically identifies two value streams representative of the focal organization. When applying QSAM to small medium enterprises (SMEs) that frequently have one or two value streams, in effect the focal organization’s entire internal supply chain is being studied. Smaller company size also allows the researchers to spend time looking into such supply chain–related areas as staff development (obviously involving the human resources discipline), strategic direction of the enterprise and leadership (strategic management discipline), and customer relationship management (marketing discipline). In short, applying QSAM to SMEs greatly expands its traditional focus to an enterprise-wide scan and a stronger focus on “soft” data such as company culture and staff motivation. This enables researchers to consider a range of organizational, attitudinal, and social issues such as performance measures, (internal and external) customer satisfaction levels, communication channels, union issues, and staff motivations that can all affect alignment with the organization’s strategy and significantly impact bottom line performance.
Arguably, applying QSAM to SMEs is an early attempt to transfer the methodology to the wider management research community. QSAM enables researchers to take a more holistic overview/understanding of the alignment between organizational strategy and culture in order to address people management issues accurately and efficiently.
Discussion
The authors have shown that the management literature, and in particular the supply chain/logistics literature, overwhelmingly exhibits a single set of (objectivist/positivist) philosophical assumptions about the nature of phenomena and what researchers believe constitutes valid discipline knowledge about those phenomena. We believe that researchers could take advantage of rigorous mixed-methods approaches like QSAM that generate relevant knowledge, in particular involving site-based studies that examine human behaviors in their natural social settings. Such approaches also serve to remind researchers to “understand and acknowledge the extent to which the perspective they adopt will focus their attention on some things and not others, and bias their perception of the phenomena they study” (Orlikowski & Baroudi, 1991, p. 23).
QSAM enables good practice, poor practice, and performance trends to be detected (e.g., Childerhouse & Towill, 2004). Working within a team ensures that the QSAM researchers view the same case situation from different perspectives and in divergent ways, particularly when individuals are tasked with using specific methods (Eisenhardt, 1989). In addition to data triangulation that maximizes its validity, Greene, Caracelli, and Graham (1989) stated four additional purposes of QSAM: complementarity (seeking elaboration, illustration, enhancement, and clarification of the results from one method with the findings from the other method), development (when the researcher uses the results from one method to develop or inform the use of another method), initiation (discovering paradoxes and contradictions that lead to the research questions being reframed), and expansion (seeking to extend the breadth and range of inquiry by using different methods for different inquiry components).
However, QSAM has limitations; first around the methodology itself due to the limited amount of time able to be spent on a Quick Scan since it is simply not possible to understand and document the entire supply chain. Hence, focus is placed on gaining in-depth knowledge of specific value streams. There is also a clear need for buy-in from organizations since without it the quality of the information and resultant understanding of the supply chain can be significantly reduced.
Conclusion
It may well be true as McGrath (1982) has stated that it is not possible to conduct an unflawed study. Every research method has its inherent flaws and choice of that method will limit the conclusions that can be drawn. However, based on our field experiences we believe that it essential to obtain correlating evidences using a variety of methods. As researchers we also have a moral obligation to put science in the service of humanity’s most pressing problems, making it essential that the knowledge we discover is relevant to practitioners.
This article has sought to describe and present the benefits and challenges of a mixed-methods approach to research that seeks to frame supply chain issues through a variety of theoretical lenses. QSAM was designed to help bridge the academia–practitioner gap by focusing on practitioner relevance and scientific rigor. In reflecting on some 15 years of application and refinement, the structured framework, administration requirements, and overarching processes were presented and justified. Some guidelines for making sense of “soft” data describing relationships between technology, people, and organizations in the supply chain were also presented.
To date, QSAM research findings have been published mainly as applied case studies (e.g., Böhme, 2009; Potter, Mason, Naim, & Lalwani, 2004) and quantitative value stream comparisons (e.g., Childerhouse et al., 2011; Towill, Childerhouse, & Disney, 2002). Some 40 experts in eight universities have used the framework and its standardized protocols to perform readily comparable assessments of real-world supply chains. The application of QSAM to SMEs has extended the scope of the initial audit to include supply chain–relevant disciplines such as marketing, communications, strategic management, and leadership.
QSAM has been described as an example of a mixed-methods approach that has proven to be extremely valuable for studying messy, complex, and real-world supply chains. Philosophically it provides an interpretive perspective, one that emphasizes the importance of subjective meanings and social–political and symbolic actions in the processes through which humans construct and reconstruct their reality (Morgan, 1983, p. 396). Deployment of QSAM into SMEs is an early attempt to transfer the methodology to other management disciplines and to the wider management research community.
The international team of QSAM researchers is keen to extend a hand to others who are interested in interpretive (supply chain) management research that extends the QSAM research scope and network. It is thus our hope that this article may serve to motivate researchers to adopt a more balanced portfolio of research methods for studying management phenomena than has been the case until now.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
