Abstract
Is strategic decision comprehensiveness beneficial for firms? Despite significant empirical attention on this research question, inconsistent findings have prevented strong insights from being formed. To help the field move forward, we address long-standing controversies surrounding whether comprehensiveness is beneficial for firms, and whether environmental dynamism enhances or diminishes its effects. We meta-analyze 37 studies and provide the most definitive evidence possible regarding the strategic value of decision comprehensiveness. Our analyses show (1) that strategic decision comprehensiveness and organizational outcomes are positively related to a meaningful degree when subjective outcome measures are used, and (2) that environmental dynamism does not have a moderating impact on this comprehensiveness–outcomes linkage. Our results indicate that measurement strategies and methodological choices may have primarily driven the effects of strategic decision comprehensiveness reported in the literature. They also suggest that long-standing ideas related to moderating effects of dynamism do not hold. We define an agenda for future research and a roadmap for empirical efforts.
Introduction
Many strategic management theorists have argued that comprehensive approaches to strategic decision making provide value for organizations (e.g. Baer et al., 2013; Dean and Sharfman, 1996; Elbanna and Child, 2007a; van den Oever and Martin, 2019). Indeed, conventional wisdom holds that in making strategic decisions, upper-echelon managers often collect a wealth of information on the issue, develop numerous options for dealing with the situation, and systematically evaluate those options using organizational goals as key screening criteria. The term “comprehensiveness” is frequently used in strategy research to refer to this specific decision-making approach (e.g. Forbes, 2007; Fredrickson and Mitchell, 1984; Miller et al., 1998). Surveys by management consultants (e.g. Rigby, 2017; Zander, 2018) and academic studies of decision analysis tools used in business firms (e.g. Cabantous and Gond, 2011) highlight both top managers’ propensity to embrace the approach and the theoretical benefits of comprehensiveness.
Many believe that comprehensiveness enables decision makers to effectively deal with complex organizational issues, reduce their cognitive biases, and increase their commitment to the decision made (e.g. Bagozzi et al., 2003; Dean and Sharfman, 1996; Idson et al., 2004; Miller, 2008). Although the research community has generally endorsed this simple positive-effects view, the value of strategic decision comprehensiveness is the subject of long-standing debates in the literature (e.g. Forbes, 2007; Hutzschenreuter and Kleindienst, 2006; van den Oever and Martin, 2019). Indeed, some scholars believe that because comprehensive approaches fails to simplify the decision process, it can hinder firm functioning (Braybrooke and Lindblom, 1963; Snyder and Paige, 1958) by creating information overload, dysfunctional delays, and misallocation of precious resources in the pursuit of a non-existent right answer.
In addition, some theorists have highlighted environmental dynamism as a critical contextual factor that determines the actual degree to which comprehensiveness provides important benefits for firms (e.g. Burgelman et al., 2018; Miller, 2008; Shepherd and Rudd, 2014). For some scholars, low levels of dynamism set the stage for positive effects of comprehensiveness, while high levels of dynamism reduce, eliminate, or reverse those positive effects (e.g. Fredrickson, 1984; Fredrickson and Mitchell, 1984; Hough and White, 2003). For others, however, the exact opposite holds true (e.g. Bourgeois and Eisenhardt, 1988; Walters and Bhuian, 2004). Furthermore, empirical studies do not consistently support either of the two positions regarding the role of dynamism. A number of studies have reported either negative moderation by dynamism (e.g. Hough and White, 2003) or positive moderation by dynamism (e.g. Priem et al., 1995), while other studies have not found moderating effects of dynamism (e.g. Dean and Sharfman, 1996).
In sum, long-standing empirical research into the main and dynamism-moderated effects of comprehensiveness has been inconclusive, leading prominent scholars of strategy process to question the efficacy of research focused on comprehensiveness (e.g. Eisenhardt and Zbaracki, 1992; Forbes, 2007; Rajagopalan et al., 1993). Furthermore, several narrative reviews have highlighted methodological factors, such as measurement strategies employed or restriction of variance in key constructs, as possible sources of inconsistent findings across primary studies (e.g. Elbanna, 2006; Papadakis et al., 2010; Shepherd and Rudd, 2014). Thus, contextual factors, both substantive (i.e. the effects of dynamism) and methodological (i.e. the effects of methodological choices), may have played a key role in the controversies surrounding the decision and firm outcomes of strategic decision comprehensiveness (i.e. organizational outcomes from here on).
As such, we propose that better understanding of the context specificity (i.e. both environment and study contexts) of the comprehensiveness–outcomes relationship may help unravel the controversies highlighted above. Accordingly, we use meta-analytic techniques to synthesize quantitative research on this relationship. Indeed, meta-analysis can be instrumental for revealing the true meaning of a set of studies (Dalton and Dalton, 2005; Gonzalez-Mulé and Aguinis, 2018; Van Iddekinge et al., 2018). In the following section, we provide a brief history of the dominant thinking related to decision making by and within organizations; this puts our work into its proper theoretical context. Next, we develop our arguments and hypotheses related to the strategic value of decision comprehensiveness. First, we scrutinize both sides of the debate regarding the main effects of comprehensiveness on organizational outcomes. Second, we discuss the controversy surrounding the moderating role of dynamism on these effects. Third, we discuss the potential influences of methodological factors on the findings reported to date. Following the theory-building section, we describe our research methods and empirical findings. By way of foreshadowing, the results of our investigations indicate that the positive findings reported in past comprehensiveness studies are more likely driven by methodological choices than by true effects on organizational outcomes. We conclude with the implications of our work for interpreting past research and planning future work.
Theory and hypotheses
Strategic decisions can be described as important and non-routine decisions that require significant resource commitments and have notable influences on the performance, survival, and health of firms (Mintzberg et al., 1976; Schwenk, 1988). They are typically made by the most senior and influential managers at the apex of organizations (i.e. upper-echelon or top managers), who frame strategic priorities, establish corporate policies, enforce operating standards, and develop management talent for their firms (Katzenbach, 1997). Although alternative decision approaches such as intuition and heuristics can be used, many top managers seem to prefer systematic and extensive approaches to making strategic decisions (e.g. Cabantous and Gond, 2011), based on the belief that more information leads to better choices. As such, they collect input from staff, middle managers, and external consultants during strategic decision-making activities (e.g. Fredrickson and Mitchell, 1984; Meissner and Wulf, 2014; Miller et al., 1998). In the paragraphs below, we evaluate the common preference for rational analysis in its proper historical context.
Rational-choice theory: a brief history
The neoclassical view of decision making provides a particularly narrow version of rationality. The main assumption of this view is that decision makers identify an exhaustive set of alternatives and systematically evaluate the future consequences of those alternatives. Utilities are well specified, and utility maximization is the central goal. Although sometimes used as a theoretical foil even today, this narrow version of rationality is considered applicable only to simple, bounded problems that rarely, if ever, occur in actual organizations. In more technical terms, the neoclassical view is generally understood as applicable only to static problems in situations characterized by certainty (March and Simon, 1958).
As a more realistic alternative, procedural rationality came to the fore in the 1970s (Simon, 1978). From this perspective, decision makers explicitly identify meaningful alternatives and pursue some understanding of the future consequences of those alternatives. However, these decision makers are usually not exhaustive or completely successful in these activities. Furthermore, in a procedurally rational world, decision makers may consider one alternative only, especially if the first alternative considered is satisfactory. While they probably adopt an approximation of consequentialist logic (see March, 1997), decision makers are not utility maximizers and indeed cannot be since their preferences tend to be ill-formed, shifting, and inconsistent (e.g. Miller, 2008). Finally, while they may believe that marginal costs of investigatory activity should not exceed marginal benefits, decision makers do not follow this basic idea in any strict or formulaic way.
Both neoclassical and procedural rationality inform modern rational choice theory (Cabantous and Gond, 2011), which includes the normative ideal of multiple alternatives for any decision, as well as thorough evaluations of those alternatives against explicit preferences. However, the theory does not generally incorporate the idea of optimization based on strict utility maximization. In essence, modern rational choice theory holds that better decisions are made when a number of alternatives are richly considered for their expected consequences, even though key aspects of the process may be subject to a series of human foibles and limitations.
Overall comprehensiveness–outcomes relationship
Arguments for negative effects
Building on some concerns originally applied to the neoclassical school, critics of modern rational choice theory argue that comprehensiveness is not a positive force in the strategy process. They often voice two particular concerns. First, the consequences of possible courses of action are not known when a decision is being made (March, 1997, 2006). Because the world is not static, the future consequences of today’s actions may not be discernable and cannot be easily reduced to a set of meaningful probabilities. Attempts to arrive at definitive or even probabilistic predictions are at best a waste of resources, and at worst a source of poor decisions. Thus, intuition, educated guesses, or perhaps imitation, rather than comprehensiveness, may be more appropriate decision tools (e.g. Albin and Foley, 1998; Dane and Pratt, 2007). Second, specific outcome preferences that are held today may not be held tomorrow (March, 1997, 2006). Indeed, preferences often change radically and rapidly (Miller, 2008). As such, comprehensiveness could create overwhelming difficulties in evaluating the environment (cf. Bennett and Lemoine, 2014).
Arguments for positive effects
Proponents of modern rational choice theory, on the contrary, argue that comprehensiveness is a positive force in the strategy process. They explain that comprehensiveness positively influences organizational outcomes for three primary reasons (Miller, 2008). First, comprehensiveness can help decision makers better identify and understand complex factors that are embedded in a strategic issue (e.g. Dean and Sharfman, 1996). Second, it can help decision makers reduce the negative effects of cognitive biases, particularly sunk-cost and confirmation biases (e.g. Idson et al., 2004). Third, it can help ensure that a decision is implemented in an effective way. Having invested a great deal of time, energy, and resources in the initial stages of the decision process, managers likely perceive procedural justice and, as a result, are motivated to implement the decision (e.g. Bagozzi et al., 2003; Miller, 2008).
Conceptual synthesis of the theoretical predictions
Despite their conflicting views on the effects of comprehensiveness, both proponents and critics of modern rational choice theory believe that comprehensiveness has non-trivial effects on outcomes. Given that sound arguments and logic can be used to support either position, the relevant question to address now is whether strategic decision comprehensiveness leads to desirable or undesirable organizational outcomes.
We draw from theories and findings in related research streams to argue that, on average, comprehensiveness has positive effects on outcomes. Recently, Samba et al. (2018) argued that information elaboration—a broad construct encompassing comprehensiveness—has positive organizational outcomes. Indeed, information elaboration involves surfacing and integrating different information elements in decision processes. It also encourages creativity in problem-solving and generation of decision alternatives. Using meta-analytic techniques, Samba et al. (2018) found that the aggregate main effect of information elaboration on organizational outcomes is significant and positive, on average. Earlier, Miller and Cardinal (1994) had argued that strategic planning—a construct related to but distinct from comprehensiveness—positively affects a firm’s financial performance. Using meta-analytic techniques, the authors found support for this prediction in the aggregate. Moreover, recent research in Economics has also supported the notion that comprehensiveness leads to desirable organizational outcomes, such as higher-quality decisions, (e.g. Sethi and Yildiz, 2016), increased profits in the book publishing context (Bar-Isaac et al., 2012), and effective uses of human capital (Brynjolfsson and McElheran, 2016). Overall, these arguments and findings considered together suggest that strategic decision comprehensiveness leads to desirable organizational outcomes.
Hypothesis 1. A positive relationship between strategic decision comprehensiveness and organizational outcomes exists in the aggregate.
The role of dynamism in the comprehensiveness–outcomes relationship
Arguments for less positive effects in dynamic environments
As discussed earlier, a number of standard criticisms of rational decision approaches exist in the literature. Although the issues raised have been cast as universal because the world is never completely static (cf., Braybrooke and Lindblom, 1963; Snyder and Paige, 1958), they may be more applicable to contexts that are high in environmental dynamism. In such contexts, changes in customer tastes, competitor tactics, and technologies are constant and very difficult to predict (Fredrickson and Iaquinto, 1989; Glick et al., 1993). Consequences of possible strategic actions can be very difficult to discern, and continued efforts to specify them through comprehensive analyses could breed frustration and conflict. Perhaps more importantly, those efforts could lead to overly strong commitments to wrong answers (i.e. sunk-cost bias).
Relatedly, preferences for a course of action may change as events unfold in unpredictable directions. Thus, judgments made one day on the basis of a rich preference-based approach to decision making could be irrelevant the next. Yet, irrational escalation of commitment borne of “carefully-made” choices can prevent change of course. Collectively, these arguments reflect the ideas emphasized by Fredrickson and colleagues in their seminal work on comprehensiveness (Fredrickson, 1984; Fredrickson and Iaquinto, 1989; Fredrickson and Mitchell, 1984). Also, there simply may not be enough time for extensive investigations in dynamic situations. Indeed, the time needed for rich information collection and deep analysis is likely unavailable in rapidly changing situations. Attempts to be comprehensive when making strategic decisions may cause harmful delays in responding to fleeting windows of opportunities and pressing problems. In fact, time pressure has been a key point for the strategy process scholars who have argued that comprehensive approaches damage firms (cf. Eisenhardt, 1989).
Arguments for more positive effects in dynamic environments
In contrast to the above reasoning, some researchers have argued that comprehensiveness may have greater value for firms in dynamic contexts (e.g. Miller et al., 1998; Walters and Bhuian, 2004). Because the world is constantly changing in these environments, creating new knowledge through careful investigation may be crucial for generating positive outcomes. Furthermore, previous learning may be irrelevant, and failing to acquire, analyze and evaluate new information on various environmental conditions could result in biased and problematic choices. Thus, when the past is not indicative of the future, identifying new decision factors and trends through systematic search and analysis is important (Bourgeois and Eisenhardt, 1988; Eisenhardt, 1989).
Relatedly, collecting and analyzing new information help managers diagnose whether environmental changes are transient or not. To miss this important cue can threaten well-being and even survival of the firm (Aldrich, 1979). As Bourgeois and Eisenhardt (1988) explained, upper-echelon managers facing dynamic environments must “put their world in order through a rational process of identifying goals and setting priorities, collecting information, and generating and evaluating alternatives in order to gain a sense of control” (p. 827). The authors traced their position to psychoanalysis, in which patients dealing with uncertain situations are encouraged to follow rational decision processes (De Board, 1978).
Stable contexts, on the contrary, are reasonably well-understood at the outset of a decision process. Thus, top managers can rely more heavily on past experiences or even rule-based choices when making strategic decisions (Dean and Sharfman, 1996). In other words, managers do not need as much collection and analysis of new information (e.g. Glick et al., 1993). Moreover, because investigations have real costs associated with information acquisition and opportunity costs related to resource expenditure, the marginal benefits of extensive investigations are unlikely to exceed marginal costs in a strong way (cf. March, 1994). Thus, the returns to extensive investigations are likely low in stable contexts.
Conceptual synthesis of theoretical predictions
As indicated above, fundamental disagreements exist regarding the strategic value of comprehensiveness in dynamic environments. Moreover, overarching theoretical ideas that can be used to adjudicate the debate do not exist. Here again, sound arguments and logic can be used to support either position; this is part of the rich tapestry of work in this research area. While we relied on theories and findings in related research streams (e.g. pre-existing meta-analyses, findings from Economics research) to develop our main-effect hypothesis, such foundations on the moderating role of dynamism on the comprehensiveness–outcomes relationship are not available. What is clear, however, is that both sides believe that dynamism influences the comprehensiveness–outcomes relationship. In other words, the controversy lies in how dynamism affects the outcomes of comprehensiveness. Thus, our hypothesis for moderation by dynamism is non-directional and builds on the common ground between the two opposing sides of the debate.
Hypothesis 2. Environmental dynamism affects the positive relationship between strategic decision comprehensiveness and organizational outcomes.
The role of statistical artifacts and measurement strategies
Statistical artifacts, such as sampling error and measurement error, may explain the inconsistencies in findings across past studies on the strategic implications of decision comprehensiveness (e.g. Hunter and Schmidt, 2014; Orlitzky et al., 2003). Furthermore, because comprehensiveness and organizational outcomes are broad meta-constructs, their specific operationalizations may substantially influence the relationships proposed above. Simply stated, differences in research methods across studies may produce differences in findings. In fact, previous meta-analytic work has established that research methods have important effects on observed relationships in management research (e.g. Miller and Cardinal, 1994; Orlitzky et al., 2003). Thus, to more rigorously estimate the main and dynamism-moderated effects of comprehensiveness, we consider the impact of statistical artifacts and measurement strategies on the findings reported to date.
We also expect that the inconsistencies in the main and dynamism-moderated effects of comprehensiveness can be partially explained by the methodological choices adopted in primary studies. Accordingly, we draw from prior literature reviews (e.g. Elbanna, 2006; Papadakis et al., 2010; Shepherd and Rudd, 2014) and other commentaries on the topic (e.g. Forbes, 2007) to predict that the following measurement strategies (c.f. Orlitzky et al., 2003) accounts for a substantial proportion of cross-study variance: (1) focus of comprehensiveness measures (i.e. whether the measures focus on efforts spent in specific steps of the decision process vs. on the data used in the overall process), (2) use of proximal versus distal outcome measures (e.g. decision effectiveness vs. firm performance), (3) objectivity of outcome measures (e.g. performance data from databases such as Compustat vs. managers’ reports of privately held financial data versus managers’ perceptions of organizational performance), and (4) measurement lag structure (e.g. comprehensiveness and outcomes were measured at the same time vs. a lag time was observed between measuring comprehensiveness and outcomes).
Focus of comprehensiveness measures
Comprehensiveness is labeled and assessed in different ways (e.g. Elbanna, 2006; Forbes, 2007). We identified two main foci of the measures used in primary studies: detailed focus on resources expended in various stages of the decision-making process, such as problem identification, alternative generation, and option vetting (i.e. detail-oriented approach, as exemplified by Fredrickson, 1984) versus a general focus on relevant and crucial information sought, collected, and analyzed in the overall decision process (i.e. big picture approach, as exemplified by Dean and Sharfman, 1996; Miller and Toulouse, 1986). For instance, survey items, such as “when confronted with an important, non-routine problem or opportunity, to what extent does your firm conduct multiple examinations of any suggested course of action” illustrate the detailed-oriented focus, while items such as “in general, how effective was the group at focusing its attention on crucial information and ignoring irrelevant information” represent the big-picture focus. In both foci, the measures emphasize extensiveness of the process of collecting and analyzing large amounts of information. In other words, both sets of measures attempt to capture the amount of investigatory activity. As such, they are used to assess comprehensiveness (e.g. Elbanna and Child, 2007a; Forbes, 2007; Priem et al., 1995). However, using a detail-oriented versus big-picture approach to measuring comprehensiveness, or vice versa, may have led to different findings on to the effects of comprehensiveness (Elbanna, 2006; Shepherd and Rudd, 2014). Indeed, the detail-oriented approach typically includes a relatively large number of survey items that explicitly emphasize the stages of classical decision theory, while the other approach often uses a few items that lack this explicit emphasis.
Hypothesis3a. The focus of comprehensiveness measures affects the main and dynamism-moderated relationships between strategic decision comprehensiveness and organizational outcomes.
Proximal versus distal outcome measure
Strategic decision-making researchers typically assess the outcomes of comprehensiveness through either decision (e.g. decision effectiveness) or firm outcome variables (e.g. firm profitability or return on assets (ROA)). Firm outcome measures often reflect organizational performance, which is a broad meta-construct (e.g. Miller et al., 2013) that is subject to many exogenous effects (e.g. Pearce et al., 1987). We consider firm outcome measures distal. Decision outcome variables, however, attempt to evaluate the immediate outcomes of decision processes. They typically link comprehensiveness to decision effectiveness (e.g. Dean and Sharfman, 1996; Ji and Dimitratos, 2013), which refers to contribution of the decision toward meeting existing organizational goals (Elbanna and Child, 2007a; Meissner and Wulf, 2014). We consider decision outcome measures proximal. Given the shorter distance between the predictor and criterion variables, we expect the effects of comprehensiveness to be greater for proximal outcomes, compared with distal outcomes (cf. Bell et al., 1997; Hough and White, 2003).
Hypothesis 3b. The proximity of outcome measures affects the main and dynamism-moderated relationships between strategic decision comprehensiveness and organizational outcomes.
Objectivity of outcome measures
In some primary comprehensiveness studies, outcome data were collected through publicly available archival sources (i.e. what we call “objective outcome data”), through survey questions that require specific outcome information (i.e. what we call “moderately objective outcome data”), or through Likert-type scale items asking managers to report their perceptions of organizational performance (i.e. perceptual outcome data, or what we call “subjective outcome data”). The effects of comprehensiveness reported in these studies are likely influenced by this set of methodological choices for two reasons. First, common methods bias could have substantially influenced the correlations when objective outcome data were not used. Indeed, a survey-survey measurement scheme is often introduced in field studies because comprehensiveness is typically assessed with survey scales. This scheme, in turn, sets a foundation for biased data and inflated observed relationships. Second, and in stark contrast to the above reasoning, more accurate data could have been generated through survey questions. Because key informants are expert judges on how their organizations are actually doing, they may provide more accurate outcome information compared with archival sources. Also, assessments involving archival market or accounting data are often confounded with earnings management, public relations, and other extraneous considerations (Miller and Cardinal, 1994).
Hypothesis 3c. The objectivity of outcome measures affects the main and dynamism-moderated relationships between strategic decision comprehensiveness and organizational outcomes.
Measurement lag structure
This potential methodological artifact indicates the temporal distance between the measurements of comprehensiveness and outcomes. In some studies (e.g. Carmeli et al., 2013; Covin et al., 2001), outcomes were measured prior to measuring comprehensiveness (i.e. reverse lag). In other studies (e.g. Forbes, 2005; Heavey et al., 2009), outcomes and comprehensiveness were measured at the same point in time (no lag). Finally, in a third category of studies (e.g. Dean and Sharfman, 1996; Ji and Dimitratos, 2013), outcomes were measured after measuring comprehensiveness (lag). We argue that the effects of comprehensiveness found are influenced by the measurement lag structure adopted (e.g. Miller and Cardinal, 1994). Because strategic decisions entail high levels of uncertainty and have long term consequences for organizations, managers cannot predict their outcomes when they make these decisions (Mintzberg et al., 1976; Papadakis and Barwise, 1998). Potential outcomes of strategic decisions are realized, observable, and measurable often after many years (e.g. Dean and Sharfman, 1996; Elbanna, 2006). Thus, measuring the outcomes of strategic decisions and the process through which they were made (i.e. comprehensiveness) at the same time likely leads to biased results.
Hypothesis 3d. The measurement lag structure influences the main and dynamism-moderated relationships between strategic decision comprehensiveness and organizational outcomes.
Data and methods
To assess the magnitude of the comprehensiveness–outcomes relationship, we used two meta-analytic techniques to aggregate the results of primary studies on this topic. First, we used the Hedges and Olkin meta-analysis technique (i.e. HOMA; for example, Hedges and Olkin, 1985; Karna et al., 2016) to explore our hypotheses with the sample size weighted mean correlations of our overall study sample and of different sets of subsamples. By aggregating the findings and correcting for statistical artifacts in our overall study sample, we began to form a more precise picture of the effects of strategic decision comprehensiveness. With subgroup analyses, we explored the possibility that measurement strategies used in primary studies explain the inconsistencies in literature and began to tease out the role of dynamism in the effects of strategic decision comprehensiveness. Overall, these preliminary analyses may provide both surprising and important insights on this important research area.
Second, we tested our hypotheses more rigorously using meta-analytic regression analysis (i.e. MARA), in which correlations from various samples are regressed onto the hypothesized moderating factors (e.g. Drees and Heugens, 2013; Karna et al., 2016; Lipsey and Wilson, 2001). This technique facilitates the robust testing of substantive moderators, such as dynamism, while controlling for cross-study methodological artifacts that may have strongly influenced the findings in primary studies. In other words, the effects of multiple moderators are examined simultaneously to determine what is truly important for progress in a research area.
Data
To meet basic qualification requirements for inclusion in our work, a study had to: (1) be focused on strategic decision making, (2) have a process construct focused on extensiveness in collecting and using information to handle specific problems/opportunities that carry immediate needs for investigation and choice, and (3) have an outcome construct related to decision or firm performance. In essence, a study had to be in the research stream associated with the work of Bourgeois and Eisenhardt (1988), Dean and Sharfman (1996), Fredrickson (1984), Fredrickson and Mitchell (1984), and Priem et al. (1995). In addition, the study had to be quantitative in nature and give sufficient statistical information for collecting or computing correlations. Finally, the study needed to have a sample not used in another study already included in the database. Keywords that past authors could have used in studying the strategic value of comprehensiveness were identified and used in our literature searches. Keywords for our comprehensiveness variable included “comprehensiveness,” “rationality,” and “procedural rationality.” Terms representing our outcomes construct included “decision effectiveness” and “firm performance.”
Importantly, unpublished research, working papers, and conference papers were not incorporated. Although some researchers include unpublished work in meta-analyses, there are dangers involved with this practice. Chief among these dangers is response bias in soliciting unpublished studies, which can yield a non-representative set of such studies. In contrast, the entire set of published studies is accessible. Moreover, Dalton et al.’s (2012) findings suggest that the impact of the “file-drawer” problem (i.e. exclusion of unpublished work) “does not produce an inflation bias and does not pose a serious threat to the validity of meta-analytically derived conclusions” (p. 222). Nonetheless, the possibility of a file drawer problem was addressed via the calculation of fail-safe N for all reported mean correlations (Rosenthal, 1991).
Initially, 81 papers were identified for potential inclusion in our study, through searches of databases (e.g. Social Sciences Citation Index), management journals (e.g. Strategic Management Journal, Academy of Management Journal, and Strategic Organization), literature reviews found in known comprehensiveness–outcome studies (e.g. Miller, 2008), theoretical syntheses of comprehensiveness–outcome research (e.g. Eisenhardt and Zbaracki, 1992; Forbes, 2007), and qualitative reviews of the strategy process literature (e.g. Hutzschenreuter and Kleindienst, 2006; Rajagopalan et al., 1993; Shepherd and Rudd, 2014). Of the initial 81 papers, 75 were quantitative in nature. After carefully reviewing these 75 papers, we found that 23 of them had an empirical focus that lies outside of the strategic comprehensiveness domain. Next, 13 out of the 52 remaining papers were redundant because their samples were already included in our analyses through other studies. Finally, comprehensiveness–outcome correlations could not be obtained from 6 of the 39 remaining papers.
The 33 usable papers yielded 37 different samples (a few papers included multiple samples). These 37 samples constitute the empirical foundation for our meta-analytic work. Additional details of our study inclusion vetting process are provided in the Supplemental Material Appendix.
Coding of variables
Comprehensiveness–outcome correlations
Product-moment correlations between comprehensiveness and organizational outcomes were obtained from the 37 primary study samples. Eleven of these samples provided multiple correlations because the authors incorporated different categories/levels of some of our potential moderator variables (e.g. a correlation between comprehensiveness and ROA, and a correlation between comprehensiveness and decision effectiveness). For such samples, all of the individual correlations were collected and treated as if they had come from independent studies—a common meta-analytic practice (e.g. Dalton and Dalton, 2005; Tracz et al., 1992). In total, we obtained 50 correlations for our analyses (k = 50). The Supplemental Material Appendix includes a robustness test showing that the non-independence in our set of correlations does not significantly affect our results.
Environmental dynamism
Two authors used their judgment to code the dynamism variable based on (1) narrative descriptions of dynamism faced by the firms in a given sample, (2) extant knowledge related to the nature of the industries represented in a sample, and (3) direct assessment for a sample, such as a scale or a score, if one existed. Dynamism was coded as a three-level categorical variable based on the level of dynamism represented in a sample: stable environments only, dynamic environments only, and both environments.
Importantly, our approach to coding dynamism considered the potential that due to its treatment in previous studies, our dynamism variable may be both a substantive and methodological moderator for the comprehensiveness–outcome relationship. Indeed, early research practice, which involved range restriction of the dynamism variable, may have provided the fuel for the controversy regarding the moderating role of dynamism on the effects of comprehensiveness. For instance, Fredrickson (1984) restricted his study sample to stable environments only, based on the argument that comprehensiveness provides value for such settings. In other oft-cited studies, the reverse was done. For example, Eisenhardt (1989) and Bourgeois and Eisenhardt (1988) analyzed samples including dynamic environments only. Overall, what may have started out as purposeful sampling—a technique widely used in qualitative research—seems to have morphed into a theoretical debate over time. Yet, restricting the range of the key variable leads to biased results (e.g. Aitken, 1935; Sackett et al., 2002). Our approach to coding dynamism allowed us to also investigate whether this methodological choice influenced findings reported in the literature. Coding was done by two authors, with an inter-rater reliability of 0.88 (Perreault and Leigh, 1989). Any discrepancy that arose in the coding process was resolved through discussion.
Measurement strategies
The focus of comprehensiveness measures was coded as a two-level categorical variable, detail-oriented versus big-picture. Some measures (i.e. the detail-oriented category) focused on efforts expended during specific steps of the decision process. In contrast, other measures focused on crucial information and/or systematic-quantitative analyses used during the overall decision process. Coding was done by two authors, with an inter-rater reliability of 0.87 (Perreault and Leigh, 1989). A representative sample of items used to measure comprehensiveness is included in the Supplemental Material Appendix.
Outcome measures were coded based on two methodological factors. One potential moderator related to the measurement of outcomes is a two-level categorical variable assessing the impact of distance between the decision process and the outcome variable. Decision outcomes (e.g. decision quality or effectiveness) are proximal outcome variables, while firm outcomes (e.g. firm performance or growth) are distal outcome variables. Coding was done by two authors, with an inter-rater reliability of 0.90 (Perreault and Leigh, 1989). Another potential moderator, which relates to the objectivity of the outcome measure, is a three-level categorical variable. Outcome measures were coded as objective, moderately objective, or subjective, based on how the outcome data were obtained. The label “objective outcome measures” indicates that publicly available performance data were used in the study. “Moderately objective outcome measures” shows that survey items requiring specific outcome information were given to key informants. The label “subjective outcome measures” means that simple Likert-type scales were used to assess general decision or firm performance. To illustrate, performance can be assessed using ROA data from Compustat (objective outcome data), by asking managers to report specific ROA numbers (moderately objective outcome data), or through survey items asking managers to rate ROA relative to competitors (subjective outcome data). Coding was done by two authors, with an inter-rater reliability of 0.97 (Perreault and Leigh, 1989).
Finally, measurement lag structure was coded as a three-level categorical variable. The three categories are as follows: studies with no lag, when outcomes and comprehensiveness were measured at the same point in time (e.g. Elbanna, 2012), studies with lag, when outcomes were measured 1 year or more after comprehensiveness had been measured (e.g. Ji and Dimitratos, 2013), and studies with reverse lag, when outcomes had been measured 1 year or more before comprehensiveness was measured (e.g. Priem et al., 1995). Coding was done by two authors, with an inter-rater reliability of 1 (Perreault and Leigh, 1989).
Meta-analytic procedures
Initially, we used random effects HOMA models to estimate sample size weighted mean correlations for the overall sample and for all subsamples (i.e. subgroup analyses). Comparing the mean correlations across subsamples provided preliminary insights into the nature and variability of the comprehensiveness–outcome relationship. Next, we used a MARA procedure to more definitively examine and test our hypotheses. In this approach, correlations constitute the dependent variable, characteristics of samples are the independent variables, and each observation is weighted by its sample size. Similar to our reasoning with the HOMA procedure, we used a random effects model with the MARA procedure (see Appendix S4 of Samba et al., 2018 for a detailed discussion of the fixed and random effects meta-analysis models).
Results
Preliminary analyses
In the initial HOMA (see Table 1), the mean estimate of the population correlation between comprehensiveness and outcomes, based on our final sample, was r = 0.177 (p ⩽ 0.001). The mean effect size was r = 0.215 in the overall sample when the observed correlations were individually corrected for measurement error—that is, by dividing the focal correlation by the product of the square roots of the reliabilities of the measures assessing the two focal variables (Hunter and Schmidt, 2004). Because the 95% confidence interval did not cross 0, and because the file-drawer test (Lipsey and Wilson, 2001) indicated that a large number of null results would be required to bring 0 into the interval (i.e. fail-safe N = 1993 samples), this positive mean effect size supported Hypothesis 1 (i.e. that strategic decision comprehensiveness and organizational outcomes are positively related in the aggregate).
Hedges and Olkin’s (1985) technique was used. HOMA requires effect sizes to be normally distributed (Rosenthal, 1991) and thus uses the Fisher z transformation for individual correlations. As a result, the technique yields estimates of true-score correlations that avoid the bias of r found in smaller samples. Furthermore, a random effect model of the technique was used because it is suitable for a meta-analysis with different subsets. This model is also more conservative than a fixed effect model because it attributes the variability in effect size to both sampling error and to variability in the population of effects, as opposed to sampling error only (Lipsey and Wilson, 2001).
For column headings: k = number of effect sizes, N = total sample size,
These calculations are based on uncorrected effect sizes, as reliabilities were not reported for every sample and an individual correction for measurement error was thus not possible.
p ⩽ 0.01; ***p ⩽ 0.001.
Furthermore, the large Q statistic confirmed our presumption of heterogeneity underlying the mean effect size. In other words, one or more moderators may have influenced the comprehensiveness–outcome relationship, making it reasonable to investigate potential moderators. To explore this possibility, we conducted subgroup analyses (see Table 1).
These preliminary moderator analyses provided a basis to reject Hypothesis 2 (i.e. that dynamism affects the positive relationship between comprehensiveness and outcomes). Indeed, if there were positive moderation by dynamism, the subgroup containing samples of firms in dynamic environments only would have shown the highest correlation among the three subgroups, and the subgroup containing samples of firms in stable settings only would have shown the lowest correlation among the three subgroups. The reverse would have been true in the case of negative moderation by dynamism.
Instead, the subgroup analyses revealed a surprising pattern of results involving dynamism. Samples of firms facing stable contexts only and samples of firms facing dynamic contexts only yielded comprehensiveness–outcome correlations that were substantially weaker (r = −0.006, n.s., and r = 0.092, n.s., respectively) than those produced by samples with both levels of dynamism represented (r = 0.293; p ⩽ 0.001). In fact, neither of the mean correlation in the subgroups of samples with restricted dynamism level (i.e. stable contexts only and dynamic environments only) was statistically significant. Thus, our findings suggested that the level of dynamism does not moderate the comprehensiveness–outcomes relationship. At the same time, they suggested that when the range of dynamism is restricted to either stable settings or dynamic environments, the comprehensiveness–outcomes correlation is significantly different from the correlation obtained when the range of dynamism is not restricted. Thus, our findings suggested that a methodological artifact, namely, range restriction of the dynamism variable, explained differences in effects previously observed on moderation by dynamism.
In addition, the subgroup analyses provided preliminary indications that other methodological artifacts may also have influenced the findings reported in the literature. For instance, studies assessing proximal outcomes of comprehensiveness seemed to produce more positive correlations than studies in which distal outcomes were measured (r = 0.302; p ⩽ 0.001 vs r = 0.144; p ⩽ 0.01). In studies with subjective outcome measures (perceptual approach, simple scale items), comprehensiveness was highly correlated with outcomes (r = 0.298; p ⩽ 0.001). In contrast, the use of either objective or moderately objective outcome data yielded non-significant findings in the aggregate (r = −0.052, n.s., and r = 0.038; n.s., respectively). Furthermore, the measurement lag structure used may have affected the correlations found in primary studies. Indeed, t-tests suggested that reverse lag (i.e. outcomes are measured before comprehensiveness; r = 0.107; p ⩽ 0.01) was statistically different from lag (i.e. comprehensiveness is measured before outcomes; r = 0.293; p ⩽ 0.001; Δr = 0.186; t = 4.950) and no lag (i.e. comprehensiveness and outcomes are measured at the same time; r = 0.217; p ⩽ 0.01; Δr = 0.110; t = 2.487). Although, lag seemed to produce the most positive mean correlation of the three categories, a test suggested that its mean correlation is not statistically different from that of the no lag category (Δr = 0.076; t = 0.994). Finally, a t-test indicated that choosing between a detail-oriented (r = 0.171; p ⩽ 0.01) and big-picture approach to measuring comprehensiveness (r = 0.183; p ⩽ 0.001) did not make a difference (Δr = 0.012; t = 0.318). Overall, the subgroup analyses suggested that how dynamism was treated, the objectivity of the outcome measures used, whether the outcome measures were proximal or distal, and the measurement lag structure adopted in primary studies had substantially influenced the moderation effects found to date. In contrast, the focus of comprehensiveness did not show an impact on these findings.
Meta-analytic regression analyses
Because subgroup analyses consider potential moderator effects in isolation, we used a maximum likelihood random effects MARA model to simultaneously investigate the effects of multiple potential moderators. With the MARA, we can shed a more definitive light on the role of dynamism and methodological factors in existing findings.
The MARA model (see Table 2) confirmed that the objectivity of outcome variables had strong effects on the comprehensiveness–outcome relationship (p ⩽ 0.001), with objective and moderately objective outcome measures both producing substantially weaker effects than subjective outcome measures (the subjective outcome measure category was the reference category for the three-category dummy variable). Furthermore, the findings indicated that when researchers studied firms in stable settings only and in dynamic environments only, comprehensiveness produced essentially equal effects, which are substantially weaker than when both environments were considered. Thus, the MARA provided strong evidence that the level of dynamism did not moderate the comprehensiveness–outcomes relationship. Rather, range restriction of the dynamism variable in primary studies explained differences in effects previously observed on the moderating role of dynamism. Finally, when all the potential moderators competed in the same model, choosing proximal over distal outcomes, measuring comprehensiveness and outcomes at the same time or with a lag, and choosing a detail-oriented versus big-picture focus on comprehensiveness did not affect the relationship between comprehensiveness and outcomes. In other words, the comprehensiveness–outcome relationship did not seem to vary based on the prevailing levels of these three methodological factors.
Results of the meta-analytic regression analysis (MARA).
The number of effect sizes, k = 50.
The total sample size, N = 5084.
p ⩽ 0.05; **p ⩽ 0.01; ***p ⩽ 0.001.
Overall, the MARA provided a more definitive answer regarding the main and dynamism-moderated effects of comprehensiveness. First, it indicated that comprehensiveness and outcomes are positively related to a meaningful degree only when subjective outcome measures (i.e. measures that capture managers’ perceptions of performance) were used in primary studies. Second, the more rigorous moderator analysis strongly suggested that dynamism does not play the role that either side of popular theorizing proposes. Rather the controversy was likely caused by range restriction of the dynamism variable in primary studies, a methodological factor.
Robustness analyses
To ensure adequate confidence in the results presented above, we conducted a number of robustness tests. First, we averaged all within-sample correlations in order to have only independent samples in our analyses (k = 37, rather than k = 50). Second, we added three studies that had not been included, but whose exclusion could be challenged on the basis of reasonable doubt about their conceptualizations and/or emphases being inconsistent with ours (Jones et al., 1992; Thomas and Ambrosini, 2015; van den Oever and Martin, 2019). Third, we deleted studies that did not have a direct focus on the comprehensiveness–outcome linkage (e.g. studies where the comprehensiveness or outcome variable had been a control variable). Finally, we checked for potential multicollinearity issues in the MARA. The patterns of our results were unchanged in all robustness tests (see the Supplemental Material Appendix for the details of the robustness tests).
We also investigated the ramifications of (1) having multiple informants take the same survey (i.e. single vs multiple informants) and (2) separating the sources of comprehensiveness and outcome data (i.e. same vs different informants for comprehensiveness and outcomes). In essence, we distinguished between common source variance (i.e. the same informants provided the predictor and outcome data) and common method variance (i.e. survey data were used to assess both comprehensiveness and outcomes). The robustness test did not suggest a bias stemming from common source. Indeed, the mean correlation in studies where multiple informants provided survey data (r = 0.20; p ⩽ 0.001; k = 17) was essentially the same as the one found in studies where a single informant provided all the data (r = 0.24; p ⩽ 0.001; k = 25). The same pattern was observed when we compared the subgroup of studies in which the informant(s) provided both comprehensiveness and outcome data against the subgroup of studies in which either the informants who provided comprehensiveness data are different from informants who provided outcome data, or the same informant(s) provided comprehensiveness and outcome data at different times. We found mean correlations of r = 0.22 (p ⩽ 0.001; k = 38) and r = 0.24 (p ⩽ 0.001; k = 4), respectively. Overall, we can conclude from this set of robustness analyses that, in the subset of primary studies using a survey-survey scheme, a bias stemming from common source did not influence the effects of strategic decision comprehensiveness.
Discussion
Although substantial theoretical and empirical efforts have been invested in better understanding, the strategic implications of decision comprehensiveness, long-standing disagreements between proponents and critics of modern rational choice theory persist. As such, our work sheds the most definitive light possible on inconsistencies found in past research. We emphasize the major findings from our analyses below and then discuss their implications for research and practice.
Main effects of comprehensiveness on outcomes
Our cumulative results suggest that the positive main effects of comprehensiveness reported in the literature are largely driven by one methodological choice that researchers make early in the study design stage: how to collect the outcome data. Indeed, the investigation of results across all environments and other study designs (e.g. Orlitzky et al., 2003) show that while the mean correlation between comprehensiveness and outcomes is substantial when informants are asked to rate the outcome variable via simple scale items (i.e. subjective outcome data), it is not significantly different from zero when researchers collect the outcome data themselves using archival sources (i.e. objective outcome data). There are two possible interpretations for this result (cf. Miller and Cardinal, 1994).
The first interpretation is that substantial and positive findings in the subjective outcome category are the result of meaningful outcome data being generated when informants are given the freedom to genuinely express themselves. In general, key informants in strategy-process research have a good sense of how their organizations are doing. Archival outcome data, on the contrary, may reflect a host of reporting and timing issues. They are subject to subtle manipulations, such as those related to earnings management (Ettredge et al., 2010), as well as more egregious ones, such as those associated with fraud (e.g. Enron and Worldcom scandals). Bias and error generated by purveyors of commercial archival databases are also a concern (e.g. Lara et al., 2006). Finally, and regardless of data quality, concerns remain because some archival sources may provide data that are too far removed from the predictor and/or too abstract to be relevant success/failure indicators of strategic decision-making processes.
The second interpretation is that the correlations of the studies in the subjective outcome category are likely inflated. That is, in this category, statistically significant findings may be largely driven by methodological artifacts such as common method bias (e.g. Podsakoff et al., 2003). Such bias can result when one method (e.g. self-report surveys) is used to assess both the independent and dependent variables. Indeed, common methods create fertile ground for informant biases such as social desirability, acquiescence, and other biasing factors (Spector, 2006). Moreover, informants in various studies may consciously or subconsciously provide responses to outcome measures that match responses to strategic-process measures. For example, an informant may assume that successful managers systematically collect and analyze large amounts of information when making important decisions, whereas poor managers are not analytical in their approach to decision making. In other words, informants may equate comprehensiveness with positive organizational outcomes (Dean and Sharfman, 1996; Feldman and March, 1981; Miller and Cardinal, 1994).
The moderately objective outcome category provides insights into which of the above interpretations is likely more valid. This subgroup contains studies in which researchers assessed the outcome variable through informants who reported specific outcome information on surveys. Therefore, if the use of archival outcome data was responsible for the non-findings observed in the objective outcome subgroup (i.e. the first assumption above), the moderately objective outcome category—in which surveys are used to assess both comprehensiveness and outcomes—would have shown significant results, similar to the subjective outcome subgroup.
Because the mean correlations in the objective and moderately objective outcome category are not significantly different from zero, we ruled out issues associated with the use of archival outcome data in primary studies as a source of our nonfindings. Therefore, the second interpretation above seems to be the most valid. That is, the statistically significant findings of comprehensiveness’ effects are likely the results of methodological artifacts. However, because studies in the moderately objective outcome category are also based on survey data for both predictor and criterion variables (i.e. a survey-survey design), common method bias is likely not the culprit for the inflated correlations (cf. Spector et al., 2019). Indeed, if common method bias were to blame, higher correlations would have been observed for both subjective and moderately objective outcome categories. Instead, the results show that the mean correlation for the moderately objective outcome category is also not significantly different from zero, similar to that of the objective outcome category. Thus, other forms of methodological artifacts, such as anchoring bias and availability heuristic—examples among many other types of biases—may have driven the positive correlations reported in the literature between comprehensiveness and organizational outcomes.
Moderating effects of dynamism
Our findings also challenge the generally held belief that the level of dynamism in the environment moderates the relationships between comprehensiveness and outcomes. In our data, the mean correlation is essentially zero when studies use samples of either stable or dynamic environments only. In fact, the comprehensiveness–outcomes relationship is significant and positive only when the sample of primary studies includes both environments. Thus, our results support neither side of the debate on whether dynamism enhances or diminishes the effects of comprehensiveness. We consider three possible explanations for this surprising finding.
First, non-significant moderating effects of dynamism could stem from an effect size that is too small to detect. However, our meta-analytic approach to addressing our research question likely rules out this explanation. Indeed, the total sample sizes for the dynamic and stable environments subgroups are N = 1718 and N = 490 firms, respectively. Because the sample sizes are fairly large, even very small effects of comprehensiveness would likely have been detected.
The second possible explanation is that theoretical assumptions about moderating effects of dynamism have been wrong all along. Indeed, a causal connection between dynamism and comprehensiveness may exist. For example, dynamism may be positively related to comprehensiveness (e.g. Meissner and Wulf, 2014). Should this be the case, the range of comprehensiveness would also be restricted in a sample of firms facing only stable or only dynamic contexts (e.g. Fredrickson, 1984; Fredrickson and Mitchell, 1984). In turn, range restriction of comprehensiveness would reduce its observed relationship with the outcome variable (e.g. Hunter and Schmidt, 2004). If this explanation is valid, comprehensiveness is endogenous to dynamism (i.e. the level of comprehensiveness varies with the level of dynamism and comprehensiveness essentially functions as a mediator of the dynamism–outcomes relationship; cf. Meissner and Wulf, 2014), and theoretical models of comprehensiveness need to be substantially revised. Moreover, studies such as those incorporating moderated-mediation models (Hayes, 2013; Preacher et al., 2007) will be needed to draw rich conclusions regarding the role of dynamism in the comprehensiveness–outcomes relationship.
The third explanation relates to the conceptualizations and measurements of dynamism in the literature. Because the categories of our dynamism variable were created from the descriptions of samples used in the primary studies, we had to assume that dynamism was reasonably explained, assessed, and/or reported in previous work. This assumption, however, may not hold for a number of reasons. From a theoretical standpoint, the simplistic, traditional characterization of dynamism is likely problematic because the external environment is a very broad and complex entity (e.g. Forbes, 2007; Thompson, 1960). Also, researchers’ conceptualizations of the environment as an external reality may significantly differ from the way managers understand it (Forbes, 2007; Meissner and Wulf, 2014). Thus, a more fine-grained conceptualization of dynamism, perhaps in terms of clarity and quantity of the information in a task environment (cf. Forbes, 2007), may be more helpful. Regarding the measurement of dynamism, construct validity may be an issue. Indeed, the level of dynamism is often deduced from extant knowledge or perceptions related to the nature of the industries represented in a sample. For example, a number of authors have argued that some firms (e.g. high-tech, pharmaceutical companies) are known to operate in dynamic environments (e.g. Covin et al., 2001; Fredrickson and Mitchell, 1984). Other scholars have held some firms (e.g. paints and coatings firms) do business in stable environments (e.g. Fredrickson, 1984). Yet, it is possible that not all high-tech firms experience high levels of dynamism, in the same way that not all paint product companies operate in stable environments. Finally, and as mentioned above, the research practice involving range restriction of dynamism to create samples of firms operating in dynamic or stable environments only may be problematic. In sum, our results show that dynamism, based on its current conceptualizations and operationalizations, neither strengthens nor weakens the comprehensiveness–outcomes relationship. Rather, the controversy surrounding moderating effects of dynamism is caused by methodological artifacts.
Research implications and future directions
In this work, we synthesize the available empirical evidence on the strategic implications of decision comprehensiveness. Our meta-analytic review of this literature has challenged conventional wisdom and generated novel insights. Sound methods and robust results aside, we note several limitations that point to new directions for future theoretical and empirical research.
A first important consideration from our work is that the common belief about positive organizational effects of strategic decision comprehensiveness was empirically supported when perceptual outcome measures were used. Importantly, our analyses suggest that common method bias does not explain the large disparity between the mean correlations observed for objective versus subjective outcome measures. Rather, other methodological factors are likely responsible. However, we cannot conclusively pinpoint those factors. Indeed, data on these potential methodological artifacts were either unavailable or insufficient for inclusion in our analyses. Therefore, studies designed to include both objective and subjective outcome assessments would be very useful in better understanding the strategic value of decision comprehensiveness.
Furthermore, a number of variables that have been proposed to account for the variance in existing studies do not appear to play important roles. Indeed, while we ultimately found non-significant moderating effects by the proximity and lag between predictor and criterion variables and by the focus of comprehensiveness measures, these factors have been highlighted as possible moderators in previous research (see Elbanna, 2006; Forbes, 2007; Shepherd and Rudd, 2014). Despite the nonfindings of moderating effects for these variables, our MARA model explains a substantial amount of between-study variance. It also suggests that other factors may be at play because 40% of the variance is still unexplained. To identify these factors, researchers may need to adopt longitudinal designs. However, despite several literature reviews (e.g. Elbanna, 2006; Papadakis and Barwise, 1998; Shepherd and Rudd, 2014) and prominent studies (e.g. Elbanna and Child, 2007a) calling for longitudinal designs in investigating the strategic value of comprehensiveness, all studies in our sample relied on cross-sectional analyses, except for Miller’s (1993).
Beyond a methodological explanation for our results, we build on the view that strategic decision comprehensiveness provides symbolic value for organizations (cf. Langley, 1989), which can be captured by managers’ accounts of organizational performance. Because institutional pressures for rationality are very strong (Feldman and March, 1981), managers may use rational processes in order to maintain legitimacy and guard against criticism in case mistakes are made (Feldman and March, 1981; Langley, 1989, 1995; Meyer and Rowan, 1977). This interpretation suggests that comprehensiveness is beneficial for firms in terms of symbolic, instead of substantive value—as captured by organizational performance measured using metrics such as accounting returns, growth, and stock market performance indicators (c.f. Combs et al., 2005; March and Sutton, 1997). We emphasize that this possible explanation neither suggests nor implies that perceptual performance data are inherently flawed or invalid outcome measures. When adequately used (cf. Aguinis et al., 2018; Bono and McNamara, 2011, etc.), perceptual performance data obtained from key respondents provide two major advantages over other measurement strategies: (1) preceptual performance data capture both financial and non-financial dimensions of performance, allowing for a more comprehensive assessment of strategic value, and (2) as mentioned earlier, managers have a good sense of how their organizations are doing. Thus, future comprehensiveness studies using perceptual outcome data should provide a reasonable justifications for such use and report evidence of construct validity (Conway and Lance, 2010).
A second important take-away is that dynamism is likely not a substantive moderator of comprehensiveness’ effects. Rather, moderating roles of dynamism found in previous studies likely are methodological in nature and stem from restricting the range of the dynamism variable. We emphasize that because our analyses were conducted at the study level, interpretations of these surprising results at the firm level should be made with caution. That said, we put forward that our evaluation of dynamism as a moderating variable is reasonable. Indeed, our approaches both to conceptualizing dynamism and to conducting our analyses are mainstream, our inter-rater reliability in coding for dynamism is strong, and our robustness tests support our findings. Therefore, we urge future researchers to consider other variables in testing for potent moderators of comprehensiveness’ effects. To start, past research has highlighted several boundary conditions as potentially important moderators of the comprehensiveness–outcomes relationship. These boundary conditions are described as decision (e.g. magnitude; urgency), managerial (e.g. CEO tenure or education; top management team cognitive or demographic diversity), organizational (e.g. past performance; slack), and environmental factors (e.g. complexity; hostility) that may substantially influence the effectiveness of comprehensiveness (e.g. Elbanna and Child, 2007a; Papadakis et al., 1998; Shepherd and Rudd, 2014).
Relatedly, an important methodological concern, which we could not address in our analyses, is endogeneity due to omitted variables (Semadeni et al., 2014). Comprehensiveness and its outcomes may be jointly related to some unobserved variables that are often excluded from existing empirical or theoretical models. For example, a parallel research stream on the antecedents of comprehensiveness considers the potential moderators highlighted above as factors that influence the use of comprehensiveness in the first place (Elbanna and Child, 2007b; Miller et al., 1998; Papadakis et al., 1998). However, in the much larger and more prominent research stream focused on the outcomes of comprehensiveness, these factors are often excluded. This situation has likely created endogeneity concerns. Stated differently, it appears that the evolution of two parallel strategy research streams, both focused on comprehensiveness (one on its antecedents, and the other on its outcomes), has led to “endogenous theorizing” in the latter research stream (see Antonakis, 2017). Because our sample did not include any study that addressed endogeneity concerns, we call for future research that simultaneously models the dynamics between some of the factors highlighted above, comprehensiveness, and outcomes. Indeed, it is possible that one or more of these factors discourages the use of comprehensiveness and, at the same time, increases its effectiveness—thus creating paradoxical effects of comprehensiveness (cf. Sitkin et al., 2011).
As a third important take-away, we note that meta-analytic procedures cannot provide insights that other techniques can. For example, we studied, in isolation, one dimension of the strategic decision-making process (i.e. comprehensiveness). While this practice is common, as prior literature reviews indicate (Elbanna, 2006; Hutzschenreuter and Kleindienst, 2006; Shepherd and Rudd, 2014), a few studies have shown that the strategic decision-making process is multidimensional (e.g. Dean and Sharfman, 1996; Hart and Banbury, 1994). Thus, there likely is more to the story than what our work can tell. To provide a more complete picture, future strategy research should simultaneously consider how multiple dimensions of the strategic decision-making process (e.g. rational, intuitive and political) shape firm and/or decision outcomes. For example, a recent qualitative study (Calabretta et al., 2017) drew on the paradox perspective to offer critical insights into how the coexistence of comprehensiveness and intuition can improve firm performance. Also, while comprehensiveness is influenced by historical events, empirical work has not incorporated its “temporal dynamics” (Langley, 2007). The same holds true for dynamism, which prior studies have often consided a fixed contingency (Royer and Langley, 2008). Therefore, we encourage the use of narrative, ethnographic, longitudinal and qualitative studies, which would frame comprehensiveness as a dynamic process and examine how it is shaped and reshaped over time.
Our final take-away involves the implications of our study for “open strategy” (Whittington et al., 2011) and “big data analytics” (e.g. McAfee and Brynjolfsson, 2012) research. As an emergent stream of strategy practice research, open strategy holds that organizations improve strategic decisions by including more input from internal and external stakeholders in the strategic decision-making process (Hautz et al., 2017). However, our findings imply that having access to more information does not necessarily translate into better decisions. Instead, the extent to which available information is substantively integrated into decision-making processes likely influences organizational outcomes (e.g. Samba et al., 2018; van Knippenberg et al., 2004). Relatedly, recent advances in the Business Analytics landscape have enabled firms to continuously collect and analyze large volumes of unstructured data from various sources (Janssen et al., 2017; Tabesh et al., 2019). These new capabilities enable organizations to include substantial amounts of diverse information elements into strategic decision-making processes. For example, large volumes of crowd-based data generated by stakeholders (e.g. customer tweets or online reviews) can now be quickly analyzed using advanced Natural Language Processing (NLP) algorithms. Useful insights from this data can, in turn, be leveraged during decision-making activities (e.g., George et al., 2014; Zeng and Glaister, 2018).
What can contributors to these emergent research streams learn from our study? First, building robust, theory-driven foundations for empirical research are crucial at the outset of research efforts (Colquitt and Zapata-Phelan, 2007). Early comprehensiveness research lacked theory building, and little has changed since Eisenhardt and Zbaracki (1992) explained that despite various central ideas of bounded rationality, a coherent theory of it had yet to emerge. Second, the range of key variables should not be restricted. As mentioned earlier, some researchers restricted the range of the dynamism variable in assessing the strategic value of comprehensiveness, despite the long-held belief that dynamism influences decision-making processes. This research practice, based on beliefs that comprehensiveness matters most in dynamic (or stable) environments, was adopted in more recent empirical efforts. For example, Souitaris and Maestro (2010) and Walter et al. (2008) included firms operating in dynamic settings only, while Thywissen et al. (2018) studied firms in stable environments only. Because restricting the range of one variable (e.g. dynamism) may result in range restriction of a key variable (e.g. comprehensiveness), we encourage future researchers to be mindful of unintentional range restriction of key variables and increase caution during research design. Finally, although advanced predictive analytics can help managers better predict complex and uncertain future states (George et al., 2014), contributors to research on the strategic implications of big data have a duty to remind readers that “unhealthy obsession with numbers” and “paralysis by analysis” remain relevant concerns (Langley, 1995).
Implications for practicing managers
Comprehensive approaches to strategic decision making are popular in many business firms. However, our findings regarding the comprehensiveness–outcomes relationship suggest that while it provides some benefits, comprehensiveness may have been oversold. After accounting for methodological artifacts, we show that it provides little value for organizations in terms of organizational performance. Thus, managers should understand that comprehensiveness is likely not the sought-after panacea. Nevertheless, our results should not discourage 21st century managers from using comprehensive processes in making strategic decisions. Indeed, the recent confluence of big data and advanced predictive analytics has changed the nature of comprehensiveness, along with the informational environment. For instance, what were considered difficult-to-understand environments several decades ago can now be both richly and quickly analyzed, making comprehensiveness even more valuable. However, these recent advances have also created a conundrum for organizations. On one hand, managers are now equipped with sophisticated tools that help them collect and analyze enormous amounts of data. On the other hand, these activities necessitate more efforts from managers, who need to transform data-driven insights into effective decisions, strategies, and actions (Tabesh et al., 2019; Zeng and Glaister, 2018). Stated differently, the sheer volume and velocity of available information in the digital era may significantly add to executive overload (cf. Hambrick et al., 2005).
Conclusion
Our article provides two definitive insights: (1) comprehensiveness and outcomes are positively related only when subjective outcome measures are used, and (2) dynamism does not play the moderating role that popular theorizing suggests. Based on these insights, we provide specific directions for future research. To better understand the strategic value of comprehensiveness, studies that are focused on the validity of outcome measures are needed, and sooner rather than later. To draw firm conclusions regarding any role for dynamism, should model the dynamics among contextual variables, comprehensiveness, and outcomes. Absent these new studies, the comprehensiveness–outcome domain will remain very difficult to understand.
Supplemental Material
SO-18-0090.R2_b-Appendix – Supplemental material for Method in the madness? A meta-analysis on the strategic implications of decision comprehensiveness
Supplemental material, SO-18-0090.R2_b-Appendix for Method in the madness? A meta-analysis on the strategic implications of decision comprehensiveness by Codou Samba, Pooya Tabesh, Ioannis C Thanos and Vassilis M Papadakis in Strategic Organization
Footnotes
Acknowledgements
We acknowledge the very helpful input of Strategic Organization Editor Ann Langley and four anonymous reviewers. We also recognize with deep gratitude the valuable insights and inputs of Russel Crook, David Ketchen, and Tim Pollock during this and/or previous revision rounds of this manuscript.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental material
Supplemental material for this article is available online.
Author biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
