Abstract
Construct measurement is a cornerstone for research in any area. In the absence of information on the caliber of measurement practices within and across studies, it is extremely difficult to synthesize research and develop normative guidelines for managers. This Feature Topic aims to advance measurement practices by strategic management scholars. The Feature Topic includes six articles from a diverse array of perspectives. We describe the context of strategic management and measurement, examine how the Feature Topic articles address critical issues, and conclude with advice for authors and reviewers on this topic.
Strategic management is considered to be one of the younger fields within the management discipline. Given strategic management’s relative youth, it is not surprising that the field has often been criticized for a lack of methodological rigor. Construct measurement stands out as an aspect of methods that would benefit from an Organizational Research Methods Feature Topic, as to date, relatively limited emphasis has been placed on measurement issues within strategic management. Therefore, we are pleased to introduce the Feature Topic on “Construct Measurement in Strategic Management.” We begin with a brief background on the role and importance of construct measurement, followed by a history of the coverage of construct measurement in strategic management research. Next, we introduce each of the articles appearing in the feature topic. The articles include commentaries and original analyses, written from both quantitative and qualitative perspectives, covering an array of prominent topics. Finally, we conclude with an agenda for future studies as well as practical suggestions for authors and reviewers. Overall, we hope this Feature Topic will encourage strategic management researchers to thoughtfully nurture the measurement of key constructs as a means of advancing the strategic management domain as an important aspect of management research.
Trends in Strategic Management Construct Measurement
Nearly 30 years ago, Venkatraman and Grant (1986) presented a then state-of-the-art assessment of construct measurement in strategic management. They began by noting several factors that underscore the importance of sound measurement: a growing trend toward the use of large-scale empirical models, the potential for measurement error to affect interpretation of results, and dangers in making normative recommendations based on weak measures. Overall, Venkatraman and Grant were critical regarding most aspects of construct measurement in strategic management studies. In particular, they highlighted the following concerns: reliance on categorical variables, use of single indicators, inadequate assessment of reliability, and ambiguity regarding levels of measurement. Similar assessments have been made in subsequent reviews and commentaries (e.g., Hitt, Boyd, & Li, 2004; Ketchen, Boyd, & Bergh, 2008; Montgomery, Wernerfelt, & Balakrishnan, 1989; Snow & Thomas, 1994). In fairness to strategic management researchers, similar problems can be found elsewhere in the management domain. For example, a content analysis of articles published in multiple journals between 1991 and 1992 revealed widespread use of single indicators and failures to routinely report reliability data, even when presenting new measures (Schriesheim, Powers, Scandura, Gardiner, & Lankau, 1993). Similarly, recent entrepreneurship research also has a similar pattern of single indicators and insufficient attention to reliability and validity (Crook, Shook, Morris, & Madden, 2010).
How well has strategic management progressed since Venkatraman and Grant’s (1986) critique? Fortunately, evidence has been developed that allows us to consider an answer to this question. Boyd, Gove, and Hitt (2005b), for example, conducted a content analysis of roughly 200 articles published in Strategic Management Journal between 1998 and 2000. Their findings revealed minimal progress on two of the key concerns raised by Venkatraman and Grant. First, Boyd and colleagues reported that the vast majority of variables—regardless whether they were independent, outcome, or control variables—were measured with single indicators. Of equal concern, the majority of articles utilizing multiple measures failed to report reliability data. Additionally, these studies typically reported either no or partial correlation matrices, making it impossible for readers to assess reliability directly. In a companion article (i.e., Boyd et al., 2005a), they empirically demonstrated how imprecise measurement of strategic management constructs can lead to attenuation and Type II errors, reinforcing Venkatraman and Grant’s concerns about the foundation for normative advice. As argued by Ghoshal (2006), practical relevance is the raison d’etre of management research, making this a very important concern.
Since Boyd et al. (2005b) is based on articles published 10 years ago, it is an open question whether measurement practices have changed in the past decade. Consequently, we replicated portions of their analysis drawing on more recent data. Specifically, we selected articles from several 2010 issues of Strategic Management Journal. Our sample represented over half of the empirical articles published that year. We collected data for three elements of Boyd et al. (2005b): use of single versus multiple measures, reporting of reliability data, and magnitude of reliabilities. Results are shown in Table 1.
An Update of the Boyd, Gove, and Hitt (2005b) Content Analysis.
Note: SMJ = Strategic Management Journal; IV = independent variable; DV = dependent variable.
Overall, results show both progress and lost ground. For the 1998-2000 studies, single indicators were the dominant approach to measuring constructs, although dependent variables were more likely than predictors or controls to be measured with multiple measures. For 2010, the proportion of single indicator measures has grown moderately for independent variables and substantially for dependent variables. Possible explanations for this finding could be a shift in the type of constructs being studied or possibly an artifact of the differing time windows of the two comparisons. Still, it does appear as if little progress has been made in this regard.
In contrast, results for reliabilities are far more encouraging. For the 1998-2000 sample, fewer than half of constructs for which reliability estimates could be calculated actually reported this information in the article. The findings for 2010 are completely reversed, where the majority of possible reliability estimates are now presented in the article. Additionally, the magnitude of reported reliabilities has grown for independent, dependent, and control variables over time. In summary, only limited progress has been made on the measurement of strategic management constructs since Venkatraman and Grant’s (1986) critique.
Role and Importance of Construct Measurement
Why is construct measurement a concern for strategic management researchers, and what actions can improve the state of affairs? Figure 1 lays out a roadmap of key issues. Shown in the upper portion of the figure is Blalock’s (1979) framework for studying social processes. This framework has three components: First is a statement of relations proposing a causal order between constructs, represented by the path from Construct A to Construct B. This component takes place at the theoretical level and, as such, is not tested directly. Second, there is the operational link between constructs, which is performed at the indicator level and represented by the path from Indicator X to Indicator Y. This is the actual test of a given hypothesis. The third component is the relationship between constructs and indicators. Traditionally, indicators are assumed to be driven by the construct, known as reflective indicators. A counterargument is that the indicators shape the construct, known as causal or formative indicators. We discuss the implications of this distinction in more detail subsequently.
Key concepts in the development of constructs. Note: The upper portion of the figure is adapted from Blalock (1979), while the construct validity text balloon is from Venkatraman and Grant (1986).
As noted previously, strategic management research has largely assumed a one-to-one correspondence between constructs and indicators, as the overwhelming majority of models rely on single indicators and provide limited tests of reliability or validity. Although many studies rely on multiple measures of performance, most constructs are not assessed via multiple measures. As a result, there is the potential that strategic management studies have consistently underestimated the true nature of relationships between many constructs (e.g., Boyd, 1991; Boyd et al., 2005b). Additionally, other analyses, such as power tests, can also be adversely affected by measurement problems (Cohen, 1987) as well as the testing and interpreting of change and other difference scores (Bergh & Fairbank, 2002). Concerns about measurement are especially relevant for macro researchers, as they often study broad constructs. For example, Godfrey and Hill (1995) compared key research streams—agency, transaction cost economics, and the resource-based view—to subatomic particles, in that “instrumentation cannot be calibrated to such a degree as to permit their direct observation” (p. 519).
In lieu of perfect measurement, it becomes critical to understand just how closely our indicators track with unobserved constructs. The shaded box in Figure 1 identifies six key components that affect the validity of constructs (Venkatraman & Grant, 1986). Content validity, which asks how closely a chosen indicator meshes with the theoretical framing of a construct, is the first element. The second and third items relate to internal consistency and are dimensionality of measures and their reliability. The fourth element is convergent validity, which examines whether different measures of the same construct show similar patterns of behavior. Fifth, discriminant validity examines how well a construct can be considered distinct from other constructs. Finally, nomological, or predictive, validity assesses whether measures in fact have a link to dependent variables as specified by theory.
A danger for strategic management studies lies in the interpretation of results. Even if a manuscript reports a substantial effect size between Indicator X and Indicator Y, confidently drawing implications for the underlying theories is impossible without assessment of these six components for all variables in a model. Each of the Feature Topic articles sheds light on a different aspect of this process and thereby provides a basis for improved construct measurement. In the following section, we summarize each of the articles included in the Feature Topic.
Articles in the Feature Topic
Our first article is “Seeking Qualitative Rigor in Inductive Research: Notes on the Gioia Methodology,” by Gioia, Corley, and Hamilton. The authors discuss the development of constructs using a qualitative lens. A key premise of the article is that too strong a focus on operationalization of constructs causes researchers to lose sight of the more fundamental—and less measureable—concepts, which are the true foundation of construct development. Next, the authors highlight the challenges of qualitative research: While qualitative studies offer the potential for novel and fresh insights about the nature of constructs, these studies are also subject to the criticism that qualitative research fails to meet the litmus test of scientific rigor. In the remainder of the article, Gioia and colleagues describe a systematic framework for the development of new concepts and the role of such concepts in the development of new theory. This framework has been developed by Gioia and various colleagues across a series of studies published in leading journals such as Academy of Management Journal, Strategic Management Journal, and Administrative Science Quarterly. Our hope is that Gioia et al.’s article will quickly become regarded as a vital resource for qualitative researchers who seek to place their work at the highest levels.
Our second article is “The Use of Archival Proxies in Strategic Management Research: Castles Made of Sand?” by Ketchen, Ireland, and Baker. This commentary highlights a central challenge for many strategic management scholars: On one hand, archival data sources have the utility of being readily available, and they also enable researchers to use identical measures across studies, furthering synthesis of research findings. However, there are also questions regarding how well an archival proxy accurately taps an underlying construct. Thus, the choice when designing a new study may be between using an imperfect measure versus not having a measure to use. In this article, the authors examine construct validity issues associated with archival data sources for three prominent strategic management measures: research and development intensity, patent counts, and patent citations. Drawing on a review of studies in three journals over multiple years, the authors find that these three measures have been used to represent an array of diverse constructs. This situation is disconcerting because in most cases there is no basis for presuming that a measure is capturing the construct that authors intend to capture rather than one or more of the alternative constructs. The article concludes with suggestions for improving the use of archival measures in future studies.
Our third article is “Measuring and Testing Industry Effects in Strategic Management Research: An Update, Assessment, and Demonstration,” by Sharp, Bergh, and Li. This article poses the dual questions “How important is it to test for industry effects?” and “How should industry be measured?” Around two decades ago, Dess, Ireland, and Hitt (1990) noted that relatively few strategic management studies systematically ruled out industry effects as an alternate explanation for hypothesis tests. Sharp and colleagues present three sets of analyses to inform best practices with regard to this issue. First, they replicate the analysis by Dess et al. using more recent studies, providing benchmarks on shifting trends on the norms for assessing industry effects. Second, they conduct a longitudinal content analysis to identify changes in the specific measurement schemes over time. Finally, they use corporate restructuring to illustrate how the measurement of industry effects can alter the results of hypothesis tests. The message for strategic management research is clear: Better assessment of industry effects will create stronger confidence in the findings produced within studies that include the industry level of analysis.
Our fourth article is “Exploring the Dimensionality of Organizational Performance: A Construct Validity Study,” by Hamann, Schiemann, Bellora, and Guenter. This article explores what is arguably the most central construct in strategic management research. There is a rich history of both conceptual and empirical discussions of what constitutes organizational performance. As a result, a wide array of measures has been used in individual studies. Hamann and colleagues trace the development of the organizational performance construct over the past two decades. Next, drawing on a recent three-dimensional model of performance (Combs, Cook, & Shook, 2005), the authors conduct additional analyses, encompassing roughly 20 indicators for nearly 5,000 firms across a 20-year window. Based on these analyses, they identify a set of robust indicators for four dimensions of organizational performance. Our expectation is that the authors’ conclusions will spark a new round of spirited discussion in the literature about how to best measure performance. Are Hamann et al.’s findings the final word on performance assessment, or are adjustments and improvements needed before researchers can be confident that they are fully capturing this vital construct?
Our fifth article is “Dummy Constructs? Binomial Categorical Variables as Representations of Constructs: CEO Duality Through Time,” by Gove and Junkunc. In their work, the authors address the challenges associated with using dichotomous variables to represent complex constructs. While the associated issues are germane to a number of strategic management variables, Gove and Junkunc focus on CEO duality, where the CEO also serves jointly as chairman of the board of directors. They begin with a review of relevant studies that have included duality as a study variable. They also discuss the role of theoretical frameworks and temporal considerations in understanding this phenomenon. Next, they empirically explore issues associated with the measurement of duality, drawing on a longitudinal analysis of 4,582 firm-years of Fortune 500 data. Their analysis concludes that use of a 0/1 representation of board leadership has only limited ability to represent the underlying aspects of this widely used variable. Meanwhile, studies of CEO duality have often generated ambiguous results. One implication is that more precise and nuanced assessment of CEO duality may enable scholars to empirically establish the nomological network that surrounds CEO duality.
Our final article is “Measurement Malaise: The Case of Corporate Governance Research,” by Aguinis and Dalton. This article explores how meta-analysis can be used as the initial phase of a protocol to improve construct validity. To illustrate the benefits of this procedure, the authors integrate two topics that are addressed elsewhere in the Feature Topic: corporate governance and firm performance. A number of governance characteristics—notably, CEO duality, board independence, and director equity—are recommended both by agency theory and by many governance codes and guidelines. However, original studies and meta-analytic reviews report consistently weak relationships between these governance variables and firm performance. The authors then offer and test a five-point protocol for overcoming measurement limitations in corporate governance research, and they report a substantial increase in empirical estimates when their process is used. Their article serves as an “illustration of the increase in explanatory power that can be achieved by improving the construct validity of independent and dependent variables.” This article holds considerable progress for improving both the measurement and the findings for any area within strategic management research that has relied upon coarse-grained measures.
Suggestions for Future Studies
Based on our content analysis and the articles contained in this Feature Topic, we would like to offer some broad suggestions for future studies.
Concepts, Constructs, and Triangulation
Our first avenue for future research extends on themes from several of our Feature Topic articles. As noted by Ketchen and colleagues, many of the central constructs in strategic management research tend to be assessed using archival data. Additionally, the proportion of studies that utilize original data—namely, either surveys or laboratory studies—has declined substantially over time (Boyd, Haynes, Hitt, Bergh, & Ketchen, 2012). Thus, for better or worse, archival data sources figure prominently on the strategic management landscape. We should be concerned about this trend—not only for how well our indicators represent their intended constructs, but also, using the terminology of Gioia and colleagues, for the underlying concepts as well. So, for instance, how well does a 0/1 archival measure of board leadership structure tap the broad themes of resource acquisition, oversight, and the political interplay of a CEO with their board?
Work to date indicates that some archival proxies are more effective than others. Finkelstein (1992), for example, developed a model of executive power comprised of 13 indicators and utilizing multiple data sources. In a supplementary analysis, Finkelstein also reported that the archival measures tracked closely with power as perceived by top managers and also that there was strong interrater reliability among managers in the same firm. Similarly, Hambrick and Abrahamson (1995) found that archival measures of managerial discretion correlated with assessments of both academic experts and industry analysts. Thus, in these two cases, we can infer that archival data are reasonable proxies of the underlying phenomena.
In contrast, there is an extensive literature on environmental uncertainty. Given the weak triangulation between measures based on survey and archival data, it is reasonable to conclude that the two are related but fundamentally different concepts (Boyd, Dess, & Rasheed, 1993). The development of meaningful proxies involves both theorizing and the search for new and better measures (Carpenter & Reilly, 2006). Consequently, one major priority for researchers is to sift through the better and less desirable proxies and identify alternative measures for the latter.
The Link Between Constructs and Indicators
Do constructs drive indicators in strategic management, or vice versa? As shown in Figure 1, there are competing perspectives on the linkage between construct and measure. This is a worthwhile discussion in terms of measurement and also as a basis for further theorizing about the basic nature of strategic management concepts. For example, when the CEO serves jointly as chairman of the board of directors, does this represent an underlying proclivity toward oversight and control? Or, does this practice combine with board size, equity ownership, representation by outsiders, and other factors to determine the level of oversight? These distinctions are very important in specifying theory: A reflective approach assumes the board has a specific intent regarding oversight, while a formative approach infers that oversight is the sum of a specific set of actions.
In a traditional model, the causal direction flows from the construct to indicator, namely, a reflective model. Alternately, though, the indicators may drive the construct, namely, a formative or causal model. While the distinction between these types of measures is not new (e.g., Blalock, 1964; Edwards & Bagozzi, 2000), discussion of this topic within the strategic management community is largely absent. The lack of debate stems from the subfield’s reliance on single indicators (Boyd et al., 2005b), making the cause-effect distinction largely moot.
Formative indicators are discussed only rarely in strategic management studies that rely on multiple measures, even those that utilize structural equation modeling. In a content analysis of strategic management articles published between 1994 and 2003, Podsakoff, Shen, and Podsakoff (2006) made the provocative argument that the outstanding majority of strategic management constructs have been mistakenly depicted as reflective constructs. In contrast, other scholars question the basic premise of formative models (e.g., Edwards, 2011; Howell, Breivik, & Wilcox, 2007). In our own manuscripts, reviewer questions about the causal direction between construct and indicator have helped immeasurably to think more deeply about the basic nature of concepts as well as address more practical considerations such as the number of indicators and their expected interrelationships. Consequently, more attention to this topic within the strategic management community would be beneficial.
Qualitative and Mixed Method Approaches to Study Constructs
Given the wealth of readily available archival data sources, combined with the challenges of extracting original data from top executives, it is not surprising that qualitative designs are seen less frequently in strategic management studies than for some other management subspecialties. However, qualitative methodologies such as case studies and field surveys can be very helpful “in situations where constructs and linkages are poorly understood” (Barr, 2006, p. 181). In particular, Barr (2006) advocated that these approaches are especially relevant for topics such as dynamic capabilities and strategic groups. Separately, a hybrid approach, or a mixed methods design, combines elements of both qualitative and quantitative studies in a single article. While these designs are more challenging to develop and execute, they also offer the potential for greater insights into both constructs and theories. A content analysis of mixed method strategic management studies found that these articles were substantially more influential than the corresponding mono-method papers (Molina-Azorin, 2012). Godfrey and Hill (1995) note that there is a tendency to discount concepts that are difficult to quantify in many strategic management studies. When this is the case, researchers tend to focus on variables that are measurable but not necessarily meaningful. Thus, rather than designing another quantitative study to explore why variables such as R&D spending or board outsiders have a poor predictive track record, it may be more fruitful to embrace qualitative and mixed method approaches in order to develop new alternatives to these measures. We suggest that senior faculty have a particular obligation to advance the field in this way, given that they do not face the pressure of a tenure clock.
Use of North American Survey Instruments in Other Settings
Despite the prevalence of archival data sources, there is still a substantial minority of strategic management articles that collect original data via surveys. Given the international scope of the field, many of these studies are based on samples of non-U.S. respondents. These research settings offer special challenges to the validity of constructs. Back-translation is widely used and is intended to preserve the original intent of a survey item. The measure is translated from English to the new context, and then that measure is translated back to English. The two English versions are then compared for inconsistencies. Research teams often have members representing each of the relevant cultural contexts to facilitate the development of comparable measures.
While these are reasonable precautions, strategic management researchers would also benefit from a closer review of practices in the cross-cultural community. In addition to semantic equivalence, which is addressed by back-translation, there are also issues of conceptual and scale equivalence when using instruments in different cultural settings (Schaffer & Riordan, 2003). However, these latter two issues are addressed much less frequently in strategic management studies. Consequently, a final avenue of opportunity is to reassess the applicability of macro measures to different settings.
Suggestions for Authors, Reviewers, and Editors
Over the course of our careers, each of the guest editors has had the opportunity to view a nontrivial number of manuscripts, from the vantage points of author, reviewer, and editor. We would like to conclude this editor’s introduction with some practical suggestions for addressing construct measurement issues in the review process.
First, when planning your study, remember that strategy is considered to be an early stage discipline in terms of paradigm development (Ketchen et al., 2008). A key characteristic of an early stage field is lack of consensus (Pfeffer, 1993). In practical terms, you should plan to have your measures challenged, even if the source is widely cited. While a strong defense for your chosen measure is helpful, you should also consider collecting backup data so that it will be possible to substitute alternate indicators. This problem is especially important when collecting original data, as it may be difficult or impossible to go back to your original data sources. We have seen multiple cases where a revise and resubmit could not go forward due to an inability to meet requests for additional measures.
Don’t “cherry pick” measures based on results—namely, what Bettis (2012) calls “data snooping.” Whether implicitly or explicitly, reviewers will ask about the validity of your constructs, and you may be hard-pressed to explain why the semilog-transformed, 3.75-year-smoothed, outlier-trimmed version of the dependent variable is the optimal measurement option. In contrast, strong theories combined with good measures should yield robust findings even in the face of small changes in measurement schemes. One encouraging sign in this regard is a supplementary section of many papers sometimes labeled as “robustness checks”—where they examine the sensitivity of hypothesis tests to variations in measurement. Not only does this practice provide utility to readers, but it also conveys to reviewers the rigor of both your theory and methodology.
Also, be mindful of making, or (when serving as a reviewer) requesting, changes to a measure that has been reported previously. On one hand, given the state of construct measurement in strategic management studies, measures in a certain area may be evolving over time. As such, slavish adherence to an early stage measurement scheme will simply institutionalize poor measurement practices. Conversely, however, continual changes in measurement practice for a given construct over time reduce the ability for researchers to synthesize and integrate findings across studies (Schriesheim et al., 1993). For instance, scales used to measure leader-member exchange (LMX) have evolved over time, with measures having been both added and deleted. A comparison of versions concluded that these changes have improved reliability, but at the expense of validity. Additionally, the changes raised questions whether content validity remained stable across versions (Keller & Dansereau, 2001). Consequently, efforts to fine-tune measures can sometimes be counterproductive.
We suggest that reviewers might balance these opposing views on construct measure development by recognizing that methodological practices can become institutionalized and repeated without critical scrutiny and by applying careful psychometric principles to their evaluative processes. For example, for many years, change has been measured as a simple difference between two measures of the same construct. However, research on the methodological viability of difference scores have demonstrated that they tend to be especially vulnerable to error (Edwards, 1994), and as such, their reliabilities can fall to unacceptable levels. For example, Bergh and Fairbank (2002, p. 360) reported that the reliability of a change variable is .4 when the reliability of its two observed measures is .7 and they have a correlation of .5. The reliability level of the change variable improves modestly to .6 when the reliability of the two observation points increases to .8. However, if the correlation among the two variables used to compute the change variable rises above .5, then the reliability of the change variable drops quickly and reaches 0 in measurement settings that are not uncommon in strategic management. Overall, we recommend that considering the applied development of measurement practices in combination with the psychometric properties of measurement can be used by reviewers to guide manuscript development in a way that is consistent with best scientific practices and contributes to the knowledge development process.
Conclusion
Construct measurement is a critical concern for strategic management researchers. In the words of one of the Feature Topic articles, research built on a shaky foundation have the potential to be nothing more than “castles made of sand.” Fortunately, macro scholars have placed growing emphasis on research methodology in more recent years—both in general and more specifically to construct measurement well. These trends are underscored by a greater presence of macro papers here at ORM, as well as more macro presenters in the Center for the Advancement of Research Methodology and Analysis (CARMA) methods webcasts. Strategic Management Journal also has a small yet influential stream of papers on methodology, and the Research Methodology in Strategy and Management book series has a number of chapters that address measurement issues. We hope that the articles included in this Feature Topic will stimulate further interest in this area.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.