Understanding the Impact of Convergent Validity on Research Results

Abstract

Using different measures of constructs in research to develop robust evidence of relationships and effects is seen as good methodological practice. This assumes these measures possess high convergent validity. However, proxies—alternative measures of the same construct—are rarely perfectly convergent. Although some convergence is preferred to none, this study demonstrates that even modest departures from perfect convergent validity can result in substantial differences in the magnitudes of findings, creating challenges for the accumulation and interpretation of research. Using data from published research, the authors find that substantial differences in findings between studies using desired and proxy variables occur even at levels of convergent validity as high as r = .85. Implications of using measures with less-than-ideal convergent validity for the interpretability of research results are examined. Convergent validities above r = .70 are recommended, whereas those below r = .50 should be avoided. Researchers are encouraged to develop and report convergent validity data.

Keywords

field research methods research design measurement design content validity reliability and validity construct validation procedures

Get full access to this article

View all access options for this article.

References

Acquaah

(2007). Managerial social capital, strategic orientation and organizational performance in an emerging economy. Strategic Management Journal, 28, 1235–1255.

Bakker

A. B.

Demerouti

Dollard

M. F.

(2008). How job demands affect partners' experience of exhaustion: Integrating work–family conflict and cross-over theory. Journal of Applied Psychology, 93, 901–911.

Bedeian

A. G.

Sturman

M. C.

Streiner

D. L.

(2009). Decimal dust, significant digits, and the search for stars. Organizational Research Methods, 12, 687–694.

Bercovitz

Mitchell

(2007). When is more better? The impact of business scale and scope on long-term business survival, while controlling for profitability. Strategic Management Journal, 28, 61–79.

Bommer

W. H.

Dierdorff

E. C.

Rubin

R. S.

(2007). Does prevalence mitigate relevance? The moderating effect of group-level OCB on employee performance. Academy of Management Journal, 50, 1481–1494.

Bommer

W. H.

Johnson

J. L.

Rich

G. A.

Podsakoff

P. M.

MacKenzie

S. B.

(1995). One the interchangeability of objective and subjective measures of employee performance: A meta-analysis. Personnel Psychology, 48, 587–605.

Bowling

N. A.

Hendricks

E. A.

Wagner

S. H.

(2008). Positive and negative affectivity and facet satisfaction: A meta-analysis. Journal of Business Psychology, 23, 115–125.

Boyd

B. K.

Gove

Hitt

M. A.

(2005). Consequences of measurement problems in strategic management research: The case of Ahmihud and Lev. Strategic Management Journal, 26, 367–375.

Budden

Hadavas

Hoffman

Pretz

(2007). Generating valid 4 × 4 correlation matrixes. Applied Mathematics E-notes, 7, 53–59.

10.

Campbell

D. T.

Fiske

D. W.

(1959). Convergent and discriminant validation by the multitrait–multimethod matrix. Psychological Bulletin, 56, 81–105.

11.

Carlson

K. D.

F. X.

(2011). Citing and building on meta-analytic findings: A review and recommendations. Organizational Research Methods, 14, 696–717.

12.

Chen

W. R.

Miller

K. D.

(2007). Situational and institutional determinants of firms' R&D search intensity. Strategic Management Journal, 28, 369–381.

13.

Combs

J. G.

Crook

T. R.

Shook

(2005). The dimensionality of organizational performance and its implications for strategic management research. In Ketchen

Bergh

(Eds.), Research methodology in strategy and management (pp. 259–286). San Diego, CA: Elsevier.

14.

Earle

J. S.

Spicer

Peter

K. S.

(2010). The normalization of deviant organizational practices: Wage arrears in Russia, 1991-1998. Academy of Management Journal, 53, 218–237.

15.

Edwards

J. R.

Bagozzi

R. P.

(2000). On the nature and relationship between constructs and measures. Psychological Methods, 5, 155–174.

16.

Fey

C. F.

Furu

(2008). Top management incentive compensation and knowledge sharing in multinational corporations. Strategic Management Journal, 29, 1301–1324.

17.

Funder

D. C.

West

S. G.

(1993). Consensus, self-other agreement, and accuracy in personality judgment. Journal of Personality and Social Psychology, 61, 457–476.

18.

Glass

Collins

(1970). Geometric proof of the restriction on the possible values of r_xy when r_yz and r_yz are fixed. Educational and Psychological Measurement, 30, 37–39.

19.

Gong

Law

K. S.

Chang

Xin

K. R.

(2009). Human resource management and firm performance: The differential role of managerial affective and continuance commitment. Journal of Applied Psychology, 94, 263–275.

20.

Guler

Guillen

M. F.

(2010). Home country networks and foreign expansion: Evidence from the venture capital industry. Academy of Management Journal, 53, 390–410.

21.

Guthrie

J. P.

(2001). High-involvement work practices, turnover and productivity: Evidence from New Zealand. Academy of Management Journal, 44, 180–190.

22.

Hunter

J. E.

Schmidt

F. L.

(2004). . Methods of meta-analysis: Correcting error and bias in research findings. Thousand Oaks, CA: Sage.

23.

Jackson

D. N.

(1976). . Personality inventory manual. Goshen, NY: Research Psychologist Press.

24.

Judge

T. A.

Jackson

C. L.

Shaw

J. C.

Scott

B. A.

Rich

B. L.

(2007). Self-efficacy and work-related performance: The integral role of individual differences. Journal of Applied Psychology, 92, 107–127.

25.

Kale

Singh

(2007). Building firm capabilities through learning: The role of the alliance learning process in alliance capability and firm-level alliance success. Strategic Management Journal, 28, 981–1000.

26.

Kristof-Brown

A. L.

Zimmerman

R. D.

Johnson

E. C.

(2005). Consequences of individual’s fit at work: A meta-analysis of person–job, person–organization, person–group, and person–supervisor fit. Personnel Psychology, 58, 281–342.

27.

Larraza-Kintana

Wiseman

R. M.

Gomez-Mejia

L. R.

Welbourne

T. M.

(2007). Disentangling compensation and employment risks using the behavioral agency model. Strategic Management Journal, 28, 1001–1019

28.

Lee

G. K.

(2007). The significance of network resources in the race to enter emerging product markets: The convergence of telephony communications and computer networking, 1989–2001. Strategic Management Journal, 28, 17–37.

29.

Maurer

T. J.

Andrews

K. D.

(2000). Traditional, Likert, and simplified measures of self-efficacy. Educational and Psychological Measurement, 60, 965–973.

30.

Maurer

T. J.

Pierce

H. R.

(1998). A comparison of Likert scale and traditional measures of self-efficacy. Journal of Applied Psychology, 83, 324–329.

31.

McDonald

M. L.

Westphal

J. D.

(2010). A little help here? Board control, CEO identification with the corporate elite, and strategic help provided to CEOs at other firms. Academy of Management Journal, 53, 343–370.

32.

Meehl

P. E.

(1990). Why summaries of research on psychological theories are often uninterpretable [Monograph]. Psychological Reports, 66, 195–244.

33.

Miner

J. B.

Raju

N. S.

(2004). Risk propensity differences between managers and entrepreneurs and between low and high growth entrepreneurs: A reply in a more conservative vein. Journal of Applied Psychology, 89, 3–13.

34.

Murphy

K. R.

DeShon

(2000). Interrater correlations do not estimate the reliability of job performance ratings. Personnel Psychology, 53, 873–900.

35.

Novick

M. R.

(1966). The axioms and principal results of classical test theory. Journal of Mathematical Psychology, 3, 1–18.

36.

Nunnally

J. C.

(1967). Psychometric theory. New York, NY: McGraw-Hill.

37.

Podsakoff

N. P.

Whiting

S. W.

Podsakoff

P. M.

Blume

B. D.

(2009). Individual- and organizational-level consequences of organizational citizenship behaviors: A meta-analysis. Journal of Applied Psychology, 94, 122–141.

38.

Scandura

T. A.

Williams

E. A.

(2000). Research methodology in management: Current practice, trends and implications for future research. Academy of Management Journal, 43, 1248–1264.

39.

Schwab

(1980). Construct validity in organizational behavior. Research in Organizational Behavior, 2, 3–43.

40.

Short

J. C.

Ketchen

D. J.

Jr. Shook

C. L.

Ireland

R. E.

(2010). The concept of opportunity in entrepreneurship research: Past accomplishments and future challenges. Journal of Management, 36, 40–65.

41.

Spangler

W. E.

(1992). Validity of questionnaire and TAT measures of need for achievement: Two meta-analyses. Psychological Bulletin, 112, 140–154.

42.

Stanley

Wang

(1969). Restrictions on the possible values of r₁₂, given r₁₃ and r₂₃ . Educational and Psychological Measurement, 29, 579–581.

43.

Stewart

W. H.

Roth

P. L.

(2001). Risk propensity differences between entrepreneurs and managers: A meta-analytic review. Journal of Applied Psychology, 86, 145–153.

44.

Stewart

W. H.

Roth

P. L.

(2004). Data quality affects meta-analytic conclusions: A response to Miner and Raju (2004) concerning entrepreneurial risk propensity. Journal of Applied Psychology, 89, 14–21.

45.

Toh

S. M.

Morgeson

F. P.

Campion

M. A.

(2008). Human resource configurations: Investigating fit with the organizational context. Journal of Applied Psychology, 93, 864–882.

46.

Traub

R. E.

(1994). Reliability for the social sciences: Theory and applications. (Vol. 3). Thousand Oaks, CA: Sage.

47.

Wall

T. D.

Michie

Patterson

Wood

S. J.

Sheehan

Cleeg

C. W.

West

(2004). On the validity of subjective measures of company performance. Personnel Psychology, 57, 95–118.

48.

Watson

Wiese

Vaidya

Tellegen

(1999). The two general activation systems of affect: Structural findings, evolutionary considerations, and psychobiological evidence. Journal of Personality and Social Psychology, 76, 820–838.