A less evaluative measure of Big Five personality: Comparison of structure and criterion validity

Abstract

Researchers and practitioners have long been concerned about detrimental effects of socially desirable responding on the structure and criterion validity of personality assessments. The current research examined the effect of reducing evaluative item content of a Big Five personality assessment on test structure and criterion validity. We developed a new public domain measure of the Big Five called the Less Evaluative Five Factor Inventory (LEFFI), adapted from the standard 50-item IPIP NEO, and intended to be less evaluative. Participants (n = 3164) then completed standard (IPIP) and neutralized (LEFFI) measures of personality. Criteria were also collected, including academic grades, age, sex, smoking, alcohol consumption, exercise, protesting, religious worship, music preferences, dental hygiene, blood donation, other-rated communication styles, other-rated HEXACO personality, and cognitive ability (ICAR). Evaluativeness of items was reduced in the neutralized measure. Cronbach's alpha and test-retest reliability were maintained. Correlations between the Big Five were reduced in the neutralized measure and criterion validity was similar or slightly reduced in the neutralized measure. The large sample size and use of objective criteria extend past research. The study also contributes to debates about whether the general factor of personality and agreement with socially desirable content reflect substance or bias.

Keywords

Personality social desirability Big Five general factor of personality faking halo

Get full access to this article

View all access options for this article.

References

Allport

G. W.

Odbert

H. S.

(1936). Trait-names: A psycho-lexical study. Psychological Monographs, 47(1), i.

Anderson

N. H.

(1968). Likableness ratings of 555 personality-trait words. Journal of Personality and Social Psychology, 9(3), 272–279. https://doi.org/10.1037/h0025907

Anglim

Lievens

Everton

Grant

S. L.

Marty

(2018). HEXACO personality predicts counterproductive work behavior and organizational citizenship behavior in low-stakes and job applicant contexts. Journal of Research in Personality, 77, 11–20. https://doi.org/10.1016/j.jrp.2018.09.003

Anglim

Morse

De Vries

R. E.

MacCann

Marty

(2017). Comparing job applicants to non-applicants using an item-level bifactor model on the HEXACO personality inventory. European Journal of Personality, 31(6), 669–684. https://doi.org/10.1002/per.2120

Anglim

Morse

Dunlop

P. D.

Minbashian

Marty

(2020). Predicting trait emotional intelligence from HEXACO personality: Domains, facets, and the general factor of personality. Journal of Personality, 88(2), 324–338. https://doi.org/10.1111/jopy.12493

Anusic

Schimmack

Pinkus

R. T.

Lockwood

(2009). The nature and structure of correlations among Big Five ratings: The halo-alpha-beta model. Journal of Personality and Social Psychology, 97(6), 1142–1156.https://doi.org/10.1037/a0017159

Ashton

M. C.

Lee

(2009). The HEXACO–60: A short measure of the major dimensions of personality. Journal of Personality Assessment, 91(4), 340–345. https://doi.org/10.1080/00223890902935878

Bäckström

(2007). Higher-order factors in a five-factor personality inventory and its relation to social desirability. European Journal of Psychological Assessment, 23(2), 63–70. https://doi.org/10.1027/1015-5759.23.2.63

Bäckström

Björklund

(2013). Social desirability in personality inventories: Symptoms, diagnosis and prescribed cure. Scandinavian Journal of Psychology, 54(2), 152–159. https://doi.org/10.1111/sjop.12015

10.

Bäckström

Björklund

(2016). Is the general factor of personality based on evaluative responding? Experimental manipulation of item-popularity in personality inventories. Personality and Individual Differences, 96, 31–35.

11.

Bäckström

Björklund

(2020). The properties and utility of less evaluative personality scales: Reduction of social desirability; increase of construct and discriminant validity. Frontiers in Psychology, 11, 560271. https://doi.org/10.3389/fpsyg.2020.560271

12.

Bäckström

Björklund

Larsson

M. R.

(2009). Five-factor inventories have a major general factor related to social desirability which can be reduced by framing items neutrally. Journal of Research in Personality, 43(3), 335–344. https://doi.org/10.1016/j.jrp.2008.12.013

13.

Bäckström

Björklund

Larsson

M. R.

(2014). Criterion validity is maintained when items are evaluatively neutralized: Evidence from a full-scale Five-Factor Model inventory. European Journal of Personality, 28(6), 620–633. https://doi.org/10.1002/per.1960

14.

Barrick

M. R.

Mount

M. K.

(1996). Effects of impression management and self-deception on the predictive validity of personality constructs. Journal of Applied Psychology, 81(3), 261–272. https://doi.org/10.1037/0021-9010.81.3.261

15.

Bartram

(2007). Increasing validity with forced-choice criterion measurement formats. International Journal of Selection and Assessment, 15(3), 263–272. https://doi.org/10.1111/j.1468-2389.2007.00386.x

16.

Biderman

M. D.

Nguyen

N. T.

Cunningham

C. J.

Ghorbani

(2011). The ubiquity of common method variance: The case of the Big Five. Journal of Research in Personality, 45(5), 417–429. https://doi.org/10.1016/j.jrp.2011.05.001

17.

Birkeland

S. A.

Manson

T. M.

Kisamore

J. L.

Brannick

M. T.

Smith

M. A.

(2006). A meta-analytic investigation of job applicant faking on personality measures. International Journal of Selection and Assessment, 14(4), 317–335. https://doi.org/10.1111/j.1468-2389.2006.00354.x

18.

Block

(1995). A contrarian view of the five-factor approach to personality description. Psychological Bulletin, 117(2), 187–215. https://doi.org/10.1037/0033-2909.117.2.187

19.

Borkenau

(1992). Implicit personality theory and the five‐factor model. Journal of Personality, 60(2), 295–327.

20.

Borkenau

Ostendorf

(1989). Descriptive consistency and social desirability in self‐and peer reports. European Journal of Personality, 3(1), 31–45.

21.

Cattell

(1986). The psychometric properties of tests: Consistency, validity, and efficiency. In R. B. Cattell & R. C. Johnson (Eds), Functional psychological testing: Principles and instruments, pp. 54--78. Brunner/Mazel.

22.

Chen

Watson

Biderman

Ghorbani

(2016). Investigating the properties of the general factor (M) in bifactor models applied to Big Five or HEXACO data in terms of method or meaning. Imagination, Cognition and Personality, 35(3), 216–243. https://doi.org/10.1177/0276236615590587

23.

Chmielewski

Watson

(2009). What is being assessed and why it matters: The impact of transient error on trait research. Journal of Personality and Social Psychology, 97(1), 186–202. https://doi.org/10.1037/a0015618

24.

Christiansen

N. D.

Goffin

R. D.

Johnston

N. G.

Rothstein

M. G.

(1994). Correcting the 16PF for faking: Effects on criterion-related validity and individual hiring decisions. Personnel Psychology, 47(4), 847–860. https://doi.org/10.1111/j.1744-6570.1994.tb01581.x

25.

Clifton

J. D. W.

(2020). Managing validity versus reliability trade-offs in scale-building decisions. Psychological Methods, 25(3), 259–270. https://doi.org/10.1037/met0000236

26.

Condon

D. M.

Revelle

(2014). The international cognitive ability resource: Development and initial validation of a public-domain measure. Intelligence, 43, 52–64. https://doi.org/10.1016/j.intell.2014.01.004

27.

Conn

S. R.

Rieke

M. L.

(1994). 16PF fifth edition technical manual. Institute for Personality & Ability Testing, Incorporated.

28.

Costa Jr P. T. McCrae

R. R.

(1992). Four ways five factors are basic. Journal of Personality and Individual Differences, 13(6), 653–665. https://doi.org/10.1016/0191-8869(92)90236-I

29.

Costa

P. T.

McCrae

R. R.

(1992). Normal personality assessment in clinical practice: The NEO Personality Inventory. Psychological Assessment, 4(1), 5–13. https://doi.org/10.1037/1040-3590.4.1.5

30.

Cronbach

L. J.

(1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/bf02310555

31.

Davies

S. E.

Connelly

B. S.

Ones

D. S.

Birkland

A. S.

(2015). The general factor of personality: The “Big One,” a self-evaluative trait, or a methodological gnat that won’t go away? Personality and Individual Differences, 81, 13–22. https://doi.org/10.1016/j.paid.2015.01.006

32.

de Vries

R. E.

Bakker-Pieper

Konings

F. E.

Schouten

(2013). The communication styles inventory (CSI) a six-dimensional behavioral model of communication styles and its relation with personality. Communication Research, 40(4), 506–532.https://doi.org/10.1177/0093650211413571

33.

de Vries

R. E.

Zettler

Hilbig

B. E.

(2014). Rethinking trait conceptions of social desirability scales: Impression management as an expression of honesty-humility. Assessment, 21(3), 286–299. https://doi.org/10.1177/1073191113504619

34.

Dodaj

(2012). Social desirability and self-reports: Testing a content and response-style model of socially desirable responding. Europe’s Journal of Psychology, 8(4), 651–666. https://doi.org/10.5964/ejop.v8i4.462

35.

Douglas

E. F.

McDaniel

M. A.

Snell

A. F.

(1996). The validity of non-cognitive measures decays when applicants fake. In Academy of Management Proceedings. Vol. 1996, No. 1, pp. 127--131. Academy of Management.

36.

Dunlop

P. D.

Bourdage

J. S.

de Vries

R. E.

McNeill

I. M.

Jorritsma

Orchard

, Austen, T., Baines, T., & Choe

W. -K.

(2020). Liar! Liar!(when stakes are higher): Understanding how the overclaiming technique can be used to measure faking in personnel selection. Journal of Applied Psychology, 105(8), 784–799. https://doi.org/10.1037/apl0000463

37.

Dwight

S. A.

Donovan

J. J.

(2003). Do warnings not to fake reduce faking? Human Performance, 16(1), 1–23. https://doi.org/10.1207/s15327043hup1601_1

38.

Edwards

A. L.

(1953). The relationship between the judged desirability of a trait and the probability that the trait will be endorsed. Journal of Applied Psychology, 37(2), 90–93. https://doi.org/10.1037/h0058073

39.

Ellingson

J. E.

Sackett

P. R.

Hough

L. M.

(1999). Social desirability corrections in personality measurement: Issues of applicant comparison and construct validity. Journal of Applied Psychology, 84(2), 155–166. https://doi.org/10.1037/0021-9010.84.2.155

40.

Funder

D. C.

(2001). Personality. Annual Review of Psychology, 52, 197–221. https://doi.org/10.1146/annurev.psych.52.1.197

41.

Goffin

R. D.

Christiansen

N. D.

(2003). Correcting personality tests for faking: A review of popular personality tests and an initial survey of researchers. International Journal of Selection and Assessment, 11(4), 340–344. https://doi.org/10.1111/j.0965-075X.2003.00256.x

42.

Goldberg

L. R.

(1990). An alternative “description of personality”: The Big-Five factor structure. Journal of Personality and Social Psychology, 59(6), 1216–1229. https://doi.org/10.1037/0022-3514.59.6.1216

43.

Goldberg

L. R.

Johnson

J. A.

Eber

H. W.

Hogan

Ashton

M. C.

Cloninger

C. R.

Gough

H. G.

(2006). The international personality item pool and the future of public-domain personality measures. Journal of Research in Personality, 40(1), 84–96. https://doi.org/10.1016/j.jrp.2005.08.007

44.

Hough

L. M.

(1997). Personality at work: Issue and evidence. In Hakel

M. D.

(Ed.), Beyond multiple choice: Evaluating alternatives to traditional testing for selection (pp. 131–166). Lawrence Erlbaum Associates.

45.

Jeong

Y. R.

Christiansen

N. D.

Robie

Kung

M. C.

Kinney

T. B.

(2017). Comparing applicants and incumbents: Effects of response distortion on mean scores and validity of personality measures. International Journal of Selection and Assessment, 25(3), 311–315. https://doi.org/10.1111/ijsa.12182

46.

John

O. P.

Robins

R. W.

(1993). Determinants of interjudge agreement on personality traits: The Big Five domains, observability, evaluativeness, and the unique perspective of the self. Journal of Personality, 61(4), 521–551. https://doi.org/10.1111/j.1467-6494.1993.tb00781.x

47.

John

O. P.

Robins

R. W.

(1994). Accuracy and bias in self-perception: Individual differences in self-enhancement and the role of narcissism. Journal of Personality and Social Psychology, 66(1), 206–219. https://doi.org/10.1037/0022-3514.66.1.206

48.

Just

(2011). A review of literature on the general factor of personality. Personality and Individual Differences, 50(6), 765–771. https://doi.org/10.1016/j.paid.2011.01.008

49.

Kam

(2013). Probing item social desirability by correlating personality items with Balanced Inventory of Desirable Responding (BIDR): A validity examination. Personality and Individual Differences, 54(4), 513–518. https://doi.org/10.1016/j.paid.2012.10.017

50.

Leising

Burger

Zimmermann

Bäckström

Oltmanns

J. R.

Connelly

B. S.

(2020). Why do items correlate with one another? A conceptual analysis with relevance for general factors and network models. PsyArXiv. https://doi.org/10.31234/osf.io/7c895

51.

Leising

Ostrovski

Borkenau

(2012). Vocabulary for describing disliked persons is more differentiated than vocabulary for describing liked persons. Journal of Research in Personality, 46(4), 393–396. https://doi.org/10.1016/j.jrp.2012.03.006

52.

Leising

Scherbaum

Locke

K. D.

Zimmermann

(2015). A model of “substance” and “evaluation” in person judgments. Journal of Research in Personality, 57, 61–71. https://doi.org/10.1016/j.jrp.2015.04.002

53.

Leising

Vogel

Waller

Zimmermann

(2020). Correlations between person-descriptive items are predictable from the product of their mid-point-centered social desirability values. European Journal of Personality, 0890207020962331. https://doi.org/10.1177/0890207020962331

54.

Loevinger

(1954). The attenuation paradox in test theory. Psychological Bulletin, 51(5), 493–504. https://doi.org/10.1037/h0058543

55.

MacCann

Pearce

Jiang

(2017). The general factor of personality is stronger and more strongly correlated with cognitive ability under instructed faking. Journal of Individual Differences, 38(1), 46–54. https://doi.org/10.1027/1614-0001/a000221

56.

McCrae

R. R.

Costa

P. T.

(1983). Social desirability scales: More substance than style. Journal of Consulting and Clinical Psychology, 51(6), 882.

57.

McCrae

R. R.

John

O. P.

(1992). An introduction to the five-factor model and its applications. Journal of Personality, 60(2), 175–215. https://doi.org/10.1111/j.1467-6494.1992.tb00970.x

58.

McCrae

R. R.

Mõttus

(2019). What personality scales measure: A new psychometrics and its implications for theory and assessment. Current Directions in Psychological Science, 28(4), 415–420. https://doi.org/10.1177/0963721419849559

59.

Mcfarland

L. A.

(2003). Warning against faking on a personality test: Effects on applicant reactions and personality test scores. International Journal of Selection and Assessment, 11(4), 265–276. https://doi.org/10.1111/j.0965-075X.2003.00250.x

60.

McFarland

L. A.

Ryan

A. M.

(2000). Variance in faking across noncognitive measures. Journal of Applied Psychology, 85(5), 812.

61.

McNeish

(2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412–433. https://doi.org/10.1037/met0000144

62.

Morgeson

F. P.

Campion

M. A.

Dipboye

R. L.

Hollenbeck

J. R.

Murphy

Schmitt

(2007). Reconsidering the use of personality tests in personnel selection contexts. Personnel Psychology, 60(3), 683–729. https://doi.org/10.1111/j.1744-6570.2007.00089.x

63.

Mueller-Hanson

Heggestad

E. D.

Thornton

G. C.

III (2003). Faking and selection: Considering the use of personality from select-in and select-out perspectives. Journal of Applied Psychology, 88(2), 348–355. https://doi.org/10.1037/0021-9010.88.2.348

64.

Musek

(2007). A general factor of personality: Evidence for the Big One in the five-factor model. Journal of Research in Personality, 41(6), 1213–1233. https://doi.org/10.1016/j.jrp.2007.02.003

65.

Nederhof

A. J.

(1985). Methods of coping with social desirability bias: A review. European Journal of Social Psychology, 15(3), 263–280. https://doi.org/10.1002/ejsp.2420150303

66.

Ones

D. S.

Viswesvaran

Reiss

A. D.

(1996). Role of social desirability in personality testing for personnel selection: The red herring. Journal of Applied Psychology, 81(6), 660–679. https://doi.org/10.1037/0021-9010.81.6.660

67.

Paulhus

D. L.

(1984). Two-component models of socially desirable responding. Journal of Personality and Social Psychology, 46(3), 598–609. https://doi.org/10.1037/0022-3514.46.3.598

68.

Paunonen

S. V.

LeBel

E. P.

(2012). Socially desirable responding and its elusive effects on the validity of personality assessments. Journal of Personality and Social Psychology, 103(1), 158.

69.

Peabody

(1967). Trait inferences: Evaluative and descriptive aspects. Journal of Personality and Social Psychology, 7(4p2), 1–18. https://doi.org/10.1037/h0025230

70.

Peabody

(1984). Personality dimensions through trait inferences. Journal of Personality and Social Psychology, 46(2), 384–403. https://doi.org/10.1037/0022-3514.46.2.384

71.

Peabody

Goldberg

L. R.

(1989). Some determinants of factor structures from personality-trait descriptors. Journal of Personality and Social Psychology, 57(3), 552–567. https://doi.org/10.1037/0022-3514.57.3.552

72.

Pettersson

Turkheimer

Horn

E. E.

Menatti

A. R.

(2012). The general factor of personality and evaluation. European Journal of Personality, 26(3), 292–302. https://doi.org/10.1002/per.839

73.

Phillips

D. L.

Clancy

K. J.

(1972). Some effects of" social desirability" in survey studies. American Journal of Sociology, 77(5), 921–940. https://doi.org/10.1086/225231

74.

Reise

S. P.

Moore

T. M.

Haviland

M. G.

(2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92(6), 544–559. https://doi.org/10.1080/00223891.2010.496477

75.

Revelle

Wilt

(2013). The general factor of personality: A general critique. Journal of Research in Personality, 47(5), 493–504. https://doi.org/10.1016/j.jrp.2013.04.012

76.

Rothstein

M. G.

Goffin

R. D.

(2006). The use of personality measures in personnel selection: What does current research support? Human Resource Management Review, 16(2), 155–180. https://doi.org/10.1016/j.hrmr.2006.03.004

77.

Saucier

(2002). Orthogonal markers for orthogonal factors: The case of the Big Five. Journal of Research in Personality, 36(1), 1–31. https://doi.org/10.1006/jrpe.2001.2335

78.

Schmit

M. J.

Ryan

A. M.

(1993). The Big Five in personnel selection: Factor structure in applicant and nonapplicant populations. Journal of Applied Psychology, 78(6), 966–974. https://doi.org/10.1037/0021-9010.78.6.966

79.

Schmitt

Oswald

F. L.

(2006). The impact of corrections for faking on the validity of noncognitive measures in selection settings. Journal of Applied Psychology, 91(3), 613–621. https://doi.org/10.1037/0021-9010.91.3.613

80.

Steiger

J. H.

(1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87(2), 245–251. https://doi.org/10.1037/0033-2909.87.2.245

81.

Tett

R. P.

Jackson

D. N.

Rothstein

(1991). Personality measures as predictors of job performance: A meta-analytic review. Personnel Psychology, 44(4), 703–742. https://doi.org/10.1111/j.1744-6570.1991.tb00696.x

82.

Topping

G. D.

O'Gorman

(1997). Effects of faking set on validity of the NEO-FFI. Personality and Individual Differences, 23(1), 117–124. https://doi.org/10.1016/s0191-8869(97)00006-8

83.

Uziel

(2010). Rethinking social desirability scales: From impression management to interpersonally oriented self-control. Perspectives on Psychological science, 5(3), 243–262. https://doi.org/10.1177/1745691610369465

84.

van der Linden

Dunkel

C. S.

Petrides

(2016). The general factor of personality (GFP) as social effectiveness: Review of the literature. Personality and Individual Differences, 101, 98–105. https://doi.org/10.1016/j.paid.2016.05.020

85.

van der Linden

Pekaar

K. A.

Bakker

A. B.

Schermer

J. A.

Vernon

P. A.

Dunkel

C. S.

Petrides

(2017). Overlap between the general factor of personality and emotional intelligence: A meta-analysis. Psychological Bulletin, 143(1), 36–52. https://doi.org/10.1037/bul0000078

86.

van der Linden

te Nijenhuis

Bakker

A. B.

(2010). The general factor of personality: A meta-analysis of Big Five intercorrelations and a criterion-related validity study. Journal of Research in Personality, 44(3), 315–327. https://doi.org/10.1016/j.jrp.2010.03.003

87.

Wessels

N. M.

Zimmermann

Biesanz

J. C.

Leising

(2020). Differential associations of knowing and liking with accuracy and positivity bias in person perception. Journal of Personality and Social Psychology, 118(1), 149–171. https://doi.org/10.1037/pspp0000218

88.

West

T. V.

Kenny

D. A.

(2011). The truth and bias model of judgment. Psychological Review, 118(2), 357.

89.

Worthington

D. L.

Schlottmann

R. S.

(1986). The predictive validity of subtle and obvious empirically derived psychological test items under faking conditions. Journal of Personality Assessment, 50(2), 171–181. https://doi.org/10.1207/s15327752jpa5002_2

90.

Zickar

M. J.

Robie

(1999). Modeling faking good on personality items: An item-level analysis. Journal of Applied Psychology, 84(4), 551–563. https://doi.org/10.1037/0021-9010.84.4.551

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.09 MB