Notes to Factor Analysis Techniques for Construct Validity

Abstract

This paper introduces and discusses factor analysis techniques for construct validity, including some suggestions for reporting using the evidence to support the construct validity from exploratory and confirmatory factor analysis techniques. Construct validity is a vital part of psychological testing and a prerequisite to every measurement instrument, including aptitude, achievement, and interests. Research, particularly in nursing and the health sciences, depends on reliable and valid measurements. Therefore, a growing emphasis is on assessing validity regarding the structure of test variables commonly estimated by factor analysis techniques. However, it is not always clear how to report the analysis and use it to support the construct validity. Both exploratory and confirmatory factor analysis techniques provide vital evidence to support the construct validity. However, these are not the only available evidence for construct validity, and the researcher should always consider other sources of evidence to develop and support the construct validity of their intended measures. In addition, the collection and presentation of this evidence are not limited to a time, but the validity of constructs is a continuous process that leads to validating the underlying theories from which constructs have emerged.

Keywords

Factor analysis Health sciences Construct validity research

Get full access to this article

View all access options for this article.

References

Alavi

Visentin

D. C.

Thapa

D. K.

Hunt

G. E.

Watson

Cleary

(2020a). Chi-square for model fit in confirmatory factor analysis. Journal of Advanced Nursing, 76(9), 2209–2211. https://doi.org/10.1111/jan.14399

Alavi

Visentin

D. C.

Thapa

D. K.

Hunt

G. E.

Watson

Cleary

(2020b). Exploratory factor analysis and principal component analysis in clinical studies: Which one should you use? Journal of Advanced Nursing, 76(8), 1886–1889. https://doi.org/10.1111/jan.14377

Andrews

F. M.

(1984). Construct validity and error components of survey measures: A structural modeling approach. The Public Opinion Quarterly, 48(2), 409–442. https://doi.org/10.1086/268840

Bandalos

D. L.

(2018). Measurement theory and applications for the social sciences. Guilford Publications.

Beavers

A. S.

Lounsbury

J. W.

Richards

J. K.

Huck

S. W.

Skolits

G. J.

Esquivel

S. L.

(2013). Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research, and Evaluation, 18(1), 6. https://doi.org/10.7275/qv2q-rk76

Boateng

G. O.

Neilands

T. B.

Frongillo

E. A.

Melgar-Quiñonez

H. R.

Young

S. L.

(2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, 149. https://doi.org/10.3389/fpubh.2018.00149

Brown

(2010). Construct validity: A unitary concept for occupational therapy assessment and measurement. Hong Kong Journal of Occupational Therapy, 20(1), 30–42. https://doi.org/10.1016/S1569-1861(10)70056-5

Brown

T. A.

(2015). Confirmatory factor analysis for applied research. Guilford publications.

Buckingham, B. R., McCall, W. A., Otis, A. S., Rugg, H. O., Trabue, M. R., & Courtis, S. A. (1921). Report of the Standardization Committee. Journal of Educational Research, 4(1), 78–80.

10.

Clark

L. A.

Watson

(2019). Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment, 31(12), 1412–1427. https://doi.org/10.1037/pas0000626

11.

Coleman

M. A.

(2006). Construct validity evidence based on internal structure: Exploring and comparing the use of Rasch measurement modeling and factor analysis with a measure of student motivation . Virginia Commonwealth University, ProQuest Dissertations Publishing, 3225841. https://www-proquest-com-s.web.bisu.edu.cn/openview/8aac11d9f8f180e71f93676351244ffa/1?cbl=18750&diss=y&parentSessionId=TJrYw9cRHox6bE%2BFWSyKyNe%2FFGamyqm4h9oKMellMEU%3D&pq-origsite=gscholar&accountid=41305

12.

Comrey

A. L.

Lee

H. B.

(1992). A first course in factor analysis. Lawrence Erlbaumm Associates, Inc.

13.

Costello

A. B.

Osborne

(2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10(1), 7. https://doi.org/10.7275/jyj1-4868

14.

Cote

J. A.

Buckley

M. R.

(1987). Estimating trait, method, and error variance: Generalizing across 70 construct validation studies. Journal of Marketing Research, 24(3), 315–318. https://doi.org/10.2307/3151642

15.

Cronbach

L. J.

Meehl

P. E.

(1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957

16.

DiStefano

Hess

(2005). Using confirmatory factor analysis for construct validation: An empirical review. Journal of Psychoeducational Assessment, 23(3), 225–241. https://doi.org/10.1177/073428290502300303

17.

Farrell

A. M.

Rudd

J. M.

(2009). Factor analysis and discriminant validity: A brief review of some practical issues. In Tojib

(Ed.), Australia and New Zealand Marketing Academy Conference 2009 Conference Proceedings. Australia: ANZMAC. https://publications.aston.ac.uk/id/eprint/7644/1/factor_analysis_ANZMAC_2009.pdf

18.

Goodwin

L. D.

Leech

N. L.

(2003). The meaning of validity in the new standards for educational and psychological testing. Measurement and Evaluation in Counseling and Development, 36(3), 181–191. https://doi.org/10.1080/07481756.2003.11909741

19.

Goodwyn

(2012). Factor scores, structure coefficients, and communality coefficients [Paper presentation]. Annual meeting of the Southwest Educational Research Association, New Orleans, February 2, 2012. https://files.eric.ed.gov/fulltext/ED529102.pdf

20.

Jackson

D. L.

Gillaspy

J. A.

Jr Purc-Stephenson

(2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods, 14(1), 6–23. https://doi.org/10.1037/a0014694

21.

Knekta

Runyon

Eddy

(2019). One size doesn't fit all: Using factor analysis to gather validity evidence when using surveys in your research. CBE Life Sciences Education, 18(1), rm1. https://doi.org/10.1187/cbe.18-04-0064

22.

Linn

R. L.

(2006). The standards for educational and psychological testing: Guidance in test development. In Downing

S. M.

Haladyna

T. M.

(Eds.), Handbook of test development (pp. 27–38). Lawrence Erlbaum Associates Publishers.

23.

Lorenzo-Seva

(2013). How to report the percentage of explained common variance in exploratory factor analysis (Technical Report). Department of Psychology, Universitat Rovira i Virgili, Tarragona. http://psico.fcep.urv.cat/utilitats/factor/

24.

MacCallum

R. C.

Austin

J. T.

(2000). Applications of structural equation modeling in psychological research. Annual Review of Psychology, 51(1), 201–226. https://doi.org/10.1146/annurev.psych.51.1.201

25.

MacKenzie

S. B.

Podsakoff

P. M.

Podsakoff

N. P.

(2011). Construct measurement and validation procedures in MIS and behavioral research: Integrating new and existing techniques. MIS Quarterly, 35(2), 293–334. https://doi.org/10.2307/23044045

26.

Matsunaga

(2010). How to factor-analyze your data right: Do's, don'ts, and how-to's. International Journal of Psychological Research, 3(1), 97–110. https://doi.org/10.21500/20112084.854

27.

Mislevy

R. J.

(1996). Test theory reconceived. Journal of Educational Measurement, 33(4), 379–416. https://doi.org/10.1111/j.1745-3984.1996.tb00498.x

28.

Mohajan

(2017). Two criteria for good measurements in research: Validity and reliability. Annals of Spiru Haret University Economic Series, 17(4), 59–82. https://doi.org/10.26458/1746

29.

Pett

M. A.

Lackey

N. R.

Sullivan

J. J.

(2003). Making sense of factor analysis: The use of factor analysis for instrument development in health care research. Sage Publications. https://doi.org/10.4135/9781412984898

30.

Rattray

Jones

M. C.

(2007). Essential elements of questionnaire design and development. Journal of Clinical Nursing, 16(2), 234–243. https://doi.org/10.1111/j.1365-2702.2006.01573.x

31.

Rios

Wells

(2014). Validity evidence based on internal structure. Psicothema, 26(1), 108–116. https://doi.org/10.7334/psicothema2013.260

32.

Rönkkö

Cho

(2020). An updated guideline for assessing discriminant validity. Organizational Research Methods, 25(1), 6–14. https://doi.org/10.1177/1094428120968614

33.

Slaney

(2017). Validating psychological constructs: Historical, philosophical, and practical dimensions. Palgrave Macmillan.

34.

Snook

S. C.

Gorsuch

R. L.

(1989). Component analysis versus common factor analysis: A Monte Carlo study. Psychological Bulletin, 106(1), 148–154. https://doi.org/10.1037/0033-2909.106.1.148

35.

Strauss

M. E.

Smith

G. T.

(2009). Construct validity: Advances in theory and methodology. Annual Review of Clinical Psychology, 5(1), 1–25. https://doi.org/10.1146/annurev.clinpsy.032408.153639

36.

Streiner

D. L.

Norman

G. R.

Cairney

(2015). Health measurement scales: A practical guide to their development and use (5th ed.). Oxford University Press. https://doi.org/10.1093/med/9780199685219.001.0001

37.

Tabachnick

B. G.

Fidell

L. S.

(2013). Using multivariate statistics (6th ed.). Pearson.

38.

Tavakol

Wetzel

(2020). Factor analysis: A means for theory and instrument development in support of construct validity. International Journal of Medical Education, 11, 245–247. https://doi.org/10.5116/ijme.5f96.0f4a

39.

Thompson

(2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. American Psychological Association.

40.

Thompson

Daniel

L. G.

(1996). Factor analytic evidence for the construct validity of scores: A historical overview and some guidelines. Educational and Psychological Measurement, 56(2), 197–208. https://doi.org/10.1177/0013164496056002001

41.

Westen

Rosenthal

(2003). Quantifying construct validity: Two simple measures. Journal of Personality and Social Psychology, 84(3), 608–618. https://doi.org/10.1037//0022-3514.84.3.608

42.

Wetzel

A. P.

(2012). Factor analysis methods and validity evidence: A review of instrument development across the medical education continuum. Academic Medicine, 87(8), 1060–1069. https://doi.org/10.1097/ACM.0b013e31825d305d

43.

Yun

Ulrich

D. A.

(2002). Estimating measurement validity: A tutorial. Adapted Physical Activity Quarterly, 19(1), 32–47. https://doi.org/10.1123/apaq.19.1.32

44.

Zhai

Haudek

K. C.

Wilson

Stuhlsatz

(2021). A framework of construct-irrelevant variance for contextualized constructed response assessment. Frontiers in Education, 6(409). https://doi.org/10.3389/feduc.2021.751283

45.

Zumbo

B. D.

Hubley

A. M.

(Eds.). (2017). Understanding and investigating response processes in validation research. Springer. https://doi.org/10.1007/978-3-319-56129-5