Researchers commonly compare means and other statistics across groups with little concern for whether the measure possesses strong factorial invariance (i.e., equal factor loadings and intercepts/thresholds). When this assumption is violated, inaccurate inferences associated with statistical and practical significance can occur. This manuscript emphasizes the importance of testing for measurement invariance (MI) and provides guidance when conducting these tests. Topics discussed are potential causes of noninvariant items, the difference between measurement bias and invariance, remedies for noninvariant measures, and considerations associated with model estimation. Using a sample of 491 teachers, a demonstration is also provided that evaluates whether a newly constructed behavior and instructional management scale is invariant across elementary and middle school teachers. Analyses revealed that the results differ slightly based on the estimation method utilized although these differences did not greatly influence the latent factor mean difference conclusions. Additional implications and considerations related to invariance testing are discussed.
Barrett, P. ( 2007). Structural equation modelling: Adjudging model fit. Personality and Individual Differences, 42, 815-824.
3.
Barry, C.L., & Finney, S.J. ( 2009). Does it matter how data are collected? A comparison of testing conditions and the implications for validity. Research & Practice in Assessment, 3, 1-15.
4.
Beauducel, A., & Herzberg, P.Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling: A Multidisciplinary Journal, 13, 186-203
5.
Browne, M.W. ( 2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36, 111-150.
6.
Byrne, B.M., Shavelson, R.J., & Muthén, B.O. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456-466.
7.
Chen, F.F. ( 2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, 464-504.
8.
Chen, F.F. ( 2008). What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural research. Journal of Personality and Social Psychology, 95, 1005-1018.
Cheung, G.W., & Rensvold, R.B. (1999). Testing factorial invariance across groups: A reconceptualization and proposed new method. Journal of Management, 25, 1-27.
11.
Cheung, G.W., & Rensvold, R.B. (2000). Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling . Journal of Cross-cultural Psychology, 31, 187-212.
Dimitrov, D.M. ( 2010). Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development , 43, 121-149
14.
Flora, D.B., & Curran, P.J. ( 2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9, 466-491.
Green, S.B., & Yang, Y.(2009) Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika, 74, 155-167
17.
Hancock, G.R., Lawrence, F.R., & Nevitt, J. ( 2000). Type I error and power of latent mean methods and manova in factorially invariant and noninvariant latent variable systems. Structural Equation Modeling, 7, 534-556.
18.
Hu, L., & Bentler, P.M. ( 1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55.
19.
Little, T.D., Slegers, D.W., & Card, N.A. ( 2006). A non-arbitrary method of identifying and scaling latent variables in SEM and MACS models. Structural Equation Modeling , 13, 59-72.
20.
Lubke, G.H., & Muthén, B.O. (2004). Applying multigroup confirmatory factor models for continuous outcomes to Likert scale data complicates meaningful group comparisons. Structural Equation Modeling , 11, 514-534.
21.
Marsh, H.W. ( 1985). The structure of masculintiy/feminity: An application of confimatory factor analysis to higher-order factor structures and factorial invariance. Multivariate Behavioral Research, 20, 427-449.
22.
Marsh, H.W., Hau, K.T., & Wen, Z.L. ( 2004) In search of golden rules: Comment on hypothesis testing approaches to setting cutoff values for fit indexes and dangers in overgeneralising Hu & Bentler (1999) findings. Structural Equation Modeling , 11, 320-341.
23.
Marsh, H.W., & Hocevar, D. ( 1985). Application of confirmatory factor analysis to the study of self-concept: First- and higher order factor models and their invariance across groups. Psychological Bulletin, 97, 562-582.
24.
Marsh, H., Liem, G., Martin, A., & Nagengast, B., & Morin, A. ( 2011). Methodological-measurement fruitfulness of exploratory structural equation modeling (ESEM): New approaches to key substantive issues in motivation and engagement. Journal of Psychoeducational Assessment, 29, 322-346.
25.
Marsh, H.W., Tracey, D.K., & Craven, R.G. ( 2006). Multidimensional self-concept structure for preadolescents with mild intellectual disabilities: A hybrid multigroup-MIMC approach to factorial invariance and latent mean differences. Educational and Psychological Measurement, 66, 795-818.
26.
Martin, N.K., & Sass, D.A. ( 2010). Construct validation of the Behavior and Instructional Management Scale. Teaching & Teacher Education, 26, 1124-1135.
27.
McDonald, R.P., Seifert, C.F., Lorenzet, S.J., Givens, S., & Jaccard, J. ( 2002). The effectiveness of methods for analyzing multivariate factorial data. Organizational Research Methods, 5, 255-274.
28.
Meredith, W. ( 1993). Measurment invariance, factor analysis and factorial invariance. Psychometrika, 58, 525-543.
29.
Millsap, R.E. ( 2001). When trivial constraints are not trivial: The choice of uniqueness constraints in confirmatory factor analysis. Structural Equation Modeling, 8, 1-17.
30.
Millsap, R.E. ( 2005). Four unresolved problems in studies of factorial invariance . In A. Maydeu-Olivares & J. J. McArdle (Eds.), Contemporary psychometrics (pp. 153-172). Mahwah, NJ: Lawrence Erlbaum.
31.
Millsap, R.E., & Kwok, O. ( 2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 9(1), 93-115.
32.
Millsap, R.E., & Meredith, W. ( 2007). Factorial invariance: Historical perspectives and new problems. In R. Cudeck & R. MacCallum (Eds.) Factor analysis at 100 (pp. 131-152) Mahwah, NJ: Lawrence Erlbaum.
33.
Millsap, R.E., & Yun-Tein, J. ( 2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39, 479-515.
Satorra, A., & Bentler, P.M. ( 2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66, 507-514.
36.
Satorra, A., & Bentler, P.M. ( 2010). Ensuring positiveness of the scaled difference chi-square test statistic. Psychometrika, 75, 243-248.
37.
Schmitt, T.A. ( 2011). Current methodological considerations in exploratory and confirmatory factor analysis. Journal of Psychoeducational Assessment, 29, 304-321.
38.
Steenkamp, J.E.M., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research , 25, 78-90.
39.
Van de Vijver, F.J.R., & Leung, K. ( 2000). Methodological issues in psychological research on culture . Journal of Cross-Cultural Psychology, 31(1), 33-35
40.
Van de Vijver, F.J.R., & Poortinga, Y.H. (2005). Conceptual and methodological issues in adapting tests. In R. K. Hambleton , P. F. Merenda, & C. D. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 39-63). Mahwah, NJ: Lawrence Erlbaum.
41.
Vandenberg, R.J., & Lance, C.E. ( 2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4-69.
42.
Walker, C.M. ( 2011). What’s the DIF? Why differential item functioning analyses are an important part of instrument development and validation. Journal of Psychoeducational Assessment, 29, 364-376.
43.
Yang, Y., & Green, S.B. ( 2011). Coefficient alpha: A reliability coefficient for the 21st century?Journal of Psychoeducational Assessment, 29, 377-392.
44.
Yu, C.Y. ( 2002). Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes. (Unpublished doctoral dissertation). University of California, Los Angeles.
45.
Yuan, K.H., & Bentler, P.M. ( 2000). Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. In M. E. Sobel & M. P. Becker (Eds.), Sociological methodology (pp. 165-200). Washington, DC: ASA.
