For organizational research on individual change, missing data can greatly reduce longitudinal sample size and potentially bias parameter estimates. Within the structural equation modeling framework, this article compares six missing data techniques (MDTs): listwise deletion, pairwise deletion, stochastic regression imputation, the expectation-maximization (EM) algorithm, full information maximization likelihood (FIML), and multiple imputation (MI). The rationale for each technique is reviewed, followed by Monte Carlo analysis based on a threewave simulation of organizational commitment and turnover intentions. Parameter estimates and standard errors for each MDT are contrasted with complete-data estimates, under three mechanisms of missingness (completely random, random, and nonrandom) and three levels of missingness (25%, 50%, and 75%; all monotone missing). Results support maximum likelihood and MI approaches, which particularly outperform listwise deletion for parameters involving many recouped cases. Better standard error estimates are derived from FIML and MI techniques. All MDTs perform worse when data are missing nonrandomly.
Get full access to this article
View all access options for this article.
References
1.
Alexander, R. A. (1988). Group homogeneity, range restriction, and range enhancement effects on correlations. Personnel Psychology, 41(4), 773-777.
2.
Alexander, R. A., Barrett, G. V., Alliger, G. M., & Carson, K. P. (1986). Toward a general model of nonrandom sampling and the impact of population correlations: Generalizations of Berkson’s Fallacy and restriction of range. British Journal of Mathematical and Statistical Psychology, 39, 90-105.
3.
Allison, P. D. (1987). Estimation of linear models with incomplete data. In C. C. Clogg (Ed.), Sociological methodology (pp. 71-103). Washington, DC: American Sociological Association.
4.
Allison, P. D. (2002). Missing data (Sage University Papers Series on Quantitative Applications in the Social Sciences, 07-136). Thousand Oaks, CA: Sage.
5.
Anderson, S. E., & Williams, L. J. (1992). Assumptions about unmeasured variables with studies of reciprocal relationships: The case of employee attitudes. Journal of Applied Psychology, 77(5), 638-650.
6.
Arbuckle, J. L. (1995). Amos user’s guide [Computer software]. Chicago: SmallWaters.
7.
Bateman, T. S., & Strasser, S. (1984). A longitudinal analysis of the antecedents of organizational commitment. Academy of Management Journal, 27, 95-112.
8.
Brett, J. M., Feldman, D. C., & Weingart, L. R. (1990). Feedback-seeking behavior of new hires and job changers. Journal of Management, 16(4), 737-749.
9.
Chan, D. (1998). The conceptualization and analysis of change over time: An integrated approach incorporating longitudinal mean and covariance structures analysis (LMACS) and multiple indicator latent growth modeling (MLGM). Organizational Research Methods, 1, 421-483.
10.
Chan, D., & Schmitt, N. (2000). Interindividual differences in intraindividual changes in proactivity during organizational entry: A latent growth modeling approach to understanding newcomer adaptation. Journal of Applied Psychology, 85(2), 190-210.
11.
Collins, L. M., Schafer, J. L., & Kam, C. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6(4), 330-351.
12.
Dempster, A. P., Laird, N. H., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, B39, 1-38.
13.
Duncan, T. E., Duncan, S. C., & Li, F. (1998). A comparison of modeland multiple-imputation-based approaches to longitudinal analyses with partial missingness. Structural Equation Modeling, 5, 1-21.
14.
Enders, C. K. (2001). A primer on maximum likelihood algorithms for use with missing data. Structural Equation Modeling, 8(1), 128-141.
15.
Enders, C. K., & Bandalos, D. L. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling, 8(3), 430-457.
16.
Farkas, A. J., & Tetrick, L. E. (1989). A three-wave longitudinal analysis of the causal ordering of satisfaction and commitment on turnover decisions. Journal of Applied Psychology, 74(6), 855-868.
17.
Finkbeiner, C. (1979). Estimation for the multiple group factor model when data are missing. Psychometrika, 44, 409-420.
18.
Fullagar, C., & Barling, J. (1989). A longitudinal test of a model of the antecedents and consequences of union loyalty. Journal of Applied Psychology, 74, 213-227.
19.
Garst, H., Frese, M., & Molenaar, P. C. M. (2000). The temporal factor of change in stressorstrain relationships: A growth curve model on a longitudinal study in East Germany. Journal of Applied Psychology, 85(3), 417-438.
20.
Gold, M. S., & Bentler, P. M. (2000). Treatments of missing data: A Monte Carlo comparison of RBHDI, iterative stochastic regression imputation, and expectation-maximization. Structural Equation Modeling, 7(3), 319-355.
21.
Goodman, J. S., & Blum, T. C. (1996). Assessing the non-random sampling effects of subject attrition in longitudinal research. Journal of Management, 22(4), 627-652.
22.
Gourieroux, C., & Monfort, A. (1981). On the problem of missing data in linear models. Review of Economics Studies, 48, 579-586.
23.
Graham, J. W. (in press). Adding missing-data-relevant variables to FIML-based structural equation models. Structural Equation Modeling.
24.
Graham, J. W., & Donaldson, S. W. (1993). Evaluating interventions with differential attrition: The importance of nonresponse mechanisms and use of follow-up data. Journal of Applied Psychology, 78, 119-128.
25.
Hartley, H. O., & Hocking, R. R. (1971). The analysis of incomplete data. Biometrics, 27, 783-823.
26.
Jamshidian, M., & Bentler, P. M. (1999). ML estimation of mean and covariance structures with missing data using complete data routines. Journal of Educational and Behavioral Statistics, 24(1), 21-41.
27.
Jöreskog, K., & Sörbom, D. (1993). PRELIS 2: A program for multivariate data screening and data summarization [Computer software]. Chicago: Scientific Software International, Inc.
28.
Jöreskog, K. G., & Sörbom, D. (1996). LISREL-8 [Computer software]. Moresville, IN: Scientific Software.
29.
Kim, J. O., & Curry, J. (1977). The treatment of missing data in multivariate analysis. Sociological Methods and Research, 6, 215-240.
30.
Lance, C. E., Vandenberg, R. J., & Self, R. M. (2000). Latent growth models of individual change: The case of newcomer adjustment. Organizational Behavior and Human Decision Processes, 83(1), 107-140.
31.
Lee, S. Y. (1986). Estimation for structural equation models with missing data. Psychometrika, 45, 309-324.
32.
Little, R. J. A., & Rubin, D. B. (1987). Statistical analysis with missing data. New York: John Wiley.
33.
Marini, M. M., Olsen, A. R., & Rubin, D. B. (1980). Maximum likelihood estimation in panel studies with missing data. In K. F. Schuessler (Ed.), Sociological methodology (pp. 314-357). San Francisco: Jossey-Bass.
34.
Marsh, H. W. (1998). Pairwise deletion for missing data in structural equation models: Nonpositive definite matrices, parameter estimates, goodness of fit, and adjusted sample sizes. Structural Equation Modeling, 5, 22-36.
35.
McArdle, J. J., & Hamagami, F. (1992). Modeling incomplete longitudinal and cross-sectional data using latent growth structural models. Experimental Aging Research, 18, 145-166.
36.
Meyer, J. P., Bobocel, D., & Allen, N. J. (1991). Development of organizational commitment during the first year of employment: A longitudinal study of preand post-entry influences. Journal of Management, 17(4), 717-733.
37.
Muthen, B., Kaplan, D., & Hollis, M. (1987). On structural equation modeling with data that are not missing completely at random. Psychometrika, 52, 431-462.
38.
Muthen, L. K., & Muthen, B. O. (1998). Mplus user’s guide. Los Angeles: Muthen & Muthen.
39.
Porter, L. W., Steers, R. M., Mowday, R. T., & Boulian, P. V. (1974). Organizational commitment, job satisfaction, and turnover among psychiatric technicians. Journal of Applied Psychology, 59, 603-609.
40.
Raymond, M. R., & Roberts, D. M. (1987). A comparison of methods for treating incomplete data in selection research. Educational and Psychological Measurement, 47, 13-26.
41.
Rogelberg, S. G., Luong, A., Sederburg, M. E., & Cristol, D. S. (2000). Employee attitude surveys: Examining the attitudes of noncompliant employees. Journal of Applied Psychology, 82(2), 284-293.
42.
Roth, P. L. (1994). Missing data: A conceptual review for applied psychologists. Personnel Psychology, 47(3), 537-560.
43.
Roth, P. L., & Switzer, F. S., III. (1995). A Monte Carlo analysis of missing data techniques in an HRM setting. Journal of Management, 21, 1003-1023.
44.
Rubin, D. B. (1976). Inference and missing data. Biometrika, 63, 581-592.
45.
Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York: John Wiley.
46.
Rovine, M. J. (1994). Latent variable models and missing data analysis. In A. von Eye & C. C. Clogg (Eds.), Latent variables analysis: Applications for developmental research (pp. 181-225). Thousand Oaks, CA: Sage.
47.
Schafer, J. L. (1997). Analysis of incomplete multivariate data. New York: Chapman & Hall.
48.
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147-177.
49.
Switzer, F. S., III, Roth, P. L., & Switzer, D. M. (1998). Systematic data loss in HRM settings: A Monte Carlo analysis. Journal of Management, 24(6), 763-779.
50.
Werts, C. E., Rock, D. A., & Grandy, J. (1979). Confirmatory factor analysis applications: Missing data problems and comparison of path models between populations. Multivariate Behavioral Research, 14, 199-213.
