The problem of missing data at the second level of a two-level hierarchical data structure was investigated. Using data generated to simulate the 1982 High School and Beyond data set, five missing data treatments—listwise deletion, overall mean substitution, group mean substitution, the expectation maximization (EM) algorithm, and multiple imputation—were examined under four manipulated conditions: number of Level 2 variables, Level 2 sample size, Level 1 intercept-slope correlation, and percentage of missing data. Listwise deletion, group mean substitution, and the EM algorithm performed equally well for the variable having missing values. For the variables having no missing data, listwise deletion and the EM algorithm performed satisfactorily. Only listwise deletion performed well in estimating random effects except when the Level 2 sample size was 30 and 40% of the data were missing. The practical implications of the findings are discussed.
Get full access to this article
View all access options for this article.
References
1.
Basilvesky, A., Sabourin, D., Hum, D., & Anderson, A. (1985). Missing data estimators in the general linear model: An evaluation of simulated data as an experimental design, Communications in Statistics, 14, 371-394.
2.
Bernstein, L., & Burstein, N. (1994, April 4-8). Bias vs. precision: Combining estimates in multisite evaluation research. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
3.
Bryk, A.S., & Raudenbush, S.W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage.
4.
Busing, F.M.T.A. (1993). Distribution characteristics of variance estimates in two-level models. Available from http://www.stat.ucla.edu/~kreft
5.
Donner, A., & Rosner, B. (1982). Missing value problems in multiple linear regression with two independent variables. Communications in Statistics-Theory and Methods, 11, 127-140.
6.
Donoghue, J.R., & Jenkins, F. (1992). A Monte Carlo study of the effects of model misspecification on HLM estimates (Rep. No. RR-92-69). Princeton, NJ: Educational Testing Service.
7.
Duncan, T.E., Duncan, S.C., & Li, F. (1998). A comparison of model- and multiple imputation-based approaches to longitudinal analyses with partial missingness. Structural Equation Modeling, 5, 1-21.
8.
Gleason, T.C., & Staelin, R. (1975). A proposal for handling missing data. Psychometrika, 40, 229-251.
9.
Goldstein, H. (1995). Multilevel statistical models. New York: Halsted.
10.
Graham J.W., Hofer, S.M., & MacKinnon, D.P. (1996). Maximizing the usefulness of data obtained with planned missing value patterns: An application of maximum likelihood procedures, Multivariate Behavioral Research, 31, 197-218.
11.
Gregorich, S. (1997). EM_COVAR.sas: A SAS Macro for estimating a covariance matrix and mean vector via the EM algorithm [Computer program]. Retrieved from http://mywebpage.netscape.com/segregorich/missing.html
12.
Hegamin-Younger, C., & Forsyth, R. (1998). A comparison of four imputation procedures in a two-variable prediction system. Educational and Psychological Measurement, 58, 197-210.
13.
Hill, P.W., & Goldstein, H. (1998). Multilevel modeling of educational data with cross-classification and missing identification for units. Journal of Educational and Behavioral Statistics, 23, 117-128.
14.
Kaiser, H.F., & Dickman, K. (1962). Sample and population score matrices and sample correlation matrices from an arbitrary population correlation matrix. Psychometrika, 27, 179-182.
15.
Kim, J., & Curry, J. (1977). The treatment of missing data in multivariate analysis. Sociological Methods & Research, 6, 215-240.
16.
Kim, K.-S. (1990). Multilevel data analysis: A comparison of analytical alternatives. Unpublished doctoral dissertation, University of California, Los Angeles.
17.
Kromrey, J.D., & Hines, C.V. (1994). Nonrandomly missing data in multiple regression: An empirical comparison of common missing-data treatments. Educational and Psychological Measurement, 54, 573-593.
18.
Littell, R.C., Milliken, G.A., Stroup, W.W., & Wolfinger, R.D. (1996). SAS® system for mixed models. Cary, NC: SAS Institute, Inc.
19.
Little, R.J.A., & Rubin, D.B (1987). Statistical analysis with missing data. New York: John Wiley.
20.
Longford, N.T. (1993). Random coefficient models. New York: Claredon.
21.
Raymond, M.R., & Roberts, D.M. (1987). A comparison of methods for treating incomplete data in selection research. Educational and Psychological Measurement, 47, 13-26.
22.
Roth, P.L. (1994). Missing data: A conceptual review for applied psychologists. Personnel Psychology, 47, 537-558.
23.
Roth, P.L., & Switzer, F.S. (1995). A Monte Carlo analysis of missing data techniques in a HRM setting. Journal of Management, 21, 1003-1023.
24.
Rubin, D.B. (1976). Inference and missing data. Biometrika, 61, 581-592.
25.
Rubin, D.B. (1989). Some applications of multilevel models to educational data. In R.D. Bock (Ed.), Multilevel analysis of educational data (pp. 1-17). San Diego, CA: Academic Press.
26.
SAS Institute, Inc. (1989). SAS/IML software: Usage and reference, Version 6 (1st ed.). Cary, NC: Author.
27.
Schafer, J.L. (1994). NORM [Computer program]. Retrieved from http://www.stat.psu.edu/%7ejls/misoftwa.html
28.
Schafer, J.L., & Olsen, M.K. (1998). Multiple imputation for multivariate missing data problems: A data analyst’s perspective. Retrieved from http://www.stat.psu.edu/~jls/index.html
29.
Singer, J. (1998). Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. Journal of Educational and Behavioral Statistics, 23, 323-355.
30.
Snijders, T.A.B., & Bosker, R.J. (1993). Standard errors and sample sizes for two level research. Journal of Educational Statistics, 18, 237-259.
31.
Witta, E.L. (1992). Seven methods of handling missing data using samples from a national data base. Unpublished doctoral dissertation, Virginia Polytechnic Institute and State University, Blacksburg, VA.
