Journal of Educational and Behavioral Statistics

Using explicit formulas for the information matrix of ML factor analysis under multivariate normal theory, gross and net information for estimating the parameters in a covariance structure gained by adding the associated mean structure are defined. It is proved that a necessary and sufficient condition for a non-null net information gain is that the identified mean structure is not saturated, provided that the factor mean is not a zero vector. Under this necessary and sufficient condition for the mean structure, asymptotic variances of the ML estimates for some covariance structure parameters will be reduced. In light of this Theorem, we discuss some recently proposed mean and covariance structure models. For some of these models, adding the associated mean structure will not improve the estimation of the parameters of interest. We argue that in such cases it may be better not to model the means. In other cases where the identified mean structure is not saturated, modeling the means should be seriously considered. Using numerical examples, we illustrate that the reduction of asymptotic variances for factor loadings can be quite substantial when a mean structure is added. We also demonstrate similar numerical results for a two-group model. Limitations and applicability of our results are discussed.

We consider maximum likelihood (ML) estimation of mean and covariance structure models when data are missing. Expectation maximization (EM), generalized expectation maximization (GEM), Fletcher-Powell, and Fisher-scoring algorithms are described for parameter estimation. It is shown how the machinery within a software that handles the complete data problem can be utilized to implement each algorithm. A numerical differentiation method for obtaining the observed information matrix and the standard errors is given. This method also uses the complete data program machinery. The likelihood ratio test is discussed for testing hypotheses. Three examples are used to compare the cost of the four algorithms mentioned above, as well as to illustrate the standard error estimation and the test of hypothesis considered. The sensitivity of the ML estimates as well as the mean imputed and listwise deletion estimates to missing data mechanisms is investigated using three artificial data sets that are missing completely at random (MCAR), missing at random (MAR), and neither MCAR nor MAR.

Three alternative procedures to adjust significance levels for multiplicity are the traditional Bonferroni technique, a sequential Bonferroni technique developed by Hochberg (1988), and a sequential approach for controlling the false discovery rate proposed by Benjamini and Hochberg (1995). These procedures are illustrated and compared using examples from the National Assessment of Educational Progress (NAEP). A prominent advantage of the Benjamini and Hochberg (B-H) procedure, as demonstrated in these examples, is the greater invariance of statistical significance for given comparisons over alternative family sizes. Simulation studies show that all three procedures maintain a false discovery rate bounded above, often grossly, by α (or α/2). For both uncorrelated and pairwise families of comparisons, the B-H technique is shown to have greater power than the Hochberg or Bonferroni procedures, and its power remains relatively stable as the number of comparisons becomes large, giving it an increasing advantage when many comparisons are involved. We recommend that results from NAEP State Assessments be reported using the B-H technique rather than the Bonferroni procedure.

Formulas for estimating sample sizes are presented to provide specified levels of power for tests of significance from a longitudinal design allowing for subject attrition. These formulas are derived for a comparison of two groups in terms of single degree-of-freedom contrasts of population means across the study timepoints. Contrasts of this type can often capture the main and interaction effects in a two-group repeated measures design. For example, a two-group comparison of either an average across time or a specific trend across time (e.g., linear or quadratic) can be considered. Since longitudinal data with attrition are often analyzed using an unbalanced repeated measures model (with a structured variance-covariance matrix for the repeated measures) or a random-effects model for incomplete longitudinal data, the variance-covariance matrix of the repeated measures is allowed to assume a variety of forms. Tables are presented listing sample size determinations assuming compound symmetry, a first-order autoregressive structure, and a non-stationary random-effects structure. Examples are provided to illustrate use of the formulas, and a computer program implementing the procedure is available from the first author.