Organizational Research Methods

Recently, there has been increased interest in tests of measurement equivalence/ invariance (ME/I). This study uses simulated data with known properties to assess the appropriateness, similarities, and differences between confirmatory factor analysis and item response theory methods of assessing ME/I. Results indicate that although neither approach is without flaw, the item response theory–based approach seems to be better suited for some types of ME/I analyses.

This article describes the use of neural networks as an alternative method to investigate the links between various dimensions of culture and perceptions of justice and demonstrates their ability to model the data relationships with higher accuracy than multiple regression analysis. A complete discussion of the development and validation of the neural network models is included as a guide to researchers in management who are interested in exploring this methodology.

Organizational data are inherently nested; consequently, lower level data are typically influenced by higher level grouping factors. Stated another way, almost all lower level organizational data have some degree of nonindependence due to work group, geographic membership, and so on. Unaccounted-for nonindependence can be problematic because it affects standard error estimates used to determine statistical significance. Currently, researchers interested in modeling higher level variables routinely use multilevel modeling techniques to avoid well-known problems with Type I error rates. In this article, however, the authors examine how nonindependence affects statistical inferences in cases in which researchers are interested only in relationships among lower level variables. They show that ignoring nonindependence when modeling only lower level variables reduces power (increases Type II errors), and through simulations, the authors show where this loss of power is most pronounced.

Employee selection often involves a series of sequential tests (or hurdles). However, validation strategies under this complex design are not found in the literature. Missing is a discussion of the statistical properties important in establishing criterion-related validity in multiple-hurdle designs. The authors address this gap in the literature by suggesting a general statistical model for range restriction corrections. Because the multiple-hurdle design includes as special cases predictive and concurrent designs, the corrections apply also to these designs. The general correction model is based on algorithms from the missing data literature. Two missing data procedures are examined: the estimation-maximization procedure and the Bayesian multiple imputation (MI) procedure. These procedures are large-sample equivalent and often yield similar results. The MI procedure, however, has the added advantage of providing easily obtainable standard errors. A hypothetical example of a multiple-hurdle design is used to illustrate the procedures.

A common research problem in validation studies is the estimation of the population correlation between predictor X and performanceY from a non–randomly selected sample. Procedures for getting unbiased estimates of population correlations in a limited set of conditions in which no rejection of job offers is assumed have been developed. However, in applied selection settings, it is very likely that some of the candidates who have received job offers reject them through a selfselection process. If an estimation model based on the assumption that there is no rejection of job offers via self-selection is used, estimates of population parameters may be biased due to model misspecification. In the current study, a procedure is developed that is applicable to a variety of realistic validation settings, including a setting in which both institutional selection and applicant’s rejection of job offers are involved. Data requirements of the procedure are also discussed.

Reviewing research on diversity and relational demography in teams and work groups, the authors compare different ways of measuring gender composition and demonstrate that existing practice can be theoretically biased. The authors conclude that within group-level analyses, the proportion of women should be used; whereas within individual-level analyses, the appropriate approach depends on whether a gender-by-gender composition interaction effect is found. The generalizability of this approach to other types of diversity is also discussed.