Applied Psychological Measurement

A graphical comparison of empirical versus simu lated residual variation is presented as one way to as sess the goodness of fit of an item response theory model. The two forms of residual variation were gen erated through the separate calibration of empirical data and data "tailored" to fit the model, given the empirical parameter estimates. A variety of techniques illustrate the utility of using tailored residuals as a specific baseline against which empirical residuals may be understood.

An IRT analysis based on p (proportion correct) and r (item-test correlation) is proposed for a group of tests having items in common. The procedure is a generalization of a procedure proposed by De Gruijter and Mooijaart (1983) which is related to procedures for the factor analysis of dichotomous data. The pro cedure results in IRT item parameters using data from examinee groups with subsets of common items; it is, therefore, particularly appropriate for calibrating items for use in small-scale item banks. Simulated data are used to illustrate the procedure.

The accuracy of parameter estimates provided by the major computer programs for confirmatory factor analysis studies is questioned. This note demonstrates an inconsistency in parameter estimates across two of the major programs (LISREL and EQS), with the incon sistency attributed to the use of LISREL VI'S automatic start values for the estimation of generalized least squares models.

This paper describes the development of an unfold ing methodology designed to analyze "pick any" or "pick any/n" binary choice data (e.g., decisions to buy or not to buy various products). Maximum likeli hood estimation procedures are used to obtain a joint space representation of both persons and objects. A review of the relevant literature concerning the spatial treatment of such binary choice data is presented. The nonlinear logistic model type is described, as well as the alternating maximum likelihood algorithm used to estimate the parameter values. The results of an appli cation of the spatial choice model to a synthetic data set in a monte carlo analysis are presented. An appli cation concerning consumer (intended) choices for nine competitive brands of sports cars is discussed. Future research may provide a means of generalizing the model to accommodate three-way choice data.

The Mokken scale is critically discussed. It is ar gued that Loevinger's H, adapted by Mokken and ad vocated as a coefficient of scalability, is sensitive to properties of the item set which are extraneous to Mokken's requirement of holomorphy of item re sponse curves. Therefore, when defined in terms of H, the Mokken scale is ambiguous. It is furthermore ar gued that item-selection free statistical inferences con ceming the latent person order appear to be insuffi ciently based on double monotony alone, and that the Rasch model is the only item response model fulfilling this requirement. Finally, it is contended that the Mokken scale is an unfruitful compromise between the requirements of a Guttman scale and the requirements of classical test theory.

The nonparametric approach to constructing and evaluating tests based on binary items proposed by Mokken has been criticized by Roskam, van den Wol lenberg, and Jansen. It is contended that their argu ments misrepresent the objectives of this approach, that their criticisms of the role of the H coefficient in the procedures are irrelevant or erroneous, and that they fail to distinguish the inherent requirements (and limitations) of general nonparametric models and pro cedures from those of parametric ones. It is concluded that Mokken's procedures provide a useful tool for re searchers in the social sciences who wish to construct and evaluate tests for measuring theoretically mean ingful latent traits while avoiding the strong parametric assumptions of traditional item response theory.

This study compared four methods of determining the dimensionality of a set of test items: linear factor analysis, nonlinear factor analysis, residual analysis, and a method developed by Bejar (1980). Five artifi cial test datasets (for 40 items and 1,500 examinees) were generated to be consistent with the three-parame ter logistic model and the assumption of either a one- or a two-dimensional latent space. Two variables were manipulated: (1) the correlation between the traits (r = .10 or r = .60) and (2) the percent of test items measuring each trait (50% measuring each trait, or 75% measuring the first trait and 25% measuring the second trait).

While linear factor analysis in all instances over estimated the number of underlying dimensions in the data, nonlinear factor analysis with linear and quad ratic terms led to correct determination of the item di mensionality in the three datasets where it was used. Both the residual analysis method and Bejar's method proved disappointing. These results suggest the need for extreme caution in using linear factor analysis, re sidual analysis, and Bejar's method until more investi gations of these methods can confirm their adequacy. Nonlinear factor analysis appears to be the most prom ising of the four methods, but more experience in ap plying the method seems necessary before wide-scale use can be recommended.

This monte carlo study explored how four com monly used test equating methods (linear, equipercen tile, and item response theory methods based on the Rasch and three-parameter models) responded to tests of different psychometric properties. The four methods were applied to generated data sets where mean item difficulty and discrimination as well as level of chance scoring were manipulated. In all cases, examinee abil ity was matched to the level of difficulty of the tests. The results showed the Rasch model not to be very robust to violations of the equal discrimination and non-chance scoring assumptions. There were also problems with the three-parameter model, but these were due primarily to estimation and linking prob lems. The recommended procedure for tests similar to those studied is the equipercentile method.