lzstarMix

Abstract

In a report for the Council of Chief State School Officers, Olson and Fremer (2013) recommended the use of person-fit assessment, in addition to other methods, to detect irregularities in answering behavior of the test takers. Mixed-format tests, which include both dichotomous and polytomous items, are becoming increasingly popular because of an increasing emphasis on performance tasks in the common-core assessments (e.g., Darling-Hammond & Adamson, 2010). However, there is a lack of research on person-fit assessment for mixed-format tests (Sinharay, 2015a; 2015b). One reason is that there exist few statistics and no publicly available software packages to assess person fit for mixed-format tests.

lzstarMix is a function written in R (R Core Team, 2015) to compute the l_z statistic for mixed-format tests (Drasgow, Levine, & Williams, 1985; Sinharay, 2015b). The function also computes the $l_{z}^{*}$ statistic for mixed-format tests that was recently suggested by Sinharay (2015a). The null distribution of $l_{z}^{*}$ is generally much closer than that of l_z to the standard normal distribution for mixed-format tests. The function lzstarMix currently works when the dichotomous items are modeled using the three-parameter logistic (3PL) model,¹ and the polytomous items are modeled using the generalized partial credit model (GPCM; Muraki, 1992).² The input to the function consists of the item-parameter estimates, and the item scores and estimated abilities (maximum likelihood estimate or weighted likelihood estimates) of several examinees. The output are l_z and $l_{z}^{*}$ for the examinees. The function can be used for tests that include only dichotomous items, in which case $l_{z}^{*}$ for mixed-format tests (Sinharay, 2015a) becomes identical to $l_{z}^{*}$ for dichotomous items (Snijders, 2001), or only polytomous items.

Availability

A free copy of the lzstarMix function, a user manual, a simulated data set, a real data set, and example input and output files can be obtained by contacting the author. The user manual shows examples of how another freely available R function can be used to compute the estimated abilities.

Footnotes

Author’s Note

Any opinions expressed in this publication are those of the author and not necessarily of Pacific Metrics Corporation.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Darling-Hammond

Adamson

(2010). Beyond basic skills: The role of performance assessment in achieving 21st century standards of learning (Technical report). Stanford, CA: Stanford Center for Opportunity Policy in Education, Stanford University.

Drasgow

Levine

M. V.

Williams

E. A.

(1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38, 67-86.

Muraki

(1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159-176.

Olson

J. F.

Fremer

(2013). TILSA test security guidebook: Preventing, detecting, and investigating test securities irregularities. Washington, DC: Council of Chief State School Officers.

R Core Team. (2015). R: A language and environment for statistical computing. Vienna, Austria: Author.

Sinharay

(2015a). Asymptotically correct standardization of person-fit statistics beyond dichotomous items. Psychometrika. Advance online publication. doi:10.1007/s11336-015-9465-x

Sinharay

(2015b). Assessment of person fit for mixed-format tests. Journal of Educational and Behavioral Statistics. Advance online publication. doi: 10.3102/1076998615589128

Snijders

(2001). Asymptotic distribution of person-fit statistics with estimated person parameter. Psychometrika, 66, 331-342.