irtplay : An R Package for Online Item Calibration,Scoring,Evaluation of Model Fit,and Useful Functions for Unidimensional IRT

Description

The R package irtplay provides practical tools for unidimensional item response theory (IRT) models that conveniently enable users to conduct many analyses related to IRT such as calibrating online items, scoring test-takers’ proficiencies, and evaluating IRT model-data fit. The irtplay software package is quite user-friendly and its functions are easy to implement. In addition, the irtplay package supports mixed-item formats consisting of dichotomous and polytomous items. For the polytomous items, the graded response model and the (generalized) partial credit models are available.

Online Item Calibration Using Stocking’s Method A

In computerized adaptive testing (CAT), online items, which include pretest items and operational items, should be calibrated precisely to ensure a healthy item bank (Stocking, 1988). The irtplay package calibrates the online items using Stocking’s (1988) Method A, which is the maximum likelihood estimation (MLE) of the item parameters given the proficiency estimates. Method A is known to result in accurate, unbiased item parameter calibration when items are randomly administered to examinees (Chen & Wang, 2016; He et al., 2017). Although Method A is the simplest and most straightforward online calibration method in current use, there is little software available to support it. The updated version of irtplay will include fixed item parameter calibration (FIPC; Kim, 2006).

Estimation of Proficiency Parameters

For estimation of proficiency parameters, six popular scoring methods can be implemented using the irtplay package. Available methods include MLE, MLE with fences (MLEF; Han, 2016), maximum a posteriori (MAP) estimation (Hambleton et al., 1991), expected a posteriori (EAP) estimation (Bock & Mislevy, 1982), EAP summed scoring (Thissen & Orlando, 2001), and inverse test characteristic curve scoring (Kolen & Tong, 2010). Although much IRT software has been developed, there are few software packages available to estimate proficiency parameter with the mixed-item format.

Evaluation of IRT Model-Data Fit on Item Level

In terms of assessing the IRT model-data fit, the irtplay package supports computation of popular item-fit statistics. Available fit statistics include χ² fit statistic (Bock, 1960; Yen, 1981), likelihood ratio χ² fit statistic (G2; McKinley & Mills, 1985), infit and outfit statistics (Ames & Penfield, 2015), and S-X² (Orlando & Thissen, 2000). In addition, irtplay provides users with graphical displays to examine residuals between the observed data and model-based prediction for items (Hambleton et al., 1991).

Useful Functions Related to IRT

The irtplay package also includes many other convenient functions such as importing item and/or proficiency parameter estimates from the output of popular IRT software—Bilog-MG, PARSCALE, flexMIRT, and mirt (R package), for example. The software also enables users to implement flexMIRT software through R console, generate simulated item response data, compute asymptotic variance–covariance matrices of item parameter estimates (Li & Lissitz, 2004), compute conditional distribution of observed scores using the Lord and Wingersky (1984) recursion formula, compute item and test information functions, compute item and test characteristic curves, and plot item and test characteristic curves and item and test information functions.

Availability and Distribution

The R package irtplay and its reference manual are available for free from the website http://www.r-project.org and works on Windows, Linux, and MacOS platforms. Version 3.6.0 (or later) of the R software should be installed for optimal functioning of the irtplay package.

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material

Supplemental Material