Identifying the Unauthorized Use of Examination Material

Abstract

Item disclosure is one of the most serious threats to the validity of high stakes examinations, and identifying examinees that may have had unauthorized access to material is an important step in ensuring the integrity of an examination. A procedure was developed to identify examinees that potentially had unauthorized prior access to examination content. A standardized difference score is created by comparing examinee ability estimates for potentially exposed items to ability estimates for unexposed items. Outliers in this distribution are then flagged for further review. The steps associated with this procedure are described and followed by an example of applying the procedure. In addition, the use of this procedure is supported by the results of a simulation that models the use of unauthorized access to examination material.

Keywords

test security person-fit statistics cheating

Get full access to this article

View all access options for this article.

References

Cohen, A.S. , & Wollack, J.A. ( 2006). Test administration, security, scoring, and reporting . In R. L. Brennan (Ed.), Educational Measurement (4th ed. pp. 355-386). Westport, CT: Praeger Publishers.

Dauphinee, W.D. , & Lee, R.S. ( 2006, September). Response to a security breach of confidential, high-stakes examination content: Medical Council of Canada’s approach and lessons learned. Oral abstract presented at the annual meeting of the Association for Medical Education in Europe, Genoa, Italy.

Drasgow, F. , Levine, M.V. , & McLaughlin, M.E. (1987). Detecting inappropriate test scores with optimal and practical appropriateness indices. Applied Psychological Measurement, 11, 59-79.

McLeod, L.D. ( 1999). Detecting item memorization in the CAT environment. Applied Psychological Measurement, 23, 147-160.

Meijer, R.R. , & Sijtsma, K. ( 2001). Methodology review: Evaluation person-fit. Applied Psychological Measurement, 25, 107-135.

O’Neill, T. , Lunz, M.E. , & Thiede, K. ( 2000). The impact of receiving the same questions on consecutive computer-adaptive test administrations. Journal of Applied Measurement, 1, 131-51.

Rudner, L.M. , Bracey, G. , & Skaggs, G. ( 1996). The use of a person-fit statistic with one high quality achievement test. Applied Measurement in Education, 9, 91-109.

Wood, T.J. ( 2009). The effect of reused questions on repeat examinees. Advances in Health Sciences Education, 14, 465-473.