The present article describes the potential utility of item response theory (IRT) and adaptive testing for scale evaluation and for web-based career assessment. The article describes the principles of both IRT and adaptive testing and then illustrates these with reference to data analyses and simulation studies of the Career Confidence Inventory (CCI). The kinds of information provided by IRT are shown to give a more precise look at scale quality across the trait continuum and also to permit the use of adaptive testing, where the items administered are tailored to the individual being tested. Such tailoring can significantly reduce testing time while maintaining high quality of measurement. This efficiency is especially useful when multiscale inventories and/or a large number of scales are to be administered. Readers are encouraged to consider using these advances in career assessment.
Betz, N. ( 1977). Effects of immediate knowledge of results and adaptive testing on ability test performance. Applied Psychological Measurement, 1, 259-266. doi:10.1177/014662167700100212.
2.
Betz, N.E., & Borgen, F.H. ( 2006). Manual for the CAPA Confidence Inventory. Columbus, OH: Career and Personality Assessments, Inc .
3.
Betz, N.E., & Borgen, F.H. ( 2009). Comparative effectiveness of CAPA and FOCUS online: Career assessment systems with undecided college students. Journal of Career Assessment, 17, 351-366. . doi:10.1177/ 1069072709334229.
4.
Betz, N.E., Borgen, F.H., Rottinghaus, P., Paulsen, A., Halper, C., & Harmon, L.W. ( 2003). The Expanded Skills Confidence Inventory: Measuring basic dimensions of vocational activity. Journal of Vocational Behavior , 62, 76-100. doi:10.1016/S0001-8791(02)00034-9.
5.
Bock, R.D. ( 1997). A brief history of item response theory. Educational Measurement: Issues and practice, 16, 21-23. doi:10.1111/j.1745-3992.1997.tb00605.x.
6.
Borgen, F., & Betz, N. ( 2008). Career self-efficacy and personality: Linking the Career Confidence Inventory and the Healthy Personality Inventory. Journal of Career Assessment, 16, 22-43. doi:10.1177/ 1069072707305770.
7.
Fowkes, K.M., & McWhirter, E.H. (2007). Evaluation of computer-assisted career guidance in middle and secondary education settings: Status, obstacles, and suggestions. Journal of Career Assessment, 15, 388-400. doi:10.1177/1069072707301234.
8.
Fukuyama, M., Probert, B., Neimeyer, G., Nevill, D., & Metzler, A. ( 1988). Effects of DISCOVER on career self-efficacy and decision making of undergraduates. Career Development Quarterly, 37, 56-62.
9.
Garis, J., & Niles, S. ( 1990). The separate and combined effects of SIGI and DISCOVER and a career planning course on undecided university students. Career Development Quarterly, 38, 261-274.
10.
Gore, P.A., Bobek, B., Robbins, S., & Shayne, L. ( 2006). Computer-based career exploration: Usage patterns and a typology of users. Journal of Career Assessment, 14, 421-436.
11.
Gore, P.A., & Leuwerke, W.C. (2000). Information technology for career assessment on the internet. Journal of Career Assessment , 8, 3-20. doi:10.1177/106907270000800102.
12.
Hafsteinsson, L.G., Donovan, J.J., & Breland, B.T. ( 2007). An item response theory examination of two popular goal orientation measures. Educational and Psychological Measurement , 67, 719-739.
13.
Harris-Bowlsby, J., & Sampson, J.P. Jr. (2001). Computer-assisted career guidance planning systems: Dreams and realties. Career Development Quarterly, 49, 250-260.
14.
Holland, J.L. ( 1997). Making vocational choices: A theory of vocational personalities and work environments (3rd ed.). Odessa, FL: Psychological Assessment.
15.
Katz, M.R. ( 1973). Career decision making: A computer-based system of interactive guidance and information (SIGI). In: Proceedings of the Invitational Conference on Testing Problems (pp. 43-69). Princeton, NJ: Educational Testing Service.
16.
Luzzo, D., & Pierce, G. ( 1996). Effects of DISCOVER on the career maturity of middle school students. Career Development Quarterly, 45, 170-172.
17.
Mills, C., & Steffen, M. ( 2000). The GRE Computer Adaptive Test: Operational Issues. In W. van der Linden & C. Glas (Ed.), Computerized adaptive testing: Theory and practice (pp. 7599). Dordrecht, Netherlands: Kluwer Academic Publishers .
18.
Oliver, L., & Chartrand, J. (Ed.). (2000).Career assessment and the internet: Special issue. Journal of Career Assessment, 8, 1-104. doi:10.1177/106907270000800101.
19.
Peterson, G.W., Ryan-Jones, R.E., Sampson, J.P. Jr., Reardon, R.C., & Shahnasarian, M. (1994). A comparison of the effectiveness of three computer-assisted career guidance systems: DISCOVER, SIGI, and SIGI-PLUS . Computers in Human Behavior, 10, 189-198.
20.
Rayman, J.R., & Harris-Bowslby, J. (1977). Discover: A model for a systematic career guidance program. Vocational Guidance Quarterly, 26, 3-12.
21.
Reile, D.M., & Harris-Bowlsby, J. (2000). Using the internet in career planning and placement. Journal of Career Assessment , 8, 69-84. . doi:10.1177/106907270000800106.
22.
Rizopoulos, D. ( 2006). ltm: An R package for latent variable modeling and item response theory analyses. Journal of Statistical Software , 17, 1-25.
23.
Samejima, F. ( 1969). Estimation of latent ability using a response pattern of graded scores. Psychometrica Monograph, 17, 1-100.
24.
Samejima, F. ( 1997). Graded response model. In: W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 85-100). New York, NY: Springer.
25.
Scherbaum, C.A., Cohen-Charash, Y., & Kern, M.J. ( 2006). Measuring general self-efficacy: A comparison of three measures using item response theory. Educational and Psychological Measurement, 66, 1047-1063. doi:10.1177/0013164406288171.
26.
Thissen, D., & Orlando, M. ( 2001). Item response theory for items scored in two categories . In: D. Thissen & H. Wainer (Eds.), Test scoring (pp. 73-140). Mahwah, NJ: Lawrence Erlbaum.
27.
Turner, B., Betz, N., Edwards, M., & Borgen, F. ( 2010). Psychometric examination of an inventory of self-efficacy for the Holland themes using Item Response Theory. Measurement and Evaluation in Counseling and Development, 43, 188-198.
28.
Wainer, H. ( 2000). Computerized adaptive testing: A primer. Mahwah, NJ: Lawrence Erlbaum.
29.
Wainer, H., & Thissen, D. ( 2001). True score theory: The traditional method. In: D. Thissen & H. Wainer (Eds.), Test scoring (pp. 73-140). Mahwah, NJ: Lawrence Erlbaum.
30.
Wall, J., Baker, H., & Sampson, J. ( 2004). Editorial comments for the special issue on the use of technology in assessment. Measurement and Evaluation in Counseling and Development, 37, 66-69.
31.
Career assessment and new technology on the internet. Journal of career assessment, 18, 315-361.
32.
Weiss, D. ( 2004). Computerized adaptive testing for effective and efficient measurement in counseling and education. Measurement and Evaluation in Counseling and Development, 37, 70-84.
33.
Zickar, M.J., Russel, S.S., Smith, C.S., Bohle, P., & Tilley, A.J. ( 2002). Evaluating two morningness scales with item response theory. Personality and Individual Differences, 33, 11-24.
