The Pearson correlation is used to depict effect sizes in the context of item response theory. Amultidimensional Rasch model is used to directly estimate the correlation between latent traits. Monte Carlo simulations were conducted to investigate whether the population correlation could be accurately estimated and whether the bootstrap method provided a good approximation of the distribution of the sample correlation. The independent variables were (a) the item response model, (b) test length, (c) sample size, and (d) the magnitude of correlation. The dependent variable was the empirical correlation coefficient. The results indicate that the population correlation can be accurately estimated and that the bootstrap method yields very a good approximation of the sampling distribution.
Adams, R. J., Wilson, M. R.,& Wang, W.-C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1-23.
2.
American Psychological Association. (2001). Publication manual of the American Psychological Association(5th ed.). Washington, DC: Author.
3.
Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-573.
4.
Baker, F. B. (1992). Item response theory: Parameter estimation techniques. NewYork: Dekker.
5.
Baugh, F. (2002). Correcting effect sizes for score reliability: A reminder that measurement and substantive issues are linked inextricably. Educational and Psychological Measurement, 62, 254-263.
6.
Béguin, A. A.,& Glas, C. A.W. (2001). MCMCestimation and model-fit analysis of multidimensional IRT models. Psychometrika, 66, 541-562.
7.
Bickel, P., & Freedman, D. (1981). Some asymptotic theory for the bootstrap. Annals of Statistics, 9, 1196-1217.
8.
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 397-479). Reading, MA: Addison-Wesley.
9.
Bock, R. D.,& Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of the EM algorithm. Psychometrika, 46, 443-459.
10.
Bock, R. D., Gibbons, R.,& Muraki, E. J. (1988). Full information item factor analysis. AppliedPsychological Measurement, 12, 261-280.
11.
Bock, R. D.,& Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444.
12.
Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, NJ: Lawrence Erlbaum.
13.
Cohen, J. (1969). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
14.
Davison, A. C., & Hinkley, D. V. (1997). Bootstrap methods and their application. Cambridge, UK: Cambridge University Press.
15.
De Boeck, P.,& Wilson, M. R. (Eds.). (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York: Springer-Verlag.
16.
DeMars, C. (2001). Group difference based on IRT scores: Does the model matter?Educational and Psychological Measurement, 61, 60-70.
17.
Dempster, A. P., Laird, N. M.,& Rubin, D. B. (1977). Maximumlikelihood fromincomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39B, 1-38.
18.
DiCicco, T. J., & Efron, B. (1996). Bootstrap confidence intervals. Statistical Science, 11, 189-228.
19.
Efron, B. (1979). Bootstrap methods: Another look at the jackknife. The Annals of Statistics, 7, 126-126.
20.
Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.
21.
Eliason, S. R. (1993). Maximum likelihood estimation: Logic and practice. Newbury Park, CA: Sage.
22.
Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
23.
Fischer, G. H. (1973). The linear logistic test model as instrument in educational research. Acta Psychologica, 37, 359-374.
24.
Fischer, G. H., & Molenaar, I. W. (Eds.). (1995). Rasch models: Foundations, recent developments, and applications. New York: Springer.
25.
Fischer, G. H.,& Pononcy, I. (1994). An extension of the partial credit model with an application to the measurement of change. Psychometrika, 59, 177-192.
26.
Fisher, R. A. (1915). Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika, 10, 507-521.
27.
Fisher, R. A. (1921). On the probable error of a coefficient of correlation deduced from a small sample. Metron, 1, 1-32.
28.
Fraser, C. (1988). NOHARM: A Fortran program for fitting unidimensional and multidimensional normal ogive models of latent trait theory [Computer software]. Armidale, Australia: University of New England, Centre for Behavioral Studies.
29.
Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer-Nijhoff.
30.
Hedges, L. V.,& Olkin, L. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
31.
Henson, R. K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34, 177-189.
32.
Hotelling, H. (1953). New light on the correlation coefficient and its transforms. Journal of the Royal Statistical Society, 15B, 193-232.
33.
LePage, R., & Billiard, L. (1992). Exploring the limits of bootstrap. New York: John Wiley.
34.
Linacre, J. M. (1989). Many-facet Rasch measurement. Chicago: Measurement, Evaluation, Statistics, and Assessment Press.
35.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
36.
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading MA: Addison-Wesley.
37.
Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
38.
Maxwell, S. E., & Delaney, H. D. (1990). Designing experiments and analyzing data: A model comparison perspective. Belmont, CA: Wadsworth.
39.
McCullagh, P., & Nelder, J. A. (1989). Generalized linear models(2nd ed.). London: Chapman & Hall.
40.
McCulloch, C. E., & Searle, S. R. (2001). Generalized, linear, and mixed models. New York: John Wiley.
41.
McDonald, R. P. (1982). Linear versus nonlinear models in latent trait theory. Applied Psychological Measurement, 6, 379-396.
42.
McKinley, R. L., & Reckase, M. D. (1983). MAXLOG: A computer program for the estimation of the parameters of a multidimensional logistic model. Behavior Research Methods and Instrumentation, 15, 389-390.
43.
Muraki, E. (1993). POLYFACT [Computer software and manual]. Princeton, NJ: Education Testing Service.
44.
Muraki, E., & Carlson, J. E. (1995). Full-information factor analysis for polytomous item responses. Applied Psychological Measurement, 19, 73-90.
45.
Nelder, J. A., & Wedderburn, R.W.M. (1972). Generalized linear models. Journal of the Royal Statistical Society, 135A, 370-384.
46.
Pearson, F. (1896). Mathematical contributions to the theory of evolution, III. Regression heredity and panmixia. PhilosophicalTransactions of the Royal Society of London, 187A,253-318.
47.
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Institute of Educational Research.
48.
Reckase, M. D. (1985). The difficulty of test items that measure more than one dimension. Applied Psychological Measurement, 9, 401-412.
49.
Rijmen, F., Tuerlinckx, F., De Boeck, P.,& Kuppens, P. (2003). Anonlinear mixed model framework for item response theory. Psychological Methods, 8, 185-205.
50.
SASInstitute, Inc. (1999). TheNLMIXEDprocedure [Computer software]. Cary, NC: Author.
51.
Thompson, B., & Snyder, P. A. (1998). Statistical significance and reliability analyses in recent JCD research articles. Journal of Counseling and Development, 76, 431-436.
52.
Thompson, B.,& Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement, 60, 174-195.
53.
Vacha-Haase, T., Ness, C., Nilsson, J., & Reetz, D. (1999). Practices regarding reporting of reliability coefficients: A review of three journals. Journal of Experimental Education, 67, 335-341.
54.
van der Linden, W. J., & Hambleton, R. K. (Eds.). (1997). Handbook of modern item response theory. New York: Springer.
55.
Volodin, N., & Adams, R. J. (1995, April). Identifying and estimating a D-dimensional Rasch model. Paper presented at the International Objective MeasurementWorkshop, University of California at Berkeley.
56.
Wang, W.-C. (1999). Direct estimation of correlations among latent traits within IRT framework. Methods of Psychological Research Online, 4(2), 47-68.
57.
Wang, W.-C.,& Chen, H.-C. (2004). The standardized mean difference within the framework of item response theory. Educational and Psychological Measurement, 64, 201-223.
58.
Wang, W.-C., & Wu, C.-I. (2004). Gain score in item response theory as an effect size measure. Educational and Psychological Measurement, 64, 758-780.
59.
Wilson, D. T., Wood, R., & Gibbons, R. (1991). TESTFACT: Test scoring, item statistics, and item factor analysis [Computer software]. Mooresville, IN: Scientific Software.
60.
Wilson, M. R. (1992). The partial order model: An extension of the partial credit model. Applied Psychological Measurement, 16, 309-325.
61.
Wolfinger, R. D., & SAS Institute, Inc. (n.d.). Fitting nonlinear mixed models with the new NLMIXED procedure. Retrieved August 17, 2003, from http://support.sas.com/rnd/app/papers/nlmixedsugi.pdf
62.
Wright, B. D., & Masters, G. N. (1982). Rating scale analysis. Chicago: Measurement, Evaluation, Statistics, and Assessment Press.
63.
Wu, M. L., Adams, R. J.,& Wilson, M. R. (1998). ACER ConQuest: Generalized item response modeling software [Computer software and manual]. Camberwell, Australia: Australian Council for Educational Research.
