Dimensionality and construct validity of language tests

Abstract

At times 'unidimensional' tests have been characterized as being unduly constrained in what they can measure, and ultimately as lacking construct validity for communicative language assessment since communicative lan guage ability is held to be 'multidimensional' in nature. This simulation study considers the effects on statistical measures of test dimensionality that result from systematic sampling variation in both a single and a double-trait assessment model. Results are offered as evidence that there are both psychological and psychometric states of test dimensionality that are distinct, and that psychometric unidimensionality is not at all exclusive of psycho logical multidimensionality. Moreover, it is argued that psychometric uni dimensionality may often be rather more reflective than detractive of construct validity, even when complex multidimensional constructs underlie performance on the test.

Get full access to this article

View all access options for this article.

References

Bachman, L.F. 1989: Response to Henning. Correspondence Language Testing 6, 217-29.

Boldt, R.F. 1991: Personal communication.

Carroll, J.B. 1945: The effect of difficulty and chance success on correlations between items or between tests Psychometrika 10, 1-19.

Davidson, F.G. 1988: An exploratory modeling of the trait structures of some existing language test datasets. A doctoral dissertation Los Angeles: University of California.

Goldstein, H. 1980: Dimensionality, bias, independence and measurement scale problems in latent trait test score models British Journal of Mathematical and Statistical Psychology 33, 234-46.

Gulliksen, H. 1987: Theory of mental tests Hillsdale, NJ : Lawrence Earlbaum.

Hambleton, R.K. and Swaminathan, H. 1985: Item response theory: Principles and applications Boston: Klumer-Nijhoff.

Hattie, J. 1985: Methodology review: Assessing uni-dimensionality of tests and items Applied Psychological Measurement 9, 139-64.

Henning, G. 1988: The influence of test and sample dimensionality on latent trait person ability and item difficulty calibrations Language Testing 5, 83-99.

10.

Henning, G. 1989: Meanings and implications of the principle of local independence Language Testing 6, 95-108.

11.

Messick, S. 1987: Validity Research Report 87-40. Princeton, NJ: Educational Testing Service.

12.

Also appears in Linn, R.L., editor, Educational Measurement. Third edition. New York: Macmillan, 1988.

13.

Traub, R.E. 1983: A priori considerations in choosing an item response model In Hambleton, R.K., editor, Applications of item response theory. Vancouver, B.C.: Educational Research Institute of British Columbia.

14.

Way, W. and McKinley, R.L. 1992: The feasibility of modeling secondary TOEFL ability dimensions using multidimensional IRT models TOEFL Technical Report TR-5. Princeton, NJ: Educational Testing Service.