A construct definition study of a standardized ESL vocabulary test

Abstract

This paper reports a construct definition study of a standardized English as a second language (ESL) vocabulary test. The test, which contained 20 stimulus word items and 17 context word items, was administered to two different samples of ESL students at different proficiency levels and to a sample of native speakers of English. The item responses for the 40 items were used as the dependent variable in a stepwise multiple regression, construct specification equation and 12 variables (measures of the stimulus words' and keyed responses' fluency, distribution, abstractness, length, affective impact, and prototypicality) were used as predictors. Only six of the predictors accounted for a significant amount of variance in the dependent variable, and the number and arrangement of significant predictors varied across proficiency strata, indicating the utility of incorporating the examinees' stages of learning into construct validation studies.

Construct definition studies such as the current one permit the statement of explicit predictions and a means to falsify construct theory, which is a perceived need in educational and psychological measurement.

The paper concludes with an agenda for future research based on the results of the present study.

Get full access to this article

View all access options for this article.

References

Anderson, R.C. 1982: Acquisition of cognitive skill. Psychological Review 89, 369-406.

Carroll, J.B. 1976: Psychometric tests as cognitive tasks: A new 'structure of intellect'. In Resnick, L.B. , editor, The nature of intelligence, Hillsdale, New Jersey: Lawrence Erlbaum Associates.

—, Davies, P. and Richman, B. 1971: Word frequency book Boston: Houghton Mifflin.

Cronbach, L.J. , and Meehl, P.E. 1955: Construct validity in psychological tests. Psychological Bulletin 52, 281-302.

Embretson, S.E. 1983: Construct validity: Construct representation versus nomothetic span. Psychological Bulletin 93, 179-97.

Guilford, J.P. 1956: The structure of intellect. Psychological Bulletin 53, 267-93.

Guralnik, D.B. , editor in chief, 1984: Webster's new world dictionary of the American language, second college edition . New York: Simon and Schuster.

Lakoff, G. 1975: Hedges: A study in meaning criteria and the logic of fuzzy sets. Papers from the Eighth Regional Meeting Chicago Linguistics Society . Chicago: University of Chicago , Linguistics Department.

Messick, S. 1984: The psychology of educational measurement. Journal of Educational Measurement 21, 215-37.

10.

Miller, S.K. , Plake, B.S. and Patience, W.M. 1987: Differential item and test performance between males and females: The influence of item arrangement. Paper presented at the annual meeting of the National Council on Measurement in Education, Washington, DC, USA.

11.

Osgood, C.E. 1964: Semantic differential technique in the comparative study of cultures. American Anthropologist 66, 171-200.

12.

—, Osgood, C.E. , Suci, J. and Tannenbaum, P.H. 1957: The measurement of meaning . Urbana, Illinois: University of Illinois Press.

13.

—, and McGuigan, F.J. 1973: Psychophysiological correlates of meaning: essences or traces? In McGuigan, F.J. and Schoonover, R.A. , editors, The psychophysiology of thinking. Studies of covert processes, New York: Academic Press.

14.

Perkins, K. and Mouw, J.T. 1987: A framework for conducting ESL/EFL construct validation studies. Paper presented at the Language Testing Research: Priorities and Prospects Symposium, Reading, UK.

15.

Rosch, E. 1973: On the internal structure of perceptual and semantic categories . In Moore, T.E. , editor, Cognitive development and the acquisition of language, New York: Academic Press.

16.

Rosch, E. and Mervis, C.B. 1975: Family resemblances: studies in the internal structure of categories. Cognitive Psychology 7, 573-605.

17.

Rumelhart, D.E. and Norman, D.A. 1978: Accretion, tuning, and restructuring: three modes of learning . In Cotton, J.W. and Klatzky, R. , editors, Semantic factors in cognition, Hillsdale, New Jersey: Lawrence Erlbaum.

18.

Schaeffer, B. and Wallace, R. 1969: Semantic similarity and the comparison of word meanings. Journal of Experimental Psychology 82, 343-46.

19.

Smith, E.E. , Shoben, E.J. and Rips, L.J. 1974: Structure and process in semantic memory: a feature model for semantic decisions. Psychological Review 81, 214-41.

20.

Stenner, A.J. , Smith, M. , III and Burdick, D.S. 1983: Toward a theory of construct definition. Journal of Educational Measurement 20, 305-16.

21.

Thurstone, L.L. 1935: Vectors of the mind. Chicago: University of Chicago Press.

22.

Upshur, J.A. and Peterson, J. 1968: Michigan test of English language proficiency, Form F . Ann Arbor, Michigan: University of Michigan, The English Language Institute Testing Service.

23.

Zadeh, L. 1965: Fuzzy sets. Information and Control 8, 338-53.