A Factor Analysis of Direct and Indirect Measures of English as a Second Language Writing Ability

Abstract

This paper reports the results of a study to determine whether an analysis of objective test measures and holistically scored composition data would support a single or a multifactor theory of writing assessment and to determine which category of measure-direct or indirect-was the more reliable. Four experimental objective tests of writing ability based on an information-processing approach were administered to one hundred foreign university students for whom English was not their native language. The subjects also wrote descriptive essays which were holistically graded. The results indicated that the holistic and objective assessments were relatively independent measures of writing ability, indicating support for a two-factor theory of ability assessment. The holistic assessment was found to be more reliable than the objective assessment.

Get full access to this article

View all access options for this article.

References

Benton, S. L. and Kiewra, K. A. (1986). Measuring the organizational aspects of writing ability. Journal of Educational Measurement, 23, 377-386.

Braddock, R. , Lloyd-Jones, R. , and Schoer, L. (1963). Research in written composition. Urbana, IL: National Council of Teachers of English.

Breland, H. and Gaynor, J. (1979). A comparison of direct and indirect assessment of writing skill. Journal of Educational Measurement, 16, 119-128.

Carroll, J. B. , Davies, P. , and Richman, B. (1971). Word frequency book. Boston: Houghton Mifflin.

Diederich, P. B. (1974). Measuring growth in English. Urbana, IL: National Council of Teachers of English.

Embretson, S. E. (1983). Construct validity: construct representation versus nomothetic span. Psychological Bulletin, 93, 179-197.

Hales, L. W. and Tokar, E. (1975). The effect of the quality of preceding responses on the grades assigned to subsequent responses to an essay question. Journal of Educational Measurement, 12, 115-117.

Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179-185.

Jacobs, H. L. , Zingraf, S. A. , Wormuth, D. R. , Hartfiel, V. F. , and Hughey, J. B. (1981). Testing ESL composition: A practical approach. Rowley, MA: Newbury House Publishers, Inc.

10.

Troyka, L. Q. (1982). Looking back and forward. In K. L. Green-berg , H. S. Weiner , and R. A. Donovan (Eds.), Notes from the National Testing Network in Writing, October. New York: City University of New York, Instructional Resource Center.

11.

van Dijk, T. A. and Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic Press.