Do visual chunks and planning impact performance on the graph description task in the SPEAK exam?

Abstract

This study examines how task characteristics (the number of visual chunks and the amount of planning time) and test-taker characteristics (graph familiarity) influence the perceptual and cognitive processes involved in graph comprehension, the strategies used in describing graphs, and the scores obtained on the graph description task in a semi-direct oral test. Specifically, it investigates whether providing planning time and reducing the number of visual chunks in bar and line graphs help mitigate the influence of graph familiarity on language performance on graph description tasks. These research questions were explored using a structural equation modeling approach in a framework of the influence of the interaction between task characteristics and test-taker characteristics on task performance. It was found that graph familiarity affected the overall communicative quality of participants’ performance on these graph description tasks. Graph familiarity thus represented a potential source of construct-irrelevant variance. However, reducing the number of visual chunks in a graph and providing planning time positively impacted the perceptual and cognitive processes involved in graph comprehension and helped mitigate the influence of graph familiarity. Practical and theoretical implications of this study are discussed.

Get full access to this article

View all access options for this article.

References

Aiken, L. S. and West, S. G. 1991: Multiple regression: testing and interpreting interactions. Newbury Park, CA: Sage .

Bachman, L. F. 1990: Fundamental considerations in language testing. Oxford: Oxford University Press .

Bachman, L. F. and Palmer, A. S. 1996: Language testing in practice. Oxford: Oxford University Press .

Bentler, P. M. 2002: EQS for Windows 6.0. Encino, CA: Multivariate Software, Inc.

Butterworth, B. 1980: Some constraints on models of language production. In Butterworth, B , editor, Language production. Volume 1. New York: Academic Press , 423-459.

Carpenter, P. A. and Shah, P. 1998: A model of the perceptual and conceptual processes in graph comprehension . Journal of Experimental Psychology: Applied 4, 75-100 .

Carswell, C. M. 1993: Stimulus complexity and information integration in the spontaneous interpretations of line graphs . Applied Cognitive Psychology 7, 341-357 .

Carswell, C. M. , Bates, J. R. , Pregliasco, N. R. , Lonon, A. and Urban, J. 1998: Finding graphs useful: linking preference to performance for one cognitive tool . Cognitive Technology 3, 4-18 .

Carswell, C. M. , Emery, C. and Lonon, A. 1993: Stimulus complexity and information integration in the spontaneous interpretation of line graphs . Applied Cognitive Psychology 7, 341-357 .

10.

Carswell, C. M. and Ramzy, C. 1997: Graphing small data sets: should we bother? Behavior and Information Technology 16, 61-71 .

11.

Crick, J. E. and Brennan, R. L. 1983: Manual for GENOVA: a generalized analysis of variance system. ACT Technical Bulletin No. 43. Iowa City, IA: ACT, Inc.

12.

Crookes, G. 1989: Planning and interlanguage variation . Studies in Second Language Acquisition 11, 367-383 .

13.

Ellis, R. 1987: Interlanguage variability in narrative discourse: style shifting in the use of the past tense . Studies in Second Language Acquisition 9, 1-20 .

14.

Ellis, R. 1994: The study of second language acquisition. Oxford: Oxford University Press .

15.

ETS (Educational Testing Service) 2001-02: TSE and SPEAK band descriptor chart: TSE and SPEAK score user guide. 2001-02 edition. Princeton, NJ: Educational Testing Service . Available online at: http://ftp.ets.org/pub/toefl/008659.pdf (December 2004).

16.

Foster, P. and Skehan, P. 1996: The influence of planning and task type on second langauge performances . Studies in Second Language Acquisition 18, 299-323 .

17.

Ginther, A. 2001: Effects of the presence and absence of visuals on performance on TOEFL CBT listening-comprehensive stimuli. TOEFL Research Report No. 66. Princeton, NJ: Educational Testing Service .

18.

Gruba, P. 1997: The role of video media in listening assessment . System 25, 335-346 .

19.

Gruba, P. 1999: The role of digital media in second language listening comprehension. Unpublished doctoral thesis, University of Melbourne, Melbourne, Australia.

20.

Iwashita, N. , McNamara, T. and Elder, C. 2001: Can we predict task difficulty in an oral proficiency test? Exploring the potential of an information-processing approach to task design . Language Learning 51, 401-436 .

21.

Katz, I. R. , Xi, X. , Kim, H. and Cheng, P. C.-H. 2002: Graph structure supports graph description . Proceedings of the Twenty-fourth Annual Meeting of the Cognitive Science Society, Hillsdale, NJ: Erlbaum, 157-162 .

22.

Katz, I. R. , Xi, X. , Kim, H. and Cheng, P. C.-H. 2004: Elicited speech from graph items on the test of spoken English. TOEFL Research Report No. 74. Princeton, NJ: Educational Testing Service .

23.

Kosslyn, S. M. 1989: Understanding charts and graphs . Applied Cognitive Psychology 3, 185-225 .

24.

Lance, C. E. 1988: Residual centering, exploratory and confirmatory moderator analysis, and decomposition of effects in path models containing interaction effects . Applied Psychological Measurement 12, 163-175 .

25.

Mehnert, U. 1998: The effects of different lengths of time for planning on second language performance . Studies in Second Language Acquisition 20, 83-106 .

26.

Peebles, D. , Cheng, P. C.-H. and Shadbolt, N. 1999: Multiple processes in graph-based reasoning . In Hahn, M. and Stoness, S. S. , editors, Proceedings of the Twenty First Annual Conference of the Cognitive Science Society. Vancouver.

27.

Pinker, S. 1990: A theory of graph comprehension. In Freedle, R. , editor, Artificial intelligence and the future of testing. Mahwah, NJ: Erlbaum , 73-126.

28.

Shah, P. and Carpenter, P. A. 1995: Conceptual limitations in comprehending line graphs . Journal of Experimental Psychology: General 124, 43-61 .

29.

Shah, P. , Hegarty, M. and Mayer, R. E. 1999: Graphs as aids to knowledge construction: signaling techniques for guiding the process of graph comprehension . Journal of Educational Psychology 91, 690-702 .

30.

Skehan, P. 1998: A cognitive approach to language learning. Oxford: Oxford University Press .

31.

Wigglesworth, G. 1995: The effect of planning time on oral test discourse . Paper presented at the 17th Language Testing Research Colloquium, March, Baltimore, MD.

32.

Wigglesworth, G. 1997: An investigation of planning time and proficiency level on oral test discourse . Language Testing 14, 85-106 .

33.

Wigglesworth, G. 2001: Influences on performance in task-based oral assessments. In Bygate, M. , Skehan, P. and Swain, M. , editors, Researching pedagogical tasks: second language learning, teaching and testing. Harlow: Pearson Education Limited , 186-209.

34.

Xi, X. 2003: Investigating language performance on the graph description task in a semi-direct oral test. Unpublished doctoral dissertation, University of California, Los Angeles, CA.