Information provided by examination of the skills that underlie holistic scores can be used not only as supporting evidence for the validity of inferences associated with performance tests but also as a way to improve the scoring rubrics, descriptors, and benchmarks associated with scoring scales. As fluency is considered a critical, perhaps foundational, component of speaking proficiency, temporal measures of fluency are expected to be strongly related to holistic ratings of speech quality.This study examines the relationships among selected temporal measures of fluency and holistic scores on a semi-direct measure of oral English proficiency. The spoken responses of 150 respondents to one item on the Oral English Proficiency Test (OEPT) were analyzed for selected temporal measures of fluency. The examinees represented three first language backgrounds (Chinese, Hindi, and English) and the range of scores on the OEPT scale. While strong and moderate correlations between OEPT scores and speech rate, speech time ratio, mean length of run, and the number and length of silent pauses were found, fluency variables alone did not distinguish adjacent levels of the OEPT scale. Temporal measures of fluency may reasonably be selected for the development of automated scoring systems for speech; however, identification of an examinee’s level remains dependent on aspects of performance only partially represented by fluency measures.
Anderson JR ( 1983). The architecture of cognition. Cambridge, MA: Harvard University Press.
2.
Anderson-Hsieh J., Koehler K. ( 1988). The effect of foreign accent and speaking rate on native speaker comprehension. Language Learning, 38, 561-598.
3.
Anderson-Hsieh J., Venkatagiri H. ( 1994). Syllable duration and pausing in the speech of Chinese ESL speakers. TESOL Quarterly, 807-812.
4.
Bialystok E. ( 1982). On the relationship between knowing and using linguistic forms. Applied Linguistics, 3, 181-206.
5.
Blake CG ( 2006). The potential of text-based internet chats for improving ESL oral fluency. Unpublished doctoral dissertation, Purdue University, USA.
6.
Boersma P., Weenink D. ( 2009). Praat, Version 5.1.15. Retrieved from http://www.fon.hum.uva.nl/praat.
7.
Brennan S., Williams M. ( 1995). The feeling of another’s knowing: Prosody and filled pauses as cues to listeners about the metacognitive states of speakers . Journal of Memory and Language, 38, 383-398.
8.
Bull J. ( 1999). Computer assisted assessment: Impact on higher education institutions. Educational Technology & Society , 2(3). Retrieved from http://www.ifets.info/journals/2_3/joanna_bull.html .
9.
Cenoz J. ( 1998). Pauses and communication strategies in second language speech. ERIC Document ED 426630. Rockville, MD : Educational Resources Information Center.
10.
Chambers F. ( 1997). What do we mean by fluency? System , 25, 535-544.
11.
Chodorow M., Burstein J. ( 2004). Beyond essay length: Evaluating e-rater®’s performance on TOEFL® Essays. TOEFL Research Report No. 73. Princeton, NJ: Educational Testing Service.
12.
Clark HH ( 2002). Speaking in time. Speech Communication , 36, 5-13.
13.
Clark HH, Fox Tree JE ( 2002). Using uh and um in spontaneous speaking. Cognition, 84, 73-111.
14.
Cucchiarini C., Strik H., and Boves L. ( 2000). Quantitative assessment of second language learners’ fluency by means of automatic speech recognition technology. Journal of the Acoustical Society of America, 107(2), 989-999.
15.
Cucchiarini C., Strik H., and Boves L. ( 2002). Quantitative assessment of second language learners’ fluency: Comparisons between read and spontaneous speech. Journal of the Acoustical Society of America, 111(6), 2862-2873.
16.
de Bot K. ( 1992). A bilingual production model: Levelt’s speaking model adapted. Applied Linguistics, 13, 1-24.
17.
Dechert HW, Mohle D., and Raupach M. ( 1984). Second language production. Tubingen , Germany: Gunter Narr Verlag.
18.
Dechert HW, Raupach M. ( 1980). Towards a cross-linguistic assessment of speech production . Frankfurt, Germany: Peter Lang .
19.
Dechert HW, Raupach M. ( 1987). Psycholinguistic models of production. Tubingen, Germany: Gunter Narr Verlag .
20.
Derwing TM ( 1990). Speech rate is no simple matter: Rate adjustment and NS-NNS communicative success. Studies in Second Language Acquisition , 12, 303-313.
21.
Ejzenberg R. ( 2000). The juggling act of oral fluency: A psycho-sociolinguistic metaphor. In H Riggenbach (ed.), Perspectives on fluency (pp. 287-314). Ann Arbor, MI: The University of Michigan Press.
22.
Erard M. ( 2004, January 2). Just like, er, words, not, um, throwaways . New York Times. Retrieved from http://www.nytimes.com.
23.
Erard M. ( 2007). Um. New York: Anchor Books.
24.
Faerch K., Haastrup K., and Phillipson R. ( 1984). Learner language and language learning. Copenhagen, Denmark: Multilingual Matters .
25.
Ferris D. ( 1994). Lexical and syntactic features of ESL writing by students at different levels of L2 proficiencies. TESOL Quarterly, 28, 414-420.
26.
Fillmore C. ( 1979). On fluency. In C Fillmore , D Kempler, and W Wang (eds.), Individual differences in language ability and language behavior (pp. 85-101). New York: Academic Press.
27.
Freed BF ( 1995). What makes us think that students who study abroad become fluent? In BF Freed (ed.), Second language acquisition in a study abroad context (pp. 123-148). Amsterdam: John Benjamin.
28.
Fulcher G. ( 1987). Tests of oral performance: The need for data-based criteria . English Language Teaching Journal, 41, 287-291.
29.
Fulcher G. ( 2003). Testing second language speaking. London: Longman/Pearson Education.
30.
Goldman-Eisler F. ( 1968). Psycholinguistics: Experiments in spontaneous speech . New York: Academic Press .
31.
Grosjean F. ( 1980). Linguistic structures and performance structures: Studies in pause distribution. In HW Dechert and M Raupach (eds.), Temporal variables in speech: Studies in honour of Frieda Goldman-Eisler (pp. 271-285). The Hague: Mouton.
32.
Grosjean F., Deschamps A. ( 1975). Analyse contrastive des variables temporelles de l’anglais et du français: Vitesse de parole et variables compostantes, phénomènes d’hésitation. Phonetica , 31, 144-184.
33.
Hasselgreen A. ( 2004). Testing the spoken English of young Norwegians: A study of test validity and the role of ‘smallwords’ in contributing to pupils’ fluency. Cambridge, UK: Cambridge University Press.
34.
Hawkins PR ( 1971). The syntactic location of hesitation pauses. Language and Speech, 14, 277-288.
35.
Heike AE ( 1981). A content-processing view of hesitation phenomena . Language and Speech, 24(2), 147-160.
36.
Heike AE ( 1985). A componential approach to oral fluency evaluation . The Modern Language Journal, 69, 135-142.
37.
Kormos J., Denes M. ( 2004). Exploring measures and perceptions of fluency in the speech of second language learners. System, 32, 145-164.
38.
Kowal S., O’Connell D. ( 1980). Pausological research at Saint Louis University . In HW Dechert and M Raupach (eds.), Temporal variables in speech: Studies in honour of Frieda Goldman-Eisler (pp. 61-68). The Hague: Mouton.
39.
Kowal S., Wiese R., and O’Connell DC (1983). The use of time in storytelling . Language and Speech, 26, 377-392.
40.
Leal CF ( 1995). A pause as a cause of change. ERIC Document ED 379935 . Rockville, MD: Educational Resources Information Center.
41.
Lee YW, Gentile C., and Kantor R. ( 2008). Analytic scoring of TOEFL CBT essays: Scores from humans and e-rater®. TOEFL Research Report No. 81. Princeton, NJ: Educational Testing Service.
42.
Lehtonen J. ( 1981). Problems of measuring fluency and normal rate of speech . Paper presented at the International Congress of Applied Linguistics.
43.
Lennon P. ( 1984). Retelling a story in English as a second language . In HW Dechert and D Mohle (eds.), Second language productions (pp. 50-68). Tubingen , Germany: Gunter Narr Verlag.
44.
Lennon P. ( 1990). Investigating fluency in EFL: A qualitative approach . Language Learning, 41, 387-417.
45.
Levelt WJM ( 1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press.
46.
McLaughlin B. ( 1990). Restructuring. Applied Linguistics , 11, 113-128.
47.
Mohle D. ( 1984). A comparison of the second language speech production of different native speakers. In HW Dechert , D Mohle, and M Raupach (eds.), Second language production (pp. 26-49). Tubingen, German: Guner Narr Verlag.
48.
Mullen KA ( 1977). Rater reliability and oral proficiency evaluations . Occasional Papers in Linguistics, 1, 1-10.
49.
Munro MJ, Derwing TM ( 1998). The effects of speaking rate on listener evaluations of native and foreign-accented speech. Language Learning, 48, 159-182.
50.
Munro MJ, Derwing TM ( 2001). Modeling perceptions of accentedness and comprehensibility of L2 speech. Studies in Second Language Acquisition, 23, 451-468.
51.
Nakano M., Kondo Y., and Tsutsui E. ( 2008). Fundamental research on automatic speech evaluation of L2 English. Presented at the Distance Learning and the Internet Conference, Tokyo, Japan.
52.
O’Brien I., Segalowitz N., Freed B., and Collentine J. ( 2007). Phonological memory predicts second language oral fluency gains in adults. Studies in Second Language Acquisition , 29, 557-582.
53.
O’Connell DC, Kowal S. ( 2004). The history of research on the filled pause as evidence of The Written Language Bias in Linguistics (Linell, 1982). Journal of Psycholinguistic Research, 33(6), 459-474.
54.
Olynyk M., D’Anglejan A., and Sankoff D. ( 1987). A quantitative and qualitative analysis of speech markers in the native and second language speech of bilinguals. Applied Psycholinguistics, 8, 121-136.
55.
Oepp (2009): http://www.purdue.edu/oepp .
56.
Ordinate (1999, 2009). http://www.ordinate.com.
57.
Python Software Foundation (2008). Python, Version 2.5.2. Retrieved from http://www.python.org/download/releases/2.5.2/
58.
Pawley A., Syder FH ( 1983). Two puzzles for linguistic theory: Nativelike selection and nativelike fluency. In JC Richards and RW Schmidt (eds.), Language and communication (pp. 191-226). London: Longman.
59.
Raupach M. ( 1980). Temporal variables in first and second language speech production. In HW Dechert and M Raupach (eds.), Temporal variables in speech: Studies in honour of Frieda Goldman-Eisler (pp. 271-285). The Hague: Mouton.
60.
Raupach M. ( 1984). Formulae in second language speech production . In HW Dechert, D Mohle, and M Raupach (Eds.), Second language productions (pp. 114-137). Tubingen, Germany: Gunter Narr Verlag.
61.
Raupach M. ( 1987). Procedural learning in advanced learners of a foreign language. In JA Coleman and R Towell (eds.), The advanced language learner (pp. 123-155). London: CILT.
62.
Riazantseva A. ( 2001). Second language proficiency and pausing: A study of Russian speakers of English. Studies in Second Language Acquisition , 23, 497-526.
63.
Riggenbach H. ( 1991). Toward an understanding of fluency: A microanalysis of nonnative speaker conversations. Discourse Processes, 14, 423-441.
64.
Rohde L. ( 1985). Compensatory fluency: A study of spoken English produced by four Danish learners. In E Glahn and A Holmen (eds.), Learner discourse: Anglica et Americana (pp. 43-69). Copenhagen: University of Copenhagen.
65.
Sajavaara K. ( 1987). Second language speech production: Factors affecting fluency. In HW Dechert and M Raupach, M (eds.), Psycholinguistic models of production (pp. 45-65). Norwood, NJ: Ablex .
66.
Sajavaara K., Lehtonen J. ( 1978). Spoken language and the concept of fluency. Language Centre News, 1, 23-57.
67.
Schmidt R. ( 1992). Psychological mechanisms underlying second language fluency . Studies in Second Language Acquisition, 14, 357-385.
68.
Schmitt-Gevers H. ( 1993). La notion d’aisance dans la production et la réception orales en langue étrangére. Mélanges - Centre de Recherches et d’Applications pédagogiques en Langues , 21, 129-148.
69.
Shiffrin RM, Schneider W. ( 1977). Controlled and automatic human information processing II: Perceptual learning, automatic attending, and a general theory. Psychological Review, 84, 127-190.
70.
Swerts M. ( 1998). Filled pauses as markers of discourse structure . Journal of Pragmatics, 30, 485-496.
71.
Tavakoli P., Skehan P. ( 2005). Strategic planning, task structure, and performance testing . In R Ellis (ed.), Planning and task performance in a second language (pp. 239-273). Amsterdam: John Benjamins.
72.
Temple L. ( 1992). Disfluencies in learner speech. Australian Review of Applied Linguistics, 15(1), 29-44.
73.
Towell R. ( 1987). Approaches to the analysis of the oral language development of the advanced learner. In JA Coleman, R Towell (eds.), The advanced language learner (pp. 157-181). London: CILT.
74.
Towell R., Hawkins R., and Bazergui N. ( 1996). The development of fluency in advanced learners of French . Applied Linguistics, 17, 84-119.
75.
Ushigusa S. ( 2008). The relationships between oral fluency, multiword units, and proficiency scores. Unpublished doctoral dissertation, Purdue University, USA.
76.
Van Gelderen A. ( 1994). Prediction of global ratings of fluency and delivery in narrative discourse by linguistic and phonetic measures - oral performances . Language Testing, 11, 291-319.
77.
Vanderplank R. ( 1993). Pacing and spacing as predictors of difficulty in speaking and understanding English. English Language Teaching Journal , 47, 117-125.
78.
Watanabe M., Hirose K., Den Y., and Minematsu N. ( 2008). Filled pauses as cues to the complexity of upcoming phrases for native and non-native listeners. Speech Communications , 50, 81-94.
79.
Wood D. ( 2001). In search of fluency: What is it and how can we teaching it?The Canadian Modern Language Review, 57, 571-589.
80.
Xi X., Higgins D., Zechner K., and Williamson D. ( 2008). Automated scoring of spontaneous speech using SpeechRaterSMv1.0. TOEFL Research Report No. 62. Princeton, NJ: Educational Testing Service.
81.
Xi X., Mollaun P. ( 2006). Investigating the utility of analytic scoring for the TOEFL academic speaking test (TAST). TOEFLiBT-01.
82.
Yang R. ( 2009). A confirmatory factory analysis of the contsruct validity of the oral English proficiency test. OEPP Technical Papers 09-01.
83.
Zechner K. - Bejar I. ( 2006). Towards automatic scoring of non-native spontaneous speech . Paper presented at the HLT-NAACL-6. New York.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
