Objective: This study examined the effects of user production (speaking and typing) and user reception (listening and reading) modes on natural language human-computer dialogue. Background: Text-based dialogue is often more efficient than speech-based dialogue, but the latter is more dynamic and more suitable for mobile environments and hands-busy situations. The respective contributions of user production and reception modes have not previously been assessed. Method: Eighteen participants performed several information search tasks using a natural language information system in four experimental conditions: phone (speaking and listening), Web (typing and reading), and mixed (speaking and reading or typing and listening). Results: Mental workload was greater and participants' repetitions of commands were more frequent when speech (speaking or listening) was used for both the user production and reception modes rather than text (typing or reading). Completion times were longer for listening than for reading. Satisfaction was lower, utterances were longer, and the interaction error rate was higher for speaking than typing. Conclusion: The production and reception modes both contribute to dialogue and mental workload. They have distinct contributions to performance, satisfaction, and the form of the discourse. Application: The most efficient configuration for interacting in natural language would appear to be speech for production and system prompts in text, as this combination decreases the time on task while improving dialogue involvement.
Get full access to this article
View all access options for this article.
References
1.
Baker, K., McKenzie, A., Biermann, A., & Webelhuth, G. (2004). Constraining user response via multimodal dialogue interface. International Journal of Speech Technology, 7, 251-258.
2.
Branigan, H.P., Pickering, M.J., Pearson, J., McLean, J.F., & Nass, C.I. (2003, July-August). Syntactic alignment between computers and people: The role of belief about mental states. Paper presented at the 25th Annual Meeting of the Cognitive Science Society , Boston, MA.
3.
Chen, F. (2006). Designing human interface in speech technology . New York: Springer.
4.
Clark, H.H. (1996). Using language. Cambridge, UK: Cambridge University Press.
5.
Clark, H.H., & Wilkes-Gibbs, D. (1986). Referring as a collaborative process. Cognition, 22, 1-39.
6.
Diaz, F.C., & Banga, E.R. (2006). A method for combining intonation modelling and speech unit selection in corpus-based speech synthesis systems. Speech Communication, 48, 941-956.
7.
Gorin, A.L., Riccardi, G., & Wright, J.H. (1997). How may I help you?Speech Communication , 23, 113-127.
8.
Hart, S.G., & Staveland, L.E. (1988). Development of a multidimensional workload rating scale: Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human mental workload (pp. 139-183). Amsterdam: Elsevier.
9.
Le Bigot, L., Jamet, E., Rouet, J.-F., & Amiel, V. (2006). Mode and modal transfer effects on performance and discourse organization with an information retrieval dialogue system in natural language. Computers in Human Behavior, 22, 467-500.
10.
Lee, H.-K., Suh, K.-S., & Benbasat, I. (2001). Effects of task-modality fit on user performance . Decision Support Systems, 32, 27-40.
11.
Lee, K.W., & Lai, J. (2005). Speech versus touch: Acomparative study of the use of speech and DTMF keypad for navigation. International Journal of Human-Computer Interaction, 19, 343-360.
12.
Leiser, R.G. (1989). Exploiting convergence to improve natural language understanding. Interacting with Computers, 1, 284-298.
13.
Levelt, W.J.M., & Kelter, S. (1982). Surface form and memory in question answering . Cognitive Psychology, 14, 78-106.
14.
Oviatt, S.L. (1999). Ten myths of multimodal interaction. Communications of the ACM, 42(11), 74-81.
15.
Oviatt, S.L., Cohen, P.R., & Wang, M. (1994). Toward interface design for human language technology: Modality and structure as determinants of linguistic complexity. Speech Communication, 15, 283-300.
16.
Parush, A. (2005). Speech-based interaction in multitask condition: Impact of prompt modality. Human Factors, 47, 591-597.
17.
Pickering, M.J., & Garrod, S. (2004). Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27, 169-226.
18.
Sadek, D., Bretier, P., & Panaget, F. (1997, August). Artimis: Natural dialogue meets rational agency. Paper presented at the 15th International Joint Conference on Artificial Intelligence (IJCAI'97), Nagoya, Japan.
19.
Sheeder, T., & Balogh, J. (2003). Say it like you mean: Priming for structure in caller responses to a spoken dialog system. International Journal of Speech Technology, 6, 103-111.
20.
Shneiderman, B. (2000). The limits of speech recognition. Communications of the ACM, 43(9), 63-65.
21.
Wilkie, J., Jack, M.A., & Littlewood, P.J. (2005). System-initiated digressive proposals in automated human-computer telephone dialogues: The use of contrasting politeness strategies. International Journal of Human-Computer Studies, 62, 41-71.
