Investigating the Relationship Between Dialogue Structure and Tutoring Effectiveness: A Hidden Markov Modeling Approach

Identifying effective tutorial dialogue strategies is a key issue for intelligent tutoring systems research. Human-human tutoring offers a valuable model for identifying effective tutorial strategies, but extracting them is a challenge because of the richness of human dialogue. This article addresses that challenge through a machine learning approach that 1) learns tutorial modes from a corpus of human tutoring, and 2) identifies the statistical relationships between student outcomes and the learned modes. The modeling approach utilizes hidden Markov models (HMMs) to capture the unobservable stochastic structure that is thought to influence the observations, in this case dialogue acts and task actions, that are generated by task-oriented tutorial dialogue. We refer to this unobservable layer as the hidden dialogue state, and interpret it as representing the tutor and students' collaborative intentions. We have applied HMMs to a corpus of annotated task-oriented tutorial dialogue to learn one model for each of two effective human tutors. Significant correlations emerged between the automatically extracted tutoring modes and student learning outcomes. Broadly, the results suggest that HMMs can learn meaningful hidden tutorial dialogue structure. More specifically, the findings point to specific mechanisms within task-oriented tutorial dialogue that are associated with increased student learning. This work has direct applications in authoring data-driven tutorial dialogue system behavior and in investigating the effectiveness of human tutoring.