International Journal of Artificial Intelligence in Education: The European Journal on Artificial Intelligence

We have augmented the Deep Thought logic tutor with a Hint Factory that generates data-driven, context-specific hints for an existing computer aided instructional tool. We investigate the impact of the Hint Factory's automatically generated hints on educational outcomes in a switching replications experiment that shows that hints help students persist in a deductive logic proofs tutor. Three instructors taught two semester-long courses, each teaching one semester using a logic tutor with hints, and one semester using the tutor without hints, controlling for the impact of different instructors on course outcomes. Our results show that students in the courses using a logic tutor augmented with automatically generated hints attempted and completed significantly more logic proof problems, were less likely to abandon the tutor, performed significantly better on a post-test implemented within the tutor, and achieved higher grades in the course.

A new type of sensor for students' mental states is a single-channel portable EEG headset simple enough to use in schools. To gauge its potential, we recorded its signal from children and adults reading text and isolated words, both aloud and silently. We used this data to train and test classifiers to detect a) when reading is difficult, b) when comprehension is lacking, and c) lexical status and word difficulty. To avoid exploiting the confound of word and sentence difficulty with length, we truncated signals to a uniform duration. The EEG data discriminated reliably better than chance between reading easy and difficult sentences. We found weak but above-chance performance for using EEG to distinguish among easy words, difficult words, pseudo-words, and unpronounceable strings, or to predict correct versus incorrect responses to a comprehension question about the read text. We also identified which EEG components appear sensitive to which lexical features. We found a strong relationship in children between a word's age-of-acquisition and activity in the Gamma frequency band (30–100 Hz). This pilot study gives hope that a school-deployable EEG device can capture information that might be useful to an intelligent tutor.

In this paper we investigate how student disengagement relates to two performance metrics in a spoken dialog computer tutoring corpus, both when disengagement is measured through manual annotation by a trained human judge, and also when disengagement is measured through automatic annotation by the system based on a machine learning model. First, we investigate whether manually labeled overall disengagement and six different disengagement types are predictive of learning and user satisfaction in the corpus. Our results show that although students' percentage of overall disengaged turns negatively correlates both with the amount they learn and their user satisfaction, the individual types of disengagement correlate differently: some negatively correlate with learning and user satisfaction, while others don't correlate with either metric at all. Moreover, these relationships change somewhat depending on student prerequisite knowledge level. Furthermore, using multiple disengagement types to predict learning improves predictive power. Overall, these manual label-based results suggest that although adapting to disengagement should improve both student learning and user satisfaction in computer tutoring, maximizing performance requires the system to detect and respond differently based on disengagement type. Next, we present an approach to automatically detecting and responding to user disengagement types based on their differing correlations with correctness. Investigation of our machine learning model of user disengagement shows that its automatic labels negatively correlate with both performance metrics in the same way as the manual labels. The similarity of the correlations across the manual and automatic labels suggests that the automatic labels are a reasonable substitute for the manual labels. Moreover, the significant negative correlations themselves suggest that redesigning ITSPOKE to automatically detect and respond to disengagement has the potential to remediate disengagement and thereby improve performance, even in the presence of noise introduced by the automatic detection process.

In this paper, we explore the possibility of a general framework for modelling engagement dynamics in software tutoring, focusing on the cases of developmental dyslexia and developmental dyscalculia. This project aims at capturing the similar engagement state patterns for the two learning disabilities. We start by presenting a model of engagement dynamics in spelling learning, which relates input behaviour to learning and explains the dynamics of engagement states. Predictive power of extracted features is increased by incorporating domain knowledge in the pre-processing. The introduced model enables the prediction of focused and receptive states, and of forgetting. In the second part, we extend the model to a more general framework, which takes into account the similarities and dissimilarities of the two studied cases. Finally, we define desirable properties of a general engagement dynamics model, while analysing the reusability of the introduced spelling model.

Cognitive disequilibrium and its affiliated affective state of confusion have been found to positively correlate with learning, presumably due to the effortful cognitive activities that accompany their experience. Although confusion naturally occurs in several learning contexts, we hypothesize that it can be induced and scaffolded to increase learning opportunities. We addressed the possibility of confusion induction in a study where learners engaged in trialogues on research methods concepts with animated tutor and student agents. Confusion was induced by staging disagreements and contradictions between the animated agents, and then inviting the (human) learners to provide their opinions. Self-reports of confusion indicated that the contradictions were successful at inducing confusion in the minds of the learners. A second, more objective, method of tracking learners' confusion consisted of analyzing learners' performance on forced-choice questions that were embedded after contradictions. This measure was also found to be revealing of learners' underlying confusion. The contradictions alone did not result in enhanced learning gains. However, when confusion had been successfully induced, learners who were presented with contradictions did show improved learning compared to a no-contradiction control. Theoretical and applied implications along with possible future directions are discussed.