International Journal of Artificial Intelligence in Education: The European Journal on Artificial Intelligence

The purpose of the study described in this paper was to evaluate the efficacy of an assessment for learning system named ACED (Adaptive Content with Evidence-based Diagnosis). We used an evidencecentered design approach to create an adaptive, diagnostic assessment system which includes five main models: competency, evidence, task, presentation, and assembly. We also included instructional support in the form of elaborated feedback. The key issue we examined was whether the inclusion of the feedback into the system (a) impairs the quality of the assessment (relative to validity, reliability, and efficiency), and (b) does, in fact, enhance student learning. Results from a controlled evaluation testing 268 high-school students showed that the quality of the assessment was unimpaired by the provision of feedback. Moreover, students using the ACED system showed significantly greater learning of the content compared with a control group. These findings suggest that assessments in other settings (e.g. state-mandated tests) might be augmented to support student learning with instructional feedback without jeopardizing the primary purpose of the assessment.

To investigate whether more concise Natural Language feedback improves learning, we developed two Natural Language generators (DIAG-NLP1 and DIAG-NLP2), to provide feedback in an Intelligent Tutoring System that teaches troubleshooting. We systematically evaluated them in a three way comparison that included the original system, which generates overly repetitive feedback. We found that DIAG-NLP2, the generator which intuitively produces the best, corpus-based language, does engender the most learning. Distinguishing features of the more effective feedback are: it obeys Grice's maxim of brevity, it is more directive and uses a specific type of referring expressions. Interestingly, simpler ways of restructuring the original repetitive feedback as done in DIAG-NLP1, such as exploiting the hierarchical structure of the domain, were not effective. Since the design of interfaces to Intelligent Tutoring Systems often includes verbal feedback, we suggest that: if the number of different contexts in which verbal feedback is provided is high, such feedback should be based on corpus studies, and generated by techniques more sophisticated than template filling.

This paper introduces AMPLIA, an intelligent learning environment employed as a resource in medical students' training. The development of AMPLIA raised several research topics, due to the convergence of Artificial Intelligence (AI) and Learning Environments. The core topics are: probabilistic diagnostic learning in the medical area, application of teaching strategies based on Pedagogical Negotiation (PN), construction of cognitive student models with probabilistic beliefs, and application of interoperability methods for pedagogical agents, and tutoring systems integration. An important aspect of AMPLIA is the utilization of PN as the main form of interaction. The impact of this approach on the system dynamics and on the student's learning is presented in detail. Considering the importance of cooperative work during the learning process, we describe how AMPLIA is used to enable cooperation. The description is based on experiments carried out with AMPLIA and its users. The main results of these experiments are reported as well.