Abstract
In this article we describe our approach towards the specification and realization of human-computer interaction within Next Generation Ambient Intelligent Environments (NGAIE). These environments are populated with numerous devices and multiple occupants or users. They exhibit increasingly intelligent behaviour, provide optimized resource usage and support consistent functionality and human-centric operation. In our approach, NGAIEs contain an encoding of local and global knowledge in the form of a set of heterogeneous ontologies, which have to be aligned. This is to provide a uniform and consistent knowledge representation. In NGAIEs humans will interact with their environments seamlessly using multimodal dialogue interaction. To enable such adaptive human-computer interaction we then focus on when and how this knowledge can be modelled and used in order to realize complex, negotiative, and collaborative tasks. The combination of heterogeneous ontologies and ontology matching algorithms allows for semantically rich interaction and information exchange. Based on an agent-based, service-oriented architecture, this combination maximizes the use of available interaction resources, while decoupling interaction specification from interfaces and modalities. We illustrate our approach with a task analysis of a scenario showing the challenges of NGAIEs. Finally, we present ontology prototypes that are required for the implementation of the scenario.