Recognising activities of daily living from patterns of object use

Abstract

To provide assistance and support to the elderly disabled and cognitively impaired, the recognition of their activities of daily living (ADL) must be accurate and precise with regards to the object use for the activity situations. Current knowledge-driven and ontology-based activity recognition techniques model object concepts from assumptions and common everyday knowledge of object use of routine activities. Modelling activities from assumptions and common everyday knowledge of object use could lead to faulty recognition of particular routine activities and possibly undermine abnormal activity trends. A significant step in the recognition of activities of daily living is the discovery of the object use for specific routine activities due to its ability to relate object use to their associated activities. The discovering particular object(s) which are used to perform routine activities could help enhance knowledge-driven ontology-based activity recognition with the object use for specific activities and the associated activities as ontology concepts. This paper focuses on the recognition of simple activities of daily living from object use and interactions in the home environment. We take advantage of the object use for routine activities discovered from a topic model process to augment activity ontology concepts for activity recognition. The experimental results obtained using the Kasteren and Ordonez datasets show it is significantly encouraging, comparable and improved on results published using the same datasets.

Keywords

Activity recognition topic model ontology model Latent Dirichlet Allocation

1. Introduction

Activity recognition is an important emerging area of research in pervasive computing due to its significance in the provision of support and assistance to the elderly disabled and cognitively impaired. It is a process involved in identifying what an individual is doing, e.g. Sleeping, Showering, and Making Food etc. Research efforts so far has focused on the use of video [1, 2, 3], wearable sensors [4, 5] and wireless sensor networks [6, 7] to monitor simple human activities. Video based activity recognition captures body images which are segmented and then classified using context based analysis. Unlike sensor based activity recognition, video based suffers from accurate segmentation of captured images which affects classification process. Sensor based activity recognition monitor object usage from the interactions of the objects in the home environment. Typically, sensor based activity recognition follows data-driven, knowledge-driven or a combination of both techniques to identify activities. Data driven activity recognition use machine learning and statistical methods which involve discovering the data patterns to make activity inferences. Research efforts by [8, 6, 7] show the strengths of the approach in the learning process. Inferences in most cases for data driven approaches are hidden and or latent, thus requiring activities identified to be expressed in understandable format for the end user. Again, data driven approach suffers from its inability to integrate context aware features to enhance activity recognition in the home environment. On the other hand, knowledge driven and ontology-based methods model activities as concepts, associating them to everyday knowledge of object use in the home environment through a knowledge engineering process. The modelling process involves associating low-level sensor data to the relevant activity to build a knowledge base of activities in relation to sensors and object. Activities are then recognised by following logical inference and or inclusion of subsumption reasoning. In comparison to data driven techniques, knowledge driven techniques are more expressive and inferences are usually in the format easily understood by the end user [9]. Knowledge driven techniques in most cases depend on everyday knowledge of activities and object use to build and construct activity ontologies. Knowledge of object use is mostly by assumptions, regular every day knowledge of what object are used for routine activities or even wiki-know-how [10].1 In this paper, we follow a sensor based activity recognition with sensors capturing object use as the result of the interactions of object in the home environment. We regard object use and interactions as atomic events leading to activities. With different objects, we could have different activities. In some cases, we could have activities with shared or similar object use. The approach we propose specifically follows object use as events and entries to recognise activities. This is in line with what is obtainable in real world situations and most home environments. Interactions of objects as object use in the home environment result to activities. These object use and interactions as atomic events result to particular routine activities. Our motivation of following this approach is such that knowledge driven activity recognition through an activity Ontology modelled and constructed from the everyday knowledge of object use may not fit into certain activity situations or capture specific routine activities in home environments but could be satisfactorily achieved with accurate and precise object use for specific routine activities. If an activity model has been developed based on the generic and or assumed knowledge of object use, the recognition model may fail due to activities and objects fittings which differs with individuals and home environments. Generic ontology models have been designed and developed as in Chen et al. [8] to emphasise re-usability and share ability. As a way forward, we argue that, it is essential to extend this ontology to accommodate object concepts that are specific to the routine activities with regards the individual and the home environment. To provide assistance and support to the elderly and cognitively impaired, the recognition of their activities of daily living (ADL) must be accurate and precise with regards to the object use events. This framework significantly comes in handy where object use for routine activities have not been predefined. So the challenge becomes discovering specific object use for particular routine activities. This paper focuses on the process of the recognition of ADL as a step towards the provision support and assistance to the elderly disabled and cognitively impaired. It also emphasises on knowledge driven techniques and shows that the knowledge acquisition process can be extended beyond generic and everyday knowledge of object use to build activity ontology. Given these, our work harnesses the complementary strengths of data and the knowledge driven techniques to provide solutions to the limitations and challenges highlighted above. In this context, the major aim of this paper is to extend knowledge driven activity recognition to include a process of acquiring knowledge of object use to describe contexts of activity situations. In particular, we seek to acquire knowledge by identifying activities within object data stream by activity-object use discovery. In addition to this, we represent this acquired knowledge ontologically to build an activity recognition prototype capable of accurately recognising activities of daily living. We propose a framework composed of a context description module and an activity ontology module as components. The context description module augments the activity ontology to acquire the knowledge of object use in the home environment through an activity-object use discovery enabled by the Latent Dirichlet Allocation topic model. The contributions of this paper are as follows.

•
Activity-object use discovery in context for the likely objects use for specific routine activities. We use topic model through activity-object use discovery to acquire knowledge for concepts formation as part of an ontology knowledge acquisition and learning system.
•
Extend the traditional activity ontology to include the knowledge concepts acquired from the activity-object use discovery and context description which is significantly important especially in home environments and scenarios where object use for routine activities have not been predefined.
•
The evaluation and validation of the proposed framework.

The remainder of the paper is organized as follows. Section 2 provides an overview of the related works, while Section 3 describes the proposed activity recognition approach. Section 4 provides experimental results based on the Kasteren and Ordonez datasets which were used to validate the proposed framework. Section 5 concludes this paper.

Figure 1.
An overview of the activity recognition framework.

2. Related work

Activity recognition approaches can be classified in two broad categories data and knowledge driven approaches. This classifications are based on the methodologies adopted, how activities are modelled and represented in the recognition process. Data driven approaches can be generative or discriminative. According to [9], the generative approach builds a complete description of the input (data) space, usually using a probabilistic model. The resulting model induces a classification boundary which can be applied to classify observations during inference. The classification boundary is implicit and a lot of activity data is required to induce it. Generative classification models includes Dynamic Bayes Networks (DBN) [11, 12], Hidden Markov Model (HMM) [11, 13, 14], Naive Bayes (NB) [15, 16, 12], Topic model Latent Dirichlet Allocation (LDA) [17, 18]. Discriminative models, as opposed to generative models, do not allow generating samples from the joint distribution of the models [9]. Discriminative classification models includes nearest neighbour [19, 20], decision trees [21], support vector machines (SVMs) [22, 23, 19], conditional random fields (CRF) [24], multiple eigenspaces [25], and k-means [19]. With regards to the state of the art approaches discussed above, data driven approaches have the advantage of handling incomplete data and managing noisy data. Not all the data driven approaches have the advantage of handling temporal information. Another drawback associated with data driven approaches is that they lack the expressiveness to represent activities as they as learnt in the model. They need to be annotated and sometime needing elements of semantics for expressiveness. Topic models inspired by the text and natural language processing community have been applied to discover and recognise human activity routines in research works by Katayoun and Gatica-Perez [17] and Huynh [4]. Huynh [4] applied the bag of words model of the Latent Dirichlet Allocation (LDA) to discover activities like dinner, commuting, office work etc. The process involved activity discovery of partitioned sensor segments of time windows. Katayoun and Gatica-Perez [17] discovered activity routines from mobile phone data, Huynh [4] used wearable sensors attached to the body parts of the user. Activities discovered in both work were latent, lacked expressiveness and minimal opportunities to integrate context rich features. Our work also significantly differs from Katayoun and Gatica-Perez [17] and Huynh [4] with the modelling of the activity-object use using an ontology activity model. Knowledge driven ontology models follow web ontology language (OWL) theories for the specification of conceptual structures and their relationships. Ontology based activity recognition are an emerging area in knowledge-driven approach. It involves the use of ontological activity modelling and representation to support activity recognition and assistance. Ontology uses the formal and explicit specification of a shared conceptualization of a problem domain [26]. Vocabulary for modelling a domain is provided by specifying the objects and concepts, properties, and relationships. This then uses domain and prior knowledge to predefine activity models to define activity ontologies [9]. Latfi and Lefebvre [27] proposed an ontology framework for a telehealth smart home aimed at providing support for elderly persons suffering from loss of cognitive autonomy. Similarly, Marjan et al. [28] proposed and ontology based framework to recognise activities and the events occurring in the home. Chen et al. [9] and Chen and Nugent [29] proposed an ontology-based approach to activity recognition in which they constructed context and activity ontologies for explicit domain modelling. Sensor activations over a period of time are mapped to individual contextual information and then fused to build a context at any specific time point. Subsumption reasoning were used to classify the activity ontologies, thus inferring the ongoing activity. Knowledge driven ontology models also follow web ontology language (OWL) theories for the specification of conceptual structures and their relationships [30]. OWL has been widely used for modelling human activities for recognition, which most times involve the description of activities by their specifications using their object and data properties [29]. In ontology modelling, domain knowledge is required to encode activity scenarios, but it also allows the use of assumptions and common sense domain knowledge to build the activity scenarios that describe the conditions that drive the derivation of the activities [31]. Recognising the activity then requires the modelled data to be fed to the ontology reasoner for classification. The authors of [9, 31] followed generic activity knowledge to develop an ontology model for the smart home users. Whilst these approaches to model activities to depending on common sense domain knowledge and its associated heuristics are commendable, they may lead to faulty activity recognition due to lack of specificity of the object use and contexts describing the activity situations. Specific considerations to object use for routine activities could be been made just as home settings and individual object usage differs which may not be applicable to ontologies developed from generic knowledge of object usage. They also do not follow evidenced patterns of object usage and activity evolution as they rely on generic know hows and hows to to build ontology models. In view of these limitations, we apply an LDA enabled activity discovery technique to discover likely object use for specific routine to augment the generic ontology modelling process by [9] in our work. The framework we propose in this paper, also extends previous works by Ihianle et al. [32, 33, 34] by the inclusion of the ontology activity model.

3. Overview of our activity recognition approach

To achieve activity recognition, the proposed framework supports object use as contexts of activity situations through activity-object use discovery, information fusion of activity and object concepts, activity ontology design, development and modelling, and then activity recognition. The proposed framework is implemented as illustrated with the architecture in Fig. 1 – having a complementary topic model context description module to augment the traditional ontology driven activity recognition. The architecture is made of two component modules – the context description module and the ontology module. Object use for specific routine activities as activity-object use distributions and activity context descriptors are discovered using a Latent Dirichlet Allocation (LDA) topic model in the context description module. An activity ontology is developed using the discovered object use for the respective activities as ontology concepts in the ontology module. Activity recognition is achieved by observed object use query on the activity ontology for the relevant activity situations. As a unified framework, the functions of these component modules are integrated to provide a seamless activity recognition platform which takes in inputs of sensor and object use observations captured in the home environment representing atomic events of object interactions. We describe in detail the component modules in the following subsections.

3.1 Context description module

The context description module augments the traditional knowledge driven activity recognition framework [8, 29, 35]. Its main function is to provide the knowledge of object use for specific routine activities as activity-object use distributions and activity context descriptors. These object use for specific routine activities are the contexts describing the specific routine activity situations, hence the name context description module. To provide the basis for an activity recognition, the knowledge of object use for respective activity concepts are required. The necessity of these object use knowledge is such that activities as high-level events are a result of low-level tasks or atomic events of object interactions. Traditional knowledge driven activity recognition frameworks [10, 8] model ontologies from generic object use assumptions or every day knowledge of object use. However, the traditional (generic) models may not be the case in every home setting or environment as this may lead to erroneous object descriptions of low-level tasks and eventually incorrect activity recognition. To design and model activity situations ontologically, there must be an accurate process of acquiring the knowledge of object use or context descriptors which describe activities or maps to activity as higher level of events. The context description module performs its function by a process of activity-object use discovery and activity context description which uses the output from the activity-object use discovery process. We briefly describe this modular process below.

3.1.1 Activity-object use discovery by Latent Dirichlet Allocation

The activity-object use discovery is dependent on the Latent Dirichlet Allocation (LDA) introduced by Blei et al. [36]. The LDA generatively classifies a corpus of documents as a multinomial distribution of latent topics. It takes advantage of the assumption that there are hidden themes or latent topics which have associations with the words contained in a corpus of documents. It then requires the bag of words (documents) from a corpus of documents as input and number of topics as a key parameter. In the context of our activity-object use discovery process, the activity topic number and bag of sensor observations corresponds to topic number and bag of words respectively of the LDA. We explain determining the activity topic number and bag of sensor observations below.

•
Activity topic number: A key parameter needed by the LDA process is the topic number. In the context of the activity recognition we propose in this paper, the number of activities corresponds to the topic number. The activities to be recognised adds up to the activity topic number. For experimental purposes we use the number of activities specified in the dataset.
•
Bag of object observations: The bag of objects observation we propose is analogous to the bag of words used in the LDA text and document analysis. In text and document analysis, a document (bag) in a corpus of texts can be represented as a set of words with their associated frequencies independent of their order of occurrence [37]. Disregarding the order of word occurrence, the bag of words is a representation of the words in the document with their frequencies. We follow the bag of word approach to represent discrete observations of objects or sensors of specific time windows generated as events in the use or interaction of home objects. In this regard, we refer to it as bag of object observations. To satisfactorily achieve bagging of the objects accordingly, the stream of observed sensor or objects data are partitioned into segments of suitable time intervals. By this, the objects and the partitioned segments then respectively corresponds to the words and documents of the bag of words. If a dataset is given by $D$ made of $x_{1},\ldots,x_{n}$ objects, $D$ can be partitioned using suitable sliding time window intervals into $d_{1},\ldots,d_{D}$ segments similar to the schema Eq. (1).

$D=\begin{bmatrix}\{x_{1},\ldots,x_{n1}\}&\text{ objects in segment }d_{1}\\ \{x_{2},\ldots,x_{n2}\}&\text{ objects in segment }d_{2}\\ \{x_{N1},\ldots,x_{N}\}&\text{ objects in segment }d_{1}\\ \end{bmatrix}$ (1)

The observed objects $x_{1},\ldots,x_{n}$ in each of the segments $d_{1},\ldots,d_{D}$ are then represented with their associated frequencies $f$ to form a segment-object-frequency matrix similar to the schema given in Eq. (2).

$\text{Bag of Object Observation }=\begin{bmatrix}d_{1}&x_{1}&f_{1}\\ d_{2}&x_{2}&f_{2}\\ &\ldots&\\ d_{D}&x_{N}&F\\ \end{bmatrix}$ (2)

Figure 2.
An overview of the activity ontology module.

The LDA takes advantage of the assumption that there are hidden themes or latent topics which have associations with the words contained in a corpus of documents. It also involves the use of bag of words in the corpus of documents which are generatively classified to latent themes or topics and word distributions. We conversely apply this assumption to the activity-object pattern discovery context that latent activity topics would have associations with the features of object data in the partitioned segments of the bag of sensor observations discussed above. The documents are presented in the form of objects segments $d_{1},\ldots,d_{D}$ composed of co-occurring object data observations. With $D$ composed of object segments $d_{1},\ldots,d_{D}$ similar to schema Eq. (1) above, $d_{i}$ would be made of objects represented as $x_{i_{1}},\ldots,x_{i_{n}}$ from $X$ objects of $x_{1},\ldots,x_{n}$ . The LDA places a dirichlet prior $P$ ( $\theta_{d}|\alpha$ ) with parameter $\alpha$ on the document-topic distributions $P$ ( $z|\theta_{d}$ ). It assumes a Dirichlet prior distribution on the topic mixture parameters $\theta$ and $\phi$ , to provide a complete generative model for documents $D$ . $\theta$ describes $D\times Z$ matrix of document-specific mixture weights for the $Z$ topics, each drawn from a Dirichlet( $\alpha$ ) prior, with hyperparameter $\alpha$ . $\phi$ is an $X\times Z$ matrix of word-specific mixture weights over $X$ objects for the $Z$ activity topics, drawn from $\beta$ which is a Dirichlet prior. The probability of a corpus or observed object dataset, is equivalent to finding parameter $\alpha$ for the dirichlet distribution and parameter $\beta$ for the topic-word distributions $P$ ( $x|z,\beta$ ) that maximize the likelihood $\mathbb{L}$ of the data for documents $d=d_{1},\ldots,d_{D}$ as given in the Eq. (3) following Gibbs sampling for LDA parameter estimation [36].

$\displaystyle\mathbb{L}(\alpha,\beta)=\prod_{d=1}^{D}\int P(\theta_{d}|\alpha)$ $\displaystyle\quad∼{}\left(\prod_{x=1}^{X_{D}}\sum_{z=1}^{Z}P(x_{xi}^{d}|z,% \beta)P(z|\theta_{d})\right)d\theta_{d}$ (3)

Given the assumptions above, the activity-object patterns can be calculated from the Eq. (4) below:

$\displaystyle P(x|z,\phi)=\prod_{z=1}^{Z}\prod_{x_{i}=1}^{X}(\phi^{x_{i}}_{z})% ^{n^{x_{i}}_{z}}$ (4)

where $n^{x_{i}}_{z}$ is the number of times an object $x_{i}$ is assigned to a topic $z$ .

In the context of the activity recognition, modelling activity concepts for recognition would rely on the probabilistic distribution of the objects given the activity topics. The LDA topic model, $P$ ( $x|z$ ) computes the object use distributions that are linked to specific activity topics. The object distributions for the activity topics through LDA topic model provides the activity context description needed to build an activity ontology model with regards to the specific object use. This provides the object use concepts and for the respective activity concepts needed to construct the activity ontology model. If we use the activity topic number from a given dataset $D$ and bag of sensor observations as inputs, then the object use distributions specific to the routine activities can be determined and results similar to the schema Eq. (5). With this, we would have discovered the activity-object use distributions as activity context descriptors for the specific routine activities. The process ends with the LDA process assigning objects to specific activity topics.

$\displaystyle P(x|z)=$ $\displaystyle\quad\begin{bmatrix}\{x_{k1a},x_{k1b},\ldots,x_{k1n}\}\text{ % Assigned to }k1\\ \{x_{k2a},x_{k2b},\ldots,x_{k2n}\}\text{ Assigned to }k2\\ \ldots\\ \{x_{Ka},x_{Kb},\ldots,x_{KN}\}\text{ Assigned to }K\\ \end{bmatrix}$ (5)
3.2 Ontology module

The ontology module is composed of the knowledge base as a repository of information consisting of the modelled activity ontology concepts, data, rules used to support activity recognition. Just like other knowledge bases, it functions as a repository where information can be collected, organized, shared and searched. The activities and the context descriptors from the context description module are designed, developed following description logic, knowledge representation and formalism and then added to the knowledge base. The knowledge base is made of the TBox, ABox and the reasoner (see Fig. 2 for an overview of the activity ontology module). The TBox is the terminological box made of the activities concepts and the relevant context descriptors of object use as defined and encoded as ontology concepts. The ontological design and development process gradually populates the TBox by encoding the activities and context descriptors from the context description module as ontology concepts. The ABox is the assertional box made of the instances and individuals of the concepts encoded in the TBox. They are asserted through properties which may be object or data properties. For all the terminological concepts in our TBox, instances and individuals of these are asserted through different properties to populate our ABox. In addition to the activities and context descriptor concepts and instances, we also added temporal concepts and instantiated them following the 4D fluent approach to allow for a realistic reflection activity evolution and transition. Also based on their temporal properties as usual time of occurrence, activities could be modelled as static and dynamic activities. The resultant activity ontology created with the fusion of likely object use and behavioural information from the activity context descriptions makes it possible for activity inferencing. The reasoner checks the relationships between the concepts in the TBox and also checks the consistencies in the ABox for the individuals and instances to perform activity recognition by information retrieval. The eventual result from the information retrieval are the activities or activities situations.

Table 1
Ontology concepts used

Ontology concepts	Descriptions
ADL	Activity of Daily Living and super activity of all activity concepts
Activity or activity situation	Concepts representing type of ADL examples include Breakfast, Dinner, Drink, Toileting etc.
Objects or resources	Concepts representing concepts of object used in the home environment examples include Microwave, Plates Cupboard, Fridge, Cup etc.
TimeSlice	Time slice of an object
TimeInterval	Time interval of an activity
Ontology properties
hasUse	An object property used to associate usage of an object for an activity
hasStartTime	An data property used to associate time reference when an object was observed
tsTimeSliceOf	Fluent property for time slice of an object
tsTimeIntervalOf	Fluent property for the interval describing range of time an activity

Table 2

Ontology notations used

Notations	Descriptions
$\sqsubseteq$	Subclass of
$\exists$	Property of
$\sqcap$	Intersection
$\sqcup$	Union
$\equiv$	Is an equivalent class of
$\rightarrow$	Implication

The process of modelling an activity concept resulting from a set of sensors and object use requires asserting all the objects concepts and with their times to be encoded to represent the activity situation (see Tables 1 and2 for a list of concepts and notations used in our analogies). If an activity situation Breakfast is the result ofMicrowave_On and Fridge_On at times t1 and t2 respectively, then Breakfast can be asserted with the properties hasUse and hasStartTime as hasUse(Microwave_On, hasStartTimet1), hasUse(Fridge_On, hasStartTimet2). The activity is therefore modelled as a list of the objects and with their times ordered temporally. The example of Breakfast from Microwave_On and Fridge_On at t1 and t2 can then be encoded by the expression Eq. (6) below.

$\displaystyle\textit{Breakfast}\sqsubseteq ADL\sqcap\exists\textit{hasUse}.$

Typically, activity situations or activities in the home environment are a result of specific objects use. To model activity situations accurately, it is important to extend the traditional activity ontology modelling to include specific resources and or objects use for the specific routine activity. The activity context descriptors resulting from the activity-object use discovery forms the resources and objects use concepts to be modelled onto the activity ontology for the specific routine activities such that:

Activities: The activity topics are annotated as activity concepts analogous to the activity situations in the home environment. This represents a class collection all types of activities set as $z_{1},\ldots,z_{k}$ .

Objects: These represents class collection of all objects as activity context descriptors in the home environment set as $x_{1},\ldots,x_{n}$ .

Figure 3.

4D-fluents with activities and resources.

Recall $P(x|z)$ computes the likely objects use for each of the activity topics, then $z_{i}$ from $z_{1},\ldots,z_{k}$ would have a subset of likely objects defined as $x_{1z},\ldots,x_{iz}$ . The function $f$ maps the activity context descriptors $x_{1z},\ldots,x_{iz}$ objects for activity $z_{i}$ as the likely objects use for this activity such that:

(7)

\displaystyle f\!:z_{i}\rightarrow x_{1z},\ldots,x_{iz}

If the function $f$ is replaced with the object property function in hasUse, then Eq. (7) becomes:

$\displaystyle z_{i}\textit{ hasUse }x_{1z},\ldots,x_{iz}$ (8)

The expressions below encodes enhanced sensors or object outputs with their temporal attributes.

$\displaystyle z_{i}{\textit{ hasUse }}\{(x_{1z}\_\textit{On},{\textit{% hasStartTime}t_{1z}})$ $\displaystyle\quad∼{}\ldots(x_{iz}\_\textit{On},{\textit{hasStartTime}t_{iz})}\}$ (9)

The activity context descriptors are then modelled as resources and objects class concepts accordingly and then added to the ABox so that:

$\displaystyle Z_{i}\sqsubseteq\textit{ADL}\sqcap\exists\textit{hasUse}.((x_{1z% }\_\textit{On}\sqcap\exists$ $\displaystyle\quad∼{}\textit{hasStartTime}.(=,t_{1z}))\ldots\sqcap\exists$ $\displaystyle\quad∼{}(x_{iz}\_\textit{On}\sqcap\exists\textit{hasStartTime}.(=% ,t_{iz})))$ (10)

3.2.1 Static and dynamic activities

In the home environment, activities are performed differently, in different ways and times within the 24 hour day path. Some of these activities can be performed – at specific times of the day making them have static times occurrence (Static activities), performed at different or varying times of the day (Dynamic activities), and in some cases, same or similar objects may be used to perform some of these activities. We make our analogy using Breakfast, Lunch, Dinner, Toileting and Showering as examples of activities in the home environment. Breakfast, Lunch and Dinner are examples of different activity concepts which can be performed with same or similar object interactions given that they are food related activities. In difference, they have specific times of the day they are performed making them static activities. Given their similarities, they can be modelled as subclasses of the activity Make Food, however, they differ with regards to their respective temporal properties. Whilst they inherit all the properties of Make Food by subsumption, they can be easily confused in the recognition process if modelled in the ontology without consideration to their usual times of performance. Distinction can only be achieved for them by the specification of the time intervals they are usually performed. On the other hand, activities like Toileting and Showering can be performed at any time of the day making the process of distinguishing them less dependent on their temporal properties, hence, they are dynamic activities. Dynamic activities are not constrained within any time interval. We therefore extend the ontology of activity situations to include static and dynamic activities using the 4D-fluent approach [38], requiring the temporal class concepts Timeslice and TimeInterval to be specified using the relational properties tsTimesliceOf and tsTimeIntervalOf respectively as illustrated in Fig. 3, the time intervals Interval1 and Interval2 holds the temporal information of the time slices for the static and dynamic activities respectively. An instance of a TimeSlice of an activity whether static or dynamic is linked by the property tsTimeSliceOf and property tsTimeInterval which then links this instance of TimeSlice with an instance of the class TimeInterval.

Modelling a static activity: A static activity is modelled by requiring the specification of the TimeSlice, TimeInterval class concepts and with the context descriptors for that activity. The activity concepts described above are extended so that the hasUse object property encodes the usage of the objects for the static activity by specifying the static activity as the domain class concept and ranges to all the object classes which describes the context descriptors. We further extend this with the temporal properties which requires tsTimeInterval to have domain TimeSlice and Resources and it ranges TimeInterval to capture specific time interval of the day through Interval (a sub class of TimeInterval). The time instants of the activity are captured through the tsTimeSliceOf with domain TimeSlice and Resources and it ranges to TimeSlice. With regards to Fig. 3, the expression Eq. (11) encodes a static activity so that Interval1 asserts the time interval of the day the static activity is performed using the object $x_{1z}$ .

$\displaystyle\textit{StaticActivity}\sqsubseteq\textit{ADL}\sqcap\exists% \textit{hasUse}.(x_{1z}\_\textit{On}$ $\displaystyle\quad∼{}\sqcap\exists\textit{tsTimeInterval}.\textit{Interval1})$ (11)

The expression Eq. (12) then asserts Interval1 to cover the time instant $t_{1z}$ through the property tsTimeSliceOf.

$\displaystyle\textit{Interval1}\sqsubseteq\textit{Interval}\sqcap\exists% \textit{tsTimeSliceOf}.$ $\displaystyle\quad∼{}(\textit{TimeSlice}\sqcap\exists\textit{hasStartTime}.(=,% t_{1z}))$ (12)

Modelling a dynamic activity: Similar to static activities, it is modelled by requiring the specification of the TimeSlice, TimeInterval class concepts and with the context descriptors for that activity. An instance of a TimeSlice of a dynamic activity is linked by the property tsTimeSliceOf and property tsTimeInterval and then links this instance of the class TimeSlice with an instance of class TimeInterval which may be Interval2. Interval2 ranges to cover the full 24 hour cycle of the day as asserted by expression Eq. (13).

$\displaystyle\textit{DynamicActivity}\sqsubseteq\textit{ADL}\sqcap\exists% \textit{hasUse}.(x_{2z}\_\textit{On}$ $\displaystyle\quad∼{}\sqcap\exists\textit{tsTimeInterval}.\textit{Interval2})$ (13)

The expression Eq. (14) then asserts Interval2 to cover the time instant $t_{2z}$ through the propertytsTimeSliceOf.

$\displaystyle\textit{Interval2}\sqsubseteq\textit{Interval}\sqcap\exists% \textit{tsTimeSliceOf}.$ $\displaystyle\quad∼{}(\textit{TimeSlice}\sqcap\exists\textit{hasStartTime}.(=,% t_{2z}))$ (14)

Table 3

A sample of sensor status and output

Sensor ID	Sensor state	Time
6	Microwave_On	09:00:00
7	Fridge_On	09:01:01
8	PlatesCupboard_On	09:01:05

3.3 Activity recognition by object use query

The activity recognition process is enabled by the Algorithm 3.3 which performs a mapping of the activity situation using the observed objects. A comparison is made through reasoning by the ontology to retrieve the closest activity situation described by the contexts of object observed as sensor data. The activity recognition uses object use query like constructs adapted from the Temporal Ontology Querying Language (TOQL) [38] on the knowledge base to retrieve activity situations fitting the requirements of the query. As an advantage, sensor states and status of object use as implemented in the activity ontology can be used in queries to reflect real situations of object usage in the home environment. A typical query is comprised of SQL like construct (SELECT-FROM-WHERE) for OWL which treats the ontology classes and properties like database tables and columns. An additional AT construct in the query compares the time interval for which a property is true with a time interval or instant. Considering the scenario in the home environment where sensor status and outputs captured are reported as given in Table 3. The Algorithm 3.3 enables the activity recognition process. The inputs are observed sensor along their time lines as $x_{1},\ldots,x_{n}$ from segments $d_{1},\ldots,d_{D}$ . The process maps sensor or object $x_{i}$ on to the ontology applying inference rules to determine if $x_{i}$ is a context descriptor for a static or dynamic activity. If the mapping using the inference rule returns activity, then it is reported as the activity recognised for the observed $x_{i}$ . Considering the scenario of observed objects in Table 3, the question would be “What activity does these sensors or object use in Table 1 represent at the particular time?” The query construct to provide the activity recognition is as given in the schema Eq. (15).

Input: Observed Sensors $X=\{x_{1},\dots,x_{n}\}$ in Partition of Sensor Segment $D=\{d_{1},\ldots,d_{D}\}$ , ADL ontology (ADL)Static Activity (SA) or Dynamic Activity (DA). Begin:data stream is active Extract observed object $x_{i}\in X=\{x_{1},\ldots,$ $x_{n}\}$ , from $d_{i}\in d_{1},\ldots,d_{D}$ for each $d_{i}\in D$ . Create Activities $Z\equiv Z(x_{1})\sqcup,\ldots,Z(x_{n})$ for each $x$ $\in$ $A$ do Map $x_{i}$ to a Static Activity (SA) or Dynamic Activity (DA)So that; $Z\equiv ADL\sqcap\exists$ hasUse.( $x_{i}\_\textit{On}\sqcap\exists$ hasStartTime.( $=,t$ ))Activity Inference

an activity $Z_{i}$ is returned Report (SA) or (DA) as recognised. for all Repeat process for next $x$ . Activity recognition algorithm.

Considering the scenario of observed objects in Table 3, the question would be “What activity does these sensors or object use in Table 1 represent at the particular time?” The query construct to provide the activity recognition is as given in the schema Eq. (15).

$\displaystyle\textbf{SELECT }\textit{Activities }\textit{zi},$ $\displaystyle\textit{Activities}.\textit{activityName}$ $\displaystyle\textbf{FROM }\textit{Activities},\textit{ Objects }\textit{As}$ $\displaystyle\textit{Object1},\textit{ Objects }\textit{As }\textit{Object2},$ Objects As Object3 $\displaystyle\textbf{WHERE }\textit{Activities}.\textit{hasUse:}$ $\displaystyle\textit{Object1}\text{ AND }\textit{Object1}.\textit{ObjectState}$ $\displaystyle\text{LIKE }``\textit{Microwave\_On}"\text{ AT }(09.00)$ $\displaystyle\text{AND }\textit{Activities}.\textit{hasUse:}$ $\displaystyle\textit{Object2}\text{ AND }\textit{Object2}.\textit{ObjectState}$ $\displaystyle\text{LIKE }``\textit{Fridge\_On}"\text{ AT }(09.00)$ $\displaystyle\text{AND }\textit{Activities}.\textit{hasUse:}$ $\displaystyle\textit{Object3}\text{ AND }\textit{Object3}.\textit{ObjectState}$ $\displaystyle\text{LIKE }``\textit{PlatesCupboard\_On}"\text{ AT }(09.00)$ (15)

4. Experiments and results

To validate the framework presented in this paper, we used the Kasteren et al. [7] and Ordonez et al. [6] datasets captured in two different home settings with similar events and activities (see Tables 4 and 5 for an overview of the home setting descriptions and activity instances). Our choice of these dataset was driven by the fact that the Kasteren and Ordonez dataset contain a lot of sensor activations as object use with dense sensing applied. Different types of sensors (for example pressure sensors, magnetic sensor etc tagged to home objects like microwave, dishes, cups) were used to capture object interactions representing the different activities. To further enhance the learning process, the activities contained therein have been performed in varied ways and accurately annotated in the ground truth.

We followed a 4-fold cross validation on the datasets.

Our criterion for evaluation is to compare recognised activities with the ground truth provided with the dataset based on the average true positives TP, false positives FP and false negatives FN per activity. The results are then further evaluated based on precision, recall and F-score.

Table 4
Home setting and description

	Kasteren A	Ordonez A
Setting	Apartment	Apartment
Rooms	3	4
Duration	22 days	14 days
Sensors	14	12

Table 5

Activity instances in the Kasteren and Ordonez dataset

Activities	Kasteren A	Ordonez A
Sleeping	25	14
Toileting	114	44
Leaving	36	14
Showering	24	14
Grooming	Na	51
Breakfast	20	14
Lunch	Na	9
Dinner	10	Na
Drink	20	Na
Snack	Na	11
Spare time	Na	11

Table 6

Activity concepts and the discovered context descriptors for Kasterens house A

Activities	Context descriptors
Leaving	Front door
Toileting	Hall toilet door, toilet flush
Showering	Hall bathroom door
Sleeping	Hall bedroom door
Make food	Fridge, plates cupboard, cups cupboard, groceries cupboard, microwave, freezer
Make drink	Fridge

Figure 4.

Visualisation of some activity-object distributions for Kasteren house A.

4.1 Activity-object use discovery and context description evaluation

To generate the context descriptors for routine activities, we followed the context description process described in Subsection 3.1 above. Recall the LDA process requires activity topic numbers and the bag of object observations. We used the number of activities as given in the dataset. Next, we partitioned the dataset using 60 seconds sliding windows to construct the bag of object observations in the form of a segment-object-frequency matrix as we have described previously Subsection 3.1 above. For the LDA activity-object use discovery process, we used the constructed bags of object observations as inputs, the activity topic numbers from the datasets and we set the dirichlet hyperparameters $\alpha$ to 50/K and $\beta$ to 0.01 as recommended by Steyvers and Griffiths [34]. Figures 4 and 5 show sample visualisation of some of the activity-object distributions for Kasteren and Ordonez datasets respectively. The activity context descriptors for the specific routine activities are achieved with dependency on these distributions from the LDA. The idea is that for an object to be a context describing an activity topic, it must have been assigned to an activity topic. Further, we annotated some of the activity topics in line the ground truth and matched them with their respective context descriptors as given in Tables 6 and 7.

Table 7
Activity Concepts and the discovered context descriptors for Ordonez house A

Activities	Context descriptors
Leaving	Main door
Toileting	Toilet, basin
Showering	Shower
Sleeping	Bed
Make food	Cupboard, fridge, microwave, toaster
Spare time	Seat
Grooming	Basin, cabinet

Figure 5.

Visualisation of some activity-object distributions for Ordonez house A.

Figure 6.

Common object concepts for Kasteren and Ordonez houses.

Unlike the 7 activity set in the Kasteren A ground truth, we discovered 6 activity topics in this process indicative of the activity sets which we annotated as Leaving, Toileting, Showering, Sleeping, Make Food and Drink. Our Make Food activity in this case represents Breakfast and Dinner due to same and similar context descriptors or object usage. We distinguish these activities through the ontology static activity modelling. This is similar for Ordonez A with 7 activities annotated as Leaving, Toileting, Showering, Sleeping, Make Food, Spare Time and Grooming. Make Food for the Ordonez house A represents Breakfast, Lunch and Snack.

4.2 Activity ontology and recognition performance

To this point, we have generated the context descriptors for the various activities for Kasteren A and Ordonez A houses. To facilitate activity recognition and eventual evaluation of the framework, we model the activities and the context descriptors from the previous Subsection in an ontology activity model. To enhance and support a unified ontology model and with common concepts shared and which can be reused across the similar home environments, we developed unified activity ontology for the Kasteren and Ordonez datasets as illustrated in Fig. 6. The green coloured rounded rectangles represents common object concepts in both homes. The blue rounded rectangle has been used specifically Ordonez concepts which are not shared in Kasteren concepts. This unified ontology model can also be extended and adapted further for similar homes thus reducing the amount of time taken to construct and develop activity ontologies.

Figure 7.

Common activity concepts for the Kasteren and Ordonez houses.

The activity concepts were also modelled accordingly, but due consideration was given to Make Food which represented a group of activities. The Make Food activity as it implies, corresponds to a group of activities involving making of food and ranges from Breakfast and Dinner. With regards to the Kasterens dataset, we class Breakfast, and Dinner as static activities with super class Make Food sharing same or similar context descriptors and also they are performed at specific times of the day. To further enhance shared ontology concepts and reuse, we harmonised the Kasteren and Ordonez activity concepts as illustrated in Fig. 7 onto the activity ontology to form a set of unified activity concepts. Similar to the object concepts, we have colour coded activity concepts in this unified set of activity concepts with static and dynamic activities as super classes so that the green rounded rectangle represents common activity concepts, blue rounded rectangle as activity concepts in the Ordonez house and not in the Kasteren House and the red rounded rectangle as activity concept in the Kasteren house and not in the Ordonez House. As part of our proposed framework, we added instances and individuals to the of the object concepts making the model more expressive (assertions used in populating the ABox) for example instantiating Microwave with Microwave_On to suggest the state of the object or sensor when in use. The ABox was further populated with assertions using object and data properties as we explained in Subsection 3.2 incorporating the context descriptors for the activity situations preparatory for activity recognition. The modelled activity situations or concepts are then linked to their respective context descriptors as object states through the properties as assertions added to the ABox. Activity recognition is enabled by an object use query to retrieve activity situations based on observed sensor or object data similar to the schema Eq. (15). The Algorithm 3.3 implements the recognition process.

Table 8

Activity recognition performance for Kasteren A and Ordonez A

Activities	True positives (%)		False positives (%)		False negatives (%)
	Kasteren A	Ordonez A	Kasteren A	Ordonez A	Kasteren A	Ordonez A
Leaving	100	100	0.00	0.00	0.00	0.00
Toileting	100	100	0.00	0.00	0.00	0.00
Showering	94.7	63.2	3.6	0.00	1.7	0.00
Grooming	Na	68.7	Na	26.6	Na	4.8
Sleeping	100	100	0.00	0.00	0.00	0.00
Spare time	Na	100	Na	0.00	Na	0.00
Breakfast	72.5	81.4	17.3	17.5	10.2	1.1
Lunch	Na	61.3	Na	29.5	Na	9.2
Dinner	69.2	Na	23.5	Na	7.3	Na
Drink	64.5	Na	29.8	Na	5.6	Na
Snack	Na	58.7	Na	31.6	Na	9.7

Figure 8.

Activity ontology and implementation of an object based query.

To facilitate execution of activity inference, the modelled activity ontology is imported into the java based TOQL environment as illustrated in Fig. 8. Object based queries are executed by mapping the observed objects and its temporal information from the dataset to the closest activity in the imported activity ontology through ontological reasoning.

We evaluated the proposed framework, thus allowing us to compare the activities inferred and recognized by our framework with the ground truths. Leaving, Toileting, Sleeping and Showering for both datasets were recognised with significantly high results as shown in Table 8. This performance can be attributed to the discovered object use and context descriptors for these activities. For these activities object use were accurately specific with minimal false positives. The process of discovering likely the object use for routine activities significantly ensured that these activities were associated to the objects used to perform them. Breakfast and Dinner for Kasteren house and Breakfast and Lunch for Ordonez house showed lower performance due to confusions from same and similar object use with Drink and Snack respectively. These activities have shared same and similar object interactions as observed with context description process hence been classed under the super activity Make Food. Recall that to further distinguish Breakfast and Dinner they were modelled as Static activities given the specific time of the day they are performed. To enhance their recognition, time interval properties and concepts enabled by 4D fluent approach were included. They were often recognised concurrently and led to high false positives in the process. However, the results achieved for them are quite encouraging. Overall, the average precision, recall and F-Score with the datasets as illustrated in Fig. 9 show impressive performance.

4.3 Learning performance of the framework

For further performance evaluation, we present the learning performance of our proposed activity recognition framework. Activities in the home environment can have different ways of being performed or the object use for activities may differ. A robust activity recognition model should have the ability of recognising activities irrespective of the object use and interactions. We evaluate the learning capability at the activity level further using the ground truth as the basis of evaluation. This comparison would involve the number instances of the different activities across folds at the activity level. A good model should be able to return almost the same number of activity traces as in the ground truth. The results as presented in Table 9 is indicative of good performance of the learning process of the framework. Leaving, Toileting, Sleeping and Showering were recognised with significantly high instances an minimal difference with the ground truth for both datasets. Breakfast, Lunch and Dinner showed lower performance due to confusions from same and similar object use with Drink. The number of correctly recognised instances in comparison to the ground truth is also very encouraging.

Table 9
Summary of correctly recognised activity instances for Kasteren and Ordonez datasets

Activities	Ground truth		Instances recognised		Differences
	Kasteren A	Ordonez A	Kasteren A	Ordonez A	Kasteren A	Ordonez A
Leaving	36	14	36	14	0	0
Toileting	114	44	114	44	0	0
Showering	24	14	22	9	2	5
Grooming	Na	51	Na	35	Na	16
Sleeping	25	14	25	14	0	0
Spare time	Na	11	Na	11	Na	0
Breakfast	20	14	14	11	6	3
Lunch	Na	9	Na	6	Na	3
Dinner	10	Na	7	Na	3	Na
Drink	20	Na	13	Na	7	Na
Snack	Na	11	Na	6	Na	5

Figure 9.

Average precision, recall and F-Score for Kasteren A and Ordonez A datasets.

Figure 10.

A summary of evaluations using the Kasteren house A dataset for A $=$ Kasteren et al. [7]; B $=$ Ordonez et al. [6]; C $=$ Ye [39]; D $=$ Riboni et al. [35]; E $=$ this paper.

4.4 Comparison with other recognition approaches

We compared the results we obtained with the results reported by Kasteren et al. [7], Ordonez et al. [6], Ye [39], Riboni et al. [35] with ours for the Kasteren and House A as illustrated in the Fig. 10. Experimental methodology differs for the reported works. Kasteren et al. [7], Ordonez et al. [6] and Riboni et al. [35] all performed the evaluations using a leave one day out methodology. Ye [39] used a 10 fold validation just as we have used a 4 fold validation methodology. Although Ye [39] did not use timeslices which lead to improved results, we have used 60 seconds timeslices similar to Kasteren et al. [7], Ordonez et al. [6] and Riboni et al. [35]. Comparing our results directly with these other methodologies, our work performed significantly better with 91.9% for the F-Score. It is assumed the weaker performance reported by Kasteren et al. [7], Ordonez et al. [6] and Riboni et al. [35] might be due to the effect of evaluation methodology a leave one day out which meant fewer day representation for the object data. Comparing our results with Ye [39], we achieved a slightly higher F-Score which meant our proposed frame work for recognition is robust and significantly good.

5. Summary and conclusion

The activity recognition enhanced using topic model we propose in this paper provides the basis to learn and recognise activities. We carried out experiments using the Kasteren and Ordonez datasets. We evaluated the performance of the framework to recognise activities. In addition to the experiments, we compared our results to the results published using the same dataset in other literature. Based on the experiments and evaluations, we discuss the benefits and limitations of the proposed approaches.

•

Activity-object use and context description process: As part of the framework, we proposed the acquiring knowledge of object use by the object use discovery and activity context descriptions for the activities. However, activities like Breakfast, Lunch and Dinner sharing same or similar object use are considered as activity situations which can be made distinct by modelling them as static activities in the ontology. The main benefit of this process is its ability to discover unique object use as context descriptors for the activity situations. Limitations may arise for other similar activity situations like Drink and Snack as we observed with the datasets.

•

Performance of the activity recognition process: Experiments carried out on the datasets suggest good recognition performance for activities. Although the performance was encouraging for most activities, recognition were confused for activities sharing same and similar object use. Notably in this case was Drink and Snack which we modelled as dynamic activities and Breakfast, Lunch and Dinner modelled as static activities. Given the general performance of this activity recognition process as illustrated with figure 8, the activity recognition process on the average is significantly comparable.

•

Model learning performance: The aim of this evaluation was to assess the model learning ability. From the results, the contexts descriptors which led to activity situations in the ground truth are similar to the contexts descriptors we discovered hence the result achieved at the activity level. We obtained almost the same number of activity traces for the datasets in comparison with the ground truth suggesting good and significant learning.

With the experiments, assessments and evaluations using publicly available datasets, we can say that, i) The process of activity-object use and context description of activity situation provides accurately the needed object and activity concepts for the ontology modelling process. ii) Modelling activities as static and dynamic activities helps to improve activity recognition especially for activities with same and similar object interactions. iii) Given the results from the activity recognition process in comparison with other results published using the same datasets, we conclude it is significantly good and encouraging. The experimental and evaluation process using these datasets suggests that the features, components and the entire activity recognition process have been fully verified.

The future work should involve extending the ontology activity model by the consideration of more contextual features for the different types of sensing devices. It should also consider extending the temporal entities and features for progressive activities as they evolve.

Footnotes

http://www.wikihow.com.

References

Akdemir

Turaga

and Chellappa

, An ontology based approach for activity recognition from video, In Proceedings of 16th ACM International Conference on Multimedia (MM, 08) (2008).

Leo

Medioni

Trivedi

Kanade

and Farinella

, Computer vision for assistive technologies, Computer Vision and Image Understanding 154 (2017), 1–15.

Nguyen

T.-H.-C.

Nebel

J.-C.

and Florez-Revuelta

, Recognition of activities of daily living with egocentric vision: A review, MDPI, Sensors 16(1) (2016), 72.

Huynh

D.T.G.

, Human Activity Recognition with Wearable Sensors, PhD Thesis, Technische Universit Darmstadt, 2008.

Labrador

M.A.

and Yejas

O.D.L.

, An ontology based approach for activity recognition from video, Human Activity Recognition: Using Wearable Sensors and Smartphones, CRC Press (2013).

Ordonez

de Toledo

and Sanchis

, Activity recognition using hybrid generative/discriminative models on home environments using binary sensors, MDPI Sensor Open Access (2013).

van Kasteren

Englebienne

and Krose

B.J.A.

, Human Activity Recognition from Wireless Sensor Network Data: Benchmark and Software, Atlantis Press Book, 2010.

Chen

Nugent

C.D.

and Wang

, A knowledge-driven approach to activity recognition in smart homes, IEEE Transactions on Knowledge and Data Engineering (2011).

Chen

Nugent

Hoey

Cook

and Yu

, Sensor-based activity recognition, In IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 42 (2012), 790–808.

10.

Okeyo

Chen

Wang

and Sterritt

, Dynamic sensor data segmentation for real-time activity recognition, Pervasive and Mobile Computing (2012).

11.

Modayil

Bai

and Kautz

, Improving the recognition of interleaved activities, In 10th International Conference on Ubiquitous Computing (Ubi Comp 2008) (2008).

12.

Tapia

E.M.

Intille

S.S.

Larson

Haskell

Wright

King

and Friedman

, Real-time recognition of physical activities and their intensities using wireless accelerometers and a heart rate monitor, In 11th IEEE International Symposium on Wearable Computers (2007).

13.

Patterson

D.J.

Liao

Fox

and Kautz

, Inferring high-level behavior from low-level sensors, In 5th Conference on Ubiquitous Computing (Ubi Comp 03) (2003).

14.

Patterson

D.J.

Kautz

D.F.H.

and Philipose

, Fine-grained activity recognition by aggregating abstract object usage, In 9th IEEE International Symposium on Wearable Computers (2005).

15.

Langley

Iba

and Thompson

, An analysis of Bayesian classifiers, In 10th National Conference on Artificial Intelligence (1992).

16.

Tapia

E.M.

Intille

S.S.

and Larson

, Activity recognition in the home using simple and ubiquitous sensor, In 2nd International Conference in Pervasive Computing (PERVASIVE 2004) (2004).

17.

Katayoun

and Gatica-Perez

, Discovering routines from large-scale human locations using probabilistic topic models, ACM Transactions on Intelligent Systems and Technology (TIST) 2(1) (2011).

18.

Huynh

Fritz

and Schiele

, Discovery of activity patterns using topic models, In 10th International Conference on Ubiquitous Computing (Ubi Comp 08) (2008).

19.

Huynh

Blanke

and Schiele

, Scalable recognition of daily activities with wearable sensors, In 3rd International Conference on Location and Context Awareness LOCA 07 (2007).

20.

Lee

and Mase

, Activity and location recognition using wearable sensor, In IEEE Pervasive Computing (2002).

21.

Bao

and Intille

S.S.

, Activity recognition from user-annotated acceleration data, In 2nd International Conference Pervasive Computing (PERVASIVE, 2004) (2004).

22.

Cao

Masoud

O.T.

Boley

and Papanikolopoulos

, Human motion recognition using support vector machines, In Computer Vision and Image Understanding 113 (2009), 1064–1075.

23.

Qian

Mao

Xiang

and Wang

, Recognition of human activities using svm multi-class classification, In Pattern Recognition Letters 31(2) (2010), 100–111.

24.

Liao

Fox

and Kautz

, Extracting places and activities from GPS traces using hierarchical conditional random fields, In Int J Robotics Res (2007).

25.

Huynh

and Schiele

, Unsupervised discovery of structure in activity data using multiple eigenspaces, In 2nd International Workshop on Location and Context Awareness (LoCA 2006) (2006).

26.

Gruber

T.R.

, A translation approach to portable ontology specifications, Knowledge Acquisition 5 (1993), 119–220.

27.

Latfi

and Lefebvre

, Ontology-based management of the telehealth smart home, dedicated to elderly in loss of cognitive autonomy, In CEUR Workshop Proceedings (2007).

28.

Alirezaie

Renoux

Köckemann

Kristoffersson

Karlsson

Blomqvist

Tsiftes

Voigt

and Loutfi

, An ontology-based context-aware system for smart homes: E-care@home, MDPI, Sensors 17(7) (2017), 1586. doi: 10.3390/s17071586.

29.

Chen

and Nugent

C.D.

, Ontology-based activity recognition in intelligent pervasive environments, International Journal of Web Information Systems 5(4) (2009), 410–430.

30.

Yamada

Sakamoto

Kunito

Yamazaki

Isoda

and Tanaka

, Applying ontology and probabilistic model to human activity recognition from surrounding things, IPSJ Digital Courier 3 (2007), 506–517.

31.

Okeyo

Chen

Wang

and Sterritt

, A hybrid ontological and temporal approach for composite activity modelling, In IEEE 11th International Conference on Trust, Security and Privacy in Computing and Communications (2012).

32.

Ihianle

I.K.

Naeem

and Tawil

, A dynamic segmentation based activity discovery through topic modelling, In IET International Conference on Technologies for Active and Assisted Living (TechAAL 2015) (2015).

33.

Ihianle

I.K.

Naeem

and Tawil

, Recognition of activities of daily living from topic model, In Procedia Computer Science (2016).

34.

Ihianle

I.K.

Naeem

and Islam

, Ontology-driven activity recognition from patterns of object use, In Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, UbiComp ’17 (2017), 654–657.

35.

Riboni

Pareschi

Radaelli

and Bettini

, Is ontology-based activity recognition really effective? In IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops) (2011).

36.

Blei

A.Y.

and Jordan

M.I.

, Latent dirichlet allocation, Machine Learning (2003).

37.

Griffths

T.L.

and Steyvers

, Finding scientific topics, In Proceedings of the National Academy of Sciences (2004).

38.

Baratis

Petrakis

E.G.M.

Batsakis

Maris

and Papadakis

, TOQL: Temporal ontology querying language, Advances in Spatial and Temporal Databases (2009).

39.

, Exploiting semantics with situation lattices in pervasive computing, PhD Thesis, University College Dublin, 2009.

Recognising activities of daily living from patterns of object use

Abstract

Keywords

1. Introduction

3. Overview of our activity recognition approach

3.1 Context description module

3.1.1 Activity-object use discovery by Latent Dirichlet Allocation

Table 1 Ontology concepts used

Table 4 Home setting and description

Table 7 Activity Concepts and the discovered context descriptors for Ordonez house A

Table 9 Summary of correctly recognised activity instances for Kasteren and Ordonez datasets

5. Summary and conclusion

Footnotes

References

Table 1
Ontology concepts used

Table 4
Home setting and description

Table 7
Activity Concepts and the discovered context descriptors for Ordonez house A

Table 9
Summary of correctly recognised activity instances for Kasteren and Ordonez datasets