An integrated semantic framework for designing context-aware Internet of Robotic Things systems

Abstract

The paradigm of Internet of Robotic Things (IoRT) extends the scope of the Internet of Things by endowing any object with the three main typical functions of any robotic system: perception, actuation, and control. This paper presents a semantic framework for context-aware IoRT systems to support the development of applications for monitoring and managing IoRT systems. A knowledge representation framework, called SmartRules, is proposed for context modeling. SmartRules is a production rules language that enables reactive reasoning based on the closed world and unique name assumptions. It allows producing actions based on contextual information represented in a dedicated ontology language, called $\mu$ -Concept. An operational platform, centered on the notion of manageable object (MO), is also proposed to abstract the access to any physical or virtual device, which can communicate through the Internet. In addition, an integrated methodology and tools are proposed for guiding the development and deployment of context-aware Semantic IoRT systems, and, in particular, for defining context semantics and creating context management rules. To show the effectiveness of the proposed framework in ambient assisted living (AAL) applications, an IoRT system dedicated to the monitoring and assistance of an elderly person during his/her daily living activities is described and evaluated.

Keywords

Internet of Things robotics ontology reasoning ambient assisted living

1. Introduction

There is a growing consensus about the need for adding cognitive capabilities to the connected devices produced today [1, 2]. Based on this consideration, the Internet of Robotic Things (IoRT) paradigm has been proposed for the first time in [3], in which intelligent set of devices connected to the Internet can monitor events, fuse sensor data from a variety of sources, use local and distributed intelligence to determine a best course of action, and then act to control or manipulate objects in the physical world. In our vision, the adoption of the IoRT paradigm makes that any connected object or thing can be endowed with perception, actuation and control, the three main typical functions of any robotic system. Moreover, IoRT systems also allow overcoming the limitations of standalone robots, by enabling smart collaborations among different types of robots and connected devices [2]. Each of these entities provides robotic services that can uniformly measure or control the world, as well as interact with people in both physical and cyber worlds.

Creating these robotic services comes with many challenges concerning how smart objects can process heterogeneous data about infrastructure and users, not only to infer the context within the system, but also to make autonomous decisions without the intervention of a teleoperator. Among the main challenges. we highlight the semantic heterogeneity of sensor data, as well as context and action modeling. The semantic heterogeneity problem usually rises from the need to integrate the multitude of protocols employed by different IoT [4] and Networked Robots platforms [5]. It also rises when integrating heterogeneous information captured by different sensors into a coherent picture of the environment. Furthermore, in order to react autonomously, IoRT systems must have a certain level of context recognition, called context awareness. They must be able to represent the entities composing the environment and how the context evolves. The designers of robotic services must specify the context within the system by defining the entities and processes present in the IoRT environment, as well as the constraints on how these entities correlate. Context awareness also includes how IoRT systems react to context changes.

Figure 1.

General architecture of a Semantic IoRT Platform. The Sensors and Actuators Platform abstracts the access to sensors (S) and Actuators (A). The Symbolic Reasoning layer interprets contextual entities from data extracted from sensors, subsequently generating actions based on domain knowledge represented in ontologies and rules. These actions are then decoded by the Platform into actual commands to actuators.

The concept of Semantic IoRT is introduced in this paper to tackle these challenges by using high-level symbolic representations and first-order logic based reasoning. In general, semantic IoRT systems work by abstracting and integrating low-level data from heterogeneous sensors into high-level knowledge structures representing context, which is subsequently processed by symbolic reasoning mechanisms to trigger actions in actuators (Fig. 1). This approach has generally several advantages. First, high-level symbolic models provide a unified view of the context to facilitate the task of integrating together heterogeneous entities in IoRT systems. In addition, contextual models and action triggers are created by the designer to have a better control of the system behavior, also making it easier to adapt robotic services to new contexts.

Knowledge in Semantic IoRT systems is frequently represented by ontologies and rules. An ontology is a logical theory expressing a shared conceptualization of a specific domain [6, 7]. One of the main aspects of ontologies is that they must be formal in order to be processable by computers. Ontologies are composed mainly by taxonomies of concepts and relations about the domain they represent. When properly formalized, ontologies can serve as semantic knowledge models in applications that need to process contextual information. A notable example of domain ontology is FOAF,1 which describes a taxonomy of concepts and relations that are common in the social networks domain. SOUPA [8] and CONON [9] are the most prominent ontology proposals for representing context according to the requirements of pervasive computing and ambient intelligence. For instance, the SOUPA ontology for ubiquitous and pervasive systems includes concepts about agents, beliefs, desires, time, space and privacy. According to [10], based on a context ontology and asserted knowledge, a system should be able to (i) automatically derive new knowledge about the current context, and (ii) detect possible inconsistencies in the context information.

Another important aspect of Semantic IoRT systems is the use of production rules. There is a long tradition of using production rules to represent procedural knowledge in reactive systems (see, for instance, [11]). They are essentially used to detect specific information patterns being generated in the system and then to infer new contextual knowledge or trigger new actions. The challenge in past years has been to develop a systematic way of combining ontologies and production rules by taking into account ontology and rule authoring (i.e., see [12] for a review).

This article presents a semantic framework for context-aware IoRT systems, developed during the SemBySem2 and Web of Objects3 projects, both conjoint research efforts aimed at developing semantic approaches for monitoring and managing IoRT systems. Focusing on context modeling in a heterogeneous IoRT environment, the problem of ontology and production rules integration is addressed in the scope of Semantic IoRT systems. This paper has three main contributions. First, a knowledge representation framework, called SmartRules, is proposed for context modeling [13, 14, 15]. SmartRules is a production rules language specific for producing actions based on contextual information represented in a dedicated ontology language, called $\mu$ -Concept. SmartRules is particularly suitable for IoRT systems that need to perform reactive, online reasoning, as it uses unique name assumption and closed-world reasoning. It also includes a series of constructs that facilitate the handling of incomplete information due to failures in perception. In this paper, we specify an abstract syntax and semantics to the $\mu$ -Concept language.

The second contribution is an operational platform for context awareness for IoRT, centered on the notion of manageable object (MO). A MO is any physical or virtual object that is manipulated by an IoT system. It includes sensors and actuators, as well as people, home objects, doors, robots and rooms. MOs are represented within the system by interfaces which abstract access to the MO’s state and operations. The proposed platform has two main layers that abstract the complexity and heterogeneity of the IoRT environment using MOs, as well as a run-time reasoning system for managing the context logics independently from the complexity and heterogeneity of a IoRT environment. This logic is defined by an ontology that describes the semantics of MOs and the reactive rules to manage the context.

The third contribution consists of an integrated methodology and tools for helping designers to develop and deploy context-aware Semantic IoRT systems, and, in particular, for defining context semantics and creating context management rules. The set of tools includes a platform to abstract low-level access to heterogeneous perception and actuation devices, as well as a SmartRules-based platform for representing, reasoning and querying contextual data. The proposed design and methodology should minimize the need to code rewriting when adapting IoRT systems from one context to another. We show that these tools allow to develop Semantic IoRT systems with relatively less effort, leveraging the flexibility of the framework.

The remainder of the paper is structured as follows. Section 2 presents an overview of related work on semantic approaches used in IoT and robotic systems. Section 3 introduces a knowledge representation framework based on SmartRules. Section 4 describes the software platform, tools and methodology for semantic IoRT systems. Some application scenarios are described and analysed in Section 5. Finally, a conclusion and some perspectives are given in Section 6.

2. Related work

Significant work has been done in recent years to enhance IoT and robotic systems with knowledge-based tools, usually by adding semantic layers to their core systems. Contributions in both areas are relevant to semantic IoRT systems. Thus, in this section, we review the literature in both semantic IoT and robotics.

A multitude of projects proposed frameworks and methods for annotating and managing sensors’ data with semantic information. For example, the SPITFIRE project introduced the concept of Semantic Web of Things [16], a network that should provide access to sensor data in a way similar to how the WWW provides access to general resources. Sensor data are recorded in triple-stores where they can be queried using a standard language, such as SPARQL.4 These data are also described using an adaptation of the Semantic Sensor Network (SSN) [17] ontology and other related ontologies. Henson et al. [18] propose SemSOS (Semantic Sensor Observation Services) a platform for semantically-enabled Sensor Observation Services. SemSOS provides the ability to query high-level knowledge of the environment as well as low-level raw sensor data. SensorML [19] aims at enabling interoperability by using ontologies, at both the syntactic and semantic levels. Lee et al. [20] defined the Semantic Ontology model and a use-case with collaboration of objects to deliver an infrastructure simplifying the management of IoT applications.

It is important to note that most of the ontologies used in semantic IoT and robotic systems are specified using the Ontology Web Language (OWL).5 OWL is the de facto ontology representation language in the Semantic Web and other technologies. It has been proposed with the goal of having a high level of expressiveness (i.e., expressive constructs), while supporting efficient reasoning services.

In the field of robotics, KNOWROB (KNOWledge processing for ROBot [21]) and OpenRobots Common Sense Ontology based platform (ORO [22]) are the main representative approaches that propose an ontology-based centralized module for robot commonsense knowledge management. Both KNOWROB and ORO propose respectively specific OWL ontologies that are defined with root concepts of the OpenCyc upper ontology [23], but these two ontologies differ in their scope. The KNOWROB ontology focuses on modeling robot perceptions and their relations with Robot tasks whereas the ORO ontology focuses on modeling robot perceptions and their relations with the interactions and dialogues between human and robot. KNOWROB proposes built-in reasoning procedures that interleave RDF Querying with the execution of Prolog functions called computable predicates. Alternatively, ORO uses SPARQL and description logics to classify and infer new relations from any asserted knowledge provided by robot perception modules. Recently, the CORA ontology was standardized in IEEE 1872–2015 to ensure an unambiguous definition of core notions of Robotics and related fields [24, 25]. Based on the SUMO top-level ontology, CORA axiomatizes concepts that describe the robot itself and its interactions with the environment and/or other robots.

With respect to the combination of production rules with ontologies, a multitude of context-aware ubiquitous computing systems employ some sort of rule language with OWL ontologies (see [11], for instance). An early example of such systems is Cobra [26, 8], a software agent capable of building representations and reasoning about contextual information. It represents context as an OWL ontology (SOUPA) and infer new information using domain-specific rules. In another approach, [27] proposes a data model for semantic IoT systems based on semantic link networks. It is capable of representing knowledge about objects and events, as well as links (i.e., relations) between them. The authors propose several rule patterns that derive new information based on objects and events. However, the authors are not clear about how the work can be implemented and the rules can be used to produce actions.

More recent systems have turned themselves to web standards. For instance, Gyrard et al. [28] introduce the M3 ontology, an extension of the SSN ontology, used to describe domain-dependent measurements. The authors also introduce what they call Linked Open Rules. These are Semantic Web rules (SWRL [29]) that allow simple reasoning on domain knowledge. For example, they allow to infer that a temperature reading of 34 ${}^{\circ}$ C is low if it is being read by a thermometer attached to a person or high if it is being used to measure weather. While the adherence to Semantic Web standards is a positive point, OWL (or OWL 2), SWRL and WRL languages do not allow for representation as rich as $\mu$ -Concept and SmartRules languages. For example, W3C rule language proposals SWRL, for Semantic Web Rule Language and WRL, for Web rules language, are Horn rules; none of them includes any form of reactive rules and does not have a native way for allowing the creation of new ontology instances, like SmartRules does. Indeed, SWRL provides an OWL (Full) ontology while WRL includes a mapping to OWL-DL6 but only at the core level that does not include rules. Moreover, SWRL and WRL propose to extend OWL into two different ways by providing means to express OWL-DL axioms. The Semantics of Business Vocabulary and Rules (SBVR) is an OMG specification that provides an unambiguous, meaning-centric, multilingual, and semantically rich capability for defining meanings of the language used in the industry to produce functional and business specifications of complex systems, with an emphasis on human understanding. However, SBVR cannot be used as machine readable language with rules that can be fired by an inference engine to trigger actions.

More in line with the approach proposed in this paper, the recent CONSERT approach [30] is an approach for context meta-modeling. It defines a meta-model for representing contextual assertions about objects, including constraints and rules. The model is instantiated as an OWL model. Rules are intended to derive new assertions based on domain data. CONSERT also includes an inference engine that manages separately different tasks such as querying, checking constraints violation and rule execution. It is similar to the approach proposed in this paper in that it provides a context representation framework composed of language and rules, with specific constructs for context management problem, as well as a reasoning framework. On the other hand, the proposed approach includes also an abstraction software layer (middleware) for manageable objects integration.

In general, the aforementioned approaches address the specific challenge of using ontologies for knowledge management in IoT and robotic systems. Most of them use SWRL rules or SPARQL-based queries to extract contextual knowledge, which constrain the expressiveness of the system or leads to inconvenient reifications. The solutions used to overcome this problem, such as in SOUPA, CONON, KNOWROB, ORO, are relatively complex and require from application designer to write reasoning procedures for different reasoning systems and link them with the OWL ontology. In addition, the reasoning in these approaches is not based on the closed world assumption, which limits the application designer in writing non-monotonic reasoning procedures that support negation as failure or assertions’ retraction or update.

Figure 2.

Example of SmartRule and $\mu$ -Concepts and their relationship: (a) Stove, Person and _SwitchOff described using the abstract syntax of $\mu$ -Concepts; (b) SmartRule that deactivate the stove if the system detects that there is no person at home; (c) Example of $\mu$ -Concept XML syntax, defining the concept Stove.

3. Contextual knowledge management language for IoRT

The cornerstone of the proposed framework isSmartRules, an ontology and rules representation language for Semantic IoRT Systems [14]. The main objective of SmartRules is to allow IoRT systems to instantiate contextual knowledge and react according to it, while abstracting the complexity and heterogeneity of the MOs populating the IoRT environment. SmartRules is composed of two sub-languages: the SmartRules sub-language itself, for the authoring of context management rules; and the $\mu$ -Concept sub-language, for defining the ontology that describes the IoRT environment and associated context. Context management rules specified in the SmartRules sub-language solely rely on the entities specified in an ontology defined with the $\mu$ -Concept sub-language.

Figure 2 shows simple first example of SmartRules and $\mu$ -Concept. It depicts examples of concepts, actions and rule defined using both languages. Theaction _SwitchOff can be seen as a kind of command that can be applied to instances of the concept Stove. The rule SafetyRule infers that the stove must be turned off if there is no person at home. The switching off action is produced by the instantiation of the action _SwitchOff. This instance can then be processed by the platform actuators. The two sub-languages are always used together for representing MOs, as well as for reasoning efficiently on these MOs to produce new and high-level contextual information or actions that can be executed on/by these MOs when needed. Next, we present both sub-languages in more detail.

3.1 The $\mu$ -Concept sub-language

$\mu$ -Concept is a sub-language for specifying ontologies describing MOs and other entities in IoRT environments. It can be used to model context, particularly physical objects and actions that can be performed by or on them. To allow the reuse of concepts defined in OWL ontologies, $\mu$ -Concept is based on RDF Schema (RDFS), one of the Semantic Web languages [31], and include constructs commonly used in description logics [32]. Like RDFS, the concrete syntax of $\mu$ -Concept language is based on the RDFS XML schema (e.g. Fig. 2c). For a clear and rigorous presentation of the $\mu$ -Concept language, its formal, abstract syntax that captures its main constructs is described in the following.

As in RDFS, $\mu$ -Concepts provide three main constructs, namely, concepts, properties and individuals. For example, the notion of parent can be represented in $\mu$ -Concept abstract syntax as follows:

Concept(Parent

subConceptOf(Person)

restriction(hasChild

minCardinality(1) range(Person)))

This axiom defines the concept of Parent based on some restrictions. The first one specifies the class Parent as a subclass of Person. The second restriction indicates that all instances of Parent must have at least one relationship hasChild with another person. Only property restrictions are explicitly defined with the keyword restriction. A concept definition axiom must be understood as a definition of necessary conditions. The language also supports other common property restrictions found in other representation languages, such as maximum cardinality, range type and interval. Furthermore, the language introduces some other restrictions that are relevant for IoRT systems, such as default and static values. The default value of a property at class level is the value that the property is assumed to have in an instance of the class if no other value is explicitly defined. For example, the axiom:

Concept(LightBulb

restriction(isOn default(false)))

specifies that instances of Light Bulb have a default value, which can be on or off. This feature is particularly useful in reactive systems when a sensor is not able to provide context information due to a given perception error. In this case, the IoRT system will infer the context from default values. On the other hand, static values set properties with unchangeable values. For example, the axiom:

Concept(BrokenLightBulb

subClassOf(LightBulb)

restriction(isOn static(false)))

specifies that a broken light bulb cannot be turned on.

Restrictions can also be defined as a function of other property values, a feature called property aggregation. For example,

Concept(SocialEvent

restriction(attendance

aggregationV: Count(attendedBy)))

describes that the value of the property attendance in Social Event is given by the number of individuals related to a social event by the relation attendedBy. The language includes several pre-defined aggregation functions, such as Union, Sum, Count, Minimum, Maximum and Average. These functions can return single values (-V) or a set of values (-S).

Besides, one can state the existence of instances. For example,

Instance(LIGHT_BULB#5

type(LightBulb)

value(isOn true))

specifies that an instance named LIGHT_BULB#5 is a Light Bulb and is turned on. Also, it is possible to define concepts extensionally through enumerations, for example:

Enumeration(DoorState OPEN# CLOSED# UNDEF#)

specifies a concept Door State which is entirely defined by a class of three instances. Furthermore, the language allows the definition of ordered lists of instances. For example,

InstanceList(LOW# MID# HIGH#)

which defines an ordered list where it is assumed that LOW# $\leqslant$ MID# $\leqslant$ HIGH#. Lists of datatype literals can be specified by XSD lists.

The $\mu$ -Concept language supports the specification of properties independently from the concepts. For example,

Property (livesIn domain(Person) range(Place))

specifies that the relation livesIn associates instances of Person with instances of Place. The language also allows one to define more elaborated restrictions on properties, in a similar way to concept restrictions. For instance,

Property (isOn

domain(Object) range(xsd: Boolean)

default(false))

describes that the default value of the isOn relation is false. Concept restrictions can override property restrictions.

The $\mu$ -Concept language distinguishes normal concepts from action concepts. Action concepts are restricted concepts that are associated to normal concepts, denoting actions that can be executed by the MOs represented by these normal concepts. All actions have at least one domain concept. The relationship indicates what entities can produce which actions. Instances of specific action concepts can be seen as action commands that should be processed by an actuator or system. For example, an action concept Open associated to the concept Door can be defined as follows:

Action (_Open domain(Door))

An instance of _Open specifies the need to execute an action on an instance of Door. In the platform presented in Section 4, these instances are captured and sent to actuators or displays. Action concepts can be described with properties, as with normal concepts. However, the language distinguishes between two types of restrictions: input restrictions, which describe the input needed to execute the action by the MO; and output restrictions, which specifies the prop- erties corresponding to the action outputs. For instance,

Action (_MoveTo domain(MobileRobot)

I: restriction(fromPosition minCardinality(1))

I: restriction(toPosition minCardinality(1))

O: restriction(position minCardinality(1)))

specifies the action _MoveTo that can be executed by a mobile robot. This action takes both the start and end positions as input and results in a new position after the movement execution.

3.1.1 Syntax and semantics

The abstract syntax is described here by means of a variation of Extended BNF ([33, 34]) and includes the most relevant constructs of the language.7 Let the vocabulary $V$ be the pairwise disjoint union $V_{C}\cup V_{A}\cup V_{P}\cup V_{D}\cup V_{o}\cup V_{d}\cup V_{f}$ , such that $V_{C}$ is set of concept names, $V_{A}$ is set of action names, $V_{P}$ is a set of property names, $V_{D}$ is a set of datatype names (i.e., XSD types8), $V_{o}$ is a set of instance names, $V_{d}$ is a set of literals and $V_{f}$ is a set of function names. Let $C\in V_{C}$ ; $A\in V_{A}$ ; $P\in V_{P}$ ; $D\in V_{D}$ ; $o\in V_{o}$ ; $d\in V_{d}$ ; $f\in V_{f}$ ; $W\in V_{C}\cup V_{A}\cup V_{P}$ (i.e., $W$ is a concept, action or property name); and $v\in V_{o}\cup V_{d}$ (i.e., $v$ is an instance name or literal). Let the variables $n,m\in\mathbb{N}$ . Also, $\mathrm{ConceptThing}\in V_{C}$ $\mathrm{PropertyThing}\in V_{P}$ are distinguished concept and property names, respectively. The syntax of the abstract syntax of the $\mu$ -Concept language is given by the following grammar:9

<Ontology> ::= ‘Ontology(’ $\{<axiom>\}$ ‘)’

<axiom> ::= ‘Concept(’ $C$ $\{<ddesc>\}$ ‘)’ ‘Enumeration(’ $C$ $\{o\}$ ‘)’ ‘Property(’ $P$ $\{<pdesc>\}$ ‘)’ ‘FunctionalProperty(’ $P$ $\{<pdesc>\}$ ‘)’ ‘Action(’ $A$ $\{$ ‘domain(’ $C$ ‘)’ $\}$ $\{<actionIO>\}$ ‘)’ ‘Instance(’ $o$ $\{<odesc>\}$ ‘)’ ‘InstanceList(’ $\{o\}$ ‘)’

<ddesc> ::= ‘subClassOf(’ $C$ ‘)’ | <desc>

<desc> ::= ‘restriction(’ $R$ $\{<restriction>\}$ ‘)’

<odesc> ::= ‘type(’ $C$ ‘)’ ‘value(’ $P$ $v$ ‘)’

<restriction> ::= ‘range(’ $C$ ‘)’ ‘range(’ $D$ ‘)’ ‘range(’ $D$ $d_{min}$ $d_{max}$ ‘)’ ‘minCardinality(’ $n$ ‘)’ ‘maxCardinality(’ $m$ ‘)’ ‘static(’ $v$ ‘)’ ‘default(’ $v$ ‘)’ ‘aggregationV:’ $f$ ‘(’ {W} ‘))’ ‘aggregationS:’ $f$ ‘(’ {W} ‘))’

<pdesc> ::= ‘subPropertyOf(’ P ‘)’ ‘domain(’ $C$ ‘)’ ‘domain(union(’ $\{C\}$ ‘)’ ‘range(’ $C$ ‘)’ ‘range(’ $D$ ‘)’ ‘range(’ $D$ $d_{min}$ $d_{max}$ ‘)’ ‘minCardinality(’ $n$ ‘)’ ‘maxCardinality(’ $m$ ‘)’ ‘static(’ $v$ ‘)’ ‘default(’ $v$ ‘)’ ‘aggregationV:’ $f$ ‘(’ {W} ‘))’ ‘aggregationS:’ $f$ ‘(’ {W} ‘))’

The actual semantics of the language is given by an interpretation $\mathcal{I}=(\Delta^{\mathcal{I}},\Delta^{\mathcal{I}}_{\mathcal{D}},\cdot^{% \mathcal{I}})$ , where $\Delta^{\mathcal{I}}$ is a non-empty set of the domain of interpretation domain and $\Delta^{\mathcal{I}}_{\mathcal{D}}$ is the datatype domain ( $\Delta^{\mathcal{I}}\cap\Delta^{\mathcal{I}}_{\mathcal{D}}=\emptyset$ ). The domain of interpretation of concept instances is separated from the domain of interpretation of action instances, such that $\Delta^{\mathcal{I}}=\Delta^{\mathcal{I}}_{\mathcal{C}}\cup\Delta^{\mathcal{I}% }_{\mathcal{A}}$ , and $\Delta^{\mathcal{I}}_{\mathcal{C}}\cap\Delta^{\mathcal{I}}_{\mathcal{A}}=\emptyset$ . The interpretation function $\cdot^{\mathcal{I}}$ gives the interpretation of expressions of the language according to Table 1. For simplicity, only a general interpretation for aggregation functions is provided here. These functions are calculated at runtime by the rule engine.

Table 1
Interpretation of main language expressions

Construct	Expression	Semantics
Atomic concept	$C$	$C^{\mathcal{I}}\subseteq\Delta^{\mathcal{I}}_{\mathcal{C}}$
Atomic action concept	$A$	$A^{\mathcal{I}}\subseteq\Delta^{\mathcal{I}}_{\mathcal{A}}$
Datatype	$D$	$D^{\mathcal{D}}\subseteq\Delta^{\mathcal{I}}_{\mathcal{D}}$
Property	$P$	$P^{\mathcal{I}}\subseteq\Delta^{\mathcal{I}}\times(\Delta^{\mathcal{I}}\cup% \Delta^{\mathcal{I}}_{\mathcal{D}})$
Individual	$o$	$o^{\mathcal{I}}\in\Delta^{\mathcal{I}}$
Datatype value	$v$	$v^{\mathcal{D}}\in\Delta^{\mathcal{I}}_{\mathcal{D}}$
TopConcept	$\mathrm{ConceptThing}$	$\mathrm{ConceptThing}^{\mathcal{I}}=\Delta^{\mathcal{I}}$
TopProperty	$\mathrm{PropertyThing}$	$\mathrm{PropertyThing}^{\mathcal{I}}=\Delta^{\mathcal{I}}\times\Delta^{% \mathcal{I}}\cup\Delta^{\mathcal{I}}_{\mathcal{D}}$
Class restrictions	Expression ( $E$ )	Semantics ( $E^{\mathcal{I}}$ )
Subclass restriction	$\textit{subclass}(C)$	$E^{\mathcal{I}}=C^{\mathcal{I}}$
Property restriction	$\textit{restriction}(P\ r_{1}\ldots r_{n}),n\geqslant 1$	$E^{\mathcal{I}}=[\textit{restriction}(P\ r_{1})]^{\mathcal{I}}\cap\ldots\cap[% \textit{restriction}(P\ r_{n})]^{\mathcal{I}}$
Range restr. (concept)	$\textit{restriction}(P\ \textit{range}(C)$	$E^{\mathcal{I}}=\{x:y.\langle x,y\rangle\in P^{\mathcal{I}}\IMPL y\in C^{% \mathcal{I}}\}$
Range restr. (datatype)	$\textit{restriction}(P\ \textit{range}(D))$	$E^{\mathcal{I}}=\{x:v.\langle x,v\rangle\in P^{\mathcal{I}}\IMPL v\in\Delta^{% \mathcal{I}}_{\mathcal{D}}\}$
Min. cardinality	$\textit{restriction}(P\ \textit{minCardinality}(n))$	$E^{\mathcal{I}}=\{x:\|\{y.\langle x,y\rangle\in P^{\mathcal{I}}\}\|\geqslant n\}$
Max. cardinality	$\textit{restriction}(P\ \textit{maxCardinality}(n))$	$E^{\mathcal{I}}=\{x:\|\{y.\langle x,y\rangle\in P^{\mathcal{I}}\}\|\leqslant n\}$
Static value	$\textit{restriction}(P\ \textit{static}(v))$	$E^{\mathcal{I}}=\{x:\Forall{y}.\langle x,y\rangle\in P^{\mathcal{I}}\IMPL y=v\}$
Default value	$\textit{restriction}(P\ \textit{default}(v))$	$E^{\mathcal{I}}=\{x:\Exists{y}.\langle x,y\rangle\in P^{\mathcal{I}}\}$
Aggregation values	$\textit{restriction}(P\ aggregationV:f(W_{1}\ \ldots\ W_{n}))$	$E^{\mathcal{I}}=\{x:\Forall{y}.\langle x,y\rangle\in P^{\mathcal{I}}\IMPL y=f^% {\mathcal{I}}(W_{1}^{\mathcal{I}},\ldots,W_{n}^{\mathcal{I}})\}$
Aggregation set	$\textit{restriction}(P\ aggregationS:f(W_{1}\ \ldots\ W_{n}))$	$E^{\mathcal{I}}=\{x:\Forall{y}.\langle x,y\rangle\in P^{\mathcal{I}}\IMPL y\in f% ^{\mathcal{I}}(W_{1}^{\mathcal{I}},\ldots,W_{n}^{\mathcal{I}})\}$
Property restriction	Expression ( $E$ )	Semantics ( $E^{\mathcal{I}}$ )
Domain restriction	$\textit{domain}(C)$	$E^{\mathcal{I}}=C^{\mathcal{I}}\times\Delta^{\mathcal{I}}$
Domain union	$\textit{domain}(\textit{union}(C_{1},\ldots,C_{n})$	$E^{\mathcal{I}}=C_{1}^{\mathcal{I}}\cup\ldots\cup C_{n}^{\mathcal{I}}\times% \Delta^{\mathcal{I}}$
Range (concept)	$\textit{range}(C)$	$E^{\mathcal{I}}=\Delta^{\mathcal{I}}\times C^{\mathcal{I}}$
Range (datatype)	$\textit{range}(D)$	$E^{\mathcal{I}}=\Delta^{\mathcal{I}}\times D^{\mathcal{D}}$
Range (min-max)	$\textit{range}(D\ d_{\min}\ d_{\max})$	$E^{\mathcal{I}}=\{\langle x,y\rangle:y.\langle x,y\rangle\in\Delta^{\mathcal{I% }}\times\Delta^{\mathcal{I}}_{\mathcal{D}}\IMPL d_{\min}\leqslant y\leqslant d% _{\max}\}$
Min. cardinality	$\textit{minCardinality}(n)$	$E^{\mathcal{I}}=\{\langle x,y\rangle:\|\{y.\langle x,y\rangle\in\Delta^{% \mathcal{I}}\times\Delta^{\mathcal{I}}_{\mathcal{D}}\cup\Delta^{\mathcal{I}}\}% \|\geqslant n\}$
Max. cardinality	$\textit{maxCardinality}(m)$	$E^{\mathcal{I}}=\{\langle x,y\rangle:\|\{y.\langle x,y\rangle\in\Delta^{% \mathcal{I}}\times\Delta^{\mathcal{I}}_{\mathcal{D}}\cup\Delta^{\mathcal{I}}\}% \|\leqslant m\}$
Default value	$\textit{static}(v)$	$E^{\mathcal{I}}=\Delta^{\mathcal{I}}\times\{v\}$
Default value	$\textit{default}(v)$	$E^{\mathcal{I}}=\Delta^{\mathcal{I}}\times\Delta^{\mathcal{I}}_{\mathcal{D}}% \cup\Delta^{\mathcal{I}}$
Aggregation value	$\textit{aggregationV}:f(W_{1},\ldots,W_{n})$	$E^{\mathcal{I}}=\Delta^{\mathcal{I}}\times{v}\text{ such that }v=f^{\mathcal{I% }}(W_{1}^{\mathcal{I}},\ldots,W_{n}^{\mathcal{I}})$
Aggregation set	$\textit{aggregationS}:f(W_{1},\ldots,W_{n})$	$E^{\mathcal{I}}=\Delta^{\mathcal{I}}\times f^{\mathcal{I}}(W_{1}^{\mathcal{I}}% ,\ldots,W_{n}^{\mathcal{I}})$
Axiom	Syntax	Semantics
Concept def.	$\textit{Concept}(C\ \textit{ddesc}_{1}\ldots\textit{ddesc}_{n})$	$C^{\mathcal{I}}\subseteq\textit{ddesc}_{1}^{\mathcal{I}}\cap\ldots\cap\textit{% ddesc}_{n}^{\mathcal{I}}$
Enumeration def.	$\textit{Enumeration}(C\ o_{1}\ldots o_{n})$	$C^{\mathcal{I}}=\{o_{1}^{\mathcal{I}},...,o_{n}^{\mathcal{I}}\}$
Property def.	$\textit{Property}(P\ \textit{pdesc}_{1}\ldots\textit{pdesc}_{n})$	$P^{\mathcal{I}}\subseteq\textit{pdesc}_{1}^{\mathcal{I}}\cap\ldots\cap\textit{% pdesc}_{n}^{\mathcal{I}}$
Functional def.	$\textit{FunctionalProperty}(P\ \textit{pdesc}_{1}\ldots\textit{pdesc}_{n})$	$\begin{multlined}\displaystyle P^{\mathcal{I}}\subseteq\textit{pdesc}_{1}^{% \mathcal{I}}\cap\ldots\cap\textit{pdesc}_{n}^{\mathcal{I}}\text{ and }\|\{y:y.% \langle x,y\rangle\in P^{\mathcal{I}}\}\|\leqslant 1{multlined}\displaystyle P^% {\mathcal{I}}\subseteq\textit{pdesc}_{1}^{\mathcal{I}}\cap\ldots\cap\textit{% pdesc}_{n}^{\mathcal{I}}\text{ and }\|\{y:y.\langle x,y\rangle\in P^{\mathcal{I% }}\}\|\leqslant 1$
Action def.	$\textit{Action}(A\ \textit{domain}(C_{1})\ \ldots\ \textit{domain}(C_{n})$	$A^{\mathcal{I}}\subseteq\textit{desc}_{11}^{\mathcal{I}}\cap\ldots\cap\textit{% desc}_{1n}^{\mathcal{I}}\cap\textit{desc}_{21}^{\mathcal{I}}\cap\ldots\cap% \textit{desc}_{2m}^{\mathcal{I}}$
	$I:\textit{desc}_{11}\ \ldots\ I:\textit{desc}_{1n}\ O:\textit{desc}_{21}\ \ldots$	$\Exists{P_{A}}\text{ such that }P_{A}=\{\langle x,y\rangle:\langle x,y\rangle% \subset A^{\mathcal{I}}\times C_{1}^{\mathcal{I}}\cup\ldots\cup C_{n}^{% \mathcal{I}}\}$
	$O:\textit{desc}_{2m})$
Individual def.	$\textit{Individual}(o\ type(C_{1})\ \ldots\ type(C_{n})$	$o^{\mathcal{I}}\in C_{1}\cap\ldots\cap C_{n}$
	$\textit{value}(P_{1},v_{1})\ldots\textit{value}(P_{m},v_{m})$	$\langle o^{\mathcal{I}},v_{1}^{\mathcal{I}}\rangle\in P_{1}^{\mathcal{I}}\AND% \ldots\AND\langle o^{\mathcal{I}},v_{m}^{\mathcal{I}}\rangle\in P_{m}^{% \mathcal{I}}$
InstanceList def.	$\textit{InstanceList}(o_{1}\ldots o_{n})$	$\Exists O=\{o_{1}^{\mathcal{I}},\ldots,o_{n}^{\mathcal{I}}\}\text{ such that }% O\in\Delta^{\mathcal{I}}\text{ and there is an }$
		$\text{ order relation }\leqslant_{R}\text{ on }O\text{ such that }o_{1}^{% \mathcal{I}}\leqslant_{R}\ldots\leqslant_{R}o_{n}^{\mathcal{I}}$

3.2 The SmartRules sub-language

SmartRules allows expressing if-then production rules where predicates are based on $\mu$ -Concept constructs. A SmartRule rule (or simply, a SmartRule) can be expressed in the following manner:

rule ‘ $\langle rule{-}name\rangle$ ’

conditions

$\quad\langle condition_{1}\rangle$ ;

$\quad\langle condition_{2}\rangle$ ;

…

actions

$\quad\langle action_{1}\rangle$ ;

$\quad\langle action_{2}\rangle$ ;

…

end

Figure 3.

Overview of Micro-Ontology taxonomy, including DUL and SSN concepts (prefixed). The detail box highlights part of the Stimulus-Sensor-Observation pattern in SSN. The dashed association hasValue was added by us in Micro Ontology in order to facilitate the representation of sensor values in observations.

The concepts and actions used to compose smart rules are entities represented in $\mu$ -Concepts language, including property restrictions. Each rule has a priority of activation and a series of conditions described as concepts and properties, also including variables. If those conditions are satisfied, the action is produced. It is also possible to define inputs and output parameters for the actions within the rules themselves.

Figure 2 shows an example of SmartRule for turning off a stove based on the presence or not of a person at home. The condition Person(not (isAtHome)) tests if there is an instance of the concept Person that has not a relation isAtHome defined. The condition Stove(isRunning, ?ID := hasID) checks if there is an instance of Stove that is on, and, at the same time, attributes the ID of that stove to the variable ?ID. SmartRules supports the use of variable attribution (symbol :=) to facilitate the rule construction. If those conjunctions are satisfied, the actions are produced. In this example, an instance of the action _SwitchOff is created and attributed to the variable ?switchOffStove, which is subsequently attributed with the ID of the stove and executed.

It is important to note that, while SmartRules are oriented to produce mainly actions, they can also produce general facts within the model. For instance, SmartRules can aggregate basic sensor measurements into higher-level measurements.

The language includes a series of functions to deal with actions and other entities. In the previous example, the functions createAction and execute were used to instantiate actions and submit them for execution. Other directives are createInstance, to instantiate concepts; insert, to add new instances to the model; update, to update any instance with new information that has changed during the rule execution; and one, to select a single instance in an instance list.

Variables in SmartRules can be used to store any construct (i.e., instance, property value or literal). The language allows the reasoner to automatically infer the type of the variable of the referenced element. The inference process called matching (unification), consists of matching/unifying the corresponding instances in the knowledge base, including the subsumed concepts defined in the premises of the rules.

SmartRules and $\mu$ -Concept together incorporate the notion of reasoning based on Closed World Assumption (CWA), which consider negation as failure. In other words, facts that do not exist in the ontology are considered to be false. This contrasts languages such as OWL, where reasoning is based on open world assumption. The effect of using CWA is to simplify the modeling task, as the designer does not need to construct negation closures around instances. Furthermore, SmartRules and $\mu$ -Concept languages assume that instances of concepts must have unique names, which also simplify modeling. Thus, there is no need to define equality axioms between instances. In turn, as models would tend to be simpler in SmartRules, CWA and unique name assumption should allow for more efficient reasoning, a property that is essential to reactive IoRT systems.

3.3 The Micro Ontology

The proposed framework includes a domain ontology that can be used as a starting point for creating context models for IoRT scenarios. The Micro Ontology includes useful concepts for modeling different aspects of reactive semantic IoRT applications, particularly in indoor scenarios. However, one can use the platform with a different ontology. The top-level of Micro Ontology is mainly constituted of axiomatically simpler versions of concepts in DOLCE Ultra Lite (DUL) ontology10 and the Semantic Sensors Network (SSN) ontology11 [17] (see Fig. 3). It also includes concepts to model indoor and outdoor environments, spatial relations, as well as sensors and observations. Next, we discuss the main concepts that we added to the ontology.

The DUL concept Object describes all physical or virtual objects present in the environment. We defined three subcategories of objects that are relevant to IoRT systems: Active Object, Static Object, and Movable Object. Active objects are physical objects such as sensors (Sensor), actuators (Actuator) or computer devices (Device); as well as virtual objects, such as web services, databases or computer files. Virtual objects can be used to represent virtual sensors that provide contextual information, such as weather parameters. Static Object and Movable Object describe entities necessary for for context interpretation.

Person is a central concept of ontology that allows one to describe the identity and profile of a user. This concept can be specialized to describe user profiles by application type (profile for a social network, profile for a cognitive assistance application, etc.).

Taking inspiration in IEEE 1872–2015 [24, 25], we defined Robot as subconcept of DUL Agent and DUL Designed Artifact. The Robot concept can be specialized to define subclasses of robots, such as a mobile companion robot or an exoskeleton robot.

Action is a DUL concept that describes all context adaptation actions. The distinction between the concept Action in DUL and the action concepts defined with the action construct in $\mu$ -Concept language is in the nature of their instances. Instances of action concepts refer to commands that must be processed by the IoRT platform. On the other hand, instances of the DUL Action are instances of action events produced by the system or by other agents. Their relationship is straightforward. The processing of instances of action concepts by the platform generates action events, which can be represented as instances of DUL Action in the Micro Ontology.

From these high-level concepts, more specific notions can be defined. For example, the concept Building Area Region is the root of a taxonomy of types of regions in home environments. This taxonomy includes subconcepts such as Indoor Space Region to describe the internal spaces of the habitat and Outdoor Space Region to describe external spaces. These two concepts are extended into more specific internal and external environments, such as Room Region, Bathroom Region, Living Room Region, Garden Region, etc. All of them are associated to subconcepts of Place in DUL.

Figure 4.

Overview of the platform architecture. The mechanics of the Reactive Reasoning Core is mostly given by explicit knowledge models composed of ontologies and rules. On the other hand, the mechanics of the Abstraction Layer is hard-coded. The modeling and rule editing tools are not necessary for the system to run, being mostly used during the design phase.

The description of sensors is based on the concept Sensor of SSN, which specializes Active Object. Sensor readings are represented by the concepts Observation and Sensor Output, which is part of the ontology pattern Stimulus-Sensor-Observation (detail in Fig. 3). The concept Observation is an abstract representation of contextual information provided by the sensors. The concept Sensor Output represents the actual output, which is described by instances of Observation Value. In DUL, observations values are seen as quality regions, which are reifications of regions in value spaces. Instances of Observation Value aggregate the actual literal values of the observations and their unit of measure. The Micro Ontology includes a taxonomy of types of observations related to different outputs that can be produced by sensors. An example of scenario is discussed in Section 5 to show how the ontology of sensors and observations can be used to represent indoor contexts.

4. Platform, tools and methodology

In this section, we propose a middleware platform for semantic IoRT systems development. It is composed of two weakly-coupled software layers, which can be replicated and deployed on multiple computers, as shown in Fig. 4. The lower Abstraction Layer coordinates access to MOs used by the system, such as objects and devices in the environment. Its main objective is to hide the heterogeneity of IoRT sources embedded in these MOs, such as sensors and actuators, from higher-level components. The top reasoning layer, called Reactive Reasoning Core (RRC), is responsible for processing information from the Abstraction Layer, reason about it and produce actions. The actions are then converted back to commands for the Abstraction Layer, closing the perceive-plan-act cycle.

4.1 Abstraction Layer

The Abstraction Layer manages the heterogeneity of low-level components, providing a simpler, homogeneous interface for the upper layers. It is a kind of semantic façade that abstracts access to objects been managed in the environment using protocols or data formatting that are too complex. Its implementation is based on a message-oriented middleware with standard messaging format. The Abstraction Layer transforms heterogeneous sensor data into instances of MOs. After the RRC processes the gathered information and produce actions, the Abstraction Layer translates them into commands that are recognized by the real-world actuators. The reasoning layer does not need to run synchronously with the Abstraction Layer.

For a better manageability of the context in IoRT environments, the Abstraction Layer can itself be distributed among different networking domains in order to be physically close to the MOs. For instance, a typical IoRT system can be composed of a RRC located in a virtual machine hosted in the cloud, with the components that form the Abstraction Layer distributed on several physical machines connected through different IoT networks.

4.2 The reactive reasoning core (RRC)

The Reactive Reasoning Core allows inferring implicit observations or the actions that will be performed by entities using explicit contextual knowledge and a set of SmartRules. This module provides a set of functionalities that are necessary to manage the content of the ontology used by the system, its consistency regarding the $\mu$ -Concept ontology and the reasoning operations that are handled by the rules and query modules. RRC also maintains data persistence for recovery in case of any failure. The complete consistent model is written in a database. Its functionality is distributed among three different (sub)modules.

The ontology module loads and manages access to the ontological knowledge base used for reasoning and querying. It is based on the representation framework described in Section 3. This module sets up new $\mu$ -Concept instances, checks constraints and updates the instances according to the messages it receives. It serves as a proxy to the other modules in RRC, taking care of coordinating access and persistence of new knowledge being produced. The ontology ( $\mu$ -Concepts, their relationships, instances, etc.) is stored and loaded from a persistent database. When the ontology is updated or modified, RRC checks the validity of the update before performing the modification regarding a set of predefined constraints.

The consistency checking module is responsible for checking the consistency of the model used, taking into account constraints defined in the ontology. The checking is done for each modification performed in the model to ensure the consistency of the model and its instances regarding the $\mu$ -Concepts and properties definitions.

The reasoning module is responsible for the execution of SmartRules in the ontology module and subsequently updates it with new knowledge. The reasoning module can also handle queries presented to the system. The reasoning process starts by the conversion of SmartRules to a target inference engine. We use Drools [35] inference engine in our implementation. However, any Rete-based production rule system could be employed. As new facts are produced, they are inserted back in the ontology. More importantly, if actions have been produced, they are sent to the Abstraction Layer to be translated to commands for actuators.

Note that it is possible to have a history of all instances and measurements. This can be useful to perform temporal reasoning, for example. This approach ensures that the ontology represents the last state of the environment and its components.

4.3 Framework tools

The proposed framework includes a set of tools to help specify the ontology. The $\mu$ -Concept Modeling Tool is basically similar to other ontology specification tools, such as Protégé [36]. It allows the user to create new $\mu$ -Concept ontologies, create action types and relate them to other concepts in the taxonomy. It is able to import OWL ontologies and translate them to $\mu$ -Concept. The SmartRules Editor [13] is intended to facilitate the editing of context management rules in SmartRules and conversion to target inference engines. The definition of the rules is restricted to the vocabulary provided by an ontology defined in $\mu$ -Concept (e.g. the Micro-Ontology). The editor also allows the user to query the knowledge base, retrieving $\mu$ -Concept instances matching certain criteria.

4.4 A method for framework instantiation

The proposed framework is flexible in how it can be instantiated in the different scenarios. We propose a standard methodology for designers to instantiate it. Its aim is to lower the amount of programming work needed to accomplish the task.

In practice, we figured that system designers would like to use pre-existing OWL ontologies representing some aspects of the domain (e.g., user and contextual info). With that in mind, in order to instantiate the framework, a designer has to:

1.
Check consistency of the OWL file with a description logics reasoner. Also, adds all inferred facts back to the ontology.
2.
Convert the OWL ontology to a $\mu$ -Concept ontology. The OWL ontology is interpreted in its RDFs form directly from RDF triples. The framework provides a tool for this conversion.
3.
Use the $\mu$ -Concept editor to enrich the converted ontology $\mu$ -Concept-specific constructs. More specifically, at this step, the designer must add context-specific knowledge to the model, including: (a) static and default values for properties, particularly to those whose value depends on perceptions processes that can fail; (b) property translations; (c) order relation between instances.
4.
Add instances to the model. Each MO in the environment, such as actuated furniture and devices, should be specified in the system by an instance of the ontology.
5.
Add actions to the model. In this step, the designer must associate actions to MOs that require them. They will serve as basis to the definition of action rules.
6.
Create rules with the SmartRules editing tool. There are two main types of rules. Abstraction rules might be needed to transform low-level data given by sensor into high-level observations to complement the work done by the Abstraction level. Reactive rules make the system react to specific context patterns, producing actions as a result. Additionally, integrity verification rules might be added if needed (e.g. check if temperature of an object is out of range).
7.
Connect low-level components to the Abstraction Layer. Eventually, there might be new types of sensors or actuators available in the IoRT infrastructure of a particular scenario that has no support in the framework. In this case, the designer must implement the support by adding an interface that is capable of communicating based on $\mu$ -Concept constructs.

These steps should produce an instance of the framework ready to be deployed in a specific scenario. Naturally, this methodology could also be incorporated into a broader software engineering methodology.

We expect that the majority of the deployments of the platform will be based on previous deployments in similar contexts. These should require just minor changes to the ontology. For example, in the scenario of house automation, after instantiating and deployment of the platform in a house, adapting it to another house should require less work. In order to achieve that, the two first steps of the previous method should be replaced a step to remove all instances of the previous context, as just instances are expected to change between similar environments. Existing instances can also be adapted, if the new context is similar enough to the previous one. We expect that fewer new concepts, actions and rules would have to be added in a similar context. More importantly, rules do not need to be removed when adapting the platform from one context to another, as they are contextual. As the system is deployed multiple times in examples of a same class of contexts (e.g. similar hotel rooms), both concepts and rules accumulate, effectively covering the range of expected situations in a class of contexts.

Figure 5.
Fall context detection and confirmation through robot and pushing button. The user is equipped with Fall detector Bracelet and the cricket Indoor location beacon.

5. Validation scenarios and experiments

The initial version of the proposed framework developed in the context of the SembySem project was validated through a series of industrial IoT demonstrators [37], such as real-time train localization, tracking, and sharing of cargo context information. It has been also validated for centralized control of multiple building management systems, based on SCADA, in a single coherent hypervision system. The framework has been evolved to deal with the requirements of semantic IoRT system in Ambient Assisted Living (AAL) scenarios, which is the context of experiments described in this section. AAL IoRT systems are designed to help people, often with perceptual and mobility impairments, to have a better quality of life in their homes. In order to demonstrate the effectiveness of the proposed framework in this kind of scenario, we implemented and analyzed an IoRT system dedicated to the monitoring and assistance of an elderly person during his/her daily living activities. Our approach can be used in conjunction with data-driven approaches to monitoring, such as [38]. We implemented three different real use cases12: (a) provide context-dependent reminders to take medicine; (b) detect and act on a fall; and (c) provide context-dependent dietary advice. In this section, we will only detail use cases (a) and (b).

5.1 Modeling environment and infrastructure

The experiments were conducted at our lab (LISSI Lab, UPEC). We adapted part of the building in order to recreate a realistic IoRT environment similar to that of a smart home. The environment includes the following components (Fig. 5):

•
A Kompai companion robot equipped with a variety of sensors (i.e., 2 RGB cameras, one microphone, 6 ultrasound sensors and 16 laser sensors), actuators (i.e., 2 differential and one free wheel) and processing unit with a touchscreen display. This robot provides several high-level services, such as the management of the medical agendas and the medical treatment; it can recognize and synthesize voice, enabling speech interaction, send emails and navigate in unknown environments. All services, sensors and navigation system of robot can be controlled remotely;
•
Display devices, such as smartphones and video screens;
•
A Cricket indoor localization system [39];
•
A switching actuator for remotely switching on or off home appliances and light bulbs;
•
A set of sensors allowing the observation of events related to the context of the elderly person at home.

The sensors used for event detection are:

•
A long-range RFID reader embedded on the robot companion that allows detecting the presence of active tags in a space of 1 to 8 meters;
•
A RFID short range reader carried by the user that allows to detect whether the user is close to objects, which are equipped with passive tags;
•
Magnetic sensors for detecting the opening/closing of doors and windows and users’ presence;
•
Sensors for measuring brightness, moisture and temperature;
•
A bracelet for detecting falls and measuring the pulse of a person.

Figure 6.
IoRT devices present in the environment and represented in the ontology. This model is a specialization of SSN ontology.

Figure 7.
Some of the instances of IoRT devices in the ontology with defined relationships. Dotted lines are instantiation relationships.

These IoRT components are instantiated as MOs and connected to the Abstraction Layer. Every MO is instantiated in the knowledge base by using its corresponding $\mu$ -Concept defined in the proposed Micro-ontology. They are subsumed by concepts in DUL and SSN. In particular, the ontology includes a sensor taxonomy as shown in Fig. 6, together with their respective observation types and some basic properties. Each specific device present in the environment is represented as an instance of a type of that taxonomy (see Fig. 7). For example, the fall sensor is represented in the ontology as instance FALL: #1235 is a Physiological Sensor. Both RFIDREADER: #101 and RFIDREADER: #102 correspond to two different instances of Radio Identification Sensor. The environment itself is also specified, such as house spaces and objects, and their relationships to IoRT devices. These relationships can be used to infer new information.

Furthermore, reacting to context changes requires the modeling of actions. In this simple scenario, we have three types of actions:

Action(_VisualMessage

domain(SpaceRegion)

I:restriction(content minCardinality(1)

range(xsd: String)))

Action(_MoveRobot

domain(Robot)

I:restriction(content range(xsd: String))

I:restriction(x minCardinality(1))

I:restriction(y minCardinality(1)))

Action(_EmergencyAlarm

domain(SmartPhone)

I:restriction(content static("Emergency Alarm!")

I:restriction(humanLocalization

minCardinality(1))

The platform is able to translate instances of these actions into commands to the respective actuators and other devices. The action _VisualMessage is the act of displaying some information (i.e. content) in some display device in a specific location (i.e. SpaceLocation). The platform is responsible for finding and choosing the appropriate display in the environment. The action _MoveRobot makes the target robot to travel to the position $(x,y)$ and show a message in content. The action _EmergencyAction is similar to _VisualMessage, but it has a fixed content and takes into consideration the current location of the user.
5.2 Defining rules

The previous concepts and actions allow us to specify SmartRules that implement the use cases in the AAL scenario. The rules are used by the system to abstract information from lower-level sensor observations into high-level entities, and also to produce actions based on contextual information. In the following, we focus on the description of the rules needed for managing two different use cases. In the first use case, the system must detect when the elderly person is at home and then advise him/her to take medicine. The advice might be given by a display or by a robot. In the second use case, the system must detect that the user fell to the floor and send a robot to investigate. If the user does not respond, the system sends an alarm. In both contexts, there is a base set of rules that abstract information from sensor data.

Beginning with medicine advice, the system needs first to detect the whereabouts of the user inside his/her home in order to deliver the advice. The system is able to sense the user’s location in a given space of the house as well as his/her proximity to display devices. The first rule generates presence observations from location observation captured by the localization sensor, while it updates the user’s location in the house:

rule "PresenceObservation"

conditions

?presence := LocationOutput(

?spacename := one (hasValue),

?producer := isProducedBy);

?spacename (isInstanceOf(SpaceNameValue));

?name := hasQualityValue);

?space := House(?name == instanceInternalUID);

?producer(?person := wearedBy);

actions

?person->locatedAt := ?space;

update(?person);

PresenceObservation ?obspresence :=

createInstance(PresenceObservation);

?obspresence->instanceInternalUID :=

"PRESENCE_OBSERVATION";

insert(?obspresence);

update(?presence);

end

This rule induces a kind of virtual presence sensor, as it derives presence data from location. The first line of the rule captures outputs of the localization sensor. The hasValue property links the observation to quality values in SSN, which describe the actual values produced by the sensors. In this case, it is an instance of Space Name Value, which has a list of location names produced by the location sensor. The property isProducedBy describes the nature of the data supplied by the sensor, such as temperature, brightness and position. The second line within the rule allows linking the instance (i.e. the real spaces, such as room, living room, bath room, etc.) to the concept House defined in the ontology. The variable ?producer associated to the localization sensor allows creating a relationship between ?person and the ?spacename. The ontology is then updated with action operations. This is equivalent to update the beliefs of the robot regarding the environment. In the rule, the observations are captured by the variable ?obspresence.

The next rule generates the necessary user proximity data from RFID sensor readings:

rule "ProximityObservation"

conditions

?ac := ActiveRFIDReaderOutput(

?tagDetected := hasValue,

?producer := isProducedBy);

?tagDetected(?qualityValue := hasQualityValue);

?producer(?installed := installedIn);

?tag := Tag(?qualityValue == hasDataValue,

?person := isReferenceOf);

actions

?person->nearTo := ?installed;

update(?person);

ProximityObservation ?obs :=

createInstance(ProximityObservation);

?obs->instanceInternalUID :=

"PROXIMITY_OBSERVATION";

?obs->observedBy := ?producer;

?obs->observationResult := ?ac;

update(?obs);

end

The concept RFID Reader Output (Fig. 6), allows checking whether the ?person with a passive Tag (i.e. isReferenceOf) is close to objects represented by the variable ?tagDetected. Thus, the observation is then updated.

Based on these basic rules, it is possible to generate reactions to changes in context and user state. If the proximity observation was observed by a display MO that is located close to the user, then the RRC triggers a _VisualMessage action on the MO corresponding to the closest display screen to the user:

rule "HealthAdvice"

conditions

ProximityObservation(?observedby :=

observedBy);

?observedby(?location := installedIn);

Person(nearTo == ?location);

actions

_VisualMessage ?message :=

createAction(?location, _VisualMessage);

?message->content := "You must take your

medicine!";

execute(?message);

end

In the contexts where the user is present, but not close to a display MO, the RRC triggers an action to move the service robot to the user’s location and display the message:

rule "HealthAdviceRobot"

conditions

PresenceObservation(

?observedby := observedBy,

?observation := observationResult);

?observation(?xValue := hasValue, ?yValue := hasValue);

?xValue(isInstanceOf(XValue),

?x := hasQuantityValue);

?yValue(isInstanceOf(YValue),

?y := hasQuantityValue);

?robot := Robot();

actions

_MoveRobot ?moverobot :=

createAction(?robot, _MoveRobot);

?moverobot->content :=

"John! You must take your medicine!";

?moverobot->x := ?x;

?moverobot->y := ?y;

execute(?moverobot);

end

This rule shows how the system can produce robot actions given sensor readings environments. The system takes location information from the presence observation and uses it to produce an action on the robot, namely, moving it to a position $(x,y)$ in its internal map. In this rule, the robot reminds a person to take medicine. This rule can be adapted to give other kinds of medical advices by just updating the content property associated with the _MoveRobot action.

Table 2
Scalability test results at different parts of the framework (explanation in the text)

MO	MO to AL	AL to RRC	RRC – Inst	RRC – Infer	RRC to AL	AL to MO
100	85 ms	90 ms	10 ms	30 ms	225 ms	476 ms
1000	85 ms	140 ms	20 ms	30 ms	300 ms	420 ms
5000	120 ms	180 ms	50 ms	120 ms	400 ms	542 ms
10000	120 ms	220 ms	40 ms	110 ms	530 ms	720 ms
20000	140 ms	220 ms	110 ms	200 ms	530 ms	720 ms

The second use case involves detection of a fall event. The following rule addresses that (Fig. 5). If the fall sensor produces an output and the user has been observed to be present, the rule fires an action on the robot, commanding it to move to the user’s location and ask for information:

rule "CheckPersonState"

conditions

?fall := FallOutput(treated == "false");

PresenceObservation(

?observedby := observedBy,

?observation :=observationResult);

?observation(?xValue := hasValue,

?yValue := hasValue);

?xValue(isInstanceOf(XValue),

?x :=hasQuantityValue);

?yValue(isInstanceOf(YValue),

?y :=hasQuantityValue);

?rob := Robot();

actions

_MoveRobot ?moverobot :=

createAction(?rob, _MoveRobot);

?moverobot->content :=

"John! If you need assistance, push the SOS button";

?moverobot->x := ?x;

?moverobot->y := ?y;

execute(?moverobot);

update(?fall);

end

It is also possible to define rules based on lack of certain inputs. If the user does not press the SOS button on the robot (Fig. 5), then a new action is triggered:

rule "Emergency"

conditions

?fall := FallOutput();

not exists (SOSButtonOutput());

?smart := SmartPhone();

?person := Person(?location := locatedAt);

actions

_EmergencyAlarm ?alarm :=

createAction(?smart, _EmergencyAlarm);

?alarm->content := "Emergency";

?alarm->humanLocalization :=

?location->instanceInternalUID;

execute(?alarm);

end

This rule specifies that an alarm action will be triggered if a fall has been detected and the user did not press the SOS button. The alarm action is executed on the MO smartphone. The same rule can be replicated to work with other devices, such as Internet residential gateways (e.g. set-top boxes), which can potentially alert family members.

5.3 Evaluation and scalability

The feasibility of the proposed framework in terms of response time is evaluated in three use-cases: (a) Remind the user when it is time to take the medication; (b) fall detection and emergency situation confirmation; and (c) Recognition of meal preparation and provision of dietary advices. Both (a) and (b) are described above. The response time includes the processing time taken by the Abstraction Layer (sensors outputs encoding, integrity constraints checking, insertion of instances of concepts in the ontology at RRC, sending action messages to actuators). In use-case (c), the system has to detect user activities while cooking and give dietary advices. The obtained mean response times for the three use-cases are lower than 3 seconds (a: 2.1 s, b: 2.6 s and c: 2.9 s) and are therefore compatible with the requirements of ambient assisted living applications.

We also conducted some synthetic tests to evaluate scalability in regard to MO number. We created a collection with synthetic scenarios having 860 concepts, 150 rules and varying number of MOs (from 100 to 20,000). For each scenario, we run the platform multiple times and measured mean execution time in different parts of the architecture. The tests were carried out on two separate machines. The RCC component run on a Intel Core i5, with 4 GB RAM, equipped with Windows 7 64-bits. The Abstraction Layer run on a 2.60 GHz Pentium (R) dual-core, with 2 Gb of RAM, equipped with Windows XP. All executions are single threaded. We setup the tests such that, for each run, only between 15 and 25 rules fired. The results are depicted in Table 2. Each column represent the processing time in different parts of the architecture:

MO
The number of MOs in standalone distributed threads. We consider sensing MO (MO-S) and Actuation MO (MO-A);
MO to AL
The highest timestamp of the sensor MO data encoding and sending to the Abstraction Layer;
AL to RRC
The highest timestamp of the routing and encoding from the Abstraction Layer to the RRC;
RRC – Inst
The highest timestamp of the instantiation of the MO data as $\mu$ -Concept instances in the ontology (in RRC);
RRC – Infer
The highest timestamp for the firing of all the rules needed for inference of new context instances and actions assertions in the ontology;
RRC to AL
The highest timestamp of the reception of the last action message by the Abstraction Layer and its encoding to low-level message to be routed to the actuator MOs;
AL to MO
The highest timestamp of the reception of the last action raw message by the MO corresponding to the actuator MO.
5.4 Discussion

Considering the use cases and evaluation above, the main advantages of our framework compared to other approaches relies on how SmartRules are built and processed. The semantic IoT platforms reviewed in Section 2 that better relate to ours are the ones that use OWL as an ontology representation language and SWRL as a rule language. Both SmartRules//muConceps and OWL/SWRL are tightly integrated languages, such that rules are defined using the restricted vocabulary of the ontology. Also, both pairs of languages support built-in functions to some extent.

However, in contrast to SWRL, SmartRules includes built-in functions and constructs that give more control of how facts are produced. Both languages allow for the instantiation of individuals. However, SmartRules provides functions for updating existing individuals, effectively allowing for non-monotonic update of the knowledge base. Furthermore, in contrast to OWL, the $\mu$ -Concept language includes special constructs that are specific helpful for IoRT systems, such as aggregation functions. As we mentioned before, these constructs substitute the definition of specific rules for data aggregation, – as it would be the case in SWRL. Default values are also relevant in situation where sensors fail to provide data.

SmartRules employs CWA and unique name assumption, in contrast to OWL and SWRL, which use open-world assumption. The impact of this choice is substantial to IoRT systems. By closing the world, we are able to avoid the need for closure axioms in the modeling, simplifying the definition of the ontology and the rules. The same contribution is given by the unique name assumption, which removes the need for stating (in)equality in the knowledge base, a common problem for OWL. Both choices simplify the modeling and impact reasoning performance, which are essential in reactive systems.

We also provide a platform and a set of tools for supporting designers in the development and deployment of semantic context-aware IoRT systems. This framework is the result of intensive work for the development of industry-compliant IoRT systems, which have been validated in the context of AAL use cases. Based on the implementation and experiments, we believe that the design work was simplified regarding earlier proposals. The complexity of the IoRT environment is hidden for the designer thanks to the Abstraction Layer, which is capable of generating semantic descriptions of contextual information. The experiment shows that response time of and IoRT system deployed on computers with limited resources remains in intervals usually defined by end users of AAL IoRT services.

6. Conclusion

This paper proposes a semantic framework of context aware IoRT systems, focused on the notions of manageable objects and reactive rules based on an ontology about the environment. The knowledge representation part of our proposal consists of SmartRules, a rule representation language, and $\mu$ -Concept, an ontology representation language. In this paper, we describe an abstract syntax and model-theoretic semantics for $\mu$ -Concept, as well as the Micro-Ontology for representation of IoT domains. We also present a platform, tools and methodology to instantiate IoRT systems based on SmartRules. The example scenarios and validation presented shows the suitability of the framework for building semantic IoRT systems and adapting existing systems to new scenarios.

The framework has some limitations. We assume that all IoT objects are known in advance. Consequently, the modeler has to populate the ontology with the set of instances the can be present in the context. Thus, the semantic framework developed in this approach cannot deal with novelty autonomously. Part of the solution lays into endowing the system with learning capabilities (i.e. [40]) and service discovery. Also, there should a better way for bridging the semantic gap between entities description at the pure sensor level and their representation at symbolic level. This is a long-standing problem in AI, and some interesting approaches exist in the direction of cognitive semantics (e.g. [41]). Furthermore, SmartRules and $\mu$ -Concept, in their current form, do not support reasoning with any ignorance model, such as probability or fuzzy logic (even though such information can be added to $\mu$ -Concept models using simple properties). The platform can only deal with some kinds of missing information by using property aggregation. We plan to address some of these issues in future work.

Footnotes

http://www.foaf-project.org/.

https://itea3.org/project/sembysem.html.

https://itea3.org/project/web-of-objects.html.

https://www.w3.org/TR/sparql11-overview/.

https://www.w3.org/TR/owl2-overview/.

https://www.w3.org/TR/owl-semantics/syntax.html.

We ommit list properties and inverse functional properties.

http://www.w3.org/TR/xmlschema-2/.

Elements enclosed in {braces} can be repeated one or more times.

http://ontologydesignpatterns.org/wiki/Ontology:DOLCE+DnS_Ultralite.

It is important to note that SSN extends DUL itself.

Videos of some of the experiments can be found in: http://www.ubiquitous-intelligence.eu/demos.

References

Ray

. Internet of robotic things: Concept, technologies, and challenges. IEEE Access2016; 4: 9489-9500. ISSN 2169-3536, doi: 10.1109/access.2017.2647747.

Chibani

Amirat

Mohammed

Matson

Hagita

Barreto

. Ubiquitous robotics: Recent challenges and future trends. Robotics and Autonomous Systems Nov2013; 61(11): 1162-1172. ISSN 0921-8890, doi: 10.1016/j.robot.2013.04.003.

Research

. The Internet of Robotic Things. ABI Research2014.

Mineraud

Mazhelis

Tarkoma

. A gap analysis of Internet-of-Things platforms. Computer Communications Sep2016; 89-90: 5-16. Internet of Things: Research Challenges and Solutions, ISSN 0140-3664, doi: 10.1016/j.comcom.2016.03.015.

Song

Goldberg

Chong

. Networked Robots. Springer International Publishing, Cham. Siciliano

Khatib

, eds. 2016; 1109-1134. doi: 10.1007/978-3-319-32552-1_44.

Guarino

. Formal ontology and information systems. Proceedings of FOIS1998; 98: 81-97.

Fensel

. Ontologies: A Silver Bullet for Knowledge Management and Electronic Commerce. Springer Berlin Heidelberg, Berlin, Heidelberg2001; 11-18. doi: 10.1007/978-3-662-04396-7_2.

Chen

Finin

Joshi

. The SOUPA Ontology for Pervasive Computing. Ontologies for Agents: Theory and Experiences. Tamma

Cranefield

Finin

Willmott

, eds. Basel: Birkhãuser Basel 2005; 233-258. doi: 10.1007/3-7643-7361-X_10.

Zhang

Wang

. Enabling context-aware smart home with semantic web technologies. International Journal of Human-friendly Welfare Robotic Systems2005; 6(4): 12-20.

10.

Bettini

Brdiczka

Henricksen

Indulska

Nicklas

Ranganathan

, et al. A survey of context modelling and reasoning techniques. Pervasive and Mobile Computing2010; 6(2): 161-180.

11.

Riboni

Bettini

. OWL 2 modeling and reasoning with complex human activities. Pervasive and Mobile Computing Jun2011; 7(3): 379-395. Knowledge-Driven Activity Recognition in Intelligent Environments, ISSN 1574-1192, doi: 10.1016/j.pmcj.2011.02.001.

12.

Yeh

Laun

. Challenges for rule systems on the web. Lecture Notes in Computer Science. Springer Berlin Heidelberg 2009; 4-16. doi: 10.1007/978-3-642-04985-9_4.

13.

Sabri

Chibani

Beck

Zarri

Amirat

Brunner

, et al. An editor for micro-concept rules design. Proceedings of the 3rd International RuleML-2009 Challenge2009.

14.

Sabri

Chibani

Amirat

Zarri

. Semantic reasoning framework to supervise and manage contexts and objects in pervasive computing environments. 2011 IEEE Workshops of International Conference on Advanced Information Networking and Applications, IEEE 2011; 47-52. doi: 10.1109/waina.2011.148.

15.

Brunner

Goudou

Gatellier

Beck

Laporte

. SEMbySEM: A framework for sensors management. Proceedings of the 1st SemSensWeb2009 Workshop on the Semantic Sensor Web CEUR-WS2009; (468): 19.

16.

Pfisterer

Romer

Bimschas

Kleine

Mietz

Truong

, et al. SPITFIRE: Toward a semantic web of things. IEEE Communications Magazine November2011; 49(11): 40-48. ISSN 0163-6804, doi: 10.1109/MCOM.2011.6069708.

17.

Compton

Barnaghi

Bermudez

García-Castro

Corcho

Cox

, et al. The SSN ontology of the W3C semantic sensor network incubator group. Web Semantics: Science, Services and Agents on the World Wide Web Dec2012; 17: 25-32. ISSN 1570-8268, doi: 10.1016/j.websem.2012.05.003.

18.

Henson

Pschorr

Sheth

Thirunarayan

. SemSOS: Semantic sensor observation service. International Symposium on Collaborative Technologies and Systems, IEEE2009; 44-53. doi: 10.1109/cts.2009.5067461.

19.

Botts

Robin

. OpenGIS Sensor Model Language (SensorML) OGC 07-000. Open Geospatical Consortium Inc (OGS)2007.

20.

Lee

Chong

. Short paper: Object collaboration model to create joint knowledge in WoO environment. Internet of Things (WF-IoT), IEEE World Forum on2014; 61-62. doi: 10.1109/WF-IoT.2014.6803119.

21.

Tenorth

Beetz

. KnowRob: A knowledge processing infrastructure for cognition-enabled robots. The International Journal of Robotics Research Apr 2013; 32(5): 566-590. doi: 10.1177/0278364913481635.

22.

Lemaignan

Ros

Mösenlechner

Alami

Beetz

. ORO, a knowledge management platform for cognitive architectures in robotics. IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE2010; 3548-3553. ISSN 2153-0858, doi: 10.1109/iros.2010.5649547.

23.

Matuszek

Cabral

Witbrock

DeOliveira

. An introduction to the syntax and content of Cyc. AAAI Spring Symposium: Formalizing and Compiling Background Knowledge and Its Applications to Knowledge Representation and Question Answering2006; 44-49.

24.

Prestes

Carbonera

Fiorini

Jorge

VAM

Abel

Madhavan

, et al. Towards a core ontology for robotics and automation. Robotics and Autonomous Systems Nov2013; 61(11): 1193-1204. Ubiquitous Robotics, ISSN 0921-8890, doi: 10.1016/j.robot.2013.04.005.

25.

IEEE. 1872–2015 – IEEE Standard Ontologies for Robotics and Automation. Institute of Electrical and Electronics Engineers (IEEE) 2015; 1-60. doi: 10.1109/IEEESTD.2015.7084073.

26.

Chen

Finin

Joshi

Kagal

Perich

Chakraborty

. Intelligent agents meet the semantic Web in smart spaces. IEEE Internet Computing Nov2004; 8(6): 69-79. ISSN 1089-7801, doi: 10.1109/mic.2004.66.

27.

Sun

Jara

. An extensible and active semantic model of information organizing for the Internet of Things. Personal and Ubiquitous Computing Aug2014; 18(8): 1821-1833. ISSN 1617-4917, doi: 10.1007/s00779-014-0786-z.

28.

Gyrard

Bonnet

Boudaoud

. Enrich machine-to-machine data with semantic web technologies for cross-domain applications. IEEE World Forum on Internet of Things (WF-IoT), IEEE2014; 559-564. doi: 10.1109/wf-iot. 2014.6803229

29.

O’Connor

Knublauch

Grosof

Dean

Grosso

, et al. Supporting rule system interoperability on the semantic web with SWRL. The Semantic Web – ISWC. Gil

Motta

Benjamins

Musen

, eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 974-986. doi: 10.1007/11574620_69.

30.

Sorici

Picard

Boissier

Zimmermann

Florea

. CONSERT: Applying semantic web technologies to context modeling in ambient intelligence. Computers & Electrical Engineering May2015; 44: 280-306. ISSN 0045-7906, doi: 10.1016/j.compeleceng.2015.03.012.

31.

Brickley

Guha

. RDF Schema 1.1. W3C Recommendation. World Wide Web Consortium2014.

32.

Baader

Calvanese

McGuinness

Nardi

Patel-Schneider

. The Description Logic Handbook: Theory, Implementation and Applications, 2nd ed. New York, NY, USA: Cambridge University Press, 2010.

33.

Horrocks

Patel-Schneider

. Reducing OWL entailment to description logic satisfiability. Web Semantics: Science, Services and Agents on the World Wide Web Oct2004; 1(4): 345-357. International Semantic Web Conference 2003, ISSN 1570-8268, doi: 10.1016/j.websem.2004.06.003.

34.

Patel-Schneider

Hayes

Horrocks

. OWL Web Ontology Language Semantics and Abstract Syntax. W3C2004.

35.

Browne

. JBoss Drools Business Rules. Packt Publishing, 2009.

36.

Noy

Sintek

Decker

Crubezy

Fergerson

Musen

. Creating Semantic Web contents with Protege-2000. IEEE Intelligent Systems Mar2001; 16(2): 60-71. ISSN 1541-1672, doi: 10.1109/5254.920601.

37.

Gatellier

. SemBySem innovation report: Semantic approach offers effective services management for the Internet of Things. Technical Report, Itea2, 2011.

38.

Maglogiannis

Ioannou

Tsanakas

. Fall detection and activity identification using wearable and hand-held devices. Integrated Computer-Aided Engineering2016; 23(2): 161-172. ISSN 1069-2509, doi: 10.3233/ICA-150509.

39.

Priyantha

Chakraborty

Balakrishnan

. The Cricket location-support system. Proceedings of the 6th Annual International Conference on Mobile Computing and Networking. MobiCom ’00. ACM Press 2000; New York, NY, USA, 32-43.

40.

Chen

Okeyo

Wang

Sterritt

Nugent

. A systematic approach to adaptive activity modeling and discovery in smart homes. 2011 4th International Conference on Biomedical Engineering and Informatics (BMEI)2011; 4: 2192-2196. ISSN 1948-2914, doi: 10.1109/BMEI.2011.6098760.

41.

Fiorini

Gärdenfors

Abel

. Representing part-whole relations in conceptual spaces. Cognitive Processing May2014; 15(2): 127-142. ISSN 1612-4790, doi: 10.1007/s10339-013-0585-x.

An integrated semantic framework for designing context-aware Internet of Robotic Things systems

Abstract

Keywords

1. Introduction

3.1 The μ -Concept sub-language

3.1.1 Syntax and semantics

Table 1 Interpretation of main language expressions

4.1 Abstraction Layer

4.2 The reactive reasoning core (RRC)

4.3 Framework tools

4.4 A method for framework instantiation

5.1 Modeling environment and infrastructure

Table 2 Scalability test results at different parts of the framework (explanation in the text)

6. Conclusion

Footnotes

References

3.1 The $\mu$ -Concept sub-language

Table 1
Interpretation of main language expressions

Table 2
Scalability test results at different parts of the framework (explanation in the text)