Semantics in the Edge: Sensors and actuators in the Web of Linked Data and Things

2.1. Maturity and Coverage of WoT Ontologies

Different standardisation bodies work towards developing data models and ontologies for the Internet and Web of Things [18,26,33,34]: the World Wide Web Consortium (W3C), the Open Geospatial Consortium (OGC), the Internet Engineering Task Force (IETF), the European Telecommunication Standards Institute (ETSI), the Open Connectivity Foundation (OCF), the IPSO Alliance, and the Open Mobile Alliance, among them. Many aspects of the data being generated in the WoT need to be described semantically and the standardisation bodies sometimes adopt conceptually different modelling perspectives. The diagram in Fig. 1 shows these aspects and presents the current state-of-the-art in ontologies available to describe those.

OPEN IN VIEWER

Fig. 1.

Data aspects that require semantics in WoT and existing established ontologies.

Real world setting A central aspect in modeling WoT applications is the description of the real world setting in which the Things/Sensors/Actuators are deployed to observe or act on some features of interest. Many ontologies [4,12,27,32,61] include patterns to describe such settings. The OGC and W3C joint standard on the Semantic Sensor Network ontology (SOSA/SSN) [26], describing networks of sensors and actuators, their capabilities, their features of interest, and their individual observations or actuations, serves as a core or as a source of inspiration to many of these ontologies, ensuring some form of interoperability between them. The ETSI SmartM2M technical committee, develops the Smart Applications REFerence (SAREF) Ontology [12],3

Extensions of SAREF and SSN have been developed for specific domains, such as CASO for the Agriculture [50], EEPSA for Buildings [22], and the SAREF4ABCD series of SAREF extensions4

In addition, SSN has a separate module, called SSN System, to model capabilities and operating/survival ranges of systems/things. Sagar et al. [60] discussed some remaining modeling issues of SSN in this regard and proposed the S3N5

Property qualification or quantification Sensors and actuators are deployed to observe and act on specific properties of features of interest. SOSA/SSN and SAREF both have the modeling of features of interests and their properties in their core, but do not prescribe whether a property can be reused across features of interest, or should belong to a specific feature of interest,6

Being able to describe quantity values and their units is a requirement that is almost ubiquitous in any WoT domain. Different ontologies have been developed to describe units, their relations, and quantities with their values. A recent survey [36] compares and evaluates eight well known ontologies for units of measurements, among which QUDT [30], OM [58] and the Units Ontology [21] are the most widely used. The survey also reports on the Wikidata corpus [68] that at the time of research contained over 4.4k measurement units and 4.1k non-prefixed units. While these ontologies are comprehensive in respect to modelling units of measurements and their relations, a comprehensive model of systems of quantity kinds is still under development, with the QUDT ontology leading the way (as mentioned above). An alternative approach relying on RDF 1.1 Datatypes is proposed by Lefrançois and Zimmermann [47], and allows for more concise representation of quantity values and queries.

Contextualization of value assignment to properties The OWL time ontology [10], the WGS84,7

Device functionality and their APIs A recent and important core ontology to describe thing affordances in terms of properties, actions, and events, is the Thing description [34] developed by the W3C Web of Things working group. However, the Thing description does not model how the enactment of these affordances is to be modeled. This aspect is covered by SOSA/SSN that describe sensors that implement procedures and make observation. Parallel to this, they describe actuators that implement procedures and make actuations. The PEP [44] ontology defines an ontology pattern as a generalization of these two parallel conceptual models. However, the aforementioned models are mainly concerned with the flow of information between devices, but there is little integration of device types (sensors, actuators, gateways) themselves. They need to be considered as more than raw data producers, but the data input and output of these devices and the process to execute them needs to undergo harmonisation through ontological models. This has not yet happened other than in some subdomains.

Events and processes While the PROV Ontology (PROV-O) [41] is an established W3C standard that provides a set of classes, properties, and restrictions that can be used to represent events and activities that happened, i.e. to document a workflow log, an established process ontology that can be used to describe the execution behaviour of complex functions of WoT devices is missing. PSL [24] and m3po [28] were early examples of process ontologies that could theoretically be used for WoT devices, but neither are described in OWL nor have they found use outside of academia. The WiLD ontology [35] proposed as an execution model for the Linked-Data Fu system is the closest to a process model to describe the execution behaviour of WoT devices, but it has yet to be used in actual implementations and it also lacks a mapping to the ontologies mentioned above for describing device functionality and their APIs, in particular the WoT Thing description.

OPEN IN VIEWER

Fig. 2.

Semantics in the Edge: using RDF abstractions for the content of Web resource.

WoT protocols To be part of the WoT, things, sensors, and actuators, need to be exposed on the Internet and reachable using Web protocols. This category focuses on ontologies that bind the thing’s affordances and their enactments to Web protocols. The architecture paradigm for WoT applications is intended to be stateless due to Thing constraints, therefore ontologies such as Hydra [39] are a good fit. More than the classical REST level 2 that is offered by Hydra, the WoT HCTL ontology8

A missing link between the description of affordances and the actual messages that will be sent as commands and received as results, is the description of the data model for these messages. The WoT JSON Schema ontology10

Human in the loop Voice and gesture-controlled interfaces are becoming increasingly popular in the WoT. In particular, in automobiles, smart homes, computer games and Augmented Reality (AR) / Virtual Reality (VR) applications, voice, gestures and sometimes even gaze has become prevalent due to its accessibility to everyone. Designers, producers, and vendors integrating gesture interfaces into their products have also increased in numbers, giving rise to a greater variation of interaction models in utilizing them. However, different modalities that are used to interact with smart environments have not yet been formalized in vocabularies and ontologies, in particular models that formally describe voice commands, gestures and gaze interactions and how they relate to affordances of WoT devices are missing.

Other considerations are of utmost importance for the WoT as it bridges the Web and the real-world where humans need to protect their privacy and integrity. Ontologies to describe access control and security of WoT application will be important for a successful deployment of the semantic WoT. The WoT SEC ontology11

While RDF has proven to be an effective data model for interoperability on the application layer, its verbose serialisation formats (e.g. RDF/XML, NTriples, or Turtle) present a challenge on the presentation layer. Other than some approaches using the HDT [19] serialisation of RDF [29] or other binary representations of RDF [8], there has been little work and even fewer uptake in industry of providing WoT devices that consume and produce RDF.

Consequently, many data formats and data models exist and they compete with each other for adoption in devices in different WoT domains. Standardisation groups rather try to solve this problem by standardising data formats and service APIs [17,33,52]. Some work aim at tackling semantic interoperability despite the heterogeneity of data formats and service API specifications, i.e., across platforms.

The use of semantic Web technologies has been investigated to facilitate semantic interoperability among these platforms [20,49,65]. One challenge is to investigate how semantic interoperability can be obtained on the edge level, i.e. between devices directly, instead of between platforms. The work in Lefrançois [45] is a starting point to investigate how constrained devices that are not natively semantic Web enabled can still be interoperable with one another.

Figure 2(a) illustrates a typical Web service that consumes and outputs resource representations, that are octet streams typed with internet media types according to the Web architecture principles.12

Figure 2(b) illustrates a combination of two WoT services that are seemingly incompatible, but that as an abstraction generate and consume RDF, respectively. The output RDF graph, generated using a certain lifting rule, could then be lowered using the second thing’s lowering rule. In this setting, the condition for the services to be composable is that the content validation rule of the first thing is more specific than the content validation rule of the second thing. However, SHACL shape containment has not been investigated yet.

As devices became powerful enough to offer storage and processing, new architectures appeared, based on edge computing. At the same time, it is now often a requirement that the final user is able to configure the intelligent environment. This poses several research questions: (i) how to embed reasoning in edge devices with various capacities; (ii) how to efficiently distribute reasoning tasks among available heterogeneous devices; and (iii) how to allow user to easily write rules for such devices.

How to embed reasoning in devices with various capacities Edge computing allows manipulating data close to sources, saving bandwidth, lowering latency and reducing communication needs. Slider [9] is an incremental reasoner optimised in memory and processing footprint. RDF4LED [40] is a lightweight RDF engine, which comprises of RDF storage and a SPARQL processor, for small query operations in lightweight edge devices. Devices on the WoT generate and consume highly dynamic data. The recent survey on stream reasoning by Dell’aglio et al. [14] discusses several open issues left to tackle in this field.

How to distribute reasoning tasks among devices Reasoning tasks can be distributed among heterogeneous devices: some are powerful computers, some are edge devices with various constraints. One challenge is to develop computationally-efficient reasoning strategies coping with such heterogeneities [62], and as close to the data sources as possible. One such approach is HyLAR that deploys incremental reasoning tasks on both, the server and the client [64]. Assuming that devices may reason with heterogeneous entailment regimes is also a starting point for a new investigation, for example, if a constrained edge node uses a poorer entailment regime than a more powerful gateway.

End-User Development In the scenario offered by such a complex network, the End-User Development (EUD) vision [48] aims at putting customization mechanisms in the hands of end users. Starting from iCAP [16], an early rule-based system for building context-aware applications, several works demonstrated the effective applicability of EUD techniques for the personalization of the functionality of smart devices and online services in different areas, including mobile environments [11] and smart home [6]. Users can take advantage of visual programming platforms such as IFTTT and Zapier to personalize the joint behaviors of their own connected entities, by adopting the trigger-action programming paradigm, i.e., they allow the definition of IF-THEN rules. Unfortunately, despite its wide adoption, the way it is implemented nowadays presents its own set of open issues: (i) low-level of abstraction (current trigger-action programming platforms adopt highly technology-dependent representation models that work with well-known connected entities, previously associated to a specific user, only. Therefore, defining IF-THEN rules becomes a complex task for non-programmers [31]), (ii) information overload (contemporary trigger-action programming platforms do not provide users with any discovery support [66], and the explosion of new smart devices and online service results in user interfaces with too much information), (iii) and run-time problems (there is the need to provide users with instruments for understanding and debugging their IF-THEN rules, i.e., to avoid possible conflicts [6] and to assess a rules’ correctness [15]).