Foundational patterns benchmark

Abstract

Keywords

1. Introduction

Recently, there has been growing interest in the use of ontology as a fundamental tool for representing domain-specific conceptual models to improve the semantics, accuracy, and relevance of domain users’ query results. Although the amount of data has grown steadily over the past decade, much data shares similar characteristics that can be captured by shared conceptualization. Thus, for describing real-world phenomena and answering user queries in computer science, conceptual modeling has become widely used in the context of cognitive science (Guizzardi, 2005; Mylopoulos, 1992). Conceptual modeling is defined as the activity of formally describing some aspects of the physical and social world around us for the purposes of understanding and communication (Mylopoulos, 1992). While the descriptions that arise from conceptual modeling activities were originally intended to be used by humans, not machines, recent works show how to create computable conceptual models, e.g. Kremen and Necaský (2019), that can be published as linked data and used as a source of truth for data modelling tasks.

While the goal of a conceptual model is to express the meaning of concepts used in a specific domain to discuss the problem and find appropriate relationships between different concepts, the Web Ontology Language (OWL 2) (W3C Consoritum, 2012) is too low-level to capture key modeling decisions like concept rigidity (Student vs. Person). This can result in semantically incorrect representation (yet consistent in OWL 2 sense) ending up in irrelevant query results. For example, events, objects, or their properties are modeled in each domain ontology differently. Foundational ontologies aim to tackle this problem by extending the basic conceptual schema (i.e. the conceptual model that doesn’t have foundational concepts that we can use with any domain) with additional constructs.

In this paper, we show how existing triple-stores perform for data compliant with foundational ontologies. In particular, we focus on existing main-stream triple stores and compare them in terms of efficiency of evaluating typical queries. Typical queries reflect the structure of a top-level ontology and which is recurrently repeating in existing domain and task ontologies. We selected the Unified Foundational Ontology (UFO) (Guizzardi and Wagner, 2005a) among other foundational ontologies because (i) it is one of the most widely used foundational ontologies (Verdonck and Gailly, 2016) in applications, (ii) our experience with using UFO in various domains (Kostov et al., 2017; Křemen et al., 2017) (iii) maturity of UFO for conceptual modeling (Guizzardi, 2005) using OntoUML, a formal UFO-based modeling language based on UML.

Fig. 1.

(a) querying scenario (b) UFO-based querying scenario (c) UFO-based querying of generated data.

Scenarios. We extend our previous work (Ahmad et al., 2018) on a new Resource Description Framework (RDF) (Klyne and Carroll, 2004) indexing approach based on UFO. The different querying scenarios are depicted in Fig. 1. The leftmost diagram shows standard querying scenario – a knowledge base (data) compliant with a domain ontology is queried by users. Typically, large knowledge bases are stored in triple stores and the user queries are formulated in SPARQL (Harris and Seaborne, 2013). The middle diagram shows one of the benefits of using foundational ontologies – it unifies the types of queries to be executed on data. Here, the types of queries are expressed by UFO query patterns, which are then instantiated as user queries in terms of domain ontologies.

The rightmost diagram shows the scenario we use in this paper – we define a benchmark consisting of a set of UFO Query Patterns and of a UFO-based data generator. Based on the created foundational-based domain ontology, the generator provides instances of the ontology, stores the generated data in triple stores, indexes the stored data with a foundational index and evaluates this data on generated foundational patterns. This benchmark can be reused not only for our foundational generated data but also for all datasets compliant with the unified foundational ontology such as INBAS,1

https://www.inbas.cz/

STAMP Hazard and Risk Ontology2

http://onto.fel.cvut.cz/ontologies/stamp-hazard-profile/

and TermIT.3

https://kbss.felk.cvut.cz/termit-demo/

Thus the main research questions this paper contributes to are:

What are recurrent patterns in queries to UFO-compliant ontologies? For example, Who participates in an event? This event can be a marriage, a party, a football game, a business operation, etc.

How to choose a triple store for UFO-based ontology storage? For this, we design a benchmark for evaluating a triple store performance and accuracy w.r.t. the typical queries of UFO-compliant domain ontologies.

How selected state-of-the-art triple stores perform w.r.t. UFO-based ontologies? We answer this question by evaluating the performance of the triple stores using the designed benchmark.

The paper is organized as follows. Section 2 presents related work. Section 3 reviews the necessary background on querying RDF and OWL. In Section 3.2 we introduce the Unified Foundational Ontology. The data generator is explained in Section 4 including the UFO model and UFO indexing technique. Foundational patterns benchmark is presented in Section 5. The evaluation of benchmark query results is given in Section 6, with the description of our use-case. The discussion of the experimental results is present in Section 7. Finally, we conclude the paper in Section 8.

2. Related work

Recently, several benchmarks have been proposed to compare query execution performance for triple stores. These benchmarks focus on different aspects of SPARQL query evaluation. A comprehensive overview of the main features of the benchmarks can be found, for example, in Saleem et al. (2019). The authors describe a benchmark as a combination of a dataset, a set of queries, and a set of performance metrics, and evaluate existing SPARQL benchmarks accordingly. We select here only some of the representative SPARQL benchmarks along with their main features.

FEASIBLE (Saleem et al., 2015) is a cluster-based SPARQL benchmark generator able to synthesize customized benchmarks from the query logs of SPARQL endpoints. The Berlin SPARQL Benchmark (BSBM) (Bizer and Schultz, 2009) is designed to compare the query performance of native RDF stores with the performance of SPARQL-to-SQL rewriters in different architectures. It is applied to several triple stores, such as Sesame (now RDF4J), Virtuoso, and Jena-TDB. The BSBM benchmark is settled in an e-commerce use case where a set of products are offered by different vendors and consumers who have posted reviews about these products on different review sites. The Lehigh University Benchmark (LUBM) (Guo et al., 2005) is a widely used benchmark for comparing the performance, completeness, and soundness of OWL reasoning engines. It is based on a customizable deterministic generator of synthetic OWL data based on the Univ-Bench ontology – an ontology of the university domain that includes universities, their departments, their professors, employees, courses, publications, and their relationships in the OWL language, and provides the features necessary for evaluation purposes. The data generated is random and repeatable and can be scaled to any size, and it uses plain SPARQL queries. The University Ontology Benchmark (UOBM)4

⁴
https://www.cs.ox.ac.uk/isg/tools/UOBMGenerator/

is a more expressive new version of LUBM that includes additional OWL constructs (such as disjoint axioms or negation). The DBpedia SPARQL Benchmark DBPSB (Morsey et al., 2011) is a benchmark for evaluating the performance of triple stores based on non-artificial data and queries, it is settled in the DBLP bibliographic database. It generates benchmark queries from the DBpedia dataset and tests them against 4 different triple stores, namely Virtuoso, Sesame, Jena-TDB and BigOWLIM (now GraphDB). A SPARQL Performance Benchmark (SP2Bench) (Schmidt et al., 2009) is a benchmark for evaluating SPARQL performance. It proposes a methodological approach for testing the performance of SPARQL engines with respect to different operator constellations, RDF access paths, typical RDF constructs, and a variety of possible optimization approaches.

Relation to our approach. The benchmarks mentioned above focus on various features and performance of the SPARQL evaluation scenario (size of the data, expressiveness of the ontology, variety of queries). However, no attention is paid to the shape of data. For example, although most top-level ontologies distinguish between objects and events, these benchmarks are agnostic to this distinction and do not reflect the top-level semantics of data and queries. For example, although a triple store with UFO index might perform poorly on star-shaped graph patterns (i.e., a tree-like pattern with a root node and depth 1) in general, particular graph patterns with specific ontological types of nodes (e.g., object or event) may be more efficient. We investigate this difference in this paper and propose a benchmark of foundational patterns generated using the Unified Foundational Ontology (UFO) (Guizzardi and Wagner, 2005a).

We claim that a top-level ontology gives a common and recurring shape to both data and queries. In our approach, the distinction between ontological categories is made explicit. For example, queries that search for objects involved in an event are covered by a foundational pattern, which can then be optimized using the UFO indexing technique presented in our previous work. Thus, our benchmark can be used to measure what is the efficiency of evaluating a class of queries corresponding to the given foundational pattern. In particular, our benchmark shows how an index based on a OWL representation of UFO5

⁵

http://onto.fel.cvut.cz/ontologies/ufo

implemented in various triple stores can help make SPARQL evaluation over such data more efficient.

3. Background

First, we present a fragment of OWL 2-DL (W3C Consoritum, 2012) in a simplified manner, as a knowledge representation language, along with simple conjunctive queries over this fragment. Next, we give an overview of UFO, one of the foundational ontologies. Then we show an example of RDF representation of the OWL-based UFO fragment and queries.

3.1. OWL 2-DL

An OWL 2-DL ontology $O = {α_{i}}$ , $i \in {1, \dots, N_{O}}$ , where $α_{i}$ axiom is either

a class assertion $A (a)$ , saying that “ $a$ is an instance of $A$ ”, e.g. $Person (Frank)$ .

an object property assertion $P (a, b)$ , saying that “ $a$ is related to $b$ through $P$ ”, e.g. $hasFriend (Frank, John)$ .

a terminological axiom of the form $C_{1} ⊑ C_{2}$ ,

where

A

is an OWL atomic class,

C_{(i)}

are OWL class expressions (discussed later),

a

is an OWL individual and

P

is an OWL object property. Typical OWL class expressions can be constructed from atomic classes as follows

each atomic class $A$ is a class expression,

boolean operators ( $C_{1} ⊓ C_{2}$ ), ( $C_{1} ⊔ C_{2}$ ), or ( $\neg C_{2}$ ), for class intersection/union and complement. For example, $(Person ⊓ Male)$ denotes the concept of men.

existential restriction $(\exists P \cdot C)$ , denoting a class, elements of which are related through P to at least one instance of C. For example, $(\exists hasChild \cdot Man)$ denotes the class of all individuals having at least one son.

universal restriction $(\forall P \cdot C)$ , denoting a class, elements of which are related through P only to instances of C. For example, $(\forall hasChild \cdot Man)$ denotes the class of all individuals having only sons as children, if any,

qualified cardinality restrictions $(⩽ n P \cdot C)$ , or $(⩾ n P \cdot C)$ , $(= n P \cdot C)$ , denoting a class, elements of which are related through $P$ to at least/at most/exactly n individuals through $P$ . For example, $(⩾ 4 hasChild \cdot Man)$ denotes a class, elements of which have at least four sons (and possibly some daughters).

Full OWL 2-DL syntax as well as its formal semantics can be found in W3C Consoritum (2012).

Having an OWL 2-DL ontology $O$ , we define a distinguished conjunctive query as $Q (? x_{1}, \dots, ? x_{n}) = μ_{1}, \dots, μ_{M}$ , where $? x_{i}$ is a variable occurring in some $μ_{i}$ , $μ_{i}$ is an atom of the form $A (y)$ or $P (y, z)$ , where A is an atomic OWL class, P is an OWL object property and y, resp. z is either a variable $? x_{i}$ , or a an OWL named individual. Intuitively, queries match the class/property assertion axioms, possibly extended by inferencing from other axioms. Full query syntax and semantics of distinguished conjunctive queries can be found in Křemen and Kouba (2011). Let us show the notions on an example.

Example 1.
Having an OWL 2-DL ontology $O = {agent ⊑ object, agent (a), performs (a, b)}$ , the query $Q (? x_{1}, ? x_{2}) = object (? x_{1}), performs (? x_{1}, ? x_{2})$ asks for all object and actions they perform. In our case, the query returns a single result binding ${(? x_{1}, ? x_{2}) \to (a, b)}$ , because $a$ is inferred to be an object ( $agent ⊑ object$ ).

3.2. Unified foundational ontology

UFO is a top-level ontology that has been developed based on a number of theories from Formal Ontology, Philosophical Logic, Philosophy of Language, Linguistics and Cognitive Psychology (Guizzardi and Wagner, 2005a).

Based on Description Logic formalization of basic UFO concepts introduced in Benevides et al. (2017) we publish Unified Foundational Ontology with dereferencable identifiers in OWL.6

⁶
http://onto.fel.cvut.cz/ontologies/ufo

Its main concepts fundamental for this work are sketched in the UML class diagram in Fig. 2. UFO describes endurants (i.e., static aspects of a domain) (UFO-A) (Guizzardi, 2005), perdurants/events (UFO-B) (Guizzardi et al., 2013) and social agents (UFO-C) built on top of UFO-A and UFO-B (Guizzardi and Wagner, 2010). UFO splits entities into endurants and perdurants which are both individuals, i.e. entities that cannot have instances (

endurant ⊑ individual

), (

perdurant ⊑ individual

). Endurants are individuals that are wholly present whenever they are present, and they can be any object (e.g. an agent, aircraft) (

object ⊑ endurant

), or its tropes (moments) (e.g. speed, location, colors, etc.) (

trope ⊑ endurant

), that exist as long as an object they inhere in exists (

trope ⊑ (= 1 inheres-in \cdot object)

), and situations (

situation ⊑ endurant

). Perdurants unfold in time accumulating temporal parts. They involve events (

event ⊑ perdurant

) and object snapshots (

object-snapshot ⊑ perdurant

). Events happen in time and cannot undergo non-relational changes which is a change that does not mean a change in a relation, e.g., a death cannot be created/changed/destructed. They can be either atomic or complex (

event ⊑ (atomic-event ⊔ complex-event)

). Complex events are aggregations of at least two disjoint events, and have participants (

event ⊑ (⩾ 1 has-participant \cdot object)

) and complex events have parts (

\exists has-event-part \cdot ⊤ ⊑ complex-event

) (Guizzardi et al., 2013). An event occurs in a certain situation at a certain point in time, and transforms it to another situation, they may change reality by changing the state of affairs from one (pre-state) situation to a (post-state) situation (Guizzardi et al., 2008). object-snapshot is an immutable state description of an object within a situation. situation is a snapshot of object snapshots valid in the given temporal range (Guizzardi and Wagner, 2011).

Moreover, UFO defines Dispositions which are Intrinsic Tropes (moments) ( $intrinsic-trope ⊑ Trope$ ), i.e. existentially dependent entities that are realizable through the occurrence of an event ( $disposition ⊑ trope$ ). This occurrence brings about a situation (Guizzardi and Wagner, 2014). In other words, UFO considers dispositions as properties that are only manifested in particular situations or the occurrence of certain triggering events, and that can also fail to be manifested ( $disposition ⊑ (= 1 is-manifested-by \cdot event)$ ). Dispositions inhere in particular objects ( $disposition ⊑ (= 1 inheres-in \cdot object)$ ). For example, a security flaw in an information system is manifested by the event of stealing sensitive data that brings about unsafe situation.

Additionally, UFO introduces the notion of agents ( $agent ⊑ endurant$ ), i.e. proactive objects that have intentions, the propositional content of intention is a goal. Intentions cause the agent to perform actions ( $\exists performs \cdot ⊤ ⊑ object$ ) (2005, b). Finally, UFO also defines services (Griffo et al., 2017), and powertypes, i.e. universal types whose instances are individuals in the subject domain (Carvalho et al., 2017; Guizzardi et al., 2015).

Representation language: UFO-A is expressed in a quantified modal logic (QML) that allows the expression of the alethic modalities of truth (viz., necessity and contingency), and UFO-B is defined in first-order logic (FOL) with the Method of Temporal Arguments (MTA) (Vila and Reichgelt, 1996). But Benevides et al. (2019) defines a method for rewriting UFO-A in FOL, with no loss of content, and consistently with a revisited UFO-B. Also, to represent UFO using Description Logic (DL), authors in Benevides et al. (2017) proposed a number of alternative translations from UFO-B’s original axiomatization in first-order logic to the description logic SROIQ, which is the formal underpinning of OWL 2 DL. UFO is used in domains such Geology, Biodiversity Management, Petroleum Reservoir Modeling, Disaster Management, Datawarehousing, Telecommunications, Petroleum and Gas, Logistics, among many others (Guizzardi et al., 2015).

Fig. 2.

Main concepts of UFO.

3.3. RDF representation of UFO models

To use a wide-spread technology for UFO index representation, we will consider RDF triple stores. Indexing techniques over RDF are discussed in Section 2. At this point, we show how to represent common OWL axioms, representing an OWL ontology in RDF and a distinguished conjunctive query over the ontology as basic graph patterns of SPARQL (Harris and Seaborne, 2013), a query language for RDF.

Consider a triple pattern, an ordered tuple $t^{v} = (s^{v} p^{v} o^{v})$ , where $s^{v}$ is its subject term, $p^{v}$ is its predicate term and $o^{v}$ is its object term. Each subject is either a variable V, or a constant7

⁷
For the purpose of this paper, we consider constant to be IRIs only.

C

Having an OWL ontology $O$ , its RDF serialization is given directly by OWL specification (W3C Consoritum, 2012). For distinguished conjunctive queries, we translate each atom of the form $A (y)$ into an RDF triple pattern $(y rdf:type A)$ 8

⁸

$rdf:type$ is a special predicate of RDF denoting instantiation.

and each atom of the form

P (y, z)

into an RDF triple pattern

(y P z)

. Note that all constants (

A

P

and possibly y, z) are represented by the corresponding IRIs.

Example:Having an OWL 2-DL ontology $O$ from example 1, its RDF serialization would be9

⁹

We use the prefix “ufo:” to denote the namespace <http://onto.fel.cvut.cz/ontologies/ufo/>, thus a translation of agent into its RDF representation would be an IRI <http://onto.fel.cvut.cz/ontologies/ufo/agent>.

and the SPARQL representation of the query Q would be a SPARQL basic graph pattern

4. Data generator

UFO-based Data Generator (UDG)10

¹⁰
https://github.com/ahmadjana/ufomodel: This is the GitHub link for the source code.

is a generator for RDF triples. It generates data based on a foundational ontology model, the generated data are stored in a triple store and indexed using the UFO index (Ahmad et al., 2018), we can access the data using JOPA.11

¹¹

https://github.com/kbss-cvut/jopa

First, we describe how the data are generated. Then we present a validator of compliance of data to the model. After that, we review the UFO indexing technique.

Fig. 3.

UFO model entities.

4.1. Generating data

Once the model based on UFO is designed, the task is to generate the data. For this we use JOPA, a persistence library for programmatic access to OWL ontologies (Ledvinka and Kremen, 2015). It is used here to create instances of the model entities, i.e., to generate an object graph and then persist it in a repository.

The structure of the generated data is depicted in Fig. 3. A predefined amount of Persons, Actions and Tropes (or moments) is generated as follows.

generate and persist Persons (OWL instances) together with their tropes;

generate and persist an initial set of Actions (OWL instance).

For each action generate a random Person (OWL instance) that participated in this event and persist them; also generate and persist parts of the actions. These parts have the form of a balanced binary tree.

4.2. Validator

The previous sections have shown how to generate the synthetic UFO-compliant RDF data. In order to use our benchmark for existing real-world datasets, we provide a validator based on SHACL (W3C, 2017) rules. The validator checks for cardinality and domain/range constraints of RDF data on the input to determine if it is usable for the benchmark queries presented in Section 5.

For example, the RDF graph in Listing 1 will not pass validation because it lacks a inheres-in link to the respective aircraft instance, but the RDF graph in Listing 2 will pass. The validator can be found at https://github.com/kbss-cvut/ufo-validator, together with example usage.

Listing 1.

RDF graph that not pass the validation

Listing 2.

RDF graph that passes the validation

4.3. UFO index

Once having the generated data in the repository, the task now is to index the generated data using the UFO-index script. In Ahmad et al. (2018), we have presented our novel approach to improve the efficiency of SPARQL12

¹²
Common SPARQL prefixes are
rdfs: to denote http://www.w3.org/2000/01/rdf-schema#,

rdf: to denote http://www.w3.org/1999/02/22-rdf-syntax-ns,

ufo: to denote http://onto.fel.cvut.cz/ontologies/ufo/,

foaf to denote http://xmlns.com/foaf/0.1/,

doc to denote http://onto.fel.cvut.cz/ontologies/documentation/,

time to denote http://onto.fel.cvut.cz/ontologies/time/ and

aviation-safety: to denote http://onto.fel.cvut.cz/ontologies/aviation-safety/.

queries by using UFO-based indexing techniques. Note that we use our UFO-index not only for generated data, but also for all UFO grounded datasets. We have created UFO-based physical design index tables that store RDF data according to the main concepts of UFO, Perdurant and Endurant. As follows:

UFO Triple Tables that store triples physically into two tables instead of one triple table as in general design (Faye et al., 2012; Abadi et al., 2007); one for Perdurant category and the other for Endurant.

UFO Property Table that builds a UFO property table for endurants and another table for perdurants, that will reduce Null values in each property table (Faye et al., 2012; Abadi et al., 2007), but we will still have them.

UFO Vertical Partitioning that applies vertical partitioning approach where each triple table includes n-two column tables where n is the number of unique properties in the data. In each of these tables, the first column contains the subjects that match the property, and the second column contains the object values for those subjects (Abadi et al., 2007,2009; Harris and Shadbolt, 2005).

For this version, we use UFO triple table technique to index the generated and existing data. Tables 1 and 2 explain how this technique works by dividing one triple table into two UFO-based categories tables. In this technique, UFO is divided mainly into two categories, Perdurant (event) and Endurant (object) (

(Event ⊔ Endurant) ⊑ ⊤

). Applying the triple table technique in UFO will store all instances of these two categories (named graphs) in one table and that leads to slowdown in query execution and non-sufficient usage of the conceptualization of UFO. Thus, our proposal in this approach is to divide physically into two tables; one for each category (Perdurant and Endurant). For example, to answer the query: what objects participate in a specific event (e.g., Action-i?)

(Action ⊑ Event) \to (Action ⊑ (= 1 hasParticipant \cdot Object))

. In Triple Table, instead of searching in the whole triple table, the query will be executed in the Perdurant table by using a perdurant named graph. In Property Table: the system will search for (has-participant) property in perdurant property table. By using UFO-based Vertical Partitioning, each table is built regarding to unique UFO-predicate, so that, the search process will look for has-participant UFO-predicate.

Table 1

Perdurant table, depicted from Ahmad et al. (2018)

Subject	Predicate	Object
Event-i	has-participant	Agent-i
Process-i	is-event-part-of	Event-i
Action-i	is-performed-by	Agent-i

Table 2

Endurant table, depicted from Ahmad et al. (2018)

Subject	Predicate	Object
Person-i	is-participant-of	Event-i
Agent-i	performs	Action-i

5. Foundational patterns benchmark

After indexing the data, the task now is to create a benchmark of queries that interest the user, i.e., queries that correspond to people’s thoughts and languages. Users are interested in asking queries about physical objects (e.g., man, woman, car, animal, etc.), properties (e.g., weight, height, color, etc.) and events (e.g., accident, party, flight, war, sale, etc.), i.e., immaterial entities that exist only in the mind of the user or a community of users of a language (Guizzardi and Wagner, 2010). For example, people look for people who perform or participate in a particular action, the characteristics or dispositions of those people, etc.

The meanings of a variety of words such as: red, John, Jana, marriage, accident, ball, process, participate, happen, party, hot, warm, play, situation, tasks reflect the essential differences between things that happen and who performs those things, i.e., the distinction between behavioral elements and structural elements. UFO distinguishes between these two categories, where behavioral elements (events) and structural elements (objects). The question word (“how” versus “what”) is often invoked to verify the different nature of these elements (Guizzardi et al., 2016). Moreover, UFO-B proposes a discrete linear ordering of time points in order to answer question word (“when”) (Benevides et al., 2019).

Thus, for a more comprehensive representation of an ontological domain, it is important to focus on the representation of endurants (e.g., objects, their parts, their properties, etc.) and perdurants (e.g., events, their parts, etc.). This representation is exactly what we consider when creating our benchmark.

The benchmark can be used in several ways. First, to test the suitability of a novel triple store towards real-world UFO-based queries. Second, to identify usability of novel UFO-based queries in existing triple stores. Both use cases help to ensure that the UFO-backed data can be reasonably exploited in real-world applications.

How this benchmark is created. Conceptually, we have created a benchmark of all possible foundational queries, i.e., patterns based on UFO concepts and entities that could be created between Perdurant–Endurant, Perdurant–Perdurant or Endurant–Endurant, i.e., foundational patterns between structural (objects, tropes, agents, situations, etc.) and dynamic aspects (events, actions, etc.) of reality, so that it must be able to characterize ontological aspects of endurants, perdurants, as well as their interplay. Table 3 shows examples of foundational patterns of the generated benchmark.13

¹³
See https://github.com/ahmadjana/ufomodel, cit. November 21, 2022.

Technically, we created these queries automatically by running SPARQL queries over UFO-based indexed triple tables (Ahmad et al., 2018), Each query selects all relations that have Perdurant or Endurant as domain or range and vice versa. Query 3, selects all relations from Endurant table that have object as a domain, such as the participation relation (ufo:is-participant-of) between events and their participants. So, the user can ask questions such as, Who participated in the Joker film? Or, inheritance relation that expresses the properties or moments that inhere in objects, for example, What is the color of Barcelona’s team jerseys?

Table 3

Foundational query patterns and their formal representations

	Patterns	Pattern formalization
$P_{1}$	What are the tropes (properties) of an object?	$(? o, ? p) \to ufo:has-trope (? o, ? p)$
$P_{2}$	What are the objects that participate in an event?	$(? o, ? e) \to ufo:is-participant-of (? o, ? e)$
$P_{3}$	What are the parts of an object?	$(? o 1, ? o 2) \to ufo:has-object-part (? o 1, ? o 2)$
$P_{4}$	What are the parts of events?	$(? e 1, ? e 2) \to ufo:has-event-part (? e 1, ? e 2)$
$P_{5}$	What are the factors that cause an event?	$(? e, ? f) \to ufo:is-manifestation-of (? e, ? f)$
$P_{6}$	What are the situations with events that changed them?	$(? e, ? s) \to ufo:has-pre-situation (? e, ? s)$
$P_{7}$	What is the resulting situation of an event?	$(? e, ? s) \to ufo:has-post-situation (? e, ? s)$
$P_{8}$	What actions that agents perform?	$(? a, ? e) \to ufo:performs (? a, ? e)$
$P_{9}$	What is the direct cause of an event?	$(? e 1, ? e 2) \to ufo:directly-causes (? e 1, ? e 2)$
$P_{1} 0$	when an event started?	$(? e, ? t) \to ufo:has-begin-point (? e, ? t)$
$P_{1} 1$	when an event finished?	$(? e, ? t) \to ufo:has-end-point (? e, ? t)$
$P_{1} 2$	What is the situation that triggers an event?	$(? s, ? e) \to ufo:triggers (? s, ? e)$
$P_{1} 3$	What is the situation an event bingsAbout?	$(? e, ? s) \to ufo:brings-about (? e, ? s)$
$P_{1} 4$	How do situations activate objects’ dispositions?	$(? s, ? d) \to ufo:activates (? s, ? d)$
$P_{1} 5$	What are the system’s parts?	$(? o, ? s) \to ufo:is-part-of (? o, ? s)$

Listing 3.

SPARQL query for endurant

Query 4 retrieves all relations that have event as a domain, i,e, the dynamic aspects of reality. Then, the user can have answers to questions such as, when did the second world war start? What are the parts of Jana’s wedding party? etc.

However, the generic generated patterns in Table 3 are simple and represent only single triple patterns generated based on UFO predicates. We therefore generate more complex patterns with multiple triples. We selected these patterns from our previous experience with UFO-based systems such as INBAS,14

¹⁴

https://www.inbas.cz/

STAMP Hazard and Risk Ontology,15

¹⁵

http://onto.fel.cvut.cz/ontologies/stamp-hazard-profile/

and TermIT.16

¹⁶

https://kbss.felk.cvut.cz/termit-demo/

These systems are information management systems developed based on Semantic Web technologies. They use UFO as a foundational ontology modeling templates of queries in which their users are interested. For example, INBAS 17

¹⁷

https://www.inbas.cz/

is a system for aviation safety data management, it concerns all aviation events (accidents and incidents) that happened in aviation domain, how they occurred (factors and situations), and who perform the actions in these domain (agents), etc. STAMP Hazard and Risk Ontology18

¹⁸

http://onto.fel.cvut.cz/ontologies/stamp-hazard-profile/, cit. November 21, 2022.

is a UFO-based ontology that helps in analyzing accidents and modeling control loop failures through the STAMP safety model. The main competency questions asked by domain users are: What accidents or failures happened (events), what are the hazards (dispositions) of systems (objects), etc. The extended patterns can be found in Table 4.

Listing 4.

SPARQL query for perdurant

Table 4

Extended foundational query patterns and their formal representations

	Patterns	Pattern formalization
$P_{1}^{'}$	What are the parts of a given object with its properties and in which event participates (meets P1, P2 and p3)?	$(? p, ? e, ? o) \leftarrow ufo:has-trope (? p, o 1)$ , $ufo:is-participant-of (o 1, ? e)$ , $ufo:has-object-part (o 1, ? o)$
$P_{2}^{'}$	What are the actions that a given agent performs and what are the dispositions of this agent (meets P5,P8)?	$(? a, ? d) \leftarrow ufo:performs (a 1, ? a)$ , $ufo:is-manifestation-of (? a, ? d)$
$P_{3}^{'}$	What are factors (i.e., dispositions of objects) of an event and how these factors are activated (meets P5 and P14)?	$(? f, ? s) \leftarrow ufo:is-manifestation-of (? f, e 1)$ $ufo:activates (? s, ? f)$
$P_{4}^{'}$	What are the pre and post situations of a given event (meets P6 and P7)?	$(? s 1, ? s 2) \leftarrow ufo:pre-state (e 1, ? s 1)$ , $ufo:post-state (e 1, ? s 2)$
$P_{5}^{'}$	What are the tropes of a given object o1 (meets P1)?	$(? p) \leftarrow$ ufo:has-trope(o1,?p)
$P_{6}^{'}$	Who are participants in a given event e1 (meets P2)?	$(? o) \leftarrow ufo:is-participant-of (? o, e 1)$
$P_{7}^{'}$	When a given event has been started, what did directly cause it and what are the results of this event e1 (meets P2)?	$(? t, ? e, ? s) \leftarrow ufo:has-begin-point (e 1, ? t), ufo:directly-causes (e 1, ? e), ufo:brings-about (e 1, ? s)$
$P_{8}^{'}$	What are the parts of a given action (event) a1 (meets P4)?	$(? p) \leftarrow ufo:has-event-part (a 1, ? p)$
$P_{9}^{'}$	What are the agents of the a given domain or system d1? (meets P15)?	$(? a) \leftarrow ufo:is-part-of (? a, d 1)$

5.1. Description of the benchmark patterns

In this section, we describe the created foundational patterns. As we mentioned in Section 3.2, UFO mainly distinguishes between events and objects. Thus, the foundational benchmark consists of all the patterns between UFO categories, i.e., the interplay between endurants and the dynamic aspects of reality (e.g., events, processes, causation, dispositions, situations, moments). Given the objective of characterizing this interplay between endurants and perdurants, these two ontologies are meant to form an integral whole. Thus, let us discuss some examples or queries of benchmark patterns from Table 3.

Who participates in an event?

Events are ontologically dependent entities in the sense that they existentially depend on their participants (things and people participate and play certain tasks) in order to exist. (event ⊑ (⩾ 1 has-participant · object)). E.g., what are the objects who participate in the department meeting? Who attends the Christmas party?

What are the object’s parts?

Endurants are entities that, whenever they exist, they exist with all their parts, while maintaining their identity, i.e., we can refer to Jana’s arm, leg and head as the same entity (object ⊑ (⩾ 1 is-object-part-of · object)), e.g., What are the parts of Jana’s body? What are the parts of the car?

What are the event’s parts?

This pattern describes how events relate to their parts. According to UFO, every complex event consists of parts (complex-event ⊑ (⩾ 2 has-event-part · event). E,g., What are all temporal precedents involved in an event?

How is an event triggered?

An event occurs in a certain situation at a certain point in time, and transforms it into another situation, they may change reality by changing the state of affairs from one (pre-state) situation ufo:has-pre-situation to a (post-state) situation ufo:has-post-situation (Guizzardi et al., 2008). An event ufo:brings-about exactly one situation $(event(e) \to \exists! s (brings-about(e,s)))$ , which holds in all endpoints of the event. Also, an event is triggered by exactly one situation $(triggers(s, e) \to situation(s) event(e))$ , which holds in all beginpoints of the event, e.g. The event car’s Accident brings-about the Situation that driverIsinjured, which triggers the event ambulance’sCall.

What does a specific situation activate?

A situation triggers an event if and only if (iff) there is a disposition (e.g skills, abilities, disabilities, weak points, etc.) that is activated by the situation ( $\exists activates \cdot ⊤ ⊑ (situation)$ ) and is manifested by an event, e.g., having a written exam (situation) activates the ability of writing (disposition) of the student to write (event).

What are the factors of a given event?

In UFO, events existentially depend on the objects that participate in them and an event is a manifestation of a disposition of an object, then an event occurs due to the dispositions of its participants ( $\exists is-manifested-by \cdot ⊤ ⊑ (disposition)$ ) (Guizzardi and Wagner, 2014). When manifested, they are manifested through the occurrence of resulting events and state changes (is-manifested-by), e,g., what are the factors of a particular accident?

What does an agent perform?

Agent has its own beliefs, intentions, and goals that are sets of intended states of affairs of an agent. He performs actions to achieve their goals performs, e.g., a doctor performs surgery operations in order to satisfy his intentions in saving peoples’ lives.

Table 5 contains the SPARQL representations of the benchmark queries.

Table 5
SPARQL representations of the foundational queries

Foundational Queries SPARQL Queries

What are the object’s parts with its tropes and in which event participates? ( $P_{1}^{'}$ )

What are the event’s parts ( $P_{4}$ )?

How an event is triggered ( $P_{1} 1$ )?

What do situations activate ( $P_{1} 4$ )?

What are the factors causing an event ( $P_{5}$ )?

What does an agent perform ( $P_{8}$ )?

What is the situation that changed an event( $P_{6}$ ) ?

When did an event start and end, what directly causes this event and what are the results of it ( $P_{1} 0$ ), ( $P_{1} 1$ ) ( $P_{7}^{'}$ )?

Foundational Queries	SPARQL Queries
What are the object’s parts with its tropes and in which event participates? ( $P_{1}^{'}$ )
What are the event’s parts ( $P_{4}$ )?
How an event is triggered ( $P_{1} 1$ )?
What do situations activate ( $P_{1} 4$ )?
What are the factors causing an event ( $P_{5}$ )?
What does an agent perform ( $P_{8}$ )?
What is the situation that changed an event( $P_{6}$ ) ?
When did an event start and end, what directly causes this event and what are the results of it ( $P_{1} 0$ ), ( $P_{1} 1$ ) ( $P_{7}^{'}$ )?

6. Benchmark experiment

For evaluation, we tested the foundational patterns in two different use cases, generated data using UDG and existing real world data by instantiating our foundational patterns, i.e., we instantiate the general patterns in real data (individuals), the instance components structure (Subject, predicate, and objects) meets the pattern structures. The comparison is done in different triple stores. We run the Foundational SPARQL Benchmark against four popular RDF stores (Sesame (or RDF4J19

¹⁹
http://rdf4j.org/

), Fuseki Jena with JenaTDB,20

²⁰

https://jena.apache.org/documentation/fuseki2/

GraphDB,21

²¹

http://graphdb.ontotext.com/

Blazegraph22

²²

https://blazegraph.com/

and Virtuoso.23

²³

https://virtuoso.openlinksw.com/

Sesame (or RDF4J):

Version 2.5.2+0dedb9c with Tomcat Version 8.0.48, Operating System Windows 10 10.0 (amd64), Java Runtime Oracle Corporation Java HotSpot(TM) 64-Bit Server VM (1.8.0-151). It is physically designed bases on B-Tree indexing triple tables with context. It allows the user to choose between three storage engines (in-memory, native, DBMS-backend).

Fuseki Jena:

Version 3.13.1 with Tomcat Version 8.0.48, Operating System Windows10 10.0 (amd64). It provides the SPARQL 1.1 protocols for query and update as well as the SPARQL Graph Store protocol. Fuseki is tightly integrated with TDB to provide a robust, transactional persistent storage layer, and incorporates Jena text query. It can be used to provide the protocol engine for other RDF query and storage systems.

GraphDB free:

Version 9.1 with Tomcat Version 8.0.48, Operating System Windows10 10.0 (amd64). It is the free standalone edition of GraphDB. It is implemented in Java and packaged as a Storage and Inference Layer (SAIL) for the RDF4J RDF framework. GraphDB Free is a native RDF rule-entailment and storage engine. The supported semantics can be configured through rule-set definition and selection. Included are rule-sets for OWL-Horst, unconstrained RDFS with OWL Lite and the OWL2 profiles RL and QL. Custom rule-sets allow tuning for optimal performance and expressiveness.

Virtuoso:

Open-Source Edition v7.02 with Tomcat Version 8.0.48, Operating System Windows10 10.0 (amd64).

Blazegraph (former Bigdata):

Version 2.1.5, open-source and commercial license), Operating System Windows10 10.0 (amd64) with Java version 8. It is an ultra-scalable, high-performance graph database with support for the RDF/SPARQL APIs.

The experiment was conducted on a Lenovo, Intel^® Core™i5-7200U CPU @2.5 GHz 2.71 GHz processor, installed memory is 8.00 GB and 64-bit operating system. The average execution time results and standard deviation of pattern instances are specified, where the given results are averages from executing each query ten times against the different triple stores.

We run the queries against different datasets to compare their execution time (performance), number of results and correctness w.r.t. each triple store (they implement different level of reasoning for OWL ontologies). To check whether the results are correct, the domain expert checks that they do not contain wrong result items and that they are complete w.r.t the input data. This check has been done for real data only. Each query will be executed 10 times either on the Perdurant table (named graph) or on the Endurant table after indexing the data by running UFO indexing script on a triple’s repository. As we proposed in Ahmad et al. (2018), this script automatically group all Perdurant statements together through a single group identifier (Named Graph), i.e., in one Perdurant table. And all Endurant triples in another Endurant table.

6.1. SPARQL features selection

To use the foundational benchmark on different datasets by running multiple queries against triple stores, we select a number of frequently executed queries based on domain experts in real data and on our previous experience of developing UFO based systems. We selected queries that cover most SPARQL features that allow us to assess the performance of foundational queries with SPARQL features. Note, all the executed queries are instances of our foundational patterns. The SPARQL features we consider are:

the overall number of triple patterns

SPARQL pattern constructors (UNION or OPTIONAL)

the solution sequences and modifiers (DISTINCT)

filter conditions and operators (FILTER, LANG, REGEX and STR)

aggregates such as (COUNT, HAVING and GROUP BY).

We used the methodology proposed in Saleem et al. (2019) to analyse the characteristics of the proposed benchmark. The diversity score of our benchmark queries is approx 1.6, which is rather a low value, similarly to synthetic benchmarks presented in the comparison. This is not surprising as our benchmark focuses on studying evaluation of rather simple foundational queries.

6.2. Generated data experiments

We instantiated the foundational patterns w.r.t. the generated data benchmark and w.r.t. SPARQL features; we tested the instances of UFO pattern for different generated data-set sizes (200000, 500000, million and 10 million triples). We create instances of foundational patterns, i.e., the individuals in triple stores. The instances meet the UFO relations.

Following are samples of the foundational patterns instances with their SPARQL queries.

Q1: Select all agents with their properties(first name, last name, etc.) instance of $P_{9}^{'}$ , $P_{1}$ .

Q2: Select participants of all events.

(Instance of P2)

Q3: return all events with their parts, properties (e.g., start and end time) and their participants. Instance of $P_{8}^{'}$ , $P_{5}^{'}$ , $P_{6}^{'}$

Q4: Select all actions with their start and end times that have (Person-444525854) as a participant with all properties of this person instance of $P_{5}^{'}$ and $P_{6}^{'}$ .

Fig. 4.

Mean value and standard deviation of execution query time for generated data.

Figure 4 shows the box plot of the results of running execution time of the instantiated queries on the different triple stores. Figure 5 shows how the query (Q1) execution time is affected by different dataset sizes on the selected triple stores. Moreover, we run query (Q1) with and without UFO. The results in Fig. 6 shows how the UFO index improves the performance by around 50%.

Fig. 5.

Mean value ϕ of execution query time for Q1 on different dataset size.

Fig. 6.

Mean value ϕ of execution query time for Q1 with and without UFO.

6.3. Real world datasets experiments: Aviation safety dataset

The ontology that we used to evaluate the benchmark is the Aviation Safety ontology. We designed the Aviation Safety Ontology24

²⁴
https://www.inbas.cz/aviation-safety-ontology

for describing safety issues in aviation organizations and to increase the awareness of analytical methods and tools in the aviation community for safety analysis in aviation domain (Kostov et al., 2017). Our strategy is to analyze safety events that lead to incidents or accidents and explain factors, that contribute to these safety events. Thus, Aviation Safety Ontology consists of the common aviation domain concepts, such as objects (e.g., aircraft, crew, aerodrome) and events (e.g. flight, accident) and all safety reports in aviation safety domain, i.e., all safety reports that are created to inform about all accidents or incidents in aviation domain (Kostov et al., 2017). The ontology consists of 19421 axioms, 6895 logical axioms, 1725 classes; around 900 of them have instances, 129 Object Properties and 2172 individuals (instances). We built Aviation Safety Ontology on top of the Unified Foundational Ontology (UFO)25

²⁵

http://onto.fel.cvut.cz/ontologies/ufo/current/index-en.html

(Guizzardi, 2005; Mylopoulos, 1992). Figure 7 depicts basic concepts in Aviation Safety Ontology that is represented in UFO. During the ontology development for the requirements election step, we constructed the following competency questions (CQs) that the ontology should answer. The competency questions were derived from an interview of domain users and experts (Kostov et al., 2017; Křemen et al., 2017). Also, we map the CQs to the foundational patterns from Table 3 to defend that user queries can be generalized to our foundational pattern.

Fig. 7.

Aviation safety ontology.

CQ1: What are the properties or qualifications of the safety agents (e.g., the air traffic control agents) (maps P1)

CQ2: Which people do necessarily participate in a particular event (e.g., specific damage event, a runway incursion)? (maps P2)

CQ3: Which safety events happen during a particular flight phase (e.g. take-off)? (maps P4)

CQ4: Who is responsible for a particular safety operation (e.g., ground handling)? (maps P8)

CQ5 What are the parts of an Aircraft? (maps P6)

CQ6 Which safety events have air traffic controller/pilot/ground service personnel as their participant? (maps P2)

CQ7 What are the conditions/properties of an aerodrome surface area? (maps P1)

CQ8 What are the factors/ dispositions that are manifested in particular events? (maps P5)

Fig. 8.

Mean value and standard deviation of execution query time for real data.

For evaluation, we answer the domain user competency questions by using foundational patterns. We ran the following queries against selected triple stores. The dataset consists of approximately 25000 triples. Figure 8 shows the mean values and standard deviation of execution time of running the following queries.

RQ1: What are the tropes (properties) that inhere in the air traffic control agent? (instance of P1 and CQ1)

RQ2: What are the participants of a damage event? (instance of P2 and CQ2)

RQ3: What are the event parts of a specific Flight which has Aircraft-i as a participant? (instance of P4 and CQ3)

RQ4: Who performs ground handling operation-i? (instance of P8 and CQ4)

RQ5: Select everyone that performs actions in aviation domain and filter all participating relation? (instance of P8,P2 and CQ2,CQ4)

RQ6: What are the parts of aircraft-i and in which event this object participates? (instance of P’6 and CQ5)

In order to compare the performance of triple stores with and without using UFO index, we ran the previous queries (RQ1, RQ2, RQ3, RQ4) on the aviation ontology without applying UFO index. The results in Fig. 9 indicates that using UFO-indexing approach makes the search process easier and faster, as we demonstrated in Ahmad et al. (2018).

Fig. 9.

Mean value of execution query time with and without UFO index.

6.4. Generated data vs real data

In this section, we optimize the foundational patterns by running the same following foundational queries on the same size of both generated and real data, i.e., safety data (around 26000 triples). Our goal is to compare triple stores performance in the same size with different UFO-based data and show how these foundational patterns are applicable for any dataset based on a unified foundational ontology, i.e., we can run these fondational patterns on different UFO based datasets.

Q’1: Who are the participants in all event in each dataset?

Q’2: What are all properties in each dataset?

Q’3: What are all actions that happened in both datasets with their parts?

Q’4: Who are the participants in all events in each dataset and what are the properties of every participant?

Figure 10 presents the mean values and standard deviations of the execution queries time on both datasets, i.e., real and generated data sets, where the box represents the standard deviation and the points represents the data labels. The figure indicates that Fuseki Jena is the worst and Virtuoso is the best.

Fig. 10.

Mean value of execution query time on generated vs real data.

7. Experiments discussion

In this section, we discuss the performance of triple stores after running the above different foundational SPARQL queries and the SPARQL features we used in those queries against them. The results of the execution time experiments we performed on different triple stores with different datasets sizes and types (i.e., generated data and existing real data) show that the performance of Fuseki Jena-TDB is the lowest among all triple stores and for all dataset sizes, and Virtuoso is better than RDF4J, Blazegraph, and GraphDB with respect to various features of SPARQL queries.

The mean values have the largest values in Q1 and Q3 in the generated dataset, as they are more complex queries with multi joins and also OPTIONAL blocks are evaluated before the parent block. However, Virtuoso has better space utilization compared to Blazegraph and other stores, and with its well-engineered query optimization engine, it is likely to be faster than others at processing optional join queries, especially for low-selective optional join queries. RDF4J (Sesame) and Fuseki perform worse on complex queries that contain many triple patterns and joins, as well as complex SPARQL clauses. Also for UNION and FILTER because of its optimization engine. For subject-object joins (e.g., Q6 in real data), Blazegraph performed best, perhaps due to its use of a B+ tree-based index nested loop join, which is more read-optimal compared to the bitmap index-based of both row- and column- store in Virtuoso. Virtuoso and Blazegraph typically use a hash join algorithm for executing subject-subject joins over the intermediary results. Subject-subject Joins (e.g., Q3 in real data) all triple-store typically use a hash join algorithm for executing subject-subject joins over the intermediary results. However, using UFO index improves the performance of all triple stores by 65% comparing without UFO index. Most interesting is the comparison of triple stores with and without UFO index which is the most interesting thing. The results show that using UFO-indexing approaches speeds up result retrieval by 68% for RDF4J, 72% for Jena, 70% for GraphDB, 67% for Virtuoso, and 65% for Blazegraph, see Fig. 9. The results also show that the performance of Fuseki Jena-TDB is the lowest and Virtuoso is the best. Figure 11 shows how the differences of triple stores performances by using UFO is lower than the differences without UFO. The results indicate that, even if Virtuoso is better than other triples stores, the performance’s difference between triple stores is lower than using UFO.

Blazegraph is better than RDF4J (Sesame) and GraphDB, taking into the consideration that in many cases, RDF4J is almost equal to GraphDB performance in simple queries; that is because GraphDB is built on top of RDF4j.

Moreover, in our experiment, we have shown a significant performance increase on a relatively small data sample for all foundational queries, i.e., the size of dataset plays an important role in a triple store performance.

Regarding the number of results, all of the selected triple stores return the same number of results. But, the result set size plays an important role on triple stores performance. Also, Fig. 5 shows that the dataset size affects the performance of triple stores. The results of the real dataset validation were checked by a domain expert, who confirmed their correctness and the usability of foundational ontologies in developing safety domain ontologies.

It is interesting to note that based on foundational patterns, we were able to run the same queries on different datasets, as we showed in Section 6.4. However, based on the results, we note that the performance of the triples was somehow better on real word data than on generated data, even though they have the same size, which could be due to the size of the result set.

In summary, the differences of performance of the benchmark on major triple stores are not huge, yet visible. This also shows that foundational queries are generic and structurally simple enough to align well with their existing indexing mechanisms. Moreover, it provides a basis for a substantial evaluation of the approach on real-world datasets that will show the applicability on larger diversity of foundational ontology-compliant data.

Fig. 11.

The differences of triple stores w.r.t mean value of execution query with and without UFO index.

8. Conclusion

In this paper, we proposed a foundational benchmark that evaluates performance of UFO-based SPARQL queries on domain ontologies. We used this benchmark for evaluating the performance of different triple stores on both real world and generated data. For this purpose, we created a foundational data generator that generates data based on the UFO model. The benchmark is applicable for any ontology based on UFO.

Furthermore, we indexed all datasets using our foundational indexing technique and evaluated selected triples stores using the designed benchmark. The performance gains are significant and are mostly comparable among the triple stores (with Jena being the winner).

Several improvements are planned for the future work to cover more SPARQL features with OWL entailment regimes. Also, we will do more evaluation for our UFO indexing approach by generating larger and more heterogeneous data and we will compare more triple stores with bigger sizes of UFO-based indexed datasets.

Footnotes

Acknowledgements

This work was supported by the Student Grant Competition of CTU, grant No. SGS19/110/OHK3/2T/13.

The experiments results

We present here the results in numbers (mean value and standard deviation of execution query time) of the benchmark experiments on triples stores in Tables 6–19.

References

Abadi, D.J., Madden, S.R. & Hollenbach, K. (2007). Scalable semantic web data management using vertical partitioning. In Proceedings of the 33rd International Conference on Very Large Data Bases VLDB ’07 (pp. 411–422), http://dl.acm.org/citation.cfm?id=1325900 .

Abadi, D.J., Marcus, A., Madden, S.R. & Hollenbach, K. (2009). SW-Store: A vertically partitioned DBMS for semantic web data management. VLDB Journal, 18(2), 385–406. doi:10.1007/s00778-008-0125-y.

Ahmad, J., Kremen, P. & Ledvinka, M. (2018). Optimization of queries based on foundational ontologies. In OTM Conferences. Lecture Notes in Computer Science (Vol. 11230). Cham: Springer. doi:10.1007/978-3-030-02671-4_22.

Benevides, A., João, P., Almeida, J. & Guizzardi, G. (2019). Towards a unified theory of endurants and perdurants: Ufo-ab. In Proceedings of the Joint Ontology Workshops 2019 Episode v: The Styrian Autumn of Ontology, Graz, Austria, September 23–25, 2019. CEUR-WS.org.

Benevides, A.B., Bourguet, J.-R., Guizzardi, G. & Peñaloza, R. (2017). Representing the ufo-b foundational ontology of events in sroiq. In Proceedings of the Joint Ontology Workshops 2017 Episode 3: The Tyrolean Autumn of Ontology, Bozen-Bolzano, Italy, September 21–23, 2017. CEUR Workshop Proceedings (Vol. 2050).

Bizer, C. & Schultz, A. (2009). The Berlin sparql benchmark. Int. J. Semantic Web Inf. Syst., 5, 1–24. doi:10.4018/jswis.2009040101.

Carvalho, V.A., Almeida, J.P.A., Fonseca, C.M. & Guizzardi, G. (2017). Multi-level ontology-based conceptual modeling. Data & Knowledge Engineering Journal, 109, 3–24. Special issue on conceptual modeling – 34th International Conference on Conceptual Modeling, http://www.sciencedirect.com/science/article/pii/S0169023X17301052. doi:10.1016/j.datak.2017.03.002.

Faye, D.C., Cure, O. & Blin, G. (2012). A survey of RDF storage approaches. ARIMA Journal, 15, 11–35. http://www.citeulike.org/user/paarnio/article/11477528 .

Griffo, C., João, P.A., Almeida, J., Guizzardi, G. & Nardi, J. (2017). From an ontology of service contracts to contract modeling in enterprise architecture. In 2017 IEEE 21st International Enterprise Distributed Object Computing Conference (Edoc). doi:10.1109/EDOC.2017.15.

10.

Guizzardi, G. (2005). Ontological foundations for structural conceptual models. Telematica instituut / ctit, Ph.D. thesis, UT.

11.

Guizzardi, G., Almeida, J.P.A., Guarino, N. & Carvalho, V.A.D.E. (2015). Towards an ontological analysis of powertypes. In Proceedings of the Joint Ontology Workshops 2015 Episode 1: The Argentine Winter of Ontology Co-Located with the 24th International Joint Conference on Artificial Intelligence (IJCAI 2015), Buenos Aires, Argentina, July 25–27, 2015, CEUR-WS.org.

12.

Guizzardi, G., Falbo, R. & Guizzardi, R.S.S. (2008). Grounding software domain ontologies in the unified foundational ontology (ufo): The case of the ode software process ontology. In 1th Iberoamerican Workshop on Requirements Engineering and Software Environments (IDEAS’2008).

13.

Guizzardi, G., Guarino, N. & Almeida, J.P.A. (2016). Ontological considerations about the representation of events and endurants in business models. In Lecture Notes in Computer Science LNCS (Vol. 9850, pp. 20–36), http://arxiv.org/abs/9780201398298 . doi:10.1007/978-3-319-45348-4_2.

14.

Guizzardi, G. & Wagner, G. (2005a). Towards ontological foundations for agent modelling concepts using the unified fundational ontology (UFO). In Lecture Notes in Computer Science. Berlin, Heidelberg: Springer Berlin Heidelberg. doi:10.1007/b136434.

15.

Guizzardi, G. & Wagner, G. (2005b). Towards ontological foundations for agent modelling concepts using the unified fundational ontology (ufo). In Proceedings of the 6th International Conference on Agent-Oriented Information Systems II, AOIS’04 (pp. 110–124). Berlin, Heidelberg: Springer. doi:10.1007/11426714_8.

16.

Guizzardi, G. & Wagner, G. (2010). Using the unified foundational ontology (UFO) as a foundation for general conceptual modeling languages. In Theory and Applications of Ontology: Computer Applications (pp. 175–196). doi:10.1007/978-90-481-8847-5_8.

17.

Guizzardi, G. & Wagner, G. (2011). Towards an ontological foundation of discrete event simulation (pp. 652–664). doi:10.1109/WSC.2010.5679121.

18.

Guizzardi, G. & Wagner, G. (2014). Dispositions and causal laws as the ontological foundation of transition rules in simulation models. In Simulation Conference (WSC), 2013 Winter (pp. 1335–1346). IEEE. doi:10.1109/WSC.2013.6721520.

19.

Guizzardi, G., Wagner, G., Almeida, J. & Guizzardi, R. (2015). Towards ontological foundations for conceptual modeling: The unified foundational ontology (ufo) story. Applied ontology, 10. doi:10.3233/AO-150157.

20.

Guizzardi, G., Wagner, G., de Almeida Falbo, R., Guizzardi, R.S.S. & Almeida, J.P.A. (2013). Towards ontological foundations for the conceptual modeling of events. In Conceptual Modeling. ER 2013. Lecture Notes in Computer Science (Vol. 8217, pp. 327–341). Berlin, Heidelberg: Springer. https://link-springer-com.web.bisu.edu.cn/chapter/10.1007/978-3-642-41924-9_27 .

21.

Guo, Y., Pan, Z. & Heflin, J. (2005). Lubm: A benchmark for owl knowledge base systems. Web Semantics: Science, Services and Agents on the World Wide Web, 3, 158–182. doi:10.1016/j.websem.2005.06.005.

22.

Harris, S. & Seaborne, A. (2013). SPARQL 1.1 query language. Tech. rep., W3C Consoritum. https://dx-doi-org.web.bisu.edu.cn/citeulike-article-id:2620569.

23.

Harris, S. & Shadbolt, N. (2005). SPARQL query processing with conventional relational database systems. In Web Information Systems Engineering – WISE 2005 Workshops, New York, New York (Vol. 3807, pp. 235–244). https://eprints.soton.ac.uk/261126/ . doi:10.1007/11581116-25.

24.

Klyne, G. & Carroll, J.J. (2004). Resource description framework (RDF): Concepts and abstract syntax, W3C Recommendation 10 (October) (2004) 1–20, http://www.w3.org/TR/2004/REC-rdf-concepts-20040210/.

25.

Kostov, B., Ahmad, J. & Křemen, P. (2017). Towards ontology-based safety information management in the aviation industry. In On the Move to Meaningful Internet Systems: OTM 2016 Workshops, LNCS (Vol. 10034). Cham: Springer.

26.

Křemen, P., Kostov, B., Blaško, M., Ahmad, J., Plos, V., Lališ, A., Stojić, S. & Vittek, P. (2017). Ontological foundations of European coordination centre for accident and incident reporting systems. Journal of Aerospace Information Systems. doi:10.2514/1.I010441.

27.

Křemen, P. & Kouba, Z. (2011). Optimization in OWL2-DL. In Lecture Notes in Computer Science (pp. 188–202). Berlin Heidelberg: Springer. doi:10.1007/978-3-642-23091-2-18.

28.

Kremen, P. & Necaský, M. (2019). Improving discoverability of open government data with rich metadata descriptions using semantic government vocabulary. J. Web Semant., 55, 1–20. doi:10.1016/j.websem.2018.12.009.

29.

Ledvinka, M. & Kremen, P. (2015). Jopa: Accessing ontologies in an object-oriented way. In ICEIS 2015 – 17th International Conference on Enterprise Information Systems.

30.

Morsey, M., Lehmann, J., Auer, S. & Ngomo, A.-C.N. (2011). Dbpedia sparql benchmark: Performance assessment with real queries on real data. In Proceedings of the 10th International Conference on the Semantic Web – Volume Part I, ISWC’11 (pp. 454–469). Berlin, Heidelberg: Springer. http://dl.acm.org/citation.cfm?id=2063016.2063046 .

31.

Mylopoulos, J. (1992). Conceptual modelling and Telos. In Conceptual Modeling, Databases, and Case an Integrated View of Information Systems Development (pp. 49–68). John Wiley & Sons.

32.

Saleem, M., Mehmood, Q. & Ngonga Ngomo, A.-C. (2015). Feasible: A featured-based sparql benchmark generation framework. In International Semantic Web Conference, Cham: Springer. doi:10.1007/978-3-319-25007-6_4.

33.

Saleem, M., Szárnyas, G., Conrads, F., Bukhari, S.A.C., Mehmood, Q. & Ngonga Ngomo, A.-C. (2019). How representative is a sparql benchmark? An analysis of rdf triplestore benchmarks. In The World Wide Web Conference, WWW ’19 (pp. 1623–1633). New York, NY, USA: Association for Computing Machinery. doi:10.1145/3308558.3313556.

34.

Schmidt, M., Schallhorn, T., Lausen, G. & Pinkel, C. (2009). Sp2bench: A sparql performance benchmark. In

De Virgilio ,

Giunchiglia ,

Tanca (Eds.), Semantic Web Information Management (pp. 222–233). Berlin, Heidelberg: Springer. doi:10.1109/ICDE.2009.28.

35.

Verdonck, M. & Gailly, F. (2016). Insights on the use and application of ontology and conceptual modeling languages in ontology-driven conceptual modeling. In

Comyn-Wattiau ,

Tanaka ,

Yamamoto and

Saeki (Eds.), Lecture Notes in Computer Science LNCS (Vol. 9974, pp. 83–97). Springer. doi:10.1007/978-3-319-46397-1_7.

36.

Vila, L. & Reichgelt, H. (1996). The token reification approach to temporal reasoning, Artificial Intelligence, 83(1), 59–74. issn: 0004-3702.

37.

W3C (2017). Shapes constraint language (SHACL). Tech. rep., W3C (Jul. 2017). https://www.w3.org/TR/shacl/.

38.

W3C Consoritum (2012). OWL 2 Web ontology language document overview. Tech. Rep. December, W3C Consoritum, http://www.w3.org/TR/owl2-overview/.

Foundational patterns benchmark

Abstract

Keywords

1. Introduction

4 https://www.cs.ox.ac.uk/isg/tools/UOBMGenerator/

3.1. OWL 2-DL

6 http://onto.fel.cvut.cz/ontologies/ufo

7 For the purpose of this paper, we consider constant to be IRIs only.

10 https://github.com/ahmadjana/ufomodel: This is the GitHub link for the source code.

4.2. Validator

13 See https://github.com/ahmadjana/ufomodel, cit. November 21, 2022.

19 http://rdf4j.org/

6.2. Generated data experiments

24 https://www.inbas.cz/aviation-safety-ontology

Footnotes

Acknowledgements

The experiments results

References

⁴
https://www.cs.ox.ac.uk/isg/tools/UOBMGenerator/

⁶
http://onto.fel.cvut.cz/ontologies/ufo

⁷
For the purpose of this paper, we consider constant to be IRIs only.

¹⁰
https://github.com/ahmadjana/ufomodel: This is the GitHub link for the source code.

¹³
See https://github.com/ahmadjana/ufomodel, cit. November 21, 2022.

¹⁹
http://rdf4j.org/

²⁴
https://www.inbas.cz/aviation-safety-ontology