Guidelines for the reuse of ontology content

Abstract

Reuse of elements from existing ontologies in the construction of new ontologies is a foundational principle in ontological design. It offers the benefits, among others, of consistency and interoperability between such knowledge structures as well as sharing resources. Reuse is widely found within important collections of established ontologies, such as BioPortal and the OBO Foundry. However, reuse comes with its own potential problems involving ontological commitment, granularity, and ambiguity. Guidelines are proposed to aid ontology developers and curators in their prospective reuse of content. These guidelines have been gleaned over years of practice in the ontology field. The guidelines are couched in experiential reports on designing and curating particular ontologies (e.g., EXACT and EXACT2) and using generally accepted approaches (e.g., MIREOT) in doing so. Various software tools to assist in ontology reuse are surveyed and discussed.

Keywords

Ontology reuse ontology modeling ontological commitment ontology reuse guidelines ontology reuse tools

1. Introduction

When one sets out to capture the knowledge in the domain of biology and biomedicine using an ontology, a natural question to ask is whether some of the content has been modeled before and can thus be reused in the present context. Invariably, the answer is yes. In fact, reuse is an important design principle in the ontology curation community. Ontology reuse is at the core of the OBO Foundry’s espoused foundational principles (OBO Foundry, 2022; Smith et al., 2007). It has also been proposed as one of the “ten commandments” of ontological engineering: “honor interoperability, so that your ontology has a long life and can be reused by others” (Jansen and Schulz, 2011). In the context of ontological engineering, reuse is considered a basic activity of the conceptual modeling phase (Kendall and McGuinness, 2019). We find support for reuse in the Web Ontology Language (OWL) via owl:imports axioms (OWL, 2022) and in the Protégé ontology editing software (Protégé, 2022; Noy et al., 2000). Additionally, top-level ontologies have been constructed with the express purpose of facilitating content reuse and alignment. An example is the Basic Formal Ontology (BFO; Grenon et al., 2004). Other “upper domain” ontologies, such as the Ontology for General Medical Sciences (OGMS, 2022) and BioTop (Beisswanger et al., 2008), have extended the BFO with more general domain knowledge that can also be reused by ontology authors.

In previous work, we have studied the kinds of reuse availed of in a large collection of ontologies housed in the BioPortal (2022). Specifically, in the paper of Ochs et al. (2017), we identified those ontologies that did indeed reuse content and investigated how and to what extent those ontologies utilized existing classes and properties to model new classes and properties. In further work (Halper et al., 2018), we examined the common kinds of modeling errors and inconsistencies that are likely to arise when reusing existing content. Examples include duplicated classes and properties as well as versioning problems.

Following up on the latter point, let us note that ontology reuse is not a panacea, and it is not as easy as it may first seem. An ontology author needs to be aware of potential issues that can affect the quality of their ontology. If reused content is not utilized correctly and/or not maintained properly, unintended erroneous concepts or properties may be introduced. And that leads us to the work presented in this paper. We wish to promote some guidelines that should help the prospective ontology author in content reuse. Since we realize that it is difficult to internalize abstract guidelines, we couch these guidelines in case studies of ontology reuse and enumerate what to look out for in order to avoid problems. The first study is in the context of the construction of the EXACT ontology (Soldatova et al., 2008) and its successor EXACT2 (Soldatova et al., 2014). The second involves general ontology construction employing OWL imports and the “Minimum Information to Reference External Ontology Terms” (MIREOT) technique (Courtot et al., 2011). Some of the issues discussed revolve around ontological commitment (Schulz and Cornet, 2009; Gruber, 1993; Kendall and McGuinness, 2019), granularity (Kumar et al., 2004), and ambiguity (Nirenburg and Raskin, 2001). We also survey some software tools that can aid an ontology author in the construction of an ontology benefiting from reuse. The advantages and disadvantages of the tools are discussed.

The remainder of this paper is organized as follows. In the next section, we review background information pertaining to biomedical ontology structure and reuse. Sections 3 and 4 constitute the results of this work. In particular, Section 3 contains the proposed guidelines for reuse along with experiential reports of ontology reuse that highlight the guidelines. Section 4 covers software tools that can facilitate such reuse tasks. Discussions and conclusions follow.

2. Background

2.1. Ontology structure and existing repositories

The exact definition of what an ontology constitutes has been an ongoing discussion for many years (Guarino et al., 2009). Ontologies can range across the spectrum from informal structures to highly formal representations specified in various logical languages (Guarino et al., 2009; Bodenreider and Stevens, 2006). In general, an ontology is a structure for capturing the knowledge of some domain of interest, such as biomedicine. It is often used in support of other applications to enable standardization and interoperability. A review of the utilization of various ontologies in the biomedical domain can be found in the paper of Rubin et al. (2008).

In this paper, we take a computer-science perspective and view ontologies toward the more formal end of the spectrum (Guarino et al., 2009; Bodenreider and Stevens, 2006). In particular, an ontology is defined using a language such as OWL or the OBO format. From a structural perspective, an ontology comprises a network of concepts organized in a hierarchy of IS-A relationships. If concept $C_{1}$ is connected to concept $C_{2}$ via an IS-A, then $C_{1}$ is a subconcept of $C_{2}$ , i.e., $C_{2}$ is more general than $C_{1}$ . For example, Glucose test IS-A Test. Each concept is further defined through a set of lateral relationships associating it with other concepts. We might find the relationship measures with the case that Glucose test measures Glucose. The generic part-of relationship can be utilized to form a mereological hierarchy defining concepts as parts and wholes in relation to each other (Halper et al., 1998). Concepts can also be defined with the use of axioms that can denote, for example, restrictions, domains, or equivalences.

The BioPortal is a site that houses a collection of over 1000 ontologies (BioPortal, 2022). These can serve as a good source of material for instances of reuse. The OBO Foundry also maintains a large collection of ontologies (OBO Foundry, 2022). The majority of ontologies appearing in the BioPortal are available in either OWL or OBO format. Other ontologies can be found through the Open Ontology Repository (OOR) Initiative (2022), as well as Ontobee (Xiang et al., 2011) and the EBI Ontology Lookup Service (EBI OLS; Jupp et al., 2015). Additionally, AberOWL consists of a collection of ontologies, allowing for ontology-focused access to biological data (AberOWL, 2022).

2.2. Ontology reuse

2.2.1. Abstract ontologies and patterns

Ontology researchers and developers have realized for some time that elements of reuse are important in ontology design and are key for successful ontology utilization and interoperability. Reuse promotes alignment of and consistency between ontologies and obviously eliminates duplication of work. The OBO Foundry has a stated principle of “commitment to collaboration” emphasizing this point (OBO Foundry, 2022). In a discussion of the conceptual modeling phase of ontology engineering, it has been stated that “we never start from scratch” (Kendall and McGuinness, 2019). In that context, the reuse is often focused on metadata, provenance, and pedigree.

To facilitate this reuse, we find a variety of ontologies focused more on abstract, foundational components of ontology content than on domain-specific knowledge. These include foundational ontologies (also called top-level ontologies or upper ontologies) and upper domain ontologies (also called top-level domain ontologies or top-domain ontologies).

A foundational ontology comprises very high-level categories and generic relationships that can be commonly used as the bases in other domain-specific ontologies (Keet, 2011; Khan and Keet, 2012). Such ontologies often define abstract concepts such as continuants and occurrents and seek to capture concepts that underlie natural languages and human discourse. Examples include the Suggested Upper Merged Ontology (SUMO; Niles and Pease, 2001), the Basic Formal Ontology (BFO; Grenon et al., 2004), the Descriptive Ontology for Linguistic and Cognitive Engineering (DOLCE; Gangemi et al., 2002), and the General Formal Ontology (GFO; Herre, 2010). In a study from the year 2015 (Ochs et al., 2017), we found that 77 out of 355 (= 21.7%) ontologies in BioPortal included at least part of the BFO in their content. According to the OBO Foundry, BFO is the upper ontology upon which OBO Foundry ontologies are built, and many OBO Foundry ontologies were among the 355 ontologies that were analyzed.

An upper domain ontology serves a similar purpose to a foundational ontology but focuses its attention on abstract concepts for a specific domain of interest, such as biomedicine. In the paper of Beisswanger et al. (2008), it was suggested that there is a need for a mediating layer between foundational ontology concepts and those comprising domain-specific knowledge. This is the role proposed for upper domain ontologies. Examples include OGMS, BioTop, and the General Formal Ontology for Biology (GFO-Bio; Hoehndorf et al., 2008).

Content reuse from upper domain ontologies like the OGMS, and domain-specific ontologies like the Gene Ontology (GO; Gene Ontology Consortium, 2022) and Chemical Entities of Biological Interest (ChEBI; 2022), is also fairly common. In recent efforts to quickly create COVID-19 ontologies, e.g., the Ontology of Coronavirus Infectious Disease (CIDO; He et al., 2020), curators employed reuse to accelerate the process. Reuse is so extensive in CIDO that out of its 7,834 concepts, only 774 are original. The rest are reused from 20 ontologies, including ChEBI, the Human Phenotype Ontology (HPO; Groza et al., 2015), GO, National Drug File – Reference Terminology (NDF-RT; Carter et al., 2006), and the Drug Ontology (DrON; Hogan et al., 2017). Studies have been conducted to determine the prevalence of ontology reuse. We find an analysis of term reuse and an accompanying analysis of error patterns in the work of Kamdar et al. (2017). A review of the orthogonality of the OBO Library ontologies can be found in the paper of Ghazvinian et al. (2011). In the work of Ochs et al. (2017), an investigation into the utilization of reused content in a sample of 355 BioPortal ontologies was carried out.

Ontology design patterns (ODPs) are “documented descriptions of proven solutions to typical problems of ontological modeling” (Zagorulko and Borovikova, 2020). Content ODPs, also known as knowledge patterns, typically model aspects of complex ontologies that frequently occur (Hitzler et al., 2016). Reusing the design pattern of ontologies in a similar domain improves consistency and alignment of ontologies. ODPs offer designers best practices and solutions that have been tried and tested by other ontology authors in the domain, thereby saving time and effort in the development process. When ontology authors reuse ontology content, it is important to take note of the commitments being made by the ODPs as well as the content and be aware of conflicts when different ontologies implementing different ODPs are reused at the same time.

In addition to direct ontology reuse, establishment and maintenance of mappings between ontologies has been employed. For example, mappings between life-science ontologies have been dealt with in the work of Groß et al. (2013).

2.2.2. Reuse approaches

An author of a new ontology O can reuse content from an existing source ontology, say, S in a couple of different ways. The different approaches have consequences in dealing with the periodic updates (e.g., error corrections and content additions) to S. An ontology author may choose to include content using the owl:imports mechanism defined in OWL syntax and implemented in the OWL API. This approach allows for the inclusion of the entire content of S into O “on the fly.” As such, updates to S will appear in O without any intervention on the part of O’s author. However, there may be unexpected consequences downstream, especially after classification.

On the other hand, O’s author may choose to reuse a fixed version of S’s content. This may be either S’s complete content or a selected subset extracted using, e.g., the MIREOT approach (Courtot et al., 2011). Reusing a fixed version of S’s content provides the author of O with greater control over when reused content is updated. But it comes at the expense of making the integration of changes to S into O more labor intensive.

2.3. Related work

As noted, reuse is accepted as a critical aspect of ontology development. Previous work has sought to facilitate the reuse process. The work of Keet and Grütter (2021) addresses the issue of resolving conflicts arising due to reuse (such as among imported concepts and relationships). Automated resolution strategies, that keep the human in the decision loop, are presented in explanatory ways. We see this work as complementary to ours. Our proposed guidelines seek to avoid the introduction of conflicts in the first place. But resolving the inevitable conflict that gets by helps assure the quality of the target ontology.

A software tool called ONSET has been developed to assist in the selection of an appropriate foundational ontology for a specific ontology development effort (Khan and Keet, 2012). This complements our guidelines with recommendations for choosing the best foundational ontology, if desired by an ontology developer. In contrast, we are not proposing criteria to distinguish the quality of different foundational ontologies.

The NeOn methodology defines a set of nine scenarios for building ontologies and ontology networks (Suárez-Figueroa et al., 2012,2015; Suárez-Figueroa, 2010). Scenario 3 of NeOn, “reusing ontological resources,” is most closely aligned with our aims, with Scenario 5, “reusing and merging ontological resources,” being related as well. Scenario 3 defines a sequence of activities, such as ontology search and ontology selection, to be carried out by the ontology practitioner toward successful reuse. To supplement the workflow, a “filling card” and assessment table are provided. Again, we see our own guidelines as complements to the NeOn approach, which has some wider goals in dealing with reengineering and the utilization of non-ontology resources. We are strictly focused on reusing existing ontologies.

As noted, ontology design patterns capture best practices for dealing with ontology modeling problems (Ontology Design Patterns . org, 2021; Gangemi and Presutti, 2009; Hitzler et al., 2016). In particular, reengineering ontology design patterns are sets of transformation rules that can be applied to a source model in order to produce a new ontology. In this approach, the source could be an ontology or even a non-ontological entity such as a data-model pattern, a UML specification, or some kind of linguistic structure. Our approach is to provide guidance based on past practice rather than transformation rules. However, we could see more formal rules emerging from the utilization of our guidelines and informing design patterns.

3. Reuse guidelines and experiences

3.1. Reuse guidelines

How does one go about initially selecting which ontology, or even which part of an ontology, to reuse? It certainly depends on the ontology design approach being applied. If a top-level ontology, such as BFO, or a top-domain ontology, such as OGMS, is being used, it often provides options within its ecosystem (i.e., among the ontologies that use the framework), making alignment easier. For example, the Infectious Disease Ontology (IDO; Cowell and Smith, 2010) is a sub-domain ontology that is based on OGMS and provides a broad framework for representing infectious diseases. Furthermore, its extensions, such as the Brucellosis Ontology (Lin et al., 2011), the Virus Infectious Disease Ontology (VIDO; 2022), the COVID-19 Infectious Disease Ontology (IDO-COVID-19; 2022), and the Malaria Ontology (Topalis et al., 2013), provide coverage of some specific areas. The same is true of the Sleep Domain Ontology (SDO; Arabandi et al., 2010). Top-level ontologies are often a good place to start when developing a new ontology, and they promote a modular way of building an ontology. Yet another option is to use an existing ontology with a broad coverage of biomedicine, such as SNOMED CT (SNOMED International, 2022),1

¹
SNOMED CT is available free of charge in SNOMED International member countries. (See https://www.snomed.org/snomed-ct/get-snomed; accessed October 14, 2022.) For example, in the USA, SNOMED CT is available via the UMLS. However, in certain circumstances, reuse of SNOMED CT may require some licensing agreement. See Guideline 8 below.

to provide previously vetted conceptual content for the new ontology structure.

The first aspect for selecting an ontology for reuse that comes to mind is certainly coverage, i.e., does the reused ontology provide good coverage of the domain of interest? (See, e.g., the paper of Kendall and McGuinness (2019).) For example, the Foundational Model of Anatomy (FMA; Rosse and Mejino, 2003) provides a broad and deep coverage of the anatomy domain, and SNOMED CT offers a broad and deep coverage of the clinical medicine domain. The Malaria Ontology, on the other hand, presents a deep coverage of malaria; however, it is a narrow ontology as it only covers malaria.

Another issue is the quality of the source ontology. Metrics derived from an ontology, or explicitly provided by its curators, can give a summary of the general quality and structure of an ontology. For example, the number of concepts defined in an ontology indicates the level of domain coverage. Ontology repositories, such as BioPortal, present basic metrics to users when viewing an ontology. And additional metrics, e.g., the number of concepts with textual definitions or the number of ontologies that reuse the ontology, could be used to guide users toward ontologies with higher quality content. As an example, when reviewing the SDO in BioPortal, one can glean these metrics: classes: 1382; properties: 100; depth: 19; maximum and average number of children: 48 and 2, respectively; and the number of classes without a definition: 611.

In previous work (Ochs et al., 2013; He et al., 2013b,a), we carried out quality-assurance studies of ontologies, e.g., SDO, the Cancer Chemoprevention Ontology (CanCO; 2022), and the Ontology for Drug Discovery Investigations (DDI; Qi et al., 2010), that were designed employing reuse. We demonstrated a number of cases of the migration of irrelevant concepts, which were not necessary for the new, target ontology. We also illustrated examples of duplicate concepts and duplicate properties, resulting from reuse.

How one reuses the content of an ontology depends on application and modeling need. If there is a need to cover anatomy in detail, then one should reuse a large portion of the FMA. However, if the need is limited to, say, anatomy of the feet, then importing the FMA in its entirety is really not necessary. Just importing a subhierarchy of the FMA will suffice. Therefore, depending on the situation, one could import an entire ontology, or a fragment thereof, or just individual classes. Related to this issue is the modularization of the source ontology, i.e., the ease with which the desired content can be separated from material that is not of interest (Kendall and McGuinness, 2019). In recent years, several ontology modularization techniques have emerged. An extensive review of these techniques can be found in the paper of Le Clair et al. (2022). Reusing a module of an ontology is one approach that allows ontology authors to limit the number of reused concepts while ensuring sufficient coverage based on their needs.

Let us note that a major issue that may arise in ontology reuse is increased difficulty in managing ontological commitments. In general, ontological commitment refers to the “precise agreement about the objects and relations [in the domain of interest that are] being denoted by the terms and concepts [in the ontology]” (Schulz and Cornet, 2009). Ontological commitment is a commitment to a certain theory or discourse that describes or explains the domain of interest. We would ask, “what entities or kinds of entity exist according to a given theory or discourse, and thus are among its ontological commitments?”2

https://plato.stanford.edu/entries/ontological-commitment/. Accessed August 21, 2022.

So, ontology developers “accept entities into our ontology via accepting theories that are ontologically committed to those entities.”3

See footnote 2.

As with all ontologies, a top-level ontology reflects a certain theory. For example, BFO, which is reused in over 200 domain ontologies, is based on realism. Consequently, those domain ontologies inherited ontological commitment to realism. While that may well be suited for biomedical ontologies describing biological and chemical entities, it may not be so for other ontologies, i.e., those that include abstract entities.

Importing an existing ontology (even partially) – and especially an upper-level ontology – to an ontology that is under development forces its developers to make ontological commitments about the structure, granularity, and the refinement of representations of the domain of interest. Imports from different ontologies may lead to difficulties in managing potentially inconsistent ontological commitments and logical entailments. Moreover, if some ontological commitments are no longer suitable for the intended applications, then that may require a fundamental redesign of the ontology.

In the following, we explore these issues from the perspective and experience of building the EXACT ontology (and its successor EXACT2) and from the use of OWL Imports and the MIREOT methodologies. Before moving on to those discussions, we enumerate the set of nine ontology reuse guidelines that we are proposing. Their relevance will be elucidated in the following. Let us again note that one of our overall aims in this work is to introduce our guidelines proposal and invite more people in the ontology community to expand on the work. We would like to see this as an ongoing effort.

Check the ontological commitments of the candidate ontology for reuse and compare them to the ontological commitments of the curated ontology before carrying out the reuse.

Consider all attributes of a reused concept, not only its name or label.

Reuse only ontology entities that come with textual definitions.

Check other concepts in the same IS-A subhierarchy where a reused concept is placed to ensure that there is no conflict created by ontological commitments of the sibling or hierarchically related classes.

Preserve the integrity of the taxonomy of the curated ontology consisting of all the hierarchical relationships in the ontology.

Check the ontological commitments of all superclasses (e.g., by axioms) in the source ontology of the entity being chosen for reuse.

Use software tools to automate some of the reuse tasks.

Obtain the proper licensing before reusing ontology content.

Reuse content only from actively maintained ontologies.

Let us comment on the last three guidelines here. Guideline 7 recommends the use of software tools, some of which are surveyed in Section 4. An important issue raised in the work of Kendall and McGuinness (2019) is licensing, listed here as Guideline 8. It is certainly mandatory to obtain the proper permission in order to reuse any ontology content, and the author of a new ontology must remember to do so before embarking on any reuse activities.

Another significant complication, addressed by Guideline 9, is that hundreds of potential reuse ontologies in BioPortal have not been updated in several years (if ever), based on the metrics provided by the site. Many of these ontologies are no longer maintained and had potentially reused old versions of source ontologies that are themselves long out of date. This leads to, for example, twelve versions of OBO REL/RO/BFO properties appearing throughout BioPortal’s ontologies (Ochs et al., 2017). This situation can impact ontology authors who decide to reuse the contents of these “dormant” ontologies (using, e.g., the BioPortal reuse plug-in for Protégé). Alternatively, the problem could be avoided by adding to reuse tools and ODEs a capability of prioritizing recently updated ontologies, based on their recorded update time-stamps, when searching for a concept to be reused. In this manner, the concept in the updated source ontology will be selected before some concept in a dormant ontology. This reinforces Guideline 7.

3.2. Checking commitments and illustrations with the EXACT ontology

Let illustrate some of the commitment issues associated with reuse on the example of the ontology EXACT (EXperimental ACTions) that was focused on the representation of functional genomics experiments run within the Robot Scientist project (Soldatova et al., 2008). EXACT employed the SUMO (Niles and Pease, 2001) as a top-level ontology to ensure interoperability with other software used in the project. Consequently, EXACT had the following top-level classes: Abstract entity, Object, Process, Proposition, Quality, Role. The set of these top-level classes reflects the ontological commitment about the structure and features of the chosen area of knowledge. For example, ontology developers often choose not to consider logical abstractions pertinent to the domain they model, and consequently the resulting ontology does not include abstract entities. In the case of EXACT, the representation of such abstract entities as Truth value and False value ware essential for the representation of pre- and post-conditions of experimental actions.

The developers of EXACT soon realized that other areas of biology may benefit from formal representation of experimental actions and their properties. Hence, “EXACT2” was designed to capture the full semantics of biomedical protocols required for their reproducibility (Soldatova et al., 2014). It reused BFO, Information Artifact Ontology (IAO; Ceusters, 2012), Phenotypic Quality Ontology (PATO; 2022), and Ontology of Biomedical Investigation (OBI; Bandrowski et al., 2016) classes to improve interoperability with other biomedical ontologies. And it reused ChEBI (Chemical Entities of Biological Interest (ChEBI), 2022) for the representation of biochemical entities, as well as the Eagle-I Research Resource Ontology (ERO; Torniai et al., 2011) for the representation of equipment employed in experiments. These reused ontologies have different commitments compared with the SUMO ontological commitments, and consequently the structure of EXACT2 differs significantly from that of EXACT (following Guideline 1).

BFO is based on the philosophy of reality that does not allow for the inclusion of abstract entities (due to the requirement that each ontological class must have instances in reality). Therefore, such classes as True value and False value were removed from EXACT2. As the result, EXACT2 does not support the representation of pre- and post-conditions of experimental actions. That was the necessary compromise between the level of expressivity and interoperability with biomedical ontologies reusing BFO. Hence, the class SUMO: Object has been replaced with the class OBI: material entity, the class SUMO: Proposition has been replaced with the class IAO: information content entity, and the class SUMO: Process has been replaced with the class OBI: planned process (definition: a process that realizes a plan which is the concretization of a plan specification) (Guideline 3).

The OBI: planned process branch has been carefully assessed, and several planned processes were imported or explicitly mapped to EXACT2 experimental actions (Guideline 4). However, from the perspective of the intended application – the reproducibility – the representation of processes in OBI lacks some essential information, because the intended applications of OBI are different (Guideline 2). For example, OBI has the following property for the class process: achieves planned objective (target: material maintenance objective), and realizes (target: concretizes some plan specification). Consequently, storage as a subclass of the class OBI: planned process inherits all its properties (Guideline 6).

Based on our analysis of the protocols (Soldatova et al., 2014), the specification of objective and plan are not essential for the reproducibility, while (storage) temperature, and period (of storage) are essential; but these latter elements are not captured by OBI. Some biochemical entities must be stored at (or below) $- 80^{\circ} C$ , $- 20^{\circ} C$ , or $+ 4^{\circ} C$ , and some may be kept at room temperature. The failure to record such essential information may result in the failure to correctly follow biomedical procedures, and thus produce erroneous results.

Fig. 1.

Example OBI and EXACT2 representations (unlabeled, solid arrows are IS-A relationships).

To overcome such limitations imposed by the commitment to OBI, EXACT2 introduces two types of descriptors of experimental actions: compulsory and optional. EXACT2: goal (that corresponds to OBI: objective) is represented as an optional descriptor, while material entity (which is being stored), storage temperature, and period are represented as EXACT2: compulsory descriptor. EXACT2 representations not only explicitly capture what information is essential for reproducibility and what is not, they are also more intuitive than those of OBI (see Fig. 1).

EXACT2 is being increasingly used to support automated experimentation, where all experimental information has to be encoded explicitly and in great detail (Williams et al., 2015). EXACT is being reengineered again to provide better support for such an application and to ensure interoperability with relevant areas. EXACT is being aligned with the Core Ontology about Robotics and Automation (CORA) proposed by an IEEE working group (Prestes et al., 2013). One of the extensions of CORA is the task ontology, and it aims to formally define principles for the representation of plans (including experimental plans), e.g., goal, task, subtask, constraint, evaluation, action, and capability. The developers of EXACT are participating in the development of the task ontology (Balakirsky et al., 2017).

Interestingly, CORA is based on SUMO. Consequently, the new version of EXACT for automated experimentation will have ontological commitments that are similar to the original EXACT.

3.3. Using OWL imports and MIREOT

The following experiences are based on reuse of ontology content by two main methodologies: OWL import and MIREOT. The latter is mostly based on two different implementations of MIREOT.

3.3.1. Picking ontology entities by textual definitions, not just by names/labels

A very common mistake is to pick entities from ontologies (classes, object properties, etc.) based on their name or label alone (see Guideline 2). This is a problem that applies not only to ontology reuse, but also to the use of controlled vocabularies and ontologies more generally. The term name or label can be very evocative, and it quite often turns out to be ambiguous upon closer inspection. A good example is the term “cell.” In biology and biomedicine, most people will immediately think about the smallest independently reproducing unit of life, while it is obvious that “cell” can have a multitude of meanings besides this, e.g., for cellular phone.

The same problem frequently occurs in reusing classes or other entities from ontologies as the following example shows: in a student exercise, students are asked to represent the fact that a trauma medical director is part of a trauma center. The students are provided with a basic ontology called the Ontology of Organizational Structures of Trauma Centers and Trauma Systems (OOSTT; 2021) and are asked to use Ontobee (2022), a browser for ontology terms, to discover entities from ontologies for reuse. OOSTT provides the representation of trauma medical director role and of trauma center. Medical director can be created as an anonymous class of Homo sapiens bearing a role trauma medical director using the object property bearer of from the Relation Ontology (Smith et al., 2005).

The most challenging part of the assignment was to find the proper relationship to connect a human being bearing a medical director role to a trauma center. One student picked the object property relation has member (from the Semanticscience Integrated Ontology (Dumontier et al., 2014)). But has member is defined as a mereological relationship between a collection and an item. It is obvious that this does not properly represent the relationship between an organization and a person being a part of that organization in the sense of filling a role therein. This becomes even more obvious looking at the definition of organization, one of the superclasses of trauma center (Guideline 5). An organization is an “entity that can bear roles, has members, and has a set of organization rules. Members of organizations are either organizations themselves or individual people. Members can bear specific organization member roles that are determined in the organization rules. The organization rules also determine how decisions are made on behalf of the organization by the organization members.” The third sentence of this definition makes it sufficiently clear that membership in an organization goes beyond a merely mereological relationship. Hence, picking that relationship was not the best option. A better option would have been to pick the relationship is member of organization from OBI, the same ontology from which organization came.

This demonstrates that in reusing entities from ontologies, we should not rely on the names or labels of entities, but we need to consult their textual definitions (Guideline 3). This also stresses how important it is for ontology developers to provide a textual definition for the entities that they represent. The best practice is to enter the definitions the moment the entities are added to the ontology to prevent later insecurity regarding what the entity was intended to represent.

This leads to the question about what to do with classes that do not come with definitions. In a July 2022 search using the “search ontology class” feature of BioPortal with the string “cell” (exact match), the returned results were matches in 77 ontologies. Of these matches, 29 (= 37.6%) did not have a definition (e.g., GO includes a textual definition, but SNOMED CT does not). Per SNOMED CT’s documentation,4

⁴
https://confluence.ihtsdotools.org/display/DOCGLOSS/textual+definition. Accessed August 21, 2022.

“textual definitions are optional and are only provided for a limited number of concepts, where there is a requirement (for) additional detail.” The “cell” example clearly demonstrates that ontology classes without a definition are a common phenomenon, often by design, in a particular ontology.

But why should we be bothered by this? Certainly, when we say “cell,” especially in the biomedical domain, the definition is clear. Reviewing all of the definitions of “cell” in the BioPortal search results demonstrates how important definitions are. The GO defines cell as: “The basic structural and functional unit of all organisms. Includes the plasma membrane and any external encapsulating structures such as the cell wall and cell envelope” (Gene Ontology Consortium, 2022). Many of the classes retrieved are defined in a similar way, and we may assume that the definitions provided are co-extensive. But there are examples of definitions that show that based on the term alone, we might have made faulty assumptions on what the class is intended to represent. In the Mosquito Gross Anatomy Ontology, the class named “cell” represents a part of the wing anatomy of mosquitoes and is defined as: “an area of wing membrane delimited by veins or by veins and the wing margin; named after the vein immediately anterior to it or after the posterior element if the vein is formed by the ‘fusion’ of two vein branches” (Mosquito Gross Anatomy Ontology, 2022). The VIVO-Integrated Semantic Framework uses a class called “cell” as a subclass of “Telephone Type” defined as: “also called mobile telephone” (VIVO-Integrated Semantic Framework, 2022). These definitions show that ontology developers looking for reuse of terms should not make assumptions based on entity names, but that they need to check the definitions to ensure the intended meaning of the term in question is as expected. Due to this, it is not recommendable to reuse ontology entities that do not come with any definition (Guideline 3).

3.3.2. Picking concepts by role textual definitions, not just names

This fairly common problem is related to the problem using names (see Guideline 2) since it also relies on picking a term based on its name or label. This mistake can easily happen, especially when using roles that have abbreviated labels. Two examples of this are metabolite from ChEBI and active ingredient from DrON (Drug Ontology, 2022). Ontology developers aiming for reuse sometimes pick these terms as parents for classes representing metabolites or active ingredients (e.g., 2,3-dihydrodipicolinate and acetaminophen, respectively). However, looking at the textual definition of active ingredient reveals that this would be a mistake: “a role of a scattered molecular aggregate that is part of a drug product that is realized by (1) administration of the drug to an organism followed by (2) some change in the structure or functioning of some part of the organism” (Guideline 3). So, we are dealing with a role of a portion of a chemical substance here. The portion of acetaminophen in a capsule of 500 mg Tylenol is a bearer of an active ingredient role. This portion’s role active ingredient is an instance of active ingredient; the portion is not. While in DrON such issues were clarified by the definition, ontology developers striving for reuse face an additional hurdle. Looking up the definition of metabolite, we find: “any intermediate or product resulting from metabolism. The term ‘metabolite’ subsumes the classes commonly known as primary and secondary metabolites” (Drug Ontology, 2022). Based on this, it seems we might safely reuse the class to represent individual portions of 2,3-dihydrodipicolinate. However, looking further into the matter, the ChEBI specification also includes in an axiom that “metabolite” is a “biochemical role.” This shows that our assumption based on the name and the textual definition is not correct and that the taxonomic relationship prevents us from using metabolite as the superclass of 2,3-dihydrodipicolinate or as a type for portions thereof (Guideline 5).

3.3.3. Confirming ontological commitments based on textual definitions in the taxonomy

Neglecting ontological commitments based on textual definitions is another reuse problem that frequently arises. It might, in fact, not be due to reuse per se, but by developers trying to re-model the structure of external information models or surveys in their own new ontologies. Suppose that a developer needs to model current patients and previous patients of a hospital. They find that patients are represented using patient roles, e.g., in OBI. The ontology developer assumes that previous patient ought to be a role, too. Nothing in the definition of the superclass role deters one from such a plan, as it reads: “b is a role means: b is a realizable entity and b exists because there is some single bearer that is in some special physical, social, or institutional set of circumstances in which this bearer does not have to be, and b is not such that, if it ceases to exist, then the physical makeup of the bearer is thereby changed” (Smith et al., 2005).

However, when reusing entities from ontologies, it is vitally important to check other entities on the same branch of the taxonomy to ensure that there is no conflict created by ontological commitment of superclasses (Guideline 4). In our example, the conflict would arise immediately at the next superclass to role, namely, realizable entity. The latter in OBI is defined as: “to say that b is a realizable entity is to say that b is a specifically dependent continuant that inheres in some independent continuant which is not a spatial region and is of a type, instances of which are realized in processes of a correlated type.” This definition clearly demonstrates that all roles are realized by a process. There are no processes that would be realized by a previous patient role. Not participating in treatment or not meeting a research nurse are clearly not processes; they are absences, and they cannot be used as grounding for the realization of a role.

Another example that would raise an ontological commitment challenge in reusing OBI is when the author needs to model a person who declines to sign an informed consent for a clinical trial. Based on the Realizable entity class in OBI, there is no action that would realize a “declined informed consent” role; hence, representing a person as a role in this instance would be an error. For developers focusing on reuse, the important guidance here is to check the ontological commitments of all superclasses (Guideline 6). This, of course, assumes that the taxonomy that the developer plans to reuse is in itself consistent and correct.

3.3.4. Confirming ontological commitments based on axioms in the taxonomy

Sometimes, it is not only the textual definition that conflicts with the reuse, but also an axiom on a class, either through a necessary and sufficient axiom or through a necessary axiom. These cases are particularly important since they can easily lead to inconsistencies of the ontology, once the development progresses and more and more axioms are added. An example of this came up in the development of an ontological representation for the divorce process based on the ontology of document acts (d-acts; 2022) and the Ontology of Medically Related Social Entities (OMRSE; 2022). Given that we wanted to represent the actual divorce (the process that legally annuls the marriage) and the agreement of the spouses to get a divorce, how would we do that? The former is clearly a document act in most legal systems. For example, in the d-acts, we find: “a social act creating, revoking or transferring a socio-legal generically dependent continuant or a role by validating (signing, stamping, publishing) a document.” In the divorce, the mutually dependent roles of the spouses are revoked.

We might now think that the mutual agreement of the two spouses to get a divorce is a declaration, as it exists in OMRSE. However, consulting the textual definition should already deter us from that idea: “a social act that brings about, transfers or revokes a socio-legal generically dependent continuant or brings about or revokes a role. Declarations do not depend on words spoken or written, but sometimes are merely actions, for instance the signing of a document.” But more importantly, there is a necessary and sufficient axiom on this class that postulates that each member of this class has to revoke a social or legal entity, transfer it, or create it. However, none of that happens when two spouses declare their willingness to get a divorce. Hence, that act should not be a member of that class. This demonstrates how important it is to check the ontological commitments created by axioms on superclasses that are being chosen (Guideline 6). Under no circumstances should these be neglected.

Table 1
Reuse checklist; “G1” is “Guideline 1,” etc.

Guideline Question Action

Yes No

G1–G6 1. Do you reuse classes and relationships that are already defined in other ontologies? Do not “reinvent the wheel” and unnecessarily proliferate representations.

G1–G6 2. Is the candidate source ontology a reputable, widely used resource? Proceed to use it. Consider using another ontology.

G1, G4, & G6 3. Do the new ontology and the source ontology share commitments? Consider importing individual classes and their modifications to have similar commitments as the source ontology.

G2, G3, & G7 4. Do you use automated tools to identify and import suitable classes? Check the definitions of imported classes to ensure that they have the intended semantics. Consider employing reuse tools to simplify your work.

G5 5. Are you sure that every imported class is placed at the correct location in the IS-A hierarchy of the new ontology? Double-check superclasses and sibling classes in the new ontology.

G2 6. Do the new ontology and the source ontology share the same naming conventions? Harmonize the naming conventions.

G7 7. Did you check the new ontology resulting from reuse for logical inconsistencies? Run a reasoner.

G7 8. Did you check the new ontology for duplications, including those due to differences in naming conventions and misspellings? Double-check the names; use a spell-checker; check existing synonyms.

G8 9. Do you have proper licensing to reuse the content? Obtain proper licensing or find an alternate ontology that is properly licensed.

G9 10. Is the ontology actively maintained? Reuse an alternate ontology, if available.

Guideline	Question	Action
G1–G6	1. Do you reuse classes and relationships that are already defined in other ontologies?		Do not “reinvent the wheel” and unnecessarily proliferate representations.
G1–G6	2. Is the candidate source ontology a reputable, widely used resource?	Proceed to use it.	Consider using another ontology.
G1, G4, & G6	3. Do the new ontology and the source ontology share commitments?		Consider importing individual classes and their modifications to have similar commitments as the source ontology.
G2, G3, & G7	4. Do you use automated tools to identify and import suitable classes?	Check the definitions of imported classes to ensure that they have the intended semantics.	Consider employing reuse tools to simplify your work.
G5	5. Are you sure that every imported class is placed at the correct location in the IS-A hierarchy of the new ontology?		Double-check superclasses and sibling classes in the new ontology.
G2	6. Do the new ontology and the source ontology share the same naming conventions?		Harmonize the naming conventions.
G7	7. Did you check the new ontology resulting from reuse for logical inconsistencies?		Run a reasoner.
G7	8. Did you check the new ontology for duplications, including those due to differences in naming conventions and misspellings?		Double-check the names; use a spell-checker; check existing synonyms.
G8	9. Do you have proper licensing to reuse the content?		Obtain proper licensing or find an alternate ontology that is properly licensed.
G9	10. Is the ontology actively maintained?		Reuse an alternate ontology, if available.

3.4. Reuse checklist

To complete our survey of the guidelines for the reuse process, we provide in Table 1 a reuse checklist for the benefit of the ontology author. The table is organized as a series of ten yes/no questions, with accompanying actions based on the answers. The checklist questions are generally related to the overall reuse process; however, some questions are more relevant to specific guidelines. In the table, we have mapped the questions to their most significant guideline(s). For example, the ontology author is asked (Question 4; Guidelines 2, 3, and 7): do you use automated tools to identify and import suitable classes? If yes, the author should check the definitions of the imported classes to ensure that they have the intended semantics. If not, the author is encouraged to employ reuse tools.

4. Software tools

Finding suitable ontologies for reuse and effectively reusing them is not easy, and will probably be time consuming, especially for novice ontology developers. In the work of Butt (2015), it was shown that the reuse challenges may deter developers. Therefore, it is important to select the right software tools for the right reuse tasks (Guideline 7). In the following sections, we discuss features of popular reuse tools to (a) support searching for relevant ontologies, and (b) import terms from the reused ontology into the new ontology.

4.1. Ontology search tools

The first task for an ontology developer, who is considering reusing already existing resources, is to identify if such suitable ontologies exist. Ontology libraries and repositories provide a convenient platform for ontology developers to publish their vocabulary of terms in an easily accessible format. The popular ontology repositories have built-in tools that allow the user to search for terms which they intend to reuse. These platforms also offer sophisticated functionalities such as visualization, querying, analysis, and mapping of terminologies. Another innovative functionality is the recommender system that takes a text description of a project or ontology and uses keyword matching to recommend ontologies the user should consider reusing. This is very useful in narrowing down the number of ontologies to explore, which can otherwise be overwhelming. There are additional services such as REST APIs and SPARQL endpoints that allow developers to easily integrate the resources from the ontology repositories into application programs. Some of the popular ontology repositories that offer the preceding features include the BioPortal (Whetzel et al., 2011), Ontobee, Linked Open Vocabularies (LOV; Vandenbussche et al., 2017), and the EBI OLS.

The BioPortal and the EBI OLS enable users to search for terms in ontologies from the biomedical domain. Although these libraries contain a large number of ontologies, the tools are restricted in being domain-dependent. The LOV and Ontobee work similarly to BioPortal and EBI OLS but have the advantage of being domain independent. LOV also has ontologies developed in languages other than English.

There are some tools that are designed to search through multiple ontology libraries and extract relevant information such as the IRIs (Internationalized Resource Identifiers), term labels, and definitions for reuse. An example of such a tool is the OntoMaton, a Google Spreadsheet widget that allows users to search for terms within ontologies (Maguire et al., 2013). OntoMaton was initially developed to find terms from the BioPortal but was later extended to search through the LOV and the EBI OLS. Figure 2 shows the user interface of the OntoMaton tool.

Fig. 2.

OntoMaton ontology search tool in Google spreadsheet.

There are several efforts towards ontology search tools which have either become obsolete or are not yet available to the public. For example, TONES Ontology Repository (Horridge, 2015) provides a general domain-independent search tool, but it appears to be deprecated. MapOnto (An et al., 2006) is also no longer maintained, while CORE (Fernández et al., 2006) and WebCORE (Cantador et al., 2020) seem to be stuck in the research phase.

4.2. Ontology import tools

Most ontology development environments (ODEs) have the capability of importing and thus reusing ontologies. Protégé is one of the most popular ontology development tools (Noy et al., 2000; Singh and Anand, 2013). It supports a direct import of an ontology from a specified file or a location on the web. A user can import an entire ontology or reuse selected terms from the ontology. Protégé’s import from a web location feature has the advantage of reusing the most up-to-date version of an ontology. Figure 3 shows the direct import feature of Protégé with the available import options.

Fig. 3.

Protégé direct ontology import wizard.

Protégé also has a range of plug-ins to support reuse, one of which is the MIREOT plug-in (Hanna et al., 2012). This plug-in offers the convenience of searching for terms and directly importing the relevant ones within the same ODE. This tool allows the user to search through a list of available ontologies or from an external ontology. Figure 4 shows the MIREOT plugin for Protégé with the available features.

Fig. 4.

MIREOT plugin for Protégé.

Protégé has built-in reasoners, such as HermiT (Glimm et al., 2014), that check for logical consistency (Question 7 of the reuse checklist). HermiT also identifies subsumption relationships between classes which can help the ontology author identify inferred classes and check that they align with the hierarchical structure of the curated ontology (Guideline 5).

Ontofox is another popular tool for reusing ontology terms (Xiang et al., 2010). This is a web-based tool that creates a file and stores only the selected terms the developer wishes to reuse from a particular ontology. This file can then be imported, for example, using Protégé’s direct import feature. Ontofox has the advantage of following the full MIREOT guidelines for reusing ontology terms (Xiang et al., 2010). Ontofox contains a database of ontologies and the tool has a feature that allows the user to provide the source of any external ontology and extract terms for reuse. Ontofox is helpful in promoting modularization by allowing the author to extract and reuse a fragment of or specific terms from an ontology based on their needs.

There are other ODEs with reuse features and plug-ins. For example, WebODE supports reuse and the merging of ontologies (Hanna et al., 2012). OntoEdit has plug-ins for importing and exporting ontologies in different formats (Sure et al., 2002).

Another reuse tool is ROBOT, which is available as either a Java library or a command-line tool to automate workflows during ontology development (Jackson et al., 2019). ROBOT is an open-source library built by packaging some useful, low-level OWL API functionalities into high-level functionalities for working with OBO ontologies (Jackson et al., 2019). ROBOT was built to support OBO Foundry ontology development according to best practices, as specified in the OBO Foundry principles (OBO Foundry, 2022; Smith et al., 2007). Among the various features of ROBOT, there are the “extract,” “remove,” and “filter” commands to create subsets of ontologies as import modules. The import modules from several reused ontologies can then be combined into one import file or into the main ontology using ROBOT’s “merge” command. It is worth mentioning that ROBOT offers a wide range of capabilities besides the reuse commands to support functions such as reasoning, querying, and reporting. In addition, ROBOT supports template driven term generation using Dead Simple OWL Design Patterns (DOS-DPs; Osumi-Sutherland et al., 2017), which fosters reusability of those newly created terms by other ontologies.

Table 2 gives a summary of the ontology reuse tools in this section highlighting the core features of each tool and the current status of each tool.

Table 2

Summary of reuse software tools highlighting the tools’ core features that support reuse, and the current usage status

Software Tool	Core Features	Status
BioPortal	Large corpus of biomedical ontologies (1,003 ontologies with 79,636,946 class mappings as of 08/07/2022) Ontologies available in different formats (OWL, OBO, and UMLS) Online search tool Visualization, ontology recommendation, annotation, and mapping	In use
Ontobee	Large corpus of biomedical ontologies and non-biomedical ontologies (252 ontologies as of 08/07/2022) Online search tool Visualization, annotation, and mapping SPARQL query	In use
Linked Open Vocabularies	Large corpus of ontologies from all domains (779 vocabularies on 08/07/2022) Online search tool SPARQL query	In use
EBI Ontology Lookup Service	Large corpus of biomedical ontologies (277 ontologies on 08/07/2022) Online search tool Tools for ontology mappings and building ontologies from spreadsheets. API tool	In use
OntoMaton	Online search tool which integrates several libraries Google spreadsheet add-on	In use
TONES Ontology Repository	Web service provides application developers with a repository of ontologies for testing purposes A corpus of OWL-DL ontologies	Obsolete
MapOnto	Ontology library for the MapOnto project	Obsolete
Protégé	Ontology development tool Plugins such as MIREOT for searching terms Direct import of ontologies from file or web Built in reasoners for logical consistency checks	In use
Ontofox	Ontology term and axiom extraction tool Follows MIREOT principles Exports reuse terms in various formats Can be used to extract terms from any ontology source code	In use
ROBOT	Command-line tool and Java library Commands for reusing external ontologies Extract, remove, filter, and merge Supports querying and reporting Provides reasoning capabilities Converts ontology files into different formats Chaining commands to create a workflow-like function	In use

5. Discussion

The reuse of ontology content is a foundational design pattern of ontologies. With more and more ontologies being created and deployed in important biomedical applications, it is critical that reuse be done correctly so as to avoid erroneous content and accompanying deleterious effects on the applications. For this purpose, we outlined a set of reuse guidelines to help ontology authors, especially novice authors. Since those guidelines are abstract, we brought use cases to illustrate what happens when the guidelines are not followed. In this way, we have surveyed some of the problematic outcomes associated with reuse and offered suggestions for avoiding them.

For example, regarding Guideline 1 (focusing on ontological commitments), we note that different ontologies can be used for various applications that may imply different ontological commitments. Such commitments can be enforced by the reused ontologies. Changes in ontological commitments may lead to a significant redesign of the conceptual framework of the supporting ontology and to a re-evaluation of which ontologies are suitable for reuse. As we discussed, the EXACT ontology was initially designed for the representation of automated experiments, and it adhered to the SUMO commitments. Then, EXACT was redesigned for a wider application, namely, the representation of biomedical experimental procedures to ensure their reproducibility. As a part of the redesign process, the suitability of SUMO as a top-level ontology was re-evaluated. It was decided that BFO was a more suitable top-level ontology for the intended application. The change of the top-level ontology (and as a result, ontological commitments) led to substantial structural changes in EXACT. The redesign of an ontology is certainly a complex and expensive process, and it is prone to errors and hard to automate. Therefore, ontology developers need to take into consideration such possible complications when reusing other ontologies, and Guideline 1 captures an important aspect of this.

We believe it is important for those venturing into the creation of a new ontology based on existing ontologies to familiarize themselves with the guidelines presented herein before starting. This way errors in the reuse of ontologies may be prevented a priori. Furthermore, we observe that many creators of ontologies are typically committed to one model of ontologies and are not familiar with other models. Thus, we made a choice to illustrate the guidelines according to different models and perspectives at the expense of some repetition of cases, e.g., confirming commitments based on definitions and not just names. This allows the prospective ontology developer to see the implications of following the guidelines in the context of different models.

Let us note that while ontology search tools can help promote reuse, they can also lead to violations of Guideline 2. This is due to the fact that such search tools match keywords to ontology term labels. The user must then manually check if the terms in the search results indeed have the intended semantics. Unfortunately, this is not always the case, as different ontologies may use the same term with different contexts or meanings, as discussed above. For example, in the Schema.org, the label ‘Place’ is a class of entities that have a fixed physical extension, while in the Event Ontology ‘place’ is the property that relates an event to a spatial object. Another common problem is that different ontologies may use different synonyms for the same term, and a search tool could fail to find some of the synonyms. Using the same example, the location of an event and the event can be related with either the property ‘place’ from Event Ontology or ‘location’ from Schema.org. However, if the user searches for the term ‘location’ in Event Ontology, it cannot return the result of ‘place’ even though they mean the same thing, and vice versa for searching in Schema.org.

The BioPortal does slightly better in this regard as it uses the Lexical OWL Ontology Matcher (LOOM) to show mappings between classes in different ontologies. LOOM is an ontology alignment tool that takes two OWL ontologies and identifies corresponding concepts by comparing the preferred names and synonyms of the concepts using an approximate matching technique (Ghazvinian et al., 2009). The technique used in LOOM allows matching of concepts despite slight misspellings as well as the use of synonyms, which is helpful in answering Question 8 of the reuse checklist to avoid duplication of terms.

It is important to bear in mind that ontology editing software, such as Protégé or similar tools, may not enforce adherence to guidelines and best-practices recommendations for ontology reuse, e.g., those provided by the OBO Foundry. The OBO Foundry requires that every term in an OBO Foundry ontology have a textual definition. Most ontologies create an annotation value that uses the IAO definition annotation. One of the OBO Foundry’s recommendations for ontology reuse is: “if an individual term is reused without change to the definition, the original term IRI should be used. If the definition of a term (either text or logical) is changed, the original term IRI should not be reused” (Smith et al., 2007).

The first part of this recommendation is typically what is implemented in tools for reuse of individual terms, such as Ontofox (Xiang et al., 2010) and the MIREOT-based Protégé plugin (Hanna et al., 2012). However, if a developer makes changes in the textual definition of a term from an external ontology, then it is important to ensure that the IRI is changed as well. This is crucial since, otherwise, entities that do not belong in the class a term refers to could end up being elements of that class. For example, let us assume that someone wants to reuse the term “journal article” from the IAO to support a network of annotating medical journal papers. The IAO term is defined as: “A report that is published in a journal.”5

⁵
http://purl.obolibrary.org/obo/IAO_0000013. Accessed October 11, 2022.

But the person wants to alter the definition to read as follows in order to more closely represent the use case: “A medical report that is published in a journal.” Doing so, the target ontology should have a new IRI for the term, since the intended meaning is more specific than the intended meaning in the source ontology. So, if the textual definition changes significantly (more than correcting grammar, spelling, or phrasing), the developer of the target ontology needs to assign a new IRI. Likewise, the developer of the target ontology needs to check whether the axioms for a term in the source ontology hold for the new intended meaning in their ontology. If not, the IRI from the source ontology should not be reused.

Thus, ontology reuse tasks are adequately supported by suitable tools and ODEs. However, it is important to be cognizant of their limitations. It would be beneficial to ontology developers if tools could include additional monitoring features that helped ensure adherence to recommendations and guidelines.

Overall, the problems ontology developers are facing regarding ontology reuse, and associated problems that arise from it, can be taken as a call for increased and improved education in knowledge representation, semantic web technologies, and ontologies. This is especially important since many ontologies are created by professionals in various disciplines where such ontology creation is required for the support of their applications. Given the existing necessity to semantically integrate data in data-intensive fields such as biomedical research and clinical care, knowledge representation and reasoning basics warrant a prominent place in curricula for students aiming to work in data science within those fields.

Let us point out that we did not run any controlled experiments creating ontologies with the use of our guidelines. The guidelines were gleaned from the many years of experience of the authors in ontology creation and curation. We do expect to be able to examine the quality of future ontologies formulated with an eye toward the reuse guidelines to assess their impact. Moreover, as mentioned, we see our set of guidelines as a starting point. We do hope that the set will be augmented and refined by the ontology community in an ongoing effort. As alluded to, a good way to aid in this would be to have the guidelines incorporated into some of the tools we have surveyed or additional software to be designed in the future. Let us mention that experiments in teaching good biomedical ontology design to novices have been carried out (Boeker et al., 2012).

It will be noted that in the scientific community there has been a push to make scholarly data, particularly data gathered in the context of publicly funded research projects, more easily accessible via automated means. A set of “FAIR” (“Findability, Accessibility, Interoperability, and Reusability”) principles has been proposed toward this end (Wilkinson et al., 2016). While the focus of FAIR may be on underlying data, it does include a knowledge component. And, in this respect, ontologies can play a key role by standardizing scientific domain knowledge. Moreover, our guidelines for ontology reusability play right into this. It is critical that different ontologies maintain consistency of their constituent knowledge to allow for consistent interpretation of disparate data sets. Our guidelines seek to promote this consistent knowledge. The process of ontology reusability also implies an element of interoperability. Lastly, the ontology community at large has sought to address the issues of findability and accessibility via the establishment of various ontology repositories, discussed in this paper.

And finally, let us mention something about the authors’ ontology experiences and development efforts. Author LNS has created EXACT (Soldatova et al., 2008) and was involved in the development of EXPO (Generic Ontology of Experiments; Soldatova and King, 2006), OBI (Bandrowski et al., 2016), OntoDM (Ontology for Data Mining; Panov et al., 2008), OntoDT (Ontology of Data Types; Panov et al., 2015), Exposé (Machine Learning Experiments; Vanschoren and Soldatova, 2010), and DDI, among others. Author FSM has worked on the development of OCL-SOP (Maikore et al., 2017), NDDO (Kostovska et al., 2019), an ontology to describe relationships between behaviors as part of the TURBBO project (Tools for Understanding Relationships Between Behaviours using Ontology (TURBBO), 2022), and an ontology for child abuse injuries (Ontology for ELEctronic tool for Clinicians, Trainers and Researchers In Child Abuse, 2022).

Author MB is the main curator or co-main curator of nine ontologies and has contributed to multiple others. The ontologies he has created or co-created are: Ontology of Biobanking (Brochhausen et al., 2016), Proper Name Ontology (2022), d-acts (2022), OOSTT, Drug-drug Interaction and Drug-Drug Interaction Evidence Ontology (DIDEO; Brochhausen et al., 2014), Apollo-SV (Hogan et al., 2016), ACGT Master Ontology (Brochhausen et al., 2011), OMRSE, and the Ontologized MIABIS (OMIABIS; Brochhausen et al., 2013).

Authors MH, CO, and YP have been involved in the analysis and application of ontology reuse efforts. They have developed a number of ontology summarization methodologies, including a variety of abstraction networks. Among other applications, these techniques have been brought to bear in quality-assurance efforts of ontologies, many of which have been created with reuse paradigms. Some of the ontologies analyzed are SNOMED CT, the National Cancer Institute thesaurus (NCIt; Sioutos et al., 2007), ChEBI, ERO, the Uber Anatomy Ontology (Uberon; Mungall et al., 2012), GO, SDO, the Ontology of Clinical Research (OCRe; Sim et al., 2014), DDI, and CanCO.

6. Conclusions

Ontologies have become critical to biomedicine and health care information processing enterprises. Given the fact that ontology reuse is widely employed – and encouraged – by the ontology community, when curators seek to formulate new ontologies, it is necessary to provide them guidance to avoid common mistakes and promote quality and consistency of the new products. Toward that end, we have presented a collection of guidelines to be followed by ontology authors as they work on their new ontologies. These guidelines have been gleaned from years of experience working on various ontologies. We presented experiential reports about the EXACT and EXACT2 ontologies, as well as work using MIREOT to illustrate these abstract guidelines in actual settings. We also surveyed some software tools available to the prospective author looking to employ reuse of existing ontology content. Overall, adherence to the suggested guidelines should lead to the curation of ontologies of better quality and enhanced interoperability. Furthermore, we suggest that ontology authors who follow these guidelines will save significant time and effort, which would otherwise be necessary for quality-assurance regimens focusing on errors arising due to improper reuse of ontologies.

Footnotes

Acknowledgements

Research reported in this publication by author MB was partially supported by the National Institute of General Medical Sciences of the National Institutes of Health under award number R01GM111324. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

References

AberOWL Repository. Available at http://aber-owl.net/#/. Accessed August 21, 2022.

An, Y., Borgida, A. & Mylopoulos, J. (2006). Discovering the semantics of relational tables through mappings. Journal on Data Semantics VII, 1–32.

Arabandi, S., Ogbuji, C., Redline, S., Chervin, R., Boero, J., Benca, R., et al. (2010). Developing a sleep domain ontology. In Proc. AMIA Clinical Research Informatics Summit.

Balakirsky, S., Schlenoff, C., Fiorini, S.R., Redfield, S., Barreto, M., Nakawala, H., Carbonera, J.L., Soldatova, L., Bermejo-Alonso, J., Maikore, F., Goncalves, P.J.S., Momi, E.D., Kumar, V.R.S. & Haidegger, T. (2017). Constructing a semantic network for legal content. In Proc. ASME Int’l Manufacturing Science and Engineering Conference, Los Angeles, CA. (Volume 3: Manufacturing Equipment and Systems) (V003T04A049 (10 pages)). doi:10.1115/MSEC2017-2783.

Bandrowski, A., Brinkman, R., Brochhausen, M., Brush, M.H., Bug, B., Chibucos, M.C., et al. (2016). The ontology for biomedical investigations. PLOS ONE, 11(4), 1–19.

Beisswanger, E., Schulz, S., Stenzhorn, H. & Hahn, U. (2008). BioTop: An upper domain ontology for the life sciences – a description of its current structure, contents, and interfaces to OBO ontologies. Applied Ontology, 3(4), 205–212. doi:10.3233/AO-2008-0057.

BioPortal. Available at https://bioportal.bioontology.org/. Accessed August 18, 2022.

Bodenreider, O. & Stevens, R. (2006). Bio-ontologies: Current trends and future directions. Briefings in Bioinformatics, 7(3), 256–274. doi:10.1093/bib/bbl027.

Boeker, M., Schober, D., Raufie, D., Grewe, N., Röhl, J., Jansen, L. & Schulz, S. (2012). Teaching good biomedical ontology design. In Proc. ICBO’12, Graz, Austria (5 pages).

10.

Brochhausen, M., Fransson, M.N., Kanaskar, N.V., Eriksson, M., Merino-Martinez, R., Hall, R.A., Norlin, L., Kjellqvist, S., Hortlund, M., Topaloglu, U., Hogan, W.R. & Litton, J.-E. (2013). Developing a semantically rich ontology for the biobank-administration domain. Journal of Biomedical Semantics, 4(1), 1–9. doi:10.1186/2041-1480-4-44.

11.

Brochhausen, M., Schneider, J., Malone, D., Empey, P.E., Hogan, W.R. & Boyce, R.D. (2014). Towards a foundational representation of potential drug–drug interaction knowledge. In CEUR Workshop Proceedings (Vol. 1309, pp. 16–31).

12.

Brochhausen, M., Spear, A.D., Cocos, C., Weiler, G., Martín, L., Anguita, A., Stenzhorn, H., Daskalaki, E., Schera, F., Schwarz, U. & Sfakianakis, S. (2011). The ACGT master ontology and its applications – towards an ontology-driven cancer research and management system. Journal of Biomedical Informatics, 44(1), 8–25. doi:10.1016/j.jbi.2010.04.008.

13.

Brochhausen, M., Zheng, J., Birtwell, D., Williams, H., Masci, A.M., Ellis, H.J. & Stoeckert, C.J. (2016). OBIB-a novel ontology for biobanking. Journal of Biomedical Semantics, 7(1), 1–9. doi:10.1186/s13326-016-0045-5.

14.

Butt, A.S. (2015). Ontology search: Finding the right ontologies on the web. In Proc. 24th Int’l Conference on World Wide Web, Florence, Italy (pp. 487–491). doi:10.1145/2740908.2741753.

15.

Cancer Chemoprevention Ontology. Available at https://bioportal.bioontology.org/. Accessed August 15, 2022.

16.

Cantador, I., Fernández, M., Castells, P. WebCORE: A Web Application for Collaborative Ontology Reuse and Evaluation. Available at researchgate.net. Accessed November 10, 2020.

17.

Carter, J.S., Brown, S.H., Bauer, B.A., Elkin, P.L., Erlbaum, M.S., Froehling, D.A., Lincoln, M.J., Rosenbloom, S.T., Wahner-Roedler, D.L. & Tuttle, M.S. (2006). Categorical information in pharmaceutical terminologies. In Proc. 2006 AMIA Annual Symposium, Washington, DC (pp. 116–120).

18.

Ceusters, W. (2012). An information artifact ontology perspective on data collections and associated representational artifacts. In

Mantas et al. (Eds.), Proc. MIE, Pisa, Italy (pp. 68–72).

19.

Chemical Entities of Biological Interest (ChEBI). Available at https://bioportal.bioontology.org/. Accessed July 28, 2022.

20.

Courtot, M., Gibson, F., Lister, A.L., Malone, J., Schober, D., Brinkman, R.R. & Ruttenberg, A. (2011). MIREOT: The minimum information to reference an external ontology term. Applied Ontology, 6(1), 23–33. doi:10.3233/AO-2011-0087.

21.

COVID-19 Infectious Disease Ontology. Available at https://bioportal.bioontology.org/. Accessed July 27, 2022.

22.

Cowell, L.G. & Smith, B. (2010). Infectious disease ontology. In

Sintchenko (Ed.), Infectious Disease Informatics (pp. 373–395). New York, NY: Springer. doi:10.1007/978-1-4419-1327-2_19.

23.

Drug Ontology. Available at https://bioportal.bioontology.org/. Accessed July 29, 2022.

24.

Dumontier, M., Baker, C.J.O., Baran, J., Callahan, A., Chepelev, L., Cruz-Toledo, J., Rio, N.R.D., Duck, G., Furlong, L.I., Keath, N., Klassen, D., McCusker, J.P., Queralt-Rosinac, N., Samwald, M., Villanueva-Rosales, N., Wilkinson, M.D. & Hoehndorf, R. (2014). The semanticscience integrated ontology (SIO) for biomedical research and knowledge discovery. Journal of Biomedical Semantics, 5, Article number: 14. doi:10.1186/2041-1480-5-14.

25.

Fernández, M., Cantador, I. & Castells, P. (2006). CORE: A tool for collaborative ontology reuse and evaluation. In Proc. 4th Int’l Workshop on Evaluation of Ontologies for the Web (EON 2006), Edinburgh, Scotland (pp. 23–26).

26.

Gangemi, A., Guarino, N., Masolo, C., Oltramari, A. & Schneider, L. (2002). Sweetening ontologies with DOLCE. In Proc. Int’l Conference on Knowledge Engineering and Knowledge Management, Sigüenza, Spain (pp. 166–181).

27.

Gangemi, A. & Presutti, V. (2009). Ontology design patterns. In Handbook on Ontologies (pp. 221–243). Springer. doi:10.1007/978-3-540-92673-3_10.

28.

Gene Ontology Consortium. Available at http://geneontology.org. Accessed August 21, 2022.

29.

Ghazvinian, A., Noy, N.F. & Musen, M.A. (2009). Creating mappings for ontologies in biomedicine: Simple methods work. In Proc. 2009 AMIA Annual Symposium, San Francisco, CA (p. 198).

30.

Ghazvinian, A., Noy, N.F. & Musen, M.A. (2011). How orthogonal are the OBO foundry ontologies? Journal of Biomedical Semantics, 2(Suppl 2), S2. doi:10.1186/2041-1480-2-S2-S2.

31.

Glimm, B., Horrocks, I., Motik, B., Stoilos, G. & Wang, Z. (2014). HermiT: An OWL 2 reasoner. Journal of Automated Reasoning, 53(3), 245–269. doi:10.1007/s10817-014-9305-1.

32.

Grenon, P., Smith, B. & Goldberg, L. (2004). Biodynamic ontology: Applying BFO in the biomedical domain. In Ontologies in Medicine (pp. 20–38).

33.

Groß, A., Reis, J.C.D., Hartung, M., Pruski, C. & Rahm, E. (2013). Semi-automatic adaptation of mappings between life science ontologies. In Proc. Int’l Conference on Data Integration in the Life Sciences (DILS ’13), Montreal, Canada (pp. 90–104).

34.

Groza, T., Köhler, S., Moldenhauer, D., Vasilevsky, N., Baynam, G., Zemojtel, T., Schriml, L.M., Kibbe, W.A., Schofield, P.N., Beck, T., Vasant, D., Brookes, A.J., Zankl, A., Washington, N.L., Mungall, C.J., Lewis, S.E., Haendel, M.A., Parkinson, H. & Robinson, P.N. (2015). The human phenotype ontology: Semantic unification of common and rare disease. The American Journal of Human Genetics, 97(1), 111–124. doi:10.1016/j.ajhg.2015.05.020.

35.

Gruber, T.R. (1993). A translation approach to portable ontology specifications. Knowledge Acquisition, 5(2), 199–220. doi:10.1006/knac.1993.1008.

36.

Guarino, N., Oberle, D. & Staab, S. (2009). What is an ontology? In Handbook on Ontologies (pp. 1–17). Springer.

37.

Halper, M., Geller, J. & Perl, Y. (1998). An OODB part-whole model: Semantics, notation, and implementation. Data & Knowledge Engineering, 27(1), 59–95. doi:10.1016/S0169-023X(97)00055-4.

38.

Halper, M., Ochs, C., Perl, Y., Arabandi, S. & Musen, M.A. (2018). Quality assurance of ontology content reuse. In

Jaiswal ,

Cooper ,

M.A.

Haendel and

C.J.

Mungall (Eds.), Proc. 9th Int’l Conference on Biomedical Ontology (ICBO 2018), Corvallis, OR (6 pages).

39.

Hanna, J., Chen, C., Crow, W.A., Hall, R., Liu, J., Pendurthi, T., Schmidt, T., Jennings, S.F., Brochhausen, M. & Hogan, W. (2012). Simplifying MIREOT: A MIREOT Protégé plugin. In Proc. 2012 Int’l Semantic Web Conference (Posters & Demos), Boston, MA (pp. 25–28).

40.

He, Y., Yu, H., Ong, E., Wang, Y., Liu, Y., Huffman, A., Huang, H.-H., Beverley, J., Hur, J., Yang, X., Chen, L., Omenn, G.S., Athey, B. & Smith, B. (2020). CIDO, a community-based ontology for coronavirus disease knowledge and data integration, sharing, and analysis. Scientific Data, 7, Article number 181. doi:10.1038/s41597-020-0523-6.

41.

He, Z., Ochs, C., Agrawal, A., Perl, Y., Zeginis, D., Tarabanis, K., Elhanan, G., Halper, M., Noy, N. & Geller, J. (2013b). A family-based framework for supporting quality assurance of biomedical ontologies in BioPortal. In Proc. 2013 AMIA Annual Symposium (pp. 581–590).

42.

He, Z., Ochs, C., Soldatova, L., Perl, Y., Arabandi, S. & Geller, J. (2013a). Auditing redundant import in reuse of a top level ontology for the drug discovery investigations ontology. In Proc. Int’l Workshop on Vaccine and Drug Ontology Studies (VDOS-2013), Montreal, Canada.

43.

Herre, H. (2010). General formal ontology (GFO): A foundational ontology for conceptual modelling. In

Poli ,

Healy and

Kameas (Eds.), Theory and Applications of Ontology: Computer Applications (pp. 297–345). Springer. doi:10.1007/978-90-481-8847-5_14.

44.

Hitzler, P., Gangemi, A. & Janowicz, K. (Eds.) (2016). Ontology Engineering with Ontology Design Patterns: Foundations and Applications (Vol. 25). IOS Press.

45.

Hoehndorf, R., Loebe, F., Poli, R., Herre, H. & Kelso, J. (2008). GFO-Bio: A biological core ontology. Applied Ontology, 3(4), 219–227. doi:10.3233/AO-2008-0055.

46.

Hogan, W.R., Hanna, J., Hicks, A., Amirova, S., Bramblett, B., Diller, M., Enderez, R., Modzelewski, T., Vasconcelos, M. & Delcher, C. (2017). Therapeutic indications and other use-case-driven updates in the drug ontology: Anti-malarials, anti-hypertensives, opioid analgesics, and a large term request. Journal of Biomedical Semantics, 8(1), 1–11. doi:10.1186/s13326-016-0099-4.

47.

Hogan, W.R., Wagner, M.M., Brochhausen, M., Levander, J., Brown, S.T., Millett, N., DePasse, J. & Hanna, J. (2016). The apollo structured vocabulary: An OWL2 ontology of phenomena in infectious disease epidemiology and population biology for use in epidemic simulation. Journal of Biomedical Semantics, 7(1), 1–12. doi:10.1186/s13326-016-0098-5.

48.

Horridge, M. (2015). TONES ontology repository (corpus). doi:10.5281/zenodo.32717.

49.

Jackson, R.C., Balhoff, J.P., Douglass, E., Harris, N.L., Mungall, C.J. & Overton, J.A. (2019). ROBOT: A tool for automating ontology workflows. BMC Bioinformatics, 20, 407. doi:10.1186/s12859-019-3002-3.

50.

Jansen, L. & Schulz, S. (2011). The ten commandments of ontological engineering. In

Herre ,

Hoehndorf and

Loebe (Eds.), Proc. 3rd Workshop of Ontologies in Biomedicine and Life Sciences, Berlin, Germany.

51.

Jupp, S., Burdett, T., Leroy, C. & Parkinson, H.E. (2015). A new ontology lookup service at EMBL-EBI. In Proc. SWAT4LS Int’l Conference, Cambridge, England (pp. 118–119).

52.

Kamdar, M.R., Tudorache, T. & Musen, M.A. (2017). A systematic analysis of term reuse and term overlap across biomedical ontologies. Semantic Web Journal, 8(6), 853–871. doi:10.3233/SW-160238.

53.

Keet, C.M. (2011). The use of foundational ontologies in ontology development: An empirical assessment. In Proc. Extended Semantic Web Conference, Heraklion, Greece (pp. 321–335).

54.

Keet, C.M. & Grütter, R. (2021). Toward a systematic conflict resolution framework for ontologies. Journal of Biomedical Semantics, 12(1), 1–15. doi:10.1186/s13326-021-00246-0.

55.

Kendall, E. & McGuinness, D.L. (2019). Ontology Engineering. Morgan & Claypool.

56.

Khan, Z. & Keet, C.M. (2012). ONSET: Automated foundational ontology selection and explanation. In Proc. 18th International Conference on Knowledge Engineering and Knowledge Management, Galway, Ireland (pp. 237–251). doi:10.1007/978-3-642-33876-2_22.

57.

Kostovska, A., Tolovski, I., Maikore, F., The Alzheimer’s Disease Neuroimaging Initiative, Soldatova, L.N. & Panov, P. (2019). Neurodegenerative disease data ontology. In

P.K.

Novak ,

Šmuc and

Džeroski (Eds.), International Conference on Discovery Science, Split, Croatia (pp. 235–245).

58.

Kumar, A., Smith, B. & Novotny, D.D. (2004). Biomedical informatics and granularity. International Journal of Genomics, 5(6–7), 501–508.

59.

Le Clair, A., Marinache, A., Ghalayini, H.E., Maccaull, W. & Khedri, R. (2022). A review on ontology modularization techniques – a multi-dimensional perspective. IEEE Transactions on Knowledge and Data Engineering. doi:10.1109/TKDE.2022.3152928.

60.

Lin, Y., Xiang, Z. & He, Y. (2011). Brucellosis ontology (IDOBRU) as an extension of the infectious disease ontology. Journal of Biomedical Semantics, 2, Article number: 9. doi:10.1186/2041-1480-2-9.

61.

Maguire, E., González-Beltrán, A., Whetzel, P.L., Sansone, S.-A. & Rocca-Serra, P. (2013). OntoMaton: A bioportal powered ontology widget for Google spreadsheets. Bioinformatics, 29(4), 525–527. doi:10.1093/bioinformatics/bts718.

62.

Maikore, F.S., Selenge, G., Olayinka, A., Abbott, P. & Soldatova, L. (2017). An ontology for clinical laboratory standard operating procedures. In Proc. Joint Ontology Workshops (JOWO 2017), Bozen-Bolzano, Italy.

63.

Mosquito Gross Anatomy Ontology. Available at https://bioportal.bioontology.org/. Accessed July 29, 2022.

64.

Mungall, C.J., Torniai, C., Gkoutos, G.V., Lewis, S.E. & Haendel, M.A. (2012). Uberon, an integrative multi-species anatomy ontology. Genome Biology, 13, R5. doi:10.1186/gb-2012-13-1-r5.

65.

Niles, I. & Pease, A. (2001). Towards a standard upper ontology. In Proc. FOIS ’01, Ogunquit, ME.

66.

Nirenburg, S. & Raskin, V. (2001). Ontological semantics, formal ontology, and ambiguity. In Proc. FOIS ’01, Ogunquit, ME (pp. 151–161). doi:10.1145/505168.505183.

67.

Noy, N.F., Fergerson, R.W. & Musen, M.A. (2000). The knowledge model of Protégé-2000: Combining interoperability and flexibility. In

Dieng and

Corby (Eds.), Proc. Int’l Conference on Knowledge Engineering and Knowledge Management, Juan-les-Pins, France (pp. 17–32).

68.

OBO Foundry. Available at http://www.obofoundry.org/. Accessed July 28, 2022.

69.

Ochs, C., He, Z., Perl, Y., Arabandi, S., Halper, M. & Geller, J. (2013). Refining the granularity of abstraction networks for the sleep domain ontology. In Proc. Fourth Int’l Conference on Biomedical Ontology (ICBO 2013), Montreal, Canada (pp. 84–89).

70.

Ochs, C., Perl, Y., Geller, J., Arabandi, S., Tudorache, T. & Musen, M.A. (2017). An empirical analysis of ontology reuse in BioPortal. Journal of Biomedical Informatics, 71, 165–177. doi:10.1016/j.jbi.2017.05.021.

71.

OGMS – Ontology for General Medical Science. Available at https://code.google.com/archive/p/ogms. Accessed August 19, 2022.

72.

Ontobee. Available at http://www.ontobee.org/. Accessed August 15, 2022.

73.

Ontology Design Patterns . org (ODP). Available at http://ontologydesignpatterns.org/wiki/Main_Page. Accessed August 5, 2021.

74.

Ontology for ELEctronic tool for Clinicians, Trainers and Researchers In Child Abuse. Available at https://bioportal.bioontology.org/. Accessed August 14, 2022.

75.

Ontology of Document Acts. Available at Ontobee (2022). Accessed August 15, 2022.

76.

Ontology of Medically Related Social Entities. Available at https://bioportal.bioontology.org/. Accessed July 27, 2022.

77.

Ontology of Organizational Structures of Trauma Centers and Trauma Systems (OOSTT). Available at https://bioportal.bioontology.org/. Accessed July 29, 2021.

78.

Open Ontology Repository (OOR) Initiative. Available at https://ontologforum.org/index.php/OpenOntologyRepository. Accessed July 29, 2022.

79.

Osumi-Sutherland, D., Courtot, M., Balhoff, J.P. & Mungall, C. (2017). Dead simple OWL design patterns. Journal of Biomedical Semantics, 8, Article number: 18. doi:10.1186/s13326-017-0126-0.

80.

OWL Web Ontology Language Reference. Available at https://www.w3.org/TR/owl-ref. Accessed July 27, 2022.

81.

Panov, P., Džeroski, S. & Soldatova, L.N. (2008). OntoDM: An ontology of data mining. In Proc. 2008 IEEE Int’l Conference on Data Mining Workshops, Pisa, Italy (pp. 752–760). doi:10.1109/ICDMW.2008.62.

82.

Panov, P., Soldatova, L.N. & Džeroski, S. (2015). Representing bioinformatics datatypes using the OntoDT ontology. In

F.M.

Couto and

Hastings (Eds.), Proc. ICBO 2015, Lisbon, Portugal (2 pages).

83.

Phenotypic Quality Ontology (PATO). Available at https://bioportal.bioontology.org/. Accessed August 21, 2022.

84.

Prestes, E., Carbonera, J.L., Fiorini, S.R., Jorge, V.A.M., Abel, M., Madhavan, R., et al. (2013). Towards a core ontology for robotics and automation. Robotics and Autonomous Systems, 61(11), 1193–1204. doi:10.1016/j.robot.2013.04.005.

85.

Proper Name Ontology. Available at http://purl.obolibrary.org/obo/iao/pno.owl. Accessed August 15, 2022.

86.

Protégé. Available at https://protege.stanford.edu. Accessed July 27, 2022.

87.

Qi, D., King, R.D., Hopkins, A.L., Bickerton, G.R. & Soldatova, L.N. (2010). An ontology for description of drug discovery investigations. Journal of Integrative Bioinformatics, 7(3).

88.

Rosse, C. & Mejino, J.L.V. (2003). A reference ontology for biomedical informatics: The foundational model of anatomy. Journal of Biomedical Informatics, 36(6), 478–500. doi:10.1016/j.jbi.2003.11.007.

89.

Rubin, D.L., Shah, N.H. & Noy, N.F. (2008). Biomedical ontologies: A functional perspective. Briefings in Bioinformatics, 9(1), 75–90. doi:10.1093/bib/bbm059.

90.

Schulz, S. & Cornet, R. (2009). SNOMED CT’s ontological commitment. In Proc. ICBO’09, Buffalo, NY (pp. 55–58).

91.

Sim, I., Tu, S.W., Carini, S., Lehmann, H.P., Pollock, B.H., Peleg, M. & Wittkowski, K.M. (2014). The ontology of clinical research (OCRe): An informatics foundation for the science of clinical research. Journal of Biomedical Informatics, 52, 78–91. doi:10.1016/j.jbi.2013.11.002.

92.

Singh, A. & Anand, P. (2013). State of art in ontology development tools. Int’l. Journal of Advances in Computer Science and Technology, 2(7), 96–101.

93.

Sioutos, N., de Coronado, S., Haber, M.W., Hartel, F.W., Shaiu, W.-L. & Wright, L.W. (2007). NCI thesaurus: A semantic model integrating cancer-related clinical and molecular information. Journal of Biomedical Informatics, 40(1), 30–43. doi:10.1016/j.jbi.2006.02.013.

94.

Smith, B., Ashburner, M., Rosse, C., Bard, J., Bug, W., Ceusters, W., et al. (2007). The OBO foundry: Coordinated evolution of ontologies to support biomedical data integration. Nature Biotechnology, 25, 1251–1255. doi:10.1038/nbt1346.

95.

Smith, B., Ceusters, W., Klagges, B., Köhler, J., Kumar, A., Lomax, J., Mungall, C., Neuhaus, F., Rector, A.L. & Rosse, C. (2005). Relations in biomedical ontologies. Genome Biology, 6, R46.

96.

SNOMED International. Available at https://www.snomed.org/. Accessed August 15, 2022.

97.

Soldatova, L.N., Aubrey, W., King, R.D. & Clare, A. (2008). The EXACT description of biomedical protocols. Bioinformatics, 24(13), i295–i303. doi:10.1093/bioinformatics/btn156.

98.

Soldatova, L.N. & King, R.D. (2006). An ontology of scientific experiments. Journal of the Royal Society Interface, 3(11), 795–803. doi:10.1098/rsif.2006.0134.

99.

Soldatova, L.N., Nadis, D., King, R.D., Basu, P.S., Haddi, E., Baumlé, V., Saunders, N., Marwan, W. & Rudkin, B. (2014). EXACT2: The semantics of biomedical protocols. BMC Bioinformatics, 15(Suppl 14), S5.

100.

Suárez-Figueroa, M.C. (2010). NeOn methodology for building ontology networks: Specification, scheduling and reuse. PhD thesis, Informatica.

101.

Suárez-Figueroa, M.C., Gómez-Pérez, A. & Fernández-López, M. (2012). The NeOn methodology for ontology engineering. In Ontology Engineering in a Networked World (pp. 9–34). Springer.

102.

Suárez-Figueroa, M.C., Gómez-Pérez, A. & Fernández-López, M. (2015). The NeOn methodology framework: A scenario-based methodology for ontology development. Applied Ontology, 10(2), 107–145. doi:10.3233/AO-150145.

103.

Sure, Y., Erdmann, M., Angele, J., Staab, S., Studer, R. & Wenke, D. (2002). OntoEdit: Collaborative ontology development for the semantic web. In Proc. 1st Int’l Semantic Web Conference, Sardinia, Italy (pp. 221–235).

104.

Tools for Understanding Relationships Between Behaviours using Ontology (TURBBO). Available at https://github.com/fatibaba/turbbo. Accessed October 7, 2022.

105.

Topalis, P., Mitraka, E., Dritsou, V., Dialynas, E. & Louis, C. (2013). IDOMAL: The malaria ontology revisited. Journal of Biomedical Semantics, 4, Article number: 16. doi:10.1186/2041-1480-4-16.

106.

Torniai, C., Brush, M.H., Vasilevsky, N., Segerdell, E., Wilson, M., Johnson, T., Corday, K., Shaffer, C. & Haendel, M. (2011). Developing an application ontology for biomedical resource annotation and retrieval: Challenges and lessons learned. In Proc. ICBO 2011, Buffalo, NY (pp. 101–108).

107.

Vandenbussche, P.Y., Atemezing, G.A., Poveda-Villalón, M. & Vatant, B. (2017). Linked open vocabularies (LOV): A gateway to reusable semantic vocabularies on the web. Semantic Web, 8(3), 437–452. doi:10.3233/SW-160213.

108.

Vanschoren, J. & Soldatova, L.N. (2010). Exposé: An ontology for data mining experiments. In Proc. Int’l Workshop on Third Generation Data Mining: Towards Service-Oriented Knowledge Discovery (SoKD-2010), Barcelona, Spain (pp. 31–46).

109.

Virus Infectious Disease Ontology. Available at https://bioportal.bioontology.org/. Accessed July 27, 2022.

110.

VIVO-Integrated Semantic Framework. Available at https://bioportal.bioontology.org/. Accessed July 29, 2022.

111.

Whetzel, P.L., Noy, N.F., Shah, N.H., Alexander, P.R., Nyulas, C., Tudorache, T., et al. (2011). BioPortal: Enhanced functionality via new web services from the national center for biomedical ontology to access and use ontologies in software applications. Nucleic Acids Research, 39(Web server issue), W541–W545. doi:10.1093/nar/gkr469.

112.

Wilkinson, M.D., Dumontier, M., Aalbersberg, I.J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L.B., Bourne, P.E., Bouwman, J., et al. (2016). The FAIR guiding principles for scientific data management and stewardship. Scientific Data, 3(1), 1–9.

113.

Williams, K., Bilsland, E., Sparkes, A., Aubrey, W., Young, M., Soldatova, L.N., Grave, K.D., Ramon, J., de Clare, M., Sirawaraporn, W., Oliver, S.G. & King, R.D. (2015). Cheaper faster drug development validated by the repositioning of drugs against neglected tropical diseases. Journal of the Royal Society Interface, 12(104). doi:10.1098/rsif.2014.1289.

114.

Xiang, Z., Courtot, M., Brinkman, R.R., Ruttenberg, A. & He, Y. (2010). OntoFox: Web-based support for ontology reuse. BMC Research Notes, 3, Article number: 175. doi:10.1186/1756-0500-3-175.

115.

Xiang, Z., Mungall, C., Ruttenberg, A. & He, Y. (2011). Ontobee: A linked data server and browser for ontology terms. In Proc. ICBO 2011, Buffalo, NY (pp. 279–281).

116.

Zagorulko, Y.A. & Borovikova, O.I. (2020). Using a system of heterogeneous ontology design patterns to develop ontologies of scientific subject domains. Programming and Computer Software, 46(4), 273–280. doi:10.1134/S0361768820040064.