Abstract
This paper describes a scenario-based methodology called the NeOn Methodology framework. The aim of this framework is to speed up the construction of ontologies and ontology networks by reusing available knowledge resources (ontologies, non-ontological resources and ontology design patterns). The methodology is founded on four pillars: (1) a glossary of processes and activities; (2) a set of nine scenarios for building ontologies and ontology networks; (3) two ontology life-cycle models; and (4) a set of prescriptive methodological guidelines for performing specific processes and activities. The paper also shows how to apply this methodology framework to develop an ontology network within the multimedia domain. Finally, it presents an empirical evaluation of the NeOn Methodology framework.
Introduction
Managing information and knowledge in organizations is a crucial issue. The main resources of the systems and applications used by organizations today are vast quantities of data and knowledge in different types of distributed formats. These data are normally structured, but some are very heterogeneous and, hence, not necessarily interoperable.
Nowadays, ontologies are used for making information explicit and allowing it to be shared. A good solution for improving data and knowledge management would be to create an ontology-based application that could properly manage all the organization’s knowledge. Such a process would involve modeling this knowledge in an ontology.
The emergence of ontology development methodologies over the last few decades has transformed the art of building ontologies into an engineering activity. Several researchers such as Gómez-Pérez et al. (2003), Stadlhofer et al. (2013) and Iqbal et al. (2013) have provided detailed surveys and comparative studies of these methodologies. METHONTOLOGY (Fernández-López et al., 1997, 1999) and On-To-Knowledge (Staab et al., 2001) are considered to be the most complete methodologies for building single ontologies from scratch. Worthy of note is the ontology-building method described by Noy and McGuinness (2001), which reports and shows how to avoid some common ontology development errors. Several methodologies for collaborative and distributed ontology development, such as DILIGENT (Pinto et al., 2004), and tools, such as WebProtégé (Tudorache et al., 2013), have recently appeared (Simperl & Luczak-Rösch, 2014). WebProtégé is actually a lightweight ontology editor for the Web that provides collaboration features that are integrated into its implicit ontology development methodology. The above methodologies include high-level guidelines for single ontology construction but do not provide sufficient details about the methods and techniques that they employ. The guidelines normally: (a) cover all issues from specification through to implementation, but only a few provide unspecific guidelines on ontology reuse; and (b) mainly target ontology researchers.
On the other hand, several initiatives in the ontology development field have revealed ad-hoc alternatives for building ontologies, most of which, however, are not included in the above mentioned methodologies. Some examples are the Knowledge Web1
These new ways of developing ontologies have led to a very large number of ontologies being embedded in ontology networks and developed by reusing and reengineering knowledge resources. The resulting ontology networks (Haase et al., 2006) are defined as collections of ontologies related by means of a variety of different meta-relationships, such as mapping, modularization, version and dependency.6
Also noteworthy in relation to the ontology networks addressed in this paper is the notion of ‘hyperontologies’ (heterogeneously structured ontologies, which offer distributed networks of ontologies written in different formalisms) (Kutz et al., 2010).
Despite the wide range of proposed ontology development methods and methodologies, none, as far as we know, supports all the new trends currently emerging in ontology network development. These trends comprehend complex scenarios requiring the reuse and reengineering of knowledge resources (ontologies, non-ontological resources and ontology design patterns) and involving software developers and ontology practitioners. The above methodologies do not provide engineers with guidelines to take them through the construction of a product that is now much more complex than it was several years ago. Even though some contributions (see, e.g., Francesconi et al., 2010; Barresi et al., 2008) take into consideration ontology reuse, transformation and integration, they focus on a particular scenario. However, if engineers opt to follow different methodologies, they may find that such methodologies contain descriptions of similar processes and activities that bear different names. This will lead to a heavier workload and, possibly, misunderstandings.
On this ground, we present the NeOn Methodology framework, a scenario-based methodology that provides accurate details for developing ontologies and ontology networks7
Meta-relationships such as mappings, modularization, version and dependency are central issues in ontology networks. Given that this paper focuses on the methodology framework rather than on the details of its activities, these relations are not described thoroughly here. Interested readers are referred to (Suárez-Figueroa et al., 2012), which contains chapters specifically addressing matching, modularization and evolution, etc.
The framework presented in this paper together with the methods and techniques that it includes (which are merely outlined here for reasons of space) are the result of abstracting our ontology network development experience as part of complex projects mostly developed with partners from several countries.
The rest of the paper is organized as follows. Section 2 describes the NeOn Methodology framework. Section 3 shows how to apply the methodological framework to develop an ontology network of multimedia issues. Section 4 reports the empirical evaluation of the NeOn Methodology framework. Finally, Section 5 discusses the conclusions and future work.
The NeOn Methodology framework (Suárez-Figueroa, 2010) is a scenario-based methodology that provides accurate details about key aspects of the ontology engineering process, paying special attention to the reuse and reengineering of ontological and non-ontological resources. This framework is founded on four pillars: (1) a glossary of processes and activities; (2) a set of nine scenarios for building ontologies and ontology networks; (3) two modes of organizing ontology developments, called ontology life-cycle models; and (4) a set of precise methodological guidelines for performing specific processes and activities.
NeOn glossary of processes and activities
Ontology engineering is an evolving discipline: new ontology building approaches, such as creating ontology networks and reusing any available knowledge resource, appear all the time, and new applications based on semantic issues, such as the Linked Data initiative, are also emerging. Because of this constant evolution, however, the terminology used by ontology practitioners is sometimes fuzzy. For instance, the difference between ontology modification and ontology update is not absolutely clear, which may cause misunderstandings. Whereas other activities have multiple natural language definitions (e.g., ontology merging), which may also obscure their meaning.
As established in Bloom’s taxonomy (Bloom, 1956), the lowest level of knowledge in any field is its terminology, and no proper knowledge transmission is possible without it. Therefore, the terminology requires clarification so that the people using the NeOn Methodology to build ontologies can communicate with no trouble.
For this reason, we have created a glossary of ontology engineering terms, called the NeOn glossary, as part of the NeOn Methodology framework. Its main purpose is to unify and specify the ontology engineering terminology.
The terms included in the glossary refer to the processes and activities involved in ontology development, where activity is an action to be performed including its required input and output information and process is a group or set of ordered activities.
The NeOn glossary8
NeOn glossary of processes and activities. (Colors are visible in the online version of the article;
Figure 1 shows the whole NeOn glossary of processes and activities. The meanings of the different arrow types are as follows: (1) arrows with dashed lines denote a process that is divided into activities; (2) arrows with solid lines denote type-of relationships; and (3) arrows with dotted lines denote synonymy. Some examples of terms defined in the glossary are: ontology enrichment (which refers to the activity of adding new conceptual structures, e.g., concepts, roles and axioms to an ontology); ontology evaluation (which refers to the activity of checking the technical quality of an ontology against a frame of reference); and ontology partitioning (which refers to the activity of dividing an ontology into a set (not necessarily disjoint) of modules which together form an ontology, but which can be treated separately).
Based on experience gained from our involvement in different types of projects, we identified three different ontology-building situations: (1) single ontologies; (2) sets of interconnected single ontologies; and (3) ontology networks. The first case refers to ontologies that have no relationship whatsoever (domain dependent or independent) with other ontologies. The second situation denotes a set of ontologies linked by some kind of domain-dependent relationship. The last instance refers to a group of interconnected single ontologies that are linked to each other by a variety of different meta-relationships or domain-independent relationships (based on Haase et al., 2006). Therefore, it can be said that when software developers and ontology practitioners explicitly define meta-relationships such as mapping, modularization, version and dependency between a set of interconnected ontologies and/or between an ontology and its components, they are developing an ontology network.
In this context, we have identified a set of nine flexible scenarios for building ontologies and ontology networks, placing special emphasis on knowledge resource reuse and reengineering. The set of the nine most widespread scenarios identified in Fig. 2 provides information on the different ontology development pathways. Figure 2 shows the directed lines with their associated numbered circles that represent the different scenarios. Scenarios are sets of ordered processes and activities defined in the NeOn glossary (Section 2.1). Processes are represented by rounded boxes with a solid black border, whereas activities are represented by circles with a solid black border. The possible inputs (knowledge resources available for reuse) and the possible outputs (ontologies, ontology networks and alignments) of the execution of some of the scenarios are represented as boxes with a dotted border.
Ontology network-building scenarios. (Colors are visible in the online version of the article;
There follows a non-exhaustive summary of the nine scenarios.
Scenario 1: From specification to implementation. This scenario refers to the development of ontologies from scratch. It consists of the core activities to be performed in any ontology development and should be combined with the other scenarios. In this scenario, ontology developers should first specify the requirements to be met by the ontology by means of the ontology requirements specification activity.9
Note that the processes and activities defined in the NeOn glossary are shown in italics in the description of scenarios.
Scenario 2: Reusing and reengineering non-ontological resources (NORs). A non-ontological resource is defined as a knowledge resource (glossary, dictionary, lexicon, classification scheme, taxonomy and thesaurus) whose semantics has not been formalized by an ontology. This scenario is applicable when ontology developers have access to NORs that, to some extent, cover the domain of the ontology network being built and that normally reflect some degree of community consensus; these resources can be used to develop the new ontology. Ontology developers should perform the non-ontological resource reuse process. This entails: (1) searching highly reliable Web sites, domain-related sites, and resources within organizations for NORs, using the terms included in the ORSD as input; (2) assessing candidate NORs using three criteria (coverage, precision and consensus); (3) selecting the most suitable NORs to be used to build the ontology network. After this selection, developers must carry out the non-ontological resource reengineering process to transform the selected NORs into ontologies. This process is divided into the following activities (Villazón-Terrazas et al., 2010): (1) non-ontological resource reverse engineering, whose goal is to analyze the selected NORs and identify their underlying components and then create representations of the resources at different levels of abstraction (design, requirements and conceptual); (2) non-ontological resource transformation, whose goal is to generate a conceptual model from each selected NOR (a transformation based on the use of the so-called patterns for reengineering non-ontological resources (PR-NOR) (Villazón-Terrazas et al., 2010)); and (3) ontology forward engineering, whose goal is to output a new implementation of the ontologies on the basis of the new conceptual models identified in the NOR transformation activity. Ontology developers should then use the resultant ontologies as input for Scenario 1. This approach can both speed up ontology development, thereby cutting development costs, and deal with agreed-upon knowledge.
If the ultimate aim is to develop an ontology network, developers should explicitly define meta-relationships, such as useImports or isReusing, in order to denote that the final ontology is importing or reusing other ontologies (these ontologies are, in this case, the result of a NOR reengineering process).
Scenario 3: Reusing ontological resources. This scenario is applicable when developers have access to ontological resources that cover, to some extent, the domain of the ontology network being built and can be reused to develop the new ontology. If the ultimate aim is to build an ontology network, then developers should explicitly define meta-relationships (e.g., useImports or isReusing) in order to model such reuse.
There are different ways of reusing ontological resources (Suárez-Figueroa, 2010). For example, ontological resources can be used as a whole, as a part or module,10
A module (d’Aquin et al., 2007) is the part of the ontology that defines the relevant set of terms for a particular purpose.
An ontology statement or triple contains the following three components: subject, predicate and object.
Scenario 4: Reusing and reengineering ontological resources. This scenario applies in cases where the outcomes of Scenario 3 are not entirely usable as they are, and the ontological resources therefore have to be reengineered to serve the intended purpose. The processes to be performed in this scenario are as follows: ontology developers should carry out first the ontological resource reuse process14
In this case, meta-relationships such as useImports or isReusing should be defined if the objective is to develop an ontology network.
The ontological resource reengineering process proposed here was inspired by the software reengineering process (Byrne, 1992).
Scenario 5: Reusing and merging ontological resources. This scenario applies in cases where: (a) several ontological resources from the same domain can be selected for reuse (Scenario 3); and (b) ontology developers need to create a new ontological resource by aligning and merging two or more, possibly overlapping, ontological resources. Alternatively, the ontology developer may only need to establish mappings or alignments among the selected ontological resources in order to create the ontology network. Ontology developers should then use the outcomes as input for Scenario 1.
If the ultimate aim is to build an ontology network, several types of meta-relationships should be defined in this scenario, including domain-independent relationships, such as useImports or isReusing, which represent the reuse of ontological resources, and meta-relationships such as hasMapping, which represent the alignments among resources.
Scenario 6: Reusing, merging and reengineering ontological resources. This scenario applies in cases where the result of Scenario 5 (a merged ontological resource) is not usable as it is (e.g., we have an ontology in F-logic with a taxonomy built according to a particular criterion A, whereas the new ontology should be in OWL with a taxonomy built according to criterion B). In such a case, the merged ontological resource should be reengineered (as explained in Scenario 4) to serve the intended purpose. The resultant ontological resource should be used as input for Scenario 1.
As in Scenario 5, when the objective is to build an ontology network, developers should include domain-independent relations such as useImports or isReusing and hasMapping to represent reuse and alignment, respectively.
Scenario 7: Reusing ontology design patterns (ODPs). Ontology design patterns (Gangemi, 2005) represent modeling solutions to some well-known ontology modeling problems. The ODP reuse process is divided into the following activities: (1) search for possible ODPs for reuse in a repository or catalogue (examples of repositories are the Ontology Design Pattern Portal,15
Note that, when the ultimate aim is to build an ontology network, developers should explicitly define meta-relationships such as useImports or isReusing in order to represent pattern reuse, since an ontology design pattern can be considered an ontology.
Scenario 8: Restructuring ontological resources. This scenario applies in cases where the set of ontological resources (output by different scenarios) to be integrated in the new ontology needs to be reorganized. Restructuring involves activities that can be combined in any manner and order. The activities are as follows: (1) modularize the ontology to facilitate reuse of the knowledge included in the network;18
In this case, meta-relationships such as containsModules should be explicitly defined.
In this case, meta-relationships such as isExtension should be explicitly defined when the aim is to build an ontology network.
In this case, meta-relationships such as isSpecialization should be explicitly defined when the aim is to build an ontology network.
Scenario 9: Localizing ontological resources. This scenario applies in cases where the terms of the ontology network have to be written in different natural languages. Developers should perform the ontology localization activity once the ontology is almost finished and ready for use. This activity requires the translation of all the ontology terms, usually in English, into a natural language (Spanish, French and German, among others) other than the language used in the conceptualization. One key task in this activity is to select the most appropriate linguistic assets (multilingual thesauri and electronic dictionaries, for example, EuroWordNet21
Note that these scenarios can be flexibly combined in different ways, and any combination of scenarios should include Scenario 1, which is composed of the core ontology development activities. In fact, as Fig. 2 shows, the results of all the other scenarios should be integrated into the results output by Scenario 1. Support activities,22
Definitions for the support activities mentioned here are available in the NeOn glossary (available at
The goal of this activity is to acquire knowledge from experts of a given domain or through some kind of (semi)automatic process, which is called ontology learning. Gómez-Pérez and Manzano-Macho (2004) presented a survey of ontology learning techniques.
This activity is related to ontology evolution. A summary of the guidelines for this activity is presented in the Appendix. Palma et al. (2012) presented detailed guidelines for evolving ontologies.
An excerpt of the guidelines for this activity is presented in the Appendix. Sabou and Fernandez (2012) presented detailed guidelines for evaluating ontologies.
Processes and activities and their related scenarios
Finally, as already mentioned, scenarios are sets of the ordered processes and activities defined in the NeOn glossary. Table 1 matches the set of scenarios to the processes and activities defined in the glossary. There are two options: (1) a process or an activity can be carried out as part of a particular scenario; or (2) a process or an activity can be carried out as part of more than one scenario.
Software engineering acknowledges that no one standard life-cycle model is valid for all software development projects and whether or not a life-cycle model is appropriate for a specific project depends on several features (Taylor, 2008). Bearing in mind the analogy between software engineering and ontology engineering, it is not very realistic to propose a single life-cycle model for all ontology network developments. For this reason, Suárez-Figueroa and Gómez-Pérez (2008) proposed a preliminary collection of ontology network life-cycle models, where an ontology network life-cycle model describes how to organize an ontology network development project. The collection includes the following five models: the waterfall model, the incremental model, the iterative model, the evolutionary prototyping model and the spiral model. All the models were designed based on the life-cycle models described and used in software engineering. The specific features of ontology network development were taken into account, assuming a rather centralized and controlled ontology engineering setting, disregarding approaches such as mining ontologies from tags.
Dzbor et al. (2009) presented preliminary evaluations of the use of the above models. Such evaluations showed that ontology developers had some trouble distinguishing among incremental, iterative, evolving prototyping and spiral ontology network life-cycle models. Therefore, we decided to modify the preliminary collection of ontology network life-cycle models and commit to only two models: the waterfall model and the iterative-incremental model. This decision was based on three factors. First, the preliminary results of the evaluation showed that ontology developers found it hard to differentiate between the different iterative (incremental, iterative, evolutionary prototyping and spiral) life-cycle models. Second, Larman (2002) suggested two different models: waterfall and iterative-incremental life-cycle models in software engineering. Third, thanks to the experience that we gained by developing ontologies as part of different research projects (Esperonto, Knowledge Web, SEEMP, among others), we found that the main models used in practice were based on notions of waterfall and iterative modeling. The NeOn Methodology framework adopts these two models, which have been adapted to the specific features of ontology network development from software engineering models (Sommerville, 2007).
Waterfall life-cycle model family
In an ontology development project, this family of models represents the stages as sequential phases, where each stage must be completed before the next stage begins and backtracking is only permitted in the maintenance phase to fix errors or include missing knowledge. This model includes a set of support activities (e.g., knowledge acquisition, content evaluation) that must be performed in all phases.
This model is applicable in the following situations:
Short ontology projects (e.g., 2 months);
Ontology projects whose goal is to develop an existing ontology in a different formalism or language;
Ontology projects with closed requirements, e.g., for implementing an ontology based on an ISO standard or previously agreed-upon resources;
Ontology projects where ontologies are to cover a small well-understood domain.
Due to the importance of reusing and reengineering knowledge resources and merging ontological resources, we have defined five different versions of the waterfall model as follows: the four-phase waterfall model, the five-phase waterfall model, the five-phase + merging phase waterfall model, the six-phase waterfall model and the six-phase + merging phase waterfall model. Figure 3(a) shows a diagram of these versions. The diagram uses a graphical variation of BNF notation. In Fig. 3(a), the rounded boxes with a solid black border represent the phases of the basic version of the model; the rounded boxes with a dotted black border represent optional phases depending on the model version; the square brackets represent model version-dependent composition options; and, finally, the dashed directed line denotes optional backtracking (in the maintenance phase) in order to fix errors or add missing knowledge.
Relationship between scenarios and waterfall life-cycle model versions
Relationship between scenarios and waterfall life-cycle model versions
Ontology network life-cycle models proposed in the NeOn Methodology framework. (a) Waterfall model family. (b) Iterative-incremental model.
This model, shown in Fig. 3(b), organizes the ontology development as a set of iterations (or short mini-projects with a fixed duration). The braces in Fig. 3(b) denote repetition, and the numbers (2 and N) indicate the number of iterations performed. Each iteration is scheduled as a single ontology project developed according to any of the versions of the waterfall model presented in Section 2.3.1. In other words, the iterative-incremental model is basically composed of a set of iterations developed according to any version of the waterfall ontology network life-cycle model. Thus, the relationship between scenarios and the iterative-incremental model depends on the different versions of the waterfall model used in the iterative-incremental model. Therefore, the relationships presented in Table 2 are equally valid for this model.
This model is applicable in the following scenarios:
Ontology projects involving large groups of developers with different profiles and roles;
Ontology projects in which development involves several different poorly understood domains;
Ontology projects where requirements are partially unknown or can change during ontology development.
The requirements specified in the ORSD can be divided into different subsets, and each subset can be covered in different iterations. The result of any iteration is a functional and partial ontology that satisfies a subset of the requirements. A partial ontology can be used, evaluated and integrated into any other ontology network.
The model is based on adding to and successively improving the ontology network by performing multiple iterations with cyclic feedback and adaptation.
Prescriptive methodological guidelines for processes and activities
The set of prescriptive methodological guidelines provides information on how to perform a particular process or a particular activity defined in the NeOn glossary. The guidelines include guiding cards, a workflow, and some examples explaining how to use the guidelines.
Guiding card for ontology requirements specification
Guiding card for ontology requirements specification
The graphical workflow shows how the process or the activity should be performed. The workflow contains the inputs, outputs, actors and details (methods, techniques and tools) prescribing how to perform the process or activity. Figure 4 illustrates the workflow for the ontology requirements specification activity.
The NeOn Methodology framework includes detailed methodological guidelines for performing both the different processes and activities involved in Scenarios 1, 2, 3, 5, 7, 8 and 9 and the support activities such as ontology evaluation and evolution.26
During this activity, developers can explicitly include meta-relationships such as hasPriorVersion to indicate that a new version of an ontology is being developed. In doing so, developers are building an ontology network.
Processes and activities with methodological guidelines
Workflow for ontology requirements specification.
We now describe how to apply the NeOn Methodology framework to a real use case for building a user-context ontology network (usercONText). This ontology network was developed as part of the mIO!27
To develop usercONText, the ontology development team (composed of people with a range of skills from the different project partner institutions) first performed the ontology requirements specification activity. This activity was carried out according to the guidelines shown in Fig. 4. As a result, the team output the ORSD, which contains more than 300 functional requirements. The requirements include the following terms: device, camera, interface, sensor, environment, user, source, localization, building, provider, service, network, access, interval, time, preference and role, among others.
The ontology development team used the glossary of terms included in the ORSD to run an exploratory search for available knowledge resources that could be reused to develop usercONText. As a result, the team collected a variety of ontologies (i.e., OWL-Time,28
Workflow for scheduling.
After the search, the team used gOntt to schedule the ontology development project (Suárez-Figueroa, 2010; Suárez-Figueroa et al., 2012). gOntt is a tool mainly used for scheduling ontology development projects. It automatically generates the initial ontology development plan in the form of a Gantt chart. In addition, it also provides ontology developers with support during the execution of the ontology development project. gOntt is founded on the methodological guidelines for scheduling projects shown in Fig. 5. These guidelines include the following tasks:
Select the ontology network life-cycle model. To help the team decide which the most appropriate life-cycle model is, gOntt asks a simple natural language question, “Are the ontology requirements assumed to be fully known at the beginning of the ontology network development?”. In the particular use case presented here, the requirements were expected to change throughout the project, and, consequently, the team selected an iterative-incremental model with three iterations.
Select the set of scenarios. To help the team with this task, gOntt asks a set of simple natural language questions, “Do you intend to use any ontological resource in your ontology network development?” and “Do you intend to develop your ontology network in different ontology languages?”. gOntt uses the responses to these questions to output the initial ontology network life cycle, that is, an initial plan for ontology network development, based on the methodological foundations presented by Suárez-Figueroa (2010). In the particular case presented here, this set of questions has to be answered once per iteration, that is, three times. The selected scenarios were as follows:
In the first iteration, the team applied Scenarios 3 and 7. They reused the ontologies (Scenario 3) in two different ways: (a) by reusing whole ontologies (e.g., SOUPA, RECO, the OWL Time ontology, the FOAF ontology) and (b) by reusing ontology modules from the CoDAMoS and DeliveryContext ontologies. Additionally, they reused the modular and the n-ary relationship patterns as part of Scenario 7.
In the second iteration, the team applied Scenarios 2, 7 and 8. They reused and reengineered the ISO 3166 NOR into an ontology about countries as part of Scenario 2. Then, as part of Scenario 7, they reused the componency pattern to denote that objects either are proper parts of other objects or have proper parts. Finally, as part of Scenario 8, they restructured the different ontological resources from the previous and the current iterations.
In the third iteration, the team deployed Scenarios 7, 8 and 9. They reused the price pattern to enrich services with their prices as part of Scenario 7. They also restructured the ontology network as part of Scenario 8. Finally, they transformed all the ontology labels into Spanish as part of Scenario 9.
Update the initial plan. The goal of this task is to modify (if necessary) the initial plan presented by gOntt. In the case presented here, the ontology development team changed the order of the processes and activities as follows: in the first iteration, the plan was to perform the ontological resource reuse processes and the ODP reuse process in parallel; in the second iteration, the plan was to perform the ODP reuse process in parallel with Scenario 2 processes.
Establish resource restrictions and assignments. In this case, information about both time scheduling (updating the iteration duration and the time automatically assigned to the planned processes and activities) and human resource assignments were included in the ontology network development plan.
The above three iterations and their respective scenarios were developed according to the methodological guidelines included in the NeOn Methodology framework. As a result, the team output usercONText.32
NeOn Methodology framework application. (Colors are visible in the online version of the article;
Figure 6 shows how the NeOn Methodology was applied to this particular use case.
As de Hoog (1998) said, “It is extremely difficult to judge the value of a methodology in an objective way. Experimentation is, of course, the proper way to do it, but it is hardly feasible because there are too many conditions that cannot be controlled”. On the one hand, the introduction of an experimental toy problem would violate the basic assumption behind the need for a methodology: the development process is complex. On the other hand, if we apply the argument provided by de Hoog for knowledge-based systems to the ontology engineering field, we find it is unlikely that anyone will be willing to pay twice for building the same complex ontology using different approaches.
Selection of use cases in which the NeOn Methodology framework was used
Selection of use cases in which the NeOn Methodology framework was used
People with different backgrounds (e.g., ontology engineers, domain experts, final users and linguists, among others) have used the NeOn Methodology framework in different ontology development projects. These projects targeted domains as far apart as satellite data processing, seeking program funding, fishery stocks, e-employment and multimedia. Table 5 summarizes six use cases (one of which is presented in Section 3) in which the NeOn Methodology framework was applied. Table 5 includes information on the ontology and ontology development process. The information includes the following items:
Ontology name: name of the ontology, URL (if the ontology is available) and references.
Description: summary describing the knowledge modeled in the ontology. The information also includes the languages (ontological and natural languages) used in the development, and whether the ontology was part of any other project.
Applied NeOn scenarios: list of the scenarios employed during ontology development, and a description explaining why such scenarios were needed.
Life-cycle model used: model used in ontology development, stating, if the iterative-incremental model was used, the number of iterations.
Employed technological support: the ontology editor(s) handled during the development, including information on whether gOntt was used to schedule the ontology development project.
As shown in Table 5, the NeOn Methodology framework is tool-independent since it has been used in a variety of development environments (NeOn Toolkit, Protégé, TopBraid Composer, etc.).
Developers’ experiences using the NeOn Methodology framework
One way of evaluating a methodology is to use surveys (Kitchenham, 1996; Palvia & Nosek, 1990) in which the users who have applied the methodology in past projects are asked to provide information about their experience.
In order to analyze developer experiences of using the NeOn Methodology framework as part of their own ontology network developments, we have designed a questionnaire to elicit their views on the methodology and draw some conclusions regarding the following general issues:
Did the NeOn Methodology work and/or was it useful and applicable?
Did the NeOn Methodology help ontology network developers to do their job?
Are some methodological components more helpful than others within the NeOn Methodology?
Are ontology network developers satisfied with the NeOn Methodology?
Would the ontology network developers use the NeOn Methodology again and recommend it to other developers?
The questionnaire is composed of two different sections: (1) one related to background knowledge, and (2) another related to the experiences gained by using the NeOn Methodology framework. The first part includes four questions (two open questions and two multiple-choice questions). The second part is composed of 14 questions: five multiple-choice questions and nine open questions asking survey participants about their opinions and comments.
The questionnaire was administered to people participating in different international research projects as part of which ontologies had to be developed. Our questionnaire is available at
Regarding background knowledge, it is noteworthy that 65% of survey respondents are specialized in ontology engineering, whereas 35% have some knowledge of the field, although they cannot be considered ontology experts. With respect to the number of ontologies developed, 54% of respondents have built more than five ontologies.
Moving on to the core part of the questionnaire, particularly the responses to the multiple-choice questions, we have found that
Most survey respondents considered that the NeOn Methodology is useful and well explained, as shown in Fig. 7(a) and (b).
Ninety six percent of respondents considered that the NeOn Methodology was helpful during ontology development, whereas 4% did not consider the methodology to be of any help whatsoever (see Fig. 7(c)).
Ninety two percent of the respondents said that they would use the NeOn Methodology again in future ontology developments, and 62% of the respondents would recommend the methodology to other colleagues (see Fig. 7(d) and (e)).
Results for the multiple-choice questions. (a) Distribution of the responses to the question, “Did you find the NeOn Methodology useful?”. (b) Distribution of the responses to the question, “Do you think the NeOn Methodology is well explained?”. (c) Distribution of the responses to the question, “In general, do you think the NeOn Methodology has helped you during the development?”. (d) Distribution of the responses to the question, “In your future ontology network developments, will you use the NeOn Methodology again?”. (e) Distribution of the responses to the question, “Would you recommend the NeOn Methodology to other colleagues involved in ontology development projects?”. (Colors are visible in the online version of the article;
We can characterize the whole population of NeOn Methodology users by taking a random experimental sample of 26 people, the same number of people who responded to our questionnaire, and asking each person the question: (a) “Did you find the NeOn methodology useful?”. Let
If instead of assuming the hypothesis of
If
Applying a similar reasoning to the other questions, the result is the same for question (b) (where again there were 26 positive answers). In the case of question (c), if we consider together yes, always, and yes, sometimes, then
Regarding the results for the open questions, it is noteworthy that:
According to the survey respondents, the advantages of the NeOn Methodology over other existing methodologies, such as METHONTOLOGY, are that: (a) this methodology is easy to understand in different scenarios, partly thanks to the guiding cards for each process and activity; (b) it is based on a scenario approach; and (c) it includes detailed and helpful documentation.
On the other hand, the disadvantages are that: (a) the methodology does not explicitly state all the steps to be performed; and (b) the methodology is time-consuming to apply. However, this methodology is generally regarded as being more robust and helpful than the others are. In this respect, one of the respondents commented literally: “I like that NeOn is oriented to collaborative development of network of ontologies and gives rather good guidance for choices. I consider METHONTOLOGY too complex and hard to apply to real ontology development process” [sic].
The respondents replied that the most useful NeOn Methodology framework components were the methodological guidelines and the set of scenarios. Thus, some of the comments were as follows: “The guidelines are useful to not forget any important step” [sic]; “I also found useful the guiding cards, which were short” [sic]; “Scenarios are a really good starting point, especially at the beginning”; “Scenarios consider different cases and provide guidelines for each one”; and “Detecting where you are, what to do, and how to proceed according to the different information sources available is very important in ontology development projects”.
Additionally, we should add that five respondents considered the guidelines for the ontology requirements specification activity to be a very useful and beneficial component because: (a) they helped them to gather the necessary information and (b) the ORSD template and the different tasks made it easier to gather ontology requirements.
It is worth mentioning that both the glossary and the set of life-cycle models were considered useful by approximately 50% of the respondents and useless by the other 50%.
Figure 8 summarizes the responses with regard to the usefulness of the methodological components.
Statistics on NeOn Methodology framework component usefulness. (Colors are visible in the online version of the article;
We will proceed as above in order to characterize the whole population who used the NeOn Methodology, albeit now focusing on the usefulness of the framework components. For each of the three components, the random variable that represents the number of positive responses follows a binomial. Applying the same reasoning as above, we can conclude that at least 73% of the population would answer yes to the question of whether the methodological guidelines are useful. We cannot reject the hypothesis
Despite the findings regarding the glossary and life-cycle models, our experience shows that both components are necessary. As we see it, the most plausible explanation for this mismatch is that methodology users already have an implicit conceptualization of these components. Therefore, we think that an interesting line of research would be to study the dependence of component usefulness on ontology engineer experience and skill.
Finally, the respondents provided some suggestions that may help improve the NeOn Methodology framework, such as:
Making the methodology more lightweight and agile because “like RUP for software engineering NeOn Methodology is good but heavy”;
Preparing a light and adapted version of the methodology for linked data practitioners;
Building NeOn Methodology framework specializations in specific domains (such as BioNeOn, EduNeOn, MedNeOn, etc.);
Providing more realistic examples of how to carry out the different processes and activities;
Improving the ‘human factor’, that is, the involvement of domain experts in the knowledge acquisition process;
Translating the methodology into different natural languages;
Including the linked data dimension in the set of scenarios;
Providing more guidelines for addressing the problem of dividing ontology development between several teams, where each team develops one or more parts of the ontology that then have to be integrated;
Producing specific guidelines for ontology conceptualization that include ontology-building criteria similar to those presented by Rector et al. (2004);
Enhancing the guidelines and criteria for reusing ontologies in order to reduce the time taken to complete the required information;
Finally, developing Protégé plug-ins to support the methodology in order to improve the technological dimension.
There has been a growing trend in recent years towards the reuse of knowledge resources in ontology development in the context of different development initiatives. Thus, the NeOn Methodology framework proposes the reuse and reengineering of available ontologies, ODPs and NORs, all agreed upon by consensus, to construct modular ontologies interconnected as networks. The NeOn Methodology framework is based on four pillars: (1) a glossary of processes and activities; (2) a set of nine flexible ontology-building scenarios; (3) two ontology life-cycle models that organize ontology development; and (4) a set of methodological guidelines for performing specific processes and activities. This framework is a methodology that defines each process or activity in a precise manner by clearly stating its purpose, inputs and outputs, actors involved, execution time, and the proposed set of methods, techniques and tools for use.
The NeOn Methodology framework is a step forward in the evolution of the ontology engineering field, which requires practical and precise methodologies. This methodology solves the problems of: (a) the need for a methodology targeting ontology practitioners (ontology engineers, domain experts, final users, linguists, etc.) rather than just ontology researchers; and (b) the need for a methodology explaining the ontology-building process in the style and with the granularity of software development methodologies. The NeOn Methodology is presented prescriptively by means of simple descriptions and diagrams so that it is easily understandable for people with different backgrounds (ontology engineers, domain experts, final users, linguists, etc.). The NeOn Methodology adopts a software engineering approach with different levels of complexity so that it can be easily taken on board. It is adaptable to specific needs and users; it can also adopt new processes or activities involved in ontology network development.
The NeOn Methodology framework has already been used flexibly in ontology development projects within a diversity of domains and areas and within the context of different European and national projects (e.g., e-employment, invoices, pharmaceutical products, user-context in tourism (Lamsfus et al., 2009), education (Clemente et al., 2011)). In addition, as already reported in the paper, people with different profiles and backgrounds have applied the methodology successfully.
We have demonstrated that the NeOn Methodology framework is independent of technology since it has been used with several development environments (NeOn Toolkit, Protégé, TopBraid Composer, etc.). The methodology is closely related to the NeOn Toolkit and its plug-ins, but it can also be used with other tools such as Protégé, TopBraid Composer and others.
According to the responses to our questionnaire on the NeOn Methodology framework, we can state that the majority of the respondents consider the methodology to be useful, helpful and well explained. Most said that they would apply it again in future ontology developments, and 62% of respondents added that they would recommend the NeOn Methodology to other colleagues. Generally, the methodology is perceived as being more robust and helpful than others, like METHONTOLOGY, for example. By contrast, the main weaknesses of the methodology are: (a) it does not explicitly state the whole sequence of steps to be performed, although this sequence is generated during the scheduling activity that can be performed using gOntt; and (b) methodology application is time consuming.
Regarding the usefulness of the components included in the NeOn Methodology framework, the most useful are the methodological guidelines and the set of scenarios; on the other hand, the glossary and the set of life-cycle models were considered useful and useless by almost the same number of respondents.
Finally, we would like to mention that we are currently working on a more lightweight and agile version of the NeOn Methodology, as well as on the inclusion of the linked data dimension in the methodology. We also plan to: (a) provide specific guidelines for ontology modeling; (b) enhance the guidelines and criteria for reusing ontologies in order to reduce the time taken to complete the required information; (c) provide technological support for ontology reuse; and (d) provide some guidelines for documenting ontologies that include the different knowledge resources that were reused.
Footnotes
Acknowledgements
This work was supported by the European NeOn project (FP6-027595) and the Spanish mIO! project (Tecnologías para prestar servicios en movilidad en el futuro universo inteligente – Technologies for mobility service provision in the future intelligent universe, CENIT-2008-1019) and is partially funded by the CDTI (Centro para el Desarrollo Tecnológico Industrial – Industrial Technology Development Center) as part of the INGENIO 2010 Program. We would like to thank Maria Poveda-Villalón for her work on the development of the user-context ontology network. We are very grateful to Rosario Plaza for her revisions and comments. We would also like to thank all the people who responded to the questionnaire, as well as the developers and practitioners (Boris Villazón-Terrazas, Agnieszka Lawrynowicz, Raúl Palma, Julia Clemente, Carlos Bobed, Germán Herrero-Carcel and Jose Manuel Gómez-Pérez) who provided us with detailed information about their ontology developments.
Excerpt of methodological guidelines
| Scenarios | Excerpt of methodological guidelines for processes and activities | |
| Scenario 1: From specification to implementation | Ontology Requirements Specification activity (Suárez-Figueroa, 2010; Suárez-Figueroa et al., 2012). Its goal is to explain why the ontology is being built, which its intended uses are, who the end-users are, and what requirements the ontology should fulfill. The result of this activity is the Ontology Requirements Specification Document (ORSD) that should be written following the ORSD template. | Task 1. Identify the purpose, scope and implementation language. The objective is to determine the main goal of the ontology, its coverage and foreseeable granularity, and its implementation language. A set of interviews with users and domain experts are conducted, taking as input a set of ontological needs. |
| Task 2. Identify the intended end users. Its goal is to establish who the intended main end-users of the ontology will be. A set of interviews with users and domain experts are conducted, taking as input a set of ontological needs. | ||
| Task 3. Identify the intended uses. Its goal is to obtain the intended uses and use scenarios of the ontology. A set of interviews with users and domain experts are performed, taking as input a set of ontological needs. The scenarios should be described in natural language; they can be expressed as use cases in UML. | ||
| Task 4. Identify requirements. Its goal is to acquire the set of requirements (functional and non-functional) that the ontology should satisfy. It is recommended to use the competency question (CQ) technique (Grüninger & Fox, 1995) to specify functional ontology requirements. This initial set of ontology requirements is obtained by means of a set of interviews with users and domain experts. Other knowledge elicitation techniques can be also applied for the creation of CQs, such as 20-questions, triad analysis, card sort (Rao et al., 2008), and the use of a goal modelling approach (Barbosa et al., 2011), in particular a methodology called Tropos. Mind map tools and Excel can be used for gathering the requirements. | ||
| Task 5. Group functional requirements. Its goal is to group into several categories the list of functional ontology requirements in the form of CQs and their associated answers obtained in Task 4. Usually this task is carried out in parallel to Task 4. | ||
| Task 6. Validate the set of requirements. The aim here is to identify possible conflicts between ontology requirements (functional and non-functional), missing ontology requirements and contradictions between them. The criteria that can be used in this validation task are the following: correctness, completeness, consistency, verifiability, understandability, no ambiguity, conciseness, realism, modifiability and traceability. | ||
| Task 7. Prioritize requirements. Its goal is to provide different levels of priority to the functional and non-functional ontology requirements identified. In the case of the functional requirements, priority should be given to the different groups of CQs, and, within each group, to the different CQs; additionally, priority could be given to each CQs independently of the groups. Priority will be used by the ontology development team for planning and scheduling the ontology development and for deciding which parts of the ontology will be developed first. This task is optional but recommended. | ||
| Task 8. Extract terminology and its frequency. Its goal is to extract a pre-glossary of terms with their frequencies from the list of CQs and their answers, which are identified in Task 4. Terminology extraction techniques and tools supporting such techniques can be used. This pre-glossary of terms is divided in three different parts: terms from the CQs, terms from the CQs’ answers, and terms identified as named entities. The set of terms appearing with high frequency will be used later on for searching knowledge resources that could be potentially reused in the ontology development. As heuristic, we can mention that normally the set of terms appearing more frequently is that which requires more effort during the ontology development; for this reason, frequency is important because it takes us towards those resources that allow saving more effort. | ||
| Scheduling activity (Suárez-Figueroa, 2010; Suárez-Figueroa et al., 2012). Its goal is to organize the different processes and activities in time, that is, to state a concrete planning or schedule that guides the ontology network development, including processes and activities, their order and time, as well as human resources restrictions. | Task 1. Select the ontology network life cycle model. Its goal is to obtain the most appropriate ontology network life cycle model for the ontology network to be developed. The ontology requirement specification document (ORSD) and the types of potential knowledge resources to be reused during the development are taken as input. | |
| Task 2. Select the set of scenarios. Its goal is to select the set of scenarios to be followed during the ontology network development. The ORSD and the set of potential knowledge resources to be used during the development are taken as input. | ||
| Task 3. Update initial plan. Its goal is to modify (if necessary) the initial plan presented by gOntt. The ORSD and the set of potential knowledge resources to be used during the development are taken as input. Ontology developers can modify the initial plan in the following ways: (a) by including or deleting processes, activities and model phases, and (b) by changing order and dependencies between processes and activities. | ||
| Task 4. Establish resource restrictions and assignments. Its goal is to include information about temporal scheduling and human resource assignments in the life cycle obtained in Task 3. The ORSD and the set of potential knowledge resources to be used during the development are taken as input. | ||
| Scenario 2: Reusing and reengineering non-ontological resources (NORs) | NOR Reuse process (Villazón-Terrazas, 2012). Its goal is to choose the most suitable non-ontological resources for building ontologies. The output is a set of non-ontological resources that, to some extent, covers the expected domain. | Activity 1. Search non-ontological resources. The goal of the activity is to search NORs from highly reliable web sites, domain-related sites and resources within organizations. The terms appearing most frequently in the ORSD are used to search for the candidate NORs that cover the terminology desired. |
| Activity 2. Assess the set of candidate non-ontological resources. The goal of this activity is to assess the set of candidate NORs. This activity is divided into the following 6 tasks: (1) extracting lexical entries of the NORs; (2) calculating precision; (3) calculating coverage; (4) evaluating the consensus; (5) evaluating the quality; and (6) building an assessment table. | ||
| Activity 3. Select the most appropriate non-ontological resources. Its goal is to select the most appropriate NORs that are to be transformed into an ontology. The selection is performed manually and it is recommended to look for resources with consensus, quality, high value of coverage and high value of precision. The activity output is a ranked list of NORs that, to some extent, covers the expected domain. These resources are ready for the reengineering process. | ||
| NOR Reengineering process (Villazón-Terrazas, 2012). Its goal is to transform a non-ontological resource into an ontology. The output of the process is an ontology. | Activity 1. Non-ontological resource reverse engineering. Its goal is to analyze a NOR, to identify its underlying terms, and to create representations of the resource at different levels of abstraction (design, requirements and conceptual). The activity is divided into 3 tasks (data gathering, conceptual abstraction and information exploration). | |
| Activity 2. Non-ontological resource transformation. Its goal is to generate a conceptual model from the NOR. It is recommended the use of patterns for reengineering non-ontological resources (PR-NOR) in order to guide the transformation process. Thus, the activity is divided into (1) the search for a suitable pattern for reengineering NORs, (2) the use of reengineering patterns to guide the transformation or perform an ad-hoc transformation and (3) the manual refinement. | ||
| Activity 3. Ontology forward engineering. Its goal is to generate the ontology. The conceptual model obtained in the second task of Activity 2 is transformed into a formalized model, according to a knowledge representation paradigm, such as description logics, first order logic or F-logic. Then, the formalized model is implemented in an ontology language. | ||
| Scenario 3: Reusing ontological resources | Domain Ontology Reuse process (Suárez-Figueroa, 2010, 2012). Its goal is to find and select one or several domain ontologies that can be reused in the ontology being developed. | Activity 1. Domain ontology search. Its goal is to search in libraries, repositories and registries for candidate domain ontologies that could satisfy the needs of the ontology network being developed. Those terms appearing in the pre-glossary of the ORSD are used as input. |
| Activity 2. Domain ontology assessment. The objective of this activity is to find out if the set of candidate domain ontologies are useful for the development of the ontology network. The proposal considers analyzing the domain coverage and deciding whether a particular domain ontology is useful or not. This means analyzing whether the domain ontology covers (totally or partially) the requirements identified in the ORSD of the ontology network being developed. | ||
| Activity 3. Domain ontology select. Its goal is to find out which domain ontologies are the most suitable for the development of the ontology network. As a result of this activity, the ontology developers should select the candidate domain ontology that scored the highest results. | ||
| Activity 4. Domain ontology integration. Its goal is to integrate the domain ontologies selected in Activity 3 into the ontology network being developed. | ||
| General Ontology Reuse process (Fernández-López et al., 2013). Its goal is to find and select general ontologies and integrate them into the ontology network being developed. | Activity 1. Conduct a comparative study of reusable general ontologies. Its goal is to find and compare general ontologies. We propose to build a table to represent the comparative study. From the point of view of the formal concept analysis (FCA) (Stumme, 2009), this table is essentially a formal context. The contextual objects are the ontologies to be compared, and the attributes are both the definitions and the axioms that these ontologies include or features of the models that they describe (e.g., the definition of time points, the assumption of different granularities). | |
| Activity 2. Make a decision on which general ontology is going to be reused. Its goal is to select the most appropriate general ontology to be reused in the ontology network being developed. The table obtained in the comparative study is used to select the general ontology that best fits the requirements of the ontology to be developed. FCA is used to make this decision by structuring the different options (candidate general ontologies) in lattices. Such lattices are obtained and used during this activity of decision making. | ||
| Activity 3. Assemble the whole ontology. Its goal is to customize and integrate the most appropriate general ontology into the ontology to be developed. | ||
| Note: Let’s note that if there is a previous comparative study, then Activity 1 does not have to be carried out, and thus the reuse process should start directly on Activity 2. | ||
| Ontology Statements Reuse process (Suárez-Figueroa, 2010, 2012). Its goal is to make use of ontology statements in the ontology network being developed. The output of this process is a set of ontology statements that will be used. | Activity 1. Ontology statement search. Its goal is to search the Web for ontology statements candidates that could satisfy the ontological needs of a particular case. The ORSD is taken as input, specifically those terms that have a high frequency. Some gateways to the Semantic Web such as Watson on-line or the Watson plug-in in the NeOn Toolkit can be used for the search. | |
| Activity 2. Ontology statement assessment. Its goal is to decide whether a concrete ontology statement is useful for the ontology network being developed. The content and granularity of ontology statements should be inspected to assess whether they satisfy the ontological needs. A set of criteria for assessing each ontology statement is provided. | ||
| Activity 3. Ontology statement selection. Its goal is to decide which of the useful ontology statements provided are the best or most convenient for the ontology network being developed. The criterion of minimum effort needed for integrating the ontology statement should be used when comparing ontology statements and selecting the most convenient ones. | ||
| Activity 4. Ontology statement integration. Its goal is to integrate the ontology statements selected in Activity 3 into the ontology network being developed. After integrating an ontology statement, the following tasks will probably have to be performed: (a) changing names (concepts, properties) to adapt them to the naming conventions used in the ontology network being developed; (b) adding range to properties and changing cardinalities; and (c) adding restrictions. | ||
| Activity 5. Local inconsistency detection. Its goal is to search for local inconsistencies in the ontology network. Such inconsistencies could have been introduced by adding new knowledge. | ||
| Scenario 5: Reusing and merging ontological resources | Ontology Matching activity (Euzenat & Le Duc, 2012). Its goal is to find or discover relationships or correspondences between entities of different ontologies or ontology modules. This activity can be seen as the first stage of Ontology Alignment, which refers to the activity of finding the correspondences between two or more ontologies and storing/exploiting them. | Task 1. Identify ontologies and characterise needs. Its goal is to identify the ontologies to be matched and to characterise the needs. |
| Task 2. Find existing alignments. Its goal is to find existing alignments that satisfy the needs. Alignments may be found on the web or through specialised directories. | ||
| Task 3. Select a matcher. Its goal is to find a suitable matcher to build a new alignment. | ||
| Task 4. Match ontologies. Its goal is to run the selected matcher against the ontologies and to collect the resulting alignment. | ||
| Task 5. Evaluate alignments. Its goal is to perform a final screening and validation. | ||
| Task 6. Enhance alignments. Its goal is to enhance the alignments either through a manual modification of the alignment or through the application of refinement procedures. | ||
| Task 7. Store and share alignments. Its goal is to save and share the alignment obtained into a declarative format and to give this alignment proper annotation in order to record its provenance and purpose. | ||
| Task 8. Render alignments. Its goal is to transform the alignment into another form or to interpret the alignment so that it can perform actions such as mediation or merging. | ||
| Scenario 7: Reusing ontology design patterns (ODPs) | Ontology Design Patterns Reuse process (Suárez-Figueroa et al., 2009; Presutti et al., 2012). Its goal is to use available ontology design patterns in order to solve different modelling problems during the development of ontologies. | Activity 1. Identify requirements to be addressed. The objective of this activity is to identify which requirement(s), from the ORSD, can be addressed by ontology design pattern reuse. The selection of requirements may be based on the following (non-exhaustive) list of criteria: which requirements are not yet addressed within other activities; which requirements do not provide trivial solutions; and which requirements can be associated with existing pattern types. |
| Activity 2. Identify available patterns. Its goal is to identify as many patterns, or rather pattern catalogues, as possible that could help solve the modelling issues proposed by the requirements selected in Activity 1. | ||
| Activity 3. Divide and transform the problem, select a partial problem. Its goal is to prepare the set of modeling problems posed by the requirements for matching the set of available patterns. In order to match the problem with such small partial solutions and to reuse them, the problem needs to be divided into manageable pieces. When the problem has been transformed and divided, one such “manageable piece” is selected as a starting point. | ||
| Activity 4. Match selected partial problem to patterns. Its goal is to identify which patterns can solve which parts of the selected sub-problem, if any. Two general suggestions are provided for performing this activity in order to assist the matching: (a) to identify the type of patterns that may be suitable for solving the problem and to match the problem primarily to these kinds of patterns and (b) depending on the type of pattern, to use the description of the intent of the pattern to match with the problem (for content patterns, this would mean to match the competency questions of the patterns with the competency questions of the problem). | ||
| Activity 5. Select patterns. Its goal is to select a set of patterns for reuse based on the results of the matching process carried out in Activity 4. | ||
| Activity 6. Apply (reuse) selected patterns and compose them. Its goal is to reuse the selected patterns and compose them. | ||
| Activity 7. Evaluate and revise with respect to partial problem. Its goal is to test the solution on the selected partial problem at hand and to ensure that it really solves the problem correctly. | ||
| Activity 8. Integrate partial solutions. Its goal is to integrate the solutions into the partial problems solved during the different iterations or during the solution of different sub-problems by the parallel teams. | ||
| Scenario 8: Restructuring ontological resources | Ontology Modularization activity (d’Aquin, 2012). Its goal is to obtain a module or a set of modules from an ontology which fits the requirements of a particular application or a particular scenario. | Task 1. Identify the purpose of modularization. Its goal is to identify both the reasons for modularizing the ontologies and the benefits expected from guiding the rest of the process. |
| Task 2. Select a modularization approach. Its goal is to choose the best modularization approach between ontology partitioning (the activity of dividing an ontology into a set of (not necessarily disjoint) modules that together form an ontology, but which can be treated separately) and ontology module extraction (the activity of obtaining from an ontology some concrete modules that could be used for a particular purpose). Whenever the purpose relates to the entire ontology (i.e. when improving maintenance and, in some cases, performance), a partitioning approach should be considered. Whenever the purpose relates to extracting specific parts of an ontology (e.g. to customize or reuse it partially), a module extraction should be considered. | ||
| Task 3. Define modularization criteria. Its goal is to define the modularization criteria. These criteria determine the basic characteristics that the resulting modules should have, that is, what should go into a module. | ||
| Task 4. Select a base modularization technique. Its goal is to select the most appropriate technique, depending on the criteria to apply. | ||
| Task 5. Parametrize the technique and applying it. Its goal is to select the different parameters required to obtain interesting and useful results when the technique is applied. Most of the techniques would, in principle, be applied in the same way, namely, taking the original ontology as input and creating modules in the form of smaller ontologies, thus allowing processing the resulting modules iteratively, in the same way as the original ontology. | ||
| Task 6. Combine results. Its goal is to refine and combine the results obtained with various parameters, techniques and approaches in an iterative process. | ||
| Task 7. Evaluate modularization. Its goal is to evaluate the result of the modularization. This evaluation is crucial to decide whether a new iteration is necessary (in such a case, a new set of criteria and a new technique is applied), or whether the current (set of) modules are satisfactory, considering the application scenario. The modularization could be evaluated in two different ways: (a) by checking the criteria and (b) by testing against the purpose of modularization. | ||
| Task 8. Finalize modularization. Its goal (if necessary) is to deploy and exploit the modularization results in an application. | ||
| Scenario 9: Localizing ontological resources | Ontology Localization activity (Espinoza Mejía et al., 2012). Its goal is to adapt an ontology to a particular language and culture. | Task 1. Select the most appropriate linguistic assets. Its goal is to select the most appropriate linguistic assets that help in the localization activity. The task output is a set of linguistic assets that can help reduce the cost, improve the quality, and increase the consistency of the localization activity. The choice of a specific resource is performed manually, taking into account that the linguistic assets comply with the following characteristics: (a) consensus, (b) broad coverage and (c) high precision. |
| Task 2. Select ontology label(s) to be localized. Its goal is to select the ontology label(s) to be localized. The task output is a set of ontology labels and their context. The context describes the meaning of a specific label in the ontology and consists of a small excerpt of ontology labels around the ontology label itself (e.g., direct hypernym labels, hyponym labels). | ||
| Task 3. Obtain ontology label translation(s). Its goal is to obtain the most appropriate translation in the target language for each ontology label. Different machine translation (MT) techniques can be used to perform this task automatically. | ||
| Task 4. Evaluate label translation(s). Its goal is to evaluate label translations in the target language. Different linguistic criteria can be used for evaluating label translations. Two levels of evaluation criteria (semantic fidelity evaluation and stylistic evaluation) are proposed. | ||
| Task 5. Ontology update. Its goal is to update the ontology with the label translations obtained for each localized label. The task output is an ontology enriched with labels associated to each localised term in the target language. | ||
| Across scenarios | Ontology Evaluation activity (Sabou & Fernandez, 2012). Its goal is to check the technical quality of an ontology against a frame of reference. | Task 1. Select individual components of the ontology network. Its goal is to identify the elements of the ontology network that need to be evaluated, including individual ontologies, ontology statements and ontology relations, among others. This decision should be based on two criteria: (a) which ontology network elements are critical for the overall network; and (b) which of these elements can actually be evaluated. |
| Task 2. Select an evaluation goal and approach. Its goal is to decide the aim of the evaluation and to select an appropriate evaluation approach. The following evaluation goals can be distinguished: (a) domain coverage, (b) quality of the modelling, (c) suitability for an application/task and (d) adoption and use. | ||
| Task 3. Identify a frame of reference and evaluation metric. Its goal is to select the concrete ingredients of the evaluation that consist of a frame of reference and a set of evaluation metric(s). | ||
| Task 4. Apply the selected evaluation approach. Its goal is to apply the selected evaluation approach. This task requires, on the one hand, a proper setup for the evaluation experiments and the implementation of software tools to compute the evaluation metrics, and on the other, the engagement of the human experts in stimulating sessions to collect their evaluations. | ||
| Task 5. Combine and present individual evaluation results. Its goal is to combine the evaluation results derived from individual components in order to reach a global understanding of the network’s quality. Finally, the results of the evaluation should be presented in an appropriate form for possible repair (corrections, additions) and improvements, and for the future evolution of the ontology network. | ||
| Ontology Evolution activity (Palma et al., 2012). Its goal is to facilitate the modification of an ontology by preserving its consistency. | Task 1. Request a change. Its goal is to initiate the change process. Changes can either be requested from users or developers, who feel that the ontology is not adequate in its current form; changes can also be discovered. | |
| Task 2. Plan the change. Its goal is to analyze the change requested and to determine both why the change needs to be made and which part of the ontology is affected by such a change. As a result of the analysis performed during this task, the ontology engineer may decide to implement the change, and if this has many side effects or if the cost of implementation is too high, he may defer the change request to a later time or not implement it at all. | ||
| Task 3. Implement the change. Its goal is to implement the changes. Changes can be as easy as adding or removing a subclass; however, some changes can require complex operations as well as the restructuration of the ontology. Change logging allows tracking which changes have been made; it also allows undoing them easily, in case something goes wrong. | ||
| Task 4. Verification and validation. Its goal is to verify and validate the ontology after this is considered completely evolved. During this assessment, the ontology modified originally is commonly verified in isolation; this activity can include the verification of other artefacts related to the ontology (as mentioned above) to ensure that they were not changed in a wrong way or that they have an unexpected behaviour. | ||