Abstract
With the increasing complexity of engineered systems, digital twins (DTs) have been widely used to support integrated modeling, simulation, and decision-making of the system of systems (SoS). However, when integrating DTs of each constituent system, it is challenging to implement complexity management, interface definition, and service integration across DTs. This study proposes a new concept called cognitive twin (CT) to support SoS development and operation. CTs have been defined as DTs with augmented semantic capabilities for promoting the understanding of interrelationships be-tween virtual models and enhancing the decision-making. First, CTs aim to integrate the information description of DTs across constituent systems using a unified ontology and semantic modeling technique. Second, CTs provide integrated simulations among DTs for decision-making of the SoS based on a high-level architecture (HLA). Finally, through reasoning ontology models, CTs provide decision-making options for the operations of real constituent systems. A case study on unmanned aerial vehicles (UAVs) landing on unmanned surface vehicles (USVs) is used to verify the flexibility of this approach. From the results, we find that the CT based on the proposed ontology provides a unified formalism of DTs across UAVs and USVs. Moreover, the reasoning based on the CT provides decision-making capabilities for UAVs by implementing cognitive computing to select target USVs for landing.
Keywords
Introduction
Developers of complex systems face an emergent challenge in the integration of heterogeneous complex systems across domains [1, 2]. Currently, systems that are integrated within an infrastructure by implementing internet connectivity between physical and cyber devices cannot exist in isolation, which drives widespread and accelerating changes in system environments [3]. Thus, a system of systems (SoS) has been proposed to define such a situation as a “set of systems or system elements that interact to provide a unique capability that none of the constituent systems can accomplish on its own” [4], such as an air traffic control system [5]. Moreover, to support SoS analysis and development, system of systems engineering (SoSE) has been proposed as “the process of planning, analyzing, organizing, and integrating the capabilities of a mix of existing and new systems into a system-of-systems capability that is greater than the sum of the capabilities of the constituent parts” [6].
When developing an SoS and its constituent systems, complexity is a key performance indicator to measure and manage the interrelationships among constituent systems and their components. Complexity refers to a system characteristic that makes it difficult, or even impossible, to accurately predict behavior over time, particularly in terms of understanding all relevant interactions among system components within the defined system boundary [7]. When managing complexity, the unified formalism of the SoS is the basis to describe the topology between each system and component [8]. However, discrepancies of system feature within one SoS require a unified specification to represent domain-specific knowledge of each constituent system, which is herein considered as challenge 1.
Digital twins (DTs) have been widely used to provide performance predictions for constituent systems as a virtual representation that serves as a real-time digital counterpart of a physical object or process [9]. Within an SoS, it is common to involve several constituent systems with their respective DTs for the system development. However, such heterogeneous DTs cannot be integrated to predict the performance of SoS operations because of the various data structures and interface specifications. Moreover, interface management when integrating such DTs is a difficult task, because of the complex topology among the DTs. Finally, when implementing the integrated simulation across DTs, run-time and synchronized communications are required to support data exchange across DTs. Thus, a standardized framework is needed to combine DTs for integrated simulation across an SoS, which is herein considered as challenge 2.
Because an SoS involves different constituent systems, gaps between the development processes of all the constituent systems lead to uncertainties when implementing such systems together. In this situation, DTs enable the implementation of real-time communications between the physical and virtual worlds to make decisions for the real constituent systems during their operational processes [10]. However, it is difficult to realize this mechanism using current hardware and software, because of the limitation of real-time computing capabilities. Thus, a viable approach is expected to provide decision-making options for the behaviors of constituent systems during SoS operation based on simulation results, which is herein considered as challenge 3.
To address these challenges, a cognitive twin (CT) concept is proposed to encompass complexity management of SoSs and their DTs, to implement integrated simulation, and to support decision-making for the SoS operations. In this approach, a unified ontology is developed to support semantic modeling for describing the topology among SoSs and their DTs. Moreover, integrated simulation is used to predict the behaviors of constituent systems using DTs. Based on the simulation results, reasoning of semantic models is implemented to make decisions during SoS operations.
The CT aims to provide a semantic modeling approach to formalize an SoS using a unified description. Moreover, high-level architecture (HLA) [11] simulations enable CTs to provide an integrated simulation environment for SoS analysis. Finally, simulation results are the basis to support decision-making for SoS operations based on semantic reasoning. Compared with traditioal SoS development, CT provides the formalization of virtual entities, SoS, and integrated simulation results. The contributions of this paper are as follows:
Support for complexity management of an SoS by defining the topology among constituent systems within the SoS and their DTs: The Basic Formal Ontology (BFO) [12] and Industrial Ontologies Foundry (IOF) [13] are referenced to define the ontology for semantic modeling of an SoS and the related DTs. The unified ontology enables the description of real constituent systems and their DTs for complexity management by transforming nonstructural complexity into transparent structural complexity. Support for managing the interfaces between heterogeneous DTs in an integrated simulation infrastructure. SoS development involves heterogeneous DTs for each constituent system; thus, HLA is required to integrate DTs for behavior prediction. Support for decision-making of system behaviors through reasoning and integrated simulation: Based on the integrated simulation results, semantic models of the SoS and DTs are extended with the simulation results referring to constituent system behaviors. The reasoning of such models enables decision-making for SoS operation.
The remainder of this paper is organized as follows. We discuss the related work in Section 2 and demonstrate our research methodology in Section 3. In Section 4, we specifically introduce a semantic modeling approach, including ontology design, integrated simulation across DTs for SoS development, and semantic integration. In Section 5, we present a case study to clarify and validate the proposed CT concept. Finally, discussions are presented in Section 6, and the conclusions of this work are drawn in Section 7.
In this section, we first identify the challenges faced by systems engineers from the perspective of SoSs. Then, we introduce DTs and their integration based on the existing research. Moreover, semantic modeling is investigated to support SoS development. Finally, the state of the art of CTs is discussed to summarize the motivations of this study.
System of systems challenges
SoSs are currently of great interest to systems engineers [14]. SoSE was proposed as a specific systems engineering approach to manage SoS development [15]. Currently, SoSE is facing several challenges [16]: 1) Ambiguity description across SoS: it leads to a large number of design errors and increased project cycle time and cost; 2) Architecture design and verification across systems: SoS is composed of many heterogeneous systems which is designed and verified its respective system developers. This leads to a challenge in terms of interoperability across domain-specific knowledge; 3) Assessing the behaviors in real-time SoS situations: an SoS involves different constituent systems, and their design gaps lead to uncertainties during SoS operations, which require flexible decision-making to decrease the introduced risks. Thus, autonomous decision-making for SoS operations is useful to provide more flexible behaviors for each system.
Digital twin integration for system of systems
The concept of a DT originated from Grieves’s speech on product lifecycle management in 2003 [17]. DTs should at least contain three basic elements: physical entities, virtual entities, and communications between them through data and information. The virtual entities of DTs are considered as digital models, referring to a virtual representation that contains all physical information and knowledge [18]. Currently, DTs are widely used to support SoS development [19]. DTs are developed for different constituent systems during development processes, which makes the integration of heterogeneous data difficult [20]. Several techniques have been proposed to support DT integration within an SoS. For example, HLA is a standard for the modeling and co-simulation of distributed processes and provides standardized interface specification which makes it possible to integrate different hardware interfaces into a single SoS [21, 22]. Moreover, distributed co-simulation protocol (DCP) is also used to support distributed simulation across DTs to enable integrated verification of SoSs [23], but it mainly addresses the integration problem of real-time simulation.
Semantic modeling for system of systems
Semantic modeling enables computers to understand human knowledge, and it has been widely used to describe domain-specific knowledge [24, 25]. Moreover, it provides solutions for data integration of DTs using web techniques [26]. Metamodeling methods provide the basis for semantic modeling. Graph, Object, Point, Property, Role, Relationship (GOPPRR) is the metamodeling language that can express almost all complex relationships. In addition, the Generic Modeling Environment (GME) is also a metamodeling method used in the area of electrical engineering [27]. To date, existing studies have attempted to integrate semantic models with DTs through ontology [28]. For example, Unified Modeling Language(UML) and ontology were integrated to create a high-level semantic model for the exchange of information between computers and humans through human-readable text and computer-readable models [29, 30].
State of the art of cognitive twins
Previous researchers explored to enhance DTs with cognitive capabilities using semantic technologies and DTs. Semantic modelling and ontologies were proposed as a concept of semantically enhanced DTs which enables to define system characteristics, as well as the topology that it interacts with other components [31]. The knowledge graph was used to connect and retrieve heterogeneous data including descriptive and simulation models, it was proposed as the paradigm of the next generation DTs [32]. In 2016, a workshop presentation from Ahmed El Adl was presented about the cognitive evolution of IoT technologies who proposed a Cognitive Digital Twin concept. It was defined as “a digital representation, augmentation and intelligent companion of its physical twin as a whole, including its subsystems across all of its life cycles and evolution phases”. Recently, the Cognitive twins (CTs) were formally proposed as “DTs with augmented semantic capabilities for identifying the dynamics of virtual model evolution, promoting the understanding of interrelationships between virtual models and enhancing the decision-making” [33]. With the similar concept, the functions of hybrid human-machine cognitive ssystems were investigated with the CT definition as “a digital expert or copilot, which can learn and evolve, and that integrates different sources of information for the considered purpose”. From the technical perspective, CT has its framework to combine semantic modeling and integrated simulation in order to support SoS development.
Summary
From the literature review, we find several key motivations for this study:
The SoS is an emergent concept of interest to system developers. SoSE has been proposed to support and manage SoS development, including methods, software, and simulations. Currently, ambiguous descriptions across an SoS, assessing SoS behaviors, and autonomous decision-making for SoS operations are three challenges within the SoSE domain. Integration of DTs is challenging because of heterogeneous data and model structures during SoS development. Currently, HLA, Distributed Co-Simulation Protocol (DCP), etc., are widely used to support DT and model integration. Such techniques are effective in supporting data communication when integrating DTs. Semantic modeling is important for constructing DTs; however, it still lacks a unified architecture and cannot solve the challenge of integrating different domain-specific knowledge. The CT is a new concept for supporting SoS development by using semantic modeling and integrated simulation. Based on this concept, a new potential solution enables managing the SoS complexity through unified descriptions of constituent systems and their topology. Moreover, it uses integrated simulation results to support decision-making for SoS operations.
In this section, the research methodology is introduced to develop and evaluate the proposed CT concept for SoS development.
Construction of cognitive twins based on systems thinking and reference ontology
Figure 1 illustrates the research design. Systems thinking is first used to capture the entities and their topology, which represent the real SoS, and their DTs. Such entities and topology represent domain knowledge about the SoS to construct DTs for each constituent system. These DTs are combined based on the domain knowledge to implement integrated simulation and to obtain the results for each constituent system behavior.
Construction of cognitive twins.
Then CT ontology is defined to contain all entities in the scope, including physical entities of SoS, virtual entities and the topology among them, and simulation results. IOF model-based systems engineering(MBSE) ontology is used to formalize the virtual entities of the constituent systems in the SoS. The IOF MBSE ontology is based on meta – meta models consisting of six key concepts with extensions: Graph, Object, Point, Property, Role, and Relationship(GOPPRRE) [34, 35]. Reference ontologies including BFO [12] and IOF specifications [13] are used to formalize the domain knowledge related to the SoS, DTs, and simulation results. Then, the ontology is aligned to the BFO specification for the final CT ontology. The CT ontology, DTs, and SoS are used to construct CTs that provide ontology models for reasoning. The reasoning results support decision-making for manipulating the constituent systems during SoS operation.
A case study is conducted to demonstrate and evaluate the proposed CT concept for SoS development through quantitative and qualitative analysis. Such analysis evaluates the viability of the CT concept by comparing different scenarios for decision-making during SoS operations. The purpose of the case study is to construct cognitive twin ontology and support SoS decision-making through reasoning, so as to prove that the proposed cognitive twin is workable and meaningful.
Case study definition: A case study of an SoS including three unmanned aerial vehicles (UAVs) and three unmanned surface vehicles (USVs) is conducted. SoSs are also constructed for cases containing more UAVs and USVs.1
Scenario design: Each UAV can land on only one USV. The scenario is to determine an optimized solution for the UAV landing with the least time. Integrated simulation of the scenario: DTs are developed for each UAV and USV. HLA is used to support integrated simulations of the SoS by implementing communications across DTs. Reasoning for decision-making: Results from HLA simulations are formalized in ontology models. Through reasoning of the ontology models, the optimized solution for UAV landing is obtained for the real SoS operations. Evaluation: Qualitative and quantitative approa-ches are used to evaluate the expected purposes of the proposed CT according to their respective metrics:
When using a qualitative approach, three metrics are proposed: 1) complete description, i.e., whether the designed ontology describes the scenario, HLA models, and DTs; 2) integrated simulation, i.e., whether the HLA enables the integration of different DTs of UAVs and USVs to predict the performance of the SoS; and 3) decision-making for SoS operations, i.e., whether the reasoning of ontology models enables decision-making for system behaviors within the SoS. When using a quantitative approach, three metrics are considered: 1) the individuals (which refer to the instances in the ontology) developed based on the ontology in the case study; 2) the HLA entities generated from ontology models; and 3) the entire landing time of the optimized landing scenario determined by reasoning.
Overview
Figure 2 presents an overview of the proposed CT. The CT includes three main components:
Virtual entities, which refer to the digital shadows, digital models, or digital presentations of the constituent systems in the SoS and the topology among them. The digital shadows contain the architecture model, simulation model, etc, as well as data information such as simulation results. Virtual entities are used to perform the system behaviors for each constituent system. Ontology, which refers to the concept definitions and topology between them, is used to support semantic modeling for formalizing the virtual entities, SoS, results of integrated simulation across DTs, and the topology among them. The ontology is developed based on the upper-level ontology BFO, IOF specifications, and the IOF model-based systems engineering ontology approach, GOPPRRE. Based on the reference ontologies, semantic modeling is used to develop ontology models for constructing the CT. SoS, which refers to the physical pairs of virtual entities. For example, in the case study, the SoS consists of three UAVs and three USVs.
SoS that is not the focus of research has been designed and manufactured. All physical entities and topologies have been defined. Virtual entities are composed of models constructed during SoS development, and the interfaces of each component system are implemented by HLA. Finally, ontology is constructed to formalize both SoS and virtual entitis.
The composition of the cognitive twins.
Apart from the main CT components, HLA is used to support integrated simulation across DTs of constituent systems. To realize the automated simulation, digital twin federates (DTFs) for each of the digital models, or virtual entities, are generated from ontology models. Then, through HLA objects, such DTFs form the basis to construct an HLA federation model.
The federation model is used to implement HLA simulations through a run-time infrastructure (RTI) that controls different DTFs to exchange their respective simulation data through a time manager. After simulation executions, the simulation results are generated and defined in the ontology models. Finally, through reasoning of the ontology models, decision-making is performed to control each constituent system to achieve better SoS performance.
Ontology class hierarchy based on BFO and IoF.
As shown in Fig. 3, the ontology for the SoS and DTs is defined based on the BFO and IOF specifications. The red nodes represent IOF and BFO ontology concepts. The purple nodes represent the domain ontology for the SoS and DTs. The blue nodes represent the scenario ontology. The SoS and DT entities class includes two subclasses: 1) Occurrent, referring to “an entity that unfolds itself in time or it is the instantaneous boundary of such an entity or it is a temporal or spatiotemporal region that such an entity occupies”; 2) Continuant, referring to “an entity that persists, endures, or continues to exist through time while maintaining its identity.” The details of other classes are provided in [36]. Through the ontology framework, the specific SoS, its constituent systems, and the operational processes, functions, and qualities in the real world are formalized with the information of the DTs, HLA simulation model, and simulation results. Based on the ontology, the Web Ontology Language (OWL) is used to design the complete ontology for the SoS and DTs under the IOF and BFO ontology framework.
Token::
Where SoS refers to the physical entities in the real world. It contains Coordinate, Artifact, Person, ArtiAggre, and Organization. Coordinate refers to the coordinate system in physical space; Artifact refers to a physical element in the SoS; Person refers to relevant people in the SoS; ArtiAggre and Organization refer to the effective sets of artifacts and persons in the SoS. VEs refer to the virtual entities of the SoS. It contains SimuResults, DT, and HLAModel. SimuResults refers to the results of one distributed simulation. DT refers to a digital model of the constituent system in the SoS. HLAModels refers to the models that describe the topology among different DTs.
In this study, we employ the IOF MBSE ontology to represent HLAModels for implementing simulations automatically [34, 35], as shown in Fig. 4. The ontology is developed based on an M0-M3 modeling framework: 1) The M0 layer represents the physical entities of DTs in the real world; 2) The M1 layer represents the virtual entities of DTs; 3) The M2 layer represents meta-models; 4) The M3 layer represents meta-meta smodels, including the elements: Graph, Object, Relationship, Property, Role, Point and Extention between them. The M0-M3 modeling framework provides standardized and semantic expressions for different model structures, which are used as the basis of the MBSE ontology definition.
GOPPRRE Formalizing HLA models.
OWL is applied to design an MBSE ontology based on GOPPRRE that can support information exchange across the DTs. To express the concepts clearly, some definitions are provided to describe the ontology concepts for the DTs and HLA models using the GOPPRRE approach [34, 35].
The definition of HLAModels includes four components:
Based on the GOPPRRE approach, there are six root classes to represent the HLA model structure. The class
The object property
Token
The object property
The object property
The object property
The object property
The object property
Datatype property
The GOPPRRE approach uses the meta-meta model Property to describe the attribute of the other five meta-meta models. Therefore, there is only one necessary datatype property
To define the connection rules among meta-models Objects and Points in each Graph, an additional constraint is defined as a connector:
Where the connector defines a binding between one Point or Object and one Role on one side of the Relationship. It contains three object properties that allow the individual
A digital twin federation based on HLA.
With the definition of connector, the concept of a connection is defined as a directed link between two different Object individuals in one model, which is realized by a Relationship individual. Token is defined as a connection that is linked from
Where the connection is defined based on connector and connector’. Through this definition, one individual
To define standardized interfaces of heterogeneous DTs, a distributed simulation based on HLA is adopted, which is a universal framework for distributed simulation standardized by the IEEE. HLA provides a set of rules based on the following main concepts to achieve interoperability and reusability when constructing the standardized interfaces for DTs:
Federation: A simulation application composed of a set of simulation components. Federate: One simulation component that represents the basic elements of HLA. RTI: Simulation-oriented middleware for managing alliance interaction.
These specifications effectively define interfaces across DTs. In HLA, a federate is the basic component for the distributed simulation. When integrating DTs and HLA, the federate and virtual entities construct a DTF. Thus, each DT has its respective DTF referring to a standardized interface to the RTI. To generate compiled code for executing co-simulations, semantic models express the interactions between DTFs.
Figure 5 shows a digital twin federation based on HLA. The federation contains a number of DTFs based on the HLA interface specification and OMT. A DTF is composed of simulation models referring to virtual entities of the DT, federate code, and local RTI component (LRC). The simulation model consists of federate objects that meet the HLA standard aiming to construct the data exchange interface of each virtual entity for the request and response to the RTI using the HLA interaction specification. The federate code is an application program that performs local DT executions, which defines interactions with the RTI. The LRC is the API component for the DTF to communicate with the RTI.
The MBSE ontology in Section 4.2 is used to describe the data exchange between different DTFs and develop the HLA models. The HLA model is defined by the federation object model (FOM), which describes the topology among HLA federates during the federation execution. The FOM includes object classes and interaction classes. An object class defines a physical entity of an SoS, and it is composed of a set of object attributes whose values define the state of the entity. An interaction class defines an event that occurred during the federation execution, and interaction parameters define the information used in the event. The publish and subscribe mechanisms are used to support the information exchange during federation execution. When one federate publishes an object class, it registers an instance of an object class, and updates its attribute values. Other federates that subscribe to the same object class can obtain the related value instances and then receive the attribute value. The instances of interaction classes are used similarly, except that they do not represent persistent entities.
Token
In summary, the integration of DTs within an SoS is supported by HLA using its standardized interface specification. The communication between heterogeneous DTs can be realized through the interfaces provided by the RTI. In addition, through the ontology based on the GOPPRRE approach mentioned in Section 4.2, the HLA models are formalized by the ontology model in CT.
The MBSE ontology of CTs that describes the topology among different DTs provides a unified description of the entire physical SoS and virtual models. However, heterogeneous interfaces and interface management when integrating DTs is supported by HLA. Thus, an integrated simulation framework is developed for generating HLA execution models using an automated transformation algorithm.
To realize this transformation, the ontology of CTs is used to generate federations through a code transformer: 1) Apache Jena is used to load an ontology model representing HLA federations and the topology among them; 2) A generator is developed to traverse the entire loaded model and generate the C++ code for HLA simulation executions. The overall workflow of the transformation is defined as shown in Algorithm 1. Algorithm 1 takes as input the MBSE ontology HLAModels and provides as output the Federation that implements the HLA simulation. All individuals of Graph representing one HLA federation are queried to execute separately to generate the federation file: 1) the individuals of each
After the federations are generated, the HLA simulation is executed through the RTI, which is a time manager application developed in accordance with the HLA interface specification. The RTI provides simulation operational services for simulation execution, such as simulation start, pause, resume, and time synchronization. Moreover, the RTI provides the communication transmission services, which implement each federate independently.
Problem statement of UAV landing.
The process of federation execution is shown in Algorithm 2. Each DTF in this federation is an independent simulation agent and executed simultaneously, and all services used by the DTF are provided by the RTI. The input of federation execution is the federation file, which is generated by Algorithm 1 with the related simulation parameters. Then, some instances of object classes or interaction classes are created during execution. The execution process involves four activities:
Create federation execution: Federation execution, which refers to an integrated simulation process, is created by one DTF. Then, all the other DTFs join in.
Start federation execution: DTF publishes and subscribes to object classes and interaction classes during this process. Then, instances of object classes are created and they exist throughout the federation execution.
Execute federation: In this process, there are two processes of data exchange that are controlled by instances of object classes or interaction classes.
Destroy federation execution: After all data exchange is completed, federation execution is destroyed by DTF and the simulation results are obtained.
Problem statement
As shown in Fig. 6, an SoS consisting of three UAVs, three USVs, and one control center is demonstrated. The UAVs are planning to land on the USVs. Each UAV lands on only one USV. Thus, each UAV has three options for landing. The control center provides decision-making for each UAV regarding which USV they land on. All the USVs and UAVs have their own locations (
The ontology of UAV landing.
All systems in the system have been designed and there are physical pairs of virtual entities. To develop a CT for this target, ontology models for this scenario are first constructed, including a scenario description, HLA model description, and simulation result descriptions (which are created after the simulations). Moreover, the DTs for each UAV and USV are developed separately. One HLA simulation model is developed to support communications among different DTs for implementing an integrated simulation of the landing time. As shown in Table 1, ten datasets are provided to define the initial locations of UAVs, and the fixed locations of USV1, USV2 and USV3 are (0,0,0), (10,10,0) and (20,0,0). Then, HLA simulations are implemented for the six landing scenarios separately (for example, UAV1 landing on USV1, UAV2 landing on USV2, UAV3 landing on USV3) under each set of initial locations. The total time of the entire landing process is captured through the HLA simulation for each landing scenario. Finally, based on the ontology models and DT integration based on HLA, a CT is constructed for operational decision-making of each UAV by the control center. All the landing scenarios are shown in Table 2.
Sets of initial locations of UAVs (km)
Six landing scenarios of UAVs and USVs
HLA model of UAV landing.
To support decision-making of the proposed SoS in the case study, ontology models are built to formalize the SoS using the ontology proposed in Sections 4.2 and 4.3. Then, HLA simulation models are generated from the ontology models as in Section 4.4. After the simulations are executed by the HLA models, the simulation results are synchronized into the ontology models. Finally, the ontology models are used to support decision-making for the SoS operations by reasoning.
As shown in Eq. (4.2), the ontology models include the topology among DTFs in different HLA models, their simulation results, and the real SoS scenarios. The SoS in the case consists of systems including UAV, USV and control center, and includes relevant people. According to BFO and IOF specifications, we have the following definition:
Where UAV, USV and ControlCenter are subclasses of Artifact UAVDriver and USVDriver are subclasses of Person. UAVs, USVs and the control center are individuals of thoes class.
The ontology model for the topology includes instances of Graph, Object, Property, Point, Role, and Relationship according to the landing scenario. Figure 7 is a individual of
There are seven DTFs in this case, three UAV DTFs, three USV DTFs, and one ControlCenter DTF. UAV DTFs publish UAV Object Class and USV DTFs publish USV Object Class, which are both subscribed to by ControlCenter DTF. When the instances of UAV Object Class and USV Object Class, which represent the real entities in the SoS, are moving, ControlCenter DTF receives their coordinate changes. Navigation Interaction Class is published by ControlCenter DTF and subscribed to by UAV DTFs. ControlCenter DTF calculates the acceleration of the UAVs according to the coordinates and the landing scenario, and then sends the acceleration information to the UAVs through the instances of Navigation Interaction Class. Finally, the UAVs land on the USVs according to the expected landing scenario.
As shown in Fig. 8, HLA models are generated from ontology models. The simulation is executed using the algorithms in Section 4.4. The ontology models for the six landing scenarios are used to generate the HLA models separately with 10 sets of initial location information, as shown in Table 1.
When executing the HLA simulation, CERTI (an open-source HLA RTI software) is used to support communications among DTs of UAVs and USVs through their respective DTF simulation applications. Such applications are managed by a run-time infrastructure gateway (RTIG), which is used to control the process of coordinating HLA federates. Each federation must have at least one RTIG process. Each federate simulation application is developed based on the ontology model and CERTI APIs.
Simulation results of UAV landing.
Sixty simulations were executed in total; the overall landing times for each landing scenario under all sets of initial locations are listed in Table 3, and one sample is shown in Fig. 9. From the results, we find that at least one landing scenario has the shortest overall landing time within each set of initial locations of HLA models. Thus, we infer that the landing scenario with the shortest overall landing time is the expected landing process when the UAVs and USVs are at the given initial locations. Based on the simulation results from HLA, the landing scenario and initial coordinates are formalized in ontology models through a developed synchronized Java plugin.
Simulation results of the overall landing time (h)
Through DT integration based on HLA and semantic modeling for the topology of HLA models, simulation results, and the real SoS, the developed CT is used to support real-time decision-making when each constituent system operates in the SoS. In this case, when a set of real-time positions is given, the reasoning of the ontology models for each initial location is used to obtain the optimal landing scenario for each UAV. The process consists of two parts: 1) the preprocessing of the real-time coordinates of each UAV and USV to confirm to which of the 10 sets of sample data (Sample in Table 1) the given real-time coordinate set belongs; 2) the reasoning algorithm is used to query the shortest landing time in the ontology model based on SPARQL queries.
Algorithm 3 is first used to compare the real-time positions of UAVs with the 10 sample datasets to determine which set to use for decision-making. The input elements of the algorithm are RealTime, which represents the set of real-time coordinates of UAVs,
After preprocessing the initial locations in Algorithm 3, the SelectedSample is provided to Algorithm 4. Algorithm 4 is developed to identify the landing scenario with the shortest landing time under the given initial locations through a SPARQL query of the developed ontology model. In the ontology model, the 10 sample datasets of initial locations and their related simulation results under each landing scenario are formalized. bfo:SelectedSample refers to the individual in the ontology that represents the SelectedSample. Through this query, all the six landing scenarios are sorted by their respective landing times, and the query result is the landing scenario with the shortest landing time. Through this query, the expected landing scenario is obtained from the CT. Based on the reasoning results, the control center in the SoS is able to make decisions regarding how each UAV lands on the determined USV.
In the Case Study, we used an SoS consisting of three UAVs, three USVs and a control center as a case. It is a complete SoS, which contains multiple systems, and each system has interaction. The UAVs, USVs and the control center are independent and heterogeneous systems, which makes complexity management and interface integration necessary. The systems need to communicate various information including speed, acceleration and attitude, and decisions need to be made as soon as possible for landing. It is necessary to construct CT ontology for this SoS and support decision-making through reasoning, so as to prove that the proposed CT is workable and meaningful.
Discussion and evaluation
In this section, quantitative and qualitative analyses of the CT are presented from three perspectives in the case study: 1) complexity management of CT for the SoS development; 2) DT integration for each constituent of the system model; 3) decision-making for real SoS operations based on CT.
Quantitative analysis
All the axioms in the entire ontology model for the use case are shown in Table 4. Among them, there are entities for the real SoS. The number of entities for real virtual models is 130. From the axiom of the ontology models, we infer that all the ontology concepts of the real SoS entity and virtual entity are developed. There are 153 individuals in the ontology models, and 46 connections among them. This is used to represent the topology among entities and manage the complexity of SoS development and operation.
OWL classes in the use case
OWL classes in the use case
In the case study, there are seven DTs including three UAVs, three USVs, and a control center in the SoS, as shown in Table 5. Moreover, all the corresponding DTs are integrated in one federation by their respective DTFs based on the HLA interface specification. This HLA federation is executed 60 times with 10 sample datasets and six landing scenarios. From Table 5, we find that all the DTs are integrated to predict the overall landing time of each landing scenario. Moreover, during the entire generation process of the HLA model, all the HLA DTFs and the federation are generated and implemented automatically.
DT and HLA concepts
In the case study, all 60 landing processes under different initial locations and different landing scenarios are executed, whose results are defined as individuals in the ontology model. As shown in Table 6, 60 individuals of the landing processes and 60 individuals of the simulation results are created. Those individuals are the cues for implementing a reasoning algorithm in Section 5.4, which is used to support real-time decision-making in the SoS. When a set of real-time positions of the UAVs and USVs is given, there are six landing scenarios as decision options. After the query, two individuals, of which one represents the optimal landing scenario and the other represents the shortest landing time, are given. The average time of this query is 0.15 s. Through this query, the optimal landing scenario is obtained through reasoning from 120 individuals.
Individuals for reasoning and query result
When constructing the CT, the CT ontology is definded by Eq. (4.2). All entities of SoS have been aligned to the BFO and IOF specification. Physical entities of UAV and USV are mapped to individuals of class Artifact; person of SoS is mapped to individuals of class Person; the simulation models and results are mapped to individulas of class HLAModel and SimuResults. Through this unified ontology, the topology between the virtual entities and real SoS is defined as a basis to implement reasoning to make decisions for the SoS operations. Moreover, the unified ontology provides a bridge between the real-world entities and their DTs, because the related ontology models provide readable information for the computer to distinguish and describe the virtual DT and the real world. Furthermore, the unified ontology provides better scalability to extend the CT to other scenarios in the future: 1) the adopted BFO ontology is one of the most powerful upper-level ontologies, which has been used by the IOF (
When constructing the CT, heterogeneous DTs are integrated through HLA specifications. When implementing HLA simulations, heterogeneous DTs are connected to the RTI by HLA federates. Through the RTI, run-time and synchronized communications can be managed and controlled among different DTs. Such HLA specifications are the basis to construct a unified platform to integrate different DTs of constituent systems in the SoS and implement simulations among them. Traditioal SoS development based on HLA has complex simulation model construction process and lack of information exchange with system entities [37]. Compared with it, CT ontology provides the formalization of HLA models to support the graphical construction of HLA federations, which greatly improves the efficiency of simulation development. This also provides a potential model-driven solution for SoS development and operation.
The CT is used to make decisions for SoS operations in the case study. First, the integrated simulations among the DTs are implemented to predict the performances of each UAV in different landing scenarios. Such simulation results are used as cues to develop ontology models in the CT. Through reasoning applied to the ontology models, one decision-making option representing a landing scenario with the least overall landing time is obtained. Based on this option, when the real SoS operates, the control center is able to manipulate each UAV for its landing target with the least landing time. Compared with traditional SoS development, the developed CT can use the simulation results to support decision-making during the real SoS operations, thereby connecting the virtual DTs to the real SoS operations.
Summary
The CT is an emerging concept that has been proposed by several studies [33, 38]. However, there is a lack of basic frameworks to support CT development and applications, particularly for SoS development and operation [38, 39]. In this study, we proposed a CT framework for decision-making for SoS operations based on DT integration and reasoning. The proposed framework uses the BFO and GOPPRRE ontologies to formalize real SoSs, their virtual models, and the topology between them. Moreover, through HLA specifications, heterogeneous DTs are integrated to implement simulations to predict the behaviors of each constituent system in the SoS. Finally, based on the simulation results, ontology models are used to provide reasoning results for decisions regarding the real SoS operations. From the case study, we found that the proposed CT approach enables decision-making for SoS operations with advanced features, which was confirmed by both quantitative and qualitative analyses.
Conclusion
This study proposes a CT to support DT integration and decision-making across systems when implementing SoS development and operation. For challenge 1, a semantic modeling approach is proposed to support a unified DT description in order to manage the complexity. The GOPPRRE approach supports the formalization of the systems, BFO and IOF specifications provide the integration of formalization of different domains to support complexity management. For challenge 2, semantic models are transformed into HLA execution models for integrated simulation among DTs for each constituent system in the real SoS through HLA-RTI. Through reasoning applied to the semantic models and simulation results, CT enables the support of SoS operations by providing decision-making options and solves challenge 3.
In conclusion, our research makes three main contributions:
We proposed a new CT concept for supporting SoS development and operations. The CT employs semantic models to manage the complexity of DTs across systems by using unified descriptions. The CT provides an integrated simulation infrastructure for managing the interfaces between heterogeneous DTs and implementing data exchange among them. The CT offers cognitive capabilities based on semantic models and results from HLA simulations. Through reasoning applied to the semantic models, the CT enables decision-making support for real SoS operations.
From the case study, our approach indeed promotes complexity management capabilities for DTs across systems using a unified semantic model. Moreover, HLA is an infrastructure that supports integrated simulation across DTs, which promotes the interoperability of heterogeneous DTs via a unified interface specification. It allows SoS developers to construct DTs for their own systems without considering the limitations of heterogeneous data structures. Furthermore, reasoning based on semantic models and results from integrated simulations enables the provision of decision-making options for manipulating system behaviors of constituent systems during SoS operations. This provides a new model-based solution to construct SoSs for autonomous systems. In the future, a complete toolkit for ontology modeling, distributed simulation, and visualization for SoS development and operation will be developed to enable automatic distributed simulation execution, DT management, and decision-making support for real-time SoS operations.
Footnotes
Available at https://gitee.com/zkhoneycomb/open-share/tree/lihan/Papers.
Acknowledgments
This work was supported by The National Key Research and Development Program of China (Grant No. 2020YFB1708100). Thanks to the IoF Systems Engineering Working Group to support the ontology specification.
