Ontology geometry distance computation using deep learning technology

1 Introduction

As a structured data storage, processing, analysis and calculation model, ontology is widely used in the mainstream of computer such as information retrieval, graphic image processing, pattern recognition and granularity computing. At the same time, due to its powerful auxiliary functions, it is applied to many disciplines such as physics, chemistry, biology, pharmacy, medicine and materials science. With the deepening research, new techniques are constantly updated and applied to practice, making the role of ontology more and more effective. See Swanson [1], Teymourlouie et al. [2], Viani et al. [3], Hippolyte et al. [3], Roth and Jornet [5], Cooper et al. [6], Pfaff and Krcmar [7], Travin et al. [8], Seipel et al. [9], and Blondet et al. [10] for more details.

In particular, ontology is a structured set of concepts that are related by certain form of structure. In general, the ontology of this association structure can be represented as the graph model. Each vertex in the graph corresponds to a concept in ontology, and the edge between two vertices indicates that there is some direct superior-subordinate relationship between the two concepts. Combined with learning theory, the goal of ontology learning is to obtain an ontology function that is used to calculate the similarity between ontology vertices. That is, by learning of ontology samples, we get S : V × V → ℝ . But the problem is that the description of concepts in the general ontology is an individual description, not a paired description. Therefore, it is not convenient to obtain an ontology sample set of paired vertex similarity without the help of field experts. On the other hand, the function S is not intuitive and can’t express accurately the meaning of the whole ontology structure. A more reasonable way is through the sample set to learn the optimal ontology function f : V → ℝ which maps each vertex to a real number, and the similarity between two vertices v _i and v _j can be measured by |f (v _i ) - f (v _j ) |. The lager |f (v _i ) - f (v _j ) | is, the small similarity between v _i and v _j ; on the contrary, the smaller |f (v _i ) - f (v _j ) | is, the higher similarity between v _i and v _j . Several ontology related algorithms and applications can refer to Hashemi et al. [11], Abdi et al. [12], Mesmer and Olewnik [13], Pan et al. [14], Rizzo et al. [15], Maldonado et al. [16], Ansari [17], Fouda et al. [18], and Zhu et al. [19].

Several papers contribute on the ontology learning algorithm from the theory and application point of view. Gao and Xu [20] presented the stability analysis of learning algorithms for ontology similarity computation. Gao et al. [21] studied the strong and weak stability of k-partite ranking based ontology learning algorithm. Gao et al. [22] proposed the ontology learning algorithm for similarity measuring and ontology mapping by means of linear programming. Gao and Farahani [23] determined the Generalization bounds and uniform bounds for multi-dividing ontology algorithms with convex ontology loss function from a statistical point of view. Gao et al. [24] discussed the distance learning tricks for ontology similarity measuring and ontology mapping.

In this paper, we focus on the ontology learning algorithm based on the geometry distance computation and deep learning technique. The rest of the paper is organized as follows: first we introduce the geometry distance calculating setting in ontology problem, and review the ontology distance computation algorithm which is projected on the positive cone in a reproducing kernel Hilbert space; then, we propose our main ontology geometry distance learning algorithm using deep learning neural networks; finally, we verify the effectiveness of the algorithm by several experiments.

2 Overview of geometry distance computation

In order to represent the ontology algorithm in a mathematical model, the information of each vertex on ontology graph is represented by a p-dimensional vector, i.e., v ∈ ℝ p . In this way, the similarity between the vertices of the ontology transforms to the geometric distance between the vectors. The most primitive way to calculate the geometric distance between vectors is to calculate the Euclidean distance between two vectors. However, this calculation method can’t reflect the essential relationship between ontology concepts. In this paper, the calculation of the optimal geometric distance d is obtained by studying ontology samples. Let’s begin from the notation and the setting of geometry distance computation. In what follows, we always assume V as an ontology set, as well as the ontology information matrix. Without causing confusion, we use the notion v (or u) to express ontology vertex, its corresponding vector and its corresponding ontology concept, which are no longer represented in bold.

Let I _n be the n × n identity matrix. The Frobenius norm of a matrix V is ∥ V ∥ F = Tr ( V T V ) , where Tr (V) is the trace of matrix V. The space of n × n symmetric positive definite matrices is denoted by S + + n . A function d : V × V → R⁺ can be regarded as an ontology distance function if for any v₁, v₂, v₃ ∈ V, it satisfies non-negativity (d (v₁, v₂) ≥0), distinguishability (d (v₁, v₂) =0 ⇔ v₁ = v₂), symmetry (d (v₁, v₂) = d (v₂, v₁)) and triangle inequality (d (v₁, v₃) ≤ d (v₁, v₂) + d (v₂, v₃)).

In this fashion, the class of Mahalanobis distances with M ∈ S + + n , W ∈ ℝ d × p (d ≤ p) and M = W ^T W can be defined as d M ( v 1 , v 2 ) = ( v 1 - v 2 ) T M ( v 1 - v 2 ) = ( v 1 - v 2 ) T W T W ( v 1 - v 2 ) = ∥ W v 1 - W v 2 ∥ 2 . It implies that learning a Mahalanobis ontology distance metric M can be transformed to determine a linear transformation W which projects each ontology vertex v ∈ ℝ p into a low-dimensional subspace, but keeping the geometric distance of two ontology vertices in the transformed space equal to the Mahalanobis distance in the original vector space.

In this section, we mainly overview the distance calculating based ontology learning algorithm connected to a reproducing kernel Hilbert space. In this setting, the training data can be denoted as ( u i , v i , y i ) i = 1 n where u i , v i ∈ ℝ p which correspond to certain ontology vertex, l _i ∈ {0, 1} with l _i = 1 if (u _i , v _i ) is the pair of similarity ontology vertices (concepts), and l _i = 0 if (u _i , v _i ) is the pair of dissimilarity ontology vertices (concepts). Let L₁ be the number of samples with l _i = 1 and L₀ be the number of samples with l _i = 0. Set Ξ d = 1 L 0 ∑ i : l i = 0 ( u i - v i ) ( u i - v i ) T , Ξ s = 1 L 1 ∑ i : l i = 1 ( u i - v i ) ( u i - v i ) T . Hence, a dissimilarity hypothesis can be stated as

ϱ ( u i , v i ) = log { 1 2 π | Ξ d | exp ( - 1 2 ( u i - v i ) T Ξ d - 1 ( u i - v i ) ) 1 2 π | Ξ d | exp ( - 1 2 ( u i - v i ) T Ξ s - 1 ( u i - v i ) ) } .

A large ϱ (u _i , v _i ) implies that u _i and v _i are dissimilar, and vice-versa. Let Proj (·) be a projection to the cone of positive definite matrices. Then, the Mahalanobis matrix in ontology setting can be obtained by M = Proj ( Ξ s - 1 - Ξ d - 1 ) where the projection is deduced by eigen-decomposition of Ξ s - 1 - Ξ d - 1 as UDU ^T then M = UD₊U ^T with D₊ = diag (max(d _i , ɛ)), D = diag (d _i ) and ɛ > 0 is a very small real number.

Let k : V × V → ℝ be a positive definite kernel defined on ontology space V. In terms of the Mercer theorem, a mapping φ : V → H to a Reproducing Kernel Hilbert Space (RKHS) H exists for any positive definite kernel. In this way, the purpose is to transform to get a Mahalanobis distance d H : H × H → ℝ + associated with H, which can be expressed by d H ( v 1 , v 2 ) = ( φ ( v 1 ) - φ ( v 2 ) ) T M H ( φ ( v 1 ) - φ ( v 2 ) ) . Moreover, the notations in this setting are re-written as Ξ H , d = 1 L 0 ∑ i : l i = 0 ( φ ( u i ) - φ ( v i ) ) ( φ ( u i ) - φ ( v i ) ) T , Ξ H , s = 1 L 1 ∑ i : l i = 1 ( φ ( u i ) - φ ( v i ) ) ( φ ( u i ) - φ ( v i ) ) T , ϱ H ( u i , v i ) = log { ( e - 1 2 ( φ ( u i ) - φ ( v i ) ) T Ξ H , d - 1 ( φ ( u i ) - φ ( v i ) ) ) 2 π | Ξ H , d | / ( e - 1 2 ( φ ( u i ) - φ ( v i ) ) T Ξ H , s - 1 ( φ ( u i ) - φ ( v i ) ) 2 π | Ξ H , d | ) } . And the clipping of the spectrum can be obtained by M H = Proj H ( Ξ H , s - 1 - Ξ H , d - 1 ) .

Next, we show how to get Ξ H , s - 1 and Ξ H , d - 1 . Given a set of ontology vertex pairs { ( u i , v i ) } i = 1 n , set Z = (u₁, ⋯, u _n , v₁, ⋯, v _n ) and J J T = 1 n [ I n - I n - I n I n ] , then we infer Ξ = 1 n ∑ i = 1 n ( u i - v i ) ( u i - v i ) T = Z J J T Z T . Set Φ _Z = (φ (u₁), ⋯, φ (u _n ), φ (v₁), ⋯, φ (v _n )), then the |H| dimensional covariance matrix Ξ _H associated with RKHS H is stated as Ξ H = Φ Z J J T Φ Z T . The kernel matrix of Z is denoted as K Z ∈ ℤ 2 n × 2 n , which is defined by [K _Z ] _ij equals to k (u _i , u _j ) if i, j ≤ n; k (v _i , v _j ) if i, j > n; and k (u _i , v _j ) in the other case. Let the SVD decomposition of J T Φ Z T Φ Z J = J T K Z J be X Z ϒ Z X Z T . Set ι > 0 be a parameter and W Z = J X Z I 2 n - ι ϒ Z - 1 . Then, the regularized estimate of Ξ _H can be formulated as Ξ ˆ H = Φ Z W Z W Z T Φ Z T + ι I H . Hence, we obtain Ξ ˆ H - 1 = ( Φ Z W Z W Z T Φ Z T + ι I H ) - 1 = 1 ι ( I H - Φ Z W Z ϒ Z - 1 W Z T Φ Z T ) and Ξ ˆ s , H - 1 - Ξ ˆ d , H - 1 = 1 ι ( Φ Z d W Z d ϒ Z d - 1 W Z d T Φ Z d T - Φ Z s W Z s ϒ Z s - 1 W Z s T Φ Z s T ) .

However, Ξ ˆ s , H - 1 - Ξ ˆ d , H - 1 can’t be computed directly. We need the other trick to solve it. Set C ∈ S + + n , otn as the n ontology training vertices, A s = W Z s ϒ Z s - 1 W Z s T and A d = W Z d ϒ Z d - 1 W Z d T . We aim to solve the following projection problem which onto the cone of positive definite matrices in H: argmin C ∈ S + + n L ( C ) = ∥ Φ otn C Φ otn T + Φ Z s A s Φ Z s T - Φ Z d A d Φ Z d T ∥ F 2 .

By computation, we yield L ( C ) = Tr ( K otn C K otn C ) + 2 Tr ( K Z s , otn C K Z s , otn T A s ) - 2 Tr ( K Z d , otn C K Z d , otn T A d ) + C , where C is a constant. By setting ▽ _C L (C) =0, we get

C * = K otn - 1 ( K Z d , otn T A d K Z d , otn - K Z s , otn T A s K Z s , otn ) K otn - 1 . Finally, the Mahalanobis distance in H can be described as

d H ( u , v ) = ( φ ( u ) - φ ( v ) ) T Φ otn C Φ otn T ( φ ( u ) - φ ( v ) ) = ( k u , otn C k u , otn T - 2 k u , otn C k v , otn T + k v , otn C k v , otn T ) 1 2 .

3 Deep learning based ontology distance calculating algorithm

Deep learning is a new research field in machine learning. In recent years, it has made breakthrough progress in large kinds of applications such as speech recognition and computer vision. The motivation of this model is to establish a model that simulates the neural connections of the human brain. When dealing with such signals as images, sounds and texts, the data features are described by stratification through multiple transformation stages, and the data interpretation is obtained (related studies can refer to Phong et al. [25], Hassan et al. [26], Oh et al. [27], Proenca and Neves [28], Biswas et al. [29], Ren et al. [30], Rad et al. [31], Lore et al. [32], Bianco et al. [33], Olmos et al. [34], and Treder et al. [35]). In this section, we introduce our main ontology distance function learning algorithm based on the deep neural network learning technique.

Assume that there are M + 1 layers in the designed network and d^(m) units in the m-th layer, where m ∈ {1, 2, ⋯, M}. Let W ( 1 ) ∈ ℝ d ( 1 ) × p be a projection matrix to be learned in the first layer, b ( 1 ) ∈ ℝ d ( 1 ) be a bias vector and s : ℝ p → ℝ p be a component-wisely nonlinear activation function. For a fixed ontology vertex v ∈ ℝ p , the output in the first layer is then given by h ( 1 ) = s ( W ( 1 ) v + b ( 1 ) ) ∈ ℝ d ( 1 ) . Let h⁽¹⁾ be the output of the first layer and as well as the input of the second layer.

Similarly, the projection matrix, bias, and nonlinear activation function in the second layer are denoted by W ( 2 ) ∈ ℝ d ( 2 ) × d ( 1 ) , b ( 2 ) ∈ ℝ d ( 2 ) and s, respectively. Then, the the output of the second layer is determined by h ( 2 ) = s ( W ( 2 ) h ( 1 ) + b ( 2 ) ∈ ℝ d ( 2 ) . Thus, the output of the m-th layer can be formulated by h ( m ) = s ( W ( m ) h ( m - 1 ) + b ( m ) ) ∈ ℝ d ( m ) and the output in the top level is f ( v ) = h ( M ) = s ( W ( M ) h ( M - 1 ) + b ( M ) ) ∈ ℝ d ( M ) , where parametric nonlinear function f : ℝ p → ℝ d ( M ) obtained by W^(m) and b^(m) for m ∈ {1, ⋯, M}. For a pair of ontology vertices v _i and v _j , then f ( v i ) = h i ( M ) , f ( v j ) = h j ( M ) , and d f 2 ( v i , v j ) = ∥ f ( v i ) - f ( v j ) ∥ 2 2 .

Define the threshold parameter τ₁, τ₂ with τ₂ > τ₁ > 0, and τ > 1 is related to τ₁ and τ₂. We use l ij ( τ - d f 2 ( v i , v j ) ) > 1 to enforce the margin between τ and d f 2 ( v i , v j ) , where τ = τ₁ + 1 = τ₂ - 1, l _ij = 1 if v _i and v _j are similar and l _ij = -1 if v _i and v _j are dissimilar. Let λ be a balance parameter, β be a sharpness parameter, [x] = max {x, 0}, g ( z ) = log ( 1 + e z β ) β .Then the ontology optimization problem can be expressed as: argmin f J = ∑ i , j g ( 1 - l ij ( τ - d f 2 ( v i , v j ) ) ) 2(1) + λ 2 ∑ m = 1 M ( ∥ W ( m ) ∥ F 2 + ∥ b ( m ) ∥ 2 2 ) . Let ⊗ be the element-wise multiplication, c = 1 - l ij ( τ - d f 2 ( v i , v j ) ) , and z i ( m ) = W ( m ) h i ( m - 1 ) + b ( m ) . For m ∈ {1, ⋯, M - 1}, we set Θ ij ( M ) = g ′ ( c ) l ij ( h i ( M ) - h j ( M ) ) ⊗ s ′ ( z i ( M ) ) , Θ ji ( M ) = g ′ ( c ) l ij ( h j ( M ) - h i ( M ) ) ⊗ s ′ ( z j ( M ) ) , Θ ij ( m ) = ( ( W ( m + 1 ) ) T Θ ij ( m + 1 ) ) ⊗ s ′ ( z i ( m ) ) , Θ ji ( m ) = ( ( W ( m + 1 ) ) T Θ ji ( m + 1 ) ) ⊗ s ′ ( z j ( m ) ) . In order to solve the ontology optimization problem (1), we apply the stochastic sub-gradient descent trick to obtain the parameters {W^(m), b^(m)} for m ∈ {1, 2, ⋯, M} with original inputs h i ( 0 ) = v i and h j ( 0 ) = v j as follows:

∂ J ∂ W ( m ) = ∑ i , j ( Θ ij ( m ) ( h i ( m - 1 ) ) T + Θ ji ( m ) ( h j ( m - 1 ) ) T ) + λ W ( m ) ,(2)

∂ J ∂ b ( m ) = ∑ i , j ( Θ ij ( m ) + Θ ji ( m ) ) + λ b ( m ) .(3) Let μ be the learning rate, the whole procedure can be expressed as follows: Algorithm: Deep learning based ontology distance function learning algorithm Input: ontology sample data S = {(v _i , v _j , l _ij )}, the number of network layers M + 1, threshold τ, learning rate μ, iterative number T, balance parameter λ, convergence error ɛ. Initialize: set bias b^(m) as zero vector, let d⁽⁰⁾ be the dimension of input layer and m ∈ {1, ⋯, M}, we set W ( m ) ∼ U [ - 6 d ( m ) + d ( m - 1 ) , 6 d ( m ) + d ( m - 1 ) ] . For t = 1, ⋯, T, do   Randomly choose an ontology vertex pair (v _i , v _j , l _ij ) from S;   Set h i ( 0 ) = v i and h j ( 0 ) = v j ;   For m = 1, ⋯, M, do     Infer h i ( m ) and h j ( m ) in light of forward propagation.   End For   For m = M, M - 1, ⋯, 1, do     Yield the gradient by means of (2) and (3).   End For   For m = 1, ⋯, M, do W ( m ) = W ( m ) - μ ∂ J ∂ W ( m ) , b ( m ) = b ( m ) - μ ∂ J ∂ b ( m ) . End For   Determine J _t in view of (1);   If t > 1 and |J _t - J_t-1| < ɛ, then break. End For Output: { W ( m ) , b ( m ) } m = 1 M .

In this section, we present four experiments to show the effectiveness of our proposed ontology learning algorithm.

Deep learning is also known as unsupervised feature learning, in which features are extracted without human design and features are learned from the data. Depth learning is essentially a non-linear combination of the methods of representation learning. It indicates that learning refers to learning representations (or features) from data to extract useful information in the data when categorizing and predicting. Depth learning begins with raw data and transforms each representation (or feature) layer by layer into a higher-level, more abstract representation, thereby discovering the intricate structure of high-dimensional data. In this paper, we design an ontology distance learning algorithm by means of deep learning. It is applied to ontology similarity measuring and ontology mapping, and been used in various engineering applications.

Conflict of Interests

The authors hereby declare that there is no conflict of interests regarding the publication of this paper.