Fundamental ideas and mathematical basis of ontology learning algorithm

Abstract

As a data utility and aided tool, ontology has been widely used in many areas of the computer. Owing to its great efficiency, ontologies have also been introduced into various engineering disciplines. In this paper, we present the fundamental ideas of how to deal with similarity measuring problem in ontology learning algorithms. The mathematical basis of ontology learning algorithms is also introduced from a statistical learning theory point of view. Finally, we present two ontology learning algorithms in multi-dividing setting and ontology sparse vector learning setting, respectively.

Keywords

1 Introduction

The term “ontology”, derived from philosophy, was introduced into the field of artificial intelligence in the early 1990 s. As a model of conceptual semantic storage, analysis and management, it has drawn great attention from the field of computer science and information technology. In the 21st century, scholars from various disciplines are using ontology tools to cope with various engineering problems, making the ontology popular in multi-science research as well, such as biology (Koehler et al. [1]), pharmacy (Przydzial et al. [2] and Bhattacharjee et al. [3]), education system (Demartini et al. [4]), psychology (Simut [5]), medicine (Shah et al. [6], Zghal and Moreno [7] (Mignard and Nicolle [8], and Brisaboa et al. [9]), neuroscience (Bowden et al. [10]), and nanotechno-logy.

In recent years, ontology methods have been applied to various ontology engineering applications. Ashraf et al. [12] discussed OUSAF (Ontology USage Analysis Framework) which is often used in large RDF data. Ongsri and Roddick [13] considered combining ontologies into three popular conceptual model approaches. Sorokine et al. [14] proposed a new ontology-driving-based information system. Groza et al. [15] explained the semantic unification in human phenotype ontology of common and rare diseases. Cannataro et al. [16] raised an ontology-based algorithm for fixed genome sequencing and gene sequencing based on the semantic similarity of each gene corresponding disease.

In general, the ontology structure is represented by the graph model G = (V, E), where each vertex on the ontology represents a concept of ontology. Each edge e = v_iv_j in the graph indicates that there is a direct relationship between the concepts v_i and v_j. The similarity between the two concepts in ontology can be expressed by the similarity of vertices. The similarity of vertices v_i and v_j is denoted by Sim (v_i, v_j), whose value is a real number between [0,1]. Sim (v_i, v_j) = 1 indicates that the concepts corresponding to v_i and v_j are the same. Sim (v_i, v_j) = 0 means that there is no relationship between v_i and v_j. The threshold M is given by experts in the field. When Sim (v_i, v_j) > M, the similarity between v_i and v_j is greater than M, and then it returns a set of concepts with higher similarity to the user.

If we consider this problem from a mathematical perspective, the goal of ontology similarity calculation is to find the optimal similarity calculation function $Sim : V \times V \to ℝ^{+} \cup {0}$ , which maps each pair of vertices to non-negative real numbers and consider the size of real numbers to determine the similarity between ontology vertices. For ontology mapping, the goal is to establish a bridge between different ontologies, the essence of which is to calculate the similarity between the vertices of different ontologies. Let G₁ and G₂ be two ontology graphs corresponding to two different ontologies O₁ and O₂ respectively. For each vertex v ∈ G₁, a set of vertices S_v ⊆ V (G₂) is found, where the concepts corresponding to the vertices in the set all have larger semantic similarity with the concepts corresponding to the v vertices. One way to get ontology mapping is to compute the similarity S (v, v_j) for each vertex v ∈ G₁, where v_j ∈ V (G₂), and select the parameter 0 < M < 1. Then S_v is the set of elements whose elements satisfy S (v, v_j) ⩾ M. From this point of view, the essence of ontology mapping is to obtain the similarity calculation function S and select the appropriate parameter M.

Ontology similarity calculation is the key of ontology algorithm and subject application. Early ontology similarity calculation methods are mostly heuristic. To sum up, these heuristic calculation methods follow the pattern below:

$\begin{matrix} Sim (v_{i}, v_{j}) = α_{1} {Sim}_{name} (v_{i}, v_{j}) + α_{2} {Sim}_{ins tan ce} (v_{i}, v_{j}) \\ + α_{3} {Sim}_{attribute} (v_{i}, v_{j}) + α_{4} {Sim}_{structure} (v_{i}, v_{j}) \end{matrix}$ (1)

where α_i (1 ≤ i ≤ 4) are all real numbers, and the values are between 0 and 1. These equilibrium parameters are given by experts in the field or empirically determined, and satisfy α₁ + α₂ + α₃ + α₄ = 1. Sim_name (v_i, v_j) represents the name similarity between the concepts corresponding to v_i and v_j. Sim_instance (v_i, v_j) represents the instance similarity between the concepts corresponding to v_i and v_j, and Sim_attribute (v_i, v_j) represents the attribute similarity between the concepts corresponding to v_i and v_j. Sim_structure (v_i, v_j) represents the structural similarity between concepts corresponding to v_i and v_j. In other words, the similarity between v_i and v_j is weighted by the similarity of their names, instances, attributes and structures. In the formula for calculating the similarity of ontology vertices, there will be other parameters to be determined by specialists in the field. Traditionally, the parameters that are multiplication coefficients in α_i(1 ⩽ i ⩽ 4) and partial ontology similarity calculation formulas are called the first type of parameters, and the parameters inside the partial ontology similarity calculation formula are called the second type of parameters (for example, equilibrium coefficient appeared in a part of the vertex similarity calculation formula molecules). The shortcomings of this heuristic ontology similarity calculation model above lie in: 1) almost all the parameters need to be decided by experienced domain experts in practical operation. When they are unable to find domain experts, the given parameters may be unreasonable, and the gap is relatively large, which will directly affect the efficiency of the algorithm; 2) This method to calculate the similarity in pairs is not clear and direct, and it needs a large amount of computation. Meanwhile, a detailed analysis for ontology structures needs to be done in advance.

To sum up, under the background of big data, the heuristic method of calculating the similarity formula has great limitations. In order to solve the above problems and break through the restriction of traditional given parameters ontology calculation model, the machine learning method is widely applied to the ontology similarity calculation. The core idea of ontology conceptual similarity calculation algorithm based on machine learning method can be described as follows: According to the learning of sample points, the optimal ontology function $V \to ℝ$ is obtained, which maps the vertices on each ontology map into real numbers and then the whole ontology map will be mapped to the real axis. On the other hand, the similarity between the corresponding vertices of the ontology concepts is described by their one-dimensional distance on the real axis. The larger the distance is, the lower the similarity is and the smaller the distance is, the higher the similarity is.

2 Two kinds of ontology algorithms

In this section, we introduce two main ontology algorithms in multi-dividing ontology setting and ontology sparse vector computing setting, respectively.

2.1 Ontology algorithm in multi-dividing setting

In recent years, many scholars have proposed a variety of techniques for calculating ontologysimilarity and ontology mapping, and used them in multidisciplinary projects. One of the methods is to use the ontology graph structure to get all the vertices of the ontology graph divided into k parts for k levels. The level of value is determined by domain experts. In this framework, for ontology function f, the real number corresponding to the vertex in level a is greater than the real number corresponding to the vertex in any level b, where 1 ⩽ a < b ⩽ k. At last, the similarity between ontology vertices is determined by the absolute value of the difference between their corresponding real numbers. Known as multi-dividing based ontology learning algorithms, these algorithms have been proven to be applicable to many engineering fields. The following two examples are listed to illustrate how the algorithm is applied to the project.

Example 1.Figure 1 shows the well-known “GO” ontology in the field of genetics. This ontology can be viewed as a dictionary to store the corresponding concepts, as well as the interrelationships among the concepts, and its structure is a tree structure. All concepts can be divided into three parts, “Molecular function”, “Biological process” and “Cellular Component”, which correspond to the three branches of the entire tree ontology graph. The corresponding positions of the following concepts are determined according to its divisions, so this structure is a hierarchical structure. It can be found that the similarities between concepts belonging to the same branch are greater than the similarities between concepts belonging to different branches. Therefore, the ontology learning method based on the idea of multi-dividing is very suitable for the gene “GO” ontology, in which case the value of k is 3, and the three levels respectively correspond to the above three branches “Molecular function”, “Biological process” and “Cellular Component”.

Fig.1

Genetics “GO”ontology graph.

Example 2. The famous “PO” ontology in Phytology, shown in Fig. 2, can also be viewed as a dictionary to store corresponding botanical concepts, as well as the interrelationships among concepts, also structured as a tree. All the concepts can be divided into two parts, “Plant anatomical entity” and “Plant structure development stage”, which correspond to the two branches of the entire tree Phytology ontology graph. The corresponding positions of the following concepts are determined according to its divisions, so this structure is also a hierarchical structure. Similarly, the similarity between conceptsbelonging to the same branch is greater than the similarity between concepts belonging to different branches. Thus, the ontology learning method based on the idea of multi-dividing is also suitable for Phytology “PO” ontology, in which case the value of k is 2, and the two levels respectively correspond to the above two branches “Plant anatomical entity” and “Plant structure development stage”.

Fig.2

Phytology “PO”ontology graph.

Example 3.Figure 3 shows the structure of the “University” ontology O₁. It is observed that the ontology is a tree structure, and the three branches are “Courses”, “Student” and “Academic Staff” respectively. Using multi-dividing to calculate the similarity of the ontology, we assume that k = 3, and the three branches correspond to three levels respectively. Because “Student” and “Academic Staff” are humans, the similarities are obviously similar, so the three levels can be arranged as: Courses (level 1), Student(level 2), and Academic Staff (level 3).

Fig.3

“University” Ontology O₁.

Example 4.Figure 4 shows the tree structure of “University” ontology O₂. The two branches correspond to “Courses” and “People”, respectively. Our goal is to establish an ontology mapping between O₁ and O₂. For this purpose, the hierarchy of the two ontologies needs to be unified. Thus, assume k = 4, the branches “Course” in O₁ and O₂ are in the same level. The corresponding ranking order will be “Courses”(level 1), “Student”(level 2), “Academic Staff”(level 3), “People”(level 4).

Fig.4

“University” Ontology O₂.

It can be seen from the above four examples, when the ontology structure is a tree, the multi-dividing framework is suitable for ontology similarity calculation and ontology mapping. However, in the field of engineering applications, most of the ontology graphs are tree-structured for the needs of hierarchical representation, so the ontology function learning algorithm under the multi-dividing framework can be well implemented in practice.

In the early years, there had been some theoretical analysis results for the algorithm of multi-dividing ontology learning (see Gao et al. [17 –27], Zhu et al. [28], Maldonado et al. [29], Ansari et al. [30], and Fouda et al. [31] for more details).

2.2 Ontology algorithm in sparse vector learning setting

In ontologies related to engineering such as physics, chemistry, materials, medicine and pharmacy, due to the complex molecular structure information of genes and others, semantic information and related structures, genes, information to be contained in the ontologies are quite huge. As a result, the design of ontology learning algorithm has higher requirements in terms of algorithm complexity and algorithm optimization. In order to obtain the similarity calculation results quickly and efficiently, the sparse algorithm learning is introduced into ontology similarity calculation and ontology mapping. Based on a particular application domain and the special structure information of the ontology graph under this background, an ontology sparse vector learning algorithm suitable for a specific application background is designed, so as to obtain the optimal ontology function, and then obtain the similarity calculation method between ontology concepts. It is the trend of the design of ontology learning algorithm in the future big data framework.

In the representation of the ontology concept information, a vector is often used to characterize all the information corresponding to the concept. Due to the complexity of information, the dimension of this vector may be very large in some specific application fields. The essence of ontology function is a dimensionality reduction operator, which uses high-dimensional data to represent one-dimensional real numbers. In this sense, the essence of ontology learning algorithm is dimensionality reduction algorithm. Based on the ontology algorithm of ontology sparse vector, the basic idea is that for a specific application area, effectively shield irrelevant feature information, thereby enhancing the valuable information. For example, in genetics, a genetic disease associated with only a few genes, and millions of other genes in the human body has no connection with the disease, the goal of ontology sparse vectors is to effectively find these genes that have decisivesignificance.

In Mathematics, a sparse vector is defined as a vector whose major components are 0, and then it multiplied the corresponding vector of any one of the ontology concept. Then, what really works is only the key information that corresponds to a small portion of the original ontology vector.

3 Introduce of fundamental of statistical machine learning

3.1 Some examples of machine learning

This main algorithm is based on statistical learning theory to analyze machine learning based ontology algorithm. We first give a few examples to illustrate what machine learning is.

Example 5. Linear regression is one of the most common examples of learning. Its goal is to find a straight line that best approximates an objective function, which in turn can effectively predict the set of data points in a plane $ℝ^{2}$ :

${(x_{i}, y_{i})}_{i = 1}^{n} = {(x_{1}, y_{1}), (x_{2}, y_{2}), \dots, (x_{n}, y_{n})} .$ (2)

Traditionally, to measure the approximation effect, we set a straight line with the least squares: $Y = aX + b,$ (3)

and the best straight line is the one makes $\sum_{i = 1}^{n} ({ax}_{i} + b - y_{i})^{2}$ become the shortest line.

Example 6. Extending the example of linear regression in Example 5 to the general case is to find a curve that can best represents the given data set. Suppose the predicted function be $f : ℝ \to ℝ$ , the selected function space follows a certain rule and consists of N real-valued parameters. For example, f is a polynomial with d degrees, then N = d + 1 and these parameters w₀, w₁, ⋯, w_d are unknown coefficients of the polynomial f. In this case, the suitable function f is learned from the data set ${(x_{i}, y_{i})}_{i = 1}^{n} = {(x_{1}, y_{1}), (x_{2}, y_{2}), \dots, (x_{n}, y_{n})}$ by using the square-minimum method. However, in the exact data set, the value of y_i happens to be exactly y_i = f (x_i) + ɛ, but in most cases, the data has a noise-dependent error, then y_i = f (x_i) + ɛ, where the error term depends on a random variable whose x_i has an average value of zero.

And then the smallest square can be expressed as minimized: $\sum_{i = 1}^{n} (f_{w} (x_{i}) - y_{i})^{2}$ (4)

where: $f_{w} (x_{i}) = \sum_{j = 0}^{d} w_{j} x^{j} .$ (5)

In general, the value of sample size n is much larger than N. It can be seen that the essence of the above learning is the learning coefficient vector, and its classical solution comes from linear algebra and numerical calculation.

Since the value of y_i is affected by the noise, for any $x \in ℝ$ , there is a probability measure ɛ_x corresponding to $ℝ$ satisfy $x \in ℝ$ and the mean value of ɛ_x is f (x). Further, y_i is a random distribution of ɛ_{x
_i}. For the sample point x_i, their selection rule obeys the probability measure ρ_X on $ℝ$ . The random distribution of the last sample point (x_i, y_i) follows the probability measure ρ on.

The approximation of the function can be stated as $f_{w} (x) = \sum_{i = 1}^{N} w_{i} φ_{i} (x),$ (6)

where φ_i is a group of bases in the function space. They are not necessarily polynomials, and the goal of learning is to get the coefficient vector w .

Example 7. Neural network training can be seen as an extension of Example 6. Specifically, a neural network can be viewed as a directed graph that contains several input vertices, output vertices, and intermediate vertices, while some unknown functions are calculated and updated throughout the learning process. Let X be the input space, the element of which is the input point; Y is the output space, the element of which is the output point. The neural network calculates the function from X to Y. The characteristics of the entire neural network depend on the functions determined by their weights $w = {w_{j}}_{j \in J} .$ (7)

Example 8. A classic example of pattern recognition is the classification and recognition of handwritten letters. Furthermore, the input space X is a matrix set. Each element in the matrix takes a value in [0,1] and represents a pixel in the handwritten alphanumeric image that has a certain gray value, or represents an extraction of letters feature. Y can be expressed as: $Y = {y \in ℝ^{26} | y = \sum_{i = 1}^{26} λ_{i} e_{i}, \sum_{i = 1}^{26} λ_{i} = 1} .$ (8)

Here, e_i is a vector in $ℝ^{26}$ , and each vector represents a corresponding letter. For any i ∈ {1, 2, ⋯, 26}, if Δ ⊂ Y is the set of y satisfy λ_i ∈ [0, 1], then, the points in Δ can be explained as the probability measure on the set {A, B, …, Z}. Further, the problem can be described that the parameter corresponding to the learning function f : X → Y is the linear combination of e_i in {P {x = A}, P {x = B}, ⋯, P {x = Z}}.

Example 9. A classical approximation method is called Probably Approximately Correct. Let T be a subset of $ℝ^{n}$ and ρ_X be a probability measure on $ℝ^{n}$ , unknown in advance. Intuitively, the closer the set $S \subset ℝ^{n}$ is to T, the smaller the measure of the Symmetric difference becomes: $S Δ T = (S ∖ T) \cup (T ∖ S) .$ (9)

Denote f_S and f_T are the characteristic functions of S and T, respectively, and then: $\int_{ℝ^{n}} | f_{S} - f_{T} | d ρ_{X} .$ (10)

Is the error of S. The values of f_S and f_T can only be 0 or 1.Therefore, the integral is equivalent to $\int_{ℝ^{n}} (f_{S} - f_{T})^{2} d ρ_{X} .$ (11)

Let C be a class of subsets of $ℝ^{n}$ , then T ∈ C. The method of constructing T in C is described as follows: First, according to ρ_X, take points x₁, x₂, ⋯, x_m in $ℝ^{n}$ , and then label it 1 or 0 according to whether they belong to T or not. Secondly, for each $f : ℝ^{n} \to {0, 1}$ , f_S ∈ C, calculate based on the labels of {x₁, x₂, ⋯, x_m}. Then, follow the principle of least error on ρ_X, we get S that is closest to T. When the sample size m is large enough, and C has a certain mathematical description, this approximation is better.

Example 10. A random sampling method commonly used in the early numerical integration algorithm is called MCI (Monte Carlo Integration). Let $f : [0, 1]^{n} \to ℝ$ . One approach of approximation is to randomly taking points x₁, x₂, ⋯, x_m ∈ [0, 1] ⁿ to compute the value of: $I_{m} (f) = \frac{1}{m} \sum_{i = 1}^{m} f (x_{i}) .$ (12)

To approximates the integral value: $\int_{x \in [0, 1]^{n}} f (x) dx .$ (13)

By setting a weak regularization condition on the function f, I_m (f) → ∫f holds at probability 1, that is, for all ɛ > 0, we have:

$lim_{m \to \infty} \underset{x_{1}, x_{2}, \dots, x_{m}}{P} {| \frac{1}{m} \sum_{i = 1}^{m} f (x_{i}) - \int_{x \in [0, 1]^{n}} f (x) dx | > ɛ} \to 0 .$ (14)

In this example, the measure makes the sample knowable (the measure on [0, 1] ⁿ inherits the characteristics of the standard Lebesgue measure on $ℝ^{n}$ ), but this method is equally applicable to cases where the measure is unknown.

3.2 Framework of learning

Let X be a tight metric space, $Y = ℝ^{k}$ , where ρ is a Borel probability measure on Z = X × Y whose regularity can be mathematically assumed as necessary. If ξ is a random variable (which can be understood as a real-valued function on the probability space Z), let E (ξ) and σ² (ξ) denote the mathematical expectation and variance of ξ respectively. So E (ξ) = ∫_Zξ and $σ^{2} (ξ) = E ((ξ - E (ξ))^{2}) = E (ξ^{2}) - (E (ξ))^{2} .$ (15)

For f : X → Y, its generalized error is defined as: $L (f) = L_{ρ} (f) = \int_{Z} (f (x) - y)^{2} d ρ .$ (16)

where (f (x) - y) ² is called local error. An important problem in learning is how to reduce the value of generalized error. A classical method is to decompose L (f) into multiple parts.

For each x ∈ X, let ρ (y|x) be the conditional probability measure of x on Y, ρ_X is the Marginal probability measure of ρ on X, which is stated as: $ρ_{X} (S) = ρ (π^{- 1} (S)),$ (17)

where π : X × Y → X is projection operator (bounded, linear operators in normed linear space). For any integral function $φ : X \times Y \to ℝ$ , based on Fubini’s theorem, we get:

∫_X×Yφ (x, y) dρ = ∫_X (∫_Yφ (x, y) dρ (y|x)) dρ_X decompose ρ into ρ (y|x) and ρ_X, divide Z into input domain X and output domain Y, and maps their direct products.

Let f_ρ : X → Y, which is stated as: $f_{ρ} (x) = \int_{Y} yd ρ (y | x) .$ (18)

The function f_ρ is called the regression function of ρ. For each x ∈ X, f_ρ (x) is the mean of the ordinate of {x} × Y. The regularity of ρ can help to get the regularity of f_ρ, and we always assume that f_ρ is bounded. Let x ∈ X, consider a function from Y to $ℝ$ that maps y to y - f_ρ (x). Since the mathematical expectation of such a function is 0, its variance is: $σ^{2} (x) = \int_{Y} (y - f_{ρ} (x))^{2} d ρ (y | x) .$ (19)

If we average on X, then $σ_{ρ}^{2} = \int_{X} σ^{2} (x) d ρ_{X} = L (f_{ρ}) .$ (20)

We can measure the quality of ρ by considering the value of $σ_{ρ}^{2}$ . For all f : X → Y, we get, $L (f) = \int_{X} (f (x) - f_{ρ} (x))^{2} d ρ_{X} + σ_{ρ}^{2} .$ (21)

Since $σ_{ρ}^{2}$ is independent from f, the above formula means that f_ρ is a function with the smallest probability error in all f : X → Y, then $σ_{ρ}^{2}$ represents the lower bound of the error L (f) with respect to the measure ρ. The goal of learning then is to find the function which is most approximate f_ρ from a randomsample of Z.

Consider the sampling, let $z = ((x_{1}, y_{1}), (x_{2}, y_{2}), \dots, (x_{m}, y_{m})) \in Z^{m} .$ (22)

Be samples on Z^m, that is, m instances follow the independent distribution of ρ, and Z^m represents the m Cartesian product on Z. The empirical error of f with respect to z is defined as: $L_{z} (f) = \frac{1}{m} \sum_{i = 1}^{m} (f (x_{i}) - y_{i})^{2} .$ (23)

If ξ is the random variable on Z, then denote the empirical mean of ξ with respect to z as E_z (ξ), and

$E_{z} (ξ) = \frac{1}{m} \sum_{i = 1}^{m} ξ (z_{i}) .$ (24)

For any function f : X → Y, denote f_Y : X × Y → Y as (x, y) ↦ f (x) - y. Then: $\begin{matrix} L (f) = E (f_{Y}^{2}), \\ L_{z} (f) = E_{z} (f_{Y}^{2}) \end{matrix}$ (25)

And the mathematical expectations of (f_ρ) _Y is 0, and the variance is $σ_{ρ}^{2}$ .

Let C (X) be a Banach space formed by a continuous function on X with the following generalizations: ${∥ f ∥}_{\infty} = sup_{x \in X} | f (x) | .$ (26)

Considering the subspace H of C (X), the learning algorithm looks for the function closest to f_ρ from H, which is called Hypothesis Space. In the process of machine learning, H is the key element. If H is too small, the performance of the optimal function will be not good enough. If H is too large, the complexity of the algorithm will be greatly increased. We always hope f_ρ ∈ H, but in practice, even f_ρ ∈ C (X) may not be guaranteed.

Considering f_H ∈ H as the objective function, we define that f_H is obtained by looking for a function in H that minimizes the error L (f)(assuming a loss function is squared loss): $min_{f \in H} \int_{Z} (f (x) - y)^{2} d ρ .$ (27)

For $L (f) = \int_{X} (f (x) - f_{ρ} (x))^{2} d ρ_{X} + σ_{ρ}^{2}$ , if we minimize f_H, we can get: $min_{f \in H} \int_{X} (f - f_{ρ})^{2} d ρ_{X}$ .

Let z ∈ Z^m be a sample, and define the empirical objective function f_H,z = f_z as a function which is obtained by minimizing L_z(f) when f ∈ H. That is, $min_{f \in H} \frac{1}{m} \sum_{i = 1}^{m} (f (x_{i}) - y_{i})^{2}$ . Thus, f_z is not actually determined by the algorithm. The above minimization depends on the sample z . In the sense that the problem depends on ρ. However, after the sample is fixed, the determination of f_z has nothing to do with ρ again. In addition, it should be noted that L (f_z) and L_z (f) have different meanings.

The following example illustrates the existence of f f_H and f_z by setting weak conditions for the hypothesis space H. If z ∈ Z^m, f : X → Y, then:

L_z (f) = L_ρ,z (f) = L (f) - L_z (f). Theoretically, the value of L (f) can not be directly calculated because of the unknown distribution, but the value of L_z (f) can be calculated directly after the sample is fixed.

Let f₁, f₂ ∈ C (X), for almost all z ∈ Z^m, estimate the value of |L_z (f₁) - L_z (f₂) | from the Lipschitz estimate and the value of ∥f₁ - f₂ ∥ _∞. Set U ⊆ Z is called full measure, if measure Z ∖ U is 0. If |f_j (x) - y| ⩽ M(where j ∈ {1, 2}) holds on the full measure set U ⊆ Z, then for all z ∈ U^m, we have |L_z (f₁) - L_z (f₂) | ⩽ 4M ∥ f₁ - f₂ ∥ _∞.

It can be seen from the results that:

① Let H ⊆ C (X), if |f (x) - y| ⩽ M holds for all f ∈ H almost everywhere, then L and L_z are continuous.

② Let H ⊆ C (X) be tight, if |f (x) - y| ⩽ M holds for all f ∈ H almost everywhere, then f_H and f_z exist.

What needs to be specially explained here is that the functions f_H and f_z are not unique. When H is a convex space, f_H is unique.

Let H be a hypothetical, the error of the function f ∈ H in H can be expressed in normal error: $L_{H} (f) = L (f) - L (f_{H}) .$ (28)

For any f ∈ H, L_H (f) ⩾0 and L_H (f_H) =0, then, L (f_H) and L_H (f) are absolutely different. By simple calculation, we get $\int_{X} (f_{z} - f_{ρ})^{2} d ρ_{X} + σ_{ρ}^{2} = L (f_{z}) = L_{H} (f_{z}) + L (f_{H})$ where $σ_{ρ}^{2}$ is the lower bound of error L, and the generalized error L (f_z) of f_z depends on ρ, H, and sample z. The square distance ∫_X (f_z - f_ρ) ²dρ_X is called the Excess Generalization Error of f_z. When ρ and H assume a certain condition, as the sample size m tends to be infinite, the extra generalized error ∫_X (f_z - f_ρ) ²dρ_X becomes arbitrarily small with a large probability.

Fig.5

Linear regression.

Consider L_H (f_z) + L (f_H). The second L (f_H) is only related to the choice of H, and has nothing to do with the sample, which is called the Approximation Error. We denote it as: $A (H) = \int_{X} (f_{H} - f_{ρ})^{2} d ρ_{X} .$ (29)

Then, the Approximation Error can be described as: $L (f_{H}) = A (H) + σ_{ρ}^{2} .$ (30)

Therefore, $σ_{ρ}^{2}$ is the lower bound of the approximation error. The first L_H (f_z) is called Sample Error or Estimation Error. In other words, the error we need to estimate is divided into two types: approximation error and sample error. After that, we estimate the two types of error to calculate the final algorithm error, which is also the classic computing approach of statistical learning theory.

The estimation of A (H) has nothing to do with the sample z, and its ρ depends on the regression function f_ρ. As for the sample error, once the sample is given, it has no relation with ρ any more, although the sample is determined by ρ.

When the hypothesis space H is fixed, the sample error decreases as the sample size m increases. With a fixed sample size m, as the H range expands, the approximation error decreases but the sample error increases. The bias problem means that when the sample size m is fixed, choosing the size of H to minimize the error L (f_z) with a high probability. Therefore, here the “bias” corresponds to the approximation error, “variance” corresponds to the sample error. “Bias” and “Variance” are like “time domain” and “frequency domain” in the Uncertainty Theorem: one’s “good” can lead to the other’s “bad”, and the two can not be obtained at the same time, so we have to find a balance between them.

If H is too small, it will lead to large bias, while if H is too large, it will produce large variance. Therefore, the size of H is the core of learning theory. Figures 6 and 7 describe the normal function, functions in the case of large bias and large variance respectively. It can be seen that in the case of large bias, the curve obtained does not reflect the actual situation; in the case of large variance, the resulting curve is in poor smooth, that is to say, the function is “shaking” violently, this learning function can’t be used for other data sets, resulting in poor generalization of the algorithm.

Fig.6

Normal curve.

Fig.7

Large bias.

In what follows, we will turn to ontology setting, and we set V as the ontology instance space because the ontology structure is presented as a graph.

4 Confidence weighted multi-dividing ontology algorithm

In this section, we aim to present our first ontology algorithm in the confidence weighted multi-dividing setting.

As the concepts mentioned in the first section, the so-called multi-segmentation ontology learning is to use the ontology graph which mostly has a tree structure. The similarity between the concepts belonging to the same branch is greater than that belonging to different branches. From the point of view of data distribution, the data corresponding to the vertices belonging to the same branch are relatively close to each other, and they can be considered to be relatively concentrated in the same area. If these data points are projected as one-dimensional real lines, the data points corresponding to the vertices of the same branch should be concentrated in a certain range. In this feature, a plurality of branches in the tree structure are divided into a plurality of levels, and the real numbers corresponding to the respective branches (rank) are sequentially arranged on the real axis.

According to this idea, let $V (G) \to ℝ$ be the ontology function, where V (G) is the vertex set of the ontology graph, which maps each vertex (corresponding to a concept) into real numbers. The similarity between the concepts is determined by the absolute value of the difference of the real value corresponding to the vertex. In other words, the similarity of v_i and v_j is determined by |f (v_i) - f (v_j) |. The larer the value of the vertex |f (v_i) - f (v_j) | is, the smaller the similarity between them become. In the multi-dividing framework, the vertex corresponding to the vertex in level α is greater than the real number corresponding to the vertex in any level b, where 1 ⩽ a < b ⩽ k. That is, for any v ∈ r (a) and v′ ∈ r (b), we have f (v) > f (v′).

Let k be the number of divisions after division, V be the input space, or the instance space, whose e1ements are the vertices of the onto1ogy graph. For Y ⊆ {1, 2, ⋯, k} is the hierarchica1 markup space, D is the unknown distribution V × Y. Set $S = {(v_{1}, y_{1}), (v_{2}, y_{2}), \dots, (v_{n}, y_{n})}$ (31)

be a set of samples with a capacity of n. The goal of the ontology learning algorithm is to obtain an optimal ontology function $f : V \to ℝ$ by S to map each vertex to real number. The similarity of the two concepts is determined by the absolute value of the difference between the corresponding vertex corresponding real numbers. In this framework, for the sake of mathematical expression and computation, when initializing the ontology, it is necessary to represent the information of each vertex with a p-dimensional vector, that is, all the information related to the vertex and concept is contained in the corresponding vector. Thus the ontology function can be expressed as $f : ℝ^{d} \to ℝ$ from this point of view, and the essence of ontology learning algorithm is a dimensionality reduction algorithm, which is a d dimension vector with a real number to represent.

Since each tag y does not represent the value of f (v) under the multiple partitioning framework, it represents the rank of the vertex v and takes an integer between 1 and k. Thus the sample set S can be expressed as $\begin{matrix} S = (S_{1}, S_{2}, \dots, S_{k}) \\ \in (X \times Y)^{n_{1}} \times (X \times Y)^{n_{2}} \times \dots \times (X \times Y)^{n_{k}}, \end{matrix}$ (32)

where for any a ∈ {1, 2, ⋯, k}, we have $S_{a} = {(v_{1}^{a}, a), (v_{2}^{a}, a), \dots, (v_{n_{a}}^{a}, a)}$ and |S_a| = n_a. In this case, $v_{j}^{i} (i \in {1, 2, \dots, k}, j \in {1, 2, \dots, n_{i}})$ (33)

represents both the vertex and the vector corresponding to the vertex, and the reader can determine the content of the vertex according to the context.

In the multi-segmentation framework, the optimal ontology function we need satisfies the following condition: For 1 ⩽ a < b ⩽ k, the vertex in any class α in f has a larger value than the vertex in any class b, ie f (v) > f (v′). If v ∈ S_a and v′ ∈ S_b. For a ∈ {1, 2, ⋯, k}, let D_a = D_V|Y=a be the conditional distribution.

In multi-dividing setting, the optimal ontology function can be expressed by minimizing the following expression: $\sum_{a = 1}^{k - 1} \sum_{b = a + 1}^{k} E_{V_{i} \sim D_{a}, V_{j} \sim D_{b}} {{I}_{f (V_{i}) ⩽ f (V_{j})}},$ (34)

where I represents the true value function, the result is 1 when the argument is true, and the result is 0 when the argument is false. Since the distribution D is unknown in advance, the above equation can not be calculated directly.

Using ontology sample set S, the empirical ontology model can be determined by minimizing $\sum_{a = 1}^{k - 1} \sum_{b = a + 1}^{k} \frac{1}{n_{a} n_{b}} \sum_{i : y_{i} = a} \sum_{j : y_{j} = b} {{I}_{f (v_{i}) ⩽ f (v_{j})}} .$ (35)

Although the distribution of the sample obeys D, but when S is given, the sample set is not directly related to the distribution. Thus this version can be calculated directly.

Although the direct calculation is very difficult, it is desirable that the essence of the truth function I is a discrete counting function that is not derivable. Thus, in most cases, it is necessary to set a continuous derivative loss function $l : ℝ^{V} \times V \times V \to ℝ^{+} \cup {0}$ (non-negative real-valued function, and symmetric in v and v’) to replace the true-valued function I. Some of the literature also known as the loss function l is the cost function c, whose role is for those who do not meet f (v) ⩾ f (v′) (where v ∈ S_a, v′ ∈ S_b and 1 ⩽ a < b ⩽ k) vertex pairs to a certain penalty. The minimizing the expected ontology model can be transformed into the following expected model: $\sum_{a = 1}^{k - 1} \sum_{b = a + 1}^{k} E_{V_{i} \sim D_{a}, V_{j} \sim D_{b}} l (f, V_{i}, V_{j}) .$ (36)

The corresponding empirical ontology framework is denoted as $\sum_{a = 1}^{k - 1} \sum_{b = a + 1}^{k} \frac{1}{n_{a} n_{b}} \sum_{i : y_{i} = a} \sum_{j : y_{j} = b} l (f, v_{i}, v_{j}) .$ (37)

There are some commonly used ontology lost function list as follows (here x is the variable of ontology loss functions):

Hinge loss function: $l (x) = {1 - x, 0};$ (38)

Least square loss function: $l (x) = (1 - x)^{2};$ (39)

Exponential loss function: $l (x) = e^{- x};$ (40)

Logarithmic loss function $l (x) = log (1 + e^{- x});$ (41)

Symmetric ɛ-insensitive logarithmic loss function: $\begin{matrix} l (x, ɛ) = log (1 + e^{x - ɛ}) + log (1 + e^{- x - ɛ}) \\ + 2 log (1 + e^{- ɛ}) . \end{matrix}$ (42)

Here, we use the hinge loss as our ontology loss function, and thus the ontology objective function we find to minimize to linear function can be formulated as follows:

$\frac{{∥ w ∥}^{2}}{2} + \sum_{a = 1}^{k - 1} \sum_{b = a + 1}^{k} C^{a, b} \sum_{i = 1}^{n_{a}} \sum_{j = 1}^{n_{b}} max (0, 1 - w (v_{i}^{a} - v_{j}^{b})) .$ (43)

where ∥w∥ is denoted as the Euclidean norm to regularize the ontology order, and for each pair of (a, b), C^a,b is the parameter to punish the sum of errors. We can reformulate the ontology objective function as a sum of two losses for a pair of vertices for each pair of (a, b), $\begin{matrix} \frac{{∥ w ∥}^{2}}{2} + \sum_{a = 1}^{k - 1} \sum_{b = a + 1}^{k} C^{a, b} \sum_{i = 1}^{T^{a, b}} {I_{y_{t}^{a, b} = a} (g_{t}^{a})^{a, b} (w) \\ + I_{y_{t}^{a, b} = b} (g_{t}^{b})^{a, b} (w)}, \end{matrix}$ (44)

where T^a,b = n_a + n_b and each pair of (a, b) $\begin{matrix} (g_{t}^{a})^{a, b} (w) = \sum_{t^{'} = 1}^{t - 1} I_{y_{t^{'}} = b} max (0, 1 - w (v_{t} - v_{t^{'}})), \\ (g_{t}^{b})^{a, b} (w) = \sum_{t^{'} = 1}^{t - 1} I_{y_{t^{'}} = a} max (0, 1 - w (v_{t^{'}} - v_{t})) . \end{matrix}$ (45)

Instead of maintaining all the obtained vertices to calculate the gradients $\nabla (g_{t}^{•})^{a, b} (w)$ for each pair of (a, b), we keep random vertices from each rate in the corresponding buffer. Hence, for each pair of (a, b), two buffers $B_{a}^{a, b}$ and $B_{b}^{a, b}$ with predefined capacity are maintained for rate a and rate b, respectively. The buffers can be updated by means of stream oblivious policy.

The framework of the online confidence weighted multi-dividing ontology algorithm is depicted in Algorithm 1.

Algorithm 1. Confidence weighted multi-dividing ontology algorithm

Input: the punish parameter C^a,b for each pair of (a, b); the capacity of buffers M_a for a ∈ {1, ⋯, k}, a parameter η^a,b for each pair of (a, b); set $θ_{i}^{a, b} = 1$ for i ∈ {1, ⋯, d} for each pair of (a, b).

For each pair of (a, b), do:

Initialize: $μ_{1}^{a, b} = (0, \dots, 0)^{d}$ , $B_{a}^{a, b} = B_{b}^{a, b} = Ø$ , $Σ_{1}^{a, b} = diag (θ)$ .

For t = 1, ⋯, T^a,b, do

Obtain a training ontology vertex (v_t, y_t);

If y_t = a, then $\begin{matrix} (B_{b}^{t + 1})^{a, b} = (B_{b}^{t})^{a, b}; \\ C_{t}^{a, b} = C^{a, b} max {1, \frac{| (B_{b}^{t})^{a, b} |}{M_{b}}}; \end{matrix}$ (46)

$\begin{matrix} (B_{a}^{t + 1})^{a, b} = Updatebuffer (v_{t}, (B_{a}^{t})^{a, b}, M_{a}); \\ [μ_{t + 1}^{a, b}, Σ_{t + 1}^{a, b}] = Update (f |_{a, b}) (μ_{t}^{a, b}, Σ_{t}^{a, b}, \\ v_{t}, y_{t}, C_{t}^{a, b}, (B_{b}^{t + 1})^{a, b}, η^{a, b}); \end{matrix}$ (47)

End If

If y_t = b, then

$(B_{a}^{t + 1})^{a, b} = (B_{a}^{t})^{a, b}$ ;

$C_{t}^{a, b} = C^{a, b} max {1, \frac{| (B_{a}^{t})^{a, b} |}{M_{a}}}$ ;

$(B_{b}^{t + 1})^{a, b} = Updatebuffer (v_{t}, (B_{b}^{t})^{a, b}, M_{b})$ ;

$[μ_{t + 1}^{a, b}, Σ_{t + 1}^{a, b}] = Update (f |_{a, b}) (μ_{t}^{a, b}, Σ_{t}^{a, b}, v_{t}, y_{t}, C_{t}^{a, b}, (B_{a}^{t + 1})^{a, b}, η^{a, b})$ ;

End If

End For

5 Ontology sparse vector learning via feature selection

In this section, we mainly present the algorithm for ontology sparse vector learning.

Let $S = {z_{i} = (v_{i}, y_{i})}_{i = 1}^{n}$ be the ontology training data, where v_i ∈ R^d and output label $y_{i} \in ℝ$ . Let v_·,j be the j-th feature vector of all ontology samples of length n. Here, we assume that both d and n are large numbers but v_·,i for i ∈ {1, ⋯, d} are sparse. Let $l (\hat{y}, y)$ be an ontology loss function and f_β (v) be an ontology function which is belong to function family F and associated with ontology sparse vector β. Denote L (β, z) = l (f_β (v), y). We can then formulate the learning purpose as a regularized minimization ontology problem: $\hat{β} = \underset{β \in ℝ^{d}}{arg min} \sum_{i = 1}^{n} L (β, z_{i}) + Ω (β) .$ (48)

In the online ontology learning setting, the algorithm gets an ontology sample z_t = (v_t, y_t) at a time, and at time t, the vector is updated in an online way with an ontology sample z_t ∈ S drawn randomly

$β_{t} = β_{t - 1} - η L^{'} (β_{t - 1}, z_{t}),$ (49)

where positive number η is the rate of learning and L′ (β_t-1, z_t) ∈ ∂_{β
_t}L (β_t-1, z_t) is a subgradient of the ontology loss function with respect to β_t. If L (β, ·) is differentiable at β, we have ∂_βL (β, ·) = {∇ _βL (β, ·)}.

For each $K \in ℕ^{+}$ iterations at step t, each of which can be denoted by (1), the ontology sparse vector is shrunk by a soft-threshold operator T with a gravity parameter $g \in ℝ^{d}$ where for i ∈ {1, ⋯, d} we have g_i ⩾ 0. For an ontology sparse vector $β = [β_{1}, \dots, β_{d}] \in ℝ^{d}$ , ${\hat{β}}_{t} = T (β_{t}, g),$ (50)

where T (β, g ) = [T (β₁, g₁), ⋯, T (β_d, g_d)] and $T (β_{i}, g_{i}) = {\begin{matrix} max (β_{i} - g_{i}, 0), if β_{i} > 0 \\ min (β_{i} + g_{i}, 0), otherwise \end{matrix} .$ (51)

The easiest process can be stated as follows: input β₀, g₀ (base gravity parameter), K and η; for each iterative (from 1 to K), randomly and uniformly draw z_t from S, and do operations as (1); at last, return $\hat{β} = T (β_{K}, g_{0} K 1_{d})$ .

Let $\tilde{k} \in ℝ^{d}$ be the number of updated features v_·,i (here i ∈ {1, ⋯, d}). Set $g = g 0 \tilde{k}$ . This simple process can be modified as follows. input β₀, g₀ (base gravity parameter), $\tilde{k} = 0_{d} \in ℝ^{d}$ , K and η; for each iterative (from 1 to K), randomly and uniformly draw z_t from S, and do operations as (1) and $\tilde{k} \leftarrow \tilde{k} + I (| v_{t} | > 0)$ ; at last, return $\hat{β} = T (β_{K}, g_{0} \tilde{k})$ and $\tilde{k}$ .

Furthermore, let V_K be a random subsample of {1, ⋯, n} with K ontology samples, and $\begin{matrix} \begin{matrix} Π_{i}^{g} = P^{*} (i \in {\hat{D}}^{g} (\hat{β}, V_{K}) = {i : | {\hat{β}}_{i} | > 0}) \\ = E_{S} [I (| {\hat{β}}_{i} | > 0)], \end{matrix} \end{matrix}$

where the probability P^* is with respect to the random subsampling of V_K. Let $Π^{g} = [Π_{1}^{g}, \dots, Π_{d}^{g}]$ .

Let positive integer n_K be the number of truncated bursts, and ${\tilde{k}}_{τ}$ be the counters of updates for burst τ (τ ∈ {1, ⋯, n_K}). With M processors, let S⁽¹⁾, ⋯, S^(m) be the random permutation of ontology training data S. with gravity g , the estimate of selection probability of i-th sparse feature can be yielded as ${\hat{Π}}_{i} = \frac{\sum_{m = 1}^{M} \sum_{τ : {\tilde{k}}_{i, τ}^{(m)} > 0} I (| {\hat{β}}_{i, τ}^{(m)} | > 0)}{\sum_{m = 1}^{M} \sum_{τ = 1}^{n_{K}} I ({\tilde{k}}_{i, τ}^{(m)} > 0)}$ (52)

if i satisfies $\sum_{m = 1}^{M} \sum_{τ = 1}^{n_{K}} {\tilde{k}}_{i, τ}^{(m)} > 0$ ; otherwise, ${\hat{Π}}_{i} = 1$ .

For threshold π₀ ∈ [0, 1], the collection of stable features with gravity parameter g is given by

${\hat{Θ}}^{g} = {i : Π_{i}^{g} ⩾ π_{0}} .$ (53)

We always write ${\hat{Θ}}^{g}$ as $\hat{Θ}$ for short.

In this fashion, the ontology sparse vector learning iterative algorithm can be described as follows. Input β₀, g₀, $\hat{Θ}$ in terms of (3) and (4), K, η, $x_{0} = (β_{0, j})_{j \in \hat{Θ}}$ and $\tilde{k} = 0_{| \hat{Θ} |}$ ; for each iterative (from 1 to K), randomly and uniformly draw z_t from S, ${\tilde{z}}_{t} = (z_{t, i})_{i \in \hat{Θ}}$ and do operations as (1) and $\tilde{k} \leftarrow \tilde{k} + I (| v_{t} | > 0)$ ; then, $\hat{u} = T (x K, g_{0} \tilde{k}), {\hat{β}}_{i} = {\hat{u}}_{i^{'}} if i \in \hat{Θ}$ , and ${\hat{Θ}}_{i^{'}} = i$ , and ${\hat{β}}_{i} = 0$ if $i \notin \hat{Θ}$ ; at last, return $\hat{β}$ and $\tilde{k}$ . We denote this procedure as Ξ.

If we consider the adaptive base gravity g₀ for a optimal rejection rate w_s ∈ [0, 1] is expressed as $\begin{matrix} g_{0, s} (w_{s}) = sup {g_{0} ⩾ 0 : \\ \frac{\sum_{m = 1}^{M} \sum_{τ = 1}^{n_{K}} \sum_{{i : i \in {\hat{Θ}}_{s}, {\tilde{k}}_{i, τ}^{(m)} > 0}} I (| \frac{β_{i, τ}^{(m)} - β_{i, τ - 1}^{(m)}}{{\tilde{k}}_{i, τ}^{(m)}} | > g_{0})}{\sum_{m = 1}^{M} \sum_{τ = 1}^{n_{K}} \sum_{i \in {\hat{Θ}}_{s}} I ({\tilde{k}}_{i, τ}^{(m)} > 0)} ⩽ w_{s}} . \end{matrix}$ (54)

Finally, our ontology sparse vector learning algorithm can be stated as follows. Input ontology sample data S, w₀, γ(it called annealing rate in machine learning), π₀, K, n_K, M, η, ${\tilde{β}}_{0}^{(m)} = 0_{d}$ (m ∈ {1, ⋯, M}), $\tilde{k} = 0_{d}$ , S^(m)(m ∈ {1, ⋯, M}) and ${\hat{Θ}}_{0} = {0, \dots, d}$ ; for each iterative (s = 1, 2, ⋯), set ${\tilde{β}}_{0}^{(m)} = {\tilde{β}}_{s - 1}^{(m)}$ , and repeat the following steps until $\bar{β}$ converges:

deduce g_0,s (w_s) according to (5);

for all m ∈ {1, ⋯, M} and τ = 1, ⋯, n_K, get ${\hat{β}}_{τ}^{(m)}$ using the procedure Ξ;

determine ${\hat{Π}}_{s}$ in terms of (3) and update ${\hat{Θ}}_{s} = {i : {\hat{Π}}_{s, i} > π_{0}}$ ;

${\hat{β}}_{s}^{(m)} = {\hat{β}}_{n_{K}}^{(m)} I_{{\hat{Θ}}_{s}}$ for m ∈ {1, ⋯, M} and

$\begin{matrix} w_{s + 1} = {\begin{matrix} w_{0} (e^{- γ \frac{| {\hat{Θ}}_{s} |}{d}} - \frac{| {\hat{Θ}}_{s} |}{d} e^{- γ}), if γ ⩾ 0 \\ w_{0} \frac{log (1 - γ (1 - \frac{| {\hat{Θ}}_{s} |}{d}))}{log (1 - γ)}, otherwise \end{matrix}; \end{matrix}$

$\bar{β} = \frac{1}{M} \sum_{m = 1}^{M} {\tilde{β}}_{s}^{(m)}$ .

At last, return $\bar{β}$ , and the whole ontology sparse vector learning algorithm is finished.

6 Conclusion

In this paper, we introduce the fundamental ideas of ontology learning algorithm in multi-dividing setting and sparse vector learning setting. Also, the fundamental of statistical learning theory used in ontology learning is introduced. Finally, we design two ontology learning algorithms: confidence weighted multi-dividing ontology algorithm and ontology sparse vector learning via feature selection.

Footnotes

Acknowledgments

We thank the reviewers for their constructive comments in improving the quality of this paper. This work was supported in part by the National Natural Science Foundation of China (51574232), the Open Research Fund by Jiangsu Key Laboratory of Recycling and Reuse Technology for Mechanical and Electronic Products (RRME-KF1612) and the Natural Science Foundation of Jiangsu University of Technology in China (KYY14013).

References

Koehler

, Doelken

S.C.

, Mungall

C.J.

, et al., The human phenotype ontology project: Linking molecular biology and disease through phenotype data, Nucleic Acids Research42(D1) (2014), 966–974.

Przydzial

M.J.

, Bhhatarai

, Koleti

B.A.

, et al., GPCR ontology: Development and application of a G protein-coupled receptor pharmacology knowledge framework, Bioinformatics29(24) (2013), 3211–3219.

Bhattacharjee

, Jayadeepa

R.M.

, Talambedu

, et al., Complex network and gene ontology in pharmacology approaches: Mapping natural compounds on potential drug target colon cancer network, Current Bioinformatics6(1) (2011), 44–52.

Demartini

, Enchev

, Gapany

, et al., The Bowlogna ontology: Fostering open curricula and agile knowledge bases for Europe’s higher education landscape, Semantic Web4(1) (2013), 53–63.

Simut

C.C.

, The between psychology, ontology, and Divinity: Fundamental aspects of the concept of Logos in the early thought of Slavoj Zizek, HTS Theological Studies701 (2014), 1–12.

Shah

, Rabhi

and Ray

, Investigating an ontology-based approach for big data analysis of inter-dependent medical and oral health conditions, Cluster Computing-The Journal of Networks Software Tools and Applications18(1) (2015), 351–367.

Zghal

H.B.

and Moreno

, A system for information retrieval in a medical digital library based on modular ontologies and query reformulation, Multimedia Tools and Applications72(3) (2014), 2393–2412.

Mignard

and Nicolle.

, Merging BIM and GIS using ontologies application to urban facility management in ACTIVe3D, Computers in Industry65(9) (2014), 1276–1290.

Brisaboa

N.R.

, Luaces

M.R.

, Places

A.S.

and Seco

, Exploiting geographic references of documents in a geographical information retrieval system using an ontology-based index, Geoinformatica14(3) (2010), 307–331.

10.

Bowden

D.M.

, Song

, Kosheleva

and Dubach

M.F.

, NeuroNames: An ontology for the braininfo portal to neuroscience on the Web, Neuroinformatics10(1) (2012), 97–114.

11.

Thomas

D.G.

, Pappu

R.V.

and Baker

N.A.

, Nanoparticle ontology for cancer nanotechnology research, Journal of Biomedical Informatics44(1) (2011), 59–74.

12.

Ashraf

, Hussain

O.K.

and Hussain

F.K.

, Making sense from Big RDF Data: OUSAF for measuring ontology usage, Experience45(8) (2015), 1051–1071.

13.

Ongsri

S.L.

and Roddick

J.F.

, Incorporating ontology-based semantics into conceptual modelling, Information Systems52 (2015), 1–20.

14.

Sorokine

, Schlicher

B.G.

and Ward

R.C.

, et al., An interactive ontology-driven information system for simulating background radiation and generating scenarios for testing special nuclear materials detection algorithms, Engineering Applications of Artificial Intelligence43 (2015), 157–165.

15.

Groza

, Kohler

and Moldenhauer

, et al., The human phenotype ontology: Semantic unification of common and rare disease, American Journal of Human Genetics97(1) (2015), 111–124.

16.

Cannataro

, Guzzi

P.H.

and Milano

, GoD: An R-package based on ontologies for prioritization of genes with respect to diseases, Journal of Computational Science9 (2015), 7–13.

17.

Gao

, Zhu

L.L.

and Wang

K.Y.

, Ontology sparse vector learning algorithm for ontology similarity measuring and ontology mapping via ADAL technology, Journal of Bifurcation and Chaos25(14) (2015), (12 pages). DOI: 10.1142/S0218127415400349.

18.

Gao

and Zhu

L.L.

, Gradient learning algorithms for ontology computing, Computational Intelligence and Neuroscience2014, Article ID 438291. 10.1155/2014-438291.

19.

Gao

and Xu

T.W.

, Stability analysis of learning algorithms for ontology similarity computation, Abstract and Applied Analysis2013, Article ID 174802. 10.1155/2013-174802.

20.

Gao

, Gao

and Zhang

Y.G.

, Strong and weak stability of-partite ranking algorithm, Information-An International Interdisciplinary Journal15(11) (2012), 4585–4590.

21.

Gao

, Zhu

L.L.

and Wang

K.Y.

, Ranking based ontology scheming using eigenpair computation, Fuzzy Systems31(4) (2016), 2411–2419.

22.

Gao

, Guo

and Wang

K.Y.

, Ontology algorithm using singular value decomposition and applied in multidisciplinary, Cluster Computing-The Journal of Networks Software Tools and Applications19(4) (2016), 2201–2210.

23.

Gao

, Baig

A.Q.

, Ali

, Sajjad

and Farahani

M.R.

, Margin based ontology sparse vector learning algorithm and applied in biology science, Saudi Journal of Biological Sciences24(1) (2017), 132–138.

24.

Gao

, Zhu

L.L.

, Guo

and Wang

K.Y.

, Ontology learning algorithm for similarity measuring and ontology mapping using linear programming, Fuzzy Systems33(5) (2017), 3153–3163.

25.

Gao

and Farahani

M.R.

, Generalization bounds and uniform bounds for multi-dividing ontology algorithms with convex ontology loss function, The Computer Journal60(9) (2017), 1289–1299.

26.

Gao

, Liang

, Xu

T.W.

and Gan

J.H.

, Topics on data transmission problem in software definition network, Open Physics15 (2017), 501–508.

27.

Gao

, Farahani

M.R.

, Aslam

and Hosamani

, Distance learning techniques for ontology similarity measuring and ontology mapping, Cluster Computing-The Journal of Networks Software Tools and Applications20(2) (2017), 959–968.

28.

Zhu

L.L.

, Pan

, Farahani

M.R.

and Gao

, Magnitude preserving based ontology regularization algorithm, Fuzzy Systems33 (2017), 3113–3122.

29.

Maldonado

, Prada

and Senosiain

M.J.

, On linear operators and bases on Kothe spaces, Applied Mathematics and Nonlinear Sciences1(2) (2016), 617–624.

30.

Ansari

A.A.

, Investigation of the effect of albedo and oblatness on the circular restricted four variable body problem, Applied Mathematics and Nonlinear Sciences2(2) (2017), 529–542.

31.

Fouda

D.A.

, Hamdy

, Nouh

, Beheary

, Bakrey

and Saad

S.M.

, Model atmosphere analysis of two new early-type O4 dwarfs stars, Applied Mathematics and Nonlinear Sciences2(2) (2017), 559–564.