Hybrid reasoning in knowledge graphs: Combing symbolic reasoning and statistical reasoning

3.1. Knowledge completion

To deal with the problem of incompleteness in knowledge graphs, much work has been done to apply statistical relational learning (SRL) models [18] to infer implicit relations between two entities in a knowledge graph. Path ranking algorithm (PRA) [19] and knowledge graph embedding (KGE) [1] are two typical kinds of methods belonging to SRL, and have shown widely used in knowledge completion. In this subsection, we first introduce path ranking algorithm. We then introduce three categories of methods for knowledge graph embedding models.

3.2. Schemaitc knowledge induction

Existing KGs contain lots of triples but lack schematic knowledge, e.g. subclassOf axioms and disjointness axioms. It brings a difficulty to infer implicit information, deal with the heterogeneity for ontology mapping [36], object reconciliation [37] and resolve contradictions [38]. Hence, learning schematic knowledge to enrich KGs becomes a critical and meaningful task.

One main category of the methods to produce schematic knowledge combines rule mining algorithms with symbolic reasoning. The works in [36,39] defined association rule patterns to generate various kinds of axioms and performed inconsistency handling for ontology construction by enriching an original schema incrementally. Considering the open world assumption adopted by KGs, Galárraga et al. [8] adopted partial completeness assumption to generate counterexamples for rules and redefined the standard measurements for support and confidence. Its extension AMIE+ [40] further improved the precision by using type hierarchy and joint reasoning when learning association rules. Inspired by these methods, Gao et al. [41] exploited a type inference algorithm and defined a mining model with the probabilistic type assertions to deal with noisy negative examples, which could generate high-quality subclassOf axioms and disjointness axioms. To improve the scalability of rule-based methods, Omran et al. [42] introduced a new sampling algorithm and embedding representations of arguments. Both of them could guide the extraction of rules. Similarly, the work in [43] employed embedding models to iteratively extracted rules by probabilistic representations of missing facts and feedback from a precomputed embedding model.

The other main category combines machine learning techniques with logical reasoning. The work in [9] used inductive logic programming, which integrated machine learning with logic programming, and defined an ALC downward refinement operator for learning concept descriptions. This operator was extended in [44] that could learn more expressive schematic knowledge like cardinality restrictions. In [38], a statistical method was proposed to extract domain and range of a property. The vector space model from information retrieval was applied to extract disjoint concepts. After the extraction finished, consistency checking was performed in parallel based on predefined inconsistency patterns. The work in [45] integrated the probabilistic inference capability of Bayesian networks with the logical formalism to learn subclassOf axioms and disjointness axioms. It used logical rules for generating more complex axioms and dealing with inconsistency during the schema construction of KGs.

3.3. Knowledge alignment

Over past decades, more and more knowledge graphs become available on the Web, but the heterogeneity and multi-linguality gap of KGs still hinder their sharing and reusing in the Semantic Web. Benefited from hybrid reasoning, the studies of knowledge alignment have obtained some encouraging results.

Cross-lingual taxonomy alignment (CLTA) refers to mapping each category in the source taxonomy of one language onto the most relevant category in the target taxonomy of another language. However, existing methods for CLTA mainly rely on features based on symbolic similarities. Wu et al. [10] proposed a bilingual topic model, called Bilingual Biterm Topic Model (BiBTM). After identified the matched categories based on string similarity, they trained BiBTM by textual contexts extracted from the Web and obtained the topic vector of the extracted textual context for each category. Finally, they utilized the cosine similarity between topic vectors to calculate the taxonomy alignment. Furthermore, they improved the performance of proposed models by merging explicit category correlations including co-occurrence correlation and structural correlation [46].

In addition, there exist some works that employ embedding-based ideas [6] for entity alignment (EA) among knowledge graphs. MTransE [11] separately trains the entity embeddings of two KGs and designed different techniques to represent cross-lingual transitions including axis calibration, translation vectors and linear transformations. JAPE [47] learns the embeddings of two KGs in a unified space and leveraged attributes of triples to refine entity embeddings. To deal with the problem of lack of prior alignment, IPTransE [48] and BootEA [49] employ an iterative process and designed several sophisticated strategies based on the structure of KGs to refine the new alignment. Chen et al. [50] proposed a method called KDCoE, which co-trains the embeddings of multilingual KGs and descriptions of entities. To utilize various features of KGs, Zhang et al. [51] proposed a framework to unify multiple views of entities and learn embeddings for entity alignment. Furthermore, they designed two cross-KG identity inference methods at the entity level as well as the relation and attribute level to preserve and enhance the alignment between different KGs.

3.4. Multi-hop reasoning for question answering

Question answering (QA) is a hot topic that has recently been facilitated by large-scale knowledge bases. However, due to the variety and complexity of questions and knowledge, question answering over knowledge bases (KBQA) is still a challenging task, especially in multi-hop QA.

There are two typical categories of multi-relation questions, a path question [52] and a conjunctive question [53]. A path question contains only one topic entity and its answer could be found by walking down an answer path consisting of a few relations and intermediate entities. A conjunctive question contains more than one subject entity and its answer could be obtained by the intersection of results from multiple path questions. At present, semantic parsing models [12] and embedding-based models [13] tailored for QA are not adequate to handle multi-hop QA because of heavy data annotations and reasoning ability. Therefore, recent works utilized hybrid ideas to improve the performance and make these results explainable.

Zhang et al. [54] proposed a probabilistic modeling framework for multi-hop QA, which could simultaneously handle uncertain topic entity and multi-hop reasoning for QA. They introduced a new propagation architecture over KGs so that logical inference could be performed in the probabilistic model. Zhou et al. [52] designed an interpretable reasoning network (IRN). It could dynamically decide which part of an input question should be analyzed at each hop, and predict a relation corresponding to the parsed results. Compared with existing methods, the intermediate entities and relations predicted by IRN could construct traceable reasoning paths to reveal how the answer was derived. Hamilton et al. [53] introduced a framework to efficiently make predictions about conjunctive logical queries. They encoded graph nodes in a low-dimensional space and represented logical operators (i.e. projection operator and intersection operator) as learned geometric operations. Moreover, they further demonstrated how to map a practical subset of logic to efficient geometric operations in an embedding space. Vakulenko et al. [55] proposed a novel approach for complex QA using unsupervised message passing. It could propagate confidence scores by parsing an input question and matching terms in a KG to a set of possible answers. This approach was implemented as a series of sparse matrix multiplications mimicking joins over small local subgraphs so that it could successfully be applied to very large KGs, such as DBpedia.

3.5. Other hybrid reasoning methods

Other hybrid reasoning methods focus on boosting the performance of NLP tasks. Most of them merge the symbolic information (e.g. the structure of a KG) into the statistic-based methods and provide the explanation for results of reasoning [56].

Wang et al. [57] proposed a joint model that takes advantage of both explicit and implicit representations for short text classification. They incorporated character level features of KG into a convolutional neural network to capture fine-grained subword information. Experiments on real data showed that their method achieved significant improvement for this task.

To alleviate the bound of number and quality of annotated data, Luo et al. [58] exploited the rich expressiveness of regular expressions at different levels within a neural network (NN). This combined framework could significantly enhance the learning effectiveness and improve the performance on the tasks of intent detection and slot filling.

To tackles the problem of learning and prediction with concept drifts, Chen et al. [59] revisited features embeddings as semantic ones (i.e. consistency vectors and entailment vectors). Such embeddings can be exploited in a context of supervised stream learning to learn statistic models, which are robust to concept drifts. Moreover, they explored an ontology-based knowledge representation and reasoning framework for the transfer learning explanation [60]. It can models a learning domain in transfer learning with expressive OWL ontologies and complement the learning domain with the prediction task-related common sense knowledge. Furthermore, the authors designed a correlative reasoning algorithm to infer three kinds of explanatory evidence for explaining a positive feature or a negative transfer from one learning domain to another.