Learning to transfer knowledge from RDF Graphs with gated recurrent units

Abstract

The Internet is a vital part of today’s ecosystem. The speedy evolution of the Internet has brought up practical issues such as the problem of information retrieval. Several methods have been proposed to solve this issue. Such approaches retrieve the information by using SPARQL queries over the Resource Description Framework (RDF) content which requires a precise match concerning the query structure and the RDF content. In this work, we propose a transfer learning-based neural learning method that helps to search RDF graphs to provide probabilistic reasoning between the queries and their results. The problem is formulated as a classification task where RDF graphs are preprocessed to abstract the N-Triples, then encode the abstracted N-triples into a transitional state that is suitable for neural transfer learning. Next, we fine-tune the neural learner to learn the semantic relationships between the N-triples. To validate the proposed approach, we employ ten-fold cross-validation. The results have shown that the anticipated approach is accurate by acquiring the average accuracy, recall, precision, and f-measure. The achieved scores are 97.52%, 96.31%, 98.45%, and 97.37%, respectively, and outperforms the baseline approaches.

Keywords

Resource Description Framework (RDF)transfer learning deep learning information retreival

1. Introduction

Owing to the availability of knowledge, the modern era comes with a range of problems for the Site. People record, upload, archive, and digitize nearly every operation in the everyday routine of life over the Internet of today’s modern society. Communication systems today have the potential to independently attach to the Internet to spread valuable information without the involvement of users. As a result, data grows on a regular basis which results in an abundance of information. The quest for such data contributed to the development of the Semantic Web and its related data. The development of machine inference [1] is considered to facilitate the flow of information that can connect data from dispersed databases to make it relevant. The expression Web 3.0 was added by this mash-up of results. For Web 3.0, which is acquired using RDF, linking the dispersed information is critical. A RDF triple can then be interpreted as an atomic representation of fact or claim[2] where each triple includes a $s$ topic with a $p$ property with a value of $o$ [3, 4].

Similar datasets can be described as linked data[5], which can be interpreted as ‘all about using the Web to build typed connections between different source data’. Due to its related connections [5, 6], Related information incorporates entities from different sources/locations to be monitored as data space. This encourages to use of the information needed from dispersed sources and creates connections that could assist in information search. RDF triples make it simple for entities to query and link together. RDF and SPARQL are used by current studies to query the content and run search results, respectively.

The RDF is massive and crucial, so collecting data for an average user is not straightforward. Nevertheless, connected data and SPARQL deliver a major increase in search techniques. To read and understand RDF data [7], however, the complexity requirements (similar triples according to the rules of RSFS and OWL) and human labor are important. In order to extract RDF components, SPARQL queries, for example, need structure consistency. These queries do not offer the opportunity for statistical analysis to check the concern against the content of RDF; for example, Basket features may not be sufficient to classify online shopping basketas an input. Several methods have been suggested to accomplish this form of RDF search utilizing related information and SPARQL[8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]. Notably, instead of calculating the resemblance within the RDF material that contributes to the initial inspiration of this work, certain methods reply to inquiries with a particular match. Hadi et al.[20], While on the other side, Used a machine learning method to query for RDF graphics. Though their approach is based on statistical estimation, when looking for RDF graphs, it does not accept semantic relationships and needs substantial improvement.

A challenging issue of neural learning-based approaches [21] is that they are problem-specific which requires training from starch with afresh data source or for a different related problem. Further, deep learning-based approaches are data-hungry which means they require training on large data set to produce satisfactory results. Moreover, now a day the information is continually evolving making it challenging to build tools that could take advantage of pre-trained tools and leverage the information that already exists. To overcome these issues, we exploit the concept of transfer learning in this work. In transfer learning, the learned knowledge from a pre-trained model is extracted and then be fine-tuned to achieve optimal performance without requiring the model training from scratch.

In this work, for RDF graphs, a deep transfer learning-based method is proposed that utilizes the historical data from the DBpedia. These documents are first validated using the W3C validation service. Then, validated documents are reprocessed to have an abstraction of features from the RDF graphs. Next, we use the attention-based pre-trained neural learner to help transfer the knowledge. The pre-trained neural model helps to abstract the pre-learned knowledge without requiring it to be retrained from scratch. The abstracted knowledge is then fine-tuned by utilizing Gated Recurrent Unit (GRU) recurrent neural network. Finally, the proposed solution is validated using a ten-fold cross-validation method. The observations of the assessment reflect that the average scores of accuracy, recall, precision, and f-measure are up to, respectively, 97.52%, 96.31%, 98.45%, and 97.37%.

The primary contributions of this work are as follows:

•
For RDF graph searching, a transfer learning-based approach is proposed that leverages pre-learned knowledge. We are the first to take advantage of transfer learning in the recovery estimation of RDF graphs.
•
Evaluation findings of the projected method indicate that the recommended approach based is reliable and exceeds the state-of-the-art.

The remainder of the article is arranged as follows: the suggested solution is outlined in Section 2. Section 3 outlines the assessment process and implications of the suggested strategy. The risks are stated in Section 4. The linked work and thesis are discussed in Sections 5 and 6, respectively.
2. Related studies

WWW is a space for knowledge. While the URLs represent RDF graphics and other web resources, That could be connected and granted access through the Internet. Because of the data explosion created by the new digital age, it is hard to get the correct URLs against requested queries. Tim Burner Lee launched the semantic web to tackle this problem, which offers a common structure that enables data to be exchanged and reused across applications. Instead of keyword matching and question answers, it considers semantics for searching. The fundamental of the semantic web is to connect knowledge from various tools together. In addition, Linking data is important for connecting and searching data across the semantic web. RDF graphs that contain RDF format data are based on related information. Several approaches to the effective search of RDF graphs have been proposed. These techniques (graph-based or keyword-based search) rely largely on classical RDF searches.

Tran et al. suggested the concept of developing summary-graphs for the initial RDF graph for the processing and classification of SPARQL queries.[22]. Then a solution to this idea was suggested by Zhang et al.[23]. In addition, Yang et al. [24] recommended tree patterns to connected user-specified keywords that are arranged in tree patterns by their significance to scale, Zheng et al. [25] suggested a tool for searching for semantically related Patterns of Structure. In the end, De Virgilio[26] Suggested RDF-based keyword inquiry for Tensor Calculus that is extended through MapReduce [27] to an apportioned world. An approach was suggested by Nhuan et al. [28] Identifies the degrees of equality indicated by different vocabulary between relationships (properties). They suggest the incidents of matched pairs of RDF triples to determine the ranges showing property equality of upper and lower degrees. As a consequence, they selected a graph of related properties where degrees of similarity among properties are indicated by the strength of the edges based on the interval.

Another method that is being adopted for information retrieval is based on fuzzy logic. Although these are not related to machine learning approaches but are worth mentioning. Nagarajan et al.[29] introduced a multi semantic data retrieval system focused on ontology. It is founded on the principle of integrating information of domains and images and uses a fuzzy set of rules to retrieve the appropriate multi-modal information. It also can provide the semantic image by designing and building visual terms using the probabilistic latent semantic. Additional Research [30, 31, 32] have suggested formalization and semantic visualization templates focused on a collection of fuzzy rules. Jaafar et al.[33] suggested a knowledge-based method based on fuzzy to recognize an operation of definition and visualize F-RDF retrieval, to help end-user improve Web data request and access. To simplify the retrieval of information, a fuzzy logic-based ranking feature was introduced by Gupta et al.[18]. The function is based on the estimation of term-weighting schemes, like frequency, inverse paper, as well as normalization.

In conclusion, researchers have suggested various methods [8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 34, 35] for information retrieval using RDF; however, major changes are needed. Besides, none of them employs algorithms for the classification of machine learning to resolve this problem. Notably, the suggested solution differs in that, since we are the first to implement the help vector machine for RDF graph recovery, the current approaches differ.

3. Approach

3.1 Overview

The overall workflow of the proposed approach is illustrated in Fig. 1. The suggested method performs the estimation of RDF graph information retrieval in the following manner:

Figure 1.

Overview of the proposed approach.

(1)

The history-data of RDF graphs are first validated and then reused for the purpose of model training.

(2)

Next to abstract a generalized feature state, we perform lemmatization, word inflection, and stop word removal.

(3)

Then we make a global vocabulary system to limit the out-of-vocabulary words.

(4)

Afterwards, we convert the abstracted features into vector encoded form that can be used for model training purposes.

(5)

Finally, we train transfer learning-based neural model that learned to predict RDF graphs being retrieved.

3.2 Problem definition

To better understand the presented approach, analyze an illustration presented in Fig. 2 to learn how the suggested solution looks for an RDF graph for information retrieval. The example presents an abstraction of the RDF graph that is taken from DBpedia.

Figure 2.

Illustrative example of a RDF graph.

An RDF $g$ is a graph that can be formalized as a set of $G$ (RDF graphs) that can be structured in the form of,

$\displaystyle g=\><\textit{trp}_{1},\textit{trp}_{2},\ldots,\textit{trp}_{i}>$ (1)

Where in the RDF graph, $\textit{trp}_{1},\textit{trp}_{2},\ldots,\textit{trp}_{i}$ corresponds to the $i$ number of triples, where each triple is combined of subject, predicate, and object. The $g$ represents the full example (Fig. 2) of the RDF graph, $\textit{trp}_{1}$ represents the first triple found, $\textit{trp}_{2}$ represents the instance’s 2nd triple, and the final triple of a case is $\textit{trp}_{i}$ .

The suggested solution takes the issue of looking for an RDF graph as a problem of classification and suggests whether or not to retrieve an RDF graph. The description of retrieval of the newly RDF graph $g$ as the prediction function pred.

$\displaystyle r=\textit{pred(g) }r\in\left\{1\>or\>0\right\},g\in G$ (2)

Where, $r$ , pred, $g$ , and $G$ are a retrieval anticipation classification (1 indicates information is revived and 0 indicates no information retrieved), classification function, an RDF map, and a series of RDF charts, respectively.

3.3 Preprocessing

It is vital to validate the syntax of each RDF graph thus, we use the Apache Jena API1

¹
http:/jena.apache.org/.

API to validate each RDF graph. Next, the valid RDF graphs are loaded into the model to transform it from RDF to N-triple serialization. We analyze all the documents for feature extraction and to build the classification model for information retrieval. We employ the natural language preprocessing techniques (Word inflection, lemmatization, and Stop word removal) to normalize the abstracted features. We employ the Natural Language Toolkit (NLTK)2
²
http://www.nltk.org/.

of Python for these tasks. To transform words into their base-form that are superlative and comparative we utilize lemmatization. The Word inflection singularize a given word into its base form and as the name suggests stop word removal helps to remove the stop words. Next for the model training purpose, we build a global vocabulary system in which each unique word corresponds to a positive integer value that helps to transform the features into a vectorized shape that is suitable for model preparation. The vectorization process results in an

n

-dimension matrix. in which each row represents. We can formalize the preprocessing of an RDF graph as,

$\displaystyle g^{\prime}=<\textit{trp}_{1}^{\prime},\textit{trp}_{2}^{\prime},% \ldots,\textit{trp}_{i}^{\prime}>$ (3)

where, $\textit{trp}_{1}^{\prime},\textit{trp}_{2}^{\prime},\ldots,\textit{trp}_{i}^{\prime}$ are the number of preprocessed $i$ triples found in the RDF graph $g$ . An RDF graph $g^{\prime}$ can be presented after preprocessing (Fig. 3) for the motivational example given in Section 3.2.

Figure 3.

Illustrative example of preprocessed RDF graph.

3.4 Deep transfer learning

In transfer learning, the information learned during solving one problem is transferred and fine-tuned for another related problem. In recent years, transfer learning methods have applied in several fields such as metric learning [36], machine learning [37], image and text classification [38, 39, 40, 41] and dimensional reduction [42]. Figures 4 and b represents the model used for transfer learning that is based on the attention model for deep feature representation and CNN-based classifier.

Figure 4.

Architecture of pre-trained neural learning model used for deep transfer learning.

The pre-trained neural model [21] is then used to transfer knowledge from RDF graphs. The model is trained for information recovery prediction of RDF graphs with the blend of word2vec embedding technique combined with convolutional neural network (CNN) for the succeeding causes, we choose the convolutionary neural network: 1) it is capable of learning deep semantic relationship among terms [43]; 2) by applying different filter sizes And avoids the gradient issue of the repeated neural network [44].

3.5 Fine tuning with gated recurrent unit

An important understanding of the anticipated approach is to freezing the knowing information and then fine-tune it with fresh knowledge. For this purpose, we utilize the GRU [45] based recurrent neural network. The GRU learner pays consideration to the RDF-specific features to accomplish optimum performance. The overall workflow of the anticipated method is illustrated in Fig. 5.

Figure 5.

Architecture of transfer learning for RDF graphs.

The construction of the RNN neuron is presented in Fig. 6, where $\tau$ is input, $c$ is context and the outcome is $y$ . The computation of the hidden state activation is a function at time step $i$ on the previous $h_{i-1}$ sideways by present state $\tau_{i}$ .

$\displaystyle h_{i}=f(\tau_{i},h_{i-1})$ (4)

Normally $f$ is combined by the affine and element-wise nonlinear transformation of $h_{i-1}$ and $\tau_{i}$ .

$\displaystyle h_{i}=\phi(W\tau_{i},Uh_{i-1})$ (5)

where the input to the hidden layer weight matrix is $W$ and the state to state matrix weight matrix is $U$ , and the activation function is $\phi$ . The plain RNN experience the disappearing gradient issue that can be revoked by using a Gated Recurrent Unit (GRU) neural network. The GRU takes advantage of the full hidden content [46] overcoming the disappearing gradient problem. It is combined by two gates, update gate $z_{i}$ and rest gate $r_{i}$ and can be expressed as;

$\displaystyle h_{i}=(1-z_{i})h_{i-1}+z_{i}\hbar_{i}$ (6)

here $h$ is preceding context and $\hbar$ freshly provided context.

$\displaystyle z_{i}=\phi(W_{z}\tau_{i}+U_{z}h_{i-1})$ (7) $\displaystyle\hbar_{i}=tanh(W\tau_{i}+r_{i}\otimes Uh_{i-1})$ (8) $\displaystyle r_{i}=\phi(W_{r}\tau_{i}+U_{r}h_{i-1})$ (9)

Figure 6.

A RNN neuron architectural structure.

The difference from Eq. (5) is that $\hbar$ is modulated with the reset gates $r_{i}$ . Here Component-wise multiplication is $\otimes$ and $\phi$ is the function of activation.

4. Evaluation

To evaluate the suggested approach, this section describes the research questions, Explanation on how to gather RDF graphs, Introduces the suggested method metrics and calculation process, While addressing the study questions and explaining the findings.

4.1 Questions regarding research

The recommended solution is assessed by investigating the issues related to research:

•
RQ1: How precise is the recommended solution in the RDF graph retrieval prediction?
•
RQ2: Does the classification model that was suggested exceed other algorithms of machine/deep learning in the recovery estimation of RDF graphs?
•
RQ3: Does the preprocessing of features affect the estimation of the RDF graph retrieval?

RQ1 tests the accuracy of the solution presented. The suggested methodology is contrasted with a state-of-the-art technique in this perspective: RDF retrieval based on graphs (GRSearch) [47] and retrieval of RDF based on a machine (MLSearch) [20]. To emphasize the effectiveness of the proposed technique proposed, The suggested solution also is equated with two basic algorithms: algorithm for unselected prediction and algorithm of No-Rule prediction.

The RQ2 analyze the performance of various machine learning and deep learning classification models to show whether RDF is estimated in retrieval, Suggested methodology enhances other machine/deep learning classification models.

RQ3 explores the effect of transfer learning and the preprocessing of features. In this sense, the efficiency of the proposed solution with and without transfer learning and preprocessing is calculated and compared.
4.2 Methods and performance measures

The dataset is compiled from DBpedia.3

³
https:/wiki.dbpedia.org/data-set-30.

The 2016–10 DBpedia release comprises 13 billion kinds of information. Of these, 1.7 billion were obtained from the Wikipedia English edition. To test the suggested solution, just 1.7 billion RDF triples (English version) are included; however as stated in Section 3.3, we disregard all syntactically invalid triples.

There are three components of the evaluation process for the optimal classification model (CNN). Cross-validation (sometimes referred to as rotation estimation) [48] $M$ is set to $D$ in the first component. We divide $D$ into ten $MiI=1,2,\ldots,10)$ segments. We deduct and mark the RDF graphs belonging to $M i$ as Test Verifying graphs from RDF, And the following graphs for RDF are labeled as Train RDF having taught graphs.

We use $M$ -fold cross-validation in the second process and classification model training/testing (MNB, SVM, RF, LR, LSTM, CNN and Transfer Learning). First, we split the experiment data-set Train and the evaluation data-set Test after Cross-validation for each iteration. Classifiers are then trained with Train and all models is tested with Test. We evaluate the (accuracy ( $A^{\prime}$ ), recall ( $R^{\prime}$ ), precision ( $P^{\prime}$ ), and F1-measure ( $F1^{\prime}$ )) measures in the last stage of each model returning the best classifier.

For performance appraisal of classification algorithms [49, 50, 51, 52, 53, 54, 55], the chosen metrics are widely accepted metrics. Therefore, accuracy, recall, precision, and f-measure for the performance assessment of the proposed method, sufficient retrieval is calculated on the given RDF graphs, which can be described as,

$\displaystyle A^{\prime}=\frac{\textit{TP}+\textit{TN}}{\textit{TP}+\textit{TN% }+\textit{FP}+\textit{FN}}$ (10) $\displaystyle P^{\prime}=\frac{\textit{TP}}{\textit{TP}+\textit{FP}}$ (11) $\displaystyle R^{\prime}=\frac{\textit{TP}}{\textit{TP}+\textit{FN}}$ (12) $\displaystyle F1^{\prime}=\frac{2*P*R}{P+R}$ (13)

TP describes the number of predictions defined by the suggested approach as hit, TN describes the number of predictions estimated by the suggested approach as miss, FP Refers to the number of predictions which are wrongly estimated by the proposed solution as hit, And FN Refers to the number of predictions wrongly estimated by the solution suggested as miss.

4.3 Results and discussion

4.3.1 RQ1: Efficiency of the suggested model

We answer to RQ1 by the recommended techniques(state-of-the-art): MLSearch and GRSearch. We as well equate the suggested technique with an unselected prediction and a no-rule prediction. We endorse these approaches since the suggested method is mainly used to predict retrieval of RDF graph applying deep learning models.

Table 1 presents test consequences for the recommended strategy and baseline methods. In the first column of the table, methods are presented. For each classifier, the effects of the output metrics ( $A^{\prime}$ , $P^{\prime}$ , $R^{\prime}$ , and $\textit{F1}^{\prime}$ ) are displayed in Columns 2–5 of the chart. The output for corresponding strategy is displayed in each row of the graph. The average $A^{\prime}$ , $P^{\prime}$ , $R^{\prime}$ , and $\textit{F1}^{\prime}$ of the projected method, MLSearch, and GRSearch are (97.52%, 83.52%, and 79.53%), (98.45%, 89.17%, and 67.31%), (96.31%, 80.19%, and 64.19%), and (97.37%, 84.44%, and 65.71), respectively.

Table 1
Comparative analysis among baseline methods

	$A^{\prime}$	$P^{\prime}$	$R^{\prime}$	$\textit{F1}^{\prime}$
Transfer learning	97.52%	98.45%	96.31%	97.37%
MLSearch	83.52%	89.17%	80.19%	84.44%
GRSearch	79.53%	67.31%	64.19%	65.71%

The performance results of the unselected prediction evaluation, no-rule, and suggested solution are seen in the Table 2. In the first column of the table, methods are presented. For each classifier, the effects of the output metrics ( $A^{\prime}$ , $P^{\prime}$ , $R^{\prime}$ , and $\textit{F1}^{\prime}$ ) are displayed in Columns 2–5 of the chart. The table rows exhibit the performance of the processes, respectively. The average $A^{\prime}$ , $P^{\prime}$ , $R^{\prime}$ , and $\textit{F1}^{\prime}$ of the projected method, unselected prediction, and no-rule are (97.52%, 65.21%, and 86.26%), (98.45%, 65.40%, and 79.65%), (96.31%, 54.36%, and 82.17%), and (97.37%, 59.37%, and 80.89), respectively.

Table 2

Comparative analysis with random prediction and zero rule

	$A^{\prime}$	$P^{\prime}$	$R^{\prime}$	$\textit{F1}^{\prime}$
Transfer learning	97.52%	98.45%	96.31%	97.37%
Unselected prediction	65.21%	65.40%	54.36%	59.37%
No-rule prediction	86.26%	79.65%	82.17%	80.89%

The following are the observations from Tables 1 and 2:

•

The proposed solution outperforms the $A^{\prime}$ , $P^{\prime}$ , $R^{\prime}$ , and $\textit{F1}^{\prime}$ baseline frameworks, unselected estimation, and no-rule classifiers, respectively.

•

The advancement of the projected method upon MLSearch in $A^{\prime}$ and $\textit{F1}^{\prime}$ is 16.76% $=$ (97.52% – 83.52%)/83.52% and 15.31% $=$ (97.37% – 84.44%)/84.44%, respectively.

•

The advancement of the projected method upon GRSearch in $A^{\prime}$ and $\textit{F1}^{\prime}$ is 22.62% $=$ (97.52% – 79.53%)/79.53% and 48.18% $=$ (97.37% – 65.71%)/65.71%, respectively.

•

The advancement in the performance of the projected method upon unselected prediction in $A^{\prime}$ and $\textit{F1}^{\prime}$ is 49.55% $=$ (97.52% – 65.21%)/65.21% and 64.01% $=$ (97.37% – 59.37%)/59.37%, respectively.

•

The advancement in the performance of the projected method upon no-rule in $A^{\prime}$ and $\textit{F1}^{\prime}$ is 13.05% $=$ (97.52% – 86.26%)/86.26% and 20.37% $=$ (97.37% – 80.89%)/80.89%, respectively.

For the recommended method and baseline methods in Fig. 8, we show the accuracy distribution for 10-fold cross-validation. The distributions of each approach are equated with $\textit{F1}^{\prime}$ and one bean is drawn against each approach.Within a bean, each short horizontal line displays $\textit{F1}^{\prime}$ in fold of $i^{th}$ , While the long horizontal line indicates the average value of $\textit{F1}^{\prime}$ .We note that the suggested model in each fold exceeds the baseline approach. Especially, relative to the best performance of the benchmark solution, the average $\textit{F1}^{\prime}$ value of the proposed approach is considerably high. Figure 7 represents the learned semantics from the RDF graphs with the proposed approach.

Table 3

Machine learning methods compared

ML classifier	$A^{\prime}$	$P^{\prime}$	$R^{\prime}$	$\textit{F1}^{\prime}$
Transfer learning	97.52%	98.45%	96.31%	97.37%
CNN	95.24%	97.12%	96.70%	96.91%
LSTM	90.69%	90.08%	88.95%	89.51%
SVM	86.56%	87.48%	80.41%	83.80%
MNB	84.96%	76.07%	84.27%	79.96%
LR	91.21%	93.41%	96.41%	94.88%
RF	90.88%	92.88%	95.27%	94.06%

Figure 7.

Visualization of learned semantics from RDF Graphs with Gated Recurrent Unit.

Figure 8.

F-measure distribution comparison.

4.3.2 RQ2: Machine/deep learning models’ efficiency comparative analysis

Due to their competitive results, [56, 57, 54, 43], we respond to RQ2 By implementing the most commonly used algorithms for machine and deep learning classification (Transfer Learning, CNN, LSTM, LR, SVM, MNB, and RF). The validation of the recommended solution with SVM yields the most detailed outcomes and the other classification methods of the dataset outperform.

The Table 3 shows Transfer Learning, CNN, LSTM, LR, SVM, MNB, and RF evaluation performance.For each classifier, the output results of $A^{\prime}$ , $P^{\prime}$ , $R^{\prime}$ and $\textit{F1}^{\prime}$ are given in columns and rows display the output of a given classifier, respectively.

The average $A^{\prime}$ , $P^{\prime}$ , $R^{\prime}$ , and $\textit{F1}^{\prime}$ of Transfer Learning, CNN, LSTM, SNM, MNB, LR, and RF are (95.24%, 90.69%, 86.56%, 84.96%, 91.21%, and 90.88%), (97.12%, 90.08%, 87.48%, 76.07%, 93.41%, and 92.88%), (96.70%, 88.95%, 80.41%, 84.27%, 95.41%, and 95.27%), and (96.91%, 89.51%, 83.79%, 79.96%, 94.88%, and 94.06), respectively.

The Table’s conclusions 3 are as follows:

•
The Transfer Learning classification algorithm exceeds any of the other algorithms in accuracy, recall, precision, and $\textit{F1}^{\prime}$ . The explanation is that Transfer Learning performs better because it takes advantage of learned knowledge and fine tunes it resulting in enhanced performance.
•
While current [58] research states that the MNB classification algorithm is successful, It’s not, however, Compatible with the provided dataset approach. One potential explanation is that input features are interconnected with the training classification model, and MNB working well if [59, 43] is independent of the features. As opposed to LR, RF, and SVM with the projected method, evaluation performance of MNB on the provided dataset are not efficient. The LR and RF output figures are indeed identical to the SVM values.

It is deduced by the prior analysis that Transfer Learning fits better for the suggested solution than other classifiers.
4.3.3 RQ3: Impact of transfer learning and preprocessing of features

There may be identical features or superlative/comparative phrases for different RDF graphs. It is an undertaking to pass on information such as attributes to a machine learning model. It decreases performance and increases the cost of machine classification methods for processing. Moreover, we analyze the impact of transfer learning to improve modeling performance.

We respond to RQ3 by analyzing the results of the evaluation of the current approach with and without the pre-processing of the features and transfer learning. The results of the assessment are displayed in Table 4. In the first column of the table, the preprocessing input settings are displayed. The output results of $A^{\prime}$ , $P^{\prime}$ , $R^{\prime}$ and $\textit{F1}^{\prime}$ are presented in columns and rows present the output of the suggested solution to the various pre-processing and transfer learning settings, respectively. In the last row of the table, a difference in the efficacy of the suggested approach without pre-processing and transfer learning is provided.

Table 4
Impact of transfer learning and preprocessing

	$A^{\prime}$	$P^{\prime}$	$R^{\prime}$	$\textit{F1}^{\prime}$
Transfer learning	97.52%	98.45%	96.31%	97.37%
$+$ Preprocessing
Transfer learning	95.07%	95.28%	96.51%	95.89%
Preprocessing	92.12%	94.17%	96.28%	95.21%
Disable	79.33%	83.74%	84.68%	84.20%

Figure 9.

Impact of transfer learning and preprocessing.

We render the following findings from the Table 4:

•

The proposed solution allowed by preprocessing and transfer learning achieves considerable performance improvement. The findings of the assessment show that the efficiency achieved in $A^{\prime}$ , $P^{\prime}$ , $R^{\prime}$ , and $\textit{F1}^{\prime}$ are 97.52%, 98.45%, 96.31%, and 97.37%.

•

The disabled preprocessing method substantially cuts $R^{\prime}$ from 96.31% to 84.68%. In $R^{\prime}$ , The reduction provides the wrong result in response to the question asked. The identical or superlative/comparative terms in the triples given are one potential explanation for the decrease in results.

The preceding review concludes that the suggested solution includes transfer learning and preprocessing characteristics.

4.4 Threats to validity

Any factors that could impact the output of the proposed solution may be present. The foregoing are the drawbacks to the validity of the recommended technique.

•
The validity threat to construct is by choosing the measurement metrics. To test the proposed method, we picked $A^{\prime}$ , $P^{\prime}$ , $R^{\prime}$ , and $\textit{F1}^{\prime}$ metrics. Since for the assessment of classification questions, they are the most adopted metrics [49, 50, 51, 52, 53, 54, 60].
•
A challenge to create validity is the leveraging of python toolkit (NLTK) for preprocessing (Section 3.4). Due to its success and popularity, [54], we prefer NLTK. The use of some other repository for processing natural language could influence the above-mentioned effects of the projected method.
•
The generalization of the solution is an external validity hazard. For the assessment of the suggested solution, we concentrate on RDF graphs from the (DBpedia) open-access dataset. In the case of other datasets, we do not guarantee the conclusions of the proposed method.

5. Conclusion

Online users are continuously posting the moments of their life on the Internet in this modern age, creating information overload. Consequently, without knowing the semantics and syntax of the content, it is challenging to extract the necessary information accurately. To this end, we presented a transfer learning-based approach to the search for RDF graphs which addresses RDF graph requests as a classification problem. For the retrieval forecast of RDF graphs, the proposed solution applies a deep learning classifier to the specified dataset. The suggested solution provides a new way of looking for RDF graphs which encourages Web users to respond to their queries. Using DBpedia (open-source) RDF graphs, the 10-fold cross-validation is employed for the assessment of the projected method. The findings of the assessment suggest that the method presented is precise with an accuracy rate of 97.52%. The proposed approach explores this direction with the intention of information retrieval in the form of classification. In our future, work we intend to further explore this direction with domain-specific and cross-domain knowledge extraction.

References

Zhou

Ding

and Finin

, How is the Semantic Web evolving? A dynamic social network perspective, Computers in Human Behavior 27 (2011), 1294–1302. doi:10.1016/j.chb.2010.07.024.

Yan

and Ma

, An approach for approximate subgraph matching in fuzzy RDF graph, Fuzzy Sets and Systems 376 (2019), 106–126, Theme: Computer Science.

and Yan

, Fuzzy RDF: A Data Model to Represent Fuzzy Metadata, 2008, pp. 1439–1445.

and Yan

, Modeling fuzzy data with RDF and fuzzy relational database models, International Journal of Intelligent Systems 33 (2018).

Bizer

et al., Linked Data – The story so far, Vol. 5, 2009, pp. 1–22.

Bizer

, The emerging web of linked data, IEEE Intelligent Systems 24(5) (2009), 87–92. doi: 10.1109/MIS.2009.102.

Casanova

M.A.

, Keyword Search over RDF Datasets, in: Conceptual Modeling Laender

A.H.F.

Pernici

Lim

E.-P.

and de Oliveira

J.P.M.

, eds, Springer International Publishing, Cham, 2019, pp. 7–10. ISBN 978-3-030-33223-5.

Singh

Zong

and Singh

A.K.

, Nearest Keyword Set Search in Multi-Dimensional Datasets, IEEE Transactions on Knowledge and Data Engineering 28(3) (2016), 741–755. doi: 10.1109/TKDE.2015.2492549.

Gani

Siddiqa

Shamshirband

and Nasaruddin

, A survey on Indexing Techniques for Big Data: Taxonomy and Performance Evaluation, Knowledge and Information Systems 46 (2015). doi: 10.1007/s10115-015-0830-y.

10.

Elleuch

Zarka

Anis

B.a.

and Alimi

, A fuzzy ontology – Based framework for reasoning in visual video content analysis and indexing, Proceedings of the 11th International Workshop on Multimedia Data Mining, MDMKDD’11 – Held in Conjunction with SIGKDD’11 (2011). doi: 10.1145/2237827.2237828.

11.

Gacto

Alcalá

and Herrera

, Integration of an Index to Preserve the Semantic Interpretability in the Multiobjective Evolutionary Rule Selection and Tuning of Linguistic Fuzzy Systems, Fuzzy Systems, IEEE Transactions on 18 (2010), 515–531. doi: 10.1109/TFUZZ.2010.2041008.

12.

Komkhao

and Halang

W.A.

, Incremental collaborative filtering based on Mahalanobis distance and fuzzy membership for recommender systems, International Journal of General Systems – INT J GEN SYSTEM 42 (2012), 1–26. doi: 10.1080/03081079.2012.710437.

13.

and Chen

C.X.

, Efficient data modeling and querying system for multi-dimensional spatial data, 2008, p. 58. doi: 10.1145/1463434.1463503.

14.

Izakian

Pedrycz

and Jamal

, Fuzzy clustering of time series data using dynamic time warping distance, Engineering Applications of Artificial Intelligence 39 (2015), 235–244. doi: 10.1016/j.engappai.2014.12.015. https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0952197614003078.

15.

Lughofer

and Pratama

, Online active learning in data stream regression using uncertainty sampling based on evolving generalized fuzzy models, IEEE Transactions on Fuzzy Systems 26(1) (2018), 292–309. doi: 10.1109/TFUZZ.2017.2654504.

16.

Idreos

Papaemmanouil

and Chaudhuri

, Overview of Data Exploration Techniques, in: Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, SIGMOD ’15, ACM, New York, NY, USA, 2015, pp. 277–281. ISBN 978-1-4503-2758-9. doi: 10.1145/2723372.2731084.

17.

Reh

Amirkhanov

Kastner

Groller

and Heinzl

, Fuzzy feature tracking: Visual analysis of industrial 4D-XCT data, Computers and Graphics 53 (2015), 177–184. doi: 10.1016/j.cag.2015.04.001.

18.

Gupta

Saini

and Saxena

A.K.

, A new fuzzy logic based ranking function for efficient Information Retrieval system, Expert Systems with Applications 42(3) (2015), 1223–1234. doi: 10.1016/j.eswa.2014.09.009. https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S095741741400548X.

19.

Arnaout

and Elbassuoni

, Effective searching of RDF knowledge graphs, Journal of Web Semantics 48 (2018), 66–84.

20.

Hadi

A.S.

Fergus

Dobbins

and Al-Bakry

A.M.

, A Machine Learning Algorithm for Searching Vectorised RDF Data, in: 2013 27th International Conference on Advanced Information Networking and Applications Workshops, 2013, pp. 613–618. doi: 10.1109/WAINA.2013.204.

21.

Soliman

, Deep learning based searching approach for RDF graphs, Plos one 15(3) (2020), e0230500.

22.

Tran

Wang

Rudolph

and Cimiano

, Top-k Exploration of Query Candidates for Efficient Keyword Search on Graph-Shaped (RDF) Data, in: Proceedings of the 2009 IEEE International Conference on Data Engineering, ICDE ’09, IEEE Computer Society, Washington, DC, USA, 2009, pp. 405–416. ISBN 978-0-7695-3545-6. doi: 10.1109/ICDE.2009.119.

23.

Zhang

Tran

and Rettinger

, Probabilistic query rewriting for efficient and effective keyword search on graph data, Proc. VLDB Endow. 6(14) (2013), 1642–1653. doi: 10.14778/2556549.2556550.

24.

Yang

Ding

Chaudhuri

and Chakrabarti

, Finding patterns in a knowledge base using keywords to compose table answers, Proc. VLDB Endow. 7(14) (2014), 1809–1820. doi: 10.14778/2733085.2733088.

25.

Zheng

Zou

Peng

Yan

Song

and Zhao

, Semantic SPARQL Similarity Search over RDF Knowledge Graphs, Proc. VLDB Endow. 9(11) (2016), 840–851. doi: 10.14778/2983200.2983201.

26.

De Virgilio

, RDF Keyword Search Query Processing via Tensor Calculus, 2012, pp. 780–788. doi: 10.1007/978-3-642-33615-7_22.

27.

De Virgilio

and Maccioni

, Distributed Keyword Search over RDF via MapReduce, in: The Semantic Web: Trends and Challenges Presutti

d’Amato

Gandon

d’Aquin

Staab

and Tordai

, eds, Springer International Publishing, Cham, 2014, pp. 208–223. ISBN 978-3-319-07443-6.

28.

N.D.

Reformat

M.Z.

and Yager

R.R.

, Linked Open Data: Uncertainty in Equivalence of Properties, in: Advances in Fuzzy Logic and Technology 2017 Kacprzyk

Szmidt

Zadrożny

Atanassov

K.T.

and Krawczak

, eds, Springer International Publishing, Cham, 2018, pp. 418–429. ISBN 978-3-319-66827-7.

29.

Nagarajan

and Minu

R.I.

, Fuzzy ontology based multi-modal semantic information retrieval, Procedia Computer Science 48 (2015), 101–106, International Conference on Computer, Communication and Convergence (ICCC 2015). doi: 10.1016/j.procs.2015.04.157. https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S1877050915006663.

30.

Dong

and Hirota

, Formalization and visualization of kansei information based on fuzzy set approach, in: Fifty Years of Fuzzy Logic and its Applications Tamir

D.E.

Rishe

N.D.

and Kandel

, eds, Springer International Publishing, Cham, 2015, pp. 169–181. ISBN 978-3-319-19683-1.

31.

Pancho

D.P.

Alonso

J.M.

and Magdalena

, Enhancing Fingrams to deal with precise fuzzy systems, Fuzzy Sets and Systems 297 (2016), 1–25, Themed Section: Fuzzy Systems. doi: 10.1016/j.fss.2015.05.019. https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S0165011415002948.

32.

Besbes

and Zghal

, Personalized and context-aware retrieval based on fuzzy ontology profiling, Integrated Computer Aided Engineering 24 (2016). doi: 10.3233/ICA-160525.

33.

Jaafar

Danyaro

K.U.

and Liew

M.S.

, Web intelligence: A fuzzy knowledge-based framework for the enhancement of querying and accessing web data, in: Handbook of Research on Trends and Future Directions in Big Data and Web Intelligence, 2015, pp. 83–104. doi: 10.4018/978-1-4666-8505-5.ch005.

34.

Kyu

K.M.

and Oo

A.N.

, Graph-based Indexing Method for Searching in RDF Data, in: 2019 International Conference on Advanced Information Technologies (ICAIT), 2019, pp. 96–101. ISSN null.

35.

Gayathri

and Rajendran

V.V.

, Semantic search on summarized RDF triples, in: 2017 International Conference on Intelligent Computing and Control (I2C2), 2017, pp. 1–6. ISSN null.

36.

and Tan

Y.-P.

, Deep transfer metric learning (2015), 325–333.

37.

Duan

Tsang

I.W.

and Maybank

S.J.

, Domain transfer svm for video concept detection (2009), 1375–1381, IEEE.

38.

Khan

Islam

Jan

Din

I.U.

and Rodrigues

J.J.C.

, A novel deep learning based framework for the detection and classification of breast cancer using transfer learning, Pattern Recognition Letters (2019).

39.

Shin

Roth

H.R.

Gao

Nogues

Yao

Mollura

and Summers

R.M.

, Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning, IEEE Transactions on Medical Imaging 35(5) (2016), 1285–1298.

40.

Yuan

Zheng

and Lu

, Hyperspectral image superresolution by transfer learning, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 10(5) (2017), 1963–1974.

41.

Hussain

Huang

Zhou

and Wang

, Deep transfer learning for source code modeling, International Journal of Software Engineering and Knowledge Engineering 30(5) (2020), 649–668.

42.

Pan

S.J.

Kwok

J.T.

Yang

, Transfer Learning via Dimensionality Reduction. 8 (2008), 677–682.

43.

Ramay

W.Y.

Umer

Yin

X.C.

Zhu

and Illahi

, Deep neural network-based severity prediction of bug reports, IEEE Access 7 (2019), 46846–46857. doi: 10.1109/ACCESS.2019.2909746.

44.

Hinton

G.E.

Srivastava

Krizhevsky

Sutskever

and Salakhutdinov

, Improving neural networks by preventing co-adaptation of feature detectors (2012). http://arxiv.org/abs/1207.0580.

45.

Chung

Gulcehre

Cho

and Bengio

, Empirical evaluation of gated recurrent neural networks on sequence modeling, arXiv preprint arXiv:1412.3555, 2014.

46.

Young

Hazarika

Poria

and Cambria

, Recent trends in deep learning based natural language processing, IEEE Computational IntelligenCe Magazine 13(3) (2018), 55–75.

47.

Gayathri

and Rajendran

V.V.

, Semantic search on summarized RDF triples, in: 2017 International Conference on Intelligent Computing and Control (I2C2), 2017, pp. 1–6. doi: 10.1109/I2C2.2017.8321904.

48.

Purushotham

and Tripathy

B.K.

, Evaluation of classifier models using stratified tenfold cross validation techniques, in: Global Trends in Information Systems and Software Applications Krishna

P.V.

Babu

M.R.

and Ariwa

, eds, Springer Berlin Heidelberg, Berlin, Heidelberg, 2012, pp. 680–690. ISBN 978-3-642-29216-3.

49.

Tian

and Sun

, Information retrieval based nearest neighbor classification for fine-grained bug severity prediction, in: Proceedings of the 2012 19th Working Conference on Reverse Engineering, WCRE ’12, IEEE Computer Society, Washington, DC, USA, 2012, pp. 215–224. ISBN 978-0-7695-4891-3. doi: 10.1109/WCRE.2012.31.

50.

Tian

and Sun

, DRONE: Predicting priority of reported bugs by multi-factor analysis, in: 2013 IEEE International Conference on Software Maintenance, 2013, pp. 200–209. ISSN 1063-6773. doi: 10.1109/ICSM.2013.31.

51.

Lamkanfi

Demeyer

Giger

and Goethals

, Predicting the severity of a reported bug, in: 2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010), 2010, pp. 1–10. ISSN 2160-1852. doi: 10.1109/MSR.2010.5463284.

52.

Lamkanfi

Demeyer

Soetens

Q.D.

and Verdonck

, Comparing mining algorithms for predicting the severity of a reported bug, in: 2011 15th European Conference on Software Maintenance and Reengineering, 2011, pp. 249–258. ISSN 1534-5351. doi: 10.1109/CSMR.2011.31.

53.

Yang

Baek

Lee

J.-W.

and Lee

, Analyzing emotion words to predict severity of software bugs: A case study of open source projects, in: Proceedings of the Symposium on Applied Computing, SAC ’17, ACM, New York, NY, USA, 2017, pp. 1280–1287. ISBN 978-1-4503-4486-9. doi: 10.1145/3019612.3019788.

54.

Umer

Liu

and Sultan

, Emotion based automated priority prediction for bug reports, IEEE Access 6 (2018), 35743–35752. doi: 10.1109/ACCESS.2018.2850910.

55.

Hussain

Huang

Zhou

and Wang

, DeepVS: an efficient and generic approach for source code modelling usage, Electronics Letters (2020).

56.

Kumar

Ross Quinlan

Ghosh

Yang

Motoda

McLachlan

G.J.

Liu

P.S.

Zhou

Z.-H.

Steinbach

Hand

D.J.

and Steinberg

, Top 10 algorithms in data mining, Knowledge and Information Systems 14(1) (2008), 1–37. doi: 10.1007/s10115-007-0114-2.

57.

Sohrawardi

S.J.

Azam

and Hosain

, A comparative study of text classification algorithms on user submitted bug reports, in: Ninth International Conference on Digital Information Management (ICDIM 2014), 2014, pp. 242–247. doi: 10.1109/ICDIM.2014.6991434.

58.

Hellerstein

Jayram

T.s.

and Rish

, Recognizing End-User Transactions in Performance Management, 2000, pp. 596–602.

59.

Umer

Liu

and Sultan

, Sentiment based approval prediction for enhancement reports, Journal of Systems and Software 155 (2019), 57–69.

60.

Hussain

Huang

Zhou

and Wang

, CodeGRU: Context-aware deep learning with gated recurrent unit for source code modeling, Information and Software Technology (2020), 106309.

Learning to transfer knowledge from RDF Graphs with gated recurrent units

Abstract

Keywords

1. Introduction

3. Approach

3.1 Overview

1 http:/jena.apache.org/.

4.1 Questions regarding research

3 https:/wiki.dbpedia.org/data-set-30.

4.3.1 RQ1: Efficiency of the suggested model

Table 1 Comparative analysis among baseline methods

Table 4 Impact of transfer learning and preprocessing

References

¹
http:/jena.apache.org/.

³
https:/wiki.dbpedia.org/data-set-30.

Table 1
Comparative analysis among baseline methods

Table 4
Impact of transfer learning and preprocessing