Over comparative study of text summarization techniques based on graph neural networks

2.2.1. Term frequency-inverse document frequency

The word frequency is characterized as the number of sentences involved in the document that includes the specific terms. These sentences are then scored by the query similarity, in which the pre-eminent scoring sentences are obtained as the segments of the summary.

2.2.2. Cluster-based technique

Generally, the documents are written in such a way for addressing numerous topics successively in a structured format. It is essential to consider that summaries are needed to highlight different themes in the document. Document clustering remains an important tool to create meaningful summaries for the collection of documents that are generated from entirely different topics. The summary sentences are selected based on the similarity of the sentence to the theme of the cluster. The sentence locality in the document is also considered an essential factor for sentence selection. The higher weightage is given to the sentence closer to the commencement of the sentence. Another factor that enhances the sentence score is the similarity of the sentence with the first sentence in the document [39].

2.2.3. Machine learning technique

While considering the machine learning techniques the summarization process is customized as the classification issue, in which the sentences are categorized into summary and non-summary sentences based on the features.

The text summarization based on machine learning is detailed in this sub-section. The semi-supervised deep learning method for text summarization was used by [24] using the context vector of the document. In this, the contextual information was added while encoding the data by the humans, and then by proper training of the deep learning classifier, the extractive features were generated and the abstractive summarization was performed based on the extractive features. For the abstractive text summarization, the context or non-context information was utilized and the summarization was performed. A better performance was obtained based on the accuracy of the summarization task. The text summarization based on the deep learning technique was devised by [38] using the Explicit Semantic Analysis (ESA) and Semantic Role Labeling (SRL) approach. In this, the words were considered as the weight vectors and then the graph representation was performed for the summarization based on the sentence level. Here, the input document is segmented and pre-processed and then, the labeling and grouping were performed. Finally, based on the interpretations, the summary was generated and obtained elevated performance based on the semantic representation.

The machine learning-based text summarization was employed by [2] for the Bengali text using the extractive strategy. Here, the Bengali news article was utilized for the evaluation of the text summarization, in which the stop word removal and the tokenization were performed as the pre-processing step, then the vector encoding was devised for the summarization using a neural network based on probability. Fuzzy-based text summarization was utilized by [44] for news articles and text documents. Here, the fuzzy rules were utilized for the classification of the text, and then the most significant sentences were utilized for the summarization [25]. The hybrid generator-pointer network was utilized for the summarization of the text. The ensemble-based text summarization was utilized by [50] using the hybrid LSTM and Fuzzy logic, in which the most informative sentence was extracted using the fuzzy logic, and the weights of the LSTM were tuned using the Adam optimization and the attention mechanism. Multi-document summarization system is utilized with the deep neural network [56] for the summarization of documents in a coherent and fluent flow of documents covering the major information in the documents. The method consists of graphs for modeling sentence relationships within clusters namely document clusters, and the G-Flow graph system is utilized that create its graph representation by drawing on the discourse relationships between sentences. In this method, By calculating discourse relation indicators, the ADG creates edges between sentences. Compared to a baseline GRU model without any graphs and typical graph-based extractive summarizers, the model exhibits a definite benefit and produces competitive outcomes with other cutting-edge MDS systems. A recursive neural network employed with the ranking framework model [9] is utilized to prioritize them for multi-document summarizing, a hierarchical regression approach that assesses the salience information of all the non-terminal nodes in the parse tree and concerns the sentence ranking task. Recursive neural networks (ReNN) are used to model this procedure. R2N2 produces some advantages for multi-document summarization, it is possible to consider more supervisory expertise, which is good for learning additional ranking features, more precise sentence selection strategy is made possible by the consistent ranking scores that deliver from words to sentences. A computational approach for tracking faces in real time has been introduced [48]. In this approach, the detection of the face is identified using the shadow of a person, motion, and color. Authentication is satisfied by some requirements which are classified as face slant and face size are invariable, variation of lights invariable, and there is no need in taking other aspects into account while integrating a new subject. Face tracking is done in three stages image processing, image search, and pattern classification. The devised method minimizes storage space and training time for the high-dimensional dataset. The accuracy of the results is increased and classification is relatively simpler after creating a subset of features. The presented deep recurrent generative decoder (DRGD) equipped sequence-to-sequence oriented encoder-decoder model [28] to model and learn the latent structural information revealed in the target summaries of the training data. For the recurrent latent variables, the neural variation inference is used to address the unsolvable posterior inference. Using back-propagation an end-to-end standard is enhanced in all the parameters of the model. The method utilized to perform unsupervised extractive and abstractive text summarization [41] using sentence embeddings. The unsupervised Text Summarization model uses an RNN with LSTM to encode texts in a vector space. Similar to the skip-gram method for learning word embeddings and sentence embeddings are utilized. This work tries to learn how to combine word embeddings to produce sentence embeddings that satisfy the property that sentences are paraphrases of each other, and are positioned close to one another in the vector space. Automatic Text summarization is utilized to reduce the vocabulary in words [42] using the continuous vector approach. Using four ATS systems and the ROUGE evaluation methods, this technique is assessed using the MultiLing corpus. The utilized research demonstrates that a modern system performs better when the vocabulary is less. Four systems are used in the automatic text summarization method to evaluate the reduced vocabulary approach specifically, TextRank, Sasi, MMR, and base. Text Summarization based on the DUC 2001 dataset was devised by [36] using Deep Learning and Ridge Regression approach. Sentence selection and sentence ranking are the two key processes used in a summarization task. Using the Document Understanding Conference (DUC) dataset, summarization methods can be tested and evaluated. Each layer of nodes in an MLP is fully connected to the layer below it in a directed graph, which has several layers. The forward pass and the reverse pass are the two primary passes in MLP training.

2.2.4. Neural network approach for text summarization

This technique includes the training of the neural network to discern different categories of the sentences that are involved in the summary. The neural network thus comprehends the patterns that are involved in the sentences and it should be added to the summary. The neural network utilized three-layered feed-forward neural networks to generate the preferable summaries of the document.

2.2.5. Graph theoretic techniques

The graph theoretic illustration of the sentence provides the method for the recognition of the themes. After the general pre-processing phases, such as stop word removal and stemming the passages in the documents are illustrated as nodes in the uni-directional graph. There are nodes for each sentence and they are connected if the edges of the sentence allot the common phrase. For the query-specific summary, the sentences are selected from the relevant subgraphs, and for the generic summary, the sentences are selected from each of the subgraphs.

The GNN-text summarization is detailed in this section. The text summarization using the GNN was utilized by [22] for the extraction of a better summary from the text. The word graph was generated initially based on the semantic relation and then the sentence block was generated, which instead of the word sentence block was created. Then, the scores were assigned to each sentence block through the reading and post-reading approaches, in which the redundancy was eliminated through the tri-gram blocking strategy. The performance was analyzed based on human evaluation and experiments using the R-1 measure. The text summarization based on enhanced extraction using the Topic aware GNN was employed by [11] for compressing text data. In this technique, the inter-sentence relationship was evaluated to reduce the topical information rejection while summarizing the text. Besides, the document-level sentence selection was done through the neural topic system to detect latent topics. The text summarization employed offered a robust performance in terms of long documents and genres by considering R-1, R-2, and R-L.

The text summarization using the heterogeneous GNN was employed by [52] through several granularity levels. Here, the cross-sentence relation was elevated through the additional nodes that perform the intermediary among the sentences. As in method [22], this technique also utilizes tri-gram blocking to remove redundant information while choosing the sentence. GNN-based abstractive summarization of text was utilized by [30] using the integrated gated GNN with the Bi-directional long short-term memory (BiLSTM), by using this encoding process the key feature elimination was eliminated that enhancing the performance of the system. Here, for the entire sentence, the word co-occurrence was generated through edge building among the nodes. In addition, the seq2seq encoding makes the system encode all the necessary features. The system obtained enhanced performance nearer to the semantics and diversities.

The Graph convolutional neural network (GCN) based text summarization was developed by [29,31], and [51]. GCN-based extractive text summarization was employed by [31] using the attention matrix. Here, the graph network obtains the necessary information through the sentence relation, and the GCN was utilized for the generation of the adjacent matrix, in which the training was not performed for the reduction of the computational complexity. The event summarization was devised by [29] using the GCN based on the semantic graph. In this, the Twitter event data was considered for the summarization based on the semantic graph and score-based salience evaluation. The tweet representation was devised using the fine-tuned approach and then the relation graph-based clustering was performed finally, the salience evaluation was made for the summarization of the Twitter data. The best performance was evaluated through a large real dataset in terms of semantic terms. The text summarization using the Graph convolutional network (GCN), a type of GNN was used by [51] for solving the issue concerning the language-specific dependencies. The word and the sentence level text representation was employed through the BERT and FGOW, feature graph of words encoder, and then the summarization was devised through the GCN. Besides, the pre-trained BERT was utilized for the representation of the semantic latent at the document level. The document was divided into a sentence for summarization using the Sentence pierce tool. Knowledge Transfer for Out-of-Knowledge-Base Entities based on a GNN is utilized by [19] to detect absent knowledge in a database. The relation triplets that are provided to the system after training contain additional entities (OOKB entities), which causes this issue. Since the system was not trained on these entities, it does not have their embeddings and is hence unable to anticipate their relations. While it can be resolved by retraining the embeddings using the newly inserted relation triplets containing the OOKB entities, a less expensive alternative would be preferable. The propagation model and output model are used in the GNN to solve the OOKB entity problem. Graph information that is distributed between the nodes is managed by the propagation model and the output model defines the task-oriented objective function over node vectors. Retrieval-augmented generation is utilized for code summarization using a Hybrid GNN [32] to add the retrieved code and the related summary as additional data for training the model and to obtain the source files from the retrieval database which is most comparable to the present program. In addition to capturing global semantics among nodes, hybrid message-passing GNN is utilized based on both static and dynamic graphs. Four components of the presented Hybrid GNN include Retrieval-augmented Static Graph Construction, Attention-based Dynamic Graph Construction, HGNN, and Decoder. The structured neural summarization approach is utilized by [15] natural language processing system in the utilized summarization approach to convert the complex input into a simple representation. Framework adapted for the summarization of input data includes sequence to sequence. A mechanism called memories is obtained by combining the inputs of each token with the final state of RNN. The neural message-passing technique shares the information through a graph. Hierarchical Heterogeneous graph attention network applied for Syntax-Aware summarization is utilized by [49] to pick out important lines from the original abstract or text that might come up with new sentences and words. Two layers are utilized from the presented GNN approach the hierarchical graph pooling layer which obtains the tree information hierarchically and the dependency of constituent pair is detected using meta-data called the syntax-aware graph attention layer. Since the model can learn a global representation for each sentence, it can be simply modified for the extractive summarization job, allowing a graph-level classifier to be trained. The datasets PubMed, WikiHow, Reddit, XSum, New York Times, and CNN/DM are employed in this approach for the summarization of single documents. Fairness dictates that the same pre-trained model is used to replace the word embedding component in DISCOBERT and HAHSum. A Graph-to-Sequence Learning Framework for Summarizing Opinionated Texts is utilized by [53] to create a valid summary for a collection of writings with strong opinions about a certain subject. When directing the decoder, the integrated MMR process can take into account both non-redundant opinions and salient opinions with marginal relevance scores, and avoid listening to outdated, repetitious viewpoints produced at earlier time stages. Opinionated words are summarized using the two-stage graph-to-sequence framework where the estimation of prominent works performed in stage one and stage two generates the summary. The datasets used in the framework of summarization are Idebate, and RottenTomatoes. The most recent technique for abstractive multi-document summarization is used in the presented approach called HT (Hierarchical Transformer). It improves the Transformer’s encoder in which each layer accepts one paragraph as input, and inter-paragraph attention is used to describe the dependency of many paragraphs. In the utilized A Graph-to-Sequence Learning Framework, for salience regression kNNR-S2S makes use of kNNR, the graph encoder is taken out by GNR-S2S, and the graph encoder is taken out by GNR-S2S. GATSum: Graph-Based Topic-Aware Abstract based on Text Summarization is utilized by [21] provides instructions on how to create a summary and helps to collect global semantic features and resolves inadequate global semantic features in text summary. A book summary aims to make it easier for readers to rapidly understand the important points of a text. Two types of summarization approaches are described in the presented method that is abstract and extractive. Important text from the source text is mostly copied by extractive models, which then combine it with other text. Abstract models can provide summaries of high quality since they are more closely related to the substance of summarization. The output of a sequence is generated under the input sequence at each decoding step, and this process is continued until a termination sign is found. The output of the final result comes from the decoder side after each time step performs input for the following time step. The excellent performance of GTASum underlines the importance of introducing topic model components and graph attention network components to improve the generation effect. The successful implementation of GTASum emphasizes the significance of adding a topic model and graph attention network components to enhance the generating effect. Text Graph Transformer for Document Classification utilized by [57] to build efficient node representations by observing heterogeneity and structure in the text graph. The document node represents all of the vocabulary from the corpus and the word node represents the entire corpus of documents are the two types of nodes in the text graph. In the devised method the text graph’s document and word kinds can be captured using heterogeneity encoding. To encrypt word nodes and document nodes utilize the values 0 and 1. As k rises from 1 to an ideal value, learning performance steadily improves and then starts to decline as k raises more. Improved Code Summarization technique based on GNN is utilized by [27] provides a method that creates these summaries by employing a graph neural architecture that closely resembles the AST’s default structure. In code metaphor when summarizing, one language input is used for source code and another for summaries. Input for the same model’s encoder would be code, and input for the decoder would be summaries. The Code is typically treated as a sequence to be input into a recurrent neural network (RNN) or other similar structure optimized for sequential information in typical applications of NMT to code summarization. Recurrent layers work well for learning sequence information since they transmit information for each token in a sequence. At each time step of the input sequence, some of the popular recurrent layers, like the GRU and LSTM, can additionally return state information. Enhancing Extractive Text Summarization with Topic-Aware GNN utilized by [12] Using a graph-structured document representation, it is possible to efficiently record inter-sentence interactions. Additionally, to find latent topics, the devised model incorporates a combined neural topic model (NTM), which can deliver document-level characteristics for sentence selection. Sentence representations can be enhanced by topical data throughout the graph propagation process, which can be thought of as a type of document-level feature that aids in our model’s ability to extract the key information from a document’s entirety. The devised model not only outperforms existing approaches on datasets of scientific papers with much longer documents when compared to state-of-the-art extractive models for news datasets but also significantly outperforms them. This shows that the proposed model is more robust to different document lengths and genres. HETFORMER, a Heterogeneous transFORMER-based pre-trained model for long-text extractive summarization using multi-granularity sparse attention is utilized by [33] employing sparse attention, a new structured pre-trained technique that may capture the diverse structure of documents. The presented method shows that both single- and multi-document extractive summarization tasks can be completed with cutting-edge performance. This method is enhanced with three categories of nodes token, phrase, and entity to handle various semantic granularities.

2.2.6. Others

The text summarization other than the machine learning and the GNN is detailed in this section. The text summarization using the rule-based technique for the Bangla documents was utilized by [16]. Here, summarization was employed for the news content to read the news with minimal time on the Internet. The scoring of the sentence was devised based on the graph strategy by considering 12 different features. Besides, the redundant information from the news document was removed for efficient text summarization, in which the news document was segmented into sentences before the summarization. The summarization employed by the method was closely similar to human summarization. The automatic text summarization was employed by [20] using the extractive approach, in which the features such as the lexical and the semantics were extracted and then the clustering was performed to group similar words and then the keywords were extracted for the summarization process. The clustering was devised based on the Genetic algorithm and then based on the ranking the text summarization was performed. The global ranking-based criterion makes the system obtain an enhanced performance for various languages. The text summarization based on the text rank was devised by [35] using the improved graph-based approach. Here, the distance-based similarity among the sentences was evaluated for the extractive text summarization. The stop words were removed for the text data and then the word tokenization was employed for the graphical representation. Then, the sentences were ranked using the ranking algorithm and summarization was performed. The weight-based text summarization technique obtained better performance.

The text summarization based on the keyword extraction was used by [47] for the information retrieval from the search engine. Here, the keyword was extracted automatically for re-framing the query given by the user from the information fetched from the search engine. The summarized text was used as a link for content retrieval. The optimization of the search engine through text summarization was employed by [37] using the meta description, which was obtained through the query-based algorithm. In this approach, the keywords are extracted from the query, and summarization was employed. Here, the frequency of the keyword was counted for scoring the sentence to summarize the text. The text summarization for the Arabic document was employed by [43] based on the semantic and statistical features. Here, the preprocessing of the text was performed the removal of the redundant information and then the key features extracted the score was assigned and the devised system obtained better performance by considering the importance of the sentence. The text summarization based on the extractive strategy was devised by [1] for the Arabic text, using the general-purpose architecture. In this, the Tatweel and the Taskeel process was utilized for the removal of noise from the input data and then the sentence embedding and the clustering were performed for the summarization of the text, in which the Arabic BERT was utilized for embedding the sentence based on the pre-trained approach. The summary of the text was obtained through the K-Means clustering method. The domain knowledge-based text summarization was utilized by [3] for the Arabic text document. Here, the segmentation of the document into a sentence was employed and then the stop words were removed from the segmented sentence for the extraction of the informative features. Finally, the scoring was performed for the summarization of the text using the Genetic algorithm. The Arabic text summarization obtained a promising performance enhancement based on the F-Measure average evaluation. The Arabic text summarization based on the rank-based strategy was employed by [14] for the extractive summarization. In this, the score for the text document was generated using the modified page rank algorithm. The cosine-based similarity among the sentence was evaluated to obtain the more informative keywords by maintaining the relation among them. Initially, the stop word removal, tokenization, and normalization were performed in the pre-processing, and then features were extracted through the distance-based measure and finally the summarization based on the extractive features was devised using the modified algorithm. Better performance was achieved while elevating the iteration numbers. A novel application of continuous vector representations to the problem of multi-document summarization is utilized by [23] to capture the important information contained in large volumes of text and present it in a brief, representative, and consistent summary. Due to the improved benefits of continuous word vectors, the framework of state-of-the-art performance is improved by evaluating the devised function of the summarization method. In the presented method of summarization, the syntactic and semantic information about the word is captured using the continuous vector and reveals the word’s similarity by comparing the words based on their size. The field of automatic summarization will advance as the word vectors’ quality rises and discover better methods of creating and comparing the vectors. The paper presents a general framework for single document summarization with side information utilized by [40] in which the selected sentences from the input document are used as the output summary by the sentence extractor, which combines this document meaning representation with an attention mechanism over the side information. A CNN and RNN network in the framework of extractive summarization connects the hierarchical decoder-encoder architecture. The compositionality of the document is exploited by the model and it shows how a text is composed of sentences that make sense and that each sentence is composed of words that make sense. Because of its general characteristic side information plays an important role in the summarization of important sentences from the document and is extracted easily. The presented paper shows a demonstration of how to expand the training of gated GNN to such massive graphs and how to create graphs from source code utilized by [5] On two tasks – VARMISUSE, where the network learns to decide which variable should be used at a specific program location, and VARNAMING, where a network tries to guess a variable’s name given its usage – we test our devised model. The abstract syntax tree (AST) of the program, which is made up of syntax nodes that correspond to non-terminals in the grammar of the programming language and syntax tokens that correspond to terminals, forms the basis of a programmed graph. Due to the difficulty of efficient batching when there are several shape types, using GGNNs for sets of big, diverse graphs takes significant engineering work. The presented model is to familiarise natural language researchers with neural approaches utilized by [17] by reviewing neural network models from the standpoint of natural language processing research. The model covers computation graph abstraction for autonomous gradient calculation, feed-forward networks, convolutional networks, recurrent networks, and recursive networks in addition to input encoding for natural language problems. In terms of the interdependencies between the various components, the graph structure specifies the order of the calculation. The provided model uses an RNN technique to encode the sequence into a vector representation, and this vector representation is then used as an auxiliary input to another RNN that serves as a decoder. Strongly learning, neural networks can do non-Markovian modeling of trees and sequences as well as non-linear classification. The extractive strategy of summarization used in this method is utilized by [13] which combines a document and sentence clustering approach. The approach guarantees adequate coverage and prevents duplication. By initially grouping related documents into clusters, the clustering-based technique next groups sentences from each document cluster into sentence clusters. The final summary is composed of the highest-scoring sentences from the sentence clusters. If a word exists in a query or a phrase, its weight is assigned to the relevant vector, and its omission from the vector is indicated by a value of 0. The benefits of using the clustering mechanism in the presented method help reduce the redundancy in the techniques of extractive summarization. The presented method was utilized by [26] to improve the summarization quality in a sentence of Arabic text by combining the methods of semantic and linguistic feature selection. In the first step of the devised method, the RST is used to generate a primary summary. In the second phase, each sentence in the primary summary is given a score depending on how semantically related it is to the text’s subtitles and title. Discourse segmentation involves dividing a source text into non-overlapping EDUs, or elementary discourse units. Identification of rhetorical relations is used in the summarization of Arabic text to pinpoint the discourse connections between text spans. Presented a method for text summarizing that integrates sentence features and grammatical structure, which is utilized by [45]. The material used for text summarization is prepared by a series of preparation stages, including word stemming, stop words removal, and tokenization. In the hybrid-based text summarization method the feature extraction, which involves the discovery of significant features, is applied to the preprocessed material. Using dependency grammar, which breaks down sentences into words and connects these words with dependency relations, the syntactic structure of the retrieved sentences is then examined.