Over comparative study of text summarization techniques based on graph neural networks

Abstract

Due to the enormous content of text available online through emails, social media, and news articles, it has become complicated to summarize the textual information from multiple documents. Text summarization automatically creates a comprehensive description of the document that retains its informative contents through the keywords, where Multi-Document Summarization (MDS) is a productive tool for data accumulation that creates a concise and informative summary from the documents. In order to extract the relevant information from the documents, Graph neural networks (GNNs) is the neural structure that detains the interrelation of the graph by progressing the messages between the graphical nodes. In the current years, the advanced version of GNNs, such as graph attention network (GAN), graph recurrent network, and graph convolutional network (GCN) provides a remarkable performance in text summarization with the advantage of deep learning techniques. Hence, in this survey, graph approaches for text summarization has been analyzed and discussed, where the recent text summarization model based on Deep learning techniques are highlighted. Further, the article provides the taxonomy to abstract the design pattern of Neural Networks and conducts a comprehensive of the existing text summarization model. Finally, the review article enlists the future direction of the researcher, which would motivate the enthusiastic and novel contributions in text summarizations.

Keywords

Text summarization graph neural networks deep learning natural language processing multi-document summarization

Nomenclature

MDS

Multi-Document Summarization

GNN

Graph Neural Networks

GAN

Graph Attention Network

GCN

Graph Convolutional Network

NLP

Natural Language Processing

NTM

Neural Topic Model

NVDM

Neural Variational Document Model

PFA

Poisson Factor Analysis

LDA

Latent Dirichlet Allocation

NMT

Neural Machine Translation

RNN

Recurrent Neural Network

GNN

Graph Neural Networks

ESA

Explicit Semantic Analysis

SRL

Semantic Role Labeling

ReNN

Recursive Neural Networks

RL-MMR

Maximal Margin Relevance Guided Reinforcement Learning Framework

GRU

Gated Recurrent Unit

DNN

Deep Neural Network

NLP

Natural Language Processing

1. Introduction

The Text summarization yields the summarized content from the source document and particularly, from the multiple input documents, the summarized contents are acquired from through the keywords. Here, the extraction of the summarized content containing significant information is reported as the most challenging task in the Natural Language Processing (NLP) as the summary yields the valuable information to the Internet users, which saves their time. For enabling the efficient information extraction, automatic text summarization technique is employed that collects the necessary information in a readable format [50]. The text summarization techniques could be abstractive or extractive form, where extractive summarization gathers the important sentences and words without any change in the objects, while abstractive summarization forms a summary as a outcome of paraphrasing [30,54]. Since large training set is required for training the abstractive model, extractive techniques are explored in this survey.

In the extractive approach, local semantic information may be obtained between paragraphs and sentences. Researchers attempt to include subject information during summarizing a context that captures the overall semantics of the input materials. The document understanding can be improved using topic models, like Neural Topic Model (NTM), Neural Variational Document Model (NVDM), Poisson Factor Analysis (PFA), and Latent Dirichlet Allocation (LDA) [11,21]. Neural Machine Translation (NMT) is the method of NLP that performs the translation function from one language to another language due to the highly stimulated data-driven methods [27]. Recent research focuses on generation-based methods that boost generalization performance. In certain studies, summaries are produced from the provided source code using Seq2Seq frameworks. For producing the summary, mechanism majority of Seq2Seq techniques treat the abstract syntax tree or source code that is parsed from the source code as a sequence and adheres to the encoder-decoder paradigm. These studies show that they solely use sequential models, which have trouble capturing the complex semantics of source code, such as control dependencies and data dependencies. Additionally, unlike retrieval-based systems, generation-based approaches often cannot benefit from comparable cases from the retrieval database [32].

The cross-sentence relation identification is the crucial step that needs to be utilized for the effective summarization of the text. The RNN detection of the cross-sentence relation is employed by several researchers. While using the long document and multiple documents for summarization, the detection of long-distance dependency is still a challenging task [4]. Thus, graph-based summarization is utilized for efficient summarization. The summarization based on the deep learning technique is developed by several researchers, but the complexity due to the irregular shape and size of the nodes in the graphical representation leads to the development of graph neural networks (GNN) [49,52].

Text summarization helps us to fetch the required content from the large document that consumes more time without skipping the required information. Numerous extractive text summarization techniques have been analyzed that will help the upcoming researchers to gain knowledge. The existing techniques fail to utilize large datasets, and inefficient summarization is due to the failure of detecting edge features. The graph model when combined with a neural network provides better accuracy, but it can be used for only one document.The text summerization techniques are categorized and visually presented in Fig. 1. So, in the upcoming research, GNN will be enhanced with optimization techniques to overcome the limitations.

Research Objective:

This study explores the complexities and excellence of various extractive text summarization techniques and the research gaps are identified.

Papers from various resources and various articles are pursued under relevant topics and read up.

The gathered text summarization techniques are categorized using machine learning models, GNN, and other methods are reviewed along with the achievements and the research gaps.

The analysis of the conventional text summarization techniques based on the performance metrics, the dataset used, and the categories help the readers to explore the widely used methods and metrics.

Research Questions:

RQ1: What are the types of text summarization techniques?

RQ2: Does the text summarization technique using GNN is effective compared to other methods?

RQ3: Why extractive text summarization technique is used for summarization?

Section 2 of the research details the related works based on the categories along with the achievements and the analysis of the conventional methods with the discussion included in Section 3. The research gaps are elaborated in Section 4 and Section 5 concludes the work.

2. Related work

The text summarization techniques developed by the conventional researchers are reviewed and the challenges faced by the techniques such as failure in considering some semantic units, to use the pre-trained model, inaccurate summarization, and failure in considering the synonyms of the words motivated to analyze the text summarization technique and to find the research gaps to develop a novel technique by fulfilling the research gaps. The existing methods regarding text summarization are reviewed in this section along with the achievements.

The summarization techniques are widely categorized into two different types such as abstractive and extractive.

Fig. 1.

Categorization of text summarization techniques.

2.1. Abstractive summarization

The abstractive summarization learns the significant concepts in the document and attempts to represent general Natural language processing. This abstractive model utilizes linguistic methods to obtain the main concept in the passage. After obtaining the knowledge about the phrases and text it sorts out the new terms and notions, which was then utilized to summarize the passage [39]. The complexities experienced in natural language processing make abstractive summarization a challenging process. Hence, the concentration is now in extractive summarization [18].

2.2. Extractive summarization

The extractive summarization determines the most significant passage, words, and lines from the original text and merges them as the summary. The linguistic or statistical features are utilized to select the passage.

Some of the methods involved in extractive summarization are described in the following section.

2.2.1. Term frequency-inverse document frequency

The word frequency is characterized as the number of sentences involved in the document that includes the specific terms. These sentences are then scored by the query similarity, in which the pre-eminent scoring sentences are obtained as the segments of the summary.

2.2.2. Cluster-based technique

Generally, the documents are written in such a way for addressing numerous topics successively in a structured format. It is essential to consider that summaries are needed to highlight different themes in the document. Document clustering remains an important tool to create meaningful summaries for the collection of documents that are generated from entirely different topics. The summary sentences are selected based on the similarity of the sentence to the theme of the cluster. The sentence locality in the document is also considered an essential factor for sentence selection. The higher weightage is given to the sentence closer to the commencement of the sentence. Another factor that enhances the sentence score is the similarity of the sentence with the first sentence in the document [39].

2.2.3. Machine learning technique

While considering the machine learning techniques the summarization process is customized as the classification issue, in which the sentences are categorized into summary and non-summary sentences based on the features.

The text summarization based on machine learning is detailed in this sub-section. The semi-supervised deep learning method for text summarization was used by [24] using the context vector of the document. In this, the contextual information was added while encoding the data by the humans, and then by proper training of the deep learning classifier, the extractive features were generated and the abstractive summarization was performed based on the extractive features. For the abstractive text summarization, the context or non-context information was utilized and the summarization was performed. A better performance was obtained based on the accuracy of the summarization task. The text summarization based on the deep learning technique was devised by [38] using the Explicit Semantic Analysis (ESA) and Semantic Role Labeling (SRL) approach. In this, the words were considered as the weight vectors and then the graph representation was performed for the summarization based on the sentence level. Here, the input document is segmented and pre-processed and then, the labeling and grouping were performed. Finally, based on the interpretations, the summary was generated and obtained elevated performance based on the semantic representation.

The machine learning-based text summarization was employed by [2] for the Bengali text using the extractive strategy. Here, the Bengali news article was utilized for the evaluation of the text summarization, in which the stop word removal and the tokenization were performed as the pre-processing step, then the vector encoding was devised for the summarization using a neural network based on probability. Fuzzy-based text summarization was utilized by [44] for news articles and text documents. Here, the fuzzy rules were utilized for the classification of the text, and then the most significant sentences were utilized for the summarization [25]. The hybrid generator-pointer network was utilized for the summarization of the text. The ensemble-based text summarization was utilized by [50] using the hybrid LSTM and Fuzzy logic, in which the most informative sentence was extracted using the fuzzy logic, and the weights of the LSTM were tuned using the Adam optimization and the attention mechanism. Multi-document summarization system is utilized with the deep neural network [56] for the summarization of documents in a coherent and fluent flow of documents covering the major information in the documents. The method consists of graphs for modeling sentence relationships within clusters namely document clusters, and the G-Flow graph system is utilized that create its graph representation by drawing on the discourse relationships between sentences. In this method, By calculating discourse relation indicators, the ADG creates edges between sentences. Compared to a baseline GRU model without any graphs and typical graph-based extractive summarizers, the model exhibits a definite benefit and produces competitive outcomes with other cutting-edge MDS systems. A recursive neural network employed with the ranking framework model [9] is utilized to prioritize them for multi-document summarizing, a hierarchical regression approach that assesses the salience information of all the non-terminal nodes in the parse tree and concerns the sentence ranking task. Recursive neural networks (ReNN) are used to model this procedure. R2N2 produces some advantages for multi-document summarization, it is possible to consider more supervisory expertise, which is good for learning additional ranking features, more precise sentence selection strategy is made possible by the consistent ranking scores that deliver from words to sentences. A computational approach for tracking faces in real time has been introduced [48]. In this approach, the detection of the face is identified using the shadow of a person, motion, and color. Authentication is satisfied by some requirements which are classified as face slant and face size are invariable, variation of lights invariable, and there is no need in taking other aspects into account while integrating a new subject. Face tracking is done in three stages image processing, image search, and pattern classification. The devised method minimizes storage space and training time for the high-dimensional dataset. The accuracy of the results is increased and classification is relatively simpler after creating a subset of features. The presented deep recurrent generative decoder (DRGD) equipped sequence-to-sequence oriented encoder-decoder model [28] to model and learn the latent structural information revealed in the target summaries of the training data. For the recurrent latent variables, the neural variation inference is used to address the unsolvable posterior inference. Using back-propagation an end-to-end standard is enhanced in all the parameters of the model. The method utilized to perform unsupervised extractive and abstractive text summarization [41] using sentence embeddings. The unsupervised Text Summarization model uses an RNN with LSTM to encode texts in a vector space. Similar to the skip-gram method for learning word embeddings and sentence embeddings are utilized. This work tries to learn how to combine word embeddings to produce sentence embeddings that satisfy the property that sentences are paraphrases of each other, and are positioned close to one another in the vector space. Automatic Text summarization is utilized to reduce the vocabulary in words [42] using the continuous vector approach. Using four ATS systems and the ROUGE evaluation methods, this technique is assessed using the MultiLing corpus. The utilized research demonstrates that a modern system performs better when the vocabulary is less. Four systems are used in the automatic text summarization method to evaluate the reduced vocabulary approach specifically, TextRank, Sasi, MMR, and base. Text Summarization based on the DUC 2001 dataset was devised by [36] using Deep Learning and Ridge Regression approach. Sentence selection and sentence ranking are the two key processes used in a summarization task. Using the Document Understanding Conference (DUC) dataset, summarization methods can be tested and evaluated. Each layer of nodes in an MLP is fully connected to the layer below it in a directed graph, which has several layers. The forward pass and the reverse pass are the two primary passes in MLP training.

2.2.4. Neural network approach for text summarization

This technique includes the training of the neural network to discern different categories of the sentences that are involved in the summary. The neural network thus comprehends the patterns that are involved in the sentences and it should be added to the summary. The neural network utilized three-layered feed-forward neural networks to generate the preferable summaries of the document.

2.2.5. Graph theoretic techniques

The graph theoretic illustration of the sentence provides the method for the recognition of the themes. After the general pre-processing phases, such as stop word removal and stemming the passages in the documents are illustrated as nodes in the uni-directional graph. There are nodes for each sentence and they are connected if the edges of the sentence allot the common phrase. For the query-specific summary, the sentences are selected from the relevant subgraphs, and for the generic summary, the sentences are selected from each of the subgraphs.

The GNN-text summarization is detailed in this section. The text summarization using the GNN was utilized by [22] for the extraction of a better summary from the text. The word graph was generated initially based on the semantic relation and then the sentence block was generated, which instead of the word sentence block was created. Then, the scores were assigned to each sentence block through the reading and post-reading approaches, in which the redundancy was eliminated through the tri-gram blocking strategy. The performance was analyzed based on human evaluation and experiments using the R-1 measure. The text summarization based on enhanced extraction using the Topic aware GNN was employed by [11] for compressing text data. In this technique, the inter-sentence relationship was evaluated to reduce the topical information rejection while summarizing the text. Besides, the document-level sentence selection was done through the neural topic system to detect latent topics. The text summarization employed offered a robust performance in terms of long documents and genres by considering R-1, R-2, and R-L.

The text summarization using the heterogeneous GNN was employed by [52] through several granularity levels. Here, the cross-sentence relation was elevated through the additional nodes that perform the intermediary among the sentences. As in method [22], this technique also utilizes tri-gram blocking to remove redundant information while choosing the sentence. GNN-based abstractive summarization of text was utilized by [30] using the integrated gated GNN with the Bi-directional long short-term memory (BiLSTM), by using this encoding process the key feature elimination was eliminated that enhancing the performance of the system. Here, for the entire sentence, the word co-occurrence was generated through edge building among the nodes. In addition, the seq2seq encoding makes the system encode all the necessary features. The system obtained enhanced performance nearer to the semantics and diversities.

The Graph convolutional neural network (GCN) based text summarization was developed by [29,31], and [51]. GCN-based extractive text summarization was employed by [31] using the attention matrix. Here, the graph network obtains the necessary information through the sentence relation, and the GCN was utilized for the generation of the adjacent matrix, in which the training was not performed for the reduction of the computational complexity. The event summarization was devised by [29] using the GCN based on the semantic graph. In this, the Twitter event data was considered for the summarization based on the semantic graph and score-based salience evaluation. The tweet representation was devised using the fine-tuned approach and then the relation graph-based clustering was performed finally, the salience evaluation was made for the summarization of the Twitter data. The best performance was evaluated through a large real dataset in terms of semantic terms. The text summarization using the Graph convolutional network (GCN), a type of GNN was used by [51] for solving the issue concerning the language-specific dependencies. The word and the sentence level text representation was employed through the BERT and FGOW, feature graph of words encoder, and then the summarization was devised through the GCN. Besides, the pre-trained BERT was utilized for the representation of the semantic latent at the document level. The document was divided into a sentence for summarization using the Sentence pierce tool. Knowledge Transfer for Out-of-Knowledge-Base Entities based on a GNN is utilized by [19] to detect absent knowledge in a database. The relation triplets that are provided to the system after training contain additional entities (OOKB entities), which causes this issue. Since the system was not trained on these entities, it does not have their embeddings and is hence unable to anticipate their relations. While it can be resolved by retraining the embeddings using the newly inserted relation triplets containing the OOKB entities, a less expensive alternative would be preferable. The propagation model and output model are used in the GNN to solve the OOKB entity problem. Graph information that is distributed between the nodes is managed by the propagation model and the output model defines the task-oriented objective function over node vectors. Retrieval-augmented generation is utilized for code summarization using a Hybrid GNN [32] to add the retrieved code and the related summary as additional data for training the model and to obtain the source files from the retrieval database which is most comparable to the present program. In addition to capturing global semantics among nodes, hybrid message-passing GNN is utilized based on both static and dynamic graphs. Four components of the presented Hybrid GNN include Retrieval-augmented Static Graph Construction, Attention-based Dynamic Graph Construction, HGNN, and Decoder. The structured neural summarization approach is utilized by [15] natural language processing system in the utilized summarization approach to convert the complex input into a simple representation. Framework adapted for the summarization of input data includes sequence to sequence. A mechanism called memories is obtained by combining the inputs of each token with the final state of RNN. The neural message-passing technique shares the information through a graph. Hierarchical Heterogeneous graph attention network applied for Syntax-Aware summarization is utilized by [49] to pick out important lines from the original abstract or text that might come up with new sentences and words. Two layers are utilized from the presented GNN approach the hierarchical graph pooling layer which obtains the tree information hierarchically and the dependency of constituent pair is detected using meta-data called the syntax-aware graph attention layer. Since the model can learn a global representation for each sentence, it can be simply modified for the extractive summarization job, allowing a graph-level classifier to be trained. The datasets PubMed, WikiHow, Reddit, XSum, New York Times, and CNN/DM are employed in this approach for the summarization of single documents. Fairness dictates that the same pre-trained model is used to replace the word embedding component in DISCOBERT and HAHSum. A Graph-to-Sequence Learning Framework for Summarizing Opinionated Texts is utilized by [53] to create a valid summary for a collection of writings with strong opinions about a certain subject. When directing the decoder, the integrated MMR process can take into account both non-redundant opinions and salient opinions with marginal relevance scores, and avoid listening to outdated, repetitious viewpoints produced at earlier time stages. Opinionated words are summarized using the two-stage graph-to-sequence framework where the estimation of prominent works performed in stage one and stage two generates the summary. The datasets used in the framework of summarization are Idebate, and RottenTomatoes. The most recent technique for abstractive multi-document summarization is used in the presented approach called HT (Hierarchical Transformer). It improves the Transformer’s encoder in which each layer accepts one paragraph as input, and inter-paragraph attention is used to describe the dependency of many paragraphs. In the utilized A Graph-to-Sequence Learning Framework, for salience regression kNNR-S2S makes use of kNNR, the graph encoder is taken out by GNR-S2S, and the graph encoder is taken out by GNR-S2S. GATSum: Graph-Based Topic-Aware Abstract based on Text Summarization is utilized by [21] provides instructions on how to create a summary and helps to collect global semantic features and resolves inadequate global semantic features in text summary. A book summary aims to make it easier for readers to rapidly understand the important points of a text. Two types of summarization approaches are described in the presented method that is abstract and extractive. Important text from the source text is mostly copied by extractive models, which then combine it with other text. Abstract models can provide summaries of high quality since they are more closely related to the substance of summarization. The output of a sequence is generated under the input sequence at each decoding step, and this process is continued until a termination sign is found. The output of the final result comes from the decoder side after each time step performs input for the following time step. The excellent performance of GTASum underlines the importance of introducing topic model components and graph attention network components to improve the generation effect. The successful implementation of GTASum emphasizes the significance of adding a topic model and graph attention network components to enhance the generating effect. Text Graph Transformer for Document Classification utilized by [57] to build efficient node representations by observing heterogeneity and structure in the text graph. The document node represents all of the vocabulary from the corpus and the word node represents the entire corpus of documents are the two types of nodes in the text graph. In the devised method the text graph’s document and word kinds can be captured using heterogeneity encoding. To encrypt word nodes and document nodes utilize the values 0 and 1. As k rises from 1 to an ideal value, learning performance steadily improves and then starts to decline as k raises more. Improved Code Summarization technique based on GNN is utilized by [27] provides a method that creates these summaries by employing a graph neural architecture that closely resembles the AST’s default structure. In code metaphor when summarizing, one language input is used for source code and another for summaries. Input for the same model’s encoder would be code, and input for the decoder would be summaries. The Code is typically treated as a sequence to be input into a recurrent neural network (RNN) or other similar structure optimized for sequential information in typical applications of NMT to code summarization. Recurrent layers work well for learning sequence information since they transmit information for each token in a sequence. At each time step of the input sequence, some of the popular recurrent layers, like the GRU and LSTM, can additionally return state information. Enhancing Extractive Text Summarization with Topic-Aware GNN utilized by [12] Using a graph-structured document representation, it is possible to efficiently record inter-sentence interactions. Additionally, to find latent topics, the devised model incorporates a combined neural topic model (NTM), which can deliver document-level characteristics for sentence selection. Sentence representations can be enhanced by topical data throughout the graph propagation process, which can be thought of as a type of document-level feature that aids in our model’s ability to extract the key information from a document’s entirety. The devised model not only outperforms existing approaches on datasets of scientific papers with much longer documents when compared to state-of-the-art extractive models for news datasets but also significantly outperforms them. This shows that the proposed model is more robust to different document lengths and genres. HETFORMER, a Heterogeneous transFORMER-based pre-trained model for long-text extractive summarization using multi-granularity sparse attention is utilized by [33] employing sparse attention, a new structured pre-trained technique that may capture the diverse structure of documents. The presented method shows that both single- and multi-document extractive summarization tasks can be completed with cutting-edge performance. This method is enhanced with three categories of nodes token, phrase, and entity to handle various semantic granularities.

2.2.6. Others

The text summarization other than the machine learning and the GNN is detailed in this section. The text summarization using the rule-based technique for the Bangla documents was utilized by [16]. Here, summarization was employed for the news content to read the news with minimal time on the Internet. The scoring of the sentence was devised based on the graph strategy by considering 12 different features. Besides, the redundant information from the news document was removed for efficient text summarization, in which the news document was segmented into sentences before the summarization. The summarization employed by the method was closely similar to human summarization. The automatic text summarization was employed by [20] using the extractive approach, in which the features such as the lexical and the semantics were extracted and then the clustering was performed to group similar words and then the keywords were extracted for the summarization process. The clustering was devised based on the Genetic algorithm and then based on the ranking the text summarization was performed. The global ranking-based criterion makes the system obtain an enhanced performance for various languages. The text summarization based on the text rank was devised by [35] using the improved graph-based approach. Here, the distance-based similarity among the sentences was evaluated for the extractive text summarization. The stop words were removed for the text data and then the word tokenization was employed for the graphical representation. Then, the sentences were ranked using the ranking algorithm and summarization was performed. The weight-based text summarization technique obtained better performance.

The text summarization based on the keyword extraction was used by [47] for the information retrieval from the search engine. Here, the keyword was extracted automatically for re-framing the query given by the user from the information fetched from the search engine. The summarized text was used as a link for content retrieval. The optimization of the search engine through text summarization was employed by [37] using the meta description, which was obtained through the query-based algorithm. In this approach, the keywords are extracted from the query, and summarization was employed. Here, the frequency of the keyword was counted for scoring the sentence to summarize the text. The text summarization for the Arabic document was employed by [43] based on the semantic and statistical features. Here, the preprocessing of the text was performed the removal of the redundant information and then the key features extracted the score was assigned and the devised system obtained better performance by considering the importance of the sentence. The text summarization based on the extractive strategy was devised by [1] for the Arabic text, using the general-purpose architecture. In this, the Tatweel and the Taskeel process was utilized for the removal of noise from the input data and then the sentence embedding and the clustering were performed for the summarization of the text, in which the Arabic BERT was utilized for embedding the sentence based on the pre-trained approach. The summary of the text was obtained through the K-Means clustering method. The domain knowledge-based text summarization was utilized by [3] for the Arabic text document. Here, the segmentation of the document into a sentence was employed and then the stop words were removed from the segmented sentence for the extraction of the informative features. Finally, the scoring was performed for the summarization of the text using the Genetic algorithm. The Arabic text summarization obtained a promising performance enhancement based on the F-Measure average evaluation. The Arabic text summarization based on the rank-based strategy was employed by [14] for the extractive summarization. In this, the score for the text document was generated using the modified page rank algorithm. The cosine-based similarity among the sentence was evaluated to obtain the more informative keywords by maintaining the relation among them. Initially, the stop word removal, tokenization, and normalization were performed in the pre-processing, and then features were extracted through the distance-based measure and finally the summarization based on the extractive features was devised using the modified algorithm. Better performance was achieved while elevating the iteration numbers. A novel application of continuous vector representations to the problem of multi-document summarization is utilized by [23] to capture the important information contained in large volumes of text and present it in a brief, representative, and consistent summary. Due to the improved benefits of continuous word vectors, the framework of state-of-the-art performance is improved by evaluating the devised function of the summarization method. In the presented method of summarization, the syntactic and semantic information about the word is captured using the continuous vector and reveals the word’s similarity by comparing the words based on their size. The field of automatic summarization will advance as the word vectors’ quality rises and discover better methods of creating and comparing the vectors. The paper presents a general framework for single document summarization with side information utilized by [40] in which the selected sentences from the input document are used as the output summary by the sentence extractor, which combines this document meaning representation with an attention mechanism over the side information. A CNN and RNN network in the framework of extractive summarization connects the hierarchical decoder-encoder architecture. The compositionality of the document is exploited by the model and it shows how a text is composed of sentences that make sense and that each sentence is composed of words that make sense. Because of its general characteristic side information plays an important role in the summarization of important sentences from the document and is extracted easily. The presented paper shows a demonstration of how to expand the training of gated GNN to such massive graphs and how to create graphs from source code utilized by [5] On two tasks – VARMISUSE, where the network learns to decide which variable should be used at a specific program location, and VARNAMING, where a network tries to guess a variable’s name given its usage – we test our devised model. The abstract syntax tree (AST) of the program, which is made up of syntax nodes that correspond to non-terminals in the grammar of the programming language and syntax tokens that correspond to terminals, forms the basis of a programmed graph. Due to the difficulty of efficient batching when there are several shape types, using GGNNs for sets of big, diverse graphs takes significant engineering work. The presented model is to familiarise natural language researchers with neural approaches utilized by [17] by reviewing neural network models from the standpoint of natural language processing research. The model covers computation graph abstraction for autonomous gradient calculation, feed-forward networks, convolutional networks, recurrent networks, and recursive networks in addition to input encoding for natural language problems. In terms of the interdependencies between the various components, the graph structure specifies the order of the calculation. The provided model uses an RNN technique to encode the sequence into a vector representation, and this vector representation is then used as an auxiliary input to another RNN that serves as a decoder. Strongly learning, neural networks can do non-Markovian modeling of trees and sequences as well as non-linear classification. The extractive strategy of summarization used in this method is utilized by [13] which combines a document and sentence clustering approach. The approach guarantees adequate coverage and prevents duplication. By initially grouping related documents into clusters, the clustering-based technique next groups sentences from each document cluster into sentence clusters. The final summary is composed of the highest-scoring sentences from the sentence clusters. If a word exists in a query or a phrase, its weight is assigned to the relevant vector, and its omission from the vector is indicated by a value of 0. The benefits of using the clustering mechanism in the presented method help reduce the redundancy in the techniques of extractive summarization. The presented method was utilized by [26] to improve the summarization quality in a sentence of Arabic text by combining the methods of semantic and linguistic feature selection. In the first step of the devised method, the RST is used to generate a primary summary. In the second phase, each sentence in the primary summary is given a score depending on how semantically related it is to the text’s subtitles and title. Discourse segmentation involves dividing a source text into non-overlapping EDUs, or elementary discourse units. Identification of rhetorical relations is used in the summarization of Arabic text to pinpoint the discourse connections between text spans. Presented a method for text summarizing that integrates sentence features and grammatical structure, which is utilized by [45]. The material used for text summarization is prepared by a series of preparation stages, including word stemming, stop words removal, and tokenization. In the hybrid-based text summarization method the feature extraction, which involves the discovery of significant features, is applied to the preprocessed material. Using dependency grammar, which breaks down sentences into words and connects these words with dependency relations, the syntactic structure of the retrieved sentences is then examined.

3. The recent models based on deep neural network

The deep neural network discerns the information from multiple representation levels and the abstract from the input data, which proves the applicability of the data in numerous research fields, such as NLP and computer vision. The deep learning algorithm performs to be an effective tool in summarization as it learns the distinctive features with the help of back-propagation to reduce the objective function. For the document summarization model, the DNN exhibits effective performance in processing large-scale documents and informative summaries as it consists of dynamic fitting abilities. This section of the review article describes the taxonomy that generates nine neural network design techniques.

3.1. Models related to recurrent neural networks (RNNs)

The RNN exhibited a better performance in constructing the sequential data that obtained the sequential relationship and syntactic information from the word sequence. The LSTM [58] is found to be a pre-eminent technique in the neural network. The RNN-based models were utilized in summarization tasks from 2015. Cao [9] presented the summarization model so-called Ranking framework in recursive neural network (R2N2), that enhanced the sentence-level features and extracted words. The RNN model transmitted the sentence scoring task into the regression process, that were utilized to estimate the relevance of the sentence and generate the parsing tree. The hierarchical RNN framework was utilized in [59] that exploits the subtopic information by extracting both the topic embedding and the sentence embedding. Soft clustering was utilized to incorporate the sentence and comment representation to attain subtopic representation.

The Gated Recurrent Unit (GRU) enabled encoder-decoder framework presented in [8] minimized the diversities in the opinion that reflected very advantageous views and created multi-review summaries. Maximal margin relevance guided reinforcement learning framework (RL-MMR) mode integrated the statistical measures and neural sequence learning. The soft attention employed in this technique enabled more exploration in the search space. The two-stage framework was represented in [6] that distills the relevant information from the document source. The first stage of the model influences the Bi-LSTM encoders that comprehended documents and word-level illustrations. The second stage of the model integrated the multi-score.

3.2. Models based on convolutional neural network

The convolutional neural network ensures a remarkable achievement in NLP, it utilized the kernel function that extricated the relevant information from the input. The pooling operations were utilized at intervals to return the complex features. The simple procedure utilized using CNN in document summarization was striding multiple filters with various window sizes over the input that attained the semantic representation. A hybrid CNN-based model was presented in [10] to obtain the latent feature representation.

The naïve CNN layer failed to capture the long-distance relation while processing the sequential data due to the limitation of fixed-size convolutional kernels. The CNN-depended model increased the receptive fields by the generation of the hierarchical structures that evaluated sequential data in parallel technique. However, it is a complex process to summarize the lengthy input articles.

3.3. GNN technique

The graph neural network was effectively employed in various summarization techniques that extricated the unique image features from the Euclidean space and struggled to process the non-Euclidean data. The NLP consisted of phrases and vocabularies with strong relationships and was illustrated with graphs [22]. The GAN and GCN have widely accepted techniques due to the simplicity of integration and the effectiveness of the neural network. These techniques constructed the relationship graph based on input graphs where the nodes can be sentences, edges, and words that obtains the similarity between them. The input documents were fed to Deep Neural Network (DNN) model that generated the embedding at various levels.

Yasunaga et al. [56] presented the GCN model for extractive summarization that obtained the interconnection between the sentences. This model constructs a sentence-depended graph that provided the pre-processed data to GCN to obtain the sentence-wise features. Each of the sentences was illustrated as nodes and the relation between the sentence pair was described as the edge. Further in the GCN model, the sentence relationship graph was generated by the cosine similarity graph and the customized discourse graph. The GCN model, known as SemSentSum was illustrated in [7], this method extended the universal embedding. This model generated the sentence semantic relationship graph by pre-trained with unrelated corpus. The sentence relation graph and the sentence embeddings were provided to GCN. These models failed to obtain the relationship between sentences, documents, and words. Hence, to obtain these relationships Wang et al. [52] generated the heterogeneous GAN model [34,46].

Table 1 depicts the current GNN-depended model utilized for MDS with the descriptions of edges, edge weights, nodes, and GNN methods.

Table 1
Analysis of the current GNN model

Author Model Nodes Edges Edge-weight GNN methods

Takuo Hamaguchi et al. [19] GNN based Knowledge Based Completion [19] Entities Triplets Not mentioned Not mentioned

Shangqing Liu et al. [32] Attention-based dynamic graph [32] Abstract syntax tree (AST) Multi-edged (reach, control, use/define) Global attention scores Hybrid GNN (HGNN)

Patrick Fernandes et al. [15] Structured Neural Summarization Sentence Node Or Super Node Parse tree, Use the edge Logistic sigmoid function Gated Graph Neural Networks

Penghui Wei et al. [53] A two-stage Graph-to-Sequence Learning Framework Words Text structure TF-IDF GAN and GCN

Zixing Song and Irwin King [49] SynapSum (as shorthand for Syntax-aware Heterogeneous Graph Attention Network for Summarization) Text units Word–sentence Not mentioned Gated GNN

Ming Jiang et al. [21] GATSum: Graph-Based Topic-Aware Abstract Text Summarization Topic nodes and sentence node Sentence–topic Sigmoid function GAN

Haopeng Zhang and Jiawei Zhang [57] Text Graph Transformer (TG-Transformer) Word node Word–word TF-IDF Heterogeneous GNN

Peng Cui et al. [12] Topic-Aware Graph Neural Networks Sentence and topic nodes Sentence–topic Attention score between each topic and the sentence Topic-Aware GNN

Ye Liu et al. [33] HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization Token, sentence, and entity Token-to-token (t2t), token to-sentence (t2s), sentence-to-token (s2t), sentence to-sentence (s2s) and entity-to-entity (e2e) Not mentioned Not mentioned

Alexander LeClair and Sakib Haque [27] Improved code summarization through GNN AST AST nodes Visualizing model weights ConvGNNs

Michihiro Yasunaga [56] Graph-based Neural MDS Sentence Sentence–sentence Personalized Discourse Graph GCN

Danqing Wang [52] HeterDoc-SumGraph Document, sentence, and word Word–document and word–sentence TF-IDF GAN

Michihiro Yasunaga [55] ScisummNet Sentence Sentence–sentence Cosine Similarity Graph GCN

Diego Antognini and Boi Faltings [7] SemSentSum Sentence Sentence–sentence Edge Removal and Cosine Similarity Graph GCN

Author	Model	Nodes	Edges	Edge-weight	GNN methods
Takuo Hamaguchi et al. [19]	GNN based Knowledge Based Completion [19]	Entities	Triplets	Not mentioned	Not mentioned
Shangqing Liu et al. [32]	Attention-based dynamic graph [32]	Abstract syntax tree (AST)	Multi-edged (reach, control, use/define)	Global attention scores	Hybrid GNN (HGNN)
Patrick Fernandes et al. [15]	Structured Neural Summarization	Sentence Node Or Super Node	Parse tree, Use the edge	Logistic sigmoid function	Gated Graph Neural Networks
Penghui Wei et al. [53]	A two-stage Graph-to-Sequence Learning Framework	Words	Text structure	TF-IDF	GAN and GCN
Zixing Song and Irwin King [49]	SynapSum (as shorthand for Syntax-aware Heterogeneous Graph Attention Network for Summarization)	Text units	Word–sentence	Not mentioned	Gated GNN
Ming Jiang et al. [21]	GATSum: Graph-Based Topic-Aware Abstract Text Summarization	Topic nodes and sentence node	Sentence–topic	Sigmoid function	GAN
Haopeng Zhang and Jiawei Zhang [57]	Text Graph Transformer (TG-Transformer)	Word node	Word–word	TF-IDF	Heterogeneous GNN
Peng Cui et al. [12]	Topic-Aware Graph Neural Networks	Sentence and topic nodes	Sentence–topic	Attention score between each topic and the sentence	Topic-Aware GNN
Ye Liu et al. [33]	HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization	Token, sentence, and entity	Token-to-token (t2t), token to-sentence (t2s), sentence-to-token (s2t), sentence to-sentence (s2s) and entity-to-entity (e2e)	Not mentioned	Not mentioned
Alexander LeClair and Sakib Haque [27]	Improved code summarization through GNN	AST	AST nodes	Visualizing model weights	ConvGNNs
Michihiro Yasunaga [56]	Graph-based Neural MDS	Sentence	Sentence–sentence	Personalized Discourse Graph	GCN
Danqing Wang [52]	HeterDoc-SumGraph	Document, sentence, and word	Word–document and word–sentence	TF-IDF	GAN
Michihiro Yasunaga [55]	ScisummNet	Sentence	Sentence–sentence	Cosine Similarity Graph	GCN
Diego Antognini and Boi Faltings [7]	SemSentSum	Sentence	Sentence–sentence	Edge Removal and Cosine Similarity Graph	GCN

4. Analysis of the text summarization techniques

The analysis of the text summarization techniques based on the performance metrics, dataset utilized, and categorization is evaluated in this section.

4.1. Analysis based on GNN models in graph-summarization model

The analysis based on the GCN model is shown in Fig. 2. The GNN-based models utilize different graph methods such as GCN, GAN, hybrid model, gated GNN model, heterogeneous GNN model, topic-aware GNN model, and convGNNs model. The GCN model is generally utilized in most of the review articles such as [7,53,56], and [55].

Fig. 2.

Analysis in terms of GNN techniques.

4.2. Analysis in terms of edge-weight technique in the GNN method

The analysis in terms of the edge-weight technique is illustrated in Fig. 3, the weight of the edge was estimated on different methods such as TF-IDF, global attention score, sigmoid function, and cosine similarity graph. The TF-IDF is found to be a widely utilized technique to estimate the edge weight due to simplicity and accuracy.

Fig. 3.

Analysis in terms of edge weight.

4.3. Analysis based on performance metrics

The performance metrics such as Rouge-1, Rouge-2, Rouge-L, Rouge-F, Rouge SL, accuracy, precision, recall, and F-Measure are utilized by the conventional techniques for the evaluation of the performance, which is depicted in Fig. 4. In this proposed study, 50 papers are chosen for analysis. In this study, 29% of papers are used based on Rouge-1 which covers with high percentage whereas Rouge-2 is used by 27% of papers, and Rouge-L is utilized by 17% of the research papers on performance metrics. Each metric of Rouge-F, Rouge SL, and accuracy, are utilized by 2% of performance metrics in the existing methods, and precision, recall, and F-Measure of all metrics are evaluated as 7% in the conventional techniques.

Fig. 4.

Analysis based on performance metrics.

From the figure, it is demonstrated that most of the existing articles utilize the rouge score as the important performance metric to analyze the effectiveness of the model. Hence, this review article evaluates the rouge score attained by the various existing methods to demonstrate the efficacy of the existing models. Tables 1, 2, and 3 depict the rouge analysis of the GCN-based model, machine learning-based model, and other text summarization models respectively.

Table 2

Rouge analysis of GCN-based text summarization techniques

Reference	Dataset	Rouge-1	Rouge-2	Rouge-L
[53]	Rotten tomato	25.56	7.87	19.65
[12]	arXiv	46.05	19.97	33.61
[49]	CNN/DM	41.52	18.82	38.29
[21]	CNN/DM	51.4	25.0	50.0
[33]	Multi-news	46.21	17.49	42.43
[21]	CNN/daily mail	42.95	19.76	39.84
[56]	DUC 2004	38.23	9.48	-
[52]	CNN/daily mail	42.95	19.76	39.23
[55]	DUC 2004	39.12	9.59	-

Table 3

Rouge analysis of machine learning-based text summarization techniques

Reference	Dataset	Rouge-1	Rouge-2	Rouge-L
[9]	DUC 2004	38.78	9.86	-
[28]	DUC 2004	36.99	24.15	34.21
[42]	MS WORD	41.52	18.82	38.29
[36]	-	26.50	5.30	-

Table 4

Rouge analysis of other text summarization techniques

Reference	Dataset	Rouge-1	Rouge-2	Rouge-L
[23]	Opinosis	20.57	2.87	6.73
[40]	DUC 2004	39.7	14.7	7.9

From analysis, it is demonstrated that Graph-text summarization models lead the other models while considered to rouge score. The Rouge analysis of other text summerization techniques are tabulated in Table 4.

4.4. Analysis based on dataset utilized

The datasets such as CNN/DailyMail, New York Times (NYT), Essex Arabic Summaries Corpus (EASC), DUC, and several others are utilized by the conventional methods, are depicted in Fig. 3. In this, the CNN/DailyMail dataset is widely used by 33% of research papers, followed by the NYT, DUC, and EASC utilized by 14% for each technique in the existing papers, the datasets such as LCSTS, Gigaword, Newsroom, Community, and KALIMAT are utilized by 5% for each paper. The analysis based on the utilized dataset is visually displayed in Fig. 5.

Fig. 5.

Analysis based on dataset utilized.

4.5. Analysis based on the categorization of techniques

The analysis based on the categorization of the techniques is illustrated in Fig. 6. In this, out of the 25 papers chosen 20 papers represents the GNN, 12 papers represent machine learning techniques and 18 papers represent the other methods of text summarization.

Fig. 6.

Analysis based on techniques used.

5. Research gaps and future scope

The challenges faced by the conventional text summarization technique are a failure in considering the synonyms identifier, failure to utilize a large dataset, inefficient summarization, and failure in using the optimization to enhance the performance of the system. Besides, many techniques failed to consider the edge features that may limit the summarization accuracy. Also, the traditional domain-based and rules-based techniques cannot be utilized for the other text summarization due to the specific nature of the technique. The coherence and the redundancy are considered significant issues experienced in the summarization model as they affect the quality of the summarized content. Hence, there must be considerations to capture cross-document dependency or relationship to reduce coherence and redundancy. Obtaining the cross-document relation attains significant consideration in the past decades in text summarization as it reduces redundancy and improves coherence. Graphical-model is proven to be one of the widely explored techniques to obtain the inter-sentence relationship between sentences. The Graph model integrated with deep learning models was effective in obtaining syntactic-rich representation and also to high-quality summaries. The existing works represent the effectiveness of the graph learning model for text summarization techniques that provides to explore more detail about the Graph-based methods in natural language processing.

While considering the GNN-based text summarization, performance enhancement can be obtained by training the network. Besides, the consideration of the edge features enhances the summarization accuracy and the graph-special class attracts the researchers to utilize the GNN-based technique. Still, the limitations such as failure to use the optimization technique, which enhances the summarization efficiency, and failure in considering the significant semantic units need to be rectified to enhance the performance.

Further, the graph model is at the initial level and it is useful for the single document process. Yet, advanced strategies like transfer learning and split learning enable us to utilize the graph models for multi-document text summarization.

6. Conclusion

The multi-document summarization is one of the emerging research topics as it generates precise and concise summaries of huge-volume of text without losing its significant terms. This research provides a review of the conventional text summarization techniques based on machine learning, GNN, and other methods apart from machine learning and GMM along with achievements and challenges presented. The challenges faced by the existing system make the researchers develop a novel technique for text summarization by fulfilling the research gaps. The analysis based on the dataset, performance metrics, and the methods utilized helps to evaluate the performance based on the widely used metrics and the datasets. Thus, the analysis of the conventional text summarization techniques along with the research gaps and the evaluation methods helps to develop a novel framework for enhancing the efficiency of the method by fulfilling the research gaps. This review provides brief insights into the methods and challenges employed in the extractive summarization models and it demonstrates that the graph-based model provides suitable output for extractive summarization. Further, it elucidates the research gaps and future scope for extractive summarization. However, this review leaves to analyze the abstractive summarization techniques that generate the semantic representation of the given document and select words from the general vocabulary that fits with semantics. Text summarization using GNN can be widely in real-time applications such as media monitoring, helpdesk and customer care, and Books and literature. This work mainly concentrates on extracting summarization and, in the future, the article related to extractive summarization will be reviewed to provide the challenges and techniques employed in text summarization, which enables the researchers to find an effective solution for the text summarization.

References

A.M.

Abu Nada,

Alajrami,

A.A.

Al-Saqqa and

S.S.

Abu-Naser, Arabic text summarization using AraBERT model using extractive text summarization approach, International Journal of Academic Information Systems Research (IJAISR) 4(8) (2020), 6–9.

Al Munzir,

M.L.

Rahman,

Abujar, Ohidujjaman and

S.A.

Hossain, Text analysis for Bengali text summarization using deep learning, in: 2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT), 2019, pp. 1–6.

Q.A.

Al-Radaideh and

D.Q.

Bataineh, A hybrid approach for Arabic text summarization using domain knowledge and genetic algorithms, Cognitive Computation (2018).

Allahyari,

Pouriyeh,

Assefi,

Safaei,

E.D.

Trippe,

J.B.

Gutierrez and

Kochut, Text summarization techniques: A brief survey, 2017, arXiv preprint arXiv:1707.02268.

Allamanis,

Brockschmidt and

Khademi, Learning to represent programs with graphs, 2017, arXiv preprint arXiv:1711.00740.

R.K.

Amplayo and

Lapata, Informative and controllable opinion summarization, 2019, arXiv preprint arXiv:1909.02322.

Antognini and

Faltings, Learning to create sentence semantic relation graphs for multi-document summarization, 2019, arXiv preprint arXiv:1909.12231.

Bražinskas,

Lapata and

Titov, Unsupervised opinion summarization as copycat-review generation, 2019, arXiv preprint arXiv:1911.02247.

Cao,

Wei,

Dong,

Li and

Zhou, Ranking with recursive neural networks and its application to multi-document summarization, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 29, 2015.

10.

Cao,

Wei,

Li,

Zhou and

Wang, Learning summary prior representation for extractive summarization, in: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Vol. 2, 2015, pp. 829–833.

11.

Cui,

Hu and

Liu, Enhancing extractive text summarization with topic-aware graph neural networks, in: Conference: Proceedings of the 28th International Conference on Computational Linguistics, 2021.

12.

Cui,

Hu and

Liu, Enhancing extractive text summarization with topic-aware graph neural networks, 2020, arXiv preprint arXiv:2010.06253.

13.

A.R.

Deshpande and

L.M.R.J.

Lobo, Text summarization using clustering technique, International Journal of Engineering Trends and Technology 4(8) (2013), 3348–3351.

14.

Elbarougy,

Behery and

El Khati, Extractive Arabic text summarization using modified PageRank algorithm, Egyptian Informatics Journal 21 (2020), 73–81. doi:10.1016/j.eij.2019.11.001.

15.

Fernandes,

Allamanis and

Brockschmidt, Structured neural summarization, 2018, arXiv preprint arXiv:1811.01824.

16.

P.P.

Ghosh,

Shahariar and

M.A.H.

Khan, A rule based extractive text summarization technique for Bangla news documents, I.J. Modern Education and Computer Science 12 (2018), 44–53.

17.

Goldberg, A primer on neural network models for natural language processing, Journal of Artificial Intelligence Research 57 (2016), 345–420. doi:10.1613/jair.4992.

18.

Gupta and

S.K.

Gupta, Abstractive summarization: An overview of the state of the art, Expert Systems with Applications 121 (2019), 49–65. doi:10.1016/j.eswa.2018.12.011.

19.

Hamaguchi,

Oiwa,

Shimbo and

Matsumoto, Knowledge transfer for out-of-knowledge-base entities: A graph neural network approach, 2017, arXiv preprint arXiv:1706.05674.

20.

Á.

Hernández-Castañeda,

R.A.

García-Hernández,

Ledeneva and

C.E.

Millán-Hernández, Extractive automatic text summarization based on lexical-semantic keywords, IEEE Access 8 (2020).

21.

Jiang,

Zou,

Xu and

Zhang, GATSum: Graph-based topic-aware abstract text summarization, Information Technology and Control 51(2) (2022), 345–355. doi:10.5755/j01.itc.51.2.30796.

22.

Jing,

You,

Yang,

Fan and

Tong, Multiplex graph neural network for extractive text summarization, 2021, arXiv preprint. doi:10.18653/v1/2021.emnlp-main.11.

23.

Kågebäck,

Mogren,

Tahmasebi and

Dubhashi, Extractive summarization using continuous vector space models, in: Proceedings of the 2nd Workshop on Continuous Vector Space Models and Their Compositionality (CVSC), 2014, pp. 31–39.

24.

Khatri,

Singh and

Parikh, Abstractive and extractive text summarization using document context vector and recurrent neural networks, 2018, arXiv preprint.

25.

Kumar and

Sharma, Systematic literature review of fuzzy logic based text summarization, Iranian Journal of Fuzzy Systems 16(5) (2019), 45–59.

26.

Lagrini,

Redjimi and

Azizi, Automatic Arabic text summarization approaches, International Journal of Computer Applications 164(5) (2017), 31–37. doi:10.5120/ijca2017913628.

27.

LeClair,

Haque,

Wu and

McMillan, Improved code summarization via a graph neural network, in: Proceedings of the 28th International Conference on Program Comprehension, 2020, pp. 184–195. doi:10.1145/3387904.3389268.

28.

Li,

Lam,

Bing and

Wang, Deep recurrent generative decoder for abstractive text summarization, 2017, arXiv preprint arXiv:1708.00625.

29.

Li and

Zhang, Twitter event summarization by exploiting semantic terms and graph network, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, 2021.

30.

Liang,

Du,

Shao and

Ji, Gated graph neural attention networks for abstractive summarization, Neurocomputing (2020).

31.

Y.-C.

Lin and

Ma, On automatic text extractive summarization based on graph and pre-trained language model attention, 2021, arXiv preprint.

32.

Liu,

Chen,

Xie,

Siow and

Liu, Retrieval-augmented generation for code summarization via hybrid gnn, 2020, arXiv preprint arXiv:2006.05405.

33.

Liu,

J.G.

Zhang,

Wan,

Xia,

He and

P.S.

Yu, HETFORMER: Heterogeneous transformer with sparse attention for long-text extractive summarization, 2021, arXiv preprint arXiv:2110.06388.

34.

Ma,

W.E.

Zhang,

Guo,

Wang and

Q.Z.

Sheng, Multi-document summarization via deep learning techniques: A survey, ACM Computing Surveys (CSUR) (2020).

35.

Mallick,

A.K.

Das,

Dutta,

A.K.

Das and

Sarkar, Graph-based text summarization using modified TextRank, in: Advances in Intelligent Systems and Computing, AISC, Vol. 758, 2018.

36.

K.B.

Mani, Text summarization using deep learning and ridge regression, 2016, arXiv preprint arXiv:1612.08333.

37.

Matošević, Text summarization techniques for meta description generation in process of search engine optimization, in: Artificial Intelligence and Algorithms in Intelligent Systems, 2019, pp. 165–173.

38.

Mohamed and

Oussalah, SRL-ESA-TextSum: A text summarization approach based on semantic role labeling and explicit semantic analysis, Information Processing & Management 56(4) (2019), 1356–1372. doi:10.1016/j.ipm.2019.04.003.

39.

Moratanch and

Chitrakala, A survey on extractive text summarization, in: Proceedings of 2017 International Conference on Computer, Communication and Signal Processing (ICCCSP), 2017, pp. 1–6.

40.

Narayan,

Papasarantopoulos,

S.B.

Cohen and

Lapata, Neural extractive summarization with side information, 2017, arXiv preprint arXiv:1704.04530.

41.

Padmakumar and

Saran, Unsupervised text summarization using sentence embeddings, Technical report, University of Texas at Austin, 2016.

42.

E.L.

Pontes,

Huet,

J.M.

Torres-Moreno and

A.C.

Linhares, Automatic text summarization with a reduced vocabulary using continuous space vectors, in: International Conference on Applications of Natural Language to Information Systems, Springer, Cham, 2016, pp. 440–446.

43.

Qaroush,

Abu Farha,

Ghanem,

Washaha and

Maali, An efficient single document Arabic text summarization using a combination of statistical and semantic features, Journal of King Saud University – Computer and Information Sciences 33 (2021), 677–692. doi:10.1016/j.jksuci.2019.03.010.

44.

Sahba,

Ebadi,

Jamshidi and

Rad, Automatic text summarization using customizable fuzzy features and attention on the context and vocabulary, in: World Automation Congress (WAC), Stevenson, WA, USA, 2018.

45.

D.Y.

Sakhare and

Kumar, Syntactic and sentence feature based hybrid approach for text summarization, International Information Technology and Computer Science 2014(3) (2014), 38–46. doi:10.5815/ijitcs.2014.03.05.

46.

C.S.

Saranyamol and

Sindhu, A survey on automatic text summarization, International Journal of Computer Science and Information Technologies 5(6) (2014), 7889–7893.

47.

Sekaran,

Chandana,

J.R.V.

Jeny,

M.N.

Meqdad and

Kadry, Design of optimal search engine using text summarization through artificial intelligence techniques, TELKOMNIKA Telecommunication, Computing, Electronics and Control 18(3) (2020), 1268–1274.

48.

N.R.

Shetty,

N.H.

Prasad and

Nalini, Emerging Research in Computing, Information, Communication and Applications, Springer, Singapore, 2016.

49.

Song and

King, Hierarchical heterogeneous graph attention network for syntax-aware summarization, 2022.

50.

Tomer and

Kumar, Improving text summarization using ensembled approach based on fuzzy with LSTM, Arabian Journal for Science and Engineering (2020).

51.

Vo, SE4ExSum: An integrated semantic-aware neural approach with graph convolutional network for extractive text summarization, ACM Transactions on Asian and Low-Resource Language Information Processing 20(6) (2021).

52.

Wang,

Liu,

Zheng,

Qiu and

Huang, Heterogeneous graph neural networks for extractive document summarization, 2020, arXiv preprint arXiv:2004.12393.

53.

Wei,

Zhao and

Mao, A graph-to-sequence learning framework for summarizing opinionated texts, IEEE/ACM Transactions on Audio, Speech, and Language Processing 29 (2021), 1650–1660.

54.

Xu,

Gan,

Cheng and

Liu, Discourse-aware neural extractive text summarization, in: Conference: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 10.18653/v1/2020.acl-main.451, 2020.

55.

Yasunaga,

Kasai,

Zhang,

A.R.

Fabbri,

Li,

Friedman and

D.R.

Radev, ScisummNet: A large annotated corpus and content-impact models for scientific paper summarization with citation networks, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 2019, pp. 7386–7393.

56.

Yasunaga,

Zhang,

Meelu,

Pareek,

Srinivasan and

Radev, Graph-based neural multi-document summarization, 2017, arXiv preprint arXiv:1706.06681.

57.

Zhang and

Zhang, Text graph transformer for document classification, in: Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.

58.

Zhang,

Liu and

Song, Sentence-state LSTM for text representation, 2018, arXiv preprint.

59.

Zheng,

Sun,

Li and

Muthuswamy, Subtopic-driven multi-document summarization, in: Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 3153–3162.