Abstract
With the exponential growth of the internet, a lot of online news reports are produced on the web every day. The news stream flows so rapidly that no one has the time to look at each and every item of information. In this situation, a person would naturally prefer to read updated information at certain time intervals. Document updating technique is very helpful for individuals to acquire new information or knowledge by eliminating out-of-date or redundant information. Existing summarization systems involve identifying the most relevant sentences from the text and putting them together to create a concise initial summary. In the process of identifying the important sentences, features influencing the relevance of sentences are determined. Based on these features the salience of the sentence is calculated and an initial summary is generated from highly important sentences at different compression rates. These types of initial summaries work on a batch of documents and do not consider the documents that may arrive at later time, so that corresponding summaries need to get updated. The update summarization system addresses this issue by taking into account the documents read by the user in the past and seeks to present only fresh or different information. The first step is to create an initial summary based on basic and additional features. The next step is to create an update summary based on the basic, additional and update features. In this paper, two approaches are proposed for generating initial and update summary from multiple documents about given news. The first approach performs semantic analysis by modifying the vector space model with dependency parse relations and applying latent semantic analysis on it to create a summary. The second approach applies sentence annotation based on aspects, prepositions and named entities to generate summary. Experimental results show that the proposed approaches generate better initial and update summaries compared with the existing systems.
1. Introduction
Data that evolves temporally is composed of many time slices, in which every time slice contains many documents, and every document consists of many sentences. Update summarization aims to incrementally capture this evolving information that changes over time. It seeks to summarize an evolutionary document collection at the current time on the supposition that the users have already read some related documents previously. Update summarization provides an effective means to acquire and manage information, which can be used in information retrieval, intelligence analysis and trend prediction. It is the natural extension of the traditional static summarization process. A good update summarization should include as many responses relevant to the given topic as possible, keeping information as important and novel as possible.
The primary task of update summarization is to provide an initial summary that could be a compressed summary of a collection of news documents about a specific topic for news domain. The secondary task is to generate an update summary for news documents on the same topic, with the assumption that the reader had already read the first set of documents. These tasks rely on a scenario, in which a user has a standing question that seeks a response from an information retrieval or summarization system at two different times. Hence this research work is directed towards proposing two different multidocument update summarization approaches, namely Update Summary Generation system (USG) and Enhanced Update Summary Generation systems (EUSG).
There are three issues to be considered for multidocument update summarization process. First and foremost, simple word-matching measures are not able to completely capture the content similarity between the sentences as news articles consist of different words to describe the same events. The traditional vector space model assumes a bag-of-words model of the document where the words within a document are independent of each other. Therefore, an effort has to be made to find the dependency between the words that are used to select semantically important sentences from the document collection. The second issue is on how to select more relevant keywords to create a well-organized fluent summary from multiple documents. The third issue is on how to select novel sentences by removing redundancy between the document sentences.
The proposed USG system addresses these issues by doing semantic analysis of news documents. Initially, the documents are mapped into vector space model and the similarity matrix is constructed by term synonym frequency–inverse sentence frequency (TSF-ISF), which utilizes WordNet. Then the similarity matrix is modified into semantic similarity matrix through identification of a grammatical relationship between the words. In order to extract one of the most fundamental semantic units from natural language text, Stanford Tagger and Parser is used. The context is intuitively extracted from typed dependency structures, which basically depict dependency relations. The dependency relations imply deep, fine-grained, labelled dependencies that encode long-distance relations and passive information. The semantic similarity matrix is reduced to have the effect of preserving the most important semantic information in the text while reducing noise and other undesirable artifacts. In text mining, a common recommendation is to use latent semantic analysis (LSA) rather than principal component analysis (PCA) because in PCA the covariance matrix is decomposed while in LSA the term document matrix is decomposed. Moreover, the PCA matrix will almost certainly be denser than that of the LSA. In the proposed USG system, LSA is utilized to reduce the semantic similarity matrix and to obtain relevant keywords. Additionally, a feature specific sentence ranking strategy is applied to all of the sentences and the high-ranking sentences are selected as candidate sentences for initial summary. An update summary is then generated by selecting novel sentences from the given document set, using a novel and topic relevance (NTRM) measure.
In order to produce a responsive, semantically enhanced summary, an oriented structural analysis is needed. To address this issue, a sentence-level semantic analysis document summarization approach called a EUSG system is developed. This approach performs deeper linguistic analysis to select the important concepts. Further individual sentences are mapped into concepts and an individual word score is calculated based on a term synonym concept frequency–inverse sentence frequency (TSCF-ISF) measure. It aims to produce a semantic summary using a list of important aspects such as what, when, how, etc. Each aspect determines the important information that must be retrieved for generating the summary. The initial summary generated is guided by pre-defined aspects, prepositions and named entities to enhance the quality and readability of the resulting summary. Further, an update summary is generated by selecting novel sentences from the given document set, using a novel sentence similarity measure (NSSM).
This paper is organized as follows. Section 2 discusses related work on multidocument update summarization. Section 3 presents an overview of the proposed USG approach. Section 4 discusses the EUSG approach for summary generation. The experimental and evaluation measures are discussed in Section 5. Finally Section 6 sums up the conclusions of this paper.
2. Related work
Genest et al. [1] developed a news symbolic summarizer (NESS) system that relies on the syntactical parser to extract linguistic knowledge from source documents. It selects sentences based on linguistic metrics. It also measures the similarity between candidate sentences and the previous articles already read by the user. It utilizes term frequency–inverse document frequency to measure the relevance of the sentence to the topic. This system could not select the most relevant sentences from documents owing to the absence of semantic analysis. The NESS system is considered as the baseline system for our evaluation task. The system proposed in Kireyev [2] and Ozsoy et al. [3] uses LSA to provide a simple and robust way of generating extractive summaries for update summarization task.
Steinberger and Jezek [4] developed a summarizer that is based on iterative residual rescaling (IRR), which that creates the latent semantic space for a set of documents under consideration. IRR utilizes generalized singular value decomposition (SVD) and enables the control of the influence of major and minor topics in the latent space.
The approach specified in Long et al. [5] describes a method for multidocument update summarization. The best summary is defined to be the one that has the minimum information distance to the entire document set. The best update summary has the minimum conditional information distance to a document cluster given the fact that a prior document cluster has already been read.
The approach specified in Huang and He [6] proposes an update summarization framework based on topic correlation analysis. The topics were first extracted from the two document sets provided in the task of update summarization by means of a latent Dirichlet allocation (LDA) topic model. Then the correlation between the new topics and the old topics was identified, based on which four categories of topic evolution patterns were defined to capture the topic shift between the two document collections. A new sentence-ranking algorithm, CorrRank, was developed, which fully incorporates the topic evolution in the process of sentence ranking and sentence selection in update summarization.
Peng et al. [7] developed a new method based on shallow parsing with rules. The rules were generated according to the syntactic features of English language, such as the tense of verbs, the usage of modal verbs and so on. The latest novel information from English news text was extracted correctly, to meet the needs of users for accessing updated information on developing events quickly and effectively.
Shiyan et al. [8] proposed a multidocument summarization, where aspects could be taken as specified queries in summarization. They also proposed a novel ranking algorithm, Decayed DivRank for guided summarization task. This addresses relevance, importance, diversity and novelty simultaneously through a decayed vertex-reinforced random walk process in sentence ranking.
Pan et al. [9] developed a new concept-based multidocument summarization system that employs discourse parsing, information extraction and information integration. The summarization process includes parsing dissertation abstracts into five standard sections, extracting research concepts and their relationships, the research methods used and the contextual relations from specific sections of the text, integrating similar concepts and relationships across different abstracts and organizing different kinds of information using a variable-based framework, and presenting them in an interactive web-based interface.
3. Update summary generation system
In a USG system, the goal is to propose an approach for modifying the vector space model with dependency parse relations. Then LSA is applied on it to create the initial summary. Finally, an update summary is generated by applying the novelty measure. In the proposed USG system, initially, the individual word weight is calculated using the TSF-ISF weight of words by incorporating the dependency between each pair of words. Also to protect the vital semantic information in the text, LSA is performed on it and to select the relevant sentences, basic features, additional features and update specific features are used.
The calculation of TSF involves synonym extraction, comparison and term frequency calculation for each term. Synonym extraction can be done with the help of WordNet, which is a lexical database consisting of synonym for terms. The usage of TSF improves the quality of the term. ISF is used to measure general importance of the term in the sentence. Word weight is calculated by TSF * ISF. In this the TSF of each word is calculated as defined in equation (1).
where wi ∈ {w U syn(w)}. In TSF calculation, the term synonym is incorporated into the term frequency (TF) of each word and its synonym is multiplied by α. According to Toutanova et al. [10], α = 1 for the word and α = 0.5 for the synonym of the word. The TF of each word is calculated as defined in equation (2):
In equation (2), the numerator denotes the number of occurrences of the ith word in document dk and the denominator denotes the number of occurrences of all words in document collection D. The ISF of each word is calculated as defined in equation (3):
In equation (3), xi is the number of sentences that contain the ith word. After calculating the TSF-ISF, a matrix is constructed which has N rows and M columns.
3.1. Semantic analysis
The similarity matrix constructed using TSF-ISF is unable to define the context between the words. In order to define the context between the words, the similarity matrix is modified by identifying the verb–object pair. Merely acquiring the verbs from sentences does not provide semantics to the similarity matrix. There is a need to find out whether these words are really important or not. For this purpose, the objects corresponding to the verbs have to be identified. The objects are nouns or adjectives for the verbs. The weight for verbs in the similarity matrix is updated based on the weight of objects affected by it. If there is more than one object for the same verb, then the maximum weight amongst all the objects corresponding to the given verb is obtained and this weight is added to the weight of the corresponding verb in the similarity matrix, which yields the semantic similarity matrix. The vector space model has been adapted to incorporate semantic information by increasing the TSF-ISF value if a semantic relation holds between the words. Semantic analysis is performed using the following steps.
Construct the similarity matrix of input documents using the TSF-ISF measure.
Assign part-of-Speech to each word in the document to get tagged documents.
Apply tagged documents to the parser to find dependencies between words in a sentence.
Identify verbs and objects (nouns/adjectives) that are affected by it in each sentence.
Find the contextual object. If more than one object is there for the same verb then the one with maximum weight amongst all the objects corresponding to the given verb is selected as the contextual object.
Add the contextual object weight with its similarity matrix weight to form the semantic similarity matrix.
Perform LSA to reduce the rank or truncate the singular value matrix and this reduction will have the effect of preserving the most important semantic information in the text.
3.1.1. Part-of-speech tagging
In corpus linguistics, part-of-speech tagging is the process of marking up the words in a corpus as corresponding to a particular part of speech, based on its definition, as well as its context or the relationship to adjacent and related words in a sentence. Traditional grammar classifies words based on eight parts of speech: the verb, the noun, the adjective, the pronoun, the adverb, the preposition, the conjunction and the interjection. In fact, the same word can be a noun in one sentence and a verb or adjective in another. In order to assign part-of-speech to each word, the proposed system seeks to utilize Stanford Log–Linear Part-Of-Speech Tagging and, according to Marneffe et al. [11], this produces tagged documents. The grammatical relationships in a sentence are then extracted and the output is represented using Stanford-type dependencies as specified in Toutanova et al. [10]. This helps in identifying the verbs in each sentence of the document and the nouns and adjectives called objects affected by it.
3.1.2. Dependency parsing
A dependency parse represents dependencies between individual words. A typed dependency parser additionally labels dependencies with grammatical relations such as subject and indirect object. Every word in the sentence is dependent on each other. The Stanford-type dependencies provide a representation of grammatical relations between words in a sentence.
3.1.3. Performing LSA
LSA is a mathematical technique for extracting and inferring relations of expected contextual usage of words in passages of discourse. It utilizes SVD to identify patterns in the relationship between terms and concepts contained in an unstructured collection of text. The sentences that are closely related, but contain no common words, will be recognized as similar under LSA analysis. Queries, or concept searches, against a set of documents that have undergone LSA will return results that are conceptually similar in meaning to the search criteria even if the results do not share a specific word or words with the search criteria. LSA overcomes two of the most problematic constraints of Boolean keyword queries: multiple words that have similar meanings (synonymy) and words that have more than one meaning (polysemy). Synonymy and polysemy are often the causes of mismatches in the vocabulary used by the authors of documents and users of information retrieval systems. As a result, Boolean keyword queries often return irrelevant results and miss information that is relevant. The proposed method involves constructing a weighted term–document matrix, performing an SVD on the matrix by transforming the single term-frequency matrix, A, into three other matrices so that a term-keyword vector matrix U, a singular value matrix G and a document–keyword vector matrix VT are created, which satisfy the following relation
In equation (4), A is the supplied M*K weighted term-document matrix. U is a computed M*r matrix of term vectors where r is the rank of A. G is a computed r*r diagonal matrix of decreasing singular values which represents the strength of each keyword, and VT is a computed r*K matrix of document vectors. Dimensionality reduction is done by setting the smallest singular values to zero and by excluding the negative metrics. After selecting the relevant document, the words with highest weights are selected as keywords which are of high relevance in summary. Thus, SVD is used to reduce the rank or truncate the singular value matrix. This reduction has the effect of preserving the most important semantic information in the text while reducing noise and other undesirable artefacts of the original space of A. JAva MAtrix package (JAMA) as specified in Hicklin et al. [12] is used to perform all of the matrix operations.
3.2 Initial summary
To generate an initial summary, it is necessary to capture the relevant sentences from dataset A documents. Relevant sentences are selected based on different features. The proposed work combines basic features with additional features. During initial summary generation, a subset of ranked sentences is selected to generate the summary. A redundancy check is done between a sentence and the summary generated so far, before selecting it for the summary. The sentences are adjusted on their order of occurrence in the original documents so as to improve readability.
3.2.1. Basic features
Word feature – the calculation of word feature involves the processing of every word in each sentence. Each word has its own significance in summary generation. Word feature is calculated by calculating the word weight for each word using the TSF*ISF measure.
Position feature – the first sentence of the document is always most vital. The position feature is defined by taking into account maximum positions of 3.
Sentence length feature – to determine the importance of the sentence in a specific document, the length of the sentence plays a very important role. It is calculated by performing the division of the number of words in a sentence and the number of words in a particular document.
Sentence centrality feature– according to Fattah and Ren [13], the sentence centrality feature defines overlap of words between the given sentence and the other sentences in the same document.
Sentence with proper noun feature – according to Fattah and Ren [13], any sentence that contains a lot of proper nouns is an important one. Hence, it should be extracted and included in the summary.
Sentence with numerical data feature – according to Fattah and Ren [13], any sentence that contains numerical data is an important one. Hence, it should be extracted and included in the summary.
3.2.2. Additional features
Word similarity between sentence and topic feature– any sentence that contains words similar to the given topic is an important one. The similarity between the word and the topic is given as 1 if both word and the topic are the same. Otherwise it is set to 0.
Sentence frequency score feature – the number of sentences in which a particular word occurred in a document set from among the total number of sentences in the document set gives the sentence frequency score.
Document frequency score feature – the number of documents in which a particular word occurred in a document set from among the total number of sentences in the document set gives the document frequency score.
The sentence score, which is calculated using all the basic and additional sentence specific features, is used to determine the informativeness of the initial summary. The highest ranking sentences are selected and the summary is generated by arranging the selected sentences in the order in which they appeared in the original documents.
3.3. Update summary
To capture the relevant sentences from dataset B documents, the proposed USG system utilizes a novel update specific feature. The update summary is generated by applying basic, additional and update features. The update specific feature is defined as follows.
NTRM
The new update feature is defined as the combination of topic relevance and novelty measure. The NTRM of a word is defined in equation (5):
where i = 1, 2, …, M. In equation (5), the novel measure (NM) of a word is represented as NM(wi) and it is used to identify the novelty of the word in a sentence. The topic relevance measure (TRM) of a word is represented as TRM(wi) and it is used to measure how much the word is relevant to the topic of the article. After calculating the NTRM of each word, the NTRM of a sentence can be calculated using equation (6).
The NM of a word is defined as in equation (6):
where KB denotes number of documents in dataset B that contain word wi, KA denotes number of documents in dataset A that contain word wi, and KB denotes number of documents in dataset B. The topic relevance between sentences and topic is defined as in equation (7):
where i = 1, 2, …, m. Now the sentence score is calculated for all sentences using basic features, additional features and update feature to generate update summary.
4. Enhanced update summary generation system
In order to retrieve higher semantic information from the documents, there is a need for deeper linguistic or semantic analysis of the source documents instead of relying only on document word frequencies to select important concepts. The summary should contain the aspect-oriented information for all aspects. The summary generated is guided by pre-defined aspects that are employed to enhance the quality and readability of the resulting summary.
4.1. EUSG steps
Initially the documents in the dataset are split into sentences and those sentences are annotated with predefined aspects, prepositions and named entities.
Sentence representation is enhanced by extracting concepts from Wikipedia.
Individual sentences are mapped into concepts and the individual word score is calculated based on the TSF-ISF.
For each sentence, the score is then calculated based on basic and additional features for dataset A documents and based on basic and additional as well as update features for dataset B documents.
The highest ranking sentences are selected and ordered in such a way that the sentences are included in the original documents and finally an initial summary is generated.
Update summary is generated after removing redundancy.
4.1.1. Sentence annotation
In order to select salient sentences, all of the sentences are annotated with predefined aspects, prepositions and named entities. The documents from the dataset are split into sentences and annotated with appropriate template tags. These annotations include both objective and subjective tags. As any standard Named Entity Recognition can only tag objective tags, the proposed EUSG system would annotate all of the documents with all possible tags. A sentence is tagged with multiple tags if it has more than one answer to the template.
Consider the following two sentences taken from the document D08021D:NYT_ENG_20050707 related to Attacks category.
Sentence 1: The next explosion occurred at 8:56 a.m. near King’s Cross Station, where the death toll was 21, the police said.
Sentence 2: Twenty-one minutes later, at 9:17 a.m., a third blast ripped through a train coming in Edgware Road underground station, killing seven.
Sentence annotation with aspects
This process aims to produce semantic summary using important aspects such as what, when, how, etc. These aspects define what counts as important information but the summary also includes other facts that are considered especially important. Annotations of the above sentences with aspects are:
The next explosion occurred at <when>8:56 a.m.<when> near <where>King’s Cross Station</where>, where the <who affected>death toll was 21</who affected>, the police said. Twenty-one minutes later, at <when>9:17 a.m.</when>, a third blast ripped through a train coming in <where>Edgware Road</where> underground station, <who affected>killing seven</who affected>.
Sentence annotation with prepositions
In English grammar, a preposition is a part of speech that links nouns and pronouns to other phrases in a sentence. A preposition generally represents the temporal, spatial or logical relationship of its object to the rest of the sentence. It is very interesting to observe how prepositions implicitly capture the key elements in a sentence. The list of prepositions used for calculating sentence importance is limited to simple single-word prepositions like ‘in’, ‘on’, ‘of’, ‘at’, ‘for’, ‘from’, ‘to’, ‘by’, ‘with’, etc. Annotations of the above sentences with prepositions are as follows:
The <preposition>next</preposition> explosion occurred <preposition>at</preposition>8:56a.m. <preposition>near</preposition> King’s Cross Station, where the death toll was 21, the police said. Twenty-one minutes later, <preposition>at</preposition> 9:17 a.m., a third blast ripped <preposition>through</preposition> a train coming <preposition>in</preposition> Edgware Road underground station, killing seven.
Sentence annotation with named entities
Prior observations in the given data lead to the understanding that, the more types of named entities that a sentence contains, the stronger is the likelihood of its capabilities in answering a set of questions like ‘What happened?’, ‘Who was involved?’, ‘And where did this happen?’ Named entities refer to the objects for which proper nouns are used in a sentence. Seven basic named entities are identified: person, location, date, time, organization, money and percentage. Stanford Named Entity Recognition, developed by Finkel et al. [14], is employed to identify person, location and organization entities. Others are extracted by applying patterns. Annotations of the above sentences with named entities are:
The next explosion occurred at <time>8:56 a.m.</time> near <location>King’s Cross Station</location>, where the death toll was 21, the <person name>police</person name> said. Twenty-one minutes later, at <time>9:17 a.m.</time>, a third blast ripped through a train coming in <location>Edgware Road</location> underground station, killing seven.
4.1.2. Semantic element extraction strategy
The words are conventionally considered to be units of the text to calculate the importance. Simple word counts, frequencies and synonym-based word frequencies in the document collection have proved to work well in the context of summarization as shown in Nastase et al. [15]. The proposed EUSG system uses semantic concepts in computing sentence importance. Wikipedia is a vast, interlinked network of documents providing multilingual database of concepts. It is a web-based, free-content encyclopaedia, a comprehensive and well-organized knowledge repository as specified in Yajie and Li [16]. Links exist in Wikipedia’s articles that are used to direct the user and to recognize related pages. Wikipedia Miner is a freely available toolkit for navigating and making use of the content of Wikipedia. The proposed EUSG system seeks to create a concept database from Wikipedia by selecting the concepts that appear explicitly in sentences. Each word in each sentence is compared with the concept database.
Let Con = {cp1, cp2, …, cpn} be the set of concepts in the concept database. To improve accuracy and calculate the weight of each word, the proposed EUSG system adopts TSCF. Every word’s TSCF is calculated by performing synset extraction, concept database construction and term frequency calculation. The word weight is calculated as defined in equation (8).
Here i = 1, 2, …, M. The TSCF of every word is obtained using equation (9):
where wi ∈ {w ∪ syn(w)}. In TSCF calculation, to include word synonym into account, the TF of each word and its synonym is multiplied by α where α = 1 for the word and α = 0.5 for the synonym of the word and β = 1 if the word itself is a concept in the concept database.
4.2 Initial summary
To generate the initial summary, the proposed work combines basic features defined in Section 3.2.1 with the following additional features.
Aspects feature – any sentence that contains important aspects is considered as an important sentence. This feature is calculated by dividing the total number of aspects present in the sentence by the length of the sentence.
Prepositions feature – a sentence is considered as important if it consists of more prepositions. This feature is calculated by dividing the total number of prepositions present in the sentence with the length of the sentence.
Named entities feature– a sentence with more named entities is an important one. This feature is calculated by dividing the total number of named entities present in the sentence by the length of the sentence.
Semantic concepts feature – if a sentence has more number of semantic concepts then it is considered as salient. This feature is calculated by dividing the total number of concepts present in the sentence by the length of the sentence.
4.3 Update summary
To generate an update summary, six basic features, four additional features and an update specific feature are used. The update specific feature is defined as follows:
Novel sentence similarity measure (NSSM) – this feature selects novel sentences that have not been included in the initial summary. All sentences in the initial summary are considered candidate sentences. New sentences that have least similarity with these candidate sentences are chosen as sentences in the update summary. The similarity between candidate sentences and sentences in dataset B is calculated by dividing the sum of the weight of the words that occur in both sentences by the sum of the weight of the words that occur in the shorter sentence. The benefit is that, if a sentence contains all the words of another sentence, that is, if one sentence is totally a part of another, then their similarity is 1.
5. Evaluation domain and result analysis
The test corpus in TAC 2008 comprises 48 topics. Two time slices called Dataset A and Dataset B are contained in each topic, and there are 10 documents in each time slice. Dataset A documents are utilized to generate initial summary and Dataset B documents are used to generate update summary.
5.1. Recall/precision
Recall is the ability of the search to find all the relevant sentences in the corpus. It is defined as the fraction of the sentences that are relevant to the topic that are successfully retrieved.
In equation (10), relevant sentences are those that are identified in the human-generated summary and retrieved sentences are those that are retrieved by the system. Precision is the ability to retrieve top-ranked sentences that are mostly relevant. It is defined as the fraction of retrieved sentences that are relevant to the search using equation (11):
To maintain standard Recall level, a Precision value is interpolated for each standard Recall level. This can be represented by rj ({0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0} such that r0 = 0.0, r1 = 0.1, …, r10 = 1.0. The interpolated Precision at the jth standard Recall level is the maximum known Precision at any Recall level between the jth and (j + 1)th level and it is defined as in equation (12):
5.2. ROUGE1 score
Recall-Oriented Understudy for Gisting Evaluation (ROUGE) calculates n-gram overlaps between automatically generated summaries and previously written human summaries [17]. A high level of overlap should indicate a high level of shared concepts between the two summaries. The ROUGE1 score is based on the overlap of unigrams between automatically generated summaries and human-generated summaries.
In equation (13), C is the count of number of unigrams that occurs in machine and human summary and T is the total number of unigrams in D.
Table 1 shows actual Recall and Precision calculated by the existing NESS system, proposed USG and EUSG systems for initial summary at 20% compression rate (CR). Table 2 shows the interpolated Recall and Precision of NESS, USG and EUSG systems for initial summary at 20% CR. The CR, also known as condensation rate, will be calculated as summary length over source length and it will vary from 0 to 100%. The quality of the summary is appropriate when the compression rate is 5–30%. The 25 relevant sentences are selected manually using human narrators. Values in Tables 1 and 2 show that better Precision and Recall values are obtained by EUSG approach owing to its deeper semantic analysis on sentences.
NESS, USG and EUSG actual Precision and Recall for initial summary at 20% CR
NESS, USG and EUSG interpolated recall and precision for initial summary at 20% CR
Figure 1 shows Recall–Precision graph at 20% CR for initial summary and Figure 2 shows Recall–Precision graph at 20% CR for update summary. The three curves show the behaviour of NESS, USG and EUSG systems. The figure shows that, at 20% CR, high Precision is obtained by EUSG system for both initial and update summary.
Recall–precision graph for initial summary. Figure is reproduced in colour in online version.
Recall–precision graph for update summary. Figure is reproduced in colour in online version.
Figure 3 shows the ROUGE1 score at 20% of initial summary at different CR for NESS, USG and EUSG systems. The figure shows that, at 20% CR, a high ROUGE1 score is obtained by the EUSG system. This shows that the proposed EUSG approach-generated initial summary highly correlates with the human-generated summary by performing annotation-based sentence extraction strategy and TSCF-ISF measure.
ROUGE1 score of initial summary. Figure is reproduced in colour in online version.
Figure 4 compares the ROUGE1 score of different summaries. The result shows that the overlap between initial summary (IS) and update summary (US) is low in the proposed EUSG system owing to the selection of unique sentences from dataset B documents through the use of update specific feature NSSM. Also IS and US of the proposed EUSG system highly correlate with human summary when compared with USG and NESS systems.
ROUGE1 score performance measure. Figure is reproduced in colour in online version.
6. Conclusion
Multidocument summarization has become an important tool for assisting and interpreting text information in today’s fast-growing information milieu. It is very difficult for human beings to manually summarize large documents of text. Furthermore, there is a need to produce new and relevant information that keeps changing over a period of time. This research work has paid careful attention to these problems and has sought to present a solution that will automatically generate an initial summary and an update summary. An attempt has been made to produce good summaries that include concise, relevant and non-redundant sentences. More specifically, extractive summarization approaches are dealt with in which important sentences are selected based on statistical and linguistic features. The proposed two approaches have focused on specified dynamic document collection, attempting to confine the changes over time. In USG approach the initial summary and then the update summary are generated. In this approach, LSA is performed in order to maintain important semantic information in the text. To select relevant sentences, basic features, additional features like word similarity between sentence and topic, sentence frequency and document frequency scores, and update specific features like NTRM are used. Relevancy is improved by employing a semantic similarity measure. The initial summary and update summary were compared with the manual summary and the results show that the summaries generated by the proposed USG system are efficient compared with the existing NESS system. The overlap between initial and update summaries are also low in the proposed USG system. In the EUSG approach, in order to produce a semantically enhanced summary, a sentence-level semantic analysis based on sentence annotation with aspects, prepositions and named entities is utilized. Semantic element extraction strategy is used to select important concepts from documents by utilizing Wikipedia as a resource. The initial summary is generated by combining six basic features with new additional features like sentence annotation with aspects, prepositions, named entities and sentences with a semantic concepts feature. The update summary is generated using the NSSM feature. The results show that the proposed EUSG system is found to produce better summaries compared with both NESS and USG approaches owing to its enhanced semantic analysis. The overlap between initial and update summaries is also very low in the proposed EUSG system.
Footnotes
Funding
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.