Recognition of English information and semantic features based on SVM and machine learning

Abstract

With the deepening of people’s research on event anaphora, a large number of methods will be used in the identification and resolution of event anaphora. Although there has been some progress in the resolution of the current event, the difficult problems have not yet been completely resolved. This study analyzes the English information anaphora resolution based on SVM and machine learning algorithms and uses the CNN three-layer network as the basis to model the structure. Moreover, this study improves the semantic features by adding semantic roles and analyzes and compares the performance of the improved semantic features with those before the improvement. In addition, this study combines semantic features to compare and analyze each feature combination and uses a dual candidate model to improve the system. Finally, this study analyzes the experimental results. The results show that the performance of the system using the dual candidate model is better than that of the single candidate model system.

Keywords

SVM machine learning English information anaphora resolution feature recognition

1 Introduction

With the rapid development of computer technology and the Internet, the informatization of society has developed to a whole new stage. The transmission and exchange of information has become an important foundation for the operation of modern social life, and all kinds of information are exploding. While enjoying the convenience provided by massive amounts of information, people are also faced with the dilemma of finding the content they need from the vast amount of Some technologies that integrate the achievements of natural language processing, such as information retrieval, information extraction, text classification, automatic summarization, machine translation, and multilingual information processing systems, meet people’s need for accurate positioning of information from different angles, which greatly facilitates users who are tired of massive information. However, people find that these systems still cannot really understand and meet the needs of users, so they must improve the level of natural language processing technology from all aspects [1]. Natural language is an interdisciplinary subject, which involves linguistics, mathematics and computer science and spans three major knowledge areas of liberal arts, science and engineering. In the information society, the knowledge level of language information processing and the total amount of information processed each year have become one of the important indicators to measure the modernization level of a country [2]. Natural language provides people with a rich means of expressing thoughts, and different people have different expression habits. When describing the same definite thing, different people have different expressions. Even for the same article, different people have different understanding of the article. It is very difficult for computers to deal with natural articles that humans can understand. How to make machines understand natural language is a huge challenge for researchers of natural language processing. One of the challenges is how to deal with anaphora. Anaphora is a common linguistic phenomenon in natural language, which refers to the use of a pronoun in a text to refer to a previously spoken language unit. The common expressions of anaphora s include abbreviations, proper nouns, pronouns and other language phenomena. This phenomenon is often carried out throughout the article and widely exists in various expressions of natural language. No matter from the discussion of the law of language itself or from the perspective of application, anaphora resolution is a problem that linguists and computational linguists cannot ignore. Anaphora digestion plays a very important role in many areas of NLP (Nature Language Process). In fact, a fascinating article must use multiple expressions to represent the same entity. However, for natural language processing systems, this undoubtedly greatly increases the difficulty of processing. With the development of other related work of natural language, anaphora dissolution has become the bottleneck of natural language processing [3].

In the field of computational linguistics, anaphora resolution also shows unprecedented importance. Anaphora resolution is an indispensable part of text understanding in natural language processing. Moreover, anaphora resolution is involved in many applications such as text summarization, machine translation (Machine Translation), multilingual information processing, and information extraction.

2 Related work

The method based on syntactic knowledge makes full use of syntactic knowledge, and it is applied to anaphora resolution in a heuristic way. The literature [4] proposed a RAP algorithm that uses McCord’s Slot Grammar to obtain the syntactic structure of the document and calculates the saliency of each antecedent candidate by manually weighting various language features (such as subject and object). Moreover, the algorithm uses filtering rules to determine the antecedent, and achieves the elimination of third-person pronouns and reflexive pronouns within and between sentences. The literature [5] modified and expanded the RAP algorithm to avoid building a complete parse tree and preprocessed with natural language processing tools to obtain shallow information such as part-of-speech tagging and syntactic function tagging to determine the antecedent. The literature [6] quantified different language features on the basis of part-of-speech tagging and used the method of calculating weights to complete the ranking of possible candidates, so as to select antecedents and solve the pronoun resolution. However, the language features in the above work are manually weighted. In order to realize the automation of language feature weighting, the literature [6] improved the RAP algorithm and explored the use of the maximum entropy model to automatically determine the weight of various language features.

With the continuous emergence of annotated corpora and the rapid development of the Internet, it is becoming more and more convenient to obtain experimental corpora. Recently, most researches on anaphora digestion tend to be based on corpus-based anaphora digestion methods. The specific methods can be roughly divided into the following three categories: 1) Rule-based methods: The literature [7] used the center theory to first classify the antecedent candidates according to the difference between the foresight or the foresight center, and then used various center acquisition algorithms to select the antecedent. As a rule-based system representing the current international advanced level, literature [8] proposed a multi-agent strategy based on restriction rules, which has achieved considerable success. The system results in the MUC.6 test corpus have a recall rate of 65.8%, an accuracy rate of 84.7%, and an F value of 73.9%. The system results in MUC.7 have a recall rate of 55.7%, an accuracy rate of 82.8% and an F value of 66.5%.2) Statistical-based methods: The literature [9] gives priority to those antecedents that have higher co-occurrence antecedent candidates as pronouns, and conducts research on the dissection of the pronoun “it”. Literature [10] proposed a method based on Bayesian probability statistical model to resolve the third person singular pronouns in the text. The literature [11] proposed to use the clustering method to resolve the same finger of noun phrases. The basic idea is to collect the basic noun phrases in the chapter, cluster the noun phrases according to the features of the phrases and determine whether the two nouns belong to the same category. The recall rate of the system in MUC.6 is 55%, the accuracy rate is 53%, and the F value is 54%) Classification-based methods: The literature [12] converted the problem of judging antecedent into a classification problem, and judged whether there is a anaphora relationship between the pronoun and each antecedent candidate through a classifier. This idea has opened up a brand-new path for the future research on anaphora resolution. The literature [13] give detailed and complete implementation steps and designed and developed a learning-based system to resolve noun phrases in unrestricted text. The system is divided into 5 parts in total:1)identify candidates, 2) construct noun phrase feature vectors, 3) construct training examples, 4) train classifiers, 5) identify test documents, and construct a referral chain. The performance of the classifier in the system [13], the classification ability of each attribute in the feature vector, and the cause of the error are analyzed in detail. The F value s that the system finally resolved on the MUC.6 and MUC.7 corpora are 62.6 and 60.4, respectively. In recent years, many researchers have expanded the system to varying degrees based on the literature [13] and have made some progress. Typical systems include: The literature [14] extracted 53 different lexical, grammatical, and semantic features. The literature [15] proposed a dual candidate model to directly learn the competitive relationship between the proactive language candidates to better determine the proactive language. The literature [16] continued to explore the role of semantic information in the antecedent candidate anaphora chain in the pronoun (especially neutral pronoun) anaphora resolution. On this basis, literature [17] further uses context information and network mining technology to automatically identify the semantic category of pronouns, thereby better solving the anaphora resolution of pronouns (especially neutral pronouns). At present, most of the anaphora resolution systems use a local optimization method, that is, for each anaphora, the best antecedent is selected according to different algorithms. In order to achieve global optimization, the literature [18] used a Bell tree to represent the search space, in order to find the optimal anaphora resolution scheme. The literature [19] selected the best partitioning scheme from different systems according to the advantages and disadvantages of different systems.

The literature [20] used the lattice framework to propose the idea of anaphora classification in a context-sensitive semantic environment and give the corresponding algorithm. The literature [21] proposed the HNC-based anaphora resolution method, which utilizes the type features of various semantic blocks and the structural features between semantic blocks to use exclusion rules within and between semantic blocks. Moreover, the principle of partial focus priority (similar to the central theory) is used for priority selection to achieve the elimination of personal pronouns between sentence sequences. The literature [22 –24] used a method based on weakened linguistic knowledge to solve the elimination of personal pronouns and combined with a anaphora selection strategy to conduct research on anaphora elimination.

3 Undirected graph model

CRFs can be regarded as an undirected graph model or Markov domain. The global condition is on the random variable X representing the observation sequence. Formally, G = (V, E) is used to represent an undirected graph. Among them, V is a set of nodes, and E is an undirected edge between nodes. The node y represents a set of continuous or scattered random variables, and each node v (v ∈ V) in V corresponds to an random variable of element Y_v of y. The graph model is related to the joint probability distribution of the random variables represented by the nodes in the graph. The parameters of these probability distributions depend on the direct conditional independence of random variables in the graph.

If the directed graph model G = (V, E) has identified the conditional independence relationship between the topologically ordered sets of nodes in G, the joint probability distribution of the directed graph model can be decomposed into the product of conditional probabilities. The joint probability of the random variable represented by node V = v₁, v₂, ⋯ , v_n can be expressed as formula (1). $p (V = v_{1}, v_{2}, \dots, v_{n}) = \prod_{i = 1}^{n} p (v_{i} | v_{π i})$ (1)

In the formula, v_πi is the parent node set of node v_i. The parameters of the Markov random domain are different from those of the directed graph model.

The first step in parameterizing the undirected graph model G = (V, E) is to find the set of nodes that each local function acts on. In order to find this set, the concept of conditional independence is introduced. A, B, C represent three disjoint subsets. If node set V_B separates V_A and V_C, then we can think that the random variable represented by node set V_A is conditionally independent of the random variable represented by V_C under the condition that node set V_B is given. For an undirected graph model, a simple graphical separation theory concept is used. Under the condition that V_B is given, V_A is independent of V_C, that is, the path from any node V_{i|i∈A} in V_A to any node V_{i|i∈C} in V_C passes through at least one node V_{i|i∈C}B in V_B. In order to identify the part of the node where the local function works on its nodes, according to the conditional independent properties of the undirected graph model G, if there are no edges between the two nodes v_i and v_j in the graph G, it means that under the condition that all other nodes in the graph are given, these two nodes must be conditionally independent. Therefore, when choosing a local function, it must be ensured that the joint probability distribution can be decomposed to prevent v_i and v_j from appearing in the same local function.

The simplest way to achieve this decomposition requirement is to ensure that the nodes acting on this local function form a fully connected node subset or clique in G. Obviously, this ensures that no local function acts on any pair of nodes that are not directly linked. Moreover, if two nodes appear in a clique at the same time, a connection is established by a local function defined on the clique where these two nodes are located. To further refine the concept of local functions, each local function is defined on a largest clique or a clique whose nodes cannot be expanded and whose nodes remain fully connected. Then, it is meaningless to define a potential function on any clique corresponding to this subset of the largest clique. Therefore, the simplest local function set corresponding to the conditional independence of graph G is the set of functions defined on V_C that may be involved in the largest clique C of G. These local functions Ψ_{V_C(V_C)} are called potential functions and are strictly non-negative real function forms.

However, the product of a set of non-negative real number functions does not satisfy the axiom of probability. Therefore, in order to satisfy these axioms and ensure that the product is indeed the joint probability distribution of the random variables represented by the nodes in G, and a normalization factor Z is defined: $Z = \sum_{v_{1}, v_{2}, \dots, v_{n}} \prod_{c \in C} Ψ_{V_{C} (V_{C})}$ (2)

Among them, C is the set of all largest cliques in graph G. Therefore, the joint probability can be calculated using the following formula: $p (V = v_{1}, \dots, v_{n}) = \frac{1}{Z} \prod_{c \in C} Ψ_{V_{C} (V_{C})}$ (3)

By using the Hammersley-Clifford (J. Hammersley, 1971; Peter Clifford, 1990) theorem, we get the above formula.

4 Graph structure of CRFs

CRFs are a framework of undirected graph models, which can be used to define the joint probability distribution of a labeled sequence, under a set of observation sequences to be labeled is gave. If it is assumed that X and Y respectively represent the joint distribution random variable of the observation sequence to be marked and its corresponding marking sequence, the conditional random field (X, Y) is an undirected graph model with the observation sequence X as the global condition.

Generally, we define G = (V, E) to be an undirected graph, and Y ={ Y_v|v ∈ V }, that is, each node in V corresponds to the component Y of the label sequence represented by a random variable. Therefore, the condition of the entire graph and the distribution category associated with the graph is X, so the form of the joint distribution category associated with G is p (Y₁, ⋯ , Y_n|X), where Y_i and x are the category sequence and the observation sequence, respectively. If each random variable Y_v satisfies the Markov attribute on G and all random variables Y_{(u|u≠v,{u,v}∈V)} except X and Y_v are given, then the probability expression of the random variable Y_v is: $P (Y_{v} | X, Y_{u,} u \neq v) = P (Y_{v} | X, Y_{u,} u \sim v)$ (4)

Among them, u ∼ v indicates that u and v are adjacent in graph G, and (X, Y) is a conditional random field.

Theoretically, if the graph G shows the conditional dependency relationship between the tag sequences to be modeled, its structure can be arbitrary, and it describes the conditional independence in the tag sequence. But when used to model sequence labeling tasks, the simplest and most common graph structure encountered is that the nodes corresponding to the elements of Y form a simple first-order chain. We refer to this conditional random field as Linear-chain CRFs, as shown in Fig. 1:

Fig. 1

Model structure of linear CRFs.

Since we want to define a probability distribution P (Y|X), the random variable representing y is only a part of graph G. In addition, there is no graph structure among the elements of X. The reason is that we only use the observation sequence as a condition and do not make any independent assumptions about X.

As a probability automaton, each state transition of CRFs corresponds to a non-normalized weight. The non-normalized nature of these weights means that the transitions in the CRFs model are treated differently. Therefore, for any given state, the probability distribution passed to the subsequent state may be enlarged or reduced, and the weight of any state sequence is given by the global normalization factor, so that CRFs avoid the problem of label bias.

5 Potential function representation of CRFs

The graph structure of CRFs can be used to decompose the joint distribution of Y_v into a product of a normalized potential function. The potential function comes from the concept of conditional independence and is strictly non-negative, real-valued function. Among them, Y_v is an element of y. The vertices in G involved in each potential function represent a subset of random variables. According to the independent definition of the undirected graph model, if there are no edges between two vertices in G, it means that the random variables represented by the two vertices are independent of other given vertices in G. Therefore, the potential function must ensure that the joint probability can be decomposed so that random variables with independent conditions will not appear in the same potential function. The easiest way to meet this requirement is to ensure that each potential function acts on a set of random variables, and the vertices corresponding to these variables form a largest clique. Under the CRFs of the chain structure, each potential function g acts on the adjacent labeled variables Y_i and Y_i+1.

In the case where the observation sequence X is given, Lafferty et al. defined that the probability of labeling sequence Y is a normalized form of the product of the potential function. Among them, the form of each factor is shown in formula (5): $exp (\sum_{j} λ_{j} t_{j} (Y_{i - 1}, Y_{i}, X, i) + \sum_{k} μ_{k} s_{k} (Y_{i}, X, i))$ (5)

Among them, t_j (Y_i-1, Y_i, X, i) is the feature function of the entire observation sequence and position and the label of i and i - 1, and s_k (Y_i, X, i) is the feature function of the entire observation sequence and position i. The parameters λ_j and μ_k are the feature weights related to the feature functions t_j and s_k respectively obtained during training. The range o F value s of j and k depends on the number of features in the template.

When defining the feature function, the set of real-valued features b (X, i) of the observation sequence can be constructed to describe the empirical distribution features of the training data. These features have the same distribution as the model. The following introduction is an example: $b (X, i) = {\begin{matrix} \begin{matrix} 1 The observed value of if position \\ i is the Chinese character " shu " \end{matrix} \\ 0 otherwise \end{matrix}$ (6)

Each feature function represents a real-valued observed feature b (X, i). If the current state (state function) or the previous state and the current state (transfer function) have specific values, then all the feature functions are real-valued. For example, the following transfer function is: $t_{j} (Y_{i - 1}, Y_{i}, X, i) = {\begin{matrix} \begin{matrix} b (X, i) if Y_{i - 1} = verb, \\ Y_{i} = noun \end{matrix} \\ 0 otherwise \end{matrix}$ (7)

In the following description, we use formula (8) to represent the state function: $s_{k} (Y_{i}, X, i) = s_{k} (Y_{i - 1}, Y_{i}, X, i)$ (8) and, $F_{j} (Y, X) = \sum_{i = 1}^{n} f_{j} (Y_{i - 1}, Y_{i}, X, i)$ (9)

In formula (9), the feature function f_j (Y_i-1, Y_i, X, i) is a state feature function s_k (Y_i, X, i) or a transfer feature function t_j (Y_i-1, Y_i, X, i). Therefore, for a given observation sequence X = X₁, X₂, ⋯ , X_i, ⋯ , X_n, the probability of its corresponding label sequence Y = Y₁, Y₂, ⋯ , Y_i, ⋯ , Y_n is: $P (Y | X, λ) = \frac{1}{Z (X)} exp (\sum_{j} λ_{j} F_{j} (Y | X))$ (10)Z (X) is the normalization factor, and its form is: $Z (X) = \sum_{Y} exp (\sum_{j} λ_{j} F_{j} (Y | X))$ (11)

Then, we can express P (Y|X).

Now, we have built a statistical model of P (Y|X) using CRFs. The labeling task for solving the sequence is to find Y* to make the maximum value of P (Y|X). Z (X) has nothing to do with Y, so Y* is: $\begin{matrix} Y * = arg max_{Y} P (Y | X) = arg max_{Y} \frac{1}{Z (X)} \\ exp (\sum_{j} λ_{j} F_{j} (Y | X)) = arg max_{Y} \sum_{j} λ_{j} F_{j} (Y | X) \end{matrix}$ (12)

The optimal solution Y* can be obtained using dynamic optimization methods such as Viterbi.

6 Parameter estimation of CRFs

The main task of establishing the CRFs model is to estimate the feature weight λ from the sample data. CRFs parameter estimation can use Maximum Likelihood Estimation (MLE) and Bayes Estimation. The following mainly introduces the parameters of CRFs model using maximum likelihood estimation.

In the training set Y* T = { 〈 X^k, Y^k 〉 }, the maximum likelihood parameter estimation is a function that assumes that P (Y|X, λ) is λ. The λ that maximizes the logarithm of P (Y|X, λ) is the estimated value, and its likelihood value is: $\begin{matrix} L_{\land} = \sum_{T} log P (Y^{k} | X^{k}, λ) = \\ \sum_{T} log \frac{1}{Z (X^{k})} exp (\sum_{j} λ_{j} F_{j} (Y^{k} {| X}^{k})) \end{matrix}$ $\begin{matrix} = \sum_{T} (\sum_{j} λ_{j} F_{j} (Y^{k} {| X}^{k}) - log (Z (X^{k}))) \end{matrix}$ (13)

The maximum value is: $\land * = arg max_{λ} \sum_{T} log P (Y^{k} | X^{k}, λ)$ (14)

Since L_∧ is a convex function and the zero point of the derivative is the maximum point, this global maximum can converge to.

To obtain the maximum value of L_∧, we have to maximize the difference between the correct path and all candidate paths. Therefore, the trained CRFs model will distinguish the correct path from all other paths, which can avoid the problem of category deviation.

Therefore, when differentiating λ, the partial derivative formula is:

$\begin{matrix} \frac{\partial L_{\land}}{\partial λ_{j}} = \sum_{T} (\sum_{j} F_{j} (Y^{k} | X^{k}) - E_{P (Y | X^{k})} \\ [F_{k} (Y | X^{k})]) \end{matrix}$ (15)

It can be abbreviated as: $\frac{\partial L_{\land}}{\partial λ_{j}} = O_{j} - E_{j} = 0$ (16)O_j is the frequency of occurrence of λ_j in the training set T, and E_j = ∑_TE_P(Y|X^k) [F_k (Y|X ^k)] is the expected feature of λ_j in the model distribution. If the direct calculation of E_j requires a large amount of calculation, it can be solved using dynamic programming methods, such as the forward-backward algorithm.

If maximum likelihood estimation is used directly, over-learning problems may occur. This problem can be solved by introducing a penalty function. For example, when using the penalty term $\frac{\sum_{j} λ_{j}^{2}}{2 σ^{2}}$ , the original problem becomes: $L_{\land^{'}} = L_{\land} - \frac{\sum_{j} λ_{j}^{2}}{2 σ^{2}} + const$ (17)

Its derivative becomes: $\frac{\partial L_{\land^{'}}}{\partial λ_{j}} = \frac{\partial L_{\land}}{\partial λ_{j}} - \frac{λ_{j}}{σ^{2}}$ (18)

Therefore, the parameter estimation problem of λ can be solved by the optimization method. The iterative methods such as GIS and IIS can be used. The implementation of this paper uses L-BFGS (Limited-memory Brayden-Fletcher-Goldfarb-Shannon) algorithm.

For a chain of CRFs, we can add a start state tag and an end state tag to each sentence to mark the sequence. Y₀ and Y_n+1 represent the start tag and end tag respectively. When an observation sequence x is given, the probability P (Y|X, λ) of labeling sequence F can be effectively calculated using a matrix.

We set Ψ to be the alphabet of marks, Y′ and Y″ are the marks from this alphabet. Moreover, we define the set of n + 1 matrices as {M_i (X) |i = 1, ⋯ , n + 1 } 1). Among them, M_i (X) is |Ψ × Ψ| matrix, and the matrix element form is: $M_{i} (Y^{'}, Y^{″} | X) = exp (\sum_{j} λ_{j} f_{j} (Y^{'}, Y^{″}, X, i))$ (19)

When the observation sequence X is given, the conditional probability of the label sequence F without normalization can be expressed as the product of the elements of the n + 1 matrix: $P (Y | X, λ) = \frac{1}{Z (X)} \prod_{i = 1}^{n + 1} M_{i} (Y_{i - 1}, Y_{i} | X)$ (20)

Similarly, the normalization factor Z (X) of the observation sequence X can be calculated from the M_i (X) matrix by using the Closed Semirings method. The algebraic structure is a general framework to deal with the path problem in the graph. The value of Z (X) is given by the product of the M_i (X) matrix from the start position to the end position. It is expressed as: $Z (X) = {[\prod_{i = 1}^{n + 1} M_{i} (X)]}_{start, end}$ (21)

Therefore, we only need to get M_i (X) to calculate the value of Z (X).

In the parameter estimation process, whether iterative convergence or gradient-based methods are used, in order to calculate the maximum likelihood parameters, for each observation sequence X^k in the training data, we must effectively calculate the expect E_P(Y|X^k) [F_k (Y|X ^k)] of each feature function related to the distribution of the CRFs model. $\begin{matrix} E_{P (Y | X^{k})} [F_{k} (Y {| X}^{k})] = \frac{1}{Z (X^{k})} \\ \sum_{Y} [exp \sum_{j} λ_{j} F_{j} (Y^{'}, X^{k}) \times \sum_{j} F_{j} (Y^{'}, X^{k})] \end{matrix}$ (22)

It is not advisable to calculate the above formula in a simple way. The reason is that it needs to be summed on the label sequence: if the observation sequence X^k has n elements, then Y^k corresponds to n^|Ψ| elements, and the cost of summing these items is very high. Therefore, the forward-backward algorithm similar to HMM is usually used to solve this problem.

We define the forward vectors α_i (X) and backward vectors β_i (X) as: $α_{0} (Y | X) = {\begin{matrix} 1 & if Y = Y_{n + 1} \\ 0 & otherwise \end{matrix}$ (23) $β_{n + 1} (Y | X) = {\begin{matrix} 1 & if Y = Y_{n + 1} \\ 0 & otherwise \end{matrix}$ (24)

The recursive definitions are: $α_{i} {(X)}^{T} = α_{i - 1} {(X)}^{T} - M_{i} (X)$ (25) $β_{i} (X) = M_{i + 1} (X) β_{i + 1} (X)$ (26)

Therefore, $\begin{matrix} E_{P (Y | X^{k})} [F_{k} (Y {| X}^{k})] = \\ \sum_{i = 1}^{n} \sum_{{YY}^{″}} \frac{α_{i - 1} (Y^{'} | X^{k}) M_{i} (Y^{'}, Y^{″} | X^{k}) β_{i} (Y^{″} | X^{k})}{Z (X^{k})} \end{matrix}$ (27) and, $Z (X^{k}) = β_{0} (X^{k}) = α_{n + 1} (X^{k})$ (28)

Then, we can quickly find the feature expectation, so that we can use the machine learning method to calculate the model feature weight λ.

7 Model construction and performance analysis

The model constructed in this study is based on SVM and machine learning technology, and the model structure is based on the CNN three-layer network. The most basic CNN neural network is usually composed of three layers: input layer, hidden layer and output layer. The data enters the hidden layer through the input layer, and the sampling layer in the hidden layer maps the features extracted by the convolution layer. Finally, the features enter the output layer to obtain the classification results. The structure is shown in Fig. 2.

Fig. 2

The structure diagram of convolutional neural network.

The anaphora resolution system of English information in this article uses the same framework as the benchmark system, as shown in Fig. 3.

Fig. 3

The anaphora resolution platform based on machine learning and SVM.

The experiment is conducted on the English corpus of OntoNotes 4.0 using the positive and negative examples balance method, and the samples were generated according to the sample generation method. The ratio of positive and negative examples reached about 1 : 11. This article then uses the first step of the method to filter the samples. The experimental results are shown in Table 1 and Fig. 4 below. After the first step of filtering, the ratio of positive and negative examples changes from about 1 : 11 to about 1 : 3.5. It can be seen from this that the first step of this method can effectively reduce negative cases. Moreover, from the experimental results of the system, it can be seen that after the first step of processing, the system’s recall rate and system performance have been improved. This shows that reducing the proportion of negative examples can indeed improve the performance of the system, but the imbalance of positive and negative examples still exists. Next, this study uses the second step of the positive and negative examples balance method to filter negative examples. The second step uses named entity recognition features. If a noun phrase belongs to a person’s name, organization name or place name, then this noun phrase is basically impossible to represent an event. Therefore, this article filters all the noun phrases in the sample. If the filtered noun phrase belongs to the name of a person, institution, or place, then all the examples containing this noun phrase will be removed from the sample. After the second step of filtering, the ratio of positive and negative examples is reduced from about 1 : 3.5 to about 1 : 3. Although the accuracy rate has decreased, the recall rate and system performance have been improved. This shows that the method of filtering negative examples using central theory and named entity recognition is effective.

Table 1

Statistical table of the impact of different positive and negative cases on the system

Positive example: Negative example	Correct rate P (%)	Recall rate R (%)	F value (%)
1 : 11	42.05	15.52	31.85
1 : 3.5	32.37	37.15	34.59
1 : 3	31.36	41.19	35.61

Fig. 4

Statistical diagram of the impact of different positive and negative cases on the system.

Through the above two steps, the ratio of positive and negative examples was reduced from about 1 : 11 to about 1 : 3. However, due to the particularity of anaphora and antecedent in event reference, this problem cannot be resolved like entity reference. The balance of positive and negative examples is achieved according to the filter rules such as part-of-speech and singular and plural of anaphora and antecedent. Therefore, based on the first two steps to filter the negative examples, the third step is carried out, and the positive examples are copied under the condition that the negative examples remain unchanged to continue to adjust the positive and negative examples. Table 2 and Fig. 5 show the performance of the event anaphora resolution system when the positive and negative examples are balanced using the positive and negative examples balance method.

Table 2

Statistical table of positive and negative balance methods

	Correct rate P (%)	Recall rate R (%)	F value (%)
1 : 0.9	22	85	38
1 : 1	24	68	37
1 : 1.1	28	66	40
1 : 1.2	29	63	40
1 : 1.3	30	56	39
1 : 1.5	31	50	38
1 : 3	32	40	35
1 : 3.5	33	37	33
1 : 11	42	24	31

Fig. 5

Statistical diagram of positive and negative balance method.

As can be seen from Fig. 5, as the ratio of positive examples to negative examples decreases, the recall rate tends to decrease, and the accuracy rate increases. The system performance showed a trend of increasing first and then decreasing. The ratio of positive and negative examples reached a stable range from 1 : 1.1 to 1 : 1.3 and the system performance reached the highest at 1 : 1.1. Therefore, the experiments in this paper use the experimental results of the ratio of positive and negative examples of 1 : 1.1 in subsequent experiments.

In this paper, after using SVM machine learning to achieve the best performance, considering the possibility of continuing to improve the performance of the system, in the model, the SVM system can be used to build the system model alone, or the machine learning can be used to build the model, or the two can be combined to build a dual model. This article uses a dual candidate model to try to improve the performance of the system. Moreover, this study analyzes the performance of these models, the experimental results are shown in Table 3 and Fig. 6. It can be seen from Table 3 and Fig. 6 that the single-candidate model has a lower recall rate than the benchmark system, but the accuracy and system performance are improved. This shows that the new semantic features given in this paper have a positive effect on the resolution of noun phrase event reference. Secondly, after using the dual candidate model, the accuracy, recall and overall performance of the system are improved. Compared with the single candidate model, the accuracy rate is increased by 1.75%, the recall rate is increased by 1.85%, and the overall performance is improved by 2.03%. Compared with the benchmark system, the accuracy rate is increased by 2.24%, the recall rate is increased by 0.29%, and the overall performance is increased by 2.23%. This shows that the dual candidate model has a positive influence on the event-referential resolution system of noun phrases.

Table 3

Comparison of results between single candidate model and double candidate model

Feature	Correct rate P (%)	Recall rate R (%)	F value (%)
Benchmark system	27.94	67.23	39.48
Single candidate model	28.43	65.67	39.68
Dual candidate model	30.18	67.52	41.71

Fig. 6

Comparison chart of results between single candidate model and double candidate model.

8 Conclusion

Anaphora resolution can be divided into entity referral resolution and event referral resolution according to different targets. Since the attention of the event anaphora is far less than that of the entity anaphora, the event anaphora resolution has been treated as a whole with the entity anaphora resolution as a whole. However, as the amount of information increases, information is often transmitted with the occurrence of events. The continuous development of information extraction technology makes the demand for event anaphora resolution increasingly higher, and event anaphora resolution is of great significance. Moreover, this paper has done related research work on English information anaphora and explored the role of structured syntactic information in the resolution of event noun phrase anaphora. In addition, this study uses two semantic roles to improve the semantic role expansion tree in structured features and combines semantic features to compare and analyze each feature combination and uses a dual candidate model to improve the system. Finally, this study analyzed the experimental results. The results show that the performance of the system using the dual candidate model is better than that of the single candidate model system.

References

Tommasel

and Godoy

, A social-aware online short-text feature selection technique for social media, Information Fusion 40(6) (2017), 1–17.

Nayak

and Nayak

A.K.

, Odia running text recognition using moment-based feature extraction and mean distance classification technique, Advances in Intelligent Systems & Computing 309(12) (2015), 497–506.

Ramalakshmi

and Golla

, An advanced fuzzy constructing algorithm for feature discovery in text mining, International Journal of Computer Applications 127(17) (2015), 30–34.

Gissel

S.T.

, Scaffolding students’ independent decoding of unfamiliar text with a prototype of an eBook-feature, Journal of Information Technology Education Research 14(2015) (2015), 439–470.

Oki

Takuro

, Scene text localization using object detection based on filtered feature channels and crosswise region merging, Growth & Change 21(3) (2015), 61–76.

Kamble

R.R.

and Kodavade

D.V.

, Relevance feature search for text mining using FClustering algorithm, International Journal of Computer Sciences & Engineering 6(7) (2018), 223–227.

Maruthupandi

and Devi

K.V.

, Multi-label text classification using optimised feature sets, International Journal of Data Mining Modelling & Management 9(3) (2017), 237.

Pandi

Maruthu

and Rajendran

Vimala Devi K.

, Efficient feature extraction for text mining, Advances in Natural & Applied Sciences 10(4) (2016), 64–73.

, Zhao

and Han

, A fingerprint feature extraction algorithm based on optimal decision for text copy detection, International Journal of Security & Its Applications 10(11) (2016), 67–78.

10.

Soleymanpour

and Marvi

, Text-independent speaker identification based on selection of the most similar feature vectors, International Journal of Speech Technology 20(1) (2016), 1–10.

11.

Mojaveriyan

Mohammad

and Hossein

Ebrahimpourkomleh

, Seyed jalaleddin Mousavirad. IGICA: a hybrid feature selection approach in text categorization, International Journal of Intelligent Systems Technologies & Applications 8(3) (2016), 42–47.

12.

Aghdam

M.H.

and Heidari

, Feature selection using particle swarm optimization in text categorization, Journal of Artificial Intelligence & Soft Computing Research 5(4) (2015), 38–43.

13.

Robati

Zahra

, Zahedi

Morteza

and Fayazi Far

Najmeh

, Feature selection and reduction for persian text classification, International Journal of Computer Applications 109(17) (2015), 1–5.

14.

Zia

Tehseen

, Abbas

Qaiser

and Pervez Akhtar

Muhammad

, Evaluation of feature selection approaches for urdu text categorization, International Journal of Intelligent Systems Technologies & Applications 07(6) (2015), 33–40.

15.

zia

Tehseen

, Pervez Akhter

Muhammad

and Abbas

Qaiser

, Comparative study of feature selection approaches for Urdu text categorization, Malaysian Journal of Computer Science 28(2) (2015), 93–109.

16.

, Zhe Jin

Xue

and Cui

LiHua

, Text recognition algorithm based on text features, International Journal of Multimedia & Ubiquitous Engineering 11(5) (2016), 209–220.

17.

Yan

, Li

and Zhang

, A New multi-instance multi-label learning approach for image and text classification, Multimedia Tools & Applications 75(13) (2016), 7875–7890.

18.

Agnihotri

, Verma

and Tripathi

, Computing symmetrical strength of N-grams: a two pass filtering approach in automatic classification of text documents, SpringerPlus 5(1) (2016), 942–956.

19.

Kumar

Ganesh

and Vivekanandan

, Intelligent model view controller based semantic webservice call through mishmash text featuring technique, Journal of Computational & Theoretical Nanoscience 14(4) (2017), 2021–2029.

20.

Shukla

Alok Kumar

, et al., Knowledge discovery in medical and biological datasets by integration of relief-f and correlation feature selection techniques, Journal of Intelligent & Fuzzy Systems 38(5), 6637–6648.

21.

Shi

, Bai

and Yao

, An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition, IEEE Transactions on Pattern Analysis & Machine Intelligence 39(11) (2017), 2298–2304.

22.

Shakir

Hina

, Rasheed

Haroon

and Rasool Khan

Tariq Mairaj

, Radiomic feature selection for lung cancer classifiers, Journal of Intelligent & Fuzzy Systems 38(5) (2020), 5847–5855.

23.

Zatarain-Cabada

Ramón

, Barrón-Estrada

María Lucia

, García-Lizárraga

Jorge

, et al., Java tutoring system with facial and text emotion recognition, International Journal of Advanced Computer Research 106(106) (2015), 49–55.

24.

Al-Tahraw

M.M.

, Polynomial neural networks versus other arabic text classifiers, Journal of Software 11(5) (2016), 418–430.