Semantic ordering of English machine translation based on fuzzy theory

Abstract

With the development and popularization of electronic computers and the Internet, the problem of language barriers has once again become prominent in the new era, and people are more in need of machine translation. However, there is currently no suitable method for effective semantic ordering of English machine translation. In order to better perform semantic ordering on English machine translation, the article combines fuzzy theory to construct an algorithm model, and analyzes the experimental results through evaluation indicators. The results show that with the increase of training concentration training examples, the semantic parser can learn more natural language sentence analysis methods from the training examples, and the natural language sentences that can be correctly parsed gradually increase, so with the training examples increased recall rate and F value gradually increased. The experimental results also show that the use of higher precision syntax analyzers can effectively improve the performance of statistical machine translation systems, whether in phrase-based or machine-based translation methods.

Keywords

Semantic ordering machine translation fuzzy theory semantic parser system performance

1 Introduction

Machine translation is essentially a comprehensive research field. It is rooted in science and engineering, basic research and practical development. It is the intersection of computer science, linguistics, artificial intelligence and software engineering. It is also the theoretical linguistic grammar theory, the most natural applications of dictionaries, pragmatics and segments, computational linguistic analysis algorithms, semantic representation and generation algorithms, and descriptive linguistic dictionaries and grammar acquisition. Machine translation is a truly multidisciplinary product combining science and technology. The study of machine translation also inevitably promotes the rapid development of these disciplines.

The Section 1 of the article is an introduction to the analysis of the overall structure of the article; Section 2 is a review of the fuzzy theory and machine translation reference; Section 3 is the constructed algorithm model and evaluation system; Section 4 is the experiment, including experimental data, experimental methods and evaluation indicators; Section 5 is the analysis and discussion of the experimental results; Section 6 is a summary of the full text.

2 Related work

The development of machine translation has a long history, and many research teams at home and abroad have long conducted in-depth research on it. In [1], the author surveyed some text analysis calculation methods, focusing on the supervised method of extending the old theory to new data and unsupervised techniques to discover hidden rules that were worthy of theorization. The author reviewed the use of these tools to develop the latest research on social insights through collective attention and reasoning through the content of communication, as well as the social state, roles and actions identified through communication processes to establish social relationships and through heterogeneous signals in communication. In [2], the author proposed a new neuromachinery translation (NMT) vocabulary reduction method that considered the morphological features of a language without considering the vocabulary of a given input corpus. The article also proposed an alternative word segmentation method based on supervised morphological analysis, which can help the author to measure the accuracy of the model. The results showed that the author’s method achieved a significant improvement of 2.3 BLEU points compared to the traditional vocabulary reduction technique, indicating that it can provide higher accuracy in open vocabulary translation of morphologically rich languages. In [3], the authors proposed a novel context-aware loop coder (CAEncoder) as an alternative to the widely used bi-directional encoder so that future and historical contexts can be fully incorporated into the source representation of the learning. Neural Machine Translation (NMT) relied heavily on its encoder to capture the basic meaning of the source sentence, resulting in a faithful translation. Experiments on Chinese-English and English-Chinese translation tasked have shown that CAEncoder had significantly improved over bidirectional RNN encoders on widely used NMT systems. In [4], the author proposed a heuristic method to train NMT models with very large vocabulary, and proved that this lexical expansion method can minimize the embedding quality. This allowed the author to provide a large number of NMT embedded vocabularies for future research and applications. Overall, the analysis of the article suggested that NMT embedding should be used in applications that need to organize word concepts based on similarity and/or vocabulary functionality, while single-language embedding was more suitable for modeling (non-specific) inter-word correlation. In [5], the author minimized these types of errors by interfacing statistical machine translation (SMT) models with linked open data (LOD) resources. The author conducted several experiments based on the SMT system Moses and evaluated various strategies for utilizing knowledge in multilingual link data in automatic translation of named entities. Finally, the authors analyzed best practices for multilingual linked data sets to optimize their benefits for multilingual and cross-language applications.

However, machine translation is an extremely difficult research topic. No matter how urgent the demand for it is now, a fully automated, high-quality machine translation system has not yet appeared. In [6], the authors evaluated the ability of the fuzzy inference system-direct search optimization algorithm (FIS-DSOA) to predict the daily runoff in the lower reaches of the Talehan River using data from nearby rain gauges and hydrological stations. The authors used five statistical indicators (ie correlation coefficient, efficiency coefficient, mean absolute error, MSE and consistency index) to test the prediction of the FIS-DSOA model and compared the results with the results of artificial neural networks, FIS and ANFIS models. The model greatly improved the accuracy of the daily runoff forecast. In [7], the author proposed an event-driven plan recognition method based on intuitionistic fuzzy theory, and proposed an algorithm based on identifying fuzzy event sequences to predict the future actions of objects. First, by analyzing the process and characteristics identified by the plan, the master plan was introduced into a series of subtasks of the event, and then an algorithm for predicting the plan target was created by identifying a series of events. Finally, we had proved our approach through experiments in the military field.

3 Method

3.1 Machine translation based sequencing model

(1) Machine Translation Evaluation Model (BLEU)

The closer the translation of the machine translation system is to the reference translation, the better the translation result and the higher the BLEU value. The specific evaluation method of the BLEU model is: put the candidate translation and the reference translation into the n-gram language model, and calculate the matching degree between the translated translation and the reference translation according to different n-meta models, using the actual under the specific n-gram. Match Number / The number of all translations obtained by the n-gram model in the machine translation system. Penalty factor BP is set in the BLEU model to penalize sentences whose translation length is too short. The specific calculation formula is as follows:

$\begin{matrix} BLEU = BP \times exp (\sum_{n = 1}^{N} w_{n} log p_{n}), BP = {\begin{matrix} 1 & ifc > r \\ e^{(1 - \frac{r}{c})} & ifc \leq r \end{matrix} \\ p_{n} = \frac{\sum_{C \in {Candidates}} \sum_{n - grame} {Count}_{clip} (n - gram)}{\sum_{C^{'} \in {(Candidates)} \sum_{n - {gram}^{'} \in C^{'}} count (n - {gram}^{'})} \end{matrix}$ (1)

Where, in the first formula, N is the maximum order of the n-gram; w_n is the weight of the specific n-gram in the calculation; in general, N = 4, the average method technique is generally used in different n-source models, i.e., $w_{n} = \frac{1}{N} = \frac{1}{4}$ . In the second formula, r represents the effective length of the reference translation; c represents the actual length of the machine translated translation; if there are multiple possible reference translations, the effective length selection rule is: the closest translation to the machine translation translation length c; in the third formula, Candidates represents the candidate target translation in the translation result; C represents the current target translation. It can be known from the formula that the value of log p_n is negative, and the range of BLEU is between (0, 1), which increases with the improvement of the quality of the translation. However, in practice, it is almost impossible for a translation system to be equal to 1, unless the translation results of the system are formulated as reference translations.

3.2 Machine translation model based on fuzzy theory

(1) Fuzzy AHP method

Experts conduct on-the-spot investigation and investigation of information systems, identify all risk factors affecting system security, construct a hierarchical structure model of information system risk factors, and score the two levels of risk factors affecting information system security by experts. The fuzzy complementary matrix of the system risk factors is used to obtain the fuzzy uniform matrix. The relative weight coefficient of the risk factors to the information system security is determined according to the fuzzy consensus matrix of the system. For the network information system, the fuzzy AHP method uses the 0.1 0.9 scale method to construct the fuzzy complementary judgment matrix A: $A = {(a_{ij})}_{n \times n}, a_{ij} + a_{ji} = 1$ (2)

Among them, a_ij is the scale value in the judgment matrix, indicating the expert’s understanding of the relative importance of the two risk factors (i, j). Based on the fuzzy complementary matrix A, the fuzzy uniform matrix C is obtained by the following formula transformation: $b_{i} = \sum_{j = 1}^{n} a_{ij}, c_{ij} = \frac{b_{i} - b_{j}}{2 n} + 0.5, C = {(c_{ij})}_{n \times n}$ (3)

In order to reflect the difference in the relative importance of each indicator, the weighting factor WI: $W_{i}^{'} = \frac{1}{n} - \frac{1}{2 α} + \frac{1}{n α} \sum_{j = 1}^{n} c_{ij} (i = 1, 2 \dots, n)$ of each risk factor is determined according to the formula. In the formula, α satisfies α = (n - 1)/2, and the larger the, the smaller the difference between the weights; the smaller the, the larger the difference between the weights. In order to highlight the difference in weight between the evaluation indicators, α is the minimum value, i.e. α = (n - 1)/2.

(2) Entropy weight method

If there are an evaluation objects and m risk factors in a network information system, the original data matrix corresponding to the system is: $X = {(x_{ij})}_{m \times n}$ (4)

Where x_ij is the raw data of the j-th evaluation object on the i-th risk factor. The original data matrix X_ij is normalized according to the formula to obtain a normalized matrix: Y = (y_ij) _m×n. The entropy value ei of the i-th risk factor in the system can be obtained by computing the normalized matrix according to the following formula: $\begin{matrix} z_{ij} = \frac{y_{ij}}{\sum_{j = 1}^{n} y_{ij}}, k = \frac{1}{ln n} > 0, e_{i} \\ = - k \sum_{i = 1}^{n} z_{ij} ln z_{ij} (1 \leq i \leq m) \end{matrix}$ (5)

For the i-th risk factor, the greater the difference in risk factors, the greater the evaluation of the information system, but the smaller the entropy value of the corresponding risk factors; conversely, the smaller the evaluation of the information system risk factors, the entropy value is bigger.

(3) Fuzzy theory comprehensive weight method

Because the fuzzy AHP method and the entropy weight method determine the weighting coefficients of the evaluation indicators are subjective and objective, respectively, which results in a large difference in the weight coefficients determined by the two methods for the same evaluation index. The purpose of the game theory comprehensive weight method is to fully consider the characteristics of the subjective and objective weighting methods, and to find the consistency or compromise of the subjective and objective weight values, so that the main and objective weights are greatly diverged. According to the formula, the integrated weight coefficient vector W^T_i of each evaluation index is determined. $W_{i}^{T} = α_{1} W_{i}^{' T} + α_{2} U_{i}^{' T}, i = 1, 2, \dots, m$ (6)

In the formula, $W_{i}^{T}, U_{i}^{T}$ is a weight vector based on the evaluation index determined by the primary and objective weighting methods; α₁ and α₂ are coefficients in which the primary and objective weights are combined.

3.3 Semantic analysis model based on machine translation

The Language Model (LM) is used to calculate the probability that a string s appears as a sentence s = w₁w₂ … w_l. Using the language model to calculate the probability of occurrence of these illegal logical expressions, it is found that their probability of occurrence is much lower than the probability of occurrence of correct logical expressions, so they can be filtered using the language model. The improved English semantic analysis model consists of three parts: SCFG rule set, SCFG derivation probability model and language model. The context-free grammar G’ of the logical expression is also used in the training process to linearize the logical expression.

After the model training is completed, the improved English semantic analytic model for the new natural language sentence s, firstly synchronously parsing and selecting the top k derivation probability generated by the synchronization, and then combining the language model from the k logic Select the optimal result t* in the expression. Select the logical expression with the first k derivation probabilities as the formula: $T = m (\underset{d \in D (G | s)}{k - best} (P (d | s)))$ (7)

The choice of k values is discussed in detail in the experimental section. Finally, for the input natural language sentence s, the optimal result can be expressed as: $t^{*} = \underset{t \in T}{arg max} (P (t) \times P (d | s))$ (8)

Where P (d|s) represents the derivation probability and P (t) represents the language model probability of the logical expression t.

4 Experiment

4.1 Experimental data

The corpus used is a database obtained from the bilingual dictionary “Chinese transliteration of foreign names”. The data contains 37,668 unique English terms and their corresponding official Chinese transliterations. The corpus is from LDC2003E01 (Chinese-English Name Entity Lists version 1.0 beta). The names of the people and the names of the places are simply sorted out to form 78, 1823 pairs of English and Chinese words. This article uses 60,000 pairs of data excerpted according to the principle of equidistance.

This article will be used to translate the GEOQUERY data set into Chinese, including 880 natural language sentences and their corresponding FUNQL logical expressions. All data is manually translated and verified. Because predicates in logical expressions have a special meaning, no predicates are translated.

In the experiment of Chinese-English translation direction (including the application of the phrase structure-based sequence method and the experiment based on the dependency structure ordering method respectively), the bilingual parallel corpus from LDC (Linguistic Data Consortium) was used. This work extracted 1M sentence pairs from the LDC Sino-British news field parallel corpus as a training set. The development set is from NIST OPEN Machine Translation 2002-2005, with a total of 4476 pairs of Chinese and English sentences. The test set is NIST OPEN Machine Translation 2006, with a total of 1664 sentences.

4.2 Experimental methods

(1) Experimental setup

This paper uses the domestic open source statistical machine translation tool NiuTrans to build a phrase-based translation system for experiments. The translation system is configured as follows: word alignment using GIZA++, training ternary grammar language model using KENLM tool; adjusting model parameters based on minimum error rate method; using BLEU-4 as evaluation index of translation system performance. The open source tool GibbsLDA++is used to train and predict the theme model. The parameters use the system defaults and the number of topics is set to 30.

(2) Corpus settings

Parallel corpus in the spoken language field, which is a parallel corpus of official spoken language provided by CWMT09 (50K in size); parallel corpus in the general field, which is from the Institute of Computing Science of the Chinese Academy of Sciences, including political, economic, legal and other fields (scale 350K). The specific statistics are shown in Table 1 below:

Table 1
Corpus settings

Training corpus Sentence logarithm Number of words

Source Target Source Target

Speaking field 50K 50K 317K 356K

General field 16M 16M 358M 396M

Training corpus	Sentence logarithm	Number of words
Speaking field	50K	50K	317K	356K
General field	16M	16M	358M	396M

4.3 Evaluation criteria

The experiment uses a ten-fold cross-validation method to divide the data set into 10 folds, each with 88 sentences. Each time, 10% (792) training, one fold (88 sentences) test, ten experiments were performed, and the average was taken as the final result. This parsing result is considered correct only if the parsing result is exactly the same as the correct result. Statistically parse all logical expressions and resolve the correct number of logical table formulas. Accuracy (P), recall (R) and F (F) are used as evaluation indicators. The calculation formulas of each indicator are as follows:

$\begin{matrix} Accuracy = \frac{Analyze the correct number of natural language sentences}{The number of natural language sentences obtained by all parsing} \\ Recall = \frac{Analyze the correct number of natural language sentences}{The number of natural language sentences in the test set} \\ F = \frac{(β^{2} + 1) \times P \times R}{β^{2} \times P + R} \end{matrix}$ (9)

The two indicators of accuracy and recall rate evaluate the results of the analysis from different aspects. It is meaningless to pursue the accuracy of one indicator separately and ignore the other one. Therefore, the F value is introduced to measure the overall performance of the algorithm. The β in the F value calculation formula is a preference preset value, the value of β is greater than 1 and the emphasis is on the recall rate, and less than 1 is more important than the accuracy, and equal to 1 is equally important. In the experiment, the value of β is 1 and the F value at this time is also called F1 value.

5 Results and discussions

5.1 Bilingual corpus analysis based on machine translation

(1) Baseline system performance

The performance of the three Baseline systems set up in this paper is shown in Table 2 below.

Table 2
Baseline translation system performance (BLEU%)

Translation system Corpus Test set

Baseline 1 General field 56.93

Baseline 2 General field 34.15

Baseline 3 Speaking field 46.97

Translation system	Corpus	Test set
Baseline 1	General field	56.93
Baseline 2	General field	34.15
Baseline 3	Speaking field	46.97

As can be seen from Table 2, the performance of Baseline 1 is better than that of Baseline 3, and the BLEU value is increased by about 10 percentage points. The reason is the difference in the size of the training corpus. The corpus in the general field is larger and contains more translation knowledge. The corpus in the spoken language field is the corpus in the field. The translation knowledge is relatively accurate, but the corpus is small and easy to translate. The problem of unregistered words has resulted in poor translation quality.

This paper also compares the impact of corpus quality on system performance under the same scale of corpus. The experimental results show that the performance of Baseline 2 is significantly lower than that of Baseline 3. This phenomenon indicates that the training corpus, which is similar to the distribution of text topics in the target domain, is superior to the relatively mixed training corpus in the random sampling field. The reason is that the target field usually contains more professional terms and unique language expressions in the field. It is difficult to effectively learn such translation knowledge from bilingual parallel corpora in other fields, resulting in poor translation performance. However, the performance of the general-purpose translation system Baseline 1 still has a huge room for improvement. The reason is that the general-purpose bilingual corpus is mixed in the field and contains more noise data, which affects the statistical data of the translation model and makes the translation knowledge in the domain less.

(2) Parallel sentence pair selection

TopN sentence pairs are extracted from the large-scale general-purpose bilingual corpus, and used as the training corpus of the translation system. The specific experimental results are shown in Fig. 1. In the figure, the abscissa indicates the size of TopN, and the ordinate indicates the translation performance of the corresponding machine translation system in the translation task for the spoken language.

Fig.1

Machine translation system performance.

It can be seen from the figure that the domain-related 1000K sentences can be extracted from the general-purpose bilingual corpus to achieve the performance of Baseline 1. This phenomenon indicates that the training corpus is not scaled when translating tasks for specific fields. The reason is that the general domain corpus is intermixed and contains more noisy data, which affects the statistical data of the translation model, so that the translation knowledge in the domain obtains a smaller translation probability.

5.2 Semantic analysis based on machine translation

In the experiment, the paper compares 10, 20, 40, 80, 160, 320, 640, and 792 sentences as training examples, and 88 sentences as test sets. The analytical results of the model are accurate. The rate, recall rate and F value are shown in Fig. 1.

The experimental results shown in Fig. 2 show that the semantic analysis results are closely related to the training examples. As the training examples increase, the analytical results are gradually improved. With the increase of training training examples, the semantic parser can learn more natural language sentence analysis methods from the training examples, and the natural language sentences that can be correctly parsed gradually increase, so the recall rate increases with the training examples. When using 792 sentences as the training set, the best analytical results were obtained: accuracy of 75.34%, recall rate of 63.53%, and F1 value of 52.17%.

Fig.2

Relationship between accuracy, recall and F value and the number of training examples.

The experiment found that the selection of the n-value of the N-gram language model is very important.

After all the parameters in the model are determined, the number of generations of the translation models M1 to M5 in the GIZA++tool is M1 = 7, M2 = 7, M3 = 5, M4 = 5, M5 = 5, N-language model n = 4, for the first sentence to select the logical expression with the highest k = 5 probabilities as the candidate, the best result of the improved English semantic analysis model on the GEOQUERY data set is; the accuracy rate is 86.72%, the recall rate is 79.21%, the F value is 73.45%.

5.3 Semantic order analysis based on machine translation

More importantly, these two systems reduce the number of sequence operations by more than about 60% compared to systems that use the phrase structure ordering method. This means that the machine translation-based sequencing method not only achieves better translation quality, but also greatly reduces the number of operations. It is foreseeable that the machine-based sequencing method will be more advantageous in terms of time when dealing with larger data. This paper argues that the main reason that machine translation based methods are more suitable for English ordering than the phrase structure based method is that when describing the same sentence, the number of nodes needed to rely on the syntax tree is usually less than the phrase structure syntax tree.

The distribution of τ values before and after the application of the rule set is given in Fig. 4. As shown in Fig. 4, after applying the rule set, the percentage of sentences with a lower τ value (τ<0.8) is reduced, while the percentage of sentences with a higher τ value (τ≥0.8) is increased. In particular, the percentage of τ≥0.9 sentences increased from 26.31% to 30.94%. The average value of τ increased from 0.831 to 0.889. The evaluation results show that after applying the machine translation-based scheduling method proposed in this paper, the Chinese sentences on the training set are generally closer to the English word order.

Fig.3

The performance of the semantic analysis model varies with the value of n.

Fig.4

Distribution of τ between the pair of training sets before and after the Chinese-English ordering rule set based on machine translation.

6 Conclusions

In order to better perform semantic ordering on English machine translation, the article combines fuzzy theory to construct an algorithm model, and analyzes the experimental results through evaluation indicators. The results show that with the increase of training concentration training examples, the semantic parser can learn more natural language sentence analysis methods from the training examples, and the natural language sentences that can be correctly parsed gradually increase, so with the training examples increased recall rate and F value gradually increased. The improved English semantic analysis model obtained the best result on the GEOQUERY data set; the accuracy rate is 86.72%, the recall rate is 79.21%, and the F value is 73.45%. When using 792 sentences as the training set, the best analytical results were obtained: accuracy of 75.34%, recall rate of 63.53%, and F1 value of 52.17%. The evaluation results show that after applying the machine translation-based scheduling method proposed in this paper, the Chinese sentences on the training set are generally closer to the English word order. The experimental results also show that the use of higher precision syntax analyzers can effectively improve the performance of statistical machine translation systems, whether in phrase-based or machine-based translation methods.

References

Evans

J.A.

and Aceves

, Machine translation: mining text for social theory, Molecular Microbiology 42(1) (2016), 539–547(9).

Ataman

, Negri

, Turchi

and Federico

, Linguistically motivated vocabulary reduction for neural machine translation from Turkish to English, Prague Bulletin of Mathematical Linguistics 108(1) (2017), 331–342.

Zhang

, Xiong

, Su

and Hong

, A context-aware recurrent encoder for neural machine translation, IEEE/ACM Transactions on Audio Speech & Language Processing 25(12) (2017), 2424–2432.

Hill

, Cho

, Jean

and Bengio

, The representational geometry of word meanings acquired by neural machine translation models, Machine Translation 31(1–2) (2017), 3–18.

Ankit

, Georg

and Felix

, Improving machine translation through linked data, Prague Bulletin of Mathematical Linguistics 108(1) (2017), 355–366.

Tabari

M.M.R.

, Prediction of river runoff using fuzzy theory and direct search optimization algorithm coupled model, Arabian Journal for Science & Engineering 41(10) (2016), 4039–4051.

Wang

, Wang

, Li

and Wang

, An event-driven plan recognition algorithm based on intuitionistic fuzzy theory, The Journal of Supercomputing 74(12) (2016), 6923–6938.