Abstract
In order to solve the problems faced in the transfer learning of artificial intelligence system modeling technology, a new method of TSK transfer learning fuzzy system was proposed to enhance knowledge transfer. The two key problems of the precursor learning and the post learning of the TSK type transfer fuzzy system were solved by using this method. And also a novel transfer fuzzy clustering method was proposed for solving the problem. At the same time, a post learning mechanism was proposed to enhance the ability of knowledge transfer which effectively improved the performance of the final model. The experimental results showed that the proposed method integrated the transfer clustering and the transfer fuzzy system modeling successfully, making the modeling process of the fuzzy system more intelligent with better learning ability. In addition, the proposed method provided a new research idea for the development of transfer learning in the field of intelligent modeling.
Introduction
The L2-TSK-TFS modeling method is a TSK type transfer learning fuzzy system with knowledge transfer capability which can effectively compensate for the lack of the current scene information by effectively using the existing history knowledge for assisted learning, so as to effectively compensate for the defect of the degradation of the fuzzy system performance caused by the lack of the current scene data information, thereby obtaining a fuzzy system that has better applicability than those learn by using only current data information. However, there are still some problems to be solved and studied for this method among which the most important are the following two: First, the L2-TSK-TFS method directly inherits the knowledge (the precursor parameters) from the source scene (historical scene) when learning precursor of the fuzzy system without learning these precursor parameters, which to some extent, forces the target domain (current domain) to maintain a high degree of similarity with the source domain, but this is contrary to the theory of transfer learning. Second, the learning mechanism of the L2-TSK-TFS method is relatively simple in the post learning of the fuzzy system, and the transfer requires a higher degree of similarity between the source domain and the target domain. Based on the above issues, a scientific problem concerning the L2-TSK-TFS method has been concisely formulated. That is, how to solve the transfer problem of the precursor parameters and improve the transfer learning ability of the post parameters is a problem that needs to be solved in the L2-TSK-TFS method. In order to solve this problem, the L2-TSK-TFS method is to be improved from two aspects, and a TSK transfer learning fuzzy system with enhanced knowledge transfer capability will be proposed, which is called IKT-TSK-TFS.
Literature review
Since it was proposed in 1995, the study of transfer learning has attracted more and more attention. There are different opinions on transfer learning, among which representative expositions are: learning guided learning, lifelong learning, and knowledge transfer, inductive transfer, multi-task learning, consolidation of knowledge, context-sensitive learning, knowledge-based inductive learning, meta-learning, incremental/cumulative learning, etc. [1, 2]. Among the various opinions, a technology closely related to transfer learning is a multitask-based learning framework that attempts to learn multiple tasks at the same time. The traditional practice of this learning method is to learn the common parts of multitasking, and then guide each independent individual’s learning [3].
In 2005, a related announcement concerning information processing technology released by the Defense Advanced Research Projects Agency’s gave a new definition to transfer learning: A system should learn from the accumulated experience and knowledge of past tasks when performing new tasks. In this definition, transfer learning is used to extract historical knowledge from one or more source tasks and apply this knowledge to new target tasks [4, 5]. This approach is different from multi-task learning. In multi-task learning, it emphasizes the simultaneous learning of source and target tasks, while the transfer learning theory focuses more on the learning of the target task. It makes the proportion of the source task and the target task in the transfer learning no longer equal [6, 7].
The current transfer learning theory has attracted attention in terms of classification, clustering, and regression. Looking at the progress in the field of transfer learning in recent years, it can be found that it has the most extensive research in the field of supervised classification. Representative work includes: The domain-adapted transfer learning method was successfully applied to the classification of large-scale emotional data. Duan et al. successfully integrated the transfer learning theory with multi-core learning and applied it to the classification field [8]. In the aspect of unsupervised clustering, there are still few related methods although transfer learning has received some attention. Related scholars proposed a transfer spectral clustering algorithm based on spectral clustering model as well as transfer learning theory which can also be used as a multitasking algorithm.
At present, a large amount of research work on transfer learning technology focuses on machine learning methods based on probability theory. But for another branch of machine learning method, i.e., the machine learning method based on fuzzy theory, there is less related research work on the use of transfer learning technology. This article will discuss how to introduce the transfer learning technology to the classic fuzzy machine learning method to make the fuzzy machine learning process becomes more intelligent. Specifically, we will mainly carry out related work in two aspects: fuzzy identification methods, that is, fuzzy clustering technology and fuzzy intelligent modeling methods, namely, fuzzy system modeling techniques.
Establishment of TSK transfer learning system
The learning algorithm for the transfer learning fuzzy system
The learning algorithm for the transfer learning fuzzy system
A comparison of generalization performance (J) of methods on simulated data sets
Transfer learning mechanism adopted by L2-TSK-TFS and IKT-TSK-TFS methods respectively.
A comparison of generalization performance of various fuzzy systems on simulated data sets.
The working framework for the L2-TSK-TFS method as well as the IKT-TSK-TFS method is shown in Fig. 1. From Fig. 1, we can clearly see the differences between the two methods in the process of knowledge transfer. In addition, the deficiency of the L2-TSK-TFS method can also be found.
IKT-TSK-TF arithmetic statement
Here, the learning algorithm for TSK transfer learning fuzzy system (IKT-TSK-TFS) aiming at enhancing knowledge transfer is presented. The specific steps are shown in Table 1.
Analysis of experimental results
Analysis of simulated data experiment
Table 2 shows the experimental results of all the methods on the simulated dataset. Figure 2 shows the data modeling effect of all the methods under the
First, both the results of Table 2 and Fig. 2 show that the generalization performance of the IKT-TSK-TFS method is the best.
Second, the results of graph Fig. 2a–c show that no matter how the data sets are combined, the traditional L2-TSK-FS modeling method that has no learning-transfer ability cannot effectively compensate for the missing information to obtain satisfactory performance.
Third, although the HiRBF method is a learning-transfer method, the results of Fig. 2d show that this method cannot fill the missing part of the information by using transfer learning. It is due to the fact that the method does not use any knowledge to assist learning during transfer learning, and its learning process is still dependent on the data itself, resulting in the poor performance of the method in the face of a data missing scenarios.
Fourth, the results of Fig. 2e and c show the excellent performance of the L2-TSK-TFS method and the IKT-TSK-TFS method in the face of data missing scenarios. This is because the above two methods both adopt the theory of knowledge transfer through the summarization of historical knowledge and effective usage in assisting learning in the current field, and using balance parameters to control the learning process, and thus get better generalization performance.
Fifth, the experimental results of the L2-TSK-TFS method and the IKT-TSK-TFS method are compared. Although it is difficult to see which is more advantageous from the effect graphs on Fig. 2e and c due to the excellent performance of both, the data value result of Table 2 clearly shows that the IKT-TSK-TFS method is more advantageous. This proves that the IKT-TSK-TFS method is feasible for the precursor transfer learning strategy and the post enhanced transfer learning strategy. The two strategies are an effective improvement on the L2-TSK-TFS method.
Experimental analysis of real data
A comparison of performance (J0) of each method on real data
A comparison of performance (J0) of each method on real data
Table 3 shows that the experimental results of each method on the real fermentation data sets are similar to those obtained from the simulated data sets. This experimental result further proves that the IKT-TSK-TFS method is more advantageous when faced with data missing scenarios compared to other fuzzy modeling methods. Because its learning-transfer mechanism can complete compensation for missing information in the modeling learning process and get better generalization performance. In addition, by comparing with the L2-TSK-TFS method, it is proved again that the two kinds of reinforcement learning strategies proposed are effective and feasible for the precursor and post parameters of the fuzzy system.
A new TSK transfer learning fuzzy system approach is proposed to enhance knowledge transfer. This method can solve the two key issues of precursor learning and post learning that face the TSK-type migration fuzzy system. And a novel transfer fuzzy clustering method is proposed to solve the precursor transfer learning problem of the fuzzy system. At the same time, a post learning mechanism that can enhance knowledge transfer ability is proposed to effectively improve the performance of the final model. The experimental results show that the proposed method successfully integrates the transfer clustering and transfer fuzzy system modeling, which makes the modeling process of the fuzzy system more intelligent with better learning ability. In addition, the proposed method also provides a new research idea for the development of transfer learning in the field of intelligent modeling.
Footnotes
Acknowledgments
This work was supported by the Scientific and Technological Research Key Project of Henan Provincial Education Office (Grant No. 17A520061), the Scientific and Technological Project of Henan Provincial Science and Technology Office (Grant No. 172102210077).