Differential fusion of multi-latent-layer temporal features for industrial process fault classification

Abstract

Industrial process data often possess characteristics such as time series correlation, high dimensionality, and noise. A fault classification method based on the Long Short-Term Memory (LSTM) combined with a Variational Autoencoder (VAE) model (LSTM-VAE) integrates the advantages of LSTM in handling long time series and the VAE in anomaly detection. However, when extracting features, the method mainly focuses on directly using the time series processed through sliding time steps to extract features via the LSTM network, which may lead to neglect or minor influence of abnormal signals during the generation of latent variables from long time series. To address these issues, this paper proposes a Difference Fusion Multi-Latent-Layer Temporal Feature (DFMLF) extraction method. The method calculates the latent variables by weighting the differences of input time series with the hidden states of the LSTM-VAE network to enhance the VAE’s ability to construct features. To further extract features from the generated latent variables that still have temporal characteristics, gated recurrent units are utilized. To prevent information loss, the latent variables before and after modeling are concatenated in dimensions and classified using a Convolutional Neural Networks. This method was evaluated on the Tennessee Eastman process and a real three-phase flow process, comparing it with other six different models. The results validate the effectiveness of the proposed model.

Keywords

Variational autoencoder long short-term memory differential fusion deep learning process fault classification

Get full access to this article

View all access options for this article.

References

Ashraf

Anowar

Setu

, et al. (2023) A survey on dimensionality reduction techniques for time-series data. IEEE Access 11: 42909–42923.

Chai

Zhao

(2019) Enhanced random forest with concurrent analysis of static and dynamic nodes for industrial fault classification. IEEE Transactions on Industrial Informatics 16(1): 54–66.

Chen

Duan

, et al. (2022) Semisupervised anomaly detection of multivariate time series based on a variational autoencoder. Applied Intelligence 53(5): 6074–6098.

Chung

Gulcehre

Cho

, et al. (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. Available at: http://arxiv.org/abs/1412.3555 (accessed 19 July 2025).

Chung

Kastner

Dinh

, et al. (2015) A recurrent latent variable model for sequential data. Available at: https://arxiv.org/abs/1506.02216 (accessed 19 July 2025).

Kano

Nakagawa

(2008) Data-based process monitoring, process control, and quality improvement: Recent developments and applications in steel industry. Computers & Chemical Engineering 32(1–2): 12–24.

Kingma

Welling

(2013) Auto-encoding variational Bayes. Available at: http://arxiv.org/abs/1312.6114 (accessed 19 July 2025).

Zhu

Song

, et al. (2025) A novel label-aware global graph construction method and spiking-coded graph neural network for intelligent process fault diagnosis. Neurocomputing 611: 128707.

Lin

Clark

Birke

, et al. (2020) Anomaly detection for time series using VAE-LSTM hybrid model. In: ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), Barcelona, 4–8 May, pp. 4322–4326. New York: IEEE.

10.

Liu

Bai

(2022) Fault diagnosis using dynamic principal component analysis and GA feature selection modeling for industrial processes. Processes 10(12): 2570.

11.

Maged

Lui

Haridy

, et al. (2024) Variational autoencoders-LSTM based fault detection of time-dependent high dimensional processes. International Journal of Production Research 62(4): 1092–1107.

12.

Malhotra

Vig

Shroff

, et al. (2015) Long short-term memory networks for anomaly detection in time series. In: European symposium on artificial neural networks, computational intelligence and machine learning (ESANN 2015), Bruges, Belgium, 22–24 April, pp. 89–94. Bruges, Belgium: i6doc.com Publishing.

13.

Niu

(2020) LSTM-based VAE-GAN for time-series anomaly detection. Sensors 20(13): 3738.

14.

Park

Hoshi

Kemp

(2018) A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder. IEEE Robotics and Automation Letters 3(3): 1544–1551.

15.

Ren

Fang

, et al. (2023) Two-view LSTM variational auto-encoder for fault detection and diagnosis in multivariable manufacturing processes. Neural Computing and Applications 35(29): 22007–22026.

16.

Ruiz-Cárcel

Cao

Mba

, et al. (2015) Statistical process monitoring of a multiphase flow facility. Control Engineering Practice 42: 74–88.

17.

Sun

Lin

, et al. (2023) Anomaly detection of power battery pack using gated recurrent units based variational autoencoder. Applied Soft Computing 132: 109903.

18.

Yang

Wang

Qian

, et al. (2025) Fault diagnosis of multi-unit nonlinear industrial processes based on a global–local attention convolutional neural network. Engineering Applications of Artificial Intelligence 159: 111747.

19.

Yin

Ding

Haghani

, et al. (2012) A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process. Journal of Process Control 22(9): 1567–1581.

20.

Yin

Ding

Xie

, et al. (2014) A review on basic data-driven approaches for industrial process monitoring. IEEE Transactions on Industrial Electronics 61(11): 6418–6428.

21.

Liu

(2020) Convolutional long short-term memory autoencoder-based feature learning for fault detection in industrial processes. IEEE Transactions on Instrumentation and Measurement 70: 1–15.

22.

Yuan

Lin

(2016) Fault diagnosis and remaining useful life estimation of aero engine using LSTM neural network. In: 2016 IEEE international conference on aircraft utility systems (AUS), Beijing, China, 10–12 October, pp. 135–140. New York: IEEE.

23.

Zhang

Qiao

, et al. (2024) A teacher–student strategy specific to transformer for machine fault diagnosis. Structural Health Monitoring. Epub ahead of print 25 December. DOI: 10.1177/14759217241302845.

24.

Zhang

, et al. (2025) A temporal-aware dual-attention network for fault diagnosis in industrial processes. Control Engineering Practice 164: 106468.

25.

Zhang

Yang

Wang

(2019) Data-driven methods for predictive maintenance of industrial equipment: A survey. IEEE Systems Journal 13(3): 2213–2227.

26.

Zhang

Zhou

Huang

, et al. (2021) Fault diagnosis of rotating machinery based on recurrent neural networks. Measurement 171: 108774.

27.

Zhou

Song

, et al. (2016) Autoregressive dynamic latent variable models for process monitoring. IEEE Transactions on Control Systems Technology 25(1): 366–373.

28.

Zhou

Ren

(2020) Probabilistic weighted copula regression model with adaptive sample selection strategy for complex industrial processes. IEEE Transactions on Industrial Informatics 16(11): 6972–6981.

29.

Zhu

Jiang

Liu

(2021) Fault detection and diagnosis in industrial processes with variational autoencoder: A comprehensive study. Sensors 22(1): 227.

30.

Zhu

Zhang

, et al. (2025a) Design of model fusion learning method based on deep bidirectional GRU neural network in fault diagnosis of industrial processes. Chemical Engineering Science 302: 120884.

31.

Zhu

Zhang

Gao

(2025b) Industrial process fault diagnosis using dilated convolutional stacking bidirectional gated recurrent unit with high and low-order feature fusion. Chemical Engineering Science 305: 121164.