An undercarriage image driven anomaly detection method for metro vehicle based on adversarial memory enhancement

Abstract

Anomaly detection is essential to ensure metro vehicles' safe operation. Error reconstruction-based anomaly detection methods have been widely studied because they only need to be trained by normal data and do not require much anomaly data, which is challenging to obtain. However, sometimes the auto-encoder network for error reconstructing “generalizes” so well that it also rebuilds the anomaly well, leading to missed anomaly detection. Therefore, this paper proposes an undercarriage image-driven anomaly detection method for metro vehicles based on adversarial memory enhancement. Firstly, this study performs component segmentation based on YOLOv5 detection results and constructs a component anomaly detection dataset. Secondly, an anomaly detection method based on memory enhancement and adversarial training of encoding-decoding-encoding structure is proposed for component anomaly detection. It enables the auto-encoder to reconstruct the image better. Thirdly, the combined indicator of the difference between potential features and reconstruction error is used as an anomaly indicator for anomaly detection of metro components, reducing the rate of fault misses. The experimental results on the established dataset demonstrate that the proposed method reduces false negative rates of 92.4%, 92.6%, 74.6%, and 59.1% compared with ${A E}_{L_{2}}$ , ${A E}_{S S I M}$ , GANomaly, and MemAE, respectively.

Keywords

Metro vehicle anomaly detection convolutional autoencoder memory enhancement adversarial training

Get full access to this article

View all access options for this article.

References

Yang

Chen

, et al. Defect detection of axle box cover device fixing bolts in metro based on convolutional neural network. In: 2020 39th Chinese Control Conference (CCC), Shengyang, China, July 27-29, 2020, IEEE, pp. 7504–7509.

Zhou

, et al. Freight car target detection in a complex background based on convolutional neural networks. Proc Inst Mech Eng F J Rail Rapid Transit 2019; 233: 298–311.

Zhang

Liu

Chen

, et al. Real-time vision-based system of fault detection for freight trains. IEEE Trans Instrum Meas 2020; 69: 5274–5284.

Zhang

Wang

Liu

, et al. An automatic fault detection method of freight train images based on BD-YOLO. IEEE Access 2022; 10: 39613–39626.

Wiley

Lucas

. Computer vision and image processing: a paper review. J Artif Intell Res 2018; 2(1): 29–36.

Zhang

Tang

Deng

, et al. A fault diagnosis method for wind turbines gearbox based on adaptive loss weighted meta-ResNet under noisy labels. Mech Syst Signal Process 2021; 161: 107963.

Liu

Qiang

, et al. The application of deep learning in computer vision. In: 2017 Chinese automation congress (CAC), Jinan, China, October, 20 - 22, 2017.

Ren

Girshick

, et al. Faster r-cnn: towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 2015; 28.

Redmon

Divvala

Girshick

, et al. You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, June 26-July 1, 2016, pp. 779–788.

10.

Redmon

FarhadiYOLO9000

: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, July 21-26, 2017, pp. 7263–7271.

11.

Redmon

Farhadi

. Yolov3: an incremental improvement. arXiv preprint arXiv:180402767 2018.

12.

Bochkovskiy

Wang

C-Y

Liao

H-YM

. Yolov4: optimal speed and accuracy of object detection. arXiv preprint arXiv:200410934 2020.

13.

Sun

Xiao

Xie

. Automatic multi-fault recognition in TFDS based on convolutional neural network. Neurocomputing 2017; 222: 127–136.

14.

Nan

Gao

. Automated visual inspection of multipattern train components using gradient information and feature fusion under the illumination-variant condition. Proc Inst Mech Eng F J Rail Rapid Transit 2018; 232: 1500–1513.

15.

Ehret

Davy

Morel

J-M

, et al. Image anomalies: a review and synthesis of detection methods. J Math Imag Vis 2019; 61: 710–743.

16.

Chen

Zhou

Huang

. One-class SVM for learning in image retrieval. In: Proceedings 2001 international conference on image processing (Cat NoIEEE 01CH37205), Thessaloniki, Greece, October 7-10, 2001, pp. 34–37.

17.

Chalapathy

Menon

Chawla

. Anomaly detection using one-class neural networks. arXiv preprint arXiv:180206360 2018.

18.

Zhang

Tang

Deng

, et al. Fault detection of wind turbines by subspace reconstruction-based robust kernel principal component analysis. IEEE Trans Instrum Meas 2021; 70: 1–11.

19.

Chalapathy

Chawla

. Deep learning for anomaly detection: a survey. arXiv preprint arXiv:190103407 2019.

20.

Zhuang

Shen

Tan

, et al. Structured binary neural networks for accurate image classification and semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Long Beach, CA, USA, June 15-20, 2019, pp. 413–422.

21.

Ruff

Vandermeulen

Goernitz

, et al. Deep one-class classification. In: International conference on machine learning, Macao, China, February 26-28, 2018, pp. 4393–4402.

22.

Zong

Song

Min

, et al. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In: International conference on learning representations, Vancouver, BC, Canada, April 30-May 3, 2018.

23.

Wang

Zhang

Tsui

K-L

. A deep generative approach for rail foreign object detections via semi-supervised learning. IEEE Trans Ind Inf 2023; 19: 459–468.

24.

Bergmann

Löwe

Fauser

, et al. Improving unsupervised defect segmentation by applying structural similarity to autoencoders. arXiv preprint arXiv:180702011 2018.

25.

Tian

Pang

Liu

, et al. Unsupervised anomaly detection in medical images with a memory-augmented multi-level cross-attentional masked autoencoder. arXiv preprint arXiv:220311725 2022.

26.

Gong

Liu

, et al. Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In: 2019 IEEE/cvf international conference on computer vision (ICCV), Seoul, South Korea, October, 20-26, 2020.

27.

Park

Noh

Ham

. Learning memory-guided normality for anomaly detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Seattle, USA, June 13-19, 2020, pp. 14372–14381.

28.

Liu

Song

Feng

, et al. Semi-supervised anomaly detection with dual prototypes autoencoder for industrial surface inspection. Opt Laser Eng 2021; 136: 106324.

29.

Goodfellow

Pouget-Abadie

Mirza

, et al. Generative adversarial networks. Commun ACM 2020; 63: 139–144.

30.

Akcay

Atapour-Abarghouei

Breckon

. Ganomaly: semi-supervised anomaly detection via adversarial training. Asian conference on computer vision. Perth, Australia, December 2–6: Springer, 2018, pp. 622–637.

31.

Creswell

White

Dumoulin

, et al. Generative adversarial networks: an overview. IEEE Signal Process Mag 2018; 35: 53–65.

32.

Radford

Metz

Chintala

. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:151106434 2015.