Computer vision and deep learning–based data anomaly detection method for structural health monitoring

Abstract

The widespread application of sophisticated structural health monitoring systems in civil infrastructures produces a large volume of data. As a result, the analysis and mining of structural health monitoring data have become hot research topics in the field of civil engineering. However, the harsh environment of civil structures causes the data measured by structural health monitoring systems to be contaminated by multiple anomalies, which seriously affect the data analysis results. This is one of the main barriers to automatic real-time warning, because it is difficult to distinguish the anomalies caused by structural damage from those related to incorrect data. Existing methods for data cleansing mainly focus on noise filtering, whereas the detection of incorrect data requires expertise and is very time-consuming. Inspired by the real-world manual inspection process, this article proposes a computer vision and deep learning–based data anomaly detection method. In particular, the framework of the proposed method includes two steps: data conversion by data visualization, and the construction and training of deep neural networks for anomaly classification. This process imitates human biological vision and logical thinking. In the data visualization step, the time series signals are transformed into image vectors that are plotted piecewise in grayscale images. In the second step, a training dataset consisting of randomly selected and manually labeled image vectors is input into a deep neural network or a cluster of deep neural networks, which are trained via techniques termed stacked autoencoders and greedy layer-wise training. The trained deep neural networks can be used to detect potential anomalies in large amounts of unchecked structural health monitoring data. To illustrate the training procedure and validate the performance of the proposed method, acceleration data from the structural health monitoring system of a real long-span bridge in China are employed. The results show that the multi-pattern anomalies of the data can be automatically detected with high accuracy.

Keywords

Structural heath monitoring data anomaly detection computer vision deep learning stacked autoencoder deep neural network

Get full access to this article

View all access options for this article.

References

Inman

Farrar

Vicente

et al . Damage prognosis: for aerospace, civil and mechanical systems. Chichester: John Wiley & Sons, 2005.

Sohn

Farrar

Hemez

et al . A review of structural health monitoring literature: 1996–2001. Los Alamos, NM: Los Alamos National Laboratory, 2004.

Liu

et al . SMC structural health monitoring benchmark problem using monitored data from an actual cable-stayed bridge. Struct Control Hlth 2014; 21: 156–172.

JP.

The state of the art in structural health monitoring of cable-stayed bridges. J Civ Struct Health Monit 2016; 6: 43–67.

Zhang

et al . Research and practice of health monitoring for long-span bridges in the mainland of China. Smart Struct Syst 2015; 15: 555–576.

Wong

Xia

. Health checks through landmark bridges to sky-high structures. Adv Struct Eng 2011; 14: 103–119.

Dalvi

Norouzi

Hunt

et al . Structural health monitoring system of Ironton-Russell Bridge during substructure construction. In: SPIE 9805, proceedings of the smart structures and materials + nondestructive evaluation and health monitoring, Las Vegas, NV, 20–24 March 2016. Bellingham, WA: SPIE.

Huang

Duan

Zhu

et al . Study on deformation monitoring of subway station deep foundation construction. Appl Mech Mater 2016; 847: 425–430.

Park

Kim

Cho

. Framework of automated construction-safety monitoring using cloud-enabled BIM and BLE mobile tracking sensors. J Constr Eng M 2016; 143: 05016019.

10.

Worden

Dulieu-Barton

. An overview of intelligent fault detection in systems and structures. Struct Health Monit 2004; 3: 85–98.

11.

Henry

Clarke

. The self-validating sensor: rationale, definitions and examples. Control Eng Pract 1993; 1: 585–610.

12.

Feng

Wang

Shida

. A review of self-validating sensor technology. Sensor Rev 2007; 27: 48–56.

13.

Shen

Wang

. Failure detection, isolation, and recovery of multifunctional self-validating sensor. IEEE T Instrum Meas 2012; 61: 3351–3362.

14.

Spencer

Park

J-W

Mechitov

et al . Next generation wireless smart sensors toward sustainable civil infrastructure. Procedia Engineer 2017; 171: 5–13.

15.

Lin

Zhang

Zhou

et al . ATPC: adaptive transmission power control for wireless sensor networks. In: Proceedings of the 4th international conference on embedded networked sensor systems, Boulder, CO, 31 October–3 November 2006, pp. 223–236. New York: ACM.

16.

Rondinone

Ansari

Riihijärvi

et al . Designing a reliable and stable link quality metric for wireless sensor networks. In: Proceedings of the workshop on real-world wireless sensor networks, Glasgow, 1 April 2008, pp. 6–10. New York: ACM.

17.

Nagayama

Moinzadeh

Mechitov

et al . Reliable multi-hop communication for structural health monitoring. Smart Struct Syst 2010; 6: 481–504.

18.

Barnett

Lewis

. Outliers in statistical data. Hoboken, NJ: John Wiley & Sons, 1978.

19.

Johnson

Wichern

. Applied multivariate statistical analysis. London: Pearson PLC, 2002.

20.

Yang

Peter

. Development of an advanced noise reduction method for vibration analysis based on singular value decomposition. NDT&E Int 2003; 36: 419–432.

21.

Žvokelj

Zupan

Prebil

. Non-linear multivariate and multiscale monitoring and signal denoising strategy using Kernel Principal Component Analysis combined with Ensemble Empirical Mode Decomposition method. Mech Syst Signal Pr 2011; 25: 2631–2653.

22.

Calabrese

Campanella

Proverbio

. Noise removal by cluster analysis after long time AE corrosion monitoring of steel reinforcement in concrete. Constr Build Mater 2012; 34: 362–371.

23.

Katicha

Flintsch

Bryce

et al . Wavelet denoising of TSD deflection slope measurements for improved pavement structural evaluation. Comput-Aided Civ Inf 2014; 29: 399–415.

24.

Jiang

Mahadevan

Adeli

. Bayesian wavelet packet denoising for structural system identification. Struct Control Hlth 2007; 14: 333–356.

25.

Yang

Nagarajaiah

. Blind denoising of structural vibration responses with outliers via principal component pursuit. Struct Control Hlth 2014; 21: 962–978.

26.

Kramer

. Autoassociative neural networks. Comput Chem Eng 1992; 16: 313–328.

27.

Tamilselvan

Wang

. Failure diagnosis using deep belief learning based health state classification. Reliab Eng Syst Safe 2013; 115: 124–135.

28.

Wei

. Clustering of vehicular cable tension of cable-stayed bridge under normal operation conditions. In: Proceedings of the 10th international workshop on structural health monitoring (IWSHM 2015), Stanford, CA, 1–3 September 2015.

29.

Sanchez

R-V

Zurita

et al . Multimodal deep support vector classification with homologous features and its application to gearbox fault diagnosis. Neurocomputing 2015; 168: 119–127.

30.

Wai

et al . A vision-based system for dynamic displacement measurement of long-span bridges: algorithm and verification. Smart Struct Syst 2013; 12: 363–379.

31.

Koch

Georgieva

Kasireddy

et al . A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure. Adv Eng Inform 2015; 29: 196–210.

32.

Cha

Y-J

Choi

Buyukozturk

. Deep learning-based crack damage detection using convolutional neural network. Comput-Aided Civ Inf 2017; 32: 361–378.

33.

Yang

Dorn

Mancini

et al . Blind identification of full-field vibration modes from video measurements with phase-based video motion magnification. Mech Syst Signal Pr 2017; 85: 567–590.

34.

Yeum

Choi

Dyke

. Autonomous image localization for visual inspection of civil infrastructure. Smart Mater Struct 2017; 26: 035051.

35.

Bengio

Lamblin

Popovici

et al . Greedy layer-wise training of deep networks. Adv Neur In 2007; 19: 153–160.

36.

LeCun

Bengio

Hinton

. Deep learning. Nature 2015; 521: 436–444.

37.

Hinton

Osindero

Teh

Y-W

. A fast learning algorithm for deep belief nets. Neural Comput 2006; 18: 1527–1554.

38.

Møller

. A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks 1993; 6: 525–533.