Damage detection is one of the most important tasks for structural health monitoring of civil infrastructure. Before a damage detection algorithm can be applied, the integrity of the data must be ensured; otherwise results may be misleading or incorrect. Indeed, sensor system malfunction, which results in anomalous data (often called faulty data), is a serious problem, as the sensors usually must operate in extremely harsh environments. Identifying and eliminating anomalies in the data is crucial to ensuring that reliable monitoring results can be achieved. Because of the vast amounts of data typically collected by a structural health monitoring system, manual removal of the anomalous data is prohibitive. Machine learning methods have the potential to automate the process of data anomaly detection. Although supervised methods have been proven to be effective for detecting data anomalies, two unresolved challenges reduce the accuracy of anomaly detection: (1) the class imbalance and (2) incompleteness of anomalous patterns of training dataset. Unsupervised methods have the potential to address these challenges, but improvements are required to deal with vast amounts of monitoring data. In this article, the generative adversarial networks are combined with a widely applied unsupervised method, that is, autoencoders, to improve the performance of existing unsupervised learning methods. In addition, the time-series data are transformed to Gramian Angular Field images so that advanced computer vision methods can be included in the network. Two structural health monitoring datasets from a full-scale bridge, including examples of anomalous data caused by sensor system malfunctions, are utilized to validate the proposed methodology. Results show that the proposed methodology can successfully identify data anomalies with good accuracy and robustness, hence can overcome one of the key difficulties in achieving automated structural health monitoring.
GhoshalASundaresanMJSchulzMJ, et al. Structural health monitoring techniques for wind turbine blades. J Wind Eng Ind Aerod2000; 85(3): 309–324.
2.
KoJNiYQ.Technology developments in structural health monitoring of large-scale bridges. Eng Struct2005; 27(12): 1715–1725.
3.
SpencerBRuiz-SandovalMEKurataN.Smart sensing technology: opportunities and challenges. Struct Control Hlth2004; 11(4): 349–368.
4.
WanHPNiYQ.Bayesian multi-task learning methodology for reconstruction of structural health monitoring data. Struct Health Monit2019; 18(4): 1282–1309.
5.
MagalhãesFCunhaACaetanoE.Vibration based structural health monitoring of an arch bridge: from automated OMA to damage detection. Mech Syst Signal Pr2012; 28: 212–228.
6.
MaoJXWangHFengDM, et al. Investigation of dynamic properties of long-span cable-stayed bridges based on one-year monitoring data under normal operating condition. Struct Control Hlth2018; 25(5): e2146.
7.
YiTHHuangHBLiHN.Development of sensor validation methodologies for structural health monitoring: a comprehensive review. Measurement2017; 109: 200–214.
8.
AbdelghaniMFriswellMI.Sensor validation for structural systems with multiplicative sensor faults. Mech Syst Signal Pr2007; 21(1): 270–279.
9.
ChangC-MChouJ-Y.Damage detection of seismically excited buildings based on prediction errors. J Aerospace Eng2018; 31(4): 04018032.
10.
KangFLiJDaiJ.Prediction of long-term temperature effect in structural health monitoring of concrete dams using support vector machines with Jaya optimizer and salp swarm algorithms. Adv Eng Softw2019; 131: 60–76.
11.
MoriotJQuaegebeurNLe DuffA, et al. A model-based approach for statistical assessment of detection and localization performance of guided wave–based imaging techniques. Struct Health Monit2018; 17(6): 1460–1472.
12.
WangHZhangYMMaoJX, et al. Modeling and forecasting of temperature-induced strain of a long-span bridge using an improved Bayesian dynamic linear model. Eng Struct2019; 192: 220–232.
13.
ThiyagarajanKKodagodaSVan NguyenL. Predictive analytics for detecting sensor failure using autoregressive integrated moving average model. In: 2017 12th IEEE conference on industrial electronics and applications (ICIEA), Siem Reap, Cambodia, 18–20 June 2017.
14.
AbdelghaniMFriswellMI.Sensor validation for structural systems with additive sensor faults. Struct Health Monit2004; 3(3): 265–275.
15.
WanHPNiYQ.Bayesian modeling approach for forecast of structural stress response using structural health monitoring data. J Struct Eng2018; 144(9): 04018130.
16.
ChandolaVBanerjeeAKumarV.Anomaly detection: a survey. ACM Comput Surv2009; 41(3): 15.
17.
KhanSYairiT.A review on the application of deep learning in system health management. Mech Syst Signal Pr2018; 107: 241–265.
18.
BaoYQChenZCWeiS, et al. The state of the art of data science and engineering in structural health monitoring. Engineering2019; 5(2): 234–242.
19.
FuYGPengCGomezF, et al. Sensor fault management techniques for wireless smart sensor networks in structural health monitoring. Struct Control Hlth2019; 26(7): e2362.
20.
MazurowskiMAHabasPAZuradaJM, et al. Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance. Neural Networks2008; 21(2–3): 427–436.
21.
BaoYQTangZYLiH, et al. Computer vision and deep learning–based data anomaly detection method for structural health monitoring. Struct Health Monit2018; 18(2): 401–421.
22.
TangZYChenZCBaoYQ, et al. Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring. Struct Control Hlth2019; 26(1): e2296.
23.
RabatelJBringaySPonceletP.Anomaly detection in monitoring sensor data for preventive maintenance. Expert Syst Appl2011; 38(6): 7003–7015.
24.
KullaaJ.Sensor validation using minimum mean square error estimation. Mech Syst Signal Pr2010; 24(5): 1444–1457.
25.
SoualhiAClercGRazikH.Detection and diagnosis of faults in induction motor using an improved artificial ant clustering technique. IEEE T Ind Electron2012; 60(9): 4053–4062.
26.
ZwetslootIMWoodallWHA review of some sampling and aggregation strategies for basic statistical process monitoring. J Qual Technol. Epub ahead of print 12July2019. DOI: 10.1080/00224065.2019.1611354.
27.
SunBLuhPBJiaQS, et al. Building energy doctors: an SPC and Kalman filter-based method for system-level fault detection in HVAC systems. IEEE T Autom Sci Eng2013; 11(1): 215–229.
28.
PathirageCSNLiJLiL, et al. Structural damage identification based on autoencoder neural networks and deep learning. Eng Struct2018; 172: 13–28.
29.
YildirimOSan TanRAcharyaUR.An efficient compression of ECG signals using deep convolutional autoencoders. Cogn Syst Res2018; 52: 198–211.
30.
CozzolinoDVerdolivaL.Single-image splicing localization through autoencoder-based anomaly detection. In: 2016 IEEE international workshop on information forensics and security (WIFS), Abu Dhabi, United Arab Emirates, 4–7 December 2016.
31.
YamashitaRNishioMDoRKG, et al. Convolutional neural networks: an overview and application in radiology. Insights Imaging2018; 9(4): 611–629.
32.
SeyfioğluMSÖzbayoğluAMGürbüzSZ.Deep convolutional autoencoder for radar-based classification of similar aided and unaided human activities. IEEE T Aero Elec Sys2018; 54(4): 1709–1723.
33.
BaurCWiestlerBAlbarqouniS, et al. Deep autoencoding models for unsupervised anomaly segmentation in brain MR images. In: International MICCAI Brainlesion Workshop, Granada, 16September2018.
34.
OhDYunI.Residual error based anomaly detection using auto-encoder in SMD machine sound. Sensors2018; 18(5): 1308.
35.
GoodfellowIPouget AbadieJMirzaM, et al. Generative adversarial nets. In: GhahramaniZWellingMCortesC, et al. (eds) Advances in neural information processing systems. Vancouver, BC, Canada: Neural Information Processing Systems Foundation, 2014, pp. 2672–2680.
36.
RadfordAMetzLChintalaS.Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.
37.
XiaYCaoXWenF, et al. Learning discriminative reconstructions for unsupervised outlier removal. In: Proceedings of the IEEE international conference on computer vision, Santiago, 7–13 December 2015.
38.
WangHHuRMXieJ, et al. Comparative study on buffeting performance of Sutong Bridge based on design and measured spectrum. J Bridge Eng2012; 18(7): 587–600.
39.
LucasJMCrosierRB.Robust CUSUM: a robustness study for CUSUM quality control schemes. Commun Stat: Theor M1982; 11(23): 2669–2687.
40.
MontgomeryDC.Introduction to statistical quality control. Hoboken, NJ: John Wiley & Sons, 2007.
41.
MaoJXWangHFuYG, et al. Automated modal identification using principal component and cluster analysis: application to a long-span cable-stayed bridge. Struct Control Hlth2019; 26(10): e2430.
42.
MaoJXWangHLiJ.Fatigue reliability assessment of a long-span cable-stayed bridge based on one-year monitoring strain data. J Bridge Eng2018; 24(1): 05018015.
43.
WangHTaoTYLiAQ, et al. Structural health monitoring system for Sutong cable-stayed bridge. Smart Struct Syst2016; 18(2): 317–334.
44.
FakhrulddinAHFeiXLiH.Convolutional neural networks (CNN) based human fall detection on body sensor networks (BSN) sensor data. In: 2017 4th international conference on systems and informatics (ICSAI), Hangzhou, China, 11–13 November 2017.
45.
YaoXJYiTHQuCX, et al. Blind modal identification using limited sensors through modified sparse component analysis by time-frequency method. Comput-Aided Civ Inf2018; 33(9): 769–782.
46.
SalehiHBurgueñoR.Pattern recognition framework using asynchronous discrete binary data for condition and damage assessment in plate-like structures. J Intel Mat Syst Str2019; 30(8): 1200–1215.
47.
WangZOatesT. Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. In: Workshops at the twenty-ninth AAAI conference on artificial intelligence, Austin, TX, 25–30 January 2015.
48.
KingmaDPBaJ.Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
49.
SmithLN. Cyclical learning rates for training neural networks. In: 2017 IEEE winter conference on applications of computer vision (WACV), Santa Rosa, CA, 24–31 March 2017.
