With the development of deep learning, object detection algorithms based on horizontal box are widely used in the field of damage identification. However, damages can be in any direction and position, and they are not necessarily horizontal or vertical. This article proposes a bolt damage identification network, namely, orientation-aware center point estimation network, which models a damage as a center point of its rotated bounding box. The proposed orientation-aware center point estimation network uses deep layer aggregation network to search center points and regress to all other damage properties, such as size and angle. A loss function is designed to improve the optimization efficiency of network. Orientation-aware center point estimation network is applied to bolt damage detection, and comparison with the well-known Faster Region-Convolutional Neural Network (a benchmark using horizontal bounding box) demonstrates the accuracy of the proposed method. Finally, videos were utilized to verify the capability of the proposed orientation-aware center point estimation network in real-time detection of bolt damages.
StephenJT.Condition monitoring of bolted joints. Doctoral Dissertation, The University of Sheffield, Sheffield, 2015.
2.
BaoYLiHSunX, et al. Compressive sampling–based data loss recovery for wireless sensor networks used in civil structural health monitoring. Struct Health Monit2013; 12(1): 78–95.
3.
HasniHJiaoPAlaviAH, et al. Structural health monitoring of steel frames using a network of self-powered strain and acceleration sensors: a numerical study. Automat Constr2018; 85: 344–357.
4.
SpencerBFJrParkJWMechitovKA, et al. Next generation wireless smart sensors toward sustainable civil infrastructure. Procedia Engineer2017; 171: 5–13.
5.
ChangPCFlatauALiuSC.Health monitoring of civil infrastructure. Struct Health Monit2003; 2(3): 257–267.
6.
YinHWangTYangD, et al. A smart washer for bolt looseness monitoring based on piezoelectric active sensing method. Appl Sci2016; 6(11): 320.
7.
HuoLChenDLiangY, et al. Impedance based bolt pre-load monitoring using piezoceramic smart washer. Smart Mater Struct2017; 26(5): 057004.
8.
OreguiE-BMLiSNunezVAA, et al. Monitoring bolt tightness of rail joints using axle box acceleration measurements. Struct Control Hlth2017; 24(2): e1848.
9.
JiangTWuQWangL, et al. Monitoring of bolt looseness-induced damage in steel truss arch structure using piezoceramic transducers. IEEE Sens J2018; 18(16): 6677–6685.
10.
WangFHoSCMSongG.Monitoring of early looseness of multi-bolt connection: a new entropy-based active sensing method without saturation. Smart Mater Struct2019; 28(10): 10LT01.
11.
DuFXuCZhangJ.A bolt preload monitoring method based on the refocusing capability of virtual time reversal. Struct Control Hlth2019; 26(8): e2370.
12.
TolaKDLeeCParkJ, et al. Bolt looseness detection based on ultrasonic wavefield energy analysis using an Nd:YAG pulsed laser scanning system. Struct Control Hlth2020; 27(9): e2590.
13.
HoSCMLiWWangB, et al. A load measuring anchor plate for rock bolt using fiber optic sensor. Smart Mater Struct2017; 26(5): 057003.
14.
ZhangYZhaoXSunX, et al. Bolt loosening detection based on audio classification. Adv Struct Eng2019; 22(13): 2882–2891.
15.
KongQZhuJHoSCM, et al. Tapping and listening: a new approach to bolt looseness monitoring. Smart Mater Struct2018; 27(7): 07LT02.
16.
YuanRLvYKongQ, et al. Percussion-based bolt looseness monitoring using intrinsic multiscale entropy analysis and BP neural network. Smart Mater Struct2019; 28(12): 125001.
17.
WangFHoSCMSongG.Modeling and analysis of an impact-acoustic method for bolt looseness identification. Mech Syst Signal Pr2019; 133: 106249.
18.
FengDFengMQ.Computer vision for SHM of civil infrastructure: from dynamic response measurement to damage detection—a review. Eng Struct2018; 156: 105–117.
19.
DavoudiRMillerGRKutzJN.Data-driven vision-based inspection for reinforced concrete beams and slabs: quantitative damage and load estimation. Automat Constr2018; 96: 292–309.
20.
DavoudiRMillerGRCalviP, et al. Computer vision–based damage and stress state estimation for reinforced concrete and steel fiber–reinforced concrete panels. Struct Health Monit2020; 19(6): 1645–1665.
21.
ChaYJChenJGBüyüköztürkO.Output-only computer vision based damage detection using phase-based optical flow and unscented Kalman filters. Eng Struct2017; 132: 300–313.
22.
DworakowskiZKohutPGallinaA, et al. Vision-based algorithms for damage detection and localization in structural health monitoring. Struct Control Hlth2016; 23(1): 35–50.
23.
ShiJXuXWangJ, et al. Beam damage detection using computer vision technology. Nondestruct Test Eva2010; 25(3): 189–204.
24.
RamanaLChoiWChaYJ.Fully automated vision-based loosened bolt detection using the Viola–Jones algorithm. Struct Health Monit2019; 18(2): 422–434.
25.
ChaYJYouKChoiW.Vision-based detection of loosened bolts using the Hough transform and support vector machines. Automat Constr2016; 71: 181–188.
26.
ParkJHHuynhTCChoiSH, et al. Vision-based technique for bolt-loosening detection in wind turbine tower. Wind Struct2015; 21(6): 709–726.
27.
O’MahonyNCampbellSCarvalhoA, et al. Deep learning vs. traditional computer vision. In: Proceedings of the science and information conference, Las Vegas, NV, 25–26 April 2019, pp. 128–144. Cham: Springer.
28.
KimSKHuhJH.Consistency of medical data using intelligent neuron faster R-CNN algorithm for smart health care application. Healthcare2020; 8(2): 185.
29.
KamnitsasKLedigCNewcombeVF, et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal2017; 36: 61–78.
30.
MortaziABagciU. Automatically designing CNN architectures for medical image segmentation. In: Proceedings of the international workshop on machine learning in medical imaging, Granada, 16 September 2018, pp. 98–106. Cham: Springer.
31.
KisilevPSasonEBarkanE, et al. Medical image description using multi-task-loss CNN. In: CarneiroGMateusDLoïcP, et al. (eds) Deep learning and data labeling for medical applications. Cham: Springer, 2016, pp. 121–129.
32.
MaXDaiZHeZ, et al. Learning traffic as images: a deep convolutional neural network for large-scale transportation network speed prediction. Sensors2017; 17(4): 818.
33.
FaroAGiordanoDSpampinatoC.Evaluation of the traffic parameters in a metropolitan area by fusing visual perceptions and CNN processing of webcam images. IEEE T Neural Networ2008; 19(6): 1108–1129.
34.
ZhangYSunXLohKJ, et al. Autonomous bolt loosening detection using deep learning. Struct Health Monit2020; 19(1): 105–122.
35.
WangZXTuXJGaoXR, et al. Bolt detection of key component for high-speed trains based on deep learning. In: Proceedings of the 2019 far east NDT new technology & application forum (FENDT), Qingdao, China, 24–27 June 2019, pp. 192–196. New York: IEEE.
36.
ZhaoXZhangYWangN.Bolt loosening angle detection technology using deep learning. Struct Control Hlth2019; 26(1): e2292.
37.
HuynhTCParkJHJungHJ, et al. Quasi-autonomous bolt-loosening detection method using vision-based deep learning and image processing. Automat Constr2019; 105: 102844.
38.
XieYSunJ. On-line bolt-loosening detection method of key components of running trains using binocular vision. In: Proceedings of the LIDAR imaging detection and target recognition, Changchun, China, 23–25 July 2017, vol. 10605, p. 1060513. Bellingham, WA: SPIE, the International Society for Optics and Photonics.
39.
PhamHCTaQBKimJT, et al. Bolt-loosening monitoring framework using an image-based deep learning and graphical model. Sensors2020; 20(12): 3382.
40.
SunJXieYChengX.A fast bolt-loosening detection method of running train’s key components based on binocular vision. IEEE Access2019; 7: 32227–32239.
LinTYDollárPGirshickR, et al. Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, 21–26 July 2017, pp. 2117–2125. New York: IEEE.
43.
RedmonJDivvalaSGirshickR, et al. You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, 27–30 June 2016, pp. 779–788. New York: IEEE.
44.
RenSHeKGirshickR, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis & Machine Intelligence2017; 39(6): 1137–1149.
45.
YuFWangDShelhamerE, et al. Deep layer aggregation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Salt Lake City, UT, 18–23 June 2018, pp. 2403–2412. New York: IEEE.
