This article proposes a new end-to-end deep super-resolution crack network (SrcNet) for improving computer vision–based automated crack detectability. The digital images acquired from large-scale civil infrastructures for crack detection using unmanned robots often suffer from motion blur and lack of pixel resolution, which may degrade the corresponding crack detectability. The proposed SrcNet is able to significantly enhance the crack detectability by augmenting the pixel resolution of the raw digital image through deep learning. SrcNet basically consists of two phases: phase I—deep learning–based super resolution (SR) image generation and phase II—deep learning–based automated crack detection. Once the raw digital images are obtained from a target bridge surface, phase I of SrcNet generates the corresponding SR images to the raw digital images. Then, phase II automatically detects cracks from the generated SR images, making it possible to remarkably improve the crack detectability. SrcNet is experimentally validated using the digital images obtained using a climbing robot and an unmanned aerial vehicle from in situ concrete bridges located in South Korea. The validation test results reveal that the proposed SrcNet shows 24% better crack detectability compared to the crack detection results using the raw digital images.
Ministry of Land, Transport and Maritime Affairs and Korea Infrastructure Safety Corporation. Guidelines for the safety inspection and precise safety diagnosis, 2017, http://law.go.kr/engLsSc.do?tabMenuId=tab45#
2.
DewolfJLauzonRCulmoM. Monitoring bridge performance. Struct Health Monit2002; 1: 129–138.
3.
KochCPaalSRashidiA, et al. Achievements and challenges in machine vision-based inspection of large concrete structures. Adv Struct Eng2014; 17: 303–318.
4.
JungJParkG. Rapid and non-invasive surface crack detection for pressed-panel products based on online image processing. Struct Health Monit2019; 18: 1928–1942.
5.
XuYLiSZhangD, et al. Identification framework for cracks on a steel structure surface by a restricted Boltzmann machine algorithm based on consumer-grade camera images. Struct Control Health Monit2018; 25: e2075.
6.
YeumCDykeS. Vision-based automated crack detection for bridge inspection. Comput Aid Civil Infrastruct Eng2015; 30: 759–770.
7.
ReaganDSabatoANiezreckiC. Feasibility of using digital image correlation for unmanned aerial vehicle structural health monitoring of bridges. Struct Health Monit2018; 17: 1056–1072.
8.
YeumCChoiJDykeS. Autonomous image localization for visual inspection of civil infrastructure. Smart Mater Struct2017; 26: 035051.
9.
JangKKimBChoS, et al. Automated crack evaluation of a high-rise bridge pier using a ring-type climbing robot. Comput Aid Civil Infrastruct Eng. Epub ahead of print 26 March 2020. DOI: 10.1111/mice.12550.
10.
GibbSLaHLeT, et al. Nondestructive evaluation sensor fusion with autonomous robotic system for civil infrastructure inspection. J Field Robot2018; 35: 998–1004.
11.
ChaY-JChoiWBüyüköztürkO. Deep learning-based crack damage detection using convolutional neural network. Comput Aid Civil Infrastruct Eng2017; 32: 361–378.
12.
XuYBaoYChenJ, et al. Surface fatigue crack identification in steel box girder of bridges by a deep fusion convolutional neural network based on consumer-grade camera images. Struct Health Monit2019; 18: 653–674.
13.
KhaniMVahidniaSGhasemzadehL, et al. Deep-learning-based crack detection with applications for the structural health monitoring of gas turbines. Struct Health Monit. Epub ahead of print 11 November 2019. DOI: 10.1177/1475921719883202.
14.
JangKKimNAnY-K. Deep learning-based autonomous concrete crack evaluation through hybrid image scanning. Struct Health Monit2019; 18: 1722–1737.
15.
ChaY-JChoiWSuhG, et al. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput Aid Civil Infrastruct Eng2017; 33: 731–747.
16.
DengJLuYLeeV. Concrete crack detection with handwriting script interferences using faster region-based convolutional neural network. Comput Aid Civil Infrastruct Eng2019; 35: 373–388.
17.
XuYWeiSBaoY, et al. Automatic seismic damage identification of reinforced concrete columns from images by a region-based deep convolutional neural network. Struct Control Health Monit2019; 26: e2313.
18.
LiuYYaoJLuX, et al. DeepCrack: a deep hierarchical feature learning architecture for crack segmentation. Neurocomputing2019; 338: 139–153.
19.
DungCAnhL. Autonomous concrete crack detection using deep fully convolutional neural network. Automat Constr2019; 99: 52–58.
20.
LiSZhaoXZhouG. Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network. Comput Aid Civil Infrastruct Eng2019; 34: 616–634.
21.
GonzalezCRichardWStevenE. Digital image processing using MATLAB. 2nd ed.New York: McGraw-Hill, 2011.
22.
ParkSParkMKangM. Super-resolution image reconstruction: a technical overview. IEEE Signal Process Mag2003; 20: 21–36.
YangJWrightJHuangT, et al. Image super-resolution via sparse representation. IEEE Trans Imag Process2010; 19: 2861–2873.
25.
KimKKwonY. Single-image super-resolution using sparse regression and natural image prior. IEEE Trans Pattern Anal Mach Intell2010; 32: 1127–1133.
26.
ShanQLiZJiaJ, et al. Fast image/video upsampling. ACM Trans Graphic2008; 27: 1–8.
27.
FattalR. Image upsampling via imposed edge statistics. ACM Trans Graphic2007; 26: 1–8.
28.
SunJXuZShumH. Image super-resolution using gradient profile prior. In: 2008 IEEE conference on computer vision and pattern recognition (CVPR), Anchorage, AK, 23–28 June 2008.
29.
LanczosC. An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. J Res Nat Bur Stand1950; 45: 255–282.
30.
IraniMPelegS. Improving resolution by image registration. CVGIP Graph Model Imag Process1991; 53: 231–239.
31.
YangC-YMaCYangM-H. Single-image super-resolution: a benchmark. In: European conference on computer vision (Lecture notes in computer science), Zurich, 6–12 September 2014, pp. 372–386. Berlin: Springer.
32.
DongCLoyCHeK, et al. Image super-resolution using deep convolutional networks. IEEE Tran Pattern Anal Mach Intell2016; 38: 295–307.
33.
LedigCTheisLHuszárF, et al. Photo-realistic single image super-resolution using a generative adversarial network. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, HI, 21–26 July 2017.
34.
TaiYYangJLiuX. Image super-resolution via deep recursive residual network. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, HI, 21–26 July 2017.
35.
KimJLeeJLeeK, et al. Accurate image super-resolution using very deep convolutional networks. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, NV, 27–30 June 2016.
36.
ZhangYLiKLiK, et al. Image super-resolution using very deep residual channel attention networks. In: Computer vision—ECCV 2018 (Lecture notes in computer science), Munich, 8–14 September 2018, pp. 294–310. Berlin: Springer.
37.
HeKZhangXRenS, et al. Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, NV, 27–30 June 2016.
38.
HuJShenLSunG. Squeeze-and-excitation networks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR), Salt Lake City, UT, 18–23 June 2018.
39.
WangZChenJHoiS. Deep learning for image super-resolution: a survey. arXiv:1902.06068v1, 2019.
LiJLiangXWeiY, et al. Perceptual generative adversarial networks for small object detection. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, HI, 21–26 July 2017.
42.
LimBSonSKimH, et al. Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, HI, 21–26 July 2017.
43.
SimonyanKZissermanA. Very deep convolutional networks for large-scale image recognition. In: 3rd international conference on learning representations (ICLR 2015), San Diego, CA, 7–9 May 2015.
44.
BadrinarayananVKendallACipollaR. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell2017; 39: 2481–2495.
45.
AgustssonETimofteR. NTIRE 2017 challenge on single image super-resolution: dataset and study. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, HI, 21–26 July 2017.
46.
KeysR. Cubic convolution interpolation for digital image processing. IEEE Trans Acoust Speech Signal Process1981; 29: 1153–1160.
47.
RobbinsHMonroS. A stochastic approximation method. Ann Math Stat1951; 22: 400.
48.
PowerD. Evaluation: from precision, recall and F-measure to ROC, informedeness, markedness & correlation. J Mach Learn Technol2011; 1: 37–63.
49.
JangKAnY-K. Multiple crack evaluation on concrete using a line laser thermography scanning system. Smart Struct Syst2018; 22: 201–207.
50.
AnY-KKwonYSohnH. Noncontact laser ultrasonic crack detection for plates with additional structural complexities. Struct Health Monit Int J2013; 12: 522–538.
51.
AnY-K. Measurement of crack-induced non-propagating lamb wave modes under varying crack widths. Int J Sol Struct2015; 62: 134–143.
