Bridges are critical transportation infrastructure in Taiwan that are at high risk of structural damage during major earthquake incidents. The structural health monitoring method currently used to assess bridge safety nationwide is labor-intensive and expensive to implement. In this study, a novel automated bridge safety assessment system was developed to facilitate post-earthquake bridge inspections using symbiotic organism search-transfer learning-PointRend (STF-PointRend) for component and damage type detection, EfficientNetB0 for damage level detection, and earthquake resistance index for safety assessment. In the case studies, the STF-PointRend model obtained good testing results, with global mean Intersection over Union values of 76.31 and 75.47% for bridge-component and damage-type detections. Furthermore, the EfficientNetB0 model obtained an average F1 score of 0.86 for damage-level detection in the testing results. The developed model was used to conduct a safety evaluation of two bridges (Lempenge I and Luk I) that had suffered damage in a 2018 earthquake in Palu, Indonesia. The earthquake-resistance scores of 56.15 and 53.21 respectively earned by the two bridges indicate that both require immediate maintenance.
BarrileVCandelaGFotiaA, et al. UAV survey of bridges and viaduct: workflow and application. In: MisraSMurganteBKorkhovV, et al. (eds.) ChamSMisraS, et al. (eds) Computational science and its applications – ICCSA 2019. Cham, Switzerland: Springer International Publishing, 2019, pp. 269–284.
2.
YehC-HLohC-HTsaiK-C.Overview of Taiwan earthquake loss estimation system. Nat Hazard2006; 37(1): 23–37.
3.
KoJMNiYQ.Technology developments in structural health monitoring of large-scale bridges. Eng Struct2005; 27(12): 1715–1725.
4.
Hsien-KeL, et al. Comparison of bridge inspection methodologies and evaluation criteria in Taiwan and foreign practices. In: ISARC. Proceedings of the international symposium on automation and robotics in constructionTaipei, Taiwan, 28 June–1 July, 2017, vol. 34. IAARC Publications.
5.
LiaoH-K.Development of a bridge maintenance decision support module for Taiwan Bridge Management System. In: Bridge maintenance, safety management, health monitoring and informatics-IABMAS’08: proceedings of the fourth international IABMAS conference, Seoul, Korea, 13–17 July 2008. Boca Raton, FL, USA: CRC Press, p.166.
6.
WeiXYangZLiuY, et al. Railway track fastener defect detection based on image processing and deep learning techniques: a comparative study. Eng Appl Artif Intell2019; 80: 66–81
7.
GastineauAJohnsonTSchultzA.Bridge health monitoring and inspections–a survey of methods. Minnesota, USA: Minnesota Department of Transportation, 2009.
8.
ChengM-YFangY-CChiuY-F, et al. Design and maintenance information integration for concrete bridge assessment and disaster prevention. J Perform Constr Facil2021; 35(3): 04021015.
9.
ZhangCChangCCJamshidiM.Bridge damage detection using a single-stage detector and field inspection images. arXiv preprint arXiv:1812.10590, 2018.
10.
ZhangCChangCCJamshidiM.Concrete bridge surface damage detection using a single-stage detector. Comput Aided Civ Infrastruct Eng2020; 35(4): 389–409.
11.
NarazakiYHoskereVYoshidaK, et al. Synthetic environments for vision-based structural condition assessment of Japanese high-speed railway viaducts. Mechanical Systems and Signal Processing2021; 160: 107850.
12.
KashikaPVenkatapurRB.Automatic tracking of objects using improvised Yolov3 algorithm and alarm human activities in case of anomalies. Int J Inf Technol2022; 14(6): 2885–2891.
13.
KirillovAWuYHeK, et al. Pointrend: image segmentation as rendering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Seattle, WA, USA, 14–19 June 2020, pp.9799–9808. New York: IEEE Computer Society.
14.
LinT-YDollárPGirshickR, et al. Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognitionHonolulu, HI, USA, 21–26 July, 2017, pp.2117–2125. New York: IEEE Computer Society.
15.
ParmarVBhatiaNNegiS, et al. Exploration of optimized semantic segmentation architectures for edge-deployment on drones. arXiv preprint arXiv:2007.02839, 2020.
16.
TanMLeQ.EfficientNet: rethinking model scaling for convolutional neural networks. In: Proceedings of the 36th international conference on machine learning, proceedings of machine learning research. [Online], https://proceedings.mlr.press/v97/tan19a.html (2019, accessed 1 December 2021).
17.
Pantaleón MarcosJRamos ÓscarROrtegaG, et al. Dynamic analysis of a composite cable-stayed bridge: Escaleritas viaduct. J Bridge Eng2010; 15(6): 653–660.
18.
RosGSellartLMaterzynskaJ, et al. The synthia dataset: a large collection of synthetic images for semantic segmentation of urban scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, 27–30 June 2016, pp.3234–3243. New York: IEEE Computer Society.
19.
BibaevaV.Using metaheuristics for hyper-parameter optimization of convolutional neural networks. In: 2018 IEEE 28th international workshop on machine learning for signal processing (MLSP)Aalborg, Denmark, 17–20 September, 2018, IEEE, pp.1–6.
20.
SwerskyKSnoekJAdamsRP.Multi-task bayesian optimization. Adv Neural Inf Process Sys2013; 26: 1–9.
21.
WangLFengMZhouB, et al. Efficient hyper-parameter optimization for NLP applications. In: Proceedings of the 2015 conference on empirical methods in natural language processing, Lisbon, Portugal, 17–21 September, 2015, pp.2112–2117. Association for Computational Linguistics.
22.
YooJYoonHKimH, et al. Optimization of hyper-parameter for CNN model using genetic algorithm. In: 2019 1st International conference on electrical, control and instrumentation engineering (ICECIE), 25–25 November 2019, pp.1–6.
23.
ChengM-YPrayogoD.Symbiotic organisms search: a new metaheuristic optimization algorithm. Comput Struct2014; 139: 98–112.
YauNJLiaoHK.Indices for fast assessment of bridge condition in Taiwan. Applied Mechanics and Materials2015; 752–753: 689–697.
26.
FaesLWagnerSKFuDJ, et al. Automated deep learning design for medical image classification by health-care professionals with no coding experience: a feasibility study. Lancet Digit Health2019; 1(5): e232–e242.
27.
RustowiczRMCheongRWangL, et al. Semantic segmentation of crop type in Africa: A novel dataset and analysis of deep learning methods. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. Long Beach, CA, USA, 16–17 June, 2019, pp.75–82. New York: IEEE Computer Society.
28.
MontagnonECernyMCadrin-ChênevertA, et al. Deep learning workflow in radiology: a primer. Insights Imaging2020; 11(1): 22.
29.
NietoDBrillAKimB, et al. Exploring deep learning as an event classification method for the Cherenkov Telescope Array. arXiv preprint arXiv:1709.05889, 2017.
KeT-WHwangJ-JLiuZ, et al. Adaptive affinity fields for semantic segmentation. In: Proceedings of the European conference on computer vision (ECCV)Munich, Germany, 8–14 September, 2018, pp.587–602. Berlin/Heidelberg, Germany: Springer.
33.
AlonsoIRiazueloLMurilloAC.MiniNet: an efficient semantic segmentation ConvNet for real-time robotic applications. IEEE transactions on robotics2020; 36(4): 1340–1347.
34.
PerezLWangJ.The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:1712.04621, 2017
35.
ShortenCKhoshgoftaarTM.A survey on image data augmentation for deep learning. J Big Data2019; 6(1): 60.
36.
MikołajczykAGrochowskiM.Data augmentation for improving deep learning in image classification problem. In: 2018 International interdisciplinary PhD workshop (IIPhDW)Świnoujście, Poland, 9–12 May 2018, pp.117–122. New York: Institute of Electrical and Electronics Engineers (IEEE).
