Automated region-of-interest localization and classification for vision-based visual assessment of civil infrastructure

Complementary advances in computer vision and new sensing platforms have mobilized the research community to pursue automated methods for vision-based visual evaluation of our civil infrastructure. Spatial and temporal limitations typically associated with sensing in large-scale structures are being torn down through the use of low-cost aerial platforms with integrated high-resolution visual sensors. Despite the enormous efforts expended to implement such technology, practical real-world challenges still hinder the application of these methods. The large volumes of complex visual data, collected under uncontrolled circumstances (e.g. varied lighting, cluttered regions, occlusions, and variations in environmental conditions), impose a major challenge to such methods, especially when only a tiny fraction of them are used for conducting the actual assessment. Such difficulties induce undesirable high rates of false-positive and false-negative errors, reducing both trustworthiness and efficiency in the methods. To overcome these inherent challenges, a novel automated image localization and classification technique is developed to extract the regions of interest on each of the images, which contain the targeted region for inspection. Regions of interest are extracted here using structure-from-motion algorithm. Less useful regions of interest, such as those corrupted by occlusions, are then filtered effectively using a robust image classification technique, based on convolutional neural networks. Then, such highly relevant regions of interest are available for visual assessment. The capability of the technique is successfully demonstrated using a full-scale highway sign truss with welded connections.