The production and manufacturing processes of steel inevitably generate various types of surface defects. The real-time and accurate detection of these surface defects is of great practical significance. To realise real-time detection of steel surface defects with significant differences in shape and size on resource constrained edge computing equipment, this paper proposes a lightweight real-time steel surface defect detection model SD-YOLO based on a dynamic parameterisation strategy. Firstly, a Dynamic Parameterised Enhancement Module is proposed, which dynamically assigns routing weights to parallel convolutional kernels based on input features, thereby enhancing the representation of defect features in the feature map and improving the network's ability to capture rich and detailed features. Secondly, the Efficient Intersection over Union loss function is employed to optimise the regression process of the prediction boxes. This enhances the model's fitting performance on bounding boxes with significant aspect ratio differences and improves the accuracy of detecting defects of various scales. Experimental results indicate that for the NEU-DET and GC10-DET datasets, SD-YOLO achieves a mean average precision of 83.1% and 74.1% respectively, with a stronger focus on defective regions, and detection speeds of 169.5 Frames Per Second (FPS) and 178.6 FPS, respectively. When SD-YOLO is deployed on the NVIDIA Jetson Orin NANO, the detection speed reaches 33.9 FPS and 66.7 FPS respectively, and maintains the same detection accuracy as the server-side, which realises real-time, accurate, and automatic detection of steel surface defects on edge computing devices with limited computational resources. Furthermore, SD-YOLO also demonstrates excellent generalisation ability and accuracy on images of steel surface defects collected in real industrial environments. In conclusion, SD-YOLO provides a practical and effective solution for real-time steel surface defect detection in resource-constrained environments, making it highly suitable for deployment in industrial applications. Source code is available at https://github.com/Xcy0512/SD-YOLO.
YangLYuHLiR,et al.Detection and evaluation mechanism of cold rolling strip edge shape. Ironmaking & Steelmaking2018; 45: 457–468.
2.
ZhaoLMOuyangQChenDF,et al.Surface defects inspection method in hot slab continuous casting process. Ironmaking & Steelmaking2011; 38: 464–470.
3.
ZhaoJLiuSTaoH,et al.Slim-YOLOv8: a fast and accurate algorithm for surface defect detection of steel strips. Ironmaking & Steelmaking2024; 51: 03019233241266717.
4.
YuJWangYLiQ, et al.Cascaded adaptive global localisation network for steel defect detection. International Journal of Production Research2024; 62: 4884–4901.
5.
GuanSChangJShiH, et al.Strip steel defect classification using the improved GAN and EfficientNet. Applied Artificial Intelligence2021; 35: 1887–1904.
6.
WilliamsTJHardinRABeckermannC. Thermophysical properties and performance of riser sleeves for steel castings. International Journal of Metalcasting2016; 10: 535–555.
7.
SophianATianGYZairiS. Pulsed magnetic flux leakage techniques for crack detection and characterisation. Sensors and Actuators A: Physical2006; 125: 186–191.
8.
LópezGBasterraL-ARamón-CuetoG,et al.Detection of singularities and subsurface defects in wood by infrared thermography. International Journal of Architectural Heritage2014; 8: 517–536.
9.
MukupaWRobertsGWHancockCM,et al.A review of the use of terrestrial laser scanning application for change detection and deformation monitoring of structures. Survey Review2017; 49: 99–116.
10.
HuHLiYLiuM,et al.Classification of defects in steel strip surface based on multiclass support vector machine. Multimedia Tools and Applications2014; 69: 199–216.
11.
GirshickR. Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision, Santiago, Chile,2015, pp.1440–1448.
12.
RenSHeKGirshickR,et al.Faster R-CNN: towards real-time object detection with region proposal networks. Advances in neural information processing systems2016; 39: 1137–1149.
13.
HeKGkioxariGDollárP, et al.Mask R-CNN. In: Proceedings of the IEEE international conference on computer vision, Venice, Italy,2017, pp.2961–2969.
14.
RedmonJDivvalaSGirshickR, et al.You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA,2016, pp.779–788.
15.
RedmonJFarhadiA. YOLOv3: an incremental improvement. arXiv preprint arXiv:1804.02767, 2018.
16.
BochkovskiyAWangC-YMark LiaoH-Y. YOLOv4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934, 2020.
17.
GeZLiuSWangF,et al.YOLOx: exceeding YOLO series in 2021. arXiv preprint arXiv:2107.08430, 2021.
18.
XingJJiaM. A convolutional neural network-based method for workpiece surface defect detection. Measurement2021; 176: 109185.
19.
LiuJCuiGXiaoC. A real-time and efficient surface defect detection method based on YOLOv4. Journal of Real-Time Image Processing2023; 20: 77.
20.
LuoQLiBSuJ, et al.CDDNet: Camouflaged defect detection network for steel surface. IEEE Transactions on Instrumentation and Measurement2023; 73: 1–13.
21.
WangCLiuHYangX, et al.STD-detector: spatial-to-depth feature-enhanced detection method for the surface defect detection of strip steel. Journal of Electronic Imaging2023; 32: 063007–063007.
22.
LiuGChuMGongR,et al.DLF-YOLOF: an improved YOLOF-based surface defect detection for steel plate. Journal of Iron and Steel Research International2024; 31: 442–451.
23.
WenXShanJHeY,et al.Steel surface defect recognition: a survey. Coatings2022; 13: 17.
24.
WangGChenMLinY, et al.An efficient parallel fusion structure of distilled and transformer-enhanced modules for lightweight image super-resolution. The Visual Computer2024; 40: 1–20.
25.
CuiWSongKJiaX, et al.An efficient targeted design for real-time defect detection of surface defects. Optics and Lasers in Engineering2024; 178: 108174.
26.
WangGChenMLinYC, et al.Efficient multi-branch dynamic fusion network for super-resolution of industrial component image. Displays2024; 82: 102633.
27.
CuiWSongKWangY, et al.A rapid screening method for suspected defects in steel pipe welds by combining correspondence mechanism and normalizing flow. IEEE Transactions on Industrial Informatics2024; 20: 11171–11180.
28.
WangGZhangCChenM, et al.YOLO-MSAPF: multiscale alignment fusion with parallel feature filtering model for high accuracy weld defect detection. IEEE Transactions on Instrumentation and Measurement2023; 72: 1–14.
29.
SongKCuiWYuH, et al.SAM era: can it segment any industrial surface defects?Computers, Materials & Continua2024; 78: 3953–3969.
30.
SongKFengHCaoT, et al.MFANet: multifeature aggregation network for cross-granularity few-shot seamless steel tubes surface defect segmentation. IEEE Transactions on Industrial Informatics2024; 20: 9725–9735.
31.
CuiWSongKFengH, et al.Autocorrelation-aware aggregation network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement2023; 72: 1–12.
32.
WangGZhangCChenM, et al.A high-accuracy and lightweight detector based on a graph convolution network for strip surface defect detection. Advanced Engineering Informatics2024; 59: 102280.
33.
LiuJLiHZuoF, et al.KD-LightNet: A lightweight network based on knowledge distillation for industrial defect detection. IEEE Transactions on Instrumentation and Measurement2023; 72: 1–13.
34.
ZhangJKangXNiH,et al.Surface defect detection of steel strips based on classification priority YOLOv3-dense network. Ironmaking & Steelmaking2021; 48: 547–558.
35.
LvSTaoCHaoZ, et al.Research on strip surface defect detection based on improved YOLOv5 algorithm. Ironmaking & Steelmaking2024; 51: 03019233241260922.
36.
WangCBochkovskiyALiaoHM. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Vancouver, Canada,2023, pp.7464–7475.
37.
LiXWangWWuL, et al.Generalized focal loss: learning qualified and distributed bounding boxes for dense object detection. Advances in Neural Information Processing Systems2020; 33: 21002–21012.
38.
ZhangYRenWZhangZ, et al.Focal and efficient IoU loss for accurate bounding box regression. Neurocomputing2022; 506: 146–157.
39.
YangBBenderGLeQV,et al.Condconv: conditionally parameterized convolutions for efficient inference. Advances in neural information processing systems2019; 32: 1–12.
40.
ChenYDaiXLiuM, et al.Dynamic convolution: attention over convolution kernels. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Seattle, WA, USA,2020, pp.11027–11036.
41.
HanKWangYGuoJ,et al.ParameterNet: parameters are all you need for large-scale visual pretraining of mobile networks. arXiv preprint arXiv:2306.14525, 2023.
42.
ShazeerNMirhoseiniAMaziarzK,et al.Outrageously large neural networks: the sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538, 2017.
43.
HeYSongKMengQ,et al.An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on Instrumentation and Measurement2019; 69: 1493–1504.
44.
LvXDuanFJiangJ-j, et al.Deep metallic surface defect detection: the new benchmark and detection network. Sensors2020; 20: 1562.
45.
WangC-YYehI-HMark LiaoH-Y. YOLOv9: learning what you want to learn using programmable gradient information. arXiv preprint arXiv:2402.13616, 2024.
ZhaoYLvWXuS, et al.DETRs beat YOLOs on real-time object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Seattle, WA, USA,2024, pp.16965–16974.
48.
YuXLyuWZhouD, et al.ES-Net: efficient scale-aware network for tiny defect detection. IEEE Transactions on Instrumentation and Measurement2022; 71: 1–14.
49.
LiSKongFWangR, et al.EFD-YOLOv4: a steel surface defect detection network with encoder-decoder residual block and feature alignment module. Measurement2023; 220: 113359.
50.
LiuRHuangMGaoZ, et al.MSC-DNet: an efficient detector with multi-scale context for defect detection on strip steel surface. Measurement2023; 209: 112467.
51.
TianRJiaM. DCC-CenterNet: a rapid detection method for steel surface defects. Measurement2022; 187: 110211.
52.
ZhaoCShuXYanX, et al.RDD-YOLO: a modified YOLO for detection of steel surface defects. Measurement2023; 214: 112776.
53.
ChenHDuYFuY, et al.DCAM-Net: a rapid detection network for strip steel surface defects based on deformable convolution and attention mechanism. IEEE Transactions on Instrumentation and Measurement2023; 72: 1–12.
54.
HanJCuiGLiZ,et al.DBCW-YOLO: a modified YOLOv5 for the detection of steel surface defects. Applied Sciences2024; 14: 4594.
55.
ZhongHFuDXiaoL, et al.STFE-Net: a multi-stage approach to enhance statistical texture feature for defect detection on metal surfaces. Advanced Engineering Informatics2024; 61: 102437.
56.
ZuoLSuSFanS, et al.A new dual-branch network with global information for the surface defect detection on solar pv wafer. IEEE Sensors Journal2024; 24: 9197–9207.
57.
RezatofighiHTsoiNGwakJ, et al.Generalized intersection over union: a metric and a loss for bounding box regression. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Long Beach, CA, USA,2019, pp.658–666.
58.
ZhengZWangPLiuW, et al.Distance-IoU loss: faster and better learning for bounding box regression. Proceedings of the AAAI Conference on Artificial Intelligence2020; 34: 12993–13000.
59.
GevorgyanZ. Siou loss: more powerful learning for bounding box regression. arXiv preprint arXiv:2205.12740, 2022.
60.
ZhangHZhangS. Shape-IoU: More accurate metric considering bounding box shape and scale. arXiv preprint arXiv:2312.17663, 2023.
61.
SelvarajuRRCogswellMDasA, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision, Venice, Italy,2017, pp.618–626.
