FOD-YOLO NET: Fasteners fault and object detection in railway tracks using deep yolo network 1

Abstract

Automated railway security systems prevent train collisions with trackside obstructions that cause accidents in high-speed railways. Rail safety is being improved and accident rates reduced through continuous research. A rapid advancement in deep learning has promoted new possibilities for research in this field. In this work, a novel deep learning-based FOD-YOLO net is proposed for detecting the fasteners faults and objects in the railway tracks. There are two basic components in the deep learning-based YOLOv8: the backbone and the head. YOLOv8 utilizes an improved version of the CSPDarknet53 network for detecting objects on the railway track. The head of YOLOv8 consists of EfficientNet with various convolutional layers with squeeze and excitation blocks for detecting any defect in the track fasteners. These layers are liable for detecting the objectness scores, bounding boxes and class probabilities structured with fully connected layers for the objects and faults in tracks. Based on the results from the Yolo network, the alert message is sent to the loco pilot to avoid accidents using fuzzy logic. The experimental fallouts of proposed FOD-YOLO net achieve higher accuracy and yields better evaluation results with 98.14% accuracy, 98.84% precision and 95.94% recall. From the experimental results, the FOD-YOLO net improves the overall accuracy range by 5.44%, 4.72%, 0.73%, and 13.18% better than Fast RCNN, YOLOv5s-VF, YOLO-GD, and 2D-SSA + Deep network respectively.

Keywords

Railway track object detection fault detection deep learning Yolo network fuzzy logic

1 Introduction

A railroad track is the most important part of a rail system, and the rails should be fastened to the crossties for a safe journey. Railways have always placed a great deal of importance on automatic track inspection as they expanded their networks [1, 2]. The nation will be greatly impacted by major advances in this field. In order to prevent accidents on the rails, it is imperative that the rails are properly monitored and maintained because of their enormous size. Many lives are affected by the absence of carelessness [3]. An accident prevention method has been introduced to prevent this from happening on railways. Specifically, this approach is focused on regions where animals are frequently seen on railroad tracks. It is simple to detect organisms using cameras, preventing accidents [4, 5].

Manual examinations and the use of specialized equipment take time and require knowledgeable personnel [6]. The use of vision-based technologies has grown in popularity as a result of the expansion of railroad networks, since they are effective at identifying components as they are based on railroad track images [7, 8]. Deep learning has rapidly advanced, creating new prospects for study in the subject. Track employees initially do track patrol and maintenance on foot before progressively transitioning to using track patrol automobiles, but they remain solely dependent on human visual inspection [9, 10]. However, manual detection of an object and defect in the railway track is time consuming. Study on railway scrutiny systems has engrossed on infrastructure systems, surface defect identification, fastener detection, etc [11 –13]. To avoid train derailments and other kind of harsh accidents caused by defective or failed railway track elements, it is essential to inspect rails, bolts, and clips, which fastener rail ends composed at a joint, and rail flexible clips that attach rails to rail sleepers. The fastener consists of clip and bolt together. Advances in image processing can efficiently prevent accidents on railway track [14, 15]. Deep neural networks and machine learning have produced promising fallouts in all real-time applications. The use of deep learning for collision identification between any organism on track can help prevent such accidents. The primary contributions of our research are listed as follows:

The key objective of this research is to present a novel FOD-YOLO model to identify the objects and faults in the railway tracks to avoid the terrible accidents.

In FOD-YOLO model, the self-attention integrated CSPDarkNet53 (SE-CSPDN53) is used as a backbone structure for feature map extraction from the initial stage to the final stage of the network to efficiently detect the objects on the track.

In FOD-YOLO, the head consists of EfficientNet with squeeze and excitation modules (SE- EfficientNet) as well as Gelu activation functions with convolution layers for detecting track fastener defects in the track.

Finally, the Fuzzy based decision making is performed using the both scenarios from the detection results of the FOD-YOLO model are compared to alert the loco-pilot and also to nearby control room.

The effectiveness of the proposed FOD-YOLO net shows high performance in terms of accuracy, specificity, precision, recall, mAP, FPS, and F1 score with respective values.

The rest of this work is structured as follows: section 2 designates the related works on object detection in railway tracks, section 3 explains the inclusive work of the proposed FOD-YOLO net, section 4 describes the experimental results and discussion and the section 5 enfolds with the conclusion and future work.

2 Literature Survey

In this section, researchers have proposed several object detection strategies for detecting the organism on the railway track in recent years. This section provides a brief overview of several recent machine and deep learning and machine learning studies on the detection of objects and faults in the railway tracks to avoid accidents.

In 2022 Wang, M., et al., [16] suggested a YOLOv5s-VF, a revolutionary rail surface fault detecting network. V-CBAM with two essential parts: SSA and FCAM. The collection of rail image is based on human labelling of flaws to train an object identification network, it is available on open source due to the scarcity of publicly available datasets. The result demonstrates the overall speed of 114.9 fps and accuracy of 93.5% that YOLOv5s-VF exceeds current rail surface object identification approaches.

In 2022, Yue, X., et al., [17] introduced a YOLO-GD model to recognize dishes in images, including cups, chop sticks, bowls, and more. The catch point calculation was created on the basis of image processing for retrieving the catch point of the various kinds of dishes. According to the trial findings, YOLO-GD mAP achieves 97.38% that was 3.41% better than YOLOv4. The inference time for an image was reduced by the YOLO-GD model of mAP is 97.42%.

In 2022 Kapoor., et al., [18] devised a deep classifier network that is effective at identifying objects on the railroad track in front of the train. The 2D-SSA and deep network combination improves the performance of obstacle recognition. This technique also offers another way to locate railroad tracks. Additionally, utilizing the OSU thermal pedestrian benchmark database, the performance of this technique was examined under various lighting scenarios. It performs better with an accuracy rate of 85.2%.

In 2022 Rahman., et al., [19] a transfer learning model built on a convolutional neural network was applied to a unique dataset. A dataset that can be useful in identifying obstructions for rail crossings or tracks was created by gathering manual shot photos from a real-world train track. This suggested MobileNetV2 was demonstrated its effectiveness in utilizing resources and its capacity to find obstructions on the railway track. Even though the other networks like YOLOv5, Resnet50, VGG19, and VGG16 also utilized on the dataset, MobileNetV2 outperformed the other chosen models with the accuracy of 97.0%.

In 2020 Li, B., et al., [20] suggested a two-stage detection approach for locating equipment that identified the signal condition of a track circuit. It improves the network architecture to detect the equipment appearance. The ACU and recall rate of the suggested method is higher than those of a single target detection algorithm. This suggested two-phase detection method for improving the efficiency of YOLOv3 for detecting objects.

In 2020 Ranyal, and Jain [21] suggested employing one-stage RetinaNet deep convolution network architecture to locate rail fasteners on train lines. DJI Phantom was flown at a height of 100 meters were used to capture aerial images with a size of 5472×3078 to undertake a quantitative assessment. Results demonstrate that the obtained deep features were used to detect rail abnormalities that may need immediate repair and hold the promise of a robust technique when trained on a bigger dataset.

In 2020 Liu, W., et al., [22] presented a unified CNN architecture for identifying all CSC modules at different scales. Initially by enhancing and optimizing Faster R-CNN, a detection network for CSCs with huge sizes was introduced. The detection network for CSCs with large scales is then combined with a cascade network that was suggested for the identification of CSCs with small scales to develop a single network design. It was found that 92.8% of detections were accurate using the unified CNN architecture.

In 2020 Ye, T., et al., [23] design a DFF-Net for detecting railway traffic objects to stop railway accidents. The object detection engine takes the aforementioned anchor boxes as input and uses feature fusion submodules to enhance semantic information and achieve accurate object detection. In tests using a dataset of railway traffic, the suggested DFF-Net outperformed the most recent state-of-the-art detectors with mAP of 90.12%.

In 2021, Wisultschew, C., et al. [24] designed and implemented an edge IoT hardware platform for locating and tracking things on railway tracks. An extremely detailed point-of-cloud map is produced with the help of a low-resolution 16-channel 3D camera and LIDAR sensor. Models are quantitatively validated by implementing them on automobile roads and simulating railway level crossings.

In 2023, Meng, C., et al., [25] introduced the SDRC-YOLO to an efficient railway object intrusion detection system by employing a lightweight up sampling operator and convolution kernel. Simulated results show that SDRC-YOLO increases mean average accuracy (mAP) on datasets RS and Pascal VOC 2012 by 2.8% and 1.8% respectively. Small targets are more accurately identified than with the baseline YOLOv5s method.

From these literature review, the efficacy of the current approach is hampered by its reliance on the time-consuming process of detecting the objects. The majority of the studies mentioned above rely on single detection as input, which lowers performance rates. The development of automatic signal decomposition and categorization is necessary to address the aforementioned challenges. In this work, deep learning based FOD-YOLO net for improving the detection accuracy for detecting objects and faults in railway track for prevent accidents.

3 Proposed Methodology

In this section, a novel FOD-YOLO net is presented to detect the objects and faults in the railway tracks to avoid collisions and accidents. The investigations are performed with a set of pre-enrolled images that are openly available. Figure 1 illustrates the overall workflow of the proposed model.

Fig. 1

Schematic representation of the proposed methodology.

3.1 Data pre-processing

Initially, the gathered video frames are converted into images and these images are denoising for lessening the noise distortions. The collected images are denoised through AGS filter to deduce the noise distortions. The image pre-processing method that manipulates the intensity range inside sub-blocks by locally adjusting the intensity by using Adaptive Gaussian star (AGS) filter. A Gaussian filter is a low pass filter used to blur portions of the images and reduce distortion (high frequency elements). In an amplitude spectrum, the pattern of a typical periodic noise is sometimes viewed as a star. In order to form the shape of a star, two 2D elliptical filters were positioned perpendicular to each other. AGS filters are represented in the frequency domain by the following equation (1), $\begin{matrix} A (x, y) = \\ {\begin{matrix} \begin{matrix} V_{1} (x_{1}, y_{1}) + V_{2} (x_{2}, y_{2}) & {ifx}_{1} \neq x_{2} {andy}_{1} \neq y_{2} \end{matrix} \\ \begin{matrix} max (V_{1} (x_{1}, y_{1}), V_{2} (x_{2}, y_{2})) & {ifx}_{1} = x_{2} {andy}_{1} = y_{2} \end{matrix} \end{matrix} \end{matrix}$ (1)

The following are the parameters of AGSF: $α_{i 1, 2} = D . \min (\sqrt{{(x_{i 1, 2} - x_{ir})}^{2} + {(y_{i 1, 2} - y_{ir})}^{2}})$ (2) $β_{i 1, 2} = D . \max (\sqrt{{(x_{i 1, 2} - x_{ir})}^{2} + {(y_{i 1, 2} - y_{ir})}^{2}})$ (3) where r signifies the neighboring pixels with the number of $i_{1, 2}^{th}$ noisy area. From equation (2) and (3), α_i1,2 and β_i1,2 are ED times the maximum and minimum value of Euclidean distances respectively.

3.2 FOD-YOLO network

The noise-free images are given as an input to the proposed FOD-YOLO net to accurate detection of objects on tracks and faults in fasteners. The FOD-YOLO net is an advanced version of YOLOv8, in which two YOLO networks are used for object detection on tracks and fault detection in railway track. In the FOD-YOLO net, there are two different modified structures are used for efficient detection of objects and faults in the railway tracks. The structure of the proposed FOD-YOLO net is displayed in Fig. 2. In the Object detection phase, the self-attention-based Cross-Stage-Partial DarkNet53 (SA-CSPDN-53) is used a modified backbone structure to detection the objects on the railway track. Controlling deeper networks with gradient paths was enable the SA-CSPDN-53 structure to learn and converge successfully. Several kinds of convolutions are used to create the CSPDN-53 structure. Simultaneously, in the fault detection phase, squeeze and excitation based EfficientNet (SE- EfficientNet) is introduced with the Gelu activation function for efficient fault detection in railway tracks. During the convolution layer, with SE module is utilized. An adaptive SE module selects the size of the 1D convolutional filter, and the dimension is preserved during local cross-channel interactions by lessening network complexity. The proposed FOD-YOLO ideally balances speed and precision to detect the object and faults in railway track.

Fig. 2

Architecture of the proposed FOD-YOLO net.

i) Object detection on Railway track

In this phase, Cross-Stage-Partial DarkNet53 (CSPDN-53) is used to detection the objects on the railway track. CSPDN-53 divides the feature maps into two parts, processes them separately through two different branches of the network, and then merges them. This helps in improving information flow and enables better feature reuse across different stages of the network. Convolutional layers in CSPDN-53 involve the mathematical operation of convolution. CSPDN-53 is the default backbone of the YOLO v8. This backbone is a development of the backbone namely DarkNet53 which has added the Cross Stage Partial Connections (CSP) feature which integrates feature maps from the initial stage to the final stage of the network. CSP implementation reduces computation thereby outperforming other state-of-the-art backbone architectures. The structure of CSPDN-53 is shown in Fig. 2. The convolutional process is used to extract features from the input data using convolutional filters.

The MP module, Conv module, SPPCSPC module and three consecutive conv modules form the main network. Self-attention is highly flexible and allows the model to weigh the different parts of the input sequence. The primary purpose of self-attention mechanisms in CSPDN-53 is to enable the YOLO model to focus on different parts of the video sequence when making predictions for each frame in that sequence. The attention mechanism calculates a set of weights for each element in the sequence based on its relationship with other elements. The self-attention mechanism is also used with CSPDN-53 which helps in achieving high level of accuracy while the depth of the network increases. Applying convolution layers to extract the feature map from the input image, it utilizes the most pooling ‘n’ times the size of the input windows to create the feature set. First, three convolution operations are performed on an input feature map with various sizes. Different kernel sizes were used for max-pooling operations. The SPPCSPC engine can retrieve information about objects at multiple scales without scaling the feature map. The image is transmitted twice once to CSPDN-53 for feature extraction. The findings are produced last by the YOLO layer. The high accuracy and other good properties of the multibranch model are preserved in the final deployed model, as well as the high efficiency, which balances speed and accuracy and improves network performance. This model analyzes and identify objects of interest of varied sizes at various scales for object detection.

ii) Fault detection in Railway track

In this phase, a modified EfficientNet is introduced with the integration of squeeze and excitation (SE) module for efficient fault detection in railway track. Squeeze and excitation module [26] are used to enhance the efficiency of the yolo to model the dependencies between the channels of convolutional kernels. The architecture of SE-EfficientNet is made with the reversed bottleneck MB-Conv is displayed in Fig. 2. An MB-Conv block comprises with several layers that increase and decrease the channel. Connections are employed during bottlenecks with a smaller channel than extension layers to extract the relevant features as t_f. In contrast to normal layers with around k² factor, depth-separable convolution minimizes computation. The kernel size k_s indicates the height and width of the 2D-convolution windows. In SE-EfficientNet, the GeLU is used as the activation function. As a result of compound scaling, the following criteria are obtained, $\begin{matrix} depth : 𝔻 = α^{μ} \\ width : 𝕎 = β^{μ} \\ resolution : ℝ = γ^{μ} \end{matrix}$ (4) Where α, β, γ ⩾ 1, these are constant factors to evaluate the grid-search with the compound coefficient μ. In the standard convolutional block, the flops are correlated to $𝔻, 𝕎^{2}, ℝ^{2}$ . When the factors are scaled as shown in equation (4) around (α, β², γ²) ^φ (αβ²γ²) ^φ, the total flops of the EfficientNet increases. However, even more fallout can be achieved by extracting t_f features around a large model instantly with α, β, and γ; though, the search cost arbitrarily increases with larger designs.

Squeeze and excitation module

Squeeze and excitation module is introduced to increase the efficiency of the network for tuning the dependencies between channels of convolutional kernels. The squeeze process is applied to compress the feature maps U∑R^w×b×c2 with the spatial dimensions (w × b) that gives the channel-wise information F_sq with global average pooling. The input of the squeeze block has a convolution layer. Then by the usage of average pooling each channel is lessen into a single value. The ReLU operation produces non-linearity which follows the dense layer. Here the sigmoid functions act as a gating function. Additionally, the squeeze block is computationally light, which improves the capability of the network and lessens computations. $Z = F_{sq} (u_{c}) = \frac{1}{H \times W} \sum_{i = 1}^{H} \sum_{j = 1}^{w} u_{c (i, j)}$ (5)u_c refers the c^th feature maps of the prior convolution layer, W refers to width and H refers to height respectively. The extracted features are input of the excitation block for further processing. The extracted features from the squeeze block are utilized for identifying the dependencies between the channels that are entirely taken in the excitation block by accepting a gating process with a sigmoid activation as given in the equation (6). $s = F_{ex} (Z^{w}) = σ (g (w)) = σ (w_{2} δ (w_{1 Z}))$ (6)

The GeLU activation function is signified as δ. The learning parameters namely w1 and w2 regarding the non-linearity is aided by two fully connected layers. After rescaling, u_c along with the activation s_c. $X_{c} = F (u_{c}, s_{c}) = s_{c} \cdot u_{c}$ (7) Where X_c means the final result. The excitation block gathers global feature descriptors in each place after acquiring raw features from the squeeze block. Feature maps that are more significant than their counterparts are given more attention. Following a locally connected2D layer, a sigmoid activation function is used to learn the weights. Furthermore, deep learning algorithms use attention blocks, a layer of neural networks, to focus on certain data points based on weights assigned to different regions. It uses each encoder step and decoder preview step to align or calculate attention weights. It multiplies the results from each encoder step with globally aligned weights to find the content value to feed to the head of the YOLO network. The accuracy is boosted by altering the hyper-parameters like epoch and learning rate for efficient detection of faults in railway track.

3.3 Fuzzy based Decision making

The images of railway track compared and an alert message will be sent to the loco-pilot and also to nearby control rooms if an object or defect is detected using fuzzy logic. Mamdani fuzzy is made with a minimal number of rules for altering the loco pilot based on the detection results. The fuzzy logic controller was used to convert a linguistic control technique based on expert data into an independent control approach. A MFIS is consists of five building blocks: a database, a rule base, a decision-making unit, a fuzzification interface unit, and a defuzzification interface unit. The fuzzification unit changes the target value of the four detection fallouts (object, no-object, fault, no-fault) into fuzzy measures. The fuzzy rules are mathematically represented as, $F = {\begin{matrix} if (D_{x}, D_{y}^{'}) ⩾ if (D_{x}^{'}, D_{y}^{'}) ⩾ 1, \\ otherwise 0 \end{matrix}$ (8) Where F is defined fuzzy decision-making variable, D_x and D_y are the nothing detected in tracks, $D_{x}^{'}$ and $D_{y}^{'}$ are the detection of objects and faults in tracks. When any fault or object in the track that will alert the loco pilot. Four fuzzy rules make up the decision-making unit, which can warn the loco pilot based on inputs such as object, no-object, fault, and no-fault in railway tracks. It uses a Mamdani Inference system with Gauss membership functions (MF) and accepts inputs between 0 and 1. Four fuzzy rules are crucial in the detection of objects and faults.

Fuzzy rules

If object detect on track and fault detect in track then alert the loco pilot

If object did not detect on track and fault detect in track then alert the loco pilot

If object detect on track and fault did not detect in track then alert the loco pilot

If object did not detect on track and fault did not detect in track then did not alert the loco pilot.

The fuzzy inference process analyzes the values of an input and allocates values to the fallouts based on a set of rules. A fuzzy interference regulation is used to the results of the YOLO network. To prevent accidents, two membership functions are incorporated into the fuzzy system to produce a result for warning the loco pilot and control room.

4 Results and discussion

In this section, the experimental setup of the proposed FOD-YOLO was implemented using MATLAB 2020b, and the ensuing experimental findings are represented. The proposed FOD-YOLO net was assessed with various measures viz., specificity, precision, F1 score, recall, and accuracy measures on the collected dataset.

4.1 Dataset acquisition

In this work, an annotated subset of 7500 training images from the public RailSem19 dataset as the training set, eliminating tramway images [27]. A total of 26,810 images were produced from these images by selecting different locations of interest and acquiring them at 224×224 resolution. Similarly, 12,81,167 images from the freely accessible ImageNet dataset [28] as additional non-railway image are also gathered for experiments. ImageNet dataset constitutes already labelled data for the better model for testing the query image type. Normally, the images are labelled by annotation and this ImageNet dataset consists of many categorizes. So, the re-labeling ImageNet training data is necessary. Original ImageNet annotation is a single label (“crack”), whereas the image contains multiple ImageNet categories (“crack”, “human”, and “animal”).

The official annotated test set from RailSem19 is used in the evaluation data collection Fishy-Rails. The resulting 7,142 images and segmentations are the result of embedding the images with objects from the public Pascal VOC object detection dataset [29] and fasteners fault detection dataset [30]. The sample images from the dataset are exposed in Fig. 3.

Fig. 3

Sample images gathered from the different dataset.

Figure 4 displays the visualization results of the proposed FOD-YOLO net for the gathered images. The input images (column:1) are pre-processed using AGS filter to eliminate the noise distortions as visualized in column:2. These pre-processed images are fed using deep learning-based YOLO V8 network for detecting the faults and objects in the railway track as displayed in column:3.

Fig. 4

Experimental results of the proposed FOD-YOLO net.

4.2 Performance analysis

The proposed FOD-YOLO model was assessed by approximately metrics accuracy, precision, specificity, recall and mAP. $Specificity = \frac{T_{neg}}{T_{neg} + F_{pos}}$ (9) $Precision = \frac{T_{pos}}{T_{pos} + F_{pos}}$ (10) $Recall = \frac{T_{pos}}{T_{pos} + F_{neg}}$ (11) $F 1 score = 2 (\frac{Precision \times Recall}{Precision + Recall})$ (12) $Accuracy = \frac{T_{pos} + T_{neg}}{Totalno . ofsamples}$ (13)

Precision and recall are used as significant indicators to measure the performance of object identification model. To evaluate the objective performance of the FOD-YOLO model, Mean Average Precisions (mAP) is calculated using equation (14). $mAP = \frac{\sum_{m} \int_{0}^{1} p (r) dr}{m}$ (14) $fps = \frac{1}{T}$ (15)

The frames per second (FPS) of an object detection algorithm signifies the number of images frames the algorithm can process per second. The parameter FPS is calculated by taking the reciprocal of the processing time in seconds. According to the parameters given for the sample images, the proposed FOD-YOLO evaluation of performance is calculated as shown in

The findings attained by the proposed YOLO network for the collected datasets i.e., for object detection and for fault detection is illustrated in Table 1. The proposed FOD-YOLO net reaches the average accuracy of 98.14% and mAP of 96.85% for the gathered dataset. Figure 5 displays the performance of the FOD-YOLO based the network parameters. Additionally, the proposed FOD-YOLO net acquires the overall precision, recall, specificity and F1 score of 97.84%, 95.94%, 98.16% and 96.98% respectively.Table 1.

Table 1

Performance scrutiny of the proposed FOD-YOLO network

Dataset	Classes	Accuracy	F1 score	mAP
Object detection	Normal	99.07	98.07	98.8
	Human	97.18	95.74	95.5
	Animal	98.12	96.25	96.4
Fault detection	Normal	99.04	97.04	98.3
	Crack	97.14	97.87	96.5
	Friction	99.25	98.03	95.6

Fig. 5

Performance analysis of the proposed FOD-YOLO net for detecting (a) objects on track (b) faults in track.

Figure 5 (a) displays the efficiency analysis of the proposed FOD-YOLO net for detecting the objects on the railway track. Figure 5 (b) displays the efficacy analysis of the proposed FOD-YOLO net for the detection of fault on the railway track.

The accuracy graph of the proposed of the proposed FOD-YOLO net is presented in Fig. 6(a) as the accuracy value on the vertical-axis and the number of epochs on the horizontal axis. The epoch and loss range in Fig. 6(b) indicates that once the epochs are elevated, the loss of FOD-YOLO net declines. The proposed FOD-YOLO net achieves high levels of accuracy in terms of detecting the objects and faults in the tracks.

Fig. 6

(a) Accuracy curve (b) Loss curve of the proposed of FOD-YOLO net.

4.3 Comparative analysis

The efficiency of traditional networks was measured for validating that the results of the proposed FOD-YOLO net attain high accuracy. This assessment was conducted in contrast with the proposed YOLO network and conventional networks SSD, Mask RCNN, and Faster RCNN.

The proposed FOD-YOLO model is contrasted with three current strategies in a comparative analysis. The effectiveness of DL models as a whole is contrasted with the proposed approach in Table 2. The FOD-YOLO network is contrasted with DL techniques such as YOLO v8, Mask RCNN, Faster RCNN, and SSD. Performance evaluation was based on various metrics like precision, specificity, recall, and accuracy of the DL technique. However, conventional networks have performed less well than the proposed networks. YOLO network progresses the detection accuracy by 8.57%, 4.82%, 5.44%, and 1.23% better than SSD, Faster R-CNN, Mask R-CNN and Yolo v8 respectively.

Table 2
Comparison of conventional object detection networks

Networks Accuracy F1 score Specificity Precision Recall

SSD 89.72 87.04 89.51 88.23 89.08

Faster R-CNN 92.08 89.14 92.87 93.51 92.58

Mask R-CNN 93.40 91.02 91.23 88.98 93.87

YOLO 96.93 95.64 90.23 87.59 91.49

FOD-YOLO 98.14 96.98 98.16 97.84 95.94

Networks	Accuracy	F1 score	Specificity	Precision	Recall
SSD	89.72	87.04	89.51	88.23	89.08
Faster R-CNN	92.08	89.14	92.87	93.51	92.58
Mask R-CNN	93.40	91.02	91.23	88.98	93.87
YOLO	96.93	95.64	90.23	87.59	91.49
FOD-YOLO	98.14	96.98	98.16	97.84	95.94

Figure 7 displays the detection results of the proposed FOD-YOLO performs faster and attains best results in object detection. Though, the conventional object detection models are not performed well when compared to the proposed FOD-YOLO. From the above comparison the proposed FOD-YOLO yields higher detection than the conventional approaches such as SSD, Faster RCNN, Mask RCNN and YOLOv8. The proposed FOD-YOLO increases system performance and reduces false positive rate by achieving precise accuracy of 0.99. So, the predicted fallouts of the proposed FOD-YOLO are highly reliable for detecting the objects and faults in the railway tracks.

Fig. 7

Detection comparison results of conventional object detection network.

Figure 8 summarizes the experimental setup with the test samples from the gathered to estimate the efficiency of prior frameworks based on different weather conditions such as normal, night, rain and snow. A performance measure was applied to compared the proposed with the prior models based on detection accuracy for different weather conditions. The FOD-YOLO net progresses the overall accuracy range by 8.57%, 6.17%, 5.19% and 1.24% better than SSD, Faster RCNN, Mask RCNN and YOLO respectively for normal conditions. Moreover, The FOD-YOLO net progresses the overall accuracy range by 14.7%, 13.0%, 9.65% and 7.51% better than SSD, Faster RCNN, Mask RCNN and YOLO respectively for rainy weather conditions. Though, the prior networks not achieved better fallouts in contrast to the proposed FOD-YOLO net in different weather conditions. However, the FOD-YOLO net does not achieve good results for rain, snow and night time.

Fig. 8

Accuracy comparison of traditional object detection techniques.

Table 3 summarizes the experimental setup with the test samples from the gathered to estimate the efficiency of prior frameworks. A performance measure was applied to compared the proposed with the prior models based on detection accuracy. The FOD-YOLO net progresses the overall accuracy range by 4.72%, 0.73%, 13.18% and 5.44% better than YOLOv5s-VF [16], YOLO-GD [17], 2D-SSA + Deep network [18] and Faster RCNN [22] respectively. Though, the prior networks not achieved better fallouts in contrast to the proposed FOD-YOLO net. The FOD-YOLO net achieves a high accuracy value of 98.14% with minimal number of GFLOPS, demonstrating excellent object detection performance. However, the superior accuracy comes at the expense of computational efficiency. Its counterpart, the FOD-YOLO net, offers a faster demonstrating its proficiency in executing object detection tasks.

Table 3

Accuracy comparison of prior techniques and Proposed FOD-YOLO net

Authors	Techniques	Accuracy	Parameter	GFLOPs
Wang, et al., (2022) [16]	YOLOv5s-VF	93.5%	2948255	8.7
Yue, et al., (2022) [17]	YOLO-GD	97.42%	2785450	8.5
Kapoor, et al., (2022) [18]	2D-SSA + Deep network	85.2%	3247591	11.2
Liu, et al., (2020) [22]	Faster R-CNN	92.8%	3107548	9.4
Proposed	FOD-YOLO net	98.14%	2745170	7.8

The proposed FOD-YOLO increases mean average precision (mAP) by 3.41% and improves frames per second (FPS) by 12.5%, as seen in Fig. 9 when compared to the YOLO-GD [17]. The proposed FOD-YOLO achieves significant gains in detection speed and accuracy when compared with YOLOv5s-VF [16], 2D-SSA + deep network [18] and Faster R-CNN [22]. Figure 7 shows the comparison of the proposed approach outperforms the current techniques. By comparing FPS and mAP across several networks, the FOD-YOLO has successfully achieved a good balance between speed and accuracy. From this comparison, our FOD-YOLO net is superior than other methods. So, the detected results of the proposed FOD-YOLO net are highly reliable for the detection of objects and faults in railway tracks.

Fig. 9

Performance comparison based on mAP and FPS.

4.4 Ablation study

An ablation study was made to measure the effectiveness of proposed FOD-YOLO based on the collected dataset. As a result of this experiment, the two modification was done in the backbone of the FOD-YOLO network for detecting the defects and objects in the railway tracks. The effectiveness of the proposed model was evaluated by removing one by one for the assessment. According to a comparison, the recognition accuracy by removing these modules was illustrated in Table 4.

Table 4
Ablation study of the proposed FOD-YOLO model

Parameters With With With SA-CSPDN and

SA-CSPDarkNet SE-EfficientNet SE-EfficientNet

Accuracy 95.21% 96. 48% 98.14%

Precision 94.08% 95.71% 97.84%

Recall 92.28% 93.53% 95.94%

Parameters	With	With	With SA-CSPDN and
Accuracy	95.21%	96. 48%	98.14%
Precision	94.08%	95.71%	97.84%
Recall	92.28%	93.53%	95.94%

The proposed FOD-YOLO network was examined in terms of the performance metrics by removing SA-CSPDNand SE-EfficientNet. In this comparison, the proposed model attains better accuracy (96.48%) with SE-EfficientNet which comparatively high than the with SA-CSPDN (95.21%). Similarly, all the metrics are varied as well as accuracy. The EfficientNet is more appropriate for assessing identifying featural changes. On the other hand, the combination of these two modules yields high level of recognition accuracy (98.14%). From this ablation analysis, the proposed FOD-YOLO network model with these modules attains high accuracy in the identification of faults and objects in the railway tracks.

5 Conclusion

This work presents novel FOD-YOLO net for accurate detection of objects on tracks and detect the faults in trach fasteners. The deep learning based YOLOv8 has a backbone with modified version of the CSPDN53 architecture for detecting the objects on the track. The head of YOLOv8 consists of EfficientNet with multiple convolutional layers for detecting any defect in the track fasteners. Objects in an image are identified using a set of fully connected layers that predict bounding boxes, objectness scores, and class probabilities. The detection findings demonstrate that the proposed FOD-YOLO net performs better by visual records from the gathered dataset. From the experimental analysis, the proposed FOD-YOLO net yields the overall accuracy of 98.14% for efficient detection of objects and faults. The proposed YOLO was contrasted with the tradition object detectors in which it increases the overall accuracy by 4.82%, 5.44%, and 1.23% better than SSD, Faster RCNN, and Mask RCNN respectively. The FOD-YOLO net improves the overall accuracy range by 5.44%, 4.72%, 0.73%, and 13.18% better than FOD-YOLO net, YOLOv5s-VF, YOLO-GD, and 2D-SSA + Deep network respectively. However, the FOD-YOLO net does not achieve good results for different weather conditions namely rain, snow and night time due to the limited number of datasets. In future, the proposed model will be trained and tested with different datasets. Moreover, the proposed model will be extended with the advanced deep learning algorithms with IoT devices for practical implementation to avoid the railway track accidents.

References

Meimetis

, Daramouskas

, Perikos

, Hatzilygeroudis

Real-time multiple object tracking using deep learning methods, Neural Computing and Applications 35 (2023), pp. 89–118.

Chandan

, Jain

Real time object detection and tracking using Deep Learning and OpenCV. In 2018 International Conference on inventive research in computing applications (ICIRCA) (2018), pp. 1305–1308.

Micheal

A.A.

, Vani

, Sanjeevi

, Lin

C.H.

Object detection and tracking with UAV data using deep learning, Journal of the Indian Society of Remote Sensing 49(2021), pp. 463–469.

Shreyas

, Sheth

M.H.

3D Object detection and tracking methods using deep learning for computer vision applications. In 2021 International Conference on Recent Trends on Electronics, Information, Communication & Technology (RTEICT) (2021), pp. 735–738.

, Zhang

, Li

, Zhou

, Cao

, Xiao

Object detection and tracking under complex environment using deep learning-based LPM, IET Computer Vision 13 (2019), pp. 157–164.

Mahamuni

C.V.

, Jalauddin

Z.M.

Intrusion monitoring in military surveillance applications using wireless sensor networks (WSNs) with deep learning for multiple object detection and tracking. In 2021 International Conference on Control, Automation, Power and Signal Processing (CAPS) (2021), pp. 1–6.

Rampriya

R.S.

, Suganya

, Sabarinathan , Ganesan

, Prathiksha

, Rakini

Object Detection in Railway Track using Deep Learning Techniques. In International Conference on Computational Techniques and Applications, Singapore: Springer Nature Singapore (2021), pp. 107–115.

, Gao

, Yuan

, Tang

, Wei

Real-time obstacle detection over rails using deep convolutional neural network. In 2019 IEEE Intelligent Transportation Systems Conference (ITSC) (2019), pp. 007–1012.

Wang

, Yang

, Tsui

K.L.

Real-time detection of railway track component via one-stage deep learning networks, Sensors 20 (2020), pp. 4325.

10.

, Li

, Chen

, Miao

, Li

, Shan

, Ren

Obstacle detection in dangerous railway track areas by a convolutional neural network, Measurement Science and Technology 32 (2021), pp. 105401.

11.

Karagiannis

, Olsen

, Pedersen

Deep learning for detection of railway signs and signals. In Advances in Computer Vision: Proceedings of the 2019 Computer Vision Conference (CVC) 11 (2020), pp. 1–15, Springer International Publishing.

12.

Guan

, Jia

, Xie

, Yin

A lightweight framework for obstacle detection in the railway image based on fast region proposal and improved YOLO-tiny network, IEEE Transactions on Instrumentation and Measurement 71 (2022), pp. 1–16.

13.

, Zhao

, Wang

, Zhou

, Gao

A stable lightweight and adaptive feature enhanced convolution neural network for efficient railway transit object detection, IEEE Transactions on Intelligent Transportation Systems 23 (2022), pp. 17952–17965.

14.

Guo

, Qian

, Shi

Real-time railroad track components inspection based on the improved YOLOv4 framework, Automation in Construction 125 (2021), pp. 103596.

15.

, Qiu

, Miao

, Zou

, Li

, Ren

, Shen

Improved Mask R-CNN for obstacle detection of rail transit, Measurement 190 (2022), pp. 110728.

16.

Wang

, Li

, Zhu

, Zhao

Detection of Surface Defects on Railway Tracks Based on Deep Learning, IEEE Access 10 (2022), pp. 126451–126465.

17.

Yue

, Li

, Shimizu

, Kawamura

, Meng

YOLO-GD: a deep learning-based object detection algorithm for empty-dish recycling robots, Machines 10 (2022), pp. 294.

18.

Kapoor

, Goel

, Sharma

An intelligent railway surveillance framework based on recognition of object and railway track using deep learning, Multimedia Tools and Applications 81 (2022), pp. 21083–21109.

19.

Rahman

F.U.

, Ahmed

M.T.

, Hasan

M.M.

, Jahan

Real-Time Obstacle Detection Over Railway Track using Deep Neural Networks, Procedia Computer Science 215 (2022), pp. 289–298.

20.

, Wu

, Wang

, Chen

, Shi

, Tan

Railway track circuit signal state check using object detection. In Journal of Physics: Conference Series 1486 (2020), pp. 042018, IOP Publishing.

21.

Ranyal

, Jain

Detection of rail fasteners from aerial images using deep convolution neural networks. In IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium (2020), pp. 2217–2220.

22.

Liu

, Liu

, Nunez

, Han

Unified deep learning architecture for the detection of all catenary support components, IEEE Access 8 (2020), pp. 17049–17059.

23.

, Zhang

, Liu

Railway traffic object detection using differential feature fusion convolution neural network, IEEE Transactions on Intelligent Transportation Systems 22 (2020), pp. 1375–1387.

24.

Wisultschew

, Mujica

, Lanza-Gutierrez

J.M.

and Portilla

, 3D-LIDAR based object detection and tracking on the edge of IoT for railway level crossing, IEEE Access 9 (2021), 35718–35729.

25.

Meng

, Wang

, Shi

, Gao

, Tao

and Wei

, SDRC-YOLO: A Novel Foreign Object Intrusion Detection Algorithm in Railway Scenarios, Electronics 12(5) (2023), 1256.

26.

, Shen

and Sun

, Squeeze-and-excitation networks. In, Proceedings of the IEEE conference on computer vision and pattern recognition (2018), 7132–7141.

27.

Zendel

, Murschitz

, Zeilinger

, Steininger

, Abbasi

, Beleznai.

Railsem19: A dataset for semantic rail scene understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, (2019), pp. 0-0.

28.

Deng

, Dong

, Socher

, Li

L.J.

, Li

, Fei-Fei.

Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition (2009), pp. 248–255.

29.

Everingham

, Eslami

S.A.

, Van Gool

, Williams

C.K.

, Winn

, Zisserman

The pascal visual object classes challenge: A retrospective, International Journal of Computer Vision 111 (2015), pp. 98–136.

30.

Lin

Y.W.

, Hsieh

C.C.

, Huang

W.H.

, Hsieh

S.L.

, Hung

W.H.

Railway track fastener’s fault detection using deep learning. In 2019 IEEE Eurasia Conference on IOT, Communication and Engineering (ECICE), Yunlin, Taiwan, (2019), pp. 187–190.

FOD-YOLO NET: Fasteners fault and object detection in railway tracks using deep yolo network 1

Abstract

Keywords

1 Introduction

2 Literature Survey

3 Proposed Methodology

4.1 Dataset acquisition

Table 4 Ablation study of the proposed FOD-YOLO model Parameters With With With SA-CSPDN and SA-CSPDarkNet SE-EfficientNet SE-EfficientNet Accuracy 95.21% 96. 48% 98.14% Precision 94.08% 95.71% 97.84% Recall 92.28% 93.53% 95.94%

References

Table 4
Ablation study of the proposed FOD-YOLO model

Parameters With With With SA-CSPDN and

SA-CSPDarkNet SE-EfficientNet SE-EfficientNet

Accuracy 95.21% 96. 48% 98.14%

Precision 94.08% 95.71% 97.84%

Recall 92.28% 93.53% 95.94%