A review of the technological developments for interlocking at level crossing

Obstacle detection sensors

In literature, ¹² the author describes Inductive Loops as “probably the most common form of vehicle detection”. It is also the earliest proposed solution to detect an object. The Inductive Loops technology has a wire embedded inside the road tarmac in a loop-like shape, which creates an electromagnetic field. The loop acts as an electrical circuit with variable frequencies ranging from 10 kHz to 200 kHz depending on the model used. The object e.g. vehicle when passed through the effective range, it induces eddy current and decreases the inductance. The decreased inductance actuates the electronics, which could be used to produce a simplified output. A detailed discussion on Inductive Loops installation and work principles is given in literature. ¹³

Since the working principle of Inductive Loops depends on inducing the electric circuits, the detectability for objects, which are made up of more composite and non-inductive materials is reduced. In the case of system failure, the embedded system is abandoned and a new system is installed, which can cause significant disruption in areas of high traffic volume and increase the risk of an accident when in proximity to rail infrastructure. Extreme weather conditions e.g. heavy rain or fog can adversely affect the system and result in low detectability. As such, inefficiency in poor weather conditions and extensive wear and tear make Inductive Loops a less favourable choice for Level Crossing application. ¹⁴

Another technique used for object detection mentioned in literature ¹¹ is using strain gauges or piezometers, which measures the deformation of a material. A strain gauge can be installed inside a crossing area, which can detect the deformation of the crossing decking when a vehicle travels over it. The object's weight causes the decking and attached piezometer to deform, which changes its conductivity and hence signals the object’s presence above the sensor. Multiple piezometers can be used to interpret the direction of the object. Because piezometers are calibrated to detect vehicles and pedestrians, they may not be able to detect lighter objects, including children, which poses a significant risk to a Level Crossing. Made from rugged, weatherproof semiconductor materials, piezometers can perform better than Inductive Loops, however, the installation of these systems in the crossing decking makes them difficult to install and maintain.

Another method for obstacle detection at a Level Crossing is having two stereo cameras in one housing as mentioned in literature. ¹⁵ These two cameras are used to create a 3D representation of the scene is placed at a specific angle to cover an entire area of Level Crossing. These stereo cameras are integrated with an image-processing unit, where the optical axis, distortions, brightness and other errors are corrected. ¹⁶ From such corrections, the main advantage is to avoid false detection from shadows as mentioned in literature. ¹⁵ The main disadvantage of such a system is the cost of installation and its inability to detect where light availability is low.

The work in literature ¹⁷ proposed the use of a thermal camera, which creates an image based on the difference in temperature. The thermal imaging relies on the phenomenon of black-body radiation produced by all objects with a temperature above absolute zero, therefore, the system is unaffected with the change in weather or varying light conditions. ¹⁸ Thermal Imaging provides a high-quality image, easy installation and more lens options. However, the system is unable to detect an object, which is inert or produce the same radiation as the background scene. ¹⁹

Another technique mentioned in literature^12,15 is ultrasonic detectors and Optical Beam method. The ultrasonic detectors use the transmission of an ultrasonic pulse and find the difference between transmitted and reflected pulse to detect the presence of moving or stationary object. The effectiveness of an ultrasonic detector relies on its working frequency and size, for instance, the ultrasonic detector with a frequency range of 40 kHz can detect an obstacle about 7 meters away. The only disadvantage of such systems mentioned is its price, cost of installation and its sensitivity in extreme weather conditions. ²⁰ The Optical beam works on the same principle, where an optical beam is transmitted from one end of a Level Crossing and received on the other end. Any interference would detect the presence of an object. Even though the optical beam is easy to replace, the system still has high installation cost and requires multiple emitters to cover the whole area of a Level Crossing. If electrical waves are passed between the transmitter and emitter, the working principle is the same but it would not be able to detect any pedestrian.^11,12

Another technique for obstacle detection mentioned in literature ²¹ is laser scanner for application in a Level Crossing area, which relies on the “the use of dense 3Dpoint clouds produced by a tilting 2D laser scanner”. This system would not be dependent on light conditions and object appearance as it is relying on binocular vision. The working principle of laser is the same as beam or SONAR, where it requires the emitter and receiver. The affectability of the laser system is reduced in adverse weather condition and since it requires multiple laser systems to cover the entire region of a Level Crossing the maintenance cost is high. In literature ²² the use of “Single Laser Rangefinders” is proposed, where the emitter and receiver are in one unit. Such systems are less complex but are unable to section areas of Level Crossing.

Another technique proposed was 3-D Laser Radar in literature. ²³ It mentions the working principle as “A 3-D laser radar emits a laser pulse to an object, and measures the time that it takes for reflected laser to return to the radar (time-of-flight method) to acquire a distance to that object”. The work in literature^24,25 proposed LiDAR, which works on the same principle of emitting near-infrared light. The reflected light allows the determination of speed, position and direction of an object. They both work on the principle of “Time of Flight (ToF)”, which can be used to determine the distance of an object. The LiDAR is compact, light and robust with high resolution, high range measuring accuracy and high scan rate. This allows the sensing system to detect even smaller objects such as a child. The initial cost and small life cycle of such systems is a limiting factor and it is hard to classify objects from such systems.

The use of “Camera” has been of much interest to many researchers for the application of object detection because of the easy installation and accessibility of the data for post-processing. In literature ¹² it is stated for video sensing system that “An automatic process must be established to only transmit relevant information to the control room and train driver like the presence of an obstacle in the crossing zone at a critical time”. The video sensing system is also proposed in literature ²⁶ and many different imaging processing techniques are applied to the sensing systems for analysing and automatically detecting objects in literature. ²⁷ The use of Closed Circuit Television (CCTV) can replace the manual operator at a Level Crossing, hence it is the most preferable choice among researchers.

The drawback of such a sensing system is that any video system requires a sufficient amount of light to work effectively. During the adverse weather conditions such as heavy rain or fog, the detectability of such systems is reduced. The ability of video systems to distinguish objects and their properties in any given scene is a difficult task and requires extensive algorithms to process and analyse. For example, the video system is unable to differentiate between cardboard and a human child at a Level Crossing and will cause some level of threat. These situations require the intervention of an operator. ²⁸ Usability of such systems is limited by an increase in false positives that is when an object is incorrectly detected. Most false positives are caused by a dynamic background, which resembles the real-world situation. This is not a failure of the algorithm, but a misunderstanding of its capability to differentiate risk between objects. However, this paper will introduce the Deep Learning algorithm, which enables the system to learn the representation on its own more effectively, hence, the model will be able to distinguish the object in a dynamic environment.

RADAR is another area of interest for many researchers and is mostly used obstacle detector at Level Crossings within the UK. The working principle of RADAR is same as emission and receiving of a signal e.g. radio waves, the received echo suggests the presence of an obstacle. Different types of RADAR are available whose working principle remains the same but differ in their post-processing to obtain information such as distance. For example, the pulse RADAR uses Time-of-Flight method and Frequency modulated RADAR determines the distance from the difference in the emitted and received frequency of an echo. ²⁹

In literature ³⁰ the implementation of RADAR in four-quadrant gate Level Crossing is discussed in detail. It discusses the use of Timed and Dynamic method of operation, where Timed method delay the closing of the barrier if some vehicle is present and in Dynamic the operation depends on the presence of the vehicle within the “Minimum Track Clearance Distance (MTCD).” Also in literature, ³¹ the RADAR is proposed as the main sensing system to detect obstacle with three safety configuration. In general, the RADAR is cheap to install or maintain and have a high life expectancy of approximately 10 years. Most of these RADAR works in the range of Gigahertz wavelength e.g. 24 GHz, which makes the system more robust in different environmental conditions. It can easily detect small and stationary objects at Level Crossings, which pose the most threat the crossing’s safety. ³²

A single type of sensor cannot provide sufficient information about the object, which is required for application at Level Crossings. For example, the functionality of video camera-based systems is limited by the availability of light and RADAR is unable to classify between cardboard and a child. Given the limitations of using a single type of sensor, the use of two or more sensors, will add another layer of resilience and therefore overcome the shortcomings of a single sensor.

In literature ³³ the fusion of colour Cameras and LiDAR is suggested, where colour Cameras can recognize distinctive object because of their colours and texture using different post-processing algorithms and LiDAR can discriminate such objects. The underlying issue with colour cameras is their inherent ambiguity due to the varying reflectivity of an object such as dry grass and soil. Also, the effect of the illumination spectrum on the perceived colour and the chromatic shift due to the atmosphere pose additional problems. The dynamic environment of a Level Crossing because of growing vegetation reduces the effectiveness of a Camera system. To overcome the mentioned problems, the LiDAR is proposed which can discriminate the moving vegetation from other smooth surfaces e.g. rocks or tree trunks. The LiDAR can eliminate the shadow effects as well as complimenting the Camera system more effectively. The problem with such fusion is the cost of colour Camera and LiDAR, particularly when two Camera are required to cover an entire area of a Level Crossing. The cost of LiDAR and the continuous maintenance make it a less preferable choice for obstacle detector at a Level Crossing.

In literature ³⁴ the fusion of Multi-beam and Sector Scanning Sonars is proposed. Sector Scanners provides a set of scan lines by mechanically moving the unit to obtain an image, hence some image compensation would be required to produce an accurate image. To avoid this limitation, the multi-beam is proposed which will scan the entire area reducing the need for stabilisation. The problem with such fusion is high false alarms since neither of these mentioned sensors can differentiate the objects. To avoid this problem, the use of traditional image processing techniques, image segmentation or feature extraction is adopted. These limitations and the cost of such systems makes it a very un-preferable choice for the application at Level Crossing.

The most commonly used sensor fusion for Level Crossing application is proposed in literature, ³⁵ which compliments the RADAR system with the LiDAR. RADAR is used as a primary obstacle detector to cover an entire area of Level Crossing using multiple-sensors. The data is post-processed to obtain further information about an object such as speed and distance. The RADAR, however, may not be able to detect small density objects. To avoid such limitation, the use of LiDAR is proposed, which can detect and obtain a detailed map e.g. 3 D scene of a scenario. The LiDAR can detect small density objects, which are missed from the RADAR. The LiDAR is normally within a glass unit; therefore, the presence of dust or small water droplets affects the functionality of the system. To overcome such problem regular maintenance is required, which further increases the cost of the system. Even though, the fusion of RADAR and LiDAR is a promising solution but have certain limitation such as the cost of the units and regular maintenance.

At this point, it is necessary to provide a summary of all the technologies mentioned below Table 2, scores each technology 1 (leading) – 5 (lagging) for each technology concerning certain parameters e.g. Cost, Maintenance etc.

OPEN IN VIEWER

Table 2.

A brief summary of all technologies discussed. The score 5 represents the worst choice, while 1 represents the best choice with respect to mentioned parameters e.g. cost or lifecycle.

	Equipment cost	Maintenance	Product Life cycle	False positive	Impact on weather
Inductive loops	5	5	4	4	4
Strain gauge/piezometers	5	5	4	4	3
CCTV	2	3	2	3	3
Stereo camera	3	4	2	3	3
Thermal camera	4	2	2	2	2
SONAR	2	4	3	3	2
Millimeter-wavelength Beam interference	4	4	1	3	4
Laser range finder beam (LRBF)	3	4	2	5	4
Light detection and ranging (LiDAR)	5	4	2	2	2
RADAR	3	2	1	1	2

Algorithms

The data received by any of the aforementioned sensors require some sort of post-processing before it can be used effectively in any given application. The post-processing techniques may include image-processing, feature extraction, image segmentation or foreground detection. The ultimate purpose of such an algorithm is to identify any present obstacle at a Level Crossing, which will either start the automated process of closing the barrier or warn the train driver for any threat.

Since one of the most commonly used sensing system at Level Crossing is Video Camera, which provides a continuous stream of live video to process, analyse and detect for any obstacle. The following discussion will highlight most of these algorithms that can detect any foreground from the background image. Among these mentioned algorithms, the most common method is to subtract the foreground from the background pixels. The problem with such an approach is that the background scene at a Level Crossing is a dynamic environment, where the algorithm must be able to adapt and update the background pixel continuously. The present work will also suggest the use of Deep Learning Techniques; a subset of Machine Learning Algorithms that automatically detects the representations from the objects and classify them with high accuracy.

The work proposed in literature ³⁶ introduces the algorithm that uses the “Single Gaussian-SG”, which fits the Gaussian probability density function on the last ‘n’ pixel’s value. To avoid fitting the density function for every new frame the mean and variance are calculated. To detect any foreground, the given value is compared with the threshold value. The work in literature ³⁷ discusses the use of Hue-Saturation-Value (HSV) compared with the traditional use of Colour Scheme (Red, Green and Blue- RGB), which is more robust to illumination changes and partial camouflage. However, the use of “Single Gaussian” is suitable for the indoor environment, where the change is minimal.

For outdoor application, the work in literature ³⁸ proposed the use of “Mixture of Gaussian-MOG”. Each pixel is modelled using “Mixture of Gaussian” and each model use one-line approximation to update the model. The optimal number of Gaussian filter is updated according to the given environment. Another work in literature ³⁹ models the background using three Gaussian filters representing the road, the vehicle and the shadows, where the intensity of each pixel is used for this categorisation; the darkest component is labelled as a shadow, from the remaining two the one with the highest variation is classified as a vehicle (obstacle) while the last one is classified as a road. The problem with such models is the noise and clutters in a given environment, therefore, the work in literature ³⁹ proposed the use of “Mixture of General Gaussian-MOGG”. The work in literature ^39,40 uses the Expectation-Maximisation (EM) and the K-Mean methods for MOGG model that adapt to the data more and is less sensitive to over-fitting. However, the model recovery time to false alarms is very slow and works poorly with a highly dynamic environment such as Level Crossing with high vegetations.

The work in literature^36,41 introduces another technique that uses the probability density function using the K estimator on a recent sample (N) of intensity values. From the normal Gaussian function, the background model is obtained using Kernel Estimation- KE Function. The foreground detection is possible using a threshold value, which is also used to obtain two more models; Short-Term background model and Long-Term background model. The Short-Term model eliminates persistent false-positives, whereas, the Long-Term model reduces the number of false-positive detection. The use of these two updated models makes the system more effective and reliable for dynamic background environments. ⁴²

Another algorithm technique is mentioned in literature, ⁴³ which suggested the use of “Subspace Learning-SL”. The Subspace Learning is divided into two main stages; Reconstructive and Discriminative. The Reconstructive Phase learns information from the data through unsupervised learning and updates the data in increments, which is beneficial for real-time applications. The Discriminative Phase requires supervision to separate the data using a linear transformation, which is used for classification. The training requires the data in batch, hence, the data should be available in advance which is a downside. Also, the model is computationally extensive and creates a lower-dimensional classification space. To avoid these limitations, different works proposed different solutions such as Subspace Learning using Principal Component Analysis (PCA), ⁴⁴ Subspace Learning using Incremental Non-Negative Matrix Factorisation,^45,46 or Subspace Learning using Incremental Rank Tensor. ⁴⁷

There are some algorithms which are used to separate the data used for classification. The work proposed in literature ⁴⁸ is “Support Vector Machine-SVM”. The SVM utilises a hyperplane in high dimensional space to separate the data by minimising the margin between the hyperplane to the data. The output is divided into two sets for unbiased training; 80% for SVM training and 20% for two parametric minimisations. If the probabilistic output is greater than the distance of hyperplane to the data, the output is considered as either a new moving foreground or newly updated background.

The model proposed in literature ⁴⁹ uses the “Support Vector Regression–SVR”, which model each pixel as a function of its intensity. The pixel values of each frame contain two outputs, one correspond to the intensity of the pixel and other value represent the confidence of that pixel belonging to the background. Once trained, the model uses a linear regression function where the output of SVR represents the confidence of each pixel belonging to the background. The SVR model keeps updating the background scene by labelling the pixel as background if the confidence lies within the threshold range.

The use of threshold value and probability function limits the functionality of the proposed algorithm. To overcome this issue ⁵⁰ proposed the use of “Support Vector Data Description-SVDD”, which uses the boundary feature. However, the optimisation is computationally intensitive and during maintenance, all SVDD must be recomputed.

The work in literature ⁵¹ proposed the use of Temporal differencing and Image Template Matching to detect and track the objects in a video sequence. The method is robust to background clutter and a slight change in the object’s presence would not significantly affect the performance.

Another technique is mentioned in literature ⁵² is the use of “Temporal Median Filter-TMF”. The TMF uses the median value of the last frame to model the background, which increases the stability of the background. However, the TMF is not able to accommodate the model in the rigorous statistical description and does not provide a deviation measure for the adoption of the threshold value.

The work in literature ⁵³ proposed the use of “Co-occurrence of Image Variation”, which works best if the background object belongs to the same object with least variation. The method can be divided into two main steps; the Learning Phase and Classification Phase. The Learning Phase calculates the eigenvector value and in Classification Phase the input value is computed along with its corresponding eigenimage variation value. If the calculated value is close to the calculated eigenvalue then the new and old pixel value is considered as a background model. However, Level Crossing is a dynamic environment with a variant distinct object, which makes the proposed algorithm a very poor choice.

A similar kind of algorithm is proposed in literature, ⁵⁴ which uses the “Principal Component Analysis-PCA”. The PCA is further divided into two main phases; training phase and online transformational phase. The training phase captures the dataset to obtain the transformation matrix U between the original and transformed space. In the online phase, the measure received are compared with the values obtained in the training phase. If the values differ significantly then it is considered as a foreground object. A more generalised form of PCA is “Independent Component Analysis-ICA”, which can differentiate background and foreground, particularly stationery and small objects. For outdoor applications, the use of colour ICA is proposed in literature.^55,56

Some other work proposed the fusion of two techniques such “K-Mean clustering” algorithm on a modified “Lucas-Kanade” method, ⁵⁷ “Lucas-Kanade” technique with “Haris Corner Points” in literature ⁵⁸ or “Histogram of Oriented Gradients-(HoG)” algorithm with a Support Vector Machine-SVM in literature. ⁵⁹

The discussed algorithms work on a principle of either subtracting the foreground pixel with the background pixel and detect the presence of an object using some threshold value or some supervised programming is required to train from the data for classification. These limitations affect the functionality of the system as the Level Crossing is a dynamic environment and most the time the required data is not sufficient for manual programming. To overcome these problems, the work in literature ⁶⁰ proposed the use of Deep Learning technique. Deep Learning is a subset of Machine Learning, which can automatically learn the representation from the images and classify the given categories without any supervision. It consists of multiple processing layers and can learn representation using backpropagation algorithms. Once enough representations are learned, very complex functions are learned used for different applications such as classification and detection.

The Deep Learning model learns features for the general-purpose avoiding the need for human-engineering, for example, the classification model learns small features like edges, orientation and location to more complex such as motifs or parts of objects with assembled features. From such learned models, it can detect and classify objects very effectively. The model does not require any supervision or regular increment of data to update the existing system. Another advantage of Deep Learning is its ability to integrate with other existing sensing systems such as RADAR, LiDAR and Video System. For example, the Deep Learning model can classify and detect objects in real-time using the live stream of video from Video Sensing system. The training of such models requires high computational power, however, once trained the network can work in real-time with even 15fps rate of processing power. The work in literature^61–63 mentions details about the training and computation power.

Some work has discussed the integration of Artificial Intelligence- A.I with different sensing systems. For example, the work in literature ⁶⁴ discussed the integration of Video Camera with the use of A.I and the work in literature ⁶⁵ discussed the use of TensorFlow (an open-source library for Machine Learning) to detect objects. Another work discussing the integration of Vision system with A.I is mentioned in literature. ⁶⁶ These work and the mentioned discussion on each given algorithms suggest the importance of Deep Learning techniques, which can automate the process of learning and classifying the objects.

These algorithms could be categorised into two main types; Detection and Classification. The detection category can detect an object by subtracting the foreground pixel from background pixel, whereas, the classification type can separate the data into different categories for classification. Below in Figure 2 is a decision tree, where the order of preference is from top to bottom.

OPEN IN VIEWER

Figure 2.

Decision tree for selection of appropriate algorithms depending on identification requirements.