Integration of scanning and image processing algorithms for lane detection based on fuzzy method

2.1 Adaptive image binarization threshold algorithm

For the lane detection algorithm based on computer vision, the accuracy of the lane detection is affected by changes in weather and light intensity on the road surface. To improve the real-time performance of lane detection, binarization processing is conducted on the RGB image captured by the camera. The binarization threshold of the image must therefore be rapidly and accurately determined. For this reason, this paper proposes an adaptive image binarization threshold algorithm, as follows:

Adaptive Image Binarization Threshold Algorithm

Step 1:n = 1;//To initiate the binarization threshold value.

i = 255;//To initiate the controlling variable of the loop.

Step 2: Fori >  = 0 do;//To determine the last coordinates of the fastest falling point.

ifArray[i] >  500 then;//Storage of binarization threshold. break; endif;

i = i-1;//endfor.

Step 3:n = i+10;//Access to the fastest falling point coordinate values, then read its 10th points’ value in the array.

Step 4: forArray[n] >  50 andn <  255 do;//To obtain the correct binarization threshold.

n = n+5; end.

Effects are induced by scenery changes ahead of the vehicle, including: clouds, houses beside the road, trees, road signs, etc. The proposed algorithm is based on statistical information to adaptively obtain the image binarization threshold.

2.2 Obtain the intersection of lane and image edge

Ideally, after binarization, the lane will extend to the bottom of the image or to the margin of the broadside image. A lane depicted by a dotted line is shown in Fig. 2, in which the green portion depicts the edge area of the image.

Because the image captured by the camera represents a certain range of road environment, conditions will not be acceptable when three lane lines simultaneously intersect with the bottom edge of the image. If the four lane lines closest to the vehicle are all dotted lines, then some lanes will intersect with the image bottom, while some lanes will intersect with the image side. An algorithm is proposed to determine the intersection between the lane and the image side by scanning, expressed as follows:

Obtain the Intersection of Lane and the Image Edge Algorithm

Step 1: //On the lower part of the image block processing, as shown in Fig. 3, the b² block pattern is used to count all pixels for the binary b² block, with b representing the length of the block.

In the block, if N _{point w}  ⩾ N _{point b} , k = 1, else k = 0 (N _{point w} is the number of the white points, and N _{point b} is the number of black points). Due to the fact that the edge of the green area is only at the image edge, the number of the blocks processed by the algorithm is shown in Equation (1): N block = ( L + 2 W - 16 ) / 8(1) where L is the length of the image, and W is the width of the image. According to Equation (1) the calculation required by the algorithm is greatly reduced.

Step 2:// Scan image edges.

Step 2.1.i = 0;

Step 2.2. Scan image bottom from left to right, i = i+1 when meeting Point _w , then retreat to the state of the white line area.

Step 2.3.i = i-1/ When black spots (Point _b ) is met, the points should be saved. If δ ≤ 5 and the distance from the identified points in the previous frame is less than 5 (d≤5), repeat Step 2.2 until all the edge points on the image edge have been scanned.

Step 3:// Scan the left side of the sidebar from the bottom up.

Step 3.1. Check k = 1. In the left points in the sidebar at the bottom of the image, if k = 1, then move up five pixels, which illustrates that these points may overlap.

Step 3.2. Reaches an end if a white point is obtained in the left sidebar in the image.

Step 4:// Scan right sidebar from the bottom up.

Step 4.1. Check k = 1. In the right points, which are at the bottom sidebar of the image, if k = 1, then move up five pixels, which illustrates that these points may overlap.

Step 4.2. Reaches an end if a white point is obtained in the right sidebar of the image.

With the application of this algorithm, the intersections of the lane and the image edge can be obtained. However, when there is a strong light on the road, the result of image binarization will be inaccurate. When the lane is denoted by a dotted line, all intersections of the lane and the image edge cannot be determined. One node, at least, can be obtained with this algorithm; some nodes can be obtained in most cases. For this reason, based on the above algorithm to get the intersection of the lane and the image side by scanning, this paper further proposes an algorithm to obtain lost intersections, described as follows.

Obtaining Lost Intersections Algorithm

Step 1: Obtain two nodes, Point _G and Point _H , intersections of the lane and the image edge, by the scanning algorithm, as shown in Fig. 4.

Step 2:Line _  _BC : turn 90 degrees counterclockwise to cobtain Line _  _AC ; turn Line _  _DE 90 degrees clockwise to obtain Line _  _DF .

Step 3: As the greatest width of the lane in the image is 5/7 of the image (the area between Point _I and Point _J in Fig. 4), the vanishing point P _K of the default lane is described as follow:

The horizontal coordinate is expressed in Equation (2): K abscissa = L / 7 + 45 L / 77(2)

The vertical coordinate is expressed in Equation (3): K ordinate = W / 2(3)

Step 4: Calculate the coordinates of Point _M according to the position of Point _K and Point _H . When Point _H is mapped to Line _  _D F, the lane slope r is obtained based on the coordinate of Point _H . Then, the coordinate of Point _M can be calculated.

Step 5: Repeat Steps 1 through 4 to calculate the coordinates of ‘Point _L , Point _I , Point _J , Point _M ’.

According to the scanning algorithm, the position of ‘Point _L , Point _I , Point _J , Point _M ’ will be easily obtained by the further application of the algorithm to obtain lost nodes as long as any two nodes of ‘Point _G , Point _I , Point _J , Point _H ’ have been obtained. In the driving process, the position of ‘Point _L , Point _I , Point _J , Point _M ’ will shift left or right with vehicle movement. Once the positions of ‘Point _L , Point _I , Point _J , Point _M ’ are obtained, they can be used to calculate all intersections of lane and image edge in the next frame.

Based on the above analysis, this algorithm can be applied to calculate all intersections on the condition that some intersections of the lane and image edge were not detected as shown in Fig. 5.

In Fig. 5(a) and (b), the blue area between the two lines is the scanning area of the left lane, the light blue area between the two lines is the scanning area of right lane, the black circle is the lane points found in the scanning area, and the red circle is the points calculated by the locations of other points. In Fig. 5(a), the white circle represents the nodes between a lane of the current frame and the bottom edge of the image calculated according to the lane position in the past frame. In Fig. 5(b), the pink points represent the nodes between the lane and left edge of the image.

This paper uses the fuzzy method to determine whether to choose the intersection algorithm by scanning area or the intersection algorithm during image processing. ‘IF X AND Y THEN Z’ fuzzy rules were used to construct a rule-based fuzzy system [2, 8]. As the image is being processed by the algorithm looking for intersections between the lane and image edge and the algorithm to obtain lost nodes, the algorithm can obtain both coordinates of ‘Point _L , Point _I , Point _J , Point _M ’ (the nodes of the lane and the image edge) and the coordinates of the lane vanishing point. In this study, the moving distance of intersection of lanes and the images was divided into three member functions by the algorithm, as follows: { DPC → Distance _ min → [ 0 , 2 ) ; DPN → Distance _ mid → [ 2 , 4 ) ; DPF → Distance _ max → [ 4 , 5 ] . where e is the distance travelled by the vanishing points in these two frames, which can be classified into three member functions, as follows: { DVC → Distance _ min → [ 0 , 1 ) ; DVN → Distance _ mid → [ 1 , 2 ) ; DVF → Distance _ max → [ 2 , 3 ] .

The fuzzy controller uses the Centroid-method to defuzzify the output variable without losing all the information of fuzzy rules. The Centroid-method is described by Equation (4): e * = ∑ i = 1 n e ci ( ω i ) ω i / ∑ i = 1 n e ci ( ω i )(4) where e^* is the determination of the output value, ω _i is the i-th value in the output field, and e _ci  (ω _i ) is the membership function of ω _i .

According to the fuzzy rules, the output is divided into three levels, as follows: { SAN → To choose scanning point algorithm ; MS → To choose the same algorithm applied in former frame ; CV → algorithm of image processing .

As a result, nine fuzzy logic rules were established, and are listed in Table 1.

In summation, the algorithm can accurately obtain the node coordinates of the lane and the image edge location, as well as the coordinates of the lane’s vanishing points. The algorithm can further establish the corresponding image scanning area to detect the lane with the application of the obtained information.

In this paper, the lane scanning area is divided into four categories: (1) when the left edge of the image and the lane line intersect, the scanning area is from left to right with the intersection point as a starting point; (2) when the lane and the image bottom intersect, the scanning area is from the bottom to the top, with the intersection point as a starting point; (3) when the right edge of the lane line intersects the image to the intersection as the starting point of the scanning area from right to left; (4) when the vehicle is moving on a larger curved lane, in order to facilitate the use of the Bezier algorithm to fit the curved lane, it is necessary to establish the lane scanning area in the distance.

After establishing the scanning area, the algorithm will obtain the efficient points of the lane line in the scanning area in three ways: (1) When the scanning area extends from the left to the middle or from the right to the middle, the algorithm will scan the points from the image bottom along the edge of the domain, and will stop after scanning nine efficient points. Because the scene in the lower part of the lane line is easier to observe than that in the upper part, when the lane line is on the edge, the scene on the left upper part of the lane line extends beyond the road, while the one in the right lower part of the domain area still is still in the scope of the, and the road is of a single color. As a result, this scan mode can greatly reduce disturbance of the road surface and the surrounding sceneries; (2) When the scanning area extends from the left to the middle, the algorithm will scan points from the right to the left, and will stop after scanning nine efficient points; (3) When the scanning area extends from the bottom of the image to the middle, the algorithm will stop after scanning nine efficient points.