Abstract
In order to solve the problems of large measurement errors and long time consuming in the measurement of the lateral distance between driverless vehicles, this paper proposes to design a lateral distance measurement technology of driverless vehicles based on machine vision. According to the preview following theory and model basis, the lateral movement process of driverless vehicles is determined, the lateral displacement deviation change rate and heading angle deviation change rate between vehicle and preview point in local coordinates are calculated, and the lateral motion dynamic model of unmanned vehicle is designed. With the help of Yolo algorithm, the boundary frame is drawn up to determine the position of the lateral moving vehicle, and the detection error is corrected with the help of loss function to complete the lateral moving target detection of the unmanned vehicle. The relationship between the machine coordinate system and the world coordinate system is determined. According to the internal and external parameter matrix of the relationship between the lateral distances of the driverless vehicle, the machine vision position is set, and the lateral distance is determined by the pitch angle between the horizontal planes. The experimental results show that the measurement error of this method is always less than 5%. This method has small measurement error, small error between test distance and actual distance, and has certain application performance.
Keywords
Introduction
With the rapid development of technology such as computer, automation, communication and information fusion, the automatic driving vehicle came into being [1]. The so-called smart car is a typical high-tech fusion body integrating computer science, modern sensing, multi information fusion, communication and automatic control technology [2]. Therefore, the smart car has the functions of automatic identification of road conditions, automatic obstacle avoidance, automatic overtaking and lane merging, which can completely replace the driver’s operation in theory [3]. With the continuous development of intelligent vehicle technology, the car will become safer and reliable, more humane, so that it has human thoughts and consciousness. The development of intelligent vehicle benefits from a variety of control, communication, information fusion and other technologies, and the realization of these technologies needs to be carried out on the premise of accurately knowing the vehicle driving state information. The vehicle environment information and driving state information are mainly measured by on-board sensors and transmitted to the controller, but with the improvement of vehicle performance, the required vehicle state information will increase significantly, and all the state information measured by on-board sensors is not practical. On the one hand, too much state information will lead to information redundancy and information confusion of on-board sensors; on the other hand, the price of on-board sensors is very expensive, and the installation of a large number of on-board sensors will greatly increase the production cost. Due to the above problems, the improvement of vehicle safety performance technology is limited [4].
Lateral motion control refers to the intelligent vehicle through the visual perception system, sensor system, GPS/GIS system to obtain vehicle real-time position information and vehicle driving state information, based on a specific control method to make it drive along the desired path [5]. When the intelligent vehicle is running, the tire can not sideslip, and the driving direction can only be along the longitudinal symmetrical line of the body. It is a highly nonlinear nonholonomic motion constraint system. Its model and environment are uncertain and the measurement is inaccurate, which makes the lateral control more complex [6]. Many scholars have done a lot of research into the lateral distance measurement of intelligent vehicles, and achieved some results.
A method for estimating the vertical and horizontal distance of vehicle system is proposed [7]. In order to effectively solve the problem that the lateral distance of vehicle is short under complex driving conditions, which may cause danger to vehicle handling stability and safety, a dual nonlinear state observer algorithm based on vehicle vertical and lateral coupling dynamics is designed in this paper to realize real-time and accurate estimation of vehicle lateral motion state under complex driving conditions, The road excitation model and vehicle system vertical and lateral coupling dynamic model are established; Then, using unscented Kalman filter (UKF) and nonlinear fuzzy observation (T-S) theory, a nonlinear state observation algorithm is designed to jointly estimate the sprung mass and roll state of vehicle system under different road excitation conditions; Finally, using CarSim dynamics software, the observation accuracy of real-time estimation of vehicle roll angle and roll rate by using the joint state observer (UKF & T-S) under j-turn test conditions on standard A-level and C-level roads is compared and analyzed. This method has high accuracy for vehicle lateral ranging, and can solve the problem of short vehicle lateral distance under complex driving conditions, which poses a threat to vehicle handling stability and safety, but the operation process is complex and not easy to popularize. The measurement error is large and the measurement speed is slow. The method of front vehicle recognition and longitudinal vehicle distance detection based on fusion of rear edge features is proposed [8]. This method is different from the traditional algorithm which uses global edge features or gray texture features to identify the front car. Firstly, the shadow under the car is segmented according to the gray distribution characteristics of the road, and then the ROI of the front car is established by using the distribution characteristics of the horizontal and vertical edges of the rear car. Furthermore, the symmetrical intensity of the vertical edge is used to verify the vehicle identity. According to the principle of small hole imaging of the camera, the longitudinal vehicle distance measurement model is established, which avoids the complex calibration of internal and external parameters. The operation process of this method is relatively simple and can identify the identity of the vehicle, but there are some differences between transverse parameters and longitudinal parameters, which need to be further improved. And the lateral motion flow of unmanned aerial vehicle cannot be determined. Machine vision is a rapidly developing branch of artificial intelligence. To replace human vision with machine vision is to make simple judgment. In the process of mass repetitive industrial production, the machine vision detection method can greatly improve the efficiency and automation of production. Due to the great development of machine vision in the fields of observation, space, automobile and so on.
In order to make up for the problems with the above methods, this paper presents a research on the measurement technology of the transverse distance of unmanned vehicles based on machine vision.
The technical route of this paper is as follows:
(1) Based on the theory and model of preview and follow, the transverse motion flow of unmanned vehicles is determined. The variation rate of lateral displacement deviation and heading angle deviation of vehicle and preview point in local coordinates is calculated, and the dynamic model design of the vehicle lateral motion is completed.
(2) The boundary frame is drawn up by Yolo algorithm, and the vehicle position is determined. The error of the detection is corrected by the loss function, and the target detection of the vehicle lateral motion is completed.
(3) The relationship between the machine coordinate system and the world coordinate system is determined. According to the obtained relationship, the internal and external reference matrix of the transverse distance of the unmanned vehicle are determined. The machine vision position is set, and the lateral distance measurement is determined by the pitching angle between the horizontal planes.
(4) Experimental analysis.
The larger the distance, the smaller the size of the car in the image, and the smaller the difference between the test distance and the actual distance. The results show that the measurement result of this method is within the allowable error range.
(5) Conclusion and future prospects.
This paper presents a new technology for measuring the lateral distance of driverless vehicles based on machine vision. The relationship between the machine coordinate system and the world coordinate system is determined. According to the internal and external parameter matrix of the relationship between the lateral spacing of driverless vehicles, the machine vision position is set, and the lateral spacing is determined by the pitch angle between the horizontal planes.
Lateral distance measurement technology of driverless vehicle based on machine vision
Dynamic model of lateral motion of driverless vehicle
As a highly nonlinear and nonholonomic motion constraint system, the uncertainty and measurement inaccuracy of intelligent vehicle model and external environment make it difficult to control the vehicle motion. Using fuzzy control logic to design lateral controller, there is no need to establish an accurate control model, because when the vehicle system model is not accurate, the fuzzy control rules can be designed according to the driver’s experience and expert knowledge, so as to design the controller. According to human driving experience, drivers will habitually observe the road ahead in advance, estimate the road conditions ahead, and obtain the distance between the preview point and the vehicle position in advance. If the vehicle turns right in front of the road, the driver will turn the steering wheel to the right according to the road curvature and driving speed. In order to make the vehicle turn smoothly, the driver needs to constantly observe the lateral displacement deviation and heading angle deviation between the actual running position of the vehicle and the road centerline, and adjust the steering wheel angle to reduce these deviations, so as to track the desired path accurately and quickly [9]. However, the process is easily affected by the surrounding environment and becomes more complex with the change of vehicle speed. The preview following principle proposed by academician Xie Konghui vividly describes the above-mentioned driver’s handling behavior, and then on this basis, the driver’s “steady-state prediction dynamic correction hypothesis”, “preview optimal curvature model” and “optimal preview acceleration model” are produced. In the research of intelligent vehicles, the theory of preview following is also applicable.
Therefore, this paper combines the preview following principle and fuzzy logic control method to study the lateral control of intelligent vehicle. Firstly, the road vehicle dynamic control model is built, and the proportional derivative (PD) lateral acceleration optimal tracking controller is designed according to the optimal preview driver Principle and model, so as to obtain the vehicle lateral control system. Secondly, taking the vehicle longitudinal speed and road curvature as the controller inputs and the preview distance as the controller output, the fuzzy optimal controller for automatic selection of preview distance is constructed, so as to realize the adaptive preview fuzzy optimal control of vehicle lateral motion. According to the above ideas, the vehicle dynamic model is designed [10]. The vehicle dynamics model is shown in Fig. 1.
Vehicle dynamics model.
Set the vehicle lateral speed is
The lateral control process of driverless vehicle is shown in Fig. 2.
Lateral control flow chart of unmanned vehicle.
The transfer function of driverless vehicle in lateral motion can be expressed as:
In the formula,
Under low frequency domain conditions, the ideal lateral pre-aiming flow control should meet:
Because of the sway or movement in different directions when the vehicle moves laterally, the change rate of lateral displacement deviation and heading angle deviation between the vehicle and the preview point in the local coordinates are expressed as follows:
In the formula,
Unmanned vehicles in lateral rotation can be considered as a first order inertia link [6], then exists:
In the formula,
Based on this, the optimal model is designed based on the theory [11]. The presight time obtained from setting the driverless vehicle presight distance to
Suppose the vehicle steering wheel angle is during lateral movement
In the design of lateral motion dynamic model of driverless vehicle, according to preview following theory and model basis, the lateral motion process of driverless vehicle is determined, and the lateral displacement deviation change rate and heading angle deviation change rate between vehicle and preview point in local coordinates are calculated to complete the design of lateral motion dynamic model of driverless vehicle.
According to the above research of lateral motion dynamics model of driverless vehicles, in order to achieve the accurate measurement of the lateral distance of driverless vehicles, it is necessary to detect the lateral movement target of driverless vehicles to reduce the risk of lateral occurrence in the movement of driverless vehicles. In this paper, the detection of YOLO object detection. The basic idea of the algorithm is that if an object falls within a grid, it predicts the object boundary frame. Set the forecast B
Each boundary box contains 5 parameters: a, b, c, d, e.
In the formula,
YOLO simultaneously predicts the presence of objects where objects belong to a certain class of posterior probability Pr (Class, 0objet), the probability of the center of a class i object within the grid. Each grid predicts only the conditional probability of one C class object, each predicts the location of the B bounding box, which shares a set of conditional probability Pr (Class, 0object) (F1, 2, 3, C) and the confidence of a bounding box at detection is:
The final YOLO output is the tensor of
YOLO regards target detection as a regression problem, so its loss function uses mean square and error. The loss function includes three parts: coordinate positioning error, confidence error, and classification error. In order to distinguish between positioning and classification errors, the loss function of YOLO adopts different weight values and large weights for the positioning error; Since most of the grids contain no target in the target detection, smaller weights are used to reduce the grid confidence without the target [13]. The expression for the loss function is:
According to the calculation of the loss function, the detection of the horizontal targets of the driverless vehicle is completed, and the detection process is as shown in Fig. 3.
Horizontal movement target detection of driverless vehicles.
In the lateral movement target detection of unmanned vehicles, the boundary frame is formulated with the YOLO algorithm to determine the position of the vehicle lateral movement vehicle. The basic network of Yolo algorithm is inspired by googlenet, but it does not adopt its inception structure, but simply uses 1
Based on the above determined horizontal target detection of driverless vehicles, this paper introduces machine vision to realize the horizontal distance measurement of driverless vehicles. In machine vision, in order to use the 2-dimensional information in the image to obtain 3-dimensional information, it is necessary to transform the coordinate space. The relevant coordinate system has image pixel coordinate system, image physical coordinate system, camera coordinate system and world coordinate system. The basic model of camera imaging adopts pinhole imaging model. Among them, the first need to consider the change of its coordinate system. Machine and world coordinate systems are shown in Fig. 4.
Machine vision coordinate system and world coordinate system.
In Fig. 4,
In the lateral distance measurement of an unmanned vehicle, considering that the origin of the machine vision coordinate system and the focal point of the image plane is the projection point
After determining the relationship between them, the inner parameter matrix and the outer parameter matrix of the lateral distance of the driverless vehicle are expressed as:
In the formula,
In the actual imaging process of machine vision, optical distortion can lead to nonlinear distortion, which can be divided into radial distortion and tangential distortion. Radial distortion is the radial position deviation of the image pixel centered on the distortion point, which is caused by lens manufacturing error; The tangential distortion is caused by the fact that the lens plane is not completely parallel to the imaging plane. In the process of image processing, it is necessary to calibrate the camera first and get the camera related parameters, including the internal parameter m, external parameter m and distortion parameter [14].
The methods of distance measurement based on machine vision include back projection transformation, fitting modeling and geometric imaging. Among them, the inverse projection method is based on the similar triangle principle, which is simple to calculate, but it needs to know the width of the measured object, so it is not suitable for vehicle distance estimation; Fitting modeling method is based on a large number of data samples to establish a nonlinear regression model, which requires high accuracy of sample data; The geometric imaging method uses the geometric relationship between the image coordinate system and the world coordinate system in the imaging model to calculate the distance. The calculation process is simple and the accuracy is high. Therefore, after detecting the center point of the transverse vehicle, the geometric imaging method is used to estimate the distance from the transverse vehicle. The schematic diagram of lateral distance measurement for driverless vehicles is shown in Fig. 5.
Schematic diagram of lateral distance measurement of unmanned vehicle.
According to Fig. 4, the distance between the driverless vehicle and the lateral vehicle can be expressed as:
In the formula,
During the actual ranging process, the machine elevation is generally less than 10 degrees and about 1.2 m, from the ground, this moment
The vehicle is measured according to Eq. (17).
In the lateral distance measurement of driverless vehicle, the relationship between the machine coordinate system and the world coordinate system is firstly determined. According to the internal and external parameter matrix of the relationship between the lateral distances of driverless vehicle, the machine vision position is set, and the lateral distance measurement is determined by the pitch angle between the horizontal planes.
Sample ranging image.
Experimental scheme
In order to verify that the proposed method can effectively realize the lateral distance measurement of driverless vehicles, an experimental analysis is carried out. In the experiment, the driving photos of driverless vehicles collected from a certain road section in a certain area are taken as the research object, and the measurement accuracy of the lateral distance of driverless vehicles is analyzed. The experimental platform is: the hardware equipment is Xiaomi air 12 laptop: 12.5 inches, inter i5 processor, CPU of 1.61 ghz, memory of 4 GB, windows10 64 bit operating system. The software programming environment is MATLAB2014, and the MATLAB camera calibration toolbox is loaded. Shoot the vehicle at certain intervals to ensure that the vehicle is in the image. Finally, the lateral distance between vehicles is calculated according to the formula given in this paper. The collected sample image is shown in Fig. 6.
According to Fig. 6, the lateral distance of four driverless vehicles is measured. The transverse distance between car a and car B on the left is about 50 cm, the transverse distance between car B and car C is about 60 cm, and the transverse distance between car C and car D is about 55 cm. In order to ensure the effectiveness of the experiment, the horizontal distance is iterated many times, and the values are average.
Error comparison of vehicle lateral distance measurement with different methods.
Under the background of the above experimental environment, this paper adopts the comparison method, the estimation method of vertical and horizontal distance of vehicle system, and the front vehicle recognition and longitudinal vehicle distance detection method based on the fusion of rear edge features. Taking the measurement error and time-consuming of driverless vehicle lateral distance as the experimental indexes, several comparative experiments were carried out. Among them, the measurement error refers to the difference between the method and the actual distance in the transverse distance measurement, expressed as a percentage; The measurement time is the time cost to complete the accurate measurement of lateral distance.
Experimental results
This paper analyzes the method, the estimation method of vertical and horizontal distance of vehicle system, the front vehicle recognition and longitudinal vehicle distance detection method based on the fusion of rear edge features, and analyzes the error of transverse distance measurement of sample vehicles. The experimental results are shown in Fig. 7.
By analyzing the data in Fig. 6, it can be seen that under the same experimental conditions, there is a certain gap in the error of the transverse distance measurement of the sample vehicles by using the method in this paper, the estimation method of the vertical and transverse distance of the vehicle system, and the front vehicle recognition and longitudinal vehicle distance detection method integrating the edge features of the rear of the vehicle. Among them, the measurement error of this method is always less than 5%, while the other two methods continue to show an upward trend. In contrast, the measurement error of the proposed method is lower than the other two methods, which verifies that the proposed method can effectively improve the vehicle lateral distance.
In order to further verify the effectiveness of this method, according to the example ranging image of Fig. 5, it can be seen that the position is accurately detected, and the value of the transverse distance decreases with the size of the vehicle in the image. Table 1 shows the experimental results of transverse distance measurement using this method.
Lateral distance measurement results
Lateral distance measurement results
It can be seen from Table 1 that the larger the distance value, the smaller the size of the car in the image, and the smaller the difference between the test distance and the actual distance. The results show that the measurement results of this method are within the allowable error range.
In order to overcome the shortcomings of the existing methods for measuring the lateral vehicle distance, this paper proposes a new technology for measuring the lateral vehicle distance of unmanned vehicles based on machine vision. According to the preview following theory and model, the lateral movement process of unmanned vehicles is determined, and the lateral displacement deviation change rate and heading angle deviation change rate of vehicles and preview points in local coordinates are calculated, complete the design of lateral motion dynamic model of driverless vehicle. With the help of Yolo algorithm, the boundary frame is drawn up to determine the position of the lateral moving vehicle, and the detection error is corrected with the help of loss function to complete the lateral moving target detection of the unmanned vehicle. The relationship between the machine coordinate system and the world coordinate system is determined. According to the internal and external parameter matrix of the relationship between the lateral distances of the driverless vehicle, the machine vision position is set, and the lateral distance is determined by the pitch angle between the horizontal planes. The experimental results show that this method has the advantages of low measurement error and fast measurement speed. Due to the short research time and few experimental test indicators, in the future development, the measurement method of lateral vehicle distance of driverless vehicles under the short-range dynamic measurement scenario will be studied.
