Development of Vision-Based Vehicle Detection and Recognition System for Intelligent Vehicles

The business plan for the Intelligent Vehicles Initiative (IVI) from the U.S. Department of Transportation contains several candidate services that address collision warning and collision avoidance. Recently, a proactive crash mitigation system that would enhance the crash avoidance and survivability components of IVI has been proposed. An accurate object detection and recognition system is a prerequisite for the proactive crash mitigation system. A vision-based approach to the detection and recognition of vehicles, calculation of their motion parameters, and tracking of multiple vehicles by using a sequence of gray-scale images taken from a moving vehicle is presented. The vision-based system consists of four models: the object detection model, the object recognition model, the object information model, and the object tracking model. Object detection, recognition, and tracking are accomplished by combining the analysis of a single image frame with the analysis of consecutive image frames. In the analysis of the single image frame, the system detects the potential objects by using their shape features and recognizes the objects by using a neural network. Once the objects are recognized, they are tracked in the consecutive image frames by processing only the pertinent areas given by previous frames. The analysis of the single image frame is performed every 10 image frames. The information model will judge whether the objects are hazardous to the host vehicle by using two parameters: time to collision and its time derivative. Experimental results demonstrated a robust system in real-time vehicle recognition, vehicle tracking, and vehicle motion analysis over thousands of image frames.