SUN ET AL.: ON-ROAD VEHICLE DETECTION: A REVIEW
direction, called “image stabilization” , . Image stabilization is based on frame-to-frame registration. Taking the first frame of the image sequence as a reference, the stabilization method registers this frame to the next frame, computes the motion parameters from the current frame to the reference frame. Then, it uses the estimated parameters to warp the current frame to get the stabilized image, which can be considered as taken from a stationary camera. The motion model employed in  contains four parameters: two for
translation, one for rotation, and one for scaling:
X Y 2 2
X Y 1 1
X Y 2 2
X i Y i X Y are the image frame coordinates at time ti, where 2 2 i s t h e t r a n s l a t i o n m e a s u r e d i n t h e i m a g e c o o r d i n a t e s y s t e m o f a frame at t2, is the rotation angle between the two frames,
and s is a scaling factor.
This model was appropriate for image sequence of distant scenes, where perspective distortion could be neglected. The motion parameters were computed by matching a small number of feature points between two frames. The extraction of feature points was done via searching a predefined small window on the very top region of the image using correlation. By constraining the selection of feature points to the very top region, features that were too close to the camera could be avoided and, consequently, less distortion was introduced to the model. It should be mentioned that image stabilization methods would fail when an image contains close scenes—a common scenario when driving a vehicle in downtown or during vehicle turns.
Critique of HV Methods
Constructing explicit models of the objects to be recognized is very difficult when object appearance varies a lot. In general, appearance-based methods are more accurate than template- based methods, however, they are more costly due to classifier training. Nevertheless, appearance-based methods are becoming more and more popular due to the exponential growth in processor speed. Analyzing the pros and cons of various appearance-based methods proposed in the litera- ture is not simple. Most studies have been performed using different data sets and performance measures making a fair evaluation of different feature extraction methods and classification schemes difficult if not impossible. In a recent study, experimental results were reported using several feature extraction methods (i.e., PCA features, wavelet features, and Gabor features) and classifiers (i.e., Neural Networks (NN) and Support Vector Machines (SVMs)) ). Testing was performed using a common data set obtained by driving Ford’s concept vehicle under different traffic condi- tions (e.g., structured highway, complex urban streets, and varying weather conditions). The best approach in terms of accuracy was found to be Gabor features with SVMs, yielding an error rate of 5.33 percent with a false positives (FP) rate of 3.46 percent and a false negatives (FN) rate of 1.88 percent. Combining Gabor and Harr wavelet features yielded a slightly better performance (i.e., an error rate of 3.89 percent with a FP rate of 2.29 percent and a FN rate of 1.6 percent) at the expense of higher time requirements. It should be mentioned that the error rate using Haar wavelet features with SVMs was 8.52 percent, the FP rate was 6.50 percent and
the FN rate was 2.02 percent. More systematic evaluations of various feature extraction methods and classification schemes are required in order to assess the performance of HV methods. In order for these comparisons to become more meaningful, it is imperative to develop first representative datasets (i.e., benchmarks) and carefully designed evaluation procedures (i.e., see Section 9.6).
An important issue in the realization of successful driver assistance applications is the design of vehicle detection and tracking systems that yield a maximum level of reliability and robustness in real-time. Although many efforts have been put into this research area, many algorithms and systems have already been reported, many prototype vehicles have already been demonstrated, a highly robust, reliable, real-time system is yet to be demonstrated. Achieving these objectives requires addressing several challenges and solving quite different problems.
From a technical point of view, the success of an on-road vehicle detection system will depend on the number of correct detections versus the number of false alarms that it produces, assuming a certain processing rate and a processor platform. Determining the desired level of accuracy for vehicle detection is not easy and depends on the nature of the application. For example, if vehicle detection is part of a warning system, then higher false positive rates can be tolerated. In contrast, systems involving active vehicle control need to be more conservative in terms of false alarms. There are a number of ways to significantly reduce the number of false positives while keeping high accuracy including improved algorithmic solutions (e.g., using multiple cues, advanced statistical, and learning models), sensor fusion (e.g., visible, IR, and radar), and telematics (e.g., vehicle-to- vehicle communication and GPS-based localization). We elaborate more on these issues next.
The design of computer vision algorithms that operate robustly and reliably in complex and wide varying environ- ments (e.g., rain, fog, night, etc.) is a major challenge. Using on-board cameras makes some well-established computer vision techniques unsuitable (e.g., background subtraction is not appropriate due to fast background changes caused by camera motion) or not directly applicable unless making certain assumptions or adding enhancements (e.g., stereo- based systems require frequent recalibration to account for camera movements caused by shocks and vibrations). Efficient implementations should also be considered (e.g., fast motion-based estimation) in order to meet real-time performance requirements.
Developing more powerful algorithms to deal with a variety of issues is thus essential. In doing so, it is important to understand first the requirements of on-road vehicle detec- tion and design customized algorithmic solutions that meet the requirements while taking advantage of domain knowl- edge and inherent constraints (e.g., exploiting temporal continuity to improve accuracy and robustness or assuming a flat road to simplify the mapping between image pixels and word coordinates). We have presented in Section 8 a number of issues associated with HG approaches and potential algorithmic solutions to deal with these issues effectively. More efforts are clearly required in this direction.