IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,
On-Road Vehicle Detection: A Review
Zehang Sun, Member, IEEE, George Bebis, Member, IEEE, and Ronald Miller
Abstract—Developing on-board automotive driver assistance systems aiming to alert drivers about driving environments, and possible collision with other vehicles has attracted a lot of attention lately. In these systems, robust and reliable vehicle detection is a critical step. This paper presents a review of recent vision-based on-road vehicle detection systems. Our focus is on systems where the camera is mounted on the vehicle rather than being fixed such as in traffic/driveway monitoring systems. First, we discuss the problem of on-road vehicle detection using optical sensors followed by a brief review of intelligent vehicle research worldwide. Then, we discuss active and passive sensors to set the stage for vision-based vehicle detection. Methods aiming to quickly hypothesize the location of vehicles in an image as well as to verify the hypothesized locations are reviewed next. Integrating detection with tracking is also reviewed to illustrate the benefits of exploiting temporal continuity for vehicle detection. Finally, we present a critical overview of the methods discussed, we assess their potential for future deployment, and we present directions for future research.
Index Terms—Vehicle detection, computer vision, intelligent vehicles.
VERY minute, on average, at least one person dies in a vehicle crash. Auto accidents also injure at least 10 million people each year, two or three million of them seriously. It is predicted that the hospital bill, damaged property, and other costs will add up to 1-3 percent of the world’s gross domestic product , . With the aim of reducing injury and accident severity, precrash sensing is becoming an area of active research among automotive manufacturers, suppliers and universities. Several national and international projects have been launched over the past several years to investigate new technologies for improving safety and accident prevention (see Section 2). E
Vehicle accident statistics disclose that the main threats a driver is facing are from other vehicles. Consequently, developing on-board automotive driver assistance systems aiming to alert a driver about driving environments and possible collision with other vehicles has attracted a lot of attention. In these systems, robust and reliable vehicle detection is the first step. Vehicle detection—and tracking
has many applications including platooning (i.e., vehicles
traveling in high speed and close distance in highways), stop and go (vehicles traveling in low speeds and close distance in cities), and autonomous driving.
This paper presents a review of recent vision-based on- road vehicle detection systems where the camera is mounted on the vehicle rather than being fixed such as in traffic/driveway monitoring systems. Vehicle detection using optical sensors is very challenging due to huge within class variabilities in vehicle appearance. Vehicles
may vary in shape (Fig. 1a), size, and color. The appearance of a vehicle depends on its pose (Fig. 1b) and is affected by nearby objects. Complex outdoor environments (e.g., illumination conditions (Fig. 1c), unpredictable interaction between traffic participants, cluttered background (Fig. 1d) are difficult to control. On-road vehicle detection also requires faster processing than other applications since the vehicle speed is bounded by the processing rate. Another key issue is robustness to vehicle’s movements and drifts.
More general overviews on various aspects of intelligent transportation systems (e.g., infrastructure-based approaches such as sensors detecting the field emitted by permanent magnetic markers or electric wires buried in the road) as well as vision-based intelligent transportation systems (e.g., driver monitoring, pedestrian detection, sign recognition, etc.) can be found in , , , , , . Several special issues have also focused on computer vision applications in intelligent transportation systems , , , .
This paper is organized as follows: In Section 2, we present a brief introduction of vision-based intelligent vehicle research worldwide. A brief review of active and passive sensors is presented in Section 3. Detailed reviews of Hypothesis Generation (HG) and Hypothesis Verification (HV) methods are presented in Sections 5 and 6 while exploring temporal continuity by integrating detection with tracking is discussed in Section 7. In Section 8, we provide a critical overview of the HG and HV methods reviewed. Challenges and future research directions are presented in Section 9. Finally, our conclusions are given in Section 10.
. Z. Sun is with eTreppid Technologies, LLC., 755 Trademark Drive, Reno, NV 89521. E-mail: email@example.com. . G. Bebis is with the Computer Vision Laboratory, Department of Computer Science and Engineering, University of Nevada, 1664 North Virginia Street, Reno, NV 89557. E-mail: firstname.lastname@example.org. . R. Miller is with the Vehicle Design R & A Department, Ford Motor Company, 1 American Road, Dearborn, MI 48126-2798. E-mail: email@example.com.
Manuscript received 11 Aug. 2004; revised 15 Aug. 2005; accepted 22 Aug. 2005; published online 13 Mar. 2006. Recommended for acceptance by H. Sawhney. For information on obtaining reprints of this article, please send e-mail to: firstname.lastname@example.org, and reference IEEECS Log Number TPAMI-0422-0804.
VISION-BASED INTELLIGENT VEHICLE RESEARCH WORLDWIDE
Vision-based vehicle detection for driver assistance has received considerable attention over the last 15 years. There are at least three reasons for the blooming research in this field: 1) the startling losses both in human lives and finance caused by vehicle accidents, 2) the availability of feasible technologies accumulated within the last 30 years of computer vision research, and 3) the exponential growth in processor speeds have paved the way for running
0162-8828/06/$20.00 2006 IEEE
Published by the IEEE Computer Society