Driver traffic violation detection and driver risk calculation through real-time image processing

Abstract

Road safety is a serious problem in many countries and affects the lives of many people. Improving road safety starts with the drivers, and the best way to make them change their habits is to offer incentives for better, safer driving styles. This project aims to make that possible by offering a means to calculate a quantified indicator of how safe a driver's habits are. This is done by developing an on-board, visual road-sign recognition system that can be coupled with a vehicle tracking system to determine how often a driver violates the rules of the road. The system detects stop signs, red traffic lights and speed limit signs, and outputs this data in a format that can be read by a vehicle tracking system, where it can be combined with speed information and sent to a central database where the driver safety rating can be calculated. Input to the system comes from a simple, standard dashboard mounted camera within the vehicle, which generates a continuous stream of images of the scene directly in front of the vehicle. The images are subjected to a number of cascaded detection sub-systems to determine if any of the target objects (road signs) appear within that video frame. The detection system software had to be optimized for minimum false positive detections, since those will unfairly punish the driver, and it also had to be optimized for speed to run on small hardware that can be installed in the vehicle. The first stage of the cascaded system consists of an image detector that detects circles within the image, since traffic lights and speed signs are circular and a stop sign can be approximated by a circle when the image is blurred or the resolution is lowered. The second stage is a neural network that is trained to recognize the target road sign in order to determine which road sign was found, or to eliminate other circular objects found in the image frame. The output of the neural network is then sent through an iterative filter with a majority voted output to eliminate detection 'jitter' and the occasional incorrect classifier output. Object tracking is applied to the 'good' detection outputs and used as an additional input for the detection phase on the next frame. In this way the continuity and robustness of the image detector are improved, since the object tracker indicates to it where the target object is most likely to appear in the next frame, based on the track it has been following through previous frames. In the final stage the detection system output is written to the chosen pins of the hardware output port, from where the detection output can be indicated to the user and also used as an input to the vehicle tracking system. To find the best detection approach, some methods found in literature were studied and the most likely candidates compared. The scale invariant feature transform (SIFT) and speeded up robust features (SURF) algorithms are too slow compared to the cascaded approach to be used for real-time detection on an in-vehicle hardware platform. In the cascaded approach used, different detection stage algorithms are tested and compared. The Hough circle transform is measured against blob detection on stop signs and speed limit signs. On traffic light state detection two approaches are tested and compared, one based on colour information and the other on direct neural network classification. To run the software in the user's vehicle, an appropriate hardware platform is chosen. A number of promising hardware platforms were studied and their specifications compared before the best candidate was selected and purchased for the project. The developed software was tested on the selected hardware in a vehicle during real public road driving for extended periods and under various conditions.