A project by Daniella Apelblat and Yaniv Firedman
Introduction and motivation
In the past year, in Israel alone, over 42 thousand drivers have reported falling asleep at the wheel. Over a quarter of a million drivers reported feeling drowsy and dosing off lightly while driving. Over 100 thousand drivers reported sliding to the margins of the road due to almost falling asleep at the wheel . Hence there is no doubt that being tired while driving is as common as it is dangerous. We therefore decided to tackle the problem of the sleepy driver.
Today, there are a number of available solutions that helps tackle the problem of driver fatigue. For example, Toyota's Kirobo Mini robot is a small hand-held robot that talks to the driver during the drive and monitors his awake state ; however, the device is only available in Japan and is not intended for global use. A second system uses hemoglobin levels, measured in the ear through an earpiece, to monitor the heart rate of the driver . When drowsiness is detected, a vibration of the driver's mobile phone is initiated in order to wake her up. In our opinion, apart from the fact that driving while handling a mobile phone is not a good idea, a drowsy driver may not notice this vibration at all. There exist a few more such systems, but we found none of these existing solutions to be both simple to use, practical and cheap.
In this project we chose to utilize Matlab for face-feature-detection in order to monitor drivers' drowsiness. The guiding logic behind the program is that when a driver is fully awake and alert, her eyes are wide open and the irises are seen and can be well identified. When the driver begins to get drowsy her eyelids become heavier and heavier, making the irises less visible. When we detect the irises as less visible, we issue a voice alert to the to wake the driver up. Our program is not only easy to use (just press Play), but it also does not require any wearable device, which may be uncomfortable and disturbing to the driving. The only requirement of our system is a small camera mounted in the car in front of the driver's face.
The implementation consists of several independent components which operate in synergy with each other. First we needed to create an interface with the camera. In building the program we used our laptops' cameras, but of course this could be done for any camera available. Once video streaming from the camera begins, the next involves the identification of the drivers' eyes. This is accomplished using Matlab's built-in Cascade Object Detector  -first identifying the face and then extracting the eyes (Figure 1).
We then used an eye-tracking algorithm published by Peter Aldrian . This algorithm was able to identify the iris in each of the eyes (Figure 2A). When the driver gets drowsy and her eyes begin to droop, the irises are not recognized very well in the picture, and tend to be recognized off-target (Figure 2B). When such off-target phenomenon is identified, we warn the driver that she needs to wake up by playing a recorded audio. In order to avoid false-alarms in general and specifically to allow blinking to go without warning, the recording is only played after a breaching a threshold of "sleepy" eye identifications for a certain period of time.
Independently of the video-processing component, a GUI was developed for user registration and sign-in to the monitoring system. The guiding principle behind the registration option is to allow future implementations of per-customer customization such as smarter individual eye detection, user-specific choice of alarm etc.
We tested the system in two modes: using live video acquisition, and using a pre-recorded video. The results for both modes are as follows:
Tracking using a pre-made video
Tracking with pre-made video resulted in correct functioning of the system. The video was created with a few episodes of enacted drowsiness and the system successfully recognized each of the drowsy episodes and warned the user by playing the correct audio message at the time of crossing the threshold.
Tracking using live video streaming
Tracking using live video input did not succeed. The alarm did not go off upon drowsy driving (see discussion).
Using the system with the GUI
Upon initiating the GUI, the system is started. First the user is introduced to the main GUI, where she may choose to register or to sign in with an existing user (Figure 3 A,B). After registration (Figure 3 C,D) the user is directed to the "Start Monitor" window begins, upon pressing the Start button, the monitoring of the driver (Figure 3E). Once the driver is identified to be drowsy, a voice alert is issued from the system.
We present a useful tool that may assist in improving drivers' safety. Our system worked well on recorded video but did not work with live-stream acquisition. We attribute this failure to the specific camera used for live acquisition, having a sampling rate which was not high enough, in addition to low computational power of the used computer which resulted slow calculations. Since both the image acquisition and the image processing times were too high, a "blink" or "drowsiness" could not be recognized correctly. We suspect that this hurdle, may easily be solved using better hardware and a system that, unlike our laptops, is designated only to the function of this program.
In order to fine-tune the comprehensively rely on the system, it should also be tested in real driving conditions. Our testing included only in-doors runs with optimal lighting conditions, whereas real driving is sometimes in the dark, in very bright lighting conditions and more.