Keywords

1 Introduction

Transport systems are an essential part of human activities. Growth in roads infrastructures, advancements in vehicles and development of road safety laws are intend to reduce road accidents [1]. However according to a report published by WHO, death count of 1.25 million people per annum on road accidents has not reduced [2]. Changes in our lifestyle resulted in increased number of traffic accidents due to driver’s drowsiness or sleepiness [3, 4]. National Highway Traffic Safety Administration (NHTSA) estimates that approximately 25% of police reported accidents involves driver fatigue [5].

According to a study by the Sleep Research Center (UK), driver drowsiness causes up to 20% of accidents on monotonous roads [6]. While designing any vehicle, comfort zone and safety are the main factors and due to these comfort environment in the vehicle driver can go in sleeping mode [7]. In this situation of driver’s drowsiness, the safety concerns are not properly address. To reduce this problem and to negate the deadly accidents, the driver continuous monitoring is required [8,9,10].

Different techniques have been reported for the detection of driver’s drowsiness [11,12,13]. Subjective technique cannot be use in a real driving situation but is helpful in simulations for determining drowsiness [14]. Psychological signals like electrocardiogram (ECG), electroencephalogram (EEG), Electrooculography (EOG) can be utilize for drowsiness detection. A technique based on a psychological signal is more reliable [9, 15,16,17]. Vehicle movement based detection is another technique. Here information is obtain from sensors attached with steering wheel, acceleration pedal and/or body of the vehicle [11, 18, 19]. Signals collected from sensors are continuously monitor for the identification of noticeable variation in order to detect driver’s drowsiness.

This paper is outlined as follows: Sect. 2 describes general drowsiness detection techniques. Computer vision based driver’s drowsiness detection is presented in Sect. 3. Section 4 is for discussion on the prevailing drowsiness detection techniques and finally Sect. 5 concludes the paper.

2 General Techniques for Drowsiness Detection

A drowsy person behave differently and can be distinguished through visual monitoring. Computer vision based measurement of the driver’s eye motion, eye blinking, head motion and/or head position utilized for the detection in Visual technique. A summary is presented in Table 1.

Table 1. Classification of drowsiness detection techniques.
Full size table

2.1 Subjective Measure Technique

Subjective measurement of level of drowsiness estimated on driver’s personal judgment. Many scales devised but most commonly used drowsiness scale is the Karolinska Sleepiness Scale (KSS) [20]. This technique is essential for the measurement of drowsiness and helpful in defining the threshold level for any other drowsiness detection systems.

Rumagit et al. used real time driving simulator for drowsiness detection using eye gaze and correlate with KSS for evaluation [21]. A roadside test was designed to measure driver’s fitness for continue driving [22]. At measurement site, volunteer drivers were stopped for the test, work-sleep data collection and KSS. Results obtained and information gathered were utilized to establish correlation and effectiveness of the test devised by Forsman et al. Jackson et al. utilized KSS for conformation of drowsiness association with slow eyelid closure with increased frequency and duration [23].

2.2 Psychological Measure Technique

For alertness and drowsiness stages, physiological signals are different. Taking advantage of this variation, psychological measures are taken with the help of dedicated sensors for the detection of drowsiness. A tabular survey on drowsiness detection with psychological technique is presented in Table 2.

Table 2. Previous work on physiological drowsiness detection.
Full size table

Eye movement detection through EoG signal to identify drowsiness was utilized by many researchers. In this technique, the eye orientation monitored through electric field generation by the potential difference between cornea and retina of the eye [24]. Disposable electrodes at outer corner of each eye and one electrode at forehead are utilized to pick Rapid eye movements (REM) and Slow Eye Movements (SEM) signals [25]. REM and SEM occur when the driver is awake and drowsy respectively.

The Heart Rate (HR) significant variation between different drowsiness stages, make it a vital sign. HR easy determination through ECG is well understood technique [26]. Another important information from ECG, Heart Rate Variability (HRV) which is the ratio of Low Frequency (LF) and High Frequency (HF) is of great interest. HRV decreases as the drowsiness rating increases on KSS [27].

Row physiological signals are prone to noises and artifacts due vehicle movement. Therefore, an effective filtering technique is a must requirement before processing a physiological signal [32]. Statistical features are extracted, using various feature extraction techniques like Discrete Wavelet Transform (DWT) and Fast Fourier Transform (FFT), before further analysis [33]. Finally, Artificial Neural Networks (ANN) [34], Linear Discriminant Analysis (LDA), or other similar analysis were utilized for the drowsiness detection.

Physiological signals usage for drowsiness detection is accurate and reliable compared with other methods. However, physiological signals measurement is intrusive [28]. Some researchers have investigated wireless technology for minimizing the intrusiveness. Some other researchers, eliminate the intrusiveness by putting sensors on the steering, on the cost of bearable errors due to improper electrode contact, considering the importance of user friendliness [27].

2.3 Vehicle Movement Based Techniques

An indirect way to measure driver drowsiness is through vehicle-based measurements. In this technique, sensors are attached with some part of the vehicle like steering wheel and/or acceleration pedal. Signals collected from the sensors are analyzed for drowsiness detection. Steering wheel based drowsiness measurement is presented by Jung et al. [27]. Relatively larger variations in driving speed due to sleep deprivation was investigated [19].

Steering Wheel Movement (SWM) measurement using steering angle sensor is widely used for determining the level of driver drowsiness. Many researchers establish that normal drivers make more steering wheel reversals than sleep deprived drivers [35]. Lane change effect is cancelled out by considering only small movement (0.5° to 5°) of the steering wheel [36]. Hence, small SWMs make driver drowsiness detection possible. SWMs are greatly dependent on road geometry and it limit its usefulness.

Standard Deviation of Lane Position (SDLP) has gained the interest of researchers due to its simplicity and usefulness for drowsiness detection. SDLP is measured through software in the simulation environment and lane position is tracked through external camera in the field experiment [37]. Through experimental verification, Ingre et al. established a direct relationship between SDLP and KSS [38]. However, above experiment reveals poor correlation due to significant difference between selected subjects. Furthermore, SDLP is dependable on external factors like climate, lighting and road marking.

3 Computer Vision and Drowsiness Detection

Drowsiness never comes instantly but appear with visually noticeable symptoms. These symptoms generally appear even well before drowsiness in every driver. These includes Eye closing for longer time, High eye blinking rate, Heavy eyelids, Rubbing eyes, Yawing, Head nodding, Inability to focus and Hard to concentrate.

3.1 Template Matching Technique

In this technique, a template image presence in a larger image is checked through comparison. The template is an already stored image, especially selected for the comparison purpose. Both close and open eye templates are provided to the system. The system use pictures from the video to check the states of the eyes, calculate eyes closure time and compare with a predefined time for sleepiness and eye-blinking [39].

This technique of template matching is easy and simple, so most of researchers have used this technique. Bergasa et al. used this technique and confirms its effectiveness for drowsiness detection [40]. For faster eyes detection, adoption of Kalman filter algorithm was investigated by Tang et al. and ended up with promising results [41].

3.2 Eye Blinking Technique

This technique detects the level of drowsiness and sleepiness by calculating the eye blinking rate and eye closure duration. The reason is that when a driver felt drowsy, his/her eye blinking rate and gaze between eyelids are different from normal situations. Using eye blinking rate drowsiness detection methodology is presented in Fig. 1 where eye-blinking classification is carried out through computer vision.

Fig. 1.
figure 1

Computer vision based eye blinking classification

Full size image

Ahmad and Borole monitor the position of irises and eye states in the technique [42]. They placed a camera at a suitable place in the vehicle and acquire video. Then by applying computer vision techniques to sequentially localize face, eyes and eyelids positions to measure ratio of closure.

3.3 PERCLOS Technique

PERCLOS means percentage of time eye closed in a given period. To sense the level of drowsiness PERCLOS is a well-known parameter. Yan et al. [43] consider that a human blinks once every 5 s on average that is 12 times blinks per minute. They tried gray-scale conversion and template matching for extracting data.

3.4 Yawning Technique

One of an important sign of fatigue is yawn. It is assume with a large vertical mouth opening compared. As compared to speaking, mouth is widely open in yawning process. Bhandari et al. detected yawn by face and then mouth tracking [44]. After yawn detection, the system alarms the driver.

4 Discussion and Recommendations

There are numerous techniques for drowsiness detection with embedded pros and cons. Each detection technique comes under a category depending upon the detection methodology and/or sensors used. The advantages and limitations of each category of drowsiness detection are presented in Table 3. The subjective technique, with real-time implementation limitation, is the simplest and provide basis for categorizing drowsiness. Psychological are promising with trustworthiness but embedded intrusiveness needed to be addressed without compromising reliability.

Table 3. Advantages and limitations of different techniques.
Full size table

Vehicular based detection solved intrusiveness issue but not reliable due to late detection. Now comes the computer vision based technique which is non-intrusive, easy to implement due to major advancements in the field including immense processing power.

Only few researchers have tried combining computer vision based detection with other technique. Tran et al. [45] combine visual and vehicular technique. They focused on eye behavior for extraction of facial expression and extract SWM as vehicular technique. Lee and Chung [46] combined the visual and psychological techniques for drowsy driver detection. An android based smart phone is utilized to process data extracted from the wireless sensors. Video sensor and a bio-signal sensor captures the driver image and photoplethysmography (PPG) signal. Dynamic Bayesian network integrate the two features for more authentic driver’s fatigue detection.

Nakamura et al. [47] faced the problem that some of the selected drivers did not show significant facial change of expression in their experiment. Their suggestion to overcome the shortfall is the use of EEG signal especially alpha channel. Authors proposed hybrid drowsiness detection is presented in Fig. 2. Here after detecting the eyes, two separate algorithms are used, one for PERCLOS and the other for eye-closing duration. Drowsiness detection in either case triggers the alarm for driver and co-passengers.

Fig. 2.
figure 2

Proposed hybrid drowsiness detection using image processing

Full size image

In this paper, a comprehensive review of various methods to detect drowsiness while driving is presented. This article mainly focuses on computer vision based techniques. It is intended for the development of a new hybrid algorithm to eliminate the drawbacks of discussed methods.

5 Conclusion

Every year thousands of people died in road accidents. Mostly, the accident occurs due to the drowsiness of the driver. Such causes can be avoided by exploiting the advanced technology. The progress in the field of image processing and computer vision made it possible to detect the drowsiness of the driver by monitoring drowsiness visual symptoms. The paper is a comprehensive review on various methods to detect drowsiness with the focus on computer vision based detection. It is intended that computer vision based detection of the hybrid symptoms eliminate the drawbacks of discussed techniques. The proposed system is more reliable and dependable for driver drowsiness detection. It is non-intrusive in its nature. A reasonable number of road accidents are preventable with the use of computer vision based drowsiness detection.