Abstract
Forewarning drowsy drivers can reduce the number of road accidents. A non-intrusive drowsiness detection system is implemented, which alerts the driver on the onset of drowsiness. A Pi camera module attached to Raspberry Pi is used to acquire and process the live video of the driver. Haar face detector in OpenCV is used for face detection followed by 68 points of facial landmark identification. Eye and Mouth Aspect Ratios, blink rate and yawning rate are the features extracted. Drowsiness detection is done using two methodologies viz. a threshold-based one and the other, employing artificial intelligence. The machine learning techniques used are LDA and SVM. Feedback is provided as an alarm if a driver is found to be drowsy. The analysis shows that machine learning-based techniques viz. LDA and SVM outperform threshold technique for the dataset considered.
Access provided by Autonomous University of Puebla. Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Drowsiness while driving is one of the principal causes of road accidents. According to the surveys conducted by the U.S National Highway Safety Administration [1], every year almost 60,000 road accidents are due to drowsiness. The above argument is also supported by Sinan et al. [2] with a statistical figure in terms of percentage, i.e., 25–30% of driving accidents are caused due to drowsiness of the driver. Drowsiness or even sleepiness while driving can be due to loss of attention, long working hours, fatigue, monotonous driving, late-night drives or may be due to consumption of some medications or alcohol [3]. An effective system which can automatically identify the onset of drowsiness can save many lives.
To address this, the factors indicating the onset of drowsiness need to be figured out first. In a broader perspective, the survey papers[2, 4,5,6] states that the aforementioned factors fall under 3major categories viz. behavioral factors, physiological factors and vehicular factors. In detection based on behavioral factors, computer vision-based techniques [7,8,9] are adopted, for capturing the videos or images of the driver’s drowsy face. Closure of eyelids, blinking rate of an eye, gaze, yawning, facial expressions, movements of the head and driving time [3] are some of the significant behavioral factors. The vehicular factors include lane detection, steering wheel angle and angular velocity. The major physiological factors are electroencephalogram (EEG) [10, 11], electro-oculogram (EOG), heart rate, breathing rate and body temperature. Analysis of driving patterns is not accurate because they are implemented by detecting steering wheel, brake and accelerator. Moreover, these characteristics depend on the type of road and drivers’ skills [12]. Physical characteristics of driver are detected by body sensors like EEG, Electro Cardiograms (ECG), EOG and blood pressure [13]. It is uncomfortable for the driver to wear all sensors at the time of driving. On the contrary, analysis of driver behavior consisting of camera-based methods [14] will be more accurate than other methods mentioned, since it takes care of the driver’s comfort without distracting his/her attention.
In this fast-moving world, in order to utilize time effectively, people prefer over-night travels, though it is dangerous. Driver drowsiness detection systems are very much crucial to safeguard people and lives. Hence an ample amount of research has been carried out in this regard. Owing to accuracy and reduced discomfort to the driver, computer vision-based techniques have opted for this analysis. Some of the related researches in this area are discussed below. Continuous monitoring is always prioritized while designing and developing driver alert systems. Similar researches can be seen in several works of literature. Ali et al. [15] designed a fatigue detection system using a web camera for continuous monitoring of drivers. The eye region was detected from the video using OpenCV libraries. Prediction of drowsiness was accomplished by Partial Least Squares Regression, using Percentage of Eyelid Closure (PERCLOS). They also computed the Eye Aspect Ratio (EAR) and Mouth Aspect Ratio (MAR). In both cases, if the threshold limit is crossed, an alert message will be generated, followed by an alarm.
To reduce the number of casualties, Mehta et al. [16] developed an android application for driver drowsiness detection using facial features. Among the machine learning techniques used for analysis, bagging classifier yielded the highest accuracy in detection. The author also suggests incorporation of the above app with the currently available cab hiring services. Accidents may be due to natural drowsiness of the driver or maybe due to alcohol consumption. Charniya et al. [17] carried out the analysis to detect the drowsy or drunk state of the driver while driving. This paper mainly focuses on the detection of drowsiness from the face and facial features with a yawning detector. Face detection was done using Haar cascade features. State of the eye was classified either as open or closed using Support Vector Machines (SVM). The visual features such as eye index and pupil activity were used to identify if he/she was either in alert or non-alert state. The relative deviation between estimated and ground-truth pupil centers was used for this and a threshold was used to give the warning. Yawn was detected by the Viola-Jones algorithm. For prohibiting the ignition of the vehicle, if the driver is drunk, an alcohol detector MQ303A was also included.
This work focuses on implementing a driver alert system, on the onset of drowsiness. This includes a Raspberry Pi with Pi camera module for acquiring the videos of driver’s face while driving. These captured images from video frames are preprocessed for facial landmark detection. 68 landmark features of the face from eyes, eyebrows, nose, mouth, and jawline are used to detect the onset of drowsiness. EAR, MAR, blink rate and yawn rate are the features used. Three types of detection techniques, including 2 supervised learning algorithms are used for detecting drowsiness. The former is a threshold technique and the latter is based on linear discriminant analysis (LDA) and Support Vector Machine (SVM). On detecting drowsiness, the driver will get alerted with the help of a buzzer. The performance of this system developed using different classifiers are measured in terms of accuracy, precision, recall and F1 scores. The paper is organized as follows: The proposed system is discussed in the next session. The detail of the hardware system developed is given in Sect. 3. The data processing part including the database, preprocessing, features acquired are elaborated in Sect. 4. A brief description of the detection techniques used is given in Sect. 5. A detailed analysis of the results obtained is given in Sect. 6, followed by a conclusion and future work.
2 Proposed System
A Raspberry Pi-based driver drowsiness detection system is proposed and developed. The Pi camera module is used to collect the real-time video of the driver’s face. Following this, the Raspberry Pi module segments the acquired video into static frames of facial images. After identifying the face, the images are resized and features are extracted. This includes the identification of facial coordinate points corresponding to eyes and mouth. EAR and MAR are computed with the eye coordinates and mouth coordinates, respectively. Further, the rate of blinking and yawning are also taken into consideration for detecting drowsiness. Two types of detection methods are adopted in this work. First one is a threshold-based technique and the second involves computer-assisted method with the help of machine learning. Based on the aforementioned parameters, if the person is identified to be drowsy, an alarm will be turned on to forewarn the driver. A display module is also attached to cross verify whether the facial landmarks are identified correctly. The performance evaluation of the proposed system is carried out with the help of two databases: one a publically available and the other one, custom made database captured using the associated hardware. The block diagram of the proposed system is as given in Fig. 1.
3 Hardware System Developed
In this project, Raspberry Pi 4, Pi Camera module and a display are used. Raspberry Pi has 2 GB RAM, USB 3.0 Ports and also supports dual 4 K video output. The Pi camera module is a portable lightweight camera that supports Raspberry Pi. It communicates with Pi using the MIPI camera serial interface protocol. Python libraries are used to control the camera module and processing of these images from the captured video. The hardware module is shown in Fig. 2a.
4 Data Processing
4.1 Database
The detection methodologies are analyzed using a publically available dataset, Closed Eyes in the Wild dataset (CEW). Further, the performance of a drowsiness detection system developed is evaluated using a custom dataset, acquired using Pi camera module.
4.1.1 Closed Eyes in the Wild Dataset (CEW)
CEW dataset was developed by Song et al. [18] for detecting closed eyes in still images. This dataset comprised of 2423 subjects. It has 1192 closed eyes, which were directly downloaded from the Internet and 1231 with open eyes, selected from another publically available database viz. Labeled Face in the Wild (LFW) [19]. 120 randomly selected samples from CEW dataset is used for analysis.
4.1.2 Custom Dataset
An image dataset is developed with the help of students in Amrita Viswa Vidyapeetham. The subjects were asked to undertake a virtual driving game for 2 h, to simulate a monotonous driving state and then the videos were acquired. The database consists of 240 images with 120 samples in closed eye state and the remaining in open eye state. Both of these states also have images while the subjects yawned. Sample image is shown in Fig. 2b.
4.2 Preprocessing and Feature Extraction
Preprocessing followed by facial feature extraction is done in four subsequent steps. The first step comprises of image extraction from video frames and converting them to grayscale. The second step involves the detection of the face from the grayscale image so that facial features can be identified easily. The salient points of the face corresponding to eye and mouth regions are detected in the third step using the Haar cascade face detector [20]. This includes 68 facial landmarks [21] in order to localize the eyes, eyebrows, nose, mouth, and jawline. The final step involves detecting the eye and mouth coordinates for computing EAR and MAR. EAR is calculated from horizontal and vertical distance of the eye, and MAR, from the respective values of mouth. The computation of these ratios is given in Eqs. (1) and (2), where pxx is the corresponding facial landmark points shown in Fig. 2b. The averaged values of EAR and MAR are taken from the adjacent 16 frames of the video for further analysis.
5 Drowsiness Detection Techniques
Two main methodologies are adopted for detecting the onset of drowsiness. First one is a threshold-based and the second is based on 2 different machine learning techniques viz. LDA and SVM.
5.1 Threshold-Based Drowsiness Detection Technique
Average values of EAR and MAR are taken over 15 images in the video. If the EAR is above 0.3, then it is considered as an awake state. If the drowsy state is detected more than 3 s then the person is considered as sleeping and alarm is turned on. Similarly, if the MAR value crosses a threshold of 0.6, for more than 3 s then it is considered as yawning, or else it will be ignored as talking. These threshold values are chosen based on a trial and error method. If the number of blinks is greater than 16 per minute then the person is considered as drowsy [22]. If the number of yawns is more than 2 per minute, then also drowsiness is detected [23].
5.2 Machine Learning-Based Drowsiness Detection Technique
The machine learning techniques used in this work are LDA and SVM. LDA tries to differentiate between the drowsy and awake states by fitting a linear discriminant, with a constraint to reduce misclassification. SVM, on the other hand, fits a hyperplane, separating the two states such that the margin is maximized. SVM’s are capable of fitting curvilinear hyperplanes with the help of kernel functions.
6 Results and Discussion
The Pi camera module used in the system for acquiring live videos is Rev 1.3. From this, the still images are collected at an average frame rate of 16 fps. For preprocessing, OpenCV’s Haar cascade face detector is used to detect the faces and subsequently resize the images to 400 × 400 pixels. Further 68 facial landmarks are identified using OpenCV libraries. Features EAR, MAR, Blink rate and yawn rate are obtained from the localized points corresponding to eyes, eyebrows, nose, mouth, and jawline.
The drowsiness detection is done using two methods. A simple threshold technique is the first one and the second one, based on supervised machine learning viz. using LDA and SVM. Both of these techniques use the above-mentioned features. Further, the system designed is validated using a publically available database as well as a custom made one. The performance measures of the system are compared in terms of accuracy, precision, recall and F1 score. Table 1 shows the performance analysis of the system using CEW as well as a custom dataset. For CEW dataset, LDA gave higher accuracy and F1 score viz. 0.908 and 0.906, respectively, compared to other techniques. On the other hand, for the custom dataset, SVM gave higher accuracy and F1 score viz. 0.9 and 0.883, respectively, than others. In short, machine learning-based techniques exhibit higher performance measures than threshold-based one.
7 Conclusions and Future Work
The drowsiness detection system is developed using a Raspberry Pi. The system is tested with 2 datasets, a publically available one and a custom made one. Drowsiness detection is carried out both using a threshold-based technique and with a supervised learning technique. The latter yielded better results in the respective datasets analyzed. Future work aims at incorporating physiological signals such as EEG along with facial features to enhance the reliability of the system.
References
Charniya NN, Nair VR (2017) Drunk driving and drowsiness detection. In: International conference on intelligent computing and control (I2C2), Coimbatore
Kaplan S, Guvensan MA, Yavuz AG, Karalurt Y (2015) Driver behavior analysis for safe driving: a survey. IEEE Trans Intell Transp Syst 16(6):3017–3032
de Naurois CJ, Bourdin C, Stratulat A, Diaz E, Vercher JL (2019) Detection and prediction of driver drowsiness using artificial neural network models. Accid Anal Prev 126:95–104
Ramzan M, Khan HU, Awan SM, Ismail A, Ilyas M, Mahmood A (2019) A survey on state-of-the-art drowsiness detection techniques. IEEE Access. https://doi.org/10.1109/ACCESS.2019.2914373
Ngxande M, Tapamo JR, Burke M (2017) Driver drowsiness detection using behavioral measures and machine learning techniques: a review of state-of-art techniques. In 2017 pattern recognition association of South Africa and robotics and mechatronics (PRASA-RobMech). IEEE, pp 156–161
Mittal A, Kumar K, Dhamija S, Kaur M (2016) Head movement-based driver drowsiness detection: a review of state-of-art techniques. In: IEEE international conference on engineering and technology (ICETECH), Coimbatore
Koresh MH, Deva J (2019) Computer vision based traffic sign sensing for smart transport. J Innovative Image Process (JIIP) 1(01):11–19
Suma V (2019) Computer vision for human-machine interaction-review. J Trends Comput Sci Smart Technol (TCSST) 1(02):131–139
Dhaya R (2020) CCTV surveillance for unprecedented violence and traffic monitoring. J Innovative Image Process (JIIP) 2(01):25–34
Poorna SS, Arsha VV, Aparna PTA, Gopal P, Nair GJ (2018) Drowsiness detection for safe driving using PCA EEG signals. Progress in computing, analytics and networking. Springer, Singapore, pp 419–428
Poorna SS, Deep A, Hariharan K, Jain RR, Krishnan S (2020) A transfer learning approach for drowsiness detection from EEG signals. In: International conference on innovations in computational intelligence and computer vision (ICICV-2020)
Bhatt PP, Trivedi JA (2017) Various methods for driver drowsiness detection: an overview. Int J Comput Sci Eng (IJCSE) 9(3):70–74
Awais M, Badruddin N, Drieberg M (2017) A hybrid approach to detect driver drowsiness utilizing physiological signals to improve system performance and wearability. Sensors 17(9):1991
Vesselenyi T, Moca S, Rus A, Mitran T, Tătaru B (2017) Driver drowsiness detection using ANN image processing. In: IOP conference series: materials science and engineering, p 252
Ali M, Abdullah S, Raizal CS, Rohith KF, Menon VG (2019) A novel and efficient real time driver Fatigue and Yawn detection-alert system. In: 2019 3rd international conference on trends in electronics and informatics (ICOEI). IEEE, pp 687–691
Mehta S, Mishra P, Bhatt AJ, Agarwal P (2019) AD3S: advanced driver drowsiness detection system using machine learning. In: 5th international conference on image information processing (ICIIP). IEEE, pp 108–113
Charniya NN, Nair VR (2017) Drunk driving and drowsiness detection. In: International conference on intelligent computing and control (I2C2). IEEE
Song F, Tan X, Liu X, Chen S (2014) Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients. Pattern Recogn 47(9):2825–2838
Huang GB, Ramesh M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report 07–49, University of Massachusetts, Amherst
Singh V, Shokeen V, Singh B (2013) Face detection by Haar cascade classifier with simple and complex backgrounds images using OpenCV implementation. Int J Adv Technol Eng Sci 1(12):33–38
Kazemi V, Sullivan J (2014) One millisecond face alignment with an ensemble of regression trees. In: IEEE conference on computer vision and pattern recognition, Columbus, OH
Nakano T, Kato M, Morito Y, Itoi S, Kitazawa S (2013) Blink-related momentary activation of the default mode network while viewing videos. Proc Nat Acad Sci 110(2):702–706
Corey TP, Shoup-Knox ML, Gordis EB, Gallup GG Jr (2012) Changes in physiology before, during, and after yawning. Front Evol Neurosci 3:7
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Madireddy, R. et al. (2021). Driver Drowsiness Detection System Using Conventional Machine Learning. In: Smys, S., Palanisamy, R., Rocha, Á., Beligiannis, G.N. (eds) Computer Networks and Inventive Communication Technologies. Lecture Notes on Data Engineering and Communications Technologies, vol 58. Springer, Singapore. https://doi.org/10.1007/978-981-15-9647-6_31
Download citation
DOI: https://doi.org/10.1007/978-981-15-9647-6_31
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-9646-9
Online ISBN: 978-981-15-9647-6
eBook Packages: EngineeringEngineering (R0)