Abstract
In the paper a problem of analysing facial images captured by depth sensor is addressed. We focus on evaluating mouth state in order to estimate the drowsiness of the observed person. In order to perform the experiments we collected visual data using standard RGB-D sensor. The imaging environment mimicked the conditions characteristic for driver’s place of work. During the investigations we trained and applied several contemporary general-purpose object detectors known to be accurate when working in visible and thermal spectra, based on Haar-like features, Histogram of Oriented Gradients, and Local Binary Patterns. Having face detected, we apply a heuristic-based approach to evaluate the mouth state and then estimate the drowsiness level. Unlike traditional, visible light-based methods, by using depth map we are able to perform such analysis in the low level of even in the absence of cabin illumination. The experiments performed on video sequences taken in simulated conditions support the final conclusions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alioua, N., Amine, A., Rziza, M.: Driver’s fatigue detection based on yawning extraction. Int. J. Veh. Technol., Article no. 678786 (2014). https://doi.org/10.1155/2014/678786
Azim, T., Jaffar, M.A., Mirza, A.M.: Fully automated real time fatigue detection of drivers through fuzzy expert systems. Appl. Soft Comput. 18, 25–38 (2014)
Burduk, R.: The AdaBoost algorithm with the imprecision determine the weights of the observations. In: Intelligent Information and Database Systems, Part II, LNCS, vol. 8398, pp. 110–116 (2014)
Chang, H., Koschan, A., Abidi, M., Kong, S.G., Won, C.-H.: Multispectral visible and infrared imaging for face recognition. In: 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–6 (2008)
Craye, C., Rashwan, A., Kamel, M.S., Karray, F.: A multi-modal driver fatigue and distraction assessment system. Int. J. Intel. Transp. Syst. Res. 14(3), 173–194 (2016)
Cyganek, B., Gruszczynski, S.: Hybrid computer vision system for drivers’ eye recognition and fatigue monitoring. Neurocomputing 126, 78–94 (2014)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 886–893 (2005)
Forczmański, P., Kukharev, G.: Comparative analysis of simple facial features extractors. J. R. Time Image Process. 1(4), 239–255 (2007)
Forczmański, P., Kukharev, G., Shchegoleva, N.: Simple and robust facial portraits recognition under variable lighting conditions based on two-dimensional orthogonal transformations. In: 7th International Conference on Image Analysis and Processing (ICIAP). LNCS, vol. 8156, pp. 602–611 (2013)
Forczmański, P.: Human face detection in thermal images using an ensemble of cascading classifiers. In: Hard and Soft Computing for Artificial Intelligence, Multimedia and Security, Advances in Intelligent Systems and Computing, vol. 534, pp. 205–215 (2016)
Forczmański, P.: Performance evaluation of selected thermal imaging-based human face detectors. In: Proceedings of the 10th International Conference on Computer Recognition Systems CORES 2017. Advances in Intelligent Systems and Computing, vol. 578, pp. 170–181 (2018)
Fornalczyk, K., Wojciechowski, A.: Robust face model based approach to head pose estimation. In: Proceedings of the 2017 Federated Conference on Computer Science and Information Systems, FedCSIS 2017, pp. 1291–1295 (2017)
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. In: Proceedings of the 2nd European Conference on Computational Learning Theory, pp. 23–37 (1995)
Fu, R., Wang, H., Zhao, W.: Dynamic driver fatigue detection using hidden Markov model in real driving condition. Exp. Syst. Appl. 63, 397–411 (2016)
Intel RealSense Camera SR300 – Embedded Coded Light 3D Imaging System with Full High Definition Color Camera Product Datasheet, rev. 1 (2016). https://software.intel.com/sites/default/files/managed/0c/ec/realsense-sr300-product-datasheet-rev-1-0.pdf. Accessed 05 Oct 2018
Jo, J., Lee, S.J., Park, K.R., Kim, I.J., Kim, J.: Detecting driver drowsiness using feature-level fusion and user-specific classification. Exp. Syst. Appl. 41(4), 1139–1152 (2014)
Kong, W., Zhou, L., Wang, Y., Zhang, J., Liu, J., Gao, S.: A system of driving fatigue detection based on machine vision and its application on smart device. J. Sens. 2015, 11 pages (2015)
Krishnasree, V., Balaji, N., Rao, P.S.: A real time improved driver fatigue monitoring system. WSEAS Trans. Signal Process. 10, 146–155 (2014)
Nowosielski, A.: Vision-based solutions for driver assistance. J. Theor. Appl. Comput. Sci. 8(4), 35–44 (2014)
Makowiec-Dabrowska, T., Siedlecka, J., Gadzicka, E., Szyjkowska, A., Dania, M., Viebig, P., Kosobudzki, M., Bortkiewicz, A.: The work fatigue for drivers of city buses. Medycyna Pracy 66(5), 661–677 (2015)
Małecki, K., Nowosielski, A., Forczmański, P.: Multispectral data acquisition in the assessment of driver’s fatigue. In: Mikulski, J. (ed.) Smart Solutions in Today’s Transport, TST 2017. Communications in Computer and Information Science, vol. 715. pp. 320–332 (2017)
Mitas, A., Czapla, Z., Bugdol, M., Ryguła, A.: Registration and evaluation of biometric parameters of the driver to improve road safety, pp. 71–79. Scientific Papers of Transport, Silesian University of Technology (2010)
Ojala, T., Pietikinen, M., Harwood, D.: Performance evaluation of texture measures with classification based on Kullback discrimination of distributions. In: Proceedings of the 12th International Conference on Pattern Recognition, vol. 1, pp. 582–585 (1994)
Smiatacz, M.: Liveness measurements using optical flow for biometric person authentication. Metrol. Meas. Syst. 19(2), 257–268 (2012)
Staniucha, R., Wojciechowski, A.: Mouth features extraction for emotion classification. In: Proceedings of the 2016 Federated Conference on Computer Science and Information Systems, FedCSIS 2016, pp. 1685–1692 (2016)
Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)
Zhang, Y., Hua, C.: Driver fatigue recognition based on facial expression analysis using local binary patterns. Opt. Int. J. Light. Electron Opt. 126(23), 4501–4505 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Forczmański, P., Kutelski, K. (2019). Driver Drowsiness Estimation by Means of Face Depth Map Analysis. In: Pejaś, J., El Fray, I., Hyla, T., Kacprzyk, J. (eds) Advances in Soft and Hard Computing. ACS 2018. Advances in Intelligent Systems and Computing, vol 889. Springer, Cham. https://doi.org/10.1007/978-3-030-03314-9_34
Download citation
DOI: https://doi.org/10.1007/978-3-030-03314-9_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03313-2
Online ISBN: 978-3-030-03314-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)