Keywords

1 Introduction

Circadian rhythm exists in human body and influences people’s life and production. The discovery of the molecular mechanism for circadian rhythm in organisms won the Nobel Prize in 2017 [1]. Circadian rhythm molecular mechanism controls body’s nervous systems and influences body behaviours, further affecting people’s life and production, such as to keep health, to work at sunrise and sleep at sunset and to lead to fatigue and further decreasing productivity, threatening production safety. The circadian rhythms of individuals are regulated by endogenous (e.g., circadian pacemaker, peripheral oscillators, clock genes) and exogenous factors (e.g., light, feeding, age, social behaviour, work and schedules) and are different from each other [2, 3]. Therefore, it is necessary to establish simple and objective methods to learn individual behaviour circadian rhythm more precisely and accurately in order to arrange individual life more efficiently, keep health and ensure production safety. There are several methods available to assess circadian rhythm including measurement of core body temperature and melatonin, cortisol production, questionnaires, actigraphy. The methods for measurement of temperature, melatonin and cortisol production require rigid detection conditions to avoid disturbances. The questionnaires are subjective and classify only rough three kinds. The actigraphy is susceptible to masking and artefacts [4]. In this paper, we established a behavioural method to finely characterize circadian rhythm through measurement of volunteers’ eye behaviours by combining convolutional neural network and the percentage of eyelid closure over the pupil over time (PERCLOS). The method was applied to 7 volunteers. The results showed that 6 young volunteers’ PERCLOS values increased from morning to night in accordance with the circadian rhythm of youth. The method is easy to operate and objective. There is reason to believe that the established method will expand to use and help individuals to learn their behavioural circadian rhythm and arrange their life more efficiently.

2 Experiment

2.1 Establishment of the Convolutional Neural Network Model

The Keras was applied to construct the neural network model on the platform of IntelCore 17, NVIDIA GTX 1080Ti GPU and the operating system of Ubuntu 16.04. The convolutional neural network (CNN) model was written in Python programming language. The model consisted of three convolutional layers, three maximum pooling layers, one fully connected layer and a sigmoid classifier (Fig. 1). The Zhejiang University (ZJU) public data (index P70) was set as the training data. The input image was a 26 * 34 greyscale image. After a convolution layer, 26 * 34 * 32 feature maps were output. After a maximum pooling layer, 13 * 17 * 32 feature maps were output. After the second convolution layer, 13 * 17 * 64 feature maps were output. After the second maximum pooling layer, 6 * 8 * 64 feature maps were output. After the third convolutional layers, 6 * 8 * 128 feature maps were output. After the third maximum pooling layer, 3 * 4 * 128 feature maps were obtained. Then, the feature maps dimension was transformed into one-dimensional vector with inputting a fully connected layer, of which the length was 512, and finally, through sigmoid classifier classification, results were output with the category of open or close status.

Fig. 1
figure 1

Structure of the convolutional neural network model for PERCLOS detection

Full size image

2.2 Data Collection and Treatment

7 healthy volunteers were recruited in this research: one was a female at the age of 51 and the other 6 volunteers were aged from 25 to 30, 3 male. During the experiment, the volunteers were on their holidays and woke up naturally without drinking coffee or tea. Each volunteer was asked to play a simple game on a personal computer with a camera to get his/her videos for 5 min on the test. The speed of collecting videos was 24 images every second. The volunteers were required to do the test 4 times a day that was just after getting up in the morning (I), at noon (II), in the evening (III) and just before going to bed (IV).

The collected videos were treated with the CNN model according to the process as shown in Fig. 2. Briefly, the video was inputted into multitask convolutional neural network (MTCNN) to detect the face and locate the eyes. Then the image was converted into greyscale image by greyscale processing. The greyscale image was processed with the combination of convolutional layer and pooling layer * 3, and the feature maps dimension was transformed into one-dimensional vector with inputting a fully connected layer. Finally, the classification results of open or close status were output through sigmoid classifier as the PERCLOS value and recorded as 0 for open or 1 for close. The PERCLOS values were calculated out as an average percentage of close status every minute.

Fig. 2
figure 2

Process of volunteer video treatment for PERCLOS

Full size image

The PECLOS data were treated with variance analysis, repeated measurement and multiple comparisons.

3 Result Discussion and Conclusion

In the recent years, the application of convolutional neural network algorithm to PERCLOS detection became prevalent because of its good performance and easy to get related resources [5]. In this study, we established a CNN model for PERCLOS detection. In order to increase the CNN model performance, we employed the ZJU public database to train and optimize the parameters of the model. There were two kinds of images in the ZJU database including 7000 eye open images and 1984 eye close images. 5770 eye open images applied to training, the other 1230 to test. And 1574 eye close images applied to training, the other 410 to test. After the training, the validation accuracy rate of the model reached about 99.96%, which was a satisfactory result in eye detection for PERCLOS values.

For face detection and eye location, we compared MTCNN algorithm and Haar-like algorithm. Results showed that the Haar-like algorithm was tended to false recognition that nares recognized as eyes. However, we found that MTCNN algorithm overcame the difficulty easily in accordance with other researchers [6].

The PERCLOS detection results for volunteers in 4 certain times (including I, II, III, IV time) shown in Fig. 3. We noted that volunteers were tending to shake their heads at the onset of tests. Therefore, we intercepted videos from 2 to 5 min for statistical analysis. In considering the influence of age, we separated the 7 volunteers into two group as young group (aged from 25 to 30, black curve in Fig. 3) and elder group (blue curve in Fig. 3). The results of young group showed that the average PERCLOS values were 0.091 (I), 0.086 (II), 0.121 (III), 0.172 (IV) for 4 times, and their standard deviations were 0.018 (I), 0.016 (II), 0.024 (III), 0.032 (IV), separately. The PERCLOS data of young group were analysed with 1-factor repeated measures analysis of variance, multiple comparison analysis and pairwise T-test. The results showed that: circadian rhythm effect was very significant (P, 0.001); the PERCLOS value of IV time was significantly different from that of I (P, 0.007), II (P, 0.016) and III (P: 0.016); while the average PERCLOS value of I time was not significantly different from that of II (P, 0.672) and that of III (P, 0.330), the average PERCLOS value of II time was not significantly different from that of III (P, 0.350). The curve of PERCLOS was stable at the onset part from I to III, appearing no significant difference among the three times and increased sharply at the IV time, appearing significant difference from I to III time. These results illustrated that people keep energetic in the daytime from morning to evening (from I to III) and fell to sleepy at night (at IV). The curve of PERCLOS was similar to cortisol production of youth which kept a stable level during daytime and changed significantly decreased at night, while melatonin (another circadian rhythm-related secrete) began to increase secretion and lead to people sleepy [7, 8]. In order to investigate the effectivity, we recruited an elder female volunteer for the test. Her PERCLOS curve (blue curve in Fig. 3) was very different from that of young persons (black curve in Fig. 3). In the morning and at noon, she showed sleepier than youth, and her energetic time was in the evening. These results indicated that our PERCLOS method was able to distinguish the different circadian rhythm effectively and to finely characterize individual circadian rhythm.

Fig. 3
figure 3

Curve of PERCLOS values

Full size image

Circadian rhythm is very important for children growing, people health and production. Several methods have established for learning individual circadian rhythm, but these methods are complex and difficultly available. PERCLOS has been applied as an index to detect driver fatigue since the recommend of the office of motor carriers, Federal highway administration, the USA in 1998 [9]. To a certain extent, there are some relationships between circadian rhythm and fatigue. The circadian rhythm is regarded as one of the important reasons to lead to fatigue [10]. To our knowledge, we, for the first time, tried to employ PERCLOS as an index to characterize circadian rhythm. We established a simple and easy to operate method to measure the PERCLOS value. The measurement results of volunteers showed that the method was successfully detecting PERCLOS values, and the change of PERCLOS values was consistent with circadian rhythm.

In conclusion, circadian rhythm is very important for child growing, health keeping, and efficient and safe working. It is necessary to learn individual circadian rhythm for arranging his/her life more efficiently. In this paper, we were inspired by fatigue detection to have established a PERCLOS method for fine characterization of the individual circadian rhythm. The method was simple and easy to operate. We believe that this method will expand to use and help individuals to learn their behavioural circadian rhythm and arrange their life more efficiently.