Keywords

1 Background and Purpose of Research

Terrorism is prevalent around the world and has been a rising threat to humanity. However, it can be minimised significantly if the country has good security measures, vigilance is thus vital in this case when combating terrorism. In Singapore, SGSecure movement was launched in September 2016 to sensitise, train and mobilise the community in fight against terror, where every member of the community must do his part by staying alert to ever-present security threats [1]. For instance, security personnel on patrol who are viewing CCTV monitors should be increasingly vigilant of unusual or out-of-place behaviour in these perilous times [2]. Low vigilance level will cause the security personnel to spot suspicious people or items at a lower frequency—taking a longer time to react. Thus, it is important to have a reliable method to facilitate assessment of the symptoms of decline in vigilance level, so as to increase the security level in a country and minimise the possibility of terrorist attacks due to low vigilance level.

While EEG is the most commonly studied physiologic measure of vigilance, various measures of eye movement have also been used [3]. There have been studies using high-end eye trackers (Eyelink 1000) to assess vigilance level [4], but there has not been any studies using low-end eye trackers, such as Tobii EyeX. In our study, we first evaluated the performance of low-cost eye trackers in tracking gaze. To do so, we evaluated the optimal conditions when using a low cost remote eye tracker, Tobii EyeX eye tracker. The optimal conditions required when using the eye tracker, which include extrinsic parameters that will ultimately affect the precision and accuracy of the eye tracker (e.g. presence of chin rest, light intensity, distance from eye tracker), can then be obtained.

We determined how the gaze can be used to quantify vigilance and lastly, how EEG and gaze features are related to assess vigilance. With these, we evaluated if the information collected from the users can be used to optimize devices, or software interfaces to maintain vigilance level. In other words, whenever vigilance levels fall below a certain value, respondents can be notified to initiate counteraction.

2 Objective and Hypothesis

Our goal is to evaluate how vigilance level can be assessed reliably using a low cost eye tracker. We hypothesised that the repetitive visual stimuli induce mental fatigue and cause the subjects to feel drowsy. This caused a decline in vigilance level which can be observed from results of subjective rating based on reaction time, the change in raw EEG data and frontal asymmetry. Blink rate (BR) can be compared with these results to find out how it can be used to evaluate user’s vigilance level.

3 Experiment Design and Data Collection

Figure 1 shows the experimental setup with different sensors. Table 1 explains the experimental design and how data collection will be done in our experiment.

Fig. 1
figure 1

Experimental setup

Full size image
Table 1 Devices and their measured parameters in experiment
Full size table

We sent out a pre-experiment survey to a high school student population who are females and received a total of 222 responses. Through this survey, we have gathered information that the average time majority can stay vigilant for is within 5–15 min and that loud music is the most common form of distraction.

3.1 Experimental Design for Sensor Evaluation

Sensor Evaluation will consist of two parts: Eye Calibration & Sensors Checking and Random Dot Experiment. The Eye Calibration & Sensors Checking would take a total of 10 min. The Random Dot Experiment will have 6 sessions with different variables and each session would last for 5 min, with a rest period of 1 min in between.

An experiment will first be designed to assess the hypotheses: What are the optimal conditions in which a low cost eye tracker can perform best in, and to evaluate the performance of a low cost eye tracker. This experiment will show a dot at different places around the screen. The subject would then be asked to look at the dot at the position it appears (Fig. 3), which the subject then have to press the spacebar, to indicate that he/she has seen the dot. In each session, we repeated the experiment and change some variables as seen in Table 2. This experiment also ensures that the data collected later in our actual controlled experiment is the most accurate and precise.

Table 2 Variables in each experiment session
Full size table

3.2 Experimental Design for Vigilance Testing

This controlled experiment is approximately 40 min in length, and consists of two tasks: Psychomotor Vigilance Task (PVT) and a Spot the hidden item (SPOT) test which is a simulation of security checks. This study is designed to assess the hypothesis that low cost remote eye trackers can objectively assess vigilance level.

At the start and during resting periods of experiment, subjects are to rate their sleepiness level on the 9-level Karolinska Sleepiness Scale (KSS) (Fig. 2) that is used for evaluating subjective sleepiness level.

Fig. 2
figure 2

Karolinska sleepiness scale

Full size image

As shown in Table 3, the sensor setup and calibration will be conducted to check the accuracy and performance of the eye tracker. After every task, rest periods are planned with KSS survey, so that the participants can start every task in the same relaxed condition (about 1 min for the subjects to close their eyes to relax), regardless of what the previous task is.

Table 3 Experiment flow with different tasks
Full size table

Subsequently, a special test will be used in our experiment to determine the relationship between the vigilance level and sleepiness of a person. In the experiment, the modified PVT is a simple task where the subject presses a button as soon as the light appears to assess the reaction time. The dot will turn on randomly every few seconds for 5 min. The test will last for 5 min with a dot appearing at the different spots, changing every 3 s. The purpose is to measure vigilance level, and give a numerical measure of sleepiness by analysing the lapses in attention of the tested subject (Fig. 3).

Fig. 3
figure 3

Screenshot of PEBL’s perceptual vigilance test

Full size image

We designed 2 parts of this test, PVT and PVT (music) where the difference is the presence and absence of loud music that act as distractor. We have chosen to use loud music as based on our survey outcomes, 34% of the respondents have stated that out of the different sound distractions (soft music, loud music, whispers, conversations), they are most distracted by loud music. Other studies have also shown that sustained attention, or vigilance, can be related to many aspects of musical influence [5].

The SPOT test will be a simulation of security check where subjects will be required to spot certain shapes that are hidden between different shapes. Subjects are then tasked to click on the 10 hidden shapes (given to subjects in on-screen instructions) that they can find to successfully complete one round. The entire test will only end after 15 min.

There would be a total of 3 different levels of this test. Starting with the easiest level to find 10 hidden yellow circles filled with many different shapes of many different colours (seen in Fig. 4). Yellow circles have been used as yellow is the most fatiguing color and causing more eye irritant. Subjects will only be given 3 min for this difficulty level, and subjects are required to finish as many rounds of this difficulty level as possible.

Fig. 4
figure 4

Screenshot of easy level of SPOT

Full size image

After 3 min, subjects enter the medium level where subjects will have to find 10 red squares, the shapes would be more hidden, and may even be hidden behind red rectangles. This makes it more difficult for subjects to find the hidden red squares. A longer period of time of 5 min will be given to complete the search task. Next stage is hard level, where subjects would be required to find 10 dark green triangles. Different shades of green triangles are around the screen to confuse the subjects, and the dark green triangles will be placed behind rectangles and squares, making this level the most difficult. This last difficulty level would take a total of 7 min. The SPOT test would therefore take a total time of 15 min.

After 1 min rest period, the subject will have to complete another round of PVT. The only difference would be that the subject will have a high possibility of losing their original vigilance level, due to the long period of time spent completing the entire controlled experiment. In order to ensure that there is a significant change in vigilance level, we had gathered the results from our pre-experiment survey, regarding the average time one can stay vigilant. From the survey of 220 respondents, more than half (55.2%) can only stay vigilant within an average time of 10–20 min. Therefore, the time period between the first and last PVT is more than 20 min. This will be the end of the controlled experiment. Subjects will then be asked again about their 9-level KSS, to evaluate the subjects’ subjective sleepiness.

From the sensor evaluation we determined the optimal conditions which a low cost eye tracker can perform best in. Subjects have to stare at the dot when it appears, showing randomly around the screen. During the 6 different sessions, variables will be changed as shown in Table 2 and the presence of a chin rest. We used the optimal conditions obtained to conduct the vigilance testing. The modified PVT assesses the reaction time of subjects. For PVT (music), loud music was played as a form of distraction, causing their vigilance level to drop. For the Spot Test, it was to simulate security checks and assess the users’ vigilance level with the sensors. Subjects were asked to find 10 hidden shapes within a limited time. While resting, we instructed subjects to complete 9-level Karolinska Sleepiness Scale (KSS) to evaluate their subjective vigilance rating. We used OpenSesame to implement data collection application and performed different statistical testing using Excel Analysis Toolpak. We then obtained reaction time for PVT and KSS rating which can be used as surrogate measures of vigilance levels to evaluate both the eye tracker and EEG.

4 Data Analysis and Evaluation

We collected data from 10 and 15 healthy students (aged 14–17, all female) in the sensor evaluation and vigilance testing respectively. Each experiment lasted approximately 1 h. Students’ EEG signals and eye gaze coordinates were recorded.

Sensor Evaluation

All eye tracker data were mapped to [0–1], making all the x and y values within the range of 0–1, which is expressed as:

$$ v' = \frac{v - min\,A}{max\,A - min\,A} \times \left( {new\_max\,A - new\_min\,A} \right) + new\_min\,A $$
(1)

Following that, we had to calculate the accuracy of each point, by firstly calculating

$$ \begin{aligned} & Pixel\;Accuracy\;Both \\ & = \sqrt {\left( {target\;X - gaze\;point\;X} \right) + \left( {target\;Y - gaze\;point\;Y} \right)} \\ \end{aligned} $$
(2)

The accuracy of each point is calculated by taking Eq. (1) and (2). To calculate the precision of each points, we first had to find the mean of the two eyes which is expressed by:

$$ dX = diff\left( {\frac{{\left( {Temp\,Gaze\,Point\,X\,Right + Temp\,Gaze\,Point\,X\,Left} \right)}}{2}} \right) $$
(3)
$$ dY = diff\left( {\frac{{\left( {Temp\,Gaze\,Point\,Y\,Right + Temp\,Gaze\,Point\,Y\,Left} \right)}}{2}} \right) $$
(4)
$$ diffs = \sqrt {dX^{2} + dY^{2} } $$
(5)

To calculate the precision of each points, we then used

$$ Pixel\,Precision\,Both = \sqrt {mean(diffs)^{2} } $$
(6)

We conducted the T-test on the accuracy and precision values of each dot during the different sessions. The objective is to find out if the different variables significantly affect the precision and accuracy of the eye tracker.

The following features were considered to assess vigilance levels. Reaction time: where eye gaze coordinates were labelled with the times participants pressed the “space”. Reaction time is used to find out the change in vigilance level, as reaction time indicates perceptual vigilance.

Blink Rate

Eye position coordinates revealed that a decline in vigilance level, was characterised by a higher blink rate. To identify instances of a decline in vigilance level during the experiment, eye position coordinates that were at 0 for both left and right eyes that last for between 0.1 and 0.4 s were labelled as blinks. The number of blinks per minute is then calculated to find the blink rate.

Frontal Asymmetry

From EEG data, we used the alpha relative band from Fp1 and Fp2 electrodes to compute the frontal asymmetry which can be expressed as:

$$ Frontal\,Asymmetry = \log \frac{{\left( {fp1 - fp2} \right)}}{{\left( {fp1 + fp2} \right)}} $$
(7)

The calculation simply means that higher scores are indicative of approach behavior—a form of behaviour in which the person is moved towards, or in this case, more vigilant, and lower scores are indicative of avoidance motivation—a behavioral act that enables an individual to avoid a situation, or in this case, less vigilant [6].

Sensor Evaluation

Using data from the random dot in the 5 sessions in sensor evaluation (the sixth session was excluded from the data analysis, as the eye tracker could not detect the subjects’ eyes), we achieved the following accuracy and precision shown in Fig. 5.

Fig. 5
figure 5

Accuracy and precision achieved across sensor evaluation

Full size image

From the accuracy and precision of 6 different sessions from the Sensor Evaluation, the optimal condition at which the eye tracker works best is with a distance of 45 cm with 250 lx light intensity. We conducted the t-test on the accuracy and precision data and realised that the presence of chin rest does not affect both the accuracy and precision of the eye tracker significantly. Hence, to make the experiment more similar to a real-life simulation, we have decided to not use the chin rest. Therefore, we have used 45 cm distance, 250 lx light intensity and absence of chinrest in the vigilance testing to ensure that the eye tracker data we collect in the experiment is the most accurate and precise.

Vigilance Testing

The KSS was analysed and we picked out the subjects who became sleepy over time. Using these subjects’ data from the eye tracker, we achieved the following blink rates trend shown in Fig. 6a. Together with the timings when the subject pressed the spacebar, we obtained the reaction time of the subjects throughout the experiment shown in Fig. 6b. An increase in blink rate and reaction time shows a decline in vigilance level.

Fig. 6
figure 6

a Blink rate during vigilance testing. b Reaction time and frontal asymmetry of alpha relative

Full size image

From above graphs, we concluded that the increase in blink rate and reaction time proves that the subject’s vigilance level declined over time. The frontal asymmetry of alpha relative correlates with the reaction time. When the reaction time increases, the frontal asymmetry becomes more negative. As mentioned previously, the higher scores are indicative of approach behavior which shows a higher vigilance level and lower scores are indicative of avoidance motivation which shows a lower vigilance level. From Fig. 6b, it can be seen that the top row of graphs became more negative over time, confirming that the vigilance level has declined over time. Moreover, the raw β and θ activities in both the temporal lobes were analysed and observed to have decreased and increased respectively as β waves associated with alertness and θ waves with drowsiness [7].

5 Discussion

While the participant’s vigilance level was detected due to the symptoms of decline in vigilance level observed, we were unable to find out the exact characteristics of when one’s vigilance level is low. True to the limitation of using a low cost eye tracker to measure vigilance level as mentioned in the introduction, vigilance level cannot be assessed using exact values, but instead trends over time have to be observed. Moreover, we designed the vigilance testing experiment in which the repetitive visual stimuli should be able to induce mental fatigue and cause subjects’ vigilance level to decrease, as mentioned in our hypothesis. However, according to our KSS results, not all subjects became less alert as the experiment progresses.

To further improve analysis, we wish to have subjects of different genders and age to get the common regularity and the possibility to get values to differentiate high vigilance level from low vigilance level. It is more accurate to have more adults as our experiment subjects since we would be applying the results to security guards. Furthermore, other than blink rates, we can analyse more features from the eye tracker data (e.g. PERCLOS) to allow us to have a better evaluation on vigilance level.

With a good understanding of the trends and symptoms of decline in vigilance, our findings could be used to optimize devices or software interfaces in tracking one’s vigilance level. This would allow jobs that require high vigilance levels to maintain a high vigilance level as if any decline in vigilance level is detected, they would be alerted to take counteractions. However, since we were only able to obtain trends, developing an algorithm to evaluate instantly one’s vigilance level will be the next step to develop a software that can quantify a person’s vigilance level which can be used in the selection of security guards or alert the supervisor if on-duty guard’s vigilance becomes too low.

6 Conclusion

From the results of sensor evaluation, we found that a low cost eye tracker performs best with a distance of 45 cm, in a 250 lx light intensity place in the absence of chinrest. These results were useful in the experimental design of the vigilance testing. We were able to evaluate the symptoms of decline in vigilance level from vigilance testing which are: increase in blink rate, decrease in β and increase in θ activities in the both temporal lobes and increase in reaction time. We are able to conclude that a low cost eye tracker can ultimately assess vigilance level. Overall, our project seeks to objectively evaluate vigilance states to maintain desirable vigilance level in an automated approach. In the long term, our project can function as a feasible method to assess and maintain people’s vigilance level using a low cost eye tracker.