1 Introduction

Many accidents in the construction industry are the consequences of incorrect estimation of various risk aspects, such as environmental danger, individual skills, and workers’ mental and physical conditions. One of the critical reasons for the lack of safety awareness is the cognitive bias against dangers [1]. In recent years, virtual reality-based systems have been applied to construction safety education and training [2,3,4], which significantly improves the operation level and risk prevention awareness of workers in practice. However, according to a recent survey about construction safety accidents in the past five years on thousands of urban construction workers in Shanghai [5], 17.3% of the workers suffer from chronic health conditions due to overtime work. These workers are constantly exposed to noise, intense light, harmful materials, ineffective protections, excessive working, lack of mental health guidance, and lack of physical examination, which implies that their physical and mental health is in urgent need of improvement.

This paper presents a novel VR-based system for safety training and health assessment for construction workers. It employs a deep EEG neural network (EEG-net) and physiology data to assess physical and mental health conditions and competence of working in high-risk situations. While VR-based and task-orientated safety training augments an individual’s functioning, real-time processing of the individual’s EEG signals is achieved via a brain–computer interface (BCI) of electroencephalography (EEG). And other physiological data (such as blood pressure and heart rate) is analyzed as the reference of health condition. We also propose a novel deep EEG network structure by adding separable convolutional layers [6] and applying batch normalization and ELU activation functions. A clipping algorithm is applied to cut the training EEG, speed up the training of the EEG network, and improve test efficiency. Finally, a Kolmogorov–Smirnov test [7] on EEG data is adopted as an evaluation standard and a reliable reference for the health and safety assessment.

We conduct a large-scale cross-discipline study of EEG datasets. It is the first in construction safety training and assessment to apply VR with deep learning and physiology analysis. The system tests 117 workers, and the assessment results are validated by further medical examinations. The VR system allows construction workers to interact with simulated situations without risk while enabling user assessment and profiling for safety training and task allocations.

2 Related work

2.1 Virtual security training

Based on the characteristics of immersion, interactivity, and multi-perception of VR, virtual reality technology is widely used in film and television, game, and other industries [8]. VR is now becoming a well-known training tool. Saunders et al. [9] proposed a semi-immersive interactive virtual environment to help the operators to gain the knowledge and skills required to operate a lathe machine safely. Corelli et al. [10] evaluated the availability of three different VR systems in firefighter training. Clifford et al. [11] reported a research on radio communication training within a virtual training environment for aerial firefighting. Virtual simulation training has a good development in the field of security training. At present, there have been examples of safety and skills training using VR technology, and some advanced enterprises have begun to apply in production activities. For example, in 2017, China Railway Electrification Bureau used VR technology to simulate the whole process of manufacturing cold shrinkable cable terminals in urban rail transit engineering skills.

Due to the rapid development of virtual reality technology at current stage, many sites have introduced virtual safety training in construction. For example, Gong Hong [2] proposed to combine virtual reality technology with practical training to improve the current situation of construction safety training. Le et al. [3] proposed an online social virtual reality system framework to enable users to complete construction safety and health education through role playing, learning by communication, and social interaction. Shamsudin et al. [4] reported the application effect of virtual reality technology in construction safety training. In the early work, we put forward the virtual reality for training and fitness assessments for construction safety [12].

However, such mentioned common virtual safety training technology provided a more realistic VR experience through audio–visual stimulation. In order to enhance the sense of immersion, some scholars have studied the impact of tactile sensation on the VR experience. The literature [13] concentrated on the impact of pain and how pain affected perception and decision making in the VR environment. The author showed through experiments that additionally including electrical pain stimulation improved the sense of immersion and raised the felt importance of made decisions. The literature [14] studied the effect of full immersion virtual reality training on balance and knee function in total knee replacement patients. Experiments have proved that VR exercise may improve the recovery efficiency in the early stage after total knee replacement, which shows clinical value. In the construction field, the VR injury experience simulation system can build a highly simulated construction site environment, based on VR technology to restore the entire occurring process of some typical construction injuries. Based on the combination of visual, audio, and tactile stimuli, the trainee can experience the injuries that commonly occur on the construction site, such as falling, mechanical injury, electric injury, collapse injury, and object strike.

2.2 EEG

Physiology data such as heart rates and blood pressure have long been used to assess a person’s physical and mental health conditions. The rapid development of various assistive technology enables convenient and wireless acquisition of biological data [15]. Clinically, EEG signals are often collected for various disease diagnoses such as epilepsy and stroke [16,17,18]. There are standard clinical guidelines to collect and interpret EEG signals accurately. In research, brain–computer interface (BCI) [19] allows researchers in various disciplines to utilize EEG measurement and understand the mental states of participants. Lin et al. [20] recorded EEG signals with virtual driving to study the relationship between distraction and mental states in driving. Ozkan et al. [21] used EEG in a virtual reality environment and forecast subsequent driving actions. Kweon et al. [22] compared EEG signals in a virtual 3D environment with a typical 2D environment. Liu et al. [23] experimented virtual marine accidents and recorded EEG signals to evaluate crew members’ actions when executing commands. Vourvopoulos et al. [24] studied the effect of game experience on brain activity regulation.

However, the collected data came with significant noise or unwanted information [25]. As a result, accurate interpretation and analysis of datasets could only be achieved by robust and systematical learning and training [26].

More recently, researchers have developed machine learning methods to interpret EEG signals, taking advantage of feature classification algorithms to assist EEG analysis. Researchers decode EEG signals by K-nearest neighbor (KNN) [27], support vector machine (SVM) [28], and random forest [29], facilitating the effective diagnosis of depression, brain injury, and Alzheimer’s disease.

A study compares KNN, SVM, artificial neural network, and deep belt network (DBN) on analyzing EEG data [30]. The experimental results show that the performance of DBN is better than traditional shallow learning methods. Hosseinifard et al. [31] used machine learning and EEG signals’ linear characteristics to identify patients with depression, proving that machine learning methods could be applied to EEG signals and the diagnosis of depression. Janjarasjitt [32] used wavelet-based features and SVM to classify epileptic EEG on average, seizure, and interictal periods.

Convolutional neural networks (CNNs) learn representations of sub-patterns in a region of input, so they are good at tasks like image processing. Recurrent neural networks (RNNs), on the other hand, learn representations of sequences, and thus, they have shown good performance in natural language processing. Researchers have proposed many different deep neural network structures to study EEG datasets [33, 34].

3 System design and overview

As the VR training system’s primary target users are construction workers, some crucial requirements during the experiment should be taken into consideration: (1) The training time should be sufficient to ensure each participant finishes all training tasks or virtual sections. (2) The training content should be specific and straightforward because the training is progressed through interactions such as watching, a small range of navigations, and handle touches. (3) The VR training system should be vivid, and thus, when a virtual safety accident occurs, workers feel fear and shock as if they were in that situation, and their awareness can be enhanced.

Fig. 1
figure 1

Structure of virtual reality training system

Full size image
Fig. 2
figure 2

The architecture of our system

Full size image

The system generally consists of six scenarios that are most likely to happen and cause construction injuries, as shown in Fig. 1. During the training, trainees first learn the basic working rules in construction and get acquainted with various safety tools via instruction tutorials to evoke awareness. Trainees then go through the six accident scenarios and respond toward the injuries. The technology framework of our proposed VR system, as shown in Fig. 2, consists of three modules: a human–computer interface (HCI) input module, a data processing module, and a visualization module (including a convolutional neural network to deal with EEG signals, a clipping training algorithm, and physical condition evaluation metrics).

The input module collects participants’ EEG signals, blood pressures, and heart rates during virtual task sessions. The EEG convolutional neural network module processes the EEG data with an EEG-net in real time. The clipping training algorithm module clips large training examples into small segments. The physical condition evaluation module sets assessment metrics, i.e., the recommendation threshold and standard for abnormal EEG levels. The visualization module creates an immersive VR environment for training and displaying workers’ physical and emotional reactions during virtual training.

3.1 HCI input module

Users’ action data are collected by the HTC VIVE headgear in the input module. The headgear includes a VIVE pro head display, two controllers, a VIVE wireless kit, and other accessories. The VIVE wireless kit expands trainees’ scope of activity, enhances portability, and facilitates the wearing of EEG equipment.

Users wear Emotiv EPOC+ EEG sensors, including a signal acquisition electrodes, a head-mounted electrode cap, and other accessories. The EEG signals are collected from sixteen channels. Moreover, in this paper, EEG is sampled at a frequency of 128Hz, and the quality of sampled EEG signals has been verified [35]. Emotiv EPOC+ is a portable Bluetooth wireless transmission module. Next, an Omron electronic sphygmomanometer (model hem-8622) collects users’ blood pressure and heart rates and transmits these data via Bluetooth.

3.2 Visualization module

VR technology simulates potential hazardous accidents and leaves a lasting impression on workers. Through the training, workers learn to stick to safety regulations and avoid such accidents.

Finally, trainees are asked to complete an online test of safety knowledge. The scores are available immediately to find their weakness and further strengthen their safety awareness.

3.3 On-site experiment setups

We conducted our study at four construction sites of Shanghai Municipal Engineering Construction and Development Co., Ltd. At each site, we set up a 10-sqft room for safety training, which included the VR safety training and health assessment system.

Fig. 3
figure 3

A set of photographs of the research and development personnel in VR training

Full size image

Figure 3 shows a set of photographs of the research and development personnel in VR training. It is indeed cumbersome and uncomfortable for the subject to wear both devices simultaneously, but it is actually bearable because the VR training does not last a long time. Users first put on EPOC+ after inserts are properly moistened with saline solution, and the electrodes are mainly positioned on the side of the brain and the top of the head (Fig. 3a). Next, Vive headwear is worn, mainly positioned in front of the eyes (Fig. 3b). Finally, the Vive wireless kit is worn on the top of the head, without close contact with the scalp (Fig. 3c). Figure 3d shows a participant being trained with EPOC+ and Vive headwear.

Fig. 4
figure 4

Visualized UI of our system

Full size image

Due to the adjustable elasticity of Vive headwear and wireless kit, users can first wear Epoc+ and then HTC Vive, which enables electrodes to work properly. Figure 4 shows our system’s visualization UI, including real-time EEG, heart rate, blood pressures, and scenes shown in the user’s headset.

4 Methods

We propose a clipping training algorithm. In addition, batch normalization and ELU activate function are adopted to improve the EEG-net. The EEG-net detects the EEG of workers in the virtual training process in real time. Based on the sampled EEG data, the concept of abnormal rate of EEG is put forward. And the standard model of EEG health assessment is constructed. Next, the normal distribution characteristics of EEG are analyzed by K-S single sample test. At last, combined with the central limit theorem, the evaluation model of workers EEG data in this system is summarized.

4.1 EEG-net

We apply an improved EEG-net to process EEG signals. Its architecture is shown in Fig. 5 and consists of three convolutional layers, two max-pooling layers, and one classification layer.

Fig. 5
figure 5

Improved EEG-net architecture

Full size image

4.1.1 Network structure

The kernel size of the first convolutional layer is (1, 64). It convolves the data in the time domain with a unit of 2Hz. The second convolutional layer employs a kernel of size (16, 1). It convolves along channels and applies spatial filtering with the FBCSP algorithm [36], thus convolving in both time and space domains. Batch normalization and an ELU activation function are applied to the collected feature maps. In each training epoch, we randomly drop some data or set inputs to 0 to avoid overfitting and expanding the data size [37]. The third convolutional layer is separable [38]. It consists of a depth convolution and a pointwise convolution with sizes of (1, 16) and (1, 1), respectively, which greatly reduces the computational complexity.

Two max-pooling layers interleave with the second convolutional layer, the third (separable) convolutional layer, and the last classification layer. The two max-pooling layers both have a stride of 4 with batch normalization and an ELU activation function.

The last layer is the classification layer. It classifies each training sample into one of N classes by their features and a Softmax activation function, where N denotes the number of classes.

4.1.2 Dataset selection

We adopt two publicly available datasets in previous BCI research and transform them into 128 Hz images. The first dataset is the feedback error-related negativity (ERN) dataset [39] and contains 146 segments for training. It records the perturbation of ERN and EEG following erroneous or abnormal cases.

The second dataset is the P300 event-related potential (P300) dataset [40] and contains 197 segments for training. It records the neural responses to visual stimuli in EEG [41]. Abnormal EEG fluctuations after target stimuli are the labels for training the classifier.

Our EEG-net is a supervised network, and the dataset is labelled by 2 invited neurologists manually. And there have been some useful labels which can be trained on two publicly available datasets.

4.2 Dataset clipping

This part gives a explanation of how our EEG-net is trained by dataset input clipping twice, and then, the clipping algorithm is introduced, and a new loss function is defined.

4.2.1 Training data processing

A supervised classification network maps each training example to class scores.

$$\begin{aligned} f(X^{i};\theta ):\mathbb {R}^{E\cdot T}\rightarrow \mathbb {R}^{C} \end{aligned}$$
(1)

where \(X^i\) denotes the input; \(\theta \) denotes the parameters of the classifier; E denotes the EEG potential; T denotes the EEG stride; and C denotes the number of classes; \(\mathbb {R}^{C}\) is a vector where each element denotes the score of the corresponding class, based on the input \(X^i\). We use a Softmax function to transform the output into a conditional probability distribution. The probability of the input \(X^i\) belonging to class \(l_c\) is:

$$\begin{aligned} p(l_c \mid f(X^i;\theta )) = \frac{exp(f_c(X^i;\theta ))}{\sum ^c_1 exp(f(X^i;\theta ))} \end{aligned}$$
(2)

After obtaining the above conditional probability distribution, the loss function is defined as a negative log-likelihood between this distribution and the correct labels as lossA:

$$\begin{aligned} \begin{aligned} lossA&=\,loss(y^i,p(l_c \mid f(X^i;\theta ))) = \sum ^C_{c=1} \\&\,- log(p(l_c \mid f(X^i;\theta ))) \cdot \delta (y^i=l_c) \end{aligned} \end{aligned}$$
(3)

Then, it is minimized by backpropagating errors and updating the model parameters using batch stochastic gradient.

$$\begin{aligned} \theta ^ *= arg \, min_\theta \sum ^N_{i=1} loss(y^i,p(l_c \mid f(X^i;\theta ))) \end{aligned}$$
(4)

4.2.2 Dataset clipping algorithm

Each raw training example includes the entire EEG for the duration of safety training. We preprocess the EEG with a clipping method. It splits each EEG signal into several segments to reduce computational complexity and give a more comprehensive representation for different training sections. We set 10-s time intervals and assign a label for each segment generated by a 2-s sliding window. In addition, before each section, we insert two relatively strong pulses (one positive and one negative) that serve as the indicator of EEG and give a better presentation of each training section.

The clipping method not only reduces the size of input data but also increases the number of examples. As shown in Fig. 6, the whole input of EEG is clipped into several segments of 2 s. Then, every five consecutive 2-s segments are combined as a 10-s segment. Loss function penalizes the discrepancy of two adjacent 2-s segments’ labels and reconstructs correlations in a 10-s segment. By this clipping method, we have more short-time input units.

Fig. 6
figure 6

An example of clipping algorithm

Full size image

Then, we obtain a new series by extracting the raw input \(X^i \in R^{E \cdot T}\) based on \(T^{'}\), where E and T are the EEG potential and its time duration, respectively.

$$\begin{aligned} C^i = \{X^i_{1...E,t+T^{'}} \mid t \in 1...T-T^{'}\} \end{aligned}$$
(5)

All \(T-T^{'}\) segments serve as new training examples. Each of them is assigned a label.

Although the time efficiency is improved by clipping training examples with a 2-second sliding window, it does not retain the correlations across the entire duration. Thus, we add a penalization term in the loss function to penalize the discrepancy of two adjacent segments labels and reconstruct correlations. The new loss function is defined as lossB, i.e., lossA plus the cross-entropy of two adjacent segments:

$$\begin{aligned} \begin{aligned} lossB= lossA + \sum ^C_{c=1} - log(p_{f,c}(X^i_{t ... t+T^{'}}) \cdot p_{f,c}(X^i_{t+1 ... t+T^{'}+1}) \end{aligned} \end{aligned}$$
(6)

where the conditional probability distribution of the succeeding segment \(p_{f,c}(X^i_{t...t+T^{'}})\) is also taken into consideration.

4.3 Physical condition evaluation metric

Fig. 7
figure 7

Background of the training workers

Full size image

We recruit 117 construction workers to participate in our VR training. Among them, 106 are male and 11 are female. More statistics of the workers are given in Fig. 7. The system settings are available in Experiment 1 of Section 5.

To evaluate workers’ health conditions, we define an abnormal rate \(X_i\) for the \(i^{th}\) training section as:

$$\begin{aligned} X_i = N_i / M_i \end{aligned}$$
(7)

where \(N_i\) denotes the number of abnormal segments and \(M_i\) denotes the total number of segments. Figure 8 provides a snapshot of the outcome of the EEG network.

Fig. 8
figure 8

Computation result of the EEG-net

Full size image
Fig. 9
figure 9

Distributions of abnormal rates in six training sections

Full size image

To test the data’s statistical significance, we start from each section’s cumulative distribution function (CDF) as:

$$\begin{aligned} F_n(x) = \frac{1}{n} \sum ^n_{i=1} I_{[-\infty ,x]}(X_i) \end{aligned}$$
(8)

where \(I_{[-\infty ,x]}(X_i)\) is an indicator function and x is the systematic evaluation standard.

$$\begin{aligned} I_{[-\infty ,x]}(X_i) = \left\{ \begin{aligned} 1 , X_i > x \\ 0 , X_i \le x \end{aligned} \right. \end{aligned}$$
(9)

Assuming that the above CDF follows a Gaussian distribution F(x), we apply the Kolmogorov–Smirnov test on the data normality as:

$$\begin{aligned} D_n = sup_x \mid F_n(x) - F(x) \mid \end{aligned}$$
(10)

where \(sup_x\) is the upper limit of the CDF. If \(X_i\) follows a Gaussian distribution \(F(x), D_n\) approaches 0 as n approaches infinity.

Table 1 One sample of Kolmogorov–Smirnov test
Full size table

Figure 9 shows the distributions of abnormal rates (horizontal axis) and accident percentages (vertical axis) in six types of injuries. We then apply a K-S test on them and calculate the mean deviation, absolute value of the most extreme difference, and 2-tailed asymptotic significance of each section’s abnormal rate, as shown in Table 1 and Figure 10. There is no statistical significance at 5\(\%\) confidence since the p values are higher than 0.05. It indicates that \(F_n(x)\) follows a Gaussian distribution F(x) in each section.

According to data, rare accidents happen with a probability below 0.05, so we set it as a threshold. Finally, based on the overall abnormal rate for all sections and workers, we choose a threshold of 0.25 for the following indicator function to identify an abnormality in general.

$$\begin{aligned} I_{[-\infty ,x]}(X_t) = \left\{ \begin{aligned} 1 , X_t > 0.25 \\ 0 , X_t \le 0.25 \end{aligned} \right. \end{aligned}$$
(11)

For each section, given its abnormal rate, we set a specific threshold. Specifically, for electrical injuries and confined space, we set a threshold of 0.08. Because in these sections, only 5\(\%\) of all trainers show abnormal rate above 0.08.

$$\begin{aligned} I_{[-\infty ,x]}(X_1) = \left\{ \begin{aligned} 1 , X_1 > 0.08 \\ 0 , X_1 \le 0.08 \end{aligned} \right. \end{aligned}$$
(12)

For the other sections, we set a threshold of 0.30, as only 5\(\%\) of all trainers show abnormal rate above 0.3.

$$\begin{aligned} I_{[-\infty ,x]}(X_2) = \left\{ \begin{aligned} 1 , X_2 > 0.30 \\ 0 , X_2 \le 0.30 \end{aligned} \right. \end{aligned}$$
(13)

Blood pressure and heart rate have nothing to do with EEG-net. These data are collected and directly displayed on UI without further analysis. The medical standard of blood pressure and heart rate is clearly established. If the systolic blood pressure is above 140 mmhg or the diastolic blood pressure is above 90 mmhg, it is hypertension. If the individuals’ average blood pressure in the training process is above the threshold, the worker may be considered to be unhealthy. If the heart rate exceeds 100 beats per minute, the worker may also be recognized as unhealthy.

According to the above identification criteria, among the 117 workers, nine have abnormal rates in several sections. After going through thorough physical examinations, six of them are diagnosed with health issues inappropriate for work, one is diagnosed with epilepsy, three are diagnosed with hypertension, and the last two are diagnosed with other diseases. And the other three are re-assigned to other positions.

Based on the sample of 117 workers, the result may suggest that high levels of abnormal EEG show positive correlation with potential high-risk health. However, we cannot provide solid evidence or verification on such relationship. We will conduct further verification of such correlation.

Fig. 10
figure 10

Summary of abnormal rate interval of 117 workers

Full size image

5 Experimental results and analysis

This section presents three experiments’ results. The first experiment includes six virtual scenarios of serious injuries at construction sites for training. The second and third experiments compare the accuracy, training time, and average processing rate between our improved EEG-net and the vanilla EEG-net on the two datasets of ERN and P300.

The system runs on computers with a CPU of Intel (R) Core (TM) i9-9900K CPU@ 3.6GHz and 32GB RAM. And VR system stabilizes at a stable rate of 48FPS. We will spare no effort to enhance the hardware and optimize our VR system to get higher FPS in the future.

Fig. 11
figure 11

Six virtual scenarios of serious injuries at construction sites

Full size image
Fig. 12
figure 12

Comparisons of our improved EEG-net and the vanilla EEG-net on ERN

Full size image

5.1 Experiment 1: VR training modules

The experiencing module is the core module of the VR training system. We design six virtual scenarios of serious injuries at construction sites, as shown in Fig. 11.

  1. (1)

    Electrical injury (Fig. 11a): the workers are trained for safety awareness of using electricity and are required to connect a damaged plug to the distribution box. Trainees use the handle to pick up the damaged plug on the ground and plug it into the yellow circle on the distribution box, to intentionally cause electric shock.

  2. (2)

    Object impact (Fig. 11b): trainees are required to move to the outside of a construction building, and then, they are hit by falling objects like steel pipes. Trainees use the handle to touch the yellow circle located on the top of their head; afterward, they will see steel pipes falling from the top of building.

  3. (3)

    Mechanical injury (Fig. 11c): the trainees are required to walk into the operating radius of a working excavator, and then, they are injured by such machinery.

  4. (4)

    Foundation collapse (Fig. 11d): trainees are required to work on a foundation with cracks and seepage, and then, the foundation collapses, and the workers should escape. In this process, they are injured by falling stones to feel how dangerous such accidents are. The trainees touch the three yellow circles with handle, which in turn leads to cracks, water seepage, and foundation collapse.

  5. (5)

    Confined space (Fig. 11e): the construction workers are instructed to enter a deep well for operation, and afterward, they experience falling due to a lack of ventilation. Following the arrow, trainees move to the manhole cover. Next, they are required to touch the yellow circle with handle to descend along the manhole, with their sight gradually turning dark.

  6. (6)

    Falling (Fig. 11f): the trainees are required to work on a high construction platform without a safety belt and then experience falling.

5.2 Experiment 2: EEG-net performance evaluation

In this subsection, we evaluate our EEG-net with the feedback error-related negativity (ERN) dataset whose abnormality has been identified in previous BCI research. It records the ERN and EEG perturbations following erroneous or abnormal cases. All data are transformed into 128Hz images.

We compare our improved EEG-net against the vanilla EEG-net. The experiment runs on an NVIDIA GeForce RTX 2080 Ti GPU, and the models are implemented in TensorFlow with CUDA9 and cuDNN v7. We summarize the accuracy and training time of the vanilla EEG-net and our improved EEG-net with and without clipping.

Figure 12 shows the models’ performance on ERN. The blue rectangles under the circles in Fig. 12a denote the standard deviations. The results indicate that the EEG-net with clipping achieves similar accuracy in much less time.

Fig. 13
figure 13

Comparison of our improved EEG-nets and the vanilla EEG-net on P300

Full size image

5.3 Experiment 3: EEG-net performance evaluation on P300 dataset

We evaluate the same models on the P300 event-related potential (P300) dataset, with the same ERN experiment settings. We also summarize the average processing durations in the two experiments.

Figure 13 shows the results of P300. Similar to the ERN experiment, our EEG-net with clipping achieves higher accuracy in a shorter time. In addition, both of our EEG-nets run faster than the vanilla EEG-net.

6 Evaluation and quality analysis

6.1 Evaluation of VR training

To evaluate the proposed virtual safety training system, we consult six senior engineers for management of the construction firm and six neurologists with more than five years of clinical experience about their feedback of involvement and evaluation. Their detailed information is shown in Table 2.

Table 2 Background of the 12 questionnaire participants
Full size table

We create two questionnaires with six questions for the experts to assess our system subjectively. The questionnaire consists of six questions. Among them, three are common for both groups (marked in black in Fig. 14), while the other three are designed according to the responders’ expertise (marked in blue and orange in Fig. 14). The experts give each question scores, where 0 denotes extremely unsatisfactory or extremely ineffective, while 5 denotes the other extreme. Figure 13 shows the feedback summary on the nine questions, where blue, gray, and orange bars denote the highest, the lowest, and the average scores, respectively.

Fig. 14
figure 14

Summary of feedback on the questionnaire

Full size image

Our system runs a simple test during safety training on workers’ physical condition. The scores show that our system performs very well in simulation and training. The hardware comfort score is a little lower because of the limitation in hardware and its development, but this will be resolved soon. Four neurologists give positive comments which confirm our system’s effectiveness according to the identified abnormality in EEG.

6.2 Quality analysis

In order to understand how users feel about the virtual reality safety training experience, we invited 10 participants to take the virtual reality safety training courses once to twice per week in two weeks time. After each training course, we conducted a face-to-face interview with them, lasting 30 minutes. A total of 21 records were finally collected. Participants taking the virtual reality safety training conducted a wide range of injury experience activities, mainly including:

  1. (1)

    Share the immersive experience of electrical injuries

  2. (2)

    Share the immersive experience of object impact

  3. (3)

    Share the immersive experience of mechanical injury

  4. (4)

    Share the immersive experience of foundation collapse

  5. (5)

    Share the immersive experience of being in confined space

  6. (6)

    Share the immersive falling

Due to space limitations, we only show some noteworthy records in this section.

P6: “With the help of this virtual reality safety training system, I have experienced far more injuries that may occur during the construction process than in normal cases. This is impossible in real life. We cannot experience all these possible hazardous situations in the real world. Otherwise, those injuries will cause dramatic damage to our bodies.”

P6 described how this system helps people understand and experience the severe injuries occurring on the construction site that they have never known or experienced before. This method could enrich people’s risk knowledge and enhance safety awareness.

P3: “It was amazing to be able to feel electric shocks, falling, etc. in the virtual world. I really felt that there was indeed a sense of fear and tension when I received electric shocks or I was falling. It was more interesting that when the researchers asked me to take one more trial, I would rather escape to the safe place, avoiding experiencing such accidents or injuries again.”

Fear is the most significant feeling of trainees, which can affect their actions in the VR training. Participants suppose such way of experiencing injuries on construction sites in the virtual world paves another way for contemporary safety training.

P8: “I think it was a good way to interact during the VR experience. Unlike just watching the video, you are able to decide your actions in the virtual world by yourself. I think I was truly a member of virtual world; I could truly understand how the environment was like and I was affected by this environment to determine my actions in the next second.” Interestingly, the use of VR interaction to simulate the injury experience in the construction environment increases the sense of immersion and reality. When participants realize their fear, it will affect their decision in the virtual world.

P2: “This system is highly simulated. It presented many scenes of injuries that occur during construction process. It evoked my awareness of dangers hiding in the construction working.”

As all participants figure out, the significance of our VR safety training is providing a realistic construction scenario for trainees. It is the experience in the virtual world that enables trainees to understand the dangers in construction process and to improve their safety awareness.

With the above feedback, we can infer that our system increases the simulation of damage experience in the construction environment through VR and improves the extent of immersion and reality. This method is proved to enrich people’s risk knowledge and enhance their safety awareness. Even though trainees may have never experienced such injuries before, they could feel the fear and know how dangerous it is to work in the construction environment with the help of our system.

7 Conclusion

This paper proposes a novel VR-based training and health assessment system based on an improved EEG-net for construction safety. An EEG-net with a clipping training algorithm, batch normalization, and ELU activation function is introduced to improve assessment accuracy and time efficiency. We select the EEG abnormality threshold via a K-S test on workers’ EEG data during VR training. Finally, 12 experts give a positive assessment, which proves that our system could effectively reduce construction safety accidents.

In the future, we will improve the realness of virtual scenes and interactions and build a more professional VR training and health assessment system for construction safety.