Keywords

1 Introduction

Among Japanese industries, the construction sector has the highest number of fatalities due to work-related accidents, with many attributed to unsafe worker behaviors [1]. Here, “unsafe behaviors” refers to actions where workers, either by neglecting established safe-work practices and safety rules or due to conditions such as ill health or momentary lapses in attention, engage in activities that compromise safety [2]. On-site hazard prediction training effectively prevents unsafe behaviors. However, the application of the so-called “36 Agreement” to the construction industry in April 2024 will create demands for completing projects within more limited labor hours and stricter deadlines [3], reducing the time available for on-site safety education. Therefore, it is crucial to conduct hazard prediction training to address unsafe behaviors before construction commences.

Japan’s Occupational Safety and Health Law mandates the provision of safety and health education to workers at the start of their employment or when entering a new site. This education aims to prevent work-related accidents by imparting safety and health knowledge related to the tasks workers will engage in [4]. Prior to newcomer orientation training conducted by the main contractor, subcontractors provide their workers with “outbound training” for safety and health education [5]. While outbound training can be conducted before construction starts, challenges such as the difficulty for safety managers and workers to grasp actual conditions and work environments at the construction site make site-specific hazards hard to recognize. Furthermore, preventing unsafe behaviors requires safety managers to point out errors in workers’ hazard predictions so that workers can recognize judgmental errors. However, existing outbound training methods primarily focus on safety managers reminding workers of safety and health management knowledge through verbal or written instructions, making it difficult to provide opportunities for workers to conduct hazard predictions related to their own actions.

In this study, we aimed to enable precise hazard prediction training by safety managers, tailored to workers’ hazard prediction capabilities and within the timeframe of outbound training. To that end, we developed a virtual reality (VR) system that facilitates hazard prediction training for unsafe behaviors during outbound training. Our objective is to demonstrate the proposed system’s effectiveness in addressing challenges associated with outbound training by clarifying the following points:

  1. 1.

    Safety managers can point out errors in workers’ incorrect hazard predictions regarding their own work activities.

  2. 2.

    Safety managers’ instructions affect workers’ activities.

Safety science focuses on scientific methods and theories to prevent accidents and disasters and to manage risk. This field studies how interactions between human behavior, organizational operations, and technical systems affect safety. Recent trends in safety science for the construction industry include the following areas [6]:

  1. 1.

    Adaptable safety climate and culture models incorporating different sites, project complexity levels, or national contexts.

  2. 2.

    Expanding established prototypes for broader application of information technology in the construction community through more testing and case studies.

  3. 3.

    Continued research on subgroup factors related to workers’ safety awareness and behavioral cognition models.

  4. 4.

    Integrating artificial intelligence and smart properties into safety program management.

  5. 5.

    Developing and applying information technology to enhance communication and coordination regarding safety between management and workers.

  6. 6.

    Assessing user acceptance and industry readiness for applying various information technologies in construction safety management.

This study aims to foster safety climates and cultures within construction industry organizations (Area 1), improve workers’ safety awareness and behaviors (Area 3), and develop and apply VR technologies that facilitate safety communication between managers and workers (Area 5).

2 Related Works

2.1 Research on Hazard Classification

Considering all possible hazards, Mihic et al. [7] proposed three categories for hazard classification to realize a more accurate hazard identification process. The first category, “self-induced hazards,” is the most easily recognizable danger type in this study, as it results from the workers’ activities and affects the workers themselves. The second category, “peer-induced hazards,” arises not from the construction workers themselves but from peers working on the same or different building elements. Other workers may pose risks not only to themselves but to all other workers present in a hazardous activity area. The final category, “global hazards,” is a special type of hazard with a very large impact range, making it impractical to identify the extent of impact. Instead, the entire construction site is considered a hazard zone. These hazards affect all construction workers and other personnel on the construction site at the time of occurrence. They are typically characterized by their high severity but low occurrence probability.

According to Khosravi et al. [8], the causes of unsafe behavior and accidents at construction sites are multifaceted and commonly related to social, organizational, project management, supervision, contractor, site condition, workgroup, or individual characteristics. They emphasize the importance of distal factors such as society, the organization, and project management, which can reduce the likelihood of hazardous behaviors and accidents more than proximal factors such as site conditions and individual characteristics.

In this study, we aim to support active training by enabling construction site workers to tag their training actions with potential hazards for comparison with safety managers’ feedback on hazard predictions for each hazard identified using the unsafe behavior tagging feature.

2.2 Advantages of Using VR in Construction Safety Training

Man et al. [9] noted that while VR is gaining attention in construction safety training, there has been no concrete evidence of its effectiveness. A meta-analysis of VR application research in computer science and technology education over the past decade showed its superiority over traditional construction training. As they mention, the benefits of using VR include the engaging nature of VR training, which, unlike textbook learning, simulates being on an actual site. VR training can increase employees’ safety motivation, improve construction techniques, and increase work speed. Through immersion and presence, VR allows trainees to experience more realistic training.

According to Babalola et al. [10], VR-based safety education provides diverse benefits. VR allows visualizations of construction site conditions, making it easier to identify hazards in advance, and simulating actual construction sites can enhance workers’ hazard identification capabilities. They also propose methods for using augmented reality (AR) systems to visually identify on-site hazards. These digital tools simulate actual site conditions more faithfully than 2D drawings and are effective in hazard management. Combining mobile-device-based VR and AR for safety education is also effective and favored by students and construction professionals.

Regarding considerations of new information technologies such as VR and mixed reality (MR) for improving safety management education and training, Yang et al. [11] focused on the lack of pedagogical guidelines for designing and developing learning content, aiming to address that issue by leveraging the Authentic Learning framework. Although VR and MR simulations could enhance participants’ motivation to learn, they did not improve knowledge or teaching effectiveness, likely due to the framework’s complexity. They suggest using 2D and 3D media in conjunction with systems that facilitate communication for more efficient learning.

Wolf et al. [12] suggested that while approaches such as verbal explanations, documentation, and testing exist for safety education and training in construction, they insufficiently promote learning. Moreover, modern learners prefer personalized feedback, necessitating the maximization of data provided during training. Learning environments utilizing VR can create safe education settings without risk of physical harm, providing both objective data for improvement and feedback on hazard recognition. Doing so motivates trainers and enhances learning by developing a wide range of training content, from hazard awareness to hazard recognition training.

From these insights, we aimed to develop a practical and experiential VR education support system that can realize precise hazard prediction training by safety managers, tailored to the hazard prediction capabilities of workers and able to be completed within outbound training timeframes.

3 System Design

3.1 Training Process

We propose a training process that combines construction task training with hazard prediction training. Table 1 outlines the training process. Workers initially perform construction tasks based on their own judgment without relying on prior instructions from safety managers. They then reflect on their actions and make hazard predictions regarding their behavior. By comparing their own judgment with that of safety managers, workers can recognize errors in their decision-making. When mistakes are identified, workers think about what they should be cautious of and verbalize these points.

Table 1. Training process by workers and safety managers
Full size table

3.2 System Design

Figure 1 illustrates the system configuration. In the “Model Registration” process, safety managers register the construction site model, unsafe behavior tags related to the construction site, and a digital twin as a model for the task. Unsafe behavior tags indicate the risk of unsafe behavior, assessed and assigned by safety managers or the workers themselves in relation to actual actions. Section 4 presents specific examples of unsafe behavior tags. In the “Model Confirmation” process, workers review the construction site model and the digital twin that models the task. During the “Task Rehearsal” process, workers mimic the work procedures verified in the “Model Confirmation” process within a virtual space and record tasks as a digital twin. In the “Tagging” process, both safety managers and workers register unsafe behavior tags for the worker’s digital twin. The “Review” process involves comparing unsafe behavior tags attached to the worker’s digital twin between the workers and safety managers, helping to identify commonalities and differences and complementing perspectives on evaluating unsafe behaviors. Figure 2 shows the sequence diagram. The “Model Registration,” “Model Confirmation,” “Work Rehearsal,” “Tagging,” and “Review” processes are executed as sequential steps.

Fig. 1.
figure 1

System configuration.

Full size image
Fig. 2.
figure 2

Sequence diagram.

Full size image

Figure 3 shows a use case diagram. The system comprises “Construction Site Model Registration and Viewing,” “Worker Digital Twin Registration and Viewing,” and “Unsafe Behavior Tag Registration and Viewing” functions. The “Construction Site Model Registration and Viewing” function is used in all processes. The “Worker Digital Twin Registration and Viewing” function is employed during the “Model Registration” process to register a model that records the worker’s tasks during the “Work Rehearsal” process and to reflect on tasks during the “Tagging” and “Perspective Sharing” processes. The “Unsafe Behavior Tag Registration and Viewing” function is used during the “Tagging” and “Perspective Sharing” processes.

Fig. 3.
figure 3

Use case diagram.

Full size image

4 System Implementation

4.1 Function Screens

Based on the outlined system design, we used the Unity game engine to develop a VR system that facilitates the training process within outbound training, employing virtual environments that simulate construction sites to enable hazard prediction training for unsafe behaviors while considering specific site hazards. The implementation details for each system function (Fig. 3) are as follows:

  1. 1.

    Construction Site Model Registration and Viewing function

Building information modeling (BIM) recreates 3D models of actual buildings, improving construction practices and serving as a standard model for construction drawings [13]. By allowing BIM of construction sites to be registered as VR objects, the system enables training within a virtual space that mimics real-world sites. We registered models imported into Autodesk Revit [14] and converted to FBX format as inputs. Figure 4 shows the unsafe behavior tags defined in this research, namely “Fall,” “Trip,” and “Crash”, which are among the top factors in construction industry occupational accidents. Unsafe behavior tags can be added or deleted, allowing safety managers to introduce new tags while considering construction site characteristics.

Fig. 4.
figure 4

Unsafe behavior tags (blue: safety manager; red: worker).

Full size image
  1. 3.

    Worker Digital Twin Registration and Viewing function

This function records tasks within the virtual space as time-series data. Tasks can later be visualized as avatar actions to review the work performed. This function is utilized when workers review construction procedures preregistered by safety managers and when workers and safety managers assess the risks associated with a worker’s actions. Figure 5 displays the worker digital twin registration function, where pressing the “Rehearsal Start” button begins task recording, and pressing the “Rehearsal Finish” button stops it. Figure 6 shows the Worker Digital Twin Viewing function, where worker digital twins are recorded as CSV files, allowing for the reproduction of specific digital twins. After loading a digital twin, pressing the play button replays the worker’s behavior as an avatar in the virtual space (Fig. 7).

Fig. 5.
figure 5

Worker Digital Twin Registration function.

Full size image
Fig. 6.
figure 6

Worker Digital Twin Viewing function.

Full size image
Fig. 7.
figure 7

Worker avatar.

Full size image
  1. 4.

    Unsafe Behavior Tag Registration and Viewing function

Tags indicating the risk of unsafe behavior can be assigned to avatars replicating workers’ construction actions at times and locations deemed risky. The unsafe behavior tags illustrated in Fig. 4 can be registered. Workers and safety managers use the Worker Digital Twin Viewing function to review actions and assign unsafe behavior tags at relevant locations and times. Figure 8 shows the process of using the registration function to register unsafe behavior tags.

Fig. 8.
figure 8

Registering an unsafe behavior tag.

Full size image

Figure 9 shows how workers and safety managers use the Unsafe Behavior Tag Viewing function to compare the unsafe behavior tags they registered. This comparison aims to help workers recognize their own judgment errors in hazard prediction and serves as an opportunity for safety managers to identify new unsafe behaviors from the worker’s perspective.

Fig. 9.
figure 9

Comparing unsafe behavior tags.

Full size image

4.2 Hardware

The proposed system utilizes the HTC VIVE Pro Eye [15] as the VR device. To capture more detailed hand and finger movements, we combine the VR base station with a Hi5 VR Glove [16] (Fig. 10).

Fig. 10.
figure 10

Data glove.

Full size image

5 Field Experiments

5.1 Objective

We conducted a field experiment with the objective of realizing a practical application to ascertain whether VR enables workers and safety managers to remotely recognize site-specific hazards and whether sharing perspectives through training could lead to more accurate safety instructions tailored to the understanding level of construction site workers.

5.2 Method

We implemented the system with the cooperation of a construction company based in Kagawa Prefecture, Japan. A training stage for a building planned for construction was prepared for one worker affiliated with the construction company, and actual outbound training was conducted twice. Two individuals from the safety and sales departments also used the system, and we conducted interviews regarding the system’s advantages, areas for improvement, ease of instruction, and changes in worker behavior.

5.3 Procedure

The targeted work processes were side and ceiling paneling tasks for interior finishing work. Figure 11 shows the side paneling, and Fig. 12 shows the ceiling paneling. The worker’s task was to transport boards from a storage area and align them at designated positions. For the ceiling paneling task, the system showed “Get On” and “Get Off” buttons when the worker approached a portable work platform, allowing the worker to move between the platform and the ground by pressing these buttons.

Fig. 11.
figure 11

Side paneling.

Full size image
Fig. 12.
figure 12

Ceiling paneling.

Full size image

The sequence of the practical application is as follows:

  1. 1.

    Preparation

We used the Construction Site Model Registration function to recreate the construction site in the virtual space. Safety managers generally use the Worker Digital Twin Creation function to register work examples as data, but in this study, the authors operated both functions.

  1. 2.

    Work procedure confirmation

Workers immerse themselves in the virtual space and view the work examples pre-registered using the Worker Digital Twin Viewing function.

  1. 3.

    Task practice

Workers perform task practice based on the examples. During this time, the Worker Digital Twin Registration function records their actions. Figure 13 shows a worker engaged in task practice.

  1. 4.

    Hazard prediction training

Using the Worker Digital Twin Viewing and Unsafe Behavior Tag Registration functions, both safety managers and workers perform hazard prediction for the worker’s construction tasks and assign unsafe behavior tags accordingly.

  1. 5.

    Confirmation of correctness and errors

Using the Unsafe Behavior Tag Viewing function, workers compare the tags assigned by safety managers and those assigned by themselves to identify commonalities and differences.

Fig. 13.
figure 13

Worker engaged in task practice.

Full size image

5.4 Results and Discussion

In the first training session, interviews with the safety manager revealed that the building structure and operations were close to reality. The manager particularly praised the practical training enabled by the Construction Site Model Registration and Viewing Function, stating that it allowed easier instructions than verbal explanations. However, feedback also highlighted difficulties in grabbing virtual objects like boards and issues with wall contact, indicating a need for improved operability. In the second training session, the manager positively noted that the worker could perform risk-checking actions like pointing, which were not possible in the first session, and that the Unsafe Behavior Tagging Function facilitated discussions.

This study aimed to demonstrate the effectiveness of this system in outbound training by clarifying the following points:

  1. 1.

    Safety managers can point out errors in workers’ incorrect hazard predictions regarding their own work activities.

  2. 2.

    Safety managers’ instructions affect workers’ activities.

From the safety manager’s perspective, visualizing workers’ reflections and remarks regarding unsafe behaviors enhanced their understanding of workers’ comprehension level. However, they noted a need for improvements in system operability, realism, UI enhancements, and addressing language barriers, particularly noting the complexity of safety manager operations. These points highlight the need for further system development and evaluations.

6 Summary

We developed a VR system to support training processes that integrate construction practice and safety training, aiming to enable safety managers to provide appropriate guidance while understanding workers’ comprehension levels. The Unsafe Behavior Tag Registration and Viewing function allowed safety managers to grasp worker understanding of hazard prediction for unsafe behaviors, facilitating instruction. We also confirmed changes in worker activities. The Construction Site Model Registration and Viewing function improved the ease of safety instruction. However, there were opinions regarding the need for interface improvements, such as unnatural object handling. The proposed system targeted training for a series of work tasks from carrying boards for siding and ceiling work to aligning them at the installation position, but it did not replicate actions such as climbing on and off portable work platforms or board weights. Climbing actions and board weights are strongly related to risks such as falls, trips, and collisions, and incorporating these factors into the training process is an outstanding issue. One solution could be an MR approach to allow the use of actual boards and portable work platforms during training. However, outbound training situations pose constraints such as limited space and tool availability, so training content may require a hybrid training process that seamlessly switches between VR and MR. In the future, we will focus on these points, advancing the development and implementation of the proposed system and aiming to establish effective safety education methods.