Keywords

1 Introduction

In the construction industry, Building Information Modeling (BIM) is a catalyst for a broad digital transition, with the objective of best responding to the major challenges of the sector. Each construction project is unique, and the construction process evolves in a constantly changing environment. Construction planning itself becomes a complex activity that is critical to the good management of a project. This planning is carried out under different methods and generally uses several tools.

In the BIM approach, 4D models are used to simulate the construction process, by a combination of a 3D model and a construction schedule. Using specific software, the objective is to generate a virtual model that can simulate construction activities. This type of simulation has become a communication tool very much appreciated by general contractor and execution teams, especially because it allows all parties to have a visual perspective of the evolution of the site as a way to better understand the extent of the work to be done [1]. However, the collaborative use of 4D simulations remains under-exploited by project teams and is often reduced by the capacity of the tools being used, compounding the difficulties faced by teams in their efforts to interact collectively with 4D simulations [2]. Recent research by Boton [2] and Sampaio [3] has suggested the use of virtual reality to optimize the use of 4D simulations. Known for its immersive and interactive nature, this booming technology in the construction industry opens up new dimensions for exploring space and other perspectives of group work. Indeed, the immersion and the feeling of presence that virtual reality provides can be of great value as it allows users to explore and inspect all the aspects of a construction site model, as a team in a virtual dynamic is brought closer to reality. In addition, interactive and intuitive functioning in a collaborative framework makes it possible to make the decision-making process more efficient thanks to the exchanges of ideas that this new work environment offers, with the objective of considering the most fluid sequencing for construction work.

Although the advantages of using virtual reality to optimize the planning process and address the limitations of current simulation tools are foreseeable [4], the integration of 4D models in virtual reality environments remains technologically difficult, given the rigidity of existing software and the exchange formats lack of adaptation to the libraries of information conveyed by 4D models [5]. On the other hand, we note that the use of OpenBIM standards (IFC-BCF) has proven its effectiveness in supporting the metadata of 3D models and in supporting a data exchange consisting of virtual reality support software, specifically for game engines [6]. From this perspective, introducing 4D models in virtual reality environments using OpenBIM standards is a promising option.

The objective of this research is to explore this opportunity as a way to propose a new method to improve the integration of 4D simulations in virtual reality environments. This article is structured in five sections. After the first section has introduced the problem, a literature review is proposed in section two. The third section presents the research methodology in the context of Design Science. In the fourth section, a prototype to foster the integration of 4D simulations in Virtual Reality environments is presented. The fifth section concludes the paper.

2 Related Work

The literature review focuses first on the use of 4D BIM models, to define their different aspects and discover recent advances. This section then explores virtual reality and the known practices around this technology in the construction industry. Finally, with the objective of linking the two themes of BIM 4D and virtual reality, recent developments are presented in order to explain the limits and issues that hinder this link. This section ends with a summary of the potential needs and objectives justifying the research approach.

2.1 4D BIM Models in Construction

The planning of construction activities is one of the most important processes in construction management; this process includes the definition of activities and the estimation and programming of resources by identifying the interrelationships between the different tasks of a project [7, 8]. Construction schedules are prepared based on quantities, available resources and other constraints related to project management. The construction activities are then organized according to a work breakdown structure (WBS) to represent a hierarchical breakdown of the construction work to be carried out by the teams in order to achieve the projects objectives [9]. The arrival of BIM has transformed planning and project management practices. By moving from a traditional 2D representation, today's construction schedules can be completed as 4D BIM models [10]. A 4D BIM model is produced by linking the elements of a threedimensional model to the activities of a construction schedule.

The added values of using 4D simulations lie in how they ease the process of communicating ideas and support collaborative work in the context of project reviews and on-site meetings [11]. This technology can also support various practical uses, including studies and analyses of constructability, the detection of spatio-temporal interference, simulation and management of logistics and resources, the planning of excavations, and on-site health and safety management [2]. Aware of the need for collaboration, Boton [2] found that 4D simulation is very useful for carrying out constructability studies. This practice brings actors together very early in a project, in order to analyze and evaluate several construction scenarios and sequences, based on their experience, in order to choose the best construction alternative and validate the feasibility of the schedule. Going beyond other modes of representation such as diagrams and two-dimensional visualization supports, 4D simulation considers the spatial aspect of the construction and becomes a powerful tool for analyzing and anticipating dangerous situations on site, as well as organizing and promoting health and safety on site [12].

The adoption of 4D simulations faces several obstacles, and the technological tools must adapt to the real needs of practitioners. Studies show that current 4D simulation practices are still too limited to support all the suggested options and meet the real needs of the construction industry [13]. Among the technological limitations of 4D simulation tools, Mazars and Francis [14] evoke the problems related to the interoperability of data and the incompatibility between 3D BIM software and 4D simulation. This gap leads to the need for repetitive and redundant processes in order to perform modifications on the model during spatio-temporal conflict detection, among other actions.

With the aim of automating and optimizing the production of construction schedules, Mazars and Francis [14] propose a new method of chronographic modeling of 4D simulations based on the interoperability between the 3D modeling software Revit and the 4D simulation in Navisworks. This interoperability can be exploited to integrate an iterative process to edit data in both software applications as well as to improve the activity of spatio-temporal conflict detection. In the same perspective, Elghaish and Abrishami [18] present an innovative solution in Navisworks using a genetic multicriteria optimization algorithm to automate the production of 4D models and improve the process of selecting construction alternatives. Indeed, to a large extent these works contribute to the integration of a unique and dynamic platform to optimize the production of 4D models. Hakkarainen et al. [19] propose an Augmented Reality-based integration system to consult and compare in real time the progress of the work on a construction site with the data related to its 4D simulation. Ratajczak et al. [20] developed an AR application in the form of a site manager connected to a database that integrates a management system based on the location-based management system (LBMS) using the coding of a location breakdown structure (LBS).

As stated by Boton [2], 4D simulations with today's tools are not able to meet the challenge of adequately representing the collaboration between stakeholders, especially in terms of visualization and interaction. As a way to address this lacuna, Boton evokes the concept of integrating virtual reality technology to support collaboration during constructability analysis meetings.

2.2 Virtual Reality (VR) Systems in the Construction Industry

The use of virtual reality in the construction industry is not new. In 1996, Bouchlaghem and Liyanage [21] were already exploring the potential use of this technology in design processes to provide layout ideas, to approve the ergonomics of future living spaces in relation to several factors, and to assess the risks associated with fires. The use of virtual reality can also be applied in education: to help students understand the essence of different architectural concepts, to conduct more in-depth architectural studies, and to learn how to solve problems with greater openness to new ideas and innovation [22].

Li et al. [23] strongly recommend virtual reality for health and safety training. Studies and tests have shown that through the VR medium, users are more motivated, attentive and remain focused on the subjects in immersive environments. Virtual reality provides an effective visual support that helps to better understand and identify complex engineering problems and thus improve the management of the risks and uncertainties of a future site. With the objective of integrating virtual reality into BIM processes and to support emerging practices, several large software and IT content publishing companies such as Autodesk, Trimble and Unity have embarked on the development of VR solutions and plug-ins to create links with the BIM models.

The work of Huang et al. [24] presents a comparative review of the most popular solutions in the construction industry, based on the features and capabilities they provide to users in an immersive experience, including navigation techniques, annotations, interaction-objects and interaction-data, as well as some criteria related to the communication and collaboration of users. Several of these plug-ins and game engines were tested during this research, using the Oculus Quest and HP Reverb headsets to closely assess the BIM models’ integration capabilities.

Overall, the integration of BIM models and data is done according to two methods, either through a transfer flow using exchange formats, or by using an API connection from BIM software (which is the case with most VR plugins). However, we can see that the workflow offered by most of these solutions still does not allow a two-way connection between the VR environment and the BIM software to be integrated so as to synchronize changes in both directions and thereby support a complete process. The exception appears to be the UnityReflect extension, which despite being still in development, is likely to support a two-way data connection from the VR environment to Revit and Navisworks.

2.3 Linking 4D Simulation and Virtual Reality

Some research works [2, 3, 25] have studied the difficult question of the integration of 4D simulations in virtual reality environments. Yerrapathruni [26] conducted multiple experiments to test the application of 4D models in an immersive virtual environment (IVE) and determined that students quickly managed to acquire the experience and instincts related to choosing construction methods, activity sequencing logic, site planning and logistics. The IVE has been a very promising tool, helping practitioners identify and resolve critical issues related to job planning in order to collaboratively develop and generate more realistic and consistent installation sequences. According to Sampaio [3], the combination of 4D models with VR technology makes it possible to explore a construction site virtually, on a real-life scale, to inspect each corner of a model and determine the quantities of materials relating to each construction task. Virtual reality is an effective medium for improving the perception of practitioners and to foster the level of collaboration required to support constructability analysis meetings. The intuitive interaction on the 4D simulation could allow users to develop a much better understanding of the problems and conflicts of a projects schedule in order to generate more realistic construction scenarios [2].

Two approaches are generally used to connect 4D simulations and VR environments. The first approach uses a pipeline connection with BIM software APIs and VR support software in order to visualize and manipulate the 4D model in an immersive environment. Kuncham [27] proposes an application that uses the APIs (.NET) of the Navisworks software to project the 4D simulation in a CAVE system. However, this application is limited by the capacity of Navisworks’ APIs; it does not accept the integration of new interactions to perform advanced manipulations on 4D simulation in the virtual reality environment, and so it is mainly used just for visualization purposes. The second approach is better known among developers and consists of integrating BIM data into game engines. We highlight the work of Bourlon and Boton [5], who propose an application for integrating 4D simulations in the virtual environment of the Unity3D game engine to be experimented with the Valyz immersive system. The workflow uses the FBX format to manage the 3D model separately from the construction schedule, which is attached from the Excel software in .CSV format to be integrated and then linked into the Unity3D game engine. The application thus makes it possible to link the temporal data to the elements of the 3D model. In immersion, we can interact with the 4D simulation and visualize the different activities of the calendar. However, the workflow adopted during the preparation and integration of the data remains tedious and does not allow the metadata of the 3D model to be preserved. In addition, the quality of the graphics rendering is not optimal and needs improvement.

Each of the two approaches has their advantages and disadvantages. Nevertheless, the use of game engines as a support for the virtual reality environment is an unlimited resource in terms of the development of interactions and connections, and the quality of the graphics is much more accomplished than the other software and systems. Moreover, in order to capitalize on the value of these experiments, ensure the conditions for interaction and extend collaboration in the practices, it is necessary to use an interoperable system as described in step 4 of the framework proposed by Boton [2]. However, to develop such a mechanism, the incompatibility of the software and exchange formats remains a formidable challenge.

3 Research Methodology

This research is anchored in the field of Design Science insofar as the objective is to propose a method for integrating 4D simulations in virtual reality environments and to validate it through the development of a prototype. Inspired by work in this field [28, 29], this research approach is organized around the seven guidelines proposed by Hevner and Chatterjee [30] to build and evaluate the artefact: to relate to the design as an artefact, emphasize the relevance of the problem, evaluate the method, assess its contribution to research, ensure research rigor, design as a research process and clearly communicate the research.

Considering that design is both a process and a product, March and Smith [29] identified two complementary and interdependent processes in design science: the construction and evaluation of an artefact. Among other things, an artefact is defined for a particular context, and when that context changes, the artefact can become obsolete. With the aim of bringing the artefact to life, the research approach is based on an evolutionary and incremental process. This process is evolutionary, as it considers the evolution of potential needs and specifications to improve and readjust the prototype to new features, and it is incremental, because it can be iterated multiple times at defined locations, resulting in a new prototype or a working product at the end of each iteration, thereby promoting the continuous improvement of the artefact.

The literature review made it possible to identify and define the research question, and to define and characterize the key concepts of the research. Next, we compared the solutions and evaluated the existing practices, which allowed us to refine the research question and reposition it accordingly. This step involved the identification of the potential needs and target goals of the artefact, and also allowed us to define the criteria for evaluating the proposed artefact. The third step then prioritizes and specifies a set of characteristics and functions elementary to the construction and evaluation of the artefact. An artefact in the form of a method is proposed to integrate 4D simulations in a virtual reality environment following an OpenBIM approach based on IFC in step four. Finally, with the objective of evaluating the proposed method, a prototype is developed in the last step using a specific ecosystem of software and tools, which is then validated according to the criteria identified in the second step.

The objective is to develop a complete method suitable for existing systems, which meets the need to optimize the integration and handling of 4D simulations in VR environments. The method is designed to respond to the work process explored by Boton [2], and is carried out with particular attention to recent developments in BIM-VR, in particular the work of Nandavar et al. [6]. The workflow presented by the method covers three work environments identified as key concepts (Pre-VR, VR and Post-VR) and triggers an iterative integration mechanism structured over six steps. To divide the main issue into several questions, the three concepts are represented by work environments in the rest of the document. This subdivision strategy makes it possible to delineate the systems in order to easily materialize and characterize the design, to bring together the relevant elements and to identify the critical aspects of each work environment to advocate the best possible connection between them.

In addition to the virtual reality (VR) environment that remains at the heart of this project, a Pre-VR environment is represented as a workspace for the preparation of the 4D simulation. Moreover, to integrate a BIM philosophy and incorporate part 4 of the “feedback” mentioned in the framework proposed by Boton [2], it is essential to introduce the concept of a Post-VR environment. This Post-VR environment refers to the definitions of Common Data Environments (CDE) presented in the literature review and requires a complete process.

In order to facilitate its applicability and generalizability, it was necessary to adopt a certain rigor in order to standardize the process and to formalize all the activities. The activities are identified as the critical elements of each step; they are characterized and then analyzed by input and output data with the aim of highlighting the prerequisites and the target objectives to be achieved through the various developments. Subsequently, a set of rules and conditions are generated to support good practices around each activity.

4 Development of a Prototype to Foster the Integration of 4D Simulations in Virtual Reality Environments

4.1 The Prototype

The power and efficiency of this prototype are based primarily on how it simplifies the manufacture and assembly of a WBS, using a new communication strategy that exploits the functionalities of the Tridify tool to automate the integration of PBS entities into WBS planning activities. The simplicity of the development method uses the technique of quickly creating simulations in a VR environment adopted from the work of Lucas [32]. Figure 1 summarizes the prototype development process.

Fig. 1.
figure 1

Summary of the prototype development process

Full size image

We chose to work with Revit Architecture to model and configure the data, then transfer and integrate them into the Unity3D game engine using the Tridify extension. Given that the motivation for creating a 4D model is to link the WBS elements with a corresponding PBS, it is therefore important to prepare a WBS from the Product Breakdown Structure corresponding to the breakdown of the model once downloaded in Unity3D. The Hierarchy window of the Unity3D engine interface contains a tree structure arranged by level, and each level contains the elements in the entities of the IFC diagram that are dedicated to them.

The code for performing the animation sequencing of WBS activities in the VR environment is in part adopted from recent work by Lucas [32]. It uses an array (Array) to contain the activities of the WBS in their Gameobject forms and has a container to serve as a countdown (increasing from zero) mechanism to trigger the tasks at each defined interaction.

The virtual reality experience was performed using an Oculus Quest headset. Unity3D in its recent versions offers an integration package for Oculus tools. These packages have been particularly useful in the modeling of the virtual character and accessories to build an immersive and intuitive experience. In order to conduct immersive constructability studies, it is useful to provide users with effective navigation techniques so that they can comfortably inspect different aspects of a changing construction site. To this end, the (walkthrough) mode seems to offer a fairly natural navigation compared to other modes such as the (flyover). To achieve this, we used the “OVR Player Controller” from the Oculus integration package. For additional character control and to adjust the camera easily during immersion, we added the possibility of manual rotation via the “SnapRotation” controllers to the automatic tracking that is already done on 6DoF (six degrees of freedom) with the Oculus headset. To achieve walkthrough experience in the VR environment, it is necessary to add collision property to the model elements, including the floors, walls and stairs.

After modeling the virtual character and arranging the immersive environment, the experience must be made interactive. Unity3D offers several packages with the accessories and toolkits necessary to develop the interactions sought by the user in a virtual environment. We used the “XR interaction toolkit” package to develop some interactive functions and capabilities. Beforehand, a Laser Ray (Raycaste) was added to the Oculus controllers to extend the range of user interactions. In order to control the simulation in the immersive environment, we added two arrow buttons to the UI to call the functions “construct the next task” and “deconstruct the existing task”. A UI menu in the form of a dashboard was created and attached to one of the user's controllers to accompany him during the immersion. This menu offers as shortcuts giving access to the available functions and interaction capabilities. We also connected the container numbers to a text space in the UI Menu (Fig. 2) in order to simultaneously generate and view the sequence number corresponding to each of the WBS activities triggered in the VR environment.

Fig. 2.
figure 2

UI-menu that gives access to VR control functions

Full size image

4.2 The Validation

In this step, we verify the applicability of the proposed method through the experimentation and the tests carried out on the prototype (the product of the method) and based on the criteria developed in the context of Design Science. The development of the prototype requires the selection of an ecosystem composed of suitable software and tools in order to perform the activities of the process under the best conditions. We validated the performance of the prototype and the effectiveness of the proposed method by testing the prototype. According to the criteria suggested by Hevner and Chatterjee [30] in the context of evaluation by experimentation, we verified the completeness (To what extent does the proposed method meet the integration objectives?), consistency (How does the method adapt to different contexts and under what conditions does it respond to challenges?), ease of use, and accuracy (A manual assessment of the accuracy of the rules that build the method).

The prototype’s completeness was assessed and compared to the objectives highlighted at each step of the method. It has been noted that the rules of interactions are insufficient in relation to the convincing objectives of the mechanism. Therefore, it is not yet feasible to use the feedback step to be able to evaluate the method through this first prototype, nor for the iterative aspect of the cycle. The method’s consistency was validated, as it is characterized by its ability to be reconfigurable. The formalism of activities can be remodeled and thus can accept different techniques and conditions depending on the software ecosystem. In addition, the method considers certain use ergonomics and accepts a simplified workflow at the heart of the two environments (Pre-VR and VR). Several activities can be automated, such as WBS development, rendering quality optimization and sequencing scheduling, which provides ease of use and accuracy of the results obtained.

5 Conclusion

Virtual reality is a developing technology at the crossroads of several disciplines. Identified as a remarkable visual medium with an immersive and interactive character, in the construction industry it is qualified to enrich and capitalize the value of our digital BIM models, as well as to improve and innovate the traditional practices. However, this technology still expresses a lack of integration, specifically to support 4D models, since the existing connections between 4D simulation software and VR support software such as game engines are still not adequate. The research presented in this article contributes to resolving this issue by creating an artefact in the form of a method that provides a new approach to strengthen the connection of 4D simulations to virtual reality environments. This method explores the initiative of an IFC-based 4D model to support a more efficient integration in virtual reality environments, optimizing data exchange with an iterative mechanism that accepts a two-way exchange link.

Following the hierarchical steps of the proposed method, a prototype was developed using an ecosystem of tools and software adapted to the evaluation of the method. The experimentation on this prototype within the framework of this project made it possible to simplify and standardize a number of activities for the preparation and integration of data, to identify rapid simulation techniques in the Unity3D game engine in order to explore the model in immersion in a virtual reality environment, and to control the dynamism of the 4D simulation following the succession of construction activities based on intuitive interaction.

However, this project did not make it possible to solve all the difficulties of 4D interactions in regards to composing and adjusting the sequencing and scheduling of construction activities in real time during a VR immersion and so it could not meet the challenges related to information return and data extraction. As indicated in the Discussions section, promising future work would be to focus on the aspects of data interaction, collaboration and feedback in order to provide a complete evaluation of the method and thereby develop a prototype capable of meeting the needs of future construction project management.