Abstract
Innovation technology in industries including manufacturing and aerospace is moving toward the use of Mixed Reality (MR) and advanced tools while Architecture, Engineering and Construction (AEC) sector is still remaining behind it. Moreover, the use of immersive technologies in the AEC digital education, as well as for professional training, is still little considered. Augmented and Mixed Reality (AR/MR) have the capability to provide a “X-ray vision”, showing hidden objects in a virtual/real overlay. This feature in the digital object visualization is extremely valuable for improving operation performance and maintenance activities. The present study gives an overview of literature about the methodologies to integrate virtual technologies such as AR/MR and Building Information Modeling (BIM) to provide an immersive technology framework for training purposes together with the Digital Twin Model (DTM)-based approach. Furthermore, the Facility Management (FM) tasks’ training on complex building systems can benefit from a virtual learning approach since it provides a collaborative environment enhancing and optimizing efficiency and productivity in FM learning strategies. For this purpose, the technological feasibility is analyzed in the proposed case study, focusing on the realization of a methodological framework prototype of immersive and interactive environment for building systems’ FM. Cloud computing technologies able to deal with complex and extensive information databases and to support users’ navigation in geo-referenced and immersive virtual interfaces are included as well. Those ones enable the DTM-based operation for building maintenance both in real-time FM operators’ training and FM tasks’ optimization.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Nowadays, Facility Management (FM) is mainly carried out through performance-based contracts stressing the importance of the building performance gap between the as-designed and the as-built configuration (Talamo et al. 2019). The interoperability between expected performance databases and actual ones remains a key obstacle for reducing the mentioned gap, but digital technologies have the potential to overcome it (Wong et al. 2018).
Indeed, monitoring the real data allows the assessment of such a deviation entailing a continuous Measurement and Verification (Manfren et al. 2020) activity that gives the chance to narrow the challenge to the predicted and actual maintenance (Atta and Talamo 2020) during operation. Quantity and quality of information depend on technologies adopted up to the level of big data calling for cloud computing, machine learning techniques and data environment readable by building experts (Ahmed et al. 2017).
Building Information Modeling (BIM) has been adopted widely in the design phase, while for the operation one, its use is often as an archive of information since the model is created after a survey of the as-built. However, great potential of this later created model can be expressed if merged with Virtual Reality (VR) technologies giving the chance to survey and inspect the buildings remotely (Sabol 2008). Furthermore, the experience can be further improved through Augmented Reality (AR) technologies overlapping the physical visit of the buildings with its virtual reconstruction. This merge provides instantaneously a batch of information to the surveyor up to a “X-Ray” vision experience. It entails those experts and non-expert who can benefit from this technology since required information to fill the skills and knowledge gap could be received using that technology. This undoubtedly leads to a widespread of this solution and expeditious adoption by workers and trainers.
2 Material and Methods
2.1 Background and Literature Review
Considering that the FM phase is the longest period of the entire building life cycle, information management utilizing BIM is still facing challenges related to on-site application since it is mainly used in the design phase (Chung et al. 2021). Due to differences in design, construction and FM BIM information requirements, the identification of specific data for operation and maintenance as well as for BIM integration with external management systems is still a research purpose.
To this end, an open-source BIM-based Facility Management information exchange system called Construction Operations Building information exchange (COBie) has been developed in the UK and USA (East and Carrasquillo-Mangual 2012).
Moreover, the evolution of digital approaches based on Virtual Reality (VR) and Mixed Reality (MR) introduced the application of immersive spaces for learning and education in medicine (Liu 2014), construction (Messner et al. 2003) and production engineering (Maffei and Onori 2019).
Focusing the attention on construction education, studies underlined the effectiveness of MR as an education tool.
Azhar et al. (2018) explored the potentials of MR and VR implementation in design review. Shirazi and Behzadan (2015) introduced complex construction processes through AR in construction management, showing that students interacting with the augmented environment were able to improve learning on tasks and activities.
The same digital approach could be potentially extended to maintenance service training, as MR provides immersive simulation in a multi-user collaborative and interactive environment for real-time live learning. Moreover, MR and other immersive technologies are promising solution tools for problem-solving and decision-making processes in operation and maintenance (Ke et al. 2016).
Considering that FM industry needs the acquisition of a large amount of data from different sources (Irizarry et al. 2013) due to the diversity of the maintained items (Rankohi and Waugh 2013), immersive technologies provide solutions to ensure collaboration between operators on the same up-to-date virtual information (Bae et al. 2013).
Moreover, as building lifecycle’s costs are mostly related to the operation and maintenance phase (Becerik-Gerber et al. 2011), the use of virtual collaborative solutions such as AR/VR/MR combined with cloud computing and artificial intelligence is significant in the FM industry (Zakiyudin et al. 2013).
In this regard, FM industry has increased its interest in digitalization since collaborative MR solutions could be useful for the optimization of maintenance activities, as it enables knowledge transfer between operators and stakeholders, providing them a context-sensitive interactive platform.
In fact, the use of MR instructions allows more than 82% error’s decreases in maintenance activities (Mekni and Lemieux 2014). In this regard, NASA’s experts use Microsoft HoloLens MR-based remote assistance in maintenance tasks (Hachman 2015).
Also, immersive solutions are utilized by Boeing (Sacco 2016) and other international companies to reduce production time and improve task location for maintenance operators.
Combining building information modeling (BIM) and MR (Chu et al. 2018) for FM (Gheisari and Irizarry 2016), remote collaboration and interaction through holograms is enabled, but it still represents a research area. In fact, the integration of BIM and MR technologies allows users to be virtually co-located even though the applicability is still under development in FM.
Therefore, the main purpose of this research is to enhance the use of BIM and immersive technologies in FM, increasing work productivity through the application of a Digital Twin-based framework. In this regard, Asset Information Model (AIM) information dataset has been defined, and Common Data Environment (CDE) and MR solutions are both used for maintenance and training activities.
2.2 System Architecture and Process Flow
The realization of the proposed framework is related to the implementation of different components and data flow in order to enable the digital Facility Management system architecture:
-
1.
BIM Model for Facility Management—Asset Information Model (AIM).
-
2.
DTM-based up-to-date data flow using mobile user interfaces.
-
3.
Cloud-based CDE platform for data storage and analysis.
-
4.
Building on-site maintenance activities using MR.
-
5.
Remote support and hybrid training using MR (Fig. 13.1).
The proposed framework is based on BIM data as the main source for physical and digital interactions. The configuration of a customized mobile user interface (UI) allows operators to gain maintenance on-site data directly updating the AIM through the CDE platform combining as-built and on-site data. The CDE platform also provides a knowledge base for data analysis, machine learning systems, prediction and reasoning.
Moreover, MR is introduced for holographic on-site operators’ experience as well as for prototyping a hybrid learning space for workers’ assistance and remote support.
BIM Model for Facility Management—Asset Information Model (AIM). The Asset Information Model (AIM), as defined by EN ISO 19650:2018, is a BIM model including specific spaces and assets of the building. This model is mainly used to support operation and maintenance phases. Typically, the AIM is the evolution of a Project Information Model (PIM) which is updated to incorporate modifications evolved during the operation phase (East and Carrasquillo-Mangual 2012). This model is also equipped with connectivity data to enable FM systems related to space, equipment and asset management. Specific operation and maintenance objects (OMO) need to be identified in the BIM model and structured according to AIM geometric and information standards which allow the interaction with other systems with a correct data flow.
A Facility Management dataset that can be associated to BIM’s OMOs is proposed in Table 13.1 defining both geometric and information standards.
The AIM is structured according to a set of control parameters which allow to keep a standardized database during the asset’s lifecycle, describing the actual status of a building, component, or asset over time.
Moreover, the AIM contains parameters related to the Construction Operation Building Information Exchange (COBie) StandardFootnote 1 (East and Carrasquillo-Mangual 2012) which is used for data transmission during the building life cycle. Such parameters are also useful for integrating AIM data with Computerized Maintenance Management Systems (CMMSs) (East and Carrasquillo-Mangual 2012).
DTM-based up-to-date data flow using mobile user interfaces. The concept of Digital Twin derives from Product Lifecycle Management (PLM) Grieves’ definition as a digital representation of physical assets and the connections between them (Grieves and Vickers 2017). As there is still no clear definition of Digital Twins, Kritzinger et al. proposed three main categories.
(i) The Digital Model (DM) is related to the use of BIM in the construction industry, as a digital representation without any form of automation in data flow; (ii) a Digital Shadow (DS) is based on real-time data flow from physical to digital asset. It is also defined a Digital Shadow when data are automatically and real-time transferred from physical to digital asset only; and (iii) a Digital Twin is a DM with automatic data flow from physical to the digital asset and vice versa.
The use of a mobile application is integrated in the proposed system architecture in order to collect on-site operational data which constantly and automatically update the AIM parameters. Such approach provides an up-to-date database based on the Digital Twin concept, as items of the operation and maintenance model (AIM) are constantly updated by on-site inputs, providing data analysis and reasoning systems to predict actions.
Cloud-based Common Data Environment (CDE) platform. Cloud computing allows end-users to access IT infrastructure and applications through the network, consisting in a combined set of technologies such as the Internet, distributed systems, virtualization. Depending on end-users’ utilization, it is defined as Infrastructure as a Service (IaaS), Platform as a Service (PaaS), or Software as a Service (SaaS). Despite many positive impacts on business management, the adoption of cloud computing solutions is still remaining behind in construction industry. The main benefits are decision support system, compatibility, information and organization, while weakness factors are mainly focused on data security and privacy.
The use of cloud computing in construction industry is strictly related to process management based on the use of BIM data organized through the implementation of CDEs. A CDE is a cloud-based and object-based system providing query, transfer, updating and management of project elements from a variety of data sources (Sacks et al. 2018) in a single multi-service platform. In such context, the configuration of a CDE platform plays a relevant role, ensuring a consistent data flow and enhancing efficiency through the connection of AIM data to on-site operations (Fig. 13.1). Moreover, the CDE platform is a cloud-based data service for data collection and analysis (Preidel et al. 2018) of both geometric and information models providing a knowledge base for the implementation of MR technologies.
Building on-site maintenance activities based on MR. Immersive technologies can help performance improvement of operators by providing virtual information about the real environment, as VR, AR and MR allow users to interact with an immersive and virtual scenario as represented by Milgram and Kishino (Fig. 13.2).
While VR is an immersive full-virtual application, AR provides a virtual-content layer over real-world scenarios, adding relevant information about the real-time captured image. MR is a combination of reality and virtuality.
The proposed framework aims to achieve FM performance improvements by introducing MR technology into on-site maintenance activities, where BIM data represent the baseline for merging the real and virtual world.
While VR is an immersive full-virtual application, AR provides a virtual-content layer over real-world scenarios, adding relevant information about the real-time captured image. MR is a combination of reality and virtuality.
The proposed framework aims to achieve FM performance improvements by introducing MR technology into on-site maintenance activities, where BIM data represent the baseline for merging the real and virtual world.
The present application proposes building on-site maintenance activities integrated to MR devices to improve maintenance process efficiency through holographic on-site implementation of the AIM. In fact, in the proposed framework, holograms are directly connected to the CDE platform, which provide maintenance geometric and information data coming from the AIM in a “X-ray” vision immersive virtual overlay. The MR extension allows operators to perform a new maintenance work process visualizing the AIM on-site through immersive overlays, checking and updating the AIM parameters and improving maintenance efficiency.
MR is also a tool for communication and alerts displaying as well as for real-time collaboration and information visualization during the entire operation and maintenance process.
Remote support and hybrid training using MR. Azhar et al. (2018) introduced VR for design communication teaching and observed that students felt unstable and needed supervision from trainers. While AR introduces digital objects overlaying on the real world, MR combines the real and virtual scenarios.
The main difference between AR and MR is compared to a sliding scale by Lehman and Tan (2021), where AR integrates digital objects in the physical world and MR allows virtual objects’ interaction in the real world. In fact, MR users are conscious of the real environment while involved in virtual actions. The use of MR devices improves students’ long-term learning as mentioned by Azhar et al. who observed the effectiveness of MR in enhancing communication capabilities in AECO education, revealing the potential of MR for practical and technical trainings.
In such context, AR/VR/MR tools are reliable solutions to train maintenance operators as (i) literature shows that the initial investment is totally repaid by training costs reductions; (ii) immersive technologies allow to visualize and interact with simulated items, according to practical learning approaches which increase trainees’ motivation and skills acquisition; (iii) MR systems allow to collect a variety of performance data in order to check and improve the training process (Borsci et al. 2015).
The objective of the proposed solution is to enable shared training experiences between physical and digital.
For the above mentioned, the proposed framework (Fig. 13.3) involves MR approach extended to training purposes (Ogunseiju et al. 2021) aiming to provide remote assistance from off-site expert to on-site operators sharing the on-site’s experience within the same holographic view.
On-site technical staff are able to connect to off-site building managers through video calls, replacing unnecessary physical inspections through the use of shared remote immersive experiences.
In this regard, an important example of cloud-based solutions to realize training metaverse has been proposed in Italy in 2022 by Microsoft Italia and Hevolus (2022), allowing the University of Naples Federico II to experiment the first didactic solution for participatory and laboratory teaching in MR.
With the same perspective, the use of MR-Hybrid Learning (MR-HL) environments can be introduced in maintenance service training with the aim of giving instructions to maintenance operators (Sepasgozar 2020) using MR environments.
In fact, head-mounted devices (HMDs) allow trainers to act and experiment with free hands (Agostinelli et al. 2021), showing maintenance practical activities using holograms. Trainees are involved in MR-HL environments being able to visualize holograms without wearing any device and by only accessing MR data streaming through a specific cloud-based spectator view platform.
3 Conclusions and Future Research
MR is a challenging digital technology for the FM industry, where digitalization is still relatively behind if compared to other sectors. The adoption of the proposed framework involves the introduction of digital collaborative solutions with the potential to increase efficiency in the FM phase.
The present study aims at contributing to research by investigating the concept of collaborative MR in operation and maintenance tasks as well as for workers’ training, exploring a possible framework architecture based on BIM an MR for different application areas of FM. The proposed solution is based on collaborative MR technology as it has a relevant potential in operative tasks as well as in education and training in FM sector, improving efficiency as workers currently have to manually get information from different sources and devices to achieve their tasks, leading to a large number of possible errors.
Moreover, despite the use of HMDs is currently restricted and uncovered in the construction sector (Khan et al. 2021), MR could provide access to a digital centralized information system obtaining substantial beneficial in work tasks’ optimization of FM.
In this regard, the proposed system architecture is aimed at finding solutions to potentially address the gap, acting as a starting point for future in-depth developments.
Moreover, structured and organized BIM data combined with MR systems provide improved understanding and are a valuable solution to resolve issues and inefficiency related to traditional design artifacts.
As technology progresses, immersive, realistic and customized training can improve education in many other fields of AECO industry such as construction site safety. MR enables workers identifying on-site hazards and gets improved context experience-based awareness. In this regard, future developments of the present research could be related to the extended use of MR to construction site safety training as a tool for enhancing hazard detection, avoidance, response and communication in order to reduce injuries and deaths in construction sites.
Notes
- 1.
COBie (Construction Operation Building Information Exchange) Standard specifies the transfer of information between the owner and the construction team. Following the last update of COBie in 2010, now it is called COBie2 and it provides readability both for humans and machinery. The human readable format is provided in Microsoft Excel Spreadsheet format.
References
Agostinelli S, Majidi Nezhad M, Cumo F, Heydari A, Muzi F (2021) Predictive maintenance strategy based on big data analysis and machine learning approach for advanced building management system. In: Proceedings of SEEP 2021 international conference, Vienna, Austria
Ahmed V, Tezel A, Aziz Z, Sibley M (2017) The future of big data in facilities management: opportunities and challenges. Facilities 35(13/14):725–745. https://doi.org/10.1108/F-06-2016-0064
Atta N, Talamo C (2020) Digital transformation in facility management (FM). IoT and big data for service innovation. In: Daniotti B, Gianinetto M, Della Torre S (eds) Digital transformation of the design, construction and management processes of the built environment. Research for development. Springer, Cham. https://doi.org/10.1007/978-3-030-33570-0_24
Azhar S, Kim J, Salman A (2018) Implementing virtual reality and mixed reality technologies in construction education: students’ perceptions and lessons learned. In: 11th annual international conference of education, 2018
Bae H, Golparvar-Fard M, White J (2013) High precision vision-based mobile augmented reality system for context-aware architectural, engineering, construction and facility management (AEC/FM) applications. Vis Eng 1(1):1–13
Becerik-Gerber B, Jazizadeh F, Li N, Calis G (2011) Application areas and data requirements for BIM-enabled facilities management. J Constr Eng Manag 138(3):431–442
Borsci S, Lawson G, Broome S (2015) Empirical evidence, evaluation criteria and challenges for the effectiveness of virtual and mixed reality tools for training operators of car service maintenance. Comput Ind. https://doi.org/10.1016/j.compind.2014.12.002
Chu M, Matthews J, Love PED (2018) Integrating mobile building information modelling and augmented reality systems: an experimental study. Autom Constr 85:1–26
Chung S, Cho C-S, Song J, Lee K, Lee S, Kwon S (2021) Smart facility management system based on open BIM and augmented reality technology. Appl Sci 11:10283. https://doi.org/10.3390/app112110283
East B, Carrasquillo-Mangual M (2012) The COBie guide. Available online: https://nibs.org. Accessed 1 Nov 2021
Gheisari M, Irizarry J (2016) Investigating human and technological requirements for successful implementation of a BIM-based mobile augmented reality environment in facility management practices. Facilities 34(1/2):69–84
Grieves M, Vickers J (2017) Digital twin: mitigating unpredictable, undesirable emergent behavior in complex systems. In: Transdisciplinary perspectives on complex systems. Springer International Publishing, Cham, pp 85–113
Hachman M (2015) NASA’s taking Microsoft’s HoloLens to the space station. PCWorld. Retrieved from https://www.pcworld.com/article/2940460/nasastaking-microsofts-hololens-to-the-space-station.html
Hevolus e Microsoft Italia annunciano Hybrid Learning Spaces: la realtà mista per una didattica sempre più ibrida. Available online: https://news.microsoft.com/. Accessed 10 Mar 2022
Irizarry J, Gheisari M, Williams G, Walker BN (2013) InfoSPOT: a mobile augmented reality method for accessing building information through a situation awareness approach. Autom Constr 33:11–23
Ke F, Lee S, Xu X (2016) Teaching training in a mixed-reality integrated learning environment. Comput Hum Behav. https://doi.org/10.1016/j.chb.2016.03.094
Khan A, Sepasgozar S, Liu T, Yu R (2021) Integration of BIM and immersive technologies for AEC: a scientometric-SWOT analysis and critical content review. Buildings 11:126. https://doi.org/10.3390/buildings11030126
Lehman SM, Tan CC (2021) Leveraging edge computing for mobile augmented reality. In: Fog/edge computing for security, privacy, and applications. Springer
Liu Y (2014) Virtual neurosurgical education for image-guided deep brain stimulation neurosurgery. In: International conference on audio, language and image processing, 2014. IEEE, pp 623–626
Maffei A, Onori M (2019) Evaluation of the potential impact of fully-immersive virtual reality on production engineering curricula. In: 2019 IEEE AFRICON, 2019. IEEE, pp 1–5
Manfren M, Aste N, Leonforte F, Del Pero C, Buzzetti M, Adhikari RS, Zhixing L (2020) Parametric energy performance analysis and monitoring of buildings—HEART project platform case study. Sustain Cities Soc 61. https://doi.org/10.1016/j.scs.2020.102296
Mekni M, Lemieux A (2014) Augmented reality: applications, challenges and future trends. Appl Comput Sci 205–214
Messner JI, Yerrapathruni SC, Baratta AJ, Whisker VE (2003) Using virtual reality to improve construction engineering education. In: American society for engineering education annual conference & exposition, 2003
Ogunseiju O, Akanmu A, Bairaktarova D (2021) Mixed reality based environment for learning sensing technology applications in construction. ITcon 26(Special issue Construction 4.0: Established and Emerging Digital Technologies within the Construction Industry (ConVR 2020)):863–885. https://doi.org/10.36680/j.itcon.2021.046
Preidel C, Borrmann A, Mattern H, König M, Schapke SE (2018) Common data environment. In: Borrmann A, König M, Koch C, Beetz J (eds) Building information modeling. Springer, Cham. https://doi.org/10.1007/978-3-319-92862-3_15
Rankohi S, Waugh L (2013) Review and analysis of augmented reality literature for construction industry. Vis Eng 1(9):1–18
Sabol L (2008) Building information modelling & facility management, the power of process in the built environment. IFMA world workplace. www.dcstrategies.net. https://pdfs.semanticscholar.org/cb40/288c07f9351120447f872a2e815f5f1a8db5.pdf
Sacco A (2016) Google Glass takes flight at Boeing. CIO. Retrieved from https://www.cio.com/article/3095132/wearable-technology/google-glass-takes-flight-atboeing
Sacks B, Eastman C, Lee G, Teicholz P (2018) BIM handbook: a guide to building information modeling for owners, designers, engineers, contractors, and facility managers, 3rd edn. Wiley
Sepasgozar SME (2020) Digital twin and web-based virtual gaming technologies for online education: a case of construction management and engineering. Appl Sci 10(13):4678. https://doi.org/10.3390/app10134678
Shirazi A, Behzadan AH (2015) Design and assessment of a mobile augmented reality-based information delivery tool for construction and civil engineering curriculum. J Prof Issues Eng Educ Pract 141(3):04014012
Talamo C, Pinto MR, Viola S, Atta N (2019) Smart cities and enabling technologies: influences on urban facility management services. IOP Conf Ser Earth Environ Sci 296:012047 https://doi.org/10.1088/1755-1315/296/1/012047
Wong JKW, Ge J, He SX (2018) Digitisation in facilities management: a literature review and future research directions. Autom Constr 92:312–326. https://doi.org/10.1016/j.autcon.2018.04.006
Zakiyudin MZ, Fathi MS, Rambat S, Tobi M, Uzairiah S, Kasim N, Latiffi AA (2013) The potential of context-aware computing for building maintenance management systems. Appl Mech Mater 405:3505–3508
Author information
Authors and Affiliations
Contributions
Conceptualization, S.A. and B.N.; methodology, S.A.; formal analysis, S.A.; investigation, S.A.; resources, S.A.; writing—original draft preparation, S.A. and B.N.; writing—review and editing, S.A.; visualization, S.A. All authors have read and agreed to the published version of the manuscript.
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2023 The Author(s)
About this paper
Cite this paper
Agostinelli, S., Nastasi, B. (2023). Immersive Facility Management—A Methodological Approach Based on BIM and Mixed Reality for Training and Maintenance Operations. In: Arbizzani, E., et al. Technological Imagination in the Green and Digital Transition. CONF.ITECH 2022. The Urban Book Series. Springer, Cham. https://doi.org/10.1007/978-3-031-29515-7_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-29515-7_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-29514-0
Online ISBN: 978-3-031-29515-7
eBook Packages: EngineeringEngineering (R0)