Keywords

1 Introduction

Architectural design is concerned with designing human experience, most commonly manifest in the physical built environment. Through the design process, drawings, models and 2D renderings are used to understand space. A full understanding of this experience often only begins during construction and becomes clear after a building is finished, thus limiting the potential for physical experience to make a meaningful contribution to the final product.

During the past few decades, the architectural, engineering and construction (AEC) industry has identified the potential that performance-driven design could have in tailoring the experience that the built environment could have for its users. Interactive interfaces and pipelines allowed users to test how well their designs performed, e.g. daylight, thermal comfort, wind or visual connectivity analyses, were developed and adopted in AEC pipelines to help designers make the right performance decisions early on. Still, the actual experiential side of what it means to be in that space was only possible through visualisations and movies. The growth of XR in terms of hardware (headsets and tablets) as well as software (Games Engines) has changed this, giving designers the opportunity to place themselves within an XR experience of their buildings to make experience-driven design decisions.

XR offers an opportunity to bring experience into the design process. Over the years, the authors’ practice has developed XR tools enabling designers to experience their designs at early stages. Glaucon is an XR toolset developed by the authors that increases user ability to collaborate in the same space and remotely, connecting designer team workflows with experiential design tools and hardware. Glaucon has been used in several high-profile real-life projects within the practice with both internal and external users using a broad set of hardware types.

In this paper, the authors will show precedent technologies that were developed to facilitate the experiential side of design decision making and how these innovative approaches amplified its collaborative aspect. Leading to that, this paper demonstrates how these efforts have led to the development of Glaucon, what is the platform’s architecture and how it has been used in the design process. The paper identifies several domains where Glaucon has been used to create a high degree of visual fidelity in a collaborative XR experiential design process and brought the experience of a built space into design processes, enhancing both design evaluation and iteration, in order to raise the quality of design outcomes.

2 State of the Art: The Development of XR and Its Incorporation in the AEC Industry

Virtual Reality is primarily seen as a platform for visualising digital or simulated environments (Machover and Tice 1994). VR (as a subset of immersive technologies and XR) can inherently benefit the AEC industry. XR technology centres on user experience and supports various types of communication and interactions in the design process, which has gradually become an area of increasing interest and development activities in the AEC industry. Research has shown that XR technology can enable better collaboration and communication between designers, stakeholders, managers and end-users (Bassanino et al. 2010; Fernando et al. 2013; Van den Berg et al. 2017) allowing users to evaluate and validate the proposed design (Bardram et al. 2002; Dunston et al. 2007; Loyola et al. 2019), and it supports designers to collaboratively evaluate the functionality and usability of proposed environments (Whyte 2002; Hilfert and König 2016) and reduce cost and waste associated with physical mock-ups (Maldovan et al. 2006; Majumdar et al. 2006).

However, the applications of XR technologies in the AEC field often reveal the tension between the multi-disciplinary nature of the workflow and bespoke XR solutions for real-life projects (Van den Berg et al. 2017). Several studies have explored the possibilities of informing decision-making processes with XR HMD systems in the AEC industry from a technical point of view and concluded that realness could be extended for experiential congruence (Otto et al. 2003; Nikolić and Whyte 2021). Various existing XR systems are focusing on visual fidelity where pipelines for bridging CAD tools and visualisers are well developed but lacking the capability of enabling remote or local collaboration between designers and stakeholders and the possibilities of allowing users to partake in the design process across a board range of XR hardware systems and collaborate with others both physically and virtually (Enscape 2022; Resolve 2022; TheWild 2022). Also, many collaborative XR tools are being developed for remote collaboration in virtual environments (Spatial 2022; Horizon 2022). However, they have not been developed specifically for the AEC industry, where most interactions are developed solely for communication without considering interactions with the virtual context.

3 Experience and Decision Making in the Design Process in Practice

The authors are employees of an architectural practice that has been investigating the potential of XR and experiential feedback in the design process for over two decades. The authors’ team has developed several immersive and interactive tools to enable designers to experience designs at 1:1 scale, thus facilitating a better understanding of geometries and spatial configurations. These interfaces and applications have been widely used in various real-life projects (from industrial design to city planning) and stages of the design process (conception to completion), offering an innovative approach to enhance the design process and workflows between design and support teams.

To that end, XR technologies have been used since their “commercial infancy” to facilitate design reviews of various projects. A stereo projector was used to immerse designers in their designs, allowing them to review boat hull geometry or even cockpit visibility while sailing the proposed design into the harbour. Similar, dome projectors were used for collaboratively evaluating architectural projects through the immersion of various stakeholders in the virtual space. The team had even backed VR HMD headsets in Kickstarter and used games engines to create VR walkthroughs (Fig. 1). These immersive experiences had proven essential for design decision-making, showcasing how the use of XR systems could – to an extent – replicate that of a physical mock-up. Nevertheless, these initial attempts with XR technologies allowed the user to experience the proposed environment in a predominantly stationary manner and were limited to specific interactions, originating from the nature of the hardware.

Fig. 1.
figure 1

Stereo projector, dome and VR use on prior projects

Full size image

4 Glaucon – An Innovative XR Platform

Following the experimentation around various XR technologies and with the exponential evolution of both software and hardware, the authors have focused on developing an XR platform that could go beyond visualisations and allow for a fully immersive, collaborative virtual environment. Glaucon was developed to address a need to bring the experience of a built project directly to live site environments in order to understand what experiencing that space would be like more closely. This needed to be accessible irrespective of physical or virtual presence and across a broad range of XR hardware systems. The core application needed to be general-purpose to be easily reused or adapted for multiple project contexts. Additionally, bringing the collaborative nature to the heart of the virtual experience was key.

4.1 Methods

In order to develop this toolset, Glaucon was developed using a high-fidelity real-time graphics engine that provided online networking support together with a core API that could be readily built upon and extended using custom scripting and code. The toolset is built on top of a games engine, running on a high end gaming PC connected to a consumer grade VR headset with an additional camera to facilitate AR tracking (Gillespie et al. 2021). Versions have also been developed for iOS and PC streaming VR. Its common set of functionality as described below.

Graphical Fidelity

The application is built on top of a game engine (Epic Games Inc. 2022) that provides high-quality visual output across a broad spectrum of XR device types. This was the same video game engine employed by our practice’s visualisation team to facilitate a smooth data pipeline when collaborating to create high-quality XR experiences.

Spatial Alignment

Tools were developed to enable virtual spaces to be aligned to their physical counterparts, establishing a correspondence between environments and allowing them to be augmented and extended virtually. These tools make use of computer vision and AR libraries to create image marker-based anchor points that could be positioned in the physical environment and mapped to corresponding virtual equivalents (Gillespie et al. 2021). When combined with spatial mapping technologies on most consumer-grade XR hardware, spaces could be aligned and tracked in real-time (Fig. 2).

Fig. 2.
figure 2

Spatial alignment establishes correspondence between reality and virtual environments

Full size image

In some cases, device-specific implementations needed to be developed for specific hardware types. Some devices natively incorporated this functionality as standard (e.g. mobile AR), whilst others required additional hardware configuration and AR libraries or alternative input systems in order for the spatial alignment process to be carried out (Fig. 3).

Fig. 3.
figure 3

Additional hardware added to VR headset to facilitate AR marker recognition

Full size image

Collaboration

Glaucon supports collaboration between physically present and virtual users who can all see and speak with one other, and it includes a range of functions that enable users to share their experiences.

Collaborative functionality, already provided by the video game engine itself, was extended by implementing intuitive and straightforward interactivity together with a mode of user avatar representation appropriate for the review context. Basic human body language could be read in a manner corresponding with reality, inferred from user inputs (Gillespie et al. 2021). Project content forming the basis for collaboration could be loaded into virtual environments at any scale and linked design workflows.

Limitations in game engine’s networking, such as data type replication and file size management, were overcome to allow collaborative virtual design environments of any scale to be created, saved and come back to as persistent experiences that could be revisited as part of a project design review process. A downloadable content pipeline (DLC) was also implemented, enabling third parties, such as visualisation teams using the same engine, to export high quality environments that Glaucon could load in.

Design Team Workflows

Glaucon was designed as a generic toolset to work alongside design team processes. The content loading system works with models exported from design applications, and a series of high-quality environments were implemented, providing several review contexts of varying scales for this.

Hardware

Glaucon has been designed to support a broad set of XR hardware types:

  • 2D Desktop (PC)

  • Tethered VR headsets (PC)

  • Standalone VR and AR headsets (Streamed from PC)

  • Web browser (Pixel Streamed from Server)

  • Mobile AR Tablets (iPad)

Complexities developing for a broad set of target platforms were mitigated using a hardware agnostic SDK (Khronos Group 2022) and adopting XR streaming from PC servers for standalone headsets (NVIDIA Corporation 2022), which enabled consistency of XR experience across device types. Mobile AR was developed separately, allowing bespoke implementations of AR libraries and device hardware limitations to be managed.

4.2 Implementations

Glaucon has been used on several projects across different work stages in varying contexts. Three application domains have been identified, showcasing distinct scenarios where Glaucon has brought the experience of built space and experience to design processes.

Physical On-Site and Mock-Up Experiences

The ability to align physical and virtual experiences has enabled Glaucon to be used to augment and extend the physical environment with the virtual at scale to experience projects as if they had been built in-situ.

Glaucon has been used to extend a physical mock-up virtually using a VR backpack PC and VR headset. It was able to extend a physical mock-up of half a lobby constructed in a warehouse (Gillespie et al. 2021). A full-scale experience of the environment was possible where physically, only half of the space could be constructed in plywood and plasterboard. The virtual allowed the extension of the physical to represent the built experience, adding materiality and views to city streets. It also allowed virtual design options to be cycled to aid decision-making (Fig. 4).

Fig. 4.
figure 4

Warehouse space used for XR mock-up

Full size image

Glaucon has also been used to enhance the experience of a physical site to understand the contribution a design would make to the urban realm in-situ in a live urban context. A virtual model was developed to an enhanced level of detail to stand up to close scrutiny (Gillespie et al. 2021) which combined with the tactile nature of the site, the feel of the live urban context combined with the virtual to create an enhanced feeling of the proposed design within its live site context (Fig. 5).

Fig. 5.
figure 5

Onsite full-scale collaborative XR experience

Full size image

Comparisons could be directly made between real and virtual by removing the headset, and the lead architect directly presented the project as a physically and virtually present user. A visitor experience was also developed that allowed the client to start their experience in the physical plaza and experience the visitor journey up the reception lift to a high-level viewing gallery, as if the project had been built (Fig. 6).

In this domain, all users were physically present, some but not all using XR hardware. Where users were represented as virtual avatars, this created a correspondence and sense of the present in physical and virtual spaces allowing the experience to be presented and discussed as if they had been physically built.

Fig. 6.
figure 6

Comparisons between reality and virtual experiences

Full size image

Virtual Review Experience

Glaucon has been used in an entirely virtual context where users sought to experience or review an environment as if it were real. Multiple users who were not physically present could engage with the design and be part of the decision-making. These reviews have been primarily experienced with backpack VR in a similar manner to the onsite examples, and streaming VR has also been used successfully with a smaller number of users (Fig. 7).

Fig. 7.
figure 7

Virtual review experience

Full size image

By combining the experience of the space with the detailed design models and enabling multiple users to collaborate virtually, the impact of the detailed design, for example, in one case, a staircase in a residential project, and its contribution to the overall experience could be understood. Several environments have also been developed in conjunction with specialist teams to create the basis for a formalised review environment, allowing live project content to be loaded, cycled and review space states saved for immediate recall at a later point.

Mobile Experience

Glaucon has been used in a mobile context both as an onsite AR experience and entirely virtual use cases. It has been used to increase accessibility and application reach and to allow users who may not have access to or be comfortable using head-mounted XR hardware. Remote app deployment has been used to increase application reach to any user with a compatible device.

Fig. 8.
figure 8

Onsite mobile AR experience combining real and virtual experiences

Full size image

Several projects have been developed using Glaucon’s spatial alignment toolset to allow users to gain an understanding of a project as built, overlaying virtual with reality to create an augmented experience with the device acting as a “magic window”. Virtual only experiences have also been developed where users have been able to experience a project using their mobile device as a window without needing to be situated on site. Collaboration between users has occurred physically through users showing others in the same space the view from the device (Fig. 8).

5 Conclusion

Glaucon has built upon prior precedent from within the practice and in the wider XR field allowing physical spaces to be enhanced through an XR experiential layer of immersive VR and tablet-based AR contexts. It is likely that given increased use as a design tool in practice, the impact of experience-driven decision making in this context will become more evident.

Glaucon has enabled design teams to understand and experience their designed spaces in a readily accessible and more immersive manner than previous tools have allowed, engaging with user experience and feel prior to the final built product. It has facilitated an enhanced level of collaboration across hybrid spatial environments, and through its approach to hardware development increased application reach both within and outside the practice, allowing virtual projects to be physically situated and experienced in-situ, has created a heightened sense of experience than has previously been possible.

Where alternatives to the experiences Glaucon enables are physical equivalents at full scale or achieved using costly mock-ups, tools like Glaucon have the potential to reduce waste from physical mock-ups whilst simultaneously allowing more options to be assessed at scale better informing any physical mock-ups constructed. Whilst the purpose of construction mock-ups is more than just visual, virtual mockups have a very clear economic benefit; in addition to that, Glaucon offers a higher degree of visual fidelity and situated physical experience than has been previously possible, which may therefore have a greater impact upon this process.

Glaucon’s testing as the basis for a collaborative design review environment is an area of ongoing research. Where Glaucon offers an enhanced experience at scale and on site, its ability to directly load and configure varied review spaces and environments means that in similar ways that Glaucon offers benefits to the mock-up process, it could offer similar improvements to more traditional review environments and whilst this may not and arguably should not eliminate physical design review processes, it may enable them to increase in meaning and value when they do occur.

Given the rate of current XR hardware development, the opportunity to design hybrid spatial experiences and the spatial alignment technologies developed may increase Glaucon’s value as a design tool. As virtual experiences themselves become key design outputs in a similar manner that BIM is and Digital Twins are becoming, future iterations of Glaucon or similar other tools may serve as test or deployment environments for augmenting and extending existing space using AR technologies to create hybrid spatial environments as part of a designed experience.