Keywords

1 Introduction

The most interesting feature in immersive technologies research is to properly define the various aspect of “R”. According to Milgram and Kishino (1994) in their “Virtuality continuum” the digital elements added to the real environment enhance the reality with the final result of a totally digital environment called VR—Virtual Reality. The steps between reality and virtuality are known as Mixed Reality (MR) that includes Augmented Reality (AR), in which virtual digital objects are added to a real environment and Augmented Virtuality (AV), in which real objects are added to virtual ones. The immersive technologies were further defined as XR (cross reality) as “fusion of ubiquitous sensor/actuator networks and shared online virtual worlds” (Lifton and Paradiso 2009). Virtual Reality (VR), as a term, was popularized by Lanier (1988) to describe experiences where the user is entirely immersed into a three-dimensional digital virtual world and interacts with it (Fig. 1).

Fig. 1
figure 1

M.I.R.A. Morandini—the perspective section cuts through the void to show the full perceived volume of the virtual museum © Gabriele Pitacco and Monica Bidoli

Full size image

A literature survey in the VR definition, presented at the beginning of almost each research paper, shows, within the architecture and architecture—related fields, some kind of homogeneity in the definition of the concept, without identifying a shared one or a prevailing one.

In 1996 Stuart identifies five features describing the characteristics of VR systems that are: (1) three-dimensional viewing, (2) dynamic display, (3) users are active navigators, (4) the image displayed in the VR is from the point of view of the user’s head, and (5) multimedia interaction.

Freina and Ott interestingly underline the existence of these two different kinds of VR based on viewing methods: non-immersive and immersive. The former is defined as “a computer-based environment that can simulate places in the real or imagined worlds; the latter takes the idea even further by giving the perception of being physically present in the non-physical world” (Freina and Ott 2015). The project analyzed within this document is developed within the latter. This chapter presents an example of local based immersive experience where virtual and real object are included in an experience where the user has the perception of being physically present in the non-physical world.

2 Architecture and Virtual Reality

According to the literature there are several cases where AR, VR, and XR is used in different fields either not related to architecture in any way or where architecture doesn’t play any role, being considered as a neutral background for other more interesting elements. According to Paranandi et al. (2002) “although AR, VR and XR have found good applications in Medicine, Flight Simulation, and Video Game Industry, its effect on architecture remains imperceptible”, but also (Wojciechowski et al. 2004; Freina and Ott 2015) and many others share a similar point of view.

If we focus on architecture and design, the role of AR, VR and XR is mainly restricted to be a (common) medium for digital representation, visualization and/or reconstruction. Digital visualization can range from the (1) reconstruction of old or destroyed building and artefacts, to (2) the prefiguration of future ones, to (3) the creation of impossible spaces and objects or to (4) other forms, like training tools.

Visualization and representation of old artifacts and objects or simulation of historical buildings can be seen in several literature survey that explore the reconstruction of lost or never built projects like, for instance a Roman Palace (Dima et al. 2014), virtual restoration of partially or fully damaged buildings and structures on historic sites to enable visitors to see them integrated into their real environment (Panou et al. 2018), visualization of incomplete or broken real objects as they were in their original state by superimposing their missing parts (Liarokapis and White 2005) or even museum objects, generic 3D gallery designed for presenting cultural objects in a virtual room, a virtual exhibition presenting museum artifacts in a 3D room being a reconstruction of a real gallery—an exhibition corridor in the Victoria and Albert Museum in London (Wojciechowski et al. 2004).

As Roussou perfectly underlined “the use of architectural detail in immersive real-time virtual reality systems is difficult due to the technical and performance restrictions placed by the realtime image generator”. From an architectural disciplinary perspective this can be considered a critical issues, especially in art and restoration, where the attention to the “materiality” and detail is crucial and the artwork is already a form of representation given by the artist or the architect (Fig. 2).

Fig. 2
figure 2

Marina Morandini (daughter of the poet Luciano Morandini) in front of a Getullio Alviani artwork, has been the very first visitor of the M.I.R.A. Morandini. © Luca D’Agostino (Phocus Agency)

Full size image

The most explored topic in literature is the visualization and representation of designed future entities being them buildings, artworks, objects or other things. According to Schnabel (2001) the manipulation of virtual environments during the design process pushes designers to better perceive space, for example its fluidity and functionality, without using 2D representations. VR gives the possibility to experience sensations and movement in an artificial environment that is a simulation of some aspects of the real world. According to Milovanovic, focusing on the field of architecture, VR applications’ utilization are wide, from design itself, construction and project’s communication as well as collaborative decision-making (Milovanovic et al. 2017).

In all the aforementioned cases AR, VR, and XR have been used as design medium, components or representation tools, just like a pencil, a design table or a CAD software, while their potential as a final outcome, as a design subject instead of a design tool, is yet to be explored and exploited.

In 1969 Superstudio turned their final outcome of a design process into strongly political drawings. The final outcome of the design was not intended to be a building or a monument but the negative utopia represented (Lang and Menning 2003). The medium turned into the final outcome of the design process. That’s the conceptual inspiration for this design of the project presented here.

3 Spatiality in (Virtual) Reality

According to Brett (2016) “Architects are trained and experienced in spatial design—spatial thinking, traditions of spatial theory, a language of spatial communication and representation, and an archive of precedent and typology are some of the skills that uniquely qualify architects to contribute to the spatial design of virtual reality experiences. Virtual reality experiences designed without attention to spatiality misses the most important opportunity of the technology.” This works show an example of how architects should be thinking about AR, VR, and XR as not just a tool to design and visualize physical spaces, but as something to design in and of itself, with attention to the spatial experience.

Virtual reality can either replicate the physical world and its rules or imagine a completely different one, detached from reality. Virtual reality space offers different opportunities and shows different limitations than physical space. Light, physics, navigation, materiality, construction, and environmental issues are different in the virtual world, and the solutions and design choices require a different approach able to incorporate and explore the possibilities of their brand new spatiality.

The virtual reality also presents its own limits and constraints. A new virtual economy based more on performances than on budget, an economy deeply based on the calculation speed of the used hardware. Optimization is the new budget—polygon counts, complex lighting calculations, high numbers of textures—these all impact processing speeds that can affect battery life, hardware requirements, and frame rates.

The choice between a physical replica, a brand new world or all the in-between options must be deeply considered. According to Brett “There is value in the familiar. Skeuomorphic design takes advantage of archetypal physical forms to present something recognizable to the user. While virtual space design should not necessarily replicate all the details of physical space, designers of virtual reality spaces can reference familiar forms and organizational structures that will help users understand and navigate virtual space. Certain physics may no longer impose the same limitations on form and movement in virtual reality space as it does in physical space, but designers should not necessarily see this as an imperative to throw away all conventions. Forms and materials may no longer have the same functionality in virtual reality space, but they may still express certain behaviours and qualities through reference to their physical analogues. Where physical constraints no longer dictate form and space, designers may look to psychological factors in usability—affordance, signifiers, and conceptual models—for guiding forces in virtual space design” (Brett 2016).

4 A Museum Exhibition of AR, VR, and XR

In the past years museum have seen the exploitation of multimedia techniques and lately the introduction of virtual reality methods to create new forms of presentation for exhibitions. Virtual Reality can offer a number of advantages to museums, offering a way to overcome some common problems like the lack of space or the need of visitors to interact with the exhibits. According to Lepouras, a broad categorisation of virtual museums reveals that they vary from fully immersive cave systems to simple multimedia presentations. (Lepouras et al. 2001). Several examples can be found from the XR Guide for the Guggenheim Museum of contemporary art in Bilbao (Abawi et al. 2004), to the Manchester Jewish Museum (Tom et al. 2016), from the tourist guide for cultural heritage sites located in the old town of Chania, Crete, Greece (Panou et al. 2018), to the British Museum (Rae and Edwards 2016), the Great North Museum (Atkinson 2015) and also the Grant Museum, DiMoDA (Digital Museum of Digital Art), immersion room at Cooper Hewitt Smithsonian Design Museum, Met Cloisters in New York City, Hermitage, MAAT, the new museum of Art, Architecture and Technology, in Lisbon, Muse in Trento, (Symbola 2019), but none of these offered the kind of experience presented here.

5 M.I.R.A. Morandini. Augmented Reality Immersive Museum “Luciano Morandini”

The M.I.R.A. Morandini (Augmented Reality Immersive Museum dedicated to Luciano Morandini) was presented on Saturday 7 September 2019, on the 10th anniversary of the passing of the intellectual and poet Luciano Morandini, in his home-town, San Giorgio di Nogaro (Udine), Italy.

The M.I.R.A. Morandini is the first prototype of an immersive museum with both real contents, such as a selection of paintings from the Morandini Legacy (which collects the works of some of the post-war Friulian masters like Zigaina, Spacal, Alviani, Bassi), and virtual ones, like volumetric holograms of the sculptures, 360° immersive videos of performances, multimedia works dedicated to the poet and the reading of the poems by Luciano Morandini. The Museum prototype, tested here for the first time, is partly real and partly virtually “augmented” with more contents, more space and a contemporary architecture and design. The Museum can be visited through the latest-generation virtual reality headset Oculus Quest, released in May 2019.

Born from an idea of the architect Gabriele Pitacco (www.gp-a.it), the project was created with the XR project concept and development by Antonio Giacomin (www.fluido.it), the support of the association S@NGIORGIO 2020 for the exhibition contents and BIC Incubatori FVG srl—center of excellence on incubation and business development as certified regional incubator. The Scientific and Technological Research Area of Trieste—Area Science Park was involved for the technical and operational coordination and the Department of Architecture and Civil Engineering of the University of Trieste for the research of perceptive and spatial aspects. Comunicarte curated the graphic design, while the Municipality of San Giorgio di Nogaro acted as a host structure.

The prototype is part of the project “Hacking Real Space” funded by the Friuli Venezia Giulia Region on the Call of the European Fund POR FESR FVG 2014-2020, Action 2.1b.2 “Grant for the financing of personalized acceleration and company consolidation programs, aimed at the entrepreneurial development of projects of cultural value, aimed at cultural and creative companies”.

6 A Museum in a Room (Il Museo, Cielo Incluso, in Una Stanza)

In the House of Poetry dedicated to Luciano Morandini, in a dark room looking empty (Fig. 3), a virtual reality headset is the passe-partout for accessing a unique museum: the space multiplies and expands to accommodate the world of the poet, iridescent and ever-changing, and the passage from one scenario to another is a journey in which poetry is the means of transport and the reader/visitor the protagonist.

Fig. 3
figure 3

The physical entrance to the M.I.R.A. Morandini in San Giorgio di Nogaro

Full size image

The Morandini Augmented Reality Interactive Museum is a unique experience, in which the visitor is immersed in a world that does not follow the same rules as the real one, where existing physical elements and virtual elements interpenetrate and complement each other. Once worn the virtual reality headset, the actual space multiplies in height, generating further floors which divide, move away from each other and expand into the virtual space to create an exciting central space.

It is a space defined by a triple height void (Figs. 4 and 5) that visually connects the different thematic areas. A void that allows the museum to be perceived in its entirety and able to provide orientation and points of reference to the visitor who enters a space different from the real one enriching the visit of unexpected and exciting points of view and gives the space a unitary vision.

Fig. 4
figure 4

M.I.R.A. Morandini—the render of the poetry floor (level-1) of the Museum shows one of the text of the Morandini poems floating in space

Full size image
Fig. 5
figure 5

M.I.R.A. Morandini—the render shows the view from the upper level to the lower ones, showing the visual connection between them and the void

Full size image

The museum is designed in sections as a sequence of floors, cut by a diagonal beam of light entering from the roof. Light creates a central void that defines a series of floors staggered in space, a raumplan of spaces with different size, height and character. Spaces tailored on the works exhibited and, at the same time, visually connected to each other.

Movement between the different levels is possible through a series of virtual elevators, portals for teleportation -quoting Star Trek- which allow you to jump from one floor to another, almost immediately.

The width of the central void, while being shifted in space, is always constant: there is a fixed dimensional relationship between the various planes, obtained—in complementary pairs—by the subdivision of the real floor. The larger floors are the result of a virtual extension to the outside, while the smaller ones are obtained by the subtraction of this same surface. By placing all the floors along a central imaginary magic line it is possible to reconstruct the real space.

The virtual museum is closely linked to real space and the existing physical building. The entrance to the virtual museum takes place on the first floor, as in the real museum. The visit then takes place freely descending to the other floors and returning to the first floor at the end, to exit.

The lower floor, at the end of the path, becomes the meeting point and the passage between virtual and real, where the real artworks of the Morandini Legacy are visible.

Exactly like the work of poet, the museum presents, within its unitary structure, a series of environments that are different from each other, but capable of interpenetrating, communicating and influencing each other.

7 Hold the (Magic) Line

To expand the limited size of the original room, a “magic line” was invented (Fig. 6). The imaginary line that separates the two halves of the real room is stretched to the thickness of the empty space. This stretched void creates a completely new virtual space, which in the virtual building corresponds to the central passage and thus separates the planes that are on one side and the other of the void. This filter, which corresponds in reality to a very narrow strip in the plan, allows to virtually generate an enormous environment that is completely permeable, through which you can look out at other planes and see what happens on them. In this way, the entire real surface of the room is exploited at its most, several times, with different proportions and contents in the different virtual levels.

Fig. 6
figure 6

M.I.R.A. Morandini—the plans (above) and section (below) diagrams shows how the magic lines expands the space creating both virtual voids (light blue), and virtual space (dark blue) that can be seen but not experienced

Full size image

The areas of the larger levels are obtained extruding the original volumes towards the outside, and although they are not walkable, they visually look like that they are, giving the idea of depth and welcoming all those works and objects that need to be observed from a distance, such as multimedia videos and blow-ups of the poet’s photo-portrait.

In the area delimited by the magic line there are the virtual elevators, the portals of teleportation: a square space projecting on the void, straddling the magic line, which gives access to the following floors.

In the same way, the central empty space may not be walkable as it is protected by glazed parapets, but for the bravest who want to try walking on the air, there is a unique experience to be tested.

8 Images and Words

The museum opens with a biographical section (Fig. 7) and a brief note on Luciano Morandini, curated by Carlo Londero, poet’s scholar, with the voice of Fabrizio Gaio recorded at the Casa della Musica (House of Music) in Trieste, together with the Luca D’Agostino’s photographic portraits, curated by Phocus Agency and a 360° immersive experience. The 360° immersive experience is meta-space that bring a whole park inside a floating solid looking sphere that can be crossed.

Fig. 7
figure 7

M.I.R.A. Morandini—the axonometric view of level 0 shows both the floating sphere (an object that can be passed through to enter the garden of Villa Dora, a meta-space that is wider than the virtual one already perceived) and the virtual space that outdistances the two photographic portraits

Full size image

The next level contains a reasoned selection of the poetic works of Luciano Morandini (Fig. 8), curated by Luisa Gastaldo Morandini with Carlo Londero, whose reading was made by Chiara Dorigo (Poesie a Manovella) and Rossella Gorgoglione at GLB sound of San Giorgio di Nogaro (Udine). The words of the poem are floating in space, close to walls without touching them.

Fig. 8
figure 8

M.I.R.A. Morandini—the axonometric view of level-1 shows the layout of the poetry floor, as in Fig. 4

Full size image

The volumetric holograms of a selection of the sculptural works that constitute the Morandini Legacy fluctuate in the next space (Fig. 9). The selection, curated by Massimo Premuda of the Casa dell’Arte (House of Art) of Trieste, emphasizes a work by Dora Bassi, the only female artist in the collection, and an abstract work in bronze by Giovanni Piccin. The volumetric holograms have been realized with a three-dimensional photographic scan realized by Fluido.it by Antonio Giacomin. The song “Vento” performed by Elsa Martin with music by Stefano Battaglia on texts by Luciano Morandini and recorded at Swiss Radio and Television RSI—Radio 2 has been selected by Gianluca La Boria from the works presented at the International Festival of Jazz and Poetry “Festival J&P” of San Giorgio di Nogaro (Udine).

Fig. 9
figure 9

M.I.R.A. Morandini—the axonometric view of level-2 shows the Statues floor with the tridimensional hologram of the Dora Bassi artwork

Full size image

Remote Transitions (Fig. 10) is a contemporary art performance and the central element of the following floor. Created by the Cultural Association S@NGIORGIO 2020 together with the Estonian Academy of Music and Theater in Tallinn and the Tartini Conservatory of Trieste, the performance can be enjoyed in an immersive 360° video with spatialized audio that allows the spectator to experience an extract of the event in first person (Fig. 11). Remote Trasitions is a performance realized between three “poles” connected at a distance. In the first two poles (the Tallinn Music Academy and the Tartini Conservatory of Trieste) there are musicians (with traditional instruments, but also computers) connected via the Lo.La system (an acronym for Low Latency) in real time. In the third pole (San Giorgio di Nogaro) the two A/V flows of Tallinn and Trieste arrive in streaming, but with a time delay; nevertheless the double-bass player Giovanni Maier improvises on the music coming from Tallinn and Trieste, and dancer Nina Alexopoulou improvises on the music, while an interactive video by Florence-based Studio RF follows her movements in real-time.

Fig. 10
figure 10

Remote Transition—the photos show the synchronized performances in Tallin (left) Trieste (top-center) and San Giorgio di Nogaro (bottom-center and left)

Full size image
Fig. 11
figure 11

M.I.R.A. Morandini, the axonometric view of level-3 shows the second meta-space dedicated to “remote transition”, an immersive 360° immersive experience showing the art performance connecting in real time Trieste, Tallin and San Giorgio di Nogaro

Full size image

At the end of the visit (Fig. 12) there is a selection of the graphic and pictorial works of the Morandini Legacy, with a selection of the works of some of the post-war Friulian masters, Alviani, Anzil, Chersicla, Marini, Spacal, Zigaina made available by the Municipality of San Giorgio di Nogaro. This selection is the bridge, the link between physical reality and virtual reality, connecting the two worlds though the overlapping of the artworks.

Fig. 12
figure 12

M.I.R.A. Morandini—the perspective plan of level-4 shows the overlap of virtual and real. The paintings and artworks are both in the real floor and the virtual one, connecting them to create a local based immersive experience where virtual and real object are included in an experience where the user has the perception of being physically present in the non-physical world

Full size image

The museum (Fig. 13) represents an all-round virtual experience, able to involve all senses thanks to a varied exposition in the forms of art and in the represented themes.

Fig. 13
figure 13

M.I.R.A. Morandini—the perspective section close up shows the different floors and the exhibition lay-out

Full size image

9 Aspects of XR Development

Implementing a technology that interacts between real and virtual world has many issues. The main issue in a Location Based Experience (LBE) is to maintain the reference between real and virtual space. The inside-out tracking method, used by the headset, analyse in real time the environment defining the borders of the virtual experience setting the common spots in the two realities. The real environment survey has to be perfectly aligned with its virtual reproduction. Once the real/virtual relationship is defined the architect/designer can modify the virtual world structure to recreate and redefine the space. Another relevant issue is the interactivity between two different virtual spaces or crossing the above mentioned “magic line”. The main research and development was focused on the teleportation system using a trigger system that allows to move virtually without moving physically enhancing the space experience.

In the prototype are also inserted a meta-spaces in form of a sphere where the user can enter in an environment bigger than it’s container.

The whole system was prototyped and developed within a game engine (Figs. 14 and 15).

Fig. 14
figure 14

M.I.R.A. Morandini—the photographs show a selection of the reactions of the first 100 users that visited the M.I.R.A. Morandini

Full size image
Fig. 15
figure 15

Luciano Morandini, portrait © Luca D’Agostino (Phocus Agency)

Full size image

10 Conclusions

Ivan Sutherland, the creator of one of the world’s first VR systems in the 1960s, stated: “The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal” (Sutherland 1965). The road to get there is still quite long but MIRA Morandini shows a concrete example and an argument for why architects can add values in the design not only of real spaces, but also for virtual ones.