Keywords

1 Introduction

The European Program Horizon 2020 includes heritage research as one of the strategical points, focusing on the promotion of digital tools for the protection, value enhancement and dissemination of the rich cultural heritage of our continent (Llopis Verdú 2017, p. 333). In the midst of the digital age, ICTs are playing an essential role in the management of cultural heritage resources.

Digitizing documents aims both to preserve the original ones and to make them accessible to a great number of users. Digital metric survey systems are more and more used and they have come to be indispensable for the heritage registration. However, all this information whose main purpose is the preservation can be also used to disseminate the cultural values of the architectural heritage. Thus, an adequate methodology is needed to generate virtual recreations of the architectural heritage (Rodríguez Moreno 2016).

The use of new technologies to make cultural heritage known among population transcend the mere graphic reconstruction, statically drawn, for it includes interactive elements allowing the user for understanding in a deeper way the heritage values of a certain cultural asset.

Developing accessible applications may convert the mere informative consultation into an immersive experience of the contents to be disseminated. In this regard, Virtual Reality (VR) and Augmented Reality (AR) systems are extending the limit of graphics enhancing the interaction with the user to reach an integral understanding of the cultural asset.

This contribution aims to describe the state of the art by reviewing some remarkable cases whose VR and AR applications are at service of the heritage dissemination. In addition, it is briefly proposed a basic methodology to create and publish this kind of applications, which is the result of our own experience developed in a research project named El Sanatorio de san Francisco de Borja de Fontilles. Modelo de análisis para la recuperación integral de complejos sanitarios de valor patrimonial (HAR2013-42060-R), funded by the National Research Program of the Ministry of Economy, Industry and Competitiveness of Spain.

2 State of the Art

Every work regarding both heritage virtualization and virtual archaeology in general (restoration, recreation, anastylosis …) have to disseminate their results as one of their main goals. These results comprise digital models, either static or dynamic ones, which represent an object or architecture—shape and space—in its actual or original state.

For them to be disseminated, it is possible to use some techniques that are widely used in other fields of knowledge such as medical science or architectural design. We are speaking about Virtual Reality and Augmented Reality. Recently, a new technique combining the previous two has broken into the ICT landscape: It is called Mixed Reality (Fig. 1). Next, we analyse the potential of these techniques for the dissemination and preservation of cultural heritage.

Fig. 1.
figure 1

Mixed Reality glasses of the Asus Company. They contain front cameras

Full size image

2.1 Virtual Reality (VR)

Virtual Reality could be defined as the environment that is generated by computer technology in which scenes or three-dimensional objects are shown and create in the observer the illusion of being immersed in it. The Virtual Reality system will have higher or lower immersion degree depending on the interface used (screens, projectors, VR glasses …) and it will increase according to the user interactivity with such environment. Thus, it is possible to simulate a complete sensory experience by immersing oneself in a completely artificial environment perceiving no exterior reality stimulus.

Virtual Reality, apart from having different uses regarding medical science, education or entertainment, is being increasingly used for the dissemination of cultural heritage. The number of museums that, lacking space or other resources, opt for virtualization systems to display their collections is bigger and bigger (Basso 2017).

As it is said before, what is sought in a Virtual Reality system is to completely isolate the user to immerse him in a totally different reality. In this regard, one of the more effective devices are VR glasses that, together with closed headphones, reach this goal in a successful way. This has been done in the museum of Via Flaminia in the ancient Roman city of Fano: having performed an exhaustive 3D survey of the archaeological remains and their subsequent historical study, the buildings were virtually reconstructed and two Virtual Reality applications were developed (Quattrini and Ruggeri 2017). The first of them allows to display spherical panoramas in which it is possible to see the archaeological remains and their hypothetical reconstruction. It is only needed a smartphone and VR glasses such as Google Cardboard or similar. The second application recreates the roman forum space which can be walked around by the user. It is also possible to enter the Vitruvius Basilica. Since in this recreation the user interacts with a rendered space in real time, it requires grater processing capacity than a mobile device have nowadays. That is the reason why they have opted for using VR HTC Vive glasses together with a computer running the application.

One of the most notable advantages of using Virtual Reality techniques is the possibility of relocating the exhibition. The fact that a virtual environment is created completely independent of physical reality enables the virtual scene to be reproduced anywhere, provided that the appropriate devices are available (computer, VR glasses …). Therefore, it would be possible to access via internet a virtual exhibition of, for instance, an archaeological recreation without needing to visit the archaeological site (Fig. 2).

Fig. 2.
figure 2

VR application developed for Fontilles. Virgen de los Desamparados pavilion

Full size image

In recent years, there have appeared several websites that allow for online viewing, publishing and sharing 3D content. These platforms are real repositories of three-dimensional models that have been created by individuals (digital artists, archaeologists, architects or just 3D enthusiasts) and institutions (museums, foundations, universities …). Among them, SketchFab must be highlighted since its functionality allows for animating the scene, editing the global illumination, adding sounds and text notes … Moreover, it is compatible with VR devices to reach a totally immersive experience.

A remarkable example of this kind of virtual musealization is the development of a virtual gallery including the Romanesc capitals of the Sant Cugat del Vallés’ Abbey’s cloister, in Barcelona (Cabezos and Rossi 2017). This paper emphasizes the complete process of the virtual museum creation: from the capitals surveying using digital photogrammetry to their virtual exhibition through SketchFab.

The British Museum’s profile may serve as another significant example. Lots of artefacts are continuously being digitally modelled and uploaded to their SketchFab page so they can be viewed from any device anywhere in the world (Fig. 3). This way heritage is being disseminated among millions of people and so an extraordinary online museum is created.

Fig. 3.
figure 3

Profile of The British Museum in Sketchfab

Full size image

2.2 Augmented Reality (AR)

La Augmented Reality is a term used to define the combination of a real physical environment with virtual elements through a technological device. Combining tangible physical elements with virtual ones resulted in the enrichment—increase—of a real scene configuration. In contrast to Virtual Reality, in Augmented Reality the user is always perceiving the reality of the physical world.

Integrating a physical environment with virtual elements enabling the user to interact with in real time comprises two type of techniques: those extending the real space itself projecting images on it and implying a collective experience such as Projection Mapping or Video Mapping, and those augmenting the reality in real time with the content available through a screen (Cazorla et al. 2015). The latter require at least a digital camera that captures reality, a processing unit that performs graphic computing (GPU) and a monitor where the results of the process are shown. Nowadays, almost every mobile device (smartphone, tablet …) with normal features contains these three elements and, although their computing power is limited, they are already capable of executing applications of Augmented Reality in a satisfactory way. There are other devices, less extended due to their high cost, in which the screen is integrated in glasses (Microsoft Holo Lens, Google Glass …) so the Augmented Reality feeling is better.

Just as the VR, Augmented Reality has also numerous applications in the field of cultural heritage dissemination (Fig. 4). Sometimes, despite the great effort to disseminate historical and constructive studies, the understanding of some archaeological remains is still difficult for most of the visitors. The loss of volumes, coatings and colours affect the historical vestige realisation (Pagliano 2017) and a reintegration, perhaps virtual, of an object or a historical architecture is needed.

Fig. 4.
figure 4

AR application developed for Fontilles. Central Pavilion

Full size image

The idea is to build a three-dimensional virtual model of what is wanted to show and exhibit to the public. For that, leaving Video Mapping techniques aside, there are two possibilities of different degree of technical complexity. The first one is the simplest and consists of using markers or targets. A marker is a relatively complex graphic code printed on paper or cardboard that, shown to the camera, allow the system for placing a 3D model on it and display it in the screen. This technique was used in the Lonja of Valencia to show several representative architectural elements on markers coded with figurative referents related to the element that they were showing (Cazorla et al. 2015).

Similarly, for the musealization of the archaeological site Il Sacello degli Augustali in Miseno explanatory panels were placed with AR markers to show virtual reconstructions of the buildings of the complex. In addition, it is possible to place the virtual model on the archaeological remains themselves, provided the marker is taken by the system camera. For that to be possible, the coordinates of the ruins in relation with the explanatory panel position were introduced in the virtual model.

The second technique is more complex from a technical point of view. It consists of “augmenting” the “reality” of a real object on itself with no need of any marker, that is, analysing the object’s morphological features to place the virtual objects in the space. This way, the visitor is able to move freely around the artefact with no restrictions for the AR elements to be displayed. For the moment, the most effective system to carry out this technique is the Tango Project developed by Google. It has to be used in special smartphones with additional environment sensors, which, together with regular sensors (accelerometer, compass …), are the basis for the Motion Tracking, Area Learning and Depth Perception technology. (Note: by now there are only two available devices: Lenovo Phab 2 Pro and Asus Zenfone AR) (Fig. 5) Tango was the chosen technique for the musealization of the Cité de l’Architecture et du Patrimoine (Palais de Chaillot, Paris), where the visitor is able to view virtual reconstructions just by framing the artefacts from any point of view (Cannella 2017).

Fig. 5.
figure 5

Asus Zenfone AR compatible with Project Tango

Full size image

3 Methodology to Develop VR and AR Applications

3.1 Historical Study

Every complete architectural survey requires, apart from a thorough fieldwork to obtain a detailed metric model of the building, an exhaustive research about it for us to understand its origin and other vicissitudes that have led to its actual state. Thus, a complete knowledge can be reached when combining all the available information, namely, both the one we generate (geometric) and the existing (documental).

In this regard, within the frame of the afore mentioned research project, an exhaustive research has been performed consisting of a complete analysis of documents regarding the Sanatorium, from its foundation to these days, emphasizing the architectural and constructive aspects. Most of this information has been found in the Fontilles Historical Archive and ranges from legal documents and staff letters to plans, budgets and contracts for the construction works in the pavilions, along with an infinity of historical photographs of the daily life of patients. The centennial magazine of Fontilles has also been studied, as well as some publications and research works prior to ours.

3.2 Digital Survey and Model Edition

A good metric survey gives us accurate and reliable technical knowledge of the current state of a given construction. Its shape and dimensions make a three-dimensional information model, through which the building can be analysed giving a new historical and constructive reading.

The new surveying techniques comprise those that capture information using active sensors, such as Terrestrial Laser Scanner, and those using passive sensors (image-based) such as photographic cameras to take pictures and use SfM (Structure from Motion) algorithms. Combining both techniques provide us with a digital model featuring millions of polygons (high-poly) faithfully textured. Such a massive amount of information is hardly manageable without specific software, so that model is inadequate for dissemination. It necessary to optimize it, minimizing the loss of detail according to the available device memory.

For this optimization, there is a procedure with three operations to be performed in a 3D modelling software, namely, retopology, mapping and baking. The retopology operation consists of reconstructing the model in a rational way using quadrangles. Next, it is necessary to create one or more UV coordinate maps (mapping) for us to be able to properly texturize the object (baking).

3.3 Graphical Reconstruction

Las Archaeological reconstructions must respond to the Principles of the Letter of Seville about the recognition of the interpretative phases, restoring the philological correction of digital reconstruction (Quattrini and Ruggeri 2017). In this regard, these works have to base on a thorough historical study considering every existing document (pictures, drawings, paintings, scientific or literary texts, comparisons with other architectures …). It is also crucial a well performed digital survey, either architectural or archaeological.

In the project of Fontilles, the graphic reconstruction of every phase has been approached using two different methods. The first one has to do with the geometric restitution of old photographs, while the second focuses in the stratigraphic analysis of the walls. For both, the digital model of the building has embodied the starting point to measure and size every graphical hypothesis.

3.4 AR and VR Applications Development

Once every digital model has been optimized, it is time to begin with the application development itself. For that, a game engine is needed, which is a special software that includes pre-programmed routines to develop and play a videogame. In our case, Unreal Engine 4 was chosen for VR development and Unity 5 along its plugin Vuforia for AR development.

Game Engines in general facilitate the development of interactivity between the user and the virtual system through the events creation. This feature may be useful to show extra information about the historical building (or object) through interactive menus, interest points, location plans … In Unity it is necessary to type simple C# scripts to design each function, whereas in Unreal everything can be managed thanks to Blueprints Visual Scripting, which is a graphic programming system with nodes to avoid typed code (Fig. 6).

Fig. 6.
figure 6

Development in Unreal Engine 4 using the Blueprints system

Full size image

Finally, before exporting the result and generating the application executable file, it is convenient to configure the resolution according to the device it is intended, since a mobile or tablet usually have less computing capacity than a computer.

4 Conclusions

In this contribution the current techniques of Virtual Reality and Augmented Reality have been described. Their practical potential serving the dissemination of cultural heritage values has been verified in several cases in which virtual technology complements positively the traditional museography.

In this regard, although these technologies are important to enrich visitors experience, the public still demands to see the objects in person experiencing them actively (Accardi et al. 2016). Keeping artefacts in deposits and replacing them virtually is not enough and that is the reason why it is so important knowing to combine technological innovation with museographical tradition.

The afore proposed workflow draw from the historical study and digital survey of a determined cultural asset (the Sanatorium of Fontilles) in order to develop VR and AR applications to virtually reconstruct its original shape. Thus, VR and AR development must enhance the cultural mediation to increase the value of the asset and its understanding, making it more attractive to the public. The procedure should follow the Principles of Seville, representing the uncertainties that underlie the undertaken philological reconstruction, avoiding historical fakes, to enhance the integral conservation of all the heritage values of a cultural good.