AR and VR to Enrich Cultural Heritage and Retail Experiences: ETT Case Studies and Guiding Principles

Abstract

In recent years, both the cultural and the industrial/retail sectors have been going through an intense period of development, characterized by suggestions from ICT. The use of Augmented and Virtual Reality features for the enjoyment of cultural and retail experiences has become increasingly demanded when it comes to engage different users and increase levels of immersion aiming to the transmission of information. ETT, specialized in technological innovation and Experience Design, has developed high level know-how in the creation of AR and VR experiences, both in Culture and Retail, though the use of cutting edge technologies and engagement and storytelling techniques.

1 Company Description

ETT S.p.A. is a Digital and Creative Industry specialized in technological innovation and Experience Design. It was established in 2000 and it currently employs over 150 people between Genoa headquarters and various offices in main Italian cities and in London. ETT has been active for years in the New Media sector, for which it creates innovative applications exploiting the potential of new technologies in contexts related to edutainment, culture, tourism, communication and marketing. It combines original design, storytelling and cutting-edge technologies to create engaging experiences for museums, corporate and retail spaces. In this fields, since 2008, ETT has delivered over 1000 multimedia installations in about 100 museums and private customers, for a total of over 4.5 million visitors. Even if ETT case histories concern application areas that at first glance may seem distant—Cultural Heritage and Industry/Retail—the approach that guides project creation and experience development is unitary. ETT’s aim is to create unique user experiences, focused on engagement, in order to improve the level of “knowledge”—whether it is about the cultural attractor or the product/industrial process—by using AR and VR immersion techniques, interactivity and storytelling.

2 Project Summary

2.1 Augmented Reality Projects

ETT has designed and developed many Augmented Reality applications, allowing users to explore cultural heritage assets and product features in an interactive way. In Augmented Reality applications, using the video stream captured by the mobile device camera and associated to image recognition, it is possible to add multimedia contents and information (e.g. texts, pictures and 3D reconstructions) on the surrounding real world or object. AR applications developed by ETT allow users to explore virtual environments, view the 3D reconstructions of ancient monuments overlaid on their real ruins, interact with 3D Avatars of historical figures. In the following paragraphs the main projects presented are: Cenacolo Vinciano App, Accademia Galleries App and UnipolSai CarViewer App.

2.2 Virtual Reality Projects

Focusing on the enhancement of cultural heritage, smart tourism, corporate spaces and retail, ETT has designed and developed many immersive tools based on Virtual Reality features. Virtual Reality is used to replace the user’s real-world environment with a simulated one. Using specific VR headsets, users can enjoy immersive virtual tours of fascinating historical sites or reproductions of industrial processes, with means of both entertainment and training. Thanks to Virtual Reality applications, users can explore virtual environments by feeling completely immersed in 360-degrees panoramic views or computer graphics 3D reconstructions of sites and scenes. In the following paragraphs the main projects presented are: Ars Excavandi, Teatro alla Scala Museum, Area X—Intesa San Paolo Assicura.

2.3 Projects Offering Both Augmented and Virtual Reality Experiences

Furthermore, many ETT projects encompass both Augmented and Virtual Reality features, in order to create immersive experiences with many degrees of immersion and engagement. In the following paragraphs the main projects presented are: The Ara as It Was, Virtual Destination Italy, Exploracity and Musei in Comune—Interactive Video Guide.

3 Project Details

3.1 Augmented Reality Projects

In the Cultural Heritage field, the main AR including projects are:

Cenacolo Vinciano App

The new Leonardo da Vinci’s Last Supper App makes the visit experience to the Museum even more comprehensive and engaging. The free App, developed both for Android and iOS devices, takes museum visitors on a unique tour. The Last Supper’s charm is revealed through narration techniques, helping visitors to appreciate the technical and compositional choices made by Leonardo. Inside the Refectory, the AR feature makes the visit even more immersive and evocative. The Stories section is dedicated to further historical and artistic information that helps to contextualize the painting through text cards supported by 360-degree videos and pictures.

Accademia Galleries APP

In the monumental estate of the Academia Galleries of Venice, ETT redesigned the visit experience by several touch exhibits and a mobile application, in order to improve the visitor experience and enjoyment of works of art. The mobile app, developed for iOS and Android platforms, is designed for different types of users—Children, Teenagers, Adults—and helps investigate aspects and details of the works that cannot be perceived at first sight. Augmented Reality contents, through which visitors can see what emerged during restoration work and the artworks in three-dimensional mode, in addition to videos with in-depth information, are available.

In the Retail, the most recent AR including project is:

UnipolSai CarViewer App

CAR Viewer is an Augmented Reality application that offers an innovative solution for discovering vehicles, exploring them virtually, deepening knowledge of the various models characteristics through 3D Real Time vehicle reconstructions. Clients can assess different vehicles until they find the ideal one, before they even touch it, in a simple, immediate and safe way. The application allows to visualize and interact with the model and to position it in the room where the client is, maintaining contact with the real world. The AR experience can be enjoyed through last-generation tablets or smartphones.

3.2 Virtual Reality Projects

In the Cultural Heritage field, the main VR including projects are:

Ars Excavandi

“Ars Excavandi” is the exhibition that opened the year of Matera 2019 as the European Capital of Culture. Thanks to visitors take part to an immersive excursus into the history of underground art and relive the magnificence of the excavated architecture. The final part of the exhibition is situated in the hypogea, characteristic underground places of the city of Matera. They’re 1200 sq. Meters underground that can go up to a depth of 12 meters. The Virtual Reality experience developed by ETT mimics a walk in the hypogea and guarantees the chance to visit the “underground Matera” even to those who are affected by physical disabilities and could not otherwise enjoy it.

Teatro Alla Scala Museum

The digital innovation project developed by ETT is related to the strategy of the Teatro alla Scala Museum for the enhancement of the exhibited works. Besides multimedia stations and a mobile app guiding visitors along the Museum itinerary, the novelties include, in particular, a Virtual reality tour of the Theatre. Wearing specific VR headsets and guided by the prima ballerina Nicoletta Manni, visitors can explore the Opera Theatre and discover its beauties and secrets in a highly engaging tour.

In the Retail field, the most recent VR including project is:

Area X: Intesa Sanpaolo Assicura

Area X is an innovative edutainment space that relies on the concept of “protection”, where live interactive and engaging experiences in Virtual Reality take place. Crossing the doors of the Intesa San Paolo exhibition area, a unique adventure starts, bringing the visitor to an evocative alien planet where protection is as important as on planet Earth. Here, as settlers, visitors are asked to choose how to act in this new world. In three different stations is possible to explore the planet in different environments: driving a jetpack to fly among suggestive floating islands, visiting an alien housing module or driving a space rover on a journey through breath-taking 3D scenarios. The experiences are linked by one guiding principle: the more you are protected, the more fascinating and full of surprises the adventures will be. Before facing an adventure, special “space” insurance policies may be selected and taken on the trip. These will protect the score from the unforeseen events that would lower it. This helps people to understand which is the best type of insurance for every circumstance in life.

Fig. 1

The Ara as It Was, Rome

3.3 Projects Offering Both Augmented and Virtual Reality Experiences

The following projects include both Augmented and Virtual Reality features, and have all been developed in Cultural Heritage and Tourism fields.

The Ara As It Was

The “Ara as It Was” experience is the first systematic Augmented and Virtual-reality project carried out in Rome about Cultural Heritage. The Ara Pacis is one of the most important Roman art masterpieces, built to celebrate the peace achieved by the Emperor Augustus throughout the Roman Empire. Using Samsung Gear VR headsets and Samsung S7 smartphones, a multimedia story unfolds, showing the original appearance and function of the altar, both using AR and VR features. In the first part of the visit, it is possible to see the alterations that happened through time to the northern part of Campo Marzio, the area of Rome chosen by Augustus to acclaim his power, through Virtual Reality. The second part of the experience is instead based on Augmented Reality. Using the device camera, it is possible to activate contents about the original colours of the monument and the stories represented by the bas-relives. Thanks to an algorithm developer by ETT, the multimedia contents are anchored directly on the monument, without ever interrupting the vision of the real monument (Fig. 1).

Virtual Destination Italy

Virtual Destination Italy is an innovative platform offering a multisensory interactive tour in which the visitor can identify himself with the character of a young nobleman of the XVII century, taking his Grand Tour in Europe and Italy. The Grand Tour is the traditional trip of Europe undertaken by mainly upper-class European young men. The Virtual Destination Italy platform, thanks to a mobile application and geolocation tools, offers immersive experiences, where the user interacts with various characters, scholars and artists of the time that, through Augmented Reality features, appear on the screen in the form of 3D avatars and tell stories and anecdotes about the different places. The platform, moreover, offers visitors Virtual Reality experiences, dynamic comparisons between past and present pictures of the same place and 360-degree panoramic images, in order to enhance the knowledge of the Cultural Heritage offer of the different places that once made the Grand Tour great.

Exploracity

Exploracity is an innovative platform for the enhancement of tourism and cultural heritage of the city of Genoa. As a pre experience, Exploracity comprises touch screen stations and VR experiences, located at the Tourist Information Offices, offering multimedia contents, photos and videos about the main points of interest of Genoa, with gaming applications dedicated both to children and adults. During the experience itself, an interactive and immersive exploration of the city is made available to users thanks to a mobile Application that, based on GPS geolocation tools, offers amazing Virtual and Augmented Reality experiences. Thanks to this platform, visitor receive the first notions and historical reconstruction of the city’s cultural development.

Musei in Comune Rome: Interactive Video Guide

The Civic Museums of the City of Rome, Italy—Musei Capitolini, Museo dell’Ara Pacis and Mercati di Traiano-Museo dei Fori Imperiali—have been equipped with a highly innovative video guide created by ETT, which combines the characteristics of a traditional audio guide with innovative technologies, thereby creating a “multimedia tour”. The interactive video guides aims to revive the visiting experience through the curiosity and emotional engagement, achieved with an indoor location system—based on Beacon (BLE) technology—that automatically suggests content to visitors that is associated with nearby artwork. Augmented and Virtual Reality also support the delivery of information content via storytelling techniques, by showing, for example, the protagonists of the frescos (Fig. 2), X-rays of paintings and 360° photos of the museums halls.

Fig. 2

Musei in Comune Interactive Video Guides, Rome

4 Feedback from End Users

Even if the main user feedbacks that ETT projects receive are positive—in terms of usability, engagement, transmission of information—“The Ara as It Was” at the Ara Pacis Museum in Rome is the main project developed by ETT that has been analysed and tracked by an academic committee, in partnership with Università degli Studi di Napoli Parthenope (Italy). The latest empirical analysis has been carried out applying the visitor experience model for mixed reality, which measures how 23 functional and experiential elements have been reshaping a traditional museum visit, enhancing visitor satisfaction (Trunfio et al., 2019). The empirical analysis findings and conclusions open up new scenarios for future research on immersive museum experiences, combining cultural heritage with technological innovation. One of the latest paper produced on the project “The Ara as It Was” tries to shed light on this topic by analysing how mixed reality functional and experiential elements have been reshaping the traditional museum visit, opening up opportunities and challenges to build innovative immersive experiences and enhance visitor satisfaction. The Ara Pacis Museum constitutes a best practice of heritage exhibition management under conditions of mixed reality, in which virtual interaction with cultural and historical artefacts enhance the immersive visitor experience (Bec et al., 2019). By integrating physical museum elements with virtual and augmented spaces, the “Ara as It Was” project reshapes the iconic Ara Pacis Museum and redesigns museum service models. Empirical analysis results show that the “Ara as It Was” project represents a successful combination of advanced technology and innovative storytelling, increasing visitor presence and participation levels. Furthermore, findings of the empirical analysis allow the introduction of fresh knowledge into the theoretical debate and propose some managerial implications for the co-creation of value in the museum experience.

5 Future Outlook/Roadmap

As the media theorist McLuhan pointed out in 1960, media types are way more than just passive channels of information, being capable of affecting people in different ways. Even if it is still not completely known what extent AR and VR will impact human thinking processes, it is clear that the use of these technologies has become an integral part of cultural, retail and training experiences. The main perspective that is expected in the field of AR and VR technologies is certainly the implementation with Artificial Intelligence mechanisms, which can be used, in the form of chatbots or natural language comprehension, to enhance and enrich user experience.

6 Conclusion

Augmented and Virtual Reality features development is changing Cultural Heritage and Retail realities impressively fast. This process allows Industries working in ICT, like ETT, to transform the relationship between users and experiences. Users turn the visit into an increasingly personal journey of contamination, shared emotion and involvement. The constant evolution of the needs of the public—both Cultural Heritage and Retail—is based on the search for experiences and new stimuli. An answer to the search for these new stimuli can be represented by the wise use of AR and VR technologies, which ETT has consolidated thanks to the know-how developed in the field. The wise use of AR and VR technologies is related to the transmission of correct and scientifically validated information, and to the use only in cases where they enrich the experience in ways that would otherwise not be possible. In conclusion, it is clear that the use of technologies is destined to improve—in terms of usability and success in conveying information—if guided by constant research and comparison with international scenarios development. These are ETT’s guiding principles.