Keywords

1 Introduction

Museums are fundamental Cultural Institutions of modern society, being places of reference, research, and dissemination of knowledge related to many disciplines. In museums exhibitions, tangible Cultural Heritage, which refers to physical artefacts and sites, and intangible Cultural Heritage, which includes practices, representations, expressions, knowledge, and skills [1], are preserved and presented to the visitor as traces that witness the culture and reveal its deep meanings.

To keep up to date with emerging social needs, museums use different approaches to communicate and exhibit Cultural Heritage. Among different approaches, digital technologies are commonly adopted to engage visitors in memorable and amusing experiences. Furthermore, museums integrate digital technologies to pursue their objectives of education, entertainment, accessibility, and exploration [2].

Among the digital technologies adopted, there seems to be a growing trend related to the enhancement of traditional museum exhibitions through the use of multisensory eXtended Reality technologies (XR) – which include Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) - to blur the line between the physical and virtual worlds and allow visitors to experience an engaging sense of immersion [3]. Indeed, multisensory XR technologies can greatly impact the realm of Cultural Heritage, bringing numerous advantages to museums such as increased visitor engagement and involvement. Furthermore, XR technologies enable the digital restoration of Cultural Heritage sites that have been lost or are inaccessible, or even the partial reconstruction of ancient artifacts to bring back their original form. The digital recreations of the artifacts are displayed to visitors as interactive and immersive experiences. The visitor is actively involved in a journey of discovery where the artifact serves as a tool for communicating cultural significance. Additionally, XR technologies can enable tangible and intangible interactions between visitors and the artifacts, allowing for exploration of pieces that may be too delicate to handle in real life. Finally, XR technologies may be crucial in increasing accessibility and inclusivity. Many studies in museums and art galleries also point out the integration of multisensory XR technologies to stimulate multiple human senses [4, 5], enhance emotional engagement and improve learning activities.

Although museums seem to recognize the substantial benefits of XR technologies, these experiences are frequently only temporarily incorporated into museum exhibitions [6]. The integration of these technologies is often related to extemporaneous attempts that make it difficult to assess the actual effect of multisensory XR technologies on visitors’ perception, learning experiences and engagement.

In addition, some researchers have highlighted several problems that may arise from incorporating technology in museum exhibitions, such as the costs associated with implementation, adoption, and maintenance of the technologies and the potential risks in their use related to the limited visitor experience [7, 8]. Indeed, some museum professionals are cautious in adopting new technologies to avoid “disturbing” the intimate relational experience created between the visitor and the artefacts [9], or to prevent the risks of favouring entertainment over educational purposes in a “Disneyfication” of the museum’s offerings [10].

However, despite the existence of a limited number of comparative studies, such as [11, 12], it appears challenging to assess the impact of multisensory XR technologies on museums and exhibitions visitors experience. Indeed, to perform a comprehensive assessment, various factors must be taken into consideration and compared. These include the use of different multisensory XR technologies, and the consequent different levels of interactivity and immersion, the levels of satisfaction and enjoyment of the experiences, and if the integration of technology is more effective for learning and engagement compared to traditional exhibitions. However, evaluating all these elements in a real-world setting is a complex task.

The paper presents a comparative evaluation by conducting a laboratory-based research activity based on a framework to compare different experiences of the same museum exhibition using a traditional approach and various XR technologies. Furthermore, the framework compares the impact of adopting multisensory approaches and different degrees of immersion on visitors’ experience and learning performances. The study outlines experimental activities that simulate a museum exhibition, allowing the user to interact with different paintings using different XR technologies within a laboratory environment.

2 Multisensory Immersive Technologies in Museum Exhibitions

Museums and other Cultural Institutions often incorporate digital technologies alongside traditional methods, either before or after the exhibit, to provide additional exploration or recall of content. However, some museums choose to enhance and supplement the viewing and comprehension of artifacts during the actual exhibition visit [13, 14].

Regarding museum exhibitions, traditional methods such as panels and audio guides may not be seen as effective by visitors in terms of user engagement and content learning and may be perceived as limiting. More often, museums include 3D reconstructions and digital heritage representations and integrate immersive multisensory technologies, such as eXtended Reality (XR), by offering Virtual Reality (AR), Augmented Reality (VR), and Mixed Reality (MR) experiences.

XR technologies have the advantage of making content more engaging for the visitor, who is often called to interact first-hand with museum artifacts, becoming an active participant in the experience. In addition, XR technologies also allow the creation of amazing and stunning experiences for the user. Thanks to the reproduction of virtual environments, they can find themselves immersed in another world estranged from the surrounding real one. This aspect can be crucial to allow visitors to focus on the proposed content, resulting in higher engagement and positively impacting learning activities. Additionally, XR technologies enable the combination of various digital media to provide multisensory stimuli, frequently engaging multiple senses such as hearing, sight, and touch at the same time. This feature is regarded as crucial to enhance content learning, as research has shown that multisensory engagement leads to better retention of information in the user's memory [15].

Many researchers explored the integration of XR technologies in museum exhibitions. Li et al. stated that the digital museum should not replace the traditional museum, but the digital contents should complement the physical ones by also considering the role that museums have for the visitor [16]. This aspect can be easily assessed by using AR to complement physical artifacts exhibited inside museums and by using VR to recreate a virtual representation of the museum or of the artifacts exposed. Geroimenko [17] investigates the potential of AR as a new artistic medium, and underlines some advantages compared to other more traditional media, stating that AR is spatially not limited, not expensive to produce and exhibit, and easy for making interactive and multimedia content.

Numerous examples can be found in literature that demonstrate the positive impact of XR technologies on enhancing the visitor experience in museum exhibitions. Weiquan et al. developed AR Muse, an iPad application where users can view the AR animated versions of six paintings after framing the original ones, showing a higher time spent looking at the artwork and better long-term identification when AR is used [18]. Ararat is an iPhone application designed by Kei et al. where it is possible to see real-time animated versions of different paintings, including those of Van Gogh and Da Vinci [19]. Baradaran used AR to display an animation on the Mona Lisa painting, adding a political and patriotic message to the original artwork [20].

Concerning VR, some Institutions have integrated this technology to provide visitors with historical content about the artist and his paintings, as in the case of “The Modigliani VR: The Ochre Atelier” experience that was developed as part of the Modigliani exhibition at the Tate Modern Museum in London and was integrated into the collection made of paintings, sculptures, and drawings [21]. Others instead adopt VR to revive the paintings themselves, such as “Mona Lisa: Beyond the Glass”, which is the Louvre’s first VR project that consists of a journey back in time to meet the real woman that da Vinci painted [22]. Also, the Salvator Dalì Museum created “Dreams of Dalí”, an immersive VR experience in which it is possible to explore Dali’s painting Archaeological Reminiscence of Millet’s Angelus [23]. And yet, in other cases, VR is used to create fully virtual displays, such as “The Kremer Museum” launched in 2017, which consists of a virtual museum where Dutch and Flemish Old Master paintings from the Kremer Collection are exposed [24].

It appears that the use of immersive, multisensory technologies is more prevalent in science and technology museums, where the focus is on enhancing the user's interactive experience. In these museums, audience engagement is typically achieved through hands-on exploration. As stated by Dalsgaard, Dindler, and Eriksson [25], science and technology Institutions build their processes by providing means to explore cultural heritage in a participative and active way. Instead, in art museums, there seems to prevail a contemplative paradigm in which the adoption of traditional technologies is preferred.

In the realm of museums, determining the effects of XR technologies on the visitor experience can be challenging, especially in the context of art museums. Measuring the impact that these technologies have on the museum visit is also difficult to define. To explore the hypothesis that XR technologies can enhance the museum experience, the authors created a framework and used it as a guide for conducting various experimental case studies to evaluate the impact of multisensory XR technologies on the museum visitors’ experience.

3 A Framework for Investigating the Factors that Influence the Visitor Experience at Museums

In this Section, we present a framework for creating and evaluating museum experiences that takes into account various elements such as the museum objectives, target audience, technological tools, digital resources, and experience modalities (Fig. 1).

Fig. 1.
figure 1

Framework including various elements for creating museum experiences.

Full size image

The purpose of the framework is to bring together various aspects for designing and evaluating museum visits. It is intended to be adaptable to any type of museum, including science and technology museums and art museums and galleries.

The framework serves as the foundation for conducting case studies to explore and compare different museum visit experiences. These case studies will be evaluated to determine which factors and combinations are most successful in fulfilling the museum objectives and meeting the needs of the target audience. The studies will be conducted in a laboratory setting as this environment offers a better control and allows for a clearer understanding of the correlation between variables, as opposed to in a real-life museum exhibit.

3.1 Framework Description

At the top level, the framework includes two dimensions: the museum objectives and the target audience, that are interconnected. These two dimensions are then linked to another dimension, related to the technological tools used within the museum exhibitions, the digital resources, which provide the means for content delivery, and the experience modalities, including elements such as immersion, sensory stimulation, collaboration, and interaction.

The museum objectives have been limited to User Experience (UX) and engagement, learning, accessibility, exposition, and exhibit. This allows us to focus on specific aspects of the visitor experience during a museum exhibition and exclude objectives such as the preservation of cultural heritage that are not related to the visitor experience. The target audience is adults or kids.

At the bottom level of the framework, the three other dimensions are included.

The technological tools, which are used by the museum to enhance the visitor experience, are categorized on a scale ranging from real to virtual. These technologies include more traditional tools, such as panels and audio-guides, and more forefront ones, such as AR and VR headsets.

Digital resources concern the modalities of conveying content through technological tools. They include representations of different sensorial stimuli, such as audio tracks, images and video, 3D models and animation, odors and tactile stimuli. Connected to these two dimensions, are the experience modalities, which include immersion, sensory stimuli, collaboration, and interaction. Each category is presented with different levels, ranging from no immersion to full immersion, with dots indicating an increase in the level.

3.2 From the Framework to the Experimental Case Studies

Among the proposed combinations, the comparison will initially focus on traditional technologies versus cutting-edge ones (see Fig. 2). Experimental case studies have been designed to compare four types of experiences, with a focus on the UX and engagement and learning museum goals and an adult target audience.

The content proposed in the four experiences is the same, enabling a comparison of the experiences from a learning perspective. In the chosen experiment, the dimensions of sensory stimuli, collaboration, and interaction have been considered null for all four experiences. Concerning the immersion aspect, the AR experience is considered not-immersive as it is delivered through a smartphone application, whereas the VR experience is considered semi-immersive.

Moreover, two of the selected experiences use more conventional methods (panel and audio-guide) with little or partial integration of digital resources.

The authors posit that XR technologies are more effective in terms of engagement and content learning for museum visits rather to more traditional approaches.

Fig. 2.
figure 2

Framework dimensions adopted for developing the first experimental case studies.

Full size image

4 Case Studies

The first comparison of experiences will be based on a visit to an art museum exhibit. A traditional piece of art (a painting) has been selected as the basis for the design, development, and testing of the experimental case studies.

4.1 Content Selected for the Experiment

The painting used for the experimental case studies is Sandro Botticelli’s La Nascita di Venere, located in the Uffizi Galleries in Florence, Italy. This painting was selected because it is a classic artistic representation displayed in an art museum, where the use of XR technologies may be seen as riskier. While multisensory XR technologies are more frequently utilized in scientific or interactive museums where visitors can experience the content by interacting with it, they may not be as widely accepted in art museums where technology may be seen as an obstacle between the visitor and the artifact. However, this technology has the potential to enhance the visitor experience.

The content for the experimental case studies has been chosen and separated into three primary areas: background information about the painting, general knowledge about the elements depicted, and an in-depth analysis of each character.

4.2 Description of the Panel Experience and the Audio-Guide Experience

The contents related to the painting have been collected in a document and used as a reference for the four experiences. During the panel experience, the user can stand in front of the real painting and read the information presented as text written on a small panel near the painting. This way, he/she can observe the painting and read some details about it. The same textual information has been used to produce an audio track, adopted for the audio-guide experience. In this experience, the user can listen to the audio delivered through a smartphone while observing the painting. The same audio track has also been used in AR and VR experiences.

4.3 Description of the AR Experience and the VR Experience

The AR experience is delivered through a smartphone application, used to frame the real painting, as shown in Fig. 3. The smartphone application presents an initial screen where the user can select La Nascita di Venere to start the experience. In the first part of the experience, some images and texts are presented to the user through screen animations, inserted to support the audio narration related to the context of the painting. Then, the external camera of the device is activated, and the user is invited to frame the painting, helped by a reference outline on the screen. In the second part of the experience, some AR elements appear on top of the real painting. To maintain the integrity of the artwork, the AR technology was used in a non-invasive manner, only utilizing lines, arrows, and labels to highlight certain aspects of the painting, such as the names of the depicted characters, and by magnifying specific details. This approach ensures that the technology serves as a supplement to the physical painting, highlighting its features and not replacing it with augmented reality content. The user can adjust the volume of the background music and the voice narration through a menu accessible through a burger menu at any point during the experience.

In the VR experience, the participants find themself inside a virtual recreation of a museum. The real painting is reproduced virtually and presented on a white wall. The experience is delivered using an Oculus Quest headset, as shown in Fig. 3. The VR experience starts automatically and presents the same sequence of images, animations, and texts used for the AR experience. Indeed, the user can view the elements superimposed on the virtual replica of the painting.

Fig. 3.
figure 3

Participants during the AR and VR experiences.

Full size image

4.4 Development of the AR and VR Applications

The AR and VR experience applications have been developed using the Unity3D game engine [26]. The first step of the development regarded preparing the images presented during the experiences as layers on top of the painting. These images were extracted from the original painting using Adobe Photoshop [27] and then imported into Unity. In Unity, images and audio tracks were incorporated, and all animations were produced. Moreover, magnifying visual effects (similar to small stars) have been used to better highlight the characters of the painting described in the last part of the explanation. The AR application was created using the Unity Mars tool [28] and the real painting was utilized as the image marker to trigger the AR content, as shown in Fig. 4.

Regarding the design of the AR application’s Graphical User Interface, neutral colors (black, sand, and cream) and rounded shapes were chosen not to distract the use from the information being presented. For the VR experience, the virtual environment and the painting were created. The environment was designed as a simple room with the painting displayed on one of the white walls. As the last step, both applications were optimised and exported for use on a smartphone and the Meta Quest device.

Fig. 4.
figure 4

Screenshot of the AR application

Full size image

5 Testing

The experiment was conducted in a laboratory setting, where a 60x90cm reproduction of the painting “La Nascita di Venere” was used to simulate a real museum visit during the testing experience.

5.1 Participants and Questionnaires

The study involved 40 participants (21 female) aged between 19 and 58, (M = 28; SD = 5,58) who voluntarily participated in the experiment. They were divided into groups of 10 people to test the four experiences equally. All the participants considered were Italian native speakers since the proposed experiences were in Italian.

The participants’ educational levels varied from high school to PhD. They indicated a technological level comprising the use of PC, tablets, smartwatches, and smart TVs, but no one mentioned using VR headsets. Demographic information and technology proficiency were gathered through a pre-experience survey questionnaire administered to the participants. The questionnaire also investigated familiarity with the painting and reported that 26 out of 40 visited the Uffizi Galleries. Only 10% of the participants considered themselves to be highly knowledgeable about the painting's details. Furthermore, their familiarity with the painting's content was evaluated through a second questionnaire that consisted of specific questions related to the painting. The questionnaire was given to the participants prior to the experience, and it included open-ended questions, such as asking for the names of the characters depicted.

After the experience, the participants were asked to compile two more questionnaires related to UX and engagement and learning performances.

The first questionnaire, previously utilized by the researchers to assess UX and engagement in other studies [29, 30], was developed using similar works found in the literature [31,32,33] and incorporated elements of the Davis’s variation Technology Acceptance Model (TAM) model [34] Regarding the technology and the modalities assessed during the experiment, the questions that aimed to explore the interactivity aspects with the content were removed since, as stated, the experiences did not include any interactive elements. The questionnaire comprises seven sections, mainly regarding quality and engagement of the overall experience, playfulness, self-efficacy, information quality, perceived usefulness, attitude toward use, and behavioral intention. The questionnaire is structured using a 7-point Likert scale, and the overall experience was also evaluated through the use of semantic differential.

The second questionnaire focused on measuring the participants’ learning outcomes. Participants were asked to provide closed responses on the information presented during the experience. There were six questions in total, covering the three aforementioned content topics. The questionnaires were delivered using Google Forms.

5.2 Analysis of the Collected Data

Regarding the questionnaire that assessed UX and engagement during the experience (as depicted in Fig. 5), the AR experience was rated as the most engaging, with an average score of 6.5 out of 7. Meanwhile, the panel experience was rated as the least engaging, with an average score of 4.6 out of 7.

The VR experience has been considered most satisfying in terms of self-efficacy, with an average score of 6.6 out of 7, and making the subjects feel more “safe in using the application autonomously.” The quality of the information was perceived as consistent across the VR, audio-guide, and panel experiences, while the AR experience received an average score of 6.7 out of 7 for information quality.

Fig. 5.
figure 5

Graph showing the UX and engagement results for the four experimental groups.

Full size image

With regards to the questionnaire evaluating learning performance, the comparison analysis showed that the group of participants who underwent the AR experience acquired more knowledge compared to the other groups. Unlike engagement, the panel experience appeared to have a greater impact on learning performance as opposed to the audio-guide. The results have been obtained considering the delta between the pre-experience and the post-experience knowledge. The results indicate that the participants retained 53.6% of the content from the AR experience and 49% of the content from the VR experience (as shown in Fig. 6).

Fig. 6.
figure 6

Results of the evaluation of the learning performance for the four experiences.

Full size image

6 Conclusion

This paper presents a study that tests the hypothesis that using multisensory XR technologies can enhance UX, engagement, and learning outcomes during visits to art museums, when compared to traditional methods. To accomplish this, a framework was created that categorizes various aspects of museum visits. This framework was then used to design four experimental case studies that were carried out in a laboratory setting to evaluate and compare different technologies and modalities for museum visits.

Tests were conducted to evaluate the UX, engagement, and learning performance associated with the experimental case studies. The analysis of the data collected showed that XR technologies, specifically AR, slightly improved user engagement and information retention. In the future, building upon the framework, further experimental case studies will be explored that incorporate different technologies, levels of immersion, sensory stimuli, and interactions.