Keywords

1 Introduction

Recently, there has been a great effort in the digitization of Cultural Heritage (CH) to support needs such as conservation of artefacts, modernization of exhibitions, reaching a wider audience, increase visitors’ engagement, etc. First, the immerging novel methods for reconstruction of CH sites and artefacts highlight the importance of rigorous 3D representation of CH [1, 2] not only for the study of the monuments and artefacts but also for their preservation in digitized version. The (re)use of such digitized artefacts are useful for CH institutions to support their preservation, conservation, and restoration work [3] and to communicate their work to the public [4].

CH institutions, through Virtual Museums (VMs) [5] and other digital applications, not only keep in touch and renew the interest of the existing audience but also reach new groups of remote digital audiences. Pandemic conditions boosted CH institutions to keep up, by opening or expanding their digital doors to the public, whereas their physical doors remained closed. The current status indicates that what has started during the pandemic is here to stay and new challenges begin to emerge. Visitors are keen on digital experiences offered by CH institutions around the world, via a variety of technological means and applications. The threshold of the users’ satisfaction from a CH digital experience will keep rising, as more and more people are being familiarized with new media and devices, and seek more meaningful experiences with added cognitive, psychomotor and affective value.

VMs can be either replicas of physical museums or representations of spaces that do not exist in the real world. Three-dimensional (3D) representations that do not correspond to real museums can be created from scratch with any specifications regarding the exhibition’s shell. Moreover, these spaces can host digitized objects from different real-world museums. In this context, there are infinite possibilities in terms of the structure and content of exhibitions that can be created in a digital 3D setting. This allows CH institutions to communicate a story that would require more resources to be told in a physical museum context. Additionally, VMs can host digitized objects, the originals of which are impossible or very difficult to be displayed in a real-world museum, due to safety restrictions (e.g. cannot be exposed due to age/maintenance work, there is no suitable/sufficient space for their exhibition, etc.). In a VM such CH objects are not only displayed in safety but also can be enriched with interactions that strengthen their meaning and make their stories easier and more fun for the visitors to extract and understand. Although it seems ideal to be able to design an exhibition without the physical space limitations, in reality such freedom brings new challenges to the museological design, whereas the shell of the museum needs to be designed as well. In addition to this, more things need to be considered when designing the visitor’s experience in a VM, as the latter comprises not only a museum but also a digital application.

Virtual Reality (VR) brings new possibilities to the VMs, where the level of immersion and sense of presence are increased [6]. Moreover, Head-mounted devices (HMDs) offer the possibility to the users to interact with 3D objects inside the virtual environment e.g. via touch controllers. Hands-on activities overcome the limitation of touching an artefact in a physical museum. The feeling of being part of a photorealistic 3D environment, combined with handling objects activities, can raise the pedagogical value of the whole VM experience especially when they are designed to promote experiential learning [7]. This approach is also aligned with modern museology, which encourages visitors to form their own interpretations about the museum objects [8,9,10]. Furthermore, the visual stimuli from the 3D world can contribute to the user’s understanding of the exhibition’s content [11]. In this context, visual activities can offer the possibility to the users to practice their photo-visual skills and understanding visual messages from artworks [12].

Offering VM experiences that not only represent a typical physical museum visit raises the usefulness of such applications in users’ perception, as well as their satisfaction in terms of entertainment and learning in a VM setting. In the context of finding ways on creating meaningful experiences in a VM that can enhance understanding of visual culture [13], our research focuses on increasing the pedagogical value of such experiences via the use of VR technology. Thus, instead of offering VR experiences that their main strength is the use of VR technology itself to trigger the interest of the audience, we try to reinforce the motivation of the users to explore the studied cultural heritage theme of the application. More specifically, we aim to create not a simple walk-and-watch VR exhibition with simple click-interactions, but a rich-in-stimuli VR exhibition that integrates experiential activities for users who interact with the artefacts in order to promote higher-order cognitive skills [14, 15] and reinforce a positive attitude towards museums and cultural heritage in general.

2 Related Work and Contribution

The VMs developed in recent years highlight the plethora of options available in terms of technologies, navigation, interactions, and scenarios that exist to serve different VM dissemination and communication needs [16, 17]. A well-known service to create a VM is Google Arts & Culture, which allows CH institutions to display their exhibitions digitized with 360° panoramas via Street View tours [18]. The user navigates in a digital replica of a museum and remotely observes the exhibits in their current location. Although 360° panoramas can be used to provide remote cost-effective access to an exhibition, the immersion is relatively low and the user can view the flattened 3D objects only from a single point of view. On the other hand, 3D VMs can offer richer experiences, with exhibitions that usually exist only in the digital world. In a 3D setting, usually created with game engines (e.g. Unity3D), the user can walk through the space and observe a 3D object from all sides. A VM like this allows the user to navigate in a 3D exhibition area and display some information about the exhibits [19].

3D VMs developed in VR, offer an increased level of detail and immersion for the user. In such museums, the visitor usually wears an HMD to navigate and interacts with the CH objects e.g. via touch controllers. In the Anthropology VM [20] an integrated 3D viewer allows the user to observe CH objects with related multimedia content and interact via multiple-choice questions. The Scan4Reco VM offered in desktop and VR, allows the users to inspect CH objects via a 3D viewer, where there is textual information about the object, and research metadata related to specific areas on the object [21]. The user can also change the texture of 3D objects to display different time instances. Another VM hosts the Antikythera Mechanism, an ancient artefact found in the shipwreck of Antikythera sparred in fragments [22]. This VM is a characteristic example of exhibiting a highly fragile CH object in the safety of the virtual environment. The visitors can grab and observe the fragments from a very short eye distance. They can also enter the geometry of each fragment to view gears and inscriptions hidden inside the fragment mass via a CT scans viewer. This is an example of an activity that is impossible or very difficult to be executed in the real world by a wide audience and exploits the possibilities of VR technologies.

More than ten years ago, the benefits of creating dynamic virtual exhibitions had been highlighted [23]. DynaMus is a 3D VM framework that is used to create virtual exhibitions with content from open data web images resources and supports 3D objects display as well [24]. VIRTUE system allows curators to set up VM exhibitions of static and dynamic 2D (paintings, photographs, videos, etc.) and 3D artefacts. Curators can add an unlimited number of rooms, which can be adjusted. Visitors may navigate through the virtual rooms, inspect the artefacts and display information about them [25]. The Invisible Museum platform allows the curators to create virtual exhibitions available in both web 3D and VR [26]. Deep learning mechanisms facilitate the users to present textual narratives based on the socio-historic context of the artworks. Curators create VMs by adding images, videos, and 3D objects in the 3D virtual space.

To sum up, 3D VMs offer a variety of experiences. One can have a quick remote visit to a real-world exhibition via 360° panorama or can visit an exhibition that exists only in a 3D digital form. In 3D virtual exhibitions, the experience of the user usually includes a walk around a room and simple interactions to view textual information. More complicated activities are integrated into VR settings with various interactions and activities according to needs. Finally, the complexity of the exhibitions that can be created with VM authoring tools is increasing.

Furthermore, for the creation of VMs we need to take into account their educational potential [27], as well as the importance of providing meaningful experiences through them [28]. Being inspired by existing VR museums and utilizing the possibilities of VR settings, we suggest an approach that can facilitate the VM users to understand the exhibition’s content and trigger them to get more interested in artefacts, contemporary art, and CH in general. In this study, a set of interaction and activity types is presented that can be used in any VM for 2D and 3D CH objects. Through these interactions and activities, the users can practice their photo-visual skills, meaning their ability to understand visual messages from artworks. We aspire to make this set useful for future VM authoring tools willing to introduce more sophisticated types of activities in their exhibitions and raise the experiences’ educational value, as well as the visitors’ satisfaction in terms of entertainment and learning.

In the context of the “2gether” project, the aim is to establish a technological and conceptual framework that allows a deeper understanding of contemporary art by a wide audience. To this end, a crowdsourcing platform and a VR environment are the main components of the system. We are developing a 3D VM in VR to explore more sophisticated interactions and activities for the users by a) increasing the pedagogical value of such experiences and b) allowing the users to practice their photo-visual skills and their understanding of contemporary art in a VM setting. In addition to the VM, a crowdsourcing web platform is being developed not only for the curators but also for the artists to upload their artworks and create their own 3D virtual exhibitions, which is not described in detail as it is out of the scope of this paper.

This study offers an approach that VM developers and museum experts can use to create meaningful experiences in VR museum settings with high educational value. More specifically, the users can practice their photo-visual skills, increase their knowledge in a deeper level than information retention (understanding) [7] and reinforce their positive perception about museums and cultural heritage in general.

The paper is organized as follows: In Sect. 3, the methodology is described. Next, in Sect. 4, the pedagogical approach is presented including interactivity and activity types that can be used in a VR museum to support learning. In Sect. 5, we describe how this approach is applied to the 2gether virtual exhibition and the VR museum application. Finally, Sect. 6 consists of the discussion and the conclusion of this research.

3 Methodology

After a review on 3D VMs and the interactions they support, we found different techniques that CH institutions use in their sites or social media channels to animate their artefacts to increase their audience’s engagement. Then we classified the techniques that we could apply to 2D and 3D CH digitized objects in a VR environment. Afterwards, a generic exhibition shell was designed to support the user’s explorative behavior [11], together with some activities that can facilitate learning [29] for any kind of exhibit in a VM. In parallel, a museological study was conducted to define the theme of the prototype digital exhibition. At a later stage, our team selected the artworks for the exhibition, defined the learning content and objectives for each artwork [29]. Then we selected the animation techniques that would be better to use for each object and designed the animation/activity scenarios. Based on the scenarios we proceeded to the implementation phase, which is currently in progress.

4 Pedagogical Approach

Initially, the design of the VM space should support the museological goals of the CH institution and be aligned with its educational policy. Our main objective is to encourage the explorative behavior of the visitors inside the digital exhibition and facilitate the users to understand the learning content for the artworks, via various interactions that highlight their concepts.

The suggested types of interactions that can enhance artefacts meanings in a VM setting are presented in Table 1, for 2D and 3D digitized objects. In the first column, one can view the interaction type. The second and the third column show the interactivity and immersion level, respectively. The fourth column presents what needs to be created for each interaction type to work from a technical point of view. The fifth column shows the intended User Experience (UX) for the visitors. In a 2D animation interaction the user views a painting hanging on the wall with a 2D animated texture and interacts with the artwork only to trigger the animation to play. The interaction level is low compared to a hands-on activity, in which the user grabs and manipulates parts of the artwork. Furthermore, in a 3D animation interaction where a 2D painting is being transformed into a 3D scene, the user enters the painting canvas. Thus, the immersion is higher with respect to a 2D animation interaction.

In an experiential activity (hands-on) the user forms a hypothesis on how to complete successfully a task regarding an artefact. Then, the user tests the hypothesis, by interacting with the artefact and receives feedback from the environment. According to the feedback, the user alters the previous hypothesis and tests something else until the riddle is solved [7]. In practice, the user has to execute a task with a specific goal. Once the user completes the task, an animation can be triggered as a reward/ feedback on the successful execution of the task. The task can foster the user to observe structural elements of the artwork, perform synthetic or analytical thinking regarding the artwork, form meanings and understand key messages and visual choices of the artist. The animation that follows the completed task can validate the hypothesis of the user by enhancing the key message derived from the learning objectives set for the specific artwork.

Table 1. Interaction types for 2D and 3D digitized CH assets
Full size table

In addition to the interaction types that can be applied in 2D/3D digitized CH objects in a VM, a set of activities can be applied horizontally for all exhibits to facilitate learning in a VM [29]. Firstly, an embodied virtual agent may present information about the artworks, as it would happen to a real-world exhibition tour. Secondly, a Points of Interest (PoIs) activity can be available for the visitors that want to delve into a deeper level of information about an artefact. When the user is successfully motivated to learn more about an artefact should be able to view more information about it. Specifically, the user can view PoIs on the artwork and select one to learn more about it. This activity resembles the real-world process of pointing at a specific area on the artwork to explain a specific visual detail on a 2D or 3D CH asset e.g. a form/shape/color, a characteristic feature of the artist/artistic movement, a visual choice of the artist, etc. Finally, to strengthen the VM effectiveness in terms of learning, we suggest to set the learning content and learning objectives of the artefacts from the very beginning in order to achieve consistency among the design, production, and assessment phase of the VM [30].

5 The “2Gether” VR Museum

The aforementioned approach was used to create the “2gether” VR museum. The virtual exhibition is about the human body and how is perceived and visualized in contemporary art. We brought together artworks from different geographical regions to exploit the possibilities of the digital exhibition. The virtual exhibition consists of both 2D (e.g. paintings) and 3D (e.g. sculptures) artworks that allow us to try different animation techniques and interactions. The decisive factor for the selection of the artworks was the acquisition of appropriate licenses by the artists or their families. Regarding the exhibition shell, we aim to encourage free exploration and support the positioning of the artefacts in chronological order.

For each artwork, we defined the basic learning concept, with which we deal in interactions and activities e.g. for Edgar Degas’s “Little Dancer of Fourteen Years” we focus on the ballerina’s pose, while in Liubov Popova’s “Portrait study” we focus on the cubo-futurismo artistic expression (see Fig. 1). Then we described the learning content [29] limited in a few sentences: the information one needs to know to understand the basic concept (e.g. main message and/or artist/ artistic movement, socio-historical context, techniques, etc.). Then we defined the learning objectives: what we expect from the visitor to gain in terms of learning through the interactions with the artwork. For example, Aleksandr Rodtchenko’s “Construction on White (Robot)” (see Fig. 1) learning objectives are: the user to recognize the male and female figures; construct part of the human figure out of geometrical shapes; understand that the geometrical visualization of the couple emphasizes their mechanical – robotic relationship. Then we selected the interaction type of Table 1 for each art piece and the specific scenarios for the animations and hands-on activities. To do so we considered a) the learning content and learning objectives; b) the opportunities that the VR environment and various 2D and 3D animation techniques can offer to the artwork, in terms of enhancing its meaning; and c) any limitation immerged from a museological perspective and/or license issues. It was also specified that when the animation of the artwork is playing or when the visitor executes an activity to interact with the artwork, it should be clear that this is an intervention on the artist’s work and not the original art piece. For this reason, visual elements indicate and distinguish the original piece from its interventions, and a relevant informative text shows up at the beginning of the experience. Finally, we also included a short description of how we expect the user to approach the learning content through the interactions and activities.

We also suggest a PoIs activity that will be available for all artworks to facilitate the visitors to focus on visual elements on the art pieces and form related meanings. These PoIs have been defined for each artwork together with the preliminary information that the user will be able to view in each PoI (see Fig. 2). Another activity available for all artworks is the embodied virtual guide who introduces basic information about the artworks to the visitor (see Fig. 3).

Fig. 1.
figure 1

© Vassilis and Eliza Goulandris Foundation Collection, b) Liubov Popova: Portrait study © Metropolitan Organisation of Museums of Visual Arts of Thessaloniki (MOMus)Footnote

https://www.momus.gr/en.

, c) Aleksandr Rodtchenko: Construction on White (Robot) © MOMus.

Images of a) Edgar Degas: Little Dancer of Fourteen Years

Full size image
Fig. 2.
figure 2

© MOMus.

Visualization of PoI activity on a 3D digitized asset (mockup). The user selects a PoI on the digitized asset to view more information. Artwork: Alex Mylona: Angel II

Full size image
Fig. 3.
figure 3

© MOMus.

The embodied guide presenting a 2D digitized asset. Artwork: Aleksandr Rodtchenko: Construction on White (Robot)

Full size image

Until now, most of the artworks have been digitized with photogrammetry and/or polygon modeling to be integrated into the game engine. The VM is a work in progress and is being developed with Unity High-Definition Render Pipeline (HDRP) to pursue high graphical realism. The VM will be available in VR with HMD (Oculus Quest) and touch controllers. To increase the learnability and usability of the system, we designed a tutorial at the beginning of the VM experience to familiarize the user with the system. We intend to reduce the cognitive load needed to explore the VM and make easier to focus on the CH content and the tasks.

6 Discussion and Conclusions

3D VMs offer a variety of experiences and can meet various CH institutions and audiences’ needs. From 360° panoramas to 3D digital exhibitions and VMs derived from authoring tools, there are various ways for the users to interact with the digitized CH assets, which are usually limited to displaying textual information. On the other hand, 3D VR museums offer more complicated interactions that can have additional educational value with hands-on activities and rich stimuli. We can utilize the possibilities of the VR technologies to create VMs with more sophisticated interactions and activities that a) are not possible to be executed in a physical museum (touch CH objects, interact, experiential activities, etc.), and b) enhance learning aspects of the whole VM experience (e.g. knowledge: understanding of the CH contents, affective: increase engagement, trigger/retain/increase motivation to explore more about the CH content, skills: photo-visual, analysis, synthesis, form own interpretations, etc.).

We presented a set of interaction and activity types that can be applied in 2D and 3D digitized assets in VMs. Our pedagogical approach focuses on experiential learning and visual stimuli to understand art pieces and highlights the importance of defining the learning content and objectives for the artworks scenarios. This approach aims to facilitate users to increase their knowledge regarding the content of a specific VM in a substantial way: information retention level, understanding and beyond. In parallel, the goal is the users to practice their photo-visual skills, which are useful for any museum setting, digital or physical, so as to transfer their knowledge outside our application. We also aspire to strengthen visitors’ confidence in forming their interpretations about the art pieces and increase their interest in the artworks, contemporary art and CH in general. The assessment of user experience will provide us with insights about the usefulness and appropriateness of the interactions and activities, as well as the effectiveness of the whole experience, in terms of user satisfaction, entertainment and learning (skills, knowledge, attitude). Finally, in our future work we foresee incorporating a classification of sophisticated interactions and activities in VM authoring tools that already tend to offer more complicated and adjustable virtual exhibitions.