1 Introduction

Virtual information systems are emerging as vital tools in enhancing the museum visitor experience. Technological advancements in immersive spatial holographic systems can project high definition virtual scenes of historical events with narrative progression to replace the traditional human tour guide model. This approach enriches the visitor experience by engaging and motivating the imagination of the user through interactive storytelling, gaming and learning.

Furthermore, virtual devices have contributed towards enhancing customer experience and sales turnover by offering synergy between the tracked object, person, or place in the space [3, 25]. The public sector and private museum industry have invested heavily in developing immersive applications for their electronic touring systems to increase the number of local and international visitors [20]. The application of Virtual Reality (VR) systems incorporates vital automotive and aviation industries. However, VR applications operate in a single digital perspective which marginalises physical and sociological interplay and heightens the potentiality of dysfunctional behavioural issues. This research explores the development of a Mixed Reality (MR) museum system utilising a Head Mounted Display (HMD) that amalgamates the physical and virtual worlds of the museum environment.

This paper aims to explore the developmental process of a Spatial Holographic Mixed Reality system which incorporates, sociological interactivity. A prototype MR application named the “MuseumEye” will be evaluated to determine its impact and effectiveness in enhancing visitor perception and touring experience. The preliminary research investigation involved conducting a literature review and examining general system observations to build the application blueprint. The evaluation process included a 102-participant survey with semi-structured inquiries to evaluate the impact of the prototype system on the museum experience. The MR system is custom developed and evaluated for use in the Egyptian Museum in Cairo. It incorporates spatial holographic images of 120,000 ancient/pharaonic antiques located in the world’s largest museum [51].

The outcome of the participant evaluation highlighted an intriguing expansion of visitor stay from a maximum of 1 h (during the previous 20 years) to a minimum of 1 h 20 min when deploying the MuseumEye application. This approach incorporates the sociological and technical capacity of MR in shaping and enhancing the museum tour experience.

1.1 Museum guidance

The most significant role in the museum industry is attracting people and enriching their knowledge [18]. Museums employ a diverse range of practical activities to engage the public and museum guidance is the primary contributor [58], This approach is applied using verbal and non-verbal instructions and information to aid visitor interaction [21]. However, the traditional human tour guide system is an organised framework to engage, amuse, educate the visitors with information along a specific route within a museum. This approach is considered an outstanding and enduring feature of museum visitor programme [2]. Furthermore, numerous key authors in the museum guidance emphasise the prominent roles of the modern museum tour guide as the ‘Pathfinder’ which requires to lead visitors around the museum through a pre-planned route [12]. A ‘Mentor’ which provides information for visitors about the site [12]. ‘Leadership’ which ensures the positive and beneficial interaction and also responsible for the social integration in the following group. [9].

Finally, the role of ‘teacher’ is to educate visitors by disseminating information [21]. However, although the human guide is one of the popular guidance methods in museums, it has numerous negative aspects. Mason and McCarthy [41] stated that young audiences consider the human guiding method to be too instructional and educational. The study concludes that younger generations prefer modern interactive methods derived from their evolving personal technologies [2]. Jamison, DeVries [32] considers two situations that indicate the ineffectiveness of the human tour guide approach: if the visitor desires to explore certain places which differ to the tour guide’s route and when the visitor disagrees with the tour guide’s bias towards the museum regarding targeted information. Furthermore, the human guidance system cannot conduct a consistent level of performance due to limited presentations in terms of the context and skills, which would reduce the visitor experience [42]. This method of interpretation affects the information according to the tour guide’s perspectives and beliefs [32].

Therefore, providing an alternative guide system that can overcome the issues highlighted in the literature and further provide entertainment is a viable approach. Furthermore, numerous scholars consider digital multimedia guides can show the ‘On-demand tour guide’ [23, 32] and deliver personalised information as a significant step towards creating a greater effective museum guide experience.

1.2 Evolution of immersive systems in museums

The conventional museum experience relies on displaying artefacts in a particular order to control and display visitor information. These studies indicate that the traditional museum environment does not fully satisfy the expectations of the modern museum visitor [11]. To modernise and enhance the typical museum experience, the exhibits, stories and artefacts are required to be accessible to all types of visitors [2, 29]. Contemporary MR devices can simultaneously take on the role of an educator, entertainer and tour guide, heightening the potential implementation of this technology for museums. Space and capacity availability are essential considerations for museums in order to incorporate the above roles. However, museum space is gradually expanding to include entertainment areas, educational venues, personal guidance tools and gaming areas. Many modern museums now incorporate similar innovations, and visitors have witnessed many technological developments alongside historical museum content. These developments explore essential museum roles to enhance and reify the meaning of the modern museum space.

The Head Mounted Display (HMD) introduced in this research is significant in fulfilling the research objectives as it is highly portable and can display information quickly. HMDs do not distract or restrict the user’s peripheral vision when observing exhibited items like screen based mobile devices. The user can utilise the full scope of the MR environment instead of holding and focusing gaze attention on a mobile screen to observe guided content.

The following literature review comparatively explores the practicalities of implementing VR and MR HMDs in museums. Previous examples of VR HMDs implemented in museums focus on entertaining visitors by unlocking virtual content to prompt interactions. The ‘Meta Museum’ project [40] blends virtual reality with artificial intelligence to produce an interactive educational tool for visitors. Comparatively, the ‘empty museum’ initiative [27] operates outside museums and facilitates the shared experiences of multiple visitors within the same virtual environment. A further study implemented a VR binocular HMD with a scanning projector system to explore museum artefacts [64]. Similar projects employed VR processing to display virtual antiques on wall-mounted screens [41] and visitor guidance displays within a museum [32]. The primary aim of these VR projects is to mediate the transference of information for visitor learning [44]. Comparable studies have explored cost-effective methods of virtual interfacing using cardboard HMDs [48] and Kinect sensors to enhance visitor interactivity and engagement with historical content [56]. A recent project used VR HMDs with low energy Bluetooth beacon to interact with virtual objects in specific areas of a digital museum [65]. These studies contribute to research within a digital museum space and the usage of VR in immersing the visitor in an interactive virtual world. Despite the advantages of VR, it is essential to engage visitors in an actual museum environment to encourage social interactions with other visitors.

Removing the visitor from the real museum environment by adopting VR HMDs restricts the natural freedom of movement to explore detailed artefacts and exhibits. Therefore, this research proposes MR technology as more effective and efficient in enhancing the visitor experience in a real-world museum environment.

2 Related research on AR/MR HMDs in museums

Public acceptance of new technologies such as MR HMDs is of significant concern for museology researchers. Rekimoto [52] highlights this issue in a project using the ‘Sony GlassTron HMD’, Fig. 1a, for scanning fiducial markers that trigger MR interactivity. The device suffered from over complicated and fragile components hardwired into a handheld camera and palmtop display. This construction made the device heavy and uncomfortable to wear while restricting the free flow of natural Kinect movement of the user due to extensive wiring. A similar MR device developed for the ‘ARCHEOGUIDE’ project in the museology field experienced comparable hardware issues. The system required a computer module installed in a backpack with an HMD attached by a wire and earphones, Fig. 2b [61]. Archeoguide used a marker-based tracking technique to configure interactive operations [19]. In the same year, a comparable AR HMD with a mobile power unit and the keyboard came into production. The computer module is harnessed in a backpack and carried by the user with a body motion sensor, MR smart glasses and headphone unit.

Fig. 1
figure 1

a Headset with a Hand-Held device which used by Rekimoto [52] (Left), Fig. 1. b Devices that the visitor should wear at ‘Archeoguide’ [61] (Center), Fig. 1. c Example of wearable device that Damala, Marchal [57]

Fig. 2
figure 2

The process of building ‘MuseumEye’

This system design permitted the user to freely walk and interact in the AR environment without restrictive wiring due to the onboard power supply [57]. In consideration of the free-roaming nature of museum visits, Damala, Marchal [13] validated this design methodology as suitable for open public spaces. Furthermore, the design incorporates sensor inputs and outputs into a single wearable device, Fig. 1c [13]. A project named ‘ARtSENSE’ used AR glasses accompanied by several types of sensors such as biosensors and acoustical sensors to synergise with the user in a more naturalistic manner [14, 16, 54]. The ‘SHAPE’ project aimed to use AR to deliver archaeological information for enhancing the educational and social aspect in the museum experience [24]. Visitors utilising the ‘SHAPE’ system had to wear a Sony Glasstron PLM-S700E (HMD) and simultaneously lift a laptop in order to manage tour guidance and enjoy the virtual experience. An MR project implemented the CyVisor HMD accompanied with a digital workstation and camera to augment 3D models overlaid on a table-top display as part of the virtual museum exhibit [63]. An AR project with a storytelling directive named the SEA CREATURES MR experience involved a see-through video HMD with narrative progression and visual 3D objects to guide the visitor through the storytelling process [31]. The Mixed Reality Agent Guide (Mira) guidance system utilised a combination of HMD with robots in museum rooms [28]. Mira worked by tracking markers within the museum and encouraging visitors to aim AR sensors towards the markers in order to make the system recognise and respond to triggers. A Similar project named ARbInI used an HMD that detects fiducial markers to superimpose images within the user interface [17]. This process requires the user to be in proximity to the tracker; these systematic requirements decreased the natural flow of the museum tour experience [1]. A monocular configurated HMD with hand-held PC tested in a museum as a navigation tool. However, the system neglects essential AR assessability features [47]. A comparable issue in the MR Display Case system neglected key MR technology by using glasses which is compatible with 3D display systems opposed to MR [35]. A system using the HTC Vive and a camera to augment both VR and AR environments into a single perspective lacks mobility and general features [65].

Despite the technological breakthroughs of these studies, there is little research that combines the following features: engaging visitors with exhibited items, mobility and guidance, amalgamating real objects with the virtual overlay, information processing, multi-user experiences, gamification and naturalistic spatial awareness. The MR device ‘HoloLens’, provides the research genuinely created these features and new functions, such as the on-location storytelling, interactive game and museum guide. Museum guidance functionality is the core of this research, and with the help of the devices advanced capabilities, the guide system can fulfil a broader range of visitor desires than other museum guides in Table 1.

Table 1 A comparative study of Projects that used VR, AR and MR HMDs

Table 1 demonstrates a comparative analysis of recent studies that implement HMDs in museum applications. This table includes the ‘MuseumEye’, an MR prototype developed for this study to demonstrate its systematic abilities alongside other comparable systems.

3 Theory of immersion

Immersion theory constitutes the experience of physical interconnectivity between an individual and an encapsulating perceptual stimulus [4, 62]. The physiological feeling of being surrounded by a different reality is similar to the perceptual awareness of experiencing virtual reality [45]. Jennett, Cox [34] demonstrated a conceptual overview and defined immersion as “a gradual, time-based, progressive experience that includes the suppression of all surroundings, together with focused attention and involvement in the sense of being in a virtual world” [46]. Games frequently discuss the immersion experience when playing highly visceral and interactive games [17]. Brown and Cairns [6] investigated the emotive feelings of players during three levels of immersion stimulus: engagement, engrossment and total immersion. Immersion theory is a crucial consideration in shaping the MR museum experience as simulated immersion has the potential lead to enhance learning and interactivity.

Bitgood [3] defines immersion as ‘The degree to which an exhibit effectively involves, absorbs, engrosses, or creates for visitors the experience of a particular time and place”. Player immersion in the visual gaming environment amalgamates perceptual immersion with the narrative progression, adding further depth and meaning to the overall experience [4]. Bitgood [3] determined four factors for constructing immersing experience: use of surrounding physical environment, environmental feedback, multisensory stimulation and object realism. The immersion into the storytelling narrative is a significant factor in enhancing the MR experience; many scholars emphasise the significance of using fictional narratives to generate immersive experience [20, 25, 33, 45]. Contextual narratives are stories that can engage visitors in identifying connections between a story and an object or event [8]. Narrative storytelling is a vehicle for propagating the immersion stimulus as it can transport the mind to another place or time known as transportation theory [26].

3.1 Presence

Immersion theory incorporates the theory of presence. Presence is the maximum sensation of awareness in a simulative environment and feeling cut off from objective reality [34]. Presence is a term used in the virtual reality field to describe illusionary aspects that administer physical reactions in the user [34]. Presence theory typically defined as “an illusionary instant which feels objectively real” [38]. Other scholars reify presence as “a psychological sense of existence in a virtual environment” [55]. The concept of presence in MR is to augment user cognitive perception to feel a different physical location and time-period [49]. Thus, immersive tools and gadgets such as MR HMDs are considered utilities for projecting high-level immersive experiences [6]. Several studies exploited the theory of presence using VR technologies in different domains to achieve their aims. For instance; A 360-degree virtual industrial environment was created to immerse engineering students and to engage them to provide a sense of presence [59]. Also, in the retail industry, a VR experience was created to give the customer a sense of immersion in a virtual supermarket in order to enhance the level of engagement [60]. In this research, the MuseumEye system aims to apply the highest level of immersion by adopting storytelling narratives and changing the visualisation of the physical environment creating a sense of presence during museum tours.

4 MuseumEye system

4.1 Selection of Cairo museum

The motivation behind using the Cairo museum is the strategic significance of this facility in Egypt. The museum houses more than 120,000 antiques from ancient Egypt, making it the most extensive museum collection in the world for pharaonic artefacts [51]. Based on the testimonies of museum curators and guides, visitors do not spend more than 1-h observing and touring the museum’s collections. In order to address this phenomenon, the MuseumEye system theory incorporates a 5-stage research approach, Fig. 2.

The design process started with a historical literature review (Stage 1A) to build a taxonomy of the standard features offered by MR systems employed in museums. A series of observations (Stage 1B) explored the behavioural traits of visitors in selected museum rooms with the most viewed antiques. This process incorporated user needs and requirements to enhance the social experience of museum visitors. The second stage used this information to produce a blueprint of the MuseumEye application and system structure (Stage 2), visual content design (Stage 3), and a development plan (Stage 4). In the final stage, the MuseumEye system was evaluated (Stage 5) using semi-structured surveys to measure the impact of the AR system towards enhancing the traditional museum experience.

4.2 Stage 1: preliminary study

(Stage 1A) literature review

A systematic review explored essential user functions to analyse the role of the tour guide, historical information displays and visitor entertainment. The studies applied in this research were synthesised based on journal publications, conference proceedings which explore AR/ MR reality applications using HMDs in museums.

(Stage 1B) participant observation

The observational analysis revealed the general behavioural patterns of the Egyptian Museum visitors.

  • Pattern 1: Group tours make the visit more exciting and consequently maximise the time spent in front/next to the exhibited item.

  • Pattern 2: Visitors who use guided methods tend to spend more time in front of the exhibited items.

  • Pattern 3: Visitors who are in groups tend to read the content of labels beside the items loudly to other members of the group.

  • Pattern 4: The tendency to take pictures and selfies was evident in many cases, especially in popular rooms.

These behavioural patterns form the foundation of the ‘MuseumEye’ guidance system. Building an MR system using the physical world and virtual objects requires a significant understanding of how the user perceives the amalgamated environment. The MR device is capable of combining the virtual and real worlds seamlessly. Therefore, user functions should operate in this context to reduced user confusion and disorientation. The primary function of the MuseumEye system is to guide visitors, but there are additional roles for system usage. These roles include enticing visitors to walk in a thematic tour, gaining historical knowledge and entertaining people through learning in a comprehensive museum experience.

4.3 Stage 2: MuseumEye structure design

4.3.1 MuseumEye as an immersive design

The MuseumEye application uses the Microsoft HoloLens HMD to deploy the MR guidance system for museums. The MuseumEye application is an MR experience projecting interactive images and characters from ancient times in the museum. Designing virtual characters of important historical peoples and objects overlaid with music and sound effects creates a unique museum experience. The MuseumEye introduced a virtual guide that can walk and speak to the visitor and provide vital visual information in the form of videos, pictures, and 3D scanned antiques. The 3D scanned antique permits visitors a close up detailed look at the object outside of the glass box. This approach allows visitors to observe the antique from different angles using control functions, giving the user a sense of holding the object.

4.3.2 Apparatus

The Microsoft HoloLens utilised in this study fits human ergonomic standards, Fig. 3. The device has a long battery life which enables museum visitors to complete their MR tour duration without significant power loss or system failure. The system is designed to support the visitors touring experience than to distract them from reading written information or viewing real antiques.

Fig. 3
figure 3

Microsoft HoloLens –source: [43]

The Microsoft HoloLensFootnote 1 allows a wider field of view in comparison to similar AR HMDs and projects real-time spatial mapping using a 3D scanner. This feature is significant for this research as during the walkthrough of different locations in the museum requires the AR HMD to adapt to alternating environments. This feature permits multi-user viewing of the same MR environment simultaneously in the same space. The hand gesture controls of the Microsoft HoloLens enable the user to control application functions in a more naturalistic manner than using touchscreen or button triggers.

4.3.3 MuseumEye immersive functions

To fulfil visitor needs and accomplish museum guide objectives, a comprehensive list of functions formulates the application system design. The system functions vary according to their purpose and their classification. The classification structure considers the function’s purpose and the particular action that the visitor will perform while using the system.

Visual communication

It is necessary to achieve direct communication between the visitor’s senses and the system’s visual and acoustic sources as part of the immersive experience. Therefore, a set of functions are designed to enrich the experience with various forms of communication during the tour.

Guidance

A set of functions that involve visual and acoustic signs and cues, which can aid and guide the visitor in order to make the system tour easier.

Interaction

A set of functions that utilise the headset’s hand gestures to interact with spatial visuals. These functions aim to open several ways of interaction between the visitor and the two realms together.

Communication

It is essential to create ways of communication between the visitor and the virtual guide to ease the transfer of knowledge, using facilities such as acoustic and visual clues that can be followed by visitors to get the directions needed.

The following section examines the functions of the MuseumEye application in the Egyptian museum.

  • Function 1: Spatial scenery – Category: Visual communication

The first function comprises of 3D representations of historical scenery, 3D scanned artefacts and animated characters positioned in the virtual environment using the parameters of the real environment.

  • Function 2: Storytelling by virtual guide performance – Category: Guidance

Authentic historical content is narrated, animated and performed by the virtual guide avatar King Tutankhamun, who directs users with hand gestures. The virtual guide is scaled in the MR environment to match the physicalities of a life-size human guide. This process creates a greater naturalistic interaction to converse contextual information. The virtual guide is customisable, permitting the user to configure the audio and visual content.

  • Function 3: Script text – Category: Guidance

Visitors who cannot hear the storytelling narrations of the virtual guide can access subtitles. This function allows visitors to catch up with an ongoing explanation if disrupted and access information displayed on exhibit tags and labels.

  • Function 4: Audio narration - Category: Guidance

Audio narration produced from academic references synchronised with animated displays encapsulates the essence of museum guidance, allowing users to listen and look at the antique simultaneously. This process addresses one of the key user behaviours outlined in the literate review concerning group visitors’ ready labels aloud.

  • Function 5: Air tap/ Hand interactions - Category: Interaction

Interaction by hand gestures such as air tapping is possible in several ways: moving between scenes, revealing item’s images, revealing item’s script text, using the User Interface (UI) navigation buttons, and spinning or rotating the virtual replica of the item. Interactions can boost the level of engagement with visitors. As long as the user keeps interacting with the system, it means the information continues to feed into the user.

  • Function 6: Knowledge scale game - Category: Interaction

The MuseumEye application incorporates an interactive game for discovering further information about specific antiques. Small interactive circles in proximity to the artefact trigger the gameplay in order to promote information retention by the visitor.

  • Function 7: Scene portal points - Category: Guidance

A geolocalisation feature in the HoloLens creates scene portals, which are interactive points placed next to each item registered by the system. When the user focuses sensor registration to a particular artefact, the system prompts the user to move to the next relevant item in that scene. This method allows the system to direct visitor groups in a structured way to stop overcrowding around popular exhibits. However, individual tour customisation allows the user to have the flexibility to access any scene and take random scenarios of the designed tour upon request.

  • Function 8: Take a photo - Category: Communication

An instant photo function operates by saying a specific word to capture and share what the user can see with others. This function is a response to the museum visitors’ performance patterns and the tendency to take photos during tours.

  • Function 9: Collaborative shared experience - Category: Communication

The HoloLens is a collaborative experience, which means all interactions are accessible to co-visitors using the same network connection. This function encourages social interaction and opens prospects for open discussion between visitors. This function considers visitor behavioural patterns and the tendency to walk in close groups.

  • Function 10: Tap to place portals - Category: Interaction

Hand gesture controls permit the user to interact with a scene portal and place information next to relevant antiques. Furthermore, a ‘tap to place’ operation opens a portal in front of a physical item to place the scene at the request of the user.

  • Function 11: Interact with an Antique virtual replica - Category: Interaction

Visitors can manipulate virtual replicas using hand gestures as compensation for handling constraints of authentic antiques. The application supports an interactive feature that explores virtual replicas from different angels to details that are not observable in the real museum. This process gives the visitor a sense of being an archaeologist rather than just an observer.

  • Function 12: User interface (UI) navigation and controls - Category: Interaction

The navigation view is a wide and curved user interface in proximity to the user. The user interface provides the operator with various controls that lead to the growth of the visitor’s interaction skills. It also provides the user with the freedom to enter or leave the scene upon request.

The following section covers the tour design and the walk cycle in the actual room of King Tutankhamun.

4.3.4 Tour design

It is an essential design methodology to develop a coherent tour and storytelling progression in an organised manner. However, for prototyping the system, it is preferable to use a system loop to test and evaluate the design integrity. Figure 4 demonstrates a walk cycle from the start and end. The tour consists of three ‘stations’, marked as red points and nine ‘stops’ marked as black points. Red points represent the ‘stations’ that comprise storytelling interventions along the tour. These stations cover general information about the king himself, his dynasty, who rules the country, his queen, old Egyptian gods and battles. The other scenes marked in black represent the exhibited antique guidance scenes which have acoustic and visual guided methods. The visitor can change the sequence shown in Fig. 4 or jump from one to another, skipping some stations as the content itself is not organised to be dependent on any other scenes.

Fig. 4
figure 4

MuseumEye tour design

4.3.5 Spatial mapping design

To augment visitor’s experience, the physical environment is mapped with the virtual to create a single perspective. This concept was a prerequisite in order to make the visitor convinced of the virtual environment. If the ground is a bit higher than in reality or the walls unevenly positioned, the user may encounter spatial confusion or injury. Therefore, to avoid this issue precise measuring of the ‘The King Tutankhamun section’ using existing plans of the floor blueprint outlined the physical parameters of the room. These dimensions formulate the spatial design of the interior of the room and remap it over the physical room, as depicted in Fig. 5.

Fig. 5
figure 5

MuseumEye mapping design

Figures 4, 5, depict one of the storytelling stations (station 1), and the mapping sequence of the king room. When a visitor triggers ‘station 1’, the scene is generated and mapped on top of the physical environment of the room. The pillars of the temple that the king used to rule the country will replace the room walls, and guards who protect the king will appear at specific spots around the antiques shown in Fig. 5. The empty spaces in the room are utilised fully as the king’s throne is surrounded by ancient Egyptian gods listening to the king’s narration. The red and black dots represent interactive glowing points in the spatial design of the system when triggered the dots initialise and build a mixed reality scene based on the users’ location. Three red dots represent storytelling narrations when triggered the king appears in front of the visitor and starts to tell his life story with floating images to visualise his exploitations.

4.3.6 Application structure and design

The application design focuses on user accessibility and structure consisting of several layers of physical and virtual objects activated by gesture controlled communication and guidance triggers. To avoid operator confusion, the number of visualisations and distance between interactive triggers are fundamental design considerations. The system aims to communicate with the user through three perceptual layers, which are separated spatially (Fig. 6).

Fig. 6
figure 6

MuseumEye structure design

The first layer represents the user interaction controls and the user interface (UI) design. The first layer is in proximity to the visitor for accurate hand gesture control. Performing a click/air tap by Microsoft HoloLens requires three operations: head movement as a pointer, gaze point, and hand gesture. The movement of the visitor’s head up, down, left, and right aims the gaze point to click and activate functions.

4.4 Stage 3: ‘MuseumEye’ visual content structure

The Microsoft HoloLens is capable of processing images, video, 3D audio and 3D holograms that initiate using interactive hand gesture control. Figure 7 demonstrates the HoloLens content design structure built on the concepts of human-centred design.

Fig. 7
figure 7

MuseumEye visual content design

4.4.1 3D content design

Holograms are 3D objects in a virtual scene; it is necessary to build the 3D characters relevant to the context of artefacts inside a specific museum room. The room utilised in the development of the MuseumEye application has the properties of King Tutankhamun [c. 1346–1328 BC] chamber. Accurate character aesthetics is an important design consideration regarding King Tutankhamun and the Egyptian people who lived in this period. 3D design tools Autodesk Maya 2016 and ZBrush produced the 3D models depicted in Fig. 8. The finished models were subject to critique through social media and reviews from anthropologists for recommendations. The 3D software, Marvellous Designer 5, was used to colourise, texture character accessories.

Fig. 8
figure 8

3D visualisation of Egyptian avatars representing – from left – the king, two maids and the queen

Historical references using books, temples and tombs inscriptions formed the virtual character designs, objects and artefacts, these items were showcased to several Egyptologists and experts in archaeology for recommendations and adaptations.

4.4.2 User interface

The concept of User Experience (UX) in spatial designs formulated the MuseumEye accessibility framework. However, a lack of existing knowledge in UX spatial design made it a challenge to design an interface to guarantee effective usability. The HoloLens assigns a limited number of hand gesture controls. To enhance application accessibility, a floating UI and clicking/air tapping controls maximised functional with minimal guidance. User instructions using text and images to aid accessibility is administered to the operator before the tour starts.

4.4.3 Spatial content design

The spatial content design considered ambient audio sources inside the HoloLens application to allocate 3D spatial sounds to the user. This process gives the operator a sense of space and ambience by reinforcing the visual aspects of the application design, creating a greater sense of immersion. The animated characters are designed and positioned within physical boundaries to avoid bottlenecks in the pathways or overcrowding in targeted rooms.

Extensive battle scenes, depicted in Fig. 9, require large open areas to operate efficiently.

Fig. 9
figure 9

Shots from what visitor can see from HMD inside the Egyptian museum in Cairo

4.4.4 Antique’s UI panel

It is vital to satisfy different visitor interests by unlocking various levels of information using images, text and audio narrations. The application content synthesises visual and acoustic information to discover antiques and control virtual handling functions. A 3D scanner named ‘Cubify Sense 3D’ replicated the physical artefacts within the museum to produce 3D virtualisations of actual objects. Refinements to the scanned virtual object in 3D design software assured accurate replication of the original artefact. The object navigation system operates using hand gesture controls allowing the user to spin the scanned objects 360°. Hints and tips appear once the user taps on the navigation controls containing information about the artefact. A game function within the application functions to motivate the user to keep discovering the item and displays the user progress in a bar named ‘Knowledge scale’.

The scale bars increase in size as the visitor uncovers secret information hidden in small tips that trigger around the item. Once the visitor reveals all secret tips, an award sound effect will play, indicating the exploration game has finished. The system presents large text labels to visitors opposed to the traditional small labels on items and displays narration subtitles of the king character, Fig. 10. Floating buttons around antiques in the MR UI help visitors see additional images and content while listening to the narrations. A replay narration button is configured to help visitors replay information. Transfer buttons permit the user to leave the current scene and enter a different one. These buttons were designed and allocated to make it easy for the user to navigate the UI.

Fig. 10
figure 10

Antique’s UI panel with the virtual guide – the avatar of King Tutankhamun

4.5 Stage 4: ‘MuseumEye’ development process

The development process takes into account all previous content design and research studies to build a pipeline for the MuseumEye system.

4.5.1 Development pipeline

Figure 11, demonstrates the development pipeline from the initial storyboard. All content is fed to the game engine ‘Unity3D’ which is responsible for creating the scenes, developing the interactions, integrating the content, and outputting the application to the HoloLens. It is essential to compile the system to prevent lags, errors or bugs to ensure sustainability. The loop of amendments continued until the testing phase ended and proved the validity of the system.

Fig. 11
figure 11

MuseumEye development pipeline

4.5.2 Virtual scenario

The system scenario was designed based on the freedom of change. This process gives the visitor an authority to control and jump from one scene to another. This concept contradicts with the prepared thematic tour that is performed by human or audio guides. When the visitor feels that he/she is the controller of their visit, it increases the level of intention and raises the possibility of learning and enjoying the tour. As depicted in Fig. 12, the intro scene starts, then it takes the visitor to station one where the king introduces himself to the visitor, explaining everything in the context of the room, and then the possibility of choosing the scenes is open to the visitor.

Fig. 12
figure 12

MuseumEye system scenario

4.6 Stage 5: system evaluation

The evaluation methodology utilised for the MuseumEye system is a purposive sample using semi-structured questionnaires of visitors to the Egyptian Museum [7, 50, 53]. Similar studies in the same context adopted qualitative methods for evaluation purposes [15, 36]. The qualitative method was designed carefully to include the research questions would arouse in the system evaluation. Also, this method can reveal the critical responses regards whether the virtual guide would be an alternative for a human tour guide in museums or not. Therefore, the questionnaire examines system functions as a way to interpret the essential roles of guides and what is needed to achieve a better museum experience. According to the museum guidance literature and the system design, the semi-structured questionnaire was built to explore the tour guide roles ‘pathfinder’, ‘mentor’, ‘leader’, and ‘educator’.

Participant recruitment invitations sent through social media invitations to a selected population sampled consisting of frequent visitors to the Egyptian museum ensured quick and relevant responses. An age limitation of 18 to 65 years assured ethical standards and quality assurance when conducting field research. The sampling ensures equality as it divided the experiment into (41.6%) male and (58.4%) female to reduce gender bias. Participants were given a short tutorial on how to use the system and perform the air tapping; then they were showed the tour design in the room and portal points. The time provided for participants to evaluate and experience the system was unlimited – as depicted in Fig. 13. After the system demonstration, participants were provided with a questionnaire incorporating 102 semi-structured surveys and lasted approximately 30 min.

Fig. 13
figure 13

The museum visitors experimenting MuseumEye system

5 Results

A cross analytical examination of the participant questionnaire results was conducted using a The qualitative methodological approach [5]. An inductive coding approach examined participant’s judgemental statements, codes and narratives, which reflected their perceptions and experiences [10].

Results of the questionnaire conform according to Braun and Clarke [5] formula depicted in Fig. 14. As previously explained, the roles of the human tour guide are translated into digital functions. However, the participants’ responses regarding the explored variables -tour guide roles- led to generate themes. Interestingly, other themes that arose during the evaluation are not explicitly around the system functions; they seemed to be conceptual themes. So, the data analysis codes divided the themes into two categories; a) functionality themes which explore a certain function that the system can do in the process of guidance; b) museum experience themes which represent a specific concept that could be mapped to the human guide roles.

Fig. 14
figure 14

The museum visitors experimenting MuseumEye system

5.1 Functionality themes

Mixed reality perception

Sixteen of all participants agreed on this feature, in particular students were most impressed with the surroundings, including the interior graphics and the ambient audios. One of the participants commented “I felt that I would like to be in this world forever, I have not seen the museum room like that and I did not predict the system to portray ancient Egypt as it does” said by a museum curator. Another participant expressed the aspect by saying “I liked being surrounded by worriers as I am in the middle of the battle, it was extremely engaging especially with the music”. Another participant said, “However, I was seeing everyone around me. I felt the system takes me away in another world neglecting people”. Moreover, a participant said an interesting comment about the sense of mixed reality “The most amazing part is being isolated from our world but also still engaged with it, it is a bit of both”. However, a participant criticised the integration of the two worlds by saying “It seemed to be mixed worlds. However, the billers were not exactly aligned with the walls and floor, and the lighting directions make it a bit odd”.

Tour navigation

After minimal instruction, 12 participants claimed to manage the navigation controls between scenes and activate user options. Although the system requires some prior experience and older participant stipulated: “I got used of it very fast, and I did not need any more instructions from the assistant”. Moreover, another participant commented, “It was effortless, and I managed to navigate the room with the system in an attractive way, and I was enjoying the guidance”. However, some of the visitors did not interact with the scene portals correctly and activated them in error due to user inexperience. Another visitor criticised the limitation of the number of the explored antiques by saying “The number of the exploration points is very low, I need to see more and navigate along the room to have more diversity”.

User interaction

System interactions require knowledge of performing air tap hand gestures. Visitors performed interactions efficiently after a practical demonstration. Eighteen participants expressed on this point as one participant said, “I won the game of knowledge scale on the first try it took from me a few seconds then I got how to spin the statue”. Also, another participant said, “The navigation of the statues makes me feel that I was engaged more”. Hand gestures are a new movement to the majority of museum visitors, and some asked for assistance during the tour. A student said, “I could not read the secret hints in the game, it was going so fast, and it should last for seconds so I can read it”. Another participant said, “Swiping and clicking are somehow cumbersome and need more instructions”. Some visitors commented on taping on the floating UI as one visitor commented: “I like the images and scripts buttons but where are the videos and the documentary movies, I believe this system needs more multimedia content”. Another commented, “It took some time to understand how to point and tap on the buttons, I wish I can tap the buttons with my fingers organically without this systematic manner”.

Storytelling narration

Participants watched the virtual guide and listened to stories while noting the MR performance. Twenty-five of the respondents expressed how interested they were in the stories and all participants watching the animation until the end. One participant noted, “It was surprising to see huge horses with its actual size in this room, I was literally at the centre of the battle”. However, some participants wanted the application to cover more content and museum objects stating: “I wish I could see a menu that can list all the museum collections which have 100 antiques”. Another commented on the language “Language is clear and simple”. Moreover, a visitor emphasised on the quantity of the stories “The system needs more animated stories”.

3D multimedia representations

Twenty participants emphasised the similarities in detail between the 3D virtual replica of the actual artefact. However, participants did not use it as a substitute for the original piece, suggesting observation of the virtual and original object at the same time. One participant suggests, “It was beneficial to see the labels in a large size, well written next to the piece. It is cumbersome to read labels on the bottom of the statue in very small font size”. Another participant commented: “I always believed that what I saw today is the future of the museums”. Other participants commented on the quality of the visuals as one said: “I like the 3d characters and the design of the system, and it was very easy”. Another commented, “Images and museum were fascinating”. However, other criticised how the visuals look “some colours and faces are not realistic to the actual characters and the billers were not identical to the real billers”. Another visitor commented, “The character of Tutankhamun was not identical to the authentic one”.

Device’s usability

Ten participants expressed concerns regarding the usability of the devices narrow field of view, stating: “The view is clipped”. Another participant said: “The Field of view is very narrow”. However, some visitors were satisfied with the usability of the device as a visitor commented: “It does not need any time and effort to understand the way it works”. Other comments regarding the device’s ergonomics suggest the device is “A little bit heavy” and “pressing on my nose”. One participant commented on the battery life “I did not finish the experience because the device’s battery was out of energy”.

5.2 Museum experience themes

Educational tool

Five participants registered as school teachers suggested using this system for educating young adults in Egyptian civilisation studies. One participant with experience in enhancing children’s performance suggested utilising the system to train her students on Egyptian themed theatre performance. Another teacher commented, “One visit wearing this device and using the system is more beneficial than hours and hours of school”. Two visitors emphasised on the information they retained from the system as one said “I learned so much today about Kind Tutankhamun” and the other one said, “It was useful, and I retained historical information for the first time”. Moreover, a visitor desires to extend the system in a different language, as he/she commented: “I wish to use AR for kids and we wish to see English and Arabic versions and others from the necessary languages”.

Independent guided tool

Twenty-two of participants agreed on replacing the personal tour guide with MuseumEye application. One of the participants said: “Yes, it could replace a human guide if the system included all the collections in the museum”. Many responses emphasised the personal independence of the tool. As one visitor responded, “I had the free well to go and see wherever I need without assistance from tour guides or curators”. Another visitor commented on the influence of using the system on other activities in the museum as he/she said: “It does not cause a disturbance in the museum”. However, others suggested adding more sources of information to satisfy visitors who seek more in-depth information. One participant suspected the replacement of the human tour guide by this system, as he/she commented: “I hope this tool cannot replace the human guide in Egypt.”

Social interactions

Despite the testing phase, this research study did not include an additional device. However, 11 respondents agreed on the role that MuseumEye can play to encourage social interaction. “It will be pretty awesome if my friends and I can see what I can see,” another participant affirms this opinion, explaining “It is better than VR systems as I can see people around me and I can talk to them while touring on the contrary with VR systems”.

Explore like an archaeologist

Eight participants admired this feature by saying “It was very innovative to spin a statue floating on air; however, it was a bit fast, but it was fascinating”. Another participant commented on the interactive secret points “I cannot imagine another way of showing these interesting points in each statue even the human guide he would point to some points out of sight”. A student in tourism school said, “It seems like our lecturer is showing a piece in his/her hand and explaining to us in our classes”. Another participant stated, “I loved with I could animate the statues out of the glass boxes”.

System entertainment factor

Fifteen participants agreed on the entertainment aspects of the system. However, not all participants found it as entertaining as expected during the preliminary studies. A participant commented “I know it was a game but not as a game as I understand. It needs to be more fun to increase enjoyment”. Another from computer science background said “This is not an actual game like we play. Usually, it might fit the museum guiding, but it needs to be more developed and include scoring and characters”. A teacher said, “I felt if it should fit the kids in the future, it should be more interesting than this game”. However, there are some positive responses suggest “It was useful to have a game on the statue, but I did not see a game word, I realised that without further instructions” and a further positive comment “It was not boring at all, it makes me stay to watch the storytelling graphics more than I expected”.

6 Discussion & conclusion

The MuseumEye application incorporates the theory of immersion and the theory of presence to introduce a new museum visitor experience. The MuseumEye system augments immersive audible storytelling and narrative progression with detailed visualisations inside specific museum rooms. The system positions the visitors in the middle of the story of the pharaohs and engages people with two-ways of interactions. The developmental framework derived from conducting a literature review and analysing previous observational studies to understand the nature of the museum visitors and applied in the application design. System evaluation conducted through prototyping, blueprint, and qualitative surveys permitted system redesign and adjustment according to the recommended amendments of study participants. Participants expressed a high-level immersive effect during their tour, while previous studies neglected the importance and incorporation of immersive system responses.

MuseumEye as a guide system proved to translate traditional tour guide roles and overcome the problems examined in the literature. For instance; ‘tour navigation/pathfinder’ allows visitors to explore independently, as some visitors do not prefer to follow the routes provided by the human tour guide [32]. Storytelling narration/mentor provide interesting and useful information prepared by curators and academics with the sense of immersion integrated with a multimedia presentation to overcome boredom problems that face human guides [41].

Systematic alterations to the navigation system by applying user feedback aligned this function with the previous studies on guidance systems. MuseumEye is a pioneer application in implementing user-interactivity in a spatial UI design. However, the hand gestures assigned to the Microsoft HoloLens are limited, these issues need addressing in future versions of the HoloLens to overcome unnatural control operations.

The MuseumEye application has a high potential to enhance the traditional museum experience and gives visitors the freedom to use cutting edge technology with minimal external instructions. The system encourages social interactivity between visitors while linking HMDs to synchronise group experiences. The virtual guide developed in the research study for storytelling and performance is a step towards replacing the human guide, which is currently the only visitor guidance method in the Egyptian Museum of Cairo. This outcome provides further grounding for the replacement of human tour guides in all museum environments due to the effectiveness of the virtual guide role. The system further indicated a high potential for utilising in educational filed and adapted to include curriculum activities in museums and non-heritage places. The advantage of the MuseumEye application over previous MR museum studies reviewed in this research considers the role of the visitor as an archaeologist. However, participants suggested improvements to the level of amusement in the system with more research in gamification mechanics and game aspects. Also, this system could boost the time people spent in the museum as it shows a significant impact on the engagement level that visitor used to sense usually in the museum. This point has significant implications for the economic state of the tourism sector in Egypt, as more prolonged visitor stays in museums can mean increased revenue. That is due to using the museum facilities, get more amusement, grasp more information and a significant change in the museum experience.

The effectiveness of MR on the aspect of the thematic environment was evident in participant responses. Visitors expressed positively towards the system when visualising the ancient Egyptian civilisation. The adaptability of the MuseumEye system permitted deployment in any museum in the world and integrated effectively into any museum information systems. However, the Microsoft HoloLens is quite expensive, future versions and new emerging MR technologies should consider the cost value to the public to increase accessibility and usability as suggested in market projections [30, 39].

6.1 System limitations

MR museum applications should display spatial visuals to challenge the traditional museum experience. However, the HoloLens struggles to cope with the complexity of overcrowding in halls, rooms and bottleneck paths as this disrupts the program. Lighter and user-friendlier devices such as ODG -7 glasses will allow visitors to see each other’s faces and their expressions. Current MR HMDs do not register facial expressions which restrict organic social interactions and shared experiences between visitors, tour guides and museum management [17, 48].

6.2 Research contributions & conclusion

This study contributes towards a systematic taxonomy AR/MR HMDs used in recent museum developments and published in journal articles (See Table 1). Furthermore, the immersive Spatial Holographic MR System was designed based on the theories of immersion. This study sheds light on user social/behavioural traits as well as the technical features of immersive MR systems. The empirical contributions in this study provide a mixed reality system that can guide, educate, and entertain visitors by presenting visuals spatially. Evidence supports that the system enhanced the visitors touring, and overall museum experience, including increasing the length of visitor stay. The MuseumEye application enables museum management to save the cost of human tour guides and invest further on archiving and entertainment elements of museum management. The MuseumEye system proves its application in museums by introducing a new form of information system design based on the behavioural traits of visitors. This study opens the prospect for mixed reality to invade museums and the cultural heritage sector, and it takes the traditional museum experience to a new level of engagement and interactive experience.