Keywords

1 Introduction

The MOL is an ancient mnemonic that works by combining visual and spatial cues to aid memorization. The method is highly effective [1, 12], and is used by champions in memory competitions worldwide. Watlin (2009) defines the MOL as “a memory strategy in which items to be learned are associated with a series of physical locations” [18, p. 489]. To apply the technique, one must choose a location (locus, pl. loci) to act as the Memory Palace. Within this palace one isolates a travel route: a fixed set of sub-locations in a certain order (e.g., your hallway, bathroom, and living room, etc.). Further, one visualizes the memory items one would like to recall at the different loci (e.g. a milk carton in the bathroom sink). To reinforce or recall the memory items, one mentally revisits the Memory Palace.

This combination of spatial and visual cues is in theory a great match for the medium of Immersive VR. By creating a Virtual Environment (VE) and enabling the user to place objects within this environment, the processes of the MOL can be simulated in VR. Potential benefits could include ease in performing the MOL and greater success in the memorization. As the medium of Immersive VR can provide an actual visual and spatial environment, rather than an imaginary one as traditionally used with the MOL, the visuo-spatial impression might prove to be powerful in terms of memory.

The implementation of the MOL to VR can provide several benefits. Potential applications areas include students’ learning, such as recalling content material they are learning, but can also exist within healthcare. The MOL itself has previously been used to aid several kinds of memory-impaired patients, with some success [13, 20]. The usefulness of the MOL, however, would be limited by the amount of time needed to train individual patients [13]. For this purpose an application that aids the user may be more adequate than the MOL alone. As the application simulates both the spatial environment and the visual cues, it may be easier to deploy a method which otherwise is complicated to perform on ones own, especially for elderly with reduced cognitive abilities.

In this paper, we review related work, describe the Mnemosyne application, the pilot study and results, and our improvements for a larger study.

2 Related Work

Several researchers have previously combined the MOL with VEs. Most related works provide the user with a VE through a screen, and the user performs the conventional MOL by mentally visualizing the memory items. Thus, the applications only provide the location for a Memory Palace, and the user has to perform the method without any aid from the system. Examples include the work by Legge et al. (2012) who compared two groups: one in which the participants used a virtual location presented on a screen, and another group which used a known physical location [8]. They found the performance of the two groups to be equivalent, a promising result for the use of VEs with the MOL. Another work is presented by Huttner and Robra-Bissantz (2017), who presented VEs to two groups: one using a laptop, and the other using an HMD. They found that the VR group outperformed the laptop group in terms of accuracy [5]. Although the applications in these studies did not let the user visualize and place memory objects within the VE, the results indicate that VEs fruitfully can be used as a basis on which to employ the MOL, and that an HMD is an effective way of displaying the Memory Palace.

Some work, however, has also been done where the MOL is performed through the application. Jund et al. (2016) simulated the MOL in a “semi-immersive” VE using a 3D monitor with 3D glasses [6]. The aim of their study was to compare the difference between an allocentric and an egocentric view, to evaluate which offered the best spatial cues to aid memorization. They found the results of the egocentric reference to be better than the allocentric. Similar to Jund et al.’s study, is the work presented by Fassbender and Heiden (2006) who created a Virtual Memory Palace for a desktop PC where the users could navigate through the palace and place images locally stored on the computer [2]. The results of their scoping study indicated a benefit for long term memory with the Virtual Memory Palace. Both the work by Jund et al. and Fassbender and Heiden, however, was done in a desktop VR scenario, and did not offer the natural orientation within the VE that head tracking with an HMD provides. This kind of orientation is important for spatial learning, as it is how we navigate and remember locations in real life. Utilizing natural orientation as a means to support spatial learning, is elementary to support the MOL. As the basic concept of the MOL is that if you remember the place (i.e. the Memory Palace), you remember the content, spatial learning may prove to be an essential part of the MOL which again could be supported by natural orientation.

Most similar to our work is the work by Mann et al. [9]. Their VE is delivered through a CAVE system, a series of wall projectors encompassing the user in the VE. They compared memory scores between the immersive group and three other groups: (1) using their usual method of memorizing, (2) the normal MOL, and (3) a desktop VR group. Although the Immersive VR group were most successful, the results were not statistically significant, and ceiling effect and uneven sample size made further analysis difficult [9]. Our work is differentiated from that of Mann et al. as we employ an HMD as the VR technology for executing the method. In short, our work is distinguished from the related works by performing the MOL through the application and visualizing the memory items within the VE in Immersive VR through an HMD.

By employing Immersive VR for our application, the user will experience the VE from a First-Person View, a feature that contributes to learning in VR [11]. Moreover, the application will allow the users to experience the spatial environment as we experience spatial environments in real life. As studies on spatial learning in VR and real life deliver similar results [4, 15], the application may allow for exploiting the way spatial information is seen and learned in the most basic sense. There is evidence suggesting that spatial knowledge from virtual environments are transferable to the real world [17, 19]. This is essential to the concept of the MOL, as the content to be recalled is embedded visually and spatially in the environment.

The hypothesis underlying this study is that the immersion, and the potential succeeding feeling of presence, will be of importance for the utilization of the visuo-spatial Memory Palace for memory recall purposes. The methodology of the pilot study which this paper presents, aims to isolate the effects of these different levels of immersion and presence and relate it to the memory scores in the experiment. For this purpose, the Mnemosyne application was created both for Immersive VR and Desktop VR, to be able to differentiate between these levels of immersion.

3 Application

Vindenes (2017) presents the “Mnemosyne” application [16]. The application was designed through a Research Through Design process, to be an “integration of many technical research contributions from a variety of disciplines into a single working system” [21, p. 498]. The design incorporates elements from research on Virtual Reality Learning Environments (VRLEs), and psychology research on cognitive load, spatial cognition, memory and learning. Effectively, this means that the design of the application is inspired by research on the important role of spatial navigation and visual impressions on memory. In addition to this, research on cognitive load and learning are taken into consideration. In this way, the Mnemosyne application functions as an “embodiment of theory and technical opportunities” [21, p. 498], and can be used to investigate the effects of the visuo-spatial dominant technology of VR on memory. The application will be used to investigate the hypothesis that Immersive Virtual Reality supports the visuo-spatial elements of the MOL. By comparing a group using the application in Immersive VR to other approaches, we will try and isolate to which degree immersiveness aids memorization.

3.1 Application Requirements

The system requirements for Mnemosyne were based on (1) a literature review, and (2) requirements set to simulate the mechanics of the MOL. From the literature review, three distinct requirements were identified. First, the users should be able to play ‘themselves’ from a FPV to get a first order experience [11], as opposed to controlling a character in a third-person view. Second, the design should allow the users to be autonomous, free to move where they want in the environment [3, 11], as opposed to navigation along a fixed route (carousel navigation). Third, the design of the rooms should be simplistic. This latter requirement draws on research on VR and cognitive load that suggests being careful and selective with content in the VEs so as to not overfill the environment [7]. Other requirements were set by the nature of the MOL itself. Requirements identified by the mechanics of the MOL state that a Memory Palace is needed, i.e. a three-dimensional VE. It has to be possible to navigate and orient oneself within this environment through an HMD. It must also be possible to instantiate and place objects, i.e. images, to any given association one might have. In this way, the MOL can be simulated, and the VE populated so it “becomes the physical representation of the knowledge to be taught” [14, p. 345]. According to Sanchez, Barreiro and Maojo (2000), this is the characteristic which distinguishes VR as an educational technology: “the possibility of creating symbolic spaces capable of embodying knowledge” [14, p. 360]. When a user has filled the Virtual Memory Palace with the memory objects, that instance of the VE will act as a virtual representation of the information he or she wants to recall.

3.2 Application Design

Mnemosyne was created with the A-Frame Web API for the Samsung GEAR VR HMD and displayed in the Carmel Developer browser. In the HMD, the Samsung S7 phone is used as the screen, while the GEAR VR provide lenses, a touchpad and a more precise internal measurement unit than what is present in the smartphone. The Memory Palace is presented as a building comprising five different rooms, and three memory items can be placed in each room (see Fig. 1). Thus, the application can be used to memorize a total of 15 items. The items to be memorized are entered into a smartphone application on the Samsung S7 as text keywords before the smartphone is inserted into the HMD. To support the ability to visualize almost any memory object, these keywords are further connected to a Web Image Search API that returns images the user can place within the VE. In this way, the user can enter any keyword he or she wants to memorize into the application. To be viewable from all sides within the 3D VE, these images are attached to the sides of a cube, creating a “memory cube” (see Fig. 2). The placement of memory cubes is performed by gaze interaction, by looking at blank memory cubes at pre-defined locations (see Fig. 1). Within the VE, the user has free navigation, and moves in the direction of the gaze by pressing the touchpad on the side of the HMD. The memory cubes are placed in the order that they were indexed in the word list. When the user has populated the Memory Palace, the environment is free to be navigated and inspected for memory reinforcement.

Fig. 1.
figure 1

Allocentric view

Full size image
Fig. 2.
figure 2

Two memory cubes

Full size image

4 Pilot Study

To evaluate the application and research methodology before a full-scale study, we conducted a pilot study with 18 participants. These participants were divided into three groups of six participants each. The participants were university students, equally distributed in gender, aged between 20 and 32, with a mean age of 25 years. The students were recruited from a variety of fields, e.g. anthropology, media studies, pedagogy, information science, marine biology, etc. Information science had most representatives, with 6 out of 18 participants. Group 1 used the application in Immersive VR; group 2 used the application in Desktop VR, and group 3 used the MOL without any technological aids (i.e., the conventional MOL). The aim of the research design was to be able to compare results from the Immersive VR group to the desktop VR group, and from the desktop VR group to the conventional MOL-group. The reason for the three groups was to isolate which effects came from being immersed in the VE, which effects came just from a visualization, and which effects simply came from the MOL itself.

As spatial learning is important in the context of the MOL, each participant across all groups were required to take a test of their spatial reasoning abilities. The spatial test consisted of five “mental rotation” tasks. Each correct answer would give a score of 1, the max score in the spatial ability test being 5. When this test was completed, the participants were taught the concept of the MOL and were informed on how to execute the method. The VR groups (i.e., group 1 & 2) were also exposed to the VE beforehand, to ensure that as little time as possible was lost in the interaction with the application. In this training phase they were given instructions on how to navigate and perform the placement of the memory cubes. While Group 1 & 2 was training in employing the MOL in VR, Group 3 was trained in using the MOL without technological aids. After the experiment, the participants answered a questionnaire, and participated in a short interview, which both addressed their experience of performing the MOL. The steps of the experiment for all three groups can be summarized as such: (1) Spatial task, (2) Training, (3) Memory task, (4) Interview & Survey.

5 Results

The group who used the conventional MOL outperformed both the VR groups in the memory experiment. More specifically, the Immersive VR group recalled on average 13.1 items, the Desktop VR group recalled 14.3 items, and the group using the conventional MOL recalled all 15 items. As for long term memory, similar results were found one week after the exposure to various experimental conditions (Immersive VR: 10.6, Desktop VR: 12, Conventional MOL: 13.8). On first glance, these results suggest that the MOL does not benefit in terms of effectiveness when combined or adapted to technological aids. We explored this further.

5.1 Spatial Ability

Further examination of the results, however, reveals other differences between the participants than just simply the mediums used. The spatial ability for instance, varied significantly between the groups. The conventional MOL group scored the highest with 3.8 points on average, the Desktop VR group scored 3.3, while the Immersive VR group had an average score of 1.8. There is thus a correlation between high spatial ability scores and high memory scores. To further examine this, we examined the relationship between spatial ability and memory scores between all participants regardless of groups. We found that those participants who had a spatial ability score of 1 remembered on average 12.25 items, while those who had a spatial ability score of 2, remembered on average 13.5 items. Those who had a spatial ability score of 3, remembered on average 14.8 items, while all of the participants who had a spatial ability score of 4 or 5, remembered 15 items.

These results support the ability-as-enhancer hypothesis as presented by Mayer and Sims, which states that high spatial ability learners should benefit more from 3D environments as they would use less cognitive effort [10]. This then allows more cognitive power for mental model construction. By this hypothesis, low spatial ability learners would not benefit from VR support in the performing of the MOL, as this would cause them to experience cognitive overload more often than the others. This theory may help to explain the low scores of the Immersive VR group.

5.2 Interaction Time

The Immersive VR group also had a higher interaction time, or method performing time, than the other groups. Having to navigate within the spatial environment “physically”, had a disadvantage in terms of time versus only performing it mentally. This makes it harder to do time-based evaluations of the effectiveness of Virtual Memory Palaces. Two out of the six participants in the Immersive VR group reported stress due to time constraints in execution of the MOL, whereas none of the participants in the other groups reported such. The number of participants who were “finished” and thus had more time to walk around to memorize the items was also far higher in the Desktop VR group and the group using the conventional MOL than in the Immersive VR group.

6 Discussion

The medium of Immersive VR provides a promising venue for new visual information environments. By designing applications that exploit our brains visuo-spatial memory schema creation, the medium could allow us to create VEs filled with the content we want to recall or learn. The results from this pilot study, however, points out that this might not equally benefit everyone. The results indicate that low spatial learners benefit less from the MOL and using the MOL in a Virtual Memory Palace. These results should be controlled in future studies with larger sample size and using statistical analysis. For example, future studies on VRLEs should evaluate the impact of spatial ability on the outcomes of the learning content. If high spatial ability is a prerequisite for benefiting from VRLEs due to cognitive load, this may impact how VRLEs should be incorporated and for whom.

Although the study shows correlations between Immersive VR and low memory scores, it is questionable whether it is the features of Immersive VR causing these results, or if it is the factors following the Immersive VR group. Because of the disadvantages of the VR groups in terms of interaction time and lower spatial abilities of the participants, this pilot study cannot with certainty comment on the effectiveness of Virtual Memory Palaces. To be able to evaluate the effectiveness of the MOL with control groups, the interaction and navigation time could be eliminated so that valuable memorization time is not lost in the interaction with the system. This could be addressed with automatic rather manual placement of memory cubes. If a user is presented to an already-filled Memory Palace, the user could focus on just navigating through the different rooms. The interaction, however, may be important, and future studies would be needed to evaluate this. As spatial ability scores may affect the execution of the MOL, participants should also be placed in groups based on their spatial ability skills.

7 Conclusions

This study presented “Mnemosyne”, an Immersive VR application simulating the MOL. A pilot study was conducted with the application, to evaluate the effects of the application on memory, and the research design of the study. Time lost in navigation with the VR interface and low spatial ability of the Immersive VR group makes it hard to conclude on the effects of the application on memory. The outcomes of this pilot study indicate that spatial ability was important for the effect of the MOL across all groups. For our future full-scale experiment, the implications of this study will be taken into consideration. In the next study, with an increased sample size, the application will be modified to reduce interaction time, and the groups will be sorted based on pre-tested spatial abilities. To avoid the ceiling effect, the number of items, and therefore also the size of the memory palace will be increased. The full-scale study will also gather more information about the participants, such as their subjective feeling of presence, as this is related to the varying degree of immersiveness. By employing statistical analysis on the results, we hope that the full-scale study will be able to isolate which effects different degrees of immersion and presence can have on their memory scores.