1 Introduction

Since the last decade of the twentieth century, computer-based virtual reality worlds have been a key research topic, especially in the entertainment industry. For 25 years, virtual reality (VR) was displayed on a PC screen with user interaction controlled by a mouse. Despite the immense possibilities of those environments, user immersion was low. However, over recent years, new hardware and software developments for virtual reality environments (VREs), such as head-mounted displays (HMD) Oculus Rift™ and Unreal Engine 4™ Game Engine, have opened the door to higher levels of user immersion in gaming, entertainment and even education. But the use of VREs for teaching purposes is still an open issue. Although it has been established in various studies that VREs increase student receptivity and learning rates (Chen et al. 2013; Roussou and Slater 2017), especially among young students (Bustillo et al. 2015; Roussou and Slater 2017), while the conclusions of other studies have suggested that they increase the presence of students, but reduce the learning rates (Makransky et al. 2019). The complexity of the human learning processes explains these contradictory results that also point to the need for further research, to identify the best way of creating high-immersion VREs for teaching purposes. In any case, one advantage that should be taken into account is the general consensus that VREs enhance both the participation and the involvement of students in learning activities. This observation has been noted in all types of education and at all levels, including higher education (Alhalabi 2016; Muller et al. 2017), primary and secondary education (Passig et al. 2016) and professional training (Chittaro and Buttussi 2015; Webster 2016).

However, beyond any agreement over the potential of VREs in education, there are many open questions about the best implementation of VREs in educational environments. Some of these questions refer to the technical issues of the implementation relating to Motion Sickness in VR (Polcar and Horejsi 2015). Other questions refer to the educational strategies: the best topics or concepts to be taught with VREs (Freina and Ott 2015), and to a high sense of presence, which can be highly motivating for students and can influence their processing of educational material (Lee et al. 2010). In this research, we seek to cast light on some of these questions by evaluating learning outcomes of a different nature in a high-immersion VRE and by comparing them with the learning outcomes of a more traditional teaching methodology, in this case watching an educational video.

History is one of the most promising knowledge areas for high-immersion VREs. Most historical sites have undergone definitive changes or no longer survive. If students feel involved in the virtual reconstruction of an historical site, then immersive VR is expected to enhance learning rates. Many different aspects of historical sites may be learnt: historical data (Kiourt et al. 2018), urban layout (Laurent et al. 2018), ways of life (De Paolis 2013), traditions (Chen et al. 2018), etc. The conclusions of those works suggest that educational goals play a central role in the definition of the VRE.

There are two steps to the development of a VRE: first, the creation of the 3D models and second, integration of the models in a game engine. There are various approaches toward the creation of 3D reconstructions that feature cultural heritage, which are linked to the historical object and its accessibility (Lucet 2009). Topographic techniques with Geodesic Stations, laser scanning and photogrammetry are preferred for large-scale monuments that have a complex geometry (Remondino et al. 2017). Whenever the built heritage is partially or completely ruined, then the above approaches are useless and CAD tools take their place (Bustillo et al. 2015; De Paolis 2013). The application of low-complex 3D models for VRE is essential to productive teaching skills, as the scope of a 3D model will clearly restrict end-user interaction with the virtual reconstruction (Lucet 2009). The second step, as mentioned, consists of the integration of these models in a game engine and the creation of the VRE. Over the past 20 years, different game engines have created virtual environments. EON studio and XVRtechnology were the reference with the first generation of VREs, e.g., in virtual museums (Loizides et al. 2014) and educational video games in VR on medicine in the Middle Ages (Lorenzini et al. 2015). Over the past few years, the fast development of computer performances (particularly of graphics cards) has allowed the representation of more realistic VREs and new game engines have assumed leading positions. Unity is the most widely used, thanks to its flexibility and ability to adjust to all types of projects. As a contender, Unreal Engine 4 is a powerful game engine that produces high-quality interactive VREs and provides reliable support for virtual reality devices.

In this paper, a teaching experience is reported for undergraduate students that employ a VRE displaying a 3D environment through Oculus Rift™ with Unreal Engine VR tools. A 3D model of Briviesca at the start of the fifteenth century (Alaguero et al. 2015) was used in the VRE. The first version of the 3D model was designed with off-line rendering and video creation in mind. It was then implemented in a VRE (Checa et al. 2016). The teaching experience narrates the establishment and growth, over the fourteenth and fifteenth centuries, of the population center known as Briviesca (Spain). This paper reflects on the strengths and the weaknesses of the VRE in relation to teaching historical knowledge and Medieval urban layout. It also compares cultural heritage teaching through both the VRE and videos. Most other works published on cultural heritage VREs have mainly focused on the hardware and software that are required to create 3D immersive environments (Andreoli et al. 2016; Carrozzino and Bergamasco 2010; De Paolis 2013; Lucet 2009). Only a few research papers have evaluated their effects on the final users (Bustillo et al. 2015; Champion 2008). In this research, we take previous studies one step further by evaluating the different kinds of knowledge that may be acquired: explicit, implicit and spatial. Besides, this research goes a step further than a first version of the VRE teaching experience in the Middle-Ages Briviesca already cited (Checa et al. 2016): It improves the major limitations of this teaching experience that could only be considered a prototype, and it is tested with a larger number of students, producing significant conclusions from the statistical point of view. These major limitations were related to (1) a very small sample of end users without any statistical value; (2) the use of first-generation HMDs with strong motion sickness effects; (3) a low VRE resolution and visual quality; (4) the inclusion of a new procedure to measure the capabilities of the students at spatial identification of the locations of the main buildings of the city; and, (5) the improvement of the first questionnaire that includes questions with 100% right answers (because these kinds of questions will not help to detect advantages or disadvantages between the different learning methodologies).

The town of Briviesca toward the end of the Middle Ages was selected for this teaching experience due to its very unusual characteristics. It is a town in the north of Spain. Historical records tell us that it was located on a hill in the thirteenth century. At the beginning of the fourteenth century, Doña Blanca de Portugal acquired a group of small hamlets close to the site of Briviesca. Known as the founder of the town, she chose to imitate the Roman urban layout, which was quite rare in Medieval town planning in the Iberian Peninsula. It is an uncommon urban layout that adds special interest to historical explanations of the development of Briviesca and its daily life toward the end of the Middle Ages.

2 Methods

Figure 1 shows the methodology followed in this work. It begins with the creation of the VRE. Although this investigation is not focused on the technical procedure to create the VRE of Briviesca in the fifteenth century, a summary of this procedure is required to analyze its potential for teaching purposes. The detailed procedure to create the VRE has already been presented in an earlier work (Alaguero et al. 2015; Checa et al. 2016).

Fig. 1
figure 1

Scheme of the proposed teaching activity

Full size image

The VRE of Briviesca includes around 300 ordinary buildings and over 20 singular buildings, based on historical–archaeological research. The ordinary buildings include dwellings, outhouses, cowsheds and sheep folds. Among the singular buildings are two churches, one fortress, some houses of the nobility and the town hall. Besides, modeling also extends to signposts, shrubs, wells and fences. The 3D model consisted of 1771,882 triangles. On average, a normal house had a total of 3000 triangles and the singular buildings such as the Collegiate of Saint Mary and the church of Saint Martin (89,000 triangles and 31,000 triangles, respectively) required many more polygons. This size of the 3D models was suitable for both off-line and online rendering. All the 3D models were created with a 3D modeling software called Blender, an open-source software with a GNU GP License. Some tasks such as the creation of collision meshes had to be completed, before the integration of the 3D online rendering model into the VR Game Engine (Checa et al. 2016). Finally, the lighting and the user interaction with the VRE procedures were tested. At this stage, some users suffered from motion sickness while testing the VRE, as other researchers have previously experienced beforehand (Hupont et al. 2015). To remedy those side effects, the VR experience was first set up at a continuous frame rate of 90 fps and then different user locomotion systems were tested to circumvent the problem. First, a simple gamepad locomotion system was designed, as in traditional videogames, with a joystick to walk around the environment. Then, an alternative locomotion system based on teleporting was also designed. With this second system, the user can stroll around the environment, but when wishing to move beyond the boundaries of the physical space (more than a couple of meters), Oculus Touch controllers can be used for teleportation to a new area within the visual range of the user. From this new location, the user can continue to stroll around the town or teleport to a new location. The HMD hardware was Oculus Rift. The computer had the following specifications: IntelR Core™ i7-4790 CPU 3.60 GHz, 32 GB RAM and a NVIDIA Titan Xp graphic card.

Validation of the teaching experience with undergraduate students followed twin strategies. Firstly, it was intended to detect the strengths and the weaknesses of the two teaching methodologies:

  • A video (Alaguero 2015) presenting Briviesca in the fifteenth century using off-line renders of the 3D model. The duration of this video is 13 min. The screening of this video was followed-up by a short discussion with the students that enlarged the video experience by up to half an hour. This teaching strategy will be referred to as the “video” in following Sections.

  • A guided VRE tour of Briviesca by means of Oculus Rift HMD that displays brief video clips. These shorter videos of 1-to-2-min duration represent seven sections of the main video that had previously been presented. The sections are placed at seven locations in the VRE that match their content. Students watch the videos on a giant screen in front of them when they arrive at each location. The videos activate automatically when the player reaches the point, and any movement during the video is blocked. The students can move through the VRE using two different locomotion systems: gamepad locomotion (“VR Gamepad Locomotion” in the following sections) and roomscale + teleporting locomotion (“VR Teleporting” in the following sections). The students were instructed to follow a path of grass and flowers on the ground that guided them through the environment. The position of the videos was visually marked by a sign, and they played automatically when the user entered a delimited area of action. Each virtual tour lasted approximately 30 min, on average, although the VR Gamepad Locomotion required an average of 32.5 min, in comparison with the VR Teleporting group that required 24.6 min on average. Although these experiences may be longer than the video experience, the students in the VR environments spent part of their time in the VRE to get used to the motion mechanic and the immersion process, thereby reducing useful exposure time to the knowledge that was subsequently tested, leading us to conclude that all the students were exposed to the knowledge that was subsequently tested for a roughly similar length of time.

  • The position of each milestone and the itinerary of the student in the VRE are shown in Fig. 2.

    Fig. 2
    figure 2

    Locations of video clips and pathways through the virtual environment and images of both the virtual environment and the actual site

    Full size image

It is worth outlining that all the students had some rough contact with a map of the Medieval city of Briviesca, as some of the experience learning indicators were evaluated on a city map. The VR groups saw that map and their position once they arrived at each of the seven milestones where they could watch the videos clips. The video group saw the map in some frames where there was detailed information on both the layout of the city and the locations of the main buildings. Neither group therefore had the opportunity to examine the map very closely. It was an intentional design decision for the teaching experiences, because continuous exposure to a 2D representation of the city might reduce the learning rate of the historical knowledge and the visual knowledge and a balanced experience was sought between those three types of knowledge.

The design of the teaching experience permitted the students to achieve certain objectives:

  • Learning central historical facts on the settlement and growth of Briviesca and its Medieval layout; information that is included in the video narration.

  • Increase of visually acquired knowledge on urban or housing structures in the Middle Ages; not specifically found in the video narration, but learnt while viewing the video and in the VRE

  • Increasing student capabilities to address the spatial positioning of the main buildings and town services.

  • Increasing student interest in history and urbanism by means of new technologies and devices that provide them with a close and almost-touchable interaction with a historical site.

A total of 100 undergraduate students participated in the teaching experience. They were enrolled on the Communications Media Bachelor Degree at the University of Burgos. According to Cohen (1988), a large effect may only be expected with at least twenty-five participants. Two equal groups of students were formed at random. The video was watched by one group, and the second group went on a VRE tour through the town (and watched the short video clips). In the group of students who enjoyed the VRE, 64% of them had never previously had a VRE. The gender balance in the video group and the VR group was, respectively, 60% and 48% females. The dataset was processed with statistical normalization to assure that possible gender differences relating to learning rates (Delgado Ana and Prieto 1996) would have no effect on the results and to avoid any influence on learning outcomes. The ages of the students in the video group and in the VR group were 21–28 and 21–29 years old, respectively. The two groups followed a brief introduction on the teaching experience and its objectives before watching it and were then administered a survey.

All 100 students filled in the survey immediately after having completed their experiences. The survey contained 16 questions and a map to be filled in, as shown in Table 1. The first three questions (codes 1–3 in Table 1) were to do with general satisfaction with the teaching experience. The other questions (codes Q1–Q13 in Table 1) were divided into three groups and were intended to assess whether the students had progressed toward the session goals in the three proposed topics. Each student selected one out of three different possible answers. The initial block of five questions (Q1–Q5) concerned aspects that the student could only learn by listening to the video narrative. This knowledge included an understanding of the central historical events related to Briviesca, its establishment and to different issues regarding the urban layout of the town. The second group contained five questions (Q6–Q10) on information gained only by watching: It was not specific to the narration and had to be absorbed by watching the rendered images (i.e., building height and house construction materials). A third block included three questions (Q11–Q13) for the evaluation of the capability of the viewer to remember the town and the singular buildings. A map with blank boxes was also provided to identify singular buildings of the town from a spatial point of view. The responses were standardized and adjusted in accordance with the total number of valid surveys to be analyzed.

Table 1 Survey administered to students upon completion of the teaching experience (see annex for full survey)
Full size table

3 Results

The satisfaction of the students with the teaching experience was rated in questions 3 from 1 (hardly satisfied) to 5 (very satisfied). Figure 3 displays the average scores for the three teaching strategies. The VRE played with the gamepad locomotion scored an average of 4.72, and the one using teleporting scored 4.67. The teaching experience of watching the rendered video yielded an average figure of 4.24 (over 4 in all cases): a slight difference, but enough to suggest greater general satisfaction with VRE. One reason might be the novelty of VR experiences for the students, because, as previously mentioned, it was a new immersive virtual reality experience for 64% of students Besides, this result is also connected with the low rates of motion sickness that are outlined later on, because novelty is insufficient in itself to assure higher satisfaction levels, if the user is not feeling comfortable with the VR experience (Bustillo et al. 2015).

Fig. 3
figure 3

Overall student satisfaction with the teaching experience

Full size image

Some motion sickness problems were experienced using the gamepad locomotion in the VRE. Motion sickness or more specifically, virtual reality sickness is produced by a difference between visual and vestibular stimulations (Polcar and Horejsi 2015). The stimuli received from the eyes differ from stimuli from the inner ear. While watching the VRE, eyesight reports body movement to the brain, although the inner ears report no movement at all. This mismatch can produce motion sickness. With the inclusion of roomscale motion, where the user can stroll around the environment and use teleportation to move longer distances, 80% of users had a motion-sickness free experience. Figure 4 shows the immense difference, in terms of motion sickness, between the locomotion system and the gamepad locomotion. Besides, the users who suffered from moderate-to-high sickness problems during the experience dropped from around 30% in the gamepad locomotion experience to 0% in the teleporting experience.

Fig. 4
figure 4

Motion sickness using different locomotion systems in VR

Full size image

The following group of questions (Q1–Q5) referred to knowledge that had to be extracted from the video narrative by the student. In Fig. 5, an average mark given to each question and the average general score for the answers given by the three groups of students is shown. Students viewing the video gained higher marks for these questions (81/100 on average) than those on the virtual tour with teleporting locomotion (68/100) or gamepad locomotion (60/100). Both VR groups achieved similar marks, because the information was learnt watching the videos and the locomotion procedure should not affect this task. Although a larger number of wrong answers were given to the questions by the students following the virtual tour, there was an especially bad case in Q5 for VRE gamepad locomotion users. This low score was explained by a design error in the virtual tour: The students bypass a video clip and finish the tour, missing a video clip that offered the necessary knowledge to give a full response to the question. Therefore, an important conclusion might be extracted from this result: In an VRE, exposure to knowledge pills should be strongly controlled. VREs provide students with high levels of freedom during the learning experience; although this freedom can improve student interest, it can also mean that the student misses out on part of the teaching experience. It is unlikely to happen in traditional teaching activities, where the teacher exercises much higher control over student exposure to knowledge with many more means of doing so.

Fig. 5
figure 5

Knowledge acquired from video narrative

Full size image

The last block of questions and its results differed greatly from the responses to questions (Q6–Q10) on knowledge that could only be acquired visually. The average mark is shown in Fig. 6 for the three groups of students and each of their responses to the block of questions and the average scores. Those students on the virtual tour gained higher marks for the five questions (on average, 69/100 and 68/100, respectively) than the group that watched the video (57/100). The better marks obtained in this block of questions (and in the next two blocks) by the students following either of the two VR experiences may be explained by the higher interest of those students in the novelty of testing an VR environment compared with watching a video, which they saw as a less attractive experience. Some studies have outlined that this novelty can negatively affect learning rates (Makransky et al. 2019). The user may feel a sensation of overload and may be distracted from the learning goals when observing the VR environment, resulting in fewer opportunities to build learning outcomes. This result may therefore not only be justified by the novelty of VR for the students, and it suggests that VR environments really do prompt visual acquisition of knowledge.

Fig. 6
figure 6

Visually acquired knowledge

Full size image

Although no differences might be expected between the results of the two groups that followed the virtual tour, there were strong differences in some questions (Q6 and Q9). A deeper analysis is required to understand this behavior. Q6 refers to the standard height of the houses of the town. We observed that the students who used the gamepad locomotion strolled the streets at a very slow pace to avoid motion sickness, while those who enjoyed the experience with teleporting locomotion took more time to explore the surroundings of singular buildings rather than common dwellings and moved quickly to those areas with the teleportation option. The teleporting students therefore failed to recall the height of the common houses and thought that the height of the singular buildings was representative of the whole city buildings, while they are usually higher than the other buildings. In contrast, Q9 inquired into the materials of the houses (what were the most common construction materials in the main buildings of the town?). The teleporting group responded better than the other groups to Q9, because they spent more time watching singular buildings and therefore retained the different materials that are used in the construction of the houses of the town.

The third group of questions (Q11–Q13) evaluated student recall of the location of some of the main (inns, water supplies…) services of the town. The average mark for the three groups of students to each response and the average general score are shown in Fig. 7. Again, the student group on the virtual tour gained higher marks for three questions (average marks of 50/100 and 54/100, respectively) than the group watching the video (36/100). The difference in the average mark was clearly significant (14–18 points). In any case, all three groups presented low marks, due to the difficulty of locating elements in a city after a 15–30-min teaching experience. A deeper analysis is required to analyze the results of Q11 (where are the town inns located?), the worst answers to which were from the video students’ group. These students could not recall the video narrative that explained the location of the town inns; instead, they turned to their own experience of Spanish cities and incorrectly in this case located them in the Main Square: a problem that was not found with the virtual tour, because the students were facing the inns at the same time as they approached the town gates (the real location of the inns) and were able to recall that piece of information later on.

Fig. 7
figure 7

Ability to recall the locations of the main buildings

Full size image

A map was provided as the final element of the survey to study student recall of buildings and their spatial locations in the VRE. Students were expected to identify singular places (services and buildings) in Briviesca from a spatial point of view. The average marks given to the three groups of students and the average score for each response are listed in Fig. 8. The first seven locations referred to singular buildings, while the last five referred to services and elements that were not visually singular. As expected, the results were not very good, due to the short experience and the intrinsic difficulty of the spatial location task. But, once again, the virtual reality groups achieved significantly better results: an average of 62/100 and 56/100 versus 41/100 for the video. Although the VR groups had a slightly longer exposure to the map, as they were immersed in the VR environment and they followed a fixed visual path (a path of flowers and grass) in their tour (without any help of the use of the map), it may be said that immersion can also reduce the impact of paying attention to the map. There was therefore no significant difference between any of the groups with regard to exposure to the map that could have justified a higher learning rate in the group of questions relating to the location of the buildings.

Fig. 8
figure 8

Spatial visualization skills in a map

Full size image

Considering the results of the previous group of questions, we observed a trend that clearly reflected the higher efficiency of VR experiences for the recall of spatial positioning. Besides, the differences between the two means of locomotion can be observed in Fig. 8. It confirmed that the teleporting students paid more attention to singular buildings and were able to place them better on the map (e.g., Loc4, Loc6); however, with some non-representative buildings (e.g., Loc9, Loc10), they received worse marks than the locomotion gamepad users.

4 Conclusions

This research has evaluated the possibilities and limitations of virtual reality environments for teaching purposes, especially in topics related to cultural heritage. A virtual urban reconstruction of the fifteenth-century Spanish town of Briviesca has been considered for this educational research, due to the special urban and historical characteristics of the town at the end of the Middle Ages. Different topics have been shared through this reconstruction: from historical knowledge to urban layout. The virtual environment used in this educational research was previously described in an earlier study (Checa et al. 2016). It is displayed with Oculus Rift running on Unreal Game Engine. In this research, two systems for user movement were selected: (1) gamepad locomotion and (2) roomscale movements combined with teleporting locomotion with Oculus Touch controls.

Furthermore, a teaching experience has been designed following two different approaches. In the first, a more conventional approach, the students viewed a 13-min video, created with rendered images of the virtual environment, on the Medieval history of the city, its urban structure and the use and location of its main buildings. In the second approach, the students were invited to a 30-min semi-guided tour in the 3D immersive environments and the screening of the aforementioned video split into seven 2-min videos. In both cases, there was a general presentation of the teaching experience to the students beforehand and they were administered the survey afterward. The survey questions were designed with a view to the evaluation of knowledge learning of different nature and general satisfaction regarding the teaching experience. In all, 100 undergraduate students on the Communications Media Bachelor Degree at Burgos University (Spain) participated in this learning experience.

The first conclusion of this research is that the students expressed their preferences for the VRE (their score of 4.7 was somewhat better than the score of 4.2 for the video), explained perhaps by the novelty of some aspects for the students. This higher level of satisfaction with VR experiences was not always presented in the bibliography and can be correlated with the absence of motion sickness thanks to the use of the roomscale movements with teleporting locomotion with Oculus Touch controls.

However, the keener interest that students showed toward VRE also yielded positive results in terms of the learning process, although this fact can be justified both by the novelty of the VR experience for the students and by the inherent capabilities of VRE to provide immersion and presence states in the user. Learning by watching was more effective on the virtual tour. There are differences in user behavior depending on the locomotion procedure in the virtual reality environment. Those users with gamepad locomotion employed their time evenly throughout the tour, spending similar lengths of time walking the streets and watching singular buildings. They did so to limit motion sickness, as quick movements while watching the surroundings will produce this undesired effect. The users who moved around using roomscale movements combined with teleportation locomotion spent very little time walking the streets, because moving with the use of the teleporting option gives no sickness effects, and the students, on the other hand, spent more time watching singular buildings. Therefore, although both groups in the virtual tour learnt more about visually acquired knowledge, the marks of the teleportation group were better for details relating to singular buildings, while the marks of the gamepad locomotion group were better for details relating to streets and normal buildings.

Besides, one of the most complex learning tasks in an urban layout is learning the location of the main buildings and sites of a city. In this case, all the marks of the students were average, due to the difficulty of this task. But VRE helped to acquire this skill, and the marks of the teleporting locomotion group were slightly better than those of the other VR group, perhaps because the students spent more time watching the singular buildings in the city. This result for the VRE groups cannot be linked to a slightly longer exposure to the map, because these students are immersed in the VR environment and they follow a fixed visual path (a path of flowers and grass) on their tour (without the help of the map). It may also be added that immersion might also reduce the impact of paying attention to the map. In this sense, there was no significant difference between any of the groups in time of exposure to the map that could justify a higher learning rate in relation to the identification of the buildings on the map.

Finally, historical aspects and urban layout appeared to be conveyed through the video narration more easily. In a VRE, the exposure to knowledge pills should be strongly controlled. The students had a high degree of freedom in the VRE during the learning experience, heightening interest, although with the risk of losing part of the learning experience.

In summary, the most balanced approach to undergraduate student learning on cultural heritage was therefore through a virtual tour with spatial location on a map and video clips. Lastly, a virtual tour design was crucial where the omission of certain steps in one part of the itinerary can place some viewers following another route at an unfair advantage. The implementation of a serious game might be a better way of attracting the attention of students following the VRE to historical concepts, which can be better transmitted by watching videos.

Subsequent work will be directed at enlarging the scope of the teaching experience to include older participants to stress the importance of our conclusions and their range of application. Besides, new teaching dynamics that allow lengthier VRE experiences with higher student involvement should be designed to improve the spatial allocation of the buildings on a map and serious games appear to be the best option for future developments in this area. The design of lengthier VRE experiences will be an aim for future comparison with shorter VRE experiences and especially the learning outcomes of both experiences.