1 Introduction

The development of the technology of virtual reality (VR) head-mounted displays (HMDs) has led to increasing awareness of VR technology and shifts in the application of VR from the game industry to the field of education (Buń et al. 2015; Anglin et al. 2017; Tamaddon and Stiefs 2017). In the past few years, VR has triggered innovative changes in the education environment (Mikropoulos 2006; Wang et al. 2017). Owing to the advancement of science and technology and the prevalence of handheld devices, more teachers can design particular learning activities and virtual learning environments within the classroom according to learners’ needs and provide learners with a set of motivational factors.

The most important feature of VR, which is also its most fascinating feature, when applied to a teaching environment, is its ability to provide users with a sense of immersion and presence. Theoretically, VR can place learners in a virtual world with extremely intense and authentic reactions, creating the feeling that they are actually in the presented situation. Past studies on learners’ experience of VR have explored the effect of presence from various perspectives in the virtual environment (VE), such as the comparison of first-person and third-party perspectives (Mikropoulos 2006), and active and didactic learning (Persky et al. 2009). In addition, some studies have explored the relationship between interaction and the sense of presence (Messinis et al. 2010). Immersion can be divided into three levels: engagement, engrossment, and total immersion (Brown and Cairns 2004). Regardless of the level, “immersion is the result of a good gaming experience” (Jennett et al. 2008) that includes disconnection from the real world and real time, and involvement in the task environment. In the past, VR-based studies have tended to focus on the impact of VR on learning outcomes and the responses of the human body when using VR (Sharples et al. 2008). However, few studies have investigated the differences in the sense of presence and immersion in a VE generated by VR HMDs and desktop computers.

In addition, self-efficacy has been observed to have a significant impact on learning outcome by various studies in numerous fields. Self-efficacy refers to an individual’s level of confidence in performing tasks in a particular field (Bandura 1986). Previous studies have determined that, owing to the sense of presence and immersion generated by VR, the technology is capable of enhancing learning effectiveness (Messinis et al. 2010). However, the differences in the sense of presence and immersion in the environment produced by different VR technologies and the impact of such differences on the individuals’ self-efficacy require further exploration.

Disaster education appears to be a suitable subject for teaching and training in a VE owing to difficulties associated with generating a physical disaster environment. The main purpose of disaster education is to train individuals’ resilience to future disasters. However, owing to the potential risks and limited feasibility, it is difficult for disaster education courses (such as earthquake preparedness and fire drills) to construct a real-life disaster situation in which students can learn and practice the requisite skills. VR technology and the corresponding VEs may be a solution for such an educational dilemma. Therefore, this study introduced VR technology to earthquake preparedness courses by developing a program that simulates an earthquake evacuation to enhance the learners’ sense of presence and immersion in the situation. Furthermore, the developed program was utilized to explore the differences in learners’ sense of presence and immersion and the self-efficacy between the VE generated by an HMD (VR HMD) and a desktop computer-facilitated (desktop VR) virtual environment.

2 Literature review

2.1 Presence and virtual learning environments

Researchers and educators have already considered virtual environments to be a tool for science education, and have named the construct a virtual learning environment (VLE) accordingly (Leong et al. 2018; Lan et al. 2018). In addition to being suitable for science education owing to the elimination of real spatial limitations, a good VE experience, which makes learners feel that they are actually located in the simulated environment, is more mentally immersive in learning activities. The sense of presence is a key factor for such an experience (Dede 2009). Scholars have proposed various definitions and descriptions of the sense of presence. Lombard and Ditton (1997) believed that sense of presence is a type of “perceptual illusion of non-mediation,” dependent on non-interference of the relevant technical equipment with users’ operational processes. Heeter (2000) classified presence into three groups: personal, social, and environmental.

  1. 1.

    Personal presence refers to the extent to which users’ senses and emotions are expanded into the VE, generating an immersion experience, such as mental engagement.

  2. 2.

    Social presence refers to the extent of coexistence of other beings (real or virtual) in the VE and the interaction between the user and these beings.

  3. 3.

    Environmental presence refers to extent to which users’ behavior in the VE reacts to the environment; this concept it also referred to as spatial presence.

Slater (2003) claimed that “presence is about form, the extent to which the unification of simulated sensory data and perceptual processing produces a coherent place that you are in and where there may be the potential for you to act.” Diemer et al. (2015) suggested that elaborate and complex simulations increase presence. The participants’ sense of presence in the experiment depended on the perceptual distance between the actual experience and the simulated experience. Mikropoulos (2006) studied the correlation between the sense of presence and learning performance and determined that a strong sense of presence in a VLE has many positive effects on learners and well-designed learning activities also contribute to learners’ sense of presence. In addition, the study observed that learners more familiar with computer operation tend to have a weaker sense of presence. According to previous studies, the sense of presence is associated with the method of interaction, modeling quality, and users’ cognitive structure.

Moreover, more educators have begun to use VR in different domains to assist learning in domain knowledge and skills. Leong et al. (2018) used a VE for radiotherapy training, compared the learning outcomes of VE learning and 3D teaching module (standard), and suggested that the integrated approach is recommended to enhance conceptual understanding and level of confidence. In language learning, Lin and Lan (2015) reviewed 29 studies from 2004 to 2013 in language learning in VR worlds, which indicate the trend of emerging educational technology in language learning. In science education, Parong and Mayer (2018) observed that slideshows benefit students’ learning better than VR, but students report higher motivation and interest while using VR.

Many studies have suggested that VLE presented by different displays appears to lead to a dissimilar sense of presence in learners (Persky et al. 2009; Hou et al. 2012). In recent years, the reduction in the cost of HMDs, alongside improvements in device performance, has increased the popularity of VR games. Compared with other VLE experiences, VR HMDs can present the VE to learners in a more realistic manner (Hendrix and Barfield 1996). Through a 2*2 experimental design, Makransky et al. (2017) determined that participants had a higher sense of presence under VR HMD compared with that under desktop VR, but they also experienced a higher cognitive load, which affected their learning performance. The present study intends to obtain a deeper understanding of the relationship between the sense of presence and learners—specifically, the differences in learners’ sense of immersion and self-efficacy in VLEs generated by different VR devices. An earthquake education game that simulates real earthquake experiences (including physical events, such as shaking and falling objects) was introduced to observe learners’ sense of presence (including spatial presence and immersion) in the VLE and changes in their self-efficacy in coping with earthquakes. Questionnaires were utilized to collect learners’ perceptions.

2.2 Earthquake education

Over the past 20 years, Taiwan has experienced several disastrous earthquakes. The “World Urbanization Prospects” released by the United Nations in 2012 analyzed the natural hazard level of 633 large cities (with a population greater than 750,000) from around the world. Three cities in Taiwan were among the top ten urban areas exposed to three or more natural disasters. According to the report on the relationship between natural disasters and sovereign ratings issued by Standard & Poor’s Financial Services LLC in 2015, Taiwan ranked fifth among the countries threatened by the risk of earthquake (second in Asia, following Japan). Therefore, earthquake education is regarded by the Taiwanese government as an important strategy to mitigate damage caused by earthquakes. In 2014, the government launched the “Earthquake School in the Cloud” project to enhance the interest in seismology, facilitate the connection between earthquake observation and life, and reinforce earthquake awareness and preparedness among students and the public. The government also established seismology zones in the science museums of large cities, which have become key outdoor education areas for primary and secondary schools in Taiwan. In addition to providing verbal and graphical earthquake-related materials, the museums have earthquake simulation areas, so that students can experience earthquakes through simulation.

At present, disaster education is attracting increasing attention globally. Many disaster prevention and control organizations have utilized online information and games to promote the knowledge of disaster preparedness. Digital games can effectively enhance the learning motivation of learners (Annetta et al. 2009; Prensky 2003). Utilizing digital games to simulate earthquake scenarios allows learners to practice and become familiarized with necessary preparedness and response skills for improving disaster awareness and preparedness. The Earthquake Country Alliance successfully developed a free online game, “Beat the Quake!,” which was originally used as a tool to prepare citizens in Southern California in the 2008 region-wide earthquake drills. Scholars, such as Tanes and Cho (2013), have studied the game and empirically proven that the game process improves the effectiveness of learning earthquake preparedness knowledge. In addition, based on location learning theory, Chou et al. (2012) developed an adventure game to enhance learners’ understanding and skills in earthquake evacuation. They also tested the effectiveness of the game on 42 sixth-year primary school students and utilized a questionnaire to collect feedback. The results showed that the game facilitates the learning of earthquake evacuation knowledge to some extent.

Existing studies have indicated that digital games can effectively enhance learners’ performance in learning disaster preparedness knowledge and their learning motivation and risk awareness. However, few studies have applied digital games in earthquake education. Moreover, no studies have applied VR HMDs to create a VLE for earthquake education or examined its effect on learners’ sense of presence. Therefore, this study developed an earthquake simulation game that allows learners to improve earthquake preparedness, experience severe shaking during earthquakes, and practice evacuation skills in a VLE through targeted guidance.

3 Method

3.1 Virtual learning environments and stimulus materials

This study developed a 3D VR earthquake simulation game and used Unity as a platform for scene development and programming. Unity was selected because the development platform facilitates integration with other devices. In addition, Unity uses JavaScript and C# as programming languages, which are common languages used by developers.

The main scene of the game was an indoor environment furnished with common household items such as cabinets, a television cabinet, television set, table, sofas, and floor-to-ceiling windows (Figs. 1, 2). To enhance the sense of authenticity, the game adopted a first-person perspective. In addition to checking the objects presented in the scene, players were allowed to perform specific actions on the objects. Players were requested to participate in the earthquake drill according to the textual and audio instructions provided in the game. The overall VR experience process lasts 15 to 18 min depending on the learners’ pace. The participants must use a controller pointing toward a specific item or place to interact with the VR environment. The process is displayed in Fig. 3, which includes five tasks for daily life preparation in an earthquake drill. The first task is to check the wall structure from the appearance of walls. The second task is to secure the furniture with metal brackets to prevent the furniture from falling (Fig. 4). The third task is to prepare an emergency kit including water, flashlight, first aid kit, food, jacket, batteries, and cash. The fourth task is to confirm the emergency shelters, which are places of stay after a strong earthquake, in a map displayed in the room. The last task is to experience the earthquake and respond to it correctly.

Fig. 1
figure 1

Plan of the designed VR environment

Full size image
Fig. 2
figure 2

Captured screenshots of the designed VR environment

Full size image
Fig. 3
figure 3

Flowchart of earthquake drill

Full size image
Fig. 4
figure 4

Scenario of securing the furniture (task 2)

Full size image

During the tasks, instructions were presented first, and arrows that indicated the items to be collected were subsequently presented on the screen to guide the players to complete the corresponding actions. The drill followed a procedure designed based on earthquake prevention and preparedness information provided by the National Fire Agency of the Ministry of the Interior, including pre-earthquake preparation and responses during earthquakes (Fig. 3).

3.2 Overview of experiment design

This study contains two experiments, and the design framework is shown in Fig. 5. The first experiment consisted of a control group and experimental group 1, whereas the second experiment was a single-group pre- and posttest experimental design for experimental group 2.

Fig. 5
figure 5

Experiment framework

Full size image

In the intervention of VR HMD and desktop VR, participants in the study were expected to act in accordance with the instructions presented at each stage in the game, starting from preparation prior to the earthquake, and ending when they escaped from the scene, following the shaking and reaching the door. The duration of the session was between seven and ten minutes.

The first experiment lasted for 3 weeks and was divided into two phases; both were conducted in a quiet room. In the first phase, participants were asked to play the game from a desktop computer and use the mouse to perform actions (Fig. 6). During the game, the participants were expected to follow the verbal and audio instructions and perform the corresponding actions, such as to secure the furniture, prepare the emergency kit, search for emergency shelters in the neighborhood, respond to the earthquake, and escape from the scene through the door.

Fig. 6
figure 6

Desktop VR

Full size image

A questionnaire was provided to all the participants to collect their feedback on the perceived spatial presence (SP), mental immersion (MI), and self-efficacy (SE) in earthquake preparedness. A week after the completion of the first phase, the participants were invited back to participate in the second phase of the experiment. In the second phase, participants were provided with an HMD and a handheld controller when playing the game (Fig. 7). The procedure of the game was the same as that in the first phase. Following the game, the participants were provided a questionnaire to collect their feedback. In addition to the questions asked in the first phase, an open question was added at the end of the questionnaire—“What were the differences between the use of the VR HMD and the desktop computer while playing the game?”

Fig. 7
figure 7

VR HMD

Full size image

The second experiment investigated the effects of the VR system on self-efficacy. A single-group pre- and posttest experimental design was used, and the process is shown on the right side of Fig. 5. Experimental group 2 received the pretest consisting of the earthquake resistance self-efficacy scale (SE scale). A week later, the VR HMD experiment was performed, and the group received the posttest of the earthquake resistance SE scale.

3.3 Participants

The participants of the experiment were 99 university students. In experiment one, 42 (male N = 25, female N = 17) participants were assigned to the experimental group 1 (three data are not valid, n = 39) and 37 participants to the control group (male N = 15, female = 22); there are 20 participants (male N = 7, female N = 13) in experiment 2. The age of the participants was between 19 and 33 years (M = 22.9, SD = 2.6). The proportion of male and female participants in the experimental group was 55% and 45%, respectively. In addition, 72% of the participants had experienced VR HMDs before.

3.4 Measurements

3.4.1 Spatial presence (SP) and mental immersion (MI) scales

The temple presence inventory questionnaire developed by Lombard et al. (2009) to measure multiple levels of presence was adopted to measure SP (seven items) and MI (six items). A seven-point Likert scale was used to rate each item (“1” = “totally disagree” and “7” = “totally agree”). Both the SP and MI scales had good reliability (Cronbach’s α was .91 and .90, respectively).

3.4.2 Perceived self-efficacy toward earthquake preparedness (SE)

Referring to the Mulilis–Lippa earthquake preparedness scale (Mulilis et al. 1990; Williamson 1997), a five-item earthquake preparedness SE scale was developed to measure the perceived self-efficacy of participants in earthquake preparedness—specifically, their perceived self-efficacy toward their ability to take precautions to minimize human injuries and damage to their property. Examples of the items include “I am able to take precautions and secure the bookshelf to prevent it from falling during an earthquake,” “I am able to take precautions prior to an earthquake,” and “I am able to identify the area in my apartment/house that is free from items that may cause danger during an earthquake.” A 7-point Likert scale was used to rate each item (“1” = “totally disagree” and “7” = “totally agree”). The scales had good test–retest reliability (r = .68, p < .001).

4 Results

4.1 Quantitative analysis

After the 3-week experimental period, the collected data were compiled and screened, and the responses of 39 participants were determined to be valid (original N = 42); three participants reported discomfort and dizziness during the experiment and could not complete the entire experimental procedure. Participants from the control group did not participate in the games. They were only asked to complete the earthquake preparedness SE scale within a month prior to and after the experiment.

The descriptive analysis results of the SP, MI, and SE scales are presented in Table 1. The mean values of participants’ SP (Mean = 5.8) and MI (Mean = 6.1) in the VR HMD environment were greater than those in a desktop VR environment, and the standard deviations of participants’ SP (standard deviation = .12) and MI (standard deviation = .14) in the VR HMD environment were lower than those in a desktop VR environment.

Table 1 Results of experiments 1 and 2
Full size table

To examine the differences in self-efficacy between participants who received and did not receive the VR-facilitated education program, an independent sample t test was introduced to compare the responses of the experimental group in the first phase and those of the control group. The results show that the self-efficacy of the two groups was significantly different (t = 5.68, p < .005). In addition, a paired sample t test was applied to test the differences in SP, MI, and SE of the experimental group between the two VLE platforms. The results show that the participants’ SP and MI were noticeably dissimilar between the two VLE platforms (p < .001); however, no significant differences were observed in their SE values between the two VLE platforms (p > .05). As for the results of the paired sample t test of experiment 2, the difference between SE pretest and SE posttest is significant (t = − 2.41, p < .05).

4.2 Qualitative analysis

Based on the results of observation, participants were asked to respond to an additional question when they finished the game in the second phase: “What are the differences between the use of the VR HMD and the desktop computer while playing the game?” The answers were analyzed from two aspects: operational differences and sensory differences.

4.2.1 Operational differences

Among the 39 participants from the experimental group, 27 had prior experience with HMD-facilitated VE, whereas the remaining 13 had no experience with the technology. The majority of the participants who had no prior experience with VR HMD environments did not report any operational issues and could follow the instructions provided in the game. A few participants expressed specific sentiments after switching from a mouse-operated mode to a HMD-facilitated operation mode, such as

  • U1: It (using the HMD) feels more real; it was as if I was actually doing all that stuff, while the desktop computer version merely involved moving the mouse and pressing the buttons.

  • U2: It (using the HMD) was easier for me to be immersed in the situation, and I know that I was surer of what I was doing when using a controller.

  • U3: Compared with the use of a mouse, it took me longer to get used to operating the controller and moving around the scene.

When using the handheld controller, the movement of the player in the game changes with the movement of his/her wrist, which is different from the operation of a mouse. Therefore, some participants required more time to familiarize themselves with the operation (U3). However, operation with a handheld controller facilitates more freedom of movement, which creates a feeling that more closely resembles that of using one’s hands compared with the operation of a mouse (U1, U2).

4.2.2 Sensory differences

The additional question asked at the end of the second phase yielded substantial positive feedback toward VR HMD from participants, in terms of sensory differences, compared with desktop computer-facilitated VR, related to the sense of movement, authenticity, and shaking of the earthquake. The 36 participants expressed positive feelings toward the VR HMD, e.g., it was “more real,” had “more sense of presence,” and was “interesting.” Some of the specific responses are as follows: (“it” refers to the VR earthquake system)

  • U1: It (VR earthquake system) felt more real than the desktop version. I felt that I was really experiencing it.

  • U2: It was much more real, and my entire vision was the scene. [It made me] engage more in the scene. My sight was the picture and I was more into the scene. Then, the image started shaking when the earthquake came. The desktop version was a bit more boring. The VR head-mounted device was great!

  • U3: With the head-mounted device, it was easier for me to be engaged in the game, and my emotions went up and down, as they would in real life. It was more fun!

  • U4: It gave me a sense of presence; very impressive!

  • U5: It enhanced my sense of presence, making me feel like I was touching those things.

  • U6: It had a stronger sense of authenticity; I was really reaching out trying to grab the stuff in the game!

  • U7: The scene when you are supposed to hide under the table felt so real, making me really want to squat.

  • U8: It felt more real, especially when I was moving around.

  • U9: The VR HMD experience was easier to remember as it felt like you were really experiencing everything, while the desktop version felt like playing a normal video game, which was difficult for the body to remember.

According to the collected feedback, VR HMD could provide players with a sensation of realism, thereby facilitating deeper memories (U4, U9). It also directly stimulated real body movement (U5, U6, U7). Some participants claimed that being able to move the body in an HMD-facilitated VE enhanced their memory of the course content (U9) and influenced their emotions (U3). It can be observed that an HMD-facilitated VE can provide players with a more realistic auditory and visual experience, directly affecting their experience and sensory responses, and making it easier to mentally engage the participants.

5 Discussion and conclusions

Previous studies have shown different task performances for HMDs and desktop computer-facilitated VEs (Patrick et al. 2000; Santos et al. 2009; Huang et al. 2016). However, few studies have measured the differences in users’ sense of presence and immersion between the two environments. In contrast to studies that examine users’ experiences by providing different devices to experimental and control groups, this study adopted a single-group repeated measure design, where the same participants were provided two treatments in both of which their self-efficacies were high, to eliminate the impact of other influential factors. Such a design could obtain an in-depth understanding of the differences in users’ experience between the HMD and desktop computer-facilitated environments.

5.1 Visual and action effects

Generally, researchers tend to avoid the application of a single-group repeated measure design. However, by repeatedly measuring the same variables in the same sample group under different conditions, this study could illustrate that, despite the presence of identical visual stimuli, HMD-facilitated and desktop computer-facilitated VEs may lead to differences in users’ SP and MI. This is illustrated by the results of qualitative analysis. First, in contrast to desktop VR, in which the images and scene of a given area are limited by the monitor screen, VR HMD can provide users with an all-around visual experience, enhance visual stimulation, and generate a sense of being in the scene and spatial presence. Consequently, the users devoted more attention and experienced a greater presence in the game. Second, SP and MI are affected by physical activity. In a desktop computer-facilitated VE, users sit in front of the desktop computer, and the only movement of their body is the movement of wrist and fingers used for mouse operation. However, in an HMD-facilitated VE, users are required to stand and turn their heads to move around the scene, and reach out to pick up items and touch fixed furniture in the virtual environment. These movements are closer to the actual actions they are expected to perform in real life when preparing for a disaster.

Although the sense of presence and immersion was observed to increase significantly in the HMD-facilitated VE, the study observed no significant differences in the self-efficacy of the participants between the two VEs. However, compared with the control group, the self-efficacy of participants in the experimental group in both VEs was substantially greater. These findings show that the application of VR technology in earthquake education is conducive to the enhancement of self-efficacy in earthquake preparedness. A likely reason that no differences in self-efficacy were observed in the two VEs is that SP and MI belong to the category of affection, whereas self-efficacy in earthquake preparedness belongs to the category of cognition. Therefore, although the participants’ affection was enhanced in the second phase, as they had already obtained the corresponding skills and knowledge in the first phase, no progression or degeneration was observed in their self-efficacy toward earthquake preparedness.

In summary, VE is a suitable method for practical training courses of hazard and disaster education. In contrast to the high-cost earthquake simulation modules and emergency evacuations drills, VR HMD can provide a real, low-cost, highly mobile, and risk-free disaster prevention and preparedness training experience.

5.2 Limitations and future studies

It was observed that some participants could not swiftly adapt to a 3D virtual environment during the experimental process. According to the struggling participants, “although VR HMD provides a greater sense of presence, it tends to make me feel dizzy,” and “if I move too fast, I feel dizzy,” suggesting that the experience can cause discomfort and dizziness. Many previous studies have suggested that the quality of the HMDs tends to have a substantial effect on VR-induced symptoms among users (Sharples et al. 2008; Moss et al. 2011). Therefore, when applying VR HMD to education, researchers should assess and balance the effects and side effects, and the necessity for the adoption of new technological devices, paying additional attention to the physiological responses of individuals to these technologies and establishing a systematic assessment method.

In addition, according to the findings of the qualitative analysis, some participants mentioned that the application of the VR HMD facilitated their memory of the corresponding knowledge and skills. However, no empirical data were collected by this study to support such a claim. It is suggested that future studies should apply the theory of embodied cognition to explore whether the body movements triggered by VR HMD are conducive to the enhancement of learners’ memory and cognition (Allen and Waterman 2015; Toumpaniari et al. 2015; Jang et al. 2017). In addition, previous studies have pointed out that affection is beneficial to the improvement of academic achievement (Kraiger et al. 1993; Lazarowitz et al. 1994), and that education in an HMD-facilitated VE tends to enhance learners’ perceived affection. The question surrounding the longer-term impact of HMD-facilitated VLE on learning achievements should be further examined. It is suggested that future studies should explore the likelihood of perceived affection generated by the HMD-facilitated VLE and body movement to produce improved learning retention.