1 Introduction

The first studies that Lanier initiated on virtual reality (VR) in the mid-1980s inspired the emergence of the easy-access, low-cost, wireless virtual reality devices used today [28]. VR allows the learned information to be put into practice by creating a sense of presence. Although students are given tasks to put the theoretical knowledge they learned into practice in some multimedia software, students can not experience learning by feeling themselves in an authentic environment in such software. VR applications differ from other educational software in that they can provide students with a unique reality experience [35]. VR supports advanced teaching activities, thanks to which it creates very lifelike images. Users adapt to the environment more easily with VR activities. In addition, users interact with media content according to changing scenarios or conditions. VR can be more effective than other technologies in terms of acquiring various skills. This is because the brain perceives activities in virtual worlds as real and facilitates the transfer of learned knowledge and skills to the real world [44]. In addition, users gain kinesthetic skills during activities in VR environment [9, 23]. Procedural learning is learning how to do something by acquiring psychomotor skills through instructions [29]. VR environments contribute to the skill development of students in authentic learning environments with procedural learning. When the literature is examined, there are different studies that contribute to learning using Head-Mounted Displays (HMD) and give students psychomotor skills [6, 8, 13, 16, 62]. In these studies, VR environments facilitated the learning process by increasing learning motivation. In addition, by making the learning process interesting, it has made it easier for students to gain psychomotor skills.

HMDs stand out as one of the most common VR technologies that enable procedural learning. HMDs are a wearable computer technology that provides hands free access in a virtual environment [48]. With the decrease in the prices of HMDs in parallel with the development of VR, these devices have become more accessible for commercial and educational access. According to Radianti, Majchrzak, Fromm, and Wohlgenannt [41], low-cost HMDs for mobile devices, such as Samsung Gear VR and Google Cardboard, enable everyone to experience immersive virtual environment. The resolution and field of view characteristics of HMDs affect the sense of presence [15]. Ergonomics and usability factors may vary according to device types. Image width, weight, and adjustability affect the usability of HMDs [4].

Developing VR applications for mobile devices brings some educational advantages. Mobile technologies allow adaptive, individual, and interactive learning applications [30]. These technologies have appropriate functionality and low cost [40]. According to Zydney and Warner [66], it transforms the traditional classroom environment into a more attractive and interactive one. Learning with mobile devices contributes positively to informal learning as well as formal learning in classroom environments [2]. According to Huang [18], HMD VR learning environments could significantly enhance students’ science self-efficacy. Considering these educational advantages of mobile technologies, the development of VR applications for mobile devices stands out as a necessity [56].

Presence is a sense that users interact with objects or each other in a three-dimensional environment as if they are in a real environment. As the sense of reality in virtual environments increases, the presence of users increases [61]. It is possible to come across various studies examining the relationship between presence in virtual environments and different variables in the literature ([5, 24, 26]; Merchant, Goetz, Cifuentes, Keeney-Kennicutt, & Davis, 2012; [57]). In these studies, users interacted with objects or people, feeling as if they were in authentic learning environments. The users have taken control by navigating in the environment as they want in these learning environments. Besides, users carried out learning activities by isolating themselves from the outside environment.

Although HMDs have a wide range of use, they have also various limitations that reduce their popularity in daily life. Preparation of content with this technology is a painstaking, time-consuming, and costly process [14]. Especially wired VR equipment can be difficult to use for users. During VR implementations carried out with such equipment, users may experience cable tangling problems [46]. Another limitation of HMDs is the high cost and large dimensions of devices that provide high resolution and a wide field of view. Long-term use of HMD can lead to health problems such as nausea, dizziness and headache [47, 49]. Besides, HMDs may cause visual fatigue and cognitive load [53], vertigo [65] and simulation sickness [43]. Considering these situations, it is a necessity to set the HMD usage times appropriately.

Considering the technological developments in today’s conditions, it is observed that not many concrete steps have been taken in order to expand the use of low-cost VR devices in education in authentic learning environments. The necessity of low-cost implementation of VR applications has been emphasized in previous studies [12, 20]. In addition, there are limited number of studies in the literature that have been carried out using low-cost VR devices and provide students with psychomotor skills. The contribution of this study to the literature is to prove the effectiveness of low-cost VR devices in learning environments and to reveal the potential for widespread use in classrooms. In addition, preventing the cable tangling problem in VR devices is another important dimension in the study. It is foreseen that the usage potential of VR applications in lessons will increase with the elimination of cable tangling and cost problems. Within the scope of the study, VR applications developed for mobile devices have the potential to gain psychomotor skills to students. These applications make learning easy, interesting, and fun. It provides students sense of presence during learning activities. Moreover, these applications are important in terms of preventing the risk factors that students may encounter in their professional life. Apart from this, very few studies in the literature [1, 25] utilized multimedia design principles while developing VR applications. HMD applications for mobile devices were developed according to the principles of multimedia design to prevent the increase in cognitive load in this study. The aim of this study is to examine the effects of VR implementations on learning by using low-cost wireless VR devices within the scope of the “Fire and Emergency Situations” course and to what extent the presence is created in virtual environments. Besides, determining student experiences in VR environments is another aim of the research.

The following research questions have been examined in this context: 1) What is the effect of VR implementations developed for HMDs on learning? 2) To what extent the presence is created in VR environments? 3) What are the students’ experiences in VR implementations developed for HMDs?

2 Method

In order to provide more comprehensive explanations to research problems, to ensure the integrity of the findings by using quantitative and qualitative data together, and to increase the validity of the research through data triangulation, emmbedded design, one of the mixed designs, was used. Because it was considered within the framework of an experimental research, the embedded design was named as the adopted embedded experimental research design. Embedded designs are mixed research designs using different data types in case a single data set is not sufficient for research questions. In the adopted embedded experimental research design, qualitative data are used in an empirically based quantitative research [10].

2.1 Participants

The research is aimed at students enrolled in the first year of a state university’s Civil Defense and Firefighting Program. The study was carried out with two experimental groups and a comparison group, a total of 96 students, 32 in each. Experimental group 1 participated in teacher-centered direct instruction and VR implementations, experimental group 2 participated only in VR implementations, and comparison group only participated in teacher-centered direct instruction. The students in experimental group 1, experimental group 2 and comparison group were randomly assigned to their groups. Participants’ ages range from 17 to 24. Most of the participants have seven years or more of computer experience. More than half of the students (58%) in the experimental groups stated that they had not experienced VR.

2.2 Development process

Samsung Gear VR SM-R323 and VR Box 2.0 goggles were used while developing VR applications. In order for wireless HMD applications to work properly, phones with motion detector G sensor and Bluetooth feature are needed. Two different controllers, Appa and VR Box 2.0, were used in order to interact with the media content in the VR applications. These controllers are compatible with cost-effective VR goggles and all phones with bluetooth feature.

Multimedia design principles (signaling, redundancy, multimedia, spatial contiguity, temporal contiguity, segmenting and modality, guided discovery learning, feedback) were used while developing VR environments in the research [34]. With the principle of signaling, the students were enabled to focus on the subject to be told. Thus, the students’ interest was not distracted. In order to prevent the increase of cognitive load on students, information that addresses more than one sensory (verbal and visual) was not presented within the scope of the redundancy principle. However, in order to provide a better learning within the scope of the multimedia principle, verbal expressions were made in addition to writing, pictures or animations during some activities. While explanatory subtitles were used to learn subjects that are difficult to understand within the scope of the spatial contiguity principle, it was ensured that the mental connections between visual and verbal presentations were better established within the scope of the temporal contiguity principle. With the principle of segmenting, students had the opportunity to learn step by step the subjects they wanted to be told. With the principle of modality, the cognitive load was shared between the visual and verbal channels, allowing students to learn more easily in VR environments. With the guided discovery learning principle, students were guided in cases where they had difficulty in carrying out activities in the VR environment. Explanatory feedback was given to the students immediately after the tasks performed in the VR environment with the feedback principle. Thus, the factors causing success or failure were explained to the students with their reasons. “Stop-Continue” function was added to VR applications so that students can run the activities at their individual pace. In Table 1, it was demonstrated which multimedia design principles are used in the study, why and how they are used in VR learning environments.

Table 1 Multimedia desing principles
Full size table

The creation of VR environments basically consist of three stages. A summary of what is done at each stage is given below.

Planning

The aim and limitations of the study were determined during the planning phase. A needs analysis was conducted with field experts and students before the content was developed.

Design

Draft ideas about the features that should be in VR environments, and how the course content should be were discussed with field experts, researchers, and development team at the beginning of the design phase. Storyboards were created and transmitted to the development team. Then, considering the multimedia design principles, the development process of the application was started under the supervision of field experts.

Development

During the development phase, the content of the subject to be taught in VR environments was prepared. First of all, text and sound files were prepared. Then, the modeling phase of three-dimensional visuals was started. Three-dimensional objects in virtual environments are modeled with 3DS Max software realistically. Texts, sounds, 3D models, and animations were brought together systematically using Unity game engine. C# programming language in the Unity game engine was used to ensure the interaction between objects in VR environments and to prepare some animations. .apk files of the application for mobile devices were created in the Unity game engine.

2.3 Procedures

During the implementation activities, firstly, teacher-centered direct instruction (theoretical knowledge) were given to the students within the scope of the “Fire and Emergency Situations” course in VR environments. Later, the students put the theoretical knowledge they learned into practice by practicing in virtual environments. Activities in VR environment have been developed to run on mobile devices. Headsets were attached to VR goggles during the implementation. Thus, the sounds that may come from the outside world are prevented. After the VR applications were installed on the students’ mobile devices, the students performed the activities in the virtual environment with the mobile phones placed in the Samsung Gear VR SM-R323 goggles. The reason for using the specified brand VR goggles is to get a quality image from them. These goggles are relatively low-cost among the VR goggles with good image quality on the market. Samsung Gear VR SM-R323 goggles only support Samsung branded phones. Students whose mobile phones are not Samsung conducted the activities with VR Box 2.0 goggles that support all 3.5–6 in. devices. Interaction with objects in the VR environment is provided by the Appa controller. VR goggles and control devices used in the implementation are presented in Fig. 1.

Fig. 1
figure 1

VR equipment: (a) Samsung Gear VR SM-R323. (b) VR Box 2.0. (c) Appa controller

Full size image

The VR tasks consists of two stages as part 1 and part 2. Part 1 consists of “Combustion”, “Combustion Products”, “Fire Fighting Principles” chapters, and part 2 consists of the “Classification of Fires” chapter. This content was chosen because of their kinetic potential for the research. Some screenshots for Chapter 1 and Chapter 2 are presented in Figs. 2 and 3.

Fig. 2
figure 2

Part 1 screenshots

Full size image
Fig. 3
figure 3

Part 2 screenshots

Full size image

Six VR goggles and six controllers were used in experimental process carried out with VR devices. Experimental group 1 participated in the teacher-centered direct instruction and VR implementation in groups of six (Fig. 4). Each group spent approximately 20 minutes in VR activities. The same process steps with experimental group 2 were carried out for about the same time the next day. The comparison group only participated in the teacher-centered direct instruction.

Fig. 4
figure 4

Experimental groups

Full size image

On the first day of the implementation, experimental group 1 and comparison group participated in a three-hour teacher-centered direct instruction. After the teacher-centered direct instruction, experimental group 1 carried out the VR activities. On the second day, the experimental group 2, who did not participate in the teacher-centered direct instruction, conducted their activities only on VR environments. Before and after the implementation process, Fire Knowledge Test (FKT) was applied to all three groups. Apart from comparison group, The “Presence Questionnaire in Virtual Environments (PQVE)” and “Three-Dimensional Virtual Learning Environments Evaluation Scale (3DVLEs)” were applied to the experimental groups after the implementations. At the end of the first day of the implementation, data of FKT, PQVE and 3DVLEs were collected from experimental group 1 on the same day. Since the experimental group 2 was included in the study in the second day, the data of FKT, PQVE and 3DVLEs were collected from this group at the end of the second day. The purpose of collecting data from experimental group 1 and experimental group 2 on the day they conducted the activities is to prevent the mortality internal validity threat. On the third day, focus group interviews were conducted with experimental group 1 and experimental group 2, a total of 12 students, 6 in each.

2.4 Data collection instruments

The quantitative data in the study were gathered with Fire Knowledge Test (FKT), Presence Questionnaire in Virtual Environments (PQVE), and Three-Dimensional Virtual Learning Environments Evaluation Scale (3DVLEs). Qualitative data were obtained using the Virtual Reality Semi-Structured Interview Form (VRSIF). Before the experimantal process, all students signed voluntary participation forms. Then, personal information forms were distributed to the students in order to gather information about their technological experiences. FKT developed by the researchers was applied to all three groups before and after the implementations in order to determine the pre-test and post-test scores of the students. FKT consists of 17 open-ended questions. The maximum score that can be obtained from FKT is 100. In order to ensure the content validity of the test, opinions were taken from 10 field experts. The Content Validity Ratio (CVR) was calculated in order to determine the degree to which each item in the test served the purpose in line with the opinions received from field experts. According to Veneziano and Hooper [59], the minimum value of CVR’s obtained by 10 field experts at α = 0.05 significance level was determined as .62. In the study, 17 questions were also used in the test because CVR values in FKT were over .62.

Developed by Witmer and Singer [61], the 32-item PQVE consists of four factors: control factors, sensory factors, distraction factors, and realism factors. The reliability coefficient (Cronbach Alpha) of the original scale consisting of 7-point Likert type items was determined as .81. PQVE was adapted to Turkish. Since it did not serve the purpose of the study, one item was removed from the original scale and the scale was converted with 31 items. This 31-item scale was administered to 125 students after some VR experience, who were studying in different departments and were not in the research group. The internal consistency coefficient (Cronbach Alpha) of the scale was calculated as .79. While positive opinions about creating presence in VR environments are scored by approaching 7, negative opinions are scored by approaching 1. In the scale with 31 items, items 8, 10, 11, 17, 21, 23, 24, and 27 with negative meanings were reversed and scored before the analysis. The maximum score that can be obtained from PQVE is 217.

3DVLEs that was developed by Shin, Biocca, and Choo [50], was used by the researchers by adapting it to Turkish. The original version of the scale is 7-point Likert type and consists of seven factors and 21 items. Fit indices were analysed to examine the fit of the model structure of the adapted scale. It was observed that the relational model was perfectly compatible. In addition, the internal consistency coefficient of the adapted scale was calculated as .94.

VRSIF, developed by the researchers, consists of nine open-ended semi-structured questions. It was aimed to obtain more detailed data from the participants by adding probes that required annotation to some questions. For the validity of the interview forms, attention was paid to the fact that the questions directed to the participants contain a single judgment. Thus, the participants were prevented from misunderstanding the questions. In order to understand the interview questions correctly, the questions were reviewed by two language experts. Interview forms were evaluated by 10 field experts whose opinions were consulted for FKT.

2.5 Data analysis

The sample size of the research was determined to meet the assumptions of parametric tests (experimental group 1 = 32, experimental group 2 = 32, comparison group = 32). ANOVA was used in the analysis of the first research problem. In the second research problem, while examining the situation of creating a sence of presence in virtual environments, the mean test scores of the users were interpreted by comparing with the threshold value (number of items × mean value). In the third research problem, qualitative data obtained from focus group interviews were categorized to support quantitative data.

2.6 Validity and reliability

In the experimental process of the study, precautions were taken to ensure internal validity [11]. While determining the experimental groups in the study, the threat of “subject characteristics” was prevented by taking care to create groups with similar technological competencies. All participants carried out the activities in the same environment in order to prevent the “location” threat. In order to eliminate the “mortality” threat, the working group was kept as large as possible. In order to eliminate the “instrumentation” threat, data collection tools was prepared in a way that will not bore the participants. In order to prevent the “implementation” threat, the course content in the groups was carried out by the same instructor. In addition, the data obtained from the participants were collected by the same researcher.

In evaluating qualitative studies, “credibility” for truth, “transferability” for applicability, “dependability” for stability, and “confirmability” for objectivity are recommended [32]. In the current study, in order to ensure credibility raw data and analyzes were examined by experts, taking into account the environment from which the data was obtained. Feedback was received from experts as a result of the review. In addition, a sincere atmosphere was created in the data collection process in order for the participants to share their thoughts more openly and sincerely. Transferability in research requires a detailed description of the situation under study [37]. Findings are described with direct quotations in order to ensure transferability in the study. The data obtained were reported in detail. Moreover, the research results were supported by the quotations obtained from the participants. The dependability reflects the stability of a qualitative study. Focus group interviews with students were recorded for dependability in the study. The researchers did not add their own interpretation to the data in any way while transcribing the interview records they obtained. In order to prevent the risk of data loss voice recordings were instantly transferred to the computer. The participants were informed about the working environment and process for the comfirmability of the study. The raw data set was stored after being read twice.

3 Results

3.1 Effects of VR implementations on gain scores

Before examining whether VR implementations made a significant difference in student achievement, the pre-test scores of the students in experimental group 1, experimental group 2 and comparison group were compared with one way ANOVA. When the ANOVA assumptions were examined, the samples whose mean scores would be investigated were unrelated. The pretest scores of the groups showed a normal distribution (p = .415). The variances for the dependent variable were homogeneously distributed for each sample. Mean scores of all three groups over 100 points were very close to each other (Table 2). There was no significant difference between the pretest scores of the groups (F (2, 93) = .889, p = .415) (Table 3). According to these results, the initial knowledge levels of experimental group 1, experimental group 2 and comparison group are similar.

Table 2 Pretest descriptive statistics
Full size table
Table 3 Pretest ANOVA results
Full size table

After determining that the pretest scores of all groups were similar, analysis of covariance (ANCOVA) was performed to determine whether there was a difference between the posttest scores of the groups. While conducting ANCOVA, it was aimed that the pre-test scores were controlled and the post-test scores were not affected by the pre-test scores. Since the variances of the scores of the dependent variable for each group was not equal (p = .415), ANCOVA was not performed. Instead, gain scores were obtained by subtracting the pre-test scores from the posttest scores. Then, whether the gain scores differed significantly between the groups was tested with ANOVA. The scores whose effects were examined in the dependent variable showed a normal distribution. The variances for the dependent variable were not homogeneously distributed for each sample. For this reason, Dunnett’s C statistic, which is used in cases where variances are not homogeneously distributed in ANOVA, was used. Thus, the analysis was continued by fulfilling the assumptions of ANOVA. It was found that the highest change in learning development scores was in experimental group 1 and the least change was in experimental group 2 (Table 4). Analysis showed that there was a significant difference between the gain scores of the groups (F (2, 93) = 395.758, p < .001) (Table 5).

Table 4 Descriptive statistics of gain scores
Full size table
Table 5 Posttest ANOVA results
Full size table

In order to determine among which groups the differentiation is, Dunnett’s C test, one of the Post-Hoc statistics, was used since the variances were not homogeneous (Table 6).

Table 6 Differentiation in gain scores based on groups
Full size table

According to Dunnett’s C test results; the mean gain score of experimental group 1 (M = 75.971) was statistically higher than experimental group 2 (M = 65.252) and comparison group (M = 46.501). The mean score of the experimental group 2 was statistically higher than the mean score of the comparison group. Considering the difference between mean scores, these differences between all groups were significant at the p = .05 level (p = .000). These findings reveal that VR implementations are more effective than teacher-centered direct instruction in increasing learning. In addition, student achievements increase by supporting teacher-centered direct instruction with VR implementations.

3.2 Effects of VR implementations on presence

Findings regarding the sense of presence obtained from the participants in VR environments are shown in Table 7. Since the middle value of the 31-item and 7-point Likert-type scale is 4, the threshold value of PQVE was calculated as 124 (31 × 4) [33]. The scores obtained from the groups vary between 125 and 203. According to Table 7, the sense of presence of all the participants in both groups in VR environments was above the threshold value (x̄> 124). Findings show that VR implementations have created a high level of presence for all students in the experimental groups.

Table 7 Presence scores in VR environments
Full size table

3.3 Students’s experiences in VR environments

3DVLEs and VRSIF were applied to the students in order to determine what experiences students had in VR environments. The mean scores of the experimental group 1 and experimental group 2 were given for the items collected under the original factor structures in 3DVLE. These values ​​were supported by student opinions for data triangulation. The opinions of the experimental groups on the “presence” and the “immersion” factors are presented in Table 8. The second factor, “immersion”, was analyzed under a single factor by combining with the first factor, “presence”, which measures items in similar direction. Negative opinions were scored as close to 1, positive opinions were scored as close to 7. Focus group interviews were held with experimental groups to evaluate VR learning environments. Two different focus groups were represented by “FG_1” and “FG_2”. The numbers next to the expressions “FG_1” and “FG_2” represent students who expressed their views in focus group interviews.

Table 8 Views on presence and immersion
Full size table

According to the group average scores in Table 8, both experimental groups have a similar sense of presence. In addition, the participants in the experimental groups immersed in the virtual environment at a similar level. Approximately one point difference has been observed between the views of experimental group 1 and experimental group 2 about experiencing a feeling of human warmth and a feeling of human contact in 3D virtual learning environments.

  • FG1_S1: With VR, we made this application as if it were a real event. You feel that you are entering the environment, you are experiencing it. I personally experienced this practice. I learned by feeling.

  • FG1_S2: As if the teacher is telling you something you know and experience. When we put on the goggles, the lesson was taught like in the classroom.

  • FG2_S4: We feel like we are in the event. We gain calmness because we feel like we are in the environment with VR.

According to the students’ views on the perceived usefulness of VR learning environments in Table 9, experimental group 1 and experimental group 2 found 3D virtual learning environments useful at a similar level. In addition, the experimental groups considered VR environments as useful environments. Students stated that these learning environments can help them in many aspects similarly.

  • FG2_S2: This practice is very important just as internship is. I think that our practice and work experience will increase even more thanks to such applications.

  • FG2_S4: It can prevent mistakes to be made. It prevents loss of life and property. We participate in drills as much as we want. We save time. These practices are catchy and effective.

  • FG2_S5: When a fire breaks out, it allows us to perceive more quickly what to do in order. It shows us what to do when we go into a fire.

Table 9 Views on perceived usefulness
Full size table

Students’ opinions in Table 10 show that experimental group 1 and experimental group 2 found the interaction in 3D virtual learning environments at a similar level clear and understandable. Student groups stated that they found learning highly easy through 3D virtual learning environments and learned easily in 3D virtual environments. When VR learning environments mediate the learning process (lecture support material in teacher-centered direct instruction), it was found easier than learning only in VR learning environments. This finding is a sign that students’ participation in VR activities after teacher-centered direct instruction facilitates their learning process.

  • FG1_S3: It is positive in terms of physiological and ease of implementation. We can do whatever we want in the VR environment. VR environments are as interesting as we expected.

  • FG1_S5: We easily fulfilled the tasks we wanted in VR environments. It is permanent and instructive.

  • FG2_S6: We will understand the lesson better and understand more quickly what we will do in real life. We have a better chance to learn with VR.

Table 10 Views on perceived ease of use
Full size table

According to the opinions of the students about meeting the expectations in VR environments under the “confirmation” factor in Table 11, 3D virtual learning environments met the expectations of the experimental groups at a similar level. In addition to the products and services provided in these learning environments, the experience was more positive than expected. The experimental group 1 reported more positive opinions than the experimental group 2.

  • FG1_S1: I did not believe the VR application would be effective. I thought I was going to see an ordinary sight.

  • FG2_S4: I didn’t think it was so realistic. After all, how real the virtual can be… Too realistic, the body releases adrenaline. It literally activates our excitement, emotions and feelings.

  • FG1_S5: I think this is a successful application.

Table 11 Opinions on VR environments meeting expectations
Full size table

The data in Table 12 show that the students participating in VR applications have a similar level of satisfaction with VR learning environments. The results also show that these students do not have any complaints about the 3D virtual learning environments. However, some students stated that they experienced some problems in these learning environments.

  • FG2_S2: I think we will be more enthusiastic with VR while entering the class. Because there are both practical and teacher-centered direct instruction in VR applications. When this happens, we come to class more willingly.

  • FG1_S4: We are already bored in the theoretical lesson. After a while, information is no longer in our minds. We do not understand what the teacher says. Topics in VR applications excite us. VR provides both learning and fun.

  • FG1_S6: VR goggles should be adjusted for eye disorders. I do not wear glasses, but I had tears when I walked in the VR environment for a long time.

Table 12 Views on satisfaction
Full size table

Student views on using VR learning environments in the future under the “intention to use” factor structure are presented in Table 13. According to these data, the intention of the students in the experimental groups to use 3D virtual learning environments in the future is similar. In line with this view, the situation of students to recommend VR learning environments to other friends is similar.

  • FG1_S5: We consider it necessary to use VR applications for the future lessons.

  • FG2_S6: It is necessary to use such technologies.

Table 13 Views on intention to use
Full size table

The 3DVLEs results applied to the experimental groups show that the majority of the students have the sense of presence in virtual environments and they are immersed in the virtual environment. The results indicate that VR environments are easy to use and beneficial learning environments. In the light of these findings, it can be said that VR learning environments can be satisfying and applicable learning environments in the future.

4 Discussion and implicationss

In this study, it was investigated to what extent the use of low-cost VR devices in virtual learning environments provides effective learning and sense of presence. VR applications have been more effective in terms of learning compared to teacher-centered direct instruction. When the teacher-centered direct instructions were supported with VR applications, the achievement of the students increased significantly. Putting the theoretical knowledge learned before entering VR environments into practice in virtual environment may be the reason for this situation. The teacher-centered direct instructions prepared the students cognitively for the subject before VR applications. To generalize, better learning will be provided if the students are theoretically familiar with the subject before entering VR activities.

When the literature was examined, VR environments contributed to the learning processes of students [19, 22, 55, 58, 64]. In these studies, VR applications were more effective in increasing students’ course achievement compared to teacher-centered direct instructions. In the current study conducted for the Fire and Emergencies course, student achievement in VR applications increased more than just teacher-centered direct instructions. This difference is thought to be due to the fact that VR environments contain course activities that allow procedural learning compared to teacher-centered lectures. The fact that students performed the activities at their own learning speed in VR environments and the factors that increase motivation in VR environments can be considered as other factors that may cause this difference in achievement. Considering today’s technological developments and learning needs, students are not very satisfied with monotonous teacher-centered direct instructions. Supporting student expectations with technologies that can create an innovation effect will make the lessons both effective and reinforce the theoretical knowledge learned. In this research, teacher-centered direct instructions were supported by VR learning environments by using low-cost VR devices that are easily accessible to every student. The use of high-resolution and expensive VR devices on the market in learning environments is not reasonable. Because procuring the desired quality device for each student will create a high budget requirement. What needs to be done is to make VR applications widespread with easily accessible devices that can provide equal opportunities to students. There are also low-cost VR goggles such as Google Cardboard on the market. However, these goggles have low resolution and allow limited interaction. Students may have difficulties in conducting their course activities efficiently with these VR goggles. For this reason, the course activities were carried out effectively by using low-cost VR devices that provide high quality images. This situation has revealed the potential for widespread use of certain low-cost VR devices in learning environments.

The students emphasized that VR applications developed within the scope of the Fire and Emergency Lessons provide them with professional skills and prepare them for their professional lives. Similar to this finding of the research, some studies in the literature show that VR applications can gain students skills in different fields [7, 17, 21, 42]. Most of the students who graduate from university may experience a period of fluster in their transition to professional life. In order for students to reach their professional competencies, they need to gain experience in the relevant field for a long time. When these students are provided VR implementations as virtual internships during their university years, these students will be able to feel themselves ready for their professional life to a certain extent.

The reason why students’ presence was high in both experimental groups can be shown as VR applications include learning tasks that allow realistic interaction. In addition, the fact that the objects in the VR environment are very close to the reality can be expressed as another reason for the high presence. It can be said that the increase in the presence is effective in increasing the level of satisfaction of students towards VR environments. Because the majority of the students who feel immersed themselves in the virtual environment emphasized that VR environments are beneficial and fun for them. Opinions were obtained from the focus group interviews that support these explanations. The fact that qualitative findings support quantitative findings can be accepted as an indicator that the developed VR applications are successful in creating presence in virtual environments. Findings obtained from different studies examining the effects of VR environments on presence are similar to the current study findings [12, 51]. Just as while playing a computer game, gamers feel themselves immersed in the flow of the game and involved in the environment, VR applications help students focus on course activities such as computer games in learning activities.

In terms of perceived usefulness, students found VR environments useful at a similar level. In line with the qualitative interviews, the students found it useful in terms of facilitating learning of VR applications, having the potential to use in education, being able to perform tasks that are difficult to fulfill in the real environment, reducing misconceptions, saving time and costs, preparing students for professional life, and reducing risk factors. Similar views have been reached in the literature that VR applications are useful [3, 6, 36, 39]. In fact, whether a system is beneficial in the long term should be evaluated according to its widespread use potential. If more low-cost VR applications are developed in different areas and their effects are tested, it will be confirmed that these applications are beneficial.

Most of the students stated that learning in VR environments is easy, and the interaction in these learning environments is clear and understandable. When VR learning environments are used as course support material in the learning process, they provide better learning compared to VR learning environments alone. The fact that the gain score of the experimental group 1 is higher than experimental group 2 coincides with this result of the study. Opinions supporting these findings were also obtained from focus group interviews. Similar opinions can be found in the literature regarding the ease of use of VR applications ([38, 60];). Using ergonomically suitable low-cost VR devices is also an important factor in terms of ease of use. If such applications are seen as easy to use by students, they will be widely used in learning environments.

Considering the “comfirmation” factor, it was concluded that the students’ expectations in VR environments were met similarly in both experimental groups and their VR experiences were similar. More than 70% of the participants in the experimental groups stated that they liked the VR environments at an advanced level and their experiences in these environments were better than they expected. When the literature is examined, there are realistic and exciting examples of VR applications [6, 27, 45]. In the light of these findings, VR applications can be evaluated as interactive environments that offer authentic experiences to users and meet the expectations of users.

Students found VR environments satisfiying in terms of motivation-enhancing, fun, interesting, and instructive. When the literature is examined, there are studies showing that VR environments create satisfaction and increase motivation in a similar direction [31, 52, 63]. Although VR environments are approved by the vast majority of students, these environments have led to short-term nausea, dizziness, balance problems, and eye watering in some of the students. In addition, some students using prescription glasses stated that they could not see the media content clearly because they had to remove their glasses while using VR goggles. In parallel with these views, there are studies mentioning that the use of HMD in VR environments creates discomfort in some users [14, 46]. The importance of satisfaction factor becomes clear in order to use low-cost VR applications widely in teaching. Students will only be willing to perform activities that they are satisfied with.

A great majority of the students expressed a positive opinion about using VR environments in the future and recommending these environments to others. Parallel to these views, Sun, Lin and Wang’s [54] study on teaching the moon and solar system with VR was suggested to other friends by the students who participated in the research. It can be said that by supporting the teacher-centered direct instructions with VR applications, the intention to use these applications may increase in the future. However, if the effectiveness and efficiency of low-cost VR applications are demonstrated, these applications can spread among large masses. Applications that will create a difference at the regional level will have the potential to be developed further by being followed by different masses. In order to see the widespread effect of such implementations, it must first reach large masses in line with the recommendations.

5 Conclusion and future work

Low-cost VR applications have increased the academic achievement of the students. These implementations have made learning easy, interesting, and fun. Moreover, low-cost VR applications have provided students “presence” during their learning activities. In short, these applications have provided effective learning. The cost of using VR in education has been reduced by using low-cost VR goggles and interaction devices. It has been concluded that these VR applications with proven effectiveness can become widespread (scale up) in formal learning environments. Low-cost VR applications have the potential to gain students’ psychomotor skills. It has been tried to prevent the risk factors that students may encounter in real life and professional life during virtual drills. In addition, it has been aimed to reduce the cognitive load by developing low-cost VR applications in reference to multimedia design principles. Considering all these findings, it can be concluded that low-cost VR applications can be used as an effective course support material in authentic learning environments.

The option of adding low-cost VR applications, which can provide students with professional skills in learning processes, to the curriculum can be offered to authorized institutions. However, it is important to consider the limitations of these technologies while developing VR applications. Long-term use of HMD applications can cause mental fatigue and simulation sickness. In order to prevent this situation, HMD usage periods should be considered during the implementation process. If funding is made with the support of organizations and governments, it can be ensured that better quality all in one VR goggles (like Oculus Quest 2, HTC Vive, etc.) are integrated into the classrooms. However, this situation may be limited to only a few schools or universities as it will cause cost problems. Wireless, low-cost, and high-quality VR goggles and interaction devices can be used in order to use VR technologies widely in the classroom. The effectiveness of alternative low-cost VR devices in learning environments can be examined. For low-cost and scalable options regarding VR, pixel streaming and other cloud-based renderings can be potential solution. This makes it possible to render the VR content in the cloud and stream it to low-cost headsets. Instead of trying to optimize 3D content for low-performing mobile processors, this solution could be more effective. However, it should be taken into account that mobile processors will develop hugely in the near future. Considering the rapid changes in technology, it is possible to talk about low-cost VR goggles that can be newly released to the market. It is expected that the technical features of VR goggles will increase and their prices will decrease over time. For this reason, application developers should realize the potential of new VR goggles. New low-cost VR applications can be developed in different domains and these applications can be scaled up in learning environments. Manufacturers may reduce prices for the widespread use of all-in-one VR goggles. Given the needs, different researchers should continue to produce low-cost solutions for similar issues. Thus, more effective and efficient activities can be carried out.