Keywords

1 Introduction

In schools, laboratories are utilized as tools to get the students more involved in the scientific process [1]. The hands-on experience learned from laboratories compliments the material learned in the class and enhances problem-solving and critical thinking skills [2, 3]. Even though in-person laboratories have their merits, the laboratories are often expensive [4], they can only be conducted during laboratories hours, and the outcomes of the laboratories were arguable [5, 6]. If a student missed a laboratory, it was almost impossible to make up the work as it needed a physical laboratory environment with specific tools to carry out the experiment. After the disruption of traditional education due to the spread of COVID-19, classes that included laboratories were the ones affected the most. The students weren’t allowed to participate in the labs in person. Alternately, the students were watching videos and carrying-out experiments at home.

We believe with the integration of Virtual Reality (VR) in education [7], we can overcome the mentioned problems of in-person laboratories and enhance student learning [8,9,10]. Due to the increase in technology and other distractions in classrooms, students start to lose interest in the subject 15 min into the lecture/laboratory [11, 12]. VR headsets immerse the user by placing them in a non-physical world and cutting their connection with the real world. The utilization of VR in education allows for (a) students to be immersed in an environment where they can learn without distractions, (b) a low-cost, one-time purchase, and (c) an available and accessible 24/7 platform. In our previous study [7], we showed a 6-min video demonstration of our VR chemistry lab and asked 109 General Chemistry students questions. Out of the 109 students, 42.7% of the students strongly agreed and 37.3% agreed that learning general chemistry with VR will be more engaging.

Virtual Reality (VR) simulation is used as a teaching methodology in several domains. The multitude of hand-skill training benefits from VR applications since VR tackles several challenges facing traditional training methods. For instance, conventional surgical training for prospective medical students and medical residents relies on cadavers and animal parts, which are expensive, messy, and hard to repeat frequently [13, 14]. As a consequence, VR training for surgical training received attention from the research community, such as cataract surgery simulation [13], hepatic surgery simulation [15], cricothyroidotomy simulation [16], and gallbladder surgery simulation [17].

In certain domains, virtual reality education and training have surpassed traditional methods in terms of efficiency and quality of learning [18,19,20,21]. Welding and painting are two such domains. In hands-on skills education and training domains such as those, traditionally, the students need to practice with actual equipment to gain hands-on expertise and learn the subtleties of the trade. With the introduction of accessible VR headsets, hands-on training has moved gradually from using real tools to using a VR setup. The student is equipped with a headset and mock tools that resemble the real tools. The student movement and tools movement are virtually represented and tracked in VR. The instructor is supplied with an external screen to see what the student is doing in the virtual world. The results of the virtual reality simulators for core skills education such as welding, and painting were significant statistically and demonstrably. For example, the VR training adoption in welding was spread into over 140 countries with curricula available in 16 languages. The rate of training certificates completed has risen by 41.6%. This significant rise can be attributed to the rate of VR simulation training [20].

Physical science classes have also been considered in VR education. VR has already been in use to teach organic chemistry [22], physical chemistry [23], and biochemistry [24, 25]. In addition, we have implemented a chemistry lab simulation that is designed to allow students to access the lab in a safe and comfortable setting to give students confidence for the time that they first approach the actual chemistry lab. The simulation will allow students to “touch,” “feel,” and “handle” the chemicals and lab equipment with the use of haptic gloves. Our initial study [7] introduced the students to the notion of attending a VR chemistry lab, and the response from the students was primarily positive.

The notion of a VR learning framework is slightly mentioned in the literature [24, 26,27,28,29]. A VR framework could refer to a learning framework applied to VR technology to enhance the pedagogy of the system. One such framework incorporates ACE teaching framework into an interactive spherical video-based virtual reality [26]. Another paper describes an immersive virtual reality application in a teaching environment. But rather than focusing on the technology, the paper describes a framework adapted from the cognitive theory of multimedia learning. The result outlines how VR teaching should be designed based on knowledge of multimedia learning [27]. A classroom Framework for training teachers is described in [28].

Some references refer to the learning framework as the ability to customize the VR for teaching purposes [24, 29]. The authors in [24] developed a customized VR to intervene with children with a neurodevelopment disorder. The application includes functionality to monitor children’s interaction with the therapist. The authors in [29] developed an open-source interactive molecular dynamics VR network for students to perform short tasks on enzymes. The open source allows multiple participants to cohabit the same VR environments as well as touch and feel the response of dynamic molecules.

Our system that we are proposing is a complete VR learning framework that allows the instructor to create their lessons. The framework provides customization by the instructor through a web-based dashboard. The customized settings can be loaded to the VR application where students can receive lessons while feeling and viewing their surroundings in the VR environment.

2 Methods

2.1 Framework

Our framework is designed to work with different laboratory types such as biology, chemistry, physics, mathematics, etc. The framework created in this study has four main modules: (a) interface, (b) companion, (c) dashboard, and (d) VR scene, as seen in Fig. 1.

Fig. 1.
figure 1

Overall architecture of the virtual reality-based pedagogical framework. The purple components are the implemented components, while the green components are future work (assessment and performance metrics).

Full size image

We decided to integrate Oculus Quest 2 [30] as the VR headset for this framework due to its low-cost and untethered nature. The interaction with the seen is carried out with the VR controllers, while the other integrated human computer devices include: keyboard, mouse, and Touch haptic device [31]. In our framework, VR headset and controllers have direct access to the VR scene module, and they are used to manipulate 3D models and navigate in the scene. The final component of the “Interface” module is voice recognition. The voice recognition captures the voice input and sends it as an input to the “Companion” module. “Companion” module replicates a laboratory assistant during laboratory work and helps the student by listening to and addressing commands from the student during the simulation. We use speech recognition to track the voice commands to the companion requested by the student. Due to its high accuracy, we used Google’s API to convert speech to text. The word error rate of Google API is the lowest, with 9% compared to other speech recognition systems [32]. Students use concise commands rather than continuous speech throughout the simulation, which causes speech recognition to stop listening. Thus, we force the companion to continuously listen for speech starting at the beginning of the simulation and ending when the simulation is complete.

Once the speech is converted to text, the command is mapped to the appropriate function. Every student has their own unique way of expressing the commands. To properly identify the commands, the simulation maps the commands to the appropriate function by using entity linking [33], which is achieved by using spaCy [34]. Entity linking allows words of interest to link to functions for the companion to carry out the commands. During the recording, there is a lot of distortion and ambient noise that can affect speech recognition from a microphone. To avoid this issue, we calibrated the energy threshold for ambient noise levels. The algorithm relies on the energy and spectral characteristics of the signal and applies a three-level two-dimensional thresholding to determine whether the input is speech or non-speech [35].

The online dashboard, accessible through a web browser, for teachers to create laboratories and track and evaluate student performance. Each teacher/supervisor can create a log in to save their laboratories/scenes as seen in Fig. 2. Each task and module’s settings are stored as .JSON files in an online database. The settings for each task can be altered by a teacher/supervisor using a simple web browser user interface as seen in Fig. 3. Once the settings are set, a unique identifier will be provided to share with the students which can be seen in Fig. 4. Using the unique identifier, students will be able to load the settings and metrics stored on the online database. Our preliminary tests show that the .JSON file that will hold a scene’s settings is <2 MB. The data transfer rate of Firebase is 10 MB per second which will allow the new scene to be downloaded in <1 s [36].

Fig. 2.
figure 2

Login page for online dashboard

Full size image
Fig. 3.
figure 3

Scene/Laboratory creation interface

Full size image
Fig. 4.
figure 4

Scene/Laboratory JSON with a unique id.

Full size image

After the scene JSON is loaded into the scene through the scene executor, model information from the JSON is passed to the command executor, and the 3D models in the scene are updated. Also, the command executor is responsible for fulfilling the mapped functions received from the “Companion” module.

3 Results

To evaluate our framework, we conducted a usability study. Our goal was to evaluate the framework through the web-based dashboard since it will be the resource available to instructors to customize their VR lessons. Five instructors from Florida Polytechnic University performed the usability study. They are in Computer Science, Electrical and Computer Engineering, Applied Mathematics, and Data Science and Business Analytics departments. The subjects varied in their experience as instructors and their previous exposure to VR technology. Two of the subjects had experience with VR education in terms of surgery simulation, and the rest didn’t have any experience with VR education.

The subjects were given a description of the VR learning framework and what we intend to do with the web-based dashboard. The subjects read the instructions on the usability form and filled out the background questionnaire (Fig. 5). The subjects then experienced a VR chemistry lab application that we have implemented using the Oculus Quest 2. Although they were not evaluating the application itself, they experienced the virtual lab to understand the importance of VR teaching, especially for applicants who did not have a lot of VR exposure, this was important. The applicants were then introduced to the dashboard and were given a series of tasks to do with no guidance:

Fig. 5.
figure 5

First page of the usability study instructions provided to the participants of the study. The page contains an introduction, purpose as well as background questionnaire.

Full size image

  • Task 1: please create a login name and password

  • Task 2: Use your credentials to log in

  • Task 3: Add a new scene and select the following elements into the scene: fire extinguisher, the interactive periodic table, eye washing station, and fume hood

  • Task 4: View the JSON file of the scene you added

  • Task 5: Delete the scene

The subjects evaluated the tasks based on perceived difficulty on a Likert scale (1 being the hardest and 5 being the easiest). Four of the subjects found all tasks to be straightforward to do. The fifth subject found the tasks easy to do but ranked task 2 and task 3 lower (the subject ranked them at 4). The average values are given in Table 1.

Table 1. Mean values for the perceived tasks difficulties
Full size table

The reset of the questionnaire contained the following questions

  • How would you like the JSON file to be loaded in the VR lesson:

    1. a)

      Automatically

    2. b)

      Having a load button.

    3. c)

      Copying the JSON file and pasting it to another software before loading the VR application

  • How secure did you feel the system is in terms of entering credentials

    (not safe) (very safe)

    1 2 3 4 5

  • Do you expect VR classes to become available in the future? (Please explain your answer):

  • Would you prefer the website to be simplistic (current version) or have more features?

  • Did you find any negative experiences while doing the tasks?

  • Any additional comments for improvement?

Except for one subject who chose ‘automatically’, the subjects prefer a load button to transfer what they have customized on the website to the VR teaching environment. One of the subjects mentioned that the load button is preferable to not load the wrong settings by mistake.

Two out of five subjects did not find the website’s features sufficient to promote safety in terms of creating and storing credentials, but three out of five felt that it was safe. The mean value for safety from the questionnaire was 3.8. This area is not directly related to the VR framework but would enhance the overall experience.

The subjects do expect VR to be part of education in the future, especially for laboratory classes, as one of the subjects commented. Another comment on the future of VR classes was “VR allows for users to immerse and carry out tasks without distractions”. One user said, “The pandemic has proved that VR is the future.”

The simplicity of the website’s user interface and features was welcomed by three out of the five subjects, while the other two subjects suggested adding more features and making the features more personable. In the additional comments section, one of the subjects suggested adding a diagram depicting what will be loaded into the VR scene. Two subjects suggested using ‘Enter’ as a way of navigating (e.g., entering credentials) rather than relying on the mouse. Overall, there was no negative experience, and the prospect of a virtual framework was positive.

4 Conclusion

This paper presents a learning framework for VR based teaching. The framework consists of a VR application for teaching, such as the chemistry laboratory that we implemented, that can be experienced through a virtual reality headset. The application provides haptic feedback to the users to feel the objects surrounding them and thus providing a rich learning environment. In addition, there are various commands that can be issued by the user through a voice user interface. The classes can be customized through a dashboard that can be viewed on a web browser. Typically, the instructor would tweak the course according to their preference and load those custom settings to the VR application. The instructors can save their VR scene settings for future access and can create multiple scene settings and store them under their username.

A usability study was conducted to get the feedback from instructors. Five instructors with various background participated in the study and provided comments on the web-based dashboard. They performed various tasks with minimum guidance and were able to successfully complete the tasks. Their outlook to the future of VR learning was very positive.

For future work, we want to implement the assessment and performance metrics components to our framework and to move our online database that holds scene information to the cloud. Adding assessment, will allow teachers to evaluate student performance through a simple UI. Also, we want to create the ability to share scenes among different instructors. Some UI elements as well as personalized features will be added according to the user’s comments in the usability study. Finally, safety web features such as authentication methods will be added to make the instructors’ experience more relaxed and pleasant.