Keywords

1 Introduction

Consider the value of elementary school students being able to take on the roles and responsibilities of STEM occupations by completing authentic, problem-based collaborative tasks set in immersive 3-D environments - all within a traditional classroom. The Virtual Access to STEM Careers (VASC) project does this by synthesizing cutting-edge virtual technologies and problem-based learning instructional practices to develop an innovative curriculum that: (1) removes traditional barriers (e.g., lack of school resources; geographic distance; poverty; disability) that prevent under-served students from participating in authentic STEM learning opportunities; (2) sparks interest in pursuing a STEM career in marine, environmental, geological, and computer sciences; (3) embeds effective gamification and reward systems to increase and maintain motivation and focus; (4) ensures students have the academic foundation to transition to STEM coursework at the middle school level; and (5) provides educators with tools and professional development to effectively implement this innovative method of instruction. To achieve the aforementioned goals, we have created several virtual reality environments that introduce the sea turtle life cycle to students. Through these virtual environments, students learn about different activities that park rangers and marine scientists regularly perform in real life. Figure 1 shows an example of one such activity students can take part in with VASC.

Fig. 1.
figure 1

Measuring Sea Turtle Exercise within VASC. Students practice using a tape measure and calipers while taking measurements of a sea turtle. Progress panel above turtle tells students what measurements they still need to complete.

Full size image

Although virtual technologies are becoming more commonplace in formal and informal learning environments, there is still a paucity of research on how to effectively synthesize virtual technologies and problem based learning in STEM curriculum [20]. Challenges posed in the literature include transferring and expanding knowledge and skills aligned to standards learned in the classroom to virtual worlds. It is also a challenge to strategically sequence virtual learning experiences that increase and maintain student engagement and motivation. Synthesizing virtual activities and STEM curriculum that yield meaningful observable and measurable outcomes can also be a challenge. Finally, the lack of educator training to effectively utilize virtual technologies in formal and informal learning environments can be an obstacle. In efforts to address some of these challenges, VASC focuses on the following research questions: (1) What learning experiences involving emerging technologies effectively enable diverse populations of students to gain familiarity and relevant competencies with these technologies, (2) What factors influence the outcomes of the learning experiences?, and (3) Does the type of hardware alter the accessibility, learning outcome, and effectiveness of learning through virtual environments?

Our research team completed an initial round of development in which we built a virtual environment for learning about different types of sea turtles featuring tasks such as measuring turtles, counting eggs, and relocating nests. Our environment includes an indoor classroom (i.e., training environment) with sand tables for students to become acquainted with virtual reality interactions and movements as well as learn the fundamental tools and skill sets applied by scientists and park rangers. This iteration of VASC also includes an outdoor beach scene in which students can explore the actual environment sea turtles inhabit and where STEM professionals do their work in the field. Future iterations will have the students transfer skill sets learned in the indoor classroom to “grand challenges” that occur in the outdoor beach scene. Examples of the scenes and tasks that make up VASC are shown in Figs. 1 and 2. Having completed development for the indoor classroom, we invited students and teachers to test out our system in two separate investigations using two different types of hardware. The investigations gathered information about how quickly and easily the participants adapt to the immersive virtual environment, evaluate the type of instructions necessary for guiding users through the system, and document the overall impressions of students and teachers using the technology. The results will inform our next round of development related to the selection of equipment, methods of interaction, delivering immediate feedback at different points of the simulation, and creating new surveys.

Fig. 2.
figure 2

Examples of different activities within VASC. Left to right, top down: 1) Measuring a turtle with calipers. 2) Exploring outdoor environment. 3) Identifying turtle species by tracks in sand. 4) Laying cage around eggs on beach.

Full size image

1.1 Paper Organization

The remainder of this paper is organized as follows. Section 2 discusses related research in integrating virtual technologies into the classroom, Sect. 3 details the design of our investigations including hardware used, participant demographics, and administered surveys. Sections 4 and 5 discuss our formal survey results, general observations of the two investigations, lessons learned for the next stages of development, and conclusion.

2 Background

Legislation including the Next Generation Science Standards [7] and Common Core State Standards for Mathematics [15] promote the integration of STEM by offering in depth connections between the STEM domains. STEM is grounded in situated cognition theory which suggests that an individual’s knowledge is rooted in the activity, context, and culture in which it was learned [26]. This theory operates under the understanding that how STEM knowledge is applied is as important as how and where the knowledge is applied because the learning is authentic and representative of a real world experience.

As real-world applications continue to gain momentum across K-12 settings, efforts to promote research and practice in STEM education have increased to ameliorate the roadblocks that separate the four disciplines through authentic, Problem-Based Learning [2]. Problem-Based Learning has garnered positive outcomes for students in the areas of collaboration [23], student engagement [1, 3], critical thinking, and problem-solving skills [21]. Problem Based Learning can offer students the type of scaffolding that enriches inquiry and increases student motivation [10, 30, 31]. The integration of technology into Problem Based Learning assists teachers because it can promote independence [17], especially for students with disabilities [6, 12] and English Language Learners [11].

Despite growing evidence of effective instructional STEM practices, recent research asserts students become disenfranchised with STEM due to cost, content rigor, time barriers, and lack of access to resources and opportunities [4]. However, proper integration of technologies into the science curriculum has been shown to overcome some of these obstacles [13, 14, 19, 25]. One of the available technologies that has a significant impact on learning, training and education is through Immersive Virtual Environments [8, 22, 25].

Immersive Virtual Environments synthesized in a Problem-Based Learning framework has the potential to support underserved and underrepresented students in accessing STEM curriculum as well as stimulating interest in pursuing later academic and career opportunities. Within Immersive Virtual Environments, students can visualize abstract concepts and complete related hands-on tasks rather than imagining them [5, 16, 32]. For example, Trindade et al. [32] utilized virtual environments to explain the atomic and molecular structures and behaviors of water when taking different forms; gas, liquid or solid. After observing virtual water, students’ conceptual understanding of the aforementioned concepts increased. Students showed the most significant improvements with the tasks that had the highest interactivity. Rousseou et al. [28] created a virtual playground for students to address a set of tasks involving arithmetical fraction problems. Their results indicate that the fully interactive virtual environment significantly improved students’ problem solving skills. In general, a growing body of research indicates that when students interact and control events in virtual environments, they become more actively involved in constructing knowledge through an immersive experience rather than learning by lecture and reading dense expository text [9, 27, 28].

Contemporary STEM classrooms offer students problem-based learning opportunities embedded in real world contexts [18, 24] versus traditional learning activities [29]. Research suggests STEM programs aim to incorporate more technology, experiential learning opportunities, and student-centered projects [29]. Through the problem-based learning process, students develop a line of inquiry, collaborate with peers, and research relevant topics to design solutions to problems. For instance, within the VASC Curriculum there is a focus on why a turtle nest requires emergency relocation (e.g., predators, high tide). Then when confronted with the pressing issue of relocation, students develop an action plan using the problem-based learning approach. After developing the action plan, the students practice discrete skills in cooperative learning activities that prepare and mimic the activities that will occur in immersive virtual environments. The activities occurring in the immersive virtual environments reflect the actual practices of STEM professionals operating in the field. Given the novelty of implementing immersive technologies in variety of classroom settings and may include 20–25 students, teachers need a streamlined process where they can independently facilitate learning activities efficiently and effectively. Incongruity between technology, curriculum, and the learning environment can lead to issues with fidelity of implementation, social validity, and teachers abandoning the project.

3 Experimental Design

To test the current state and usability of VASC, we ran two investigations using different HMD’s. Different groups of students and teachers were invited to participate in both investigations. The research team used both pre- and post-surveys and structured observation to collect data. The first investigation was deployed on the HTC Vive at a local school and the second on the Oculus Quest 2 in a university research lab. Figure 3 shows participants interacting with VASC during each investigation. While the physical locations and type of HMD changed between investigations, participants saw and experienced the exact same virtual environment and were asked to complete the same pre- and post-surveys.

Fig. 3.
figure 3

Participants testing VASC on the HTC Vive (left) and the Oculus Quest 2 (right).

Full size image

3.1 Equipment Used

The following list of equipment was set-up and used during the first investigation:

  • 2\(\times \) HTC Vive Headsets

  • 4\(\times \) Towers/Sensors for the Vive

  • 2\(\times \) Desktops capable of running the project

  • 2\(\times \) monitors for the computer

  • cables and peripherals for the computers

This hardware was packed up, loaded into a vehicle, hauled to a local school, and set-up in the classroom on the day of the investigation. Our team had to determine on the day how best to set-up this equipment in the classroom to make sure each testing station had enough room.

For the second investigation our equipment list was simply an Oculus Quest 2 and a computer for users to complete the pre- and post-surveys.

3.2 Demographics

There were 10 participants in the first investigation. All participants were either students in the target grade range or were teachers working in or near the target grade range for VASC:

  • 2 students - each in grade 4

  • 8 teachers - 1 in kindergarten and 7 in grades 2–5

There were 8 participants in the second investigation. While the students and educators in this group were mainly outside of VASC’s target grade range, their feedback still offers extra perspectives to consider:

  • 2 students - 1 in grade 5 and 1 in grade 10

  • 6 teachers - 2 elementary school teachers (1 in special education) and 4 university professors

The sample size and demographics of each investigation reflects the logistical limitations presented from Covid-19 protocols involved in IRB approval and state mandates.

3.3 Survey Design

We administered surveys at the beginning and end of each investigation. The purpose of the pre-survey was to gather participants’ demographic information and background with gaming and virtual reality. The post-surveys were designed to gauge how comfortable participants had been understanding given directions, navigating the system, performing tasks, and what they would like to see added or improved. We have designed one set of surveys for students and another for teachers. All surveys were created and administered using Qualtrics, an experience management software.

Survey for Students. The goal of the student survey is to solicit feedback from the students’ perspective, as they will be the primary users of the simulation. Since VASC is designed for students between 3rd and 5th grades, the survey was kept simple with the two following sections:

  • Demographics - basic demographic and video game familiarity questions

  • User Experience - questions on user’s enjoyment, immersion, and understanding of the simulation

Questions on the student survey are presented in different formats, including written and pictorial questions, examples of which are shown in Fig. 4. The goal is to keep these surveys short overall and for questions to be easy to understand and answer for all elementary age students.

Fig. 4.
figure 4

Examples of formats used for written (left) and pictorial (right) questions on student survey.

Full size image

Survey for Teachers. The goal of the teacher survey is to solicit feedback on the functionality and usability of the simulation. It is important to our design process that teachers easily understand the VASC system so that they can seamlessly use it in their classrooms. We also need to ensure that the system is compatible to content learned “in-person” and is accessible to the diverse needs of learners. The pre-survey had the following two sections:

  • Demographics - general demographic information and user’s familiarity and comfort level with video games

  • Technology Acceptance - user’s general familiarity with and how closely they follow emerging technology

The post-survey had the following three sections:

  • User Experience - user’s experience and feedback on the content and functionality of the simulation

  • Slater-Usoh-Steed Questionnaire - user’s sense of immersion in the simulation

  • NASA-TLX - questions on the simulation’s difficulty

We administered the teacher survey to all participants in our two investigations, including the students as all participants experienced the exact same simulation. Eventually teachers and students will use different simulations that are more targeted toward their roles in the classroom and the different surveys will take this into account. The time taken and difficulties encountered during our investigations did emphasize the need to give students different, shorter, and more pictorial surveys as shown in Fig. 4.

4 Results

Data from the investigation was collected in two different fashions. The first was the administering of surveys before and after the participants tested VASC. The second was our research team documenting general observations about the running of the overall investigation including how equipment was set-up, participant behavior, and back-end data collection.

4.1 Survey Results

Table 1. Summary of Pre- and Post-Survey Results for Students (S) and Teachers (T) over two investigations. Yellow (left side): Investigation 1 on HTC Vive. Blue (right side): Investigation 2 on Oculus Quest 2.
Full size table

Y: Yes/Strongly Agree, y: Somewhat Agree, yn: Neither Agree nor Disagree,

n: Somewhat Disagree, N: No/Strongly Disagree,

L: a Little, M: a Moderate amount

Table 1 summarizes the types of data collected in our pre- and post-surveys. Both students and teachers were asked a number of questions measuring their familiarity with gaming and VR and how comfortable and successful they were using all aspects of the VASC system. Overall the participants reported that they were able to successfully complete all tasks tested in the VASC system and enjoyed doing so with few problems. All of the students in our test groups reported being able to understand how to use the simulation and complete all tasks, and had little trouble doing so. This group also wanted to explore the system in greater detail and tended to request that more open-world exploration and interactions be added to VASC. Adults in our tests tended to have a little more trouble getting comfortable and navigating the simulation. This may indicate that when our system is expanded to have one version for students and one for teachers that a different level and type of instructions should be made available for each group. Our testing also showed that participants moved through the tasks faster on average when using the Quest 2 over the Vive.

The administered surveys also included several open-ended questions asking for more detailed opinions and suggestions on different aspects of the system. These questions helped us to identify what aspects of VASC are working well and are on the right track and what areas need improvement. Participants wrote that they were impressed with the realism of the scene, the overall accuracy of moving within the environment, and the ability to interact with objects. They felt that the way in which they interacted with tools made sense. Many participants felt they had enough information throughout the investigation to successfully use the program. However, some participants felt they needed more directions whether they be auditory or visual and requested they be placed throughout the simulation to explain each individual task. Some responses also said it took time to learn how to interact with the environment and said having a period to learn and practice at the beginning would be very helpful. The tasks that were most commonly reported as being difficult were holding objects, shoveling, picking up and moving objects, teleporting, and teleporting while doing a second task (like holding an object or scrolling). Dropping objects and trying to pick them back up also caused frustration for several participants. These results are inline with the data collected from NASA-TLX which indicates that users were mainly concerned about their performance when no time constraint was present. Users’ physical demands were one of the lowest workload factors which indicates the tasks were not physically demanding and were easy to perform in virtual reality (Fig. 5). We did not do a factor analysis to see whether the six items represent a single dimension or multiple dimensions for this initial study. As far as requests for expanding the system, several participants requested the simulation include animations showing baby turtles hatching from shells and walking toward the ocean. Students wanted to be able to interact with even more objects and buildings in the environment to add to the sense of immersion and realism.

Fig. 5.
figure 5

Average scores for the six workload factors. Lowest are Time Constraints at 38 and Physical Demand at 48 and highest are Performance at 76 and Mental Demand at 73.

Full size image

4.2 General Observations

In addition to data collected from surveys, our team made several general observations while setting up and running the investigations that will be used to modify and improve future investigations.

First, it took too long for participants, especially students, to complete the survey. Some students needed assistance from parents to complete the surveys and had trouble staying focused through the whole process. The surveys will be condensed in future investigations to only include the most relevant questions. Additionally, students will be given different surveys that are formatted in a way more targeted for grades 3–5 as discussed in Sect. 3.3.

We also decided that the equipment we used in the first investigation is impractical for future tests. It took too long to move and set-up all the computers and sensors required for using the HTC Vive. This particular set of equipment also requires too much space to be practical for having several testing stations in a standard classroom. We were only able to use 2 out of the 3 sets of hardware we brought for this experiment as the towers interfered with each other. For the second investigation, we used the Oculus Quest 2 which does not require any towers or any connection to computers. We also did not have the problem of separate units interfering with each other when using the Quest 2. We will only run future investigations and experimental manipulations on the Quest 2 so that the only equipment that will need to be transported to classrooms are the headsets and we will be able to perform larger scale tests.

As far as the actual running of the investigations went, we observed that users need more guidance throughout the testing process and more feedback from the UI. We will add written and auditory instructions especially when users complete or fail to complete a task as the current feedback method was not intuitive to all users. Additionally, we need to include instructions when users fail a task multiple times so they can reach their goals via step-by-step guidance as needed. We are also updating our environment so that once a user picks up an object it snaps to his or her hand. Several users dropped objects, became frustrated, and lost focus on the primary goals of the program. Our next iterative investigation will explore whether or not this change improves user satisfaction with the program and decreases the time it takes them to complete activities.

5 Conclusion

We are developing a virtual environment to expose students in 3rd through 5th grades to STEM careers in marine, environmental, computer, and geological sciences. At a recent stage of our process we ran two investigations with teachers and students to gain feedback on the usability of our system and gather information to inform design decisions in the next stages. By having 18 participants pick up VASC for the first time, complete a number of virtual tasks, and fill out surveys about their experiences, we learned several things that we will take into account in both our system design and future experimental set-ups. We will add more instructions through both visual and auditory means inside the simulation to guide participants through the exact steps of the system. We will examine exactly what level of detail is necessary and at what point instructions can be taken away or made optional so that participants can be successful and still have freedom to explore. We are also updating how participants interact with objects used to complete measuring tasks so that they do not waste time and become frustrated dropping, searching for, and picking up virtual tools. To maximize how much data can be collected in each investigation we will use Oculus Quest 2 that are easier to transport and a higher number of which can be used by multiple people in a tight space. We will also shorten our surveys and administer different surveys to students and teachers that are more targeted at their age ranges.

We have found that VASC was well-received by students and teachers and with enthusiasm. Our system has the potential to be a powerful tool in the classroom to engage students in a unique way and educate them on STEM concepts and career opportunities. We will continue to develop, test, and solicit educator feedback on VASC to ensure it is accessible to students and teachers and meets STEM learning objectives and national standards.