Keywords

1 Introduction

In recent years, the use of Virtual Reality (VR)-based simulators for training is becoming more widespread because traditional methods of training have some major drawbacks [12]. In the military context, these drawbacks related to traditional training methods based on field exercises are high costs, logistics complexity, and spatial-temporal constraints [13]. VR-based simulation environments can help address these drawbacks while serving as a platform for supplementing current training approaches [12]. Besides, computer simulations provide more convenience, accessibility, and flexibility [13].

Studies prove that computer-simulated environments can effectively help establish the link between theory and practice while providing a safe environment in which to acquire experience [8]. As safety is an important issue in the military training context, Army Forces have seen an increased interest in using virtual environments for training purposes [22].

Even though there has been an increasing interest in using VR simulators that mock real scenarios and allow professionals to perform operations that previously would only be possible in a real environment [8], creating a VR application from scratch takes up a lot of time and research. Virtual Reality frameworks simplify this process of development by providing generic functions and features such as head and hand tracking, controller mapping, object manipulation, scene and task management, etc. These functions or features are reusable such that they can be used to any kind of VR application, and scalable such that customization can still be done depending on what the developer needs. Having a VR framework allows the developers to dedicate their time to the user interface (UI) and the 3D navigation/interaction systems rather than dealing with the core functionalities of a VR application [6]. This perception is very important because, according to [20], in a VR application, the key elements to offer a valuable experience to the end user are: virtual world, immersion, sensory feedback, and interactivity.

During the literature review, it was found that, although there is a lot of research related to the use of virtual reality in military training, there is not much emphasis on the use of frameworks to support the development of these applications. Considering this identified void, this paper discusses aspects that are crucial to a military training VR application and presents IME\(^{VR}\), the first Brazilian Army VR framework, which seeks to facilitate the development and the reuse of solutions for common issues in different military training VR simulators.

2 Method

To achieve the objective proposed in the introduction, the study was structured in three phases:

  1. 1.

    Theoretical phase: a bibliographic review to identify the key aspects of military training VR simulators (Sect. 3).

  2. 2.

    Practical phase: a high-level view of the IME\(^{VR}\) framework’s architecture, linking its components to the aspects identified in the theoretical phase (Sect. 4).

  3. 3.

    Test Phase: the usage of the framework in the implementation of SAOA, the first Brazilian Army VR simulator for Army Artillery observer training (Sect. 5).

3 Related Work

This section presents recent works (published in the last five years) linked to military training VR simulators.

According to these criteria, the following papers were chosen and analyzed: [1,2,3,4,5, 7, 9, 11, 14,15,16,17], and [18].

[1] discussed the virtual reality applied for enhanced education learning, military training, and sports.

[2] analyzed experiments and survey results of a VR application developed for Pakistani Army military physical training.

[3] showed a U.S. Navy VR simulator for training maintenance procedures.

[4] presented a 3D interactive virtual reality (VR) military system to train Chinese students in live firing.

[5] presented a study using VR simulation to assess performance in Canadian Navy emergency lifeboat launches.

[7] presented a Thai aircraft recognition training simulator using virtual reality.

[9] described a virtual reality soldier simulator with body area networks for Taiwanese Army team training.

[11] presented a study where virtual reality was used to train U.S. Army Landing Signal Officers (LSOs).

[14] presented a Chinese field operation rescue simulation system based on VR.

[15] described the design and usability evaluation of a mixed reality prototype to simulate the role of a U.S. Army tank platoon leader.

[16] investigated the impact of VR technology on 46 immersive gamified simulations with serious purposes. Military training applications were included in this set of simulators analyzed.

[17] analyzed the use of VR in the U.S. Air Force initial flight training.

[18] showed a VR simulator for Army Dismounted Soldier Training used by the Armies of the United States, Canada, and Denmark.

3.1 Key Aspects of Military Training VR Simulators

After analysing the related work, it was identified the following key aspects related to military training VR simulators:

  1. 1.

    Data model: considering roles and relationships, [2,3,4, 9, 11, 14, 15], and [18] implemented a data model based on an instructor evaluating a student during the execution of a training task.

  2. 2.

    Network: [2,3,4, 9, 11, 14, 15], and [18] presented a training workflow implemented by a communication protocol between simulator’s terminals in a network-based application.

  3. 3.

    Game engine: the VR simulators were designed with Unity game engine in [2,3,4, 11], and [15]. In the other papers, the game engine was not explicitly stated.

  4. 4.

    HMD: [2] used Samsung GearVR. [3, 9, 11], and [15] used Oculus Rift. [17] used HTC Vive. [16] analyzed simulators with different types of HMDs. In the other papers, the HMD was not explicitly stated.

  5. 5.

    Performance measurement: in all analyzed simulators, some type of performance measurement was considered (interactive feedback, report generation, etc.).

Table 1 summarizes the key aspects identified in the analyzed military training VR simulators.

Table 1. Key aspects of military training VR simulators
Full size table

4 Framework

Considering the key aspects identified in Sect. 3.1, this section details the framework specified, designed and tested in this work. The IME\(^{VR}\) framework is presented by two topics: solution specification and technologies used.

4.1 Solution Specification

Based on the instructor-student relationship presented in Sect. 3.1, it was specified that the framework should be composed of two terminals: the instructor terminal and the student terminal. These terminals are connected by a Wi-Fi network, as shown in Fig. 1.

Fig. 1.
figure 1

General structure of the framework

Full size image

In addition, based on the need to implement a communication protocol between terminals (presented in Sect. 3.1), the framework interaction diagram provides two base messages (task designation and finalization) and the necessary structure for the communication protocol messages implementation, as shown in Fig. 2.

Fig. 2.
figure 2

Framework interaction diagram

Full size image

As there are multiple ways to view and interact with data in a VR application, the framework uses the MVC (Model-View-Controller) architecture. This pattern is structured into three logical components that interact with each other. The Model component manages the system data and associated operations on that data. The View component defines and manages how the data is presented to the user. The Controller component manages user interaction (e.g., keypresses, mouse clicks, etc.) and passes these interactions to the View and the Model [21]. Following the MVC architecture, the framework specification was structured into two parts: Model and View-Controller.

Before detailing the MVC architecture of the framework, it is necessary to note that a framework is a generic structure that is extended to create a more specific subsystem or application [21]. To extend a framework, you cannot change its internal code. You must add concrete classes that inherit abstract class operations and/or define callbacks, methods that are called in response to events recognized by the framework. This feature of frameworks is known as inversion of control [19]. Based on the inversion of control, the framework specification includes two layers: FRAMEWORK and TO DO. In the FRAMEWORK layer are packages and classes that are part of the framework core and must not be modified by the user. In the TO DO layer are the packages and classes that can be edited, complemented, or implemented by the user.

Considering the key aspects presented in Sect. 3.1, as shown in Figs. 3 and 4, the framework was structured as follows:

  1. 1.

    Model

    1. (a)

      FRAMEWORK layer: data (data model, data persistence, and report generation), communication protocol (message handler, base/interface message, and message parser), and network (multithreaded TCP/IP client/server communication).

    2. (b)

      TO DO layer: data (definition of the task to be performed by the student), communication protocol (definition of the protocol messages related to the training task), and network (definition of the network parameters).

  2. 2.

    View-Controller

    1. (a)

      FRAMEWORK layer: application UI base structure and base controller for the integrated VR HMDs (Google Cardboard, Google Daydream, and Oculus Quest).

    2. (b)

      TO DO layer: user interface/interaction complementation.

Fig. 3.
figure 3

Framework overview

Full size image
Fig. 4.
figure 4

Framework UML diagrams

Full size image

4.2 Technologies Used

Table 2 presents the technologies used in the implementation of the IME\(^{VR}\) framework.

Table 2. Technologies used in the IME\(^{VR}\) implementation
Full size table

About the VR HMDs, five candidates for the use in the framework were analyzed: Google Cardboard, Samsung GearVR, Google Daydream, Oculus Quest, and HTC Vive (Table 3). Considering that the framework, in its initial version, would include one HMD within each range of visual quality and cost, it was integrated with Google HMDs (Google Cardboard and Google Daydream) and Oculus Quest.

Table 3. VR HMDs analysis
Full size table

5 Case Study

To validate the usage of the framework in the implementation of a military training VR simulator, an application for Army Artillery observer training called SAOA was developed. This simulator is detailed in [10]. The case study is presented considering 2 points of analysis: application and development effort.

5.1 Application Analysis

Figure 5 shows the main screen of the instructor terminal. Through this terminal, the instructor can create new assessment simulations, load previous assessment simulations, configure network-related data, manage training missions, and track observer’s evaluation results on each of these missions.

Fig. 5.
figure 5

Instructor terminal

Full size image

In addition, the instructor terminal has a built-in copy of the observer terminal, called the observer monitoring terminal. This tool is very useful because it allows the instructor to follow, in real time, what is being visualized in the observer terminal. The observer’s point of view is a 3-DOF rotational experience through which he or she can see the terrain and the targets/fires to be rendered. As shown in Fig. 6, to support the observation process, the system provides five auxiliary targets to be used as a reference for target location and correction adjustments.

Fig. 6.
figure 6

Observer monitoring terminal (overview)

Full size image

Another supporting feature provided to the observer is the binocular tool. Figure 7 shows the binocular tool being used to observe a fire burst onto an enemy Artillery battery. The binocular reticle is graduated in mils to assist the observer in distances calculation.

Fig. 7.
figure 7

Observer monitoring terminal (binocular tool)

Full size image

Figure 8 shows a screenshot of the observer terminal version for Oculus Quest. Through this terminal, the observer can visualize all the objects involved in the simulation managed by the instructor terminal.

Fig. 8.
figure 8

Observer terminal

Full size image

To perform the tests with this simulator, a sample of 13 users was used. All elements of the sample are Artillery officers formed by the Brazilian Army Military Academy and have extensive practical experience in observation. As shown in [10], the tests validated the simulator according to 2 parameters: sense of presence and effectiveness. The sense of presence analysis was based on the IPQ specification. The evaluation of effectiveness was made in terms of time and performance.

5.2 Development Effort Analysis

The software cost estimation model COCOMO was used to estimate the effort spent on developing the SAOA in 2 scenarios: from the scratch and using the IME\(^{VR}\) framework. Table 4 presents the data considering the application development from the scratch.

Table 4. SAOA development from the scratch
Full size table

Considering the adjustment values related to communication, distribution, complexity, reuse, installation, and user interface, the function points calculated were 1493.42. Using the COCOMO formula, the development effort was estimated at 128.79 person-month.

Table 5 shows the data considering the application development using the IME\(^{VR}\) framework.

Table 5. SAOA development using the IME\(^{VR}\) framework
Full size table

Adjusting the complexity values due to the support provided by the framework, the function points calculated were 1013.35. Using the COCOMO formula, the development effort was estimated at 85.92 person-month.

With this analysis, it was possible to estimate that the use of the IME\(^{VR}\) framework provided savings of 33.29% (approximately one third) in the SAOA development effort.

6 Discussions and Conclusion

In this paper, we presented the first Brazilian Army VR framework, IME\(^{VR}\), an MVC framework for military training VR simulators. IME\(^{VR}\) seeks to facilitate the development and the reuse of solutions for common issues in different military training VR applications.

This framework was used in the implementation of SAOA, the first Brazilian Army VR simulator for Army Artillery observer training. The validation of this simulator proved the effectiveness and versatility of the framework in VR development. Besides, this case study showed that the use of the IME\(^{VR}\) framework provided savings of 33.29% (approximately one third) in the SAOA development effort.

Even though the framework provides some facilities, it is important to note that the final quality of a VR training application lies in the hands of those responsible for the design process. The proper use of the features and capabilities of the framework and the creation of a veritable environment are the responsibility of the developers.

As limitations, it can be pointed out that the framework does not include integration with haptic devices in addition to those included in the supported HMDs.

About future work, the following possibilities can be indicated:

  • The usage of the IME\(^{VR}\) framework in the implementation of other military training VR simulators.

  • The integration of other hardware into the framework (other HMDs, haptic devices, etc.).

  • The evolution of the IME\(^{VR}\) framework to IME\(^{XR}\) through the inclusion of supporting components for the development of military training Augmented Reality (AR) applications.