Keywords

1 Introduction

Virtual Reality, VR, has a significant growth in recent times, providing an immersive experience by creating a virtual space that interacts with the human [1, 2] in collaborative training and learning environments imitating a real-life process or situation [3]. VR environments are obtained using immersive multimedia technology that generates scenes that can be viewed by the user through devices such as: Oculus Rift VR, HTC Vive [1], Samsung Gear VR, or smartphones [2]. To capture the movements of users in VR environments can be used devices such as Wii Mote, Wii MotionPlus, Kinect by Microsoft [3], RealSense by Intel, among others, all devices use different technologies such as video cameras, depth sensors, accelerometers, gyroscopes, pressure sensors, etc. that combined with different actuators achieves the stimulation of the senses in the virtual environment.

VR-based applications aim to simulate the real world with virtual environments; VR has been applied in areas such as medicine; for anatomy and surgery training [4]; aerospace engineering, for maintenance and repair activities [5]; graphic design, for product design and manufacturing [6], as well as in the automotive industry where the development of virtual environments has been found, mainly oriented to the following fields: (i) Design where VR can be used for layout and concept evaluation during an early stage of the development process [7]; (ii) Virtual Prototyping (VP), physical models can be replicated in VR that allow for cost and time reduction derived from omitting the construction of physical models; (iii) Virtual Manufacturing (VM) encompasses the processes of modeling, simulation and optimization of critical operations in a process related to automotive engineering; (iv) Training in automotive maintenance and service tasks [8] and skills enhancement in immersive 3D environments [9]; (v) Virtual Assembly (VA) facilitates the assembly and disassembly of virtual objects, complementing the training process.

In this context, the applications that are developed in VR can be oriented to the processes of teaching–learning [10], in the academic part and to the training - qualification in the industrial scope, these processes previously mentioned can be applied individually or in collaborative works between users, for which it is considered: (i) environments with a user in which tasks that can be performed individually are considered such as assembly of mechanical parts, doors [11], spot welding, precision welding [12, 13], electronic control units, alternators [14], among others; and (ii) Multi-user environments, where more than one agent interacts with each other in the development of a task, these environments aim to strengthen collaborative work [9], in the development of collaborative work VR-Studio [15] of the Volkswagen group is one of the pioneers, however, multi-user environments have not been properly explored in the industry.

This article presents the implementation of a multi-user virtual reality application focused on the assembly process in the area of automotive engineering. The developed application allows interacting in a controlled 3D environment with other virtual users from different points in order to meet the training tasks of industrial processes.

The structure of the publication consists of six sections, in the first the introduction indicates the work that has been done with virtual reality and its advantages. The second part focuses on the problems that exist at the time of teaching and training of the technical part. The third section shows the structure of a virtual environment in which industrial processes are developed, the fourth part describes how the virtual environment was developed and how it works, the analysis of results and the experimental part are shown in the fifth section.

2 Problem Formulation

In engineering, by complementing the theoretical part with the practical part, it is intended that future professionals are competent when making decisions. Academically, most higher education institutions do not have sufficient means to achieve meaningful learning through the teaching-learning process. The lack of laboratories and workshops in which the knowledge acquired in the classroom can be applied (theory) represents a disadvantage in the training of students. In the field of engineering, certain industrial processes such as assembly lines or the technical path of an oil field allow students to become familiar with these processes and develop skills that allow high reliability and efficiency during the process.

At this point, it is worth mentioning that in professional life, training is essential because having highly trained personnel allows companies to adapt quickly to new market conditions. This ability of companies depends to a large extent on the ability of operators at all hierarchical levels to act in a self-organised manner in unknown situations and to find creative solutions [16, 17].

Another disadvantage for the realization of industrial processes is the traditional laboratories without adequate maintenance of modules and equipment, deterioration of physical space and lack of investment to acquire upgrades, among others [18]. These problems can be solved through the virtualization of laboratories, with an open structure that allows the manipulation of all the devices, in addition to optimizing the physical space and the use of different modules present in industrial processes, as well as the execution of emergency events, so that virtual teaching can focus on induction and professional training [19].

The development of industrial processes in a virtual environment can be individual or multi-user, additionally allows for teacher-student, student-student interaction, thus achieving the optimization of resources. In this context, it is proposed to develop a virtual environment for multi-user training in industrial processes, as shown in Fig. 1.

Fig. 1.
figure 1

Multi-user training structure

Full size image

In the specific case of an assembly line in the field of automotive engineering. Assembling a bodywork or assembling an engine involves knowing the function of each of its components (theoretical support) so that during the assembly an established order can be followed and the process can be carried out efficiently (practical support).

The implementation of virtual environments is intended to optimize resources in teaching-learning processes of higher-level students (engineering or technology) and the training of technical staff in the automotive area, and to standardize industrial processes with virtual reality. New processes can be experimented with without the need to stop the operation of the plant. Finally, constant training for the expansion of skills together with work experience will enable us to meet the high requirements in the field of engineering.

3 Application Structure

The virtual application is implemented considering a block scheme, in which the following are considered: the block of the scenes, the input and output blocks, and the block of the scripts, see Fig. 2.

Fig. 2.
figure 2

System Structure

Full size image

The Scenes block has the menus and modules that control the 3D components and are linked by a controller to the activity to be performed (engine assembly or body assembly), considering the physical details and assembly characteristics such as rigidity, vibration and strength, among others. In addition, it is composed of several interaction environments, because the system offers different assembly stages in the same production line.

In the Inputs and Outputs blocks, all the devices that allow the user to interact with the virtual environment are considered, using HTC VIVE glasses, which have headset, controllers and base stations (infrared cameras) as input devices, and headphones, HMD device and haptic device with which the output responses that allow the user to dive can be appreciated. For the operation of this hardware in the 3D graphic engine, connection complements are required, which perform the communication of the input and output devices with a specific action, for what is considered the steam vr plugin. The audio modules allow to generate the sound effects in each assembly station (welding, painting, knocking), which are sent to the virtual environment to obtain sound effects according to the location of the audio listener with respect to the audio source (echoes, noise and attenuation), and to the user through the headphones.

In the SCRIPTS block, modules are generated that allow the coordinated interaction between the aforementioned blocks and the elements that facilitate the operation of the system, for which the following modules are considered: (i) Manipulation Scripts: The controller grab object is used to rotate or reposition the manipulated object by means of signals from an input device; on the other hand, when the user needs to place the 3D models in a destination or assemble a finished product depending on the workstation. The controller of 3D Model Assemble is configured, which generates haptic responses in the HTC VIVE; (ii) Interface Scprits: presents the user-configurable options, the user role, the workstation and difficulty level can be selected; in the user roles, the user can choose between teacher and student, obtaining access to specific features depending on each role.

For the workstation, the assembly of an engine or the installation of a bus body is available; and, there are three levels of difficulty, which modify the behavior of 3D Model Assemble and module evaluation; and finally (iii) Scripts for the module of Network manager: which provides multi-user support, so that several users on each Assembly Stage module can intervene in real time on the same assembly line, in addition to the audible voice exchange of each user, this module is linked to the user interface, in which you can choose between teacher or student, by default, the teacher has server and student privileges.

4 Export the Models CAD to 3D Graphics Engine

The application of virtual reality for training and multi-user training in a production line of engine assembly and implementation of a bus body, has been built through a multi-layer scheme in order to achieve the necessary immersion and interaction in a virtual environment that fits the reality, see Fig. 3. Thus, we have the following layers:

Fig. 3.
figure 3

Multilayer

Full size image

Layer 1:

3D Models Built, to build the 3D elements that make up the virtual environment is used CAD software, because this tool allows creating 3D models identical to reality, considering the physical and mechanical characteristics to be analyzed in the implementation of the production line, the tool used is SolidWork because it has the necessary characteristics in a single package.

Layer 2:

Fix Axis and Hierarchy 3D Models, after building the 3D models, the game engine requires conditioning these elements, this includes 2 sublayers: Sublayer 2.1, the pivot point of each piece is fixed at the origin of the same and the orientation of the local reference system is changed to the global reference system, necessary for the manipulation of the elements when the user performs the manipulation and movement tasks in the activity being performed; and Sublayer 2.2, in which the hierarchies between the parts that make up the model are established, this is necessary to carry out the animation and transmission of movement required for the task that can be the assembly of the engine or the assembly of the carriageway. for this layer the elements are edited in 3DS Max, because it is a tool widely used in workflows, between modelling software and game engine.

Layer 3:

Once the 3D models have been built and conditioned, they are exported to the Unity3D environment to carry out the programming that provides animation and realism to the objects, and allows the user to interact with the virtual environment. First, the workstations of the production line are built and virtual spaces are assigned; for the installation of the bodywork, several stations are available: chassis welding, bodywork welding, painting, engine installation and operation tests; for each station, scripts are programmed to control the behavior of each workstation; and, the respective 3D models of each workstation are located in the space assigned on the production line. See Fig. 4.

Fig. 4.
figure 4

General assembly line

Full size image

The graphic characteristics of the materials of each object are assigned, using photographs of real textures with which the seamless technique (image editor) is applied, which allows the texture to be homogenized (smoothed contour), thus obtaining an object with a realistic and uniform photo texture, see Fig. 5. In this stage the animation of the mobile elements that take part in the tasks of the workstations is carried out, the physical changes of: texture (painting), colour and shape (welding) are programmed. In addition, visual effects are included for each of the tasks, e.g., in the case of welding, the sparks resulting from consuming the melting material are shown and for the painting station, the aerosol effect of the paint particles in the air is programmed, using the game engine particle system.

Fig. 5.
figure 5

Seamless texture.

Full size image

Subsequently, the application is configured so that several users can interact in the same environment at the same time through multi-user support, and the characteristics are limited according to the user’s role and the activities they can carry out on each workstation. The network administrator module allows an object to be manipulated by several operators, following the Unity structure. Users generate their own movements and transmit it to the server, allowing all users to recreate it, on the other hand, assembly line models are running on the server and their status is transmitted to all users, as shown in Fig. 6.

Fig. 6.
figure 6

Unity structure vs implemented structure

Full size image

Finally, the evaluation module is developed that considers the level of difficulty, task execution times, efficiency in the use of materials, initial state of the task and final product, with which the performance is quantified using the evaluation algorithm that differentiates the status of student or operator. The teacher to provide feedback through the student’s correction and grading activities uses the results obtained. In addition, a control panel is implemented so that the user can manage all the scenes and activities that are carried out in the assembly stations; these tools are accessible by means of graphic interface components (buttons, text boxes, sliders and others), that the user can use using the HTC VIVE controls; and they allow visualizing the information about attendance, status and results of the objectives of each stage of the assembly line, this information is segmented depending on the type of user.

5 Experimental Results

In order to evaluate the multi-user application it is proposed to execute several experiments in which each command and component developed in the virtual environment is used.

To start the application the menu is set up, in which the operator chooses the industrial process he wishes to carry out. See Fig. 7.

Fig. 7.
figure 7

Home de la app

Full size image

Within the industrial process of the bodywork there are 5 stages: welding, bodywork assembly, equipment, painting and testing stage; each of them contains a specific task for the operator. In the welding area the operator proceeds to weld the part of the car bodywork, within the virtual environment you can count on other users within it, one of them can be the supervisor of the production part while the other users are operators in the same task, each user has the possibility of intercommunicating with each other to make questions or ask for help with the task they are running. The welding of the bodywork has a maximum time of execution, during which the objective must be fulfilled in order to proceed in a repetitive way with the welding of the bodywork. The experiment carried out inside the bodywork in the welding area can be seen in Figs. 8, 9 and 10.

Fig. 8.
figure 8

Selection of materials to start the task.

Full size image
Fig. 9.
figure 9

Performing the welding task.

Full size image
Fig. 10.
figure 10

Verify welding task.

Full size image

At the end of the process, the product is available for use in another part of production. For the development of the mechanical part within the system menu, you enter the option of assembling the vehicle’s engine, the user or users present in the production stage make the incorporation of the parts that are part of the engine, simultaneously in the environment you can count on multiple users who exchange information among them. Figures 11 and 12 show the results of the experiment carried out for the engine assembly.

Fig. 11.
figure 11

Engine assembly stage.

Full size image
Fig. 12.
figure 12

Attaching the engine bases to the bus bodywork.

Full size image

At the end of production, it unifies the products developed in each sub-stage, thus fulfilling the objective of obtaining a final product. In order to unify the products, the equipment option is entered, a stage in which all the necessary equipment is placed to obtain the final product of production. Figure 13 shows the result of the experiment carried out in the bus equipment stage.

Fig. 13.
figure 13

Supervision of the assembly between bus and motor.

Full size image

The results presented below indicate the efficiency of the usability of virtual environments to carry out an industrial process, in our specific case: a body assembly line. To this end, the SUS [20] summary assessment method is used. in which a Likert style scale is obtained [21] that generates a single number, represented by an average composed by the usability of the global system under study.

The selected questions are 10, see Table 1, out of 50 that frame the most consistent and polarized answers. The selected points have a correlation of between 0.7 and 0.9. The weighting ranges from 1 to 5, meaning complete disagreement and complete agreement respectively.

Table 1. Questionnaire results
Full size table

The total number, obtained from the sum of the operation in each question results in 30. Based on this result, the SUS score is calculated and expressed by a multiplication of 2.5, which means that the software needs to implement improvements to achieve a higher usability feature score.

The assessment of a virtual environment by users is important because it allows them to identify possible shortcomings and at the same time increase the use of headsets such as HTC VIVE, so that the immersion in a virtual environment becomes an experience that facilitates learning and training in new tasks.

6 Conclusions

By using VR you can reproduce industrial processes that are related to engineering. Virtual environments can be used for the collaborative teaching-learning process because it is multi-user and optimizes resources. When carrying out processes involving the assembly, in our case the assembly of a bodywork in a virtual environment offers the advantage of repeating the process until the expected results are achieved, that is to say that users become familiar with the process.

The constant training in the technical area to carry out procedures and the updating of knowledge allows certain industrial processes to be carried out efficiently. In a virtual environment the interaction between users to carry out an activity avoids taking risks, it should be stressed that the experience in a virtual environment must be as close as possible to real life.

From the experiments carried out for the present investigation it was concluded that the users were familiarized with the activities to be carried out during an industrial process, in our case with a body assembly line. During the dives in the virtual environment the users had no difficulty in manipulating the controls and following the instructions. Significant learning was achieved by complementing theoretical knowledge and practical through a process.