Keywords

1 Introduction

Virtual reality is an organized and simulated environment. It is an interactive environment where any user can observe his action and responses in the so-called virtual environment (VE) through some human–computer interface equipment. The mode of interaction between user and VE using some input devices like keyboard, mouse or some multimodal devices like multidirectional treadmill, wired glove, polhemus boom arm, joystick, trackball, exoskeleton (external hardware on hands), eye tracking, video analysis, brain wave electroencephalography (EEG), electromyography (EMG) help in perception of the real-time virtual 3D objects and subsequent analysis of the object.

Virtual reality is highly flexible because construction and visualization of different models are possible according to requirements. VR allows immersion, i.e. user can storm inside the data, interact with objects and manipulate, reshape dimension and scales according to preferences. VR is a dynamic experience because users play the role of both user and observer. Virtual reality consists of three levels: first one is non-immersive, i.e. virtual creation of 3D objects in the computer system without use of any special purpose hardware system. This is useful for training purposes. Flight simulator is a suitable example for this level of VE. Second one is semi-immersive or sensory immersive which has wide applications in robot navigation, airplane simulation and construction modelling where an user can find himself in the virtual environment and perform the necessary action according to his own perception to simulate reality. The last but not the least is fully immersive or neural-direct where connection lies between human brain, database and the user’s location and orientation. In this level, a sensory input was imparted to the human brain and simultaneously user’s thought process is injected to the virtual environment for visualization and manipulation.

In recent years, production safety scenario is not optimistic; hoisting and conveying machinery operation accidents occur from time to time. The primary reasons for these accidents include the operators’ lack of professional knowledge, lack of skills, inadequate standard operating procedure. At present, the crane is recognized as special equipment for heavy operations, but the training process is lagging behind OSHA Safety Training Courses (retrieved from http://www.bebsoft.com/osha-safety-training). Inadequate training material leads to unorganized training of crane operators and they do not really acquire the practical feel of the real-time operation, leaving a potential for accident risk. Therefore, it is an urgent task to change the current training method of the crane operators, improve the professional quality and operation skills of the workers, and ensure the safety and stability of crane operations. At present, virtual reality technology has been applied in many fields such as medical, military, aviation, robotics and manufacturing, construction, education and entertainment. It will also have a significant effect on improving the training of crane operators.

2 Virtual Prototype

Virtual prototyping (VP) is creation of simulated environment of a product where analysis and testing of the product can be done for design or qualitative performance analysis. It will help in product development owing to its availability and low prices (Choi et al. 2015) along with replacing the problems such as deployment of dedicated equipment and space in case of conventional training approaches (Goulding et al. 2012) . In safety domain of a manufacturing industry, VP can be useful in solving the issues related to layout, operation and dynamic analysis of operating environment (Guo et al. 2013) .

In order to change the status quo of the crane operator training, development of an overhead crane operation simulation system based on virtual reality technology is our goal. The developed simulated VR environment can be used for overhead crane operation training, skill improvement training, safety education and practice examination. The simulated system of overhead crane is composed of several parts, including visual system, human–computer system and hardware system (“The Virtual Crane” retrieved from http://venus.web.cern.ch/VENUS/vcs/virtcra2.2).

The use of virtual reality for simulating EOT crane operation improves the understanding of the mechanical structure and working principle of crane system. System architect uses 3D virtual simulation technology to produce realistic 3D overhead crane working scenes and working conditions. Trained person operates the overhead crane in 3D visualization of the scene using a large screen projector or a stereo imaging lens. Virtual reality system can be helpful for the operator to match real-time interaction in the three-dimensional environment, and they can operate trolley, hook and all kinds of safety devices to acquire knowledge of the operating procedure. Virtual environment gives the opportunity to operators for repetitive training. Accident reconstruction and analysis are possible for identification of errors occurred in training process.

3 Virtual Reality Application in Training

Application of virtual reality nowadays makes it possible to reconstruct, simulate and visualize the whole operation process and find out the deficiencies that are present in the operation. It is being widely applicable in chemical, mining, construction, health care industries but its applicability in manufacturing industries is still an issue to be taken care of. Accident prevention training and operational safety programmes will be beneficial for a worker in a theoretical sense but practical approach to training through immersion and visualization is possible through virtual reality. Visualization of accident that had happened in the past and providing training to the employees in the virtual environment which will give them the feel of real environment to avoid those in future is to be added in safety training programmes. Virtual reality platform can emulate immersive interactive scenarios which reflect real-life situations, simulating the way equipment responds, replicating soft skills such as human actions and behaviour. The simulation of unpredicted control situations can reduce cost, complexity and time associated with the understanding and knowledge acquisition process needed for the real-life problem solution.

4 Advantages of the VR Training System

Compared with previous training and assessment methods, the overhead crane operating system based on virtual reality technology will have following advantages:

  1. i.

    High Security: Using the overhead crane simulation system, one can safely carry out high-speed, extreme driving and very dangerous safety experiments, which cannot be achieved in the day-to-day experiment.

  2. ii.

    Good reproducibility: Because the overhead crane is used in the complex environment in reality, the real-time experiment is hard to perform which can be easily achieved by VR technique, and the overhead crane simulator based on virtual reality technology can conveniently carry out data acquisition, model selection and setting of the simulation environment.

  3. iii.

    Economic: Compared with the real vehicle test, the simulator occupies a small area and consumes less energy, and it can also be more convenient to set up various experimental conditions and experimental parameters in the software environment.

5 An Introduction to Simulators

Machine simulators are now widely used for giving training to the operators. In industry, education, military, the application of these simulators are increasing day by day. The current increasing trend of virtual reality is making that application more affordable. Virtual reality-based simulators are now used in various fields, and it is giving high return of interest. Virtual crane simulator is described below.

5.1 Virtual Crane Simulator

5.1.1 Environment and Needs

Workplace having cranes is very complex scenario. Workers should be properly trained and experienced to cope up with this complex environment. It is sometimes very difficult to reproduce the exact training scenario in real situations. Training process in real scenario is also hazardous in nature. Trainee may not be allowed to do any mistakes at the time of training in the real scenario but repetitive training can be accommodated in virtual reality training.

5.1.2 The Crane Simulator

The crane simulator is in general a virtual crane in computer-made 3D environment. The environment is the replication of a real environment. When operator or user wears Head Mounted Display, he/she will be immersed in that computer’s simulated environment. He/she will feel that they are in that environment. They can walk through the environment and interact with the objects. The realism of the environment means not only the photorealistic view of the environment, but the rigid body physics, dynamics, kinematics, gravity, collision, elasticity also. All the intractable factors must be incorporated in the environment. Users can practice as much time as they want and make mistakes there. By making mistake they will understand the path which leads to accident. They will also understand about the barriers that must be incorporated to get rid of from those accidents.

6 Methodology for Development of Crane Simulator

The methodology considered here for developing virtual reality -based crane simulator model is mentioned step by step. Some prerequisites for building virtual reality-based scenario are learning of 3D modelling software, texturing, programming language (C++, C#, python) and game engine. The methodological flow work is given in Fig. 2.1 for development of virtual reality-based prototype for crane operation.

Fig. 2.1
figure 1

Flow diagram of method

Full size image

6.1 3D Modelling and Texturing

3D modelling is the description of spatial vector location of objects, environments through a computer system. This section provides the basic concepts of the modelling technique. The method of using basic building block that is used for creating objects and the process of moving, rotating and resizing the objects are also explored here. The fundamental steps for 3D modelling are described below (retrieved from https://www.unrealengine.com).

6.1.1 Coordinate Systems

There are three perpendicular axes, which are labelled as the x, y and z axes in the field of 3D design. The origin is called the world origin that is used as the reference in 3D modelling space, and the three axes intersect here. Distance of a point in the space from the origin has been described by the x, y and z values. Rotation is also possible about the three axes, and these rotations are often referred to as pitch (rotation about the x axis), yaw (rotation about the y axis) and roll (rotation about the z axis). This coordinate system is named as the Cartesian coordinate system.

6.1.2 Points, Lines and Faces

The basic elements in 3D modelling are point, face and line also known as vertices, polygons and splines, respectively. A point is presented by the values along x, y and z axes, a line is also described defined by the XYZ values of its two end points, a face can be similarly defined by the XYZ values of the points that cover it (the points on a face must all be planar). Figure 2.2 shows an example of 3D models with coordinates.

Fig. 2.2
figure 2

Vertex, line, face

Full size image

Lines are made using points; multiple lines together make a face, and number of faces is combined to produce a shape or model. Some complex models, such as a character, may require millions of faces, while a simple box can be produced with just six faces. So it can be inferred in a common way that 3D objects are nothing but some combination of vertices. For reducing simulation time, least number of vertices in a model should be used.

6.1.3 Image Mapping (Texture Mapping)

The 3D model above explained is only a mesh. Now to make that mesh look real, texturing is very much important. Texture mapping is a method for giving detailed surface texture (a bitmap or raster image) or colour to the 3D mesh. A shape or polygon gets live after applying texture map on their surfaces. In a simple way, texturing can be said as pasting some colourful pattern on white box. Every vertex of a polygon is a texture coordinate (in the 2D model, these vertexes are known as a UV coordinate). Figures 2.3 and 2.4 describe the essence of image mapping.

Fig. 2.3
figure 3

3D model without texture

Full size image
Fig. 2.4
figure 4

3D model with texture

Full size image

Blender software is used for texturing. The following are the essential things that are very much useful for proper texturing.

6.1.3.1 UV Mapping

The process of projecting 2D images on the surface of 3D mesh. Figure 2.5 provides the UV map of a cube.

Fig. 2.5
figure 5

UV mapping

Full size image
6.1.3.2 Normal Mapping

Normal map is used to provide the direction of normal of a geometric shape. Following gives the normal maps of cube (Fig. 2.6).

Fig. 2.6
figure 6

Normal mapping

Full size image
6.1.3.3 Bump Mapping

It is provided for visualization of irregular displacements on the surfaces of plane mesh.

6.1.3.4 Reflection Mapping

This is needed to give the reflection of a 3D mesh in the environments.

After the texturing, the texture model has to be imported in virtual reality engine that is used for creating complex virtual scenario. Unreal engine or unity 3D can be used as per user’s requirement.

6.2 Animation

Animation is to give life to a 3D model. The 3D static mesh, after animation, behaves like real character. Blend Space is one of the concepts that is widely used for animation. In Blend Space, different types of animation can be added and made a single animation. That makes the model more real. Suppose in one animation, the 3D avatar will walk and in another animation it will run. Now in Blend Space, these two animations can be grouped together.

6.3 Immersive and Interactive Environment

For making the virtual environment , geometric transformation, lighting and shadows and texturing have to be implemented for making immersive environment.

6.3.1 Geometric Transformations

Geometric transformations are used for modifying the size, shape of model proportion to other model and also used for the spatial position of the object in the world. The most common transformations that are used for this are translation, rotation and scaling. Transformations can be applied to entire world, single objects, or single primitives (points, lines and faces). The value of the axis should be specified in which the transformation is to be applied and which origin in the world has to be used at the time of transformation.

6.3.2 Lights, Cameras and Shading and Surface Characteristics

Overview of virtual lights and camera used for modelling technique and also the description of shading and surface options available in modelling software are given below.

6.3.2.1 Lighting and Shadows

Lights are used for illumination in 3D environments for focusing objects and are very important for the perfect realism of final virtual scene. When there is no light, the objects will looks like the following.

There are various types of light sources in 3D software; each of it gives different atmospheric situations. One can also control intensity and colour of light by manipulating it and give a different scenario according to user’s specification. Following are various types of lighting sources that are used in this study.

Directional lights. The directional light provides parallel rays. Distant light sources such as sun where lights are parallel and coherent are simulated using this light. Following figure gives the overview of directional light

Point light. The point light rays come from a single point, outwards in all direction. It simulates various light sources such as candle, bulb. Following is the figure of point lights.

Spot lights. A spot light looks like a cone that transmits its light in a specific direction. This is modified by using the cone angle.

Figure 2.7 shows all the different lights and its visualization in the environment.

Fig. 2.7
figure 7

Different light sources of the environment

Full size image

6.4 Interaction Implementation

Dynamic components in the environments are identified, and each are to be given the interaction effect with the help of blueprint or visual scripting in UE4. In UE4 game engine, blueprints are used to define object-oriented classes and add functionality to them like other scripting languages. Blueprints are created inside of UE4 editor visually, instead of by typing code, and saved as assets in a content package. These essentially define a new class or type of actor. Blueprints do not always need to contain scripted behaviour. Using the components present in the environment, blueprint can be made reusable to accelerate your level design process through event graph which is otherwise called as level blueprint. Modification in the class blueprint will have impact on complete project package. Event graphs contain the design-time and game-time behaviour of your blueprints. The event graph of a blueprint contains a node graph that uses events and function calls to perform actions in response to gameplay events associated with the blueprint. This is used to add functionality that is common to all instances of a blueprint. This is where interactivity and dynamic responses are set up. For adding functionality, model must be rigged to make it visually real.

6.4.1 Rigging

Rigging is like adding some skeleton in the static mesh, so that the model can move or can be deformed. Before animation, the model should be rigged otherwise proper animation cannot be availed. Figures 2.8, 2.9 and 2.10 are some examples of the process of rigging of the 3D model (http://www.autodesk.in/products/maya).

Fig. 2.8
figure 8

Liver

Full size image
Fig. 2.9
figure 9

Four views of liver

Full size image
Fig. 2.10
figure 10

Skeleton added to the liver

Full size image

6.5 Setting up Collision Among Components

In virtual environment , to avoid user’s walk through the static/dynamic models, setting up collision is important. Different types of collision detection techniques are available in UE4 game engine. Box collision technique is used in video games and other simulator, but here the collision is not proper in visualization, and computational time is more. Spherical type of collision is modified version of box collision where computational time is lesser than box collision but improper collision is still there. Convex hull collision is the most developed version of collision detection. Here collision is given according to convex hull algorithm that will detect all the boundary point of the model. Computation time of this algorithm is very high but still it is used everywhere due its efficiency.

7 Results

Three-dimensional modelling is done using GOOGLE Sketch up Pro. EOT crane operation simulation is the focus of the current work for which pictures, and video data are collected from a steel industry where EOT crane operation is needed for steel-making process (https://www.sketchup.com). Glimpses of component modelling are shown in Fig. 2.11.

Fig. 2.11
figure 11

a Caster, b nozzle

Full size image

7.1 Textured Models

For giving custom texture to the model, those must be segmented as per its different sections. Model segmentation is done in Blender software. Those sections are then given different colour for understanding that this section is different from the other section. After the segmentation of the model, relevant images have to be collected for giving texture. Then the collected image has to be converted in normal map, occlusion map and specular map. Those map are then integrated with the original image using visual scripting language. We can change various property of the images using unreal engine software. The projection view can be taken care of using normal map. Reflection map can be used to give shininess of the model. Occlusion map can then be integrated for giving the property where indirect light falls on the model. Figures 2.12 and 2.13 show the actual mesh and textured mesh of caster and slag-spot stand.

Fig. 2.12
figure 12

a Caster (without texture), b caster (with texture)

Full size image
Fig. 2.13
figure 13

a Slag pot (without texture), b slag pot (with texture)

Full size image

7.2 Developed Virtual Environment

The developed environment can be simulated using keyboard and mouse of the computer. The benefits of this crane simulator is operator can be trained using virtual simulator and make mistakes as much as possible that is not possible in real scenario. Also for providing training to operator in real scenario whole operational process must be stopped to avoid mishaps. Stopping the operation means a huge loss to the organization. But using this simulator, operator can be trained without stopping the whole operation as this training is carried out remotely in digital computer. The simulator can be enriched by using various sensors technology with this virtual environment. Specific operations in the developed environment can be simulated using sensor-integrated hand data gloves. And using haptic technology, we can also the feel the touch of each operations carried out in the virtual environment. We can mount Head Mounted Display for being immersed in the environment.

8 Conclusions

Dangerous, impractical and expensive scenarios in real world are achievable nowadays owing to the courtesy of virtual reality . Due to more and more use of these VR applications, its cost is going down and it is becoming mainstream where we can expect its further applications in occupational safety at manufacturing industries. Application of virtual reality technique in modelling and simulation of overhead crane enabled trainees to perform planned and repetitive operations in a risk-free environment. VR training will enhance the operator’s acquaintance with the workplace and different hazards present. It will be a boost for the operator’s confidence, and the developed simulator will reduce the training cost of crane operators. Hazard identification and prevention strategies can be thought of in the further work.