Keywords

1 Introduction

This paper is focused on the problem of the strain injuries in hands that occur due to repetitive execution of a certain hand movement, causing an excessive use in certain muscles or joints [1]. These injuries are mostly related to professional activities, especially in people that work at high levels of industrialization, patchwork or in use of advanced technology in productive process [2], as they are subject to a high rate of work where the performed tasks often lead to repetitive actions.

The main injuries in hands caused by repetitive efforts are the carpal tunnel syndrome, tendonitis and tenosynovitis (DeQuervain disease). These injuries are responsible for causing pain and functional incapacity on upper limbs, which involve tendons, muscles, joints, nerves and blood vessels [3].

In the literature there are studies on repetitive strain injuries in several fields and it was identified that one of the most used body segments were the upper limbs [4, 5]. There is a need to exercise them in order to prevent repetitive strain injuries, as well as to improve the quality life of the people in general and the working class in particular [5]. For this purpose classes of labour gymnastics were created. This consists in the performance of physical activities during working days in companies [6]. However, in addition to the recommendation of physical exercise or the labour gymnastics there is no method for the prevention of these injuries [3, 5,6,7,8,9,10].

The lack of awareness for prevention often leads the employers to believe that costs associated to prevention are unnecessary expenditure. However, in the United States of America the workers compensation costs $20 billion, and another $100 billion on lost productivity, employee turnover, and other indirect expenses [11].

The objective of this paper is to develop a sequence of serious games to prevent strain injuries in hands through stretching off and warm-up exercises. The game activities promote hands detection and their movements which are automatically detected by the proposed system. To accomplish the goal, it is first necessary to define the adequate exercises to prevent strain injuries through exercises on the wrists and fingers for each hand. Given the nature of the problem, the tests should be performed for a long period of time to detect results. To perform these tests there will be defined two groups with similar functions and workloads, where one group should perform this set of games and the other does not.

Being the wrist where all the nerves, tendons and blood vessels pass to the hand it is very important to perform an appropriate stretching. So, the stretching exercises should be performed at least twice a day, at the beginning and end of daily functions, as stated in [1]. Performing regular stretching exercises increases flexibility, and it may promote injury prevention and improve the recovery ability, in case of the worker eventually suffers from any injury [12].

The warm-up exercises activate blood circulation and warm up muscles and joints, as stated in [13], and will prepare the user for intense muscular activities [1].

The execution of these exercises are important, since the main reason for the existence of strain injuries results from the fact that the hands and wrists usually are not sufficiently exercised before performing a repetitive task [14].

This paper is organized in five sections, where Sect. 2 presents the proposed system. Section 3 refers to the games developed and the respective movements implemented. Section 4 presents the preliminary results of the detection of the hands with their tracking points in laboratory. The article finishes with the final comments.

2 Implemented System

Figure 1 presents the implemented system. The system consists of the Intel RealSense 3D Camera F200 and a personal computer. The user is placed in front of the camera to perform the warm-up and the stretching exercises to be tracked.

Fig. 1.
figure 1

Implemented System, adapted from [15]

Full size image

The Intel camera is responsible for the 3D hands detection and it has a color camera, infrared laser projector and a depth camera [15]. The effective range for gesture capture is between 20 and 60 cm, as stated in [16].

The computer is responsible for the game environment, data acquisition from the camera, and processing data to recognize the movements (the warm-up and the stretching exercises) made by the player as commands in the game.

Figure 2 shows all points tracked from the hand by the camera using the image processing modules from Intel. However, on the application only the red tracking points are tracked, therefore the frame processing is faster and lighter [14].

Fig. 2.
figure 2

Hand Tracking Points, adapted from [16]

Full size image

The game application is developed in Unity 3D software, a flexible software which allows game development in 2D or 3D, compatible with C# programing. It has available several free assets and it allows to target many devices more easily, as stated in [17, 18].

3 Developed Games

In this section, there will be discussed the implementation of the games developed.

To prevent the strain injuries, as previously referred, there are two sets of exercises that can be implemented: warming up and stretching off exercises, to the wrists and fingers for each hand, as stated in [5, 6, 15, 19] - these movements were defined by a physiotherapist. So, several games were developed, where the game controls are the movements corresponding to the warm-up and stretching exercises.

The use of serious games have several benefits, because they can be played on any computer, being a low cost solution, especially in companies where the daily use of computers is given. Moreover, in relation to the classes of labour gymnastics [4, 6], it has the advantage of not being necessary to create a space in the company for the performance of classes, neither the hiring of a teacher, besides being able to be used at any time of the day, without having to join the employees. Besides that, a serious game playing can also be a stimulating and entertaining activity [20].

3.1 Ping Pong Game

The Ping Pong Game, Fig. 3, is based on the classic game Pong, one of the first computer games. The game features are two paddles and a ball; the goal is to defeat an opponent by being the first to gain 10 points; a player gets a point once the opponent misses a ball, as stated in [21].

Fig. 3.
figure 3

Ping Pong Game

Full size image

In this game, the objective is to suffer the least number of goals in one minute of play. Since, the player controls both paddles with the movement of one hand, each hand controls both paddles during 30 s.

Figure 4 represents the movement correspondent to move the paddles, up and down, with the hand. This is the warming-up exercise for the wrist with the vertical movement.

Fig. 4.
figure 4

Wrist warm-up exercise, vertical movement, adapted from [15]

Full size image

With this game, it is possible to perform one of the several warm-up exercises, to prevent strain injuries.

To play the Ping Pong Game, the system needs the wrist point and the center point to move the paddles.

3.2 Space Ship Game

Based on the arcade game Space Invaders [22], the Space Ship game consists on a spaceship that is traveling through space and must avoid collisions with the meteorites that are in its way, Fig. 5. The player can also destroy these meteorites with the shooting weapons implemented on the ship.

Fig. 5.
figure 5

Space Ship Game

Full size image

So, to move the ship from one side to another, the player must do the wrist warm-up exercise with the horizontal movement, Fig. 6. To activate the shooting system from the ship, the player must do the exercise from Fig. 7.

Fig. 6.
figure 6

Wrist warm-up exercise, horizontal movement, adapted from [15]

Full size image
Fig. 7.
figure 7

Warm-up exercise, stretching and flexion of the fingers, adapted from [15]

Full size image

The player should control the ship for 30 s with each hand, to prevent destroying it. So, the player can perform both warming up exercises, one to the wrist another to the fingers, in both hands.

To play the Space Ship Game, the system needs the wrist point and the center point and, also need the tracking points corresponding to the fingertips. The wrist point and the center point are responsible for the ship movement, but the fingertips are needed to use the shooting guns implemented on the ship.

3.3 Hedgehog Invasion Game

The Hedgehog Invasion Game, Fig. 8, is based on the Angry Birds game [23], the Siege Hero game [24], and the Crush the Castle game [25], where the goal is to eliminate all enemies while destroying their buildings, with as few attempts as possible.

Fig. 8.
figure 8

Hedgehog Invasion Game

Full size image

To launch the hedgehogs, it is first necessary to close the hand in fist form and then to load the catapult. To do so, it is necessary to apply the movement shown in Fig. 9 several times. In the end, open again the hand to release the hedgehog and throw it against the construction.

Fig. 9.
figure 9

Wrist warm-up exercise, wrist rotation, adapted from [15]

Full size image

As it can be seen in Fig. 8, there are two catapults, so to launch the hedgehog from the right catapult it is necessary to perform the game exercise with the right hand and to use the left catapult it is used the left hand.

The player has five shots on each catapult to progress in the game levels. There are ten levels in case the player eliminates all the enemies of the level with a single shot.

To launch the hedgehog, the system uses all the tracking points because it is first necessary to recognize that the hand is closed in a fist form. Then, it is needed to use the wrist point and the center point to load the catapult; to recognize this circular movement it is necessary to recognize several positions on each lap. It is necessary to use all the tracking points to recognize the movement when the hand is opened to launch the hedgehog.

These game activities cover the warming-up exercises. Figure 10, presents some examples of the stretching exercises that will be translated in a game scenario.

Fig. 10.
figure 10

Stretching exercises, adapted from [15]

Full size image

4 Results

In this section there will be presented the results of the developed games. Figure 11 represents a 3D tracking result for both hands, where the hand with the blue tracking fingers represents the right hand and the other, the left hand. Moreover, the hand is represented in 3D, but during the execution of the games, the hand will be represented in 2D.

Fig. 11.
figure 11

Hands points tracked by Intel camera in Unity, adapted from [14]

Full size image

The three game scenarios were tested in laboratory environment. These preliminary tests allow evaluating system constraints and functioning. It is worth pointing out that it is still necessary to conduct more tests with all games with several people, to obtain a more consistent feedback.

In these first tests, the games presented a good gameplay, the commands were correctly detected and hands movements were correctly detected.

The use of both hands simultaneously (tested at the beginning) offered some detection problems, causing recognition errors. The game where this problem was most felt is in the game of Ping Pong, where one hand could control both paddles at the same time. To overcome this problem, all games were adapted to be controlled by one hand at a time. So, in both Ping Pong and Space Ship games it was necessary to create a playing time for each hand, corresponding to 30 s each.

Figure 12 shows a game scenario of Ping Pong. On this position, the tracking points are directing the paddles to go down. The central value is the game time and the score of the game is given by the remaining two numbers.

Fig. 12.
figure 12

Ping Pong Game scenario

Full size image

In Fig. 13 it is being played the Space Ship Game and the tracking points are in position to activate the firing command, because to firing the fingertips must be closer to the center point. The player’s score is written in yellow, and the ship’s health in blue.

Fig. 13.
figure 13

Space Ship Game, shooting result

Full size image

In Fig. 14, due the tracking points location, the ship is moving to the left position, to avoid collision with meteorites.

Fig. 14.
figure 14

Space Ship Game, movement result

Full size image

In Fig. 15, the hedgehog was thrown against the enemy buildings, beginning to overthrow them.

Fig. 15.
figure 15

Hedgehog Invasion Game, launching result

Full size image

5 Final Comments

The objective of the work presented in this paper was to design and develop a sequence of serious games where with the hands detection and their movements it is possible to prevent strain injuries through warm-up and stretching off exercises.

The games were developed, tested and optimized in laboratory environment. The next step in the research is to design and implement the stretching exercises and some other warming-up scenarios. When all the game scenarios are implemented, the overall system will go under extensive tests for validation. Moreover, there will be used a questionnaire in order to evaluate the comfort of the users in the daily work and about their game experience.

The growing concern about strain injuries caused by professional activities leads to a higher pursuit for solutions to perform labour gymnastics in companies. So, the implementation of this type of games can be an asset, as it can lead, in a more fun and motived way, to a reduction or even help to eliminate strain injuries in employees.