Keywords

1 Introduction

Currently the teaching-learning process uses new emerging technologies such as: virtual reality and haptic devices [1]. These components have participated in various studies, having a high degree of success and usefulness [2]. Haptic devices are used in different fields such as: rehabilitation through virtual games and education, which allow to attract the attention of the user in order to develop the tasks in a proper way [3, 4]. In this context, there are jobs related to this topic, in [5] presents a multi-touch panel that includes virtual reality in games for infants with a dynamic interface to make drawings and identify figures. In [6] shows the interaction between virtual reality and the visual haptic device that allows to increase the fluency in the production of handwriting by hand in kindergarten children. A virtual reality game to improve the performance of students in the area of mathematics is presented in [7].

The development of the virtual system has three main features: the simulation of an environment, a didactic and interactive element and an immersive component. The immersion is integrated through haptic technology, which has been used, for example as a guide in systems of navigation and feedback of forces in which the touch sense allows the interaction between the user and the system [8,9,10].

The basis for educational development must be fostered from the home to the school, all with new technologies that allow structuring levels such as: interaction, which is the capacity of the human being to interact with a virtual environment. The immersion must have as a primary goal to capture the full attention of the user, which with its imagination will seek that the experience in virtual reality resemble as much as possible to the reality. The virtual tools applied to children present results of great validity, because they have high motivational degree and interaction [11,12,13]. In the development of educational systems have highlighted virtual applications aimed at teaching and didactic guide in the management of processes as it appears in [14,15,16].

Virtual reality (VR) allows to create realistic and dynamic images, which acquire sensory information that allow the interaction with the 3d virtual model [15]. This is presented in applications oriented to different purposes, from entertainment to interactive teaching for engineering or medical training [17, 18]. It has also been found that the application of VR for teaching increases the interest of students, either by the novel of the technology or by the challenges that may arise during the use of the application which motivates the user to finish the task in a correct way [19].

The children have a great cognitive capacity since the first years of life which allows them to learn quickly from their surroundings. By applying the appropriate stimuli to them in a continuous teaching process is possible to develop their skills quickly and upgraded [20,21,22]. Initial education is intended to be based on the best techniques and methods of teaching basic knowledge in early ages [23]. The methods for taking a proper initial education are varied, but all of them require resources that are attractive to the students, from colorful images to interactive games and much more, therefore, the qualitative method is very important in validating the learning process in children of young age [24]. At present a considerable percentage of students present a lack of interest in the conventional methods of didactic teaching taught by the teacher, being necessary the integration of new tools to capture the attention of the students in the classrooms [25]. In this context, the integration of VR into systems for the initial teaching process can provide great educational benefits for both the student and the teacher. The use of haptic devices offers an improved experience when interacting with the environment, the new advances of these devices have made it possible to stimulate other senses in people, besides the visual and auditory [26], which would allow to cover some methods of motor learning in the teaching of children.

With the aim of updating the methods of teaching of initial education, this work proposes the creation of a virtual tool didactic, for the teaching-learning of basic knowledge on: primary colors, secondary, strokes to realize the writing of basic figures, uppercase and lowercase vowels of the alphabet. The system is implemented through the use of interactive virtual reality interfaces and controlled through the analysis of signals from a haptic device.

The present work is divided into five sections, including the introduction, Sect. 2 methodology used for the development of the system, Sect. 3 multilayer development plan, Sect. 4 developed environments, Sect. 5 use mode, Sect. 6 shows the tests and results and, finally, Sect. 7 conclusions.

2 Methodology Used for the Development of the System

This section describes the stages of the virtual system as shown in Fig. 1.

Fig. 1.
figure 1

Operative Scheme

Full size image
  1. A.

    Input peripherals

The virtual system uses the haptic device Geomagic Touch as an input peripheral. It has 6 degrees of freedom and digital encoders that allow the user to move into the three spatial dimensions and translate these positions into readable variables for the system, the communication used is Ethernet.

  1. B.

    Script development

Scripts developed in c# language in visual studio are responsible for system administration and the control of outputs depending on input. The control algorithm of each game starts with the acquisition of signals, manages the required information according to the interaction interface, runs, and if the object has weight or collides with another object, the haptic device supplies a feedback of forces as shown in Fig. 2. The scripts are compatible with unity and communicate with the haptic device geomagic touch via plugins. This allows its manipulation by making the times of a pointer, also control the audio output, for which, starts with the selection of the desired game. An audio feedback is obtained with the instructions and the objective that the user must reach in the environment, indicating at the same time if the actions performed are correct, incorrect or if it should improve.

Fig. 2.
figure 2

Flowchart of information management in the virtual system

Full size image

The validation and trajectory-generation controls have been developed in MATLAB software, which, through interaction with unity, receives the information necessary to send a response to the haptic device or to unity, as the case might be. To validate the input data, classifiers are used, which allow to calculate the Euclidean distance, to validate the stroke made by the child with a pattern stroke. The goal of the trajectory generation control is to get the haptic geomagic Touch device to move according to a variable signal in the time received from Unity, a sinusoidal signal for instance. For this procedure, a PID control is applied to each of the device’s position inputs.

3 Multilayer Development Plan

This section describes the multi-layered scheme for the development of virtual applications of a teaching-learning system in children of initial education. This schema presents in an orderly manner the development of the application, has the following layers: (i) Layer 1 preselection of materials to be used in the objects that compose the interfaces; (ii) Layer 2 2D and 3D object editing; (iii) Layer 3 development and implementation of the virtual environment in Unity; (iv) Layer 4 Script development and interaction with virtual interfaces; and finally (v) Layer 5 data acquisition for the validation of the educational activities carried out by the child, as shown in Fig. 3.

Fig. 3.
figure 3

Multi-layer diagram

Full size image

Layer 1. This layer defines the resources and materials used in the design process, based on the theory of Vygotsky, which indicates that children acquire their knowledge through games related to their daily environment. To which you have: images that relate colors with objects (orange color-image carrot), vowels with images of animals (the vowel ‘a’ with a picture of a spider), sounds, audio of basic instructions, colors and textures.

Layer 2. Design of the objects used in the virtual environment, through the Fireworks 8 software and taking into consideration the resources of the previous layer, 2d drawings are designed; Textures, palettes and shapes highlight their details to make them attractive to the user. 3d objects are built with the blender software, giving realism to the object by implementing meshes, rendering and creation of bones in a hierarchical way that allows to give life to it through the animations provided by Unity.

Layer 3. All resources to be used are exported to a working environment in 3d Unity. Each virtual object is assigned the respective properties according to the function that must be fulfilled within the design of the game. Each element that could be touched has different features to provide the sense of reality to the environment. Materials used in the writing area imitate the shape and color of an academic blackboard. The other elements in the environment are presented with colors and shapes that are pleasant and appealing to children. All games have audio to explain the procedures and indicate if the task is done in the correct way to orientate the child during the development of the activity. There are animations in the Unity environment which help the user in the development of the task and dynamize the interfaces.

Layer 4. The reproduction of the programs is done through the scripts of each interface, they run the script to start each game with their respective elements, they also allow the functions to be handled in an organized way to link the different environments and access each one. Once started the game these scripts activate the corresponding audio source to make known the objective of the activity to be realized. The scripts contain the plugins for the use of the haptic device as input and output of the system. They accept the input data which are evaluated according to determined parameters and depending on these one has a feedback of forces of the haptic device.

The development of writing in the virtual environment is done editing the texture of the blackboard object. By using several scripts is calculated the position of the cursor on the 3d element and the force applied to it, with these calculations modify the color of the original texture at the exact point where the cursor is placed multiplying by a certain radius and establishing the definition of the stroke on the blackboard that will be proportional to the force applied. It means with a small force applied the stroke will be very subtle, and with a stronger the stroke will be very definite and with more color. The environment has a reset button to return the original texture to the blackboard object when you want to delete the strokes made, as well as you can choose between seven different colors to do the writing.

Layer 5. Finally, in this layer there are two controllers that meet different functions. The first one is a controller by classifiers which performs the evaluation of the strokes made, checking the trajectory followed and similarity of the stroke with a pattern. It is found both in writing of the vowels of the alphabet as in writing of initial strokes. The second controller is used for the generation of movement in the haptic device, while the child holds the manipulator of the geomagic, it moves on a surface (blackboard) indicating the movements that the child should perform with his hand to do the stroke. This process uses a PID control [27] that is applied on the output of the geomagic to carry out the feedback of forces.

4 Developed Environments

Three different games were designed with their results interface. In the application of secondary colors are used 2D and 3D objects. It presents a palette of colors that can be perceived by the haptic device as a real object, also there are figures of vegetables and fruits to be colored, these are only visible to user, but you can’t touch it with the cursor. The writing games show an object similar to a blackboard on which the haptic device simulates writing, there are some buttons with different functions. There is an avatar with animations in all environments and each application has a second interface where the results of each game are indicated, as shown in Fig. 4

Fig. 4.
figure 4

Environment development (a) Secondary Colors Interface, (b) y (c) Figures drawing Interface and (d) Vowel Writing Interface

Full size image

5 How to Use

The application displays a main menu, it has 3 games with the following parameters: secondary colors, stroke drawing and vowel writing respectively.

In Fig. 4 the virtual environment of the first game is shown where the theme of this game are the secondary colors. It starts with an introduction of the combination that has to be performed between the primary colors to form each one of the secondary colors (Fig. 5(a)); it presents an illustration with the silhouette of a vegetable or fruit (Fig. 5(b)), the audio mentions that type of vegetable or fruit and its color, asking the child to select the respective primary colors to form the requested color. When the correct colors are chosen, the illustration is colored and continues with the next secondary color (Fig. 5(c)), the audible response indicates when the selected color is correct or incorrect, objects appear and disappear according to the color secondary to be colored. Once the game is complete, a button appears that when pressed, it leads to a different environment where the child’s performance during the activity will be shown (Fig. 5(d)).

Fig. 5.
figure 5

First game virtual environment (Color figure online)

Full size image

In the second game there is a surface as an academic blackboard in order to the child can learn to make the first strokes as an initial help for the learning of writing. There are five buttons to the left of the blackboard, which are chosen according to the stroke of the desired figure to practice (circle, heart, spiral, wave and straight line). When one of the buttons is selected an audio sounds, which in conjunction with the animations, explain how the stroke should be performed (trajectory and starting point). On the right side of the blackboard there are five more buttons, the first one is for trajectory generation. The following buttons are complete, delete, menu and results.

For the third game, the upper and lowercase vowels must be written. Like the previous game, the auditory feedback and animations are presented to indicate the trajectory to follow in each letter. Each vowel is chosen with the buttons on the left and on the right, there are buttons for ending, deleting, menu and results. Similarly, colorful animations are presented at the end of each letter correctly.

6 Tests and Results

  1. A.

    Test

The system should always be used under the supervision of the teacher or tutor’s child, who will evaluate the performance of it. The main menu of the game is presented where the teacher in charge selects the game that he believes is suitable according to the age, knowledge or skills of the child.

In the first level the child hears the way the secondary colors are formed and proceeds to perform the task (Fig. 6(a)). When the selection is incorrect, it is sent a denial hearing response (Fig. 6(b)) for that the child understands and chooses the right color. Once this is done, the silhouette is colored on the screen with the resulting color (Fig. 6(c)) and continues with the next color until the game ends (Fig. 6(d)).

Fig. 6.
figure 6

First game used by the user

Full size image

For the development of the second level a button is pressed to perform one stroke at a time. When the desired figure has been chosen, the indications are heard of how the stroke should be performed (Fig. 7(a)). By pressing the button to view the tutorial, the scene changes and is possible to perceive the displacement of the haptic device following the path of the stroke to be written (Fig. 7(b)). The child proceeds to complete the dotted lines forming the figure (Fig. 7(c); If the stroke is performed correctly, a positive auditory response is heard and the respective animations in the environment (Fig. 7(d)). Otherwise, the audio indicates to the child that try to do it again.

Fig. 7.
figure 7

Second game used by the user

Full size image

In the last game the vowels must be written (Fig. 8(a)). Just as in the previous game the sounds along with the animations show to child how the vowel should be written (Fig. 8(b)). The child performs the layout of each vowel and at the end presses the button with the tick (Fig. 8(c)); The system evaluates the trajectory and the order, if they are correct it presents the animations and positive sounds (Fig. 8(d)), when one makes a mistake the child is told to continue practicing.

Fig. 8.
figure 8

Third game used by the user

Full size image
  1. B.

    Results

The results tables for each of the games are presented in the next section. In Fig. 9 the evaluation of the game of the secondary colors showing the successes and failures are indicated, as well as the respective qualitative qualification.

Fig. 9.
figure 9

Game results secondary colors

Full size image

In the second game the user chooses between: (a) learn how to perform the stroke or (b) perform the stroke, to which if the user chooses option (a) a trajectory control is executed, in which two aspects are experimental: the trajectory described by haptic device geomagic touch (Fig. 10(a)), and the control errors (Fig. 10(b)), where you can see that the errors tend asymptotically to zero.

Fig. 10.
figure 10

Trajectory Control

Full size image

For the results corresponding to the second game is necessary to do a control by sorter which compares the similarity of the stroke made by the child with the data of a pattern, in Fig. 11(a) The pattern of the heart figure (blue Line) is indicated along with the child’s stroke (red color line). In the same way for the third game the same control by sorter is applied, as shown in Fig. 11(b) the pattern and the stroke made by the child in the vowel ‘e’ lowercase are shown. Once the comparison is made the control reveals a numerical result, this value varies according to the type of stroke (if it is a capital, lowercase vowel or a figure) that is performed, this result is transformed to a qualitative qualification (form of qualification Managed by the initial education teachers) with the following evaluation parameters: A, EP and I (see Fig. 12(a)) and (Fig. 12(b)); where they correspond to acquired (very satisfactory performance), in process (satisfactory performance) and insufficient (incorrect performance), respectively.

Fig. 11.
figure 11

Validation of letters and figures using a control by classifier (Color figure online)

Full size image
Fig. 12.
figure 12

Results (a) game strokes basic figures (b) game vocal stroke

Full size image

7 Conclusion

The teaching-learning process for children of initial education can be complemented with the virtual system presented in this article, and it is very useful as a support for the teacher. In order children to focus their attention on the virtual environments designed. It must be eye-catching and most of their objects should be related to elements of daily life. As well as it is necessary to get the attention of the infant and encourage him to learn with the use of audio and animations within the virtual environment. The motion control developed in MATLAB allows to execute a trajectory tracking in the haptic device geomagic Touch and helps to develop skills for the writing of the children.