Keywords

1 Introduction

Respiratory function is sometimes affected by hereditary genetic diseases or problems, but today, gas emissions and chemicals generated by industrial development and urban growth produce harmful agents that affect people’s health. These factors cause allergic manifestations that cause damage to the organs of the respiratory system i.e. nasal cavity, pharynx, larynx, trachea, bronchi and lungs, so there are a greater number of people with respiratory affections e.g. asthma, rhinitis, pulmonary hypertension, sinusitis and others [1,2,3]. The treatment of patients is through medication e.g. antibiotics, corticosteroids, anti-inflammatory drugs, or surgical procedures, the same ones that are performed in specialized medical centers by a team of professionals such as pulmonologists, nutritionists, physiotherapists, among other experts whose aim is to improve people’s quality of life [4].

Respiratory rehabilitation is an alternative that complements medical treatment for patients of all ages and includes a range of techniques and instruments applied to reduce the symptoms of respiratory disease [5], including the following: (i) FR respiratory physiotherapy, is a set of techniques that are applied so that the patient can eliminate secretions from the respiratory tract, as well as control breathing to improve pulmonary ventilation; (ii) mechanical instruments as well as physiotherapy are applied to eliminate secretions and clear the respiratory tract, generally applied in cases where the patient has more difficulty to perform the activity on his or her own and requires assistance through these instruments e.g. flutter, cornet, intrapulmonary ventilator; and (iii) exercise as part of any treatment, physical activity is a constant recommendation by physicians, in this case it favors the functioning of the cardiovascular and respiratory systems, and should always be supervised by the treating physician [6, 7].

The application of the different types of respiratory therapies depends on the respiratory physiopathology, age and psychoemotional state of the patient. In the case of young children and infants, the application of mechanical instruments is the most commonly used alternative [8]. FR is a technique that involves the patient, family members and the physiotherapist, who work together to correctly execute the exercise sequences, providing the patient with comfort and avoiding muscle fatigue and breathing difficulty. However, because FR requires prolonged sessions during treatment, it can be observed that children have difficulty maintaining concentration in their activity. For this reason, it is necessary to propose an attractive and entertaining way to assist the children in each FR session and to facilitate the physiotherapist in the execution of the therapy [9, 10]. One of these forms is through games, which are currently an alternative in several areas where children work with e.g., health, education, psychology among others.

The insertion of new technological trends in different areas of medicine makes it possible to propose alternatives to traditional medical treatments, an example of which is the application of virtual reality VR in various areas of health such as: (i) traumatology where VR and haptic devices are applied to recreate similar scenarios to the real ones for the execution of rehabilitation sessions of the upper and lower extremities in patients with motor problems due to injuries or degenerative diseases [11,12,13]; (ii) paediatrics with the development of VR applications based on games for the therapy of children with cerebral palsy [14, 15]. In [16] a virtual reality program is presented to analyze the emotional consequences caused by the treatment of catastrophic illnesses in hospitalized children and adolescents; (iii) psychology has been one of the most developed areas for the treatment of different illnesses e.g. application of virtual environments to conduct studies on eating disorders [17]. For the treatment of phobias, where the patient is immersed in a real environment according to the type and degree of phobia and where it is possible to record the levels of biometric signals to be evaluated by the specialist during therapy [18, 19]. VR applications to reduce the level of anxiety in cancer patients prior to receiving chemotherapy sessions [20].

For this reason, this work proposes an application in a 3D virtual environment in the field of pneumology to perform FR as part of conventional clinical treatment in children with respiratory diseases and/or pathologies. For this purpose, games will be developed in virtual environments where the patient performs different exercises that, depending on the inhalation and exhalation time, allow him to complete the activity and move on to the next exercise until the therapy session is completed. These exercises will be supervised by the specialist, who will guarantee the functioning of the developed application.

This paper is divided into the following sections: Sect. 2 describes the methodology used for the development of this application for FR; Sect. 3 presents how the FR system is structured; Sect. 4 indicates the results obtained using the application developed and Sect. 5 indicates the conclusions of the work developed.

2 Methodology

The work consists in the development of an application for mobile devices or virtual reality devices, which will support the performance of three types of breathing exercises, using a non-invasive data acquisition system and a virtual reality graphic environment to motivate the patient to perform the therapy correctly.

The exercises selected for therapy consist of: (a) one nasal and one oral aspiration and one oral exhalation, (b) two nasal and one oral exhalation and (c) three nasal and one oral inhalation and one oral exhalation; when these exercises are performed, a displacement of air into and out of the body is caused, thus generating a sound. In this case, this is the signal to be picked up by means of a microphone, constituting the patient’s data signal that will be used to carry out the comparison with the standard signals, Fig. 1 shows the signal acquired in each of the exercises.

Fig. 1.
figure 1

Signals acquired by the microphone

Full size image

These signals show the presence of certain patterns such as the presence of peaks in aspiration and exhalation, and the lack of synchronization of each of them. The treatment of the information will be carried out through a process involving the criteria of amplitude and time. The technique used is the Dynamic Temporal Alignment, DTW; in addition, this lack of alignment in the acquired signals does not obey a fixed law, that is, a constant delay, but it occurs in a heterogeneous way, producing localised variations that increase or decrease the duration of the analysis section. The associated problem refers to the added difficulty in the process of measuring the distance between standards, since sections that may correspond to different units will be compared. It will therefore be necessary to temporarily align the sweeper to proceed to a distance measurement between patterns whose new time axis has homogenized the initial variations. Therefore, this process will allow the comparison of each exercise with its respective pattern which was obtained by processing 100 samples of each of the exercises, calculating the DTW of the formed matrix and obtaining the average value of the matrix whose value is the DTW distance to be taken as a reference for this application using a threshold of 80% of maximum coincidence of tolerance to validate that the exercise of the therapy is correctly performed. Figure 2 shows the signals taken for each of the exercises and their respective patterns for comparison and validation.

Fig. 2.
figure 2figure 2

Representation of input signals and patterns, with vectors X and Y of the DTW calculation between both signals

Full size image

3 System Structure

The system proposed in this work considers that the user enters the virtual environment where the instructions are indicated and the exercise to be performed is modeled, the user performs the proposed exercise when acquiring the signal the system compares it if it has a coincidence level greater than 80% it is given as a satisfactorily completed exercise which allows him to continue with the remaining two exercises of the therapy will allow a maximum of 3 erroneous repetitions per exercise to avoid hyperventilation in the patient. The results of the exercises are stored in a database in order to generate a report with the exercises performed, the time the exercise lasted, the date, time and patient data. The application developed for this experimentation consists of 5 stages: Inputs, processing, scripts, graphic platform and output, see Fig. 3.

Fig. 3.
figure 3

System diagram

Full size image

The input stage synthesizes the data to be acquired from the environment. In this case, the microphone is the sensor to be used to capture the information which will be recorded and entered by the device’s audio card.

The data processing is carried out in several steps. The data acquired by the sensor is stored in a vector, this vector is compared with the previously trained pattern, the technique used for the comparison is the DTW implemented using the Matlab software, the result of this comparison is transmitted to the script developed in Unity for the control of the graphical interface, this data is communicated through a TCP/IP port in real time.

The scripting stage is made up of the following modules: (i) APP Controller unit responsible for processing, managing and storing the input and output data of the system. This driver is associated with the modules; (ii) Audio Effects which commands the sound effects of the virtual environment; (iii) Operations Control manages the commands registered within the system for interaction with the user defining the activities to be performed; (iv) Report Creator records the data generated by the user in its interaction with the system; (v) Data Structure manages the data structure that comes from the I/O communication ports; (vi) Steam VR Plugin manages the data entered by the controls or commands into the VR environment and communicates with the APP Controller; (vii) Training is the module that manages the training instructions. This module is trained with the conditions required to carry out the exercises; (viii) Activity Checker allows you to compare the proposed exercise by classifying the exercises according to the previously established parameters.

The graphic stage, which contains the environments in which the exercise will take place, consists of several Game Objects: Audio G.O. which is an audio source that generates sounds for the environments where the activity takes place; Avatar G.O. which highlights the avatar that represents the user; U.I.G.O. allows you to navigate within the application and select the training scene or exercise activity; VR Device represents the virtual reality glasses within the environment; the Training Scene and Activity Scene modules are the training events prior to the execution of the therapy exercise to develop the sequences present visual characteristics both in time and space and allow to continue to the next exercise if the therapy exercise was correctly completed and the Virtual Reality Environment module is the Home screen where you can choose one of the two scenarios for the start of therapy.

In stage 5 there are: the Audio Results output where the sound is originated, which gives more realism to the virtual environment, the Headset Display module that allows the visualization of the environment in response to the inhalation and exhalation activity on the microphone at the beginning for the execution of the therapy and in the Data Report where the recording of the informative data of the activity carried out by the user is recorded.

In the virtualization process, replica 3D models of real models such as luminaires, bridges, benches and others are built to include them in the application environment. The sequence for the design of the environment is shown in Fig. 4.

Fig. 4.
figure 4

3D object construction diagram

Full size image

The design of the virtual environments uses the 3D models replica of the real models for which CAD software is used, in this case SolidWorks was used. Once the 3D objects are obtained, their properties related to the axes, orientation and hierarchy of elements are modified through 3ds MAX, which provides agility in the manipulation and location of the objects in the virtual environment. The generated file is imported to the 3D Unity in.fbx format where the texturization and programming for the use of each object within the virtual environment is carried out. As a final product, a virtual representation of the real object in functionality and aesthetics is obtained.

The entire game environment has been developed with the unity tools because it offers a good tool to make realistic scenarios for the app. The principal environment features (see Fig. 5) used to develop the app are the next: (i) Ambient Lightning place light points in strategic places to feel sense of a road to follow, (ii) Heightmap using raise/lower terrain modeling tool and a brush style from the options panel it’s possible to make some mountains and small valleys in the environment, (iii) Forest Texture using the resources of Environment package it’s possible to assign textures and grass, soil, rocks and so on, (iv) Trees there are many types of trees inside the environment package that will be placed manual or randomly in a specific area, (v) Street Furniture its necessary place bridges, light posts, park benches and others from a CAD-CAM (SolidWorks) or 3D modelling software (3dsMAX, blender), (vi) Water effects to bring the effect of a small river through the park, there are a prefab inside the Environment package.

Fig. 5.
figure 5

Game environment develop

Full size image

4 Analysis and Results

This section presents the execution of the preliminary tests using the developed app, and also shows the results obtained after the use of the application. The system developed allows the evaluation of respiratory physiotherapy (RP) techniques aimed at children with lung disorders, in order to strengthen the performance of the therapy in an entertaining and motivating way for the patient. The special characteristics of respiratory physiotherapy in children suffering from lung problems are based on the activity-disposition relationship, due to the complexity of maintaining the child’s readiness with the activity of the therapy to be performed.

The application shows an open world environment in which the patient must perform FR micro-tasks to interact with the environment and unlock new sequential missions in the game. The HTC VIVE headset connected to a VR Ready computer (see Fig. 6) with Core i7-6700HQ processor, GTX1070 video card and 16 GB RAM are used for testing. The headset HTC VIVE was elected because their workspace of 12 m2 that enables free movement in any direction by: walking, running, bending and jumping in virtual reality. These features bring immersive experience to the patient that make exercises without think in make exercises of therapy. The software of HTC VIVE has a plugin to connect directly to unity and execute the program while is in development to check details in VR without build all project in an executable file.

Fig. 6.
figure 6

HTC VIVE Glasses and VR Ready laptop

Full size image

When the application is opened, the Home screen is displayed, showing the name of the application and its opening to the other screens as shown in Fig. 7. The application has constant visual and auditory support that allows the user to interact intuitively with the application without the need for a third person to constantly guide the user. On the Training screen the user receives a tutorial to perform the 3 exercises that will be asked of the user, see Fig. 8.

Fig. 7.
figure 7

Home screen of the app

Full size image
Fig. 8.
figure 8

Training screen of the app

Full size image

In the next level the user appears at the starting point of the forest where he can follow the visual and auditory instructions that guide the user to the next point where he will have to perform a task to complete the passage through that place. As a first point, exercise N°1 is performed (see Fig. 9). If the execution is successful, the user is allowed to advance to the next point; if the execution fails, the user is encouraged to perform a second time; if he or she fails the test, the system will enter a process loop until the user succeeds in performing the task.

Fig. 9.
figure 9

Execution of the first exercise

Full size image

The exercise performed is checked in Matlab with the controller running at the same time as the application. If the Euclidean distance and the result of the DTW algorithm are within the training values see Fig. 10, it is resolved that the exercise signal matches what was requested in this favorable case, which visualizes the mission achievement animation. For an unfavorable case, the algorithm is repeated until the user enters a signal within the parameters of the classifier.

Fig. 10.
figure 10

Resulting data in Matlab for exercice N° 1

Full size image

As a second point, there is a problem with a luminaire, which does not work properly. The user’s task is to repair the luminaire by repeating exercise 2. Once the luminaire has been fully repaired, the patient is compensated for the advance to the next point, see Fig. 11, the analysis performed in Matlab is shown in Figs. 12 and 13. The optical response inside the virtual environment are shown in the Fig. 14.

Fig. 11.
figure 11

Implementation of exercise 2

Full size image
Fig. 12.
figure 12

Comparing two respiratory signals in Matlab

Full size image
Fig. 13.
figure 13

DTW results of the exercise 2 in Matlab

Full size image
Fig. 14.
figure 14

Graphical response made after the successful completion of exercise 2

Full size image

As a next activity, the user is shown the need to continue advancing in the points of interest of the environment. The user is asked to perform exercise 3 which consists of 3 breaths and 1 exhalation to continue to the virtual environment bridge as shown in Fig. 15. The graphical results are shown in the Figs. 16 and 17.

Fig. 15.
figure 15

Hearing of the request for implementation for the exercise 3

Full size image
Fig. 16.
figure 16

Comparing of respiratory user signs with the reference sign in Matlab

Full size image
Fig. 17.
figure 17

DTW results of the implementation of the third exercise in Matlab

Full size image

As the user progresses within the virtual environment, the system stores important session data such as: runtime, number of successes and failure to perform tests. These data will be available to the physiotherapist who is responsible for interpreting and verifying the progress of the therapy. The activity is validated by the completion of the task according to the development in the execution of the exercise performed by children with lung disease in the execution of the RF according to the activity indicated in each of the exercises, and the level of user satisfaction in the use of the application developed is also established.

4.1 Task Fulfillment

The techniques and strategies used are based on the therapist defining the objective of the therapy and independently activating the stage according to the established inhalation and exhalation times and at the end of the three stages of established exercises, the therapy will be evaluated verifying the degree of compliance with the activity performed by the children.

For the evaluation process, the group consisted of 10 children with lung disease problems between the ages of 6 and 10 years old who have the same level of disease. The indicator to be validated is the fulfilment of the activities defined by the therapist, which are indicated in Table 1.

Table 1. Parameters to evaluate.
Full size table

According to the data, it is observed that the evaluated group registers a greater compliance with the parameters, which indicates that the work carried out in the virtual environment of the therapy is interactive and the attention of the child is constant due to the presence of virtual scenarios, whose purpose is to guide the child in the process of compliance with the therapy in a correct way, see Fig. 18.

Fig. 18.
figure 18

Results of task accomplishment.

Full size image

As the application is fun, it can be observed that it has a great acceptance by the user since it allows him/her to interact in virtual environments different from the traditional ones, such as the medical offices, maintaining the concentration and mental stimulation during the process. Therefore, applications of this type developed in virtual environments encourage the realization of the therapy, improving the recovery of the patients.

5 Conclusions

The results obtained from the implementation of the virtual reality system for the assistance in the treatment of respiratory disorders in child patients prove to be efficient for the rehabilitation treatment under the supervision of the therapist, since it allowed the patient to maintain concentration and perform the therapy in a very entertaining and productive way. The system developed considers three stages as virtual games that adjust to the degree of inhalation and exhalation acquired in real time from patients through the HTC VIVE device, which allowed the patient to interact with the respective exercise for therapy in the virtual environment.