1 Introduction

Knee osteoarthritis is a prevalent condition among the senior population that significantly impacts their quality of life, causing knee function limitations and pain [1,2,3]. In China alone, the incidence of KOA is 8.1%, with approximately 120 million individuals suffering from knee osteoarthritis pain [4]. As the aging population continues to grow, the prevalence of KOA is expected to increase, reaching 50% among people over 60 years of age [5]. Exercise therapy is a crucial rehabilitation method for KOA patients and is highly recommended as a cornerstone of prevention and treatment in major guidelines [6, 7].

Typically, rehabilitation physicians determine the appropriate type and intensity of rehabilitation exercises based on individual pathology and physiological conditions, providing exercise recommendations in the form of exercise prescriptions. However, the rehabilitation process for KOA patients often lasts at least 8–12 weeks, and many patients do not have the necessary conditions to undergo long-term training sessions in a hospital setting, making home-based self-rehabilitation a more practical approach [8]. Nevertheless, accurate implementation of exercise prescriptions is essential for achieving health benefits. Without the supervision and guidance of a physician, patients may perform exercises incorrectly, engage in insufficient or excessive exercise, leading to reduced training effectiveness or even secondary injuries [9]. Moreover, many patients find exercise monotonous, which diminishes their motivation to initiate training and comply with exercise prescriptions. According to a study, only 31% of rehabilitated patients adhere to the exercise routines recommended by their physicians [10]. Thus, enhancing patients' motivation for active rehabilitation is also crucial.

Exergame, which combines games with exercise instruction, leverages gamification elements such as interactivity, game tasks, rewards, and entertainment to encourage patients to engage in exercise [11, 12]. Research has shown that exergames have significant effects on motivating exercise [13], improving cognitive function [14], and increasing physiological energy expenditure [15]. For example, O'Connor et al. utilized video games to enhance response and motivation physiologically in people suffering from spinal cord injuries. It is noteworthy that 87% of the participants reported that these games effectively motivated them to engage in physical activity [16]. Betker et al. also demonstrated that exergame environments motivated participants to engage in dynamic exercise tasks [17]. Exergames have demonstrated beneficial effects on physical and mental health in older adults as well, improving physical function and motor capacity [18,19,20].

Combining exergames with motion capture technology further enhances the precision of motion guidance and the enjoyment of the game. Motion capture technologies such as Nintendo Wii, Microsoft Kinect, and other motion sensors can capture patient gesture data, enabling the acquisition of joint characteristics and movement recognition. In recent years, motion-capture-based exergames have been increasingly utilized in physical medicine and rehabilitation [21,22,23,24]. For example, Gillian Barry et al. successfully employed exergaming to promote balance training in healthy adults, yielding positive motivational effects [25]. Ying-Yi Liao et al. utilized Kinect-based exergames to instruct the elderly in performing aerobic exercise and balance training, effectively improving their frailty [26]. Shih Y H et al. developed a Kinect-based flight simulation control game to assist patients in exercising their neck and shoulder muscles, utilizing real-time human joint positions translated into control commands for flight simulation games [27]. Chen M et al. designed an exergame system to facilitate lower limb muscle exercise in the elderly, reducing the risk of falls by acquiring patient movement information for game control and movement assessment using Kinect [28]. Chi-Min Yang et al. also verified the feasibility, safety, and effectiveness of Kinect-based exergames in guiding exercise for the elderly. Exergames are also beginning to be used in knee rehabilitation [29]. Donald G Manlapaz et al. employed Nintendo Wii Fit™ for exergaming and designed an eight-week exercise program to promote exercise in KOA patients, effectively reducing the risk of falls [30]. Chung-Ho Su designed a motion-capture system based on Kinect technology for the rehabilitation of sufferers who underwent total knee replacement (TKR), using a bicycle boarding movement as an exercise motion for joint mobility. The system recognized pedaling motions with Kinect to drive a game character forward, while body swaying controlled character turns and obstacle avoidance. A controlled trial demonstrated that exergames could enhance postoperative exercise intentions in TKR patients with significant effects [31].

In summary, exergame based on motion capture technology provides an effective way for rehabilitation training. Rehabilitation training for KAO patients is a gradual process. The type, frequency, intensity, and duration of rehabilitation training vary significantly from patient to patient or in different rehabilitation stages of the same patient. These characteristics imply that rehabilitation training for KOA requires individualization, the accuracy of exercise instruction, and the survivability of the process. With the advent of new depth sensors, a more reliable and convenient method for exergame movement guidance is available. Based on the above analysis, we combined KOA patient exercise prescription with exergame and used Azure Kinect technology and action recognition algorithm to achieve personalized exercise guidance and fun exercise. The idea of the system is that a physician makes a personalized exercise prescription, and the exergame system reads the exercise prescription and gamification to a personalized exergame for patients. The primary objective of this article is to introduce the design and implementation of the system, evaluate the accuracy of the system's motion guidance, and assess the usability and user experience of the system through a clinical controlled trial.

2 System design

2.1 System design

This system is an exergame that helps KOA patients to rehabilitate at home and achieve accurate guidance and interest in their exercise. The system first digitizes the KOA exercise prescription developed by the physician and then automatically generates a personalized exergame based on the exercise parameters. The game parameters of the exergame are matched with the exercise prescription parameters to ensure the accuracy of the exercise prescription. The exergame uses Azure Kinect to obtain information about the user's major joint positions and calculates the major joint angles to drive the real-time status of the character in the game. At the same time, the recognition method of rehabilitation training actions is designed based on the patient's joint angle data, and the virtual character is controlled to run the game. The rehabilitation physician views the patient's information and the data information of exercise actions through the physician's Web module of the remote system [32]. The physician formulates exercise prescriptions for KOA patients based on their disease and function of the knee and then can evaluate and adjust exercise prescription parameters based on the exercise feedback information. Figure 1 shows the general research approach of this research.

Fig. 1
figure 1

The Components of the Knee Exergame System

Full size image

The system possesses the following characteristics:

  • Scientific and Effective: The exercise prescription developed by the rehabilitation physician ensures that the rehabilitation training aligns with the patient's knee joint's functional state. The exergame is designed to follow the exercise prescription, ensuring a scientific and effective approach to rehabilitation. The numerical game planning is tailored to the parameters of each patient's exercise prescription, ensuring personalized and targeted training.

  • Accurate Execution: The system utilizes Kinect somatosensory technology to perceive and capture key action parameters. This allows patients to view their actions in real-time and provides monitoring of action completion. If a patient deviates from the specified amplitude, the system provides reminders, ensuring accurate execution of each action.

  • Active Rehabilitation: The user-centric game design, coupled with somatosensory interaction, enhances the system's interest and encourages patients to actively engage in rehabilitation training. By incorporating gamification elements, the system promotes motivation and compliance with exercise execution, fostering an active approach to rehabilitation.

  • Simple and User-Friendly: The system is designed with simplicity in mind, featuring a straightforward structure and easy installation process. It can be quickly deployed in various locations, such as homes or offices. Users do not need to wear any additional sensors, and they can easily engage in personalized rehabilitation training guided by the system.

These characteristics collectively contribute to a system that offers a scientific, accurate, and engaging rehabilitation experience for KOA patients. By providing targeted exercise guidance, real-time feedback, and an enjoyable user interface, the system aims to enhance patient compliance and outcomes in their rehabilitation journey.

2.1.1 The hardware architecture

The exergame architecture comprises a computer, a Kinect (Azure), a large screen, and a soundbox used for data processing, motion tracking, game environment visualization and sound playback. Figure 2 shows the exergame's hardware architecture.

Fig. 2
figure 2

The hardware architecture of the exergame

Full size image

Azure Kinect is a new generation of depth sensors released by Microsoft in 2019 [33]. Azure Kinect can acquire 32 joints of humans and has higher acquisition accuracy [34]. The system first obtains the position data of the user's hip, knee, and ankle joints through Kinect, then calculates the joint angles for motion recognition and implements game interaction. Thus, the user can interact with the computer through various gestures and postures without needing to wear sensors. Kinect eases the interaction process and enables a more seamless mapping process, making it easier for older people who lack gaming experience to access the game. Figure 3 shows the structure of the Azure Kinect setup and human skeleton acquisition.

Fig. 3
figure 3

The structure of the Azure Kinect setup and human skeleton acquisition

Full size image

2.1.2 The software setup

The software system consists of user management, exercise parameter confirmation, exergame, and exercise report. User management records user information, including user's basic information, exercise prescription information, etc. Exercise parameter confirmation is for users to view and confirm the exercise parameters of this time of exergame. Exergame is the system's main body. The exergame numerical planning is generated according to the user's exercise prescription parameters. The exercise parameters are converted into the game level design and the action guidance. Exercise report records the performance of the user's exercise prescription, so that the rehabilitation physician can regularly check and adjust the prescription. Figure 4 shows the flowchart.

Fig. 4
figure 4

The flowchart of the KOA exergame system

Full size image

The system mainly adopts Unity 2019.3, C#, and MySQL. The Azure Kinect development kit is used to obtain the user's skeleton data. The user's clothes should be as tight as possible, improving the accuracy of skeleton tracking. Figure 5 shows the exergame running effects.

Fig. 5
figure 5

The running effects of knee exergame

Full size image

2.2 Exercise prescription digitization

Rehabilitation physicians develop KOA exercise prescriptions based on various factors such as age, gender, symptoms, medical examinations, and functional tests. These prescriptions aim to achieve scientific and planned rehabilitation treatment or fitness prevention [35]. The exercise prescription specifies the type of exercise, exercise intensity, exercise time, frequency, and provides instructions for attention during exercise.

KOA rehabilitation training primarily focuses on the muscles surrounding the knee. It typically includes muscle strength training, aerobic exercise, range of motion training, balance training, and flexibility training [36]. For this study, six rehabilitation exercises were selected after careful screening by rehabilitation physicians. These exercises include stepping, leg raises and knee bend, straight leg raises forward (SLR-F), straight leg raises backward (SLR-B), squat, and up and down stairs (UDS). These exercises meet the rehabilitation training needs of KOA patients and can be easily performed independently.

In this study, the movement specifications and intensity requirements were further refined, focusing on limb requirements and joint angles.

  1. (1)

    Limb requirements: KOA patients may have functional differences between their left and right knees, leading to variations in exercise intensity for each knee. To standardize exercise movements, such as straight leg raises forward, separate strength requirements were defined for the left and right legs.

  2. (2)

    Joint angle: Joint angle is associated with the joint's range of motion, which can vary among KOA patients. It is crucial to define reasonable joint angles that patients should reach during exercise. Reaching these angles indicates the completion of the movement. This study considered hip and knee angles in the coronal and sagittal planes, respectively.

  3. (3)

    Time to remain: Time to remain refers to the duration of completing a movement and maintaining a static position. For instance, when a user achieves the required squatting angle, they remain in that position for a specific period, which is beneficial for muscle strength and endurance. The duration of remain can vary depending on the patient's medical condition and stage of recovery. It serves as an important indicator for functional assessment.

To enable computer understanding and processing, exercise prescriptions are digitized, converting them into a computer-readable format. The concept of "structure" in computer language is used to digitally represent exercise prescriptions, as demonstrated in Table 1. This digital representation facilitates the implementation and customization of exercise programs within the exergame system.

Table 1 The structure of exercise prescription
Full size table

The digitization of exercise prescriptions allows the computer to interpret and implement the prescribed exercises accurately, ensuring personalized and effective rehabilitation training within the exergame system.

2.3 Exercise prescription gamification

Exercise prescription gamification involves transforming exercise prescriptions into game-like experiences. This requires matching game levels and game numerical values with exercise prescription parameters, which prompts the user to perform the rehabilitation exercise precisely as encouraged and guided by the interactive game. In this study, the gamification process begins with designing game levels that correspond to each specific exercise. The game levels are tailored to align with the exercise prescription parameters and provide an immersive and engaging experience for the user.

Next, action guidance is developed for each exercise to assist users in executing the movements correctly. The action guidance provides instructions, visual cues, or audio prompts to guide users through the proper form and technique of each exercise. This ensures that users perform the exercises accurately and safely.

To evaluate the user's performance, an action recognition method is implemented. This method utilizes the user's skeleton tracking data obtained through technologies like Azure Kinect. By analyzing the user's skeletal movements, the system can assess whether the user accurately performs the exercise according to the prescribed form and range of motion.

By combining game design, action guidance, and action recognition, the exercise prescription gamification approach enhances user engagement, motivation, and adherence to the prescribed exercises. It provides a more interactive and enjoyable experience for users while ensuring they perform the exercises correctly and reap the maximum benefits of the rehabilitation program.

2.3.1 KOA exercise prescription game level design

Exergame level design involves creating game interaction scenes that correspond to each rehabilitation training action. In this study, a wilderness adventure game is designed as the backdrop for the exergame. The levels are set according to the exercise order and exercise prescription.

Within the game, various types of gameplay mechanics are utilized, such as obstacle avoidance, attack, and defense, to guide users in executing the corresponding exercises. For instance, the design may include a fallen tree that requires users to perform a squatting motion to pass through. Different states of large tree lodgings may require users to squat at varying knee angles, while the number of trees may influence the duration of the squatting exercise. Additionally, game incentive design is incorporated to enhance the enjoyment and engagement of the game.

The combination of exercise prescription parameters and game design elements ensures that users perform the prescribed exercises while immersed in an entertaining gaming experience. The level design and interaction design of the exergame are presented in Table 2, showcasing the alignment between the exercise prescription and the corresponding game elements.

Table 2 The level design and interactive design of the exergame
Full size table

The game is designed to match the parameters of the exercise. Straight leg raises forward, or backward is used as an example for the introduction. In the game, different types of monsters are used to indicate whether the user should raise their left or right leg and whether it should be done in a forward or backward motion. The number of monsters matches to the number of exercise sets, and the life value of each monster is related to the number of repetitions in each set. To defeat a monster, the user must complete the specified number of exercises. The time interval between monster appearances corresponds to the rest interval between exercise sets. When the user raises their leg to the specified angle and maintains it for the specified time, it is recognized as a successful attack effect against the monster.

The exergame design follows a user-centered (UCD) approach [37, 38]. Extensive preliminary research and user communication were conducted to ensure that the game details highlight its characteristics and promote active user participation. The user experience is a crucial aspect, particularly for the elderly KOA patients, as it influences their willingness to engage in the exercise program continuously. The game design takes into account factors such as psychological acceptance, hearing, eyesight, motor response ability, and coordination skills of elderly users. These considerations aim to enhance the users' motivation and active participation.

The game design process involved pre-design and process design stages, which included the participation of several elderly KOA patients. Game incentive design, safety considerations, and work-rest combination design were incorporated. Through five iterations of design and testing, a final version of the exergame was developed for the experimental testing. The exergame design process based on the UCD method is illustrated in Fig. 6, demonstrating the iterative and user-centric approach taken in the design process.

Fig. 6
figure 6

Exergame Design Process based on the User-Centered Design (UCD) Method

Full size image

2.3.2 Action guidance design

Action guidance involves providing users with instructions on how to correctly execute the prescribed training actions. The system utilizes various digital media forms to guide users and ensure they perform the actions accurately.

  1. (1)

    Images or animations: The system designs images or animations for each action, serving as visual guides for users to understand the movements they need to execute. Simultaneously, voice prompts are provided to instruct and encourage users to perform the prescribed actions.

  2. (2)

    Augmented reality (AR) technology: AR technology is employed to assist users in correctly performing the training actions. By superimposing virtual information on real-life scenes, a small screen window is created to display Kinect-captured human images with bone stick overlays. This allows users to visualize the effects of their skeletal recognition and ensure proper alignment and form during the exercises.

  3. (3)

    Real-time user skeleton data: The system utilizes the user's skeleton data obtained by the Kinect to drive a game character in real-time. The user's skeletal motion data is assigned to the joints of the game character, enabling the character to replicate the user's actions. This real-time feedback assists users in observing the effects of their actions as they control the game character.

  4. (4)

    Game safety design: The system prioritizes user safety during the exercise sessions. If a user feels unwell at any point, they can stop the game immediately. Adequate rest periods are incorporated between different exercise categories and exercise sets to allow users to recover. The interval time between each exercise is primarily based on the user's completion of the previous action, ensuring sufficient time for rest and avoiding potential risks associated with excessive strain. However, the maximum interval time value for subsequent exercises is also considered to maintain a suitable pace and flow throughout the workout.

Through these action guidance strategies and safety measures, the system aims to provide users with clear instructions, real-time visual feedback, and a safe exercise environment, enhancing their understanding, performance, and overall experience during the gamified rehabilitation training.

2.3.3 Action recognition design

Action recognition is a crucial parameter that determines whether the user has successfully executed the prescribed action. The accuracy of action recognition is vital for the independent and effective use of the rehabilitation training system [39]. Kinect, a motion sensing device, is used to capture the position information of human joints, and its capability for accurately acquiring action data has been well demonstrated [40, 41].

The action recognition technology employed in the system calculates the position relationships and angles of the joints based on the Kinect data. This information is then processed using threshold-based recognition algorithms [42] or random forest recognition algorithms [43]. These algorithms analyze the joint data to recognize the specific action being performed and determine whether it has been accurately completed. The action recognition process allows the system to assess whether the user has executed the exercise as required, enabling the progression of the game and the overall rehabilitation training.

By utilizing the accurate action recognition capabilities of Kinect and implementing suitable recognition algorithms, the system gives feedback on user performance in real time, ensuring that the prescribed actions are executed correctly. This feedback not only promotes the game's progress but also encourages users to maintain proper form and alignment during the rehabilitation exercises.

The definition and calculation of the user's joint angle is the basis of the system's recognition. Figure 7 illustrates the coronal and sagittal angles of the knee, respectively. Taking the calculation of the angle of the left knee \(\theta_{i}^{lk}\) as an example, \(i\) represents a certain frame of data acquisition. \(p_{i}^{lh} = \left( {x_{i}^{lh} {, }y_{i}^{lh} {, }z_{i}^{lh} } \right)^{{\text{T}}}\) is the position of the left hip, \(p_{i}^{lk} = \left( {x_{i}^{lk} {, }y_{i}^{lk} {, }z_{i}^{lk} } \right)^{{\text{T}}}\) is the position of the left knee, \(p_{i}^{la} = \left( {x_{i}^{la} , y_{i}^{la} , z_{i}^{la} } \right)^{{\text{T}}}\) is the position of the left ankle, \(v_{i}^{1}\) is the definition vector represents the connection vector between the knee and the hip, the direction points to the hip joint; \(v_{i}^{2}\) is the definition vector represents the connection between the knee and the ankle, and the direction points to the ankle joint; the knee joint angle calculation formula (1) as follows:

Fig. 7
figure 7

Schematic diagram of knee angle calculation

Full size image
$${\text{cos}}\;{\theta }_{i}^{lk}=\frac{{v}_{i}^{1}\cdot {v}_{i}^{2}}{\Vert {v}_{i}^{1}\Vert \Vert {v}_{i}^{2}\Vert }=\frac{\left({p}_{i}^{lh}-{p}_{i}^{lk}\right)\cdot \left({p}_{i}^{la}-{p}_{i}^{lk}\right)}{\Vert {p}_{i}^{lh}-{p}_{i}^{lk}\Vert \Vert {p}_{i}^{la}-{p}_{i}^{lk}\Vert }$$
(1)

The ROM of the knee usually refers to the action angle of the knee joint in the sagittal plane of the human body. In this system, we can simplify the calculation formula (2) of the ROM angle of the left knee in the sagittal plane as follows:

$${\text{cos}}\;{\theta }_{i}^{lk*}=\frac{{({y}_{i}^{lh}-{y}_{i}^{lk},{z}_{i}^{lh}-{z}_{i}^{lk})}^{\text{T}}\cdot {({y}_{i}^{la}-{y}_{i}^{lk},{z}_{i}^{la}-{z}_{it}^{lk})}^{\text{T}}}{\Vert {({y}_{i}^{lh}-{y}_{i}^{lk},{z}_{i}^{lh}-{z}_{i}^{lk})}^{\text{T}}\Vert \Vert {({y}_{i}^{la}-{y}_{i}^{lk},{z}_{i}^{la}-{z}_{it}^{lk})}^{\text{T}}\Vert }$$
(2)

This system adopts two kinds of action recognition methods. The algorithm design and process are as follows:

  1. (1)

    Random forest algorithm: The system utilizes a random forest algorithm for the classification and recognition of complex and continuous actions, such as stepping and going up and down stairs. The random forest algorithm is a robust classifier that combines multiple decision trees and makes predictions based on the voting results of each tree in the forest. It offers advantages such as simplicity, ease of implementation, and low computational overhead [44].

To train the random forest algorithm, the system first constructs an action data database. Data from 34 individuals, including 20 healthy young adults, 8 healthy older adults, and 6 KOA patients, were collected for the stepping and stair-related actions. Since the rehabilitation training requires users to face the camera, frontal action execution data were primarily used for action recognition training. Each segment of the action data consists of five cycles of action, with each individual performing five segments of action. To focus on the lower limb actions and minimize interference from other limbs, the system calculates data for four angles in the sagittal plane of the human body at two hip and two knee joints. The mean and variance of the four-angle data are calculated for each action data segment, serving as the feature data for the action database. The Bootstrap sampling method is employed to generate the training set and construct decision trees for each tree in the random forest. Finally, multiple decision trees are trained and integrated into the random forest algorithm, which can be used to recognize whether stepping or stair-related actions are being performed during a given time.

Once the action type is confirmed, the system performs action counting. Specifically, the system calculates angular data for two knees in the sagittal plane. The data is first smoothed, and then the first-order derivatives of the angle data are calculated. K-means cluster analysis is utilized to identify the maximum peak or minimum trough data points representing the angle data changes, and the number of actions is counted accordingly. This counting process determines the number of stepping or stair-related actions performed during a specific period.

  1. (2)

    Threshold judgment algorithm. We used threshold judgment methods to recognize actions that need to reach a certain posture, such as T-pos, squat, Leg raises and knee bend and other actions. We directly match the relative positions and angles of joints to recognize actions. For example, squat refers to a user squatting to a certain angle synchronously with both knees. The action recognition method calculates the ROM to reach the squatting angle \(\hat{\theta }\) required by the user and sets a threshold \(\varepsilon\) value to determine whether it approximately meets the requirements. The judgment formula (3) is as follows:

    $$flag = \left\{ {\begin{array}{*{20}c} {1,} & {{\text{if}}\;|\theta_{i}^{lk} - \hat{\theta }| \le \varepsilon {\text{ and }}|\theta_{i}^{rk} - \hat{\theta }| \le \varepsilon } \\ {0,} & {{\text{Otherwise}}} \\ \end{array} } \right.$$
    (3)

where, \(\theta_{i}^{lk}\) is the angle of the left knee, \(\theta_{i}^{rk}\) is the angle of the right knee, the value of 1 indicates that the action executed by the user is recognized as a squat action, and the value of 0 indicates that the squat action has not been completed.

In the system, action recognition is designed as a binary classification task to optimize efficiency. For example, when the squat game level requires a squat action, the system only recognizes whether the user has executed the squat action. This design aligns with the system's task requirements and improves overall efficiency in action recognition.

3 Experimental design

3.1 The reliability of system

The exergame system for knee rehabilitation, which is assessed in this study, was evaluated based on two primary dimensions. We conducted a clinical randomized controlled trial involving older patients with knee osteoarthritis to validate the usability. The experimental group was tested using the exergame system, while the control group used a control system that could acquire joint angle data through Kinect but lacked exercise movement guidance and feedback. The usability of the system was assessed based on the accuracy of action execution and action recognition.

In order to assess the system's usability, two metrics were established: action execution precision and action recognition precision. The action execution precision gauges the user's proficiency in carrying out prescribed rehabilitation actions prompted by the system, thereby evaluating the efficacy of the guidance provided. On the other hand, the action recognition precision measures the system's capability to accurately identify and interpret the user's actions, allowing for an assessment of the system's viability.

During the experiment, users followed the exergame's sequential action requests and performed the actions accordingly. The system detects user actions and uses the algorithms described in Section 2.3.3 to recognize and determine whether each user action is performed correctly. Simultaneously, the system recorded videos of the users performing the actions during the exergame. A group of rehabilitation physicians (consisting of three physicians) reviewed the videos and judged whether the user's actions were executed correctly. If the physician group determines, based on a combined judgment, that a training action was performed correctly, it is recorded as being performed correctly by the user. Otherwise, it is recorded as incorrectly performed.

We assessed the accuracy of movement execution by comparing the number of times the user completed the movement correctly to the total number of movements prescribed in the exercise prescription by the rehabilitation physician. We assumed that the number of exercises required by the system fully met the exercise prescription issued by the rehabilitation physician.

Through this evaluation method, we aimed to assess the user's ability to execute the actions accurately and determine the system's capability to recognize and provide feedback on the user's actions. The results of this evaluation will help analyze the feasibility and effectiveness of the exergame system for use in the rehabilitation of patients.

The accuracy of the action execution of the action \(i\) is defined as follows:

$$Accuracy\_exe_{i} = \frac{{N\_ec_{i} }}{{N_{i} }}$$
(4)

where, \(N\_ec_{i}\) represents the number of actions \(i\) executed correctly by the user, and \(N_{i}\) represents the number of actions \(i\) required by the exercise prescription.

The accuracy of action recognition can be evaluated by computing the ratio between the frequency of accurate execution of an action by a participant and the frequency of precise recognition of that action by the system. The specific formula for this calculation is as follows:

$$Accuracy\_rec_{i} = \frac{{N\_rc_{i} }}{{N\_ec_{i} }}$$
(5)

where, \(N\_ec_{i}\) represents the number of actions \(i\) executed correctly by the user, and \(N\_rc_{i}\) represents the number of actions \(i\) which correctly recognizes by system based on the number of correct actions executed by the user.

3.2 User experience analysis

User experience is a crucial aspect of evaluating users' subjective experience and is closely related to system usability. It plays a significant role in facilitating high-quality interactions between users and systems. As technology advances, there is a wide range of smart products available for the elderly. However, previous research has often overlooked the user experience of older adults. Most motion-sensing games are primarily designed for the entertainment of young people, neglecting the specific interaction design needs of older adults. Consequently, this has led to challenges in the independent use of such systems by older adults and hindered the successful implementation of rehabilitation intervention programs. To address this gap, the knee joint rehabilitation training system proposed in this article adopts a user-centered design approach throughout its design and development process, fully integrating the principles of user experience design. Therefore, it is essential to further validate the system's design through user experience analysis.

Several user experience measurement scales are available, including the System Usability Scale, the Questionnaire for User Interface Satisfaction, the User Experience Questionnaire (UEQ), etc. User experience evaluation generally encompasses two dimensions: usability quality and hedonic quality. We employed the short version of the User Experience Questionnaire (UEQ-S) [45]. The UEQ-S retains the core dimensions of the original UEQ while reducing the number of evaluation items, resulting in a more concise and efficient questionnaire. The UEQ-S is particularly suitable for rapid user experience assessment. In this experiment, the subjects are older people, and we try to use this convenient and effective tool to analyses the user experience. Previous research has demonstrated that the UEQ-S is a fast, straightforward, and reliable survey questionnaire with good reliability and validity [46]. As indicated in Table 3, the UEQ-S comprises eight items that capture both hedonic and usability qualities.

Table 3 The evaluation items of the UEQ-S
Full size table

We utilized the UEQ-S analysis tool to process the data obtained from the questionnaire. The tool uses a 7-point Likert scale to adjust the score of each question answer, ranging from -3 (lowest value) to +3 (highest value). Any scales that exhibited a significant difference (> 3) between the highest and lowest values assessed were excluded as abnormal questionnaires. A final score above 0.8 indicates a positive evaluation and below -0.8 indicates a negative evaluation. Scores between 1.5 and 2 represent excellent quality. Furthermore, confidence intervals and scale consistency have been calculated. The confidence interval can be used to assess the accuracy of the mean estimate. The smaller the confidence interval, the more precise the estimate and the more reliable the result.

4 Experimental result

4.1 Participants

We undertook an examination involving 30 participants with knee osteoarthritis who were randomly assigned to two groups: an experimental group consisting of 20 participants and a control group consisting of 10 participants. The average age of the participants was 60 ± 5 years. To ensure accurate diagnostic information, we recruited the experimental subjects primarily from a hospital between March 2021 and July 2022.

To develop exercise prescriptions tailored to each participant, three rehabilitation physicians with extensive clinical experience were invited to develop the prescription and evaluate the effectiveness of the system. These physicians were specialists in rehabilitation medicine from three different hospitals. The rehabilitation physician team developed an exercise prescription for each participant based on patient information. The experimental group performed rehabilitation exercises using the personalised exergame generated from the exercise prescription. On the other hand, the control group performed exercises based on the prescribed exercise regimen without the assistance of the exergame system. The exercise data of the control group was obtained through the control system.

Prior to the experiments, each recruited KOA patient received training on the system's usage specifications to ensure they understood and could follow the system's operating procedures. Each participant from both the experimental and control groups underwent two independent exergame experiments, and the entire process was recorded on video.

Following the conclusion of the user trials, the experimental group members were asked to respond to the User Experience Questionnaire to assess their user interaction. Concurrently, the three rehabilitation physicians evaluated each participant's motions based on the recorded videos, and the quantification of correctly executed exercise movements was performed.

The research assessed the precision of action execution, the accuracy of recognition, and user satisfaction using the statistical findings, thereby appraising the system's efficacy. The investigation adhered to the principles articulated in the Declaration of Helsinki and secured approval from the Ethics Committee of the Hefei Institute of Materials Science, Chinese Academy of Sciences. All individuals partaking in the study provided explicit consent after being informed of the research's objectives.

4.2 The result of system reliability

Based on the assessment conducted by the rehabilitation physician consortium, the actions of the 20 participants in the experimental group during the two exercise experiments were meticulously enumerated and categorized. The outcomes about the precision of action execution and the correctness of action recognition are detailed in Table 4. The findings of the accuracy of the action execution of the control group are shown in Table 5.

Table 4 The results of the accuracy of the action execution and the accuracy of the action recognition of the experimental group
Full size table
Table 5 The results of the accuracy of the action execution of the control group
Full size table

The execution rate of exercise prescription in the experimental group reached 97.47%, the system's accuracy in recognizing actions achieved a rate of 95.42%, as indicated by the data in Table 4. This result suggests that the exergame system demonstrates high effectiveness in promoting exercise prescription execution and system reliability, as evaluated by the consortium of rehabilitation physicians.

Based on the information provided in Table 5, the physician group evaluated the exercise prescription execution rate of the 10 participants in the control group, revealing a rate of 75.96%. Among the exercise movements involving specific angular requirements, patients had difficulty executing them accurately by themselves in the absence of guidance.

4.3 The result of user experience analysis

We assessed the user experience within the experimental group of the exergame. The questionnaire, which was performed only on the 20 patients in the experimental group, was recovered with reasonable data and no unusually answers. Table 6 shows the mean scores of the UEQ-S questionnaire indicators at the 95% confidence interval. These values indicate that the experimental group gives a good user experience assessment of the exergame.

Table 6 The results of the mean statistics of UEQ-S values
Full size table

In addition, we plotted the benchmark results of the UEQ-S data analysis tool, as shown in Fig. 8. Relatively speaking, the ratings of hedonic quality are significantly higher than those of pragmatic quality. This indicates that exergame has better game attributes that can increase the interest of KOA patients in exercising.

Fig. 8
figure 8

The benchmark result of the UEQ-S scales

Full size image

5 Discussion

In this study, we designed an exergame based entirely on the exercise prescription of KOA patients. By digitizing the exercise prescription and numericizing the game, we guided the patients in accurately executing patient-personalized rehabilitation exercises. Analysis of the clinical control experiment revealed that the accuracy of movement execution under exergame guidance reached 97.77%, which was significantly higher than that of the control group of KOA patients who underwent rehabilitation training without guidance. Additionally, the accuracy of action recognition reached 97.52%, indicating that the exergame system is highly effective in promoting execution and system reliability. The evaluation by three rehabilitation physicians further confirmed the reliability and usability of the system through video analysis. It is a valuable tool for assisting KOA patients in self-rehabilitation training.

The user experience analysis revealed that the patients obtained a high score of (1.68 ± 0.893) on the User Experience Questionnaire, indicating an excellent user experience with the exergame. This suggests that the exergame can improve compliance with rehabilitation training. The system's hardware structure is simple and easy to deploy, as it does not require users to wear specialized equipment. Although we recommended users to wear tight clothes to facilitate Kinect data acquisition, the experiment showed that even volunteers wearing ordinary pants achieved a high rate of action recognition. The exergame provides visualizations and multiple cues to guide patients, making the system user-friendly. Therefore, this system is suitable for KOA patients in community and home settings and has promising applications.

However, there are limitations in this study that should be addressed in future research:

The effectiveness and compliance of the rehabilitation training effects need to be validated. While this study verified the reliability and positive user experience of the system, further research is required to assess its effectiveness and long-term compliance in promoting treatment outcomes for KOA patients. This could be achieved through long-term cohort studies and controlled trials.

They are expanding the repertoire of movements and games. Presently, the system provides a constrained set of rehabilitation training actions, which might not meet the diverse needs of all users. In the future, additional suitable rehabilitation actions can be incorporated into the system to facilitate a wider range of rehabilitation training. Moreover, designing more game options for users to choose from would enhance the enjoyment of rehabilitation training and improve compliance.

Further improvement of the user experience in the game. Although the present analysis of user experience yielded favorable outcomes, there may be variations in user age and cognitive abilities that need to be further explored and addressed to cater to the needs of a broader range of users.

6 Conclusion

This article proposes an exergame method based on Azure Kinect to assist patients with knee osteoarthritis (KOA) in accurately performing prescribed exercises and improving their motivation for rehabilitation training. The system digitizes the KOA exercise prescription provided by doctors and designs personalized exergames to guide KOA patients in correctly performing rehabilitation exercises. Additionally, the system utilizes Azure Kinect to collect the users' major joint position information, assisting in identifying the users' rehabilitation training movements. We conducted a clinical controlled trial to validate the usability and user experience of the system. The accuracy of exercise execution in the exergame experimental group reached 97.47%, significantly higher than the control group, indicating that the system can effectively guide KOA patients in their rehabilitation training. The action recognition accuracy of the system reached 95.42%, demonstrating reliable interactivity of the exergame. Through analysis using the UEQ-S, the user experience score in the experimental group was (1.68 ± 0.893), indicating that the system provided a good user experience and improved patients' willingness and compliance to engage in active exercise. The proposed exergame system in this article has high usability and an ideal user experience, not only providing scientific and effective guidance for rehabilitation training for KOA patients but also suitable for home-based remote rehabilitation training for KOA patients.