Keywords

1 Introduction

Virtual Reality (VR) is an often-used instrument in sports science and sports exercise [48]. It offers many advantages such as standardized experimental conditions and immersion due to spatial information and realistic visualization. Authentic sport settings can be repeated and manipulated, which is not possible in real settings [1, 38, 43, 49]. Further advantages of VR are the possibility of a standalone training [5, 44], the safe training environment, and the individual design [13, 14, 49]. Other authors [8, 46] highlight the importance of a realistic visualization of VR environments and their virtual characters to ensure a natural behavior of real athletes. [33] demand the development of sport specific settings that are close to normal training and competition conditions. It is a known fact, that expert athletes can only use their expertise in familiar and natural settings [29]. Nevertheless, further demands exist to develop realistic virtual environment (VE) systems in sports including high immersion: the reduction of latencies, and cybersickness (symptoms of illness, nausea and dizziness) [2, 23, 36, 47].

Several VE systems already exist in different sports: e.g. rowing [38, 42], rugby [26], basketball [13, 14], handball [6, 7], baseball [22, 35, 49], soccer [15], dancing [11] and martial arts [4, 34]. Despite of these VE sports systems, only few inherit autonomous behavior of the virtual characters. [27, 36] emphasize the importance of interactivity in VR applications, which reduces cybersickness. Through interactivity athletes have more influence in the VE sports system and hence have a greater feeling of immersion. This is also described through the model of “Interlocking” [45], where the user’s intentions and possibilities and the affordances of the environment should fit together. Presence is a subjective sensation of being and interacting in a virtual environment. The more realistic the VE is and the greater are the affordances and possibilities to act, the higher is the feeling of presence and thus of immersion. Therefore, it is extremely important that users feel comfortable within and accept VR as a new world [41]. In this context it is also important that users have a natural perception in VR [37].

First approaches of autonomous, adaptive VE systems in sports already exist. [3] used an autonomous system to analyse anxiety and pressure in pistol shooting. [17] developed an interactive cricket simulator for training. Further, there are few autonomous virtual coaches in sports: [20] evolved a first approach for an adaptive immersive virtual golf training system. [11] presented a virtual dance tool for autonomous learning. [16, 19] created a multimodal coaching system for squats with a virtual coach giving real-time feedback. [9] developed an interactive virtual rugby environment to examine deceptive body movements.

In conclusion, autonomous systems can also be used in training as well as in research. However, there is still a research gap in the development and use of autonomous VE sports systems, which adapt independently to the athlete’s movements to ensure realistic sports specific behavior. Especially, autonomous VE systems in fast reacting sports and studies in karate-kumite are rare [32]. Therefore, our research is focused on the development of an autonomous character (AC) in karate kumite, which is able to execute a karate attack autonomously against a real athlete. The development is described from a sports science point of view and can also be used for the creation of ACs in other sports. In the discussion we present future investigations and training programs, which are possible with the ACs. Furthermore, insights regarding the preferable display mode (CAVE or HMD) are given.

2 Methods

2.1 Development of an AC from a Sports Science’s Perspective of View

General Approach

Autonomous characters (AC) for karate kumite were created by the following steps: competition analysis of several high-class international karate kumite matches, design of the AC based on 3D motion capture data, and development of a decision system for autonomy of the ACs. The aim of the current work is that an AC can choose and conduct an appropriate attack towards a real athlete, who is moving karate specifically (stepping with varying distances), but does not perform an attack himself. The athlete is aware that he is going to be attacked and has to react as in a real competition. The AC recognizes the position of the real athlete permanently through the real-time tracking of the athlete’s head and two hand targets.

Competition analysis: selection of relevant karate techniques

A male karate expert with long-term international success as athlete and experience as coach carried out the competition analysis of 45 high-ranking international karate competitions (including world championships, European championships and high-class international cups in the years 2012–2013. The results show that the arm techniques Gyaku-Zuki (GZ), Gyaku-Zuki overrun (GZo) and Kizami-Zuki (KZ) are the most often and most successfully applied attacks with 87% of all attacks. The remaining 13% of attacks are foot techniques, which are not included in the AC. Based on these results, which are in line with [28], the above mentioned arm attacking techniques were chosen for the AC. GZ, a punch with the back arm, is performed from a stable position, while GZo is executed during a forward motion. KZ is a punch with the front arm. GZ can be performed jodan (attack towards the head/chin) and chudan (attack towards the chest/solarplexus), while KZ and GZo are jodan techniques only.

Creation of the AC

Three (n = 3) healthy karate athletes (one woman and two men, age: 24–31 years, long-term and successful international experiences in karate kumite, with coaching experiences, shotokan style) were chosen for animating three ACs. All athletes were informed about the procedure, risks and the aim of the study. They took part on voluntary basis and gave their written consents. The study was conducted under the ethical guidelines of the academic institutions. All employed attacking techniques, sports specific stepping and turning movements were recorded using an infrared motion capturing system (VICON Tracker by using in A.R.T., Weilheim, Germany, 120 Hz) with twelve cameras (MX-13, VICON, Oxford, UK). The targets were placed according to the guidelines of A.R.T, with the exception, that the abdomen target and both foot targets were replaced by self-created targets using 3D-print and CAD models, because the original targets were not suitable for karate specific movements (either they hurt or they were too loose). All attacks started from a pre-defined position and were executed towards a target, replacing an imaginary opponent. The distance and height of the target varied according to the skill of the athlete and the feasibility for a correct attack execution due to the karate regime. For further detail see Table 1. All conditions were repeated three times. All movements were monitored by a karate expert.

Table 1. List of movement recordings for each of the three ACs. GZ: Gyaku-Zuki, GZo, Gyaku-Zuki overrun, KZ: Kizami-Zuki.
Full size table

Development of a decision system for an AC

The autonomy of the ACs is based on a decision system developed in a previous work [31], and aims to ensure a natural (not predictable) behavior of the AC. Based on the competition analysis it was defined that the ACs execute the GZ and GZo with the right hand, and the KZ with the left. In normal stepping position (kamae position) the left foot and left hand are ahead, the right foot is behind, and the right hand positioned near the hip.

The decision system runs through four processing stages: Firstly, the AC has to be aligned towards the real athlete. The alignment is achieved when the AC’s front hand points towards the athlete’s head and front hand being tracked continuously. Oriental adaptations are attained by use of one or several recorded turning techniques. Secondly, the AC follows the athlete’s movements to get in a distance which is appropriate for an attack. The ACs’ positional adaptations are achieved by using recorded forward, backward and sideways steps. All karate athletes executed their attacks within distances of 1.80 m to 2.40 m (distance between rear foot of the attacker and the target), the AC measures the distances (from its rear foot to the plain of the athlete’s head) continuously. Thirdly, the AC is in a possible attacking distance and executes an offensive. The distance between the real athlete and the AC is divided into 20 cm steps resulting in six ranges (under 1.60 m, 1.60 m–1.80 m, 1.80 m–2.00 m, 2.00 m–2.20 m, 2.20 m–2.40 m, over 2.40 m). For each range we used cumulative probabilities (given in percent and based on the competition analysis) for executing attacks. Fourthly, the AC selects the appropriate target position on the opponent’s body. Depending on the position of the hands and thus the defensive position of the real athlete, this can either be the chin (jodan) or the chest (chudan). For more technical details see [50].

Technical implementations

The motion data of each real athlete was captured with the described motion capturing system and a frequency of 120 Hz. The captured data represent target positions of each body limb and are saved as raw data files for each athlete and type of attack. Target information that was lost during the capturing process was interpolated linearly using valid target positions before and after the gaps. Since the gaps were not bigger than a few frames in most cases, no relevant features of the movement were lost during this process. All tracking raw data were filtered using different operations such as LULU smoothing, Savitzky-Golay-Filter and Gaussian Smoothing. To gain best results the parameters of the filtering process were determined separately for each data file. The raw tracking data were then converted to a standard motion representation which describes the orientations of the joints of a skeleton. The 3D model of the AC was designed analogously to a real athlete who perceives the opponent’s movements, decides on an appropriate reaction and performs it. The perception of the AC was realized by an online tracking system that yields the positions of markers mounted to the athlete’s glasses and attached to his hands. This data was provided to the decision system which then chose an appropriate movement for the AC (attack or a step movement). The decision system worked at a frequency of 30 Hz which enables the AC to adapt to changes in the environment. The motion generator produced a motion defined by the decision system (using probability distribution related to empirical findings for hand techniques: 46% are GZ, 23% are GZo and 31% are KZ) and ensured that given constraints as e.g. distances, positions and orientations are met. For the decision on target height (jodan or chudan) the positions of the athlete’s hands were taken into account by computing of protecting area and comparison with the width of two fists. Finally, the created motion data was used to animate the 3D model which was visualized in a stereoscopic displaying system, a CAVE or a HMD. The system was implemented using C++ with Qt library published under the LGPL license. The response time (time between the action of the real athlete and reaction of the AC) was around 150 ms. This includes the processing time of the tracking system, network delay, latency of the displaying hardware, and the time needed for all computations involved. For further technical details see [50].

2.2 Evaluation

Participants

Six (n = 6) experts took part in the interviews on voluntary basis. Four experts are also coaches in karate kumite and thus can give a reliable feedback if the ACs could be used as a tool for karate training in the future.

Procedures

The first aim of the interviews was to analyze whether the ACs and the virtual environment are realistic enough and if the experts feel comfortable in VR. Parts of the interview are based on the questionnaires IPQ [40] and SSQ [21]. The evaluation method is based on to the one described in [17]. It is also possible to evaluate such systems by questionnaires or different scales (e.g. MEC-SPQ or SPES), but we chose expert interviews to get a specific evaluation for our ACs to assess, inter alia, the movement behavior and interactivity. The second aim was to analyze whether a CAVE or a HMD would be more appropriate for future investigations. The experts examined the ACs in two immersive VE systems: in a CAVE (Cave Automatic, Virtual Environment with four screens, size 2.30 m × 2.30 m, resolution of 1400 × 1050) and by using a HMD (Head Mounted Display (Oculus Rift, DK2). They observed the ACs in each setting for 15 min (five minutes per AC in both virtual environments). To reach a higher feeling of immersion within the virtual environment of the HMD, the user’s hands were visualized. Unfortunately, our HMD was not wireless, so the cable was held away to reduce the risk of tripping. The participants were free to move and response karate specifically towards the offences.

Subsequently, the interview (open questions) was made and the answers were both journalized and recorded. The interview consisted of seven complexes, had a total of 35 questions (three to seven items per complex) and lasted 30 min. Complex 1 included seven questions concerning realistic and natural attack behavior and movement behavior of the ACs. Furthermore it was asked, if movement velocities and behavior of different parts of the ACs’ body (hands, posture of head and trunk, face and hip rotation) were displayed realistically. Complex 2 included five questions. It was questioned, if the ACs followed the athletes correctly and if they attacked from realistic positions and distances and hit adequate body regions of the athletes. Complex 3 contained four questions regarding the realistic appearance of the ACs and the virtual environment. The participants were also asked if they felt immersed in the VE system and if they had still perceived the real environment. In complex 4 five questions were used to determine, if further feedback or implementations are needed for future sports applications. Furthermore, the participants were asked, if they need an acoustical or a tactile feedback, natural facial expressions in the ACs or if the visual appearance is sufficient enough. The athletes could give further ideas for improvements, additionally. Complex 5 consisted of four questions. The participants were asked if they took the VE system seriously, if they had fun, and if they felt comfortable in VR. Furthermore, they were asked for symptoms of cybersickness. Complex 6 was composed of three questions. It was assessed if the attacks were recognizable and correctly performed, and if one could respond adequately. Furthermore, the participants should indicate if they were able to behave in the VE system as they would have done in reality. Complex 7 (six questions) inquired if the participants recommended HMD or CAVE for future investigations.

Analysis of data

The participant’s answers were summarized and analyzed accordingly to the methods of [18, 25]. All answers were scaled into three categories (good, moderate and poor), due to certain keywords or phrases. These are shown as descriptive statistics in Table 2. Answers to complex 7 were scaled into seven categories (−3 to +3).

Table 2. Results of the expert interviews in [%]
Full size table

3 Results

Complex 1 (realistic behavior of the ACs) is sufficient, but still developable. Complex 2 (realistic interaction) is only moderate. The results of complex 3 (realistic visualization of VR and immersion) indicate that CAVE and HMD have a different level of realistic visualization and immersion. While visualization and immersion are only moderate in the CAVE, they are satisfactory using the HMD. The visual presentation of the ACs (karate Gi), the environment (a gym and mats), the presented movements, and the attack distances were rated realistic. Moreover, the decision system works correctly and fulfills the demands for VE sports systems, allowing a sport specific behavior. Complex 4 (integration of further feedback and implementations) confirms that the visual implementation is sufficient. Ideas for further feedback and implementations are described in the discussion. Complex 5 shows that the participant’s feelings, experiences and cognition are good in both virtual environments, CAVE and HMD. All participants felt great motivation and no one reported symptoms of cybersickness, although the latency of the AC’s movements were around 150 ms. However, the latency did not lead to changes in the reaction behavior or seem to disturb the athletes. Nevertheless, all participants recommend further decrease of latencies to ensure greater immersion and more natural training feeling. Complex 6 (participant’s movements, reactions and attack recognition) reveals that they can move and respond better with the HMD. In the discussion we show possible improvements for the ACs and possible future usage in regards of anticipation research and as a tool for competition and training.

In complex 7 the participants were asked if they prefer CAVE or HMD for future investigations in karate kumite. All athletes rate the HMD as preferable tool. In a seven-scale (−3 to +3) the CAVE was valued with 0.25 ± 0.5 (mean ± SD), while the HMD was valued with 1.875 ± 0.25 (mean ± SD). The interviews showed that all users prefer the use of a HMD due to greater immersion and more freedom of movements, which allow sports specific motions and responses. The CAVE used in the current study was too small, which resulted in the participant’s feeling of being constricted. The HMD was very light and compact and thus did not disturb the users. While the HMD encloses both eyes and thus completely excluded the real world, it was easier to accept the VR here as a real environment. In regard to the question, if the experts would recommend using the current autonomous VE system for future sports application, such as anticipation research and training, all stated that after some improvements the system would be appropriate for further utilization.

4 Discussion and Outlook

The present study provides sports scientific foundations for the development of an autonomous character (AC) in karate kumite with regard to future anticipation research and training. Autonomous characters have the advantage, that they can be used without the need of further training partners or coaches. Nowadays, new visualization technologies are more accessible and there is a need to understand if and how VR can be used to train athletes in sport. This work makes a contribution in showing how to create an autonomous VE sports system and how this system could be used to analyze and train motor response and decision making in karate. It allows both, to study and to routinely test athletic responses to realistic and controllable stimuli. The current work also shows that karate athletes prefer a HMD over a CAVE, so for future investigations and training in VR, we recommend the use of compact and small HMDs in a room which is big enough for sports specific movements. The presented procedure can also be transferred to other sports (e.g. martial arts and racquet sports). Finally, our study provides essential steps for the development of an autonomous character (AC), which can be used in a great number of sports to perform interactions with real athletes. These include the selection of relevant techniques, the creation of an animated model of the AC, the development of a decision system, and its evaluation. In the following sections we present possible improvements for the ACs and potential uses.

Possible improvements for the ACs

The study shows the development and evaluation of an AC in karate kumite from a sports scientific point of view. The results underpin the development of a realistic autonomous setting and hence achieve the demands of [4, 8, 32, 33, 46].

There has been a rapid improvement of techniques in the last years, so future virtual sports systems with reduced latencies should be available soon [17, 36]. The usage of a prediction system would also be helpful to decrease latencies of the decision system [27]. Already [10] present a prediction system in table tennis, what could also be used in other fast reacting sports. We intend to increase the interaction, what is also demanded by our athletes, so the ACs would be able to react to a real athlete’s attack. Therefore, a movement analysis and a cue detection, which are relevant to anticipate an upcoming attack, are necessary.

Our experts recommend further feedback in the autonomous VE system like acoustics (e.g. stepping noises) and haptics, but especially haptic feedback is still rare in VE sports systems [43, 49]. A possible setup for haptic feedback would be to attach e.g. vibration sensors at the body of the real athlete (for a correct AC’s hit detection), and at the athlete’s hand for a correct hit detection of the athlete’s attack. The approach of [12] would therefore be a possible solution, where piezoelectric sensors were attached to detect upcoming forces (correct hits). However, feedback should be implemented cautiously. Firstly, it could evoke a dependency of users to it, and secondly the athletes could be disturbed by further technical devices [43]. In regards to the importance of natural facial expressions of the ACs different opinions exist. Currently, the ACs have a stiff face with a neutral expression. While some athletes state that facial expressions are very important, others declare that they do not look into the opponent’s face. Using eye tracking, [39] showed that expert karate athletes focus on the opponent’s head, what could be a cue that the face and especially facial expressions may play a role. However, the use of eye tracking alone inherits the problem that actual attention as well as information in peripheral vision cannot be captured and measured directly. Although athletes focus on the head, which is a central point, they can detect small peripheral changes in the opponent’s kinematics and thus have a overall view. As a result the influence of facial expressions and the detection of relevant anticipatory cues need further examinations in future studies.

Another possibility for improving the ACs for karate training in VR is to capture more athletes. The athletes embodying the ACs in the study at hand are all from the same sports club which is organized in the German Judo-Karate Association (DJKB) e.V. Athletes from different national and international clubs and different associations could be included and be the basis for several ACs with different fight behaviors. Moreover, further attack techniques (e.g. foot techniques) should be integrated in the ACs. [30] already demonstrated that high-class experts also have little individual variations within their movement patterns. A further potential approach could be to mix all the movements of all current ACs. Specifically, the same character would attack, but with all recorded movement variations (all distances and all heights for attacks), which would not be possible in reality.

Potential uses of the ACs

One of the aims of VE sports systems are to implement realistic sports scenarios for practical training (e.g. [13, 14, 38]), and to analyze relevant anticipatory cues (e.g. [4, 9]). All experts of the present study have the opinion that an AC is a useful tool for anticipation and reaction training and further investigations in anticipation research. Anticipation investigations can be made using the ACs in combination with the methods from [32]. Here, the real athlete’s first response to virtual attacks is being detected.

Another possibility to analyze anticipation is the usage of eye tracking in VR in combination with temporal and spatial occlusion techniques. The reaction to spatial and/or temporal occluded attacks might give insights to relevant anticipatory cues. Further it is possible to use the ACs for practical training, e.g. the improvement of response times and type of response. Virtual opponents have the advantage that they do not get tired and thus can be a suitable tool for speed and reaction training. Additionally, the speed of the virtual attacks could be manipulated to create different levels of difficulties. ACs can also be used for competition preparation. The AC can be manipulated to act and attack like an actual future opponent. Different fighting types (offensive, defensive) can be embodied, too. However, when using HMDs in VR training, the athlete’s body should be virtualized [24]. It would also be desirable to use a wireless HMD to create even more freedom of movement.