1 Introduction

Sonification is a domain of interdisciplinary sciences defined as the use of non-speech sound to convey information. It gathers several techniques such as audification, auditory icons, earcons, parameter mapping sonification, and model-based sonification. The most common techniques have been described and discussed extensively in the Sonification Handbook [12], providing an exhaustive overview of the field. Due to the temporal nature of sound, sonification is particularly well suited for applications and tasks related to time, e.g., monitoring [1] or synchronisation [27]. In addition, the strong relationship between auditory and sensorimotor systems makes it suitable for augmenting the perception of movements, in particular the perception of one’s own body motion (i.e., kinesthesia). By combining these two aspects, sonification has the capacity to enhance applications related to physical training and rehabilitation in a promising way. Other common applications of sonification include data mining, sensory substitution (e.g., to improve accessibility), artistic works, and complement to scientific visualisation.

In this work, we apply sonification to rowing. Rowing biomechanics have been studied extensively [26], bringing into play complex interactions between various kinematic and kinetic quantities. Yet no model for optimal behaviour has been proposed through balancing these quantities in a way that would prove to be the most efficient. On the other hand, guidelines to improve efficiency of the rowing cycle have been provided, mainly by focusing on the different sources of energy waste [15]. Discussions with trainers and rowers confirmed that they were aware of these guidelines and endeavoured to comply with them to improve the rowing technique. The objective of the present work is to investigate whether an interactive auditory feedback can improve a rower’s technique in relation to these guidelines.

Certainly due to practical challenges inherent to the nature of rowing, many of the studies on rowing biomechanics have been performed in laboratory with help from ergometers, i.e., rowing machines aiming at simulating the movements of a rower. In several cases, these ergometers were modified to simulate the motion of the boat and take into account energy exchanges with its environment. It is of course sensible to extend results obtained with ergometers to on-water rowing. Nevertheless, recent improvements in mobile technology enable in situ collection of kinematic, kinetic, and physiological data in a relatively straightforward way, as well as on-the-fly data processing, and real-time synthesis of augmented feedback. The present work provides an example of such a data flow implementation using a somewhat reduced and easily accessible technical setup that could be extended by incorporating additional sensors and by displaying multimodal feedback.

In a previous article, we introduced an evaluation of four models for the sonification of elite rowing [4]. This evaluation was conducted using listening tests performed in a non-interactive context. The four models evaluated, described in detail in [4], consisted in a pure tone with gliding frequency, sounds of musical instruments varying in pitch, wind sound varying in loudness, and car engine sound varying in brightness. Design choices were explicited, compared with existing sonification methods, and four different models were implemented and evaluated via listening tests in a non-interactive context. Two of these models (Wind and Car engine) are based on environmental sounds implementing two different metaphors: the experience of the wind blowing stronger when moving at a higher velocity, and the characteristic shift in brightness of the sound of a car engine when pressing the gas pedal for accelerating. Such auditory displays using an ecological approach call on everyday listening as described by Gaver [8]. On the other hand, two other models (including the Pure tone model used in the present article) make use of sonification mappings without prior metaphorical value from an ecological point of view, and therefore have to be learnt by experience (e.g., when interacting with the system). Auditory displays of this type call on musical listening instead of everyday listening, bringing into play different levels of cognition.

In our previous study, elite and casual rowers assessed the sonification models with respect to aesthetics and perceived functionality. From the analysis of the results emerged a correlation between aesthetic appreciation and perceived functionality, i.e., the rowers associated the aesthetic value with the amount of information they assumed to be able to extract from a particular sonification model. A ranking of the models could be established with respect to both aesthetic and overall preferences, the latter being rather influenced by aesthetic aspects than by functional aspects. The wind sound was rated as the most preferred and pleasant sound.

The objective of the present article is to introduce a follow-up study aiming at describing the implementation of an interactive sonification system, as well as characterising its observable effects on the rowing style. Interactive sonification may play a role in the context of multi-rower crews, e.g., acting on the synchronisation of the crew. However, since we were primarily focused on alterations of the individual technique, experiments were conducted with single scullers only.

2 Method

2.1 Motivations

Several systems for interactive sonification of human body motion have been introduced in the past few years [6, 11, 28], aiming to improve the self-perception of one’s gestures for purposes of training or rehabilitation. Many practical applications have appeared in diverse sports such as speed skating [9], swimming [13], running [2], or weightlifting [21]. Interactive sonification has the potential to enhance the training of elite athletes by creating an additional channel of perception of body motion. In the specific case of elite rowing, the “body” can be extended to include the boat and the oars through the use of appropriate sensors to collect kinematic and kinetic data [20]. By communicating these data in real time using the auditory modality, interactive sonification creates a musician-instrument learning paradigm allowing the rower to make the timing and magnitude of body movements and forces applied more efficient [29]. In other words, a parallel could be drawn between learning how to play a musical instrument and learning how to row with help from an interactive sonification system. In music education, the auditory feedback is naturally provided by the instrument. However, it is interesting to note that interactive sonification has even been used in that context [10, 22]. It has been shown that better rowing skills reduce energy waste, making the rowing technique more efficient [17]. Any skill improvement due to interactive sonification would, therefore, be reflected on rowing efficiency. We also expect the sonification system to create a rhythmical experience within an interactive sensorimotor loop, therefore helping rowers to maintain a stable stroke rate.

Acceptance of the system is critical when designing technological aids for elite athletes. Designing a suitable continuous auditory feedback is a challenging task, and evaluation of rowing sonification models in a non-interactive context revealed a relatively high rejection rate [4]. In this study, it was hypothesised that the acceptance rate would be higher for athletes having tested the system during on-water experiments. As mentioned by Hunt et al. [18], sounds that would normally be judged annoying can be acceptable to people placed in an intimate control loop with the sonification system. Moreover, recent investigations on the effect of musical agency have shown that interacting with a musical auditory feedback reduces the perceived fatigue during workout [7]. This suggests that the use of interactive sonification could lead to a decreased exertion in the context of elite rowing, and therefore improve the acceptance of the system.

2.2 Experiment 1: discrete sound feedback

Observation of training practice, along with discussions with coaches and rowers, led us to focus on stroke rate regularity. A training session is often characterised by a predefined stroke rate that the athlete attempts to match during a certain time usually comprised between 5 and 30 min. Stroke rate during practice in a single scull normally ranges from 18 to 34 strokes per minute, the latter being the approximate pace reached during race. Although not considered a strong indicator of performance like average velocity or power production, stroke rate regularity may characterise a certain level of control that a rower has on the timing of forces and movements, therefore potentially unveiling some of the rower’s skills, or her fitness on that particular day.

To help them maintaining a stable stroke rate, rowers use most of the time a small electronic stroke rate meter such as StrokeCoach from Nielsen–Kellerman providing the instantaneous stroke rate, time elapsed, and stroke count in real time. This interactive feedback is essentially visual and is apparently not considered intrusive by the rowers. As mentioned previously, using the auditory modality is a natural way of providing feedback about motion, especially in the case of a synchronisation task like stroke rate matching. It is not a case that, on Roman galleys, the pace and synchronisation of the rowing slaves was controlled by a drummer, nor that music is naturally paired with dancing as the body easily synchronises on the beat of the music: rhythm is intrinsically related to motor system and coordination. Repp and Penel demonstrated the better ability for humans to synchronise with rhythmical stimuli in the auditory modality rather than in the visual modality [23].

Therefore, we believe that interactive sonification has the potential to provide an aid for the rowers to synchronise with the desired stroke rate. Listening tests performed in a non-interactive context [4] showed that both casual and elite rowers were able to distinguish randomised data samples belonging to two subgroups of training stroke rate values—low (17–18 strokes per minute) and medium (26 strokes per minute)—independently of the sound model that was used to display the data.

To assess the effects of sonification on stroke rate regularity, a pilot experiment was designed and conducted with a female rower from the Swedish national team (age: 24, rowing experience: 14 years) on lake Magelungen in Stockholm, Sweden. The primary goal of the experiment was to investigate whether a discrete sonification could complement the visualisation system StrokeCoach that she had been using for a long time. A secondary objective consisted in providing an overview of kinematics of a single scull at different stroke rates.

A tri-axis accelerometer (Witilt v3.0 from SparkFun Electronics) was used to stream acceleration data into a smartphone (N95 from Nokia, running Symbian OS). The data was fed directly at a frequency of 10 Hz into a Python script via the program Python for S60 installed on the smartphone. Another tri-axis accelerometer (x-IMU from x-io Technologies) was used to record kinematic data on a micro SD card at a sampling rate of 389 Hz. Two amplified monophonic loudspeakers (Soundball from Goobay) were connected to the sound output of the smartphone using a parallel Y cable. The whole setup was compact enough to be taken onboard. All devices were taped on the scull behind the rower’s seat as shown in Fig. 1, therefore streaming and logging quantities related to the motion of the boat.

Fig. 1
figure 1

The equipment used in Experiment 1 consisted in a smartphone running Symbian OS, two monophonic loudspeakers, and two accelerometers. Boat acceleration was streamed to the smartphone by the first accelerometer and logged at a higher sample rate by the second one. The auditory feedback was synthesised using the MIDI synthesiser of the smartphone

Full size image

The sonification used in this experiment consisted in the discrete part of the sound model Musical instruments described in details in [4], i.e., earcons aiming at providing an interactive feedback concerning the instantaneous stroke rate. In this model, peaks of maximum acceleration are detected by the algorithm and used to render the time lag of the current stroke. The sound of a drum hit is played as soon as an acceleration peak is detected, followed by the sound of a ringing bell triggered after a constant time delay corresponding to the period of the rowing cycle for the chosen stroke rate. The objective for the rower is to synchronise the sound of the bell with the next drum hit. For the rower to experience a rhythmical flow, additional tones in the form of low-pitched percussive impulses were added to the original model. The target time interval following a detected peak, which depends on the chosen stroke rate, is divided into four equally spaced beats. In this way, the resulting inter-onset interval implied by a typical stroke rate value is comprised between 440 and 800 ms, corresponding to rhythmic duration scale, i.e., relative changes of the timing of events within the auditory stream [5]. This particular time scale is propitious for creating an intimate action–feedback loop to guide the movement of the rower into the correct timing.

The auditory display can in fact be described as a sort of adaptive metronome: rowing cycles can then be associated with bars of time signature , segueing all the more smoothly into one another as the actual stroke rate is close to the target stroke rate. The remaining part of the model Musical instruments, consisting of a continuous mapping of velocity fluctuations to the center frequency and inter-tone duration of a trill, was not included in this experiment. The sound synthesis was performed by the Python script running on the smartphone using a wrapper enabling direct and instantaneous control of the MIDI synthesiser built in the mobile phone.

The rower was asked to row at the following stroke rates: 20, 25, 30, and 34 strokes per minute, during 2 min at the lower values (20 and 25 min\(^{-1}\)) or 1 min at the higher values (30 and 34 min\(^{-1}\)). For each stroke rate value, two trials were conducted: at first using only the visual feedback system, then with both visualisation and interactive sonification. For the rower to warm up progressively and to be in a similar physical shape in the control and experimental conditions corresponding to a given stroke rate, the trials were not randomised: the stroke rate value increased stepwise as the experiment went on.

2.3 Experiment 2: continuous sound feedback

From biomechanical studies of rowing, it is well known that large intra-cyclic velocity fluctuations are detrimental to rowing efficiency [14, 19]. In a steady-state regime, i.e., when the average velocity of the boat is constant from one cycle to another, the following power equation applies: the power produced by the rower is equal to the sum of the power dissipated by drag forces (due to friction of the boat hull with water, proportional to a power function of velocity) and the power dissipated at the oar blades (by transferring kinematic energy to water throughout the drive phase). To investigate the dependency of power distribution to stroke rate, Hofmijster et al. [16] introduced several measures of efficiency: velocity efficiency takes into account power lost due to velocity fluctuations, propelling efficiency is related to power lost at the blades, and net efficiency combines the two to provide information on the quality of a rower’s own technique in the scope of minimising power loss. In a recent study, de Brouwer et al. [3] showed that, in the case of a crew of several rowers, antiphase coordination might result in an improved velocity efficiency due to the reduction of velocity fluctuations. This innovative technique, although more unstable that in-phase coordination, and requiring consequent material modifications (e.g., longer boats), illustrates the issue of power loss due to velocity fluctuations.

The object of Experiment 2 is to evaluate the effect of an interactive auditory feedback within the musician-instrument paradigm described in Sect. 2.1. More specifically, the experiment aims at assessing whether an enhanced perception of boat motion leads the rower to modify her technique to reduce energy loss due to velocity fluctuations. To evaluate the effects of a continuous sound feedback on rowing kinematics, we conducted a pilot experiment with the same rower who participated in Experiment 1 (referred to as R1) in the bay Tallaröfjärden in Vaxholm, Sweden. The same model of tri-axis accelerometer (x-IMU) was used to record kinematic quantities related to the boat. However, interactive sound feedback was generated by a more recent smartphone (Galaxy SII from Samsung running Android OS). Having noticed that extreme values of acceleration in the range of stroke rate values tested in Experiment 1 did not exceed 13 m s\(^{-2}\) in absolute value, we decided to use the tri-axis accelerometer built in the smartphone ranging up to 19.6 m s\(^{-2}\) to stream input data into the sound synthesis program. The design of the different sound models was conducted on a personal computer using the software PureData. A customised version of the program ScenePlayer (a port of PureData on Android) was used to run the patches on the smartphone. The original ScenePlayer program, embedding the PureData vanilla distribution, was recompiled from source to include various externals, among which hip (high-pass filter for float streams) and fifo(first-in, first-out buffer structure for float numbers).

The technique of parameter-mapping sonification [12] was used in this experiment with the sonification models Pure tone (using the mapping Velocity \(\rightarrow\) Pitch), Wind (Velocity \(\rightarrow\) Loudness) and Car engine (Acceleration \(\rightarrow\) Spectral centroid), described in details in [4]. All the corresponding mappings functions are linear. A fourth model consisting in a superimposition of Wind and Car engine was called Wind + Car engine. One amplified monophonic loudspeaker (Portasound from Roxcore) was connected to the sound output of the smartphone. The whole setup was compact enough to be taken onboard. All devices were taped on the scull behind the rower’s seat as shown in Fig. 2, therefore streaming and logging quantities related to the motion of the boat.

In a first part of the experiment, the rower was instructed to try out each of the four models during a short time to get a sense of the link between her actions and the resulting sound. The order of the models was randomised under the following constraint: the Wind + Car Engine model had to be tested after both Wind and Car Engine models. The rower was briefly introduced to each model before testing it via an explanation of the sonification mappings involved. After each trial, she was asked to fill a questionnaire (presented in the next section) reflecting her appreciation of the models. She was then asked to establish an overall ranking of the models. The most preferred model was then chosen for the next part of the experiment, which consisted in a 20-min long rowing session divided in ten sequences of 2 min where sonification was turned on (experimental condition) and off (control condition) alternately, the first sequence being silent. The rower was instructed to pay attention to the sound whenever the sonification was on, but was free to choose her pace and rowing strategy. To collect impressions about how the rower experienced her interaction with the sonification system, an interview was conducted immediately after the session.

Experiment 2 was repeated with 7 elite rowers (4 male, 3 female; mean age: 21.7 years; average rowing experience: 9.7 years; referred to as R2–R8) and 2 national coaches (both male; mean age: 46 years; average rowing experience: 35 years; referred to as C1–C2) during a training camp of the Swedish national team on the artificial flat-water course in Račice, Czech Republic. Participants (including the rower who took part in the pilot experiment) are presented in Table 1.

Table 1 The participants to Experiment 2 were 8 elite rowers (R1–R8) and 2 national coaches (C1–C2)
Full size table
Fig. 2
figure 2

The equipment used in Experiment 2 consisted in a smartphone running Android OS, one monophonic loudspeaker, and one external accelerometer. Boat acceleration was collected by the smartphone’s internal accelerometer and fed into the sonification model directly, as well as logged by an external accelerometer at a higher sample rate. The auditory feedback was synthesised using the program ScenePlayer running on the smartphone

Full size image

2.4 Feedback from the rowers

A particular focus was set on the degree of acceptance of the system by the rowers. As mentioned in the previous subsection, all participants were given the possibility to express thoughts and criticism during personal interviews realised immediately after Experiment 2. Free to mention any issue they considered important, they were asked the following questions:

  • How did you interact with the sound?

  • Did you come up with a particular strategy?

  • Was the experiment design suitable, with respect to your training habits?

  • Would you like the sound to come more often? Less often?

  • What was your general impression?

Finally, they were offered the opportunity to provide suggestions about different types of sound design, either in the form of a verbal description or through voice sketching.

The evaluation of the models was conducted in a similar way as for the listening tests of our previous study [4]: after each trial of the first phase of Experiment 2, the participants were asked to assess the corresponding sonification model by answering a set of questions in the form of eleven-step Likert bipolar scales. These questions, referred to as Questions Q1–Q8, correspond to rowers’ preferences in terms of function and aesthetics. They are presented in Table 2. The participants were then asked, in the form of a polar question, if they would agree to use this kind of sound during their training (Question Q9). Finally, they had the opportunity to write free comments concerning the model.

Table 2 Questions used for the evaluation of sonification models in Experiment 2
Full size table

All the rowers had Swedish as native language and had good knowledge of English. Since the questionnaires were written in English, the experimenter provided a purely linguistic assistance whenever they were unsure about a word or a question. The interviews were conducted in Swedish and later transcribed in English by the experimenter.

3 Results and analysis

3.1 Data processing

Fig. 3
figure 3

The process to compute velocity fluctuations around average velocity from accelerometer measurements was the following: raw acceleration was integrated, giving unrealistic velocity values due to the accelerometer drift (blue curve on the left). A Gaussian convolution with a relatively large parameter was applied to the resulting curve to get an average value of the unrealistic curve (red curve on the left). Values resulting from the convolution were then substracted from the unrealistic velocity to give velocity fluctuations around the average velocity in the case of steady-state rowing (curve on the right)

Full size image

Acceleration samples were collected by the x-IMU tri-axis accelerometer at a sampling frequency of 389 Hz. Acceleration data contain unpredictable error, that is not constant nor linear, called accelerometer drift. Due to this phenomenon, a direct integration of the acceleration to get velocity would give unrealistic results. Yet we believe that boat velocity is a very important quantity to control in rowing training; therefore, it should be one of the physical quantities to consider as input data when designing an interactive sonification system. The ultimate objective when looking for the optimal rowing technique is to optimise the kinematic efficiency, i.e., maximise average velocity of the boat at a given power production level. While the actual value of boat velocity can not be derived from accelerometer measurements alone due to the drift issue, it is possible to compute an approximation of velocity fluctuations around the average velocity.

In the implementation of the sonification models mapping velocity to a given auditory parameter, this problem was solved using the following algorithm: raw acceleration was integrated to give a quantity homogeneous to velocity (yet unrealistic), and a moving filter was applied to the resulting data stream to provide an approximation of its average value over two rowing cycles. This locally averaged value was then substracted from the unrealistic velocity value computed in the first place to give an approximation of local velocity fluctuations.

For a data analysis performed in off-line conditions, as in the following subsections, more powerful tools can be used to get a better approximation of local velocity fluctuations. Firstly, a pre-processing of the acceleration data was performed to remove its DC component. The principle used for deriving velocity fluctuations from acceleration data was the same as previously, but the locally averaged value could be computed more precisely via a Gaussian convolution with a relatively large parameter (\(\sigma =\)1,000). This process is illustrated in Fig. 3.

In the following, we assume steady-state rowing, i.e., the average velocity does not vary from one cycle to another. Therefore, acceleration samples corresponding to any sort of transient state (e.g., initial acceleration, pauses, turning, or steadily increasing stroke rate) were not considered in the analysis.

3.2 Stroke rate stability

To assess the stability of the stroke rate, the quasi-period of each rowing cycle of a given trial was extracted to compute the corresponding instantaneous stroke rate. The peak detection algorithm used in the sonification model in Experiment 1 took local acceleration maxima as reference points. However, we found local velocity minima to be more appropriate for separating cycles because they occurred at a well-defined instant. In a rowing cycle, minimal velocity occurs always shortly after the catch (the moment when the oars enter the water) while the point of maximal acceleration depends on the stroke rate.

In each trial of Experiment 1, the rower attempted to match exactly a target stroke rate value (TSR). By computing the average stroke rate and comparing to the TSR, we could calculate the average discrepancy between the rower’s target motion and its actual execution. By computing the standard deviation of the actual stroke rate, we could observe the stability of the performance. These results are presented in the upper part of Table 3. In every trial, the actual stroke rate was found to be slightly higher than the TSR. Pairwise Student’s \(t\) tests were performed between the mean values of the actual stroke rate in the two conditions. For all values of the TSR, the actual stroke rate was found to be higher in average when the sonification was activated (\(\mathrm{\textit{SR} }_1\)) than in the control condition (\(\mathrm{\textit{SR} }_0\)). This means that the discrepancy with respect to the TSR was greater with the sonification than without. However, this difference was significant (\(\alpha = 0.01\)) in only one case, namely at 20 strokes per minute. The standard deviation was found to be dependent on the stroke rate, increasing with the TSR. No influence of the sonification on the standard deviation could be determined.

Table 3 Comparison of stroke rate values computed in the two experiments
Full size table

Although this was not the main objective of Experiment 2, the influence of a continuous auditory display on the stroke rate could be studied as well. No TSR was given to the rowers during this experiment, to interfere as little as possible with their training routine. Nevertheless, we knew from experience that rowers very often organise a training session by choosing a specific stroke rate, using a visual feedback device to try to row at this pace as regularly as possible. The analysis of acceleration data collected in the 20-min long session of Experiment 2 confirmed this fact: participants held approximately the same stroke rate during the whole session. Only one participant tried to “play” with the sonification at some point by increasing steadily the stroke rate over a 2-min long sequence, which was not considered for the analysis of stroke rate regularity. In the lower part of Table 3, mean values of the actual stroke rate computed for each participant are displayed in both conditions (with and without sonification) along with the standard deviation. Pairwise Student’s \(t\) tests were performed between the mean values of the actual stroke rate, indicating whether a participant used a significantly different stroke rate in the two conditions. Such a significant difference was found for 4 participants (\(\alpha =0.01\)): two had a greater stroke rate in the experimental condition, while two others rowed at a greater stroke rate in the control condition. Standard deviation of the actual stroke rate was slightly smaller in the experimental condition for 7 participants out of 10.

3.3 Velocity efficiency

To assess velocity efficiency, i.e., to determine the amount of power lost due to velocity fluctuations around the mean velocity \(v_0\), we studied the personal technique of each rower. Velocity fluctuations, computed as specified in Sect. 3.1 to approximate \(\Delta v (t) = v(t) - v_0\), were extracted from each rowing cycle taken into account in the analysis of the stroke rate and were rescaled in time to a normalised interval \([0,1]\). For each trial, the set of all resulting normalised cycles was then averaged to get the average shape of the velocity curve within a cycle at steady state.

To compare the efficiency of the different shapes obtained, we use the Euclidean norm for a function \(f\) defined and Riemann integrable over \([0,1]\):

$$\begin{aligned} \Vert f\Vert = \sqrt{\int _0^1(f(\varphi )^2)d\varphi } \end{aligned}$$
(1)

The norm of the average cycle of velocity fluctuations, \(\Vert \Delta v\Vert = \Vert v-v_0\Vert\), represents the distance of the actual velocity cycle to the optimal velocity, in theory occurring if the boat were travelling at constant velocity \(v_0\) [14].

In Experiment 1, a trial corresponded to a given stroke rate value. The average shape of velocity fluctuations was found to depend strongly on the stroke rate, as illustrated in Fig. 4 representing the averaged normalised cycle for each of the four values tested. Only curves corresponding to the silent condition are displayed, the curves being identical when the discrete sonification was activated. The discrete sonification was actually designed to help the rower to adust her timing, and not to affect the rowing technique. We can observe that the peak of maximum velocity occurs later in the normalised cycle as the stroke rate increases. To investigate the efficiency of the four curves, \(\Vert \Delta v\Vert\) was computed and is displayed in Table 4. A regression shows that the relationship between \(\Vert \Delta v\Vert\) and the actual stroke rate \(\mathrm{\textit{SR} }_0\) is strongly linear (\(r^2=0.999\)), the norm of the fluctuations increasing with the stroke rate.

Fig. 4
figure 4

To observe the shape of velocity fluctuations (\(\Delta v\)) for each TSR tested in Experiment 1, the phase of every rowing cycle detected was extracted and the cycle was normalised in time accordingly. The resulting series of normalised cycles was used to compute an average cycle. The four curves correspond to four different stroke rate values in the control condition (without sonification). The structural difference of the average cycle can be observed depending on the stroke rate value: the peak of highest velocity occurs towards the end of the drive phase at a lower stroke rate, while it takes place at the end of the recovery phase for higher stroke rates. We can also notice that the amplitude of the fluctuations is larger for a higher stroke rate

Full size image
Table 4 For each target stroke rate (TSR) tested in Experiment 1, the mean stroke rate value measured in the control condition (\(\mathrm{\textit{SR} }_0\)) is displayed, together with the Euclidean norm of the average cycle of velocity fluctuations (\({\Vert \Delta v\Vert }\))
Full size table
Fig. 5
figure 5

For each rower participating in Experiment 2, average cycles of velocity fluctuations computed in the control condition (solid blue curve) and in the experimental condition (dashed red curve) are compared. Very few differences can be observed, suggesting that the sonification did not have a significant effect on velocity fluctuations

Full size image

For each participant in Experiment 2, two normalised curves of velocity fluctuations were extracted using the same process as in Experiment 1, corresponding respectively to control condition and experimental condition. The curves of the ten participants are displayed in Fig. 5, together with the approximate stroke rate held during the 20-min long session regardless of the condition (control or experimental). Close values of the average stroke rate allow us to observe differences in individual technique, e.g., for participants R1, R2, and C1, or for participants R5 and R6. For the majority of rowers, the curves corresponding to the two conditions are almost identical, showing that sonification did not have any effect on velocity fluctuations. In addition, \(\Vert \Delta v\Vert\) was computed for each participant in the two conditions. The results, presented in Table 5, evidence no significant difference on velocity fluctuations when rowing with and without sonification.

Table 5 For each rower participating in Experiment 2, the mean stroke rate value measured in the control condition (\(\mathrm{\textit{SR} }_0\)) and in the experimental condition (\(\mathrm{\textit{SR} }_1\)) is displayed, together with the Euclidean norm of the average cycle of velocity fluctuations (\({\Vert \Delta v\Vert }_0\), respectively, \({\Vert \Delta v\Vert }_1\))
Full size table

3.4 Perception of the four models by the rowers

3.4.1 Questionnaires

A one-way ANOVA with factor Sonification model was conducted on the participants’ values separately for each of the questions listed in Table 2 (Q1–Q8). A significant effect of this factor (\(p<0.05\)) was observed for Questions Q4 (\(F=4.727\)), Q5 (\(F=4.005\)) and Q7 (\(F=3.263\)). Pairwise comparisons were analysed to find significant differences for the mean of this factor (Least Square Difference post hoc comparison, \(p < 0.05\)). The results show that Wind was found better associated with rowing than Pure tone and Car engine (Question Q4). Wind was also judged more pleasant than both Car engine and Wind + Car engine, and Pure tone was judged more pleasant than Car engine (Question Q5). The Car engine model was found more intrusive (Question Q7) than the two models Pure tone and Wind.

For each model, we define the acceptance rate as the percentage of positive answers to the polar question Q9. From the acceptance rate we could derive the following ranking of the models: 1. Wind (80 %), 2. Wind + Car engine (60 %), 3. Pure tone (44 %), 4. Car engine (10 %). This ranking is consistent with the explicit ranking expressed by the rowers before the 20-min long session, which was conducted with their favourite model. It is also consistent with the ranking according to aesthetic qualities that could be derived from non-interactive listening tests performed in our previous study [4].

3.4.2 Interviews

When interviewed after having completed Experiment 1, the participant expressed several concerns. First, she found the experiment very tiresome. Second, she could not really relate her actions to the resulting sound. Third, she thought that the moment when the discrete feedback was triggered was not suitable: corresponding to the peak of maximum acceleration, it occurred when she was relaxed at the end of the catch phase, while she would have expected it to come at the beginning of the catch when the force applied to the oars was maximal. Fourth, she had trouble associating the bright sound of the ringing bell with an event that had been happening a few seconds before (the previous acceleration peak), and suggested to use it for the current acceleration peak instead.

Interviews conducted after Experiment 2 focused on several aspects of the experiment: the strategy that the rowers had been using in relation to the auditory feedback, how they were experiencing the interaction with the different models, aesthetic considerations, and how they thought that the models could be improved. Illustrative statements extracted from these interviews, referenced hereunder, have been gathered in Table 6.

Table 6 Statements by the participants of Experiment 2 after having tested the four different sonification models
Full size table

Rowers were not instructed about a particular strategy to use. Instead they were explained the models in theory and instructed to try to find a particular strategy. Since the model chosen by the rowers was Wind in 80 % of the cases, their strategy could be compared: some of them tried to have a longer sweeping sound at the end of the stroke (R1, R2, R4, C1), others tested how much variation could be heard when rowing softer or harder (R2, R3, R6). Despite the instructions, two participants reported that they tried to forget about the sound because it disturbed them (R5, C2). The rower who chose the Car engine model (R7) tried to get a sound as high-pitched as possible because she understood that it meant a greater acceleration. She stated that it led to a rowing technique that was somewhat unusual for her. In the case of the Wind + Car engine model, the rower expressed how he combined the two simultaneous feedbacks (R8–a).

When describing how they interacted with the four different sonification models, some participants underlined the drawbacks of a given model (R4–a, R7–a). They also commented on the structure of the experiment, most of them finding it suitable and not interfering with their training habits (R1–a). However, a few participants would have liked to hear the sonification without interruption throughout the whole session (R2, R7, C1–a). In general, they accepted the principle of using an interactive sonification system for training, and they found the experiment exciting (R1–b, C1–b). However, some participants expressed doubts regarding the potential of such a system to improve their performance (R5–a, R8–b, C2–a).

They reviewed the aesthetics of the sound models, stating what they were expecting in terms of design (R1–c, R8–c, C1–c). Some of them emphasised the advantage of using environmental sounds, such as the sound of water or wind (R2–a, R4–b). On the other hand, others suggested to redesign some of the models (R5–b, C1–d, C2–b). Interestingly, one rower (R1) found the Car engine model to sound like an ergometer wheel, a sound that was rather associated with Wind by two other participants (R2, R3). Finally, one participant commented on the technical setup (R2–b).

4 Discussion

Experiment 1 was aimed to assess the effects of a discrete sonification system on stroke rate stability. For four values of a target stroke rate (TSR), the actual stroke rate was computed, both without and with interactive auditory feedback. In all cases, the actual stroke rate was found to be slightly larger than the TSR. For all values of the TSR, the actual stroke rate was found to be slightly higher in the experimental condition than in the control condition, although this difference was significant in one case only. Hence, the discrepancy with the TSR was larger with the sonification, i.e., the interactive feedback did not help the rower to match the intended stroke rate. The computation of stroke rate standard deviation did not reveal any effect of sonification on stroke rate stability. This absence of effect could be explained by the simplistic character of the auditory display, reducing communication bandwidth to a discrete interaction. Another reason could be the level of excellence of the participant who was an experienced rower belonging to the Swedish national team, capable of achieving a close-to-optimal performance in the control condition.

In a study of interactive sonification of rowing, Schaffert et al. [25] observed an increase of average velocity for different boat categories—including single scull—when a continuous auditory display was provided to the rowers. Their sonification model was similar to the Pure tone model introduced in the present work, using the mapping Acceleration \(\rightarrow\) Pitch instead of Velocity \(\rightarrow\) Pitch in our case. The fact that they also noticed a modification of the structure of the acceleration cycle in the condition with sonification suggests that the actual stroke rate might have been slightly higher. Since it is known that stroke rate and average velocity are correlated [16], this would contribute to the effect observed by Schaffert et al. together with other factors such as a better crew synchronisation thanks to the auditory feedback. Another explanation could lie in the design differences between the two experiments: the mapping of our Wind model involves loudness as output auditory parameter whereas Schaffert et al. used pitch to reflect variations in the data, possibly enabling the rowers to better focus on parts of the cycle where the boat is slowing down and therefore improve rowing efficiency. Indeed, in a preliminary version of their experiment, some participants claimed to have tried to minimise the deceleration of the boat at specific key points of the cycle [24].

A second objective of Experiment 1 was to study the dependency of some rowing kinematic quantities to stroke rate. Stroke rate standard deviation increased with the TSR, reflecting a higher variability in the timing of the boat motion. This suggests a decreased level of control by the athlete when rowing at a higher pace. An analysis of velocity fluctuations around average velocity was conducted by computing a normalised average rowing cycle for each value of the TSR. A measure of velocity fluctuations (Euclidean norm) was found to be linearly dependent on the actual stroke rate value, indicating a decrease of velocity efficiency at a higher pace. These two results are consistent with a previous study by Hofmijster et al. [16] on the effect of stroke rate on mechanical power distribution, where velocity efficiency was found to decrease linearly when stroke rate increased.

In Experiment 2, we investigated the influence of a continuous auditory display on velocity efficiency and stroke rate. In a first phase, participants had the chance to get familiarised with four models for interactive sonification. In a second phase, their favourite model was used for a 20-min long rowing session to provide an interactive auditory feedback during sequences of 2 min. Although no TSR was given in instruction, the actual stroke rate was found to be relatively stable for all participants, showing that rowers usually try to keep a constant tempo during their training. No effect of sonification on stroke rate value could be evidenced. Nevertheless, the presence of an auditory feedback led a majority of participants to a slightly reduced stroke rate standard deviation, suggesting a tendency for a more regular rowing action. An average cycle representing velocity fluctuations around average velocity was computed in the two conditions (with and without sonification) for each participant. Analysis of these average cycles revealed no effect of sonification on velocity fluctuations.

Questionnaires and interviews were conducted to get a direct feedback about how rowers experienced the system. The principle of sonification was well accepted by most participants, and aesthetics of the models were criticised: one model (Wind) was preferred by a large proportion of participants, while another model (Car engine) was very unpopular. For two models (Wind and Pure tone), the acceptance rate was higher in the case of on-water experiments than in the previous study based on listening tests [4], while it decreased for another model (Car engine). Although most participants claimed to have used a personal strategy in relation to the auditory feedback, no corresponding effect could be observed in the average curves of velocity fluctuations displayed in Fig. 5.

If no positive effect of the system on kinematic quantities could be highlighted in Experiment 2, a longer training period might be required to learn an efficient way to interact with the sonification. Confronting the produced sounds with feedback from the coach could allow the rower to learn how it “should” sound, and then to try to reproduce the corresponding movement patterns with help from the auditory feedback. Although our continuous auditory displays were expected to be judged intrusive by some participants (based on mixed reviews collected in a non-interactive context [4]), no negative effect could be shown either. These preliminary results suggest that the system, even if intrusive, did not change the performance of the participants and this can be seen as a sign of easy adoption of such a system. This also emerges from some of the interviews where the participants declared to be actively making use of the sonification. This suggests that the system could be tested over a longer period, provided that the athletes validate the aesthetics of the model they would use.

5 Conclusion

We presented an application of interactive sonification aimed at augmenting information provided to elite rowers during training. Two experiments were conducted to assess potential effects of an interactive auditory feedback on two kinematic quantities related to the boat: stroke rate and velocity fluctuations. A measure of velocity fluctuations was introduced and applied on average curves representing the shape of velocity fluctuations in steady-state rowing. Sonification was found to affect the stroke rate of a few participants; however, no general trend could be underlined. No effect of sonification could be observed on the measure of velocity fluctuations, which was found to depend linearly on the stroke rate. Interviews and questionnaires were conducted with the participants to evaluate the degree of acceptance of each continuous auditory feedback, and to adapt them to the needs and preferences of the rowers. In this way, the sonification models could be improved for future experiments, in which the evolution of kinematic quantities could be monitored over longer trainig periods.