1 Introduction

Coordination dynamics, as a quantitative field established in the mid 1980s, opened a new level of applications to synergetics outside the hard core sciences. Hermann Haken’s proposition that phase transitions far from thermal equilibrium play a much bigger role than widely thought was open for a new test. The concepts of synergetics had been applied with great success to the laser and hydrodynamic instabilities, systems where the mesoscopic or even microscopic dynamics of the subsystems are well known and from which the order parameters guiding the macroscopic dynamics can be derived—a so-called bottom-up approach. In coordination dynamics this situation is reversed: There are transition phenomena in movements that have been found experimentally and the goal is to establish a dynamics of order parameters at the macroscopic level and then find a description on a lower level from which the macroscopic dynamics can be derived. Such an approach is a top-down example of synergetics. The two approaches, top-down and bottom-up, are of course complementary [1].

Here we start with a brief review of the original Haken-Kelso-Bunz (HKB) model [2] and its generalizations and then focus on three applications of coordination dynamics, namely to probe the human brain of individual subjects, to relate brain and behavior in social coordination, and to establish the building blocks of complex movements in professional ballet dancing.

2 The Origin of Coordination Dynamics

In what ways can you move your two index fingers rhythmically? According to the lore of coordination dynamics, most people without special training are able to produce two stable movement patterns at low rates, i.e. flexion of one finger while the other extends (anti-phase) and simultaneous flexion and extension of both fingers (in-phase). Surprisingly, only the in-phase movement can be performed beyond a certain frequency of the movement that may vary across individuals. When a movement starts in anti-phase and the movement rate is increased, the coordination pattern switches spontaneously and involuntarily to in-phase; if the movement starts in in-phase no switch is observed. These observations by one of us [3, 4] led to a fruitful collaboration with Hermann Haken in Stuttgart and to publications (e.g., [2, 5]) that are the pillars of coordination dynamics as a quantitative science. The importance of these publications is at least twofold: First, they take the phenomenon described above from the real world of two fingers doing some seemingly trivial wagging movements to an abstract level of identifying order parameters, and deriving them using the theory of coupled nonlinear oscillators and stochastic dynamics. The movement rate becomes a control parameter (that leads to destabilizing the coordination between the limbs), and the relative phase \(\phi \) is established as the order parameter, i.e. the relevant variable for the description of the two rhythmically moving fingers at a macroscopic level. The switching that occurs when the movement speeds up is seen as a ball moving in a potential landscape in an overdamped fashion that changes its shape as a function of rate expressed in the famous equations

$$\begin{aligned} \begin{array}{ll} &{} \quad \dot{\phi } = -\frac{dV(\phi )}{d\phi } + \sqrt{Q} \xi _t = -a \sin \phi - 2 b \sin 2\phi + \sqrt{Q} \xi _t \\ &{}<\xi _t> \, =\,0; \quad <\xi _t \xi _{t'}>\,=\, \delta (t-t'); V(\phi )=-a \cos \phi - b \cos 2\phi \end{array} \end{aligned}$$
(1)

where \(\sqrt{Q} \xi _t\) is Gaussian white noise of strength Q and b/a represents the movement rate. A decrease in b/a corresponds to an increase in rate with a critical value of \(b/a=0.25\). The shape of the potential function below and above the critical rate is shown in Fig. 1. Second, and maybe even more important, the theory that was formulated in these initial papers more than 30 years ago modeled the switch in the movement patterns from the viewpoint of synergetics as a second order phase transition and made predictions that could be tested experimentally (see [6, 7], for reviews). On the deterministic side, an oscillator, known as the hybrid, was established that follows the amplitude-frequency relation found in human finger movements [8]; the model was extended to situations where the individual components are not symmetric (see below) and to gaits and gait changes in quadrupeds [9]. On the stochastic side it was shown that the switch exhibits the features of a phase transition in non-equilibrium systems: The system has to become unstable before the switch actually occurs. Such an instability leaves footprints like an increase in the variance of the movement (enhanced fluctuations) and a longer time to recover from a perturbation (critical slowing down) when the critical point is approached but before the actual transition takes place. The experimental discovery of these hallmarks showed that the spontaneous change from anti-phase to in-phase is not simply a replacement of one motor program by another but a self-organized pattern forming process that occurs in systems far from thermal equilibrium. Moreover, the dynamic principle seems to be universal: Whether it is two fingers, two hands, an arm and a leg [10, 11] or even the legs of two different people watching each other [12], it is always the anti-phase movement that becomes unstable at increasing rate and the coordination pattern switches to in-phase.

Fig. 1
figure 1

Potential landscape from (1) for slow movement rates (\(b/a=0.7\), left) with minima for anti-phase and in-phase. For high rates (\(b/a=0.15\), right), where only one minimum exists, the only stable movement pattern is in-phase

Fig. 2
figure 2

When the interacting components are not symmetric, the potential landscape is tilted due to a linear term \(\delta _{\omega }\phi \). The fixed points are shifted away from 0 and \(\pi \), the system becomes unstable at slower rates (here \(b/a=0.4)\), and the transition has a preferred direction

In the original work, where the interacting limbs were two fingers with the same eigenfrequencies, the system in terms of the oscillating limbs has a symmetry with respect to an exchange of the components. When we look at the coordination between an arm and a leg or the syncopation of one finger with an external stimulus, this symmetry is no longer present, i.e. we are now dealing with a system of oscillators with different eigenfrequencies. If the equation for the relative phase (1) is derived for such a system, an additional term, usually termed \(\delta _{\omega }\), appears in the equation of motion and becomes a term \(\delta _{\omega } \phi \) in the potential function [13]. The two main effects of this term for the dynamics of the system are: First, the coordination patterns are no longer strictly in-phase or anti-phase, i.e. at a relative phase of 0 or \(\pi \), respectively, but the stable states are shifted with respect to these values. As shown in Fig. 2 the potential function is now tilted. As a consequence, the states \(\phi =0\) and \(\phi =2\pi \) are no longer the same. For a system that is initially in an anti-phase pattern at \(\phi \approx \pi \) it is more likely to switch to \(\phi \approx 2\pi \) than \(\phi \approx 0\). Even though after the switch it is not possible to tell whether a system is close to 0 or \(2\pi \), by following the continuous relative phase it can be detected whether the direction was to the left or right. The coordination patterns with broken symmetry between arms and legs were studied intensively using the device shown in Fig. 3 with one of the authorsFootnote 1 as the animal under investigation. This device, known as MAC (multi-articulation coordination), allows for perturbing the movements by applying a brake for a short moment and for manipulating the degree of symmetry breaking. The latter is achieved by using weights that are either attached to the wrist, which decreases the amount of symmetry breaking or on the ankle, which actually increases it. The predictions derived from the theoretical model have been found in good agreement with the experimental results [10, 11].

Fig. 3
figure 3

A subject in the MAC (Multi-Articulation Coordination). This device allows for perturbations of the movements by applying a brake for a short moment and for varying the degree of symmetry breaking by attaching weights to either the wrist or ankle. As the legs naturally have a smaller eigenfrequency than the arms, the degree of symmetry breaking can either be further increased or reduced with loads at the ankles or wrists, respectively

3 Coordination Patterns in Brain Signals

In the early 1990s coordination dynamics was used for probing the human brain, where the syncopation-synchronization paradigm became the vehicle of choice in many experiments. In contrast to the bimanual setup used in many of the behavioral studies, here the subjects are instructed to flex (or extend) their index finger in between two beats of a metronome, i.e. to syncopate with the stimulus. As the rate of the metronome increases, this behavior becomes unstable and the subjects switch to a pattern synchronized with the beats [13]. During the trials the magnetic fields originating from electrical activity in the brain were recorded using magnetoencephalography (MEG), a technology in its infancy at the time. In the first experiment a magnetometer with 37 SQuIDs (Superconducting Quantum Interference Devices) was positioned over left auditory and motor cortex covering roughly 2/3 of a hemisphere. The hallmarks of non-equilibrium phase transitions previously found in behavioral data were now detected in the brain signals [14,15,16]. The experiment was later repeated with a 143-channel device with full-head coverage that allowed for a better comparison of brain activity during the coordination task and signals from control condition of auditory and motor only [17]. One of the main findings from these experiments remains the switch in the phase of the first Fourier component of the brain signal, shown in Fig. 4, accompanying the switch from syncopation to synchronization in the behavior.

Fig. 4
figure 4

Phase shift by \(\pi \) in the relative phase between the Fourier component of the MEG signal at the coordination frequency and the stimulus in sensors over the left hemisphere that coincides with the behavioral switch from syncopation to synchronization. The underlying color represents the signal power

Fig. 5
figure 5

Setup for a dual-EEG social coordination experiment: Two subjects are instructed to perform a certain coordination pattern with their index fingers. For the first 20 seconds they cannot see the other’s movement and therefore cannot coordinate. Then the screen turns transparent and a coordination pattern is established. During the trials the movements and EEG signals from both subjects are recorded

During the last decade, social coordination, i.e. the interaction between two or more people has become an expanding field where not only behavioral measures are recorded from interacting individuals but also brain signals using a dual-EEG for instance as shown in Fig. 5. In this experimental setting two subjects (we call them Red and Blue) are facing each other and are instructed to perform a certain coordination pattern say in-phase with their right index finger when possible. For the first 20 seconds of each trial an opaque screen prevents them from seeing the other’s finger and they move at a comfortable rate. Then the screen turns transparent allowing the subjects to synchronize their movements. During the trials EEG is recorded from both subjects using caps with 60 electrodes each. In earlier work [18,19,20] certain wave patterns were identified in the \(\alpha \)-range around 10 Hz that are tied to coordinated action. Here we introduce a procedure for the segmentation of multi-channel EEG recordings that are band-pass filtered, in our case 7–13 Hz, and apply it to the data recorded in this experiment.

A short sequence of one second from a 60-channel EEG recording is shown as a butterfly plot in Fig. 6 together with the spatial patterns of the electric potential at the maxima and minima and an insert on the right, which shows the color coding of the electrode locations on the scalp. Aside from the oscillation there is also a change in the shape of the pattern from a dipole to a pattern with a single maximum or minimum changing to a quadrupolar shape at the end of the time series. Such changes in the electric potential at the scalp originate from a change in the underlying electrical activity in the cortex, primarily in the macrocolumns of the gray matter. It is our goal to identify segments of the EEG signal where the shape of the spatial pattern is essentially constant while only reversing polarity and relate them to the coordination behavior.

Fig. 6
figure 6

Butterfly plot of 60 EEG channels for one second from the recording of one subject together with the spatial patterns of the electric potential on the scalp at the maxima and minima. There are segments where the signals only reverse polarity while the spatial shape remains unchanged and other time intervals where the pattern goes from dipolar to a single maximum or minimum and on to a quadrupole. Insert Color coding for the electrode locations

In a first step we reduce the dimensionality of the system by applying a principal component analysis (PCA). In PCA a set of patterns is calculated that are best suited to represent the variance in the recordings whilst drastically reducing the number of time series that are needed. PCA is performed by calculating the eigenvalues and eigenvectors of the covariance matrix given by

$$\begin{aligned} C_{ij}=\frac{1}{T} \int _0^T \!\! dt \, \{ e_i(t) - \bar{e}_i\} \{ e_j(t) - \bar{e}_j\} \end{aligned}$$
(2)

where \(e_k(t)\) represents the time series recorded from electrode k and \(\bar{e}_k\) is its temporal mean. As the recordings were done with 60 electrodes for each subject, C is a 60 \(\times \) 60 matrix. The eigenvectors corresponding to the largest eigenvalues of this matrix represent the spatial patterns that carry most of the variance in the EEG signal. The patterns calculated from the time series in Fig. 6 are shown on the left of Fig. 7. The corresponding eigenvalues \(\lambda \) (if properly normalized) quantify how much of the variance is carried by a given pattern. The four eigenvalues for the patterns in Fig. 7 sum up to 0.956, which means the four patterns represent almost 96% of the variance. Because C is a real symmetric matrix its eigenvectors are orthogonal and the amplitude for a pattern k at a time t, \(p_k(t)\), can be computed by projecting the EEG signal vector \(\varvec{e}(t)\) onto the eigenvector \(\varvec{v}^{(k)}\)

$$\begin{aligned} p_k(t) = \varvec{e}(t) \cdot \varvec{v}^{(k)} \end{aligned}$$
(3)
Fig. 7
figure 7

Principal component analysis (PCA) applied to the EEG data shown in Fig. 6. The spatial patterns corresponding to the four largest eigenvalues (left column) carry almost 96% of the variance in the signal. Time series of the corresponding amplitudes (cyan) show oscillations around 10 Hz like the data from the single electrodes. This fast oscillation is removed by calculating complex slowly varying amplitudes shown in red and green for their real- and imaginary part, respectively. The magnitude of these functions represents the envelope plotted in magenta

The amplitudes for the first four modes are plotted in cyan in Fig. 7 next to the patterns. Like the time series from the electrodes they show oscillations around a frequency of 10 Hz. The second step of the procedure is intended to eliminate these fast oscillations and replace them by slowly varying time series like the envelope plotted in magenta in Fig. 7. To this end we perform a Fourier transform on the amplitudes \(p_k(t)\) leading to complex valued functions \(\tilde{p}_k(\omega )\). As the \(p_k(t)\) are real, their Fourier coefficients for \(\omega \) and \(-\omega \) are complex conjugates \(\tilde{p}_k(-\omega )=\tilde{p}_k^*(\omega )\). Next we set the coefficients for all negative frequencies to zero, shift the remaining functions by 10 Hz to the left and apply an inverse Fourier transform. This process eliminates the 10 Hz component from the signal and leads to complex valued amplitudes \(a_k(t)\) whose real- and imaginary part are plotted in red and green, respectively, in Fig. 7. The envelope of the amplitudes \(p_k(t)\) is given by the magnitudes \(|a_k(t)|\).

The dynamics recorded by EEG is now described by the four complex amplitudes \(a_k(t)\), which represent the journey of a trajectory in an 8-dimensional space. If this trajectory stays within a small region of that space for a certain time the pattern is stationary. If the trajectory moves around the pattern changes. In analogy to lower dimensions we can define a velocity for the trajectory as the derivatives of the real- and imaginary parts of \(a_k(t)\) and a scalar speed, s(t), as the square root of the sum of these derivatives squared

$$\begin{aligned} s(t)=\sqrt{\sum _k \left\{ \left[ \frac{d}{dt}\mathfrak {R}\{a_k(t)\}\right] ^2+\left[ \frac{d}{dt}\mathfrak {I}\{a_k(t)\}\right] ^2 \right\} } \end{aligned}$$
(4)

This speed is plotted in cyan in Fig. 8 together with the original data in a butterfly plot and spatial pattern at the maxima and minima. Changes in the spatial shape occur where s(t) has maxima (dashed vertical white lines), which are found as the time points where the temporal derivative of s(t) (plotted in white) intersects the horizontal axis with a negative slope. The segments in-between the vertical lines are time intervals where the shape of the pattern does not change or changes only slightly.

Fig. 8
figure 8

Segmentation of the EEG data shown in a butterfly plot and patterns at the minima and maxima of the signal. The speed along the trajectory in the 8-dimensional space is plotted in cyan and its derivative in white. The time points where the derivative intersects the horizontal axis with a negative slope mark the boundaries of segments (vertical white dashed lines) within which the shape of the pattern is relatively constant

Now we apply this segmentation to the dual-EEG recordings described above and relate it to the coordination behavior of the subjects. As mentioned, for the first 20 seconds of each trial the view of the partner’s finger was blocked by an opaque screen and each individual moved at her own rate. In order to synchronize the movement after the screen has turned transparent these rates have to become the same and there are various ways how this can happen: The faster one may slow down, the slower one speed up or they both change and meet in the middle or elsewhere. Here, we look at the data from 20 trials of a pair, where the red subject, in almost all trials, started at a slower movement rate and sped up to the blue subject, whose frequency stayed pretty much the same. The angular frequencies from a typical trial around the time when the screen turns transparent (from 15 to 26 s) are shown in Fig. 9.

Fig. 9
figure 9

Angular frequencies for the red and blue subject, \(\omega _R\) and \(\omega _B\) for a time span from \(t=15\,\mathrm{s}\) to \(t=26\,\mathrm{s}\). The screen turns transparent at \(t=20\,\mathrm{s}\) indicated by the dashed black vertical line. As in most of the trials Red starts at a lower rate and speeds up to Blue

Fig. 10
figure 10

Histograms of the locations measured by the phases where the segment boundaries fall within the movement cycles for all four brain-finger combinations. Only the boundaries from the red brain show a relation to the movement of the blue finger

We aim to determine whether there is any relation between the segment boundaries extracted from the two EEG data sets and the coordination behavior by determining where these boundaries fall into the movement cycle. This was done by calculating the phase of the movements at each boundary. There are four cases: The phase of the blue (red) finger at the boundaries from the blue (red) brain, and the phase of the blue (red) finger at the boundaries from the red (blue) brain. As expected, no relation was found for the first 20 seconds when the subjects moved individually without vision of the other’s finger movements. But also after movement synchrony had been established for times greater than about 2 seconds after the screen had turned transparent, no relation could be seen. Only during the transition interval from \(t=20\,\mathrm{s}\) to \(t=21.5\,\mathrm{s}\) the histograms exhibited in Fig. 10 were found, showing that the segment boundaries from the red brain are more than twice as likely to fall around a phase of \(\pi \) of the blue finger than a phase of 0 or \(2\pi \) (right upper diagram). This finding is an interesting reflection of the behavioral pattern where Red adjusts her movement rate to the blue partner, whereas Blue seems to simply move at her own rate largely independent of what the Red is doing, hinting at a master-slave or leader-follower relation in both brain and behavior.

4 Coordination Patterns in Complex Movements: Ballet Dancing

In the Fall of 2014 we invited the God daughter of one of us (JASK), the professional ballet dancer (Makaila Wallace from Ballet B.C.) and asked her to perform a choreography of her choosing lasting about 20 seconds in a number of different experimental conditions, e.g. with and without music, fast versus slow, expressing different emotions, etc. The performances were recorded using a Vicon motion capture system with eight infrared cameras and 32 infrared markers attached to the dancer’s body that allowed for a reconstruction of the trajectories of the markers in 3-dimensional space. Snapshots of such trajectories together with stick figure representations of the dancer during two short time intervals of the performanceFootnote 2 are shown in Fig. 11.

Fig. 11
figure 11

Snapshots of the stick figure representing the dancer together with trajectories of the 32 infrared markers attached to different parts of her body. Each of the traces corresponds to a duration of one second. The colored spheres represent the six degrees of freedom for the locations and orientations of the dancer’s body center in space

When the dancer performs the choreography, the movement can be split into a movement of her body center and movements of her body and limbs internal degrees of freedom relative to the center movement. We define the dancer’s center as the center of gravity of a triangle given by the markers at her left and right hip, and a marker at her back close to the lowest lumbar vertebra (L5). The location of these points in the 3-dimensional space of the laboratory are given by three vectors we call \(\varvec{h_l}\), \(\varvec{h_r}\) and \(\varvec{l_5}\), respectively, and determine the vector for the body center

$$\begin{aligned} \varvec{c}=\frac{1}{3} \{ \varvec{h_l} + \varvec{h_r} + \varvec{l_5} \} \end{aligned}$$
(5)

Next, we define a coordinate system for the dancer’s body with the x-, y- and z-axis given by

$$\begin{aligned} \varvec{x}=\frac{(\varvec{h_r}-\varvec{l_5}) + (\varvec{h_l}-\varvec{l_5})}{|\varvec{h_r}+\varvec{h_r}-2\varvec{l_5}|} \qquad \varvec{y}=\frac{(\varvec{h_r}-\varvec{l_5})\times (\varvec{h_l}-\varvec{l_5})}{|(\varvec{h_r}-\varvec{l_5})\times (\varvec{h_l}-\varvec{l_5}) |} \qquad \varvec{z}=\varvec{x}\times \varvec{y} \end{aligned}$$
(6)

After subtracting the translation of the center \(\varvec{c}\) from all marker coordinates and applying a rotation that transforms the laboratory coordinate system into the body frame (6), all movements are relative to the body center location and orientation in space. A snapshot of the trajectories for such a coordinated movement is shown in Fig. 12.

Fig. 12
figure 12

Snapshot of the stick figure together with marker trajectories after the translations and rotations of the body center have been removed. Such transformed time series are much better suited for an extraction of the relevant features of the other body parts, e.g. arms and legs, than the original data

The time series after the center translations and rotations have been removed are much better suited for extracting the relevant features of the movement of other body parts. For instance the movement of the torso and the head can now be described in terms of two additional rigid bodies as translations and rotations relative to the center. Here we will restrict ourselves to the dynamics of the arms and legs, which have 14 and 8 markers attached, and therefore their trajectories in 3-dimensional space are given by 42 and 24 time series (\(a_n(t)\), \(n=1 \ldots 42\) and \(l_m(t)\), \(m=1 \ldots 24\)), respectively. To extract the basic movement patterns from these time series we perform a principal component analysis (PCA) separately for the markers on the arms and the markers on the legs. To this end we build the covariance matrix \(C^{(a)}\)

$$\begin{aligned} C^{(a)}_{ij}=\frac{1}{T} \int _0^T \!\! \{a_i(t)-\bar{a}_i\}\{a_j(t)-\bar{a}_j\} \, dt \end{aligned}$$
(7)

for the arms, where \(\bar{a}_{i,j}\) represents the mean of the corresponding time series. In the same way \(C^{(l)}\) for the legs is found; both are symmetric matrices that have real non-negative eigenvalues and orthogonal eigenvectors. The eigenvectors represent basic movement patterns, whereas the corresponding eigenvalues are a measure of how much a given pattern contributes to the variance in the original time series. Specifically, the eigenvectors describe the deviation from a given state, i.e., each triple of components represents the magnitude and direction of the movement for a certain marker. Therefore, each of the eigenvectors represents a movement pattern of all markers for either the arms or the legs. Interestingly, for both the arms and legs, the vectors corresponding to the six largest eigenvalues show coordination patterns that are either in-phase or anti-phase movements. Three examples from the first six modes for the arms and legs are shown in Figs. 13 and 14, respectively. The dominating arm pattern, accounting for almost 35% of the variance is an in-phase up and down movement (Fig. 13 left). The corresponding anti-phase pattern covering about 12% corresponds to the fourth largest eigenvalue (middle). On the right is the second mode, an in-phase back and forth movement covering about 26% of the variance. In total the six dominating eigenvectors cover more than 97% of the arm movements.

Fig. 13
figure 13

Three examples from the first six modes for arm movement. The dominating pattern is an in-phase up and down movement (left); the corresponding anti-phase pattern is the forth mode (middle); the movement corresponding to the second largest eigenvalue is in-phase back and forth (right)

For the legs, shown in Fig. 14, the two dominant coordination patterns are an anti-phase (56%) and in-phase movement (18%) in the yz-plane (left and middle). The sixth mode is an anti-phase walking pattern in the x-direction and the fifth mode (not shown) is the corresponding in-phase movement. The total variance captured by the first six modes for the legs sums up to more than \(98\%\).

Fig. 14
figure 14

Three examples from the first six modes for leg movement. Modes one and two (left and middle) are an anti-phase and an in-phase pattern in the yz-plane. The sixth mode (right) is an anti-phase walking movement in the x-direction; the corresponding in-phase pattern is the fifth mode (not shown)

It is most intriguing that the complex arm and leg movement by a professional ballet dancer can be captured almost completely by only six basic coordination pattern for both the arms and the legs. Moreover, these patterns fall into groups and can be classified as anti-phase or in-phase movements along three different directions in 3-dimensional space.

5 Summary and Conclusion

We have tried to point to some of the highlights from more than 30 years of our research in coordination dynamics and its relation to synergetics, where the systems’ components form entities and the interaction of such entities, modes or patterns gives rise to self-organization and macroscopic structures. On the macroscopic level, the coordination pattern is not described by the dynamics of the individual finger movements but by their relative phase. The transition in the behavior of this order parameter is also found in the brain signal of the dominating pattern. The way coordination patterns are established (or lost) in social settings leaves footprints in EEG recordings and may allow conclusions regarding the relationship between individuals. The results from the analysis of the ballet dancer in a certain sense take us back to the question we raised at the beginning of Sect. 2, which we can now rephrase as: In what ways do you have to be able to move your limbs in order to perform movements as complex as the choreography of a classical ballet dance? The answer, as we have seen here, is: In-phase and anti-phase in three different directions. Our results attest to a quite remarkable aspect of complex biological systems: They compress their high-dimensionality into lower-dimensional, fundamental patterns that are context-specific. These functional synergies or coordinative structures [21] constitute the building blocks of coordinated behavior in living things. There are good reasons why all healthy humans have these coordination patterns in their repertoire.