1 Introduction

Social neuroscience seeks to bridge the gap between the neural, the behavioral and the social. Such an agenda contrasts with cognitive science and the shortcoming of its brain-centered and individualistic approach to the mind. Recently, several approaches have proposed to go beyond a third person representational account of others by investigating social interaction from developmental, dynamical and relational viewpoints. This departure from a strictly reductionist view calls for new manners of empirical investigation of social systems along with a theoretical account of their various scales of organization. With those advances, one aims to integrate complementary aspects of the problem of social coordination into a coherent, comprehensive and parsimonious whole. In this respect, non-linear dynamical systems theory has already proved a good formalism to relate biological, psychological and more recently social levels [1]. This paper discusses a new experimental paradigm grounded in the framework of Coordination Dynamics [2–4]. We describe the development of Virtual Partner Interaction (VPI) , a system allowing to couple a human with a theoretical model of movement coordination in real time [5, 6]. We review its generalization into the “Human Dynamic Clamp” (HDC) , a new paradigm for Cognitive Science to study the multiple scales of coordination that govern human brain and behavior.

This novel paradigm pursues an already ongoing grip of Cognitive Science toward multiscale coordination [1, 3, 7]. In the exemplary case of hand movements for instance, social interactions span multiple scales in time: from position, phase and frequency of movements to the turn-taking between people (e.g. [8]). Such social interaction also gives rise to neural coordination within and across brains [9–11]. Multiple scales are also present in space, from the processing of information at synaptic levels to the level of large neural assemblies giving rise to different brain rhythms [12]. Moreover, neurophysiology shows how temporal and spatial dimensions are intertwined: neural oscillations at large time-scales (i.e. low frequencies) tend to cover larger scales in space, whereas shorter time-scales (i.e. high frequencies) appear to be more localized [13]. Thus, both brain and behavior are meshed together across multiple scales of time and space.

Since the present scientific approach aims to combine experimental studies with theoretical models, the key challenge is to connect these observations across scales and levels of organization within a coherent theoretical framework [14]. Coordination dynamics aims at such understanding through the synergetic concepts of self-organization [15] and the mathematical tools of dynamical systems theory [3, 16, 17]. It seeks both general principles and functionally-specific mechanisms of coordination [2] and aims at connecting multiple scales by emphasizing reciprocal coupling between levels, upward and downward [1]. In this perspective, coordination between humans represents an operational playground for experimental investigation at the crossroad of the neural, the behavioral and the social.

Recently, hyperscanning techniques have offered access to the simultaneous recording of brain activity from interacting people and thus to the study of brain and behavior coordination at both intra- and inter-individual scales [11, 18–22,19,20,21,]. In doing so, this technique has also reintroduced real social interaction into laboratory studies of human behavior, a key feature that was oddly lacking from earlier work within a (cognitively-inspired) social neuroscience , as it resorted to exposing one subject to social “stimuli” rather than examining interactions [23–26]. Further, the use of reciprocal paradigms and a real second-person approach of social cognition do not necessarily require the presence of two or more subjects in the experimental task [26]. Instead, one of the interacting partners can be substituted with a virtual agent whose design sustains bi-directional coupling between real and simulated partners [6, 27, 28].

2 Human-Machine Interaction as a Research Tool

Meanwhile , in other areas of science and engineering, a plethora of studies was concentrating on subjective perception of artificial agents by humans, with the goal of designing realistic avatars for potential applications to, e.g. video games, cinema, or eLearning assistants [29] to name just a few. In this line of research, the exercise was to mimic facets of human behavior rather than to model foundational neurobehavioral mechanisms. Interestingly, participants’ beliefs of realism were influenced by emotionally and behaviorally contingent responses made by the artificial agent [30]; see also [31]; this finding hints at the importance of reciprocal coupling with the human.

The development of realistic artificial agents extended the toolset available to social psychological research [32], with more to come as those agents are embedded in virtual realities that are increasingly indistinguishable from “normal” reality. The breakthrough of virtualization has reconciled ecological validity and experimental control, e.g. in the study of visual perception, spatial cognition and social interaction [5, 33].

A first level of social interaction is the mere presence of someone else [34]. Regarding this issue, virtual reality fits particularly well since it creates a sense of presence through mediated environments carrying dynamic animations of virtual characters [35]. Virtual characters are readily perceived as social agents and are thus capable of exerting social influence on humans [36]. Those virtual characters with strong similarity to real human interactions [37] can easily and valuably be combined with neuroimaging recording [32].

Human-machine interaction was also used to investigate motor coordination: for instance a finger tapping study by Repp and Keller [38] used a simple linear phase correction model to drive a virtual agent. It showed that subjects’ behavior was systematically modulated by the computational parameters governing that agent. Reframed in a functional neuroimaging study by Fairhurst et al. [39], the same paradigm uncovered some neural basis for motor synchronization and more importantly, for the socio-emotional consequences of different degrees of entrainment success.

In the following, we describe another paradigm, the Human Dynamic Clamp (HDC) , that embraces a continuous, multiscale and nonlinear coupling between a human and a machine. By departing from information processing approaches and design-oriented modeling, the HDC offers: (a) a new way to bridge the gap between theory, experiment and models; and (b) an integrative solution to linking neural, behavioral, and social dynamics. HDC puts well established equations of human coordination dynamics into the machine and studies real-time interactions between human and virtual partners. This opens up the possibility to explore and understand a wide variety of interactions [5, 6, 40]. Ultimately, HDC may prove useful to establishing a much friendlier union of man and machine, based on sound interactional design, and perhaps it will even lead to the creation of a different kind of machine altogether.

3 A Principle-Based Virtual Partner

The study of movement coordination is at the core of coordination dynamics and for the last 30-odd years the catchy phrase “let your fingers do the walking” has opened a rich experimental window into human behavior at both intra-individual and inter-individual levels. In a first move, it is important to clarify what we are looking at [1]. What is the behavior? What are the relevant variables and control parameters? These fundamental issues are addressed by uncovering qualitative changes in collective variables from the system called order parameters [15, 41]. Qualitative changes appear in two main flavors within the formalism of dynamical system theory : phase transitions and bifurcations . Although they are both revealed in the phenomenon of transition in collective dynamics, the first is related to the switch between potential modes of behavior simultaneously accessible to the system, and the second concerns global changes of the system’s behavioral landscape. The landscape is usually described with a manifold in phase space (the frame of reference representing the relationship between variables associated with each degree of freedom). The challenge then is to uncover the most parsimonious model that can exhibit these qualitative changes, and fit its parameters according to the experimental data (see the discussion of Phenomenological Synergetics in [42]. One key issue to keep in mind lies with the biological constraints that make it possible to link a model to actual physiological mechanisms. In this perspective, it is fundamental to recognize that all models are false by definition. However, dynamical system theory offers good candidates for a universal class of models, given the needed parsimony for elegant theory [43, 44].

Born from this aim was our recently developed Human Dynamic Clamp , a paradigm that took inspiration from the electrophysiological dynamic clamp [45, 46] to allow real-time interaction between a human subject and a computational model. Using empirically-grounded models not only validated reciprocal and fully dynamical design protocols for experimenters to use, but also provided the opportunity to explore parameter ranges and perturbations that were out of reach of traditional experimental designs with live interactions. The symmetry between the human and the machine and the fact that they carry the same laws of coordination dynamics were keys to our approach [6]. The design of the virtual partner (VP) was grounded in the equations of motion for the coordination of the human neurobehavioral system. These laws were obtained from accumulated studies over the last 30-odd years to describe how parts of the human body and brain self-organize, and to address the issue of self-reference, a condition leading to complexity.

The first version of the Human Dynamic Clamp called Virtual Partner Interaction [6] embodied the Haken–Kelso–Bunz (HKB) model [47]. The original form of HKB describes and predicts the coordination dynamics of two rhythmically moving fingers, with its characteristically complex phenomena such as multistability, phase transitions, hysteresis, critical slowing-down and fluctuation enhancement ([42, 17] for reviews). Since then, the model has also been successfully validated experimentally for the coordination between different limbs (e.g. [4]), between people (e.g. [48]) and even between species [49], within unimodal and multimodal contexts [50]. It has been supported by empirical evidence ranging from brain dynamics within [51, 52] and between brain areas [46, 53–55,54,], to coordination with external stimuli [56] and neural counterparts thereof [57, 58]. This universal characteristic supports HKB as an ideal candidate for the Human Dynamic Clamp .

In its original implementation, the VPI system was composed of a goniometer continuously digitizing the finger position of a human participant; a computational circuit simulating the HKB model; and a screen rendering the virtual partner’s behavior (see Fig. 1a–b). The computational circuit calculates the position of VP continuously according to the differential equations of HKB (Fig. 1b), and the resulting dynamics is mapped onto a virtual avatar displayed on the screen.

Fig. 1
figure 1

The VPI system. a presents it key components (goniometer to transduce human movement behavior and screen to display Virtual partner’s behavior) from the human viewpoint. Task and coupling are outlined in (b). Human’s behavior is digitized and fed into a computer whose software computes the corresponding position of the VP in real time, following a theoretical model of behavioral coordination—here HKB. The picture of the VP is updated on the screen (a) according to the output of the model. Data are stored for further study (c) to test hypotheses about the relationship between the agent’s intrinsic properties, coupling parameters and emergent collective behavior

Full size image

The HKB model at the collective level describes the equation of motion of the relative phase, a variable that distills the coordination of two oscillatory components . In this form, the HKB model reads:

$$ \dot{\phi } = {\text{a}}\sin \phi + b\,\sin 2\phi , $$
(1)

where \( \phi \) is the relative phase between human and VP’s finger position, and a and b are constants (for more details, see [59]).

However, since computers do not have direct access to the relative phase, the internal dynamics of VP is governed by the HKB model at the component level (see Fig. 1b). In this form, two non-linearly coupled non-linear oscillators represent the interaction between the two fingers. The collective form in Eq. (1) can be derived from the equations at the component level (Fig. 1b). At the component level, variables are no longer the relative phase but the individual finger positions (and velocities by derivation). x and y represent VP’s and human’s finger positions, α, β and γ are constants associated with the intrinsic dynamics of VP, ω is VP’s pulsation (frequency), A and B are constants associated with the coupling from VP to human, and finally μ is a constant fixed to either +1 or −1, indicating VP’s preference for in-phase or anti-phase coordination.

In the original study [6], VP and human behaviors were chosen to be quite simple. Both partners were tasked to coordinate finger movements with one another, the human with the intention of achieving in-phase coordination with the VP (trying to synchronize his/her flexion and extension movements with VP’s). On the VP side, the parameter μ was set to −1, inducing a VP preference for anti-phase coordination and thus a goal opposite to human’s. Subjects were instructed to maintain a smooth and continuous rhythmic movement with their right index finger (flexion-extension) and to avoid stopping their movement at any time. Visual coupling was experimentally manipulated: from unidirectional in two conditions (VP “perceives” human movement but human does not perceive VP’s behavior; or reciprocally), to bi-directional in another (both VP and human have access to each other’s finger movement). VPI accommodated the whole set of behavioral coordination modes described by the HKB model. For instance, when VP and human participants did not have the same preferred movement frequency, their relative phase conformed to predictions by the extended version of HKB [56] and exhibited phase wrapping (not shown) or metastability (see Fig. 1c). Pitting machine against human through opposing task demands is a way to enhance the formation of emergent behavior, and also allowed us to examine each partner’s individual contribution to the collective behavior. An intriguing outcome of the experiments was that subjects ascribed intentions to the machine, reporting that it was “messing” with them. A later study further suggested that VP elicits emotional experiences in human: subjects’ emotional arousal was greatest when VP interactions were (falsely) deemed to be with a human rather than with a machine [31].

In summary, Kelso et al. [6] initial VPI experiment demonstrated the feasibility of the Human Dynamic Clamp in the context of a continuous coordination of rhythmic movements. It uncovered unexpected behaviors, which were theoretically tested afterward. In the following, we show how to explore a new set of behaviors with other theoretical models of human behavior.

4 Expanding the Behavioral Repertoire

Embedding the HKB model in a Virtual Partner demonstrated that the explicit use of non-linear relational dynamics in an experimental paradigm can lead to new observations of emergent phenomena that linear models may miss out on. The Human Dynamic Clamp paradigm is about developing this idea by integrating other principle-based models grounded on canonical behaviors observed in experimental work. More complex behaviors can then be approached through the combination of canonical models in a modular and hierarchical manner [5, 60], see also Fig. 2.

Fig. 2
figure 2

Examples of interactions between a human participant (red) and a VP embedding alternative models of relational dynamics (blue). a The Excitator model (with parameters \( {\text{a}} = 0;{\text{b}} = 0;{\text{A}} = 1;{\text{B}} = - 0.2;{\tau } = 1;{ \omega } = 1 \); dashed line indicates switch from discrete to rhythmic movement in the human participant); b the adaptive Excitator model (\( {\text{a}} = 0;{\text{b}} = 0;{\text{A}} = 1;{\text{B}} = - 0.2;\uptau = 1;\upomega = 1;{\text{K}} = 1) \); c a modified HKB with an intended relative phase of pi/2 (\( {\text{a}} = 0.641;{\text{b}} = 0.00709;{\text{A}} = 0.12;{\text{B}} = 0.025;{\text{C}} = 1;\upomega = 1 \); dashed line indicates release of the VPI intentional forcing, i.e. switch to normal HKB model)

Full size image

4.1 Discrete Behavior: Phase-Space Sculpture

Although it is undeniable that living organisms rely both on rhythmic and discrete behaviors, the field of motor control has traditionally studied them separately. This led to two different ways of theoretically approaching and modeling them. While rhythmic movements have been extensively studied through the prism of dynamical systems, discrete movements’ modeling has focused on equilibrium points or control signals [61]. Unifying rhythmic and discrete movements is often posed to be a key theoretical challenge in behavioral science [62, 63]. However, there is no specific need to invoke two separate mechanisms for discrete and rhythmic behavior [64–66]. For instance, Schöner [17] extended the HKB model to the case of discrete bistable coordination by changing the intrinsic dynamics. Sternad et al. [67] proposed another model for unimanual coordination with two mutually inhibiting subsystems, each of which handled the discrete and the continuous cases respectively.

Along similar lines, Jirsa and Kelso [68] modeled discrete and rhythmic movement based on the phase flow topology of the so-called “Excitator” model (see also [63]). The Excitator defines a universal class of two-dimensional dynamical systems able to exhibit limit cycles for rhythmic movement, and fixed point dynamics for discrete movement. This model is based on topological considerations and is a parsimonious way to handle discrete and continuous behaviors simultaneously. Furthermore, in line with the approach of HKB modeling, the Excitator provides predictions regarding false-start phenomena that have been confirmed experimentally [69]. Finally, it is a biologically realistic model since it follows the self-excitable property that the FitzHugh-Nagumo model drew from single neurons [70].

The structure of the model contains three characteristics related to topological constraints: boundedness of the trajectory, existence of a separatrix marking the boundary between two separate regimes in phase space, and existence of a limit cycle for rhythmic movements and of one or two stable fixed point(s) for monostable and bi-stable discrete movements respectively.

$$ {\text{The}}\;{\text{equations}}\;{\text{read}}\;{\text{as}}\;{\text{follows}}:\left\{ {\begin{array}{*{20}l} {\mathop {x_{1} }\limits^{ \cdot } = \omega \left( {x_{1} + x_{2} - g_{1} \left( {x_{1} } \right)} \right)\tau } \hfill \\ {\mathop {x_{2} }\limits^{ \cdot } = - \omega \left( {x_{1} - a + g_{2} \left( {x_{1,} x_{2} } \right) - I} \right)/\tau ,} \hfill \\ \end{array} } \right. $$
(2)

where \( x_{1} \) and \( x_{2} \) are internal variables of the oscillator, \( \omega \) is the pulsation (frequency) of VP, \( a \) the term controlling the position of the separatrix, \( b \) the term controlling the angle of the separatrix, \( I \) an instantaneous external input, and \( \tau \) the time constant of the system.

Note that the choice of functions \( g_{1} \) and \( g_{2} \) is not fixed but must nevertheless guarantee the boundedness of the system so that the system belongs to the class of self-excitable systems . Here we take

$$ g_{1} \left( {x_{1} } \right) = \frac{1}{3}x_{1}^{3} \,\, {\text{and}}\,\,g_{2} \left( {x_{1,} x_{2} } \right) = - bx_{2} $$
(3)

When this is put in unidimensional form, we retain the same coupling terms as HKB model’s. The coupling causes either convergence or divergence of the trajectories in phase space depending on initial conditions. Since trajectories are bounded, constraints lead to in-phase or anti-phase modes of coordination (for more details, see [68]).

Implementation of the Excitator dynamics in an HDC is quite straightforward: one only needs to substitute the relevant equations from (2) in the software controlling VP’s behavior. Note that these equations introduce a new term of importance: parameter I allows to modify the phase flow according to an external input. An external input can originate from the experimenter or the human partner him/herself. It is a key component for modeling discrete behaviors , which rely on external information and is non-autonomous in a mathematical sense. The introduction of the new variable allows VP and human to coordinate diverse movements that range from simple rhythms to discrete actions. Figure 2a presents an interaction between a human and a VP governed by the Excitator model , and shows a transition from discrete movement (flexions and extensions interrupted by quiescent behavior) to continuous movement.

4.2 Adaptive Behavior: Parameter Dynamics and Modularity

The Excitator model shows how a single dynamical system may give rise to different behavioral modes of coordination between human and virtual partner. However, each mode required a different set of parameters. Once those parameters are fixed, the differential equations set the functional structure of the system for a specific behavioral context. But structure, function and dynamics are not separated in nature; everything is constantly evolving on different time scales [7, 3]. In biology, organisms change their own behavior and learn new ones to better face the world, and interact with their peers in a more effective manner. Robert Rosen even associated adaptation as the most characteristic property of living things [71]. The process of adaptation is ubiquitous in so-called complex adaptive systems that may also encompass physical or artificial aspects [72]. In the case of the brain, it is not surprising to observe such ongoing anticipation continuously [73]. Adaptation is especially important in social behavior, for instance mimicry at the morphological level [74] or interactional synchrony during cooperative imitation and skill learning [75].

Coordination may be seen as a subtle blend of reaction and adaptation to the other [76]. Whereas reaction takes place at a given time, adaptation builds up over time. For instance, humans may have a preferred movement frequency but they can adapt to different partners by slowing down or speeding up their movements. In the case of the Human Dynamic Clamp , frequency adjustment is a good candidate to address adaptive behavior in a manner that is fully compatible with the previously described systems, and uses the same formalism. Basically, frequency adaptation requires a new equation in the system of differential equations that manages the rate of change of frequency ω. At a more conceptual level, it fits with the idea that adaptation depends on the system’s ongoing intrinsic dynamics. Furthermore, adaptation can enhance the realism of the interaction, by expanding beyond an instantaneous coordination with the position of a finger or the phase of a movement.

Different strategies for modeling frequency adaptation have been proposed. In a pure Artificial Intelligence (AI) tradition, a specific module detects the frequency of the human partner which then controls VP’s actual frequency. This shows that it is possible to successfully design an artificial device that is able to do the job. In the Bayesian approach, adaptation is error-based and relies on reinforcement learning [77]. This approach is inspired from real physiological processes. In predictive coding, adaptation of model parameters is associated with Hebbian and synaptic plasticity in the brain [78]. Other bottom-up strategies have been developed in the fields of signal processing [79] and robotics [80]. Here we continue to follow the strategy of Coordination Dynamics and Dynamical System Theory . That approach was shown to better account for frequency adaptation in fireflies [81] and in tempo adaptation to musical rhythms [82]; see also [83]. In contrast with the AI approach, it is worth noting that the equations stay totally continuous and do not relate to an artificial measurement of the human frequency. This illustrates how adaptation relies on parameter dynamics according to the scale of observation [84].

Following Righetti and colleagues [85, 86], we introduce frequency adaptation through the addition of a new dimension—related to ω—in the set of differential equations:

$$\left\{ {\begin{array}{*{20}l} {\dot{x} = f_{x} \left( {x,v,\omega } \right) + KF\left( t \right)} \hfill \\ {\dot{v} = f_{v} \left( {x,v,\omega } \right)} \hfill \\ \end{array} } \right.{\text{and }}\dot{\omega } = \pm KF\left( t \right)\frac{v}{{\sqrt {x^{2} + v^{2} } }}.$$
(4)

where \( K \) is the coupling strength of the adaptation, x and v are variables capable of producing a limit cycle, and \( F\left( t \right) \) is the coupling part of the system. Figure 2b shows how a VP governed by the extended Excitator equations is able to follow changes in movement frequency. Addition of a third dimension also leads to unstable dynamics, less predictable from the human point of view. This may be associated with the emergence of chaotic regimes that are typical of 3-dimensional nonlinear dynamical systems [87]. Such unpredictability can be associated with a form of intention [88]: a model of intentional behavior could be further designed. That is what we will see in the next section.

4.3 Intentional Behavior: Symmetry Breaking and Forcing

In the case of an adaptive system, we have seen that adding a third dimension renders the dynamics less predictable. The system is nevertheless not random and appears more autonomous while still being governed by deterministic rules. This balance between autonomy and coupling creates successful agency illusion and can trigger an attribution of intention to the human observer [89, 90]. Keeping in mind that the Human Dynamic Clamp aims at operationalizing models for experimental purposes, a teleonomic system is not adequate, because its intention is not directly controllable by the experimenter.

In the initial VPI experiment [6], the control parameter μ (Eq. (1)) modulated intention attribution in some participants. In general, adopting a principle-based modeling requires redefining the boundary conditions of the model. Until now, we were dealing with spontaneous coordination. It has been shown experimentally, however, that intention affects the spontaneous potential landscape by stabilizing and destabilizing specific dynamic patterns [91] including at the brain level [92]. The former empirical findings motivated an extension of the HKB model [17]; see also [5] through the introduction of new term in the relative phase equation:

$$\dot{\phi } = {a}\,\sin \phi + b\,\sin 2\phi + c\,\sin \psi - \phi ,$$
(5)

where \( \psi \) is the intended relative phase. By incorporating an intentional forcing term c which stabilizes or destabilizes particular patterns, the model was able to explain experimental observations related to intentional switching between in-phase and anti-phase.

We recently generalized the Schöner and Kelso coupling model [5], so the intended relative phase angle \( \psi \) can take on any value between \( - \pi \) and \( + \pi \):

$$ C_{\text{int}} = - C\left( {cos\left( \psi \right)\left( {\dot{x} - \dot{y}} \right) + sin\left( \psi \right)\omega y} \right). $$
(6)

This modification of VP dynamics makes it possible to direct a collective behavior towards any desired pattern of coordination (see Fig. 2c). This offers new experimental perspectives, e.g. to study how new dynamical patterns are learned on top of a subject’s spontaneous behavioral repertoire [40].

5 Conclusion

In this chapter, we have seen how a hybrid system called the Human Dynamic Clamp allows for real-time interaction between humans and virtual partners, based on the equations of coordination dynamics built originally from HKB and its extensions. A key aspect is that the human and its virtual partner are reciprocally coupled: the human acquires information about the partner’s behavior through perception, and the virtual partner continuously detects the human’s behavior through the input of sensors. Our approach is analogous to that of the original dynamic clamp used to study the dynamics of interactions between neurons, but now scaled up to the level of behaving humans. This principle-based approach offers a new paradigm for the study of social interaction. While stable and intermittent coordination behaviors emerged that had previously been observed in ordinary human social interactions, we also discovered novel behaviors or strategies that had never been observed in human social behavior. Those novel behaviors pertained to unexplored regions of the theoretical model and were possible ways of coordination for people to interact with each other. Such emergence of novel behaviors demonstrates the scientific potential of HDC as a human-machine framework. Modifying the dynamics of the virtual partner with the purpose of inducing a desired human behavior, such as learning a new skill or as a tool for therapy and rehabilitation, is one of several applications of VPI.

HDC allows to study social interaction with more experimental control than other recent social neuroscience methods (e.g. hyperscanning) ; it is also a test bed for theoretical models. HDC moves away from simple protocols in which systems are ‘poked’ by virtue of ‘stimuli’ to address more complex, reciprocally connected systems where meaningful interactions occur. Thus, the Human Dynamic Clamp supports the development of a computational social neuroscience where theory, experiment and modeling work hand-in-hand across neural, behavioral and social scales [93].