Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Thinking is easy, acting is difficult, and to put one’s thoughts into action is the most difficult thing in the world. Johann Wolfgang von Goethe

It has now been more than 140 years since Eduard Hitzig and Gustav Fritsch performed their seminal experiments on the effect of electrical stimulation to the cerebrum (Fritsch and Hitzig 1870, 2009), yet we are still puzzled by what function the ‘motor strips’ identified in their experiments yield. For a long time the prevailing idea was that motor cortex and the adjacent premotor cortices are solely concerned with the control of muscles of the body. Little recognition had thus been given to whether the function of these regions may extend beyond connecting the brain to the lower motor neurons via the spinal cord, and to signal which particular muscles to contract. Part of this view originates from classical views of brain organisation that have emphasised that motor control is part of a serially organised system in which sensory information is transformed into neural signals for motor planning and execution through different anatomically distinct stages (Flash and Hogan 1985; Kawato et al. 1990; Bhushan and Shadmehr 1999). This transformation process ultimately culminates in M1, where descending motor commands are generated.

Perception, cognitive processing (e.g. learning, attention and working memory) and action selection have therefore traditionally been considered (and studied) as largely independent processes. The brain, however, is unlikely to adhere to these text book divisions, and while functional specialisation is critical, activity deployment in the brain does not follow these strict boundaries (Cisek 2007b; Pesaran 2010; Ledberg et al. 2007; Bullier and Nowak 1995; Hubbard et al. 2005; Smid et al. 1991; Mesulam 1990). Studies from both human and non-human primates have now, indeed, established that the premotor and motor regions of cortex, rather than being merely concerned with generating muscle commands, are intimately involved in the processing of higher order signals for action selection (Romo et al. 2004; Cisek 2007b; Cisek and Kalaska 2010; Gold and Shadlen 2001, 2007). This body of work has thus led to challenge the view of strictly serial processing as an organisational principle for brain function, and thus, our view on the foundations of action selection. For example, decisions for actions are now assumed to involve the parallel activation of multiple options, with the commitment to a specific action when activity associated to that particular action reaches a given threshold (Ivry and Spencer 2004; Cisek 2007a, b).

Studies in non-human primates, for example, show that responses of single neurons in dorsal premotor cortex (PMd) and primary motor cortex (M1) correlate with a variety of processes, such as prior expectation, time, reward, motivation or uncertainty (Bastian et al. 1998; Roesch and Olson 2003, 2004, 2007; Weinrich et al. 1984; Wise et al. 1983, 1986; Nakamura 2006; Cisek and Kalaska 2004; Crammond and Kalaska 2000, 1994; Rubino et al. 2006; Renoult et al. 2006; Requin et al. 1988; Roux et al. 2003, 2006). These signals do not directly reflect the production of the actual descending signals required for movement. At the same time, brain regions commonly regarded as specialised in, for example, decision making, learning or attention also contain neurons responding to planning and performing movements (Carello and Krauzlis 2004; Cisek and Kalaska 2005; Coe et al. 2002; Gold and Shadlen 2001; Horwitz et al. 2004; Hoshi et al. 2000; Platt and Glimcher 1999; Romo et al. 2004; Schall 2001).

The structures of the motor system are characterised by anatomical connections with a number of areas concerned with higher level computations that could provide routes through which information is transmitted to the motor system. These include regions in parietal, prefrontal and cingulate cortex, and the basal ganglia: for example, connections exist to different parietal areas (Rizzolatti et al. 1998) involved in the representation of probabilistic information (Yang and Shadlen 2007) and reward (Platt and Glimcher 1999), but also in guiding decisions about hand choice (Oliveira et al. 2010); regions of cingulate cortex (Van Hoesen and Solodkin 1993) encoding uncertainty about reward expectation and the value of actions (Rudebeck et al. 2008); and regions of prefrontal cortex (Dum and Strick 2005; Lu et al. 1994) and subcortical basal ganglia-thalamic circuits (Alexander et al. 1990) involved in processing motivational decision variables (Schultz 2006). Within prefrontal cortex, value-based influences from ventromedial prefrontal cortex (Boorman et al. 2009) may reach the motor cortices via the anterior cingulate sulcus.

In this chapter, we address how one can usefully study the functional role and neural underpinnings of signals that might influence and bias the selection of actions in humans. How can we explain the apparent richness of signals observed in the motor system that appear to be not strictly motor related, and how these can studied in the complex and flexible behaviour only humans are capable of?

The advent of techniques for non-invasive stimulation of cortex has opened the possibility to address such issues. These techniques complement direct recordings and microstimulation approaches in animals and their impact on the functional state of M1. In this chapter, we specifically ask how one can use transcranial magnetic stimulation (TMS) to study the functional role of M1 (and PMD) for action preparation and selection, and how activity in these regions might be influenced by cognitive operations that ensure our flexible and accurate movements in an uncertain and ever-changing world. This is of relevance because it not only allows for novel insights into M1/PMd function, but also allows for inferences about the hierarchical processing for action selection.

1 Using Motor-Evoked Potentials to Read-Out the Functional State of M1/PMD During Behaviour

First, we briefly review how we can utilise cortical stimulation techniques such as TMS to investigate the role of the human motor system in cognition. When applied to M1, TMS can evoke descending cortico-spinal volleys that cause contralateral muscle movement. This movement can be quantified using surface electromyography and provides a direct measure of corticospinal excitability. TMS can therefore be used in humans to non-invasively assess the functional state of the corticospinal motor system (Box 10.1). The excellent temporal resolution of TMS allows for measuring the excitability of the corticospinal system at various time points within a task. Any task-specific change in the size of evoked electromyographic responses reflects changes in the functional state or excitability of the motor output system at the time of stimulation. This is a crucial asset of the technique because, in principle, changes in the functional state can now be read out with millisecond precision.

Moreover, it is often neglected that this technique provides a causal measure in its true meaning—the signal evoked in contralateral muscles is caused by the stimulation, and nothing else. Changes in MEP size can have different origins, including noise, variations in wakefulness and attention, or even small variations in the precise stimulation point and can have both cortical and spinal origins. Critically, however, when carefully conducted, a significant proportion of variance will originate from changes in the functional state of the motor system at the time the TMS pulse is applied.

TMS provides a unique and complimentary measure to other techniques because, for example, the effector-specific changes in corticospinal excitability (CSE) are not easily observed otherwise, e.g. with EEG or fMRI approaches. Moreover, the possibility to assess systematic changes in intracortical inhibition and excitation (Ziemann and Rothwell 2000; Di Lazzaro et al. 1999, 2004; Kujirai et al. 1993; Chen 2004) offers unprecedented non-invasive information about their specific role in action control. Finally, as discussed in more detail elsewhere in this book, double-coil approaches (Civardi et al. 2001; Mochizuki et al. 2004) can be used together with complementary neuroimaging techniques (Bestmann et al. 2008b; Ruff et al. 2009) to highlight the functional interactions among interconnected networks in the brain, and how these may relate to the preparation and selection of action.

3 Assessing the Functional State of M1 During Action Execution/Voluntary Movement

One question TMS allows for addressing is how the functional state of M1 changes during the execution phase of an action. This can be investigated by probing M1 excitability at different times immediately prior to, during and after the overt response during simple or choice RT and stop-signal tasks (Chen et al. 1998; Davey et al. 1998; Soto et al. 2010; Leocani et al. 2000; Hiraoka et al. 2010; Chen and Hallett 1999; Hasbroucq et al. 2000; Romaiguere et al. 1997). Interestingly, as discussed below, the specific changes in M1 during this period may differ from those in the preceding period of action preparation.

One consistently observed finding is that CSE increases immediately before the electromyographic burst in the agonist muscle that precedes the actual voluntary movement (Hoshiyama et al. 1996a, 1997), which often corresponds to the period starting around 100 ms after the appearance of an imperative (‘Go’) stimulus. Moreover, this effect is accompanied by a reduction in intracortical inhibition preceding the voluntary movement by around 100 ms (Reynolds and Ashby 1999). It, therefore, seems that intracortical inhibition is involved in simple and choice RT tasks, even when there is no requirement to stop the movement (Burle et al. 2004). This increase is furthermore mirrored by a (usually much smaller) CSE increase (Hoshiyama et al. 1996b; Duque and Ivry 2009; Koch et al. 2006; Leocani et al. 2000), or even significant decrease (Michelet et al. 2010; Tandonnet et al. 2010; Liepert et al. 2001) in muscles not involved in an action. This latter pattern may depend on whether a muscle is merely not involved, or an antagonistic to the selected muscle. It is important to recall, however, that intracortical facilitation (ICF) and ICI as well as different dynamic behaviours of ICF and pre-movement facilitation may change simultaneously and influence one another. Depending on their relative contribution in a specific task, and the specific time at which we measure CSE, the specific pattern of CSE changes that can be observed thus may vary significantly (see above). The consistent observation is that there are very specific and mostly antagonistic changes in CSE between the selected and unselected effectors during the execution of an action. Taken together, these results show how TMS can provide unique insight into the physiological processes in M1 during the execution of selected actions.

4 Assessing the Functional State of M1 During Action Preparation

It is generally thought that actions are prepared and selected before they are executed. For example, we know from behavioural reaction time experiments that prior information (such as visual cues) can be used to prepare actions (Rosenbaum 1980). One hallmark feature in these experiments is a shortening of reaction times for more predictable stimuli, which has been taken as evidence that the respective action has been ‘mentally’ prepared, and therefore a response can be made with greater speed once the imperative stimulus is presented.

The most successful approach to study preparation experimentally are instructed delay tasks, in which a cue stimulus provides information about the likely action, which can only be executed after a delay when a subsequent imperative stimulus has been presented. This period prior to an overt action is therefore unconfounded by descending motor commands, and thus can provide useful insights into the physiological mechanism that underpin the transformation of perceptual and cognitive signals into action. The point is that under a parallel processing account, we expect to see gradual changes in CSE in the period prior to action, when actions are prepared.

In non-human primates, neurons representing the selected action progressively increase their firing rates during a delay period inserted between presentation of a visual cue and the appearance of a visual stimulus. This increase depends, for example, on the degree of predictability of the forthcoming movement—on average, more predictable sensory information leads to a stronger gradual activity build-up in premotor and motor cortex (Roux et al. 2006; Wise et al. 1983; Tanji and Evarts 1976; Bastian et al. 1998; Cisek and Kalaska 2005; Cisek 2005; Crammond and Kalaska 2000, 1994; Nakamura 2006). This observation is seen as a physiological correlate of action preparation.

In humans, TMS has been used to assess preparatory activity changes non-invasively, using delayed-response or instructed delay tasks. A rich set of studies have now established the general finding that CSE undergoes significant changes during action preparation (Bestmann et al. 2008a; van den et al. 2007; Coxon et al. 2006; van Elswijk et al. 2007, 2008; Mars et al. 2008; Duque and Ivry 2009; Duque et al. 2010; Hasbroucq et al. 1997, 1999a, b; Sinclair and Hammond 2008, 2009; Touge et al. 1998; Mars et al. 2007). These findings demonstrate that in humans, TMS can serve as a non-invasive analogue to invasive direct recordings of delay-period activity in non-human primates. TMS allows for differentiating between different intrinsic muscles, and thus provides sufficient resolution to distinguish the physiological underpinnings of action preparation and selection for different unilateral finger movements (Bestmann et al. 2008a). Moreover, the ability to assess intracortical inhibition and facilitation non-invasively through the use of paired-pulse protocols (Kujirai et al. 1993; Di Lazzaro et al. 2004) provides an additional window into the physiological interplay between intracortical excitation and inhibition during the preparation and selection of different actions.

6 Studying Dynamic, Trial-By-Trial Changes in M1 Excitability During Action Preparation

A fundamental feature of human movement is that anticipatory knowledge of an impending action improves the speed and accuracy of the response. This implies that we learn about the predictability of sensory information, and modify activity at the level of motor output accordingly, while preparing an action. For example, although sensory information provides useful cues for guiding actions, it is also inherently uncertain, and learning about this uncertainty can enable the nervous system to prepare motor output for action prior to an event. There is a good deal of information showing that, on average, predictable sensory information guiding actions leads to a gradual activity build-up in premotor and motor cortex during preparation for action. This may also be reflected in specific excitability changes of corticospinal projections, in line with a burgeoning set of studies demonstrating quantifiable effects of visual information on the motor system, including the spinal cord. Taken together, this implies that the predictability of sensory information is learned and represented explicitly in the brain, and that its representation is directed to the level of motor output for anticipatory action preparation.

The important point here is that predictability and prior expectations can only be established through learning, and that the functional state of the motor system should, therefore, reflect such learning. This interesting dynamic, however, cannot be revealed by inspecting M1 excitability changes on average alone. Understanding how the brain uses the predictability of events to inform preparation for action requires models of how, for example, the predictability of an event is learnt and represented over time. One solution to this is the use of model-based approaches (Mars et al. 2010; Corrado and Doya 2007; O’Doherty et al. 2007). These can provide trial-by-trial predictions about the possible rules used by the motor system to harness the probability of future events for action decisions.

In a first study investigating whether predictive (in this case, information theoretic) models can explain a substantial amount of CSE changes during action preparation, we previously measured CSE during an instructed delay task, in which an instruction cue provided information about the forthcoming movement with varying degree of certainty (Bestmann et al. 2008a). Thus, participants had to respond to an imperative cue stimulus but critically, in different blocks of trials, a preceding instruction cue predicted the identity of this imperative cue only with 85, 70 or 55% validity, respectively (Fig. 10.1a). This, therefore, varied the uncertainty about the required action, given the instruction cue. The important point here is that these probabilities were unknown to the participants, and thus had to be learned over time in each block.

Fig. 10.1
figure 1

Influence of uncertainty and surprise on corticospinal excitability during action preparation. a Schematic of the task. On valid trials, a preparatory CS predicted the identity of a subsequent IS, cueing a button press with the right thumb or little finger. On invalid trials, the CS-IS mapping was invalid as the CS was followed by the alternative IS. The validity of the CS varied across blocks of 105 trials between 85:15, 70:30 and 55:45%, respectively, creating blocks with, low, medium and high uncertainty about imperative stimuli. A single TMS pulse was applied during every trial, 200 ms before IS appearance, to read out changes in M1 excitability. b Average behavioural and electrophysiological results. Reaction times (grey/black) are, on average, significantly shorter for more predictable (i.e. more validly cued) actions. Average changes in corticospinal excitability parallel this effect, with CSE being largest for prepared actions in a predictable context. c CSE (top) for validly and invalidly cued trials from all subjects, plotted against the average uncertainty (entropy; left) and trial-by-trial surprise (right). CSE was generally higher when uncertainty (entropy) was low, and trials were preceded by surprising events. Reaction times for validly and invalidly cued trials plotted against average uncertainty (entropy; left) and trial-by-trial surprise (right). Reaction times were generally faster when uncertainty (entropy) and surprise were low. These results suggest that predictions about events based on both the average uncertainty, and trial-by-trial surprise conveyed by visual events can modulate the voluntary motor system on a trial-by-trial basis. Furthermore, these results show how measurements of corticospinal excitability with TMS can provide a window to examine computational processes about how humans implement decisions in real time.

Full size image

Behavioural data showed that participants reacted, on average, faster on trials in which the instruction cue was more reliable, indicating some sort of learning that enabled more efficient action preparation (Fig. 10.1b). But this does not reveal the dynamics through which this learning may take place, and influence action preparation. We therefore quantified the uncertainty about the forthcoming movement on a trial-by-trial basis for each block basis, using a simple information theoretic model that quantified the uncertainties inherent in trial sequences. We therefore asked whether these quantities might predict subject’s responses and their preparatory state prior to these (as measured through CSE). RTs and muscle-specific CSE changes were indeed influenced by both entropy and surprise: High uncertainty (high entropy) about the upcoming imperative cue was associated with decreases in CSE during the preparatory period (Fig. 10.1c). Moreover, a surprising imperative cue on the preceding trial resulted in a similar decrease in CSE. Thus, delay-period CSE, which provides an index of the preparatory state of a subject was lower when preparatory cues resolved less uncertainty (entropy), and when surprise in the preceding trigger cues was large (Fig. 10.1c). This effect was mirrored in the behavioural reaction time data—subjects were slower to respond when average uncertainty about the forthcoming movement was high, and furthermore, when imperative stimuli were surprising given the preceding cue (Fig. 10.1c).

One important aspect is the use of Bayesian model comparison, to test whether the specific model chosen indeed provides a good explanation of the data, compared to other competing models. This model comparison showed that there was indeed more evidence supporting the specific information theoretic model (given the observed RT and the CSE data), compared to a small group of alternative models that did not, or not fully, account for the contextual uncertainty inherent in the sequence of trials. The novel point made by these data is that human motor cortex is dynamically biased according to (inferred or learned) representations of contextual probabilities inherent in imperative visual events. These representations are likely encoded in the brain upstream of M1, but dynamically influence action preparation and selection. Recent support that this may reflect a general mechanism through which representations for actions are shaped by current contextual requirements comes from work on value-based decisions for actions (Klein-Flügge and Bestmann 2012). If action selection in motor regions emerges from a competitive process that is gradually biased by evidence signals originating in other regions, then biases reflecting the evaluation of more or less valuable choice options should be traceable in the motor system, before the decision process is complete. Using TMS to read-out CSE changes during such value-based decisions, Klein-Flügge and Bestmann (2012) found that excitability for chosen versus unchosen actions indeed distinguishes the forthcoming choice, but critically so could demonstrate that this occurs before the decision process is complete. Importantly, this required a trial-by-trial quantification of the value that participants assigned to their choices. The subjective value that participants assigned to the options offered on each trial was inferred using cumulative prospect theory (cf. Tversky and Kahnemann 1992), which could then be regressed against the dynamic trial-by-trial changes in CSE. Support for the idea that the observed dynamic changes in CSE during the choice process were indeed value-driven comes from the finding that both excitability and reaction times varied as a function of the subjective value difference between chosen and unchosen actions, and that such a relationship does not occur in the absence of a decision. This provides novel evidence in humans, using non-invasive TMS as read-out of the functional state of motor cortex, that internally generated value-based decisions influence the competition between action representations in motor cortex before the decision process is complete. More generally, these results demonstrate the importance of studying the dynamic, trial-by-trial changes that guide the preparation and selection of movements in an ever-changing world, and that such dynamics can now be usefully studied with the combination of TMS and model-based approaches.

7 Frameworks for Action Selection

The studies reviewed above clearly show that the human motor system is dynamically shaped by our prior expectations about forthcoming movements. Such prior expectations can be instilled by variables that are currently relevant for action selection, such as the expected reward that can be obtained following an action, or the likelihood of an event occurring. TMS can reveal the dynamic changes in the functional state of M1 during and prior to actions, and how these relate to our prior expectations.

However, these findings themselves do not yet provide a mechanistic account that can explain the functional role of such modulations and influences, nor how such signals actually reach the motor system. It is now established that M1 exhibits responses to sensory signals in a variety of modalities including vision and somatosensation (Hatsopoulos and Suminski 2011), and thus is likely to integrate such signals for the selection and preparation of appropriate movements. Two recent theories provide architectures and mechanistic accounts on how this may actually happen—the affordance competition hypothesis (Cisek 2006, 2007a, b; Cisek and Kalaska 2010) and active inference (Friston et al. 2009, 2011b). We note that these two accounts are not competing or mutually exclusive—in fact, it is the subject of intense ongoing research to explore their commonalities. We briefly outline the core concepts of these accounts and how they help to explain the findings reviewed above, but any discussion of these accounts in the present chapter can only be brief, and the reader is referred to the original work for in-depth details.

7.1 Affordance Competition

The key point of the affordance competition hypothesis (Cisek and Kalaska 2010) is that sensory information is continuously used to specify potentially available, and competing actions, whilst other kinds of information (such as motivation, reward expectation, new sensory information) will be accumulated and provide evidence that ultimately leads to selecting one action from the available set of actions (Cisek 2006, 2007b; Cisek and Kalaska 2010). Potential actions therefore ‘compete’ with one another, and internal representations influence this competition. As initially introduced by (Gibson 1979), the concept of affordances reflects the idea that these internal representations are opportunities, or affordances, for action defined by the environment (Fig. 10.2).

Fig. 10.2
figure 2

Graphical illustration of the affordance competition hypothesis. Sensory information (here illustrated for visual information flow) continuously influences the representation of potentially available actions, which ‘compete’ with one another. Internal representations and signals such as motivation, reward expectation, new sensory information provide additional evidence that ultimately leads to selecting one action from the available set of actions. As shown here, starting with the visual cortex, information is passed to the parietal lobe, where visual information is likely transformed into representations of potential actions. These representations are captured by different, and potentially competing neural populations in parietal cortex. This competition is biased by additional input from e.g. the basal ganglia and prefrontal cortical regions that accumulate and process additional information required for action selection (here indicated by the red arrows). Once competition has led to the selection of an action over its potential action alternatives, it is unleashed. This also caused feedback from the induced changes in the environment (dotted blue arrow) and feedback caused by the predictive collateral via the cerebellum. Modified with permission from Cisek and Kalaska (2010)

Full size image

These affordances for action are based on incoming sensory information and internally represented decision variables (e.g. subjective reward, motivation, wakefulness, hunger), which are continuously transformed into parameters of action (Cisek and Kalaska 2010). This also implies that multiple actions might initially be available, but competition between these alternative options ultimately leads to the commitment to one specific action. This competition is thought to be driven by mutual inhibition among cells with different tuning properties (Cisek 2006), and/or through differential selection in corticostriatal circuits as likely physiological substrates.

Critically, the competition at the level of PMd/M1 is driven by inputs from other regions, such as parietal and prefrontal cortex and the basal ganglia that contain information or evidence about the most appropriate action, given the context. These regions therefore bias the competition among actions until some (unspecified) threshold is reached and a commitment to an action is made.

The TMS work in humans reviewed above indeed provides strong support for this idea. The observation that CSE changes relate to the prior expectation of an event, for example, suggest that our expectation of what will happen (and consequently what we will have to do) ‘biases’ the competition among available actions. In humans, this bias can be quantified using TMS. For example, a recent study by Michelet and co-workers shows that CSE during the reaction time of the Eriksen flanker task increases gradually for the agonist muscle, and decreases for the antagonistic muscle (Michelet et al. 2010). Critically, the opposite is initially observed in an incongruent condition—when information about which action to perform is misleading, the competition among two actions initially favours the erroneous action, and only later reverses as sensory information provides sufficient evidence for the correct action. This is reflected in the observed CSE changes which initially increase for the erroneous action, and then reverse gradually. These findings elegantly show that the dynamic modulation in CSE resembles the competition among alternative actions, which ultimately leads to the selection of one response and the rejection of the other.

7.2 Active Inference and Predictive Coding

Active inference is a corollary of the free-energy principle and the predictive coding account (Friston and Kiebel 2009). In short, this idea states that a self-organising system like the brain should minimise the free energy of sensory states it samples. Here, free energy itself is an upper bound on the surprise (prediction error) associated with sensory signals, such as visual cues. Simply put, free energy is the (Bayesian) evidence for the brain’s model of its world. This means that when the brain minimises free energy, it reduces surprising exchanges with the world. Equivalently, it means that it maximises the evidence for its own internal model of its sensory world (see Fig. 10.3).

Fig. 10.3
figure 3

Active inference and predictive coding. Active inference is a generalisation of predictive coding that covers motor behaviours and itself is a special instance of the principle of free energy minimisation (cf. Friston et al. 2011b). Free energy is a statistical quantity that bounds the surprise (self-information) associated with sensory signals. This surprise is quantified in relation to a generative model of how those signals were caused. Predictive coding uses prediction error as a proxy for free energy (cf. surprise) and rests on a hierarchical model, in which prediction errors are passed up the hierarchy (red arrows) to optimise high-level representations that provide top-down predictions (black arrows). In this schematic, prediction error units are portrayed in red and units encoding the conditional expectations of the hidden causes of sensory input are shown in blue. During perception, the best explanation for sensory input emerges when the top-down predictions can explain as much of the prediction error (at each hierarchical level) as possible. Active inference takes this one step further and notes that certain sensory modalities can use prediction errors to drive motoneurons to eliminate prediction error directly (through classical motor reflex arcs). This is shown schematically on the lower left, using units in the dorsal and ventral horns of the spinal cord. Under active inference, a movement just fulfils the predictions afforded by percepts that predict both exteroceptive (e.g. visual) and interoceptive (e.g. stretch receptor) consequences. This high-level (sensorimotor) percept is activated by an exteroceptive (sensory) cue, and the ensuing top-down predictions propagate to both sensory cortex (to suppress exteroceptive prediction error) and the motor system. However, in the motor system, the predictions engender a prediction error that is eliminated by movement (adapted from Brown et al. 2011)

Full size image

More specifically, predictive coding is based on the assumption that the brain makes inferences about the causes of its own sensations and percepts (see Feldman and Friston 2010; Friston 2006, 2009, 2010; Friston et al. 2010; Friston and Stephan 2007). These inferences are driven (or inhibited) by bottom-up or feed-forward sensory information (see Fig. 10.3). This information is conveyed to higher brain areas in the form of prediction errors (Rao and Ballard 1999; Friston et al. 2008). By contrast, top-down or backward connections signal the predictions the brain makes about the information that will be received at the lower level. These predictions aim at suppressing prediction errors. An ideal state would be when predictions are optimal, and prediction errors therefore are minimal, i.e. the brain would perfectly explain (predict) the world which it samples through its own sensations.

One important concept is that of top-down first-order and contextual second-order predictions. The former drive (or inhibit) neurons reporting prediction errors, whereas the latter reflect the precision (or reliability) of these predictions errors. Put simply, the precision can be regarded as representing the reliability, ambiguity, or uncertainty about sensory signals, such as visual cues. As seen previously, there is a rich body of work that shows that the motor system is highly sensitive to such second-order effects, e.g. changes in the reliability of visual cues instructing movement (Bestmann et al. 2008a; Brown et al. 2011; Mars et al. 2007). Top-down predictions can therefore have a direct (first-order) or a modulatory (second-order) effect on the responses of prediction error units that make the ensuing predictions as efficient as possible.

Active inference (Brown et al. 2011; Friston et al. 2011b) extends this architecture, suggesting that exactly the same recursive message passing also applies to the motor system. The only difference here is that prediction errors at the lowest level (i.e. the cranial nerve nuclei and spinal cord) are suppressed by movement, through classical reflex arcs. In this view, descending (corticospinal) signals are not motor commands in the traditional sense per se, but predictions of the proprioceptive signals that arise from movement. The peripheral motor system, through movement, therefore tries to fulfil its predictions about proprioceptive signals (see Friston (2009, 2010) for an in-depth treatment). In this view, a sensory cued movement is generated by a high-level (sensorimotor) representation that predicts a particular pattern of proprioceptive and exteroceptive sensory signals. This representation arises to explain prediction errors caused by, e.g., a visual cue, while motor reflexes suppress the ensuing prediction errors in the proprioceptive domain. This framework has been used to explain several features of the motor system and a series of behaviours, from visual tracking (Friston 2009; Friston and Kiebel 2009), motor preparation (Brown et al. 2011), to action observation (Friston et al. 2010). An obvious appeal of this idea is that the same architecture and principle about hierarchical message passing and integration can now be applied to both sensory and motor systems. This has intuitive appeal because it assumes that the brain does not use different approaches for dealing with similar problems.

In other words, this framework provides a unifying account for the organizational principles underpinning sensory perception and action: if ascending sensory signals are prediction errors and descending motor commands are predictions, then the optimisation of predictions (and the resulting movements) should depend on optimising precision (i.e. reliability) in exactly the same way as in sensory processing. Initial modelling work and behavioural experiments (Brown et al. 2011) support this view. These suggest that motor preparation (and selection) is ultimately directed towards proprioceptive sensations, i.e., the predicted sensory feedback that will be elicited from the anticipated motor response (Brown et al. 2011).

Importantly, both concepts can be brought together when considering that high-level sensorimotor representations are often dynamic in nature. Time variant neural dynamics represent prior beliefs or expectations about, for example, the sequence of sensorimotor events or trajectories that will arise in the near future (Friston et al. 2011a). One way of viewing these is as attractors that provide proprioceptive and sensory predictions for sensorimotor integration. These, in other words, are the representations of affordance. The selection of an action relies on accurate bottom-up prediction errors conveying salient sensory information that has yet to be explained. Or, phrased differently, the brain aims to select those representations with an affordance that best explains sensory input, which is equivalent to affordance competition. Put simply, bottom-up prediction errors bias competition amongst high(er) level sensorimotor representations (attractors).

The key point here is that both accounts introduced here predict that our motor system will be influenced by the predictions our brain makes about forthcoming movements, and that they make specific statements on how such influences originate. For example, the affordance competition model provides testable predictions about the likely routes through which specific types of information will be conveyed to the motor system; this depends on the functional specialisation of regions in parietal or frontal cortex, and the basal ganglia. Recent double-coil TMS studies have indeed addressed how, for example, premotor and parietal regions influence the functional state of M1 during different types of movement tasks, and at rest (Koch et al. 2006, 2007, 2008). Active inference, and the hierarchical predictive coding account it is resting on, makes specific predictions about the type of connections that convey the information that allows for an action to be chosen and executed. Bottom-up sensory information (prediction errors) that has yet to be explained by top-down predictions is generally associated with the activity of superficial pyramidal cells (Mumford 1992; Friston et al. 2010; Brown et al. 2011). With regards to action selection, descending (cortico-spinal) signals are not motor commands in the traditional sense per se, but predictions of the proprioceptive signals that arise from movement. In the future, it will therefore be of interest to record from the different descending and ascending pathways, and to determine whether information in these may indeed reflect movement-related predictions and prediction errors, respectively. Both accounts vary in their specific aims, complexity and architecture. However, they provide frameworks in which to address how information influences our motor system to ensure that our actions remain flexible and accurate in an uncertain and ever-changing world.

8 Summary

Transcranial magnetic stimulation (TMS) provides a window to examine computational processes that the brain may use to implement actions in real time, and their influence on output stage. In humans, TMS can thus be used to read-out the functional state of motor system during action preparation and selection, and thus provide insights into their physiological underpinnings in an unprecedented way.