Abstract
Recent theory proposes that the brain, when confronted with several action possibilities, prepares multiple competing movements before deciding among them. Psychophysical supporting evidence for this idea comes from the observation that when reaching towards multiple potential targets, the initial movement is directed towards the average location of the targets, consistent with multiple prepared reaches being executed simultaneously. However, reach planning involves far more than specifying movement direction; it requires the specification of a sensorimotor control policy that sets feedback gains shaping how the motor system responds to errors induced by noise or external perturbations. Here we found that, when a subject is reaching towards multiple potential targets, the feedback gain corresponds to an average of the gains specified when reaching to each target presented alone. Our findings provide evidence that the brain, when presented with multiple action options, computes multiple competing sensorimotor control policies in parallel before implementing one of them.
Similar content being viewed by others
References
Cisek, P. & Kalaska, J.F. Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of action. Neuron 45, 801–814 (2005).
Thura, D. & Cisek, P. Deliberation and commitment in the premotor and primary motor cortex during dynamic decision making. Neuron 81, 1401–1416 (2014).
Cisek, P. Cortical mechanisms of action selection: the affordance competition hypothesis. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362, 1585–1599 (2007).
Baumann, M.A., Fluet, M.C. & Scherberger, H. Context-specific grasp movement representation in the macaque anterior intraparietal area. J. Neurosci. 29, 6436–6448 (2009).
McPeek, R.M., Han, J.H. & Keller, E.L. Competition between saccade goals in the superior colliculus produces saccade curvature. J. Neurophysiol. 89, 2577–2590 (2003).
Chapman, C.S. et al. Reaching for the unknown: multiple target encoding and real-time decision-making in a rapid reach task. Cognition 116, 168–176 (2010).
Ghez, C. et al. Discrete and continuous planning of hand movements and isometric force trajectories. Exp. Brain Res. 115, 217–233 (1997).
Van der Stigchel, S., Meeter, M. & Theeuwes, J. Eye movement trajectories and what they tell us. Neurosci. Biobehav. Rev. 30, 666–679 (2006).
Stewart, B.M., Gallivan, J.P., Baugh, L.A. & Flanagan, J.R. Motor, not visual, encoding of potential reach targets. Curr. Biol. 24, R953–R954 (2014).
Stewart, B.M., Baugh, L.A., Gallivan, J.P. & Flanagan, J.R. Simultaneous encoding of the direction and orientation of potential targets during reach planning: evidence of multiple competing reach plans. J. Neurophysiol. 110, 807–816 (2013).
Shadmehr, R., Smith, M.A. & Krakauer, J.W. Error correction, sensory prediction, and adaptation in motor control. Annu. Rev. Neurosci. 33, 89–108 (2010).
Wolpert, D.M., Diedrichsen, J. & Flanagan, J.R. Principles of sensorimotor learning. Nat. Rev. Neurosci. 12, 739–751 (2011).
Scott, S.H. Optimal feedback control and the neural basis of volitional motor control. Nat. Rev. Neurosci. 5, 532–546 (2004).
Todorov, E. & Jordan, M.I. Optimal feedback control as a theory of motor coordination. Nat. Neurosci. 5, 1226–1235 (2002).
Wolpert, D.M. & Flanagan, J.R. Motor learning. Curr. Biol. 20, R467–R472 (2010).
Todorov, E. Optimality principles in sensorimotor control. Nat. Neurosci. 7, 907–915 (2004).
Cluff, T. & Scott, S.H. Rapid feedback responses correlate with reach adaptation and properties of novel upper limb loads. J. Neurosci. 33, 15903–15914 (2013).
Franklin, S., Wolpert, D.M. & Franklin, D.W. Visuomotor feedback gains upregulate during the learning of novel dynamics. J. Neurophysiol. 108, 467–478 (2012).
Diedrichsen, J. Optimal task-dependent changes of bimanual feedback control and adaptation. Curr. Biol. 17, 1675–1679 (2007).
Dimitriou, M., Franklin, D.W. & Wolpert, D.M. Task-dependent coordination of rapid bimanual motor responses. J. Neurophysiol. 107, 890–901 (2012).
Pruszynski, J.A. et al. Primary motor cortex underlies multi-joint integration for fast feedback control. Nature 478, 387–390 (2011).
Franklin, D.W. & Wolpert, D.M. Specificity of reflex adaptation for task-relevant variability. J. Neurosci. 28, 14165–14175 (2008).
Nashed, J.Y., Crevecoeur, F. & Scott, S.H. Influence of the behavioral goal and environmental obstacles on rapid feedback responses. J. Neurophysiol. 108, 999–1009 (2012).
Knill, D.C., Bondada, A. & Chhabra, M. Flexible, task-dependent use of sensory feedback to control hand movements. J. Neurosci. 31, 1219–1237 (2011).
Dimitriou, M., Wolpert, D.M. & Franklin, D.W. The temporal evolution of feedback gains rapidly update to task demands. J. Neurosci. 33, 10898–10909 (2013).
Smith, M.A., Ghazizadeh, A. & Shadmehr, R. Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 4, e179 (2006).
Haith, A.M., Huberdeau, D.M. & Krakauer, J.W. Hedging your bets: intermediate movements as optimal behavior in the context of an incomplete decision. PLOS Comput. Biol. 11, e1004171 (2015).
Gallivan, J.P. et al. One to four, and nothing more: nonconscious parallel individuation of objects during action planning. Psychol. Sci. 22, 803–811 (2011).
Hudson, T.E., Maloney, L.T. & Landy, M.S. Movement planning with probabilistic target information. J. Neurophysiol. 98, 3034–3046 (2007).
Izawa, J. & Shadmehr, R. On-line processing of uncertain information in visuomotor control. J. Neurosci. 28, 11360–11368 (2008).
Gallivan, J.P., Barton, K.S., Chapman, C.S., Wolpert, D.M. & Randall Flanagan, J. Action plan co-optimization reveals the parallel encoding of competing reach movements. Nat. Commun. 6, 7428 (2015).
Corneil, B.D., Olivier, E. & Munoz, D.P. Visual responses on neck muscles reveal selective gating that prevents express saccades. Neuron 42, 831–841 (2004).
Saijo, N., Murakami, I., Nishida, S. & Gomi, H. Large-field visual motion directly induces an involuntary rapid manual following response. J. Neurosci. 25, 4941–4951 (2005).
Gomi, H., Abekawa, N. & Nishida, S. Spatiotemporal tuning of rapid interactions between visual-motion analysis and reaching movement. J. Neurosci. 26, 5301–5308 (2006).
Prablanc, C. & Martin, O. Automatic control during hand reaching at undetected two-dimensional target displacements. J. Neurophysiol. 67, 455–469 (1992).
Day, B.L. & Lyon, I.N. Voluntary modification of automatic arm movements evoked by motion of a visual target. Exp. Brain Res. 130, 159–168 (2000).
Saunders, J.A. & Knill, D.C. Visual feedback control of hand movements. J. Neurosci. 24, 3223–3234 (2004).
Brenner, E. & Smeets, J.B. Perceptual requirements for fast manual responses. Exp. Brain Res. 153, 246–252 (2003).
Goodale, M.A., Pelisson, D. & Prablanc, C. Large adjustments in visually guided reaching do not depend on vision of the hand or perception of target displacement. Nature 320, 748–750 (1986).
Körding, K.P. & Wolpert, D.M. Bayesian integration in sensorimotor learning. Nature 427, 244–247 (2004).
Braun, D.A., Aertsen, A., Wolpert, D.M. & Mehring, C. Learning optimal adaptation strategies in unpredictable motor tasks. J. Neurosci. 29, 6472–6478 (2009).
Orbán, G. & Wolpert, D.M. Representations of uncertainty in sensorimotor control. Curr. Opin. Neurobiol. 21, 629–635 (2011).
Franklin, D.W., Osu, R., Burdet, E., Kawato, M. & Milner, T.E. Adaptation to stable and unstable dynamics achieved by combined impedance control and inverse dynamics model. J. Neurophysiol. 90, 3270–3282 (2003).
Crevecoeur, F., McIntyre, J., Thonnard, J.L. & Lefèvre, P. Movement stability under uncertain internal models of dynamics. J. Neurophysiol. 104, 1301–1313 (2010).
Munoz, D.P. & Wurtz, R.H. Saccade-related activity in monkey superior colliculus. II. Spread of activity during saccades. J. Neurophysiol. 73, 2334–2348 (1995).
Song, J.H. & Nakayama, K. Hidden cognitive states revealed in choice reaching tasks. Trends Cogn. Sci. 13, 360–366 (2009).
Gallivan, J.P. & Chapman, C.S. Three-dimensional reach trajectories as a probe of real-time decision-making between multiple competing targets. Front. Neurosci. 8, 215 (2014).
Churchland, M.M., Santhanam, G. & Shenoy, K.V. Preparatory activity in premotor and motor cortex reflects the speed of the upcoming reach. J. Neurophysiol. 96, 3130–3146 (2006).
Manohar, S.G. et al. Reward Pays the Cost of Noise Reduction in Motor and Cognitive Control. Curr. Biol. 25, 1707–1716 (2015).
Christopoulos, V. & Schrater, P.R. Dynamic Integration of Value Information into a Common Probability Currency as a Theory for Flexible Decision Making. PLOS Comput. Biol. 11, e1004402 (2015).
Howard, I.S., Ingram, J.N. & Wolpert, D.M. A modular planar robotic manipulandum with end-point torque control. J. Neurosci. Methods 181, 199–211 (2009).
Liu, D. & Todorov, E. Evidence for the flexible sensorimotor strategies predicted by optimal feedback control. J. Neurosci. 27, 9354–9368 (2007).
Ingram, J.N., Flanagan, J.R. & Wolpert, D.M. Context-dependent decay of motor memories during skill acquisition. Curr. Biol. 23, 1107–1112 (2013).
Huang, V.S., Haith, A., Mazzoni, P. & Krakauer, J.W. Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. Neuron 70, 787–801 (2011).
Galea, J.M., Mallia, E., Rothwell, J. & Diedrichsen, J. The dissociable effects of punishment and reward on motor learning. Nat. Neurosci. 18, 597–602 (2015).
Baugh, L.A., Kao, M., Johansson, R.S. & Flanagan, J.R. Material evidence: interaction of well-learned priors and sensorimotor memory when lifting objects. J. Neurophysiol. 108, 1262–1269 (2012).
Ingram, J.N., Howard, I.S., Flanagan, J.R. & Wolpert, D.M. Multiple grasp-specific representations of tool dynamics mediate skillful manipulation. Curr. Biol. 20, 618–623 (2010).
Acknowledgements
We thank D. Franklin for providing helpful advice and suggestions. This research was supported by the Natural Sciences and Engineering Research Council of Canada, the Wellcome Trust, the Human Frontiers Science Program, and by the Royal Society Noreen Murray Professorship in Neurobiology (to D.M.W.). J.P.G. was supported by a Canadian Institutes of Health Research fellowship award.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Rights and permissions
About this article
Cite this article
Gallivan, J., Logan, L., Wolpert, D. et al. Parallel specification of competing sensorimotor control policies for alternative action options. Nat Neurosci 19, 320–326 (2016). https://doi.org/10.1038/nn.4214
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.4214
- Springer Nature America, Inc.
This article is cited by
-
A reservoir of foraging decision variables in the mouse brain
Nature Neuroscience (2023)
-
Spatial coding for action across spatial scales
Nature Reviews Psychology (2022)
-
Sensorimotor strategy selection under time constraints in the presence of two motor targets with different values
Scientific Reports (2021)
-
Flexible planning of corrective responses for double-step reduction in the number of potential targets
Scientific Reports (2021)
-
Influence of a light touch reference on cutaneous reflexes from the hand during standing
Experimental Brain Research (2021)