Abstract
The behavioral strategies that mammals use to learn multi-step routes are unknown. In this study, we investigated how mice navigate to shelter in response to threats when the direct path is blocked. Initially, they fled toward the shelter and negotiated obstacles using sensory cues. Within 20 min, they spontaneously adopted a subgoal strategy, initiating escapes by running directly to the obstacle’s edge. Mice continued to escape in this manner even after the obstacle had been removed, indicating use of spatial memory. However, standard models of spatial learning—habitual movement repetition and internal map building—did not explain how subgoal memories formed. Instead, mice used a hybrid approach: memorizing salient locations encountered during spontaneous ‘practice runs’ to the shelter. This strategy was also used during a geometrically identical food-seeking task. These results suggest that subgoal memorization is a fundamental strategy by which rodents learn efficient multi-step routes in new environments.
Similar content being viewed by others
Data availability
Data supporting the findings of this study (mouse tracking data and stimulus information) are available from https://figshare.com/articles/dataset/Subgoal_paper_data_zip/14610135. Any additional data are available from the corresponding author upon reasonable request.
Code availability
The custom code used to perform analysis for this study is available at https://github.com/BrancoLab/escape-analysis; code for registering videos to a common coordinate framework is available at https://github.com/BrancoLab/Common-Coordinate-Behaviour; and code for creating the visualizations found in the figures and supplementary videos is available at https://github.com/BrancoLab/escape-visualization.
References
Lima, S. L. & Dill, L. M. Behavioral decisions made under the risk of predation: a review and prospectus. Can. J. Zool. 68, 619–640 (1990).
Drickamer, L. C. & Stuart, J. Peromyscus: snow tracking and possible cues used for navigation. Am. Midl. Nat. 111, 202–204 (1984).
McMillan, B. R. & Kaufman, D. W. Travel path characteristics for free-living white-footed mice (Peromyscus leucopus). Can. J. Zool. 73, 1474–1478 (1995).
Benhamou, S. An analysis of movements of the wood mouse Apodemus sylvaticus in its home range. Behav. Processes 22, 235–250 (1991).
Thompson, S. D. Spatial utilization and foraging behavior of the desert woodrat, Neotoma lepida lepida. J. Mammal. 63, 570–581 (1982).
Morris, R. G. M. Spatial localization does not require the presence of local cues. Learn. Motiv. 12, 239–260 (1981).
Edvardsen, V., Bicanski, A. & Burgess, N. Navigating with grid and place cells in cluttered environments. Hippocampus 30, 220–232 (2020).
Spiers, H. J. & Gilbert, S. J. Solving the detour problem in navigation: a model of prefrontal and hippocampal interactions. Front. Hum. Neurosci. 9, 125 (2015).
Stachenfeld, K. L., Botvinick, M. M. & Gershman, S. J. The hippocampus as a predictive map. Nat. Neurosci. 20, 1643–1653 (2017).
Tolman, E. C. Cognitive maps in rats and men. Psychol. Rev. 55, 189–208 (1948).
O’Keefe, J. & Nadel, L. The Hippocampus as a Cognitive Map (Clarendon Press, 1978).
Etienne, A. S. & Jeffery, K. J. Path integration in mammals. Hippocampus 14, 180–192 (2004).
Restle, F. Discrimination of cues in mazes: a resolution of the ‘place-vs.-response’ question. Psychol. Rev. 64, 217–228 (1957).
Sutherland, R. J. & Dyck, R. H. Place navigation by rats in a swimming pool. Can. J. Psychol. 38, 322–347 (1984).
Hamilton, D. A., Rosenfelt, C. S. & Whishaw, I. Q. Sequential control of navigation by locale and taxon cues in the Morris water task. Behav. Brain Res. 154, 385–397 (2004).
Cooper, Jr., W. E. & Blumstein, D. T., eds. Escaping from Predators: An Integrative View of Escape Decisions (Cambridge Univ. Press, 2015).
Vale, R., Evans, D. A. & Branco, T. Rapid spatial learning controls instinctive defensive behavior in mice. Curr. Biol. 27, 1342–1349 (2017).
Yilmaz, M. & Meister, M. Rapid innate defensive responses of mice to looming visual stimuli. Curr. Biol. 23, 2011–2015 (2013).
Alyan, S. & Jander, R. Short-range homing in the house mouse, Mus musculus: stages in the learning of directions. Anim. Behav. 48, 285–298 (1994).
Etienne, A. S., Teroni, E., Maurer, R., Portenier, V. & Saucy, F. Short-distance homing in a small mammal: the role of exteroceptive cues and path integration. Experientia 41, 122–125 (1985).
Harrison, F. E., Reiserer, R. S., Tomarken, A. J. & McDonald, M. P. Spatial and nonspatial escape strategies in the Barnes maze. Learn. Mem. 13, 809–819 (2006).
Ellard, C. G. & Eller, M. C. Spatial cognition in the gerbil: computing optimal escape routes from visual threats. Anim. Cogn. 12, 333–345 (2009).
Collett, T. S. Do toads plan routes? A study of the detour behaviour of Bufo viridis. J. Comp. Physiol. 146, 261–271 (1982).
Layne, J. E. Mechanisms of homing in the fiddler crab Uca rapax 1. Spatial and temporal characteristics of a system of small-scale navigation. J. Exp. Biol. 206, 4413–4423 (2003).
McCreery, H. F., Dix, Z. A., Breed, M. D. & Nagpal, R. Collective strategy for obstacle navigation during cooperative transport by ants. J. Exp. Biol. 219, 3366–3375 (2016).
Liu, A. et al. Mouse navigation strategies for odor source localization. Front. Neurosci. 14, 218 (2020).
Wallace, D. G., Gorny, B. & Whishaw, I. Q. Rats can track odors, other rats, and themselves: implications for the study of spatial behavior. Behav. Brain Res. 131, 185–192 (2002).
Burgess, N., Recce, M. & O’Keefe, J. A model of hippocampal function. Neural Netw. 7, 1065–1081 (1994).
Schölkopf, B. & Mallot, H. A. View-based cognitive mapping and path planning. Adapt. Behav. 3, 311–348 (1995).
Collett, T. S. Making learning easy: the acquisition of visual information during the orientation flights of social wasps. J. Comp. Physiol. A 177, 737–747 (1995).
Mobbs, D., Headley, D. B., Ding, W. & Dayan, P. Space, time, and fear: survival computations along defensive circuits. Trends Cogn. Sci. 24, 228–241 (2020).
Sutton, R. S., Precup, D. & Singh, S. Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning. Artificial Intell. 112, 181–211 (1999).
Jadhav, S. P., Kemere, C., German, P. W. & Frank, L. M. Awake hippocampal sharp-wave ripples support spatial memory. Science 336, 1454–1458 (2012).
O’Keefe, J. & Recce, M. L. Phase relationship between hippocampal place units and the EEG theta rhythm. Hippocampus 3, 317–330 (1993).
Wilson, M. & McNaughton, B. Reactivation of hippocampal ensemble memories during sleep. Science 265, 676–679 (1994).
Doeller, C. F., King, J. A. & Burgess, N. Parallel striatal and hippocampal systems for landmarks and boundaries in spatial memory. Proc. Natl Acad. Sci. USA 105, 5915–5920 (2008).
Packard, M., Hirsh, R. & White, N. Differential effects of fornix and caudate nucleus lesions on two radial maze tasks: evidence for multiple memory systems. J. Neurosci. 9, 1465–1472 (1989).
Datta, S. R., Anderson, D. J., Branson, K., Perona, P. & Leifer, A. Computational neuroethology: a call to action. Neuron 104, 11–24 (2019).
Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A. & Poeppel, D. Neuroscience needs behavior: correcting a reductionist bias. Neuron 93, 480–490 (2017).
Mobbs, D., Trimmer, P. C., Blumstein, D. T. & Dayan, P. Foraging for foundations in decision neuroscience: insights from ethology. Nat. Rev. Neurosci. 19, 419–427 (2018).
Bradski, G. The OpenCV Library. Dr. Dobb’s Journal of Software Tools 120, 122–125 (2000).
Mathis, A. et al. DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nat. Neurosci. 21, 1281–1289 (2018).
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V. & Thirion, B. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
Acknowledgements
This work was funded by a Wellcome Senior Research Fellowship (214352/Z/18/Z) and by the Sainsbury Wellcome Centre Core Grant from the Gatsby Charitable Foundation and Wellcome (090843/F/09/Z) (T.B.) and the Sainsbury Wellcome Centre PhD Programme (P.S. and S.O.). We thank members of the Branco lab and T. Mrsic-Flogel for discussions; J. Rapela for advice on statistical analysis; T. Mrsic-Flogel, T. Behrens, C. Barry, M. Stephenson-Jones, Y. Isogai, C. Clopath, Y. L. Tan, F. Claudi, R. Vale and three anonymous reviewers for comments on the manuscript; the Sainsbury Wellcome Centre Neurobiological Research Facility and FabLabs for technical support; and K. Betsios for programming the data acquisition software.
Author information
Authors and Affiliations
Contributions
P.S. and T.B. conceived the project, designed the experiments and wrote the manuscript. P.S. performed the escape experiments with help from P.I. P.S. analyzed the data. D.C. and N.B. designed the food reward training protocol. S.F.O. and P.S. performed the food reward experiments.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Peer review information Nature Neuroscience thanks Kiah Hardcastle, Mackenzie Mathis, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data Fig. 1 Escapes in the presence of an obstacle.
a, Speed profile for all escape trials (3 trials per mouse). Mice were stimulated with sound (dotted white line) while in the threat zone of the open field (N = 10 mice; left) or the platform with the obstacle (N = 24 mice; right). In the platform with the wall obstacle, the probability of threat-evoked escape to shelter was 93%, while the probability of escape in a no-stimulus control was 12%. Trials are sorted by shelter arrival time. b, Exploration trajectories around the obstacle area (each color represents the movements of one mouse prior to trial 1). 3 randomly selected sessions are displayed. All mice approached and explored the area within 5 cm of the obstacle prior to the first escape trial (median time in this region: 37 sec, IQR: 29 - 45 seconds, minimum 14 seconds; median total distance explored in this region: 417 cm, IQR 259 - 493 cm, minimum 127 cm). c, Example trial and escape trajectories for experiment with an unprotective hole obstacle instead of the wall obstacle (N = 23 escapes, 8 mice). Grey indicates homing-vector paths, and blue indicates obstacle-edge vector paths. The black rectangle represents the hole obstacle. d, Summary of escape trajectories with the wall and hole obstacles. e, Escape duration measured as the duration from escape initiation until reaching the shelter. To compare across conditions and trials, this value is normalized by the shortest possible path length to the shelter from the mouse’s starting point. Each dot represents one escape. f, Running speed during escape is the average speed of the mouse from escape initiation until reaching the shelter. There is a small effect of trial on running speed. F(2, 37)=4.4, P = .02, two-sided repeated measures ANOVA on trials 1-3. Open field total: N = 23 escapes, 10 mice; obstacle trial 1: N = 21 escapes, 21 mice; obstacle trial 2: N = 24 escapes, 24 mice; obstacle trial 3: N = 21 escapes, 21 mice.
Extended Data Fig. 2 Platforms with the wall obstacle and hole obstacle.
a, Picture of the platform with the wall obstacle. The platform is 92 cm in diameter, and the wall obstacle is 50 cm long x 12.5 cm tall. The shelter is 10 cm long x 10 cm wide x 12.5 cm tall. It is made from red acrylic that is opaque to the mouse but transparent to red and infrared light. b, Picture of the platform with the hole obstacle. The platform is 92 cm in diameter and has a 50 cm long x 10 cm wide x 1 m deep rectangular hole in the middle. Both platforms are raised from the floor at a height of 1 m.
Extended Data Fig. 3 The role of visual input in efficient obstacle avoidance.
a, Example trial and all escape trajectories when the obstacle arises simultaneous with stimulus onset. This trial occurs after 20 minutes with three baseline escape trials in the open field. Putative visual avoidance occurs when the mouse turns toward an obstacle edge in the region between 5-10 cm away from the obstacle. Putative tactile obstacle avoidance occurs when the mouse turns toward an obstacle edge only after its head is already within 5 cm of the obstacle. N = 10 escapes, 10 mice b, Escape trajectories for experiments in which naïve mice escape to the shelter in complete darkness (the four experiments in panels B and C are the only experiments in this paper performed in the dark). Dot-and-arrow plots display the distribution of escape targets in each condition. Open field: N = 41 escapes, 14 mice; obstacle: N = 33 escapes, 14 mice; obstacle + exploration in light: N = 33 escapes, 14 mice. c, Mice with 20 minutes of experience in the light, including three escape trials. Even after removing all light, mice execute edge-vector responses. N = 32 escapes, 14 mice. d, Summary of escape trajectories in complete darkness. One-sided permutation test on edge vectors in the dark, obstacle vs. open field; after 20 mins in the light: P = 0.002 (**); after 10 mins in the light: P = 0.2; exploration in dark: P = 0.2.
Extended Data Fig. 4 Obstacle manipulation experiments.
a, Chronic obstacle removal experiments (combined data from the experiment with three baseline escape trials and the experiment with zero). After the obstacle had been removed and prior to the escape trials, 100% of mice had previously visited the area where the obstacle used to be (black bar). Each colored trace represents the movements of one mouse after the obstacle was removed and prior to an escape trial. Six mice were randomly selected for visualization. b, Escape trajectories in a control condition for the obstacle length-change experiment. In this condition, the obstacle is always short. N = 15 escapes, 9 mice. Grey indicates trajectories targeting the shorter obstacle edge location, and blue indicates trajectories targeting the longer obstacle edge location. Arrow plots: dark green indicates overshooting the current edge or shelter position, and light green indicates undershooting (see Fig. 2c). c, Escape trajectories in a control condition for the obstacle length-change experiment. In this condition, the obstacle is always long. N = 10 escapes, 8 mice. d, Escape trajectories in an obstacle length-change experiment. The obstacle starts out long (dotted line) and is shortened after 20 minutes and 3 escape trials. N = 14 escapes, 9 mice. e, Escape trajectories in an obstacle length-change experiment. The obstacle starts out short (dotted line) and is lengthened after 20 minutes and 3 escape trials. N = 13 escapes, 9 mice.
Extended Data Fig. 5 Homing runs, turn angles, and heading directions.
a, Left: histogram of homing runs’ initial condition. This shows, for an average escape in the CORE-ZB, how many prior homing runs fell into different proximity bins. Each bin reflects proximity in both the position (x-axis) and body orientation (y-axis) of the homing’s starting point. Right: example of homing runs extracted from exploration during the CORE-ZB’s exploration period, and example of a subsequent escape in that experiment. b, Experiment with narrow corridors that constrain movements during exploration and escape. Histogram and homing runs are computed in the same manner as in panel a. c, Correlation between the predicted escape target (using the procedure illustrated in Fig. 3b) and the actual escape target. Here, escape targets are predicted using the homing run with the most similar turn angle to the escape turn angle. Data are from homings and escapes in the platform with narrow corridor. The correlation coefficient r = 0.98; P = 2×10-22. This prediction thus yields an R2 value of 0.97, which here corresponds to a mean absolute error of 0.025 in escape-target units. Note that we are using R2 values as a measure of explained variance and not as a measure of goodness-of-fit of the linear relationship. d, Same analysis as in panel c, but with escapes and homings from the CORE-ZB. The correlation coefficient r = -0.03; P = 0.9. This prediction thus yields an R2 value of 0.0009, which corresponds to a mean absolute error of 0.38 in escape-target units. For comparison, predicting that every escape will be equal to the CORE-ZB’s mean escape target generates a mean absolute error of 0.31. e, Correlation between the allocentric heading direction required to target the obstacle edge location (x-axis) and the allocentric heading direction during escape (y-axis). The y-axis heading direction is measured when the mouse is 15 cm away from the escape initiation point (that is, after the initial turn movement is complete). The vector from the center of the platform to the shelter (pointing south) is set as 0˚, and a vector pointing west or east is ±90˚. The absolute value of the heading direction is taken so that escapes toward the left and right edges can be considered together. Homing-vector escapes are not included. Data are from the two COREs of Fig. 2a,b. Lines show the linear regression fit, and the shaded area shows the prediction interval within 1 standard deviation.
Extended Data Fig. 6 Illustration of escape initiation points in the main experiments.
a, Escapes in the open field. The escape initiation points mark the beginning of a turn-and-run movement from inside the threat zone (gray area) to outside the threat zone. This is computed the same way for escapes (shown here) and for spontaneous homing runs (for example Fig. 3, Extended Data Fig. 5). Top left: red dots show the mouse’s position when the stimulus comes on for all trials, and blue dots show the mouse’s position at the escape initiation point. Bottom left: single escape trials are color-coded by trajectory type (homing-vector paths are black/gray, edge-vector paths are blue). Movements between the stimulus onset and the escape movement onset are shown in dark, bold traces. The rest of the escape is shown in a lighter hue. Right: example escape. The red mouse silhouettes mark the path between stimulus onset and escape initiation. The bright blue silhouette marks the escape initiation point, which is where the analysis of escape paths and turn angles begins. b, Escapes with an obstacle (trial 1). c, Escapes with an obstacle (trial 2). d, Escapes with an obstacle (trial 3). e, Escapes after chronic obstacle removal (zero baseline escape trials). f, Escapes after chronic obstacle removal (three baseline escape trials). g, Food-seeking trials in the open field. (H) Food-seeking trials after chronic obstacle removal.
Extended Data Fig. 7 Mice memorize previously targeted subgoal locations.
a, Correlation between escape targets in the CORE-ZB and the amount of exploration in different sections of the platform. Red outlines indicate the section of the platform in which the distance explored is measured. For exploration near the obstacle edge, only the edge that was targeted during the escape (that is, left vs. right) is considered. Boxes show the correlation coefficients and respective p-values; significant correlations have green outlines. b, Spatial efficiency of escapes on the first trial in the presence of an obstacle (same data as in Fig. 1). Here, runs from the threat area to the 10 cm in the center of the obstacle are considered. Zero homing vectors: N = 9 escapes, 9 mice; ≥1 homing vector: N = 12 escapes, 12 mice. White squares show the median, thick lines show the IQR, and thin lines show the range excluding outliers. Two-sided permutation test on escape efficiency. c, Same as panel b, but here, runs from the threat area to the obstacle edge that was not used during the escape are considered. Zero edge vectors: N = 8 escapes; ≥1 edge vector: N = 13 escapes. d, Same as panel b and c, but for runs from the shelter area to the obstacle edge used during the escape. Zero edge vectors: N = 2 escapes; ≥1 edge vector: N = 19 escapes. e, Escapes from an experiment acutely removing the obstacle on the first trial, after 10 minutes of exploration. N = 10 escapes, 10 mice. f, Correlation of different running movements to escape target score, in the trial-1 acute removal experiment. These include homing runs from the threat area to different parts of the obstacle, as well as runs from the shelter area to the obstacle edge. Runs toward the same edge targeted in the escape (here, the right edge) are considered separately from runs toward the opposite edge (here, the left). g, Correlation between escape targets in the trial-1 acute removal experiment and the amount of exploration in different sections of the platform. Post-removal exploration is not applicable in this experiment, since the obstacle is removed just before the escape begins.
Extended Data Fig. 8 Edge-directed movements in different environments.
a, Frequency of spontaneous movements toward the obstacle edges (all sessions with an obstacle). b, Frequency of edge-directed movements for different conditions. Two-sided permutation test on number of edge-directed movements; wall obstacle vs. open field: P = 0.0005. Open field (no shelter): N = 6 mice; hole obstacle (no shelter): N = 7 mice; wall obstacle (no shelter): N = 16 mice. For movements directed toward the center of the platform, there are no significant differences across conditions. Each dot is one session. White squares show the median, thick lines show the IQR, and thin lines show the range excluding outliers. c, Escape trajectories for a hole obstacle (N = 53 escapes, 8 mice). The black rectangle represents the hole obstacle. d, Evolution of escape targets for increasing trial numbers with the hole obstacle and the wall obstacle.
Extended Data Fig. 9 Edge vectors persist over many trials and minutes after obstacle removal.
a, Escape targets vs. trial number in the chronic obstacle removal experiments (three and zero baseline escape trials combined, and now including the minority of mice that performed >3 trials). For this plot, only successful escape trials are counted toward the trial number. The correlation between escape target and the number of stimulus-evoked escapes following obstacle removal is not significant: correlation coefficient r = -0.07, P = 0.6. b, Escape targets vs. time. The correlation between escape target and amount of time since obstacle removal is not significant: correlation coefficient r = -0.12, P = 0.3. See Extended Data Fig. 6 for correlations between escape targets and the amount of post-removal exploration in various parts of the platform. Lines show the linear regression fit, and the shaded area shows the prediction interval within 1 standard deviation.
Extended Data Fig. 10 Training mice to approach and lick a spout in response to a tone.
a, Lick raster plots for an example mouse during the first (top) and the last training day (bottom). During food-approach training, a 9-second, 10-kHz tone is associated with the availability of condensed milk at a metal spout. For the lick raster plots, licks were plotted at 5 licks/sec when the sensor was tonically triggered by licking; this does not affect the quantifications in panels b-c. b, Summary data for lick probability during training. Relative lick probability is the average probability of licking the spout within a 4.5-second window during the stimulus, divided by the lick probability during the 20 seconds before or after the stimulus. Mice lick the spout specifically in response to the tone on the fifth day of training (relative lick probability > 1, P = 0.002, one-sided permutation test) but not on the first day (P = 0.09). c, Summary data for reward-port approach probability during training. Relative approach probability is the average probability of moving from the back of the conditioning box to the side where the spout is located in response to the tone, divided by the approach probability at other random time points during the session. Mice approach reward specifically in response to the tone on the fifth day of training (relative approach probability > 1, P = 0.02, one-sided permutation test) but not on the first day (P = 0.41). For panels b and c, gray lines are individual mice and the green line is the mean. N = 5 mice.
Supplementary information
Supplementary Information
Supplementary Figs. 1 and 2.
Supplementary Audio 1
Threat stimulus 1—crashing sound
Supplementary Audio 2
Threat stimulus 2—crackling sound
Supplementary Video 1
Visualizing escapes—raw video of multiple escapes alongside our custom visualization
Supplementary Video 2
Escapes in the open field
Supplementary Video 3
Escapes with an obstacle
Supplementary Video 4
Escapes with an obstacle that rises up at the stimulus onset
Supplementary Video 5
Escapes with an obstacle in the dark—naive versus experienced mice
Supplementary Video 6
Escapes with an obstacle that disappears at stimulus onset and escapes several minutes after the obstacle had disappeared
Supplementary Video 7
Escapes with a short obstacle after experience with a long obstacle
Supplementary Video 8
Spontaneous homing movements
Supplementary Video 9
Food-seeking task before and after obstacle removal
Supplementary Video 10
Automated obstacle removal
Supplementary Video 11
Tracking the mouse’s position, heading direction and speed with DeepLabCut
Rights and permissions
About this article
Cite this article
Shamash, P., Olesen, S.F., Iordanidou, P. et al. Mice learn multi-step routes by memorizing subgoal locations. Nat Neurosci 24, 1270–1279 (2021). https://doi.org/10.1038/s41593-021-00884-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41593-021-00884-8
- Springer Nature America, Inc.
This article is cited by
-
Interactions between rodent visual and spatial systems during navigation
Nature Reviews Neuroscience (2023)
-
A step-by-step guide home
Nature Neuroscience (2021)