1 Introduction

Territoriality is a wide-spread phenomenon throughout nature. A territory is an area of space used exclusively by an organism, or a group of organisms (Burt 1943). It is formed by deliberately excluding others of the same species (called conspecifics) from the area, either by aggressive confrontations or mutual consent (Adams 2001). In the last two decades, there have been a number of studies that show analytically how territorial patterns can form from the movements and interactions of animals (Lewis and Murray 1993; Moorcroft and Lewis 2006; Potts and Lewis 2014). These use mean-field approximations to model the animals’ behavioral decisions as partial differential equations (PDEs), and so enable territory formation to be analyzed using standard tools from PDE theory (Murray 2002).

Despite their success in uncovering drivers behind space use patterns (Moorcroft et al. 2006), previous analytical models assume an attractive potential influencing the animals’ movements. This could either be fidelity to a central place such as a den- or nest-site (Lewis et al. 1997), or a tendency to move towards places that the animal has previously visited (Briscoe et al. 2002). However, it is not clear that such an attractive potential is in fact necessary for territory formation (Moorcroft 2012).

In this paper, we present a PDE model of territorial pattern formation based purely on conspecific avoidance, with no attractive potential. It is based on an individual based model (IBM) of so-called territorial random walkers (Giuggioli et al. 2011a). Previous work used simulation analysis to demonstrate empirically that territories can form in this system (Giuggioli et al. 2011a). Here, we show analytically the circumstances under which territorial patterns may form. Specifically, necessary conditions for territorial pattern formation include

  • spatial aversion to scent marks

  • scent marks that persist for longer than it takes the animal to traverse the terrain, and

  • a reaction to conspecific scent averaged over a small area around the animal.

As is often the case in ecological applications, it is important that the discrete spatial nature of each interaction is present in the model (Durrett and Levin 1994). In the case of territorial interactions, this discreteness is inherent in the fact that animals have a non-zero perceptive radius for determining the presence of scent. As part of this study, we develop a limiting procedure that enables the transition from IBM to PDE without losing this important aspect of spatially discrete interactions. This has the potential for general use, as previous limiting procedures have often failed in this regard (Durrett and Levin 1994).

The paper is organized as follows. Section 2 derives the PDE from the IBM model. Sections 3 and 4 investigate the conditions under which patterns may form. Section 5 gives some concluding remarks.

2 From the individual-level description to a system of PDEs

2.1 Description of model

The individual based model (IBM) is based on a 1D model of territoriality which was recently proposed by Giuggioli et al. (2011a), but then slightly modified and studied in detail by Giuggioli et al. (2011b) and Potts et al. (2012). The model consists of two agents moving on a 1D lattice. The agents represent either a single individual responsible for territorial defense, or a group of individuals moving together, such as a pack or a flock. For example, the former is appropriate when modelling fox (Vulpes vulpes) behaviour where the dominant male in each group marks and secures the territory (Harris 1980), whereas the latter may be more appropriate for modelling wolf (Canis lupus) packs (Lewis and Murray 1993).

Agents move as discrete-time discrete-space nearest-neighbour random walkers, depositing scent marks as they move. In the model of Giuggioli et al. (2011b), the scent remains present for a finite amount of time, called the active scent time and denoted by the symbol \(T_\mathrm{AS}\). Once this time is up, provided the lattice site has not been re-scented, the mark is no longer considered by conspecifics to be ‘active’. Agents cannot move into any lattice site that contains the active scent of another agent (Fig. 1a).

Fig. 1
figure 1

Pictorial representation of the underlying model. The territory of each agent represents the sites containing that agent’s scent. In a, agent 1 is unable to move to the right in the next step, since there is active scent of agent 2 there. However, agent 2 can move in either direction. In b, we show the case where the lattice spacing, \(a\), is halved, so response to scent is averaged over several sites, given by the grey ovals. As the lattice spacing is reduced by a factor of \(h(a)\), so the response to scent is averaged over \(2h(a)-1\) sites. In this situation, agent 1 has a higher probability of moving left than right, while agent 2 has equal probability of moving in both directions

Our model set-up will take three stages. Stage 1 uses the formalism of coupled step selection functions (Potts et al. 2014) to describe a stochastic IBM algorithm which generalizes that of Giuggioli et al. (2011b). Stage 2 describes how to derive a mean-field probabilistic model from the IBM. Stage 3 involves taking the PDE limit of the probabilistic model . The various parameters used in the study are detailed in Table 1.

Table 1 Glossary of symbols

2.2 Stochastic algorithm for the individual based model

If unconstrained by scent marks, an agent is simply a nearest-neighbor random walker. Therefore the probability that agent \(i\) (\(i \in \{1,2\}\)) moves from site \(n'\) to \(n\) is \(\phi _i(n|n')=1/2\) if \(|n-n'|=1\) and \(\phi _i(n|n')=0\) otherwise. This function \(\phi _i(n|n')\) is the environment-independent movement kernel.

Now we add the effect of scent marks, which for this paper are thought of as constituting of the animal’s ‘environment’. For each agent \(i\), let the environment, \({E}_i(n,m)\), be the probability that there is conspecific scent at lattice site \(n\) and timestep \(m\). We give two possible definitions for \({E}_i(n,m+1)\), denoted by \({E}^1_i(n,m+1)\) and \({E}^2_i(n,m+1)\), and both defined in terms of the state of the system at timestep \(m\). The first is given by

$$\begin{aligned} {E}^1_i(n,m+1)={\left\{ \begin{array}{ll}1 &{}\quad \text {if an agent }\, j\ne i\,\hbox { is at position }n\hbox { at any time } \\ &{}\quad \text {between }\, m-T_{AS}+1\hbox { and }m,\\ 0 &{}\quad \text {otherwise.} \end{array}\right. } \end{aligned}$$
(1)

If \({E}^1_i(n,m)=1\) then there is conspecific scent present, otherwise there is not. This is the definition used by Giuggioli et al. (2011b) and Potts et al. (2012).

An alternative to Eq. (1) is the following definition

$$\begin{aligned} {E}^2_i(n,m+1)={\left\{ \begin{array}{ll}1-\mu \tau &{}\quad \text {with probability }\, \kappa (a),\hbox { if an agent }j\ne i\\ &{}\quad \hbox { is at} \hbox { position }n\hbox { at}\,\,\hbox {timestep }m,\\ (1-\mu \tau ){E}^2_i(n,m) &{} \quad \text {otherwise,} \end{array}\right. } \end{aligned}$$
(2)

where \(\tau \) is the length of a timestep and \(\kappa (a)\) is the probability that scent is deposited when the animal visits a lattice site. Notice that scent left at timestep \(m\) has a probability \(1-\mu \tau \) of remaining present at timestep \(m+1\).

Introducing \(\kappa (a)\) allows us to change the lattice spacing \(a\) without changing the average distance moved between scent depositions, by insisting that \(a/\kappa (a)\) is kept constant. From Sect. 2.3 onwards, we will use Eq. (2) to describe scent deposition and decay. However, the stochastic algorithm of this section can be defined equally well using either Eq. (1) or (2).

We now define the interaction term, which denotes how the scent affects the agent’s movement. Animals typically have a perceptive radius that determines the spatial area over which they respond to scent. The model of Giuggioli et al. (2011b) implicitly identified this perceptive radius with the lattice spacing \(a\). However, this limits the model’s flexibility: if the lattice spacing is changed then the model assumptions about the animal’s perceptive radius are also changed. Therefore, to ensure our model is not constrained by the choice of \(a\), we define the interaction term, \({C}_i^j(n,m)\), to be a Bernoulli random variable taking value 1 with probability

$$\begin{aligned} \varphi =1-\frac{1}{2h(a)-1}\sum _{l=1-h(a)}^{h(a)-1}{E}^j_i(n+l,m), \end{aligned}$$
(3)

where \(j \in \{1,2\}\), \(h(a)\) is defined so that \(ah(a)\) is the perceptive radius of the animal, and \({C}_i^j(n,m)\) takes value 0 with probability \(1-\varphi \). The model from Giuggioli et al. (2011b) implicitly had \(h(a)=1\). In general, to change the lattice spacing whilst keeping the perceptive radius \(\delta \) constant requires setting \(h(a)=\delta /a\), which holds as long as \(h(a)\) is an integer (see Fig. 1b).

The probability \(f_i^m(n|n')\) of agent \(i\) moving to \(n\) at timestep \(m\), given that is was previously at position \(n'\), is a combination of \(\phi _i(n|n')\) and \({C}_i^j(n,m)\), written as follows

$$\begin{aligned}&f_i^m(n|n')\nonumber \\&\quad ={\left\{ \begin{array}{ll}\frac{\phi _i(n|n'){C}_i^j(n,m)}{\phi _i(n'\!+\!1|n'){C}_i^j(n'\!+\!1,m)\!+\!\phi _i(n'\!-\!1|n'){C}_i^j(n'\!-\!1,m)} &{}\quad \text {if}\, {C}_i^j(n'\!+\!1,m)\!+\! {C}_i^j(n'\!-\!1,m)\ne 0,\\ \delta _k(n-n') &{}\quad \text {otherwise,} \end{array}\right. } \end{aligned}$$
(4)

where \(\delta _k\) is the Kronecker delta.

Equation (4) allows us to describe the stochastic algorithm. This is a one-step Markov process, so can be fully described by determining the possible states of the system at timestep \(m+1\), given the state at time \(m\). Suppose that, for some \(m\), we know \({E}^j_i(n,m)\) for every \(n\). Suppose further that animal \(i\) is at position \(n_i\) at timestep \(m\). Then the algorithm is as follows

  1. 1.

    Calculate \({E}^j_i(n,m+1)\) for each \(n\).

  2. 2.

    Define a categorical distribution taking one of three values \(n_i-1,n_i,n_i+1\) with probabilities given by \(f_i^m(n_i-1|n_i)\), \(f_i^m(n_i|n_i)\) and \(f_i^m(n_i+1|n_i)\) respectively. These values are the possible future positions of animal \(i\).

  3. 3.

    Draw a random variable from this categorical distribution and move the animal to the position just drawn.

  4. 4.

    Repeat steps 2 and 3 for each animal in turn.

2.3 Probability distribution of an agent in a given scent distribution

To construct a probabilistic master equation describing the above stochastic process, we first assume that the evolution of the scent marks can be decoupled from the movement of the agent. In other words, we calculate the equation governing a single step of each agent’s movement that is true for any fixed, arbitrary scent distribution of the other agent. This is a so-called mean-field approximation, that assumes covariates between the agent and conspecific scent are small enough to ignore.

Let \(U(n,m)\) (resp. \(V(n,m)\)) be the probability of agent \(1\) (resp. \(2\)) being at position \(n\) at timestep \(m\) and \(P(n,m)\) (resp. \(Q(n,m)\)) the probability of there being scent present of agent \(1\) (resp. \(2\)) at position \(n\) at timestep \(m\). By analysing the probability of moving to site \(n\) from either site \(n-1\), \(n\), or \(n+1\) in one timestep, we eventually arrive at the following discrete space-time master equations

$$\begin{aligned} U(n,m+1)=&[1-Q(n+i,m)]\biggl \{ \frac{1}{2}U(n-1,m)[1+Q(n+i-2,m)]\nonumber \\&+\,\frac{1}{2}U(n+1,m)[1+Q(n+i+2,m)]\biggr \} \nonumber \\&+\,U(n,m)Q(n+i-1,m)Q(n+i+1,m), \end{aligned}$$
(5)
$$\begin{aligned} V(n,m+1)=&[1-P(n+i,m)]\biggl \{ \frac{1}{2}V(n-1,m)[1+P(n+i-2,m)]\nonumber \\&+\,\frac{1}{2}V(n+1,m)[1+P(n+i+2,m)]\biggr \} \nonumber \\&+\,V(n,m)P(n+i-1,m)P(n+i+1,m), \end{aligned}$$
(6)

where the following implicit summation notation (Einstein 1916) is used

$$\begin{aligned} P(n+i,t):=\frac{1}{2h(a)-1}\sum _{i=1-h(a)}^{h(a)-1} P(n+i,t), \nonumber \\ Q(n+i,t):=\frac{1}{2h(a)-1}\sum _{i=1-h(a)}^{h(a)-1} Q(n+i,t), \end{aligned}$$
(7)

and \(a\) is the lattice spacing and the product \(ah(a)\) is the perceptive radius of the agent.

To give some intuition behind Eqs. (5) and (6), we focus on Eq. (5), and note that all of the comments in this paragraph hold equally well for Eq. (6). The initial \([1-Q(n+i,m)]\) factor in Eq. (5) ensures that there is a low probability of moving to position \(n\) if there is a high probability of active conspecific scent being present at or around position \(n\). The factor \([1+Q(n+i-2,m)]\) (resp. \([1+Q(n+i+2,m)]\)) means that if scent is likely to be present at or around \(n-2\) (\(n+2\)) and the animal is at \(n-1\) (\(n+1\)) at time \(m\) then it will be likely to move to \(n\) at time \(m+1\). The final summand \(U(n,m)Q(n+i-1,m)Q(n+i+1,m)\) means that if the presence of scent is highly probable both to the left and right of an animal at time \(m\), then it is likely to stay where it is. Notice that if \(\sum _n U(n,m)=1\) then \(\sum _n U(n,m+1)=1\) so that probabilities are conserved.

Let \(\tau \) be the waiting-time between successive jumps. Then Eqs. (5) and (6) rearrange to give

$$\begin{aligned} \frac{U(n,m+1)-U(n,m)}{\tau }=&\frac{1}{2\tau }[1-Q(n+i,m)]\{U(n-1,m)[1+Q(n+i-2,m)]\nonumber \\&+\,U(n+1,m)[1+Q(n+i+2,m)]\}\nonumber \\&-\,\frac{1}{\tau }[1-Q(n+i-1,m)Q(n+i+1,m)]U(n,m), \end{aligned}$$
(8)
$$\begin{aligned} \frac{V(n,m+1)-V(n,m)}{\tau }=&\frac{1}{2\tau }[1-P(n+i,m)]\{V(n-1,m)[1+P(n+i-2,m)]\nonumber \\&+\,V(n+1,m)[1+P(n+i+2,m)]\}\nonumber \\&-\,\frac{1}{\tau }[1-P(n+i-1,m)P(n+i+1,m)]V(n,m),\nonumber \\ \end{aligned}$$
(9)

Equation (8) can be re-written as follows

$$\begin{aligned}&\frac{U(n,m\!+\!1)\!-\!U(n,m)}{\tau }\!=\!\frac{a^2}{2\tau }\biggl \{\frac{1}{a}\left[ \frac{U(n\!+\!1,t)\!-\!U(n,t)}{a}\!-\!\frac{U(n,t)-U(n-1,t)}{a}\right] \nonumber \\&\quad +\,\frac{1}{2a}\biggl [4U(n+1,t)\frac{Q(n+i+2,t)-Q(n+i,t)}{2a}\nonumber \\&\quad -\,4U(n-1,t)\frac{Q(n+i,t)-Q(n+i-2,t)}{2a}\biggr ] \nonumber \\&\quad +\,\frac{1}{a}\biggl [\frac{U(n,t)Q(n\!+\!i\!+\!1,t)Q(n\!+\!i\!-\!1,t)\!-\!U(n\!-\!1,t)Q(n\!+\!i,t)Q(n\!+\!i\!-\!2,t)}{a}\nonumber \\&\quad -\,\frac{U(n\!+\!1,t)Q(n\!+\!i\!+\!2,t)Q(n\!+\!i,t)\!-\!U(n,t)Q(n\!+\!i\!+\!1,t)Q(n\!+\!i\!-\!1,t)}{a}\biggr ]\biggr \},\nonumber \\ \end{aligned}$$
(10)

and similarly for Eq. (9). Taking the limit as \(a,\tau \rightarrow 0\) and \(n,m,h(a)\rightarrow \infty \) such that \(D=a^2/(2\tau )\), \(x=na\), \(t=m\tau \), \(ah(a)=\delta \) in the limit, and writing \(u(x,t)\) (resp. \(v(x,t)\)) for the probability density functions of agent 1’s (resp. 2’s) position and \(p(x,t)\) (resp. \(q(x,t)\)) for the probability that agent 1’s (resp. 2’s) active scent is present at position \(x\) at time \(t\), we arrive at the following PDE (see Appendix A for a full derivation)

$$\begin{aligned} \frac{\partial u}{\partial t}=D\frac{\partial ^2}{\partial x^2}[(1-\bar{q}^2)u]+4D\frac{\partial }{\partial x}\left[ \frac{\partial \bar{q}}{\partial x}u\right] . \end{aligned}$$
(11)

The equation governing the evolution of \(v(x,t)\) over time is analogous

$$\begin{aligned} \frac{\partial v}{\partial t}=D\frac{\partial ^2}{\partial x^2}[(1-\bar{p}^2)v]+4D\frac{\partial }{\partial x}\left[ \frac{\partial \bar{p}}{\partial x}v\right] . \end{aligned}$$
(12)

Here, \(\bar{p}(x,t)\) and \(\bar{q}(x,t)\) are the locally averaged scent of agents 1 and 2, respectively

$$\begin{aligned}&\bar{p}(x,t)=\frac{1}{2\delta }\int _{-\delta }^{\delta }p(x+z,t)\mathrm{d}z, \end{aligned}$$
(13)
$$\begin{aligned}&\bar{q}(x,t)=\frac{1}{2\delta }\int _{-\delta }^{\delta }q(x+z,t)\mathrm{d}z. \end{aligned}$$
(14)

2.4 Evolution of the scent distribution

Recall that we gave two different formulae for the scent decay process, Eqs. (1) and (2). For the purposes of our mean-field analysis, it is convenient to use Eq. (2). In other words, the probability of scent being present at lattice site \(n\) decays by a factor of \(1-\tau \mu \) each timestep of length \(\tau \). Additionally, when a site is visited by the animal, the probability that there is active scent present jumps to 1 with probability \(\kappa (a)\).

The following master equation follows directly from taking the expectation of either side of Eq. (2)

$$\begin{aligned} P(n,m+1)=(1-\mu \tau ) U(n,m)\kappa (a) + (1-\mu \tau )[1- U(n,m)\kappa (a)]P(n,m). \end{aligned}$$
(15)

The probability density version of Eq. (15) is the limit as \(a,\kappa (a),\tau \rightarrow 0\) and \(n,m\rightarrow \infty \) of

$$\begin{aligned} p(na,m\tau \!+\!\tau )\!=&\!(1\!-\!\mu \tau ) u(na,m\tau )a\kappa (a) \!+\! (1\!-\!\mu \tau )[1\!-\! u(na,m\tau )a\kappa (a)]p(na,m\tau ), \end{aligned}$$
(16)

such that \(x=na\), \(t=m\tau \) and \(\lambda =a\kappa (a)/\tau \) in this limit.

Subtracting \(p(na,m\tau )\) from both sides of Eq. (16), dividing by \(\tau \) and taking this limit leads to the following ordinary differential equation (ODE) governing \(p(x,t)\)

$$\begin{aligned} \frac{\partial p}{\partial t}= \lambda (1-p)u - \mu p. \end{aligned}$$
(17)

We can interpret \(\lambda \) as representing the amount of scent deposited over a unit of space in a single unit of time. The derivation for \(q(x,t)\) is similar and gives

$$\begin{aligned} \frac{\partial q}{\partial t}= \lambda (1-q)v - \mu q. \end{aligned}$$
(18)

Analyzing the system of Eqs. (11), (12), (17) and (18) requires choosing an appropriate domain and boundary conditions. A simple and biologically realistic choice is to assume that agents are confined in a domain \([0,L]\) with zero flux boundary conditions. The boundary conditions could either come about by being confined in a valley or on a small island. Alternatively, the conditions could model a situation where the rate of migration of animals into the domain is equal to the rate of movement outwards. In other words, the population is assumed to be exhibiting a certain spatial and temporal stability. These boundary conditions are given as follows

$$\begin{aligned} \left\{ \frac{\partial }{\partial x}[(1-\bar{q}^2)u]+4\left[ \frac{\partial \bar{q}}{\partial x}u\right] \right\} \bigg |_{x=0}=\left\{ \frac{\partial }{\partial x}[(1-\bar{q}^2)u]+4\left[ \frac{\partial \bar{q}}{\partial x}u\right] \right\} \bigg |_{x=L}= & {} 0, \nonumber \\ \end{aligned}$$
(19)
$$\begin{aligned} \left\{ \frac{\partial }{\partial x}[(1-\bar{p}^2)v]+4\left[ \frac{\partial \bar{p}}{\partial x}v\right] \right\} \bigg |_{x=0}=\left\{ \frac{\partial }{\partial x}[(1-\bar{p}^2)v]+4\left[ \frac{\partial \bar{p}}{\partial x}v\right] \right\} \bigg |_{x=L}= & {} 0.\qquad \nonumber \\ \end{aligned}$$
(20)

The existence of the boundary requires that we need to redefine \(\bar{p}(x,t)\) and \(\bar{q}(x,t)\) in the cases where \(x<\delta \) and \(x>L-\delta \), as follows

$$\begin{aligned} \bar{p}(x,t)= & {} {\left\{ \begin{array}{ll} \frac{1}{x+\delta }\int _{-x}^{\delta }p(x+z,t)\mathrm{d}z &{} \quad \text{ if } x<\delta , \\ \frac{1}{2\delta }\int _{-\delta }^{\delta }p(x+z,t)\mathrm{d}z &{} \quad \text{ if } \delta \le x\le L-\delta , \\ \frac{1}{L-x+\delta }\int _{-\delta }^{L-x}p(x+z,t)\mathrm{d}z &{} \quad \text{ if } x>L-\delta , \end{array}\right. } \end{aligned}$$
(21)
$$\begin{aligned} \bar{q}(x,t)= & {} {\left\{ \begin{array}{ll} \frac{1}{x+\delta }\int _{-x}^{\delta }q(x+z,t)\mathrm{d}z &{} \quad \text{ if } x<\delta , \\ \frac{1}{2\delta }\int _{-\delta }^{\delta }q(x+z,t)\mathrm{d}z &{} \quad \text{ if } \delta \le x\le L-\delta , \\ \frac{1}{L-x+\delta }\int _{-\delta }^{L-x}q(x+z,t)\mathrm{d}z &{}\quad \text{ if } x>L-\delta . \end{array}\right. } \end{aligned}$$
(22)

In addition to the boundary conditions, it is necessary to impose integral conditions on the initial probability distributions \(u(x,0)\) and \(v(x,0)\), to ensure that probability is conserved. In other words

$$\begin{aligned} \int _0^L u(x,0)\mathrm{d}x=\int _0^L v(x,0)\mathrm{d}x = 1. \end{aligned}$$
(23)

A consequence of Eqs. (19) and (20) is that the time-derivative of \(\int _0^L u(x,t)\mathrm{d}x\) is zero. Therefore the initial conditions from Eq. (23) imply that probabilities are conserved at every point in time, i.e.

$$\begin{aligned} \int _0^L u(x,t)\mathrm{d}x=\int _0^L v(x,t)\mathrm{d}x = 1. \end{aligned}$$
(24)

2.5 A dimensionless version of the model

To minimize the number of model parameters, we re-write Eqs. (11), (12), (17), and (18), using the following dimensionless parameters

$$\begin{aligned} \tilde{u}=Lu,\,\,\tilde{v}=Lv,\,\,\tilde{x}=\frac{x}{L},\,\,\tilde{t}=\frac{tD}{L^2},\,\,m=\frac{\mu L}{\lambda },\,\,\varepsilon =\frac{D}{L\lambda }. \end{aligned}$$
(25)

Dropping the tildes over the letters to ease notation, we arrive at the following dimensionless system of equations, which will be the object of study for the rest of this paper

$$\begin{aligned} \frac{\partial u}{\partial t}= & {} \frac{\partial ^2}{\partial x^2}[(1-\bar{q}^2)u ]+4\frac{\partial }{\partial x}\left[ \frac{\partial \bar{q}}{\partial x}u\right] , \end{aligned}$$
(26)
$$\begin{aligned} \frac{\partial v}{\partial t}= & {} \frac{\partial ^2}{\partial x^2}[(1-\bar{p}^2)v ]+4\frac{\partial }{\partial x}\left[ \frac{\partial \bar{p}}{\partial x}v\right] , \end{aligned}$$
(27)
$$\begin{aligned} \varepsilon \frac{\partial p}{\partial t}= & {} (1-p)u - mp, \end{aligned}$$
(28)
$$\begin{aligned} \varepsilon \frac{\partial q}{\partial t}= & {} (1-q)v - mq. \end{aligned}$$
(29)

3 Territorial patterns

We define a territorial pattern to be a non-trivial steady-state solution to Eqs. (26)–(29). These are found by setting to zero the left-hand sides of Eqs. (26)–(29). Setting Eq. (28) (resp. Eq. (29)) to zero enables the steady state solution of \(p(x,t)\) (resp. \(q(x,t)\)), denoted by \(p^*(x)\) (resp. \(q^*(x)\)), to be written in terms of the steady state solution of \(u(x,t)\) (resp. \(v(x,t)\)), denoted by \(u^*(x)\) (resp. \(v^*(x)\)) as follows

$$\begin{aligned} p^*(x)= & {} \frac{u^*(x)}{m + u^*(x)}, \end{aligned}$$
(30)
$$\begin{aligned} q^*(x)= & {} \frac{v^*(x)}{m + v^*(x)}. \end{aligned}$$
(31)

To ease notation, we will henceforth drop the asterisks. By setting Eqs. (26) and (27) to zero and integrating with respect to \(x\), we have that

$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d} x} \{(1-\bar{q}[v(\cdot ),x]^2)u(x)\}+4\left[ \frac{\mathrm{d} \bar{q}}{\mathrm{d} x}u(x)\right]= & {} {c}_{1}, \end{aligned}$$
(32)
$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d} x}\{(1-\bar{p}[u(\cdot ),x]^2)v(x)\}+4\left[ \frac{\mathrm{d} \bar{p}}{\mathrm{d} x}v(x)\right]= & {} {c}_{2}, \end{aligned}$$
(33)

for constants \(c_1\) and \(c_2\). The boundary conditions given by Eqs. (19) and (20) imply that \(c_1=c_2=0\).

We use the notation \(\bar{p}[u(\cdot ),x]\) and \(\bar{q}[v(\cdot ),x]\) to emphasize the fact that \(\bar{p}\) and \(\bar{q}\) are functionals. That is, they map the functions \(u(\cdot )\) and \(v(\cdot )\), respectively, to the interval \([0,1]\). These functionals are given by the following formulae

$$\begin{aligned} \bar{p}[u(\cdot ),x]= & {} {\left\{ \begin{array}{ll} \frac{1}{x+\delta }\int _{-x}^{\delta } \frac{u(x+z)}{m+u(x+z)} \mathrm{d}z, &{}\quad \text{ if } x<\delta , \\ \frac{1}{2\delta }\int _{-\delta }^{\delta } \frac{u(x+z)}{m+u(x+z)} \mathrm{d}z &{} \quad \text{ if } \delta \le x\le 1-\delta , \\ \frac{1}{1-x+\delta }\int _{-\delta }^{1-x}\frac{u(x+z)}{m+u(x+z)}\mathrm{d}z &{} \quad \text{ if } x>1-\delta , \end{array}\right. } \end{aligned}$$
(34)
$$\begin{aligned} \bar{q}[v(\cdot ),x]= & {} {\left\{ \begin{array}{ll} \frac{1}{x+\delta }\int _{-x}^{\delta } \frac{v(x+z)}{m+v(x+z)} \mathrm{d}z, &{} \quad \text{ if } x<\delta , \\ \frac{1}{2\delta }\int _{-\delta }^{\delta } \frac{v(x+z)}{m+v(x+z)} \mathrm{d}z &{} \quad \text{ if } \delta \le x\le 1-\delta , \\ \frac{1}{1-x+\delta }\int _{-\delta }^{1-x}\frac{v(x+z)}{m+v(x+z)}\mathrm{d}z &{} \quad \text{ if } x>1-\delta , \end{array}\right. } \end{aligned}$$
(35)

In sum, as well as Eqs. (34) and (35), we have the following system of equations, whose non-constant solutions correspond to territorial patterns

$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d} x}\{(1-\bar{q}[v(\cdot ),x]^2)u(x)\}+4\left[ \frac{\mathrm{d} \bar{q}}{\mathrm{d} x}u(x)\right]= & {} 0, \end{aligned}$$
(36)
$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d} x}\{(1-\bar{p}[u(\cdot ),x]^2)v(x)\}+4\left[ \frac{\mathrm{d} \bar{p}}{\mathrm{d} x}v(x)\right]= & {} 0, \end{aligned}$$
(37)
$$\begin{aligned} {p(x)}= & {} \frac{u(x)}{m + u(x)}, \end{aligned}$$
(38)
$$\begin{aligned} {q(x)}= & {} \frac{v(x)}{m + v(x)}. \end{aligned}$$
(39)

3.1 Territorial patterns with only local interactions

We first examine the case where \(\delta \rightarrow 0\) so that agents only respond to scent at the exact position where they are situated. This means Eqs. (36) and (37) become

$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d} x}\{(1-{q(x)}^2)u(x)\}+4\left[ \frac{\mathrm{d} q}{\mathrm{d} x}u(x)\right]= & {} 0, \end{aligned}$$
(40)
$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d} x}\{(1-{p(x)}^2)v\}+4\left[ \frac{\mathrm{d} p}{\mathrm{d} x}v(x)\right]= & {} 0. \end{aligned}$$
(41)

The limit \(\delta \rightarrow 0\) means that the functionals \(\bar{p}[u(\cdot ),x]\) and \(\bar{q}[v(\cdot ),x]\) have been replaced by functions \(p(x)\) and \(q(x)\), which makes analysis tractable. To ease notation, we hencefore drop the explicit dependence of the functions \(u\), \(v\), \(p\), and \(q\) on \(x\).

By substituting Eqs. (38) and (39) into (40) and (41), the following system of ODEs for the steady state solution of \((u,v)\) is found

$$\begin{aligned} A\dot{\mathbf{u}}&=0, \nonumber \\ A&=m\left( \begin{array}{cc} (m+2v)(m+v) &{} 2u(2m+v) \\ 2v(2m +u) &{} (m +2u)(m + u) \end{array} \right) , \nonumber \\ \dot{\mathbf{u}}&= \left( \begin{array}{c} \mathrm{d}u{/}\mathrm{d}x \\ \mathrm{d}v{/}\mathrm{d}x \end{array} \right) . \end{aligned}$$
(42)

The system of ODEs in Eq. (42) is simple enough to analyze mathematically. The results of this analysis are summarized in the following

Theorem 1

  1. 1.

    No scent decay. If \(m=0\) then \(p(x)=q(x)=1\) and \(u(x)\), \(v(x)\) can take any value.

  2. 2.

    Positive scent decay. If \(m>0\) then there are no non-constant solutions to Eq. (42). Hence no territorial patterns can form in this case.

Proof

See Appendix B. \(\square \)

3.2 Territorial patterns with non-local interactions

In the case where \(\delta >0\), Eqs. (36)–(39) give a system of integral-ODEs, so are harder to analyse analytically. Instead, we solve them numerically using the method of false transients (Mallinson and Vahl 1973). This involves solving Eqs. (26)–(29) forward in time until the solution is unchanging.

Our algorithm uses a forward-difference approximation for time and a central difference approximation for space. We divide the interval \([0,1]\) into \(1000\) equal, non-intersecting, sub-intervals of length \(0.001\). We iterate finite-difference versions of Eqs. (26)–(29) using timesteps of \(0.01\), until all of the \(u(x,t)\) or \(v(x,t)\) values in all of the sub-sections are increasing by less than \(10^{-8}\) over each timestep. The initial conditions have all of \(u(x)\) concentrated on the sub-interval \([0.25,0.251)\) and all of \(v(x)\) on the sub-interval \([0.75,0.751)\). This means \(u(x)\) and \(v(x)\) are zero outside the sub-intervals \([0.25,0.251)\) and \([0.75,0.751)\) respectively, and each integrate to 1 over \([0,1]\).

Numerical analysis shows that patterns emerge from this system corresponding to two territories: \(u(x)\) on the left and \(v(x)\) on the right (Fig. 2a, b). Notice that a larger scent averaging radius leads to wider overlap of the probability distributions, meaning that the perceptive scale of the animal plays a large role in the territorial patterns that emerge.

Fig. 2
figure 2

Numerical steady state solutions of the model. Solid red (resp. black) lines denote values of \(u(x)\) (resp. \(v(x)\)), whereas dotted red (resp. black) lines show values of \(p(x)\) (resp. \(q(x)\)). In both panels, \(m=0.4\) and \(\varepsilon =0.01\). In a, \(\delta =0.1\), whilst b has \(\delta =0.01\). Notice that a larger the scent-averaging radius, \(\delta \), gives a larger overlap between territories. c Compares steady states of simulations of the original IBM (dashed lines) with numerical results from the PDE approximation (solid lines). As in a and b, dotted red (resp. black) lines show values of \(p(x)\) (resp. \(q(x)\)). Here, \(\delta =0.01\), \(m=0.4\), and \(\varepsilon =0.01\). This corresponds, in the IBM, to \(T_\mathrm{AS}/\tau =500\) and \(N=100\)

These can be compared with the territories that form in the original IBM with the interaction rules from Giuggioli et al. (2011b). Although there is some qualitative agreement, the patterns generated by the IBM are still quite different to the PDE. In the IBM, at any point in time, there is a border between the two territories. This border fluctuates about the central point, typically much slower than the movement of the agent. Each agent is free to move within its territory borders. Consequently, the probability density of both agents combined (\(u+v\)) ends up being roughly uniform (Fig. 2c). This does not happen in the mean field approximation studied here. Indeed, the value of \(u+v\) appears to be lower in the middle of the terrain. Since this is just an artifact of the assumptions made in using the PDE limit, it is necessary to be cautious when inferring biological lessons from such pattern features.

4 Investigating pattern formation via linear analysis

A common technique for examining whether patterns spontaneously form in a dynamical system is to linearize the system about the uniform steady state and examine the resulting dispersion relation, e.g. (Murray (2002), chapter 2). For our system, the uniform steady state is

$$\begin{aligned} (u_s,v_s,p_s,q_s)=\left( 1,1,\frac{1}{1+m},\frac{1}{1+m}\right) . \end{aligned}$$
(43)

That \(u_s=v_s=1\) arises from the integral conditions (Eq. 24). The values for \(p_s\) and \(q_s\) then follow from Eqs. (38) and (39).

Letting \(\mathbf{w}=(\hat{u},\hat{v},\hat{p},\hat{q})=(u-u_s,v-v_s,p-p_s,q-q_s)\), we use Eqs. (2629) to give the linearized system

$$\begin{aligned} \frac{\partial \hat{u}}{\partial t}&= \left[ 1-\frac{1}{(1+m)^2}\right] \frac{\partial ^2\hat{u}}{\partial x^2}+2\left[ 2-\frac{1}{1+m}\right] \frac{\partial ^2 \hat{\bar{q}}}{\partial x^2}, \nonumber \\ \frac{\partial \hat{v}}{\partial t}&= \left[ 1-\frac{1}{(1+m)^2}\right] \frac{\partial ^2\hat{v}}{\partial x^2}+2\left[ 2-\frac{1}{1+m}\right] \frac{\partial ^2 \hat{\bar{p}}}{\partial x^2}, \nonumber \\ \frac{\partial \hat{p}}{\partial t}&= \frac{m}{\varepsilon (1+m)}\hat{u}-\frac{1+m}{\varepsilon }\hat{p}, \nonumber \\ \frac{\partial \hat{q}}{\partial t}&= \frac{m}{\varepsilon (1+m)}\hat{v}-\frac{1+m}{\varepsilon }\hat{q}. \end{aligned}$$
(44)

Searching for solutions of the form \(w=(u_0,v_0,p_0,q_0)\exp (\sigma t+\mathrm{i}kx)\), we obtain the following eigenvector equation

$$\begin{aligned}&A\mathbf{w}=\sigma \mathbf{w} \nonumber \\&A=\left( \begin{array}{cccc} \left[ \frac{1}{(1+m)^2}-1\right] k^2 &{}\quad 0&{}\quad 0 &{}\quad -2\left[ 2-\frac{1}{1+m}\right] \frac{k}{\delta }\sin {\delta k} \\ 0 &{}\quad \left[ \frac{1}{(1+m)^2}-1\right] k^2 &{}\quad \quad -2\left[ 2-\frac{1}{1+m}\right] \frac{k}{\delta }\sin {\delta k} &{}\quad 0 \\ \frac{m}{\varepsilon (1+m)} &{}\quad 0 &{}\quad -\frac{1+m}{\varepsilon } &{}\quad 0 \\ 0 &{}\quad \frac{m}{\varepsilon (1+m)} &{}\quad 0 &{}\quad -\frac{1+m}{\varepsilon } \end{array}\right) . \end{aligned}$$
(45)

The dispersion relation is given by plotting the real values of \(\sigma \) as a function of the wave number \(k\), wherever \(\text{ det }(A-\sigma I)=0\). As shown in Fig. 3a, patterns can form for a finite range of wavelengths as long as \(m<1\); that is, as long as the scent decay is not too rapid.

Fig. 3
figure 3

Dispersion relations. a Show the dispersion relation for the dynamical system in Eqs. (2629), where \(m=0.1\) (solid line), \(m=0.5\) (dotted), \(m=1\) (dashed), and \(m=3\) (dot-dashed). We set \(\varepsilon =0.01\) and \(\delta =0.01\) throughout. b Shows the same dispersion relations, but this time the animals respond only to the scent density at the particular point in which they reside, i.e. we take the limit \(\delta \rightarrow 0\). This system is given in Eqs. (28), (29), (46) and (47). The values of \(\varepsilon \) and \(m\) are identical to those in a

We can gain biological insight by relating this result back to the underlying IBM. Recall that \(m=\mu L/\lambda \) (Eq. 25). Recall also that \(\lambda \) is the limit of \(a\kappa (a)/\tau \). In the original lattice model, where \(\kappa (a)=1\), \(a\kappa (a)/\tau \) is simply the speed of the animal. Then \(m<1\) if and only if the time it would take a freely moving animal on the lattice to traverse the whole terrain is less than the characteristic timescale for scent-mark decay \(1/\mu \).

The dispersion relation changes somewhat if we examine the case where \(\delta \rightarrow 0\), so that animals only respond to scent in the exact place that they are located at any point in time. In this case, \(\bar{p}\) and \(\bar{q}\) are replaced by \(p\) and \(q\) respectively, so that Eqs. (26) and (27) are replaced by

$$\begin{aligned} \frac{\partial u}{\partial t}= & {} \frac{\partial ^2}{\partial x^2}[(1-{q}^2)u ]+4\frac{\partial }{\partial x}\left[ \frac{\partial {q}}{\partial x}u\right] , \end{aligned}$$
(46)
$$\begin{aligned} \frac{\partial v}{\partial t}= & {} \frac{\partial ^2}{\partial x^2}[(1-{p}^2)v ]+4\frac{\partial }{\partial x}\left[ \frac{\partial {p}}{\partial x}v\right] . \end{aligned}$$
(47)

Equation (45) becomes

$$\begin{aligned} A&=\left( \begin{array}{cccc} \left[ \frac{1}{(1+m)^2}-1\right] k^2 &{} 0 &{}\quad 0 &{} -2\left[ 2-\frac{1}{1+m}\right] k^2 \\ 0 &{} \left[ \frac{1}{(1+m)^2}-1\right] k^2 &{}\quad -2\left[ 2-\frac{1}{1+m}\right] k^2 &{} 0 \\ \frac{m}{\varepsilon (1+m)} &{} 0 &{}\quad -\frac{1+m}{\varepsilon } &{}0 \\ 0 &{} \frac{m}{\varepsilon (1+m)} &{}\quad 0 &{} -\frac{1+m}{\varepsilon } \end{array}\right) , \end{aligned}$$
(48)

which is the limit as \(\delta \rightarrow 0\) of Eq. (45). The corresponding dispersion relation is given in Fig. 3b. Here, for \(0<m<1\), \(\sigma \) is an increasing function of \(k\), indicating that the steady state is unstable but arbitrarily large wave numbers grow fastest. In other words, this is an ill-posed problem.

5 Discussion and conclusions

We have shown how stable territorial patterns can form purely from a conspecific avoidance mechanism, without requiring any attractive potential. Our model is constructed by taking the continuous space-time limit of a discrete lattice model. Therefore it can be rigorously linked to the underlying movement and interaction processes. We have demonstrated that patterns will only form if the scent marks last for a sufficiently long time. If they decay too quickly, i.e. \(m\ge 1\), the territorial structure breaks down. This can be interpreted as saying territories can only emerge if the animal is able to patrol its territory faster than the scent marks decay.

Similarly, patterns will only form reliably if the animals react to the averaged scent density across the local vicinity of the animal. From a biological perspective, an animal will always have a perceptive radius over which it will react to scent. Therefore this spatial averaging is implicit in the system being modeled. As such, our study demonstrates the importance of ensuring that the mathematical limiting process, moving from discrete to continuous space, does not destroy a key feature of the underlying biology. Our procedure for performing this limiting process has the potential for broad application, since there are many examples where the discreteness of ecological interactions is known to be an important feature of the modeling process (Durrett and Levin 1994).

The model is derived from an individual-based model, previously studied using stochastic simulations (Giuggioli et al. 2011a; Potts et al. 2012). As noted in recent reviews (Giuggioli and Kenkre 2014; Potts and Lewis 2014), one of the advantages of this approach is that it gives a clear delineation between the related notions of ‘home range’ and ‘territory’. The territory of an animal is defined as the area containing active scent marks of the animal (Burt 1943). Therefore, in the model presented here, \(p(x,t)\) and \(q(x,t)\) can be considered the probabilities of position \(x\) being part of the animals’ territories at time \(t\).

On the other hand, the home range of an animal is its utilization distribution (Burt 1943). Therefore \(u(x,t)\) and \(v(x,t)\) can be considered as the home ranges of the animals at time \(t\). The utilization distribution of an animal is typically much easier to measure in the field than the fluctuating locations of the territory border (Potts et al. 2012). In our approach, the concepts of territory and home range are related by rather simple steady-state equations (30) and (31). This gives an explicit way to calculate the probable location of a territory border, given data on its home range.

A key reason for studying PDE limits of IBMs is to provide mathematical analysis of the conditions under which patterns may form, rather than relying on empirical evidence from computer simulations. However, as shown here, patterns that form from numerically solving the PDE may to be quantitatively different from those formed by simulating the IBM. Therefore, if such PDE models were fitted to data on real systems, it is important for the user to check that the PDE results are not significantly different to those given by the IBM. Otherwise, there is a danger of making incorrect inferences about biological patterns, that may merely arise as artifacts of the mean-field approximation and/or limiting procedure.

Models such as ours could be of use in analyzing territory formation when there is no reason to believe the animals have any fidelity towards particular locations, or where these locations are not known, e.g. Bateman et al. (2015). Though memory processes have recently been invoked to explain pattern formation (Briscoe et al. 2002; Moorcroft 2012), it is unclear how to find out what is going on inside the minds of the animals using current science. This makes conjectures about memory difficult to falsify. Conspecific avoidance mechanisms, on the other hand, can be measured directly, e.g. Arnold et al. (2011). Therefore our model of territorial emergence has the potential to be parameterized from empirically measured interaction mechanisms.