Protection zone in a diffusive predator–prey model with Beddington–DeAngelis functional response

Abstract

In any reaction–diffusion system of predator–prey models, the population densities of species are determined by the interactions between them, together with the influences from the spatial environments surrounding them. Generally, the prey species would die out when their birth rate is too low, the habitat size is too small, the predator grows too fast, or the predation pressure is too high. To save the endangered prey species, some human interference is useful, such as creating a protection zone where the prey could cross the boundary freely but the predator is prohibited from entering. This paper studies the existence of positive steady states to a predator–prey model with reaction–diffusion terms, Beddington–DeAngelis type functional response and non-flux boundary conditions. It is shown that there is a threshold value $\theta _0$ which characterizes the refuge ability of prey such that the positivity of prey population can be ensured if either the prey’s birth rate satisfies $\theta \ge \theta _0$ (no matter how large the predator’s growth rate is) or the predator’s growth rate satisfies $\mu \le 0$, while a protection zone $\Omega _0$ is necessary for such positive solutions if $\theta <\theta _0$ with $\mu >0$ properly large. The more interesting finding is that there is another threshold value $\theta ^*=\theta ^*(\mu ,\Omega _0)<\theta _0$, such that the positive solutions do exist for all $\theta \in (\theta ^*,\theta _0)$. Letting $\mu \rightarrow \infty$, we get the third threshold value $\theta _1=\theta _1(\Omega _0)$ such that if $\theta >\theta _1(\Omega _0)$, prey species could survive no matter how large the predator’s growth rate is. In addition, we get the fourth threshold value $\theta _*$ for negative $\mu$ such that the system admits positive steady states if and only if $\theta >\theta _*$. All these results match well with the mechanistic derivation for the B-D type functional response recently given by Geritz and Gyllenberg (J Theoret Biol 314:106–108, 2012). Finally, we obtain the uniqueness of positive steady states for $\mu$ properly large, as well as the asymptotic behavior of the unique positive steady state as $\mu \rightarrow \infty$.

1 Introduction

Biological resources are renewable, but many have been exploited unreasonably. Nowadays, some species cannot survive in their habitat without human intervention. Such interventions have included establishing banned fishing areas and fishing periods to cope with over-fishing in fishery production, setting up nature reserves to protect the endangered species, etc. These phenomena are usually described via diffusive predator–prey models, where the population evolution of the species relies on the interactions between predator and prey, as well as the influences from the spatial environments surrounding them. Naturally, prey species would die out when the prey’s birth rate is too low, the habitat size is too small, the predator’s growth rate is too fast, or the predation rate is too high. To save the endangered prey species, various human interferences are proposed such as creating a protection zone where the prey could cross the boundary freely but the predator is prohibited from entering. Refer to the works on protection zones by Du et al for the Lotka–Voltera type competition system (Du and Liang 2008), Holling II type predator–prey system (Du and Shi 2006), Leslie type predator–prey system (Du et al. 2009), as well as predator–prey systems with protection coefficients (Du and Shi 2007). Oeda studied the effects of a cross-diffusive Lotka–Voltera type predator–prey system with a protection zone (Oeda 2011). A cross-diffusive Lotka–Voltera type competition system with a protection zone was investigated by Wang and Li (2013). Zou and Wang studied an ODE model of protection zones, where the sizes of protection zones are reflected by restricting the functionals’ coefficient for the predator (Zou and Wang 2011). Recently, Cui et al. (2014) observed the strong Allee effect in a diffusive predator–prey system with protection zones.

In this paper, we study the steady states to the following diffusive predator–prey system with Beddington–DeAngelis type functional response

$$\begin{aligned} \left\{ \begin{array}{l@{\quad }l}\displaystyle u_t-d_1\Delta u=u\left( \theta -u-\frac{a(x)v}{1+mu+kv}\right) ,&{}x\in \Omega ,~~t>0,\\ \displaystyle v_t-d_2\Delta v=v\left( \mu -v+\frac{cu}{1+mu+kv}\right) ,&{} x\in \Omega _1,~~t>0,\\ \displaystyle \frac{\partial u}{\partial {n}}=0 ,&{} x\in \partial \Omega ,~~t>0,\\ \displaystyle \frac{\partial v}{\partial {n}}=0 ,&{} x\in \partial \Omega _1,~~t>0,\\ \displaystyle u(x,0)=u_0(x)\ge (\not \equiv ) 0,&{}x\in \Omega ,\\ \displaystyle v(x,0)=v_0(x)\ge (\not \equiv ) 0, &{} x\in \Omega _1, \end{array}\right. \end{aligned}$$

(1.1)

where $\Omega$ is a bounded domain in $\mathbb {R}^N$ $(N\le 3)$ with smooth boundary $\partial \Omega$, ${\Omega }_0\Subset \Omega$ with $\partial \Omega _0$ smooth, $\Omega _1=\Omega \backslash \overline{\Omega }_0$, constants $d_1,d_2,\theta ,c,m,k>0$, $\mu \in \mathbb {R}$, $\frac{\partial }{\partial n}$ is the outward normal derivative on the boundary, and

$$\begin{aligned} a(x)=\left\{ \begin{array}{llll} 0,&{}x\in \overline{\Omega }_0,\\ a,&{}x\in \Omega _1.\end{array}\right. \end{aligned}$$

(1.2)

The fact of $a(x)=0$ in ${\Omega }_0$ implies that no predation could take place there.

Equation (1.1) is a reaction–diffusion system of species u and v, and the dynamical behavior of species would be determined not only by the mechanism of the functional response between u and v, but also by the interaction between their reaction and diffusion. Here prey u and predator v disperse at rates $d_1$ and $d_2$, and grow at rates $\theta$ and $\mu$, respectively. The prey is consumed with the functional response of Beddington–DeAngelis type $\frac{a(x)uv}{1+mu+kv}$ in $\Omega$, and contributes to the predator with growth rate $\frac{cuv}{1+mu+kv}$ in $\Omega _1$. Non-flux boundary conditions mean that the habitat of the two species is closed. The B-D type functional response was introduced by Beddington (1975) and DeAngelis et al. (1975). Refer to Beddington (1975), DeAngelis et al. (1975) and Dimitrov and Kojouharov (2005) for the background of the original predator–prey model with B-D type functional response. Guo and Wu studied the existence, multiplicity, uniqueness and stability of the positive solutions under homogeneous Dirichlet boundary conditions in Guo and Wu (2010), as well as the effect of large k in Guo and Wu (2012). Chen and Wang established the existence of nonconstant positive steady-states under Neumann boundary conditions (Chen and Wang 2005; Pang and Wang 2003).

In particular, a mechanistic derivation for the B-D type functional response has been given by Geritz and Gyllenberg (2012) recently, where predators v were divided into searchers $v_S$ with attack rate a and handlers $v_H$ with handling time h, while preys u were structured into two classes: active preys $u_P$ and those prey individuals $u_R$ who have found a refuge with total refuge number b and sojourn time $\tau$. In these terms, the parameters in B-D type functional response of (1.1) can be understood as that $m=ah$ reflects the handling time of $v_H$, and $k=b\tau$ describes the refuge ability of the prey.

The prey’s refuge may come from its aggregation, reduction of its activity, or places where its predation risk is somehow reduced (Sih 1987). Dynamic consequences of prey refuges were observed by González-Olivares and Ramos-jiliberto (2003) with more prey, fewer predators and enhanced stability. On the other hand, refuges usually cost the prey in terms of reduced feeding or mating opportunities (Sih 1987), and hence their population could not be very large. In contrast, the protection zones, as refuges from humans, always benefit the endangered species. Refer to Haque et al. (2014), Ko and Ryu (2006), Mukherjee (2016), Sarwardi et al. (2012), Wang and Wang (2012), Wei and Fu (2016) and Yang and Zhang (2016) for more backgrounds on prey refuges and their affections. In this paper, we will show the effect of the prey’s refuge and the size of the protection zone have on the coexistence and stability of the predator–prey system with B-D type functional response. The results obtained here observe the general law that refuges and protection zones benefit the coexistence of species (González-Olivares and Ramos-jiliberto 2003; Sih 1987; Zou and Wang 2011).

Since the model (1.1) contains different coefficients a(x) and c in the B-D type functional response terms for u and v respectively, without loss of generality, suppose $d_1=d_2=1$ for simplicity. The steady-state problem corresponding to (1.1) takes the form

$$\begin{aligned} \left\{ \begin{array}{llll} \displaystyle -\Delta u=u\left( \theta -u-\frac{a(x)v}{1+mu+kv}\right) &{}\text{ in }~\Omega ,\\ \displaystyle -\Delta v=v\left( \mu -v+\frac{cu}{1+mu+kv}\right) &{}\text{ in }~\Omega _1,\\ \displaystyle \frac{\partial u}{\partial {n}}\Big |_{\partial \Omega }=0,~~ \frac{\partial v}{\partial {n}}\Big |_{\partial \Omega _1}=0.&{}\end{array}\right. \end{aligned}$$

(1.3)

Denote by $\lambda _1(q)$ the first eigenvalue of $-\Delta +q$ over $\Omega$ under homogeneous Neumann boundary conditions with $q=q(x)\in L^\infty (\Omega )$. The following properties of $\lambda _1(q)$ are well known:

• (i) $\lambda _1(0)=0$;

• (ii) $\lambda _1(q_1)>\lambda _1(q_2)$ if $q_1\ge q_2$ and $q_1\not \equiv q_2$;

• (iii) $\lambda _1(q)$ is continuous with respect to $q\in L^\infty (\Omega )$.

Define

$$\begin{aligned} \theta ^*(\mu ,\Omega _0)=\lambda _1(q(x)),~~\theta _0=\frac{a}{k},~~\theta _1(\Omega _0)=\lambda _1(q_0(x)), \end{aligned}$$

(1.4)

with

$$\begin{aligned} q(x)=\frac{a(x)\mu }{1+k\mu },~~q_0(x)= \left\{ \begin{array}{lll}0, &{}x\in \overline{\Omega }_0,\\ \theta _0, &{}x\in \Omega _1. \end{array}\right. \end{aligned}$$

(1.5)

Denote by $U_{\theta ,q_0}$ the solution of the scalar problem

$$\begin{aligned} -\Delta u=u(\theta -u-q_0(x))~\mathrm{in}~\Omega ,~~\frac{\partial u}{\partial {n}}=0~\mathrm{on}~\partial \Omega . \end{aligned}$$

(1.6)

Due to $\theta _1=\inf \limits _{\phi \in H^1(\Omega ),\int _{\Omega }\phi ^2dx>0}\frac{\int _{\Omega }|\nabla \phi |^2dx+\frac{a}{k}\int _{\Omega _1}\phi ^2dx}{\int _{\Omega }\phi ^2dx}$, the properties (i)–(iii) of $\lambda _1(q)$ imply the following lemma:

Lemma 1.1

$\theta ^*{(\mu ,\Omega _0)}$ is strictly increasing with respect to $\mu$ and decreasing when $\Omega _0$ enlarging, $\theta ^*(0,\Omega _0)=0$, $\theta ^*(\mu ,\Omega _0)<\theta _0$, $\lim _{\mu \rightarrow \infty }\theta ^*(\mu ,\Omega _0)=\theta _1(\Omega _0)\le \frac{a|\Omega _1|}{k|\Omega |}$, $\lim _{|\Omega _1|\rightarrow 0}\theta ^*(\mu ,\Omega _0)=0$, $\lim _{|\Omega _0|\rightarrow 0}\theta ^*(\mu ,\Omega _0)=\frac{a\mu }{1+k\mu }$. $\Box$

Biologically, we are interested in the positivity of the prey u in the diffusive predator–prey model (1.3). We state the main results of the paper one by one as follows.

Obviously, either large $\theta$ or small $\mu$ benefits the prey u. In the first theorem, we give two sufficient conditions for keeping the prey positive without protection zones.

Theorem 1

If $\theta \ge \theta _0$ or $\mu \le 0$, then the positivity of u would be ensured automatically without any protections zones.

The next theorem implies that a suitable protection zone guarantees the existence of positive solutions to (1.3) under $\theta <\theta _0$ with $\mu >0$.

Theorem 2

Suppose $\mu > 0$. If $\theta ^*(\mu ,\Omega _0)< \theta < \theta _0$, then Eq. (1.3) has at least one positive solution. Furthermore, if $\theta \le \theta ^*(\mu ,\Omega _0)$ with $m\le \frac{(k\mu +1)^2}{a\mu }$, then Eq. (1.3) has no positive solutions.

In the third theorem, we give a necessary and sufficient condition for the coexistence of u and v under $\mu \in (-\frac{c}{m},0]$.

Theorem 3

Suppose $-\frac{c}{m}<\mu \le 0$. Then Eq. (1.3) has at least one positive solution if and only if $\theta >\theta _*=-\frac{\mu }{c+m\mu }=\frac{|\mu |}{c-m|\mu |}\ge 0$.

Remark 1

Since $\lim _{|\Omega _1|\rightarrow 0}\theta ^*(\mu ,\Omega _0)=0$ by Lemma 1.1, for any $\theta >0$ and $\mu \ge 0$ fixed, the key condition $\theta >\theta ^*(\mu ,\Omega _0)$ in Theorem 2 can be realized by enlarging the size of the protection zone $\Omega _0=\Omega {\setminus }\overline{\Omega }_1$. So does the condition $\theta >\theta _1$ in the following Theorem 4.

Remark 2

Theorem 1 shows that no protection zones are necessary for the positivity of u if $\mu \le 0$. It is known by Theorem 3 that in addition to the positivity of u, the positivity of v can be ensured also if the death rate of the predator v is not too high with $\mu \in (-\frac{c}{m},0]\subset (-\infty ,0]$ and the birth rate of the prey u is properly large such that $\theta >\theta _*$.

Finally, the last theorem says the positive solutions of (1.3) are in fact unique if $\theta$ is even larger than $\theta _1$ under large $\mu$, and determines the asymptotic behavior of the unique positive solution as $\mu \rightarrow \infty$. In fact, from Lemma 1.1 and Theorem 2 that if $\theta >\theta _1$, prey species could be alive no matter how large the predator’s growth rate is.

Theorem 4

If $\theta >\theta _1(\Omega _0)$, then there exists $\mu ^*>0$ such that the positive solution of (1.3) is unique and linearly stable when $\mu \ge \mu ^*$. Furthermore, the unique positive solution satisfies $(u,v-\mu )\rightarrow (U_{\theta ,q_0},0)$ uniformly on $\overline{\Omega }$ and $\overline{\Omega }_1$, respectively, as $\mu \rightarrow \infty$.

This paper is arranged as follows. In the next two sections, we prove Theorems 1–3 and Theorem 4, respectively. The last section is devoted to a discussion of the obtained results, by analyzing them with the mechanistic derivation for the B-D type functional response in Geritz and Gyllenberg (2012).

2 Existence of positive solutions

At first we deal with the proof of Theorem 1.

Proof of Theorem 1

Assume $\mu \le -\frac{c}{m}$. Integrate the second equation of (1.3) over $\Omega _1$ to get

$$\begin{aligned} 0=\int _{\Omega _1}v\left( \mu -v+\frac{cu}{1+mu+kv}\right) {d}x, \end{aligned}$$

and hence

$$\begin{aligned} 0\le \int _{\Omega _1}v^2{d}x=\int _{\Omega _1}v\left( \mu +\frac{cu}{1+mu+kv}\right) {d}x\le \left( \mu +\frac{c}{m}\right) \int _{\Omega _1}v{d}x\le 0. \end{aligned}$$

This concludes $v\equiv 0$, and so u satisfies

$$\begin{aligned} -\Delta u=u(\theta -u)\quad \text{ in }\quad \Omega ,\quad \frac{\partial u}{\partial n}=0\quad \mathrm{on}\quad \partial \Omega . \end{aligned}$$

(2.1)

Obviously, (2.1) admits the solution $u=\theta >0$.

The desired result for $-\frac{c}{m}<\mu \le 0$ is substantially concluded from Theorem 3. Indeed, the subcase of $\theta >\theta _*$ is covered by Theorem 3, while for $\theta \le \theta _*$, it can be found in the proof of Theorem 3 that $v\equiv 0$, and so $u=\theta >0$.

Next consider the first equation of (1.3) with $\theta \ge \theta _0$. It is easy to know that $\frac{a(x)v}{1+mu+kv}<\frac{a(x)}{k}\le \frac{a}{k}=\theta _0$ for $v\ge 0$. Thus, for any $v(x)\ge 0$, there is $\tilde{\theta }_0\in (0,\theta _0)$ such that

$$\begin{aligned} -\Delta u=u(\theta -u-\frac{a(x)v}{1+mu+kv})> u(\theta -\tilde{\theta }_0-u)\quad \text{ in }\quad \Omega ,\quad \frac{\partial u}{\partial n}=0\quad \mathrm{on}\quad \partial \Omega . \end{aligned}$$

This ensures that $u\ge \theta -\tilde{\theta }_0>0$. $\square$

We need some preliminaries represented as lemmas and propositions for the proof of Theorem 2, and begin with two known results on the maximum principle and the Harnack inequality.

Lemma 2.1

(Maximum Principle Lou and Ni 1996) Let $g\in C(\overline{\Omega }\times \mathbb {R})$, $w\in C^2(\Omega )\bigcap C^1(\overline{\Omega })$, where $\Omega$ is a bounded domain in $\mathbb {R}^N$ with smooth boundary.

• (a) If  $\Delta w+g(x,w)\le 0$ in $\Omega$, $\frac{\partial w}{\partial {n}}\ge 0$ on $\partial \Omega$ and $\min _{\overline{\Omega }} w=w(x_0)$, then $g(x_0,w(x_0))\le 0$.

• (b) If  $\Delta w+g(x,w)\ge 0$ in $\Omega$, $\frac{\partial w}{\partial {n}}\le 0$ on $\partial \Omega$ and $\max _{\overline{\Omega }} w=w(x_0)$, then $g(x_0,w(x_0))\ge 0$.$\square$

Lemma 2.2

(Harnack Inequality Lou and Ni 1999; Lin et al. 1988) Let $f\in L^p(\Omega )$ with $p>\max \{\frac{N}{2},1\}$, and w be a non-negative solution of $\Delta w+f(x)w=0$ in a bounded domain $\Omega \subset \mathbb {R}^N$ with smooth boundary under homogeneous Neumann boundary condition. Then there exists a positive constant $C=C(p,N,\Omega ,\Vert f\Vert _{L^p(\Omega )})$ such that

$$\begin{aligned} \max \limits _{\overline{\Omega }}w\le C\min \limits _{\overline{\Omega }}w. \end{aligned}$$

$\square$

The following a priori estimates are easy to get.

Lemma 2.3

Let (u, v) be a nontrivial non-negative solution of (1.3). Then

$$\begin{aligned} 0<u\le \theta ,\quad \mu _+<v\le \mu _++\frac{c\theta }{1+m\theta +k\mu _+},\quad \Vert u\Vert _{C^{1,\alpha }(\overline{\Omega })}+\Vert v\Vert _{C^{1,\alpha }(\overline{\Omega }_1)}\le C,\end{aligned}$$

with $\mu _+=\max \{\mu ,0\}$, $\alpha \in (0,1)$, $C=C(\theta ,\mu ,\Omega _0)>0$.

Proof

Suppose $u(x_0)=\max _{\overline{\Omega }}u(x)>0$. By Lemma 2.1(b), we have

$$\begin{aligned} u(x_0)\left( \theta -u(x_0)-\frac{a(x_0)v(x_0)}{1+mu(x_0)+kv(x_0)}\right) \ge 0, \end{aligned}$$

and then

$$\begin{aligned} u(x_0)\le \theta -\frac{a(x_0)v(x_0)}{1+mu(x_0)+kv(x_0)}\le \theta . \end{aligned}$$

Due to Lemma 2.2, we arrive at $0<u\le \theta$ on $\overline{\Omega }$. Similarly, we can show $\mu _+<v\le \mu _++\frac{c\theta }{1+m\theta +k\mu _+}$ on ${\overline{\Omega }}_1$.

The $C^{1,\alpha }$ boundedness of solutions comes from the elliptic regularity theory together with the Sobolev embedding theorem. $\square$

We will use the local bifurcation theorem of Crandall and Rabinowitz (1971) and the global bifurcation theorem of Rabinowitz (1971) to prove Theorem 2.

Denote the semitrivial solution curves by

$$\begin{aligned} \Gamma _u=\{(\theta ,u,v)=(\theta ,0,\mu );\,\theta>0\}, \quad \Gamma _v=\{(\theta ,u,v)=(\theta ,\theta ,0);\,\theta >0\}. \end{aligned}$$

Define

$$\begin{aligned} X= & {} W^{2,p}_n(\Omega )\times W^{2,p}_n(\Omega _1),~Y=L^p(\Omega )\times L^p(\Omega _1)~\mathrm{with}~p>N,\\ Z= & {} C_n^1(\overline{\Omega })\times C_n^1(\overline{\Omega }_1), \end{aligned}$$

where

$$\begin{aligned} W^{2,p}_n(\Omega )= & {} \left\{ w\in W^{2,p}(\Omega );~\frac{\partial w}{\partial n}=0~\mathrm{on}~\partial \Omega \right\} ,\\ C_n^1(\overline{\Omega })= & {} \left\{ w\in C^1(\overline{\Omega });\frac{\partial w}{\partial n}=0~\mathrm{on}~\partial \Omega \right\} . \end{aligned}$$

The Sobolev embedding theorem implies $X\subseteq Z$.

Let $(\phi ^*,\psi ^*)$ solve

$$\begin{aligned}&\Delta \phi ^*+(\theta ^*-\frac{a(x)\mu }{1+k\mu })\phi ^*=0~\mathrm{in}~\Omega ,~\frac{\partial \phi ^*}{\partial {n}}=0~\mathrm{on}~\partial \Omega ,\\&\Delta \psi ^*-\mu \psi ^*+\frac{c\mu }{1+k\mu }\phi ^*=0~\mathrm{in}~\Omega _1,~\frac{\partial \psi ^*}{\partial {n}}=0~\mathrm{on}~\partial \Omega _1. \end{aligned}$$

Then $\psi ^*=(-\Delta +\mu I)^{-1}_{\Omega _1}\frac{c\mu }{1+k\mu }\phi ^*$.

Proposition 2.1

Let $\mu >0$. Then there are positive solutions of (1.3) bifurcating from $\Gamma _u$ if and only if $\theta >\theta ^*(\mu ,\Omega _0)$, possessing the form

$$\begin{aligned} \ \Gamma _1=\{(\theta ,u,v)=(\theta (s),s\phi ^*+o(|s|),\mu +s\psi ^*+o(|s|));\quad s\in (0,\sigma )\}\quad \end{aligned}$$

(2.2)

with $(\theta (0),u(0),v(0))=(\theta ^*,0,\mu )$ for some $\sigma >0$ in a neighborhood of $(\theta ^*,0,\mu )\in \mathbb {R}\times X$.

Proof

Denote by $V=v-\mu$,

$$\begin{aligned} F(\theta ,u,V)= & {} \left( \begin{array}{clcr} \ \Delta u+f_1(\theta ,u,V+\mu )\\ \Delta V+f_2(\mu ,u,V+\mu )\end{array}\right) ^T\quad \mathrm{and}\nonumber \\ F_1(\theta ,u,v)= & {} \left( \begin{array}{clcr} \ \Delta u+f_1(\theta ,u,v) \\ \Delta v+f_2(\mu ,u,v) \end{array}\right) ^T\end{aligned}$$

(2.3)

with

$$\begin{aligned} f_1(\theta ,u,v)=u(\theta -u-\frac{a(x)v}{1+mu+kv}),\quad f_2(\mu ,u,v)=v(\mu -v+\frac{cu}{1+mu+kv}). \end{aligned}$$

Obviously, $F(\theta ,u,V)=0$ is equivalent to $F_1(\theta ,u,v)=0$, and $F_1(\theta ,0,\mu )=F(\theta ,0,0)=0$ for $\theta \in \mathbb {R}$. A direct calculation yields

$$\begin{aligned} \ F_{(u,V)}(\theta ,0,0)[\phi ,\psi ]=\Big (\begin{array}{clcr}\ \Delta \phi +(\theta -\frac{a(x)\mu }{1+k\mu })\phi \\ \Delta \psi -\mu \psi +\frac{c\mu }{1+k\mu }\phi \end{array}\Big )^T. \end{aligned}$$

(2.4)

By the Krein–Rutman theorem, $F_{(u,V)}(\theta ,0,0)[\phi ,\psi ]=(0,0)$ has a solution $\phi >0$ if and only if $\theta =\theta ^*$. So $(\theta ^*,0,\mu )$ is the only possible bifurcation point from which positive solutions of (1.3) bifurcate from $\Gamma _u$. Besides, we have

$$\begin{aligned} \mathrm{Ker} F_{(u,V)}(\theta ^*,0,0)=\mathrm{Span}\,\{(\phi ^*,\psi ^*)\},\quad \text{ dim } \text{ Ker }\, F_{(u,V)}(\theta ^*,0,0)=1. \end{aligned}$$

For $(\bar{\phi },\bar{\psi })\in Y\cap \text{ Range }\,F_{(u,V)}(\theta ^*,0,0)$, choose $(\phi ,\psi )\in X$ such that

$$\begin{aligned} \left\{ \begin{array}{llll} \displaystyle \Delta \phi +(\theta -\frac{a(x)\mu }{1+k\mu })\phi =\bar{\phi },\\ \displaystyle \Delta \psi -\mu \psi +\frac{c\mu }{1+k\mu }\phi =\bar{\psi }. \end{array} \right. \end{aligned}$$

(2.5)

Multiplying by $\phi ^*$ on both sides of the first equation of (2.5) and integrating by parts over $\Omega$, we get $\int _\Omega \bar{\phi }\phi ^* dx=0$. Then

$$\begin{aligned} \text{ Range }\,F_{(u,V)}(\theta ^*,0,0)=\Big \{(\bar{\phi },\bar{\psi })\in Y;\int _\Omega \bar{\phi }\phi ^* dx=0\Big \}, \end{aligned}$$

(2.6)

and thus

$$\begin{aligned} \text{ codim } \text{ Range }\,F_{(u,V)}(\theta ^*,0,0)=1. \end{aligned}$$

By a simple calculation,

$$\begin{aligned}&F_\theta (\theta ^*,0,0)=F_{\theta \theta }(\theta ^*,0,0)=(0,0),\\ {}&F_{\theta (u,V)}(\theta ^*,0,0)[\phi ^*,\psi ^*]= (\phi ^*,0) \notin \text{ Range } F_{(u,V)}(\theta ^*,0,0). \end{aligned}$$

In conclusion, the proposition is proved by the local bifurcation theorem (Crandall and Rabinowitz 1971). $\square$

Proposition 2.2

Let $-\frac{c}{m}<\mu <0$. Then there are positive solutions of (1.3) bifurcating from $\Gamma _v$ if and only if $\theta >\theta _*=-\frac{\mu }{c+m\mu }$, having the form

$$\begin{aligned} \Gamma _2=\{(\theta ,u,v)=(\tilde{\theta }(s),\theta +s\phi _*(x)+o(|s|),s+o(|s|));s\in (0,\tilde{\sigma })\} \end{aligned}$$

(2.7)

with $\tilde{\theta }(0)=-\frac{\mu }{c+m\mu }$, $\phi _*=(\Delta -\theta I)^{-1}\frac{a(x)\theta }{1+m\theta }$ for some $\tilde{\sigma }>0$ near $(\theta ,\theta ,0)\in \mathbb {R}\times X$.

Proof

Let $w=u-\theta$,

$$\begin{aligned} G(\theta ,w,v)=\left( \begin{array}{clcr}\ \Delta w+(w+\theta )(-w-\frac{a(x)v}{1+m(w+\theta )+kv})\\ \Delta v+v(\mu -v+\frac{c(w+\theta )}{1+m(w+\theta )+kv}) \end{array}\right) ^T, \end{aligned}$$

(2.8)

Then $F_1(\theta ,u,v)=0$ is equivalent to $G(\theta ,w,v)=0$. We have

$$\begin{aligned}&G_{(w,v)}(\theta ,w,v)[\phi ,\psi ]\\&\quad =\left( \begin{array}{clcr}\ \Delta \phi -(2w+\theta )\phi -\frac{a(x)v}{1+m(w+\theta )+kv}\phi +\frac{a(x)mv(w+\theta )}{(1+m(w+\theta )+kv)^2}\phi -\frac{a(x)(w+\theta )(1+m(w+\theta ))}{(1+m(w+\theta )+kv)^2}\psi \\ \Delta \psi +(\mu -2v)\psi +\frac{c(w+\theta )}{1+m(w+\theta )+kv}\psi -\frac{ckv(w+\theta )}{(1+m(w+\theta )+kv)^2}\psi +\frac{cv(1+kv)}{(1+m(w+\theta )+kv)^2}\phi \end{array}\right) ^T, \\&G_\theta (\theta ,w,v)= \Big (\begin{array}{clcr} -w-\frac{a(x)v}{1+m(w+\theta )+kv}+\frac{a(x)mv(w+\theta )}{(1+m(w+\theta )+kv)^2}\\ \frac{cv(1+kv)}{(1+m(w+\theta )+kv)^2} \end{array}\Big )^T. \end{aligned}$$

The equation $G_{(w,v)}(\theta ,0,0)[\phi ,\psi ]=(0,0)$ is equivalent to

$$\begin{aligned} \left\{ \begin{array}{ll} \Delta \phi -\theta \phi -\frac{a(x)\theta }{1+m\theta }\psi =0&{}\mathrm{in}~\Omega ,\\ \Delta \psi +\mu \psi +\frac{c\theta }{1+m\theta }\psi =0&{}\mathrm{in}~\Omega _1,\\ \frac{\partial \phi }{\partial {n}}=0~\mathrm{on}~\partial \Omega ,~~\frac{\partial \psi }{\partial {n}}=0~&{}\mathrm{on}~\partial \Omega _1. \end{array}\right. \end{aligned}$$

(2.9)

The second equation of (2.9) has a solution $\psi >0$ if and only if $\mu =-\frac{c\theta }{1+m\theta }$, i.e. $\theta =-\frac{\mu }{c+m\mu }=\theta _*$. Thus $(\theta _*,\theta ,0)$ is the only possible bifurcation point along $\Gamma _v$, and $\phi _*$ solves the first equation of (2.9) with $\theta =\theta _*$ and $\psi \equiv 1$. It is easy to verify that

$$\begin{aligned} \text{ Ker }\,G_{(w,v)}(\theta _*,0,0)= \text{ Span } \{(\phi _*,1)\},~~ \text{ dim } \text{ Ker } G_{(w,v)}(\theta ,0,0)=1. \end{aligned}$$

A direct calculation shows

$$\begin{aligned}&G_\theta (\theta _*,0,0)=G_{\theta \theta }(\theta _*,0,0)=(0,0),\\&\text{ Range } G_{(w,v)}(\theta _*,0,0)=\left\{ (f,g)\in Y;\int _\Omega gdx=0\right\} ,\\&\quad \text{ codim } \text{ Range } G_{(w,v)}(\theta _*,0,0)=1,\\&G_{\theta (w,v)}(\theta _*,0,0)[\phi _*,1]=\left( -\phi _*-\frac{a(x)}{(1+m\theta )^2}, \frac{c}{(1+m\theta )^2}\right) \notin \text{ Range } G_{(w,v)}(\theta _*,0,0). \end{aligned}$$

By the local bifurcation theorem (Crandall and Rabinowitz 1971), we get the desired results of the Proposition 2.2. $\square$

In order to use the global bifurcation theorem for $\mu >0$, define $F_2:\mathbb {R}\times Z\rightarrow Z$ by

$$\begin{aligned} \ F_2(\theta ,u,v)= \left( \begin{array}{clcr}\ u \\ v-\mu \end{array}\ \right) ^T-\left( \begin{array}{clcr}\ (-\Delta +I)^{-1}_\Omega (u+f_1(\theta ,u,v))\\ (-\Delta +I)^{-1}_{\Omega _1}(v-\mu +f_2(\mu ,u,v)) \end{array}\ \right) ^T.\qquad \end{aligned}$$

(2.10)

Then (1.3) is equivalent to $F_2(\theta ,u,v)=0$. Let $\tilde{\Gamma }_1\subset \mathbb {R}\times Z$ be the maximal connected set satisfying

$$\begin{aligned} \Gamma _1\subset \tilde{\Gamma }_1\subset \{(\theta ,u,v)\in \mathbb {R}\times Z\backslash \{(\theta ^*,0,\mu )\};F_2(\theta ,u,v)=(0,0)\}. \end{aligned}$$

From the global bifurcation theory of Rabinowitz (1971), one of the following non-excluding results must be true (see Theorem 6.4.3 in López-Gómez 2001):

• (a) $\tilde{\Gamma }_1$ is unbounded in $\mathbb {R}\times Z$.

• (b) There exists a constant $\bar{\theta }\ne \theta ^*$ such that $(\bar{\theta },0,\mu )\in \tilde{\Gamma }_1$.

• (c) There exists $(\tilde{\theta },\tilde{\phi },\tilde{\psi })\in \mathbb {R}\times (Y_1\backslash \{(0,\mu )\})$ with $Y_1=\{(\bar{\phi },\bar{\psi })\in Z;\int _\Omega \bar{\phi }\phi ^*dx=0\}$ such that $(\tilde{\theta },\tilde{\phi },\tilde{\psi })\in \tilde{\Gamma }_1$.

Now we give the proofs of Theorems 2 and 3.

Proof of Theorem 2

At first we know that $u,v>0$ for any $(\theta ,u,v)\in \tilde{\Gamma }_1$ which means that the case (c) above cannot occur by $\phi ^*>0$. Otherwise there is a $(\bar{\theta },\bar{u},\bar{v})\in \tilde{\Gamma }_1$ such that (1) $\bar{u}>0$ with $\bar{v}(x_0)=0$ for some $x_0\in \Omega _1$, or (2) $u(x_1)=v(x_2)=0$ for some $x_1\in \Omega$ and $x_2\in \Omega _1$, or (3) $\bar{v}>0$ with $\bar{u}(x_3)=0$ for some $x_3\in \Omega$. Denote by $\mathscr {B}_\Omega =\{\phi \in C^1_n(\overline{\Omega });\,\phi >0~\text{ on }~\overline{\Omega }\}$. Choose a sequence $\{(\theta _i,u_i,v_i)\}_{i=1}^\infty \subset \tilde{\Gamma }_1\cap (\mathbb {R}\times \mathscr {B}_\Omega \times \mathscr {B}_{\Omega _1})$ such that $\lim \limits _{i\rightarrow \infty }(\theta _i,u_i,v_i)=(\bar{\theta },\bar{u},\bar{v})$ in $\mathbb {R}\times Z$, where $\bar{\theta }$ can be $\infty$. Obviously, $(\bar{u},\bar{v})$ is a non-negative solution of (1.3) with $\theta =\bar{\theta }$. By Lemma 2.2, one of the following must hold:

$$\begin{aligned} \text{(1) }~\bar{u}>0,\bar{v}\equiv 0;~~ \text{(2) }~\bar{u}\equiv 0, \bar{v}\equiv 0; ~~\text{(3) }~ \bar{u}\equiv 0,\bar{v}>0. \end{aligned}$$

For (3), we have $-\Delta \bar{v}=\bar{v}(\mu -\bar{v})$ in $\Omega _1$, $\frac{\partial \bar{v}}{\partial {n}}=0$ on $\partial \Omega _1$, and thus $\bar{v}\equiv \mu$. By Proposition 2.1, this implies $\bar{\theta }=\theta ^*$, a contradiction to the definition of $\tilde{\Gamma }_1$.

Suppose (1) or (2) is true. Integrate the second equation of (1.3) on $\Omega _1$ with $(u,v)=({u}_i,{v}_i)$ to obtain

$$\begin{aligned} \int _{\Omega _1}v_i\left( \mu -v_i+\frac{cu_i}{1+mu_i+kv_i}\right) dx=0, \quad i\in \mathbb {N}. \end{aligned}$$

On the other hand, $\mu >0$ and $\bar{v}\equiv 0$ ensure $\mu -v_i>0$, and thus $\mu -v_i+\frac{cu_i}{1+mu_i+kv_i}>0$ for i large enough, also a contradiction.

The case (b) is excluded by Proposition 2.1. So, the only true case is (a).

From Lemma 2.3, (u, v) are uniformly bounded in Z as $(\theta ,u,v)\in \tilde{\Gamma }_1$ which shows that $\theta$ is unbounded. Combining this with Proposition 2.1, we know that (1.3) has at least one positive solution for $\theta >\theta ^*(\mu ,\Omega _0)$ with $\mu >0$.

Now, let (u, v) be a positive solution of (1.3) with $m\le \frac{(1+k\mu )^2}{a\mu }$. A direct calculation yields that $u+\frac{a(x)v}{1+mu+kv}>\frac{a(x)\mu }{1+k\mu }$. By the monotonicity of the eigenvalue, we conclude that

$$\begin{aligned} 0=\lambda _1\left( -\theta +u+\frac{a(x)v}{1+mu+kv}\right) >\lambda _1\left( -\theta +\frac{a(x)\mu }{1+k\mu }\right) . \end{aligned}$$

Then

$$\begin{aligned} \theta >\lambda _1\left( \frac{a(x)\mu }{1+k\mu }\right) =\theta ^*(\mu ,\Omega _0). \end{aligned}$$

This shows that (1.3) has no positive solution whenever $\theta \le \theta ^*(\mu ,\Omega _0)$ and $m\le \frac{(1+k\mu )^2}{a\mu }$. $\square$

Proof of Theorem 3

When $\mu =0$, fix $\theta >0$. By Lemma 1.1 and Theorem 2, we can take a sequence $\{(\mu _i,u_i,v_i)\}_{i=1}^\infty$ such that $(u_i,v_i)$ is a positive solution of (1.3) with $\mu =\mu _i>0$, $\lim _{i\rightarrow \infty }\mu _i=0$. By Lemma 2.3 and embedding theorem, we can choose a subsequence (still denoted by $\{(\mu _i,u_i,v_i)\}_{i=1}^\infty$) such that $(u_i,v_i)$ converges to $(\tilde{u},\tilde{v})\in Z$, a non-negative solution of (1.3). By Lemma 2.2, $\tilde{u}>0$ or $\tilde{u}\equiv 0$ in $\Omega$; $\tilde{v}>0$ or $\tilde{v}\equiv 0$ in $\Omega _1$.

If $\tilde{u}\equiv 0$ and $\tilde{v}>0$, then $\mu _i-v_i+\frac{cu_i}{1+mu_i+kv_i}<0$ in $\Omega _1$ for i large enough. This contradicts $\int _{\Omega _1}v_i(\mu _i-v_i+\frac{cu_i}{1+mu_i+kv_i})dx=0$.

If $\tilde{u}>0$ and $\tilde{v}\equiv 0$, then $\mu _i-v_i+\frac{cu_i}{1+mu_i+kv_i}>0$ in $\Omega _1$ for i large enough, also a contradiction with $\int _{\Omega _1}v_i(\mu _i-v_i+\frac{cu_i}{1+mu_i+kv_i})dx=0$.

If $\tilde{u}\equiv 0$ and $\tilde{v}\equiv 0$, then $\theta -u_i+\frac{a(x)v_i}{1+mu_i+kv_i}>0$ in $\Omega$ for i large enough, a contradiction to $\int _{\Omega }u_i(\theta -u_i+\frac{a(x)v_i}{1+mu_i+kv_i})dx=0$.

In summary, we must have $\tilde{u},\tilde{v}>0$ in $\Omega$ and $\Omega _1$, respectively. This means that (1.3) possesses positive solutions for all $\theta >0$ if $\mu =0$.

Now, suppose $-\frac{c}{m}<\mu <0$. For $\theta>-\frac{\mu }{c+m\mu }>0$, the existence of positive solutions can be obtained from Proposition 2.2 by a similar global bifurcation analysis of $\Gamma _u$ as that for the branch $\Gamma _v$ with $\mu >0$. We omit the details.

Conversely, let (u, v) be a positive solution of (1.3) with $\mu \in (-\frac{c}{m},0]$. Then $0<u\le \theta$ by Lemma 2.3, and hence

$$\begin{aligned} \mu =\lambda _1\left( v-\frac{cu}{1+mu+kv}\right) >\lambda _1\left( -\frac{cu}{1+mu}\right) \ge \lambda _1\left( -\frac{c\theta }{1+m\theta }\right) =-\frac{c\theta }{1+m\theta }, \end{aligned}$$

namely, $\theta >-\frac{\mu }{c+m\mu }$. $\square$

3 Uniqueness of positive solutions

In this section, we use topological degree to prove Theorem 4 for $\theta >\theta _1$ and large $\mu$. At first, introduce an auxiliary problem

$$\begin{aligned} \left\{ \begin{array}{llll} \displaystyle -\Delta u=u\left( \theta -u-\frac{a(x)v}{1+mu+kv}\right) &{}\text{ in }~\Omega ,\\ \displaystyle -\Delta v=v\left( \mu -v+t\frac{cu}{1+mu+kv}\right) &{}\text{ in }~\Omega _1,\\ \displaystyle \frac{\partial u}{\partial {n}}\Big |_{\partial \Omega }=0,\quad \frac{\partial v}{\partial {n}}\Big |_{\partial \Omega _1}=0 \end{array}\right. \end{aligned}$$

(3.1)

with the parameter $t\in [0,1]$. Eq. (3.1) reverts back to (1.3) if $t=1$. When $t=0$, we have from the second equation of (3.1) that $v\equiv \mu$, and then obtain the scalar problem

$$\begin{aligned} \left\{ \begin{array}{llll} \displaystyle -\Delta u=u\left( \theta -u-\frac{a(x)\mu }{1+mu+k\mu }\right) &{}\mathrm{in}~\Omega ,\\ \displaystyle \frac{\partial u}{\partial {n}}=0&{}\mathrm{on}~\partial \Omega , \end{array}\right. \end{aligned}$$

(3.2)

which yields Eq. (1.6) as $\mu \rightarrow \infty$.

Lemma 3.1

Problem (1.6) has a unique positive solution if and only if $\theta >\theta _1$.

Proof

Suppose $\theta >\theta _1$. Let $\phi >0$ be the normalized eigenfunction with respect to $\theta _1$. Set $\underline{u}=\epsilon \phi$. Then

$$\begin{aligned} -\Delta \underline{u}=-\epsilon \Delta \phi =\epsilon (\theta _1-q_0(x))\phi =\epsilon \phi (\theta -q_0(x)-\epsilon \phi )+\epsilon \phi (\theta _1-\theta +\epsilon \phi ). \end{aligned}$$

Choose $\epsilon$ small enough such that $\theta _1-\theta +\epsilon \phi <0$ to get

$$\begin{aligned} -\Delta \underline{u}\le \underline{u}(\theta -q_0(x)-\underline{u})~\mathrm{in}~\Omega , ~~\frac{\partial \underline{u}}{\partial n}=0~\mathrm{on}~\partial \Omega . \end{aligned}$$

Obviously, $\underline{u}=\epsilon \phi$ and $\overline{u}=\theta$ are a pair of positive sub- and supersolutions of (1.6) with $\underline{u}\le \overline{u}$. We can get a positive solution of Eq. (1.6) by the sub-supersolution method. Let $\tilde{u}$ and $\hat{u}$ be the minimal and maximal positive solutions to (1.6), respectively. Since

$$\begin{aligned} \int _{\Omega }\nabla \tilde{u}\cdot \nabla \hat{u}dx=\int _{\Omega }\tilde{u}\hat{u}(\theta -\tilde{u}-q_0(x))dx= \int _{\Omega }\tilde{u}\hat{u}(\theta -\hat{u}-q_0(x))dx, \end{aligned}$$

we conclude

$$\begin{aligned} \int _{\Omega }\tilde{u}\hat{u}(\tilde{u}-\hat{u})dx=0. \end{aligned}$$

Therefore $\tilde{u}\equiv \hat{u}$.

On the other hand, it is obviously true for any positive solution $u_1$ of (1.6) that $\theta =\lambda _1(u_1+q_0(x))>\lambda _1(q_0(x))=\theta _1$. $\square$

Next, we show the uniqueness of positive solutions to (3.2).

Proposition 3.1

Suppose $\theta >\theta _1$. There is a $\tilde{\mu }=\tilde{\mu }(\theta )>0$ such that for any $\mu >\tilde{\mu }$, problem (3.2) has an unique positive solution.

Proof

Since $-q_0(x)<-\frac{a(x)\mu }{1+mU_{\theta ,q_0}+k\mu }$, then $U_{\theta ,q_0}$ is a subsolution of (3.2). Obviously, $\theta$ is a supersolution of (3.2) and $U_{\theta ,q_0}\le \theta$. Then there exist positive solutions to (3.2).

To prove the uniqueness of the positive solutions to (3.2), we at first show that the positive solutions of (3.2) are linearly stable for large $\mu$. Let U be a positive solution of (3.2). Consider the eigenvalue problem

$$\begin{aligned} -\Delta \phi =\theta \phi -2U\phi -\frac{a(x)\mu (1+k\mu )}{(1+mU+k\mu )^2}\phi +\eta \phi ~\mathrm{in}~\Omega , ~~\frac{\partial \phi }{\partial {n}}=0~\mathrm{on}~\partial \Omega \qquad \end{aligned}$$

(3.3)

with the principal eigenvalues denoted by

$$\begin{aligned} \eta =\eta (\mu )=\inf _{\phi \in H^{1}(\Omega ),\,\Vert \phi \Vert _2=1}\int _{\Omega }[|\nabla \phi |^2-\theta \phi ^2+2U\phi ^2+\frac{a(x)\mu (1+k\mu )}{(1+mU+k\mu )^2}\phi ^2]dx.\nonumber \\ \end{aligned}$$

(3.4)

We have

$$\begin{aligned} 0=\lambda _1\left( -\theta +2U+\frac{a(x)\mu (1+k\mu )}{(1+mU+k\mu )^2}-\eta \right) >\lambda _1(-\theta -\eta )=-\theta -\eta , \end{aligned}$$

i.e., $\eta >-\theta$. Denote by $\eta ^*$ the principal eigenvalue of the problem

$$\begin{aligned} -\Delta \phi =\theta \phi -2U_{\theta ,q_0}(x)\phi -q_0(x)\phi +\eta ^*\phi ~\mathrm{in}~\Omega ,~~ \frac{\partial \phi }{\partial {n}}=0~\mathrm{on}~\partial \Omega \end{aligned}$$

(3.5)

with the normalized eigenfunction $\phi ^*>0$. Then $\eta ^*=\frac{\int _\Omega U_{\theta ,q_0}^2\phi ^* dx}{\int _\Omega U_{\theta ,q_0}\phi ^* dx}>0$. Due to $\frac{a(x)\mu }{1+mu+k\mu }\rightarrow q_0(x)$ uniformly on $\overline{\Omega }$ as $\mu \rightarrow \infty$, we know $U\rightarrow U_{\theta ,q_0}(x)$ uniformly on $\overline{\Omega }$. It follows from (3.4) that

$$\begin{aligned} \eta&\le \int _{\Omega }\left[ |\nabla \phi ^*|^2-\theta {\phi ^*}^2+2U{\phi ^*}^2+\frac{a(x)\mu (1+k\mu )}{(1+mU+k\mu )^2}{\phi ^*}^2\right] dx \nonumber \\&=\eta ^*+\int _\Omega \left[ 2(U-U_{\theta ,q_0})+\frac{a(x)\mu (1+k\mu )}{(1+mu+k\mu )^2}-q_0(x)\right] {\phi ^*}^2dx. \end{aligned}$$

(3.6)

Thus $-\theta<\eta <M$ with $M>0$ independent of $\mu$. We claim that $\lim \inf _{\mu \rightarrow \infty }\eta =r>0$. In fact, choose a sequence $\mu _n\rightarrow \infty$ such that $\eta _n\rightarrow r$, and

$$\begin{aligned} -\Delta \phi _n=\theta \phi _n-2u_n\phi _n-\frac{a(x) \mu _n(1+k\mu _n)}{(1+mu_n+k\mu _n)^2}\phi _n+\eta _n\phi _n~\mathrm{in}~\Omega ,~~\frac{\partial \phi _n}{\partial {n}}=0~\mathrm{on}~\partial \Omega \nonumber \\ \end{aligned}$$

(3.7)

with normalized $\phi _n>0$, i.e. $\Vert \phi _n\Vert _2=1$. As $\int _\Omega |\nabla \phi _n|^2dx$ are uniformly bounded with respect to n, there exists a subsequence $\phi _{n_k}\rightharpoonup \phi _0$ weakly in $H^1(\Omega )$. Obviously, $\phi _0\ge 0$ and $\Vert \phi _0\Vert _2=1$. Multiply (3.7) by $\varphi \in C_0^{\infty }(\Omega )$ and integrate by parts to have

$$\begin{aligned} \int _\Omega \nabla \phi _n\cdot \nabla \varphi dx=\int _\Omega [\theta \phi _n\varphi -2u_n\phi _n\varphi -\frac{a(x)\mu _n(1+k\mu _n)}{(1+mu_n+k\mu _n)^2}\phi _n\varphi +\eta _n\phi _n\varphi ]dx. \end{aligned}$$

Since $u_n\rightarrow U_{\theta ,q_0}$ uniformly on $\overline{\Omega }$ as $n\rightarrow \infty$, we have

$$\begin{aligned} \int _\Omega \nabla \phi _0\cdot \nabla \varphi dx=\int _\Omega [\theta \phi _0\varphi -2U_{\theta ,q_0}\phi _0\varphi -q_0(x)\phi _0\varphi +r\phi _0\varphi ]dx. \end{aligned}$$

Comparing with (3.5), we prove the claim that $r=\eta ^*>0$. So, there exists $\tilde{\mu }>0$ such that $\eta =\eta (\mu )>0$ when $\mu >\tilde{\mu }$, which implies the linear stability of the positive solutions to (3.2).

Let

$$\begin{aligned} H(t,u)&=[MI-\Delta ]^{-1}\left( M+\theta -u-t\frac{a(x)\mu }{1+mu+k\mu }\right) u,\nonumber \\ A&=\{u\in C(\overline{\Omega });~\varepsilon _0<u<\theta +1\} \end{aligned}$$

(3.8)

with $0<\varepsilon _0<\min _{x\in \overline{\Omega }}U_{\theta ,q_0}(x)$, $0\le t\le 1$, M large. Define

$$\begin{aligned} S(t,u)=u-H(t,u). \end{aligned}$$

It is easy to see that $S(t,u)\ne 0$ for all $u\in \partial A$, $0\le t\le 1$. For large M, by the compactness of H, there are only finitely many isolated fixed points in A, denoted by $u_1,\dots ,u_m$. Together with the linear stability of the positive solutions and the homotopy invariance of fixed point index, we have

$$\begin{aligned} 1=\text{ index }(S(0,u),A,0)=\text{ index }(S(1,u),A,0)=\sum _{i=1}^{m}\text{ index }(H,u_i)=m. \end{aligned}$$

Therefore, there is an unique positive fixed point to (3.8) with $t=1$ whenever $\mu >\tilde{\mu }$, i.e. problem (3.2) has an unique positive solution. $\square$

Now, we can deal with the uniqueness Theorem 4.

Proof of Theorem 4

Let (u, v) be a positive solution of (1.3) with large $\mu$. Linearize the eigenvalue problem of (1.3) at (u, v) to have

$$\begin{aligned} \left\{ \begin{array}{llll}\displaystyle -\Delta \phi =\theta \phi -2u\phi -\frac{a(x)v(1+kv)}{(1+mu+kv)^2}\phi -\frac{a(x)u(1+mu)}{(1+mu+kv)^2}\psi +\eta \phi &{}\mathrm{in}~\Omega , \\ \displaystyle -\Delta \psi =\mu \psi -2v\psi +\frac{cu(1+mu)}{(1+mu+kv)^2}\psi +\frac{cv(1+kv)}{(1+mu+kv)^2}\phi +\eta \psi &{}\mathrm{in}~\Omega _1, \\ \displaystyle \frac{\partial \phi }{\partial n}=0~\mathrm{on}~\partial \Omega ,~~~~\frac{\partial \psi }{\partial n}=0~\mathrm{on}~\partial \Omega _1. \end{array}\right. \nonumber \\ \end{aligned}$$

(3.9)

Here $\phi$, $\psi$ and $\eta$ may be complex-valued.

From Kato’s inequality, we have

$$\begin{aligned}&-\Delta |\phi |\le -\mathrm{Re}\left( \frac{\bar{\phi }}{|\phi |}\Delta \phi \right) \nonumber \\&\quad =\mathrm{Re}\left( \theta |\phi |-2u|\phi |-\frac{a(x)v(1+kv)}{(1+mu+kv)^2}|\phi |+\frac{a(x)u(1+mu)}{(1+mu+kv)^2}\psi \cdot \frac{\bar{\phi }}{|\phi |}+\eta |\phi |\right) \nonumber \\&\quad \le \theta |\phi |-2u|\phi |-\frac{a(x)v(1+kv)}{(1+mu+kv)^2}|\phi |+\frac{a(x)u(1+mu)}{(1+mu+kv)^2}|\psi |+\mathrm{Re}(\eta )|\phi |. \end{aligned}$$

(3.10)

To obtain the linear stability, it suffices to prove that for any $\delta >0$, there exists $\mu _{\delta }>0$ such that the eigenvalues $\eta$ of (3.9) satisfy $\mathrm{Re}(\eta )\ge \eta ^*-\delta$ when $\mu \ge \mu _{\delta }$. Otherwise, there exist a $\delta _0>0$ and a sequences $\{(\mu _n,\eta _n,u_n,v_n,\phi _n,\psi _n)\}^{\infty }_{n=1}$ satisfying (3.9) with $\Vert \phi _n\Vert _2+\Vert \psi _n\Vert _2=1$, and $\mu _n\rightarrow \infty$ as $n\rightarrow \infty$ such that $\text{ Re }(\eta _n)<\eta ^*-\delta _0$. Replace $(\mu ,\eta ,u,v,\phi ,\psi )$ in (3.10) with $(\mu _n,\eta _n,u_n,v_n,\phi _n,\psi _n)$, multiply by $|\phi _n|$, and then integrate by parts over $\Omega$ to have

$$\begin{aligned} \int _\Omega |\nabla |\phi _n||^2dx&\le \int _\Omega \big (\theta |\phi _n|^2-2u_n|\phi _n|^2-\frac{a(x)v_n(1+kv_n)}{(1+mu_n+kv_n)}|\phi _n|^2\nonumber \\&~~+\frac{a(x)u_n(1+mu_n)}{(1+mu_n+kv_n)^2}|\psi _n||\phi _n|\big ) dx+(\eta ^*-\delta _0)\int _\Omega |\phi _n|^2dx.\nonumber \\ \end{aligned}$$

(3.11)

Let $r_n$ be the principal eigenvalue of the eigenvalue problem

$$\begin{aligned} -\Delta \varphi =\theta \varphi -2u_n\varphi -\frac{a(x)v_n(1+kv_n)}{(1+mu_n+kv_n)^2}\varphi +r_n\varphi ~\mathrm{in}~\Omega ,~~\frac{\partial \varphi }{\partial n}=0~\mathrm{on}~\partial \Omega . \end{aligned}$$

We know that

$$\begin{aligned} r_n-\eta ^*=\inf _{\varphi \in H^1(\Omega )}\frac{\int _\Omega \left[ |\nabla \varphi |^2-\theta \varphi ^2+2u_n\varphi ^2+\frac{a(x)v_n(1+kv_n)}{(1+mu_n+kv_n)^2}\varphi ^2-\eta ^*\varphi ^2\right] dx}{\int _\Omega \varphi ^2dx}, \end{aligned}$$

and $r_n\rightarrow \eta ^*$ by the proof of Proposition 3.1. So, there exists a $N>0$ such that $r_n-\eta ^*>-\frac{\delta _0}{2}$ for $n>N$. Thus by (3.11),

$$\begin{aligned} -\frac{\delta _0}{2}\int _\Omega |\phi _n|^2dx&<(r_n-\eta ^*)\int _\Omega |\phi _n|^2dx \\&\le \int _\Omega [|\nabla \phi _n|^2-\theta |\phi _n|^2+2u_n|\phi _n|^2\\&\quad +\frac{a(x)v_n(1+kv_n)}{(1+mu_n+kv_n)^2}|\phi _n|^2-\eta ^*|\phi _n|^2]dx \\&\le -\delta _0\int _{\Omega }|\phi _n|^2dx+\int _{\Omega _1}\frac{au_n(1+mu_n)}{(1+mu_n+kv_n)^2}|\psi _n||\phi _n| dx. \end{aligned}$$

Since $\frac{au_n(1+mu_n)}{(1+mu_n+kv_n)^2}\rightarrow 0$ in $C(\overline{\Omega }_1)$ as $n\rightarrow \infty$, then $\int _\Omega |\phi _n|^2dx\rightarrow 0$.

Using Kato’s inequality again, we have

$$\begin{aligned} -\Delta |\psi _n|&\le \mu _n|\psi _n|-2v_n|\psi _n|+\frac{cu_n(1+mu_n)}{(1+mu_n+kv_n)^2}|\psi _n|\\&\quad + \frac{cv_n(1+kv_n)}{(1+mu_n+kv_n)^2}|\phi _n|+(\eta ^*-\delta _0)|\psi _n|. \end{aligned}$$

Multiply by $|\psi _n|$ and integrate by parts over $\Omega _1$ to get

$$\begin{aligned} \int _{\Omega _1}|\nabla |\psi _n||^2dx&\le \int _{\Omega _1}\Bigg [\mu _n|\psi _n|^2-2v_n|\psi _n|^2+ \frac{cu_n(1+mu_n)}{(1+mu_n+kv_n)^2}|\psi _n|^2\\&\quad +\frac{cv_n(1+kv_n)}{(1+mu_n+kv_n)^2}|\phi _n||\psi _n|+(\eta ^*-\delta _0)|\psi _n|^2\Bigg ]dx \\&\le \int _{\Omega _1}\left[ -\mu _n+\frac{cu_n(1+mu_n)}{(1+mu_n+kv_n)^2}+\eta ^*-\delta _0\right] |\psi _n|^2dx \\&\quad +\int _{\Omega _1}\frac{cv_n(1+kv_n)}{(1+mu_n+kv_n)^2}|\phi _n||\psi _n| dx. \end{aligned}$$

Consequently,

$$\begin{aligned} \int _{\Omega _1}|\psi _n|^2dx&\le \frac{1}{\mu _n}\int _{\Omega _1}\left[ \frac{cu_n(1+mu_n)}{(1+mu_n+kv_n)^2}+\eta ^*-\delta _0\right] |\psi _n|^2dx\\&\quad +\frac{1}{\mu _n}\int _{\Omega _1}\frac{cv_n(1+kv_n)}{(1+mu_n+kv_n)^2}|\phi _n||\psi _n|dx\\&\le \frac{1}{\mu _n}\int _{\Omega _1}\left( \frac{c\theta }{1+m\theta }+\eta ^*-\delta _0\right) |\psi _n|^2dx +\frac{1}{\mu _n}\int _{\Omega _1}\frac{c}{k}|\phi _n||\psi _n|dx. \end{aligned}$$

This concludes $\int _{\Omega _1}|\psi _n|^2dx\rightarrow 0$ as $n\rightarrow \infty$, since $\mu _n\rightarrow \infty$ and $|\phi _n|,|\psi _n|$ are bounded in $L^2(\Omega _1)$.

In summary, we have obtained $\int _\Omega |\phi _n|^2dx$,$\int _{\Omega _1}|\psi _n|^2dx\rightarrow 0$, as $n\rightarrow \infty$ which contradict with $\Vert \phi _n\Vert _2+\Vert \psi _n\Vert _2=1$.

By using a similar argument to that in the proof of Proposition 3.1, we get from the linear stability of the positive solutions to (1.3) and Proposition 3.1 that the solution of (1.3) must be unique when $\mu >\max \{\tilde{\mu },\mu _0\}$ with $\mu _0=\inf \{\mu _\delta ; \delta \in (0,\eta ^*)\}$.

Finally, we consider the asymptotic behavior of the unique positive solution (u, v) as $\mu \rightarrow \infty$. Since $\frac{cu}{1+mu+kv}\le \frac{c\theta }{1+m\theta +k\mu }\rightarrow 0$ as $\mu \rightarrow \infty$, for any $\epsilon >0$, there is a $\mu _\epsilon >0$ such that $\frac{cu}{1+mu+kv}<\epsilon$ for $\mu >\mu _\epsilon$. Then

$$\begin{aligned} \mu v-v^2\le -\Delta v\le (\mu +\epsilon )v-v^2~\mathrm{in}~\Omega _1, \end{aligned}$$

which yields $\mu \le v\le \mu +\epsilon$ for $\mu >\mu _\epsilon$. And thus $v-\mu \rightarrow 0$ as $\mu \rightarrow \infty$. We know that $\frac{a(x)v}{1+mu+kv}\rightarrow q_0(x)$, and then $u\rightarrow U_{\theta ,q_0}(x)$ uniformly on $\overline{\Omega }$ as $\mu \rightarrow \infty$. $\square$

4 Discussion

In a reaction–diffusion system of predator–prey PDE model, in addition to the interaction mechanism between the species, the behavior of the species is also affected by the diffusion of the species, as well as the size and geometry of the habitat. Obviously, the prey species would die out under excessive predation from nature or humans. The results obtained in this paper show the way in which the created protection zone saves the endangered prey species in the diffusive predator–prey model with Beddington–DeAngelis type functional response and non-flux boundary conditions.

Compared with previous results on protection zone problems with various functional responses such as the Lotka–Voltera type competition system (Du and Liang 2008), Holling II type predator–prey system (Cui et al. 2014; Du and Shi 2006), and Leslie type predator–prey system (Du et al. 2009), richer dynamic properties have been observed for the model (1.1) with B-D type functional response in this paper. It can be found that a total of four threshold values are obtained here for the prey birth rate $\theta$, i.e., $\theta _0$, $\theta ^*$, $\theta _1$ (for the predator growth rate $\mu >0$) and $\theta _*$ (for $\mu \le 0$).

The first threshold value $\theta _0$ gives the necessary condition for establishing a protection zone to save the prey u. By Theorem 1, the survival of u could be automatically ensured without protection zones whenever $\theta >\theta _0=\frac{a}{k}$, which can be realized when the refuge ability of the prey is properly large that $k>\frac{a}{\theta }$, or the predation rate is small that $a<{\theta }{k}$. In other words, the protection zones have to be made only if the prey’s refuge ability is too weak with respect to its birth rate $\theta$ and the predation rate a. This matches with the mechanistic derivation for the B-D type functional response proposed in Geritz and Gyllenberg (2012). In addition, Theorem 1 says also that the protection zones are unnecessary if the predator’s growth rate $\mu \le 0$, where the predator species v can not live without the prey u, and thus the extinction of v cannot take place after of u.

The second threshold value is $\theta ^*=\theta ^*(\mu ,\Omega _0)=\lambda _1(q(x))$ with $q(x)=\frac{a(x)\mu }{1+k\mu }$ and $\mu >0$. By Theorem 2, the positive steady states can be attained for $\theta \in (\theta ^*,\theta _0)$. Due to the monotonicity of the principal eigenvalue $\lambda _1=\lambda _1(q(x))$ with respect to q(x), we know that the threshold value $\theta ^*$ would be enlarged (and hence harmful for the prey u) when the predation rate a(x) or the predator’s growth rate $\mu$ increase, or when the prey refuge k or the size of the protection zone $\Omega _0$ decrease. Conversely, Theorem 2 says also that the prey u must become extinct when $\theta \le \theta ^*$ with the handling time m of $v_H$ being shorter than $\frac{(k\mu +1)^2}{a\mu }$. All of these match those in Geritz and Gyllenberg (2012). In addition, since $\theta ^*{(\mu ,\Omega _0)}\le \theta _0$ is strictly increasing with respect to $\mu$ and decreasing when enlarging $\Omega _0$, letting $\mu \rightarrow \infty$, we get the third threshold value $\theta _1=\theta _1(\Omega _0)$ such that if $\theta >\theta _1(\Omega _0)$, prey species could survive no matter how large the predator’s growth rate is. The critical $\theta =\theta _1(\Omega _0)$ implies a critical size of the protection zone as well, namely, if the real protection zone $\widetilde{\Omega }_0\Supset \Omega _0$, the survival of the prey with such birth rate $\theta$ is independent of the predator’s growth rate. Also, the uniqueness and linear stability obtained in Theorem 4 for $\mu$ large enough are reasonable because $\frac{cu}{1+mu+kv}\rightarrow 0$, and hence $v-\mu \rightarrow 0$ as $\mu \rightarrow \infty$.

Since the condition $\mu \le 0$ yields the survival of u without protection zones by Theorem 1, the fourth threshold value $\theta _*$ obtained in Theorem 3 with $\mu \in (-\frac{c}{m},0]$ is only made for v alive. In fact, the conversion of prey is limited by $\frac{c}{m}$, as shown in the proof of Theorem 1, the predator v must be die out if its growth rate $\mu \le -\frac{c}{m}$. With such non-positive growth rate $\mu \in (-\frac{c}{m},0]$, there should be properly large number of prey to survive the predator, just as described via the criterion $\theta >\theta _*=\frac{|\mu |}{c-m|\mu |}\ge 0$ in Theorem 3. It is worth noting that the threshold value $\theta _*$ for alive predator v would be enlarged as m (the handling time of $v_H$) is increasing. This well matches the mechanism in Geritz and Gyllenberg (2012).

We have shown the effect of refuge ability of the prey and protection zone have on the coexistence and stability of predator–prey species in this paper. In fact, protection zones can be regarded as another refuge offered by human intervention, which is necessary if the prey’s refuge ability is too weak in the predator–prey system to prevent the extinction of the prey populations. The critical sizes of protection zones, obtained in this paper and represented by the principal eigenvalues $\theta ^*=\lambda _1(q(x))$ and $\theta _1=\lambda _1(q_0(x))$, show the basic requirement (depending on the predator’s growth rate) and the sufficient one (working under any predator’s growth rate), respectively. The results of the present paper would be helpful to the designing of nature reserves and no-fishing zones, etc.