1 Introduction

In the past few decades, the dynamical models in the form of reaction-diffusion equations have been frequently used to solve problems related to spatial ecology and evolution, see [5, 22, 23, 25, 27]. In the real world, due to reproductive maturity or other time lags in biological processes, historical information may have a significant impact on the dynamics of population systems, so many delayed reaction-diffusion equations are used to describe the evolution of population distribution [1, 4, 6, 15, 30].

The dispersal by random diffusion is one of the most basic dispersal strategies. In reality, the movements of species may be a combination of both random and biased ones. Since species are intelligent, many organisms can sense their environment and pay attention to moving in a direction that is favorable to them. Based on this observation, Belgacem and Cosner [2] assumed that the population can exhibit a taxis in the direction of increasing environmental favorability, and studied the reaction–diffusion–advection logistic model

$$\begin{aligned} {\left\{ \begin{array}{ll} u_t=\nabla \cdot [d\nabla u-au\nabla m]+m(x)u-cu^2, &{} x\in \varOmega ,t>0, \\ u(x,t)=0, &{} x\in \partial \varOmega ,t>0, \end{array}\right. } \end{aligned}$$
(1.1)

where u(xt) denotes the species density at location x and time t. In model (1.1), the term \(-d\nabla u\) accounts for random diffusion, \(a u\nabla m(x)\) represents the migration of species along the gradient direction of resource function m(x). The results of Belgacem and Cosner [2] and the subsequent literature [9] show that for single species, migration along the gradient direction of the food distribution of the species will generally contribute to the survival of the species.

We would like to know what spatiotemporal patterns can be indeced by the joint effect of time delays, spatial diffusion, advection, heterogeneous environment and population interaction. In a reaction–diffusion–advection model with time delay effect, the effects of dispersal and time delays are not independent of each other, and an individual that was previously at location x may now not be at the same point in space [3, 6, 12]. Therefore, it is more reasonable to consider the model with nonlocal time delay. Recently, Jin and Yuan [19] investigated the following general delayed reaction–diffusion–advection equation

$$\begin{aligned} {\left\{ \begin{array}{ll} u_t=\nabla \cdot [d\nabla u-au\nabla m]+u(x,t)f\left( x,u(x,t),\displaystyle \int _{\varOmega }k(x,y)u(y,t-\tau ) \mathrm {d}y \right) , &{} x\in \varOmega ,t>0, \\ u(x,t)=0, &{} x\in \partial \varOmega ,t>0, \end{array}\right. }\nonumber \\ \end{aligned}$$
(1.2)

where \(f(x,0,0)=m(x)\) and the term \(\int _{\varOmega }k(x,y)u(y,t-\tau )\mathrm {d}y\) is called the nonlocal delayed term, which represents the spatial weighted time delays according to distance from the original position. In [19], Jin and Yuan showed the existence of spatially nonhomogeneous steady-state solutions of (1.2) and investigated whether time delay \(\tau \) can induce Hopf bifurcation near the steady-state solution. They also showed the influence of the advection rate a on Hopf bifurcation.

The dynamics of the two-species model has been extensively studied, including the global stability of (non-)constant steady states [7, 10, 17, 20, 24, 26] or Hopf bifurcations induced by time delays at the (non-)constant steady states [13, 18, 29, 31, 33]. For instance, in [14, 15], the authors considered the diffusive two-species model with nonlocal delay effect and investigated the stability of spatially nonhomogeneous positive steady state and the corresponding Hopf bifurcation problem. Recently, Li and Dai [21] have studied the following Lotka–Volterra competition–diffusion–advection model with time delay effect:

$$\begin{aligned} {\left\{ \begin{array}{ll} u_t=\nabla \cdot [d\nabla u-au\nabla m]+u\left[ m(x)-a_{11}u(x,t-\tau )-a_{12}v(x,t-\tau ) \right] , &{} x\in \varOmega ,t>0, \\ v_t=\nabla \cdot [d\nabla v-av\nabla m]+v\left[ m(x)-a_{21}u(x,t-\tau )-a_{22}v(x,t-\tau ) \right] , &{} x\in \varOmega ,t>0, \\ u(x,t)=v(x,t)=0, &{} x\in \partial \varOmega ,t>0. \end{array}\right. }\nonumber \\ \end{aligned}$$
(1.3)

They obtained the existence of spatially nonhomogeneous positive steady state and showed that this positive steady state loses its stability for a large delay \(\tau \) and a Hopf bifurcation occurs such that system (1.3) exhibits oscillatory pattern.

Motivated by [6, 14, 19], we can assume that in a two-species model, the per-capita growth rates of two species do not depend on its density at the current positions and time but on all positions in region \(\varOmega \) and previous time \(\tau \). Hence, the localized density-dependent per capita growth rates \(m(x)-a_{11}u(x,t-\tau )-a_{12}v(x,t-\tau )\) and \(m(x)-a_{21}u(x,t-\tau )-a_{22}v(x,t-\tau )\) in (1.3) are not realistic. Instead, it is more reasonable to consider the following general reaction–diffusion–advection two species model with nonlocal delay effect as follows:

$$\begin{aligned} {\left\{ \begin{array}{ll} u_t=\nabla \cdot [d\nabla u-au\nabla m]+u(x,t)f_1\big (x,(k_{11}*u)(x,t-\tau ),(k_{12}*v)(x,t-\tau ) \big ), &{} x\in \varOmega ,t>0, \\ v_t=\nabla \cdot [d\nabla v-av\nabla m]+v(x,t)f_2\big (x,(k_{21}*u)(x,t-\tau ),(k_{22}*v)(x,t-\tau ) \big ), &{} x\in \varOmega ,t>0, \\ u(x,t)=v(x,t)=0, &{} x\in \partial \varOmega ,t>0. \end{array}\right. }\nonumber \\ \end{aligned}$$
(1.4)

Here u(xt) and v(xt) denote the species densities at time t and location x, respectively; the two species have the same diffusion rate \(d>0\) and the same advection rate \(a>0\); \(\varOmega \) is a bounded domain in \({\mathbb {R}}^n(1\le n\le 3)\) with smooth boundary \(\partial \varOmega \); \(\tau \) is the time delay representing the maturation time; \(k_{ij}(i,j=1,2)\) are continuous kernel functions on \(\varOmega \times \varOmega \) which describe the dispersal behavior of the populations and

$$\begin{aligned} (k_{i1}*u)(x,t)=\int _{\varOmega }k_{i1}(x,y)u(y,t)\mathrm {d}y,\ (k_{i2}*v)(x,t)=\int _{\varOmega }k_{i1}(x,y)v(y,t)\mathrm {d}y,\ i=1,2; \end{aligned}$$

the nonlinear smooth functions \(f_i(x,s_1,s_2)(i=1,2):\varOmega \times {\mathbb {R}}\times {\mathbb {R}}\rightarrow {\mathbb {R}}\) are called the general per capita growth rates and satisfy the condition

  • \(\mathbf{(H_1)}\) \(f_1(x,0,0)=f_2(x,0,0)=m(x)\), where \(m(x)\in C^2({\overline{\varOmega }})\) and \(\max _{{\overline{\varOmega }}}m(x)> 0\).

The Dirichlet boundary conditions imply that the exterior environment is hostile and the two species cannot move across the boundary of environment. We consider model (1.4) with the following initial condition:

$$\begin{aligned} u(x,s)=\varphi _1(x,s),\ v(x,0)=\varphi _2(x,s),\ \ x\in \varOmega ,s\in [-\tau ,0], \end{aligned}$$

where the initial data \(\varphi _1,\varphi _2\in {\mathscr {C}}\triangleq C([-\tau ,0],{\mathbb {Y}})\) with \({\mathbb {Y}}=L^2(\varOmega )\).

For the convenience of analysis, we first make a variable transformation. Letting \({\tilde{u}}=e^{(-a/d)m(x)}u, {\tilde{v}}=e^{(-a/d)m(x)}v, t={\tilde{t}}/d\), denoting \(\lambda =1/d,\alpha =a/d,\tau ={\tilde{\tau }}/d\), and dropping the tilde sign, model (1.4) can be transformed as follows:

$$\begin{aligned} {\left\{ \begin{array}{ll} u_t=e^{-\alpha m(x)}\nabla \cdot \left[ e^{\alpha m(x)}\nabla u\right] +\lambda u(x,t)f_1\big (x,(K_{11}*u)(x,t-\tau ),(K_{12}*v)(x,t-\tau ) \big ), &{} x\in \varOmega ,t>0, \\ v_t=e^{-\alpha m(x)}\nabla \cdot \left[ e^{\alpha m(x)}\nabla v\right] +\lambda v(x,t)f_2\big (x,(K_{21}*u)(x,t-\tau ),(K_{22}*v)(x,t-\tau ) \big ), &{} x\in \varOmega ,t>0, \\ u(x,t)=v(x,t)=0, &{} x\in \partial \varOmega ,t>0, \end{array}\right. }\nonumber \\ \end{aligned}$$
(1.5)

where

$$\begin{aligned} (K_{ij}*g)(x,t)=\int _{\varOmega }K_{ij}(x,y)g(y,t)\mathrm {d}y, \end{aligned}$$

with \(K_{ij}(x,y)=k_{ij}(x,y)e^{\alpha m(y)}\) for \(i,j=1,2\).

For the simplification of calculation, denote

$$\begin{aligned} r^i_1(x)=\frac{\partial f_i}{\partial s_1}(x,0,0),\ \ r^i_2(x)=\frac{\partial f_i}{\partial s_2}(x,0,0), \end{aligned}$$

and

$$\begin{aligned} \kappa _{ij}=\int _{\varOmega }\int _{\varOmega }e^{\alpha m(x)}r_j^i(x)K_{ij}(x,y)\phi ^2(x)\phi (y)\mathrm {d}y\mathrm {d}x\ne 0 \end{aligned}$$

for \(i,j=1,2\). Suppose further the following two assumptions hold:

  • \(\mathbf{(H_2)}\) \((\kappa _{21}-\kappa _{11})(\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21})>0\) and \((\kappa _{12}-\kappa _{22})(\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21})>0\);

  • \(\mathbf{(H_3)}\) \(\kappa _{11}\kappa _{22}\ge 0\) and \(\kappa _{12}\kappa _{21}\ge 0\).

It follows from [2, 5, 23] that, under the assumption \(\mathbf{(H_1)}\), the following eigenvalue problem

$$\begin{aligned} {\left\{ \begin{array}{ll} -e^{-\alpha m(x)}\nabla \cdot [e^{\alpha m(x)}\nabla u]=\lambda m(x)u, &{} x\in \varOmega , \\ u(x)=0, &{} x\in \partial \varOmega , \end{array}\right. } \end{aligned}$$
(1.6)

has a positive principal eigenvalue \(\lambda _*\) and the corresponding eigenfunction \(\phi \in C^{1+\delta }({\overline{\varOmega }})\) can be chosen strictly positive in \(\varOmega \), where \(\delta \in (0,1)\). In this paper our main results are in the spirit of [21] for the local growth rate case: under the assumptions \(\mathbf{(H_1)}-\mathbf{(H_3)}\), there exists a \(\lambda ^*\) with \(0<\lambda ^*-\lambda _*\ll 1\), such that for any \(\lambda \in (\lambda _*,\lambda ^*]\), system (1.5) admits a spatially nonhomogeneous positive steady state \((u_{\lambda },v_{\lambda })\) and there exists a sequence of values \(\{ \tau _n(\lambda ) \}_{n=0}^{\infty }\) such that \((u_{\lambda },v_{\lambda })\) is locally asymptotically stable when \(\tau \in [0,\tau _0(\lambda ))\), unstable when \(\tau \in (\tau _0(\lambda ),\infty )\), and for system (1.5), a forward Hopf bifurcation occurs at \(\tau _n(\lambda )\) from the positive steady state \((u_{\lambda },v_{\lambda })\). Moreover, Hopf bifurcation is more likely to occur when adding a term describing advection along the environmental gradients for the diffusive Lotka–Volterra competition model with nonlocal delay. Here, the assumption \(\mathbf{(H_2)}\) is used to guarantee the existence of positive steady-state solutions. \(\mathbf{(H_3)}\) is imposed to make sure the simplicity of pure imaginary eigenvalue and is actually satisfied by many population biological models.

The rest part of this paper is organized as follows. In Sect. 2, we establish the existence of the positive steady state of model (1.5). Sections 3 and 4 are devoted to the stability and Hopf bifurcation of the positive steady state through analyzing the corresponding eigenvalue problem. Then the normal form of Hopf bifurcation is derived in Sect. 5 to determine the bifurcation direction and stability of the bifurcating periodic solutions. In Sect. 6, the general results are applied to a competition–diffusion–advection model with nonlocal delay effect.

Notice that the elliptic operator in (1.6) is not self-adjoint because of advection term, which causes some technical difficulties. In view of these difficulties we introduce some weighted spaces. The weighted space plays a vital role in the Hopf bifurcation analysis of system (1.5). Throughout the paper, we use the following notations. Denote by \(L^2_w(\varOmega )\) the weighted \(L^2\) space with a weighted norm

$$\begin{aligned} \Vert u\Vert _{L^2_w(\varOmega )}=\left( \int _{\varOmega }e^{\alpha m(x)}|u(x)|^2\mathrm {d}x \right) ^{1/2}. \end{aligned}$$

Let \(H^k_w(\varOmega )(k\ge 0)\) be the weighted Sobolev space of the \(L^2_w\)-function u(x) defined on \(\varOmega \), and the norm of space \(H^k_w(\varOmega )(k\ge 0)\) is defined by

$$\begin{aligned} \Vert u\Vert _{H^k_w(\varOmega )}=\left( \sum _{|j|\le k}\int _{\varOmega }e^{\alpha m(x)}|\partial ^ju|^2\mathrm {d}x \right) ^{1/2}. \end{aligned}$$

Define the space \({\mathbb {X}}=H^2_w(\varOmega )\cap H_{0,w}^1(\varOmega )\) and \({\mathbb {Y}}=L^2_w(\varOmega )\), where

$$\begin{aligned} H_{0,w}^1(\varOmega )=\left\{ u\in H^1_w(\varOmega )|u(x)=0,\forall x\in \partial \varOmega \right\} . \end{aligned}$$

For a space Z, we also define the complexification of Z to be \(Z_{{\mathbb {C}}}:=Z\oplus \mathrm i Z=\{ x_1+\mathrm i x_2|x_1,x_2\in Z \}\). Let \({\mathcal {C}}=C([-\tau ,0],{\mathbb {Y}})\) be the Banach space of continuous mapping from \([-\tau ,0]\) into \({\mathbb {Y}}\), and \(\langle \cdot ,\cdot \rangle _w\) be the \(L^2_w\) inner product on complex-valued Hilbert space \({\mathbb {Y}}_{{\mathbb {C}}}\) or \({\mathbb {Y}}_{{\mathbb {C}}}^2\), defined as

$$\begin{aligned} \langle u,v \rangle _w=\int _{\varOmega }e^{\alpha m(x)}{\overline{u}}(x)^Tv(x)\mathrm {d}x. \end{aligned}$$
(1.7)

2 Existence of Positive Steady State

This section is devoted to the the existence of the positive steady state of model (1.5), which satisfies the following boundary value equation:

$$\begin{aligned} {\left\{ \begin{array}{ll} e^{-\alpha m(x)}\nabla \cdot \left[ e^{\alpha m(x)}\nabla u\right] +\lambda u(x)f_1\big (x,(K_{11}*u)(x),(K_{12}*v)(x) \big )=0, &{} x\in \varOmega , \\ e^{-\alpha m(x)}\nabla \cdot \left[ e^{\alpha m(x)}\nabla v\right] +\lambda v(x)f_2\big (x,(K_{21}*u)(x),(K_{22}*v)(x) \big )=0, &{} x\in \varOmega , \\ u(x)=v(x)=0, &{} x\in \partial \varOmega . \end{array}\right. }\nonumber \\ \end{aligned}$$
(2.1)

To solve problem (2.1), define \(F:{\mathbb {X}}^2\times R^+\rightarrow {\mathbb {Y}}^2\) as

$$\begin{aligned} F(U,\lambda )=\left[ \begin{array}{c} e^{-\alpha m}\nabla \cdot \left[ e^{\alpha m}\nabla u\right] +\lambda uf_1\big (x,K_{11}*u,K_{12}*v \big ) \\ e^{-\alpha m}\nabla \cdot \left[ e^{\alpha m}\nabla v\right] +\lambda vf_2\big (x,K_{21}*u,K_{22}*v \big ) \end{array}\right] \end{aligned}$$

for all \(U=(u,v)^T\). At first, for any fixed \(\lambda \in {\mathbb {R}}^+\), \(F(U,\lambda )\) always has a trivial steady state (0, 0). Denote

$$\begin{aligned} {\mathcal {L}}:= \begin{pmatrix} e^{-\alpha m(x)}\nabla \cdot [e^{\alpha m(x)}\nabla ]+\lambda _* m(x) &{} 0 \\ 0 &{} e^{-\alpha m(x)}\nabla \cdot [e^{\alpha m(x)}\nabla ]+\lambda _* m(x) \end{pmatrix}, \end{aligned}$$
(2.2)

which is the Frechét derivative of F with respect to U at \((0,\lambda _*)\). It is easy to check that \({\mathcal {L}}\) is a self-adjoint operator in the sense of weighted inner product, and \({\mathscr {N}}({\mathcal {L}})=\mathrm {span}\{ q_1,q_2 \}\), where \(q_1=(\phi ,0)^T\) and \(q_2=(0,\phi )^T\). Then operator \({\mathcal {L}}:{\mathbb {X}}^2\rightarrow {\mathbb {Y}}^2\) is Fredholm with index zero. Clearly, the Crandall-Rabinowitz bifurcation theorem cannot be applied here to show the existence of positive solution of (2.1) since \(\mathrm {dim}{\mathscr {N}}({\mathcal {L}})=2\). Now, we deal with this situation by implicit function theorem. For later discussion, we decompose the spaces \({\mathbb {X}}^2,{\mathbb {Y}}^2\):

$$\begin{aligned} {\mathbb {X}}^2={\mathscr {N}}({\mathcal {L}})\oplus X_1^2,\ \ \ {\mathbb {Y}}^2={\mathscr {N}}({\mathcal {L}})\oplus Y_1^2, \end{aligned}$$

where

$$\begin{aligned} X_1=\left\{ y\in {\mathbb {X}}:\langle \phi ,y\rangle _w=0 \right\} ,\ Y_1=\left\{ y\in {\mathbb {Y}}:\langle \phi ,y\rangle _w=0 \right\} . \end{aligned}$$

Then we have the following result on the existence of positive steady states for system (1.5).

Theorem 1

Suppose that \(\mathbf{(H_1)}-\mathbf{(H_3)}\) hold. Then there exist \(\lambda ^*>\lambda _*\) and a continuously differential mapping \(\lambda \mapsto (\xi _{\lambda },\eta _{\lambda },\beta _{\lambda },c_{\lambda })\) from \([\lambda _*,\lambda ^*]\) to \(X_1^2\times {\mathbb {R}}^+ \times {\mathbb {R}}^+\) such that, for \(\lambda \in (\lambda _*,\lambda ^*]\), system (1.5) has a positive steady state \((u_\lambda (x),v_\lambda (x))\), where

$$\begin{aligned} {\left\{ \begin{array}{ll} u_{\lambda }=\beta _{\lambda }(\lambda -\lambda _*)[\phi +(\lambda -\lambda _*)\xi _{\lambda }], \\ v_{\lambda }=c_{\lambda }(\lambda -\lambda _*)[\phi +(\lambda -\lambda _*)\eta _{\lambda }]. \end{array}\right. } \end{aligned}$$
(2.3)

Moreover, for \(\lambda =\lambda _*\),

$$\begin{aligned} \left\{ \begin{aligned} \beta _{\lambda _*}&=\dfrac{(\kappa _{12}-\kappa _{22})\int _{\varOmega }m(x)e^{\alpha m(x)}\phi ^2(x)dx}{\lambda _* (\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21})}, \\ c_{\lambda _*}&=\dfrac{(\kappa _{21}-\kappa _{11})\int _{\varOmega }m(x)e^{\alpha m(x)}\phi ^2(x)dx}{\lambda _* (\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21})}, \end{aligned}\right. \end{aligned}$$
(2.4)

and \((\xi _{\lambda _*},\eta _{\lambda _*})^T\in X_1^2\) is the unique solution of the following equation

$$\begin{aligned}&{\mathcal {L}} \begin{pmatrix} \xi \\ \eta \end{pmatrix} + \begin{pmatrix} m(x)+\lambda _*\beta _{\lambda _*}r_1^1(x)\int _{\varOmega }K_{11}(x,y)\phi (y)\mathrm {d}y+\lambda _*c_{\lambda _*}r_2^1(x)\int _{\varOmega }K_{12}(x,y)\phi (y)\mathrm {d}y \\ m(x)+\lambda _*\beta _{\lambda _*}r_1^2(x)\int _{\varOmega }K_{21}(x,y)\phi (y)\mathrm {d}y+\lambda _*c_{\lambda _*}r_2^2(x)\int _{\varOmega }K_{22}(x,y)\phi (y)\mathrm {d}y \end{pmatrix}\phi \nonumber \\&\quad =0. \end{aligned}$$
(2.5)

Proof

One can easily have

$$\begin{aligned} \lambda _*\displaystyle \int _{\varOmega }m(x)e^{\alpha m(x)}\phi ^2(x)dx=\displaystyle \int _{\varOmega }e^{\alpha m(x)}|\nabla \phi (x)|^2dx>0, \end{aligned}$$

and then \(\beta _{\lambda _*}\) and \(c_{\lambda _*}\) are well defined and positive. From (2.4), we see that

$$\begin{aligned} {\left\{ \begin{array}{ll} \left\langle \phi , \big (m(x)+\lambda _*\beta _{\lambda _*}r_1^1(x)\int _{\varOmega }K_{11}(x,y)\phi (y)\mathrm {d}y+\lambda _*c_{\lambda _*}r_2^1(x)\int _{\varOmega }K_{12}(x,y)\phi (y)\mathrm {d}y \big )\phi \right\rangle _w=0, \\ \left\langle \phi , \big (m(x)+\lambda _*\beta _{\lambda _*}r_1^2(x)\int _{\varOmega }K_{21}(x,y)\phi (y)\mathrm {d}y+\lambda _*c_{\lambda _*}r_2^2(x)\int _{\varOmega }K_{22}(x,y)\phi (y)\mathrm {d}y\big ) \phi \right\rangle _w=0, \end{array}\right. } \end{aligned}$$

and hence \(\xi _{\lambda _*}\) and \(\eta _{\lambda _*}\) are also well defined. Now, setting \(u=\beta (\lambda -\lambda _*)[\phi +(\lambda -\lambda _*)\xi ]\) and \(v=c(\lambda -\lambda _*)[\phi +(\lambda -\lambda _*)\eta ]\) into \(F(U,\lambda )=0\), we obtain that \((\beta ,c,\xi ,\eta )\) satisfies

$$\begin{aligned} {\mathcal {F}}(\xi ,\eta ,\beta ,c,\lambda ) ={\mathcal {L}} \begin{pmatrix} \xi \\ \eta \end{pmatrix} +m(x) \begin{pmatrix} \phi +(\lambda -\lambda _*)\xi \\ \phi +(\lambda -\lambda _*)\eta \end{pmatrix} +\lambda \begin{pmatrix} [\phi +(\lambda -\lambda _*)\xi ]h_1(\xi ,\eta ,\beta ,c,\lambda ) \\ [\phi +(\lambda -\lambda _*)\eta ]h_2(\xi ,\eta ,\beta ,c,\lambda ) \end{pmatrix}=0, \end{aligned}$$

where

$$\begin{aligned} h_i(\xi ,\eta ,\beta ,c,\lambda )={\left\{ \begin{array}{ll} \dfrac{f_i(x,K_{i1}*u,K_{i2}*v)-m(x)}{\lambda -\lambda _*}, &{} \lambda \ne \lambda _*, \\ \beta r_1^i(x){\widetilde{k}}_{i1}(x)+c r_2^i(x){\widetilde{k}}_{i2}(x), &{} \lambda =\lambda _* \end{array}\right. } \end{aligned}$$
(2.6)

with

$$\begin{aligned} {\widetilde{k}}_{ij}(x)=\int _{\varOmega }K_{ij}(x,y)\phi (y)\mathrm {d}y\ \ \mathrm {for}\ \ i,j=1,2. \end{aligned}$$
(2.7)

That is, the existence problem of positive solution of (2.1) is reduced to solving \({\mathcal {F}}(\xi ,\eta ,\beta ,c,\lambda )=0\). Seeing that \(\varOmega \) is a bounded domain in \({\mathbb {R}}^n(1\le n\le 3)\) with smooth boundary, we can deduce that \(X_1^2\) is compactly imbedded into \(C^{\delta }({\overline{\varOmega }}) \times C^{\delta }({\overline{\varOmega }})\) for some \(\delta \in (0,1)\). Then \({\mathcal {F}}(\xi ,\eta ,\beta ,c,\lambda )\) is a function from \(X_1^2\times {\mathbb {R}}^3\) to \({\mathbb {Y}}^2\). From the definition of \(\xi _{\lambda _*},\eta _{\lambda _*},\beta _{\lambda _*},c_{\lambda _*}\), it holds that \({\mathcal {F}}(\xi _{\lambda _*},\eta _{\lambda _*},\beta _{\lambda _*},c_{\lambda _*},\lambda _*)=0\). Let \(D_{(\xi ,\eta ,\beta ,c)}{\mathcal {F}}(\xi _{\lambda _*},\eta _{\lambda _*},\beta _{\lambda _*},c_{\lambda _*},\lambda _*)[\chi ,\kappa ,\rho ,\varsigma ]\) be the Fréchet derivative of \({\mathcal {F}}\) with respect to \((\xi ,\eta ,\beta ,c)\) evaluated at \((\xi _{\lambda _*},\eta _{\lambda _*},\beta _{\lambda _*},c_{\lambda _*},\lambda _*)\). Then a calculation gives that

$$\begin{aligned}&D_{(\xi ,\eta ,\beta ,c)}{\mathcal {F}}(\xi _{\lambda _*},\eta _{\lambda _*},\beta _{\lambda _*},c_{\lambda _*},\lambda _*)[\chi ,\kappa ,\rho ,\varsigma ]\nonumber \\&\quad ={\mathcal {L}} \begin{pmatrix} \chi \\ \kappa \end{pmatrix} +\lambda _*\phi \begin{pmatrix} \rho r_1^1(x){\widetilde{k}}_{11}(x)+\varsigma r_2^1(x){\widetilde{k}}_{12}(x) \\ \rho r_1^2(x){\widetilde{k}}_{21}(x)+\varsigma r_2^2(x){\widetilde{k}}_{22}(x) \end{pmatrix}. \end{aligned}$$

For applying the implitic function theorem, we will verify that \(D_{(\xi ,\eta ,\beta ,c)}{\mathcal {F}}(\xi _{\lambda _*},\eta _{\lambda _*},\beta _{\lambda _*},c_{\lambda _*},\lambda _*)\) is a bijection from \(X_1^2\times {\mathbb {R}}^2\) to \({\mathbb {Y}}^2\). Since \(\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21}\ne 0\) due to assumption \(\mathbf{(H_2)}\), one can deduce that

$$\begin{aligned} (\xi ,\eta ,\beta ,c)=(0,0,0,0)\ \ \mathrm {if}\ \ D_{(\xi ,\eta ,\beta ,c)}{\mathcal {F}}(\xi _{\lambda _*},\eta _{\lambda _*},\beta _{\lambda _*},c_{\lambda _*},\lambda _*)[\chi ,\kappa ,\rho ,\varsigma ]=0. \end{aligned}$$

That is, \(D_{(\xi ,\eta ,\beta ,c)}{\mathcal {F}}(\xi _{\lambda _*},\eta _{\lambda _*},\beta _{\lambda _*},c_{\lambda _*},\lambda _*)\) is injective. Next, we show that it is a surjection. For any \(({\hat{u}},{\hat{v}})^{T}\in {\mathbb {Y}}^2\), we have the following decomposition

$$\begin{aligned} \begin{pmatrix} {\hat{u}} \\ {\hat{v}} \end{pmatrix}=\begin{pmatrix} u_1 \\ v_1 \end{pmatrix}+\begin{pmatrix} u_2 \\ v_2 \end{pmatrix},\ \ \mathrm {where}\ \begin{pmatrix} u_1 \\ v_1 \end{pmatrix}\in {\mathscr {N}}({\mathcal {L}}),\begin{pmatrix} u_2 \\ v_2 \end{pmatrix}\in Y_1^2. \end{aligned}$$

By choosing

$$\begin{aligned} \rho _0=\dfrac{\kappa _{22}\left\langle \phi ,u_1 \right\rangle _w-\kappa _{12}\left\langle \phi ,v_1 \right\rangle _w}{\lambda _* (\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21})} \ \mathrm {and}\ \ \varsigma _0=\dfrac{\kappa _{11}\left\langle \phi ,u_1 \right\rangle _w-\kappa _{21}\left\langle \phi ,v_1 \right\rangle _w}{\lambda _* (\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21})}, \end{aligned}$$

there holds that

$$\begin{aligned} -\lambda _*\phi \begin{pmatrix} \rho _0 r_1^1(x){\widetilde{k}}_{11}(x)+\varsigma _0 r_2^1(x){\widetilde{k}}_{12}(x) \\ \rho _0 r_1^2(x){\widetilde{k}}_{21}(x)+\varsigma _0 r_2^2(x){\widetilde{k}}_{22}(x) \end{pmatrix}+\begin{pmatrix} u_1 \\ v_1 \end{pmatrix}\in {\mathscr {R}}({\mathcal {L}}). \end{aligned}$$

Notice that \((u_2,v_2)^T\in Y_1^2={\mathscr {R}}({\mathcal {L}})\) and \({\mathcal {L}}:X_1^2\rightarrow Y_1^2\) is a bijection, then there is \((\chi _0,\kappa _0)\in X_1^2\) such that

$$\begin{aligned} D_{(\xi ,\eta ,\beta ,c)}{\mathcal {F}}(\xi _{\lambda _*},\eta _{\lambda _*},\beta _{\lambda _*},c_{\lambda _*},\lambda _*)[\chi _0,\kappa _0,\rho _0,\varsigma _0]=({\hat{u}},{\hat{v}})^T. \end{aligned}$$

Consequently, \(D_{(\xi ,\eta ,\beta ,c)}{\mathcal {F}}(\xi _{\lambda _*},\eta _{\lambda _*},\beta _{\lambda _*},c_{\lambda _*},\lambda _*)\) is bijective from \(X_1^2\times {\mathbb {R}}^2\) to \({\mathbb {Y}}^2\). Then from implicit function theorem, there exist \(\lambda ^*>\lambda _*\) and a continuously differential mapping \(\lambda \mapsto (\xi _{\lambda },\eta _{\lambda },\beta _{\lambda },c_{\lambda })\) from \([\lambda _*,\lambda ^*]\) to \(X_1^2\times {\mathbb {R}}^+ \times {\mathbb {R}}^+\) such that

$$\begin{aligned} {\mathcal {F}}(\xi _{\lambda },\eta _{\lambda },\beta _{\lambda },c_{\lambda },\lambda )=0,\ \ \lambda \in [\lambda _*,\lambda ^*]. \end{aligned}$$

Therefore, \((u_{\lambda },v_{\lambda })\) is a positive solution of Eq. (2.1). The proof is completed. \(\square \)

3 Eigenvalue Problems

In this section, we will study the eigenvalue problem associated with the positive steady state \(U_{\lambda }=(u_{\lambda },v_{\lambda })^T\) defined in Theorem 1. Unless otherwise specified, we always assume that \(\lambda \in [\lambda _*,\lambda ^*]\) with \(0<\lambda ^*-\lambda _* \ll 1\), and \(\mathbf{(H_1)}-\mathbf{(H_3)}\) hold. Linearizing system (1.5) at \(U_{\lambda }\), we obtain

$$\begin{aligned} {\left\{ \begin{array}{ll} \dfrac{\partial u}{\partial t}=e^{-\alpha m(x)}\nabla \cdot \left[ e^{\alpha m(x)}\nabla u\right] +\lambda f_1\big (x,K_{11}*u_{\lambda },K_{12}*v_{\lambda } \big )u(x,t) &{} \\ \qquad \quad \, \, +\lambda u_{\lambda }\big [r_1^{1\lambda }(x)K_{11}*u(x,t-\tau )+r_2^{1\lambda }(x)K_{12}*v(x,t-\tau )\big ], &{} x\in \varOmega ,t>0, \\ \dfrac{\partial v}{\partial t}=e^{-\alpha m(x)}\nabla \cdot \left[ e^{\alpha m(x)}\nabla v\right] +\lambda f_2\big (x,K_{21}*u_{\lambda },K_{22}*v_{\lambda } \big )v(x,t) &{} \\ \qquad \quad \, \, +\lambda v_{\lambda }\big [r_1^{2\lambda }(x)K_{21}*u(x,t-\tau )+r_2^{2\lambda }(x)K_{22}*v(x,t-\tau )\big ], &{} x\in \varOmega ,t>0, \\ u(x,t)=v(x,t)=0 &{} x\in \partial \varOmega ,t>0, \end{array}\right. } \end{aligned}$$
(3.1)

where

$$\begin{aligned} r^{i\lambda }_1(x)=\frac{\partial f_i}{\partial s_1}\big (x,K_{i1}*u_{\lambda },K_{i2}*v_{\lambda } \big ),\ \ r^{i\lambda }_2(x)=\frac{\partial f_i}{\partial s_2}\big (x,K_{i1}*u_{\lambda },K_{i2}*v_{\lambda } \big ) \end{aligned}$$

are the Fréchet derivative of \(f_i(i=1,2)\) about the second term and the third term respectively. Define two linear operators \(A_\lambda :{\mathbb {X}}_{{\mathbb {C}}}^2\rightarrow {\mathbb {Y}}_{{\mathbb {C}}}^2\) and \(B_\lambda :{\mathbb {Y}}_{{\mathbb {C}}}^2\rightarrow {\mathbb {Y}}_{{\mathbb {C}}}^2\) by

$$\begin{aligned} A_\lambda :=\begin{pmatrix} \begin{array}{l} e^{-\alpha m(x)}\nabla \cdot \left[ e^{\alpha m(x)}\nabla \right] \\ \ \ +\lambda f_1\big (x,K_{11}*u_{\lambda },K_{12}*v_{\lambda } \big ) \end{array} &{} 0 \\ 0 &{} \begin{array}{l} e^{-\alpha m(x)}\nabla \cdot \left[ e^{\alpha m(x)}\nabla \right] \\ \ \ +\lambda f_2\big (x,K_{21}*u_{\lambda },K_{22}*v_{\lambda } \big ) \end{array} \end{pmatrix} \end{aligned}$$

and

$$\begin{aligned} B_\lambda \psi :=\begin{pmatrix} \lambda u_{\lambda }(r_1^{1\lambda }(x)K_{11}*\psi _1+r_2^{1\lambda }(x)K_{12}*\psi _2) \\ \lambda v_{\lambda }(r_1^{2\lambda }(x)K_{21}*\psi _1+r_2^{2\lambda }(x)K_{22}*\psi _2) \end{pmatrix} \end{aligned}$$

for \(\psi =(\psi _1,\psi _2)^T\in {\mathbb {Y}}_{{\mathbb {C}}}^2\). Then \(A_\lambda \) is an infinitesimal generator of a compact \(C_0\) semigroup [28]. From [32], the solution semigroup of Eq. (3.1) has an infinitesimal generator \(A_{\tau ,\lambda }\) defined by

$$\begin{aligned} A_{\tau ,\lambda }\psi ={\dot{\psi }} \end{aligned}$$
(3.2)

with the domain

$$\begin{aligned} {\mathscr {D}}(A_{\tau ,\lambda })=\left\{ \psi \in C_{{\mathbb {C}}}\cap C_{{\mathbb {C}}}^1:\psi (0)\in {\mathbb {X}}_{{\mathbb {C}}}^2,{\dot{\psi }}(0)=A_\lambda \psi (0)+B_\lambda \psi (-\tau ) \right\} , \end{aligned}$$

where \(C_{{\mathbb {C}}}^1=C^1([-\tau ,0],{\mathbb {X}}_{{\mathbb {C}}}^2)\). Now the stability of \((u_{\lambda },v_{\lambda })\) is determined by the point spectrum of \(A_{\tau ,\lambda }\), which is

$$\begin{aligned} \sigma _p(A_{\tau ,\lambda })=\left\{ \mu \in {\mathbb {C}}: \varLambda (\lambda ,\mu ,\tau )\psi =0\ \mathrm {for\ some}\ \psi \in {\mathbb {X}}_{{\mathbb {C}}}^2\setminus \{(0,0)\} \right\} , \end{aligned}$$

with

$$\begin{aligned} \varLambda (\lambda ,\mu ,\tau )\psi =A_{\lambda }\psi +B_{\lambda }\psi e^{-\mu \tau }-\mu \psi . \end{aligned}$$
(3.3)

As in [14], we also need to consider the adjoint operator \(A_{\tau ,\lambda }^*\) of \(A_{\tau ,\lambda }\) in the sense of weighted inner product, which is defined as

$$\begin{aligned} A_{\tau ,\lambda }^*\psi =-{\dot{\psi }} \end{aligned}$$
(3.4)

with the domain

$$\begin{aligned} {\mathscr {D}}(A_{\tau ,\lambda }^*)=\left\{ \psi =(\psi _1,\psi _2)^T\in C^1([0,\tau ],{\mathbb {X}}_{{\mathbb {C}}}^2):\psi (0)\in {\mathbb {X}}_{{\mathbb {C}}}^2,-{\dot{\psi }}(0)=A_\lambda \psi (0)+B_\lambda ^*\psi (\tau ) \right\} , \end{aligned}$$

where \(B_\lambda ^*:{\mathbb {Y}}^2 \rightarrow {\mathbb {Y}}^2\) is given by

$$\begin{aligned} B_{\lambda }^*\psi =\lambda \begin{pmatrix} \int _{\varOmega }r_1^{1\lambda }(x){\widetilde{K}}_{11}(x,\cdot )\psi _1(x)u_{\lambda }(x)\mathrm {d}x+\int _{\varOmega }r_1^{2\lambda }(x){\widetilde{K}}_{21}(x,\cdot )\psi _2(x)v_{\lambda }(x)\mathrm {d}x \\ \int _{\varOmega }r^{1\lambda }_2(x){\widetilde{K}}_{12}(x,\cdot )\psi _1(x)u_{\lambda }(x)\mathrm {d}x+\int _{\varOmega }r^{2\lambda }_2(x){\widetilde{K}}_{22}(x,\cdot )\psi _2(x)v_{\lambda }(x)\mathrm {d}x \end{pmatrix} \end{aligned}$$

for \(\psi =(\psi _1,\psi _2)^T\in {\mathbb {Y}}^2\) with \({\widetilde{K}}_{ij}(x,\cdot )=k_{ij}(x,\cdot )e^{\alpha m(x)}\) for \(i,j=1,2\). The spectral set of \(A_{\tau ,\lambda }^*\) is

$$\begin{aligned} \sigma (A_{\tau ,\lambda }^*)=\left\{ \mu \in {\mathbb {C}}: {{\tilde{\varLambda }}}(\lambda ,\mu ,\tau )\psi =0\ \mathrm {for\ some}\ \psi \in {\mathbb {X}}_{{\mathbb {C}}}^2\setminus \{(0,0)\} \right\} , \end{aligned}$$

with

$$\begin{aligned} {{\tilde{\varLambda }}}(\lambda ,\mu ,\tau )\psi =A_{\lambda }\psi +B_{\lambda }^*\psi e^{-\mu \tau }-\mu \psi . \end{aligned}$$
(3.5)

One can easily check that

$$\begin{aligned} \left\langle {{\tilde{\psi }}},\varLambda (\lambda ,\mu ,\tau )\psi \right\rangle _w =\left\langle {{\tilde{\varLambda }}}(\lambda ,{{\bar{\mu }}},\tau ){{\tilde{\psi }}},\psi \right\rangle _w. \end{aligned}$$
(3.6)

In the sense of weighted inner product, \({{\tilde{\varLambda }}}(\lambda ,{{\bar{\mu }}},\tau )\) is the adjoint operator of \(\varLambda (\lambda ,\mu ,\tau )\) and they has the same point spectrum, i.e.

$$\begin{aligned} \sigma (\varLambda (\lambda ,\mu ,\tau )) =\sigma ({{\tilde{\varLambda }}}(\lambda ,{{\bar{\mu }}},\tau )), \end{aligned}$$

which means \(\mu \in \sigma (A_{\tau ,\lambda })\) if and only if \({{\bar{\mu }}}\in \sigma (A_{\tau ,\lambda }^*)\).

Theorem 2

Under assumptions \(\mathbf{(H_1)}-\mathbf{(H_3)}\), for \(\lambda \in [\lambda _*,\lambda ^*]\) and \(\tau \ge 0\), 0 is not an eigenvalue of \(A_{\tau ,\lambda }\).

Proof

Suppose to the contrary that 0 is an eigenvalue of \(A_{\tau ,\lambda }\), then there exists \(\psi \in {\mathbb {X}}_{{\mathbb {C}}}^2\setminus \{(0,0)\}\) such that

$$\begin{aligned} \varLambda (\lambda ,0,\tau )\psi =0. \end{aligned}$$
(3.7)

By the decomposition \({\mathbb {X}}^2={\mathscr {N}}({\mathcal {L}}_{\lambda _*})\oplus X_1^2\), \(\psi \) takes the following form

$$\begin{aligned} \psi (x)={\tilde{a}}(\lambda )\phi (x)+(\lambda -\lambda _*)b(\lambda ,x) \end{aligned}$$
(3.8)

where \({\tilde{a}}(\lambda )\in {\mathbb {R}}^2\) and \(b(\lambda ,\cdot )\in X_1^2\). Then substituting (3.8) to (3.7) gives

$$\begin{aligned} \varLambda (\lambda ,0,\tau ){\tilde{a}}(\lambda )\phi +(\lambda -\lambda _*)\varLambda (\lambda ,0,\tau )b(\lambda ,\cdot )=0. \end{aligned}$$
(3.9)

Since \((u_{\lambda },v_{\lambda })\rightarrow (0,0)\) as \(\lambda \rightarrow \lambda _*\), then \(r^{i\lambda }_j(x)\rightarrow r^i_j(x)\) uniformly in \(\varOmega \) as \(\lambda \rightarrow \lambda _*\). A straightforward calculation yields

$$\begin{aligned} \lim _{\lambda \rightarrow \lambda _*}\dfrac{\varLambda (\lambda ,0,\tau ){\tilde{a}}(\lambda )\phi }{\lambda -\lambda _*}=m(x)\phi {\tilde{a}}(\lambda _*)+\lambda _* \phi \overline{{\mathcal {K}}}{\tilde{a}}(\lambda _*)+\lambda _* \phi \varrho \widetilde{{\mathcal {K}}} {\tilde{a}}(\lambda _*), \end{aligned}$$

where

$$\begin{aligned} \overline{{\mathcal {K}}}=\begin{pmatrix} \beta _{\lambda _*}r^1_1{\widetilde{k}}_{11}+c_{\lambda _*}r^1_2{\widetilde{k}}_{12} &{} 0 \\ 0 &{} \beta _{\lambda _*}r^2_1{\widetilde{k}}_{21}+c_{\lambda _*}r^2_2{\widetilde{k}}_{22} \end{pmatrix},\ \varrho =\begin{pmatrix} \beta _{\lambda _*} &{} 0 \\ 0 &{} c_{\lambda _*} \end{pmatrix},\ \widetilde{{\mathcal {K}}}=\begin{pmatrix} r^1_1{\widetilde{k}}_{11} &{} r^1_2{\widetilde{k}}_{12} \\ r^2_1{\widetilde{k}}_{21} &{} r^2_2{\widetilde{k}}_{22} \end{pmatrix}, \end{aligned}$$

in which \({\widetilde{k}}_{ij}\) is defined as in (2.7). By expanding \({\tilde{a}}(\lambda )\) and \(b(\lambda ,x)\) near \(\lambda _*\), we obtain

$$\begin{aligned} {\tilde{a}}(\lambda )=\sum _{i=0}^{\infty }{\tilde{a}}_i(\lambda -\lambda _*)^i,\ \ b(\lambda ,x)=\sum _{i=0}^{\infty }b_i(x)(\lambda -\lambda _*)^i. \end{aligned}$$
(3.10)

Note that \(\varLambda (\lambda _*,0,\tau )={\mathcal {L}}\), where \({\mathcal {L}}\) is defined as (2.2). Then from (3.9) and (3.10) we see that

$$\begin{aligned} {\mathcal {L}} b_0(\cdot )=-m(x)\phi {\tilde{a}}_0-\lambda _*\phi \overline{{\mathcal {K}}}{\tilde{a}}_0-\lambda _*\phi \varrho \widetilde{{\mathcal {K}}} {\tilde{a}}_0. \end{aligned}$$

Calculating the weighted inner product of above equation with \(\phi \), we have \(\varrho \widetilde{{\mathcal {K}}} {\tilde{a}}_0=0\). Due to the positivity of \(\beta _{\lambda _*},c_{\lambda _*}\) and the assumption \(\mathbf{(H_2)}\), we can deduce that \({\tilde{a}}_0=0\). Since \({\mathcal {L}}_{\lambda _*}|_{X_1^2}: X_1^2 \rightarrow Y_1^2\) is invertible, then \(b_0(x)=0\) for all x. Likewise, considering the term of \((\lambda -\lambda _*)^i(i\ge 1)\), we still have that \({\tilde{a}}_i=0\) and \(b_i(x)=0\) for all x. Consequently, \(\psi =0\) is the unique solution of \(\varLambda (\lambda ,0,\tau )\psi =0\), a contradiction with \(\psi \in {\mathbb {X}}_{{\mathbb {C}}}^2\setminus \{(0,0)\}\). The proof is completed. \(\square \)

In the following, we show the situation when \(A_{\tau ,\lambda }\) has a pair of purely imaginary eigenvalues \(\mu =\pm i\omega (\omega >0)\) for some \(\tau >0\). From previous argument, \(\mu =i\omega \in \sigma _p(A_{\tau ,\lambda })\) for some \(\tau >0\) if and only if

$$\begin{aligned} A_{\lambda }\psi +B_{\lambda }\psi e^{-i\theta }-i\omega \psi =0 \end{aligned}$$
(3.11)

is solvable for some value of \(\omega >0, \theta \in [0,2\pi )\) and \(\psi \in {\mathbb {X}}_{{\mathbb {C}}}^2\setminus \{(0,0)\}\), where \(\theta :=\omega \tau \). We first show the following lemma for further discussion.

Lemma 1

Recall that \(\lambda _*\) is the principal eigenvalue of problem (1.6), the following results hold:

  1. (i)

    if \(z\in {\mathbb {X}}_{{\mathbb {C}}}\) and \(\langle \phi ,z \rangle _w=0\), then \(|\langle Lz,z \rangle _w|\ge \lambda _2\Vert z\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}^2\), where the operator \(L:{\mathbb {X}}\rightarrow {\mathbb {Y}}\) is defined by

    $$\begin{aligned}L=e^{-\alpha m(x)}\nabla \cdot [e^{\alpha m(x)}\nabla ]+\lambda _*m(x)\end{aligned}$$

    and \(\lambda _2\) is the second eigenvalue of operator \(-L\);

  2. (ii)

    if \((\omega ,\theta ,\psi )\) is a solution of Eq. (3.11) with \(\omega >0,\theta \in [0,2\pi )\) and \(\psi \in {\mathbb {X}}_{{\mathbb {C}}}^2\setminus \{(0,0)\}\), then \(\dfrac{\omega }{\lambda -\lambda _*}\) is bounded for \(\lambda \in (\lambda _*,\lambda ^*]\).

Proof

Part (i) can be proved as in [8, Lemma 2.3]. We only discuss part (ii). By calculating the weighted inner product of Eq. (3.11) with \(\psi \), one can obtain

$$\begin{aligned} \left\langle \psi ,A_{\lambda }\psi +B_{\lambda }\psi e^{-i\theta }-i\omega \psi \right\rangle _w=0. \end{aligned}$$
(3.12)

Choose some \(\theta _0\in [0,2\pi )\) such that

$$\begin{aligned} \left\langle \psi ,B_{\lambda }\psi e^{-i\theta } \right\rangle _w=\left| \Big \langle \psi ,B_{\lambda }\psi \Big \rangle _w \right| e^{i(\theta _0-\theta )}. \end{aligned}$$

Note that \(A_{\lambda }\) is self-adjoint in the sense of weighted inner product, then \(\big \langle \psi ,A_{\lambda } \psi \big \rangle _w\) is real. Separating the real and imaginary parts of (3.12) gives that

$$\begin{aligned} \omega \big \langle \psi ,\psi \big \rangle _w=\left| \big \langle \psi ,B_{\lambda }\psi \big \rangle _w \right| \sin (\theta _0-\theta ). \end{aligned}$$

Therefore,

$$\begin{aligned} \left| \frac{\omega }{\lambda -\lambda _*} \right|&=\dfrac{\left| \lambda \sin (\theta _0-\theta ) \left\langle \psi , \begin{pmatrix} \beta _{\lambda }[\phi +(\lambda -\lambda _*)\xi _{\lambda }](r^{1\lambda }_1(x)K_{11}*\psi _1+r^{1\lambda }_2(x)K_{12}*\psi _2) \\ c_{\lambda }[\phi +(\lambda -\lambda _*)\eta _{\lambda }](r^{2\lambda }_1(x)K_{21}*\psi _1+r^{2\lambda }_2(x)K_{22}*\psi _2) \end{pmatrix} \right\rangle _w \right| }{\left\langle \psi ,\psi \right\rangle _w} \\&\le 2\lambda M_1M_2 e^{\max _{\varOmega }(\alpha m(x))} \{ |K_{11}|, |K_{12}|, |K_{21}|, |K_{22}| \}|\varOmega |, \end{aligned}$$

where

$$\begin{aligned} M_1= & {} \max \{ \beta _{\lambda }\left[ \Vert \phi \Vert _{\infty }+(\lambda -\lambda _*)\Vert \xi _{\lambda }\Vert _{\infty } \right] ,c_{\lambda }\left[ \Vert \phi \Vert _{\infty }+(\lambda -\lambda _*)\Vert \eta _{\lambda }\Vert _{\infty } \right] \}, \\ M_2= & {} \max \{\Vert r^{1\lambda }_1(x)\Vert _{\infty },\Vert r^{1\lambda }_2(x)\Vert _{\infty },\Vert r^{2\lambda }_1(x)\Vert _{\infty },\Vert r^{2\lambda }_2(x)\Vert _{\infty }\}. \end{aligned}$$

Since \((u_{\lambda },v_{\lambda })\) is bounded for \(\lambda \in (\lambda _*,\lambda ^*]\), then there is a constant \(M>0\) such that \(\Vert r^{i\lambda }_j\Vert _{\infty }<M (i,j=1,2)\). Now, the boundedness of \(\dfrac{\omega }{\lambda -\lambda _*}\) for \(\lambda \in (\lambda _*,\lambda ^*]\) can be obtained from the continuity of \(\lambda \mapsto (\beta _{\lambda }, c_{\lambda }, \Vert \xi _{\lambda }\Vert _{\infty }, \Vert \eta _{\lambda }\Vert _{\infty })\). The proof is finished. \(\square \)

By the decomposition \({\mathbb {X}}^2={\mathscr {N}}({\mathcal {L}}_{\lambda _*})\oplus X_1^2\), for \(\lambda \in (\lambda _*,\lambda ^*]\), ignoring a scalar factor, we can rewrite \(\psi =(\psi _1,\psi _2)^T\) in (3.11) as the form

$$\begin{aligned} \psi _1= & {} \phi +(\lambda -\lambda _*)z_1,\ \ \langle \phi ,z_1 \rangle _w=0, \nonumber \\ \psi _2= & {} (p_1+ip_2)\phi +(\lambda -\lambda _*)z_2,\ \ \langle \phi ,z_2 \rangle _w=0, \ p_1>0. \end{aligned}$$
(3.13)

Setting \(h:=\omega /(\lambda -\lambda _*)\), and substituting (2.3), (3.13) and \(\omega =(\lambda -\lambda _*)h\) into Eq. (3.11), then Eq. (3.11) can be transformed as the following equivalent system:

$$\begin{aligned} g_1(z_1,z_2,p_1,p_2,h,\theta ,\lambda ):= & {} Lz_1+\Big \{ m(x)-i h +\lambda h_1(\xi _{\lambda },\eta _{\lambda },\beta _{\lambda },c_{\lambda },\lambda ) \Big \}\cdot \big [ \phi +(\lambda -\lambda _*)z_1 \big ] \nonumber \\&+\Big \{ r^{1\lambda }_1(x)K_{11}*\big [ \phi +(\lambda -\lambda _*)z_1 \big ]\nonumber \\&+r^{1\lambda }_2(x)K_{12}*\big [ (p_1+ip_2)\phi +(\lambda -\lambda _*)z_2 \big ] \Big \} \nonumber \\&\cdot \lambda \beta _{\lambda }[\phi +(\lambda -\lambda _*)\xi _{\lambda }] e^{-i\theta }=0, \nonumber \\ g_2(z_1,z_2,p_1,p_2,h,\theta ,\lambda ):= & {} Lz_2+\Big \{ m(x)-i h +\lambda h_2(\xi _{\lambda },\eta _{\lambda },\beta _{\lambda },c_{\lambda },\lambda ) \Big \} \nonumber \\&\cdot \big [(p_1+ip_2)\phi +(\lambda -\lambda _*)z_2 \big ] \nonumber \\&+\Big \{ r^{2\lambda }_1(x)K_{21}*\big [\phi +(\lambda -\lambda _*)z_1 \big ]\nonumber \\&+r^{2\lambda }_2(x)K_{22}*\big [(p_1+ip_2)\phi +(\lambda -\lambda _*)z_2 \big ] \Big \} \nonumber \\&\cdot \lambda c_{\lambda }\big [\phi +(\lambda -\lambda _*)\eta _{\lambda } \big ] e^{-i\theta }=0, \end{aligned}$$
(3.14)

where \(h_i(\xi _{\lambda },\eta _{\lambda },\beta _{\lambda },c_{\lambda },\lambda )\) is defined in (2.6), and the operator L is defined as in Lemma 1. Define \(G:X_1^2\times {\mathbb {R}}^4\times {\mathbb {R}}\rightarrow {\mathbb {Y}}_{{\mathbb {C}}}^2\) by

$$\begin{aligned} G(z_1,z_2,p_1,p_2,h,\theta ,\lambda ):=(g_1,g_2). \end{aligned}$$

Now, we show that \(G(z_1,z_2,p_1,p_2,h,\theta ,\lambda )=0\) is uniquely solvable when \(\lambda =\lambda _*\).

Lemma 2

Under assumptions \(\mathbf{(H_1)}-\mathbf{(H_3)}\), the equation

$$\begin{aligned} {\left\{ \begin{array}{ll} G(z_1,z_2,p_1,p_2,h,\theta ,\lambda _*)=0, \\ z_1,z_2\in X_{1{\mathbb {C}}},\ h\ge 0,\ p_1\ge 0,\ \theta \in [0,2\pi ) \end{array}\right. } \end{aligned}$$

has a unique solution \((z_{1\lambda _*},z_{2\lambda _*},p_{1\lambda _*},p_{2\lambda _*},h_{\lambda _*},\theta _{\lambda _*})\) satisfying that \(p_{2\lambda _*}=0, \theta _{\lambda _*}=\dfrac{\pi }{2}\), \(p_{1\lambda _*}\) is the positive root of the following equation

$$\begin{aligned} \kappa _{12}(\kappa _{12}-\kappa _{22})p^2+(\kappa _{11}\kappa _{12}-\kappa _{22}\kappa _{21})p-\kappa _{21}(\kappa _{21}-\kappa _{11})=0,\end{aligned}$$

and

$$\begin{aligned} h_{\lambda _*}&=\dfrac{(\kappa _{22}-\kappa _{12})(\kappa _{11}+\kappa _{12}p_{1\lambda _*})\displaystyle \int _{\varOmega }m(x)e^{\alpha m(x)}\phi ^2 dx}{(\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21})\displaystyle \int _{\varOmega }e^{\alpha m(x)}\phi ^2 dx} \\&=\dfrac{(\kappa _{11}-\kappa _{21})(\kappa _{21}+\kappa _{22}p_{1\lambda _*})\displaystyle \int _{\varOmega }m(x)e^{\alpha m(x)}\phi ^2 dx}{p_{1\lambda _*}(\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21})\displaystyle \int _{\varOmega }e^{\alpha m(x)}\phi ^2 dx}, \end{aligned}$$

and \((z_{1\lambda _*},z_{2\lambda _*})^T\in X_{1{\mathbb {C}}}^2\) is the unique solution of

$$\begin{aligned} {\mathcal {L}}\begin{pmatrix} z_{1\lambda _*} \\ z_{2\lambda _*} \end{pmatrix}&=-m(x)\phi \begin{pmatrix} 1 \\ p_{1\lambda _*} \end{pmatrix} -\lambda _*\phi \begin{pmatrix} \beta _{\lambda _*}r^1_1(x){\widetilde{k}}_{11}(x)+r^1_2(x){\widetilde{k}}_{12}(x) \\ p_{1\lambda _*}\big ( \beta _{\lambda _*}r^2_1(x){\widetilde{k}}_{21}(x)+r^2_2(x){\widetilde{k}}_{22}(x) \big ) \end{pmatrix} \\&\quad \, \, +i\lambda _*\phi \begin{pmatrix} \beta _{\lambda _*}\big (r^1_1(x){\widetilde{k}}_{11}(x)+p_{1\lambda _*}r^1_2(x){\widetilde{k}}_{12}(x) \big ) \\ c_{\lambda _*}\big (r^2_1(x){\widetilde{k}}_{21}(x)+p_{1\lambda _*}r^2_2(x){\widetilde{k}}_{22}(x) \big ) \end{pmatrix} +ih_{\lambda _*} \phi \begin{pmatrix} 1 \\ p_{1\lambda _*} \end{pmatrix}, \end{aligned}$$

where \({\mathcal {L}}\) is defined as (2.2)

Proof

When \(\lambda =\lambda _*\), we have

$$\begin{aligned} g_1(z_1,z_2,p_1,p_2,h,\theta ,\lambda _*)&:=Lz_1+\Big \{ m(x)-i h +\lambda _* \big ( \beta _{\lambda _*}r^1_1(x){\widetilde{k}}_{11}(x)+r^1_2(x){\widetilde{k}}_{12}(x) \big ) \Big \}\cdot \phi \\&\qquad +\Big \{ r^1_1(x){\widetilde{k}}_{11}(x)+(p_1+ip_2)r^1_2(x){\widetilde{k}}_{12}(x) \Big \}\cdot \lambda _* \beta _{\lambda _*}\phi e^{-i\theta }=0, \\ g_2(z_1,z_2,p_1,p_2,h,\theta ,\lambda _*)&:=Lz_2+\Big \{ m(x)-i h +\lambda _* \big ( \beta _{\lambda _*}r^2_1(x){\widetilde{k}}_{21}(x)+r^2_2(x){\widetilde{k}}_{22}(x) \big ) \Big \} \cdot (p_1+ip_2)\phi \\&\qquad +\Big \{ r^2_1(x){\widetilde{k}}_{21}(x)+(p_1+ip_2)r^2_2(x){\widetilde{k}}_{22}(x) \Big \}\cdot \lambda _* c_{\lambda _*}\phi e^{-i\theta }=0. \end{aligned}$$

Then

$$\begin{aligned} {\left\{ \begin{array}{ll} G(z_1,z_2,p_1,p_2,h,\theta ,\lambda _*)=0, \\ z_1,z_2\in X_{1{\mathbb {C}}},\ h\ge 0,\ p_1\ge 0,\ \theta \in [0,2\pi ) \end{array}\right. } \end{aligned}$$

is solvable if and only if

$$\begin{aligned}&\left\langle \phi ,\Big \{ m(x)-i h +\lambda _* \big ( \beta _{\lambda _*}r^1_1(x){\widetilde{k}}_{11}(x)+r^1_2(x){\widetilde{k}}_{12}(x) \big ) \Big \}\cdot \phi \right. \\&\quad \left. +\Big \{ r^1_1(x){\widetilde{k}}_{11}(x)+(p_1+ip_2)r^1_2(x){\widetilde{k}}_{12}(x) \Big \}\cdot \lambda _* \beta _{\lambda _*}\phi e^{-i\theta } \right\rangle _w=0, \\&\quad \left\langle \phi ,\Big \{ m(x)-i h +\lambda _* \big ( \beta _{\lambda _*}r^2_1(x){\widetilde{k}}_{21}(x)+r^2_2(x){\widetilde{k}}_{22}(x) \big ) \Big \} \cdot (p_1+ip_2)\phi \right. \\&\quad \left. +\Big \{ r^2_1(x){\widetilde{k}}_{21}(x)+(p_1+ip_2)r^2_2(x){\widetilde{k}}_{22}(x) \Big \}\cdot \lambda _* c_{\lambda _*}\phi e^{-i\theta } \right\rangle _w=0. \end{aligned}$$

That is,

$$\begin{aligned} \begin{aligned} ih\int _{\varOmega }e^{\alpha m(x)}\phi ^2dx&=(\cos \theta -i\sin \theta )\lambda _*\beta _{\lambda _*}\big ( \kappa _{11}+\kappa _{12}(p_1+ip_2) \big ), \\ ih(p_1+ip_2)\int _{\varOmega }e^{\alpha m(x)}\phi ^2dx&=(\cos \theta -i\sin \theta )\lambda _* c_{\lambda _*}\big ( \kappa _{21}+\kappa _{22}(p_1+ip_2) \big ). \end{aligned} \end{aligned}$$
(3.15)

From (3.15), we see that \(p_1+ip_2\) is a root of

$$\begin{aligned} \kappa _{12}(\kappa _{12}-\kappa _{22})p^2+(\kappa _{11}\kappa _{12}-\kappa _{22}\kappa _{21})p-\kappa _{21}(\kappa _{21}-\kappa _{11})=0. \end{aligned}$$

Due to assumptions \(\mathbf{(H_2)},\mathbf{(H_3)}\) and \(p_1\ge 0\), there holds that \(p_2=p_{2\lambda _*}=0\) and \(p_1=p_{1\lambda _*}\) is the unique positive root of above quadratic equation. Now, it can be derived from (3.15) that \(\theta =\theta _{\lambda _*}=\frac{\pi }{2}\) and

$$\begin{aligned} h=h_{\lambda _*}&=\dfrac{(\kappa _{22}-\kappa _{12})(\kappa _{11}+\kappa _{12}p_{1\lambda _*})\displaystyle \int _{\varOmega }m(x)e^{\alpha m(x)}\phi ^2 dx}{(\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21})\displaystyle \int _{\varOmega }e^{\alpha m(x)}\phi ^2 dx} \\&=\dfrac{(\kappa _{11}-\kappa _{21})(\kappa _{21}+\kappa _{22}p_{1\lambda _*})\displaystyle \int _{\varOmega }m(x)e^{\alpha m(x)}\phi ^2 dx}{p_{1\lambda _*}(\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21})\displaystyle \int _{\varOmega }e^{\alpha m(x)}\phi ^2 dx}. \end{aligned}$$

The proof is finished. \(\square \)

In what follows, we will provide the solvability result of the equation \(G=0\) for \(\lambda \) near \(\lambda _*\) by applying the implicit function theorem.

Theorem 3

Assume that the assumptions \(\mathbf{(H_1)}-\mathbf{(H_3)}\) hold, then the following statements are true:

  1. (i)

    there exist \({{\tilde{\lambda }}}^*>\lambda _*\) and a continuously differentiable mapping \(\lambda \mapsto (z_{1\lambda },z_{2\lambda },p_{1\lambda },p_{2\lambda },h_{\lambda },\theta _{\lambda })\) from \([\lambda _*,{{\tilde{\lambda }}}^*]\) to \(X_{1{\mathbb {C}}}^2\times {\mathbb {R}}^4\) such that \(G(z_{1\lambda },z_{2\lambda },p_{1\lambda },p_{2\lambda },h_{\lambda },\theta _{\lambda },\lambda )=0\);

  2. (ii)

    if \((z_1^{\lambda },z_{2}^{\lambda },r_1^{\lambda },r_2^{\lambda },h^{\lambda },\theta ^{\lambda })\) with \(h^{\lambda }>0,\theta ^{\lambda }>0\) solves \(G(\cdot ,\lambda )=0\) for \(\lambda \in [\lambda _*,{{\tilde{\lambda }}}^*]\), then \((z_1^{\lambda },z_2^{\lambda },r_1^{\lambda },r_2^{\lambda },\) \(h^{\lambda },\theta ^{\lambda })=(z_{1\lambda },z_{2\lambda },r_{1\lambda },r_{2\lambda },h_{\lambda },\theta _{\lambda })\).

Proof

Denote by \(T=(T_1,T_2): X_{1{\mathbb {C}}}^2\times {\mathbb {R}}^4\rightarrow {\mathbb {Y}}_{{\mathbb {C}}}^2\) the Fréchet derivative of G with respect to \((z_1,z_2,p_1,p_2,h,\theta )\) evaluated at \((z_{1\lambda _*},z_{2\lambda _*},p_{1\lambda _*},p_{2\lambda _*},h_{\lambda _*},\theta _{\lambda _*},\lambda _*)\). A direct calculation gives

$$\begin{aligned} T_1(\chi _1,\chi _2,\nu _1,\nu _2,\epsilon ,\vartheta )&=L\chi _1-i\epsilon \phi -\vartheta \lambda _*\beta _{\lambda _*}\phi \Big [r^1_1(x){\widetilde{k}}_{11}(x)+p_{1\lambda _*}r^1_2(x){\widetilde{k}}_{12}(x) \Big ] \\&\quad \, \, -(i\nu _1-\nu _2)\lambda _*\beta _{\lambda _*}\phi r^1_2(x){\widetilde{k}}_{12}(x), \\ T_2(\chi _1,\chi _2,\nu _1,\nu _2,\epsilon ,\vartheta )&=L\chi _2-i\epsilon p_{1\lambda _*}\phi -\vartheta \lambda _* c_{\lambda _*}\phi \Big [r^2_1(x){\widetilde{k}}_{21}(x)+p_{1\lambda _*}r^2_2(x){\widetilde{k}}_{22}(x) \Big ] \\&\quad \, \, +(\nu _1+i\nu _2)\phi \left[ m(x)-ih_{\lambda _*}+\lambda _*\big (\beta _{\lambda _*}r^2_1(x){\widetilde{k}}_{21}(x) \right. \\&\quad \, \, \left. +c_{\lambda _*}r^2_2(x){\widetilde{k}}_{22}(x)\big )-i \lambda _*c_{\lambda _*}r^2_2(x){\widetilde{k}}_{22}(x) \right] . \end{aligned}$$

Next, we show that T is a bijection from \(X_{1{\mathbb {C}}}^2\times {\mathbb {R}}^4\) to \({\mathbb {Y}}_{{\mathbb {C}}}^2\). We first prove T is injective. If \(T(\chi _1,\chi _2,\nu _1,\nu _2,\epsilon ,\vartheta )=0\), then

$$\begin{aligned}&\displaystyle -i\epsilon \int _{\varOmega }e^{\alpha m(x)}\phi ^2\mathrm {d}x-\vartheta \lambda _*\beta _{\lambda _*}(\kappa _{11}+p_{1\lambda _*}\kappa _{12})=(i\nu _1-\nu _2)\lambda _*\beta _{\lambda _*}\kappa _{12}, \nonumber \\&\qquad \displaystyle -i\epsilon p_{1\lambda _*}\int _{\varOmega }e^{\alpha m(x)}\phi ^2\mathrm {d}x-\vartheta \lambda _*c_{\lambda _*}(\kappa _{21}+p_{1\lambda _*}\kappa _{22})\nonumber \\&\quad =(i\nu _1-\nu _2)(h_{\lambda _*}\int _{\varOmega }e^{\alpha m(x)}\phi ^2\mathrm {d}x+\lambda _*c_{\lambda _*}\kappa _{22}). \end{aligned}$$
(3.16)

It can be seen from (3.15) that

$$\begin{aligned} -p_{1\lambda _*}h_{\lambda _*}\int _{\varOmega }e^{\alpha m(x)}\phi ^2\mathrm {d}x=p_{1\lambda _*}\lambda _*\beta _{\lambda _*}(\kappa _{11}+p_{1\lambda _*}\kappa _{12})=\lambda _*c_{\lambda _*}(\kappa _{21}+p_{1\lambda _*}\kappa _{22}).\nonumber \\ \end{aligned}$$
(3.17)

This result combined with (3.16) leads to that

$$\begin{aligned} (i\nu _1-\nu _2)\left[ h_{\lambda _*}\int _{\varOmega }e^{\alpha m(x)}\phi ^2\mathrm {d}x+\lambda _*c_{\lambda _*}\kappa _{22}-p_{1\lambda _*}\lambda _*\beta _{\lambda _*}\kappa _{12} \right] =0. \end{aligned}$$

Then from assumption \(\mathbf{(H_2)},\mathbf{(H_3)}\), (3.17) and the definitions of \(\beta _{\lambda _*},c_{\lambda _*}\) and \(h_{\lambda _*}\), we obtain that

$$\begin{aligned}&h_{\lambda _*}\int _{\varOmega }e^{\alpha m(x)}\phi ^2\mathrm {d}x+\lambda _*c_{\lambda _*}\kappa _{22}-p_{1\lambda _*}\lambda _*\beta _{\lambda _*}\kappa _{12} \\&\quad = \frac{\kappa _{12}(\kappa _{22}-\kappa _{12})p_{1\lambda _*}^2+\kappa _{21}(\kappa _{11}-\kappa _{21})}{p_{1\lambda _*}(\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21})}\int _{\varOmega }m(x)e^{\alpha m(x)}\phi ^2\mathrm {d}x\ne 0, \end{aligned}$$

which implies that \(i\nu _1-\nu _2=0\), i.e. \(\nu _1=\nu _2=0\). Substituting \(i\nu _1-\nu _2=0\) into (3.16), we must have \(\epsilon =0\) and \(\vartheta =0\). Consequently, \(\chi _1=\chi _2=0\). So, T is injective from \(X_{1{\mathbb {C}}}^2\times {\mathbb {R}}^4\) to \({\mathbb {Y}}_{{\mathbb {C}}}^2\). By a similar manner to the proof of \(D_{(\xi ,\eta ,\beta ,c)}{\mathcal {F}}(\xi _{\lambda _*},\eta _{\lambda _*},\beta _{\lambda _*},c_{\lambda _*},\lambda _*)\) in Theorem 1, we can also prove T is surjective. Now, we have prove T is bijective. Therefore, part (i) can be obtained from the implicit function theorem immediately.

To show part (ii), we will check that if \(G(z_1^{\lambda },z_2^{\lambda },p_1^{\lambda },p_2^{\lambda },h^{\lambda },\theta ^{\lambda },\lambda )=0\) for \(h^{\lambda }>0,\theta ^{\lambda }\in [0,2\pi )\), then

$$\begin{aligned} (z_1^{\lambda },z_2^{\lambda },p_1^{\lambda },p_2^{\lambda },h^{\lambda },\theta ^{\lambda }) \rightarrow (z_{1\lambda _*},z_{2\lambda _*},p_{1\lambda _*},p_{2\lambda _*},h_{\lambda _*},\theta _{\lambda _*}) \end{aligned}$$

as \(\lambda \rightarrow \lambda _*\) in the norm of \(X_{1{\mathbb {C}}}^2\times {\mathbb {R}}^4\). First, it follows from Lemma 1 (ii) that \(\{h^{\lambda }\}\) is bounded. As in Theorem 2.4 of [4], due to the boundedness of \(\{h^{\lambda }\},\{\theta ^{\lambda }\},\{\beta _{\lambda }\},\{c_{\lambda }\},\{\xi _{\lambda }\},\{\eta _{\lambda }\}\), there are \(M_1,M_2,M_3>0\) such that

$$\begin{aligned} \begin{aligned} \lambda _2\Vert z_1^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}^2&\le \left| \langle Lz_1^{\lambda },z_1^{\lambda } \rangle _w \right| \\&\le M_1 \Vert \phi \Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}\Vert z_1^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}+\left[ M_2(|p_1^{\lambda }|+|p_2^{\lambda }|)\right. \\&\left. \quad +M_3(\lambda -\lambda _*)(\Vert z_1^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}+\Vert z_2^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}) \right] \Vert z_1^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}, \\ \lambda _2\Vert z_2^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}^2&\le \left| \langle Lz_2^{\lambda },z_2^{\lambda } \rangle _w \right| \\&\le M_1 \Vert \phi \Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}\Vert z_2^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}+\left[ M_2(|p_1^{\lambda }|+|p_2^{\lambda }|)\right. \\&\left. \quad +M_3(\lambda -\lambda _*)(\Vert z_1^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}+\Vert z_2^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}) \right] \Vert z_2^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}, \end{aligned} \end{aligned}$$
(3.18)

where \(\lambda _2\) is defined in Lemma 1 (i). By setting \(0<{{\tilde{\lambda }}}^*-\lambda \ll 1\), we see that

$$\begin{aligned} \Vert z_1^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}+\Vert z_2^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}\le M_4 \Vert \phi \Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}+M_5(|p_1^{\lambda }|+|p_2^{\lambda }|) \end{aligned}$$
(3.19)

for some constants \(M_4,M_5>0\). On the other hand, note that \(\langle \phi , L z_1^{\lambda } \rangle _w=0\), then it follows from the first equation of (3.14) that

$$\begin{aligned}&\bigg \langle \phi , \big [ \phi +(\lambda -\lambda _*)z_1 \big ]\cdot \Big \{ m(x)-i h +\lambda h_1(\xi _{\lambda },\eta _{\lambda },\beta _{\lambda },c_{\lambda },\lambda ) \Big \}+ \lambda \beta _{\lambda }[\phi +(\lambda -\lambda _*)\xi _{\lambda }] e^{-i\theta } \\&\quad \cdot \Big \{ r^{1\lambda }_1(x)K_{11}*\big [ \phi +(\lambda -\lambda _*)z_1 \big ]+r^{1\lambda }_2(x)K_{12}*\big [ (p_1+ip_2)\phi +(\lambda -\lambda _*)z_2 \big ] \Big \} \bigg \rangle _w=0. \end{aligned}$$

By separating the real and imaginary parts of the above identity, we have

$$\begin{aligned} \begin{aligned} |p_1^{\lambda }|&\le M_6 \Vert \phi \Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}+M_7(\lambda -\lambda _*)(\Vert z_1^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}+\Vert z_2^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}), \\ |p_2^{\lambda }|&\le M_6 \Vert \phi \Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}+M_7(\lambda -\lambda _*)(\Vert z_1^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}+\Vert z_2^{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}) \end{aligned} \end{aligned}$$
(3.20)

for some constants \(M_6,M_7>0\). Since \(0<{{\tilde{\lambda }}}^*-\lambda _* \ll 1\), then from (3.19) and (3.20) we get the boundedness of \(\{z_1^{\lambda }\}, \{z_2^{\lambda }\}, \{p_1^{\lambda }\}, \{p_2^{\lambda }\}\). Recall that the operator L has a bounded inverse from \(X_{1{\mathbb {C}}}\) to \(Y_{1{\mathbb {C}}}\). By acting \(L^{-1}\) on \(g_1\left( z_1^{\lambda },z_2^{\lambda },p_1^{\lambda },p_2^{\lambda },h^{\lambda },\theta ^{\lambda },\lambda \right) =0\) and \(g_2\left( z_1^{\lambda },z_2^{\lambda },p_1^{\lambda },p_2^{\lambda },h^{\lambda },\theta ^{\lambda },\lambda \right) =0\), we obtain that \(\{z_1^{\lambda }\}, \{z_2^{\lambda }\}\) are also bounded in \(X_{1{\mathbb {C}}}\) and hence \(\left\{ \left( z_1^{\lambda },z_2^{\lambda },p_1^{\lambda },p_2^{\lambda },h^{\lambda },\theta ^{\lambda } \right) :\lambda \in (\lambda _*,{{\tilde{\lambda }}}^*] \right\} \) is precompact in \({\mathbb {Y}}_{{\mathbb {C}}}^2\times {\mathbb {R}}^4\) due to the embedding theorem. Let \(\left\{ \left( z_1^{\lambda ^n},z_2^{\lambda ^n},p_1^{\lambda ^n},p_2^{\lambda ^n},h^{\lambda ^n},\theta ^{\lambda ^n} \right) \right\} \) be any convergent subsequence satisfying

$$\begin{aligned}&\left( z_1^{\lambda ^n},z_2^{\lambda ^n},p_1^{\lambda ^n},p_2^{\lambda ^n},h^{\lambda ^n},\theta ^{\lambda ^n} \right) \rightarrow \left( z_1^{\lambda _*},z_2^{\lambda _*},p_1^{\lambda _*},p_2^{\lambda _*},h^{\lambda _*},\theta ^{\lambda _*} \right) \ \mathrm {in}\ {\mathbb {Y}}_{{\mathbb {C}}}^2\times {\mathbb {R}}^4,\\&\quad \lambda ^n \rightarrow \lambda _*\ \mathrm {as}\ n\rightarrow \infty . \end{aligned}$$

By taking the limit of equations

$$\begin{aligned} L^{-1}g_1\left( z_1^{\lambda ^n},z_2^{\lambda ^n},r_1^{\lambda ^n},r_2^{\lambda ^n},h^{\lambda ^n},\theta ^{\lambda ^n},\lambda ^n \right) =0 \end{aligned}$$

and

$$\begin{aligned} L^{-1}g_2\left( z_1^{\lambda ^n},z_2^{\lambda ^n},r_1^{\lambda ^n},r_2^{\lambda ^n},h^{\lambda ^n},\theta ^{\lambda ^n},\lambda ^n \right) =0 \end{aligned}$$

as \(n\rightarrow \infty \), there holds that

$$\begin{aligned}&\left( z_1^{\lambda ^n},z_2^{\lambda ^n},p_1^{\lambda ^n},p_2^{\lambda ^n},h^{\lambda ^n},\theta ^{\lambda ^n} \right) \rightarrow \left( z_1^{\lambda _*},z_2^{\lambda _*},p_1^{\lambda _*},p_2^{\lambda _*},h^{\lambda _*},\theta ^{\lambda _*} \right) \ \mathrm {in}\ X_{1{\mathbb {C}}}^2\times {\mathbb {R}}^4, \\&\quad \lambda ^n \rightarrow \lambda _*\ \mathrm {as}\ n\rightarrow \infty , \end{aligned}$$

and \(G(z_1^{\lambda _*},z_2^{\lambda _*},p_1^{\lambda _*},p_2^{\lambda _*},h^{\lambda _*},\theta ^{\lambda _*},\lambda _*)=0\). Now we can oatain from the unique solvability of \(G(\cdot ,\lambda _*)=0\) in Lemma 2 that

$$\begin{aligned} (z_1^{\lambda _*},z_2^{\lambda _*},p_1^{\lambda _*},p_2^{\lambda _*},h^{\lambda _*},\theta ^{\lambda _*})=(z_{1\lambda _*},z_{2\lambda _*},p_{1\lambda _*},p_{2\lambda _*},h_{\lambda _*},\theta _{\lambda _*}). \end{aligned}$$

This completes part (ii). \(\square \)

Now, Theorem 3 implies the following theorem immediately.

Theorem 4

Under assumptions \(\mathbf{(H_1)}-\mathbf{(H_3)}\), for \(\lambda \in (\lambda _*,{{\tilde{\lambda }}}^*]\), the eigenvalue problem

$$\begin{aligned} \varLambda (\lambda ,i\omega ,\tau )\psi =0,\ \omega >0,\ \tau \ge 0,\ \psi \in {\mathbb {X}}_{{\mathbb {C}}}^2\setminus \{(0,0)\} \end{aligned}$$

has a solution \((\omega ,\tau ,\psi )\), or equivalently, \(i\omega \in \sigma (A_{\tau ,\lambda })\) if and only if

$$\begin{aligned} \begin{aligned}&\omega =\omega _{\lambda }=(\lambda -\lambda _*)h_{\lambda },\ \ \tau =\tau _n=\frac{\theta _{\lambda }+2n\pi }{\omega _{\lambda }},\ n=0,1,2,\ldots , \\&\quad \psi =e \psi _{\lambda } =e \begin{pmatrix} \phi +(\lambda -\lambda _*)z_{1\lambda } \\ (p_{1\lambda }+ip_{2\lambda })\phi +(\lambda -\lambda _*)z_{2\lambda } \end{pmatrix}, \end{aligned} \end{aligned}$$
(3.21)

where e is a nonzero constant and \(\left( z_{1\lambda },z_{2\lambda },p_{1\lambda },p_{2\lambda },h_{\lambda },\theta _{\lambda }\right) \) is defined as in Theorem 3.

From Theorem 4, we know that \(i\omega _{\lambda } \in \sigma (A_{\tau _n,\lambda })\) with the associated eigenvector \(\psi _{\lambda }e^{i\omega _{\lambda }(\cdot )}\), which implies that \(-i\omega _{\lambda } \in \sigma (A_{\tau _n,\lambda }^*)\) and the corresponding adjoint equation

$$\begin{aligned} {{\tilde{\varLambda }}}(\lambda ,-i\omega _{\lambda },\tau _n){{\tilde{\psi }}}=0, \end{aligned}$$

or equivalently,

$$\begin{aligned} A_{\lambda }{{\tilde{\psi }}}+B_{\lambda }^*{{\tilde{\psi }}} e^{i\theta _{\lambda }}+i\omega _{\lambda }{{\tilde{\psi }}} =0 \end{aligned}$$
(3.22)

is solvable for \({{\tilde{\psi }}} \in {\mathbb {X}}_{{\mathbb {C}}}^2\setminus \{(0,0)\}\). Similarly, ignoring a scalar factor, \({{\tilde{\psi }}}=({{\tilde{\psi }}}_1,{{\tilde{\psi }}}_2)^T\) can also be taken as the form

$$\begin{aligned} {{\tilde{\psi }}}_1= & {} \phi +(\lambda -\lambda _*){\tilde{z}}_1,\ \ \langle \phi ,{\tilde{z}}_1 \rangle _w=0, \nonumber \\ {{\tilde{\psi }}}_2= & {} ({\tilde{p}}_1+i{\tilde{p}}_2)\phi +(\lambda -\lambda _*){\tilde{z}}_2,\ \ \langle \phi ,{\tilde{z}}_2 \rangle _w=0, \ {\tilde{p}}_1>0. \end{aligned}$$
(3.23)

Substituting (2.3), (3.23) and \(\omega _{\lambda }=(\lambda -\lambda _*)h_{\lambda }\) into Eq. (3.22), we have the following equivalent system to Eq. (3.22):

$$\begin{aligned} {\tilde{g}}_1({\tilde{z}}_1,{\tilde{z}}_2,{\tilde{p}}_1,{\tilde{p}}_2,\lambda )&:=L{\tilde{z}}_1+\Big \{ m(x)+i h_{\lambda } +\lambda h_1(\xi _{\lambda },\eta _{\lambda },\beta _{\lambda },c_{\lambda },\lambda ) \Big \}\cdot \big [ \phi +(\lambda -\lambda _*){\tilde{z}}_1 \big ] \nonumber \\&\quad \, \, +\lambda \beta _{\lambda }e^{i\theta _{\lambda }}\int _{\varOmega }r_1^{1\lambda }(x){\widetilde{K}}_{11}(x,\cdot )\big [ \phi (x)+(\lambda -\lambda _*){\tilde{z}}_1(x) \big ]\nonumber \\&\qquad \big [\phi (x)+(\lambda -\lambda _*)\xi _{\lambda }(x)\big ]\mathrm {d}x \nonumber \\&\qquad +\lambda c_{\lambda }e^{i\theta _{\lambda }}\int _{\varOmega }r_1^{2\lambda }(x){\widetilde{K}}_{21}(x,\cdot )\big [ ({\tilde{p}}_1+i{\tilde{p}}_2)\phi (x)+(\lambda -\lambda _*){\tilde{z}}_2(x) \big ] \nonumber \\&\qquad \times \big [\phi (x)+(\lambda -\lambda _*)\eta _{\lambda }(x)\big ]\mathrm {d}x=0, \nonumber \\ {\tilde{g}}_2({\tilde{z}}_1,{\tilde{z}}_2,{\tilde{p}}_1,{\tilde{p}}_2,\lambda )&:=L{\tilde{z}}_2+\Big \{ m(x)+i h_{\lambda } +\lambda h_2(\xi _{\lambda },\eta _{\lambda },\beta _{\lambda },c_{\lambda },\lambda ) \Big \}\nonumber \\&\quad \cdot \big [ ({\tilde{p}}_1+i{\tilde{p}}_2)\phi +(\lambda -\lambda _*){\tilde{z}}_2 \big ] \nonumber \\&\qquad +\lambda \beta _{\lambda }e^{i\theta _{\lambda }}\int _{\varOmega }r_2^{1\lambda }(x){\widetilde{K}}_{12}(x,\cdot )\big [ \phi (x)+(\lambda -\lambda _*){\tilde{z}}_1(x) \big ]\nonumber \\&\qquad \big [\phi (x)+(\lambda -\lambda _*)\xi _{\lambda }(x)\big ]\mathrm {d}x \nonumber \\&\qquad +\lambda c_{\lambda }e^{i\theta _{\lambda }}\int _{\varOmega }r_2^{2\lambda }(x){\widetilde{K}}_{22}(x,\cdot )\big [ ({\tilde{p}}_1+i{\tilde{p}}_2)\phi (x)+(\lambda -\lambda _*){\tilde{z}}_2(x) \big ] \nonumber \\&\qquad \times \big [\phi (x)+(\lambda -\lambda _*)\eta _{\lambda }(x)\big ]\mathrm {d}x=0. \end{aligned}$$
(3.24)

Define \({\tilde{G}}:{\mathbb {X}}_{{\mathbb {C}}}^2\times {\mathbb {R}}^2\times {\mathbb {R}}\rightarrow {\mathbb {Y}}_{{\mathbb {C}}}^2\) as

$$\begin{aligned} {\tilde{G}}({\tilde{z}}_1,{\tilde{z}}_2,{\tilde{p}}_1,{\tilde{p}}_2,\lambda ):=({\tilde{g}}_1,{\tilde{g}}_2,{\tilde{g}}_3,{\tilde{g}}_4,{\tilde{g}}_5,{\tilde{g}}_6). \end{aligned}$$

By a similar argument as in Theorem 3 and 4 , we can prove the following conclusions.

Theorem 5

Assume that \(\mathbf{(H_1)}-\mathbf{(H_3)}\) hold. Then the following statements are true:

  1. (i)

    there exists a continuously differentiable mapping \(\lambda \mapsto ({\tilde{z}}_{1\lambda },{\tilde{z}}_{2\lambda },{\tilde{p}}_{1\lambda },{\tilde{p}}_{2\lambda })\) from \([\lambda _*,{{\tilde{\lambda }}}^*]\) to \(X_{1{\mathbb {C}}}^2\times {\mathbb {R}}^2\) such that \({\tilde{G}}({\tilde{z}}_{1\lambda },{\tilde{z}}_{2\lambda },{\tilde{p}}_{1\lambda },{\tilde{p}}_{2\lambda },\lambda )=0\) with \({\tilde{p}}_{2\lambda _*}=0\), \({\tilde{p}}_{1\lambda _*}\) is the positive root of the following equation

    $$\begin{aligned} \kappa _{21}(\kappa _{21}-\kappa _{11})p^2+(\kappa _{11}\kappa _{12}-\kappa _{22}\kappa _{21})p-\kappa _{12}(\kappa _{12}-\kappa _{22})=0, \end{aligned}$$

    and \(({\tilde{z}}_{1\lambda _*},{\tilde{z}}_{2\lambda _*})^T\in X_{1{\mathbb {C}}}^2\) is the unique solution of

    $$\begin{aligned} {\mathcal {L}}\begin{pmatrix} {\tilde{z}}_{1\lambda _*} \\ {\tilde{z}}_{2\lambda _*} \end{pmatrix}&=-(m(x)+ih_{\lambda _*})\phi \begin{pmatrix} 1 \\ {\tilde{p}}_{1\lambda _*} \end{pmatrix} -\lambda _*\phi \begin{pmatrix} \beta _{\lambda _*}r^1_1(x){\widetilde{k}}_{11}(x)+r^1_2(x){\widetilde{k}}_{12}(x) \\ {\tilde{p}}_{1\lambda _*}\big ( \beta _{\lambda _*}r^2_1(x){\widetilde{k}}_{21}(x)+r^2_2(x){\widetilde{k}}_{22}(x) \big ) \end{pmatrix} \\&\quad \, \, -i\lambda _* \begin{pmatrix} \beta _{\lambda _*}\int _{\varOmega }r^1_1(x){\widetilde{K}}_{11}(x,\cdot )\phi ^2(x)\mathrm {d}x+{\tilde{p}}_{1\lambda _*}c_{\lambda _*}\int _{\varOmega }r^2_1(x){\widetilde{K}}_{21}(x,\cdot )\phi ^2(x)\mathrm {d}x \\ \beta _{\lambda _*}\int _{\varOmega }r^1_2(x){\widetilde{K}}_{12}(x,\cdot )\phi ^2(x)\mathrm {d}x+{\tilde{p}}_{1\lambda _*}c_{\lambda _*}\int _{\varOmega }r^2_2(x){\widetilde{K}}_{22}(x,\cdot )\phi ^2(x)\mathrm {d}x \end{pmatrix}, \end{aligned}$$

    where \({\mathcal {L}}\) is defined as (2.2). Moreover, if there is \(({\tilde{z}}_1^{\lambda },{\tilde{z}}_{2}^{\lambda },{\tilde{p}}_1^{\lambda },{\tilde{p}}_2^{\lambda })\) such that \({\tilde{G}}({\tilde{z}}_1^{\lambda },{\tilde{z}}_2^{\lambda },{\tilde{p}}_1^{\lambda },{\tilde{p}}_2^{\lambda },\lambda )=0\), then \(({\tilde{z}}_1^{\lambda },{\tilde{z}}_2^{\lambda },{\tilde{p}}_1^{\lambda },{\tilde{p}}_2^{\lambda })=({\tilde{z}}_{1\lambda },{\tilde{z}}_{2\lambda },{\tilde{p}}_{1\lambda },{\tilde{p}}_{2\lambda })\).

  2. (ii)

    for \(\lambda \in (\lambda _*,{{\tilde{\lambda }}}^*]\), the eigenvalue problem

    $$\begin{aligned} {{\tilde{\varLambda }}}(\lambda ,-i\omega ,\tau ){{\tilde{\psi }}}=0,\ \omega >0,\ \tau \ge 0,\ {{\tilde{\psi }}}\in {\mathbb {X}}_{{\mathbb {C}}}^2\setminus \{(0,0)\} \end{aligned}$$

    has a solution \((\omega ,\tau ,\psi )\), or equivalently, \(-i\omega \in \sigma (A_{\tau ,\lambda }^*)\) if and only if

    $$\begin{aligned} \begin{aligned}&\omega =\omega _{\lambda }=(\lambda -\lambda _*)h_{\lambda },\ \ \tau =\tau _n=\frac{\theta _{\lambda }+2n\pi }{\omega _{\lambda }},\ n=0,1,2,\ldots , \\&\quad {{\tilde{\psi }}}={\tilde{e}}{{\tilde{\psi }}}_{\lambda } ={\tilde{e}} \begin{pmatrix} \phi +(\lambda -\lambda _*){\tilde{z}}_{1\lambda } \\ ({\tilde{p}}_{1\lambda }+i{\tilde{p}}_{2\lambda })\phi +(\lambda -\lambda _*){\tilde{z}}_{2\lambda } \end{pmatrix}, \end{aligned} \end{aligned}$$
    (3.25)

    where \({\tilde{e}}\) is a nonzero constant and \(\left( {\tilde{z}}_{1\lambda },{\tilde{z}}_{2\lambda },{\tilde{p}}_{1\lambda },{\tilde{p}}_{2\lambda }\right) \) is defined in part (i), \((h_{\lambda },\theta _{\lambda })\) is defined in Theorem 3.

Remark 1

Theorem 5 shows that \(-i\omega _{\lambda } \in \sigma (A_{\tau _n,\lambda }^*)\) with the associated eigenvector \({{\tilde{\psi }}}_{\lambda }e^{-i\omega _{\lambda }(\cdot )}\).

4 Stability and Hopf Bifurcation

Notice that system (1.5) always has the steady state (0, 0). Then we first consider the stability of (0, 0). Linearizing system (1.5) at (0, 0), we obtain the linear eigenvalue problem

$$\begin{aligned} {\left\{ \begin{array}{ll} Lu+(\lambda -\lambda _*)m(x)u=\sigma u, &{} x\in \varOmega , \\ Lv+(\lambda -\lambda _*)m(x)v=\sigma v, &{} x\in \varOmega , \\ u(x)=v(x)=0, &{} x\in \partial \varOmega . \end{array}\right. } \end{aligned}$$
(4.1)

It follows from [5] that \(\sigma <0\) if and only if \(\lambda <\lambda _*\). Therefore, the stability result of the trivial steady state (0, 0) is as follows.

Lemma 3

Assume that \(\mathbf{(H_1)}\) holds. Then the trivial solution (0, 0) of system (1.2) is locally asymptotically stable when \(\lambda <\lambda _*\) and unstable when \(\lambda >\lambda _*\).

In the following, we pay attention to the stability and associated Hopf bifurcation of the positive steady state \(U_{\lambda }=(u_{\lambda },v_{\lambda })^T\) of Eq. (1.5) by regarding the parameter \(\tau \) as the bifurcation parameter. Firstly, we show the stability of \(U_{\lambda }\) for \(\tau =0\).

Theorem 6

Assume that assumptions \(\mathbf{(H_1)}-\mathbf{(H_3)}\) hold, then for each \(\lambda \in (\lambda _*,{{\tilde{\lambda }}}^*]\), all the eigenvalues of \(A_{\tau ,\lambda }\) have negative real parts when \(\tau =0\). That is, the positive steady state \((u_{\lambda },v_{\lambda })\) of Eq. (1.5) is locally asymptotically stable when \(\tau =0\).

Proof

Suppose to the contrary that there exists a sequence \(\{\lambda ^n\}_{n=1}^{\infty }\) such that \(\lambda ^n>\lambda _*\) for \(n\ge 1\), \(\lim \limits _{n\rightarrow \infty }\lambda ^n=\lambda _*\), and for every n, the eigenvalue equation

$$\begin{aligned} {\left\{ \begin{array}{ll} A_{\lambda ^n}\psi +B_{\lambda ^n}\psi =\mu \psi , &{} x\in \varOmega , \\ \psi (x)=0, &{} x\in \partial \varOmega \end{array}\right. } \end{aligned}$$
(4.2)

admits an eigenvalue \(\mu _{\lambda ^n}\) with nonnegative real part, whose corresponding eigenfunction \(\psi _{\lambda ^n}\) satisfies \(\Vert \psi _{\lambda ^n}\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}=1\). We can take \(\psi _{\lambda ^n}\) as \(\psi _{\lambda ^n}=s_{\lambda ^n}U_{\lambda ^n}+V_{\lambda ^n}\) for each \(n\ge 1\), where \(U_{\lambda }^{n}\) is the positive steady state of Eq. (1.5) with \(\lambda =\lambda ^n\), \(s_{\lambda ^n}\in {\mathbb {C}}\) and \(V_{\lambda ^n}=(V_{1\lambda ^n},V_{2\lambda ^n})^T\in {\mathbb {X}}_{{\mathbb {C}}}^2\) satisfy that

$$\begin{aligned} s_{\lambda ^n}=\frac{\langle U_{\lambda ^n},\psi _{\lambda ^n} \rangle _w}{\langle U_{\lambda ^n},U_{\lambda ^n} \rangle _w},\ \ \langle U_{\lambda ^n},V_{\lambda ^n} \rangle _w=0. \end{aligned}$$

Notice that

$$\begin{aligned} \big \langle U_{\lambda ^n},A_{\lambda ^n}V_{\lambda ^n} \big \rangle _w=\big \langle A_{\lambda ^n}U_{\lambda ^n},V_{\lambda ^n} \big \rangle _w \ \ \mathrm {and}\ \ A_{\lambda ^n}U_{\lambda ^n}=0, \end{aligned}$$

then by substituting \(\psi _{\lambda ^n}=s_{\lambda ^n}U_{\lambda ^n}+V_{\lambda ^n}\) and \(\mu =\mu _{\lambda ^n}\) into the first equation of Eq. (4.2), and computing the weighted inner product with \(\psi _{\lambda ^n}\), there holds that

$$\begin{aligned} \big \langle V_{\lambda ^n},A_{\lambda ^n}V_{\lambda ^n} \big \rangle _w+\big \langle \psi _{\lambda ^n},B_{\lambda ^n} \psi _{\lambda ^n} \big \rangle _w=\mu _{\lambda ^n}. \end{aligned}$$

Furthermore,

$$\begin{aligned} \begin{aligned} \left| \big \langle \psi _{\lambda ^n},B_{\lambda ^n} \psi _{\lambda ^n} \big \rangle _w \right|&\le \left| \lambda ^n \left\langle \psi _{\lambda ^n}, \begin{pmatrix} u_{\lambda }(r_1^{1\lambda ^n}(x)K_{11}*\psi _{1\lambda ^n}+r_2^{1\lambda ^n}(x)K_{12}*\psi _{2\lambda ^n}) \\ v_{\lambda }(r_1^{2\lambda ^n}(x)K_{21}*\psi _{1\lambda ^n}+r_2^{2\lambda ^n}(x)K_{22}*\psi _{2\lambda ^n}) \end{pmatrix} \right\rangle _w \right| \\&\le 2\lambda ^n(\lambda ^n-\lambda _*)M_1M_2e^{\max _{\varOmega }(\alpha m(x))} \max _{\varOmega \times \varOmega } \{ |K_{11}|, |K_{12}|, |K_{21}|, |K_{22}| \}|\varOmega | \\&\rightarrow 0,\ \mathrm {as}\ n\rightarrow \infty , \end{aligned}\nonumber \\ \end{aligned}$$
(4.3)

where

$$\begin{aligned} M_1= & {} \max \{ \beta _{\lambda ^n}\left[ \Vert \phi \Vert _{\infty }+(\lambda ^n-\lambda _*)\Vert \xi _{\lambda ^n}\Vert _{\infty } \right] ,c_{\lambda ^n}\left[ \Vert \phi \Vert _{\infty }+(\lambda ^n-\lambda _*)\Vert \eta _{\lambda ^n}\Vert _{\infty } \right] \}, \\ M_2= & {} \max \{\Vert r^{1\lambda ^n}_1(x)\Vert _{\infty },\Vert r^{1\lambda ^n}_2(x)\Vert _{\infty },\Vert r^{2\lambda ^n}_1(x)\Vert _{\infty },\Vert r^{2\lambda ^n}_2(x)\Vert _{\infty }\}. \end{aligned}$$

Define the operator \(A_{i\lambda }: {\mathbb {X}}_{{\mathbb {C}}}\rightarrow {\mathbb {Y}}_{{\mathbb {C}}}(i=1,2)\) by

$$\begin{aligned} A_{i\lambda }\varphi =e^{-\alpha m(x)}\nabla \cdot \left[ e^{\alpha m(x)}\nabla \varphi \right] +\lambda f_i\big (x,K_{i1}*u_{\lambda },K_{i2}*v_{\lambda } \big )\varphi \end{aligned}$$

for \(\varphi \in {\mathbb {X}}_{{\mathbb {C}}}\). Since 0 is the principal eigenvalue of \(A_{1\lambda ^n}\) (resp. \(A_{2\lambda ^n}\)) with the corresponding eigenfunction \(u_{\lambda ^n}\) (resp. \(v_{\lambda ^n}\)), we get that

$$\begin{aligned} \big \langle V_{1\lambda ^n},A_{1\lambda ^n}V_{1\lambda ^n} \big \rangle _w \le 0\ \mathrm {and}\ \big \langle V_{2\lambda ^n},A_{2\lambda ^n}V_{2\lambda ^n} \big \rangle _w \le 0. \end{aligned}$$

This result leads to

$$\begin{aligned} 0\le \mathrm {Re}(\mu _{\lambda ^n}),|\mathrm {Im}(\mu _{\lambda ^n})| \le \left| \big \langle \psi _{\lambda ^n},B_{\lambda ^n} \psi _{\lambda ^n} \big \rangle _w \right| , \end{aligned}$$

and hence

$$\begin{aligned} \lim _{n\rightarrow \infty }\mathrm {Re}(\mu _{\lambda ^n})=\lim _{j\rightarrow \infty }\mathrm {Im}(\mu _{\lambda ^n})=0. \end{aligned}$$

From the fact that \(\left| \big \langle V_{i\lambda ^n},A_{i\lambda ^n}V_{i\lambda ^n} \big \rangle _w\right| \ge |\lambda _2^{(i)}(\lambda ^n)|\cdot \Vert V_{i\lambda ^n}\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}^2\) (the proof of this inequality is similar to that of Lemma 1 (i)), where \(\lambda _2^{(i)}(\lambda ^n)\) is the second eigenvalue of \(A_{i\lambda ^n}\), we obtain

$$\begin{aligned} \left| -\big \langle \psi _{\lambda ^n},B_{\lambda ^n} \psi _{\lambda ^n} \big \rangle _w \right| +|\mu _{\lambda ^n}|\ge |\gamma ^n|\cdot \Vert V_{\lambda ^n}\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}^2, \end{aligned}$$
(4.4)

in which \(\gamma ^n=\min \left\{ \lambda _2^{(1)}(\lambda ^n),\lambda _2^{(2)}(\lambda ^n) \right\} \). In view of (4.4), using the limit \(\lim \limits _{n\rightarrow \infty }\left| \big \langle \psi _{\lambda ^n},B_{\lambda ^n} \psi _{\lambda ^n} \big \rangle _w \right| =\lim \limits _{n\rightarrow \infty }|\mu _{\lambda ^n}|=0\), we see that \(\lim \limits _{n\rightarrow \infty }\Vert V_{\lambda ^n}\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}=0\).

Since \(\psi _{\lambda ^n}=s_{\lambda ^n}U_{\lambda ^n}+V_{\lambda ^n}\) and \(\Vert \psi _{\lambda ^n}\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}=1\), we see that

$$\begin{aligned} \lim _{n\rightarrow \infty }|s_{\lambda ^n}|^2(\lambda ^n-\lambda _*)^2\lim _{n\rightarrow \infty }\left( \left\| \dfrac{u_{\lambda ^n}}{\lambda ^n-\lambda _*} \right\| _{{\mathbb {Y}}_{{\mathbb {C}}}}^2+\left\| \dfrac{v_{\lambda ^n}}{\lambda ^n-\lambda _*} \right\| _{{\mathbb {Y}}_{{\mathbb {C}}}}^2 \right) =1, \end{aligned}$$

which means \(\lim _{n\rightarrow \infty }|s_{\lambda ^n}|(\lambda ^n-\lambda _*)=\dfrac{1}{\sqrt{\beta _{\lambda _*}^2+c_{\lambda _*}^2}\Vert \phi \Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}}>0\), where \(\phi \) is the eigenfunction of eigenvalue problem (1.6) with the principal eigenvalue \(\lambda _*\). We now calculate that

$$\begin{aligned}&\frac{\big \langle \psi _{\lambda ^n},B_{\lambda ^n} \psi _{\lambda ^n} \big \rangle _w}{\lambda ^n-\lambda _*} \\&\quad =\frac{\lambda ^n\left\langle (s_{\lambda ^n}U_{\lambda ^n}+V_{\lambda ^n}),\begin{pmatrix} u_{\lambda }[r_1^{1\lambda ^n}(x)K_{11}*(s_{\lambda ^n}u_{\lambda ^n}+V_{1\lambda ^n})+r_2^{1\lambda ^n}(x)K_{12}*(s_{\lambda ^n}v_{\lambda ^n}+V_{2\lambda ^n})] \\ v_{\lambda }[r_1^{2\lambda ^n}(x)K_{21}*(s_{\lambda ^n}u_{\lambda ^n}+V_{1\lambda ^n})+r_2^{2\lambda ^n}(x)K_{22}*(s_{\lambda ^n}v_{\lambda ^n}+V_{2\lambda ^n})] \end{pmatrix} \right\rangle _w }{\lambda ^n-\lambda _*} \\&\quad =\lambda ^n\left( |s_{\lambda ^n}|^2(\lambda ^n-\lambda _*)^2J_1+s_{\lambda ^n}(\lambda ^n-\lambda _*)J_2+\overline{s_{\lambda ^n}}(\lambda ^n-\lambda _*)J_3+J_4 \right) , \end{aligned}$$

where

$$\begin{aligned} J_1&=\int _{\varOmega }\int _{\varOmega }\frac{e^{\alpha m(x)}\left[ r_1^{1\lambda ^n}(x)K_{11}(x,y)u_{\lambda ^n}^2(x)u_{\lambda ^n}(y)+r_2^{1\lambda ^n}(x)K_{12}(x,y)u_{\lambda ^n}^2(x)v_{\lambda ^n}(y) \right] }{(\lambda ^n-\lambda _*)^3} \mathrm {d}y\mathrm {d}x \\&\quad +\int _{\varOmega }\int _{\varOmega }\frac{e^{\alpha m(x)}\left[ r_1^{2\lambda ^n}(x)K_{21}(x,y)v_{\lambda ^n}^2(x)u_{\lambda ^n}(y)+r_2^{2\lambda ^n}(x)K_{22}(x,y)v_{\lambda ^n}^2(x)v_{\lambda ^n}(y) \right] }{(\lambda ^n-\lambda _*)^3} \mathrm {d}y\mathrm {d}x, \\ J_2&=\int _{\varOmega }\int _{\varOmega }\frac{e^{\alpha m(x)}\left[ r_1^{1\lambda ^n}(x)K_{11}(x,y)u_{\lambda ^n}(x)\overline{V_{1\lambda ^n}}(x)u_{\lambda ^n}(y)+r_2^{1\lambda ^n}(x)K_{12}(x,y)u_{\lambda ^n}(x)\overline{V_{1\lambda ^n}}(x)v_{\lambda ^n}(y) \right] }{(\lambda ^j-\lambda _*)^2} \mathrm {d}y\mathrm {d}x \\&\quad +\int _{\varOmega }\int _{\varOmega }\frac{e^{\alpha m(x)}\left[ r_1^{2\lambda ^n}(x)K_{21}(x,y)v_{\lambda ^n}(x)\overline{V_{2\lambda ^n}}(x)u_{\lambda ^n}(y)+r_2^{2\lambda ^n}(x)K_{22}(x,y)v_{\lambda ^n}(x)\overline{V_{2\lambda ^n}}(x)v_{\lambda ^n}(y) \right] }{(\lambda ^j-\lambda _*)^2} \mathrm {d}y\mathrm {d}x, \\ J_3&=\int _{\varOmega }\int _{\varOmega }\frac{e^{\alpha m(x)}\left[ r_1^{1\lambda ^n}(x)K_{11}(x,y)u_{\lambda ^n}^2(x)V_{1\lambda ^n}(y)+r_2^{1\lambda ^n}(x)K_{12}(x,y)u_{\lambda ^n}^2(x)V_{2\lambda ^n}(y) \right] }{(\lambda ^n-\lambda _*)^2} \mathrm {d}y\mathrm {d}x \\&\quad +\int _{\varOmega }\int _{\varOmega }\frac{e^{\alpha m(x)}\left[ r_1^{2\lambda ^n}(x)K_{21}(x,y)v_{\lambda ^n}^2(x)V_{1\lambda ^n}(y)+r_2^{2\lambda ^n}(x)K_{22}(x,y)v_{\lambda ^n}^2(x)V_{2\lambda ^n}(y) \right] }{(\lambda ^n-\lambda _*)^2} \mathrm {d}y\mathrm {d}x, \\ J_4&=\int _{\varOmega }\int _{\varOmega }\frac{e^{\alpha m(x)}\left[ r_1^{1\lambda ^n}(x)K_{11}(x,y)u_{\lambda ^n}(x)\overline{V_{1\lambda ^n}}(x)V_{1\lambda ^n}(y)+r_2^{1\lambda ^n}(x)K_{12}(x,y)u_{\lambda ^n}(x)\overline{V_{1\lambda ^n}}(x)V_{2\lambda ^n}(y) \right] }{\lambda ^n-\lambda _*} \mathrm {d}y\mathrm {d}x \\&\quad +\int _{\varOmega }\int _{\varOmega }\frac{e^{\alpha m(x)}\left[ r_1^{2\lambda ^n}(x)K_{21}(x,y)v_{\lambda ^n}(x)\overline{V_{2\lambda ^n}}(x)V_{1\lambda ^n}(y)+r_2^{2\lambda ^n}(x)K_{22}(x,y)v_{\lambda ^n}(x)\overline{V_{2\lambda ^n}}(x)V_{2\lambda ^n}(y) \right] }{\lambda ^n-\lambda _*} \mathrm {d}y\mathrm {d}x. \end{aligned}$$

It follows from Hölder inequality that

$$\begin{aligned} \langle u,v \rangle _w&=\int _{\varOmega }e^{\frac{\alpha m(x)}{2}}u(x)e^{\frac{\alpha m(x)}{2}}v(x) \mathrm {d}x \\&\le \left( \int _{\varOmega }e^{\alpha m(x)}u^2(x) \mathrm {d}x \right) ^{1/2}\left( \int _{\varOmega }e^{\alpha m(x)}v^2(x) \mathrm {d}x \right) ^{1/2}=\Vert u\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}\Vert v\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}. \end{aligned}$$

Then by using the limit \(\lim \limits _{n\rightarrow \infty }\Vert V_{\lambda ^n}\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}=0\), there holds that

$$\begin{aligned} \lim \limits _{n\rightarrow \infty }J_2=\lim \limits _{n\rightarrow \infty }J_3=\lim \limits _{n\rightarrow \infty }J_4=0. \end{aligned}$$

We also have that

$$\begin{aligned} \lim _{n\rightarrow \infty }J_1= & {} \kappa _{11}\beta _{\lambda _*}^3+\kappa _{12}\beta _{\lambda _*}^2c_{\lambda _*}+\kappa _{21}\beta _{\lambda _*}c_{\lambda _*}^2+\kappa _{22}c_{\lambda _*}^3\\= & {} -\frac{\beta _{\lambda _*}^2+c_{\lambda _*}^2}{\lambda _*} \int _{\varOmega }m(x)e^{\alpha m(x)}\phi ^2(x)\mathrm {d}x <0. \end{aligned}$$

The above argument implies that there exists \(N_*\in {\mathbb {N}}\) such that

$$\begin{aligned} \mathrm {Re}\left( \big \langle \psi _{\lambda ^n},B_{\lambda ^n} \psi _{\lambda ^n} \big \rangle _w \right) <0\ \ \mathrm {for\ each}\ n\ge N_*. \end{aligned}$$

Consequently,

$$\begin{aligned} \mathrm {Re}(\mu _{\lambda ^n})= \big \langle V_{\lambda ^n},A_{\lambda ^n}V_{\lambda ^n} \big \rangle _w+\mathrm {Re}\left( \big \langle \psi _{\lambda ^n},B_{\lambda ^n} \psi _{\lambda ^n} \big \rangle _w \right) <0, \end{aligned}$$

which is a contradiction with that \(\mathrm {Re}(\mu _{\lambda ^n})\ge 0\) for \(n\ge 1\). That is to say, \(A_{\tau ,\lambda }\) has no eigenvalue with nonnegative real parts when \(\tau =0\). The proof is finished. \(\square \)

Lemma 4

Assume that \(\mathbf{(H_1)}-\mathbf{(H_3)}\) hold. Then for each \(\lambda \in (\lambda _*,{{\tilde{\lambda }}}^*]\), \(\mu =i\omega _{\lambda }\) is a simple eigenvalue of \(A_{\tau _n,\lambda }\) for \(n=0,1,2,\ldots \).

Proof

From Theorem 4, we see that \({\mathscr {N}}[A_{\tau _n,\lambda }-i\omega _{\lambda }]=\mathrm {Span}[e^{i\omega _{\lambda }s}\psi _{\lambda }]\), where \(s\in [-\tau _n,0]\) and \(\psi _{\lambda }\) is defined as in Theorem 4. Suppose that \(\mu =i\omega _{\lambda }\) is not a simple eigenvalue, then there exists \({{\hat{\phi }}} \in {\mathscr {N}}[A_{\tau _n,\lambda }-i\omega _{\lambda }]^2\), i.e.,

$$\begin{aligned} {[}A_{\tau _n,\lambda }-i\omega _{\lambda }]{\hat{\phi \in }} {\mathscr {N}}[A_{\tau _n,\lambda }-i\omega _{\lambda }] =\mathrm {Span}[e^{i\omega _{\lambda }s}\psi _{\lambda }]. \end{aligned}$$

Hence, we can pick a constant a such that \([A_{\tau _n,\lambda }-i\omega _{\lambda }]{{\hat{\phi }}}=ae^{i\omega _{\lambda }s}\psi _{\lambda }\). Then there holds that

$$\begin{aligned} \begin{aligned} {{\hat{\phi }}}'(s)&=i\omega _{\lambda } {{\hat{\phi }}}+ae^{i\omega _{\lambda }s}\psi _{\lambda },\ \ s\in [-\tau _n,0], \\ {{\hat{\phi }}}'(0)&=A_{\lambda }{{\hat{\phi }}}(0)+B_{\lambda }{{\hat{\phi }}}(-\tau _n). \end{aligned} \end{aligned}$$
(4.5)

In view of the first equation of Eq. (4.5), we obtain that

$$\begin{aligned} \begin{aligned}&{{\hat{\phi }}}(s) ={{\hat{\phi }}}(0)e^{i\omega _{\lambda }s}+as e^{i\omega _{\lambda }s}\psi _{\lambda }, \\&\quad \quad {{\hat{\phi }}}'(0) =i\omega _{\lambda } {{\hat{\phi }}}(0)+a\psi _{\lambda }. \end{aligned} \end{aligned}$$
(4.6)

It follows from the second equation of (4.5) and (4.6) that

$$\begin{aligned} \varLambda (\lambda ,i\omega _{\lambda },\tau _n){{\hat{\phi }}}(0)&=\left[ A_{\lambda }+B_{\lambda }e^{-i\theta _{\lambda }}-i\omega _{\lambda } \right] {{\hat{\phi }}}(0) \\&=a \left( \psi _{\lambda }+\tau _n e^{-i\theta _{\lambda }}B_{\lambda }\psi _{\lambda } \right) , \end{aligned}$$

where we have used the identity \({{\hat{\phi }}}(-\tau _n)={{\hat{\phi }}}(0)e^{-i\theta _{\lambda }}-a\tau _n e^{-i\theta _{\lambda }}\psi _{\lambda }\). From (3.6), we have

$$\begin{aligned} 0&=\left\langle {{\tilde{\varLambda }}}(\lambda ,-i\omega _{\lambda },\tau _n){{\tilde{\psi }}}_{\lambda },{{\hat{\phi }}}(0) \right\rangle _w =\left\langle {{\tilde{\psi }}}_{\lambda },\varLambda (\lambda ,i\omega _{\lambda },\tau _n){{\hat{\phi }}}(0) \right\rangle _w \nonumber \\&=a\left\langle {{\tilde{\psi }}}_{\lambda },\psi _{\lambda }+\tau _n e^{-i\theta _{\lambda }}B_{\lambda }\psi _{\lambda } \right\rangle _w :=a S_{n\lambda }. \end{aligned}$$
(4.7)

Let \(\lambda \rightarrow \lambda _*\), then it follows that

$$\begin{aligned} \lim _{\lambda \rightarrow \lambda _*}S_{n\lambda }&=(1+{\tilde{p}}_{1\lambda _*}p_{1\lambda _*})\int _{\varOmega }e^{\alpha m(x)}\phi ^2 dx \\&\quad \, \, -i\lambda _*\frac{\beta _{\lambda _*}(\kappa _{11}+\kappa _{12}p_{1\lambda _*})+c_{\lambda _*}{\tilde{p}}_{1\lambda _*}(\kappa _{21}+\kappa _{22}p_{1\lambda _*})}{h_{\lambda _*}}(\frac{\pi }{2}+2n\pi ) \\&=\left[ 1+i(\frac{\pi }{2}+2n\pi ) \right] (1+{\tilde{p}}_{1\lambda _*}p_{1\lambda _*})\int _{\varOmega }e^{\alpha m(x)}\phi ^2 dx, \end{aligned}$$

which implies that \(S_{n\lambda }\ne 0\) and hence \(a=0\). Therefore, \({\hat{\phi \in }} {\mathscr {N}}[A_{\tau _n,\lambda }-i\omega _{\lambda }]\). By induction it can be derived that

$$\begin{aligned} {\mathscr {N}}[A_{\tau _n,\lambda }-i\omega _{\lambda }]^j={\mathscr {N}}[A_{\tau _n,\lambda }-i\omega _{\lambda }],\ \ j=2,3,\ldots , \end{aligned}$$

and \(\mu =i\omega _{\lambda }\) is a simple eigenvalue of \(A_{\tau _n,\lambda }\) for \(n=0,1,2,\ldots \). \(\square \)

Now it can be inferred from the implicit function theorem that there is a neighborhood \(O_n\times D_n\times H_n^2\subset {\mathbb {R}}\times {\mathbb {C}}\times {\mathbb {X}}_{{\mathbb {C}}}^2\) of \(\left( \tau _n,i\omega _{\lambda },\psi _{\lambda } \right) \) and a continuous differential function \((\mu ,\psi ):O_n\rightarrow D_n\times H_n^2\) satisfying \(\mu (\tau _n)=i\omega _{\lambda }\) and \(\psi (\tau _n)=\psi _{\lambda }\) such that, for each \(\tau \in O_n\), \(A_{\tau _n,\lambda }\) in \(D_n\) has the unique eigenvalue \(\mu (\tau )\) with its associated eigenvector \(\psi (\tau )e^{\mu (\tau )(\cdot )}\) and there holds that

$$\begin{aligned} \varLambda (\lambda ,\mu (\tau ),\tau )\psi (\tau ) =\left[ A_{\lambda }+B_{\lambda }e^{-\mu (\tau )\tau }-\mu (\tau ) \right] \psi (\tau )=0. \end{aligned}$$
(4.8)

In the following, we verify the transversality condition for Hopf bifurcation.

Lemma 5

Under the assumptions \(\mathbf{(H_1)}-\mathbf{(H_3)}\), for \(\lambda \in (\lambda _*,{{\tilde{\lambda }}}^*]\),

$$\begin{aligned} \mathrm {Re}\left( \dfrac{d\mu }{d\tau }\left( \tau _n \right) \right) >0. \end{aligned}$$

Proof

Firstly, by differentiating Eq. (4.8) with respect to \(\tau \) at \(\tau =\tau _n\), we have

$$\begin{aligned} \frac{d\mu }{d\tau }(\tau _n)\left( \psi _{\lambda }+\tau _n e^{-i\theta _{\lambda }}B_{\lambda }\psi _{\lambda } \right) =\varLambda (\lambda ,i\omega _{\lambda },\tau _n) \frac{d\psi }{d\tau }(\tau _n)-i\omega _{\lambda }e^{-i\theta _{\lambda }}B_{\lambda }\psi _{\lambda }. \end{aligned}$$

Then calculating the weighted inner product of above equation with \({{\tilde{\psi }}}_{\lambda }\) gives that

$$\begin{aligned} \frac{d\mu }{d\tau }(\tau _n)=-\frac{\left\langle {{\tilde{\psi }}}_{\lambda },i\omega _{\lambda }e^{-i\theta _{\lambda }}B_{\lambda }\psi _{\lambda } \right\rangle _w}{S_{n\lambda }}=-\frac{I_1+I_2}{|S_{n\lambda }|^2}. \end{aligned}$$

where \(S_{n\lambda }\) is defined as in (4.7),

$$\begin{aligned} I_1&=\left\langle \psi _{\lambda },{{\tilde{\psi }}}_{\lambda } \right\rangle _w \left\langle {{\tilde{\psi }}}_{\lambda },i\omega _{\lambda }e^{-i\theta _{\lambda }}B_{\lambda }\psi _{\lambda } \right\rangle _w, \\ I_2&=i\omega _{\lambda }\tau _n \left| \left\langle {{\tilde{\psi }}}_{\lambda },B_{\lambda }\psi _{\lambda } \right\rangle _w \right| ^2. \end{aligned}$$

Direct computation yields

$$\begin{aligned}&\lim _{\lambda \rightarrow \lambda _*}\left\langle \psi _{\lambda },{{\tilde{\psi }}}_{\lambda } \right\rangle _w =(1+{\tilde{p}}_{1\lambda _*}p_{1\lambda _*})\int _{\varOmega }e^{\alpha m(x)}\phi ^2 dx, \\&\quad \lim _{\lambda \rightarrow \lambda _*}\frac{1}{(\lambda -\lambda _*)^2}\left\langle {{\tilde{\psi }}}_{\lambda },i\omega _{\lambda }e^{-i\theta _{\lambda }}B_{\lambda }\psi _{\lambda } \right\rangle _w =-(1+{\tilde{p}}_{1\lambda _*}p_{1\lambda _*})h_{\lambda _*}^2\int _{\varOmega }e^{\alpha m(x)}\phi ^2 dx, \end{aligned}$$

which implies that

$$\begin{aligned} \lim _{\lambda \rightarrow \lambda _*}\frac{1}{(\lambda -\lambda _*)^2}\mathrm {Re}\left( \dfrac{d\mu }{d\tau }\left( \tau _n \right) \right) =\frac{(1+{\tilde{p}}_{1\lambda _*}p_{1\lambda _*})^2h_{\lambda _*}^2}{\lim _{\lambda \rightarrow \lambda _*}\left| S_{n\lambda } \right| ^2 }\left( \int _{\varOmega }e^{\alpha m(x)}\phi ^2 dx \right) ^2>0. \end{aligned}$$

The proof is finished. \(\square \)

From Theorems 46 and Lemmas 45, we can now conclude the stability result of the positive steady state \(U_{\lambda }\) and the associated Hopf bifurcation of Eq. (1.5) as follows.

Theorem 7

Assume that \(\mathbf{(H_1)}-\mathbf{(H_3)}\) hold. For \(\lambda \in (\lambda _*,{{\tilde{\lambda }}}^*]\), the following statements are ture:

  1. (i)

    there exists an increasing sequence \(\{\tau _n\}_{n=0}^{\infty }\) such that all the eigenvalues of \(A_{\tau ,\lambda }\) have negative real parts when \(\tau \in ( 0,\tau _0 )\), \(A_{\tau ,\lambda }\) has a pair of purely imaginary eigenvalues \(\pm i\omega _{\lambda }\left( \omega _{\lambda }>0 \right) \) when \(\tau =\tau _n\), and \(A_{\tau ,\lambda }\) has exactly \(2(n+1)\) eigenvalues with positive real parts when \(\tau \in ( \tau _n,\tau _{n+1} ], n=0,1,2,\ldots \);

  2. (ii)

    the positive steady state \(U_{\lambda }\) of Eq. (1.5) is locally asymptotically stable when \(\tau \in [0,\tau _0)\), and unstable when \(\tau \in (\tau _0,\infty )\);

  3. (iii)

    Hopf bifurcation occurs as the delay \(\tau \) increasingly crosses through each \(\tau _n(n=0,1,2,\ldots )\), and there exist \(\varepsilon _0>0\) and a continuous family of periodic orbits of (1.5) in form of

    $$\begin{aligned} \left\{ \left( \tau _n(\varepsilon ),u_n(x,t,\varepsilon ),v_n(x,t,\varepsilon ),T_n(\varepsilon ) \right) :\varepsilon \in (0,\varepsilon _0) \right\} , \end{aligned}$$

    where \((u_n(x,t,\varepsilon ),v_n(x,t,\varepsilon ))^T\) is a \(T_n(\varepsilon )\)-periodic solution of (1.5) with \(\tau =\tau _n(\varepsilon )\), and \(\tau _n(0)=\tau _n,\) \(\lim \limits _{\varepsilon \rightarrow 0^+}(u_n(x,t,\varepsilon ),v_n(x,t,\varepsilon ))^T=(u_\lambda ,v_\lambda )^T\) and \(\lim \limits _{\varepsilon \rightarrow 0^+}T_n(\varepsilon )=2\pi /\omega _{\lambda }\).

5 The Properties of Hopf Bifurcation

In this section, we will compute the normal form of the Hopf bifurcation to determine the direction and stability of bifurcating periodic solutions emerging from \((U_{\lambda },\tau _n)\) by applying the methods in Faria [11] and Hassard et al. [16]. At first, we let \({\tilde{U}}(t)=(U_1(t),U_2(t))^T=(u(\cdot ,t)-u_{\lambda },v(\cdot ,t)-v_{\lambda })^T,\gamma =\tau -\tau _n\) such that the steady state \(U_{\lambda }=(u_{\lambda },v_{\lambda })^T\) and parameter \(\tau \) is translated to the origin. Re-scale the time \({\tilde{t}}= t/\tau \) and drop the tilde signs for simplification of notations. Thus, \(\gamma =0\) is the Hopf bifurcation value. For the simplicity of writing, we define

$$\begin{aligned} r^{i\lambda }_{kl}(x)=\frac{\partial ^{k+l}f_i}{\partial s_1^ks_2^l}(x,K_{i1}*u_{\lambda },K_{i2}*v_{\lambda }), \ \ i=1,2,\ k,l=1,2,\ldots . \end{aligned}$$

Then we can rewrite Eq. (1.5) as the following abstract functional differential equation

$$\begin{aligned} \frac{d U(t)}{dt}=\tau _nL_0(U_t)+J(U_t,\gamma ), \end{aligned}$$
(5.1)

where \(U_t=U(t+s)\in {\mathcal {C}}=C([-\tau ,0],{\mathbb {Y}}^2)\), and

$$\begin{aligned}&L_0(U_t)=A_{\lambda }U(t)+B_{\lambda }U(t-1), \\&\quad J(U_t,\gamma )=\gamma L_0(U_t)+(\gamma +\tau _n)\lambda F(U_t), \end{aligned}$$

and \(F(U_t)=(F_1(U_t),F_2(U_t))^T\) is defined by

$$\begin{aligned} F_1(U_t)&=U_1(t)\Big [r^{1\lambda }_1(x)K_{11}*U_1(t-1)+r^{1\lambda }_2(x)K_{12}*U_2(t-1) \Big ] \\&\quad +\frac{1}{2!}(U_1(t)+u_{\lambda })\Big [ r^{1\lambda }_{20}(x)(K_{11}*U_1(t-1))^2\\&\quad +2r^{1\lambda }_{11}(x)(K_{11}*U_1(t-1))(K_{12}*U_2(t-1)) \\&\quad +r^{1\lambda }_{02}(x)(K_{12}*U_2(t-1))^2 \Big ]+\frac{1}{3!}u_{\lambda }\Big [ r^{1\lambda }_{30}(x)(K_{11}*U_1(t-1))^3 \\&\quad +3r^{1\lambda }_{21}(x)(K_{11}*U_1(t-1))^2(K_{12}*U_2(t-1))\\&\quad +3r^{1\lambda }_{12}(x)(K_{11}*U_1(t-1))(K_{12}*U_2(t-1))^2 \\&\quad +r^{1\lambda }_{03}(x)(K_{12}*U_2(t-1))^3 \Big ]+h.o.t, \\ F_2(U_t)&=U_2(t)\Big [r^{2\lambda }_1(x)K_{21}*U_1(t-1)+r^{2\lambda }_2(x)K_{22}*U_2(t-1) \Big ] \\&\quad \, +\frac{1}{2!}(U_2(t)+v_{\lambda })\Big [ r^{2\lambda }_{20}(x)(K_{21}*U_1(t-1))^2\\&\quad +2r^{2\lambda }_{11}(x)(K_{21}*U_1(t-1))(K_{22}*U_2(t-1)) \\&\quad +r^{2\lambda }_{02}(x)(K_{22}*U_2(t-1))^2 \Big ]+\frac{1}{3!}v_{\lambda }\Big [ r^{2\lambda }_{30}(x)(K_{21}*U_1(t-1))^3 \\&\quad +3r^{2\lambda }_{21}(x)(K_{21}*U_1(t-1))^2(K_{22}*U_2(t-1))\\&\quad +3r^{2\lambda }_{12}(x)(K_{21}*U_1(t-1))(K_{22}*U_2(t-1))^2 \\&\quad +r^{2\lambda }_{03}(x)(K_{22}*U_2(t-1))^3 \Big ]+h.o.t, \end{aligned}$$

in which h.o.t stands for “high order terms”. Denote by \({\mathcal {A}}_{\tau _n}\) the infinitesimal generator of the linearized equation

$$\begin{aligned} \frac{d U(t)}{dt}=\tau _nL_0(U_t). \end{aligned}$$
(5.2)

Then from [32], we have

$$\begin{aligned} {\mathcal {A}}_{\tau _n}\varPsi ={\dot{\varPsi }}, \end{aligned}$$

with the domain

$$\begin{aligned} {\mathscr {D}}(A_{\tau })=\left\{ \varPsi \in {\mathcal {C}}_{{\mathbb {C}}}\cap {\mathcal {C}}_{{\mathbb {C}}}^1:\varPsi (0)\in {\mathbb {X}}_{{\mathbb {C}}}^2,{\dot{\varPsi }}(0)=\tau _n A_{\lambda }\varPsi (0)+\tau _n B_{\lambda }\varPsi (-1) \right\} , \end{aligned}$$

where \({\mathcal {C}}_{{\mathbb {C}}}^1=C^1([-1,0],{\mathbb {Y}}_{{\mathbb {C}}}^2)\). So, we can rewrite Eq. (5.1) in the abstract form:

$$\begin{aligned} \dfrac{d U_t}{dt}={\mathcal {A}}_{\tau _n}U_t+X_0 J(U_t,\gamma ), \end{aligned}$$
(5.3)

where

$$\begin{aligned} X_0(s)= {\left\{ \begin{array}{ll} 0, &{} s\in [-1,0), \\ I, &{} s=0. \end{array}\right. } \end{aligned}$$

On the other hand, let

$$\begin{aligned} {\mathcal {A}}_{\tau _n}^*{{\tilde{\varPhi }}}&=-\dot{{{\tilde{\varPsi }}}}, \\ {\mathscr {D}}({\mathcal {A}}_{\tau _n}^*)&=\left\{ {{\tilde{\varPsi }}}\in {\mathcal {C}}_{{\mathbb {C}}}^*\cap ({\mathcal {C}}_{{\mathbb {C}}}^*)^1: {{\tilde{\varPsi }}}(0)\in {\mathbb {X}}_{{\mathbb {C}}}^2,-\dot{{{\tilde{\varPsi }}}}(0)=\tau _n A_{\lambda } {{\tilde{\varPsi }}}(0)+\tau _n B_{\lambda }^* {{\tilde{\varPsi }}}(1) \right\} , \end{aligned}$$

where \({\mathcal {C}}_{{\mathbb {C}}}^*=C([0,1],{\mathbb {Y}}_{{\mathbb {C}}}^2),({\mathcal {C}}_{{\mathbb {C}}}^*)^1=C^1([0,1],{\mathbb {Y}}_{{\mathbb {C}}}^2)\). Define a formal duality \(\langle \langle \cdot ,\cdot \rangle \rangle \) by

$$\begin{aligned} \langle \langle {{\tilde{\varPsi }}},\varPsi \rangle \rangle =\left\langle {{\tilde{\varPsi }}}(0),\varPsi (0) \right\rangle _w-\tau _n \int _{-1}^0\left\langle {{\tilde{\varPsi }}}(s+1),B_{\lambda }\varPsi (s) \right\rangle _w \mathrm {d}s. \end{aligned}$$
(5.4)

for \({{\tilde{\varPsi }}}\in {\mathscr {D}}({\mathcal {A}}_{\tau _n}^*)\) and \(\varPsi \in {\mathscr {D}}({\mathcal {A}}_{\tau _n})\). Then \({\mathcal {A}}_{\tau _n}^*\) and \({\mathcal {A}}_{\tau _n}\) satisfy

$$\begin{aligned} \langle \langle {\mathcal {A}}_{\tau _n}^* {{\tilde{\varPsi }}},\varPsi \rangle \rangle = \langle \langle {{\tilde{\varPsi }}},{\mathcal {A}}_{\tau _n}\varPsi \rangle \rangle \end{aligned}$$

for \({{\tilde{\varPsi }}}\in {\mathscr {D}}({\mathcal {A}}_{\tau _n}^*)\) and \(\varPsi \in {\mathscr {D}}({\mathcal {A}}_{\tau _n})\). The above equality means that \({\mathcal {A}}_{\tau _n}^*\) and \({\mathcal {A}}_{\tau _n}\) are adjoint operators under the bilinear form (5.4).

It can be seen from Theorem 4 that \({\mathcal {A}}_{\tau _n}\) has a pair of simple purely imaginary eigenvalues \(\pm i\omega _{\lambda }\tau _n\). Then the eigenfunction corresponding to \(i\omega _{\lambda }\tau _n\) (resp. \(-i\omega _{\lambda }\tau _n\)) is \(p(s)=\psi _{\lambda }e^{i\omega _{\lambda }\tau _ns}\) (resp. \({\overline{p}}(s)={\overline{\psi }}_{\lambda }e^{-i\omega _{\lambda }\tau _ns}\)) for \(s\in [-1,0]\), where \(\psi _{\lambda }\) is defined as in (3.21). At the same time, it follows from Theorem 5 and Remark 1 that \(\pm i\omega _{\lambda }\tau _n\) are also a pair of simple purely imaginary eigenvalues of the operator \({\mathcal {A}}_{\tau _n}^*\) and the eigenfunction associated with \(-i\omega _{\lambda }\tau _n\) (resp. \(i\omega _{\lambda }\tau _n\)) is \(q({\tilde{s}})={{\tilde{\psi }}}_{\lambda }e^{i\omega _{\lambda }\tau _n{\tilde{s}}}\) (resp. \({\overline{q}}({\tilde{s}})=\overline{{{\tilde{\psi }}}}_{\lambda }e^{-i\omega _{\lambda }\tau _n{\tilde{s}}}\)) for \({\tilde{s}}\in [0,1]\), where \({{\tilde{\psi }}}_{\lambda }\) is defined in Theorem 5. Following from [32], the center subspace of Eq. (5.1) is \(P=\mathrm {span}\{ p(s),{\overline{p}}(s) \}\), and the formal adjoint subspace of P with respect to the bilinear form (5.4) is \(P^*=\mathrm {span}\{ q({\tilde{s}}),{\overline{q}}({\tilde{s}}) \}\). Let \(\varPhi (s)=(p(s),{\overline{p}}(s)),\varPsi ({\tilde{s}})=\left( \dfrac{q({\tilde{s}})}{\overline{S_{n\lambda }}},\dfrac{{\overline{q}}({\tilde{s}})}{S_{n\lambda }} \right) ^T\). It is easy to verify that \(\langle \langle \varPsi ,\varPhi \rangle \rangle =I\), where \(I\in {\mathbb {R}}^{2\times 2}\) is a identity matrix. Actually, we can decompose \({\mathcal {C}}_{{\mathbb {C}}}\) as \({\mathcal {C}}_{{\mathbb {C}}}=P\oplus Q\), where

$$\begin{aligned} Q=\left\{ \varPsi \in {\mathcal {C}}_{{\mathbb {C}}}:\langle \langle {{\tilde{\varPsi }}},\varPsi \rangle \rangle =0\ \mathrm {for}\ {{\tilde{\varPsi }}}\in P^* \right\} .\end{aligned}$$

Following the idea of Hassard et al. [16], the formulas determining the bifurcation direction and stability are all relative to \(\gamma =0\). In the remainder of this section, we take \(\gamma =0\) and define

$$\begin{aligned} z(t)=\dfrac{1}{S_{n\lambda }}\langle \langle q,U_t \rangle \rangle ,\ \ W(z(t),{\overline{z}}(t))=U_t-2\mathrm {Re}\{z(t)p\}. \end{aligned}$$
(5.5)

Then we obtain a center manifold \(\mathbf{C}_0\):

$$\begin{aligned} W(z,{\overline{z}})(s)=W_{20}(s)\frac{z^2}{2}+W_{11}(s)z{\overline{z}}+W_{02}(s)\frac{{\overline{z}}^2}{2}+\cdots , \end{aligned}$$

where z and \({\overline{z}}\) are local coordinates for the center manifold \(\mathbf{C}_0\) in the direction of q and \({\overline{q}}\). From (5.5), for \(\gamma =0\), we see that

$$\begin{aligned} \begin{aligned} \dot{z}(t)&=\frac{1}{S_{n\lambda }}\cdot \frac{d}{dt}\langle \langle q({\tilde{s}}),U_t \rangle \rangle =\frac{1}{S_{n\lambda }}\langle \langle q({\tilde{s}}),{\mathcal {A}}_{\tau _n}U_t \rangle \rangle + \frac{1}{S_{n\lambda }}\langle \langle q({\tilde{s}}),X_0 J(U_t,0) \rangle \rangle \\&=i\omega _{\lambda }\tau _n z(t)+\frac{1}{S_{n\lambda }}\langle q(0),J(2\mathrm {Re}\{z(t)p\}+W(z(t),{\overline{z}}(t)),0) \rangle _w \\&=i\omega _{\lambda }\tau _n z(t)+g(z,{\overline{z}}). \end{aligned} \end{aligned}$$
(5.6)

Then,

$$\begin{aligned} \begin{aligned} g(z,{\overline{z}})&=\frac{1}{S_{n\lambda }}\langle q(0),J(2\mathrm {Re}\{z(t)p\}+W(z(t),{\overline{z}}(t)),0) \rangle _w \\&=g_{20}\frac{z^2}{2}+g_{11}z{\bar{z}}+g_{02}\frac{{\bar{z}}^2}{2}+g_{21}\frac{z^2{\bar{z}}}{2}+\ldots . \end{aligned} \end{aligned}$$
(5.7)

We now calculate that

$$\begin{aligned} g_{20}&=\frac{2\lambda \tau _n e^{-i\omega _{\lambda }\tau _n}}{S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_1(\psi _{\lambda },\psi _{\lambda }) \right\rangle _w +\frac{\lambda \tau _n e^{-2i\omega _{\lambda }\tau _n}}{S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_2(\psi _{\lambda },\psi _{\lambda }) \right\rangle _w, \nonumber \\ g_{11}&=\frac{\lambda \tau _n e^{i\omega _{\lambda }\tau _n}}{S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_1(\psi _{\lambda },{\overline{\psi }}_{\lambda }) \right\rangle _w +\frac{\lambda \tau _n e^{-i\omega _{\lambda }\tau _n}}{S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_1({\overline{\psi }}_{\lambda },\psi _{\lambda }) \right\rangle _w \nonumber \\&\quad +\frac{\lambda \tau _n}{2S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_2(\psi _{\lambda },{\overline{\psi }}_{\lambda })+{\mathcal {T}}_2({\overline{\psi }}_{\lambda },\psi _{\lambda }) \right\rangle _w, \nonumber \\ g_{02}&=\frac{2\lambda \tau _n e^{i\omega _{\lambda }\tau _n}}{S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_1({\overline{\psi }}_{\lambda },{\overline{\psi }}_{\lambda }) \right\rangle _w +\frac{\lambda \tau _n e^{2i\omega _{\lambda }\tau _n}}{S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_2({\overline{\psi }}_{\lambda },{\overline{\psi }}_{\lambda }) \right\rangle _w, \nonumber \\ g_{21}&=\frac{2\lambda \tau _n}{S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_1(\psi _{\lambda },W_{11}(-1)) \right\rangle _w+\frac{\lambda \tau _n}{S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_1({\overline{\psi }}_{\lambda },W_{20}(-1)) \right\rangle _w \nonumber \\&\quad +\frac{\lambda \tau _n e^{i\omega _{\lambda }\tau _n}}{S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_1(W_{20}(0),{\overline{\psi }}_{\lambda }) \right\rangle _w+\frac{2\lambda \tau _n e^{-i\omega _{\lambda }\tau _n}}{S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_1(W_{11}(0),\psi _{\lambda }) \right\rangle _w \nonumber \\&\quad +\frac{\lambda \tau _n e^{-i\omega _{\lambda }\tau _n}}{S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_2(\psi _{\lambda },W_{11}(-1))+{\mathcal {T}}_2(W_{11}(-1),\psi _{\lambda }) \right\rangle _w \nonumber \\&\quad +\frac{\lambda \tau _n e^{i\omega _{\lambda }\tau _n}}{2S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_2(W_{20}(-1),{\overline{\psi }}_{\lambda })+{\mathcal {T}}_2({\overline{\psi }}_{\lambda },W_{20}(-1)) \right\rangle _w \nonumber \\&\quad +\frac{\lambda \tau _n}{S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_3(\psi _{\lambda },\psi _{\lambda },{\overline{\psi }}_{\lambda })+{\mathcal {T}}_3(\psi _{\lambda },{\overline{\psi }}_{\lambda },\psi _{\lambda }) \right\rangle _w+\frac{\lambda \tau _n e^{-2i\omega _{\lambda }\tau _n}}{S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_3({\overline{\psi }}_{\lambda },\psi _{\lambda },\psi _{\lambda }) \right\rangle _w \nonumber \\&\quad +\frac{\lambda \tau _n e^{-i\omega _{\lambda }\tau _n}}{S_{n\lambda }}\left\langle {{\tilde{\psi }}}_{\lambda },{\mathcal {T}}_4(\psi _{\lambda },\psi _{\lambda },{\overline{\psi }}_{\lambda })+{\mathcal {T}}_4(\psi _{\lambda },{\overline{\psi }}_{\lambda },\psi _{\lambda })+{\mathcal {T}}_4({\overline{\psi }}_{\lambda },\psi _{\lambda },\psi _{\lambda }) \right\rangle _w, \end{aligned}$$
(5.8)

where \({\mathcal {T}}_1\) is given by

$$\begin{aligned} {\mathcal {T}}_1(\varphi _1,\varphi _2)=\begin{pmatrix} \varphi _1^{(1)}\Big (r^{1\lambda }_1(x)K_{11}*\varphi _2^{(1)}+r^{1\lambda }_2(x)K_{12}*\varphi _2^{(2)}\Big ) \\ \varphi _1^{(2)}\Big (r^{2\lambda }_1(x)K_{21}*\varphi _2^{(1)}+r^{2\lambda }_2(x)K_{22}*\varphi _2^{(2)}\Big ) \end{pmatrix}, \end{aligned}$$

\({\mathcal {T}}_2=({\mathcal {T}}_2^{(1)},{\mathcal {T}}_2^{(2)})^T\) is given by

$$\begin{aligned} {\mathcal {T}}_2^{(1)}(\varphi _1,\varphi _2)&= u_{\lambda }\Big (r^{1\lambda }_{20}(x)(K_{11}*\varphi _1^{(1)})(K_{11}*\varphi _2^{(1)})+2r^{1\lambda }_{11}(x)(K_{11}*\varphi _1^{(1)})(K_{12}*\varphi _2^{(2)}) \\&\quad +r^{1\lambda }_{02}(x)(K_{12}*\varphi _1^{(2)})(K_{12}*\varphi _2^{(2)}) \Big ), \\ {\mathcal {T}}_2^{(2)}(\varphi _1,\varphi _2)&= v_{\lambda }\Big (r^{2\lambda }_{20}(x)(K_{21}*\varphi _1^{(1)})(K_{21}*\varphi _2^{(1)})+2r^{2\lambda }_{11}(x)(K_{21}*\varphi _1^{(1)})(K_{22}*\varphi _2^{(2)}) \\&\quad +r^{2\lambda }_{02}(x)(K_{22}*\varphi _1^{(2)})(K_{22}*\varphi _2^{(2)}) \Big ), \end{aligned}$$

\({\mathcal {T}}_3=({\mathcal {T}}_3^{(1)},{\mathcal {T}}_3^{(2)})^T\) is given by

$$\begin{aligned} {\mathcal {T}}_3^{(1)}(\varphi _1,\varphi _2,\varphi _3)&= \varphi _1^{(1)}\Big (r^{1\lambda }_{20}(x)(K_{11}*\varphi _2^{(1)})(K_{11}*\varphi _3^{(1)})+2r^{1\lambda }_{11}(x)(K_{11}*\varphi _2^{(1)})(K_{12}*\varphi _3^{(2)}) \\&\quad +r^{1\lambda }_{02}(x)(K_{12}*\varphi _2^{(2)})(K_{12}*\varphi _3^{(2)}) \Big ), \\ {\mathcal {T}}_3^{(2)}(\varphi _1,\varphi _2,\varphi _3)&= \varphi _1^{(2)}\Big (r^{2\lambda }_{20}(x)(K_{21}*\varphi _2^{(1)})(K_{21}*\varphi _3^{(1)})+2r^{2\lambda }_{11}(x)(K_{21}*\varphi _2^{(1)})(K_{22}*\varphi _3^{(2)}) \\&\quad +r^{2\lambda }_{02}(x)(K_{22}*\varphi _2^{(2)})(K_{22}*\varphi _3^{(2)}) \Big ), \end{aligned}$$

and \({\mathcal {T}}_4=({\mathcal {T}}_4^{(1)},{\mathcal {T}}_4^{(2)})^T\) is given by

$$\begin{aligned}&{\mathcal {T}}_4^{(1)}(\varphi _1,\varphi _2,\varphi _3) \\&\quad =u_{\lambda }\Big (r^{1\lambda }_{30}(x)(K_{11}*\varphi _1^{(1)})(K_{11}*\varphi _2^{(1)})(K_{11}*\varphi _3^{(1)})+3r^{1\lambda }_{21}(x)(K_{11}*\varphi _1^{(1)})(K_{11}*\varphi _2^{(1)})(K_{12}*\varphi _3^{(2)}) \\&\qquad +3r^{1\lambda }_{12}(x)(K_{11}*\varphi _1^{(1)})(K_{12}*\varphi _2^{(2)})(K_{12}*\varphi _3^{(2)}+r^{1\lambda }_{03}(x)(K_{12}*\varphi _1^{(2)})(K_{12}*\varphi _2^{(2)})(K_{12}*\varphi _3^{(2)}) \Big ), \\&\quad {\mathcal {T}}_4^{(2)}(\varphi _1,\varphi _2,\varphi _3) \\&\quad =v_{\lambda }\Big (r^{2\lambda }_{30}(x)(K_{21}*\varphi _1^{(1)})(K_{21}*\varphi _2^{(1)})(K_{21}*\varphi _3^{(1)})+3r^{2\lambda }_{21}(x)(K_{21}*\varphi _1^{(1)})(K_{21}*\varphi _2^{(1)})(K_{22}*\varphi _3^{(2)}) \\&\qquad +3r^{2\lambda }_{12}(x)(K_{21}*\varphi _1^{(1)})(K_{22}*\varphi _2^{(2)})(K_{22}*\varphi _3^{(2)}+r^{2\lambda }_{03}(x)(K_{22}*\varphi _1^{(2)})(K_{22}*\varphi _2^{(2)})(K_{212}*\varphi _3^{(2)}) \Big ), \end{aligned}$$

for \(\varphi _i=(\varphi _i^{(1)},\varphi _i^{(2)})^T\in {\mathbb {Y}}^2,i=1,2,3\). From (5.8), we see that there are only \(W_{20}(s)\) and \(W_{11}(s)\) in \(g_{21}\) left to calculate.

It can be deduced from (5.3) and (5.5) that

$$\begin{aligned} {\dot{W}}={\left\{ \begin{array}{ll} {\mathcal {A}}_{\tau _n} W-gp(s)-\overline{g}\overline{p}(s), &{}s \in [-1,0), \\ {\mathcal {A}}_{\tau _n} W-gp(0)-\overline{g}\overline{p}(0)+J(2\mathrm {Re}\{z(t)p\}+W(z(t),{\overline{z}}(t)),0), &{} s=0. \end{array}\right. }\qquad \end{aligned}$$
(5.9)

Meanwhile, W also satisfies that

$$\begin{aligned} {\dot{W}}&=W_z {\dot{z}}+W_{{\bar{z}}} \dot{\overline{z}} \\&=\left[ W_{20}(s)z+W_{11}(s)\overline{z} \right] {\dot{z}}+\left[ W_{11}(s)z+W_{02}(s)\overline{z} \right] \dot{\overline{z}}+\cdots \\&=\left[ W_{20}(s)z+W_{11}(s)\overline{z} \right] (i\theta _{n\lambda }z+g(z,\overline{z})) \\&\quad +\left[ W_{11}(s)z+W_{02}(s)\overline{z} \right] (-i\theta _{n\lambda }\overline{z} +\overline{g}(z,\overline{z}))+\cdots , \end{aligned}$$

on the center manifold \(\mathbf{C}_0\) near the origin. Combining the above equation with Eq. (5.9), we have

$$\begin{aligned} (2i\theta _{n\lambda }I-{\mathcal {A}}_{\tau _n})W_{20}(s)= {\left\{ \begin{array}{ll} -g_{20}p(s)-\overline{g}_{02}\overline{p}(s), &{} s \in [-1,0), \\ -g_{20}p(0)-\overline{g}_{02}\overline{p}(0)+2\lambda \tau _n e^{-i\omega _{\lambda }\tau _n}{\mathcal {T}}_1(\psi _{\lambda },\psi _{\lambda }) &{} \\ +\lambda \tau _n e^{-2i\omega _{\lambda }\tau _n}{\mathcal {T}}_2(\psi _{\lambda },\psi _{\lambda }), &{} s=0, \end{array}\right. } \end{aligned}$$
(5.10)

and

$$\begin{aligned} -{\mathcal {A}}_{\tau _n} W_{11}(s)= {\left\{ \begin{array}{ll} -g_{11}p(s)-\overline{g}_{11}\overline{p}(s), &{} s\in [-1,0), \\ -g_{11}p(0)-\overline{g}_{11}\overline{p}(0)+\lambda \tau _n e^{-i\omega _{\lambda }\tau _n}{\mathcal {T}}_1({\overline{\psi }}_{\lambda },\psi _{\lambda }) &{} \\ +\lambda \tau _n e^{i\omega _{\lambda }\tau _n}{\mathcal {T}}_1(\psi _{\lambda },{\overline{\psi }}_{\lambda })+\frac{\lambda \tau _n}{2}\left( {\mathcal {T}}_2(\psi _{\lambda },{\overline{\psi }}_{\lambda })+{\mathcal {T}}_2({\overline{\psi }}_{\lambda },\psi _{\lambda }) \right) , &{} s=0. \end{array}\right. } \end{aligned}$$
(5.11)

To compute \(W_{20}\), from (5.10), we have

$$\begin{aligned} W_{20}'(s)=2i\theta _{n\lambda }W_{20}(s)+g_{20}p(s)+\overline{g}_{02}\overline{p}(s),\ \ s \in [-1,0). \end{aligned}$$

Note that \(p(s)=\psi _{\lambda }e^{i\omega _{\lambda }\tau _ns}\), then there holds that

$$\begin{aligned} W_{20}(s)=\frac{ig_{20}}{\omega _{\lambda }\tau _n}p(s)+\frac{i\overline{g}_{02}}{3\omega _{\lambda }\tau _n}\overline{p}(s)+E e^{2i\omega _{\lambda }\tau _n s}. \end{aligned}$$
(5.12)

Especially, Eqs. (5.10) and (5.12) imply that

$$\begin{aligned} (2i\omega _{\lambda }\tau _n I-{\mathcal {A}}_{\tau _n})E e^{2i\theta _{n\lambda }s}\bigg |_{s=0}=2\lambda \tau _n e^{-i\omega _{\lambda }\tau _n}{\mathcal {T}}_1(\psi _{\lambda },\psi _{\lambda })+\lambda \tau _n e^{-2i\omega _{\lambda }\tau _n}{\mathcal {T}}_2(\psi _{\lambda },\psi _{\lambda }), \end{aligned}$$

or equivalently,

$$\begin{aligned} \varLambda (\lambda ,2i\omega _{\lambda },\tau _n)E=-2\lambda e^{-i\omega _{\lambda }\tau _n}{\mathcal {T}}_1(\psi _{\lambda },\psi _{\lambda })-\lambda e^{-2i\omega _{\lambda }\tau _n}{\mathcal {T}}_2(\psi _{\lambda },\psi _{\lambda }). \end{aligned}$$
(5.13)

Notice that \(2i\omega _{\lambda }\) is not the eigenvalue of \(A_{\tau _n,\lambda }\) for \(\lambda \in (\lambda _*,{{\tilde{\lambda }}}^*]\). Then

$$\begin{aligned} E=-2\lambda e^{-i\omega _{\lambda }\tau _n}\varLambda (\lambda ,2i\omega _{\lambda },\tau _n)^{-1}{\mathcal {T}}_1(\psi _{\lambda },\psi _{\lambda })-\lambda e^{-2i\omega _{\lambda }\tau _n}\varLambda (\lambda ,2i\omega _{\lambda },\tau _n)^{-1}{\mathcal {T}}_2(\psi _{\lambda },\psi _{\lambda }). \end{aligned}$$

Similarly, we can derive from (5.11) that, for \(s \in [-1,0)\),

$$\begin{aligned} W_{11}(s)=-\frac{ig_{11}}{\omega _{\lambda }\tau _n}p(s) +\frac{i\overline{g}_{11}}{\omega _{\lambda }\tau _n}\overline{p}(s)+F, \end{aligned}$$
(5.14)

and when \(s=0\), F satisfies

$$\begin{aligned} -{\mathcal {A}}_\tau F=\lambda \tau _n e^{-i\omega _{\lambda }\tau _n}{\mathcal {T}}_1({\overline{\psi }}_{\lambda },\psi _{\lambda })+\lambda \tau _n e^{i\omega _{\lambda }\tau _n}{\mathcal {T}}_1(\psi _{\lambda },{\overline{\psi }}_{\lambda })+\frac{\lambda \tau _n}{2}\left( {\mathcal {T}}_2(\psi _{\lambda },{\overline{\psi }}_{\lambda })+{\mathcal {T}}_2({\overline{\psi }}_{\lambda },\psi _{\lambda }) \right) . \end{aligned}$$

Thus, we obtain

$$\begin{aligned} F= & {} -\lambda \varLambda (\lambda ,0,\tau _{n\lambda })^{-1}\left[ e^{-i\omega _{\lambda }\tau _n}{\mathcal {T}}_1({\overline{\psi }}_{\lambda },\psi _{\lambda })\right. \nonumber \\&\left. +e^{i\omega _{\lambda }\tau _n}{\mathcal {T}}_1(\psi _{\lambda },{\overline{\psi }}_{\lambda })+\frac{1}{2}\left( {\mathcal {T}}_2(\psi _{\lambda },{\overline{\psi }}_{\lambda })+{\mathcal {T}}_2({\overline{\psi }}_{\lambda },\psi _{\lambda }) \right) \right] . \end{aligned}$$
(5.15)

Lemma 6

Let E and F be defined in (5.13) and (5.15), respectively. Assume that \(\mathbf{(H_1)}-\mathbf{(H_3)}\) hold, then for \(\lambda \in (\lambda _*,{{\tilde{\lambda }}}^*]\),

$$\begin{aligned} E=\dfrac{1}{\lambda -\lambda _*}(\rho _{\lambda }U_{\lambda }+\varphi _{\lambda }),\ \ F=\dfrac{{{\tilde{\varphi }}}_{\lambda }}{\lambda -\lambda _*}, \end{aligned}$$
(5.16)

where \(U_{\lambda }=(u_{\lambda },v_{\lambda })^T\) is defined in (2.3), \(\varphi _{\lambda }\) and \({{\tilde{\varphi }}}_{\lambda }\) satisfy

$$\begin{aligned} \langle U_{\lambda },\varphi _{\lambda } \rangle =0,\ \ \lim _{\lambda \rightarrow \lambda _*}\Vert \varphi _{\lambda } \Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}=0,\ \ \lim _{\lambda \rightarrow \lambda _*}\Vert {{\tilde{\varphi }}}_{\lambda } \Vert _{{\mathbb {Y}}_{{\mathbb {C}}}}=0, \end{aligned}$$

and the constant \(\rho _{\lambda }\) satisfies

$$\begin{aligned} \lim _{\lambda \rightarrow \lambda _*}(\lambda -\lambda _*)\rho _{\lambda }= \dfrac{2i(1+p_{1\lambda _*}^2)(\kappa _{22}-\kappa _{12})(\kappa _{11}+\kappa _{12}p_{1\lambda _*})}{(\beta _{\lambda _*}^{2}+c_{\lambda _*}^2)\left[ 2i(\kappa _{22}-\kappa _{12})(\kappa _{11}+\kappa _{12}p_{1\lambda _*})-(\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21}) \right] }. \end{aligned}$$

Proof

We only show the estimate for E, and that for F can be proved in a similar manner. Since \(A_{\lambda }U_{\lambda }=0\), by substituting E, defined as in (5.16), into Eq. (5.13), we obtain

$$\begin{aligned} \begin{aligned}&A_{\lambda }\varphi _{\lambda }+B_{\lambda }(\rho _{\lambda }U_{\lambda }+\varphi _{\lambda })e^{-2i\omega _{\lambda }\tau _n}-2i\omega _{\lambda }(\rho _{\lambda }U_{\lambda }+\varphi _{\lambda }) \\&\quad =-2\lambda (\lambda -\lambda _*) e^{-i\omega _{\lambda }\tau _n}{\mathcal {T}}_1(\psi _{\lambda },\psi _{\lambda })-\lambda (\lambda -\lambda _*) e^{-2i\omega _{\lambda }\tau _n}{\mathcal {T}}_2(\psi _{\lambda },\psi _{\lambda }). \end{aligned} \end{aligned}$$
(5.17)

Calculating the weighted inner product of Eq. (5.17) with \(U_{\lambda }\) gives that

$$\begin{aligned} \begin{aligned}&\rho _{\lambda }\left[ e^{-2i\omega _{\lambda }\tau _n}\left\langle U_{\lambda },B_{\lambda }U_{\lambda } \right\rangle _w-2i\omega _{\lambda }\left\langle U_{\lambda },U_{\lambda } \right\rangle _w \right] \\&\quad =-e^{-2i\omega _{\lambda }\tau _n}\left\langle U_{\lambda },B_{\lambda }\varphi _{\lambda } \right\rangle _w+2i\omega _{\lambda }\left\langle U_{\lambda },\varphi _{\lambda } \right\rangle _w\\&\qquad -2\lambda (\lambda -\lambda _*) e^{-i\omega _{\lambda }\tau _n}\left\langle U_{\lambda },{\mathcal {T}}_1(\psi _{\lambda },\psi _{\lambda }) \right\rangle _w \\&\qquad -\lambda (\lambda -\lambda _*) e^{-2i\omega _{\lambda }\tau _n}\left\langle U_{\lambda },{\mathcal {T}}_2(\psi _{\lambda },\psi _{\lambda }) \right\rangle _w. \end{aligned} \end{aligned}$$
(5.18)

By calculating the weighted inner product of Eq. (5.17) with \(\varphi _{\lambda }\), we see

$$\begin{aligned} \begin{aligned}&\left\langle \varphi _{\lambda },A_{\lambda }\varphi _{\lambda } \right\rangle _w +\rho _{\lambda } e^{-2i\omega _{\lambda }\tau _n}\left\langle \varphi _{\lambda },B_{\lambda }U_{\lambda } \right\rangle _w -2i\omega _{\lambda }\rho _{\lambda } \left\langle \varphi _{\lambda },U_{\lambda } \right\rangle _w \\&\quad =-e^{-2i\omega _{\lambda }\tau _n}\left\langle \varphi _{\lambda },B_{\lambda }\varphi _{\lambda } \right\rangle _w +2i\omega _{\lambda } \left\langle \varphi _{\lambda },\varphi _{\lambda } \right\rangle _w \\&\qquad -2\lambda (\lambda -\lambda _*) e^{-i\omega _{\lambda }\tau _n}\left\langle \varphi _{\lambda },{\mathcal {T}}_1(\psi _{\lambda },\psi _{\lambda }) \right\rangle _w \\&\qquad -\lambda (\lambda -\lambda _*) e^{-2i\omega _{\lambda }\tau _n}\left\langle \varphi _{\lambda },{\mathcal {T}}_2(\psi _{\lambda },\psi _{\lambda }) \right\rangle _w. \end{aligned} \end{aligned}$$
(5.19)

Recall that

$$\begin{aligned} \begin{aligned}&\psi _{\lambda }\rightarrow \begin{pmatrix} \phi \\ p_{1\lambda _*}\phi \end{pmatrix},\ \ U_{\lambda }/(\lambda -\lambda _*)\rightarrow \begin{pmatrix} \beta _{\lambda _*}\phi \\ c_{\lambda _*}\phi \end{pmatrix}\ \mathrm {in}\ C({\overline{\varOmega }})\times C({\overline{\varOmega }}), \\&\omega _{\lambda }/(\lambda -\lambda _*)\rightarrow h_{\lambda _*}, \ \ \omega _{\lambda }\tau _n\rightarrow \frac{\pi }{2}+2n\pi (n=0,1,2,\ldots ),\ \ \mathrm {as}\ \lambda \rightarrow \lambda _*. \end{aligned} \end{aligned}$$
(5.20)

Then we have

$$\begin{aligned}&\lim _{\lambda \rightarrow \lambda _*}\frac{\left\langle U_{\lambda },B_{\lambda }U_{\lambda } \right\rangle _w}{(\lambda -\lambda _*)^3} =-(\beta _{\lambda _*}^2+c_{\lambda _*}^2)\int _{\varOmega }m(x)e^{\alpha m(x)}\phi ^2\mathrm {d}x, \\&\quad \lim _{\lambda \rightarrow \lambda _*}\frac{\lambda \left\langle U_{\lambda },{\mathcal {T}}_1(\psi _{\lambda },\psi _{\lambda }) \right\rangle _w}{\lambda -\lambda _*} =-(1+p_{1\lambda _*}^2)h_{\lambda _*}\int _{\varOmega }e^{\alpha m(x)}\phi ^2\mathrm {d}x, \\&\quad \lim _{\lambda \rightarrow \lambda _*}\frac{\lambda \left\langle U_{\lambda },{\mathcal {T}}_2(\psi _{\lambda },\psi _{\lambda }) \right\rangle _w}{\lambda -\lambda _*}=0, \\&\quad \lim _{\lambda \rightarrow \lambda _*}\frac{\omega _{\lambda }\left\langle U_{\lambda },U_{\lambda } \right\rangle _w}{(\lambda -\lambda _*)^3} =h_{\lambda _*}(\beta _{\lambda _*}^2+c_{\lambda _*}^2)\int _{\varOmega }e^{\alpha m(x)}\phi ^2\mathrm {d}x. \end{aligned}$$

Hence from Eq. (5.18), there exist constants \({{\tilde{\lambda }}}^*>\lambda _*\) and \(M_1,M_2>0\) so that for any \(\lambda \in (\lambda _*,{{\tilde{\lambda }}}^*)\),

$$\begin{aligned} |(\lambda -\lambda _*)\rho _{\lambda }|\le M_1\Vert \varphi _{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}^2}+M_2. \end{aligned}$$
(5.21)

Similar to the proof of Lemma 2.3 of [4], we have

$$\begin{aligned} |\langle \varphi _{\lambda },A_{\lambda }\varphi _{\lambda } \rangle _w|\ge |\lambda _2(\lambda )|\Vert \varphi _{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}^2}^2, \end{aligned}$$

where \(\lambda _2(\lambda )\) is the second eigenvalue of \(-A(\lambda )\). On the other hand, we can also obtain from (5.19), (5.20) and (5.21) that there exist constants \(M_3,M_4>0\) such that for any \(\lambda \in (\lambda _*,{{\tilde{\lambda }}})\),

$$\begin{aligned} |\lambda _2(\lambda )|\cdot \Vert \varphi _{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}^2}^2\le (\lambda -\lambda _*)M_3\Vert \varphi _{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}^2}^2 +(\lambda -\lambda _*)M_4\Vert \varphi _{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}^2}. \end{aligned}$$

Note that \(\lim \limits _{\lambda \rightarrow \lambda _*}\lambda _2(\lambda )=\lambda _2>0\), where \(\lambda _2\), defined as in Lemma 1 (i), is the second eigenvalue of \(-L\), then \(\lim \limits _{\lambda \rightarrow \lambda _*}\Vert \varphi _{\lambda }\Vert _{{\mathbb {Y}}_{{\mathbb {C}}}^2}=0\). Now, we can derive from (5.18) that

$$\begin{aligned} \lim _{\lambda \rightarrow \lambda _*}(\lambda -\lambda _*)\rho _{\lambda }= \dfrac{2i(1+p_{1\lambda _*}^2)(\kappa _{22}-\kappa _{12})(\kappa _{11}+\kappa _{12}p_{1\lambda _*})}{(\beta _{\lambda _*}^{2}+c_{\lambda _*}^2)\left[ 2i(\kappa _{22}-\kappa _{12})(\kappa _{11}+\kappa _{12}p_{1\lambda _*})-(\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21}) \right] }. \end{aligned}$$

The proof is finished. \(\square \)

From (5.8), we see that each \(g_{kl}\) is determined by the parameters of original system (1.5). Notice that

$$\begin{aligned}&\lim _{\lambda \rightarrow \lambda _*}S_{n\lambda }=(1+{\tilde{p}}_{1\lambda _*}p_{1\lambda _*})\left[ 1+i(\frac{\pi }{2}+2n\pi ) \right] \int _{\varOmega }e^{\alpha m(x)}\phi ^2 dx, \\&\quad \lim _{\lambda \rightarrow \lambda _*}(\lambda -\lambda _*)\tau _n=\frac{\frac{\pi }{2}+2n\pi }{h_{\lambda _*}},\ \ \lim _{\lambda \rightarrow \lambda _*}(\lambda -\lambda _*)F=\begin{pmatrix}0 \\ 0 \end{pmatrix}, \\&\quad \lim _{\lambda \rightarrow \lambda _*}(\lambda -\lambda _*)E= \dfrac{2i(1+p_{1\lambda _*}^2)(\kappa _{22}-\kappa _{12})(\kappa _{11}+\kappa _{12}p_{1\lambda _*})}{(\beta _{\lambda _*}^2+c_{\lambda _*}^2)\left[ 2i(\kappa _{22}-\kappa _{12})(\kappa _{11}+\kappa _{12}p_{1\lambda _*})-(\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21}) \right] } \begin{pmatrix} \beta _{\lambda _*}\phi \\ c_{\lambda _*}\phi \end{pmatrix}. \end{aligned}$$

Then we can compute that

$$\begin{aligned} \begin{aligned} \lim _{\lambda \rightarrow \lambda _*}(\lambda -\lambda _*)g_{20}&=\frac{2i(c_{\lambda _*}+\beta _{\lambda _*}{\tilde{p}}_{1\lambda _*}p_{1\lambda _*}^2)(\pi +4n\pi )}{\beta _{\lambda _*}c_{\lambda _*}(1+{\tilde{p}}_{1\lambda _*}p_{1\lambda _*})(2+i(\pi +4n\pi ))}, \\ \lim _{\lambda \rightarrow \lambda _*}(\lambda -\lambda _*)g_{11}&=0, \\ \lim _{\lambda \rightarrow \lambda _*}(\lambda -\lambda _*)g_{02}&=-\frac{2i(c_{\lambda _*}+\beta _{\lambda _*}{\tilde{p}}_{1\lambda _*}p_{1\lambda _*}^2)(\pi +4n\pi )}{\beta _{\lambda _*}c_{\lambda _*}(1+{\tilde{p}}_{1\lambda _*}p_{1\lambda _*})(2+i(\pi +4n\pi ))}, \end{aligned} \end{aligned}$$
(5.22)

which combined with (5.12) and (5.14) yields

$$\begin{aligned} \lim _{\lambda \rightarrow \lambda _*}(\lambda -\lambda _*)^2g_{21}= & {} \frac{8i(\pi +4n\pi )}{3(1+{\tilde{p}}_{1\lambda _*}p_{1\lambda _*})^2|2+i(\pi +4n\pi )|^2}\left[ \frac{1}{\beta _{\lambda _*}}+\frac{{\tilde{p}}_{1\lambda _*}p_{1\lambda _*}^2}{c_{\lambda _*}} \right] ^2 \nonumber \\&+ \, \frac{2(1+i)(1+p_{1\lambda _*}^2)(\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21})}{(\beta _{\lambda _*}^2+c_{\lambda _*}^2)\left[ 2i(\kappa _{22}-\kappa _{12})(\kappa _{11}+\kappa _{12}p_{1\lambda _*})-(\kappa _{11}\kappa _{22}-\kappa _{12}\kappa _{21}) \right] } \nonumber \\&\times \, \frac{\pi +4n\pi }{2+i(\pi +4n\pi )}. \end{aligned}$$
(5.23)

Consequently, we have obtained the normal form (5.6) restricted on the center manifold \(\mathbf{C}_0\) by computing the coefficients \(g_{20},g_{11},g_{02}\) and \(g_{21}\). Denote

$$\begin{aligned} C_1(0)=\frac{i}{2\theta _{n\lambda }}\left[ g_{11}g_{20}-2|g_{11}|^2-\frac{|g_{02}|^2}{3} \right] +\frac{g_{21}}{2}. \end{aligned}$$

Then, we have

$$\begin{aligned} \mu _2&=-\frac{\mathrm {Re}(C_1(0))}{\mathrm {Re}(\mu '(\tau _n))}, \\ \beta _2&=2\mathrm {Re}(C_1(0)), \\ T_2&=-\frac{\mathrm {Im}(C_1(0))+\mu _2\mathrm {Im}(\mu '(\tau _n))}{\tau _n}, \end{aligned}$$

which determine the properties of bifurcating periodic solutions at critical value \(\tau _n\), that is,

  1. (i)

    \(\mu _2\) determines the direction of the Hopf bifurcation: if \(\mu _2>0(<0)\), then the direction of the Hopf bifurcation is forward (backward) and the bifurcating periodic solutions exist for \(\tau > \tau _n(\tau < \tau _n)\);

  2. (ii)

    \(\beta _2\) determines the stability of the bifurcating periodic solutions: if \(\beta _2 <0( >0)\), then the bifurcating periodic solutions are orbitally asymptotically stable (unstable) on the center manifold;

  3. (iii)

    \(T_2\) determines the period of bifurcating periodic solutions: if \(T_2 >0(< 0)\), then the period of the bifurcating periodic solutions increases (decreases).

Under assumptions \(\mathbf{(H_2)}\) and \(\mathbf{(H_3)}\), there holds that

$$\begin{aligned} \lim _{\lambda \rightarrow \lambda _*}\mathrm {Re}((\lambda -\lambda _*)^2g_{21})<0, \ \ \lim _{\lambda \rightarrow \lambda _*}\mathrm {Re}((\lambda -\lambda _*)^2C_1(0))<0. \end{aligned}$$

Now, the following result is obtained.

Theorem 8

Assume that \(\mathbf{(H_1)}-\mathbf{(H_3)}\) hold, and \(\lambda \in (\lambda _*,\lambda ^*]\) with \(0<\lambda ^*-\lambda _*\ll 1\). Let \(\tau _n(\lambda )\) given as in (3.21) be the Hopf bifurcation points for Eq. (1.5) where spatially nonhomogeneous periodic orbits of Eq. (1.5) emerge from \((\tau _n,u_{\lambda },v_{\lambda })\). Then for \(n\in {\mathbb {N}} \cup \{0\}\), the direction of the Hopf bifurcation at \(\tau =\tau _n\) is forward and the bifurcating periodic solutions are orbitally asymptotically stable on the center manifold. Especially, there exist \(\varepsilon _0>0\) such that (1.5) has a locally asymptotically stable spatially nonhomogeneous periodic solution for \(\tau \in (\tau _0,\tau _0+\varepsilon _0)\).

6 A Lotka–Volterra Competition–Diffusion–Advection Model with Nonlocal Delay

In this section we choose the following Lotka–Volterra competitive system as examples

$$\begin{aligned} {\left\{ \begin{array}{ll} u_t=\big [ du_x-aum_x \big ]_x+u(x,t) &{} \\ \qquad \,\, \times \Big [m(x)-a_{11}\int _0^{\pi }k(x,y)u(y,t-\tau )\mathrm {d}y-a_{12}\int _0^{\pi }k(x,y)v(y,t-\tau )\mathrm {d}y \Big ], &{} x\in (0,\pi ),t>0, \\ v_t=\big [ dv_x-avm_x \big ]_x+v(x,t) &{} \\ \qquad \,\, \times \Big [m(x)-a_{21}\int _0^{\pi }k(x,y)u(y,t-\tau )\mathrm {d}y-a_{22}\int _0^{\pi }k(x,y)v(y,t-\tau )\mathrm {d}y \Big ], &{} x\in (0,\pi ),t>0, \\ u(x,t)=v(x,t)=0, &{} x=0,\pi ,\ t>0, \end{array}\right. }\nonumber \\ \end{aligned}$$
(6.1)

where \(a_{11},a_{22},a_{12},a_{21}>0\) and k(xy) is a continuous nonnegative function on \(\varOmega \times \varOmega \). Then (6.1) is a competitive system. Similar to Eq. (1.5), we can also obtain an equivalent model of Eq. (6.1) as follows:

$$\begin{aligned} {\left\{ \begin{array}{ll} u_t=e^{-\alpha m(x)}\nabla \cdot \left[ e^{\alpha m(x)}\nabla u\right] +\lambda u(x,t) &{} \\ \qquad \,\, \times \Big [m(x)-a_{11}\int _0^{\pi }K(x,y)u(y,t-\tau )\mathrm {d}y-a_{12}\int _0^{\pi }K(x,y)v(y,t-\tau )\mathrm {d}y \Big ], &{} x\in (0,\pi ),t>0, \\ v_t=e^{-\alpha m(x)}\nabla \cdot \left[ e^{\alpha m(x)}\nabla v\right] +\lambda v(x,t) &{} \\ \qquad \,\, \times \Big [m(x)-a_{21}\int _0^{\pi }K(x,y)u(y,t-\tau )\mathrm {d}y-a_{22}\int _0^{\pi }K(x,y)v(y,t-\tau )\mathrm {d}y \Big ], &{} x\in (0,\pi ),t>0, \\ u(x,t)=v(x,t)=0, &{} x=0,\pi ,\ t>0, \end{array}\right. }\nonumber \\ \end{aligned}$$
(6.2)

where \(\lambda =1/d,\alpha =a/d\) and \(K(x,y)=k(x,y)e^{\alpha m(y)}\). Here m(x) satisfies the assumption \(\mathbf{(H_1)}\).

Suppose that (uv) is a positive steady state of (6.2) satisfying

$$\begin{aligned} {\left\{ \begin{array}{ll} e^{-\alpha m(x)}\nabla \cdot \left[ e^{\alpha m(x)}\nabla u\right] +\lambda u(x)\Big [m(x)-a_{11}\int _0^{\pi }K(x,y)u(y)\mathrm {d}y-a_{12}\int _0^{\pi }K(x,y)v(y)\mathrm {d}y \Big ]=0, &{} x\in (0,\pi ), \\ e^{-\alpha m(x)}\nabla \cdot \left[ e^{\alpha m(x)}\nabla v\right] +\lambda v(x)\Big [m(x)-a_{21}\int _0^{\pi }K(x,y)u(y)\mathrm {d}y-a_{22}\int _0^{\pi }K(x,y)v(y)\mathrm {d}y \Big ]=0, &{} x\in (0,\pi ), \\ u(x)=v(x)=0, &{} x=0,\pi . \end{array}\right. }\nonumber \\ \end{aligned}$$
(6.3)

Multiplying the first equation of (6.3) by \(e^{\alpha m(x)}\phi \) and integrating the result over \(\varOmega \), we have

$$\begin{aligned} \lambda _*\int _0^{\pi }m(x)e^{\alpha m(x)}\phi u\mathrm {d}x&=-\int _0^{\pi }u\nabla \cdot [e^{\alpha m(x)}\nabla \phi ]\mathrm {d}x=-\int _0^{\pi }\phi \nabla \cdot [e^{\alpha m(x)}\nabla u]\mathrm {d}x \\&=\lambda \int _0^{\pi }\phi e^{\alpha m(x)}u\left[ m(x)-a_{11}\int _0^{\pi }K(x,y)u(y)\right. \\&\quad \left. -a_{12}\int _0^{\pi }K(x,y)v(y) \right] \mathrm {d}x \\&\le \lambda \int _0^{\pi }m(x)e^{\alpha m(x)}\phi u \mathrm {d}x. \end{aligned}$$

Therefore, the problem (6.2) has no positive steady state if \(\lambda <\lambda _*\).

Let

$$\begin{aligned} f_1(x,K_{11}*u,K_{12}*v)&=m(x)-a_{11}\int _0^{\pi }K(x,y)u\mathrm {d}y-a_{12}\int _0^{\pi }K(x,y)v\mathrm {d}y, \\ f_2(x,K_{21}*u,K_{22}*v)&=m(x)-a_{21}\int _0^{\pi }K(x,y)u\mathrm {d}y-a_{22}\int _0^{\pi }K(x,y)v\mathrm {d}y. \end{aligned}$$

Suppose that \(\frac{a_{11}}{a_{21}}>1>\frac{a_{12}}{a_{22}}\). Then one can easily check that \(\mathbf{(H_2)}\) and \(\mathbf{(H_3)}\) are satisfied. According to Lemma 3, Theorems 17 and 8 , we obtain the following conclusions for (6.2) which is competitive:

  1. (i)

    When \(0<\lambda <\lambda _*\), the trivial steady state (0, 0) is the unique nonnegative steady state of (6.2), which is locally asymptotically stable;

  2. (ii)

    When \(\lambda \in (\lambda _*,{{\tilde{\lambda }}}^*]\) with \(0<{{\tilde{\lambda }}}^*-\lambda _*\ll 1\), system (6.2) admits a spatially nonhomogeneous positive steady state \((u_{\lambda },v_{\lambda })\);

  3. (iii)

    For \(\lambda \in (\lambda _*,{{\tilde{\lambda }}}^*]\), there exists a critical point \(\tau _0\) such that the positive steady state \((u_{\lambda },v_{\lambda })\) is locally asymptotically stable if \(\tau \in [0,\tau _0)\), and unstable if \(\tau \in (\tau _0,\infty )\);

  4. (iv)

    System (6.2) undergoes a supercritical Hopf bifurcation at the positive steady state \((u_{\lambda },v_{\lambda })\) when \(\tau =\tau _0\), and there exists a locally stable spatially nonhomogeneous time-periodic solution for \(\tau \in (\tau _0,\tau _0+\varepsilon _0)\), where \(\varepsilon _0>0\) is small.

It follows from Lemma 2 and Theorem 4 that

$$\begin{aligned} \theta _{\lambda _*(\alpha )}(\alpha )=\frac{\pi }{2},\ \ h_{\lambda _*(\alpha )}(\alpha )=\dfrac{(a_{22}-a_{12})(a_{11}+a_{12}p_{1\lambda _*})\displaystyle \int _{\varOmega }m(x)e^{\alpha m(x)}\phi ^2 dx}{(a_{11}a_{22}-a_{12}a_{21})\displaystyle \int _{\varOmega }e^{\alpha m(x)}\phi ^2 dx}, \end{aligned}$$

where \(p_{1\lambda _*}\) is the positive root of the equation

$$\begin{aligned} a_{12}(a_{12}-a_{22})p^2+(a_{11}a_{12}-a_{22}a_{21})p-a_{21}(a_{21}-a_{11})=0, \end{aligned}$$

and the first Hopf bifurcation value satisfies

$$\begin{aligned} \tau _0(\lambda ,\alpha )=\frac{\theta _{\lambda }(\alpha )}{(\lambda -\lambda _*(\alpha ))h_{\lambda }(\alpha )}. \end{aligned}$$

Since \(\frac{a_{11}}{a_{21}}>1>\frac{a_{12}}{a_{22}}\), we have \(\frac{(a_{22}-a_{12})(a_{11}+a_{12}p_{1\lambda _*})}{(a_{11}a_{22}-a_{12}a_{21})}>0\). By the similar argument to [8, Proposition 4.7], we can derive that \(h_{\lambda _*(\alpha )}(\alpha )\) is strictly increasing with respect to \(\alpha \in [0,\infty )\). Then we show how the advection rate affects the Hopf bifurcation value with respect to sufficiently small \(\alpha >0\).

Proposition 1

Assume that \(\frac{a_{11}}{a_{21}}>1>\frac{a_{12}}{a_{22}}\) and the non-constant function m(x) satisfies \(\mathbf{(H_1)}\) and

$$\begin{aligned} m(x)>0 \ \ \mathrm {and}\ \ \varDelta m<0\ \ \mathrm {for}\ \ x\in \varOmega . \end{aligned}$$

Then there exist \(\delta _1,\delta _2>0\) such that \(\tau _0(\lambda ,\alpha )\) is strictly decreasing with respect to \(\alpha \in [0,\delta _1)\) for \(\lambda \in (\lambda _*(0),\lambda _*(0)+\delta _2)\).

Remark 2

Proposition 1 implies that Hopf bifurcation is more likely to occur when adding a term describing advection along the environmental gradients for the diffusive Lotka–Volterra competition model with nonlocal delay. One can use an argument similar to [19, Theorem 5.1] to prove Proposition 1.

In the following we give some numerical simulations to verify our analysis results. In order to maintain the real time scale, we will simulate the original competitive system (6.1), then the critical value for stability switch is \(\tau _0/d\). We take \(\varOmega =(0,\pi )\), the space step as \(\pi /50\) and the time step as 0.001. In numerical simulations, different types of patterns are observed and we have found that the distribution of species u and v is always of the same type. For the sake of simplicity, only the patterns of the distribution of species u are given here for instance. Choose the following parameter set:

$$\begin{aligned} (P)\ \ k(x,y)\equiv 1, a_{11}=0.4, a_{12}=0.1, a_{21}=0.1, a_{22}=0.4, m(x)=\sin x, x\in (0,\pi ) \end{aligned}$$

and initial condition:

$$\begin{aligned} (IC)\ \ u(x,t)=v(x,t)=0.1\sin x,\ x\in {\overline{\varOmega }},t\in [-\tau ,0]. \end{aligned}$$
Fig. 1
figure 1

Numerical simulations of (6.1) for \(d=1,a=0.01\) with parameter set (P) and initial condition (IC). Left: \(\tau =0\); Right: \(\tau =2\)

Fig. 2
figure 2

Numerical simulations of (6.1) for \(d=0.06,a=0.01\) with parameter set (P) and initial condition (IC). Left: \(\tau =1.5\); Right: \(\tau =2\)

Fig. 3
figure 3

Numerical simulations of (6.1) for \(d=0.06,a=1\) with parameter set (P) and initial condition (IC). Left: \(\tau =0.2\); Right: \(\tau =1.5\)

Fig. 4
figure 4

Numerical simulations of (6.1) for discrete delay case with \(d=0.06\), parameter set (P) and initial condition (IC). Here \(a=0.01\). Left: \(\tau =1.5\); Right: \(\tau =2\)

It follows from previous argument that system (6.1) admits no positive steady state if \(d>1/\lambda _*\). Then we choose \(d=1\) in Fig. 1, and observe that the solution of (6.1) converges to trivial steady state (0, 0) both when \(\tau =0\) and \(\tau =2\).

The influence of the time delay \(\tau \) on the solution of (6.1) can be observed clearly in Fig. 2. We first set \(d=0.06,a=0.01\). According to our theoretical analysis, when \(\tau <\tau _0/d\), the positive steady state \((u_d,v_d)=(e^{am(x)/d}{\tilde{u}}_{\lambda },e^{am(x)/d}{\tilde{v}}_{\lambda })\) of (6.1) is locally asymptotically stable, while if \(\tau >\tau _0/d\) a forward Hopf bifurcation occurs, the positive steady state \((u_d,v_d)\) loses its stability and the bifurcation periodic solution is stable. The left graph in Fig. 2 show the existence of stable nonhomogeneous positive steady state and the right graph in Fig. 2 depicts the occurrence of stable periodic solutions with obvious oscillation. In Fig. 3, letting \(a=1\), we see that the solution of (6.1) converges to a positive steady state when \(\tau =0.2\); when \(\tau =1.5\), the solution of (6.1) converges to a time-periodic solution. Then Figs. 2, 3 show that, the critical Hopf bifurcation value for stability switch decreases as the advection rate a increasing.

In Fig. 4, we show the simulation for the discrete delay case that \(k(x,y)=\delta (x-y)\) in (6.1), which has been studied in [21]. By comparing Figs. 2 and 4, we see that the nonlocal delay makes the positive steady state and time-periodic solution smaller. In biology, this means that the nonlocal delay causes the intraspecific and interspecific competitions more fiercely.