1 Introduction and Main Results

In this paper, we concerned with the following nonlinear Dirac equation with critical nonlinearities

$$\begin{aligned} -i\varepsilon \alpha \cdot \nabla u+a\beta u+V(x)u=K(x)|u|^{p-2}u+Q(x)|u|u, \end{aligned}$$
(1.1)

where \(u:\mathbb {R}^{3}\rightarrow \mathbb {C}^{4}\) is a spinor field, \(\nabla =(\frac{\partial }{\partial x_{1}},\frac{\partial }{\partial x_{2}},\frac{\partial }{\partial x_{3}})\), \(a>0\) is a constant. \(\alpha _{1}\), \(\alpha _{2}\), \(\alpha _{3}\) and \(\beta \) are \(4\times 4\) Pauli-Dirac matrices:

$$\begin{aligned} \alpha _{k}=\left( \begin{array}{cc} 0 &{} \sigma _{k}^{*} \\ \sigma _{k} &{} 0 \\ \end{array} \right) ~~1\le k\le 3,~~~\beta =\left( \begin{array}{cc} I_{2} &{} 0 \\ 0 &{} -I_{2} \\ \end{array} \right) \end{aligned}$$

with

$$\begin{aligned} \sigma _{1}=\left( \begin{array}{cc} 0 &{} 1 \\ 1 &{} 0 \\ \end{array} \right) ,~~~\sigma _{2}=\left( \begin{array}{cc} 0 &{} -i \\ i &{} 0 \\ \end{array} \right) ,~~~\sigma _{3}=\left( \begin{array}{cc} 1 &{} 0 \\ 0 &{} -1 \\ \end{array} \right) , \end{aligned}$$

where \(\sigma _{k}^{*}\) is the conjugate transpose of \(\sigma _{k}\). It is well known that the most general form of Eq. (1.1) is

$$\begin{aligned} -i\hbar \partial _{t}\psi =ic\hbar \Sigma _{k=1}^{3}\alpha _{k}\partial _{k}\psi -mc^{2}\beta \psi -M(x)\psi +F_{\psi }(x,\psi ), \end{aligned}$$
(1.2)

where \(\hbar \) stands for Planck constant, denotes the mass of particle, c is speed of light. Equation (1.2) plays an important role in quantum electrodynamics [21]. In mathematics, under the assumptions \(F(x,e^{i\theta }\psi )=F(x,\psi )\) for any \(\theta \in [0,2\pi ]\) and \(\psi (t,x)=e^{\frac{i\mu }{\hbar }t}w(x)\), then the Eq.  (1.2) is equivalent to the following stationary equation

$$\begin{aligned} i\hbar \Sigma _{k=1}^{3}\alpha _{k}\partial _{k} w+a\beta w+V(x)w=F_{w}(x,w), \end{aligned}$$
(1.3)

where \(a=mc\), \(V(x)=(\frac{M(x)}{c}+\mu )I_{4}\) and \(F_{w}(x,w)=\frac{1}{c}F_{\psi }(x,\psi )\). Especially, Eq.  (1.1) can be regarded as a generalized stationary equation of (1.2) in the case that \(F=\frac{1}{p}K(x)|\psi |^{p}+\frac{1}{3}Q(x)|\psi |^{3}\) and \(\varepsilon =\hbar \). The external fields in (1.3) arise in models of mathematical models of particle physics for many years [22, 24]. The most common examples of nonlinear Dirac equation are the massive Thirring model [29] (vector self-interaction) and the Soler model [27] (scalar self-interaction). Various nonlinearities appear in models for unified field theories. For more physical background one can refer to [28].

For the Soler model \(F(w)=\frac{1}{2}H(w\overline{w})\), \(H\in C^{2}(\mathbb {R},\mathbb {R})\), by using variational methods, Esteban and Séré [19] obtained infinitely many solutions under the following assumptions:

$$\begin{aligned} V(x)\equiv \omega ,~H'(s)\cdot s\ge \theta H(s),~F(-w)=F(w)~\text {and }\omega \in (-a,0) \end{aligned}$$

for all \(s\in \mathbb {R}\) and some \(\theta >1\). This may be the first literature to study the nonlinear Dirac equation by using variational theory. After that, Bartsch and Ding [2] obtained the standarding wave solution of Eq. (1.3) under V(x) and F(xw) are period depend on x. This is a change in the study of nonlinear Dirac equations from autonomous systems to non-autonomous systems. Their work benefits from the critical point theory of strongly indefinite functional developed in [1]. Further, Ding and Ruf [16] considered the Coulomb-type potential and obtained the existence and multiplicity of solutions for asymptotically quadratic nonlinearities. For more results on the existence and multiplicity of solutions of (1.3), we refer to the literature [12, 20] and their references.

According to [13], when the Plank constant \(\hbar >0\) is small enough and tends to zero, the solution of (1.3) is called semiclassical states. From physical point of view, this is related to the correspondence principle proposed by Niels. Bohr in the early development of quantum mechanics. This principle describes a corresponding relationship between quantum mechanics and classical mechanics, it provides a new view of physics. To the best of our knowledge, there have been many literatures seeking the existence and concentration phenomenon of the semiclassical states for nonlinear Dirac equations. Under the condition \(V(x)=0\) and \(F_{w}(x,w)=P(x)|w|^{p-2}w\), \(2<p<3\), Ding [13] obtained ground state solutions of (1.3) which concentrate the maximum points of P(x) as \(\hbar \rightarrow 0\), it is the first result about semiclassical state of the nonlinear Dirac equation. This results was later generalized to the case

$$\begin{aligned} V(x)\not \equiv 0,~~\min _{x\in \mathbb {R}^{3}}V(x)<\liminf _{|x|\rightarrow \infty }V(x) \end{aligned}$$
(1.4)

and the nonlinearity with the form \(F_{w}(x,w)=f(|w|)w\) in [15], where nonlinearity is subcritical. When the potential V satisfies (1.4), Ding and Ruf [17] also considered Eq. (1.3) with the nonlinearity \(F_{w}(x,w)=P(x)(g(|w|)+|w|)w\). In [18], Ding and Xu proposed the following local condition of the potential V(x): there is a bounded domain \(\Lambda \subset \mathbb {R}^{3}\) such that

$$\begin{aligned} \min _{x\in \overline{\Lambda }}V(x)<\min _{x\in \partial \Lambda }V(x) \end{aligned}$$
(1.5)

and they established the same conclusion as [15]. It is worth mentioning that this local condition (1.5) weakens (1.4). In fact, (1.5) is similar to the classical global condition proposed by Rabinowitz [26] in nonlinear Schödinger equation. For more semiclassical results, we refer the reader to the surveys [5,6,7, 14, 31, 32, 34] for reference to the literature.

In this paper, we first construct the semiclassical states of the critical Dirac equation with degenerate potential(the potential V may attain \(\pm a\) or approach \(\pm a\) at \(\infty \)), and then discuss the concentration phenomenon of the semiclassical state as \(\hbar =\varepsilon \rightarrow 0\). To state our main results, we need the following assumptions.

  1. (V)

    \(V\in C^{1}(\mathbb {R}^{3},\mathbb {R})\) satisfies \(\sup _{\mathbb {R}^{3}}|V(x)|\le a\), there exist the constants \(\tau \in (0,2)\) and \(\nu \in (0,+\infty ),\) such that

    $$\begin{aligned} a-|V(x)|\ge \frac{\nu }{1+|x|^{\tau }}. \end{aligned}$$
  2. (K)

    \(K\in C^{1}(\mathbb {R}^{3},\mathbb {R})\) and \(0< k_{1}\le K(x)\le k_{2}(1+|x|)^{\tau '}\) for any \(x\in \mathbb {R}^{3}\) with constants \(k_{1}>0\), \(k_{2}>0\) and \(\tau '>0\).

  3. (Q)

    \(Q\in C^{1}(\mathbb {R}^{3},\mathbb {R})\) and \(0< q_{1}\le Q(x)\le q_{2}<\infty \) for any \(x\in \mathbb {R}^{3}\) with constants \(q_{1}>0\) and \(q_{2}>0\).

  4. (S)

    There is a bounded domain \(\Lambda \subset \mathbb {R}^{3}\) with smooth boundary \(\partial \Lambda \) such that

    $$\begin{aligned}&\vec {n}(x)*\nabla V(x)>0,~\nabla K(x)*\nabla V(x)<0~\text {for any }x\in \partial \Lambda , \\&\nabla Q(x)*\nabla V(x)<0,~\nabla Q(x)*\nabla K(x)>0~\text {for any }x\in \partial \Lambda , \end{aligned}$$

    where \(\vec {n}(x)\) denotes the unit outward normal vector to \(\partial \Lambda \) at x.

Without loss of generality, we assume \(0\in \Lambda \). For any set \(\Omega \subset \mathbb {R}^{3}\), \(\delta >0\), \(\varepsilon >0\), we define

$$\begin{aligned}&\Omega ^{\delta }=\left\{ x\in \mathbb {R}^{3}:\text {dist}(x,\Omega ):=\inf _{y\in \Omega }|x-y|<\delta \right\} ,\\&\Omega _{\varepsilon }=\left\{ x\in \mathbb {R}^{3}:\varepsilon x\in \Omega \right\} . \end{aligned}$$

Denote for \(\delta >0\) small \(\mathcal {O}(\delta )=\{x\in \Lambda :\text {dist}(x,\partial \Lambda )>\delta \}\). Then there is \(\delta _{0}>0\) such that \(\sup _{\Lambda ^{\delta _{0}}\backslash \mathcal {O}(\delta _{0})}\nabla K(x)*\nabla V(x)<0\) and \(\sup _{\Lambda ^{\delta _{0}}\backslash \mathcal {O}(\delta _{0})}\nabla Q(x)*\nabla V(x)<0\). The main results of this paper are as follows.

Theorem 1.1

Suppose that assumptions (V),(K),(Q) and (S) hold. Then, for \(p\in (5/2,3)\), there exists \(\varepsilon _{0}>0\) such that if \(0<\varepsilon <\varepsilon _{0}\), Eq. (1.1) has a nontrivial solution \(u_{\varepsilon }\), satisfying that for any \(\delta >0\), there exist \(C_{1}=C_{1}(\delta )>0\) and \(C_{2}=C_{2}(\delta )>0\) such that

$$\begin{aligned} |u_{\varepsilon }|\le C_{2} \exp \left( -C_{1}\left( \frac{{\text {dist}}(x,\mathcal {O}(\delta ))}{\varepsilon }\right) ^{\frac{2-\tau }{2}}\right) . \end{aligned}$$

Our problem concerns the Sobolev critical situations, so it is difficult to deal with compactness in order to get semiclassical state. As we will see, the energy functional associated to Eq.  (1.1) is strongly indefinite. Thus, we cannot use the standard critical point theory [33] to solve it. On the other hand, we allow the potential V(x) can be reach a or tends to a at infinite. This potential V destroys the linking structure of the energy functional. In order to overcome these difficulties, we follow the methods in references [6] and [32]. We first introduce a truncation function and adjust the nonlinear term appropriately. Secondly, we make use of an idea of the penalization approach similar to that used in [4, 8, 9] in the energy functional by subtracting a penalized functional term \(P_{\varepsilon }\), which ensures the linking structure of the energy functional. Combining truncation techniques and the penalization functional \(P_{\varepsilon }\), it makes the Palais-Smale sequences bounded and relatively compact, so we can deal with the modified problem. Finally, by some regularity and \(L^{\infty }\) estimate of solutions which solves modified problem, we can get the semiclassical state of Eq. (1.1).

The paper is organised as follows. In the next section we present some preliminary notions on the Dirac operator, introduce the modified functional and give some basic lemmas. In Sect. 3, by using an abstract linking theorem, we prove the existence of nontrivial solutions of the modified problems when \(\varepsilon \) is small. In Sect.  4, we give a profile decomposition with respect to a family of solution \(\{u_{\varepsilon }\}\) which obtained in Sect.  3 and get some regularity estimates on the \(\{u_{\varepsilon }\}\). Finally, in Sect.  5, we finish the proof of the main theorem.

2 Preliminaries

Firstly, using the scaling \(w(x)=u(\varepsilon x)\), we can rewrite the Eq. (1.1) as the following equivalent equation

$$\begin{aligned} -i\alpha \cdot \nabla w+a\beta w+V(\varepsilon x)w=K(\varepsilon x)|w|^{p-2}w+Q(\varepsilon x)|w|w~~~x\in \mathbb {R}^{3}. \end{aligned}$$
(2.1)

If w is a solution of Eq. (2.1), then \(u(x):=w(x/\varepsilon )\) is a solution of the Eq. (1.1). Therefore, we will mainly focus on this equivalent equation in the remaining part of the paper.

For convenience, let \(H_{0}:=-i\alpha \cdot \nabla +a\beta \) denotes the Dirac operator, it is a self-adjoint operator on \(L^{2}(\mathbb {R}^{3},\mathbb {C}^{4})\) with domain \(\mathcal {D}(H_{0})=H^{1}(\mathbb {R}^{3},\mathbb {C}^{4})\). According to [19], we know that

$$\begin{aligned} \sigma (H_{0})=\sigma _{c}(H_{0})=\mathbb {R}\backslash (-a,a), \end{aligned}$$

where \(\sigma (H_{0})\) and \(\sigma _{c}(H_{0})\) denote the spectrum and the continuous spectrum of \(H_{0}\), respectively. Consequently, the space \(L^{2}(\mathbb {R}^{3},\mathbb {C}^{4})\) possesses the orthogonal decomposition:

$$\begin{aligned} L^{2}(\mathbb {R}^{3},\mathbb {C}^{4})=L^{+}\oplus L^{-},~~u=u^{+}+u^{-} \end{aligned}$$

such that \(H_{0}\) is positive definite in \(L^{+}\) and negative in \(L^{-}\). Let \(|H_{0}|\) denote the absolute value of \(H_{0}\) and \(|H_{0}|^{\frac{1}{2}}\) denote its square root. We define \(E:=\mathcal {D}(|H_{0}|^{\frac{1}{2}})\), then by [19], we know that E is a Hilbert space if endowed with the inner product

$$\begin{aligned} (u,v)=\text {Re}\left( |H_{0}|^{\frac{1}{2}}u,|H_{0}|^{\frac{1}{2}}v \right) _{L_{2}}, \end{aligned}$$

and the induced norm \(\Vert u\Vert ^{2}=(u,u)\), where Re stands for the real part of a complex number. By [19], this norm is equivalent to the usual \(H^{\frac{1}{2}}(\mathbb {R}^{3},\mathbb {C}^{4})\)-norm, therefore, E embeds continuously into \(L^{q}(\mathbb {R}^{3},\mathbb {C}^{4})\) for all \(q\in [2,3]\) and compactly into \(L_{loc}^{q}(\mathbb {R}^{3},\mathbb {C}^{4})\) for all \(q\in [1,3)\). Moreover, since \(\sigma (H_{0})=\mathbb {R}\backslash (-a,a)\), we have

$$\begin{aligned} a|u|_{2}^{2}\le \Vert u\Vert ^{2},~~\text {for all }u\in E. \end{aligned}$$
(2.2)

Furthermore, E can be decomposed as follows

$$\begin{aligned} E=E^{+}\oplus E^{-}, \end{aligned}$$

where \(E^{+}=E\cap L^{+}\) and \(E^{-}=E\cap L^{-}\) and the sum is orthogonal with respect to inner product \((\cdot ,\cdot )\) and \((\cdot ,\cdot )_{L_{2}}\). In addition, it follows from [18, Proposition 2.1] that

$$\begin{aligned} c_{q}\Vert u^{\pm }\Vert _{q}^{q}\le \Vert u\Vert _{q}^{q}~~\text {for all }u\in E, \end{aligned}$$

where \(c_{q}>0\) is a constant.

The energy functional of (2.1) is

$$\begin{aligned} J_{\varepsilon }(w)=&\frac{1}{2}\int _{\mathbb {R}^{3}}(-i\alpha \cdot \nabla w,w)+(a\beta w,w){\text{ d }}x+\frac{1}{2}\int _{\mathbb {R}^{3}}(V(\varepsilon x)w,w){\text{ d }}x\\&-\frac{1}{p}\int _{\mathbb {R}^{3}}K(\varepsilon x)|w|^{p}{\text{ d }}x-\frac{1}{3}\int _{\mathbb {R}^{3}}Q(\varepsilon x)|w|^{3}{\text{ d }}x. \end{aligned}$$

By the decomposition \(E=E^{+}\oplus E^{-}\), we can rewrite \(J_{\varepsilon }\) as follows

$$\begin{aligned} J_{\varepsilon }(w)=&\frac{1}{2}(\Vert w^{+}\Vert ^{2}-\Vert w^{-}\Vert ^{2})+\frac{1}{2}\int _{\mathbb {R}^{3}}(V(\varepsilon x)w,w){\text{ d }}x\\&-\frac{1}{p}\int _{\mathbb {R}^{3}}K(\varepsilon x)|w|^{p}dx-\frac{1}{3}\int _{\mathbb {R}^{3}}Q(\varepsilon x)|w|^{3}{\text{ d }}x. \end{aligned}$$

According to standard arguments, we know that \(J_{\varepsilon }:E\rightarrow \mathbb {R}\) is of class \(C^{1}\). For \(w,v\in E\), there holds

$$\begin{aligned} J_{\varepsilon }'(w)v=\text {Re}\int _{\mathbb {R}^{3}}(H_{0}w+V(\varepsilon x)w-K(\varepsilon x)|w|^{p-2}w-Q(\varepsilon x)|w|w)\cdot v {\text{ d }}x, \end{aligned}$$

where \(w\cdot v\) express the usual inner product in \(\mathbb {C}^{4}\). Moreover, in [15, Lemma 2.1] it is proved that critical points of \(J_{\varepsilon }\) are weak solutions of nonlinear Dirac Eq. (2.1).

From now on, we will construct a penalized functional \(P_{\varepsilon }\) as that used in [4, 10, 11] and a truncation function as that used in [6] and so that our modified functional have nontrivial critical points.

Let \(\varphi \in C^{\infty }(\mathbb {R}^{+},[0,1])\) be a cut-off function such that \(\varphi (t)=1\) if \(0\le t\le 1\), \(\varphi (t)=0\) if \(t\ge 2\) and for any \(t\ge 0\). Set \(b_{\varepsilon }(t)=\varphi (\varepsilon t)\) and \(m_{\varepsilon }(t)=\int _{0}^{t}b_{\varepsilon }(s){\text{ d }}s\) for any \(t\ge 0\).

Let \(\zeta \in C^{\infty }(\mathbb {R}^{+},[0,1])\) be a cut-off function such that \(\zeta (t)=0\) if \(t\ge \delta _{0}\), and \(\zeta (t)=1\) if \(0\le t\le \delta _{0}/2\), and \(\zeta '(t)\le 0\) for any \(t\ge 0\). Define \(\chi (x)=\zeta (\text {dist}(x,\Lambda ))\) and

$$\begin{aligned} g_{\varepsilon }(x,t)=\min \{h_{\varepsilon }(x,t),\phi (x)\}~~\text {for any }t\ge 0,x\in \mathbb {R}^{3}, \end{aligned}$$

where \(\phi (x)=\frac{\kappa }{1+|x|^{4+\tau '}}\) and

$$\begin{aligned} h_{\varepsilon }(x,t)&=K(\varepsilon x)t^{p-2}+\frac{p}{3}Q(\varepsilon x)t^{p-2}\left( m_{\varepsilon }(t^{2})\right) ^{\frac{3-p}{2}}\\&\quad + \frac{3-p}{3}Q(\varepsilon x)t^{p}\left( m_{\varepsilon }(t^{2})\right) ^{\frac{3-p}{2}-1}b_{\varepsilon }(t^{2}). \end{aligned}$$

Let us define

$$\begin{aligned} f_{\varepsilon }(x,t)=\chi (\varepsilon x) h_{\varepsilon }(x,t)+(1-\chi (\varepsilon x))g_{\varepsilon }(x,t), \end{aligned}$$

then for \((x,t)\in \mathbb {R}^{3}\times \mathbb {R}^{+}\) and \(G_{\varepsilon }(x,t)=\int _{0}^{t}g_{\varepsilon }(x,s)s{\text{ d }}s\)

$$\begin{aligned} F_{\varepsilon }(x,t)&=\int _{0}^{t}f_{\varepsilon }(x,s)s{\text{ d }}s\\&=\chi (\varepsilon x)\left( \frac{1}{p}K(\varepsilon x)t^{p}+\frac{1}{3}Q(\varepsilon x)\left( m_{\varepsilon }(t^{2})\right) ^{\frac{3-p}{2}}\right) +\left( 1-\chi (\varepsilon x)\right) G_{\varepsilon }(x,t). \end{aligned}$$

We denote the sets \(\mathcal {V}_{\pm }:=\{x\in \mathbb {R}^{3}:V(x)=\pm a\}\) and \(\mathcal {V}:=\mathcal {V}_{+}\cup \mathcal {V}_{-}\). By (V), we can choose \(l_{0}\) large enough, such that \((\mathcal {V})^{2\delta }\subset B(0,l_{0/2})\). Setting \(\chi _{+}\) and \(\chi _{-}\) be the characteristic function of the sets

$$\begin{aligned} \mathcal {B}_{+}:=(\mathcal {V}_{+})^{\delta }\cup \left\{ |x|\ge l_{0}:V(x)\ge \frac{3a}{4}\right\} ,~\mathcal {B}_{-}:=(\mathcal {V}_{-})^{\delta }\cup \left\{ |x|\ge l_{0}:V(x)\le -\frac{3a}{4} \right\} . \end{aligned}$$

Without loss of generality, assume that \(\delta \) is small enough, there exists a \(\theta \in (0,1)\) satisfying

$$\begin{aligned} \pm V(x)\ge \frac{3a}{4}~\text { for }x\in \mathcal {B}_{\pm }~~\text {and }V(x)\in [-\theta a,\theta a]~\text {for }x\notin \mathcal {B}=\mathcal {B}_{+}\cup \mathcal {B}_{-}. \end{aligned}$$

For \(\phi (x)=\frac{\kappa }{1+|x|^{4+\tau '}}\), we define \(\xi ,\widehat{\xi }:\mathbb {R}^{3}\times \mathbb {R}\rightarrow \mathbb {R}\) by

$$\begin{aligned} \xi (x,t)= \left\{ \begin{array}{ll} 0, &{} t\le \phi (x); \\ \frac{1}{\phi (x)}(t-\phi (x))^{2}, &{} \phi (x)<t<2\phi (x);\\ 2t-3\phi (x),&{} t\ge 2\phi (x), \end{array} \right. ~\widehat{\xi }(x,t)=\int _{-\infty }^{t}\xi (x,s){\text{ d }}s, \end{aligned}$$

and define the penalized functional \(P_{\varepsilon }:E\rightarrow \mathbb {R}\) by

$$\begin{aligned} P_{\varepsilon }(w)=\frac{a}{8}\int _{\mathbb {R}^{3}}\widetilde{\chi }(\varepsilon x)\widehat{\xi }(x,|w|){\text{ d }}x, \end{aligned}$$

where \(\widetilde{\chi }(x)=\chi _{+}(x)-\chi _{-}(x)\). It is clear that \(P_{\varepsilon }:E\rightarrow \mathbb {R}\) is of class \(C^{1}\) and

$$\begin{aligned} P'_{\varepsilon }(w)v=\frac{a}{8}\text {Re}\int _{\mathbb {R}^{3}}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w|)w\cdot v{\text{ d }}x ~~\text {for any } v\in C_{0}^{\infty }(\mathbb {R}^{3},\mathbb {C}^{4}), \end{aligned}$$
(2.3)

where \(\widetilde{\xi }(x,t)\in C^{1}(\mathbb {R}^{3}\times \mathbb {R},[0,2])\),

$$\begin{aligned} \widetilde{\xi }(x,t):=\frac{1}{t}\xi (x,t) \left\{ \begin{array}{ll} 0, &{} t\le \phi (x); \\ \frac{(t-\phi (x))^{2}}{t\phi (x)}, &{} \phi (x)<t<2\phi (x);\\ \frac{2t-3\phi (x)}{t},&{} t\ge 2\phi (x). \end{array} \right. \end{aligned}$$

Moreover, for \(w_{n}\rightharpoonup w\) weakly in E, there holds

$$\begin{aligned} P'_{\varepsilon }(w_{n})v\rightarrow P'_{\varepsilon }(w)v ~~\text {for any } v\in C_{0}^{\infty }(\mathbb {R}^{3},\mathbb {C}^{4}). \end{aligned}$$

Now we define the modified functional \(\Phi _{\varepsilon }:E\rightarrow \mathbb {R}\)

$$\begin{aligned} \Phi _{\varepsilon }(w)&=\frac{1}{2}\int _{\mathbb {R}^{3}}\left( -i\alpha \cdot \nabla +a\beta \right) w\cdot w{\text{ d }}x+\frac{1}{2}\int _{\mathbb {R}^{3}}\left( V(\varepsilon x)w,w \right) {\text{ d }}x \\&\quad -P_{\varepsilon }(w)-\int _{\mathbb {R}^{3}}F_{\varepsilon }(x,|w|){\text{ d }}x\\&=\frac{1}{2}\left( \Vert w^{+}\Vert ^{2}-\Vert w^{-}\Vert ^{2}\right) +\frac{1}{2}\int _{\mathbb {R}^{3}}\left( V(\varepsilon x)w,w \right) {\text{ d }}x-P_{\varepsilon }(w)-\int _{\mathbb {R}^{3}}F_{\varepsilon }(x,|w|){\text{ d }}x. \end{aligned}$$

By (V), (K), (Q) and (2.3), we know that \(\Phi _{\varepsilon }\) is of class \(C^{1}\), and for \(w,v\in E\), there holds

$$\begin{aligned} \Phi _{\varepsilon }'(w)v=\text {Re}\int _{\mathbb {R}^{3}}(H_{0}w+V(\varepsilon x)w-\frac{a}{8}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w|)w-f_{\varepsilon }(x,|w|)w)\cdot v {\text{ d }}x, \end{aligned}$$

and the critical points correspond to weak solutions of

$$\begin{aligned} -i\alpha \cdot \nabla w+a\beta w +V(\varepsilon x)w-\frac{a}{8}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w|)w=f_{\varepsilon }(x,|w|)w. \end{aligned}$$

Lemma 2.1

For small \(\varepsilon _{0}>0\) and \(\varepsilon \in (0,\varepsilon _{0})\), the energy functional \(\Phi _{\varepsilon }\) satisfies the Palais-Smale condition.

Proof

Assuming \(\{w_{n}\}\subset E\) is a Palais-Smale sequence for \(\Phi _{\varepsilon }\), i.e., \(\{\Phi _{\varepsilon }(w_{n})\}\subset \mathbb {R}\) is bounded and \(\Phi '_{\varepsilon }(w_{n})\rightarrow 0\) in \(E^{*}\), we shall show that \(\{w_{n}\}\) has a convergent subsequence in E. We first verify the bounded-ness of \(\{w_{n}\}\) in E. Observing that

$$\begin{aligned} o_{n}(1)\Vert w_{n}\Vert&=\Phi '_{\varepsilon }(w_{n})(w_{n}^{+}-w_{n}^{-}) \nonumber \\&=\,\Vert w_{n}\Vert ^{2}+\text {Re}\int _{\mathbb {R}^{3}}V(\varepsilon x)w_{n}\cdot (w_{n}^{+}-w_{n}^{-}){\text{ d }}x\nonumber \\&\quad -\text {Re}\int _{\mathbb {R}^{3}}f_{\varepsilon }(x,|w_{n}|)w_{n}\cdot (w_{n}^{+}-w_{n}^{-}){\text{ d }}x \nonumber \\&\quad -\frac{a}{8}\text {Re}\int _{\mathbb {R}^{3}}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w_{n}|)w_{n}\cdot (w_{n}^{+}-w_{n}^{-}){\text{ d }}x \nonumber \\&=\,\Vert w_{n}\Vert ^{2}+\int _{\mathbb {R}^{3}}(V(\varepsilon )-\frac{a}{8}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w_{n}|))(|w_{n}^{+}|^{2}-|w_{n}^{-}|^{2}){\text{ d }}x \nonumber \\&\quad -\text {Re}\int _{\mathbb {R}^{3}}f_{\varepsilon }(x,|w_{n}|)w_{n} \cdot (w_{n}^{+}-w_{n}^{-}){\text{ d }}x. \end{aligned}$$
(2.4)

By the similar argument as [32, Lemma 2.2], we get

$$\begin{aligned} \int _{\mathbb {R}^{3}}&\left( V(\varepsilon x)-\frac{a}{8}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w_{n}|)\right) \left( |w_{n}^{+}|^{2}-|w_{n}^{-}|^{2}\right) {\text{ d }}x \nonumber \\&\ge -\max \{\theta ,\frac{7}{8}\}\Vert w_{n}\Vert ^{2}-C\varepsilon ^{2\tau '+5}. \end{aligned}$$
(2.5)

On the other hand, it may be assumed that \(\Phi _{\varepsilon }(w_{n})\rightarrow c\), then

$$\begin{aligned} c+\Vert w_{n}\Vert&\ge \Phi _{\varepsilon }(w_{n})-\frac{1}{2}\Phi '_{\varepsilon }(w_{n})w_{n} \nonumber \\&=\int _{\mathbb {R}^{3}}\left( \frac{1}{2}f_{\varepsilon }(x,|w_{n}|)|w_{n}|^{2}-F_{\varepsilon }(x,|w_{n}|)\right) {\text{ d }}x+ \frac{1}{2}P'_{\varepsilon }(w_{n})w_{n}-P_{\varepsilon }(w_{n}). \end{aligned}$$
(2.6)

By the definition of \(P_{\varepsilon }\), (2.2) and the fact \(\Vert \chi _{-}(\varepsilon x)\phi \Vert _{2}\le C\varepsilon ^{\tau '+5/2}\), we deduce

$$\begin{aligned} \frac{1}{2}P'_{\varepsilon }(w_{n})w_{n}-P_{\varepsilon }(w_{n})&\ge -\frac{3a}{2}\int _{\mathbb {R}^{3}}\chi _{-}(\varepsilon x)|w_{n}|\phi {\text{ d }}x \nonumber \\&\ge -C\Vert \chi _{-}(\varepsilon x)\phi \Vert _{2}\Vert w_{n}\Vert _{2}\ge -C\varepsilon ^{\tau '+5/2}\Vert w_{n}\Vert . \end{aligned}$$
(2.7)

If \(h_{\varepsilon }(x,t)\ge \phi (x)\), then by the definition of \(g_{\varepsilon }(x,t)\) and \(G_{\varepsilon }(x,t)\), we have

$$\begin{aligned} G_{\varepsilon }(x,t)=\frac{1}{2}\phi (x)t^{2}-\frac{1}{2}\phi (x)t_{0}^{2}+H_{\varepsilon }(x,t_{0}),~~h_{\varepsilon }(x,t_{0}) =\phi (x), \end{aligned}$$

where \(H_{\varepsilon }(x,t_{0})=\int _{0}^{t_{0}}h_{\varepsilon }(x,s)s{\text{ d }}s=\frac{1}{p}K(\varepsilon x)t_{0}^{p}+\frac{1}{3}Q(\varepsilon x)t_{0}^{p}(m_{\varepsilon }(t_{0}^{2}))^{\frac{3-p}{2}}\). So there holds

$$\begin{aligned} \left| \frac{1}{2}g_{\varepsilon }(x,t_{0})t_{0}^{2}-G_{\varepsilon }(x,t_{0})\right| \le \left| \frac{1}{2}\phi (x)t_{0}^{2}-H_{\varepsilon }(x,t_{0})\right| \end{aligned}$$
(2.8)

Since \(h_{\varepsilon }(x,t_{0})=\phi (x)\), i.e.,

$$\begin{aligned}&K(\varepsilon x)t_{0}^{p-2}+\frac{p}{3}Q(\varepsilon x)t_{0}^{p-2}\left( m_{\varepsilon }(t_{0}^{2})\right) ^{\frac{3-p}{2}}\\&\quad + \frac{3-p}{3}Q(\varepsilon x)t_{0}^{p}\left( m_{\varepsilon }(t_{0}^{2})\right) ^{\frac{3-p}{2}-1}b_{\varepsilon }(t_{0}^{2})=\phi (x). \end{aligned}$$

If \(t_{0}\gg 1\), then we have

$$\begin{aligned} K(\varepsilon x)t_{0}^{p-2}\le K(\varepsilon x)t_{0}^{p-2}+\frac{p}{3}Q(\varepsilon x)t_{0}^{p-2}\left( m_{\varepsilon }(t_{0}^{2})\right) ^{\frac{3-p}{2}}=\phi (x). \end{aligned}$$

From above, we know that \(t_{0}\) has an upper bound, i.e., there exists a constant \(M>0\), such that \(t_{0}\le M\). Similarly, if \(t_{0}\ll 1\), then there holds \(K(\varepsilon x)t_{0}^{p-2}+Q(\varepsilon x)t_{0}^{p}=\phi (x)\). It follows that

$$\begin{aligned} \left| \frac{1}{2}g_{\varepsilon }(x,t_{0})t_{0}^{2}-G_{\varepsilon }(x,t_{0})\right| \le \frac{1}{2}\left| K(\varepsilon x)\right| ^{-\frac{2}{p-2}}\left| \phi (x)\right| ^{\frac{p}{p-2}}. \end{aligned}$$

Then

$$\begin{aligned}&\left| \int _{\mathbb {R}^{3}}\left( 1-\chi (\varepsilon x)\right) \left( \frac{1}{2}g_{\varepsilon }(x,|w_{n}|)|w_{n}|^{2}-G_{\varepsilon }(x,|w_{n}|){\text{ d }}x \right) \right| \nonumber \\&\quad \le C\int _{\mathbb {R}^{3}\backslash (\Lambda ^{\delta })_{\varepsilon }}|K(\varepsilon x)|^{{-\frac{2}{p-2}}}|\phi (x)|^{\frac{p}{p-2}}\le C\varepsilon ^{\frac{p(\tau '+4)}{p-2}-3}. \end{aligned}$$
(2.9)

Injecting (2.7), (2.8) and (2.9) into (2.6), we have

$$\begin{aligned} C(1+\Vert w_{n}\Vert )&\ge \int _{\mathbb {R}^{3}}\left( \frac{1}{2}f_{\varepsilon }(x,|w_{n}|)|w_{n}|^{2}-F_{\varepsilon }(x,|w_{n}|)\right) {\text{ d }}x+ \frac{1}{2}P'_{\varepsilon }(w_{n})w_{n}-P_{\varepsilon }(w_{n}){\text{ d }}x\nonumber \\&\ge \int _{\mathbb {R}^{3}}\left( \frac{1}{2}\chi (\varepsilon x)h_{\varepsilon }(x,|w_{n}|)|w_{n}|^{2}-\chi (\varepsilon x)H_{\varepsilon }(x,|w_{n}|)\right) {\text{ d }}x-C\varepsilon ^{\tau '+5/2}\Vert w_{n}\Vert \nonumber \\&=\int _{\mathbb {R}^{3}}\chi (\varepsilon x)\left( \frac{1}{2}h_{\varepsilon }(x,|w_{n}|)|w_{n}|^{2}-H_{\varepsilon }(x,|w_{n}|)\right) {\text{ d }}x-C\varepsilon ^{\tau '+5/2}\Vert w_{n}\Vert , \end{aligned}$$
(2.10)

where \(H_{\varepsilon }(x,|w_{n}|)=\int _{0}^{|w_{n}|}h_{\varepsilon }(x,s)s{\text{ d }}s\). By Hölder inequality and (2.10), we have

$$\begin{aligned}&\left| \int _{\mathbb {R}^{3}}\chi (\varepsilon x)h_{\varepsilon }(x,|w_{n}|)w_{n}\cdot (w_{n}^{+}-w_{n}^{-}){\text{ d }}x\right| \\&\quad \le \left| \int _{\mathbb {R}^{3}}K(\varepsilon x)\chi (\varepsilon x)|w_{n}|^{p-2}w_{n}\cdot (w_{n}^{+}-w_{n}^{-}){\text{ d }}x\right| \\&\qquad +\left| \int _{\mathbb {R}^{3}}\chi (\varepsilon x)\widetilde{h}_{\varepsilon }(x,|w_{n}|)w_{n}\cdot (w_{n}^{+}-w_{n}^{-}){\text{ d }}x\right| \\&\quad \le \Vert K^{1/p}|w_{n}|\Vert _{L^{p}((\Lambda ^{\delta })_{\varepsilon })}^{p-1} \Vert K^{1/p}|w_{n}^{+}-w_{n}^{-}|\Vert _{L^{p}((\Lambda ^{\delta })_{\varepsilon })}\\&\qquad + \left| \int _{\mathbb {R}^{3}}\chi (\varepsilon x)\widetilde{h}_{\varepsilon }(x,|w_{n}|)w_{n}\cdot (w_{n}^{+}-w_{n}^{-}){\text{ d }}x\right| \\&\quad \le C\left( 1+\Vert w_{n}\Vert \right) ^{\frac{p-1}{p}}\varepsilon ^{\frac{p-3}{p}}\Vert K\Vert _{L^{\frac{3-p}{3}} (\Lambda ^{\delta })}^{\frac{1}{p}}\Vert w_{n}^{+}-w_{n}^{-}\Vert \\&\qquad +\left| \int _{\mathbb {R}^{3}}\chi (\varepsilon x)\widetilde{h}_{\varepsilon }(x,|w_{n}|)w_{n}\cdot (w_{n}^{+}-w_{n}^{-}){\text{ d }}x\right| \\&\quad \le C\varepsilon ^{\frac{p-3}{p}}\left( \Vert w_{n}\Vert +\Vert w_{n}\Vert ^{\frac{2p-1}{p}}\right) +\left( \int _{\mathbb {R}^{3}}\chi (\varepsilon x)(\widetilde{h}_{\varepsilon }(x,|w_{n}|)|w_{n}|)^{\frac{3}{2}}{\text{ d }}x\right) ^{\frac{2}{3}} \\&\qquad \times \left( \int _{\mathbb {R}^{3}}|(w_{n}^{+}-w_{n}^{-})|^{3}{\text{ d }}x\right) ^{\frac{1}{3}}, \end{aligned}$$

where \(\widetilde{h}_{\varepsilon }(x,t)=\frac{p}{3}Q(\varepsilon x)t^{p-2}\left( m_{\varepsilon }(t^{2})\right) ^{\frac{3-p}{2}}+ \frac{3-p}{3}Q(\varepsilon x)t^{p}\left( m_{\varepsilon }(t^{2})\right) ^{\frac{3-p}{2}-1}b_{\varepsilon }(t^{2})\). Then (2.10) can be rewrite

$$\begin{aligned} C(1+\Vert w_{n}\Vert )&\ge -C\varepsilon ^{\tau '+5/2}\Vert w_{n}\Vert +\int _{\mathbb {R}^{3}}\chi (\varepsilon x)\left( \frac{1}{2}\widetilde{h}_{\varepsilon }(x,|w_{n}|)|w_{n}|^{2}-\widetilde{H}_{\varepsilon }(x,|w_{n}|)\right) {\text{ d }}x \nonumber \\&\ge -C\varepsilon ^{\tau '+5/2}\Vert w_{n}\Vert +c\int _{\mathbb {R}^{3}}\chi (\varepsilon x)\left( \widetilde{h}_{\varepsilon }(x,|w_{n}|)|w_{n}| \right) ^{\frac{3}{2}}{\text{ d }}x. \end{aligned}$$
(2.11)

By the definition of \(g_{\varepsilon }(x,t)\), we have

$$\begin{aligned} \left| \int _{\mathbb {R}^{3}}\left( 1-\chi (\varepsilon x)\right) g_{\varepsilon }(x,|w_{n}|)w_{n}\cdot (w_{n}^{+}-w_{n}^{-}){\text{ d }}x \right| \le C\varepsilon ^{\tau '+3}\Vert w_{n}\Vert ^{2}. \end{aligned}$$
(2.12)

Combining (2.4), (2.5), (2.11) and (2.12), we have

$$\begin{aligned} \min \left\{ 1-\theta ,\frac{1}{8}\right\} \Vert w_{n}\Vert ^{2}-C\varepsilon ^{\tau '+3}\Vert w_{n}\Vert ^{2}&\le C\varepsilon ^{\frac{p-3}{p}}\left( \Vert w_{n}\Vert +\Vert w_{n}\Vert ^{\frac{2p-1}{p}}\right) \\&\quad +C(1+\Vert w_{n}\Vert )^{\frac{2}{3}}\Vert w_{n}\Vert . \end{aligned}$$

This implies the bounded-ness of \(\{w_{n}\}\) in E for small \(\varepsilon _{0}\) and \(\varepsilon \in (0,\varepsilon _{0})\).

Next we prove \(w_{n}\rightarrow w\) in E as \(n\rightarrow \infty \), denoting \(z_{n}=w_{n}-w\), we have

$$\begin{aligned} \Phi '_{\varepsilon }(w_{n})(z_{n}^{+}-z_{n}^{-})=o_{n}(1),~~~\Phi '_{\varepsilon }(w)(z_{n}^{+}-z_{n}^{-})=o_{n}(1). \end{aligned}$$

It follows that

$$\begin{aligned} o_{n}(1)=&\,\text {Re}( w_{n}^{+},z_{n}^{+})+\text {Re}( w_{n}^{-},z_{n}^{-})+\text {Re}\int _{\mathbb {R}^{3}}V(\varepsilon x)w_{n}\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x\\&-\text {Re}\int _{\mathbb {R}^{3}}\frac{a}{8}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w_{n}|)w_{n}\cdot (z_{n}^{+}-z_{n}^{-})+f_{\varepsilon }(x,|w_{n}|)w_{n}\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x; \end{aligned}$$

and

$$\begin{aligned} 0 =&\,\text {Re}( w^{+},z_{n}^{+})+\text {Re}( w^{-},z_{n}^{-})+\text {Re}\int _{\mathbb {R}^{3}}V(\varepsilon x)w\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x\\&-\text {Re}\int _{\mathbb {R}^{3}}\frac{a}{8}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w|)w\cdot (z_{n}^{+}-z_{n}^{-})+f_{\varepsilon }(x,|w|)w\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x. \end{aligned}$$

Then there holds

$$\begin{aligned} o_{n}(1)=&\,\Phi '_{\varepsilon }(w_{n})(z_{n}^{+}-z_{n}^{-})-\Phi '_{\varepsilon }(w)(z_{n}^{+}-z_{n}^{-}) =\Vert z_{n}\Vert ^{2}\nonumber \\&+\text {Re}\int _{\mathbb {R}^{3}}V(\varepsilon x)z_{n}\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x \nonumber \\&-\text {Re}\int _{\mathbb {R}^{3}}\frac{a}{8}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w_{n}|)w_{n}\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x \nonumber \\&+\text {Re}\int _{\mathbb {R}^{3}}\frac{a}{8}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w|)w\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x \nonumber \\&-\text {Re}\int _{\mathbb {R}^{3}}\chi (\varepsilon x)(h_{\varepsilon }(x,|w_{n}|)w_{n}-h_{\varepsilon }(x,|w|)w)\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x \nonumber \\&-\text {Re}\int _{\mathbb {R}^{3}}(1-\chi (\varepsilon x))(g_{\varepsilon }(x,|w_{n}|)w_{n}-g_{\varepsilon }(x,|w|)w)\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x. \end{aligned}$$
(2.13)

By the definition of \(\widetilde{\chi }\) and \(\widetilde{\xi }(x,t)\), it follows that

$$\begin{aligned}&\lim _{n\rightarrow \infty }\text {Re}\int _{\mathbb {R}^{3}}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w_{n}|)w\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x \\&\quad =\lim _{n\rightarrow \infty } \text {Re}\int _{\mathbb {R}^{3}}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w|)w\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x=0. \end{aligned}$$

Moreover, we have

$$\begin{aligned} \left( g_{\varepsilon }(x,|w_{n}|)-g_{\varepsilon }(x,|w|)\right) \cdot (z_{n}^{+}-z_{n}^{-})\rightharpoonup 0~~\text {in }L^{2}(\mathbb {R}^{3},\mathbb {C}^{4}), \end{aligned}$$

which leads to

$$\begin{aligned} \lim _{n\rightarrow \infty }\left| \int _{\mathbb {R}^{3}}\left( 1-\chi (\varepsilon x)\left( g_{\varepsilon }(x,|w_{n}|)-g_{\varepsilon }(x,|w|)\right) w\cdot (z_{n}^{+}-z_{n}^{-})\right) {\text{ d }}x\right| =0. \end{aligned}$$

Hence, (2.13) can be rewritten as follows

$$\begin{aligned} o_{n}(1)=&\Vert z_{n}\Vert ^{2}+\text {Re}\int _{\mathbb {R}^{3}}\left[ V(\varepsilon x)-\frac{a}{8}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w_{n}|)\right] z_{n}\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x \nonumber \\&-\text {Re}\int _{\mathbb {R}^{3}}\chi (\varepsilon x)\left( h_{\varepsilon }(x,|w_{n}|)w_{n}-h_{\varepsilon }(x,|w|)w\right) \cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x. \end{aligned}$$
(2.14)

By [34], we know that

$$\begin{aligned}&\int _{\mathbb {R}^{3}}V(\varepsilon x)|z_{n}^{+}|^{2}-\frac{a}{8}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w_{n}|)|z_{n}^{+}|^{2}{\text{ d }}x\\&\quad \ge \int _{\mathbb {R}^{3}}-\theta a(1-\chi (\varepsilon x))|z_{n}^{+}|^{2}{\text{ d }}x-\frac{7a}{8}\int _{|w_{n}|\ge 3\phi (x)}\chi _{-}(\varepsilon x)|z_{n}^{+}|^{2}{\text{ d }}x\\&\qquad +\frac{a}{2}\int _{\mathbb {R}^{3}}\chi _{+}(\varepsilon x)|z_{n}^{+}|^{2}{\text{ d }}x-a\int _{\mathbb {R}^{3}\backslash (|w_{n}|\ge 3\phi (x))}\chi _{-}(\varepsilon x)|z_{n}^{+}|^{2}{\text{ d }}x+o_{n}(1) \end{aligned}$$

and

$$\begin{aligned}&\int _{\mathbb {R}^{3}}-V(\varepsilon x)|z_{n}^{-}|^{2}+\frac{a}{8}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w_{n}|)|z_{n}^{-}|^{2}{\text{ d }}x\\&\quad \ge \int _{\mathbb {R}^{3}}-\theta a(1-\chi (\varepsilon x))|z_{n}^{-}|^{2}{\text{ d }}x-\frac{7a}{8}\int _{|w_{n}|\ge 3\phi (x)}\chi _{+}(\varepsilon x)|z_{n}^{-}|^{2}{\text{ d }}x\\&\qquad +\frac{a}{2}\int _{\mathbb {R}^{3}}\chi _{-}(\varepsilon x)|z_{n}^{-}|^{2}{\text{ d }}x. \end{aligned}$$

Combining the above two inequalities and (2.14), we obtain

$$\begin{aligned}&\min \left\{ 1-\theta ,\frac{1}{8}\right\} \Vert z_{n}\Vert ^{2}-\text {Re}\int _{\mathbb {R}^{3}}\chi (\varepsilon x)(h_{\varepsilon }(x,|w_{n}|)w_{n}-h_{\varepsilon }(x,|w|)w)\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x\le 0. \end{aligned}$$
(2.15)

By mean value theorem, there exists a function \(\theta _{n}\) such that

$$\begin{aligned}&\left| \int _{\mathbb {R}^{3}}\chi (\varepsilon x)\left( K(\varepsilon x)|w_{n}|^{p-2}w_{n}-K(\varepsilon x)|w|^{p-2}w\right) \cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x\right| \nonumber \\&\quad \le (p-1)\left| \int _{\mathbb {R}^{3}}\chi (\varepsilon x)(K(\varepsilon x)|\theta _{n}|^{p-2}z_{n}\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x\right| \nonumber \\&\quad \le (p-1)\int _{\mathbb {R}^{3}}\chi (\varepsilon x)(K(\varepsilon x)|\theta _{n}|^{p-2}|z_{n}|\cdot |z_{n}^{+}-z_{n}^{-}|{\text{ d }}x \nonumber \\&\quad \le (p-1)\left( \int _{\mathbb {R}^{3}}\chi (\varepsilon x)(K(\varepsilon x)|\theta _{n}|^{p-2})^{\frac{p}{p-2}}{\text{ d }}x\right) ^{\frac{p-2}{p}} \nonumber \\&\qquad \times \left( \int _{\mathbb {R}^{3}}|z_{n}|^{p}{\text{ d }}x\right) ^{\frac{1}{p}} \left( \int _{\mathbb {R}^{3}}|z_{n}^{+}-z_{n}^{-}|^{p}{\text{ d }}x\right) ^{\frac{1}{p}} \nonumber \\&\quad \le (p-1)\left\{ \left( \int _{\mathbb {R}^{3}}(\chi (\varepsilon x)|K(\varepsilon x)|^{\frac{p}{p-2}})^{\frac{3}{3-p}}{\text{ d }}x\right) ^{\frac{3-p}{3}}\cdot |\theta _{n}|_{3}^{p}\right\} ^{\frac{p-2}{p}} \cdot |z_{n}|_{p}\cdot |z_{n}^{+}-z_{n}^{-}|_{p} \nonumber \\&\quad =o_{n}(1). \end{aligned}$$
(2.16)

Similarly, we have

(2.17)

and

$$\begin{aligned}&\Big |\int _{\mathbb {R}^{3}}\chi (\varepsilon x)Q(\varepsilon x)|w_{n}|^{p-2}\left( m_{\varepsilon }(|w_{n}|^{2})\right) ^{\frac{3-p}{2}-1}b_{\varepsilon }(|w_{n}|^{2})w_{n}\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x \nonumber \\&-\int _{\mathbb {R}^{3}}\chi (\varepsilon x)Q(\varepsilon x)|w|^{p-2}\left( m_{\varepsilon }(|w|^{2})\right) ^{\frac{3-p}{2}-1}b_{\varepsilon }(|w|^{2})w\cdot (z_{n}^{+}-z_{n}^{-}){\text{ d }}x\Big | =o_{n}(1). \end{aligned}$$
(2.18)

Taking (2.16), (2.17) and (2.18) into (2.15), and we can obtain

$$\begin{aligned} \min \{1-\theta ,\frac{1}{8}\}\Vert z_{n}\Vert ^{2}\le o_{n}(1). \end{aligned}$$

Therefore, \(\{w_{n}\}\) has a convergent subsequence in E, and the proof is completed. \(\square \)

3 The Solutions of Modified Equation

In this section, we will use an abstract linking theorem [12] to obtain nontrivial critical points for the modified variational functional. Let’s write the modified equation as follows

$$\begin{aligned} -i\alpha \cdot \nabla w+a\beta w +V(\varepsilon x)w-\frac{a}{8}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w|)w=f_{\varepsilon }(x,|w|)w. \end{aligned}$$
(3.1)

For the convenience, we give the following notations.

$$\begin{aligned}&B_{r}=\{w\in E:\Vert w\Vert \le r\},~~S_{r}=\{w\in E:\Vert w\Vert = r\};\\&E(e)=\{w\in E:w=se+v,~s\ge 0\text { and }v\in E^{-}\}. \end{aligned}$$

In order to obtain the linking structure of the modified functional, we first give the following lemma.

Lemma 3.1

([34, Lemma 3.1.]) Assume that (V) holds. Then there exists a constant \(C>0\) which independent of \(\varepsilon \), such that for any \(w\in E\),

$$\begin{aligned} \left| \int _{\mathbb {R}^{3}}V(\varepsilon x)|w|^{2}-\frac{a}{4}\widetilde{\chi }(\varepsilon x)\widehat{\xi }(x,|w|){\text{ d }}x\right| \le \max \left\{ \theta ,\frac{7}{8}\right\} \Vert w\Vert ^{2}+C\varepsilon ^{2\tau '+5}. \end{aligned}$$

Lemma 3.2

Assume that (V), (K) and (Q) hold, then there exist constants \(r_{0}>0\) and \(\rho >0\), such that

$$\begin{aligned} \inf _{w\in E^{+},\Vert w\Vert =r_{0}}\Phi _{\varepsilon }(w)\ge \rho , ~~for~any~~\varepsilon \in (0,\varepsilon _{0}). \end{aligned}$$

Proof

Taking \(w\in E^{+}\), by Lemma 3.1, there holds

$$\begin{aligned} \Phi _{\varepsilon }(w)&=\frac{1}{2}\Vert w\Vert ^{2}+\frac{1}{2}\int _{\mathbb {R}^{3}}V(\varepsilon x)|w|^{2}{\text{ d }}x-P_{\varepsilon }(w)-\int _{\mathbb {R}^{3}}F_{\varepsilon }(x,|w|){\text{ d }}x\\&=\frac{1}{2}\Vert w\Vert ^{2}+\frac{1}{2}\int _{\mathbb {R}^{3}}V(\varepsilon x)|w|^{2}-\frac{a}{4}\widetilde{\chi }(\varepsilon x)\widehat{\xi }(x,|w|){\text{ d }}x-\int _{\mathbb {R}^{3}}F_{\varepsilon }(x,|w|){\text{ d }}x\\&\ge \frac{1}{2}\min \left\{ 1-\theta ,\frac{1}{8}\right\} \Vert w\Vert ^{2}-C\varepsilon ^{2\tau '+5}-\int _{\mathbb {R}^{3}}F_{\varepsilon }(x,|w|){\text{ d }}x. \end{aligned}$$

By the definition of \(F_{\varepsilon }(x,t)\), we have

$$\begin{aligned} \int _{\mathbb {R}^{3}}F_{\varepsilon }(x,|w|){\text{ d }}x&\le \int _{(\Lambda ^{\delta })_{\varepsilon }}\frac{1}{p}|K(\varepsilon x)||w|^{p}+\frac{1}{3}Q(\varepsilon x)|w|^{p}\left( m_{\varepsilon }(|w|^{2})\right) ^{\frac{3-p}{2}}{\text{ d }}x\nonumber \\&\qquad +\int _{\mathbb {R}^{3}\backslash (\Lambda ^{\delta })_{\varepsilon }}G_{\varepsilon }(x,|w|){\text{ d }}x \nonumber \\&\quad \le \frac{1}{p}\left( \int _{(\Lambda ^{\delta })_{\varepsilon }}|K(\varepsilon x)|^{\frac{3}{3-p}}{\text{ d }}x\right) ^{\frac{3-p}{3}}\cdot \left( \int _{(\Lambda ^{\delta })_{\varepsilon }}|w|^{3}{\text{ d }}x\right) ^{\frac{p}{3}}\nonumber \\&\qquad +\int _{(\Lambda ^{\delta })_{\varepsilon }}|w|^{3}{\text{ d }}x +\frac{1}{2}\int _{\mathbb {R}^{3}\backslash (\Lambda ^{\delta })_{\varepsilon }}\phi |w|^{2}{\text{ d }}x \nonumber \\&\quad \le C\varepsilon ^{p-3}\Vert w\Vert ^{p}+\Vert w\Vert ^{3}+C\varepsilon ^{\tau '+4}\Vert w\Vert ^{2}. \end{aligned}$$

Therefore, by the above two estimates, we have

$$\begin{aligned} \Phi _{\varepsilon }(w)&\ge \frac{1}{2}\min \left\{ 1-\theta ,\frac{1}{8}\right\} \Vert w\Vert ^{2}-C\varepsilon ^{2\tau '+5} -C\varepsilon ^{p-3}\Vert w\Vert ^{p}-\Vert w\Vert ^{3}-C\varepsilon ^{\tau '+4}\Vert w\Vert ^{2}\\&\ge \frac{1}{4}\min \left\{ 1-\theta ,\frac{1}{8}\right\} \Vert w\Vert ^{2}-C\varepsilon ^{2\tau '+5}-C\varepsilon ^{p-3}\Vert w\Vert ^{p}-\Vert w\Vert ^{3}. \end{aligned}$$

Let \(\Vert w\Vert =\varepsilon <1\), in the light of \(p\in (5/2,3)\), then

$$\begin{aligned} \Phi _{\varepsilon }(w)&\ge \frac{1}{4}\min \left\{ 1-\theta ,\frac{1}{8}\right\} \varepsilon ^{2} -C\varepsilon ^{2\tau '+5}-C\varepsilon ^{2p-3}-\varepsilon ^{3}\\&\ge \frac{1}{4}\min \left\{ 1-\theta ,\frac{1}{8}\right\} \varepsilon ^{2}-C'\varepsilon ^{2p-3}. \end{aligned}$$

We complete the proof of this lemma. \(\square \)

Lemma 3.3

Assume that (V), (K) and (Q) hold. Fix \(e_{0}\in E^{+}\), then there exist \(\varepsilon _{0}>0\) and \(R_{0}>0\), such that for any \(\varepsilon \in (0,\varepsilon _{0})\), there holds

$$\begin{aligned} \sup _{w\in E(e_{0}),\Vert w\Vert \ge R_{0}}\Phi _{\varepsilon }(w)\le 0. \end{aligned}$$

Moreover, \(\sup _{w\in E(e_{0})}\Phi _{\varepsilon }(w)\le 2R_{0}^{2}\).

Proof

Taking \(w\in E(e_{0})\), denote \(w=se_{0}+v\) with \(s\ge 0,~v\in E^{-}\) , we deduce

$$\begin{aligned} \Phi _{\varepsilon }(w)=&\,\frac{s^{2}}{2}\Vert e_{0}\Vert ^{2}-\frac{1}{2}\Vert v\Vert ^{2}+\frac{1}{2}\int _{\mathbb {R}^{3}}V(\varepsilon x)|se_{0}+v|^{2}{\text{ d }}x \nonumber \\&-\int _{\mathbb {R}^{3}}\frac{a}{8}\widetilde{\chi }(\varepsilon x)\widehat{\xi }(x,|se_{0}+v|){\text{ d }}x-\int _{\mathbb {R}^{3}}F_{\varepsilon }(x,|se_{0}+v|){\text{ d }}x. \end{aligned}$$
(3.2)

Now we will discuss three cases:

Case 1: If \(s=0\) and \(v\ne 0\), then by (3.2) and Lemma 3.1, we have

$$\begin{aligned} \Phi _{\varepsilon }(w)&=-\frac{1}{2}\Vert v\Vert ^{2}+\frac{1}{2}\int _{\mathbb {R}^{3}}\left[ V(\varepsilon x)|v|^{2}{\text{ d }}x -\frac{a}{4}\widetilde{\chi }(\varepsilon x)\widehat{\xi }(x,|v|)\right] {\text{ d }}x-\int _{\mathbb {R}^{3}}F_{\varepsilon }(x,|v|){\text{ d }}x\\&\le -\frac{1}{2}\Vert v\Vert ^{2}+\frac{1}{2}\left( \max \left\{ \theta ,\frac{7}{8}\right\} \Vert v\Vert ^{2}+C\varepsilon ^{2\tau '+5}\right) -\int _{\mathbb {R}^{3}}F_{\varepsilon }(x,|v|){\text{ d }}x\\&\le -\frac{1}{2}\min \left\{ 1-\theta ,\frac{1}{8}\right\} \Vert v\Vert ^{2}+C\varepsilon ^{2\tau '+5}. \end{aligned}$$

It follows that \(\Phi _{\varepsilon }(w)\rightarrow -\infty \) as \(\Vert w\Vert =\Vert v\Vert \rightarrow \infty \).

Case 2: If \(w=se_{0}\ne 0\), then by (3.2) and Lemma 3.1, there holds

$$\begin{aligned} \Phi _{\varepsilon }(w)&=\frac{s^{2}}{2}\Vert e_{0}\Vert ^{2}+\frac{s^{2}}{2}\int _{\mathbb {R}^{3}}\left[ V(\varepsilon x)|e_{0}|^{2}{\text{ d }}x -\frac{a}{4}\widetilde{\chi }(\varepsilon x)\widehat{\xi }(x,|se_{0}|)\right] {\text{ d }}x-\int _{\mathbb {R}^{3}}F_{\varepsilon }(x,|se_{0}|){\text{ d }}x\\&\le \frac{s^{2}}{2}\Vert e_{0}\Vert ^{2}+\frac{s^{2}}{2}\left( \max \left\{ \theta ,\frac{7}{8}\right\} \Vert e_{0}\Vert ^{2}+C\varepsilon ^{2\tau '+5}\right) -\int _{\mathbb {R}^{3}}F_{\varepsilon }(x,|se_{0}|){\text{ d }}x\\&\le \frac{1}{2}\max \left\{ 1+\theta ,\frac{15}{8}\right\} \Vert se_{0}\Vert ^{2}+C\varepsilon ^{2\tau '+5} -\frac{1}{p}\int _{\mathbb {R}^{3}}\chi (\varepsilon x)K(\varepsilon x)|se_{0}|^{p}{\text{ d }}x\\&\le C_{1}\Vert e_{0}\Vert ^{2}s^{2}-C_{2}\Vert e_{0}\Vert ^{p}s^{p}+C\varepsilon ^{2\tau '+5}. \end{aligned}$$

Therefore, \(\Phi _{\varepsilon }(w)\rightarrow -\infty \) as \(s\rightarrow \infty \). Define \(\varrho _{1}:=\frac{1}{2}\max \{1+\theta ,\frac{15}{8}\}\), \(\varrho _{2}:=\frac{1}{2}\min \{1-\theta ,\frac{1}{8}\}\).

Case 3: If \(se_{0}\ne 0\) and \(v\ne 0\), then (3.2) and Lemma 3.1 leads to

$$\begin{aligned} \Phi _{\varepsilon }(w)&= \Phi _{\varepsilon }(se_{0}+v) \le \,\frac{1}{2}\Vert se_{0}\Vert ^{2}-\frac{1}{2}\Vert v\Vert +\frac{1}{2}\max \left\{ \theta ,\frac{7}{8}\right\} \Vert se_{0}+v\Vert ^{2} \nonumber \\&\qquad -\int _{\mathbb {R}^{3}}F_{\varepsilon }(x,|se_{0}+v|){\text{ d }}x+C\varepsilon ^{2\tau '+5} \nonumber \\&\le \,\frac{1}{2}\max \left\{ 1+\theta ,\frac{15}{8}\right\} \Vert se_{0}\Vert ^{2}-\frac{1}{2}\min \left\{ 1-\theta ,\frac{1}{8}\right\} \Vert v\Vert ^{2} \nonumber \\&\qquad -\int _{\mathbb {R}^{3}}F_{\varepsilon }(x,|se_{0}+v|){\text{ d }}x +C\varepsilon ^{2\tau '+5} \nonumber \\&\le \, \varrho _{1}\Vert se_{0}\Vert ^{2}-\varrho _{2}\Vert v\Vert ^{2}\nonumber \\&\qquad -\int _{\mathbb {R}^{3}}\chi (\varepsilon x)\left( \frac{1}{p}K(\varepsilon x)|se_{0}+v|^{p}+\frac{1}{3}Q(\varepsilon x)\left( m_{\varepsilon }(|se_{0}+v|^{2})\right) ^{\frac{3-p}{2}}\right) {\text{ d }}x \nonumber \\&\qquad -\int _{\mathbb {R}^{3}}(1-\chi (\varepsilon x))G_{\varepsilon }(x,|se_{0}+v|){\text{ d }}x+C\varepsilon ^{2\tau '+5} \nonumber \\ \le&\,\Vert w\Vert ^{2}\left( \varrho _{1}\Vert \frac{se_{0}}{\Vert w\Vert }\Vert ^{2}-\varrho _{2}\Vert \frac{v}{\Vert w\Vert }\Vert ^{2} -\frac{\Vert w\Vert ^{p-2}}{p}\int _{\mathbb {R}^{3}}\chi (\varepsilon x)K(\varepsilon x)\frac{|w|^{p}}{\Vert w\Vert ^{p}}{\text{ d }}x\right) \nonumber \\&\qquad +C\varepsilon ^{2\tau '+5}. \end{aligned}$$
(3.3)

If \(\Phi _{\varepsilon }(w)\rightarrow -\infty \) as \(\Vert w\Vert \rightarrow \infty \), we can get the conclusion. Otherwise there exist \(M>0\) and a sequence \(\{w_{n}\}\subset E(e_{0})\), such that \(\Phi _{\varepsilon }(w_{n})>-M\) as \(\Vert w_{n}\Vert \rightarrow \infty \). Hence, by (3.3) we can get

$$\begin{aligned} -\frac{M}{\Vert w_{n}\Vert ^{2}}&\le \varrho _{1}\cdot \Vert \frac{s_{n}e_{0}}{\Vert w_{n}\Vert }\Vert ^{2}-\varrho _{2}\cdot \Vert \frac{v_{n}}{\Vert w_{n}\Vert }\Vert ^{2} -\frac{\Vert w_{n}\Vert ^{p-2}}{p}\int _{\mathbb {R}^{3}}\chi (\varepsilon x)K(\varepsilon x)\nonumber \\&\quad \times \frac{|w_{n}|^{p}}{\Vert w_{n}\Vert ^{p}}{\text{ d }}x+o_{n}(1). \end{aligned}$$
(3.4)

Denote \(\frac{w_{n}}{\Vert w_{n}\Vert }=\frac{s_{n}e_{0}}{\Vert w_{n}\Vert }+\frac{v_{n}}{\Vert w_{n}\Vert }\), \(\Vert \frac{w_{n}}{\Vert w_{n}\Vert }\Vert =1\), by (3.4) and (K), we know that \(\frac{s_{n}e_{0}}{\Vert w_{n}\Vert }\rightarrow w_{0}\ne 0\) since \(p\in (5/2,3)\). Otherwise we can get \(1=\Vert \frac{w_{n}}{\Vert w_{n}\Vert }\Vert \rightarrow 0\). Therefore, by (3.3), we have

$$\begin{aligned} 0\le \frac{\Phi _{\varepsilon }(w_{n})}{\Vert w_{n}\Vert ^{2}}\le \varrho _{1}\cdot \Vert \frac{s_{n}e_{0}}{\Vert w_{n}\Vert }\Vert ^{2}+C_{1}- \frac{\Vert w_{n}\Vert ^{p-2}}{p}\int _{\mathbb {R}^{3}}\chi (\varepsilon x)K(\varepsilon x)\frac{|w_{n}|^{p}}{\Vert w_{n}\Vert ^{p}}{\text{ d }}x\rightarrow -\infty . \end{aligned}$$

This is a contradiction, so we have \(\Phi _{\varepsilon }(w)\rightarrow -\infty \) as \(\Vert w\Vert \rightarrow \infty \). Combining the above three cases, we can get \(\sup _{w\in E(e_{0}),\Vert w\Vert \ge R_{0}}\Phi _{\varepsilon }(w)\le 0\). Furthermore, for any \(w\in B_{R_{0}}\), there holds

$$\begin{aligned} \Phi _{\varepsilon }(w)\le \frac{1}{2}\Vert w^{+}\Vert ^{2}-\frac{1}{2}\Vert w^{}\Vert ^{2}+ \frac{3a}{4}\int _{\mathbb {R}^{3}}|w|^{2}{\text{ d }}x\le 2R_{0}. \end{aligned}$$

Now the proof is complete. \(\square \)

Let X be a reflexive Banach space, and X can be decompose to \(X=X^{+}\oplus X^{-}\). Take \(\mathcal {S}\subset (X^{-})^{*}\) be a dense subset and \(\mathcal {P}\) be the family of semi-norms on X, it consisting of all semi-norm as follow

$$\begin{aligned} p_{s}:X=X^{+}\oplus X^{-}\rightarrow \mathbb {R},~~p_{s}(x^{+}+x^{-}):=|s(x^{-})|+\Vert x^{+}\Vert ,~s\in \mathcal {S}. \end{aligned}$$

Thus \(\mathcal {P}\) induces the product topology on X, it is contained in the product topology \((X^{-},\mathcal {T}_{w})\times (X^{+},\Vert \cdot \Vert )\) on X. The associated topology is denote \(\mathcal {T}_{\mathcal {P}}\). We denote the weak\(^{*}\) topology on \(X^{*}\) by \((X^{*},\mathcal {T}_{w^{*}})\). For more detail about the \(\mathcal {T}_{\mathcal {P}}\) topology, one can see [12, Chapter4]. From now on, we take \(X=E\) and denote \(\Phi _{\varepsilon ,c}=\{w\in E:\Phi _{\varepsilon }\ge c\}\).

Lemma 3.4

Assume that (V), (K) and (Q) hold , then the functional \(\Phi _{\varepsilon }:E\rightarrow \mathbb {R}\) is sequence \(\mathcal {P}\)-upper semicontinuous and \(\Phi _{_{\varepsilon }}':(\Phi _{\varepsilon ,c},\mathcal {T_{\mathcal {P}}})\rightarrow (E^{*},\mathcal {T}_{w^{*}})\) is continuous for every \(c\in \mathbb {R}\).

Proof

The argument is similar to [32, Lemma 3.4], so we omit it. \(\square \)

Combining above lemmas and Lemma 2.1, we have the following theorem.

Theorem 3.5

([12, Theorem 4.4]) Suppose that assumptions (V),(K),(Q) hold. Then for every \((0,\varepsilon _{0})\), the modified Eq. (3.1) has a nontrivial solution \(w_{\varepsilon }\) which satisfy \(\Phi _{\varepsilon }(w_{n})\in [\rho ,\sup _{w\in E(e_{0})}\Phi _{\varepsilon }]\). Moreover, there holds \(\rho _{0}\le \Vert w_{\varepsilon }\Vert \le C_{R_{0}}\), where \(\rho _{0}>0\) and \(C_{R_{0}}>0\).

4 Profile Decomposition of Solutions and Regularity

By Theorem 3.5, we know that for any \(\varepsilon \in (0,\varepsilon _{0})\), the modified Eq. (3.1) has a nontrivial solution \(w_{\varepsilon }\). In order to show these solutions are actually solutions of the original problem (1.1), we need following several lemmas. Firstly, since \(\rho _{0}\le \Vert w_{\varepsilon }\Vert \le C_{R_{0}}\), then we have the following profile decomposition with respect to \(\{w_{\varepsilon }\}\).

Lemma 4.1

Assume \(\{\varepsilon _{n}\}\subset \mathbb {R}^{+}\) is a sequence of real numbers, and \(\varepsilon _{n}\rightarrow 0\) as \(n\rightarrow \infty \). Then there exist a sequence \(\{\sigma _{j,n}\}\subset \mathbb {R}^{+}\) and sequence \(\{x_{i,n}\}\subset \mathbb {R}^{3}\), \(\{x_{j,n}\}\subset \mathbb {R}^{3}\), such that \(\lim _{n\rightarrow \infty }\sigma _{j,n}=\infty \) and \(\{w_{\varepsilon _{n}}\}\) has following properties.

$$\begin{aligned} w_{\varepsilon _{n}}=\sum _{i\in \Lambda _{1}}W_{i}(\cdot -x_{i,n})+ \sum _{j\in \Lambda _{\infty }}\sigma _{j,n}W_{j}(\sigma _{j,n}(\cdot -x_{j,n}))+r_{n}, \end{aligned}$$

where \(\Lambda _{1}\) and \(\Lambda _{\infty }\) are finite index sets. In addition,

$$\begin{aligned} \lim _{n\rightarrow \infty }|x_{i,n}-x_{i^{'},n}|=\infty ~~for~ i,i^{'}\in \Lambda _{1}~ and~ i\ne i^{'}. \end{aligned}$$

Moreover,

(i) For any \(i\in \Lambda _{1}\), \(w_{\varepsilon _{n}}(\cdot +x_{i,n})\rightharpoonup W_{i}\ne 0\) in \(H^{1/2}(\mathbb {R}^{3},\mathbb {C}^{4})\) as \(n\rightarrow \infty \), and for any \(j\in \Lambda _{\infty }\), \(\sigma _{j,n}^{-1}w_{\varepsilon _{n}}(\sigma _{j,n}^{-1}\cdot +x_{j,n})\rightharpoonup W_{j}\ne 0\) in \(\dot{H}^{1/2}(\mathbb {R}^{3},\mathbb {C}^{4})\) as \(n\rightarrow \infty \), where \(\dot{H}^{1/2}(\mathbb {R}^{3},\mathbb {C}^{4})\) is defined by

$$\begin{aligned} \dot{H}^{1/2}(\mathbb {R}^{3},\mathbb {C}^{4}):=\{w\in L^{3}(\mathbb {R}^{3},\mathbb {C}^{4}):(-\Delta )^{1/4}w\in L^{2}(\mathbb {R}^{3},\mathbb {C}^{4})\} \end{aligned}$$

with the inner product \((w,v)=((-\Delta )^{1/4}w,(-\Delta )^{1/4}v)_{2}\) and the norm \(\Vert w\Vert _{\dot{H}^{1/2}}^{2}=(w,w)\) for any \(w,v\in \dot{H}^{1/2}(\mathbb {R}^{3},\mathbb {C}^{4})\).

(ii) There holds

$$\begin{aligned} \sum _{i\in \Lambda _{1}}\int _{\mathbb {R}^{3}}|W_{i}|^{3}{\text{ d }}x+ \sum _{j\in \Lambda _{\infty }}\int _{\mathbb {R}^{3}}|W_{j}|^{3}{\text{ d }}x \le \liminf _{n\rightarrow \infty }\int _{\mathbb {R}^{3}}|w_{\varepsilon _{n}}|^{3}{\text{ d }}x. \end{aligned}$$

(iii) \(r_{n}\rightarrow 0\) in \(L^{3}(\mathbb {R}^{3},\mathbb {C}^{4})\) as \(n\rightarrow \infty \).

(iv) \(W_{j}\) satisfies the equation

$$\begin{aligned} -i\alpha \cdot \nabla W_{j}=\chi (x_{j})E_{j}(x,|W_{j}|)W_{j}, \end{aligned}$$

where \(E_{j}(x,t)\) is defined below (4.1). \(x_{j}=\lim _{n\rightarrow \infty }\varepsilon _{n}x_{j,n}\), \(x_{j}\in \Lambda ^{\delta _{0}}\). Moreover, there holds

$$\begin{aligned} |W_{j}(x)|\le \frac{C}{1+|x|^{2}}~~for~any~x\in \mathbb {R}^{3}. \end{aligned}$$

(iv) \(W_{i}\) satisfies the equation

$$\begin{aligned} -i\alpha \cdot \nabla W_{i}+a\beta W_{i}+V(x_{i})W_{i}-\frac{a}{4}\widetilde{\chi }(x_{i})W_{i}=\widetilde{E}(x_{i},|W_{i}|)W_{i}, \end{aligned}$$

where \(\widetilde{E}(x,t)\) is given by (4.17), \(x_{i}=\lim _{n\rightarrow \infty }\varepsilon _{n}x_{i,n}\), \(x_{i}\in \Lambda ^{\delta }\). Moreover, there holds

$$\begin{aligned} |W_{i}(x)|\le C\exp (-c|x|)~~for~any~x\in \mathbb {R}^{3}, \end{aligned}$$

where C and c are positive constants.

Remark 4.2

For more information about the homogeneous Sobolev space \(\dot{H}^{1/2}(\mathbb {R}^{3},\mathbb {C}^{4})\) and the relationship between \(\dot{H}^{1/2}(\mathbb {R}^{3},\mathbb {C}^{4})\) and \(L^{p}(\mathbb {R}^{3},\mathbb {C}^{4})\), one can refer to [30]. For details of operator \((-\Delta )^{1/4}\), we refer to [23].

Proof

According to [6, Lemma 4.2], it is not difficult to know that (i), (ii) and (iii) are hold. Hence we only need to prove (iv) and (v). We first introduce the following piecewise function, which will be used to construct the equation satisfied by \(W_{j}\). Denote \(\rho _{j}=\lim _{n\rightarrow \infty }\varepsilon _{n}\sigma _{j,n}^{2}\). We define

$$\begin{aligned} E_{j}(x,t):= \left\{ \begin{array}{ll} 0, &{} \rho _{j}=+\infty ; \\ Q(0)t, &{} \rho _{j}=0;\\ \frac{p}{3}Q(0)\rho _{j}^{-\frac{3-p}{2}}t^{p-2}A^{\frac{3-p}{2}}+ \frac{3-p}{3}Q(0)\rho _{j}^{\frac{p-1}{2}}t^{p}A^{\frac{3-p}{2}-1}\varphi (\rho _{j}t^{2}),&{} 0<\rho _{j}<+\infty , \end{array} \right. \end{aligned}$$
(4.1)

where \(A=\Psi (\rho _{j}t^{2})\) and \(\Psi (t)=\int _{0}^{t}\varphi (s){\text{ d }}s\). By (Q), we know that

$$\begin{aligned} \sup _{x\in \mathbb {R}^{3}}\sup _{t>0}t^{-1}E_{j}(x,t)<+\infty . \end{aligned}$$

Let \(u_{j,n}=\sigma _{j,n}^{-1}w_{\varepsilon _{n}}(\sigma _{j,n}^{-1}\cdot +x_{j,n})\). Since \(w_{\varepsilon _{n}}\) satisfies Eq.  (3.1) with \(\varepsilon =\varepsilon _{n}\), then \(u_{j,n}\) satisfies the equation

$$\begin{aligned}&-i\alpha \cdot \nabla u_{j,n}+\sigma _{j,n}^{-1} a\beta u_{j,n}+\sigma _{j,n}^{-1}V\left( \varepsilon _{n}(\sigma _{j,n}^{-1}\cdot +x_{j,n})\right) u_{j,n} \nonumber \\&\quad -\sigma _{j,n}^{-1}\frac{a}{8}\widetilde{\chi }\left( \varepsilon _{n}(\sigma _{j,n}^{-1}\cdot +x_{j,n})\right) \cdot \widetilde{\xi }\left( \sigma _{j,n}^{-1}\cdot +x_{j,n},\sigma _{j,n}|u_{j,n}| \right) u_{j,n}\nonumber \\&\quad =\sigma _{j,n}^{-1}f_{\varepsilon _{n}}\left( \sigma _{j,n}^{-1}\cdot +x_{j,n},\sigma _{j,n}|u_{j,n}| \right) u_{j,n}. \end{aligned}$$
(4.2)

Since \(\sigma _{j,n}\rightarrow \infty \) as \(n\rightarrow \infty \), hence, for any \(\varphi \in C_{0}^{\infty }(\mathbb {R}^{3},\mathbb {C}^{4})\),

(4.3)

By the definition of \(\widetilde{\xi }\), we know that \(\widetilde{\xi }(x,t)\in C^{1}(\mathbb {R}^{3}\times \mathbb {R},[0,2])\), then

(4.4)

Similarly,

(4.5)

Now we prove \(x_{j}=\lim _{n\rightarrow \infty }\varepsilon _{n}x_{j,n}\in \Lambda ^{\delta _{0}}\). We assume that \(|\varepsilon _{n}x_{j,n}|\rightarrow \infty \) or \(\varepsilon _{n}x_{j,n}\rightarrow x_{0}\notin \Lambda ^{\delta _{0}}\) as \(n\rightarrow \infty \), then

(4.6)

Thus, combining (4.2), (4.3), (4.4), (4.5) and (4.6), we can get

$$\begin{aligned} \int _{\mathbb {R}^{3}}-i\alpha \cdot \nabla u_{j,n}\cdot \varphi {\text{ d }}x\rightarrow 0~~\text {for any }\varphi \in C_{0}^{\infty }(\mathbb {R}^{3},\mathbb {C}^{4}). \end{aligned}$$

By (i), there holds \(u_{j,n}\rightharpoonup W_{j}\), consequently,

$$\begin{aligned} -i\alpha \cdot W_{j}=0. \end{aligned}$$

It follows that \(W_{j}=0\), which contradicts (i). Therefore, \(x_{j}=\lim _{n\rightarrow \infty }\varepsilon _{n}x_{j,n}\in \Lambda ^{\delta _{0}}\). By the definition of \(h_{\varepsilon _{n}}(x,t)\) and \(E_{j}(x,t)\), we claim that

$$\begin{aligned} \sigma _{j,n}^{-1}h_{\varepsilon _{n}}(x,\sigma _{j,n}t)\rightarrow E_{j}(x,t)~\text { for any }x\in \mathbb {R}^{3},~t\in [0,\infty )\text { as }n\rightarrow \infty . \end{aligned}$$
(4.7)

If \(\rho _{j}:=\lim _{n\rightarrow \infty }\varepsilon _{n}\sigma _{j,n}^{2}\in (0,\infty )\), then for any \(x\in \mathbb {R}^{3}\) and \(t\in [0,\infty )\), there holds

$$\begin{aligned}&\sigma _{j,n}^{-1}h_{\varepsilon _{n}}(x,\sigma _{j,n}t)\nonumber \\&\quad =\sigma _{j,n}^{-1}\left\{ K(\varepsilon _{n} x)(\sigma _{j,n}t)^{p-2}+\frac{p}{3}Q(\varepsilon _{n} x)(\sigma _{j,n}t)^{p-2}(m_{\varepsilon _{n}}(\sigma _{j,n}t)^{2})^{\frac{3-p}{2}}\right\} \nonumber \\&\qquad +\sigma _{j,n}^{-1}\frac{3-p}{3}Q(\varepsilon _{n} x)(\sigma _{j,n}t)^{p}(m_{\varepsilon _{n}}(\sigma _{j,n}t)^{2})^{\frac{3-p}{2}-1}b_{\varepsilon _{n}}((\sigma _{j,n}t)^{2}). \end{aligned}$$
(4.8)

Observed that

$$\begin{aligned} m_{\varepsilon _{n}}((\sigma _{j,n}t)^{2})=\int _{0}^{(\sigma _{j,n}t)^{2}}b_{\varepsilon _{n}}(s){\text{ d }}s =\frac{1}{\varepsilon _{n}}\int _{0}^{\varepsilon _{n}\sigma _{j,n}^{2}t^{2}}\varphi (s){\text{ d }}s \end{aligned}$$

Hence, we have

$$\begin{aligned}&\sigma _{j,n}^{-1}\left\{ K(\varepsilon _{n} x)(\sigma _{j,n}t)^{p-2}+\frac{p}{3}Q(\varepsilon _{n} x)(\sigma _{j,n}t)^{p-2}\left( m_{\varepsilon _{n}}(\sigma _{j,n}t)^{2}\right) ^{\frac{3-p}{2}}\right\} \nonumber \\&=\sigma _{j,n}^{p-3}t^{p-2}\left\{ K(\varepsilon _{n} x)+\frac{p}{3}Q(\varepsilon _{n} x)(\varepsilon _{n})^{\frac{p-3}{2}}\left( \int _{0}^{\varepsilon _{n}\sigma _{j,n}^{2}t^{2}}\varphi (s){\text{ d }}s \right) ^{\frac{3-p}{2}}\right\} \nonumber \\&\rightarrow \frac{p}{3}Q(0)\rho _{j}^{-\frac{3-p}{2}}\left( \Psi (\rho _{j}t^{2})\right) ^{\frac{3-p}{2}}~~\text {as }n\rightarrow \infty . \end{aligned}$$
(4.9)

and

$$\begin{aligned}&\sigma _{j,n}^{-1}\frac{3-p}{3}Q(\varepsilon _{n} x)(\sigma _{j,n}t)^{p}\left( m_{\varepsilon _{n}}\left( \sigma _{j,n}t \right) ^{2}\right) ^{\frac{3-p}{2}-1}b_{\varepsilon _{n}}\left( (\sigma _{j,n}t)^{2}\right) \nonumber \\&=\frac{3-p}{3}Q(\varepsilon _{n} x)(\sigma _{j,n})^{p-1}t^{p}(\varepsilon _{n})^{\frac{1-p}{2}} \left( \int _{0}^{\varepsilon _{n}\sigma _{j,n}^{2}t^{2}}\varphi (s){\text{ d }}s\right) ^{\frac{3-p}{2}-1}\varphi (\varepsilon _{n}\sigma _{j,n}^{2}t^{2}) \nonumber \\&\rightarrow \frac{3-p}{3}Q(0)\rho _{j}^{\frac{p-1}{2}}t^{p}\left( \Psi (\rho _{j}t^{2})\right) ^{\frac{3-p}{2}-1}\varphi (\rho _{j}t^{2})~~\text {as }n\rightarrow \infty . \end{aligned}$$
(4.10)

Taking (4.9) and (4.10) into (4.8), we obtain that for any \(x\in \mathbb {R}^{3}\), \(t\in [0,\infty )\),

$$\begin{aligned} \lim _{n\rightarrow \infty }\sigma _{j,n}^{-1}h_{\varepsilon _{n}}(x,\sigma _{j,n}t)= E_{j}(x,t)~~\text { for }0<\rho _{j}<+\infty . \end{aligned}$$

Similarly, we can derive that

$$\begin{aligned} \lim _{n\rightarrow \infty }\sigma _{j,n}^{-1}h_{\varepsilon _{n}}(x,\sigma _{j,n}t)=0~~\text {with }\rho _{j}=+\infty . \end{aligned}$$

and

$$\begin{aligned} \lim _{n\rightarrow \infty }\sigma _{j,n}^{-1}h_{\varepsilon _{n}}(x,\sigma _{j,n}t)=Q(0)t~~\text {with }\rho _{j}=0. \end{aligned}$$

Then the claim is true. From (i), we know that

$$\begin{aligned} u_{j,n}=\sigma _{j,n}^{-1}w_{\varepsilon _{n}}(\sigma _{j,n}^{-1}\cdot +x_{j,n})\rightarrow W_{j}(x)~~\text {a.e. }x\in \mathbb {R}^{3}~\text {as }n\rightarrow \infty . \end{aligned}$$
(4.11)

Combining the (4.7) and (4.11), there holds

$$\begin{aligned} \lim _{n\rightarrow \infty }\sigma _{j,n}^{-1}h_{\varepsilon _{n}}(x,\sigma _{j,n}|u_{j,n}|)u_{j,n}=E_{j}(x,|W_{j}|)W_{j}~~\text {a.e. }x\in \mathbb {R}^{3}~\text {as }n\rightarrow \infty . \end{aligned}$$

Then from Lebesgue dominated convergence theorem, it follows that

$$\begin{aligned}&\sigma _{j,n}^{-1}\int _{\mathbb {R}^{3}}\chi (\varepsilon _{n}(\sigma _{j,n}^{-1} x+x_{j,n})h_{\varepsilon _{n}}(\sigma _{j,n}^{-1} x+x_{j,n},\sigma _{j,n}|u_{j,n}|)u_{j,n}\cdot \varphi {\text{ d }}x \nonumber \\&\rightarrow \int _{\mathbb {R}^{3}}\chi (x_{j})E_{j}(x,|W_{j}|)W_{j}\cdot \varphi {\text{ d }}x~\text {as }n\rightarrow \infty . \end{aligned}$$
(4.12)

By (4.2), (4.3), (4.4), (4.5) and (4.12), we have

$$\begin{aligned} -i\alpha \cdot \nabla W_{j}=\chi (x_{j})E_{j}(x,|W_{j}|)W_{j}. \end{aligned}$$

Thus, from [3, Theorem 1.1], we can obtain

$$\begin{aligned} |W_{j}(x)|\le \frac{C}{1+|x|^{2}}~~\text {for any}~x\in \mathbb {R}^{3}. \end{aligned}$$

We finish the proof of (iv).

To prove (v). Since \(w_{\varepsilon _{n}}\) satisfies Eq.  (3.1) with \(\varepsilon =\varepsilon _{n}\), i.e.,

$$\begin{aligned} -i\alpha \cdot \nabla w_{\varepsilon _{n}}+a\beta w_{\varepsilon _{n}} +V(\varepsilon _{n} x)w_{\varepsilon _{n}}-\frac{a}{8}\widetilde{\chi }(\varepsilon _{n} x)\widetilde{\xi }(x,|w_{\varepsilon _{n}}|)w_{\varepsilon _{n}}=f_{\varepsilon _{n}}(x,|w_{\varepsilon _{n}}|)w_{\varepsilon _{n}}. \end{aligned}$$

From (i), we know that \(w_{\varepsilon _{n}}(\cdot +x_{i,n})\rightharpoonup W_{i}\ne 0\) in \(H^{1/2}(\mathbb {R}^{3},\mathbb {C}^{4})\) as \(n\rightarrow \infty \). Denote \(u_{i,n}:=w_{\varepsilon _{n}}(\cdot +x_{i,n})\), then

$$\begin{aligned}&-i\alpha \cdot \nabla u_{i,n}+a\beta u_{i,n} +V(\varepsilon _{n}(x+x_{i,n}))u_{i,n}-\frac{a}{8}\widetilde{\chi }(\varepsilon _{n}(x+x_{i,n}) )\widetilde{\xi }(x+x_{i,n},|u_{i,n}|)u_{i,n} \nonumber \\&\qquad =f_{\varepsilon _{n}}((x+x_{i,n}),|u_{i,n}|)u_{i,n}. \end{aligned}$$
(4.13)

If \(x_{i}:=\lim _{n\rightarrow \infty }\varepsilon _{n}x_{i,n}\in \Lambda ^{\delta _{0}}\), then for any \(\varphi \in C_{0}^{\infty }(\mathbb {R}^{3},\mathbb {C}^{4})\), we have

$$\begin{aligned}&\int _{\mathbb {R}^{3}}-i\alpha \cdot \nabla u_{i,n}\cdot \varphi {\text{ d }}x\rightarrow \int _{\mathbb {R}^{3}}-i\alpha \cdot \nabla W_{i}\cdot \varphi {\text{ d }}x, \nonumber \\&\int _{\mathbb {R}^{3}}a\beta u_{i,n}\cdot \varphi {\text{ d }}x\rightarrow \int _{\mathbb {R}^{3}}a\beta W_{i}\cdot \varphi {\text{ d }}x, \nonumber \\&\int _{\mathbb {R}^{3}}V(\varepsilon _{n}(x+x_{i,n}))u_{i,n}\cdot \varphi {\text{ d }}x\rightarrow \int _{\mathbb {R}^{3}}V(x_{i})W_{i}\cdot \varphi {\text{ d }}x, \nonumber \\&\int _{\mathbb {R}^{3}}\frac{a}{8}\widetilde{\chi }(\varepsilon _{n}(x+x_{i,n}) )\widetilde{\xi }(x+x_{i,n},|u_{i,n}|)u_{i,n}\cdot \varphi {\text{ d }}x \nonumber \\&\rightarrow \int _{\mathbb {R}^{3}}\frac{a}{8}\widetilde{\chi }(x_{i})2\cdot W_{i}\cdot \varphi {\text{ d }}x=\frac{a}{4}\int _{\mathbb {R}^{3}}\widetilde{\chi }(x_{i}) W_{i}\cdot \varphi {\text{ d }}x. \end{aligned}$$
(4.14)

In additional, there holds

$$\begin{aligned}&\int _{\mathbb {R}^{3}} f_{\varepsilon _{n}}\left( (x+x_{i,n}),|u_{i,n}| \right) u_{i,n}\cdot \varphi {\text{ d }}x\nonumber \\&\quad =\int _{\mathbb {R}^{3}}\chi \left( \varepsilon _{n}(x+x_{i,n})h_{\varepsilon _{n}}((x+x_{i,n}),|u_{i,n}| \right) u_{i,n}\cdot \varphi {\text{ d }}x \nonumber \\&\qquad +\int _{\mathbb {R}^{3}}\left( 1-\chi (\varepsilon _{n}(x+x_{i,n})\right) g_{\varepsilon _{n}}((x+x_{i,n}),|u_{i,n}|)u_{i,n}\cdot \varphi {\text{ d }}x. \end{aligned}$$
(4.15)

Since

$$\begin{aligned}&h_{\varepsilon _{n}}((x+x_{i,n}),|u_{i,n}|)\\ {}&=K(\varepsilon _{n}(x+x_{i,n}))|u_{i,n}|^{p-2}+ \frac{p}{3}Q(\varepsilon _{n}(x+x_{i,n}))|u_{i,n}|^{p-2}\left( m_{\varepsilon _{n}}(|u_{i,n}|^{2})\right) ^{\frac{3-p}{2}}\\&+\frac{3-p}{3}Q(\varepsilon _{n}(x+x_{i,n}))|u_{i,n}|^{p}\left( m_{\varepsilon _{n}}(|u_{i,n}|^{2})\right) ^{\frac{3-p}{2}-1} b_{\varepsilon _{n}}(|u_{i,n}|^{2})\\&\rightarrow K(x_{i})|W_{i}|^{p-2}+Q(x_{i})|W_{i}|^{2}~~\text {a.e. }x\in \mathbb {R}^{3}~~\text {as }n\rightarrow \infty , \end{aligned}$$

and

$$\begin{aligned}&g_{\varepsilon _{n}}((x+x_{i,n}),|u_{i,n}|)=\min \left\{ h_{\varepsilon _{n}}((x+x_{i,n}),|u_{i,n}|),\phi (x+x_{i,n})\right\} \\&\rightarrow \min \left\{ K(x_{i})|W_{i}|^{p-2}+Q(x_{i})|W_{i}|^{2},0 \right\} ~~\text {a.e. }x\in \mathbb {R}^{3}~~\text {as }n\rightarrow \infty , \end{aligned}$$

Consequently, by (4.15), there holds

$$\begin{aligned}&\int _{\mathbb {R}^{3}} f_{\varepsilon _{n}}\left( (x+x_{i,n}),|u_{i,n}| \right) u_{i,n}\cdot \varphi {\text{ d }}x\\&\quad \rightarrow \int _{\mathbb {R}^{3}}\chi (x_{i}) \left( K(x_{i})|W_{i}|^{p-2}+Q(x_{i})|W_{i}|^{2} \right) W_{i}\cdot \varphi {\text{ d }}x\\&\quad +\int _{\mathbb {R}^{3}}\left( 1-\chi (x_{i})\right) \min \left\{ K(x_{i})|W_{i}|^{p-2}+Q(x_{i})|W_{i}|^{2},0 \right\} W_{i}\cdot \varphi {\text{ d }}x~\text {as }n\rightarrow \infty . \end{aligned}$$

By (K) and (Q), it follows that

$$\begin{aligned}&\int _{\mathbb {R}^{3}} f_{\varepsilon _{n}}\left( (x+x_{i,n}),|u_{i,n}| \right) u_{i,n}\cdot \varphi {\text{ d }}x \nonumber \\&\quad \rightarrow \int _{\mathbb {R}^{3}}\chi (x_{i})\left( K(x_{i})|W_{i}|^{p-2}+Q(x_{i})|W_{i}|^{2}\right) W_{i}\cdot \varphi {\text{ d }}x~\text {as }n\rightarrow \infty . \end{aligned}$$
(4.16)

We define

$$\begin{aligned} \widetilde{E}(x,t)=\chi (x)K(x)|t|^{p-2}+\chi (x)Q(x)|t|^{2}. \end{aligned}$$
(4.17)

Combining (4.13), (4.14), (4.15), (4.16) and (4.17), there holds

$$\begin{aligned}&\int _{\mathbb {R}^{3}}\left( -i\alpha \cdot \nabla u_{i,n}+a\beta u_{i,n} +V(\varepsilon _{n}(x+x_{i,n}))u_{i,n}\right) \cdot \varphi {\text{ d }}x\\&-\int _{\mathbb {R}^{3}}\left( \frac{a}{8}\widetilde{\chi }\left( \varepsilon _{n}(x+x_{i,n})\right) \widetilde{\xi }(x+x_{i,n},|u_{i,n}|)+ f_{\varepsilon _{n}}\left( (x+x_{i,n}),|u_{i,n}| \right) u_{i,n} \right) \cdot \varphi {\text{ d }}x\\&\rightarrow \int _{\mathbb {R}^{3}} \left[ -i\alpha \cdot \nabla W_{i}+a\beta W_{i}+V(x_{i})W_{i}-\frac{a}{4}\widetilde{\chi }(x_{i})W_{i}-\widetilde{E}(x_{i},|W_{i}|)W_{i}\right] \cdot \varphi {\text{ d }}x. \end{aligned}$$

Then, we have

$$\begin{aligned} -i\alpha \cdot \nabla W_{i}+a\beta W_{i}+V(x_{i})W_{i}-\frac{a}{4}\widetilde{\chi }(x_{i})W_{i}=\widetilde{E}(x_{i},|W_{i}|)W_{i}. \end{aligned}$$
(4.18)

Now we will show that \(x_{i}:=\lim _{n\rightarrow \infty }\varepsilon _{n}x_{i,n}\in \Lambda ^{\delta _{0}}\). We assume that \(x_{i}\notin \Lambda ^{\delta _{0}}\), by the definition of \(f_{\varepsilon _{n}}\) and \(\widetilde{\xi }\), then \(W_{i}\) satisfies the equation

$$\begin{aligned} -i\alpha \cdot \nabla W_{i}+a\beta W_{i}+V(x_{i})W_{i}-\frac{a}{4}\widetilde{\chi }(x_{i})W_{i}=0. \end{aligned}$$

Take the scalar product with \(\left( W_{i}^{+}-W_{i}^{-}\right) \) and integrate in \(\mathbb {R}^{3}\), we have

$$\begin{aligned} 0&=\Vert W_{i}\Vert ^{2}+\text {Re}\int _{\mathbb {R}^{3}}V(x_{i})W_{i}\cdot \left( W_{i}^{+}-W_{i}^{-}\right) {\text{ d }}x\\&\quad -\text {Re}\frac{a}{4}\int _{\mathbb {R}^{3}}\widetilde{\chi }(x_{i})W_{i}\cdot \left( W_{i}^{+}-W_{i}^{-}\right) {\text{ d }}x\\&\ge a\Vert W_{i}\Vert _{2}^{2}-\frac{3a}{4}\Vert W_{i}\Vert _{2}^{2}=\frac{a}{4}\Vert W_{i}\Vert _{2}^{2}. \end{aligned}$$

Therefore, we obtain \(W_{i}=0\), which contradicts (i). Consequently, \(x_{i}\in \Lambda ^{\delta _{0}}\). Since \(W_{i}\) satisfies (4.18), then according to [31, Lemma 4.6], there holds

$$\begin{aligned} |W_{i}(x)|\le C\exp (-c|x|)~~\text {for any}~x\in \mathbb {R}^{3}. \end{aligned}$$

The proof is now completed. \(\square \)

Lemma 4.3

Assume that (P), (Q) and (K) hold, \(5/2<p<3\), then the index set \(\Lambda _{\infty }=\emptyset \).

Proof

The proof of this lemma is similar to the one of [6, Lemma 4.21] with the help of Lemma 4.1 in this paper, therefore, we omit its proof. \(\square \)

Now we give the \(L^{\infty }\) estimate for the solutions which solves modified Eq.  (3.1).

Lemma 4.4

Assume that (P), (Q) and (K) hold, \(5/2<p<3\), let \(\{w_{\epsilon }\}\) be a family of critical points of (3.1) which obtained in Theorem 3.5. Then there exist \(M>0\) and \(\varepsilon _{0}>0\), such that for any \(0<\varepsilon <\varepsilon _{0}\),

$$\begin{aligned} \sup _{x\in \mathbb {R}^{3}}|w_{\varepsilon }(x)|\le M. \end{aligned}$$

Before prove the Lemma 4.4, we need the following two lemmas.

Lemma 4.5

[25, Lemma 4.2] For any \(p\in (1,\infty )\), there exists a constant \(C>0\) such that

$$\begin{aligned} \Vert \nabla \psi \Vert _{L^{p}(\mathbb {R}^{3})}\le C\Vert i\alpha \cdot \nabla \psi \Vert _{L^{p}(\mathbb {R}^{3})}~~\text {for any }\psi \in C_{0}^{\infty }(\mathbb {R}^{3},\mathbb {C}^{4}). \end{aligned}$$

Lemma 4.6

Let \(\zeta \) be a cut-off function such that \(\zeta (x)=1\) for \(x\in B_{R/2}(0)\), \(\zeta (x)=0\) for \(x\notin B_{R}(0)\) and \(|\nabla \zeta (x)|\le R/4\), \(w_{\varepsilon _{n}}\) is solution of (3.1), then there holds

$$\begin{aligned} \Vert \zeta w_{\varepsilon _{n}}\Vert _{W^{1,p}(\mathbb {R}^{3})}\le C_{p,R}\Vert w_{\varepsilon _{n}}\Vert _{L^{p}(B_{R}(0))}+C_{p}\Vert \zeta |w_{\varepsilon _{n}}|^{2}\Vert _{L^{p}(B_{R}(0))}. \end{aligned}$$

Proof

Since \(\{w_{\varepsilon _{n}}\}\) solves Eq.  (3.1), i.e.,

$$\begin{aligned} -i\alpha \cdot \nabla w_{\varepsilon _{n}}+a\beta w_{\varepsilon _{n}} +V(\varepsilon _{n} x)w_{\varepsilon _{n}}-\frac{a}{8}\widetilde{\chi }(\varepsilon _{n} x)\widetilde{\xi }(x,|w_{\varepsilon _{n}}|)w_{\varepsilon _{n}}=f_{\varepsilon _{n}}(x,|w_{\varepsilon _{n}}|)w_{\varepsilon _{n}}. \end{aligned}$$

By multiplying \(w_{\varepsilon _{n}}\) with \(\zeta \) and substituting the product into above formula, there holds

$$\begin{aligned}&a\beta (\zeta w_{\varepsilon _{n}}) +V(\varepsilon _{n} x)(\zeta w_{\varepsilon _{n}})-\frac{a}{8}\widetilde{\chi }(\varepsilon _{n} x)\widetilde{\xi }(x,|\zeta w_{\varepsilon _{n}}|)(\zeta w_{\varepsilon _{n}}) -f_{\varepsilon _{n}}(x,|\zeta w_{\varepsilon _{n}}|)(\zeta w_{\varepsilon _{n}}) \nonumber \\&=i\alpha \cdot \nabla (\zeta w_{\varepsilon _{n}})=\zeta (i\alpha \cdot \nabla w_{\varepsilon _{n}})+i\sum _{k=1}^{3}(\partial _{k}\zeta )\alpha _{k}\cdot w_{\varepsilon _{n}}. \end{aligned}$$
(4.19)

It is clear that

$$\begin{aligned} \int _{\mathbb {R}^{3}}|a\beta (\zeta w_{\varepsilon _{n}})|^{p}{\text{ d }}x\le a^{p}\int _{B_{R}(0)}|w_{\varepsilon _{n}}|^{p}{\text{ d }}x. \end{aligned}$$

By the definition of \(\widetilde{\chi }\) and \(\widetilde{\xi }\), we know that

$$\begin{aligned}&\int _{\mathbb {R}^{3}}\left| V(\varepsilon _{n} x)(\zeta w_{\varepsilon _{n}})-\frac{a}{8}\widetilde{\chi }(\varepsilon _{n} x)\widetilde{\xi }(x,|\zeta w_{\varepsilon _{n}}|)(\zeta w_{\varepsilon _{n}})\right| ^{p}{\text{ d }}x\\&\le \int _{B_{R}(0)}\left| V(\varepsilon _{n} x)w_{\varepsilon _{n}}\right| ^{p}{\text{ d }}x+\int _{B_{R}(0)}\left| \frac{a}{8}\widetilde{\chi }(\varepsilon _{n} x)\widetilde{\xi }(x,| w_{\varepsilon _{n}}|) w_{\varepsilon _{n}}\right| ^{p}{\text{ d }}x\\&\le a^{p}\int _{B_{R}(0)}|w_{\varepsilon _{n}}|^{p}{\text{ d }}x+\left( \frac{a}{4}\right) ^{p}\int _{B_{R}(0)}|w_{\varepsilon _{n}}|^{p}{\text{ d }}x\\&= \left( 1+\frac{1}{4^{p}}\right) a^{p}\int _{B_{R}(0)}|w_{\varepsilon _{n}}|^{p}{\text{ d }}x. \end{aligned}$$

Combining this and (4.19), there holds

$$\begin{aligned} \Vert i\alpha \cdot \nabla (\zeta w_{\varepsilon _{n}})\Vert _{L^{p}(\mathbb {R}^{3})}\le C\Vert w_{\varepsilon _{n}}\Vert _{L^{p}(B_{R}(0))}+\Vert f_{\varepsilon _{n}}(x,|\zeta w_{\varepsilon _{n}}|)|\zeta w_{\varepsilon _{n}}|\Vert _{L^{p}(\mathbb {R}^{3})}. \end{aligned}$$
(4.20)

By the definition of \(f_{\varepsilon _{n}}\), we have

$$\begin{aligned}&\Vert f_{\varepsilon _{n}}(x,|\zeta w_{\varepsilon _{n}}|)|\zeta w_{\varepsilon _{n}}|\Vert _{L^{p}(\mathbb {R}^{3})}^{p}=\int _{\mathbb {R}^{3}}f_{\varepsilon _{n}}(x,|\zeta w_{\varepsilon _{n}}|)|\zeta w_{\varepsilon _{n}}|^{p}{\text{ d }}x \nonumber \\&\le \int _{B_{R}(0)}f_{\varepsilon _{n}}(x,|\zeta w_{\varepsilon _{n}}|)| w_{\varepsilon _{n}}|^{p}{\text{ d }}x\le C\int _{B_{R}(0)}|(K(\varepsilon _{n}x)+Q(\varepsilon _{n}x)|\zeta w_{\varepsilon _{n}}|)|\zeta w_{\varepsilon _{n}}||^{p}{\text{ d }}x \nonumber \\&\le C_{1}\int _{B_{R}(0)}|K(\varepsilon _{n}x)w_{\varepsilon _{n}}|^{p}{\text{ d }}x+ C_{2}\int _{B_{R}(0)}|Q(\varepsilon _{n}x)\zeta |w_{\varepsilon _{n}}|^{2}|^{p}{\text{ d }}x \nonumber \\&\le C_{3}\int _{B_{R}(0)}|w_{\varepsilon _{n}}|^{p}+|\zeta |w_{\varepsilon _{n}}|^{2}|^{p}{\text{ d }}x. \end{aligned}$$
(4.21)

Using Lemma 4.5, there holds

$$\begin{aligned} \Vert \zeta w_{\varepsilon _{n}}\Vert _{W^{1,p}(\mathbb {R}^{3})}&=\Vert \zeta w_{\varepsilon _{n}}\Vert _{L^{p}(\mathbb {R}^{3})}+\Vert \nabla (\zeta w_{\varepsilon _{n}})\Vert _{L^{p}(\mathbb {R}^{3})} \nonumber \\&\le \Vert w_{\varepsilon _{n}}\Vert _{L^{p}(B_{R}(0))}+C_{p}\Vert \alpha \cdot \nabla (\zeta w_{\varepsilon _{n}})\Vert _{L^{p}(\mathbb {R}^{3})}. \end{aligned}$$
(4.22)

Combining (4.20), (4.21) and (4.22), there holds

$$\begin{aligned} \Vert \zeta w_{\varepsilon _{n}}\Vert _{W^{1,p}(\mathbb {R}^{3})}\le C_{p,R}\Vert w_{\varepsilon _{n}}\Vert _{L^{p}(B_{R}(0))}+C_{p}\Vert \zeta |w_{\varepsilon _{n}}|^{2}\Vert _{L^{p}(B_{R}(0))}. \end{aligned}$$

The proof of Lemma 4.6 is now complete. \(\square \)

Proof of Lemma 4.4

We assume that there exist a sequence of \(\{\varepsilon _{n}\}\) such that \(\varepsilon _{n}\rightarrow 0\) as \(n\rightarrow \infty \) and a sequence of critical points \(\{w_{\varepsilon _{n}}\}\subset E\) of (3.1) such that

$$\begin{aligned} \sup _{x\in \mathbb {R}^{3}}|w_{\varepsilon _{n}}(x)|\rightarrow \infty ~~\text {as }n\rightarrow \infty . \end{aligned}$$

By Lemma 4.3 and (i) of Lemma 4.1, we have

$$\begin{aligned} w_{\varepsilon _{n}}=\sum _{i\in \Lambda _{1}}W_{i}(\cdot -x_{i,n})+r_{n}. \end{aligned}$$

Moreover, by (iii) of Lemma 4.1, there holds

$$\begin{aligned} r_{n}\rightarrow 0~~\text {in }L^{3}(\mathbb {R}^{3},\mathbb {R})~\text {as }n\rightarrow \infty . \end{aligned}$$

Since \(\{w_{\varepsilon _{n}}\}\) solves Eq.  (3.1), i.e.,

$$\begin{aligned} -i\alpha \cdot \nabla w_{\varepsilon _{n}}+a\beta w_{\varepsilon _{n}} +V(\varepsilon _{n} x)w_{\varepsilon _{n}}-\frac{a}{8}\widetilde{\chi }(\varepsilon _{n} x)\widetilde{\xi }(x,|w_{\varepsilon _{n}}|)w_{\varepsilon _{n}}=f_{\varepsilon _{n}}(x,|w_{\varepsilon _{n}}|)w_{\varepsilon _{n}}. \end{aligned}$$
(4.23)

By (v) of Lemma 4.1, we know that \(|W_{i}|\in L^{\infty }(\mathbb {R}^{3},\mathbb {R})\) for any \(i\in \Lambda _{1}\). Using this and (4.20), we can deduce there exist \(N_{\gamma }>0\) and \(\varrho >0\), such that

$$\begin{aligned} \sup _{y\in \mathbb {R}^{3}}\int _{B_{\varrho }(y)}|w_{\varepsilon _{n}}|^{3}{\text{ d }}x\le \gamma ~~\text {for any } n>N_{\gamma }. \end{aligned}$$
(4.24)

Define \(\eta \in C_{0}^{\infty }(\mathbb {R}^{3},[0,1])\) such that \(\eta (x)=1\) for \(x\in B_{\varrho /2}(y)\), \(\eta (x)=0\) for \(x\notin B_{\varrho }(y)\) and \(|\nabla \eta (x)|\le 4/\varrho \) for \(x\in \mathbb {R}^{3}\). By multiplying \(w_{\varepsilon _{n}}\) with \(\eta \) and substituting the product into (4.23), there holds

$$\begin{aligned}&a\beta (\eta w_{\varepsilon _{n}}) +V(\varepsilon _{n} x)(\eta w_{\varepsilon _{n}})-\frac{a}{8}\widetilde{\chi }(\varepsilon _{n} x)\widetilde{\xi }(x,|\eta w_{\varepsilon _{n}}|)(\eta w_{\varepsilon _{n}}) -f_{\varepsilon _{n}}(x,|\eta w_{\varepsilon _{n}}|)(\eta w_{\varepsilon _{n}})\\&=i\alpha \cdot \nabla w_{\varepsilon _{n}}=\eta (i\alpha \cdot \nabla w_{\varepsilon _{n}})+i\sum _{k=1}^{3}(\partial _{k}\eta )\alpha _{k}\cdot w_{\varepsilon _{n}}. \end{aligned}$$

By Lemma 4.6, we have

$$\begin{aligned} \Vert \eta w_{\varepsilon _{n}}\Vert _{W^{1,p}(\mathbb {R}^{3})}\le C_{p,\varrho }\Vert w_{\varepsilon _{n}}\Vert _{L^{p}(B_{\varrho }(y))}+C_{p}\Vert \eta |w_{\varepsilon _{n}}|^{2}\Vert _{L^{p}(B_{\varrho }(y))}. \end{aligned}$$

Then Hölder inequality and (4.24) implies

$$\begin{aligned} \Vert \eta |w_{\varepsilon _{n}}|^{2}\Vert _{L^{p}(B_{\varrho }(y))}\le \Vert w_{\varepsilon _{n}}\Vert _{L^{3}(B_{\varrho }(y))}\cdot \Vert \eta |w_{\varepsilon _{n}}|\Vert _{L^{p^{*}}(B_{\varrho }(y))}\le \gamma ^{1/3}\Vert \eta |w_{\varepsilon _{n}}|\Vert _{L^{p^{*}}(B_{\varrho }(y))}, \end{aligned}$$

where \(p^{*}=\frac{3p}{3-p}\). Hence, when \(\gamma >0\) small enough,

$$\begin{aligned} \Vert w_{\varepsilon _{n}}\Vert _{L^{p^{*}}(B_{\varrho /2}(y))}&\le \Vert \eta |w_{\varepsilon _{n}}|\Vert _{L^{p^{*}}(B_{\varrho }(y))} \le \frac{1}{S_{p}}\Vert \eta w_{\varepsilon _{n}}\Vert _{W^{1,p}(\mathbb {R}^{3})}\\&\le \frac{1}{S_{p}}\left[ C_{p,\varrho }\Vert w_{\varepsilon _{n}}\Vert _{L^{p}(B_{\varrho }(y))} +C_{p}\gamma ^{1/3}\Vert \eta w_{\varepsilon _{n}}\Vert _{L^{p^{*}}(B_{\varrho }(y))}\right] , \end{aligned}$$

where \(S_{p}\) is Sobolev constant, which deduce that

$$\begin{aligned} \Vert w_{\varepsilon _{n}}\Vert _{L^{p^{*}}(B_{\varrho /2}(y))}^{3}\le \frac{C}{S_{p}}\Vert w_{\varepsilon _{n}}\Vert _{L^{p}(B_{\varrho }(y))} \le C'\Vert w_{\varepsilon _{n}}\Vert _{L^{3}(B_{\varrho }(y))}^{3}\le C'\gamma . \end{aligned}$$
(4.25)

Since \(p\in \left( \frac{5}{2},3 \right) \), it follows that \(p^{*}=\frac{3p}{3-p}\in (15,+\infty )\). Therefore, by (4.25), there holds

$$\begin{aligned} \Vert w_{\varepsilon _{n}}\Vert _{L^{15}(B_{\varrho /2}(y))}^{3}\le C'\gamma . \end{aligned}$$
(4.26)

Denote \(\widetilde{\eta }(x)=\eta (2x)\), then using Lemma 4.6 again, we can get

$$\begin{aligned} \Vert \widetilde{\eta } w_{\varepsilon _{n}}\Vert _{W^{1,p}(\mathbb {R}^{3})}\le C\Vert w_{\varepsilon _{n}}\Vert _{L^{p}(B_{\varrho /2}(y))}+ C_{p}\Vert \widetilde{\eta }|w_{\varepsilon _{n}}|^{2}\Vert _{L^{p}(B_{\varrho /2}(y))}. \end{aligned}$$
(4.27)

If we take \(3<p'<15/2\), then by (4.26), (4.27) and Hölder inequality, there holds

$$\begin{aligned} \Vert \widetilde{\eta } w_{\varepsilon _{n}}\Vert _{W^{1,p'}(\mathbb {R}^{3})}&\le C\Vert w_{\varepsilon _{n}}\Vert _{L^{15}(B_{\varrho /2}(y))}+ C_{p'}\Vert w_{\varepsilon _{n}}\Vert _{L^{15}(B_{\varrho /2}(y))}^{2}\\&\le C_{\varrho ,\gamma }. \end{aligned}$$

By Sobolev embedding theorem, \(W^{1,p'}(\mathbb {R}^{3})\hookrightarrow C^{0}(\mathbb {R}^{3})\) is continuous. Therefore, we have \(\Vert \widetilde{\eta } w_{\varepsilon _{n}}\Vert _{L^{\infty }}(\mathbb {R}^{3})\le C_{\varrho ,\gamma }\), i.e., \(\Vert w_{\varepsilon _{n}}\Vert _{L^{\infty }(B_{\varrho /4}(y))}\le C_{\varrho ,\gamma }\). By the arbitrariness of y, there holds

$$\begin{aligned} \Vert w_{\varepsilon _{n}}\Vert _{L^{\infty }(\mathbb {R}^{3},\mathbb {C}^{4})}\le C_{\varrho ,\gamma }. \end{aligned}$$

This contradicts \(\sup _{x\in \mathbb {R}^{3}}|w_{\varepsilon _{n}}(x)|\rightarrow \infty ~~\text {as }n\rightarrow \infty .\) Consequently, there exists a constant \(M>0\) such that \(\sup _{x\in \mathbb {R}^{3}}|w_{\varepsilon _{n}}(x)|\le M\). The proof is completed. \(\square \)

5 Proof of Theorem 1.1

Proof of Theorem 1.1

By Lemma 4.4, we know that there exists a \(\varepsilon _{0}>0\), such that for any \(0<\varepsilon <\varepsilon _{0}\), \(|w_{\varepsilon }(x)|\le M\). Recalling the definition of the \(b_{\varepsilon }(t)\) and \(m_{\varepsilon }(t)\), it is clear that

$$\begin{aligned} m_{\varepsilon }(|w_{\varepsilon }|^{2})=\int _{0}^{|w_{\varepsilon }|^{2}}b_{\varepsilon }(s){\text{ d }}s=|w_{\varepsilon }|^{2},~~ b_{\varepsilon }(|w_{\varepsilon }|^{2})=\varphi (\varepsilon |w_{\varepsilon }|^{2})=1, \end{aligned}$$

then we deduce that

$$\begin{aligned} h_{\varepsilon }(x,|w_{\varepsilon }|)&=K(\varepsilon x)|w_{\varepsilon }|^{p-2}+\frac{p}{3}Q(\varepsilon x)|w_{\varepsilon }|^{p-2}\left( |w_{\varepsilon }|^{2}\right) ^{\frac{3-p}{2}}\\&\quad +\frac{3-p}{3}Q(\varepsilon x)|w_{\varepsilon }|^{p}(|w_{\varepsilon }|^{2})^{\frac{3-p}{2}-1}\\&=K(\varepsilon x)|w_{\varepsilon }|^{p-2}+Q(\varepsilon x)|w_{\varepsilon }|. \end{aligned}$$

Using this and the definition of \(f_{\varepsilon }\), we obtain

$$\begin{aligned}&f_{\varepsilon }(x,|w_{\varepsilon }|)=\chi (\varepsilon x)(K(\varepsilon x)|w_{\varepsilon }|^{p-2}+Q(\varepsilon x)|w_{\varepsilon }|)+(1-\chi (\varepsilon x))g_{\varepsilon }(x,|w_{\varepsilon }|). \end{aligned}$$

By Lemma 4.3 and Lemma 4.1 (i), we know that for any sequence of solutions \(\{w_{\varepsilon _{n}}\}\) will not concentrate at a single point, then we can treat the situation as the subcritical equations like [32]. By the similar argument as [32, Lemma 4.6, Proposition 5.2], we can get

$$\begin{aligned} |w_{\varepsilon }|\le C_{1} \exp \left( -C_{2}\left( \frac{\text {dist}(x,\mathcal {O}(\delta ))}{\varepsilon }\right) ^{\frac{2-\tau }{2}}\right) , \end{aligned}$$
(5.1)

where \(C_{1}\), \(C_{2}\) are positive constants. Then, by choose \(\kappa \) large enough, we have

$$\begin{aligned} g_{\varepsilon }(x,|w_{\varepsilon }|)&=\min \left\{ K(\varepsilon x)|w_{\varepsilon }|^{p-2}+Q(\varepsilon x)|w_{\varepsilon }|,\frac{\kappa }{1+|x|^{\tau '+4}}\right\} \\&=K(\varepsilon x)|w_{\varepsilon }|^{p-2}+Q(\varepsilon x)|w_{\varepsilon }|. \end{aligned}$$

Therefore, \(f_{\varepsilon }(x,|w_{\varepsilon }|)=K(\varepsilon x)|w_{\varepsilon }|^{p-2}+Q(\varepsilon x)|w_{\varepsilon }|\). Then (3.1) can be rewritten as follows

$$\begin{aligned}&-i\alpha \cdot \nabla w_{\varepsilon }+a\beta w_{\varepsilon }+V(\varepsilon x)w_{\varepsilon }-\frac{a}{8}\widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w_{\varepsilon }|)w_{\varepsilon }\\&\quad =K(\varepsilon x)|w_{\varepsilon }|^{p-2}w_{\varepsilon }+Q(\varepsilon x)|w_{\varepsilon }|w_{\varepsilon }. \end{aligned}$$

By the definition of \(\widetilde{\chi }\), \(\widetilde{\xi }\) and (5.1), it is not difficult to know that

$$\begin{aligned} \widetilde{\chi }(\varepsilon x)\widetilde{\xi }(x,|w_{\varepsilon }|)=0. \end{aligned}$$

Then

$$\begin{aligned} -i\alpha \cdot \nabla w_{\varepsilon }+a\beta w_{\varepsilon }+V(\varepsilon x)w_{\varepsilon }=K(\varepsilon x)|w_{\varepsilon }|^{p-2}w_{\varepsilon }+Q(\varepsilon x)|w_{\varepsilon }|w_{\varepsilon }. \end{aligned}$$

This means that we can obtain the desire result and the proof of Theorem 1.1 is completed. \(\square \)