1 Introduction and main results

In this paper, we consider the following Hamiltonian elliptic system:

$$ \left \{ \begin{array}{@{}l} -\Delta u+V(x)u=H_{v}(x,u,v),\quad x\in\mathbb{R}^{N},\\ -\Delta v+V(x)v=H_{u}(x,u,v),\quad x\in\mathbb{R}^{N},\\ u(x)\to0,\qquad v(x)\to0 \quad\mbox{as } |x|\to\infty, \end{array} \right . $$
(1.1)

where \(z=(u,v):\mathbb{R}^{N}\to\mathbb{R} \times \mathbb{R}\), \(N\ge3\), \(V\in C(\mathbb{R}^{N},\mathbb{R})\), and \(H \in C^{1}(\mathbb{R}^{N}\times\mathbb{R}^{2},\mathbb{R})\).

For the case of a bounded domain these systems were studied by a number of authors. For instance, see [16] and the references therein. The problem (1.1) or similar to (1.1) in the whole space \(\mathbb{R}^{N}\) was considered recently in some works. For instance, see [730] and the references therein. Most of them focused on the case \(V \equiv1\). The main difficulty of this problem is the lack of compactness for Sobolev’s embedding theorem. A usual way to overcome this difficulty is working on the radically symmetric function space which possesses compact embedding. By the means, De Figueiredo and Yang [7] obtained a positive radially symmetric solution which decays exponentially to 0 at infinity. Their results were generalized by Sirakov [13] in a different way. Later, Bartsch and De Figueiredo [9] proved that the system admits infinitely many radial as well as non-radial solutions. By a generalized linking theorem, Li and Yang [8] proved the system has a positive ground state solution for \(V=1\) and an asymptotically quadratic nonlinearity. Another usual way is avoiding the indefinite character of the original functional by using the dual variational method; see for instance [1012, 14].

Recently, problem (1.1) with general periodic potential was considered in [1518, 20, 21, 23, 24, 28, 30]. By applying a generalized linking theorem for the strongly indefinite functionals developed recently by Bartsch and Ding [31] (also see [32, 33]), the authors obtained the infinitely many geometrically distinct solutions under different assumptions. For the non-periodic case, see [19, 22, 26, 27]. Moreover, for the semiclassical solutions’ problem, we refer the readers to [25, 29, 34, 35] and the references therein.

Motivated by these works, we continue to consider the superlinear periodic case. In a recent paper [23], Zhang et al. have studied this case and obtained infinitely many geometrically distinct solutions. However, we do not know if these solutions are large energy solutions. Here we call that a sequence \(\{z_{k}\}\) of solutions of the problem (1.1) is large energy solutions if the corresponding energy \(\Phi(z_{k})\rightarrow\infty\) as \(k\rightarrow\infty\). In this paper, our aim is to establish infinitely many large energy solutions by the generalized variant fountain theorem developed recently by Batkam and Colin [36]. To the best of our knowledge, there is no work focused on this case. Hence our result is new and different from the one in [23].

More precisely, we make the following assumptions:

(V):

\(V\in C(\mathbb{R}^{N}, \mathbb{R})\) is 1-periodic in x, and 0 lies in a gap of the spectrum of \(-\Delta+V\);

(H1):

\(H\in C^{1}(\mathbb{R}^{N}\times\mathbb{R}^{2}, [0,\infty))\) is 1-periodic in x, and there is a constant \(C>0\) such that

$$ \bigl|H_{z}(x,z)\bigr|\leq C \bigl(1+\xi(x)|z|^{p-1} \bigr),\quad p \in \bigl(2, 2^{*} \bigr), $$

where \(\xi(x)>0\), \(\xi(x)\in L^{\infty}(\mathbb{R}^{N})\cap L^{r}(\mathbb{R}^{N})\), and \(\frac{1}{r}+\frac{p}{2^{*}}=1\);

(H2):

\(H_{z}(x,z)=o(|z|)\) as \(|z|\rightarrow0\), and \(\frac {H(x,z)}{|z|^{2}}\rightarrow\infty\) as \(|z|\rightarrow\infty\) uniformly in x;

(H3):

there exists a function \(W(x)\in L^{1}(\mathbb {R}^{N})\) such that

$$ H(x,z+w)-H(x,z)-r \bigl(H_{z}(x,z),w \bigr)+\frac{(r-1)^{2}}{2} \bigl(H_{z}(x,z),z \bigr)\geq-W(x), $$

\(\forall x\in\mathbb{R}^{N}\), \(z,w\in\mathbb{R}^{2}\) and \(r\in[0,1]\);

(H4):

\(H(x,z)=H(x,-z)\), \(\forall(x,z)\in\mathbb{R}^{N}\times \mathbb{R}^{2}\).

The main result of this paper is the following theorem.

Theorem 1.1

Suppose that (V) and (H1)-(H4) are satisfied. Then problem (1.1) has infinitely many large energy solutions.

The single Schrödinger equation

$$ \left \{ \begin{array}{@{}l} -\Delta u+V(x)u=f(x,u) \quad\mbox{in } \mathbb{R}^{N},\\ u(x)\to0, \quad\mbox{as } |x|\to\infty, \end{array} \right . $$
(1.2)

has been studied recently in Batkam and Colin [36]. Under the assumption (V) and the following assumptions:

(f1):

\(f(x,u)\) is continuous and 1-periodic in x, there is a constant \(c>0\) such that \(|f(x,u)|\leq c(1+|u|^{p-1})\), where \(p\in(2, 2^{*})\);

(f2):

\(f(x,u)=o(|u|)\) as \(|u|\rightarrow0\), and \(\frac {F(x,u)}{|u|^{\mu}}\rightarrow\infty\) as \(|u|\rightarrow\infty\) uniformly in x, where \(\mu>2\) and \(F(x,u)=\int_{0}^{u}f(x,s)\,ds\);

(f3):

\(u\rightarrow\frac{f(x,u)}{|u|}\) is strictly increasing in \(\mathbb{R}\backslash\{0\}\);

(f4):

\(f(x,u)=-f(x,-u)\), \(\forall(x,u)\in\mathbb {R}^{N}\times\mathbb{R}\);

the existence of large energy solutions is obtained for problem (1.2).

Remark 1.2

There is similarity to the condition (H3) for the semilinear Schrödinger equation which was first introduced in [37]. Namely,

  1. (S)

    there exists a function \(W(x)\in L^{1}(\mathbb{R}^{N})\) such that

    $$ 2F(x,u+v)-2F(x,u)- \bigl(2rv-(r-1)^{2}u \bigr)f(x,u) \geq-W(x), $$

    \(\forall x\in\mathbb{R}^{N}\), \(u,v\in\mathbb{R}\) and \(r\in[0,1]\).

Clearly, the condition (S) is much weaker than the condition (f3) (see [37]), and the condition (f2) implies the condition (H2) since \(\mu>2\).

From Remark 1.2 we know, on the one hand, Theorem 1.1 improves the result in [36] by weakening the corresponding condition. On the other hand, Theorem 1.1 is a generalization of the result in [36] from single elliptic equation to elliptic system. Compared to the single equation (1.2), the system (1.1) becomes more complex in nature. We must face two kinds of indefiniteness: one comes from the system itself and the other comes from each equation in the system. Moreover, there are at least three difficulties in our problem. First, there is a lack of compactness of the Sobolev embedding since the domain is the whole \(\mathbb{R}^{N}\). Second, the variational setting for our problem is more complex and different from the case where \(V=1\) since the potential V is a general periodic function. Third, the energy function is strongly indefinite and it has a more complex geometry structure than functions which have the mountain pass structure.

This paper is organized as follows. In Section 2, we formulate the variational setting, and introduce the generalized variant fountain theorem. The existence of large energy solutions is proved in Section 3.

2 Variational setting and generalized fountain theorem

Below by \(|\cdot|_{q}\) we denote the usual \(L^{q}\)-norm, \((\cdot,\cdot)_{2}\) denote the usual \(L^{2}\) inner product, c, \(c_{i}\) or \(C_{i}\) stand for different positive constants. Let X and Y be two Banach spaces with norms \(\|\cdot\|_{X}\) and \(\|\cdot\|_{Y}\). We always choose the equivalent norm \(\|(x,y)\|_{X\times Y}=(\|x\|^{2}_{X}+\|y\|^{2}_{Y})^{\frac{1}{2}}\) on the product space \(X\times Y\). In particular, if X and Y are two Hilbert spaces with inner products \((\cdot,\cdot)_{X}\) and \((\cdot,\cdot)_{Y}\), we choose the inner product \(((x,y),(w,z))=(x,w)_{X}+(y,z)_{Y}\) on the product space \(X\times Y\).

Let \(A=-\Delta+V\) and \(\{F_{\lambda}\}_{\lambda\in\mathbb{R}}\) be the spectral family of A. Assumption (V) implies an orthogonal decomposition:

$$ L^{2}:=L^{2} \bigl(\mathbb{R}^{N}, \mathbb{R} \bigr)=L^{-}\oplus L^{+},\qquad u=u^{-}+u^{+}, $$

where \(L^{-}=F_{0}L^{2}\) and \(L^{+}=(\operatorname{id}-F_{0})L^{2}\). Denoting by \(|A|\) the absolute value of A its square root operator is

$$ |A|^{1/2}=\int_{-\infty}^{+\infty}| \lambda|^{1/2}\,dF_{\lambda}: \mathcal {D} \bigl(|A|^{1/2} \bigr)\rightarrow L^{2}, $$

where

$$ \mathcal{D} \bigl(|A|^{1/2} \bigr)= \biggl\{ u\in L^{2}: \int _{-\infty}^{+\infty }| \lambda|^{1/2}\,d(F_{\lambda}u,u)_{2} <\infty \biggr\} . $$

Let \(X=\mathcal{D}(|A|^{1/2})\) be the Hilbert space with the inner product

$$ (u,v)_{X}= \bigl(|A|^{1/2}u, |A|^{1/2}v \bigr)_{2} $$

and the corresponding norm \(\|u\|_{X}=(u,v)_{X}^{1/2}\). There is an induced decomposition

$$ X=X^{-}\oplus X^{+}, \quad\mbox{where } X^{\pm}=X \cap L^{\pm}, $$

which is orthogonal with respect to the inner products \((\cdot,\cdot )_{2}\) and \((\cdot,\cdot)_{X}\).

Let \(E=X\times X\) with the inner product

$$ \bigl((u,v),(\varphi,\psi) \bigr)=(u,\varphi)_{X}+(v, \psi)_{X} $$

and the corresponding norm

$$ \bigl\| (u,v)\bigr\| = \bigl(\|u\|_{X}^{2}+\|v \|_{X}^{2} \bigr)^{1/2}. $$

Setting

$$ E^{+}=X^{+}\times X^{-},\qquad E^{-}=X^{-}\times X^{+}, $$

then for any \(z=(u,v)\in E\), we have

$$ z=z^{+}+z^{-}, \quad\mbox{where } z^{+}= \bigl(u^{+},v^{-} \bigr), z^{-}= \bigl(u^{-},v^{+} \bigr). $$

Clearly, \(E^{+}\) and \(E^{-}\) are orthogonal with respect to the products \((\cdot,\cdot)_{2}\) and \((\cdot,\cdot)\). Hence

$$ E=E^{+}\oplus E^{-}. $$

Recall that \(E\hookrightarrow L^{p}:=L^{p}(\mathbb{R}^{N},\mathbb {R}^{2})\) is continuous for \(p\in[2,2^{*}]\) and \(E\hookrightarrow L_{\mathrm{loc}}^{p}\) is compact for \(p\in[2,2^{*})\), where 2 is the Sobolev critical exponent.

In what follows, we introduce the generalized variant fountain theorem, and consider the \(C^{1}\)-functional \(\Phi_{\lambda}:E\rightarrow\mathbb{R}\) defined by

$$ \Phi_{\lambda}(u)=I(u)-\lambda J(u), \quad\lambda\in[1,2]. $$

Let \(\{e_{j}\}_{j\geq0}\) be an orthogonal basis of \(E^{+}\). We adopt the following notations:

$$ E_{k}^{-}:=E^{-}\oplus \Biggl( \bigoplus_{j=0}^{k}\mathbb{R}e_{j} \Biggr) \quad \mbox{and}\quad E_{k}^{+}:=\overline{\bigoplus_{j=k}^{\infty} \mathbb{R}e_{j}}, $$

\(B_{k}:=\{u\in E_{k}^{-}|\|u\|\leq\rho_{k}\}\), \(N_{k}:=\{u\in E_{k}^{+}|\|u\|=r_{k}\}\), where \(0< r_{k}<\rho_{k}\), \(k\geq1\), \(\Gamma _{k}(\lambda)\) is the class of maps \(\gamma: B_{k}\rightarrow E\) such that:

  1. (a)

    γ is odd and τ-continuous, and \(\gamma |_{\partial B_{k}}=\operatorname{id}\);

  2. (b)

    for every \(u\in \operatorname{int}(B_{k})\) has a τ-neighborhood \(N_{u}\) in \(E_{k}^{-}\) such that \((\operatorname{id}-\gamma)(N_{u}\cap \operatorname{int}(B_{k}))\) is contained in a finite dimensional subspace of E;

  3. (c)

    \(\Phi_{\lambda}(\gamma(u))\leq\Phi_{\lambda}(u)\), \(\forall u\in B_{k}\),

where the definition of τ-topology can be found in [32].

The following version of the fountain theorem for a strongly indefinite functional is due to Batkam and Colin [36].

Proposition 2.1

Assume that the above functional \(\Phi _{\lambda}\) satisfies

(F1):

\(\Phi_{\lambda}\) maps bounded sets to bounded sets uniformly for \(\lambda\in[1,2]\), and \(\Phi_{\lambda}(-u)=\Phi_{\lambda }(u)\) for all \((\lambda,u)\in[1,2]\times E\),

(F2):

\(J(u)\geq0\) for all \(u\in E\), and \(I(u)\rightarrow \infty\) or \(J(u)\rightarrow\infty\) as \(\|u\|\rightarrow\infty\),

(F3):

for every \(\lambda\in[1,2]\), \(\Phi_{\lambda}\) is τ-upper semicontinuous and \(\Phi_{\lambda}'\) is weakly sequentially continuous.

If there are \(0< r_{k}<\rho_{k}\) such that

$$\begin{aligned} b_{k}(\lambda):=\inf_{u\in E_{k}^{+}, \|u\|=r_{k}}\Phi_{\lambda}(u) \geq a_{k}(\lambda):=\sup_{u\in E_{k}^{-},\|u\|=\rho_{k}}\Phi_{\lambda}(u), \quad \forall\lambda\in[1,2], \end{aligned}$$

then

$$\begin{aligned} c_{k}(\lambda):=\inf_{\gamma\in\Gamma_{k}(\lambda)}\sup_{u\in B_{k}} \Phi _{\lambda} \bigl(\gamma(u) \bigr)\geq b_{k}(\lambda),\quad \forall \lambda\in[1,2]. \end{aligned}$$

Moreover, for a.e. \(\lambda\in[1,2]\) there exists a sequence \(\{ u_{k}^{n}(\lambda)\}\subset E\) such that

$$\begin{aligned} \sup_{n}\bigl\| u_{k}^{n}(\lambda)\bigr\| <\infty,\qquad \Phi_{\lambda}' \bigl(u_{k}^{n}(\lambda ) \bigr)\rightarrow0 \quad\textit{and}\quad \Phi_{\lambda} \bigl(u_{k}^{n}( \lambda) \bigr)\rightarrow c_{k}(\lambda) \quad\textit{as } n\rightarrow\infty. \end{aligned}$$

In order to apply Proposition 2.1 to prove our main result, by the assumption (V) and decomposition of E, we define the following functional on E:

$$\begin{aligned} \Phi_{\lambda}(z)=\frac{1}{2}\bigl\| z^{+}\bigr\| ^{2}- \lambda \biggl(\frac{1}{2}\bigl\| z^{-}\bigr\| ^{2}+\int _{\mathbb{R}^{N}}H(x,z) \biggr), \end{aligned}$$

and

$$\begin{aligned} I(z)=\frac{1}{2}\bigl\| z^{+}\bigr\| ^{2},\qquad J(z)=\frac{1}{2} \bigl\| z^{-}\bigr\| ^{2}+\int_{\mathbb{R}^{N}}H(x,z) \end{aligned}$$

for all \(z=z^{+}+z^{-}\in E\), \(\lambda\in[1,2]\). Clearly, \(\Phi_{\lambda }\) is strongly indefinite, and our hypotheses imply that \(\Phi_{\lambda}\in C^{1}(E, \mathbb{R})\). In particular, a standard argument shows that critical points of \(\Phi_{1}\) are solutions of problem (1.1) (see [33] and [38]).

3 Proof of theorem

Before giving the proof of the main theorem, we need some preliminary results.

Lemma 3.1

For every \(\lambda\in[1,2]\), \(\Phi_{\lambda}\) is τ-upper semicontinuous and \(\Phi_{\lambda}'\) is weakly sequentially continuous. Moreover, \(\Phi_{\lambda}\) maps bounded sets to bounded sets.

Proof

The proof was given in [33] (also see [31, 38]); here we omit the details. □

Similar to Lemma 3.8 in [38] (also see [39]), we have the following lemma, which will be helpful for our argument.

Lemma 3.2

Assume that \(\xi(x)>0\), \(\xi(x)\in L^{\infty }(\mathbb{R}^{N})\cap L^{r}(\mathbb{R}^{N})\), and \(\frac{1}{r}+\frac {p}{2^{*}}=1\). Then

$$\begin{aligned} \beta_{k}=\sup_{z\in E^{+}_{k},\|z\|=1} \biggl(\int _{\mathbb{R}^{N}}\xi (x)|z|^{p} \biggr)^{\frac{1}{p}} \rightarrow0, \quad\textit{as } k\rightarrow \infty. \end{aligned}$$
(3.1)

Proof

Clearly, \(0\leq\beta_{k+1}\leq\beta_{k}\), hence \(\beta _{k}\rightarrow\beta\geq0\). For every k, there exists \(z_{k}\in E^{+}_{k}\) such that \(\|z_{k}\|=1\) and

$$\begin{aligned} 0\leq\beta_{k}^{p}-\int_{\mathbb{R}^{N}} \xi(x)|z_{k}|^{p}<\frac{1}{k}. \end{aligned}$$

Up to a subsequence, we have \(z_{k}\rightharpoonup z\) in E. By the definition of \(E^{+}_{k}\) we have \(z=0\). Moreover, by Sobolev imbedding theorem we have \(|z_{k}|_{2^{*}}\leq C\) for some \(C>0\). Observe that \(\xi(x)\in L^{r}\) implies that for every \(\varepsilon>0\), there exists \(R>0\) such that

$$\begin{aligned} \biggl(\int_{B^{c}_{R}(0)}\bigl|\xi(x)\bigr|^{r} \biggr)^{\frac{1}{r}}<\varepsilon. \end{aligned}$$

Since \(E\hookrightarrow L_{\mathrm{loc}}^{p}\) compactly for \(p\in[2, 2^{*})\). Hence, there exists \(k_{0}>0\) such that

$$\begin{aligned} \int_{B_{R}(0)}\xi(x)|z_{k}|^{p}< \varepsilon, \quad\forall k\geq k_{0}. \end{aligned}$$

Now since \(\frac{1}{r}+\frac{p}{2^{*}}=1\), for k large enough, we deduce by using the Hölder inequality that

$$\begin{aligned} \int_{\mathbb{R}^{N}}\xi(x)|z_{k}|^{p} =&\int _{B_{R}(0)}\xi (x)|z_{k}|^{p}+\int _{B^{c}_{R}(0)}\xi(x)|z_{k}|^{p} \\ \leq&\int_{B_{R}(0)}\xi(x)|z_{k}|^{p}+ \biggl(\int_{B^{c}_{R}(0)}\bigl|\xi (x)\bigr|^{r} \biggr)^{1/r} \biggl(\int_{B^{c}_{R}(0)}|z_{k}|^{2^{*}} \biggr)^{p/2^{*}} \\ \leq& \varepsilon+\varepsilon|z_{k}|^{p}_{2^{*}} \\ \leq&\varepsilon \bigl(1+C^{p} \bigr). \end{aligned}$$

We see that the desired conclusion holds by taking the limit. □

Lemma 3.3

Let (V), (H1)-(H4) be satisfied. There exists \(r_{k}>0\) independent of \(\lambda\in[1,2]\) such that

$$\begin{aligned} b_{k}(\lambda):=\inf_{z\in E_{k}^{+}, \|z\|=r_{k}}\Phi_{\lambda}(z)>0, \end{aligned}$$

and \(b_{k}(\lambda)\rightarrow\infty\) uniformly in λ as \(k\rightarrow\infty\).

Proof

Observe that, given \(\varepsilon>0\), there is \(C_{\varepsilon}>0\) such that for all \(z\in E_{k}^{+}\)

$$\begin{aligned} \bigl|H_{z}(x,z)\bigr|\leq\varepsilon|z|+C_{\varepsilon} \xi(x)|z|^{p-1} \end{aligned}$$
(3.2)

and

$$\begin{aligned} \bigl|H(x,z)\bigr|\leq\frac{1}{2}\varepsilon|z|^{2}+ \frac{1}{p}C_{\varepsilon}\xi (x)|z|^{p}. \end{aligned}$$
(3.3)

It then follows that

$$\begin{aligned} \Phi_{\lambda}(z) \geq&\frac{1}{2}\|z\|^{2}- \frac{1}{2}\lambda \varepsilon\int_{\mathbb{R}^{N}}|z|^{2}- \frac{1}{p}\lambda C_{\varepsilon}\int_{\mathbb{R}^{N}} \xi(x)|z|^{p} \\ \geq&\frac{1}{2}\|z\|^{2}-\frac{1}{2}c\lambda\varepsilon \|z\|^{2}-\frac {1}{p}\lambda C_{\varepsilon}\int _{\mathbb{R}^{N}}\xi(x)|z|^{p}. \end{aligned}$$

By choosing \(\varepsilon=\frac{1}{2\lambda c}\), we obtain

$$\begin{aligned} \Phi_{\lambda}(z)\geq\frac{1}{4}\|z\|^{2}- \frac{1}{p}\lambda C_{\varepsilon}\int_{\mathbb{R}^{N}} \xi(x)|z|^{p}. \end{aligned}$$

By (3.1), we have

$$\begin{aligned} \Phi_{\lambda}(z)\geq\frac{1}{4}\|z\|^{2}-C_{1} \beta_{k}^{p}\|z\| ^{p}=\frac{1}{4} \biggl( \|z\|^{2}-\frac{C_{1}}{4}\beta_{k}^{p} \|z \|^{p} \biggr). \end{aligned}$$

If we set \(r_{k}=(\frac{C_{1}}{2}\beta_{k}^{p})^{\frac{1}{2-p}}\), then for every \(z\in E_{k}^{+}\) such that \(\|z\|=r_{k}\), we have

$$\begin{aligned} \Phi_{\lambda}(z)\geq\tilde{b}_{k}:=\frac{1}{8} \biggl( \frac{C_{1}}{2}\beta _{k}^{p} \biggr)^{\frac{2}{2-p}}= \frac{1}{8}r_{k}^{2}>0. \end{aligned}$$

Moreover, by (3.1) again, \(\beta_{k}\rightarrow0\) as \(k\rightarrow\infty\), we have \(\tilde{b}_{k}\rightarrow\infty\), and hence \(b_{k}(\lambda)\rightarrow\infty\) uniformly in λ as \(k\rightarrow\infty\). □

Lemma 3.4

Let (V), (H1)-(H4) be satisfied. There exists \(\rho_{k}>0\) independent of \(\lambda\in[1,2]\) such that

$$\begin{aligned} a_{k}(\lambda):=\sup_{z\in E_{k}^{-},\|z\|=\rho_{k}}\Phi_{\lambda }(z) \leq0. \end{aligned}$$

Proof

Since \(\Phi_{\lambda}\leq\Phi_{1}\) for all \(\lambda\in [1,2]\), it suffices to show the conclusion holds for \(\lambda=1\). Suppose to the contrary that there exists a sequence \(z_{n}\in E_{k}^{-}\) such that \(\Phi_{1}(z_{n})>0\) for all n and \(\|z_{n}\|\rightarrow\infty\) as \(n\rightarrow\infty\). Set \(w_{n}=\frac{z_{n}}{\|z_{n}\|}=w_{n}^{+}+w_{n}^{-}\), where \(w_{n}^{-}\in E^{-}\), \(w_{n}^{+}\in\bigoplus_{j=0}^{k}\mathbb {R}e_{j}\subset E^{+}\). Then \(1=\|w_{n}\|^{2}=\|w_{n}^{+}\|^{2}+\|w_{n}^{-}\|^{2}\) and

$$ 0<\frac{\Phi(z_{n})}{\|z_{n}\|^{2}}=\frac{1}{2} \bigl( \bigl\| w_{n}^{+} \bigr\| ^{2}-\bigl\| w_{n}^{-} \bigr\| ^{2} \bigr)-\int_{\mathbb{R}^{N}}\frac{H(x,z_{n})}{|z_{n}|^{2}}|w_{n}|^{2}. $$
(3.4)

By (H1), we have

$$ \bigl\| w_{n}^{-}\bigr\| ^{2}<\bigl\| w_{n}^{+} \bigr\| ^{2}=1-\bigl\| w_{n}^{-}\bigr\| ^{2}, $$

therefore

$$ 0\leq\bigl\| w_{n}^{-}\bigr\| ^{2}\leq \frac{1}{2} \quad\mbox{and}\quad \frac{1}{2}\leq \bigl\| w_{n}^{+} \bigr\| ^{2}\leq1. $$

Since \(\bigoplus_{j=0}^{k}\mathbb{R}e_{j}\) is finite dimensional, going to a subsequence if necessary, we may assume \(w_{n}^{+}\rightarrow w^{+}\neq0\), \(w_{n}^{-}\rightharpoonup w^{-}\), and \(w_{n}(x)\rightarrow w(x)\) a.e. on \(\mathbb{R}^{N}\). Hence \(w\neq0\) and \(|z_{n}|=\|z_{n}\||w_{n}|\rightarrow\infty\). By (H2) and Fatou’s lemma, we have

$$\begin{aligned} \int_{\mathbb{R}^{N}}\frac {H(x,z_{n})}{|z_{n}|^{2}}|w_{n}|^{2} \rightarrow\infty, \quad\mbox{as } n\rightarrow\infty. \end{aligned}$$

By (3.4), we get a contradiction. □

Combining Lemmas 3.1, 3.3, 3.4, and Proposition 2.1, we have the following lemma.

Lemma 3.5

Let (V), (H1)-(H4) be satisfied, for almost every \(\lambda\in[1,2]\), there exists a sequence \(\{ z_{k}^{n}(\lambda)\}\) such that

$$\begin{aligned} \sup_{n}\bigl\| z_{k}^{n}(\lambda)\bigr\| <\infty,\qquad \Phi'_{\lambda} \bigl(z_{k}^{n}(\lambda ) \bigr)\rightarrow0,\qquad \Phi_{\lambda} \bigl(z_{k}^{n}( \lambda) \bigr)\rightarrow c_{k}(\lambda), \quad\textit{as } n \rightarrow\infty. \end{aligned}$$

Lemma 3.6

Let (V), (H1)-(H4) be satisfied, for almost every \(\lambda\in[1,2]\), there exists some \(\{z_{k}(\lambda)\}\) such that

$$\begin{aligned} \Phi'_{\lambda} \bigl(z_{k}(\lambda) \bigr)=0, \qquad \Phi_{\lambda} \bigl(z_{k}(\lambda ) \bigr)=c_{k}( \lambda). \end{aligned}$$

Proof

Let \(\{z_{k}^{n}(\lambda)\}\) be the sequence obtained in Lemma 3.5. Here for notational simplicity, we write \(z_{k}^{n}(\lambda )=z_{k}^{n}\). Since \(\{z_{k}^{n}\}\) is bounded, without restriction we can assume that \(\{z_{k}^{n}\}\) is either vanishing or nonvanishing. If \(\{ z_{k}^{n}\}\) is vanishing, i.e.,

$$\begin{aligned} \lim_{n\rightarrow\infty}\sup_{y\in\mathbb{R^{N}}}\int _{B(y,R)}\bigl|z_{k}^{n}\bigr|^{2}=0,\quad \forall R>0, \end{aligned}$$

by Lions’ concentration compactness principle in [40] (also see [38]), \(z_{k}^{n}\rightarrow0\) in \(L^{p}\) for all \(p\in(2,2^{*})\). By using the Hölder inequality and (3.2), we have

$$\begin{aligned} \biggl\vert \int_{\mathbb{R}^{N}}H_{z} \bigl(x,z_{k}^{n} \bigr)\cdot z_{k}^{n} \biggr\vert \leq \varepsilon\int_{\mathbb{R}^{N}}\bigl|z_{k}^{n}\bigr|^{2}+C_{\varepsilon} \int_{\mathbb{R}^{N}}\bigl|z_{k}^{n}\bigr|^{p} \rightarrow0 \end{aligned}$$

as \(n\rightarrow\infty\). Therefore, we have

$$\begin{aligned} 0< b_{k}(\lambda)\leq\Phi_{\lambda} \bigl(z_{k}^{n} \bigr)\leq\bigl\| \bigl(z_{k}^{n} \bigr)^{+}\bigr\| ^{2}-\bigl\| \bigl(z_{k}^{n} \bigr)^{-} \bigr\| ^{2} =\Phi'_{\lambda} \bigl(z_{k}^{n} \bigr)z_{k}^{n}+\lambda\int_{\mathbb {R}^{N}}H_{z} \bigl(x,z_{k}^{n} \bigr)\cdot z_{k}^{n} \rightarrow0 \end{aligned}$$

as \(n\rightarrow\infty\). This implies that \(\{z_{k}^{n}\}\) is nonvanishing, i.e., there exist \(R, \delta>0\), and a sequence \(\{y_{n}\}\subset\mathbb{R}^{N}\) such that

$$\begin{aligned} \liminf_{n\rightarrow\infty}\int_{B(y_{n}, R)}\bigl|z_{k}^{n}\bigr|^{2} \geq\delta. \end{aligned}$$

Taking a subsequence if necessary we may suppose that

$$\begin{aligned} \int_{B(y_{n}, R)}\bigl|z_{k}^{n}\bigr|^{2}\geq \frac{\delta}{2} \end{aligned}$$

for all \(n\in\mathbb{N}\). Since \(\Phi_{\lambda}\) is invariant under translation, by a standard argument, we have

$$\begin{aligned} \int_{B(0, R+\frac{\sqrt{N}}{2})}\bigl|w_{k}^{n}\bigr|^{2} \geq\frac{\delta}{2}, \end{aligned}$$
(3.5)

where \(w_{k}^{n}=z_{k}^{n}(\cdot-k_{n})\). Observe that \(\|z_{k}^{n}\|=\| w_{k}^{n}\|\). Hence \(\{w_{k}^{n}\}\) is bounded, up to a subsequence we may assume

$$\begin{aligned} w_{k}^{n}\rightharpoonup z_{k} \quad \mbox{in } E,\qquad w_{k}^{n}\rightarrow z_{k} \quad\mbox{in } L^{2}_{\mathrm{loc}} \quad\mbox{and} \quad w_{k}^{n} \rightarrow z_{k} \quad\mbox{a.e. in } \mathbb{R}^{N}. \end{aligned}$$
(3.6)

In view of (3.5), we know \(z_{k}\neq0\). By Lemma 3.1, it is easy to see that \(\Phi '_{\lambda}(z_{k})=0\).

Now, let us show that

$$\begin{aligned} \int_{\mathbb{R}^{N}}H \bigl(x,w_{k}^{n}-z_{k} \bigr)\rightarrow0, \quad\mbox{as } n\rightarrow\infty. \end{aligned}$$
(3.7)

Indeed, in virtue of assumption (H1), we know that for any \(\varepsilon>0\), there exists \(R>0\) such that

$$\begin{aligned} \biggl(\int_{B^{c}_{R}(0)}\bigl|\xi(x)\bigr|^{r} \biggr)^{\frac{1}{r}}<\varepsilon. \end{aligned}$$

Thus, by the Hölder inequality and the Sobolev embedding theorem, we have

$$\begin{aligned} &\int_{\mathbb{R}^{N}}H \bigl(x,w_{k}^{n}-z_{k} \bigr)\\ &\quad\leq\int_{\mathbb {R}^{N}}\varepsilon\bigl|w_{k}^{n}-z_{k}\bigr|^{2}+C_{\varepsilon} \xi (x)\bigl|w_{k}^{n}-z_{k}\bigr|^{p} \\ &\quad=\int_{\mathbb{R}^{N}}\varepsilon\bigl|w_{k}^{n}-z_{k}\bigr|^{2}+C_{\varepsilon } \int_{B_{R}(0)}\xi(x)\bigl|w_{k}^{n}-z_{k}\bigr|^{p} +C_{\varepsilon}\int_{B^{c}_{R}(0)}\xi(x)\bigl|w_{k}^{n}-z_{k}\bigr|^{p} \\ &\quad\leq\varepsilon\bigl|w_{k}^{n}-z_{k}\bigr|_{2}^{2}+C_{\varepsilon}| \xi|_{\infty }\int_{B_{R}(0)}\bigl|w_{k}^{n}-z_{k}\bigr|^{p} +C_{\varepsilon} \biggl(\int_{B^{c}_{R}(0)}\bigl|\xi(x)\bigr|^{r} \biggr)^{\frac {1}{r}}\bigl|w_{k}^{n}-z_{k}\bigr|_{2^{*}}^{p} \\ &\quad\leq\varepsilon(C_{2}+C_{3}+C_{4}). \end{aligned}$$

Hence (3.7) holds. Similarly, we also have

$$\begin{aligned} \int_{\mathbb {R}^{N}}H_{z} \bigl(x,w_{k}^{n}-z_{k} \bigr) \bigl(w_{k}^{n}-z_{k} \bigr)\rightarrow0, \quad \mbox{as } n\rightarrow\infty. \end{aligned}$$
(3.8)

Moreover, by (3.7) and (3.8), it is easy to show that

$$\begin{aligned} \int_{\mathbb{R}^{N}}H \bigl(x,w_{k}^{n} \bigr)\rightarrow\int_{\mathbb {R}^{N}}H(x,z_{k}), \quad \mbox{as } n \rightarrow\infty \end{aligned}$$
(3.9)

and

$$\begin{aligned} \int_{\mathbb{R}^{N}}H_{z} \bigl(x,w_{k}^{n} \bigr)w_{k}^{n} \rightarrow\int_{\mathbb {R}^{N}}H_{z}(x,z_{k})z_{k}, \quad\mbox{as } n\rightarrow\infty. \end{aligned}$$
(3.10)

Therefore, by (3.9) and (3.10) we obtain

$$\begin{aligned} \Phi_{\lambda} \bigl(w_{k}^{n} \bigr) =& \frac{1}{2} \bigl\langle \Phi'_{\lambda } \bigl(w_{k}^{n} \bigr),w_{k}^{n} \bigr\rangle +\frac{\lambda}{2}\int_{\mathbb {R}^{N}}H_{z} \bigl(x,w_{k}^{n} \bigr)w_{k}^{n} - \lambda\int_{\mathbb{R}^{N}}H \bigl(x,w_{k}^{n} \bigr) \\ \rightarrow&\frac{1}{2} \bigl\langle \Phi'_{\lambda}(z_{k}),z_{k} \bigr\rangle +\frac {\lambda}{2}\int_{\mathbb{R}^{N}}H_{z}(x,z_{k})z_{k} -\lambda\int_{\mathbb{R}^{N}}H(x,z_{k}) \\ =&\Phi_{\lambda}(z_{k}), \end{aligned}$$

which implies \(\Phi_{\lambda}(z_{k})=c_{k}(\lambda)\). The proof is complete. □

By the preceding lemma, we directly obtain the following lemma.

Lemma 3.7

Let (V), (H1)-(H4) be satisfied, there exist a sequence \(\{\lambda_{n}\}\) and a sequence \(\{z_{k}(\lambda _{n})\}\) such that

$$\begin{aligned} \lambda_{n}\rightarrow1, \quad\Phi_{\lambda_{n}} \bigl(z_{k}( \lambda _{n}) \bigr)=c_{k}(\lambda_{n}),\qquad \Phi'_{\lambda_{n}} \bigl(z_{k}(\lambda_{n}) \bigr)=0. \end{aligned}$$

Lemma 3.8

Let (H4) be satisfied, then

$$\begin{aligned} \int_{\mathbb{R}^{N}} \biggl(H(x,z)-H(x,r\phi)+r^{2} \bigl(H_{z}(x,z),\phi \bigr)-\frac{1+r^{2}}{2} \bigl(H_{z}(x,z),z \bigr) \biggr)\leq C, \end{aligned}$$

where \(z\in E\), \(\phi\in E^{+}\), \(0\leq r\leq1\), and the constant \(C:=\int_{\mathbb{R}^{N}}|W(x)|\) does not depend on z, ϕ, r.

Proof

This follows from (H4) if we take \(z=z\) and \(w=r\phi-z\). □

Lemma 3.9

Let (V), (H1)-(H4) be satisfied. The sequences \(\{z_{k}(\lambda_{n})\}\) given in Lemma  3.7 are bounded.

Proof

For notational simplicity, we write \(z_{k}^{n}:=z_{k}(\lambda_{n})\). First, we claim that there is a constant C independent of \(z_{k}^{n}\) and \(\lambda_{n}\) such that

$$\begin{aligned} \Phi_{\lambda_{n}} \bigl(r \bigl(z_{k}^{n} \bigr)^{+} \bigr)-\Phi_{\lambda_{n}} \bigl(z_{k}^{n} \bigr)\leq C,\quad \forall r\in[0,1]. \end{aligned}$$
(3.11)

Since

$$\begin{aligned} \bigl\langle \Phi'_{\lambda_{n}} \bigl(z_{k}^{n} \bigr),\varphi \bigr\rangle = \bigl( \bigl(z_{k}^{n} \bigr)^{+},\varphi^{+} \bigr)-\lambda_{n} \bigl( \bigl(z_{k}^{n} \bigr)^{-},\varphi^{-} \bigr) -\lambda_{n}\int_{\mathbb{R}^{N}}H_{z} \bigl(x,z_{k}^{n} \bigr)\varphi=0,\quad \forall \varphi\in E, \end{aligned}$$

it follows from the definition of \(\Phi_{\lambda}\) that

$$\begin{aligned} \Phi_{\lambda_{n}} \bigl(r \bigl(z_{k}^{n} \bigr)^{+} \bigr)-\Phi_{\lambda _{n}} \bigl(z_{k}^{n} \bigr) =&\frac{1}{2} \bigl(r^{2}-1 \bigr)\bigl\| \bigl(z_{k}^{n} \bigr)^{+}\bigr\| ^{2}+\frac {1}{2}\lambda_{n}\bigl\| \bigl(z_{k}^{n} \bigr)^{-}\bigr\| ^{2} \\ &{}+\lambda_{n}\int_{\mathbb{R}^{N}}H \bigl(x,z_{k}^{n} \bigr)-\lambda_{n}\int_{\mathbb{R}^{N}}H \bigl(x,r \bigl(z_{k}^{n} \bigr)^{+} \bigr) \\ &{}+\frac{1}{2} \bigl( \bigl(z_{k}^{n} \bigr)^{+},\varphi^{+} \bigr)-\frac{1}{2}\lambda _{n} \bigl( \bigl(z_{k}^{n} \bigr)^{-}, \varphi^{-} \bigr) -\frac{1}{2} \lambda_{n}\int_{\mathbb{R}^{N}}H_{z} \bigl(x,z_{k}^{n} \bigr)\varphi. \end{aligned}$$

Take

$$\begin{aligned} \varphi = \bigl(r^{2}+1 \bigr) \bigl(z_{k}^{n} \bigr)^{-}- \bigl(r^{2}-1 \bigr) \bigl(z_{k}^{n} \bigr)^{+}= \bigl(r^{2}+1 \bigr)z_{k}^{n}-2r^{2} \bigl(z_{k}^{n} \bigr)^{+}, \end{aligned}$$

which together with Lemma 3.8 implies that

$$\begin{aligned} \Phi_{\lambda_{n}} \bigl(r \bigl(z_{k}^{n} \bigr)^{+} \bigr)-\Phi_{\lambda _{n}} \bigl(z_{k}^{n} \bigr) =&-\frac{r^{2}}{2}\lambda_{n}\bigl\| \bigl(z_{k}^{n} \bigr)^{-}\bigr\| ^{2} +\lambda_{n}\int _{\mathbb{R}^{N}} \bigl(H \bigl(x,z_{k}^{n} \bigr)-H \bigl(x,r \bigl(z_{k}^{n} \bigr)^{+} \bigr) \bigr) \\ &{}+\lambda_{n}\int_{\mathbb{R}^{N}} \biggl(r^{2} \bigl(H_{z} \bigl(x,z_{k}^{n} \bigr), \bigl(z_{k}^{n} \bigr)^{+} \bigr)- \frac {1+r^{2}}{2} \bigl(H_{z} \bigl(x,z_{k}^{n} \bigr),z_{k}^{n} \bigr) \biggr) \\ \leq& C. \end{aligned}$$

Hence (3.11) holds.

To show the boundness of \(\{z_{k}^{n}\}\), we argue by contradiction that \(\|z_{k}^{n}\|\rightarrow\infty\) as \(n\rightarrow\infty\). Since \(\Phi_{\lambda}\geq0\), we know \(\|(z_{k}^{n})^{+}\|^{2}\geq\| (z_{k}^{n})^{-}\|^{2}\). Let \(w_{k}^{n}=z_{k}^{n}/\|z_{k}^{n}\|\), then \(w_{k}^{n}=(w_{k}^{n})^{-}+(w_{k}^{n})^{+}\) and \(\frac{1}{2}\leq\| (w_{k}^{n})^{+}\|^{2}\leq1\). Passing to a subsequence, \((w_{k}^{n})^{+}\rightharpoonup w_{k}^{+}\) in E, \((w_{k}^{n})^{+}\rightarrow w_{k}^{+}\) in \(L^{2}_{\mathrm{loc}}\), and \((w_{k}^{n})^{+}\rightarrow w_{k}^{+}\) a.e. on \(\mathbb{R}^{N}\). We see that either \(\{(w_{k}^{n})^{+}\}\) is vanishing, i.e.,

$$\begin{aligned} \lim_{n\rightarrow\infty}\sup_{y\in\mathbb{R}^{N}}\int _{B(y,R)}\bigl| \bigl(w_{k}^{n} \bigr)^{+}\bigr|^{2}=0, \quad\forall R>0, \end{aligned}$$

or nonvanishing, i.e., there exist \(R, \delta>0\) and a sequence \(\{y_{n}\}\subset\mathbb{R}^{N}\) such that

$$\begin{aligned} \liminf_{n\rightarrow\infty}\int_{B(y_{n},R)}\bigl| \bigl(w_{k}^{n} \bigr)^{+}\bigr|^{2}\geq \delta. \end{aligned}$$

If \((w_{k}^{n})^{+}\) is vanishing, Lions’ concentration compactness principle implies \((w_{k}^{n})^{+}\rightarrow0\) in \(L^{p}\) for \(p\in (2,2^{*})\). Lebesgue’s dominated convergence theorem and (3.3) imply that

$$\begin{aligned} \int_{\mathbb{R}^{N}}H \bigl(x,s \bigl(w_{k}^{n} \bigr)^{+} \bigr)\rightarrow0, \quad\mbox{for any } s\in\mathbb{R}. \end{aligned}$$
(3.12)

Let \(r_{n}=\frac{s}{\|z_{k}^{n}\|}\rightarrow0\) as \(n\rightarrow\infty \). Therefore, by (3.11) and (3.12), we have

$$\begin{aligned} c_{k}(\lambda_{n})+C =&C+\Phi_{\lambda_{n}} \bigl(z_{k}^{n} \bigr)\geq\Phi_{\lambda _{n}} \bigl(r_{n} \bigl(z_{k}^{n} \bigr)^{+} \bigr) \\ =&\frac{s^{2}}{2}\bigl\| \bigl(w_{k}^{n} \bigr)^{+}\bigr\| ^{2}-\lambda_{n}\int _{\mathbb {R}^{N}}H \bigl(x,s \bigl(w_{k}^{n} \bigr)^{+} \bigr) \\ \geq&\frac{s^{2}}{4}+o(1), \end{aligned}$$

which implies a contradiction if s is large enough. Hence \((w_{k}^{n})^{+}\) is nonvanishing, i.e., there exist \(R, \delta >0\) and a sequence \(\{y_{n}\}\subset\mathbb{R}^{N}\) such that

$$\begin{aligned} \liminf_{n\rightarrow\infty}\int_{B(y_{n},R)}\bigl| \bigl(w_{k}^{n} \bigr)^{+}\bigr|^{2}\geq \delta. \end{aligned}$$

Going to a subsequence if necessary, we have

$$\begin{aligned} \int_{B(y_{n},R)}\bigl| \bigl(w_{k}^{n} \bigr)^{+}\bigr|^{2}\geq\frac{\delta}{2}, \quad\mbox{for all } n \in \mathbb{N}. \end{aligned}$$

By a standard argument, we have

$$\begin{aligned} \int_{B(0, R+\frac{\sqrt{N}}{2})}\bigl| \bigl(u_{k}^{n} \bigr)^{+}\bigr|^{2}\geq\frac{\delta}{2}, \end{aligned}$$
(3.13)

where \((u_{k}^{n})^{+}=(w_{k}^{n})^{+}(\cdot-k_{n})\). Thus (3.13) implies that \(u_{k}^{+}\neq0\) and \(|z_{k}^{n}|\rightarrow\infty\). It follows from (H2) and Fatou’s lemma that

$$\begin{aligned} \int_{\mathbb{R}^{N}}\frac {H(x,z_{k}^{n})}{|z_{k}^{n}|^{2}}\bigl|w_{k}^{n}\bigr|^{2} \rightarrow\infty, \quad\mbox{as } n\rightarrow\infty, \end{aligned}$$

and

$$\begin{aligned} 0\leq\frac{\Phi_{\lambda_{n}}(z_{k}^{n})}{\|z_{k}^{n}\|^{2}}=\frac {1}{2}\bigl\| \bigl(w_{k}^{n} \bigr)^{+}\bigr\| ^{2}-\lambda_{n} \biggl( \frac{1}{2}\bigl\| \bigl(w_{k}^{n} \bigr)^{-} \bigr\| ^{2} +\int_{\mathbb{R}^{N}}\frac {H(x,z_{k}^{n})}{|z_{k}^{n}|^{2}}\bigl|w_{k}^{n}\bigr|^{2} \biggr)\rightarrow -\infty, \end{aligned}$$

as \(n\rightarrow\infty\), a contradiction. Therefore, \(\{z_{k}^{n}\}\) is bounded. □

Proof of Theorem 1.1

Clearly, the condition (F2) holds and \(\Phi_{\lambda}(-z)=\Phi_{\lambda}(z)\) for all \((\lambda,z)\in [1,2]\times E\). Lemma 3.1 implies that the conditions (F1) and (F3) hold. Lemmas 3.3 and 3.4 show that \(\Phi_{\lambda}\) possesses the geometric property of Proposition 2.1. Therefore, we can obtain the sequence \(\{z_{k}^{n}\} \) given in Lemma 3.7 by Proposition 2.1. From the relations

$$\begin{aligned} \Phi_{1} \bigl(z_{k}^{n} \bigr)= \Phi_{\lambda_{n}} \bigl(z_{k}^{n} \bigr)+( \lambda_{n}-1) \biggl(\frac{1}{2}\bigl\| \bigl(z_{k}^{n} \bigr)^{-}\bigr\| ^{2}+\int_{\mathbb {R}^{N}}H \bigl(x,z_{k}^{n} \bigr) \biggr) \end{aligned}$$

and

$$\begin{aligned} \bigl\langle \Phi'_{1} \bigl(z_{k}^{n} \bigr)-\Phi'_{\lambda_{n}} \bigl(z_{k}^{n} \bigr),\varphi \bigr\rangle = (\lambda_{n}-1) \biggl( \bigl( \bigl(z_{k}^{n} \bigr)^{-},\varphi \bigr)+\int _{\mathbb {R}^{N}}H_{z} \bigl(x,z_{k}^{n} \bigr)\varphi \biggr), \quad\forall\varphi\in E, \end{aligned}$$

we deduce, since the sequence \(\{c_{k}(\lambda_{n})\}\) is nondecreasing and bounded from above, that \(\{z_{k}^{n}\}\) is a \((PS)\)-sequence for \(\Phi_{1}\) at level \(c_{k}(1)\). By repeating the argument of Lemma 3.6 we obtain the existence of \(z_{k}\in E\) such that \(\Phi'_{1}(z_{k})=0\) and \(\Phi_{1}(z_{k})\geq b_{k}\), moreover, \(b_{k}\rightarrow\infty\) as \(k\rightarrow\infty\). This ends the proof of Theorem 1.1. □