Ground state solutions for Hamiltonian elliptic systems with super or asymptotically quadratic nonlinearity

Abstract

This article concerns the Hamiltonian elliptic system:

$$ \textstyle\begin{cases} -\Delta \varphi +V(x)\varphi =G_{\psi }(x,\varphi ,\psi ) & \mbox{in } \mathbb {R}^{N}, \\ -\Delta \psi +V(x)\psi =G_{\varphi }(x,\varphi ,\psi ) & \mbox{in } \mathbb {R}^{N}, \\ \varphi , \psi \in H^{1}(\mathbb {R}^{N}). \end{cases} $$

Assuming that the potential V is periodic and 0 lies in a spectral gap of $\sigma (-\Delta +V)$, least energy solution of the system is obtained for the super-quadratic case with a new technical condition, and the existence of ground state solutions of Nehari–Pankov type is established for the asymptotically quadratic case. The results obtained in the paper generalize and improve related ones in the literature.

1 Introduction and main results

Consider the following Hamiltonian elliptic system:

$$ \textstyle\begin{cases} -\Delta \varphi +V(x)\varphi =G_{\psi }(x,\varphi ,\psi )& \mbox{in } \mathbb {R}^{N}, \\ -\Delta \psi +V(x)\psi =G_{\varphi }(x,\varphi ,\psi )& \mbox{in } \mathbb {R}^{N}, \\ \varphi , \psi \in H^{1}(\mathbb {R}^{N}), \end{cases} $$

(1.1)

where $\varphi ,\psi :\mathbb {R}^{N}\to \mathbb {R}$, $V\in C(\mathbb {R}^{N},\mathbb {R})$, and $G\in C^{1}(\mathbb {R}^{N}\times \mathbb {R}^{2},\mathbb {R})$ with gradient $\nabla G=(G _{\varphi },G_{\psi })$. Problem (1.1) or similar to (1.1) has been extensively investigated in the literature based on various assumptions on the potential $V(x)$ and the nonlinearity $G(x,\eta )$ with $\eta =(\varphi ,\psi )$.

For the case of a bounded domain, assuming moreover $V(x)\equiv 0$, systems like or similar to (1.1) have been studied by some authors, see [7] for sublinear systems, [8, 9, 17] for the superlinear case, [18] for asymptotically linear systems, and [28] for a singularly perturbed problem. Recently, an Orlitz space approach was used by De Figueiredo et al. [11] dealing the system with $G(x,\varphi ,\psi )=F(\varphi )+H( \psi )$.

In [44], the authors studied ground state solutions for a Hamiltonian elliptic system with inverse square potential, and there are other papers concerned with the system in the whole space $\mathbb {R}^{N}$, see [10, 21,22,23, 31, 35, 38,39,40, 45,46,47,48] and the references therein. The main difficulty of such a problem is lack of the compactness of Sobolev embedding. The usual way to overcome this difficulty is working on the radially symmetric function space, which possesses compact embedding. Using spectral family theory of non-compact operator, De Figueiredo and Yang [10] proved the existence of a positive radially symmetric solution which decays exponentially to 0 at infinity for the case that $V\equiv 1$, see also [39] for the case that $G(x,\varphi ,\psi )=F(x,\varphi )+H(x, \psi )$ is asymptotically autonomous. Later, this result was generalized by Sirakov [31] in a different way. Infinitely many radial as well as non-radial solutions were obtained by Bartsch and De Figueiredo [4] provided that $G(x,\eta )$ is even in η. In [21], Li and Yang proved, via a generalized linking theorem, that (1.1) has a positive ground state solution with $V\equiv 1$ and asymptotically autonomous quadratic nonlinearity, see also [48] for a super-quadratic case. The case of general periodic potential was considered in [38, 47] for super-quadratic systems with $G(x,\varphi ,\psi )=F(x,\varphi )+H(x, \psi )$ and [45, 46] for asymptotically quadratic systems. Assuming that V is periodic and positive, nontrivial solutions were obtained by Liao et al. in the recent paper [22] under some new super-quadratic conditions. The nonperiodic and asymptotically quadratic case was studied in [36, 41, 42]. For semiclassical problems with subcritical and critical nonlinearities, we refer to [14, 22].

Here, we mention the dual variational method, another usual way to avoid the indefinite character of the original functional. See, for instance, Ávila and Yang [2, 3], Alves et al. [1], Yang [39], and the references therein.

As pointed out in [40, 46, 47], besides the lack of the compactness of Sobolev embedding, there are two kinds of indefiniteness we have to face: one comes from the system itself and the other comes from each equation in the system. Although a proper variational setting for (1.1) can be established, the functional associated with it is strongly indefinite. Therefore the periodic assumptions on V and G are usually needed, see, for instance, [19, 33,34,35, 40, 43, 45,46,47]. Applying the critical point theory developed by Bartsch and Ding [5], Zhao et al. [46] obtained a least energy solution for asymptotically quadratic system (1.1), i.e., a nontrivial solution $\bar{\eta }\in E$ satisfying $I(\bar{\eta })=\inf_{\mathcal{M}} I$, where

$$ {\mathcal{M}}:=\bigl\{ \eta \in E\setminus \bigl\{ (0,0)\bigr\} : I'(\eta )=0\bigr\} , $$

(1.2)

$E=E^{-}\oplus E^{+}$ is the working space on which the energy functional I associated with (1.1) is defined, see (2.12). For the super-quadratic case, a least energy solution of (1.1) was also established by Zhang et al. [40] via the critical point theory constructed in [5]. Based on the work [34], Tang [35] developed a direct approach and found a ground state solution of Nehari–Pankov type for (1.1), i.e., a nontrivial solution $\bar{\eta }\in E$ satisfying $I(\bar{\eta })= \inf_{\mathcal{N}^{-}} I$, where

$$ \mathcal{N}^{-}=\bigl\{ \eta \in E\setminus E^{-}: \bigl\langle I'(\eta ), \eta \bigr\rangle =\bigl\langle I'(\eta ),w\bigr\rangle =0, \forall w\in E^{-}\bigr\} . $$

(1.3)

The main ingredients in this approach are the observations that, for any $\eta \in E\setminus E^{-}$, there exist $t=t(\eta )>0$ and $w=w(\eta )\in E^{-}$ such that $w+t\eta \in \mathcal{N}^{-}$; moreover, a minimizing Cerami sequence for the energy functional I can be found outside $\mathcal{N}^{-}$ by using the diagonal method. The set $\mathcal{N}^{-}$ first introduced by Pankov [27] is a subset of the Nehari manifold

$$ \mathcal{N} = \bigl\{ \eta \in E\setminus \bigl\{ (0,0)\bigr\} : \bigl\langle I'( \eta ), \eta \bigr\rangle =0 \bigr\} . $$

(1.4)

Note that, if $\eta \neq (0,0)$ and $I'(\eta )=0$, then $\eta \in \mathcal{N}^{-}$. Hence $\mathcal{N}^{-}$ contains all nontrivial critical points of I, i.e., $\mathcal{M}$ is a very small subset of $\mathcal{N}^{-}$. In general, it is more difficult to find a solution η̄ for (1.1) that satisfies $I(\bar{\eta })= \inf_{\mathcal{N}^{-}} I$ than the one satisfying $I(\bar{\eta })= \inf_{\mathcal{M}} I$.

Let

$$ \Lambda := \max \{ -\underline{\Lambda },\overline{\Lambda } \} ,\qquad \Lambda _{0}=\min \{ -\underline{\Lambda },\overline{\Lambda } \}. $$

(1.5)

It follows from (1.6) that $\Lambda _{0}>0$. The following assumptions were introduced in [40, 46] for super and asymptotically quadratic system (1.1):

(V):

$V\in C ({\mathbb {R}}^{N} )$ are 1-periodic in $x_{i}$, $i=1, 2, \ldots , N$, and 0 lies in a gap of the spectrum of $-\Delta +V$, i.e.,

$$ \sup \bigl[\sigma (-\triangle +V)\cap (-\infty , 0)\bigr]:= \underline{\Lambda }< 0< \overline{ \Lambda }:=\inf \bigl[\sigma (-\triangle +V)\cap (0, \infty )\bigr]; $$

(1.6)

(G1):

$G\in C^{1}({\mathbb {R}}^{N}\times {\mathbb {R}}^{2},[0,\infty ))$ is 1-periodic in $x_{i}$, $i=1, 2, \ldots , N$, and $|G_{\eta }(x,\eta )|=o(| \eta |)$ as $|\eta |\to 0$ uniformly in x, where $G_{\eta }(x,\eta )=\nabla G(x,\eta )$;

(SQ):

$\lim_{|\eta |\to \infty }\frac{G(x,\eta )}{|\eta |^{2}}= \infty$ uniformly in $x\in \mathbb {R}^{N}$;

(DL):

$\hat{{G}}(x,\eta ):=\frac{1}{2}G_{\eta }(x,\eta )\cdot \eta -G(x,\eta )>0$ if $\eta \neq (0,0)$, and there exist $r_{0}>0$, $c_{0}>0$, and $\sigma >\max \{1,N/2\}$ such that

$$ \bigl\vert G_{\eta }(x,\eta ) \bigr\vert ^{\sigma }\le c_{0}\hat{{G}}(x,\eta ) \vert \eta \vert ^{ \sigma },\quad \forall (x,\eta )\in {\mathbb {R}}^{N}\times {\mathbb {R}}^{2}, \vert \eta \vert \ge r_{0}, $$

where and in the sequel, the dot denotes the inner product in $\mathbb {R}^{2}$;

(G₂):

$G_{\eta }(x, \eta )=V_{\infty }(x)\eta +\bar{G}_{\eta }(x, \eta )$ with $\bar{G}_{\eta }(x,\eta )=\nabla \bar{G}(x, \eta )$, where $V_{\infty }\in C(\mathbb {R}^{N})$ is 1-periodic in each of $x_{1}, x_{2}, \ldots , x_{N}$, $\inf V_{\infty }> \Lambda$, and $|\bar{G}_{\eta }(x, \eta )|=o(|\eta |)$ as $|\eta |\to \infty$ uniformly in $x\in \mathbb {R}^{N}$;

(G₃):

$\hat{{G}}(x,\eta )>0$ if $\eta \neq (0,0)$, and $\hat{{G}}(x,\eta )\to \infty$ as $|\eta |\to \infty$;

(G₄):

$\hat{{G}}(x,\eta )\ge 0$, and there exist $\delta _{0} \in (0,\Lambda _{0})$, $\delta _{1}>0$ such that $\hat{{G}}(x,\eta )>0$ if $0<|\eta |\le \delta _{1}$, and $\hat{{G}}(x,\eta )\ge \delta _{0}$ whenever $|\hat{{G}}_{ \eta }(x,\eta )|\ge (\Lambda _{0}-\delta _{0})|\eta |$.

More precisely, the following theorems were established in [40, 46].

Theorem 1.1

([40])

Let (V), (G1), (SQ), and (DL) be satisfied. Then (1.1) has a least energy solution, i.e., a nontrivial solution $\bar{\eta }\in E$ such that $I(\bar{\eta })=\inf_{\mathcal{M}}I$.

Theorem 1.2

([46])

Let (V), (G1), (G₂), and (G₃) or (G₄) be satisfied. Then (1.1) has a least energy solution.

It follows from (G1), (SQ), and (DL) that the following condition holds, see Lemma 2.2:

• (G0) $G\in C^{1}({\mathbb {R}}^{N}\times {\mathbb {R}}^{2},\mathbb {R})$, and there exist constants $p\in (2,2^{*})$ and $C_{1}>0$ such that

  $$ \bigl\vert G_{\eta }(x,\eta ) \bigr\vert \le C_{1}\bigl(1+ \vert \eta \vert ^{p-1}\bigr),\quad \forall (x,\eta )\in \mathbb {R}^{N}\times \mathbb {R}^{2}, $$

  where $2^{*}:=2N/(N-2)$ if $N\geq 3$ and $2^{*}:=+\infty$ if $N=1$ or 2.

Condition (DL) was first introduced by Ding and Lee [13] for a single Schrödinger equation, and it together with (SQ) is commonly used instead of the following classic condition, see, for instance, [26, 40, 42].

• (AR) There exists a constant $\mu >2$ such that

  $$ 0< \mu G(x,\eta )\le G_{\eta }(x,\eta )\cdot \eta , \quad \forall x\in {\mathbb {R}}^{N}, \eta \in {\mathbb {R}}^{2}\setminus \bigl\{ (0,0)\bigr\} . $$

It is clear that (AR) implies (SQ); moreover, by a similar argument as in the proof of [13, Lemma 1.3], (DL) holds provided that (AR) and (G0) with $\mu \ge p$ are satisfied. The idea of using the more natural super-quadratic condition (SQ) under a Nehari type setting to replace (AR) goes back to Liu and Wang [25]. Afterwards, condition (SQ) was used in many papers, see [13, 14, 26, 33, 38, 42, 47] and the references therein. In a recent paper [30], a local superquadratic condition instead of (SQ) was introduced by Qin, Tang, and Wu, which combines with a technical condition to guarantee the existence of ground state solutions for Schrödinger systems like (1.1). Note that the assumption $\hat{{G}}(x,\eta )>0$ in (DL), (G₃), and (G₄) plays an important role in the arguments of Zhang [40] and Zhao [46], see also Ding and Lee [13].

Inspired by the aforementioned works, we continue to study problem (1.1) in this paper and construct two types of ground state solutions, i.e., the least energy solution and the Nehari–Pankov type. We first use a technical condition introduced in [29] to consider the super-quadratic case and obtain a least energy solution for (1.1) with the aid of a generalized linking theorem established in [20]. Then we lay emphasis on the asymptotically quadratic case and discuss the existence of ground state solution of Nehari–Pankov type for (1.1). Different from the super-quadratic case, the Nehari–Pankov manifold $\mathcal{N}^{-}$ is not homeomorphic to the unit sphere for the asymptotically quadratic case, so the generalized Nehari manifold method introduced by Szulkin and Weth [33] does not apply even under strict monotonicity on G. More precisely, for each $\eta \in E\setminus E^{-}$, $\mathcal{N}^{-}$ does not necessarily intersect the set $E^{-}\oplus \mathbb {R}^{+}\eta$, so one cannot directly adopt the method used in [35] either. On the other hand, the argument in [40, 46, see Lemma 4.1] becomes invalid due to the lack of positive assumption on $\hat{{G}}(x,\eta )$ (i.e., $\hat{{G}}(x,\eta )>0$). Motivated by the works [27, 33, 40, 46], we further develop the approach in [29, 34, 35] to find ground state solution of Nehari–Pankov type for (1.1). Our approach is based on finding a proper subset $E_{0}^{+}$ of $E\setminus E^{-}$ (see (3.4) for the definition) such that, for any $\eta \in E_{0}^{+}$, there exist $t=t(\eta )>0$ and $w=w(\eta )\in E^{-}$ satisfying $w+t\eta \in \mathcal{N}^{-}$, then we can derive a minimizing sequence on the Nehari–Pankov manifold by using the diagonal method, see Lemma 3.9.

Before stating our main results, we first introduce the following weaker version of (DL):

• (G2) $\hat{{G}}(x,\eta )\ge 0$, and there exist $C_{0}>0$, $\delta _{0}\in (0,\Lambda _{0})$, and $\sigma >\max \{1,N/2\}$ such that

  $$ \frac{ \vert G_{\eta }(x,\eta ) \vert }{ \vert \eta \vert }\ge \frac{1}{2} ( \Lambda _{0}-\delta _{0} )\quad \Longrightarrow \quad \bigl\vert G_{\eta }(x,\eta ) \bigr\vert ^{\sigma }\le C_{0}\hat{{G}}(x,\eta ) \vert \eta \vert ^{\sigma }. $$

Clearly, (G2) holds under (DL) and (G1). Thus (G2) weakens (DL), and there are some functions satisfying (G2), but not (DL) and (AR), see Examples 1.9 and 1.10. Moreover, (G0) holds under (G1), (G2), and (SQ), see Lemma 2.2.

We are now in a position to state the first result of this paper.

Theorem 1.3

Let (V), (G1), (G2), and (SQ) be satisfied. Then (1.1) has a least energy solution.

Next, we consider the asymptotically quadratic case and introduce the following assumptions:

(G3):

$G_{\eta }(x, \eta )=V_{\infty }(x)\eta +\bar{G}_{\eta }(x, \eta )$, where $V_{\infty }\in C(\mathbb {R}^{N})$ is 1-periodic in each of $x_{1}, x_{2}, \ldots , x_{N}$ with $\inf V_{\infty }>0$, and there exists $\eta _{0}\in E^{+} \setminus \{0\}$ such that

$$ \Vert \eta _{0} \Vert ^{2}- \Vert w \Vert ^{2}- \int _{\mathbb {R}^{N}}V_{\infty }(x) \vert \eta _{0}+w \vert ^{2} \,\mathrm{d}x< 0, \quad \forall w\in E^{-}; $$

(1.7)

${G}_{\eta }(x, \eta )\cdot \bar{G}_{\eta }(x, \eta )\le 0$ and ${G}_{\eta }(x, \eta ) \cdot \bar{G}_{\eta }(x, \eta )<0$ for $0<|\eta |\le \alpha _{0}$ for some $\alpha _{0}>0$, $|\bar{G}_{\eta }(x, \eta )|=o(|\eta |)$ as $|\eta |\to \infty$ uniformly in $x\in \mathbb {R}^{N}$;

(G3′):

$G_{\eta }(x, \eta )=V_{\infty }(x)\eta +\bar{G}_{\eta }(x, \eta )$, where $V_{\infty }\in C(\mathbb {R}^{N})$ is 1-periodic in each of $x_{1}, x_{2}, \ldots , x_{N}$ with $\inf V_{\infty }>{\Lambda }$, $|\bar{G}_{\eta }(x, \eta )|=o(| \eta |)$ as $|\eta |\to \infty$ uniformly in $x\in \mathbb {R}^{N}$, and ${G}_{\eta }(x, \eta )\cdot \bar{G}_{\eta }(x, \eta )<0$ for $x\in \mathbb {R}^{N}$ and $\eta \neq (0,0)$;

(G4):

For all $\theta \ge 0$, $x\in \mathbb {R}^{N}$, $\eta , w\in \mathbb {R}^{2}$, there holds

$$ \frac{1-\theta ^{2}}{2}G_{\eta }(x,\eta )\cdot \eta -\theta G_{\eta }(x, \eta )\cdot w+G(x,\theta \eta +w) -G(x,\eta )\ge 0; $$

Our main results for the asymptotically quadratic case read as follows.

Theorem 1.4

Let (V), (G1), (G3), and (G4) be satisfied. Then (1.1) has a nontrivial solution $\bar{\eta }\in E$ such that $I({\bar{\eta }})=\inf_{\mathcal{N}^{-}}I \geq \kappa$, where $\kappa >0$ is a constant.

Corollary 1.5

Let (V), (G1), (G3′), and (G4) be satisfied. Then (1.1) has a nontrivial solution $\bar{\eta }\in E$ such that $I(\bar{\eta })=\inf_{\mathcal{N}^{-}}I \geq \kappa$, where $\kappa >0$ is a constant.

In general, $\mathcal{N}^{-}$ contains infinitely many elements of E. In fact, for any $\eta \in E_{0}^{+}$ defined later by (3.4), there exist $t=t(\eta )>0$ and $w=w(\eta )\in E^{-}$ such that $w+t\eta \in \mathcal{N}^{-}$ which is the global maximum of $I|_{E^{-}\oplus \mathbb {R}^{+}\eta }$, see Corollary 3.2 and Lemma 3.7. Since η̄ is a solution at which I has least “energy” in set $\mathcal{N}^{-}$, it was called a ground state solution of Nehari–Pankov type in [33, 35].

Remark 1.6

It is easy to see that (G4), together with $|G_{\eta }(x,\eta )|=o(|\eta |)$ as $|\eta | \to 0$ uniformly in $x\in \mathbb {R}^{N}$, implies that ${G}(x,\eta )\ge 0$ and $\hat{{G}}(x,\eta )\ge 0$ for any $(x,\eta )\in \mathbb {R}^{N}\times \mathbb {R}^{2}$. Condition (G0) holds under (G1) and (G3); moreover, (G3′) implies (G3), see (2.14) for details. The crucial condition $\hat{{G}}(x,\eta )>0$ in (DL), (G₃), and (G₄) is not needed in (G2) and (G4); moreover, there are many functions satisfying (G2), (G3), and (G4), but not (DL), (G3′), and (G4′) resp., see Examples 1.10 and 1.11. Thus Theorems 1.3 and 1.4 generalize and improve the results in [35, 40, 46].

Remark 1.7

In [35], Tang introduced the following class of functions which satisfies (G4):

$$ G(x,\varphi ,\psi )=\sum^{k}_{i=1} \int ^{\alpha _{i}\varphi +\beta _{i}\psi }_{0}g_{i}(x,t)t \,\mathrm{d}t + \sum^{l}_{j=1} \int ^{\sqrt{\varphi ^{2}+2b_{j}\varphi \psi +a_{j}\psi ^{2}}}_{0}h _{j}(x,t)t \,\mathrm{d}t, $$

where $\alpha _{i}, \beta _{i}, a_{j}, b_{j}\in \mathbb {R}$ with $\alpha ^{2}_{i}+\beta ^{2}_{i}\neq 0$ and $a_{j}>b_{j}^{2}$, $g_{i}(x,t)$ and $h_{j}(x,t)$ are nondecreasing in $t\in \mathbb {R}^{+}$ for every $x\in \mathbb {R}^{N}$ and $g_{i}(x,0)=h_{j}(x,0)=0$.

Condition (G4) is also satisfied under assumptions (G1), (G₂) and the following condition introduced by Bartsch and Mederski [6], the proof of which will be given in the Appendix.

(G4′):

$\hat{{G}}(x,\eta )\ge 0$, and for any $x\in \mathbb {R}^{N}$, $\eta , w\in \mathbb {R}^{2}$, there holds

$$ {2G_{\eta }(x,\eta )\cdot \eta }\bigl[G(x,\eta )-G(x,w)\bigr]\le {{ \bigl(G_{\eta }(x, \eta )\cdot \eta \bigr)^{2}- \bigl(G_{\eta }(x,\eta )\cdot w\bigr)^{2}}}, $$

whenever $G_{\eta }(x,\eta )\cdot w=G_{\eta }(x,w)\cdot \eta \neq 0$.

Moreover, (G4) holds also under conditions (G1), (G4′), and (SQ).

Remark 1.8

For the asymptotically quadratic case, Nehari–Pankov manifold $\mathcal{N}^{-}$ does not necessarily intersect the set $\hat{E}(\eta ):=E^{-}\oplus \mathbb {R}^{+}\eta$ for each $\eta \in E\setminus E^{-}$, and so $\mathcal{N}^{-}$ is not homeomorphic to the unit sphere $S^{+}\subset E^{+}$ even under the strict monotonic condition. Thus, it seems infeasible to find a minimizing sequence on the Nehari–Pankov manifold by reducing the problem on a sphere under assumptions (G1), (G3), and (G4).

Before proceeding to the proof of main results, we give some nonlinear examples to illustrate condition (G2).

Example 1.9

$G(x,\eta )=\alpha (x)|\eta |^{2} \ln [1+|\eta |]$, where $\alpha \in C({\mathbb {R}}^{N},(0,\infty ))$ is 1-periodic in $x_{i}$, $i=1, 2, \ldots , N$.

It is not difficult to verify that G satisfies (G1), (G2), and (SQ), but it does not satisfy (AR).

Example 1.10

$G(x,\eta )=a ({8}/{5}|\eta |^{13/4}-4| \eta |^{11/4}+{9}/{2}|\eta |^{9/4} )$, where $a>0$ and $N\le 4$.

By simple computation, one has $\hat{{G}}(x,\eta )=a|\eta |^{9/4} (\sqrt{ |\eta |}-{3}/{4} )^{2}\ge 0$. Then G does not satisfy (AR) and (DL), but it satisfies (G2) with $\sigma =12/5$ if $a\in (0,8\Lambda _{0}/81)$.

For the asymptotically quadratic case, it is not difficult to verify that the following functions satisfy (G1), (G3), (G4), or (G3′) by virtue of Remark 1.7, but Example 1.11 does not satisfy (G₃) and (G₄) since $\hat{{G}}(x,\eta )=0$ if $|\eta |\ge 1$.

Example 1.11

$G_{\eta }(x,\eta )={V_{\infty }(x)} \max \{ |\eta |^{\alpha }, 1 \} \eta$, where $\alpha >0$ and $V_{\infty }\in C(\mathbb {R}^{N})$ is 1-periodic in each of $x_{1}, x_{2}, \ldots , x_{N}$ with $\inf V_{\infty }>\Lambda$.

Example 1.12

$G_{\eta }(x,\eta )={V_{\infty }(x)} [1-\frac{1}{1+|\eta |^{\alpha }} ]\eta$, where $\alpha >0$ and $V_{\infty }\in C(\mathbb {R}^{N})$ is 1-periodic in each of $x_{1}, x_{2}, \ldots , x_{N}$ with $\inf V_{\infty }>\Lambda$.

Example 1.13

$G_{\eta }(x,\eta )={V_{\infty }(x)} [1-\frac{1}{\ln (e+|\eta |)} ]\eta$, where $V_{\infty } \in C(\mathbb {R}^{N})$ is 1-periodic in each of $x_{1}, x_{2}, \ldots , x _{N}$ with $\inf V_{\infty }>\Lambda$.

The remainder of this paper is organized as follows. In Sect. 2, some preliminary results are presented. The proofs of the main results will be given in the last section.

2 Variational setting and preliminaries

Let $\mathcal{A}=-\triangle +V$. Then $\mathcal{A}$ is self-adjoint in $L^{2}(\mathbb {R}^{N})$ with domain $\mathfrak{D}(\mathcal{A})=H^{2}(\mathbb {R}^{N})$ (see [16, Theorem 4.26]). Let $\{\mathcal{E}(\lambda ): - \infty \le \lambda \le +\infty \}$ and $|\mathcal{A}|$ be the spectral family and the absolute value of $\mathcal{A}$, respectively, and $|\mathcal{A}|^{1/2}$ be the square root of $|\mathcal{A}|$. Set $\mathcal{U}=\mathrm{id}-\mathcal{E}(0) -\mathcal{E}(0-)$. Then $\mathcal{U}$ commutes with $\mathcal{A}$, $|\mathcal{A}|$, and $|\mathcal{A}|^{1/2}$, and $\mathcal{A} = \mathcal{U}|\mathcal{A}|$ is the polar decomposition of $\mathcal{A}$ (see [15, Theorem IV 3.3]). Let

$$ H:=\mathfrak{D}\bigl( \vert \mathcal{A} \vert ^{1/2} \bigr), \qquad H^{-}:=\mathcal{E}(0)H, \qquad H^{+}:=\bigl[ \mathrm{id}-\mathcal{E}(0)\bigr]H. $$

(2.1)

For any $u\in H$, it is easy to see that

$$ u={u}^{-}+{u}^{+}, \qquad u^{-}:= \mathcal{E}(0)u\in H^{-}, \qquad u^{+}:=\bigl[\mathrm{id}- \mathcal{E}(0)\bigr]u\in H^{+} $$

(2.2)

and

$$ \mathcal{A}u^{-}=- \vert \mathcal{A} \vert u^{-}, \qquad \mathcal{A}u^{+}= \vert \mathcal{A} \vert u^{+}, \quad \forall u\in H\cap \mathfrak{D}(\mathcal{A}). $$

(2.3)

Define an inner product

$$ (u, v)_{H}= \bigl( \vert \mathcal{A} \vert ^{1/2}u, \vert \mathcal{A} \vert ^{1/2}v \bigr) _{L^{2}}, \quad u, v\in H $$

(2.4)

and the corresponding norm

$$ \Vert u \Vert _{H}= \bigl\Vert \vert \mathcal{A} \vert ^{1/2}u \bigr\Vert _{L^{2}}, \quad u\in H, $$

(2.5)

where $(\cdot , \cdot )_{L^{2}}$ denotes the inner product of $L^{2}(\mathbb {R}^{N})$, $\|\cdot \|_{L^{s}}$ stands for the usual $L^{s}(\mathbb {R}^{N})$ norm. By (V), $H=H^{1}(\mathbb {R}^{N})$ with equivalent norms. Therefore, H embeds continuously in $L^{s}(\mathbb {R}^{N})$ for all $2\le s\le 2^{*}$. In addition, one has the decomposition $H=H^{-} \oplus H^{+}$ which are orthogonal with respect to $(\cdot , \cdot )_{L ^{2}}$ and $(\cdot , \cdot )_{H}$. Then there hold

$$ \int _{\mathbb {R}^{N}} \bigl( \vert \nabla u \vert ^{2}+V(x) \vert u \vert ^{2} \bigr)\,\mathrm{d}x= \bigl\Vert u ^{+} \bigr\Vert _{H}^{2}- \bigl\Vert u^{-} \bigr\Vert _{H}^{2}, \quad \forall u=u^{-}+u^{+} \in H. $$

(2.6)

Let $E=H\times H$ with the inner product

$$ \bigl( (u,v),(\varphi ,\psi ) \bigr)=(u,\varphi )_{H}+(v,\psi )_{H}, $$

(2.7)

and the corresponding norm

$$ \Vert z \Vert = \bigl[ \Vert u \Vert ^{2}_{H}+ \Vert v \Vert ^{2}_{H} \bigr]^{1/2},\quad \forall z=(u,v)\in E. $$

(2.8)

By (1.6), (1.5), and (2.8), it is easy to see that $\|z\|^{2}\ge \Lambda _{0}\|z\|^{2}_{2}$ for any $z\in E$, where $\|\cdot \|_{s}$ stands for the usual $L^{s}(\mathbb {R}^{N},\mathbb {R}^{2})$ norm. Moreover, we have the following lemma.

Lemma 2.1

Let (V) be satisfied. Then the embedding $E\hookrightarrow L^{p}(\mathbb {R}^{N}, \mathbb {R}^{2})$ is continuous for all $p\in [2,2^{*}]$ and $E\hookrightarrow L^{p}_{\mathrm{loc}}(\mathbb {R}^{N}, \mathbb {R}^{2})$ is compact for all $p\in [2,2^{*})$.

Lemma 2.2

([29, Lemma 2.2])

Suppose that (G1), (G2), and (SQ) are satisfied. Then (G0) holds. Moreover, for any $\varepsilon >0$, there exist $C_{\varepsilon }>0$ and $p\in (2,2^{*})$ such that

$$ \bigl\vert G_{\eta }(x,\eta ) \bigr\vert \le \varepsilon \vert \eta \vert +C_{\varepsilon } \vert \eta \vert ^{p-1},\quad \forall (x,\eta )\in \mathbb {R}^{N}\times \mathbb {R}^{2}, $$

(2.9)

and $\hat{{G}}(x,\eta )\to \infty$ as $|\eta |\to \infty$ uniformly in x.

Under (V), (G0), and (G1), a standard argument (see [12, 32, 37]) shows that the solutions of problem (1.1) are critical points of the functional

$$ I(\eta )= \int _{{\mathbb {R}}^{N}} \bigl(\nabla \varphi \nabla \psi +V(x) \varphi \psi \bigr)\,\mathrm{d}x- \int _{{\mathbb {R}}^{N}}G(x, \eta ) \,\mathrm{d}x, \quad \forall \eta =(\varphi ,\psi )\in E, $$

(2.10)

I is of class $C^{1}(E, \mathbb {R})$, and

$$\begin{aligned} \bigl\langle I'(\eta ), w\bigr\rangle = & \int _{{\mathbb {R}}^{N}} \bigl(\nabla \varphi \nabla \phi +V(x)\varphi \phi \bigr)\,\mathrm{d}x+ \int _{{\mathbb {R}} ^{N}} \bigl(\nabla \psi \nabla \xi +V(x)\psi \xi \bigr) \,\mathrm{d}x \\ &{} - \int _{{\mathbb {R}}^{N}} \bigl(G_{\varphi }(x, \eta ) \xi +G_{\psi }(x, \eta )\phi \bigr)\,\mathrm{d}x,\quad \forall \eta =(\varphi , \psi ), w= (\xi ,\phi )\in E. \end{aligned}$$

(2.11)

The following generalized linking theorem plays an important role in proving our main results.

Let X be a Hilbert space with $X=X^{-}\oplus X^{+}$ and $X^{-} \perp X^{+}$. For a functional $I\in C^{1}(X,\mathbb {R})$, I is said to be weakly sequentially lower semi-continuous if, for any $u_{n}\rightharpoonup u$ in X, one has $I(u)\leq \liminf_{n\rightarrow \infty }I(u_{n})$, and $I'$ is said to be weakly sequentially continuous if $\lim_{n\rightarrow \infty }\langle I'(u_{n}),v\rangle =\langle I'(u),v \rangle$ for each $v\in X$.

Lemma 2.3

([19], [20, Theorem 2.1])

Let X be a Hilbert space with $X=X^{-}\oplus X^{+}$ and $X^{-}\perp X^{+}$, and let $I\in C^{1}(X,\mathbb {R})$ of the form

$$ I(u)=\frac{1}{2}\bigl( \bigl\Vert u^{+} \bigr\Vert ^{2}- \bigl\Vert u^{-} \bigr\Vert ^{2}\bigr)- \phi (u), \quad u=u^{+}+u^{-}\in X^{+}\perp X^{-}. $$

Suppose that the following assumptions are satisfied:

• (I1) $\phi \in C^{1}(X,\mathbb {R})$ is bounded from below and weakly sequentially lower semi-continuous;

• (I2) $\phi '$ is weakly sequentially continuous;

• (I3) there exist $r>\rho >0$ and $e\in X^{+}$ with $\|e\|=1$ such that

  $$ \kappa :=\inf I(S_{\rho })>\sup I(\partial Q), $$

  where

  $$ S_{\rho }=\bigl\{ u\in X^{+}: \Vert u \Vert =\rho \bigr\} , \qquad Q=\bigl\{ se+v: v\in X^{-}, s\geq 0, \Vert se+v \Vert \leq r \bigr\} . $$

Then, for some $c\geq \kappa$, there exists a sequence $\{u_{n}\} \subset X$ satisfying

$$ I (u_{n})\rightarrow c, \qquad \bigl\Vert I'(u_{n}) \bigr\Vert \bigl(1+ \Vert u_{n} \Vert \bigr)\rightarrow 0. $$

Let

$$ E^{+}=H^{+}\times H^{-},\qquad E^{-}=H^{-}\times H^{+}, $$

(2.12)

then, for any $z=(u,v)\in E$, there holds

$$ z=z^{+}+z^{-},\qquad z^{+}= \bigl(u^{+},v^{-}\bigr)\in E^{+},\qquad z^{-}=\bigl(u^{-},v^{+}\bigr)\in E^{-}. $$

(2.13)

Moreover, $E^{+}$ and $E^{-}$ are orthogonal with respect to the inner products $(\cdot , \cdot )_{2}$ and $(\cdot , \cdot )$, where $(\cdot , \cdot )_{2}$ is chosen by $((u,v), (\varphi ,\psi ) ) _{2}=(u,\varphi )_{L^{2}}+(v,\psi )_{L^{2}}$ for any $(u,v), ( \varphi ,\psi ) \in L^{2}(\mathbb {R}^{N},\mathbb {R}^{2})$. Hence $E=E^{+}\oplus E ^{-}$.

Next we show that (G3′) implies (G3). If $\inf V_{\infty }>{\Lambda }$, take $\bar{\mu }\in ({\Lambda },\inf V_{\infty })$, then there exists $\eta _{0}\in (\mathcal{E}(\bar{\mu })-\mathcal{E}(0) )H \times (\mathcal{E}(0)-\mathcal{E}(-\bar{\mu }))H \subset E^{+}\setminus \{0\}$ such that $\Lambda \|\eta _{0}\|_{2}^{2}\le \|\eta _{0}\|^{2} \le \bar{\mu }\|\eta _{0}\|_{2}^{2}$, thus for any $w\in E^{-}$, one has

$$\begin{aligned}& \Vert \eta _{0} \Vert ^{2}- \Vert w \Vert ^{2}- \int _{\mathbb {R}^{N}}V_{\infty }(x) \vert \eta _{0}+w \vert ^{2} \,\mathrm{d}x \\& \quad \le \bar{\mu } \Vert \eta _{0} \Vert _{2}^{2}- \Vert w \Vert ^{2}-\inf V_{\infty } \bigl( \Vert \eta _{0} \Vert ^{2}_{2}+ \Vert w \Vert _{2}^{2} \bigr) \\& \quad = - \bigl[(\inf V_{\infty }-\bar{\mu }) \Vert \eta _{0} \Vert _{2}^{2}+ \Vert w \Vert ^{2}+\inf V_{\infty } \bigl( \Vert \eta _{0} \Vert ^{2}_{2}+ \Vert w \Vert _{2}^{2} \bigr) \bigr] \\& \quad < 0. \end{aligned}$$

(2.14)

This shows that (1.7) holds.

As in [46], we introduce a change of variable

$$ \textstyle\begin{cases} \varphi =\frac{u+v}{\sqrt{2}}, \\ \psi =\frac{u-v}{\sqrt{2}}, \end{cases} $$

(2.15)

and set $H(x,z)=H(x,u,v):=G(x,\frac{u+v}{\sqrt{2}},\frac{u-v}{ \sqrt{2}})$, $\bar{H}(x,z)=\bar{G}(x, \frac{u+v}{\sqrt{2}},\frac{u-v}{ \sqrt{2}})$, here and in the sequel, we write $z=(u,v)$ and $|z|= (|u|^{2}+|v|^{2} )^{\frac{1}{2}}$.

It is not difficult to verify that assumptions (G2′), (G4′), and (G0)–(G5) on G imply that H satisfies:

(S0):

There exist constants $p\in (2,2^{*})$ and $C_{1}>0$ such that

$$ \bigl\vert H_{z}(x,z) \bigr\vert \le C_{1}\bigl(1+ \vert z \vert ^{p-1}\bigr), \quad \forall (x,z)\in \mathbb {R}^{N} \times \mathbb {R}^{2}; $$

(S1):

$H\in C^{1}({\mathbb {R}}^{N}\times {\mathbb {R}}^{2},[0,\infty ))$ is 1-periodic in $x_{i}$, $i=1, 2, \ldots , N$, and $|H_{z}(x,z)|=o(|z|)$ as $|z|\to 0$ uniformly in x;

(S2):

$\hat{{H}}(x,z):=\frac{1}{2}{{H}_{z}}(x,z)\cdot z-{{H}}(x,z) \ge 0$, and there exist constants $C_{0}>0$, $\delta _{0}\in (0,\Lambda _{0})$ and $\sigma >\max \{1,N/2\}$ such that

$$ \frac{ \vert H_{z}(x,z) \vert }{ \vert z \vert }\ge \frac{1}{2} (\Lambda _{0}-\delta _{0} )\quad \Longrightarrow\quad \bigl\vert H_{z}(x,z) \bigr\vert ^{\sigma }\le C_{0}\hat{{H}}(x,z) \vert z \vert ^{ \sigma }; $$

(S3):

$H_{z}(x, z)=V_{\infty }(x)z+\bar{H}_{z}(x, z)$, where $V_{\infty }\in C(\mathbb {R}^{N})$ is 1-periodic in each of $x_{1}, x_{2}, \ldots , x_{N}$ with $\inf V_{\infty }>0$, and there exists $z_{0}\in E^{+}\setminus \{0\}$ such that

$$ \tau ^{2} \Vert z_{0} \Vert ^{2}- \Vert w \Vert ^{2}- \int _{\mathbb {R}^{N}}V_{\infty }(x) (\tau z _{0}+w)^{2} \,\mathrm{d}x< 0, \quad \forall \tau >0, w\in E^{-}; $$

${H}_{z}(x,z)\cdot \bar{H}_{z}(x,z)\le 0$ and ${H}_{z}(x,z)\cdot \bar{H}_{z}(x, z)<0$ for $0<|z|\le \alpha _{0}$ for some $\alpha _{0}>0$, $|\bar{H}_{z}(x, z)|=o(|z|)$ as $|z|\to \infty$ uniformly in $x\in \mathbb {R}^{N}$;

(S3′):

$H_{z}(x, z)=V_{\infty }(x)z+\bar{H}_{z}(x, z)$, where $V_{\infty }\in C(\mathbb {R}^{N})$ is 1-periodic in each of $x_{1}, x_{2}, \ldots , x_{N}$ with $\inf V_{\infty }>\Lambda$, $|\bar{H}_{z}(x, z)|=o(|z|)$ as $|z|\to \infty$ uniformly in $x\in \mathbb {R}^{N}$, and ${H}_{z}(x,z) \cdot \bar{H}_{z}(x, z)<0$ for $x\in \mathbb {R}^{N}$ and $z\neq (0,0)$;

(S4):

For all $\theta \ge 0$, $x\in \mathbb {R}^{N}$, $z, \zeta \in E$, there holds

$$ \frac{1-\theta ^{2}}{2}H_{z}(x,z)\cdot z-\theta H_{z}(x,z)\cdot \zeta +H(x,\theta z+\zeta ) -H(x,z)\ge 0; $$

(S4′):

$\hat{{H}}(x,z)\ge 0$, and for any $x\in \mathbb {R}^{N}$, $z, \zeta \in \mathbb {R}^{2}$, there holds

$$ {2H_{z}(x,z)\cdot z}\bigl[H(x,z)-H(x,\zeta )\bigr]\le {{ \bigl(H_{z}(x,z)\cdot z\bigr)^{2}-\bigl(H _{z}(x,z)\cdot \zeta \bigr)^{2}}}, $$

whenever $H_{z}(x,z)\cdot \zeta =H_{z}(x,\zeta )\cdot z\neq 0$.

Under (V), (S0), and (S1), it follows from Lemma 2.2 that, for any $\varepsilon >0$, there exist $C_{\varepsilon }>0$ and $p\in (2,2^{*})$ such that

$$ \bigl\vert H_{z}(x,z) \bigr\vert \le \varepsilon \vert z \vert +C_{\varepsilon } \vert z \vert ^{p-1},\quad \forall (x,z)\in \mathbb {R}^{N}\times \mathbb {R}^{2}. $$

(2.16)

Clearly, (S0) holds under (S1) and (S3), as well as under (S1), (S2), and (SQ). For any $z=(u,v)\in E$, it follows from (2.6) and (2.15) that $\eta =(\varphi ,\psi )=(\frac{u+v}{\sqrt{2}},\frac{u-v}{ \sqrt{2}})$ and

$$\begin{aligned}& \int _{{\mathbb {R}}^{N}} \bigl(\nabla \varphi \nabla \psi +V(x)\varphi \psi \bigr)\,\mathrm{d}x \\& \quad = \frac{1}{2} \int _{{\mathbb {R}}^{N}} \bigl( \vert \nabla u \vert ^{2}+V(x) \vert u \vert ^{2}- \vert \nabla v \vert ^{2}-V(x) \vert v \vert ^{2} \bigr)\,\mathrm{d}x \\& \quad = \frac{1}{2} \bigl( \bigl\Vert u^{+} \bigr\Vert _{H}^{2}- \bigl\Vert u^{-} \bigr\Vert _{H}^{2}- \bigl\Vert v^{+} \bigr\Vert _{H}^{2}+ \bigl\Vert v^{-} \bigr\Vert _{H}^{2} \bigr) \\& \quad = \frac{1}{2} \bigl( \bigl\Vert z^{+} \bigr\Vert ^{2}- \bigl\Vert z^{-} \bigr\Vert ^{2} \bigr). \end{aligned}$$

(2.17)

Thus, we have an equivalent functional

$$ \Phi (z)=\Phi (u,v)=\frac{1}{2} \bigl( \bigl\Vert z^{+} \bigr\Vert ^{2}- \bigl\Vert z^{-} \bigr\Vert ^{2} \bigr)- \Psi (z) $$

(2.18)

and

$$ \bigl\langle \Phi '(z),\zeta \bigr\rangle = \bigl(z^{+},\zeta ^{+}\bigr)-\bigl(z ^{-},\zeta ^{-}\bigr)- \bigl\langle \Psi '(z),\zeta \bigr\rangle ,\quad \forall z, \zeta \in E, $$

(2.19)

where

$$ \Psi (z)= \int _{{\mathbb {R}}^{N}}H(x,z)\,\mathrm{d}x,\qquad \bigl\langle \Psi '(z),\zeta \bigr\rangle = \int _{{\mathbb {R}}^{N}}H_{z}(x,z)\cdot \zeta \,\mathrm{d}x. $$

(2.20)

It is obvious that $z=(u,v)\in E$ is a critical point of Φ if and only if $(\frac{u+v}{\sqrt{2}},\frac{u-v}{\sqrt{2}} )$ is a critical point of I. In what follows, we shall seek for the critical points of Φ under the assumptions on H. Obviously, the sets $\mathcal{M}$ and $\mathcal{N}^{-}$ defined by (1.2) and (1.3) can be rewritten as follows:

$$ {\mathcal{M}}:=\bigl\{ z\in E\setminus \bigl\{ (0,0)\bigr\} : \Phi '(z)=0\bigr\} , $$

(2.21)

and

$$ \mathcal{N}^{-}=\bigl\{ z\in E\setminus E^{-}: \bigl\langle \Phi '(z),z\bigr\rangle =\bigl\langle \Phi '(z),\zeta \bigr\rangle =0, \forall \zeta \in E^{-} \bigr\} , $$

(2.22)

By (V), the functional Φ is strongly indefinite, the type of which has been largely investigated by authors via critical point theory, see [8, 12, 13, 18, 33, 37] and the references therein.

Before proceeding to the proof of the main results, we need some preliminaries.

Lemma 2.4

Suppose that (V), (S0), and (S1) are satisfied. If $\inf V_{\infty }>0$, then

$$\begin{aligned} \tau \bigl\langle \Phi '(z), \tau z+2\zeta \bigr\rangle \ge & \tau ^{2} \bigl\Vert z^{+} \bigr\Vert ^{2}- \bigl\Vert \tau z^{-}+\zeta \bigr\Vert ^{2}+ \Vert \zeta \Vert ^{2}- \int _{\mathbb {R}^{N}}V_{ \infty }(x) \vert \tau z+\zeta \vert ^{2}\,\mathrm{d}x \\ &{} +\tau ^{2} \int _{\mathbb {R}^{N}}\frac{z\cdot H_{z}(x, z)V_{\infty }(x)- \vert H _{z}(x, z) \vert ^{2}}{V_{\infty }(x)}\,\mathrm{d}x, \\ &\forall z\in E, \tau \in \mathbb {R}, \zeta \in E^{-}. \end{aligned}$$

(2.23)

Proof

In view of (2.19) and $\inf V_{\infty }>0$, we have

$$\begin{aligned}& \tau \bigl\langle \Phi '(z), \tau z+2\zeta \bigr\rangle \\& \quad = \tau ^{2} \bigl\Vert z^{+} \bigr\Vert ^{2}-\tau ^{2} \bigl\Vert z^{-} \bigr\Vert ^{2}-2\tau \bigl(z^{-}, \zeta \bigr)-\tau \int _{\mathbb {R}^{N}}H_{z}(x, z)\cdot (\tau z+2\zeta ) \,\mathrm{d}x \\& \quad = \tau ^{2} \bigl\Vert z^{+} \bigr\Vert ^{2}- \bigl\Vert \tau z^{-}+\zeta \bigr\Vert ^{2}+ \Vert \zeta \Vert ^{2}- \int _{\mathbb {R}^{N}}V_{\infty }(x) \vert \tau z+\zeta \vert ^{2}\,\mathrm{d}x \\& \qquad {} + \int _{\mathbb {R}^{N}} \bigl[V_{\infty }(x) \vert \tau z+\zeta \vert ^{2}-\tau H _{z}(x, z)\cdot (\tau z+2\zeta ) \bigr] \,\mathrm{d}x \\& \quad = \tau ^{2} \bigl\Vert z^{+} \bigr\Vert ^{2}- \bigl\Vert \tau z^{-}+\zeta \bigr\Vert ^{2}+ \Vert \zeta \Vert ^{2}- \int _{\mathbb {R}^{N}}V_{\infty }(x) \vert \tau z+\zeta \vert ^{2}\,\mathrm{d}x \\& \qquad {} + \int _{\mathbb {R}^{N}} \bigl\{ V_{\infty }(x) \vert \zeta \vert ^{2}+2\tau \bigl[V _{\infty }(x)z-H_{z}(x, z) \bigr] \cdot \zeta \\& \qquad {} + \bigl[V_{\infty }(x) \vert z \vert ^{2}-z\cdot H_{z}(x, z) \bigr] \tau ^{2} \bigr\} \,\mathrm{d}x \\& \quad \ge \tau ^{2} \bigl\Vert z^{+} \bigr\Vert ^{2}- \bigl\Vert \tau z^{-}+\zeta \bigr\Vert ^{2}+ \Vert \zeta \Vert ^{2}- \int _{\mathbb {R}^{N}}V_{\infty }(x) \vert \tau z+\zeta \vert ^{2}\,\mathrm{d}x \\& \qquad {} + \int _{\mathbb {R}^{N}}\frac{ \vert V_{\infty }(x)\zeta +\tau [V_{ \infty }(x)z-H_{z}(x, z) ] \vert ^{2}}{V_{\infty }(x)}\,\mathrm{d}x \\& \qquad {} +\tau ^{2} \int _{\mathbb {R}^{N}}\frac{z\cdot H_{z}(x, z)V_{\infty }(x)- \vert H _{z}(x, z) \vert ^{2}}{V_{\infty }(x)}\,\mathrm{d}x \\& \quad \ge \tau ^{2} \bigl\Vert z^{+} \bigr\Vert ^{2}- \bigl\Vert \tau z^{-}+\zeta \bigr\Vert ^{2}+ \Vert \zeta \Vert ^{2}- \int _{\mathbb {R}^{N}}V_{\infty }(x) \vert \tau z+\zeta \vert ^{2}\,\mathrm{d}x \\& \qquad {} +\tau ^{2} \int _{\mathbb {R}^{N}}\frac{z\cdot H_{z}(x, z)V_{\infty }(x)- \vert H _{z}(x, z) \vert ^{2}}{V_{\infty }(x)}\,\mathrm{d}x,\quad \forall z\in E, \tau \in \mathbb {R}, \zeta \in E^{-}, \end{aligned}$$

which shows that (2.23) holds. □

Corollary 2.5

Suppose that (V), (S0), and (S1) are satisfied, and that $\inf V_{\infty }>0$. Then

$$\begin{aligned}& \bigl\Vert z^{+} \bigr\Vert ^{2}- \bigl\Vert z^{-}+\zeta \bigr\Vert ^{2}- \int _{\mathbb {R}^{N}}V_{\infty }(x) \vert z+ \zeta \vert ^{2}\,\mathrm{d}x \\& \quad \le - \Vert \zeta \Vert ^{2}- \int _{\mathbb {R}^{N}}\frac{z\cdot H_{z}(x, z)V_{ \infty }(x)- \vert H_{z}(x, z) \vert ^{2}}{V_{\infty }(x)}\,\mathrm{d}x, \quad \forall z\in \mathcal{N}^{-}, \zeta \in E^{-}. \end{aligned}$$

(2.24)

Employing a standard argument (see [37] and [40]), one can easily check the following lemma.

Lemma 2.6

Let (V), (S0), and (S1) be satisfied. Then Ψ is nonnegative and weakly sequentially lower semi-continuous, and $\Psi '$ is weakly sequentially continuous.

3 Proof of main results

Lemma 3.1

Let (V), (S0), (S1), and (S4) be satisfied. Then, for any $z\in E$,

$$ \Phi (z)\geq \Phi (\tau z+\zeta )+\frac{1}{2} \Vert \zeta \Vert ^{2} +\frac{1- \tau ^{2}}{2}\bigl\langle \Phi '(z),z \bigr\rangle -\tau \bigl\langle \Phi '(z),\zeta \bigr\rangle ,\quad \forall \zeta \in E^{-}, \tau \geq 0. $$

(3.1)

Proof

The proof is the same as the one of [35, Lemma 3.3], here we omit the details. □

From Lemma 3.1, we have the following two corollaries.

Corollary 3.2

Let (V), (S0), (S1), and (S4) be satisfied. Then, for any $z\in \mathcal{N}^{-}$,

$$ \Phi (z)\geq \Phi (\tau z+\zeta )+\frac{1}{2} \Vert \zeta \Vert ^{2}, \quad \forall \zeta \in E^{-}, \tau \geq 0. $$

(3.2)

Corollary 3.3

Let (V), (S0), (S1), and (S4) be satisfied. Then, for any $z\in E$,

$$ \Phi (z)\geq \Phi \bigl(\tau z^{+}\bigr)+\frac{\tau ^{2} \Vert z^{-} \Vert ^{2}}{2} + \frac{1- \tau ^{2}}{2}\bigl\langle \Phi '(z),z\bigr\rangle +\tau ^{2}\bigl\langle \Phi '(z),z ^{-}\bigr\rangle , \quad \forall \tau \geq 0. $$

(3.3)

Applying Corollary 3.2, we can prove the following lemma in the same way as [33, Lemma 2.6].

Lemma 3.4

Let (V), (S0), (S1), and (S4) be satisfied. Then

• (i) there exists $\rho >0$ such that $m:=\inf_{\mathcal{N} ^{-}}\Phi \geq \kappa :=\inf \{ \Phi (z):z\in E^{+}, \|z\|= \rho \} >0$;

• (ii) $\|z^{+}\|\geq \max \{ \|z^{-}\|,\sqrt{2m} \}$ for all $z\in \mathcal{N}^{-}$.

Define a set $E_{0}^{+}$ as follows:

$$ E_{0}^{+}= \biggl\{ z\in E^{+} \setminus \{0\} : \Vert z \Vert ^{2}- \Vert \zeta \Vert ^{2}- \int _{\mathbb {R}^{N}}V_{\infty }(x) \vert z+\zeta \vert ^{2}\,\mathrm{d}x< 0, \forall \zeta \in E^{-} \biggr\} . $$

(3.4)

Obviously, (S3) shows that the set $E_{0}^{+}$ is not empty.

Lemma 3.5

Let (V), (S1), and (S3) be satisfied. Then, for any $e\in E_{0}^{+}$, there is a constant $r_{1}>\rho$ such that $\sup \Phi (\partial Q)\leq 0$ for $r\ge r _{1}$, where

$$ Q=\bigl\{ \zeta +se: \zeta \in E^{-}, s\geq 0, \Vert \zeta +se \Vert \leq r\bigr\} . $$

(3.5)

Proof

It is sufficient to show $\Phi (z)\le 0$ for $z\in E^{-}\oplus \mathbb {R}e$ and $\|z\|\ge R$ for large R. Arguing indirectly, assume that, for some sequence $\{z_{n}\}\subset E^{-} \oplus \mathbb {R}^{+}e$ with $\|z_{n}\|\rightarrow \infty$, $\Phi (z_{n}) \geq 0$ for all $n\in \mathbb {N}$. Set $w_{n}=\frac{z_{n}}{\|z_{n}\|}=w_{n} ^{-}+s_{n} e$, then $\|w_{n}\|=1$. Passing to a subsequence, we may assume that $w_{n}\rightharpoonup w$ in E. By Lemma 2.1 and (2.18), then $w_{n}\rightarrow w$ a.e. on $\mathbb {R}^{N}$, $s_{n} \rightarrow s$ and

$$ 0\le \frac{\Phi (z_{n})}{ \Vert z_{n} \Vert ^{2}} =\frac{1}{2}s^{2}_{n} \Vert e \Vert ^{2}-\frac{1}{2} \bigl\Vert w^{-}_{n} \bigr\Vert ^{2}- \int _{\mathbb {R}^{N}} \frac{H(x,z_{n})}{ \vert z _{n} \vert ^{2}} \vert w_{n} \vert ^{2}\,\mathrm{d}x. $$

(3.6)

If $s=0$, it follows from (S1) and (3.6) that

$$ 0\leq \frac{1}{2} \bigl\Vert w^{-}_{n} \bigr\Vert ^{2}+ \int _{\mathbb {R}^{N}} \frac{H(x,z_{n})}{ \vert z _{n} \vert ^{2}} \vert w_{n} \vert ^{2}\,\mathrm{d}x \leq \frac{s_{n}^{2}}{2} \Vert e \Vert ^{2} \rightarrow 0, $$

which yields $\|w^{-}_{n}\|\rightarrow 0$, and so $1=\|w_{n}\|\rightarrow 0$, a contradiction.

If $s\neq 0$, then $w\neq 0$. Since $e\in E_{0}^{+}$, there exists a bounded domain $\Omega \subset \mathbb {R}^{N}$ such that

$$ s^{2} \Vert e \Vert ^{2}- \bigl\Vert w^{-} \bigr\Vert ^{2}- \int _{\Omega }V_{\infty }(x) \bigl\vert s e+w^{-} \bigr\vert ^{2} \,\mathrm{d}x< 0. $$

(3.7)

By (S3), $H(x, z)=\frac{1}{2}V_{\infty }(x)|z|^{2}+\bar{H}(x, z)$. It follows from (3.6) that

$$\begin{aligned} 0 \le & \frac{s_{n}^{2}}{2} \Vert e \Vert ^{2}-\frac{1}{2} \bigl\Vert w_{n}^{-} \bigr\Vert ^{2}- \int _{\Omega }\frac{H(x, z_{n})}{ \Vert z_{n} \Vert ^{2}}\,\mathrm{d}x \\ = & \frac{s_{n}^{2}}{2} \Vert e \Vert ^{2}-\frac{1}{2} \bigl\Vert w_{n}^{-} \bigr\Vert ^{2}- \frac{1}{2} \int _{\Omega }V_{\infty }(x) \vert w_{n} \vert ^{2}\,\mathrm{d}x - \int _{\Omega }\frac{\bar{H}(x, z_{n})}{ \Vert z_{n} \Vert ^{2}}\,\mathrm{d}x. \end{aligned}$$

Clearly, $|\bar{H}(x, z)|\le C_{1}|z|^{2}$ for some $C_{1}>0$ and $|\bar{H}(x, z)|/|z|^{2} \rightarrow 0$ as $|z|\to \infty$. Since $w_{n} \rightharpoonup w$ in E, then $w_{n} \rightarrow w$ in $L^{2}(\Omega )$, and it is easy to see from the Lebesgue dominated convergence theorem that

$$ \int _{\Omega }\frac{H(x, z_{n})}{ \Vert z_{n} \Vert ^{2}}\,\mathrm{d}x= \int _{ \Omega }\frac{F(x, z_{n})}{ \vert z_{n} \vert ^{2}} \vert w_{n} \vert ^{2}\,\mathrm{d}x=o(1). $$

Hence

$$ 0\le s^{2} \Vert e \Vert ^{2}- \bigl\Vert w^{-} \bigr\Vert ^{2}- \int _{\Omega }V_{\infty }(x) \bigl\vert s e+w ^{-} \bigr\vert ^{2}\,\mathrm{d}x, $$

a contradiction to (3.7). □

Lemma 3.6

Let (V), (S1), (S2), and (SQ) be satisfied. Then, for any $e\in E^{+}$ with $\|e\|=1$, there is a constant $r_{1}>\rho$ such that $\sup \Phi (\partial Q)\leq 0$ for $r\ge r_{1}$, where

$$ Q=\bigl\{ \zeta +se: \zeta \in E^{-}, s\geq 0, \Vert \zeta +se \Vert \leq r\bigr\} . $$

(3.8)

Proof

The argument is similar to the proof of Lemma 3.5, we only need to modify the last part of it. Indeed, for the case $s\neq 0$, it follows from (S1), (SQ), (3.6), and Fatou’s lemma that

$$\begin{aligned} 0 \leq & \limsup_{n\rightarrow \infty } \biggl[\frac{s^{2}_{n}}{2}- \frac{1}{2} \bigl\Vert w^{-}_{n} \bigr\Vert ^{2}- \int _{\mathbb {R}^{N}} \frac{H(x,z_{n})}{ \vert z_{n} \vert ^{2}} \vert w_{n} \vert ^{2}\,\mathrm{d}x \biggr] \\ \leq &\lim_{n\rightarrow \infty }\frac{s^{2}_{n}}{2}- \liminf _{n\rightarrow \infty } \int _{\mathbb {R}^{N}} \frac{H(x,z_{n})}{ \vert z _{n} \vert ^{2}} \vert w_{n} \vert ^{2}\,\mathrm{d}x \\ \leq & \frac{s^{2}}{2}- \int _{\mathbb {R}^{N}}\liminf_{n\rightarrow \infty } \frac{H(x,z _{n})}{ \vert z_{n} \vert ^{2}} \vert w_{n} \vert ^{2}\,\mathrm{d}x = -\infty , \end{aligned}$$

a contradiction. □

By virtue of Lemmas 3.4 and 3.5, one can prove the following lemma by using the same argument as in the proof of [33, Lemma 2.6].

Lemma 3.7

Let (V), (S1), (S3), and (S4) be satisfied. Then, for any $z\in E_{0}^{+}$, $\mathcal{N}^{-} \cap (E^{-}\oplus \mathbb {R}^{+}z)\ne \emptyset$, i.e., there exist $\tau (z)>0$ and $w(z)\in E^{-}$ such that $\tau (z)z+w(z)\in \mathcal{N}^{-}$.

Applying Lemmas 2.3, 2.6, 3.4(i), 3.5, and 3.6, one gets directly the following.

Lemma 3.8

Under (V) and (S1), either (S2), (SQ) are satisfied or (S3) holds. Then there exist a constant $c\in [\kappa , \sup \Phi (Q)]$ and a sequence $\{z_{n}\}\subset E$ such that

$$ \Phi (z_{n})\rightarrow c, \qquad \bigl\Vert \Phi '(z_{n}) \bigr\Vert \bigl(1+ \Vert z_{n} \Vert \bigr)\rightarrow 0, $$

(3.9)

where Q is defined by (3.8).

The following lemma shows that a minimizing Cerami sequence for the energy functional can be found outside the Nehari–Pankov manifold, from which one can easily demonstrate a ground state solution of Nehari–Pankov type for problem (1.1).

Lemma 3.9

Let (V), (S1), (S3), and (S4) be satisfied. Then there exist a constant $c_{*}\in [\kappa , m]$ and a sequence $\{z_{n}\}\subset E$ satisfying

$$ \Phi (z_{n})\rightarrow c_{*}, \qquad \bigl\Vert \Phi '(z_{n}) \bigr\Vert \bigl(1+ \Vert z_{n} \Vert \bigr)\rightarrow 0. $$

(3.10)

Proof

The proof is essentially contained in [34], we omit it here. □

Lemma 3.10

Let (V), (S1), and (S3) be satisfied. Then for any sequence $\{z_{n}\}\subset E$ satisfying (3.10) is bounded in E.

Proof

To prove the boundedness of $\{z_{n}\}$, arguing by contradiction, suppose that $\|z_{n}\| \to \infty$. Let $w_{n}=z_{n}/ \|z_{n}\|$, then $\|w_{n}\|=1$. By Lemma 2.1, there exists a constant $C_{2}>0$ such that $\|w^{+}_{n}\|_{2} \le C_{2}$. If

$$ \delta :=\limsup_{n\to \infty }\sup_{y\in \mathbb {R}^{N}} \int _{B_{1}(y)} \bigl\vert w _{n}^{+} \bigr\vert ^{2}\,\mathrm{d}x=0, $$

then by Lions’s concentration compactness principle [24] or [37, Lemma 1.21] (usually this lemma is stated for $\{z_{n}\} \subset H^{1}(\mathbb {R}^{N})$; however, a simple modification of the argument in [24] shows that the conclusion remains valid for E), $w_{n}^{+}\rightarrow 0$ in $L^{s}(\mathbb {R}^{N},\mathbb {R}^{2})$ for $2< s<2^{*}$. Fix $R>[2(1+c)]^{1/2}$. By virtue of (2.16), for $\varepsilon =1/4(RC _{2})^{2}>0$, we have

$$ \limsup_{n\to \infty } \int _{\mathbb {R}^{N}}H\bigl(x, Rw_{n}^{+}\bigr) \,\mathrm{d}x \le \varepsilon (RC_{2})^{2} +R^{p}C_{\varepsilon } \lim_{n\to \infty } \bigl\Vert w_{n}^{+} \bigr\Vert _{p}^{p}=\frac{1}{4}. $$

(3.11)

Let $\tau _{n}=R/\|z_{n}\|$. Hence, by virtue of (3.10), (3.11), and Corollary 3.3, one can get that

$$\begin{aligned} c+o(1) = & \Phi (z_{n}) \ge \frac{\tau _{n}^{2}}{2} \bigl( \bigl\Vert z_{n} ^{+} \bigr\Vert ^{2}+ \bigl\Vert z_{n}^{-} \bigr\Vert ^{2} \bigr)- \int _{\mathbb {R}^{N}}H\bigl(x, \tau _{n}z_{n} ^{+}\bigr)\,\mathrm{d}x +\frac{1-\tau _{n}^{2}}{2}\bigl\langle \Phi '(z_{n}), z _{n} \bigr\rangle \\ &{} +\tau _{n}^{2}\bigl\langle \Phi '(z_{n}), z_{n}^{-} \bigr\rangle \\ = & \frac{R^{2}}{2} \bigl( \bigl\Vert w_{n}^{+} \bigr\Vert ^{2}+ \bigl\Vert w_{n}^{-} \bigr\Vert ^{2} \bigr)- \int _{\mathbb {R}^{N}}H\bigl(x, Rw_{n}^{+}\bigr) \,\mathrm{d}x + \biggl(\frac{1}{2}-\frac{R ^{2}}{2 \Vert z_{n} \Vert ^{2}} \biggr)\bigl\langle \Phi '(z_{n}), z_{n} \bigr\rangle \\ & {} +\frac{R^{2}}{ \Vert z_{n} \Vert ^{2}}\bigl\langle \Phi '(z_{n}), z_{n}^{-}\bigr\rangle \\ = & \frac{R^{2}}{2}- \int _{\mathbb {R}^{N}}H\bigl(x, Rw_{n}^{+}\bigr) \,\mathrm{d}x+o(1) \ge \frac{R^{2}}{2}-\frac{1}{4} +o(1) > c+ \frac{3}{4} +o(1). \end{aligned}$$

This contradiction shows that $\delta >0$.

Going if necessary to a subsequence, we may assume the existence of $k_{n}\in \mathbb {Z}^{N}$ such that $\int _{B_{1+\sqrt{N}}(k_{n})}|w_{n} ^{+}|^{2}\,\mathrm{d}x> \frac{\delta }{2}$. Let $\tilde{w}_{n}(x)=w_{n}(x+k _{n})$. Since $V(x)$ is 1-periodic in x, then $\|\tilde{w}_{n}\|=\|w _{n}\|=1$, and

$$ \int _{B_{1+\sqrt{N}}(0)} \bigl\vert \tilde{w}_{n}^{+} \bigr\vert ^{2}\,\mathrm{d}x> \frac{ \delta }{2}. $$

(3.12)

Passing to a subsequence, we have $\tilde{w}_{n}\rightharpoonup \tilde{w}$ in E, $\tilde{w}_{n}\rightarrow \tilde{w}$ in $L^{s}_{ \mathrm{loc}}(\mathbb {R}^{N},\mathbb {R}^{2})$, $2\le s<2^{*}$, $\tilde{w}_{n}\rightarrow \tilde{w}$ a.e. on $\mathbb {R}^{N}$. It follows from (3.12) that $\tilde{w}^{+}\ne 0$, and so $\tilde{w}\ne 0$.

Now we define ${\tilde{z}}_{n}(x)=z_{n}(x+k_{n})$, then ${\tilde{z}} _{n}/\|z_{n}\|=\tilde{w}_{n}\rightarrow \tilde{w}$ a.e. on $\mathbb {R}^{N}$, $\tilde{w}\ne 0$. For $x\in \Omega :=\{y\in \mathbb {R}^{N} : \tilde{w}(y) \ne 0\}$, we have $\lim_{n\to \infty }|{\tilde{z}}_{n}(x)|=\infty$. For any $\varsigma \in C_{0}^{\infty }(\mathbb {R}^{N},\mathbb {R}^{2})$, setting $\varsigma _{n}(x)=\varsigma (x-k_{n})$, then

$$\begin{aligned} \bigl\langle \Phi '(z_{n}), \varsigma _{n} \bigr\rangle = & \bigl(z_{n}^{+}-z _{n}^{-}, \varsigma _{n} \bigr)-(V_{\infty }z_{n}, \varsigma _{n})_{2}- \int _{\mathbb {R}^{N}}\bar{H}_{z}(x, z_{n})\cdot \varsigma _{n}\,\mathrm{d}x \\ = & \Vert z_{n} \Vert \biggl[ \bigl(w_{n}^{+}-w_{n}^{-}, \varsigma _{n} \bigr)-(V _{\infty }w_{n}, \varsigma _{n})_{2} - \int _{\mathbb {R}^{N}}\frac{\bar{H}_{z}(x, z_{n})}{ \vert z_{n} \vert }\cdot \varsigma _{n} \vert w_{n} \vert \,\mathrm{d}x \biggr] \\ = & \Vert z_{n} \Vert \biggl[ \bigl(\tilde{w}_{n}^{+}- \tilde{w}_{n}^{-}, \varsigma \bigr)-(V_{\infty } \tilde{w}_{n}, \varsigma )_{2} - \int _{\mathbb {R}^{N}}\frac{\bar{H}_{z}(x, \tilde{z}_{n})}{ \vert \tilde{z}_{n} \vert } \cdot \varsigma \vert \tilde{w}_{n} \vert \,\mathrm{d}x \biggr], \end{aligned}$$

which, together with (3.10), yields that

$$ \bigl(\tilde{w}_{n}^{+}- \tilde{w}_{n}^{-}, \varsigma \bigr)-(V_{ \infty } \tilde{w}_{n}, \varsigma )_{2} - \int _{\mathbb {R}^{N}}\frac{\bar{H} _{z}(x, \tilde{z}_{n})}{ \vert \tilde{z}_{n} \vert }\cdot \varsigma \vert \tilde{w} _{n} \vert \,\mathrm{d}x=o(1). $$

(3.13)

Note that, for some constant $C>0$, there holds

$$\begin{aligned} \biggl\vert \int _{\mathbb {R}^{N}}\frac{\bar{H}_{z}(x, \tilde{z}_{n})}{ \vert \tilde{z} _{n} \vert }\cdot \varsigma \biggr\vert \tilde{w}_{n} \vert \,\mathrm{d}x \vert \le & \int _{\mathbb {R}^{N}} \biggl\vert \frac{\bar{H}_{z}(x, \tilde{z}_{n})}{ \vert \tilde{z} _{n} \vert } \biggr\vert \vert \tilde{w}_{n}-\tilde{w} \vert \vert \varsigma \vert \,\mathrm{d}x + \int _{\mathbb {R}^{N}} \biggl\vert \frac{\bar{H}_{z}(x, \tilde{z}_{n})}{\tilde{z} _{n}} \biggr\vert \vert \tilde{w} \vert \vert \varsigma \vert \,\mathrm{d}x \\ \le & C \int _{\mathrm{supp} \varsigma } \vert \tilde{w}_{n}-\tilde{w} \vert \vert \varsigma \vert \,\mathrm{d}x + \int _{\Omega } \biggl\vert \frac{\bar{H}_{z}(x, \tilde{z}_{n})}{\tilde{z}_{n}} \biggr\vert \vert \tilde{w} \vert \vert \varsigma \vert \,\mathrm{d}x=o(1). \end{aligned}$$

Hence,

$$ \bigl(\tilde{w}^{+}-\tilde{w}^{-}, \varsigma \bigr)-(V_{\infty } \tilde{w}, \varsigma )_{2}=0, \quad \forall \varsigma =(\varphi ,\psi )\in C_{0}^{\infty }\bigl(\mathbb {R}^{N},\mathbb {R}^{2}\bigr). $$

(3.14)

Let $\tilde{w}=(u,v)$, then $\tilde{w}^{+}-\tilde{w}^{-}= (u^{+}-u ^{-},v^{-}-v^{+})$. Thus from (3.14) we have

$$ \bigl(u^{+}-u^{-}, \varphi \bigr)_{H}+ \bigl(v^{-}-v^{+},\psi \bigr)_{H}-(V_{\infty }u, \varphi )_{L^{2}}-(V_{\infty }v,\psi )_{L^{2}}=0, $$

which implies that

$$ \textstyle\begin{cases} -\Delta u+V(x)u =V_{\infty }u, \\ \Delta v+V(x)v=V_{\infty }v. \end{cases} $$

Hence u is an eigenfunction of operator $\mathcal{B}_{1}:=-\triangle +(V-V_{\infty })$, and v is an eigenfunction of operator $\mathcal{B}_{2}:=-\triangle +(V_{\infty }-V)$, which contradicts the fact that $\mathcal{B}_{i}$ has only continuous spectrum for $i=1,2$ since $V-V_{\infty }$ is 1-periodic. This contradiction shows that $\{z_{n}\}$ is bounded. □

Lemma 3.11

Let (V), (S1), (S2), and (SQ) be satisfied. Then for any sequence $\{z_{n}\}\subset E$ satisfying

$$ \Phi (z_{n})\rightarrow c\ge 0, \qquad \bigl\langle \Phi '(z_{n}),z^{\pm }_{n} \bigr\rangle \rightarrow 0, $$

(3.15)

is bounded in E.

Proof

By virtue of (S4) and (3.15), there exists a constant $C_{3}>0$ such that

$$ C_{3}\ge \Phi (z_{n})-\frac{1}{2} \bigl\langle \Phi '(z_{n}),z_{n} \bigr\rangle = \int _{\mathbb {R}^{N}}\hat{H}(x,z_{n})\,\mathrm{d}x\ge 0. $$

(3.16)

To prove the boundedness of $\{z_{n}\}$, arguing by contradiction, suppose that $\|z_{n}\| \to \infty$. Let $w_{n}=z_{n}/\|z_{n}\|$, then $\|w_{n}\|=1$. If $\delta :=\limsup_{n\to \infty }\sup_{y\in \mathbb {R}^{N}} \int _{B_{1}(y)}|w_{n}^{+}|^{2}\,\mathrm{d}x=0$, then by Lions’s concentration compactness principle, $w_{n}^{+}\rightarrow 0$ in $L^{s}(\mathbb {R}^{N},\mathbb {R}^{2})$ for $2< s<2^{*}$. Set $\sigma '={\sigma }/( \sigma -1)$ and

$$ \Omega _{n}:= \biggl\{ x\in \mathbb {R}^{N}: \frac{ \vert H_{z}(x,z) \vert }{ \vert z \vert }\le \frac{1}{2} (\Lambda _{0}-\delta _{0} ) \biggr\} . $$

(3.17)

Then $2\sigma '\in (2,2^{*})$ since $\sigma >\max \{1,N/2\}$. Note that $\Lambda _{0}\|z\|_{2}^{2}\le \|z\|^{2}$ for any $z\in E$, then

$$ \int _{\Omega _{n}}\frac{ \vert H_{z}(x,z_{n}) \vert }{ \vert z_{n} \vert } \vert w_{n} \vert \bigl\vert w^{+}_{n} \bigr\vert \,\mathrm{d}x \le \frac{1}{2}(\Lambda _{0}-\delta _{0}) \bigl\Vert w_{n}^{+} \bigr\Vert _{2} \Vert w_{n} \Vert _{2}\le \frac{1}{2}-\frac{\delta _{0}}{2\Lambda _{0}}. $$

(3.18)

On the other hand, by virtue of (S4), (3.16), Lemma 2.1, and the Hölder inequality, there exists a constant $C_{4}>0$ such that

$$\begin{aligned} \int _{\mathbb {R}^{N}\setminus {\Omega _{n}}}\frac{ \vert H_{z}(x,z_{n}) \vert }{ \vert z_{n} \vert } \vert w _{n} \vert \bigl\vert w^{+}_{n} \bigr\vert \,\mathrm{d}x \le & \biggl[ \int _{\mathbb {R}^{N}\setminus {\Omega _{n}}} \biggl(\frac{ \vert H_{z}(x,z_{n}) \vert }{ \vert z_{n} \vert } \biggr)^{\sigma } \biggr] ^{1/\sigma } \bigl\Vert w_{n}^{+} \bigr\Vert _{2\sigma '} \Vert w_{n} \Vert _{2\sigma '} \\ \le & C_{4} \biggl( \int _{\mathbb {R}^{N}\setminus {\Omega _{n}}}C_{0}\hat{H}(x,z _{n}) \,\mathrm{d}x \biggr)^{1/\sigma } \bigl\Vert w_{n}^{+} \bigr\Vert _{2\sigma '} \\ \le & C_{4}(C_{0}C_{3})^{1/\sigma } \bigl\Vert w_{n}^{+} \bigr\Vert _{2\sigma '}=o(1). \end{aligned}$$

(3.19)

In view of (S1), (2.18), and (3.15), one has

$$ 2c+o(1) = \bigl\Vert z_{n}^{+} \bigr\Vert ^{2}- \bigl\Vert z_{n}^{-} \bigr\Vert ^{2}-2 \int _{\mathbb {R}^{N}}H(x,z _{n})\,\mathrm{d}x\le \bigl\Vert z_{n}^{+} \bigr\Vert ^{2}- \bigl\Vert z_{n}^{-} \bigr\Vert ^{2}. $$

(3.20)

Combining (3.18) and (3.19) and using (2.19), (3.15), and (3.20), we have

$$\begin{aligned} \frac{1}{2}+o(1) \le & \frac{ \Vert z_{n}^{+} \Vert ^{2}-\langle \Phi '(z_{n}),z _{n}^{+}\rangle }{ \Vert z_{n} \Vert ^{2}} \\ = & \frac{1}{ \Vert z_{n} \Vert } \int _{ \vert z^{+}_{n} \vert \neq 0}\frac{H_{z}(x,z_{n}) \cdot z_{n}^{+}}{ \vert z^{+}_{n} \vert } \bigl\vert w^{+}_{n} \bigr\vert \,\mathrm{d}x \\ \le & \int _{ \vert z_{n} \vert \neq 0} \frac{ \vert H_{z}(x,z_{n}) \vert }{ \vert z_{n} \vert } \vert w_{n} \vert \bigl\vert w ^{+}_{n} \bigr\vert \,\mathrm{d}x \\ \le & \int _{\Omega _{n}}\frac{ \vert H_{z}(x,z_{n}) \vert }{ \vert z_{n} \vert } \vert w_{n} \vert \bigl\vert w ^{+}_{n} \bigr\vert \,\mathrm{d}x + \int _{\mathbb {R}^{N}\setminus {\Omega _{n}}}\frac{ \vert H _{z}(x,z_{n}) \vert }{ \vert z_{n} \vert } \vert w_{n} \vert \bigl\vert w^{+}_{n} \bigr\vert \,\mathrm{d}x \\ \le & \frac{1}{2}-\frac{\delta _{0}}{2\Lambda _{0}}+ o(1). \end{aligned}$$

(3.21)

This contradiction shows that $\delta >0$. The rest of the argument is standard, so we omit it. □

We first prove Theorem 1.4, then sketch the proof of Theorem 1.3.

Proof of Theorem 1.4

Applying Lemmas 3.9 and 3.10, there exists a bounded sequence $\{z_{n}\}\subset E$ satisfying (3.10). By Lemma 2.1, there is a constant $C_{5}>0$ such that $\|z_{n}\|^{2} _{2}+\|z_{n}\|^{p}_{p}\le C_{5}$. If

$$ \delta :=\limsup_{n\rightarrow \infty }\sup_{y\in \mathbb {R}^{N}} \int _{B(y,1)} \vert z _{n} \vert \,\mathrm{d}x=0, $$

then by Lions’s concentration compactness principle, $z_{n}\rightarrow 0$ in $L^{s}(\mathbb {R}^{N},\mathbb {R}^{2})$ for $2< s<2^{*}$. From (S1), (2.16), (2.18), (2.19), and (3.10), one has

$$\begin{aligned} 2c_{*}+o(1) =& \bigl\Vert z_{n}^{+} \bigr\Vert ^{2}- \bigl\Vert z_{n}^{-} \bigr\Vert ^{2}-2 \int _{\mathbb {R}^{N}}H(x,z _{n})\,\mathrm{d}x \\ \leq & \bigl\Vert z_{n}^{+} \bigr\Vert ^{2}= \bigl\langle \Phi '(z_{n}),z_{n}^{+} \bigr\rangle + \int _{\mathbb {R}^{N}}H_{z}(x,z_{n})z^{+}_{n} \,\mathrm{d}x \\ \leq & \varepsilon \Vert z_{n} \Vert _{2} \bigl\Vert z_{n}^{+} \bigr\Vert _{2}+ C_{\varepsilon } \Vert z_{n} \Vert ^{p-1}_{p} \bigl\Vert z_{n}^{+} \bigr\Vert _{p}+o(1) \\ \leq & \varepsilon C_{5}+o(1). \end{aligned}$$

This is a contradiction since $\varepsilon >0$ is arbitrary. Thus $\delta >0$.

Going if necessary to a subsequence, we may assume the existence of $k_{n}\in \mathbb {Z}^{N}$ such that

$$ \int _{B(k_{n},1+\sqrt{N})} \vert z_{n} \vert ^{2} \,\mathrm{d}x>\frac{\delta }{2}. $$

Let us define $w_{n}(x)=z_{n}(x+k_{n})$ so that

$$ \int _{B(0,1+\sqrt{N})} \vert w_{n} \vert ^{2}> \frac{\delta }{2}. $$

(3.22)

Since $V(x)$, $H(x,z)$ are periodic in x, together with (3.10), we have $\|w_{n}\|=\|z_{n}\|$ and

$$ \Phi (w_{n})\rightarrow c_{*}\in [\kappa , m], \qquad \bigl\Vert \Phi '(w_{n}) \bigr\Vert \bigl(1+ \Vert w_{n} \Vert \bigr)\rightarrow 0. $$

(3.23)

Passing to a subsequence, we have $w_{n}\rightharpoonup w_{0}$ in E, $w_{n}\rightarrow w_{0}$ in $L^{s}_{\mathrm{loc}}(\mathbb {R}^{N},\mathbb {R}^{2})$ for $2\leq s<2^{*}$ and $w_{n}\rightarrow w_{0}$ a.e. on $\mathbb {R}^{N}$. Hence it follows from (3.22) and (3.23) that $\Phi '(w_{0})=0$ and $w_{0}\neq 0$. This shows that $w_{0}\in \mathcal{N}^{-}$, and so $\Phi (w_{0})\geq m$. On the other hand, by (S1),(S4), (2.18), (2.19), (3.23), and Fatou’s lemma, we have

$$\begin{aligned} m \geq & c_{*}=\lim_{n\rightarrow \infty } \biggl[\Phi (w_{n})- \frac{1}{2}\bigl\langle \Phi '(w_{n}),w_{n} \bigr\rangle \biggr] \\ =&\lim_{n\rightarrow \infty } \int _{\mathbb {R}^{N}}\hat{H}(x,w_{n}) \,\mathrm{d}x \geq \int _{\mathbb {R}^{N}}\liminf_{n\rightarrow \infty }\hat{H}(x,w _{n})\,\mathrm{d}x \\ =& \int _{\mathbb {R}^{N}}\hat{H}(x,w_{0})\,\mathrm{d}x =\Phi (w_{0})- \frac{1}{2}\bigl\langle \Phi '(w_{0}),w_{0} \bigr\rangle \\ = & \Phi (w_{0}). \end{aligned}$$

This shows that $\Phi (w_{0})\leq m$, and so $\Phi (w_{0})=m= \inf_{\mathcal{N}^{-}}\Phi \ge \kappa >0$ by Lemma 3.4(i). □

Proof of Theorem 1.3

It follows from Lemma 2.2 that (S1), (SQ), and (S2) yield (S0). Applying Lemmas 3.8 and 3.11, there exists a bounded sequence $\{z_{n}\}\subset E$ satisfying (3.9). Similar to the argument as in the proof of Theorem 1.3, we can show that $\Phi '( \bar{z})=0$ for some $\bar{z}\in E\setminus \{(0,0)\}$, i.e., $\mathcal{M}\neq 0$. Let $\hat{c}:=\inf_{\mathcal{M}}\Phi$. By (S4), for any $z\in \mathcal{M}$, one has

$$ \Phi (z)=\Phi (z)-\frac{1}{2}\bigl\langle \Phi '(z),z\bigr\rangle = \int _{\mathbb {R}^{N}}\hat{H}(x,z)\,\mathrm{d}x\ge 0, $$

therefore $\hat{c}\ge 0$. Let $\{z_{n}\}\subset \mathcal{M}$ such that $\Phi (z_{n})\to \hat{c}$. Then $\langle \Phi '(z_{n}),\zeta \rangle =0$ for any $\zeta \in E$. It follows from Lemma 3.11 that $\{z_{n}\}$ is bounded in E. The rest of the argument is the same as in the proof of Theorem 1.4 by using (S2) instead of (S4). □

Appendix

Here, we prove that (G4) holds under (G1), (G4′), and (G₂). It is equivalent to showing that (S4′) together with (S1), (S3′) yields (S4). For any $z, \zeta \in \mathbb {R}^{2}$, let

$$ h(t):=\frac{1-t^{2}}{2}H_{z}(x, z)\cdot z-tH_{z}(x, z) \cdot \zeta +H(x,tz+ \zeta )-H(x, z), \quad \forall t\ge 0. $$

It is sufficient to show that $h(t)\ge 0$ for $t\ge 0$. If $z=0$, then $h(t)=H(x,\zeta )\ge 0$ by (S1). Assume $z\neq 0$. It follows from (S1), (S4′) and $z\cdot \bar{H}_{z}(x, z)< 0$ for $z\neq 0$ (deduced from (S3′)) that

$$ h(0)=\frac{1}{2}H_{z}(x, z)\cdot z-H(x, z)+H(x,\zeta )\ge 0,\quad \text{and}\quad h(t)\ge 0 \quad \text{for any } t \text{ large}. $$

Assume that $h(t)$ reaches a minimum at some point $t_{0}\in [0, \infty )$. Clearly, $h(t)\ge 0$ for any $t\ge 0$ if $t_{0}=0$. Assume $t_{0}>0$. Then $h'(t_{0})=0$, i.e.,

$$ H_{z}(x,t_{0}z+\zeta )\cdot z-H_{z}(x, z)\cdot (t_{0}z+\zeta )=0. $$

If $H_{z}(x,t_{0}z+\zeta )\cdot z=0$, (S1) and (S4′) imply that

$$ h(t_{0})=\frac{1+t_{0}^{2}}{2}H_{z}(x, z)\cdot z-H(x, z)+H(x,t_{0}z+ \zeta )\ge 0. $$

If $H_{z}(x,t_{0}z+\zeta )\cdot z\neq 0$, (S4′) yields that $H_{z}(x,z)\cdot z\neq 0$. Using (S1) and (S4′) again, we have

$$\begin{aligned} h(t_{0}) \ge & \frac{1-t_{0}^{2}}{2}H_{z}(x, z)\cdot z-t_{0}H_{z}(x, z)\cdot \zeta - \bigl[{2H_{z}(x,z) \cdot z} \bigr]^{-1} \\ &{}\times \bigl\{ \bigl[H_{z}(x,z)\cdot z \bigr]^{2}- \bigl[H_{z}(x,z) \cdot (t_{0}z+ \zeta ) \bigr]^{2} \bigr\} \\ \ge & \bigl[{2H_{z}(x,z)\cdot z} \bigr]^{-1}{ \bigl[t_{0}H_{z}(x,z) \cdot z- H_{z}(x,z)\cdot (t_{0}z+ \zeta ) \bigr]^{2}} \\ \ge &0. \end{aligned}$$

Then $h(t)\ge 0$ for $t\ge 0$, and so (S4) holds.