The Schrödinger–Bopp–Podolsky Equation Under the Effect of Nonlinearities

Abstract

In this paper, we study the following nonlinear Schrödinger–Bopp–Podolsky system

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta u+V(x)u+q\phi u=f(u)&{}\\ -\Delta \phi +a^2\Delta ^2\phi =4\pi u^2 \end{array} \right. \ \,\mathrm{in }\,{\mathbb {R}}^3, \end{aligned}$$

where $a>0$, $q>0$ and $V\in {\mathcal {C}}({\mathbb {R}}^3,{\mathbb {R}})$. By means of the variational methods, we prove the existence of infinitely many nontrivial solutions, the existence of a ground state solution for $f(u)=|u|^{p-2}u+h(u)$ with $p\in [4,6)$ and the existence of at least one positive solution for $f(u)=P(x)u^5+\mu |u|^{p-2}u$ with $p\in (2,6)$ under some certain assumptions.

1 Introduction

In the recent paper [12], the following Schrödinger–Bopp–Podolsky system has been studied for the first time

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta u+\omega u+q^2\phi u=|u|^{p-2}u&{}\\ -\Delta \phi +a^2\Delta ^2\phi =4\pi u^2 \end{array}\right. \ \,\mathrm{in }\,{\mathbb {R}}^3, \end{aligned}$$

(1.1)

where $a, \omega >0$, $q\ne 0$ and $p\in (2,6)$. The system appears when one couples the Schrödinger field $\Psi =\Psi (t,x)$ with the Bopp–Podolsky Lagrangian of the electromagnetic field and considers standing wave $\Psi (t,x)=\mathrm{e}^{iwt}u(x)$ in the purely electrostatic case. The Bopp–Podolsky theory, which is a second-order gauge theory of the electromagnetic field, was developed by Bopp [3] and then independently by Podolsky [20]. According to Mie theory [19] and its generalizations in [4,5,6,7], the Bopp–Podolsky theory was introduced to solve the alleged infinity problem in classical Maxwell theory.

d’Avenia and Siciliano [12] used variational methods to prove the existence results of problem (1.1). Indeed, the solutions can be found as critical points of a smooth energy functional. When $p \in (2,6)$ and |q| small enough or $p \in (3,6)$ and $q \ne 0$, they proved that the energy functional has the mountain pass geometry; hence, the above system has a nontrivial solution. In addition, they proved the above system does not admit any nontrivial solution for $p\ge 6$ by using Pohozaev-type identity, and in the radial case, as $a\rightarrow 0$, the solutions they got tend to solutions of the classical Schrödinger–Poisson system. They also showed that, if $\rho$ is the distribution density of the given charge, then the electrostatic potential $\Phi$ satisfies the following equation

$$\begin{aligned} -\Delta \Phi =\rho \quad \text {in }{\mathbb {R}}^3. \end{aligned}$$

(1.2)

If $\rho =4\pi \delta _{x_0}$ with $x_0\in {\mathbb {R}}^3$, then ${\mathcal {G}}(x-x_0)=\frac{1}{|x-x_0|}$ is the fundamental solution of (1.2). And the electrostatic energy is

$$\begin{aligned} \mathcal {\varepsilon }_M({\mathcal {G}})=\frac{1}{2}\int _{{\mathbb {R}}^3} |\nabla {\mathcal {G}}|^2 =+\infty . \end{aligned}$$

Hence, Eq. (1.2) is replaced by

$$\begin{aligned} -\Delta \phi +a^2\Delta ^2\phi =\rho \quad \,\mathrm{in }\,{\mathbb {R}}^3 \end{aligned}$$

in the Bopp–Podolsky theory. Moreover, we know that ${\mathcal {K}}(x-x_0)$ is the fundamental solution of the equation

$$\begin{aligned} -\Delta \phi +a^2\Delta ^2\phi =4\pi \delta _{x_0}, \end{aligned}$$

where

$$\begin{aligned} {\mathcal {K}}(x):=\frac{1-\mathrm{e}^{-|x|/a}}{|x|},\quad \lim \limits _{x\rightarrow x_0}{\mathcal {K}}(x-x_0)=\frac{1}{a}. \end{aligned}$$

And its energy is

$$\begin{aligned} \mathcal {\varepsilon }_{\mathrm{BP}}({\mathcal {K}})=\frac{1}{2}\int _{{\mathbb {R}}^3} |\nabla {\mathcal {K}}|^2+\frac{a^2}{2}\int _{{\mathbb {R}}^3}|\Delta {\mathcal {K}}|^2 <+\infty , \end{aligned}$$

more details can be found in [12].

Remark 1.1

The operator $-\Delta +\Delta ^2$ appears also in other different physical and mathematical problems (see [2, 14]).

After that, Gaetano and Kaye [15] supplemented and improved some results in [12]. It has been showed that (1.1) has no solution for large values of q’s and has two radial solutions for small q’s.

Chen and Tang [10] extended the subcritical case to more general cases and dealt with the following system

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta u+V(x) u+\phi u=\mu f(u)+u^5&{}\\ -\Delta \phi +a^2\Delta ^2\phi =4\pi u^2 \end{array}\right. \ \,\mathrm{in }\,{\mathbb {R}}^3, \end{aligned}$$

where $a>0$, $\mu >0$, $V\in {\mathcal {C}}({\mathbb {R}}^3,[0,\infty ))$ with $V_\infty =\lim _{|y|\rightarrow \infty }V(y)\ge \text {sup}_{x\in {\mathbb {R}}^3}V(x)>0$, and $f\in {\mathcal {C}}({\mathbb {R}},{\mathbb {R}})$ satisfies that there exists a constant $p\in (2,6)$ such that $\int ^t_0f(s){\mathrm{d}}s\ge t^p$ for all $t\ge 0$. They obtained the existence of ground state solutions for $p\in (4,6)$ and $\mu >0$ or $p\in (2,4]$ and $\mu >\mu _*$, where $\mu _*$ is a positive constant.

Motivated by the cited papers [10, 12], our aim here is to study the existence of ground state solutions, positive solutions and infinitely many nontrivial solutions for the following Schrödinger–Bopp–Podolsky system

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta u+V(x)u+q\phi u=f(u)&{}\\ -\Delta \phi +a^2\Delta ^2\phi =4\pi u^2 \end{array}\right. \ \,\mathrm{in }\,{\mathbb {R}}^3, \end{aligned}$$

(1.3)

where $a>0$, $q>0$ and V satisfies the following conditions:

$(V_1)$:

$V\in {\mathcal {C}}({\mathbb {R}}^3,{\mathbb {R}})$ and $\inf _{x\in {\mathbb {R}}^3}V(x)>0$;

$(V_2)$:

there exists a constant $d_0>0$ such that

$$\begin{aligned} \lim \limits _{|y|\rightarrow +\infty }\text {meas}\left\{ x\in {\mathbb {R}}^3:|x-y|\le d_0,V(x)\le M\right\} =0,\quad \forall M>0. \end{aligned}$$

We work in the Hilbert space

$$\begin{aligned} E:=\left\{ u\in H^1({\mathbb {R}}^3):\int _{{\mathbb {R}}^3}|\nabla u|^2+V(x)u^2{\mathrm{d}}x<+\infty \right\} , \end{aligned}$$

endowed with the norm

$$\begin{aligned} \Vert u\Vert :=\left( \int _{{\mathbb {R}}^3}|\nabla u|^2+V(x)u^2{\mathrm{d}}x\right) ^{\frac{1}{2}}. \end{aligned}$$

Theorem 1.2

[23] Assume that $\left( V_{1}\right)$–$\left( V_{2}\right)$ are satisfied. Then, E is continuously embedded into $L^s({\mathbb {R}}^3)$ for $s\in [2,6]$. Moreover, the embedding $E\hookrightarrow L^s({\mathbb {R}}^3)$ is compact for $s\in [2,6)$.

We first consider the following system:

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta u+V(x)u+\phi u=f(u)&{}\\ -\Delta \phi +a^2\Delta ^2\phi =4\pi u^2 \end{array} \right. \ \,\mathrm{in }\,{\mathbb {R}}^3. \end{aligned}$$

(1.4)

Assume that f is a continuous function and satisfies the following conditions:

$(f_{1})$:

$f(t)=-f(-t)$;

$(f_{2})$:

there exist $1<\iota <5$ such that $\lim _{|t|\rightarrow 0}\frac{f(t)}{|t|}=\lim _{|t|\rightarrow +\infty }\frac{f(t)}{|t|^\iota }=0$;

$(f_{3})$:

$\lim _{|t|\rightarrow +\infty }\frac{F(t)}{|t|^4}=\infty$, and there exists $\mu \ge 4, \kappa >0$ such that

$$\begin{aligned} \mu F(t) \le t f(t)+\kappa t^{2}, \end{aligned}$$

where $F(t)=\int _{0}^{t} f(r) \,{\mathrm {d}} r$.

Our first result is as follows.

Theorem 1.3

Assume that $\left( V_{1}\right)$–$\left( V_{2}\right)$ and $\left( f_{1}\right)$–$\left( f_{3}\right)$ are satisfied. Then, the problem (1.4) possesses infinitely many nontrivial solutions.

Remark 1.4

To our knowledge, there are few results about existence of multiply solutions to Schrödinger–Bopp–Podolsky equation.

Remark 1.5

In order to assure the boundedness of the Palais–Smale sequences of the energy functional, the following condition is usually supposed.

(AR) There exists $\mu >4$ such that

$$\begin{aligned} 0<\mu H(s)\le sh(s),\quad \text {for all}\quad s\ne 0. \end{aligned}$$

Obviously, $(f_3)$ is weaker than Ambrosetti–Rabinowitz condition.

We next consider the case of $f(u)=|u|^{p-2}u+h(u)$, namely the following system:

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta u+V(x)u+\phi u=|u|^{p-2}u+h(u)&{}\\ -\Delta \phi +a^2\Delta ^2\phi =4\pi u^2 \end{array} \right. \ \,\mathrm{in }\,{\mathbb {R}}^3, \end{aligned}$$

(1.5)

where $h\in {\mathcal {C}}({\mathbb {R}},{\mathbb {R}})$ satisfies the following conditions:

$\left( h_{1}\right)$:

for some $\varrho \in (1,5)$,

$$\begin{aligned} \lim \limits _{|u| \rightarrow +\infty } \frac{h(u)}{|u|^{\varrho }}=\lim \limits _{|u| \rightarrow 0} \frac{h(u)}{|u|}=0; \end{aligned}$$

$\left( h_{2}\right)$:

for any $p\in [4,6)$, $H(u)+\frac{1}{p}|u|^p\ge 0$, where $H(u)=\int _{0}^{u} h(r) \,{\mathrm {d}} r$,

$\left( h_{3}\right)$:

there exists $\theta _0\in (0,1)$ such that for any $t>0$ and $u\ne 0$,

$$\begin{aligned} \left[ \frac{h(u)}{u^3}-\frac{h(tu)}{(tu)^3}\right] \text {sign}(1-t)+\theta _0\frac{V(x)|1-t^2|}{(tu)^2}\ge 0. \end{aligned}$$

The following is the second result of our paper.

Theorem 1.6

Assume that $\left( V_{1}\right)$–$\left( V_{2}\right)$ and $\left( h_{1}\right)$–$\left( h_{3}\right)$ are satisfied. If $p\in [4,6)$, then the problem (1.5) has a ground state solution.

Remark 1.7

The condition $(h_3)$ comes from [11], and it is weaker than Ambrosetti–Rabinowitz condition and also weaker than the following condition:

$$\begin{aligned} \frac{h(u)}{|u|^3}\,\,\ \text {is nondecreasing on}\,\,\ (-\infty ,0)\cup (0,+\infty ). \end{aligned}$$

Finally, we consider the case of $f(u)=P(x)u^5+\mu |u|^{p-2}u$, namely the following system:

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta u+V(x)u+q\phi u=P(x)u^5+\mu |u|^{p-2}u&{}\\ -\Delta \phi +a^2\Delta ^2\phi =4\pi u^2 \end{array}\right. \ \,\mathrm{in }\,{\mathbb {R}}^3. \end{aligned}$$

(1.6)

We assume that P(x) is a continuous function and satisfies the following conditions:

$\left( P_{1}\right)$:

$P(x)> 0$ and $P(x)\in L^{\infty }({\mathbb {R}}^3)$;

$\left( P_{2}\right)$:

$|P(x)-P(x_0)|=O(|x-x_0|^2)$ as $x\rightarrow x_0$, where $P(x_0)=\sup _{x\in {\mathbb {R}}^3}P(x)$.

The difficulty in obtaining solutions of the problem (1.6) lies in two aspects. On the one hand, since the problem involves critical exponent, the difficulty lies in the lack of compactness. On the other hand, when $p\in (2,4)$, the boundedness of the Palais–Smale sequences of the energy functional is hard to get. Inspired by [1, 9, 12, 13, 16, 21], we construct a truncated function and use the mountain pass theorem with mountain pass lever value under $\frac{1}{3}S^{\frac{3}{2}}\Vert P(x)\Vert ^{\frac{1}{2}}_\infty$.

Now we are in a position to state the following results of our paper on the existence of positive solutions.

Theorem 1.8

Assume that $\left( V_{1}\right)$–$\left( V_{2}\right)$ and $\left( P_{1}\right)$–$\left( P_{2}\right)$ are satisfied. If $p\in (4,6)$ with $\mu >0$ or $p=4$ with $\mu$ sufficiently large, then the problem (1.6) has at least one positive solution for any $q>0$.

Theorem 1.9

Assume that $\left( V_{1}\right)$–$\left( V_{2}\right)$ and $\left( P_{1}\right)$–$\left( P_{2}\right)$ are satisfied. If $p\in (2,4)$ with $\mu$ sufficiently large, then there exists $q_*>0$ such that, for all $q\in (0,q_*)$, the problem (1.6) has at least one positive solution.

Notations   Throughout the paper, we denote by $\text {meas}(\cdot )$ the Lebesgue measure in ${\mathbb {R}}^3$. $E^*$ is the dual space of E. $\left\| \cdot \right\| _p$ denotes the usual norm of Lebesgue space $L^p({\mathbb {R}}^3)$. S is the best Sobolev constant for the embedding $D^{1,2}({\mathbb {R}}^3)\hookrightarrow L^6({\mathbb {R}}^3)$. The symbol C denotes a positive constant and may vary from line to line.

2 Preliminary

In this section, we want to introduce the variational setting, the functional setting and some main results.

2.1 The Variational Setting

We first consider the nonlinear Schrödinger Lagrangian density

$$\begin{aligned} {\mathcal {L}}_{\mathrm{Sc}}=i{\hbar }{\bar{\psi }}\partial _t\psi -\frac{{\hbar }^2}{2m}|\nabla \psi |^2+2F(\psi ), \end{aligned}$$

(2.1)

where $\psi :{\mathbb {R}}\times {\mathbb {R}}^3\rightarrow {\mathbb {C}}$ and ${\hbar }$,$m>0$. If the gauge potential of the electromagnetic field $({\mathbf {E}},{\mathbf {H}})$ is $(\phi ,{\mathbf {A}})$, where $\phi :{\mathbb {R}}^3\rightarrow {\mathbb {R}}$ and ${\mathbf {A}}:{\mathbb {R}}^3\rightarrow {\mathbb {R}}^3$, then the following equations hold

$$\begin{aligned} {\mathbf {E}}=-\nabla \phi -\frac{1}{c}\partial _t{\mathbf {A}}, \quad {\mathbf {H}}=\nabla \times {\mathbf {A}}. \end{aligned}$$

To study the interaction between $\psi$ and its electromagnetic field $({\mathbf {E}},{\mathbf {H}})$, we can replace the derivatives $\partial _t$ and $\nabla$ in (2.1) with the covariant ones

$$\begin{aligned} D_t=\partial _t+\frac{i\sqrt{q}}{{\hbar }}\phi , \quad {\mathbf {D}}=\nabla -\frac{i\sqrt{q}}{{\hbar } c}{\mathbf {A}}. \end{aligned}$$

Thus, we have

$$\begin{aligned} {\mathcal {L}}_{\mathrm{CSc}}= & {} i{\hbar }{\bar{\psi }}D_t\psi -\frac{{\hbar }^2}{2m}|{\mathbf {D}}\psi |^2+2F(\psi )\\= & {} i{\hbar }{\bar{\psi }}\left( \partial _t+\frac{i\sqrt{q}}{{\hbar }}\phi \right) \psi -\frac{{\hbar }^2}{2m}\left| \left( \nabla -\frac{i\sqrt{q}}{{\hbar } c}{\mathbf {A}}\right) \psi \right| ^2+2F(\psi ). \end{aligned}$$

According to [3], the Bopp–Podolsky Lagrangian density is

$$\begin{aligned} {\mathcal {L}}_{\mathrm{BP}}= & {} \frac{1}{8\pi }\left\{ |{\mathbf {E}}|^2-|{\mathbf {H}}|^2+a^2\left[ (\text {div}{\mathbf {E}})^2-\left| \nabla \times {\mathbf {H}}-\frac{1}{c}\partial _t{\mathbf {E}}\right| ^2\right] \right\} \\= & {} \frac{1}{8\pi }\Bigg \{|\nabla \phi +\frac{1}{c}\partial _t{\mathbf {A}}|^2-|\nabla \times {\mathbf {A}}|^2\\&\left. +\,a^2\left[ \left( \Delta \phi +\frac{1}{c}\text {div}\partial _t{\mathbf {A}}\right) ^2-\left| \nabla \times \nabla \times {\mathbf {A}}+\frac{1}{c}\partial _t(\nabla \phi +\frac{1}{c}\partial _t{\mathbf {A}})\right| ^2\right] \right\} . \end{aligned}$$

Now, we add the Lagrangian density of the electromagnetic field to ${\mathcal {L}}_{\mathrm{CSc}}$ so that we can get the total Lagrangian density

$$\begin{aligned} {\mathcal {L}}:={\mathcal {L}}_{\mathrm{CSc}}+{\mathcal {L}}_{\mathrm{BP}}. \end{aligned}$$

Therefore, the total action is

$$\begin{aligned} {\mathcal {S}}(\psi ,\phi ,{\mathbf {A}})=\int _{{\mathbb {R}}^3}{\mathcal {L}}{\mathrm{d}}x{\mathrm{d}}t. \end{aligned}$$

More details can be found in [12].

2.2 The Operator $-\Delta +a^2\Delta ^2$

Let ${\mathcal {D}}$ be the completion of $C_c^\infty ({\mathbb {R}}^3)$ with the inner product

$$\begin{aligned} \langle \varphi ,\psi \rangle _{\mathcal {D}}:=\int _{{\mathbb {R}}^3}\nabla \varphi \nabla \psi +a^2\int _{{\mathbb {R}}^3}\Delta \varphi \Delta \psi . \end{aligned}$$

Obviously, ${\mathcal {D}}$ is continuously embedded into $D^{1,2}({\mathbb {R}}^3)$ and therefore in $L^6({\mathbb {R}}^3)$.

The following lemmas were obtained in [12].

Lemma 2.1

[12] The Hilbert space ${\mathcal {D}}$ is continuously embedded in $L^\infty ({\mathbb {R}}^3)$.

Lemma 2.2

[12] The space $C_c^\infty ({\mathbb {R}}^3)$ is dense in ${\mathcal {A}}$, where

$$\begin{aligned} {\mathcal {A}}:=\left\{ \phi \in D^{1,2}({\mathbb {R}}^3):\Delta \phi \in L^2({\mathbb {R}}^3)\right\} \end{aligned}$$

normed by $\sqrt{\langle \phi ,\phi \rangle _{\mathcal {D}}}$ and, consequently ${\mathcal {D}}={\mathcal {A}}$.

Lemma 2.3

[12] For all $y\in {\mathbb {R}}^3$, ${\mathcal {K}}(\cdot -y)=\frac{1-\mathrm{e}^{-|\cdot -y|/a}}{|\cdot -y|}$ is the fundamental solution of

$$\begin{aligned} -\Delta \phi +a^2\Delta ^2\phi ^2=4\pi \delta _y. \end{aligned}$$

Furthermore,

• (i) if $f\in L^1_{\mathrm{loc}}({\mathbb {R}}^3)$ and, for almost everywhere $x\in {\mathbb {R}}^3$, the map $y\in {\mathbb {R}}^3\mapsto \frac{f(y)}{|x-y|}$ is summable, then ${\mathcal {K}}*f\in L^1_{\mathrm{loc}}({\mathbb {R}}^3)$;

• (ii) if $f\in L^s({\mathbb {R}}^3)$ with $1\le s < \frac{3}{2}$, then ${\mathcal {K}}*f\in L^r({\mathbb {R}}^3)$ for $r\in \left( \frac{3s}{3-2s},+\infty \right] .$

In both cases, ${\mathcal {K}}*f$ solves

$$\begin{aligned} -\Delta \phi +a^2\Delta ^2\phi ^2=4\pi f. \end{aligned}$$

Moreover, almost everywhere in ${\mathbb {R}}^3$, we have

$$\begin{aligned} \nabla ({\mathcal {K}}*f)=(\nabla {\mathcal {K}})*f \quad { \mathrm{and} }\quad \Delta ({\mathcal {K}}*f)=(\Delta {\mathcal {K}})*f. \end{aligned}$$

Then, for any fixed $u\in H^1({\mathbb {R}}^3)$, $\phi _u:={\mathcal {K}}*u^2$ is the unique solution in ${\mathcal {D}}$ of the second equation in (1.3), and the following lemma holds.

Lemma 2.4

[12] For every $u\in H^1({\mathbb {R}}^3)$, we have:

• (i) for every $y\in {\mathbb {R}}^3$, $\phi _{u(\cdot +y)}=\phi _u(\cdot +y)$;

• (ii) $\phi _u\ge 0$;

• (iii) for every $s\in (3,+\infty ]$, $\phi _u\in L^s({\mathbb {R}}^3)\bigcap C_0({\mathbb {R}}^3)$;

• (iv) for every $s\in \left( \frac{3}{2},+\infty \right]$, $\nabla \phi _u=\nabla {\mathcal {K}}*u^2\in L^s({\mathbb {R}}^3)\bigcap C_0({\mathbb {R}}^3)$;

• (v) $\phi _u\in {\mathcal {D}}$;

• (vi) $\Vert \phi _u\Vert _6\le C\Vert u\Vert ^2$;

• (vii) $\phi _u$ is the unique minimizer of the functional

  $$\begin{aligned} E(\phi )=\frac{1}{2}\Vert \nabla \phi \Vert ^2_2+\frac{a^2}{2}\Vert \Delta \phi \Vert _2^2-\int _{{\mathbb {R}}^3}\phi u^2, \quad \phi \in {\mathcal {D}}; \end{aligned}$$

• (viii) if $v_n\rightharpoonup v$ in $H^1({\mathbb {R}}^3)$, then $\phi _{v_n}\rightharpoonup \phi _v$ in ${\mathcal {D}}$.

2.3 The Functional Setting

It is easy to see that the weak solutions of (1.3) are the critical points of the $C^1$ energy functional

$$\begin{aligned} R(u,\phi )=\frac{1}{2}\Vert u\Vert ^2+\frac{q}{2}\int _{{\mathbb {R}}^3}\phi u^2{\mathrm{d}}x-\frac{q}{16\pi }\Vert \phi \Vert ^2_{\mathcal {D}}-\int _{{\mathbb {R}}^3}F(u){\mathrm{d}}x \end{aligned}$$

on $E\times {\mathcal {D}}$. If $(u,\phi )\in E\times {\mathcal {D}}$ is a critical point of R, then we have

$$\begin{aligned} 0= & {} \partial _uR(u,\phi )[v]=\int _{{\mathbb {R}}^3}\left[ \nabla u\nabla v+V(x)uv+q\phi uv\right] {\mathrm{d}}x\\&-\,\int _{{\mathbb {R}}^3}f(u)v{\mathrm{d}}x,\quad \text {for all}\ \ v\in E, \end{aligned}$$

and

$$\begin{aligned} 0=\partial _\phi R(u,\phi )[\xi ]=\frac{q}{2}\int _{{\mathbb {R}}^3}\xi u^2{\mathrm{d}}x-\frac{q}{8\pi }\int _{{\mathbb {R}}^3}\left[ \nabla \phi \nabla \xi +a^2\Delta \phi \Delta \xi \right] {\mathrm{d}}x, \quad \text{ for } \text{ all }\ \ \xi \in {\mathcal {D}}. \end{aligned}$$

Distinctly, the functional R is strongly indefiniteness, and hence, we adopt a reduction procedure which used in [12]. Noting that $\partial _\phi R\in {\mathcal {C}}^1({\mathbb {R}}^3)$, by implicit function theorem, we have

$$\begin{aligned} G_\Phi =\left\{ (u,\phi )\in E\times {\mathcal {D}}:\partial _\phi R(u,\phi )=0\right\} \quad \text {and }\quad \Phi \in {\mathcal {C}}^1(E,{\mathcal {D}}), \end{aligned}$$

where $G_\Phi$ is the graph of the map $\Phi :u\in E\mapsto \phi _u\in {\mathcal {D}}$. Hence, we have the reduced form

$$\begin{aligned} I(u)=\frac{1}{2}\int _{{\mathbb {R}}^3}\left[ |\nabla u|^2+V(x)u^2\right] {\mathrm{d}}x+\frac{q}{4}\int _{{\mathbb {R}}^3}\phi _uu^2{\mathrm{d}}x-\int _{{\mathbb {R}}^3}F(u){\mathrm{d}}x, \end{aligned}$$

and it is of class ${\mathcal {C}}^1$ on E. For all $u,v\in E$,

$$\begin{aligned} I'(u)[v]= & {} \partial _uR(u,\Phi (u))[v]+\partial _\phi R(u,\Phi (u))\circ \Phi '(u)[v]\\= & {} \partial _uR(u,\Phi (u))[v]\\= & {} \int _{{\mathbb {R}}^3}\left[ \nabla u\nabla v+V(x)uv+q\phi _uuv\right] {\mathrm{d}}x-\int _{{\mathbb {R}}^3}f(u)v{\mathrm{d}}x. \end{aligned}$$

In fact, to find solutions of Eq. (1.3), we only need to find critical points of I.

3 Existence of Multiple Solutions

In this section, assume that $\left( V_{1}\right)$–$\left( V_{2}\right)$ and $\left( f_{1}\right)$–$\left( f_{3}\right)$ are satisfied. We prove that the problem (1.4) possesses infinitely many nontrivial solutions. And we give the energy functional I by

$$\begin{aligned} I(u)=\frac{1}{2}\int _{{\mathbb {R}}^3}|\nabla u|^2+V(x)u^2{\mathrm{d}}x+\frac{1}{4}\int _{{\mathbb {R}}^3}\phi _uu^2{\mathrm{d}}x-\int _{{\mathbb {R}}^3}F(u){\mathrm{d}}x. \end{aligned}$$

Lemma 3.1

[22] Let X be an infinite-dimensional Banach space, and there exists a finite-dimensional space W such that $X=W \oplus V$. $I \in {\mathcal {C}}^{1}(X,{\mathbb {R}})$ satisfies the (PS) condition, and

• (i) $I(u)=I(-u)$ for all $u \in X$, $I(0)=0$;

• (ii) there exist $\rho >0$, $\alpha >0$ such that $\left. I\right| _{\partial B_{\rho } \cap V} \ge \alpha$;

• (iii) for any finite-dimensional subspace $Y \subset X$, there exists $R=R(Y)>0$ such that $I(u) \le 0$ on $Y \backslash B_{R}$.

Then, I possesses an unbounded sequence of critical values.

Lemma 3.2

Under the assumptions of Theorem 1.3, if $\left\{ u_{n}\right\} \subset E$ satisfies the following conditions:

• (i) $\left\{ I\left( u_{n}\right) \right\}$ is bounded above by a positive constant;

• (ii) $I^{\prime }\left( u_{n}\right) [u_{n}] \rightarrow 0$, then $\left\{ u_{n}\right\}$ is bounded in E.

Proof

If $\left\{ u_{n}\right\}$ is unbounded in E, we can find a subsequence still denoted by $\left\{ u_{n}\right\}$ such that $\Vert u_{n}\Vert \rightarrow +\infty$. Let $v_{n}=\frac{u_{n}}{\left\| u_{n}\right\| }$, we have $\left\| v_{n}\right\| =1$. Thus, we may assume that $v_{n}\rightharpoonup v$ in E. It follows from $\left( f_{3}\right)$ and (i) that there exists a constant $c>0$ such that

$$\begin{aligned} c+1\ge & {} I\left( u_{n}\right) -\frac{1}{\mu }I^{\prime }\left( u_{n}\right) [u_{n}] \\\ge & {} \frac{\mu -2}{2 \mu }\left\| u_{n}\right\| ^{2}-\frac{\kappa }{\mu }\left\| u_{n}\right\| _{2}^{2}\\= & {} \frac{\mu -2}{2 \mu }\left\| u_{n}\right\| ^{2}\left\| v_{n}\right\| ^{2}-\frac{\kappa }{\mu }\left\| u_{n}\right\| ^{2}\left\| v_{n}\right\| _{2}^{2}, \end{aligned}$$

as $n \rightarrow \infty$, which implies $1 \le \frac{2 \kappa }{\mu -2} \limsup _{n \rightarrow \infty }\left\| v_{n}\right\| _{2}^{2}$. Therefore, $v \not \equiv 0$.

By $\left( f_{3}\right)$ and Fatou’s lemma, one has

$$\begin{aligned} 0= & {} \lim _{n \rightarrow \infty } \frac{c}{\left\| u_{n}\right\| ^{4}}=\lim _{n \rightarrow \infty } \frac{I\left( u_{n}\right) }{\left\| u_{n}\right\| ^{4}}\\\le & {} \lim _{n \rightarrow \infty }\left( \frac{1}{2\Vert u_n\Vert ^2}+\frac{\Vert u_n\Vert _2^4}{4a\Vert u_n\Vert ^4}-\int _{{\mathbb {R}}^{N}} \frac{F(u)}{u_{n}^{4}} v_{n}^{4}{\mathrm{d}}x\right) =-\infty , \end{aligned}$$

as $n\rightarrow +\infty$. This is a contradiction, and hence, $\left\{ u_{n}\right\}$ is bounded in E. $\square$

Lemma 3.3

Under the assumptions of Theorem 1.3, then the functional I satisfies the (PS) condition.

Proof

To prove that I satisfies the (PS) condition, we only need to prove $\{u_n\}\subset E$ which obtained by Lemma 3.2, has a convergent subsequence. As $\{u_n\}$ is bounded in E, there exists a subsequence still denoted by $\{u_n\}$ and $u_0\in E$ such that $u_n\rightharpoonup u_0$ in E and $u_n\rightarrow u_0$ in $L^q({\mathbb {R}}^3)$ for $2< q<6$. It follows from $u_n\rightharpoonup u_0$ in E and $I'(u_0)\in E^*$ that $\left\langle I^{\prime }(u_0), u_{n}-u_0\right\rangle \rightarrow 0$. And as $I'(u_n)\rightarrow 0$ in $E^*$, it is easy to obtain

$$\begin{aligned} \left\langle I^{\prime }(u_n), u_{n}-u_0\right\rangle \le \Vert I^{\prime }(u_n)\Vert _{E^*}\Vert u_{n}-u_0\Vert \rightarrow 0. \end{aligned}$$

Therefore,

$$\begin{aligned} \left\langle I^{\prime }\left( u_{n}\right) -I^{\prime }(u_0), u_{n}-u_0\right\rangle =\left\langle I^{\prime }(u_0), u_{n}-u_0\right\rangle -\left\langle I^{\prime }(u_n), u_{n}-u_0\right\rangle \rightarrow 0. \end{aligned}$$

By $(f_2)$, for any $\epsilon >0$, there exists $C_\epsilon >0$ such that

$$\begin{aligned} f(u)\le \epsilon |u|+C_\epsilon |u|^\iota . \end{aligned}$$

Hence, as $n\rightarrow +\infty$, we have

$$\begin{aligned}&\int _{{\mathbb {R}}^3}\left( \left( f\left( u_{n}\right) -f\left( u_{0}\right) \right) \left( u_{n}-u_{0}\right) \right) \,{\mathrm {d}}x\\&\quad \le \int _{{\mathbb {R}}^3}|f\left( u_{n}\right) ||u_{n}-u_{0}|\,{\mathrm {d}}x+\int _{{\mathbb {R}}^3}|f\left( u_{0}\right) ||u_{n}-u_{0}|\,{\mathrm {d}}x\\&\quad \le \int _{{\mathbb {R}}^3}\left( \epsilon |u_n|+C_\varepsilon |u_n|^\iota \right) |u_{n}-u_{0}|\,{\mathrm {d}}x+\int _{{\mathbb {R}}^3}\left( \epsilon |u_0|+C_\varepsilon |u_0|^\iota \right) |u_{n}-u_{0}|\,{\mathrm {d}}x\\&\quad \le \epsilon \left( \Vert u_n\Vert _2+\Vert u_0\Vert _2\right) \Vert u_n-u_0\Vert _2\\&\qquad +\,C_\epsilon \left( \Vert u_n\Vert ^\iota _{\iota +1}+\Vert u_0\Vert ^\iota _{\iota +1}\right) \Vert u_n-u_0\Vert _{\iota +1} \rightarrow 0, \end{aligned}$$

as $\epsilon \rightarrow 0$ and $\iota \in (1,5)$.

And then,

$$\begin{aligned} \left\| u_{n}-u_{0}\right\| ^{2}= & {} \left\langle I^{\prime }\left( u_{n}\right) -I^{\prime }(u_0), u_{n}-u\right\rangle \\&+\,\int _{{\mathbb {R}}^3}\left( \left( f\left( u_{n}\right) -f\left( u_{0}\right) \right) \left( u_{n}-u_{0}\right) \right) \,{\mathrm {d}}x\rightarrow 0. \end{aligned}$$

This completes the proof.$\square$

Proof of Theorem 1.3

We have verified that I satisfies the (PS) condition. It follows from $(f_1)$ that I is an even functional. As E is a separable space, E has orthonormal basis $\{e_i\}$. For all $k,j\in {\mathbb {Z}}$, we define $E_{j}:={\mathbb {R}} e_{j}$, $W_{k}:=\oplus _{j=1}^{k} E_{j}$, $V_{k}:=\overline{\oplus _{j=k+1}^{\infty }E_{j}}$. Let $W=W_{k}$, $V=V_{k}$, clearly $E=W \oplus V$ and ${\text {dim}} W<\infty$.

Next, we verify that I satisfies (ii) in Lemma 3.1. By $(f_2)$, for any $\epsilon >0$, there exists $C_\epsilon >0$ such that

$$\begin{aligned} I(u)= & {} \frac{1}{2}\Vert u\Vert ^2+\frac{1}{4}\int _{{\mathbb {R}}^3}\phi _{u}u^2\,{\mathrm {d}} x-\int _{{\mathbb {R}}^3}F(u){\mathrm{d}}x\\\ge & {} \frac{1}{2}\Vert u\Vert ^2+\frac{1}{4}\int _{{\mathbb {R}}^3}\phi _{u}u^2\,{\mathrm {d}} x-\epsilon \int _{{\mathbb {R}}^3}u^2{\mathrm{d}}x-C_\epsilon \int _{{\mathbb {R}}^3}u^{\iota +1}{\mathrm{d}}x\\\ge & {} \frac{1}{2}\Vert u\Vert ^2-C\left( \epsilon \Vert u\Vert ^2+C_\epsilon \Vert u\Vert ^{\iota +1}\right) . \end{aligned}$$

Therefore, there exists $\rho >0$ small enough, $\alpha >0$ such that $I(u) \ge \alpha >0$ as $\Vert u\Vert =\rho$.

Now, we verify that I satisfies (iii) in Lemma 3.1. If $I(u)>0$ for any finite-dimensional subspace $Y\subset E$, then we can find a sequence $\{u_n\}$ such that $\Vert u_n\Vert \rightarrow +\infty$ and $I(u_n)>0$. Let $v_{n}=\frac{u_{n}}{\left\| u_{n}\right\| }$, from the proof in Lemma 3.2, we have

$$\begin{aligned} 0=\lim _{n \rightarrow \infty } \frac{I\left( u_{n}\right) }{\left\| u_{n}\right\| ^{4}}\le \lim _{n \rightarrow \infty }\left( \frac{1}{2\Vert u_n\Vert ^2}+\frac{\Vert u_n\Vert _2^4}{4a\Vert u_n\Vert ^4}-\int _{{\mathbb {R}}^{N}} \frac{F(u)}{u_{n}^{4}} v_{n}^{4}{\mathrm{d}}x\right) =-\infty , \end{aligned}$$

as $n\rightarrow +\infty$, which is a contradiction. This shows that there exists $R=R(Y)>0$ such that $I(u) \le 0$ on $Y \backslash B_{R}$. The proof is completed.$\square$

4 Existence of Ground State Solutions for $p\in [4,6)$

In this section, to solve the problem (1.5), we apply Jeanjean’s trick [17] and give a family of energy functionals

$$\begin{aligned} I_\lambda (u)=\frac{1}{2}\int _{{\mathbb {R}}^3}\left[ |\nabla u|^2+V(x)u^2\right] {\mathrm{d}}x+\frac{1}{4}\int _{{\mathbb {R}}^3}\phi _uu^2{\mathrm{d}}x-\lambda \int _{{\mathbb {R}}^3}\left[ \frac{1}{p}|u|^p+H(u)\right] {\mathrm{d}}x. \end{aligned}$$

Lemma 4.1

[24] Assume that $\left\{ u_{n}\right\}$ is a bounded sequence in $H^1({\mathbb {R}}^3)$. If

$$\begin{aligned} \lim _{n \rightarrow \infty } \sup _{y\in {\mathbb {R}}^{3}} \int \limits _{B_{r}(y)}\left| u_{n}\right| ^{2}{\mathrm{d}}x=0 \quad \,\mathrm{for\, some } \quad r>0, \end{aligned}$$

then $u_{n} \rightarrow 0$ in $L^{q}\left( {\mathbb {R}}^{3}\right)$ for all $q \in (2, 6)$.

Lemma 4.2

[17] Let $(X,\Vert \cdot \Vert )$ be a Banach space and $T\subset {{\mathbb {R}}}^+$ be an interval. Consider a family of ${\mathcal {C}}^{1}$ functionals on X of the form

$$\begin{aligned} I_{\lambda }(u)=A(u)-\lambda B(u) \quad \forall \, \lambda \in T, \end{aligned}$$

with $B(u) \ge 0$ and either $A(u) \rightarrow +\infty$ or $B(u) \rightarrow +\infty$ as $\Vert u\Vert \rightarrow +\infty$. If there are two points $v_{1}, v_{2} \in X$ such that

$$\begin{aligned} c_{\lambda }=\inf _{\gamma \in \Gamma } \max _{t \in [0,1]} I_{\lambda }(\gamma (t))>\max \left\{ I_{\lambda }\left( v_{1}\right) , I_{\lambda }\left( v_{2}\right) \right\} \quad \forall \lambda \in T, \end{aligned}$$

where

$$\begin{aligned} \Gamma =\left\{ \gamma \in {\mathcal {C}}([0,1], X): \gamma (0)=v_{1}, \gamma (1)=v_{2}\right\} . \end{aligned}$$

Then, for almost every $\lambda \in T$, there exists a bounded $(PS)_{c_{\lambda }}$ sequence in X, and the mapping $\lambda \rightarrow c_{\lambda }$ is non-increasing and continuous from the left.

Lemma 4.3

Suppose that V, h satisfy $(V_1)$–$(V_2)$ and $(h_1)$–$(h_2)$. Then, for almost every $\lambda \in [\frac{1}{2},1]$, there is a bounded sequence $\{v_m\}$, such that $I_\lambda (v_m)\rightarrow c_\lambda$ in E and $I'_\lambda (v_m)\rightarrow 0$ in the dual $E^*$ of E.

Proof

Obviously that

$$\begin{aligned} \int _{{\mathbb {R}}^3}\left[ \frac{1}{p}|u|^p+H(u)\right] {\mathrm{d}}x\ge 0, \end{aligned}$$

and

$$\begin{aligned} \frac{1}{2}\int _{{\mathbb {R}}^3}\left[ |\nabla u|^2+V(x)u^2\right] {\mathrm{d}}x+\frac{1}{4}\int _{{\mathbb {R}}^3}\phi _uu^2{\mathrm{d}}x\rightarrow +\infty \quad \mathrm{as}\quad \Vert u\Vert \rightarrow +\infty . \end{aligned}$$

Let $u_\tau =\tau ^2u(\tau \cdot )$, then

$$\begin{aligned} I_\lambda (u_\tau )= & {} \frac{\tau ^3}{2}\Vert \nabla u\Vert ^2_2+\frac{\tau }{2}\int _{{\mathbb {R}}^3}V(x)u^2\,{\mathrm {d}} x+\frac{\tau ^3}{4}\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{1-\mathrm{e}^{-\frac{|x-y|}{\tau a}}}{|x-y|}u^2(x)u^2(y)\,{\mathrm {d}} x\,{\mathrm {d}} y\\&-\,\frac{\lambda \tau ^{2p-3}}{p}\Vert u\Vert ^p_p-\lambda \tau ^3\int _{{\mathbb {R}}^3}H(u)\,{\mathrm {d}} x\\\le & {} \frac{\tau ^3}{2}\Vert \nabla u\Vert ^2_2+\frac{\tau }{2}\int _{{\mathbb {R}}^3}V(x)u^2\,{\mathrm {d}} x+\frac{\tau ^3}{4}\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{u^2(x)u^2(y)}{|x-y|}\,{\mathrm {d}} x\,{\mathrm {d}} y\\&-\,\frac{\lambda \tau ^{2p-3}}{p}\Vert u\Vert ^p_p. \end{aligned}$$

Since $p\in [4,6)$, there exists $\tau _{0}>0$ large enough such that $I_{\lambda }\left( \tau _{0} u\right) <0$. By taking $v=u_{\tau _o}$, we have $I_{\lambda }(v)<0$.

Moreover, it follows from $(h_1)$ that

$$\begin{aligned} |h(u)| \le \epsilon |u|+C_{\epsilon }|u|^{\varrho },\quad \text {for any}\,\ \epsilon >0. \end{aligned}$$

By simple calculations, we derive

$$\begin{aligned} \int _{{\mathbb {R}}^{3}} H\left( u\right) \,{\mathrm {d}} x \le \frac{\epsilon }{2}\left\| u\right\| ^2_{2}+\frac{C_{\epsilon }}{\varrho +1}\left\| u\right\| _{\varrho +1}^{\varrho +1}. \end{aligned}$$

Hence,

$$\begin{aligned} I_\lambda (u)= & {} \frac{1}{2}\Vert u\Vert ^2_2+\frac{1}{4}\int _{{\mathbb {R}}^3}\phi _uu^2\,{\mathrm {d}} x-\lambda \frac{1}{p}\Vert u\Vert ^p_p-\lambda \int _{{\mathbb {R}}^3}H(u)\,{\mathrm {d}}x\\\ge & {} \frac{1}{2}\Vert u\Vert ^2_2-\frac{1}{p}\Vert u\Vert ^p_p-\frac{\epsilon }{2}\Vert u\Vert ^2_2-\frac{C_\epsilon }{\varrho +1}\Vert u\Vert ^{\varrho +1}_{\varrho +1}\\\ge & {} \frac{1-\epsilon C}{2}\Vert u\Vert ^2-\frac{C}{p}\Vert u\Vert ^p-\frac{CC_\epsilon }{\varrho +1}\Vert u\Vert ^{\varrho +1}. \end{aligned}$$

Then, there exists $\rho >0$ small enough such that $I_\lambda (u)>0$ as $\Vert u\Vert =\rho$. Now, Lemma 4.2 leads to the conclusion. $\square$

Lemma 4.4

If $\left\{ v_{m}\right\}$ is a bounded sequence in E obtained by Lemma 4.3, then

$$\begin{aligned} \lim \limits _{m \rightarrow +\infty } \sup \limits _{y \in {\mathbb {R}}^{3}}\int \limits _{B_{1}( y)}\left| v_{m}\right| ^{2}{\mathrm{d}}x>0. \end{aligned}$$

Proof

Assume that $\lim _{m \rightarrow +\infty } \sup \limits _{y \in {\mathbb {R}}^{3}}\int \limits _{B_{1}( y)}\left| v_{m}\right| ^{2}{\mathrm{d}}x=0$. From Lemma 4.1, we have $v_{m} \rightarrow 0$ in $L^{q}\left( {\mathbb {R}}^{3}\right)$ for all $q \in (2, 6)$.

Hence,

$$\begin{aligned} \int _{{\mathbb {R}}^3}h(v_m)v_m\,{\mathrm {d}} x\le \epsilon \Vert v_m\Vert _2^2+C_\epsilon \Vert v_m\Vert ^{\varrho +1}_{\varrho +1}=o(1). \end{aligned}$$

Moreover, by the Hardy–Littlewood–Sobolev inequality (see [18]), we obtain

$$\begin{aligned} \int _{{\mathbb {R}}^3}\phi _{v_m}v_m^2\,{\mathrm {d}} x\le \int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{v_m^2(x)v_m^2(y)}{|x-y|}\,{\mathrm {d}} x\,{\mathrm {d}} y\le \frac{8\root 3 \of {2}}{3\root 3 \of {\pi }}\Vert v_m\Vert ^4_{\frac{12}{5}}=o(1). \end{aligned}$$

Thus, we have

$$\begin{aligned} o(1)= & {} I'_\lambda (v_m)[v_m]=\Vert v_m\Vert ^2+\int _{{\mathbb {R}}^3}\phi _{v_m}v_m^2\,{\mathrm {d}} x-\lambda \Vert v_m\Vert ^p_p-\lambda \int _{{\mathbb {R}}^3}h(v_m)v_m\,{\mathrm {d}} x\\= & {} \Vert v_m\Vert ^2+o(1), \end{aligned}$$

and then

$$\begin{aligned} c_\lambda +o(1)= & {} I_\lambda (v_m)=\frac{1}{2}\Vert v_m\Vert ^2+\frac{1}{4}\int _{{\mathbb {R}}^3}\phi _{v_m}v_m^2\,{\mathrm {d}} x-\frac{\lambda }{p}\Vert v_m\Vert ^p_p\\&-\,\lambda \int _{{\mathbb {R}}^3}H(v_m)\,{\mathrm {d}} x=o(1), \end{aligned}$$

which is a contradiction.$\square$

Lemma 4.5

If $\left\{ v_{m}\right\} \subset E$ is the sequence obtained by Lemma 4.3, then for a.e. $\lambda \in \left[ \frac{1}{2}, 1\right]$, there exists a sequence of points $\left\{ y_{m}\right\} \subset {\mathbb {R}}^{3}$, set $u_{m}(x):=v_{m}\left( x-y_{m}\right)$, we have

• (i) $u_{m}\rightharpoonup u_{\lambda } \not \equiv 0$ in E;

• (ii) $I_{\lambda }^{\prime }\left( u_{\lambda }\right) =0$ in $E^{*}$;

• (iii) $I_{\lambda }\left( u_{\lambda }\right) \le c_{\lambda }$ in E;

• (iv) there exists $M>0$ such that $I_{\lambda }\left( u_{\lambda }\right) \ge M$.

Proof

Putting together Lemmas 4.3 and 4.4, we know that for almost every $\lambda \in \left[ \frac{1}{2}, 1\right]$, there exists a bounded sequence $\left\{ v_{m}\right\}$ which satisfies $I_{\lambda }\left( v_{m}\right) \rightarrow c_{\lambda }$ in E and $I_{\lambda }^{\prime }\left( v_{m}\right) \rightarrow 0$ in $E^{*}$ as $m \rightarrow +\infty$. Furthermore, by Lemma 4.4, there exists a sequence of points $\left\{ y_{m}\right\} \subset {\mathbb {R}}^3$ and $\alpha >0$, such that

$$\begin{aligned} \lim \limits _{m\rightarrow +\infty }\int _{B_{1}( y_{m})} v_{m}^{2} \,{\mathrm {d}} x \ge \alpha >0. \end{aligned}$$

(4.1)

Let $u_{m}(x):=v_{m}\left( x-y_{m}\right)$. By the invariance translations of $I_{\lambda }$, as $m \rightarrow +\infty$, we have that $I_{\lambda }\left( u_{m}\right) \rightarrow c_{\lambda }$ in E and $I_{\lambda }^{\prime }\left( u_{m}\right) \rightarrow 0$ in $E^{*}$. Since $\left\{ u_{m}\right\}$ is bounded, there exists $u_{\lambda } \in E$ such that

$$\begin{aligned} \left\{ \begin{array}{ll} u_{m} \rightharpoonup u_{\lambda }\quad \mathrm{in }\,E ;&{}\\ u_{m} \rightarrow u_{\lambda }\quad \mathrm{in }\,L^s_{\mathrm{loc}}({\mathbb {R}}^3),\,\, \forall s\in [1,6];&{}\\ u_{m} \rightarrow u_{\lambda }\quad \mathrm{a.e.\, in }\,{\mathbb {R}}^3.&{} \end{array}\right. \end{aligned}$$

(4.2)

In the following, we complete the proof of this lemma.

• (i) It follows from (4.1) that

  $$\begin{aligned} C\left\| u_{\lambda }\right\| ^{2} \ge \left\| u_{\lambda }\right\| _{2}^{2} \ge \int _{B_{1}(0)} u_{\lambda }^{2} \,{\mathrm {d}} x=\lim \limits _{m \rightarrow +\infty } \int _{B_{1}(0)} u_{m}^{2} \,{\mathrm {d}} x \ge \alpha >0, \end{aligned}$$

  and thus $u_{\lambda } \not \equiv 0$ in E.

• (ii) We only need to show that $\left\langle I_{\lambda }^{\prime }\left( u_{\lambda }\right) , \varphi \right\rangle = 0$ for any $\varphi \in E$. Observe that

  $$\begin{aligned}&\left\langle I_{\lambda }^{\prime }\left( u_{m}\right) , \varphi \right\rangle -\left\langle I_{\lambda }^{\prime }\left( u_{\lambda }\right) , \varphi \right\rangle \\&\quad =\int _{{\mathbb {R}}^3}\left( \left( \nabla u_m-\nabla u_\lambda \right) \varphi +V(x)\left( u_m-u_\lambda \right) \varphi \right) \,{\mathrm {d}} x+\int _{{\mathbb {R}}^3}\left( \phi _{u_m}u_m-\phi _{u_\lambda }u_\lambda \right) \varphi \,{\mathrm {d}} x\\&\qquad -\,\lambda \int _{{\mathbb {R}}^3}\left( f(u_m)-f(u_\lambda )\right) \varphi \,{\mathrm {d}} x-\lambda \int _{{\mathbb {R}}^3}\left( |u_m|^{p-2}u_m-|u_\lambda |^{p-2}u_\lambda \right) \varphi \,{\mathrm {d}} x. \end{aligned}$$

  Moreover,

  $$\begin{aligned} \left| \int _{{\mathbb {R}}^3}\left( \phi _{u_m}u_m-\phi _{u_\lambda }u_\lambda \right) \varphi \,{\mathrm {d}} x\right|\le & {} \int _{{\mathbb {R}}^3}\phi _{u_m}|u_m-u_{\lambda }||\varphi |\,{\mathrm {d}} x\\&+\,\int _{{\mathbb {R}}^3}|\phi _{u_m}-\phi _{u_\lambda }||u_\lambda \varphi |\,{\mathrm {d}} x:=J_1+J_2. \end{aligned}$$

  By Lemma 2.4, $\{\phi _{u_m}\}$ is bounded in $L^6({\mathbb {R}}^3)$ and $\phi _{u_m}\rightharpoonup \phi _{u_\lambda }$ in $L^6({\mathbb {R}}^3)$. Using the Hölder inequality and the strong convergence of $u_m$ to $u_\lambda$ in $L^3({\mathbb {R}}^3)$, we get

  $$\begin{aligned} J_1\le \Vert \phi _{u_m}\Vert _6\Vert u_m-u_\lambda \Vert _{3}\Vert \varphi \Vert _{2}\rightarrow 0. \end{aligned}$$

  Since $\varphi \in E$, for any $\epsilon >0$ there exists $R>0$ such that

  $$\begin{aligned} \left( \int _{B_R^c(0)}|\varphi |^s\,{\mathrm {d}} x\right) ^{\frac{1}{s}}<\epsilon ,\quad s\in [2,6]. \end{aligned}$$

  (4.3)

  Hence,

  $$\begin{aligned} J_2= & {} \int _{B_R(0)}|\phi _{u_m}-\phi _{u_\lambda }||u_\lambda \varphi |\,{\mathrm {d}} x+\int _{B^c_R(0)}|\phi _{u_m}-\phi _{u_\lambda }||u_\lambda \varphi |\,{\mathrm {d}} x\\\le & {} \Vert \phi _{u_m}-\phi _{u_\lambda }\Vert _{L^6(B_R(0))}\Vert u_\lambda \Vert _2\Vert \varphi \Vert _3+\Vert \phi _{u_m}-\phi _{u_\lambda }\Vert _{6}\Vert u_\lambda \Vert _2\Vert \varphi \Vert _{L^3(B_R^c(0))}\\\rightarrow & {} 0, \end{aligned}$$

  as $m\rightarrow +\infty$ and $R\rightarrow +\infty$. Furthermore, notice that

  $$\begin{aligned}&\left| \int _{{\mathbb {R}}^3}\left( |u_m|^{p-2}u_m-|u_\lambda |^{p-2}u_\lambda \right) \varphi \,{\mathrm {d}}x\right| \\&\quad \le \int _{B_R(0)}\left| |u_m|^{p-2}u_m-|u_\lambda |^{p-2}u_\lambda \right| \left| \varphi \right| \,{\mathrm {d}} x\\&\quad \quad +\,\int _{B^c_R(0)}\left| |u_m|^{p-2}u_m-|u_\lambda |^{p-2}u_\lambda \right| \left| \varphi \right| \,{\mathrm {d}} x\\&\quad :=J_3+J_4. \end{aligned}$$

  From (4.2), we infer that there exists $Q(x)\in L^p(B_R(0))$ such that $|u_m|\le Q(x)$. It follows from dominated convergence theorem that $J_3\rightarrow 0$ as $m\rightarrow 0$. On the other hand, as $R\rightarrow \infty$, we obtain

  $$\begin{aligned} J_4\le & {} \int _{B^c_R(0)}|u_m|^{p-1}\left| \varphi \right| +\int _{B^c_R(0)}|u_\lambda |^{p-1}\left| \varphi \right| \,{\mathrm {d}} x\\\le & {} \Vert u_m\Vert _p^{p-1}\Vert \varphi \Vert _{L^p(B^c_R(0))}+\Vert u_\lambda \Vert _p^{p-1}\Vert \varphi \Vert _{L^p(B^c_R(0))}\\\le & {} \epsilon C\rightarrow 0. \end{aligned}$$

  Moreover, as $u_{m} \rightarrow u_{\lambda }\ \,\mathrm{a.e.\, in}\,{\mathbb {R}}^3$, by continuity,

  $$\begin{aligned} h(u_m)\rightarrow h(u_\lambda )\quad \text {a.e.}\, \text {in}\quad {\mathbb {R}}^3. \end{aligned}$$

  It follows from $(h_1)$ that $\{h(u_m)\}$ is bounded in $L^{\frac{6}{5}}({\mathbb {R}}^3)$, and so $h(u_m)\rightharpoonup h(u_\lambda )$ in $L^{\frac{6}{5}}({\mathbb {R}}^3)$. Hence, as $R\rightarrow \infty$, we have

  $$\begin{aligned}&\left| \int _{{\mathbb {R}}^3}\left( h(u_m)-h(u_\lambda )\right) \varphi \,{\mathrm {d}} x\right| \\&\quad \le \int _{B_R(0)}\left| h(u_m)-h(u_\lambda )\right| \left| \varphi \right| \,{\mathrm {d}} x+\int _{B^c_R(0)}\left| h(u_m)-h(u_\lambda )\right| \left| \varphi \right| \,{\mathrm {d}}x\\&\quad \le \int _{B_R(0)}\left| h(u_m)-h(u_\lambda )\right| \left| \varphi \right| \,{\mathrm {d}} x+\int _{B^c_R(0)}|h(u_m)|\left| \varphi \right| \,{\mathrm {d}} x\\&\qquad +\,\int _{B^c_R(0)}|h(u_\lambda )|\left| \varphi \right| \,{\mathrm {d}} x\\&\quad \le \Vert h(u_m)-h(u_\lambda )\Vert _{L^{\frac{6}{5}}(B_R(0))}\Vert \varphi \Vert _6+\Vert u_m\Vert _{\frac{6}{5}}\Vert \varphi \Vert _{L^6(B^c_R(0))}\\&\qquad +\,\Vert u_\lambda \Vert _{\frac{6}{5}}\Vert \varphi \Vert _{L^6(B^c_R(0))}\\&\quad \le o(1)+\epsilon C\rightarrow 0. \end{aligned}$$

  Now, we get $I'_\lambda (u_\lambda )=0$.

• (iii) It follows from $\left( h_{3}\right)$ that

  $$\begin{aligned}&\frac{1}{4}h(u)u-H(u)+\frac{\theta _0V(x)}{4}u^2\\&\quad =\int ^1_0\left[ \frac{h(u)}{u^3}-\frac{h(su)}{{su}^3}+\frac{\theta _0V(x)(1-s^2)}{(su)^2}\right] s^3u^4{\mathrm{d}}s\ge 0. \end{aligned}$$

  Hence, by Fatou’s lemma we get

  $$\begin{aligned} c_{\lambda }= & {} \lim \limits _{m \rightarrow +\infty }\left[ I_{\lambda }\left( u_{m}\right) -\frac{1}{4}\left\langle I_{\lambda }^{\prime }\left( u_{m}\right) , u_{m}\right\rangle \right] \\\ge & {} \left( \frac{\lambda }{4}-\frac{\lambda }{p}\right) \liminf _{m \rightarrow +\infty }\Vert u_m\Vert _p^p+ \frac{1}{4}\liminf _{m \rightarrow +\infty }\left( \Vert u_m\Vert ^2-\lambda \theta _0\int _{{\mathbb {R}}^3}V(x)|u_m|^2{\mathrm{d}}x\right) \\&+\,\lambda \liminf _{m \rightarrow +\infty }\int _{{\mathbb {R}}^3}\left( \frac{1}{4}h(u_m)u_m-H(u_m)+\frac{\theta _0V(x)|u_m|^2}{4}\right) {\mathrm{d}}x\\\ge & {} \left( \frac{\lambda }{4}-\frac{\lambda }{p}\right) \Vert u_\lambda \Vert _p^p+ \frac{1}{4}\left( \Vert u_\lambda \Vert ^2-\lambda \theta _0\int _{{\mathbb {R}}^3}V(x)|u_\lambda |^2{\mathrm{d}}x\right) \\&+\,\lambda \int _{{\mathbb {R}}^3}\left( \frac{1}{4}h(u_\lambda )u_\lambda -H(u_\lambda )+\frac{\theta _0V(x)|u_\lambda |^2}{4}\right) {\mathrm{d}}x\\= & {} I_{\lambda }\left( u_{\lambda }\right) -\frac{1}{4}\left\langle I_{\lambda }^{\prime }\left( u_{\lambda }\right) , u_{\lambda }\right\rangle =I_{\lambda }\left( u_{\lambda }\right) >0. \end{aligned}$$

• (iv) Recall that $C\Vert u_\lambda \Vert ^2\ge \alpha$, we infer

  $$\begin{aligned} I_{\lambda }\left( u_{\lambda }\right) \ge \frac{1-\theta _0}{4}\Vert u_\lambda \Vert ^2\ge \frac{\alpha }{C}:=M. \end{aligned}$$

The proof is completed.$\square$

Now, according to Lemmas 4.3 and 4.5, there exists a sequence $\left\{ \left( \lambda _{n}, u_{\lambda _{n}}\right) \right\} \subset \left[ \frac{1}{2}, 1\right] \times E$, such that

• (i) $\lambda _{n} \rightarrow 1$ as $n \rightarrow +\infty$;

• (ii) $u_{\lambda _{n}} \not \equiv 0$, $M \le I_{\lambda _{n}}\left( u_{\lambda _{n}}\right) \le c_{\lambda _{n}}$;

• (iii) $I_{\lambda _{n}}^{\prime }\left( u_{\lambda _{n}}\right) =0$.

Proof of Theorem 1.6

By Lemma 4.5, we have

$$\begin{aligned} c_{1}\ge & {} I_{\lambda _{n}}\left( u_{\lambda _{n}}\right) =I_{\lambda _{n}}\left( u_{\lambda _{n}}\right) -\frac{1}{4}I'_{\lambda _{n}}(u_{\lambda _{n}})[u_{\lambda _{n}}]\\= & {} \frac{1}{4}\Vert u_{\lambda _{n}}\Vert ^2+(\frac{\lambda }{4}-\frac{\lambda }{p})\Vert u_{\lambda _{n}}\Vert _p^p+\lambda _n\int _{{\mathbb {R}}^3}\left( \frac{1}{4}h(u_{\lambda _{n}})u_{\lambda _{n}}-H(u_{\lambda _{n}})\right) \,{\mathrm {d}} x\\\ge & {} \frac{1}{4}\Vert u_{\lambda _{n}}\Vert ^2-\frac{\lambda _n\theta _0}{4}\int _{{\mathbb {R}}^3}V(x)|u_{\lambda _{n}}|^2{\mathrm{d}}x\\\ge & {} \frac{1-\theta _0}{4}\Vert u_{\lambda _{n}}\Vert ^2, \end{aligned}$$

which implies that there exists a constant $K>0$ such that$\Vert u_{\lambda _{n}}\Vert \le K.$

Using the facts that for any $\varphi \in E$,

$$\begin{aligned} \left\langle I^{\prime }\left( u_{\lambda _{n}}\right) , \varphi \right\rangle= & {} \left\langle I_{\lambda _{n}}^{\prime }\left( u_{\lambda _{n}}\right) , \varphi \right\rangle +\left( \lambda _{n}-1\right) \int \limits _{{\mathbb {R}}^{N}} h\left( u_{\lambda _{n}}\right) \varphi \,{\mathrm {d}} x ,\\ I\left( u_{\lambda _{n}}\right)= & {} I_{\lambda _{n}}\left( u_{\lambda _{n}}\right) +\left( \lambda _{n}-1\right) \int \limits _{{\mathbb {R}}^{N}} H\left( u_{\lambda _{n}}\right) \,{\mathrm {d}} x , \end{aligned}$$

and $\left\{ u_{\lambda _{n}}\right\}$ is bounded in E, we obtain that $M \le \lim _{n \rightarrow +\infty } I\left( u_{\lambda _{n}}\right) \le c_{1}$ and $\lim _{n \rightarrow +\infty } I^{\prime }\left( u_{\lambda _{n}}\right) =0$. Up to a subsequence, there exists a subsequence still denoted by $\left\{ u_{\lambda _{n}}\right\}$ and $u_{0} \in E$ such that $u_{\lambda _{n}}\rightharpoonup u_{0}$ in E. By using the preceding method in Lemma 4.5, we can obtain the existence of a nontrivial solution $u_{0}$ for I such that $I^{\prime }\left( u_{0}\right) =0$ and $M\le I\left( u_{0}\right) \le c_{1}$. Thus, $u_{0}$ is a nontrivial solution of (1.5). Define $m:=\inf \left\{ I(u): u \not \equiv 0, I^{\prime }(u)=0\right\}$. Let $\left\{ u_{n}\right\}$ be a sequence such that $I^{\prime }\left( u_{n}\right) =0$ and $I\left( u_{n}\right) \rightarrow m$. Similar to the argument in Lemma 4.5, we can prove that there exists ${\bar{u}} \in E$ such that $I^{\prime }({\bar{u}})=0$ and $I({\bar{u}}) \le m$. By the definition of m,  we have $m \le I({\bar{u}})$. Hence, $I({\bar{u}})=m$, which shows that ${\bar{u}}$ is a ground state solution of (1.5).$\square$

5 Existence of Positive Solutions for $p\in (2,6)$

In this section, assume that $\left( V_{1}\right)$–$\left( V_{2}\right)$ and $\left( P_{1}\right)$–$\left( P_{2}\right)$ are satisfied and $p\in (2,6)$. We prove the existence of positive solutions for problem (1.6).

By simple calculation, we have

$$\begin{aligned} I(u)= & {} \frac{1}{2}\int _{{\mathbb {R}}^3}\left( |\nabla u|^2+V(x)u^2\right) {\mathrm{d}}x+\frac{q}{4}\int _{{\mathbb {R}}^3}\phi _uu^2{\mathrm{d}}x\\&-\,\int _{{\mathbb {R}}^3}\left( \frac{1}{6}P(x)|u|^6+\frac{\mu }{p}|u|^p\right) {\mathrm{d}}x,\\ I'(u)[v]= & {} \int _{{\mathbb {R}}^3}\left( \nabla u\nabla v+V(x)uv+q\phi _uuv\right) {\mathrm{d}}x\\&-\,\int _{{\mathbb {R}}^3}\left( P(x)u^5v+\mu |u|^{p-2}uv\right) {\mathrm{d}}x. \end{aligned}$$

Lemma 5.1

[12] If $v_n\rightharpoonup v$ in $H^1({\mathbb {R}}^3)$, then

$$\begin{aligned} \int _{{\mathbb {R}}^3}\phi _{v_n}v^2_n{\mathrm{d}}x-\int _{{\mathbb {R}}^3}\phi _{v}v^2{\mathrm{d}}x-\int _{{\mathbb {R}}^3}\phi _{v_n-v}(v_n-v)^2{\mathrm{d}}x\rightarrow 0. \end{aligned}$$

Lemma 5.2

Let $\{v_n\}$ be a sequence such that $v_n\rightharpoonup v$ in E, then

$$\begin{aligned} \int _{{\mathbb {R}}^3}\phi _{v_n}v^2_n{\mathrm{d}}x\rightarrow \int _{{\mathbb {R}}^3}\phi _{v}v^2{\mathrm{d}}x\quad {\mathrm{as}}\quad n\rightarrow +\infty . \end{aligned}$$

Proof

To prove $\int _{{\mathbb {R}}^3}\phi _{v_n}v^2_n\rightarrow \int _{{\mathbb {R}}^3}\phi _{v}v^2$, we only need to prove that

$$\begin{aligned} \left| \int _{{\mathbb {R}}^3}\left( \phi _{v_n}v_n^2-\phi _{v}v^2\right) \,{\mathrm {d}} x\right|\le & {} \left| \int _{{\mathbb {R}}^3}\left( \phi _{v_n}v_n-\phi _{v}v\right) v_n\,{\mathrm {d}} x\right| +\left| \int _{{\mathbb {R}}^3}\left( \phi _{v_n}v_n-\phi _{v}v\right) v\,{\mathrm {d}} x\right| \\\le & {} \int _{{\mathbb {R}}^3}\phi _{v_n}|v_n-v||v_n|\,{\mathrm {d}} x+\int _{{\mathbb {R}}^3}|\phi _{v_n}-\phi _{v}||v_nv|\,{\mathrm {d}} x\\&+\,\int _{{\mathbb {R}}^3}\phi _{v_n}|v_n-v||v|\,{\mathrm {d}} x+\int _{{\mathbb {R}}^3}|\phi _{v_n}-\phi _{v}||v|^2\,{\mathrm {d}} x\\:= & {} J_1+J_2+J_3+J_4\rightarrow 0. \end{aligned}$$

We just give the proof of $J_1\rightarrow 0$, and the proof of $J_3\rightarrow 0$ can be proved in the same way. Lemma 2.4 and Theorem 1.2 imply that

$$\begin{aligned} J_1\le \Vert \phi _{v_n}\Vert _6\Vert v_n-v\Vert _3\Vert v_n\Vert _2\rightarrow 0\,\,\ \,\mathrm{as } n\rightarrow +\infty . \end{aligned}$$

Moreover, since $v\in E$, for any $\epsilon >0$ there exists $R>0$ such that

$$\begin{aligned} \left( \int _{B_R^c(0)}|v|^s\,{\mathrm {d}} x\right) ^{\frac{1}{s}}<\epsilon ,\quad s\in [2,6]. \end{aligned}$$

Obviously,

$$\begin{aligned} J_2= & {} \int _{B_R(0)}|\phi _{v_n}-\phi _{v}||v_nv|\,{\mathrm {d}} x+\int _{B^c_R(0)}|\phi _{v_n}-\phi _{v}||v_nv|\,{\mathrm {d}} x\\\le & {} \Vert \phi _{v_n}-\phi _{v}\Vert _{L^6(B_R(0))}\Vert v_n\Vert _2\Vert v\Vert _3+\Vert \phi _{v_n}-\phi _{v}\Vert _{6}\Vert v_n\Vert _2\Vert v\Vert _{L^3(B_R^c(0))}\\\rightarrow & {} 0, \end{aligned}$$

as $m\rightarrow +\infty$ and $R\rightarrow +\infty$, and $J_4\rightarrow 0$ can be proved in the same way. $\square$

5.1 The Case $p\in [4,6)$

Lemma 5.3

Assume that $\left( V_{1}\right)$–$\left( V_{2}\right)$ and $(P_1)$–$\left( P_{2}\right)$ are satisfied. Then, the functional I verifies the mountain pass geometry, that is,

• (i) there exist $\alpha ,\rho >0$ such that $I(v)\ge \alpha$ for all $\Vert v\Vert =\rho$;

• (ii) there exists $e\in H^1(\mathbb (R)^3){\setminus }\{0\}$ such that $I(e)<0$ with $\Vert e\Vert >\rho$.

Proof

Since $P(x)\le P(x_0)$, we have

$$\begin{aligned} I(u)\ge & {} \frac{1}{2}\int _{{\mathbb {R}}^3}\left( |\nabla u|^2+V(x)u^2\right) {\mathrm{d}}x-\int _{{\mathbb {R}}^3}\left( \frac{1}{6}P(x)|u|^6+\frac{\mu }{p}|u|^p\right) {\mathrm{d}}x\\\ge & {} \frac{1}{2}\Vert u\Vert ^2-\frac{1}{6}P(x_0)\Vert u\Vert _6^6-\frac{\mu }{p}\Vert u\Vert _p^p\\\ge & {} \frac{1}{2}\Vert u\Vert ^2-C\left( \frac{1}{6}P(x_0)\Vert u\Vert ^6+\frac{\mu }{p}\Vert u\Vert ^p\right) . \end{aligned}$$

Then, there exist $\alpha >0$ and $\rho >0$ small enough such that $I(u)\ge \alpha$ for all $\Vert u\Vert =\rho$.

Moreover, let $u_\tau =\tau u$, then

$$\begin{aligned} I(u_\tau )=\frac{\tau ^2}{2}\Vert u\Vert ^2+\frac{\tau ^4q}{4}\int _{{\mathbb {R}}^3}\phi _{u}u^2\,{\mathrm {d}} x-\frac{\mu \tau ^{p}}{p}\int _{{\mathbb {R}}^3}|u|^p{\mathrm{d}}x-\frac{\tau ^6}{6}\int _{{\mathbb {R}}^3}P(x)|u|^6\,{\mathrm {d}} x. \end{aligned}$$

Hence, there exists $\tau _{0}>0$ large enough such that $I\left( \tau _{0} u\right) <0$. By taking $e=u_{\tau _o}$, we have $I(e)<0$. $\square$

Combine Lemma 5.3 and Mountain–Pass lemma, we infer that there exists a sequence $\{v_n\}$ in E at the level c, such that

$$\begin{aligned} I(v_n)\rightarrow c \quad \text {and}\quad I'(v_n)\rightarrow 0,\quad \text {as}\quad n\rightarrow +\infty , \end{aligned}$$

(5.1)

where

$$\begin{aligned}&c:=\inf \limits _{\gamma \in \Gamma } \max \limits _{t \in [0,1]} I(\gamma (t))>0,\\&\Gamma =\{\gamma \in {\mathcal {C}}([0,1],H^1(\mathbb (R)^3)): \gamma (0)=0, \gamma (1)=e\}. \end{aligned}$$

Lemma 5.4

The sequence $\{v_n\}$ in (5.1) is bounded in E for $p\in [4,6)$.

Proof

As $n\rightarrow +\infty$, we have

$$\begin{aligned} c+o(1)= & {} I\left( v_{n}\right) -\frac{1}{4}I'\left( v_{n}\right) [v_{n}]\\= & {} \frac{1}{4}\Vert v_n\Vert ^2+\left( \frac{1}{4}-\frac{1}{p}\right) \int _{{\mathbb {R}}^3}\mu |v_n|^p{\mathrm{d}}x+\left( \frac{1}{4}-\frac{1}{6}\right) \int _{{\mathbb {R}}^3}P(x)|v_n|^6{\mathrm{d}}x\\\ge & {} \frac{1}{4}\Vert v_n\Vert ^2. \end{aligned}$$

This prove Lemma  5.4.$\square$

Lemma 5.5

The sequence $\{v_n\}$ in (5.1) is compact in E if

$$\begin{aligned} c<\frac{1}{3}S^{\frac{3}{2}}\Vert P(x)\Vert _\infty ^{-\frac{1}{2}}. \end{aligned}$$

Proof

As the sequence $\{v_n\}$ is given by (5.1), it satisfies that

$$\begin{aligned} I(v_n)= & {} \frac{1}{2}\int _{{\mathbb {R}}^3}\left( |\nabla v_n|^2+V(x)v_n^2\right) {\mathrm{d}}x+\frac{q}{4}\int _{{\mathbb {R}}^3}\phi _{v_n}v_n^2{\mathrm{d}}x\nonumber \\&-\,\int _{{\mathbb {R}}^3}\left( \frac{1}{6}P(x)|v_n|^6+\frac{\mu }{p}|v_n|^p\right) {\mathrm{d}}x=c+o(1), \end{aligned}$$

(5.2)

$$\begin{aligned} I'(v_n)[v_n]= & {} \int _{{\mathbb {R}}^3}\left( |\nabla v_n|^2+V(x)v_n^2\right) {\mathrm{d}}x+q\int _{{\mathbb {R}}^3}\phi _{v_n}v_n^2{\mathrm{d}}x\nonumber \\&-\,\int _{{\mathbb {R}}^3}\left( P(x)v_n^6+\mu |v_n|^{p}\right) {\mathrm{d}}x=o(1). \end{aligned}$$

(5.3)

Since the sequence $\{v_n\}$ is bounded, there exists $v \in E{\setminus }\{0\}$ such that

$$\begin{aligned} \left\{ \begin{array}{ll} v_{n} \rightharpoonup v\quad \mathrm{in }\,E ;&{}\\ v_{n} \rightarrow v\quad \mathrm{in }\,L^s_{\mathrm{loc}}({\mathbb {R}}^3),\,\, \forall s\in [1,6];&{}\\ v_{n} \rightarrow v\quad \mathrm{a.e.\, in }\,{\mathbb {R}}^3.&{} \end{array}\right. \end{aligned}$$

(5.4)

From $I'(v_n)\rightarrow 0$, we can obtain

$$\begin{aligned} I'(v)[v]= & {} \int _{{\mathbb {R}}^3}\left( |\nabla v|^2+V(x)v^2\right) {\mathrm{d}}x+q\int _{{\mathbb {R}}^3}\phi _{v}v^2{\mathrm{d}}x\nonumber \\&-\,\int _{{\mathbb {R}}^3}\left( P(x)|v|^6+\mu |v|^p\right) {\mathrm{d}}x=0. \end{aligned}$$

(5.5)

Set $v'_n=v_n-v$. By Brezis–Lieb’s lemma in [8],

$$\begin{aligned}&I(v_n)-I(v)-I(v'_n)\rightarrow 0\quad \text {as}\quad n\rightarrow +\infty , \end{aligned}$$

(5.6)

$$\begin{aligned}&I'(v_n)[v_n]-I'(v)[v]-I'(v'_n)[v'_n]\rightarrow 0\quad \text {as}\quad n\rightarrow +\infty . \end{aligned}$$

(5.7)

Theorem 1.2 and Lemmas 5.1, 5.2 and (5.4) imply that

$$\begin{aligned} \lim \limits _{n\rightarrow \infty }\int _{{\mathbb {R}}^3}|v'_n|^p{\mathrm{d}}x=0,\quad \lim \limits _{n\rightarrow \infty }\int _{{\mathbb {R}}^3}\phi _{v'_n}|v'_n|^2{\mathrm{d}}x=0\quad \text {as}\quad n\rightarrow +\infty . \end{aligned}$$

(5.8)

Putting (5.8) in (5.6) and (5.7), as $n\rightarrow +\infty$, we have

$$\begin{aligned} I(v)= & {} c+o(1)-I(v'_n)\nonumber \\= & {} c+o(1)-\frac{1}{2}\Vert v'_n\Vert ^2-\frac{q}{4}\int _{{\mathbb {R}}^3}\phi _{v'_n}|v'_n|^2{\mathrm{d}}x\nonumber \\&+\,\int _{{\mathbb {R}}^3}\left( \frac{1}{6}P(x)|v'_n|^6+\frac{\mu }{p}|v'_n|^p\right) {\mathrm{d}}x\nonumber \\= & {} c-\frac{1}{2}\Vert v'_n\Vert ^2+\int _{{\mathbb {R}}^3}\frac{1}{6}P(x)|v'_n|^6{\mathrm{d}}x+o(1), \end{aligned}$$

(5.9)

and

$$\begin{aligned} o(1)= & {} I'(v)[v]+I'(v'_n)[v'_n]\\= & {} \int _{{\mathbb {R}}^3}\left( |\nabla v'_n|^2+V(x)|v'_n|^2\right) {\mathrm{d}}x+q\int _{{\mathbb {R}}^3}\phi _{v'_n}|v'_n|^2{\mathrm{d}}x\\&-\,\int _{{\mathbb {R}}^3}\left( P(x)|v'_n|^6+\mu |v'_n|^{p}\right) {\mathrm{d}}x\\= & {} \int _{{\mathbb {R}}^3}\left( |\nabla v'_n|^2+V(x)|v'_n|^2\right) {\mathrm{d}}x-\int _{{\mathbb {R}}^3}P(x)|v'_n|^6{\mathrm{d}}x+o(1). \end{aligned}$$

Assume that $\int _{{\mathbb {R}}^3}P(x)|v'_n|^6{\mathrm{d}}x\rightarrow l$ as $n\rightarrow +\infty$, then $\int _{{\mathbb {R}}^3}|\nabla v'_n|^2+V(x)|v'_n|^2{\mathrm{d}}x\rightarrow l$.

It follows from Sobolev inequality that

$$\begin{aligned} \int _{{\mathbb {R}}^3}\left( |\nabla v'_n|^2+V(x)|v'_n|^2\right) {\mathrm{d}}x{\mathrm{d}}x\ge & {} S\left( \int _{{\mathbb {R}}^3}|v'_n|^6{\mathrm{d}}x\right) ^{\frac{1}{3}}\\\ge & {} S\Vert P(x)\Vert _\infty ^{-\frac{1}{3}}\left( \int _{{\mathbb {R}}^3}P(x)|v'_n|^6{\mathrm{d}}x\right) ^{\frac{1}{3}}. \end{aligned}$$

If $l>0$, we have

$$\begin{aligned} l\ge S^{\frac{3}{2}}\Vert P(x)\Vert _\infty ^{-\frac{1}{2}}. \end{aligned}$$

(5.10)

(5.9), (5.10) and Lemma 5.4 imply that

$$\begin{aligned} I(v)\rightarrow c-\frac{1}{3}l\le c-\frac{1}{3}S^{\frac{3}{2}}\Vert P(x)\Vert _\infty ^{-\frac{1}{2}}<0. \end{aligned}$$

(5.11)

However, (5.5) shows that

$$\begin{aligned} I(v)= & {} \frac{1}{2}\int _{{\mathbb {R}}^3}\left( |\nabla v'_n|^2+V(x)|v'_n|^2\right) {\mathrm{d}}x+\frac{q}{4}\int _{{\mathbb {R}}^3}\phi _vv^2{\mathrm{d}}x\\&-\,\int _{{\mathbb {R}}^3}\left( \frac{1}{6}P(x)|v|^6+\frac{\mu }{p}|v|^p\right) {\mathrm{d}}x,\\= & {} \frac{1}{2}\int _{{\mathbb {R}}^3}\left( P(x)|v|^6+\mu |v|^p\right) {\mathrm{d}}x-\int _{{\mathbb {R}}^3}\left( \frac{1}{6}P(x)|v|^6+\frac{\mu }{p}|v|^p\right) {\mathrm{d}}x\\\ge & {} 0, \end{aligned}$$

which is a contradiction to (5.11), thus $l=0$. This concludes that $\{v_n\}$ is compact.

$\square$

Proof of Theorem 1.8

Assume that $\left( V_{1}\right)$–$\left( V_{2}\right)$ and $(P_1)$–$\left( P_{2}\right)$ are satisfied. For each $\epsilon >0$, $x_0$ is chosen such that $P(x_0)=\sup _{x\in {\mathbb {R}}^3}P(x)$, we consider the following function

$$\begin{aligned} \omega _{\epsilon ,x_0}(x)=\frac{(3\epsilon )^{\frac{1}{4}}}{\left( \epsilon +|x-x_0|^2\right) ^{\frac{1}{2}}}, \end{aligned}$$

which is a solution of the minimization problem $S=\inf \{\Vert \nabla v\Vert ^2_2:v\in {\mathcal {D}}^{1,2}({\mathbb {R}}^3),\Vert v\Vert _6=1\}$, and $\omega _{\epsilon ,x_0}$ satisfies

$$\begin{aligned} \frac{\Vert \nabla \omega _{\epsilon ,x_0}\Vert ^2_2}{\Vert \omega _{\epsilon ,x_0}\Vert ^2_6}=S. \end{aligned}$$

Let $\eta \in {\mathcal {C}}^\infty _0({\mathbb {R}}^3,[0,1])$ be a cutoff function such that $\eta (x)=1$ for $|x-x_0|\le r$, $0<\eta (x)<1$ for $r<|x-x_0|\le 2r$ and $\eta (x)=0$ for $|x-x_0|\ge 2r$. Set $v_\epsilon (x)=\eta (x)\omega _{\epsilon ,x_0}(x)$, then as $\epsilon \rightarrow 0$, the following estimations hold (see [9, 25])

$$\begin{aligned} \Vert \nabla v_{\epsilon }\Vert ^2_2= & {} S^{\frac{3}{2}}+O(\epsilon ^{\frac{1}{2}}),\\ \Vert v_{\epsilon }\Vert ^6_6= & {} S^{\frac{3}{2}}+O(\epsilon ^{\frac{3}{2}}),\\ \Vert v_{\epsilon }\Vert ^s_s= & {} \left\{ \begin{array}{ll} O(\epsilon ^{\frac{s}{4}}),\quad s\in [2,3),&{}\\ O(\epsilon ^{\frac{s}{4}}|\ln \epsilon |),\quad s=3,&{}\\ O(\epsilon ^{\frac{6-s}{4}}),\quad s\in (3,6).&{} \end{array}\right. \end{aligned}$$

As $I(0)=0$, $\lim _{t\rightarrow +\infty }I(tv_\epsilon )=-\infty$, there exist $t_\epsilon >0$ and $M>0$ such that

$$\begin{aligned} I(t_\epsilon v_\epsilon )=\sup \limits _{t\ge 0}I(t v_\epsilon )\ge M\quad \text {and}\quad \frac{{\mathrm{d}}I(tv_\epsilon )}{{\mathrm{d}}t}|_{t=t_\epsilon }=0. \end{aligned}$$

Hence, we get

$$\begin{aligned} t_\epsilon \Vert \nabla v_\epsilon \Vert ^2_2= & {} t_\epsilon \left( S^{\frac{3}{2}}+O(\epsilon ^{\frac{1}{2}})\right) \\\le & {} t_\epsilon \Vert \nabla v_\epsilon \Vert ^2_2+t_\epsilon \int _{{\mathbb {R}}^3}V(x)v_\epsilon ^2{\mathrm{d}}x+qt^3_\epsilon \int _{{\mathbb {R}}^3}\phi _{v_\epsilon }v_\epsilon ^2{\mathrm{d}}x\\= & {} t^5_\epsilon \int _{{\mathbb {R}}^3}P(x)v_\epsilon ^6{\mathrm{d}}x+\frac{\mu }{t_\epsilon }\int _{{\mathbb {R}}^3}|t_\epsilon v_\epsilon |^p{\mathrm{d}}x\\\le & {} t^5_\epsilon P(x_0)\left( S^{\frac{3}{2}}+O(\epsilon ^{\frac{3}{2}})\right) +\mu \left( t_\epsilon O(\epsilon ^{\frac{1}{2}})+t^5_\epsilon S^{\frac{3}{2}}+t^5_\epsilon O(\epsilon ^{\frac{3}{2}})\right) , \end{aligned}$$

which shows that there exists a constant $A_1>0$ such that $t_\epsilon \ge A_1$.

Moreover,

$$\begin{aligned} M\le & {} I(t_\epsilon v_\epsilon ) \le \frac{t_\epsilon ^2}{2}\Vert \nabla v_\epsilon \Vert ^2_2+\frac{t_\epsilon ^2}{2}\left( \max \limits _{x\in B_{2r}(x_0)}V(x)\right) \Vert v_\epsilon \Vert ^2_2+\frac{qt_\epsilon ^4}{4a}\Vert v_\epsilon \Vert ^4_2\\&-\,\frac{t_\epsilon ^6}{6}\left( \min \limits _{x\in B_{2r}(x_0)}P(x)\right) \Vert v_\epsilon \Vert ^6_6\\= & {} \frac{t_\epsilon ^2}{2}\left( S^{\frac{3}{2}}+O(\epsilon ^{\frac{1}{2}})+\left( \max \limits _{x\in B_{2r}(x_0)}V(x)\right) O(\epsilon ^{\frac{1}{2}})\right) +\frac{qt_\epsilon ^4}{4a}\left( O(\epsilon ^{\frac{1}{2}})\right) ^2\\&-\,\frac{t_\epsilon ^6}{6}\left( \min \limits _{x\in B_{2r}(x_0)}P(x)\right) \left( S^{\frac{3}{2}}+O(\epsilon ^{\frac{3}{2}})\right) , \end{aligned}$$

which gives that there exists a constant $A_2>0$ such that $t_\epsilon \le A_2$.

Next, we claim that

$$\begin{aligned} \sup _{t>0}\left( \frac{t_\epsilon ^2}{2}\int _{{\mathbb {R}}^3}|\nabla v_\epsilon |^2{\mathrm{d}}x-\frac{t_\epsilon ^6}{6}\int _{{\mathbb {R}}^3}P(x)|v_\epsilon |^6{\mathrm{d}}x\right) \le \frac{1}{3}S^{\frac{3}{2}}\Vert P\Vert _\infty ^{-\frac{1}{2}}+O(\epsilon ^{\frac{1}{2}}). \end{aligned}$$

Choose $\delta >0$ small enough such that $|x-x_0|<\delta \epsilon ^{\frac{1}{2}}<r$. Using the condition $(P_2)$, there exists $\gamma >0$ such that

$$\begin{aligned} |P(x)-P(x_0)|\le \gamma |x-x_0|^2,\quad \text {for}\quad |x-x_0|<\delta \epsilon ^{\frac{1}{2}}, \end{aligned}$$

hence

$$\begin{aligned} \int _{{\mathbb {R}}^3}|P(x)-P(x_0)||v_\epsilon |^6{\mathrm{d}}x\le & {} \int _{|x-x_0|<\delta \epsilon ^{\frac{1}{2}}}\gamma |x-x_0|^2|v_\epsilon |^6{\mathrm{d}}x\\&+\,C\int _{|x-x_0|\ge \delta \epsilon ^{\frac{1}{2}}}|v_\epsilon |^6{\mathrm{d}}x\\\le & {} \gamma \delta \epsilon \Vert v_\epsilon \Vert _6^6+C\epsilon ^{\frac{3}{2}}\int ^{|x-x_0|=2r}_{|x-x_0|=\delta \epsilon ^{\frac{1}{2}}}\frac{1}{\left( \epsilon +|x-x_0|^2\right) ^3}{\mathrm{d}}x\\\le & {} C\epsilon +O(\epsilon ^{\frac{5}{2}})+O(\epsilon ^2). \end{aligned}$$

Then, we have that

$$\begin{aligned}&\frac{t_\epsilon ^2}{2}\int _{{\mathbb {R}}^3}|\nabla v_\epsilon |^2{\mathrm{d}}x-\frac{t_\epsilon ^6}{6}\int _{{\mathbb {R}}^3}P(x)|v_\epsilon |^6{\mathrm{d}}x\\&\quad =\frac{t^2_\epsilon }{2}\int _{{\mathbb {R}}^3}|\nabla v_\epsilon |^2{\mathrm{d}}x-\frac{t^6_\epsilon }{6}\int _{{\mathbb {R}}^3}P(x_0)|v_\epsilon |^6{\mathrm{d}}x+\int _{{\mathbb {R}}^3}|P(x_0)-P(x)||v_\epsilon |^6{\mathrm{d}}x\\&\quad \le \frac{1}{3}S^{\frac{3}{2}}\Vert P\Vert _\infty ^{-\frac{1}{2}}+O(\epsilon ^{\frac{1}{2}})+O(\epsilon ^{\frac{5}{2}})+O(\epsilon ^{\frac{3}{2}})+C\epsilon +O(\epsilon ^{2}), \end{aligned}$$

which implies the claim holds.

Thus,

$$\begin{aligned} I(t_\epsilon v_\epsilon )= & {} \frac{t_\epsilon ^2}{2}\int _{{\mathbb {R}}^3}|\nabla v_\epsilon |^2{\mathrm{d}}x+\frac{t_\epsilon ^2}{2}\int _{{\mathbb {R}}^3}V(x)v_\epsilon ^2{\mathrm{d}}x+\frac{qt_\epsilon ^4}{4}\int _{{\mathbb {R}}^3}\phi _{v_\epsilon }v_\epsilon ^2{\mathrm{d}}x\\&-\,\frac{\mu t_\epsilon ^p}{p}\int _{{\mathbb {R}}^3}|v_\epsilon |^p{\mathrm{d}}x-\frac{t_\epsilon ^6}{6}\int _{{\mathbb {R}}^3}P(x)|v_\epsilon |^6{\mathrm{d}}x\\\le & {} \frac{1}{3}S^{\frac{3}{2}}\Vert P\Vert _\infty ^{-\frac{1}{2}}+\frac{qA_2^4}{4a}\Vert v_\epsilon \Vert ^4_2-\mu \frac{A_1^p}{p}\Vert v_\epsilon \Vert ^p_p\\&+\,O(\epsilon ^{\frac{1}{2}})+C\epsilon +O(\epsilon ^{\frac{3}{2}})+O(\epsilon ^{\frac{5}{2}})+O(\epsilon ^{2})\\\le & {} \frac{1}{3}S^{\frac{3}{2}}\Vert P\Vert _\infty ^{-\frac{1}{2}}+O(\epsilon ^{\frac{1}{2}})+O(\epsilon )+O(\epsilon ^{\frac{3}{2}})+O(\epsilon ^{\frac{5}{2}})+O(\epsilon ^{2})-C\mu \epsilon ^{\frac{6-p}{4}}\\:= & {} \frac{1}{3}S^{\frac{3}{2}}\Vert P\Vert _\infty ^{-\frac{1}{2}}+A. \end{aligned}$$

If $4<p<6$ with $\mu >0$ or $p=4$ with $\mu$ sufficient large, we have $A<0$.

Combine Lemmas 5.2 and 5.4, we can easily verity that if there exists $v_0\in E$ and $v_0\not \equiv 0$ such that

$$\begin{aligned} \sup \limits _{t\ge 0}I(t v_0)<\frac{1}{3}S^{\frac{3}{2}}\Vert P(x)\Vert _\infty ^{-\frac{1}{2}}. \end{aligned}$$

(5.12)

Then, the problem (1.6) has at least one positive solution. Taking $v_0=v_\epsilon$ for $\epsilon$ small enough, then (5.12) holds, this completes our proof. $\square$

5.2 The Case of $p\in (2,4)$

Inspired by [12], we consider the truncated functional

$$\begin{aligned} I_T(u):= & {} \frac{1}{2}\int _{{\mathbb {R}}^3}\left( |\nabla u|^2+V(x)u^2\right) {\mathrm{d}}x+\frac{q}{4}\chi \left( \frac{\Vert u\Vert ^2}{T^2}\right) \int _{{\mathbb {R}}^3}\phi _uu^2{\mathrm{d}}x\\&-\,\int _{{\mathbb {R}}^3}\left( \frac{1}{6}P(x)|u|^6+\frac{\mu }{p}|u|^p\right) {\mathrm{d}}x, \end{aligned}$$

where $\chi (s)=1$ for $0\le s\le 1$, $\chi (s)\in (0,1)$ for $1\le s\le 2$, $\chi (s)=0$ for $2\le s$ and $\Vert \chi '\Vert _\infty \le 2$.

Lemma 5.6

The functional $I_T$ satisfies the mountain pass geometry:

• (i) there exist $\alpha ,\rho >0$ such that $I(u)\ge \alpha$ for all $\Vert u\Vert =\rho$;

• (ii) there exists $e\in H^1(\mathbb (R)^3){\setminus }\{0\}$ such that $I(e)<0$ with $\Vert e\Vert >\rho$.

Proof

As $q>0$, we have

$$\begin{aligned} I_T(u)\ge & {} \frac{1}{2}\int _{{\mathbb {R}}^3}\left( |\nabla u|^2+V(x)u^2\right) {\mathrm{d}}x-\int _{{\mathbb {R}}^3}\left( \frac{1}{6}P(x)|u|^6+\frac{\mu }{p}|u|^p\right) {\mathrm{d}}x\\\ge & {} \frac{1}{2}\Vert u\Vert ^2-C\left( \frac{P(x_0)}{6}\Vert u\Vert ^6+\frac{\mu }{p}\Vert u\Vert ^p\right) . \end{aligned}$$

Then, there exists $\rho >0$ small enough such that $I_T(u)>0$ as $\Vert u\Vert =\rho$.

In addition, consider $u_\tau =\tau u$, we get

$$\begin{aligned} I_T(\tau u)= & {} \frac{\tau ^2}{2}\Vert u\Vert ^2+\frac{q\tau ^4}{4}\chi \left( \tau ^2\frac{\Vert u\Vert ^2}{T^2}\right) \int _{{\mathbb {R}}^3}\phi _uu^2{\mathrm{d}}x\\&-\,\int _{{\mathbb {R}}^3}\left( \frac{\tau ^6}{6}P(x)|u|^6+\frac{\mu \tau ^p}{p}|u|^p\right) {\mathrm{d}}x, \end{aligned}$$

and there exists $\tau _{0}>0$ large enough such that $I_{\lambda }\left( \tau _{0} u\right) <0$. By taking $v=u_{\tau _o}$, we have $I_T{}(v)<0$.$\square$

By mountain pass theorem, there exists a sequence $\{u_n\}$ in E at the level $c_T$, such that

$$\begin{aligned} I_T(u_n)\rightarrow c_T \quad \text {and}\quad I_T'(u_n)\rightarrow 0,\quad \text {as}\quad n\rightarrow +\infty , \end{aligned}$$

(5.13)

where

$$\begin{aligned} c_T:= & {} \inf \limits _{\gamma \in \Gamma _T} \max \limits _{t \in [0,1]} I_T(\gamma (t))>0,\\ \Gamma _T= & {} \{\gamma \in {\mathcal {C}}([0,1],H^1(\mathbb (R)^3)): \gamma (0)=0, I_T(\gamma (1))<0\}. \end{aligned}$$

Remark 5.7

It follows from $I_T\le I$ that $c_T\le c$, where the constant c is given in (5.1).

Lemma 5.8

There exist $T_0>0$ independent of q and $q_*>0$ dependent on $T_0$ such that if $q<q_*$, then the sequence $\{u_n\}$ in (5.13) satisfies

$$\begin{aligned} \limsup \limits _n\Vert u_n\Vert <T_0 . \end{aligned}$$

Proof

Similar to the proof in [12]. Arguing by contradiction, suppose for any $T>0$, there exists $q>0$ such that

$$\begin{aligned} \limsup \limits _n\Vert u_n\Vert \ge T. \end{aligned}$$

(5.14)

Notice that

$$\begin{aligned} pI_T(u_n)-I'_T(u_n)[u_n]= & {} \left( \frac{p}{2}-1\right) \Vert u_n\Vert ^2+\left( \frac{pq}{4}-q\right) \chi \left( \frac{\Vert u_n\Vert ^2}{T^2}\right) \int _{{\mathbb {R}}^3}\phi _{u_n}u_n^2{\mathrm{d}}x\nonumber \\&-\,\frac{q\Vert u_n\Vert ^2}{2T^2}\chi '\left( \frac{\Vert u_n\Vert ^2}{T^2}\right) \int _{{\mathbb {R}}^3}\phi _{u_n}u_n^2{\mathrm{d}}x\nonumber \\&-\,\left( \frac{p}{6}-1\right) \int _{{\mathbb {R}}^3}P(x)|u_n|^6{\mathrm{d}}x. \end{aligned}$$

(5.15)

Let e be as in Lemma 5.6. As $n\rightarrow +\infty$, we have

$$\begin{aligned} I_T(u_n)\le & {} 2c_T\le 2\max \limits _{t\in [0,1]}I_T(te)\\\le & {} \Vert e\Vert ^2\max \limits _{t\in [0,1]}t^2+\frac{q}{2}\max \limits _{t\in [0,1]}\left[ t^4\chi \left( \frac{t^2\Vert e\Vert ^2}{T^2}\right) \int _{{\mathbb {R}}^3}\phi _ee^2{\mathrm{d}}x\right] \\:= & {} C_1+\frac{q}{2}J_1. \end{aligned}$$

If $t^2\Vert e\Vert ^2>2T^2$ then $J_1=0$ and if $t^2\Vert e\Vert ^2\le 2T^2$, then

$$\begin{aligned} J_1\le \frac{4T^4}{\Vert e\Vert ^4}\int _{{\mathbb {R}}^3}\phi _ee^2{\mathrm{d}}x \le \frac{4T^4}{a\Vert e\Vert ^4}\Vert e\Vert _2^4. \end{aligned}$$

By Theorem 1.2, we have

$$\begin{aligned} I_T(u_n)\le C_1+C_2qT^4. \end{aligned}$$

(5.16)

Through similar discussions, we obtain

$$\begin{aligned} \chi \left( \frac{\Vert u_n\Vert ^2}{T^2}\right) \int _{{\mathbb {R}}^3}\phi _{u_n}u_n^2{\mathrm{d}}x\le C_3 T^4, \end{aligned}$$

(5.17)

and

$$\begin{aligned} \Vert u_n\Vert ^2\chi '\left( \frac{\Vert u_n\Vert ^2}{T^2}\right) \int _{{\mathbb {R}}^3}\phi _{u_n}u_n^2{\mathrm{d}}x\le \Vert u_n\Vert ^2\left| \chi '\left( \frac{\Vert u_n\Vert ^2}{T^2}\right) \right| \int _{{\mathbb {R}}^3}\phi _{u_n}u_n^2{\mathrm{d}}x\le C_4T^6.\nonumber \\ \end{aligned}$$

(5.18)

Combining (5.14)–(5.18), we get

$$\begin{aligned}&C_5T^2-T\nonumber \\&\quad \le \left( \frac{p}{2}-1\right) \Vert u_n\Vert ^2-I'_T(u_n)[u_n]\nonumber \\&\quad \le pI_T(u_n)+\left( q^2-\frac{pq^2}{4}\right) \chi \left( \frac{\Vert u_n\Vert ^2}{T^2}\right) \int _{{\mathbb {R}}^3}\phi _{u_n}u_n^2{\mathrm{d}}x\nonumber \\&\qquad +\,\frac{q\Vert u_n\Vert ^2}{2T^2}\chi '\left( \frac{\Vert u_n\Vert ^2}{T^2}\right) \int _{{\mathbb {R}}^3}\phi _{u_n}u_n^2{\mathrm{d}}x\nonumber \\&\quad \quad -\,\left( 1-\frac{p}{6}\right) \int _{{\mathbb {R}}^3}P(x)|u_n|^6{\mathrm{d}}x\nonumber \\&\quad \le pI_T(u_n)+\left( q-\frac{pq}{4}\right) \chi \left( \frac{\Vert u_n\Vert ^2}{T^2}\right) \int _{{\mathbb {R}}^3}\phi _{u_n}u_n^2{\mathrm{d}}x\nonumber \\&\qquad +\,\frac{q\Vert u_n\Vert ^2}{2T^2}\chi '\left( \frac{\Vert u_n\Vert ^2}{T^2}\right) \int _{{\mathbb {R}}^3}\phi _{u_n}u_n^2{\mathrm{d}}x\nonumber \\&\quad \le C_6+qC_7T^4. \end{aligned}$$

(5.19)

Obviously, the inequality (5.19) does not hold if q small enough and for large T. $\square$

Remark 5.9

Since $\limsup \limits _n\Vert u_n\Vert <T_0$, we have $I_{T_0}(u_n)=I(u_n)$.

Proof of Theorem 1.9

Assume that $\left( V_{1}\right)$–$\left( V_{2}\right)$ and $(P_1)$–$\left( P_{2}\right)$ are satisfied. From Lemmas 5.6, 5.8 and Remark 5.9, there exists $q_*>0$ such that if $q\in (0,q_*)$ then I possesses a bounded (PS) sequence $\{u_n\}$ at level $c_{T_0}$. Moreover, we already know that if $c_{T_0}<\frac{1}{3}S^{\frac{3}{2}}\Vert P(x)\Vert _\infty ^{-\frac{1}{2}}$, then $\{u_n\}$ is compact in E. It remains to show that there exists $v_0\in E$ and $v_0\not \equiv 0$ such that

$$\begin{aligned} \sup \limits _{t\ge 0}I(t v_0)<\frac{1}{3}S^{\frac{3}{2}}\Vert P(x)\Vert _\infty ^{-\frac{1}{2}}, \end{aligned}$$

for $p\in (2,4)$ with $\mu$ sufficiently large. As its proof is similar with the Proof of Theorem 1.8, we omit it here.$\square$