Existence and Asymptotical Behavior of Multiple Solutions for the Critical Choquard Equation

Abstract

We deal with the following critical Choquard equation

$$\begin{aligned} -\varepsilon ^2\Delta u+ V(x)u= & {} P(x)|u|^{p-2}u\\&+\varepsilon ^{\mu -3}\Big (\int _{{\mathbb {R}}^3}\frac{Q(y)|u(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}y\Big )Q(x)|u|^{4-\mu }u, \quad x\in {\mathbb {R}}^3, \end{aligned}$$

where $\varepsilon >0$ is a small parameter, $0<\mu <3$, $p\in (4,6)$. Under some conditions on the potential functions V(x), P(x), and Q(x), we obtain the existence of multiple solutions and their asymptotical behavior as $\varepsilon \rightarrow 0$.

1 Introduction and Main Results

In this paper, we consider the following critical Choquard equation

$$\begin{aligned} -\varepsilon ^2\Delta u+ V(x)u= & {} P(x)|u|^{p-2}u\nonumber \\&+\varepsilon ^{\mu -3}\Big (\int _{{\mathbb {R}}^3} \frac{Q(y)|u(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}y\Big )Q(x)|u|^{4-\mu }u, \quad x\in {\mathbb {R}}^3,\qquad \quad \end{aligned}$$

(1.1)

where $\varepsilon >0$ is a parameter, $0<\mu <3$, $p\in (4,6)$. The potential functions V(x), P(x), and Q(x) are three bounded and continuous functions in ${\mathbb {R}}^3$ satisfying $\inf \limits _{x\in {\mathbb {R}}^3} V(x)>0$, $\inf \limits _{x\in {\mathbb {R}}^3} P(x)>0$ and $\inf \limits _{x\in {\mathbb {R}}^3} Q(x)>0$.

The Choquard equation

$$\begin{aligned} -\Delta u+ u=\Big (\int _{{\mathbb {R}}^3}\frac{u^2(y)}{|x-y|}\mathrm{{d}}y\Big )u, \quad x\in {\mathbb {R}}^3, \end{aligned}$$

was used by Pekar [17] to describe the quantum theory of polaron at rest. Then it was introduced by Choquard [10] as an approximation to Hartree–Fock theory of one-component plasma. Penrose [18] also derived it as a model of self-gravitating matter, in which quantum state reduction is understood as a gravitational phenomenon. Lieb [10] proved the existence and uniqueness (up to translations) of solutions by using symmetric decreasing rearrangement inequalities. Lions [11] obtained the existence of infinitely many spherically symmetric solutions. Ma and Zhao [14] showed the positive solutions of this equation must be radially symmetric and monotone decreasing about some fixed point by the method of moving planes. Moroz and Van Schaftingen [15] studied the generalized Choquard equation

$$\begin{aligned} -\Delta u+ u=(I_\alpha *|u|^p)|u|^{p-2}u, \quad x\in {\mathbb {R}}^3, \end{aligned}$$

where $I_\alpha$ is a Riesz potential and $p>1$. For an optimal range of parameters, they showed the regularity, positivity, and radial symmetry of the ground states and derived decay property at infinity as well.

Gao and Yang [6] studied the Brezis–Nirenberg type problem of the nonlinear Choquard equation

$$\begin{aligned} -\Delta u-\lambda u=\Big (\int _{\Omega }\frac{|u(y)|^{2^*_\mu }}{|x-y|^\mu }\mathrm{{d}}y\Big )|u|^{2^*_\mu -2}u, \quad x\in {\mathbb {R}}^3, \end{aligned}$$

where $\Omega$ is a bounded domain and $\lambda$ is a parameter, $N\ge 3$ and $2^*_\mu =\frac{2N-\mu }{N-2}$ is the critical exponent under the sense of Hardy–Littlewood–Sobolev inequality. They established some existence results for this equation. Shen, Gao, and Yang [19] investigated the critical Choquard equation with potential well

$$\begin{aligned} -\Delta u+ (\lambda V(x)-\beta )u=(|x|^{-\mu }*|u|^{2^*_\mu })|u|^{2^*_\mu -2}u, \quad x\in {\mathbb {R}}^N, \end{aligned}$$

where $\lambda ,\beta >0$, $0<\mu <N$, $N\ge 4$, $2^*_\mu$ is the critical exponent. They proved the existence of ground state solutions which localize near the potential well $\inf V^{-1}(0)$ and also characterize the asymptotic behavior as $\lambda \rightarrow \infty$. Furthermore, the multiple solutions were also established by Lusternik–Schnirelmann category theory.

For the semiclassical problem, Liu and Tang [12] studied the following subcritical equation

$$\begin{aligned} -\varepsilon ^2\Delta w+ V(x)w=\varepsilon ^{-\theta }W(x)(I_\theta *(W|w|^p))|w|^{p-2}w, \quad w\in H^1({\mathbb {R}}^N), \end{aligned}$$

where $\varepsilon >0$, $N>2$, $\theta \in [2.\frac{N+\theta }{N-2})$. The potential functions V(x), W(x) are bounded positive functions. By using pseudo-index theory, they established the multiplicity of solutions. Alves et al. [1] studied the following critical equation

$$\begin{aligned} -\varepsilon ^2\Delta u+ V(x)u= & {} \varepsilon ^{\mu -3}\Big (\int _{{\mathbb {R}}^3}\frac{Q(y)(|u(y)|^{6-\mu }+F(u(y)))}{|x-y|^\mu }\mathrm{{d}}y\Big )\\&\quad \Big (Q(x)(|u|^{4-\mu }u+\frac{1}{6-\mu }f(u))\Big ), \quad x\in {\mathbb {R}}^3, \end{aligned}$$

where $\varepsilon >0$ is a parameter, $0<\mu <3$. The potential functions V(x) and Q(x) are two bounded and continuous functions in ${\mathbb {R}}^3$ satisfying $\inf _{x\in {\mathbb {R}}^3} V(x)>0$ and $\inf _{x\in {\mathbb {R}}^3} Q(x)>0$. When $Q(x)\equiv 1$ and V(x) satisfies

$$\begin{aligned} \min _{x\in {\mathbb {R}}^3}V(x)<\liminf _{|x|\rightarrow \infty }V(x), \end{aligned}$$

they proved the existence of ground state solution and multiple solutions. Moreover, the concentration phenomenon was also considered. Zhang and Zhang [29] considered the following critical Choquard equation

$$\begin{aligned} -\varepsilon ^2\Delta u+ V(x)u= & {} \varepsilon ^{\mu -3}\Big (\int _{{\mathbb {R}}^3}\frac{|u(y)|^{6-\mu }+Q(y)F(u(y))}{|x-y|^\mu }\mathrm{{d}}y\Big )\\&\quad \Big (|u|^{4-\mu }u+\frac{1}{6-\mu }Q(x)f(u)\Big ), \quad x\in {\mathbb {R}}^3, \end{aligned}$$

where $\varepsilon >0$ is a parameter, $0<\mu <3$. The potential functions V(x) and Q(x) are two bounded and continuous functions. Under the condition,

$$\begin{aligned} Q(x)\ge \lim _{|x|\rightarrow \infty }Q(x),\ x\in {\mathbb {R}}^3, \end{aligned}$$

and

$$\begin{aligned} \mathcal {V\cap Q}=\{x\in {\mathbb {R}}^3: V(x)=V_{\min }, Q(x)=Q_\mathrm{{max}}\}\ne \varnothing , \end{aligned}$$

they established a relationship between the category of the set $\mathcal {V\cap Q}$ and the number of solutions by employing the Lusternik–Schnirelmann category theory.

On the other hand, the reduction methods are also used to study the Choquard equation. Wei and Winter [22] considered

$$\begin{aligned} -\varepsilon ^2\Delta u+ V(x)u=\frac{1}{8\pi \varepsilon ^2}(\frac{1}{|x|}*u^2)u, \quad x\in {\mathbb {R}}^3, \end{aligned}$$

where $\varepsilon >0$, $V\in C^2({\mathbb {R}}^3)$ and $\inf _{x\in {\mathbb {R}}^3} V(x)>0$. They proved that for any given positive integer K, if $P_1, P_2, ... , P_K\in {\mathbb {R}}^3$ were given nondegenerate critical points of V(x), then for $\varepsilon$ sufficiently small, there existed a positive solution for the equation and this solution had exactly K local maximum points $Q^{\varepsilon }_i(i=1,2,...,K)$ with $Q^{\varepsilon }_i\rightarrow P_i$ as $\varepsilon \rightarrow 0$. Luo, Peng and Wang [13] also investigated the above problem. For $\varepsilon$ small enough, by using a local Pohozaev type of identity, blow-up analysis, and the maximum principle, they showed the uniqueness of positive solutions concentrating at the nondegenerate critical points of V(x). For more results about Choquard equations, we refer to [5, 7,8,9, 16, 23, 26, 28, 31] and the references therein.

Motivated by the above works, we are concerned with the existence and concentration behavior of positive solutions for (1.1). We note that (1.1) involves three different potentials. This brings a competition between the potentials V, P, and Q: each one would like to attract ground states to their minimum or maximum points, respectively. It makes difficulties in determining the concentration position of solutions. This kind of problem can be traced back to [20, 21] for the semilinear Schrödinger equation. See also [24, 25, 27, 30] for other related results. We first recall the following famous Hardy–Littlewood–Sobolev inequality.

Proposition 1.1

(Hardy–Littlewood–Sobolev inequality). Let $t,r>1$ and $0<\mu <3$ with $\frac{1}{t}+\frac{\mu }{3}+\frac{1}{r}=2$, $f\in L^t({\mathbb {R}}^3)$ and $h\in L^r({\mathbb {R}}^3)$. There exists a sharp constant $C(t,\mu ,r)$, independent of f, h, such that

$$\begin{aligned} \int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{f(x)h(y)}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y\le C(t,\mu ,r)|f|_t|h|_r. \end{aligned}$$

Remark 1.2

By Proposition 1.1, the term

$$\begin{aligned} \int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{|u(x)|^r|u(y)|^r}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y \end{aligned}$$

is well defined if $|u|^r\in L^s({\mathbb {R}}^3)$ satisfies $\frac{2}{s}+\frac{\mu }{3}=2$. Therefore, for $u\in H^1({\mathbb {R}}^3)$, we will require $sr\in [2,6]$. Then $\frac{6-\mu }{3}\le r\le 6-\mu$. Here, $\frac{6-\mu }{3}$ is called the lower critical exponent and $6-\mu$ is called the upper critical exponent in the sense of Hardy–Littlewood–Sobolev inequality.

Proposition 1.3

(Optimizers for $S_{H,L}$). [6] Define

$$\begin{aligned} S_{H,L}=\inf _{u\in D^{1,2}({\mathbb {R}}^3)\setminus \{0\}}\frac{\int _{{\mathbb {R}}^3}|\nabla u|^2\mathrm{{d}}x}{(\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{|u(x)|^{6-\mu }|u(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y)^{\frac{1}{6-\mu }}}. \end{aligned}$$

Then $S_{H,L}$ is achieved if and only if

$$\begin{aligned} u=C\Big (\frac{b}{b^2+|x-a|^2}\Big )^{\frac{1}{2}}, \end{aligned}$$

where $C>0$ is a fixed constant, $a\in {\mathbb {R}}^3$ and $b>0$ are parameters.

Remark 1.4

[6] In fact,

$$\begin{aligned} U(x)=\frac{3^{\frac{1}{4}}}{(1+|x|^2)^{\frac{1}{2}}} \end{aligned}$$

is a minimizer for S, the best Sobolev constant, and is also the minimizer for $S_{H,L}$. Moreover,

$$\begin{aligned} S_{H,L}=\frac{S}{C(3,\mu )^{\frac{1}{6-\mu }}}, \end{aligned}$$

where $C(3,\mu )$ is the sharp constant in Proposition 1.1.

To state our main results, some hypotheses about the potential functions are needed as follows:

$(H_1)$:

$V_{\infty }>V_\mathrm{{min}}$ or $P_\mathrm{{max}}>P^{\infty }$,

$(H_2)$:

$Q(x)\le Q^{\infty }$ for $x\in {\mathbb {R}}^3$,

$(H_3)$:

$\mathcal {V\cap P\cap Q}=\{x\in {\mathbb {R}}^3: V(x)=V_{\min }, P(x)=P_\mathrm{{max}}, Q(x)=Q_\mathrm{{max}}\}\ne \varnothing$,

where

$$\begin{aligned} \begin{aligned}&V_{\min }:=\min _{x\in {\mathbb {R}}^3} V(x),~~{\mathcal {V}}:=\{x\in {\mathbb {R}}^3:V(x)=V_{\min }\},~~V_\infty :=\liminf _{|x|\rightarrow \infty }V(x),\\&P_{\max }:=\max _{x\in {\mathbb {R}}^3}P(x),~~{\mathcal {P}}:=\{x\in {\mathbb {R}}^3:P(x)=P_{\max }\},\ P^\infty :=\limsup _{|x|\rightarrow \infty }P(x),\\&Q_{\max }:=\max _{x\in {\mathbb {R}}^3}Q(x),~~{\mathcal {Q}}:=\{x\in {\mathbb {R}}^3:Q(x)=Q_{\max }\},\ Q^\infty :=\limsup _{|x|\rightarrow \infty }Q(x).\\ \end{aligned} \end{aligned}$$

Obviously, under the assumptions $(H_1)$, the set $\mathcal {V\cap P\cap Q}$ is bounded.

Our main results are as follows:

Theorem 1.5

Suppose that the potentials V(x), P(x), Q(x) satisfy conditions $(H_1)$, $(H_2)$ and $(H_3)$. Then

• (i) For any $\delta >0$, there exists $\varepsilon _\delta >0$ such that problem (1.1) has at least $cat_{({\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}})_\delta }({\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}})$ solutions for $\varepsilon \in (0,\varepsilon _\delta )$, where $({\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}})_\delta =\{x\in {\mathbb {R}}^3: dist(x,{\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}})\le \delta \}$.

• (ii) For $\varepsilon _n\rightarrow 0$ as $n\rightarrow \infty ,$ up to a subsequence, there exists $y_n$ such that $u_{\varepsilon _n}(x+y_n)$, where $u_{\varepsilon _n}$ is a solution in (i), converges in $H^1({\mathbb {R}}^3)$ to a ground state solution u of

  $$\begin{aligned} -\Delta u+ V_\mathrm{{min}}u=P_\mathrm{{max}}|u|^{p-2}u+\Big (\int _{{\mathbb {R}}^3}\frac{Q^2_\mathrm{{max}}|u(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}y\Big )|u|^{4-\mu }u, \ \ x\in {\mathbb {R}}^3,\nonumber \\ \end{aligned}$$

  (1.2)

The proof of our main results is based on the variational method. The main difficulties lie in two aspects: (i) The unboundedness of the domain ${\mathbb {R}}^3$ and the critical exponent under the sense of Hardy–Littlewood–Sobolev inequality lead to the lack of compactness. Some arguments developed by Brezis and Nirenberg [3] can be applied to prove that the functional associated with (1.1) satisfies the Palais-Smale (PS) condition under some energy level. (ii) When the critical term has a potential Q(x), the proof of the existence of multiple solutions become more complicated. As far as we know, there are no results about this problem. By using Lusternik–Schnirelmann theory, we establish the relationship between the category of the set ${\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}}$ and the number of solutions.

This paper is organized as follows. In the forthcoming section, we collect some necessary preliminary lemmas which will be used later. In Sect. 3, we are devoted to the energy functional with constant coefficients. In Sect. 4, the PS condition is given. In Sect. 5, the Lusternik–Schnirelmann theory is applied to prove the existence of multiple solutions.

Notation. In this paper, we make use of the following notations.

• For any $R>0$ and $x\in {\mathbb {R}}^3$, $B_{R}(x)$ denotes the open ball of radius R centered at x.

• The letter C stands for positive constants (possibly different from line to line).

• ”$\rightarrow$” denotes the strong convergence and "$\rightharpoonup$" denotes the weak convergence.

• $|u|_q=(\int _{{\mathbb {R}}^3}|u|^q\mathrm{{d}}x)^{\frac{1}{q}}$denotes the norm of u in $L^q({\mathbb {R}}^3)$ for $2\le q\le 6$.

2 Preliminaries

The standard norm of $E:=H^1({\mathbb {R}}^3)$ is given by

$$\begin{aligned} \Vert u\Vert =\Big (\int _{{\mathbb {R}}^3}(|\nabla u|^2+u^2)\mathrm{{d}}x\Big )^{1/2}. \end{aligned}$$

Since V(x) is bounded and $\inf _{x\in {\mathbb {R}}^3} V(x)>0$, we have the following equivalent norm

$$\begin{aligned} \Vert u\Vert _\varepsilon =\Big (\int _{{\mathbb {R}}^3}(|\nabla u|^2+V(\varepsilon x)u^2)\mathrm{{d}}x\Big )^{1/2}. \end{aligned}$$

For $f\in L^1_{loc}({\mathbb {R}}^3)$, define

$$\begin{aligned} I_\mu *f(x)=\int _{{\mathbb {R}}^3}\frac{f(y)}{|x-y|^\mu }\mathrm{{d}}y, \end{aligned}$$

and this integral converges in the classical Lebesgue sense for a.e. $x\in {\mathbb {R}}^3$ if and only if $f\in L^1({\mathbb {R}}^3,(1+|x|)^{-\mu }\mathrm{{d}}x).$

Remark 2.1

By Hardy–Littlewood–Sobolev inequality, $I_\mu$ defines a linear continuous map from $L^{\frac{6}{6-\mu }}({\mathbb {R}}^3)$ to $L^{\frac{6}{\mu }}({\mathbb {R}}^3)$.

Define $F:E\rightarrow {\mathbb {R}}$ by

$$\begin{aligned}F(u)=\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3} \frac{Q(\varepsilon x)Q(\varepsilon y)|u(x)|^{6-\mu }|u(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y. \end{aligned}$$

To prove the properties about $F(\cdot )$, for simplicity, we assume that $Q(x)\equiv 1$ in the following three Lemmas.

Lemma 2.2

Let $u_n\rightharpoonup u$ in E and $u_n\rightarrow u$, a.e. in ${\mathbb {R}}^3$. Then

$$\begin{aligned} I_\mu *|u_n|^{6-\mu }\rightarrow I_\mu *|u|^{6-\mu }, \ \text {a.e. in}\ {\mathbb {R}}^3, \quad \text {as}\ n\rightarrow \infty . \end{aligned}$$

Proof

By Hardy–Littlewood–Sobolev inequality, $I_\mu *|u_n|^{6-\mu }\in L^{\frac{6}{\mu }}({\mathbb {R}}^3)$. Choose a function $v\in L^{\frac{6}{6-\mu }}({\mathbb {R}}^3)$ satisfying $v>0$ in ${\mathbb {R}}^3$. Then

$$\begin{aligned} \begin{aligned}&\int _{{\mathbb {R}}^3}\Big |I_\mu *|u_n|^{6-\mu }-I_\mu *|u|^{6-\mu }\Big |v\mathrm{{d}}x\\&\le \int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{\big ||u_n(y)|^{6-\mu }-|u(y)|^{6-\mu }\big |v(x)}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y\\&=\int _{{\mathbb {R}}^3}I_\mu *v\big ||u_n(y)|^{6-\mu }-|u(y)|^{6-\mu }\big |\mathrm{{d}}y. \end{aligned} \end{aligned}$$

Since $I_\mu *v\in L^{\frac{6}{\mu }}({\mathbb {R}}^3)$, and $\Big ||u_n(y)|^{6-\mu }-|u(y)|^{6-\mu }\Big |\rightharpoonup 0$ in $L^{\frac{6}{6-\mu }}({\mathbb {R}}^3)$, we can obtain

$$\begin{aligned} \int _{{\mathbb {R}}^3}\Big |I_\mu *|u_n|^{6-\mu }-I_\mu *|u|^{6-\mu }\Big |v\mathrm{{d}}x\rightarrow 0,\ \text {as}\ n\rightarrow \infty . \end{aligned}$$

It follows from $v>0$ that the result holds. $\square$

Lemma 2.3

Let $u_n\rightharpoonup u$ in E and $u_n\rightarrow u$, a.e. in ${\mathbb {R}}^3$. Then

$$\begin{aligned} \int _{{\mathbb {R}}^3}(I_\mu *|u_n-u|^{6-\mu }|u|^{5-\mu })^{\frac{6}{5}}\mathrm{{d}}x\rightarrow 0, \ \text {a.e. in}\ {\mathbb {R}}^3, \quad \text {as}\ n\rightarrow \infty , \end{aligned}$$

and

$$\begin{aligned} \int _{{\mathbb {R}}^3}(I_\mu *|u|^{6-\mu }|u_n-u|^{5-\mu })^{\frac{6}{5}}\mathrm{{d}}x\rightarrow 0, \ \text {a.e. in}\ {\mathbb {R}}^3, \quad \text {as}\ n\rightarrow \infty . \end{aligned}$$

Proof

Let $v_n=u_n-u$, then

$$\begin{aligned} \int _{{\mathbb {R}}^3}(I_\mu *|u_n-u|^{6-\mu }|u|^{5-\mu })^{\frac{6}{5}}\mathrm{{d}}x= \int _{{\mathbb {R}}^3}(I_\mu *|v_n|^{6-\mu })^{\frac{6}{5}}|u|^{\frac{6(5-\mu )}{5}}\mathrm{{d}}x. \end{aligned}$$

By Hardy–Littlewood–Sobolev inequality, $(I_\mu *|v_n|^{6-\mu })^{\frac{6}{5}}\in L^{\frac{5}{\mu }}({\mathbb {R}}^3)$, and is bounded in $L^{\frac{5}{\mu }}({\mathbb {R}}^3)$. From Lemma 2.2, $I_\mu *|v_n|^{6-\mu }\rightarrow 0, \ \text {a.e. in}\ {\mathbb {R}}^3, \ \text {as}\ n\rightarrow \infty$. Then, we have $(I_\mu *|v_n|^{6-\mu })^{\frac{6}{5}}\rightharpoonup 0$ in $L^{\frac{5}{\mu }}({\mathbb {R}}^3)$. It follows from $|u|^{\frac{6(5-\mu )}{5}}\in L^{\frac{5}{5-\mu }}({\mathbb {R}}^3)$ that the first result holds. Similarly, the second limit can be obtained. $\square$

Lemma 2.4

Let $u_n\rightharpoonup u$ in E and $u_n\rightarrow u$, a.e. in ${\mathbb {R}}^3$. Then

• (i) $F(u_n-u)=F(u_n)-F(u)+o_n(1)$;

• (ii) $F^\prime (u_n-u)=F^\prime (u_n)-F^\prime (u)+o_n(1)$, in $(H^1({\mathbb {R}}^3))^{-1}$.

Proof

The first part (i) has been proved in [6]. We just prove the second part (ii). In fact, for any $\phi \in H^1({\mathbb {R}}^3)$,

$$\begin{aligned} \begin{aligned}&\Big |\langle F^\prime (u_n-u),\phi \rangle -\langle F^\prime (u_n),\phi \rangle +\langle F^\prime (u),\phi \rangle \Big |\\&\le C\Big |\int _{{\mathbb {R}}^3}\big (I_\mu *|u_n-u|^{6-\mu }|u_n-u|^{4-\mu }(u_n-u)-I_\mu *|u_n|^{6-\mu }|u_n|^{4-\mu }u_n\\&\quad +I_\mu *|u|^{6-\mu }|u|^{4-\mu }u \big )\phi \Big |\\&\le C\Big |I_\mu *|u_n-u|^{6-\mu }|u_n-u|^{4-\mu }(u_n-u)-I_\mu *|u_n|^{6-\mu }|u_n|^{4-\mu }u_n\\&\quad +I_\mu *|u|^{6-\mu }|u|^{4-\mu }u\Big |_{\frac{6}{5}}|\phi |_6\\&\le C\Big |I_\mu *|u_n-u|^{6-\mu }|u_n-u|^{4-\mu }(u_n-u)-I_\mu *|u_n|^{6-\mu }|u_n|^{4-\mu }u_n\\&\quad +I_\mu *|u|^{6-\mu }|u|^{4-\mu }u\Big |_{\frac{6}{5}}\Vert \phi \Vert . \end{aligned} \end{aligned}$$

Next, we prove that

$$\begin{aligned}&\Big |I_\mu *|u_n-u|^{6-\mu }|u_n-u|^{4-\mu }(u_n-u)-I_\mu *|u_n|^{6-\mu }|u_n|^{4-\mu }u_n\\&\quad +I_\mu *|u|^{6-\mu }|u|^{4-\mu }u\Big |_{\frac{6}{5}}=o_n(1). \end{aligned}$$

Let $v_n=u_n-u$. Then, for any small $\delta >0$,

$$\begin{aligned} \begin{aligned}&\Big |I_\mu *|u_n-u|^{6-\mu }|u_n-u|^{4-\mu }(u_n-u)-I_\mu *|u_n|^{6-\mu }|u_n|^{4-\mu }u_n\Big |\\&=\Big |I_\mu *|v_n|^{6-\mu }|v_n|^{4-\mu }v_n-I_\mu *|v_n+u|^{6-\mu }|v_n+u|^{4-\mu }(v_n+u)\Big |\\&=\Big |\int _0^1\frac{\mathrm{{d}}}{\mathrm{{d}}t}\Big (I_\mu *|v_n+tu|^{6-\mu }|v_n+tu|^{4-\mu }(v_n+tu)\Big )\mathrm{{d}}t\Big |\\&\le C\Big [I_\mu *(|v_n|^{5-\mu }|u|+|u|^{6-\mu })(|v_n|^{5-\mu }+|u|^{5-\mu })+I_\mu *(|v_n|^{6-\mu }\\&\quad +|u|^{6-\mu })(|v_n|^{4-\mu }|u|+|u|^{5-\mu })\Big ]\\&\le C\Big [(\delta I_\mu *|v_n|^{6-\mu }+C(\delta )I_\mu *|u|^{6-\mu })(|v_n|^{5-\mu }+|u|^{5-\mu })\\&\quad +I_\mu *(|v_n|^{6-\mu }+|u|^{6-\mu })(\delta |v_n|^{5-\mu }+C(\delta )|u|^{5-\mu })\Big ]\\&\le C\Big [\delta (I_\mu *|v_n|^{6-\mu }|v_n|^{5-\mu }+I_\mu *|v_n|^{6-\mu }|u|^{5-\mu }+I_\mu *|u|^{6-\mu }|v_n|^{5-\mu })\\&\quad +C(\delta )(I_\mu *|u|^{6-\mu }|v_n|^{5-\mu }+I_\mu *|v_n|^{6-\mu }|u|^{5-\mu })+C(\delta )I_\mu *|u|^{6-\mu }|u|^{5-\mu }\Big ]\\&=C\Big [\delta f_n+C(\delta )g_n+C(\delta )I_\mu *|u|^{6-\mu }|u|^{5-\mu } \Big ], \end{aligned} \end{aligned}$$

where

$$\begin{aligned} f_n=I_\mu *|v_n|^{6-\mu }|v_n|^{5-\mu }+I_\mu *|v_n|^{6-\mu }|u|^{5-\mu }+I_\mu *|u|^{6-\mu }|v_n|^{5-\mu }, \end{aligned}$$

and

$$\begin{aligned} g_n=I_\mu *|u|^{6-\mu }|v_n|^{5-\mu }+I_\mu *|v_n|^{6-\mu }|u|^{5-\mu }. \end{aligned}$$

Therefore,

$$\begin{aligned} \begin{aligned}&\Big |I_\mu *|v_n|^{6-\mu }|v_n|^{4-\mu }v_n-I_\mu *|u_n|^{6-\mu }|u_n|^{4-\mu }u_n+I_\mu *|u|^{6-\mu }|u|^{4-\mu }u\Big |^{\frac{6}{5}}\\&\le C\Big [\delta (f_n)^{\frac{6}{5}}+C(\delta )(g_n)^{\frac{6}{5}}+C(\delta )(I_\mu *|u|^{6-\mu }|u|^{5-\mu })^{\frac{6}{5}} \Big ]. \end{aligned} \end{aligned}$$

Define

$$\begin{aligned} G_{\delta ,n}(x)= & {} \max \Big \{\Big |I_\mu *|v_n|^{6-\mu }|v_n|^{4-\mu }v_n-I_\mu *|u_n|^{6-\mu }|u_n|^{4-\mu }u_n +I_\mu *|u|^{6-\mu }|u|^{4-\mu }u\Big |^{\frac{6}{5}}\\&-C\delta (f_n)^{\frac{6}{5}}-CC(\delta )(g_n)^{\frac{6}{5}},0 \Big \}. \end{aligned}$$

It is easy to see that

$$\begin{aligned} 0\le G_{\delta ,n}(x)\le C(\delta )(I_\mu *|u|^{6-\mu }|u|^{5-\mu })^{\frac{6}{5}}\in L^1({\mathbb {R}}^3). \end{aligned}$$

By Lemma 2.2, we can obtain

$$\begin{aligned} f_n\rightarrow 0,\ \text {a.e. in}\ {\mathbb {R}}^3,\ \text {as}\ n\rightarrow \infty ,\\ g_n\rightarrow 0,\ \text {a.e. in}\ {\mathbb {R}}^3,\ \text {as}\ n\rightarrow \infty , \end{aligned}$$

and

$$\begin{aligned}&\Big |I_\mu *|v_n|^{6-\mu }|v_n|^{4-\mu }v_n-I_\mu *|u_n|^{6-\mu }|u_n|^{4-\mu }u_n +I_\mu *|u|^{6-\mu }|u|^{4-\mu }u\Big |\rightarrow 0,\\&\quad \ \text {a.e. in}\ {\mathbb {R}}^3,\ \text {as}\ n\rightarrow \infty . \end{aligned}$$

Thus,

$$\begin{aligned} G_{\delta ,n}(x)\rightarrow 0,\ \text {a.e. in}\ {\mathbb {R}}^3,\ \text {as}\ n\rightarrow \infty . \end{aligned}$$

Then, we have

$$\begin{aligned} \int _{{\mathbb {R}}^3}G_{\delta ,n}(x)\rightarrow 0,\ \text {as}\ n\rightarrow \infty . \end{aligned}$$

(2.1)

By the definition of $G_{\delta ,n(x)}$ and the boundedness of $f_n$ in $L^{\frac{6}{5}}({\mathbb {R}}^3)$,

$$\begin{aligned} \begin{aligned}&\int _{{\mathbb {R}}^3}\Big |I_\mu *|v_n|^{6-\mu }|v_n|^{4-\mu }v_n-I_\mu *|u_n|^{6-\mu } |u_n|^{4-\mu }u_n+I_\mu *|u|^{6-\mu }|u|^{4-\mu }u\Big |^{\frac{6}{5}}\mathrm{{d}}x\\&\le C\Big [\delta \int _{{\mathbb {R}}^3}(f_n)^{\frac{6}{5}}\mathrm{{d}}x+C(\delta ) \int _{{\mathbb {R}}^3}(g_n)^{\frac{6}{5}}\mathrm{{d}}x \Big ]+\int _{{\mathbb {R}}^3}G_{\delta ,n}(x)\mathrm{{d}}x\\&\le C\Big [C\delta +C(\delta )\int _{{\mathbb {R}}^3}(g_n)^{\frac{6}{5}}\mathrm{{d}}x \Big ]+\int _{{\mathbb {R}}^3}G_{\delta ,n}(x)\mathrm{{d}}x \end{aligned} \end{aligned}$$

Thus, by Lemma 2.2 and (2.1),

$$\begin{aligned} \limsup _{n\rightarrow \infty }\int _{{\mathbb {R}}^3}\Big |I_\mu *|v_n|^{6-\mu }|v_n|^{4-\mu }v_n -I_\mu *|u_n|^{6-\mu }|u_n|^{4-\mu }u_n+I_\mu *|u|^{6-\mu }|u|^{4-\mu }u\Big |^{\frac{6}{5}}\mathrm{{d}}x\le C\delta . \end{aligned}$$

It follows from the arbitrariness of $\delta$ that

$$\begin{aligned} \Big |I_\mu *|v_n|^{6-\mu }|v_n|^{4-\mu }v_n-I_\mu *|u_n|^{6-\mu }|u_n|^{4-\mu }u_n+I_ \mu *|u|^{6-\mu }|u|^{4-\mu }u\Big |_{\frac{6}{5}}=o_n(1). \end{aligned}$$

$\square$

Making the change of variable $x\rightarrow \varepsilon x$, we can rewrite problem (1.1) as

$$\begin{aligned} -\Delta u+ V(\varepsilon x)u=P(\varepsilon x)|u|^{p-2}u+\Big (\int _{{\mathbb {R}}^3}\frac{Q(\varepsilon y)|u(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}y\Big )Q(\varepsilon x)|u|^{4-\mu }u, \quad x\in {\mathbb {R}}^3.\nonumber \\ \end{aligned}$$

(2.2)

Thus, the corresponding energy functional is

$$\begin{aligned} I_\varepsilon (u)=\frac{1}{2}\Vert u\Vert ^2_\varepsilon -\frac{1}{p}\int _{{\mathbb {R}}^3}P(\varepsilon x)|u|^{p}\mathrm{{d}}x-\frac{1}{2(6-\mu )}F(u). \end{aligned}$$

It is easy to check that $I_\varepsilon$ is well defined on E and $I_\varepsilon \in C^1(E,{\mathbb {R}}).$ Then we can define the Nehari manifold

$$\begin{aligned} {\mathcal {N}}_\varepsilon =\{u\in E\setminus \{0\}\ |\ \langle I'_\varepsilon (u), u\rangle =0\}. \end{aligned}$$

Lemma 2.5

There exists $C_0>0$ which is independent of $\varepsilon$ such that

$$\begin{aligned} \Vert u\Vert _\varepsilon >C_0 \quad \text {and}\quad I_\varepsilon (u)\ge \frac{p-2}{2p}C_0^2, \quad \text {for all}\ u\in {\mathcal {N}}_\varepsilon . \end{aligned}$$

Proof

For any $u\in {\mathcal {N}}_\varepsilon$, we have

$$\begin{aligned} \Vert u\Vert ^2_\varepsilon =\int _{{\mathbb {R}}^3}P(\varepsilon x)|u|^{p}\mathrm{{d}}x+F(u). \end{aligned}$$

It follows from the Hardy–Littlewood–Sobolev inequality and Sobolev embedding theorem that

$$\begin{aligned} F(u)=\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3} \frac{Q(\varepsilon x)Q(\varepsilon y)|u(x)|^{6-\mu }|u(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y\le C |u|^{2(6-\mu )}_6\le C \Vert u\Vert ^{2(6-\mu )}_\varepsilon . \end{aligned}$$

Without loss of generality, we assume that $\Vert u\Vert _\varepsilon \le 1$. Then

$$\begin{aligned} \Vert u\Vert ^2_\varepsilon \le C(\Vert u\Vert ^{p}_\varepsilon +\Vert u\Vert ^{2(6-\mu )}_\varepsilon )\le C\Vert u\Vert ^{p}_\varepsilon . \end{aligned}$$

Thus, the first desired result follows. On the other hand, we have

$$\begin{aligned} \begin{aligned} I_\varepsilon (u)&=\frac{1}{2}\Vert u\Vert ^2_\varepsilon -\frac{1}{p}\int _{{\mathbb {R}}^3}P(\varepsilon x)|u|^{p}\mathrm{{d}}x-\frac{1}{2(6-\mu )}F(u)\\&\ge \frac{1}{2}\Vert u\Vert ^2_\varepsilon -\frac{1}{p}\int _{{\mathbb {R}}^3}P(\varepsilon x)|u|^{p}\mathrm{{d}}x-\frac{1}{p}F(u)\\&=\frac{1}{2}\Vert u\Vert ^2_\varepsilon -\frac{1}{p}(\int _{{\mathbb {R}}^3}P(\varepsilon x)|u|^{p}\mathrm{{d}}x+F(u))\\&\ge (\frac{1}{2}-\frac{1}{p})\Vert u\Vert ^2_\varepsilon \\&\ge \frac{p-2}{2p}C_0^2. \end{aligned} \end{aligned}$$

$\square$

Lemma 2.6

For any $u\in E\setminus \{0\}$, there exists a unique $t(u)>0$ such that $t(u)u\in ({\mathcal {N}})_\varepsilon$ and

$$\begin{aligned}I_\varepsilon (t(u)u)=\max _{t\ge 0}I_\varepsilon (tu).\end{aligned}$$

Proof

For any $u\in E\setminus \left\{ 0\right\}$, define $g(t)=I_\varepsilon (tu),\ t\in [0,+\infty ).$ Then

$$\begin{aligned} g(t)=\frac{t^2}{2}\Vert u\Vert ^2_\varepsilon -\frac{t^{p}}{p}\int _{{\mathbb {R}}^3}P(\varepsilon x)|u|^{p}\mathrm{{d}}x -\frac{t^{2(6-\mu )}}{2(6-\mu )}F(u). \end{aligned}$$

It is easy to see that $g(t)>0$ for $t>0$ small and $g(t)<0$ for $t>0$ large enough, so there exists $t_0>0$ such that

$$\begin{aligned} g'(t_0)=0\quad \hbox {and}\quad g(t_0)=\max _{t\ge 0}g(t)=\max _{t\ge 0}I_\varepsilon (tu). \end{aligned}$$

It follows from $g'(t_0)=0$ that $t_0u\in {\mathcal {N}}_\varepsilon$.

If there exist $0<t_1<t_2$ such that $t_1u\in {\mathcal {N}}_\varepsilon$ and $t_2u\in {\mathcal {N}}_\varepsilon$. Then

$$\begin{aligned} \frac{1}{t_1^{p-2}}\Vert u\Vert ^2_\varepsilon =\int _{{\mathbb {R}}^3}P(\varepsilon x)|u|^{p}\mathrm{{d}}x+t_1^{2(6-\mu )-p}F(u), \end{aligned}$$

and

$$\begin{aligned} \frac{1}{t_2^{p-2}}\Vert u\Vert ^2_\varepsilon =\int _{{\mathbb {R}}^3}P(\varepsilon x)|u|^{p}\mathrm{{d}}x+t_2^{2(6-\mu )-p}F(u). \end{aligned}$$

It follows that

$$\begin{aligned} \left( \frac{1}{t_1^{p-2}}-\frac{1}{t_2^{p-2}}\right) \Vert u\Vert ^2_\varepsilon =\left( t_1^{2(6-\mu )-p}-t_2^{2(6-\mu )-p}\right) F(u), \end{aligned}$$

which is a contradiction. $\square$

Lemma 2.7

For any $\varepsilon >0$, let

$$\begin{aligned} c_\varepsilon =\inf _{u\in {\mathcal {N}}_\varepsilon }I_\varepsilon (u),\quad c_\varepsilon ^*=\inf _{u\in E\setminus \{0\}}\max _{t\ge 0}I_\varepsilon (tu), \quad c_\varepsilon ^{**}=\inf _{\gamma \in \Gamma }\sup _{t\in [0,1]}I_\varepsilon (\gamma (t)), \end{aligned}$$

where

$$\begin{aligned} \Gamma _\varepsilon =\{\gamma (t)\in C([0,1],E)\ |\ \gamma (0)=0,\ I_\varepsilon (\gamma (1))<0\}. \end{aligned}$$

Then, $c_\varepsilon =c_\varepsilon ^*=c_\varepsilon ^{**}.$

Proof

We divide the proof into three steps.

Step1. $c_\varepsilon ^*=c_\varepsilon$. By Lemma 2.6, we have

$$\begin{aligned}c_\varepsilon ^*=\inf _{u\in E\setminus \{0\}}\max _{t\ge 0}I_\varepsilon (tu) =\inf _{u\in E\setminus \{0\}}I_\varepsilon (t(u)u)=\inf _{u\in {\mathcal {N}}_\varepsilon }I_\varepsilon (u)=c_\varepsilon . \end{aligned}$$

Step2. $c_\varepsilon ^*\ge c_\varepsilon ^{**}.$ For any $u\in E\setminus \{0\}$, there exists T large enough, such that $I_\varepsilon (Tu)<0$. Define $\gamma (t)=tTu$, $t\in [0,1]$. Then we have $\gamma (t)\in \Gamma _\varepsilon$ and, therefore,

$$\begin{aligned} c_\varepsilon ^{**}=\inf _{\gamma \in \Gamma _\varepsilon }\sup _{t\in [0,1]} I_\varepsilon (\gamma (t))\le \sup _{t\in [0,1]}I_\varepsilon (\gamma (t))\le \max _{t\ge 0}I_\varepsilon (tu). \end{aligned}$$

It follows that $c_\varepsilon ^*\ge c_\varepsilon ^{**}$.

Step3. $c_\varepsilon ^{**}\ge c_\varepsilon$. For any $u\in E\setminus \{0\}$ with $\Vert u\Vert _\varepsilon$ small, we know

$$\begin{aligned} \Vert u\Vert ^2_\varepsilon >\int _{{\mathbb {R}}^3}P(\varepsilon x)|u|^{p}\mathrm{{d}}x+F(u). \end{aligned}$$

(2.3)

We claim that every $\gamma (t)\in \Gamma _\varepsilon$ has to cross ${\mathcal {N}}_\varepsilon$. Otherwise, by the continuity of $\gamma (t)$, (2.3) still holds when u is replaced by $\gamma (1)$. Then, we can obtain

$$\begin{aligned} \begin{aligned} I_\varepsilon (\gamma (1))&=\frac{1}{2}\Vert \gamma (1)\Vert ^2_\varepsilon -\frac{1}{p}\int _{{\mathbb {R}}^3}P(\varepsilon x)|\gamma (1)|^{p}\mathrm{{d}}x-\frac{1}{2(6-\mu )}F(\gamma (1))\\&\ge \frac{1}{2}\Vert \gamma (1)\Vert ^2_\varepsilon -\frac{1}{p}\int _{{\mathbb {R}}^3}P(\varepsilon x)|\gamma (1)|^{p}\mathrm{{d}}x-\frac{1}{p}F(\gamma (1))\\&\ge \frac{1}{2}\Vert \gamma (1)\Vert ^2_\varepsilon -\frac{1}{p}\Vert \gamma (1)\Vert ^2_\varepsilon \\&> 0, \end{aligned} \end{aligned}$$

which contradicts the definition of $\gamma (1)$. It follows from the claim that $c_\varepsilon ^{**}\ge c_\varepsilon$. $\square$

One can easily check that the functional $I_\varepsilon$ satisfies the mountain-pass geometry that is the following lemma holds ( [26]).

Lemma 2.8

$I_\varepsilon$ has the mountain geometry structure.

• (i) There exist $a_0,r_0>0$ independent of $\varepsilon$, such that $I_\varepsilon (u)\ge a_0$, for all $u\in E$ with $\Vert u\Vert _\varepsilon =r_0.$

• (ii) For any $u\in E\setminus \{0\}$, $\lim _{t\rightarrow \infty }I_\varepsilon (tu)=-\infty .$

Lemma 2.9

For any $\varepsilon >0$ and $Q(x)\equiv q$, we have $c_\varepsilon <\displaystyle \frac{5-\mu }{2(6-\mu )} S^{\frac{6-\mu }{5-\mu }}_{H,L}q^{\frac{-2}{5-\mu }}$, where q is a positive constant.

Proof

For any $\epsilon >0$, define

$$\begin{aligned} U_\epsilon (x)=\frac{1}{\sqrt{\epsilon }}U\left( \frac{x}{\epsilon }\right) ,\ u_\epsilon (x)=\phi (x)U_\epsilon (x),\ x\in {\mathbb {R}}^3, \end{aligned}$$

where $\phi (x)\in C_0^\infty ({\mathbb {R}}^3)$ is such that $\phi =1$ on $B_1(0)$ and $\phi =0$ on $B^c_2(0)$. From Lemma 2.6 in [1], we know that

$$\begin{aligned} \int _{{\mathbb {R}}^3}|\nabla u_\epsilon |^2\mathrm{{d}}x=C(3,\mu )^{\frac{3}{2(6-\mu )}}S_{H,L}^{\frac{3}{2}}+O(\epsilon ), \end{aligned}$$

(2.4)

$$\begin{aligned} \int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3} \frac{|u_\epsilon (x)|^{6-\mu }|u_\epsilon (y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y\ge C(3,\mu )^{\frac{3}{2}}S_{H,L}^{\frac{6-\mu }{2}}-O(\epsilon ^{\frac{6-\mu }{2}}), \end{aligned}$$

(2.5)

and

$$\begin{aligned} \int _{{\mathbb {R}}^3}|u_\epsilon |^t\mathrm{{d}}x= {\left\{ \begin{array}{ll} O(\epsilon ^{\frac{6-t}{2}}),\quad \quad &{}t\in (3,6),\\ O(\epsilon ^{\frac{3}{2}}|ln\epsilon |),&{}t=3,\\ O(\epsilon ^{\frac{t}{2}}),&{}t\in [2,3). \end{array}\right. } \end{aligned}$$

(2.6)

Then, for $t>0$,

$$\begin{aligned} \begin{aligned} I_\varepsilon (tu_\epsilon )=&\frac{t^2}{2}\int _{{\mathbb {R}}^3}(|\nabla u_\epsilon |^2 +V(\varepsilon x)u^2_\epsilon ) \mathrm{{d}}x-\frac{t^p}{p}\int _{{\mathbb {R}}^3}P(\varepsilon x)|u_\epsilon |^p \mathrm{{d}}x\\&-\frac{q^2t^{2(6-\mu )}}{2(6-\mu )}\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3} \frac{|u_\epsilon (x)|^{6-\mu }|u_\epsilon (y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y\\ \le&\frac{t^2}{2}(C(3,\mu )^{\frac{3}{2(6-\mu )}}S_{H,L}^{\frac{3}{2}}+O(\epsilon )) -\frac{q^2t^{2(6-\mu )}}{2(6-\mu )}( C(3,\mu )^{\frac{3}{2}}S_{H,L}^{\frac{6-\mu }{2}}-O(\epsilon ^{\frac{6-\mu }{2}}))\\&+C(t^2O(\epsilon )-t^pO(\epsilon ^{\frac{6-p}{2}})):=h(t). \end{aligned} \end{aligned}$$

It is easy to see that $h(t)\rightarrow -\infty$ as $t\rightarrow +\infty$, $h(0)=0$ and $h(t)>0$ as t is small. Therefore, there exists $t_\varepsilon >0$ such that h(t) attains its maximum. Then, differentiating h at $t_\epsilon$, we can obtain

$$\begin{aligned} \begin{aligned}&(C(3,\mu )^{\frac{3}{2(6-\mu )}}S_{H,L}^{\frac{3}{2}}+O(\epsilon )) -t_\epsilon ^{2(6-\mu )-2}q^2(C(3,\mu )^{\frac{3}{2}}S_{H,L}^{\frac{6-\mu }{2}}-O(\epsilon ^{\frac{6-\mu }{2}}))\\&\quad =-C(O(\epsilon )-t_\epsilon ^{p-2}O(\epsilon ^{\frac{6-p}{2}})). \end{aligned} \end{aligned}$$

When $\epsilon$ is small enough, it follows from the above expression that there exist $t_1,t_2>0$ independent of $\epsilon$ such that $t_1<t_\epsilon <t_2$. Noting

$$\begin{aligned} \frac{t^2}{2}(C(3,\mu )^{\frac{3}{2(6-\mu )}}S_{H,L}^{\frac{3}{2}}+O(\epsilon )) -\frac{q^2t^{2(6-\mu )}}{2(6-\mu )}( C(3,\mu )^{\frac{3}{2}}S_{H,L}^{\frac{6-\mu }{2}}-O(\epsilon ^{\frac{6-\mu }{2}})) \end{aligned}$$

attains its maximum at

$$\begin{aligned} \Big (\frac{C(3,\mu )^{\frac{3}{2(6-\mu )}}S_{H,L}^{\frac{3}{2}}+O(\epsilon )}{q^2(C(3,\mu )^{\frac{3}{2}}S_{H,L}^{\frac{6-\mu }{2}}-O(\epsilon ^{\frac{6-\mu }{2}}))}\Big )^{\frac{1}{2(6-\mu )-2}}. \end{aligned}$$

Then, we have

$$\begin{aligned} \begin{aligned} h(t_\epsilon )\le&\frac{5-\mu }{2(6-\mu )}q^{\frac{-2}{5-\mu }}\Big (\frac{C(3,\mu )^{\frac{3}{2(6-\mu )}}S_{H,L}^{\frac{3}{2}}+O(\epsilon )}{(C(3,\mu )^{\frac{3}{2}}S_{H,L}^{\frac{6-\mu }{2}}-O(\epsilon ^{\frac{6-\mu }{2}}))^\frac{1}{6-\mu }}\Big )^{\frac{6-\mu }{5-\mu }}+C\left( t_\epsilon ^2O(\epsilon )-t_\epsilon ^pO(\epsilon ^{\frac{6-p}{2}})\right) \\ \le&\frac{5-\mu }{2(6-\mu )}q^{\frac{-2}{5-\mu }}\Big (\frac{C(3,\mu )^{\frac{3}{2(6-\mu )}}S_{H,L}^{\frac{3}{2}}+O(\epsilon )}{(C(3,\mu )^{\frac{3}{2}}S_{H,L}^{\frac{6-\mu }{2}}-O(\epsilon ^{\frac{6-\mu }{2}}))^\frac{1}{6-\mu }}\Big )^{\frac{6-\mu }{5-\mu }}+C\left( t_2^2O(\epsilon )-t_1^pO(\epsilon ^{\frac{6-p}{2}})\right) \\ \le&\frac{5-\mu }{2(6-\mu )}q^{\frac{-2}{5-\mu }}\Big (\frac{C(3,\mu )^{\frac{3}{2(6-\mu )}}S_{H,L}^{\frac{3}{2}})}{(C(3,\mu )^{\frac{3}{2}}S_{H,L}^{\frac{6-\mu }{2}})^\frac{1}{6-\mu }}\Big )^{\frac{6-\mu }{5-\mu }}+O(\epsilon )+C\left( t_2^2O(\epsilon )-t_1^pO(\epsilon ^{\frac{6-p}{2}})\right) .\\ \end{aligned} \end{aligned}$$

Since $p\in (4,6)$, then $0<\frac{6-p}{2}<1$. Thus, as $\epsilon$ is small enough, we have

$$\begin{aligned} O(\epsilon )+C(t_2^2O(\epsilon )-t_1^pO(\epsilon ^{\frac{6-p}{2}}))<0. \end{aligned}$$

Then, we can get

$$\begin{aligned} h(t_\epsilon )<\frac{5-\mu }{2(6-\mu )}q^{\frac{-2}{5-\mu }}\Big (\frac{C(3,\mu )^{\frac{3}{2(6-\mu )}} S_{H,L}^{\frac{3}{2}})}{(C(3,\mu )^{\frac{3}{2}}S_{H,L}^{\frac{6-\mu }{2}})^\frac{1}{6-\mu }}\Big )^{\frac{6-\mu }{5-\mu }}=\frac{5-\mu }{2(6-\mu )}q^{\frac{-2}{5-\mu }}S^{\frac{6-\mu }{5-\mu }}_{H,L}. \end{aligned}$$

By Lemma 2.7, the proof is completed. $\square$

Lemma 2.10

Any $(PS)_{c}$ sequence $\left\{ u_n\right\}$ for $I_\varepsilon$ is bounded, and

$$\begin{aligned} \limsup _{n\rightarrow \infty } \Vert u_n\Vert _\varepsilon \le \sqrt{\frac{2pc}{p-2}}. \end{aligned}$$

Proof

Suppose that $\left\{ u_n\right\}$ is a $(PS)_{c}$ sequence of $I_\varepsilon$, we have

$$\begin{aligned}I_\varepsilon (u_n)\rightarrow c,\quad I'_\varepsilon (u_n)\rightarrow 0 . \end{aligned}$$

Thus

$$\begin{aligned} \begin{aligned} c+o_n(1)+o_n(1)\Vert u_n\Vert _\varepsilon&=I_\varepsilon (u_n)-\frac{1}{p}\langle I'_\varepsilon (u_n), v_n\rangle \\&=\left( \frac{1}{2}-\frac{1}{p}\right) \Vert u_n\Vert ^2_\varepsilon +\left( \frac{1}{p}-\frac{1}{2(6-\mu )}\right) F(u). \end{aligned} \end{aligned}$$

It follows that

$$\begin{aligned} \left( \frac{1}{2}-\frac{1}{p}\right) \Vert u_n\Vert ^2_\varepsilon \le c+o_n(1)+o_n(1)\Vert u_n\Vert _\varepsilon . \end{aligned}$$

Then $\left\{ u_n\right\}$ is bounded in E, and the second result holds. $\square$

Lemma 2.11

If u is a critical point of $I_\varepsilon$ on ${\mathcal {N}}_\varepsilon$, then u is a critical point of $I_\varepsilon$ in E.

Proof

Since u is a critical point of $I_\varepsilon$ on ${\mathcal {N}}_\varepsilon$, there exists $\theta \in {\mathbb {R}}$ such that

$$\begin{aligned} I'_\varepsilon (u)=\theta J'_\varepsilon (u), \end{aligned}$$

where $J_\varepsilon (u)=\langle I'_\varepsilon (u), u\rangle$.

It follows from $u\in {\mathcal {N}}_\varepsilon$ that

$$\begin{aligned} \begin{aligned} \langle J'_\varepsilon (u), u\rangle&= 2\Vert u\Vert ^2_\varepsilon -p\int _{{\mathbb {R}}^3}P(\varepsilon x)|u|^{p}\mathrm{{d}}x-2(6-\mu )F(u)\\&=(2-p)\Vert u\Vert ^2_\varepsilon +(p-2(6-\mu ))F(u)<0. \end{aligned} \end{aligned}$$

Then, by $0=\langle I'_\varepsilon (u), u\rangle =\theta \langle J'_\varepsilon (u), u\rangle$, we have $I'_\varepsilon (u)=0$. $\square$

3 The Energy Functional with Constant Coefficients

We need some results about Eq. (2.2) with constant coefficients. Consider the following problem

$$\begin{aligned} -\Delta u+ ku=\tau |u|^{p-2}u+\nu ^2\Big (\int _{{\mathbb {R}}^3}\frac{|u(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}y\Big )|u|^{4-\mu }u, \quad x\in {\mathbb {R}}^3, \end{aligned}$$

(3.1)

where k, $\tau$, and $\nu$ are positive constants. The associated energy functional is

$$\begin{aligned} I_{k\tau \nu }(u)=\frac{1}{2}\Vert u\Vert ^2_k-\frac{\tau }{p}\int _{{\mathbb {R}}^3}|u|^{p}\mathrm{{d}}x-\frac{\nu ^2}{2(6-\mu )}{\widetilde{F}}(u), \end{aligned}$$

where

$$\begin{aligned} \Vert u\Vert _k= & {} \Big (\int _{{\mathbb {R}}^3}(|\nabla u|^2+ku^2)\mathrm{{d}}x\Big )^{1/2}, \\ {\widetilde{F}}(u)= & {} \int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{|u(x)|^{6-\mu }|u(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y. \end{aligned}$$

By Lemma 2.7, we have

$$\begin{aligned}m_{k\tau \nu }:=\inf _{u\in {\mathcal {N}}_{k\tau \nu }}I_{k\tau \nu }(u)=\inf _{u\in E\setminus \{0\}}\max _{t\ge 0}I_{k\tau \nu }(tu), \end{aligned}$$

where ${\mathcal {N}}_{k\tau \nu }=\left\{ u\in E\setminus \left\{ 0\right\} | \langle I_{k\tau \nu }'(u), u\rangle =0\right\}$. Especially, $I_\infty (u)$, $m_\infty$, and ${\mathcal {N}}_\infty$ mean $I_{V_\infty P^\infty Q^\infty }(u)$, $m_{V_\infty P^\infty Q^\infty }$, and ${\mathcal {N}}_{V_\infty P^\infty Q^\infty }$, respectively.

Lemma 3.1

Problem (3.1) has at least one ground state solution.

Proof

By Lemma 2.7 and Lemma 2.8, there exits a sequence $\left\{ u_n\right\}$ which is a $(PS)_{m_{k\tau \nu }}$ sequence of $I_{k\tau \nu }$. By Lemma 2.10, we know that $\left\{ u_n\right\}$ is bounded in E. Hence, up to a subsequence, we have

$$\begin{aligned} \begin{aligned}&u_n\rightharpoonup u\quad \hbox {in}\; E,\\&u_n\rightarrow u\quad \hbox {a.e. in}\ {\mathbb {R}}^3,\\&u_n\rightharpoonup u \quad \hbox {in}\ L^{q}({\mathbb {R}}^3), \ \hbox {for }\ 2\le q \le 6. \end{aligned} \end{aligned}$$

It is easy to verify that $I_{k\tau \nu }'(u)=0$ .

Case1. $u\ne 0$.

For this case, we have $u\in {\mathcal {N}}_{k\tau \nu }$. Therefore, $I_{k\tau \nu }(u)\ge m_{k\tau \nu }$. Then we get

$$\begin{aligned} \begin{aligned} m_{k\tau \nu }=\lim _{n\rightarrow \infty }I_{k\tau \nu }(u_n)&=\lim _{n\rightarrow \infty }\Big [I_{k\tau \nu }(u_n)-\frac{1}{p}\langle I_{k\tau \nu }'(u_n), u_n\rangle \Big ]\\&=\lim _{n\rightarrow \infty }\Big [\left( \frac{1}{2}-\frac{1}{p}\right) \Vert u_n\Vert ^2_k+\left( \frac{1}{p}-\frac{1}{2(6-\mu )}\right) \nu ^2{\widetilde{F}}(u_n)\Big ]\\&\ge \left( \frac{1}{2}-\frac{1}{p}\right) \Vert u\Vert ^2_k+\left( \frac{1}{p}-\frac{1}{2(6-\mu )}\right) \nu ^2{\widetilde{F}}(u)\\&=I_{k\tau \nu }(u)-\frac{1}{p}\langle I_{k\tau \nu }'(u), u\rangle \\&=I_{k\tau \nu }(u)\ge m_{k\tau \nu }. \end{aligned} \end{aligned}$$

Thus, $I_{k\tau \nu }(u)= m_{k\tau \nu }$. Moreover, we have $u_n\rightarrow u$ in E.

Case2. $u=0$.

Since $\{u_n\}$ is a $(PS)_{m_{k\tau \nu }}$ sequence of $I_{k\tau \nu }$, we have

$$\begin{aligned} o_n(1)=\langle I'_{k\tau \nu }(u_n), u_n\rangle =\Vert u_n\Vert ^2_k-\tau \int _{{\mathbb {R}}^3}|u_n|^p\mathrm{{d}}x-\nu ^2{\widetilde{F}}(u_n). \end{aligned}$$

Assume that

$$\begin{aligned} \Vert u_n\Vert ^2_k\rightarrow l \ \ \text {and}\ \ \tau \int _{{\mathbb {R}}^3}|u_n|^p\mathrm{{d}}x+\nu ^2{\widetilde{F}}(u_n)\rightarrow l. \end{aligned}$$

It is easy to see that $l\ne 0$. If $\int _{{\mathbb {R}}^3}|u_n|^p\mathrm{{d}}x\rightarrow 0$, then $\nu ^2{\widetilde{F}}(u_n)\rightarrow l$. By the definition of $S_{H,L}$, we can get

$$\begin{aligned} \nu ^2{\widetilde{F}}(u_n)\le \nu ^2S_{H,L}^{-(6-\mu )}\Vert u_n\Vert ^{2(6-\mu )}_k. \end{aligned}$$

Letting $n\rightarrow \infty$, we have

$$\begin{aligned} l\le \nu ^2S_{H,L}^{-(6-\mu )}l^{6-\mu }. \end{aligned}$$

(3.2)

Then,

$$\begin{aligned} l\ge \nu ^{\frac{-2}{5-\mu }}S_{H,L}^{\frac{6-\mu }{5-\mu }}. \end{aligned}$$

Thus,

$$\begin{aligned} \begin{aligned} \frac{5-\mu }{2(6-\mu )} S^{\frac{6-\mu }{5-\mu }}_{H,L}\nu ^{\frac{-2}{5-\mu }}>m_{k\tau \nu }&=\lim _{n\rightarrow \infty }I_{k\tau \nu }(u_n)\\&=\lim _{n\rightarrow \infty }\Big [\frac{1}{2}\Vert u_n\Vert ^2_k-\frac{\tau }{p}\int _{{\mathbb {R}}^3}|u_n|^p\mathrm{{d}}x-\frac{\nu ^2}{2(6-\mu )}{\widetilde{F}}(u_n)\Big ]\\&=\frac{1}{2}l-\frac{1}{2(6-\mu )}l\\&\ge \frac{5-\mu }{2(6-\mu )} S^{\frac{6-\mu }{5-\mu }}_{H,L}\nu ^{\frac{-2}{5-\mu }}, \end{aligned} \end{aligned}$$

which is a contradiction. Therefore, $\int _{{\mathbb {R}}^3}|u_n|^p\mathrm{{d}}x\rightarrow b>0$ as $n\rightarrow \infty .$ Thus, by Lions’s Lemma, there exists $\{y_n\}\subset {\mathbb {R}}^3$, $\rho ,\eta >0$ such that

$$\begin{aligned} \int _{B_\rho (y_n)}|u_n|^2\mathrm{{d}}x\ge \eta . \end{aligned}$$

(3.3)

Let ${\widetilde{u}}_n(x)=u_n(x+y_n)$. Then $||{\widetilde{u}}_n||\le C$ in E. This implies that there exists ${\widetilde{u}}\in E$ such that ${\widetilde{u}}_n\rightharpoonup {\widetilde{u}}$ in E and ${\widetilde{u}}_n\rightarrow {\widetilde{u}}$ a.e. in ${\mathbb {R}}^3$. By (3.3), we get ${\widetilde{u}}\ne 0$. It is easy to prove that

$$\begin{aligned} I_{k\tau \nu }({\widetilde{u}}_n)\rightarrow m_{k\tau \nu }\ \hbox {and} \ I_{k\tau \nu }'({\widetilde{u}}_n)\rightarrow 0. \end{aligned}$$

Thus, we have $I_{k\tau \nu }'({\widetilde{u}})=0$ and ${\widetilde{u}}\in {\mathcal {N}}_{k\tau \nu }$. Then the proof follows from the argument used in the case of $u\ne 0.$ $\square$

Lemma 3.2

For $k_i>0,$ $\tau _i>0$ and $\nu _i>0,$ $i=1,2$. If

$$\begin{aligned} \min \left\{ k_2-k_1,\tau _1-\tau _2,\nu _1-\nu _2\right\} \ge 0, \end{aligned}$$

then $m_{k_1\tau _1\nu _1}\le m_{k_2\tau _2\nu _2}$. Additionally, if $\max \left\{ k_2-k_1,\tau _1-\tau _2,\nu _1-\nu _2\right\} > 0,$ then $m_{k_1\tau _1\nu _1}< m_{k_2\tau _2\nu _2}$.

Proof

By Lemma 3.1, there exists $u\in E$ satisfying $I_{k_2\tau _2\nu _2}(v)= m_{k_2\tau _2\nu _2}=\displaystyle \max _{t\ge 0}I_{k_2\tau _2\nu _2}(tu)$. By Lemma 2.6, there exists $t_0>0$ such that $I_{k_1\tau _1\nu _1}(t_0u)=\displaystyle \max _{t\ge 0}I_{k_1\tau _1\nu _1}(tu)$. Then

$$\begin{aligned} m_{k_1\tau _1\nu _1}\le \displaystyle \max _{t\ge 0}I_{k_1\tau _1\nu _1}(tv)=I_{k_1\tau _1\nu _1} (t_0v)\le I_{k_2\tau _2\nu _2}(t_0v)\le I_{k_2\tau _2\nu _2}(v)=m_{k_2\tau _2\nu _2}. \end{aligned}$$

$\square$

Lemma 3.3

For any $\xi \in {\mathbb {R}}^3$, $\displaystyle \limsup _{\varepsilon \rightarrow 0}c_\varepsilon \le m_{V(\xi )P(\xi )Q(\xi )}$.

Proof

For any $\xi \in {\mathbb {R}}^3$, by Lemma 3.1, we assume that u is a ground state solution to the equation corresponding to the functional $I_{V(\xi )P(\xi )Q(\xi )}$. Set $u_\varepsilon (x)=\varphi (\varepsilon x-\xi ) u(x-\frac{\xi }{\varepsilon })$, where $\varphi \in C_0^\infty ({\mathbb {R}}^3,[0,1])$ is a cut-off function satisfying $\varphi =1$, $|x|<1$ and $\varphi =0$, $|x|\ge 2$. Then, there exists T large enough, such that $I_\varepsilon (Tu_\varepsilon )<0$. Define $\gamma _\varepsilon (t)=tTu_\varepsilon$, $t\in [0,1]$. It is easy to see that $\gamma _\varepsilon (t)\in \Gamma _\varepsilon$ in Lemma 2.7. By direct computation, we have

$$\begin{aligned} \int _{{\mathbb {R}}^3}(|\nabla u_\varepsilon |^2+V(\varepsilon x)|u_\varepsilon |^2)\mathrm{{d}}x= & {} \int _{{\mathbb {R}}^3}(|\nabla u|^2+V(\xi )|u|^2)\mathrm{{d}}x+o_\varepsilon (1), \\ \int _{{\mathbb {R}}^3}P(\varepsilon x)|u_\varepsilon |^p\mathrm{{d}}x= & {} \int _{{\mathbb {R}}^3}P(\xi )|u|^p\mathrm{{d}}x+o_\varepsilon (1) \\ F(u_\varepsilon )= & {} Q^2(\xi ){\widetilde{F}}(u)+o_\varepsilon (1), \end{aligned}$$

Therefore,

$$\begin{aligned} \begin{aligned} I_\varepsilon (\gamma _\varepsilon (t))&=\frac{(tT)^2}{2}\Vert u_\varepsilon \Vert ^2_\varepsilon -\frac{(tT)^{p}}{p}\int _{{\mathbb {R}}^3}P(\varepsilon x)|u_\varepsilon |^p\mathrm{{d}}x-\frac{(tT)^{2(6-\mu )}}{2(6-\mu )}F(u_\varepsilon )\\&=\frac{(tT)^2}{2}\Vert u\Vert ^2_{V(\xi )}-\frac{(tT)^{p}}{p}\int _{{\mathbb {R}}^3}P(\xi )|u|^p\mathrm{{d}}x-\frac{(tT)^{2(6-\mu )}}{2(6-\mu )}Q^2(\xi ){\widetilde{F}}(u)+o_\varepsilon (1)\\&=I_{V(\xi )P(\xi )Q(\xi )}(tTu)+o_\varepsilon (1)\\&\le I_{V(\xi )P(\xi )Q(\xi )}(u)+o_\varepsilon (1)=m_{V(\xi )P(\xi )Q(\xi )}+o_\varepsilon (1) \end{aligned} \end{aligned}$$

Thus,

$$\begin{aligned} c_\varepsilon \le \max _{0\le t\le 1}I_\varepsilon (\gamma _\varepsilon (t)) \le m_{V(\xi )P(\xi )Q(\xi )}+o_\varepsilon (1). \end{aligned}$$

It follows that $\displaystyle \limsup _{\varepsilon \rightarrow 0}c_\varepsilon \le m_{V(\xi )P(\xi )Q(\xi )}$. $\square$

4 The Palais-Smale Condition

Lemma 4.1

Suppose that the condition $(H_2)$ holds. Let $\{u_n\}\subset E$ be a $(PS)_c$ sequence for $I_\varepsilon$ with $c<\displaystyle \frac{5-\mu }{2(6-\mu )} S^{\frac{6-\mu }{5-\mu }}_{H,L}(Q^\infty )^{\frac{-2}{5-\mu }}$ and such that $u_n\rightharpoonup 0$ in E. Then, one of the following conclusions holds.

• (i) $u_n\rightarrow 0$ in E;

• (ii) There exists a sequence $\{y_n\}\subset {\mathbb {R}}^3$ and constants $R,\beta >0$ such that

  $$\begin{aligned}\liminf _{n\rightarrow \infty }\int _{B_R(y_n)}|u_n|^2\mathrm{{d}}x\ge \beta .\end{aligned}$$

Proof

Suppose that (ii) does not occur. Then, for any $R>0$, one has

$$\begin{aligned} \lim _{n\rightarrow \infty }\sup _{y\in {\mathbb {R}}^3}\int _{B_R(y)}|u_n|^2\mathrm{{d}}x=0. \end{aligned}$$

Then, we have

$$\begin{aligned}u_n\rightarrow u \quad \hbox {in}\ L^{q}({\mathbb {R}}^3), \ \hbox {for }\ 2<q< 6. \end{aligned}$$

Noting $o_n(1)=\langle I'_\varepsilon (u_n),u_n\rangle$, we can obtain

$$\begin{aligned} \Vert u_n\Vert ^2_\varepsilon =F(u_n)+o_n(1). \end{aligned}$$

By Lemma 2.10, $\{u_n\}$ is bounded in E. Up to a subsequence, we can assume that

$$\begin{aligned} \Vert u_n\Vert ^2_\varepsilon \rightarrow l \ \hbox {and} \ F(u_n)\rightarrow l. \end{aligned}$$

Assume by contradiction that $l>0$. From condition $(H_2)$,

$$\begin{aligned} F(u_n)\le (Q^{\infty })^2 {\widetilde{F}}(u_n). \end{aligned}$$

By the definition of $S_{H,L}$, we can get

$$\begin{aligned} F(u_n)\le (Q^{\infty })^2 S_{H,L}^{-(6-\mu )}\Vert u_n\Vert _\varepsilon ^{2(6-\mu )}. \end{aligned}$$

It follows that

$$\begin{aligned} l\ge S^{\frac{6-\mu }{5-\mu }}_{H,L}(Q^\infty )^{\frac{-2}{5-\mu }}. \end{aligned}$$

Since $I_\varepsilon (u_n)=c+o_n(1)$, we can deduce that

$$\begin{aligned} c\ge \frac{5-\mu }{2(6-\mu )}S^{\frac{6-\mu }{5-\mu }}_{H,L}(Q^\infty )^{\frac{-2}{5-\mu }}, \end{aligned}$$

which is a contradiction with our assumption. Therefore, $l=0$ and the conclusion follows. $\square$

Lemma 4.2

Suppose that the condition $(H_2)$ holds. Let $\{u_n\}\subset E$ be a $(PS)_c$ sequence for $I_\varepsilon$ with $c<m_\infty$ and $u_n\rightharpoonup 0$ in E. Then $u_n\rightarrow 0$ in E.

Proof

Assume that $u_n\nrightarrow 0$ in E. Let $\{t_n\}\subset (0,+\infty )$ be a sequence such that $\{t_nu_n\}\subset {\mathcal {N}}_\infty$. Then, we claim that the sequence $\{t_n\}$ satisfies that $\displaystyle \limsup _{n\rightarrow \infty }t_n\le 1$.

Assume by contradiction that there exists $\delta >0$ and a subsequence still denoted by $\{t_n\}$, such that, for all $n\in \mathbb {N}$,

$$\begin{aligned} t_n\ge 1+\delta . \end{aligned}$$

Since $\langle I'_\varepsilon (u_n),u_n\rangle =o_n(1)$, we get

$$\begin{aligned} \Vert u_n\Vert ^2_\varepsilon =\int _{{\mathbb {R}}^3}P(\varepsilon x)|u_n|^{p}\mathrm{{d}}x+F(u_n)+o_n(1). \end{aligned}$$

(4.1)

Using $t_nu_n\in {\mathcal {N}}_\infty$, we have

$$\begin{aligned} t_n^2\Vert u_n\Vert ^2_\infty =t_n^{p}P^\infty \int _{{\mathbb {R}}^3}|u_n|^{p}\mathrm{{d}}x+t_n^{2(6-\mu )}(Q^\infty )^2{\widetilde{F}}(u_n). \end{aligned}$$

Then, we can obtain

$$\begin{aligned} \begin{aligned}&\left( \frac{1}{t_n^{p-2}}-1\right) \int _{{\mathbb {R}}^3}|\nabla u_n|^2\mathrm{{d}}x+\int _{{\mathbb {R}}^3}\left( \frac{V_\infty }{t_n^{p-2}}-V(\varepsilon x)\right) |u_n|^2\mathrm{{d}}x\\&\quad =\int _{{\mathbb {R}}^3}[P^\infty -P(\varepsilon x)]|u_n|^{p}\mathrm{{d}}x\\&\qquad +\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}[t_n^{(12-2\mu -p)}(Q^\infty )^2-Q(\varepsilon x)Q(\varepsilon y)]\frac{|u_n(x)|^{6-\mu }|u_n(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y+o_n(1). \end{aligned} \end{aligned}$$

By the definition of $V_\infty$ and $P^\infty$, for any $\sigma >0$, there exists $R=R(\sigma )>0$, such that, for $|\varepsilon x|\ge R$,

$$\begin{aligned} V(\varepsilon x)> V_\infty -\sigma >\frac{V_\infty }{t_n^{p-2}}-\sigma \end{aligned}$$

(4.2)

and

$$\begin{aligned} P(\varepsilon x)< P^\infty +\sigma . \end{aligned}$$

(4.3)

Moreover, $\Vert u_n\Vert _\varepsilon$ is bounded and $u_n\rightarrow 0\ \hbox {in}\ L^{q}_{loc}({\mathbb {R}}^3), \ \hbox {for }\ 2\le q < 6$. Then, we can obtain

$$\begin{aligned} \int _{{\mathbb {R}}^3}\left( \frac{V_\infty }{t_n^{p-2}}-V(\varepsilon x)\right) |u_n|^2\mathrm{{d}}x\le C\sigma +o_n(1) \end{aligned}$$

and

$$\begin{aligned} \int _{{\mathbb {R}}^3}[P^\infty -P(\varepsilon x)]|u_n|^{p}\mathrm{{d}}x\ge -C\sigma +o_n(1). \end{aligned}$$

Therefore,

$$\begin{aligned} \int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}[t_n^{2(6-\mu )-p}(Q^\infty )^2-Q(\varepsilon x)Q(\varepsilon y)]\frac{|u_n(x)|^{6-\mu }|u_n(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y\le C\sigma +o_n(1). \end{aligned}$$

Since $t_n>1+\delta$ and $Q(\varepsilon x)\le Q^\infty$, it follows from the above inequality that

$$\begin{aligned} \int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{|u_n(x)|^{6-\mu }|u_n(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y\le C\sigma +o_n(1). \end{aligned}$$

By the arbitrariness of $\sigma$, we can obtain

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{|u_n(x)|^{6-\mu }|u_n(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y=0. \end{aligned}$$

(4.4)

Since $u_n\nrightarrow 0$ in E, by Lemma 4.1, we know that there exists a sequence $\{y_n\}\subset {\mathbb {R}}^3$ and constants $R,\beta >0$ such that

$$\begin{aligned} \int _{B_R(y_n)}|u_n|^2\mathrm{{d}}x\ge \beta . \end{aligned}$$

(4.5)

Set $v_n(x)=u_n(x+y_n)$. Then $\{v_n(x)\}$ is a bounded sequence in E. Therefore, there exists $v\in E$ such that

$$\begin{aligned}v_n\rightarrow v, \ \text {a.e. in}\ {\mathbb {R}}^3.\end{aligned}$$

By (4.5), $v\ne 0$ in E. Then, it follows from Fatou Lemma and (4.4) that

$$\begin{aligned} \begin{aligned} 0<\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{|v(x)|^{6-\mu }|v(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y&\le \lim _{n\rightarrow \infty }\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{|v_n(x)|^{6-\mu }|v_n(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y\\&=\lim _{n\rightarrow \infty }\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{|u_n(x)|^{6-\mu }|u_n(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y=0, \end{aligned} \end{aligned}$$

which is a contradiction.

We next distinguish the following two cases.

Case 1: $\displaystyle \limsup _{n\rightarrow \infty }t_n=1$.

In this case, there exists a subsequence, still denoted by $\{t_n\}$ such that $t_n\rightarrow 1$ as $n\rightarrow \infty$. Then,

$$\begin{aligned} \begin{aligned}&I_\infty (t_nu_n)\\&\quad =\frac{t_n^2}{2}\int _{{\mathbb {R}}^3}(|\nabla u_n|^2+V_\infty |u_n|^2)\mathrm{{d}}x-\frac{t_n^p}{p}\int _{{\mathbb {R}}^3}P^\infty |u_n|^{p}\mathrm{{d}}x-\frac{t_n^{2(6-\mu )}}{2(6-\mu )}(Q^\infty )^2{\widetilde{F}}(u_n)\\&\quad =\frac{1}{2}\int _{{\mathbb {R}}^3}(|\nabla u_n|^2+V_\infty |u_n|^2)\mathrm{{d}}x-\frac{1}{p}\int _{{\mathbb {R}}^3}P^\infty |u_n|^{p}\mathrm{{d}}x-\frac{1}{2(6-\mu )}(Q^\infty )^2{\widetilde{F}}(u_n)+o_n(1). \end{aligned} \end{aligned}$$

Therefore,

$$\begin{aligned} \begin{aligned}&I_\varepsilon (u_n)-I_\infty (t_nu_n)=\frac{1}{2}\int _{{\mathbb {R}}^3}(V(\varepsilon x)-V_\infty )|u_n|^2\mathrm{{d}}x+\frac{1}{p}\int _{{\mathbb {R}}^3}(P^\infty -P(\varepsilon x))|u_n|^{p}\mathrm{{d}}x\\&\quad +\frac{1}{2(6-\mu )}\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}[(Q^\infty )^2-Q(\varepsilon x)Q(\varepsilon y)]\frac{|u_n(x)|^{6-\mu }|u_n(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y\\&\quad +o_n(1). \end{aligned} \end{aligned}$$

By (4.2) and $u_n\rightarrow 0\ \hbox {in}\ L^{2}_{loc}({\mathbb {R}}^3)$,

$$\begin{aligned} \int _{{\mathbb {R}}^3}(V(\varepsilon x)-V_\infty )|u_n|^2\mathrm{{d}}x\ge -C\sigma +o_n(1). \end{aligned}$$

Similarly,

$$\begin{aligned} \int _{{\mathbb {R}}^3}(P^\infty -P(\varepsilon x))|u_n|^p\mathrm{{d}}x\ge -C\sigma +o_n(1). \end{aligned}$$

Then, noting $Q(\varepsilon x)\le Q^\infty$, we can obtain

$$\begin{aligned} c\ge m_\infty -C\sigma +o_n(1). \end{aligned}$$

By the arbitrariness of $\sigma$, we have $c\ge m_\infty$, which is a contradiction.

Case 2: $\displaystyle \limsup _{n\rightarrow \infty }t_n<1$.

In this case, we may suppose that $t_n<1$ for all $n\in \mathbb {N}$. From (4.1), we can deduce that

$$\begin{aligned} \begin{aligned} I_\varepsilon (t_nu_n)=&\frac{t_n^2}{2}\Vert u_n\Vert ^2_\varepsilon -\frac{t_n^p}{p}\int _{{\mathbb {R}}^3}P(\varepsilon x)|u_n|^{p}\mathrm{{d}}x-\frac{t_n^{2(6-\mu )}}{2(6-\mu )}F(u_n)\\ =&(\frac{t_n^2}{2}-\frac{t_n^{2(6-\mu )}}{2(6-\mu )})\Vert u_n\Vert ^2_\varepsilon +(\frac{t_n^{2(6-\mu )}}{2(6-\mu )}-\frac{t_n^p}{p})\int _{{\mathbb {R}}^3}P(\varepsilon x)|u_n|^{p}\mathrm{{d}}x+o_n(1)\\ \le&(\frac{1}{2}-\frac{1}{2(6-\mu )})\Vert u_n\Vert ^2_\varepsilon +(\frac{1}{2(6-\mu )}-\frac{1}{p})\int _{{\mathbb {R}}^3}P(\varepsilon x)|u_n|^{p}\mathrm{{d}}x+o_n(1)\\ =&I_\varepsilon (u_n)-\frac{1}{2(6-\mu )}\langle I'_\varepsilon (u_n),u_n\rangle +o_n(1)\\ =&I_\varepsilon (u_n)+o_n(1). \end{aligned} \end{aligned}$$

Using this result, we have

$$\begin{aligned} \begin{aligned} m_\infty&\le I_\infty (t_nu_n)\\&=I_\varepsilon (t_nu_n)+\frac{t_n^2}{2}\int _{{\mathbb {R}}^3}(V_\infty -V(\varepsilon x))|u_n|^2\mathrm{{d}}x\\&\quad -\frac{t_n^p}{p}\int _{{\mathbb {R}}^3}(P^\infty -P(\varepsilon x))|u_n|^{p}\mathrm{{d}}x-\frac{t_n^{2(6-\mu )}}{2(6-\mu )}\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}[(Q^\infty )^2-Q(\varepsilon x)Q(\varepsilon y)]\\&\quad \frac{|u_n(x)|^{6-\mu }|u_n(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y\\&\le I_\varepsilon (t_nu_n)+\frac{t_n^2}{2}\int _{{\mathbb {R}}^3}(V_\infty -V(\varepsilon x))|u_n|^2\mathrm{{d}}x -\frac{t_n^p}{p}\int _{{\mathbb {R}}^3}(P^\infty -P(\varepsilon x))|u_n|^{p}\mathrm{{d}}x\\&\le I_\varepsilon (t_nu_n)+C\sigma +o_n(1)\\&\le I_\varepsilon (u_n)+C\sigma +o_n(1), \end{aligned} \end{aligned}$$

which means that $c\ge m_\infty$, a contradiction. $\square$

Lemma 4.3

Suppose that the condition $(H_2)$ holds. Then $I_\varepsilon$ satisfies the $(PS)_c$ condition at any level $c<m_\infty$.

Proof

Let $\{u_n\}$ be a $(PS)_c$ sequence. By Lemma 2.10, $\{u_n\}$ is bounded in E. Then there exists $u\in E$ such that $u_n\rightharpoonup u$ in E. By standard argument, $I'_\varepsilon (u)=0$ and $I_\varepsilon (u)\ge 0$. Set $w_n=u_n-u$. It follows from Lemma 2.4 and Brezis–Lieb’ Lemma that $\{w_n\}$ is a $(PS)_{c-I_\varepsilon (u)}$ sequence. Since $c-I_\varepsilon (u)<m_\infty$, by Lemma 4.2, $w_n\rightarrow 0$ in E. Therefore, $u_n\rightarrow u$ in E. $\square$

Lemma 4.4

Suppose that the condition $(H_2)$ holds. Let $\{u_n\}$ be a $(PS)_c$ sequence restricted on ${\mathcal {N}}_\varepsilon$ and assume $c<m_\infty$. Then $\{u_n\}$ has a convergent subsequence in E.

Proof

Let $\{u_n\}$ be a $(PS)_c$ sequence restricted on ${\mathcal {N}}_\varepsilon$. Then, there exist $\{\theta _n\}\subset {\mathbb {R}}$ such that

$$\begin{aligned} I'_\varepsilon (u_n)=\theta _n J'_\varepsilon (u_n)+o_n(1) \end{aligned}$$

where $J_\varepsilon (u)=\langle I'_\varepsilon (u), u\rangle$.

It follows from $u_n\in {\mathcal {N}}_\varepsilon$ and Lemma 2.5 that

$$\begin{aligned} \begin{aligned} \langle J'_\varepsilon (u_n), u_n\rangle&= 2\Vert u_n\Vert ^2_\varepsilon -p\int _{{\mathbb {R}}^3} P(\varepsilon x)|u_n|^{p}\mathrm{{d}}x-2(6-\mu )F(u_n)\\&=(2-p)\Vert u_n\Vert ^2_\varepsilon +(p-2(6-\mu ))F(u_n)\\&<(2-p)\Vert u_n\Vert ^2_\varepsilon \le (2-p)C_0^2. \end{aligned} \end{aligned}$$

From $0=\langle I'_\varepsilon (u_n), u_n\rangle$ and the above inequality, we have $\theta _n=o_n(1)$. Therefore, $I'_\varepsilon (u_n)=o_n(1)$. Thus, by Lemma 4.3, the proof is completed. $\square$

5 The Existence of Multiple Solutions

We assume that the conditions $(H_1)$, $(H_2)$, and $(H_3)$ hold in this section. Let us consider a cut-off function $\eta \in C_0^\infty ({\mathbb {R}}^3,[0,1])$ such that $\eta (x)=1$ if $|x|<1,$ $\eta (x)=0$ if $|x|>2$ and $|\nabla \eta |\le C$. Choose $w\in E$ with $I^\prime _{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}(w)=0$ and $I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}(w)=m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}$. For each $\xi \in {\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}}$, let

$$\begin{aligned} \Psi _{\varepsilon ,\xi }(x)=\eta (|\varepsilon x-\xi |)w\big (\frac{\varepsilon x-\xi }{\varepsilon }\big ). \end{aligned}$$

Then, there exists a unique $t_\varepsilon >0$ such that $t_\varepsilon \Psi _{\varepsilon ,y}\in {\mathcal {N}}_\varepsilon$. Define $\Phi _\varepsilon : {\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}}\rightarrow {\mathcal {N}}_\varepsilon$ by setting $\Phi _\varepsilon (\xi )=t_\varepsilon \Psi _{\varepsilon ,\xi }$.

Lemma 5.1

$\lim \limits _{\varepsilon \rightarrow 0}I_\varepsilon (\Phi _\varepsilon (\xi ))=m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}~~\hbox {uniformly~in}~\xi \in {\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}}.$

Proof

Suppose that the result is false. Then, there exists some $\alpha >0$, $\{\xi _n\}\subset {\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}}$ and $\varepsilon _n\rightarrow 0$ such that

$$\begin{aligned} |I_{\varepsilon _n}(\Phi _{\varepsilon _n}(\xi _n))-m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}|\ge \alpha . \end{aligned}$$

The compactness of ${\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}}$ implies that there exists $\xi \in {\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}}$ such that $\xi _n\rightarrow \xi$, up to a subsequence if necessary. Now we claim that $\displaystyle \lim _{n\rightarrow \infty }t_{\varepsilon _n}=1$. Indeed, from $t_{\varepsilon _n}\Psi _{\varepsilon _n,\xi _n}\in {\mathcal {N}}_{\varepsilon _n}$, we have

$$\begin{aligned} t^2_{\varepsilon _n}\Vert \Psi _{\varepsilon _n,\xi _n}\Vert ^2_{\varepsilon _n}=t^p_{\varepsilon _n}\int _{{\mathbb {R}}^3}P(\varepsilon _n x)|\Psi _{\varepsilon _n,\xi _n}|^{p}\mathrm{{d}}x+t^{2(6-\mu )}_{\varepsilon _n}F(\Psi _{\varepsilon _n,\xi _n}). \end{aligned}$$

By using a change of variables and Lebesgue Dominated Convergence Theorem, we can obtain

$$\begin{aligned} \Vert \Psi _{\varepsilon _n,\xi _n}\Vert ^2_{\varepsilon _n}= & {} \int _{{\mathbb {R}}^3}(|\nabla w|^2+V(\xi )w^2)\mathrm{{d}}x+o_n(1), \\ \int _{{\mathbb {R}}^3}P(\varepsilon _n x)|\Psi _{\varepsilon _n,\xi _n}|^{p}\mathrm{{d}}x= & {} \int _{{\mathbb {R}}^3}P(\xi )|w|^p\mathrm{{d}}x+o_n(1) \end{aligned}$$

and

$$\begin{aligned} F(\Psi _{\varepsilon _n,\xi _n})=Q^2(\xi ){\widetilde{F}}(w)+o_n(1). \end{aligned}$$

Then $t_n$ is bounded from above. Thus we can obtain

$$\begin{aligned} t^2_{\varepsilon _n}\int _{{\mathbb {R}}^3}(|\nabla w|^2+V(\xi )w^2)\mathrm{{d}}x=t^p_{\varepsilon _n}\int _{{\mathbb {R}}^3}P(\xi )|w|^{p}\mathrm{{d}}x+t^{2(6-\mu )}_{\varepsilon _n}Q^2(\xi ){\widetilde{F}}(w)+o_n(1). \end{aligned}$$

It follows from Lemma 2.5 that $t_n$ is has a positive lower bound. Without loss of generality, we assume that $t_{\varepsilon _n}\rightarrow T>0$. Letting $n\rightarrow \infty$ in the above expression, we can get

$$\begin{aligned} T^2\int _{{\mathbb {R}}^3}(|\nabla w|^2+V(\xi )w^2)\mathrm{{d}}x=T^p\int _{{\mathbb {R}}^3}P(\xi )|w|^{p}\mathrm{{d}}x+T^{2(6-\mu )}Q^2(\xi ){\widetilde{F}}(w). \end{aligned}$$

It follows from $w\in {\mathcal {N}}_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}$ that $T=1$. Then, we have

$$\begin{aligned} \lim _{n\rightarrow \infty }I_{\varepsilon _n}(\Phi _{\varepsilon _n}(\xi _n))=I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}(w)=m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}, \end{aligned}$$

which is a contradiction. $\square$

For any $\delta >0,$ let $\rho =\rho (\delta )>0$ such that $({\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}})_\delta \subset B_\rho (0)$. Consider $\chi :{\mathbb {R}}^3\rightarrow {\mathbb {R}}^3$ defined as $\chi (x)=x$ for $|x|\le \rho$ and $\chi (x)=\frac{\rho x}{|x|}$ for $|x|\ge \rho$. Define $\beta _\varepsilon : {\mathcal {N}}_\varepsilon \rightarrow {\mathbb {R}}^3$ given by

$$\begin{aligned} \beta _\varepsilon (u)=\frac{\int _{{\mathbb {R}}^3}\chi (\varepsilon x)u^2(x)\mathrm{{d}}x}{\int _{{\mathbb {R}}^3}u^2(x)\mathrm{{d}}x}. \end{aligned}$$

Lemma 5.2

$\lim \limits _{\varepsilon \rightarrow 0}\beta _{\varepsilon }(\Phi _\varepsilon (\xi ))=\xi ~\text {uniformly}~\text {in}~\xi \in {\mathbb {V}}\cap {\mathbb {P}}\cap {\mathbb {Q}}.$

Proof

Suppose by contradiction that there exist $\delta _0>0,~\{\xi _n\}\subset {\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}}$ and $\varepsilon _n\rightarrow 0$ such that

$$\begin{aligned} |\beta _{\varepsilon _n}(\Phi _{\varepsilon _n}(\xi _n))-\xi _n|\ge \delta _0. \end{aligned}$$

(5.1)

By the definition of $\beta _\varepsilon$, we have

$$\begin{aligned} \beta _{\varepsilon _n}(\Phi _{\varepsilon _n}(\xi _n))=\xi _n+\frac{\int _{{\mathbb {R}}^3}(\chi (\varepsilon _n x+\xi _n)-\xi _n)|\eta (\varepsilon _n x)w(x)|^2\mathrm{{d}}x}{\int _{{\mathbb {R}}^3}|\eta (\varepsilon _n x)w(x)|^2\mathrm{{d}}x}. \end{aligned}$$

Since $\{\xi _n\}\subset {\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}}\subset B_\rho (0)$ and $\chi \big |_{B_\rho }\equiv id$, we conclude that

$$\begin{aligned} |\beta _{\varepsilon _n}(\Phi _{\varepsilon _n}(\xi _n))-\xi _n|=o_n(1), \end{aligned}$$

which contradicts (5.1) and the desired conclusion holds. $\square$

Define the set

$$\begin{aligned} \tilde{{\mathcal {N}}}_\varepsilon =\{u\in {\mathcal {N}}_\varepsilon :I_\varepsilon (u)\le m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}+h(\varepsilon )\}. \end{aligned}$$

where $h(\varepsilon )=\sup \limits _{\xi \in {\mathcal {V}} \cap {\mathcal {P}} \cap {\mathcal {Q}}}|I_\varepsilon (\Phi _\varepsilon (\xi ))-m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}|$. We conclude from Lemma 5.1 that $h(\varepsilon )\rightarrow 0$ as $\varepsilon \rightarrow 0$. By the definition of $h(\varepsilon )$, for any $\xi \in {\mathcal {V}} \cap {\mathcal {P}} \cap {\mathcal {Q}}$ and $\varepsilon >0$, $\Phi _\varepsilon (\xi )\in \tilde{{\mathcal {N}}}_\varepsilon$ and $\tilde{ {\mathcal {N}}}_\varepsilon \ne \emptyset$.

Lemma 5.3

Let $\varepsilon _{n} \rightarrow 0$ and $u_{n} \in {\mathcal {N}}_{\varepsilon _{n}}$ such that $I_{\varepsilon _{n}}\left( u_{n}\right) \rightarrow m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}.$ Then there exists $\left\{ y_{n}\right\} \subset {\mathbb {R}}^{N}$ such that the sequence $u_{n}\left( x+y_{n}\right)$ has a convergent subsequence in E. Moreover, up to a subsequence, $\varepsilon _{n} y_{n} \rightarrow \xi \in {\mathcal {V}} \cap {\mathcal {P}} \cap {\mathcal {Q}}$.

Proof

Since

$$\begin{aligned} \begin{aligned} m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}&=I_{\varepsilon _{n}}\left( u_{n}\right) -\frac{1}{p}\left\langle I_{\varepsilon _{n}}^{\prime }\left( u_{n}\right) , u_{n}\right\rangle +o_n(1)\\&=\left( \frac{1}{2}-\frac{1}{2(6-\mu )}\right) \Vert u_n\Vert ^2_{\varepsilon _{n}}+\left( \frac{1}{p}-\frac{1}{2(6-\mu )}\right) F(u_n)+o_n(1)\\&\ge \left( \frac{1}{2}-\frac{1}{2(6-\mu )}\right) \Vert u_n\Vert ^2_{\varepsilon _{n}}+o_n(1), \end{aligned} \end{aligned}$$

then $\left\{ u_{n}\right\}$ is bounded in E. We can have a sequence $\left\{ y_{n}\right\} \subset {\mathbb {R}}^{3}$ and positive constants $R, \beta$ such that

$$\begin{aligned} \int _{B_{R}\left( y_{n}\right) }\left| u_{n}\right| ^{2} \ge \beta >0. \end{aligned}$$

If not, for any $R>0$, one has

$$\begin{aligned}\lim _{n\rightarrow \infty }\sup _{y\in {\mathbb {R}}^3}\int _{B_R(y)}|u_n|^2\mathrm{{d}}x=0.\end{aligned}$$

Then, we have

$$\begin{aligned}u_n\rightarrow u \quad \hbox {in}\ L^{q}({\mathbb {R}}^3), \ \hbox {for }\ 2<q< 6.\end{aligned}$$

Noting $0=\langle I'_\varepsilon (u_n),u_n\rangle$, we can obtain

$$\begin{aligned} \Vert u_n\Vert ^2_{\varepsilon _n}=F(u_n)+o_n(1). \end{aligned}$$

Up to a subsequence, assume that

$$\begin{aligned} \Vert u_n\Vert ^2_{\varepsilon _n}\rightarrow l \ \hbox {and} \ F(u_n)\rightarrow l. \end{aligned}$$

It follows from Lemma 2.5 that $l>0$. From condition $(H_2)$,

$$\begin{aligned} F(u_n)\le (Q^{\infty })^2 {\widetilde{F}}(u_n). \end{aligned}$$

By the definition of $S_{H,L}$, we can get

$$\begin{aligned} F(u_n)\le (Q^{\infty })^2 S_{H,L}^{-(6-\mu )}\Vert u_n\Vert _\varepsilon ^{2(6-\mu )}. \end{aligned}$$

It follows that

$$\begin{aligned} l\ge S^{\frac{6-\mu }{5-\mu }}_{H,L}(Q^\infty )^{\frac{-2}{5-\mu }}. \end{aligned}$$

Since $I_{\varepsilon _n}(u_n)=m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}+o_n(1)$, we can deduce that

$$\begin{aligned} m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}\ge \frac{5-\mu }{2(6-\mu )}S^{\frac{6-\mu }{5-\mu }}_{H,L}(Q^\infty )^{\frac{-2}{5-\mu }}, \end{aligned}$$

which is a contradiction with Lemma 2.9. Therefore, the conclusion follows. Denote ${\tilde{u}}_{n}(x)=u_{n}\left( x+y_{n}\right)$, going if necessary to a subsequence, we can assume that

$$\begin{aligned} {\tilde{u}}_{n} \rightharpoonup {\tilde{u}} \ne 0 \text{ in } E. \end{aligned}$$

Let $t_{n}>0$ be such that $t_{n} {\tilde{u}}_{n} \in {\mathcal {N}}_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}$. By the definition of $I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}$ and $m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}$, we obtain

$$\begin{aligned} \begin{aligned} m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}&\le I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}\left( t_{n} {\tilde{u}}_{n}\right) \\&=I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}\left( t_{n} u_{n}\right) \\&\le I_{\varepsilon _{n}}\left( t_{n} u_{n}\right) \le I_{\varepsilon _{n}}\left( u_{n}\right) =m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}+o_n(1), \end{aligned} \end{aligned}$$

so $I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}\left( t_{n} {\tilde{u}}_{n}\right) \rightarrow m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}$. Then $\left\{ t_{n} {\tilde{u}}_{n}\right\}$ is bounded in E. Since $t_{n} {\tilde{u}}_{n} \in {\mathcal {N}}_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}$, it follows from Lemma 2.5 that $\Vert t_{n} {\tilde{u}}_{n}\Vert \ge C_0$. Noting $u_n$ is bounded in E, then there exists $C>0$ such that $t_nC\ge \Vert t_{n} {\tilde{u}}_{n}\Vert \ge C_0$. Thus $t_n$ has a positive lower bound. On the other hand, ${\tilde{u}}_{n}$ does not converge to 0 in E, so there exists a $\delta ^{\prime }>0$ such that $\left\| {\tilde{u}}_{n}\right\| \ge \delta ^{\prime }$. Therefore, $t_{n} \delta ^{\prime } \le$ $\left\| t_{n} {\tilde{u}}_{n}\right\| \le C .$ Thus $\left\{ t_{n}\right\}$ is bounded from above. Then, up to a subsequence, $t_{n} \rightarrow t_{0}>0$.

Denote ${\hat{u}}_{n}:=t_{n} {\tilde{u}}_{n}, {\hat{u}}:=t_{0} {\tilde{u}}$, we have

$$\begin{aligned} I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}\left( {\hat{u}}_{n}\right) \rightarrow m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}, {\hat{u}}_{n} \rightharpoonup {\hat{u}} \text{ in } E. \end{aligned}$$

By the Ekeland’s Variational Principle, there exists a sequence $\left\{ {\hat{w}}_{n}\right\} \subset$ ${\mathcal {N}}_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}$ satisfying

$$\begin{aligned} {\hat{w}}_{n}-{\hat{u}}_{n} \rightarrow 0 \text{ in } E, I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}\left( {\hat{w}}_{n}\right) \rightarrow m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}, I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}^{\prime }\left( {\hat{w}}_{n}\right) \rightarrow 0. \end{aligned}$$

Therefore,

$$\begin{aligned} {\hat{w}}_{n} \rightharpoonup {\hat{u}} \text{ in } E \end{aligned}$$

and ${\hat{u}}=t_{0} {\tilde{u}}$ is a nontrivial critical point of $I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}$. Then

$$\begin{aligned}&m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}\\&\quad \le I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}({\hat{u}})-\frac{1}{p}\left\langle I^{\prime }_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}({\hat{u}}), {\hat{u}}\right\rangle \\&\quad =\left( \frac{1}{2}-\frac{1}{p}\right) \int _{{\mathbb {R}}^{N}}(|\nabla {\hat{u}}|^{2}+V_\mathrm{{min}}|{\hat{u}}|^{2})\mathrm{{d}}x +\left( \frac{1}{p}-\frac{1}{2(6-\mu )}\right) Q^2_\mathrm{{max}}{\widetilde{F}}({\hat{u}})\\&\quad \le \liminf _{n \rightarrow \infty }\left( \left( \frac{1}{2}-\frac{1}{p}\right) \int _{{\mathbb {R}}^{N}}\left( |\nabla {\hat{w}}_n|^{2}+V_\mathrm{{min}}|{\hat{w}}_n|^{2}\right) \mathrm{{d}}x+\left( \frac{1}{p}-\frac{1}{2(6-\mu )}\right) Q^2_\mathrm{{max}}{\widetilde{F}}({\hat{w}}_n) \right) \\&\quad =\liminf _{n \rightarrow \infty } \left( I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}({\hat{w}}_{n})-\frac{1}{p}\left\langle I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}^{\prime }\left( {\hat{w}}_{n}\right) , {\hat{w}}_{n}\right\rangle \right) \\&\quad =m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}. \end{aligned}$$

Thus

$$\begin{aligned} {\tilde{u}}_{n} \rightarrow {\tilde{u}} \text{ in } E. \end{aligned}$$

(5.2)

Now, we are going to prove that $\varepsilon _{n} y_{n} \rightarrow \xi \in {\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}}$. We first claim that $\{\varepsilon _ny_n\}$ must be bounded. Otherwise, $|\varepsilon _ny_n|\rightarrow \infty$ as $n\rightarrow \infty$. For any small $\delta >0$, there exists $\rho =\rho (\delta )>0$, such that, for $|x|\ge \rho$,

$$\begin{aligned} V(x)>V_\infty -\delta , P(x)<P^\infty +\delta \ \hbox {and}\ Q(x)<Q^\infty +\delta . \end{aligned}$$

(5.3)

For $u\in E$, define

$$\begin{aligned} \begin{aligned} I_\delta (u)&=\frac{1}{2}\int _{{\mathbb {R}}^3}\left( |\nabla u|^2+(V_\infty -\delta )|u|^2\right) \mathrm{{d}}x\\&\quad -\frac{1}{p}\int _{{\mathbb {R}}^3}(P^\infty +\delta )|u|^p\mathrm{{d}}x-\frac{1}{2(6-\mu )}(Q^\infty +\delta )^2{\widetilde{F}}(u). \end{aligned} \end{aligned}$$

Then, we can introduce

$$\begin{aligned} m_\delta =\inf _{u\in {\mathcal {N}}_\delta }I_\delta (u), \end{aligned}$$

where ${\mathcal {N}}_\delta =\{u\in E: \langle I'_\delta (u), u\rangle =0\}$.

By Lemma 3.2 and condition $(H_1)$, we have $m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}<m_\infty$. Noting the continuity of $m_\delta$ about $\delta$, we can obtain $m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}<m_\delta$ for $\delta$ small. For $u_n$, there exists ${\tilde{t}}_n>0$ such that ${\tilde{t}}_nu_n\in {\mathcal {N}}_\delta$. It is easy to see that $\{{\tilde{t}}_n\}$ is bounded. For any small $\sigma >0$, from (5.2), there exists $R>0$ and N big enough, such that

$$\begin{aligned} \int _{B^c_R(0)}(|\nabla u_n(x+y_n)|^2+u^2_n(x+y_n))\mathrm{{d}}x<\sigma , \ \text {for any}\ n\ge N. \end{aligned}$$

Thus,

$$\begin{aligned} \int _{B^c_R(y_n)}(|\nabla u_n(x)|^2+u^2_n(x))\mathrm{{d}}x<\sigma , \ \text {for any}\ n\ge N. \end{aligned}$$

(5.4)

From $|\varepsilon _ny_n|\rightarrow \infty$ as $n\rightarrow \infty$, we can get $B_R(y_n)\cap B_{\frac{\rho }{\varepsilon _n}}(0)=\emptyset$. Then, by using (5.4), we have

$$\begin{aligned} \Big |\int _{B_{\frac{\rho }{\varepsilon _n}}(0)}(V(\varepsilon _n x)-V^\infty )|{\tilde{t}}_nu_n|^2\mathrm{{d}}x\Big |<C\sigma \end{aligned}$$

and

$$\begin{aligned} \Big |\int _{B_{\frac{\rho }{\varepsilon _n}}(0)}(P(\varepsilon _n x)-P^\infty )|{\tilde{t}}_nu_n|^p\mathrm{{d}}x\Big |<C\sigma . \end{aligned}$$

Thus, noting (5.3), we can get

$$\begin{aligned} \begin{aligned} I_{\varepsilon _n}(u_n)&\ge I_{\varepsilon _n}({\tilde{t}}_nu_n)\\&=I_{\delta }({\tilde{t}}_nu_n)+\frac{1}{2}\int _{{\mathbb {R}}^3}(V(\varepsilon _n x)-(V^\infty -\delta ))|{\tilde{t}}_nu_n|^2\mathrm{{d}}x\\&\quad -\frac{1}{p}\int _{{\mathbb {R}}^3}(P(\varepsilon _n x)-(P^\infty +\delta ))|{\tilde{t}}_nu_n|^p\mathrm{{d}}x\\&\quad -\frac{1}{2(6-\mu )}\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{(Q(\varepsilon _n x)Q(\varepsilon _n y)-(Q^\infty +\delta )^2)|{\tilde{t}}_nu_n(x)|^{6-\mu }|{\tilde{t}}_nu_n(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y\\&\ge I_{\delta }({\tilde{t}}_nu_n)+\frac{1}{2}\int _{{\mathbb {R}}^3}(V(\varepsilon _n x)-(V^\infty -\delta ))|{\tilde{t}}_nu_n|^2\mathrm{{d}}x\\&\quad -\frac{1}{p}\int _{{\mathbb {R}}^3}(P(\varepsilon _n x)-(P^\infty +\delta ))|{\tilde{t}}_nu_n|^p\mathrm{{d}}x\\&\ge I_{\delta }({\tilde{t}}_nu_n)+\frac{1}{2}\int _{B_{\frac{\rho }{\varepsilon _n}}(0)}(V(\varepsilon _n x)-(V^\infty -\delta ))|{\tilde{t}}_nu_n|^2\mathrm{{d}}x\\&\quad -\frac{1}{p}\int _{B_{\frac{\rho }{\varepsilon _n}}(0)}(P(\varepsilon _n x)-(P^\infty +\delta ))|{\tilde{t}}_nu_n|^p\mathrm{{d}}x\\&\ge m_\delta -C\sigma . \end{aligned} \end{aligned}$$

Therefore, $m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}\ge m_\delta$, which is a contradiction.

Up to a subsequence, assume that $\varepsilon _{n} y_{n} \rightarrow \xi$. Hence, it suffices to show that $V(\xi )=V_\mathrm{{min}}$, $P(\xi )=P_\mathrm{{max}}$ and $Q(\xi )=Q_\mathrm{{max}}$. Arguing by contradiction again, we assume that $V(\xi )>V_\mathrm{{min}}$, $P(\xi )<P_\mathrm{{max}}$ or $Q(\xi )<Q_\mathrm{{max}}$. Since

$$\begin{aligned} \lim _{n \rightarrow \infty } \int _{{\mathbb {R}}^{3}} V\left( \varepsilon _{n} y_{n}+\varepsilon _{n} x\right) {\hat{u}}^{2}_{n}\mathrm{{d}}x=\int _{{\mathbb {R}}^{3}} V(\xi ) {\hat{u}}^{2}\mathrm{{d}}x, \\ \lim _{n \rightarrow \infty } \int _{{\mathbb {R}}^{3}} P\left( \varepsilon _{n} y_{n}+\varepsilon _{n} x\right) |{\hat{u}}_{n}|^{p}\mathrm{{d}}x=\int _{{\mathbb {R}}^{3}} P(\xi )|{\hat{u}}|^{p}\mathrm{{d}}x \end{aligned}$$

and

$$\begin{aligned}&\lim _{n \rightarrow \infty } \int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{Q(\varepsilon _n x_n+\varepsilon _{n} x)Q(\varepsilon _n y_n+\varepsilon _{n} y)|{\hat{u}}_n(x)|^{6-\mu }|{\hat{u}}_n(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y\\&\quad =Q^2(\xi )\int _{{\mathbb {R}}^3}\int _{{\mathbb {R}}^3}\frac{|{\hat{u}}(x)|^{6-\mu }|{\hat{u}}(y)|^{6-\mu }}{|x-y|^\mu }\mathrm{{d}}x\mathrm{{d}}y, \end{aligned}$$

we can obtain

$$\begin{aligned} \begin{aligned} m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}&\le I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}({\hat{u}}) \\&<\frac{1}{2} \int _{{\mathbb {R}}^{N}}|\nabla {\hat{u}}|^{2}+\frac{1}{2} \int _{{\mathbb {R}}^{N}} V(\xi ) {\hat{u}}^{2}-\frac{1}{p} \int _{{\mathbb {R}}^{N}} P(\xi ) |{\hat{u}}|^{p}-\frac{1}{2(6-\mu )}Q^2(\xi ) {\tilde{F}}({\hat{u}})\\&=\lim _{n \rightarrow \infty } I_{\varepsilon _{n}}\left( t_{n} u_{n}\right) \le \lim _{n \rightarrow \infty } I_{\varepsilon _{n}}\left( u_{n}\right) =m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}, \end{aligned} \end{aligned}$$

which is a contradiction. Therefore, $V(\xi )=V_\mathrm{{min}}$, $P(\xi )=P_\mathrm{{max}}$, and $Q(\xi )=Q_\mathrm{{max}}$, and the proof is completed. $\square$

Lemma 5.4

For any $\delta >0$, there holds that

$$\begin{aligned} \lim \limits _{\varepsilon \rightarrow 0}\sup _{u\in \tilde{{\mathcal {N}}}_\varepsilon }dist(\beta _\varepsilon (u), ({\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}})_\delta )=0. \end{aligned}$$

Proof

Let $\{\varepsilon _n\}\subset (0,+\infty )$ be such that $\varepsilon _n\rightarrow 0$. By definition, there exists $\{u_n\}\subset \tilde{{\mathcal {N}}}_{\varepsilon _n}$ such that

$$\begin{aligned} dist(\beta _{\varepsilon _n}(u_n),({\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}})_\delta ) =\sup _{u\in \tilde{{\mathcal {N}}}_{\varepsilon _n}}dist(\beta _{\varepsilon _n}(u),({\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}})_\delta )+o_n(1). \end{aligned}$$

So, it suffices to find a sequence $\{\xi _n\}\subset ({\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}})_\delta$ satisfying

$$\begin{aligned} \lim \limits _{n\rightarrow +\infty }|\beta _{\varepsilon _n}(u_n)-\xi _n|=0. \end{aligned}$$

(5.5)

By Lemma 5.3, we can obtain ${\tilde{u}}\in E$ such that $u_n(x+y_n)\rightarrow {\tilde{u}}$ in E, and, up to a subsequence, $\varepsilon _ny_n\rightarrow \xi \in {\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}}.$ Thus, $\varepsilon _ny_n\in ({\mathcal {V}}\cap {\mathcal {P}}\cap {\mathcal {Q}})_\delta$ for n large enough. It is easy to see that

$$\begin{aligned} \beta _{\varepsilon _n}(u_n)=\varepsilon _ny_n+\frac{\int _{{\mathbb {R}}^3} (\chi (\varepsilon _nx+\varepsilon _ny_n)-\varepsilon _ny_n)|u_n(x+y_n)|^2\mathrm{{d}}x}{\int _{{\mathbb {R}}^3}|u_n(x+y_n)|^2\mathrm{{d}}x}. \end{aligned}$$

Set $\xi _n=\varepsilon _ny_n$. We have that the sequence $\{\xi _n\}$ satisfies (5.5). This completes the proof. $\square$

Lemma 5.5

[4] Let I be a $C^1$ functional defined on a $C^1$ Finsler manifold M. If I is bounded from below and satisfies the (PS) condition, then I has at least $cat_MM$ distinct critical points.

Lemma 5.6

[2] Let $\Gamma ,\Omega ^+,\Omega ^-$ be closed sets with $\Omega ^-\subset \Omega ^+.$ Let $\Phi :\Omega ^-\rightarrow \Gamma ,$ $\beta :\Gamma \rightarrow \Omega ^+$ be two continuous maps such that $\beta \circ \Phi$ is homotopically equivalent to the embedding $Id:\Omega ^-\rightarrow \Omega ^+.$ Then $cat_\Gamma \Gamma \ge cat_{\Omega ^+}\Omega ^-.$

The proof of Theorem 1.1:

(i) For a fixed $\delta >0$, by Lemmas 5.1 and 5.4, we know that there exists a $\varepsilon _\delta >0$ such that for any $\varepsilon \in (0,\varepsilon _\delta )$, the diagram

$$\begin{aligned} {\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}} {\mathop {\longrightarrow }\limits ^{\Phi _\varepsilon }} \tilde{{\mathcal {N}}}_\varepsilon {\mathop {\longrightarrow }\limits ^{\beta _\varepsilon }}({\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}})_\delta \end{aligned}$$

is well defined. From Lemma 5.2, for $\varepsilon$ small enough, there is a function $\lambda (\xi )$ with $|\lambda (\xi )|<\frac{\delta }{2}$ uniformly in $\xi \in {\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}}$, such that $\beta _\varepsilon (\Phi _\varepsilon (\xi ))=\xi +\lambda (\xi )$ for all $\xi \in {\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}}$. Define $H(t,\xi )=\xi +(1-t)\lambda (\xi )$. Then, $H:[0,1]\times {\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}}\rightarrow ({\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}})_\delta$ is continuous. Obviously, $H(0,\xi )=\beta _\varepsilon (\Phi _\varepsilon (\xi )),H(1,\xi )=\xi$ for all $\xi \in {\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}}$. That is, $H(t,\xi )$ is homotopy between $\beta _\varepsilon \circ \Phi _\varepsilon$ and the inclusion map $id:{\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}}\rightarrow ({\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}})_\delta$. By Lemma 5.6, we obtain

$$\begin{aligned} cat_{\tilde{{\mathcal {N}}}_\varepsilon }\tilde{{\mathcal {N}}}_\varepsilon \ge cat_{({\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}})_\delta }({\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}}). \end{aligned}$$

On the other hand, using the definition of ${\tilde{\mathcal {N}}}_\varepsilon$ and choosing $\varepsilon _\delta$ small if necessary, we see that $I_\varepsilon$ satisfies the (PS) condition in ${\tilde{N}}_\varepsilon$ recalling $(H_1)$ and Lemma 4.4. By Lemma 5.5, we know that $I_\varepsilon$ has at least $cat_{\tilde{\mathcal {N}}_\varepsilon }(\tilde{\mathcal {N}}_\varepsilon )$ critical points on ${\mathcal {N}}_\varepsilon$. By Lemma 2.11, these points are critical points of $I_\varepsilon$ in E. Consequently, we see that the problem (1.1) has at least $cat_{({\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}})_\delta }({\mathcal {V}} \cap {\mathcal {P}}\cap {\mathcal {Q}})$ solutions.

(ii) By the definition of $c_\varepsilon$ and $m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}$, we have $m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}\le c_\varepsilon$. Then,

$$\begin{aligned} \liminf _{\varepsilon \rightarrow 0}c_\varepsilon \ge m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}. \end{aligned}$$

It follows from Lemma 3.3 that

$$\begin{aligned} \lim _{\varepsilon \rightarrow 0}c_\varepsilon = m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}. \end{aligned}$$

By Lemma 5.3, there exist $\{y_n\}\subset {\mathbb {R}}^3$ and $v\in E$ such that

$$\begin{aligned} u_n(x+y_n)\rightarrow v,\ \text {in } E. \end{aligned}$$

Now we prove that v is a ground state solution of equation 1.2. For any $\psi \in \mathcal {C}_0^\infty ({\mathbb {R}}^3)$, since $I'_{\varepsilon _n}(u_n)=0$, we have $\langle I'_{\varepsilon _n}(u_n(x)), \psi (x-y_n)\rangle =0$. By direct computation, it is easy to get

$$\begin{aligned}\langle I'_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}(v), \psi \rangle =0. \end{aligned}$$

On the other hand,

$$\begin{aligned}&m_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}\\&\quad =\lim _{n\rightarrow \infty }I_{\varepsilon _n}(u_n)\\&\quad =\lim _{n\rightarrow \infty }\Big [I_{\varepsilon _n}(u_n)-\frac{1}{2(6-\mu )} \langle I_{\varepsilon _n}'(u_n), u_n\rangle \Big ]\\&\quad =\lim _{n\rightarrow \infty }\Big [\left( \frac{1}{2}-\frac{1}{2(6-\mu )}\right) \int _{{\mathbb {R}}^3}(|\nabla u_n|^2+V(\varepsilon _n x)u_n^2)\mathrm{{d}}x-\left( \frac{1}{p}-\frac{1}{2(6-\mu )}\right) \int _{{\mathbb {R}}^3}P(\varepsilon _n x)|u_n|^p\mathrm{{d}}x\Big ]\\&\quad =\left( \frac{1}{2}-\frac{1}{2(6-\mu )}\right) \int _{{\mathbb {R}}^3}(|\nabla v|^2+V(\xi )v^2)\mathrm{{d}}x-\left( \frac{1}{p}-\frac{1}{2(6-\mu )}\right) \int _{{\mathbb {R}}^3}P(\xi )|v|^p\mathrm{{d}}x\\&\quad =\left( \frac{1}{2}-\frac{1}{2(6-\mu )}\right) \int _{{\mathbb {R}}^3}(|\nabla v|^2+V_\mathrm{{min}}v^2)\mathrm{{d}}x-\left( \frac{1}{p}-\frac{1}{2(6-\mu )}\right) \int _{{\mathbb {R}}^3}P_\mathrm{{max}}|v|^p\mathrm{{d}}x\\&\quad =I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}(v)-\frac{1}{2(6-\mu )}\langle I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}'(v), v\rangle \\&\quad =I_{V_\mathrm{{min}}P_\mathrm{{max}}Q_\mathrm{{max}}}(v). \end{aligned}$$

Thus, v is a ground solution of Eq. (1.2). $\square$