1 Introduction and main results

Recently, the following nonlinear Choquard problem

$$\begin{aligned} -\Delta u+V(x)u=\big (I_{\alpha }*|u|^{p}\big )|u|^{p-2}u, \quad x\in {\mathbb {R}}^{N} \end{aligned}$$
(1.1)

has been investigated by many authors, where \(I_{\alpha }:{\mathbb {R}}^N{\setminus } \{0\}\rightarrow {\mathbb {R}}\) is the Riesz potential defined by

$$\begin{aligned} I_\alpha (x):=\frac{A_\alpha }{|x|^{N-\alpha }}=\frac{\Gamma (\frac{N-\alpha }{2})}{\Gamma (\frac{\alpha }{2})\pi ^{N/2}2^{\alpha }|x|^{N-\alpha }},\quad \alpha \in (0,N) \end{aligned}$$

and \(\Gamma \) is the Gamma function, see [31, 34].

Equation (1.1) is usually called the nonlinear Choquard or Choquard–Pekar equation. It has several physical motivations. In the physical case \(N = 3\), \(p = 2\) and \(\alpha = 2\), the problem

$$\begin{aligned} -\Delta u+u=\big (I_2 *|u|^{2}\big )u, \quad x\in {\mathbb {R}}^{3} \end{aligned}$$
(1.2)

appeared as early as in 1954, in a work by Pekar describing the quantum mechanics of a polaron at rest [33]. See also [24, 30] for more physical background of Eqs. (1.1)–(1.2). In particular, Lieb [21] proved that the ground state solution of Eq. (1.2) is radial and unique up to translations (see also [25]). Later, Wei and Winter [37] showed that the ground state solution is nondegenerate.

Problem (1.1) has a variational structure, setting \(V(x)\equiv 1\) for example, the corresponding energy functional is defined by

$$\begin{aligned} E_{\alpha ,p}(u)=\frac{1}{2}\int _{{\mathbb {R}}^N}\big (|\nabla u|^2+u^2\big )dx-\frac{1}{2p}\int _{{\mathbb {R}}^N}\big (I_\alpha *|u|^p\big )|u|^pdx, \,\, u\in W^{1,2}({\mathbb {R}}^N)\cap L^{\frac{2Np}{N+\alpha }}({\mathbb {R}}^N). \end{aligned}$$
(1.3)

It follows by the Hardy–Littlewood–Sobolev inequality that the functional \(E_{\alpha ,p}(u)\) is well defined and belongs to \(\mathcal {C}^1(H^1({\mathbb {R}}^N),{\mathbb {R}})\) if \(p\in [\frac{N+\alpha }{N},\,\frac{N+\alpha }{N-2}]\). Moreover, the critical points of \(E_{\alpha ,p}\) are weak solutions of Eq. (1.1).

Theorem A

(See [22, 23], Hardy–Littlewood–Sobolev inequality) Suppose \(\alpha \in (0,N)\), and p, \(r>1\) with \( \frac{1}{p}+\frac{1}{r}=1+\frac{\alpha }{ N}\). Let \(f\in L^{p}({\mathbb {R}}^N)\), \(g\in L^{r}({\mathbb {R}}^N)\), then there exists a sharp constant \(C(p,r,\alpha ,N)\), independent of f and g, such that

$$\begin{aligned} \Big |\int _{{\mathbb {R}}^N}\int _{{\mathbb {R}}^N}\frac{f(x)g(y)}{|x-y|^{N-\alpha }}dxdy \Big | \le C(p,\alpha ,r,N)\Vert f\Vert _{L^p} \Vert g\Vert _{L^r}, \end{aligned}$$
(1.4)

where \(\Vert \cdot \Vert _{L^p}=\left( \int _{{\mathbb {R}}^N}|u|^{p}dx\right) ^{\frac{1}{p}}\). If \(p=r=\frac{2N}{N+\alpha }\), then

$$\begin{aligned} C(p,r,\alpha ,N)=C(N,\alpha )=\pi ^{\frac{N-\alpha }{2}}\frac{\Gamma (\frac{\alpha }{2})}{\Gamma (\frac{N+\alpha }{2})} \Big \{\frac{\Gamma (\frac{N}{2})}{\Gamma (N)}\Big \}^{-\frac{\alpha }{N}}. \end{aligned}$$
(1.5)

In this case, the equality in (1.4) is achieved if and only if \(f\equiv \text {(const.)}g\) and

$$\begin{aligned} g(x)=A(\widetilde{\gamma }^{2}+|x-\widetilde{a}|^{2})^{-\frac{(N+\alpha )}{2}} \end{aligned}$$

for some \(A\in \mathbb {C}\), \(\widetilde{a}\in {\mathbb {R}}^N\) and \(0\ne \widetilde{\gamma }\in {\mathbb {R}}\).

For \(N\ge 3\), \(0<\alpha <N\), let \(2^\alpha _*=\frac{N+\alpha }{N}\) and \(2^*_\alpha =\frac{N+\alpha }{N-2}\). By the Sobolev embedding theorem, \(W^{1,2}({\mathbb {R}}^N)\subset L^{\frac{2Np}{N+\alpha }}({\mathbb {R}}^N)\) if and only if \(p\in [2^\alpha _*, 2^*_\alpha ]\). In [31], Moroz and Van Schaftingen proved that \(E_{\alpha , p}(u)\) has no nontrivial critical points when \(p\not \in \left[ 2^\alpha _*, 2^*_\alpha \right] \). Hence, \(2^\alpha _*\) and \(2^*_\alpha \) are critical exponents for existence and nonexistence of solutions to Eq. (1.1). In the past few years, there is plenty of work dealt with Eq. (1.1) with \(p\in (2^\alpha _*, 2^*_\alpha )\) by variational methods, see for example [2, 28,29,30,31,32, 37]. When \(p=2^\alpha _*\), Moroz and Van Schaftingen [32] proved the existence of one nontrivial solution to Eq. (1.1) if V(x) satisfies

$$\begin{aligned} \liminf \limits _{|x|\rightarrow +\infty }\big (1-|x|^2\big )V(x)>\frac{N^2(N-2)}{4(N+1)}. \end{aligned}$$

As for the upper Hardy–Littlewood–Sobolev exponent, Gao and Yang [12] considered the following Brézis–Nirenberg type problem on bounded domains

$$\begin{aligned} -\Delta u =\left( \displaystyle \int _{\Omega }\frac{|u(y)|^{2_\alpha ^*}}{|x-y|^{N-\alpha }}dy\right) |u|^{2_\alpha ^*-2}u+\lambda u, \quad x\in \Omega , \quad u\in H^1_0(\Omega ). \end{aligned}$$

In this paper, we first consider

$$\begin{aligned} -\Delta u=\big (I_\alpha *|u|^{2_\alpha ^*}\big )|u|^{2_\alpha ^*-2}u, \quad u\in \mathcal {D}^{1,2}({\mathbb {R}}^{N}). \end{aligned}$$
(1.6)

By using Theorem A, one can verify that, up to translations and scalings, the ground state solution of Eq. (1.6) is unique and has the form

$$\begin{aligned} u(x)=c_{\alpha }\left( \frac{t}{t^2+|x-x_0|^2}\right) ^{\frac{N-2}{2}} \end{aligned}$$
(1.7)

where \(t>0\), \(x_0\in {\mathbb {R}}^N\) and

$$\begin{aligned} c_\alpha =\frac{[N(N-2)]^{\frac{N-2}{2}}}{\big [C(N,\alpha )A_\alpha S^{\frac{\alpha }{2}}\big ]^{\frac{N-2}{4+2\alpha }}}, \end{aligned}$$
(1.8)

here S is the best Sobolev constant for the embedding \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\hookrightarrow L^{2^{*}}({\mathbb {R}}^N)\).

A natural question is whether positive solution of Eq. (1.6) is unique and has the form of (1.7). Our result on this aspect can be stated as follows.

Theorem 1.1

Suppose \(0<\alpha <N\) if \(N=3\) or 4, and \(N-4\le \alpha <N\) if \(N\ge 5\), let u(x) be a positive solution of Eq. (1.6), then u(x) is radially symmetric and monotone decreasing about some point \(x_0\in {\mathbb {R}}^N\). Moreover, u(x) has the form of (1.7).

Remark 1.1

  1. (i)

      If \(p<\frac{N+\alpha }{N-2}\), by Pohozaev type identity, the following equation

    $$\begin{aligned} -\Delta u=\big (I_{\alpha }*|u|^p\big )|u|^{p-2}u, \quad x\in {\mathbb {R}}^{N} \end{aligned}$$
    (1.9)

    has no nontrivial solution \(u\in W^{1,2}({\mathbb {R}}^N)\cap L^{\frac{2Np}{N+\alpha }}({\mathbb {R}}^N)\) with \(\nabla u\in W_{loc}^{1,2}({\mathbb {R}}^N)\cap L_{loc}^{\frac{2Np}{N+\alpha }}({\mathbb {R}}^N)\).

  2. (ii)

      We prove Theorem 1.1 by a moving plane method, which was invented by Alexanderov in [1]. Later, it was further developed by Serrin [35], Gidas et al. [14], Caffarelli et al. [5] when classifying the solutions of semilinear elliptic equation

    $$\begin{aligned} -\Delta u=u^{\frac{N+2}{N-2}}, \quad x\in {\mathbb {R}}^N. \end{aligned}$$

    Subsequently, Chen and Li [8] and Li [17] simplified the proof, Wei and Xu [38] and Chen et al. [11] generalized the classification result to the solutions of higher order conformally invariant equations

    $$\begin{aligned} (-\Delta )^{s} u=u^{\frac{N+s}{N-s}}, \quad x\in {\mathbb {R}}^N,\ 0<s<N. \end{aligned}$$

    Li [18] used the method of moving spheres to obtain the same classification result as that in [11]. For other applications, we refer the readers to [7, 9, 10, 16, 28].

Based on the uniqueness result, we can investigate the following Choquard equation

$$\begin{aligned} -\Delta u+V(x)u=\big (I_\alpha *|u|^{2_{\alpha }^{*}}\big ) |u|^{2_{\alpha }^{*}-2}u, \quad x\in {\mathbb {R}}^N,\ N\ge 3, \end{aligned}$$
(1.10)

where the potential function \(V(x)\in L^{\frac{N}{2}}({\mathbb {R}}^N)\cap \mathcal {C}^{\gamma }({\mathbb {R}}^N)\) is nonnegative for some \(\gamma \in (0,1)\). Define the energy functionals I, \(I_{\infty }\) corresponding to Eqs. (1.10), (1.6) respectively by

$$\begin{aligned} I(u)=\frac{1}{2}\int _{{\mathbb {R}}^N}\big (|\nabla u|^{2}+V(x)u^{2}\big )dx -\frac{1}{2\cdot 2_{\alpha }^{*}}\int _{{\mathbb {R}}^N}\big (I_\alpha *|u|^{2_{\alpha }^*}\big )|u|^{2_{\alpha }^*}dx, \quad u\in \mathcal {D}^{1,2}({\mathbb {R}}^N) \end{aligned}$$

and

$$\begin{aligned} I_{\infty }(u)=\frac{1}{2}\int _{{\mathbb {R}}^N}|\nabla u|^{2}dx-\frac{1}{2\cdot 2_{\alpha }^{*}}\int _{{\mathbb {R}}^N}\big (I_\alpha *|u|^{2_{\alpha }^*}\big )|u|^{2_{\alpha }^*}dx, \quad u\in \mathcal {D}^{1,2}({\mathbb {R}}^N). \end{aligned}$$

The Nehari manifolds corresponding to I and \(I_{\infty }\) denoted by \(\mathcal {N}\) and \(\mathcal {N}_{\infty }\) respectively are

$$\begin{aligned}&\mathcal {N}:=\Big \{u \in \mathcal {D}^{1,2}({\mathbb {R}}^N){\setminus }\{0\}: \langle I'(u),u\rangle =0\Big \},\\&\mathcal {N}_{\infty }:=\Big \{u \in \mathcal {D}^{1,2}({\mathbb {R}}^N){\setminus }\{0\}: \langle I'_{\infty }(u),u\rangle =0\Big \}. \end{aligned}$$

Moreover, we define

$$\begin{aligned} m:=\inf _{u\in \mathcal {N}}I(u) \end{aligned}$$

and

$$\begin{aligned} m_{\infty }:=\inf _{u\in \mathcal {N}_{\infty }}I_{\infty }(u). \end{aligned}$$

Obviously, m is the mountain pass level of the functional I and

$$\begin{aligned} m=\inf _{u\in \mathcal {D}^{1,2}({\mathbb {R}}^N){\setminus }\{0\}}\max _{t>0}I(tu)>0. \end{aligned}$$

Our main result on Eq. (1.10) can be stated as follows.

Theorem 1.2

Let \(0<\alpha <N\) if \(N=3\) or 4, and \(N-4\le \alpha <N\) if \(N\ge 5\), and suppose that \(V(x)\in L^{\frac{N}{2}}({\mathbb {R}}^N)\cap \mathcal {C}^{\gamma }({\mathbb {R}}^N)\) is nonnegative for some \(\gamma \in (0,1)\), then \(m=m_{\infty }\) holds and m is not achieved. If V(x) in addition satisfies

$$\begin{aligned} 0<\Vert V(x)\Vert _{L^{\frac{N}{2}}}:=\left( \int _{{\mathbb {R}}^N}|V(x)|^{\frac{N}{2}}dx\right) ^{\frac{2}{N}} <(2^{\frac{\alpha +2}{N+\alpha }}-1)S{{,}} \end{aligned}$$

then Eq. (1.10) possesses at least one positive solution.

We prove Theorem 1.2 by following the variational approach developed by Benci and Cerami [3], in which a similar result was proved for the following Schrödinger equation

$$\begin{aligned} -\Delta u+ V(x)u=u^{\frac{N+2}{N-2}}, \quad x\in {\mathbb {R}}^N,\ N\ge 3. \end{aligned}$$
(1.11)

However, we cannot apply this approach directly, several difficulties arise because of the nonlocal nonlinearity with critical exponent. The main obstacle is lack of compactness, even if we get a \((PS)_c\) sequence with \(c\in (m_\infty , 2m_\infty )\), we still cannot obtain the strongly convergence of \((PS)_c\) sequence, because the nodal solutions of Eq. (1.6) doesn’t possess the double energy property (see [39]), i.e. there may exist nodal solutions of Eq. (1.6) with energy between \(m_\infty \) and \(2m_\infty \) (see Theorem 3, [13]), but the double energy property is crucial for proving the main result in [3]. We solve this difficulty by using Linking Theorem to seek a nonnegative \((PS)_c\) sequence with \(c\in (m_\infty , 2m_\infty )\) and analysing carefully the nonlocal nonlinearity. To this end, a nonlocal version of the Concentration-Compactness Principle (see Lemma 2.1, [27]) is used, which is totally different from the usual local case.

The following splitting result for Palais–Smale sequences is crucial for proving Theorem 1.2, while the local case on bounded domain has been established by Struwe [36].

Theorem 1.3

Suppose \(V(x)\ge 0\) and \(V(x)\in L^{\frac{N}{2}}({\mathbb {R}}^N)\), let \(\{u_n\}\) be a Palais–Smale sequence of I at level c. Then \(\{u_n\}\) has a subsequence which converges strongly in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\), or otherwise, replacing \(\{u_n\}\) if necessary by a subsequence, there exists a function \(\bar{u}\in \mathcal {D}^{1,2}({\mathbb {R}}^N)\) satisfying \(u_n\rightharpoonup \bar{u}\) in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\). Moreover, there exists a number \(k\in {\mathbb N}\), k functions \(u^1, \ldots , u^k\in \mathcal {D}^{1,2}({\mathbb {R}}^N)\); k sequences of points \(\{y_{n}^{i}\}\subset {\mathbb {R}}^N\), \(1\le i \le k\) and k sequences of positive numbers \(\{\sigma _{n}^{i}\}\), \(1\le i\le k\), such that

$$\begin{aligned} \left\| u_n(\cdot )-\bar{u}(\cdot )-\sum _{i=1}^{k}(\sigma _{n}^{i})^{-\frac{N-2}{2}}u^{i} \left( \frac{\cdot -y_{n}^{i}}{\sigma _{n}^{i}}\right) \right\| \rightarrow 0, \end{aligned}$$
(1.12)

where \(\bar{u}\) is a nontrivial solution of Eq. (1.10) and \(u^{i}\), \(1 \le i \le k\), are the nontrivial solutions of Eq. (1.6). Moreover, as \(n\rightarrow +\infty \), we have

$$\begin{aligned} \Vert u_n\Vert ^{2}\rightarrow \Vert \bar{u}\Vert ^2+\sum _{i=1}^{k}\Vert u^{i}\Vert ^2 \end{aligned}$$
(1.13)

and

$$\begin{aligned} I(u_n)\rightarrow I(\bar{u})+\sum _{i=1}^{k}I_{\infty }(u^{i}). \end{aligned}$$
(1.14)

where \(\Vert u\Vert ^2=\int _{{\mathbb {R}}^N}|\nabla u|^2dx\) for \(u\in \mathcal {D}^{1,2}({\mathbb {R}}^N)\).

The paper is organized as follows. In Sect. 2, via the moving plane method, we prove that, up to translations and scalings, the positive solution of Eq. (1.6) is unique. In Sect. 3, by studying the behavior of Pslais–Smale sequences, we obtain a global compactness result, which provides a complete description of Palais–Smale sequences. In Sect. 4, we first show that the mountain pass value is not achieved. Then, combining Linking Theorem with Theorem 1.3, we prove the existence of at least one positive solution for Eq. (1.10).

2 Uniqueness of positive solution

In this section, we set \(A_\alpha \equiv 1\) for convenience. We will use the moving planes method to show the uniqueness of the positive solution of Eq. (1.6). To do this, we first show the invariance of (1.6) under Kelvin transform. Denote \(K_u\) the Kelvin transform of u, that is,

$$\begin{aligned} K_u(x)=\frac{1}{|x|^{N-2}}u\left( \frac{x}{|x|^2}\right) . \end{aligned}$$

Lemma 2.1

Let u(x) be a solution of Eq. (1.6), then, \(U=K_u\) is still a solution of Eq. (1.6).

Proof

Note that

$$\begin{aligned} \Delta K_u(x)=\frac{1}{|x|^{N+2}}\Delta u\left( \frac{x}{|x|^2}\right) . \end{aligned}$$

On the other hand,

$$\begin{aligned} \left( \frac{1}{|\cdot |^{N-\alpha }}*|K_u|^{2_\alpha ^*}\right) (x)&=\int _{{\mathbb {R}}^N}\frac{\big |u\left( \frac{y}{|y|^2}\right) \big |^{2_\alpha ^*}}{|x-y|^{N-\alpha } |y|^{N+\alpha }}dy = \int _{{\mathbb {R}}^N}\frac{|u(y)|^{2_\alpha ^*}}{|x-\frac{y}{|y|^2}|^{N-\alpha }}|y|^{N+\alpha }|y|^{-2N}dy \\&= |x|^{\alpha -N}\int _{{\mathbb {R}}^N}\frac{|u(y)|^{2_\alpha ^*}}{|\frac{x}{|x|^2}-y|^{N-\alpha }}dy = |x|^{\alpha -N}\left( \frac{1}{|\cdot |^{N-\alpha }}*|u|^{2_\alpha ^*}\right) \left( \frac{x}{|x|^2}\right) , \end{aligned}$$

where we use the identity

$$\begin{aligned} |y|\big |x-\frac{y}{|y|^2}\big |=|x|\big |\frac{x}{|x|^2}-y\big | \end{aligned}$$

in the third step. Therefore, we have

$$\begin{aligned} \Delta K_u(x)= & {} |x|^{-2-N}(-\Delta u)\left( \frac{x}{|x|^2}\right) =|x|^{\alpha -N}\left( \frac{1}{|\cdot |^{N-\alpha }}*|u|^{2_\alpha ^*}\right) \left( \frac{x}{|x|^2}\right) \\&\times |x|^{-\alpha -2}\big |u\left( \frac{x}{|x|^2}\right) \big |^{2_\alpha ^*-1}u\left( \frac{x}{|x|^2}\right) . \end{aligned}$$

This shows that \(U=K_u\) is also a solution of Eq. (1.6), which implies that Eq. (1.6) is invariant under Kelvin transform. \(\square \)

Now, we transform Eq. (1.6) to an equivalent integral system. Let \(v(x)=|x|^{-N+\alpha }*|u|^{2^*_\alpha }\). Then, up to a normalization constant, Eq. (1.6) is equivalent to

$$\begin{aligned} {\left\{ \begin{array}{ll} \displaystyle u(x)=\int _{{\mathbb {R}}^N}\frac{|u(y)|^{2^*_\alpha -2}u(y)v(y)}{|x-y|^{N-2}}dy, \\ \displaystyle v(x)=\int _{{\mathbb {R}}^N}\frac{|u(y)|^{2^*_\alpha }}{|x-y|^{N-\alpha }}dy. \end{array}\right. } \end{aligned}$$
(2.1)

By \(u\in L^{\frac{2N}{N-2}}({\mathbb {R}}^N)\) and Hardy–Littlewood–Sobolev inequality, we know that \(v\in L^{\frac{2N}{N-\alpha }}({\mathbb {R}}^N)\). Making use of the moving plane method in integral forms, we show that each positive solution (uv) of system (2.1) in \(L^{\frac{2N}{N-2}}({\mathbb {R}}^N)\times L^{\frac{2N}{N-\alpha }}({\mathbb {R}}^N)\) is radially symmetric and monotone decreasing about some point \(x_0\in {\mathbb {R}}^N\).

For this purpose, we first introduce some notation. For \(x=(x_1,x_2,\dots ,x_N)\in {\mathbb {R}}^N\), \(\lambda \in {\mathbb {R}}\), we define \(x^{\lambda }=(2\lambda -x_1, x_2,\dots ,x_N)\) and

$$\begin{aligned} u_{\lambda }(x)=u(x^{\lambda }), \quad v_{\lambda }(x)=v(x^{\lambda }). \end{aligned}$$

Let \(\Sigma _{\lambda }=\{x=(x_1,x_2,\dots ,x_N)\in {\mathbb {R}}^N:x_1\ge \lambda \}\). We set

$$\begin{aligned}&\Sigma _{\lambda }^{u}:=\{x\in \Sigma _{\lambda }: u(x)<u_{\lambda }(x)\}, \quad \overline{\Sigma _{\lambda }^{u}}:=\{x\in \Sigma _{\lambda }: u(x)\le u_{\lambda }(x)\},\\&\quad \Sigma _{\lambda }^{v}:=\{x\in \Sigma _{\lambda }: v(x)<v_{\lambda }(x)\}. \end{aligned}$$

Moreover, we denote the complement of \(\Sigma _\lambda \) in \({\mathbb {R}}^N\) by \(\Sigma _{\lambda }^{c}\), and the reflection of \(\Sigma _{\lambda }^{u}\) about the plane \(x_1=\lambda \) by \(\big (\Sigma _{\lambda }^{u}\big )^*\).

We decompose \(u_{\lambda }(x)\), u(x) in \(\Sigma _{\lambda }\) and \(v_{\lambda }(x)\), v(x) in \(\Sigma _{\lambda }\) as follows.

Lemma 2.2

For each positive solution (uv) of system (2.1), we have

$$\begin{aligned} u_{\lambda }(x)-u(x)=\int _{\Sigma _{\lambda }}\left( \frac{1}{|x-y|^{N-2}}-\frac{1}{|x^{\lambda }-y|^{N-2}}\right) \big (|u_\lambda (y)|^{2^*_\alpha -1}v_{\lambda }(y)-|u(y)|^{2^*_\alpha -1}v(y)\big )dy \end{aligned}$$
(2.2)

and

$$\begin{aligned} v_{\lambda }(x)-v(x)=\int _{\Sigma _{\lambda }} \left( \frac{1}{|x-y|^{N-\alpha }}-\frac{1}{|x^{\lambda }-y|^{N-\alpha }}\right) \big (|u_{\lambda }(y)|^{2^*_\alpha }-|u(y)|^{2^*_\alpha }\big )dy. \end{aligned}$$
(2.3)

Proof

By (2.1) and the fact that \(|x-y^{\lambda }|=|x^{\lambda }-y|\), we then obtain

$$\begin{aligned} u(x)&= \int _{{\mathbb {R}}^N} \frac{|u(y)|^{2^*_\alpha -1}v(y)}{|x-y|^{N-2}}dy \nonumber \\&= \int _{\Sigma _{\lambda }}\frac{|u(y)|^{2^*_\alpha -1}v(y)}{|x-y|^{N-2}}dy + \int _{\Sigma _{\lambda }^c}\frac{|u(y)|^{2^*_\alpha -1}v(y)}{|x-y|^{N-2}}dy \nonumber \\&= \int _{\Sigma _{\lambda }}\left( \frac{|u(y)|^{2^*_\alpha -1}v(y)}{|x-y|^{N-2}}+ \frac{|u_\lambda (y)|^{2^*_\alpha -1}v_\lambda (y)}{|x^\lambda -y|^{N-2}}\right) dy, \end{aligned}$$
(2.4)

which leads to

$$\begin{aligned} u_{\lambda }(x)=u(x^\lambda )=\int _{\Sigma _{\lambda }} \left( \frac{|u(y)|^{2^*_\alpha -1}v(y)}{|x^\lambda -y|^{N-2}}+ \frac{|u_\lambda (y)|^{2^*_\alpha -1}v_\lambda (y)}{|x-y|^{N-2}}\right) dy. \end{aligned}$$
(2.5)

From (2.4) and (2.5), we then get (2.2). By a similar argument, we can also prove (2.3). \(\square \)

Using the above preliminaries, we then prove the following proposition.

Proposition 2.3

Suppose \(0<\alpha <N\) if \(N=3\) or 4 and \(N-4\le \alpha <N\) if \(N\ge 5\), and let (uv) be a positive solution of system (2.1) in \(L^{\frac{2N}{N-2}}({\mathbb {R}}^N)\times L^{\frac{2N}{N-\alpha }}({\mathbb {R}}^N)\). Then u and v are both radially symmetric and decreasing about some point \(x_0\in {\mathbb {R}}^N\).

Proof

The proof consists of three steps.

Step 1 There exists \(l_0>0\) such that for any \(\lambda <-l_0\), we have

$$\begin{aligned} u(x)\ge u_{\lambda }(x) \quad and \quad v(x)\ge v_{\lambda }(x), \quad \text {for all} \quad x \in \Sigma _{\lambda }. \end{aligned}$$
(2.6)

For the sufficiently negative value of \(\lambda \), we show that both \(\Sigma _{\lambda }^{u}\) and \(\Sigma _{\lambda }^{v}\) must be empty.

In fact, for any \(x\in \Sigma ^u_{\lambda }\), we have

$$\begin{aligned} 0<u_{\lambda }(x)-u(x)&=\int _{\Sigma _{\lambda }} \left( \frac{1}{|x-y|^{N-2}}-\frac{1}{|x^{\lambda }-y|^{N-2}}\right) \big (|u_\lambda (y)|^{2^*_\alpha -1}v_{\lambda }(y)-|u(y)|^{2^*_\alpha -1}v(y)\big )dy\\&\le \int _{\Sigma _{\lambda }\cap \{|u_\lambda |^{2^*_\alpha -1} v_\lambda >|u|^{2^*_\alpha -1} v\}}\frac{1}{|x-y|^{N-2}} \big (|u_\lambda (y)|^{2^*_\alpha -1}v_{\lambda }(y)-|u(y)|^{2^*_\alpha -1}v(y)\big )dy. \end{aligned}$$

Hence, if \(2^*_\alpha \ge 2\), we then get

$$\begin{aligned} 0&<u_\lambda (x)-u(x) \nonumber \\&\le \int _{\Sigma ^u_\lambda }\frac{(2^*_\alpha -1)|u_\lambda (y)|^{2^*_\alpha -2} v_\lambda (y) (u_\lambda (y)-u(y))}{|x-y|^{N-2}}dy +\int _{\Sigma ^v_\lambda }\frac{|u(y)|^{2^*_\alpha -1}(v_\lambda (y)-v(y))}{|x-y|^{N-2}}dy. \end{aligned}$$
(2.7)

By Lemma 2.2 and Hölder’s inequality, we obtain

$$\begin{aligned} \Vert u_\lambda -u\Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{u})}&\le C_1\Vert u_\lambda ^{2^*_\alpha -2}v_\lambda (u_{\lambda }-u)\Vert _{L^{\frac{2N}{N+2}}(\Sigma _{\lambda }^{u})} + C_2\Vert u^{2^*_\alpha -1}(v_{\lambda }-v)\Vert _{L^{\frac{2N}{N+2}}(\Sigma _{\lambda }^{v})} \nonumber \\&\le C_1\Vert u_\lambda \Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{u})}^{2^*_\alpha -2}\Vert v_\lambda \Vert _{L^{\frac{2N}{N-\alpha }}(\Sigma _{\lambda }^{u})} \Vert u_{\lambda }-u\Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{u})}\nonumber \\&\quad +\,C_2\Vert u\Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{v})}^{2^*_\alpha -1} \Vert v_{\lambda }-v\Vert _{L^{\frac{2N}{N-\alpha }}(\Sigma _{\lambda }^{v})} \end{aligned}$$
(2.8)

and

$$\begin{aligned} \Vert v_\lambda - v\Vert _{L^{\frac{2N}{N-\alpha }}(\Sigma _{\lambda }^{v})}&\le C_3 \Vert u_{\lambda }\Vert ^{2_\alpha ^*-1}_{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{u})} \Vert u_\lambda -u\Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{u})}. \end{aligned}$$
(2.9)

Hence, substituting (2.9) into (2.8), we then obtain

$$\begin{aligned} \Vert u_\lambda -u\Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{u})}&\le C_1\Vert u_\lambda \Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{u})}^{2^*_\alpha -2}\Vert v_\lambda \Vert _{L^{\frac{2N}{N-\alpha }}(\Sigma _{\lambda }^{u})} \Vert u_{\lambda }-u\Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{u})}\nonumber \\&\quad +\,C_4\Vert u\Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{v})}^{2^*_\alpha -1} \Vert u_{\lambda }\Vert ^{2_\alpha ^*-1}_{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{u})} \Vert u_\lambda -u\Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{u})}. \end{aligned}$$
(2.10)

Recalling that \(u\in L^{\frac{2N}{N-2}}({\mathbb {R}}^N)\), \(v\in L^{\frac{2N}{N-\alpha }}({\mathbb {R}}^N)\), by the dominated convergence theorem that we can choose \(l_0\) sufficiently large such that \(\lambda <-l_0\) and

$$\begin{aligned}&C_1\Vert u_\lambda \Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{u})}^{2^*_\alpha -2}\Vert v_\lambda \Vert _{L^{\frac{2N}{N-\alpha }}(\Sigma _{\lambda }^{u})} +C_4\Vert u\Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{v})}^{2^*_\alpha -1} \Vert u_{\lambda }\Vert ^{2_\alpha ^*-1}_{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{u})}\nonumber \\&\quad \le C_1\Vert u_\lambda \Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda })}^{2^*_\alpha -2}\Vert v\Vert _{L^{\frac{2N}{N-\alpha }}(\Sigma _{\lambda }^c)} +C_4\Vert u\Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda })}^{2^*_\alpha -1} \Vert u\Vert ^{2_\alpha ^*-1}_{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^c)}\le \frac{1}{2}. \end{aligned}$$
(2.11)

Thus, it follows by (2.10) and (2.11) that

$$\begin{aligned} \Vert u_\lambda -u\Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{u})}=0. \end{aligned}$$

This implies that \(\Sigma _{\lambda }^{u}\) must be a set with zero measure, hence must be empty up to a set with zero measure. By (2.9), \(\Sigma _{\lambda }^{v}\) must be empty.

Step 2 We move the plane continuously from \(\lambda <-l_0\) to the right as long as (2.6) holds. We show that if the procedure stops at \(x_1=\lambda _0\) for some \(\lambda _0\), then u(x) and v(x) must be symmetric and monotone about the plane \(x_1=\lambda _0\). Otherwise, we can move the plane all the way to the right.

Moving the plane \(x_1=\lambda \) to the right as long as (2.6) holds. Suppose that at some \(\lambda _0\), we have

$$\begin{aligned} u(x)\ge u_{\lambda _0}(x) \quad \text {and} \quad v(x)\ge v_{\lambda _0}(x) \quad \text {on}~\Sigma _{\lambda _0}, \end{aligned}$$

but

$$\begin{aligned} u(x)\not \equiv u_{\lambda _0}(x) \quad \text {or} \quad v(x)\not \equiv v_{\lambda _0}(x) \quad \text {on}~ \Sigma _{\lambda _0}. \end{aligned}$$

In the following, we show that the plane can be moved further to the right. More precisely, there exists \(\delta =\delta (N, u, v)\) such that \(u(x)\ge u_{\lambda }(x)\) and \(v(x)\ge v_{\lambda }(x)\) on \(\Sigma _{\lambda }\) for all \(\lambda \in [\lambda _0,\lambda _0+\delta )\).

Assume that

$$\begin{aligned} v(x)\not \equiv v_{\lambda _0}(x) \quad \text {on}~ \Sigma _{\lambda _0}. \end{aligned}$$

By (2.2), we have \(u(x)> u_{\lambda _0}(x)\) in the interior of \(\Sigma _{\lambda _0}\). Note that

$$\begin{aligned} meas(\overline{\Sigma _{\lambda _0}^{u}})=0 \quad \text {and} \quad \lim _{\lambda \rightarrow \lambda _0}\Sigma _{\lambda }^{u}\subseteq \overline{\Sigma _{\lambda _0}^{u}}, \end{aligned}$$

where \(meas(\overline{\Sigma _{\lambda _0}^{u}})\) denotes the Lebesgue measure of \(\overline{\Sigma _{\lambda _0}^{u}}\). Since \(u\in L^{\frac{2N}{N-2}}({\mathbb {R}}^N)\), \(v\in L^{\frac{2N}{N-\alpha }}({\mathbb {R}}^N)\) and \(meas(\overline{\Sigma _{\lambda _0}^{u}})=0\), then using the dominated convergence theorem, we can choose \(\delta >0\) sufficiently small, such that for all \(\lambda \in [\lambda _0,\lambda _0+\delta )\), we have

$$\begin{aligned} C_1\Vert u_\lambda \Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda })}^{2^*_\alpha -2}\Vert v\Vert _{L^{\frac{2N}{N-\alpha }}\big ((\Sigma _{\lambda }^{u})^*\big )} +C_4\Vert u\Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda })}^{2^*_\alpha -1} \Vert u\Vert ^{2_\alpha ^*-1}_{L^{\frac{2N}{N-2}}\big ((\Sigma _{\lambda }^{u})^*\big )}\le \frac{1}{2}. \end{aligned}$$

It follows from (2.10) that

$$\begin{aligned} \Vert u_\lambda - u\Vert _{L^{\frac{2N}{N-2}}(\Sigma _{\lambda }^{u})}=0. \end{aligned}$$

Hence, \(\Sigma _{\lambda }^{u}\) must be empty for all \(\lambda \in [\lambda _0, \lambda _0+\delta )\), which also implies that \(\Sigma _{\lambda }^{v}\) is empty for all \(\lambda \in [\lambda _0,\lambda _0+\delta )\).

Assume that

$$\begin{aligned} u(x)\not \equiv u_{\lambda _0}(x) \quad \text {on} \quad \Sigma _{\lambda _0}. \end{aligned}$$

By (2.3), we see \(v(x)> v_{\lambda _0}(x)\) in the interior of \(\Sigma _{\lambda _0}\). By the above analysis, we know that \(\Sigma _{\lambda }^{u}\) and \(\Sigma _{\lambda }^{v}\) must also be empty for all \(\lambda \in [\lambda _0,\lambda _0+\delta )\). This completes the proof.

Step 3 By step 1, we know that the plane cannot keep moving all the way to the right in Step 2. That is, the plane will eventually stop at some point. In fact, with the similar analysis as that in Step 1 and Step 2, we then assert that there exists a large \(\bar{l}\), such that for \(\lambda >\bar{l}\),

$$\begin{aligned} u(x)\le u_{\lambda }(x) \quad \text {and} \quad v(x)\le v_{\lambda }(x), \quad \text {for all} \quad x \in \Sigma _{\lambda }. \end{aligned}$$
(2.12)

Now we can move the plane continuously from \(\lambda >\bar{l}\) to the left as long as the above fact holds. The planes moved from the left and the right will eventually meet at some point. Finally, since the direction of \(x_1\) can be chosen arbitrarily, we deduce that u(x) and v(x) must be radially symmetric and decreasing about some point. \(\square \)

Now we use the elliptic regularity theory to show the following proposition.

Proposition 2.4

Assume that u is a positive solution of (1.6). Then u is uniformly bounded in \({\mathbb {R}}^N\). Furthermore, u is \(\mathcal {C}^{\infty }({\mathbb {R}}^N)\) and

$$\begin{aligned} \lim _{|x|\rightarrow +\infty }|x|^{N-2}u(x)=u_{\infty } \end{aligned}$$
(2.13)

for some positive constant \(u_{\infty }\).

Proof

Step 1 We first show that u is uniformly bounded and smooth. For \(A>0\), we define

$$\begin{aligned} \Omega =\{x\in {\mathbb {R}}^N : u(x)> A\} \quad \text {and} \quad u_{A}(x)= {\left\{ \begin{array}{ll} u(x), \quad &{}x \in \Omega , \\ 0, \quad &{}x\in {\mathbb {R}}^N {\setminus } \Omega . \end{array}\right. } \end{aligned}$$

Hence

$$\begin{aligned} u-u_{A}\in L^{\frac{2N}{N-2}}({\mathbb {R}}^N)\cap L^{\infty }({\mathbb {R}}^N), \quad \text {for any} \ A>0. \end{aligned}$$
(2.14)

Since u is a solution of Eq. (1.6), we have

$$\begin{aligned} u(x)=\int _{{\mathbb {R}}^N}\frac{\left( |\cdot |^{\alpha -N}*|u|^{2_\alpha ^*}\right) |u(y)|^{2_\alpha ^*-1}}{|x-y|^{N-2}}dy, \quad \forall x \in {\mathbb {R}}^N, \end{aligned}$$

which implies that for any \(x\in \Omega \),

$$\begin{aligned} u_{A}(x)&=\int _{{\mathbb {R}}^N}\frac{\left( |\cdot |^{\alpha -N}*|u|^{2_\alpha ^*}\right) |u(y)|^{2_\alpha ^*-1}}{|x-y|^{N-2}}dy\\&=\int _{{\mathbb {R}}^N}\frac{\left( |\cdot |^{\alpha -N}*|u_A|^{2_\alpha ^*}\right) |u_A(y)|^{2_\alpha ^*-1}}{|x-y|^{N-2}}dy\\&\quad +\int _{{\mathbb {R}}^N}\frac{\left( |\cdot |^{\alpha -N}*|u-u_A|^{2_\alpha ^*}\right) |u_A(y)|^{2_\alpha ^*-1}}{|x-y|^{N-2}}dy\\&\quad +\int _{{\mathbb {R}}^N}\frac{\left( |\cdot |^{\alpha -N}*|u_A|^{2_\alpha ^*}\right) |u-u_A(y)|^{2_\alpha ^*-1}}{|x-y|^{N-2}}dy\\&\quad +\int _{{\mathbb {R}}^N}\frac{\left( |\cdot |^{\alpha -N}*|u-u_A|^{2_\alpha ^*}\right) |u-u_A(y)|^{2_\alpha ^*-1}}{|x-y|^{N-2}}dy. \end{aligned}$$

Next we divide our argument into three cases.

Case 1 \(0<\alpha \le 2\). For any \(r\ge \frac{2N}{N-2}\), by Hardy–Littlewood–Sobolev inequality, we see

$$\begin{aligned}&\Big \Vert \int _{{\mathbb {R}}^N}\frac{\big (I_\alpha *|u_A(y)|^{2_\alpha ^*}\big )|u_A(y)|^{2_\alpha ^*-1}}{|x-y|^{N-2}}dy\Big \Vert _{L^r}\le \Big [\int _{{\mathbb {R}}^N} \big ( I_\alpha *|u_A|^{2_\alpha ^*}\big )^{\frac{Nr}{N+2r}}|u_A(y)|^{(2_\alpha ^*-1){\frac{Nr}{N+2r}}}dx\Big ]^{\frac{N+2r}{Nr}}\\&\quad \le \Big [\int _{{\mathbb {R}}^N} \big ( I_\alpha *|u_A|^{2_\alpha ^*}\big )^{p\frac{Nr}{N+2r}}dx\Big ]^{\frac{N+2r}{pNr}} \Big [\int _{{\mathbb {R}}^N}|u_A(y)|^{q(2_\alpha ^*-1){\frac{Nr}{N+2r}}}dx\Big ]^{\frac{N+2r}{qNr}}\\&\quad \le {{\Big [\int _{{\mathbb {R}}^N} |u_A|^{2^*}dx\Big ]^{\frac{2_\alpha ^*-1}{2^*}} \Big [\int _{{\mathbb {R}}^N} |u_A|^rdx\Big ]^{\frac{1}{r}}}}\Big [\int _{{\mathbb {R}}^N} |u_A|^{2^*}dx \Big ]^{\frac{2_\alpha ^*-1}{2^*}}, \end{aligned}$$

where \(q=\frac{2N+4r}{r(2+\alpha )}\) and \(1/p+ 1/q=1\). One can easily check that \(q>1\) for every \(r\ge \frac{2N}{N-2}\) and \(0<\alpha \le 2\). Thus, using the Hardy–Littlewood–Sobolev inequality again, one finds

$$\begin{aligned} \Vert u_A\Vert _{L^{r}}&\le C\Vert u_A\Vert _{L^{\frac{2N}{N-2}}}^{2(2_\alpha ^*-1)} \Vert u_A\Vert _{L^{r}} + C \Vert u_A\Vert ^{2_\alpha ^*-1}_{L^{\frac{2N}{N-2}}} \Vert u-u_A\Vert _{L^{\frac{2N}{N-2}}}^{2_\alpha ^*-1} \Vert u-u_A\Vert _{L^{r}} \nonumber \\&\quad +C\Vert u_A\Vert _{L^{\frac{2N}{N-2}}}^{2(2_\alpha ^*-1)} \Vert u-u_A\Vert _{L^{r}}+ C\Vert u-u_A\Vert _{L^{\frac{2N}{N-2}}}^{2(2_\alpha ^*-1)} \Vert u-u_A\Vert _{L^{r}}. \end{aligned}$$
(2.15)

On one hand, by \(u\in L^{\frac{2N}{N-2}}({\mathbb {R}}^N)\), we can choose A large enough, such that

$$\begin{aligned} C\Vert u_A\Vert _{L^{\frac{2N}{N-2}}}^{2(2_\alpha ^*-1)}\le \frac{1}{2}. \end{aligned}$$
(2.16)

On the other hand, by \(u\in L^{\frac{2N}{N-2}}({\mathbb {R}}^N)\) and (2.14), we verify that

$$\begin{aligned}&C \Vert u_A\Vert ^{2_\alpha ^*-1}_{L^{\frac{2N}{N-2}}} \Vert u-u_A\Vert _{L^{\frac{2N}{N-2}}}^{2_\alpha ^*-1} \Vert u-u_A\Vert _{L^{r}} +C\Vert u_A\Vert _{L^{\frac{2N}{N-2}}}^{2(2_\alpha ^*-1)} \Vert u-u_A\Vert _{L^{r}}\nonumber \\&\quad + {{C\Vert u-u_A\Vert _{L^{\frac{2N}{N-2}}}^{2(2_\alpha ^*-1)} \Vert u-u_A\Vert _{L^{r}}\le C(A).}} \end{aligned}$$
(2.17)

Substituting (2.16) and (2.17) into (2.15), we then assert that, for any \(r\ge \frac{2N}{N-2}\)

$$\begin{aligned} \Vert u_A\Vert _{L^{r}}\le \frac{1}{2}\Vert u_A\Vert _{L^{r}}+ C(A), \end{aligned}$$
(2.18)

which implies that \(u_A \in L^{r}({\mathbb {R}}^N)\) for any \(r\ge \frac{2N}{N-2}\). Therefore, we have \(u\in L^{r}({\mathbb {R}}^N)\) for any \(r\ge \frac{2N}{N-2}\). Using Hardy–Littlewood–Sobolev inequality again, we get

$$\begin{aligned} -\Delta u =\big (|\cdot |^{\alpha -N}*|u|^{2_\alpha ^*}\big )|u|^{2_\alpha ^*-2}u \in L^{p}({\mathbb {R}}^N), \quad \text {for any} \quad p\ge \frac{2N}{N-2}. \end{aligned}$$

Using the \(L^{p}\)-theory and Sobolev embedding theorem (see Theorem 9.9, [15]), we know that u is uniformly bounded and belongs to \(\mathcal {C}^{0,s}({\mathbb {R}}^N)\) for all \(0<s<1\). In fact, we also conclude \(u\in \mathcal {C}^{\infty }({\mathbb {R}}^N)\) from Theorem 4.4.8 in [6].

Case 2 \(2<\alpha <N-4\). Let \(p=\frac{N+\alpha }{N-2}<2\). First, we claim that \(u_A\in L^{s}\) for every \(2^*=\frac{2N}{N-2} \le s\le \frac{Np}{\alpha }\). Set \(s_0=2^*\), we assume that \(u_A\in L^s\) for every \(s\in [2^*, s_n]\) and \(s_n<\frac{Np}{\alpha }\). We will prove that \(u_A\in L^r\) if \(r\ge s_n\) satisfies

$$\begin{aligned}&\frac{1}{r}>\frac{p-1}{s_n}-\frac{2}{N}, \end{aligned}$$
(2.19)
$$\begin{aligned}&\frac{1}{r}<\frac{2p-1}{s_n}-\frac{2+\alpha }{N}. \end{aligned}$$
(2.20)

Moreover, we compare \(r_0=(\frac{p-1}{s_n}-\frac{2}{N})^{-1}\) with \(\frac{N p}{\alpha }\). If \(r_0\ge \frac{Np}{\alpha }\), then the claim is proved. If \(r_0<\frac{Np}{\alpha }\), set \(s_{n+1}=r_0\) and proceed again. Since

$$\begin{aligned} \frac{1}{s_n}-\frac{1}{s_{n+1}}=\frac{2-p}{s_n}+\frac{2}{N}>\frac{1}{N}, \end{aligned}$$

our argument must terminate at a finite number of steps. We should note that if \(s_n<\frac{N}{\alpha }p\),

$$\begin{aligned} s_n Nr>(N+2r)s_n-(p-1)Nr+s_nr\alpha . \end{aligned}$$
(2.21)

Then using the Hardy–Littlewood–Sobolev inequality and the condition (2.19)–(2.21), we find

$$\begin{aligned} \begin{aligned}&\Big \Vert \int _{{\mathbb {R}}^N}\frac{\big (I_\alpha *|u_A|^{p}\big )|u_A(y)|^{p-1}}{|x-y|^{N-2}}dy\Big \Vert _{L^r} \le \left\| I_\alpha *|u_A|^{p} \cdot u_A^{p-1}\right\| _{L^{\frac{Nr}{N+2r}}}\\&\quad \le \left\| (I_\alpha *|u_A|^{p})^{\frac{Nr}{N+2r}}\right\| ^{\frac{N+2r}{Nr}}_{L^{\frac{s_n (N+2r)}{s_n (N+2r)-(p-1)Nr}}}\times \Big \Vert u_A^{(p-1){\frac{Nr}{N+2r}}}\Big \Vert ^{\frac{N+2r}{Nr}}_{L^{\frac{s_n (N+2r)}{(p-1)Nr}}}\\&\quad \le \left\| I_\alpha *|u_A|^{p}\right\| _{L^{\frac{s_n Nr}{s_n (N+2r)-(p-1)Nr}}}\times \left\| u_A\right\| ^{p-1}_{L^{s_n}} \end{aligned} \end{aligned}$$

and

$$\begin{aligned} \left\| I_\alpha *|u_A|^{p}\right\| _{L^{\frac{s_n Nr}{s_n (N+2r)-(p-1)Nr}}} \le \left\| u_A\right\| ^p_{L^{\frac{s_n Nrp}{s_n (N+2r)-(p-1)Nr+s_nr\alpha }}}. \end{aligned}$$

Setting \(t={\frac{s_n Nrp}{s_n (N+2r)-(p-1)Nr+s_nr\alpha }}\), we know that \(s_n<t<r\). Hence \(t=(1-\theta )s_n+\theta r\) where \(\theta =\frac{t-s_n}{r-s_n}\). It yields that

$$\begin{aligned} \Vert u_A\Vert ^p_{L^t}\le \Vert u_A\Vert _{L^{s_n}}^{(1-\theta )p}\Vert u_A\Vert _{L^r}^{\theta p}. \end{aligned}$$

Similarly to (2.18), we have

$$\begin{aligned} \Vert u_A\Vert _{L^r}\le \Vert u_A\Vert _{L^{s_n}}^{p-1+(1-\theta )p}\times \Vert u_A\Vert _{L^r}^{\theta p}+\Vert u-u_A\Vert _{L^{s_n}}^{p-1}\Vert u_A\Vert _{L^{s_n}}^{(1-\theta )p}\Vert u_A\Vert _{L^r}^{\theta p}+C(A). \end{aligned}$$

Then we choose \(A>0\) sufficiently large such that

$$\begin{aligned} 2 \Vert u_A\Vert _{L^r}\le \Vert u_A\Vert _{L^r}^{\theta p}+C(A). \end{aligned}$$
(2.22)

Note that \(\theta p<1\). To see this, we only need to prove

$$\begin{aligned} t-s=\frac{(2p-1)sNr-s^2(N+2r+r\alpha )}{s(N+2r+r\alpha )-(p-1)Nr}<\frac{r-s}{p}, \end{aligned}$$
(2.23)

which is equivalent to

$$\begin{aligned} \text {LHS}=2psNr+Nr^2<s^2(N+2r+r\alpha )+sr^2(N-2)=\text {RHS}. \end{aligned}$$

Since \(s\ge \frac{2N}{N-2}\), we compute that

$$\begin{aligned} \text {RHS}&\ge s^2N+s^2r(N-2)(p-1)+2Nr^2 \\&\ge s^2N+2Nsr(p-1)+2Nr^2\\&\ge 2Nsrp+Nr^2+N(s-r)^2 \\&> 2Nsrp+Nr^2=\text {LHS}. \end{aligned}$$

From this, by (2.22) we know

$$\begin{aligned} \Vert u_A\Vert _{L^r}\le \max \{1, C(A)^{\frac{1}{\theta p}}\}. \end{aligned}$$

It follows that \(u_A\in L^{r}({\mathbb {R}}^N)\) for \(A>0\) sufficiently large. Thus \(u\in L^{\frac{N p}{\alpha }}({\mathbb {R}}^N)\) and \(I_\alpha *|u|^p\in L^\infty ({\mathbb {R}}^N)\).

Finally, since u satisfies

$$\begin{aligned} -\Delta u=(I_\alpha *|u|^{2_\alpha ^*})|u|^{2_\alpha ^*-2}u,\quad \text {in }{\mathbb {R}}^N. \end{aligned}$$

Then, by standard elliptic regularity theory, \(u\in \mathcal {C}^\infty ({\mathbb {R}}^N)\).

Case 3 \(N-4\le \alpha < N\). In this case, \(2_\alpha ^*=\frac{N+\alpha }{N-2}\ge 2\). Then \(a(x):=(I_\alpha *|u|^{2_\alpha ^*})u^{2_\alpha ^*-2}\in L^{N/2}({\mathbb {R}}^N)\). The Brézis–Kato theorem [4] implies that \(u\in L^t_{loc}({\mathbb {R}}^N)\) for all \(1\le t<\infty \). Thus, \(u\in W^{2,t}({\mathbb {R}}^N)\) for all \(1\le t<\infty \). By elliptic regularity theory, \(u\in \mathcal {C}^\infty ({\mathbb {R}}^N)\).

Step 2 We want to prove the asymptotic behavior at infinity of u. We prove it by contradiction. Consider the Kelvin transform:

$$\begin{aligned} U(x)=\frac{1}{|x|^{N-2}}u\left( \frac{x}{|x|^2}\right) ~~\Rightarrow ~~ |x|^{N-2}u(x)=U\left( \frac{x}{|x|^2}\right) . \end{aligned}$$

Applying Proposition 2.3 to U(x), we conclude that U(x) must be radially symmetric about some point and continuous. Hence

$$\begin{aligned} \lim _{|x|\rightarrow +\infty }|x|^{N-2}u(x)=U(0)>0, \end{aligned}$$

which completes the proof of Proposition 2.4. \(\square \)

Lemma 2.5

Let u be a solution of Eq. (1.6), then there exist \(\lambda >0\) and \(x\in {\mathbb {R}}^N\) such that

$$\begin{aligned} u(y)=\left( \frac{\lambda }{|y-x|}\right) ^{N-2}u\left( x+\frac{\lambda ^2(y-x)}{|y-x|^2}\right) . \end{aligned}$$
(2.24)

Proof

Let u be a solution of Eq. (1.6). By Proposition 2.3, we can assume that u(x) is symmetric about the origin, and we prove this lemma with \(x=0\). Moreover, without loss of generality we assume that \(\lambda =1\). Otherwise, we just need to make a translation or a scaling.

By Proposition 2.4, suppose that \(\lim \limits _{|x|\rightarrow +\infty }|x|^{N-2}u(x)=u_\infty =u(0)\). Let e be any unit vector in \({\mathbb {R}}^N\). We define

$$\begin{aligned} w(y)=\left( \frac{1}{|y|}\right) ^{N-2}u\left( \frac{y}{|y|^2}-e\right) . \end{aligned}$$

Obviously, w(y) is the Kelvin transform of \(u(y-e)\). By Lemma 2.1, w satisfies the Eq. (1.6) and hence should be radially symmetric about some point \(z_0\in {\mathbb {R}}^N\). Note that

$$\begin{aligned} w(0)=u_\infty \quad \text {and} \quad w(e)=u(0). \end{aligned}$$

Thus, w must be symmetric about the plane \(\Pi =\{x: (x-\frac{e}{2})\cdot e=0\}\). Now, choosing \(y=\frac{e}{2}-he\) for any \(h>0\), similarly to the proof of Lemma 3.1 in [11], we can prove that

$$\begin{aligned} w\left( \frac{e}{2}-he\right) =\left( \frac{1}{|\frac{1}{2}-h|}\right) ^{N-2} u\left( \frac{\frac{e}{2}+he}{|\frac{1}{2}-h|}\right) . \end{aligned}$$
(2.25)

Taking \(y=\frac{e}{2}+he\), \(h>0\), we have

$$\begin{aligned} w\left( \frac{e}{2}+he\right) = \left( \frac{1}{|\frac{1}{2}+h|}\right) ^{N-2}u\left( \frac{\frac{e}{2}-he}{|\frac{1}{2}+h|}\right) . \end{aligned}$$
(2.26)

Combining (2.25) with (2.26) and noticing the radial symmetry of u, we find

$$\begin{aligned} \left( \frac{1}{|\frac{1}{2}-h|}\right) ^{N-2} u\left( \left| \frac{\frac{1}{2}+h}{\frac{1}{2}-h}\right| e\right) = \left( \frac{1}{|\frac{1}{2}+h|}\right) ^{N-2} u\left( \left| \frac{\frac{1}{2}-h}{\frac{1}{2}+h}\right| e\right) . \end{aligned}$$

Let \(t=(\frac{1}{2}-h)/(\frac{1}{2}+h)\), then

$$\begin{aligned} u\left( \frac{e}{|t|}\right) =|t|^{N-2}u(|t| e). \end{aligned}$$

Replacing |t|, e by 1 / |y|, y / |y|, respectively, we obtain

$$\begin{aligned} u(y)=\frac{1}{|y|^{N-2}}u\left( \frac{y}{|y|^2}\right) . \end{aligned}$$

Furthermore, we can take a translation transform to obtain (2.24). \(\square \)

To prove Theorem 1.1, we also need the following proposition from Li and Zhang [19]. Earlier version with stronger assumptions was first proved by Li and Zhu [20].

Proposition 2.6

[19] Let \(f\in \mathcal {C}^1({\mathbb {R}}^N,{\mathbb {R}})\), \(\lambda >0\) and \(\mu >0\). Suppose that for every \(x\in {\mathbb {R}}^N\), there exists \(\lambda (x)>0\) such that

$$\begin{aligned} f(y)=\left( \frac{\lambda }{|y-x|}\right) ^\mu f\left( x+\frac{\lambda ^2(y-x)}{|y-x|^2}\right) , \quad y\in {\mathbb {R}}^N{\setminus } \{x\}. \end{aligned}$$

Then,

$$\begin{aligned} f(x)\equiv \pm a\left( \frac{1}{d+|x-\bar{x}|^2}\right) ^{\mu /2} \end{aligned}$$

for some \(a\ge 0\), \(d>0\) and \(\bar{x}\in {\mathbb {R}}^N\).

Proof of Theorem 1.1

Using Lemma 2.5 and Proposition 2.6, we obtain that the solution of Eq. (1.6) must be of form (1.7). \(\square \)

3 A global compactness result

In this section, we study the behavior of Palais–Smale sequences of the energy functional I and then prove Theorem 1.3. The following result is a Brézis–Lieb’s type lemma for problem (1.10), and the proof is similar as Lemma 2.4 in [31].

Lemma 3.1

Let \(N\ge 3\) and \(\alpha \in (0,N)\). If \(\{u_n\}\) is a bounded sequence in \(L^{\frac{2N}{N-2}}({\mathbb {R}}^N)\) such that \(u_n\rightarrow u\) almost everywhere in \({\mathbb {R}}^N\) as \(n\rightarrow +\infty \), then

$$\begin{aligned} \int _{{\mathbb {R}}^N}\big (I_{\alpha }*|u_n|^{2_{\alpha }^{*}}\big )|u_n|^{2_{\alpha }^{*}}dx- \int _{{\mathbb {R}}^N}\big (I_{\alpha }*|u_n-u|^{2_{\alpha }^{*}}\big )|u_n-u|^{2_{\alpha }^{*}}dx\rightarrow \int _{{\mathbb {R}}^N}\big (I_{\alpha }*|u|^{2_{\alpha }^{*}}\big )|u|^{2_{\alpha }^{*}}dx \end{aligned}$$
(3.1)

and

$$\begin{aligned}&\big (I_{\alpha }*|u_n|^{2_{\alpha }^{*}}\big )|u_n|^{2_{\alpha -2}^{*}}u_n- \big (I_{\alpha }*|u_n-u|^{2_{\alpha }^{*}}\big )|u_n-u|^{2_{\alpha }^{*}-2}(u_n-u)\rightharpoonup \big (I_{\alpha }*|u|^{2_{\alpha }^{*}}\big )|u|^{2_{\alpha }^{*}-2}u,\nonumber \\&\quad \text {in}\,\,\big (\mathcal {D}^{1,2}({\mathbb {R}}^N)\big )', \end{aligned}$$
(3.2)

where \(\big (\mathcal {D}^{1,2}({\mathbb {R}}^N)\big )'\) is the dual space of \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\).

In order to prove Theorem 1.3, we need the following concentration principle for Riesz potential.

Lemma 3.2

Let \(\{u_n\} \subset \mathcal {D}^{1,2}({\mathbb {R}}^N)\) be a sequence of functions such that

$$\begin{aligned} u_n \rightharpoonup 0 ~~\text {weakly in}~~ \mathcal {D}^{1,2}({\mathbb {R}}^N). \end{aligned}$$

Assume that there exist a bounded open set \(Q\subset {\mathbb {R}}^N\) and a positive constant \(\varrho >0\) such that

$$\begin{aligned} \int _{Q}|\nabla u_n|^2dx\ge \varrho \end{aligned}$$
(3.3)

and

$$\begin{aligned} \int _{Q}(I_{\alpha } * |u_n|^{2_\alpha ^*})|u_n|^{2_{\alpha }^{*}}dx\ge \varrho . \end{aligned}$$
(3.4)

Moreover, suppose that

$$\begin{aligned} \Delta u_n+ (I_{\alpha }* |u_n|^{2_{\alpha }^{*}})|u_n|^{2_{\alpha }^{*}-2}u_n=\chi _n, \end{aligned}$$
(3.5)

where \(\chi _n \in \big (\mathcal {D}^{1,2}({\mathbb {R}}^N)\big )'\) and

$$\begin{aligned} \langle \chi _n, \psi \rangle \le \varepsilon _n \Vert \psi \Vert , \quad \text {for all} \quad \psi \in C_{0}^{\infty }(\Omega ), \end{aligned}$$
(3.6)

with \(\Omega \) being an open neighborhood of Q and \(\{\varepsilon _n\}\) being a sequence of positive numbers converging to 0. Then there exist a sequence of positive numbers \(\{\sigma _n\}\) and a sequence of points \(\{y_n\}\subset \bar{Q}\) such that

$$\begin{aligned} v_{n}(x):=\sigma _{n}^{\frac{N-2}{2}}u_{n}(\sigma _n x+ y_n) \end{aligned}$$

converges weakly in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) to v, which is a nontrivial solution of Eq. (1.6).

Proof

Since \(u_n \rightharpoonup 0\) in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\), then by Concentration Compactness Principle II (see Lemma I.1, [26]), we obtain an at most countable index set \(\Gamma \), a sequence of \(\{ x_i\}_{i\in \Gamma }\subset {\mathbb {R}}^N \) and a family of \(\{\nu _i\}_{i\in \Gamma }\subset (0,+\infty )\) such that

$$\begin{aligned} |u_n\phi _{\Omega }|^{2^*} \rightharpoonup \sum _{i\in \Gamma }\nu _{i}\delta _{x_i}, \end{aligned}$$

where \(\phi _{\Omega }(x)\) is a cut-off function with \(\phi _{\Omega }(x)=1\) in Q; \(\phi _{\Omega }(x)=0\) in \({\mathbb {R}}^N{\setminus } \Omega \) and \(0\le \phi _{\Omega }(x)\le 1\).

For the readers’ convenience, we will prove this lemma through three claims.

Claim 1

There is at least one \(i_0\in \Gamma \) such that \(x_{i_0}\in \bar{Q}\) with \(\nu _{i_0}>0\).

Proof

Otherwise, then \(u_n\rightarrow 0\) in \(L^{2^*}(Q)\), which together with Hardy–Littlewood–Sobolev inequality implies that

$$\begin{aligned} \int _{Q}\big (I_{\alpha } * |u_n|^{2_\alpha ^*}\big )|u_n|^{2_{\alpha }^{*}}dx\rightarrow 0. \end{aligned}$$

This is a contradiction to the assumption (3.4) and the claim is proved. \(\square \)

Now, we define the concentration function

$$\begin{aligned} G_{n}(r):=\sup _{z\in \bar{Q}}\int _{B_{r}(z)}|u_n|^{2^*}dx. \end{aligned}$$

For a given small \(\tau \in \big (0, \big [\frac{S}{A_{\alpha } C(N,\alpha )}\big ]^{\frac{N}{\alpha +2}}\big )\), we choose \(\sigma _n=\sigma _n(\tau )>0\), \(y_n \in \bar{Q}\) such that

$$\begin{aligned} \int _{B_{\sigma _n}(y_n)}|u_n|^{2^*}dx=G_n(\sigma _n)=\tau . \end{aligned}$$
(3.7)

Let \(v_n(x):=\sigma _{n}^{\frac{N-2}{2}}u_{n}(\sigma _n x+ y_n)\), then

$$\begin{aligned} \widetilde{G_n}(r):=\sup _{z\in \bar{Q}_{n}}\int _{B_{r}(z)}|v_n|^{2^*}dx=\sup _{z\in \bar{Q}} \int _{B_{\sigma _n r}(z)}|u_n|^{2^*}dx=G_n(\sigma _n r), \end{aligned}$$
(3.8)

where \(\bar{Q}_n:=\{x\in {\mathbb {R}}^N: \sigma _n x+y_n \in \bar{Q}\}\). It follows by (3.7) and (3.8) that

$$\begin{aligned} \widetilde{G_n}(1)=\int _{B_{1}(0)}|v_n|^{2*}dx=\int _{B_{\sigma _n}(y_n)}|u_n|^{2^*}dx=G_{n}(\sigma _n)=\tau . \end{aligned}$$
(3.9)

Claim 2

There exists some \(\tau \in \big (0, \big [\frac{S}{A_{\alpha } C(N,\alpha )}\big ]^{\frac{N}{\alpha +2}}\big )\) such that \(\sigma _{n}(\tau )\rightarrow 0\) as \(n\rightarrow +\infty \).

Proof

Assume by contradiction, for any \(\varepsilon >0\), that there exists \(r_0>0\) such that \(\sigma _n(\varepsilon )\ge r_0\). Then a direct calculation shows that

$$\begin{aligned} \int _{B_{r_0}(x_{i_0})}|u_n|^{2^*}dx \le \sup _{z\in \bar{Q}}\int _{B_{\sigma _n(\varepsilon )}(z)}|u_n|^{2^*}dx=G_{n}(\sigma _n(\varepsilon ))=\varepsilon . \end{aligned}$$
(3.10)

In particular

$$\begin{aligned} \nu _{i_0}\le \int _{B_{r_0}(x_{i_0})}|u_n|^{2^*}dx +o_{n}(1)\le \varepsilon +o_{n}(1), \quad \text {for any} \quad \varepsilon >0, \end{aligned}$$
(3.11)

where \(o_{n}(1)\rightarrow 0\) as \(n\rightarrow +\infty \). Then, it follows by (3.11) that we have \(\nu _{i_0}\le 0\), which contradicts Claim 1.

By the definition of \(v_n\), we have \(\int _{{\mathbb {R}}^N}|\nabla v_n|^2dx=\int _{{\mathbb {R}}^N}|\nabla u_n|^2dx\), which together with the boundness of \(\{u_n\}\) in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) implies that \(\{v_n\}\) is bounded in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\). Without loss of generality, we may assume that there exists some \(v\in \mathcal {D}^{1,2}({\mathbb {R}}^N)\) such that \(v_n\rightharpoonup v\) in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) up to a subsequence.

Claim 3

v is a nontrivial solution of Eq. (1.6).

Proof

In fact, for any \(\varphi \in C_{0}^{\infty }({\mathbb {R}}^N)\), we define

$$\begin{aligned} \widetilde{\varphi }_{n}(x)=\sigma _n^{\frac{2-N}{2}}\varphi \left( \frac{x-y_n}{\sigma _n}\right) . \end{aligned}$$
(3.12)

Since \(\sigma _n\rightarrow 0\) and \(y_n \in \bar{Q}\), then we assert that \(\widetilde{\varphi }_{n}(x)\in C_{0}^{\infty }(\bar{\Omega })\) for n large enough. In virtue of (3.5) and (3.6), we obtain that

$$\begin{aligned} o_{n}(1)\Vert \varphi \Vert =o_n(1)\Vert \widetilde{\varphi }_{n}\Vert&=\displaystyle \int _{{\mathbb {R}}^N}\nabla u_n \nabla \widetilde{\varphi }_{n} dx -\int _{{\mathbb {R}}^N}\big (I_{\alpha }*|u_n|^{2_{\alpha }^{*}}\big )|u_n|^{2_{\alpha }^{*}-2}u_n\widetilde{\varphi }_{n}dx \nonumber \\&=\displaystyle \int _{{\mathbb {R}}^N}\nabla v_n \nabla \varphi dx -\int _{{\mathbb {R}}^N}\big (I_{\alpha }*|v_n|^{2_{\alpha }^{*}}\big )|v_n|^{2_{\alpha }^{*}-2}v_n\varphi dx. \end{aligned}$$
(3.13)

Thus, v is a weak solution of Eq. (1.6). Before concluding the proof, we still need to prove \(v\ne 0\). To this end, it is sufficient to prove that, up to a subsequence,

$$\begin{aligned} v_n\rightarrow v \quad \text {strongly in} \quad L^{2^*}(B_1(0)). \end{aligned}$$
(3.14)

Since \(v_n\rightharpoonup v\) in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\), by Concentration Compactness Principle (see Lemma I.1 in [26] and Lemma 2.1 in [27]), we may assume that there exist three bounded nonnegative measures \(\widetilde{\mu }\), \(\widetilde{\nu }\), \(\widetilde{\omega }\), such that \(|\nabla v_n|^2 \rightharpoonup \widetilde{\mu }\), \(|v_n|^{2^*}\rightharpoonup \widetilde{\nu }\) and \(\big |I_{\alpha }* |v_n|^{2_{\alpha }^{*}}\big |^{\frac{2N}{N-\alpha }}\rightharpoonup \widetilde{\omega }\) weakly in finite measure space \(\mathcal {M}({\mathbb {R}}^N)\) (see Page 26 in [40]). Moreover,

$$\begin{aligned} \widetilde{\mu }\ge |\nabla v|^2 + \displaystyle \sum _{j\in \widetilde{\Gamma }}\widetilde{\mu }_{j}\delta _{x_j},\, \, \, \widetilde{\nu }=|v|^{2^*}+ \displaystyle \sum _{j\in \widetilde{\Gamma }}\widetilde{\nu }_{j}\delta _{x_j},\, \, \, \widetilde{\omega }=\big |I_{\alpha }* |v|^{2_{\alpha }^{*}}\big |^{\frac{2N}{N-\alpha }}+ \displaystyle \sum _{j\in \widetilde{\Gamma }}\widetilde{\omega }_{j}\delta _{x_j} \quad \text {in}\ \mathcal {M}({\mathbb {R}}^N) \end{aligned}$$
(3.15)

and

$$\begin{aligned} \widetilde{\mu }_{j} \ge S\widetilde{\nu }_{j}^{\frac{N-2}{N}}, \quad \widetilde{\nu }_{j}\ge \left( \frac{1}{A_\alpha C(N,\alpha )}\right) ^{\frac{2N}{N+\alpha }}\widetilde{\omega }_j^{\frac{N-\alpha }{N+\alpha }}, \end{aligned}$$
(3.16)

where \(\widetilde{\Gamma }\) is an at most countable index set. In order to prove (3.14), we only need to prove

$$\begin{aligned} \{x_j\}_{j\in \widetilde{\Gamma }}\cap \overline{B_1(0)}=\emptyset . \end{aligned}$$

If not, we suppose that there exists \(x_{j_0}\in \overline{B_{1}(0)}\) for some \(j_0 \in \widetilde{\Gamma }\) and define \(\phi _{\rho }(x):=\phi \big (\frac{x-x_{j_0}}{\rho }\big )\), \(\phi \) is a cut-off function which satisfies \(\phi =1\) on \(B_{1}(0)\), \(supp \phi \subset B_{2}(0)\) and \(0\le \phi \le 1\). Denote by \(\widetilde{\phi }_{\rho ,n}(x)=\phi _{\rho }\big (\frac{x-y_n}{\sigma _n}\big )\), by the facts that \(y_n\in \bar{Q}\), \(x_{j_0}\in \overline{B_{1}(0)}\) and \(\sigma _n\rightarrow 0\), we then observe that \(supp \widetilde{\phi }_{\rho ,n}(x) \subset B_{2\sigma _n\rho }(y_n+\sigma _n x_{j_0})\subset \Omega \), which implies \(\widetilde{\phi }_{\rho ,n}(x) u_n \in \mathcal {D}_{0}^{1,2}(\Omega )\). A direct calculation yields that

$$\begin{aligned} \displaystyle \int _{{\mathbb {R}}^N}|\nabla (\widetilde{\phi }_{\rho ,n}u_n )|^{2}dx&\le \displaystyle C\int _{{\mathbb {R}}^N}|\nabla \widetilde{\phi }_{\rho ,n}|^2 u_{n}^{2}dx+\displaystyle C\int _{{\mathbb {R}}^N} |\widetilde{\phi }_{\rho ,n}|^2 |\nabla u_{n}|^{2}dx \nonumber \\&\le C\left( \displaystyle \int _{B_{2\sigma _n \rho }(y_n+\sigma _n x_{i_0})}|\nabla \widetilde{\phi }_{\rho ,n}|^{N}dx\right) ^{\frac{2}{N}}\cdot \left( \displaystyle \int _{{\mathbb {R}}^N}|u_n|^{2^*}dx\right) ^{\frac{N-2}{N}} \nonumber \\&\quad +C \displaystyle \int _{{\mathbb {R}}^N}|\nabla u_n|^2dx \nonumber \\&\le C. \end{aligned}$$
(3.17)

Hence, \(\{ \widetilde{\phi }_{\rho ,n} u_n\}\) is bounded in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) and the bound is independent of \(\rho \). Combining (3.5), (3.6) with the fact that \(C_{0}^{\infty }({\mathbb {R}}^N)\) is dense in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\), we then get

$$\begin{aligned}&\int _{{\mathbb {R}}^N}\nabla v_n \nabla (\phi _{\rho } v_n)dx -\int _{{\mathbb {R}}^N}(I_{\alpha }* |v_n|^{2_{\alpha }^{*}})|v_n|^{2_{\alpha }^{*}-2}v_n(\phi _{\rho }v_n)dx \nonumber \\&\quad = \displaystyle \int _{{\mathbb {R}}^N}\nabla u_n \cdot \nabla (\widetilde{\phi }_{\rho ,n} u_n)dx -\int _{{\mathbb {R}}^N}(I_{\alpha }*|u_n|^{2_{\alpha }^{*}})|u_n|^{2_{\alpha }^{*}-2} u_n (\widetilde{\phi }_{\rho ,n}u_n)dx =o_n(1). \end{aligned}$$
(3.18)

Let \(\rho \rightarrow 0\), then

$$\begin{aligned} \displaystyle \limsup _{n\rightarrow \infty } \left| \int _{{\mathbb {R}}^N}(\nabla v_n \cdot \nabla \phi _{\rho })v_ndx\right|&\le \displaystyle \limsup _{n\rightarrow \infty }\left( \int _{{\mathbb {R}}^N}|\nabla v_n|^2 dx\right) ^{\frac{1}{2}}\left( \int _{B_{2\rho }(x_{j_0})}v_{n}^{2}|\nabla \phi _{\rho }|^{2} dx \right) ^{\frac{1}{2}} \nonumber \\&\le C \displaystyle \left( \int _{B_{2\rho }(x_{j_0})}|\nabla \phi _{\rho }|^{N}dx\right) ^{\frac{1}{N}} \left( \int _{B_{2\rho }(x_{j_0})}| v_n|^{2^*}dx\right) ^{\frac{1}{2^*}}\rightarrow 0. \end{aligned}$$
(3.19)

Moreover,

$$\begin{aligned} \displaystyle \limsup _{n\rightarrow \infty }\int _{{\mathbb {R}}^N}|\nabla v_n|^2 {{\phi _{\rho }}}dx \ge \int _{{\mathbb {R}}^N}|\nabla v|^2\phi _{\rho }dx+\widetilde{\mu }_{j_0}\rightarrow \widetilde{\mu }_{j_0} \end{aligned}$$
(3.20)

and

$$\begin{aligned} \displaystyle \int _{{\mathbb {R}}^N}(I_{\alpha }* |v_n|^{2_{\alpha }^{*}})|v_n|^{2_{\alpha }^{*}}\phi _{\rho } dx&\le \displaystyle \left( \int _{supp \phi } (I_{\alpha }*|v_n|^{2_{\alpha }^{*}})^{\frac{2N}{N-\alpha }}dx \right) ^{\frac{N-\alpha }{2N}} \left( \int _{supp \phi } |v_n|^{2^{*}}dx\right) ^{\frac{N+\alpha }{2N}}\nonumber \\&\rightarrow \widetilde{\omega }_{j_0}^{\frac{N-\alpha }{2N}}\widetilde{\nu }_{j_0}^{\frac{N+\alpha }{2N}} \le A_{\alpha } C(N,\alpha )\widetilde{\nu }_{j_0}^{\frac{N+\alpha }{N}}. \end{aligned}$$
(3.21)

It follows from (3.18)–(3.20) that

$$\begin{aligned} S\widetilde{\nu }_{j_0}^{\frac{N-2}{N}} \le \widetilde{\mu }_{j_0}\le A_{\alpha } C(N,\alpha )\widetilde{\nu }_{j_0}^{\frac{N+\alpha }{N}}, \end{aligned}$$

then

$$\begin{aligned} \nu _{j_0}\ge \left[ \frac{S}{A_{\alpha } C(N,\alpha )}\right] ^{\frac{N}{\alpha +2}}. \end{aligned}$$

Combining the inequality above and (3.9), then we get

$$\begin{aligned} \left[ \frac{S}{A_{\alpha } C(N,\alpha )}\right] ^{\frac{N}{\alpha +2}}\le \nu _{j_0}\le \int _{B_{1}(0)}|v_{n}|^{2^*}dx=\tau , \end{aligned}$$

which contradicts the assumption \(\tau \in \big (0, \left[ \frac{S}{A_{\alpha } C(N,\alpha )}\big ]^{\frac{N}{\alpha +2}}\right) \). Therefore, (3.14) is proved. Combining (3.9) and (3.14), we have

$$\begin{aligned} \int _{B_{1}(0)}|v|^{2^*}dx=\lim _{n\rightarrow +\infty }\int _{B_1(0)}|v_n|^{2^*}dx=\tau >0, \end{aligned}$$

which implies that \(v\ne 0\). Thus, combining Claims 13, we can complete the proof. \(\square \)

Lemma 3.3

Let \(\{u_n\}\) be a Palais–Smale sequence for \(I_\infty \), such that \(u_n \in C_{0}^{\infty }({\mathbb {R}}^N)\) and

$$\begin{aligned} u_{n}\rightharpoonup 0 \quad \text {weakly in}~~ \mathcal {D}^{1,2}({\mathbb {R}}^N); ~~ u_{n}\nrightarrow 0 ~~\text {strongly in}~~ \mathcal {D}^{1,2}({\mathbb {R}}^N). \end{aligned}$$

Then there exist a sequence of points \(\{y_n\}\subset {\mathbb {R}}^N\), a sequence of positive numbers \(\{\sigma _n\}\) such that

$$\begin{aligned} v_{n}(x):=\sigma _{n}^{\frac{N-2}{2}}u_{n}(\sigma _nx+y_n) \end{aligned}$$

converges weakly in \(\mathcal {\mathcal {D}}^{1,2}({\mathbb {R}}^N)\) to v, which is a nontrivial solution of Eq. (1.6). Moreover,

$$\begin{aligned} I_\infty (u_n)= & {} I_{\infty }(v)+I_{\infty }(v_n-v)+o_n(1); \end{aligned}$$
(3.22)
$$\begin{aligned} \Vert u_n\Vert ^{2}= & {} \Vert v\Vert ^{2}+\Vert v_n-v\Vert ^{2}+o_n(1). \end{aligned}$$
(3.23)

Proof

Since \(u_n\rightharpoonup 0\) in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\), then \(\{u_n\}\) is bounded in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\). Furthermore, as \(\{u_n\}\) is a Palais–Smale sequence for \(I_\infty \), we then know that

$$\begin{aligned} \Delta u_n-( I_\alpha *|u_n|^{2_\alpha ^*} )|u_n|^{2_\alpha ^*-2} u_n=\chi _n, \end{aligned}$$
(3.24)

where \(\chi _n \in \big (\mathcal {D}^{1,2}({\mathbb {R}}^N)\big )'\) satisfies

$$\begin{aligned} \langle \chi _n, \psi \rangle \le \varepsilon _n \Vert \psi \Vert , \quad \text {for all}~~ \psi \in C_{0}^{\infty }(\Omega ). \end{aligned}$$
(3.25)

Multiplying by \(u_n\) on both sides of (3.24) and integrating on \({\mathbb {R}}^N\), we then have

$$\begin{aligned} \int _{{\mathbb {R}}^N}|\nabla u_n|^2dx =\int _{{\mathbb {R}}^N}\big (I_{\alpha }*|{u}_{n}|^{2_{\alpha }^{*}}\big )|{u}_{n}|^{2_{\alpha }^{*}}dx+o_n(1). \end{aligned}$$
(3.26)

Let us decompose \({\mathbb {R}}^N\) in N-dimensional hypercubes \(Q_i\) with unitary sides and vertices with integer coordinates. Next, we assert that for any \(n\in {\mathbb N}\), there exists some \(\widetilde{\varrho }>0\) satisfying

$$\begin{aligned} d_{n}:= \sup _{Q_{i}} \int _{Q_{i}}\big (I_{\alpha }*|{u}_{n}|^{2_{\alpha }^{*}}\big )|{u}_{n}|^{2_{\alpha }^{*}}dx \ge {\widetilde{\varrho }}. \end{aligned}$$

If not, then we have \(d_n \rightarrow 0\) as \(n\rightarrow +\infty \). A direct calculation shows that

$$\begin{aligned} \int _{{\mathbb {R}}^N}\big (I_{\alpha }*|{u}_{n}|^{2_{\alpha }^{*}}\big )|{u}_{n}|^{2_{\alpha }^{*}}dx&\le d_n^{1-\frac{1}{2_{\alpha }^{*}}}\sum _{i} \left( \int _{Q_{i}}(I_{\alpha }*|{u}_{n}|^{2_{\alpha }^{*}})|{u}_{n}|^{2_{\alpha }^{*}}dx\right) ^{\frac{1}{2_{\alpha }^{*}}}\nonumber \\&\le d_n^{1-\frac{1}{2_{\alpha }^{*}}} \big (C(N,\alpha )A_{\alpha }\big )^{\frac{1}{2_{\alpha }^{*}}}\sum _{i}\left( \int _{Q_i}|{u}_n|^{2^*}dx\right) ^{\frac{1}{2^*}} \left( \int _{{\mathbb {R}}^N}|{u}_n|^{2^*}dx \right) ^{\frac{1}{2^*}}\nonumber \\&\le C d_n^{1-\frac{1}{2_{\alpha }^{*}}} \big (C(N,\alpha )A_{\alpha }\big )^{\frac{1}{2_{\alpha }^{*}}}\Vert {u}_{n}\Vert ^2. \end{aligned}$$
(3.27)

Combining (3.26) with (3.27) and letting \(d_n\rightarrow 0\) as \(n\rightarrow +\infty \), we observe that \(\Vert {u}_n\Vert \rightarrow 0\), which leads to a contradiction.

In the following, let \(\widetilde{y}_n\) be the center of a hypercube \(Q_i\) such that

$$\begin{aligned} \int _{Q_i}(I_{\alpha }* |u_n|^{2_{\alpha }^{*}})|u_n|^{2_{\alpha }^{*}}dx \ge \frac{\widetilde{\varrho }}{2}>0. \end{aligned}$$
(3.28)

Set \(w_n=u_n(x+\widetilde{y}_{n})\), then

$$\begin{aligned} \int _{Q}(I_{\alpha }* |w_n|^{2_{\alpha }^{*}})|w_n|^{2_{\alpha }^{*}}dx \ge \frac{\widetilde{\varrho }}{2}>0, \end{aligned}$$
(3.29)

where Q denote a hypercube of unitary side centered at the origin. Using the Hardy–Littlewood–Sobolev inequality and the boundedness of \(\{u_n\}\) in \(\mathcal D^{1,2}({\mathbb {R}}^N)\) again, we get

$$\begin{aligned} \frac{\widetilde{\varrho }}{2} \le \int _{Q}(I_{\alpha }* |w_n|^{2_{\alpha }^{*}})|w_n|^{2_{\alpha }^{*}}dx \le C\left( \int _{Q}|w_n|^{2^*}dx\right) ^{\frac{N+\alpha }{N}}. \end{aligned}$$

Hence we can deduce that there exists \(\bar{\varrho }>0\) such that

$$\begin{aligned} \int _{Q}|w_n|^{2^*}dx> \bar{\varrho } . \end{aligned}$$

At this point, we have verified the conditions (3.3)–(3.5) in Lemma 3.2 for \(\{w_n\}\). The first part of Lemma 3.3 follows from Lemma 3.2. Obviously,

$$\begin{aligned} \int _{{\mathbb {R}}^N}|\nabla u_n|^2dx=\int _{{\mathbb {R}}^N}|\nabla v_n|^2dx, \quad \int _{{\mathbb {R}}^N}\big (I_\alpha *|u_n|^{2_\alpha ^*}\big ) |u_n|^{2_\alpha ^*}dx =\int _{{\mathbb {R}}^N}\big (I_\alpha *|v_n|^{2_\alpha ^*}\big ) |v_n|^{2_\alpha ^*}dx. \end{aligned}$$

Then we can prove (3.23). Similarly, (3.22) follows from (3.1). \(\square \)

It follows from Theorems A and 1.1 that

$$\begin{aligned} S_\alpha :=\inf _{\mathcal {D}^{1,2}({\mathbb {R}}^N){\setminus } \{0\}}\frac{\int _{{\mathbb {R}}^N} |\nabla u|^{2}dx}{\left( \int _{{\mathbb {R}}^N}(I_\alpha *|u|^{2_{\alpha }^*})|u(x)|^{2_{\alpha }^*} dx\right) ^{\frac{N-2}{N+\alpha }}} =\frac{S}{\left[ C(N,\alpha )A_\alpha \right] ^{\frac{N-2}{N+\alpha }}}. \end{aligned}$$
(3.30)

Now, we are ready to prove Theorem 1.3.

Proof of Theorem 1.3

Since \(\{u_n\}\) is a Palais–Smale sequence for I at level c, then it is easy to prove that \(\{u_n\}\) is bounded in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) and consequently bounded in \(L^{2^*}({\mathbb {R}}^N)\). Without loss of generality, we may assume that \(u_n\rightharpoonup \bar{u}\) in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) and \(L^{2^*}({\mathbb {R}}^N)\) as \(n\rightarrow +\infty \). Moreover, \(\bar{u}\) is a weak solution of Eq. (1.10). In fact, for any \(\varphi _1 \in C_{0}^{\infty }({\mathbb {R}}^N)\), we have

$$\begin{aligned} \langle I'(\bar{u}),\varphi _1\rangle&=\displaystyle \langle I'(u_n), \varphi _1\rangle +\int _{{\mathbb {R}}^N} V(x)(\bar{u}-u_n)\varphi _1dx+\int _{{\mathbb {R}}^N}\nabla (\bar{u}-u_n)\nabla \varphi _1dx \nonumber \\&\quad \displaystyle -\int _{{\mathbb {R}}^N}(I_{\alpha }* |\bar{u}|^{2_{\alpha }^{*}})|\bar{u}|^{2_{\alpha }^{*}-2}\bar{u}\varphi _1dx+\int _{{\mathbb {R}}^N}(I_{\alpha }* |u_n|^{2_{\alpha }^{*}})|u_n|^{2_{\alpha }^{*}-2}u_{n}\varphi _1dx. \end{aligned}$$
(3.31)

By Lemma 3.1, we know

$$\begin{aligned} \int _{{\mathbb {R}}^N}(I_{\alpha }* |u_n|^{2_{\alpha }^{*}})|u_n|^{2_{\alpha }^{*}-2}u_{n}\varphi _1dx-\int _{{\mathbb {R}}^N}(I_{\alpha }* |\bar{u}|^{2_{\alpha }^{*}})|\bar{u}|^{2_{\alpha }^{*}-2}\bar{u}\varphi _1 dx=o_n(1). \end{aligned}$$
(3.32)

Moreover, by Lemma 2.13 [40], we have

$$\begin{aligned} \int _{{\mathbb {R}}^N}V(x)(\bar{u}-u_{n})^{2}dx \rightarrow 0, \quad \text {as}~~ n\rightarrow +\infty \end{aligned}$$

and

$$\begin{aligned} \int _{{\mathbb {R}}^N} V(x)(\bar{u}-u_n)\varphi _1 dx \rightarrow 0; \quad \int _{{\mathbb {R}}^N}\nabla (\bar{u}-u_n)\nabla \varphi _1dx \rightarrow 0, \quad \text {as}~~ n\rightarrow +\infty . \end{aligned}$$
(3.33)

Thus, it follows by (3.31)–(3.33) that

$$\begin{aligned} \langle I'(\bar{u}), \varphi _1\rangle = \langle I'(u_n),\varphi _1 \rangle +o_n(1), \end{aligned}$$

which leads to \(I'(\bar{u})=0\), \(I(\bar{u})=I(u_n)-I_\infty (u_n-\bar{u})+o_n(1)\).

Let \(z_{n}^{1}:=u_n-\bar{u}\), then \(z_{n}^{1}\rightharpoonup 0\) in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) and \(\{z_{n}^{1}\}\) is a Palais–Smale sequence for \(I_\infty \). In fact, for any \(\varphi _{2}\in C_{0}^{\infty }({\mathbb {R}}^N)\), we have

$$\begin{aligned} \langle I'_\infty (z_{n}^{1}), \varphi _{2}\rangle&=\displaystyle \langle I'(u_n),\varphi _{2}\rangle - \displaystyle \langle I'(\bar{u}), \varphi _{2}\rangle +\int _{{\mathbb {R}}^N}V(x)(\bar{u}-u_n)\varphi _{2}dx\\&\quad -\int _{{\mathbb {R}}^N}(I_{\alpha }* |\bar{u}|^{2_{\alpha }^{*}})|\bar{u}|^{2_{\alpha }^{*}-2}\bar{u}\varphi _{2}dx\\&\quad \displaystyle -\int _{{\mathbb {R}}^N}(I_{\alpha }* |z_{n}^{1}|^{2_{\alpha }^{*}})|z_{n}^{1}|^{2_{\alpha }^{*}-2}z_{n}^{1}\varphi _{2}dx +\int _{{\mathbb {R}}^N}(I_{\alpha }* |u_n|^{2_{\alpha }^{*}})|u_n|^{2_{\alpha }^{*}-2}u_{n}\varphi _{2}dx\\&=o_n(1)\Vert \varphi _{2}\Vert , \end{aligned}$$

where (3.33) and (3.2) are used. Hence \(\{z_{n}^{1}\}\) is a Palais–Smale sequence of \(I_\infty \).

For any \(n\in {\mathbb N}^{+}\), there exists a sequence \(\{u_{n}^{1}\}\subset C_{0}^{\infty }({\mathbb {R}}^N)\) such that

$$\begin{aligned} \Vert u_{n}^{1}-z_{n}^{1}\Vert< \frac{1}{n}~~ \text {and}~~ \Vert I_\infty '(u_{n}^{1})-I_\infty '(z_{n}^{1})\Vert <\frac{1}{n}. \end{aligned}$$
(3.34)

It is not difficult to verify that

$$\begin{aligned} \Vert u_n^1\Vert ^2=\Vert z_n^1\Vert ^2+o_n(1); \quad I_\infty (u_n^1)=I_\infty (z_n^1)+o_n(1); \quad I_\infty '(u_n^1)=I_\infty '(z_n^1)+o_n(1).\quad \end{aligned}$$
(3.35)

Furthermore, one has

$$\begin{aligned} \Vert u_{n}^{1}\Vert ^{2}=\Vert z_n^1\Vert ^2+o_n(1)=\Vert u_n\Vert ^2-\Vert \bar{u}\Vert ^2+o_n(1) \end{aligned}$$
(3.36)

and

$$\begin{aligned} I_\infty (u_n^1)=I_\infty (z_n^1)+o_n(1)=I(u_n)-I(\bar{u})+o_n(1). \end{aligned}$$
(3.37)

If \(u_n^1\rightarrow 0\) in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\), then we have done. Now we suppose that \(u_{n}^{1}\nrightarrow 0\) strongly in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\). From (3.35) that we know that \(\{u_n^1\}\) is a Palais–Smale sequence of \(I_\infty \) and \(\{u_{n}^{1}\}\subset C_{0}^{\infty }({\mathbb {R}}^N)\) satisfies

$$\begin{aligned} u_n^1 \rightharpoonup 0 ~~\text {in}~~ \mathcal {D}^{1,2}({\mathbb {R}}^N)~~\text {and}~~ u_{n}^{1}\nrightarrow 0 \text { strongly in} ~~ \mathcal {D}^{1,2}({\mathbb {R}}^N). \end{aligned}$$

Applying Lemma 3.3 to \(\{u_n^1\}\), we assert that there exist a sequence of points \(\{x_n^{1}\}\subset {\mathbb {R}}^N\), a sequence of positive numbers \(\{\eta _{n}^{1}\}\subset {\mathbb {R}}\) such that

$$\begin{aligned} v_{n}^{1}:=(\eta _{n}^{1})^{\frac{N-2}{2}}u_{n}^{1}(\eta _{n}^{1}\cdot +x_{n}^{1}) \end{aligned}$$

converges weakly in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) to a nontrivial solution \(u^{1}\) of Eq. (1.6). Moreover,

$$\begin{aligned} I_\infty (u_n^1)=I_{\infty }(u^1)+I_{\infty }(v_n^1-u^1)+o_n(1)~~\text {and}~~ \Vert u_n^1\Vert ^{2}=\Vert u^1\Vert ^2+\Vert v_n^1-u^1\Vert ^2+o_n(1). \end{aligned}$$
(3.38)

Combining (3.38) with (3.35), we obtain that

$$\begin{aligned} I(u_n)=I(\bar{u})+I_{\infty }(u^1)+I_{\infty }(v_{n}^{1}-u^1)+o_n(1) \end{aligned}$$
(3.39)

and

$$\begin{aligned} \Vert u_n\Vert ^2=\Vert \bar{u}\Vert ^2+\Vert v_n^1-u^1\Vert ^2+\Vert u^1\Vert ^2+o_n(1). \end{aligned}$$
(3.40)

Let \(z_{n}^{j}=v_{n}^{j-1}-u^{j-1}\) and repeat the above procedure. If \(z_{n}^{j}\rightarrow 0\) in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\), we have done. If \(z_{n}^{j}\nrightarrow 0\) in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\), the analogously \(\{z_{n}^{j}\}\) is a Palais–Smale sequence of \(I_\infty \), then we can find \(\{u_n^{j}\}\subset C_{0}^{\infty }({\mathbb {R}}^N)\) such that

$$\begin{aligned} \Vert u_{n}^{j}-z_{n}^{j}\Vert< \frac{1}{n}~~ \text {and}~~ \Vert I'_{\infty }(u_{n}^{j})-I'_{\infty }(z_{n}^{j})\Vert <\frac{1}{n}, \end{aligned}$$
(3.41)

and there exist a sequence of positive numbers \(\{\eta _{n}^{j}\} \subset {\mathbb {R}}\) and a sequence of points \(\{x_{n}^{j}\} \subset {\mathbb {R}}^N\) such that

$$\begin{aligned} v_{n}^{j}:=(\eta _{n}^{j})^{\frac{N-2}{2}}u^j_{n}(\eta _{n}^{j}\cdot +x_{n}^{j}) \end{aligned}$$

converges weakly in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) to a nontrivial solution \(u^j\) of Eq. (1.6). Moreover, the following properties hold:

$$\begin{aligned} I_{\infty }(v_n^j)=I_{\infty }(u^j)+I_{\infty }(v_n^j-u^j)+o_n(1)~~\text {and}~~ \Vert v_n^j\Vert ^{2}=\Vert u^j\Vert ^2+\Vert v_n^j-u^j\Vert ^2+o_n(1).\quad \end{aligned}$$
(3.42)

Furthermore, we deduce that

$$\begin{aligned} I(u_n)=I(\bar{u})+\sum _{i=1}^{j-1}I_{\infty }(u^{i})+I_{\infty }(v_{n}^{j}-u^j)+o_n(1) \end{aligned}$$
(3.43)

and

$$\begin{aligned} \Vert u_n\Vert ^2=\Vert \bar{u}\Vert ^2+\sum _{i=1}^{j-1}\Vert u^i\Vert ^2+\Vert v_{n}^{j}-u^{j}\Vert ^2+o_n(1). \end{aligned}$$
(3.44)

Since \(u^{j}\) is a nontrivial weak solution of Eq. (1.6), then \(\Vert u^j\Vert ^2\ge S_{\alpha }^{\frac{N+\alpha }{\alpha +2}}\), which together with (3.44) and the fact that \(u_n\) is bounded in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) tells us that the iteration procedure must terminate after finitely-many steps. Therefore, we complete the proof of Theorem 1.3. \(\square \)

4 Existence of positive bound state solution

In this section, we prove the existence of bound state solutions to Eq. (1.10). Firstly, we show that, providing \(V(x)\ge 0\) and \(V(x) \in L^{\frac{N}{2}}({\mathbb {R}}^N)\), then there is no minimizer for functional I restrict on the Nehari manifold \(\mathcal {N}\).

Proposition 4.1

Assume that \(V(x)\ge 0\) and \(V(x) \in L^{\frac{N}{2}}({\mathbb {R}}^N)\), then \(m=m_{\infty }\) holds and m is not attained.

Proof

Obviously, for \(u\in \mathcal {D}^{1,2}({\mathbb {R}}^N){\setminus } \{0\}\), there exist unique \(t_u\), \(s_u>0\) such that \(t_uu\in \mathcal {N}\) \(s_uu\in \mathcal {N}_{\infty }\), moreover \(I(t_uu)=\max _{t>0}I(tu)\) and \(I_{\infty }(s_uu)=\max _{s>0}I_{\infty }(su)\). Especially, if \(u\in \mathcal {N}\) and \(s_uu \in \mathcal {N}_{\infty }\), then we have \(s_u\in (0,1]\). Therefore, for \(u\in \mathcal {N}\),

$$\begin{aligned} m_{\infty }\le I_{\infty }(s_uu)&=\displaystyle \frac{s_{u}^2}{2}\Vert u\Vert ^{2}-\frac{s_{u}^{2\cdot 2_{\alpha }^{*}}}{2\cdot 2_{\alpha }^{*}}\int _{{\mathbb {R}}^N}\big (I_{\alpha }*|u|^{2_{\alpha }^{*}}\big )|u|^{2_{\alpha }^{*}}dx \nonumber \\&\le \displaystyle \frac{s_{u}^2}{2}\Vert u\Vert ^{2}+ \frac{s_{u}^2}{2}\int _{{\mathbb {R}}^N}V(x)u^2dx-\frac{s_{u}^{2\cdot 2_{\alpha }^{*}}}{2\cdot 2_{\alpha }^{*}}\int _{{\mathbb {R}}^N}\big (I_{\alpha }*|u|^{2_{\alpha }^{*}}\big )|u|^{2_{\alpha }^{*}}dx \nonumber \\&\le I(u), \end{aligned}$$
(4.1)

which implies that \(m_{\infty } \le m\).

Next, we prove \(m\le m_\infty \). In fact, we consider a sequence \(\{u_n:=t_nw_n \} \subset \mathcal {N}\), where \(w_n(\cdot )=w(\cdot -z_n)\) with w being a positive solution centered at zero to Eq. (1.6), \(\{z_n\}\subset {\mathbb {R}}^N\) satisfying \(|z_n|\rightarrow +\infty \) as \(n\rightarrow +\infty \) and \(t_n:=t_{w_n}\). It follows by the definition of \(w_n\) that

$$\begin{aligned} w_n \rightharpoonup 0 \quad \text {in} \quad \mathcal {D}^{1,2}({\mathbb {R}}^N); \quad \Vert w_n\Vert =\Vert w\Vert \ne 0 \end{aligned}$$
(4.2)

and

$$\begin{aligned} \int _{{\mathbb {R}}^N}(I_{\alpha }*|w_n|^{2_{\alpha }^{*}})|w_n|^{2_{\alpha }^{*}}dx =\int _{{\mathbb {R}}^N}(I_{\alpha }*|w|^{2_{\alpha }^{*}})|w|^{2_{\alpha }^{*}}dx, \quad \text {as}~~ n\rightarrow +\infty . \end{aligned}$$
(4.3)

Furthermore, by Lemma 2.13 [40], we know that

$$\begin{aligned} \int _{{\mathbb {R}}^N}V(x)w_n^{2}dx \rightarrow 0 ~~ \text {as}~~ n \rightarrow +\infty . \end{aligned}$$
(4.4)

Thus, in virtue of (4.2)–(4.4), we can prove easily that

$$\begin{aligned} I(u_n)=I(t_n w_n)=\frac{t_n^2}{2}\Vert w\Vert ^2+\frac{t_n^2}{2} o_{n}(1)-\frac{t_n^{2\cdot 2_{\alpha }^{*}}}{2\cdot 2_{\alpha }^{*}}\int _{{\mathbb {R}}^N} (I_{\alpha }*|w|^{2_{\alpha }^{*}})|w|^{2_{\alpha }^{*}}dx. \end{aligned}$$
(4.5)

Since \(w_n \in \mathcal {N}_{\infty }\) and \(t_nw_n\in \mathcal {N}\), then

$$\begin{aligned} \Vert w_n\Vert ^2=\int _{{\mathbb {R}}^N}(I_{\alpha }*|w_n|^{2_{\alpha }^{*}})|w_n|^{2_{\alpha }^{*}}dx \end{aligned}$$
(4.6)

and

$$\begin{aligned} t_{n}^{2}\Vert w_n\Vert ^2+t_{n}^2\int _{{\mathbb {R}}^N}V(x)w_n^{2}dx=t_{n}^{2 \cdot 2_{\alpha }^{*}}\int _{{\mathbb {R}}^N}(I_{\alpha }*|w_n|^{2_{\alpha }^{*}})|w_n|^{2_{\alpha }^{*}}dx. \end{aligned}$$
(4.7)

Combining (4.6) and (4.7), we then have

$$\begin{aligned} \Vert w\Vert ^2 + o_{n}(1)= t_{n}^{22_{\alpha }^{*}-2}\int _{{\mathbb {R}}^N}\big (I_{\alpha }* |w|^{2_{\alpha }^{*}}\big )|w|^{2_{\alpha }^{*}}dx. \end{aligned}$$
(4.8)

From (4.2), (4.3) and (4.8), then \(\{t_n\}\) is bounded and \(t_n\rightarrow 1\) as \(n\rightarrow +\infty \). Therefore, we have \(I(u_n)\rightarrow m_{\infty }\) as \(n\rightarrow +\infty \) which implies that \(m\le m_{\infty }\). Thus \(m = m_{\infty }\).

In the following, we prove that m cannot be attained. If not, we suppose that there exists \(u_0\in \mathcal {N}\) such that \(I(u_0)=m\) and \(s_{u_0}u_0\in \mathcal {N}_{\infty }\) with \(s_{u_0}\in (0,1]\). With a direct calculation, we get

$$\begin{aligned} m_{\infty }&\le I_{\infty }(s_{u_0}u_0) =\displaystyle \frac{s_{u_0}^{2}}{2}\Vert u_0\Vert ^2-\frac{s_{u_0}^{2\cdot 2_{\alpha }^{*}}}{2\cdot 2_{\alpha }^{*}}\int _{{\mathbb {R}}^N}(I_{\alpha }*|u_0|^{2_{\alpha }^{*}})|u_0|^{2_{\alpha }^{*}}dx \nonumber \\&\le \displaystyle \frac{s_{u_0}^{2}}{2}\Vert u_0\Vert ^2 +\frac{s_{u_0}^{2}}{2}\int _{{\mathbb {R}}^N}V(x)u_0^2dx -\frac{s_{u_0}^{2\cdot 2_{\alpha }^{*}}}{2\cdot 2_{\alpha }^{*}}\int _{{\mathbb {R}}^N}(I_{\alpha }*|u_0|^{2_{\alpha }^{*}})|u_0|^{2_{\alpha }^{*}}dx \nonumber \\&\le I(u_0)\le m_{\infty }, \end{aligned}$$
(4.9)

which leads to

$$\begin{aligned} \int _{{\mathbb {R}}^N} V(x)u_0^2dx=0 ~~ \text {and}~~ s_{u_0}=1. \end{aligned}$$
(4.10)

Thus, \(u_0\in \mathcal {N}_{\infty }\) and \(I_{\infty }(u_0)=m_{\infty }\). Recalling that \(u_0\) must be of form (1.7) and \(u_0>0\), then

$$\begin{aligned} \int _{{\mathbb {R}}^N}V(x)u_0^2 dx>0, \end{aligned}$$

which contradicts to (4.10). Thus, m is not achieved. \(\square \)

The following corollaries can be regarded as a direct consequence of Theorem 1.3 and Proposition 4.1.

Corollary 4.2

Let \(\{u_n\}\subset \mathcal {D}^{1,2}({\mathbb {R}}^N)\) be a nonnegative Palais–Smale sequence satisfying the assumptions of Theorem 1.3 with \(c\in (m,2m)\), then up to a subsequence, \(\{u_n\}\) converges to a nonnegative nontrivial solution \(\bar{u}\) of Eq. (1.10) strongly in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\).

Proof

Obviously, \(u_n\rightharpoonup \bar{u}\) in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) and \(\bar{u}\) is nonnegative. Since \(c\in (m,2m)\), we conclude \(k\le 1\) in (1.12). If \(\bar{u}\ne 0\) and \(k=1\), then \(c\ge 2m\) by (1.14). If \(\bar{u}= 0\) in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) and \(k=1\), then \(u^1\) is a nonnegative solution of Eq. (1.6). By using the property of super harmonic function, we deduce that \(u^1\) is positive and \(c=m\). This is a contradiction, since \(c\in (m,2m)\).

\(\square \)

Corollary 4.3

If \(\{u_n\}\) is a minimizing sequence for I on \(\mathcal {N}\), then there exist a sequence of points \(\{y_n\} \subset {\mathbb {R}}^N\), a sequence of positive numbers \(\{\delta _n\}\subset {\mathbb {R}}^{+}\) and \(\{w_n\}\subset \mathcal {D}^{1,2}({\mathbb {R}}^N)\) such that

$$\begin{aligned} u_n(x)=w_n(x)+\psi _{\delta _n,y_n}(x), \end{aligned}$$
(4.11)

where

$$\begin{aligned} \psi _{\delta _n,y_n}(x):=c_\alpha \left( \frac{\delta _n}{\delta _n^2+|x-y_n|^2}\right) ^{\frac{N-2}{2}} \end{aligned}$$

and \(w_n \rightarrow 0\) strongly in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\).

Now, we prove the existence of positive solutions of Eq. (1.10) via classical Linking Theorem. A direct calculation shows that

$$\begin{aligned} \int _{{\mathbb {R}}^N}|\nabla \psi _{\delta ,y}|^{2}dx=\int _{{\mathbb {R}}^N}\big (I_\alpha *|\psi _{\delta ,y}|^{2_\alpha ^*}\big )|\psi _{\delta ,y}(x)|^{2_\alpha ^*}dx =S_\alpha ^{\frac{N+\alpha }{\alpha +2}}. \end{aligned}$$
(4.12)

In order to build a suitable min–max sequence for our problem, we introduce a barycenter type function and define \(\mathcal {G} : \mathcal {D}^{1,2}({\mathbb {R}}^N)\rightarrow {\mathbb {R}}^N \times {\mathbb {R}}^{+}\) by

$$\begin{aligned} \mathcal {G}(u)=\left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{{\mathbb {R}}^N}\left( \frac{x}{|x|}, \zeta (x)\right) |\nabla u|^2 dx:= (\beta (u), \vartheta (u)), \end{aligned}$$

where \(\zeta (x)\) is a cut-off function such that

$$\begin{aligned} \zeta (x)= {\left\{ \begin{array}{ll} 0, \quad \quad \text {if} \quad |x|<1; \\ 1, \quad \quad \text {if} \quad |x|\ge 1. \end{array}\right. } \end{aligned}$$
(4.13)

Moreover,

$$\begin{aligned} \beta (u)=\left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{{\mathbb {R}}^N}\frac{x}{|x|}|\nabla u|^2 dx \end{aligned}$$

and

$$\begin{aligned} \vartheta (u)=\left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{{\mathbb {R}}^N}\zeta (x)|\nabla u|^2 dx. \end{aligned}$$

Lemma 4.4

If \(|y|\ge \frac{1}{2}\), then

$$\begin{aligned} \beta (\psi _{\delta ,y})=\frac{y}{|y|}+o_n(1) \quad \text {as} \quad \delta \rightarrow 0. \end{aligned}$$

Proof

A direct calculation shows that

$$\begin{aligned} \displaystyle \int _{{\mathbb {R}}^N {\setminus } B_{\varepsilon }(y)}|\nabla \psi _{\delta ,y}|^2dx&\le C\delta ^{N-2}\displaystyle \int _{{\mathbb {R}}^N {\setminus } B_{\varepsilon }(y)} \frac{|x-y|^2}{(\delta ^2+|x-y|^2)^{N}}dx \nonumber \\&=\displaystyle C \delta ^{N-2}\displaystyle \int _{\varepsilon }^{+\infty }\frac{\tilde{\rho }^{N+1}}{(\delta ^2+\tilde{\rho }^2)^{N}}d\tilde{\rho } \le C \delta ^{N-2}\displaystyle \int _{\varepsilon }^{+\infty }\frac{1}{\tilde{\rho }^{N-1}}d\tilde{\rho }. \end{aligned}$$
(4.14)

Then, for each \(\varepsilon >0\), there exists \(\delta _0:=\delta _0(\varepsilon )\) such that for any \(\delta \in (0, \delta _0]\),

$$\begin{aligned} \int _{{\mathbb {R}}^N{\setminus } B_{\varepsilon }(y)}|\nabla \psi _{\delta ,y}|^2dx<\varepsilon . \end{aligned}$$
(4.15)

Furthermore

$$\begin{aligned} \Big |\beta (\psi _{\delta ,y})-\left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{B_{\varepsilon }(y)}\frac{x}{|x|}|\nabla \psi _{\delta ,y}|^2 dx\Big |<\varepsilon . \end{aligned}$$
(4.16)

Let \(\varepsilon \) be small enough such that for \(|y|\ge \frac{1}{2}\), the following property holds

$$\begin{aligned} \Big | \frac{x}{|x|}-\frac{y}{|y|}\Big |<\varepsilon \quad \text {for any} \ x\in B_{\varepsilon }(y). \end{aligned}$$

Then, by (4.15), we have

$$\begin{aligned}&\Big | \displaystyle \frac{y}{|y|}-\left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{B_{\varepsilon }(y)} \frac{x}{|x|}|\nabla \psi _{\delta ,y}|^2dx \Big | \nonumber \\&\quad =\left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}} \Big | \displaystyle \int _{B_{\varepsilon }(y)}\left( \frac{y}{|y|}- \frac{x}{|x|}\right) |\nabla \psi _{\delta ,y}|^2dx + \displaystyle \int _{{\mathbb {R}}^N {\setminus } B_{\varepsilon }(y)}\frac{y}{|y|}|\nabla \psi _{\delta ,y}|^2dx\Big | \nonumber \\&\quad \le \frac{\varepsilon }{S_\alpha ^{\frac{N+\alpha }{\alpha +2}}}\displaystyle \int _{B_{\varepsilon }(y)}|\nabla \psi _{\delta ,y}|^2dx+\varepsilon \le 2\varepsilon . \end{aligned}$$
(4.17)

Therefore, it follows by (4.16) and (4.17) that we can easily deduce that

$$\begin{aligned} \Big |\beta (\psi _{\delta ,y})-\frac{y}{|y|}\Big |\le 3\varepsilon \end{aligned}$$

and then we complete the proof of lemma. \(\square \)

In the sequel, we denote by

$$\begin{aligned} \mathcal {M}:=\Big \{u\in \mathcal {N}:\mathcal {G}(u)=\big (\beta (u), \vartheta (u)\big )=\left( 0,\frac{1}{2}\right) \Big \} \end{aligned}$$

a subset of Nehari manifold \(\mathcal {N}\) and define \(c_{\mathcal {M}}:=\displaystyle \inf _{u\in \mathcal {M}}I(u)\).

Lemma 4.5

Let \(V(x)\ge 0\) and \(V(x) \in L^{\frac{N}{2}}({\mathbb {R}}^N)\). Then \(c_{\mathcal {M}}>m\).

Proof

Obviously \(c_{\mathcal {M}}\ge m\). In order to show the identity cannot hold, we shall argue by contradiction and then assume that there exists a sequence of \(\{u_n\} \subset \mathcal {M}\) such that

$$\begin{aligned} \lim _{n\rightarrow +\infty }I(u_n)=m. \end{aligned}$$

Moreover, for any \(n\in {\mathbb N}\),

$$\begin{aligned} \beta (u_n)=0 \quad \text {and} \quad \vartheta (u_n)=\frac{1}{2}. \end{aligned}$$
(4.18)

By Corollary 4.3, we deduce that there exist a sequence of points \(\{y_n\}\subset {\mathbb {R}}^N\), a sequence of positive numbers \(\{\delta _n\}\subset {\mathbb {R}}^N\) and a sequence of functions \(\{w_n\}\subset \mathcal {D}^{1,2}({\mathbb {R}}^N)\) with \(w_n\rightarrow 0\) in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) such that \(u_n(x)=w_{n}(x)+\psi _{\delta _n,y_n}\). By the definition of \(\mathcal {G}\), we get

$$\begin{aligned} \mathcal {G}(w_n+\psi _{\delta _n,y_n})&= \left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}} \int _{{\mathbb {R}}^N}\left( \frac{x}{|x|}, \zeta (x)\right) |\nabla (w_n+\psi _{\delta _n,y_n})|^2 dx \\&=\left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{{\mathbb {R}}^N}\left( \frac{x}{|x|}, \zeta (x)\right) |\nabla w_n|^2 dx \\&\quad + \frac{2}{S_{\alpha }^{\frac{N+\alpha }{\alpha +2}}}\int _{{\mathbb {R}}^N}\left( \frac{x}{|x|}, \zeta (x)\right) (\nabla w_n \nabla \psi _{\delta _n,y_n}) dx\\&\quad +\left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{{\mathbb {R}}^N}\left( \frac{x}{|x|}, \zeta (x)\right) |\nabla \psi _{\delta _n,y_n}|^{2} dx \\&=\left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{{\mathbb {R}}^N}\left( \frac{x}{|x|}, \zeta (x)\right) |\nabla \psi _{\delta _n,y_n}|^{2} dx + o_n(1) = \mathcal {G}(\psi _{\delta _n,y_n})+o_n(1). \end{aligned}$$

Therefore, by (4.18), we deduce that

$$\begin{aligned} \beta (\psi _{\delta _n,y_n})\rightarrow 0~~ \text {and}~~ \vartheta (\psi _{\delta _n,y_n})\rightarrow \frac{1}{2} ~~\text {as}~~ n \rightarrow +\infty . \end{aligned}$$
(4.19)

There exists a subsequence \((\delta _n, y_n)\) such that one of the following cases may happen

  1. (1)

       \(\delta _{n}\rightarrow +\infty \)  as  \(n\rightarrow \infty \);

  2. (2)

       \(\delta _{n}\rightarrow \bar{\delta }\ne 0\)  as \(n\rightarrow \infty \);

  3. (3)

       \(\delta _{n}\rightarrow 0 \) and  \(y_n \rightarrow \bar{y}\), \(|\bar{y}|<\frac{1}{2}\) as \(n\rightarrow \infty \);

  4. (4)

       \(\delta _{n}\rightarrow 0 \)  as  \(n\rightarrow \infty \)  and  \(|y_n|\ge \frac{1}{2}\) for n large.

Now we prove that none of the possibilities listed above can be true. Obviously, by Lemma 4.4 and (4.19), case (4) can not happen. If (1) holds, then

$$\begin{aligned} \vartheta (\psi _{\delta _n,y_n})&=\displaystyle \left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{{\mathbb {R}}^N}\zeta (x)|\nabla \psi _{\delta _n,y_n}|^2dx\\&= \displaystyle \left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{{\mathbb {R}}^N {\setminus } B_1(0)}|\nabla \psi _{\delta _n,y_n}|^2dx \\&= \displaystyle 1-\left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{ B_1(0)}|\nabla \psi _{\delta _n,y_n}|^2dx =1-o_n(1), \end{aligned}$$

which contradicts to (4.19). If (2) happens, we first assert that \(|y_n|\rightarrow +\infty \). If not, up to a subsequence, we notice that \(\psi _{\delta _n, y_n}\) would converge strongly in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\), so \(u_n\) converges strongly in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\), which is impossible by Proposition 4.1. Thus, for \(n\rightarrow +\infty \), we have

$$\begin{aligned} \vartheta (\psi _{\delta _n,y_n})&= \vartheta (\psi _{\bar{\delta },y_n})+o_n(1)\\&=\left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{{\mathbb {R}}^N}\zeta (x)|\nabla \psi _{\bar{\delta },y_n}|^2dx +o_n(1)\\&= \displaystyle \left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{{\mathbb {R}}^N}\zeta (x-y_n)|\nabla \psi _{\bar{\delta },0}|^2dx +o_n(1)\\&=1-\left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{ B_1(y_n)}|\nabla \psi _{\bar{\delta },0}|^2dx+o_n(1)=1+o_n(1), \end{aligned}$$

which is absurd in the sense of (4.19). If (3) is true, then for n large,

$$\begin{aligned} \vartheta (\psi _{\delta _n,y_n})&= \displaystyle \left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{{\mathbb {R}}^N{\setminus } B_1(0)}|\nabla \psi _{\delta _n,y_n}|^2dx +o_n(1) \\&\le \displaystyle \left( \frac{1}{S_\alpha }\right) ^{\frac{N+\alpha }{\alpha +2}}\int _{{\mathbb {R}}^N{\setminus } B_1(y_n)}|\nabla \psi _{\delta _n,0}|^2dx =o_n(1), \end{aligned}$$

which is also impossible. Then the proof is completed. \(\square \)

In the following, we define a mapping \(\theta : \mathcal {D}^{1,2}({\mathbb {R}}^N){\setminus } \{0\} \rightarrow \mathcal {N}\) by

$$\begin{aligned} \theta (u)=t_{u}u, \end{aligned}$$

where \(t_u\) is the unique positive number such that \(t_u u \in \mathcal {N}\). Also we define the operator \(T:{\mathbb {R}}^N \times (0,+\infty )\rightarrow \mathcal {D}^{1,2}({\mathbb {R}}^N)\) by

$$\begin{aligned} T(y,\delta )=\psi _{\delta ,y}(x). \end{aligned}$$

Then we have the following lemma.

Lemma 4.6

Assume that \(V(x)\ge 0\) and \(V(x)\in L^{\frac{N}{2}}({\mathbb {R}}^N)\). Then for any \(\varepsilon >0\), there exists \(\delta _1=\delta _1(\varepsilon )\) and \(\delta _2=\delta _2(\varepsilon )\) (without loss of generality, we assume that \(\delta _1\le \delta _2\)) such that

$$\begin{aligned} I(\theta \circ T(y,\delta ))<m+\varepsilon \end{aligned}$$

for any \(\delta \in (0, \delta _1]\cup [\delta _2, +\infty )\) and \(y\in {\mathbb {R}}^N\).

Proof

Since \(V(x)\in L^{\frac{N}{2}}({\mathbb {R}}^N)\), then for any \(\varepsilon >0\), there exists \(r>0\) small enough such that

$$\begin{aligned} \sup _{y \in {\mathbb {R}}^N}\left( \int _{B_r(y)}|V(x)|^{\frac{N}{2}}dx\right) ^{\frac{2}{N}}<\varepsilon . \end{aligned}$$
(4.20)

A direct calculation shows that

$$\begin{aligned} \lim _{\delta \rightarrow 0}\int _{B_{r}(y)}|\psi _{\delta ,y}|^{2^*}dx=\int _{{\mathbb {R}}^N}|\psi _{1,0}|^{2^*}dx= \int _{{\mathbb {R}}^N}|\psi _{\delta ,y}|^{2^*}dx. \end{aligned}$$
(4.21)

Thus, there exists \(\delta _1=\delta _{1}(\varepsilon )\) small enough, such that for any \(\delta \in (0, \delta _1)\),

$$\begin{aligned} \left( \int _{{\mathbb {R}}^N{\setminus } B_{r}(y)}|\psi _{\delta ,y}|^{2^*}dx \right) ^{\frac{N-2}{N}}< \varepsilon . \end{aligned}$$
(4.22)

From (4.20) and (4.22), we obtain

$$\begin{aligned}&\int _{{\mathbb {R}}^N}V(x)|\psi _{\delta ,y}|^2 dx =\int _{B_{r}(y)}V(x)|\psi _{\delta , y}|^2dx+\int _{{\mathbb {R}}^N {\setminus } B_{r}(y)}V(x)|\psi _{\delta , y}|^2dx \nonumber \\&\quad \le \left( \int _{B_{r}(y)}|V(x)|^{\frac{N}{2}}dx\right) ^{\frac{2}{N}} \left( \int _{B_{r}(y)}|\psi _{\delta , y}|^{2^*}dx\right) ^{\frac{N-2}{N}}\nonumber \\&\qquad +\left( \int _{{\mathbb {R}}^N {\setminus } B_{r}(y)}|V(x)|^{\frac{N}{2}}dx\right) ^{\frac{2}{N}} \left( \int _{{\mathbb {R}}^N {\setminus } B_{r}(y)}|\psi _{\delta , y}|^{2^*}dx\right) ^{\frac{N-2}{N}}\nonumber \\&\quad \le C\varepsilon . \end{aligned}$$
(4.23)

Using \(V(x)\in L^{\frac{N}{2}}({\mathbb {R}}^N)\) again, we assume that for any \(\varepsilon >0\), there exists \(R>0\) big enough such that

$$\begin{aligned} \left( \int _{{\mathbb {R}}^N {\setminus } B_{R}(0)}|V(x)|^{\frac{N}{2}}dx\right) ^{\frac{2}{N}}<\varepsilon . \end{aligned}$$
(4.24)

Recalling that \(\displaystyle \lim _{\delta \rightarrow +\infty }\sup _{x\in {\mathbb {R}}^N}|\psi _{\delta ,y}|=0\), then we obtain that

$$\begin{aligned} \lim _{\delta \rightarrow +\infty }\int _{B_{R}(0)}|\psi _{\delta ,y}|^{2^*}dx=0, \end{aligned}$$
(4.25)

which implies that there exists \(\delta _2:=\delta _2(\varepsilon )>0\), such that for any \(\delta \ge \delta _2\),

$$\begin{aligned} \left( \int _{B_{R}(0)}|\psi _{\delta ,y}|^{2^*}dx\right) ^{\frac{N-2}{N}}\le \varepsilon . \end{aligned}$$
(4.26)

In virtue of (4.24) and (4.26), we can prove that that for any \( \delta \ge \delta _2\),

$$\begin{aligned} \displaystyle \int _{{\mathbb {R}}^N}V(x)|\psi _{\delta ,y}|^2 dx = \displaystyle \int _{B_{R}(0)}V(x)|\psi _{\delta , y}|^2dx+ \displaystyle \int _{{\mathbb {R}}^N {\setminus } B_{R}(0)}V(x)|\psi _{\delta , y}|^2dx\le C\varepsilon . \end{aligned}$$
(4.27)

Thus, combining (4.23) and (4.27) that we can conclude that

$$\begin{aligned} \int _{{\mathbb {R}}^N}V(x)|\psi _{\delta ,y}|^{2}dx<\varepsilon , \end{aligned}$$
(4.28)

for any \(y\in {\mathbb {R}}^N \) and \(\delta \in (0,\delta _1]\cup [\delta _2, \infty )\).

For any \(\psi _{\delta ,y}\), there exists \(t_{\psi }:=t(\psi _{\delta ,y})\ge 1\) such that \(t_{\psi }\psi _{\delta ,y}\in \mathcal {N}\). With a similar argument to the proof in (4.6)–(4.8), we prove that for uniformly \(y\in {\mathbb {R}}^N\), \( t_{\psi }\rightarrow 1\) as \(\delta \rightarrow 0\) or \(\delta \rightarrow +\infty \). Thus, inspired by (4.28), for any \(\delta \in (0,\delta _1]\cup [\delta _2,+\infty )\),

$$\begin{aligned} I(\theta \circ T(y,\delta ))&=\displaystyle \frac{t_{\psi }^{2}}{2}\Vert \psi _{\delta ,y}\Vert ^2+ \frac{t_{\psi }^{2}}{2}\int _{{\mathbb {R}}^N}V(x)\psi _{\delta ,y}^{2}dx -\frac{t_{\psi }^{2\cdot 2_{\alpha }^{*}}}{2\cdot 2_{\alpha }^{*}}\displaystyle \int _{{\mathbb {R}}^N}(I_{\alpha }*|\psi _{\delta ,y}|^{2_{\alpha }^{*}})|\psi _{\delta ,y}|^{2_{\alpha }^{*}}dx \\&=I_{\infty }(t_{\psi }\psi _{\delta ,y})+\displaystyle \frac{t_{\psi }^{2}}{2}\displaystyle \int _{{\mathbb {R}}^N}V(x)\psi _{\delta ,y}^{2}dx <I_{\infty }(\psi _{\delta ,y})+\varepsilon =m+\varepsilon . \end{aligned}$$

\(\square \)

Lemma 4.7

Assume that \(V(x) \ge 0\) and \(V(x) \in L^{\frac{N}{2}}({\mathbb {R}}^N)\). Then for any fixed \(\delta >0\),

$$\begin{aligned} \lim _{|y|\rightarrow +\infty } I(\theta \circ T(y,\delta ))=m. \end{aligned}$$

Proof

First, we claim that for any fixed \(\delta >0\),

$$\begin{aligned} \lim _{|y|\rightarrow +\infty }\int _{{\mathbb {R}}^N}V(x)|\psi _{\delta ,y}|^{2}dx=0. \end{aligned}$$
(4.29)

Indeed, for a given \(\varepsilon >0\), we can choose some \(R>0\) large enough such that

$$\begin{aligned} \int _{{\mathbb {R}}^N{\setminus } B_{R}(0)}|V(x)|^{\frac{N}{2}}dx<\varepsilon \end{aligned}$$

and

$$\begin{aligned} \int _{{\mathbb {R}}^N{\setminus } B_{R}(y)}|\psi _{\delta ,y}|^{2^*}dx =\int _{{\mathbb {R}}^N{\setminus } B_{R}(0)}|\psi _{\delta ,0}|^{2^*}dx<\varepsilon . \end{aligned}$$

Taking y with \(|y|>2R\), we see

$$\begin{aligned} \displaystyle \int _{{\mathbb {R}}^N}V(x)|\psi _{\delta ,y}|^2dx&=\displaystyle \int _{{\mathbb {R}}^N {\setminus } (B_{R}(y)\cup B_{R}(0))}V(x)|\psi _{\delta ,y}|^2dx\nonumber \\&\quad + \displaystyle \int _{B_{R}(y)}V(x)|\psi _{\delta ,y}|^2dx + \displaystyle \int _{B_{R}(0)}V(x)|\psi _{\delta ,y}|^2dx \nonumber \\&\le \displaystyle \left( \int _{{\mathbb {R}}^N {\setminus } B_{R}(0)}|V(x)|^{\frac{N}{2}}dx \right) ^{\frac{2}{N}} \displaystyle \left( \int _{{\mathbb {R}}^N {\setminus } B_{R}(y)}|\psi _{\delta ,y}|^{2^*}dx \right) ^{\frac{N-2}{N}} \nonumber \\&\quad + \displaystyle \left( \int _{{\mathbb {R}}^N {\setminus } B_{R}(0)}|V(x)|^{\frac{N}{2}}dx \right) ^{\frac{2}{N}} \Vert \psi _{\delta ,y}\Vert _{L^{2^*}}^{2} \nonumber \\&\quad + \Vert V(x)\Vert _{L^{\frac{N}{2}}} \displaystyle \left( \int _{{\mathbb {R}}^N {\setminus } B_{R}(y)}|\psi _{\delta ,y}|^{2^*}dx \right) ^{\frac{N-2}{N}} \nonumber \\&\le C\varepsilon . \end{aligned}$$
(4.30)

With a similar argument as the proof in (4.6)–(4.8) again, we can also prove that \(t_{\psi }\rightarrow 1\) as \(|y|\rightarrow +\infty \), where \(t_{\psi }\) satisfies \(t_{\psi } \psi _{\delta ,y}\in \mathcal {N}\). Thus, as \(|y|\rightarrow +\infty \), by (4.30),

$$\begin{aligned} m&\le I(\theta \circ T(y,\delta )) \nonumber \\&=\displaystyle \frac{t_{\psi }^{2}}{2}\Vert \psi _{\delta ,y}\Vert ^2+ \frac{t_{\psi }^{2}}{2}\int _{{\mathbb {R}}^N}V(x)|\psi _{\delta ,y}|^{2}dx -\frac{t_{\psi }^{2\cdot 2_{\alpha }^{*}}}{2\cdot 2_{\alpha }^{*}}\int _{{\mathbb {R}}^N}(I_{\alpha }*|\psi _{\delta ,y}|^{2_{\alpha }^{*}})|\psi _{\delta ,y}|^{2_{\alpha }^{*}}dx\nonumber \\&= \displaystyle \frac{1}{2}\Vert \psi _{\delta ,y}\Vert ^2+ \frac{1}{2}\int _{{\mathbb {R}}^N}V(x)|\psi _{\delta ,y}|^{2}dx -\frac{1}{2\cdot 2_{\alpha }^{*}}\int _{{\mathbb {R}}^N}(I_{\alpha }*|\psi _{\delta ,y}|^{2_{\alpha }^{*}})|\psi _{\delta ,y}|^{2_{\alpha }^{*}}dx+o(1) \nonumber \\&=m+o(1). \end{aligned}$$
(4.31)

Thus \( \displaystyle \lim _{|y|\rightarrow +\infty } I(\theta \circ T(y,\delta ))=m\). \(\square \)

From Lemma 4.5, we can deduce that there exists some \(\sigma >0\) such that \(m+\sigma <c_{\mathcal {M}}\). In the following, we give some estimates.

Lemma 4.8

There exists \(\delta _1\in (0,\frac{1}{2})\) such that for any \(0<\delta \le \delta _1\), the following properties hold.

(a):

\(I(\theta \circ T(y,\delta ))< m+\sigma ,\)   for any \(y \in {\mathbb {R}}^N\);

(b):

\( \big | \beta (\theta \circ T(y,\delta ))-\frac{y}{|y|}\big |<\frac{1}{4},\) for any \(y\in {\mathbb {R}}^N\) with \(|y|\ge \frac{1}{2}\);

(c):

\(\vartheta (\theta \circ T(y,\delta ))<\frac{1}{2}\), for any \(y\in {\mathbb {R}}^N\) with \(|y|<\frac{1}{2}\).

Proof

(a) and (b) are easy to prove. In fact, (a) can be seen as a direct consequence of Lemma 4.6. In Lemma 4.6, we have proved that \(t_{\psi }\rightarrow 1\) as \(\delta \rightarrow 0\), which together with Lemma 4.4 yields (b). Now we only need to prove (c). A direct calculation shows that

$$\begin{aligned} \vartheta (\theta \circ T(y,\delta ))&=\displaystyle \frac{t_{\psi }^{2}}{S_{\alpha }^{\frac{N+\alpha }{\alpha +2}}}\int _{{\mathbb {R}}^N}\zeta (x)|\nabla \psi _{\delta ,y}|^2dx =\displaystyle \frac{t_{\psi }^{2}}{S_{\alpha }^{\frac{N+\alpha }{\alpha +2}}}\int _{{\mathbb {R}}^N{\setminus } B_1(0)}|\nabla \psi _{\delta ,y}|^2dx \nonumber \\&=\displaystyle \frac{t_{\psi }^{2}}{S_{\alpha }^{\frac{N+\alpha }{\alpha +2}}}\int _{{\mathbb {R}}^N{\setminus } B_1(y)}|\nabla \psi _{\delta ,0}|^2dx \rightarrow 0, \end{aligned}$$
(4.32)

where in the last equality we have used the fact \(\int _{{\mathbb {R}}^N{\setminus } B_1(y)}|\nabla \psi _{\delta ,0}|^2dx \rightarrow 0\) for \(|y|<\frac{1}{2}\) as \(\delta \rightarrow 0\). \(\square \)

Lemma 4.9

There exist \(\delta _2\in (\frac{1}{2}, +\infty )\) such that for any \(\delta \ge \delta _2\), the following properties hold.

(a):

\(I(\theta \circ T(y,\delta ))< m+\sigma \), for any \(y \in {\mathbb {R}}^N\);

(b):

\(\vartheta (\theta \circ T(y,\delta ))>\frac{1}{2}\), for any \(y\in {\mathbb {R}}^N\).

Proof

By Lemma 4.6, (a) is true. Since

$$\begin{aligned} \lim _{\delta \rightarrow +\infty }\int _{B_{1}(0)}|\nabla \psi _{\delta ,y}|^2dx=0 \end{aligned}$$

and \(t_{\psi }\rightarrow 1\) as \(\delta \rightarrow +\infty \), we obtain

$$\begin{aligned} \vartheta (\theta \circ T(y,\delta ))=t_{\psi }^{2}\left( 1-\frac{1}{S_\alpha ^{\frac{N+\alpha }{\alpha +2}}}\int _{B_1(0)}|\nabla \psi _{\delta ,y}|^2dx\right) \rightarrow 1 ~~\text {as}~~ \delta \rightarrow +\infty . \end{aligned}$$

Hence (b) holds. \(\square \)

Lemma 4.10

There exists some \(R>0\) such that for any \(\delta \in [\delta _1,\delta _2]\), the following properties hold.

(a):

\(I(\theta \circ T(y,\delta ))< m+\sigma ,\) for any \(y \in {\mathbb {R}}^N\) with \(|y|\ge R\);

(b):

\(\langle \beta (\theta \circ T(y,\delta )),y\rangle >0,\) for any \(y \in {\mathbb {R}}^N\) with \(|y|\ge R\).

Proof

For any fixed \(\delta \), let \(|y|\rightarrow +\infty \) and repeating the argument in the proof of (4.6)–(4.8) again, we know \(t_{\psi }=t(\psi _{\delta ,y})\rightarrow 1\), where \(t_{\psi }\) satisfies \(t_{\psi }\psi _{\delta ,y}\in \mathcal {N}\). Using Lemma 4.7 and the compactness of \([\delta _1,\delta _2]\), we deduce that there exists some \(R_1>0\) such that

$$\begin{aligned} I(\theta \circ T(y,\delta ))< m+\sigma ~~\text {for any}~~\delta \in [\delta _1,\delta _2]~~\text {and}~~ |y|\ge R_1. \end{aligned}$$

Let \(({\mathbb {R}}^N)_{y}^{+}=\{x\in {\mathbb {R}}^N:\langle x, y\rangle >0\}\) and \(({\mathbb {R}}^N)_{y}^{-}={\mathbb {R}}^N {\setminus } ({\mathbb {R}}^N)_{y}^{+}\). Since \(\delta \in [\delta _1, \delta _2]\), we assert that there exists \(R_2>0\) large enough and \(r\in (0,\frac{1}{4})\) such that the following properties holds: for any y with \(|y|\ge R_2\),

$$\begin{aligned} B_{r}(\widetilde{y})=\{x\in {\mathbb {R}}^N: |x-\widetilde{y}|<r\}\subset ({\mathbb {R}}^N)_{y}^{+} \end{aligned}$$

with \(|\widetilde{y}-y |=\frac{1}{2}\) and for any \(x\in B_{r}(\widetilde{y})\),

$$\begin{aligned} |\nabla \psi _{\delta ,y}(x)|^2=K_1 \delta ^{N-2}\frac{|x-y|^2}{\big (\delta ^2+|x-y|^2\big )^{N}} \ge H_1>0, \end{aligned}$$

where \(K_1\) only depend on N and \(\alpha \), \(H_1\) is a positive constant. Moreover, for each \(x\in ({\mathbb {R}}^N)_{y}^{-} \),

$$\begin{aligned} |\nabla \psi _{\delta ,y}(x)|^2 \le \frac{H_{2}}{|x-y|^{2N-2}}, \quad H_{2}\equiv const. \end{aligned}$$

Thus, for any y satisfying \(|y|\ge R_2\), we have

$$\begin{aligned} \langle \beta (\theta \circ T(y,\delta )),y\rangle&= \frac{t_{\psi }^2}{S_{\alpha }}\int _{({\mathbb {R}}^N)_{y}^{+}}|\nabla \psi _{\delta ,y}(x)|^2\frac{\langle x,y\rangle }{|x|}dx+\frac{t_{\psi }^2}{S_{\alpha }}\int _{({\mathbb {R}}^N)_{y}^{-}}|\nabla \psi _{\delta ,y}(x)|^2\frac{\langle x,y\rangle }{|x|}dx \\&\ge \frac{t_{\psi }^2}{S_{\alpha }}\int _{B_{r}(\widetilde{y})}H_1\frac{\langle x,y\rangle }{|x|}dx- \frac{t_{\psi }^2}{S_{\alpha }}\int _{({\mathbb {R}}^N)_{y}^{-}} H_2\frac{|y|}{|x-y|^{2N-2}}dx \\&\ge H_{3}|y|-C\frac{1}{|y|^{N-3}}\int _{{\mathbb {R}}^{N-1}}\frac{1}{1+|z|^{2N-2}}dz>0, \end{aligned}$$

where \(H_{3}\) is a positive constant. Taking \(R=\max \{R_1,R_2\} \), we then complete the proof. \(\square \)

In the sequel, we define a bounded domain \(D\subset {\mathbb {R}}^N \times {\mathbb {R}}\) by

$$\begin{aligned} D:=\{(y,\delta )\in {\mathbb {R}}^N \times {\mathbb {R}}~:~ |y|\le R, ~ \delta _1 \le \delta \le \delta _2\}, \end{aligned}$$

where \(\delta _1\), \(\delta _2\) and R are given in Lemmas 4.84.10.

Lemma 4.11

Define a mapping \(\Upsilon : D\rightarrow {\mathbb {R}}^N \times {\mathbb {R}}^+\) by

$$\begin{aligned} \Upsilon (y,\delta )=\big (\beta \circ \theta \circ T(y,\delta ), ~~\vartheta \circ \theta \circ T(y,\delta )\big ). \end{aligned}$$

Then

$$\begin{aligned} deg\left( \Upsilon , D, \left( 0,\frac{1}{2}\right) \right) =1. \end{aligned}$$

Proof

Consider the following homotopy

$$\begin{aligned} \mathscr {F}(y,\delta ,s)=(1-s)(y,\delta )+s \Upsilon (y,\delta ). \end{aligned}$$

Since \(deg\big (id, D, \left( 0,\frac{1}{2})\right) =1\), then by the homotopy invariance of topological degree, we can complete the proof. In order to use the homotopy invariance of the topological degree, we must prove

$$\begin{aligned} \mathscr {F}(y,\delta ,s)\ne \left( 0,\frac{1}{2}\right) ~~ \text {for any}~~ (y,\delta )\in \partial D ~~ \text {and}~~ s\in [0,1]. \end{aligned}$$
(4.33)

For the readers’ convenience, we divide the proof into several cases and discuss them respectively.

Case 1 If \(|y|<\frac{1}{2}\) and \(\delta =\delta _1\), by Lemma 4.8(c), we know

$$\begin{aligned} (1-s)\delta _1+ s\vartheta \circ \theta \circ T(y,\delta _1)<\frac{1}{2} \end{aligned}$$

for any \(s\in [0,1]\).

Case 2 If \(\frac{1}{2}\le |y|\le R\) and \(\delta =\delta _1\), then it follows by Lemma 4.8(b) that

$$\begin{aligned} \Big |\beta (\theta \circ T(y,\delta ))-\frac{y}{|y|}\Big |<\frac{1}{4}. \end{aligned}$$

Thus

$$\begin{aligned} |(1-s)y+s\beta (\theta \circ T(y,\delta _1))|&\ge \displaystyle \Big |(1-s)y+s\frac{y}{|y|}\Big |-\Big |s\beta (\theta \circ T(y,\delta ))-s\frac{y}{|y|}\Big | \\&\ge s+(1-s)|y|-\displaystyle \frac{s}{4} \ge \frac{1}{4}\ne 0. \end{aligned}$$

Case 3 If \(|y|\le R\) and \(\delta =\delta _2\), from Lemma 4.9(b), we know that

$$\begin{aligned} (1-s)\delta _2+s \vartheta \circ \theta \circ T(y,\delta _2)>\frac{1}{2} \end{aligned}$$

for any \(s\in [0,1]\).

Case 4 If \(|y|=R\) and \(\delta \in [\delta _1,\delta _2]\), by Lemma 4.10(b),

$$\begin{aligned} \langle (1-s)y+s \beta \circ \theta \circ T(y,\delta ), ~y\rangle >0 \end{aligned}$$

for \(s\in [0,1]\). \(\square \)

Proof of Theorem 1.2

Obviously, the first part of Theorem 1.2 follows from Proposition 4.1. In order to apply the classical Linking Theorem (see [40]), we define

$$\begin{aligned} \mathcal {H}=\theta \circ T(D) \end{aligned}$$

and

$$\begin{aligned} \mathcal {M} =\left\{ u\in \mathcal {N}: \mathcal {G}(u)=(\beta (u),\vartheta (u))=\left( 0,\frac{1}{2}\right) \right\} . \end{aligned}$$

We claim that \(\mathcal {M}\) and \(\partial \mathcal {H}\) is a link, that is

(a):

   \(\partial \mathcal {H}\cap \mathcal {M}=\emptyset \);

(b):

   \(h(\mathcal {H})\cap \mathcal {M} \ne \emptyset \) for any \(h\in \Lambda =\{h\in \mathcal {C}(\mathcal {H}, \mathcal {N}): h(\partial \mathcal {H})=id\}\).

In fact, if \(u\in \theta \circ T(\partial D)\), then it follows from Lemmas 4.8(a), 4.9(a) and 4.10(a) that

$$\begin{aligned} I(u)<m+ \sigma < c_{\mathcal {M}}, \end{aligned}$$

which implies \(u\notin \mathcal {M}\) and we prove (a).

Next, we prove (b). In fact, for any \(h\in \Lambda \), we define a continuous mapping \(\widetilde{\eta }: D\rightarrow {\mathbb {R}}^N \times {\mathbb {R}}^{+}\) by

$$\begin{aligned} \widetilde{\eta } (y,\delta )=(\beta \circ h \circ \theta \circ T(y,\delta ), ~ \vartheta \circ h \circ \theta \circ T(y,\delta )). \end{aligned}$$

If \((y,\delta )\in \partial D\), then \(\theta \circ T(y,\delta )\in \partial \mathcal {H}\), hence \(h\circ \theta \circ T(y,\delta )=\theta \circ T(y,\delta )\). Therefore

$$\begin{aligned} \widetilde{\eta }(y,\delta )=(\beta \circ \theta \circ T(y,\delta ), ~ \vartheta \circ \theta \circ T(y,\delta ))= \Upsilon (y,\delta )~~ \text {on}~~\partial D. \end{aligned}$$

By the homotopy invariance of the topological degree and Lemma 4.11, we have

$$\begin{aligned} deg\left( \widetilde{\eta },D,\left( 0,\frac{1}{2}\right) \right) =deg\left( \Upsilon ,D,\left( 0,\frac{1}{2}\right) \right) =1, \end{aligned}$$

which implies that there exists \((y',\delta ')\in D\) such that \(h\circ \theta \circ T(y',\delta ')\in \mathcal {M}\). Hence (b) holds.

Since \(\mathcal {N}\) is a natural constraint for I, with classical minimal arguments we obtain a Palais–Smale sequence for I at level d with

$$\begin{aligned} d:=\inf _{h\in \Lambda }\max _{u\in \mathcal {H}}I(h(u)). \end{aligned}$$

From (b) and Lemma 4.5, we have

$$\begin{aligned} m< c_{\mathcal {M}} \le d. \end{aligned}$$

Moreover, by definition of d and \(\mathcal {H}\), we get

$$\begin{aligned} d\le \max _{u\in \mathcal {H}}I(u)\le \sup _{(\delta ,y)\in D}I(t_{\psi }\psi _{\delta ,y}), \end{aligned}$$

As \(t_{\psi }\psi _{\delta ,y}\in \mathcal {N}\), we know that

$$\begin{aligned} I(t_{\psi }\psi _{\delta ,y})&=\displaystyle \frac{t_{\psi }^{2}}{2}\Vert \psi _{\delta ,y}\Vert ^2+ \frac{t_{\psi }^{2}}{2}\int _{{\mathbb {R}}^N}V(x)|\psi _{\delta ,y}|^{2}dx -\frac{t_{\psi }^{2\cdot 2_{\alpha }^{*}}}{2\cdot 2_{\alpha }^{*}}\int _{{\mathbb {R}}^N}(I_{\alpha }*|\psi _{\delta ,y}|^{2_{\alpha }^{*}})|\psi _{\delta ,y}|^{2_{\alpha }^{*}}dx \nonumber \\&=\displaystyle \left( \frac{1}{2}-\frac{1}{2\cdot 2_{\alpha }^{*}}\right) t_{\psi }^{2\cdot 2_{\alpha }^{*}}\int _{{\mathbb {R}}^N}(I_{\alpha }*|\psi _{\delta ,y}|^{2_{\alpha }^{*}})|\psi _{\delta ,y}|^{2_{\alpha }^{*}}dx. \end{aligned}$$
(4.34)

On the other hand,

$$\begin{aligned} t_{\psi }^{2\cdot 2_{\alpha }^{*}-2}\displaystyle \int _{{\mathbb {R}}^N}(I_{\alpha }*|\psi _{\delta ,y}|^{2_{\alpha }^{*}})|\psi _{\delta ,y}|^{2_{\alpha }^{*}}dx&= \displaystyle \Vert \psi _{\delta ,y}\Vert ^2+\int _{{\mathbb {R}}^N}V(x)|\psi _{\delta ,y}|^{2}dx \nonumber \\&\le \Vert \psi _{\delta ,y}\Vert ^2+ \Vert V(x)\Vert _{L^{\frac{N}{2}}}\cdot \Vert \psi _{\delta ,y}\Vert _{L^{2^*}}^{2}. \end{aligned}$$
(4.35)

Recall that

$$\begin{aligned} \int _{{\mathbb {R}}^N}\big (I_\alpha *|\psi _{\delta ,y}|^{2_\alpha ^*}\big )|\psi _{\delta ,y}(x)|^{2_\alpha ^*}dx =C(N,\alpha )A_\alpha \Vert \psi _{\delta , y}\Vert _{L^{2^*}}^{2 2^*_\alpha } =S_\alpha ^{\frac{N+\alpha }{\alpha +2}}. \end{aligned}$$

By (4.35), we obtain that

$$\begin{aligned} t_{\psi }^{2\cdot 2_{\alpha }^{*}-2}\le 1+ \Vert V(x)\Vert _{L^{\frac{N}{2}}} \left( \frac{1}{C(N,\alpha )A_\alpha }\right) ^{\frac{N-2}{\alpha +2}}{S_{\alpha }^{-1}} = 1+\frac{\Vert V(x)\Vert _{L^{\frac{N}{2}}}}{S}, \end{aligned}$$
(4.36)

which implies that \(t_{\psi }^{2\cdot 2_{\alpha }^{*}}\le \left( 1+\frac{\Vert V(x)\Vert _{L^{\frac{N}{2}}}}{S}\right) ^{\frac{N+\alpha }{\alpha +2}}\). Since \(\Vert V(x)\Vert _{L^{\frac{N}{2}}}<(2^{\frac{\alpha +2}{N+\alpha }}-1)S\), we have

$$\begin{aligned} t_{\psi }^{2\cdot 2_{\alpha }^{*}}\le \left( 1+\frac{\Vert V(x)\Vert _{L^{\frac{N}{2}}}}{S}\right) ^{\frac{N+\alpha }{\alpha +2}}<2, \end{aligned}$$

which combining together with (4.34) and the fact that

$$\begin{aligned} m_{\infty }=m=\displaystyle \left( \frac{1}{2}-\frac{1}{2\cdot 2_{\alpha }^{*}}\right) \int _{{\mathbb {R}}^N}(I_{\alpha }*|\psi _{\delta ,y}|^{2_{\alpha }^{*}})|\psi _{\delta ,y}|^{2_{\alpha }^{*}}dx \end{aligned}$$

yields \(m<d<2m\).

We claim that there exists a nonnegative \((PS)_d\) sequence of I with \(d\in (m,2m)\). In fact, we can modify the energy functional I into

$$\begin{aligned} \widetilde{I}(u)=\frac{1}{2}\int _{{\mathbb {R}}^N}\big (|\nabla u|^{2}+V(x)u^{2}\big )dx -\frac{1}{2\cdot 2_{\alpha }^{*}}\int _{{\mathbb {R}}^N}\big (I_\alpha *|u^+|^{2_{\alpha }^*}\big )|u^+|^{2_{\alpha }^*}dx, \quad u\in \mathcal {D}^{1,2}({\mathbb {R}}^N). \end{aligned}$$

Suppose \(\{u_n\}\) is a \((PS)_d\) sequence of \(\widetilde{I}\) with \(d\in (m,2m)\), then \(\{u_n\}\) is bounded in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) and

$$\begin{aligned} \langle \widetilde{I}'(u_n),u_n^- \rangle =\Vert u_n^-\Vert ^2=o_n(1). \end{aligned}$$

It follows that

$$\begin{aligned} \widetilde{I}(u^+_n)\rightarrow d \in (m,2m), \quad \widetilde{I}'(u^+_n)\rightarrow 0. \end{aligned}$$

Thus, \(\{u_n^+\}\) is a nonnegative \((PS)_d\) sequence of \(\widetilde{I}\) with \(d\in (m,2m)\).

As a direct consequence of Corollary 4.2, up to a subsequence, we may suppose that \(u_n^+\rightarrow u\) strongly in \(\mathcal {D}^{1,2}({\mathbb {R}}^N)\) , and u is a nonnegative of (1.10). Since \(V(x)\in L^{\frac{N}{2}}({\mathbb {R}}^N)\cap \mathcal {C}^{\gamma }({\mathbb {R}}^N)\) is nonnegative for some \(\gamma \in (0,1)\), by a similar argument as the proof of Proposition 2.4, one can deduce that \(u\in \mathcal {C}^{2,\iota }({\mathbb {R}}^N)\) for some \(0<\iota <\gamma \). Then, the positivity of u follows from the strong maximum principle. Thus we complete the proof of the Theorem 1.2. \(\square \)