1 Introduction and main results

In this paper, we consider the following critical nonlinear Schrödinger equation involving the fractional Laplacian:

$$\begin{aligned} (-\Delta )^{s}u=\lambda u +\mu |u|^{q-2}u+|u|^{2_{s}^{*}-2}u&\ \ x\in \mathbb {R}^{N}, \end{aligned}$$
(1.1)

and possessing prescribed mass

$$\begin{aligned} \int _{\mathbb {R}^{N}}u^{2}dx=a^{2}, \end{aligned}$$
(1.2)

where \((-\Delta )^{s}\) is the fractional Laplacian, \(0<s<1,\ 2<q<2_{s}^{*}=2N/(N-2s)\) is a fractional critical Sobolev exponent. The fractional Laplacian \((-\Delta )^{s}\) is defined by

$$\begin{aligned} (-\Delta )^{s}u(x)=C(N,s) \mathrm{P.V.}\int _{\mathbb {R}^{N}}\frac{u(x)-u(y)}{|x-y|^{N+2s}}dy,\quad x\in \mathbb {R}^{N} \end{aligned}$$

for \(u \in C^\infty _0(\mathbb {R}^{N})\), where C(Ns) is a suitable positive normalizing constant and P.V. denotes the Cauchy principle value. We refer to [6, 15, 16, 19, 29, 35] for a simple introduction to basic properties of the fractional Laplace operator and concrete applications based on variational methods.

Our main driving force for the study of (1.1) arises in the study of the following time-dependent fractional Schrödinger equation with combined power nonlinearities:

$$\begin{aligned} i\psi _t-(-\Delta )^{s}\psi +\mu |\psi |^{q-2}\psi +|\psi |^{2_{s}^{*}-2}\psi =0\ \ \text{ in }\ \mathbb {R}^{N}. \end{aligned}$$
(1.3)

When searching for stationary waves of the form \(\psi (t,x)=e^{-i\lambda t}u(x)\), where \(\lambda \in \mathbb {R}\) is the chemical potential and \(u(x):\mathbb {R}^N\rightarrow \mathbb {C}\) is a time-independent function, one is led to studying (1.1). In this case, particular attention is paid to ground state solutions, a.e., solutions minimizing an energy functional among all non-trivial solutions. An alternative choice is to look for solutions to (1.1) having prescribed mass, and in this case \(\lambda \in \mathbb {R}\) is part of the unknown. This approach seems particularly meaningful from the physical point of view, since, in addition to being a conserved quantity for the time dependent (1.3), the mass has often an evident physical meaning, for example, it indicates the power supply in nonlinear optics, or the total number of atoms in Bose-Einstein condensation. Moreover, this approach gives a better insight of the properties of the stationary solutions for (1.1), for example, stability or instability, see [31] for more details.

The existence of normalized stationary states can be summarized as follows: given \(a>0\) and \(\mu \in \mathbb {R}\), \(2<q<2_s^*\), our aim is to find \((\lambda , u)\in \mathbb {R}\times H^{s}(\mathbb {R}^N, \mathbb {C})\) such that (1.1) and (1.2) hold. For the Laplacian case, i.e., \(s=1\) in (1.1), we would like to mention a seminal paper by Jeanjean in [26], which dealt with the existence of normalized solutions when the energy function is unbounded from below on the \(L^{2}\) constraint. In fact, the normalized solutions for nonlinear Schrödinger equation or system have attracted much attention in recent years, both for their interesting theoretical structure and their concrete applications (see [1,2,3, 31, 32] and references therein). Since \(\lambda \) and \(\mu \) are parts of the unknown, the Nehari manifold method is not available in the framework of normalized solutions. Meanwhile, the appearance of the \(L^{2}\) constraint makes some classical methods, used to prove the boundedness of any Palais–Smale sequence for the unconstrained problem, difficult to implement. It is well known that a new \(L^{2}\)-critical exponent \(\widetilde{p}=2+4/N\) plays a special role. Indeed, if the problem is \(L^{2}\)-subcritical, i.e., \(2<q<p<\widetilde{p}\), the energy functional \(E_{\mu }\) (defined in (1.4)) is bounded from below on the constraint \(\overline{S}_{a}=\{u\in H^1(\mathbb {R}^N, \mathbb {C}): \int _{\mathbb {R}^{N}}u^{2}dx=a^{2}\}\), so the ground state solution can be found as global minimizers of \(E_{\mu }|_{\overline{S}_{a}}\). Moreover, if the problem is \(L^{2}\)-supercritical, i.e., \(\widetilde{p}<q<p<2^{*}=2N/(N-2)\), then the energy functional \(E_{\mu }\) is unbounded both from above and from below on \(\overline{S}_{a}\). In this case, the ideas introduced by Jeanjean in [26] can be employed to consider the existence of normalized solutions for any \(a,\mu >0.\)

Compared to the semilinear case that corresponds to the Laplace operator, the fractional Laplacian problems are nonlocal and more challenging. For fractional Laplacian equations or systems with fixed \(\lambda _{i}\), the existence and non-degeneracy of solutions have been studied by a lot of researchers and there are many results about existence, nonexistence, multiplicity of solutions for fractional Laplacian equation, since it seems almost impossible for us to provided a complete list of references, we just refer the readers to [5, 9,10,11, 13, 14, 20, 21, 23,24,25, 36,37,38] and references therein.

Recently, Soave in [31, 32] first investigated the existence and properties of ground states for the nonlinear Schrödinger equation with combined power type nonlinearities and also gave new criteria for global existence and finite time blow-up in the associated dispersive equation. More precisely, Soave in [31] considered the normalized solutions for subcritical exponent and give a complete classification about the existence and nonexistence of normalized solution for \(L^{2}\)-subcritical, \(L^{2}\)-critical and \(L^{2}\)-supercritical. For the critical case, the problem is also interesting and challenging. By focusing the leading nonlinearity and analysing how the introduction of lower order term modifies the energy functional structure, Soave in [32] obtained the existence and nonexistence of normalized solutions for \(L^{2}\)-subcritical, \(L^{2}\)-critical and \(L^{2}\)-supercritical in the Sobolev critical case. Due to the lack of compactness of the Sobolev embedding \(H^{1}(\mathbb {R}^{N})\hookrightarrow L^{2^{*}}(\mathbb {R}^{N})\), the problem is more complicated, however, the difficulty was overcome ingeniously by combining some ideas from [8, 26].

Inspired by the above-mentioned works, especially by [31, 32], in the present paper our goal is two-fold. One is to show the existence and nonexistence of normalized ground states for fractional elliptic equations with critical exponent. Another is to give some results about the behavior of ground state solutions obtained above as \(\mu \rightarrow 0^{+}\). The method we use is Jeanjean’s method [26] combined with Pohozaev manifold argument. By using the test function as in [30], we show that the least energy of the equation is below the critical energy \(\frac{s}{N}S^{N/(2s)}_{s}\) under the proper conditions given on \(N, s,p,\lambda \) under which the Palais–Smale condition is satisfied. The main difficulty is to prove the convergence of constrained Palais–Smale sequence. Indeed, if we find a bounded Palais–Smale sequence, according to the compactness of the embedding \(H^{s}_{rad}(\mathbb {R}^{N})\hookrightarrow L^{p}(\mathbb {R}^{N}),\ 2<p<2_{s}^{*}\), we just get a strongly convergent subsequence in \(L^{p}(\mathbb {R}^{N})\), but we cannot deduce the strong convergence in \(L^{2}(\mathbb {R}^{N}).\) Hence we require new arguments to overcome the lack of compactness of the embedding \(H^{s}_{rad}(\mathbb {R}^{N})\hookrightarrow L^{2}(\mathbb {R}^{N}).\) To this end, we adopt some ideas of [18] to obtain a Liouville-type result.

Before we state our main results, we first introduce some notations. Let \({H^{s}(\mathbb {R}^{N})}\) be the Hilbert space of function in \(\mathbb {R}^N\) endowed with the standard inner product and norm

$$\begin{aligned} \langle u,v\rangle =\int _{\mathbb {R}^{N}}\left( (-\Delta )^{\frac{s}{2}} u(-\Delta )^{\frac{s}{2}}v+uv\right) dx,\ \ \Vert u\Vert _{H^{s}(\mathbb {R}^{N})}^{2}=\langle u,u\rangle . \end{aligned}$$

Let \(D_{s}(\mathbb {R}^N)\) be the Hilbert space defined as the completion of \(C_{c}^{\infty }(\mathbb {R}^N)\) with the inner product

$$\begin{aligned} \langle u,v\rangle _{D_{s}(\mathbb {R}^N)}=\frac{C(N,s)}{2}\iint _{\mathbb {R}^{2N}}\frac{(u(x)-u(y))(v(x)-v(y))}{|y-x|^{N+2s}}dxdy \end{aligned}$$

and norm

$$\begin{aligned} \Vert u\Vert ^{2}_{D_{s}(\mathbb {R}^{N})}=\int _{\mathbb {R}^{N}}|(-\Delta )^{\frac{s}{2}}u|^{2}dx=\frac{C(N,s)}{2}\iint _{\mathbb {R}^{2N}}\frac{|u(x)-u(y)|^{2}}{|y-x|^{N+2s}}dxdy. \end{aligned}$$

The energy functional associated with (1.1) and the constraint are given by

$$\begin{aligned} E_{\mu }(u)&=\frac{1}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})} -\frac{\mu }{q}\int _{\mathbb {R}^{N}}|u|^{q}dx-\frac{1}{2_{s}^{*}}\int _{\mathbb {R}^{N}}|u|^{2_{s}^{*}}dx \end{aligned}$$
(1.4)

and

$$\begin{aligned} S_{a}=\left\{ u\in H^s(\mathbb {R}^N, \mathbb {C}): \int _{\mathbb {R}^{N}}u^{2}dx=a^{2}\right\} . \end{aligned}$$

Let \(S_s\) be the sharp imbedding constant of \({D_{s}(\mathbb {R}^{N})}\hookrightarrow L^{2^{*}}(\mathbb {R}^{N}),\)

$$\begin{aligned} S_{s}=\inf \limits _{u\in {D_{s}(\mathbb {R}^{N})})\setminus \{0\}}\frac{\Vert u\Vert ^{2}_{D_{s}(\mathbb {R}^{N})}}{(\int _{\mathbb {R}^{N}}|u|^{2^{*}}dx)^{\frac{2}{2^{*}}}}. \end{aligned}$$
(1.5)

From [14] we know that \(S_{s}\) is attained in \(\mathbb {R}^{N}\) by

$$\begin{aligned} U_{\epsilon ,x_{0}}(x)=\kappa (\epsilon ^{2}+|x-x_{0}|^{2})^{-\frac{N-2s}{2}} \end{aligned}$$
(1.6)

where \(\kappa \ne 0 \in \mathbb {R},\ \varepsilon >0 \) are fixed constants and \(x_{0}\in \mathbb {R}^{N}\).

To present our main results, put

$$\begin{aligned} \gamma _{q,s}=&\frac{N(q-2)}{2qs},\nonumber \\ C'=&\frac{q(2_{s}^{*}-2)}{2C^{q}_{N,q,s}(2_{s}^{*}-q\gamma _{q,s})}\left( \frac{(2-q\gamma _{q,s})2_{s}^{*}}{2(2_{s}^{*}-q\gamma _{q,s})}S^{\frac{2_{s}^{*}}{2}}_{s}\right) ^{\frac{2-q\gamma _{q,s}}{2_{s}^{*}-2}}, \end{aligned}$$
(1.7)
$$\begin{aligned} C''=&\frac{22_{s}^{\star }}{(2_{s}^{\star }-q\gamma _{q,s})C^{q}_{N,q,s}}\left( \frac{Nq\gamma ^{2}_{q,s}S^{\frac{N}{2s}}_{s}}{(2-q\gamma _{q,s})s}\right) ^{\frac{2-q\gamma _{q,s}}{2}}. \end{aligned}$$
(1.8)

Theorem 1.1

Let \(N>2s,\ a,\mu >0\) and \(2<q<\overline{p}:=2+4s/N\). If there exists a constant \(\alpha =\alpha (N,q)>0\) such that

$$\begin{aligned} \mu a^{q(1-\gamma _{q,s})}<\alpha :=\min \{C', C''\}, \end{aligned}$$
(1.9)

then \(E_{\mu }|_{S_{a}}\) has a ground state \(\widetilde{u}\) with the following properties: \(\widetilde{u}\) is a positive, radially symmetric function and solves (1.1)–(1.2) for some \(\widetilde{\lambda }<0.\) Moreover, \(m(a,\mu )<0\) and \(\widetilde{u}\) is an interior local minimizer of \(E_{\mu }(u)\) on the set

$$\begin{aligned} A_{k}=\{u\in S_{a}: ||u||_{D_{s}(\mathbb {R}^{N})}< k\}, \end{aligned}$$

for suitable k small enough. Any other ground state solution of \(E_{\mu }\) on \(S_{a}\) is a local minimizer of \(E_{\mu }\) on \(A_{k}\).

Theorem 1.2

Let \(N>2s,\ a,\mu >0\) and \(2<q=\overline{p}\). If

$$\begin{aligned} \mu a^{\frac{4s}{N}}< \overline{p}\left( 2C^{\overline{p}}_{N,\overline{p},s}\right) ^{-1}, \end{aligned}$$
(1.10)

then \(E_{\mu }|_{S_{a}}\) has a ground state \(\widetilde{u}\) with the following properties: \(\widetilde{u}\) is a positive, radially symmetric function and solves (1.1)–(1.2) for some \(\widetilde{\lambda }<0.\) Moreover, \(0<m(a,\mu )<\frac{s}{N}S^{N/(2s)}_{s},\) and \(\widetilde{u}\) is a critical point of Mountain Pass type.

Theorem 1.3

Let \(N>2s,\ a,\mu >0\) and \(\overline{p}<q<2_{s}^{*}\). If one of the following conditions holds:

  1. (1)

    \( N>4s\ \text {and}\ \mu a^{q(1-\gamma _{q,s})}<\frac{S^{\frac{N}{4s}q(1-\gamma _{q,s})}_{s}}{\gamma _{q,s}},\)

  2. (2)

    \( N=\frac{q}{q-1}2s\ \text {and}\ \mu a^{q(1-\gamma _{q,s})}<\frac{S^{\frac{N}{4s}q(1-\gamma _{q,s})}_{s}}{\gamma _{q,s}},\)

  3. (3)

    \(N=4s \ \text {or}\ \frac{q}{q-1}2s<N<4s\ \text {or}\ 2s<N<\frac{q}{q-1}2s, \)

then \(E_{\mu }|_{S_{a}}\) has a ground state \(\widetilde{u}\) with the following properties: \(\widetilde{u}\) is a positive, radially symmetric function and solves (1.1)–(1.2) for some \(\widetilde{\lambda }<0.\) Moreover, \(0<m(a,\mu )<\frac{s}{N}S^{N/(2s)}_{s},\) and \(\widetilde{u}\) is a critical point of Mountain Pass type.

Theorem 1.4

Let \(a>0\) and \(\mu =0\). Then we have the following conclusions:

  1. (1)

    If \(N>4s\), then \(E_{0}\) on \(S_{a}\) has a unique positive radial ground state \(U_{\epsilon ,0}\) defined in (1.6) for the unique choice of \(\epsilon >0\) which gives \(||U_{\epsilon ,0}||_{L^{2}(\mathbb {R}^{N})}=a.\)

  2. (2)

    If \(2s<N\le 4s\), then (1.1) has no positive solutions in \(S_{a}\) for any \(\lambda \in \mathbb {R}\).

Theorem 1.5

Let \(u_{\mu }\) be the corresponding positive ground state solution obtained in Theorems 1.11.3 with energy level \(m(a,\mu )\). Then the following conclusions hold:

  1. (1)

    If \(2<q<\overline{p}\), then \(m(a,\mu )\rightarrow 0\), and \(\Vert u_{\mu }\Vert ^{2}_{D_{s}(\mathbb {R}^{N})}\rightarrow 0\) as \(\mu \rightarrow 0^{+}\).

  2. (2)

    If \(\overline{p}\le q<2^{*}\), then \(m(a,\mu )\rightarrow \frac{s}{N}S^{\frac{N}{2s}}_{s}\), and \(\Vert u_{\mu }\Vert ^{2}_{D_{s}(\mathbb {R}^{N})}\rightarrow S^{\frac{N}{2s}}_{s}\) as \(\mu \rightarrow 0^{+}\).

Remark 1.1

The assumptions (1.7), (1.8) and (1.10) are used to describe the geometry of \(E_{\mu }\). Meanwhile, the assumptions in Theorem 1.3 are applied to overcome the lack of compactness.

Remark 1.2

We should point out that Luo and Zhang in [28] considered the subcritical fractional equation with combined nonlinearities and proved the existence and nonexistence of normalized solutions, however, in this paper we consider the existence and nonexistence of normalized solutions for the critical fractional equation with combined nonlinearities. Compared with the subcritical case, the critical case is more complicated and needs to overcome the lack of compactness.

In this paper, we invoke some ideas proposed by Soave in [31, 32]. Compared to the Laplacian problems, the fractional Laplacian problems are nonlocal and more challenging. Indeed, when we consider the fractional Laplacian problem, the corresponding algebraic equation is about fractional order, which is more complicated to deal with than an integer-order algebraic equation. Moreover, one of the main difficulties is to analyze the convergence of constrained Palais–Smale sequence. To overcome the lack of compactness, we employ delicate methods which can be found in [8], that is, cut-off technique and energy estimate. To show the least energy strictly less than “the threshold energy”, our analysis is more difficult and complicated. Indeed, when we deal with the \(L^{2}\)-critical and \(L^{2}\)-supercritical case, we need to give an exact classification of dimensions N, which depends on s and q, and give exact estimate for energy function in each cases. In particular, for \(2s<N<4s\) and \(\mu =0\), to show Theorem 1.4 (2), we need to prove that all solutions for equation \((-\Delta )^{s}v=v^{2_{s}^{*}-1}, \ v\ge 0\ \text {in}\ \mathbb {R}^{N},\) must be \(\alpha U_{\epsilon ,0}\) for some \(\alpha ,\epsilon >0\).

Finally, let us sketch the proof of above theorems. In general, this study can be considered as a counterpart of the fractional Brézis-Nirenberg problem in the context of normalized solutions. To overcome the lack of compactness which is a crucial step for the critical case, we show that the least energy strictly less than “the threshold energy”, we employ delicate methods which can be found in [8], that is, cut-off technique and energy estimate. The convergence of Palais–Smale sequence (see Proposition 2.2) is one of the most delicate ingredient in the proofs of our main results. We introduce a fiber maps \(\Psi ^{\mu }_{u}(t)\) [see (2.9)], it is well known that any critical point of \(E_{\mu }|_{S_{a}}\) stays in \(\mathcal {P}_{a,\mu }\) [see (2.5)], the monotonicity and convexity properties of \(\Psi ^{\mu }_{u}(t)\) strongly affect the structure of \(\mathcal {P}_{a,\mu }\). It is easy to see that \((\Psi ^{\mu }_{u})'(t)=P_{\mu }(t\star u)\), so that t is a critical point of \(\Psi ^{\mu }_{u}(t)\) if and only if \(t\star u\in \mathcal {P}_{(a,\mu )}\) and in particular \(u\in \mathcal {P}_{(a,\mu )}\) if and only if 0 is a critical point of \(\Psi ^{\mu }_{u}(t)\). In this spirit, we split \(\mathcal {P}_{a,\mu }\) into three parts, then we prove that \(\mathcal {P}^{0}_{a,\mu }=\emptyset \) and \(\mathcal {P}_{a,\mu }\) is a smooth manifold of codimension 1 in \(S_{a}\) under suitable conditions. For \(L^{2}\)-subcritical case, we restricted the energy function \(E_{\mu }\) on the \(\mathcal {P}_{a,\mu }\) and we can prove that \(E_{\mu }|_{\mathcal {P}_{a,\mu }}\) is bounded from below, so a local minimizer \(\widetilde{u}\) for \(E_{\mu }\) on the \(\mathcal {P}_{a,\mu }\) can be obtained. For \(L^{2}\)-critical/supercritical, we construct different linking structures to obtain the Mountain Pass type solutions.

The paper is organized as follows. In Sect. 2, we introduce some preliminaries that will be used to prove Theorems 1.11.3. In Sect. 3, we give some lemmas for \(L^{2}\)-subcritical perturbation. In Sect. 4, we give some preliminaries for \(L^{2}\)-critical perturbation. In Sect. 5, we give some lemmas for \(L^{2}\)-supercritical perturbation. In Sect. 6, we prove Theorem 1.1. In Sect. 7, we prove Theorems 1.21.3. In Sect. 8, we prove Theorem 1.4. Finally, the proof of Theorem 1.5 will be given in Sect. 9.

2 Preliminaries

Let \(S_s\) be the sharp embedding constant of \({D^{s}(\mathbb {R}^{N})}\hookrightarrow L^{2_{s}^{*}}(\mathbb {R}^{N}),\)

$$\begin{aligned} S_{s}=\inf \limits _{u\in {D^{s}(\mathbb {R}^{N})})\setminus \{0\}}\frac{\Vert u\Vert ^{2}_{D^{s}(\mathbb {R}^{N})}}{(\int _{\mathbb {R}^{N}}|u|^{2_{s}^{*}}dx)^{\frac{2}{2_{s}^{*}}}}. \end{aligned}$$
(2.1)

from [14] \(S_{s}\) is attained in \(\mathbb {R}^{N}\) by \(\widetilde{u}(x)=\kappa (\varepsilon ^{2}+|x-x_{0}|^{2})^{-\frac{N-2s}{2}}\), where \(\kappa \ne 0 \in \mathbb {R},\ \varepsilon >0 \) are fixed constants and \(x_{0}\in \mathbb {R}^{N}\).

It is useful to introduce the fractional Gagliardo–Nirenberg–Sobolev inequality (see[21])

$$\begin{aligned} \int _{\mathbb {R}^{N}}|u|^{p}dx\le & {} C_{N,p,s}\left( \int _{\mathbb {R}^{N}}|(-\Delta )^{\frac{s}{2}}u|^{2}dx\right) ^{\frac{N(p-2)}{4s}}\nonumber \\&\left( \int _{\mathbb {R}^{N}}|u|^{2}dx\right) ^{\frac{p}{2}-\frac{N(p-2)}{4s}}\ \ \text {for all} \ \ u\in H^{s}(\mathbb {R}^{N}). \end{aligned}$$
(2.2)

Define

$$\begin{aligned} \gamma _{p,s}=\frac{N(p-2)}{2ps}, \end{aligned}$$

it is easy to see that

$$\begin{aligned} p\gamma _{p,s}\left\{ \begin{array}{ll}<{\displaystyle 2},&{} \text {if} \ 2<p<\overline{p},\\ ={\displaystyle 2},&{} \text {if} \ p=\overline{p},\\ >{\displaystyle 2},&{} \text {if} \ \overline{p}<p<2_{s}^{*}, \end{array}\right. \text {and that}\ \ \gamma _{2_{s}^{*}}=1, \end{aligned}$$
(2.3)

and

$$\begin{aligned} \Vert u\Vert _{L^{p}}\le C_{N,p,s}\Vert (-\Delta )^{s}u\Vert ^{\gamma _{p,s}}_{L^{2}}\Vert u\Vert ^{1-\gamma _{p,s}}_{L^{2}}\ \text {for all} \ \ u\in H^{s}(\mathbb {R}^{N}). \end{aligned}$$
(2.4)

We first give the following key Pohozaev identity for the fractional Laplace operator.

Proposition 2.1

[Theorem A.1 in [7]] Let \(u\in H^{s}(\mathbb {R}^{N})\bigcap L^{\infty }(\mathbb {R}^{N})\) be a positive solution of \((-\Delta )^{s}u= f(u)\) and \(F(u)\in L^{1}(\mathbb {R}^{N})\), then it holds that

$$\begin{aligned} \frac{N-2s}{2}\int _{\mathbb {R}^{N}}|(-\Delta )^{\frac{s}{2}}u|^{2}dx=N\int _{\mathbb {R}^{N}}F(u)dx, \end{aligned}$$

where \(F(u)=\int ^{u}_{0}f(t)dt\).

Remark 2.1

Since \(u\in H^{s}(\mathbb {R}^{N})\), by the fractional Sobolev embedding theorem (see [33, Theorem 2.2]), it is easy to see \(u\in L^{p}(\mathbb {R}^{N}),\ p\in [2,2_{s}^{*}],\) which implies that \(F(u)=\frac{\lambda }{2} u^{2} +\frac{\mu }{q}|u|^{q}+\frac{1}{2_{s}^{*}}|u|^{2_{s}^{*}}\in L^{1}(\mathbb {R}^{N})\), hence we can modify the proof of Proposition 5.1 in [4] to obtain that \(u\in L^{\infty }B(0,\frac{r}{2})\). Using the same arguments for a neighborhood of any \(x\in \mathbb {R}^{N}\), we get \(u\in L_{loc}^{\infty }(\mathbb {R}^{N})\). Thus we can use similar arguments as in the proof of Theorem 3.4 in [17] to obtain that \(u\in L^{\infty }(\mathbb {R}^{N})\), which implies that the above Pohozaev identity can be applied to our equation. In fact, similar method to prove \(u\in L^{\infty }(\mathbb {R}^{N})\) can also be used to prove Proposition 4.1 in [12].

Lemma 2.1

Let \(u\in H^{s}(\mathbb {R}^{N})\) be a solution of (1.1), then

$$\begin{aligned} \mathcal {P}_{a,\mu }=\{u\in S_{a}:P_{\mu }(u)=0\}, \end{aligned}$$
(2.5)

where

$$\begin{aligned} P_{\mu }(u)=s||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \gamma _{q,s}s\int _{\mathbb {R}^{N}} |u|^{q}dx-s\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx. \end{aligned}$$

Proof

From Proposition 2.1, we have

$$\begin{aligned} \frac{N-2s}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}=\lambda \frac{ N}{2}\int _{\mathbb {R}^{N}} u^{2}dx+\frac{ N\mu }{q}\int _{\mathbb {R}^{N}} |u|^{q}dx+\frac{ N }{2_{s}^{*}}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx. \end{aligned}$$
(2.6)

Since u is a solution of (1.1), we have

$$\begin{aligned} ||u||^{2}_{D_{s}(\mathbb {R}^{N})}=\lambda \int _{\mathbb {R}^{N}} u^{2}dx+\mu \int _{\mathbb {R}^{N}} |u|^{q}dx+\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx. \end{aligned}$$
(2.7)

Combining (2.6) with (2.7), we obtain

$$\begin{aligned} s||u||^{2}_{D_{s}(\mathbb {R}^{N})}=\mu \gamma _{q,s}s\int _{\mathbb {R}^{N}}|u|^{q}dx+s\int _{\mathbb {R}^{N}}|u|^{2_{s}^{*}}dx. \end{aligned}$$

As desired.\(\square \)

Define

$$\begin{aligned} (t\star u)(x)=e^{\frac{Nt}{2}}u(e^{t}x)\ \text {for a.e. } x\in \mathbb {R}^{N}, \end{aligned}$$
(2.8)

it is easy to see that \(t\star u\in S_{a}.\) We define the fiber map as follows:

$$\begin{aligned} \Psi ^{\mu }_{u}(t)=E_{\mu }(t\star u)=\frac{e^{2st}}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \frac{e^{q\gamma _{q,s}st}}{q}\int _{\mathbb {R}^{N}} |u|^{q}dx-\frac{e^{2_{s}^{*}st}}{2_{s}^{*}}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx. \end{aligned}$$
(2.9)

It is easy to see that \((\Psi ^{\mu }_{u})'(t)=P_{\mu }(t\star u)\), so that t is a critical point of \(\Psi ^{\mu }_{u}(t)\) if and only if \(t\star u\in \mathcal {P}_{(a,\mu )}\) and in particular \(u\in \mathcal {P}_{(a,\mu )}\) if and only if 0 is a critical point of \(\Psi ^{\mu }_{u}(t)\).

We split \(\mathcal {P}_{a,\mu }\) into three parts.

$$\begin{aligned} \mathcal {P}^{+}_{a,\mu }&=\bigg \{u\in \mathcal {P}_{a,\mu }\mid (\Psi ^{\mu }_{u})''(0)>0\bigg \}\nonumber \\&=\bigg \{u\in \mathcal {P}_{a,\mu }\mid 2s^{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})} >\mu q\gamma ^{2}_{q,s}s^{2}\int _{\mathbb {R}^{N}} |u|^{q}dx+2_{s}^{*}s^{2}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\bigg \},\nonumber \\ \mathcal {P}^{0}_{a,\mu }&=\bigg \{u\in \mathcal {P}_{a,\mu }\mid (\Psi ^{\mu }_{u})''(0) =0\bigg \}\nonumber \\&=\bigg \{u\in \mathcal {P}_{a,\mu }\mid 2s^{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})} =\mu q\gamma ^{2}_{q,s}s^{2}\int _{\mathbb {R}^{N}} |u|^{q}dx+2_{s}^{*}s^{2}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\bigg \},\nonumber \\ \mathcal {P}^{-}_{a,\mu }&=\bigg \{u\in \mathcal {P}_{a,\mu }\mid (\Psi ^{\mu }_{u})''(0)<0\bigg \}\nonumber \\&=\bigg \{u\in \mathcal {P}_{a,\mu }\mid 2s^{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}< \mu q\gamma ^{2}_{q,s}s^{2}\int _{\mathbb {R}^{N}} |u|^{q}dx+2_{s}^{*}s^{2}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\bigg \}. \end{aligned}$$
(2.10)

It is easy to see that

$$\begin{aligned} \mathcal {P}_{a,\mu }=\mathcal {P}^{+}_{a,\mu }\cup \mathcal {P}^{0}_{a,\mu }\cup \mathcal {P}^{-}_{a,\mu }. \end{aligned}$$

Lemma 2.2

Let \(N>2s,\ 2<q<2_{s}^{*}\) and \(a,\mu >0\). Let \(\{u_{n}\}\subset S_{a,r}=S_{a}\cap H^{s}(\mathbb {R}^{N})\) be a Palais–Smale sequence for \(E_{\mu }|_{S_{a}}\) at level \(m(a,\mu )\). Then \(\{u_{n}\}\) is bounded in \(H^{s}(\mathbb {R}^{N})\).

Proof

Case 1 \(q<\overline{p}\). This yields that \(\gamma _{q,s}q<2\). Since \(P_{\mu }(u_{n})\rightarrow 0 \), we have

$$\begin{aligned} s||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \gamma _{q,s}s\int _{\mathbb {R}^{N}} |u|^{q}dx-s\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx=o_{n}(1). \end{aligned}$$

Thus, by fractional Gagliardo–Nirenberg–Sobolev inequality (2.4), we have

$$\begin{aligned} E_{\mu }(u_{n})&=\frac{s}{N}||u_{n}||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{\mu }{q}\left( 1-\frac{q\gamma _{q,s}}{2_{s}^{*}}\right) \int _{\mathbb {R}^{N}} |u_{n}|^{q}dx+o_{n}(1)\\&\ge \frac{s}{N}||u_{n}||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{\mu }{q}\left( 1-\frac{q\gamma _{q,s}}{2_{s}^{*}}\right) C^{q}_{N,q,s}||u_{n}||^{q\gamma _{q,s}}_{D_{s}(\mathbb {R}^{N})}a^{q(1-\gamma _{q,s})}. \end{aligned}$$

Since \(\{u_{n}\}\) is a Palais–Smale sequence for \(E_{\mu }|_{S_{a}}\) at level \(m(a,\mu )\), we have \(E_{\mu }(u_{n})\le m+1\) for n large. Hence

$$\begin{aligned} \frac{s}{N}||u_{n}||^{2}_{D_{s}(\mathbb {R}^{N})}\le \frac{\mu }{q}\left( 1-\frac{q\gamma _{q,s}}{2_{s}^{*}}\right) C^{q}_{N,q,s}||u_{n}||^{q\gamma _{q,s}}_{D_{s}(\mathbb {R}^{N})}a^{q(1-\gamma _{q,s})}+m(a,\mu )+2 , \end{aligned}$$

which implies that \(\{u_{n}\}\) is bounded in \(H^{s}(\mathbb {R}^{N})\).

Case 2 \(q=\overline{p}\). Then \(\gamma _{\overline{p},s}\overline{p}=2\). Since \(P_{\mu }(u_{n})\rightarrow 0 \), we know

$$\begin{aligned} ||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \gamma _{\overline{p},s}\int _{\mathbb {R}^{N}} |u|^{\overline{p}}dx-\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx=o_{n}(1). \end{aligned}$$
(2.11)

Thus,

$$\begin{aligned} E_{\mu }(u_{n})=\frac{s}{N}\int _{\mathbb {R}^{N}} |u_{n}|^{2_{s}^{*}}dx+o_{n}(1)\le +m(a,\mu )+1\Rightarrow \int _{\mathbb {R}^{N}} |u_{n}|^{2_{s}^{*}}dx\le C. \end{aligned}$$

Since \(q\in (2,2_{s}^{*})\), we have \(q=\alpha 2+(1-\alpha )2_{s}^{*}\) for suitable \(\alpha \in (0,1)\), so by Hölder’s inequality, we have

$$\begin{aligned} \int _{\mathbb {R}^{N}} |u_{n}|^{q}dx\le \left( \int _{\mathbb {R}^{N}} |u_{n}|^{2}dx\right) ^{\alpha }\left( \int _{\mathbb {R}^{N}} |u_{n}|^{2_{s}^{*}}dx\right) ^{1-\alpha }\le C. \end{aligned}$$

Thus, from (2.11), we know that

$$\begin{aligned} ||u||^{2}_{D_{s}(\mathbb {R}^{N})}=\mu \gamma _{\overline{p},s}\int _{\mathbb {R}^{N}} |u|^{\overline{p}}dx+\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\le C. \end{aligned}$$

Case 3 \(\overline{p}<q<2_{s}^{*}\). This implies that \(\gamma _{q,s}q>2\). Since \(P_{\mu }(u_{n})\rightarrow 0 \), we know

$$\begin{aligned} ||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx-\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx=o_{n}(1). \end{aligned}$$

Thus

$$\begin{aligned} E_{\mu }(u_{n})=\frac{\mu }{q}\left( \frac{\gamma _{q,s}q}{2}-1\right) \int _{\mathbb {R}^{N}} |u|^{q}dx+\frac{s}{N}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\le m(a,\mu )+1. \end{aligned}$$

So \(\int _{\mathbb {R}^{N}} |u|^{q}dx\) and \(\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\) are both bounded. Hence

$$\begin{aligned} ||u||^{2}_{D_{s}(\mathbb {R}^{N})}=\mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx+\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx+o_{n}(1)\le C. \end{aligned}$$

This completes the proof.\(\square \)

Proposition 2.2

Let \(N>2s, 2<q<2_{s}^{*}\) and \(a,\mu >0\). Let \(\{u_{n}\}\subset S_{a,r}=S_{a}\bigcap H^{s}(\mathbb {R}^{N})\) be a Palais–Smale sequence for \(E_{\mu }|_{S_{a}}\) at level \(m(a,\mu )\) with

$$\begin{aligned} m(a,\mu )<\frac{s}{N}S^{\frac{N}{2s}}_{s}\ \text {and}\ m\ne 0. \end{aligned}$$

Suppose in addition that \(\mathcal {P}_{a,\mu }(u_{n})\rightarrow 0 \ \text {as} \ n\rightarrow +\infty .\) Then one of the following alternatives holds:

  1. (i)

    either up to a subsequence \(u_{n}\rightharpoonup u\) weakly in \(H^{s}(\mathbb {R}^{N})\) but not strongly , where \(u\not \equiv 0\) is a solution of (1.1) for some \(\lambda <0\), and

    $$\begin{aligned} E_{\mu }(u)\le m(a,\mu )- \frac{s}{N}S^{\frac{N}{2s}}_{s}. \end{aligned}$$
  2. (ii)

    or up to a subsequence \(u_{n}\rightarrow u\) strongly in \(H^{s}(\mathbb {R}^{N}),\) \(E_{\mu }(u)=m(a,\mu )\) and u solves (1.1)–(1.2) for some \(\lambda <0.\)

Proof

By Lemma 2.2, we know that the sequence \(\{u_{n}\}\) is bounded in \(H^{s}(\mathbb {R}^{N})\), which is radial functions, and by compactness of \(H_{rad}^{s}(\mathbb {R}^{N})\hookrightarrow \hookrightarrow L^{q}(\mathbb {R}^{N})\), which implies that

$$\begin{aligned} u_{n}\rightharpoonup u \ \text {in}\ H^{s}(\mathbb {R}^{N}),\ \ u_{n}\rightarrow u \ \text {in}\ L^{q}(\mathbb {R}^{N}) \ a.e\ \text {in}\ \mathbb {R}^{N} . \end{aligned}$$

Since \(\{u_{n}\}\) is a bounded Palais–Smale sequence for \(E_{\mu }|_{S_{a}}\) at level \(m(a,\mu )\), by Lagrange multipliers rule, there exists \(\{\lambda _{n}\}\subset \mathbb {R}\) such that for every \(\varphi \in H^{s}(\mathbb {R}^{N}) \)

$$\begin{aligned}&\int _{\mathbb {R}^{N}} \left( (-\Delta )^{\frac{s}{2}}u (-\Delta )^{\frac{s}{2}}\varphi dx-\lambda _{n}u_{n}\varphi -\mu |u|^{q-2}u_{n}\varphi -|u|^{2_{s}^{*}-2}u_{n}\varphi \right) dx=o_{n}(1)\Vert \varphi \Vert ,\nonumber \\&\quad \ \text {as}\ n\rightarrow +\infty . \end{aligned}$$
(2.12)

If we choose that \(\varphi =u_{n}\), from (2.12), it is easy to see that \(\{{u_{n}}\}\) is bounded, hence up to a subsequence \(\lambda _{n}\rightarrow \lambda \in \mathbb {R} \). By the fact that \(P_{\mu }(u_{n})\rightarrow 0 \) and \(\gamma _{q,s}<1\), we deduce that

$$\begin{aligned} \lambda a^{2}&=\lim _{n\rightarrow +\infty }\lambda _{n}\int _{\mathbb {R}^{N}} |u_{n}|^{2}dx=\lim _{n\rightarrow +\infty }\left( ||u_{n}||^{2}_{D_{s}(\mathbb {R}^{N})}-\int _{\mathbb {R}^{N}}\left( \mu |u_{n}|^{q}+|u_{n}|^{2_{s}^{*}}\right) dx\right) \nonumber \\&=\lim _{n\rightarrow +\infty }\mu (\gamma _{q,s}-1)\int _{\mathbb {R}^{N}}|u_{n}|^{q}dx=\mu (\gamma _{q,s}-1)\int _{\mathbb {R}^{N}}|u|^{q}dx\le 0. \end{aligned}$$
(2.13)

It is easy to see that \(\lambda =0\) if and only if \(u\equiv 0\). Next, we show that the \(u\not \equiv 0.\) Assume by contradiction that \(u\equiv 0\), by \(\{u_{n}\}\) is bounded in \(H^{s}(\mathbb {R}^{N})\), hence up to a subsequence \(||u_{n}||^{2}_{D_{s}(\mathbb {R}^{N})}\rightarrow \ell \in \mathbb {R}\). From \(P_{\mu }(u_{n})\rightarrow 0\) and \(u_{n}\rightarrow 0\) strongly in \(L^{q}(\mathbb {R}^{N})\), hence

$$\begin{aligned} \int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx=||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx\rightarrow \ell , \end{aligned}$$

Therefore, by the definition of \(S_{s}\) in (1.5), we have \(\ell \ge S_{s}\ell ^{\frac{2}{2_{s}^{*}}}\), we can deduce

$$\begin{aligned} \ell =0\ \text { or}\ \ell \ge S^{\frac{N}{2s}}_{s}. \end{aligned}$$

Case 1 If \(||u_{n}||^{2}_{D_{s}(\mathbb {R}^{N})}\rightarrow \ell =0\), then \(\int _{\mathbb {R}^{N}}|u_{n}|^{q}dx\rightarrow 0,\ \ \int _{\mathbb {R}^{N}}|u_{n}|^{2_{s}^{*}}dx\rightarrow 0 \), which implies that \(E_{\mu }(u_{n})\rightarrow 0\), this contradict the fact that \(E_{\mu }(u_{n})\rightarrow m(a,\mu )\).

Case 2 If \(\ell \ge S^{\frac{N}{2s}}_{s}\), from \(P_{\mu }(u_{n})\rightarrow 0\) and \(E_{\mu }(u_{n})\rightarrow m(a,\mu )\), we obtain

$$\begin{aligned} m(a,\mu )+o_{n}(1)&=E_{\mu }(u_{n})=\frac{s}{N}||u_{n}||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{\mu }{q}\left( 1-\frac{q\gamma _{q,s}}{2_{s}^{*}}\right) \int _{\mathbb {R}^{N}} |u|^{q}dx+o_{n}(1)\\&=\frac{s}{N}||u_{n}||^{2}_{D_{s}(\mathbb {R}^{N})}+o_{n}(1)= \frac{s}{N}\ell +o_{n}(1) , \end{aligned}$$

which implies that

$$\begin{aligned} m(a,\mu )=\frac{s}{N}\ell \ge \frac{s}{N}S^{\frac{N}{2s}}_{s}, \end{aligned}$$

which contradicts our assumptions. Thus, \(u\not \equiv 0.\) From (2.13), we know that \(\lambda <0\). Pass to the limit in (2.12) by the weak convergence, we obtain that

$$\begin{aligned} (-\Delta )^{s}u=\lambda u +\mu |u|^{q-2}u+|u|^{2_{s}^{*}-2}u,\&x\in \mathbb {R}^{N}. \end{aligned}$$
(2.14)

By the Pohozaev identity, \(P_{\mu }(u)=0.\) Let \(\sigma _{n}=u_{n}-u\), then \(\sigma _{n}\rightharpoonup 0\) in \(H^{s}(\mathbb {R}^{N})\). Since

$$\begin{aligned} ||u_{n}||^{2}_{D_{s}(\mathbb {R}^{N})}=||\sigma _{n}||^{2}_{D_{s}(\mathbb {R}^{N})}+||u||^{2}_{D_{s}(\mathbb {R}^{N})}+o_{n}(1) \end{aligned}$$
(2.15)

and by the well-known Brézis–Lieb lemma, we get

$$\begin{aligned} \int _{\mathbb {R}^{N}} |u_{n}|^{2_{s}^{*}}dx=\int _{\mathbb {R}^{N}} |\sigma _{n}|^{2_{s}^{*}}dx+\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx+o_{n}(1) \end{aligned}$$
(2.16)

Therefore, from \(P_{\mu }(u_{n})\rightarrow 0\) and \(u_{n}\rightarrow u\) in \(L^{q}(\mathbb {R}^{N})\), we have

$$\begin{aligned}&||\sigma _{n}||^{2}_{D_{s}(\mathbb {R}^{N})}+||u||^{2}_{D_{s}(\mathbb {R}^{N})}=\mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx\\&\quad +\int _{\mathbb {R}^{N}} |\sigma _{n}|^{2_{s}^{*}}dx+\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx+o_{n}(1). \end{aligned}$$

Combining this with \(P_{\mu }(u)=0,\) we know that \(||\sigma _{n}||^{2}_{D_{s}(\mathbb {R}^{N})}=\int _{\mathbb {R}^{N}} |\sigma _{n}|^{2_{s}^{*}}dx+o_{n}(1),\) thus

$$\begin{aligned} \lim _{n\rightarrow +\infty }||\sigma _{n}||^{2}_{D_{s}(\mathbb {R}^{N})}=\lim _{n\rightarrow +\infty }\int _{\mathbb {R}^{N}} |\sigma _{n}|^{2_{s}^{*}}dx=\ell \ge 0. \end{aligned}$$

By the definition of \(S_{s}\) in (1.5), we have \(\ell \ge S_{s}\ell ^{\frac{2}{2_{s}^{*}}}\), hence we can deduce

$$\begin{aligned} \ell =0\ \text { or}\ \ell \ge S^{\frac{N}{2s}}_{s}. \end{aligned}$$

Case 1 \(\ell \ge S^{\frac{N}{2s}}_{s}\). By (2.15) and (2.16), we have

$$\begin{aligned} m(a,\mu )&=\lim _{n\rightarrow +\infty }E_{\mu }(u_{n})=\lim _{n\rightarrow +\infty }\left( E_{\mu }(u)+\frac{1}{2}||\sigma _{n}||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{1}{2_{s}^{*}}\int _{\mathbb {R}^{N}} |\sigma _{n}|^{2_{s}^{*}}dx\right) \\&=E_{\mu }(u)+\frac{s}{N}\ell \ge E_{\mu }(u)+\frac{s}{N}S^{\frac{N}{2s}}_{s}. \end{aligned}$$

Thus, the conclusion i) holds, i.e., up to a subsequence, \(u_{n}\rightharpoonup u\) weakly in \(H^{s}(\mathbb {R}^{N})\) but not strongly, where \(u\not \equiv 0\) is a solution of (1.1) for some \(\lambda <0\), and

$$\begin{aligned} E_{\mu }(u)\le m(a,\mu )- \frac{s}{N}S^{\frac{N}{2s}}_{s}. \end{aligned}$$

Case 2 \(\ell =0.\) Then \(u_{n}\rightarrow u\) strongly in \(D_{s}(\mathbb {R}^{N})\), which implies that \(u_{n}\rightarrow u\) strongly in \(L^{2_{s}^{*}}(\mathbb {R}^{N})\) by Sobolev embedding inequality. Next, we show that \(u_{n}\rightarrow u\) strongly in \(L^{2}(\mathbb {R}^{N}).\) If we let \(\varphi =u_{n}-u\) in (2.12) and multiply \(u_{n}-u\) on both side of (2.14), we obtain

$$\begin{aligned}&||u_{n}-u||^{2}_{D_{s}(\mathbb {R}^{N})}-\int _{\mathbb {R}^{N}}(\lambda _{n}u_{n}-\lambda u)(u_{n}-u)dx\\&\quad =\int _{\mathbb {R}^{N}}(|u_{n}|^{q-2}u_{n}-|u|^{q-2}u)(u_{n}-u)dx\\&\quad +\int _{\mathbb {R}^{N}}(|u_{n}|^{2_{s}^{*}-2}u_{n}-|u|^{2_{s}^{*}-2}u)(u_{n}-u)dx+o_{n}(1) \end{aligned}$$

Thus, by \(u_{n}\rightarrow u\) strongly in \(D_{s}(\mathbb {R}^{N})\) and \(u_{n}\rightarrow u\) strongly in \(L^{2_{s}^{*}}(\mathbb {R}^{N})\), we have

$$\begin{aligned} 0=\lim _{n\rightarrow +\infty }\int _{\mathbb {R}^{N}}(\lambda _{n}u_{n}-\lambda u)(u_{n}-u)dx=\lim _{n\rightarrow +\infty }\lambda \int _{\mathbb {R}^{N}}(u_{n}-u)^{2}dx, \end{aligned}$$

which implies that \(u_{n}\rightarrow u\) strongly in \(L^{2}(\mathbb {R}^{N})\) by \(\lambda <0\). Thus, the conclusion ii) holds, i.e. up to a subsequence \(u_{n}\rightarrow u\) strongly in \(H^{s}(\mathbb {R}^{N}),\) \(E_{\mu }(u)=m(a,\mu )\) and u solves (1.1)–(1.2) for some \(\lambda <0.\) The proof is thus complete.\(\square \)

By the similar arguments as in Proposition 2.2, we can obtain the following proposition.

Proposition 2.3

Let \(N>2s, 2<q<2_{s}^{*}\) and \(a,\mu >0\). Let \(\{u_{n}\}\subset S_{a}\) be a Palais–Smale sequence for \(E_{\mu }|_{S_{a}}\) at level \(m(a,\mu )\) with

$$\begin{aligned} m(a,\mu )<\frac{s}{N}S^{\frac{N}{2s}}_{s}\ \text {and}\ m\ne 0. \end{aligned}$$

Suppose in addition that \(\mathcal {P}_{a,\mu }(u_{n})\rightarrow 0 \ \text {as} \ n\rightarrow +\infty ,\) and that there exists \(\{v_{n}\}\subset S_{a}\) and \(v_{n}\) is radially symmetric for every n such that \(\Vert u_{n}-v_{n}\Vert \rightarrow 0\) as \(n\rightarrow +\infty .\) Then one of the alternatives (i) and (ii) in Proposition 2.2 holds.

3 \(L^{2}\)-subcritical perturbation

For \(N>2s\) and \(2<q<2+4s/N\), let us recall \(C'\) in (1.7). We consider the constrained functional \(E_{\mu }|_{S_{a}}\). For every \(u\in S_{a}\), by fractional Gagliardo–Nirenberg–Sobolev inequality (2.4) and Sobolev inequality (1.5)

$$\begin{aligned} E_{\mu }(u)\ge \frac{1}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})} -\frac{\mu }{q}C^{q}_{N,q,s}||u||^{q\gamma _{q,s}}_{D_{s}(\mathbb {R}^{N})}a^{q(1-\gamma _{p,s})}-\frac{1}{2_{s}^{*}}S^{-\frac{2_{s}^{*}}{2}}_{s}||u||^{2_{s}^{*}}_{D_{s}(\mathbb {R}^{N})}. \end{aligned}$$
(3.1)

Therefore, we consider the function \(h:\mathbb {R}^{+}\rightarrow \mathbb {R}\)

$$\begin{aligned} h(t)=\frac{1}{2}t^{2}-\frac{\mu }{q}C^{q}_{N,q,s}a^{q(1-\gamma _{q,s})}t^{q\gamma _{q,s}}-\frac{1}{2_{s}^{*}}S^{-\frac{2_{s}^{*}}{2}}_{s}t^{2_{s}^{*}}. \end{aligned}$$
(3.2)

Since \(\mu >0\) and \(q\gamma _{q,s}<2<2_{s}^{*}\), we have \(h(0^{+})=0^{-}\) and \(h(+\infty )=-\infty .\)

Lemma 3.1

Under the assumption that \(\mu a^{(1-\gamma _{q,s})q}< C'\) [see (1.7)], the function h has a local strict minimum at negative level, a global strict maximum at positive level, and no other critical points, and there exists a \(R_{0}\) and \(R_{1}\) both depending on a and \(\mu \), such that \(h(R_{0})=0=h(R_{1})\) and \(h(t)\ge 0\) if and only if \(t\in (R_{0}, R_{1}).\)

Proof

For \(t>0\), we have \(h(t)>0\) if and only if

$$\begin{aligned} \varphi (t)>\frac{\mu }{q}C^{q}_{N,q,s}a^{q(1-\gamma _{q,s})},\ \text {with}\ \varphi (t)=\frac{1}{2}t^{2-q\gamma _{q,s}}-\frac{1}{2_{s}^{*}}S^{-\frac{2_{s}^{*}}{2}}_{s}t^{2_{s}^{*}-q\gamma _{q,s}} . \end{aligned}$$

Since

$$\begin{aligned} \varphi '(t)=\frac{2-q\gamma _{q,s}}{2}t^{1-q\gamma _{q,s}}-\frac{2_{s}^{*}-q\gamma _{q,s}}{2_{s}^{*}}S^{-\frac{2_{s}^{*}}{2}}_{s}t^{2_{s}^{*}-1-q\gamma _{q,s}}, \end{aligned}$$

it is easy to see that \(\varphi (t)\) is increasing on \((0, \overline{t})\) and decreasing on \((\overline{t},+\infty )\) and has a unique global maximum point at positive level on \((0,+\infty )\), where \(\overline{t}=\left( \frac{(2-q\gamma _{q,s})2_{s}^{*}}{2(2_{s}^{*}-q\gamma _{q,s})}S^{\frac{2_{s}^{*}}{2}}_{s}\right) ^{\frac{1}{2_{s}^{*}-2}}\). Thus the maximum level is

$$\begin{aligned} \varphi (\overline{t})=\frac{2_{s}^{*}-2}{2(2_{s}^{*}-q\gamma _{q,s})}\left( \frac{(2-q\gamma _{q,s})2_{s}^{*}}{2(2_{s}^{*}-q\gamma _{q,s})}S^{\frac{2_{s}^{*}}{2}}_{s}\right) ^{\frac{2-q\gamma _{q,s}}{2_{s}^{*}-2}}. \end{aligned}$$

Therefore, h is positive on an open interval \((R_{0},R_{1})\) if and only if \(\varphi (\overline{t})>\frac{\mu }{q}C^{q}_{N,q,s}a^{q(1-\gamma _{q,s})}\), which implies that

$$\begin{aligned} \mu a^{q(1-\gamma _{q,s})} < \frac{q(2_{s}^{*}-2)}{2C^{q}_{N,q,s}(2_{s}^{*}-q\gamma _{q,s})}\left( \frac{(2-q\gamma _{q,s})2_{s}^{*}}{2(2_{s}^{*}-q\gamma _{q,s})}S^{\frac{2_{s}^{*}}{2}}_{s}\right) ^{\frac{2-q\gamma _{q,s}}{2_{s}^{*}-2}}. \end{aligned}$$

Since \(h(0^{+})=0^{-}\) , \(h(+\infty )=-\infty \) and h is positive on an open interval \((R_{0},R_{1})\) , it is easy to see that h has a global maximum at positive level in \((R_{0},R_{1})\) and has a local minimum point at negative level in \((0,R_{0})\). Since \(h'(t)=t^{q\gamma _{q,s}-1}\left[ t^{2-q\gamma _{q,s}}-\mu \gamma _{q,s}C^{q}_{N,q,s}a^{q(1-\gamma _{q,s})}-S^{-\frac{2_{s}^{*}}{2}}_{s}t^{2_{s}^{*}-q\gamma _{q,s}}\right] =0\) if and only if

$$\begin{aligned} \psi (t)=\mu \gamma _{q,s}C^{q}_{N,q,s}a^{q(1-\gamma _{q,s})}\ \text {where}\ \psi (t)=t^{2-q\gamma _{q,s}}-S^{-\frac{2_{s}^{*}}{2}}_{s}t^{2_{s}^{*}-q\gamma _{q,s}}. \end{aligned}$$

Obviously, \(\psi (t)\) has only one critical point, which is a strict maximum. Therefore, if \(\psi (t)_{max}\le \mu \gamma _{q,s}C^{q}_{N,q,s}a^{q(1-\gamma _{q,s})},\) then it is easy to see that contract with h is positive on an open interval \((R_{0},R_{1})\). Thus, \(\psi (t)_{max}>\mu \gamma _{q,s}C^{q}_{N,q,s}a^{q(1-\gamma _{q,s})}\), which implies that h only has a local strict minimum at negative level, a global strict maximum at positive level, and no other critical points.\(\square \)

Lemma 3.2

Under the condition of \(\mu a^{(1-\gamma _{q,s})q}< C'\) [see (1.7)], then \(\mathcal {P}^{0}_{a,\mu }=\emptyset \) and \(\mathcal {P}_{a,\mu }\) is a smooth manifold of codimension 1 in \(S_{a}\).

Proof

Assume by contradiction that there exists a \(u\in \mathcal {P}^{0}_{a,\mu }\) such that

$$\begin{aligned} ||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx-\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx=0, \end{aligned}$$
(3.3)

and

$$\begin{aligned} 2||u||^{2}_{D_{s}(\mathbb {R}^{N})} =\mu q\gamma ^{2}_{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx+2_{s}^{*}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx. \end{aligned}$$
(3.4)

Thus, from (1.5), (2.4), (3.3), and (3.4), we have

$$\begin{aligned}&\mu \gamma _{q,s}(2-q\gamma _{q,s})\int _{\mathbb {R}^{N}} |u|^{q}dx=(2_{s}^{*}-2)\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx,\nonumber \\&||u||^{2}_{D_{s}(\mathbb {R}^{N})} =\frac{2_{s}^{*}-q\gamma _{q,s}}{2-q\gamma _{q,s}}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\le \frac{2_{s}^{*}-q\gamma _{q,s}}{2-q\gamma _{q,s}} S^{-\frac{2_{s}^{*}}{2}}_{s}||u||^{2_{s}^{*}}_{D_{s}(\mathbb {R}^{N})}, \end{aligned}$$
(3.5)
$$\begin{aligned}&||u||^{2}_{D_{s}(\mathbb {R}^{N})} =\mu \gamma _{q,s}\frac{2_{s}^{*}-q\gamma _{q,s}}{2_{s}^{*}-2}\int _{\mathbb {R}^{N}} |u|^{q}dx \le \mu \gamma _{q,s}\frac{2_{s}^{*}-q\gamma _{q,s}}{2_{s}^{*}-2} C^{q}_{N,q,s}a^{q(1-\gamma _{q,s})}||u||^{q\gamma _{q,s}}_{D_{s}(\mathbb {R}^{N})}, \end{aligned}$$
(3.6)

From (3.5) and (3.6), we can infer that

$$\begin{aligned} \left[ \frac{2-q\gamma _{q,s}}{2_{s}^{*}-q\gamma _{q,s}} S^{\frac{2_{s}^{*}}{2}}_{s}\right] ^{\frac{1}{2_{s}^{*}-2}}\le \left[ \mu \gamma _{q,s}\frac{2_{s}^{*}-q\gamma _{q,s}}{2_{s}^{*}-2} C^{q}_{N,q,s}a^{q(1-\gamma _{q,s})}\right] ^{\frac{1}{2-q\gamma _{q,s}}}, \end{aligned}$$

which implies that

$$\begin{aligned} \mu a^{q(1-\gamma _{q,s})}\ge \frac{2_{s}^{*}-2}{\gamma _{q,s}C^{q}_{N,q,s}(2_{s}^{*}-q\gamma _{q,s})} \left[ \frac{2-q\gamma _{q,s}}{2_{s}^{*}-q\gamma _{q,s}} S^{\frac{2_{s}^{*}}{2}}_{s}\right] ^{\frac{2-q\gamma _{q,s}}{2_{s}^{*}-2}}. \end{aligned}$$
(3.7)

Next, we show that the right hand of (3.7) is greater than or equal to \(C'\). To show

$$\begin{aligned}&\frac{2_{s}^{*}-2}{\gamma _{q,s}C^{q}_{N,q,s}(2_{s}^{*}-q\gamma _{q,s})} \left[ \frac{2-q\gamma _{q,s}}{2_{s}^{*}-q\gamma _{q,s}} S^{\frac{2_{s}^{*}}{2}}_{s}\right] ^{\frac{2-q\gamma _{q,s}}{2_{s}^{*}-2}}\\&\quad \ge \frac{q(2_{s}^{*}-2)}{2C^{q}_{N,q,s}(2_{s}^{*}-q\gamma _{q,s})}\left( \frac{(2-q\gamma _{q,s})2_{s}^{*}}{2(2_{s}^{*}-q\gamma _{q,s})}S^{\frac{2_{s}^{*}}{2}}_{s}\right) ^{\frac{2-q\gamma _{q,s}}{2_{s}^{*}-2}}, \end{aligned}$$

we only need to prove that

$$\begin{aligned} \left( \frac{q\gamma _{q,s}}{2}\right) ^{2_{s}^{*}-2}\left( \frac{2_{s}^{*}}{2}\right) ^{2-q\gamma _{q,s}}\le 1, \ \text {for every} \ \ 2<q<\overline{p}<2_{s}^{*}. \end{aligned}$$

Let \(q\gamma _{q,s}=x\in (0,2)\), we need to show that \(\left( \frac{x}{2}\right) ^{2_{s}^{*}-2}\left( \frac{2_{s}^{*}}{2}\right) ^{2-x}\le 1.\) For this, we set \(f(x)=\left( \frac{x}{2}\right) ^{2_{s}^{*}-2}\left( \frac{2_{s}^{*}}{2}\right) ^{2-x}\), it is easy to see that f(x) is strictly increasing on \((0, \frac{2_{s}^{*}-2}{\ln 2_{s}^{*}-\ln 2})\) and decreasing on \(( \frac{2_{s}^{*}-2}{\ln 2_{s}^{*}-\ln 2},+\infty )\). Thus, when \(x\in (0,2)\) \(f(x)\le f(2)=1,\) which implies that

$$\begin{aligned} \left( \frac{q\gamma _{q,s}}{2}\right) ^{2_{s}^{*}-2}\left( \frac{2_{s}^{*}}{2}\right) ^{2-q\gamma _{q,s}}\le 1. \end{aligned}$$

This contradicts the assumption \(\mu a^{q(1-\gamma _{q,s})}<C'.\) Thus, \(\mathcal {P}^{0}_{a,\mu }=\emptyset .\)

Next, we show that \(\mathcal {P}_{a,\mu }\) is a smooth manifold of codimension 1 on \(S_{a}\). Since \(\mathcal {P}_{a,\mu }=\bigg \{u\in S_{a}:||u||^{2}_{D_{s}(\mathbb {R}^{N})}=\mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx+\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\bigg \}\), we know that \(\mathcal {P}_{a,\mu }\) is defined by \(P_{\mu }(u)=0\), \(G(u)=0\), where

$$\begin{aligned}&P_{\mu }(u)=s||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \gamma _{q,s}s\int _{\mathbb {R}^{N}} |u|^{q}dx\\&-s\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx \ \ \text {and}\ \ G(u)=\int _{\mathbb {R}^{N}} |u|^{2}dx=a^{2} . \end{aligned}$$

Since \(P_{\mu }(u)\) and G(u) are class of \(C^{1}\), we only need to check that \(d(P_{\mu }(u),G(u))\): \(H^{s}(\mathbb {R}^{N})\rightarrow \mathbb {R}^{2}\) is surjective. If this not true, \(dP_{\mu }(u)\) has to be linearly dependent from dG(u) i.e. there exist a \(\nu \in \mathbb {R}\) such that

$$\begin{aligned}&2s\int _{\mathbb {R}^{N}} (-\Delta )^{\frac{s}{2}}u(-\Delta )^{\frac{s}{2}}\varphi dx-\mu q\gamma _{q,s}s\int _{\mathbb {R}^{N}} |u|^{q-2}u\varphi dx-s2_{s}^{*}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}-2}u\varphi dx\\&\quad =\nu \int _{\mathbb {R}^{N}} u\varphi dx \end{aligned}$$

for every \(\varphi \in H^{s}(\mathbb {R}^{N})\), which implies that

$$\begin{aligned} 2s(-\Delta )^{2}u=\nu u+\mu q\gamma _{q,s}su^{q-1}+2_{s}^{*}s u^{2_{s}^{*}-1}\ \ \text {in}\ \mathbb {R}^{N}. \end{aligned}$$

By the Pohozaev identity for above equation, we know that

$$\begin{aligned} 2s^{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}= \mu q\gamma ^{2}_{q,s}s^{2}\int _{\mathbb {R}^{N}} |u|^{q}dx+2_{s}^{*}s^{2}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx, \end{aligned}$$

that is \(u\in \mathcal {P}^{0}_{a,\mu }\), a contradiction. Hence, \(\mathcal {P}_{a,\mu }\) is a natural constraint.\(\square \)

Lemma 3.3

For every \(u\in S_{a}\), the function \(\Psi ^{\mu }_{u}(t)\) has exactly two critical points \(a_{u}<t_{u}\in \mathbb {R}\) and two zeros \(c_{u}<d_{u}\in \mathbb {R}\), with \(a_{u}<c_{u}<t_{u}<d_{u}\). Moreover,

  1. (1)

    \(a_{u}\star u\in \mathcal {P}^{+}_{a,\mu }\) and \(t_{u}\star u\in \mathcal {P}^{-}_{a,\mu }\), and if \(t\star u\in \mathcal {P}_{a,\mu }\), then either \(t=a_{u}\) or \(t=t_{u}.\)

  2. (2)

    \(||t\star u||_{D_{s}(\mathbb {R}^{N})}\le R_{0}\) for every \(t\le c_{u},\) and

    $$\begin{aligned} E_{\mu }(u)(a_{u}\star u)=\min \{E_{\mu }(t\star u):t\in \mathbb {R}\ \text {and}\ ||t\star u||_{D_{s}(\mathbb {R}^{N})}< R_{0} \}<0. \end{aligned}$$
  3. (3)

    We have

    $$\begin{aligned} E_{\mu }(u)(t_{u}\star u)=\max \{E_{\mu }(t\star u):t\in \mathbb {R} \}>0 \end{aligned}$$

    and \(\Psi ^{\mu }_{u}(t)\) is strictly decreasing and concave on \((t_{u},+\infty )\). In particular, if \(t_{u}<0\), then \(P_{\mu }(u)<0.\)

  4. (4)

    The maps \(u\in S_{a}: a_{u} \in \mathbb {R} \) and \(u\in S_{a}: t_{u} \in \mathbb {R} \) are of class \(C^{1}\).

Proof

Let \(u\in S_{a}\), since \(t\star u\in \mathcal {P}_{a,\mu } \) if and only if \((\Psi ^{\mu }_{u})'(t)=0\). Thus, we first show that \(\Psi ^{\mu }_{u}(t)\) has at least two critical points. From (3.1), we have

$$\begin{aligned} \Psi ^{\mu }_{u}(t)=E_{\mu }(t\star u)\ge h(||t\star u||_{D_{s}(\mathbb {R}^{N})})=h(e^{st}|| u||_{D_{s}(\mathbb {R}^{N})}). \end{aligned}$$

Thus, the \(C^{2}\) function \(\Psi ^{\mu }_{u}(t)\) is positive on \(\left( \frac{\ln \left( \frac{R_{0}}{|| u||_{D_{s}(\mathbb {R}^{N})}}\right) }{s}, \frac{\ln \left( \frac{R_{1}}{|| u||_{D_{s}(\mathbb {R}^{N})}}\right) }{s}\right) \) and \(\Psi ^{\mu }_{u}(-\infty )=0^{-}\), \(\Psi ^{\mu }_{u}(+\infty )=-\infty \), thus it is easy to see that \(\Psi ^{\mu }_{u}(t)\) has a local minimum point \(a_{u}\) at negative level in \((0,\frac{\ln \left( \frac{R_{0}}{|| u||_{D_{s}(\mathbb {R}^{N})}}\right) }{s})\) and has a global maximum point \(t_{u}\) at positive level in \(\left( \frac{\ln \left( \frac{R_{0}}{|| u||_{D_{s}(\mathbb {R}^{N})}}\right) }{s}, \frac{\ln \left( \frac{R_{1}}{|| u||_{D_{s}(\mathbb {R}^{N})}}\right) }{s}\right) .\) Next, we show that \(\Psi ^{\mu }_{u}(t)\) has no other critical points. Indeed, \((\Psi ^{\mu }_{u})'(t)=0\) implies that

$$\begin{aligned} \Psi (t)= & {} \mu \gamma _{q,s}s\int _{\mathbb {R}^{N}} |u|^{q}dx\ \text {where}\\ \Psi (t)= & {} se^{(2-q\gamma _{q,s})st}\Vert u\Vert ^{2}_{D_{s}(\mathbb {R}^{N})}-se^{(2_{s}^{*}-q\gamma _{q,s})st}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx. \end{aligned}$$

It is easy to see that \(\Psi (t)\) has a unique maximum point, thus the above equation has at most two solutions. From \(u\in S_{a},\) \(t\in \mathbb {R}\) is a critical point of \(\Psi ^{\mu }_{u}(t)\) if and only if \(t\star u\in \mathcal {P}_{a,\mu } ,\) we have \(a_{u}\star u,\ \ t_{u}\star u\in \mathcal {P}_{a,\mu } \) and \(t\star u \in \mathcal {P}_{a,\mu }\) if and only if \(t=a_{u}\) or \(t=t_{u}\). Since \(a_{u}\) is a local minimum point of \(\Psi ^{\mu }_{u}(t)\), we know that \((\Psi ^{\mu }_{a_{u}\star u})''(0)=(\Psi ^{\mu }_{u})''(a_{u})\ge 0\), since \(\mathcal {P}^{0}_{a,\mu }=\emptyset \), we know that \((\Psi ^{\mu }_{u})''(a_{u})\ne 0\), thus \((\Psi ^{\mu }_{a_{u}\star u})''(0)=(\Psi ^{\mu }_{u})''(a_{u})>0\), which implies that \(a_{u}\star u\in \mathcal {P}^{+}_{a,\mu },\) similarly, we have \(t_{u}\star u\in \mathcal {P}^{-}_{a,\mu }.\) By the monotonicity and the behavior at infinity of \(\Psi ^{\mu }_{u}\), we know that \(\Psi ^{\mu }_{u}\) has exactly two zeros \(c_{u}<d_{u}\) with \(a_{u}<c_{u}<t_{u}<d_{u}\) and \(\Psi ^{\mu }_{u}\) has exactly two inflection points, in particular, \(\Psi ^{\mu }_{u}\) is concave on \([t_{u},+\infty )\) and hence if \(t_{u}<0\), then \(P_{\mu }(u)=(\Psi ^{\mu }_{u})'(0)<0.\) Finally, we prove that \(u\in S_{a}: a_{u} \in \mathbb {R} \) and \(u\in S_{a}: t_{u} \in \mathbb {R} \) are of class \(C^{1}\). Indeed, we can apply the implicit function theorem on the \(C^{1}\) function \(\Phi (t,u)=(\Psi ^{\mu }_{u})'(t)\), then \(\Phi (a_{u},u)=(\Psi ^{\mu }_{u})'(a_{u})=0, \partial _{s}\Phi (a_{u},u)=(\Psi ^{\mu }_{u})''(a_{u})<0\), by the implicit function theorem, we know that \(u\in S_{a}: a_{u} \in \mathbb {R} \) is class of \(C^{1}\), similarly, we can prove that \(u\in S_{a}: t_{u} \in \mathbb {R} \) is class of \(C^{1}\).\(\square \)

For \(k>0\), set

$$\begin{aligned} A_{k}=\{u\in S_{a}:\Vert u\Vert ^{2}_{D_{s}(\mathbb {R}^{N})}<k\}, \ \text {and}\ m(a,\mu )=\inf _{u\in A_{R_{0}}}E_{\mu }(u). \end{aligned}$$

From Lemma 3.3, we immediately have the following corollary:

Corollary 3.1

The set \(\mathcal {P}^{+}_{a,\mu }\) is contained in

$$\begin{aligned} A_{R_{0}}=\{u\in S_{a}: || u||_{D_{s}(\mathbb {R}^{N})}< R_{0}\}\ \text { and }\ \sup _{\mathcal {P}^{+}_{a,\mu }}E_{\mu }\le 0\le \inf _{\mathcal {P}^{-}_{a,\mu }}E_{\mu }. \end{aligned}$$

Lemma 3.4

We have \(m(a,\mu )\in (-\infty ,0)\) that

$$\begin{aligned} m(a,\mu )=\inf _{\mathcal {P}_{a,\mu }}E_{\mu }=\inf _{\mathcal {P}^{+}_{a,\mu }}E_{\mu }\ \text {and that}\ m(a,\mu )< \inf _{\overline{A_{R_{0}}}\setminus A_{R_{0}-\rho }}E_{\mu } \end{aligned}$$

for \(\rho >0\) small enough.

Proof

For \(u\in A_{R_{0}}\), we have

$$\begin{aligned} E_{\mu }(u)\ge h(\Vert u\Vert _{D_{s}(\mathbb {R}^{N})})\ge \min _{t\in [0,R_{0}]}h(t)>-\infty . \end{aligned}$$

Therefore, \(m(a,\mu )>-\infty .\) Moreover, for any \(u\in S_{a}\), we obtain \(|| t\star u||_{D_{s}(\mathbb {R}^{N})}< R_{0}\) and \(E_{\mu }(t\star u)<0\) for \(t\ll -1\) and hence \(m(a,\mu )<0.\) Since \(\mathcal {P}^{+}_{a,\mu }\subset A_{R_{0}}\), we know that \(m(a,\mu )\le \inf _{\mathcal {P}^{+}_{a,\mu }}E_{\mu } .\) On the other hand, if \(u\in A_{R_{0}} \), then \(a_{u}\star u \in \mathcal {P}^{+}_{a,\mu }\subset A_{R_{0}}\) and

$$\begin{aligned} E_{\mu }(u)(a_{u}\star u)=\min \{E_{\mu }(t\star u):t\in \mathbb {R}\ \text {and}\ ||t\star u||_{D_{s}(\mathbb {R}^{N})}< R_{0} \}\le E_{\mu }(u), \end{aligned}$$

which implies that \(\inf _{\mathcal {P}^{+}_{a,\mu }}E_{\mu }\le m(a,\mu )\). Since \(E_{\mu }>0\) on \(\mathcal {P}^{-}_{a,\mu }\), we know that \(\inf _{\mathcal {P}^{+}_{a,\mu }}E_{\mu }=\inf _{\mathcal {P}_{a,\mu }}E_{\mu }.\) Finally, by the continuity of h there exists \(\rho >0\) such that \(h(t)\ge \frac{m(a,\mu )}{2}\) if \(t\in [R_{0}-\rho ,R_{0}]\). Therefore,

$$\begin{aligned} E_{\mu }(u)\ge h(\Vert u\Vert _{D_{s}(\mathbb {R}^{N})})\ge \frac{m(a,\mu )}{2}>m(a,\mu ) \end{aligned}$$

for every \(u\in S_{a}\) with \(R_{0}-\rho \le \Vert u\Vert _{D_{s}(\mathbb {R}^{N})}\le R_{0}\). This completes the proof.\(\square \)

4 \(L^{2}\)-critical perturbation

In this section, we consider the case \(N>2s\) and \(2<q=\overline{p}\). Assume that

$$\begin{aligned} \mu a^{\frac{4s}{N}}< \overline{p}\left( 2C^{\overline{p}}_{N,\overline{p},s}\right) ^{-1}. \end{aligned}$$
(4.1)

We recall the decomposition of

$$\begin{aligned} \mathcal {P}_{a,\mu }=\mathcal {P}^{+}_{a,\mu }\cup \mathcal {P}^{0}_{a,\mu }\cup \mathcal {P}^{-}_{a,\mu }. \end{aligned}$$

Lemma 4.1

\(\mathcal {P}^{0}_{a,\mu }=\emptyset \) and \(\mathcal {P}_{a,\mu }\) is a smooth manifold of codimension 1 in \(S_{a}\).

Proof

Assume by contradiction that if there exists a \(u\in \mathcal {P}^{0}_{a,\mu }\), then

$$\begin{aligned} ||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx-\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx=0, \end{aligned}$$
(4.2)

and

$$\begin{aligned} 2||u||^{2}_{D_{s}(\mathbb {R}^{N})} =\mu 2\gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx+2_{s}^{*}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx, \end{aligned}$$
(4.3)

thus, from (4.2) and (4.3), we have \(\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx=0\), which is not possible since \(u\in S_{a}.\) The rest of the proof is similar to that of Lemma 3.2, so we omit the details here.\(\square \)

Lemma 4.2

Under the condition of (4.1), for every \(u\in S_{a}\), there is a unique \(t_{u}\in \mathbb {R}\) such that \(t_{u}\star u \in \mathcal {P}_{a,\mu },\) where \(t_{u}\) is the unique critical point of the function of \(\Psi ^{\mu }_{u}\) and is a strict maximum point at positive level. Moreover,

  1. (1)

    \(\mathcal {P}_{a,\mu }=\mathcal {P}^{-}_{a,\mu }\).

  2. (2)

    \(\Psi ^{\mu }_{u}(t)\) is strictly decreasing and concave on \((t_{u}, +\infty )\) and \(t_{u}<0\) implies that \(P_{\mu }(u)<0.\)

  3. (3)

    The map \(u\in S_{a}: t_{u}\in \mathbb {R}\) os of class \(C^{1}\).

  4. (4)

    If \(P_{\mu }(u)<0\), then \(t_{u}<0\).

Proof

Since

$$\begin{aligned} \Psi ^{\mu }_{u}(t)=E_{\mu }(t\star u)=\left[ \frac{1}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{\mu }{\overline{p}}\int _{\mathbb {R}^{N}} |u|^{\overline{p}}dx\right] e^{2st}-\frac{e^{2_{s}^{*}st}}{2_{s}^{*}}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx, \end{aligned}$$

and \(t\star u\in \mathcal {P}_{a,\mu } \) if and only if \((\Psi ^{\mu }_{u})'(t)=0\), it is easy to see that if \(\left[ \frac{1}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{\mu }{\overline{p}}\int _{\mathbb {R}^{N}} |u|^{\overline{p}}dx\right] \) is positive, then \(\Psi ^{\mu }_{u}(t)\) has a unique critical point \(t_{u}\), which is is a strict maximum point at positive level. By the fractional Gagliardo–Nirenberg–Sobolev inequality (2.4), we have

$$\begin{aligned} \frac{1}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{\mu }{\overline{p}}\int _{\mathbb {R}^{N}} |u|^{\overline{p}}dx\ge \left( \frac{1}{2}-\frac{\mu }{\overline{p}}C^{\overline{p}}_{N,\overline{p},s}a^{\frac{4s}{N}}\right) ||u||^{2}_{D_{s}(\mathbb {R}^{N})}, \end{aligned}$$

so under the condition of \(\mu a^{\frac{4s}{N}}< \overline{p}(2C^{\overline{p}}_{N,\overline{p},s})^{-1},\) we know that \(\frac{1}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{\mu }{\overline{p}}\int _{\mathbb {R}^{N}} |u|^{\overline{p}}dx>0.\) Since, if \(u\in \mathcal {P}_{a,\mu }\), then \(t_{u}\) is a maximum point, we have that \(\Psi ^{\mu }_{u}(0)\le 0.\) Since \(\mathcal {P}^{0}_{a,\mu }=\emptyset ,\) we have \(\Psi ^{\mu }_{u}(0)<0.\) Thus, \(\mathcal {P}_{a,\mu }=\mathcal {P}^{-}_{a,\mu }.\) To prove that the map \(u\in S_{a}: t_{u}\in \mathbb {R}\) is of class \(C^{1}\), we can apply the implicit function theorem as in Lemma 3.3. Finally, since \((\Psi ^{\mu }_{u})'(t)<0\) if and only if \(t>t_{u}\), so \(P_{\mu }(u)=(\Psi ^{\mu }_{u})'(0)<0\) if and only if \(t_{u}<0\).\(\square \)

Lemma 4.3

$$\begin{aligned} m(a,\mu )=\inf _{\mathcal {P}_{a,\mu }}E_{\mu }>0. \end{aligned}$$

Proof

If \(u\in \mathcal {P}_{a,\mu },\) then \(P_{\mu }(u)=0\), and then by fractional Gagliardo–Nirenberg–Sobolev inequality (2.4) and Sobolev inequality (1.5), we have

$$\begin{aligned}&||u||^{2}_{D_{s}(\mathbb {R}^{N})}=\mu \frac{2}{\overline{p}}\int _{\mathbb {R}^{N}} |u|^{\overline{p}}dx+\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\\&\le \mu \frac{2}{\overline{p}}C^{\overline{p}}_{N,\overline{p},s}a^{\frac{4s}{N}}||u||^{2}_{D_{s}(\mathbb {R}^{N})}+S^{-\frac{2_{s}^{*}}{2}}_{s}||u||^{2_{s}^{*}}_{D_{s}(\mathbb {R}^{N})} \end{aligned}$$

From (4.1) and above inequality, we have

$$\begin{aligned} ||u||^{2_{s}^{*}}_{D_{s}(\mathbb {R}^{N})}\ge S^{\frac{2_{s}^{*}}{2}}_{s}\left( 1-\mu \frac{2}{\overline{p}}C^{\overline{p}}_{N,\overline{p},s}a^{\frac{4s}{N}}\right) ||u||^{2}_{D_{s}(\mathbb {R}^{N})}\Rightarrow \inf _{\mathcal {P}_{a,\mu }}||u||_{D_{s}(\mathbb {R}^{N})}>0. \end{aligned}$$
(4.4)

Thus, from \(P_{\mu }(u)=0\) and above inequality, we have

$$\begin{aligned} E_{\mu }(u)&=\frac{s}{N}\left( ||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{2\mu }{\overline{p}}\int _{\mathbb {R}^{N}}|u|^{\overline{p}}dx\right) \\&\ge \frac{s}{N} \left( 1-\mu \frac{2}{\overline{p}}C^{\overline{p}}_{N,\overline{p},s}a^{\frac{4s}{N}}\right) ||u||^{2}_{D_{s}(\mathbb {R}^{N})}> 0. \end{aligned}$$

Therefore,

$$\begin{aligned} m(a,\mu )=\inf _{\mathcal {P}_{a,\mu }}E_{\mu }>0. \end{aligned}$$

As required.\(\square \)

Lemma 4.4

There exists \(k>0\) sufficiently small such that

$$\begin{aligned} 0<\sup _{\overline{A_{k}}}E_{\mu }<m(a,\mu )\ \text {and}\ u\in \overline{A_{k}}\Rightarrow E_{\mu }(u), P_{\mu }(u)>0. \end{aligned}$$

where \( A_{k}=\left\{ u\in S_{a}: ||u||^{2}_{D_{s}(\mathbb {R}^{N})}<k\right\} .\)

Proof

By fractional Gagliardo–Nirenberg–Sobolev inequality (2.4) and Sobolev inequality (1.5), we have

$$\begin{aligned} E_{\mu }(u) \ge \left( \frac{1}{2}-\frac{\mu }{\overline{p}}C^{\overline{p}}_{N,\overline{p},s}a^{\frac{4s}{N}}\right) ||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{1}{2_{s}^{*}}S^{-\frac{2_{s}^{*}}{2}}_{s}||u||^{2_{s}^{*}}_{D_{s}(\mathbb {R}^{N})}>0, \end{aligned}$$

and

$$\begin{aligned} P_{\mu }(u)&=s||u||^{2}_{D_{s}(\mathbb {R}^{N})}-s\mu \frac{2}{\overline{p}}\int _{\mathbb {R}^{N}} |u|^{\overline{p}}dx-s\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\\&\ge s\left( 1-\frac{2\mu }{\overline{p}}C^{\overline{p}}_{N,\overline{p},s}a^{\frac{4s}{N}}\right) ||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{s}{2_{s}^{*}}S^{-\frac{2_{s}^{*}}{2}}_{s}||u||^{2_{s}^{*}}_{D_{s}(\mathbb {R}^{N})}>0, \end{aligned}$$

provided that \(u\in \overline{A_{k}}\) for k small enough. By Lemma 4.4, we know that \(m(a,\mu )>0\), thus if necessary replacing k with smaller quantity, we also have

$$\begin{aligned} E_{\mu }(u)\le \frac{1}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}< m(a,\mu ). \end{aligned}$$

The proof is complete.\(\square \)

In order to apply Proposition 2.2 and recover compactness, we need an estimate from above on \(m_{r}(a,\mu )=\inf _{\mathcal {P}_{a,\mu }\bigcap S^{r}_{a}}E_{\mu }\), where \(S^{r}_{a}\) is the subset of the radial functions in \(S_{a}.\)

Lemma 4.5

Under condition (4.1), we have \(m_{r}(a,\mu )< \frac{s}{N}S^{\frac{N}{2s}}_{s}.\)

Proof

From [14], we know that \(S_{s}\) is attained in \(\mathbb {R}^{N}\) by

$$\begin{aligned} U_0(x)=C(N,s)\left( \frac{1}{1+|x|^{2}}\right) ^{\frac{N-2s}{2}} \end{aligned}$$

with C(Ns) chosen so that

$$\begin{aligned} \Vert U_0(x)\Vert ^{2}_{D_{s}(\mathbb {R}^{N})}=\int _{\mathbb {R}^{N}}|U_0(x)|^{2_{s}^{*}}dx=S^{\frac{N}{2s}}_{s}. \end{aligned}$$

Take \(\eta (x)\in C^{\infty }_{0}(\mathbb {R}^{N},[0,1])\) be a cut-off function such that \(0\le \eta \le 1,\eta =1\ \text {on}\ B(0,\delta )\) and \(\eta =1 \text { on }\ \mathbb {R}^{N}\setminus B(0,2\delta )\). Let

$$\begin{aligned} u_{\epsilon }=\eta (x)U_{\epsilon }(x),\ \ \ v_{\epsilon }=a\frac{u_{\epsilon }}{\Vert u_{\epsilon }\Vert _{L^{2}(\mathbb {R}^{N})}} \end{aligned}$$

and

$$\begin{aligned} U_{\epsilon }(x)=\epsilon ^{-\frac{N-2s}{2}}U_{0}(\frac{x}{\epsilon }). \end{aligned}$$

From Proposition 21 and Proposition 22 in [30], it is easy to deduce that the following estimates hold true

$$\begin{aligned} \Vert v_{\epsilon }\Vert ^{2}_{D_{s}(\mathbb {R}^{N})}\le & {} \Vert U_{0}\Vert ^{2}_{D_{s}(\mathbb {R}^{N})}+o(\epsilon ^{N-2s}), \end{aligned}$$
(4.5)
$$\begin{aligned} \int _{\mathbb {R}^{N}}|v_{\epsilon }|^{2}dx= & {} \left\{ \begin{array}{rcl} C\epsilon ^{2s}+o(\epsilon ^{N-2s}), &{} &{} if\ N>4s,\\ C\epsilon ^{2s}\log (\frac{1}{\epsilon })+o(\epsilon ^{2s}), &{} &{} if\ N=4s,\\ C\epsilon ^{N-2s}+o(\epsilon ^{2s}), &{} &{} if\ N<4s, \end{array} \right. \end{aligned}$$
(4.6)

It is easy to see that

$$\begin{aligned} \int _{\mathbb {R}^{N}}|v_{\epsilon }|^{^{2_{s}^{*}}}dx= \int _{\mathbb {R}^{N}}|U_{0}|^{^{2_{s}^{*}}}dx+o(\epsilon ^{N}). \end{aligned}$$
(4.7)

By the similar arguments as Proposition 22 in [30], we can deduce that

$$\begin{aligned} \int _{\mathbb {R}^{N}}|v_{\epsilon }|^{\overline{p}}dx=\left\{ \begin{array}{rcl} &{}C\epsilon ^{N-\frac{ N-2 s}{2}\overline{p}}+o(\epsilon ^{\frac{N-2s}{2}\overline{p}}), &{} if\ N>\frac{\overline{p}}{\overline{p}-1}2s,\\ &{}C\epsilon ^{\frac{N}{2}}\log (\frac{1}{\epsilon })+o(\epsilon ^{\frac{N}{2}}), &{} if\ N=\frac{\overline{p}}{\overline{p}-1}2s,\\ &{}C\epsilon ^{\frac{N-2s}{2}\overline{p}}+o(\epsilon ^{N-\frac{ N-2 s}{2}\overline{p}}), &{} if\ N<\frac{\overline{p}}{\overline{p}-1}2s. \end{array} \right. \end{aligned}$$
(4.8)

It is easy to see that \(u_{\epsilon }\in C^{\infty }_{0}(\mathbb {R}^{N},[0,1])\) and \(v_{\epsilon }\in S_{a}^{r}.\) By Lemma 4.2, we know that

$$\begin{aligned} m_{r}(a,\mu )=\inf _{\mathcal {P}_{a,\mu }\bigcap S^{r}_{a}}E_{\mu }\le E_{\mu }(t_{v_{\epsilon }}\star v_{\epsilon })=\max _{t\in \mathbb {R}}E_{\mu }(t\star v_{\epsilon }). \end{aligned}$$

Next, we give a upper estimate of

$$\begin{aligned} E_{\mu }(t_{v_{\epsilon }}\star v_{\epsilon })=\max _{t\in \mathbb {R}}E_{\mu }(t\star v_{\epsilon }). \end{aligned}$$

Step 1 Consider the case \(\mu =0\) and estimate

$$\begin{aligned} \max _{t\in \mathbb {R}}\Psi ^{0}_{v_{\epsilon }}(t)=E_{0}(t\star v_{\epsilon }). \end{aligned}$$

Since

$$\begin{aligned} \Psi ^{0}_{v_{\epsilon }}(t)=\frac{e^{2st}}{2}||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{e^{2_{s}^{*}st}}{2_{s}^{*}}\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx. \end{aligned}$$

It is easy to see that for every \(v_{\epsilon }\in S_{a}\) the function \(\Psi ^{0}_{v_{\epsilon }}(t)\) has a unique critical point \(t_{v_{\epsilon },0}\), which is a strict maximum point and is given by

$$\begin{aligned} e^{st_{v_{\epsilon },0}}=\left( \frac{||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx}\right) ^{\frac{1}{2_{s}^{*}-2}}. \end{aligned}$$
(4.9)

Thus, from the estimates (4.5)–(4.7), we have

$$\begin{aligned}&\max _{t\in \mathbb {R}}E_{0}(t\star v_{\epsilon })=E_{0}(t_{v_{\epsilon },0}\star v_{\epsilon })=\Psi ^{0}_{v_{\epsilon }}(t_{v_{\epsilon },0})\\&\quad = \left[ \frac{1}{2}\left( \frac{||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx}\right) ^{\frac{2}{2_{s}^{*}-2}}||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{1}{2_{s}^{*}}\left( \frac{||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx}\right) ^{\frac{2_{s}^{*}}{2_{s}^{*}-2}}\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx\right] \\&\quad = \frac{s}{N}\left( \frac{||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\left( \int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx\right) ^{\frac{2}{2_{s}^{*}}}} \right) ^{\frac{2_{s}^{*}}{2_{s}^{*}-2}}\\&\quad =\frac{s}{N}\left[ \frac{S^{\frac{N}{2s}}_{s}+O(\epsilon ^{N-2s})}{\left( S^{\frac{N}{2s}}_{s}+O(\epsilon ^{N})\right) ^{\frac{2}{2_{s}^{*}}}}\right] ^{\frac{N}{2s}}=\frac{s}{N}S^{\frac{N}{2s}}_{s}+O(\epsilon ^{N-2s}) \end{aligned}$$

Step 2 Estimate on \(t_{v_{\epsilon }, \mu }\). Since

$$\begin{aligned} \Psi ^{\mu }_{v_{\epsilon }}(t)=E_{\mu }(t\star v_{\epsilon })=\frac{e^{2st}}{2}||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \frac{e^{2st}}{\overline{p}}\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{\overline{p}}dx-\frac{e^{2_{s}^{*}st}}{2_{s}^{*}}\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx. \end{aligned}$$

Let \(t_{v_{\epsilon },\mu }\) be the unique maximum point of \(\Psi ^{\mu }_{v_{\epsilon }}(t)\), then by \((\Psi ^{\mu }_{v_{\epsilon }})'(t)=P_{\mu }(t_{v_{\epsilon },\mu }\star v_{\epsilon })=0\) and fractional Gagliardo–Nirenberg–Sobolev inequality (2.4), we have

$$\begin{aligned} e^{(2_{s}^{*}-2)st}=\frac{||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx}-\frac{2\mu }{\overline{p}}\frac{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{\overline{p}}dx}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx}\ge \left( 1-\frac{2\mu }{\overline{p}}C^{\overline{p}}_{N,\overline{p},s}a^{\frac{4s}{N}}\right) \frac{||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx}. \end{aligned}$$

Step 3 Estimate on \(\max _{t\in \mathbb {R}}\Psi ^{\mu }_{v_{\epsilon }}(t)\). Since

$$\begin{aligned}&\max _{t\in \mathbb {R}}\Psi ^{\mu }_{v_{\epsilon }}(t)=\Psi ^{\mu }_{v_{\epsilon }}(t_{v_{\epsilon },\mu })=\Psi ^{0}_{v_{\epsilon }}(t_{v_{\epsilon },\mu })-\mu \frac{e^{2st_{v_{\epsilon },\mu }}}{\overline{p}}\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{\overline{p}}dx\\&\quad \le \sup _{\mathbb {R}}\Psi ^{0}_{v_{\epsilon }}-\frac{\mu }{\overline{p}}\left( 1-\frac{2\mu }{\overline{p}}C^{\overline{p}}_{N,\overline{p},s}a^{\frac{4s}{N}}\right) ^{\frac{2}{2_{s}^{*}-2}}\left( \frac{||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx}\right) ^{\frac{2}{2_{s}^{*}-2}}\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{\overline{p}}dx\\&\quad \le \frac{s}{N}S^{\frac{N}{2s}}_{s}+O(\epsilon ^{N-2s})\\&\qquad -\frac{\mu }{\overline{p}}\left( 1-\frac{2\mu }{\overline{p}}C^{\overline{p}}_{N,\overline{p},s}a^{\frac{4s}{N}}\right) ^{\frac{2}{2_{s}^{*}-2}}\frac{a^{\frac{4s}{N}}}{\Vert u_{\epsilon }\Vert ^{\frac{4s}{N}}_{L^{2}(\mathbb {R}^{N})}}\frac{||u_{\epsilon }||^{\frac{4}{2_{s}^{*}-2}}_{D_{s}(\mathbb {R}^{N})}\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{\overline{p}}dx}{\left( \int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx\right) ^{\frac{2}{2_{s}^{*}-2}}}\\&\quad \le \frac{s}{N}S^{\frac{N}{2s}}_{s}+O(\epsilon ^{N-2s})-C_{N,a,\mu } \frac{\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{\overline{p}}dx}{\Vert u_{\epsilon }\Vert ^{\frac{4s}{N}}_{L^{2}(\mathbb {R}^{N})}} \end{aligned}$$

From (4.6) and (4.8), we have the following estimate:

$$\begin{aligned}&\frac{\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{\overline{p}}dx}{\Vert u_{\epsilon }\Vert ^{\frac{4s}{N}}_{L^{2}(\mathbb {R}^{N})}} = \left\{ \begin{aligned}&C\epsilon ^{N-\frac{N-2s}{2}\overline{p}-\frac{4s^{2}}{N}}=C,&\text {if}\&N>4s, \\&C\epsilon ^{4s-s\overline{p}-s}|\ln \epsilon |^{-\frac{1}{2}}=C|\ln \epsilon |^{-\frac{1}{2}},&\text {if}\&N=4s, \\&C\epsilon ^{N-\frac{N-2s}{2}\overline{p}-\frac{N-2s}{2}\frac{4s}{N}}=C\epsilon ^{\frac{2s(4s-N)}{N}},&\text {if} \&\frac{\overline{p}}{\overline{p}-1}2s<N<4s,\\&C\epsilon ^{\frac{N}{2}-\frac{N-2s}{2}\frac{4s}{N}}|\ln \epsilon | ,&\text {if} \&N=\frac{\overline{p}}{\overline{p}-1}2s,\\&C\epsilon ^{\frac{N-2s}{2}\overline{p}-\frac{N-2s}{2}\frac{4s}{N}}=C\epsilon ^{N-2s},&\text {if} \&2s<N<\frac{\overline{p}}{\overline{p}-1}2s. \end{aligned} \right. \end{aligned}$$

Thus,

$$\begin{aligned} \max _{t\in \mathbb {R}}\Psi ^{\mu }_{v_{\epsilon }}(t)\le \frac{s}{N}S^{\frac{N}{2s}}_{s}. \end{aligned}$$

The proof is thus finished.\(\square \)

5 \(L^{2}\)-supercritical perturbation

In this section, we consider \(N>2s\) and \(\overline{p}<q<2_{s}^{*}\). We recall the decomposition of

$$\begin{aligned} \mathcal {P}_{a,\mu }=\mathcal {P}^{+}_{a,\mu }\cup \mathcal {P}^{0}_{a,\mu }\cup \mathcal {P}^{-}_{a,\mu }. \end{aligned}$$

Lemma 5.1

\(\mathcal {P}^{0}_{a,\mu }=\emptyset \) and \(\mathcal {P}_{a,\mu }\) is a smooth manifold of codimension 1 in \(S_{a}\).

Proof

Assume by contradiction that there exists a \(u\in \mathcal {P}^{0}_{a,\mu }\), then

$$\begin{aligned} ||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx-\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx=0, \end{aligned}$$
(5.1)

and

$$\begin{aligned} 2||u||^{2}_{D_{s}(\mathbb {R}^{N})} =\mu q\gamma ^{2}_{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx+2_{s}^{*}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx. \end{aligned}$$
(5.2)

Thus, from (5.1) and (5.2), we have

$$\begin{aligned} (2-q\gamma _{q,s})\mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx=(2_{s}^{*}-2)\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx=0. \end{aligned}$$

Since \(2-q\gamma _{q,s}<0, 2_{s}^{*}-2>0\), we have \(u=0,\) which is not possible, thanks to \(u\in S_{a}.\) The rest of the proof is similar to the one of Lemma 3.2, so we omit the details here.\(\square \)

Lemma 5.2

For every \(u\in S_{a}\), there is a unique \(t_{u}\in \mathbb {R}\) such that \(t_{u}\star u \in \mathcal {P}_{a,\mu },\) where \(t_{u}\) is the unique critical point of the function of \(\Psi ^{\mu }_{u}\) and is a strict maximum point at positive level, moreover,

  1. (1)

    \(\mathcal {P}_{a,\mu }=\mathcal {P}^{-}_{a,\mu }\).

  2. (2)

    \(\Psi ^{\mu }_{u}(t)\) is strictly decreasing and concave on \((t_{u}, +\infty )\) and \(t_{u}<0\) implies that \(P_{\mu }(u)<0.\)

  3. (3)

    The map \(u\in S_{a}: t_{u}\in \mathbb {R}\) os of class \(C^{1}\).

  4. (4)

    If \(P_{\mu }(u)<0\), then \(t_{u}<0\).

Proof

Since

$$\begin{aligned} \Psi ^{\mu }_{u}(t)=E_{\mu }(t\star u)=\frac{e^{2st}}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \frac{e^{q\gamma _{q,s}st}}{q}\int _{\mathbb {R}^{N}} |u|^{q}dx-\frac{e^{2_{s}^{*}st}}{2_{s}^{*}}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx, \end{aligned}$$

and

$$\begin{aligned} (\Psi ^{\mu }_{u})'(t)=se^{2st}||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \gamma _{q,s}se^{q\gamma _{q,s}st}\int _{\mathbb {R}^{N}} |u|^{q}dx-se^{2_{s}^{*}st} \int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx, \end{aligned}$$

it follows that \((\Psi ^{\mu }_{u})'(t)=0\) if and only if

$$\begin{aligned} ||u||^{2}_{D_{s}(\mathbb {R}^{N})}= f(t):=\mu \gamma _{q,s}e^{(q\gamma _{q,s}-2)st}\int _{\mathbb {R}^{N}} |u|^{q}dx+e^{(2_{s}^{*}-2)st} \int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx. \end{aligned}$$

It is easy to see that f(t) is positive , continuous, monotone increasing and \(f(t)\rightarrow 0^{+}\) as \(t\rightarrow -\infty \) and \(f(t)\rightarrow +\infty \) as \(t\rightarrow +\infty \). Thus, there exists a unique point \(t_{u,s}\) such that \(f(t)=||u||^{2}_{D_{s}(\mathbb {R}^{N})}\). Since \(\Psi ^{\mu }_{u}\rightarrow 0^{+}\) as \(s\rightarrow -\infty \) and \(\Psi ^{\mu }_{u}\rightarrow -\infty \) as \(s\rightarrow +\infty \), we know that there is a unique \(t_{u}\in \mathbb {R}\) such that \(t_{u}\star u \in \mathcal {P}_{a,\mu },\) where \(t_{u}\) is the unique critical point of the function of \(\Psi ^{\mu }_{u}\) and is a strict maximum point at positive level. Since \(t_{u}\) is a strict maximum point, we know that \((\Psi ^{\mu }_{u})''(t_{u})\le 0.\) Because \(\mathcal {P}^{0}_{a,\mu }=\emptyset ,\) we have \((\Psi ^{\mu }_{u})''(t_{u})\ne 0,\) which implies that \(t_{u}\star u \in \mathcal {P}^{-}_{a,\mu }\), since \(\Psi ^{\mu }_{u}(t)\) has exactly one maximum point, so \(\mathcal {P}_{a,\mu }=\mathcal {P}^{-}_{a,\mu }.\) To prove that the map \(u\in S_{a}: t_{u}\in \mathbb {R}\) os of class \(C^{1}\), we can apply the implicit function theorem as Lemma 3.3. Finally, since \((\Psi ^{\mu }_{u})'(t)<0\) if and only if \(t>t_{u}\), so \(P_{\mu }(u)=(\Psi ^{\mu }_{u})'(0)<0\) if and only if \(t_{u}<0\).\(\square \)

Lemma 5.3

There holds

$$\begin{aligned} m(a,\mu )=\inf _{\mathcal {P}_{a,\mu }}E_{\mu }>0. \end{aligned}$$

Proof

If \(u\in \mathcal {P}_{a,\mu },\) then \(P_{\mu }(u)=0\), then by fractional Gagliardo–Nirenberg–Sobolev inequality (2.4) and Sobolev inequality (1.5), we have

$$\begin{aligned} ||u||^{2}_{D_{s}(\mathbb {R}^{N})}&=\mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx+\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\\&\le \mu \gamma _{q,s}C^{q}_{N,q,s}a^{(1-\gamma _{q,s})q}||u||^{q\gamma _{q,s}}_{D_{s}(\mathbb {R}^{N})}+S^{-\frac{2_{s}^{*}}{2}}_{s}||u||^{2_{s}^{*}}_{D_{s}(\mathbb {R}^{N})} \end{aligned}$$

Thus, from above inequality and \(||u||^{2}_{D_{s}(\mathbb {R}^{N})}\ne 0\) (since \(u\in S_{a}\)), we have

$$\begin{aligned} \mu \gamma _{q,s}C^{q}_{N,q,s}a^{(1-\gamma _{q,s})q}||u||^{q\gamma _{q,s}-2}_{D_{s}(\mathbb {R}^{N})}+S^{-\frac{2_{s}^{*}}{2}}_{s}||u||^{2_{s}^{*}-2}_{D_{s}(\mathbb {R}^{N})}\ge 1, \ \forall \ u\in \mathcal {P}_{a,\mu }, \end{aligned}$$

which implies that \(\inf _{u\in \mathcal {P}_{a,\mu }}||u||_{D_{s}(\mathbb {R}^{N})}>0\). Since

$$\begin{aligned} \mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx+\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx=||u||^{2}_{D_{s}(\mathbb {R}^{N})}, \end{aligned}$$

we have

$$\begin{aligned} \inf _{u\in \mathcal {P}_{a,\mu }}\left[ \mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx+\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\right] >0. \end{aligned}$$

Thus, from \(P_{\mu }(u)=0\) and above inequality, we have

$$\begin{aligned} \inf _{u\in \mathcal {P}_{a,\mu }}E_{\mu }(u)&=\inf _{u\in \mathcal {P}_{a,\mu }}\left[ \frac{1}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})} -\frac{\mu }{q}\int _{\mathbb {R}^{N}}|u|^{q}dx-\frac{1}{2_{s}^{*}}\int _{\mathbb {R}^{N}}|u|^{2_{s}^{*}}dx\right] \\&=\inf _{u\in \mathcal {P}_{a,\mu }}\left[ \frac{\mu }{q}\left( \frac{q\gamma _{q,s}}{2}-1\right) \int _{\mathbb {R}^{N}}|u|^{q}dx+\frac{s}{N}\int _{\mathbb {R}^{N}}|u|^{2_{s}^{*}}dx\right] > 0. \end{aligned}$$

Therefore,

$$\begin{aligned} m(a,\mu )=\inf _{\mathcal {P}_{a,\mu }}E_{\mu }>0. \end{aligned}$$

This finishes the proof.\(\square \)

Lemma 5.4

There exists \(k>0\) sufficiently small such that

$$\begin{aligned} 0<\sup _{\overline{A_{k}}}E_{\mu }<m(a,\mu )\ \text {and}\ u\in \overline{A_{k}}\Rightarrow E_{\mu }(u), P_{\mu }(u)>0, \end{aligned}$$

where \( A_{k}=\left\{ u\in S_{a}: ||u||^{2}_{D_{s}(\mathbb {R}^{N})}<k\right\} .\)

Proof

By fractional Gagliardo–Nirenberg–Sobolev inequality (2.4) and Sobolev inequality (1.5), we have

$$\begin{aligned} E_{\mu }(u)\ge \frac{1}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{\mu }{q}C^{q}_{N,q,s}a^{q(1-\gamma _{q,s})}||u||^{q\gamma _{q,s}}_{D_{s}(\mathbb {R}^{N})}-\frac{1}{2_{s}^{*}}S^{-\frac{2_{s}^{*}}{2}}_{s}||u||^{2_{s}^{*}}_{D_{s}(\mathbb {R}^{N})}>0, \end{aligned}$$

and

$$\begin{aligned} P_{\mu }(u)&=s||u||^{2}_{D_{s}(\mathbb {R}^{N})}-s\mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx-s\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\\&\ge s||u||^{2}_{D_{s}(\mathbb {R}^{N})}-s\mu \gamma _{q,s}C^{q}_{N,q,s}a^{q(1-\gamma _{q,s})}||u||^{q\gamma _{q,s}}_{D_{s}(\mathbb {R}^{N})}-sS^{-\frac{2_{s}^{*}}{2}}_{s}||u||^{2_{s}^{*}}_{D_{s}(\mathbb {R}^{N})}>0. \end{aligned}$$

If \(u\in \overline{A_{k}}\) for k small enough. By Lemma 5.3, we know that \(m(a,\mu )>0\), thus if necessary replacing k with smaller quantity, we also have

$$\begin{aligned} E_{\mu }(u)\le \frac{1}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}< m(a,\mu ). \end{aligned}$$

This ends the proof.\(\square \)

In order to apply Proposition 2.2 and recover compactness, we need an estimate from above on \(m_{r}(a,\mu )=\inf _{\mathcal {P}_{a,\mu }\bigcap S^{r}_{a}}E_{\mu }\), where \(S^{r}_{a}\) is the subset of the radial functions in \(S_{a}.\)

Lemma 5.5

If one of following conditions holds:

  1. (1)

    \( N>4s\ \text {and}\ \mu a^{q(1-\gamma _{q,s})}<\frac{S^{\frac{N}{4s}q(1-\gamma _{q,s})}_{s}}{\gamma _{q,s}}\);

  2. (2)

    \( N=\frac{q}{q-1}2s\ \text {and}\ \mu a^{q(1-\gamma _{q,s})}<\frac{S^{\frac{N}{4s}q(1-\gamma _{q,s})}_{s}}{\gamma _{q,s}}\);

  3. (3)

    \(N=4s \ \text {or}\ \frac{q}{q-1}2s<N<4s\ \text {or}\ 2s<N<\frac{q}{q-1}2s \),

then we have \(m_{r}(a,\mu )< \frac{s}{N}S^{\frac{N}{2s}}_{s}.\)

Proof

Let us recall the definition of \(u_{\epsilon }\) and \(v_{\epsilon }\) as Lemma 4.5. It is easy to see that \(u_{\epsilon }\in C^{\infty }_{0}(\mathbb {R}^{N},[0,1])\) and \(v_{\epsilon }\in S_{a}^{r}.\) By Lemma 4.2, we know that

$$\begin{aligned} m_{r}(a,\mu )=\inf _{\mathcal {P}_{a,\mu }\bigcap S^{r}_{a}}E_{\mu }\le E_{\mu }(t_{v_{\epsilon },\mu }\star v_{\epsilon })=\max _{t\in \mathbb {R}}E_{\mu }(t\star v_{\epsilon }). \end{aligned}$$

By the same argument as step 1 in Lemma 4.5, we have

$$\begin{aligned} \Psi ^{0}_{v_{\epsilon }}(t_{v_{\epsilon },0})=\frac{s}{N}S^{\frac{N}{2s}}_{s}+O(\epsilon ^{N-2s}). \end{aligned}$$

Step 1 Estimate on \(t_{v_{\epsilon }, \mu }\). Since

$$\begin{aligned} \Psi ^{\mu }_{v_{\epsilon }}(t)=E_{\mu }(t\star v_{\epsilon })=\frac{e^{2st}}{2}||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \frac{e^{q\gamma _{q,s}st}}{q}\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{q}dx-\frac{e^{2_{s}^{*}st}}{2_{s}^{*}}\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx \end{aligned}$$

and \(t_{v_{\epsilon },\mu }\) be the unique maximum point of \(\Psi ^{\mu }_{v_{\epsilon }}(t)\), then by \((\Psi ^{\mu }_{v_{\epsilon }})'(t)=P_{\mu }(t_{v_{\epsilon },\mu }\star v_{\epsilon })=0,\) we have

$$\begin{aligned}&e^{2_{s}^{*}st_{v_{\epsilon }, \mu }} \int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx=e^{2st_{v_{\epsilon }, \mu }}||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}\\&\quad -\mu \gamma _{q,s}e^{q\gamma _{q,s}st_{v_{\epsilon }, \mu }}\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{q}dx\le e^{2st_{v_{\epsilon }, \mu }}||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}, \end{aligned}$$

which means that

$$\begin{aligned} e^{st_{v_{\epsilon }, \mu }}\le \left( \frac{||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx}\right) ^{\frac{1}{2_{s}^{*}-2}}. \end{aligned}$$
(5.3)

By (5.3), \(q\gamma _{q,s}>2\) and \(v_{\epsilon }=\frac{au_{\epsilon }}{\Vert u_{\epsilon }\Vert _{L^{2}(\mathbb {R}^{N})}}\), we have

$$\begin{aligned}&e^{(2_{s}^{*}-2)st_{v_{\epsilon }, \mu }}\nonumber \\&\quad =\frac{||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx}-\mu \gamma _{q,s}e^{(q\gamma _{q,s}-2)st_{v_{\epsilon }, \mu }}\frac{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{q}dx}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx}\nonumber \\&\quad \ge \frac{||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx} -\mu \gamma _{q,s}\frac{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{q}dx}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx}\left( \frac{||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx}\right) ^{\frac{q\gamma _{q,s}-2}{2_{s}^{*}-2}}\nonumber \\&\quad \ge \frac{\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}} \frac{||u_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx}\nonumber \\&\qquad -\mu \gamma _{q,s}\frac{\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-q}_{L^{2} (\mathbb {R}^{N})}}{a^{2_{s}^{*}-q}}\frac{\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{q}dx}{\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx}\left( \frac{\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}} \frac{||u_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx}\right) ^{\frac{q\gamma _{q,s}-2}{2_{s}^{*}-2}}\nonumber \\&\quad \ge \frac{\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}} \frac{\left( ||u_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}\right) ^{\frac{q\gamma _{q,s}-2}{2_{s}^{*}-2}}}{\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx}\nonumber \\&\qquad \left[ \left( ||u_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}\right) ^{\frac{2_{s}^{*}-q\gamma _{q,s}}{2_{s}^{*}-2}}-\frac{\mu \gamma _{q,s}a^{q(1-\gamma _{q,s})}\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{q}dx}{\left( \int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx\right) ^{\frac{q\gamma _{q,s}-2}{2_{s}^{*}-2}}\Vert u_{\epsilon }\Vert ^{q(1-\gamma _{q,s})}_{L^{2}(\mathbb {R}^{N})}}\right] . \end{aligned}$$
(5.4)

By the estimates in (4.5), (4.6), (4.7) and (4.8), we can infer that there exist \(C_{1}, C_{2}, C_{3}>0\) (depending on Nq) such that

$$\begin{aligned} \left( ||u_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}\right) ^{\frac{2_{s}^{*}-q\gamma _{q,s}}{2_{s}^{*}-2}}\ge C_{1} \ \text {and}\ C_{2}\le \left( \int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx\right) ^{\frac{q\gamma _{q,s}-2}{2_{s}^{*}-2}}\le \frac{1}{C_{2}} \end{aligned}$$
(5.5)

and

$$\begin{aligned}&\frac{\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{q}dx}{\Vert u_{\epsilon }\Vert ^{q(1-q\gamma _{q,s})}_{L^{2}(\mathbb {R}^{N})}} \le \left\{ \begin{aligned}&C\epsilon ^{N-\frac{N-2s}{2}q-q(1-\gamma _{q,s})},&\text {if}\&N>4s, \\&C\epsilon ^{N-\frac{N-2s}{2}q-q(1-\gamma _{q,s})}|\ln \epsilon |^{\frac{q(\gamma _{q,s}-1)}{2}},&\text {if}\&N=4s, \\&C\epsilon ^{N-\frac{N-2s}{2}q-\frac{N-2s}{2}q(1-\gamma _{q,s})},&\text {if} \&\frac{q}{q-1}2s<N<4s,\\&C\epsilon ^{\frac{N}{2}-\frac{N-2s}{2}q(1-\gamma _{q,s})}|\ln \epsilon | ,&\text {if} \&N=\frac{q}{q-1}2s,\\&C\epsilon ^{\frac{N-2s}{2}q-\frac{N-2s}{2}q(1-\gamma _{q,s})},&\text {if} \&2s<N<\frac{q}{q-1}2s. \end{aligned} \right. \end{aligned}$$
(5.6)

Next, we claim that

$$\begin{aligned} e^{(2_{s}^{*}-2)st_{v_{\epsilon }, \mu }}\ge C\frac{\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}}, \end{aligned}$$

under suitable conditions.

Case 1 \(N>4s\). Since \(\overline{p}<q<2_{s}^{*}\), we can deduce that

$$\begin{aligned} N-\frac{N-2s}{2}q-q(1-\gamma _{q,s})<0. \end{aligned}$$
(5.7)

Indeed, since \(\overline{p}<q<2_{s}^{*}\), we have \(4s/N<q-2<4s/(N-2s)\), so

$$\begin{aligned}&N-\frac{N-2s}{2}q-q(1-\gamma _{q,s})\\&\quad =N-\frac{N-2s}{2}(q-2)-(N-2s)-(q-2)-2+\frac{N(q-2)}{2s}:=f(q-2), \end{aligned}$$

it is easy to deduce that \(f(q-2)\) is strictly increasing about \(q-2\), since \(f(\frac{4s}{N-2s})=0\), thus we obtain

$$\begin{aligned} N-\frac{N-2s}{2}q-q(1-\gamma _{q,s})<0. \end{aligned}$$

So we can not get

$$\begin{aligned} e^{(2_{s}^{*}-2)st_{v_{\epsilon }, \mu }}\ge C\frac{\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}} \left[ C_{1}-\mu \gamma _{q,s}a^{q(1-\gamma _{q,s})}\frac{C_{3}}{C_{2}}o_{\epsilon }(1)\right] \ge C\frac{\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}} \end{aligned}$$

for a positive constant \(C=C(N,q,\mu ,a)>0\) for every \(\epsilon \in (0,\epsilon _{0})\) with \(\epsilon _{0}\) sufficiently small. Thus, we have to give a more precise estimate, let us recall the inequality about \(e^{(2_{s}^{*}-2)st_{v_{\epsilon }, \mu }}\) in (5.4), by well-known interpolation inequality, we have

$$\begin{aligned} \frac{\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{q}dx}{\left( \int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx\right) ^{\frac{q\gamma _{q,s}-2}{2_{s}^{*}-2}}\Vert u_{\epsilon }\Vert ^{q(1-\gamma _{q,s})}_{L^{2}(\mathbb {R}^{N})}}&\le \frac{\left( \int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx\right) ^{\frac{q-2}{2_{s}^{*}-2}}\left( \int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2}dx\right) ^{\frac{2_{s}^{*}-q}{2_{s}^{*}-2}}}{\left( \int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx\right) ^{\frac{q\gamma _{q,s}-2}{2_{s}^{*}-2}}\Vert u_{\epsilon }\Vert ^{q(1-\gamma _{q,s})}_{L^{2}(\mathbb {R}^{N})}}\nonumber \\&\le \left( \int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx\right) ^{\frac{q(1-\gamma _{q,s})}{2_{s}^{*}-2}}=\left( \left( \int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx\right) ^{\frac{2}{2_{s}^{*}}}\right) ^{\frac{2_{s}^{*}-q\gamma _{q,s}}{2_{s}^{*}-2}}. \end{aligned}$$
(5.8)

Therefore, by (5.4) and (5.8), we have

$$\begin{aligned}&e^{(2_{s}^{*}-2)st_{v_{\epsilon }, \mu }} =\frac{||v_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx}-\mu \gamma _{q,s}e^{(q\gamma _{q,s}-2)st_{v_{\epsilon }, \mu }}\frac{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{q}dx}{\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{2_{s}^{*}}dx}\nonumber \\&\quad \ge \frac{\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}} \frac{\left( ||u_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}\right) ^{\frac{q\gamma _{q,s}-2}{2_{s}^{*}-2}}}{\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx}\nonumber \\&\quad \left[ \left( ||u_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}\right) ^{\frac{2_{s}^{*}-q\gamma _{q,s}}{2_{s}^{*}-2}}-\mu \gamma _{q,s}a^{q(1-\gamma _{q,s})}\left( \left( \int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx\right) ^{\frac{2}{2_{s}^{*}}}\right) ^{\frac{2_{s}^{*}-q\gamma _{q,s}}{2_{s}^{*}-2}}\right] . \end{aligned}$$
(5.9)

Thus, if the right hand of above is positive provided that

$$\begin{aligned} \mu \gamma _{q,s}a^{q(1-\gamma _{q,s})}<\left( \frac{||u_{\epsilon }||^{2}_{D_{s}(\mathbb {R}^{N})}}{\left( \int _{\mathbb {R}^{N}} |u_{\epsilon }|^{2_{s}^{*}}dx\right) ^{\frac{2}{2_{s}^{*}}}}\right) ^{{\frac{2_{s}^{*}-q\gamma _{q,s}}{2_{s}^{*}-2}}}=S^{\frac{N}{4s}q(1-\gamma _{q,s})}_{s}+O(\epsilon ^{N-2s}) \end{aligned}$$

Thus, if \(N>4s\ \text {and}\ \mu a^{q(1-\gamma _{q,s})}<\frac{S^{\frac{N}{4s}q(1-\gamma _{q,s})}_{s}}{\gamma _{q,s}}\), we have

$$\begin{aligned} e^{(2_{s}^{*}-2)st_{v_{\epsilon }, \mu }}\ge \frac{C\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}} . \end{aligned}$$

Case 2 \(N=4s\). Then we have \(3<q<4\) and \(|\ln \epsilon |\backsim \frac{1}{\epsilon }\) as \(\epsilon \rightarrow 0.\) Thus

$$\begin{aligned} \epsilon ^{N-\frac{N-2s}{2}q-q(1-\gamma _{q,s})}|\ln \epsilon |^{\frac{q(\gamma _{q,s}-1)}{2}}=\epsilon ^{(4-q)(s-1)}|\ln \epsilon |^{q-4}\rightarrow 0\ \text {as} \ \epsilon \rightarrow 0. \end{aligned}$$

Furthermore,

$$\begin{aligned} \frac{\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{q}dx}{\Vert u_{\epsilon }\Vert ^{q(1-q\gamma _{q,s})}_{L^{2}(\mathbb {R}^{N})}}\le C\epsilon ^{N-\frac{N-2s}{2}q-q(1-\gamma _{q,s})}|\ln \epsilon |^{\frac{q(\gamma _{q,s}-1)}{2}}=o_{\epsilon }(1). \end{aligned}$$

So, we have

$$\begin{aligned} e^{(2_{s}^{*}-2)st_{v_{\epsilon }, \mu }}\ge C\frac{\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}} \left[ C_{1}-\mu \gamma _{q,s}a^{q(1-\gamma _{q,s})}\frac{C_{3}}{C_{2}}o_{\epsilon }(1)\right] \ge C\frac{\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}}. \end{aligned}$$

Case 3 \(\frac{q}{q-1}2s<N<4s\). By the same arguments as (5.7), we have

$$\begin{aligned} N-\frac{N-2s}{2}q-\frac{N-2s}{2}q(1-\gamma _{q,s})>0. \end{aligned}$$

Thus,

$$\begin{aligned} \epsilon ^{N-\frac{N-2s}{2}q-\frac{N-2s}{2}q(1-\gamma _{q,s})} \rightarrow 0 \ \text {as} \ \epsilon \rightarrow 0. \end{aligned}$$

Therefore,

$$\begin{aligned} \frac{\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{q}dx}{\Vert u_{\epsilon }\Vert ^{q(1-q\gamma _{q,s})}_{L^{2}(\mathbb {R}^{N})}}\le C\epsilon ^{N-\frac{N-2s}{2}q-\frac{N-2s}{2}q(1-\gamma _{q,s})}=o_{\epsilon }(1). \end{aligned}$$

So, we have

$$\begin{aligned} e^{(2_{s}^{*}-2)st_{v_{\epsilon }, \mu }}\ge C\frac{\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}} \left[ C_{1}-\mu \gamma _{q,s}a^{q(1-\gamma _{q,s})}\frac{C_{3}}{C_{2}}o_{\epsilon }(1)\right] \ge C\frac{\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}}. \end{aligned}$$

Case 4 \(N=\frac{q}{q-1}2s\). By the similar arguments as Case 1, we get

$$\begin{aligned} C\epsilon ^{\frac{N}{2}-\frac{N-2s}{2}q(1-\gamma _{q,s})}|\ln \epsilon |\rightarrow +\infty \ \text {as} \ \epsilon \rightarrow 0. \end{aligned}$$

Thus, by the same argument as Case 1, we know that if \(N=\frac{q}{q-1}2s\ \text {and}\ \mu a^{q(1-\gamma _{q,s})}<\frac{S^{\frac{N}{4s}q(1-\gamma _{q,s})}_{s}}{\gamma _{q,s}}\), then we have

$$\begin{aligned} e^{(2_{s}^{*}-2)st_{v_{\epsilon }, \mu }}\ge \frac{C\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}} . \end{aligned}$$

Case 5 \(2s<N<\frac{q}{q-1}2s\). It is easy to see that

$$\begin{aligned} \frac{\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{q}dx}{\Vert u_{\epsilon }\Vert ^{q(1-q\gamma _{q,s})}_{L^{2}(\mathbb {R}^{N})}}\le C\epsilon ^{\frac{N-2s}{2}q-\frac{N-2s}{2}q(1-\gamma _{q,s})}=o_{\epsilon }(1). \end{aligned}$$

Then we have

$$\begin{aligned} e^{(2_{s}^{*}-2)st_{v_{\epsilon }, \mu }}\ge C\frac{\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}} \left[ C_{1}-\mu \gamma _{q,s}a^{q(1-\gamma _{q,s})}\frac{C_{3}}{C_{2}}o_{\epsilon }(1)\right] \ge C\frac{\Vert u_{\epsilon }\Vert ^{2_{s}^{*}-2}_{L^{2}(\mathbb {R}^{N})}}{a^{2_{s}^{*}-2}}. \end{aligned}$$

Step 2 Estimate on \(\max _{t\in \mathbb {R}}\Psi ^{\mu }_{v_{\epsilon }}(t)\).

$$\begin{aligned}&\max _{t\in \mathbb {R}}\Psi ^{\mu }_{v_{\epsilon }}(t)=\Psi ^{\mu }_{v_{\epsilon }}(t_{v_{\epsilon },\mu })=\Psi ^{0}_{v_{\epsilon }}(t_{v_{\epsilon },\mu })-\mu \frac{e^{q \gamma _{q,s} st_{v_{\epsilon },\mu }}}{q}\int _{\mathbb {R}^{N}} |v_{\epsilon }|^{q}dx\\&\quad \le \sup _{\mathbb {R}}\Psi ^{0}_{v_{\epsilon }}-\frac{\mu C}{q}\frac{\Vert u_{\epsilon }\Vert ^{q\gamma _{q,s}}_{L^{2}(\mathbb {R}^{N})}}{a^{q\gamma _{q,s}}}\frac{a^{q}}{\Vert u_{\epsilon }\Vert ^{q}_{L^{2}(\mathbb {R}^{N})}} \int _{\mathbb {R}^{N}} |u_{\epsilon }|^{q}dx\\&\quad \le \frac{s}{N}S^{\frac{N}{2s}}_{s}+O(\epsilon ^{N-2s})-C\frac{\mu }{q}\gamma _{q,s}a^{q(1-\gamma _{q,s})}\frac{\int _{\mathbb {R}^{N}} |u_{\epsilon }|^{q}dx}{\Vert u_{\epsilon }\Vert ^{q(1-q\gamma _{q,s})}_{L^{2}(\mathbb {R}^{N})}}. \end{aligned}$$

By (5.6), we know that

$$\begin{aligned} \max _{t\in \mathbb {R}}\Psi ^{\mu }_{v_{\epsilon }}(t)\le \frac{s}{N}S^{\frac{N}{2s}}_{s}, \end{aligned}$$

for \(\epsilon \) small enough. This completes the proof.\(\square \)

6 Proof of Theorem 1.1

Let \(\{v_{n}\}\) be a minimizing sequence for \(\inf _{A_{R_{0}}}E_{\mu }(u)\). By Lemma 3.3, for every n we can take \(t_{v_{n}}\star v_{n}\in \mathcal {P}^{+}_{a,\mu }\) such that \(||t_{v_{n}}\star v_{n}||_{D_{s}(\mathbb {R}^{N})}\le R_{0}\) and

$$\begin{aligned} E_{\mu }(u)(t_{v_{n}}\star v_{n})=\min \{E_{\mu }(t\star v_{n}):t\in \mathbb {R}\ \text {and}\ ||t\star v_{n}||_{D_{s}(\mathbb {R}^{N})}< R_{0} \}\le E_{\mu }(v_{n}). \end{aligned}$$

Thus, we obtain a new minimizing sequence \(\{w_{n}=t_{v_{n}}\star v_{n}\}\) with \(w_{n}\in S^{r}_{a}\cap \mathcal {P}^{+}_{a,\mu } \) radially decreasing for every n. By Lemma 3.4, we have \(||w_{n}||_{D_{s}(\mathbb {R}^{N})}<R_{0}-\rho \) for every n and hence by Ekeland’s variational principle in a standard way, we know the existence of a new minimizing sequence for \(\{u_{n}\}\subset A_{R_{0}}\) for \(m(a,\mu )\) with \(\Vert u_{n}-w_{n}\Vert \rightarrow 0\) as \(n\rightarrow +\infty \), which is also a Palais–Smale sequence for \(E_{\mu }\) on \(S_{a}\). By the boundedness of \(\{w_{n}\}\), \(\Vert u_{n}-w_{n}\Vert \rightarrow 0\), Brézis–Lieb lemma and Sobolev embedding theorem, we have

$$\begin{aligned} \Vert u_{n}\Vert ^{2}_{D_{s}(\mathbb {R}^{N})}= & {} \Vert u_{n}-w_{n}\Vert ^{2}_{D_{s}(\mathbb {R}^{N})}+\Vert w_{n}\Vert ^{2}_{D_{s}(\mathbb {R}^{N})})+o_{n}(1)=\Vert w_{n}\Vert ^{2}_{D_{s}(\mathbb {R}^{N})})+o_{n}(1),\\ \int _{\mathbb {R}^{N}}|u_{n}|^{p}dx= & {} \int _{\mathbb {R}^{N}}|u_{n}-w_{n}|^{p}dx+\int _{\mathbb {R}^{N}}|w_{n}|^{p}+o_{n}(1)\\= & {} \int _{\mathbb {R}^{N}}|w_{n}|^{p}+o_{n}(1), \ \text {for}\ \ p\in [2,2_{s}^{*}]. \end{aligned}$$

Thus,

$$\begin{aligned} P_{\mu }(u_{n})=P_{\mu }(u_{n})+o_{n}(1)\rightarrow \ \text {as}\ n\rightarrow +\infty . \end{aligned}$$

Therefore, one of the alternatives in Proposition 2.2 holds. We prove that the second alternative in Proposition 2.2 occurs. Assume by contradiction that there exists a sequence \(u_{n}\rightharpoonup u\) weakly in \(H^{s}(\mathbb {R}^{N})\) but not strongly, where \(u\not \equiv 0\) is a solution of (1.1) for some \(\lambda <0\), and

$$\begin{aligned} E_{\mu }(u)\le m(a,\mu )- \frac{s}{N}S^{\frac{N}{2s}}_{s}. \end{aligned}$$

Since u is a solution of(1.1), we have \(P_{\mu }(u)=0,\) which implies that

$$\begin{aligned} ||u||^{2}_{D_{s}(\mathbb {R}^{N})}=\mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u|^{q}dx+\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx. \end{aligned}$$

Therefore

$$\begin{aligned} m(a,\mu )&\ge E_{\mu }(u)+ \frac{s}{N}S^{\frac{N}{2s}}_{s}= \frac{s}{N}S^{\frac{N}{2s}}_{s}+\frac{s}{N}||u||^{2}_{D_{s}(\mathbb {R}^{N})} -\frac{\mu }{q}\left( 1-\frac{q\gamma _{q,s}}{2_{s}^{*}}\right) \int _{\mathbb {R}^{N}}|u|^{q}dx\\&\ge \frac{s}{N}S^{\frac{N}{2s}}_{s}+\frac{s}{N}||u||^{2}_{D_{s}(\mathbb {R}^{N})} -\frac{\mu }{q}\left( 1-\frac{q\gamma _{q,s}}{2_{s}^{*}}\right) C^{q}_{N,q,s}a^{q(1-\gamma _{q,s})}||u||^{q\gamma _{q,s}}_{D_{s}(\mathbb {R}^{N})}. \end{aligned}$$

Next, we show that the right hand side of above inequality is positive under suitable conditions, then we can get a contradiction with \(m(a,\mu )<0\). Let

$$\begin{aligned} \vartheta (t)=\frac{s}{N}t^{2} -\frac{\mu }{q}\left( 1-\frac{q\gamma _{q,s}}{2_{s}^{*}}\right) C^{q}_{N,q,s}a^{q(1-\gamma _{q,s})}t^{q\gamma _{q,s}}. \end{aligned}$$

Then it is easy to see that the function \(\vartheta (t)\) has a unique minimum point \(\overline{t}\) and

$$\begin{aligned} \vartheta (\overline{t})=-\frac{2-q\gamma _{q,s}}{q}\left[ \frac{N\gamma _{q,s}}{s}\right] ^{\frac{q\gamma _{q,s}}{2-q\gamma _{q,s}}}\left[ \frac{2_{s}^{\star }-q\gamma _{q,s}}{22_{s}^{\star }}C^{q}_{N,q,s}\right] ^{\frac{2}{2-q\gamma _{q,s}}}[\mu a^{q(1-\gamma _{q,s})}]^{\frac{2}{2-q\gamma _{q,s}}}<0. \end{aligned}$$

If

$$\begin{aligned} \vartheta (\overline{t})>-\frac{s}{N}S^{\frac{N}{2s}}_{s}, \end{aligned}$$

which yields that

$$\begin{aligned} \mu a^{q(1-\gamma _{q,s})}<\frac{22_{s}^{\star }}{(2_{s}^{\star }-q\gamma _{q,s})C^{q}_{N,q,s}}\left( \frac{Nq\gamma ^{2}_{q,s}S^{\frac{N}{2s}}_{s}}{(2-q\gamma _{q,s})s}\right) ^{\frac{2-q\gamma _{q,s}}{2}}, \end{aligned}$$

then we have

$$\begin{aligned} m(a,\mu )\ge \frac{s}{N}S^{\frac{N}{2s}}_{s}+\vartheta (||u||_{D_{s}(\mathbb {R}^{N})})\ge \frac{s}{N}S^{\frac{N}{2s}}_{s}+\vartheta (\overline{t})>0, \end{aligned}$$

which contradicts the fact that \(m(a,\mu )<0\). Thus \(u_{n}\rightarrow u\) strongly in \(H^{s}(\mathbb {R}^{N}),\) \(E_{\mu }(u)=m(a,\mu )\) and u solves (1.1)–(1.2) for some \(\lambda <0.\) It remains to show that any ground state is a local minimizer for \(E_{\mu }\) on \(A_{R_{0}}\). Since \(E_{\mu }(u)=m(a,\mu )\), then \(u\in \mathcal {P}_{a,\mu }\) and \(E_{\mu }(u)<0\), so by Lemma 3.3 we have that \(u\in \mathcal {P}^{+}_{a,\mu } \subset A_{R_{0}}\) and

$$\begin{aligned} E_{\mu }(u)=m(a,\mu )=\inf _{ A_{R_{0}}}E_{\mu }(u)\ \text {and}\ ||u||_{D_{s}(\mathbb {R}^{N})}<R_{0}. \end{aligned}$$

Therefore, the proof of Theorem 1.1 is complete.

7 Proof of Theorems 1.21.3

We first list some well-known results, which will be used to prove Theorems 1.21.3. For this purpose, we give the following definition.

Definition 7.1

(see [22, Definition 3.1]) Let B be a closed subset of X. We shall say that a class \(\mathcal {F}\) of compact subsets of X is a homotopy-stable family with boundary B provided that

  1. (a)

    every set in \(\mathcal {F}\) contains B.

  2. (b)

    for any set A in \(\mathcal {F}\) and any \(\eta \in ([0,1]\times X;X)\) satisfying \(\eta (t,x)=x\) for all \((t,x)\in ({0}\times X)\cup ([0,1]\times B)\), we have \(\eta ({1}\times A)\in \mathcal {F}.\)

Theorem 7.1

(see [22, Theorem 3.2]) Let \(\varphi \) be a \(C^{1}\) function on a complete connected \(C^{1}\)-Finsler manifold X (without boundary) and consider a homotopy-stable family \(\mathcal {F}\) of compact subsets of X with a closed boundary B. Set \(c=c(\varphi ,\mathcal {F})=\inf \limits _{A\in \mathcal {F}}\max \limits _{x\in A}\varphi (x)\) and suppose that

$$\begin{aligned} \sup \varphi (B)<c. \end{aligned}$$

Then, for any sequence of sets \((A_{n})_{n}\) in \(\mathcal {F}\) such that \(\lim \limits _{n}\sup \limits _{A_{n}}\varphi =c,\) there exists a sequence \((x_{n})_{n}\) in X such that

\(\mathrm{(i)}\) \(\lim \limits _{n}\varphi (x_{n})=c \ \ \)(ii)\(\ \lim \limits _{n}\Vert d\varphi (x_{n})\Vert =0 \ \ \)(iii)\(\ \lim \limits _{n}\mathrm{dist}(x_{n}, A_{n})=0.\)

Moreover, if \(d\varphi \) is uniformly continuous, then \(x_{n}\) can be chosen to be in \(A_{n}\) for each n.

Now we are in a position to prove Theorems 1.21.3.

Case 1 \(L^{2}\)-critical perturbation, i.e., \(q=\overline{p}\). Let \(k>0\) be defined by Lemma 4.4, we follow the ideas introduced in [26] and consider the following functional \(\overline{E}_{\mu }(s,\mu ): \mathbb {R}\times H^{s}(\mathbb {R}^{N})\rightarrow \mathbb {R}\):

$$\begin{aligned} \widetilde{E}_{\mu }(t,\mu )=E_{\mu }(t\star u)=\left[ \frac{1}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{\mu }{\overline{p}}\int _{\mathbb {R}^{N}} |u|^{\overline{p}}dx\right] e^{2st}-\frac{e^{2_{s}^{*}st}}{2_{s}^{*}}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx. \end{aligned}$$
(7.1)

It is easy to see that \(\overline{E}_{\mu }(t,\mu )\) is of class of \(C^{1}\), since \(\overline{E}_{\mu }(t,\mu )\) is invariant under rotations applied to u, a Palais–Smale sequence for \(\overline{E}_{\mu }(t,\mu )|_{\mathbb {R}\times S^{r}_{a}}\) is a Palais–Smale sequence for \(\overline{E}_{\mu }(t,\mu )|_{\mathbb {R}\times S_{a}}\). Let \(E^{c}\) be the closed sublevel set \(\{u\in S_{a}: E_{\mu }\le c\},\) we introduce the minimax class

$$\begin{aligned} \Gamma :=\{\gamma =(\alpha ,\beta )\in C([0,1],\mathbb {R}\times S^{r}_{a}): \gamma (0)\in (0,\overline{A_{k}}),\gamma (1)\in (0,E^{0}) \} \end{aligned}$$
(7.2)

with associated minimax level

$$\begin{aligned} \sigma (a,\mu ):=\inf _{\gamma \in \Gamma }\max _{(t,u)\in \gamma ([0,1])}\widetilde{E}_{\mu }(t,\mu ). \end{aligned}$$

Since \(||t\star u||^{2}_{D_{s}(\mathbb {R}^{N})}\rightarrow 0^{+}\) as \(t\rightarrow -\infty \) and \(E_{\mu }(t\star u)\rightarrow -\infty \) as \(t\rightarrow +\infty \). Let \(u\in S^{r}_{a}\). There exist \(t_{0}\ll -1\) and \(t_{1}\gg 1\) such that

$$\begin{aligned} \gamma _{u}: \tau \in [0,1]\rightarrow (0,((1- \tau )t_{0}+\tau t_{1})\star u)\in \mathbb {R}\times S^{r}_{a} \end{aligned}$$
(7.3)

is a path in \(\Gamma \). Then \(\sigma (a,\mu )\) is a real number. For any \(\gamma =(\alpha ,\beta )\in \Gamma \), let us consider the function

$$\begin{aligned} P_{\gamma }:\tau \in [0,1]\rightarrow P(\alpha (\tau )\star \beta (\tau ))\in \mathbb {R}. \end{aligned}$$

We have \(P_{\gamma }(0)=P(\beta (0))>0\), by Lemmas 4.3 and 4.4. Since \(\Psi _{\beta (1)}(t)>0\) for every \(t\in (-\infty , t_{\beta (1)})\) and \(\Psi _{\beta (1)}(0)=E_{\mu }(\beta (1))\le 0,\) we have that \(t_{\beta (1)}<0.\) Thus, by Lemma 4.2, we have \(P_{\gamma }(1)=P(\beta (1))<0.\) Moreover, the map \(\tau :\alpha (\tau )\star \beta (\tau )\) is continuous from [0, 1] to \(H^{s}(\mathbb {R}^{N})\), hence we deduce that there exists \(\tau _{\gamma }\in (0,1)\) such that \(P_{\gamma }(\tau _{\gamma })=0\), namely \(\alpha (\tau _{\gamma })\star \beta (\tau _{\gamma })\in \mathcal {P}_{a,\mu } \), this implies that

$$\begin{aligned} \max _{\gamma ([0,1])}\widetilde{E}_{\mu }\ge \widetilde{E}_{\mu }(\gamma (\tau _{\gamma }))=E_{\mu }(\alpha (\tau _{\gamma })\star \beta (\tau _{\gamma }))\ge \inf _{\mathcal {P}_{a,\mu }\cap S^{r}_{a}}E_{\mu }=m_{r}(a,\mu ). \end{aligned}$$

Consequently, \(\sigma (a,\mu )\ge m_{r}(a,\mu )\). On the other hand, if \(u\in \mathcal {P}_{a,\mu } \cap S^{r}_{a}\), then \(\gamma _{u}\) defined in (7.3) ia s path in \(\Gamma \) with

$$\begin{aligned} E_{\mu }(u)=\max _{\gamma _{u}([0,1])}\widetilde{E}_{\mu }\ge \sigma (a,\mu ), \end{aligned}$$

which implies that

$$\begin{aligned} m_{r}(a,\mu )\ge \sigma (a,\mu ). \end{aligned}$$

Combining this with Lemmas 4.34.4, we have that

$$\begin{aligned} \sigma (a,\mu )=m_{r}(a,\mu )>\sup _{(\overline{A_{k}}\cup E^{0})\cap S^{r}_{a}}E_{\mu }=\sup _{((0,\overline{A_{k}})\cup (0,E^{0}))\cap (\mathbb {R}\times S^{r}_{a})}\widetilde{E}_{\mu }. \end{aligned}$$

By Theorem 7.1, we know that \(\{\gamma ([0,1]): \gamma \in \Gamma \}\) is a homotopy stable family of compact subsets of \(\mathbb {R}\times S^{r}_{a}\) with closed boundary \((0, \overline{A}_{k})\cup (0,E^{0})\) and the superlevel set \(\{\widetilde{E}\ge \sigma (a,\mu )\}\) is a dual set for \(\Gamma \). By Theorem 7.1 we can taking any minimizing sequence \(\{\gamma _{n}=(\alpha _{n},\beta _{n})\}\subset \Gamma _{n}\) for \(\sigma (a,\mu )\) with the property that \(\alpha _{n}=0\) and \(\beta _{n}(\tau )\ge 0\) a.e in \(\mathbb {R}^{N}\), there exists a Palais–Smale sequence \(\{(t_{n},w_{n})\}\subset \mathbb {R}\times S^{r}_{a}\) for \(\widetilde{E}|_{\mathbb {R}\times S^{r}_{a}}\) at level \(\sigma (a,\mu )\), that is

$$\begin{aligned} \partial _{t}\widetilde{E}_{\mu } (t_{n},w_{n}) \rightarrow 0\ \text {and}\ \ \Vert \partial _{u}\widetilde{E}_{\mu } (t_{n},w_{n})\Vert \rightarrow 0 \ \text {as}\ \ n\rightarrow +\infty . \end{aligned}$$
(7.4)

with the additional property that

$$\begin{aligned} |t_{n}|+\mathrm{dist}_{H^{s}} (w_{n},\beta _{n}([0,1]))\rightarrow 0 \ \text {as}\ \ n\rightarrow +\infty . \end{aligned}$$
(7.5)

By the definition of \(\widetilde{E}_{\mu } (t_{n},w_{n})\) in (7.1), from (7.4) we know that \(P (t_{n},w_{n}) \rightarrow 0\), that is

$$\begin{aligned} dE_{\mu } (t_{n}\star w_{n})[t_{n}\star \varphi ]=o(1) \Vert \varphi \Vert =o(1) \Vert t_{n}\star \varphi \Vert \ \text {as}\ \ n\rightarrow +\infty \ \text {for every} \ \varphi \in T_{w_{n}}S^{r}_{a}. \end{aligned}$$
(7.6)

Let \(u_{n}=t_{n}\star w_{n}\), by (7.6), we know that \(\{u_{n}\}\) is a Palais–Smale sequence for \(E_{\mu }|_{S^{r}_{a}}\) at the level \(\sigma (a,\mu )=m_{r}(a,\mu )\) and \(P(u_{n})\rightarrow 0\). Thus, by Lemmas 4.34.5, we obtain that \(m_{r}(a,\mu )\in (0, \frac{s}{N}S^{\frac{N}{2s}}_{s})\), so by Proposition 2.2, one of the alternatives occurs. Assume (i) occurs in Proposition 2.2, then up to a subsequence \(u_{n}\rightharpoonup \widetilde{u}\) weakly in \(H^{s}(\mathbb {R}^{N})\) but not strongly, where \(\widetilde{u}\not \equiv 0\) is a solution of (1.1) for some \(\lambda <0\), and

$$\begin{aligned} E_{\mu }(\widetilde{u})\le m(a,\mu )- \frac{s}{N}S^{\frac{N}{2s}}_{s}<0. \end{aligned}$$

Hence by the Pohozaev identity, \(P(\widetilde{u})=0\) holds, which implies that

$$\begin{aligned} ||\widetilde{u}||^{2}_{D_{s}(\mathbb {R}^{N})} -\frac{2\mu }{\overline{p}}\int _{\mathbb {R}^{N}}|\widetilde{u}|^{\overline{p}}dx-\int _{\mathbb {R}^{N}}|\widetilde{u}|^{2_{s}^{*}}dx=0. \end{aligned}$$

Thus

$$\begin{aligned} E_{\mu }(u)=\frac{1}{2}||\widetilde{u}||^{2}_{D_{s}(\mathbb {R}^{N})} -\frac{2\mu }{\overline{p}}\int _{\mathbb {R}^{N}}|\widetilde{u}|^{\overline{p}}dx-\frac{1}{2_{s}^{*}}\int _{\mathbb {R}^{N}}|\widetilde{u}|^{2_{s}^{*}}dx=\frac{s}{N}\int _{\mathbb {R}^{N}}|\widetilde{u}|^{2_{s}^{*}}dx>0, \end{aligned}$$

which contradicts the fact that

$$\begin{aligned} E_{\mu }(\widetilde{u})\le m(a,\mu )- \frac{s}{N}S^{\frac{N}{2s}}_{s}<0. \end{aligned}$$

Therefore, the alternative (ii) in Proposition 2.2 holds. There exists a subsequence \(u_{n}\rightarrow \widetilde{u}\) strongly in \(H^{s}(\mathbb {R}^{N}),\) \(E_{\mu }(\widetilde{u})=m(a,\mu )\) and \(\widetilde{u}\) solves (1.1)–(1.2) for some \(\lambda <0.\) By \(\beta _{n}(\tau )\ge 0\) a.e in \(\mathbb {R}^{N}\), (7.5) and the convergence implies that \(\widetilde{u}\ge 0\), by the strong maximum principle for the fractional Laplacian, see Proposition 2.17 in [34], we have \(\overline{u}\) is positive. Finally, we prove that \(\widetilde{u}\) is a ground state solution. Since any normalized solutions in \(\mathcal {P}_{a,\mu }\) and

$$\begin{aligned} E_{\mu }(\overline{u})=m_{r}(a,\mu )=\inf _{\mathcal {P}_{a,\mu }\cap S_{a}}E_{\mu }. \end{aligned}$$

It is sufficient to show that

$$\begin{aligned} \inf _{\mathcal {P}_{a,\mu }\cap S_{a}}E_{\mu }=\inf _{\mathcal {P}_{a,\mu }}E_{\mu }=m(a,\mu ). \end{aligned}$$

Assume by contradiction that there is a \(u\in \mathcal {P}_{a,\mu }\setminus S^{r}_{a}\) such that \(E_{\mu }(u)<\inf _{\mathcal {P}_{a,\mu }\cap S_{a}}E_{\mu }\) and there exists a minimizer u, let \(v=|u|^{*}\) the symmetric decreasing rearrangement of u. Then by the properties of symmetric decreasing rearrangement, we have

$$\begin{aligned} ||v||^{2}_{D_{s}(\mathbb {R}^{N})}\le ||u||^{2}_{D_{s}(\mathbb {R}^{N})},\ E_{\mu }(v)\le E_{\mu }(u)\ \text {and}\ P_{\mu }(v)\le 0=P_{\mu }(u). \end{aligned}$$

If \(P_{\mu }(v)=0,\) then \(P_{\mu }(v)=P_{\mu }(v)=0,\) which is a contradiction with above inequalities. If \(P_{\mu }(v)<0,\) then by Lemma 4.2, we know that \(t_{v,\mu }<0\), thus

$$\begin{aligned} E_{\mu }(u)\le & {} E_{\mu }(t_{v,\mu }\star u)=\frac{se^{2_{s}^{\star }st_{v,\mu }}}{N}\int _{\mathbb {R}^{N}}|v|^{2_{s}^{*}}dx= \frac{se^{2_{s}^{\star }st_{v,\mu }}}{N}\int _{\mathbb {R}^{N}}|v|^{2_{s}^{*}}dx\\= & {} e^{2_{s}^{\star }st_{v,\mu }}E_{\mu }(u)<E_{\mu }(u), \end{aligned}$$

which is a contraction. Thus

$$\begin{aligned} m(a,\mu )=m_{r}(a,\mu ) \end{aligned}$$

and hence \(\widetilde{u}\) is a ground state solution.

Case 2 \(L^{2}\)-supercritical perturbation, i.e., \(2+4s/N<q<2_{s}^{*}\). Proceeding exactly as in the case \(q=\overline{p}\), we obtain a Palais–Smale sequence \(\{u_{n}\}\subset S^{r}_{a}\) for \(E_{\mu }|_{S_{a}}\) at the level \(\sigma (a,\mu )=m_{r}(a,\mu )\) and \(P(u_{n})\rightarrow 0\). Thus, by Lemma 5.5, we obtain that \(m_{r}(a,\mu )\in (0, \frac{s}{N}S^{\frac{N}{2s}}_{s})\), so by Proposition 2.2, one of the alternatives occurs. Assume (i) occurs in Proposition 2.2, then up to a subsequence \(u_{n}\rightharpoonup \widetilde{u}\) weakly in \(H^{s}(\mathbb {R}^{N})\) but not strongly, where \(\widetilde{u}\not \equiv 0\) is a solution of (1.1) for some \(\lambda <0\), and

$$\begin{aligned} E_{\mu }(\widetilde{u})\le m(a,\mu )- \frac{s}{N}S^{\frac{N}{2s}}_{s}<0, \end{aligned}$$

hence by the Pohozaev identity \(P(\widetilde{u})=0\) holds, which implies that

$$\begin{aligned} ||\widetilde{u}||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |\widetilde{u}|^{q}dx-\int _{\mathbb {R}^{N}} |\widetilde{u}|^{2_{s}^{*}}dx=0,\end{aligned}$$

thus, by \(q\gamma _{q,s}>2\), we have

$$\begin{aligned} E_{\mu }(u)&=\frac{1}{2}||\widetilde{u}||^{2}_{D_{s}(\mathbb {R}^{N})} -\frac{\mu }{q}\int _{\mathbb {R}^{N}}|\widetilde{u}|^{q}dx-\frac{1}{2_{s}^{*}}\int _{\mathbb {R}^{N}}|\widetilde{u}|^{2_{s}^{*}}dx\\&=\frac{\mu }{q}\left( \frac{q\gamma _{q,s}}{2}-1\right) \int _{\mathbb {R}^{N}}|\widetilde{u}|^{q}dx+\frac{s}{N}\int _{\mathbb {R}^{N}}|\widetilde{u}|^{2_{s}^{*}}dx>0, \end{aligned}$$

which contradicts the fact that

$$\begin{aligned} E_{\mu }(\widetilde{u})\le m(a,\mu )- \frac{s}{N}S^{\frac{N}{2s}}_{s}<0. \end{aligned}$$

Therefore, the alternative (ii) in Proposition 2.2 holds. There exists a subsequence \(u_{n}\rightarrow \widetilde{u}\) strongly in \(H^{s}(\mathbb {R}^{N}),\) \(E_{\mu }(\widetilde{u})=m(a,\mu )\) and \(\widetilde{u}\) solves (1.1)–(1.2) for some \(\lambda <0.\) By \(\beta _{n}(\tau )\ge 0\) a.e in \(\mathbb {R}^{N}\), (7.5) and the convergence implies that \(\widetilde{u}\ge 0\), by the strong maximum principle for fractional Laplacian (see Proposition 2.17 in [34]), we have \(\overline{u}\) is positive. The next arguments are the same as case 1. This completes the proof.

8 Proof of Theorem 1.4

Proof of Theorem 1.4

If we focus on the case \(\mu =0\), then

$$\begin{aligned} E_{0}(u)&=\frac{1}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})} -\frac{1}{2_{s}^{*}}\int _{\mathbb {R}^{N}}|u|^{2_{s}^{*}}dx \end{aligned}$$

on \(S_{a}\). The associated Pohozaev manifold is

$$\begin{aligned} \mathcal {P}_{a,0}=\bigg \{u\in S_{a}:s||u||^{2}_{D_{s}(\mathbb {R}^{N})}=s\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\bigg \}=\bigg \{u\in S_{a}:(\Psi ^{0}_{u})'(0) =0\bigg \},\end{aligned}$$

where

$$\begin{aligned} \Psi ^{0}_{u}(t)=\frac{e^{2st}}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{e^{2_{s}^{*}st}}{2_{s}^{*}}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx. \end{aligned}$$

Recall the decomposition

$$\begin{aligned} \mathcal {P}_{a,0}=\mathcal {P}^{+}_{a,0}\cup \mathcal {P}^{0}_{a,0}\cup \mathcal {P}^{-}_{a,0}. \end{aligned}$$

Since

$$\begin{aligned} \Psi ^{0}_{u}(t)=\frac{e^{2st}}{2}||u||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{e^{2_{s}^{*}st}}{2_{s}^{*}}\int _{\mathbb {R}^{N}} u^{2_{s}^{*}}dx. \end{aligned}$$

It is easy to see that for every \(u\in S_{a}\), the function \(\Psi ^{0}_{u}(t)\) has a unique critical point \(t_{u,0}\), which is a strict maximum point and is given by

$$\begin{aligned} e^{st_{u,0}}=\left( \frac{||u||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx}\right) ^{\frac{1}{2_{s}^{*}-2}}. \end{aligned}$$
(8.1)

By the definition of \(\mathcal {P}^{+}_{a,0}\), we know that \(\mathcal {P}^{+}_{a,0}=\emptyset \). If \(u\in \mathcal {P}^{0}_{a,0}\), then \(u\in \mathcal {P}_{a,0} \) and \( (\Psi ^{\mu }_{u})''(0) =0\), which implies that

$$\begin{aligned} 2||u||^{2}_{D_{s}(\mathbb {R}^{N})}=2_{s}^{*}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx=2_{s}^{*}||u||^{2}_{D_{s}(\mathbb {R}^{N})}\Rightarrow ||u||_{D_{s}(\mathbb {R}^{N})}=0,\end{aligned}$$

which is not possible since \(u\in S_{a}\). Then \(\mathcal {P}_{a,0}=\mathcal {P}^{-}_{a,0}\).

Next, we show that \(\mathcal {P}_{a,0}\) is a smooth manifold of codimension 1 on \(S_{a}\). Since

$$\begin{aligned} \mathcal {P}_{a,0}=\bigg \{u\in S_{a}:||u||^{2}_{D_{s}(\mathbb {R}^{N})}=\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\bigg \}, \end{aligned}$$

we know that \(\mathcal {P}_{a,0}\) is defined by \(P_{0}(u)=0\), \(G(u)=0\), where

$$\begin{aligned} P_{0}(u)=s||u||^{2}_{D_{s}(\mathbb {R}^{N})}-s\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx \ \ \text {and}\ \ G(u)=\int _{\mathbb {R}^{N}} |u|^{2}dx=a^{2} . \end{aligned}$$

Since \(P_{0}(u)\) and G(u) are class of \(C^{1}\), we only need to check that \(d(P_{0}(u),G(u))\): \(H^{s}(\mathbb {R}^{N})\rightarrow \mathbb {R}^{2}\) is surjective. If this not true, \(dP_{0}(u)\) has to be linearly dependent from dG(u) i.e. there exist a \(\nu \in \mathbb {R}\) such that

$$\begin{aligned}&2s\int _{\mathbb {R}^{N}} (-\Delta )^{\frac{s}{2}}u(-\Delta )^{\frac{s}{2}}\varphi dx-s2_{s}^{*}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}-2}u\varphi dx\\&=\nu \int _{\mathbb {R}^{N}} u\varphi dx\ \ \text {for every } \ \ \varphi \in H^{s}(\mathbb {R}^{N}), \end{aligned}$$

which implies that

$$\begin{aligned} 2s(-\Delta )^{2}u=\nu u+2_{s}^{*}s u^{2_{s}^{*}-1}\ \ \text {in}\ \mathbb {R}^{N}. \end{aligned}$$

By the Pohozaev identity for above equation, we know that

$$\begin{aligned} 2s||u||^{2}_{D_{s}(\mathbb {R}^{N})}=2_{s}^{*}s\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx, \end{aligned}$$

that is, \(u\in \mathcal {P}^{+}_{a,0}\), a contradiction. Hence \(\mathcal {P}_{a,0}\) is a natural constraint.

Indeed, if \(u\in \mathcal {P}_{a,0}\) is a critical point of \(E_{0}|_{\mathcal {P}_{a,0}}\), then u is a critical point of \(E_{0}|_{S_{a}}\). Thus, for every \(u\in S_{a}\) there exist a unique \(t_{u,0}\in \mathbb {R}\) such that \(t_{u,0}\star u \in \mathcal {P}_{a,0} \) and \(t_{u,0}\) is a strict maximum point of \(\Psi ^{0}_{u}(t)\), if \(u\in \mathcal {P}_{a,0}\), we have that \(t_{u,0}=0\) and

$$\begin{aligned} E_{0}(u)=\max _{t\in \mathbb {R}}E_{0}(t\star u)\ge \inf _{v\in S_{a}} \max _{t\in \mathbb {R}}E_{0}(t\star u). \end{aligned}$$

On the other hand, if \(u\in S_{a},\) then \(t_{u,0}\star u \in \mathcal {P}_{a,0} \), so

$$\begin{aligned} \max _{t\in \mathbb {R}}E_{0}(t\star u)=E_{0}(t_{u,0}\star u)\ge \inf _{v\in \mathcal {P}_{a,0}} E_{0}(v). \end{aligned}$$

Thus

$$\begin{aligned} \inf _{u\in \mathcal {P}_{a,0}} E_{0}(u)=\inf _{u\in S_{a}} \max _{t\in \mathbb {R}}E_{0}(t\star u). \end{aligned}$$

Now, by (8.1), we have

$$\begin{aligned} \inf _{u\in \mathcal {P}_{a,0}} E_{0}(u)&=\inf _{u\in S_{a}} \max _{t\in \mathbb {R}}E_{0}(t\star u)\\&=\inf _{u\in S_{a}} \left[ \frac{1}{2}\left( \frac{||u||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx}\right) ^{\frac{2}{2_{s}^{*}-2}}||u||^{2}_{D_{s}(\mathbb {R}^{N})}\right. \\&\quad \left. -\frac{1}{2_{s}^{*}}\left( \frac{||u||^{2}_{D_{s}(\mathbb {R}^{N})}}{\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx}\right) ^{\frac{2_{s}^{*}}{2_{s}^{*}-2}}\int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\right] \\&=\inf _{u\in S_{a}} \frac{s}{N}\left( \frac{||u||^{2}_{D_{s}(\mathbb {R}^{N})}}{\left( \int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\right) ^{\frac{2}{2_{s}^{*}}}}\right) ^{\frac{2_{s}^{*}}{2_{s}^{*}-2}}\\&=\inf _{u\in H^{s}(\mathbb {R}^{N})\setminus \{0\}} \frac{s}{N}\left( \frac{||u||^{2}_{D_{s}(\mathbb {R}^{N})}}{\left( \int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\right) ^{\frac{2}{2_{s}^{*}}}}\right) ^{\frac{N}{2s}}. \end{aligned}$$

So it follows that

$$\begin{aligned} \inf _{u\in H^{s}(\mathbb {R}^{N})\setminus \{0\}} \frac{s}{N}\left( \frac{||u||^{2}_{D_{s}(\mathbb {R}^{N})}}{\left( \int _{\mathbb {R}^{N}} |u|^{2_{s}^{*}}dx\right) ^{\frac{2}{2_{s}^{*}}}}\right) ^{\frac{N}{2s}}=\frac{s}{N}S^{\frac{N}{2s}}_{s} \end{aligned}$$

and the infimum is achieved if and only if by the extremal functions \(U_{\epsilon ,y}\) defined in (1.6) when \(N>4s\) and stay in \(L^{2}(\mathbb {R}^{N})\). If \(2s<N\le 4s\), we show that the infimum of \(E_{0}\) on \(\mathcal {P}_{a,0}\) is not achieved. Assume by contradiction that there exists a minimizer u, let \(v=|u|^{*}\) the symmetric decreasing rearrangement of u. Then by the properties of symmetric decreasing rearrangement, we have

$$\begin{aligned} ||v||^{2}_{D_{s}(\mathbb {R}^{N})}\le ||u||^{2}_{D_{s}(\mathbb {R}^{N})},\ E_{0}(v)\le E_{0}(u)\ \text {and}\ P_{0}(v)\le 0=P_{0}(u). \end{aligned}$$

If \(P_{0}(v)<0,\) then by (8.1), we know that \(t_{v,0}<0\), thus

$$\begin{aligned}&E_{0}(u)\le E_{0}(t_{v,0}\star u)=\frac{se^{2st_{v,0}}}{N}||v||^{2}_{D_{s}(\mathbb {R}^{N})}\\&\le \frac{se^{2st_{v,0}}}{N}||u||^{2}_{D_{s}(\mathbb {R}^{N})}=e^{2st_{v,0}}E_{0}(u)<E_{0}(u), \end{aligned}$$

which is a contradiction. Thus \(P_{0}(v)=0\Rightarrow v\in \mathcal {P}_{a,0} \). Since \(\mathcal {P}_{a,0}\) is a natural constraint, we obtain

$$\begin{aligned} (-\Delta )^{s}v=\lambda v +v^{2_{s}^{*}-1}, \ v\ge 0\ \text {in}\ \mathbb {R}^{N}, \end{aligned}$$
(8.2)

for some \(\lambda \in \mathbb {R}\). Since \(P_{0}(v)=0\), which implies that \(\lambda =0\). By the strong maximum principle, we have \(v>0\) in \(\mathbb {R}^{N}\). From [27], we know that \(v=\alpha U_{\epsilon ,0}\) for some \(\alpha ,\epsilon >0\), this is not possible, since \(U_{\epsilon ,0}\notin H^{s}(\mathbb {R}^{N})\) for \(2s<N\le 4s.\) The proof is thus complete.

9 Proof of Theorem 1.5

In this section, we prove Theorem 1.5. Before the proof, we give some lemmas.

Lemma 9.1

Let \(a>0\) , \(\mu \ge 0,\ \overline{p}\le q< 2_{s}^{*}\) and (1.9) holds. Then

$$\begin{aligned} \inf _{u\in \mathcal {P}_{a,\mu } }E_{\mu }(u)=\inf _{u\in S_{a} }\max _{t\in \mathbb {R}}E_{\mu }(t\star u). \end{aligned}$$

Proof

Since \(\overline{p}\le q< 2_{s}^{*}\) and \(\mu \ge 0\), by Lemmas 4.2 and 5.2, we know that \(\mathcal {P}_{a,\mu }=\mathcal {P}^{-}_{a,\mu }\), for every \(u\in S_{a}\), there is a unique \(t_{u,\mu }\in \mathbb {R}\) such that \(t_{u,\mu }\star u \in \mathcal {P}_{a,\mu },\) where \(t_{u,\mu }\) is the unique critical point of the function of \(\Psi ^{\mu }_{u}\) (see Proposition 1.4 for \(\mu =0\)). So, if \(u\in \mathcal {P}_{a,\mu },\) we have that \(t_{u,\mu }=0\) and

$$\begin{aligned} E_{\mu }(u)=\max _{t\in \mathbb {R}}E_{\mu }(t\star u)\ge \inf _{v\in S_{a}}\max _{t\in \mathbb {R}}E_{\mu }(t\star v) . \end{aligned}$$

On the other hand, if \(u\in S_{a},\) then \(t\star u \in \mathcal {P}_{a,\mu }\) and hence

$$\begin{aligned} \max _{t\in \mathbb {R}}E_{\mu }(t\star u)=E_{\mu }(t_{u,\mu }\star u)\ge \inf _{v\in \mathcal {P}_{a,\mu }}E_{\mu }( v). \end{aligned}$$

This ends the proof.\(\square \)

Lemma 9.2

Let \(a>0\), \(\overline{p}\le q< 2_{s}^{*}\), \(\widetilde{\mu }\ge 0\) satisfy (1.9) holds. Then the function \(\mu \in [0,\widetilde{\mu }]\rightarrow m(a,\mu )\in \mathbb {R}\) is monotone non-increasing.

Proof

Let \(0\le \mu _{1}\le \mu _{2}\le \widetilde{\mu }\), by Lemma 9.1, we know that

$$\begin{aligned} m(a,\mu _{2})&=\inf _{u\in S_{a} }\max _{t\in \mathbb {R}}E_{\mu _{2}}(t\star u)=\inf _{u\in S_{a} }E_{\mu _{2}}(t_{u,\mu _{2}}\star u)\\&=\inf _{u\in S_{a} }\left[ E_{\mu _{1}}(t_{u,\mu _{2}}\star u)+(\mu _{1}-\mu _{2})\frac{e^{q\gamma _{q,s}st}}{q}\int _{\mathbb {R}^{N}} |u|^{q}dx\right] \\&\le \inf _{u\in S_{a} }\max _{t\in \mathbb {R}}E_{\mu _{1}}(t\star u)=m(a,\mu _{1}). \end{aligned}$$

As desired.\(\square \)

Proof of Theorem 1.5

We divide the proof into two cases.

Case 1 \(2<q<\overline{p}\). Since \(u_{\mu }\) is a positive ground state solution of \(E_{\mu }(u)\) on \(\{u\in S_{a}: ||u_{\mu }||^{2}_{D_{s}(\mathbb {R}^{N})}< R_{0}\}\), where \(R_{0}(a,\mu )\) is defined by Lemma 3.1, since \(R_{0}\) is defined by \(h(R_{0})=0\), see h in (3.2), we can check that \(R_{0}=R_{0}(a,\mu )\rightarrow 0\) as \(\mu \rightarrow 0^{+}\), thus \(||u_{\mu }||^{2}_{D_{s}(\mathbb {R}^{N})}< R_{0}\rightarrow 0\) as \(\mu \rightarrow 0^{+}\). Since for every \(u\in S_{a}\), by fractional Gagliardo–Nirenberg–Sobolev inequality (2.4) and Sobolev inequality (1.5)

$$\begin{aligned} 0>m(a,\mu )=&E_{\mu }(u_{\mu })\ge \frac{1}{2}||u_{\mu }||^{2}_{D_{s}(\mathbb {R}^{N})} -\frac{\mu }{q}C^{q}_{N,q,s}||u_{\mu }||^{q\gamma _{q,s}}_{D_{s}(\mathbb {R}^{N})}a^{q (1-\gamma _{p,s})}\\&-\frac{1}{2_{s}^{*}}S^{-\frac{2_{s}^{*}}{2}}_{s}||u_{\mu }||^{2_{s}^{*}}_{D_{s}(\mathbb {R}^{N})}\rightarrow 0 \end{aligned}$$

as \(\mu \rightarrow 0^{+}\).

Case 2 \(\overline{p}\le q<2_{s}^{*}\). Let \(\widetilde{\mu }\ge 0\) and (1.9) holds. Firstly, we show that the family of positive radial ground states \(\{u_{\mu }: 0<\mu <\widetilde{\mu }\}\) is a bounded in \(H^{s}(\mathbb {R}^{N})\). If \(q=\overline{p}=2+4s/N\), then by Lemma 9.2 and \(P_{\mu }(u_{\mu })=0\), we have

$$\begin{aligned} m(a,0)\ge m(a,\mu )=E_{\mu }(u_{\mu })&=\frac{s}{N}\left( ||u_{\mu }||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{2\mu }{\overline{p}}\int _{\mathbb {R}^{N}} |u|^{\overline{p}}dx\right) \\&\ge \frac{s}{N}\left( 1-\frac{2\mu }{\overline{p}}C^{\overline{p}}_{N,\overline{p},s}a^{\frac{4s}{N}}\right) ||u||^{2}_{D_{s}(\mathbb {R}^{N})}. \end{aligned}$$

If \(\overline{p}<q<2_{s}^{*}\), by the similar arguments as above, we have

$$\begin{aligned} m(a,0)\ge m(a,\mu )=E_{\mu }(u_{\mu })=\frac{s}{N}\int _{\mathbb {R}^{N}} |u_{\mu }|^{2_{s}^{*}}dx+\frac{\mu }{q}\left( \frac{q\gamma _{q,s}}{2}-1\right) \int _{\mathbb {R}^{N}} |u_{\mu }|^{q}dx. \end{aligned}$$

Thus, \(\{u_{\mu }\}\) is bounded in \(L^{q}(\mathbb {R}^{N})\cap L^{2_{s}^{*}}(\mathbb {R}^{N}) \). From \(P_{\mu }(u_{\mu })=0,\) we also have \(\{u_{\mu }\}\) is bounded in \(H^{s}(\mathbb {R}^{N}).\) Since

$$\begin{aligned} \widetilde{\lambda }_{\mu }a^{2}= & {} ||u_{\mu }||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \int _{\mathbb {R}^{N}} |u_{\mu }|^{q}dx-\int _{\mathbb {R}^{N}} |u_{\mu }|^{2_{s}^{*}}dx\\= & {} \mu (\gamma _{q,s}-1)\int _{\mathbb {R}^{N}} |u_{\mu }|^{q}dx\rightarrow 0 \end{aligned}$$

as \(\mu \rightarrow 0^{+}\). Therefore \(u_{\mu }\rightharpoonup u\) weakly in \(H^{s}(\mathbb {R}^{N}),\ D_{s}(\mathbb {R}^{N}),\ L^{2_{s}^{*}}(\mathbb {R}^{N})\) and \(u_{\mu }\rightharpoonup u\) strongly in \(L^{q}(\mathbb {R}^{N})\), \(\widetilde{\lambda }_{\mu }\rightarrow 0\). Let \(||u_{\mu }||^{2}_{D_{s}(\mathbb {R}^{N})}\rightarrow \ell \ge 0\), if \(\ell =0\), then \(u_{\mu }\rightarrow 0\) strongly in \(D_{s}(\mathbb {R}^{N})\), so \(E_{\mu }(u_{\mu })\rightarrow 0.\) However, by Lemma 9.2, we know that \(E_{\mu }(u_{\mu })\ge m(a,\widetilde{\mu })>0\) for every \(0<\mu <\widetilde{\mu }\), a contradiction. Thus \(\ell >0\). Since \(P_{\mu }(u_{\mu })=0\), we have

$$\begin{aligned} \int _{\mathbb {R}^{N}} |u_{\mu }|^{2_{s}^{*}}dx=||u_{\mu }||^{2}_{D_{s}(\mathbb {R}^{N})}-\mu \gamma _{q,s}\int _{\mathbb {R}^{N}} |u_{\mu }|^{q}dx\rightarrow \ell ,\ \ \text {as}\ \mu \rightarrow 0^{+}. \end{aligned}$$

Therefore, by the Sobolev embedding \(\ell \ge S_{s}\ell ^{\frac{2}{2_{s}^{*}}}\), which implies that \(\ell \ge S^{\frac{N}{2s}}_{s}\). On the other hand, we have

$$\begin{aligned} \frac{\ell }{N}= & {} \lim _{\mu \rightarrow 0^{+}}\left[ \frac{s}{N}||u_{\mu }||^{2}_{D_{s}(\mathbb {R}^{N})}-\frac{\mu }{q}\left( 1-\frac{q\gamma _{q,s}}{2_{s}^{*}}\right) \int _{\mathbb {R}^{N}} |u_{\mu }|^{q}dx\right] \\= & {} \lim _{\mu \rightarrow 0^{+}}E_{\mu }(u)\le m(a,0)=\frac{s}{N}S^{\frac{N}{2s}}_{s}. \end{aligned}$$

Thus, \(\ell =S^{\frac{N}{2s}}_{s}\) and the desired conclusion follows.