1 Introduction

The purpose of this paper is to consider the following biharmonic equation:

$$\begin{aligned} \left\{ \begin{array}{ll} \Delta ^2u+V_\lambda (x)u=\mu f(x)u^{-\gamma }+g(x)u^{p-1},&{}\quad \textrm{in}~{\mathbb {R}}^N,\\ u>0,&{}\quad \textrm{in}~{\mathbb {R}}^N, \end{array}\right. \end{aligned}$$
(1.1)

where \(\Delta ^2:=\Delta (\Delta )\) is the biharmonic operator with \(N\ge 1\), and \(0<\gamma <1\), \(2<p<2^{**}(2^{**}=\frac{2N}{N-4})\). \(\lambda ,~\mu >0\) are parameters and the potential \(V_\lambda (x)=\lambda a(x)-b(x)\). We assume that a(x) and b(x) satisfy the following conditions:

\((V_1)\) \(a\in C({\mathbb {R}}^N)\) and \(a(x)\ge 0\) for all \(x\in {\mathbb {R}}^N\) and there exists \(a_0>0\) such that the set

$$\begin{aligned} \{a<a_0\}:=\{x\in {\mathbb {R}}^N|a(x)<a_0\} \end{aligned}$$

has finite positive Lebesgue measure for \(N\ge 4\) and

$$\begin{aligned} |\{a<a_0\}|<S_\infty ^{-2}\left( 1+\frac{A_0^2}{2}\right) ^{-1}~~~\textrm{for}~N\le 3, \end{aligned}$$

where \(|\cdot |\) is the Lebesgue measure, \(S_\infty \) is the best Sobolev constant for the embedding of \(H^2({\mathbb {R}}^N)\) in \(L^\infty ({\mathbb {R}}^N)\) with \(N\le 3\), and \(A_0\) is defined in Lemma 2.1;

\((V_2)\) \(\Omega =\textrm{int}\{x\in {\mathbb {R}}^N:a(x)=0\}\) is nonempty and has a smooth boundary with \({\bar{\Omega }}=\{x\in {\mathbb {R}}^N:a(x)=0\}\);

\((V_3)\) b(x) is a measurable function on \({\mathbb {R}}^N\) and there exists \(0<b_0<{\bar{\gamma }}\) such that \(0\le b(x)\le \frac{b_0}{|x|^4}\) for all \(x\in {\mathbb {R}}^N\), where \({\bar{\gamma }}:=\frac{N^2(N-4)^2}{16}\) is a critical Hardy-Sobolev constant.

The potential \(V_\lambda \) satisfies \((V_1),~(V_2)\) is called the steep well potential, which was first introduced by Bartsch and Wang [4] in the study of the nonlinear Schrödinger equations.

When \(\Omega \) is a bounded domain of \({\mathbb {R}}^N\), the researchers mainly focused on the following Navier boundary value problem:

$$\begin{aligned} \left\{ \begin{array}{l} \Delta ^2u+c\Delta u=f(x,u), ~~x\in \Omega , \\ u=\Delta u=0,~~x\in \partial \Omega , \end{array}\right. \end{aligned}$$
(1.2)

which arises in the study of traveling waves in suspension bridges, see [5, 9, 14] and the study of the static deflection of an elastic plate in a fluid. In the last decades, many authors have attached their attention to the existence and multiplicity of nontrivial solutions for biharmonic equations, we refer the readers to [2, 6, 10, 12].

Recently, biharmonic equations on unbounded domain \({\mathbb {R}}^N\) have attracted a lot of attention. Especially, the researchers mainly investigated the following problems with the steep potential:

$$\begin{aligned} \left\{ \begin{array}{l} \Delta ^2u-\Delta u+\lambda V(x)u=f(x,u)~~~\textrm{in}~{\mathbb {R}}^N, \\ u\in H^2({\mathbb {R}}^N). \end{array}\right. \end{aligned}$$
(1.3)

With the aid of \(\lambda \), they proved that the energy functional possesses the property of being locally compact, see [8, 11, 16, 18] and their references therein. Especially, Ye and Tang [18] assumed that f(xu) was superlinear and subcritical at infinity, when \(\lambda \) was large enough, they obtained the existence and multiplicity of nontrivial solutions. Later, Zhang, Tang, Zhang and Luo [19] improved their results and obtained the existence of infinite nontrivial solutions when \(\lambda >0\) was large enough. Badiale, Greco and Rolando [3] obtained two nontrivial solutions for the case \(f(x,u)=g(x,u)+\mu \xi (x)|u|^{p-2}u\) when g(xu), \(\xi (x)\) satisfied some assumptions, \(\lambda \) was large enough and \(\mu \) was small enough. Mao and Zhao [13] considered (1.3) with Kirchhoff terms and concave-convex nonlinearities, existence and multiplicity of solutions were proved using the variational method.

Very recently, replacing Laplacian with p-Laplacian in (1.3), Sun, Chu and Wu [15] studied the following biharmonic equation

$$\begin{aligned} \left\{ \begin{array}{l} \Delta ^2u-\beta \Delta _p u+\lambda V(x)u=f(x,u)~~~\textrm{in}~{\mathbb {R}}^N, \\ u\in H^2({\mathbb {R}}^N), \end{array}\right. \end{aligned}$$

where \(N\ge 1\), \(p\ge 2\) and \(\beta >0\) small enough or \(\beta <0\). Using the mountain pass theorem, and under some suitable assumptions on V(x) and f(xu), they obtained the existence and multiplicity of nontrivial solutions for \(\lambda \) large enough. Later, Jiang and Zhai [7] supplemented their results, when \(\beta \in {\mathbb {R}}\) and \(\lambda V(x)\) was replaced by \(V_\lambda (x)\), which was singular, the multiplicity of nontrivial solutions was obtained.

Motivated by the above papers, in the present paper, we consider a biharmonic problem with steep well potential and singular nonlinearity. To the best of knowledge, few works concerning this case up to now. To this end, we need some assumptions on f(x) and g(x) and make the following hypotheses:

(F) \(f\in L^{\frac{p}{p+\gamma -1}}({\mathbb {R}}^N)\) is a positive continuous function.

(G) \(g\in L^{\infty }({\mathbb {R}}^N)\) is a sign-changing function such that \(|g^+|_\infty >0\), where \(g^+=\max \{g(x),0\}\).

Now, we state our main result.

Theorem 1.1

Let \(0<\gamma <1\) and \(2<p<2^{**}\). Suppose that f,  g and \(V_\lambda \) satisfy (F),  (G) and \((V_1)-(V_3)\), then there exist \(\lambda ^*>0\) and \(\mu ^*>0\) such that problem (1.1) has at least two solutions for all \((\lambda ,\mu )\in [\lambda ^*,+\infty )\times (0,\mu ^*)\).

Remark 1.2

From the condition \((V_3)\), it is easy to obtain that the function b(x) could be singular at the origin. Moreover, the improved Hardy–Sobolev inequality (see Lemma 1.1 in [17]) gives

$$\begin{aligned} \int _{{\mathbb {R}}^N}b(x)u^2dx\le b_0\int _{{\mathbb {R}}^N}\frac{u^2}{|x|^4}dx\le \frac{b_0}{{\bar{\gamma }}}\int _{{\mathbb {R}}^N} |\Delta u|^2dx. \end{aligned}$$

2 Preliminaries

Let

$$\begin{aligned} X=\left\{ u\in H^2({\mathbb {R}}^N)|\int _{{\mathbb {R}}^N}(|\Delta u|^2+a(x)u^2)dx<+\infty \right\} \end{aligned}$$

be equipped with the inner product and norm

$$\begin{aligned} \langle u,v\rangle =\int _{{\mathbb {R}}^N}(\Delta u\Delta v+a(x)uv)dx,~\Vert u\Vert =\langle u,u\rangle ^{(1/2)}. \end{aligned}$$

For \(\lambda >0\), we also need the inner product and norm

$$\begin{aligned} \langle u,v\rangle _\lambda =\int _{{\mathbb {R}}^N}(\Delta u\Delta v+\lambda a(x)uv)dx,~\Vert u\Vert _\lambda =\langle u,u\rangle _\lambda ^{(1/2)}. \end{aligned}$$

It is clear that \(\Vert u\Vert \le \Vert u\Vert _\lambda \) for \(\lambda \ge 1\). For simplicity, we let

$$\begin{aligned} \Vert u\Vert ^2_{\lambda ,V}:=\displaystyle \int _{{\mathbb {R}}^N}\left( |\Delta u|^2dx+V_\lambda u^2\right) dx, \end{aligned}$$

then by Remark 1.2, one has

$$\begin{aligned} \Vert u\Vert _\lambda ^2\ge \Vert u\Vert _{\lambda ,V}^2\ge \frac{\mu _0-1}{\mu _0}\Vert u\Vert _\lambda ^2,~~\lambda >0, \end{aligned}$$
(2.1)

where \(\mu _0=\frac{{\bar{\gamma }}}{b_0}>1\). Hence, \(\Vert u\Vert _{\lambda ,V}\) and \(\Vert u\Vert _\lambda \) are equivalent in \(X_\lambda \), where

$$\begin{aligned} X_{\lambda }=\left\{ u\in H^2({\mathbb {R}}^N)|\int _{{\mathbb {R}}^N}(|\Delta u|^2+\lambda a(x)u^2)dx<+\infty \right\} . \end{aligned}$$

Lemma 2.1

([15]). Under assumptions \((V_1),(V_2)\), the continuous embedding \(X_\lambda \hookrightarrow L^r({\mathbb {R}}^N)\) is compact for \(2\le r<2^{**}\), and there holds \(\displaystyle \int _{{\mathbb {R}}^N}|u|^rdx\le \Theta _r\Vert u\Vert ^r_\lambda \) for \(\lambda \ge \lambda _*\), where

$$\begin{aligned} \Theta _r:=\left\{ \begin{array}{l} S_\infty ^{-(r-2)}\left[ (1+\frac{A_0^2}{2})^{-1}-S_\infty ^2|\{a<a_0\}|\right] ^{-r/2}~~\qquad \textrm{if}~N\le 3,\\ S_r^{-r}\left( 1+\frac{A_0^2}{2}\right) ^{r/2}\qquad \qquad \qquad \quad \qquad \quad \quad \qquad \textrm{if}~N=4,\\ C_0^{N(r-2)/4}\left( 1+\frac{A_0^2}{2}\right) ^{r/2}~~~~~~~~~~~~~~~~~~~~~~~~~~\qquad \qquad \qquad \qquad \quad \quad \textrm{if}~N>4, \end{array}\right. \end{aligned}$$

and

$$\begin{aligned} \lambda _*:=\left\{ \begin{array}{ll} \frac{1}{a_0}&{} \quad \textrm{if}~N\le 3,\\ \frac{2(1+B_0^4|\{a<a_0\}|)}{a_0}&{} \quad \textrm{if}~N=4,\\ \frac{1+C_0^2|\{a<a_0\}|^{N/4}}{a_0}&{} \quad \textrm{if}~N>4, \end{array}\right. \end{aligned}$$

where \(A_0\), \(B_0\), \(C_0\) are positive constants, and \(S_r\) is the best Sobolev constant for the embedding of \(H^2({\mathbb {R}}^N)\) in \(L^r({\mathbb {R}}^N)\) for \(2\le r<2^{**}\).

In this paper, we make use of the following notations: the \(L^r\)-norm (\(1\le r\le +\infty \)) by \(|\cdot |_r\). C denotes various positive constants, which may vary from line to line. By \((V_1),~(V_2)\), the Hölder inequality and the Sobolev inequality, we have

$$\begin{aligned} \int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx\le |f|_{\frac{p}{p+\gamma -1}} \Theta _p^{\frac{1-\gamma }{p}}\Vert u\Vert _\lambda ^{1-\gamma }. \end{aligned}$$
(2.2)

The energy functional corresponding to (1.1) given by

$$\begin{aligned} \left. \begin{array}{l} I_{\lambda ,\mu }(u)=\displaystyle \frac{1}{2}\Vert u\Vert ^2_\lambda -\frac{1}{2}\int _{{\mathbb {R}}^N}b(x)u^2dx -\frac{\mu }{1-\gamma }\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx\\ \quad \qquad \quad - \frac{1}{p}\int _{{\mathbb {R}}^N}g|u|^{p}dx,~~{ for}~u\in X_\lambda . \end{array}\right. \end{aligned}$$
(2.3)

It is clear that \(I_{\lambda ,\mu }\) is a \(C^1\) functional. Since \(I_{\lambda ,\mu }\) is not bounded below on \(X_\lambda \), it is useful to consider the functional on the Nehari manifold

$$\begin{aligned} {\mathcal {N}}_{\lambda ,\mu }=\{u\in X_\lambda \backslash \{0\}: \langle I_{\lambda ,\mu }'(u),u\rangle =0\}. \end{aligned}$$

We analyze \({\mathcal {N}}_{\lambda ,\mu }\) in terms of the stationary points of fibering maps \(N_u:(0,+\infty )\rightarrow {\mathbb {R}}\) given by

$$\begin{aligned} N_u(t)=I_{\lambda ,\mu }(tu),~~t>0. \end{aligned}$$

Then for each \(u\in {\mathcal {N}}_{\lambda ,\mu }\), we have

$$\begin{aligned} N'_u(t)= & {} t\Vert u\Vert _{\lambda ,V}^2-\mu t^{-\gamma }\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx- t^{p-1}\int _{{\mathbb {R}}^N}g|u|^{p}dx, \\ N''_u(t)= & {} \Vert u\Vert _{\lambda ,V}^2+\mu \gamma t^{-\gamma -1}\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx- (p-1)t^{p-2}\int _{{\mathbb {R}}^N}g|u|^{p}dx. \end{aligned}$$

It is easy to see that

$$\begin{aligned} tN'_u(t)=t^2\Vert u\Vert _{\lambda ,V}^2-\mu t^{1-\gamma }\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx- t^{p}\int _{{\mathbb {R}}^N}g|u|^{p}dx, \end{aligned}$$

and for \(u\in X_\lambda \backslash \{0\}\) and \(t>0\), then \(tu\in {\mathcal {N}}_{\lambda ,\mu }\) if and only if \(N'_u(t)=0\), that is, the critical points of \(N_u(t)\) correspond to the points on the Nehari manifold. In particular, \(u\in {\mathcal {N}}_{\lambda ,\mu }\) if and only if \(N'_u(1)=0\). Then we define

$$\begin{aligned} {\mathcal {N}}_{\lambda ,\mu }^+=\{u\in {\mathcal {N}}_{\lambda ,\mu }:N''_u(1)>0\}, \\ {\mathcal {N}}_{\lambda ,\mu }^0=\{u\in {\mathcal {N}}_{\lambda ,\mu }:N''_u(1)=0\}, \\ {\mathcal {N}}_{\lambda ,\mu }^-=\{u\in {\mathcal {N}}_{\lambda ,\mu }:N''_u(1)<0\}. \end{aligned}$$

The existence of solutions to (1.1) can be studied by considering the existence of minimizers to \(I_{\lambda ,\mu }\) on \({\mathcal {N}}_{\lambda ,\mu }\). Furthermore, for each \(u\in {\mathcal {N}}_{\lambda ,\mu }\), we know that

$$\begin{aligned} \left. \begin{array}{rcl} N''_u(1)&{}=&{}\displaystyle \Vert u\Vert ^2_{\lambda ,V}+\mu \gamma \int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx- (p-1)\int _{{\mathbb {R}}^N}g|u|^{p}dx\\ &{}=&{}\displaystyle (1+\gamma )\Vert u\Vert ^2_{\lambda ,V}-(p+\gamma -1)\int _{{\mathbb {R}}^N}g|u|^{p}dx\\ &{}=&{}\displaystyle (2-p)\Vert u\Vert ^2_{\lambda ,V}+\mu (p+\gamma -1)\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx. \end{array}\right. \nonumber \\ \end{aligned}$$
(2.4)

Lemma 2.2

The energy functional \(I_{\lambda ,\mu }\) is coercive and bounded from below on \({\mathcal {N}}_{\lambda ,\mu }\).

Proof

For \(u\in {\mathcal {N}}_{\lambda ,\mu }\), we have

$$\begin{aligned} \Vert u\Vert _{\lambda ,V}^2-\mu \int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx- \int _{{\mathbb {R}}^N}g|u|^{p}dx=0. \end{aligned}$$

Therefore, by (2.1), (2.2), (2.3) and Lemma 2.1,

$$\begin{aligned} \left. \begin{array}{rcl} I_{\lambda ,\mu }(u)&{}=&{}\displaystyle \left( \frac{1}{2}-\frac{1}{p}\right) \Vert u\Vert _{\lambda ,V}^2 -\frac{\mu (p+\gamma -1)}{p(1-\gamma )}\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx\\ &{}\ge &{}\displaystyle \frac{(p-2)(\mu _0-1)}{2p\mu _0}\Vert u\Vert _\lambda ^2 -\frac{\mu (p+\gamma -1)}{p(1-\gamma )}|f|_{\frac{p}{p+\gamma -1}} \Theta _p^{\frac{1-\gamma }{p}}\Vert u\Vert _\lambda ^{1-\gamma }. \end{array}\right. \end{aligned}$$

For \(0<\gamma <1\), thus we get the conclusion.    \(\square \)

Before the following lemma, we define

$$\begin{aligned} \mu ^*=\displaystyle \frac{(\mu _0-1)(p-2)}{\mu _0(p+\gamma -1)|f|_{\frac{p}{p+\gamma -1}} \Theta _p^{\frac{1-\gamma }{p}}}\times \left( \frac{(\mu _0-1)(1+\gamma )}{\mu _0(p+\gamma -1)|g^+|_\infty \Theta _p}\right) ^{\frac{1+\gamma }{p-2}}. \end{aligned}$$

Lemma 2.3

Suppose that \((F),~(G),~(V_1)-(V_3)\) are satisfied. Then the set \({\mathcal {N}}_{\lambda ,\mu }^0\) is empty for \((\lambda ,\mu )\in [\lambda ^*,+\infty )\times (0,\mu ^*)\).

Proof

If \({\mathcal {N}}_{\lambda ,\mu }^0\ne \emptyset \), by (2.4), we have

$$\begin{aligned} \displaystyle (1+\gamma )\Vert u\Vert ^2_{\lambda ,V}-(p+\gamma -1)\int _{{\mathbb {R}}^N}g|u|^{p}dx=0 \end{aligned}$$

and

$$\begin{aligned} (2-p)\Vert u\Vert ^2_{\lambda ,V}+\mu (p+\gamma -1)\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx=0. \end{aligned}$$

By (2.1), (2.2) and Lemma 2.1, we get that

$$\begin{aligned} \frac{\mu _0-1}{\mu _0}\Vert u\Vert _\lambda ^2\le \frac{p+\gamma -1}{1+\gamma } \int _{{\mathbb {R}}^N}g|u|^{p}dx\le \frac{p+\gamma -1}{1+\gamma }|g^+|_\infty \Theta _p\Vert u\Vert _\lambda ^p \end{aligned}$$

and

$$\begin{aligned} \frac{\mu _0-1}{\mu _0}\Vert u\Vert _\lambda ^2\le \frac{\mu (p+\gamma -1)}{p-2} \int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx\le \frac{\mu (p+\gamma -1)}{p-2}|f|_{\frac{p}{p+\gamma -1}} \Theta _p^{\frac{1-\gamma }{p}}\Vert u\Vert _\lambda ^{1-\gamma }. \end{aligned}$$

Then we get

$$\begin{aligned} \Vert u\Vert _\lambda \ge \left( \frac{(\mu _0-1)(1+\gamma )}{\mu _0(p+\gamma -1)|g^+|_\infty \Theta _p}\right) ^{\frac{1}{p-2}} \end{aligned}$$

and

$$\begin{aligned} \Vert u\Vert _\lambda \le \left( \frac{\mu _0\mu (p+\gamma -1)}{(\mu _0-1)(p-2)}|f|_{\frac{p}{p+\gamma -1}} \Theta _p^{\frac{1-\gamma }{p}}\right) ^{\frac{1}{1+\gamma }}. \end{aligned}$$

Hence, we obtain \(\mu \ge \mu ^*\), which is impossible. Thus we get the conclusion.

\(\square \)

Lemma 2.4

Suppose that \((F),~(G),~(V_1)-(V_3)\) are satisfied. Then

(i) if \(\displaystyle \int _{{\mathbb {R}}^N}g|u|^pdx\le 0\), then there is a unique \(0<t^+<t_{\max }\), such that \(t^+u\in {\mathcal {N}}_{\lambda ,\mu }^+\) and

$$\begin{aligned} I_{\lambda ,\mu }(t^+u)=\inf _{t>0}I_{\lambda ,\mu }(tu); \end{aligned}$$

(ii) if \(\displaystyle \int _{{\mathbb {R}}^N}g|u|^pdx>0\), then there are unique \(t^+\) and \(t^-\) with \(t^->t_{\max }>t^+>0\), such that \(t^-u\in {\mathcal {N}}_{\lambda ,\mu }^-\), \(t^+u\in {\mathcal {N}}_{\lambda ,\mu }^+\) and

$$\begin{aligned} I_{\lambda ,\mu }(t^+u)=\inf _{0\le 0\le t_{\max }}I_{\lambda ,\mu }(tu),~~I_{\lambda ,\mu }(t^-u)=\sup _{t\ge t_{\max }}I_{\lambda ,\mu }(tu). \end{aligned}$$

Proof

Fix \(u\in X_\lambda \backslash \{0\}\) with \(\displaystyle \int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx>0\). Note that

$$\begin{aligned} N'_u(t)=t\Vert u\Vert _{\lambda ,V}^2-\mu t^{-\gamma }\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx- t^{p-1}\int _{{\mathbb {R}}^N}g|u|^{p}dx. \end{aligned}$$

For \(t>0\), we define

$$\begin{aligned} H(t):=t^{2-p}\Vert u\Vert _{\lambda ,V}^2-\mu t^{1-\gamma -p}\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx. \end{aligned}$$

Then for \(t>0\) and \(tu\in {\mathcal {N}}_{\lambda ,\mu }\) if and only if t is a solution for \(H(t)=\int _{{\mathbb {R}}^N}g|u|^{p}dx\), and \(H(t)\rightarrow -\infty \) as \(t\rightarrow 0^+\), \(H(t)\rightarrow 0\) as \(t\rightarrow \infty \). Since

$$\begin{aligned} H'(t)=(2-p)t^{1-p}\Vert u\Vert _{\lambda ,V}^2-\mu (1-\gamma -p) t^{-\gamma -p}\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx, \end{aligned}$$

then H(t) possesses a unique maximum point

$$\begin{aligned} t_{\max }=\displaystyle \left( \frac{\mu (1-\gamma -p)\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx }{(2-p)\Vert u\Vert _{\lambda ,V}^2}\right) ^{\frac{1}{\gamma +1}}, \end{aligned}$$

and

$$\begin{aligned} \left. \begin{array}{rcl} H(t_{\max })&{}=&{}\displaystyle \left[ \left( \frac{\mu (1-\gamma -p)}{2-p}\right) ^{\frac{2-p}{\gamma +1}} -\mu \left( \frac{\mu (1-\gamma -p)}{(2-p)}\right) ^{\frac{1-\gamma -p}{\gamma +1}}\right] \frac{(\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx)^{\frac{2-p}{\gamma +1}}}{\Vert u\Vert _{\lambda ,V} ^\frac{2(1-\gamma -p)}{\gamma +1}}\\ &{}\ge &{}\mu ^{\frac{2-p}{\gamma +1}}\Vert u\Vert _{\lambda ,V}^p\frac{\gamma +1}{p-2} \left( \frac{1-\gamma -p}{2-p}\right) ^{\frac{1-\gamma -p}{\gamma +1}}\left( (\frac{\mu _0}{\mu _0-1}) ^{\frac{1-\gamma }{2}}|f|_{\frac{p}{p+\gamma -1}} \Theta _p^{\frac{1-\gamma }{p}}\right) ^{\frac{2-p}{\gamma +1}}. \end{array}\right. \nonumber \\ \end{aligned}$$
(2.5)

Moreover, H(t) is increasing on \((0,t_{\max })\) and decreasing on \((t_{\max },\infty )\).

(i) if \(\displaystyle \int _{{\mathbb {R}}^N}g|u|^pdx\le 0\), then there is a unique \(0<t^+<t_{\max }\), such that

$$\begin{aligned} H(t^+)=\displaystyle \int _{{\mathbb {R}}^N}g|u|^pdx,~~H'(t^+)>0. \end{aligned}$$

Thus, \(t^+u\in {\mathcal {N}}_{\lambda ,\mu }\) and one has

$$\begin{aligned} \left. \begin{array}{rcl} N''_{t^+u}(1)&{}=&{}\displaystyle (2-p)(t^+)^2\Vert u\Vert _{\lambda ,V}^2+\mu (p+\gamma -1)(t^+)^{1-\gamma } \int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx\\ &{}=&{}t^{1+p}H'(t^+)>0. \end{array}\right. \end{aligned}$$

Then \(t^+u\in {\mathcal {N}}_{\lambda ,\mu }^+\). Since for \(0<t<t_{\max }\), one has

$$\begin{aligned} \frac{d}{dt}I_{\lambda ,\mu }(tu)=t\Vert u\Vert _{\lambda ,V}^2-\mu t^{-\gamma }\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx- t^{p-1}\int _{{\mathbb {R}}^N}g|u|^{p}dx=0 \end{aligned}$$

and

$$\begin{aligned} \frac{d^2}{dt^2}I_{\lambda ,\mu }(tu)=(2-p)t^2\Vert u\Vert _{\lambda ,V}^2+\mu (p+\gamma -1)t^{1-\gamma }\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx>0 \end{aligned}$$

for \(t=t^+\). Therefore, \(I_{\lambda ,\mu }(t^+u)=\inf _{t>0}I_{\lambda ,\mu }(tu)\) holds.

(ii) if \(\displaystyle \int _{{\mathbb {R}}^N}g|u|^pdx>0\), by (2.2),(2.5) and \(\mu \in (0,\mu ^*)\), we have

$$\begin{aligned} \left. \begin{array}{rcl} 0&{}<&{}\displaystyle \int _{{\mathbb {R}}^N}g|u|^pdx\le (\frac{\mu _0}{\mu _0-1})^{p/2} |g^+|_\infty \Theta _p^p\Vert u\Vert _{\lambda ,V}^p\\ &{}&{}=\displaystyle (\mu ^*)^{\frac{2-p}{\gamma +1}}\Vert u\Vert _{\lambda ,V}^p\frac{1+\gamma }{p+\gamma -1} \left( \frac{p-2}{p+\gamma -1}\right) ^{\frac{p-2}{1+\gamma }}\left( (\frac{\mu _0}{\mu _0-1}) ^{\frac{1-\gamma }{2}}|f|_{\frac{p}{p+\gamma -1}} \Theta _p^{\frac{1-\gamma }{p}}\right) ^{\frac{2-p}{\gamma +1}}\\ &{}&{}<H(t_{\max }). \end{array}\right. \end{aligned}$$

There are \(t^+\) and \(t^-\) such that \(0<t^+<t_{\max }<t^-\),

$$\begin{aligned} H(t^+)=\int _{{\mathbb {R}}^N}g|u|^pdx=H(t^-) \end{aligned}$$

and

$$\begin{aligned} H'(t^+)>0>H'(t^-). \end{aligned}$$

As in (i), we have \(t^+u\in {\mathcal {N}}^+_{\lambda ,\mu }\), \(t^-u\in {\mathcal {N}}^-_{\lambda ,\mu }\), and \(I_{\lambda ,\mu }(t^-u)\ge I_{\lambda ,\mu }(tu)\ge I_{\lambda ,\mu }(t^+u)\) for each \(t\in [t^+,t^-]\) and \(I_{\lambda ,\mu }(t^+u)=\inf _{0\le 0\le t_{\max }}I_{\lambda ,\mu }(tu)\), \(I_{\lambda ,\mu }(t^-u)=\sup _{t\ge t_{\max }}I_{\lambda ,\mu }(tu).\) Thus we get the conclusion. \(\square \)

We remark that from Lemmas 2.3 and 2.4, one has \({\mathcal {N}}_{\lambda ,\mu }={\mathcal {N}}^+_{\lambda ,\mu }\cup {\mathcal {N}}^-_{\lambda ,\mu }\) for all \((\lambda ,\mu )\in [\lambda ^*,+\infty )\times (0,\mu ^*)\). Since \({\mathcal {N}}^+_{\lambda ,\mu }\) and \({\mathcal {N}}^-_{\lambda ,\mu }\) are non-empty, thus, by Lemma 2.4, we may define

$$\begin{aligned} c^+_{\lambda ,\mu }=\inf _{u\in {\mathcal {N}}^+_{\lambda ,\mu }}I_{\lambda ,\mu }(u),~~ c^-_{\lambda ,\mu }=\inf _{u\in {\mathcal {N}}^-_{\lambda ,\mu }}I_{\lambda ,\mu }(u) \end{aligned}$$

Then we have the following results.

Lemma 2.5

Suppose that the functions f,  g and V satisfy the conditions (F),  (G) and \((V_1)-(V_3)\). Then for \((\lambda ,\mu )\in [\lambda ^*,+\infty )\times (0,\mu ^*)\), there exists a positive constant \(C_0\) such that \(c^+_{\lambda ,\mu }<0<C_0<c^-_{\lambda ,\mu }\).

Proof

(i) Let \(u\in {\mathcal {N}}^+_{\lambda ,\mu }\subset {\mathcal {N}}_{\lambda ,\mu }\), then we have

$$\begin{aligned} (1+\gamma )\Vert u\Vert ^2_{\lambda ,V}-(p+\gamma -1)\int _{{\mathbb {R}}^N}g|u|^pdx>0. \end{aligned}$$

It follows that

$$\begin{aligned} \left. \begin{array}{rcl} I_{\lambda ,\mu }(u)&{}=&{}\displaystyle \frac{1}{2}\Vert u\Vert ^2_\lambda -\frac{1}{2}\int _{{\mathbb {R}}^N}b(x)u^2dx -\frac{\mu }{1-\gamma }\int _{{\mathbb {R}}^N}f|u|^{1-\gamma }dx- \frac{1}{p}\int _{{\mathbb {R}}^N}g|u|^{p}dx\\ &{}=&{}\displaystyle -\frac{1+\gamma }{2(1-\gamma )}\Vert u\Vert _{\lambda ,V}^2 +\frac{p+\gamma -1}{p(1-\gamma )}\int _{{\mathbb {R}}^N}g|u|^pdx\\ &{}<&{}-\frac{(p-2)(1+\gamma )}{2p(1-\gamma )}\Vert u\Vert ^2_{\lambda ,V}<0. \end{array}\right. \end{aligned}$$

Therefore, \(c_{\lambda ,\mu }^+<0\).

(ii) Let \(u\in {\mathcal {N}}^-_{\lambda ,\mu }\), then we have

$$\begin{aligned} (1+\gamma )\Vert u\Vert ^2_{\lambda ,V}-(p+\gamma -1)\int _{{\mathbb {R}}^N}g|u|^pdx<0. \end{aligned}$$

According to (2.1), we get

$$\begin{aligned} \frac{\mu _0-1}{\mu _0}\Vert u\Vert ^2_{\lambda }\le \Vert u\Vert ^2_{\lambda ,V} <\frac{p+\gamma -1}{1+\gamma }\int _{{\mathbb {R}}^N}g|u|^pdx\le \frac{p+\gamma -1}{1+\gamma }|g^+|_{\infty } \Theta _p\Vert u\Vert ^p_{\lambda }. \end{aligned}$$

Therefore, we can show that

$$\begin{aligned} \Vert u\Vert _{\lambda }>\left( \frac{(\mu _0-1)(1+\gamma )}{\mu _0(p+\gamma -1)|g^+|_{\infty }} \Theta _p\right) ^{\frac{1}{p-2}}:=C. \end{aligned}$$

Then, we know

$$\begin{aligned} \left. \begin{array}{rcl} I_{\lambda ,\mu }(u)&{}\ge &{}\displaystyle \frac{(p-2)(\mu _0-1)}{2p\mu _0}\Vert u\Vert ^2_\lambda -\frac{\mu (p-1+\gamma )}{p(1-\gamma )}|f|_{\frac{p}{p-1+\gamma }} \Theta _p^{1-\gamma }\Vert u\Vert ^{1-\gamma }_\lambda \\ &{}>&{}C^{1-\gamma }\left[ \frac{(p-2)(\mu _0-1)}{2p\mu _0}C^{1+\gamma } -\frac{\mu (p-1+\gamma )}{p(1-\gamma )}|f|_{\frac{p}{p-1+\gamma }} \Theta _p^{1-\gamma }\right] :=C_0. \end{array}\right. \end{aligned}$$

Since \((\lambda ,\mu )\in [\lambda _*,+\infty )\times (0,\mu ^*)\), we can verify that \(C_0>0\). Hence \(I_{\lambda ,\mu }(u)>C_0>0\) for all \(u\in {\mathcal {N}}^-_{\lambda ,\mu }\) and the proof is completed. \(\square \)

Lemma 2.6

Suppose that the functions f,  g and V satisfy the conditions (F),  (G) and \((V_1)-(V_3)\). Then \({\mathcal {N}}^-_{\lambda ,\mu }\) is a closed subset in \(X_{\lambda }\) for \((\lambda ,\mu )\in [\lambda _*,+\infty )\times (0,\mu ^*)\).

Proof

In order to prove that \({\mathcal {N}}^-_{\lambda ,\mu }\) is a closed subset in \(X_{\lambda }\), let us consider a sequence \(\{u_n\}\subset {\mathcal {N}}^-_{\lambda ,\mu }\) such that \(u_n\rightarrow u\) in \(X_{\lambda }\). It is obvious that \(\langle I'_{\lambda ,\mu }(u),u\rangle =0\). By the proof of Lemma 2.5, we have

$$\begin{aligned} \Vert u\Vert _{\lambda }=\lim _{n\rightarrow \infty }\Vert u_n\Vert _{\lambda }\ge C>0. \end{aligned}$$

Thus, \(u\in {\mathcal {N}}_{\lambda ,\mu }\). By the definition of \({\mathcal {N}}^-_{\lambda ,\mu }\), it holds

$$\begin{aligned} (1+\gamma )\Vert u_n\Vert ^2_{\lambda ,V}-(p+\gamma -1)\int _{{\mathbb {R}}^N}g|u_n|^pdx<0. \end{aligned}$$

Combining with Lemma 2.1, one has

$$\begin{aligned} (1+\gamma )\Vert u\Vert ^2_{\lambda ,V}-(p+\gamma -1)\int _{{\mathbb {R}}^N}g|u|^pdx\le 0, \end{aligned}$$

which implies that \(u\in {\mathcal {N}}_{\lambda ,\mu }^-\cup {\mathcal {N}}_{\lambda ,\mu }^0\). By Lemma 2.3, we know \({\mathcal {N}}_{\lambda ,\mu }^0=\emptyset \). Therefore, \(u\in {\mathcal {N}}_{\lambda ,\mu }^-\). Thus, \({\mathcal {N}}_{\lambda ,\mu }^-\) is a closed subset in \(X_{\lambda }\). \(\square \)

Lemma 2.7

Suppose \(u\in {\mathcal {N}}_{\lambda ,\mu }^+\) and \(v\in {\mathcal {N}}_{\lambda ,\mu }^-\) are minimizers of \(I_{\lambda ,\mu }\) on \({\mathcal {N}}_{\lambda ,\mu }^+\) and \({\mathcal {N}}_{\lambda ,\mu }^-\). Then for every nonnegative \(w\in X_{\lambda }\), we have

(i) there exists \(\varepsilon _0>0\) such that \(I_{\lambda ,\mu }(u+\varepsilon w)\ge I_{\lambda ,\mu }(u)\) for all \(0\le \varepsilon \le \varepsilon _0\).

(ii) \(t_{\varepsilon }\rightarrow 1\) as \(\varepsilon \rightarrow 0^+\), for \(\varepsilon \ge 0\), where \(t_\varepsilon \) is the unique positive real number satisfying \(t_\varepsilon (v+\varepsilon w)\in {\mathcal {N}}_{\lambda ,\mu }^-\).

Proof

(i) Let \(w\ge 0\) and for each \(\varepsilon \ge 0\), set

$$\begin{aligned} \sigma (\varepsilon )=\Vert u+\varepsilon w\Vert _{\lambda ,V}^2+\mu \gamma \int _{{\mathbb {R}}^N}f|u+\varepsilon w|^{1-\gamma }dx- (p-1)\int _{{\mathbb {R}}^N}g|u+\varepsilon w|^{p}dx. \end{aligned}$$

Then by using continuity of \(\sigma \) and \(\sigma (0)=N''_u(1)>0\), there exists \(\varepsilon _0>0\) such that \(\sigma (\varepsilon )>0\) for all \(0\le \varepsilon \le \varepsilon _{0}\). Similar to the proof of Lemma 2.4, for each \(\varepsilon >0\), there exists \(s_{\varepsilon }>0\) such that \(s_\varepsilon (u+\varepsilon w)\in {\mathcal {N}}_{\lambda ,\mu }^+\), such that \(I_{\lambda ,\mu }(s_\varepsilon (u+\varepsilon w))=\inf _{t>0}I_{\lambda ,\mu }(t(u+\varepsilon w))\), then for each \(\varepsilon \in [0,\varepsilon _0]\), we have

$$\begin{aligned} I_{\lambda ,\mu }(u+\varepsilon w)\ge I_{\lambda ,\mu }(s_\varepsilon (u+\varepsilon w))\ge I_{\lambda ,\mu }(u). \end{aligned}$$

(ii) For each \(v\in {\mathcal {N}}_{\lambda ,\mu }^-\), we define \(J:(0,\infty )\times {\mathbb {R}}^3\rightarrow {\mathbb {R}}\) by

$$\begin{aligned} J(t,l_1,l_2,l_3)=l_1t-\mu l_2t^{-\gamma }-l_3t^{p-1}, \end{aligned}$$

for \((t,l_1,l_2,l_3)\in (0,\infty )\times {\mathbb {R}}^3\). Since \(v\in {\mathcal {N}}_{\lambda ,\mu }^-\), one obtains

$$\begin{aligned} \frac{\partial J}{\partial t}(1,\Vert v\Vert _{\lambda ,V}^2,\int _{{\mathbb {R}}^N}f|v|^{1-\gamma }dx,\int _{{\mathbb {R}}^N}g|v|^pdx) =N''_v(1)<0. \end{aligned}$$

Moreover, for each \(\varepsilon >0\),

$$\begin{aligned} J(t_\varepsilon ,\Vert v+\varepsilon w\Vert _{\lambda ,V}^2,\int _{{\mathbb {R}}^N}f|v+\varepsilon w|^{1-\gamma }dx,\int _{{\mathbb {R}}^N}g|v+\varepsilon w|^pdx)=0. \end{aligned}$$

We also have

$$\begin{aligned} J(1,\Vert v\Vert _{\lambda ,V}^2,\int _{{\mathbb {R}}^N}f|v|^{1-\gamma }dx,\int _{{\mathbb {R}}^N}g|v|^pdx) =N'_v(1)=0. \end{aligned}$$

Applying the implicit function theorem, there exists an open neighbourhood \(A\subset (0,\infty )\) and \(B\subset {\mathbb {R}}^3\) containing 1 and \((\Vert v\Vert _{\lambda ,V}^2,\int _{{\mathbb {R}}^N}f|v|^{1-\gamma }dx,\int _{{\mathbb {R}}^N}g|v|^pdx)\) respectively such that for all \(J(t,y)=0\) has a unique solution \(t=j(y)\) with \(j:B\rightarrow A\) being a smooth function. Then one has

$$\begin{aligned} (\Vert v+\varepsilon w\Vert _{\lambda ,V}^2,\int _{{\mathbb {R}}^N}f|v+\varepsilon w|^{1-\gamma }dx,\int _{{\mathbb {R}}^N}g|v+\varepsilon w|^pdx)\in B, \end{aligned}$$

and

$$\begin{aligned} j(\Vert v+\varepsilon w\Vert _{\lambda ,V}^2,\int _{{\mathbb {R}}^N}f|v+\varepsilon w|^{1-\gamma }dx,\int _{{\mathbb {R}}^N}g|v+\varepsilon w|^pdx)=t_\varepsilon . \end{aligned}$$

Since

$$\begin{aligned} J(t_\varepsilon ,\Vert v+\varepsilon w\Vert _{\lambda ,V}^2,\int _{{\mathbb {R}}^N}f|v+\varepsilon w|^{1-\gamma }dx,\int _{{\mathbb {R}}^N}g|v+\varepsilon w|^pdx)=0. \end{aligned}$$

Thus, by continuity of g, we get \(t_{\varepsilon }\rightarrow 1\) as \(\varepsilon \rightarrow 0^+\). \(\square \)

Lemma 2.8

Suppose \(u\in {\mathcal {N}}_{\lambda ,\mu }^+\) and \(v\in {\mathcal {N}}_{\lambda ,\mu }^-\) are minimizers of \(I_{\lambda ,\mu }\) on \({\mathcal {N}}_{\lambda ,\mu }^+\) and \({\mathcal {N}}_{\lambda ,\mu }^-\). Then for every nonnegative \(w\in X_{\lambda }\), we have

$$\begin{aligned}{} & {} \langle u,w\rangle _{\lambda ,V}-\mu \int _{{\mathbb {R}}^N}fu^{-\gamma }wdx- \int _{{\mathbb {R}}^N}gu^{p-1}wdx\ge 0, \\{} & {} \langle v,w\rangle _{\lambda ,V}-\mu \int _{{\mathbb {R}}^N}fv^{-\gamma }wdx- \int _{{\mathbb {R}}^N}gv^{p-1}wdx\ge 0. \end{aligned}$$

Proof

Let \(w\in X_{\lambda }\) be a nonnegative function, then by Lemma 2.7, for each \(\varepsilon \in (0,\varepsilon _0)\), we have

$$\begin{aligned} \left. \begin{array}{rcl} 0&{}\le &{}\displaystyle \frac{I_{\lambda ,\mu }(u+\varepsilon w)-I_{\lambda ,\mu }(u)}{\varepsilon }\\ &{}=&{}\displaystyle \frac{1}{2\varepsilon }(\Vert u+\varepsilon w\Vert ^2_{\lambda ,V}-\Vert w\Vert ^2_{\lambda ,V}) -\frac{\mu }{(1-\gamma )}\int _{{\mathbb {R}}^N}f\frac{(u+\varepsilon w)^{1-\gamma }-u^{1-\gamma }}{\varepsilon }dx\\ &{}&{}-\displaystyle \frac{1}{p}\int _{{\mathbb {R}}^N}g\frac{(u+\varepsilon w)^{p}-u^{p}}{\varepsilon }dx. \end{array}\right. \nonumber \\ \end{aligned}$$
(2.6)

By (G) and the Lebesgue dominate convergence theorem, we have

$$\begin{aligned} \lim _{\varepsilon \rightarrow 0^+}\displaystyle \frac{1}{p}\int _{{\mathbb {R}}^N}g\frac{(u+\varepsilon w)^{p}-u^{p}}{\varepsilon }dx=\int _{{\mathbb {R}}^N}gu^{p-1}wdx. \end{aligned}$$

For \(0<\gamma <1\) and f is a positive continuous function, we have

$$\begin{aligned} f((u+\varepsilon w)^{1-\gamma }-u^{1-\gamma })\ge 0. \end{aligned}$$

It follows from (2.6) that

$$\begin{aligned} \displaystyle \liminf _{\varepsilon \rightarrow 0^+}\int _{{\mathbb {R}}^N}f\frac{(u+\varepsilon w)^{1-\gamma }-u^{1-\gamma }}{\varepsilon }dx<\infty . \end{aligned}$$

Then, by (2.6) and Fatou’s lemma, we get

$$\begin{aligned} \left. \begin{array}{rcl} \displaystyle \mu \int _{{\mathbb {R}}^N}fu^{-\gamma }wdx&{}\le &{}\displaystyle \frac{\mu }{1-\gamma }\liminf _{\varepsilon \rightarrow 0^+}\int _{{\mathbb {R}}^N}f\frac{(u+\varepsilon w)^{1-\gamma }-u^{1-\gamma }}{\varepsilon }dx\\ &{}\le &{}\displaystyle \langle u,w\rangle _{\lambda ,V}- \int _{{\mathbb {R}}^N}gu^{p-1}wdx, \end{array}\right. \end{aligned}$$

consequently, for each nonnegative \(w\in X_{\lambda }\), we have

$$\begin{aligned} \langle u,w\rangle _{\lambda ,V}-\mu \int _{{\mathbb {R}}^N}fu^{-\gamma }wdx- \int _{{\mathbb {R}}^N}gu^{p-1}wdx\ge 0. \end{aligned}$$

Next, we will show that these properties are also held for \(v\in {\mathcal {N}}_{\lambda ,\mu }^-\). For each \(\varepsilon >0\), there exists \(t_\varepsilon >0\) such that \(t_\varepsilon (v+\varepsilon w)\in {\mathcal {N}}_{\lambda ,\mu }^-\). By Lemma 2.7, for \(\varepsilon >0\) small enough, we get

$$\begin{aligned} I_{\lambda ,\mu }(t_\varepsilon (v+\varepsilon w))\ge I_{\lambda ,\mu }(v), \end{aligned}$$

which implies \(I_{\lambda ,\mu }(t_\varepsilon (v+\varepsilon w))-I_{\lambda ,\mu }(v)\ge 0\). Thus, one obtains

$$\begin{aligned} \left. \begin{array}{rcl} \displaystyle \frac{\mu t_\varepsilon ^{1-\gamma }}{(1-\gamma )}\int _{{\mathbb {R}}^N}f\frac{(v+\varepsilon w)^{1-\gamma }-v^{1-\gamma }}{\varepsilon }dx&{}\le &{}\displaystyle \frac{t_\varepsilon ^2}{2\varepsilon }(\Vert v+\varepsilon w\Vert ^2_{\lambda ,V}-\Vert v\Vert ^2_{\lambda ,V})\\ &{}&{}-\displaystyle \frac{t_\varepsilon ^{p}}{p}\int _{{\mathbb {R}}^N}g\frac{(v+\varepsilon w)^{p}-v^{p}}{\varepsilon }dx. \end{array}\right. \end{aligned}$$

Using the similar argument as in the previous case, we have

$$\begin{aligned} \langle v,w\rangle _{\lambda ,V}-\mu \int _{{\mathbb {R}}^N}fv^{-\gamma }wdx- \int _{{\mathbb {R}}^N}gv^{p-1}wdx\ge 0. \end{aligned}$$

\(\square \)

3 Proof of Theorem 1.1

Since \(I_{\lambda ,\mu }(u)=I_{\lambda ,\mu }(|u|)\), we can assume that \(u\ge 0\) for every \(u\in X_\lambda \). To get the main result, it is necessary to prove the following lemmas.

Lemma 3.1

Suppose that \(0<\gamma <1\) and \(2<p<2^{**}\), and the conditions (F),  (G) and \((V_1)-(V_3)\) are satisfied. Then for \((\lambda ,\mu )\in [\lambda ^*,+\infty )\times (0,\mu ^*)\), \(I_{\lambda ,\mu }\) has a minimizer \(u_0\) in \({\mathcal {N}}_{\lambda ,\mu }^+\) such that \(I_{\lambda ,\mu }(u_0)=c_{\lambda ,\mu }^+\).

Proof

By the Ekeland variational principle ([1]), there exists a minimizing sequence \(\{u_n\}\subset {\mathcal {N}}_{\lambda ,\mu }^+\) satisfying

(i) \(c^+_{\lambda ,\mu }<I_{\lambda ,\mu }(u_n)<c^+_{\lambda ,\mu }+\frac{1}{n}\),

(ii) \(I_{\lambda ,\mu }(u)\ge I_{\lambda ,\mu }(u_n)-\frac{1}{n}\Vert u_n-u\Vert \).

Moreover, by Lemma 2.2, one has \(\{u_n\}\) is bounded in \(X_{\lambda }\). Then there exists a subsequence of \(\{u_n\}\)(still denotes\(\{u_n\}\)) such that

$$\begin{aligned}{} & {} u_n\rightharpoonup u_0,~~~\textrm{in}~X_\lambda , \\{} & {} u_n\rightarrow u_0,~~~\textrm{in}~L^p({\mathbb {R}}^N),~p\in [2,2^{**}), \end{aligned}$$

with \(u_0\ge 0\). For \(0<\gamma <1\), \(f\in L^{\frac{p}{p+\gamma -1}}({\mathbb {R}}^N)\) is a positive continuous function, by the Vitali convergence theorem, one has

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _{{\mathbb {R}}^N}f|u_n|^{1-\gamma }dx= \int _{{\mathbb {R}}^N}f|u_0|^{1-\gamma }dx. \end{aligned}$$

\(\textbf{Step1}\): We prove that \(u_n\rightarrow u_0\) in \(X_\lambda \) and \(u_0\in {\mathcal {N}}_{\lambda ,\mu }^+\).

First, we show that \(u_0\ne 0\). Using the weak lower semi-continuity norm, we have

$$\begin{aligned} I_{\lambda ,\mu }(u_0)\le \liminf _{n\rightarrow \infty }I_{\lambda ,\mu }(u_n)=c^+_{\lambda ,\mu }<0. \end{aligned}$$

If \(u_0=0\), then \(I_{\lambda ,\mu }(u_0)=0\), which is a contradiction.

Next, we prove that \(u_n\rightarrow u_0\) in \(X_\lambda \). Suppose the contrary, by (2.1), one has

$$\begin{aligned} \Vert u_0\Vert ^2_{\lambda , V}<\liminf _{n\rightarrow \infty }\Vert u_n\Vert ^2_{\lambda ,V}. \end{aligned}$$

For \(u_n\in {\mathcal {N}}_{\lambda ,\mu }^+\), one has

$$\begin{aligned} \Vert u_0\Vert _{\lambda ,V}^2-\mu \int _{{\mathbb {R}}^N}f|u_0|^{1-\gamma }dx- \int _{{\mathbb {R}}^N}g|u_0|^{p}dx<0. \end{aligned}$$
(3.1)

Now, we prove that for \(u_0\), there exists \(0<t^+\ne 1\) such that \(t^+u_0\in {\mathcal {N}}_{\lambda ,\mu }^+\).

If \(\displaystyle \int _{{\mathbb {R}}^N}g|u|^{p}dx\le 0\), then by Lemma 2.4(i), there exists \(t^+>0\) such that \(t^+u_0\in {\mathcal {N}}_{\lambda ,\mu }^+\) and \(I'_{\lambda ,\mu }(t^+u_0)=0\). By (3.1), we obtain that \(I'_{\lambda ,\mu }(u_0)\ne 0\). Hence, \(t^+\ne 1\).

If \(\displaystyle \int _{{\mathbb {R}}^N}g|u|^{p}dx>0\), then by Lemma 2.4(ii), there exists \(0<t^+\ne 1\) such that \(t^+u_0\in {\mathcal {N}}_{\lambda ,\mu }^+\).

Since \(t^+u_0\) is a minimizer of \(I_{\lambda ,\mu }\) in \(X_{\lambda }\), then

$$\begin{aligned} I_{\lambda ,\mu }(t^+u_0)<I_{\lambda ,\mu }(u_0)\le \lim _{n\rightarrow \infty }I_{\lambda ,\mu }(u_n) =c_{\lambda ,\mu }^+, \end{aligned}$$

which contradicts \(c_{\lambda ,\mu }^+=\inf _{u\in {\mathcal {N}}_{\lambda ,\mu }^+}I_{\lambda ,\mu }(u)\). Then, we obtain \(u_n\rightarrow u_0\) in \(X_\lambda \).

Finally, we claim that \(u_0\in {\mathcal {N}}_{\lambda ,\mu }^+\). Suppose the contrary, assume that \(u_0\in {\mathcal {N}}_{\lambda ,\mu }^-\). It follows from (2.4) and \(u_0\in {\mathcal {N}}_{\lambda ,\mu }^-\) that

$$\begin{aligned} \int _{{\mathbb {R}}^N}g|u_0|^{p}dx>0. \end{aligned}$$

Then, by Lemma 2.4(ii), there exist unique \(t^+>0,~t^->0\) with \(t^->t^+>0\), such that \(t^+u_0\in {\mathcal {N}}_{\lambda ,\mu }^+\), \(t^-u_0\in {\mathcal {N}}_{\lambda ,\mu }^-\) and

$$\begin{aligned} I_{\lambda ,\mu }(t^+u_0)=\inf _{0\le 0\le t_{\max }}I_{\lambda ,\mu }(tu_0),~~I_{\lambda ,\mu }(t^-u_0)=\sup _{t\ge t_{\max }}I_{\lambda ,\mu }(tu_0). \end{aligned}$$

For \(u_0\in {\mathcal {N}}_{\lambda ,\mu }^-\), it suffices to prove that

$$\begin{aligned} \frac{d}{dt}I_{\lambda ,\mu }(u_0)=0,~~\frac{d^2}{dt^2}I_{\lambda ,\mu }(u_0)<0. \end{aligned}$$

This indicates \(t^-=1\). Also, since

$$\begin{aligned} \frac{d}{dt}I_{\lambda ,\mu }(t^+u_0)=0,~~\frac{d^2}{dt^2}I_{\lambda ,\mu }(t^+u_0)>0, \end{aligned}$$

then there exists \(t\in (t^+,1]\), such that

$$\begin{aligned} c_{\lambda ,\mu }^+\le I_{\lambda ,\mu }(t^+u_0)<I_{\lambda ,\mu }(tu_0)\le I_{\lambda ,\mu }(u_0)=c_{\lambda ,\mu }^+, \end{aligned}$$

this is a contradiction. Therefore, \(u_0\in {\mathcal {N}}_{\lambda ,\mu }^+\).

\(\textbf{Step2}\): \(u_0\) is a solution of (1.1).

In the following, we show the solution \(u_0\) is a weak solution of (1.1). Let \(v\in X_{\lambda }\) and \(\varepsilon >0\). Set \(\Omega _+=\{x\in {\mathbb {R}}^N:u_0+\varepsilon v\ge 0\}\) and \(\Omega _-=\{x\in {\mathbb {R}}^N:u_0+\varepsilon v<0\}\), then by Lemma 2.8, we obtain that

$$\begin{aligned} \left. \begin{array}{rcl} 0&{}\le &{}\displaystyle \int _{\Omega _+}\left( \Delta u_0\Delta (u_0+\varepsilon v)+V_\lambda (x)u_0(u_0+\varepsilon v)\right) dx-\mu \int _{\Omega _+}fu_0^{-\gamma }(u_0+\varepsilon v)dx\\ &{}&{}\displaystyle -\int _{\Omega _+}gu_0^{p-1}(u_0+\varepsilon v)dx\\ &{}=&{}\displaystyle \Vert u_0\Vert _{\lambda ,V}^2-\mu \int _{{\mathbb {R}}^N}fu_0^{1-\gamma }dx- \int _{{\mathbb {R}}^N}gu_0^{p}dx\\ &{}&{}+\displaystyle \varepsilon \left( \langle u_0,v\rangle _{\lambda ,V}-\mu \int _{{\mathbb {R}}^N}fu_0^{-\gamma }vdx- \int _{{\mathbb {R}}^N}gu_0^{p-1}vdx\right) \\ &{}&{}-\displaystyle \bigg (\int _{\Omega _-}(\Delta u_0\Delta (u_0+\varepsilon v)+V_\lambda (x)u_0(u_0+\varepsilon v))dx-\mu \int _{\Omega _-}fu_0^{-\gamma }(u_0+\varepsilon v)dx\\ &{}&{}-\displaystyle \int _{\Omega _-}gu_0^{p-1}(u_0+\varepsilon v)dx\bigg ). \end{array}\right. \end{aligned}$$

Then, for the fact \(u_0\in {\mathcal {N}}_{\lambda ,\mu }^+\) and f(x) is a positive continuous function, we have

$$\begin{aligned} \left. \begin{array}{rcl} 0&{}\le &{}\displaystyle \varepsilon \left( \langle u_0,v\rangle _{\lambda ,V}-\mu \int _{{\mathbb {R}}^N}fu_0^{-\gamma }vdx- \int _{{\mathbb {R}}^N}gu_0^{p-1}vdx\right) \\ &{}&{}-\displaystyle \varepsilon \int _{\Omega _-}\left( \Delta u_0\Delta v+V_\lambda (x)u_0v\right) dx+\int _{\Omega _-}gu_0^{p-1}(u_0+\varepsilon v)dx. \end{array}\right. \nonumber \\ \end{aligned}$$
(3.2)

Since the measure of the domain of integration \(\Omega _-=\{x\in {\mathbb {R}}^N:u_0+\varepsilon v<0\}\) tends to 0 as \(\varepsilon \rightarrow 0^+\), it follows that

$$\begin{aligned} |\int _{\Omega _-}\left( \Delta u_0\Delta v+V_\lambda (x)u_0v\right) dx|\rightarrow 0. \end{aligned}$$

Moreover, by (G) and Lemma 2.1, when \(\varepsilon \rightarrow 0^+\), one has

$$\begin{aligned} \bigg |\int _{\Omega _-}gu_0^{p-1}(u_0+\varepsilon v)dx\bigg |\le |g|_\infty \int _{\Omega _-}g|u_0|^{p}dx+\varepsilon |g|_\infty \bigg |\int _{\Omega _-}g|u_0|^{p-1}vdx\bigg |\rightarrow 0. \end{aligned}$$

Dividing by \(\varepsilon \) and letting \(\varepsilon \rightarrow 0\) in (3.2), one obtains

$$\begin{aligned} \langle u_0,v\rangle _{\lambda ,V}-\mu \int _{{\mathbb {R}}^N}fu_0^{-\gamma }vdx- \int _{{\mathbb {R}}^N}gu_0^{p-1}vdx\ge 0. \end{aligned}$$

Since v is arbitrary, the inequality above holds for \(-v\). Hence, for all \(v\in X_\lambda \), one has

$$\begin{aligned} \langle u_0,v\rangle _{\lambda ,V}-\mu \int _{{\mathbb {R}}^N}fu_0^{-\gamma }vdx- \int _{{\mathbb {R}}^N}gu_0^{p-1}vdx=0. \end{aligned}$$

Then \(u_0\) is a positive solution for (1.1). \(\square \)

Lemma 3.2

Suppose that \(0<\gamma <1\) and \(2<p<2^{**}\), and the conditions (F),  (G) and \((V_1)-(V_3)\) are satisfied. Then for \((\lambda ,\mu )\in [\lambda ^*,+\infty )\times (0,\mu ^*)\), \(I_{\lambda ,\mu }\) has a minimizer \(v_0\) in \({\mathcal {N}}_{\lambda ,\mu }^-\) such that \(I_{\lambda ,\mu }(v_0)=c_{\lambda ,\mu }^-\).

Proof

On account of \(I_{\lambda ,\mu }\) is also coercive on \({\mathcal {N}}_{\lambda ,\mu }^-\), we apply the Ekeland’s variational principle to the minimization problem \(c^-_{\lambda ,\mu }=\inf _{u\in {\mathcal {N}}^-_{\lambda ,\mu }}I_{\lambda ,\mu }(u)\), there exists a minimizing sequence \(\{v_n\}\subset {\mathcal {N}}^-_{\lambda ,\mu }\) of \(I_{\lambda ,\mu }\) with the following properties

(i) \(c^-_{\lambda ,\mu }<I_{\lambda ,\mu }(v_n)<c^-_{\lambda ,\mu }+\frac{1}{n}\),

(ii) \(I_{\lambda ,\mu }(v)\ge I_{\lambda ,\mu }(v_n)-\frac{1}{n}\Vert v_n-v\Vert \).

Moreover, \(\{v_n\}\) is bounded in \(X_\lambda \), then there exists a subsequence of \(\{v_n\}\)(still denotes\(\{v_n\}\)) such that

$$\begin{aligned}{} & {} v_n\rightharpoonup v_0,~~~\textrm{in}~X_\lambda , \\{} & {} v_n\rightarrow v_0,~~~\textrm{in}~L^p({\mathbb {R}}^N),~p\in [2,2^{**}), \end{aligned}$$

with \(v_0\ge 0\). Then we have

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _{{\mathbb {R}}^N}f|v_n|^{1-\gamma }dx= \int _{{\mathbb {R}}^N}f|v_0|^{1-\gamma }dx \end{aligned}$$

and

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _{{\mathbb {R}}^N}g|v_n|^{p}dx= \int _{{\mathbb {R}}^N}g|v_0|^{p}dx. \end{aligned}$$

We will show that \(v_0\ne 0\). If \(v_0=0\), then \(v_n\) converges to 0 strongly in \(X_\lambda \), which contradicts Lemma 2.5. Next, we prove that \(v_n\rightarrow v_0\) in \(X_\lambda \). If \(v_n\not \rightarrow v_0\) in \(X_\lambda \) then

$$\begin{aligned} \left. \begin{array}{rcl} &{}&{}\displaystyle \Vert v_0\Vert _{\lambda ,V}^2-\mu \int _{{\mathbb {R}}^N}f|v_0|^{1-\gamma }dx- \int _{{\mathbb {R}}^N}g|v_0|^{p}dx\\ &{}&{}\quad <\displaystyle \liminf _{n\rightarrow \infty }\bigg [\Vert v_n\Vert _{\lambda ,V}^2-\mu \int _{{\mathbb {R}}^N}f|v_n| ^{1-\gamma }dx- \int _{{\mathbb {R}}^N}g|v_n|^{p}dx\bigg ]=0. \end{array}\right. \nonumber \\ \end{aligned}$$
(3.3)

Since \(\{v_n\}\subset {\mathcal {N}}^-_{\lambda ,\mu }\), we deduce from (2.4) that

$$\begin{aligned} \mu (1+\gamma )\int _{{\mathbb {R}}^N}f|v_0|^{1-\gamma }dx+(2-p)\int _{{\mathbb {R}}^N}g|v_0|^{p}dx\le 0. \end{aligned}$$

Consequently, one has \(\int _{{\mathbb {R}}^N}g|v_0|^{p}dx>0\). Then by Lemma 2.5(ii), there exists a \(t^->0\) such that \(I'_{\lambda ,\mu }(t^-v_0)=0\) and \(t^-v_0\in {\mathcal {N}}^-_{\lambda ,\mu }\). Note that \(I'_{\lambda ,\mu }(v_0)\ne 0\) by (3.3). Thus, \(t^-\ne 1\). Since \(t^-v_n\rightharpoonup t^-v_0\) and \(t^-v_n\not \rightarrow t^-v_0\) in \(X_\lambda \). Hence,

$$\begin{aligned} I_{\lambda ,\mu }(t^-v_0)<\liminf _{n\rightarrow \infty }I_{\lambda ,\mu }(t^-v_n). \end{aligned}$$

Observe that \(I_{\lambda ,\mu }(tv_n)\) attains its maximum at \(t=1\). Thus, one obtains

$$\begin{aligned} I_{\lambda ,\mu }(t^-v_0)<\liminf _{n\rightarrow \infty }I_{\lambda ,\mu }(t^-v_n) \le \lim _{n\rightarrow \infty }I_{\lambda ,\mu }(v_n) =c_{\lambda ,\mu }^-, \end{aligned}$$

which is absurd. Therefore, we obtain that \(v_n\rightarrow v_0\) in \(X_\lambda \). Since \({\mathcal {N}}^-_{\lambda ,\mu }\) is closed by Lemma 2.6, it follows that \(v_0\in {\mathcal {N}}^-_{\lambda ,\mu }\). By Lemmas 2.7 and 2.8, similar to Lemma 3.1, we deduce that \(v_0\) is also a positive solution of (1.1).

\(\square \)

Proof of Theorem 1.1

According to Lemmas 3.1 and 3.2, for \((\lambda ,\mu )\in [\lambda ^*,+\infty )\times (0,\mu ^*)\), we know that (1.1) admits at least two positive solutions \(u_0\in {\mathcal {N}}^+_{\lambda ,\mu }\) and \(v_0\in {\mathcal {N}}^-_{\lambda ,\mu }\). Since \({\mathcal {N}}^+_{\lambda ,\mu }\cap {\mathcal {N}}^-_{\lambda ,\mu }=\emptyset \), the two solutions are different. This finishes the proof. \(\square \)