1 Introduction

We firstly study the multiplicity of nontrivial weak solutions to the singular quasilinear Schrödinger equations involving the critical nonlinearity as follows:

$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle -\Delta u-\frac{\alpha }{2}\Delta (|u|^{\alpha })|u|^{\alpha -2}u=\theta |u|^{k-2}u+|u|^{2^{*}-2}u+\lambda f(u), x\in \Omega ,\\ \hspace{1.65in}u=\,0, x\in \partial \Omega , \end{array} \right. \end{aligned}$$
(1)

where \(\theta , \lambda >0\), \(0<\alpha <1\), \(1<k<2\), \(2^{*}=\frac{2N}{N-2}\), \(N\ge 3\) and \(\Omega \) is a bounded smooth domain in \(\mathbb {R}^{N}\).

Solutions of Eq. (1) have relation to the existence of standing waves solutions for the time-dependent quasilinear Schrödinger problem:

$$\begin{aligned} i\psi _{t}+\Delta \psi -V(x)\psi +\frac{\alpha }{2}\Delta (|\psi |^{\alpha })|\psi |^{\alpha -2}\psi +h(\psi )=0, x\in \mathbb {R}^N, \end{aligned}$$
(2)

where h is a nonlinearity and \(V(x):\mathbb {R}^N\rightarrow \mathbb {R}\) is a potential. Quasilinear Schrödinger equation (2) is derived as models of several physical phenomena, such as [10]. The quasilinear case corresponding to \(\alpha =2\) has been researched extensively in the last 15 years; we refer to [15, 16, 21, 24,25,26] and references therein. For \(\alpha >1\), the existence of ground state solutions of (2) with \(h(s)=\lambda |s|^{p-2}s\)(\(2<p<\alpha 2^{*}\)), was established by Liu and Wang in [14]. Later, Adachi et al. ([1, 2]), respectively, studied the existence of positive solution with \(\alpha \ge \frac{3}{2}\) and the uniqueness of ground state solution for \(\alpha >1\). In [3], Adachi et al. considered the following problem

$$\begin{aligned} -\Delta u-\Delta (|u|^{\alpha })|u|^{\alpha -2}u=|u|^{p-2}u,\ x\in \mathbb {R}^N. \end{aligned}$$
(3)

They established that if \(2<p<\alpha 2^{*}\) with \(\alpha >1\), then Eq. (3) has a positive radical solution in \(D^{1,2}(\mathbb {R}^N)\). In [22], Shen et al. considered the existence of nontrivial solutions of (2) with \(\alpha \ge \frac{3}{2}\) and \(h(s)=|s|^{\alpha 2^{*}-2}s+f(s)\). And, more remarkable in Lemma 3.1 of [22], the condition \(\alpha \ge \frac{3}{2}\) is very important to estimate the critical level. This phenomenon which the condition \(\alpha \ge \frac{3}{2}\) is needed, has also been studied by Deng et al. in [9]. As we will see later, our problem (1) is very different with the case \(\alpha \ge \frac{3}{2}\); for example the critical exponent is \(2^{*}\) for (3) if \(0<\alpha \le 1\), while if \(\alpha \ge \frac{3}{2}\), \(\alpha 2^{*}\) behaves like a critical exponent, see [22].

To the best of our knowledge, there are few results in \(0<\alpha \le 1\). Recently, Wang et al. [23] studied (2) with \(\frac{k}{2}<\alpha <1(k\in (1,2))\) and \(h(s)=\lambda V(x)|s|^{k-2}s+\beta K(x)|s|^{2^{*}-2}s\); they verified the multiplicity of solutions. And on this basis, by adding a low-order perturbation term f(s) in nonlinearity, the argument in [23] is not valid to apply the case \(h(s)=\theta |s|^{k-2}s+|s|^{2^{*}-2}s+\lambda f(s)\) directly. This is because the lower bound estimation of the truncated energy functional of the equation will be much more complicated.

To begin our study, we give the following hypotheses:

\((f_{1})\)::

there exist \(C_{1},C_{2}>0\) and \(k<p,q<2^{*}\) such that

$$\begin{aligned} |f(s)|\le C_{1}|s|^{p-1}+C_{2}|s|^{q-1},\ \text{ for } \text{ all }\ s\in \mathbb {R}; \end{aligned}$$
\((f_{2})\)::

there exist \(C_3>0\) and \(\beta \in (1,2)\) such that for all \(s\in \mathbb {R}\)

$$\begin{aligned} f(s)s-2^{*}F(s)\ge -C_{3}|s|^{\beta }, \end{aligned}$$

where \(F(s)=\int _{0}^{s}f(t)\text{ d }t\);

\((f_{3})\)::

\(f(s)>0\) for \(s\in \mathbb {R}^{+}\);

\((f_{4})\)::

\(f(-s)=-f(s)\) for \(s\in \mathbb {R}\).

Under the above assumptions, we obtain our main theorems.

Theorem 1.1

Assume \(1<k<2\), \(\frac{k}{2}<\alpha <1\), \(N\ge 3\), f satisfies conditions \((f_1)-(f_{4})\). Then there exist \(\lambda _{0},\theta _{0}>0\), such that for all \(\lambda \in (0,\lambda _{0})\) and all \(\theta \in (0,\theta _{0})\), problem (1) possesses a sequence of solutions with negative energies.

Remark 1.1

It is easy to see the assumption \((f_2)\) is weaker than the Ambrosetti-Rabinowitz condition.

Remark 1.2

It is not difficult to find the nonlinearities satisfying \((f_1)-(f_4)\).

  1. (a):

    \(f(s)=|s|^{\beta -2}s\) with \(k<\beta <2^{*}\),

  2. (b):

    \(f(s)=(C_{4}+\cos s)|s|^{\beta -2}s\) with \(k<\beta <2^{*}\) and \(C_{4}>1\).

To prove Theorem 1.1, we are faced with several difficulties. From the point of variational method, we have to find a suitable working space. Unfortunately, the term \(\frac{\alpha ^{2}}{2}\int _{\Omega }(u^{2})^{\alpha -1}|\nabla u|^{2}\text{ d }x\) is not well defined in \(H^{1}_{0}(\Omega )\). An even more difficult is that it becomes singular for \(0<\alpha <1\). The technical difficulty in handling problem (1) is that the corresponding energy functional seems hard to satisfy (PS)-conditions due to the lack of compactness of the embedding: \(H^{1}_{0}(\Omega )\hookrightarrow L^{2^{*}}(\Omega )\). In order to solve this difficulty we use the concentration-compactness principle. Under the symmetric hypotheses, we try to use Lusternik-Schnirelmann theory for \(Z_{2}\)-invariant functional (see, e.g., [19]). But the energy functional is not bounded from below, we could not directly apply this theory. Since the existence of the disturbance term f(s), the more refined analysis is needed.

Without odd condition about f(s), we obtain the conclusion below.

Theorem 1.2

Assume \(1<k<2\), \(\frac{k}{2}<\alpha <1\), \(N\ge 3\), f satisfies conditions \((f_1)-(f_{3})\). Then there exist \(\lambda _{0},\theta _{0}>0\), such that for all \(\lambda \in (0,\lambda _{0})\) and all \(\theta \in (0,\theta _{0})\), problem (1) has at least one weak solution.

Remark 1.3

The key point is that for \(1<k<2\), \(u=\textbf{0}\) fails to be a local minimizer of the functional. It is well known that the mountain pass theorem is not applicable in this situation.

In what follows, we try to study the existence of infinitely many solutions for the singular quasilinear Schrödinger elliptic equations with more general nonlinearities:

$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle -\Delta u-\frac{\alpha }{2}\Delta (|u|^{\alpha })|u|^{\alpha -2}u=h(x,u),\hspace{1cm} x\in \Omega ,\\ \hspace{1.65in}u=\,0, \hspace{1.9cm} x\in \partial \Omega , \end{array} \right. \end{aligned}$$
(4)

where \(N\ge 3\) and \(\frac{k}{2}<\alpha <1\) (where k will be defined in \((h_1)\)). For the nonlinearity h, we assume that it is continuous and satisfies the following conditions which give its behavior only in a neighborhood of the origin:

\((h_1)\)::

there exist \(\delta >0\), \(1<k<2\) and \(C_5>0\) such that \(h\in C(\mathbb {R}^N\times [-\delta ,\delta ],\mathbb {R})\), h is odd in t and

$$\begin{aligned} |h(x,t)|\le C_5|t|^{k-1}; \end{aligned}$$
\((h_2)\)::

there exist an \(x_0\in \mathbb {R}^N\), a constant \(r_0>0\) such that

$$\begin{aligned} \lim _{t\rightarrow 0}\left( \inf _{x\in B_{r_{0}}(x_0)}\frac{H(x,t)}{t^{2\alpha }}\right) =+\infty , \end{aligned}$$

where \(B_{r_0}(x_0)\) is a ball in \(\Omega \) centered at \(x_0\) with radius \(r_0\) and

$$\begin{aligned} H(x,t)=\int _{0}^{t}h(x,s)\text{ d }s. \end{aligned}$$

Note that (4) is the Euler-Lagrange equation associated to the natural energy functional:

$$\begin{aligned} J(u)=\frac{1}{2}\int _{\Omega }(1+\frac{\alpha ^{2}}{2}(u^{2})^{\alpha -1})|\nabla u|^{2}\text{ d }x-\int _{\Omega }H(x,u)\text{ d }x, \end{aligned}$$

which is not well-defined in \(H^{1}_{0}(\Omega )\). Two difficulties prevent us from applying the usual variational methods directly. On one hand, since we do not have information about the function h(x, t) at infinity, the term \(\int _{\mathbb {R}^{N}}H(x,t)\text{ d }x\) may be not well defined in \(H^{1}_{0}(\Omega )\). On the other hand, the presence of the quasilinear term \(\int _{\Omega }(u^{2})^{\alpha -1}|\nabla u|^{2}\text{ d }x\) keeps us from studying Eq. (4) in the usual Sobolev space.

Inspired by the work of Costa and Wang [8], we obtain a nontrivial solution for a modified singular quasilinear Schrödinger equation

$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle -\text{ div }(g^{2}(u)\nabla u)+g(u)g'(u)|\nabla u|^{2}=\widetilde{h}(x,u),\hspace{1cm} x\in \Omega ,\\ \hspace{2.2in}u=\,0, \hspace{1.9cm} x\in \partial \Omega , \end{array} \right. \end{aligned}$$
(5)

where g(t) will be defined in Sect. 2, and \(\widetilde{h}(x,t)\) is a new nonlinearity which will be well-defined. Furthermore, we use the variant Clark’s theorem due to [18] to prove (5) has a sequence of weak solutions. Then by Moser iteration we get \(L^\infty \)-estimates for these weak solutions. We shall show that they are also solutions of problem (4).

Next, we give the more general result.

Theorem 1.3

Assume \(1<k<2\), \(\frac{k}{2}<\alpha <1\), \((h_{1})\) and \((h_{2})\) hold. Then more general problem (4) possesses a sequence of weak solutions \(u_n\in H^{1}_{0}(\Omega )\) with \(J(u_n)\rightarrow 0\).

Remark 1.4

It is easy to say \(h(x,t)=\theta |t|^{k-2}t+|t|^{2^{*}-2}t+\lambda f(t)\) satisfies conditions \((h_1)\) and \((h_2)\). Hence Theorem 1.3 is more general than Theorem 1.1.

Remark 1.5

Notice that the nonlinearities h(x, t) are only assumed in a neighborhood of the origin, we can give the supercritical example of h(x, t) satisfying \((h_1)\) and \((h_2)\):

$$\begin{aligned} h(x,t)=\lambda |t|^{k-2}t+\mu |t|^{r-2}t \end{aligned}$$

with \(1<k<2\), \(\frac{k}{2}<\alpha <1\) and \(r>2^{*}\).

This paper is organized into four sections. We describe the preliminary results in Sect. 2, which we shall use in this paper. Theorem 1.1 is proved in Sect. 3 by the application of \(Z_2\) index theory. In Sect. 4, depending on appropriated assumptions, we can show problem (1) has at least one weak solution. In Sect. 5, we prove the more general result.

2 Preliminaries

In this paper, \(\parallel \cdot \parallel _{p}\) and \(\Vert \cdot \Vert \) denote the usual \(L^{p}(\Omega )\) norm and \(H^{1}_{0}(\Omega )\) norm, respectively

$$\begin{aligned} \Vert u\Vert _{p}=\left( \int _{\Omega }|u|^{p}\text{ d }x\right) ^{1/p}, \Vert u\Vert =\left( \int _{\Omega }|\nabla u|^{2}\text{ d }x\right) ^{1/2}. \end{aligned}$$

Let S be the usually Sobolev constant as follows:

$$\begin{aligned} S=\inf \limits _{u\in H^{1}_{0}(\Omega )}\frac{\Vert u\Vert ^{2}}{\Vert u\Vert _{2^{*}}^{2}}. \end{aligned}$$

Set

$$\begin{aligned} g(s)=\sqrt{1+\frac{\alpha ^{2}}{2}(s^{2})^{\alpha -1}},\ 0<\alpha <1. \end{aligned}$$
(6)

It is easy to say that \(g(s)=g(-s)(s\ne 0)\) and \(s=0\) is a singular point of g(s). Furthermore, g(s) is increasing in \((-\infty ,0)\) and is decreasing in \((0,+\infty )\). \(\lim \limits _{s\rightarrow 0}g(s)=+\infty \) and \(\lim \limits _{s\rightarrow \infty }g(s)=1\).

Letting

$$\begin{aligned} G(t)=\int _{0}^{t}g(s)\text{ d }s, \end{aligned}$$

then, we have

$$\begin{aligned} |G(t)|\le \int _{0}^{|t|}\left( 1+\frac{\alpha }{\sqrt{2}}s^{\alpha -1}\right) \text{ d }s=|t|+\frac{1}{\sqrt{2}}|t|^{\alpha },\ \ \text{ for } \text{ all }\ t\in \mathbb {R}. \end{aligned}$$

Furthermore, \(G'(t)=g(t)>0(t\ne 0)\), it implies that G(t) is strictly monotone and therefore there exists a continuous, odd and invertible function by \(t=G^{-1}(s)\).

Lemma 2.1

[23] If \(\alpha \in (0,1)\), then the function \(G^{-1}(s)\) satisfies:

  1. (i)

    \(\lim \limits _{s\rightarrow 0}\frac{|G^{-1}(s)|^{\alpha }}{|s|}=\sqrt{2}\);

  2. (ii)

    \(\lim \limits _{s\rightarrow \infty }\frac{|G^{-1}(s)|}{|s|}=1\);

  3. (iii)

    \(|G^{-1}(s)|\le |s|\), for all \(s\in \mathbb {R}\);

  4. (iv)

    \(\alpha |s|<|G^{-1}(s)g(G^{-1}(s))|\le |s|\), for all \(s\in \mathbb {R}\);

  5. (v)
    $$\begin{aligned} \left( G^{-1}(s)g(G^{-1}(s))\right) '\left\{ \begin{array}{ll} \displaystyle \in (\alpha ,1),\ s\ne 0,\\ \hspace{0in}=\alpha ,\ s=0; \end{array} \right. \end{aligned}$$
  6. (vi)

    \(\lim \limits _{s\rightarrow 0}\left( G^{-1}(s)g(G^{-1}(s))\right) '=\alpha ;\)

  7. (vii)

    \(\left| \frac{(r-1)g(s)-g'(s)s}{g^{2}(s)}\right| <r-\alpha \), for all \(r>1\), \(s\in \mathbb {R}\setminus \{0\}\).

Corollary 2.1

For all \(v\in H^{1}_{0}(\Omega )\) and \(\alpha >0\), then

$$\begin{aligned} \alpha |\nabla v|\le |\nabla \left( G^{-1}(v)g(G^{-1}(v))\right) |\le |\nabla v| \end{aligned}$$
(7)

and

$$\begin{aligned} |\nabla G^{-1}(v)|=\frac{1}{g(G^{-1}(v))}|\nabla v|\le |\nabla v|,\ \text{ for } \text{ all }\ v\in H^{1}_{0}(\Omega ). \end{aligned}$$
(8)

Hence, we obtain \(G^{-1}(v)g(G^{-1}(v))\), \(G^{-1}(v)\in H^{1}_{0}(\Omega )\). Besides, we get

$$\begin{aligned} |\nabla G^{-1}(v)|^{2}\le \nabla \left( G^{-1}(v)g(G^{-1}(v))\right) \nabla v. \end{aligned}$$
(9)

Lemma 2.2

[19]

  1. (i)

    The functional

    $$\begin{aligned} E(v)=\int _{\Omega }|G^{-1}(v)|^{k}\text{ d }x \end{aligned}$$

    is well defined and weakly continuous on \(H^{1}_{0}(\Omega )\). Moreover, E(v) is continuously differentiable; its derivative \(E': H^{1}_{0}(\Omega )\rightarrow H^{1}_{0}(\Omega )\) is given by

    $$\begin{aligned} <E'(v),\psi >=k\int _{\Omega }\frac{|G^{-1}(v)|^{k-2}G^{-1}(v)}{g(G^{-1}(v))}\psi \text{ d }x,\ \text{ for } \text{ all }\ \psi \in H^{1}_{0}(\Omega ). \end{aligned}$$
  2. (ii)

    The functional

    $$\begin{aligned} D(v)=\int _{\Omega }|G^{-1}(v)|^{2^{*}}\text{ d }x \end{aligned}$$

    is well defined on \(H^{1}_{0}(\Omega )\). D(v) is continuously differentiable; its derivative \(D':\ H^{1}_{0}(\Omega )\rightarrow H^{1}_{0}(\Omega )\) is given by

    $$\begin{aligned} <D'(v),\psi >=2^{*}\int _{\Omega }\frac{|G^{-1}(v)|^{2^{*}-2}G^{-1}(v)}{g(G^{-1}(v))}\psi \text{ d }x,\ \text{ for } \text{ all }\ \psi \in H^{1}_{0}(\Omega ). \end{aligned}$$
  3. (iii)

    If \(u_{n}\rightharpoonup u\) in \(H^{1}_{0}(\Omega )\), then

    $$\begin{aligned} G^{-1}(u_{n})\rightharpoonup G^{-1}(u)\ \text{ in }\ H^{1}_{0}(\Omega ). \end{aligned}$$
    (10)

Throughout this paper, we deal with problem (1) in \(H^{1}_{0}(\Omega )\) and consider the following functional

$$\begin{aligned} \Psi (u)=\frac{1}{2}\int _{\Omega }g^{2}(u)|\nabla u|^{2}\text{ d }x-\frac{\theta }{k}\int _{\Omega }|u|^{k}\text{ d }x-\frac{1}{2^{*}}\int _{\Omega }|u|^{2^{*}}\text{ d }x-\lambda \int _{\Omega }F(u)\text{ d }x.\nonumber \\ \end{aligned}$$
(11)

Notice that the first difficulty associated with (11) is that \(\Psi \) is not well defined in \(H_{0}^{1}(\Omega )\); thus, we cannot apply variational methods to deal with (11) directly. To overcome this difficulty, as in [20], we introduce the following change of variables:

$$\begin{aligned} u=G^{-1}(v)\ \text{ and }\ v=G(u)=\int _{0}^{u}g(s)\text{ d }s. \end{aligned}$$

After the changing of variables, \(\Psi (u)\) can be written by the following functional:

$$\begin{aligned} \begin{aligned} \Phi (v)=&\frac{1}{2}\int _{\Omega }|\nabla v|^{2}\text{ d }x-\frac{\theta }{k}\int _{\Omega }|G^{-1}(v)|^{k}\text{ d }x-\frac{1}{2^{*}}\int _{\Omega }|G^{-1}(v)|^{2^{*}}\text{ d }x\\&-\lambda \int _{\Omega }F(G^{-1}(v))\text{ d }x. \end{aligned} \end{aligned}$$
(12)

From the above lemmas, \(\Phi \) is well defined in \(H^{1}_{0}(\Omega )\), \(\Phi \in C^{1}(H^{1}_{0}(\Omega ),\mathbb {R})\) and for all \(\psi \in H_{0}^{1}(\Omega )\)

$$\begin{aligned} \begin{aligned} \langle \Phi '(v),\psi \rangle =&\int _{\Omega }\nabla v\nabla \psi \text{ d }x-\theta \int _{\Omega } \frac{|G^{-1}(v)|^{k-2}G^{-1}(v)}{g(G^{-1}(v))}\psi \text{ d }x\\&-\int _{\Omega }\frac{|G^{-1}(v)|^{2^{*}-2}G^{-1}(v)}{g(G^{-1}(v))}\psi \text{ d }x-\lambda \int _{\Omega }\frac{f(G^{-1}(v))}{g(G^{-1}(v))}\psi \text{ d }x. \end{aligned} \end{aligned}$$

It is easy to see that every critical points of (12) are the weak solutions of the equation:

$$\begin{aligned} -\Delta v=\theta \frac{|G^{-1}(v)|^{k-2}G^{-1}(v)}{g(G^{-1}(v))}+\frac{|G^{-1}(v)|^{2^{*}-2}G^{-1}(v)}{g(G^{-1}(v))}+\lambda \frac{f(G^{-1}(v))}{g(G^{-1}(v))}. \end{aligned}$$
(13)

This means to find the nontrivial weak solutions of (1); it suffices to find the critical points of (12).

We now give the definitions of the \(\mathrm (PS)_c\) sequence and \(\mathrm (PS)_{c}\)-conditions in X as follows.

Definition 2.1

  1. (i)

    Let X be a Banach space and \(f\in C^{1}(X,\mathbb {R})\). A sequence \(\{u_{k}\}\subseteq X\) such that \(\Phi (u_{k})\rightarrow c\) and \(\Phi '(u_{k})\rightarrow 0\)(in \(X^{*}\)) as \(k\rightarrow \infty \), is called a \(\mathrm (PS)_c\) sequence for \(\Phi \).

  2. (ii)

    Any \(\mathrm (PS)_c\) sequence for \(\Phi \) has a converging subsequence (in X); we call that \(\Phi \) satisfies \(\mathrm (PS)_c\)-conditions.

Next, we introduce the following concentration compactness lemma, which help us to prove the \(\mathrm (PS)_{c}\)-conditions.

Lemma 2.3

[12, 13] Let \(G^{-1}(v_n)\rightharpoonup G^{-1}(v)\) in \(H^{1}_{0}(\Omega )\) such that \(|G^{-1}(v_{n})|^{2^{*}}\rightharpoonup \nu \) and \(|\nabla G^{-1}(v_{n})|^{2}\rightharpoonup \mu \). Define the quantities

$$\begin{aligned}&\nu _{\infty }:=\lim \limits _{R\rightarrow \infty }\limsup \limits _{n\rightarrow \infty }\int _{|x|>R}|G^{-1}(v_n)|^{2^{*}}\text{ d }x, \\&\mu _{\infty }:=\lim \limits _{R\rightarrow \infty }\limsup \limits _{n\rightarrow \infty }\int _{|x|>R}|\nabla G^{-1}(v_n)|^{2}\text{ d }x. \end{aligned}$$

Then, for some at most countable set J, we have

$$\begin{aligned} \begin{aligned}&\displaystyle (1)\ \nu =|G^{-1}(v)|^{2^{*}}+\sum _{j\in J}\nu _{j}\delta _{x_{j}}\ \text{ and }\ \sum _{j\in J}\nu _{j}^{\frac{2}{2^{*}}}<\infty ;\\&\displaystyle (2)\ \mu \ge |\nabla G^{-1}(v)|^{2}+\sum _{j\in J}\mu _{j}\delta _{x_{j}};\\&\displaystyle (3)\ S\nu _{j}^{\frac{2}{2^{*}}}\le \mu _{j};\\&\displaystyle (4)\ \limsup \limits _{n\rightarrow \infty }\int _{\Omega }|G^{-1}(v_n)|^{2^{*}}\text{ d }x=\int _{\Omega }\text{ d }\nu +\nu _{\infty };\\&\displaystyle (5)\ \limsup \limits _{n\rightarrow \infty }\int _{\Omega }|\nabla G^{-1}(v_n)|^{2}\text{ d }x=\int _{\Omega }\text{ d }\mu +\mu _{\infty }, \end{aligned} \end{aligned}$$
(14)

where \(x_{j}\in \Omega \), \(\nu _{j}\), \(\mu _{j}\) are positive constants and \(\delta _{x_{j}}\) denotes the Dirac measure at \(x_{j}\).

Lemma 2.4

Assume \(1<k<2\), \(\frac{k}{2}<\alpha <1\), \(N\ge 3\) and \((f_1)-(f_{3})\) hold. Let \(\{v_{n}\}\subset H^{1}_{0}(\Omega )\) be a \(\mathrm (PS)_c\) sequence for \(\Phi (v)\). Then \(\{v_{n}\}\) is bounded in \(H^{1}_{0}(\Omega )\).

Proof

Assume \(\{v_{n}\}\subset H^{1}_{0}(\Omega )\) is a \(\mathrm (PS)_c\) sequence of \(\Phi (v)\), i.e.,

$$\begin{aligned} \begin{aligned} \Phi (v_n)&=\frac{1}{2}\int _{\Omega }|\nabla v_n|^{2}\text{ d }x-\frac{\theta }{k}\int _{\Omega }|G^{-1}(v_n)|^{k}\text{ d }x\\&~~-\frac{1}{2^{*}}\int _{\Omega }|G^{-1}(v_n)|^{2^{*}}\text{ d }x-\lambda \int _{\Omega }F(G^{-1}(v_n))\text{ d }x\\&=c+o_{n}(1), \end{aligned} \end{aligned}$$
(15)

and

$$\begin{aligned} \begin{aligned} \langle \Phi '(v_n),\psi \rangle&=\int _{\Omega }\nabla v_{n}\nabla \psi \text{ d }x-\theta \int _{\Omega } \frac{|G^{-1}(v_{n})|^{k-2}G^{-1}(v_{n})}{g(G^{-1}(v_{n}))}\psi \text{ d }x\\&~~-\int _{\Omega }\frac{|G^{-1}(v_{n})|^{2^{*}-2}G^{-1}(v_{n})}{g(G^{-1}(v_{n}))}\psi \text{ d }x-\lambda \int _{\Omega }\frac{f(G^{-1}(v_n))}{g(G^{-1}(v_n))}\psi \text{ d }x\\&=o_{n}(1)\Vert \psi \Vert , \end{aligned} \end{aligned}$$
(16)

where \(o_{n}(1)\rightarrow 0\ \text{ as }\ n\rightarrow \infty \), \(\psi \in H^{1}_{0}(\Omega )\). Taking \(\psi =G^{-1}(v_n)g(G^{-1}(v_n))\in H^{1}_{0}(\Omega )\)(see (7)), from (7) and (16) we have

$$\begin{aligned} o_{n}(1)\Vert v_n\Vert&=\langle \Phi '(v_n),G^{-1}(v_n)g(G^{-1}(v_n))\rangle \nonumber \\&=\int _{\Omega }\nabla v_{n}\nabla \left( G^{-1}(v_n)g(G^{-1}(v_n))\right) \text{ d }x-\theta \int _{\Omega } |G^{-1}(v_{n})|^{k}\text{ d }x\nonumber \\&~~-\int _{\Omega }|G^{-1}(v_{n})|^{2^{*}}\text{ d }x-\lambda \int _{\Omega }f(G^{-1}(v_n))G^{-1}(v_n)\text{ d }x\nonumber \\&\le \int _{\Omega }|\nabla v_{n}|^{2}\text{ d }x-\theta \int _{\Omega }|G^{-1}(v_{n})|^{k}\text{ d }x\nonumber \\&~~-\int _{\Omega }|G^{-1}(v_{n})|^{2^{*}}\text{ d }x-\lambda \int _{\Omega }f(G^{-1}(v_n))G^{-1}(v_n)\text{ d }x. \end{aligned}$$
(17)

Therefore, by Lemma 2.1-(iii), \((f_2)\), (15) and (17), we have

$$\begin{aligned} \begin{aligned} 2^{*}c+o_{n}(1)+o_{n}(1)\Vert v_n\Vert&\ge 2^{*}\Phi (v_{n})-\langle \Phi '(v_n),G^{-1}(v_n)g(G^{-1}(v_n))\rangle \\&\ge \frac{2^{*}-2}{2}\Vert v_n\Vert ^{2}-\frac{2^{*}-k}{k}\theta \int _{\Omega } |G^{-1}(v_{n})|^{k}\text{ d }x\\&\quad -\lambda \int _{\Omega }\left( 2^{*}F(G^{-1}(v_n))-f(G^{-1}(v_n))G^{-1}(v_n)\right) \text{ d }x\\&\ge \frac{2^{*}-2}{2}\Vert v_n\Vert ^{2}-C_{6}\Vert v_n\Vert ^{k}-\lambda C_{7}\Vert v_n\Vert ^{\beta }.\\ \end{aligned} \end{aligned}$$
(18)

where \(C_6=\frac{2^{*}-k}{k}\theta |\Omega |^{1-\frac{k}{2^{*}}}S^{-\frac{k}{2}}\) and \(C_7=C_{3}|\Omega |^{1-\frac{\beta }{2^{*}}}S^{-\frac{\beta }{2}}\). \(\square \)

Lemma 2.5

Assume \(1<k<2\), \(\frac{k}{2}<\alpha <1\), \(N\ge 3\) and \((f_1)-(f_{3})\) hold. Then there exist \(\widetilde{\lambda },\widetilde{\theta }>0\), such that for all \(\lambda \in (0,\widetilde{\lambda })\), all \(\theta \in (0,\widetilde{\theta })\) and \(c<0\), the functional \(\Phi \) satisfies \((PS)_c\)-conditions.

Proof

Assume \(\{v_n\}\) is a \((PS)_c\) sequence. By Lemma 2.4, we have the boundedness of \(\{v_{n}\}\) in \(H^{1}_{0}(\Omega )\). Then there exists \(v\in H^{1}_{0}(\Omega )\) such that, up to subsequence, \(v_{n}\rightharpoonup v\) in \(H^{1}_{0}(\Omega )\). Then by Lemma 2.2-(iii), we imply \(G^{-1}(v_{n})\rightharpoonup G^{-1}(v)\) in \(H^{1}_{0}(\Omega )\).

Define \(\psi \in C^{\infty }_{0}(\Omega )\) by \(\psi (x)=0\) for \(|x|>1\), \(\psi (x)=1\) for \(|x|\le \frac{1}{2}\), \(0\le \psi (x)\le 1,\ x\in \Omega \). For any fixed j and \(\varepsilon \in (0,1)\), consider

$$\begin{aligned} \psi _{\varepsilon }(x):=\psi \left( \frac{x-x_{j}}{\varepsilon }\right) . \end{aligned}$$

Due to the fact that \(\langle \Phi '(v_{n}),\psi _{\varepsilon }G^{-1}(v_n)g(G^{-1}(v_n))\rangle \rightarrow 0\) we have

$$\begin{aligned} \begin{array}{ll} \displaystyle \hspace{0cm} \displaystyle \int _{\Omega }\nabla [G^{-1}(v_n)g(G^{-1}(v_n))]\nabla v_{n}\psi _{\varepsilon }\text{ d }x+\int _{\Omega }G^{-1}(v_n)g(G^{-1}(v_n))\nabla v_{n}\nabla \psi _{\varepsilon }\text{ d }x\\ \hspace{-0.5cm} \displaystyle =\theta \int _{\Omega }|G^{-1}(v_n)|^{k}\psi _{\varepsilon }\text{ d }x+\int _{\Omega }|G^{-1}(v_n)|^{2^{*}}\psi _{\varepsilon }\text{ d }x\\ \hspace{-0.1cm} \displaystyle +\lambda \int _{\Omega }f(G^{-1}(v_n))G^{-1}(v_n)\psi _{\varepsilon }\text{ d }x+o_{n}(1). \end{array} \end{aligned}$$
(19)

By (9) we get

$$\begin{aligned} \begin{array}{ll} \displaystyle \hspace{0cm} \displaystyle \int _{\Omega }|\nabla G^{-1}(v_n)|^{2}\psi _{\varepsilon }\text{ d }x+\int _{\Omega }G^{-1}(v_n)g(G^{-1}(v_n))\nabla v_{n}\nabla \psi _{\varepsilon }\text{ d }x\\ \hspace{-0.5cm} \displaystyle \le \theta \int _{\Omega }|G^{-1}(v_n)|^{k}\psi _{\varepsilon }\text{ d }x+\int _{\Omega }|G^{-1}(v_n)|^{2^{*}}\psi _{\varepsilon }\text{ d }x\\ \hspace{-0.1cm} \displaystyle +\lambda \int _{\Omega }f(G^{-1}(v_n))G^{-1}(v_n)\psi _{\varepsilon }\text{ d }x+o_{n}(1), \end{array} \end{aligned}$$
(20)

and since \(\psi _{\varepsilon }\in L^{\infty }(\Omega )\), by Lemma 2.3 and (20), we get

$$\begin{aligned} \begin{array}{ll} \displaystyle \hspace{0cm} \displaystyle \int _{\Omega }\psi _{\varepsilon }\text{ d }\mu -\theta \int _{\Omega }|G^{-1}(v)|^{k}\psi _{\varepsilon }\text{ d }x-\int _{\Omega }\psi _{\varepsilon }\text{ d }\nu -\lambda \int _{\Omega }f(G^{-1}(v))G^{-1}(v)\psi _{\varepsilon }\text{ d }x\\ \hspace{-0.5cm} \displaystyle \le -\lim \limits _{n\rightarrow \infty }\int _{\Omega }G^{-1}(v_n)g(G^{-1}(v_n))\nabla v_{n}\nabla \psi _{\varepsilon }\text{ d }x. \end{array} \end{aligned}$$
(21)

By Lemma 2.1-(iv) and the Hölder inequality, we obtain

$$\begin{aligned} \begin{aligned} \left| \int _{\Omega }G^{-1}(v_n)g(G^{-1}(v_n))\nabla v_{n}\nabla \psi _{\varepsilon }\text{ d }x\right|&\le \int _{\Omega } |\nabla v_{n}||v_{n}\nabla \psi _{\varepsilon }|\text{ d }x\\&\le \Vert v_n\Vert \left( \int _{B_{\varepsilon }(x_{j})}|v_n|^{2}|\nabla \psi _{\varepsilon }|^{2}\text{ d }x\right) ^{1/2}.\\ \end{aligned} \end{aligned}$$

Furthermore, by \(|v_n\nabla \psi _{\varepsilon }|\rightarrow |v\nabla \psi _{\varepsilon }|\) in \(L^{2}(\Omega )\) we get

$$\begin{aligned} \begin{aligned}&\lim \limits _{n\rightarrow +\infty }\left( \Vert v_n\Vert \left( \int _{B_{\varepsilon }(x_{j})}|v_n|^{2}|\nabla \psi _{\varepsilon }|^{2}\text{ d }x\right) ^{1/2}\right) \\&\le C\left( \int _{B_{\varepsilon }(x_{j})}|v|^{2}|\nabla \psi _{\varepsilon }|^{2}\text{ d }x\right) ^{1/2}\\&\le C \left( \int _{B_{\varepsilon }(x_{j})}|v|^{2^{*}}\text{ d }x\right) ^{1/2^{*}}\left( \int _{B_{\varepsilon }(x_{j})}|\nabla \psi _{\varepsilon }|^{N}\text{ d }x\right) ^{1/N}\\&\le C\left( \int _{B_{\varepsilon }(x_{j})}|v|^{2^{*}}\text{ d }x\right) ^{1/2^{*}}\rightarrow 0,\ \text{ as }\ \varepsilon \rightarrow 0^{+}. \end{aligned} \end{aligned}$$
(22)

Hence, taking \(\varepsilon \rightarrow 0 \) in (21), we obtain

$$\begin{aligned} \nu _{j}\ge \mu _{j}. \end{aligned}$$
(23)

Clearly, by Lemma 2.3-(3) and (23), we have

(a) \(\nu _{j}=0\) or (b) \(\nu _{j}\ge S^{\frac{N}{2}}\).

We now claim that (b) is impossible if \(\lambda \) and \(\theta \) are taken small enough. Indeed, due to \(\{v_{n}\}\) is a \((\mathrm PS)_{c}\) sequence, it implies

$$\begin{aligned} \begin{aligned} 0>c+o_{n}(1)&=\Phi (v_{n})-\frac{1}{2^*}\langle \Phi '(v_{n}),G^{-1}(v_{n})g(G^{-1}(v_{n}))\rangle \\&\ge \frac{1}{N}\int _{\Omega }|\nabla v_n|^{2}\text{ d }x-\frac{(2^{*}-k)\theta }{k2^*}\int _{\Omega }|G^{-1}(v_{n})|^{k}\text{ d }x\\&-\frac{\lambda }{2^{*}}\int _{\Omega }\left( 2^{*}F(G^{-1}(v_n))-f(G^{-1}(v_n))G^{-1}(v_n)\right) \text{ d }x\\&\ge \frac{S}{N}\Vert v_{n}\Vert ^{2}_{2^*}-\theta A\Vert v_{n}\Vert _{2^*}^{k}-\lambda B\Vert v_{n}\Vert ^{\beta }_{2^*}, \end{aligned} \end{aligned}$$
(24)

where \(A=\frac{(2^{*}-k)}{2^*k}|\Omega |^{\frac{2^{*}-k}{2^{*}}}\) and \(B=\frac{c_3}{2^{*}}|\Omega |^{\frac{2^{*}-\beta }{2^{*}}}\). Without loss of generality, we assume \(k>\beta \) and \(\Vert v_n\Vert \le 1\). This implies

$$\begin{aligned} \frac{S}{N}\Vert v_{n}\Vert ^{2}_{2^*}-\theta A\Vert v_{n}\Vert _{2^*}^{k}-\lambda B\Vert v_{n}\Vert ^{\beta }_{2^*}\ge \frac{S}{N}\Vert v_{n}\Vert ^{2}_{2^*}-\theta A\Vert v_{n}\Vert ^{\beta }_{2^*}-\lambda B\Vert v_{n}\Vert ^{\beta }_{2^*}, \end{aligned}$$

then

$$\begin{aligned} \Vert v_n\Vert _{2^{*}}\le \left( \frac{N(\theta A+\lambda B)}{S}\right) ^{1/(2-\beta )}. \end{aligned}$$
(25)

On the other hand, for n large enough and if (b) occurs, we obtain

$$\begin{aligned} \begin{aligned} 0>c+o_{n}(1)&=\Phi (v_{n})-\frac{1}{2^*}\langle \Phi '(v_{n}),G^{-1}(v_{n})g(G^{-1}(v_{n}))\rangle \\&\ge \frac{1}{N}\int _{\Omega }|\nabla v_n|^{2}\text{ d }x-\frac{(2^{*}-k)\theta }{k2^*}\int _{\Omega }|G^{-1}(v_{n})|^{k}\text{ d }x\\&-\frac{\lambda }{2^{*}}\int _{\Omega }\left( 2^{*}F(G^{-1}(v_n))-f(G^{-1}(v_n))G^{-1}(v_n)\right) \text{ d }x\\&\ge \frac{1}{N}S^{\frac{N}{2}}-\theta A\Vert v_{n}\Vert _{2^*}^{k}-\lambda B\Vert v_{n}\Vert ^{\beta }_{2^*}. \end{aligned} \end{aligned}$$
(26)

Therefore we can choose \(\widetilde{\lambda },\widetilde{\theta }>0\) such that for every \(\lambda \in (0,\widetilde{\lambda })\) and \(\theta \in (0,\widetilde{\theta })\), we get

$$\begin{aligned} \begin{aligned}&\frac{1}{N}S^{\frac{N}{2}}-\theta A\Vert v_{n}\Vert _{2^*}^{k}-\lambda B\Vert v_{n}\Vert ^{\beta }_{2^*}\\&>\frac{1}{N}S^{\frac{N}{2}}-\theta A\left( \frac{N(\theta A+\lambda B)}{S}\right) ^{\frac{k}{2-\beta }}-\lambda B\left( \frac{N(\theta A+\lambda B)}{S}\right) ^{\frac{\beta }{2-\beta }}\\&>0, \end{aligned} \end{aligned}$$

which is a contradiction to (26). Thus, \(\mu _i=\nu _i=0\). From Lemma 2.3, we have

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _{\Omega }|G^{-1}(v_{n})|^{2^{*}}\text{ d }x=\int _{\Omega }|G^{-1}(v)|^{2^{*}}\text{ d }x. \end{aligned}$$
(27)

From Brezis-Lieb lemma [17], we imply

$$\begin{aligned} \lim _{n\rightarrow \infty }\int _{\Omega }|G^{-1}(v_{n})-G^{-1}(v)|^{2^{*}}\text{ d }x=0, \end{aligned}$$
(28)

and

$$\begin{aligned} \begin{aligned}&\lim _{n,m\rightarrow \infty }\int _{\Omega }|G^{-1}(v_{n})-G^{-1}(v_{m})|^{2^{*}}\text{ d }x\\&\le \lim \limits _{n\rightarrow \infty }\int _{\Omega }|G^{-1}(v_{n})-G^{-1}(v)|^{2^{*}}\text{ d }x+\lim \limits _{m\rightarrow \infty }\int _{\Omega }|G^{-1}(v_{m})-G^{-1}(v)|^{2^{*}}\text{ d }x\\&=0. \end{aligned} \end{aligned}$$
(29)

Due to Lemma 2.1-(vii), Hölder inequality and mean value theorem, we have

$$\begin{aligned} \begin{aligned}&\int _{\Omega }\left| \frac{G^{-1}(v_{n})^{2^{*}-1}}{g(G^{-1}(v_n))}-\frac{G^{-1}(v_{m})^{2^{*}-1}}{g(G^{-1}(v_m))}\right| |v_{n}-v_m|\text{ d }x\\&\le (2^{*}-\alpha )\int _{\Omega }|w_{n,m}|^{2^{*}-2}|G^{-1}(v_n)-G^{-1}(v_m)||v_{n}-v_{m}|\text{ d }x\\&\le (2^{*}-\alpha )\left( \int _{\Omega }|w_{n,m}|^{2^{*}}\text{ d }x\right) ^{\frac{2^{*}-2}{2^{*}}}\left( \int _{\Omega }|G^{-1}(v_n)-G^{-1}(v_m)|^{2^{*}}\text{ d }x\right) ^{\frac{1}{2^{*}}}\\&\left( \int _{\Omega }|v_{n}-v_{m}|^{2^{*}}\text{ d }x\right) ^{\frac{1}{2^{*}}}, \end{aligned} \end{aligned}$$
(30)

where \(w_{n,m}=\kappa G^{-1}(v_n)+(1-\kappa ) G^{-1}(v_m)\in L^{2^{*}}(\Omega )\), \(\kappa \in (0,1)\). Hence, we imply

$$\begin{aligned} \lim _{n,m\rightarrow \infty }\int _{\Omega }\left| \frac{G^{-1}(v_{n})^{2^{*}-1}}{g(G^{-1}(v_n))}-\frac{G^{-1}(v_m)^{2^{*}-1}}{g(G^{-1}(v_m))}\right| |v_{n}-v_m|\text{ d }x=0. \end{aligned}$$
(31)

By similar argument, we get

$$\begin{aligned} \lim _{n,m\rightarrow \infty }\int _{\Omega }\left| \frac{G^{-1}(v_{n})^{k-1}}{g(G^{-1}(v_n))}-\frac{G^{-1}(v_m)^{k-1}}{g(G^{-1}(v_m))}\right| |v_{n}-v_m|\text{ d }x=0. \end{aligned}$$
(32)

Next, we will get the similar estimate about the disturbance term as follows:

$$\begin{aligned} \begin{aligned}&\lim _{n,m\rightarrow \infty }\int _{\Omega }\left| \frac{f(G^{-1}(v_n))}{g(G^{-1}(v_n))}-\frac{f(G^{-1}(v_m))}{g(G^{-1}(v_m))}\right| |v_{n}-v_m|\text{ d }x\\&\le \lim _{n,m\rightarrow \infty }C\int _{\Omega }\left( \left| G^{-1}(v_n)\right| ^{p-1}+\left| G^{-1}(v_n)\right| ^{q-1}\right. \\ {}&\quad \left. +\left| G^{-1}(v_m)\right| ^{p-1}+\left| G^{-1}(v_m)\right| ^{q-1}\right) |v_{n}-v_m|\text{ d }x\\&\le \lim _{n,m\rightarrow \infty }C\int _{\Omega }\left( \left| v_n\right| ^{p-1}+\left| v_n\right| ^{q-1}+\left| v_m\right| ^{p-1}+\left| v_m\right| ^{q-1}\right) |v_{n}-v_m|\text{ d }x\\&=0, \end{aligned} \end{aligned}$$
(33)

where we use \((f_2)\), Lemma 2.1-(ii), Hölder inequality and \(v_n\rightarrow v\) in \(L^{s}(\Omega )\)( \(s\in (1,2^{*})\)).

By using \(\lim \limits _{n,m\rightarrow \infty }\langle \Phi '(v_{n})-\Phi '(v_{m}),v_{n}-v_m\rangle =0\), taking \(n,m\rightarrow \infty \), we have

$$\begin{aligned} \begin{aligned}&\int _{\Omega }|\nabla (v_n-v_m)|^{2}\text{ d }x-\theta \int _{\Omega }\left| \frac{G^{-1}(v_{n})^{k-1}}{g(G^{-1}(v_n))}-\frac{G^{-1}(v_m)^{k-1}}{g(G^{-1}(v_m))}\right| |v_{n}-v_m|\text{ d }x\\&\qquad -\int _{\Omega }\left| \frac{G^{-1}(v_{n})^{2^{*}-1}}{g(G^{-1}(v_n))}-\frac{G^{-1}(v_m)^{2^{*}-1}}{g(G^{-1}(v_m))}\right| |v_{n}-v_m|\text{ d }x\\&\qquad -\lambda \int _{\Omega }\left| \frac{f(G^{-1}(v_n))}{g(G^{-1}(v_n))}-\frac{f(G^{-1}(v_m))}{g(G^{-1}(v_m))}\right| |v_{n}-v_m|\text{ d }x\rightarrow 0. \end{aligned} \end{aligned}$$
(34)

Using (31)–(34), we get

$$\begin{aligned} \lim _{n,m\rightarrow \infty }\int _{\Omega }|\nabla (v_n-v_m)|^{2}\text{ d }x=0. \end{aligned}$$

Hence, we deduce that \(\{v_n\}\) is a Cauchy sequence in \(H^{1}_{0}(\Omega )\). Since \(H^{1}_{0}(\Omega )\) is a Banach space, up to subsequence, we get \(v_n\rightarrow v\) in \(H^{1}_{0}(\Omega )\). The proof is finished. \(\square \)

Before proceeding further, let us truncate the energy functional \(\Phi (u)\). By Sobolev embedding theorem with the hypothesis \(1<k<2\), (K) and Lemma 2.1-(iii), for all \(v\in H^{1}_{0}(\Omega )\), we have

$$\begin{aligned} \begin{aligned} \Phi (v)&=\frac{1}{2}\int _{\Omega }|\nabla v|^{2}\text{ d }x-\frac{\theta }{k}\int _{\Omega }|G^{-1}(v)|^{k}\text{ d }x-\frac{1}{2^{*}}\int _{\Omega }|G^{-1}(v)|^{2^{*}}\text{ d }x\\&\quad -\lambda \int _{\Omega }F(G^{-1}(v))\text{ d }x\\&\ge \frac{1}{2}\Vert v\Vert ^{2}-\frac{\theta }{k}C_{8}\Vert v\Vert ^{k}-\frac{S^{\frac{-2^{*}}{2}}}{2^{*}}\Vert v\Vert ^{2^*}-\lambda \int _{\Omega }F(G^{-1}(v))\text{ d }x\\&\ge \frac{1}{2}\Vert v\Vert ^{2}-\frac{\theta }{k}C_{8}\Vert v\Vert ^{k}-\frac{S^{\frac{-2^{*}}{2}}}{2^{*}}\Vert v\Vert ^{2^*}\\&-\lambda \frac{C_1}{p}|\Omega |^{\frac{2^{*}-p}{2^{*}}}S^{-\frac{p}{2}}\Vert v\Vert ^{p}-\lambda \frac{C_2}{q}|\Omega |^{\frac{2^{*}-q}{2^{*}}}S^{-\frac{q}{2}}\Vert v\Vert ^{q}\\&:=\frac{1}{2}\Vert v\Vert ^{2}-C_{9}\Vert v\Vert ^{2^*}-C_{10}\theta \Vert v\Vert ^{k}-C_{11}\lambda \Vert v\Vert ^{p}-C_{12}\lambda \Vert v\Vert ^{q}, \end{aligned} \end{aligned}$$
(35)

where \(C_8=|\Omega |^{1-\frac{k}{2^{*}}}S^{-\frac{k}{2}}\), \(C_{9}=\frac{1}{2^{*}}S^{\frac{-2^{*}}{2}}\), \(C_{10}=\frac{C_{8}}{k}\), \(C_{11}=\frac{c_1}{p}|\Omega |^{\frac{2^{*}-p}{2^{*}}}S^{-\frac{p}{2}}\) and \(C_{12}=\frac{c_2}{q}|\Omega |^{\frac{2^{*}-q}{2^{*}}}S^{-\frac{q}{2}}\) are all positive constants.

Letting \(l(t)=\frac{1}{2}t^{2}-C_{9}t^{2^*}-C_{10}\theta t^{k}-C_{11}\lambda t^{p}-C_{12}\lambda t^{q}\), we need to study the properties of l(t). By the hypothesis \(1<k<2\) and \(1<p,q<2^{*}\), we easily know that there exists a positive constant \(\lambda _{0}\le \widetilde{\lambda }\) such that for any \(\lambda \in (0,\lambda _{0})\), \(l_{0}(t)=\frac{1}{2}t^{2}-C_{9}t^{2^*}-C_{11}\lambda t^{p}-C_{12}\lambda t^{q}\) possesses positive value for some \(t_{0}>0\). Then for any \(\lambda \in (0,\lambda _{0})\), there exists a \(\theta _{0}(\lambda )(0<\theta _{0}<\widetilde{\theta })\) such that for any \(\theta \in (0,\theta _{0})\), the following result holds:

$$\begin{aligned} l(t)\ \text{ reaches } \text{ its } \text{ positive } \text{ maximum. } \end{aligned}$$
(36)

From the structure of l(t), we see that there are finite positive solutions of \(l(t)=0\). Assume the positive solutions as follows

$$\begin{aligned} 0<T_{1}<T_{2}<\cdot \cdot \cdot<T_{m}<\infty . \end{aligned}$$

Then we can easily know that

$$\begin{aligned} \begin{aligned}&l(t)<0, t\in (0,T_{1})\cup (T_{2},T_{3})\cup \cdot \cdot \cdot \cup (T_{m},\infty ),\\&l(t)>0, t\in (T_{1},T_{2})\cup (T_{3},T_{4})\cup \cdot \cdot \cdot \cup (T_{m-1},T_{m}). \end{aligned} \end{aligned}$$

Secondly, we denote \((0,T_{1})\cup (T_{2},T_{3})\cup \cdot \cdot \cdot \cup (T_{m},\infty )\) by P and denote \(Q=P\setminus (T_{m},\infty )\). We define a \(C^{\infty }\) function \(\tau : R^{+}\rightarrow [0,1]\) as

$$\begin{aligned} \begin{aligned}&\tau (t)=1,\quad \text{ if }~~t\in Q,\\&\tau (t)=0,\quad \text{ if }~~t\in (T_{m},\infty ). \end{aligned} \end{aligned}$$

Considering the truncated functional as follows:

$$\begin{aligned} \Phi _{\infty }(v)=&\frac{1}{2}\int _{\Omega }|\nabla v|^{2}\text{ d }x-\frac{\theta }{k}\int _{\Omega }|G^{-1}(v)|^{k}\text{ d }x-\frac{1}{2^{*}}\int _{\Omega }\tau (\Vert v\Vert )|G^{-1}(v)|^{2^{*}}\text{ d }x\\&-\lambda \int _{\Omega }\tau (\Vert v\Vert )F(G^{-1}(v))\text{ d }x. \end{aligned}$$

Similar as above, we denote the function

$$\begin{aligned} \overline{l}(t)=\frac{1}{2}t^{2}-C_{9}t^{2^*}\tau (t)-C_{10}\theta t^{k}-C_{11}\lambda t^{p}\tau (t)-C_{12}\lambda t^{q}\tau (t), \end{aligned}$$

and we can find

$$\begin{aligned} \Phi _{\infty }(v)\ge \overline{l}(\Vert v\Vert ). \end{aligned}$$

Finally, we notice that \(\overline{l}(t)\ge l(t)\), if \(t>0\); \(\overline{l}(t)= l(t)\) if \(t\in Q\); \(0\le \overline{l}(t)\), if \(t>T_{m}\). Moreover, due to \(\tau \in C^{\infty }\), we have \(\Phi _{\infty }(v)\in C^{1}(H^{1}_{0}(\Omega ),\mathbb {R})\). Thus, we get the following lemma.

Lemma 2.6

  1. (a).

    If \(\Phi _{\infty }(v)<0\), then \(\Vert v\Vert \in Q\), and \(\Phi (w)=\Phi _{\infty }(w)\) for all w in a small enough neighborhood of v.

  2. (b).

    For all \(\lambda \in (0,\widetilde{\lambda })\), there exists a \(\widetilde{\theta }>0\), such that when \(\theta \in (0,\widetilde{\theta })\) and \(c<0\), \(\Phi _{\infty }(v)\) satisfies the \(\mathrm (PS)_{c}\)-conditions.

We next recall some concepts and properties about \(Z_{2}\) index theory.

Let X be a Banach space and denote

$$\begin{aligned} \Sigma =\{A\subset X\setminus \{0\}:\ A~\text{ is } \text{ closed } \text{ and } \text{ symmetric }\}. \end{aligned}$$

For \(A\in \Sigma \), the \(Z_{2}\) genus of A denoted by

$$\begin{aligned} \gamma (A)=\min \{n\in \mathbb {N}:\ \text{ there } \text{ exists } \text{ an } \text{ odd } \text{ continuous }~\phi :~A\rightarrow \mathbb {R}^{n}\setminus \{0\}\}, \end{aligned}$$

if \(\{\cdots \}=\emptyset \), then \(\gamma (A)=0\), in particular \(0\in A\), then \(\gamma (A)=+\infty .\)

Note that the genus has the following properties (see [19]):

Lemma 2.7

Let \(A,B\in \Sigma \) and \(f\in (X,X)\) an odd map. Then

  1. (1)

    \(\gamma (A)\le \gamma (\overline{f(A)})\);

  2. (2)

    \(A\subset B\Rightarrow \gamma (A)\le \gamma (B)\);

  3. (3)

    \(\gamma (A)<\infty \Rightarrow \gamma (\overline{A-B})\ge \gamma (A)-\gamma (B)\);

  4. (4)

    If \(S^{N-1}\) is the sphere in \(\mathbb {R}^{N}\), then \(\gamma (S^{N-1})=N\);

  5. (5)

    If A is compact, then \(\gamma (A)<\infty \), and there exists \(\delta >0\) such that \(\gamma (A)=\gamma (N_{\delta }(A))\), where \(N_{\delta }(A)=\{x\in X:\ d(x,A)\le \delta \}\).

The last lemma in this section is the well-known deformation lemma (see [27]):

Lemma 2.8

Let X be Banach space. Suppose \(f\in C^{1}(X,\mathbb {R})\) satisfies the (PS)-conditions. If \(c\in \mathbb {R}\), \(\overline{\epsilon }>0\) and N is any neighborhood of \(K_{c}\triangleq \{u\in X:\ f(u)=c, f'(u)=0\}\), there exist \(\eta (t,u)\equiv \eta _{t}(u)\in C([0,1]\times X,X)\) and a number \(\epsilon \in (0,\overline{\epsilon }\), with the properties

  1. (1)

    \(\eta _{t}(u)=u\), if \(t=0\), or \(\forall u\notin f^{-1}[c-\overline{\epsilon },c+\overline{\epsilon }]\);

  2. (2)

    \(f(\eta _{t}(u))\le f(u)\) is nonincreasing in t for \(\forall u\in X\);

  3. (3)

    \(\eta _{1}(f^{c+\epsilon }\setminus N)\subset f^{c-\epsilon }\), where \(f^{c}=\{u\in X: f(u)\le c\}\), \(\forall c\in \mathbb {R}\);

  4. (4)

    if f is even, \(\eta _{t}\) is odd in u.

3 Proof of Theorem 1.1

We now prove Theorem 1.1 via the application of genus. For \(1\le j\le n\), we define

$$\begin{aligned} c_{j}=\inf _{A\in \sum _{j}}\sup _{u\in A}\Phi _{\infty }(v), \end{aligned}$$

with

$$\begin{aligned} \Sigma _{j}=\{A\subset X\setminus \{0\}:\ A~\text{ is } \text{ closed } \text{ in }~X,\ -A=A,\ \gamma (A)\ge j\}. \end{aligned}$$

Set \(K_{c}=\{u\in X:\ \Phi _{\infty }(v)=c, \ \Phi _{\infty }'(v)=0\}\) and \(\lambda \in (0,\widetilde{\lambda })\), \(\theta \in (0,\widetilde{\theta })\) where \(\widetilde{\lambda }\) and \(\widetilde{\theta }\) are given by Lemma 2.6.

Firstly, we claim that if \(j\in \mathbb {N}\), there is an \(\varepsilon _j=\varepsilon (j)>0\), such that \(\gamma (\Phi _{\infty }^{-\varepsilon _j})\ge j,\) where \(\Phi _{\infty }^{-\varepsilon }=\{u\in X:\ \Phi _{\infty }(v)\le -\varepsilon \}\).

Let \(X_{j}\) be a j-dimensional subspace of \(H^{1}_{0}(\Omega )\). Since \(X_{j}\) is a finite-dimension space, all the norms in \(X_{j}\) are equivalent. Therefore, for every \(v\in X_{j}\) with \(\Vert v\Vert =1\) we can define

$$\begin{aligned} \begin{array}{ll} \displaystyle \hspace{1cm} \displaystyle a_{j}=\inf \{\int _{\Omega }|v|^{\frac{k}{\alpha }}\text{ d }x:~v\in X_{j}, \Vert v\Vert =1\}<\infty ,\\ \hspace{1cm} \displaystyle b_{j}=\sup \{\int _{\Omega }|v|^{2^{*}}\text{ d }x:~v\in X_{j}, \Vert v\Vert =1\}<\infty . \end{array} \end{aligned}$$
(37)

On the other hand, for \(0<t<T_{1}\), we notice that

$$\begin{aligned} \begin{aligned} \Phi _{\infty }(tv)&=\Phi (tv)\\&=\frac{1}{2}t^{2}-\frac{\theta }{k}\int _{\Omega }|G^{-1}(tv)|^{k}\text{ d }x-\frac{1}{2^{*}}\int _{\Omega }|G^{-1}(tv)|^{2^{*}}\text{ d }x\\&\quad -\lambda \int _{\Omega }F(G^{-1}(tv))\text{ d }x. \end{aligned} \end{aligned}$$

Then, from Lemma 2.1-(1), there exists \(\delta >0\) such that \(|G^{-1}(t)|\ge \frac{1}{\root k \of {2}}|t|^{\frac{1}{\alpha }}\) for \(|t|\le \delta \), and

$$\begin{aligned} \begin{aligned} \Phi _{\infty }(tv)&\le \frac{1}{2}t^{2}-\frac{\theta }{k}\int _{\Omega }|G^{-1}(tv)|^{k}\text{ d }x-\frac{1}{2^{*}}\int _{\Omega }|G^{-1}(tv)|^{2^{*}}\text{ d }x\\&\le \frac{1}{2}t^{2}-\frac{\theta t^\frac{k}{\alpha }}{2k}\int _{\Omega }|v|^{\frac{k}{\alpha }}\text{ d }x+\frac{\theta t^\frac{k}{\alpha } }{2k}\int _{\{x\in \Omega :\ |tv(x)|\ge \delta \}}|v|^{\frac{k}{\alpha }}\text{ d }x-\frac{1}{2^{*}}\int _{\Omega }|G^{-1}(tv)|^{2^{*}}\text{ d }x\\&\le \frac{1}{2}t^{2}-\frac{\theta \sigma a_j}{2k}t^\frac{k}{\alpha }+\frac{\theta }{2k}|\Omega |\delta ^{\frac{k}{\alpha }-2^{*}}b_{j}t^{2^{*}}. \end{aligned} \end{aligned}$$
(38)

Therefore given \(\theta \in (0,\widetilde{\theta })\), we can choose \(t_{0}\in (0,T_{1})\) sufficiently small such that

$$\begin{aligned} \Phi _{\infty }(t_{0}v)\le -\varepsilon _j<0, \end{aligned}$$

where \(\varepsilon _{j}=-\frac{1}{2}t_{0}^{2}+\frac{\theta \sigma a_j}{k}t_{0}^\frac{k}{\alpha }-\frac{\theta }{2k}|\Omega |\delta ^{\frac{k}{\alpha }-2^{*}}b_{j}t_{0}^{2^{*}}\). Defining \(S_{t_{0}}=\{v\in X:~\Vert v\Vert =t_{0}\}\), then \(S_{t_{0}}\cap X_{j}\subset \Phi _{\infty }^{-\varepsilon _j}\). By Lemma 2.7, we obtain

$$\begin{aligned} \gamma (\Phi ^{-\varepsilon _j}_{\infty })\ge \gamma (S_{t_{0}}\cap X_{j})\ge j. \end{aligned}$$

Since \(\Phi _{\infty }\) is continuous and even, with above claim, we have \(\Phi _{\infty }^{-\varepsilon _{j}}\in \Sigma _{j}\) and \(c_{j}\le -\varepsilon _{j}<0\). As \(\Phi _{\infty }\) is bounded from below, we know that \(c_{j}>-\infty \). Using Lemma 2.5, we deduce \(K_{c}\) is a compact set.

Secondly, we claim that if for some \(j\in \mathbb {N}\), and there exists an \(i\ge 0\) such that \(c=c_{j}=c_{j+1}=\cdots =c_{j+i}\), then \(\gamma (K_{c})\ge i+1\).

Assume by contradiction that if \(\gamma (K_{c})\le i\), then we can choose a closed and symmetric set U with \(K_{c}\subset U\) and \(\gamma (U)\le i\). As \(c<0\), it implies \(U\subset \Phi ^{0}_{\infty }\). By Lemma 2.8, there exists an odd assumption

$$\begin{aligned} \eta : [0,1]\times X\rightarrow X, \end{aligned}$$

such that \(\eta (E^{c+\delta }_{\infty }\backslash U)\subset E^{c-\delta }_{\infty }\) for some \(\delta \in (0,-c)\).

On the other hand, by the assumption of \(c=c_{j+i}\), there exists an \(A\in \Sigma _{j+i}\) such that

$$\begin{aligned} \sup _{u\in A}\Phi _{\infty }(v)<c+\delta . \end{aligned}$$

Hence

$$\begin{aligned} \eta (A\backslash U)\subset \eta (\Phi _{\infty }^{c+\delta }\backslash U)\subset \Phi _{\infty }^{c-\delta }, \end{aligned}$$

this implies

$$\begin{aligned} \sup _{v\in \eta (A\backslash U)}\Phi _{\infty }(v)\le c-\delta . \end{aligned}$$

However, from Lemma 2.7, we have

$$\begin{aligned} \gamma (\overline{\eta (A\backslash U)})\ge \gamma (\overline{A\backslash U})\ge \gamma (A)-\gamma (U)\ge j. \end{aligned}$$

Thus \(\overline{\eta (A\backslash U)}\in \Sigma _{j}\) and

$$\begin{aligned} c=c_{j}\le \sup _{v\in \overline{\eta (A\backslash U)}}\Phi _{\infty }(u)=\sup _{v\in \eta (A\backslash U)}\Phi _{\infty }(u)\le c-\delta . \end{aligned}$$

So the main claim is proved.

Our next goal is to complete the proof of Theorem 1.1. It is well known that \(\Sigma _{j+1}\subset \Sigma _{j}\ \text{ and }\ c_{j}\le c_{j+1}<0\). Suppose all \(c_{j}\)s are distinct. Then we know \(\gamma (K_{c_{j}})\ge 1\), that is, \(c_{j}\)s are distinct negative critical values of \(\Phi _{\infty }\). Suppose for some \(j_{0}\) and \(i\ge 1\),

$$\begin{aligned} c=c_{j_{0}}=c_{j_{0}+1}=\cdots =c_{j_{0}+i}. \end{aligned}$$

Then by the second claim, we obtain

$$\begin{aligned} \gamma (K_{c_{j_{0}}})\ge i+1, \end{aligned}$$

which implies that \(K_{c_{j_{0}}}\) possesses infinitely many distinct elements.

From Lemma 2.6, for \(\Phi _{\infty }(v)<0\), it implies \(\Phi (v)=\Phi _{\infty }(v)\). Hence we complete the proof.

4 Proof of Theorem 1.2

In this section, we prove the existence theorem which can yield one nontrivial solution for (1) without \((f_{4})\).

Proof of Theorem 1.2

For \(v\in H_{0}^{1}(\Omega )\), by (35), we have

$$\begin{aligned} \Phi (v)\ge l(\Vert v\Vert ). \end{aligned}$$

From (36), for \(\lambda \in (0,\lambda _{0})\) and \(\theta \in (0,\theta _{0})\), there exists \(t_{0}\in \mathbb {R}\) such that

$$\begin{aligned} 0<\chi =\inf _{\partial B_{t_{0}}}\Phi , \end{aligned}$$

where \( B_{t_{0}}=\{v\in H_{0}^{1}(\Omega ):\ \Vert v\Vert <t_{0}\}\). In order to find a nontrivial solution of (1), we need the following fact

$$\begin{aligned} -\infty<\inf _{\overline{B}_{t_{0}}}\Phi <0. \end{aligned}$$
(39)

It is straightforward to show that this infimum is finite. To prove \(\inf _{\overline{B}_{t_{0}}}\Phi <0\), we take \(t>0\), \(\varphi \in C^\infty _0(\Omega )\) such that \(\varphi (x)\ge 0\) for all \(x\in \Omega \) with \(\Vert \varphi \Vert =1\). Then

$$\begin{aligned} \begin{aligned} \Phi (t\varphi )&\le \frac{1}{2}t^{2}\Vert \varphi \Vert ^{2}-\frac{\theta }{k}\int _{\Omega }|G^{-1}(t\varphi )|^{k}\text{ d }x-\frac{1}{2^{*}}\int _{\Omega }|G^{-1}(t\varphi )|^{2^{*}}\text{ d }x\\&\le \frac{1}{2}t^{2}\Vert \varphi \Vert ^{2}-\frac{\theta t^\frac{k}{\alpha }}{2k}\int _{\Omega }|\varphi |^{\frac{k}{\alpha }}\text{ d }x+\frac{\theta }{2k}\int _{\{x\in \Omega :|t\varphi (x)|\ge \delta \}}|t\varphi |^{\frac{k}{\alpha }}\text{ d }x\\&\le \frac{1}{2}t^{2}\Vert \varphi \Vert ^{2}-\frac{\theta \sigma }{2k}t^\frac{k}{\alpha }\int _{\Omega }|\varphi |^{\frac{k}{\alpha }}\text{ d }x+\frac{\theta }{2k}|\Omega |\delta ^{\frac{k}{\alpha }-2^{*}}t^{2^{*}}\int _{\Omega }|\varphi |^{2^{*}}\text{ d }x. \end{aligned} \end{aligned}$$
(40)

Since \(\frac{k}{\alpha }<2<2^{*}\),

$$\begin{aligned} \Phi (t\varphi )<0~~\text{ and }~~t\varphi \in \overline{B}_{t_{0}} \end{aligned}$$

are true for \(t\in (0,t_{1})\), where \(t_{1}\in (0,t_{0})\) is small enough.

Set

$$\begin{aligned} D=\inf _{\partial B_{t_{0}}}\Phi -\inf _{\overline{B}_{t_{0}}}\Phi >0. \end{aligned}$$

For \(d\in (0,D)\), invoking the Ekeland variational principle, there must be a \(v_{d}\in \overline{B}_{t_{0}}\), such that

$$\begin{aligned} \Phi (v_{d})\le \inf _{\overline{B}_{t_{0}}}\Phi +d \end{aligned}$$
(41)

and

$$\begin{aligned} \Phi (v_{d})\le \Phi (w)+d\Vert w-v_{d}\Vert ,\ \text{ for } \text{ any }\ w\in \overline{B}_{t_{0}}. \end{aligned}$$
(42)

From (41), we have

$$\begin{aligned} \Phi (v_{d})<\inf _{\partial B_{t_{0}}}E, \end{aligned}$$

which implies

$$\begin{aligned} v_{d}\in B_{t_{0}}. \end{aligned}$$
(43)

Let \(\psi \in H^{1}_{0}(\Omega )\). From (42) and (43), for \(t\in (0,1)\) small enough, we have

$$\begin{aligned} -d t\Vert \psi \Vert \le \Phi (v_{d}+t\psi )-\Phi (v_{d}), \end{aligned}$$

which yields

$$\begin{aligned} -d \Vert \psi \Vert \le \left<\Phi '(v_{d}),\psi \right>. \end{aligned}$$

By replacing \(\psi \) with \(-\psi \), one has

$$\begin{aligned} \Vert \Phi '(v_{d})\Vert \le d. \end{aligned}$$
(44)

Take \(d_{n}=\frac{1}{n}\) and set \(v_{n}= v_{d_{n}}\) for all \(n\ge 1\). Then as \(n\rightarrow \infty \)

$$\begin{aligned} \Phi (v_{n})\rightarrow \inf _{\overline{B}_{t_{0}}}\Phi \quad \text{ and }~~\Phi '(v_{n})\rightarrow 0. \end{aligned}$$

Since \(\Phi \) satisfies \((PS)_{c}\)-conditions(\(c<0\)), we get

$$\begin{aligned} v_{n}\rightarrow \overline{v},\quad \text{ in }~~H^{1}_{0}(\Omega ). \end{aligned}$$

So

$$\begin{aligned} \Phi (\overline{v})=\inf _{\overline{B}_{t_{0}}}\Phi <0 \end{aligned}$$

is true from (39). We complete the proof. \(\square \)

5 Proof of Theorem 1.3

For fixed \(\delta >0\) in \((h_1)\), we now let \(d(t)\in C(\mathbb {R},\mathbb {R})\) be a cut-off function satisfying:

$$\begin{aligned} d(t)=\left\{ \begin{array}{ll} 1,\ \text{ if }\ |t|\le \frac{\delta }{2},\\ 0,\ \text{ if }\ |t|\ge \delta , \end{array} \right. \end{aligned}$$

\(d(-t)=d(t)\) and \(0\le d(t)\le 1\) for \(t\in \mathbb {R}\). Denote

$$\begin{aligned} \widetilde{h}(x,t):=d(t)h(x,t),\ \text{ for } \text{ all }\ x\in \Omega ,\ t\in \mathbb {R} \end{aligned}$$

and

$$\begin{aligned} \widetilde{H}(x,t)=\int _{\Omega }\widetilde{h}(x,s)\text{ d }s. \end{aligned}$$

Inspired by [8], we consider the following quasilinear Schrödinger equation:

$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle -\text{ div }(g^{2}(u)\nabla u)+g(u)g'(u)|\nabla u|^{2}=\widetilde{h}(x,u),\hspace{1cm} x\in \Omega ,\\ \hspace{2.2in}u=\,0, \hspace{1.9cm} x\in \partial \Omega . \end{array} \right. \end{aligned}$$
(45)

Note that (45) is the Euler-Lagrange equation associated to the natural energy functional

$$\begin{aligned} \widetilde{I}(u)=\frac{1}{2}\int _{\Omega }g^{2}(u)|\nabla u|^{2}\text{ d }x-\int _{\Omega }\widetilde{H}(x,u)\text{ d }x. \end{aligned}$$

As in Sect. 2, taking the change variable

$$\begin{aligned} v=G(u)=\int _{0}^{u}g(s)\text{ d }s, \end{aligned}$$

we observe that the functional \(\widetilde{I}(u)\) can be written of the following way

$$\begin{aligned} J(v)=\frac{1}{2}\int _{\Omega }|\nabla v|^{2}\text{ d }x-\int _{\Omega }\widetilde{H}(x,G^{-1}(v))\text{ d }x. \end{aligned}$$
(46)

We remark that the critical points of J(v) with \(L^{\infty }\)-norm not more than \(\frac{\delta }{2}\) are also weak solutions of (4). To prove this, we need to introduce the following variant Clark’s theorem due to [17] to prove that (46) possesses a sequence of critical points.

Proposition 5.1

Let X be a Banach space, \(J\in C^{1}(X,\mathbb {R})\) is an even functional with \(J(0)=0\). Assume that J satisfies

  1.  (a)

    J is bounded from below and satisfies (PS) condition;

  2.  (b)

    for each \(l\in \mathbb {N}\), there exists a l-dimensional subspace \(X^{l}\) of \(H^{1}_{0}(\Omega )\) and \(\rho _l>0\) such that

$$\begin{aligned} \sup \limits _{X^{l}\cap S_{\rho _l}}J<0,\ \text{ where }\ S_{\rho }=\{v\in H^{1}_{0}(\Omega )|\ \Vert v\Vert =\rho \}. \end{aligned}$$

Then at least one of the following conclusions holds.

  1.  (i)

    There exists a critical point sequence \(\{v_k\}\) such that \(J(v_k)<0\) and \(v_k\rightarrow 0\) in X;

  2.  (ii)

    There exists \(r>0\) such that for any \(0<a<r\) there exists a critical point v such that \(\Vert v\Vert =a\) and \(J(v)=0\).

Next two lemmas ensure that J(v) satisfies all assumptions of Proposition 5.1.

Lemma 5.1

Assume \((h_1)\) hold. Then the functional J is bounded from below and satisfies (PS) condition.

Proof

To prove the functional J is bounded from below, it suffices to show that J is coercive. For \(v\in H^{1}_{0}(\Omega )\), by \((h_1)\) and Lemma 2.2-(iii)

$$\begin{aligned} \begin{aligned} \left| \int _{\Omega }\widetilde{H}(x,G^{-1}(v))\text{ d }x\right|&\le C\int _{\Omega }|G^{-1}(v)|^{k}\text{ d }x\\&\le C\int _{\Omega }|v|^{k}\text{ d }x\\&\le C\Vert v\Vert ^{k}. \end{aligned} \end{aligned}$$

Thus,

$$\begin{aligned} \begin{aligned} J(v)&=\frac{1}{2}\int _{\Omega }|\nabla v|^{2}\text{ d }x-\int _{\Omega }\widetilde{H}(x,G^{-1}(v))\text{ d }x\\&\ge \frac{1}{2}\Vert v\Vert ^{2}-C\Vert v\Vert ^{k} \end{aligned} \end{aligned}$$

and then the functional J is coercive.

Next, we show that J satisfies (PS) condition. Let \(\{v_n\}\) be a (PS) sequence, that is

$$\begin{aligned} |J(v_n)|\le C\ \text{ and }\ J'(v_n)\rightarrow 0. \end{aligned}$$

By the functional J is coercive, this implies the sequence \(\{v_n\}\) is bounded in \(H^{1}_{0}(\Omega )\). Assume without loss of generality that

$$\begin{aligned} \begin{aligned} v_{n}\rightharpoonup v\ \text{ in }\ H^{1}_{0}(\Omega )\ \text{ and }\ v_{n}\rightarrow v\ \text{ in }\ L^{k}(\Omega ), k\in (1,2). \end{aligned} \end{aligned}$$

Consider

$$\begin{aligned} \begin{aligned} o(\Vert v_n-v\Vert )&=\langle J'(v_n)-J'(v),v_n-v\rangle \\&=\int _{\Omega }|\nabla (v_n-v)|^{2}\text{ d }x-\int _{\Omega }\left( \frac{\widetilde{h}(x,G^{-1}(v_n))}{g(G^{-1}(v_n))}-\frac{\widetilde{h}(x,G^{-1}(v))}{g(G^{-1}(v))}\right) (v_n-v)\text{ d }x\\&\ge \Vert v_n-v\Vert ^{2}-C\int _{\Omega }\left( \frac{|G^{-1}(v_n)|^{k-1}}{g(G^{-1}(v_n))}+\frac{|G^{-1}(v)|^{k-1}}{g(G^{-1}(v))}\right) (v_n-v)\text{ d }x\\&\ge \Vert v_n-v\Vert ^{2}-C\int _{\Omega }\left( |G^{-1}(v_n)|^{k-1}+|G^{-1}(v)|^{k-1}\right) (v_n-v)\text{ d }x\\&\ge \Vert v_n-v\Vert ^{2}-C\int _{\Omega }\left( |v_n|^{k-1}+|v|^{k-1}\right) (v_n-v)\text{ d }x\\&\ge \Vert v_n-v\Vert ^{2}-C\Vert v_n-v\Vert _{k}. \end{aligned} \end{aligned}$$

Thus, \(v_n\rightarrow v\) in \(H^{1}_{0}(\Omega )\) and the (PS) condition holds for J. \(\square \)

Lemma 5.2

Assume that \((h_1)\) and \((h_{2})\) hold. Then for each \(l\in \mathbb {N}\), there exists a l-dimensional subspace \(X^{l}\) of \(H^{1}_{0}(\Omega )\) and \(\rho _l>0\), such that

$$\begin{aligned} \sup \limits _{X^{l}\cap S_{\rho _l}}J<0, \end{aligned}$$

where \(S_{\rho }=\{v\in H^{1}_{0}(\Omega ):\ \Vert v\Vert =\rho \}\).

Proof

For any \(M>0\), by \((h_2)\), there exists a constant \(a=a(M)>0\) such that if \(G^{-1}(v)\in C^{\infty }_{0}(B_{r}(x_0))\) and \(\Vert G^{-1}(v)\Vert _{\infty }<a\),

$$\begin{aligned} \widetilde{H}(x,G^{-1}(v))\ge M|G^{-1}(v)|^{2\alpha }. \end{aligned}$$
(47)

On the other hand, from Lemma 2.1-(i), there exist \(b>0\) and a constant \(C>0\) such that

$$\begin{aligned} |G^{-1}(t)|\ge C|t|^{\frac{1}{\alpha }}\ \text{ for }\ |t|\le b. \end{aligned}$$
(48)

For any \(l\in \mathbb {N}\), if \(X^{l}\) is a l-dimensional subspace of \(C^{\infty }_{0}(B_{r}(x_0))\) and \(\rho _l>0\) is sufficiently small and for all \(v\in X^{l}\cap S_{\rho _l}\), we have

$$\begin{aligned} \begin{aligned} J(v)&=\frac{1}{2}\int _{\Omega }|\nabla v|^{2}\text{ d }x-\int _{\Omega }\widetilde{H}(x,G^{-1}(v))\text{ d }x\\&\le \frac{1}{2}\rho _{l}^{2}-M\int _{\Omega }|G^{-1}(v)|^{2\alpha }\text{ d }x\\&\le \frac{1}{2}\rho _{l}^{2}-CM\int _{\Omega }|v|^{2}\text{ d }x\\&\le \left( \frac{1}{2}-CMC_{l}\right) \rho _{l}^{2}, \end{aligned} \end{aligned}$$

where we use (47), (48) and the fact that \(X_{l}\) is a finite-dimension space, all the norms in \(X_{l}\) are equivalent. Hence, from the arbitrary of M, we complete the proof. \(\square \)

We recall that the critical points of (46) with \(L^{\infty }\)-norm not more than \(\frac{\delta }{2}\) are also weak solutions of problem (4). So next step we shall study the \(L^{\infty }\) estimates of the critical points of J.

Lemma 5.3

If \(\{v_i\}\subset H^{1}_{0}(\Omega )\) is a critical point sequence of J satisfying \(v_i\rightarrow 0\) in \(H^{1}_{0}(\Omega )\), then \(v_i\rightarrow 0\) in \(L^{\infty }(\mathbb {R}^N)\).

Proof

Let \(v\in H^{1}_{0}(\Omega )\) be a weak solution of \(-\Delta v=\frac{\widetilde{h}(x,G^{-1}(v))}{g(G^{-1}(v))}\), i.e.,

$$\begin{aligned} \int _{\Omega }\nabla v\nabla \varphi \text{ d }x=\int _{\Omega }\frac{\widetilde{h}(x,G^{-1}(v))}{g(G^{-1}(v))}\varphi \text{ d }x,\ \text{ for } \text{ all }\ \varphi \in H^{1}_{0}(\Omega ). \end{aligned}$$
(49)

Let \(T>0\), and define

$$\begin{aligned} v_{T}=\left\{ \begin{array}{ll} v,\ \text{ if }\ 0< |v|\le T,\\ T,\ \text{ if }\ |v|\ge T. \end{array} \right. \end{aligned}$$

Choosing \(\varphi =|v_{T}|^{2(\eta -1)}v_{T}\) in (49), where \(\eta >1\), we get

$$\begin{aligned} \begin{aligned} \int _{\Omega }\nabla v\cdot \nabla (v_{T}|v_{T}|^{2(\eta -1)})\text{ d }x =\int _{\Omega }\frac{\widetilde{h}(x,G^{-1}(v))}{g(G^{-1}(v))}|v_{T}|^{2(\eta -1)}v_{T}\text{ d }x. \end{aligned} \end{aligned}$$

From Lemma 2.1, we obtain

$$\begin{aligned} \begin{aligned} \frac{1}{\eta ^2}\int _{\Omega }|\nabla |v_T|^{\eta }|^{2}\text{ d }x\le C\int _{\Omega }|v|^{2\eta +k-2}\text{ d }x. \end{aligned} \end{aligned}$$
(50)

On the other hand, using the Sobolev inequality, we have

$$\begin{aligned} \begin{aligned} \frac{S}{4\eta ^{2}}\Vert v_{T}\Vert ^{2\eta }_{L^{\eta 2^{*}}}\le C\Vert v\Vert ^{2\eta +k-2}_{{2\eta +k-2}}, \end{aligned} \end{aligned}$$

where we used that \(S=\inf \{\int _{\Omega }|\nabla v|^{2}\text{ d }x|\ \int _{\Omega }|v|^{2^{*}}\text{ d }x=1\}\). From the Fatou’s lemma, sending \(T\rightarrow \infty \), it follows that

$$\begin{aligned} \Vert v\Vert _{L^{\eta 2^{*}}}\le \left( C\eta \right) ^{1/\eta }\Vert v\Vert ^{(2\eta +k-2)/2\eta }_{{2\eta +k-2}}. \end{aligned}$$
(51)

Let us define \(\eta _{i}=\frac{2^{*}\eta _{i-1}+2-k}{2}\) where \(i=1,2,...\) and \(\eta _{0}=\frac{2^{*}+2-k}{2}\). Since

$$\begin{aligned} \left( 2C\eta _i\right) ^{(2\eta _j+k-2)/(2\eta _j)}\le 2C\eta _i\ \text{ for }\ i<j,\ \text{ where }\ C>1, \end{aligned}$$

by Moser’s iteration method we have

$$\begin{aligned} \Vert v\Vert _{L^{2\eta _{i+1}+k-2}}\le \exp \left( \sum _{m=0}^{i}\frac{\text{ ln }(2C\eta _m)}{\eta _m}\right) \Vert v\Vert ^{\mu _i}_{L^{2^{*}}}, \end{aligned}$$

where \(\mu _i=\prod _{m=0}^{i}\frac{2\eta _m+k-2}{2\eta _{m}}\). Letting \(i\rightarrow \infty \), we obtain that

$$\begin{aligned} \Vert v\Vert _{L^{\infty }}\le \exp \left( \sum _{m=0}^{\infty }\frac{\text{ ln }(2C\eta _m)}{\eta _m}\right) \Vert v\Vert ^{\mu }_{L^{2^{*}}}, \end{aligned}$$

where \(\mu =\prod _{m=0}^{\infty }\frac{2\eta _m+k-2}{2\eta _{m}}\) is a number in (0, 1) and \(\exp \left( \sum _{m=0}^{k}\frac{\text{ ln }(2C\eta _m)}{\eta _m}\right) \) is a positive constant. This together with the Sobolev inequality shows that if \(\{v_i\}\) is a critical point sequence of J satisfying \(v_i\rightarrow 0\) in \(H^{1}_{0}(\Omega )\) as \(i\rightarrow \infty \), then \(v_i\rightarrow 0\) in \(L^{\infty }(\Omega )\). This ends the proof. \(\square \)

Proof of Theorem 1.3

Obviously, the functional J is an even functional with \(J(0)=0\). Besides, by Lemma 5.1 and Lemma 5.2, all conditions of Proposition 5.1 are satisfied. Thus J has a sequence of critical points \(\{v_n\}\) with \(J(v_n)\rightarrow 0\) and \(v_n\rightarrow 0\) in \(H^{1}_{0}(\Omega )\). By virtue of Lemma 5.3, we know that \(\{v_n\}\) is a sequence of critical points for (46) with \(v_n\rightarrow 0\) in \(L^{\infty }(\Omega )\). Letting \(u_n=G^{-1}(v_n)\), there exists \(n_0\in \mathbb {N}\) such that \(\{u_n\}\) is a sequence of weak solutions of (4) for each \(n\ge n_0\). The proof is completed. \(\square \)