1 Introduction and main results

In this paper, we investigate the existence of high-energy solutions and sign-changing solutions to the following Kirchhoff type elliptic equation

$$\begin{aligned} \left\{ \begin{array}{ll} -(a+b\int _{\Omega }|\nabla u|^{2})\triangle u = \mu g(x,u)+f(x,u), &{}\quad \hbox {in}\ \ \Omega ,\\ u=0, &{}\quad \hbox {on}\ \ \partial \Omega ,\end{array}\right. \end{aligned}$$
(1.1)

where \(\Omega \subset \mathbb {R}^{3}\) is a bounded domain with smooth boundary \(\partial \Omega \) and \(a>0\), \(b\ge 0\). The nonlinearity \(\mu g(x,u)+f(x,u)\) may involve a combination of concave and convex terms. When \(a\equiv 1\), and \(b\equiv \mu \equiv 0\), the problem (1.1) turns out to be the following elliptic equation:

$$\begin{aligned} \left\{ \begin{array}{ll} -\triangle u=f(x,u), &{}\quad \hbox {in}\ \ \Omega ,\\ u=0,&{}\quad \hbox {on}\ \ \partial \Omega .\end{array}\right. \end{aligned}$$
(1.2)

Note that Kirchhoff type problem on a smooth-bounded domain \(\Omega \subset \mathbb {R}^{3}\) takes the form

$$\begin{aligned} \left\{ \begin{array}{ll} -(a+b\int _{\Omega }|\nabla u|^{2})\triangle u = f(x,u), &{}\quad \hbox {in}\ \ \Omega ,\\ u=0, &{}\quad \hbox {on}\ \ \partial \Omega ,\end{array}\right. \end{aligned}$$
(1.3)

and has been studied extensively. Indeed, such a class of problems is viewed as being nonlocal because of the presence of the term \((\int _{\Omega }|\nabla u|^{2})\triangle u\), which implies that the equation in (1.3) is no longer a pointwise identity and is very different from classical elliptic equations. That is to say, such a phenomenon provokes some mathematical difficulties, which makes the study of such a class of problems particularly interesting. On the other hand, problem (1.3) has its physical motivation. Moreover, the equation of (1.3) related to the following stationary analogue equation:

(1.4)

where u denotes the displacement, f(xu) the external force and b the initial tension while a is related to the intrinsic properties of the string (such as Young’s modulus). Equation of type (1.4) was first proposed by Kirchhoff [15] and began to attract researchers’ attention mainly after the work of Lions [16]. Some interesting results can be found, for example, in [1, 5, 7, 9, 10, 14, 21, 22, 24,25,26,27]. Similar nonlocal problems also appear in several fields as biological systems (for example, population density, see [2, 8]).

In the last two decades, there are a large number of papers devoted to problems like (1.3) on the existence of positive solutions, radial and non-radial solutions, ground states, and sign-changing (see, e.g., [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23]). Recently, in almost all the results concerning equation (1.3), the nonlinear term f is assumed to be superlinear or sublinear, and little has been done for the combination of concave and convex terms. It is the first purpose of our paper to investigate the Kirchhoff type problems with concave and convex nonlinearities.

Another topic which has increasingly received interest in recent years is the existence of sign-changing solutions of problems like (1.3). Mao and Zhang [23] considered the existence of sign-changing and multiple solutions of problems like (1.3) without the (PS) condition. Deng et al. [11] studied the following Kirchhoff problem:

$$\begin{aligned} -(a+b\int _{\mathbb {R}^{3}}|\nabla u|^{2})\triangle u +V(|x|)u=f(|x|,u),\ \ \ u\in H^{1}(\mathbb {R}^{3}) \end{aligned}$$
(1.5)

and they proved that, for any positive integer k, the problem has a sign-changing solution \(u^{b}_{k}\). In [13], Figueiredo et al. studied the existence of a nodal solution with minimal energy of (1.3). Using constrained minimization on the sign-changing Nehari manifold, Ye in [31] proved the following nonlinear Kirchhoff equation:

$$\begin{aligned} -(a+b\int _{\mathbb {R}^{N}}|\nabla u|^{2})\triangle u +V(x)u=f(u),\ \ \ u\in H^{1}(\mathbb {R}^{N}) \end{aligned}$$
(1.6)

has a least energy nodal solution. In [28] W. S studied the existence of least energy sign-changing solutions for a class of Kirchhoff-type problem in bounded domains using quantitative deformation lemma and degree theory.

Motivated by and different from above-mentioned papers, we expect to use the method of invariant sets of descending flow to prove the existence of sign-changing solutions. Mao and Luan [24] obtained existence of signed and sign-changing solutions with asymptotically three-linear-bounded nonlinearity via variational methods and invariant sets of descent flow. Batkam [4] established a new sign-changing version of the symmetric Mountain Pass theorem and then applied it to prove the existence of a sequence of sign-changing solutions to (1.3) with higher energy. Note that the nonlinear term f in [4] is superlinear. Different from the works in the literature, the second aim of our paper is to study the problem (1.1) with more general nonlinearity involving concave and convex nonlinearities.

Before stating our main results, we introduce some conditions for \(f,g\in C(\overline{\Omega }\times \mathbb {R},\mathbb {R})\).

\((g_{1})\) There exist constants \(1<q_{1}<q_{2}<\cdots<q_{m}<2\) and functions \(h_{i}(x)\in L^{\frac{2}{2-q_{i}}}(\overline{\Omega },\mathbb {R}^{+})(i=1,\ldots ,m)\) such that

$$\begin{aligned} |g(x,u)|\le \displaystyle \sum ^{m}_{i=1} h_{i}(x)|u|^{q_{i}-1},\ \ \forall x\in \Omega , \ \ u\in \mathbb {R}. \end{aligned}$$

\((g_{2})\) There exists \(\nu \in (1,2)\) such that

$$\begin{aligned}0<ug(x,u)\le \nu G(x,u),\ \ \forall x\in \Omega , \ \ u\in \mathbb {R},\end{aligned}$$

where \(G(x,u)=\int ^{u}_{0}g(x,t)\mathrm{d}t\).

\((f_{1})\) There exist \(C_{1}>0\) and \(2<p<6\) such that

$$\begin{aligned}|f(x,u)|\le C_{1}(1+|u|^{p-1}),\ \ \forall x\in \Omega , \ \ u\in \mathbb {R}.\end{aligned}$$

\((f_{2})\) \(\lim _{|u|\rightarrow 0}\frac{f(x,u)}{u}=0 \) uniformly in \(\Omega \).

\((f_{3})\) There exists \(\theta >4\) such that

$$\begin{aligned}0<\theta F(x,u)\le uf(x,u), \ \ \forall x\in \Omega ,\ \ u\in \mathbb {R},\end{aligned}$$

where \(F(x,u)=\int ^{u}_{0}f(x,t)\mathrm{d}t\).

\((f_{4})\) There exists \( R>0\) such that

$$\begin{aligned} \inf _{x\in \Omega ,|u|\ge R}F(x,u):=\beta >0.\end{aligned}$$

Our main results read as follows.

Theorem 1.1

Let \(a>0\), \(b\ge 0\). Assume \((g_{1}),(g_{2})\) and \((f_{1})-(f_{4})\) hold. Then there exists \(\tilde{\mu }>0\) such that for \( \mu \in [-\tilde{\mu },\tilde{\mu }]\), problem (1.1) has at least one nontrivial solutions.

Theorem 1.2

Let \(a>0\), \(b\ge 0\). Assume \((g_{1}),(g_{2})\) and \((f_{1}){\text {--}}(f_{4})\) hold and fg is odd in u. Then there exists \(\tilde{\mu }>0\) such that for \( \mu \in [-\tilde{\mu },\tilde{\mu }]\), problem (1.1) has a sequence of solutions \((u_{k})\) with \(I_{\mu }(u_{k})\rightarrow +\infty \) as \(k\rightarrow \infty \).

Remark 1.3

In general, one uses the Mountain Pass theorem to get the \((PS)_{c}\) sequences. However, it is difficult to obtain a \((PS)_{c}\) sequence by this method in the case of sublinear. Moreover, it is hard in the situation of concave and convex nonlinearities. Thus, we adopt a new technique to prove the existence of a Mountain Pass geometry in the case of concave and convex nonlinearities.

The combined effect of concave and convex nonlinearities was first investigated by Ambrosetti et al. [3] on the following elliptic equation:

$$\begin{aligned}\left\{ \begin{aligned}&-\triangle u= \mu |u|^{q-2}u+\lambda |u|^{p-2}u \\&u\in H^{1}_{0}(\Omega ) \end{aligned}\right. \end{aligned}$$
(1.4)

with \(\Omega \subset \mathbb {R}^{N}\) a bounded domain, \(1<q<2<p<2^{*}\), \(2^{*}=\frac{2N}{N-2}\). They proved the existence of infinitely many solutions with negative energy for \(0<\mu \ll \lambda =1\). Bartsch and Willem extended the results in [29], they proved the existence of infinitely many solutions with high energy for \(\lambda >0,\mu \in \mathbb {R}\) and negative energy for \(\mu >0,\lambda \in \mathbb {R}\) using Fountain theorem and Dual Fountain theorem, respectively. Li et al. also considered an elliptic equation with concave and convex nonlinearities in [19]. In [12], Ding and Liu studied multiplicity of periodic solutions of a Dirac equation with concave and convex nonlinearities.

Theorem 1.4

Let \(a>0\), \(b\ge 0\). Assume \((g_{1}),(g_{2})\) and \((f_{1}){\text {--}}(f_{4})\) hold. Then there exists \(\mu _{0}>0\) such that for \(\mu \in (0,\mu _{0}]\), problem (1.1) has at least one sign-changing solutions.

Remark 1.5

The class of nonlinearities \(\mu g(x,u)+f(x,u)\) satisfying the assumptions of Theorem 1.4 includes the nonlinearity \(\mu |u|^{q-2}u+|u|^{p-2}u\) with \(1<q<2\), \(4<p<6\). Even in this special case, Theorem 1.4 seems to be the first attempt in finding sign-changing solutions to (1.1).

The method of invariant sets of descending flow plays an important role in the study of sign-changing solutions of elliptic problems. We refer to [17, 23, 32] and the references therein. Since the pseudo-gradient flow must be constructed in such a way that keeps the positive and negative cones invariant. However, this invariance property makes the construction of the flow very complicated because of the nonlocal term \((\int _{\Omega }|\nabla u|^{2})\triangle u\). Thus, to construct the pseudo-gradient flows using an auxiliary operator is more important.

The idea of the proofs of Theorems 1.4 is to use suitable minimax arguments in the presence of invariant sets of a descending flow for the variational formulation. In particular, we make use of an abstract critical point theory developed by Wang et al. [17] on the nonlinear Schrödinger systems and by Liu et al. [18] on the Schrödinger–Poisson system

$$\begin{aligned} \left\{ \begin{aligned}&-\triangle u + V(x)u + \phi u= f(u),&\quad \hbox {in}\ \ \mathbb {R}^{3},\\&-\triangle \phi = u^{2},&\quad \hbox {in}\ \ \mathbb {R}^{3},\end{aligned}\right. \end{aligned}$$
(1.7)

where the pseudo-gradient flows were constructed using an auxiliary operator.

This paper is organized as follows. In Sect. 2, we state the variational framework of our problem and some preliminary setting. Section 3 is devoted to the proof of Theorems 1.1 and 1.2. Finally, the proof of Theorem 1.4 is presented in Sect. 4.

2 Preliminaries and functional setting

Throughout this paper we denote by \(\rightarrow \) (resp.\(\rightharpoonup \)) the strong (resp.weak) convergence. We use \(C_i\) to denote various positive constants which may vary from lines to lines and are not essential to the problem.

We make use of the following notations, let \(H:=H_{0}^{1}(\Omega )\) with the norm and the inner product

$$\begin{aligned} \Vert u\Vert :=\left( \int _{\Omega }|\nabla u|^{2}\right) ^{\frac{1}{2}},\ \ (u,v)=\int _{\Omega }(\nabla u\nabla v). \end{aligned}$$

As usual, for \(1\le p<+\infty \), \( |u|_{p}:=\left( \int _{\Omega }|u|^{p}\right) ^{\frac{1}{p}}, \ \ u\in L^{p}(\Omega ).\)

Lemma 2.1

[29, Theorem 1.9]. If \(|\Omega |<\infty \), the following embeddings are compact:

$$\begin{aligned} H_{0}^{1}(\Omega )\hookrightarrow L^{p}(\Omega ),\ \ \ 1\le p<2^{*}, \end{aligned}$$

and so there exists \(\eta _{s}\) such that

$$\begin{aligned} |u|_{s}\le \eta _{s}\Vert u\Vert ,\ \ \forall u\in H_{0}^{1}(\Omega ). \end{aligned}$$
(2.1)

Next we define the energy functional \(I_{\mu }\) on H by

$$\begin{aligned} I_{\mu }(u)=\frac{a}{2}\int _{\Omega }|\nabla u|^{2}+\frac{b}{4}\left( \int _{\Omega }|\nabla u|^{2}\right) ^{2}- \mu \int _{\Omega }G(x,u)-\int _{\Omega }F(x,u) \end{aligned}$$
(2.2)

under the assumptions\((g_{1})\) and \((f_{1})\), \(I_{\mu }\in C^{1}(H,\mathbb {R})\) and

$$\begin{aligned} (I_{\mu }^{'}(u),v)=\int _{\Omega }(\nabla u\nabla v)+\int _{\Omega }|\nabla u|^{2}\nabla u\nabla v-\mu \int _{\Omega }g(x,u)v-\int _{\Omega }f(x,u)v.\nonumber \\ \end{aligned}$$
(2.3)

It is easy to verify that \(u\in H \) is a solution of system (1.1) if and only if \(u\in H\) is a critical point of \(I_{\mu }\).

Let X be a Banach space with the norm \(\Vert \cdot \Vert \) and \(X=\overline{\bigoplus ^{\infty }_{i=1}X_{i}}\) with \(\dim X_{i}<+\infty \) for each \(i\in \mathbb {N}\). Further, we set

$$\begin{aligned} Y_{k}=\bigoplus ^{k}_{i=1}X_{i},\ \ Z_{k}=\overline{\bigoplus ^{\infty }_{i=k}X_{i}}. \end{aligned}$$

Definition 2.2

[29].  Let X be a Banach space, \(I\in C^{1}(X,\mathbb {R})\), and \(c\in \mathbb {R}\). The function I satisfies the \((PS)_{c}\) condition if any sequence \(\{u_{n}\}\subset X\) such that

$$\begin{aligned}I(u_{n})\rightarrow c,\ \ I^{'}(u_{n})\rightarrow 0\ \ \hbox {as}\ \ n\rightarrow \infty \end{aligned}$$

has a convergent subsequence.

Lemma 2.3

(Mountain Pass theorem [29]). Let X be a Banach space, \(I\in C^{1}(X,\mathbb {R})\), \(e\in X\) and \(r>0\) such that \(\Vert e\Vert >r\) and

$$\begin{aligned} \inf _{\Vert u\Vert =r}I(u)>I(0)\ge I(e).\end{aligned}$$

If I satisfies the \((PS)_{c}\) condition with

$$\begin{aligned} c:= & {} \inf _{\gamma \in \Gamma } \max _{t \in [0,1]}I(\gamma (t)),\\ \Gamma:= & {} \{\gamma \in \mathcal {C}([0,1],X):\gamma (0)=0,\gamma (1)=e\}. \end{aligned}$$

then c is a critical value of I.

Lemma 2.4

[29].  If \(1\le p<2^{*}\) then we have that

$$\begin{aligned}\beta _{k}:=\sup _{u\in Z_{k},\Vert u\Vert =1}|u|_{p}\rightarrow 0\quad \hbox {as} \ \ k\rightarrow \infty .\end{aligned}$$

To give the proof of our main result, we need the following critical point theorem.

Lemma 2.5

(Fountain theorem, Bartsch, 1992 [29]). Let \(I\in C^{1}(X,\mathbb {R})\) satisfy \(I(-u)=I(u)\). Assume that, for every \(k\in \mathbb {N}\), there exists \(\rho _{k}>\gamma _{k}>0\) such that

\((A_{1})\ \) \(a_{k}:=\max _{u\in Y_{k},\Vert u\Vert =\rho _{k}} I(u)\le 0\),

\((A_{2})\ \) \(b_{k}:=\inf _{u\in Z_{k},\Vert u\Vert =\gamma _{k}} I(u)\rightarrow +\infty \) as \( k\rightarrow +\infty \),

\((A_{3})\ \) I satisfies the \((PS)_{c}\) condition for every \(c>0\),

then I has an unbounded sequence of critical values.

3 Proof of Theorems 1.1 and 1.2

3.1 Proof of Theorem 1.1

Lemma 3.1

If \((g_{1})\), \((f_{1})-(f_{4})\) hold, then there exist \(e\in H\) and \(r>0\) such that \(\inf _{\Vert u\Vert =r}I_{\mu }(u)>0\) and \(I_{\mu }(e)<0\) with \(\Vert e\Vert >r\).

Proof

Since \(1<q_{1}<q_{2}<\cdots<q_{m}<2\), \((g_{1})\) together with (2.1) and Hölder’s inequality imply that, for \(\Vert u\Vert \) large enough,

$$\begin{aligned} \begin{aligned}\int _{\Omega }|G(x,u)|&\le \int _{\Omega }\displaystyle \sum ^{m}_{i=1}\frac{1}{q_{i}}|h_{i}(x)||u|^{q_{i}}\\&= \displaystyle \sum ^{m}_{i=1}\int _{\Omega }\frac{1}{q_{i}}|h_{i}(x)||u|^{q_{i}}\\&\le \displaystyle \sum ^{m}_{i=1}\frac{1}{q_{i}}|h_{i}(x)|_{\frac{2}{2-q_{i}}}|u|^{q_{i}}_{2}\\&\le \displaystyle \sum ^{m}_{i=1}\frac{1}{q_{i}}|h_{i}(x)|_{\frac{2}{2-q_{i}}}\eta ^{q_{i}}_{2}\Vert u\Vert ^{q_{i}}\\&\le C_{3}\Vert u\Vert ^{q_{m}}.\end{aligned} \end{aligned}$$
(3.1)

\((f_{1})\) and \((f_{2})\) imply that for any \(\delta >0 \) there exists \( C_\delta >0\) such that

$$\begin{aligned} |f(x,u)|\le \delta |u|+C_{\delta }|u|^{p-1}, \quad \forall u\in \mathbb {R},\quad x\in \Omega , \end{aligned}$$
(3.2)

and then

$$\begin{aligned} |F(x,u)|\le \frac{\delta }{2} |u|^{2}+\frac{C_{\delta }}{p}|u|^{p},\quad \forall u\in \mathbb {R},\quad x\in \Omega . \end{aligned}$$
(3.3)

It follows from (3.1) and (3.3) that

$$\begin{aligned} \begin{aligned} I_{\mu }(u)&=\frac{a}{2}\int _{\Omega }|\nabla u|^{2}+\frac{b}{4}\left( \int _{\Omega }|\nabla u|^{2}\right) ^{2}- \mu \int _{\Omega }G(x,u)-\int _{\Omega }F(x,u)\\&=\frac{a}{2}\Vert u\Vert ^{2}+\frac{b}{4}\Vert u\Vert ^{4}-\mu \int _{\Omega }G(x,u)-\int _{\Omega }F(x,u)\\&\ge \frac{a}{2}\Vert u\Vert ^{2}-|\mu |\int _{\Omega }G(x,u)-\int _{\Omega }F(x,u)\\&\ge \frac{a}{2}\Vert u\Vert ^{2}-|\mu |C_{3}\Vert u\Vert ^{q_{m}}-\int _{\Omega }\left( \frac{\delta }{2} |u|^{2}+\frac{C_{\delta }}{p}|u|^{p}\right) \\&\ge \frac{a}{2}\Vert u\Vert ^{2}-|\mu |C_{3}\Vert u\Vert ^{q_{m}}-\frac{\delta }{2} \eta ^{2}_{2}\Vert u\Vert ^{2}-\frac{C_{\delta }}{p}\eta ^{p}_{p}\Vert u\Vert ^{p}\\&=\Vert u\Vert ^{2}\left( \frac{a}{2}-\frac{\delta }{2}\eta ^{2}_{2}-|\mu |C_{3}\Vert u\Vert ^{q_{m}-2}- \frac{C_{\delta }}{p}\eta ^{p}_{p}\Vert u\Vert ^{p-2}\right) . \end{aligned} \end{aligned}$$

For \(\delta \le \frac{2a-1}{2\eta ^{2}_{2}}\) satisfying \(\frac{a}{2}-\frac{\delta }{2}\eta ^{2}_{2}\ge \frac{1}{4}\), we have

$$\begin{aligned} I_{\mu }(u)\ge \Vert u\Vert ^{2}\left( \frac{1}{4}- |\mu |C_{3}\Vert u\Vert ^{q_{m}-2}-\frac{C_{\delta }}{p}\eta ^{p}_{p}\Vert u\Vert ^{p-2}\right) . \end{aligned}$$
(3.4)

Let

$$\begin{aligned} k(s)=|\mu |C_{3}s^{q_{m}-2}+\frac{C_{\delta }}{p}\eta ^{p}_{p}s^{p-2},\ \ s\ge 0, \end{aligned}$$
(3.5)

and then we get \(\lim _{s\rightarrow +\infty } k(s)=\lim _{s\rightarrow 0^{+}} k(s)=+\infty ,\) which implies that k(s) is bounded below, thus k(s) admits a minimizer \(s_{0}\):

$$\begin{aligned} s_{0}=\left( \frac{|\mu |C_{3}p(2-q_{m})}{ C_{\delta }\eta _{p}^{p}(p-2)}\right) ^{\frac{1}{p-q_{m}}}. \end{aligned}$$

From (3.5)

$$\begin{aligned}\begin{aligned} \inf _{s\in [0,+\infty )}k(s)&=k(s_{0})\\&=|\mu |C_{3}\left( \frac{|\mu |C_{3}p(2-q_{m})}{ C_{\delta }\eta _{p}^{p}(p-2)}\right) ^{\frac{q_{m}-2}{p-q_{m}}}+\frac{C_{\delta }}{p}\eta ^{p}_{p} \left( \frac{|\mu |C_{3}p(2-q_{m})}{ C_{\delta }\eta _{p}^{p}(p-2)}\right) ^{\frac{p-2}{p-q_{m}}}\\&=|\mu |^{\frac{p-2}{p-q_{m}}} \left[ C_{3}\left( \frac{C_{3}p(2-q_{m})}{C_{\delta }\eta ^{p}_{p}(p-2)}\right) ^{\frac{q_{m}-2}{p-q_{m}}}+ \frac{C_{\delta }\eta ^{p}_{p}}{p}\left( \frac{C_{3}p(2-q_{m})}{C_{\delta }\eta ^{p}_{p}(p-2)} \right) ^{\frac{q_{m}-2}{p-q_{m}}} \right] .\end{aligned} \end{aligned}$$

Let \(K= C_{3}\left( \frac{C_{3}p(2-q_{m})}{C_{\delta }\eta ^{p}_{p}(p-2)}\right) ^{\frac{q_{m}-2}{p-q_{m}}}+ \frac{C_{\delta }\eta ^{p}_{p}}{p}\left( \frac{C_{3}p(2-q_{m})}{C_{\delta }\eta ^{p}_{p}(p-2)} \right) ^{\frac{q_{m}-2}{p-q_{m}}}\), then \( k(s_{0})=|\mu |^{\frac{p-2}{p-q_{m}}}K\),

if

$$\begin{aligned}|\mu |\le \left( \frac{1}{4K}\right) ^{\frac{p-q_{m}}{p-2}}:=\tilde{\mu }\end{aligned}$$

then

$$\begin{aligned} k(s_{0})<\frac{1}{4},\end{aligned}$$

thus, there exists \(r:=s_{0}>0\) such that \(\inf _{\Vert u\Vert =r}I_{\mu }(u)>0=I(0)\).

After integrating, we obtain from \((f_{1}){\text {--}}(f_{4})\) the existence of \(C_{4}>0\) such that

$$\begin{aligned} F(x,u)\ge C_{4}(|u|^{\theta }-|u|^{2}),\ \ \forall x \in \Omega ,\ \ u\in \mathbb {R}. \end{aligned}$$
(3.6)

For \(t>0\), note that \(\theta >4\), we have, for some \(u_{0}\in E\)

$$\begin{aligned} \begin{aligned} I_{\mu }(tu_{0})&=\frac{a}{2}t^{2}\Vert u_{0}\Vert ^{2}+\frac{b}{4}t^{4}\Vert u_{0}\Vert ^{4} -\mu \int _{\Omega }G(x,tu_{0})-\int _{\Omega }F(x,tu_{0})\\&\le \frac{a}{2}t^{2}\Vert u_{0}\Vert ^{2}_{E}+\frac{b}{4}t^{4}\Vert u_{0}\Vert ^{4}+|\mu |C_{3}t^{q_{m}}\Vert u_{0}\Vert ^{q_{m}}- C_{4}t^{\theta }|u_{0}|_{\theta }^{\theta }+ C_{4} t^{2}|u_{0}|_{2}^{2}\\&\quad \rightarrow -\infty \ \ (t\rightarrow +\infty ), \end{aligned}\end{aligned}$$

so, there exists \(e=t_{0}u_{0}\) such that \(\Vert e\Vert >r\) and \(I_{\mu }(e)< 0\).

Lemma 3.2

Let \(a>0\), \(b\ge 0\) and assume \((g_{1})\) hold. Set

$$\begin{aligned}\psi (u):=\int _{\Omega }G(x,u),\end{aligned}$$

then \( \psi ^{'}\) is weakly continuous and compact.

Proof

The proof is similar to Lemma 2.2 in [30], so we omit it.

Lemma 3.3

Assume \((g_{1}),(g_{2})\) and \((f_{1}){\text {--}}(f_{3})\) hold. Then for every \(\mu \in \mathbb {R}\), \(I_{\mu }\) satisfies \((PS)_{c}\) condition for every c.

Proof

Let \(\{u_{n}\}\subset H\) be such that \(I_{\mu }(u_{n})\rightarrow c\) and \(I_{\mu }^{'}(u_{n})\rightarrow 0\ \ \hbox {as}\ \ n\rightarrow \infty \). First, we verify the boundedness of \(\{u_{n}\}\). For n large enough, by \((g_{2})\) and \((f_{3})\) we have

$$\begin{aligned}\begin{aligned} c+1+\Vert u_{n}\Vert&\ge I_{\mu }(u_{n})-\frac{1}{\theta }(I_{\mu }^{'}(u_{n}),u_{n})\\&= a\left( \frac{1}{2}-\frac{1}{\theta }\right) \Vert u_{n}\Vert ^{2}+b\left( \frac{1}{4}-\frac{1}{\theta }\right) \Vert u_{n}\Vert ^{2}-\mu \int _{\Omega }(G(x,u_{n})\\&\ \ \ -\frac{1}{\theta }g(x,u_{n})u_{n})+\int _{\Omega }\frac{1}{\theta }(f(x,u_{n})u_{n}-F(x,u_{n}))\\&\ge a\left( \frac{1}{2}-\frac{1}{\theta }\right) \Vert u_{n}\Vert ^{2}-\mu \int _{\mathbb {R}^{3}}(G(x,u_{n})-\frac{1}{\theta }g(x,u_{n})u_{n})\\&\ge a\left( \frac{1}{2}-\frac{1}{\theta }\right) \Vert u_{n}\Vert ^{2}-|\mu |\int _{\Omega }(G(x,u_{n})-\frac{1}{\theta }g(x,u_{n})u_{n}). \end{aligned}\end{aligned}$$

By \((g_{1})\), Lemma 2.1 and Hölder’s inequality we can obtain

$$\begin{aligned} \left| \int _{\Omega }(G(x,u_{n})-\frac{1}{\theta }g(x,u_{n})u_{n})\right|\le & {} \int _{\Omega }|G(x,u_{n}|+|\frac{1}{\theta }g(x,u_{n})u_{n}|\nonumber \\\le & {} \displaystyle \sum ^{m}_{i=1}\left( \frac{1}{q_{i}}+\frac{1}{\theta }\right) |h_{i}(x)|_{\frac{2}{2-q_{i}}}\eta ^{q_{i}}_{2}\Vert u_{n}\Vert ^{q_{i}} \qquad \end{aligned}$$
(3.7)

Thus,

$$\begin{aligned} \begin{aligned} c+1+\Vert u_{n}\Vert&\ge a\left( \frac{1}{2}-\frac{1}{\theta }\right) \Vert u_{n}\Vert ^{2}- |\mu |\displaystyle \sum ^{m}_{i=1}\left( \frac{1}{q_{i}}+\frac{1}{\theta }\right) |h_{i}(x)|_{\frac{2}{2-q_{i}}}\eta ^{q_{i}}_{2}\Vert u_{n}\Vert ^{q_{i}} \end{aligned} \end{aligned}$$

where \( 1<q_{i}<2\), \(i=1,\ldots ,m,\) which gives a bound for \(\{u_{n}\}\).

Next, we prove that the sequence \(\{u_{n}\}\) has a convergent subsequence. Going if necessary to a subsequence, we can assume that \(u_{n}\rightharpoonup u\) in H and \(u_{n}\rightarrow u\) in \(L^{s}(\Omega )\), \(2\le s<6 \). Theorem A.2 in [29] implies that \(f(x,u_{n})\rightarrow f(x,u)\) in \(L^{q}(\Omega )\), \(q=\frac{p}{p-1}\). Observe that

$$\begin{aligned} \begin{aligned}&(a+b\Vert u_{n}\Vert ^{2})\Vert u_{n}-u\Vert ^{2}\\&\quad =(I_{\mu }^{'}(u_{n})-I_{\mu }^{'}(u),u_{n}-u)+b\left( \int _{\Omega }(|\nabla u|^{2}-|\nabla u_{n}|^{2})\int _{\Omega }\nabla u\nabla (u_{n}-u)\right) \\&\qquad + \mu \int _{\Omega }(g(x,u_{n})-g(x,u))(u_{n}-u)+\int _{\Omega }(f(x,u_{n})-f(x,u))(u_{n}-u). \end{aligned}\end{aligned}$$

It is clear that

$$\begin{aligned} (I_{\mu }^{'}(u_{n})-I_{\mu }^{'}(u),u_{n}-u)\rightarrow 0\ \ \hbox {as} \ \ n\rightarrow \infty . \end{aligned}$$
(3.8)

Since \(\{u_{n}\}\) is bounded and \(u_{n}\rightharpoonup u\) in H,

$$\begin{aligned} b\int _{\Omega }(|\nabla u|^{2}-|\nabla u_{n}|^{2})\int _{\Omega }\nabla u\nabla (u_{n}-u)\rightarrow 0\ \ \hbox {as} \ \ n\rightarrow \infty . \end{aligned}$$
(3.9)

By Lemma 3.2, we know that \( \psi ^{'}\) is compact, thus

$$\begin{aligned} \left| \int _{\Omega }(g(x,u)-g(x,u))(u_{n}-u)\right| \rightarrow 0\quad \hbox {as} \ \ n\rightarrow \infty . \end{aligned}$$
(3.10)

It follows from the Hölder’s inequality that

$$\begin{aligned} \begin{aligned}&\left| \int _{\Omega }(f(x,u_{n})-f(x,u))(u_{n}-u)\right| \le |f(x,u_{n})-f(x,u)|_{q}|u_{n}-u|_{p} \rightarrow 0\\&\quad \hbox {as} \ \ n\rightarrow \infty . \end{aligned} \end{aligned}$$
(3.11)

By (3.8)–(3.11), we obtain that \(\Vert u_{n}-u\Vert \rightarrow 0\ \ \hbox {as}\ \ n\rightarrow \infty .\) \(\square \)

Proof of Theorem 1.1

By Lemmas 3.13.3 and using Lemma 2.3, we complete the proof of Theorem 1.1. \(\square \)

3.2 Proof of Theorem 1.2

In this section, we prove the existence of infinitely many solutions with high energy to system (1.1).

Proof of Theorem 1.2

On \(Z_{k}\), by Lemma 2.3, we see that

$$\begin{aligned}\beta _{k}:=\sup _{u\in Z_{k},\Vert u\Vert =1}|u|_{p}\rightarrow 0, \ \ k\rightarrow \infty ,\end{aligned}$$

which implies that

$$\begin{aligned} |u|_{p}\le \beta _{k}\Vert u\Vert ,\ \ 1 \le p<2^{*}. \end{aligned}$$
(3.12)

Thus, by (3.1), (3.3), Lemma 2.1 and (3.12) we have,

$$\begin{aligned} \begin{aligned} I_{\mu }(u)&=\frac{a}{2}\Vert u\Vert ^{2}+\frac{b}{4}\Vert u\Vert ^{4}-\mu \int _{\Omega }G(x,u)-\int _{\Omega }F(x,u)\\&\ge \frac{a}{2}\Vert u\Vert ^{2}-\mu \int _{\Omega }G(x,u)-\int _{\Omega }F(x,u)\\&\ge \frac{1}{2}\Vert u\Vert ^{2}-|\mu |C_{3}\Vert u\Vert ^{q_{m}}-\int _{\Omega }\left( \frac{\delta }{2} |u|^{2}+\frac{C_{\delta }}{p}|u|^{p}\right) \\&\ge \frac{1}{2}\Vert u\Vert ^{2}-|\mu |C_{3}\Vert u\Vert ^{q_{m}}-\frac{\delta }{2} \eta ^{2}_{2}\Vert u\Vert ^{2}-\frac{C_{\delta }}{p}\beta ^{p}_{k}\Vert u\Vert ^{p}\\&=\Vert u\Vert ^{2}\left( \frac{1}{2}-\frac{\delta }{2}\eta ^{2}_{2}-|\mu |C_{3}\Vert u\Vert ^{q_{m}-2}- \frac{C_{\delta }}{p}\beta ^{p}_{k}\Vert u\Vert ^{p-2}\right) . \end{aligned} \end{aligned}$$

For \(\delta \le \frac{2a-1}{2\eta ^{2}_{2}}\) satisfying \(\frac{a}{2}-\frac{\delta }{2}\eta ^{2}_{2}\ge \frac{1}{4}\), we have

$$\begin{aligned} I_{\mu }(u)\ge \Vert u\Vert ^{2}\left( \frac{1}{4}- |\mu |C_{3}\Vert u\Vert ^{q_{m}-2}-\frac{C_{\delta }}{p}\beta ^{p}_{k}\Vert u\Vert ^{p-2}\right) \end{aligned}$$
(3.13)

and there exists

$$\begin{aligned} \gamma _{k}=\left( \frac{(2-q_{m})C_{3}p|\mu |}{(p-2)C_{\delta }\beta ^{p}_{k}}\right) ^{\frac{1}{p-q_{m}}}, \end{aligned}$$

such that

$$\begin{aligned} b_{k}:=\inf _{u\in Z_{k},\Vert u\Vert =\gamma _{k}} I_{\mu }(u) \ge \frac{1}{8}\gamma ^{2}_{k}. \end{aligned}$$

By Lemma 2.3 we know that \(\beta _{k}\rightarrow 0\ \hbox {as} \ \ k\rightarrow \infty \). Moreover, \(1<q_{m}<2\), \(2\le p<6\), thus \(b_{k}\rightarrow +\infty \ \ \hbox {as}\ \ k\rightarrow \infty \), \((A_{2})\) is proved.

On \(Y_{k}\), by (3.1) and (3.6) we have

$$\begin{aligned} \begin{aligned} I_{\mu }(u)&=\frac{a}{2}\Vert u\Vert ^{2}+\frac{b}{4}\Vert u\Vert ^{4}-\mu \int _{\Omega }G(x,u)-\int _{\Omega }F(x,u)\\&\le \frac{a}{2}\Vert u\Vert ^{2}+\frac{b}{4}\Vert u\Vert ^{4}+|\mu |\int _{\Omega }G(x,u)-\int _{\Omega }C_{4}(| u|^{\theta }-|u|^{2})\\&\le \frac{a}{2}\Vert u\Vert ^{2}+\frac{b}{4}\Vert u\Vert ^{4}+|\mu |C_{3}\Vert u\Vert ^{q_{m}}-C_{4}|u|_{\theta }^{\theta }+ C_{4}|u|_{2}^{2}. \end{aligned}\end{aligned}$$

Since on the finite dimensional space \(Y_{k}\) all norms are equivalent, thus there exists \(\rho _{k}>\gamma _{k}>0\) such that \(\Vert u\Vert =\rho _{k}\) and condition \((A_{1})\) is satisfied for \(\rho _{k}\) large enough.

As in the proof of Lemma 3.3 one sees that \(I_{\mu }\) satisfies \((PS)_{c}\) condition, i.e. \((A_{3})\) hold. Therefore, using Lemma 2.5 we obtain that system (1.1) has a sequence of of solutions \(u_{k}\) such that \(I_{\mu }(u_{k})\rightarrow \infty \) as \(k\rightarrow \infty \). \(\square \)

4 Proof of Theorem 1.3

In this section, we prove the existence of sign-changing solutions to problem (1.1).

4.1 Preliminaries of invariant subset of descending flow

To obtain sign-changing solutions, we make use of the positive and negative cones. Precisely, define

$$\begin{aligned} P^{+}:=\{u\in X:u\ge 0\}\ \ \text{ and }\ \ P^{-}:=\{u\in X:u\le 0\}. \end{aligned}$$

For \(\varepsilon >0\), we also define

$$\begin{aligned} P_{\varepsilon }^{+}:=\{u\in X:\hbox {dist}(u,P^{+})<\varepsilon \}\quad \text{ and }\quad P_{\varepsilon }^{-}:=\{u\in X:\hbox {dist}(u,P^{-})<\varepsilon \}, \end{aligned}$$

where \(\hbox {dist}(u,P^{\pm })=\inf _{v\in P^{\pm }}\Vert u-v\Vert .\)

Let X be a Banach space, \(I\in C^{1}(X,R)\), \( P,Q\subset X\) be open sets. We denote by K the set of critical point of I that is \(K=\{u\in X:I^{'}(u)=0\}\) and \(E=X{\setminus } K\). For \(c \in R\), \(K_{c} = \{x \in X : I(x) = c, I^{'}(x) = 0\}\) and \(I_{c} = \{x \in X : I(x)\le c\}\).

Definition 4.1

[18].  \(\{P,Q\}\) is called an admissible family of invariant sets with respect to I at level c, provided that the following deformation property holds: if \(K_{c}{\setminus } (P\cap Q)=\emptyset \), then there exists \(\varepsilon _{0}> 0\) such that for \(\varepsilon \in (0,\varepsilon _{0})\), there exists \(\eta \in C(X,X)\) satisfying

  1. (i)

    \(\eta (\overline{P})\subset \overline{P}\), \(\eta (\overline{Q})\subset \overline{Q}\);

  2. (ii)

    \(\eta \mid _{{I}^{c-2\varepsilon }}=id\);

  3. (iii)

    \(\eta (I^{c+\varepsilon }{\setminus } (P\cap Q))\subset {I}^{c-\varepsilon }\).

4.2 Properties of operator A

In the following, we take \(X=H\), \(P=P^{+}_{\varepsilon }\), \(Q=P^{-}_{\varepsilon }\) and \(I=I_{\mu }\). Importantly, we need an auxiliary operator A to construct a locally Lipschitz continuous operator B on E to define the descending flow. Precisely, the operator A is defined as follows: for any \(u\in H\), \(v=A(u)\) is the unique solution to the equation

$$\begin{aligned} -(a+b\Vert u\Vert ^{2})\triangle v=\mu g(x,u)+f(x,u). \end{aligned}$$
(4.1)

Then the set of fixed points of A coincide with the set K of critical points of \(I_{\mu }\). Moreover, the operator A has the following important properties.

Lemma 4.2

The operator A is well defined and is continuous.

Proof

For \(u\in H\) fixed, we consider the functional

$$\begin{aligned} J_{u}(v)=\frac{1}{2}(a+b\Vert u\Vert ^{2})\int _{\Omega }|\nabla v|^{2}-\mu \int _{\Omega }g(x,u)v-\int _{\Omega }f(x,u)v. \end{aligned}$$
(4.2)

Obviously, \(J_{u}(v)\in C^{1}(H,R)\), coercive, bounded below, weakly lower semicontinuous and strictly convex. Therefore, \(J_{u}\) admits a unique minimizer which is the unique solution to the problem (4.1).

In the following, we prove that A is continuous. Let \(\{u_{n}\}\subset H\) such that \(u_{n}\rightarrow u\) in H. Let \(v=A(u)\) and \(v_{n}=A(u_{n})\). We need to prove \(\Vert v_{n}-v\Vert \rightarrow 0\). By the definition of A we have for any \(w\in H\)

$$\begin{aligned} (a+b\Vert u_{n}\Vert ^{2})\int _{\Omega }\nabla A(u_{n})\nabla w= & {} \mu \int _{\Omega }g(x,u_{n})w+\int _{\Omega }f(x,u_{n})w, \end{aligned}$$
(4.3)
$$\begin{aligned} (a+b\Vert u\Vert ^{2})\int _{\Omega }\nabla A(u)\nabla w= & {} \mu \int _{\Omega }g(x,u)w+\int _{\Omega }f(x,u)w. \end{aligned}$$
(4.4)

Taking \(w=v_{n}-v\) in (4.3) and (4.4) and subtracting we obtain

$$\begin{aligned} \begin{aligned}&(a+b\Vert u_{n}\Vert ^{2})\Vert v_{n}-v\Vert ^{2}\\&\quad =b(\Vert u\Vert ^{2}-\Vert u_{n}\Vert ^{2})\int _{\Omega }\nabla v\nabla (v_{n}-v)\\&\qquad +\mu \int _{\Omega }(g(x,u_{n})-g(x,u))(v_{n}-v)+\int _{\Omega }(f(x,u_{n})-f(x,u))(v_{n}-v)\\&\quad \le b(\Vert u\Vert ^{2}-\Vert u_{n}\Vert ^{2}) \Vert v\Vert \Vert v_{n}-v\Vert \\&\qquad +\mu \int _{\Omega }(g(x,u_{n})-g(x,u))(v_{n}-v)+|f(x,u_{n})-f(x,u)|_{q}|v_{n}-v|_{p}. \end{aligned} \end{aligned}$$

Since \( \psi ^{'}\) is compact and by Theorem A.2 in [29] , we have \(\Vert v_{n}-v\Vert \rightarrow 0\), that is A is continuous.

Lemma 4.3

For any \(u\in H\) we have

  1. (i)

    \(\langle I_{\mu }^{'}(u),u-A(u)\rangle \ge a\Vert u-A(u)\Vert ^{2}\);

  2. (ii)

    \(\Vert I_{\mu }^{'}(u)\Vert \le (a+b\Vert u\Vert ^{2})\Vert u-A(u)\Vert \).

Proof

Since A(u) is a solution of (4.1), then for any \( w\in H\), we have \(\langle J_{u}^{'}(A(u)),w\rangle =0\) and

$$\begin{aligned} \begin{aligned} \langle I_{\mu }^{'}(u),u-A(u)\rangle&=\langle I_{\mu }^{'}(u),u-A(u)\rangle -\langle J_{u}^{'}(A(u)),u-A(u)\rangle \\&=(a+b\Vert u\Vert ^{2})\Vert u-A(u)\Vert ^{2}\\&\ge a\Vert u-A(u)\Vert ^{2}. \end{aligned} \end{aligned}$$

For any \(w\in H\),

$$\begin{aligned} \begin{aligned} \langle I_{\mu }^{'}(u),w\rangle&=\langle I_{\mu }^{'}(u),w\rangle -\langle J_{u}^{'}(A(u)),w\rangle \\&=(a+b\Vert u\Vert ^{2})\int _{\Omega }\nabla (u-A(u))\nabla w\\&\le (a+b\Vert u\Vert ^{2})\Vert u-A(u)\Vert \Vert w\Vert , \end{aligned} \end{aligned}$$

so

$$\begin{aligned}\Vert I_{\mu }^{'}(u)\Vert \le (a+b\Vert u\Vert ^{2})\Vert u-A(u)\Vert . \end{aligned}$$

Lemma 4.4

There exist \(\varepsilon _{0}>0\) and \(\mu _{0}>0\) such that \(A(P_{\varepsilon }^{\pm })\subset P_{\varepsilon }^{\pm }\), \(\forall \varepsilon \in (0,\varepsilon _{0})\), \(\forall \mu \in (0,\mu _{0}]\).

Proof

We only prove \(A(P_{\varepsilon }^{-})\subset P_{\varepsilon }^{-}\), \(A(P_{\varepsilon }^{+})\subset P_{\varepsilon }^{+}\) is similar. Let \(u\in H\), and \(v=A(u)\). As usual we denote \(v^{+}=\max \{0,v\}\) and \(v^{-}=\min \{0,v\}\), for any \(v\in H\). Obviously, \(\hbox {dist}(v,P^{-})\le \Vert v^{+}\Vert \). Then, by \((g_1)\), (3.2), Lemma 2.1 and Hölder’s inequality, we have

$$\begin{aligned} \begin{aligned}&(a+b\Vert u\Vert ^{2})\hbox {dist}(v,P^{-})\Vert v^{+}\Vert \\&\quad \le (a+b\Vert u\Vert ^{2})\Vert v^{+}\Vert ^{2}\\&\quad =(a+b\Vert u\Vert ^{2})(v,v^{+})\\&\quad =(a+b\Vert u\Vert ^{2})\int _{\Omega }\nabla v^{+}\nabla v^{+}\\&\quad =\mu \int _{\Omega }g(x,u)v^{+}+\int _{\Omega }f(x,u)v^{+}\\&\quad \le \int _{\Omega }[\mu g(x,u^{+})+f(x,u^{+})]v^{+}\\&\quad \le \left( \mu \eta _{2}\displaystyle \sum ^{m}_{i=1}(|h_{i}(x)|_{\frac{2}{2-q_{i}}}|u^{+}|_{2}^{q_{i}-1})+\varepsilon \eta _{2} |u^{+}|_{2}+C_{\varepsilon }\eta _{p}|u^{+}|^{p-1}_{p}\right) \Vert v^{+}\Vert \\&\quad \le \left( \mu \eta _{2}\displaystyle \sum ^{m}_{i=1}(|h_{i}(x)|_{\frac{2}{2-q_{i}}}\hbox {dist}(u,P^{-})^{q_{i}-1})\right. \\&\qquad +\left. \delta \eta _{2}\hbox {dis}t(u,P^{-})+C_{\delta }\eta _{p}\hbox {dist}(u,P^{-})^{p-1}\right) \Vert v^{+}\Vert . \end{aligned}\end{aligned}$$

It follows that

$$\begin{aligned} \hbox {dist}(v,P^{-}) \le \frac{ \mu \eta _{2}\displaystyle \sum \nolimits ^{m}_{i=1}(|h_{i}(x)|_{\frac{2}{2-q_{i}}}\hbox {dist}(u,P^{-})^{q_{i}-1}) +\delta \eta _{2}\hbox {dist}(u,P^{-})+C_{\delta }\eta _{p}\hbox {dist}(u,P^{-})^{p-1} }{a+b\Vert u\Vert ^{2}} . \end{aligned}$$

For \(\hbox {dist}(v,P^{-})\) small enough, by \(1<q_{1}<q_{2}<\cdots<q_{m}<2\), we have

$$\begin{aligned} \hbox {dist}(v,P^{-})\le \frac{ \mu \eta _{2}C^{'}_{3}\hbox {dist}(u,P^{-})^{q_{1}-1} +\delta \eta _{2}\hbox {dist}(u,P^{-})+C_{\delta }\eta _{p}\hbox {dist}(u,P^{-})^{p-1} }{a+b\Vert u\Vert ^{2}}. \end{aligned}$$

Let

$$\begin{aligned} \frac{\delta \eta _{2}\varepsilon +C_{\delta }\eta _{p}\varepsilon ^{p-1}}{a+b\Vert u\Vert ^{2}}\le \frac{\varepsilon }{2} \quad \hbox {and}\quad \mu \frac{\eta _{2}C_3'\varepsilon ^{q_{1}-1}}{a+b\Vert u\Vert ^{2}}\le \frac{\varepsilon }{2}, \end{aligned}$$

and then there exist \(\varepsilon _{0}>0\) and \(\mu _{0}>0\) such that for \(\varepsilon \in (0,\varepsilon _{0})\) and \(\mu \in [0,\mu _{0}]\)

$$\begin{aligned} \hbox {dist}(A(u),P^{-})=\hbox {dist}(v,P^{-})<\varepsilon , \end{aligned}$$

so, for any \(u\in P^{-}_\varepsilon \), we have \(A(u)\in P^{-}_\varepsilon \). \(\square \)

Remark 4.5

Note that the operator A is not compact and not locally Lipschitz continuous. However, it can be used to construct a locally Lipschitz continuous vector field which inherits its properties.

Lemma 4.6

There exists a locally Lipschitz continuous operator \(B:E\rightarrow H\) such that

  1. (i)

    \(\langle I_{\mu }^{'}(u),u-B(u)\rangle \ge \frac{1}{2}\Vert u-A(u)\Vert ^{2}\);

  2. (ii)

    \(\frac{1}{2}\Vert u-B(u)\Vert \le \Vert u-A(u)\Vert \le 2\Vert u-B(u)\Vert \);

  3. (iii)

    \(B(P_{\varepsilon }^{\pm })\subset P_{\varepsilon }^{\pm }\), \(\forall \varepsilon \in (0,\varepsilon _{0})\);

  4. (iv)

    if \(I_{\mu }\) is even, then B is odd.

The proof of this theorem follows the lines of [18].

Remark 4.7

Lemmas 4.34.6 imply that

$$\begin{aligned} \langle I_{\mu }^{'}(u),u-B(u)\rangle \ge \frac{1}{8}\Vert u-B(u)\Vert ^{2}, \end{aligned}$$

and

$$\begin{aligned} \Vert I_{\mu }^{'}(u)\Vert \le 2(a+b\Vert u\Vert ^{2})\Vert u-B(u)\Vert . \end{aligned}$$

4.3 Existence of sign-changing solution

Lemma 4.8

Let \(m_{1} < m_{2}\), \(\alpha >0\). Then, there exists \(\beta >0\) such that \(\Vert u-B(u)\Vert \ge \beta \), if \(u\in H\), \(I_{\mu }(u)\in [m_{1},m_{2}]\) and \(\Vert I_{\mu }^{'}(u)\Vert \ge \alpha .\)

Proof

By the definition of A, for all \(u\in H\),

$$\begin{aligned} (a+b\Vert u\Vert ^{2})\int _{\Omega }\nabla A(u)\nabla u=\mu \int _{\Omega }g(x,u)u+\int _{\Omega }f(x,u)u \end{aligned}$$

so, we have

$$\begin{aligned} \begin{aligned}&I_{\mu }(u)-\frac{1}{\theta }(a+b\Vert u\Vert ^{2})(u,u-A(u))\\&\quad =\frac{a}{2}\Vert u\Vert ^{2}+\frac{b}{4}\Vert u\Vert ^{4}-\mu \int _{\Omega }G(x,u)-\int _{\Omega }F(x,u)\\&\qquad -\frac{1}{\theta }(a+b\Vert u\Vert ^2)\int _{\Omega }\nabla u \nabla (u-A(u))\\&\quad =a\left( \frac{1}{2}-\frac{1}{\theta }\right) \Vert u\Vert ^{2}+b\left( \frac{1}{4}-\frac{1}{\theta }\right) \Vert u\Vert ^{4}+\frac{1}{\theta } (a+b\Vert u\Vert ^{2})\int _{\Omega }\nabla u\nabla A(u)\\&\qquad -\mu \int _{\Omega }G(x,u)- \int _{\Omega }F(x,u)\\&\quad =a\left( \frac{1}{2}-\frac{1}{\theta }\right) \Vert u\Vert ^{2}+b\left( \frac{1}{4}-\frac{1}{\theta }\right) \Vert u\Vert ^{4}+\frac{1}{\theta } \left( \mu \int _{\Omega }g(x,u)u+\int _{\Omega }f(x,u)u\right) \\&\qquad -\mu \int _{\Omega }G(x,u)- \int _{\Omega }F(x,u)\\&\quad =a\left( \frac{1}{2}-\frac{1}{\theta }\right) \Vert u\Vert ^{2}+b\left( \frac{1}{4}-\frac{1}{\theta }\right) \Vert u\Vert ^{4}+ \mu \int _{\Omega }\left( \frac{1}{\theta }g(x,u)u-G(x,u)\right) \\&\qquad -\int _{\Omega }\left( \frac{1}{\theta }f(x,u)u-F(x,u)\right) \end{aligned} \end{aligned}$$

thus, by (3.7) and (ii) of Lemma 4.6,

$$\begin{aligned} \begin{aligned}&b\left( \frac{1}{4}-\frac{1}{\theta }\right) \Vert u\Vert ^{4}\\&\quad =I_{\mu }(u)-\frac{1}{\theta }(a+b\Vert u\Vert ^{2})(u,u-A(u)) -a\left( \frac{1}{2}-\frac{1}{\theta }\right) \Vert u\Vert ^{2}\\&\quad \ \ \ -\mu \int _{\Omega }\left( \frac{1}{\theta }g(x,u)u-G(x,u)\right) -\int _{\Omega }\left( \frac{1}{\theta }f(x,u)u)-F(x,u)\right) \\&\quad \le |I_{\mu }(u)|+\frac{1}{\theta }(a+b\Vert u\Vert ^{2})\Vert u\Vert \Vert u-A(u)\Vert +\mu \int _{\Omega }(G(x,u)-\frac{1}{\theta }g(x,u)u)\\&\quad \le |I_{\mu }(u)|\!+\!\frac{2}{\theta }(a+b\Vert u\Vert ^{2})\Vert u\Vert \Vert u-B(u)\Vert \!+\!\mu \displaystyle \sum ^{m}_{i=1}\left( \frac{1}{q_{i}}+\frac{1}{\theta }\right) |h_{i}(x)|_{\frac{2}{2-q_{i}}}\eta ^{q_{i}}_{2}\Vert u_{n}\Vert ^{q_{i}}. \end{aligned} \end{aligned}$$
(4.5)

Arguing toward a contradiction, if there exists \(\{u_{n}\}\subset H\) with \(I_{\mu }(u_{n})\in [m_{1},m_{2}]\) and \(\Vert I_{\mu }^{'}(u_{n})\Vert \ge \alpha \) such that \(\Vert u_n-B(u_{n})\Vert \rightarrow 0\) as \(n\rightarrow \infty \), then it follows from (4.5) that \(\{u_{n}\}\) is bounded, and by Remark 4.7 we see that \(\Vert I_{\mu }^{'}(u_{n})\Vert \rightarrow 0\) as \(n\rightarrow \infty \), which is a contradiction. \(\square \)

Lemma 4.9

(Deformation lemma). Let \(S\subset H\). Assume there exists a \(\varepsilon >0\) such that Lemmas 4.6 and 4.8 is satisfied. Let \(c\in \mathbb {R}\) and \(\varepsilon _0>0\) such that

$$\begin{aligned} \forall u\in I_{\mu }^{-1}([c-2\varepsilon _{0},c+2\varepsilon _0])\cap S_{2\delta }:\ \ \Vert I_{\mu }^{'}(u)\Vert \ge \varepsilon _0. \end{aligned}$$

Then for some \(\varepsilon _{1}\in (0,\varepsilon _0)\) there exists \(\eta \in \mathcal {C}([0,1]\times H,H)\) such that

  1. (i)

    \( \eta (t,u)=u\), if \(t=0\) or if \(u\notin I_{\mu }^{-1}([c-2\varepsilon _{1},c+2\varepsilon _{1}])\),

  2. (ii)

    \(\eta (1,I_{\mu }^{c+\varepsilon _{1}}\cap S)\subset I_{\mu }^{c-\varepsilon _{1}}\),

  3. (iii)

    \(I_{\mu }(\eta (\cdot ,u))\) is not increasing, \(\forall u\in H\),

  4. (iv)

    \(\eta (t,\overline{P^{+}_{\varepsilon }})\subset \overline{P^{+}_{\varepsilon }}\), \(\eta (t,\overline{P^{-}_{\varepsilon }})\subset \overline{P^{-}_{\varepsilon }}\), \(\forall t\in [0,1]\),

  5. (v)

    \(\eta (t,\cdot )\) is odd, \(\forall t\in [0,1]\).

7

Proof

The proof is similar to the proof of [5, Lemma 2.1.] and we include it here for completeness. Define

$$\begin{aligned} \begin{aligned}&A_1:=I_{\mu }^{-1}([c-2\varepsilon _1,c+2\varepsilon _1])\cap S_{2\delta },\\&A_2:=I_{\mu }^{-1}([c-\varepsilon _1,c+\varepsilon _1])\cap S_{\delta },\\&h(u):=\frac{\hbox {dist}(u,H\backslash A_1)}{\hbox {dist}(u,H\backslash A_1)+\hbox {dist}(A_2)},\quad u\in H.\end{aligned} \end{aligned}$$

Then \(h(u)=0\ \ \hbox {on} \ \ H\backslash A_1,\) \(h(x)=1 \ \ \hbox {on}\ \ A_2\), \(0\le g\le 1\), and g is Lipschitz continuous on H. Consider the vector field

$$\begin{aligned} W(u):= \left\{ \begin{array}{ll} -g(u)\Vert V(u)\Vert ^{-2}V(u),&{}\quad u\in A_1,\\ 0,&{}\quad u\in H\backslash A_1,\end{array}\right. \end{aligned}$$

where V is defined as follows:

$$\begin{aligned} V: E\rightarrow H,\ \ u\mapsto V(u):=u-B(u). \end{aligned}$$

Obviously, W is locally Lipschitz continuous and odd in u if \(I_{\mu }\) is even. Moreover, by our choice of \(\varepsilon _1\) above we have

$$\begin{aligned} \Vert W(u)\Vert =\Vert g(u)\Vert \frac{1}{\Vert u-B(u)\Vert }\le \frac{2(a+b\Vert u\Vert ^2)}{\Vert I_{\mu }^{'}(u)\Vert }\le \frac{2(a+b\Vert u\Vert ^2)}{\varepsilon _0}. \end{aligned}$$
(4.6)

Consider the following initial value problem

$$\begin{aligned} \left\{ \begin{aligned}&\frac{\mathrm{d}}{\mathrm{d}t}\sigma (t,u)=W(u)\\&\sigma (0,u)=u\end{aligned}\right. \end{aligned}$$
(4.7)

The basic existence-uniqueness theorem for ordinary differential equations implies that for each \(u\in H\) (4.7) has a unique solution \(\sigma (\cdot ,u)\in C(\mathbb {R^{+}},H).\)

By (4.6), we have

$$\begin{aligned} \Vert \sigma (t,u)-u\Vert \le \int ^{t}_{0}\Vert W(\sigma (s,u))\Vert \mathrm{d}s\le \frac{2(a+b\Vert u\Vert ^2)}{\varepsilon _0}t, \end{aligned}$$
(4.8)

and by Remark 4.7 we have

$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d}t}I(\sigma (t,u))&=-\langle I_{\mu }^{'}(\sigma (t,u)),g(\sigma (t,u))\Vert V(\sigma (t,u))\Vert ^{-2}V(\sigma (t,u))\rangle \nonumber \\&=-g(\sigma (t,u))\Vert V(\sigma (t,u))\Vert ^{-2}\langle I^{'}(\sigma (t,u)),V(\sigma (t,u))\rangle \nonumber \\&= -g(\sigma (t,u))\Vert \sigma (t,u)-B(\sigma (t,u))\Vert ^{-2}\langle I^{'}(\sigma (t,u)),\sigma (t,u)-B(\sigma (t,u))\rangle \nonumber \\&\le -g(\sigma (t,u))\Vert \sigma (t,u)-B(\sigma (t,u))\Vert ^{-2}\cdot \frac{1}{8}\Vert \sigma (t,u)-B(\sigma (t,u))\Vert ^{2}\nonumber \\&=-\frac{1}{8}g(\sigma (t,u)). \end{aligned}$$
(4.9)

Define

$$\begin{aligned} \eta : [0,1]\times H\rightarrow H,\ \ u\mapsto \eta (t,u):=\sigma (16\varepsilon _1t,u). \end{aligned}$$

(i)  If \(t=0\), then \(\eta (t,u)=\eta (0,u)=\sigma (0,u)=u.\)

For any u, we have \(W(u)=0\). Since \(A_1\) is a closed subset of H, so \(H\backslash A_1\) is open, then there exists \(r=r(u)>0\) such that \(B_r(u)\subset H\backslash A_1\). And so, for any \(v\in B_r(u)\), \(W(u)=0\), \(\sigma (\cdot ,u)\in C(\mathbb {R^{+}},H)\) then there exists \(\xi>16\varepsilon _1>0\), such that \(|\sigma (t,u)-\sigma (0,u)|<r\), \(\forall t\in [0,\xi ]\), which implies that \(\sigma (t,u)\in B_r(u)\), \(W(\sigma (t,u))=0\), \(\forall t\in [0,\xi ]\).

Thus,

$$\begin{aligned} \int ^{16\varepsilon _{1}t}_{0}\frac{\mathrm{d}\sigma (s,u)}{\mathrm{d}s}\mathrm{d}s=\int ^{16\varepsilon _{1}t}_{0}-W(\sigma (t,u))\mathrm{d}s=0. \end{aligned}$$

It follows that

$$\begin{aligned}\eta (t,u)=\sigma (16\varepsilon _1)=\sigma (0,u)=u,\quad \forall t\in [0,1], \quad \forall u\notin A_1. \end{aligned}$$

(ii)  For any \(u\in I_{\mu }^{c+\varepsilon _{1}\cap S}\), we need to prove \(I_{\mu }(\eta (1,u))=I_{\mu }(\sigma (16\varepsilon _1,u))\le c-\varepsilon _1\).

Let \(u\in I_{\mu }^{c+\varepsilon _{1}}\cap S\). If there is \(t\in [0,16\varepsilon _1]\) such that \(I_{\mu }(\sigma (t,u))\le c-\varepsilon _1,\) then it follows from (4.9) that \(I_{\mu }(\sigma (16\varepsilon _1))<c-\varepsilon _1\) and (ii) is satisfied. If

$$\begin{aligned} \sigma (t,u)\in I_{\mu }^{-1}[c-\varepsilon _1,c+\varepsilon _1],\quad \forall t\in [0,16\varepsilon _{1}], \end{aligned}$$

we obtain from (4.8) and (4.9)

$$\begin{aligned} \begin{aligned} I_{\mu }(\sigma (16\varepsilon _{1},u))&= I_{\mu }(\sigma (0,u))+\int ^{16\varepsilon _1}_{0}\frac{\mathrm{d}}{\mathrm{d}t}I(\sigma (t,u))\mathrm{d}t\\&\le I_{\mu }(\sigma (0,u))-\int ^{16\varepsilon _1}_{0}\frac{g(\sigma (t,u))}{8}\mathrm{d}t\\&\le I_{\mu }(\sigma (0,u))-\frac{1}{8}\cdot 16\varepsilon _1 \\&\le c+\varepsilon _1- 2\varepsilon _1=c-\varepsilon _1. \end{aligned} \end{aligned}$$

and (ii) is also satisfied. Obviously, (iii) is satisfied by (4.9). Finally, (iv) is a consequence of Lemma 4.6 (see [20] for a detailed proof).

The following theorem is a corollary of [17, Theorem 2.4].

Lemma 4.10

Assume there exists \(\varepsilon >0\) such that Lemmas 4.6 and 4.8 is satisfied. Assume also that there exists a map \(\varphi _{0}:\triangle \rightarrow H\) satisfying

  1. 1.

    \(\varphi _{0}(\partial _{1}\triangle )\subset P^{+}_{\varepsilon }\) and \(\varphi _{0}(\partial _{2}\triangle )\subset P^{-}_{\varepsilon }\),

  2. 2.

    \(\varphi _{0}(\partial _{0}\triangle )\cap P^{+}_{\varepsilon }\cap P^{-}_{\varepsilon }=\emptyset \),

  3. 3.

    \(c_{0}:=\underset{{u\in \varphi _{0}(\partial _{0}\triangle )}}{\sup }I_{\mu }(u)<c_{*}:=\underset{{u\in \partial P^{+}_{\varepsilon }\cap \partial P^{-}_{\varepsilon }}}{\inf }I_{\mu }(u).\)

where \(\triangle =\{(t_{1},t_{2}\in R^{2}:t_{1},t_{2}\ge 0, t_{1}+t_{2}\le 1\}\), \(\partial _{1}\triangle =\{0\}\times [0,1],\) \(\partial _{2}\triangle =[0,1]\times \{0\}\) and \(\partial _{0}\triangle =\{(t_{1},t_{2})\in R^{2}:t_{1},t_{2}\ge 0, t_{1}+t_{2}= 1\}\).

Then there exists a sequence \(\{u_{n}\}\subset \overline{(H\backslash (P^{+}_{\varepsilon }\cap P^{-}_{\varepsilon }))_{2\delta }}\) such that

$$\begin{aligned} I_{\mu }^{'}(u_{n})\rightarrow 0,\ \ \hbox {and}\ \ I_{\mu }(u_{n})\rightarrow c:=\underset{\varphi \in \Gamma }{\inf }\underset{u\in \varphi (\triangle )\cap (H\backslash (P^{+}_{\varepsilon }\cap P^{-}_{\varepsilon })}{\sup }I_{\mu }(u)\ge c_{*}, \end{aligned}$$
(4.10)

where

$$\begin{aligned}\Gamma :=\{\varphi \in C(\Delta ,X):\varphi (\partial _{1}\triangle )\subset P^{+}_{\varepsilon },\varphi (\partial _{2}\triangle )\subset P^{-}_{\varepsilon } \ \ \hbox {and} \ \ \varphi \mid _{\partial _{0}\Delta }=\varphi _{0}\}.\end{aligned}$$

If in addition \(I_{\mu }\) satisfies the \((PS)_{c}\) condition for any \(c>0\), then \(I_{\mu }\) has a sign-changing critical point.

Proof

Lemma 2.1 in [17] implies that \(\varphi (\Delta )\cap \partial P^{+}_{\varepsilon }\cap \partial P^{-}_{\varepsilon }\ne \emptyset \) for any \(\varphi \in \Gamma \). This intersection property implies that \(c\ge c_{*}>c_{0}.\)

We claim that

$$\begin{aligned}&\forall \varepsilon _0\in (0,\frac{c-c_0}{2}), \exists u\in I_{\mu }^{-1}([c-2\varepsilon _0,c+2\varepsilon _0])\cap \overline{(H\backslash (P^{+}_{\varepsilon }\cup P^{-}_{\varepsilon }))_{2\delta }} \nonumber \\&\quad \hbox {such}\ \ \hbox {that} \ \ \Vert I^{'}(u)\Vert <\varepsilon _0. \end{aligned}$$
(4.11)

Arguing indirectly, if there exists \(\varepsilon _0\in (0,\frac{c-c_0}{2})\) such that

$$\begin{aligned} \Vert I^{'}(u)\Vert \ge \varepsilon _0,\ \ \forall u\in I_{\mu }^{-1}([c-2\varepsilon _0,c+2\varepsilon _0])\cap \overline{(H\backslash (P^{+}_{\varepsilon }\cup P^{-}_{\varepsilon }))_{2\delta }}. \end{aligned}$$

Taking \(S=H\backslash (P^{+}_{\varepsilon }\cup P^{-}_{\varepsilon })\) in Lemma 4.9 and we define the map

$$\begin{aligned} \tau : \Delta \rightarrow X,\ \ x\mapsto \tau (x):=\eta (1,\varphi (x)), \end{aligned}$$

where \(\varepsilon _1\) and \(\eta \) are given by Lemma 4.9 and \(\varphi \) is chosen such that

$$\begin{aligned} \underset{u\in \varphi (\triangle )\cap (H\backslash (P^{+}_{\varepsilon }\cap P^{-}_{\varepsilon }))}{\sup }I_{\mu }(u)\le c+\varepsilon _1. \end{aligned}$$
(4.12)

Since \(c_0<c-2\varepsilon _1\), by assumption (3) we have \(I_{\mu }(\varphi _{0}(\partial _{0}\triangle ))<c-2\varepsilon _1\), which implies that \(\varphi _{0}(\partial _{0}\triangle )\subset I_{\mu }^{c-2\varepsilon _1}\). It follows that \(\tau \in \Gamma \).

Combining (ii), (iv) of Lemma 4.9 with (4.12), we have

$$\begin{aligned} \eta (1,\varphi (\Delta ))\cap (H\backslash (P^{+}_{\varepsilon }\cup P^{-}_{\varepsilon }))\subset \eta (1,I^{c+\varepsilon _1}_{\mu })\cap (H\backslash (P^{+}_{\varepsilon }\cup P^{-}_{\varepsilon }))\subset I^{c-\varepsilon _1}_{\mu }, \end{aligned}$$

which is in contradiction with the definition of c. So, (4.10) is satisfied.

Taking limit \(\varepsilon _0\rightarrow 0,\) then there exists a sequence \(\{u_n\}\) satisfying (4.10). Since \(I_{\mu }\) satisfies the \((PS)_c\) condition, which implies that \(u_n\rightarrow u\) as \(n\rightarrow \infty \), up to a subsequence. Moreover, \(I_{\mu }\in C^{1}(H,\mathbb {R})\), then \(u\in \overline{(H\backslash (P^{+}_{\varepsilon }\cup P^{-}_{\varepsilon }))_{2\delta }}\), and u is sign-changing.

Lemma 4.11

For \(p\in [2,6]\) there exists \(k>0\) independent of \(\varepsilon \) such that \(\Vert u\Vert _{p}\le k\varepsilon \) for \(u\in P^{+}_{\varepsilon }\cap P^{-}_{\varepsilon }\).

Proof

\(\forall u\in P^{+}_{\varepsilon }\cap P^{-}_{\varepsilon }\), by Lemma 2.1, we can obtain

$$\begin{aligned} |u^{\pm }|_{p}=\inf _{v\in P^{\mp }}|u-v|_{p}\le \eta _{p}\inf _{v\in P^{\mp }}\Vert u-v\Vert \le \eta _{p}\hbox {dist}(u,P^{\mp }), \end{aligned}$$

thus \(\Vert u\Vert _{p}\le k\varepsilon \) for \(u\in P^{+}_{\varepsilon }\cap P^{-}_{\varepsilon }\). \(\square \)

Lemma 4.12

If \(\varepsilon >0\) small enough then there exists \(\mu _{0}>0\) such that \(\mu \in (0,\mu _{0})\) have \(I_{\mu }(u)\ge \frac{a}{8}\varepsilon ^{2}\) for \(u\in \partial P^{+}_{\varepsilon }\cap \partial P^{-}_{\varepsilon }\).

Proof

Let \(u\in \partial P^{+}_{\varepsilon }\cap \partial P^{-}_{\varepsilon }\), we have \(\Vert u^{+}\Vert \ge \hbox {dist}(u,P^{\mp })=\varepsilon \). By Lemma 4.11 and (3.3), we have

$$\begin{aligned} \begin{aligned} I_{\mu }(u)&=\frac{a}{2}\Vert u\Vert ^{2}+\frac{b}{4}\Vert u\Vert ^{4}-\mu \int _{\Omega }G(x,u)-\int _{\Omega }F(x,u)\\&\ge \frac{a}{2}\varepsilon ^{2}+\frac{b}{4}\varepsilon ^{4}-\mu C_{5}|u|^{\nu }_{\nu }-\frac{\delta }{2}|u|^{2}_{2}-\frac{C_{\delta }}{p}|u|^{p}_{p}\\&\ge \frac{a}{2}\varepsilon ^{2}+\frac{b}{4}\varepsilon ^{4}-\mu C_{5}\varepsilon ^{\nu }-\frac{\delta }{2}\varepsilon ^{2}-\frac{C_{\delta }}{p}\varepsilon ^{p}\\&\ge \frac{a}{2}\varepsilon ^{2}-\mu kC_{5}\varepsilon ^{\nu }-\frac{\delta }{2}k\varepsilon ^{2}-\frac{C_{\delta }}{p}k\varepsilon ^{p}\\&\ge \frac{a}{2}\varepsilon ^{2}-\mu kC_{5}\varepsilon ^{\nu }-\frac{C_{\delta }}{p}k\varepsilon ^{p}\\&\ge \frac{a}{4}\varepsilon ^{2}-\mu kC_{5}\varepsilon ^{\nu }. \end{aligned} \end{aligned}$$

For \(\mu \le \frac{a\varepsilon ^{2-\nu }}{8C_{5}k}\) satisfying \( \frac{a}{4}\varepsilon ^{2}-\mu kC_{5}\varepsilon ^{\nu }\ge \frac{a}{8}\varepsilon ^{2}\), we have

$$\begin{aligned}I(u)\ge \frac{a}{8}\varepsilon ^{2}.\end{aligned}$$

\(\square \)

Proof of Theorem 1.4

It suffices to verify assumptions (1)–(3) in applying Lemma 4.10.

Choose \(v_{1},v_{2}\in C^{\infty }_{0}(\Omega ){\setminus }\{0\}\) satisfying \(\sup (v_{1})\cap \sup (v_{2})=\emptyset \), and \(v_{1}\le 0\), \(v_{2}\ge 0\). Let \(\varphi _{0}(t,s):=R(tv_{1}+sv_{2})\) for \((t,s)\in \Delta \), where R is a positive constant to be determined later. Obviously, for \(t,s\in [0,1]\), \(\varphi _{0}(0,s)=Rsv_{2}\in P^{+}_{\varepsilon }\) and \(\varphi _{0}(t,0)=Rtv_{1}\in P^{-}_{\varepsilon }\). Hence, the assumption (1) is satisfied.

Observe that \(\rho =\min \{|tv_{1}+(1-t)v_{2}|_{2}:0\le t\le 1\}>0\). Then, \(|u|_{2}\ge \rho R\) for \(u\in \varphi _{0}(\partial _{0}\Delta )\) and it follows from Lemma 4.11 that \(\varphi _{0}(\partial _{0}\Delta )\cap P^{+}_{\varepsilon }\cap P^{-}_{\varepsilon }=\emptyset \) for R large enough.

By (3.6) and \((g_{2})\), we have

$$\begin{aligned} \begin{aligned} I_{\mu }(u)&=\frac{a}{2}\Vert u\Vert ^{2}+\frac{b}{4}\Vert u\Vert ^{4}-\mu \int _{\Omega }G(x,u)-\int _{\Omega }F(x,u)\\&\le \frac{a}{2}\Vert u\Vert ^{2}+\frac{b}{4}\Vert u\Vert ^{4}+|\mu |C_{5}|u|^{\nu }_{\nu }-C_{4}|u|_{\theta }^{\theta }+ C_{4}|u|_{2}^{2} \end{aligned} \end{aligned}$$

since \(\theta >4\), by Lemma 4.12, for R large enough

$$\begin{aligned} c_{0}:=\underset{{u\in \varphi _{0}(\partial _{0}\triangle )}}{\sup }I_{\mu }(u)<c_{*}:=\underset{{u\in \partial P^{+}_{\varepsilon }\cap \partial P^{-}_{\varepsilon }}}{\inf }I(u). \end{aligned}$$

By Lemma 3.3, we can know that \(I_{\mu }\) satisfies the \((PS)_{c}\) condition for any \(c>0\). According to Lemma 4.10, \(I_{\mu }\) has at least one sign-changing critical point, which is a sign-changing solution of equation (1.1).