Infinitely many sign-changing solutions for the nonlinear Schrödinger–Poisson system

Abstract

In this paper, we consider the following Schrödinger–Poisson system

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta u+V(x)u+\phi u=f(u)&{}\quad \text{ in }\ \mathbb {R}^3,\\ -\Delta \phi =u^2&{}\quad \text{ in }\ \mathbb {R}^3. \end{array} \right. \end{aligned}$$

We investigate the existence of multiple bound state solutions, in particular sign-changing solutions. By using the method of invariant sets of descending flow, we prove that this system has infinitely many sign-changing solutions. In particular, the nonlinear term includes the power-type nonlinearity $f(u)=|u|^{p-2}u$ for the well-studied case $p\in (4,6)$, and the less studied case $p\in (3,4)$, and for the latter case, few existence results are available in the literature.

1 Introduction and main results

In this paper, we are concerned with the existence of bound state solutions, in particular sign-changing solutions, to the following nonlinear Schrödinger–Poisson system

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta u+V(x)u+\phi u=f(u)&{}\quad \text{ in }\ \mathbb {R}^3,\\ -\Delta \phi =u^2&{}\quad \text{ in }\; \mathbb {R}^3. \end{array} \right. \end{aligned}$$

(1.1)

In the last two decades, system (1.1) has been studied extensively due to its strong physical background. From a physical point of view, it describes systems of identical charged particles interacting each other in the case that magnetic effects could be ignored and its solution is a standing wave for such a system. The nonlinear term $f$ models the interaction between the particles [28]. The first equation of (1.1) is coupled with a Poisson equation, which means that the potential is determined by the charge of the wave function. The term $\phi u$ is nonlocal and concerns the interaction with the electric field. For more detailed physical aspects of systems like (1.1) and for further mathematical and physical interpretation, we refer to [3, 12, 13] and the references therein.

In recent years, there has been increasing attention to systems like (1.1) on the existence of positive solutions, ground states, radial and non-radial solutions and semiclassical states. Ruiz [26] considered the following problem

$$\begin{aligned} \left\{ \begin{array}{ll} -\Delta u+u+\lambda \phi u=|u|^{p-2}u&{}\quad \text{ in }\ \mathbb {R}^3,\\ -\Delta \phi =u^2&{}\quad \text{ in }\ \mathbb {R}^3 \end{array} \right. \end{aligned}$$

(1.2)

and gave existence and nonexistence results, depending on the parameters $p\in (2,6)$ and $\lambda >0$. In particular, if $\lambda \ge \frac{1}{4}$, the author showed that $p=3$ is a critical value for the existence of positive solutions. By using the concentration compactness principle, Azzollini and Pomponio [5] proved the existence of a ground state solution of (1.1) when $f(u)=|u|^{p-2}u$ and $p\in (3,6)$. But no symmetry information concerning, this ground state solution was given. In [27], Ruiz studied the profile of the radial ground state solutions to (1.2) as $\lambda \rightarrow 0$ for $p\in \left( \frac{18}{7},3\right)$. Using variational method together with a perturbation argument, Ambrosetti [2] investigated the multiplicity of solutions and semiclassical states to systems like (1.1). Here, we would also like to mention the papers [4, 14, 15, 17, 21, 29] for related topics.

Another topic which has increasingly received interest in recent years is the existence of sign-changing solutions of systems like (1.1). Recall that a solution $(u,\phi )$ to (1.1) is called a sign-changing solution if $u$ changes its sign. Using a Nehari-type manifold and gluing solution pieces together, Kim and Seok [20] proved the existence of radial sign-changing solutions with prescribed numbers of nodal domains for (1.1) in the case where $V(x)=1$, $f(u)=|u|^{p-2}u$, and $p\in (4,6)$. Ianni [16] obtained a similar result to [20] for $p\in [4,6)$, via a heat flow approach together with a limit procedure. Recently, with a Lyapunov–Schmidt reduction argument, Ianni and Vaira [18] constructed non-radial multi-peak solutions with arbitrary large numbers of positive peaks and arbitrary large numbers of negative peaks to the Schrödinger–Poisson system

$$\begin{aligned} \left\{ \begin{array}{ll} -\varepsilon ^2\Delta u+u+\phi u=f(u)&{}\quad \text{ in }\ \mathbb {R}^N,\\ -\Delta \phi =a_Nu^2&{}\quad \text{ in }\ \mathbb {R}^N \end{array} \right. \end{aligned}$$

(1.3)

for $\epsilon >0$ small, where $3\le N\le 6$ and $a_N$ is a positive constant. All the sign-changing solutions obtained in [16, 18, 20] have certain types of symmetries; they are either $O(N)$-invariant or $G$-invariant for some finite subgroup $G$ of $O(N)$, and thus the system is required to have a certain group invariance. Based on variational method and Brouwer degree theory, Wang and Zhou [30] obtained a least energy sign-changing solution to (1.1) without any symmetry by seeking minimizer of the energy functional on the sign-changing Nehari manifold when $f(u)=|u|^{p-2}u$ and $p\in (4,6)$. More recently, in the case where the system is considered on bounded domains $\Omega \subset \mathbb {R}^3$, Alves and Souto [1] obtained a similar result to [30] for a more general nonlinear term $f$.

To the best of our knowledge, there is no result in the literature on the existence of multiple sign-changing solutions as bound states to problem (1.1) without any symmetry and thus to prove the existence of infinitely many sign-changing solutions to problem (1.1) without any symmetry is the first purpose of the present paper. Since the approaches in [1, 16, 20, 30], when applied to the monomial nonlinearity $f(u)=|u|^{p-2}u$, are only valid for $p\ge 4$, we want to provide an argument which covers the case $p\in (3,4)$ and this is the second purpose of the present paper. Moreover, our method does not depend on existence of the Nehari manifold.

In what follows, we assume $V\in C(\mathbb {R}^3,\mathbb {R}^+)$ satisfies the following condition.

($V_0$):

$V$ is coercive, i.e., $\lim \nolimits _{|x|\rightarrow \infty }V(x)=\infty$.

Moreover, we assume $f$ satisfies the following hypotheses.

($f_1$):

$f\in C(\mathbb {R},\mathbb {R})$ and $\lim \nolimits _{s\rightarrow 0}\frac{f(s)}{s}=0$.

($f_2$):

$\limsup \nolimits _{|s|\rightarrow +\infty }\frac{|f(s)|}{|s|^{p-1}}<\infty$ for some $p\in (3,6)$.

($f_3$):

There exists $\mu >3$ such that $tf(t)\ge \mu F(t)>0$ for all $t\not =0$, where $F(t)=\int _0^tf(s)ds$.

As a consequence of $(f_2)$ and $(f_3)$, one has $3<\mu \le p<6$. Our first result reads as

Theorem 1.1

If $(V_0)$ and $(f_1)$–$(f_3)$ hold and $\mu >4$, then problem (1.1) has one sign-changing solution. If moreover $f$ is odd, then problem (1.1) has infinitely many sign-changing solutions.

Remark 1.1

Assumption $(V_0)$ is used only in deriving compactness (the (PS) condition) of the energy functional associated with (1.1). If $\mathbb {R}^3$ in problem (1.1) is replaced with a smooth bounded domain $\Omega \subset \mathbb {R}^3$, Theorem 1.1 without $(V_0)$ and any symmetry assumption on $\Omega$ still holds.

$(f_3)$ is the so-called Ambrosetti–Rabinowitz condition ((AR) for short). Since the nonlocal term $\int _{\mathbb {R}^3}\phi _uu^2$ in the expression of $I$ (see Sect. 2) is homogeneous of degree 4, if $\mu$ from $(f_3)$ satisfies $\mu >4$ then (AR) guarantees boundedness of (PS)-sequences as well as existence of a mountain pass geometry in the sense that $I(tu)\rightarrow -\infty$ as $t\rightarrow \infty$ for each $u\ne 0$. If $\mu <4$, (PS)-sequences may not be bounded and one has $I(tu)\rightarrow \infty$ as $t\rightarrow \infty$ for each $u\ne 0$. To overcome these difficulties, in the case $\mu <4$, we impose on $V$ an additional condition

($V_1$):

$V$ is differentiable, $\nabla V(x)\cdot x\in L^r(\mathbb {R}^3)$ for some $r\in [\frac{3}{2},\infty ]$ and

$$\begin{aligned} 2V(x)+\nabla V(x)\cdot x\ge 0\quad \text{ for } \text{ a.e. }\quad x\in \mathbb {R}^3. \end{aligned}$$

This assumption was introduced in [31, 32] in order to prove compactness with the monotonicity trick of Jeanjean [19]. That $\nabla V(x)\cdot x\in L^r(\mathbb {R}^3)$ for some $r\in [\frac{3}{2},\infty ]$ plays a role only in deriving the Pohoz$\check{\mathrm{a}}$ev identity for solutions of (4.1) in Sect. 4, and it can clearly be weakened since solutions of (4.1) decay at infinity. Nevertheless, we do not want to go further in that direction. We state our second result as follows.

Theorem 1.2

If $(V_0)$–$(V_1)$ and $(f_1)$–$(f_3)$ hold, then problem (1.1) has one sign-changing solution. If in addition $f$ is odd, then problem (1.1) has infinitely many sign-changing solutions.

Remark 1.2

The class of nonlinearities $f$ satisfying the assumptions of Theorem 1.2 includes the monomial nonlinearity $f(u)=|u|^{p-2}u$ with $p\in (3,4)$. Even in this special case, Theorem 1.2 seems to be the first attempt in finding sign-changing solutions to (1.1).

The idea of the proofs of Theorems 1.1 and 1.2 is to use suitable minimax arguments in the presence of invariant sets of a descending flow for the variational formulation. In particular, we make use of an abstract critical point theory developed by Liu et al. [23]. The method of invariant sets of descending flow plays an important role in the study of sign-changing solutions of elliptic problems; we refer to [6–11, 24, 25] and the references therein. However, with the presence of the coupling term $\phi u$, the techniques of constructing invariant sets of descending flow in [6–11, 24, 25] cannot be directly applied to system (1.1), which makes the problem more complicated. The reason is that $\phi u$ is a non-local term and the decomposition

$$\begin{aligned} \int _{\mathbb {R}^3}\phi _u|u|^2=\int _{\mathbb {R}^3}\phi _{u^+}|u^+|^2+\int _{\mathbb {R}^3}\phi _{u^-}|u^-|^2 \end{aligned}$$

does not hold in general for $u\in H^1(\mathbb {R}^3)$. To overcome this difficulty, we adopt an idea from [23] to construct an auxiliary operator $A$ (see Sect. 2), which is the starting point in constructing a pseudo-gradient vector field guaranteeing existence of the desired invariant sets of the flow. Since $f\in C(\mathbb {R},\mathbb {R})$ and $A$ is merely continuous, $A$ itself cannot be used to define the flow. Instead, $A$ is used in a similar way to [8] to construct a locally Lipschitz continuous operator $B$ inheriting the main properties of $A$, and we use $B$ to define the flow. Finally, by minimax arguments in the presence of invariant sets, we obtain the existence of sign-changing solutions to (1.1), proving Theorem 1.1. For the proof of Theorem 1.2, the above framework is not directly applicable due to changes of geometric nature of the variational formulation. We use a perturbation approach by adding a term growing faster than monomial of degree $4$ with a small coefficient $\lambda >0$. For the perturbed problems, we apply the program above to establish the existence of multiple sign-changing solutions, and a convergence argument allows us to pass limit to the original system.

The paper is organized as follows. Section 2 contains the variational framework of our problem and some preliminary properties of $\phi _u$. Section 3 is devoted to the proof of Theorem 1.1. In Sect. 4, we use a perturbation approach to prove Theorem 1.2.

2 Preliminaries and functional setting

In this paper, we make use of the following notations.

• $\Vert u\Vert _p:=\left( \int _{\mathbb {R}^3}|u|^p\right) ^{1/p}$ for $p\in [2,\infty )$ and $u\in L^p(\mathbb {R}^3)$;

• $\Vert u\Vert :=\left( \Vert u\Vert _2^2+\Vert \nabla u\Vert _2^2\right) ^{1/2}$ for $u\in H^1(\mathbb {R}^3)$;

• $C,C_j$ denote (possibly different) positive constants.

For any given $u\in H^1(\mathbb {R}^3)$, the Lax–Milgram theorem implies that there exists a unique $\phi _u\in \mathcal {D}^{1,2}(\mathbb {R}^3)$ such that $-\Delta \phi _u=u^2$. It is well known that

$$\begin{aligned} \phi _u(x)=\int _{\mathbb {R}^3}\frac{u^2(y)}{4\pi |x-y|}\hbox {d}y. \end{aligned}$$

We now summarize some properties of $\phi _u$, which will be used later. See, for instance, [26] for a proof.

Lemma 2.1

• (1) $\phi _u(x)\ge 0,\ x\in \mathbb {R}^3$;

• (2) there exists $C>0$ independent of $u$ such that

  $$\begin{aligned} \int _{\mathbb {R}^3}\phi _uu^2\le C\Vert u\Vert ^4; \end{aligned}$$

• (3) if $u$ is a radial function, then so is $\phi _u$;

• (4) if $u_n\rightarrow u$ strongly in $L^{\frac{12}{5}}(\mathbb {R}^3)$, then $\phi _{u_n}\rightarrow \phi _u$ strongly in $\mathcal {D}^{1,2}(\mathbb {R}^3)$.

Define the Sobolev space

$$\begin{aligned} E=\left\{ u\in \mathcal {D}^{1,2}(\mathbb {R}^3):\int _{\mathbb {R}^3}V(x)u^2<\infty \right\} \end{aligned}$$

with the norm

$$\begin{aligned} \Vert u\Vert _E=\left( \int _{\mathbb {R}^3}\left( |\nabla u|^2+V(x)u^2\right) \right) ^\frac{1}{2}. \end{aligned}$$

This is a Hilbert space, and its inner product is denoted by $(\cdot ,\cdot )_E$.

Remark 2.1

By $(V_0)$, the embedding $E\hookrightarrow L^q(\mathbb {R}^3)\ (2\le q<6)$ is compact. This fact implies the (PS) condition; see, e.g., [10]. As in [9], $(V_0)$ can be replaced with the weaker condition:

$(V_0)'$ :

There exists $r>0$ such that for any $b>0$,

$$\begin{aligned} \lim \limits _{|y|\rightarrow \infty }m\left( \{x\in \mathbb {R}^3:V(x)\le b\}\cap B_r(y)\right) =0, \end{aligned}$$

where $B_r(y)=\{x\in \mathbb {R}^3:\ |x-y|<r\}$ and $m$ is the Lebesgue measure in $\mathbb {R}^3$.

Let us define

$$\begin{aligned} D(f,g)=\int _{\mathbb {R}^3}\int _{\mathbb {R}^3}\frac{f(x)g(y)}{4\pi |x-y|}\hbox {d}x\hbox {d}y. \end{aligned}$$

In particular, for $u\in H^1(\mathbb {R}^3)$, $D(u^2,u^2)=\int _{\mathbb {R}^3}\phi _uu^2$. Moreover, we have the following properties. For a proof, we refer to [22, p. 250] and [27].

Lemma 2.2

• (1) $D(f,g)^2\le D(f,f)D(g,g)$ for any $f,g\in L^{\frac{6}{5}}(\mathbb {R}^3)$;

• (2) $D(uv,uv)^2\le D(u^2,u^2)D(v^2,v^2)$ for any $u,v\in L^{\frac{12}{5}}(\mathbb {R}^3)$.

Substituting $\phi =\phi _u$ into system (1.1), we can rewrite system (1.1) as the single equation

$$\begin{aligned} -\Delta u+V(x)u+\phi _uu=f(u),\quad u\in E. \end{aligned}$$

(2.1)

We define the energy functional $I$ on $E$ by

$$\begin{aligned} I(u)=\frac{1}{2}\int _{\mathbb {R}^3}\left( |\nabla u|^2+V(x)u^2\right) +\frac{1}{4}\int _{\mathbb {R}^3}\phi _uu^2-\int _{\mathbb {R}^3}F(u). \end{aligned}$$

It is standard to show that $I\in C^1(E,\mathbb {R})$ and

$$\begin{aligned} \langle I'(u),v\rangle =\int _{\mathbb {R}^3}\left( \nabla u\cdot \nabla v+V(x)uv+\phi _uuv-f(u)v\right) ,\quad u,\ v\in E. \end{aligned}$$

It is easy to verify that $(u,\phi _u)\in E\times \mathcal {D}^{1,2}(\mathbb {R}^3)$ is a solution of (1.1) if and only if $u\in E$ is a critical point of $I$.

3 Proof of Theorem 1.1

In this section, we prove the existence of sign-changing solutions to system (1.1) in the case $\mu >4$, working with (2.1).

3.1 Properties of operator $A$

We introduce an auxiliary operator $A$, which will be used to construct the descending flow for the functional $I$. Precisely, the operator $A$ is defined as follows: for any $u\in E$, $v=A(u)\in E$ is the unique solution to the equation

$$\begin{aligned} -\Delta v+V(x)v+\phi _uv=f(u),\quad v\in E. \end{aligned}$$

(3.1)

Clearly, the three statements are equivalent: $u$ is a solution of (2.1), $u$ is a critical point of $I$, and $u$ is a fixed point of $A$.

Lemma 3.1

The operator $A$ is well defined and is continuous and compact.

Proof

Let $u\in E$ and define

$$\begin{aligned} J_0(v)=\frac{1}{2}\int _{\mathbb {R}^3}\left( |\nabla v|^2+(V(x)+\phi _u)v^2\right) -\int _{\mathbb {R}^3}f(u)v,\quad v\in E. \end{aligned}$$

Then $J_0\in C^1(E,\mathbb {R})$. By $(f_1)$–$(f_2)$ and Remark 2.1, $J_0$ is coercive, bounded below, weakly lower semicontinuous, and strictly convex. Thus, $J_0$ admits a unique minimizer $v=A(u)\in E$, which is the unique solution to (3.1). Moreover, $A$ maps bounded sets into bounded sets.

In the following, we prove that $A$ is continuous. Let $\{u_n\}\subset E$ with $u_n\rightarrow u\in E$ strongly in $E$. Let $v=A(u)$ and $v_n=A(u_n)$. We need to prove $\Vert v_n-v\Vert _E\rightarrow 0$. We have

$$\begin{aligned} \Vert v-v_n\Vert _E^2= & {} \int _{\mathbb {R}^3}(\phi _{u_n}v_n-\phi _uv)(v-v_n) +\int _{\mathbb {R}^3}(f(u)-f(u_n))(v-v_n)\\= & {} I_1+I_2. \end{aligned}$$

By Lemmas 2.1 and 2.2,

$$\begin{aligned} I_1\le & {} \int _{\mathbb {R}^3}(\phi _{u_n}v-\phi _uv)(v-v_n)\\= & {} D(u_n^2-u^2,v(v-v_n))\\\le & {} D\left( u_n^2-u^2,u_n^2-u^2\right) ^{\frac{1}{2}}D\left( v(v-v_n),v(v-v_n)\right) ^{\frac{1}{2}}\\\le & {} D\left( (u_n-u)^2,(u_n-u)^2\right) ^{\frac{1}{4}}D\left( (u_n+u)^2,(u_n+u)^2\right) ^{\frac{1}{4}}\\&\times D\left( v^2,v^2\right) ^{\frac{1}{4}}D\left( (v-v_n)^2,(v-v_n)^2\right) ^{\frac{1}{4}}\\\le & {} C_1\Vert u_n-u\Vert \Vert u_n+u\Vert \Vert v\Vert \Vert v-v_n\Vert \\\le & {} C_1\Vert u_n-u\Vert _E\Vert v-v_n\Vert _E. \end{aligned}$$

Now, we estimate the second term $I_2$. Let $\phi \in C_0^\infty (\mathbb {R})$ be such that $\phi (t)\in [0,1]$ for $t\in \mathbb {R}$, $\phi (t)=1$ for $|t|\le 1$ and $\phi (t)=0$ for $|t|\ge 2$. Setting

$$\begin{aligned} g_1(t)=\phi (t)f(t),\ \ g_2(t)=f(t)-g_1(t). \end{aligned}$$

By $(f_1)$–$(f_2)$, there exists $C_2>0$ such that $|g_1(s)|\le C_2|s|$ and $|g_2(s)|\le C_2|s|^5$ for $s\in \mathbb {R}$. Then,

$$\begin{aligned} I_2= & {} \int _{\mathbb {R}^3}(g_1(u)-g_1(u_n))(v-v_n)+\int _{\mathbb {R}^3}(g_2(u)-g_2(u_n))(v-v_n)\\\le & {} \left( \int _{\mathbb {R}^3}|g_1(u_n)-g_1(u)|^2\right) ^{\frac{1}{2}} \left( \int _{\mathbb {R}^3}|v-v_n|^2\right) ^{\frac{1}{2}}\\&+\left( \int _{\mathbb {R}^3}|g_2(u_n)-g_2(u)|^{\frac{6}{5}}\right) ^{\frac{5}{6}} \left( \int _{\mathbb {R}^3}|v-v_n|^6\right) ^{\frac{1}{6}}\\\le & {} C_3\Vert v-v_n\Vert _E\left[ \left( \int _{\mathbb {R}^3}|g_1(u_n)-g_1(u)|^2\right) ^{\frac{1}{2}} +\left( \int _{\mathbb {R}^3}|g_2(u_n)-g_2(u)|^{\frac{6}{5}}\right) ^{\frac{5}{6}}\right] . \end{aligned}$$

Thus,

$$\begin{aligned} \Vert v-v_n\Vert _E\le & {} C_4\left[ \Vert u-u_n\Vert _E +\left( \int _{\mathbb {R}^3}|g_1(u_n)-g_1(u)|^2\right) ^{\frac{1}{2}}\right. \\&\left. +\left( \int _{\mathbb {R}^3}|g_2(u_n)-g_2(u)|^{\frac{6}{5}}\right) ^{\frac{5}{6}}\right] . \end{aligned}$$

Therefore, by the dominated convergence theorem, $\Vert v-v_n\Vert _E\rightarrow 0$ as $n\rightarrow \infty$.

Finally, we show that $A$ is compact. Let $\{u_n\}\subset E$ be a bounded sequence. Then $\{v_n\}\subset E$ is a bounded sequence, where, as above, $v_n=A(u_n)$. Passing to a subsequence, by Remark 2.1, we may assume that $u_n\rightarrow u$ and $v_n\rightarrow v$ weakly in $E$ and strongly in $L^q(\mathbb {R}^3)$ as $n\rightarrow \infty$ for $q\in [2,6)$. Consider the identity

$$\begin{aligned} \int _{\mathbb R^3}\left( \nabla v_n\cdot \nabla \xi +Vv_n\xi +\phi _{u_n}v_n\xi \right) =\int _{\mathbb R^3}f(u_n)\xi ,\quad \xi \in E. \end{aligned}$$

(3.2)

Since $u_n\rightarrow u$ strongly in $L^\frac{12}{5}(\mathbb R^3)$, it follows from Lemma 2.1 (4) and the Sobolev imbedding theorem that $\phi _{u_n}\rightarrow \phi _u$ strongly in $L^6(\mathbb R^3)$. Since, in addition, $v_n\rightarrow v$ strongly in $L^\frac{12}{5}(\mathbb R^3)$, using the Hölder inequality, we have

$$\begin{aligned} \left| \int _{\mathbb R^3}(\phi _{u_n}v_n-\phi _uv)\xi \right| \le \Vert \phi _{u_n}\Vert _6\Vert v_n-v\Vert _\frac{12}{5}\Vert \xi \Vert _\frac{12}{5} +\Vert \phi _{u_n}-\phi _u\Vert _6\Vert v\Vert _\frac{12}{5}\Vert \xi \Vert _\frac{12}{5}\rightarrow 0 \end{aligned}$$

for any $\xi \in E$. Taking limit as $n\rightarrow \infty$ in (3.2) yields

$$\begin{aligned} \int _{\mathbb R^3}\left( \nabla v\cdot \nabla \xi +Vv\xi +\phi _{u}v\xi \right) =\int _{\mathbb R^3}f(u)\xi ,\quad \xi \in E. \end{aligned}$$

This means $v=A(u)$ and thus

$$\begin{aligned} \Vert v-v_n\Vert _E^2=\int _{\mathbb {R}^3}\left( \phi _uv(v_n-v)-\phi _{u_n}v_n(v_n-v)\right) +\int _{\mathbb {R}^3}(f(u_n)-f(u))(v_n-v). \end{aligned}$$

Hence, in the same way as above, $\Vert v-v_n\Vert _E\rightarrow 0$, i.e., $A(u_n)\rightarrow A(u)$ in $E$ as $n\rightarrow \infty$. $\square$

Remark 3.1

Obviously, if $f$ is odd then $A$ is odd.

Lemma 3.2

• (1) $\langle I'(u),u-A(u)\rangle \ge \Vert u-A(u)\Vert _E^2$ for all $u\in E$;

• (2) $\Vert I'(u)\Vert \le \Vert u-A(u)\Vert _E(1+C\Vert u\Vert _E^2)$ for some $C>0$ and all $u\in E$.

Proof

Since $A(u)$ is the solution of Eq. (3.1), we see that

$$\begin{aligned} \langle I'(u),u-A(u)\rangle =\Vert u-A(u)\Vert _E^2+\int _{\mathbb {R}^3}\phi _u(u-A(u))^2, \end{aligned}$$

(3.3)

which implies $\langle I'(u),u-A(u)\rangle \ge \Vert u-A(u)\Vert _E^2$ for all $u\in E$. For any $\varphi \in E$, we have

$$\begin{aligned} \langle I'(u),\varphi \rangle&=(u-A(u),\varphi )_E+\int _{\mathbb {R}^3}\phi _u(u-A(u))\varphi \\&=(u-A(u),\varphi )_E+D(u^2,(u-A(u))\varphi ). \end{aligned}$$

By Lemmas 2.1 and 2.2,

$$\begin{aligned} \left| D(u^2,(u-A(u))\varphi )\right| \le C\Vert u\Vert _E^2\Vert u-A(u)\Vert _E\Vert \varphi \Vert _E. \end{aligned}$$

Thus, $\Vert I'(u)\Vert \le \Vert u-A(u)\Vert _E(1+C\Vert u\Vert _E^2)$ for all $u\in E$. $\square$

Lemma 3.3

For $a<b$ and $\alpha >0$, there exists $\beta >0$ such that $\Vert u-A(u)\Vert _E\ge \beta$ if $u\in E$, $I(u)\in [a,b]$ and $\Vert I'(u)\Vert \ge \alpha$.

Proof

For $u\in E$, by $(f_3)$, we have

$$\begin{aligned} I(u)&-\frac{1}{\mu }(u,u-A(u))_E\\ =&\left( \frac{1}{2}-\frac{1}{\mu }\right) \Vert u\Vert _E^2 +\left( \frac{1}{4}-\frac{1}{\mu }\right) \int _{\mathbb {R}^3}\phi _uu^2\\&+\frac{1}{\mu }\int _{\mathbb {R}^3}\phi _uu(u-A(u)) +\int _{\mathbb {R}^3}\left( \frac{1}{\mu }f(u)u-F(u)\right) \\ \ge&\left( \frac{1}{2}-\frac{1}{\mu }\right) \Vert u\Vert _E^2 +\left( \frac{1}{4}-\frac{1}{\mu }\right) \int _{\mathbb {R}^3}\phi _uu^2 +\frac{1}{\mu }\int _{\mathbb {R}^3}\phi _uu(u-A(u)). \end{aligned}$$

Then,

$$\begin{aligned} \Vert u\Vert _E^2+\int _{\mathbb {R}^3}\phi _uu^2\le C_1\left( |I(u)|+\Vert u\Vert _E\Vert u-A(u)\Vert _E+\left| \int _{\mathbb {R}^3}\phi _uu(u-A(u))\right| \right) . \end{aligned}$$

(3.4)

By Hölder’s inequality and Lemmas 2.1 and 2.2,

$$\begin{aligned} \left| \int _{\mathbb {R}^3}\phi _uu(u-A(u))\right|&\le \left( \int _{\mathbb {R}^3}\phi _u(u-A(u))^2\right) ^{\frac{1}{2}} \left( \int _{\mathbb {R}^3}\phi _uu^2\right) ^{\frac{1}{2}}\\&\le C_2\Vert u\Vert _E\Vert u-A(u)\Vert _E\left( \int _{\mathbb {R}^3}\phi _uu^2\right) ^{\frac{1}{2}}. \end{aligned}$$

Thus, it follows from (3.4) that

$$\begin{aligned} \Vert u\Vert _E^2\le C_3\left( |I(u)|+\Vert u\Vert _E\Vert u-A(u)\Vert _E+\Vert u\Vert _E^2\Vert u-A(u)\Vert _E^2\right) . \end{aligned}$$

(3.5)

If there exists $\{u_n\}\subset E$ with $I(u_n)\in [a,b]$ and $\Vert I'(u_n)\Vert \ge \alpha$ such that $\Vert u_n-A(u_n)\Vert _E\rightarrow 0$ as $n\rightarrow \infty$, then it follows from (3.5) that $\{\Vert u_n\Vert _E\}$ is bounded, and by Lemma 3.2, we see that $\Vert I'(u_n)\Vert \rightarrow 0$ as $n\rightarrow \infty$, which is a contradiction. Thus, the proof is completed. $\square$

3.2 Invariant subsets of descending flow

To obtain sign-changing solutions, we make use of the positive and negative cones as in many references such as [7, 8, 11, 23]. Precisely, define

$$\begin{aligned} P^+:=\{u\in E:u\ge 0\}\ \ \text{ and }\ \ P^-:=\{u\in E:u\le 0\}. \end{aligned}$$

Set for $\varepsilon >0$,

$$\begin{aligned} P_\varepsilon ^+:=\{u\in E: \text{ dist }(u,P^+)<\varepsilon \}\ \ \text{ and }\ \ P_\varepsilon ^-:=\{u\in E: \text{ dist }(u,P^-)<\varepsilon \}, \end{aligned}$$

where $\text{ dist }(u,P^\pm )=\inf \limits _{v\in P^\pm }\Vert u-v\Vert _E$. Obviously, $P_\varepsilon ^-=-P_\varepsilon ^+$. Let $W=P_\varepsilon ^+\cup P_\varepsilon ^-$. Then, $W$ is an open and symmetric subset of $E$ and $E{\setminus } W$ contains only sign-changing functions. On the other hand, the next lemma shows that for $\varepsilon$ small, all sign-changing solutions to (2.1) are contained in $E{\setminus } W$.

Lemma 3.4

There exists $\varepsilon _0>0$ such that for $\varepsilon \in (0,\varepsilon _0)$,

• (1) $A(\partial P_\varepsilon ^-)\subset P_\varepsilon ^-$ and every nontrivial solution $u\in P_\varepsilon ^-$ is negative,

• (2) $A(\partial P_\varepsilon ^+)\subset P_\varepsilon ^+$ and every nontrivial solution $u\in P_\varepsilon ^+$ is positive.

Proof

Since the two conclusions are similar, we only prove the first one. By $(f_1)$–$(f_2)$, for any fixed $\delta >0$, there exists $C_\delta >0$ such that

$$\begin{aligned} |f(t)|\le \delta |t|+C_\delta |t|^p,\quad t\in \mathbb {R}. \end{aligned}$$

Let $u\in E$ and $v=A(u)$. By Remark 2.1, for any $q\in [2,6]$, there exists $m_q>0$ such that

$$\begin{aligned} \Vert u^\pm \Vert _q=\inf \limits _{w\in P^\mp }\Vert u-w\Vert _q\le m_q\inf \limits _{w\in P^\mp }\Vert u-w\Vert _E=m_q\text{ dist }(u,P^\mp ). \end{aligned}$$

(3.6)

Obviously, $\text{ dist }(v,P^-)\le \Vert v^+\Vert _E$. Then, by $(f_3)$, we estimate

$$\begin{aligned} \text{ dist }(v,P^-)\Vert v^+\Vert _E&\le \Vert v^+\Vert _E^2=(v,v^+)_E\\&=\int _{\mathbb {R}^3}\left( f(u)v^+-\phi _uvv^+\right) \\&\le \int _{\mathbb {R}^3}f(u)v^+\le \int _{\mathbb {R}^3}f(u^+)v^+\\&\le \int _{\mathbb {R}^3}\left( \delta |u^+|+C_\delta |u^+|^{p-1}\right) |v^+|\\&\le \delta \Vert u^+\Vert _2\Vert v^+\Vert _2+C_\delta \Vert u^+\Vert _p^{p-1}\Vert v^+\Vert _{p}\\&\le C\left( \delta \text{ dist }(u,P^-)+C_\delta \text{ dist }(u,P^-)^{p-1}\right) \Vert v^+\Vert _E. \end{aligned}$$

It follows that

$$\begin{aligned} \text{ dist }(A(u),P^-)\le C\left( \delta \text{ dist }(u,P^-)+C_\delta \text{ dist }(u,P^-)^{p-1}\right) . \end{aligned}$$

Thus, choosing $\delta$ small enough, there exists $\varepsilon _0>0$ such that for $\varepsilon \in (0,\varepsilon _0)$,

$$\begin{aligned} \text{ dist }(A(u),P^-)\le \frac{1}{2}\text{ dist }(u,P^-) \quad \text{ for } \text{ any }\ \ u\in P_\varepsilon ^-. \end{aligned}$$

This implies that $A(\partial P_\varepsilon ^-)\subset P_\varepsilon ^-$. If there exists $u\in P_\varepsilon ^-$ such that $A(u)=u$, then $u\in P^-$. If $u\not \equiv 0$, by the maximum principle, $u<0$ in $\mathbb {R}^3$. $\square$

Denote the set of fixed points of $A$ by $K$, which is exactly the set of critical points of $I$. Since $A$ is merely continuous, $A$ itself is not applicable to construct a descending flow for $I$, and we have to construct a locally Lipschitz continuous operator $B$ on $E_0:=E{\setminus } K$ which inherits the main properties of $A$.

Lemma 3.5

There exists a locally Lipschitz continuous operator $B:E_0\rightarrow E$ such that

• (1) $B(\partial P_\varepsilon ^+)\subset P_\varepsilon ^+$ and $B(\partial P_\varepsilon ^-)\subset P_\varepsilon ^-$ for $\varepsilon \in (0,\varepsilon _0)$;

• (2) $\frac{1}{2}\Vert u-B(u)\Vert _E\le \Vert u-A(u)\Vert _E\le 2\Vert u-B(u)\Vert _E$ for all $u\in E_0$;

• (3) $\langle I'(u),u-B(u)\rangle \ge \frac{1}{2}\Vert u-A(u)\Vert _E^2$ for all $u\in E_0$;

• (4) if $f$ is odd then $B$ is odd.

Proof

The proof is similar to the proofs of [6, Lemma 4.1] and [8, Lemma 2.1]. We omit the details. $\square$

3.3 Existence of one sign-changing solution

In this subsection, we will find one sign-changing solution of (2.1) via minimax method incorporated with invariant sets of descending flow. First of all, we introduce the critical point theorem [23, Theorem 2.4]. For more details, we refer to [23].

Let $X$ be a Banach space, $J\in C^1(X,\mathbb {R})$, $P,Q\subset X$ be open sets, $M=P\cap Q$, $\Sigma =\partial P\cap \partial Q$ and $W=P\cup Q$. For $c\in \mathbb {R}$, $K_c=\{x\in X: J(x)=c, J'(x)=0\}$ and $J^c=\{x\in X: J(x)\le c\}$. In [23], a critical point theory on metric spaces was given, but here we only need a Banach space version of the theory.

Definition 3.1

([23]) $\{P,Q\}$ is called an admissible family of invariant sets with respect to $J$ at level $c$ provided that the following deformation property holds: if $K_c{\setminus } W=\emptyset$, then, there exists $\varepsilon _0>0$ such that for $\varepsilon \in (0,\varepsilon _0)$, there exists $\eta \in C(X,X)$ satisfying

• (1) $\eta (\overline{P})\subset \overline{P}$, $\eta (\overline{Q})\subset \overline{Q}$;

• (2) $\eta \mid _{J^{c-2\varepsilon }}=id$;

• (3) $\eta (J^{c+\varepsilon }{\setminus } W)\subset J^{c-\varepsilon }$.

Theorem A

([23]) Assume that $\{P,Q\}$ is an admissible family of invariant sets with respect to $J$ at any level $c\ge c_{*}:=\inf _{u\in \Sigma }J(u)$ and there exists a map $\varphi _0:\Delta \rightarrow X$ satisfying

• (1) $\varphi _0(\partial _1\Delta )\subset P$ and $\varphi _0(\partial _2\Delta )\subset Q$,

• (2) $\varphi _0(\partial _0\Delta )\cap M=\emptyset$,

• (3) $\sup \nolimits _{u\in \varphi _0(\partial _0\Delta )}J(u)<c_*$,

where $\Delta =\{(t_1,t_2)\in \mathbb {R}^2:t_1,t_2\ge 0,\ t_1+t_2\le 1\}$, $\partial _1\Delta =\{0\}\times [0,1]$, $\partial _2\Delta =[0,1]\times \{0\}$ and $\partial _0\Delta =\{(t_1,t_2)\in \mathbb {R}^2:t_1,t_2\ge 0,\ t_1+t_2=1\}$. Define

$$\begin{aligned} c=\inf \limits _{\varphi \in \Gamma }\sup \limits _{u\in \varphi (\triangle ){\setminus } W}J(u), \end{aligned}$$

where $\Gamma :=\left\{ \varphi \in C(\triangle ,X):\varphi (\partial _1\triangle )\subset P,\ \varphi (\partial _2\triangle )\subset Q,\ \varphi |_{\partial _0\triangle }=\varphi _0|_{\partial _0\triangle }\right\} .$ Then $c\ge c_*$ and $K_c{\setminus } W\not =\emptyset$.

Now, we use Theorem A to prove the existence of a sign-changing solution to problem (2.1), and for this, we take $X=E$, $P=P_\varepsilon ^+$, $Q=P_\varepsilon ^-$, and $J=I$. We will show that $\{P_\varepsilon ^+,P_\varepsilon ^-\}$ is an admissible family of invariant sets for the functional $I$ at any level $c\in \mathbb {R}$. Indeed, if $K_c{\setminus } W=\emptyset$, then $K_c\subset W$. Since $\mu >4$, by Remark 2.1, it is easy to see that $I$ satisfies the (PS)-condition and therefore $K_c$ is compact. Thus, $2\delta :=\text{ dist }(K_c,\partial W)>0$.

Lemma 3.6

If $K_c{\setminus } W=\emptyset$, then there exists $\varepsilon _0>0$ such that, for $0<\varepsilon <\varepsilon '<\varepsilon _0$, there exists a continuous map $\sigma :[0,1]\times E\rightarrow E$ satisfying

• (1) $\sigma (0,u)=u$ for $u\in E$;

• (2) $\sigma (t,u)=u$ for $t\in [0,1]$, $u\not \in I^{-1}[c-\varepsilon ',c+\varepsilon ']$;

• (3) $\sigma (1,I^{c+\varepsilon }{\setminus } W)\subset I^{c-\varepsilon }$;

• (4) $\sigma (t,\overline{P_\varepsilon ^+})\subset \overline{P_\varepsilon ^+}$ and $\sigma (t,\overline{P_\varepsilon ^-})\subset \overline{P_\varepsilon ^-}$ for $t\in [0,1]$.

Proof

The proof is similar to the proof of [23, Lemma 3.5]. For the sake of completeness, we give the details here. For $G\subset E$ and $a>0$, let $N_a(G):=\{u\in E: \text{ dist }(u,G)<a\}$. Then $N_\delta (K_c)\subset W$. Since $I$ satisfies the (PS)-condition, there exist $\varepsilon _0, \alpha >0$ such that

$$\begin{aligned} \Vert I'(u)\Vert \ge \alpha \quad \text{ for }\ \ u\in I^{-1}([c-\varepsilon _0,c+\varepsilon _0]){\setminus } N_\frac{\delta }{2}(K_c). \end{aligned}$$

By Lemmas 3.3 and 3.5, there exists $\beta >0$ such that

$$\begin{aligned} \Vert u-B(u)\Vert _E\ge \beta \quad \text{ for }\ \ u\in I^{-1}([c-\varepsilon _0,c+\varepsilon _0]){\setminus } N_\frac{\delta }{2}(K_c). \end{aligned}$$

Without loss of generality, assume that $\varepsilon _0\le \frac{\beta \delta }{32}$. Let

$$\begin{aligned} V(u)=\frac{u-B(u)}{\Vert u-B(u)\Vert _E}\quad \text{ for }\ \ u\in E_0=E{\setminus } K, \end{aligned}$$

and take a cut-off function $g: E\rightarrow [0,1]$, which is locally Lipschitz continuous, such that

$$\begin{aligned} g(u)=\left\{ \begin{array}{lll} 0,\quad \text{ if }\ \ u\not \in I^{-1}[c-\varepsilon ',c+\varepsilon ']\ \ \text{ or }\ \ u\in N_{\frac{\delta }{4}}(K_c),\\ 1,\quad \text{ if }\ \ u\in I^{-1}[c-\varepsilon ,c+\varepsilon ]\ \ \text{ and }\ \ u\not \in N_{\frac{\delta }{2}}(K_c). \end{array} \right. \end{aligned}$$

Decreasing $\varepsilon _0$ if necessary, one may find a $\nu >0$ such that $I^{-1}[c-\varepsilon _0,c+\varepsilon _0]\cap N_\nu (K)\subset N_{\delta /4}(K_c)$, and this can be seen as a consequence of the (PS) condition. Thus, $g(u)=0$ for any $u\in N_\nu (K)$. By Lemma 3.5, $g(\cdot )V(\cdot )$ is locally Lipschitz continuous on $E$.

Consider the following initial value problem

$$\begin{aligned} \left\{ \begin{array}{lll} \frac{\hbox {d}\tau }{\hbox {d}t}=-g(\tau )V(\tau ),\\ \tau (0,u)=u. \end{array} \right. \end{aligned}$$

(3.7)

For any $u\in E$, one sees that problem (3.7) admits a unique solution $\tau (\cdot ,u)\in C(\mathbb {R}^+,E)$. Define $\sigma (t,u)=\tau (\frac{16\varepsilon }{\beta }t,u)$. It suffices to check (3) and (4) since $(1)$ and $(2)$ are obvious.

To verify $(3)$, we let $u\in I^{c+\varepsilon }{\setminus } W$. By Lemma 3.5, $I(\tau (t,u))$ is decreasing for $t\ge 0$. If there exists $t_0\in [0,\frac{16\varepsilon }{\beta }]$ such that $I(\tau (t_0,u))<c-\varepsilon$ then $I(\sigma (1,u))=I\left( \tau \left( \frac{16\varepsilon }{\beta },u\right) \right) <c-\varepsilon$. Otherwise, for any $t\in [0,\frac{16\varepsilon }{\beta }]$, $I(\tau (t,u))\ge c-\varepsilon$. Then, $\tau (t,u)\in I^{-1}[c-\varepsilon ,c+\varepsilon ]$ for $t\in [0,\frac{16\varepsilon }{\beta }]$. We claim that for any $t\in \left[ 0,\frac{16\varepsilon }{\beta }\right]$, $\tau (t,u)\not \in N_{\frac{\delta }{2}}(K_c)$. If, for some $t_1\in \left[ 0,\frac{16\varepsilon }{\beta }\right]$, $\tau (t_1,u)\in N_{\frac{\delta }{2}}(K_c)$, then, since $u\not \in N_\delta (K_c)$,

$$\begin{aligned} \frac{\delta }{2}\le \Vert \tau (t_1,u)-u\Vert _E\le \int _0^{t_1}\Vert \tau '(s,u)\Vert _Eds\le t_1\le \frac{16\varepsilon }{\beta }, \end{aligned}$$

which contradicts the fact that $\varepsilon <\varepsilon _0\le \frac{\beta \delta }{32}$. So $g(\tau (t,u))\equiv 1$ for $t\in \left[ 0,\frac{16\varepsilon }{\beta }\right]$. Then by (2) and (3) of Lemma 3.5,

$$\begin{aligned} I\left( \tau \left( \frac{16\varepsilon }{\beta },u\right) \right)&=I(u)-\int _0^{\frac{16\varepsilon }{\beta }}\left\langle I'(\tau (s,u)),V(\tau (s,u))\right\rangle \\&\le I(u)-\int _0^{\frac{16\varepsilon }{\beta }}\frac{1}{8}\Vert \tau (s,u)-B\tau (s,u)\Vert _E\\&\le c+\varepsilon -\frac{16\varepsilon }{\beta }\frac{\beta }{8}=c-\varepsilon . \end{aligned}$$

Finally, $(4)$ is a consequence of (1) of Lemma 3.5 (see [24] for a detailed proof). $\square$

Corollary 3.1

$\{P_\varepsilon ^+,P_\varepsilon ^-\}$ is an admissible family of invariant sets for the functional $I$ at any level $c\in \mathbb {R}$.

Proof

The conclusion follows from Lemma 3.6. $\square$

In the following, we will construct $\varphi _0$ satisfying the hypotheses in Theorem A. Choose $v_1,v_2\in C_0^\infty (\mathbb R^3){\setminus }\{0\}$ satisfying $\text{ supp }(v_1)\cap \text{ supp }(v_2)=\emptyset$ and $v_1\le 0,v_2\ge 0$. Let $\varphi _0(t,s):=R(tv_1+sv_2)$ for $(t,s)\in \Delta$, where $R$ is a positive constant to be determined later. Obviously, for $t,s\in [0,1]$, $\varphi _0(0,s)=Rsv_2\in P_\varepsilon ^+$ and $\varphi _0(t,0)=Rtv_1\in P_\varepsilon ^-$.

Lemma 3.7

For $q\in [2,6]$, there exists $m_q>0$ independent of $\varepsilon$ such that $\Vert u\Vert _q\le m_q\varepsilon$ for $u\in M=P_\varepsilon ^+\cap P_\varepsilon ^-$.

Proof

This follows from (3.6). $\square$

Lemma 3.8

If $\varepsilon >0$ is small enough, then $I(u)\ge \frac{\varepsilon ^2}{2}$ for $u\in \Sigma =\partial P_\varepsilon ^+\cap \partial P_\varepsilon ^-$, that is, $c_*\ge \frac{\varepsilon ^2}{2}$.

Proof

For $u\in \partial P_\varepsilon ^+\cap \partial P_\varepsilon ^-$, we have $\Vert u^\pm \Vert _E\ge \text{ dist }(u,P^\mp )=\varepsilon$. By $(f_1)$–$(f_2)$, we have $F(t)\le \frac{1}{3m_2^2}|t|^2+C_1|t|^{p}$ for all $t\in \mathbb {R}$. Then, using Lemma 3.7, we see that

$$\begin{aligned} I(u)\ge \varepsilon ^2-\frac{1}{3}\varepsilon ^2-C_2\varepsilon ^{p}\ge \frac{\varepsilon ^2}{2}, \end{aligned}$$

for $\varepsilon$ small enough. $\square$

Proof of Theorem 1.1 (Existence part)

It suffices to verify assumptions $(2)$–$(3)$ in applying Theorem A. Observe that $\rho =\min \{\Vert tv_1+(1-t)v_2\Vert _2:\ 0\le t\le 1\}>0$. Then, $\Vert u\Vert _2\ge \rho R$ for $u\in \varphi _0(\partial _0\Delta )$ and it follows from Lemma 3.7 that $\varphi _0(\partial _0\Delta )\cap M=\emptyset$ for $R$ large enough. By $(f_3)$, we have $F(t)\ge C_1|t|^\mu -C_2$ for any $t\in \mathbb {R}$. For any $u\in \varphi _0(\partial _0\Delta )$, by Lemma 2.1,

$$\begin{aligned} I(u)&\le \frac{1}{2}\Vert u\Vert _E^2+C_3\Vert u\Vert _E^4 -\int _{\text{ supp }(v_1)\cup \text{ supp }(v_2)}F(u)\\&\le \frac{1}{2}\Vert u\Vert _E^2+C_3\Vert u\Vert _E^4-C_1\Vert u\Vert _\mu ^\mu +C_4, \end{aligned}$$

which together with Lemma 3.8 implies that, for $R$ large enough and $\varepsilon$ small enough,

$$\begin{aligned} \sup \limits _{u\in \varphi _0(\partial _0\Delta )}I(u)<0<c_*. \end{aligned}$$

According to Theorem A, $I$ has at least one critical point $u$ in $E{\setminus }(P_\varepsilon ^+\cup P_\varepsilon ^-)$, which is a sign-changing solution of Eq. (2.1). Then $(u,\phi _u)$ is a sign-changing solution of system (1.1). $\square$

3.4 Existence of infinitely many sign-changing solutions

In this section, we prove the existence of infinitely many sign-changing solutions to system (1.1). For this, we will make use of [23, Theorem 2.5], which we recall below.

We will use the notations from Sect. 3.3. Assume $G:X\rightarrow X$ to be an isometric involution, that is, $G^2=id$ and $d(Gx,Gy)=d(x,y)$ for $x,y\in X$. We assume $J$ is $G$-invariant on $X$ in the sense that $J(Gx)=J(x)$ for any $x\in X$. We also assume $Q=GP$. A subset $F\subset X$ is said to be symmetric if $Gx\in F$ for any $x\in F$. The genus of a closed symmetric subset $F$ of $X{\setminus }\{0\}$ is denoted by $\gamma (F)$.

Definition 3.2

([23]) $P$ is called a $G$-admissible invariant set with respect to $J$ at level $c$, if the following deformation property holds, there exist $\varepsilon _0>0$ and a symmetric open neighborhood $N$ of $K_c{\setminus } W$ with $\gamma (\overline{N})<\infty$, such that for $\varepsilon \in (0,\varepsilon _0)$ there exists $\eta \in C(X,X)$ satisfying

• (1) $\eta (\overline{P})\subset \overline{P}$, $\eta (\overline{Q})\subset \overline{Q}$;

• (2) $\eta \circ G=G\circ \eta$;

• (3) $\eta \mid _{J^{c-2\varepsilon }}=id$;

• (4) $\eta (J^{c+\varepsilon }{\setminus }(N\cup W))\subset J^{c-\varepsilon }$.

Theorem B

([23]) Assume that $P$ is a $G$-admissible invariant set with respect to $J$ at any level $c\ge c^*:=\inf _{u\in \Sigma }J(u)$ and for any $n\in \mathbb {N}$, there exists a continuous map $\varphi _n:B_n:=\{x\in \mathbb {R}^n:|x|\le 1\}\rightarrow X$ satisfying

• (1) $\varphi _n(0)\in M:=P\cap Q$, $\varphi _n(-t)=G\varphi _n(t)$ for $t\in B_n$,

• (2) $\varphi _n(\partial B_n)\cap M=\emptyset$,

• (3) $\sup _{u\in \mathrm{Fix}_G\cup \varphi _n(\partial B_n)}J(u)<c^*,$ where $\mathrm{Fix}_G:=\{u\in X:Gu=u\}$.

For $j\in \mathbb {N}$, define

$$\begin{aligned} c_j=\inf \limits _{B\in \Gamma _j}\sup \limits _{u\in B{\setminus } W}J(u), \end{aligned}$$

where

$$\begin{aligned} \Gamma _j:=\left\{ B\Big |\ \begin{array}{ll} B=\varphi (B_n{\setminus } Y)\, \mathrm { for\,some}\, \varphi \in G_n,\, n\ge j,\,\mathrm {and\,open}\, Y\subset B_n\\ \mathrm {such\,that}\, -Y=Y\, \mathrm {and}\, \gamma (\bar{Y})\le n-j \end{array}\right\} \end{aligned}$$

and

$$\begin{aligned} G_n:=\left\{ \varphi \Big |\ \begin{array}{ll} \varphi \in C(B_n,X),\ \varphi (-t)=G\varphi (t)\,\mathrm {for}\,t\in B_n,\\ \varphi (0)\in M\,\mathrm {and}\, \varphi |_{\partial B_n}=\varphi _n|_{\partial B_n} \end{array}\right\} . \end{aligned}$$

Then for $j\ge 2$, $c_j\ge c_*$, $K_{c_j}{\setminus } W\not =\emptyset$ and $c_j\rightarrow \infty$ as $j\rightarrow \infty$.

To apply Theorem B, we take $X=E$, $G=-id$, $J=I$ and $P=P_\varepsilon ^+$. Then $M=P_\varepsilon ^+\cap P_\varepsilon ^-$, $\Sigma =\partial P_\varepsilon ^+\cap \partial P_\varepsilon ^-$, and $W=P_\varepsilon ^+\cup P_\varepsilon ^-$. In this subsection, $f$ is assumed to be odd, and, as a consequence, $I$ is even. Now, we show that $P_\varepsilon ^+$ is a $G$-admissible invariant set for the functional $I$ at any level $c$. Since $K_c$ is compact, there exists a symmetric open neighborhood $N$ of $K_c{\setminus } W$ such that $\gamma (\overline{N})<\infty$.

Lemma 3.9

There exists $\varepsilon _0>0$ such that for $0<\varepsilon <\varepsilon '<\varepsilon _0$, there exists a continuous map $\sigma :[0,1]\times E\rightarrow E$ satisfying

• (1) $\sigma (0,u)=u$ for $u\in E$.

• (2) $\sigma (t,u)=u$ for $t\in [0,1]$, $u\not \in I^{-1}[c-\varepsilon ',c+\varepsilon ']$.

• (3) $\sigma (t,-u)=-\sigma (t,u)$ for $(t,u)\in [0,1]\times E$.

• (4) $\sigma (1,I^{c+\varepsilon }{\setminus } (N\cup W))\subset I^{c-\varepsilon }$.

• (5) $\sigma (t,\overline{P_\varepsilon ^+})\subset \overline{P_\varepsilon ^+}$, $\sigma (t,\overline{P_\varepsilon ^-})\subset \overline{P_\varepsilon ^-}$ for $t\in [0,1]$.

Proof

The proof is similar to the proof of Lemma 3.6. Since $I$ is even, $B$ is odd and thus $\sigma$ is odd in $u$.

Corollary 3.2

$P_\varepsilon ^+$ is a $G$-admissible invariant set for the functional $I$ at any level $c$.

Proof of Theorem 1.1 (Multiplicity part)

According to Theorem B, if $\varphi _n$ exists and satisfies the assumptions in Theorem B then $I$ has infinitely many critical points in $E{\setminus }(P_\varepsilon ^+\cup P_\varepsilon ^-)$, which are sign-changing solutions to (2.1) and thus yield sign-changing solution to (1.1). It suffices to construct $\varphi _n$. For any $n\in \mathbb {N}$, choose $\{v_i\}_1^n\subset C_0^\infty (\mathbb R^3){\setminus }\{0\}$ such that $\text{ supp }(v_i)\cap \text{ supp }(v_j)=\emptyset$ for $i\not =j$. We define $\varphi _n\in C(B_n,E)$ as

$$\begin{aligned} \varphi _n(t)=R_n\sum _{i=1}^nt_iv_i,\quad t=(t_1,t_2,\cdots ,t_n)\in B_n, \end{aligned}$$

where $R_n>0$. For $R_n$ large enough, it is easy to check that all the assumptions of Theorem B are satisfied. $\square$

4 Proof of Theorem 1.2

In this section, we do not assume $\mu >4$ and thus the argument of Sect. 3 which essentially depends on the assumption $\mu >4$ is not valid in the present case. This obstacle will be overcome via a perturbation approach which is originally due to [23]. The method from Sect. 3 can be used for the perturbed problem. By passing to the limit, we then obtain sign-changing solutions of the original problem (1.1).

Fix a number $r\in (\max \{4,p\},6)$. For any fixed $\lambda \in (0,1]$, we consider the modified problem

$$\begin{aligned} -\Delta u+V(x)u+\phi _uu=f(u)+\lambda |u|^{r-2}u,\quad u\in E \end{aligned}$$

(4.1)

and its associated functional

$$\begin{aligned} I_\lambda (u)=I(u)-\frac{\lambda }{r}\int _{{\mathbb {R}}^N}|u|^r. \end{aligned}$$

It is standard to show that $I_\lambda \in C^1(E,\mathbb {R})$ and

$$\begin{aligned} \langle I_\lambda '(u),v\rangle =\langle I'(u),v\rangle -\lambda \int _{{\mathbb {R}}^N}|u|^{r-2}uv,\quad u,\ v\in E. \end{aligned}$$

For any $u\in E$, we denote by $v=A_\lambda (u)\in E$ the unique solution to the problem

$$\begin{aligned} -\Delta v+V(x)v+\phi _uv=f(u)+\lambda |u|^{r-2}u,\quad v\in E. \end{aligned}$$

As in Sect. 3, one verifies that the operator $A_\lambda :E\rightarrow E$ is well defined and is continuous and compact. In the following, if the proof of a result is similar to its counterpart in Sect. 3, it will not be written out.

Lemma 4.1

• (1) $\langle I_\lambda '(u),u-A_\lambda (u)\rangle \ge \Vert u-A_\lambda (u)\Vert _E^2$ for all $u\in E$;

• (2) there exists $C>0$ independent of $\lambda$ such that $\Vert I_\lambda '(u)\Vert \le \Vert u-A_\lambda (u)\Vert _E(1+C\Vert u\Vert _E^2)$ for all $u\in E$.

Lemma 4.2

For any $\lambda \in (0,1)$, $a<b$ and $\alpha >0$, there exists $\beta (\lambda )>0$ such that $\Vert u-A_\lambda (u)\Vert _E\ge \beta (\lambda )$ for any $u\in E$ with $I_\lambda (u)\in [a,b]$ and $\Vert I_\lambda '(u)\Vert \ge \alpha$.

Proof

Fix a number $\gamma \in (4,r)$. For $u\in E$,

$$\begin{aligned}&I_\lambda (u)-\frac{1}{\gamma }(u,u-A_\lambda (u))_E\\&\quad =\left( \frac{1}{2}-\frac{1}{\gamma }\right) \Vert u\Vert _E^2 +\left( \frac{1}{4}-\frac{1}{\gamma }\right) \int _{\mathbb {R}^3}\phi _uu^2\\&\qquad +\frac{1}{\gamma }\int _{\mathbb {R}^3}\phi _uu(u-A_\lambda (u)) +\int _{\mathbb {R}^3}\left( \frac{1}{\gamma }f(u)u-F(u)\right) +\lambda \left( \frac{1}{\gamma }-\frac{1}{r}\right) \Vert u\Vert _r^r. \end{aligned}$$

Then, by $(f_1)$–$(f_2)$,

$$\begin{aligned}&\Vert u\Vert _E^2+\int _{\mathbb {R}^3}\phi _uu^2+\lambda \Vert u\Vert _r^r\\&\quad \le C_1\left( |I_\lambda (u)|+\Vert u\Vert _E\Vert u-A_\lambda (u)\Vert _E+\Vert u\Vert _p^p +\left| \int _{\mathbb {R}^3}\phi _uu(u-A_\lambda (u))\right| \right) . \end{aligned}$$

Since

$$\begin{aligned} \left| \int _{\mathbb {R}^3}\phi _uu(u-A_\lambda (u))\right| \le C_2\Vert u\Vert _E\Vert u-A_\lambda (u)\Vert _E\left( \int _{\mathbb {R}^3}\phi _uu^2\right) ^{\frac{1}{2}}, \end{aligned}$$

one sees that

$$\begin{aligned}&\Vert u\Vert _E^2+\int _{\mathbb {R}^3}\phi _uu^2+\lambda \Vert u\Vert _r^r\nonumber \\&\quad \le C_3\left( |I_\lambda (u)|+\Vert u\Vert _p^p+\Vert u\Vert _E\Vert u-A_\lambda (u)\Vert _E +\Vert u\Vert _E^2\Vert u-A_\lambda (u)\Vert _E^2\right) . \end{aligned}$$

(4.2)

If there exists $\{u_n\}\subset E$ with $I_\lambda (u_n)\in [a,b]$ and $\Vert I_\lambda '(u_n)\Vert \ge \alpha$ such that $\Vert u_n-A_\lambda (u_n)\Vert _E\rightarrow 0$ as $n\rightarrow \infty$, then it follows from (4.2) that, for large $n$,

$$\begin{aligned} \Vert u_n\Vert _E^2+\int _{\mathbb {R}^3}\phi _{u_n}u_n^2+\lambda \Vert u_n\Vert _r^r\le C_4(1+\Vert u_n\Vert _p^p). \end{aligned}$$

Claim: $\{u_n\}$ is bounded in $E$. Otherwise, assume that $\Vert u_n\Vert _E\rightarrow \infty$ as $n\rightarrow \infty$. Then

$$\begin{aligned} \Vert u_n\Vert _E^2+\int _{\mathbb {R}^3}\phi _{u_n}u_n^2+\lambda \Vert u_n\Vert _r^r\le C_5\Vert u_n\Vert _p^p. \end{aligned}$$

(4.3)

By (4.3), there exists $C(\lambda )>0$ such that for large $n$,

$$\begin{aligned} \Vert u_n\Vert _2^2+\Vert u_n\Vert _r^r\le C(\lambda )\Vert u_n\Vert _p^p. \end{aligned}$$

Let $t\in (0,1)$ be such that

$$\begin{aligned} \frac{1}{p}=\frac{t}{2}+\frac{1-t}{r}. \end{aligned}$$

Then, by the interpolation inequality,

$$\begin{aligned} \Vert u_n\Vert _2^2+\Vert u_n\Vert _r^r\le C(\lambda )\Vert u_n\Vert _p^p\le C(\lambda )\Vert u_n\Vert _2^{tp}\Vert u_n\Vert _r^{(1-t)p}, \end{aligned}$$

from which it follows that there exist $C_1(\lambda ),C_2(\lambda )>0$ such that, for large $n$,

$$\begin{aligned} C_1(\lambda )\Vert u_n\Vert _2^{\frac{2}{r}}\le \Vert u_n\Vert _r\le C_2(\lambda )\Vert u_n\Vert _2^{\frac{2}{r}}. \end{aligned}$$

Thus $\Vert u_n\Vert _p^p\le C_3(\lambda )\Vert u_n\Vert _2^2$ and, by (4.3) again,

$$\begin{aligned} \Vert u_n\Vert _E^2+\int _{\mathbb {R}^3}\phi _{u_n}u_n^2+\lambda \Vert u_n\Vert _r^r\le C_4(\lambda )\Vert u_n\Vert _2^2. \end{aligned}$$

Let $w_n=\frac{u_n}{\Vert u_n\Vert _E}$. The last inequality implies that

$$\begin{aligned} \Vert w_n\Vert _2^2\ge (C_4(\lambda ))^{-1} \end{aligned}$$

(4.4)

and

$$\begin{aligned} \int _{\mathbb {R}^3}\phi _{w_n}w_n^2\le C_5(\lambda )\Vert u_n\Vert _E^{-2}. \end{aligned}$$

(4.5)

From (4.5), we have $\int _{\mathbb {R}^3}\phi _{w_n}w_n^2\rightarrow 0$ as $n\rightarrow \infty$. Since $\Vert w_n\Vert _E=1$, we assume that $w_n\rightarrow w$ weakly in $E$ and strongly both in $L^\frac{12}{5}(\mathbb {R}^3)$ and in $L^2(\mathbb {R}^3)$. Note that

$$\begin{aligned} \left| \int _{\mathbb {R}^3}(\phi _{w_n}w_n^2-\phi _ww^2)\right| \le&\int _{\mathbb {R}^3}|\phi _{w_n}-\phi _w|w_n^2+\int _{\mathbb {R}^3}\phi _w|w_n^2-w^2|\\ \le&\Vert \phi _{w_n}-\phi _w\Vert _6\Vert w_n\Vert _{\frac{12}{5}}^2 +\Vert \phi _w\Vert _6\Vert w_n-w\Vert _{\frac{12}{5}}\Vert w_n+w\Vert _{\frac{12}{5}}. \end{aligned}$$

Since $w_n\rightarrow w$ strongly in $L^\frac{12}{5}(\mathbb {R}^3)$ and, by Lemma 2.1, $\phi _{w_n}\rightarrow \phi _w$ strongly in $L^6(\mathbb {R}^3)$, we have

$$\begin{aligned} \int _{\mathbb {R}^3}\phi _ww^2=\lim _{n\rightarrow \infty }\int _{\mathbb {R}^3}\phi _{w_n}w_n^2=0, \end{aligned}$$

which implies $w=0$. But (4.4) implies $\Vert w\Vert _2^2\ge (C_4(\lambda ))^{-1}$, and thus we have a contradiction and finish the proof of the claim. The claim combined with Lemma 4.1 implies $\Vert I_\lambda '(u_n)\Vert \rightarrow 0$ as $n\rightarrow \infty$, which is again a contradiction. $\square$

Lemma 4.3

There exists $\varepsilon _1>0$ independent of $\lambda$ such that for $\varepsilon \in (0,\varepsilon _1)$,

• (1) $A_\lambda (\partial P_\varepsilon ^-)\subset P_\varepsilon ^-$ and every nontrivial solution $u\in P_\varepsilon ^-$ is negative.

• (2) $A_\lambda (\partial P_\varepsilon ^+)\subset P_\varepsilon ^+$ and every nontrivial solution $u\in P_\varepsilon ^+$ is positive.

Lemma 4.4

There exists a locally Lipschitz continuous map $B_\lambda : E{\setminus } K_\lambda \rightarrow E$, where $K_\lambda :=\mathrm{Fix}(A_\lambda )$, such that

• (1) $B_\lambda (\partial P_\varepsilon ^+)\subset P_\varepsilon ^+$, $B_\lambda (\partial P_\varepsilon ^-)\subset P_\varepsilon ^-$ for $\varepsilon \in (0,\varepsilon _1)$;

• (2) $\frac{1}{2}\Vert u-B_\lambda (u)\Vert _E\le \Vert u-A_\lambda (u)\Vert _E\le 2\Vert u-B_\lambda (u)\Vert _E$ for all $u\in E{\setminus } K_\lambda$;

• (3) $\langle I_\lambda '(u),u-B_\lambda (u)\rangle \ge \frac{1}{2}\Vert u-A_\lambda (u)\Vert _E^2$ for all $u\in E{\setminus } K_\lambda$;

• (4) if $f$ is odd then $B_\lambda$ is odd.

We are ready to prove Theorem 1.2.

Proof of Theorem 1.2 (Existence part)

Step 1. We use Theorem A for $J=I_\lambda$. We claim that $\{P_\varepsilon ^+,P_\varepsilon ^-\}$ is an admissible family of invariant sets for the functional $I_\lambda$ at any level $c$. In view of the approach in Sect. 3 and the fact that we have already had Lemmas 4.1–4.4, we need only to prove that for any fixed $\lambda \in (0,1)$, $I_\lambda$ satisfies the (PS)-condition. Assume that there exist $\{u_n\}\subset E$ and $c\in \mathbb {R}$ such that $I_\lambda (u_n)\rightarrow c$ and $I_\lambda '(u_n)\rightarrow 0$ as $n\rightarrow \infty$. Similar to the proof of Lemma 4.2, we have, for $\gamma \in (4,r)$,

$$\begin{aligned}&I_\lambda (u_n)-\frac{1}{\gamma }\langle I_\lambda '(u_n),u_n\rangle \\&\quad =\left( \frac{1}{2}-\frac{1}{\gamma }\right) \Vert u_n\Vert _E^2 +\left( \frac{1}{4}-\frac{1}{\gamma }\right) \int _{\mathbb {R}^3}\phi _{u_n}u_n^2\\&\qquad +\int _{\mathbb {R}^3}\left( \frac{1}{\gamma }f(u_n)u_n-F(u_n)\right) +\lambda \left( \frac{1}{\gamma }-\frac{1}{r}\right) \Vert u_n\Vert _r^r. \end{aligned}$$

By $(f_1)$–$(f_2)$,

$$\begin{aligned}&\Vert u_n\Vert _E^2+\int _{\mathbb {R}^3}\phi _{u_n}u_n^2+\lambda \Vert u_n\Vert _r^r \le C_1\left( |I_\lambda (u_n)|+\Vert u_n\Vert _E\Vert I_\lambda '(u_n)\Vert +\Vert u_n\Vert _p^p\right) . \end{aligned}$$

Hence, for large $n$,

$$\begin{aligned} \Vert u_n\Vert _E^2+\int _{\mathbb {R}^3}\phi _{u_n}u_n^2+\lambda \Vert u_n\Vert _r^r\le C_2(1+\Vert u_n\Vert _p^p). \end{aligned}$$

As in the proof of Lemma 4.2, one sees that $\{u_n\}$ is bounded in $E$. Then, by Remark 2.1, one can show that $\{u_n\}$ has a convergent subsequence, verifying the (PS)-condition.

Step 2. Choose $v_1,v_2\in C_0^\infty (B_1(0)){\setminus }\{0\}$ such that $\text{ supp }(v_1)\cap \text{ supp }(v_2)=\emptyset$ and $v_1\le 0,v_2\ge 0$, where $B_r(0):=\{x\in \mathbb {R}^3:\ |x|<r\}$. For $(t,s)\in \Delta$, let

$$\begin{aligned} \varphi _0(t,s)(\cdot ):=R^2\left( tv_1(R\cdot )+sv_2(R\cdot )\right) , \end{aligned}$$

(4.6)

where $R$ is a positive constant to be determined later. Obviously, for $t,s\in [0,1]$, $\varphi _0(0,s)(\cdot )=R^2sv_2(R\cdot )\in P_\varepsilon ^+$ and $\varphi _0(t,0)(\cdot )=R^2tv_1(R\cdot )\in P_\varepsilon ^-$. Similar to Lemma 3.8, for small $\varepsilon >0$,

$$\begin{aligned} I_\lambda (u)\ge I_1(u)\ge \frac{\varepsilon ^2}{2}\ \text{ for }\ u\in \Sigma :=\partial P_\varepsilon ^+\cap \partial P_\varepsilon ^-,\ \lambda \in (0,1), \end{aligned}$$

which implies that $c_\lambda ^*:=\inf _{u\in \Sigma }I_\lambda (u)\ge \frac{\varepsilon ^2}{2}$ for $\lambda \in (0,1)$. Let $u_t=\varphi _0(t,1-t)$ for $t\in [0,1]$. Then a direct computation shows that

• (i) $\int _{\mathbb {R}^3}|\nabla u_t|^2=R^3\int _{\mathbb {R}^3}\left( t^2|\nabla v_1|^2+(1-t)^2|\nabla v_2|^2\right)$,

• (ii) $\int _{\mathbb {R}^3}|u_t|^2=R\int _{\mathbb {R}^3}\left( t^2v_1^2+(1-t)^2v_2^2\right)$,

• (iii) $\int _{\mathbb {R}^3}|u_t|^\mu =R^{2\mu -3}\int _{\mathbb {R}^3}\left( t^\mu |v_1|^\mu +(1-t)^\mu |v_2|^\mu \right)$,

• (iv) $\int _{\mathbb {R}^3}\phi _{u_t}|u_t|^2=R^3\int _{\mathbb {R}^3}\phi _{\tilde{u}_t}|\tilde{u}_t|^2$, where $\tilde{u}_t=tv_1+(1-t)v_2$.

Since $F(t)\ge C_3|t|^\mu -C_4$ for any $t\in \mathbb {R}$, by (i)–(iv) we have, for $\lambda \in (0,1)$ and $t\in [0,1]$,

$$\begin{aligned} I_\lambda (u_t)\le & {} \frac{1}{2}\Vert u_t\Vert _E^2 +\frac{1}{4}\int _{\mathbb {R}^3}\phi _{u_t}|u_t|^2 -\int _{B_{R^{-1}}(0)}F(u_t)\\\le & {} \frac{R^3}{2}\int _{\mathbb {R}^3}\left( t^2|\nabla v_1|^2+(1-t)^2|\nabla v_2|^2\right) +\frac{R^3}{4}\int _{\mathbb {R}^3}\phi _{\tilde{u}_t}|\tilde{u}_t|^2\\&+\frac{R}{2}\max _{|x|\le 1}V(x)\int _{\mathbb {R}^3}\left( t^2v_1^2+(1-t)^2v_2^2\right) \\&-C_3R^{2\mu -3}\int _{\mathbb {R}^3}\left( t^\mu |v_1|^\mu +(1-t)^\mu |v_2|^\mu \right) +C_5R^{-3}. \end{aligned}$$

Since $\mu >3$, one sees that $I_\lambda (u_t)\rightarrow -\infty$ as $R\rightarrow \infty$ uniformly for $\lambda \in (0,1),\ t\in [0,1]$. Hence, choosing $R$ independent of $\lambda$ and large enough, we have

$$\begin{aligned} \sup \limits _{u\in \varphi _0(\partial _0\Delta )}I_\lambda (u)<c_\lambda ^*:=\inf _{u\in \Sigma }I_\lambda (u),\quad \lambda \in (0,1). \end{aligned}$$

Since $\Vert u_t\Vert _2\rightarrow \infty$ as $R\rightarrow \infty$ uniformly for $t\in [0,1]$, it follows from Lemma 3.7 that $\varphi _0(\partial _0\Delta )\cap M=\emptyset$ for $R$ large enough. Thus, $\varphi _0$ with a large $R$ independent of $\lambda$ satisfies the assumptions of Theorem A. Therefore, the number

$$\begin{aligned} c_\lambda =\inf \limits _{\varphi \in \Gamma }\sup \limits _{u\in \varphi (\triangle ){\setminus } W}I_\lambda (u), \end{aligned}$$

is a critical value of $I_\lambda$ satisfying $c_\lambda \ge c_\lambda ^*$, and there exists $u_\lambda \in E{\setminus } (P_\varepsilon ^+\cup P_\varepsilon ^-)$ such that $I_\lambda (u_\lambda )=c_\lambda$ and $I_\lambda '(u_\lambda )=0$.

Step 3. Passing to the limit as $\lambda \rightarrow 0$. By the definition of $c_\lambda$, we see that for $\lambda \in (0,1)$,

$$\begin{aligned} c_\lambda \le c(R):=\sup \limits _{u\in \varphi _0(\triangle )}I(u)<\infty . \end{aligned}$$

We claim that $\{u_\lambda \}_{\lambda \in (0,1)}$ is bounded in $E$. We first have

$$\begin{aligned} c_\lambda =\frac{1}{2}\int _{\mathbb {R}^3}\left( |\nabla u_\lambda |^2+V(x)u_\lambda ^2\right) +\frac{1}{4}\int _{\mathbb {R}^3}\phi _{u_\lambda }u_\lambda ^2 -\int _{\mathbb {R}^3}\left( F(u_\lambda )+\frac{\lambda }{r}|u_\lambda |^r\right) \end{aligned}$$

(4.7)

and

$$\begin{aligned} \int _{\mathbb {R}^3}\left( |\nabla u_\lambda |^2+V(x)u_\lambda ^2+\phi _{u_\lambda }u_\lambda ^2 -u_\lambda f(u_\lambda )-\lambda |u_\lambda |^r\right) =0. \end{aligned}$$

(4.8)

Moreover, we have the Pohoz$\check{\mathrm{a}}$ev identity

$$\begin{aligned}&\frac{1}{2}\int _{\mathbb {R}^3}|\nabla u_\lambda |^2+\frac{3}{2}\int _{\mathbb {R}^3}V(x)u_\lambda ^2+\frac{1}{2}\int _{\mathbb {R}^3}u_\lambda ^2\nabla V(x)\cdot x\nonumber \\&\quad +\frac{5}{4}\int _{\mathbb {R}^3}\phi _{u_\lambda }u_\lambda ^2 -\int _{\mathbb {R}^3}\left( 3F(u_\lambda )+\frac{3\lambda }{r}|u_\lambda |^r\right) =0. \end{aligned}$$

(4.9)

Multiplying (4.7) by $3-\frac{\mu }{2}$, (4.8) by $-1$ and (4.9) by $\frac{\mu }{2}-1$ and adding them up, we obtain

$$\begin{aligned} \left( 3-\frac{\mu }{2}\right) c_\lambda= & {} \left( \frac{\mu }{4}-\frac{1}{2}\right) \int _{\mathbb {R}^3}(2V(x)+\nabla V(x)\cdot x)u_\lambda ^2\nonumber \\&+\left( \frac{\mu }{2}-\frac{3}{2}\right) \int _{\mathbb {R}^3}\phi _{u_\lambda }u_\lambda ^2 +\left( 1-\frac{\mu }{r}\right) \lambda \int _{\mathbb {R}^3}|u_\lambda |^r\nonumber \\&+\int _{\mathbb {R}^3}\left( u_\lambda f(u_\lambda )-\mu F(u_\lambda )\right) . \end{aligned}$$

(4.10)

Using $(V_1)$, $(f_3)$ and the fact that $3<\mu \le p<r$, one sees that $\{\int _{\mathbb {R}^3}\phi _{u_\lambda }u_\lambda ^2\}_{\lambda \in (0,1)}$ is bounded. From this fact, it can be deduced from $(f_3)$, (4.7), and (4.8) that $\{u_\lambda \}_{\lambda \in (0,1)}$ is bounded in $E$.

Assume that up to a subsequence, $u_\lambda \rightarrow u$ weakly in $E$ as $\lambda \rightarrow 0^+$. By Remark 2.1, $u_\lambda \rightarrow u$ strongly in $L^q(\mathbb {R}^3)$ for $q\in [2,6)$. Then, by Lemma 2.1, $\phi _{u_\lambda }\rightarrow \phi _u$ strongly in $\mathcal {D}^{1,2}(\mathbb {R}^3)$. By a standard argument, we see that $I'(u)=0$ and $u_\lambda \rightarrow u$ strongly in $E$ as $\lambda \rightarrow 0^+$. Moreover, the fact that $u_\lambda \in E{\setminus } (P_\varepsilon ^+\cup P_\varepsilon ^-)$ and $c_\lambda \ge \frac{\varepsilon ^2}{2}$ for $\lambda \in (0,1)$ implies $u\in E{\setminus } (P_\varepsilon ^+\cup P_\varepsilon ^-)$ and $I(u)\ge \frac{\varepsilon ^2}{2}$. Therefore, $u$ is a sign-changing solution of (2.1). $\square$

In the following, we prove the existence of infinitely many sign-changing solutions to (2.1). We assume that $f$ is odd. Thanks to Lemmas 4.1–4.4, we have seen that $P_\varepsilon ^+$ is a $G-$admissible invariant set for the functional $I_\lambda \ (0<\lambda <1)$ at any level $c$.

Proof Theorem 1.2 (Multiplicity part)

Step 1. We construct $\varphi _n$ satisfying the assumptions in Theorem B. For any $n\in \mathbb {N}$, we choose $\{v_i\}_1^n\subset C_0^\infty (\mathbb {R}^3){\setminus }\{0\}$ such that

$\text{ supp }(v_i)\cap \text{ supp }(v_j)=\emptyset$ for $i\not =j$. Define $\varphi _n\in C(B_n,E)$ as

$$\begin{aligned} \varphi _n(t)(\cdot )=R_n^2\sum _{i=1}^nt_iv_i(R_n\cdot ), \quad t=(t_1,t_2,\cdots ,t_n)\in B_n, \end{aligned}$$

(4.11)

where $R_n>0$ is a large number independent of $\lambda$ such that $\varphi _n(\partial B_n)\cap (P_\varepsilon ^+\cap P_\varepsilon ^-)=\emptyset$ and

$$\begin{aligned} \sup \limits _{u\in \varphi _n(\partial B_n)}I_{\lambda }(u)<0<\inf \limits _{u\in \Sigma }I_\lambda (u). \end{aligned}$$

Obviously, $\varphi _n(0)=0\in P_\varepsilon ^+\cap P_\varepsilon ^-$ and $\varphi _n(-t)=-\varphi _n(t)$ for $t\in B_n$.

Step 2. For any $j\in \mathbb {N}$ and $\lambda \in (0,1)$, we define

$$\begin{aligned} c_j(\lambda )=\inf \limits _{B\in \Gamma _j}\sup \limits _{u\in B{\setminus } W}I_\lambda (u), \end{aligned}$$

where $W:=P_\varepsilon ^+\cup P_\varepsilon ^-$ and $\Gamma _j$ is as in Theorem B. According to Theorem B, for any $0<\lambda <1$ and $j\ge 2$,

$$\begin{aligned} 0<\inf _{u\in \Sigma }I_\lambda (u):=c^*(\lambda )\le c_j(\lambda )\rightarrow \infty \ \text{ as }\ j\rightarrow \infty \end{aligned}$$

and there exists $\{u_{\lambda ,j}\}_{j\ge 2}\subset E{\setminus } W$ such that $I_\lambda (u_{\lambda ,j})=c_j(\lambda )$ and $I_\lambda '(u_{\lambda ,j})=0$.

Step 3. In a similar way to the above, for any fixed $j\ge 2$, $\{u_{\lambda ,j}\}_{\lambda \in (0,1)}$ is bounded in $E$. Without loss of generality, we assume that $u_{\lambda ,j}\rightarrow u_j$ weakly in $E$ as $\lambda \rightarrow 0^+$. Observe that $c_j(\lambda )$ is decreasing in $\lambda$. Let $c_j=\lim _{\lambda \rightarrow 0^+}c_j(\lambda )$. Clearly $c_j(\lambda )\le c_j<\infty$ for $\lambda \in (0,1)$. Then, we may assume that $u_{\lambda ,j}\rightarrow u_j$ strongly in $E$ as $\lambda \rightarrow 0^+$ for some $u_j\in E{\setminus } W$ such that $I'(u_j)=0$, $I(u_j)=c_j$. Since $c_j\ge c_j(\lambda )$ and $\lim _{j\rightarrow \infty }c_j(\lambda )=\infty$, $\lim _{j\rightarrow \infty }c_j=\infty$. Therefore, equation (2.1) and thus system (1.1) has infinitely many sign-changing solutions. The proof is completed. $\square$