1 Introduction and main results

In this manuscript we are interested in prove the existence of solution for the problem

figure a

where \(1<p<N\), \(N\ge 2,\) \(f: {\mathbb {R}} \rightarrow {\mathbb {R}}\) is a continuous function and \(a,b \in {\mathcal {W}}\), which denotes the set of the functions \(k:{\mathbb {R}}^+\rightarrow {\mathbb {R}}^+\) that satisfy the following hypotheses

\((k_1)\):

\( k \in C^1\) and there are constants \(a_1,a_2>0\) that satisfy

$$\begin{aligned} a_1t^p+t^N\le k(t^p)t^p\le a_2t^p+t^N, \text { for } t>0; \end{aligned}$$
\((k_2)\):

the function \(t \mapsto {\mathcal {K}}(t^p),\) is convex, for \(t >0\), where \({\mathcal {K}}\) is the primitive of k,  that is, \({\mathcal {K}}(t):=\int _{0}^{t}k(\tau )\,d\tau ;\)

\((k_3)\):

the function \(t \mapsto \frac{k(t^p)}{t^{N-p}}\) is nonincreasing, for \(t >0\);

\((k_4)\):

the function \(t \mapsto k(t^p)t^{p-2}\) is increasing, for \(t >0\).

From the growth condition \((k_1)\) it follows the inequality

$$\begin{aligned} a_1t^p+\frac{p}{N}t^N\le {\mathcal {K}}(t^p)\le a_2t^p+ \frac{p}{N}t^N, t>0 \end{aligned}$$
(1.1)

for \(k \in {\mathcal {W}}\). Since we intend to use variational methods, the assumptions above are also important to prove that there is an associated \(C^1\)-class functional.

It will be considered that f satisfies

\((f_1)\):

\(\lim \limits _{t\rightarrow 0}\frac{f(t)}{|t|^{N-1}}=0\)

and the exponential critical growth

\((f_2)\):

there is \(\alpha _0>0\) satisfying

$$\begin{aligned} \lim \limits _{t\rightarrow +\infty }\frac{f(t)}{e^{\alpha |t|^{\frac{N}{N-1}}}}= \left\{ \begin{array}{rll} 0 &{}\text { if }&{} \alpha >\alpha _0.\\ +\infty &{}\text { if }&{} \alpha <\alpha _0; \end{array}\right. \end{aligned}$$

Before presenting the other conditions on f, we will exhibit the hypotheses on the function c, that were motivated by [2], and given by

\((c_1)\):

\(c: {\mathbb {R}}^N\rightarrow {\mathbb {R}}\) is a bounded continuous function which change its sign;

\((c_2)\):

\(\text {dist}(\Omega ^+,\Omega ^-)>0\), where \({\Omega ^+}:= \{x \in {\mathbb {R}}^N;c(x)>0\}\) and \({\Omega ^-}:= \{x \in {\mathbb {R}}^N;c(x)<0\}\);

\((c_3)\):

there is \(R>0\) with \(c(x)<0\) for all \(|x|\ge R\).

Assumption \((c_2)\) ensures the existence of \(\zeta \in C^{\infty }({\mathbb {R}}^N, [0,1])\) satisfying

$$\begin{aligned} \zeta \equiv 1, \text { in } \Omega ^+,\qquad \zeta \equiv 0, \text { in } \Omega ^-, \qquad {\mathcal {M}}:=\sup _{{\mathbb {R}}^N}|\nabla \zeta |<\infty . \end{aligned}$$

Now, we are able to state the remaining conditions on f.

\((f_3)\):

there is \(\nu >N\) and

$$\begin{aligned} 0<\theta < \min \left\{ \dfrac{\nu }{N+(N-1){\mathcal {M}}},\dfrac{\nu a_1}{pa_2+{\mathcal {M}}a_2\min \{1,p-1\}}\right\} =:\theta _0, \end{aligned}$$

such that

$$\begin{aligned} 0< \frac{\nu }{\theta }F(t) \le f(t)t, \ \text {for} \ \,|t|>0, \end{aligned}$$

where \( F(t):= \int _{0}^{t}f(\tau )\,d\tau .\)

\((f_4)\):

there are constants \(K_0, \,R_0>0\) satisfying

$$\begin{aligned} 0<F(t)\le K_0|f(t)|, \ \text {for} \ |t|\ge R_0; \end{aligned}$$
\((f_5)\):

if \(x_0 \in \Omega ^+\) and \(r>0\) satisfy \(B_r(x_0)\subset \subset \Omega ^+\), by denoting \(c_0=\inf _{x \in B_r(x_0)}c(x)>0\), it will be considered that

$$\begin{aligned} \lim \limits _{|t| \rightarrow +\infty }tf(t)e^{-\alpha _0 |t|^{\frac{N}{N-1}}}\ge \beta _0>\frac{N^N}{c_0\alpha _0^{N-1}r^N}. \end{aligned}$$

An example of function satisfying \((f_1)-(f_5)\) can be found in [2] and is given by

$$\begin{aligned} f(s)=\left( q|s|^{q-1}s+ \frac{\alpha _0 N}{N-1}|s|^{q+\frac{N}{N-1}-2}s\right) e^{\alpha _0 |s|^{\frac{N}{N-1}}} \end{aligned}$$

for \(s \in {\mathbb {R}}\) and \(q>\frac{\nu }{\theta }\). In this case \(F(s)=|s|^{q}e^{\alpha _0|s^{\frac{N}{N-1}}}\).

In the recent decades, problems related to P has been attracting the attention of researchers due to its applicability in mathematical models that arise in several branches of science such as biophysics, plasma physics and chemical reaction design driven by the parabolic reaction-diffusion system

$$\begin{aligned} u_{t} = \text {div}[( |\nabla u |^{p-2} + |\nabla u |^{N-2})\nabla u] + c(x,u). \end{aligned}$$

In the mentioned applications, the solution u describes mathematically the concentration, the divergent term provides informations of the diffusion D(u); whereas the term c is the reaction and is related to loss processes and the source. In several application in Chemistry and Biology, the reaction function c(xu) exhibits a polynomial growth in the term u and has variable coefficients. Without intention to present a complete list of references, we quote the classical ones [7, 11] for more details regarding the mentioned applications.

From the mathematical point of view, the main motivations for (P) are [2, 9], where it was considered a version of (P) for the \(N-\)Laplacian operator in an exterior domain and a problem for the general nonhomogeneous operator considered in (P) with a nonlinearity exhibiting a critical exponential growth, respectively.

Note that the hypotheses considered in the functions a and b allow to one consider a wide class of problem. For example, by considering \(a(t) = 1 + t^{\frac{N-p}{p}},~ b(t) = 1 + t^{\frac{N-p}{p}} \), we obtain \(a,b \in {\mathcal {W}}\) with \(a_1=a_2=1,\) that provide \( p \& N-\)Laplacian equation

$$\begin{aligned} -\Delta _p u - \Delta _N u + |u|^{p-2}u + |u|^{N-2}u = c(x)f(u) \ \text {in} \ {\mathbb {R}}^N, \end{aligned}$$

which arises in the study of reaction–diffusion systems as described before. If

$$\begin{aligned} a(t) = 1+ t^{\frac{N-p}{p}} + \frac{1}{(1+t)^{\frac{p-2}{p}} },~ b(t)=1+ t^{\frac{N-p}{p}} + \frac{1}{(1+t)^{\frac{p-2}{p}} }, \end{aligned}$$

we have \(a,b \in {\mathcal {W}}\) with \(a_1 =1 \) and \(a_2 =2\). In such case one can consider the mean curvature type problem

$$\begin{aligned} \begin{aligned}&-\Delta _p u - \Delta _N u - \text {div} \left( \frac{|\nabla u |^{p-2}\nabla u}{(1 + |\nabla u|^{p})^{\frac{p-2}{p}}}\right) \\&+|u|^{p-2}u + |u|^{N-2}u + \left( \frac{|u|^{p-2}u}{(1+|u|^{p})^{\frac{p-2}{p}}}\right) = c(x)f(u) \ \text {in} \ {\mathbb {R}}^{N}. \end{aligned} \end{aligned}$$

In what follows, we present the result obtained in this paper.

Theorem 1.1

Consider that \(a,b \in {\mathcal {W}}\) and \((c_1)-(c_3)\), \((f_1)-(f_5)\) hold. Then there exists a nontrivial solution for P.

The proof of the result consists in an application of the Mountain Pass theorem. The main difficulty is to prove the boundness of the Palais–Smale sequences which occurs due to the sign changing potential c. Another mathematical difficulty is the lack of compactness which is handled by considering the assumption \((f_4)\) (see [8]) and the technical difficulties related to the minimax level may be solved by combining the hypothesis \((f_5)\), a Trudinger–Moser inequality and appropriate estimates involving the Moser’s functions.

The rest of the manuscript is organized as follows: in Section 2 it is presented some preliminary facts to consider the problem through a variational approach; in Section 3 it is studied the Palais–Smale sequences associated to the problem, the Mountain Pass level and, finally, it is proved Theorem 1.1.

2 Preliminaries

We start this section with a substantial lemma proved in [1]. Before state the result let us introduce the following notation: if \(N\ge 2\), we denote by

$$\begin{aligned} S_{N-2}(\alpha ,t)=\sum _{j=0}^{N-2}\frac{\alpha ^j}{j!}|t|^{\frac{Nj}{(N-1)}}, \end{aligned}$$

for \(\alpha >0\) and \(t\in {\mathbb {R}}\).

Lemma 2.1

Consider \((u_n)\) a sequence in \(W^{1,N}(\mathbb R^N)\) such that

$$\begin{aligned} \limsup \limits _{n \rightarrow +\infty }\Vert u_n\Vert _{1,N}^N<\left( \frac{\alpha _N}{\alpha _0}\right) ^{N-1}, \end{aligned}$$
(2.1)

where

$$\begin{aligned} \Vert u\Vert _{1,N}=\left( \displaystyle \int _{{\mathbb {R}}^N}(|\nabla u|^N+|u|^N)\,dx\right) ^{1/N},u \in W^{1,N}(\mathbb R^N), \end{aligned}$$

\(\alpha _N:=N\omega _{N-1}^{\frac{1}{N-1}},\) and \(\omega _{N-1}\) is the measure of the unit sphere in \({\mathbb {R}}^N\).

Then, there are constants \(\alpha >\alpha _0\), \(s>1\), \(C>0,\) which does not depend on n,  such that

$$\begin{aligned} \int \limits _{{\mathbb {R}}^N}\left[ \exp (\alpha |u_n|^{\frac{N}{N-1}})-S_{N-2}(\alpha ,u_n)\right] ^sdx\le C, \text { for all }n \ge n_0. \end{aligned}$$
(2.2)

Consider \(\alpha >\alpha _0\) and \(q\ge 1\). From the hypotheses \((f_1)-(f_2)\) it follows that, for an arbitrary \(\varepsilon >0\), there are constants \(C_\varepsilon , c_\varepsilon >0\) satisfying

$$\begin{aligned} \begin{aligned} |f(t)|&\le \varepsilon |t|^{N-1} +c_\varepsilon |t|^{q-1}(e^{\alpha |t|^{\frac{N}{N-1}}}-S_{N-2}(\alpha ,t)), \\ |F(t)|&\le \frac{\varepsilon }{N} |t|^N +C_\varepsilon |t|^{q}(e^{\alpha |t|^{\frac{N}{N-1}}}-S_{N-2}(\alpha ,t)), \end{aligned} \end{aligned}$$
(2.3)

for all \(t \in {\mathbb {R}}\).

Regarding to obtain solutions for P it will be considered the space

$$\begin{aligned} X=W^{1,p}({\mathbb {R}}^N)\cap W^{1,N}({\mathbb {R}}^N), \end{aligned}$$

which is a Banach space with the norm \(\Vert u\Vert =\Vert u\Vert _{1,p}+\Vert u\Vert _{1,N}\), where

$$\begin{aligned} \Vert u\Vert _{1,m}=\left( \displaystyle \int _{{\mathbb {R}}^N}(|\nabla u|^m+|u|^m)\,dx\right) ^{1/m},~m\ge 1. \end{aligned}$$

From the hypotheses \(a,b \in {\mathcal {W}}\), \((c_1)\), the inequalities in (2.3) and the Trudinger–Moser inequality (see [5, 6]), it follows that the functional \(I:X\rightarrow {\mathbb {R}}\) defined by

$$\begin{aligned} I(u)=\frac{1}{p}\displaystyle \int _{{\mathbb {R}}^N}{\mathcal {A}}(|\nabla u|^p)\,dx+\frac{1}{p}\displaystyle \int _{{\mathbb {R}}^N}{\mathcal {B}}(|u|^p)\,dx-\displaystyle \int _{{\mathbb {R}}^N}c(x)F(u)\,dx \end{aligned}$$

belongs to \(C^1(X,R)\) and

$$\begin{aligned} \begin{aligned} I'(u)v&=\displaystyle \int _{{\mathbb {R}}^N}a(|\nabla u|^p)|\nabla u|^{p-2}\nabla u\nabla v\,dx+\displaystyle \int _{{\mathbb {R}}^N}b(|u|^p)|u|^{p-2}uv\,dx\\&\quad -\displaystyle \int _{{\mathbb {R}}^N}c(x)f(u)v\,dx, \text { for all }u,v \in X. \end{aligned} \end{aligned}$$

Therefore, the critical points of I are weak solutions for P.

In the next result it is obtained the Mountain Pass geometry for the functional I at the origin.

Lemma 2.2

Consider that a and b verify \((k_1)\) and suppose that the conditions \((c_1)\), \((c_3)\), \((f_1)-(f_3)\) hold. Then, there are \(\xi ,\,\rho >0\) such that

$$\begin{aligned} I(u) \ge \xi , \ \text {for all} \ u \in X\cap \partial B_{\rho }(0). \end{aligned}$$

Proof

From (1.1) we have

$$\begin{aligned} I(u)\ge \frac{1}{N}\Vert u\Vert ^N_{1,N}-\displaystyle \int _{{\mathbb {R}}^N}c(x)F(u)\,dx, \text { for all } u \in X. \end{aligned}$$
(2.4)

By using Lemma 2.1 and the Hölder’s inequality it follows that there are \(\alpha >\alpha _0\), \(s>1\) and \(C>0\) satisfying

$$\begin{aligned} \displaystyle \int _{{\mathbb {R}}^N}|u|^q(\exp (\alpha |u|^{\frac{N}{N-1}}-S_{N-2}(\alpha ,u)))\,dx\le C\Vert u\Vert _{qs'}^q, \text { for all } \Vert u\Vert _{1,N}^N<\left( \frac{\alpha _N}{\alpha _0}\right) ^{N-1}, \end{aligned}$$
(2.5)

for a fixed \(q>N\), \(C>0\) not depending on u and \(s'\) is the conjugated exponent of s.

Consider an arbitrary \(\varepsilon >0\). By using (2.3), the above inequality and the continuous embeddings \(W^{1,N}({\mathbb {R}}^N)\hookrightarrow L^N({\mathbb {R}}^N)\), \(W^{1,N}({\mathbb {R}}^N)\hookrightarrow L^{qs'}({\mathbb {R}}^N)\) we obtain that

$$\begin{aligned} \displaystyle \int _{{\mathbb {R}}^N}c(x)F(u)\,dx{} & {} \le \displaystyle \int _{\Omega ^+}c(x)F(u)\,dx\le \frac{\varepsilon }{N}C_0\Vert u\Vert ^N_N +\\{} & {} \quad +C_0C_{\varepsilon }\displaystyle \int _{{\mathbb {R}}^N}|u|^q(e^{\alpha |u|^{N/(N-1)}}-S_{N-2}(\alpha ,u))\,dx.\\{} & {} \le \frac{\varepsilon }{N}C_1\Vert u\Vert _{1,N}^N+C_2\Vert u\Vert _{1,N}^q, \end{aligned}$$

for \(\Vert u\Vert _{1,N}^N<(\alpha _N/\alpha _0)^{N-1}\) and with \(C_0:=\sup _{x \in \Omega ^+}c(x)>0\). From (2.4) and the previous inequality we get

$$\begin{aligned} I(u)\ge \frac{1}{N}\Vert u\Vert ^N_{1,N}-\frac{\varepsilon }{N}C_1\Vert u\Vert _{1,N}^N-C_2\Vert u\Vert _{1,N}^q, \ \text {for all} \ \Vert u\Vert _{1,N}^N<\left( \frac{\alpha _N}{\alpha _0}\right) ^{N-1}. \end{aligned}$$

Thus, by considering \(\varepsilon >0\) such that \((1-\varepsilon C_1)=C_3>0\) we can use the above inequality to get

$$\begin{aligned} I(u) \ge \frac{1}{N}C_3\Vert u\Vert ^N_{1,N}-C_2\Vert u\Vert _{1,N}^q, \end{aligned}$$

which proves the result for

$$\begin{aligned} \rho := \min \left\{ \frac{1}{2}\left( \frac{\alpha _N}{\alpha _0}\right) ^{\frac{N-1}{N}}, \left( \frac{C_3}{2NC_2}\right) ^{1/(q-N)}\right\} \text { and } \xi := \frac{\rho ^NC_3}{2N}. \end{aligned}$$

\(\square \)

By considering a nonnegative function \(\varphi \in C^{\infty }_0(\Omega ^+) {\setminus } \{0\},\) it follows from \((f_3)\) that \(I(t\varphi ) \rightarrow -\infty \) as \(t\rightarrow +\infty \). Thus, there is \(e \in X\) satisfying \(\Vert e\Vert >\rho \) and \(I(e)<0\). This and the previous result imply that there is a Palais-Smale sequence at the mountain pass level (see [4] and [14, Theorem 1.15]), that is, a sequence \((u_n) \subset X\) satisfying

$$\begin{aligned} \lim _{n\rightarrow +\infty } I'(u_n)=0,\qquad \lim _{n\rightarrow +\infty } I(u_n)=c:= \inf _{\gamma \in \Gamma }\max _{s \in [0,1]} I(\gamma (s)), \end{aligned}$$

with \(\Gamma := \{\gamma \in C([0,1],X); \gamma (0)=0,\, \gamma (1)=e\}\).

Some prior definitions are needed for the next step. Let \(x_0 \in \Omega ^+\) and \(r>0\) given \((f_5)\). As in [5], we consider the Moser’s functions [13] defined by

$$\begin{aligned} {\widetilde{M}}_n(x):=\frac{1}{\omega _{N-1}^{\frac{1}{N}}} \cdot \left\{ \begin{array}{lll}(\log n)^{\frac{N-1}{N}}, &{}\text { if }&{} |x-x_0|\le r/n,\quad \\ \frac{\log \left( r/|x-x_0|\right) }{(\log n)^{1/N}}, &{}\text { if }&{} r/n\le |x-x_0|<r,\\ 0,&{}\text { if }&{} |x-x_0|\ge r. \end{array}\right. \end{aligned}$$

Note that there is no loss of generality by considering \(x_0=0\). We have \({\widetilde{M}}_n \in W^{1,N}({\mathbb {R}}^N)\cap C_0({\mathbb {R}}^N)\) (which implies that \({\widetilde{M}}_n \in X\)) and \(\text{ supp }({\widetilde{M}}_n)\subset {\overline{B}}_r(0)\). Moreover, we have the result below.

Lemma 2.3

The assertions below hold.

  1. (i)

    \(\Vert \nabla {\widetilde{M}}_n\Vert _{N} = 1\), for all \(n \in {\mathbb {N}}\);

  2. (ii)

    \(\displaystyle \int _{{\mathbb {R}}^N}|{\widetilde{M}}_n|^N\,dx=O(1/\log (n)) \rightarrow 0\) as \(n \rightarrow +\infty \);

  3. (iii)

    Defining \(M_n:= {\widetilde{M}}_n / \Vert {\widetilde{M}}_n\Vert _{1,N}\), there is a sequence \((d_n) \subset {\mathbb {R}}\) satisfying

    $$\begin{aligned} M_n^{\frac{N}{N-1}}=\frac{N}{\alpha _N}\log {n}+d_n, \qquad \lim \limits _{n \rightarrow +\infty }d_n/ \log n=0, \text { for } |x|\le r/n; \end{aligned}$$
    (2.6)
  4. (iv)

    \(\Vert \nabla {\widetilde{M}}_n\Vert _{p}\rightarrow 0\) and \(\Vert {\widetilde{M}}_n\Vert _{p}\rightarrow 0\) as \(n \rightarrow +\infty ,\) for all \(1<p<N\).

Proof

The proof of properties \((i)-(iii)\) can be found in [5]. Regarding (iv),  note that

$$\begin{aligned} \begin{aligned} \int _{{\mathbb {R}}^N}|{\widetilde{M}}_n|^p&=\frac{1}{\omega _{N-1}^{p/N}}\left[ \int _{\{|x|\le r/n\}}(\log (n))^{\frac{N-1}{N}p}+\int _{\{r/n<|x|<r\}}\frac{\left( \log \left( \frac{r}{|x|}\right) \right) ^p}{(\log (n))^{p/N}}\right] \\&=\frac{1}{\omega _{N-1}^{p/N}}\left[ \omega _{N-1}\frac{r^N}{Nn^N}\left( \log (n)\right) ^{\frac{N-1}{N}p}+\int _{\{r/n<|x|<r\}}\frac{\left( \log \left( \frac{r}{|x|}\right) \right) ^p}{(\log (n))^{p/N}}\right] . \end{aligned} \end{aligned}$$
(2.7)

The fact that \(p<N\) implies

$$\begin{aligned} 0\le \frac{\left( \log (n)\right) ^{\frac{N-1}{N}p}}{n^N}\le \frac{n^{\frac{N-1}{N}p}}{n^N}{n^N}\le n^{\frac{N-p}{N}p-N}\rightarrow 0, \end{aligned}$$
(2.8)

as \(n \rightarrow +\infty \).

Regarding to estimate the right-hand side of (2.7), note that the inequalities \(p<N\) and \(\log (s)\le s\), for all \(s>0,\) provide that

$$\begin{aligned} 0\le \int _{\{r/n<|x|<r\}}\frac{\left( \log \left( \frac{r}{|x|}\right) \right) ^p}{(\log (n))^{p/N}}\le \frac{r^p}{(\log (n))^{p/N}}\omega _{N-1}\frac{r^{N-p}}{N-p}\left( 1-\frac{1}{n^{N-p}}\right) \rightarrow 0, \end{aligned}$$
(2.9)

as \(n \rightarrow +\infty \). From (2.7), (2.8) and the previous inequality we obtain that \(\lim _{n \rightarrow +\infty }\Vert {\widetilde{M}}_n\Vert _p\rightarrow 0\).

In order to prove the gradient estimate, it follows from the definition of \({\widetilde{M}}_n\) and the inequality \(p<N\) that

$$\begin{aligned} \int _{{\mathbb {R}}^N}|\nabla {\widetilde{M}}_n|^p&=\frac{1}{(\omega _{N-1}\log (n))^{\frac{p}{N}}}\int _{\{r/n<|x|<r\}}\frac{1}{|x|^p}\\&=\frac{\omega _{N-1}^{\frac{N-p}{N}}}{(\log (n))^{\frac{p}{N}}}\frac{r^{N-p}}{N-p}\left( 1-\frac{1}{n^{N-p}}\right) . \end{aligned}$$

Since

$$\begin{aligned} \frac{1}{(\log (n))^{p/N}}\left( 1-\frac{1}{n^{N-p}}\right) \rightarrow 0, \end{aligned}$$

as \(n \rightarrow +\infty \), the result follows. \(\square \)

The previous properties will play an important role in the following result:

Lemma 2.4

Consider that a and b satisfy \((k_1)\) and suppose that \((c_1)-(c_3)\), \((f_3)\), \((f_5)\) hold. Then there is \(n\in {\mathbb {N}}\) satisfying

$$\begin{aligned} \max _{t\ge 0} I(tM_n)< \frac{1}{N}\left( \frac{\alpha _N}{\alpha _0}\right) ^{N-1}. \end{aligned}$$

Proof

For each \(n\in {\mathbb {N}}\) define the function

$$\begin{aligned} g_n(t):= \frac{a_2}{p}t^p\Vert M_n\Vert _{1,p}^p+\frac{1}{N}t^N-\displaystyle \int _{{\mathbb {R}}^N}c(x)F(tM_n)\,dx,~~t \ge 0. \end{aligned}$$

Thus, it follows from (1.1) and \(\Vert M_n\Vert _{1,N}=1\) that

$$\begin{aligned} I(tM_n)\le g_n(t), \text { for all }t\ge 0. \end{aligned}$$

Note that, it is enough to obtain the existence of \(n \in {\mathbb {N}}\) such that

$$\begin{aligned} \max _{t\ge 0}g_n(t)\le \frac{1}{N}\left( \frac{\alpha _N}{\alpha _0}\right) ^{N-1}. \end{aligned}$$
(2.10)

By using the fact that \(\nu /\theta >N\) and the hypothesis \((f_2)\) we have \(g_n(t)\rightarrow -\infty \), as \(t \rightarrow +\infty \). Thus, \(g_n\) attains its global maximum at \(t_n>0\) which satisfies \(0=g_n'(t_n),\) which is equivalent to

$$\begin{aligned} t_n^N=\displaystyle \int _{B_r(0)}c(x)f(t_nM_n)t_nM_n\,dx-a_2t_n^p\Vert M_n\Vert _{1,p}^p \end{aligned}$$
(2.11)

If \(g_n(t_n)\ge 1/N\left( \alpha _N/\alpha _0\right) ^{N-1},\) we can use the expression of \(g_n\), the fact that \(F\ge 0\) and \(\text{ supp }(M_n) \subset \Omega ^+\) to obtain

$$\begin{aligned} t_n^N\ge \left( \frac{\alpha _N}{\alpha _0}\right) ^{N-1}-\frac{Na_2t_n^p}{p}\Vert M_n\Vert _{1,p}^p. \end{aligned}$$
(2.12)

Since \(\Vert M_n\Vert _{1,p}\rightarrow 0\), as \(n \rightarrow +\infty \) we can use the previous inequality to obtain a constant \({\widetilde{C}}>0\) satisfying

$$\begin{aligned} t_n\ge \tilde{C}, \ \text {for all} \ n \in {\mathbb {N}}. \end{aligned}$$
(2.13)

Consider \(\beta _0>0\) given in \((f_5)\). If \(0<\varepsilon <\beta _0\), there is \(R_\varepsilon >0\) with

$$\begin{aligned} tf(t)\ge ( \beta _0-\varepsilon )e^{\alpha _0|t|^{N/(N-1)}}, \ \text {for all} \ |t|\ge R_\varepsilon . \end{aligned}$$
(2.14)

By using the definition of \(M_n\) and (2.13) we get

$$\begin{aligned} t_nM_n(x)=t_n\frac{\tilde{M}_n}{\Vert \tilde{M}_n\Vert _{1,N}} \ge \frac{\tilde{C}}{\Vert \tilde{M}_n\Vert _{1,N}}\left( \frac{(\log {n})^{N-1}}{\omega _{N-1}}\right) ^{\frac{1}{N}}\ge R_\varepsilon , \end{aligned}$$

for all \(|x|<r/n\) and n large enough. Thus, we have from (2.11), the choice of \(r>0\) in \((f_5)\), (2.14), the previous inequality and the definition of \(M_n\) that

$$\begin{aligned} t_n^N\ge & {} \displaystyle \int _{B_{r/n}(0)}c(x)f(t_nM_n)t_nM_n\,dx-a_2t_n^p\Vert M_n\Vert _{1,p}^p\nonumber \\\ge & {} c_0(\beta _0-\varepsilon )\displaystyle \int _{B_{r/n}(0)}\exp (\alpha _0(t_nM_n)^{N/(N-1)})\,dx-a_2t_n^p\Vert M_n\Vert _{1,p}^p, \end{aligned}$$
(2.15)

with \(c_0:=\min _{B_r(0)}c(x)\). Replacing the definition of \(M_n\) in \(B_{r/n}(0)\) we get

$$\begin{aligned} t_n^N\ge & {} c_0(\beta _0-\varepsilon )\displaystyle \int _{B_{r/n}(0)}\exp \left( \alpha _0t_n^{\frac{N}{N-1}}\frac{\log {n}}{\omega _{N-1}^{1/(N-1)}\Vert {\widetilde{M}}_n\Vert _{1,N}^{N/(N-1)}}\right) \,dx \\{} & {} \quad -a_2t_n^p\Vert M_n\Vert _{1,p}^p \\= & {} c_0(\beta _0-\varepsilon )\frac{\omega _{N-1} r^N}{Nn^N}\exp \left( \frac{\alpha _0t_n^{\frac{N}{N-1}}\log {n}}{\omega _{N-1}^{1/(N-1)}\Vert {\widetilde{M}}_n\Vert _{1,N}^{N/(N-1)}}\right) -a_2t_n^p\Vert M_n\Vert _{1,p}^p. \end{aligned}$$

Since \(t^N=\exp (N\log t)\) and \(1/n^N=\exp (-N\log {n})\) we obtain that

$$\begin{aligned} 1&\ge c_0(\beta _0-\varepsilon )\frac{\omega _{N-1} r^N}{N}\exp \left( \frac{\alpha _0t_n^{\frac{N}{N-1}}\log {n}}{\omega _{N-1}^{1/(N-1)}\Vert {\widetilde{M}}_n\Vert _{1,N}^{N/(N-1)}}-N\log n-N\log t_n\right) \\&\quad -a_2t_n^{p-N}\Vert M_n\Vert _{1,p}^p . \end{aligned}$$

Using that \(1<p<N\), \(\Vert {\widetilde{M}}_n\Vert _{1,N}\rightarrow 1\) and \(\Vert M_n\Vert _{1,p}\rightarrow 0\), as \(n \rightarrow +\infty ,\) the previous inequality implies that \((t_n)\) is a bounded sequence. By using again that \(\Vert M_n\Vert _{1,p}\rightarrow 0\) and (2.12) we obtain, up to a subsequence, that \(t_n^N \rightarrow \gamma \ge (\alpha _N/\alpha _0)^{N-1}\).

Since \(1/n^N=\exp (-N\log {n})\) and \(\exp (t)\ge t, t \in {\mathbb {R}},\) it follows from (2.15) that

$$\begin{aligned} t_n^N\ge c_0(\beta _0-\varepsilon )\omega _{N-1} r^N\left[ \frac{\alpha _0t_n^{\frac{N}{N-1}}}{N\omega _{N-1}^{1/(N-1)}\Vert {\widetilde{M}}_n\Vert _{1,N}^{N/(N-1)}}-1\right] \log {n} -a_2t_n^p\Vert M_n\Vert _{1,p}^p. \end{aligned}$$

Hence, \(\gamma =(\alpha _N/{\alpha _0})^{N-1}\), otherwise we contradict the previous inequality. By using (2.6), (2.12) and (2.15) we have

$$\begin{aligned}&t_n^N\ge -a_2t_n^{p-N}\Vert M_n\Vert _{1,p}^p \\&\quad + \frac{\omega _{N-1}}{N}\frac{r^N}{n^N}\exp {\left( \alpha _0 \left[ \left( \frac{\alpha _N}{\alpha _0}\right) -\left( \frac{Na_2t_n^p}{p}\right) ^{1/(N-1)}\Vert M_n\Vert _{1,p}^{\frac{p}{N-1}}\right] \left[ \frac{N}{\alpha _N}\log n+d_n\right] \right) }\\&\quad \times c_0(\beta _0-\varepsilon )\\&=-a_2t_n^{p-N}\Vert M_n\Vert _{1,p}^p \\&\quad + \frac{\omega _{N-1}}{N}\frac{r^N}{n^N}\exp {\left( \left[ N+\alpha _N\frac{d_n}{\log n}-c_1t^{\frac{p}{N-1}}\Vert M_n\Vert _{1,p}^{\frac{p}{N-1}}-c_2t_n^{\frac{p}{N-1}}\Vert M_n\Vert _{1,p}^{\frac{p}{N-1}}\frac{d_n}{\log n}\right] \log n\right) }\\&\quad \times c_0(\beta _0-\varepsilon )\\&=c_0(\beta _0-\varepsilon )\frac{\omega _{N-1}}{N}\frac{r^N}{n^N}n^{\left( N+\alpha _N\frac{d_n}{\log n}-c_1t^{\frac{p}{N-1}}\Vert M_n\Vert _{1,p}^{\frac{p}{N-1}}-c_2t_n^{\frac{p}{N-1}}\Vert M_n\Vert _{1,p}^{\frac{p}{N-1}}\frac{d_n}{\log n}\right) }\\&\quad -a_2t_n^{p-N}\Vert M _n\Vert _{1,p}^p , \end{aligned}$$

with \(c_1=\frac{N\alpha _0}{\alpha _N}\left( \frac{Na_2}{p}\right) ^{1/(N-1)}\) and \(c_2=\left( \frac{Na_2}{p}\right) ^{1/(N-1)}\).

Considering the limit as \(n \rightarrow +\infty \), using that \(\Vert M_n\Vert _{1,p}\rightarrow 0\), \(\gamma =(\alpha _N/\alpha _0)^{N-1}\) and (2.6) we get \((\alpha _N /\alpha _0)^{N-1} \ge c_0(\beta _0-\varepsilon )\frac{\omega _{N-1}}{N}r^N\). Passing to the limit as \(\varepsilon \rightarrow 0^{+}\) we have

$$\begin{aligned} \beta _0\le \frac{N}{c_0\omega _{N-1}r^N}(\alpha _N /\alpha _0)^{N-1}. \end{aligned}$$

Using the definition of \(\alpha _N=N\omega _{N-1}^{1/(N-1)}\), we obtain a contradiction with \((f_5)\). Thus, there is \(n \in {\mathbb {N}}\) for which (2.10) is verified. \(\square \)

We have \(M_n \in X\) and \(\text{ supp }(M_n) \subset \Omega ^+\), thus \(e:= t_0M_n\) satisfies the mountain pass geometry for \(t_0>0\) large enough. The path \(\gamma (t):=tt_0M_n\) belongs to \(\Gamma \) and it follows, as a consequence of the previous lemma and Lemma 2.2, that the mountain pass level satisfies

$$\begin{aligned} c_M \in \left( 0, \frac{1}{N} \left( \frac{\alpha _N}{\alpha _0}\right) ^{N-1}\right) . \end{aligned}$$
(2.16)

3 Proof of Theorem 1.1

Regarding to prove the main result, it will be needed to study some properties of the Palais–Smale sequences. In order to prove the result, let us rewrite the functional I as

$$\begin{aligned} I(u)=J(u)-\displaystyle \int _{{\mathbb {R}}^N}c(x)F(u_n)\,dx, \end{aligned}$$

where \(J: X\rightarrow {\mathbb {R}}\) defined by

$$\begin{aligned} J(u)=\frac{1}{p}\displaystyle \int _{{\mathbb {R}}^N}{\mathcal {A}}(|\nabla u_n|^p)\,dx+\frac{1}{p}\displaystyle \int _{{\mathbb {R}}^N}{\mathcal {B}}(| u_n|^p)\,dx. \end{aligned}$$

Lemma 3.1

If \((u_n) \subset X\) is a \((PS)_c\)-sequence for I, then, up to a subsequence

  1. (i)

    \((u_n)\) is bounded

  2. (ii)

    \( u_n \rightharpoonup u_0 \,\text{ weakly } \text{ in } X\)

  3. (iii)

    \(\frac{\partial u_n}{\partial x_i}(x) \rightarrow \frac{\partial u_0}{\partial x_i}(x)\) a.e in \({\mathbb {R}}^N\)

  4. (iv)

    \(J'(u_n)\psi \rightarrow J'(u_0)\psi ,\) for all \(\psi \in X\)

Proof

Since \((u_n) \subset X\) is a \((PS)_c\)-sequence we obtain

$$\begin{aligned} I(u_n)-\theta /\nu I'(u_n)( \zeta u_n)=c+o_n(1)+o_n(1)\Vert u_n\Vert . \end{aligned}$$

The definition \(\zeta \), \((k_1)\) and (1.1) imply

$$\begin{aligned}&I(u_n)-\theta /\nu I'(u_n)( \zeta u_n)\ge \frac{1}{p}\displaystyle \int _{{\mathbb {R}}^N}{\mathcal {A}}(|\nabla u_n|^p)\,dx+\frac{1}{p}\displaystyle \int _{{\mathbb {R}}^N}{\mathcal {B}}(| u_n|^p)\,dx\\&\qquad -\displaystyle \int _{{\mathbb {R}}^N}c(x)F(u_n)\,dx\\&\qquad -\frac{\theta }{\nu }\displaystyle \int _{{\mathbb {R}}^N}\left[ a(|\nabla u_n|^p)|\nabla u_n|^{p-2}\nabla u_n\nabla (\zeta u_n)+b(|u_n|^p)| u_n|^{p}\zeta -c(x)f(u_n)\zeta u_n\right] \,dx\\&\quad \ge \left( \frac{1}{p}a_1-\frac{\theta }{\nu }a_2\right) \Vert u_n\Vert _{1,p}^p+\left( \frac{1}{N}-\frac{\theta }{\nu }\right) \Vert u_n\Vert _{1,n}^N\\&\qquad -\frac{\theta {\mathcal {M}}}{\nu }\displaystyle \int _{{\mathbb {R}}^N}a(|\nabla u_n|^p)|\nabla u_n|^{p-1}|u_n|\,dx+\displaystyle \int _{\Omega ^+}c(x)\left[ \frac{\theta }{\nu }f(u_n)u_n-F(u_n)\right] \,dx. \end{aligned}$$

Thus, by using \((k_1)\) and \((f_3)\) we have

$$\begin{aligned} c+o_n(1)+o_n(1)\Vert u_n\Vert&\ge \left( \frac{1}{p}a_1-\frac{\theta }{\nu }a_2\right) \Vert u_n\Vert _{1,p}^p+\left( \frac{1}{N}-\frac{\theta }{\nu }\right) \Vert u_n\Vert _{1,n}^N \nonumber \\&\quad -\frac{\theta {\mathcal {M}}}{\nu }\displaystyle \int _{{\mathbb {R}}^N}\left( a_2|\nabla u_n|^{p-1}|u_n|+|\nabla u_n|^{N-1}|u_n|\right) \,dx. \end{aligned}$$
(3.1)

From Young’s inequality we have

$$\begin{aligned} \displaystyle \int _{{\mathbb {R}}^N}\left( a_2|\nabla u_n|^{p-1}|u_n|+|\nabla u_n|^{N-1}|u_n|\right) \,dx\le & {} \frac{a_2\max \{1,p-1\}}{p}\Vert u_n\Vert _{1,p}^p \\{} & {} \quad +\frac{N-1}{N}\Vert u_n\Vert _{1,n}^N. \end{aligned}$$

Using (3.1) and the previous inequality we get

$$\begin{aligned} c+o_n(1)+o_n(1)\Vert u_n\Vert\ge & {} \frac{1}{p}\left( a_1-\frac{\theta }{\nu }\left( a_2p+ {\mathcal {M}}a_2\max \{1,p-1\}\right) \right) \Vert u_n\Vert _{1,p}^p\\{} & {} \quad +\frac{1}{N}\left( 1-\frac{\theta }{\nu }\left( N+{\mathcal {M}}(N-1)\right) \right) \Vert u_n\Vert _{1,N}^N. \end{aligned}$$

By using \((f_3)\) it follows that the terms into parenthesis in the right-hand side of the previous expression are positive, which implies (i). Hence, there exists \(u_0 \in X\) such that,

$$\begin{aligned} u_n \rightharpoonup u_0 \,\text{ weakly } \text{ in } X, \quad u_n \rightarrow u_0 \text{ in } L^s_{loc}({\mathbb {R}}^N), \quad u_n(x) \rightarrow u_0(x) \text{ a.e. } \text{ in } {\mathbb {R}}^N, \end{aligned}$$

for some subsequence, still denoted by \((u_n)\) and for any \(s \ge 1\). Then, we (ii) is also verified.For the proofs of the properties (iii) and (iv) see [3, Lema 3.2]. \(\square \)

The result below, whose proof can be found in [2], is needed to prove that \(u_0\) is a critical point of I.

Lemma 3.2

Consider that \((c_1)-(c_3)\) and \((f_1)-(f_4)\) hold. If \(c^{\pm }(x):= \max \{\pm c(x),0\}\), then \(c^{\pm }(x)f(u_n) \rightarrow c^{\pm }(x)f(u_0)\) and \(c^{\pm }(x)F(u_n) \rightarrow c^{\pm }(x)F(u_0)\) in \( L^1_{loc}({\mathbb {R}}^N)\).

Now we are in position to prove Theorem 1.1.

Proof of Theorem 1.1

It will be proved that \(u_0\) is a nontrivial solution for P. From Lemma 3.2 we have

$$\begin{aligned} \lim \limits _{n \rightarrow +\infty }\displaystyle \int _{{\mathbb {R}}^N}c(x)f(u_n)\varphi \,dx =\displaystyle \int _{{\mathbb {R}}^N}c(x)f(u_0)\varphi \,dx \end{aligned}$$
(3.2)

for all \(\varphi \in C^{\infty }_0({\mathbb {R}}^N).\) We shall prove that the previous limit also holds considering test functions in the space X. In the spirit of [10], we notice that given any \(\psi \in X\) there exist sequences of mollifiers and cut-off functions \((\rho _k)_k\) and \((\zeta _k)_k\), respectively, such that \(\psi _k:=\zeta _k(\rho _k *\psi ) \in C^{\infty }_0({\mathbb {R}}^N)\) satisfies the following properties:

  1. (i)

    \(\psi _k(x)\rightarrow \psi (x)\), \(|\nabla \psi _k(x)|\rightarrow |\nabla \psi (x)|\), a.e. \( x \in {\mathbb {R}}^N\);

  2. (ii)

    \(|\psi _k(x)|, |\nabla \psi _k(x)|\le h_N(x)\) and \(|\psi _k(x)|, |\nabla \psi _k(x)|\le h_p(x)\), a.e. \(x \in {\mathbb {R}}^N\), for some functions \(h_N \in L^N({\mathbb {R}}^N)\) and \(h_p \in L^p({\mathbb {R}}^N)\),

for all \(k \in {\mathbb {N}}\). Since (3.2) holds for \(\psi _k\), for all \(k \in {\mathbb {N}}\), passing to the limit as \(k \rightarrow +\infty \), using properties \((i)-(ii)\) above and the Lebesgue’s dominated convergence theorem we obtain that (3.2) holds for all \(\psi \in X\). This together with item (iv) of Lemma 3.1 imply that \(u_0\) is a critical point of I.

In what follows it will be proved that \(u_0 \ne 0\). Suppose that \(u_0=0\). By using that \(\Omega ^+\) is bounded,we obtain from Lemma 3.2 that \(\int _{\Omega ^+}c(x)F(u_n)=o_n(1).\) Thus from (1.1) we have

$$\begin{aligned} c+o_n(1)= I(u_n)&\ge \frac{a_1}{p}\Vert u_n\Vert _{1,p}^p+\frac{1}{N}\Vert u_n\Vert _{1,N}^N-\displaystyle \int _{\Omega ^+}c(x)F(u_n)\,dx\\ {}&\ge \frac{1}{N}\Vert u_n\Vert _{1,N}^N+o_n(1). \end{aligned}$$

Now, we can proceed as in [2] to get that

$$\begin{aligned} \lim \limits _{n \rightarrow +\infty }\displaystyle \int _{\Omega ^+}c(x)f(u_n)u_n\,dx=0. \end{aligned}$$

Since \(I'(u_n)u_n=o_n(1)\), it follows from \((k_1)\) and the previous limit that

$$\begin{aligned} o_n(1)\ge a_1\Vert u_n\Vert _{1,p}^{p}+\Vert u_n\Vert _{1,N}^N, \end{aligned}$$

which implies that \(\Vert u_n\Vert =\Vert u_n\Vert _{1,p}+\Vert u_n\Vert _{1,N} \rightarrow 0\). Thus, \(u_n \rightarrow 0\) strongly in X which provides that \(c=0\). This contradicts (2.16) and the result is proved. \(\square \)