Liouville type theorems for nonlinear elliptic equations and systems involving fractional Laplacian in the half space

Abstract

We study the nonexistence of solutions for fractional elliptic problems via a monotonicity result which obtained by the method of moving planes with an improved Aleksandrov–Bakelman–Pucci type estimate for the fractional Laplacian in unbounded domain.

1 Introduction

This paper is devoted to the study of nonexistence results for positive solutions of a class of fractional elliptic equations and systems in the half space ${\mathbb {R}}^N_+$, i.e.,

$$\begin{aligned} \left\{ \begin{array}{l@{\quad }l} (-\Delta )^\alpha u=u^p &{} \hbox {in }\ {\mathbb {R}}^N_+,\\ u=0 &{} \hbox {in } \ {\mathbb {R}}^N{\setminus }{\mathbb {R}}^N_+ \end{array} \right. \end{aligned}$$

(1.1)

and

$$\begin{aligned} \left\{ \begin{array}{l@{\quad }l} (-\Delta )^\alpha u=v^q &{} \hbox {in }\ {\mathbb {R}}^N_+,\\ (-\Delta )^\alpha v=u^p &{} \hbox {in }\ {\mathbb {R}}^N_+,\\ u=v=0 &{} \hbox {in }\ {\mathbb {R}}^N{\setminus }{\mathbb {R}}^N_+ \end{array} \right. \end{aligned}$$

(1.2)

where ${\mathbb {R}}^N_+=\{x=(\tilde{x},x_N)\in {\mathbb {R}}^{N-1}\times {\mathbb {R}}\,|\, x_N>0\}$, $\alpha \in (0,1)$ and the fractional Laplacian operator $(-\Delta )^\alpha$ is defined as

$$\begin{aligned} (-\Delta )^\alpha u(x)=C_{N,\alpha } P.V.\int _{{\mathbb {R}}^N}\frac{u(x)-u(y)}{|x-y|^{N+2\alpha }}\,dy\quad \hbox {for all } \ x\in {\mathbb {R}}^N. \end{aligned}$$

(1.3)

Here $P.V.$ denotes the principal value of the integral, that for notational simplicity we omit in what follows and we without lose of generality take $C_{N,\alpha }=1$.

If $\alpha =1$, Dancer [11] studied the nonexistence of positive solutions for the following nonlinear elliptic equation

$$\begin{aligned} \left\{ \begin{array}{l@{\quad }l} -\Delta u=u^p &{}\hbox {in }\ {\mathbb {R}}^N_+,\\ u=0 &{}\hbox {in }\ \partial {\mathbb {R}}^N_+ \end{array} \right. \end{aligned}$$

(1.4)

and proved that problem (1.4) have no bounded positive solutions if $1< p <\frac{N+1}{N-3}$. We note that $\frac{N+2}{N-2}<\frac{N+1}{N-3}$ if $N>3$, so this improves Theorem 1.3 in Gidas and Spruck [19] which established that the problem (1.4) have no nonnegative classical solution if $1< p <\frac{N+2}{N-2}$. The main idea of [11], which consist in the following: if there is a solution of (1.4) in $\{x_N>0\}$, and if one is able to show that any such solution is increasing in $x_N$-direction, then, after eventually some supplementary work and bounded assumption, one should be able to pass at the limit as $x_N\rightarrow \infty$ and thus get a solution of the same problem in ${\mathbb {R}}^{N-1}$, which in turn permits to use the nonexistence result for the whole space. Also, there is a simplify version of the argument by Dancer, see [2]. For more general operators, similar problem for equations and systems have been studied by the Quaas and Sirakov [24, 25] and reference therein for other related results.

During the last years there has been a renewed and increasing interest in the study of linear and nonlinear integral operators, especially the fractional Laplacian, so one naturally wonders if the method in [2] and [11] is still applicable for the fractional Laplacian operator.

Our first main result is

Theorem 1.1

If $N>2\alpha +1$ and $1< p< \frac{(N-1)+2\alpha }{(N-1)-2\alpha }$, there are no positive viscosity bounded solutions of Eq. (1.1).

Remark 1.1

This Theorem improves Corollary 1.6 of [15] in the case $\frac{N+2\alpha }{N-2\alpha }<p< \frac{(N-1)+2\alpha }{(N-1)-2\alpha }$, see also Theorem 1.2 in [14].

In order to complete the proof of Theorem 1.1 we need a Liouville type result for the corresponding entire space problem

$$\begin{aligned} (-\Delta )^\alpha u=u^p\quad \hbox {in }\ {\mathbb {R}}^N. \end{aligned}$$

(1.5)

By Theorem 3 in [9], Theorem 4.5 in [10] and some regularity results (see Theorems 2.5 and 2.6 below), we have the following result. See also [22] and [14].

Theorem 1.2

For $1<p<\frac{N+2\alpha }{N-2\alpha }$ and $N>2\alpha$, there are no positive viscosity bounded solutions of Eq. (1.5).

For system (1.2), we can get similar result as Eq. (1.1). The same we first consider the corresponding system in the entire space

$$\begin{aligned} \left\{ \begin{array}{l@{\quad }l} (-\Delta )^\alpha u=v^q &{}\hbox {in }\ {\mathbb {R}}^N,\\ (-\Delta )^\alpha v=u^p &{} \hbox {in }\ {\mathbb {R}}^N. \end{array} \right. \end{aligned}$$

(1.6)

We can deduce the following result from Theorem 1 in [23] and regularity results Theorems 2.5 and 2.6 below.

Theorem 1.3

If $N>2\alpha$ and $(p,q)$ satisfy

$$\begin{aligned} \frac{N}{N-2\alpha }<p,q\le \frac{N+2\alpha }{N-2\alpha }\quad \hbox {but not both equal to } \ \frac{N+2\alpha }{N-2\alpha }, \end{aligned}$$

then there are no positive viscosity bounded solutions of system (1.6).

We note that Theorem 1.3 does not considered the case $1<p,q\le \frac{N}{N-2\alpha }$. In [12], Dahmani–Karami–Kerbal use weak formulation approach and rely on a suitable choice of test functions, they proved that there are no nonnegative bounded weak solutions of system (1.6) if

$$\begin{aligned} \max \left\{ \frac{2\alpha (p+1)}{pq-1},\frac{2\alpha (q+1)}{pq-1}\right\} >N-2\alpha ,\quad \hbox {where }\ p,q>1. \end{aligned}$$

(1.7)

Noticed that if $1<p,q\le \frac{N}{N-2\alpha }$, then condition (1.7) is satisfied. So Theorem 1 in [12] and regularity results Theorems 2.5 and 2.6 below can imply that

Theorem 1.4

Suppose $N>2\alpha$, then there are no positive viscosity bounded solution of (1.6) if $(p,q)$ satisfy (1.7).

Remark 1.2

If we replace the fractional Laplacian operator in system (1.6) by more general integro differential operator, we can get similar result as Theorem 1.4 by considering the fundamental solutions as in [17] and [24], see [26].

Thus use the same method as the proof of Theorem 1.1, we can establish our second result.

Theorem 1.5

Suppose $p,q>1$ and $N>2\alpha +1$, then there are no positive viscosity bounded solutions of system (1.2) if and only if

$$\begin{aligned} \frac{(N-1)}{(N-1)-2\alpha }\le p,q\le \frac{(N-1)+2\alpha }{(N-1)-2\alpha } \hbox {but not both equal to } \frac{N-1+2\alpha }{N-1-2\alpha } \end{aligned}$$

and

$$\begin{aligned} \max \left\{ \frac{2\alpha (p+1)}{pq-1},\frac{2\alpha (q+1)}{pq-1}\right\} >N-1-2\alpha . \end{aligned}$$

This article is organized as follows. In Sect. 2 we present some preliminaries to introduce the notion of viscosity solutions, weak Harnack inequality, maximum principle in narrow domain and the Aleksandrov–Bakelman–Pucci (ABP) estimate in unbounded domain as in [3] for fractional Laplacian. In Sects. 3 and 4 are devoted to prove the nonexistence of positive viscosity bounded solutions of Eq. (1.1) and system (1.2) respectively and the respective monotonicity results.

2 Preliminaries

The purpose of this section is to introduce some preliminaries and prove the ABP estimate in unbounded domain for fractional Laplacian. We start this section by defining the notion of viscosity solution for nonlocal equation. For a given domain $\Omega$ of ${\mathbb {R}}^N$ and functions $h$ and $g$, we consider the equation of the form:

$$\begin{aligned} (-\Delta )^\alpha u=h \ \hbox { in }\ \Omega , \qquad u=g \ \hbox { in }\ {\mathbb {R}}^N{\setminus }\Omega . \end{aligned}$$

(2.1)

Definition 2.1

We say that a function $u:{\mathbb {R}}^N\rightarrow {\mathbb {R}}$ continuous in $\Omega$ is a viscosity super-solution (sub-solution) of (2.1) if

$$\begin{aligned} u\ge g \ (\hbox {resp. } \ u\le g) \ \hbox { in } \ {\mathbb {R}}^N{\setminus }\bar{\Omega } \end{aligned}$$

and for every point $x_0\in \Omega$ and some neighborhood $V$ of $x_0$ with $\bar{V}\subset \Omega$ and for any $\phi \in C^2(\bar{V})$ such that $u(x_0)=\phi (x_0)$ and

$$\begin{aligned} u(x)\ge \phi (x)\ (\hbox {resp. }\ u(x)\le \phi (x)) \ \hbox { for all }\ x\in V, \end{aligned}$$

defining

$$\begin{aligned} \tilde{u}= {\left\{ \begin{array}{ll} \phi &{} \hbox {in } \ V,\\ u &{} \hbox {in } \ {\mathbb {R}}^N{\setminus } V, \end{array}\right. } \end{aligned}$$

(2.2)

we have

$$\begin{aligned} (-\Delta )^\alpha \tilde{u}(x_0)\ge h(x_0)\ (\hbox {resp. }\ (-\Delta )^\alpha \tilde{u}(x_0)\le h(x_0)). \end{aligned}$$

Remark 2.1

• (a) This definition is equivalent in the case of super-solution to take $\phi$ punctually in $C^{1,1}$ in $x_0\in \Omega$ such that $u-\phi$ as a zero at $x_0$ that is a global minimum then $(-\Delta )^\alpha \phi (x_0)\ge h(x_0)$, see Lemma 4.3 of [6]. The analogous results holds for sub-solution.

• (b) Other definition and their equivalence can be found in [1].

In order to prove Theorems 1.1 and 1.5, we will use the classical method of moving planes. A key tool in the use of the method of moving planes is the maximum principle for narrow domain, which is a consequence of the ABP estimate in unbounded domain for fractional Laplacian, for local operators see [3]. The ABP estimate for bounded domain with nonlocal operator see [21]. In this section we will prove the ABP estimate in an unbounded domain for the fractional Laplacian.

We first recall the following definition of [3]. For a given domain $\Omega \in {\mathbb {R}}^N$, the quantity $R(\Omega )$ is defined to be the smallest positive constant $R$ such that

$$\begin{aligned} \hbox {meas}(B_R(x){\setminus } \Omega )\ge \frac{1}{2}\hbox {meas}(B_R(x)) \quad \hbox {for all }\ x\in \Omega . \end{aligned}$$

(2.3)

If no such radius $R$ exists, we define $R(\Omega )=+\infty$. It is easy to say that whenever the domain $\Omega$ contained between two parallel hyperplanes at a distance $d$, we have

$$\begin{aligned} R(\Omega )\le \frac{2^Nd}{\omega _N}, \end{aligned}$$

where $\omega _N$ is the volume of unit ball in ${\mathbb {R}}^N$.

We define the open cube $Q_r$ in ${\mathbb {R}}^N$ is centered at $x_0$ with side-length $r$. Then we have the following version weak $L^\varepsilon$ estimate for fractional Laplacian see for a instance Lemma 9.2 in [6]. See also Theorem 10.3 in [21].

Lemma 2.1

(weak $L^\varepsilon$estimate) There exist universal constants $\zeta _0>0$, $0<\mu <1$ and $M>1$ such that if $u\in C(\bar{Q}_{4\sqrt{N}})$ satisfies:

• (1) $u\ge 0$ in ${\mathbb {R}}^N$,

• (2) $\inf _{Q_{3}}u\le 1$ and

• (3) $\Delta ^\alpha u(x)\le \zeta _0$ in $Q_{4\sqrt{N}}$.

Then

$$\begin{aligned} |\{x\in Q_1:u(x)>M^k\}|\le (1-\mu )^k \end{aligned}$$

for $k=1,2,3,\ldots .$

As a consequence, we have that

$$\begin{aligned} |\{x\in Q_1:u(x)\ge t\}|\le d t^{-\varepsilon }, \quad \forall \ t>0, \end{aligned}$$

where $d$ and $\varepsilon$ are positive universal constants.

Now use Lemma 2.1 and the same argument as Theorem 4.8 in [5] (see also [20]) to get the following result by noticing that $L^N$ norm is replaced by $L^\infty$ norm.

Theorem 2.1

Let $u\in C(\bar{Q}_{1})$ satisfies $\Delta ^\alpha u\le h$ in $Q_{1}$ and $u\ge 0$ in ${\mathbb {R}}^N$, where $h\in C(\bar{Q}_{1})$. Then

$$\begin{aligned} \Vert u\Vert _{L^{p_0}({Q_{1/4}})}\le C\left( \inf _{Q_{1/2}} u+\Vert h\Vert _{L^\infty (Q_{1})}\right) , \end{aligned}$$

where $p_0$ and $C$ are positive universal constants.

Next, by Theorem 2.1 and some covering arguments we can get the following weak Harnack inequality.

Theorem 2.2

(weak Harnack inequality) Let $u\in C(\bar{B}_{2R})$ satisfies $\Delta ^\alpha u\le h$ in $B_{2R}$ and $u\ge 0$ in ${\mathbb {R}}^N$, where $h\in C(\bar{B}_{2R})$. Then

$$\begin{aligned} \left( \frac{1}{|B_R|}\int _{B_R} u^{p_0}\right) ^{1/{p_0}}\le C\left\{ \inf _{B_R} u+R^{2\alpha }\Vert h\Vert _{L^\infty (B_{2R})}\right\} , \end{aligned}$$

where $p_0$ and $C$ are positive universal constants.

Proof

Let $t=4/\sqrt{N}$ and $v(x)=u(tRx)$, then $\Delta ^\alpha v(x)=(tR)^{2\alpha }\Delta ^\alpha u(tRx)\le \tilde{h}$ in $B_{\sqrt{N}/2}$, where $\tilde{h}(x)=(tR)^{2\alpha }h(tRx)$. We note that $Q_1\subset B_{\sqrt{N}/2}$, so by Theorem 2.1 we have

$$\begin{aligned} \Vert v\Vert _{L^{p}({Q_{1/4}})}\le C\left\{ \inf _{Q_{1/2}} v+\Vert \tilde{h}\Vert _{L^\infty (B_{\sqrt{N}/2})}\right\} . \end{aligned}$$

Since $B_{1/8}\subset Q_{1/4}\subset Q_{1/2}$, then

$$\begin{aligned} \Vert v\Vert _{L^{p}({B_{1/8}})}\le C\left\{ \inf _{B_{1/8}} v+\Vert \tilde{h}\Vert _{L^\infty (B_{\sqrt{N}/2})}\right\} . \end{aligned}$$

By the definition of $t$ and changing variables, we have

$$\begin{aligned} \left( \frac{1}{|B_\delta |}\int _{B_\delta } u^{p}\right) ^{1/{p_0}}\le C\left\{ \inf _{B_{\delta }} u+R^{2\alpha }\Vert h\Vert _{L^\infty (B_{2R})}\right\} , \end{aligned}$$

(2.4)

where $\delta =\frac{1}{2\sqrt{N}}R$.

Next, since $u\in C(\bar{B}_{2R})$ and then exist $x_0\in \bar{B}_{R}$ such that

$$\begin{aligned} u(x_0)= \inf _{B_{R}} u. \end{aligned}$$

(2.5)

By the Fubini’s theorem and changing variables,

$$\begin{aligned} \int _{B_{R+\frac{\delta }{4}}}\int _{B_{\frac{\delta }{4}}(y)}u^p\,dx\,dy&= \int _{B_\frac{\delta }{4}}\int _{B_{R+\frac{\delta }{4}}}u^p(y+\xi )\,dy\,d\xi \\&\ge |B_\frac{\delta }{4}|\int _{B_{R}}u^p(y)\,dy. \end{aligned}$$

On the other hand, applying the mean value principle for integral, there exist $y_0\in B_{R+\frac{\delta }{4}}$ such that

$$\begin{aligned} \int _{B_{\frac{\delta }{4}}(y_0)}u^p\,dx\ge \left( \frac{\delta }{4R+\delta }\right) ^N\int _{B_{R}}u^p(x)\,dx. \end{aligned}$$

(2.6)

Next we construct a sequence of balls $\{B_{\frac{\delta }{4}}(x_k)\}_{k=1}^n$ such that $x_k\in \bar{B}_{R+\frac{\delta }{4}}$, $x_{k+1}\in B_{\frac{\delta }{4}}(x_k)$ $(k=0,1,2,\ldots ,n-1)$, $x_n=y_0$, $n\le \frac{24}{R\delta }$. To apply (2.4) in ball $B_{\frac{\delta }{4}}(x_k)$, then

$$\begin{aligned} \inf _{B_{\frac{\delta }{4}}(x_k)} u&\ge \frac{1}{C}\left( \frac{1}{|B_{\frac{\delta }{4}}|}\int _{B_{\frac{\delta }{4}}(x_k)} u^{p}\right) ^{1/{p}}-R^{2\alpha }\Vert h\Vert _{L^\infty (B_{\frac{R}{2}}(x_k))}\\&\ge \frac{1}{C}\left( \frac{1}{|B_{\frac{\delta }{4}}|}\int _{B_{\frac{\delta }{4}}(x_k)\cap B_{\frac{\delta }{4}}(x_{k+1})} u^{p}\right) ^{1/{p}}-R^{2\alpha }\Vert h\Vert _{L^\infty (B_{2R})}\\&\ge \frac{1}{\tilde{C}}\inf _{B_{\frac{\delta }{4}}(x_{k+1})} u-R^{2\alpha }\Vert h\Vert _{L^\infty (B_{2R})}. \end{aligned}$$

Without loss of generality we let $\tilde{C}\ge 2$ and replying the process above, we can obtain

$$\begin{aligned} \inf _{B_{\frac{\delta }{4}}(x_0)} u\ge \frac{1}{\tilde{C}^n}\inf _{B_{\frac{\delta }{4}}(y_0)} u-2R^{2\alpha }\Vert h\Vert _{L^\infty (B_{2R})}. \end{aligned}$$

(2.7)

Next, by the definition of $\delta$, (2.6) and applying (2.4) in $B_{\frac{\delta }{4}}(y_0)$, we have

$$\begin{aligned} \inf _{B_{\frac{\delta }{4}}(y_0)} u&\ge \frac{1}{C}\left( \frac{1}{|B_{\frac{\delta }{4}}|}\int _{B_{\frac{\delta }{4}}(y_0)} u^{p}\right) ^{1/{p}}-R^{2\alpha }\Vert h\Vert _{L^\infty (B_{2R})}\nonumber \\&\ge \frac{1}{C^\prime }\left( \frac{1}{|B_{R}|}\int _{B_{R}} u^{p}\right) ^{1/{p}}-R^{2\alpha }\Vert h\Vert _{L^\infty (B_{2R})}. \end{aligned}$$

(2.8)

Finally, we complete our proof by combining (2.7), (2.8) and (2.5). $\square$

The following is an improved ABP estimate. It applies in any domain satisfying $R(\Omega )<+\infty$. Notice that here we do not need the domain is bounded.

Theorem 2.3

Let $\Omega$ be an open domain with $R(\Omega )<+\infty$. Suppose $u\in C(\bar{\Omega })$ and $h\in C(\bar{\Omega })$ satisfy $\sup _\Omega u<\infty$ and

$$\begin{aligned} \left\{ \begin{array}{l@{\quad }l} \Delta ^\alpha u\ge h &{} \hbox {in }\ \Omega ,\\ u=0 &{} \hbox {in }\ {\mathbb {R}}^N{\setminus }\Omega . \end{array} \right. \end{aligned}$$

(2.9)

Then

$$\begin{aligned} \sup _\Omega u\le CR(\Omega )^{2\alpha }\Vert h\Vert _{L^\infty (\Omega )}, \end{aligned}$$

where $C$ is a positive constant.

Proof

Here we follow the idea of [3] (see also [4]). To prove this, assume first that $\Omega$ is bounded. Then the supremum of $u$ is achieved, so that there exist $\tilde{x}\in \Omega$ such that

$$\begin{aligned} M:= \sup _\Omega u=u(\tilde{x}). \end{aligned}$$

To simplify notation, we write $R :=R(\Omega )$ and $B_R :=B_R(\tilde{x})$. We know that

$$\begin{aligned} \frac{\hbox {meas}(B_R{\setminus } \Omega )}{\hbox {meas}(B_R)}\ge \frac{1}{2}. \end{aligned}$$

(2.10)

Consider the function

$$\begin{aligned} v=M-u. \end{aligned}$$

We claim that $v$ satisfies $\Delta ^\alpha v\le -\tilde{h}$ in ${\mathbb {R}}^N$, with $\tilde{h}$ continuous and $\Vert h^-\Vert _{L^\infty (\Omega )}= \Vert \tilde{h}\Vert _{L^\infty ({\mathbb {R}}^N)}$, where $h^-(x)=\min \{h(x),0\}$. In fact, let $\phi$ be a test function so that $v-\phi$ has a global minimum at $x_0 \in {\mathbb {R}}^N$ such that $v(x_0)=\phi (x_0)$. If $\phi (x_0)=M$ then $\Delta ^\alpha \phi (x_0)\le 0$ and if $\phi (x_0)<M$ then $x_0 \in \Omega$ so by Eq. (2.9) we have $\Delta ^\alpha \phi (x_0)\le -h(x_0)\le (-h^-)(x_0)$. Now since $h$ is continuous in $\bar{\Omega }$ we can extended $h^-$ with a non positive continuous function $\tilde{h}$ such that $\Vert h^-\Vert _{L^\infty (\Omega )}= \Vert \tilde{h}\Vert _{L^\infty ({\mathbb {R}}^N)}$ and therefore $\Delta ^\alpha \phi (x_0) \le -\tilde{h}(x_0)$ so the claim follows.

Using (2.10), $v(x_0)=0$ and Theorem 2.2 applied to $v$ in $B_{2R}$, we have

$$\begin{aligned} (1/2)^{1/p} M&\le \left( \frac{\hbox {meas}(B_R{\setminus } \Omega )}{\hbox {meas}(B_R)}\right) ^{1/p} M\\&\le \left( \frac{1}{\hbox {meas}(B_R)}\int _{B_R{\setminus } \Omega } v^p\right) ^{1/p} \\&\le \left( \frac{1}{\hbox {meas}(B_R)}\int _{B_R} v^p\right) ^{1/p} \\&\le C\left\{ \inf _{B_R} v+R^{2\alpha }\Vert \tilde{h}\Vert _{L^\infty (B_{2R})}\right\} \\&= CR^{2\alpha }\Vert h^-\Vert _{L^\infty (B_{2R}\cap \Omega )}, \end{aligned}$$

where $p>0$. This proves the desired inequalities.

In case the $\Omega$ is unbounded, the proof is same with minor changes. We define $M :=\sup _\Omega u$ and we take, for any $\eta >0$, a point $x_0$ such that $M-\eta \le u(x_0)$. We now have that $v(x_0)\le \eta$. We proceed as before and get desired estimate by letting $\eta \rightarrow 0$. $\square$

It is not difficult to deduce from Theorem 2.3 the following maximum principle in domains (not necessarily bounded) for which $R(\Omega )$ is sufficiently small.

Theorem 2.4

Let $\Omega$ be an open domain. Suppose that $\phi :\Omega \rightarrow {\mathbb {R}}$ is in $L^\infty (\Omega )$ and $u\in C(\bar{\Omega })$ is a solution of

$$\begin{aligned} \left\{ \begin{array}{l@{\quad }l@{\quad }l} (-\Delta )^\alpha u\ge \phi (x)u(x)\quad &{} \hbox {in }\ \Omega ,\\ u\ge 0 \quad &{} \hbox {in } {\mathbb {R}}^N{\setminus }\Omega , \end{array} \right. \end{aligned}$$

(2.11)

with $\phi u \in C(\bar{\Omega })$. Then there exist a number $\bar{R}$ such that $R(\Omega ^-)\le \bar{R}$ implies that each solution satisfies $u\ge 0$ in $\Omega$.

Proof

By (2.11), we observe that

$$\begin{aligned} \left\{ \begin{array}{l@{\quad }l@{\quad }l} \Delta ^\alpha \hat{u}(x)\ge -\phi (x)\hat{u}(x) \quad &{} \hbox {in } \ \Omega ^-,\\ \hat{u}(x) = 0\quad &{} \hbox {in }\ {\mathbb {R}}^N{\setminus }\Omega ^-, \end{array} \right. \end{aligned}$$

(2.12)

where $\hat{u}(x)=-\min \{u(x),0\}$, i.e., $\hat{u}=-u^-$ and $\Omega ^-=\{x\in \Omega \ | \ u(x)<0\}$.

By Theorem 2.3 with $h(x)=-\phi (x)\hat{u}(x)$, we obtain

$$\begin{aligned} \Vert \hat{u}\Vert _{L^\infty (\Omega ^-)}= \sup _{\Omega ^-}\hat{u}\le CR(\Omega ^-)^{2\alpha }\Vert \phi (x)\hat{u}(x)\Vert _{L^\infty (\Omega ^-)}. \end{aligned}$$

Thus

$$\begin{aligned} \Vert \hat{u}\Vert _{L^\infty (\Omega ^-)}\le CR(\Omega ^-)^{2\alpha }\Vert \phi (x)\Vert _{L^\infty (\Omega )}\Vert \hat{u}(x)\Vert _{L^\infty (\Omega ^-)}. \end{aligned}$$

We see that, if choose $\bar{R}$ such that $CR(\Omega ^-)^{2\alpha }\Vert \phi (x)\Vert _{L^\infty (\Omega )}<1$, then we have

$$\begin{aligned} \Vert \hat{u}\Vert _{L^\infty (\Omega ^-)}=0. \end{aligned}$$

This implies $|\Omega ^-|=0$ and since $\Omega ^-$ is open, we have $\Omega ^-=\varnothing$. Then we complete the proof. $\square$

Next we give a regularity theorem.

Theorem 2.5

Let $g$ bounded in ${\mathbb {R}}^N{\setminus }\Omega$ and $f\in C^\beta _{loc}(\Omega )$ and $u$ be a viscosity solution of

$$\begin{aligned} (-\Delta )^\alpha u=f \quad \text{ in } \ \Omega ,\quad u=g \quad \text{ on } \ {\mathbb {R}}^N{\setminus }\Omega , \end{aligned}$$

then there exists $\gamma$ such that $u\in C^{2 \alpha +\gamma }_{loc}(\Omega )$.

Proof

Here we use ideas of Silvestre [27] and the argument of Chen et al. [8]. Suppose without loss of generality that $B_1\subset \Omega$ and $f \in C^\beta (B_1)$. Let $\eta$ be a non-negative, smooth function with support in $B_1$, such that $\eta =1$ in $B_{1/2}$. Now we discuss the following equation

$$\begin{aligned} -\Delta w=-\eta f \quad \hbox {in }\ {\mathbb {R}}^N. \end{aligned}$$

By Hölder regularity theory for the Laplacian we find $w\in C^{2,\beta }$, so that $(-\Delta )^{1-\alpha }w\in C^{2\alpha +\beta }$, see [28] or for example Theorem 3.1 in [16]. Then, since

$$\begin{aligned} (-\Delta )^\alpha (u-(-\Delta )^{1-\alpha }w)=0 \quad \hbox {in } \ B_1 , \end{aligned}$$

we can use Theorem 1.1 and Remark 9.4 of [7] (see also Theorem 4.1 there), to obtain that there exist $\tilde{\beta }$ such that $u-(-\Delta )^{1-\alpha }w \in C^{2\alpha +\tilde{\beta }}$, from where we conclude. $\square$

Remark 2.2

We say that a function $u$ continuous in $\Omega$ and bounded in ${\mathbb {R}}^N$ is a classical solution of (2.1) if $(-\Delta )^\alpha u(x)$ is well defined for all $x\in \Omega$,

$$\begin{aligned} (-\Delta )^\alpha u(x)=h,\quad \hbox {for all } \ x\in \Omega \end{aligned}$$

and $u(x) = g$ a.e. in ${\mathbb {R}}^N{\setminus }\Omega$. Classical super and sub-solutions are defined similarly.

The Maximum Principle is key tool in the analysis, one can see that a nonnegative solution $u$ is either strictly positive or identically zero in ${\mathbb {R}}^N$. A more general case can be found in [27].

Proposition 2.1

Let $\Omega$ be an open and bounded domain of ${\mathbb {R}}^N$, and let $u$ be a classical solution of

$$\begin{aligned} (-\Delta )^\alpha u\ge 0\ \hbox {in}\ \Omega \quad \hbox {and} \quad u\ge 0 \ \hbox {in} \ {\mathbb {R}}^N{\setminus }\Omega . \end{aligned}$$

Then $u\ge 0$ in ${\mathbb {R}}^N$. Moreover, if $u(x)=0$ for some point inside $\Omega$, then $u\equiv 0$ in all ${\mathbb {R}}^N$.

Proof

If the conclusion is false, then there exists $x^\prime \in \Omega$ such that $u(x^\prime )<0$. Since $\Omega$ is bounded and $u\ge 0$ in ${\mathbb {R}}^N{\setminus }\Omega$, then the continuity of $u$ implies that there exists $x_0 \in \Omega$ such that

$$\begin{aligned} u(x_0)=\min _{x\in \Omega }u(x)=\min _{x\in {\mathbb {R}}^N}u(x). \end{aligned}$$

So we can find some points $y\in {\mathbb {R}}^N$ such that $u(y)>u(x_0)$ and finally by the definition of fractional Laplacian operator (see (1.3)) we can obtain $(-\Delta )^\alpha u(x_0)<0$, which contradicts our assumption. Therefore $u\ge 0$ in ${\mathbb {R}}^N$.

On the other hand, if $u(x_1)=0$ for some point $x_1\in \Omega$ and $u\not \equiv 0$ in ${\mathbb {R}}^N$, as above we have $(-\Delta )^\alpha u(x_1)<0$ which contradicts the assumption. $\square$

We also need the following $C^\beta$ estimate, which is a direct conclusion of Theorem 2.6 in [6].

Theorem 2.6

Let $\Omega$ be a regular domain. If $u\in C(\bar{\Omega })$ satisfies the inequalities

$$\begin{aligned} \Delta ^\alpha u\ge -C_0\quad \hbox {and} \quad \Delta ^\alpha u\le C_0 \quad \hbox {in} \ \Omega , \end{aligned}$$

then for any $\Omega ^\prime \Subset \Omega$ there exist constant $\beta >0$ such that $u\in C^\beta (\Omega ^\prime )$ and

$$\begin{aligned} \Vert u\Vert _{C^\beta (\Omega ^\prime )}\le C \left\{ \sup _{\Omega }|u| + \Vert u\Vert _{L^\infty (\Omega )} + C_0\right\} \end{aligned}$$

for some constant $C > 0$ which depends on $N$.

Remark 2.3

Theorems 2.6 and 2.5 imply that if $u$ is a viscosity and bounded solution of $(-\Delta )^\alpha u=u^p$ in $\Omega$ with $p>0$, then $u$ is classical. In fact, if $u$ is bounded, we have $u^p$ is bounded. So Theorem 2.6 implies there exist constants $\beta ,\gamma >0$ such that $u\in C^\beta$ and then $u^p\in C^\gamma$. Finally by Theorem 2.5 $u$ is a classical solution.

We are going to use the following convergence result for fractional Laplacian (see Corollary 4.6 in [6] for integro differential equation).

Theorem 2.7

Let $\{u_k\}$, $k\in {\mathbb {N}}$ be a sequence of functions that are bounded in ${\mathbb {R}}^N$ and continuous in $\Omega$, $f_k$ and $f$ are continuous in $\Omega$ such that

• (1) $\Delta ^\alpha u_k =f_k$ in $\Omega$ in viscosity sense.

• (2) $u_k\rightarrow u$ locally uniformly in $\Omega$.

• (3) $u_k\rightarrow u$ a.e. in ${\mathbb {R}}^N$.

• (4) $f_k\rightarrow f$ locally uniformly in $\Omega$.

Then $\Delta ^\alpha u=f$ in $\Omega$ in viscosity sense.

3 Proof of Theorem 1.1

This section is devoted to the proof of Theorem 1.1. We consider first the monotonicity of solutions. In fact, the monotonicity results used in the proof of Theorem 1.1 can be applied to much more general nonlinearities, we prove the following:

Theorem 3.1

Suppose we have a positive viscosity bounded solution $u$ of

$$\begin{aligned} \left\{ \begin{array}{l@{\quad }l} (-\Delta )^\alpha u=f(u) &{} x\in {\mathbb {R}}^N_+,\\ u=0 &{}x\in {\mathbb {R}}^N{\setminus }{\mathbb {R}}^N_+, \end{array} \right. \end{aligned}$$

(3.1)

where $f(u)$ is a locally Lipschitz continuous function with $f(0)\ge 0$ and nondecreasing in $u$. Then $u$ is strictly increasing in $x_N$-direction.

In order to prove Theorem 3.1, we use the method of moving planes. For which we give some preliminary notations, we define

$$\begin{aligned} \Sigma _{\mu }=\left\{ x=(\tilde{x}, x_N)\in {\mathbb {R}}^N_+\,|\,0<x_N<\mu \right\} ,\\ T_{\mu }=\left\{ x=(\tilde{x}, x_N)\in {\mathbb {R}}^N_+\,|\,x_N=\mu \right\} ,\\ u_{\mu }(x)=u(x_{\mu }), \ w_{\mu }(x)=u_{\mu }(x)-u(x), \end{aligned}$$

where $\mu >0$ and $x_{\mu }=(\tilde{x},2\mu - x_N)$ for all $(\tilde{x}, x_N)\in {\mathbb {R}}^{N-1}\times {\mathbb {R}}$. For any subset $A$ of ${\mathbb {R}}^N$, we write $A_{\mu }=\{x_{\mu }\,|\,x\in A\}$, the reflection of $A$ with respect to $T_{\mu }$.

Proof of Theorem 3.1

We divide the proof in two steps.

Step 1: We prove that if $\mu >0$ is small enough, then $w_{\mu }>0$. For this purpose, we first prove that $w_{\mu }\ge 0$ if $\mu >0$ is small enough. If we define

$$\begin{aligned} \Sigma _{\mu }^-=\{x\in \Sigma _{\mu }\,|\,w_{\mu }(x)<0\}, \end{aligned}$$

then we just need to show that $\Sigma _{\mu }^-$ is empty for $\mu >0$ is small enough. By contradiction, we assume that $\Sigma _{\mu }^-$ is not empty. In order to overcome the difficulty introduced by the non local character of the differential operator in the application of the method of moving planes, we use a truncation technique as in [18]. We define

$$\begin{aligned} w_1(x) = \left\{ \begin{array}{lll} w_{\mu }(x) &{} \quad x\in \Sigma _{\mu }^-,\\ 0 &{} \quad x\in {\mathbb {R}}^N{\setminus }\Sigma _{\mu }^-, \end{array} \right. \end{aligned}$$

(3.2)

$$\begin{aligned} w_2(x) = \left\{ \begin{array}{lll} 0 &{} \quad x\in \Sigma _{\mu }^-,\\ w_{\mu }(x) &{} \quad x\in {\mathbb {R}}^N{\setminus }\Sigma _{\mu }^- \end{array} \right. \end{aligned}$$

(3.3)

and we observe that $w_1(x)= w_{\mu }(x)- w_2(x)$ for all $x\in {\mathbb {R}}^N$. Next we claim that for all $\mu >0$, we have

$$\begin{aligned} (-\Delta )^\alpha w_2(x) \le 0\quad \quad \forall \,x\in \Sigma _{\mu }^-. \end{aligned}$$

(3.4)

By the definition of fractional Laplacian, for $x\in \Sigma _{\mu }^-$, we have

$$\begin{aligned} (-\Delta )^\alpha w_2(x)&= \int _{{\mathbb {R}}^N}\frac{w_2(x)-w_2(y)}{|x-y|^{N+2\alpha }}\,dy =-\int _{{\mathbb {R}}^N{\setminus }\Sigma _{\mu }^-}\frac{w_{\mu }(y)}{|x-y|^{N+2\alpha }}\,dy\\&= -\int _{(\Sigma _\mu {\setminus }\Sigma _{\mu }^-)\cup (\Sigma _\mu {\setminus }\Sigma _{\mu }^-)_\mu } \frac{w_{\mu }(y)}{|x-y|^{N+2\alpha }}\,dy\\&- \int _{({\mathbb {R}}^N{\setminus }{\mathbb {R}}_+^N)\cup ({\mathbb {R}}^N{\setminus }{\mathbb {R}}_+^N)_\mu }\frac{w_{\mu }(y)}{|x-y|^{N+2\alpha }}\,dy -\int _{(\Sigma _{\mu }^-)_\mu }\frac{w_{\mu }(y)}{|x-y|^{N+2\alpha }}\,dy\\&= - A_1-A_2-A_3. \end{aligned}$$

Next we estimate each of these integrals separately. We first observe that $w_{\mu }(y_\mu )=-w_{\mu }(y)$ for any $y\in {\mathbb {R}}^N$ and $w_{\mu }(y)\ge 0$ in $\Sigma _\mu {\setminus }\Sigma _{\mu }^-$. Then

$$\begin{aligned} A_1&= \int _{(\Sigma _\mu {\setminus }\Sigma _{\mu }^-)\cup (\Sigma _\mu {\setminus }\Sigma _{\mu }^-)_\mu } \frac{w_{\mu }(y)}{|x-y|^{N+2\alpha }}\,dy\\&= \int _{\Sigma _\mu {\setminus }\Sigma _{\mu }^-}\frac{w_{\mu }(y)}{|x-y|^{N+2\alpha }}\,dy +\int _{\Sigma _\mu {\setminus }\Sigma _{\mu }^-}\frac{w_{\mu }(y_\mu )}{|x-y_\mu |^{N+2\alpha }}\,dy\\&= \int _{\Sigma _\mu {\setminus }\Sigma _{\mu }^-}w_{\mu }(y)\left( \frac{1}{|x-y|^{N+2\alpha }}-\frac{1}{|x-y_\mu |^{N+2\alpha }}\right) \,dy\ge 0, \end{aligned}$$

since $|x-y_\mu |>|x-y|$ for $x\in \Sigma _{\mu }^-$ and $y\in \Sigma _\mu {\setminus }\Sigma _{\mu }^-$. In order to study the sign of $A_2$ we observe that $u=0$ in ${\mathbb {R}}^N{\setminus }{\mathbb {R}}_+^N$ and $u_\mu =0$ in $({\mathbb {R}}^N{\setminus }{\mathbb {R}}_+^N)_\mu$, so we have

$$\begin{aligned} A_2&= \int _{({\mathbb {R}}^N{\setminus }{\mathbb {R}}_+^N)\cup ({\mathbb {R}}^N{\setminus }{\mathbb {R}}_+^N)_\mu }\frac{w_{\mu }(y)}{|x-y|^{N+2\alpha }}\,dy\\&= \int _{{\mathbb {R}}^N{\setminus }{\mathbb {R}}_+^N}\frac{u_\mu (y)}{|x-y|^{N+2\alpha }}\,dy -\int _{({\mathbb {R}}^N{\setminus }{\mathbb {R}}_+^N)_\mu }\frac{u(y)}{|x-y|^{N+2\alpha }}\,dy\\&= \int _{{\mathbb {R}}^N{\setminus }{\mathbb {R}}_+^N}u_\mu (y)\left( \frac{1}{|x-y|^{N+2\alpha }}-\frac{1}{|x-y_\mu |^{N+2\alpha }}\right) \,dy\ge 0, \end{aligned}$$

since $u_\mu (y)\ge 0$ in ${\mathbb {R}}^N{\setminus }{\mathbb {R}}_+^N$ and $|x-y_\mu |>|x-y|$ for all $x\in \Sigma _{\mu }^-$ and $y\in {\mathbb {R}}^N{\setminus }{\mathbb {R}}_+^N$. Finally, since $w_{\mu }(y)<0$ for $y\in \Sigma _{\mu }^-$, we have

$$\begin{aligned} A_3&= \int _{(\Sigma _{\mu }^-)_\mu }\frac{w_{\mu }(y)}{|x-y|^{N+2\alpha }}\,dy =\int _{\Sigma _{\mu }^-}\frac{w_{\mu }(y_\mu )}{|x-y_\mu |^{N+2\alpha }}\,dy\\&= -\int _{\Sigma _{\mu }^-}\frac{w_{\mu }(y)}{|x-y_\mu |^{N+2\alpha }}\,dy\ge 0. \end{aligned}$$

Hence, we can obtain (3.4). Now we apply (3.4) and the linearity of fractional Laplacian to obtain that, for $x\in \Sigma _{\mu }^-$,

$$\begin{aligned} (-\Delta )^\alpha w_1(x)\ge (-\Delta )^\alpha w_{\mu }(x)=(-\Delta )^\alpha u_{\mu }(x)-(-\Delta )^\alpha u(x). \end{aligned}$$

(3.5)

Combining Eq. (3.1) with (3.5), for $x\in \Sigma _{\mu }^-$ we have

$$\begin{aligned} (-\Delta )^\alpha w_1(x)&\ge (-\Delta )^\alpha u_{\mu }(x)-(-\Delta )^\alpha u(x)\\&= f(u_{\mu }(x))-f(u(x))\\&= \frac{f(u_{\mu }(x))-f(u(x))}{u_{\mu }(x)- u(x)}w_1. \end{aligned}$$

Let us define $\varphi (x)=\frac{f(u_{\mu }(x))-f(u(x))}{u_{\mu }(x)- u(x)}$ for $x\in \Sigma _{\mu }^-$. Since $f$ is locally Lipschitz continuous, we have that $\varphi (x)\in L^\infty (\Sigma _{\mu }^-)$ and $\varphi w_1$ is continuous. Hence we have

$$\begin{aligned} (-\Delta )^\alpha w_1(x)\ge \varphi (x)w_1(x)\quad \quad x\in \Sigma _{\mu }^-, \end{aligned}$$

(3.6)

and since $w_1(x)=0$ in ${\mathbb {R}}^N{\setminus }\Sigma _{\mu }^-$, we may apply Theorem 2.4. Choosing $\mu >0$ small enough and then

$$\begin{aligned} w_{\mu }(x)=w_1(x)\ge 0\quad \quad x\in \Sigma _{\mu }^-. \end{aligned}$$

But this is a contradiction with our assumption, and therefore $\Sigma _{\mu }^-$ is empty. Hence we have

$$\begin{aligned} w_{\mu }(x)\ge 0\quad \quad x\in \Sigma _{\mu }. \end{aligned}$$

In order to complete Step 1, we claim that for $\mu >0$, if $w_\mu \ge 0$ and $w_\mu \not \equiv 0$ in $\Sigma _{\mu }$, then $w_\mu >0$ in $\Sigma _{\mu }$. Assuming the claim is true, we complete the proof, in fact the function $u$ is positive in ${\mathbb {R}}^N_+$ and $u=0$ in ${\mathbb {R}}^N{\setminus }{\mathbb {R}}^N_+$, so that $w_\mu$ is positive in $\{x_N=0\}$ and then by continuity $w_\mu \not \equiv 0$ in $\Sigma _{\mu }$.

Now we prove the claim. Assume there exist $x_0\in \Sigma _{\mu }$ such that $w_\mu (x_0)=0$, that is $u_\mu (x_0)=u(x_0)$. Then we have that

$$\begin{aligned} (-\Delta )^\alpha w_\mu (x_0)=(-\Delta )^\alpha u_\mu (x_0)-(-\Delta )^\alpha u(x_0)=0. \end{aligned}$$

(3.7)

On the other hand, defining $A_\mu =\{x\in {\mathbb {R}}^N|\,x_N<\mu \}$. Since $w_\mu (y_\mu )=-w_\mu (y)$ for any $y\in {\mathbb {R}}^N$ and $w_\mu (x_0)=0$, then

$$\begin{aligned} (-\Delta )^\alpha w_\mu (x_0)&= -\int _{A_\mu }\frac{w_\mu (y)}{|x_0-y|^{N+2\alpha }}\,dy-\int _{{\mathbb {R}}^N{\setminus } A_\mu }\frac{w_\mu (y)}{|x_0-y|^{N+2\alpha }}\,dy\\&= -\int _{A_\mu }\frac{w_\mu (y)}{|x_0-y|^{N+2\alpha }}\,dy-\int _{ A_\mu }\frac{w_\mu (y_\mu )}{|x_0-y_\mu |^{N+2\alpha }}\,dy\\&= -\int _{A_\mu }w_\mu (y)\left( {\frac{1}{|x_0-y|^{N+2\alpha }}-\frac{1}{|x_0-y_\mu |^{N+2\alpha }}}\right) \,dy. \end{aligned}$$

Since $|x_0-y_\mu |>|x_0-y|$ for $y\in A_\mu$, $w_\mu (y)\ge 0$ and $w_\mu (y)\not \equiv 0$ in $A_\mu$, then we have

$$\begin{aligned} (-\Delta )^\alpha w_\mu (x_0)<0, \end{aligned}$$

(3.8)

which contradicts (3.7), completing the proof of the claim.

Step 2: We define

$$\begin{aligned} \mu ^*=\hbox {sup}\{\mu \,|\,w_{\nu }>0\ \hbox { in }\ \Sigma _\nu ,\forall \nu <\mu \}>0. \end{aligned}$$

We see that for each $0<\mu <\mu ^*$ the function $w_{\mu }>0$ in $\Sigma _\mu$, which implies $u$ is strictly increasing in $x_N$ direction for $x\in \Sigma _\mu$ with $0<\mu <\mu ^*$. In fact, for $0<x_N<\tilde{x}_N<\mu ^*$ and let $\mu =\frac{x_N+\tilde{x}_N}{2}$. Then

$$\begin{aligned} w_\mu (x)>0\quad \hbox {in }\ \Sigma _\mu . \end{aligned}$$

Hence

$$\begin{aligned} 0<w_\mu (\tilde{x},x_N)&= u_\mu (\tilde{x},x_N)-u(\tilde{x},x_N)\\&= u(\tilde{x},\tilde{x}_N)-u(\tilde{x},x_N), \end{aligned}$$

that is $u(\tilde{x},\tilde{x}_N)>u(\tilde{x},x_N)$, so $u$ is strictly increasing in $x_N$ direction. Therefore, the theorem is proved if we show that $\mu ^*=+\infty$.

Suppose for contradiction that $\mu ^*$ is finite. By Theorem 2.4 we can fix $\varepsilon _0$ such that the operator $\Delta ^\alpha -\varphi (x)$ (where $\varphi (x)=\frac{f(u_{\mu }(x))-f(u(x))}{u_{\mu }(x)- u(x)}$) satisfies the maximum principle in the domain $\Sigma _{\mu ^*+\varepsilon _0}{\setminus }\Sigma _{\mu ^*-\varepsilon _0}$. For instance, we can take $\varepsilon _0 = (\omega _N/2^{ (N+1)})\bar{R}$, where $\bar{R}$ is number from Theorem 2.4. $\square$

Lemma 3.1

There exists $\delta _0\in (0,\varepsilon _0]$, such that for each $\delta \in (0,\delta _0]$ we have

$$\begin{aligned} w_{\mu ^*+\delta }(x)>0\quad \hbox {in} \ \Sigma _{\mu ^*-\varepsilon _0}{\setminus }\Sigma _{\varepsilon _0}. \end{aligned}$$

Suppose this lemma is proved. Then we repeat the proof of Step 1 and apply Theorem 2.4 to equation in $\Sigma _{\mu ^*+\delta }{\setminus }\Sigma _{\mu ^*-\varepsilon _0}$ and in $\Sigma _{\varepsilon _0}$ (those domains are narrow enough) to conclude that $w_{\mu ^*+\delta }\ge 0$ in $\Sigma _{\mu ^*+\delta }$ for each $\delta \in (0,\delta _0)$. This contradicts the maximal choice of $\mu ^*$.

Proof of Lemma 3.1

Suppose the Lemma is false, that is exist sequences $\delta _m\rightarrow 0$ and $x^{(m)}=(\tilde{x}^{(m)},x_N^{(m)})\in \Sigma _{\mu ^*-\varepsilon _0}{\setminus }\Sigma _{\varepsilon _0}$ such that

$$\begin{aligned} w_{\mu ^*+\delta _m}(x^{(m)})\le 0. \end{aligned}$$

(3.9)

We can suppose that $x_N^{(m)}\rightarrow x_N^{0}\in [\varepsilon _0,\mu ^*-\varepsilon _0]$ as $m\rightarrow \infty$.

We define the functions

$$\begin{aligned} u^{(m)}(\tilde{x},x_N)=u(\tilde{x}+\tilde{x}^{(m)},x_N) \end{aligned}$$

and, respectively

$$\begin{aligned} w_\mu ^{(m)}(\tilde{x},x_N)=u^{(m)}(\tilde{x},2\mu -x_N)-u^{(m)}(\tilde{x},x_N). \end{aligned}$$

Note that $u^{(m)}$ satisfies the same equation as $u$, and

$$\begin{aligned} (-\Delta )^\alpha u^{(m)}=f(u^{(m)})\ge f(0). \end{aligned}$$

So we can infer from Theorem 2.6 that

$$\begin{aligned} \Vert u^{(m)}\Vert _{C^\beta (K)}\le C, \end{aligned}$$

for each compact set $K$ in the closure of ${\mathbb {R}}^N_+$ (the constant $C$ depends on $K$ ). It follows from Theorems 2.6 and 2.7, that $u^{(m)}$ converges uniformly to a solution $\widetilde{u}$ of (3.1) and $\tilde{u}$ satisfies $\Delta ^\alpha \tilde{u}=-f(\tilde{u})\le -f(0)\le 0$.

By the strong maximum principle (see Proposition 2.1) we have that either $\tilde{u}$ is strictly positive in ${\mathbb {R}}^N_+$ or $\tilde{u}$ vanishes identically in ${\mathbb {R}}^N_+$. Suppose first that $\tilde{u}$ is strictly positive in ${\mathbb {R}}^N_+$. By what we have already shown we know that $w_\mu ^{(m)}(y,x_N)=w_\mu (y+y^{(m)},x_N)>0$ in $\Sigma _\mu$ for all $\mu \le \mu ^*$. Hence the limit function $\tilde{w}_\mu =\lim \nolimits _{m\rightarrow \infty }w_\mu ^{(m)}$ is nonnegative in $\Sigma _\mu$ for all $\mu \le \mu ^*$.

So we can repeat the moving plane arguments for $\tilde{u}$, and get $\tilde{\mu }^*\ge \mu ^*$ such that $\tilde{w}_\mu >0$ in $\Sigma _\mu$ for all $\mu \le \tilde{\mu }^*$, where $\tilde{\mu }^*$ to $\tilde{u}$ what $\mu ^*$ to $u$. On the other hand, by continuity and (3.9) we have $\tilde{w}_{\mu ^*}(0,x_N^0)=0$, and $x_N^0\in (0,\mu ^*-\varepsilon _0]$, a contradiction.

Suppose next $\tilde{u}\equiv 0$ in ${\mathbb {R}}^N_+$. We fix the rectangular domains

$$\begin{aligned} Q_1&= \left\{ x\in {\mathbb {R}}^N_+\,|\,-1<x_1<1,\ldots ,-1<x_{N-1}<1,\varepsilon _0<x_N<2\mu ^*+1\right\} ,\\ Q_2&= \left\{ x\in {\mathbb {R}}^N_+\,|\,-2<x_1<2,\ldots ,-2<x_{N-1}<2,\frac{\varepsilon _0}{2}<x_N<2\mu ^*+2\right\} . \end{aligned}$$

Since $u^{(m)}$ converges uniformly to zero in $Q_2$, we can suppose that $u^{(m)}\le 1$ in $Q_2$.

We set

$$\begin{aligned} \alpha _m=u^{(m)}(0,x_N^{(m)})\quad \hbox {and} \quad v^{(m)}=\frac{u^{(m)}}{\alpha _m}. \end{aligned}$$

Now, by (3.1) the function $v^{(m)}$ satisfies

$$\begin{aligned} \Delta ^\alpha v^{(m)}+\frac{f(u^{(m)})}{u^{(m)}}v^{(m)}=0\quad \hbox {in }\ Q_2. \end{aligned}$$

(3.10)

By applying Harnack inequality (see Theorem 1.1 in [29] and Proposition 2.4 in[22]) in these cubes we infer

$$\begin{aligned} \sup _{Q_1} v^{(m)}\le C_1 \inf _{Q_1} v^{(m)}\le C_1. \end{aligned}$$

Next we recall that $w_\mu ^*>0$ in $\Sigma _{\mu ^*}$, which implies

$$\begin{aligned} v^{(m)}(y,x_N)\le v^{(m)}(y,2\mu ^*-x_N)\le C_1,\quad \hbox {for } \ (y,x_N)\in \Sigma _{\mu ^*}. \end{aligned}$$

Hence

$$\begin{aligned} \Vert v^{(m)}\Vert _{L^\infty (Q)}\le C_1, \end{aligned}$$

where

$$\begin{aligned} Q_1=\{x\in {\mathbb {R}}^N_+\,|\,-1<x_1<1,\ldots ,-1<x_{N-1}<1,0<x_N<2\mu ^*+1\}. \end{aligned}$$

Since $f$ is locally Lipschitz, then we have $|(f(u)-f(0))/u|\le C$, where $C$ is a positive constant. Hence, $f(u)\ge -Cu$ since $f(0)\ge 0$. By applying Theorems 2.6 and 2.7 to (3.10) we get that $v^{(m)}\rightrightarrows v$ on compacts and $v$ satisfies

$$\begin{aligned} \Delta ^\alpha v+lv\le 0, \end{aligned}$$

where $l=\liminf _{t\rightarrow 0}\frac{f(t)}{t}$. By the strong maximum principle $v$ vanishes identically in $Q$ or $v>0$ in $Q$. The first possibility is excluded by $v(0,x_N^0)=1$.

Introduce the functions

$$\begin{aligned} z^\beta (y,x_N)=v(y,2\beta -x_N)-v(y,x_N) \end{aligned}$$

define in $\Sigma _\beta \cap \overline{Q}$ for $\beta \le \beta ^*+1/2$. We have, by continuity,

$$\begin{aligned} z^{\beta ^*}\ge 0\quad \hbox {and}\quad z^{\beta ^*}(0,x_N^0)=0. \end{aligned}$$

Since $(-\Delta )^\alpha z^{\beta ^*}\le lz^{\beta ^*}$, the strong maximum principle implies $z^{\beta ^*}=0$ in $\Sigma _\beta ^*\cap \overline{Q}$. This contradicts the fact that $v=0$ on $\{x_N=0\}$ and $v>0$ on $\{x_N=2\beta ^*\}$. $\square$

Theorem 3.2

Under the hypotheses of Theorem 3.1, if

$$\begin{aligned} \Delta ^\alpha u+f(u)= 0 \end{aligned}$$

(3.11)

has a positive bounded solution in ${\mathbb {R}}^N_+$ such that $u=0$ on ${\mathbb {R}}^N{\setminus }{\mathbb {R}}_+^N$, then the same problem has a positive solution in ${\mathbb {R}}^{N-1}$.

Proof

Suppose $u$ is a solution of (3.11), $u\not \equiv 0$, $0\le u\le M$. For each $x\in (y,x_N)$ in the strip $\Sigma _1=\{x\in {\mathbb {R}}^N|0<x_N<1\}$ we set

$$\begin{aligned} u_m(x^\prime ,x_N)=u(x^\prime ,x_N+m). \end{aligned}$$

Now $u_m$ satisfies the same system as $u$. Then, using the $C^\beta$ regularity, Theorem 2.6, we see that $\{u_m\}$ is bounded in $C^\beta$ and hence a subsequence of it converge uniformly on compact subsets of $\Sigma _1$ to a function $\tilde{u}$. By Theorem 2.7 $\tilde{u}$ satisfies

$$\begin{aligned} \Delta ^\alpha \tilde{u}+f(\tilde{u})= 0 \quad \hbox {in }\ \Sigma _1. \end{aligned}$$

(3.12)

The monotonicity result of Theorem 3.1 trivially implies that $\tilde{u}$ is strictly positive and independent of the $x_N$-variable.

Otherwise, by the definition of fractional Laplacian

$$\begin{aligned} (-\Delta )^\alpha \tilde{u}(x)&= \int _{{\mathbb {R}}^{N-1}}\int _{{\mathbb {R}}}\frac{\tilde{u}(\tilde{x})-\tilde{u}(\tilde{y})}{(|\tilde{x}-\tilde{y}|^2+(x_N-y_N)^2)^{\frac{N+2\alpha }{2}}}\,dy_N\,d\tilde{y}\\&= \int _{{\mathbb {R}}^{N-1}}\int _{{\mathbb {R}}}\frac{\tilde{u}(\tilde{x})-\tilde{u}(\tilde{x}-\tilde{y})}{(|\tilde{y}|^2+y_N^2)^{\frac{N+2\alpha }{2}}}\,dy_N\,d\tilde{y}. \end{aligned}$$

Let $y_N=|\tilde{y}|\tan \theta$, where $\theta \in (-\frac{\pi }{2},\frac{\pi }{2})$, then

$$\begin{aligned} (-\Delta )^\alpha \tilde{u}(x)&= \int _{{\mathbb {R}}^{N-1}}\int ^{\frac{\pi }{2}}_{-\frac{\pi }{2}}\frac{\tilde{u}(\tilde{x})-\tilde{u}(\tilde{x}-\tilde{y})}{|\tilde{y}|^{N-1+2\alpha }}(\cos \theta )^{N-2+2\alpha }\,d\theta \,d\tilde{y}\\&= \int _{{\mathbb {R}}^{N-1}}\frac{\tilde{u}(\tilde{x})-\tilde{u}(\tilde{x}-\tilde{y})}{|\tilde{y}|^{N-1+2\alpha }}\,d\tilde{y} \int ^{\frac{\pi }{2}}_{-\frac{\pi }{2}}(\cos \theta )^{N-2+2\alpha }\,d\theta , \end{aligned}$$

and

$$\begin{aligned} \int ^{\frac{\pi }{2}}_{-\frac{\pi }{2}}(\cos \theta )^{N-2+2\alpha }\,d\theta =2\int ^{\frac{\pi }{2}}_{0}(\cos \theta )^{N-2+2\alpha }\,d\theta <+\infty , \end{aligned}$$

since $N-2+2\alpha >0$. This mean that the $N$-dimension fractional Laplacian operator is actually $(N-1)$-dimension, and we have (3.12) in ${\mathbb {R}}^{N-1}$. $\square$

Proof of Theorem 1.1

Theorem 1.1 is an immediate consequence of Theorems 1.2 and 3.2. $\square$

4 Proof of Theorem 1.5

The aim of this section is to prove Theorem 1.5 by the method of moving plane applied to system in ${\mathbb {R}}^N_+$. As the proof of Theorem 1.1, we first show the following theorem:

Theorem 4.1

Suppose we have a positive viscosity bounded solution $(u,v)$ of

$$\begin{aligned} \left\{ \begin{array}{l@{\quad }l} (-\Delta )^\alpha u=f(v) &{} x\in {\mathbb {R}}^N_+,\\ (-\Delta )^\alpha v=g(u) &{} x\in {\mathbb {R}}^N_+,\\ u=v=0 &{}x\in {\mathbb {R}}^N{\setminus }{\mathbb {R}}^N_+ \end{array} \right. \end{aligned}$$

(4.1)

where $f(v)$ and $g(u)$ are locally Lipschitz continuous functions with $f(0)\ge 0,\ g(0)\ge 0$ and nondecreasing in $v$ and $u$ respectively. Then $(u,v)$ is strictly increasing in $x_N$-direction.

Let $\Sigma _\mu$ and $T_\mu$ be defined as in Sect. 3. For $x=(\tilde{x}, x_N)\in {\mathbb {R}}^N$, we denote

$$\begin{aligned} u_{\mu }(x)=u(x_{\mu }), \ w_{\mu ,u}(x)=u_{\mu }(x)-u(x),\\ v_{\mu }(x)=v(x_{\mu }),\quad \hbox {and} \quad w_{\mu ,v}(x)=v_{\mu }(x)-v(x), \end{aligned}$$

where $\mu >0$ and $x_{\mu }=(\tilde{x},2\mu - x_N)$.

Proof of Theorem 4.1

We divide the proof in two steps.

Step 1: We first prove that if $\mu$ is small enough, then $w_{\mu ,u}>0$ and $w_{\mu ,v}>0$ in $\Sigma _{\mu }$. For this purpose, we define

$$\begin{aligned} \Sigma _{\mu ,u}^-=\{x\in \Sigma _{\mu }\,|\,w_{\mu ,u}(x)<0\}\quad \hbox {and}\quad \Sigma _{\mu ,v}^-=\{x\in \Sigma _{\mu }\,|\,w_{\mu ,v}(x)<0\}. \end{aligned}$$

We will show that $\Sigma _{\mu ,u}^-$ is empty if $\mu$ is small enough. First we assume by contradiction that $\Sigma _{\mu ,u}^-$ is not empty and define

$$\begin{aligned} w_{\mu ,u}^1(x) = \left\{ \begin{array}{lll} w_{\mu ,u}(x) &{} \quad x\in \Sigma _{\mu ,u}^-,\\ 0 &{} \quad x\in {\mathbb {R}}^N{\setminus }\Sigma _{\mu ,u}^-, \end{array} \right. \end{aligned}$$

(4.2)

$$\begin{aligned} w_{\mu ,u}^2(x) = \left\{ \begin{array}{lll} 0 &{} \quad x\in \Sigma _{\mu ,u}^-,\\ w_{\mu ,u}(x) &{} \quad x\in {\mathbb {R}}^N{\setminus }\Sigma _{\mu ,u}^-. \end{array} \right. \end{aligned}$$

(4.3)

Using the arguments given in Step 1 of the proof of Theorem 3.1, we can obtain that

$$\begin{aligned} (-\Delta )^\alpha w^1_{\mu ,u}(x)\ge (-\Delta )^\alpha w_{\mu ,u}(x)\quad \hbox {and}\quad (-\Delta )^\alpha w^2_{\mu ,u}(x)\le 0,\quad \hbox {for all }\ x\in \Sigma _{\mu ,u}^-. \end{aligned}$$

From here, for $x\in \Sigma _{\mu ,u}^-$ we have

$$\begin{aligned} (-\Delta )^\alpha w^1_{\mu ,u}&\ge (-\Delta )^\alpha u_{\mu }(x)-(-\Delta )^\alpha u(x)\\&= f(v(x))-f(v_{\mu }(x))\\&= \frac{f(v(x))-f(v_{\mu }(x))}{v_{\mu }(x)- v(x)}w_{\mu ,v}. \end{aligned}$$

Let us define $\varphi _v(x)=(f(v(x))-f(v_{\mu }(x)))/(v_{\mu }(x)- v(x))$ for $x\in \Sigma _{\mu ,u}^-$. By assumption we have that $\varphi _v(x)\in L^\infty (\Sigma _{\mu ,u}^-)$ and $\varphi _vw_{\mu ,v}$ is continuous. On the other hand, since $w^1_{\mu ,u}=0$ in ${\mathbb {R}}^N{\setminus }\Sigma _{\mu ,u}^-$ using Theorem 2.3 we have

$$\begin{aligned} \Vert w^1_{\mu ,u}\Vert _{L^\infty (\Sigma _{\mu ,u}^-)}\le CR(\Sigma _{\mu ,u}^-)^{2\alpha }\Vert \varphi _v(x)w_{\mu ,v}\Vert _{L^\infty (\Sigma _{\mu ,u}^-)}. \end{aligned}$$

Since $f(v)$ is nondecreasing in $v$, we have

$$\begin{aligned} \varphi _v(x)w_{\mu ,v}&= f(v(x))-f(v_{\mu }(x))\le 0 \quad \hbox {in } \ \Sigma _\mu {\setminus }\Sigma _{\mu ,v}^- \quad \hbox {and}\\ \varphi _v(x)w_{\mu ,v}&= f(v(x))-f(v_{\mu }(x))>0 \quad \hbox {in } \ \Sigma _{\mu ,v}^- \end{aligned}$$

Denote $\Sigma _{\mu }^-=\Sigma _{\mu ,u}^-\cap \Sigma _{\mu ,v}^-$, we can obtain that

$$\begin{aligned} \Vert w^1_{\mu ,u}\Vert _{L^\infty (\Sigma _{\mu ,u}^-)}&\le CR(\Sigma _{\mu ,u}^-)^{2\alpha }\Vert \varphi _v(x)w_{\mu ,v}\Vert _{L^\infty (\Sigma _{\mu }^-)}\\&\le CR(\Sigma _{\mu ,u}^-)^{2\alpha }\Vert \varphi _v(x)\Vert _{L^\infty (\Sigma _{\mu }^-)}\Vert w_{\mu ,v}\Vert _{L^\infty (\Sigma _{\mu }^-)}\\&\le CR(\Sigma _{\mu ,u}^-)^{2\alpha }\Vert w_{\mu ,v}\Vert _{L^\infty (\Sigma _{\mu }^-)}, \end{aligned}$$

where in the last inequality we use the fact $\varphi _v(x)$ is locally Lipschitz continuous and we have changed the constant $C$ if necessary. Similar to (4.2) and (4.3), we define

$$\begin{aligned} w_{\mu ,v}^1(x) = \left\{ \begin{array}{lll} w_{\mu ,v}(x), &{} \quad x\in \Sigma _{\mu ,v}^-,\\ 0, &{} \quad x\in {\mathbb {R}}^N{\setminus }\Sigma _{\mu ,v}^-, \end{array} \right. \end{aligned}$$

(4.4)

$$\begin{aligned} w_{\mu ,v}^2(x) = \left\{ \begin{array}{lll} 0, &{} \quad x\in \Sigma _{\mu ,v}^-,\\ w_{\mu ,v}(x), &{} \quad x\in {\mathbb {R}}^N{\setminus }\Sigma _{\mu ,v}^- \end{array} \right. \end{aligned}$$

(4.5)

and argue in a completely analogous way to obtain

$$\begin{aligned} \Vert w^1_{\mu ,v}\Vert _{L^\infty (\Sigma _{\mu ,v}^-)}\le CR(\Sigma _{\mu ,v}^-)^{2\alpha }\Vert w_{\mu ,u}^1\Vert _{L^\infty (\Sigma _{\mu }^-)}. \end{aligned}$$

Thus

$$\begin{aligned} \Vert w^1_{\mu ,u}\Vert _{L^\infty (\Sigma _{\mu ,u}^-)}\le CR(\Sigma _{\mu ,v}^-)^{2\alpha }R(\Sigma _{\mu ,u}^-)^{2\alpha }\Vert w_{\mu ,u}^1\Vert _{L^\infty (\Sigma _{\mu ,u}^-)} \end{aligned}$$

and

$$\begin{aligned} \Vert w^1_{\mu ,v}\Vert _{L^\infty (\Sigma _{\mu ,v}^-)}\le CR(\Sigma _{\mu ,v}^-)^{2\alpha }R(\Sigma _{\mu ,u}^-)^{2\alpha }\Vert w_{\mu ,v}^1\Vert _{L^\infty (\Sigma _{\mu ,v}^-)}. \end{aligned}$$

Now we just choose $\mu$ small enough such that $CR(\Sigma _{\mu ,v}^-)^{2\alpha }R(\Sigma _{\mu ,u}^-)^{2\alpha }<1$ and we conclude that $\Vert w^1_{\mu ,u}\Vert _{L^\infty (\Sigma _{\mu ,u}^-)}=0$, so $|\Sigma _{\mu ,u}^-|=0$. Since $\Sigma _{\mu ,u}^-$ is open, we have that $\Sigma _{\mu ,u}^-$ is empty, which is a contradiction.

Thus we have $w_{\mu ,u}\ge 0$ in $\Sigma _\mu$ when $\mu$ is small enough. Similarly, we can obtain $w_{\mu ,v}\ge 0$ in $\Sigma _\mu$ when $\mu$ is small enough. In order to complete Step 1 we will prove a bit more general statement that will be useful later, that is, given $\mu >0$, if $w_{\mu ,u}\ge 0$, $w_{\mu ,v}\ge 0$, $w_{\mu ,u}\not \equiv 0$ and $w_{\mu ,v}\not \equiv 0$ in $\Sigma _\mu$, then $w_{\mu ,u}>0$ and $w_{\mu ,v}>0$ in $\Sigma _\mu$. For proving this property suppose there exists $x_0\in \Sigma _\mu$ such that $w_{\mu ,u}(x_0)=0$.

On one hand, by using similar arguments yielding (3.8) we find that

$$\begin{aligned} (-\Delta )^\alpha w_{\mu ,u}(x_0)<0. \end{aligned}$$

(4.6)

On the other hand, by our assumption $w_{\mu ,v}(x_0)=v_\mu (x_0)-v(x_0)\ge 0$, from the monotonicity on $f$, we obtain

$$\begin{aligned} (-\Delta )^\alpha w_{\mu ,u}(x_0)=f(v_\mu (x_0))-f(v(x_0))\ge 0, \end{aligned}$$

which is impossible with (4.6). This completes Step 1.

Step 2. We define

$$\begin{aligned} \mu ^*=\hbox {sup}\{{\mu }\,|\,w_{\nu ,u}>0, w_{\nu ,v}>0 \ \hbox {in }\ \Sigma _\nu ,\ \forall \nu <\mu \}>0. \end{aligned}$$

We see that for each $0<\mu \le \mu ^*$ the function $w_{\mu ,u}\ge 0$ and $w_{\mu ,v}\ge 0$ in $\Sigma _\mu$ which imply $(u,v)$ is strictly increasing in $x_N$ direction. Therefore, the theorem is proved if we show that $\mu ^*=+\infty$.

Suppose for contradiction that $\mu ^*$ is finite. By Theorem 2.4 we can fix $\varepsilon _0$ such that the operator $\Delta ^\alpha -\varphi _u(x)$ and $\Delta ^\alpha -\varphi _v(x)$ (where $\varphi _u(x)=\frac{g(u(x))-g(u_{\mu }(x))}{u_{\mu }(x)- u(x)}$ and $\varphi _v(x)=\frac{f(v(x))-f(v_{\mu }(x))}{v_{\mu }(x)- v(x)}$) satisfies the maximum principle in the domain $\Sigma _{\mu ^*+\varepsilon _0}{\setminus }\Sigma _{\mu ^*-\varepsilon _0}$. For instance, we can take $\varepsilon _0 = (\omega _N/2^{ (N+1)})\bar{R}$, where $\bar{R}$ is number from Theorem 2.4. $\square$

Similar discus as Lemma 3.1 (see also Lemma 3.1 in [13]), we have

Lemma 4.1

There exists $\delta _0\in (0,\varepsilon _0]$, such that for each $\delta \in (0,\delta _0]$ we have

$$\begin{aligned} w_{\mu ^*+\delta ,u}&> 0 \quad \hbox {in}\quad \Sigma _{\mu ^*-\varepsilon _0}{\setminus }\Sigma _{\varepsilon _0},\\ w_{\mu ^*+\delta ,v}&> 0 \quad \hbox {in} \quad \Sigma _{\mu ^*-\varepsilon _0}{\setminus }\Sigma _{\varepsilon _0}. \end{aligned}$$

Then we can repeat the Step 1 and apply Theorem 2.4 to equation in $\Sigma _{\mu ^*+\delta }{\setminus }\Sigma _{\mu ^*-\varepsilon _0}$ and in $\Sigma _{\varepsilon _0}$ (those domains are narrow enough) to conclude that $w_{\mu ^*+\delta ,u}\ge 0$ in $\Sigma _{\mu ^*+\delta }$ for each $\delta \in (0,\delta _0)$. This contradicts the maximal choice of $\mu ^*$.

The same argument as Theorem 3.2 we have

Theorem 4.2

Under the hypotheses of Theorem 4.1, if

$$\begin{aligned} \left\{ \begin{array}{l@{\quad }l} (-\Delta )^\alpha u=f(v) &{} x\in {\mathbb {R}}^N_+,\\ (-\Delta )^\alpha v=g(u) &{} x\in {\mathbb {R}}^N_+,\\ u=v=0 &{}x\in {\mathbb {R}}^N{\setminus }{\mathbb {R}}^N_+ \end{array} \right. \end{aligned}$$

(4.7)

has a positive bounded solution, then the same problem has a positive solution in ${\mathbb {R}}^{N-1}$.

Proof of Theorem 1.5

Theorem 1.5 is a direct conclusion of Theorems 1.3, 1.4 and 4.2. $\square$