Rigidity of Free Boundary Minimal Disks in Mean Convex Three-Manifolds

Abstract

The purpose of this article is to study rigidity of free boundary minimal two-disks that locally maximize the modified Hawking mass on a Riemannian three-manifold with a positive lower bound on its scalar curvature and mean convex boundary. Assuming the strict stability of $\Sigma$, we prove that a neighborhood of it in M is isometric to one of the half de Sitter–Schwarzschild space.

1 Introduction and Statement of the Main Results

In the last decades very much attention has been given to study Riemannian manifolds with scalar curvature bounded from below since it has strong connections with theory of minimal surfaces.

Several rigidity results involving the scalar curvature have been obtained assuming the existence of area-minimizing surfaces. In Cai and Galloway showed that if a Riemannian three-manifold with nonnegative scalar curvature contains an embedded, two-sided, locally area-minimizing two-torus $\Sigma$, then the metric is flat in some neighborhood of $\Sigma$. Later, in [3] Bray, Brendle and Neves showed that if $(M^3, g)$ is a Riemannian three-manifold with scalar curvature $R\ge 2$ and $\Sigma ^2\subset M^3$ is an embedded two-sphere which is locally area-minimizing, then $\Sigma$ has area less than or equal to $4\pi$ and if moreover the equality holds, then with the induced metric $\Sigma$ has constant Gauss curvature equal to 1 and locally M splits along $\Sigma$. Afterward, Nunes studied the hyperbolic setting; he proved that if $(M^3, g)$ is a Riemannian three-manifold with scalar curvature $R\ge -2$ and $\Sigma ^2\subset M^3$ is a two-sided compact embedded Riemann surface of genus $\gamma (\Sigma )\ge 2$ which is locally area-minimizing, then the area of $\Sigma$ with respect to the induced metric is greater than or equal to $4\pi (\gamma (\Sigma )-1)$. Moreover, if equality holds, then $\Sigma$ has constant Gauss curvature equal to $-1$ and locally M splits along $\Sigma$. In the nice work [15], Micallef and Moraru unified the approach of the results proved in, [3] and. Finally, similar rigidity results have been obtained for closed stable MOTS in initial data sets by Galloway and Mendes in [14], as well as by Mendes in [13].

Motivated by these results, Máximo and Nunes [12] established a local rigidity result for the de Sitter–Schwarzschild space, involving strictly stable minimal surfaces and the Hawking mass. For this, it is important to recall the quasi-local, so-called Hawking mass of a compact surface $\Sigma ^2\subset (M^3,g)$, which is defined as

$$\begin{aligned} m_{H}(\Sigma )=\sqrt{\dfrac{|\Sigma |}{16\pi }}\left( \frac{\chi (\Sigma ) }{2}-\frac{1}{16\pi }\int _{\Sigma }\left( H^2+\frac{2}{3}\Lambda \right) d\sigma \right) , \end{aligned}$$

(1.1)

where $\chi (\Sigma )$ is the Euler characteristics of $\Sigma$, H is the mean curvature of $\Sigma$, $|\Sigma |$ is the area of $\Sigma$ and $\Lambda$ is the infimum of the scalar curvature. This notion of quasi-local mass plays a crucial role in the proof of the Riemannian Penrose inequality for assymptotically flat manifolds, discovered independently by Huisken and Ilmanen and Bray.

More precisely, Máximo and Nunes proved the following result.

Theorem 1

(Máximo and Nunes) Let (M, g) be a Riemannian three-manifold with scalar curvature $R\ge 2$. If $\Sigma \subset M$ is an embedded strictly stable minimal two-sphere which locally maximizes the Hawking mass, then the Gauss curvature of $\Sigma$ is constant, equal to $1/a^2$ for some $a\in (0, 1)$ and a neighborhood of $\Sigma$ in (M, g) is isometric to the de Sitter–Schwarzschild space $((-\varepsilon , \varepsilon )\times \Sigma , g_a)$ for some $\varepsilon >0$.

In the presence of a nonempty boundary, the objects of study are the free boundary minimal surfaces. These surfaces arise as critical points of the area functional for surfaces in a three dimensional manifold M with boundary in $\partial M$. It follows from the first variation area formula that such surfaces intersect $\partial M$ orthogonally.

In this context, Ambrozio [2] established the following boundary version of the aforementioned rigidity results.

Theorem 2

(Ambrozio) Let (M, g) be a Riemannian three-manifold with boundary $\partial M$. Assume that R and $H^{\partial M}$ are bounded from below. If $\Sigma$ is a properly immersed, two-sided, free boundary stable minimal surface, then

$$\begin{aligned} I (\Sigma )=\frac{1}{2}\textrm{inf} R|\Sigma |+\textrm{inf} H^{\partial M} \le 2\pi \chi (\Sigma ). \end{aligned}$$

Assume that (M, g) has mean convex boundary and $\Sigma$ is a properly embedded, two-sided, locally area-minimizing free boundary surface such that

$$\begin{aligned} I (\Sigma ) = 2\pi \chi (\Sigma ). \end{aligned}$$

In addition, if one of the following hypotheses holds:

• (i) each component of $\partial \Sigma$ is locally length-minimizing in $\partial M$; or

• (ii) $\textrm{inf}H^{\partial M} = 0$;

then there exists a neighborhood of $\Sigma$ in (M, g) that is isometric to $((-\varepsilon ,\varepsilon )\times \Sigma ,dt^2+g_{\Sigma })$, where $(\Sigma , g_{\Sigma })$ has constant Gaussian curvature $\frac{1}{2}\textrm{inf} R$ and $\partial \Sigma$ has constant geodesic curvature $\textrm{inf} H^{\partial M}$ in $\Sigma$.

A similar result was obtained by Alaee, Lesourd and Yau [1], in the context of free boundary marginally outer trapped surfaces in initial data sets.

Our main goal is to present a connection between free boundary minimal surfaces and General Relativity using the modified Hawking mass. We recall that the modified Hawking mass of a compact free boundary $\Sigma ^2$ in a Riemannian three-manifold $(M^3, g)$ is defined to be

$$\begin{aligned} \widetilde{m}_{H}(\Sigma )=\sqrt{\dfrac{|\Sigma |}{8\pi }}\left( \chi (\Sigma )-\frac{1}{8\pi }\int _{\Sigma }\left( H^2+\frac{2}{3}\Lambda \right) d\sigma \right) , \end{aligned}$$

(1.2)

where where $\chi (\Sigma )$ is the Euler characteristics of $\Sigma$, H is the mean curvature of $\Sigma$, $|\Sigma |$ is the area of $\Sigma$ and $\Lambda$ is the infimum of the scalar curvature. This notion was introduced by Marquardt [11] to study weak solutions of the free boundary inverse mean curvature flow, and played an important role in the proof of the Riemannian Penrose inequality for asymptotically flat manifolds with a non-compact boundary given by Koerber [9].

Before we proceed, let us provide a precise definition of model space of this present paper, namely half de Sitter–Schwarzschild manifold which was first introduced by de Lima; see Remark 4.8 of [7]. Fix $0<m<\frac{1}{3\sqrt{3}}$, the half de Sitter–Schwarzschild metric with mass parameter m and scalar curvature equal to 2 is defined as the metric

$$\begin{aligned} g_{m}=\frac{1}{1-\frac{r^2}{3}-\frac{2m}{r}}dr^2+r^2g_{\mathbb {S}_{+}^2}, \end{aligned}$$

(1.3)

define on $(r_-(m),r_+(m))\times \mathbb {S}_{+}^2$, where $0< r_{-}(m)< r_{+}(m) < 1$ are the only positive solution of $f_m(r) = 1 -r^2 -2mr^{-1}$, $f_m(r)>0$ for all $r\in (r_-(m),r_+(m))$ and $g_{\mathbb {S}_{+}^2}$ is the standard metric on $\mathbb {S}_{+}^2$ with constant Gauss curvature equal to 1. Additionally, this manifold carries a totally geodesic inner boundary $(r_-(m),r_+(m))\times \mathbb {S}^1$.

Through a change of variable, the half de Sitter–Schwarzschild metric can be rewritten as

$$\begin{aligned} g_{m} = ds^2 + u(s)^2g_{\mathbb {S}_{+}^2},\ \ \ \textrm{on}\ \ \ [a, b]\times \mathbb {S}^2, \end{aligned}$$

(1.4)

where $u: (a, b)\rightarrow (r_-(m),r_+(m))$ is a function that extends continuously to [a, b] with $u(a) = r_-(m)$, $u(b) = r_+(m)$ and $\frac{ds}{dr}=f_m(r)^{-1/2}>0$ on $(r_-(m),r_+(m))$.

After reflection of the metric $g_{m}$, we can define a complete periodic rotationally symmetric metric on $\mathbb {R}\times \mathbb {S}_{+}^2$ with scalar curvature to $R=2$ and totally geodesic boundary $\mathbb {R}\times \mathbb {S}^1$. Moreover, the function u solves the following second-order nonlinear differential equation

$$\begin{aligned} u''(s)=\frac{1}{2}\Big (\frac{1-u'(s)^2}{u(s)}\Big )-\frac{u(s)}{2}. \end{aligned}$$

(1.5)

Moreover, any positive solutions u(r) to (1.5) that is defined for all $r\in \mathbb {R}$, define a periodic rotationally symmetric metric $g_a = dr^2 + u_a(r)^2g_{\mathbb {S}_+^{2}}$ with constant scalar curvature equal to 2, where $a\in (0, 1)$ and $u_a(r)$ satisfies $u_a(0) = a = \min u$ and $u_a'(0) = 0$; we refer to [17], Sect. 2.1] for details. This metric is precisely the half de Sitter–Schwarzschild metrics on $\mathbb {R}\times \mathbb {S}_+^{2}$ defined above.

In our first result we show that slices of the half de Sitter–Schwarzschild are local maxima in the following sense.

Theorem 3

Let $\Sigma _r = \{r\}\times \mathbb {S}_{+}^2$ be a slice of the half de Sitter–Schwarzschild manifold $(\mathbb {R}\times \mathbb {S}_{+}^2, g_m)$. Then there exists an $\varepsilon =\varepsilon (r)>0$ such that if $\Sigma \subset \mathbb {R}\times \mathbb {S}_{+}^2$ is a free boundary properly embedded two-disk, which is a normal graph over $\Sigma _r$ given by $\phi \in C^{2}(\Sigma _r)$ with $||\phi ||_{C^{2}(\Sigma _r)}<\varepsilon$, one has

• (i) either $\widetilde{m}_H(\Sigma ) < \widetilde{m}_H(\Sigma _r)$;

• (ii) or $\Sigma$ is a slice $\Sigma _s$ for some s.

One should point out that the problem of investigating the rigidity/flexibility of the half de Sitter–Schwarzschild manifolds appeared in the good survey due to de Lima [Remark 4.8, [7]].

In our next result, we establish the following local rigidity theorem:

Theorem 4

Let (M, g) be a Riemannian three-manifold with boundary $\partial M$ and satisfies $R\ge 2$ and $H^{\partial M}\ge 0$. If $\Sigma$ is a properly embedded, two-sided, free boundary strictly stable minimal two-disk which locally maximizes the modified Hawking mass, then the Gauss curvature of $\Sigma$ is constant equal to $1/a^2$ for some $a\in (0, 1)$, the geodesic curvature of $\partial \Sigma$ vanishes and a neighborhood of $\Sigma$ in (M, g) is isometric to the half de Sitter–Schwarzschild space $((-\varepsilon , \varepsilon )\times \Sigma , g_a)$ for some $\varepsilon >0$.

We point out that Thoerem 4 is a free boundary version of the results due to Máximo and Nunes [12]. Furthermore, a relevant observation is that the metric $g_m$ defined in (1.4) converges to the standard product metric $dr^2 + g_{\mathbb {S}_{+}^2}$ on $\mathbb {R}\times \mathbb {S}_{+}^2$ when $a\rightarrow 1$. Moreover, one can easily verify that $\Sigma _0 =\{0\}\times \mathbb {S}_{+}^2$ is a free boundary strictly stable minimal (in fact, totally geodesic) two-disk of area $2\pi a^2$ in $(\mathbb {R}\times \mathbb {S}_{+}^{2}, g_m)$, for each $a\in (0, 1)$. However, in the standard product metric $dr^2 + g_{\mathbb {S}_{+}^2}$, that is, in the limit as $a\rightarrow 1$, $\Sigma _0$ is only stable and not strictly so. Consequently, our theorem provides, in some sense, a generalization of the theorem due to Ambrozio [2].

The outline of the paper is as follows: In Sect. 2, we review some key aspects of the stability of free boundary minimal surfaces and derive the variation formulae of the modified Hawking mass. In Sect. 3, inspired by arguments in [2, 12] we use the strictly stability and the locally maximizes of the modified Hawking mass to construct a foliation of M in a neighborhood of $\Sigma$ by constant mean curvature (CMC) embedded free boundary surfaces. Finally, in Sect. 4 we prove Theorems 3 and 4.

2 Preliminaries

During this section we will gather some basic facts and key lemmas that will be useful for establishment of the main results. Let (M, g) be a Riemannian three-manifold with nonempty boundary $\partial M$. Denote by R the scalar curvature of M and denote the mean curvature of $\partial M$ by $H^{\partial M}$. Let $\Sigma ^2$ be a smooth compact manifold with nonempty boundary, and suppose that $\Sigma$ is properly embedded into M, that is, $\Sigma$ is embedded into $M^3$ and $\partial \Sigma =\Sigma \cap \partial M$.

Moreover, we assume that $\Sigma$ is two-sided, that is, there exists a unit vector field N along $\Sigma$ that is normal to $\Sigma$. Fix a unit normal vector field X outward pointing unit normal for $\partial M$, and let $\nu$ be the outward pointing conormal along $\partial \Sigma$ in $\Sigma$. Let $A(U,V) = g(-\nabla _U N,V)$ be the second fundamental form of $\Sigma$, and let $\Pi (u,v)=g(\nabla _uX,v)$ be the second fundamental form of $\partial M$ with respect to $-X$. We say that $\Sigma$ is free boundary if $\Sigma$ meets $\partial M$ orthogonally. In other words, $\Sigma$ is free boundary if $\nu = X$ along $\partial \Sigma$.

Let $f:\Sigma \times (-\varepsilon ,\varepsilon )\rightarrow M$ be a properly smooth normal variation of $\Sigma$, that is, f is a smooth map such that,

• for every $t\in (-\varepsilon ,\varepsilon )$, the map $f_t =f(\cdot ,t):\Sigma \rightarrow M$ is a properly embedded into M;

• $f(x,0)=x$ for every $x\in \Sigma$;

• $\frac{\partial f}{\partial t} (x,0) = \phi (x)N(x)$ for each $x\in \Sigma$, where $\phi \in C^{\infty }(\Sigma )$.

We now recall some well-knowns evolution equations of relevant geometric quantities. For this purpose, we will use the subscript t to denote quantities associated to $\Sigma _t = f_t(\Sigma )$. More precisely, $N_t$ will denote a local unit vector field normal to $\Sigma _t$, $H_t$ the mean curvature of $\Sigma _t$, $\nu _t$ the outward pointing conormal along $\partial \Sigma _t$ and $\phi _t$ is the function on $\Sigma _t$ defined by $\phi _t = g\big (\frac{\partial f}{\partial t},N_t\big )$.

First, we recall the variation of the mean curvature

$$\begin{aligned} \left\{ \begin{array}{ll}\frac{d}{dt}H(t)\Big |_{t=0}=L_{\Sigma }\phi \ \ on\ \ \Sigma ,\\ \frac{d}{dt}g(N_t, X)\Big |_{t=0}=-\frac{\partial \phi }{\partial \nu }+\Pi (N,N)\phi \ \ along\ \ \partial \Sigma ,\end{array}\right. \end{aligned}$$

(2.1)

where $L_{\Sigma }=\Delta _{\Sigma }+Ric(N,N)+|A|^2$ is the Jacobi operator of $\Sigma$, Ric is the Ricci tensor of M and $\Delta _{\Sigma }$ is the Laplace operator of $\Sigma$ with respect to the induced metric from M.

In particular, if each $\Sigma _t$ is a constant mean curvature free boundary surface, then

$$\begin{aligned} \left\{ \begin{array}{ll}\frac{d}{dt}H(t)=L_{\Sigma _t}\phi _t \ \ \textrm{on}\ \ \Sigma ,\\ \frac{\partial \phi _t}{\partial \nu _t}=\Pi (N_t,N_t)\phi _t\ \ \textrm{along}\ \ \partial \Sigma .\end{array}\right. \end{aligned}$$

(2.2)

The first variation of area is given by

$$\begin{aligned} \frac{d}{dt}|\Sigma _t|\Big |_{t=0}=-\int _{\Sigma }H\phi d\sigma +\int _{\partial \Sigma }\phi g(N, \nu )ds, \end{aligned}$$

(2.3)

where $H = tr A$ is the mean curvature of $\Sigma$ in M. From (2.3), it follows that $\Sigma$ is a critical point for the area functional if and only if $\Sigma$ is minimal with free boundary. For more details see [2].

If $\Sigma$ is minimal with free boundary, then the second variation of area is given by

$$\begin{aligned} \frac{d^2}{dt^2}|\Sigma _t|\Big |_{t=0}=-\int _{\Sigma }\phi L_{\Sigma }\phi d\sigma +\int _{\partial \Sigma }\Big (\frac{\partial \phi }{\partial \nu }-\Pi (N,N)\phi \Big )\phi ds. \end{aligned}$$

(2.4)

Alternatively, in terms of the quadratic form this can be written

$$\begin{aligned} \frac{d^2}{dt^2}|\Sigma _t|\Big |_{t=0}= \mathcal {Q}(\phi ,\phi ), \end{aligned}$$

where the quadratic form $\mathcal {Q}:C^{\infty }(\Sigma )\times C^{\infty }(\Sigma )\rightarrow \mathbb {R}$ is called the index form of $\Sigma$ given by

$$\begin{aligned} \mathcal {Q}(\phi ,\psi )= & {} -\int _{\Sigma }\phi (\Delta _{\Sigma }\psi +(Ric(N,N)+|A|^2)\psi ) d\sigma \nonumber \\{} & {} +\int _{\partial \Sigma }\phi \Big (\frac{\partial \psi }{\partial \nu }-\Pi (N,N)\psi \Big ) ds. \end{aligned}$$

(2.5)

The boundary Robin condition

$$\begin{aligned} \frac{\partial \phi }{\partial \nu }=\Pi (N,N)\phi \end{aligned}$$

is an elliptic boundary condition for $L_{\Sigma }$, therefore there exists a non-decreasing and diverging sequence $\lambda _1\le \lambda _2\le \cdots \le \lambda _k\nearrow \infty$ of eigenvalues associated to a $L^2(M, d\sigma )$ orthonormal basis of solutions to the eigenvalue problem

$$\begin{aligned} \left\{ \begin{array}{ll}L_{\Sigma }\phi +\lambda \phi =0\ \ \textrm{on}\ \ \Sigma ,\\ \frac{\partial \phi }{\partial \nu }=\Pi (N,N)\phi \ \ \textrm{along}\ \ \partial \Sigma .\end{array}\right. \end{aligned}$$

(2.6)

Then $\Sigma$ is called stable if and only if $\mathcal {Q}(\phi ,\phi )\ge 0$ for all $\phi \in C^{\infty }(\Sigma )$, where the lowest eigenvalue of $L_{\Sigma }$ is nonnegative. If the lowest eigenvalue is positive we say the surface is strictly stable. Therefore, if $\phi$ is an eigenfunction of the Jacobi operator associated with that first eigenvalue, that is, $\mathcal {Q}(\phi ,\phi )=\lambda _1(L_{\Sigma })\int _{\Sigma }\phi ^2d\sigma$, then

$$\begin{aligned} \lambda _1(L_{\Sigma })\int _{\Sigma }\phi ^2d\sigma =-\int _{\Sigma }\phi (\Delta _{\Sigma }\phi +(Ric(N,N)+|A|^2)\phi )d\sigma . \end{aligned}$$

(2.7)

Next, we derive first and the second variation formulae for the modified Hawking mass.

Proposition 1

Let (M, g) be a Riemannian three-manifold with boundary $\partial M$ and scalar curvature R bounded below. Let $\Sigma$ be a properly embedded, two-sided, free boundary surface. For a given normal variation as in the discussions above, we have

$$\begin{aligned} \left. \frac{d}{dt}\widetilde{m}_{H}(\Sigma _{t})\right| _{t=0}= & {} -\frac{2|\Sigma |^{-\frac{1}{2}}}{(8\pi )^{\frac{3}{2}}}\int _{\Sigma }H\Delta _{\Sigma }\phi d\sigma +\frac{|\Sigma |^{\frac{1}{2}}}{(8\pi )^{\frac{3}{2}}}\int _{\Sigma }(\Lambda -R)H\phi d\sigma \\{} & {} +\frac{|\Sigma |^{\frac{1}{2}}}{(8\pi )^{\frac{3}{2}}}\int _{\Sigma }\left[ 2K_{\Sigma }-\frac{4\pi \chi (\Sigma )}{|\Sigma |}+\frac{1}{2|\Sigma |}\int _{\Sigma }H^2 d\sigma -|A|^2\right] H\phi d\sigma , \end{aligned}$$

where $\Lambda =\inf R$ and $K_{\Sigma }$ is the Gaussian curvature of $\Sigma$.

Proof

Taking the derivative of the modified Hawking mass given in (1.2) for the variation $f_t$ of $\Sigma$ provided previously to obtain

$$\begin{aligned} \frac{d}{dt}\widetilde{m}_{H}(\Sigma _{t})= & {} \frac{1}{2}\frac{|\Sigma _{t}|^{-\frac{1}{2}}}{(8\pi )^{\frac{1}{2}}}\left( 1-\frac{1}{8\pi }\int _{\Sigma _{t}}H_{t}^2d\sigma _{t}-\frac{1}{12\pi }\Lambda |\Sigma _{t}|\right) \frac{d}{dt}d(\sigma _{t})\\{} & {} + \frac{|\Sigma _{t}|^{\frac{1}{2}}}{(8\pi )^{\frac{1}{2}}}\left( -\frac{1}{8\pi }\int _{\Sigma _{t}}\left[ 2H_{t}\frac{d}{d t}(H_{t})d\sigma _{t}+H_{t}^2\frac{d}{dt}(d\sigma _{t})\right] -\frac{1}{12\pi }\Lambda \frac{d}{dt}(d\sigma _{t})\right) , \end{aligned}$$

where $\frac{d}{dt}d\sigma _t$ denotes the first variation of area functional. Hence, using the fact that $\Sigma$ is free boundary at $t=0$ and the Gauss equation

$$\begin{aligned} 2Ric(N,N)=R-2K_{\Sigma }+H^2-|A|^2, \end{aligned}$$

(2.8)

the first variation formula for the modified Hawking mass follows by a straightforward computation using the following identities (2.3) and (2.1). $\square$

In order to set the stage for the proof of the second variation formula of the modified Hawking mass, it is crucial to recall the first variation of the Jacobi operator obtained by Máximo and Nunes [12].

Proposition 2

([12], Proposition 6.2) For each function $\psi \in C^{\infty }(\Sigma )$, we have:

$$\begin{aligned} L_{\Sigma }'(0)\psi= & {} \frac{d}{dt}L_{\Sigma _t}\Big |_{t=0}\psi = 2\phi g(A, Hess\psi )+2\psi g(A, Hess\phi )- 2\phi \omega (\nabla \psi )\\{} & {} -\quad 2\psi \omega (\nabla \phi )+\phi g(\nabla H, \nabla \psi )-Hg(\nabla \psi ,\nabla \phi )\\{} & {} +\quad 2A(\nabla \phi ,\nabla \psi ) -\psi div_{\Sigma }(div_{\Sigma }\omega )\\{} & {} -\quad \phi \psi HK_{\Sigma }+\phi \psi HRic(N,N)\\{} & {} +\quad \phi \psi H|A|^2+\phi \psi A_{ij}A_{ik}A_{jk}+\phi \psi R_{iNNj}A_{ij} \end{aligned}$$

where $\omega$ is the 1-form on $\Sigma$ defined by $\omega (X) = Ric(X,N)$.

Now, following the procedure adopted in [12], we shall present the second variation formula, which can be expressed as follows:

Proposition 3

Under the considerations of Proposition 1. If $\Sigma \subset M$ is a critical point of the modified Hawking mass, then

$$\begin{aligned} \left. \frac{d^2}{dt^2}\widetilde{m}_{H}(\Sigma _{t})\right| _{t=0}= & {} -\frac{3\widetilde{m}_{H}(\Sigma )}{4|\Sigma |^2}\left( \int _{\Sigma }H\phi d\sigma \right) ^2-\frac{2|\Sigma |^{\frac{1}{2}}}{(8\pi )^{\frac{3}{2}}}\int _{\Sigma }((L_{\Sigma }\phi )^2+HL_{\Sigma }'(0)\phi )d\sigma \\{} & {} +\frac{|\Sigma |^{\frac{1}{2}}}{(8\pi )^{\frac{3}{2}}}\int _{\Sigma }(H^2+\frac{2}{3}\Lambda )(\phi L_{\Sigma }\phi -H^2\phi ^2)d\sigma \\{} & {} +\frac{4|\Sigma |^{\frac{1}{2}}}{(8\pi )^{\frac{3}{2}}}\int _{\Sigma }H^2 \phi L_{\Sigma }\phi d\sigma \\{} & {} -\frac{\widetilde{m}_{H}(\Sigma )}{2|\Sigma |}\int _{\Sigma }(\phi L_{\Sigma }\phi -H^2 \phi ^2+div_{\Sigma }(\nabla _{X}X) )d\sigma \\{} & {} -\frac{|\Sigma |^{\frac{1}{2}}}{(8\pi )^{\frac{3}{2}}}\int _{\partial \Sigma }(H^2+\frac{2}{3}\Lambda )\left( \frac{\partial \phi }{\partial \nu }-g(N, \nabla _{N}X)\phi \right) \phi ds \\{} & {} + \frac{\widetilde{m}_{H}(\Sigma )}{2|\Sigma |}\int _{\partial \Sigma }\left( \frac{\partial \phi }{\partial \nu }-g(N, \nabla _{N}X)\phi \right) \phi ds, \end{aligned}$$

where $X(x)=\frac{\partial f}{\partial t}(x,0)$.

Proof

Once Proposition 1 is established, the above follows after a direct computation using the first variation formula of the modified Hawking mass and the fact that the second variation formula of the area element is given by

$$\begin{aligned} \frac{d^2}{dt^2}|\Sigma _t|\Big |_{t=0}=\int _{\Sigma }(-\phi L_{\Sigma }\phi +H^2\phi -Hdiv_{\Sigma }(\nabla _XX)) d\sigma +\int _{\partial \Sigma }\Big (\frac{\partial \phi }{\partial \nu }-\Pi (N,N)\phi \Big )\phi ds \end{aligned}$$

together with our assumption that $\Sigma$ is a critical point for the modified Hawking mass

$$\begin{aligned} \frac{\widetilde{m}_{H}(\Sigma )}{2|\Sigma |}\left( \int _{\Sigma }H\phi d\sigma -\int _{\partial \Sigma }g(\nu , X)ds\right)= & {} \frac{-|\Sigma |^{\frac{1}{2}}}{(8\pi )^{\frac{3}{2}}}\Big [\int _{\Sigma }2HL_{\Sigma }\phi d\sigma \\{} & {} -\int _{\Sigma }(H^2+\frac{2\Lambda }{3})H\phi d\sigma \\{} & {} +\int _{\partial \Sigma }(H^2+\frac{2\Lambda }{3})g(\nu , X)ds\Big ]. \end{aligned}$$

$\square$

3 Stability Result for Modified Hawking Mass

In this section, we seek to construct a foliation around $\Sigma$ by CMC embedded free boundary surfaces under the assumption of the strictly stability and locally maximizes of the modified Hawking mass, which play an important role in a proof of our main theorem.

Proposition 4

Let $(M^3, g)$ be a Riemannian manifold with boundary $\partial M$ which satisfies $R\ge 2$ and $H^{\partial M}\ge 0$. If $\Sigma$ is an properly embedded, two-sided, free boundary strictly stable minimal two-disk which locally maximizes the modified Hawking mass, then

$$\begin{aligned} |\Sigma |= \frac{2\pi }{\lambda _1(L_{\Sigma })+1}, \end{aligned}$$

where $\lambda _1(L_{\Sigma })$ is the first eigenvalue of the stability operator. Moreover, along $\Sigma$, we have $A = 0$, $R = 2$, $Ric(N,N) = -\lambda _1(L_{\Sigma })$, and its Gaussian curvature $K_{\Sigma } = \frac{2\pi }{|\Sigma |}$, and along $\partial \Sigma$, geodesic curvature is zero and $H^{\partial M}=0$. In particular, $\Sigma$ is a hemisphere.

Proof

Since $\Sigma$ is strictly stable, we obtain that $\lambda _{1}(L_{\Sigma }) \int _{\Sigma }\phi ^2 d\sigma \le \mathcal {Q}(\phi , \phi )$ for any smooth function $\phi$ on $\Sigma$. Choosing $\phi =1$, we obtain

$$\begin{aligned} \lambda _{1}(L_{\Sigma })|\Sigma |\le -\int _{\Sigma }\left( Ric(N, N)+|A|^2\right) d\sigma -\int _{\partial \Sigma }\Pi (N, N)ds. \end{aligned}$$

(3.1)

Note that the Gauss equation implies

$$\begin{aligned} Ric(N, N)+|A|^2=\frac{R}{2}-K_{\Sigma }+\frac{|A|^2}{2}\ge 1-K_{\Sigma }, \end{aligned}$$

where in the last inequality we use that $R\ge 2$.

Then, using the Gauss–Bonnet Theorem and the fact that $H^{\partial M} =k_{g}+\Pi (N, N)$ along $\partial \Sigma$, we deduce

$$\begin{aligned} (1+\lambda _{1}(L_{\Sigma }))|\Sigma |\le & {} \int _{\Sigma }K_{\Sigma }d\sigma -\int _{\partial \Sigma }\Pi (N, N)ds\\= & {} 2\pi -\int _{\partial \Sigma }(k_g+\Pi (N,N))ds\\= & {} 2\pi -\int _{\partial \Sigma }H^{\partial M}ds, \end{aligned}$$

where $k_g$ denotes the geodesic curvature of $\partial \Sigma$ in $\Sigma$. Thus, from $H^{\partial M} \ge 0$ we also have

$$\begin{aligned} |\Sigma |\le \frac{2\pi }{\lambda _{1}(L_{\Sigma })+1}. \end{aligned}$$

(3.2)

Moreover, if equality holds, all inequalities above must be equalities. Then, $\mathcal {Q}(1,1)=\lambda _{1}(L_{\Sigma })|\Sigma |$, $\Sigma$ is totally geodesic, $R=2$ and $H^{\partial M}=0$ along $\Sigma$. Next, it follows from $\mathcal {Q}(\phi , \phi )\ge \lambda _{1}(L_{\Sigma })\int _{\Sigma }\phi ^2d\sigma$ for any $\phi \in C^{\infty }(\Sigma )$, that

$$\begin{aligned} \mathcal {F}(\phi , h):=\mathcal {Q}(\phi , h)-\lambda _{1}(L_{\Sigma })\int _{\Sigma }\phi h d\sigma , \end{aligned}$$

satisfies $\mathcal {F}(\phi , \phi )\ge 0$ and $\mathcal {F}(1, 1)= 0$ for every $\phi \in C^{\infty }(\Sigma )$. At the same time, one easily verifies that $\mathcal {F}(1,\phi )=0$ for any $\phi \in C^{\infty }(\Sigma )$, and hence we may use this data to deduce

$$\begin{aligned} 0=\mathcal {F}(1, \phi ) =-\int _{\Sigma }(Ric(N, N)+\lambda _{1}(L_{\Sigma }))\phi -\int _{\partial \Sigma }\Pi (N, N)\phi . \end{aligned}$$

(3.3)

This allow us to conclude that, along $\Sigma$, $Ric(N, N) = -\lambda _1(L_{\Sigma })$, its Gaussian curvature $K_{\Sigma } = \frac{2\pi }{|\Sigma |}$ and $\Pi (N, N)=k_g=0$ along $\partial \Sigma$.

Finally, we derive the reverse of inequality (3.2); using the fact that $\Sigma$ locally maximizes the modified Hawking mass in the Propositon 3, we achieve

$$\begin{aligned} 0\ge & {} -\frac{2|\Sigma |^{\frac{1}{2}}}{(8\pi )^{\frac{3}{2}}}\int _{\Sigma }(L_{\Sigma }\phi )^2d\sigma -\left( \frac{\widetilde{m}_{H}(\Sigma )}{2|\Sigma |}-\frac{4|\Sigma |^{\frac{1}{2}}}{3(8\pi )^{\frac{3}{2}}}\right) \int _{\Sigma }\phi L_{\Sigma }\phi d\sigma \\- & {} \left( \frac{4|\Sigma |^{\frac{1}{2}}}{3(8\pi )^{\frac{3}{2}}}-\frac{\widetilde{m}_{H}(\Sigma )}{2|\Sigma |}\right) \int _{\partial \Sigma }\left( \frac{\partial \phi }{\partial \nu }-g(N, \nabla _{N}X)\phi \right) \phi ds. \\ \end{aligned}$$

Furthermore, substituting into the last inequality an eigenfunction of the problem (2.6) satisfying $\int _{\Sigma }\phi ^2d\sigma =1$, we have

$$\begin{aligned} -\frac{2|\Sigma |^{\frac{1}{2}}}{(8\pi )^{\frac{3}{2}}}\lambda _1(L_{\Sigma })^2+\frac{\widetilde{m}_{H}(\Sigma )}{2|\Sigma |}\lambda _1(L_{\Sigma })-\frac{4|\Sigma |^{\frac{1}{2}}}{3(8\pi )^{\frac{3}{2}}}\lambda _1(L_{\Sigma })\le 0, \end{aligned}$$

using the definition of modified Hawking mass, we conclude that

$$\begin{aligned} ( 8\pi -4|\Sigma |)\lambda _{1}(L_{\Sigma })\le 4|\Sigma |\lambda _{1}(L_{\Sigma })^2, \end{aligned}$$

and, since $\lambda _{1}(L_{\Sigma })>0$, we infer

$$\begin{aligned} |\Sigma |\ge \frac{2\pi }{\lambda _{1}(L_{\Sigma })+1} \end{aligned}$$

which finishes the proof of the proposition. $\square$

The next result is a crucial step in the proof of the main result. Its proof follows the same arguments used in [Proposition 5.1, [12]] and [Proposition 10, [2]]. One can construct a one-parameter family $\Sigma (t)$ in the neighborhood of $\Sigma$ such that $\Sigma (t)$ is a free boundary surface and has constant mean curvature.

Proposition 5

Let $(M^3, g)$ be a Riemannian manifold with boundary $\partial M$ which satisfies $R\ge 2$ and $H^{\partial M}\ge 0$. If $\Sigma$ is a properly embedded, two-sided, free boundary strictly stable minimal two-disk satisfying

$$\begin{aligned} |\Sigma |= \frac{2\pi }{\lambda _1(L_{\Sigma })+1}, \end{aligned}$$

then there exists $\varepsilon > 0$ and a smooth function $\mu : (-\varepsilon ,\varepsilon )\times \Sigma \rightarrow \mathbb {R}$ such that

$$\begin{aligned} \Sigma _t = \{\exp _x(\mu (t, x)N(x)); x \in \Sigma \} \end{aligned}$$

is a family of free boundary surfaces with constant mean curvature and N is the unit normal vector field along $\Sigma$. Moreover, the following properties hold:

$$\begin{aligned} \mu (0, x) = 0,\ \ \ \frac{\partial \mu }{\partial t} (0, x) = 1\ \ \ \text {and}\ \ \ \int _{\Sigma }(\mu (t,x)-t)d\sigma = 0 \end{aligned}$$

for each $x \in \Sigma$ and for each $t\in (-\varepsilon , \varepsilon )$. In particular, for some smaller $\varepsilon$, $\{\Sigma _t\}_{t\in (-\varepsilon ,\varepsilon )}$ is a foliation of a neighborhood of $\Sigma _0=\Sigma$ in M.

Proof

Since this proposition is crucial for the establishment of Theorem 4, we include its proof here for the sake of completeness.

Considering the notation used in the Sect. 2, let N be the unit normal vector field of $\Sigma$, and let X denote the unit normal vector field of $\partial M$ that coincides with the exterior conormal $\nu$ of $\partial \Sigma$.

For a function $u\in C^{2,\alpha }(\Sigma )$, $0<\alpha <1$, we define $\Sigma _{u}=\{\exp _{x}(u(x)N(x)); x\in \Sigma \}$. Note that $\Sigma _u$ is a properly embedded surface when $||u||_{2,\alpha }$ is sufficiently small. Next, consider the Banach spaces $Z=\{u \in C^{2,\alpha }(\Sigma ); \int _{\Sigma } ud\sigma = 0\}$ and $Y = \{u\in C^{0,\alpha }(\Sigma ); \int _{\Sigma } ud\sigma = 0\}$. Given sufficiently small constants $\varepsilon >0$ and $\delta >0$, we define the map $\Upsilon :(-\varepsilon ,\varepsilon )\times (B(0,\delta )\cap Z)\rightarrow Y\times C^{1,\alpha }(\partial \Sigma )$ by

$$\begin{aligned} \Upsilon (t,u)=\left( H_{\Sigma _{t+u}}-\frac{1}{|\Sigma |}\int _{\Sigma } H_{\Sigma _{u+t}}d\sigma , g(N_{\Sigma _{u+t}},X_{\Sigma _{u+t}})\right) , \end{aligned}$$

where $B(0,\delta ) = \{u\in C^{2,\alpha }(\Sigma ); ||u||_{2,\alpha } < \delta \}$, $H_{\Sigma _{t+u}}$ denotes the mean curvature of $\Sigma _{u+t}$, $N_{\Sigma _{u+t}}$ denote the unit normal vector field of $\Sigma _{u+t}$, and $X_{\Sigma _{u+t}}$ denotes the restriction of X to $\partial \Sigma _{u+t}$.

Note that $\Upsilon (0,0)=(0,0)$, because $\Sigma =\Sigma _0$. By Proposition 4, the Jacobi operator of $\Sigma$ is given by $L_{\Sigma }=\Delta _{\Sigma }-\lambda _1(L_{\Sigma }).$

Consequently, for each $\upsilon \in Z$, we use (2.1) to obtain

$$\begin{aligned} D\Upsilon (0, 0)\cdot \upsilon= & {} \frac{d}{ds}\Big |_{s=0}\Upsilon (0, s\upsilon )\\= & {} \frac{d}{ds}\Big |_{s=0}\left( H_{\Sigma _{s\upsilon }}-\frac{1}{|\Sigma |}\int _{\Sigma }H_{\Sigma _{s\upsilon }}d\sigma , g(N_{\Sigma _{s\upsilon }},X_{\Sigma _{s\upsilon }})\right) \\= & {} \left( L_{\Sigma }\upsilon -\frac{1}{|\Sigma |}\int _{\Sigma }\Delta _{\Sigma }\upsilon d\sigma + \frac{\lambda _1(L_{\Sigma })}{|\Sigma |}\int _{\Sigma }\upsilon d\sigma ,-\frac{\partial \upsilon }{\partial \nu }\right) \\= & {} \left( \Delta _{\Sigma }\upsilon -\lambda _1(L_{\Sigma })\upsilon -\frac{1}{|\Sigma |}\int _{\partial \Sigma }\frac{\partial \upsilon }{\partial \nu }ds,-\frac{\partial \upsilon }{\partial \nu }\right) . \end{aligned}$$

Now, we claim that $D\Upsilon (0,0)$ is an isomorphism when restricted to $0\times Z$. To prove this, it suffices to show that there exists a unique function $\varphi \in Z$ solving the following Neumann boundary problem

$$\begin{aligned} \left\{ \begin{array}{ll}\Delta _{\Sigma }\varphi -\lambda _1(L_{\Sigma })\varphi =f+\frac{1}{|\Sigma |}\int _{\partial \Sigma }zds\ \ \textrm{in}\ \ \Sigma ,\\ \frac{\partial \varphi }{\partial \nu }=z\ \ \textrm{on}\ \ \partial \Sigma .\end{array}\right. \end{aligned}$$

(3.4)

for given $f\in Y$ and $z\in C^{1,\alpha }(\partial \Sigma )$. So, it suffices to apply [10], Theorem 3.2] or [16], Theorem 3.1] to solve the Neumann boundary problem (3.4).

Next, we may invoke the Implicit Function Theorem to conclude for some smaller $\varepsilon >0$, there exists $(t,u(t))\in (-\varepsilon ,\varepsilon )\times B(0,\delta )$ such that $u(0) = 0$ and $\Upsilon (u(t), t ) = (0, 0)$ for any $t \in (-\varepsilon ,\varepsilon )$. More precisely, the surfaces

$$\begin{aligned} \Sigma _{t+u(t)}=\{\exp _x((t+u(t)(x))N(x)); x\in \Sigma \} \end{aligned}$$

are free boundary constant mean curvature surfaces.

Proceeding, it is easy to see that smooth function $\mu :(-\varepsilon ,\varepsilon )\times \Sigma \rightarrow \mathbb {R}$ defined by $\mu (t,x)=t+u(t)(x)$ satisfies $\mu (0,x)=0$ for each $x\in \Sigma$ and $\int _{\Sigma }(\mu (t,x)-t)d\sigma =0$ for each $t\in (-\varepsilon ,\varepsilon )$. From here it follows that

$$\begin{aligned} \int _{\Sigma }\frac{\partial \mu }{\partial t}(0,\cdot )d\sigma =|\Sigma |. \end{aligned}$$

(3.5)

Moreover, since that $\Sigma _{\mu (t,x)}$ is a CMC free boundary surfaces, for every $t\in (-\varepsilon ,\varepsilon )$, we have that

$$\begin{aligned} H_{\mu (t,\cdot )}=\frac{1}{|\Sigma |}\int _{\Sigma }H_{\mu (t,\cdot )}d\sigma . \end{aligned}$$

Thus, after differentiating at $t = 0$ and invoking (2.2), we deduce

$$\begin{aligned} L_{\Sigma }\left( \frac{\partial \mu }{\partial t}(0,\cdot )\right)= & {} \frac{1}{|\Sigma |}\int _{\Sigma }(\Delta _{\Sigma }-\lambda _1(L_{\Sigma }))\frac{\partial \mu }{\partial t}(0,\cdot )d\sigma \\= & {} \frac{1}{|\Sigma |}\int _{\Sigma }\frac{\partial }{\partial \nu }\left( \frac{\partial \mu }{\partial t}(0,\cdot )\right) d\sigma -\frac{\lambda _1(L_{\Sigma })}{|\Sigma |}\int _{\Sigma }\frac{\partial \mu }{\partial t}(0,\cdot )d\sigma \end{aligned}$$

and

$$\begin{aligned} \frac{\partial }{\partial \nu }\left( \frac{\partial \mu }{\partial t}(0,\cdot )\right) =0\ \ \ \textrm{on}\ \ \ \partial \Sigma , \end{aligned}$$

since from by Proposition 3.2 we get $\Pi (N,N)=0$ on $\Sigma$. From this and by (3.5) it follows that

$$\begin{aligned} \left\{ \begin{array}{ll}L_{\Sigma }\left( \frac{\partial \mu }{\partial t}(0,\cdot )\right) =L_{\Sigma }(1)\ \ \textrm{on}\ \ \Sigma _t,\\ \frac{\partial }{\partial \nu }\left( \frac{\partial \mu }{\partial t}(0,\cdot )\right) =0\ \ \textrm{along}\ \ \partial \Sigma _t.\end{array}\right. \end{aligned}$$

From here it follows that $\frac{\partial \mu }{\partial t} (0, x) = 1$ for each $x\in \Sigma$ which concludes the proof of the proposition. $\square$

In order to set the stage for the proofs to follow, we use Proposition 5 to define a mapping $f_t: \Sigma \rightarrow M$ by $f_t (x) = \exp _x(\mu (x, t)N(x))$ for each $t \in (-\varepsilon , \varepsilon )$. Let $N_t (x)$ be the unit vector field normal along $\Sigma _t$ such that $N_0(x) = N(x)$ for all $x\in \Sigma$ and let us denote $d\sigma _t$ and $ds_t$ the element of area of $\Sigma _t$ and $\partial \Sigma$, respectively, with respect to the induced metric by $f_t$.

Moreover, let H(t) denote the mean curvature of $\Sigma _t$ with respct to $\nu _{t}$, as well as the lapse function $\rho _{t}:\Sigma _{t}\rightarrow \mathbb {R}$ which is defined by

$$\begin{aligned} \rho _{t}(x)=\langle N_{t}(x), \frac{\partial }{\partial t}f_{t}(x)\rangle , \end{aligned}$$

for each $t \in (-\varepsilon , \varepsilon )$. Consequently, since $\{\Sigma _t\}$ is a foliation of $\Sigma$ by CMC free boundary surfaces, it follows from (2.1) and (2.2) that

$$\begin{aligned} \left\{ \begin{array}{ll}H'(t)=L_{\Sigma _t}\rho _t\ \ \textrm{on}\ \ \Sigma _t,\\ \frac{\partial \rho _t}{\partial \nu _t}=g(N_t,\nabla _{ N_t}X)\rho _t\ \ \textrm{along}\ \ \partial \Sigma _t,\end{array}\right. \end{aligned}$$

(3.6)

where $L_{\Sigma _t}$ is the Jacobi operator with associated surface $\Sigma _t$.

From Proposition 5, it is easy to check that $\rho _{0}\equiv 1$. Then we use Proposition 4 and (3.6) to conclude that

$$\begin{aligned} \left. \frac{d}{dt}\right| _{t=0}H(t)=L_{\Sigma }(1)=-\lambda _{1}(L_{\Sigma })<0. \end{aligned}$$

(3.7)

This implies that we can choose $\varepsilon$ sufficiently small such that $H(t)>0$ for $t\in (-\varepsilon ,0)$ and $H(t)<0$ for $t\in (0,\varepsilon )$, hence H is a decreasing function on t. In order to prove the main Theorem, we need to provide the monotonicity of the modified Hawking mass along this foliation. This is the content of the following lemma.

Lemma 1

Let $\{\Sigma _{t}\}_t$ be a family of surfaces obtained in the Proposition 5. Then

$$\begin{aligned} \int _{ \Sigma _{t}}(Ric (N_{t}, N_{t})+ & {} |A_{t}|^2)\rho _{t}d\sigma _{t}=\overline{\rho _{t}}\int _{\Sigma _{t}}(Ric(N_{t}, N_{t})+|A_{t}|^2)d\sigma _{t}+H'(t)\theta (t, x)\\+ & {} \overline{\rho _{t}}\int _{\Sigma _{t}}\frac{|\nabla \rho _{t}|^2}{\rho _{t}^2}d\sigma _{t} +\overline{\rho _{t}}\int _{\partial \Sigma _{t}}g(N_{t}, \nabla _{N_{t}}X)ds_{t} -\int _{\Sigma _{t}}(\Delta _{\Sigma _{t}}\rho _{t})d\sigma _{t}, \end{aligned}$$

where $\overline{\rho _{t}}=\frac{1}{|\Sigma _{t}|}\int _{\Sigma _{t}}\rho _{t}d\sigma _{t}$ and $\theta (x, t)$ is a non-positive function.

Proof

Since $\rho _{0}\equiv 1$, by continuity we can assume $\rho _t>0$ in a neighborhood. Thus, multiplying equation (3.6) by $1/\rho _t$ and integrating over $\Sigma _t$ we conclude that

$$\begin{aligned} H'(t)\int _{\Sigma _{t}}\frac{1}{\rho _{t}}d\sigma _{t}= & {} \int _{\Sigma _{t}}\frac{\Delta _{\Sigma _{t}}\rho _{t}}{\rho _{t}}d\sigma _{t}+\int _{\Sigma _{t}}(Ric(N_{t}, N_{t})+|A_{t}|^2)d\sigma _{t}\\= & {} \int _{\partial \Sigma _{t}}\frac{1}{\rho _{t}}\frac{\partial \rho _{t}}{\partial \nu _{t}}ds_{t}+\int _{\Sigma _{t}}\frac{|\nabla \rho _{t}|^2}{\rho _{t}^2}d\sigma _{t}+\int _{\Sigma _{t}}(Ric(N_{t}, N_{t})+|A_{t}|^2)d\sigma _{t}. \end{aligned}$$

Multiplying the last equation by $\overline{\rho _{t}}=\frac{1}{|\Sigma _{t}|}\int _{\Sigma _{t}}\rho _{t}d\sigma _{t}$ and subtracting it from the integral of Eq. (3.6), we find:

$$\begin{aligned} H'(t)\left( \overline{\rho _{t}}\int _{\Sigma _{t}}\frac{1}{\rho _{t}}d\sigma _{t}-|\Sigma _{t}|\right)= & {} \overline{\rho _{t}}\int _{\partial \Sigma _{t}}\frac{1}{\rho _{t}}\frac{\partial \rho _{t}}{\partial \eta _{t}}ds_{t}+\overline{\rho _{t}}\int _{\Sigma _{t}}\frac{|\nabla \rho _{t}|^2}{\rho _{t}^2}d\sigma _{t}\\{} & {} +\overline{\rho _{t}}\int _{\Sigma _{t}}(Ric(N_{t}, N_{t})+|A_{t}|^2)d\sigma _{t} -\int _{\Sigma _{t}}\Delta _{\Sigma _{t}}\rho _{t}d\sigma _{t}\\{} & {} -\int _{ \Sigma _{t}}(Ric (N_{t}, N_{t})+|A_{t}|^2)\rho _{t}d\sigma _{t}. \end{aligned}$$

Hence, it follows from $\frac{\partial \rho _{t}}{\partial \nu _{t}}=g(N_{t}, \nabla _{N_{t}}X)\rho _{t}$ that:

$$\begin{aligned} \int _{ \Sigma _{t}}(Ric (N_{t}, N_{t})+|A_{t}|^2)\rho _{t}d\sigma _{t}{} & {} =\overline{\rho _{t}}\int _{\Sigma _{t}}(Ric(N_{t}, N_{t})+|A_{t}|^2)d\sigma _{t}+H'(t)\theta (t, x) \\{} & {} \quad +\overline{\rho _{t}}\int _{\Sigma _{t}}\frac{|\nabla \rho _{t}|^2}{\rho _{t}^2}d\sigma _{t} +\overline{\rho _{t}}\int _{\partial \Sigma _{t}}g(N_{t}, \nabla _{N_{t}}X)ds_{t}\\{} & {} \quad -\int _{\Sigma _{t}}(\Delta _{\Sigma _{t}}\rho _{t})d\sigma _{t}, \end{aligned}$$

where $\theta (t, x)=|\Sigma _{t}|-\overline{\rho _{t}}\int _{\Sigma _{t}}\frac{1}{\rho _{t}}d\sigma _{t}$ is a non-positive function. This concludes the proof of the lemma. $\square$

4 Proof of the Results

4.1 Proof of Theorem 3

Proof

To begin, let $(M=\mathbb {R}\times \mathbb {S}^2_{+},g_m)$ be the half de Sitter–Schwarzschild with mass parameter $m>0$. Now, consider the double $(\widetilde{M},\widetilde{g}_m)$ of $(M,g_m)$ along $\partial M$. More precisely, $\widetilde{M} = M\times \{0, 1\}/\sim$, where $(x, 0)\sim (x, 1)$ for all $x\in \partial M$, and $\widetilde{g}_m(x, j) = g_m(x)$ for all $x\in M$ and $j = 0, 1$. It follows that $\partial M=\mathbb {R}\times \mathbb {S}^1$ is totally geodesic that $(\widetilde{M},\widetilde{g}_m)$ is a $C^{\infty }$ Riemannian manifold and $(\widetilde{M}=\mathbb {R}\times \mathbb {S}^2,\widetilde{g}_m)$ is one de Sitter–Schwarzschild.

Next, suppose $\Sigma$ is a normal graph over slice $\Sigma _r=\{r\}\times \mathbb {S}_+^2$ given by $\phi \in C^2(\Sigma _r)$ with $||\phi ||_{C^2(\Sigma _r)} < \varepsilon$, which meets $\partial M$ orthogonally. It easy to check that the modified Hawking mass satisfies $\widetilde{m}_{H}(\Sigma _r)=m$ for any slice of M and $\widetilde{\Sigma }\subset \widetilde{M}$ the double of $\Sigma$, is a closed surface given as a graph over the slice $\widetilde{\Sigma }_r=\{r\}\times \mathbb {S}^2$ that satisfies $|\widetilde{\Sigma }|_{\widetilde{g}_m} = 2|\Sigma |_{g_m}$ and $|\widetilde{\Sigma }_r|_{\widetilde{g}_m} = 2|\Sigma _r|_{g_m}$. This implies that $m_{H}(\widetilde{\Sigma })=2\widetilde{m}_{H}(\Sigma )$ and $m_{H}(\widetilde{\Sigma }_r)=2\widetilde{m}_{H}(\Sigma _r)$ where $m_H(\cdot )$ and $\widetilde{m}_H(\cdot )$ stand, respectively, for the Hawking mass and the modified Hawking mass. So, it suffices to apply Theorem 1.2 in [12] to conclude that one has either $m_H(\widetilde{\Sigma }) < m_H(\widetilde{\Sigma }_r)$ or $\widetilde{\Sigma }$ is a slice $\widetilde{\Sigma }_s$ for some s. It follows that

• (i) either $\widetilde{m}_H(\Sigma ) < \widetilde{m}_H(\Sigma _r)$;

• (ii) or $\Sigma$ is a slice $\Sigma _s$ for some s,

which finishes the proof of the theorem. $\square$

4.2 Proof of Theorem 4

Proof

Let $(M^3, g)$ be a Riemannian manifold and $\Sigma \subset M$ be a free boundary surface under the hypotheses of Theorem 4. As a consequence of Propositions 4 and 5, the Jacobi opertator of $\Sigma$ is given by $L_{\Sigma }=\Delta -\lambda _{1}(\Sigma )$, and there is a family of free boundary surfaces with constant mean curvature $\{\Sigma _t\}_{|t|<\varepsilon }$ around $\Sigma =\Sigma _{0}$. Thus, we may invoke Proposition 1 and the first variation of the modified Hawking mass, to infer

$$\begin{aligned} \frac{d}{dt}\widetilde{m}_{H}(\Sigma _{t})= & {} -\frac{1}{2}\frac{|\Sigma _{t}|^{-\frac{1}{2}}}{(8\pi )^{\frac{1}{2}}}\left( 1-\frac{1}{8\pi }\int _{ \Sigma _{t}}(H_{t}^2+\frac{4}{3})d\sigma _{t}\right) \left( \int _{ \Sigma _{t}}H_{t}\rho _{t}d\sigma _{t}-\int _{\partial \Sigma _{t}}g(\nu _{t}, X)ds_{t}\right) \\{} & {} +\frac{|\Sigma _{t}|^{\frac{1}{2}}}{(8\pi )^\frac{1}{2}}\left( -\frac{1}{8\pi }\int _{ \Sigma _{t}}\left[ 2H_{t}(\Delta _{\Sigma _{t}}\rho _{t}+Ric(N_{t}, N_{t})\rho _{t}+|A_{t}|^2\rho _{t})\right] d\sigma _{t}\right. \\{} & {} +\left. \frac{1}{8\pi }\int _{ \Sigma _{t}}\Big (H_{t}^2+\frac{4}{3}\Big )H_{t}\rho _{t}d\sigma _{t}-\frac{1}{8\pi }\int _{\partial \Sigma _{t}}\Big (H_{t}^2+\frac{4}{3}\Big )g(\nu _{t}, X)d s_{t} \right) \\= & {} \frac{|\Sigma _{t}|^{\frac{1}{2}}}{(8\pi )^{\frac{3}{2}}}H_t\Big [-\frac{4\pi }{|\Sigma _t|}\int _{\Sigma _{t}}\rho _{t}d\sigma _{t}+\left( \frac{3H_{t}^2}{2}+2\right) \int _{\Sigma _{t}}\rho _{t}d\sigma _{t} -2\int _{ \Sigma _{t}}\Delta _{\Sigma _{t}}\rho _{t}d\sigma _{t}\\{} & {} -2\int _{ \Sigma _{t}}(Ric(N_{t}, N_{t})+|A_{t}|^2)\rho _{t}d\sigma _{t}\Big ], \end{aligned}$$

where, in the last equality, we use that $\{\Sigma _t\}_t$ is a family of CMC free boundary surfaces.

In conjunction with Lemma 1 and (2.8), one sees that

$$\begin{aligned} \frac{d}{dt}\widetilde{m}_{H}(\Sigma _{t})= & {} \frac{-|\Sigma _{t}|^{\frac{1}{2}}}{(8\pi )^{\frac{3}{2}}}H_t\Big [4\pi \overline{\rho _{t}}-\left( \frac{3H_{t}^2}{2}+2\right) |\Sigma _t|\overline{\rho _{t}}\\{} & {} +2\overline{\rho _{t}}\int _{ \Sigma _{t}}(Ric(N_{t}, N_{t})+|A_{t}|^2)d\sigma _{t}\\{} & {} +2H'(t)\theta (t,x)+\overline{\rho _{t}}\int _{\Sigma _{t}}\frac{|\nabla \rho _{t}|^2}{\rho _{t}^2}d\sigma _{t}+2\overline{\rho _{t}}\int _{\partial \Sigma _{t}}g(N_{t}, \nabla _{N_{t}}X)ds_t\Big ]\\= & {} \frac{-|\Sigma _{t}|^{\frac{1}{2}}}{(8\pi )^{\frac{3}{2}}}H_t\Big [4\pi \overline{\rho _{t}}-2\overline{\rho _{t}}\int _{\Sigma _t}K_{\Sigma _t}d\sigma _{t} +\overline{\rho _{t}}\int _{ \Sigma _{t}}(R-2)d\sigma _{t}+2H'(t)\theta (t,x)\\{} & {} +\overline{\rho _{t}}\int _{ \Sigma _{t}}\left( |A_t|^2-\frac{H_t^2}{2}\right) d\sigma _{t}\\{} & {} +\overline{\rho _{t}}\int _{\Sigma _{t}}\frac{|\nabla \rho _{t}|^2}{\rho _{t}^2}d\sigma _{t}\\{} & {} +2\overline{\rho _{t}}\int _{\partial \Sigma _{t}}g(N_{t}, \nabla _{N_{t}}X)ds_t\Big ], \end{aligned}$$

and hence, by utilizing the identity $H^{\partial M} = k_g + \Pi (N_t,N_t)$ and the Gauss-Bonnet Theorem, we obtain

$$\begin{aligned} \frac{d}{dt}\widetilde{m}_{H}(\Sigma _{t})= & {} -\frac{|\Sigma _{t}|^{\frac{1}{2}}}{(8\pi )^{\frac{3}{2}}}H_t\Big [2\overline{\rho _t}\int _{\partial \Sigma _t}H^{\partial M}ds_t+\overline{\rho _{t}}\int _{ \Sigma _{t}}(R-2)d\sigma _{t}+2H'(t)\theta (t,x)\nonumber \\{} & {} +\overline{\rho _{t}}\int _{ \Sigma _{t}}\left( |A_t|^2-\frac{H_t^2}{2}\right) d\sigma _{t}+\overline{\rho _{t}}\int _{\Sigma _{t}}\frac{|\nabla \rho _{t}|^2}{\rho _{t}^2}d\sigma _{t}\Big ]. \end{aligned}$$

(4.1)

Since $\rho _{0}(x)=1$ for all $\in \Sigma$, we can choose $\varepsilon >0$ sufficiently small such that $\rho _{t}(x)>0$ for each $x\in \Sigma _t$ and $t\in (-\varepsilon ,\varepsilon )$. Thus, it suffices to use $H^{\partial M}\ge 0$ jointly with $R\ge 2$ to conclude that $\frac{d}{dt}\widetilde{m}_{H}(\Sigma _{t})\ge 0$ for $t \in \left[ 0, \varepsilon \right)$ and $\frac{d}{dt}\widetilde{m}_{H}(\Sigma _{t})\le 0$ for $t \in \left( -\varepsilon , 0\right]$. This implies that

$$\begin{aligned} \widetilde{m}_{H}(\Sigma _{t})\ge \widetilde{m}_{H}(\Sigma ), \end{aligned}$$

for all $t \in (-\varepsilon , \varepsilon )$. Consequently, taking into account that $\Sigma$ locally maximizes the modified Hawking mass, we concluded that $\frac{d}{dt}\widetilde{m}_{H}(\Sigma _{t})\equiv 0$. This immediately guarantees that $\Sigma _{t}$ is umbilic, $R=2$ along $\Sigma _t$, and $H^{\partial M}=0$ on $\partial \Sigma _{t}$. Then, we use (4.1) to infer

$$\begin{aligned} \frac{\overline{\rho _{t}}}{2}\int _{\Sigma _{t}}\frac{|\nabla \rho _{t}|^2}{\rho _{t}^2}d\sigma _{t}+H'(t)\theta (t, x)=0 \end{aligned}$$

for each $t \in (-\varepsilon , \varepsilon )$. On the other hand, from (3.7) and Lemma 1 we have $H'(t)\theta (t, x)\ge 0$, this enables to arrive at $\rho _t=1$ $\forall t\in (-\varepsilon , \varepsilon )$. Therefore, up to isometry, we can choose a small neighbourhood of $\Sigma$ such that the metric is given by $g=dt^2+g_{\Sigma _t}$, where $g_{\Sigma _t}$ is the induced metric by the isometry $f(x, t)=\exp _{x}(tN(x))$.

Now, it suffices to apply Theorem 3.2 in [8] to conclude that

$$\begin{aligned} \frac{\partial }{\partial t}g_{\Sigma _{t}}= & {} -2\rho _{t}A_{t}\\= & {} -H_{t}g_{\Sigma _{t}}, \end{aligned}$$

where we have used $\rho _{t}\equiv 1$, $H_{t}$ is constant and $\Sigma _{t}$ is umbilic for all $t \in (-\varepsilon , \varepsilon )$.

Consequently, $g_{\Sigma _{t}}=u_{a}(t)^2g_{\mathbb {S}_{+}^2}$ for all $t \in (-\varepsilon , \varepsilon )$, where $u_{a}(t)=ae^{-\frac{1}{2}\int _{0}^{t}H(s)ds}$ and $a^2=\frac{|\Sigma |}{2\pi }$.

Therefore, the induced metric by isometry $f(x, t)=\exp _{x}(tN(x))$ implies that $g=dt^2+u_{a}(t)^2g_{\mathbb {S}_{+}^2}$ on $\Sigma \times \left( -\varepsilon , \varepsilon \right)$. Moreover, it is easy to see that the function $u_{a}(t)$ is a solution of (1.5). By uniqueness of the solution to the ODE, we conclude that g is exactly the half de Sitter–Schwarzschild metric on $\Sigma \times \left( -\varepsilon , \varepsilon \right)$. Thus, the proof is completed. $\square$