Rigidity of Area-Minimizing Hyperbolic Surfaces in Three-Manifolds

Abstract

We prove that if M is a three-manifold with scalar curvature greater than or equal to −2 and Σ⊂M is a two-sided compact embedded Riemann surface of genus greater than 1 which is locally area-minimizing, then the area of Σ is greater than or equal to 4π(g(Σ)−1), where g(Σ) denotes the genus of Σ. In the equality case, we prove that the induced metric on Σ has constant Gauss curvature equal to −1 and locally M splits along Σ. We also obtain a rigidity result for cylinders (I×Σ,dt ²+g _Σ), where I=[a,b]⊂ℝ and g _Σ is a Riemannian metric on Σ with constant Gauss curvature equal to −1.

1 Introduction

It is an interesting fact in differential geometry that if M is a three-manifold with lower bounded scalar curvature, then the existence of an area-minimizing surface can influence the geometry of M.

For instance, it was shown by R. Schoen and S.T. Yau [24] that if M is a compact orientable three-manifold with nonnegative scalar curvature and Σ⊂M is an incompressible two-torus (i.e., the fundamental group of Σ injects into that of M), then M is flat. To prove that result, they first show that any such manifold contains a stable minimal two-torus. Next, they observe, using the second variation formula of area, that if M has positive scalar curvature, then every compact stable minimal surface in M is a two-sphere. The result follows because if M admits a non-flat metric of nonnegative scalar curvature, then M also admits a metric of positive scalar curvature (see [16]).

In [11], D. Fischer-Colbrie and R. Schoen conjectured that in Schoen and Yau’s result above it is sufficient that M contains an area-minimizing two-torus (not necessarily incompressible). This conjecture was proved in [7], by M. Cai and G. Galloway. They proved that if M has nonnegative scalar curvature and Σ⊂M is a two-sided embedded two-torus which is area-minimizing in its isotopy class, then M is flat. This result is obtained as a corollary of the following local statement.

Theorem 1

(M. Cai, G. Galloway)

Let (M ³,g) be a Riemannian three-manifold with nonnegative scalar curvature. If Σ is a two-sided embedded two-torus in M which is locally area-minimizing, then M is flat in a neighborhood of Σ.

It follows that the induced metric on Σ is flat and that locally M splits along Σ. The proof of Theorem 1 uses an argument based on a local deformation around Σ to obtain a metric with positive scalar curvature.

Recently, H. Bray, S. Brendle, and A. Neves studied in [3] the case where M has scalar curvature greater than or equal to 2 and Σ⊂M is a locally area-minimizing embedded two-sphere. In their case, the model is the Riemannian manifold (ℝ×S ²,dt ²+g), where g is the standard metric on S ² with constant Gauss curvature equal to 1. They proved the following result.

Theorem 2

(H. Bray, S. Brendle, A. Neves)

Let (M ³,g) be a Riemannian three-manifold with scalar curvature R _g ≥2. If Σ is an embedded two-sphere which is locally area-minimizing, then Σ has area less than or equal to 4π. Moreover, if equality holds, then Σ with the induced metric has constant Gauss curvature equal to 1 and locally M splits along Σ.

The proof in [3] is based on a construction of a one-parameter family of constant mean curvature two-spheres. A global result was also obtained using the local one above. More precisely, it was proved that if Σ is area-minimizing in its homotopy class and has area equal to 4π, then the universal cover of M is isometric to (ℝ×S ²,dt ²+g). A similar rigidity result for area-minimizing projective planes was obtained in [2].

Remark 1

There is a relation between the rigidity in Theorem 2 and the Hawking mass. If Σ⊂(M ³,g) is a surface and R _g ≥Λ,Λ∈ℝ, then the Hawking mass of Σ, denoted by m _H (Σ), is defined to be

$$m_H(\Sigma)=|\Sigma|^{1/2}\biggl(8\pi\chi(\Sigma)-\int_{\Sigma}\biggl(H^2+\frac {2}{3}\Lambda\biggr)\,d\sigma\biggr),$$

where H is the mean curvature of Σ.

Now, if Σ is a locally area-minimizing two-sphere with area equal to 4π and Λ=2, then Σ attains the maximum possible value of the Hawking mass. Suppose we have a family of two-spheres Σ _t ⊂M,Σ₀=Σ, that solves the inverse mean curvature flow. Thus, it is well known that m _H (Σ _t ) is non-decreasing. Since m _H (Σ) is the maximum of the Hawking mass, we have that m _H (Σ _t )=m _H (Σ) for all t and consequently, all the two-spheres Σ _t are minimal and have area equal to 4π.

A natural question is to know what happens when the model is the Riemannian product manifold (ℝ×Σ,dt ²+g _Σ), where Σ is a Riemann surface of genus greater than 1 and g _Σ is a Riemannian metric on Σ with constant Gauss curvature equal to −1.

In the present paper, we prove that the analogous result is true in this case. The first theorem of this paper is stated below.

Theorem 3

Let (M ³,g) be a Riemannian three-manifold with R _g ≥−2, where R _g denotes the scalar curvature of M. If Σ⊂M is a two-sided compact embedded Riemann surface of genus g(Σ)≥2 which is locally area-minimizing, then

$$|\Sigma|_g\geq4\pi(g(\Sigma)-1), $$

(1)

where |Σ| _g is the area of Σ with respect to the induced metric. Moreover, if equality holds, then Σ has a neighborhood which is isometric to ((−ϵ,ϵ)×Σ,dt ²+g _Σ), where ϵ>0 and g _Σ is the induced metric on Σ which has constant Gauss curvature equal to −1. More precisely, the isometry is given by f(t,x)=exp _x (tν(x)),(t,x)∈(−ϵ,ϵ)×Σ, where ν is the unit normal vector field to Σ.

Remark 2

Note that if |Σ|=4π(g(Σ)−1) in Theorem 3, then m _H (Σ) is the minimum (not the maximum) possible value of the Hawking mass for minimal surfaces of genus equal to g(Σ). It is interesting that rigidity still holds despite the failure of the heuristic argument of Remark 1.

We note that a related rigidity result for constant mean curvature surfaces of genus 1 was obtained in [1]. We also refer the reader to the excellent surveys [12] and [4] on rigidity problems associated with scalar curvature.

Let us give an idea of the proof of Theorem 3. The inequality (1) follows from the second variation of area using the Gauss equation, the lower bound of the scalar curvature, and the Gauss–Bonnet theorem. In the equality case, we construct, using the implicit function theorem, a one-parameter family of constant mean curvature surfaces, denoted by Σ _t , with Σ₀=Σ and all having the same genus. The next argument in the proof is the fundamental one. Arguing by contradiction and using the solution of the Yamabe problem for compact manifolds with boundary and Hopf’s maximum principle, we are able to conclude that each Σ _t has the same area. Finally, we obtain from this that Σ has a neighborhood isometric to ((−ϵ,ϵ)×Σ,dt ²+g _Σ).

If we suppose that Σ minimizes area in its homotopy class, then we obtain global rigidity using a standard continuation argument contained in [3, 7].

Corollary 1

Let (M ³,g) be a complete Riemannian three-manifold with R _g ≥−2. Moreover, suppose that Σ⊂M is a two-sided compact embedded Riemann surface of genus g(Σ)≥2 which minimizes area in its homotopy class. Then |Σ| _g satisfies inequality (1), and if equality holds, then (ℝ×Σ,dt ²+g _Σ) is an isometric covering of (M ³,g), where g _Σ is the induced metric on Σ which has constant Gauss curvature equal to −1. The covering is given by f(t,x)=exp _x (tν(x)),(t,x)∈ℝ×Σ, where ν is the unit normal vector to Σ.

In recent years, several results were obtained concerning the problem of recognizing the geometry of a compact manifold with boundary, provided the geometry of the boundary is known and some curvature conditions are satisfied. For example, in [17], P. Miao observed that the positive mass theorem (see [23, 28]) implies the following rigidity result for the unit ball B ⁿ⊂ℝⁿ.

Theorem 4

(P. Miao)

Let g be a smooth Riemannian metric on B ⁿ with nonnegative scalar curvature such that ∂B ⁿ=S ⁿ⁻¹ with the induced metric has mean curvature greater than or equal to n−1 and is isometric to S ⁿ⁻¹ with the standard metric. Then g is isometric to the standard metric of B ⁿ.

The theorem above was generalized by Y. Shi and L. Tam in [25]. In [19], M. Min-Oo proved a scalar rigidity result for the hyperbolic space. Moreover, analogues of the positive mass theorem for asymptotically hyperbolic manifolds were obtained in [8] and [27]. We note that these results imply the analogue of the Miao’s theorem for geodesic balls in the hyperbolic space.

Inspired by the above results, M. Min-Oo [20] conjectured the following scalar curvature rigidity for the hemisphere $S^{n}_{+}$:

Min-Oo’ s Conjecture

Let g be a smooth metric on the hemisphere $S^{n}_{+}$ with scalar curvature R _g ≥n(n−1) such that the induced metric on $\partial S^{n}_{+}$ agrees with the standard metric on $\partial S^{n}_{+}$ and is totally geodesic. Then g is isometric to the standard metric on $S^{n}_{+}$.

This conjecture is true for n=2, in which case it follows by a theorem of Toponogov [26] (see also [13]). Recently, counterexamples were constructed by S. Brendle, F.C. Marques and A. Neves in [6] for n≥3.

We refer the reader to [9, 13, 15] for partial results concerning Min-Oo’s conjecture. In [5], a rigidity result for small geodesic balls of S ⁿ was proved.

The next theorem is a rigidity result for cylinders ([a,b]×Σ,dt ²+g _Σ), where Σ is a Riemann surface of genus greater than 1 and constant Gauss curvature equal to −1. This is similar to Miao’s result and Min-Oo’s conjecture, the difference being that in our setting we are dealing with cylinders instead of geodesic balls.

Recall that a three-manifold is irreducible if every embedded 2-sphere in M bounds a 3-ball embedded in M.

Theorem 5

Let Σ be a compact Riemann surface of genus g(Σ)≥2 and g _Σ a metric on Σ with K _Σ≡−1. Let (Ω³,g) be a compact orientable irreducible connected Riemannian three-manifold with boundary satisfying the following properties:

• R _g ≥−2.

• H _g ≥0. (H _g is the mean curvature of ∂Ω, and the convention for the mean curvature is $\vec{H}_{g}=-H_{g}\eta$, where $\vec{H}$ is the mean curvature vector and η is the outward normal vector.)

• Some connected component of ∂Ω is incompressible in Ω and with the induced metric is isometric to (Σ,g _Σ).

Moreover, suppose that Ω does not contain any one-sided compact embedded surface. Then (Ω,g) is isometric to ([a,b]×Σ,dt ²+g _Σ).

We note that the similar result for cylinders [a,b]×S ², where S ² is the round unit sphere, does not hold. In fact, consider a rotationally symmetric metric $g=u(t)^{4}(dt^{2}+g_{S^{2}})$ on ℝ×S ² with constant scalar curvature equal to 2 such that u(0)>1 and u′(0)=0 (see [22]). Choosing a>0 such that u(a)=u(0), we have that the rescaled metric $\overline {g}=u(0)^{-4}\,g$ on [0,a]×S ² gives a counterexample.

The following example justifies the requirement that Ω does not contain any one-sided compact embedded surface. Let $(\hat{\Sigma },g_{\hat{\Sigma}})$ be a compact non-orientable surface with constant Gauss curvature equal to −1. Denote by Σ the orientable double covering of $\hat{\Sigma}$ and by π the covering map. Now, let $g_{\Sigma}=\pi^{*}g_{\hat{\Sigma}}$ and consider (M=[−k,k]×Σ,g=dt ²+g _Σ). Take the subgroup Γ={id,f}⊂Iso(M,g), where f is defined by f(t,x)=(−t,ϕ(x)) and ϕ∈Iso(Σ,g _Σ) is the non-trivial deck transformation of $\pi:\Sigma\longrightarrow\hat{\Sigma}$. Now, consider the Riemannian manifold (Ω,g _Ω), where Ω=M/Γ and g _Ω is the quotient metric. Note that Ω is orientable and irreducible, $R_{g_{\Omega}}=-2$, $H_{g_{\Omega}}=0$, ∂Ω is incompressible in Ω and with the induced metric is isometric to (Σ,g _Σ). Finally, observe that ∂Ω has only one component and that the image of {0}×Σ is a one-sided compact embedded surface in Ω.

We note that recently M. Micallef and V. Moraru provided an alternative argument to prove Theorem 3 (see [18]).

2 Proof of Inequality (1)

Let ν be the unit normal vector field to Σ. For each function ϕ∈C ^∞(Σ), we have, by the second variation formula of area and the fact that Σ is locally area minimizing, that

$$\int_{\Sigma}(\operatorname{Ric}(\nu,\nu)+|A|^2)\,\phi^2\,d\sigma\leq\int_{\Sigma }|\nabla\phi|^2\,d\sigma,$$

where A and dσ denote the second fundamental form and the area element of Σ, respectively. Choosing ϕ=1, we obtain

$$\int_{\Sigma}(\operatorname{Ric}(\nu,\nu)+|A|^2)\,d\sigma\leq0. $$

(2)

Now, the Gauss equation implies

$$\operatorname{Ric}(\nu,\nu)=\frac{1}{2}R_g-K_{\Sigma}-\frac{1}{2}|A|^2, $$

(3)

where K _Σ denotes the Gauss curvature of Σ.

Substituting (3) in (2), we get

$$\frac{1}{2}\int_{\Sigma}(R_g+|A|^2)\,d\sigma\leq\int_{\Sigma}K_{\Sigma}\,d\sigma. $$

(4)

By the Gauss–Bonnet theorem and the fact that R _g ≥−2 and |A|²≥0, we have

$$-|\Sigma|_g\leq4\pi(1-g(\Sigma)).$$

Therefore, |Σ| _g ≥4π(g(Σ)−1).

3 Equality Case

Proposition 1

If Σ attains the equality in (1), then Σ is totally geodesic. Moreover, Ric (ν,ν)=0 and R _g =−2 on Σ, and Σ has constant Gauss curvature equal to −1 with the induced metric.

Proof

If |Σ| _g =4π(g(Σ)−1), then it follows from the proof of (1) that inequalities (2) and (4) are in fact equalities. The equality in (2) together with the stability of Σ implies that the constant functions are in the kernel of the Jacobi operator L=Δ_Σ+Ric (ν,ν)+|A|² of Σ. Therefore, Ric (ν,ν)+|A|²=0 on Σ.

Now, equality in (4) implies that R _g =−2 and A=0 on Σ. Finally, by (3), we conclude that Σ has constant Gauss curvature equal to −1 with the induced metric. □

The construction in the next proposition is fundamental to conclude the rigidity in Theorem 3. The same construction was used in [1] and [3] to prove similar rigidity results. We prove it here for completeness.

Proposition 2

If Σ attains the equality in (1), then there exist ϵ>0 and a smooth family Σ _t ⊂M, t∈(−ϵ,ϵ) of compact embedded surfaces satisfying:

• Σ _t ={exp _x (w(t,x)ν(x)):x∈M}, where w:(−ϵ,ϵ)×Σ⟶ℝ is a smooth function such that

  $$w(0,x)=0,\qquad \frac{\partial w}{\partial t}(0,x)=1\quad \text{\textit{and}}\quad \int _{\Sigma}(w(t,\cdot)-t)\,d\sigma=0.$$

• Σ _t has constant mean curvature for all t∈(−ϵ,ϵ).

Proof

By the previous proposition, we have L=Δ_Σ. Fix α∈(0,1) and consider the Banach spaces X = {u ∈ C ^2,α(Σ):∫_Σ u  dσ = 0} and Y = {u ∈ C ^0,α(Σ):∫_Σ u  dσ = 0}. For each real function u defined on Σ, let Σ _u ={exp _x (u(x)ν(x)):x∈Σ}, where ν is the unit normal vector field to Σ.

Choose ϵ>0 and δ>0 such that Σ _u+t is a compact surface of class C ^2,α for all (t,u)∈(−ϵ,ϵ)×B(0,δ), where $B(0,\delta)=\{u\in X:\Arrowvert u\Arrowvert_{C^{2,\alpha}}<\delta\}$. Denote by $H_{\Sigma_{u+t}}$ the mean curvature of Σ _u+t .

Now, consider the map Ψ:(−ϵ,ϵ)×B(0,δ)⟶Y defined by

$$\Psi(t,u)=H_{\Sigma_{u+t}}-\frac{1}{|\Sigma|}\int_{\Sigma}H_{\Sigma _{u+t}}\,d\sigma.$$

Notice that Ψ(0,0)=0 because Σ₀=Σ.

The next step is to compute DΨ(0,0)⋅v, for v∈X. We have

where the last equality follows from the fact that L=Δ_Σ.

Since Δ_Σ:X⟶Y is a linear isomorphism, we have, by the implicit function theorem, that there exist 0<ϵ ₁<ϵ and u(t)=u(t,⋅)∈B(0,δ) for t∈(−ϵ ₁,ϵ ₁) such that

$$u(0)=0\quad \textrm{and}\quad \Psi(t,u(t))=0,\quad\forall t\in(-\epsilon _1,\epsilon_1).$$

Thus, defining w(t,x)=u(t,x)+t, for (t,x)∈(−ϵ ₁,ϵ ₁)×Σ, we have that all the surfaces Σ _t ={exp _x (w(t,x)ν(x)):x∈Σ} have constant mean curvature. It is easy to see that w(t,x) satisfies all the conditions stated in the proposition. □

Let ν(t) denote the unit normal vector field to Σ _t such that ν(0)=ν. In our convention, the mean curvature H(t) of Σ _t satisfies $\vec{H}(t)=-H(t)\nu(t)$, where $\vec{H}(t)$ is the mean curvature vector of Σ _t . In this case, we have

$$\frac{d}{dt}|\Sigma_t|_g=H(t)\int_{\Sigma_t}\biggl\langle\nu(t),\frac {\partial f}{\partial t}(t,\cdot)\biggr\rangle\,d\sigma_t, $$

(5)

where f(t,x)=exp _x (w(t,x) ν(x)), x∈Σ. Notice that $\frac{\partial f}{\partial t}(0,x)=\nu(x)$, so we can suppose, decreasing ϵ if necessary, that $\int_{\Sigma_{t}}\langle\nu (t),\frac{\partial f}{\partial t}(t,\cdot)\rangle\,\,d\sigma_{t}$ is positive for all t∈(−ϵ,ϵ). Moreover, we can assume that |Σ| _g ≤|Σ _t | _g for all t∈(−ϵ,ϵ), because Σ is locally area-minimizing.

Before we prove the next proposition, we will recall some facts about the Yamabe problem on manifolds with boundary which was first studied by J.F. Escobar [10]. Let (M ⁿ,g) be a compact Riemannian manifold with boundary ∂M≠∅. It is a basic fact that the existence of a metric $\overline{g}$ in the conformal class of g having scalar curvature equal to C∈ℝ and the boundary being a minimal hypersurface is equivalent to the existence of a positive smooth function u∈C ^∞(M) satisfying

$$\begin{cases}\Delta_gu-\frac{n-2}{4(n-1)}R_gu+\frac {n-2}{4(n-1)}Cu^{(n+2)/(n-2)}=0&\textrm{on}\ M,\\\frac{\partial u}{\partial\eta}+\frac{n-2}{2(n-1)}H_gu=0& \textrm{on}\ \partial M\end{cases} $$

(6)

where η is the outward normal vector with respect to the metric g.

If u is a solution of the equation above, then u is a critical point of the following functional:

$$Q_g(\phi)=\frac{\int_M(|\nabla_g\phi|_g^2+\frac{n-2}{4(n-1)}\,R_g\,\phi ^2)\,dv+\frac{n-2}{2(n-1)}\int_{\partial M}H_g\phi^2\,d\sigma}{(\int _M|\phi|^{2n/(n-2)}\,dv)^{(n-2)/n}}.$$

The Sobolev quotient Q(M) is then defined by

$$Q(M)=\inf\{Q_g(\phi):\phi\in C^1(M), \phi\neq0\}.$$

It is a well-known fact that $Q(M)\leq Q(S_{+}^{n})$, where $S_{+}^{n}$ is the upper standard hemisphere, and that if $Q(M)<Q(S_{+}^{n})$, then there exists a smooth minimizer for the functional above. This function turns out to be a positive solution of (6), with a constant C that has the same sign as Q(M).

Proposition 3

There exists 0<ϵ ₁<ϵ such that H(t)≤0 ∀t∈[0,ϵ ₁).

Proof

Suppose, by contradiction, that there exists a sequence ϵ _k →0, ϵ _k >0, such that H(ϵ _k )>0 for all k. Consider (V _k ,g _k ), where V _k =[0,ϵ _k ]×Σ and g _k is the pullback of the metric g by $f|_{V_{k}}:V_{k}\longrightarrow M$. Therefore, V _k is a compact three-manifold with boundary satisfying

• $R_{g_{k}}\geq-2$.

• The mean curvature of ∂V _k with respect to the outward normal vector, denoted by $H_{g_{k}}$, is nonnegative. More precisely, $\partial V_{k}=\Sigma\cup\Sigma_{\epsilon_{k}}$, where Σ is a minimal surface and $\Sigma_{\epsilon_{k}}$ has positive constant mean curvature.

• $|\Sigma|_{g_{k}}=4\pi(g(\Sigma)-1)$.

Claim 1

For k sufficiently large, we have Q(V _k )<0. In particular, this implies $Q(V_{k})<Q(S_{+}^{3})$.

Proof

By Proposition 1, we have R _g =−2 on Σ. Therefore, by continuity, we have $-2\leq R_{g_{k}}\leq-1$ on V _k for k sufficiently large. Choosing ϕ=1, we obtain

Since $\frac{\partial f}{\partial t}(0,x)=\nu(x)$ and the stability operator of Σ is equal to Δ_Σ, we obtain that $\frac{d}{dt}H(t)|_{t=0}=0$. Therefore, we conclude that $H(\epsilon_{k})=O(\epsilon_{k}^{2})$ because H(0)=H _Σ=0. Moreover, if V(t)=[0,t]×Σ and g _t =(f| _V(t))^∗ g, we have that

where h is defined by $h(s,x)=dv(\frac{\partial f}{\partial s}(s,x),Df(s,x)\,e_{1}, Df(s,x)\,e_{2})$ and {e ₁,e ₂}⊂TΣ is a positive orthonormal basis with respect to the induced metric on Σ. From this, we get

$$\frac{d}{dt}\operatorname{Vol}(V(t))|_{t=0}=\int_{\Sigma}h(0,x)\,d\sigma.$$

Since $\frac{\partial f}{\partial s}(0,x)=\nu(x)$, we have h(0,x)=1. Hence, $\frac{d}{dt}\operatorname{Vol}(V(t))|_{t=0}=|\Sigma|_{g}$. From this, we obtain that $\operatorname{Vol}(V_{k})=\epsilon_{k}|\Sigma|_{g_{k}}+O(\epsilon_{k}^{2})$. Finally, it is easy to see that for k sufficiently large we have $Q(V_{k})\leq Q_{g_{k}}(\phi)<0$. This concludes the proof of the claim. □

Choose k sufficiently large such that Q(V _k )<0. Thus, we have that there exists a positive function u∈C ^∞(V _k ) such that the metric $\overline{g}=u^{4}g_{k}$ satisfies

$$R_{\overline{g}}=C<0,\quad C\in\mathbb{R},\ \textrm{on}\ V_k \quad \textrm {and}\quad H_{\overline{g}}=0\quad \textrm{on}\ \partial V_k.$$

After scaling the metric $\overline{g}$ if necessary, we can suppose that C=−2.

In analytic terms, this means that u solves

$$\begin{cases}\Delta_{g_k}u-\frac{1}{8}R_{g_k}u-\frac{1}{4}u^5=0& \textrm{on}\ V_k,\\\frac{\partial u}{\partial\eta}+\frac{1}{4}H_{g_k}u=0& \textrm {on}\ \partial V_k. \end{cases}$$

(7)

By (7) and the fact that $R_{g_{k}}\geq-2$, we have

$$\Delta_{g_k}u+\frac{1}{4}u-\frac{1}{4}u^5\geq0\quad \textrm{on} \ V_k.$$

Consider x ₀∈V _k such that $u(x_{0})=\max_{x\in V_{k}}u(x)$. If x ₀∈V _k ∖∂V _k , then we get

$$\frac{1}{4}u(x_0)\geq\frac{1}{4}u(x_0)^5.$$

Thus, u(x ₀)≤1. It follows, by the maximum principle, that either u≡1 or u<1. The first possibility does not occur because the mean curvature of $\Sigma_{g_{k}}$ with respect to g _k is positive, and with respect to $\overline{g}$ is equal to zero. It follows that u<1.

Now, suppose x ₀∈∂V _k . If u(x ₀)≥1, we obtain, by Hopf’s boundary point lemma, that either u is constant or $\frac {\partial u}{\partial\eta}(x_{0})>0$. The first possibility does not occur by the same argument used in the interior maximum case. But, since $H_{g_{k}}\geq0$, (7) implies that $\frac {\partial u}{\partial\eta}(x_{0})\leq0$. Thus, the second possibility is also not possible. Hence, u(x ₀)<1.

Therefore, we conclude that u(x)<1 for all x∈V _k . From this, we obtain that $|\Sigma|_{\overline{g}}<|\Sigma|_{g_{k}}=4\pi(g(\Sigma)-1)$.

Finally, denote by ℑ(Σ) the isotopy class of Σ in V _k . Observe that Σ is incompressible in V _k . Moreover, we have that V _k is orientable and irreducible and does not contain any one-sided compact embedded surface. Since $H_{\overline{g}}=0$, we can directly apply the version for three-manifolds with boundary of the Theorem 5.1 in [14] (see also [21]) to obtain a compact embedded surface $\overline{\Sigma}\in\mathfrak{I}(\Sigma)$ such that

$$|\overline{\Sigma}|_{\overline{g}}=\inf_{\hat{\Sigma}\in \mathfrak{I}(\Sigma)}|\hat{\Sigma}|_{\overline{g}}.$$

Therefore, $|\overline{\Sigma}|_{\overline{g}}\leq|\Sigma |_{\overline{g}}<4\pi(g(\Sigma)-1)$. But this is a contradiction with (1), since we have proven, by using the lower bound $R_{\overline{g}}\geq-2$ and the second variation of area, that we must have $|\overline{\Sigma}|_{\overline{g}}\geq4\pi(g(\Sigma )-1)$. This concludes the proof of the proposition.  □

Next, we will conclude the rigidity in Theorem 3. Observe that Proposition 3 implies $\frac{d}{dt}|\Sigma_{t}|_{g}\leq0$ for all t∈[0,ϵ ₁). Thus, |Σ _t | _g ≤|Σ| _g for all t∈[0,ϵ ₁) and this implies |Σ _t | _g =|Σ| _g for all t∈[0,ϵ ₁) because Σ is locally area-minimizing. Therefore, by Proposition 1, we have that Σ _t is totally geodesic and Ric (ν(t),ν(t))=0 on Σ _t for all t∈[0,ϵ ₁). In particular, we have that all the surfaces Σ _t are minimal and the stability operator of Σ _t , denoted by $L_{\Sigma_{t}}$, is equal to $\Delta_{\Sigma_{t}}$.

Define $\rho(t)(x)=\rho(t,x)=\langle\nu(t,x),\frac{\partial f}{\partial t}(t,x)\rangle$. We have

$$L_{\Sigma_t}\rho(t)=-H^{\prime}(t),$$

so $\Delta_{\Sigma_{t}}\rho(t)=0$. Thus, ρ(t) does not depend on x.

Since Σ _t is totally geodesic, we have that $\nabla_{\frac {\partial f}{\partial x_{i}}}\nu(t)=0$ for all i=1,2, where (x ₁,x ₂) are local coordinates on Σ. Moreover, by the fact that 〈ν(t),ν(t)〉=1, we have that $\nabla_{\frac{\partial f}{\partial t}}\nu(t)$ is tangent to Σ _t . Hence, it follows that

for all i=1,2. Hence, $\nabla_{\frac{\partial f}{\partial t}}\nu (t)=0$. This means that, for all x∈Σ, ν(t,x) is a parallel vector field along the curve α _x :[0,ϵ ₁)⟶M given by α _x (t)=f(t,x)=exp _x (w(t,x)ν(x)).

Observe that D(exp _x ) _w(t,x)ν(x)(ν(x)) is also a parallel vector field along the curve α _x . Thus, ν(t,x)=D(exp _x ) _w(t,x)ν(x)(ν(x)) because w(0,x)=0 by Proposition 1. From this, we conclude that $\rho(t)=\frac{\partial w}{\partial t}(t,x)$.

By Proposition 1, we have

$$\int_\Sigma(w(t,x)-t)\,d\sigma=0,$$

so

$$\int_{\Sigma}\frac{\partial w}{\partial t}(t,x)\,d\sigma=|\Sigma|_g.$$

Therefore, since $\frac{\partial w}{\partial t}(t,x)$ does not depend on x, we get $\frac{\partial w}{\partial t}(t,x)=1$. This implies that w(t,x)=t for all (t,x)∈[0,ϵ ₁)×Σ because w(0,x)=0. Thus, we conclude that f(t,x)=exp _x (tν(x)) and, since Σ _t are totally geodesic, the pullback of g by $f|_{[0,\epsilon _{1})\times\Sigma}$ is the product metric dt ²+g _Σ, where g _Σ is the induced metric on Σ.

Arguing similarly for t≤0, we obtain the following proposition which is the rigidity in Theorem 3.

Proposition 4

If Σ attains the equality in (1), then Σ has a neighborhood which is isometric to ((−ϵ,ϵ)×Σ,dt ²+g _Σ), where ϵ>0 and g _Σ is the induced metric on Σ which has constant Gauss curvature equal to −1.

Now, we will prove Corollary 1. Suppose Σ minimizes area in its homotopy class and Σ attains the equality in (3). Define f:ℝ×Σ⟶M by f(t,x)=exp _x (tν(x)), where ν is the unit normal vector field to Σ.

Proposition 5

f:(ℝ×Σ,dt ²+g _Σ)⟶(M,g) is a local isometry.

Proof

Consider A={t>0:f|_[0,t]×Σ is a local isometry}. By Proposition 4, this set is nonempty. Moreover, A is closed. Let us prove that A is open. Given t∈A, consider the immersed surface Σ _t ={exp _x (tν(x)):x∈Σ} with the metric induced by f. We have that Σ _t is homotopic to Σ and |Σ _t |=|Σ|. Hence, Σ _t minimizes area in its homotopy class and attains the equality in (1). Therefore, by Proposition 4, we conclude that there exists ϵ>0 such that f|_[0,t+ϵ]×Σ is a local isometry. It follows that A is open and consequently f|_[0,∞)×Σ is a local isometry. Arguing similarly for t<0, we conclude the proposition. □

To conclude Corollary 1, observe that the proposition above implies that f:(ℝ×Σ,dt ²+g _Σ)⟶(M,g) is a covering map.

4 Proof of Theorem 5

Let ∂Ω⁽¹⁾ be a connected component of ∂Ω which is isometric to (Σ,g _Σ). Consider $\alpha=\inf\{|\hat{\Sigma}|_{g}:\hat{\Sigma}\in\mathfrak{I}(\partial\Omega^{(1)})\}$, where ℑ(∂Ω⁽¹⁾) is the isotopy class of ℑ(∂Ω⁽¹⁾). By hypothesis, ∂Ω⁽¹⁾ is incompressible in Ω, H _g ≥0, and Ω is orientable and irreducible and does not contain one-sided compact embedded surfaces. Therefore, we can apply the version for three-manifolds with boundary of the Theorem 5.1 in [14] (see also [21]) to obtain a compact embedded surface $\overline{\Sigma}\in \mathfrak{I}(\partial\Omega^{(1)})$ such that $|\overline{\Sigma }|=\alpha$. Note that $\overline{\Sigma}\in\mathfrak{I}(\partial\Omega ^{(1)})$ implies $\overline{\Sigma}$ has genus equal to g(Σ).

Since all connected components of ∂Ω have nonnegative mean curvature, it follows from the maximum principle that either $\overline{\Sigma}$ is a boundary component of Ω or $\overline {\Sigma}$ is in the interior of Ω. If $\overline{\Sigma}$ is in the interior of Ω, then we obtain, by Theorem 3, that $|\overline{\Sigma}|\geq4\pi(g(\Sigma)-1)$ since R _g ≥−2 and $\overline{\Sigma}$ has genus equal to g(Σ). On the other hand, we have |∂Ω⁽¹⁾|=4π(g(Σ)−1) because ∂Ω⁽¹⁾ is isometric to (Σ,g _Σ). From this, we get $|\overline{\Sigma}|=4\pi(g(\Sigma)-1)$. Now, if $\overline {\Sigma}$ is a boundary component of Ω, then we have that $\overline{\Sigma}$ is a minimal surface because $\overline{\Sigma}$ is area-minimizing and, by hypothesis, has nonnegative mean curvature. This implies, using Theorem 3, that $|\overline{\Sigma }|\geq4\pi(g(\Sigma)-1)$. Again we conclude that $|\overline {\Sigma}|=4\pi(g(\Sigma)-1)$. It follows from the previous arguments that we can suppose $\overline{\Sigma}=\partial\Omega^{(1)}$.

By the proof of the rigidity in Theorem 3, we have that there exists ϵ>0 such that the normal exponential map $f:[0,\epsilon )\times\overline{\Sigma}\longrightarrow\Omega$ defined by f(t,x)=exp _x (tν(x)), where ν is the inward normal vector, is an injective local isometry.

Define $l=\sup\{t>0:f(t,x)=\exp_{x}(t\nu(x))\text{ is defined on }[0,t)\times\overline{\Sigma}\text{ and is an}\allowbreak \text{injective local isometry}\}$. Since Ω is complete, we have that the normal geodesics to $\overline{\Sigma}$ extend to t=l. Thus, f is defined on $[0,l]\times\overline{\Sigma}$. By continuity and the definition of l, we obtain that $f:[0,l]\times\overline{\Sigma}\longrightarrow\Omega$ is a local isometry. In particular, by continuity, the immersion $f:\overline{\Sigma}_{l}\longrightarrow\Omega$ is totally geodesic, where $\overline{\Sigma}_{l}=\{l\}\times\overline{\Sigma}$.

Again using the maximum principle, we obtain that either $f(\overline {\Sigma}_{l})$ is a boundary component of Ω, different from $\overline{\Sigma}$ because of the injectivity of f on $[0,l)\times \overline{\Sigma}$, or $f(\overline{\Sigma}_{l})$ is in the interior of Ω.

Suppose $f(\overline{\Sigma}_{l})$ is a boundary component of Ω. Since f is a local isometry on $[0,l]\times\overline{\Sigma}$, we have $\frac{\partial f}{\partial t}(l,x)$ is a unit normal vector to $\overline{\Sigma}_{l}$. It follows from this that $f:\overline{\Sigma }_{l}\longrightarrow\Omega$ is injective because $\overline{\Sigma}_{l}$ is a boundary component of Ω. Thus, $f:[0,l]\times\overline{\Sigma }\longrightarrow\Omega$ is an injective local isometry. Since Ω is connected, we obtain $f([0,l]\times\overline{\Sigma})=\Omega$. Therefore, we have that Ω is isometric to $[0,l]\times\overline {\Sigma}$.

Let us analyze the case where $f(\overline{\Sigma}_{l})$ is in the interior of Ω. First, we have that $f:\overline{\Sigma }_{l}\longrightarrow\Omega$ cannot be injective. In fact, suppose $f:\overline{\Sigma}_{l}\longrightarrow\Omega$ is injective. Thus, by the rigidity in Theorem 3, there exists ϵ>0 such that f:[0,l+ϵ)⟶Ω is an injective local isometry, which is a contradiction because of the maximality of l. Therefore, there exist $x,y\in\overline{\Sigma}, x\neq y$, such that f(l,x)=f(l,y). We have $Df(l,x)(T\overline{\Sigma }_{l})=Df(l,y)(T\overline{\Sigma}_{l})$, since otherwise f would not be injective on $[0,l)\times\overline{\Sigma}$. This implies $\frac {\partial f}{\partial t}(l,x)=-\frac{\partial f}{\partial t}(l,y)$. Thus, since $f:\overline{\Sigma}_{l}\longrightarrow\Omega$ is totally geodesic, there exist neighborhoods of x and y in $\overline{\Sigma }_{l}$, respectively, such that the images by f of these neighborhoods coincide. We conclude that $\hat{\Sigma}_{l}=f(\overline{\Sigma}_{l})$ is a one-sided embedded compact surface in Ω. But, this is a contradiction because, by hypothesis, Ω does not contain any one-sided embedded compact surface. This concludes the proof of Theorem 5.