j. differentialgeometry 84 (2010)231-265 CURVATURE ESTIMATES FOR STABLE MARGINALLY TRAPPED SURFACES Lars Andersson & Jan Metzger Abstract We derive local integral and sup- estimates for the curvature of stable marginallyouter trapped surfacesin a sliced space-time. The estimates boundthe shearofamarginallyoutertrappedsur- face in terms of the intrinsic and extrinsic curvature of a slice containing the surface. These estimates are well adapted to situ- ations of physical interest, such as dynamical horizons. 1. Introduction The celebrated regularity result for stable minimal surfaces, due to Schoen, Simon, and Yau [SSY75], gives a bound on the second funda- mental form in terms of ambient curvature and area of the surface. The proof of the main result of [SSY75] makes use of the Simons formula [Sim68] for the Laplacian of the second fundamental form, together with the non-negativity of the second variation of area. In this paper we will prove a generalization of the regularity result of Schoen, Simon, and Yau to the natural analogue of stable minimal surfaces in the con- text of Lorentz geometry, stable marginally trapped surfaces. In this case, a generalization of the Simons formula holds for the null second fundamental form, and the appropriate notion of stability is that of sta- bly outermost in the sense of [AMS05, New87]. A local area estimate for stable marginally trapped surfaces, a generalization of a result due to Pogorelov [Pog81], allows us to give a curvature bound independent of assumptions on the area of the surface. An interesting feature of our estimates is that they imply curvature bounds for stable minimal surfaces or surfaces of constant mean curvature that do not depend on bounds for the derivative of the ambient curvature. Let Σ be a spacelike surface of co-dimension two in a (3+1)-dimen- sional Lorentz manifold L and let l± be the two independent future Supported in part by the NSF, under contract no. DMS 0407732 with the Uni- versity of Miami. Received 7/11/2007. 231 232 L. ANDERSSON & J. METZGER directed null sections of the normal bundle of Σ, with corresponding mean curvatures, or null expansions, θ±. Σ is called trapped if the future directed null rays starting at Σ converge, i.e., if θ± < 0. If L contains a trapped surface and satisfies certain causal conditions, then, if in addition, the null energy condition is satisfied, L is future causally incomplete [Pen65]. Let l+ be the outgoing null normal. If L is an asymptotically flat spacetime this notion is well defined; otherwise the outgoing direction can be fixed by convention. We call Σ a marginally outer trapped surface (MOTS) if the outgoing lightrays are marginally converging, i.e., if θ+ = 0. No assumption is made on the ingoing null expansion θ− of a MOTS.If Σ is contained in a time symmetric Cauchy surface, then θ+ = 0 if and only if Σ is minimal. Marginally trappedsurfacesareof central importanceingeneral rela- tivity, wherethey play the role of apparent horizons, or quasilocal black hole boundaries. Theconjectured Penrose inequality, proved in theRie- mannian case by Huisken and Ilmanen [HI01] and Bray [Bra01], may beformulated as an inequality relating thearea of theoutermost appar- ent horizon and the ADM mass. The technique of excising the interior of black holes using apparent horizons as excision boundaries plays a crucial role in current work in numerical relativity, where much of the focus is on modelling binary black hole collisions. In spite of the importance of marginally trapped surfaces in the ge- ometry ofspace-times, theextent ofourknowledgeof theregularity and existence of these objects is rather limited compared to thesituation for minimal surfaces. Asmoothmarginally outer trappedsurfaceis stationary with respect tovariations ofareawithinitsoutgoingnullcone, inviewoftheformula δ µ = fθ+µ fl+ Σ Σ where f is a function on Σ. The second variation of area at a MOTS in the direction l+ is δ θ+ = −(|χ+|2+G(l+,l+))f fl+ where G denotes the Einstein tensor of L, and χ+ is the second fun- damental form of Σ with respect to l+. For minimal surfaces in a Rie- mannianmanifold, or maximal hypersurfacesina Lorentz manifold, the second variation operator is an elliptic operator of second order. Incon- trast, the above equation shows that the second variation operator for areaof aMOTS,withrespectto variations inthenulldirection l+, isan operator of orderzero. Therefore, although MOTScan becharacterized as stationary points of area, this point of view alone is not sufficient to yield a useful regularity result. In spite of this, as we shall see below, there is a natural generalization of the stability condition for minimal surfaces, as well as of the regularity result of Schoen, Simon, and Yau, to marginally outer trapped surfaces. CURVATURE ESTIMATES FOR STABLE MOTS 233 It is worth remarking at this point that if we consider variations of area of spacelike hypersurfaces in a Lorentz manifold, the stationary points are maximal surfaces. Maximal surfaces satisfy a quasilinear non-uniformly elliptic equation closely related to the minimal surface equation. Duetothefactthatmaximalhypersurfacesarespacelike,they are Lipschitz submanifolds. Moreover, in a space-time satisfying the timelike convergence condition, every maximal surface is stable. Hence, the regularity theory for maximal surfaces is of a different flavor than the regularity theory for minimal surfaces (cf. [Bar84]). Assume that L is provided with a reference foliation consisting of spacelike hypersurfaces {M }, and that Σ is contained in one of the t leaves M of this foliation. Let (g,K) bethe induced metric and the sec- ond fundamental form of M with respect to the futuredirected timelike normal n. Further, let ν be the outward pointing normal of Σ in M and let A bethe second fundamental form of Σ with respect to ν. After possibly changing normalization, l± = n±ν, we have θ± = H ±tr K Σ whereH = trA is the mean curvatureof Σ and tr K is the trace of the Σ projection of K to Σ. Thus the condition for Σ to be a MOTS, θ+ = 0, is a prescribed mean curvature equation. The condition that plays the role of stability for MOTS is the stably outermost condition (see [AMS05, New87]). Suppose Σ is contained in a spatial hypersurface M. Then Σ is stably locally outermost in M if there is an outward infinitesimal deformation of Σ, within M, which does not decrease θ+. This condition, which is equivalent to the condition that Σ is stable in case M is time symmetric, turns out to be sufficient to apply the technique of [SSY75] to prove a bound on the second fundamental form A of Σ in M. In contrast to the situation for minimalsurfacesthestability operator definedbythedeformation ofθ+ is not self-adjoint. Nevertheless, it has a real principal eigenvalue with a corresponding principal eigenfunction which does not change sign. Thetechniquesof[SSY75]werefirstappliedinthecontextofgeneral relativity bySchoenandYau[SY81],whereexistenceandregularity for Jang’s equation were proved. Jang’s equation is an equation for agraph in N = M ×R, and is of a form closely related to the equation θ+ = 0. Let u be a function on M, and let K¯ be the pull-back to N of K along the projection N → M. Jang’s equation is the equation D D u g¯ij i j +K¯ = 0 ij 1+|Du|2 ! where g¯ij = gij − DiuDju pis the induced metric on the graph Σ¯ of u in 1+|Du|2 N. Thus Jang’s equation can be written as θ¯= 0 with θ¯= H¯ +tr K¯, Σ¯ 234 L. ANDERSSON & J. METZGER where H¯ is the mean curvature of Σ¯ in N. This shows that Jang’s equation θ¯= 0 is a close analog to the equation θ+ = 0 characterizing a MOTS. Solutions to Jang’s equation satisfy a stability condition closely related to the stably outermost condition stated above. This is due to the fact that Jang’s equation is translation invariant in the sense that if u solves Jang’s equation, then also u+ c is a solution where c is a constant. Thus, in the sense of section 5, graphical solutions to Jang’s equation are stable. This fact allows Schoen and Yau [SY81] to apply the technique of [SSY75] to prove regularity for solutions of Jang’s equation. It is worth remarking that although the dominant energy conditionisassumedtoholdthroughout[SY81], infacttheproofofthe existenceandregularity resultforsolutionsofJang’sequationpresented in [SY81] can be carried out without this assumption. In the present paper, the dominant energy condition is not used in the proof of our main regularity result (cf. Theorem 1.2 below). It was proved by Galloway and Schoen [GS06], based on an argu- ment for solutions of the Jang’s equation in [SY81], that the stability ofMOTSimpliesa“symmetrized”stability condition, whichstates that the spectrum of a certain self-adjoint operator analogous to the second variation operator for minimal surfaces is non-negative. The fact that stability in the sense of stably outermost implies this symmetrized ver- sion of stability was used in [GS06] to give conditions on the Yamabe type of stable marginal surfaces in general dimension. It turns out that this weaker symmetrized notion of stability is in fact sufficient for the curvature estimates proved here. The symmetrized notion of stability is also used in our local area estimates. However, since this notion has no direct interpretation in terms of the geometry of the ambient space- time, we prefer to state our results in terms of the stably outermost condition. Statement of results.The stability condition for MOTS which re- places the stability condition for minimal surfaces and which allows one to apply the technique of [SSY75] is the following. Definition 1.1. Σ is stably outermost if there is a function f ≥ 0 on Σ, f 6= 0 somewhere, such that δ θ+ ≥ 0. fν When there is no room for confusion we will refer to a stably outer- mostMOTSsimplyasastableMOTS.Thisisanalogous tothestability condition for a minimal surface N ⊂ M. The condition that there ex- ists a function f on N with f ≥ 0 and f 6= 0 somewhere, such that δ H ≥ 0, is equivalent to the condition that N is stable. fν The main result of this paper is the following theorem (cf. theorem 6.10, corollary 6.11 as well as theorem 7.1). CURVATURE ESTIMATES FOR STABLE MOTS 235 Theorem 1.2. Suppose Σ is a stable MOTS in (M,g,K). Then the second fundamental form A satisfies the inequality kAk ≤ C(kKk ,k∇Kk ,kMRmk ,inj(M,g)−1). ∞ ∞ ∞ ∞ Here k ·k denotes the sup-norm of the respective quantity, taken ∞ on Σ. As an application we prove a compactness result for MOTS (cf. theorem 8.1). Theorem 1.3. Let (g ,K ) be a sequence of initial data sets on a n n manifold M. Let (g,K) be another initial data set on M such that kMRmk ≤ C, ∞ kKk∞ +kM∇Kk∞ ≤ C, inj(M,g) ≥ C−1, for some constant C. Assume that g → g in C2 (M,g) and, n loc K → K in C1 (M,g). n loc Furthermore, let Σ ⊂ M be a sequence of immersed surfaces which n are stable marginally outer trapped with respect to (g ,K ) and have n n an accumulation point in M. In addition, assume that the Σ have n uniformly locally finite area, that is, for all x ∈ M there exists 0 < r = r(x) and a = a(x) <∞ such that |Σ ∩B (x,r)| ≤ ar2 uniformly in n, n Mtn where B (x,r) denotes the ball in M around x with radius r. Mtn Thenasubsequenceof the Σ convergesto asmooth immersed surface n Σ locally in the sense of C1,α graphs. Σ is a MOTS with respect to (g,K). If Σ is compact, then it is also stable. Outline of the paper.In sections 2 and 3 we discuss the notation and preliminary results, as well as a Simonsidentity which holds for the shear of a MOTS. Section 4 introduces the linearization of the operator θ+ acting on surfaces represented as graph over a MOTS. The stability conditions we use are discussed in section 5. The curvature estimates are derived in section 6 under the assumption of local area bounds. In section 7 we show how these bounds can be derived in terms of the ambient geometry. Finally section 8 uses the established curvature bounds to prove the compactness theorem. Acknowledgments. The authors wish to thank Walter Simon, Marc Mars, GregGalloway, Rick Schoen,andGerhardHuiskenforusefulcon- versations. We are grateful for the hospitality and support of the Isaac Newton Institute, as part of this work was done during the workshop Global Problems in Mathematical Relativity. 236 L. ANDERSSON & J. METZGER 2. Preliminaries and notation In this section we set up notation and recall some preliminaries from differential geometry. In the sequel we will consider two-dimensional spacelike submanifolds Σ of a four-dimensional manifold L. As a space- time manifold, L is equipped with a metric h of signature (−,+,+,+). The inner product induced by h will frequently be denoted by h·,·i. In addition, wewillassume,thatΣiscontained inaspacelike hypersurface M inL. ThemetriconM inducedbyhwillbedenotedbyg, themetric on Σ by γ. We will denote the tangent bundles by TL,TM, and TΣ, and the space of smooth tangential vector fields along the respective manifolds by X(Σ), X(M), and X(L). Unless otherwise stated, we will assume that all manifolds and fields are smooth. We denote by n the future directed unit timelike normal of M in L, which we will assume to be a well-defined vector field along M. The normal of Σ in M will be denoted by ν, which again is assumed to be a well-defined vector field along Σ. The two directions n and ν span the normal bundle NΣ of Σ in L, and moreover, we can use them to define two canonical null directions, which also span this bundle, namely, l± := n±ν. Inadditiontothemetrics,handitsLevi-CivitaconnectionL∇induce the second fundamental form K of M in L. It is the normal part of L∇, in the sense that for all vector fields X,Y ∈ X(M), (1) L∇XY = M∇XY +K(X,Y)n. The second fundamental form of Σ in M will be denoted by A. For vector fields X,Y ∈ X(Σ) we have (2) M∇XY = Σ∇XY −A(X,Y)ν. For vector fields X,Y ∈ X(Σ), the connection of L therefore splits according to (3) L∇XY = Σ∇XY +KΣ(X,Y)n−A(X,Y)ν = Σ∇XY −II(X,Y), where II(X,Y) = A(X,Y)ν − KΣ(X,Y)n is the second fundamental form of Σ in L. Here KΣ denotes the restriction of K to TΣ, the tangential space of Σ. The trace of II with respect to γ, which is a vector in the normal bundle of Σ, is called the mean curvature vector and is denoted by (4) H = II(e ,e ), i i i X for an orthonormal basis e ,e of Σ. Since H is normal to Σ, it satisfies 1 2 (5) H = Hν −Pn where H = γijA is the trace of A and P = γijKΣ is the trace of KΣ, ij ij with respect to γ. For completeness, we note that the norms of II and CURVATURE ESTIMATES FOR STABLE MOTS 237 H are given by (6) |II|2 = |A|2 −|KΣ|2 and (7) |H|2 = H2−P2. Recall that since H and II have values normal to Σ, the norms are taken with respect to h and are therefore not necessarily non-negative. We use the following convention to represent the Riemann curvature tensor ΣRm, the Ricci tensor ΣRc, and the scalar curvature ΣSc of Σ. Here X,Y,U,V ∈ X(Σ) are vector fields. ΣRm(X,Y,U,V) = Σ∇XΣ∇YU −Σ∇YΣ∇XU −Σ∇[X,Y]U,V , (cid:10) (cid:11) ΣRc(X,Y) = ΣRm(X,e ,e ,Y), i i i X ΣSc= ΣRc(e ,e ). i i i X Analogous definitions hold for MRm, MRc, and MSc as well as LRm, LRc, and LSc, with the exception that for LRc and LSc we take the trace with respect to the indefinite metric h. We recall the Gauss and Codazzi equations of Σ in L, which relate the respective curvatures. The Riemann curvature tensors ΣRm and LRm of Σ and L, respectively, are related by the Gauss equation. For vector fields X,Y,U,V we have (8) ΣRm(X,Y,U,V)= LRm(X,Y,U,V)+ II(X,V),II(Y,U) − II(X,U),II(Y,V) . In two dimensions, all curva(cid:10)ture information(cid:11)of (cid:10)Σ is contained in(cid:11) its scalar curvature, which we will denote by ΣSc. The scalar curvature of L will be denoted by LSc. The information of the Gauss equation above is fully contained in the following equation, which emerges from the above one by first taking the trace with respect to Y,U and then with respect to X,V: (9) ΣSc= LSc+2LRc(n,n)−2LRc(ν,ν)−2LRm(ν,n,n,ν)+|H|2−|II|2. The Codazzi equation, which relates LRm to II, has the following form: (10) L∇XII(Y,Z),S = ∇YII(X,Z),S +LRm(X,Y,S,Z) for vector(cid:10)fields X,Y,Z ∈(cid:11)X(Σ(cid:10)) and S ∈ Γ(N(cid:11)Σ). There is also a version of the Gauss and Codazzi equations for the embedding of M in L. They relate the curvature LRm of L to the 238 L. ANDERSSON & J. METZGER curvature MRm of M. For vector fields X,Y,U,V ∈ X(M) we have MRm(X,Y,U,V) (11) = LRm(X,Y,U,V)−K(Y,U)K(X,V)+K(X,U)K(Y,V), (12) M∇XK(Y,U)−M∇YK(X,U) = LRm(X,Y,n,U). These equations also have a traced form, namely, (13) MSc= LSc+2LRc(n,n)−(trK)2+|K|2 and (14) MdivK −M∇trK =LRc(·,n). We now investigate the connection N∇ on the normal bundle NΣ of Σ. Recall that for sections N of NΣ and X ∈ X(Σ), this connection is defined as N∇XN = L∇XN ⊥, where again (·)⊥ means taking the normal part. We have (cid:0) (cid:1) 0 = X(1) = X hn,ni = 2hN∇Xn,ni, and similarly hN∇Xν,νi= 0. T(cid:0)herefo(cid:1)re the relevant component of N∇ is N∇Xν,n = L∇Xν,n =−K(X,ν). Recall that X is tangential to Σ. This lead us to define the 1-form S (cid:10) (cid:11) (cid:10) (cid:11) along Σ by the restriction of K(·,ν) to TΣ. (15) S(X) := K(X,ν). Then, for an arbitrary section N of NΣ with N = fν +gn, we have N∇XN =X(f)ν +X(g)n+S(X) fn+gν). In particular (cid:0) (16) N∇Xl± = ±S(X)l±. We will later consider the decomposition of II into its null components. For X,Y ∈ X(Σ) let (17) χ±(X,Y) := II(X,Y),l±i = K(X,Y)±A(X,Y). The traces of χ± respec(cid:10)tively will be called θ±: (18) θ± = hH,l±i = P ±H. The Codazzi equation (10) implies a Codazzi equation for χ±. Lemma 2.1. For vector fields X,Y,Z ∈ X(Σ) the following relation holds: ∇ χ±(Y,Z) = ∇ χ±(X,Z)+Q±(X,Y,Z) X Y (19) ∓χ±(X,Z)S(Y)±χ±(Y,Z)S(X). Here, (20) Q±(X,Y,Z) = LRm(X,Y,l±,Z). CURVATURE ESTIMATES FOR STABLE MOTS 239 3. A Simons identity for χ± We use the Codazzi equation we derived in the previous section to compute an identity for the Laplacian of χ±, which is very similar to the Simons identity for the second fundamental form of a hypersurface [Sim68, SSY75]. The Laplacian on the surface Σ is defined as the operator Σ∆ = γijΣ∇2 . ij In the sequel, we will drop the superscript on Σ∆ and Σ∇, since all tensors below will be defined only along Σ. We will switch to index notation, since this is convenient for the computations to follow. In this notation Ti1···ip j1···jq denotes a (p,q)-tensor T as the collection of its components in an ar- bitrary basis {∂ }2 for the tangent spaces. To make the subsequent i i=1 computations easier, we will usually pick a basis of normal coordinate vectors. Also note that we use Latin indices ranging from 1 to 2 to denote components tangential to the surface Σ. Recall, that the commutator of the connection is given by the Rie- mann curvature tensor, such that for a (0,2)-tensor T , ij (21) ∇ ∇ T −∇ ∇ T = ΣRm T +ΣRm T . k l ij l k ij klmi mj klmj im Note that we use the shorthand ΣRm T = ΣRm T γpq when klmj im klpj iq there is no ambiguity. That is, we assume that we are in normal coordi- nateswhereγ = γij = δ . Alsonotethatthisfixesthesignconvention ij ij for ΣRm such that ΣRc = ΣRm is positive on the round sphere. ijkl ij ikkj Lemma 3.1. The Laplacian of χ = χ+ satisfies the following iden- tity: χ ∆χ = χ ∇ ∇ θ++χ LRm χ +LRm χ ij ij ij i j ij kilk lj kilj kl +χij∇k Qkij −χ(cid:0)kjSi+χijSk (cid:1) +χij∇i(cid:0)Qkjk−θ+Sj +χjkSk(cid:1) −|II|2|χ(cid:0)|2+θ+χ+ijχ+jkχ+ki−θ+χ(cid:1)+ijχ+jkKkΣi−Pχ+ijχ+jkχ+ki where P = γijKΣ is the trace of KΣ. ij Proof. Recall that in coordinates the Codazzi equation (19) for χ ij reads (22) ∇ χ = ∇ χ +Q −χ S +χ S . i jk j ik ijk ik j jk i 240 L. ANDERSSON & J. METZGER Then compute, using (22) in the first and third step, and the commu- tator relation (21) in the second, to obtain (23) ∇ ∇ χ = ∇ ∇ χ +∇ Q −χ S +χ S k l ij k i lj k lij lj i ij l = ∇ ∇ χ +ΣRm χ +ΣRm χ i k lj (cid:0) kiml mj kim(cid:1)j lm +∇ Q −χ S +χ S k lij lj i ij l = ∇ ∇ χ +ΣRm χ +ΣRm χ i (cid:0)j kl kiml mj (cid:1) kimj lm +∇ Q −χ S +χ S +∇ Q −χ S +χ S . k lij lj i ij l i kjl kl j jl k We will use the Ga(cid:0)uss equation (8) to r(cid:1)eplace(cid:0)the ΣRm-terms by LR(cid:1)m- terms. Observe that II = −1χ+l−− 1χ−l+. ij 2 ij 2 ij Plugging this into the Gauss equation (8) gives ΣRm = LRm + 1 χ+χ−+χ−χ+−χ+χ− −χ−χ+ . ijkl ijkl 2 ik jl ik jl il jk il jk Combining with (23), we infe(cid:0)r that (cid:1) ∇ ∇ χ = ∇ ∇ χ +LRm χ +LRm χ k l ij i j kl kiml mj kimj lm + 1 χ+χ− +χ−χ+ −χ+χ− −χ−χ+ χ+ 2 il km il km kl im kl im mj + 1(cid:0)χ+ χ− +χ− χ+ −χ+χ− −χ−χ+(cid:1)χ+ 2 km ij km ij kj im kj im lm +∇(cid:0)k Qlij −χljSi+χijSl +∇i Qkjl−χ(cid:1)klSj +χjlSk . Taking the trace w(cid:0)ith respect to k,l yi(cid:1)elds (cid:0) (cid:1) ∆χ = ∇ ∇ θ++LRm χ +LRm χ ij i j kilk lj kilj kl +∇ Q −χ S +χ S +∇ Q −θ+S +χ S k kij kj i ij k i kjk j jk k 1 + χ(cid:0)−|χ+|2+hχ+,χ−iχ+(cid:1)−θ+χ(cid:0)+χ− −θ−χ+χ+ (cid:1) 2 ij ij jk ki jk ki 1(cid:0) (cid:1) + χ+χ−χ+−χ−χ+χ+ . 2 jk kl li jk kl li We contract th(cid:0)is equation with χ+ an(cid:1)d obtain ij χ ∆χ = χ ∇ ∇ θ++χ LRm χ +LRm χ ij ij ij i j ij kilk lj kilj kl +χij∇k Qkij −χ(cid:0)kjSi+χijSk (cid:1) +χij∇i(cid:0)Qkjk−θ+Sj +χjkSk(cid:1) +hχ+,χ(cid:0)−i|χ|2 − 1θ+χ+χ+χ−(cid:1)− 1θ−χ+χ+χ+ . 2 ij jk ki 2 ij jk ki Now observe that χ− = 2KΣ −χ+ and θ− = 2P − θ+. Substituting ij ij ij this into the last two terms, together with hχ+,χ−i = −|II|2, we arrive at the identity we claimed. q.e.d.