$\mathrm {SO}(p,q)$-Higgs bundles and Higher Teichmüller components

Abstract

Some connected components of a moduli space are mundane in the sense that they are distinguished only by obvious topological invariants or have no special characteristics. Others are more alluring and unusual either because they are not detected by primary invariants, or because they have special geometric significance, or both. In this paper we describe new examples of such ‘exotic’ components in moduli spaces of $\mathrm {SO}(p,q)$-Higgs bundles on closed Riemann surfaces or, equivalently, moduli spaces of surface group representations into the Lie group $\mathrm {SO}(p,q)$. Furthermore, we discuss how these exotic components are related to the notion of positive Anosov representations recently developed by Guichard and Wienhard. We also provide a complete count of the connected components of these moduli spaces (except for $\mathrm {SO}(2,q)$, with $q\geqslant 4$).

1 Introduction

For a fixed closed orientable surface S and a semisimple Lie group $\mathrm {G}$, the representation variety $\mathcal {R}(S,\mathrm {G})$, i.e. the moduli space of group homomorphisms from the fundamental group of S into $\mathrm {G}$, has multiple connected components. Some of the components are mundane in the sense that they are distinguished by obvious topological invariants and have no known special characteristics. Others are more alluring and unusual, either because they are not detected by the primary invariants or because they parametrize objects of special significance, or both.

Instances of such ‘exotic’ components are well understood in two situations. The first is the case where $\mathrm {G}$ is the split real form of a complex semisimple Lie group, in which case the exotic components are known as Hitchin components (see [35]). The second occurs when $\mathrm {G}$ is the isometry group of a non-compact Hermitian symmetric space, in which case the subspace with so-called maximal Toledo invariant has exotic components (see [12]). In [16], both of these classes of exotic components of representation varieties have been called higher Teichmüller components since they enjoy many of the geometric features of Teichmüller space.

One common feature to all higher Teichmüller components is that the representations which they parametrize are all Anosov, a concept introduced by Labourie [37]. Anosov representations have many interesting dynamical and geometric properties, generalizing convex cocompact representations into rank one Lie groups. In particular, higher Teichmüller components consist entirely of discrete and faithful representations [37] which are holonomies of geometric structures on certain closed manifolds [31]. In general, the Anosov condition is open in the representation variety and so does not by itself distinguish connected components. More recently, in [32], Guichard and Wienhard defined a notion of positivity which refines the Anosov property and is still an open condition. They conjecture that such positivity for Anosov representations is also a closed condition, and hence should detect connected components of a representation variety. They showed, moreover, that apart from the split real forms and the real forms of Hermitian type, the only other non-exceptional groups which allow positive representations are the groups locally isomorphic to $\mathrm {SO}(p,q)$ for $1<p<q$, i.e. to the special orthogonal groups with signature (p, q). This leads directly to the conjecture that ${{\,\mathrm{\mathcal {R}}\,}}(S,\mathrm {SO}(p,q))$ should have ‘exotic’ connected components, fitting in the higher Teichmüller components framework in the above sense.

In this paper we establish the existence of such exotic components, count them, and show that each exotic component contains positive Anosov representations. Our methods exploit the Non-Abelian Hodge (NAH) correspondence which defines a homeomorphism between $\mathcal {R}(S,\mathrm {G})$ and the moduli space of polystable $\mathrm {G}$-Higgs bundles on a Riemann surface, say X, homeomorphic to S. Denoting these moduli spaces by ${\mathcal M}(X,\mathrm {G})$ or simply ${\mathcal M}(\mathrm {G})$ (where we drop the X from the notation unless explicitly needed for clarity or emphasis) our results thus actually address the connected components of ${\mathcal M}(\mathrm {SO}(p,q))$. Our main theorem¹ has two parts — one is an existence result and one is a non-existence result. Namely we prove

• 1. the existence of a class of explicitly described exotic components of ${\mathcal M}(\mathrm {SO}(p,q))$ for $1<p\leqslant q$, and

• 2. the non-existence of any other exotic components of ${\mathcal M}(\mathrm {SO}(p,q))$ for both $p=1$ and $2<p\leqslant q$.

Combining these two results, and including the $2^{2g+2}$ ‘mundane’ components, yields a complete count of the connected components for the moduli spaces of $\mathrm {SO}(p,q)$-Higgs bundles ${\mathcal M}(X,\mathrm {SO}(p,q))$ or, equivalently, the representation varieties ${{\,\mathrm{\mathcal {R}}\,}}(S,\mathrm {SO}(p,q))$, for $2<p\leqslant q$.

Theorem 6.1

Let X be a compact Riemann surface of genus $g\geqslant 2$ and denote the moduli space of $\mathrm {SO}(p,q)$-Higgs bundles on X by ${\mathcal M}(\mathrm {SO}(p,q)).$ For $2< p\leqslant q$, we have

$$\begin{aligned} |\pi _0({\mathcal M}(\mathrm {SO}(p,q)))|=2^{2g+2}+{\left\{ \begin{array}{ll} 2^{2g+1}+2p(g-1)-1&{}\text {if }q=p+1\\ 2^{2g+1}&{}\text {otherwise}. \end{array}\right. } \end{aligned}$$

Remark 1.2

Our methods also show that ${\mathcal M}(\mathrm {SO}(1,q))$ does not have exotic components for $q>2$, yielding $2^{2g+1}$ connected components. For $q\geqslant 4$ our techniques fall short of a component count of ${\mathcal M}(\mathrm {SO}(2,q))$. However, we expect no new exotic components to exist (see Sect. 6.2 for details).

Except for the special cases $p=2$, $q=p$ or $q=p+1$, the group $\mathrm {SO}(p,q)$ is neither split nor of Hermitian type, so the relation between topological invariants and connected components in the representation varieties or related Higgs bundle moduli spaces cannot be inferred from previously known mechanisms.

The primary topological invariants are apparent from the structure of the Higgs bundles. In the case of $\mathrm {SO}(p,q)$-Higgs bundles on X, the objects are described by a triple $(V,W,\eta )$, where V and W are holomorphic orthogonal bundles of rank p and q respectively, such that $\Lambda ^pV\cong \Lambda ^qW$, and $\eta$ is a holomorphic section of the bundle ${{\,\mathrm{Hom}\,}}(W,V)\otimes K$, where K is the canonical bundle of X. The topological invariants are then the first and second Stiefel–Whitney classes of V and W, subject to the constraint that $sw_1(V)=sw_1(W)$. These invariants provide a primary decomposition of the moduli space ${\mathcal M}(\mathrm {SO}(p,q))$ into (not necessarily connected) components labeled by triples $(a,b,c)\in H^1(S,\mathbb {Z}_2)\times H^2(S,\mathbb {Z}_2)\times H^2(S,\mathbb {Z}_2)$. Using the notation ${\mathcal M}^{a,b,c}(\mathrm {SO}(p,q))$ to denote the union of components labeled by (a, b, c), we can thus write

$$\begin{aligned} {\mathcal M}(\mathrm {SO}(p,q))=\coprod _{(a,b,c)\in \mathbb {Z}_2^{2g}\times \mathbb {Z}_2\times \mathbb {Z}_2}{\mathcal M}^{a,b,c}(\mathrm {SO}(p,q)). \end{aligned}$$

(1.1)

Each space ${\mathcal M}^{a,b,c}(\mathrm {SO}(p,q))$ has one connected component characterized entirely by the topological invariants (a, b, c). This is the connected component which contains the moduli space of polystable orthogonal bundles with these invariants, corresponding to Higgs bundles for the maximal compact subgroup of $\mathrm {SO}(p,q)$. Denoted by ${\mathcal M}^{a,b,c}(\mathrm {SO}(p,q))_{\mathrm {top}}$, these comprise the $2^{2g+2}$ ‘mundane’ components for $2< p\leqslant q$. Our existence result identifies additional components disjoint from the ${\mathcal M}^{a,b,c}(\mathrm {SO}(p,q))_{\mathrm {top}}$ components. Identifying the topological invariants of each component of Theorem 6.1 gives the following precise component count.

Corollary 6.4

For $2<p< q-1$ and $(a,b,c)\in H^1(S,\mathbb {Z}_2)\times H^2(S,\mathbb {Z}_2)\times H^2(S,\mathbb {Z}_2)$, we have

$$\begin{aligned} |\pi _0({\mathcal M}^{a,b,c}(\mathrm {SO}(p,q)))|={\left\{ \begin{array}{ll} 2&{}\text {if }p\text { is odd and }b=0\\ 2^{2g}+1&{}\text {if }p\text { is even, }a=0\text { and }b=0\\ 1&{}\text {otherwise.} \end{array}\right. } \end{aligned}$$

Remark 1.4

For $p=1$ and $p=2$, the primary topological invariants are slightly different. For $p=q$ and $p=q-1,$ the connected component count of ${\mathcal M}^{a,b,c}(\mathrm {SO}(p,q))$ is different (see Corollaries 6.5 and 6.6). For $p=q$ and $p=q-1,$ all components had been previously detected in [17, 35] respectively. Nevertheless, the nonexistence of additional components is new.

One advantage of working on the Higgs bundle side of the NAH correspondence is that Higgs bundles and their moduli spaces possess a rich structure that provides tools which are not readily available in the representation varieties. Two of these tools, which we exploit, are a real-valued proper function defined by the $\mathrm {L}^2$-norm of the Higgs field, called the Hitchin function, and a natural holomorphic $\mathbb {C}^*$-action. These are related since the critical points of the Hitchin function occur at fixed points of the $\mathbb {C}^*$-action. When the moduli space is smooth the Hitchin function is a perfect Morse–Bott function. While this is not the case in general, the properness of the Hitchin function implies that it attains its minimum on each connected component. This allows useful information about $\pi _0$ to be extracted from the loci of local minima which, in turn, can be described using information about the corresponding $\mathbb {C}^*$-fixed points.

For many groups $\mathrm {G}$ the Hitchin function has no local minima on $\mathcal {M}(\mathrm {G})$ other than those defining the mundane components (see for example [25, 26]). In such cases these local minima yield enough information to completely count the components of ${\mathcal M}(\mathrm {G})$. The group $\mathrm {SO}(p,q)$ is not of this type. While we are able to classify all the local minima on ${\mathcal M}(\mathrm {SO}(p,q))$, the singularities in the space render this insufficient for completely determining the number of connected components. The classification of local minima nevertheless plays a crucial role in the non-existence part of our main result. In the proof of the main existence theorem, the $\mathbb {C}^*$-fixed points are helpful but the new exotic components are detected by a more direct approach.

To show that the components exist, we first describe a model for the supposed components. We then construct a map from the model to ${\mathcal M}(\mathrm {SO}(p,q))$ and show that the map has open and closed image. The description of the model invokes a variant of Higgs bundles in which the canonical bundle K is replaced by the $p^\text {th}$ power of K.

Theorem 4.1

Let X be a compact Riemann surface with genus $g\geqslant 2$ and canonical bundle K. Denote the moduli space of $K^p$-twisted $\mathrm {SO}(1,q-p+1)$-Higgs bundles on X by ${\mathcal M}_{K^p}(\mathrm {SO}(1,q-p+1))$ and the moduli space of K-twisted $\mathrm {SO}(p,q)$-Higgs bundles on X by ${\mathcal M}(\mathrm {SO}(p,q)).$ For $1\leqslant p\leqslant q,$ there is a well defined map

(1.2)

which is an isomorphism onto its image and has an open and closed image. Furthermore, if $p>1,$ then every Higgs bundle in the image of $\Psi$ has a nowhere vanishing Higgs field.

In the case $p=2$, the model described in this theorem coincides exactly with the description of the ‘exotic’ maximal components of ${\mathcal M}(\mathrm {SO}(2,q))$ (see [7, 12]), where the objects parametrized by the components are described by $K^2$-twisted Higgs bundles referred to as Cayley partners. In that setting, the emergence of the Cayley partners is a consequence of the fact that $\mathrm {SO}(2,q)$ is a group of Hermitian type; our new results for $\mathrm {SO}(p,q)$ with $p\geqslant 2$ show that the phenomenon has a more fundamental origin. In this regard, we note that our new components generalize both the afore-mentioned Cayley partners in the Hermitian case (i.e. for $p=2$) and also the Hitchin components for the split real forms $\mathrm {SO}(p,p)$ and $\mathrm {SO}(p,p+1)$ (see Sect. 7.3 for more details).

A key technical detail required to show that the map (1.2) has open image, is the fact that the spaces (both the model and its image under the map) are essentially smooth. This means that all points are either smooth points or mildly singular, thus allowing the use of Kuranishi’s methods to describe open neighborhoods of all points. The proof of this key technical detail uses the relation between the tangent spaces for points in ${\mathcal M}(\mathrm {SO}(p,q))$ and hypercohomology spaces computed from a deformation complex. This complex has three terms, with the first term coming from infinitesimal automorphisms and the third term encoding integrability obstructions. The crucial lemma establishes the vanishing of the second hypercohomology, i.e. of integrability obstructions for infinitesimal deformations. This is the first place where we exploit the natural $\mathbb {C}^*$-action on the moduli space. More precisely, it is the special structure of the fixed points of the action which allows us to prove the vanishing results for the deformation complexes at those points. We then use an upper-semicontinuity argument to extend the result to all points where it is needed. To show that the image of the map (1.2) is closed, the properness of the Hitchin fibration is exploited.

The non-existence part of the main theorem follows from a careful analysis of all the $\mathbb {C}^*$-fixed points, most of which is devoted to identifying which fixed points correspond to local minima of the Hitchin function. We show that these are of two types, namely those where the Higgs field is identically zero, and those which lie in the new exotic components. Since the former label the known ‘mundane’ components, this proves that we have not missed any components.

We now discuss a few consequences of our work for the $\mathrm {SO}(p,q)$-representation variety ${\mathcal R}(S,\mathrm {SO}(p,q))$. Recall that a representation $\rho :\pi _1(S)\rightarrow \mathrm {SO}_0(2,1)$ is called Fuchsian if it is discrete and faithful. Recall also that there is a unique (up to conjugation) principal embedding

$$\begin{aligned} \iota :\mathrm {SO}_0(2,1)\rightarrow \mathrm {SO}_0(p,p-1). \end{aligned}$$

(1.3)

One consequence of our techniques is a dichotomy for polystable $\mathrm {SO}(p,q)$-Higgs bundles (see Corollary 6.3). Translating this statement across the NAH correspondence leads to the following dichotomy for surface group representations into $\mathrm {SO}(p,q)$.

Theorem 7.6

Let S be a closed surface of genus $g\geqslant 2$. For $2<p<q-1$, the representation variety ${\mathcal R}(S,\mathrm {SO}(p,q))$ is a disjoint union of two sets,

$$\begin{aligned} {\mathcal R}(S,\mathrm {SO}(p,q))~=~{\mathcal R}^{cpt}(S,\mathrm {SO}(p,q))~\sqcup ~ {\mathcal R}^{ex}(S,S,\mathrm {SO}(p,q)), \end{aligned}$$

(1.4)

where

• $[\rho ]\in {\mathcal R}^{cpt}(S,\mathrm {SO}(p,q))$ if and only if $\rho$ can be deformed to a compact representation,

• $[\rho ]\in {\mathcal R}^{ex}(S,\mathrm {SO}(p,q))$ if and only if $\rho$ can be deformed to a representation

  $$\begin{aligned} \rho '=\alpha \oplus (\iota \circ \rho _{\mathrm {Fuch}})\otimes \det (\alpha ), \end{aligned}$$

  (1.5)

  where $\alpha$ is a representation of $\pi _1(S)$ into the compact group $\mathrm {O}(q-p+1)$, $\rho _{\mathrm {Fuch}}$ is a Fuchsian representation of $\pi _1(S)$ into $\mathrm {SO}_0(2,1)$, and $\iota$ is the principal embedding from (1.3).

Remark 1.7

The above theorem still holds for $p=q>2$, with ${\mathcal R}^{ex}(S,\mathrm {SO}(p,p))$ being the union of the Hitchin components, but it does not hold when $2<p=q-1$. Namely, there are exactly $2p(g-1)$ exotic components of ${\mathcal R}(S,\mathrm {SO}(p,p+1))$ for which the result fails. With the exception of the Hitchin component, in [17] it is conjectured that all representations in these components are Zariski dense.

The model representations (1.5) connect our work on the Higgs bundle side of the NAH correspondence to the theory of positive Anosov representations. For a parabolic subgroup $\mathrm {P}<\mathrm {G},$ the set of P-Anosov representations (see Definition 7.7) defines an open set in the representation variety consisting of representations with desirable dynamic and geometric properties [37]. In [32], Guichard and Wienhard show that for certain pairs $(\mathrm {G},\mathrm {P})$ Anosov representations can satisfy an additional positivity property. The set of positive Anosov representations is open in ${\mathcal R}(S,\mathrm {G})$ and also conjectured by Guichard, Labourie and Wienhard to be closed [32], and hence to define connected components. Moreover the connected components of this set carry natural labels determined by the topological types of principal $\mathrm {P}$-bundles (see [30]). For the classical groups, the pairs $(\mathrm {G},\mathrm {P})$ which admit a notion of positivity come in three families: one with $\mathrm {G}$ a split real form, one with $\mathrm {G}$ a Hermitian group of tube type, and a third in which $\mathrm {G}$ is locally isomorphic to $\mathrm {SO}(p,q)$. In the first two families the set of positive Anosov representations corresponds exactly to the connected components of Hitchin representations and maximal representations respectively; thus, for these families, positivity is indeed a closed condition. In the case of $\mathrm {SO}(p,q)$ the conjecture is open. However, it follows from the work of Guichard and Wienhard that the model representations (1.5) are positive Anosov representations with respect to a parabolic subgroup $\mathrm {P}$ which stabilizes a partial isotropic flag in $\mathbb {R}^{p+q}$. Hence as a corollary to Theorem 7.6 we have:

Proposition 7.13

Let $\mathrm {P}\subset \mathrm {SO}(p,q)$ be the stabilizer of the partial flag $V_1\subset V_2\subset \cdots \subset V_{p-1},$ where $V_j\subset \mathbb {R}^{p+q}$ is an isotropic j-plane. If $2<p<q-1$, then each connected component of ${\mathcal R}^{ex}(S,\mathrm {SO}(p,q))$ from (1.4) contains a nonempty open set of positive $\mathrm {P}$-Anosov representations.

Assuming the conjecture of Guichard and Wienhard, it would follow from Proposition 7.13 that the exotic components described in this paper correspond exactly to the components in ${\mathcal R}(S,\mathrm {SO}(p,q))$ containing positive Anosov $\mathrm {SO}(p,q)$-representations. As further evidence for this conclusion it is noteworthy that in the cases where positivity is known to be a closed condition, the representations all satisfy a certain irreducibility condition, namely they do not factor through any proper parabolic subgroup of $\mathrm {G}$. For the components of ${\mathcal R}(S,\mathrm {SO}(p,q))$ which do not contain representations into compact groups, we can establish this irreducibility property as a corollary to Theorem 4.1. In particular, it holds for all representations in the components ${\mathcal R}^{ex}(S,\mathrm {SO}(p,q))$ from Theorem 7.6.

Proposition 7.15

Let ${\mathcal R}^{cpt}(S,\mathrm {SO}(p,q))$ be the union of the connected components of ${\mathcal R}(S,\mathrm {SO}(p,q))$ containing compact representations. Let $2<p\leqslant q$ and $\rho \in {\mathcal R}(S,\mathrm {SO}(p,q)){{\setminus }}{\mathcal R}^{cpt}(S,\mathrm {SO}(p,q))$. Then $\rho$ does not factor through any proper parabolic subgroup of $\mathrm {SO}(p,q)$.

Though our main results are the first to prove the existence of exotic components outside the realm of higher Teichmüller theory for groups of split and Hermitian type, evidence for such components has been building for some time. As mentioned above, considerations based on the Guichard–Wienhard positivity property had placed ${\mathcal R}(S,\mathrm {SO}(p,q))$ among the representation varieties expected to have such components. Even earlier indications had come from a study of the local minima of the Hitchin function on ${\mathcal M}(\mathrm {SO}(p,q))$. While the absolute minimum, i.e. the zero level, of the function is attained on the components ${\mathcal M}^{a,b,c}(\mathrm {SO}(p,q))_{\mathrm {top}}$, in [3] the first author described additional smooth local minima at non-zero values, thus opening up the possibility that further components exist.

The special case $q=p+1$ provided the first confirmation of this possibility. Hitchin components were known to exist in ${\mathcal M}(\mathrm {SO}(p,p+1))$ by virtue of the fact that the group $\mathrm {SO}(p,p+1)$ is the split real form of $\mathrm {SO}(2p+1,\mathbb {C})$. The results in [17] show that these are not the only exotic components. With the luxury of hindsight, we now see that the additional components in ${\mathcal M}(\mathrm {SO}(p,p+1))$ coincide exactly with the exotic components described by our main results for the case $q=p+1$.

We note finally that additional features of the connected components of ${\mathcal M}(\mathrm {SO}(p,q))$ have been detected by Baraglia and Schaposnik (in [5]) by examining spectral data on generic fibers of the Hitchin fibration for ${\mathcal M}(\mathrm {SO}(p+q,\mathbb {C}))$. Their methods cannot distinguish connected components because of the genericity assumption on the fibers, but, where they apply, their methods provide an intriguing alternative perspective.

2 Higgs bundle background

In this section we recall the necessary background on $\mathrm {G}$-Higgs bundles on a compact Riemann surface and their deformation theory. Special attention is then placed on the group $\mathrm {SO}(p,q).$ Higgs bundles were introduced by Hitchin in [34] and Simpson in [44], and have been studied extensively by many authors. For real groups we will mostly follow [24]. For the rest of the paper, let X be a compact Riemann surface of genus $g\geqslant 2$ and with canonical bundle $K\rightarrow X$.

2.1 General definitions

Let $\mathrm {G}$ be a real reductive Lie group with Lie algebra $\mathfrak {g}$ and choose a maximal compact subgroup $\mathrm {H}\subset \mathrm {G}$ with Lie algebra $\mathfrak {h}\subset \mathfrak {g}$. Fix a Cartan splitting $\mathfrak {g}\cong \mathfrak {h}\oplus \mathfrak {m}$, where $\mathfrak {m}$ is the orthogonal complement of $\mathfrak {h}\subset \mathfrak {g}$ with respect to a nondegenerate ${{\,\mathrm{Ad}\,}}(\mathrm {G})$-invariant bilinear form. In particular, $[\mathfrak {h},\mathfrak {m}]\subset \mathfrak {m}$ and $[\mathfrak {m},\mathfrak {m}]\subset \mathfrak {h}$, thus such a splitting is preserved by the adjoint action of $\mathrm {H}$ on $\mathfrak {g}$, giving a linear representation $\mathrm {H}\rightarrow \mathrm {GL}(\mathfrak {m}).$ Complexifying everything yields an ${{\,\mathrm{Ad}\,}}(\mathrm {H}^\mathbb {C})$-invariant splitting $\mathfrak {g}^\mathbb {C}\cong \mathfrak {h}^\mathbb {C}\oplus \mathfrak {m}^\mathbb {C}$.

For any group $\mathrm {G}$, if P is a principal $\mathrm {G}$-bundle and $\alpha :\mathrm {G}\rightarrow \mathrm {GL}(V)$ is a linear representation, denote the associated vector bundle $P\times _{\mathrm {G}} V$ by P[V].

Definition 2.1

Fix a holomorphic line bundle $L\rightarrow X$. An L-twisted$\mathrm {G}$-Higgs bundle is a pair $({\mathcal E},\varphi )$ where ${\mathcal E}$ is a holomorphic principal $\mathrm {H}^\mathbb {C}$-bundle and $\varphi \in H^0(X,{\mathcal E}[\mathfrak {m}^\mathbb {C}]\otimes L)$ is a holomorphic section of the associated $\mathfrak {m}^\mathbb {C}$-bundle twisted by L. The section $\varphi$ is called the Higgs field.

Remark 2.2

As usual, when the line bundle L is the canonical bundle K of the Riemann surface, we refer to a K-twisted Higgs bundle as a Higgs bundle. We are mainly interested in the case $L=K,$ however, taking $L=K^p$ will also play an important role.

Example 2.3

When $\mathrm {G}$ is a compact group, we have $\mathrm {G}^\mathbb {C}=\mathrm {H}^\mathbb {C}$ and $\mathfrak {m}^\mathbb {C}=0$, so a $\mathrm {G}$-Higgs bundle is just a holomorphic $\mathrm {G}^\mathbb {C}$-bundle on X. When $\mathrm {G}$ is a complex group, we have $\mathrm {G}=\mathrm {H}^\mathbb {C}$ and $\mathfrak {m}^\mathbb {C}\cong \mathfrak {g}.$ In this case, the Higgs field is just an L-twisted section of the adjoint bundle.

Rather than dealing with principal bundles, we will use a linear representation $\alpha :\mathrm {H}^\mathbb {C}\rightarrow \mathrm {GL}(V)$ and work with vector bundles and sections of associated bundles. A holomorphic principal $\mathrm {GL}(n,\mathbb {C})$-bundle is equivalent to a rank n holomorphic vector bundle E. For $\mathrm {SL}(n,\mathbb {C})$ we obtain an oriented vector bundle $(E,\omega )$, where $\omega \in H^0(\Lambda ^nE)$ is nowhere vanishing. For $\mathrm {O}(n,\mathbb {C})$ we get an orthogonal vector bundle (E, Q), where $Q\in H^0(\mathrm {Sym}^2E^*)$ such that $\det (Q)$ is nowhere vanishing. Finally, for $\mathrm {SO}(n,\mathbb {C})$ we obtain an oriented orthogonal vector bundle $(E,Q,\omega ),$ where $\det (Q)(\omega ,\omega )=1.$

The standard representations give the following vector bundle definitions, which are equivalent to the principal bundle formulations given by Definition 2.1.

Definition 2.4

An L-twisted$\mathrm {GL}(n,\mathbb {C})$-Higgs bundle over X is a pair $(E,\Phi )$, where $E\rightarrow X$ is a rank n holomorphic vector bundle and $\Phi \in H^0({{\,\mathrm{End}\,}}(E)\otimes L)$.

An L-twisted$\mathrm {SL}(n,\mathbb {C})$-Higgs bundle is a triple $(E,\omega ,\Phi )$, where $(E,\omega )$ is a rank n holomorphic oriented vector bundle and $\Phi \in H^0({{\,\mathrm{End}\,}}(E)\otimes L)$ satisfies ${{\,\mathrm{tr}\,}}(\Phi )=0$.

An L-twisted$\mathrm {O}(n,\mathbb {C})$-Higgs bundle is a triple $(E,Q,\Phi )$, where (E, Q) is a rank n holomorphic orthogonal vector bundle and $\Phi \in H^0({{\,\mathrm{End}\,}}(E)\otimes L)$ satisfies $\Phi ^TQ+Q\Phi =0.$

An L-twisted$\mathrm {SO}(n,\mathbb {C})$-Higgs bundle is a quadruple $(E,Q,\omega ,\Phi )$, where $(E,Q,\omega )$ is a rank n holomorphic oriented orthogonal vector bundle and $\Phi \in H^0({{\,\mathrm{End}\,}}(E)\otimes L)$ satisfies $\Phi ^TQ+Q\Phi =0.$

Two $\mathrm {GL}(n,\mathbb {C})$-Higgs bundles $(E_1,\Phi _1)$ and $(E_2,\Phi _2)$ are isomorphic if there exists a holomorphic bundle isomorphism $f:E_1\rightarrow E_2$ so that $f^*\Phi _2=\Phi _1.$ For $\mathrm {SL}(n,\mathbb {C}),$$\mathrm {O}(n,\mathbb {C})$ and $\mathrm {SO}(n,\mathbb {C})$-Higgs bundles we require that the isomorphism f pulls back the additional structure.

The group $\mathrm {O}(p,q)$ is the group of linear automorphisms of $\mathbb {R}^{p+q}$ which preserve a nondegenerate symmetric quadratic form of signature (p, q). Note that $\mathrm {O}(p,q)$ and $\mathrm {O}(q,p)$ are isomorphic groups, so we can assume that $p\leqslant q$ without loosing any generality. We are mainly interested in the subgroup $\mathrm {G}=\mathrm {SO}(p,q)$ of $\mathrm {O}(p,q)$ which also preserves an orientation of $\mathbb {R}^{p+q}$. This group has two connected components provided p and q are both positive, and the connected component of the identity is denoted by $\mathrm {SO}_0(p,q)$. We shall assume throughout the paper that $0<p\leqslant q$.

If $Q_p$ and $Q_q$ are positive definite symmetric $p\times p$ and $q\times q$ matrices, then the Lie algebra $\mathfrak {so}(p,q)$ is defined by the matrices

$$\begin{aligned} \mathfrak {so}(p,q)\cong \left\{ \left( {\begin{matrix}A&{}B\\ C&{}D\end{matrix}}\right) \ \left| \ \left( {\begin{matrix}A&{}B\\ C&{}D\end{matrix}}\right) ^T\right. \left( {\begin{matrix}Q_p&{}\\ {} &{}-Q_q\end{matrix}}\right) +\left( {\begin{matrix}Q_p&{}\\ {} &{}-Q_q\end{matrix}}\right) \left( {\begin{matrix}A&{}B\\ C&{}D\end{matrix}}\right) =0\right\} , \end{aligned}$$

where A is a $p\times p$ matrix, B is a $p\times q$ matrix, C is a $q\times p$ matrix and D is a $q\times q$ matrix. Thus,

(2.1)

The maximal compact subgroup of $\mathrm {O}(p,q)$ is $\mathrm {O}(p)\times \mathrm {O}(q)$ and the maximal compact subgroup of $\mathrm {SO}(p,q)$ is $\mathrm {S}(\mathrm {O}(p)\times \mathrm {O}(q))$. Using (2.1), the complexified Cartan decomposition of $\mathfrak {so}(p,q)$ is

$$\begin{aligned} \mathfrak {so}(p+q,\mathbb {C})\cong (\mathfrak {so}(p,\mathbb {C})\oplus \mathfrak {so}(q,\mathbb {C}))\oplus {{\,\mathrm{Hom}\,}}(W,V), \end{aligned}$$

where V and W are the standard representations of $\mathrm {O}(p,\mathbb {C})$ and $\mathrm {O}(q,\mathbb {C}).$ Using these representations, we have the following vector bundle definition of an $\mathrm {SO}(p,q)$-Higgs bundle.

Definition 2.5

An L-twisted$\mathrm {O}(p,q)$-Higgs bundle is a tuple $(V,Q_V,W,Q_W,\eta )$, where $(V,Q_V),$$(W,Q_W)$ are rank p, q holomorphic orthogonal vector bundles respectively and $\eta \in H^0({{\,\mathrm{Hom}\,}}(W,V)\otimes L).$

An L-twisted$\mathrm {SO}(p,q)$-Higgs bundle is a tuple $(V,Q_V,W,Q_W,\omega ,\eta ),$ where $(V,Q_V,W,Q_W,\eta )$ is an L-twisted $\mathrm {O}(p,q)$-Higgs bundle and $\left( V\oplus W,\left( {\begin{matrix}Q_V\\ {} &{}-Q_W\end{matrix}}\right) ,\omega \right)$ is an oriented orthogonal vector bundle.

Remark 2.6

We will usually interpret the orthogonal structures and the orientation as isomorphisms:

$$\begin{aligned} Q_V:V\xrightarrow {\ \ \cong \ \ } V^*, \ \ \ \ Q_W:W\xrightarrow {\ \ \cong \ \ } W^*\ \ \ \ \ \text {and}\ \ \ \ \ \ \omega :\Lambda ^p V\xrightarrow {\ \ \cong \ \ }\Lambda ^qW. \end{aligned}$$

Moreover, we will usually suppress the orthogonal structures and orientation from the notation.

Two $\mathrm {SO}(p,q)$-Higgs bundles $(V_1,Q_{V_1},W_1,Q_{W_1},\omega _1,\eta _1)$ and $(V_2,Q_{V_2},W_2,Q_{W_2},\omega _2,\eta _2)$ are isomorphic if there exists bundle isomorphisms $g_V:V_1\rightarrow V_2$ and $g_W:W_1\rightarrow W_2$ so that

Given an L-twisted $\mathrm {SO}(p,q)$-Higgs bundle $(V,Q_V,W,Q_W,\eta )$, let

$$\begin{aligned} \eta ^*=(Q_W^{-1}\otimes {{\,\mathrm{Id}\,}}_L)(\eta ^T\otimes {{\,\mathrm{Id}\,}}_L)Q_V, \end{aligned}$$

where $\eta ^T:V^*\otimes L^{-1}\rightarrow W^*$ is the dual map. The L-twisted $\mathrm {SO}(p+q,\mathbb {C})$-Higgs bundle associated to $(V,Q_V,W,Q_W,\eta )$ is given by

$$\begin{aligned} (E,Q,\omega ,\Phi )=\Big (V\oplus W,\left( {\begin{matrix}Q_V&{}0\\ 0&{}-Q_W\end{matrix}}\right) ,\omega ,\left( {\begin{matrix}0&{}\eta \\ \eta ^*&{}0\end{matrix}}\right) \Big ). \end{aligned}$$

(2.2)

In subsequent sections, we will also need the notions of $\mathrm {U}(p,q)$-Higgs bundles and $\mathrm {GL}(n,\mathbb {R})$-Higgs bundles. The complexified Cartan decompositions for these groups are given by

$$\begin{aligned}&\mathfrak {u}(p,q)^\mathbb {C}\cong (\mathfrak {gl}(p,\mathbb {C})\oplus \mathfrak {gl}(q,\mathbb {C})) \oplus ({{\,\mathrm{Hom}\,}}(E,F)\oplus {{\,\mathrm{Hom}\,}}(F,E)),\\&\mathfrak {gl}(n,\mathbb {R})^\mathbb {C}\cong \mathfrak {o}(n,\mathbb {C})\oplus \mathrm {sym}(\mathbb {C}^n), \end{aligned}$$

where E and F are respectively the standard representations of $\mathrm {GL}(p,\mathbb {C})$ and $\mathrm {GL}(q,\mathbb {C})$ and $\mathrm {sym}(\mathbb {C}^n)$ denotes the set of symmetric endomorphisms of $\mathbb {C}^n.$

Definition 2.7

An L-twisted$\mathrm {U}(p,q)$-Higgs bundle is a tuple $(E,F,\beta ,\gamma )$, where E, F are holomorphic vector bundles of rank p, q respectively, $\beta \in H^0({{\,\mathrm{Hom}\,}}(F,E)\otimes L)$ and $\gamma \in H^0({{\,\mathrm{Hom}\,}}(E,F)\otimes L)$.

An L-twisted$\mathrm {GL}(n,\mathbb {R})$-Higgs bundle is a tuple $(E,Q,\Phi )$ where (E, Q) is a holomorphic rank n orthogonal vector bundle and $\Phi \in H^0({{\,\mathrm{End}\,}}(E)\otimes L)$ such that $\Phi ^TQ=Q\Phi$.

2.2 The Higgs bundle moduli space and deformation theory

To form a moduli space of $\mathrm {G}$-Higgs bundles we need a notion of stability for these objects. In general, these stability notions involve the interaction of the Higgs field with certain parabolic reductions of structure group. For the above groups, stability can be simplified and expressed in vector bundle terms (see [24]).

Proposition 2.8

An L-twisted $\mathrm {SL}(n,\mathbb {C})$-Higgs bundle $(E,\Phi )$ is

• semistable if for every holomorphic subbundle $F\subset E$ with $\Phi (F)\subset F\otimes L$ we have $\deg (F)\leqslant 0,$

• stable if for every proper holomorphic subbundle $F\subset E$ with $\Phi (F)\subset F\otimes L$ we have $\deg (F)<0,$

• polystable if it is semistable and for every degree zero subbundle $F\subset E$ with $\Phi (F)\subset F\otimes L$, there is a subbundle $F'$ with $\Phi (F')\subset F'\otimes L$ so that $E\cong F\oplus F'.$ That is,

  $$\begin{aligned} (E,\Phi )=\Big (F\oplus F',\left( {\begin{matrix}\Phi _F&{}0\\ 0&{}\Phi _{F'}\end{matrix}}\right) \Big ). \end{aligned}$$

Remark 2.9

For the notions of stability, semistability and polystability for an L-twisted $\mathrm {O}(n,\mathbb {C})$-Higgs bundles $(E,Q,\Phi ),$ one only needs to consider isotropic subbundles $F\subset E$ with $\Phi (F)\subset F\otimes K$. Here a subbundle $F\subset E$ is isotropic if $F\subset F^\perp$, where $F^\perp$ is the perpendicular subbundle defined by Q. For a polystable L-twisted $\mathrm {O}(n,\mathbb {C})$-Higgs bundle, if $F\subset E$ is a degree zero isotropic subbundle with $\Phi (F)\subset F\otimes L,$ then $E\cong F\oplus F'$ where $F'$ is a degree zero coisotropic subbundle satisfying $\Phi (F')\subset F'\otimes L.$ We note also that the polystability of (E, Q) as an orthogonal vector bundle is equivalent to the polystability of E as a vector bundle [40].

For real groups, the notions of semistability, stability and polystability are a bit more involved. However, for the purpose of defining the moduli spaces we are interested in, it is sufficient to use the following result of [24].

Proposition 2.10

Let $\mathrm {G}$ be a real form of a simple subgroup of $\mathrm {SL}(n,\mathbb {C}).$ An L-twisted $\mathrm {G}$-Higgs bundle $({\mathcal E},\varphi )$ is polystable if and only if the induced $\mathrm {SL}(n,\mathbb {C})$-Higgs bundle is polystable in the sense of Proposition 2.8. The analogous statement for semistability also holds.

Definition 2.11

The moduli space ofL-twisted$\mathrm {G}$-Higgs bundles onX is the set ${\mathcal M}_L(\mathrm {G})$ of isomorphism classes of polystable L-twisted $\mathrm {G}$-Higgs bundles. The subset where $\mathcal {E}$ has fixed topological type a is denoted by ${\mathcal M}^a_L(\mathrm {G})\subset {\mathcal M}_L(\mathrm {G})$. In the case $L=K$, we shall denote the corresponding moduli spaces just by ${\mathcal M}^ a(\mathrm {G})\subset {\mathcal M}(\mathrm {G})$.

Remark 2.12

The above defines the moduli space as a set. It can be given the structure of a complex analytic variety using standard methods, as we briefly outline in Sect. 2.3 below. Alternatively, the moduli space can be constructed algebraically as the set of ${\mathcal S}$-equivalence classes of semistable $\mathrm {G}$-Higgs bundles as a particular case of a construction of Schmitt [42] using geometric invariant theory. When the maximal compact subgroup of $\mathrm {G}$ is semisimple and $\deg (L)\geqslant 2g-2$, the expected dimension of ${\mathcal M}_L(\mathrm {G})$ is $\dim (\mathfrak {h})(g-1)+\dim (\mathfrak {m})(\deg (L)+1-g)$.

The automorphism group ${{\,\mathrm{Aut}\,}}({\mathcal E},\varphi )$ of a $\mathrm {G}$-Higgs bundle $({\mathcal E},\varphi )$ consists of holomorphic bundle automorphisms which act trivially on the Higgs field:

$$\begin{aligned} {{\,\mathrm{Aut}\,}}({\mathcal E},\varphi )=\{f:{\mathcal E}\rightarrow {\mathcal E}|\ {{\,\mathrm{Ad}\,}}_f\varphi =\varphi \}. \end{aligned}$$

(2.3)

The center ${\mathcal Z}(\mathrm {G}^\mathbb {C})$ of $\mathrm {G}^\mathbb {C}$ is the intersection of the center of $\mathrm {H}^\mathbb {C}$ and the kernel of the representation ${{\,\mathrm{Ad}\,}}:\mathrm {H}^\mathbb {C}\rightarrow \mathrm {GL}(\mathfrak {m}^\mathbb {C})$. Thus, we always have ${\mathcal Z}(\mathrm {G}^\mathbb {C})\subset {{\,\mathrm{Aut}\,}}({\mathcal E},\varphi )$.

Remark 2.13

If $\mathrm {G}^\mathbb {C}$ is semisimple, then a $\mathrm {G}$-Higgs bundle is stable if and only if it is polystable with finite automorphism group.

The deformation theory of a $\mathrm {G}$-Higgs bundle $({\mathcal E},\varphi )$ is governed by the complex of sheaves

(2.4)

indeed $\mathbb {H}^0(C^\bullet ({\mathcal E},\varphi ))$ can be identified with the Lie algebra of ${{\,\mathrm{Aut}\,}}({\mathcal E},\varphi )$ and $\mathbb {H}^1(C^\bullet ({\mathcal E},\varphi ))$ is the infinitesimal deformation space (see [8]). There is a long exact sequence in hypercohomology:

(2.5)

Remark 2.14

When the group $\mathrm {G}$ is complex, Serre duality implies that the second hypercohomology group in this deformation complex is isomorphic to the dual of the zeroth hypercohomology group [24, Proposition 3.17]. In particular, this implies that for complex semisimple groups $\mathbb {H}^2(C^\bullet ({\mathcal E},\varphi ))$ vanishes if and only if the Higgs bundle $({\mathcal E},\varphi )$ is stable.

2.3 The complex analytic point of view on the moduli space

Fix a $C^\infty$ principal $\mathrm {H}^\mathbb {C}$-bundle P. The set of holomorphic structures on P is an affine space modeled on $\Omega ^{0,1}(X,P[\mathfrak {h}^\mathbb {C}])$: indeed, since $\dim (X)=1$, any partial connection on P is integrable and thus defines a holomorphic structure on P. A partial connection induces a Dolbeault operator ${\bar{\partial }}_P$ on any vector bundle associated to P and we shall by a slight abuse of notation denote the partial connection itself by the same symbol. We can now introduce the space of L-twisted Higgs bundle structures on P by

$$\begin{aligned} {\mathcal H}_L(\mathrm {G},P)=\{({\bar{\partial }}_P,\varphi )\ |\ {\bar{\partial }}_P\varphi =0\}, \end{aligned}$$

(2.6)

where $\varphi \in \Omega ^{0}(X,P[\mathfrak {m}^\mathbb {C}]\otimes L)$ is the Higgs field.

The complex gauge group ${\mathcal G}_{\mathrm {H}^\mathbb {C}}$ of $C^\infty$ bundle automorphisms of P acts on the space ${\mathcal H}_L(\mathrm {G},P)$. Moreover, this action preserves the subspace ${\mathcal H}_L(\mathrm {G},P)^{ps}\subset {\mathcal H}_L(\mathrm {G},P)$ of polystable L-twisted Higgs bundles. If we denote the topological type of P by a, we thus have an identification

$$\begin{aligned} {\mathcal M}^a_L(\mathrm {G}) = {\mathcal H}_L(P,\mathrm {G})^{ps}/{\mathcal G}_{\mathrm {H}^\mathbb {C}}. \end{aligned}$$

In order to give the moduli space a topology, suitable Sobolev completions must be used in standard fashion; see [4], and also [33, Sec. 8] where the straightforward adaptation to Higgs bundles is discussed in the case $\mathrm {G}=\mathrm {GL}(n,\mathbb {C})$. Then the orbits of the ${\mathcal G}_{\mathrm {H}^\mathbb {C}}$-action on ${\mathcal H}_L(\mathrm {G},P)^{ps}$ are closed in the space of semistable $\mathrm {G}$-Higgs bundles and the moduli space $\mathcal {M}_L(\mathrm {G})$ becomes a Hausdorff topological space.

Remark 2.15

If ${\mathcal H}^s_L(P,\mathrm {G})\subset {\mathcal H}^{ps}_L(P,\mathrm {G})$ denotes the subset of stable Higgs bundle structures, then ${\mathcal H}^s_L(P,\mathrm {G})$ is open in ${\mathcal H}^{ps}_L(P,\mathrm {G})$. The stable objects thus define open subsets of ${\mathcal M}^a_L(\mathrm {G})$.

The moduli space can be given the structure of a complex analytic variety using the Kuranishi model in a way analogous to the case of vector bundles on algebraic surfaces; see, e.g., [20, Sec 6.4.1] or [23, Chap. 4]. We briefly recall this for a point represented by a Higgs bundle $({\mathcal E},\varphi )$ with vanishing $\mathbb {H}^2(C^\bullet ({\mathcal E},\varphi ))$, this being the only case we shall need. For any such $({\mathcal E},\varphi )$, there is an open neighborhood U of zero in $\mathbb {H}^1(C^\bullet ({\mathcal E},\varphi ))$ and a local versal family of $\mathrm {G}$-Higgs bundles parametrized by U which restricts to $({\mathcal E},\varphi )$ over $\{0\}\times X$. Moreover, if $({\mathcal E},\varphi )$ is polystable then (semistability being an open condition) U can be taken to consist only of semistable $\mathrm {G}$-Higgs bundles, and the map taking a semistable $\mathrm {G}$-Higgs bundle in U to the polystable representative of its ${\mathcal S}$-equivalence class projects U onto an open neighborhood of $({\mathcal E},\varphi )$ in the moduli space.

Though we shall not need this, we note that the neighborhood U can be taken to be ${{\,\mathrm{Aut}\,}}({\mathcal E},\varphi )$-invariant and then an open neighborhood of $({\mathcal E},\varphi )$ in the moduli space is modeled on the GIT quotient When the automorphism group ${{\,\mathrm{Aut}\,}}({\mathcal E},\varphi )$ is finite, the GIT quotient simplifies to a regular quotient, and the isomorphism class $({\mathcal E},\varphi )$ defines (at worst) an orbifold point of ${\mathcal M}_L(\mathrm {G})$.

2.4 Stability and deformation complex for $\mathrm {G}=\mathrm {SO}(p,q)$

We shall need the precise notion of stability for $\mathrm {SO}(p,q)$-Higgs bundles. The derivation of the following simplification of the stability notion for $\mathrm {SO}(p,q)$-Higgs bundles is very similar to many cases treated in the literature. For example, see [26] for the case $\mathrm {G}=\mathrm {Sp}(2p,2q).$

Proposition 2.16

Let $(V,Q_V,W,Q_W,\omega ,\eta )$ be an L-twisted $\mathrm {SO}(p,q)$-Higgs bundle and let $\eta ^*=Q_W^{-1}\eta ^TQ_V$. Then it is

• semistable if and only if for any pair of isotropic subbundles $V_1\subset V$ and $W_1\subset W$ such that $\eta (W_1)\subset V_1\otimes L$ and $\eta ^*(V_1)\subset W_1\otimes L$, we have $\deg (V_1)+\deg (W_1)\leqslant 0,$

• stable if and only if for any pair of isotropic subbundles $V_1\subset V$ and $W_1\subset W$, at least one of which is a proper² subbundle, and such that $\eta (W_1)\subset V_1\otimes L$ and $\eta ^*(V_1)\subset W_1\otimes L$, we have $\deg (V_1)+\deg (W_1)< 0,$

• polystable if and only if it is semistable and whenever $V_1\subset V$ and $W_1\subset W$ are isotropic subbundles with $\eta (W_1)\subset V_1\otimes L$, $\eta ^*(V_1)\subset W_1\otimes L$ and $\deg (V_1)+\deg (W_1)=0,$ there are coisotropic bundles $V_2\subset V$ and $W_2\subset W$, complementary to $V_1$ and $W_1$ respectively, so that $\eta (W_2)\subset V_2\otimes L$ and $\eta ^*(V_2)\subset W_2\otimes L$. That is,

  $$\begin{aligned} (V,W,\eta )=\Big (V_1\oplus V_2, W_1\oplus W_2, \left( {\begin{matrix}\eta _1&{}0\\ 0&{}\eta _2\end{matrix}}\right) \Big ). \end{aligned}$$

We now give a recursive classification of strictly polystable $\mathrm {SO}(p,q)$-Higgs bundles, which will be important in the following sections of the paper.

Given a $\mathrm {U}(p,q)$-Higgs bundle $(E,F,\beta ,\gamma )$ with $\deg (E\oplus F)=0$, consider the associated $\mathrm {SO}(2p,2q)$-Higgs bundle

$$\begin{aligned} (V,Q_V,W,Q_W,\eta )=\Big (E\oplus E^*,\left( {\begin{matrix}0&{}{{\,\mathrm{Id}\,}}\\ {{\,\mathrm{Id}\,}}&{}0\end{matrix}}\right) ,F\oplus F^*,\left( {\begin{matrix}0&{}{{\,\mathrm{Id}\,}}\\ {{\,\mathrm{Id}\,}}&{}0\end{matrix}}\right) ,\left( {\begin{matrix}\beta &{}0\\ 0&{}\gamma ^T\end{matrix}}\right) \Big ). \end{aligned}$$

If $(E,F,\beta ,\gamma )$ is a polystable $\mathrm {U}(p,q)$-Higgs bundle, then this $\mathrm {SO}(2p,2q)$-Higgs bundle is strictly polystable. Indeed, E, $E^*,$F and $F^*$ are all isotropic subbundles with $\deg (E)+\deg (F)=0$ and

Proposition 2.17

An $\mathrm {SO}(p,q)$-Higgs bundle $({V,}{Q_V,}{W,}{Q_W,}\eta )$ is polystable if and only if it is isomorphic to

$$\begin{aligned} \bigg (E\oplus E^*\oplus V_0, \left( {\begin{matrix}0&{}{{\,\mathrm{Id}\,}}&{}0\\ {{\,\mathrm{Id}\,}}&{}\quad 0&{}\quad 0\\ 0&{}\quad 0&{}\quad Q_{V_0}\end{matrix}}\right) , F\oplus F^*\oplus W_0, \left( {\begin{matrix}0&{}\quad {{\,\mathrm{Id}\,}}&{}\quad 0\\ {{\,\mathrm{Id}\,}}&{}\quad 0&{}\quad 0\\ 0&{}\quad 0&{}\quad Q_{W_0}\end{matrix}}\right) ,\left( {\begin{matrix}\beta &{}\quad 0&{}\quad 0\\ 0&{}\quad \gamma ^T&{}\quad 0\\ 0&{}\quad 0&{}\quad \eta _0\end{matrix}}\right) \bigg ), \end{aligned}$$

(2.7)

where $(E,F,\beta ,\gamma )$ is a polystable $\mathrm {U}(p_1,q_1)$-Higgs bundle with $\deg (E)+\deg (F)=0$, and $(V_0,Q_{V_0},W_0,Q_{W_0},\eta _0)$ is a stable $\mathrm {SO}(p-2p_1,q-2q_1)$-Higgs bundle.

Proof

If $(V,Q_V,W,Q_V,\eta )$ is stable, take $p_1=q_1=0$. Suppose that $(V,Q_V,W,Q_W,\eta )$ is strictly polystable and that $E\subset V$ and $F\subset W$ are isotropic subbundles of rank $p_1$ and $q_1$ respectively, such that $\deg (E)+\deg (F)=0$ and

Since $(V,W,\eta )$ is polystable, the bundles V and W split as $V=E\oplus V'$ and $W=F\oplus W'$, where $V'$ and $W'$ are both coisotropic subbundles with the property

Since the bundles E and F are isotropic, the bundles $V'$ and $W'$ are extensions of the form:

We claim that the above extension classes vanish. For the bundle V we have a holomorphic splitting $E\oplus V'$ and a smooth splitting $E\oplus E^\perp /E\oplus E^*$. In this smooth splitting, the orthogonal structure $Q_V$ and the ${\bar{\partial }}$-operator on V are isomorphic to

where $\alpha \in \Omega ^{0,1}({{\,\mathrm{Hom}\,}}(E^*,E^\perp /E)).$ However, since the orthogonal structure $Q_V$ is holomorphic, we have $\alpha =0.$ By applying the same argument to the bundle W, we have the following holomorphic splitting

$$\begin{aligned} (W,Q_W)\cong \bigg (F\oplus F^\perp /F\oplus F^*,\left( {\begin{matrix}0&{}0&{}{{\,\mathrm{Id}\,}}\\ 0&{}Q_{F^\perp /F}&{}0\\ {{\,\mathrm{Id}\,}}&{}0&{}0\end{matrix}}\right) \bigg ). \end{aligned}$$

The conditions $\eta (F)\subset E\otimes K$, $\eta ^*(E)\subset F\otimes K$ and $\eta (W')\subset V'\otimes K$ imply that $\eta$ is given by

$$\begin{aligned} \eta =\left( {\begin{matrix}\beta &{}0&{}0\\ 0&{}\eta _0&{}0\\ 0&{}0&{}\gamma ^T\end{matrix}}\right) :F\oplus F^\perp /F\oplus F^*\longrightarrow E\oplus E^\perp /E\oplus E^*. \end{aligned}$$

The tuple $(E,F,\beta ,\gamma )$ defines a polystable $\mathrm {U}(p_1,q_1)$-Higgs bundle and

$$\begin{aligned} (V_0,Q_{V_0},W_0,Q_{W_0},\eta _0)=(E^\perp /E,Q_{E^\perp /E},F^\perp /F,Q_{F^\perp /F},\eta _0) \end{aligned}$$

defines a polystable $\mathrm {SO}(p-2p_1,q-2q_1)$-Higgs bundle. By iterating this process if necessary, we may assume $(V_0,W_0,\eta _0)$ is stable.

The converse statement is clear. $\square$

For an L-twisted $\mathrm {SO}(p,q)$-Higgs bundle $(V,Q_V,W,Q_W,\omega ,\eta )$, write

Then the Lie algebra bundles ${\mathcal E}[\mathfrak {h}^\mathbb {C}]$ and ${\mathcal E}[\mathfrak {m}^\mathbb {C}]\otimes L$ are given by

The deformation complex (2.4) becomes

(2.8)

and the long exact sequence (2.5) is given by

(2.9)

We will use the above complex and long exact sequence extensively throughout the paper.

Finally, we make explicit the gauge theoretic perspective for $\mathrm {SO}(p,q)$-Higgs bundles. Fix $C^\infty$ rank p and q orthogonal vector bundles $(\underline{V},Q_V)$ and $(\underline{W},Q_W)$ respectively, and a smooth nowhere vanishing section $\omega$ of $\Lambda ^{p+q}\underline{(}V\oplus W)$ so that $\det (Q_V\oplus -Q_W)(\omega ,\omega )=1$. An $\mathrm {SO}(p,q)$-Higgs bundle structure on $(\underline{V},Q_V,\underline{W},Q_W,\omega )$ consists of a triple $({\bar{\partial }}_V,{\bar{\partial }}_W,\eta )$, where $\eta \in \Omega ^{0,1}({{\,\mathrm{Hom}\,}}(W,V)\otimes K)$ and ${\bar{\partial }}_V$ and ${\bar{\partial }}_W$ are Dolbeault operators on $\underline{V}$ and $\underline{W}$ with respect to which $Q_V,$$Q_W,$$\omega$ and $\eta$ are each holomorphic. An isomorphism between two such Higgs bundle structures $({\bar{\partial }}_V,{\bar{\partial }}_W,\eta )$ and $({\bar{\partial }}_V',{\bar{\partial }}_W',\eta ')$ is given by an element of the $\mathrm {S}(\mathrm {O}(p,\mathbb {C})\times \mathrm {O}(q,\mathbb {C}))$-gauge group. That is, a pair of $C^\infty$ bundle automorphism $f_V:\underline{V}\rightarrow \underline{ V}$ and $f_W:\underline{W}\rightarrow \underline{W}$ so that

with the property that $(f_V^*{\bar{\partial }}_V',f_W^*{\bar{\partial }}_W',f_W^{-1}\eta 'f_V)=({\bar{\partial }}_V,{\bar{\partial }}_W,\eta ).$

2.5 The Hitchin fibration and Hitchin component

Let $\mathrm {G}^\mathbb {C}$ be a complex semisimple Lie group of rank $\ell$ and let $p_1,\ldots ,p_{\ell }$ be a basis of $\mathrm {G}^\mathbb {C}$-invariant homogeneous polynomials on $\mathfrak {g}^\mathbb {C}$ with $\deg (p_j)=m_j+1.$ Given an L-twisted $\mathrm {G}^\mathbb {C}$-Higgs bundle $({\mathcal E},\varphi ),$ the tensor $p_j(\varphi )$ is a holomorphic section of $L^{m_j+1}.$ The map $({\mathcal E},\varphi )\mapsto (p_1(\varphi ),\ldots ,p_{\ell }(\varphi ))$ descends to a map

(2.10)

known as the Hitchin fibration. In [34], Hitchin showed that h is a proper map for $L=K$, and for general L properness was shown by Nitsure in [39].

Another important aspect of the Hitchin fibration for this paper is the Hitchin section.

Theorem 2.18

(Hitchin [35]) Let $\mathrm {G}$ be the split real form of a complex semisimple Lie group $\mathrm {G}^\mathbb {C}$ of rank $\ell$. There is a section of the fibration (2.10) with $L=K$ such that the image consists of $\mathrm {G}$-Higgs bundles and defines a connected component of ${\mathcal M}(\mathrm {G}).$

Remark 2.19

For a split real group $\mathrm {G}$, a connected component of ${\mathcal M}(\mathrm {G})$ described by Theorem 2.18 is called a Hitchin component. Since the Hitchin components are smooth, the automorphism group of a Higgs bundle in such a component is as small as possible. For $\mathrm {O}(p,p-1)$, it is given by $\pm ({{\,\mathrm{Id}\,}}_V,{{\,\mathrm{Id}\,}}_W).$

We now describe an explicit construction of a section of (2.10) for the group $\mathrm {G}^\mathbb {C}=\mathrm {O}(2p-1,\mathbb {C})$. This construction will be used in Sect. 4. We will construct one such section $s_H^I$ for each choice of a holomorphic line bundle I with $I^2\cong {\mathcal O}$. In this case, the rank is $p-1$, the integers $m_j+1$ equal to 2j and the split real form is isomorphic to $\mathrm {O}(p,p-1)$. Therefore the Hitchin section is given by

$$\begin{aligned} s_H^I:\bigoplus \limits _{j=1}^{p-1}H^0(K^{2j})\rightarrow {\mathcal M}(\mathrm {O}(2p-1,\mathbb {C})). \end{aligned}$$

For each n, consider the holomorphic orthogonal bundle

(2.11)

For $(q_2,\ldots ,q_{2p-2})\in \bigoplus \limits _{j=1}^{p-1}H^0(K^{2j})$, the $\mathrm {O}(p,p-1)$-Higgs bundle $(V,Q_V,W,Q_W,\eta )$ in the image of a Hitchin section $s^I_H$ is given by

$$\begin{aligned} s^I_H(q_2,\ldots ,q_{2p-2})=(I\otimes {\mathcal K}_{p-1},Q_{p-1},I\otimes {\mathcal K}_{p-2},Q_{p-2},\eta (q_2,\ldots ,q_{2p-2})), \end{aligned}$$

(2.12)

where $\eta (q_2,\ldots ,q_{2p-2})$ depends on a choice of the basis of invariant polynomials. Notice that, in particular, the holomorphic structures on $V=I\otimes {\mathcal K}_{p-1}$ and $W=I\otimes {\mathcal K}_{p-2}$ are fixed. One choice for $\eta (q_2,\ldots ,q_{2p-2})$ is given by

(2.13)

For example, when $p=3$ we have

$$\begin{aligned}&(V,Q_V,W,Q_W,\eta (q_2,q_4))\\&\quad =\Big (IK^2\oplus I\oplus IK^{-2},\left( {\begin{matrix}&{}&{}1\\ {} &{}1&{}\\ 1\end{matrix}}\right) , IK\oplus IK^{-1},\left( {\begin{matrix}&{}1\\ 1\end{matrix}}\right) ,\left( {\begin{matrix}q_2&{}q_4\\ 1&{}q_2\\ 0&{}1\end{matrix}}\right) \Big ). \end{aligned}$$

If $(E,Q,\Phi )$ is the associated $\mathrm {O}(5,\mathbb {C})$-Higgs bundle from (2.2), then ${{\,\mathrm{tr}\,}}(\Phi ^2)=8q_2$ and ${{\,\mathrm{tr}\,}}(\Phi ^4)=20q_2^2+8q_4.$ So the above description describes the Hitchin section for the basis $p_1(\Phi )=\frac{1}{8}{{\,\mathrm{tr}\,}}(\Phi ^2)$ and $p_2=\frac{1}{8}{{\,\mathrm{tr}\,}}(\Phi ^4)-\frac{20}{64}({{\,\mathrm{tr}\,}}(\Phi ^2))^2$.

2.6 Topological invariants

Since $\mathrm {H}^\mathbb {C}$ and $\mathrm {G}$ are both homotopy equivalent to $\mathrm {H},$ the set of equivalence classes of topological $\mathrm {H}^\mathbb {C}$-bundles on X is the same as the set of equivalence classes of topological $\mathrm {G}$-bundles on X. Denote this set by $\mathrm {Bun}_X(\mathrm {G})$. This gives a decomposition of the Higgs bundle moduli space,

$$\begin{aligned} {\mathcal M}_L(\mathrm {G})=\coprod \limits _{a\in \mathrm {Bun}_X(\mathrm {G})}{\mathcal M}_L^a(\mathrm {G}), \end{aligned}$$

where $a\in \mathrm {Bun}_X(\mathrm {G})$ is the topological type of the underlying $\mathrm {H}^\mathbb {C}$-bundle of the Higgs bundles in ${\mathcal M}_L^a(\mathrm {G}).$

In general, the number of connected components of the moduli space of $\mathrm {G}$-Higgs has not been established. However, when $\mathrm {G}$ is compact and semisimple, the spaces ${\mathcal M}^a(\mathrm {G})$ are connected and nonempty [41]. Using Example 2.3, this implies the following proposition.

Proposition 2.20

If $\mathrm {G}$ is a connected real semisimple Lie group such that the maximal compact subgroup $\mathrm {H}$ is semisimple, then, for each $a\in \mathrm {Bun}_X(\mathrm {G}),$ the space ${\mathcal M}^{a}(\mathrm {G})$ is nonempty. Moreover, each space ${\mathcal M}^a(\mathrm {G})$ contains a unique connected component with the property that every Higgs bundle in it can be deformed to a Higgs bundle with zero Higgs field.

The above proposition implies that, when $\mathrm {G}$ is a semisimple complex Lie group, the space ${\mathcal M}^a(\mathrm {G})$ is nonempty for each $a\in \mathrm {Bun}_X(\mathrm {G})$. In fact, each of the spaces ${\mathcal M}^a(\mathrm {G})$ is connected. This was proven for connected groups by Li [38] and in general in [25]. In particular, we have the following:

Corollary 2.21

If $\mathrm {G}$ is a semisimple complex Lie group, then every Higgs bundle $({\mathcal E},\varphi )\in {\mathcal M}(\mathrm {G})$ can be deformed to a Higgs bundle with vanishing Higgs field. In particular,

$$\begin{aligned} |\pi _0({\mathcal M}(\mathrm {G}))|=|\mathrm {Bun}_X(\mathrm {G})|. \end{aligned}$$

A semisimple Lie group $\mathrm {G}$ whose maximal compact subgroup is not semisimple but only reductive is called a group of Hermitian type. We will discuss this case in more detail in Sect. 6.2.

We have $\mathrm {O}(1)\cong \mathbb {Z}_2$ and $\mathrm {O}(1)$-bundles are classified by their first Stiefel–Whitney class $sw_1\in H^1(X,\mathbb {Z}_2).$ For $p\geqslant 2$, topological $\mathrm {O}(p)$-bundles have two characteristic classes, a first Stiefel–Whitney class and a second Stiefel–Whitney class $sw_2\in H^2(X,\mathbb {Z}_2).$ When the first Stiefel–Whitney class vanishes, the structure group can be reduced to $\mathrm {SO}(p).$ Since $\mathrm {SO}(2)$ is a circle, the second Stiefel–Whitney class of an $\mathrm {O}(2)$-bundle lifts to the degree of a circle bundle when $sw_1=0.$ However, as an $\mathrm {O}(2)$-bundle, it is only the absolute value of the degree which is a topological invariant. For $p>2,$ the first and second Steifel–Whitney classes classify topological $\mathrm {O}(p)$-bundles over X, while the $\mathrm {SO}(p)$-bundles are classified topologically just by $sw_2$.

We will be particularly interested in the case of $K^p$-twisted $\mathrm {SO}(1,n)$-Higgs bundles and K-twisted $\mathrm {SO}(p,q)$-Higgs bundles. Since the maximal compact subgroup of $\mathrm {SO}(p,q)$ is $\mathrm {S}(\mathrm {O}(p)\times \mathrm {O}(q))$, the Higgs bundles are determined by two orthogonal bundles which have the same first Stiefel–Whitney class. Let ${\mathcal M}^{a,b,c}_L(\mathrm {SO}(p,q))$ denote the subset of $\mathrm {SO}(p,q)$-Higgs bundles $(V,Q_V,W,Q_V,\eta )$ so that

These invariants are constant on connected components, thus we have a decomposition

$$\begin{aligned} {\mathcal M}_L(\mathrm {SO}(p,q)) = \coprod {\mathcal M}_L^{a,b,c}(\mathrm {SO}(p,q)). \end{aligned}$$

(2.14)

Note that when $p=1$ the invariant b is zero, while when $q=1$ then $c=0$.

The case of $\mathrm {SO}(2,q)$ with vanishing first Stiefel–Whitney class behaves differently. Let $(V,W,\eta )$ be a polystable $K^p$-twisted $\mathrm {SO}(2,q)$-Higgs bundle with $sw_1(V)=0$. Then there is a line bundle N so that the $\mathrm {SO}(2,\mathbb {C})$-bundle $(V,Q_V)$ is isomorphic to

$$\begin{aligned} (V,Q_V)\cong (N\oplus N^{-1}, \left( {\begin{matrix}0&{}1\\ 1&{}0\end{matrix}}\right) ). \end{aligned}$$

(2.15)

With respect to this splitting, the Higgs field $\eta :W\rightarrow V\otimes K^p$ decomposes as

$$\begin{aligned} \eta =\left( {\begin{matrix}\gamma \\ \beta \end{matrix}}\right) :W\rightarrow (N\oplus N^{-1})\otimes K^p. \end{aligned}$$

3 The $\mathbb {C}^*$-action and its fixed points

In this section we recall the definition of the $\mathbb {C}^*$-action on the Higgs bundle moduli space and discuss its importance for the study of the connected components of the moduli space of $\mathrm {G}$-Higgs bundles. This method was pioneered by Hitchin [34, 35] using gauge theoretic methods. For completeness we have included in Appendix A a brief review of some essential facts coming from the gauge theoretic approach and how they translate into the language of holomorphic geometry used in the main body of the paper.

3.1 Definition and basic properties of the action

The action of $\mathbb {C}^*$ on the L-twisted Higgs bundle moduli space is defined by scaling the Higgs field. Namely, $\lambda \cdot ({\mathcal E},\varphi )=({\mathcal E},\lambda \varphi )$ for $\lambda \in \mathbb {C}^*$. Since this preserves the notions of (poly)stability, it induces a holomorphic action on the moduli space. By properness of the Hitchin fibration, if $({\mathcal E},\varphi )$ is the isomorphism class of a polystable L-twisted $\mathrm {G}$-Higgs bundle, then the limit $\lim \limits _{\lambda \rightarrow 0}({\mathcal E},\lambda \varphi )$ exists and is a polystable fixed point of the $\mathbb {C}^*$-action [44].

Notation 3.1

Note that we have denoted the isomorphism class of a Higgs bundle and the Higgs bundle itself with the same symbol. The context will always clarify which object we are referring to.

Consider the function on the moduli space of $\mathrm {G}$-Higgs bundles which assigns the $\mathrm {L}^2$-norm of the Higgs field with respect to the harmonic metric solving the self-duality equations (cf. (A.5)):

$$\begin{aligned} f:{\mathcal M}(\mathrm {G})\rightarrow \mathbb {R},\ \ \ ({\mathcal E},\varphi )\mapsto \int _X||\varphi ||^2. \end{aligned}$$

(3.1)

We will refer to the function f as the Hitchin function. Note that f is non-negative and zero if and only if $\varphi =0$. Using Uhlenbeck compactness, Hitchin showed that the map f is proper and hence it attains local minima on each closed subset of ${\mathcal M}(\mathrm {G})$ [34]. In particular, we have

$$\begin{aligned} |\pi _0({\mathcal M}(\mathrm {G}))|\leqslant |\pi _0(\mathrm {Min}({\mathcal M}(\mathrm {G})))|, \end{aligned}$$

where $\mathrm {Min}({\mathcal M}(\mathrm {G}))\subset {\mathcal M}(\mathrm {G})$ denotes the subset where f attains a local minimum.

The starting point for determining the local minima of f is the following result (Lemma A.8):

Proposition 3.2

Let $({\mathcal E},\varphi )$ be a $\mathrm {G}$-Higgs bundle such that $\mathbb {H}^0(C^\bullet ({\mathcal E},\varphi ))=0$ and $\mathbb {H}^2(C^\bullet ({\mathcal E},\varphi ))=0$. If $({\mathcal E},\varphi )$ is a local minimum of f then it is a fixed point of the $\mathbb {C}^*$-action.

In the situation of Proposition 3.2 there is a weight space splitting (see Proposition A.16, (A.6) and also Sect. 3.2 for $\mathrm {G}=\mathrm {SO}(p,q)$) of the Lie algebra bundle ${\mathcal E}[\mathfrak {g}^\mathbb {C}] = {\mathcal E}[\mathfrak {h}^\mathbb {C}] \oplus {\mathcal E}[\mathfrak {m}^\mathbb {C}]$ as

with $\varphi \in H^0({\mathcal E}[\mathfrak {m}^\mathbb {C}]_1\otimes K)$. Thus, the complex $C^\bullet =C^\bullet ({\mathcal E},\varphi )$ defined in (2.4) splits (see (A.7)) as $C^\bullet =\bigoplus C^\bullet _k$, where

(3.2)

yielding corresponding splittings $\mathbb {H}^i(C^\bullet ({\mathcal E},\varphi )) = \bigoplus _k \mathbb {H}^i(C^\bullet _{k}({\mathcal E},\varphi )).$ There is also a corresponding splitting of the long exact sequence in cohomology from (2.5):

(3.3)

We have the following criterion for local minima of f (see Lemma A.19).

Proposition 3.3

Let $({\mathcal E},\varphi )$ be a $\mathrm {G}$-Higgs bundle which is a fixed point of the $\mathbb {C}^*$-action such that $\mathbb {H}^0(C^\bullet ({\mathcal E},\varphi ))=0$ and $\mathbb {H}^2(C^\bullet ({\mathcal E},\varphi ))=0$. Then $({\mathcal E},\varphi )$ is a local minimum of the Hitchin function f if and only if $\mathbb {H}^1(C^\bullet ({\mathcal E},\varphi ))_k=0$ for all $k>0$.

The following criterion for the vanishing in Proposition 3.3 will be useful (see [13, Section 3.4]).

Proposition 3.4

If $({\mathcal E},\varphi )$ is a $\mathrm {G}$-Higgs bundle which is a fixed point of the $\mathbb {C}^*$-action such that $\mathbb {H}^0(C^\bullet )=0$ and $\mathbb {H}^2(C^\bullet )=0$, then $({\mathcal E},\varphi )$ is a local minimum of the Hitchin function f if and only if either $\varphi =0$ or the map (3.2) is an isomorphism of sheaves for every $k>0.$

To classify the local minima of f, the following two results are needed (with proofs given in the Appendix, where they appear as Lemmas A.9 and A.20 respectively).

Proposition 3.5

Let $\mathrm {G}'\subset \mathrm {G}$ be a reductive subgroup. Suppose $({\mathcal E},\varphi )$ is a $\mathrm {G}$-Higgs bundle which reduces to a $\mathrm {G}'$-Higgs bundle. If $({\mathcal E},\varphi )$ is a minimum of the Hitchin function on ${\mathcal M}(\mathrm {G})$ then it is a minimum of the Hitchin function on ${\mathcal M}(\mathrm {G}')$.

Proposition 3.6

Let $({\mathcal E}_0,\varphi _0)\in \mathcal {M}(\mathrm {G})$ be a fixed point of the $\mathbb {C}^*$-action. Suppose there exists a semistable $\mathrm {G}$-Higgs bundle $({\mathcal E},\varphi )$, which is not ${\mathcal S}$-equivalent to $({\mathcal E}_0,\varphi _0)$, and such that $\lim _{t\rightarrow \infty }({\mathcal E},t\varphi )=({\mathcal E}_0,\varphi _0)$ in $\mathcal {M}(\mathrm {G})$. Then $({\mathcal E}_0,\varphi _0)$ is not a local minimum of f.

The following result will help us show the vanishing of $\mathbb {H}^2(C^\bullet )$ for relevant Higgs bundles.

Lemma 3.7

If $({\mathcal E},\varphi )$ is a polystable L-twisted Higgs bundle and $({\mathcal E}',\varphi ')=\lim \limits _{\lambda \rightarrow 0}({\mathcal E},\lambda \varphi )$, then

$$\begin{aligned} \dim \big (\mathbb {H}^2(C^\bullet ({\mathcal E},\varphi ))\big )\leqslant \dim \big (\mathbb {H}^2(C^\bullet ({\mathcal E}',\varphi '))\big ). \end{aligned}$$

Proof

If $({\mathcal E},\varphi )$ is fixed by the $\mathbb {C}^*$-action then we are done. If $({\mathcal E},\varphi )$ is not fixed by $\mathbb {C}^*$, then consider the $\mathbb {C}^*$-family $({\mathcal E},\lambda \varphi ).$ Since $\lim \limits _{\lambda \rightarrow 0}({\mathcal E},\lambda \eta )$ exists, we can extend this to a family over $\mathbb A^1$, hence the result follows by semi-continuity of $\mathbb {H}^2$. $\square$

Example 3.8

The above minima criterion was used in [9] to classify all local minima for the group $\mathrm {GL}(n,\mathbb {R})$, with $n\geqslant 2$, and in [11] for the group $\mathrm {U}(p,q)$ (cf. Definition 2.7). For $\mathrm {U}(p,q)$, all minima $(E,F,\beta ,\gamma )$ have either $\beta =0$ or $\gamma =0$. For $\mathrm {GL}(n,\mathbb {R})$, and $n\geqslant 3$, the only local minima $(E,Q,\Phi )$ with non-zero Higgs field are the ones defining the Hitchin components. More precisely, they are given by

(3.4)

with I a 2-torsion line bundle. If $n=2$, the non-zero local minima are of the form

$$\begin{aligned} E=L\oplus L^{-1}\ \ Q=\left( {\begin{matrix}0&{} 1\\ 1 &{}0\end{matrix}}\right) \text { and }\ \Phi =\left( {\begin{matrix}0&{}0\\ \Phi _1&{}0\end{matrix}}\right) , \end{aligned}$$

(3.5)

with $\Phi _1:L\rightarrow L^{-1}K$ non-zero and $0<\deg (L)\leqslant g-1$.

3.2 $\mathrm {SO}(p,q)$-fixed points

We now focus on the details of fixed points of the $\mathbb {C}^*$-action on the L-twisted $\mathrm {SO}(p,q)$-Higgs bundle moduli space. In order to get a precise picture, the simplest approach is to analyze these directly, following Simpson’s procedure for usual Higgs (vector) bundles [45].

Let $(V,W,\eta )$ be a polystable $\mathrm {SO}(p,q)$-Higgs bundle with $(V,W,\eta )\cong (V,W,\lambda \eta )$ for all $\lambda \in \mathbb {C}^*$. If $\eta \ne 0$, then for each $\lambda$ there are holomorphic orthogonal automorphisms $g_\lambda ^V$ and $g^W_\lambda$ of V and W such that $(g^V_\lambda )^{-1}\cdot \eta \cdot g^W_\lambda =\lambda \eta$. Following Simpson, we take a $\lambda$ which is not a root of unity. If we additionally take $\lambda \in S^1$ we may, using the gauge theoretic machinery of Appendix A, take the automorphisms in the maximal compact subgroup, thus avoiding the generalized eigenspaces considered by Simpson.

Let $V=\bigoplus _{\nu \in \mathbb {R}} V_\nu$ and $W=\bigoplus _{\mu \in \mathbb {R}} W_\mu$ denote the eigenbundle decompositions of $g^V_\lambda$ and $g^W_\lambda$ respectively, so that $g^V_\lambda |_{V_\nu }=\lambda ^\nu \cdot {{\,\mathrm{Id}\,}}_{V_\nu }$ and $g_\lambda ^W|_{W_\mu }=\lambda ^\mu \cdot {{\,\mathrm{Id}\,}}_{W_\mu }.$ Since the gauge transformations $g_\lambda ^V$ and $g_\lambda ^W$ are orthogonal, two eigenbundles $V_{\nu }$ and $V_{\nu '}$ or $W_{\mu }$ and $W_{\mu '}$ are orthogonal if $\nu +\nu '\ne 0$ or $\mu +\mu '\ne 0$. Moreover, the quadratic forms define isomorphisms $V_{\nu }\cong V_{-\nu }^*$ and $W_{\mu }\cong W_{-\mu }^*$.

For all weights $\mu$ and $\nu$, we have $\eta (W_\mu )\subset V_{\mu +1}\otimes L$ and $\eta ^*(V_\nu )\subset W_{\nu +1}\otimes L$. Thus, $\eta =\sum \eta _\mu$ and $\eta ^*=\sum \eta ^*_\nu$, where

(3.6)

We may decompose $V\oplus W$ into a direct sum of minimal unbroken chains of $V_{\nu }$’s and $W_{\mu }$’s connected by non-zero Higgs fields. Consider such a chain

$$\begin{aligned} V_{a} \xrightarrow {\eta ^*_{-a-1}} W_{a+1} \xrightarrow {\eta _{a+1}} \cdots \end{aligned}$$

For simplicity of notation, we have suppressed the twisting by L from the Higgs field. This will be done every time we use these chain representations. We now consider two cases. (Of course similar arguments will apply for chains starting with a $W_\mu$.)

Case 1. Suppose $V_{-a} \cong V_{a}^*$ is among the bundles of the chain. Then $W_{-a-1} \cong W_{a+1}^*$ is also among the bundles of the chain, because the non-zero map $V_{a} \rightarrow W_{a+1}$ is dual to $W_{-a-1} \rightarrow V_{-a}$. Moreover, $V_{-a}$ is evidently the last bundle of the chain. Thus, the weights must be integers and the restriction of the quadratic forms on V and W to the chain is non-degenerate.

Case 2. Suppose now that $V_{-a} \cong V_{a}^*$ is not among the bundles of the chain. Then, arguing in a similar way to case (1), we see that $W_{-a-1}$ cannot be in the chain either. In this case the chain is isotropic for the quadratic forms on V and W. Note that the weights are only well defined up to overall translation on such a chain.

We summarize the above characterization of $\mathbb {C}^*$-fixed points in the following proposition.

Proposition 3.9

If $(V,W,\eta )$ is a polystable L-twisted $\mathrm {SO}(p,q)$-Higgs bundle which is a fixed point of the $\mathbb {C}^*$-action with $\eta \ne 0$, then it is a direct sum of holomorphic chains with non-zero Higgs fields of the following two types:

(3.7)

or

(3.8)

where the corresponding quadratic forms define isomorphisms $V_j\cong (V_{-j})^*$ and $W_j\cong (W_{-j})^*$. The two chains in (3.8) are dual to each other.

Proposition 3.9 provides a characterization of polystable $\mathbb {C}^*$-fixed points with non-vanishing Higgs field. The next result shows that stability imposes further conditions on such fixed points.

Proposition 3.10

Suppose $(p,q)\ne (2,2)$. If $(V,W,\eta )$ is a stable L-twisted $\mathrm {SO}(p,q)$-Higgs bundle which is a $\mathbb {C}^*$-fixed point, then it is represented by a chain of type (3.7).

Proof

Suppose $(V,W,\eta )$ is represented by (3.8). Consider the subbundles $V'\subset V$ and $W'\subset W$ formed by the summands of the first chain. This is a pair of isotropic $\eta$-invariant subbundles (at least one of which is proper because $(p,q)\ne (2,2)$), and the same is true for the pair $V'^*\subset V$ and $W'^*\subset W$ formed by the summands of the second chain. Since $\deg (V')+\deg (W')=-\deg (V'^*)-\deg (W'^*)$, such an $\mathrm {SO}(p,q)$-Higgs bundle is not stable. This argument also shows that if $(V,W,\eta )$ has a summand given by (3.8), then it is not stable. $\square$

3.3 Special fixed points on ${\mathcal M}(\mathrm {SO}(2,q))$

When $p=2$, we have special fixed points of the form

$$\begin{aligned} V_{-1}\xrightarrow {\eta _0^*}W_0\xrightarrow {\eta _0}V_{1}, \end{aligned}$$

(3.9)

where $V_{-1}\cong V_1^*$ and $\eta _0\ne 0$. Note that $\deg (V_1)<0$ by polystability. Also, such a Higgs bundle is of the form (2.15) with either$N=V_1$, $\gamma =\eta _0$ and $\beta =0$, or$N^{-1}=V_1$, $\beta =\eta _0$ and $\gamma =0$. Conversely, an $\mathrm {SO}(2,q)$-Higgs bundle of the form (2.15) with exactly one of $\beta$ or $\gamma$ zero is such a fixed point.

Proposition 3.11

Any $\mathrm {SO}(2,q)$-Higgs bundle $(V,W,\eta )$ which is a fixed point of the $\mathbb {C}^*$-action of the form (3.9) has $sw_1(V)=sw_1(W)=0$ and represents a local minimum of the Hitchin function.

Proof

The vanishing of the first Stiefel–Whitney class is immediate from $V=V_1^*\oplus V_1$. To see that such a fixed point is a minimum, associate to it the $\mathrm {U}(1,q)$-Higgs bundle $(V_1,W_0,\eta _0,0)$. Since a $\mathrm {U}(1,q)$-Higgs bundle with $\gamma =0$ is a minimum of the Hitchin function on its respective moduli space [11] the conclusion follows by Proposition 3.5. $\square$

Fixed points of the $\mathbb {C}^*$-action in ${\mathcal M}(\mathrm {SO}(2,2))$ are particularly easy to describe using (3.7) and (3.8). Let $(V,W,\eta )$ be an $\mathrm {SO}(2,2)$-Higgs bundle. If $sw_1(V)=sw_1(W)\ne 0$, then neither V nor W have holomorphic isotropic subbundles, thus $(V,W,\eta )$ is a fixed point if and only if $\eta =0.$ If $sw_1(V)=sw_1(W)=0,$ then $V=N\oplus N^{-1}$ and $W=M\oplus M^{-1}$ where N and M are isotropic line bundles. Up to switching the roles of N, M, $N^{-1}$ and $M^{-1}$, the holomorphic chains are given by

(3.10)

which are of the form (3.9). Hence, in view of Proposition 3.11, we have the following result.

Proposition 3.12

Every fixed point in ${\mathcal M}(\mathrm {SO}(2,2))$ is a local minimum.

3.4 $\mathrm {SO}(1,n)$-fixed points and local structure of ${\mathcal M}_{K^p}(\mathrm {SO}(1,n))$

Recall from Definition 2.5 that a $K^p$-twisted $\mathrm {SO}(1,n)$-Higgs bundle is a tuple $(I,Q_I,W,Q_W,\omega ,\eta )$.

Note that the isomorphism $(-{{\,\mathrm{Id}\,}}_I\oplus {{\,\mathrm{Id}\,}}_W):I\oplus W\rightarrow I\oplus W$ acts on such a tuple by $(I,Q_I,W,Q_W,\omega ,\eta )\mapsto (I,Q_I,W,Q_W,-\omega ,-\eta )$. In particular, for $\mathbb {C}^*$-fixed points, the isomorphism class is independent of the choice of $\omega .$ This implies that the two choices of orientation define $\mathrm {SO}(1,n)$-Higgs bundles which are in the same connected component. For this reason, we ignore the orientation in this section.

Lemma 3.13

If $(I,W,\eta )$ is a polystable $K^p$-twisted $\mathrm {SO}(1,n)$-Higgs bundle which is a $\mathbb {C}^*$-fixed point with $\eta \ne 0$, then it decomposes as

$$\begin{aligned} (I,W,\eta )\cong \Big (I,W_{-1}\oplus W_0\oplus W_{1},\ \big (\eta _{-1}\ \ \ 0\ \ \ 0\big )\Big ), \end{aligned}$$

where $(W_0,Q_0)$ is a polystable orthogonal bundle and $W_{1}\cong W_{-1}^*.$ Furthermore, $\big (I,W_{-1}\oplus W_1, \big (\eta _{-1}\ \ 0\big )\big )$ is a stable $K^p$-twisted $\mathrm {O}(1,n')$-Higgs bundle which is stable as a $K^p$-twisted $\mathrm {O}(n'+1,\mathbb {C})$-Higgs bundle. In the notation of (3.7), such an $(I,W,\eta )$ is given by the chain

Proof

The first part of the statement follows directly from Proposition 3.9. Since the bundles $W_1$ and $W_{-1}$ are isotropic, if $W_1$ has a degree zero subbundle U, then $W_{-1}$ has $U^*$ as a subbundle contained in the kernel of $\eta _{-1}$ by polystability. We may thus assume that the invariant polystable orthogonal subbundle $U^*\oplus U$ is a summand of $W_0$. Now since $(W_{-1}\oplus W_{1},I,\left( {\begin{matrix}\eta _{-1}&0\end{matrix}}\right) )$ is a stable $\mathrm {O}(1,n')$-Higgs bundle, the associated $\mathrm {O}(n'+1,\mathbb {C})$-Higgs bundle is stable by [2, Proposition 2.7]. $\square$

As in (3.2) with K replaced with $K^p$, at a $\mathbb {C}^*$-fixed point $(I,W,\eta )\cong (I,W_{-1}\oplus W_0\oplus W_{1},\ (\eta _{-1}\ \ \ 0\ \ \ 0))$ in ${\mathcal M}_{K^p}(\mathrm {SO}(1,n))$ the deformation complex (2.8) splits as $C^\bullet (I,W,\eta ) = \bigoplus C^\bullet _k,$ where

We have $\mathfrak {so}(I)=0$ and ${{\,\mathrm{End}\,}}(W_{-1}\oplus W_0\oplus W_1)=\bigoplus \limits _{j=-2}^2{{\,\mathrm{End}\,}}_{j}(W)$, where

This gives the grading on $\mathfrak {so}(W)=\bigoplus \limits _{j=-2}^{2}\mathfrak {so}_j(W)$, where

Notice that $\mathfrak {so}_0(W)\cong \mathfrak {so}(W_0)\oplus {{\,\mathrm{End}\,}}(W_{-1})$, where $\mathfrak {so}(W_0)$ is the bundle of skew-symmetric endomorphisms of $W_0$ with respect to $Q_0$. Also, ${{\,\mathrm{Hom}\,}}(W,I)\otimes K^p={{\,\mathrm{Hom}\,}}_{-1}(W,I)\otimes K^p\oplus {{\,\mathrm{Hom}\,}}_0(W,I)\otimes K^p\oplus {{\,\mathrm{Hom}\,}}_1(W,I)\otimes K^p$, where

For each $k=-2,\ldots , 2$, the above splittings give ${{\,\mathrm{ad}\,}}_\eta :\mathfrak {so}_k(W)\rightarrow {{\,\mathrm{Hom}\,}}_{k+1}(W,I)\otimes K^p$, where ${{\,\mathrm{ad}\,}}_\eta$ is defined by composing with $\eta _{-1}$. This yields long exact sequences in cohomology

(3.11)

Lemma 3.14

For $p>1$, if $(I,W,\eta )$ is a polystable $K^p$-twisted $\mathrm {SO}(1,n)$-Higgs bundle, then the second hypercohomology group $\mathbb {H}^2(C^\bullet (I,W,\eta ))$ vanishes.

Proof

By Lemma 3.7, to show that $\mathbb {H}^2(C^\bullet (I,W,\eta ))$ vanishes it suffices to show the vanishing of each graded piece of (3.11) at a fixed point of the $\mathbb {C}^*$-action. Such fixed points are given by Lemma 3.13.

First note that $\mathbb {H}^2(C^\bullet _k)=0$ for $k\geqslant 1$ since ${{\,\mathrm{Hom}\,}}_{k+1}(W,I)=0$ for $k\geqslant 1.$ Stability implies $W_1$ and $W_0$ have no positive degree subbundles, and, by Serre duality, we have

$$\begin{aligned} H^1({{\,\mathrm{Hom}\,}}_{k+1}(W,IK^p))\cong {\left\{ \begin{array}{ll} H^0({{\,\mathrm{Hom}\,}}(IK^{p-1},W_1))^* &{}k=-2\\ H^0({{\,\mathrm{Hom}\,}}(IK^{p-1}, W_0))^* &{}k=-1. \end{array}\right. } \end{aligned}$$

Thus, since $p>1,$$H^1({{\,\mathrm{Hom}\,}}_{k+1}(W,IK^p))=0$ for $k\leqslant -1.$

Finally, the form of the Higgs field implies the kernel of ${{\,\mathrm{ad}\,}}_\eta :\mathfrak {so}_0(W)\rightarrow {{\,\mathrm{Hom}\,}}_1(W,I)\otimes K^p$ is $\mathfrak {so}( W_0)$. Hence, $\mathbb {H}^2(C^\bullet _0)$ injects into the second hypercohomology group of the stable $\mathrm {O}(1,n')$-Higgs bundle $\big (I,W_{-1}\oplus W_1, \big (\eta _{-1}\ \ 0\big )\big ).$ The associated $\mathrm {O}(n'+1,\mathbb {C})$-Higgs bundle is stable by Lemma 3.13, so this hypercohomology group vanishes by Remark 2.14. $\square$

Lemma 3.15

If $p>1$ and $(I,W,\eta )=\big (I,W_{-1}\oplus W_0\oplus W_1,\big (\eta _{-1}\ \ 0\ \ 0\big )\big )$ is a polystable $K^p$-twisted $\mathrm {SO}(1,n)$-Higgs bundle which is a $\mathbb {C}^*$-fixed point, then

$$\begin{aligned} \mathbb {H}^0(C^\bullet )\cong H^0(\mathfrak {so}(W_0))\ \ \text { and }\ \ \mathbb {H}^1(C^\bullet )=\bigoplus \limits _{k=-2}^2 \mathbb {H}^1(C^\bullet _k). \end{aligned}$$

Moreover,

• $\mathbb {H}^1(C^\bullet _2)\cong H^1(\mathfrak {so}_{2}(W))$,

• $\mathbb {H}^1(C^\bullet _1)\cong H^1({{\,\mathrm{Hom}\,}}(W_{-1},W_0))$,

• $\mathbb {H}^1(C^\bullet _0)\cong H^1(\mathfrak {so}(W_0))\oplus \mathbb {H}^1_0$, where $\mathbb {H}^1_0$ is defined by the sequence

  

• $\mathbb {H}^1(C^\bullet _{-1})$ is defined by the sequence

  

• $\mathbb {H}^1(C^\bullet _{-2})$ is defined by the sequence

  

Proof

By Lemma 3.13, a $\mathbb {C}^*$-fixed point is given by $(I,W,\eta )=\big (I,W_{-1}\oplus W_0\oplus W_1,\big (\eta _{-1}\ \ 0\ \ 0\big )\big )$, where $W_0$ is a polystable orthogonal bundle and $(I,W_{-1}\oplus W_1,\left( {\begin{matrix}\eta _{-1}&0\end{matrix}}\right) )$ is a stable $\mathrm {O}(1,n')$-Higgs bundle such that the associated $\mathrm {O}(n'+1,\mathbb {C})$-Higgs bundle is also stable. In particular, $W_1$ has no non-negative degree subbundles and $W_0$ has no positive degree subbundles. Recall that in the proof of Lemma 3.14 it was shown that $H^1({{\,\mathrm{Hom}\,}}_{k+1}(W,IK^p))=0$ for $k\leqslant -1.$

For $k=2,$ we have $C_2^\bullet : \mathfrak {so}_2(W)\rightarrow 0$, thus, $\mathbb {H}^0(C_2^\bullet )=H^0(\mathfrak {so}_2(W))$ and $\mathbb {H}^1(C_2^\bullet )=H^1(\mathfrak {so}_2(W))$. In particular, $\mathbb {H}^0(C_2^\bullet )$ injects into the zeroth hypercohomology group of the deformation complex of the $\mathrm {O}(1,n')$-Higgs bundle $(I,W_{-1}\oplus W_1,\left( {\begin{matrix}\eta _{-1}&0\end{matrix}}\right) ),$ which vanishes by stability.

For $k=1,$$\mathfrak {so}_1(W)\cong {{\,\mathrm{Hom}\,}}(W_{-1},W_0)$ and $C_1^\bullet :\mathfrak {so}_1(W)\rightarrow 0$ imply $\mathbb {H}^0(C_1^\bullet )=H^0({{\,\mathrm{Hom}\,}}(W_{-1},W_0))$ and $\mathbb {H}^1(C_1^\bullet )=H^1({{\,\mathrm{Hom}\,}}(W_{-1},W_0))$. The vanishing of $H^0({{\,\mathrm{Hom}\,}}(W_{-1},W_0))\cong H^0({{\,\mathrm{Hom}\,}}(W_0,W_1))$ follows from stability. Namely, any non-zero homomorphism $f:W_0\rightarrow W_1$ defines a non-negative degree subbundle of $W_1,$ contradicting the stability of $(I,W_{-1}\oplus W_1,\left( {\begin{matrix}\eta _{-1}&0\end{matrix}}\right) )$.

For $k=0$, $C_0^\bullet :\mathfrak {so}_0(W)\rightarrow {{\,\mathrm{Hom}\,}}_1(W,I)\otimes K^p$ is given by

$$\begin{aligned} C_0^\bullet :{{\,\mathrm{End}\,}}(W_{-1})\oplus \mathfrak {so}(W_0)\rightarrow {{\,\mathrm{Hom}\,}}(W_{-1},I)\otimes K^p,\ \ \ \ (\beta _{-1},\beta _0)\mapsto \eta _{-1}\beta _{-1}. \end{aligned}$$

Thus, we can split $C_0^\bullet$ as $C_0^\bullet =C_0^{\bullet ,\prime }\oplus C_0^{\bullet ,\prime \prime }$ with $C_0^{\bullet ,\prime }:{{\,\mathrm{End}\,}}(W_{-1})\xrightarrow {\eta _{-1}}{{\,\mathrm{Hom}\,}}(W_{-1},I)\otimes K^p$ and $C_0^{\bullet ,\prime \prime }:\mathfrak {so}(W_0)\rightarrow 0$. The hypercohomology groups split accordingly, hence

For $C_0^{\bullet ,\prime }$, $\mathbb {H}^0(C_0^{\bullet ,\prime })=0$ by stability of $\big (I,W_{-1}\oplus W_1, \big (\eta _{-1}\ \ 0\big )\big )$. Thus, if $\mathbb {H}^1_0=\mathbb {H}^1(C_0^{\bullet ,\prime })$, we have

For $k=-1,$ we have $H^1({{\,\mathrm{Hom}\,}}_{0}(W,IK^p))=0$ and $C_{-1}^\bullet : {{\,\mathrm{Hom}\,}}(W_0,W_{-1})\xrightarrow {\eta _{-1}}{{\,\mathrm{Hom}\,}}(W_0,I)\otimes K^p$. Thus,

It remains to show that $\mathbb {H}^0(C_{-1}^\bullet )=0$. If N is the kernel of $\eta _{-1}:W_{-1}\rightarrow IK^p$, then $\mathbb {H}^0(C_{-1}^\bullet )\cong H^0({{\,\mathrm{Hom}\,}}(W_0,N))$. If $N=0$ we are done so suppose $N\ne 0$. Stability of $\big (I,W_{-1}\oplus W_1, \big (\eta _{-1}\ \ 0\big )\big )$ implies $\deg (N)<0$ and moreover N has no non-negative degree subbundles. A non-zero section $\beta \in H^0({{\,\mathrm{Hom}\,}}(W_0,N))$ must have a non-trivial kernel since otherwise $\beta (W_0)\subset N$ would define a non-negative degree subbundle. However, this implies that $\deg (\ker (\beta ))>0$, contradicting the polystability of $W_0$. We conclude that $H^0({{\,\mathrm{Hom}\,}}(W_0,N))=0,$ and thus $\mathbb {H}^0(C_{-1}^\bullet )=0$.

Finally consider the case of $C_{-2}^\bullet : \mathfrak {so}_{-2}(W)\xrightarrow {{{\,\mathrm{ad}\,}}_\eta }{{\,\mathrm{Hom}\,}}(W_1,I)\otimes K^p$. As in the case $k=2,$ stability of the $\mathrm {O}(1,n')$-Higgs bundle $(I,W_{-1}\oplus W_1,\left( {\begin{matrix}\eta _{-1}&0\end{matrix}}\right) )$ implies $\mathbb {H}^0(C_{-2}^\bullet )=0$. The group $\mathbb {H}^1(C_{-2}^\bullet )$ is defined by the exact sequence in the statement of the lemma since $H^1({{\,\mathrm{Hom}\,}}(W_1,IK^p))=0$. $\square$

4 Existence of exotic components of ${\mathcal M}(\mathrm {SO}(p,q))$

In this section we will prove the following theorem exhibiting connected components of ${\mathcal M}(\mathrm {SO}(p,q))$ which are not distinguished by primary characteristic classes for $p\geqslant 2$.

Theorem 4.1

Let X be a compact Riemann surface with genus $g\geqslant 2$ and canonical bundle K. Denote the moduli space of $K^p$-twisted $\mathrm {SO}(1,q-p+1)$-Higgs bundles on X by ${\mathcal M}_{K^p}(\mathrm {SO}(1,q-p+1))$ and the moduli space of K-twisted $\mathrm {SO}(p,q)$-Higgs bundles on X by ${\mathcal M}(\mathrm {SO}(p,q)).$ For $1\leqslant p\leqslant q,$ there is a well defined map

(4.1)

which is an isomorphism onto its image and has an open and closed image. Furthermore, if $p\geqslant 2$, then every Higgs bundle in the image of $\Psi$ has a nowhere vanishing Higgs field.

Remark 4.2

As a direct corollary of the above theorem, we have that, for $p>2$,

$$\begin{aligned} \left| \pi _0\big ({\mathcal M}(\mathrm {SO}(p,q))\big )\right| \ \geqslant \ 2^{2g+2}+\left| \pi _0\big ({\mathcal M}_{K^p}(\mathrm {SO}(1,q-p+1))\big )\right| . \end{aligned}$$

In Theorem 6.1 we will show that the above inequality is in fact an equality.

Remark 4.3

The space of holomorphic differentials $H^0(K^{2j})$ can be identified with the moduli space ${\mathcal M}_{K^{2j}}(\mathbb {R}^+)$. This will be used in Sect. 7.3, to interpret Theorem 4.1 as a generalized Cayley correspondence.

4.1 Defining the map $\Psi$

Recall that a $K^p$-twisted $\mathrm {SO}(1,n)$-Higgs bundle is a triple $(I,\widehat{W},\hat{\eta })$, where $\widehat{W}$ is a rank n vector bundle with an orthogonal structure $Q_{\widehat{W}}$, $I\cong \Lambda ^n\widehat{W}$ and $\hat{\eta }\in H^0({{\,\mathrm{Hom}\,}}(\widehat{W},I)\otimes K^p)$.

Let ${\mathcal H}_{K^p}(\mathrm {SO}(1,q-p+1))$ denote the configuration space of all $K^p$-twisted $\mathrm {SO}(1,q-p+1)$-Higgs bundles and let ${\mathcal H}(\mathrm {SO}(p,q))$ denote the configuration space of all $\mathrm {SO}(p,q)$-Higgs bundles. That is, ${\mathcal H}_{K^p}(\mathrm {SO}(1,q-p+1))$ consists of pairs $({\bar{\partial }}_{\widehat{W}},\hat{\eta })$ where ${\bar{\partial }}_{\widehat{W}}$ is a Dolbeault operator on $\widehat{W}$, $\hat{\eta }\in \Omega ^{1,0}({{\,\mathrm{Hom}\,}}(\widehat{W},\Lambda ^{q-p+1}\widehat{W}))$ such that ${\bar{\partial }}_{\widehat{W}}\hat{\eta }=0$ and ${\bar{\partial }}_{\widehat{W}}Q_{\widehat{W}}=0$. The space ${\mathcal H}(\mathrm {SO}(p,q))$ is defined analogously.

Recall that the Hitchin section $s_H^I:\bigoplus \limits _{j=1}^{p-1}H^0(K^{2j})\rightarrow {\mathcal M}(\mathrm {SO}(p,p-1))$ is given by (2.12), and that

Recall that the Higgs field in the image of $s_H^I$ is given by $\eta (q_2,\ldots ,q_{2p-2}):I\otimes {\mathcal K}_{p-2}\rightarrow I\otimes {\mathcal K}_{p-1}\otimes K$, as in (2.13).

Define the map

(4.2)

by

$$\begin{aligned} \widetilde{\Psi }((I, \widehat{W},\hat{\eta }),q_2,\ldots ,q_{2p-2})=\left( I\otimes {\mathcal K}_{p-1},\widehat{W}\oplus I\otimes {\mathcal K}_{p-2},\Big (\eta _{\widehat{W}}\ \ \eta (q_2,\ldots ,q_{2p-2})\Big )\right) \end{aligned}$$

(4.3)

where

It is clear that the map $\widetilde{\Psi }$ is continuous.

Remark 4.4

When defining the map $\widetilde{\Psi },$ we have ignored the orientations of the $\mathrm {SO}(1,n)$ and $\mathrm {SO}(p,q)$-Higgs bundles. An orientation $\hat{\omega }:I\otimes \Lambda ^{q-p+1}\widehat{W}\rightarrow {\mathcal O}$ clearly induces an orientation $\omega :I^{p}\otimes I^{p-1}\otimes \Lambda ^{q-p+1}\widehat{W}\rightarrow {\mathcal O}$ on the image. Moreover, the two choices of orientation will not define different components of the moduli space (see Remark 6.2).

Lemma 4.5

For $(I,\widehat{W},\hat{\eta },q_2,\ldots ,q_{2p-2})\in {\mathcal H}_{K^p}(\mathrm {SO}(1,q-p+1))\times \bigoplus \limits _{j=1}^{p-1}H^0(K^{2j})$, the $\mathrm {SO}(p,q)$-Higgs bundle $\widetilde{\Psi }(I,\widehat{W},\hat{\eta },q_2,\ldots ,q_{2p-2})$ is (poly)stable if and only if the $K^p$-twisted $\mathrm {SO}(1,q-p+1)$-Higgs bundle $(I,\widehat{W},\hat{\eta })$ is (poly)stable.

Proof

Fix $(I,\widehat{W},\hat{\eta },q_2,\ldots ,q_{2p-2})\in {\mathcal H}_{K^p}(\mathrm {SO}(1,q-p+1))\times \bigoplus \limits _{j=1}^{p-1}H^0(K^{2j})$. Recall that an $\mathrm {SO}(p,q)$-Higgs bundle is polystable if and only if the associated $\mathrm {SL}(p+q,\mathbb {C})$-Higgs bundle is polystable. Suppose first that $q_{2j}=0$ for all j. Then the $\mathrm {SL}(p+q,\mathbb {C})$-Higgs bundle associated to the image of $\widetilde{\Psi }(I,\widehat{W},\hat{\eta },0,\ldots ,0)$ is represented by

To check (poly)stability for such a “cyclic” Higgs bundle, it suffices to show that each of the bundles in the above cycle do not contain an invariant destabilizing subbundle (see Proposition 6.3 of [43]). Thus $\widetilde{\Psi }(I,\widehat{W},\hat{\eta },0,\ldots ,0)$ is polystable if and only if there are no destabilizing subbundles of $\widehat{W}$ in the kernel of $\hat{\eta }$, that is, if and only if $(I,\widehat{W},\hat{\eta })$ is polystable. Furthermore, since $\widetilde{\Psi }(I,\widehat{W},\hat{\eta },0,\ldots ,0)$ is strictly polystable if and only if $\widehat{W}$ contains a degree zero isotropic subbundle in the kernel of $\hat{\eta }$, we conclude that $\widetilde{\Psi }(I,\widehat{W},\hat{\eta },0,\ldots ,0)$ is stable if and only if $(I,\widehat{W},\hat{\eta })$ is stable.

Now suppose $(q_2,\ldots ,q_{2p-2})\ne (0,\ldots ,0)$ and let $(V,W,\eta )=\widetilde{\Psi }(I,\widehat{W},\hat{\eta },q_2,\ldots ,q_{2p-2})$ be given by (4.3). For $\lambda \in \mathbb {C}^*,$ consider the following holomorphic orthogonal gauge transformations of V and W

$$\begin{aligned} g_V=\left( {\begin{matrix}\lambda ^{1-p}&{}&{}&{}\\ {} &{}\lambda ^{3-p}&{}&{}\\ &{}&{}\ddots &{}\\ &{}&{}&{}\lambda ^{p-1}\end{matrix}}\right) \ \ \ \text {and}\ \ \ g_W=\left( {\begin{matrix}{{\,\mathrm{Id}\,}}_{\widehat{W}}&{}&{}&{}&{}\\ {} &{}\lambda ^{2-p}&{}&{}&{}\\ &{}&{}\lambda ^{4-p}&{}&{}\\ &{}&{}&{}\ddots &{}\\ &{}&{}&{}&{}\lambda ^{p-2}\end{matrix}}\right) . \end{aligned}$$

Using the description of $s_H^I$ from (2.12) and (2.13), a straightforward computation shows that

$$\begin{aligned} (g_V,g_W)\cdot (V,W,\lambda \eta )=\widetilde{\Psi }(I,\widehat{W},\lambda ^{p}\hat{\eta },\lambda ^2 q_2,\lambda ^4q_4,\ldots ,\lambda ^{2p-2} q_{2p-2}). \end{aligned}$$

(4.4)

Assume $(I,\widehat{W},\hat{\eta })$ is stable. In particular, $(I,\widehat{W},\lambda ^{p}\hat{\eta })$ is a stable $K^p$-twisted $\mathrm {SO}(1,q-p+1)$-Higgs bundle for all $\lambda \in \mathbb {C}^*$. By the above argument, $\widetilde{\Psi }(I,\widehat{W},\lambda ^{p}\hat{\eta },0,\ldots ,0)$ is also stable for all $\lambda \in \mathbb {C}^*.$ Hence, by the continuity of $\widetilde{\Psi }$ and since stability is an open condition (cf. Remark 2.13), there is a neighborhood U of $(0,\ldots ,0)\in \bigoplus \limits _{j=1}^{p-1}H^0(K^{2j})$ such that $\widetilde{\Psi }(I,\widehat{W},\lambda ^p\hat{\eta },\lambda ^2 q_2,\lambda ^4q_4,\ldots ,\lambda ^{2p-2} q_{2p-2})$ is stable for $(\lambda ^2q_2,\ldots ,\lambda ^{2p-2}q_{2p-2})\in U$ i.e. for small $\lambda$. From (4.4), $(V,W,\lambda \eta )$ is stable, and thus, $(V,W,\eta )$ is also stable. This argument is reversible, so $(V,W,\eta )$ is stable if and only if $(I,\widehat{W},\hat{\eta })$ is stable.

Assume now that $(I,\widehat{W},\hat{\eta })$ is strictly polystable. By Proposition 2.17, there is $q'$ satisfying $p-1\leqslant q'<q$, such that

$$\begin{aligned} (I,\widehat{W},\hat{\eta })=\left( \widehat{W}'\oplus \widehat{W}'', \big (\hat{\eta }'\ \ 0\big )\right) , \end{aligned}$$

where $(I,\widehat{W}',\hat{\eta }')$ is a stable $K^p$-twisted $\mathrm {O}(1,q'-p+1)$-Higgs bundle and $\widehat{W}''$ is a polystable orthogonal bundle of rank $q-q'$. In this case, we have

$$\begin{aligned} \widetilde{\Psi }(I,\widehat{W},\hat{\eta },q_2,\ldots ,q_{2p-2})=\left( V, \widehat{W}'\oplus \widehat{W}'',\big (\hat{\eta }' \ \ 0\big )\right) \end{aligned}$$

where

$$\begin{aligned} (V,W',\hat{\eta }')=\widetilde{\Psi }(I,\widehat{W}',\hat{\eta }',q_2,\ldots ,q_{2p-2}), \end{aligned}$$

(4.5)

and the map $\widetilde{\Psi }$ in (4.5) is defined as in (4.2) and (4.3), but with q replaced by $q'$. By the above argument, $\widetilde{\Psi }(I,\widehat{W}',\hat{\eta }',q_2,\ldots ,q_{2p-2})$ is a stable $\mathrm {O}(p,q')$-Higgs bundle. Since $\widehat{W}''$ is a polystable orthogonal bundle, we conclude that $\widetilde{\Psi }(I,\widehat{W},\hat{\eta },q_2,\ldots ,q_{2p-2})$ is a strictly polystable $\mathrm {SO}(p,q)$-Higgs bundle. Again, the argument is reversible, hence the converse also holds. $\square$

The next lemma shows that $\widetilde{\Psi }$ both respects isomorphism classes of the corresponding objects and is injective on such classes.

Lemma 4.6

Two $\mathrm {SO}(p,q)$-Higgs bundles $\widetilde{\Psi }(I,\widehat{W},\hat{\eta },q_2,\ldots ,q_{2p-2})$ and $\widetilde{\Psi }(I',\widehat{W}',\hat{\eta }',q_2',\ldots ,q_{2p-2}')$ are in the same $\mathrm {S}(\mathrm {O}(p,\mathbb {C})\times \mathrm {O}(q,\mathbb {C}))$-gauge orbit if and only if $(I,\widehat{W},\hat{\eta })$ and $(I',\widehat{W}',\hat{\eta }')$ are in the same $\mathrm {S}(\mathrm {O}(1,\mathbb {C})\times \mathrm {O}(q-p+1,\mathbb {C}))$-gauge orbit and $q_{2j}=q_{2j}'$ for all $1\leqslant j\leqslant p-1$. Furthermore, each $\mathrm {S}(\mathrm {O}(1,\mathbb {C})\times \mathrm {O}(q-p+1,\mathbb {C}))$-gauge transformation between $(I,\widehat{W},\eta )$ and $(I',\widehat{W}',\hat{\eta }')$ uniquely determines an $\mathrm {S}(\mathrm {O}(p,\mathbb {C})\times \mathrm {O}(q,\mathbb {C}))$-gauge transformation between the Higgs bundles $\widetilde{\Psi }(I,\widehat{W},\hat{\eta },q_2,\ldots ,q_{2p-2})$ and $\widetilde{\Psi }(I',\widehat{W}',\hat{\eta }',q_2,\ldots ,q_{2p-2}).$

Proof

Let $(I,\widehat{W},\hat{\eta })$ and $(I',\widehat{W}',\hat{\eta }')$ be two points in ${\mathcal H}_{K^p}(\mathrm {SO}(1,q\!-\!p\!+1))$, and $(q_2,\dots ,q_{2p-2})$ and $(q_2',\dots ,q_{2p-2}')$ be two points in $\bigoplus \limits _{j=1}^{p-1}H^0(K^{2j})$. Denote the associated points in the image of the map $\widetilde{\Psi }$ from (4.3) by

and recall that $V=I\otimes {\mathcal K}_{p-1}$ and $W=\widehat{W}\oplus I\otimes {\mathcal K}_{p-2}.$

First suppose $(\det (g_{\widehat{W}}),g_{\widehat{W}})$ is an $\mathrm {S}(\mathrm {O}(1,\mathbb {C})\times \mathrm {O}(q-p+1,\mathbb {C}))$-gauge transformation with

$$\begin{aligned} (\det (g_{\widehat{W}}),g_{\widehat{W}})\cdot (I,\widehat{W},\hat{\eta })=(I',\widehat{W}',\hat{\eta }'). \end{aligned}$$

A straightforward computation shows that the $\mathrm {S}(\mathrm {O}(p,\mathbb {C})\times \mathrm {O}(q,\mathbb {C}))$-gauge transformation

$$\begin{aligned} (g_V,g_W)=\Big (\det (g_{\widehat{W}}){{\,\mathrm{Id}\,}}_V,\left( {\begin{matrix}g_{\widehat{W}}&{}0\\ 0&{}\det (g_{\widehat{W}}){{\,\mathrm{Id}\,}}_{{\mathcal K}_{p-2}})\end{matrix}}\right) \Big ) \end{aligned}$$

(4.6)

acts on $(V,W,\eta )$ as

$$\begin{aligned} (g_V,g_W)\cdot (V,W,\eta )=\widetilde{\Psi }(I',\widehat{W}',\hat{\eta }',q_2,\ldots ,q_{2p-2}). \end{aligned}$$

Thus, if $(I,W,\eta )$ and $(I',W',\eta ')$ are in the same $\mathrm {S}(\mathrm {O}(1,\mathbb {C})\times \mathrm {O}(q-p+1,\mathbb {C}))$-gauge orbit, then $\widetilde{\Psi }(I,W,\eta ,q_2,\dots ,q_{2p-2})$ and $\widetilde{\Psi }(I',W',\eta ',q_2,\dots ,q_{2p-2})$ are in the same $\mathrm {S}(\mathrm {O}(p,\mathbb {C})\times \mathrm {O}(q,\mathbb {C}))$-gauge orbit.

Now suppose $(V,W,\eta )$ and $(V',W',\eta ')$ are in the same $\mathrm {S}(\mathrm {O}(p,\mathbb {C})\times \mathrm {O}(q,\mathbb {C}))$-gauge orbit. The action of $(g_V,g_W)$ on $(V,W,\eta )$ is given by

$$\begin{aligned} (g_V,g_W)\cdot ({\bar{\partial }}_V,{\bar{\partial }}_W,\eta )=(g_V^{-1}{\bar{\partial }}_V g_V\ ,\ g_W^{-1}{\bar{\partial }}_W g_W\ ,\ g_V^{-1}\eta g_W). \end{aligned}$$

With respect to the decompositions $W= \widehat{W}\oplus I\otimes {\mathcal K}_{p-2}$ and $W'= \widehat{W}'\oplus I'\otimes {\mathcal K}_{p-2}$, write

$$\begin{aligned} g_W=\left( {\begin{matrix}g_{\widehat{W}}&{}A\\ B&{}g_{{\mathcal K}_{p-2}}\end{matrix}}\right) \ \ \ \ \ \ \ \ \text {and} \ \ \ \ \ \ \ \eta =\Big (\eta _{\widehat{W}}\ \ \ \eta (q_2,\ldots ,q_{2p-2})\Big ). \end{aligned}$$

The gauge transformation $(g_V,g_W)$ acts on the Higgs field by

$$\begin{aligned}g_V^{-1}\eta g_W=g_V^{-1}\cdot \Big (\eta _{\widehat{W}}g_{\widehat{W}}+\eta (q_2,\dots ,q_{2p-2})B\ \ \ \eta _{\widehat{W}}A+\eta (q_2,\ldots ,q_{2p-2})g_{{\mathcal K}_{p-2}}\Big ), \end{aligned}$$

and hence

$$\begin{aligned}&\Big (\eta _{\widehat{W}}'\ \ \ \eta (q_2',\ldots ,q_{2p-2}')\Big )\nonumber \\&=g_V^{-1}\cdot \Big (\eta _{\widehat{W}}g_{\widehat{W}}+\eta (q_2,\dots ,q_{2p-2})B\ \ \ \eta _{\widehat{W}}A+\eta (q_2,\ldots ,q_{2p-2})g_{{\mathcal K}_{p-2}}\Big ).\nonumber \\ \end{aligned}$$

(4.7)

We now use the description of $\eta (q_2,\dots ,q_{2p-2})$ from (2.13). Since $g_V^{-1}$ is invertible and holomorphic, its matrix representation in the decompositions $V=I\otimes {\mathcal K}_{p-1}$ and $V'=I'\otimes {\mathcal K}_{p-1}$ is upper triangular with non-zero diagonal entries. A straightforward computation, using the form of $\eta (q_2',\ldots ,q_{2p-2}')$ and the fact that $g_V^{-1}\eta _{\widehat{W}}g_{\widehat{W}}$ has the form , shows that $B=0$. By orthogonality of $g_W$ we conclude also that $A=0$, $g_{\widehat{W}}$ is an $Q_{\widehat{W}}$-orthogonal gauge transformation and $g_{{\mathcal K}_{p-2}}$ is a $Q_{{\mathcal K}_{p-2}}$-orthogonal gauge transformation.

We now have $\eta (q_{2}',\ldots ,q_{2p-2}')=g_V^{-1}\eta (q_{2},\dots ,q_{2p-2})g_{{\mathcal K}_{p-2}}$. Since $(I\otimes {\mathcal K}_{p-1},I\otimes {\mathcal K}_{p-2},\eta (q_2,\dots ,q_{2p-2}))$ and $(I'\otimes {\mathcal K}_{p-1},I'\otimes {\mathcal K}_{p-2},\eta (q_2',\dots ,q_{2p-2}'))$ define gauge equivalent Higgs bundle in an $\mathrm {O}(p,p-1)$-Hitchin component, we have $(q_2,\dots ,q_{2p-2})=(q_2',\dots ,q_{2p-2}').$ By Remark 2.19, this implies

$$\begin{aligned} (g_V,g_{{\mathcal K}_{p-2}})=\pm ({{\,\mathrm{Id}\,}}_V,{{\,\mathrm{Id}\,}}_{{\mathcal K}_{p-2}}). \end{aligned}$$

Finally, the determinant of $g_{\widehat{W}}$ determines the above sign since $\det (-{{\,\mathrm{Id}\,}}_V)\det (-{{\,\mathrm{Id}\,}}_{{\mathcal K}_{p-2}})=-1$ and

$$\begin{aligned} 1=\det (g_V)\det (g_W)=\det (g_V)\det (g_{{\mathcal K}_{p-2}})\det (g_{\widehat{W}}). \end{aligned}$$

Thus, the gauge transformation $g_{\widehat{W}}$ uniquely determines $g_{{\mathcal K}_{p-2}}$ and $g_V$. This shows that $(g_V,g_W)$ is given by (4.6), completing the proof. $\square$

As a consequence of the two previous lemmas, we have the following proposition.

Proposition 4.7

The map $\widetilde{\Psi }$ from (4.3) descends to a continuous map of moduli spaces

$$\begin{aligned} \Psi :{\mathcal M}_{K^p}(\mathrm {SO}(1,q-p+1))\times \displaystyle \bigoplus \limits _{j=1}^{p-1}H^0(K^{2j})\longrightarrow {\mathcal M}(\mathrm {SO}(p,q)), \end{aligned}$$

(4.8)

which is a homeomorphism onto its image.

Remark 4.8

From Remark 2.12, one can check that the dimension of ${\mathcal M}_{K^p}(\mathrm {SO}(1,q-p+1))\times \bigoplus \nolimits _{j=1}^{p-1}H^0(K^{2j})$ is the expected dimension of ${\mathcal M}(\mathrm {SO}(p,q)).$ In particular, the map $\Psi$ is open on the smooth locus. Since the spaces ${\mathcal M}(\mathrm {SO}(p,q))$ and ${\mathcal M}_{K^p}(\mathrm {SO}(1,q-p+1))$ are singular, we have to examine the local structures of each space to prove openness of $\Psi$ at singular points.

4.2 Local structure of fixed points in the image of $\Psi$

We will now analyze the local structure of fixed points of the $\mathbb {C}^*$-action in ${\mathcal M}(\mathrm {SO}(p,q))$ which lie in the image of the map $\Psi$. The following lemma follows immediately from Lemma 3.13 and Proposition 4.7.

Lemma 4.9

An $\mathrm {SO}(p,q)$-Higgs bundle $(V,W,\eta )$ in the image of $\Psi$ is a fixed point of the $\mathbb {C}^*$-action if and only if $(V,W,\eta )=\Psi (I,\widehat{W},\hat{\eta },0,\ldots ,0)$, where $(I,\widehat{W},\hat{\eta })$ is a fixed point of the $\mathbb {C}^*$-action in ${\mathcal M}_{K^p}(\mathrm {SO}(1,q-p+1))$. In particular, such a fixed point is given by³

$$\begin{aligned} (I,\widehat{W},\hat{\eta })=\big (I,W_{-p}\oplus W_0'\oplus W_p,\big (\eta _{-p}\ \ \ 0\ \ \ 0\big )\big ), \end{aligned}$$

where $W_0'$ is a polystable orthogonal bundle of rank $q-p+1-2{{\,\mathrm{rk}\,}}(W_p)$ and $\det (W_0')=I$, $W_p$ is either zero or a negative degree vector bundle with no non-negative degree subbundles, $W_{-p}\cong W_{p}^*$ and $\eta _{-p}$ is non-zero if $W_{-p}$ is non-zero. The associated $\mathrm {SO}(p,q)$-Higgs bundle will be represented by

(4.9)

Let $(V,W,\eta )$ be a polystable $\mathrm {SO}(p,q)$-Higgs bundle in the image of $\Psi$ of the form (4.9). This will be fixed until the end of Sect. 4.2. If $W_p$ is zero, some of the considerations below simplify.

We will repeatedly use the following bundle decompositions of V and W from (4.9):

$$\begin{aligned} \begin{gathered} V= V_{1-p}\oplus V_{3-p}\oplus \cdots \oplus V_{p-3}\oplus V_{p-1},\\ W=W_{-p}\oplus W_{2-p}\oplus \cdots \oplus W_0\oplus \cdots \oplus W_{p-2}\oplus W_{p},\\ V_j=IK^{-j}\ \text {for all }j,\ \ \ W_j=IK^{-j}\ \text {if}\ 0<|j|<p,\ \ \ \text { and }\ \ \ W_0={\left\{ \begin{array}{ll}W_0' &{}\text {if } p\text { odd}\\ I\oplus W_0'&{}\text {if } p\text { even.} \end{array}\right. } \end{gathered} \end{aligned}$$

(4.10)

In particular, even though $(V,W,\eta )$ is not assumed to be stable, we get a weight decomposition like (3.2) of the deformation complex (2.8) as $C^\bullet (V,W,\eta ) = \bigoplus C^\bullet _k,$ where

In terms of the above splittings, we have ${{\,\mathrm{End}\,}}(V)=\displaystyle \bigoplus \nolimits _{k=2-2p}^{2p-2}{{\,\mathrm{End}\,}}_k(V)$, where ${{\,\mathrm{End}\,}}_{2k+1}(V)=0$ and

$$\begin{aligned} {{\,\mathrm{End}\,}}_{2k}(V)={\left\{ \begin{array}{ll} \bigoplus \limits _{j=0}^{p-1-k}{{\,\mathrm{Hom}\,}}(V_{1-p+2j},V_{1-p+2j+2k})&{}k\geqslant 0\\ \bigoplus \limits _{j=0}^{p-1+k}{{\,\mathrm{Hom}\,}}(V_{p-1-2j},V_{p-1-2j+2k})&{}k<0. \end{array}\right. } \end{aligned}$$

(4.11)

Similarly, ${{\,\mathrm{End}\,}}(W)=\displaystyle \bigoplus \nolimits _{k=-2p}^{2p}{{\,\mathrm{End}\,}}_k(W)$, where

$$\begin{aligned} {{\,\mathrm{End}\,}}_{2k}(W)={\left\{ \begin{array}{ll} {{\,\mathrm{End}\,}}(W_0)\oplus \bigoplus \limits _{j=0}^p {{\,\mathrm{End}\,}}(W_{p-2j}) &{}k=0 \text { and }p\text { odd}\\ \bigoplus \limits _{j=0}^{p-k}{{\,\mathrm{Hom}\,}}(W_{-p+2j},W_{-p+2j+2k})&{}k>0\text { or }k=0 \text { and }p\text { even}\\ \bigoplus \limits _{j=0}^{p+k}{{\,\mathrm{Hom}\,}}(W_{p-2j},W_{p-2j+2k})&{}k<0 \end{array}\right. } \end{aligned}$$

(4.12)

and

$$\begin{aligned} {{\,\mathrm{End}\,}}_{2k+1}(W)={\left\{ \begin{array}{ll} {{\,\mathrm{Hom}\,}}(W_{-2k-1},W_{0})\oplus {{\,\mathrm{Hom}\,}}(W_0,W_{2k+1})&{}2k+1\leqslant p\text { and }p\text { odd}\\ 0&{}\text {otherwise}. \end{array}\right. } \end{aligned}$$

(4.13)

Finally, ${{\,\mathrm{Hom}\,}}(W,V)=\displaystyle \bigoplus \nolimits _{k=1-2p}^{2p-1}{{\,\mathrm{Hom}\,}}_k(W,V)$, where

$$\begin{aligned} {{\,\mathrm{Hom}\,}}_{2k+1}(W,V)={\left\{ \begin{array}{ll} \bigoplus \limits _{j=0}^{p-1-k}{{\,\mathrm{Hom}\,}}(W_{-p+2j},V_{1-p+2j+2k})&{}2k+1\geqslant 0\\ \bigoplus \limits _{j=0}^{p+k}{{\,\mathrm{Hom}\,}}(W_{p-2j},V_{p-2j+1+2k})&{}2k+1<0, \end{array}\right. } \end{aligned}$$

(4.14)

and

$$\begin{aligned} {{\,\mathrm{Hom}\,}}_{2k}(W,V)={\left\{ \begin{array}{ll} {{\,\mathrm{Hom}\,}}(W_{0},V_{2k})&{}1-p\leqslant 2k\leqslant p-1\text { and }p\text { odd}\\ 0&{}\text {otherwise}. \end{array}\right. } \end{aligned}$$

(4.15)

Note that the Higgs field $\eta$ is a holomorphic section of ${{\,\mathrm{Hom}\,}}_1(W,V)\otimes K$.

The Lie algebra bundle $\mathfrak {so}(V)\oplus \mathfrak {so}(W)\subset {{\,\mathrm{End}\,}}(V)\oplus {{\,\mathrm{End}\,}}(W)$ with fiber $\mathfrak {so}(p,\mathbb {C})\oplus \mathfrak {so}(q,\mathbb {C})$ consists of $Q_V$ and $-Q_W$ skew symmetric endomorphisms of V and W respectively. The decompositions (4.11), (4.12) and (4.13) induce the following decomposition of $\mathfrak {so}(V)\oplus \mathfrak {so}(W)\subset {{\,\mathrm{End}\,}}(V)\oplus {{\,\mathrm{End}\,}}(V)$:

$$\begin{aligned} \mathfrak {so}(V)=\bigoplus \limits _{k=2-2p}^{2p-2}\mathfrak {so}_k(V) \text { and } \mathfrak {so}(W)=\bigoplus \limits _{k=-2p}^{2p}\mathfrak {so}_k(W). \end{aligned}$$

Here $\mathfrak {so}_{2k+1}(V)=0$ and, using (4.11),

$$\begin{aligned} \mathfrak {so}_{2k}(V)={\left\{ \begin{array}{ll}\{(\alpha _{1-p},\alpha _{3-p},\ldots ,\alpha _{p-1-2k})\in {{\,\mathrm{End}\,}}_{2k}(V)\ |\ \alpha _i=-\alpha ^*_{-2k-i}\}&{} k\geqslant 0\\ \{(\alpha _{p-1},\alpha _{p-3},\ldots ,\alpha _{1-p-2k})\in {{\,\mathrm{End}\,}}_{2k}(V)\ |\ \alpha _i=-\alpha ^*_{-2k-i}\}&{} k<0, \end{array}\right. } \end{aligned}$$

(4.16)

where the index of each homomorphism corresponds to the index of its domain, i.e.,

$$\begin{aligned} \alpha _i:V_i\rightarrow V_{i+2k}. \end{aligned}$$

For $\mathfrak {so}(W)$, using (4.12) we have

$$\begin{aligned} \mathfrak {so}_{2k}(W)={\left\{ \begin{array}{ll} \{(\beta ',\beta _p,\beta _{p-2},\ldots ,\beta _{-p})\in {{\,\mathrm{End}\,}}_{0}(W) |\ \beta '=-(\beta ')^*,\ \beta _i=-\beta ^*_{-i}\} &{}k=0 \text { and }p\text { odd}\\ \{(\beta _{-p},\beta _{2-p},\ldots ,\beta _{p-2k})\in {{\,\mathrm{End}\,}}_{2k}(W)\ |\ \beta _i=-\beta ^*_{-2k-i}\}&{} k>0\text { or }k=0 \text { and }p\text { even}\\ \{(\beta _p,\beta _{p-2},\ldots ,\beta _{-p+2k})\in {{\,\mathrm{End}\,}}_{2k}(W)\ |\ \beta _i=-\beta ^*_{-2k-i}\}&{} k<0, \end{array}\right. } \end{aligned}$$

(4.17)

where $\beta ':W_0\rightarrow W_0$ and, as above, $\beta _i:W_i\rightarrow W_{i+2k}.$ For odd weights, using (4.13) we have

$$\begin{aligned} \mathfrak {so}_{2k+1}(W)={\left\{ \begin{array}{ll} \{(\beta _{-2k-1},-\beta _{-2k-1}^*)\in {{\,\mathrm{Hom}\,}}(W_{-2k-1},W_{0})\oplus {{\,\mathrm{Hom}\,}}(W_0,W_{2k+1})\}&{}2k+1\leqslant p\text { and }p\text { odd}\\ 0&{}\text {otherwise}. \end{array}\right. } \end{aligned}$$

(4.18)

Since $\eta \in H^0({{\,\mathrm{Hom}\,}}_1(W,V)\otimes K)$, the map ${{\,\mathrm{ad}\,}}_\eta$ restricts to $\mathfrak {so}_k(V)\oplus \mathfrak {so}_k(W)\rightarrow {{\,\mathrm{Hom}\,}}_{k+1}(W,V)\otimes K$, yielding the subcomplex $C^\bullet _k$ of $C^\bullet$ of weight k as in (3.2)

$$\begin{aligned} C^\bullet _k= & {} C^\bullet (V,W,\eta )_k:\mathfrak {so}_k(V)\oplus \mathfrak {so}_k(W)\\&\xrightarrow {{{\,\mathrm{ad}\,}}_\eta }{{\,\mathrm{Hom}\,}}_{k+1}(W,V)\otimes K,\quad (\alpha ,\beta )\mapsto \eta \circ \beta -\alpha \circ \eta . \end{aligned}$$

This gives rise to a splitting of the hypercohomology sequence associated to $C^\bullet$:

(4.19)

For all k,  we will compute $\mathbb {H}^1(C^\bullet _k)$ and show $\mathbb {H}^2(C^\bullet _k)$ vanishes in a series of lemmas. Using (4.10) and the decomposition of ${{\,\mathrm{Hom}\,}}_1(W,V)\otimes K$ from (4.14), we write

$$\begin{aligned} \eta =(\eta _{-p},\eta _{2-p},\ldots ,\eta _{p-2})\in \bigoplus _{j=0}^{p-1}H^0({{\,\mathrm{Hom}\,}}(W_{-p+2j},V_{1-p+2j})\otimes K), \end{aligned}$$

(4.20)

where

$$\begin{aligned} {\left\{ \begin{array}{ll} \eta _{-p}:W_{-p}\rightarrow V_{1-p}\otimes K &{}\text { is defined in Lemma }4.9,\\ \eta _0=\left( {\begin{matrix}1&{}0\end{matrix}}\right) :I\oplus W_0'\rightarrow V_1\otimes K &{}\text { if }p\text { even},\\ \eta _i=1:W_i\rightarrow V_{i+1}\otimes K &{}\text { otherwise}. \end{array}\right. } \end{aligned}$$

(4.21)

Lemma 4.10

The map ${{\,\mathrm{ad}\,}}_\eta :\mathfrak {so}_k(V)\oplus \mathfrak {so}_k(W)\rightarrow {{\,\mathrm{Hom}\,}}_{k+1}(W,V)\otimes K$ is an isomorphism for each positive weight $k\notin \{p,2p\}$. In particular,

Proof

We start by considering the case $C^\bullet _{2k+1}$ with $0<2k+1$ and $2k+1\ne p$. If p is even or $p<2k+1,$ the result is immediate since $\mathfrak {so}_{2k+1}(V)$, $\mathfrak {so}_{2k+1}(W)$ and ${{\,\mathrm{Hom}\,}}_{2k+2}(W,V)\otimes K$ are all zero by (4.13) and (4.15). For p odd and $2k+1< p$, we have $\mathfrak {so}_{2k+1}(V)=0$, $\mathfrak {so}_{2k+1}(W)=\{(\beta _{-2k-1},-\beta _{-2k-1}^*)\in {{\,\mathrm{Hom}\,}}(W_{-2k-1},W_0)\oplus {{\,\mathrm{Hom}\,}}(W_0,W_{2k+1})\}$ and ${{\,\mathrm{Hom}\,}}_{2k+2}(W,V)\otimes K={{\,\mathrm{Hom}\,}}(W_0,V_{2k+2})\otimes K$. Using (4.20), the map ${{\,\mathrm{ad}\,}}_\eta$ is the isomorphism sending $\beta _{-2k-1}$ to the composition of $-\beta _{-2k-1}^*$ with $1=\eta _{2k+1}$:

Now consider the case $C^\bullet _{2k}$ with $0<2k$ and $2k\notin \{p,2p\}$. We first show $\mathfrak {so}_{2k}(V)\oplus \mathfrak {so}_{2k}(W)$ and ${{\,\mathrm{Hom}\,}}_{2k+1}(W,V)\otimes K$ are isomorphic. Using (4.10) and (4.14), we have

$$\begin{aligned}&{{\,\mathrm{Hom}\,}}_{2k+1}(W,V)\otimes K\nonumber \\&\quad \cong {\left\{ \begin{array}{ll} {{\,\mathrm{Hom}\,}}(W_{-p},IK^{p-2k})\oplus \underbrace{K^{-2k}\oplus \cdots \oplus K^{-2k}}_{p-k-1\text { times}} &{}{2k> p\text { or }p\text { odd,}}\\ {{\,\mathrm{Hom}\,}}(W_{-p},IK^{p-2k})\oplus {{\,\mathrm{Hom}\,}}(W_0',IK^{-2k})\oplus \underbrace{K^{-2k}\oplus \cdots \oplus K^{-2k}}_{p-k-1\text { times}} &{}\text {otherwise}. \end{array}\right. } \end{aligned}$$

(4.22)

On the other hand, by (4.16) and since the weight is positive, we have

$$\begin{aligned} \mathfrak {so}_{2k}(V)\cong \bigoplus \limits _{j=0}^{\left\lfloor \frac{p-k}{2}\right\rfloor -1} {{\,\mathrm{Hom}\,}}(V_{2j-p+1},V_{2j-p+1+2k})\cong \underbrace{K^{-2k}\oplus \cdots \oplus K^{-2k}}_{\left\lfloor \frac{p-k}{2}\right\rfloor \text { times}}. \end{aligned}$$

(4.23)

Similarly, by (4.17), $\mathfrak {so}_{2k}(W)\cong \bigoplus \nolimits _{j=0}^{\left\lfloor \frac{p-k-1}{2}\right\rfloor } {{\,\mathrm{Hom}\,}}(W_{2j-p},W_{2j-p+2k})$, and thus,

$$\begin{aligned} \mathfrak {so}_{2k}(W)\cong {\left\{ \begin{array}{ll} {{\,\mathrm{Hom}\,}}(W_{-p},IK^{p-2k})\oplus \underbrace{K^{-2k}\oplus \cdots \oplus K^{-2k}}_{\left\lfloor \frac{p-k-1}{2}\right\rfloor \text { times}} &{} 2k> p\text { or }p\text { odd}\\ {{\,\mathrm{Hom}\,}}(W_{-p}, IK^{p-2k})\oplus {{\,\mathrm{Hom}\,}}(W_0',IK^{-2k})\oplus \underbrace{K^{-2k}\oplus \cdots \oplus K^{-2k}}_{\left\lfloor \frac{p-k-1}{2}\right\rfloor \text { times}}&{}\text {otherwise}. \end{array}\right. } \end{aligned}$$

(4.24)

From (4.22), (4.23) and (4.24), we see that $\mathfrak {so}_{2k}(V)\oplus \mathfrak {so}_{2k}(W)$ is isomorphic to ${{\,\mathrm{Hom}\,}}_{2k+1}(W,V)\otimes K$.

Now we will show

$$\begin{aligned} C_{2k}^\bullet :\mathfrak {so}_{2k}(V)\oplus \mathfrak {so}_{2k}(W)\xrightarrow {{{\,\mathrm{ad}\,}}_\eta }{{\,\mathrm{Hom}\,}}_{2k+1}(W,V)\otimes K,\quad {{\,\mathrm{ad}\,}}_\eta (\alpha ,\beta )=\eta \circ \beta -\alpha \circ \eta \end{aligned}$$

is an isomorphism. Using the notations of (4.16), (4.17) (for positive weight) and (4.20), if

then

$$\begin{aligned}&{{\,\mathrm{ad}\,}}_\eta (\alpha ,\beta )=(\eta _{-p+2k}\beta _{-p}-\alpha _{1-p}\eta _{-p},\eta _{2-p+2k}\beta _{2-p}\\&\quad -\alpha _{3-p}\eta _{2-p},\ldots ,\eta _{p-2}\beta _{p-2-2k}-\alpha _{p-1-2k}\eta _{p-2-2k}). \end{aligned}$$

First assume $p-k$ is even. In this case we have

For p odd or $2k>p$, we have $\eta _i=1$ for all $i\ne -p$ by (4.21). Hence ${{\,\mathrm{ad}\,}}_\eta (\alpha ,\beta )$ is given by

$$\begin{aligned}&(\beta _{-p}-\alpha _{1-p}\eta _{-p},\beta _{2-p}-\alpha _{3-p},\ldots ,\beta _{-k-2}-\alpha _{-k-1},\alpha _{-k-1}^*,\nonumber \\&\quad -\beta _{-k-2}^*+\alpha _{-k-3}^*,\ldots ,-\beta _{2-p}^*+\alpha _{1-p}^*). \end{aligned}$$

(4.25)

This vanishes if and only if $\alpha$ and $\beta$ are both identically zero, so ${{\,\mathrm{ad}\,}}_\eta$ is an isomorphism. For p even and $2k\leqslant p$, the only difference is that $W_0=I\oplus W_0'$. Therefore, if we write

$$\begin{aligned} \beta _0=\Big (\beta _0^I\ \ \ \beta _0'\Big ):I\oplus W_0'\rightarrow W_{2k}, \end{aligned}$$

then the terms $W_0\rightarrow V_{2k+1}\otimes K$ and $W_{-2k}\rightarrow V_1\otimes K$ of ${{\,\mathrm{ad}\,}}_\eta$ are given by

$$\begin{aligned} \Big (\beta _0^I-\alpha _1\ \ \ \beta _0'\Big ):I\oplus W_0'\rightarrow V_{2k+1}\otimes K\ \ \ \ \ \text {and}\ \ \ \ \ \ \ -\beta _0^{I*}+\alpha _1^*:W_{-2k}\rightarrow V_1\otimes K. \end{aligned}$$

(4.26)

Again, ${{\,\mathrm{ad}\,}}_\eta$ vanishes if and only if $\alpha$ and $\beta$ both vanish, and is therefore an isomorphism.

Now suppose $p-k$ is odd. In this case, (4.16) and (4.17) imply that

For p odd or $2k>p$, $ad_\eta (\alpha ,\beta )$ is given by

$$\begin{aligned}&(\beta _{-p}-\alpha _{1-p}\eta _{-p},\beta _{2-p}-\alpha _{3-p},\ldots ,\beta _{-k-3}-\alpha _{-k-2},\beta _{-k-1},\\&\quad -\beta _{-k-1}^*+\alpha _{-k-2}^*,\ldots ,-\beta _{2-p}^*+\alpha _{1-p}^*). \end{aligned}$$

Since this vanishes if and only if $\alpha$ and $\beta$ both vanish, ${{\,\mathrm{ad}\,}}_\eta$ is an isomorphism. The case of p even and $2k\leqslant p$ follows from a similar calculation as the one done above.

Since $\mathfrak {so}_k(V)\oplus \mathfrak {so}_{k}(W)\xrightarrow {{{\,\mathrm{ad}\,}}_\eta } {{\,\mathrm{Hom}\,}}_{k+1}(W,V)\otimes K$ is an isomorphism for all positive weights k different than p and 2p, we conclude that the hypercohomology groups $\mathbb {H}^*(C^\bullet _k)$ all vanish for such k. $\square$

Next we consider the subcomplexes of weight p and 2p.

Lemma 4.11

The hypercohomology groups $\mathbb {H}^*(C^\bullet _p)$ and $\mathbb {H}^*(C^\bullet _{2p})$ are given by

where $\mathfrak {so}_{2p}(W)=\{\beta \in {{\,\mathrm{Hom}\,}}(W_{-p},W_p)| \beta +\beta ^*=0\}.$

Proof

First note that $\mathfrak {so}_{2p}(V)=0$, $\mathfrak {so}_{2p}(W)=\{\beta \in {{\,\mathrm{Hom}\,}}(W_{-p},W_p)| \beta +\beta ^*=0\}$ and ${{\,\mathrm{Hom}\,}}_{2p+1}(W,V)=0$, hence

If p is odd, then $W_0=W_0',$$\mathfrak {so}_{p}(W)\cong {{\,\mathrm{Hom}\,}}(W_{-p},W_0'),$$\mathfrak {so}_p(V)=0$ and ${{\,\mathrm{Hom}\,}}_{p+1}(W,V)=0$, thus

Moreover, $H^0(\mathfrak {so}_{2p}(W))$ and $H^0({{\,\mathrm{Hom}\,}}(W_{-p},W_0'))$ were shown to vanish in the proof of Lemma 3.15, completing the proof for the case 2p and when p is odd.

Now suppose p is even, then $W_0=I\oplus W_0'$ and, from (4.11), (4.12) and (4.14), we have

$$\begin{aligned} \mathfrak {so}_{p}(V)\cong \underbrace{K^{-p}\oplus \cdots \oplus K^{-p}}_{\left\lfloor \frac{p}{4}\right\rfloor \text { times}}, \end{aligned}$$

$$\begin{aligned} \mathfrak {so}_p(W)\cong {{\,\mathrm{Hom}\,}}(W_{-p},I)\oplus {{\,\mathrm{Hom}\,}}(W_{-p},W_0')\oplus \underbrace{K^{-p}\oplus \cdots \oplus K^{-p}}_{\left\lfloor \frac{p-2}{4}\right\rfloor \text { times}}\oplus {{\,\mathrm{Hom}\,}}(W_0',K^{-p})~ \end{aligned}$$

and

$$\begin{aligned} {{\,\mathrm{Hom}\,}}_{p+1}(W,V)\otimes K\cong {{\,\mathrm{Hom}\,}}(W_{-p},I)\oplus \underbrace{K^{-p}\oplus \cdots \oplus K^{-p}}_{\frac{p}{2}-1\text { times}}. \end{aligned}$$

Thus, $\mathfrak {so}_p(V)\oplus \mathfrak {so}_p(W)\cong {{\,\mathrm{Hom}\,}}(W_{-p},W_0')\oplus {{\,\mathrm{Hom}\,}}_{p+1}(W,V)\otimes K.$

If $\frac{p}{2}$ is even and $(\alpha ,\beta )\in \mathfrak {so}_p(V)\oplus \mathfrak {so}_p(W)$, then

Using the decomposition of $\eta$ from (4.20) and (4.21), we see that ${{\,\mathrm{ad}\,}}(\alpha ,\beta )$ is given by

$$\begin{aligned}&(\eta _0\beta _{-p}-\alpha _{1-p}\eta _{-p},\beta _{2-p}-\alpha _{3-p},\ldots ,\beta _{-\frac{p}{2}-2}-\alpha _{-\frac{p}{2}-1},\alpha _{-\frac{p}{2}-1}^*,-\beta _{-\frac{p}{2}-2}^*\\&\quad +\,\alpha _{-\frac{p}{2}-3}^*,\ldots ,-\beta _{2-p}^*+\alpha _{1-p}^*). \end{aligned}$$

If we write $\beta _{-p}=\left( {\begin{matrix}\beta _{-p}^I\\ \beta _{-p}'\end{matrix}}\right) :W_{-p}\rightarrow I\oplus W_0'$, then $\eta _0\beta _{-p}=\left( {\begin{matrix}1&0\end{matrix}}\right) \left( {\begin{matrix}\beta _{-p}^I\\ \beta _{-p}'\end{matrix}}\right) =\beta _{-p}^I$. Hence ${{\,\mathrm{Hom}\,}}(W_{-p},W_0')$ is in the kernel of ${{\,\mathrm{ad}\,}}_\eta$ and $\eta _0\beta _{-p}-\alpha _{1-p}\eta _{-p}=\beta _{-p}^I-\alpha _{1-p}\eta _{-p}$. We conclude that the map induced by ${{\,\mathrm{ad}\,}}_\eta$ on $(\mathfrak {so}_p(V)\oplus \mathfrak {so}_p(W))/{{\,\mathrm{Hom}\,}}(W_{-p},W_0')\rightarrow {{\,\mathrm{Hom}\,}}_{p+1}(W,V)\otimes K$ is given by

$$\begin{aligned}&{{\,\mathrm{ad}\,}}_\eta :{{\,\mathrm{Hom}\,}}(W_{-p},W_0')\oplus (\mathfrak {so}_p(V)\oplus \mathfrak {so}_p(W))/{{\,\mathrm{Hom}\,}}(W_{-p},W_0')\\&\quad \xrightarrow {\left( {\begin{matrix}0&\delta \end{matrix}}\right) } {{\,\mathrm{Hom}\,}}_{p+1}(W,V)\otimes K \end{aligned}$$

with $\delta$ an isomorphism. In particular, this implies that

Moreover, $H^0({{\,\mathrm{Hom}\,}}(W_{-p},W_0'))$ was shown to vanish in the proof of Lemma 3.15. The proof for $\frac{p}{2}$ odd follows from similar arguments. $\square$

Now we consider negative odd weights different from $-p$.

Lemma 4.12

The map ${{\,\mathrm{ad}\,}}_\eta :\mathfrak {so}_{2k+1}(V)\oplus \mathfrak {so}_{2k+1}(W)\rightarrow {{\,\mathrm{Hom}\,}}_{2k+2}(W,V)\otimes K$ is an isomorphism for $2k+1<0$ and $2k+1\ne -p$. In particular,

Proof

First, note that $\mathfrak {so}_{2k+1}(V)=0$. Also, if p is even or $2k+1<-p$, then $\mathfrak {so}_{2k+1}(W)=0$ and ${{\,\mathrm{Hom}\,}}_{2k+2}(W,V)=0$. For p odd and $2k+1>-p$,

$$\begin{aligned} \mathfrak {so}_{2k+1}(W)=\{(\beta _{-2k-1},-\beta _{-2k-1}^*){\in }{{\,\mathrm{Hom}\,}}(W_{-2k-1},W_0){\oplus }{{\,\mathrm{Hom}\,}}(W_0,W_{2k+1})\} \end{aligned}$$

and ${{\,\mathrm{Hom}\,}}_{2k+2}(W,V)\otimes K={{\,\mathrm{Hom}\,}}(W_0,V_{2k+2})\otimes K$. Moreover, ${{\,\mathrm{ad}\,}}_\eta :\mathfrak {so}_{2k+1}(W)\rightarrow {{\,\mathrm{Hom}\,}}_{2k+2}(W,V)\otimes K$ is given by

which is an isomorphism. $\square$

Next we deal with negative even weights different from $-p$ and $-2p$.

Lemma 4.13

For $2k<0$ and $2k\notin \{-p,-2p\}$, ${{\,\mathrm{Hom}\,}}_{2k+1}(W,V)\otimes K\cong \mathfrak {so}_{2k}(W)\oplus \mathfrak {so}_{2k}(V)\oplus K^{-2k}$ and ${{\,\mathrm{ad}\,}}_\eta$ decomposes as

$$\begin{aligned} {{\,\mathrm{ad}\,}}_\eta =\left( {\begin{matrix}a\\ b\end{matrix}}\right) :\mathfrak {so}_{2k}(W)\oplus \mathfrak {so}_{2k}(V)\rightarrow \big (\mathfrak {so}_{2k}(W)\oplus \mathfrak {so}_{2k}(V)\big )\oplus K^{-2k}, \end{aligned}$$

where a is an isomorphism. In particular,

Proof

Using (4.14), we have that

$$\begin{aligned}&{{\,\mathrm{Hom}\,}}_{2k+1}(W,V)\otimes K\\&\quad \cong {\left\{ \begin{array}{ll} {{\,\mathrm{Hom}\,}}(W_p, IK^{-p-2k})\oplus \underbrace{K^{-2k}\oplus \cdots \oplus K^{-2k}}_{p+k \text { times}}&{}{\text {if }p\text { odd or }2k<-p}\\ {{\,\mathrm{Hom}\,}}(W_p, IK^{-p-2k})\oplus {{\,\mathrm{Hom}\,}}(W_0',IK^{-2k})\oplus \underbrace{K^{-2k}\oplus \cdots \oplus K^{-2k}}_{p+k \text { times}}&{}\text {otherwise.} \end{array}\right. } \end{aligned}$$

If $p+k$ is even, then by (4.16) and (4.17) we have

$$\begin{aligned}&\mathfrak {so}_{2k}(V)\cong \Big \{(\alpha _{p-1},\ldots ,\alpha _{-k+1},-\alpha _{-k+1}^*,\ldots ,-\alpha _{p-1}^*)\in \\&\quad \bigoplus \limits _{j=0}^{p-1+k}{{\,\mathrm{Hom}\,}}(V_{p-1-2j},V_{p-1-2j+2k})\Big \} \end{aligned}$$

$$\begin{aligned}&\mathfrak {so}_{2k}(W)\cong \Big \{(\beta _p,\ldots ,\beta _{-k+2},0,-\beta _{-k+2}^*,\ldots ,-\beta _p^*)\in \\&\quad \bigoplus \limits _{j=0}^{p+k}{{\,\mathrm{Hom}\,}}(W_{p-2j},W_{p-2j+2k})\Big \}. \end{aligned}$$

Thus,

$$\begin{aligned} \mathfrak {so}_{2k}(V)\cong \underbrace{K^{-2k}\oplus \cdots \oplus K^{-2k}}_{\frac{p+k}{2}\text { times}} \end{aligned}$$

and

$$\begin{aligned} \mathfrak {so}_{2k}(W)\cong {\left\{ \begin{array}{ll} {{\,\mathrm{Hom}\,}}(W_p,IK^{-p-2k})\oplus \underbrace{K^{-2k}\oplus \cdots \oplus K^{-2k}}_{\frac{p+k}{2}-1\text { times}}&{}\text {if }p\text { odd or }2k<-p\\ {{\,\mathrm{Hom}\,}}(W_p,IK^{-p-2k})\oplus {{\,\mathrm{Hom}\,}}(W_0',IK^{-2k})\oplus \underbrace{K^{-2k}\oplus \cdots \oplus K^{-2k}}_{\frac{p+k}{2}-1\text { times}}&{}\text {otherwise.} \end{array}\right. } \end{aligned}$$

Hence we conclude that ${{\,\mathrm{Hom}\,}}_{2k+1}(W,V)\otimes K\cong \mathfrak {so}_{2k}(W)\oplus \mathfrak {so}_{2k}(V)\oplus K^{-2k}$. By a similar argument, the conclusion also holds for the case $p+k$ odd.

For the form of ${{\,\mathrm{ad}\,}}_\eta$ in this splitting, first assume p is odd or $2k<-p.$ If $p+k$ is even, then the map ${{\,\mathrm{ad}\,}}_\eta :\mathfrak {so}_{2k}(V)\oplus \mathfrak {so}_{2k}(W)\rightarrow {{\,\mathrm{Hom}\,}}_{2k+1}(W,V)\otimes K$ is given by

$$\begin{aligned} \begin{aligned}&{{\,\mathrm{ad}\,}}_\eta (\alpha ,\beta )=(\beta _p,\beta _{p-2}-\alpha _{p-1},\ldots ,\beta _{-k+2}-\alpha _{-k+3},\\&-\alpha _{-k+1},-\beta _{-k+2}^* +\alpha _{-k+1}^*,\ldots ,\alpha _{p-1}^*-\eta _{-p}\beta _p^*). \end{aligned} \end{aligned}$$

(4.27)

Consider the summand $K^{-2k}\cong {{\,\mathrm{Hom}\,}}(W_{-k},V_{k+1})\otimes K$ of ${{\,\mathrm{Hom}\,}}_{2k+1}(W,V)\otimes K$ and take the corresponding quotient $({{\,\mathrm{Hom}\,}}_{2k+1}(W,V)\otimes K)/K^{-2k}$. Then ${{\,\mathrm{Hom}\,}}_{2k+1}(W,V)\otimes K=({{\,\mathrm{Hom}\,}}_{2k+1}(W,V)\otimes K)/K^{-2k}\oplus K^{-2k}$ and, from (4.27), we conclude that ${{\,\mathrm{ad}\,}}_\eta$ can be written as

$$\begin{aligned} {{\,\mathrm{ad}\,}}_\eta =\left( {\begin{matrix}a\\ b\end{matrix}}\right) :\mathfrak {so}_{2k}(V)\oplus \mathfrak {so}_{2k}(W)\rightarrow \big ({{\,\mathrm{Hom}\,}}_{2k+1}(W,V)\otimes K\big )/K^{-2k}\oplus K^{-2k} \end{aligned}$$

where a is an isomorphism. If $p+k$ is odd, a similar conclusion holds.

If p is even and $-p<2k$, the only difference is that we have the following decompositions

$$\begin{aligned} \beta _0=\Big (\beta _0^I\ \ \ \beta _0'\Big ):I\oplus W_0'\rightarrow W_{2k}\ \ \ \ \ \text {and}\ \ \ \ \beta _0^*=\left( {\begin{matrix}(\beta _0^I)^*\\ (\beta _0')^*\end{matrix}}\right) :W_{-2k}\rightarrow I\oplus W_0'. \end{aligned}$$

With these decompositions, the terms of ${{\,\mathrm{ad}\,}}_\eta$ which involve $\beta _0$ and $\beta _0^*$ are given by

(4.28)

The map $I\oplus W_0'\rightarrow V_{2k+1}\otimes K$ is given by $\left( {\begin{matrix}\beta _0^I-\alpha _1\,&\,\beta _0'\end{matrix}}\right)$ and the map $W_{-2k}\rightarrow W_1\otimes K$ is given by $-(\beta _0^I)^*+\alpha _{-1}$. In particular, we have ${{\,\mathrm{ad}\,}}_\eta =\left( {\begin{matrix}a\\ b\end{matrix}}\right) :\mathfrak {so}_{2k}(W)\oplus \mathfrak {so}_{2k}(V)\rightarrow \big (\mathfrak {so}_{2k}(W)\oplus \mathfrak {so}_{2k}(V)\big )\oplus K^{-2k}$ with a an isomorphism.

This implies that in the long exact sequence (4.19), for $2k<0$ and $2k\notin \{-p,-2p\},$ we have

completing the proof. $\square$

The next lemma deals with $\mathbb {H}^*(C^\bullet _{-p})$ and $\mathbb {H}^*(C^\bullet _{-2p})$.

Lemma 4.14

In weight $-2p$ we have $\mathbb {H}^0(C^\bullet _{-2p})=0,$$\mathbb {H}^2(C^\bullet _{2k})=0$ and

(4.29)

where $\mathfrak {so}_{-2p}(W)=\{\beta \in {{\,\mathrm{Hom}\,}}(W_p,W_{-p})|\beta +\beta ^*=0\}.$ For p odd, we have

$$\begin{aligned} \mathbb {H}^0(C^\bullet _{-p})=0,\mathbb {H}^1(C^\bullet _{-p})\cong \mathbb {H}^1_{-p}\text {and}\mathbb {H}^2(C^\bullet _{-p})=0, \end{aligned}$$

where

(4.30)

For p even,

and with respect to this splitting ${{\,\mathrm{ad}\,}}_\eta =\left( {\begin{matrix}0&{}b\\ \eta _{-p}&{}0\\ 0&{}a\end{matrix}}\right)$, where $a:A\rightarrow A$ is an isomorphism. In particular, for p even,

$$\begin{aligned} \mathbb {H}^0(C^\bullet _{-p})=0,\quad \quad \mathbb {H}^1(C^\bullet _{-p})\cong H^0(K^p)\oplus \mathbb {H}^1_{-p} \quad \text {and}\quad \mathbb {H}^2(C^\bullet _{-p})=0. \end{aligned}$$

Proof

For weight $-2p$ we have $\mathfrak {so}_{-2p}(V)=0,$$\mathfrak {so}_{-2p}(W)=\{\beta \in {{\,\mathrm{Hom}\,}}(W_p,W_{-p})|\beta +\beta ^*=0\}$ and ${{\,\mathrm{Hom}\,}}_{-2p+1}(W,V)\otimes K\cong {{\,\mathrm{Hom}\,}}(W_{p},IK^p).$ The map ${{\,\mathrm{ad}\,}}_\eta :\mathfrak {so}_{-2p}(W)\rightarrow {{\,\mathrm{Hom}\,}}(W_{p},K^p)$ is given by ${{\,\mathrm{ad}\,}}_\eta (\beta )=\eta _{-p}\beta$. The result now follows from Lemmas 3.14 and 3.15.

If p is odd, then by (4.18) and (4.15) we have

The map ${{\,\mathrm{ad}\,}}_{\eta }:{{\,\mathrm{Hom}\,}}(W_p,W_0')\rightarrow {{\,\mathrm{Hom}\,}}(W_0',IK^p)$ is given by ${{\,\mathrm{ad}\,}}_\eta (\beta _p)=-\eta _{-p}\beta _p^*$. Again, the result now follows from Lemmas 3.14 and 3.15.

If p is even, then

$$\begin{aligned} \begin{aligned} \mathfrak {so}_{-p}(V)&\cong \underbrace{K^p\oplus \cdots \oplus K^p}_{\left\lfloor \frac{p}{4}\right\rfloor \text { times}},\\ \mathfrak {so}_{-p}(W)&\cong {{\,\mathrm{Hom}\,}}(W_p,I)\oplus {{\,\mathrm{Hom}\,}}(W_p,W_0')\oplus \underbrace{K^p\oplus \cdots \oplus K^p}_{\left\lfloor \frac{p-2}{4}\right\rfloor \text { times}},\\ {{\,\mathrm{Hom}\,}}_{1-p}(W,V)\otimes K&\cong {{\,\mathrm{Hom}\,}}(W_p, I)\oplus {{\,\mathrm{Hom}\,}}(W_0',IK^p)\oplus \underbrace{K^p\oplus \cdots \oplus K^p}_{\frac{p}{2}\text { times}}. \end{aligned} \end{aligned}$$

Setting $A={{\,\mathrm{Hom}\,}}(W_p,I)\oplus \underbrace{K^p\oplus \cdots \oplus K^p}_{\frac{p}{2}-1\ \text {times}}$ we have $\mathfrak {so}_{-p}(V)\oplus \mathfrak {so}_{-p}(W)\cong {{\,\mathrm{Hom}\,}}(W_p,W_0')\oplus A$ and ${{\,\mathrm{Hom}\,}}_{-p}(W,V)\otimes K\cong K^{p}\oplus {{\,\mathrm{Hom}\,}}(W_0',IK^p)\oplus A.$ The map ${{\,\mathrm{ad}\,}}_\eta$ is analogous to the one in the proof of Lemma 4.13 except that (4.28) is given by

Thus, ${{\,\mathrm{ad}\,}}_\eta$ restricted to ${{\,\mathrm{Hom}\,}}(W_p,W_0')$ is given by $\beta _p'\mapsto -\eta _{-p}\beta _p^{\prime *}.$ Hence,

$$\begin{aligned} {{\,\mathrm{ad}\,}}_\eta =\left( {\begin{matrix}0&{}b\\ \eta _{-p}&{}0\\ 0&{}a\end{matrix}}\right) :{{\,\mathrm{Hom}\,}}(W_p,W_0')\oplus A\longrightarrow K^{p}\oplus {{\,\mathrm{Hom}\,}}(W_0',K^p)\oplus A \end{aligned}$$

where $a:A\rightarrow A$ is an isomorphism.

Since $H^1(K^p)=0$, we have $\mathbb {H}^2(C^\bullet _{-p})=0$. As in the odd case, we also find that $\mathbb {H}^0(C^\bullet _{-p})=0$. Moreover, $\mathbb {H}^1(C^\bullet _{-p})\cong H^0(K^p)\oplus \mathbb {H}^1_{-p}$ where $\mathbb {H}^1_{-p}$ is given by (4.30). $\square$

The final case concerns the weight zero subcomplex.

Lemma 4.15

There is a bundle A so that

where $\mathfrak {so}(W_0')$ is the bundle of skew-symmetric endomorphisms of $W_0'$ (with respect to to $Q_{W_0'}$). With respect to this splitting,

where $a:A\rightarrow A$ is an isomorphism. In particular,

where

Proof

By (4.14) we have

$$\begin{aligned} {{\,\mathrm{Hom}\,}}_1(W,V)\otimes K\cong {\left\{ \begin{array}{ll} {{\,\mathrm{Hom}\,}}(W_{-p},IK^p)\oplus \underbrace{{\mathcal O}\oplus \cdots \oplus {\mathcal O}}_{p-1\ \text {times}} &{} p\text { odd}\\ {{\,\mathrm{Hom}\,}}(W_{-p},IK^p)\oplus \underbrace{{\mathcal O}\oplus \cdots \oplus {\mathcal O}}_{p-1\ \text {times}}\oplus {{\,\mathrm{Hom}\,}}(W_0',I)&{} p\text { even} \end{array}\right. } \end{aligned}$$

and by (4.16) and (4.17),

$$\begin{aligned} \mathfrak {so}_0(V)\oplus \mathfrak {so}_0(W)\cong {\left\{ \begin{array}{ll} {{\,\mathrm{End}\,}}(W_{-p}) \oplus \underbrace{{\mathcal O}\oplus \cdots \oplus {\mathcal O}}_{p-1\ \text {times}}\oplus \,\mathfrak {so}(W_0')&{} p\text { odd} \\ {{\,\mathrm{End}\,}}(W_{-p})\oplus \underbrace{{\mathcal O}\oplus \cdots \oplus {\mathcal O}}_{p-1\ \text {times}}\oplus {{\,\mathrm{Hom}\,}}(W_0',I)\oplus \mathfrak {so}(W_0')&{} p\text { even.} \end{array}\right. } \end{aligned}$$

Hence, setting A to be

$$\begin{aligned} A={\left\{ \begin{array}{ll} \underbrace{{\mathcal O}\oplus \cdots \oplus {\mathcal O}}_{p-1\ \text {times}}&{} p\text { odd}\\ \underbrace{{\mathcal O}\oplus \cdots \oplus {\mathcal O}}_{p-1\ \text {times}}\oplus {{\,\mathrm{Hom}\,}}(W_0',I)&{} p\text { even} \end{array}\right. } \end{aligned}$$

yields $\mathfrak {so}_0(V)\oplus \mathfrak {so}_0(W)=\mathfrak {so}(W_0')\oplus {{\,\mathrm{End}\,}}(W_{-p})\oplus A$ and ${{\,\mathrm{Hom}\,}}(W,V)_1\otimes K= {{\,\mathrm{Hom}\,}}(W_{-p},IK^p)\oplus A.$

Since, $W_0'$ is an invariant bundle, the restriction of the map ${{\,\mathrm{ad}\,}}_\eta :\mathfrak {so}_0(W)\oplus \mathfrak {so}_0(V)\rightarrow {{\,\mathrm{Hom}\,}}_1(W,V)\otimes K$ to $\mathfrak {so}(W_0')$ is identically zero. The restriction of the map ${{\,\mathrm{ad}\,}}_\eta$ to ${{\,\mathrm{End}\,}}(W_{-p})\oplus A$ is similar to (4.25) with the exception that the term $W_{-p}\rightarrow V_{1-p}\otimes K$ is given by

In particular, it is given by $\left( {\begin{matrix}\eta _{-p}&{}0\\ b&{}a\end{matrix}}\right) :{{\,\mathrm{End}\,}}(W_{-p})\oplus A\rightarrow {{\,\mathrm{Hom}\,}}(W_{-p},IK^p)\oplus A$ where a is an isomorphism.

The hypercohomology complex for $C^\bullet$ splits as a direct sum of the following two complexes

and

By Lemma 3.13, $\left( W_p\oplus I\oplus W_{-p},\left( {\begin{matrix}0&{}0&{}0\\ \eta _{-p}&{}0&{}0\\ 0&{}\eta _{-p}^*&{}0\end{matrix}}\right) \right)$ is a stable $K^p$-twisted $\mathrm {O}(2{{\,\mathrm{rk}\,}}(W_p)+1,\mathbb {C})$-Higgs bundle, so the hypercohomology groups $\mathbb {H}^0_{0,p}$ and $\mathbb {H}^2_{0,p}$ both vanish and $\mathbb {H}^1(C^\bullet _0)=H^1(\mathfrak {so}(W_0'))\oplus \mathbb {H}^1_{0,p}.$$\square$

4.3 Proof of Theorem 4.1

We are now set up to prove Theorem 4.1. We start by describing a neighborhood of the image of the map $\Psi$ which is open in ${\mathcal M}(\mathrm {SO}(p,q)).$

Proposition 4.16

For each $(I,\widehat{W},\hat{\eta },q_2,\ldots ,q_{2p-2})$ in ${\mathcal M}_{K^p}(\mathrm {SO}(1,q\!-\!p\!+1))\times \bigoplus \limits _{j=1}^{p-1}H^0(K^{2j})$, the second hypercohomology group for the associated $\mathrm {SO}(p,q)$-Higgs bundle vanishes

$$\begin{aligned} \mathbb {H}^2(C^\bullet (\Psi (I,\widehat{W},\hat{\eta },q_2,\ldots ,q_{2p-2})))=0. \end{aligned}$$

In particular, an open neighborhood of $\Psi (I,\widehat{W},\hat{\eta },q_2,\ldots ,q_{2p-2})$ in the moduli space ${\mathcal M}(\mathrm {SO}(p,q))$ is isomorphic to an open neighborhood of zero in

Proof

By Lemma 3.7, it suffices to prove the above proposition at the fixed points of the $\mathbb {C}^*$-action in the image of $\Psi .$ These are the Higgs bundles given in Lemma 4.9. In Lemmas 4.10, 4.11, 4.12, 4.13, 4.14 and 4.15 it is shown that if $(W,V,\eta )$ is a fixed point of the $\mathbb {C}^*$-action in the image of $\Psi$, then each of the graded pieces of $\mathbb {H}^2(C^\bullet (W,V,\eta ))$ vanish. $\square$

Proposition 4.17

For all $\Psi ((I,\widehat{W},\hat{\eta }),0,\ldots ,0)$ which are fixed points of the $\mathbb {C}^*$-action we have an isomorphism induced by $\Psi$.

Proof

Let $\Psi (I, \widehat{W},\hat{\eta },0,\ldots ,0)$ be a fixed point of the $\mathbb {C}^*$-action. For the $\mathrm {SO}(1,q-p+1)$-Higgs bundle $(I,\widehat{W},\hat{\eta }),$ the first hypercohomology group $\mathbb {H}^1(C^\bullet (I,\widehat{W},\hat{\eta }))$ of the deformation complex was computed in Lemma 3.15. In Lemmas 4.10, 4.11, 4.12, 4.13, 4.14 and 4.15 it was shown that the first hypercohomology group of the deformation complex of the $\mathrm {SO}(p,q)$-Higgs bundle is given by

$$\begin{aligned} \mathbb {H}^1(C^\bullet (\Psi (I,\widehat{W},\hat{\eta },0,\ldots ,0)))\cong \mathbb {H}^1(C^\bullet (I,\widehat{W},\hat{\eta }))\times \bigoplus \limits _{j=1}^{p-1}H^0(K^{2j}). \end{aligned}$$

It is clear from our constructions that the isomorphism is induced by $\Psi$.

By Lemma 4.6, every $\mathrm {S}(\mathrm {O}(1,\mathbb {C})\times \mathrm {O}(q-p+1,\mathbb {C}))$ automorphism $(\det (g_{\widehat{W}}),g_{\widehat{W}})$ of $(I,\widehat{W},\hat{\eta })$ determines a unique automorphism of $\Psi (I,\widehat{W},\hat{\eta },0\dots , 0)$

$$\begin{aligned} (g_V,g_W)=\left( \det (g_{\widehat{W}}){{\,\mathrm{Id}\,}}_{{\mathcal K}_{p-1}},\left( {\begin{matrix}g_{\widehat{W}}&{}0\\ 0&{}\det (g_{\widehat{W}}){{\,\mathrm{Id}\,}}_{{\mathcal K}_{p-2}}\end{matrix}}\right) \right) . \end{aligned}$$

Moreover, the action of such an automorphism on the holomorphic differentials in the above description of $\mathbb {H}^1(C^\bullet (\Psi (I,\widehat{W},\hat{\eta },0,\ldots ,0))$ is trivial. Thus,

as claimed. $\square$

Theorem 4.18

The image of the map $\Psi$ from (4.1) is open and closed.

Proof

By Propositions 4.16 and 4.17, the map $\Psi$ is open at all fixed points of the $\mathbb {C}^*$-action. For $(V,W,\eta )$ in the image of $\Psi$, there is $\lambda$ sufficiently close to zero such that $(V,W,\lambda \eta )$ is in a sufficiently small open neighborhood of a fixed point of the $\mathbb {C}^*$-action. Thus, $\Psi$ is open at all points.

To show the image of $\Psi$ is closed, we use the properness of the Hitchin fibration. Namely, suppose $(I,\widehat{W}_i,\hat{\eta }_i,q_2^i,\dots ,q_{2p-2}^i)$ is a sequence of points in ${\mathcal M}_{K^p}(\mathrm {SO}(1,q-p+1))\times \bigoplus \nolimits _{j=1}^{p-1} H^0(K^{2j})$ which diverges. Denote the associated Hitchin fibrations by

By the properness of $h_p$, $(q_2^i,\ldots ,q^i_{2p-2},h_p(I,\widehat{W}_i,\hat{\eta }_i))$ diverges in $\bigoplus \nolimits _{j=1}^pH^0(K^{2j})$. Moreover, by the definition of the map $\Psi ,$ applying the $\mathrm {SO}(p,q)$-Hitchin fibration to the image sequence yields

$$\begin{aligned} h(\Psi (I,\widehat{W}_i,\hat{\eta }_i,q_2^i,\ldots ,q^i_{2p-2}))=(q_2^i,\ldots ,q_{2p-2}^i,h_p(I,\widehat{W}_i,\hat{\eta }_i)). \end{aligned}$$

Since h is proper, we conclude that $\Psi (I,\widehat{W}_i,\hat{\eta }_i,q_2^i,\ldots ,q^i_{2p-2})$ also diverges in ${\mathcal M}(\mathrm {SO}(p,q)).$$\square$

The following direct consequence of the construction of the map $\Psi$ will be used in Sect. 7.

Corollary 4.19

Consider the subgroup $\mathrm {GL}(n,\mathbb {R})\times \mathrm {SO}(p-n,q-n)\subset \mathrm {SO}(p,q)$ defined by the embedding

$$\begin{aligned} (A,B)\mapsto \left( {\begin{matrix}A&{}\\ {} &{}B\\ &{}&{}A^{-1}\end{matrix}}\right) . \end{aligned}$$

Then no Higgs bundle in the image of $\Psi$ reduces to such a subgroup.

5 Classification of local minima of the Hitchin function for ${\mathcal M}(\mathrm {SO}(p,q))$

In this section we will prove Theorem 5.10 which classifies all local minima of the Hitchin function (3.1) on ${\mathcal M}(\mathrm {SO}(p,q))$. The strategy of proof is to divide the objects into the following three families:

• 1. stable $\mathrm {SO}(p,q)$-Higgs bundles with $\mathbb {H}^2(C^\bullet (V,W,\eta ))=0$,

• 2. stable $\mathrm {SO}(p,q)$-Higgs bundles whose corresponding $\mathrm {SO}(p+q,\mathbb {C})$-Higgs bundle is strictly polystable,

• 3. strictly polystable $\mathrm {SO}(p,q)$-Higgs bundles.

The first family consists of points which are either smooth or orbifold points of ${\mathcal M}(\mathrm {SO}(p,q))$. For these points we can use Proposition 3.4 to classify such local minimum. The local minima in the other two families will be described by a direct study of their deformations.

Recall from (3.2) that the deformation complex of an $\mathrm {SO}(p,q)$-Higgs bundle $(V,W,\eta )$ which is a $\mathbb {C}^*$-fixed point decomposes as

$$\begin{aligned} C^\bullet _k:\mathfrak {so}_k(V)\oplus \mathfrak {so}_k(W)\xrightarrow {\ \ {{\,\mathrm{ad}\,}}_\eta \ \ }{{\,\mathrm{Hom}\,}}_{k+1}(W,V)\otimes K. \end{aligned}$$

(5.1)

Each graded piece gives rise to the long exact sequence (3.3) in hypercohomology.

5.1 Stable minima with vanishing $\mathbb {H}^2(C^\bullet )$

By Proposition 3.10, stable $\mathbb {C}^*$-fixed points are given by (3.7). We start by studying the constraints on these chains imposed by the local minima condition for stable $\mathrm {SO}(p,q)$-Higgs bundles with vanishing $\mathbb {H}^2(C^\bullet )$. This will be done by first proving two lemmas.

Lemma 5.1

Let $(V,W,\eta )$ be a stable $\mathrm {SO}(p,q)$-Higgs bundle with $\eta \ne 0$ and $\mathbb {H}^2(C^\bullet (V,W,\eta ))=0$. If $(V,W,\eta )$ is a local minimum of f, then the chain given by (3.7) must have one of the following forms (with $\eta _i\ne 0$ for all i):

(5.2)

(5.3)

(5.4)

(5.5)

Proof

If one of the chains in (3.7) vanishes, we are done. Assume both chains are non-zero chains. Let $r\geqslant 0$ be the maximal weight of the first chain and $s\geqslant 0$ be the maximal weight of the second chain. We have $r>0$ or $s>0$ since $\eta \ne 0$. Since $(V,W,\eta )$ is a stable local minimum of the Hitchin function with $\mathbb {H}^2(C^\bullet )=0$, the subcomplexes from (5.1) are isomorphisms for $k\geqslant 1$ by Proposition 3.4.

If r and s have different parity, then both of the chains start and end with a summand of W if r is odd and start and end with a summand of V if r is even. In either case, ${{\,\mathrm{Hom}\,}}_{r+s+1}(W,V)\otimes K=0$ but $\mathfrak {so}_{r+s}(W)\oplus \mathfrak {so}_{r+s}(V)$ is non-zero. Hence, the subcomplex $C^\bullet _{r+s}$ from (5.1) is not an isomorphism for $k=r+s$, contradicting $(V,W,\eta )$ being a stable minima with $\mathbb {H}^2(C^\bullet )=0.$

Now assume r and s have the same parity, so the first chain starts and ends with a summand of W if and only if r is odd and the second chain starts and ends with a summand of W if only only if s is even. If $r\geqslant s,$ then ${{\,\mathrm{Hom}\,}}_{2r+1}(W,V)\otimes K=0$ and $\mathfrak {so}_{2r}(V)\oplus \mathfrak {so}_{2r}(W)=\Lambda ^2V_r\oplus \Lambda ^2W_r$. So the isomorphism of $C^\bullet _{2r}:\Lambda ^2V_r\oplus \Lambda ^2W_r\rightarrow 0$ implies that, whenever $V_r$ and $W_r$ are non-zero, they must be line bundles; more precisely we must have: (i) if r is odd, ${{\,\mathrm{rk}\,}}(W_r)=1$ and, if $s=r$, ${{\,\mathrm{rk}\,}}(V_r)=1$ (if $s<r$, $V_r=0$), or (ii) if r is even, ${{\,\mathrm{rk}\,}}(V_r)=1$ and, if $s=r$, ${{\,\mathrm{rk}\,}}(W_r)=1$ (if $s<r$, $W_r=0$). Since $r+s-1$ is odd, we have:

$$\begin{aligned} \mathfrak {so}_{r+s-1}(V)={\left\{ \begin{array}{ll}\{(\alpha ,-\alpha ^*)\in {{\,\mathrm{Hom}\,}}(V_{-s},V_{r-1})\oplus {{\,\mathrm{Hom}\,}}(V_{1-r},V_{s})\} &{}\text {if }r\text { is odd}\\ \{(\alpha ,-\alpha ^*)\in {{\,\mathrm{Hom}\,}}(V_{-r},V_{s-1})\oplus {{\,\mathrm{Hom}\,}}(V_{1-s},V_{r})\}&{}\text {if }r\text { is even} \end{array}\right. } \end{aligned}$$

$$\begin{aligned} \mathfrak {so}_{r+s-1}(W)={\left\{ \begin{array}{ll} \{(\beta ,-\beta ^*)\in {{\,\mathrm{Hom}\,}}(W_{-r},W_{s-1})\oplus {{\,\mathrm{Hom}\,}}(W_{1-s},W_{r})\}&{}\text {if }r\text { is odd}\\ \{(\beta ,-\beta ^*)\in {{\,\mathrm{Hom}\,}}(W_{-s},W_{r-1})\oplus {{\,\mathrm{Hom}\,}}(W_{1-r},W_{s})\}&{}\text {if }r\text { is even} \end{array}\right. } \end{aligned}$$

$$\begin{aligned} {{\,\mathrm{Hom}\,}}_{r+s}(W,V)\otimes K\cong {\left\{ \begin{array}{ll} {{\,\mathrm{Hom}\,}}(W_{-r},V_s)\otimes K&{}\text {if }r\text { is odd}\\ {{\,\mathrm{Hom}\,}}(W_{-s},V_r)\otimes K&{}\text {if }r\text { is even.} \end{array}\right. } \end{aligned}$$

If $s>0$, then $r+s-1\geqslant 1$ so the isomorphism $C^\bullet _{r+s-1}:\mathfrak {so}_{r+s-1}(V)\oplus \mathfrak {so}_{r+s-1}(W)\rightarrow {{\,\mathrm{Hom}\,}}_{r+s}(W,V)\otimes K$ gives

$$\begin{aligned} {\left\{ \begin{array}{ll} {{\,\mathrm{rk}\,}}(V_{s}){{\,\mathrm{rk}\,}}(V_{r-1})+{{\,\mathrm{rk}\,}}(W_{s-1})={{\,\mathrm{rk}\,}}(V_s)&{}\text {if }r\text { is odd}\\ {{\,\mathrm{rk}\,}}(W_{s}){{\,\mathrm{rk}\,}}(W_{r-1})+{{\,\mathrm{rk}\,}}(V_{s-1})={{\,\mathrm{rk}\,}}(W_s)&{}\text {if }r\text { is even.} \end{array}\right. }\end{aligned}$$

This implies either ${{\,\mathrm{rk}\,}}(W_{s-1})=0$ or ${{\,\mathrm{rk}\,}}(V_{r-1})=0$ if r is odd, and that either ${{\,\mathrm{rk}\,}}(V_{s-1})=0$ or ${{\,\mathrm{rk}\,}}(W_{r-1})=0$ if r is even. Any of these conclusions contradicts Proposition 3.10. Thus, we conclude that $s=0$ and thus r is even, so the holomorphic chain is given by (5.4). A similar argument shows that the holomorphic chain is of the form (5.5) for $s> r$. $\square$

Lemma 5.2

Let $(V,W,\eta )$ be a stable $\mathrm {SO}(p,q)$-Higgs bundle which is a local minimum of the Hitchin function with $\eta \ne 0$ and $\mathbb {H}^2(C^\bullet (V,W,\eta ))=0$; the associated holomorphic chain is given by (5.2), (5.3), (5.4) or (5.5). For all $j\ne 0,$ we have ${{\,\mathrm{rk}\,}}(W_j)=1$ and ${{\,\mathrm{rk}\,}}(V_j)=1.$ Moreover:

• In case (5.2), $V_j\cong V_{-1} K^{-j-1}$ and $W_j\cong V_{-1}K^{-j-1}$ for $0<|j|<s$.

• In case (5.3), $V_j\cong W_{-1}K^{-j-1}$ and $W_j\cong W_{-1}K^{-j-1}$ for $0<|j|<r.$

• In case (5.4), ${{\,\mathrm{rk}\,}}(V_0)=1$, and $V_j\cong V_0K^{-j}$ and $W_j\cong V_0K^{-j}$ for $0<|j|<r$.

• In case (5.5), ${{\,\mathrm{rk}\,}}(W_0)=1$, and $V_j\cong V_0K^{-j}$ and $W_j\cong V_0K^{-j}$ for $0<|j|<s$.

Proof

The proof involves an inductive argument on the weights. We first consider the case where $(V,W,\eta )$ is the holomorphic chain (5.4). We have the following decompositions

For $2k>0$ we have ${{\,\mathrm{Hom}\,}}_{2k+1}(W,V)=\displaystyle \bigoplus \limits _{j=0}^{r-k-1}{{\,\mathrm{Hom}\,}}(W_{1-r+2j},V_{2-r+2j+2k})$,

(5.6)

With respect to these splittings, $\mathfrak {so}(V)=\displaystyle \bigoplus \mathfrak {so}_k(V)$ and $\mathfrak {so}(W)=\displaystyle \bigoplus \mathfrak {so}_k(W)$ where, for $k>0$

(5.7)

Since $(V,W,\eta )$ is a stable minima of the Hitchin function with $\mathbb {H}^2(C^\bullet )=0,$ for all $k>0$ we have $\mathfrak {so}_{2k}(V)\oplus \mathfrak {so}_{2k}(W)\cong {{\,\mathrm{Hom}\,}}_{2k+1}(W,V)\otimes K$. Note that r is even and non-zero. The isomorphism for $k=2r$ implies $\Lambda ^2V_r\cong 0,$ hence ${{\,\mathrm{rk}\,}}(V_r)=1$.

The isomorphism for $k=2r-2$ implies ${{\,\mathrm{Hom}\,}}(V_{-r},V_{r-2})\oplus \Lambda ^2W_{r-1}\cong {{\,\mathrm{Hom}\,}}(W_{1-r},V_r)\otimes K.$ Thus,

$$\begin{aligned} {{\,\mathrm{rk}\,}}(V_{r-2})+{{\,\mathrm{rk}\,}}(\Lambda ^2W_{1-r})={{\,\mathrm{rk}\,}}(W_{1-r}), \end{aligned}$$

which implies ${{\,\mathrm{rk}\,}}(W_{1-r})$ is either one or two. If ${{\,\mathrm{rk}\,}}(W_{1-r})=2$, taking the determinant of the isomorphism $C^\bullet _{2r-2}$ implies $V_rK^2=V_{r-2}$. Also, the kernels of the maps $\eta _{r-1}:W_{r-1}\rightarrow V_r\otimes K$ and $\eta _{1-r}:W_{1-r}\rightarrow V_{2-r}\otimes K$ have negative degree by stability. Using $V_j^*\cong V_{-j}$ and $W_j^*\cong W_{-j}$, we have

$$\begin{aligned} \deg (V_{r-2})-2g+2<\deg (W_{r-1})<\deg (V_{r})+2g-2, \end{aligned}$$

which contradicts $V_rK^2=V_{r-2}$. So rank $W_{r-1}=1$ and the isomorphism for $C^\bullet _{2r-2}$ gives the base case of our induction:

If $r=2$ we are done, so assume $r\geqslant 4$ and that for an integer $k\in [1,\frac{r}{2}-1]$ we have

$$\begin{aligned} W_{1-r}\cong V_{2-r}K\cong W_{3-r}K^2\cong \cdots \cong W_{2k-1-r}K^{2k-2}\cong V_{2k-r}K^{2k-1}. \end{aligned}$$

(5.8)

We will prove that $V_{2k-r}\cong W_{2k+1-r}K\cong V_{2k+2-r}K^2$.

The isomorphism $C^\bullet _{2r-2-2k}$ gives

(5.9)

since $\Lambda ^2V_{r-k-1}=0$ for k odd and $\Lambda ^2W_{r-k-1}=0$ for k even by (5.8). Using (5.8), computing the ranks of both sides gives ${{\,\mathrm{rk}\,}}(V_{2k+2-r})+\left\lfloor \frac{k}{2}\right\rfloor +{{\,\mathrm{rk}\,}}(W_{2k+1-r})+\left\lfloor \frac{k-1}{2}\right\rfloor =k+{{\,\mathrm{rk}\,}}(W_{2k+1-r})$. Thus,

$$\begin{aligned} {{\,\mathrm{rk}\,}}(V_{2k+2-r})=1. \end{aligned}$$

The isomorphism $C^\bullet _{2r-2-4k}$ implies

$$\begin{aligned}&\displaystyle \bigoplus \limits _{j=0}^k{{\,\mathrm{Hom}\,}}(V_{2j-r},V_{r+2j-2-4k})\oplus \displaystyle \bigoplus \limits _{j=0}^{k-1}{{\,\mathrm{Hom}\,}}(W_{2j+1-r},W_{r+2j-1-4k})\oplus \Lambda ^2W_{r-1-2k}\\&\quad \cong \ \ \displaystyle \bigoplus _{j=0}^{2k}{{\,\mathrm{Hom}\,}}(W_{2j+1-r},V_{r+2j-4k})\otimes K. \end{aligned}$$

Using (5.8), this gives the following equality on ranks

$$\begin{aligned}&\sum \limits _{j=0}^k {{\,\mathrm{rk}\,}}(V_{r+2j-2-4k})+\sum \limits _{j=0}^{k-1}{{\,\mathrm{rk}\,}}(W_{r+2j-1-4k})+{{\,\mathrm{rk}\,}}(\Lambda ^2 W_{r-1-2k})\\&\quad = \sum \limits _{j=0}^{k-1}{{\,\mathrm{rk}\,}}(V_{r+2j-4k})+\sum \limits _{j=k}^{2k}{{\,\mathrm{rk}\,}}(W_{2j+1-r}). \end{aligned}$$

Simplifying, yields ${{\,\mathrm{rk}\,}}(V_{4k+2-r})+{{\,\mathrm{rk}\,}}(\Lambda ^2W_{2k+1-r})={{\,\mathrm{rk}\,}}(W_{2k+1-r})$. Thus, ${{\,\mathrm{rk}\,}}(W_{2k+1-r})$ is one or two.

If ${{\,\mathrm{rk}\,}}(W_{2k+1-r})=2,$ then the determinant of the isomorphism in (5.9) gives

(5.10)

By (5.8), the above terms satisfy

$$\begin{aligned} V^2_{r-2k}K^{2-2k}\cong {\left\{ \begin{array}{ll} V_{r-2j}V_{r+2j-2-2k},&{}\text { for }j=1,\ldots ,\lfloor \frac{k}{2}\rfloor \\ W_{r-2j-1}W_{r+2j-1-2k},&{}\text { for }j=1,\ldots ,\lfloor \frac{k-1}{2}\rfloor \\ W_{r-2j-1}V_{r+2j-2k}K,&{}\text { for }j=0,\ldots ,k-1. \end{array}\right. } \end{aligned}$$

(5.11)

Hence, simplifying (5.10) yields $V_{r-2k-2}\cong V_rK^{2+2k}.$ The Higgs field gives rise to non-zero maps $V_{r-2k-2}\rightarrow V_{r-2k}K^2$ and $V_{r-2k}\rightarrow V_{r}K^{2k}$ by Proposition 3.10. Thus, $\deg (V_{r-2k-2})-\deg (V_{r-2k})=4g-4$. As in the base case, this leads to a contradiction of stability. Namely, stability implies that the kernels of $\eta _{2k+1-r}:W_{2k+1-r}\rightarrow V_{2k+2-r} K$ and of $\eta _{r-1-2k}:W_{r-1-2k}\rightarrow V_{r-2k} K$ have negative degree, so that $\deg (V_{2k-r})-2g+2<\deg (W_{2k+1-r})<\deg (V_{2k+2-r})+2g-2$. So ${{\,\mathrm{rk}\,}}(W_{2k+1-r})=1.$

Using ${{\,\mathrm{rk}\,}}(W_{2k+1-r})=1,$ (5.8) and (5.11), the determinant of (5.9) gives

$$\begin{aligned} \begin{aligned} V_rV_{r-2k-2}\otimes \displaystyle \bigotimes \limits _{j=1}^{\left\lfloor \frac{k}{2}\right\rfloor }(V_{r-2k}^2K^{2-2k}) \otimes V_{r-2k}K^{1-2k}W_{r-1-2k}\otimes \displaystyle \bigotimes \limits _{j=1}^{\left\lfloor \frac{k-1}{2}\right\rfloor }(V_{r-2k}^2K^{2-2k})\\ \cong \ \ \ \ \ \displaystyle \bigotimes \limits _{j=0}^{k-1} (V_{r-2k}^2 K^{2-2k}) \otimes W_{r-1-2k}V_rK, \end{aligned} \end{aligned}$$

which simplifies to $V_{2k-r}\cong V_{2k+2-r}K^2.$ The Higgs field defines a non-zero map $V_{2k-r}\rightarrow W_{2k+1-r}K\rightarrow V_{2k+2-r}K^2$. Thus,

$$\begin{aligned} V_{2k-r}\cong W_{2k+1-r}K\cong V_{2k+2-r}K^2. \end{aligned}$$

(5.12)

Recall that k was an integer between 1 and $\frac{r-2}{2}$. Since r is even, we can take $k=(r-2)/2$, and hence (5.12) gives $V_{-2}\cong W_{-1}K\cong V_0K^2$. This completes the proof for the chain (5.4).

The difference for the chain (5.3) is that r is odd and instead of (5.8) we must assume

$$\begin{aligned} V_{1-r}\cong W_{2-r}K\cong V_{3-r}K^2\cong \cdots \cong V_{2k-1-r}K^{2k-2}\cong W_{2k-r}K^{2k-1}, \end{aligned}$$

where k is an integer satisfying $1\leqslant k\leqslant (r-3)/2$. The same proof as above shows that $W_{2k-r}\cong V_{2k+1-r}K\cong W_{2k+2-r}K^2$. By taking $k=(r-3)/2$ we have $W_{-3}\cong V_{-2}K\cong W_{-1}K^2$, and no condition on $V_0$ is imposed. Switching the roles of V and W gives the proof for the chains (5.2) and (5.5). $\square$

We can now complete the classification of the stable minima with vanishing $\mathbb {H}^2(C^\bullet )$.

Theorem 5.3

A stable $\mathrm {SO}(p,q)$-Higgs bundle $(V,W,\eta )$ with $p\leqslant q$, $\eta \ne 0$ and $\mathbb {H}^2(C^\bullet (V,W,\eta ))=0$ defines a local minimum of the Hitchin function if and only if it is a holomorphic chain of the form (5.2), (5.3), (5.4) or (5.5) which satisfies one of the following:

• 1. The chain is given by (5.2) with $p=2$ and $0<\deg (V_{-1})<2g-2$.

• 2. The chain is given by (5.2) with $p\geqslant 2$, $s=p-1$ and the bundle $W_0$ decomposes as $W_0=I\oplus W_0'$, where $W_0'$ is a stable $\mathrm {O}(q-p+1,\mathbb {C})$-bundle with $\det (W_0')=I$. Moreover, $V_j=IK^{-j}$ and $W_j=IK^{-j}$ for all $j\ne 0,$ and with respect to the splitting of $W_0$, the chain is given by

  

  (5.13)

• 3. The chain is of the form (5.3) with $q=p$, and for some 2-torsion line bundle I, $V_0=I\oplus I$, $V_j=IK^{-j}$ and $W_{j}=IK^{-j}$ for all $j\ne 0,$ and the chain is given by

  

  (5.14)

• 4. The chain is of the form (5.3) with $q=p+1$, $V_j=K^{-j}$ and $W_j=K^{-j}$ for all $|j|<p$ and $W_{-p}$ is a line bundle satisfying $\deg (W_{-p})\in (0,p(2g-2)].$

• 5. The chain is of the form (5.4) where $W_0$ is a stable $\mathrm {O}(q-p+1,\mathbb {C})$-bundle with $\det (W_0)=I$, and $V_j=IK^{-j}$ and $W_j=IK^{-j}$ for all $j\ne 0.$

• 6. The chain is of the form (5.5) with $q=p+1$, $V_0=0$, $W_0\cong {\mathcal O}$, $V_j=K^{-j}$ and $W_{j}=K^{-j}$ for $0<|j|<p$ and $W_{-p}$ is a line bundle satisfying $\deg (W_{-p})\in (0,p(2g-2)].$

• 7. The chain is of the form (5.5) with $q=p$, and for some 2-torsion line bundle I, $V_j=IK^{-j}$ and $W_{j}=IK^{-j}$ for all j.

Remark 5.4

Cases (2)–(7) are special cases of the fixed points considered in Lemma 4.9. In case (2), the Higgs bundle is still a local minimum of the Hitchin function if the invariant bundle $W_0'$ is strictly polystable. Similarly, replacing the stable orthogonal bundle $W_0$ in case (5) with a strictly polystable orthogonal bundle still defines a local minimum. We will prove that these are the only local minima apart from $\eta =0.$ Note also that none of the above cases have $p=1$ and $q>2$.

Proof

We first show that cases (1) and (2) are sufficient for the chain (5.2) to be a stable minima with $\mathbb {H}^2(C^\bullet )=0$ by invoking Proposition 3.4. For case (1), $C^\bullet _{2}$ is the only isomorphism to consider. We have $\mathfrak {so}_2(V)\oplus \mathfrak {so}_2(W)=\Lambda ^2 V_1$ and ${{\,\mathrm{Hom}\,}}_3(W,V)\otimes K=0,$ which is an isomorphism since ${{\,\mathrm{rk}\,}}(V_{-1})=1.$ For case (2), the holomorphic chain (5.13) is a fixed point considered in Lemma 4.9 with $W_p=0.$ By Lemma 4.10, $C^\bullet _k:\mathfrak {so}_k(V)\oplus \mathfrak {so}_k(W)\xrightarrow {{{\,\mathrm{ad}\,}}_\eta }{{\,\mathrm{Hom}\,}}_{k+1}(W,V)\otimes K$ is an isomorphism for all $k>0.$

We now show that cases (1) and (2) are necessary for chains of the form (5.2). We have a chain

with $s\geqslant 1$ odd. By Lemma 5.2 each of the bundles in the chain is a line bundle except $W_0$. So $p=s+1$ is even and ${{\,\mathrm{rk}\,}}(W_0)=q-p+2\geqslant 2$. Note that ${\mathcal O}=\det (V)=\det (W)=\det (W_0).$

If $N=\ker (\eta _0)$, then $\eta _0^*$ maps $V_{-1}$ to $N^\perp K\subset W_0\otimes K$. By Proposition 3.10, $\eta _0^*$ is non-zero, hence $\deg (N^\perp )-\deg (V_{-1})+2g-2\geqslant 0$. If N is coisotropic then $N^\perp$ is isotropic, and stability implies $\deg (V_{-1})+\deg (N^\perp )<0$, which implies $\deg (V_{-1})<g-1$. If N is not coisotropic, then $\eta _0\eta _0^*$ is a non-zero section of the line bundle $V_1^2K^2$. Thus,

$$\begin{aligned} \deg (V_{-1})\leqslant 2g-2. \end{aligned}$$

(5.15)

If $p=2$ and $\deg (V_{-1})< 2g-2$ we are done. If $\deg (V_{-1})=2g-2,$ then $\eta _0\eta _0^*$ is a nowhere vanishing section of the line bundle $V_1^2K^2,$ and hence the kernel of $\eta _0$ is a holomorphic orthogonal bundle $W_0'\subset W_0$ of rank $q-p+1.$ Furthermore, stability of $(V,W,\eta )$ forces $W_0'$ to be stable. Taking orthogonal complements gives a decomposition $W_0=W_0'\oplus I$ where $KV_{1}=I=\det (W_0')$ since ${\mathcal O}=\det (W_0).$ By Lemma 5.2, the holomorphic chain is given by (5.13). Thus, for $p=2$ we are done. For $p>2$ we will show that stability forces $\deg (V_{-1})=2g-2$ and $V_{-s}=K^sI.$

For $p\geqslant 4$ and even, we have $s\geqslant 3$ and odd. Using decompositions analogous to (5.6) and (5.7) and ${{\,\mathrm{rk}\,}}(V_j)={{\,\mathrm{rk}\,}}(W_j)=1$ for $j\ne 0$, the isomorphism of $C^\bullet _{s-1}$ gives

$$\begin{aligned}&\mathfrak {so}_{s-1}(V)\oplus \mathfrak {so}_{s-1}(W)\cong \displaystyle \bigoplus \limits _{j=0}^{\left\lfloor \frac{s-1}{4}\right\rfloor }{{\,\mathrm{Hom}\,}}(V_{2j-s},V_{2j-1})\\&\qquad \qquad \oplus \displaystyle \bigoplus _{j=0}^{\left\lfloor \frac{s-3}{4}\right\rfloor }{{\,\mathrm{Hom}\,}}(W_{2j+1-s},W_{2j}) \end{aligned}$$

$$\begin{aligned} {{\,\mathrm{Hom}\,}}_{s}(W,V)\otimes K\ \cong \ \displaystyle \bigoplus \limits _{j=0}^{\frac{s-1}{2}} {{\,\mathrm{Hom}\,}}(W_{2j+1-s},V_{2j+1})\otimes K. \end{aligned}$$

Since $\det (W_0)={\mathcal O}$, the determinant of both sides of the isomorphism $C^\bullet _{s-1}$ is given by

$$\begin{aligned}&V_sV_{-1}\otimes \bigotimes _{j=1}^{\left\lfloor \frac{s-1}{4}\right\rfloor }V_{s-2j}V_{2j-1}\otimes W_{s-1}^{{{\,\mathrm{rk}\,}}(W_0)}\otimes \bigotimes \limits _{j=1}^{\left\lfloor \frac{s-3}{4}\right\rfloor } W_{s-1-2j}W_{2j}\nonumber \\&\quad \cong \bigotimes _{j=0}^\frac{s-3}{2}W_{s-1-2j}V_{2j+1}K\otimes (V_sK)^{{{\,\mathrm{rk}\,}}(W_0)}. \end{aligned}$$

(5.16)

From Lemma 5.2, we have $W_{s-1}\cong V_1K^{2-s}$ and

$$\begin{aligned} V_1^2K^{3-s}\cong {\left\{ \begin{array}{ll} V_{s-2j}V_{2j-1}, &{}\text { for }j=1,\ldots ,\left\lfloor (s-1)/4\right\rfloor \\ W_{s-1-2j}W_{2j}, &{}\text { for }j=1,\ldots ,\left\lfloor (s-3)/4\right\rfloor \\ W_{s-1-2j}V_{2j+1}K, &{}\text { for }j=0,\ldots ,(s-3)/2. \end{array}\right. } \end{aligned}$$

This simplifies (5.16) to $(V_sV_{-1}K^{s-1})^{p-q-1}\cong (V_1K)^2.$ As in the proof of Lemma 5.2, the Higgs field gives a non-zero map $V_1\rightarrow V_s K^{s-1}$. Therefore,

$$\begin{aligned}&0\geqslant (p-q-1)(\deg (V_s)-\deg (V_1)+(s-1)(2g-2))\\&\quad =2(\deg (V_1)+2g-2), \end{aligned}$$

and hence $\deg (V_1)\leqslant 2-2g$. By (5.15), we conclude that $\deg (V_{-1})=2g-2$ and $\deg (V_s)=-s(2g-2).$ As above, since $\deg (V_{-1})=2g-2$, the bundle $W_0$ decomposes as $W_0'\oplus I,$ where $W_0'$ is the kernel of $\eta _0$ and $\det (W_0)'=I=V_1K.$ Moreover, we have $V_s=IK^{-s}$ since, by Lemma 5.2, $W_{s-1}=IK^{1-s}$ and $\eta _{s-1}:W_{s-1}\rightarrow V_{s}\otimes K$ is non-zero. This completes the proof of (2).

Case (3) is similar to case (2), and cases (4) and (6) are almost identical. By Lemma 5.2, the holomorphic chain (5.3) is given by

where ${{\,\mathrm{rk}\,}}(W_j)=1$ for all j. Thus, $r=q-1$ and either ${{\,\mathrm{rk}\,}}(V_0)=1$ and $q=p+1$ or ${{\,\mathrm{rk}\,}}(V_0)=2$ and $q=p$. If $q=p$, then, the holomorphic chain is identical to case (2) with the roles of V and W switched. By the same argument as case (2), we conclude that the holomorphic chain must be of the form (5.14).

We now assume ${{\,\mathrm{rk}\,}}(V_0)=1$ and $q=p+1.$ Moreover, $V_0={\mathcal O}$ since ${\mathcal O}\cong \det (V)\cong V_0$. Since the Higgs field defines a non-zero maps $W_{-1}\rightarrow {\mathcal O}\otimes K$ and $W_{-1}\rightarrow W_1\otimes K^2,$ we conclude that $W_{-1}\cong K$. Thus, $W_j=K^{-j}$ and $V_j=K^{-j}$ for all $|j|<r$ by Lemma 5.2. Since $W_p$ is an invariant isotopic subbundle and the Higgs field $\eta _{-p}:W_{-p}\rightarrow V_{-p+1}K$ is non-zero, we conclude

$$\begin{aligned} 0<\deg (W_{-p})\leqslant p(2g-2). \end{aligned}$$

Thus, the conditions in case (4) are necessary.

The holomorphic chain from case (4) is a fixed point considered in Lemma 4.9 with $W_0'=0$ and ${{\,\mathrm{rk}\,}}(W_{-p})=1.$ By Lemmas 4.10 and 4.11, $C^\bullet _k:\mathfrak {so}_k(V)\oplus \mathfrak {so}_k(W)\xrightarrow {{{\,\mathrm{ad}\,}}_\eta }{{\,\mathrm{Hom}\,}}_{k+1}(W,V)\otimes K$ is an isomorphism for all $k>0$. Thus, the conditions in case (4) are also sufficient.

The holomorphic chain from case (5) is a fixed point considered in Lemma 4.9 with $W_{-p}=0$. By Lemma 4.10, $C^\bullet _k:\mathfrak {so}_k(V)\oplus \mathfrak {so}_k(W)\xrightarrow {{{\,\mathrm{ad}\,}}_\eta }{{\,\mathrm{Hom}\,}}_{k+1}(W,V)\otimes K$ is an isomorphism for all $k>0$. Thus, the conditions in case (5) are sufficient.

To show the conditions of (5) are necessary, note that the holomorphic chain (5.4) is given by

By Lemma 5.2, ${{\,\mathrm{rk}\,}}(V_j)=1$ for all j, thus $r=p-1$ and ${{\,\mathrm{rk}\,}}(W_0)\geqslant 1$. Also, if $V_0=I$, then $I=\det (V)=\det (W)=\det (W_0)$, and $V_j=IK^{-j}$ for all $|j|<p-1$ and $W_j=K^{-1}I$ for all $j\ne 0.$ Since $W_0\ne 0$, $\mathfrak {so}_{p-2}(V)\oplus \mathfrak {so}_{p-2}(W)\cong {{\,\mathrm{Hom}\,}}(W_{1-r},W_0)$ and ${{\,\mathrm{Hom}\,}}_{p-1}(W,V)\otimes K\cong {{\,\mathrm{Hom}\,}}(W_0,V_{p-1}K).$ Taking the determinant of this isomorphism and using $W_{2-p}=K^{p-2}I$ we conclude that $V_{1-p}=IK^{p-1}$, finishing the proof of case (5).

Finally, for case (7) the holomorphic chain (5.5) is given by

By Lemma 5.2, ${{\,\mathrm{rk}\,}}(W_j)=1$ for all j. Thus $s=q-1=p-1$ and $V_0$ is a rank one orthogonal bundle I with $I=\det (V)=\det (W)=W_0,$$V_j=IK^{-j}$ for all j and $W_j=IK^{-j}$ for all $|j|<s.$ Similar to case (4), we have $\mathfrak {so}_{p-2}(V)\oplus \mathfrak {so}_{p-2}(W)\cong {{\,\mathrm{Hom}\,}}(V_0,V_{p-2})$ and ${{\,\mathrm{Hom}\,}}_{p-1}(W,V)\otimes K\cong {{\,\mathrm{Hom}\,}}(W_{1-p},V_0K).$ Thus, the isomorphism $C^\bullet _{p-2}$ implies $W_{1-p}\cong IK^{p-1}.$ Thus, the conditions of (7) are necessary. As with the other cases, the conditions of case (7) are sufficient by Lemmas 4.9 and 4.10. $\square$

5.2 Stable minima with non-vanishing $\mathbb {H}^2(C^\bullet )$

We now classify stable $\mathrm {SO}(p,q)$-Higgs bundles such that the associated $\mathrm {SO}(p+q,\mathbb {C})$-Higgs bundle is strictly polystable. By Remark 2.14, these are exactly the stable $\mathrm {SO}(p,q)$-Higgs bundles which may have $\mathbb {H}^2(C^\bullet )\ne 0$. We will prove that such $\mathrm {SO}(p,q)$-Higgs bundles define minima of the Hitchin function if and only if the Higgs field $\eta$ is zero.

The $\mathrm {SO}(p+q,\mathbb {C})$-Higgs bundle associated to an $\mathrm {SO}(p,q)$-Higgs bundle $(V,Q_V,W,Q_W,\eta )$ is

$$\begin{aligned} (E,Q,\Phi )=\Big (V\oplus W,\left( {\begin{matrix}Q_V&{}0\\ 0&{}-Q_W\end{matrix}}\right) ,\left( {\begin{matrix}0&{}\eta \\ \eta ^*&{}0\end{matrix}}\right) \Big ). \end{aligned}$$

(5.17)

Recall that a $\mathrm {GL}(p,\mathbb {R})$-Higgs bundle is defined by a triple $(V,Q_V,\eta )$ where $(V,Q_V)$ is a rank p orthogonal vector bundle and $\eta :V\rightarrow V\otimes K$ is a holomorphic map satisfying $\eta ^*=Q_V^{-1}\eta ^T Q_V=\eta$. Given such a $\mathrm {GL}(p,\mathbb {R})$-Higgs bundle, we construct the $\mathrm {SO}(p,p)$-Higgs bundle $(V,Q_V,V,Q_V,\eta )$. Using the symmetry $\eta ^*=\eta$, the corresponding $\mathrm {SO}(2p,\mathbb {C})$-Higgs bundle is

$$\begin{aligned} (E,Q,\Phi )=\Big (V\oplus V,\left( {\begin{matrix}Q_V&{}0\\ 0&{}-Q_V\end{matrix}}\right) ,\left( {\begin{matrix}0&{}\eta \\ \eta &{}0\end{matrix}}\right) \Big ). \end{aligned}$$

Even if the $\mathrm {SO}(p,p)$-Higgs bundle $(V,V,\eta )$ is stable, the above $\mathrm {SO}(2p,\mathbb {C})$-Higgs bundle is strictly polystable. Indeed, the following pair of disjoint degree zero isotropic subbundles are both $\Phi$-invariant

The following proposition shows that this example characterizes the stable $\mathrm {SO}(p,q)$-Higgs bundles which are not stable as $\mathrm {SO}(p+q,\mathbb {C})$-Higgs bundles.

Proposition 5.5

Let $(V,W,\eta )$ be a stable $\mathrm {SO}(p,q)$-Higgs bundle. The associated $\mathrm {SO}(p+q,\mathbb {C})$-Higgs bundle (5.17) is strictly polystable if and only if

$$\begin{aligned} (V,Q_V,W,Q_W,\eta )\cong \Big (V_1\oplus V_2, \left( {\begin{matrix}Q_{V_1}&{}0\\ 0&{}Q_{V_2}\end{matrix}}\right) , V_1\oplus W_2, \left( {\begin{matrix}Q_{V_1}&{}0\\ 0&{}Q_{W_2}\end{matrix}}\right) ,\left( {\begin{matrix}\eta _1&{}0\\ 0&{}\eta _2\end{matrix}}\right) \Big ), \end{aligned}$$

(5.18)

where $(V_1,Q_{V_1},V_1,Q_{V_1},\eta _1)$ is a stable $\mathrm {SO}(p_1,p_1)$-Higgs bundle with $\eta _1^*=\eta _1$ and $(V_2,Q_{V_2},W_2,Q_{W_2},\eta _2)$ is a stable $\mathrm {SO}(p_2,q_2)$-Higgs bundle.

Proof

By the above discussion, the condition (5.18) is sufficient. We now show that it is necessary. Let $(V,W,\eta )$ be a stable $\mathrm {SO}(p,q)$-Higgs bundle and suppose the associated $\mathrm {SO}(p+q,\mathbb {C})$-Higgs bundle $(E,Q,\Phi )$ given by (5.17) is strictly polystable, so that there is a degree zero proper subbundle $U\subset V\oplus W$, which is isotropic with respect to Q and satisfies $\Phi (U)\subset U\otimes K$. Let $V_1\subset V$ and $W_1\subset W$ be the respective image sheaves of the projection of U onto each summand of $V\oplus W.$ The subsheaf $V_1\oplus W_1$ is preserved by $\Phi$, thus $\deg (V_1)+\deg (W_1)\leqslant 0$ by polystability of the associated $\mathrm {SL}(p+q,\mathbb {C})$-Higgs bundle $(V\oplus W,\Phi ).$

Consider the sequences

where the subsheaf $U^v\subset V$ is $Q_V$-isotropic, the subsheaf $U^w\subset W$ is $Q_W$-isotropic, $\eta (U^w)\subset U^v\otimes K$ and $\eta ^*(U^v)\subset U^w\otimes K.$ Stability of the $\mathrm {SO}(p,q)$-Higgs bundle gives $\deg (U^v)+\deg (U^w)<0$, which implies $\deg (V_1)+\deg (W_1)>0.$ But, since $V_1\oplus W_1$ is preserved by $\Phi$, $\deg (V_1)+\deg (W_1)\leqslant 0$ by polystability of the Higgs bundle $(V\oplus W,\Phi ).$ This contradiction implies

$$\begin{aligned} V_1\cong U\cong W_1. \end{aligned}$$

We claim that $V_1$ and $W_1$ are both orthogonal subbundles. Let $Q_{V_1}$ and $Q_{W_1}$ be the restrictions of $Q_V$ and $Q_W$ to $V_1$ and $W_1$ respectively. Consider the following sequences

where, by definition, $V_1^{\perp _{V_1}}=V_1^{\perp _V}\cap V_1$ and $W_1^{\perp _{W_1}}=W_1^{\perp _W}\cap W_1$. Since $V_1^{\perp _{V_1}}$ and $W_1^{\perp _{W_1}}$ are maximally isotropic subbundles of $V_1$ and $W_1$ respectively, both $V_1/V_1^{\perp _{V_1}}$ and $W_1/W_1^{\perp _{W_1}}$ are orthogonal bundles. In particular, $V_1^{\perp _{V_1}}$ and $W_1^{\perp _{W_1}}$ are degree zero isotropic subbundles of V and W respectively. Moreover, since $\eta ^*=Q_W^{-1}\eta ^TQ_V$, we have $Q_V(\eta (-),-)=Q_W(-,\eta ^*(-))$. This, together with the fact that $\eta (W_1)\subset V_1\otimes K$ and $\eta ^*(V_1)\subset W_1\otimes K$ (by $\Phi$-invariance of U), shows that

Again, stability of the $\mathrm {SO}(p,q)$-Higgs bundle $(V,W,\eta )$ implies both $V_1^{\perp _{V_1}}$ and $W_1^{\perp _{W_1}}$ are zero, which implies $V_1\subset V$ and $W_1\subset W$ are both orthogonal subbundles.

If $p_1={{\,\mathrm{rk}\,}}(W_1)={{\,\mathrm{rk}\,}}(V_1)$, then $(V_1,W_1,\eta |_{W_1})$ is an $\mathrm {SO}(p_1,p_1)$-Higgs bundle. Note that isomorphism between $V_1$ and $W_1$ is given by including $V_1$ into $V\oplus W$ and projecting onto W. Denoting this isomorphism by $g:V_1\rightarrow W_1$, we have $\eta |_{W_1} g=(g^{-1}\otimes {{\,\mathrm{Id}\,}}_K)\eta |_{W_1}^*$. Moreover, g is orthogonal since for any $x,y\in V_1$ we have $(x,g(x)),(y,g(y))\in U$, and

$$\begin{aligned} 0=Q((x,g(x)),(y,g(y)))=Q_{V_1}(x,y)-Q_{W_1}(g(x),g(y)) \end{aligned}$$

since U is isotropic. Therefore the pair $({{\,\mathrm{Id}\,}}_V,g^{-1})$ defines an isomorphism between $(V_1,W_1,\eta |_{W_1})$ and $(V_1,V_1,\eta _1)$ with $\eta _{1}=\eta |_{W_1}g$. In particular, $\eta _1=\eta _1^*$.

Let $V_2$ and $W_2$ be the orthogonal complements of $V_1$ and $W_1$ respectively and let $\eta _2:W_2\rightarrow V_2\otimes K$ be the restriction of $\eta$ to $W_2$. By the above discussion, we obtain the desired decomposition (5.18) of the $\mathrm {SO}(p,q)$-Higgs bundle $(V,W,\eta )$. $\square$

If a stable $\mathrm {SO}(p,q)$-Higgs bundle

$$\begin{aligned}&(V,Q_V,W,Q_W,\eta )\\&\quad \cong \Big (V_1\oplus V_2, \left( {\begin{matrix}Q_{V_1}&{}0\\ 0&{}Q_{V_2}\end{matrix}}\right) , V_1\oplus W_2, \left( {\begin{matrix}Q_{V_1}0\\ 0&{}Q_{W_2}\end{matrix}}\right) ,\left( {\begin{matrix}\eta _1&{}0\\ 0&{}\eta _2\end{matrix}}\right) \Big ), \end{aligned}$$

with $\eta _1^*=\eta _1$, is a local minimum of the Hitchin function, then $(V_1,Q_{V_1},\eta _1)$ is a local minimum of the Hitchin function on the $\mathrm {GL}(p_1,\mathbb {R})$-Higgs bundle moduli space and $(V_2,Q_{V_2},W_2,Q_{W_2},\eta _2)$ is a local minimum of the Hitchin function on the $\mathrm {SO}(p_2,q_2)$-Higgs bundle moduli space.

For $p>1$, the local minima in the $\mathrm {GL}(p,\mathbb {R})$-Higgs bundle moduli space with non-zero Higgs field are described in Example 3.8. When $p=2$, such local minima are of the form (3.5), and hence the holomorphic chain of the corresponding $\mathrm {SO}(2,2)$-Higgs bundle is of the form

(5.19)

When a stable $\mathrm {SO}(p_2,q_2)$-Higgs bundle is added to (5.19), the resulting $\mathrm {SO}(2+p_2,2+q_2)$-Higgs bundle is not stable since L and $L^{-1}$ are a pair of proper isotropic subbundles exchanged by the Higgs field.

For $p\geqslant 3$, the holomorphic chain associated to a $\mathrm {GL}(p,\mathbb {R})$ local minimum is given by

where $V_j=IK^{-j}$ for all j and some 2-torsion line bundle I. The holomorphic chain of the associated $\mathrm {SO}(p,p)$-Higgs bundle is

(5.20)

By Proposition 3.10, such an $\mathrm {SO}(p,p)$-Higgs bundle is not stable if $p\geqslant 4$ is even.

By the above discussion, the potential stable $\mathrm {SO}(p,q)$ local minima which are not stable as $\mathrm {SO}(p+q,\mathbb {C})$-Higgs bundles have the form (5.18), where $(V_2,W_2,\eta _2)$ is a stable $\mathrm {SO}(p_2,q_2)$ local minimum and $(V_1,V_1,\eta _1)$ is either a stable $\mathrm {SO}(p_1,p_1)$-Higgs bundle with $\eta _1=0$ or $p_1>2$ is odd and $(V_1,V_1,\eta _1)$ is a holomorphic chain of the form (5.20). The next two propositions address these cases.

Proposition 5.6

For $p\geqslant 3$ odd, the stable $\mathrm {SO}(p,p)$-Higgs bundle given by (5.20) with $V_j=IK^{-j}$ for all j and some 2-torsion line bundle I is not a minimum of the Hitchin function.

Proof

By assumption $r=\frac{p-1}{2}$ is a positive integer. Set $V=\bigoplus \nolimits _{j=0}^{2r}V_{j-r}$ and $W=\bigoplus \nolimits _{j=0}^{2r}W_{j-r}$ with $V_j=IK^{-j}$ and $W_j=IK^{-j}$ for all j and some 2-torsion line bundle I. The holomorphic chain (5.20) is given by

Let $\beta \in \Omega ^{0,1}(K^{1-2r})$ which is non-zero in cohomology and, with respect to the above splittings of V and W,  consider the deformed orthogonal holomorphic structures:

In the above splittings of V and W, the Higgs field is given by

$$\begin{aligned} \eta =\left( {\begin{matrix}0&{}&{}&{}\\ 1&{}0&{}&{}\\ {} &{}\ddots &{}\ddots &{}\\ &{}&{}1&{}0\end{matrix}}\right) :W\rightarrow V\otimes K, \end{aligned}$$

and a calculation shows that $\eta$ is still holomorphic with respect ${\bar{\partial }}_W^\beta$ and ${\bar{\partial }}_V^\beta$. So ${{\bar{\partial }}}^\beta _V$, ${{\bar{\partial }}}^\beta _W$, together with the corresponding orthogonal structures, and $\eta$, define an $\mathrm {SO}(p,p)$-Higgs bundle $(\mathbb {V},\mathbb {W},\eta )$. Since (5.20) is stable, and stability is an open condition, $(\mathbb {V},\mathbb {W},\eta )$ is also stable. Moreover, since $\beta$ is non-zero in cohomology, $(\mathbb {V},\mathbb {W},\eta )$ is not isomorphic to (5.20), so it is not ${\mathcal S}$-equivalent to it.

Consider the following orthogonal gauge transformations of V and W

$$\begin{aligned} g_t^V=g_t^W=\left( {\begin{matrix}t^r &{}&{}&{}\\ &{}t^{r-1}&{}&{}\\ &{}&{}\ddots &{}\\ &{}&{}&{}t^{-r}\end{matrix}}\right) . \end{aligned}$$

For each $t\in \mathbb {C}^*$, we see that

$g^W_t{{\bar{\partial }}}^\beta _W(g^W_t)^{-1}={\bar{\partial }}^{t^{1-2r}\beta }_W$ and $g^V_t(t\eta )(g_t^V)^{-1}=\eta$. Thus $\lim _{t\rightarrow \infty }(\mathbb {V},\mathbb {W},t\eta )$ is equal to the Higgs bundle given by (5.20). By Proposition 3.6 we conclude that (5.20) is not a local minimum. $\square$

Proposition 5.7

Let $(V,W,\eta )$ be a stable $\mathrm {SO}(p,q)$-Higgs bundle of the form

$$\begin{aligned} (V,Q_V,W,Q_W,\eta )\cong \Big (U\oplus V', \left( {\begin{matrix}Q_{U}&{}0\\ 0&{}Q_{V'}\end{matrix}}\right) , U\oplus W', \left( {\begin{matrix}Q_{U}&{}0\\ 0&{}Q_{W'}\end{matrix}}\right) ,\left( {\begin{matrix}0&{}0\\ 0&{}\eta '\end{matrix}}\right) \Big ) \end{aligned}$$

where $(U,Q_{U},U,Q_{U},0)$ is a stable $\mathrm {S}(\mathrm {O}(p_1)\times \mathrm {O}(p_1))$-Higgs bundle and $(V',Q_{V'},W',Q_{W'},\eta ')$ is a stable $\mathrm {SO}(p_2,q_2)$ local minimum from Theorem 5.3. Then $(V,W,\eta )$ is not a local minimum.

Proof

Suppose that $(V',Q_{V'},W',Q_{W'},\eta ')$ is a minimum of type (1) from Theorem 5.3. Then $(V,W,\eta )$ can be represented by

Since, $\deg (V_{-1})>0,$ we have $H^1({{\,\mathrm{Hom}\,}}(U,V_1))\ne 0$ by Riemann-Roch. Hence, $\alpha \in H^1({{\,\mathrm{Hom}\,}}(U,V_1)) 0$ defines a rank p holomorphic orthogonal bundle $\mathbb {V}$. In the $C^\infty$ splitting $V_{-1}\oplus U\oplus V_1$ the ${\bar{\partial }}$-operator is

$$\begin{aligned} \bar{\partial }_{\mathbb {V}}=\left( {\begin{matrix}{{\bar{\partial }}}_{V_{-1}}&{}&{}\\ -\alpha ^*&{}{{\bar{\partial }}}_U&{}\\ &{} \alpha &{}{{\bar{\partial }}}_{V_{1}}\end{matrix}}\right) . \end{aligned}$$

Analogously to the previous proposition, $(\mathbb {V},W,\eta )$, with $\eta |_U=0$ and $\eta |_{W_0}:W_0\rightarrow V_1\otimes K\hookrightarrow \mathbb {V}\otimes K$ is a stable $\mathrm {SO}(p,q)$-Higgs bundle which is not isomorphic to $(V,W,\eta )$. Using the gauge transformations $g^V_t=\left( {\begin{matrix}t&{}&{}\\ {} &{}{{\,\mathrm{Id}\,}}_U&{}\\ &{}&{}t^{-1}\end{matrix}}\right)$ and $g^W_t={{\,\mathrm{Id}\,}}_W$, one computes $\lim _{t\rightarrow \infty }(\mathbb {V},W,t\eta )=(V,W,\eta )$. By Proposition 3.6$(V,W,\eta )$ is not a local minimum. For the other types of local minima from Theorem 5.3, the argument is similar. Namely, one can take the summand of $U\oplus U$ in V or in W according to where the highest weight summand of the minimum $(V',Q_{V'},W',Q_{W'},\eta ')$ lies. $\square$

We conclude that the only stable $\mathrm {SO}(p,q)$-local minima with $\mathbb {H}^2(C^\bullet )\ne 0$ have vanishing Higgs field, and that the stable local minima with non-zero Higgs field are classified by Theorem 5.3.

5.3 Strictly polystable minima

Recall from Proposition 2.17 that a strictly polystable $\mathrm {SO}(p,q)$-Higgs bundle is isomorphic to

$$\begin{aligned} \Big (E\oplus E^*\oplus V,\left( {\begin{matrix}0&{}{{\,\mathrm{Id}\,}}&{}0\\ {{\,\mathrm{Id}\,}}&{}0&{}0\\ 0&{}0&{}Q_V\end{matrix}}\right) ,F\oplus F^*\oplus W,\left( {\begin{matrix}0&{}{{\,\mathrm{Id}\,}}&{}0\\ {{\,\mathrm{Id}\,}}&{}0&{}0\\ 0&{}0&{}Q_W\end{matrix}}\right) ,\left( {\begin{matrix}\beta &{}0&{}0\\ 0&{}\gamma ^T&{}0\\ 0&{}0&{}\eta \end{matrix}}\right) \Big ), \end{aligned}$$

where $(E,F,\beta ,\gamma )$ is a polystable $\mathrm {U}(p_1,q_1)$-Higgs bundle with $\deg (E)+\deg (F)=0$, and $(V,W,\eta )$ is a stable $\mathrm {SO}(p-2p_1,q-2q_1)$-Higgs bundle. Here $0\leqslant p_1\leqslant p/2$, $0\leqslant q_1\leqslant q/2$, and $(p_1,q_1)\ne (0,0)$.

Proposition 5.8

Let $(E,F,\beta ,\gamma )$ be a polystable $\mathrm {U}(p,q)$-Higgs bundle with $\deg (E)+\deg (F)=0$ which is a local minimum in ${\mathcal M}(\mathrm {U}(p,q))$. The associated strictly polystable $\mathrm {SO}(2p,2q)$-Higgs bundle

$$\begin{aligned} \Big (E\oplus E^*,\left( {\begin{matrix}0&{}{{\,\mathrm{Id}\,}}\\ {{\,\mathrm{Id}\,}}&{}0\end{matrix}}\right) ,F\oplus F^*,\left( {\begin{matrix}0&{}{{\,\mathrm{Id}\,}}\\ {{\,\mathrm{Id}\,}}&{}0\end{matrix}}\right) ,\left( {\begin{matrix}\beta &{}0\\ 0&{}\gamma ^T\end{matrix}}\right) \Big ) \end{aligned}$$

(5.21)

is a local minimum of the Hitchin function if and only if $\beta =\gamma =0$ or $p\leqslant 1$ or $q\leqslant 1.$

Proof

If $\beta =\gamma =0$, the Higgs field is identically zero and we have a minimum. In particular, if $p=0$ or $q=0$ we have $\beta =\gamma =0.$ Now suppose $p,q>0$ and that the $\mathrm {SO}(2p,2q)$-Higgs bundle (5.21) is a local minimum with non-zero Higgs field. Then the $\mathrm {U}(p,q)$-Higgs bundle $(E,F,\beta ,\gamma )$ is a local minimum in ${\mathcal M}(\mathrm {U}(p,q))$. Thus, either $\beta =0$ or $\gamma =0$ (cf. Example 3.8). Up to switching the roles of E, F, $E^*$ and $F^*$, the relevant holomorphic chain for the $\mathrm {SO}(2p,2q)$-Higgs bundle is

(5.22)

Since the $\mathrm {U}(p,q)$-Higgs bundle $(E,F,\beta ,0)$ is polystable and $\beta \ne 0$, we must have $\deg (E)<0<\deg (F)$. For $p=1$ or $q=1$, the associated $\mathrm {SO}(2p,2q)$-Higgs bundle is a local minimum by Proposition 3.11.

We now show that (5.21) is not a local minimum if $p,q>1$ and $\beta ,\gamma$ not both zero. First assume $(E,F,\beta ,0)$ is a stable $\mathrm {U}(p,q)$-Higgs bundle. Consider the chain (5.22). By stability, $\deg (F^*)$ and $\deg (E)$ are both negative. When $p,q>1,$ Riemann-Roch implies there exist $\alpha \in H^1(\Lambda ^2F^*){{\setminus }} 0$ and $\sigma \in H^1(\Lambda ^2E){\setminus } 0$. These classes define holomorphic orthogonal bundles with ${\bar{\partial }}$-operators

Define the Higgs field $\eta :W\rightarrow V\otimes K$ by the composition $W\rightarrow F\xrightarrow {\beta } E\otimes K\rightarrow V\otimes K$. Since semistability is an open condition, $(V,W,\eta )$ is a semistable $\mathrm {SO}(2p,2q)$-Higgs bundle. Furthermore, since $(E,F,\beta ,0)$ is stable, the pairs of isotropic subbundles (E, F) and $(E^*,F^*)$ are the only destabilizing pairs of (5.22). However, these are not destabilizing pairs of $(V,W,\eta )$ because F and $E^*$ are not subbundles, hence $(V,W,\eta )$ and (5.22) are not ${\mathcal S}$-equivalent. For each $t\in \mathbb {C}^*$, the gauge transformations

act as

$$\begin{aligned} ({{\bar{\partial }}}_V,{{\bar{\partial }}}_W,t\eta )\mapsto \left( \left( {\begin{matrix}{{\bar{\partial }}}_E&{} t^{-1}\sigma \\ 0 &{}{{\bar{\partial }}}_{E^*}\end{matrix}}\right) ,\left( {\begin{matrix}{{\bar{\partial }}}_{F^*}&{} t^{-1}\alpha \\ 0 &{}{{\bar{\partial }}}_F\end{matrix}}\right) ,\eta \right) . \end{aligned}$$

Thus the limit $\lim _{t\rightarrow \infty }(V,W,t\eta )$ is isomorphic to (5.22). By Proposition 3.6, (5.22) is not a minimum when $(E,F,\beta ,0)$ is a stable $\mathrm {U}(p,q)$-Higgs bundle.

Finally, let $(E,F,\beta ,0$) be a strictly polystable $\mathrm {U}(p,q)$-Higgs bundle. The integer $\tau =|{{\,\mathrm{rk}\,}}(F)\deg (E)-{{\,\mathrm{rk}\,}}(E)\deg (F)|$ satisfies $\tau \leqslant p(2g-2)$ [11, Theorem A]. For $p\leqslant q$ and $\tau <p(2g-2)$ or $p=q$ and $\tau =p(2g-2)$, it follows from [10, Theorem 5.1] that there is a path $\varepsilon :[0,1]\rightarrow {\mathcal M}(\mathrm {U}(p,q))$ such that $\varepsilon (0)=(E,F,\beta ,0)$ and $\varepsilon (t)$ is a stable $\mathrm {U}(p,p)$-Higgs bundle which is a minimum in ${\mathcal M}(\mathrm {U}(p,q))$ for all $t>0$. By the previous argument, the $\mathrm {SO}(2p,2p)$-Higgs bundle corresponding to $\varepsilon (t)$ is not a minimum for $t> 0$. Hence, it is also not a minimum when $t=0$. For $p<q$ and $\tau =p(2g-2)$, [11, Theorem B] implies that $(E,F,\beta ,0)$ is isomorphic to $(E,F',\beta ',0)\oplus F''$, where $(E,F',\beta ',0)$ is a polystable $\mathrm {U}(p,p)$-local minimum. By the previous arguments, we conclude that (5.21) is not a local minimum in ${\mathcal M}(\mathrm {SO}(2p,2q))$ when $p,q>1$. $\square$

The next proposition shows that adding a stable $\mathrm {SO}(p,q)$ local minimum from Theorem 5.3 to a certain local minimum from Proposition 5.8 is not a local minimum.

Proposition 5.9

Let $(E,F,\beta ,\gamma )$ be a polystable $\mathrm {U}(m,n)$-Higgs bundle with $\deg (E)+\deg (F)=0$. Suppose that either $m=1$, $\beta =0$ and $\gamma \ne 0$, or $n=1$, $\gamma =0$ and $\beta \ne 0$. If $(V',W',\eta ')$ is a stable $\mathrm {SO}(p,q)$-local minimum with $\eta '\ne 0$, then the $\mathrm {SO}(p+2m,q+2n)$-Higgs bundle

$$\begin{aligned}&(V,Q_V,W,Q_W,\eta )=\Big (E\oplus E^*\oplus V', \left( {\begin{matrix}0&{}{{\,\mathrm{Id}\,}}&{}0\\ {{\,\mathrm{Id}\,}}&{}0&{}0\\ 0&{}0&{}Q_{V'}\end{matrix}}\right) ,\\&\quad F\oplus F^*\oplus W', \left( {\begin{matrix}0&{}{{\,\mathrm{Id}\,}}&{}0\\ {{\,\mathrm{Id}\,}}&{}0&{}0\\ 0&{}0&{}Q_{W'}\end{matrix}}\right) , \left( {\begin{matrix}\beta &{}0&{}0\\ 0&{}\gamma ^T&{}0\\ 0&{}0&{}\eta '\end{matrix}}\right) \Big ) \end{aligned}$$

is not a local minimum.

Proof

Up to switching the roles of E,  $V'$, F,  and $W'$, it suffices to consider holomorphic chains of one of the following six types:

(5.23)

where ${{\,\mathrm{rk}\,}}(V_{-1})=1$ and $0<\deg (V_{-1})\leqslant 2g-2$;

(5.24)

where $V_j=IK^{-j}$ and $W_j=IK^{-j}$ for all j and some I with $I^2\cong {\mathcal O}$;

(5.25)

where $V_j=K^{-j}$ and $W_j=K^{-j}$ for all $|j|<p$, ${{\,\mathrm{rk}\,}}(W_{-p})=1$, $0<\deg (W_{-p})\leqslant p(2g-2)$ and $\eta _{-p}\ne 0.$

Furthermore, in (5.23), (5.24) and (5.25), the first chain has $m=1$, $n>0$, $\deg (F)\leqslant 0\leqslant \deg (E)$ and $\gamma \ne 0$, while the second chain has $n=1$, $m>0$, $\deg (E)\leqslant 0\leqslant \deg (F)$ and $\beta \ne 0$. We will show that each of the above holomorphic chains is not a minimum. As in the proof of Proposition 5.8, we may assume the $\mathrm {U}(m,n)$-Higgs bundle is stable by the results of [10, 11].

Since ${{\,\mathrm{Hom}\,}}(E,V_1)$ is in the kernel of ${{\,\mathrm{ad}\,}}_\eta :\mathfrak {so}(V)\oplus \mathfrak {so}(W)\rightarrow {{\,\mathrm{Hom}\,}}(W,V)\otimes K$, we may use $\alpha \in H^1({{\,\mathrm{Hom}\,}}(E,V_1)){\setminus } 0$ to deform the holomorphic structure on V by considering non-zero extension

Namely, $\tilde{V}\oplus \tilde{V}^*$ is a rank p holomorphic orthogonal bundle. Defining $\tilde{\eta }:F\oplus F^*\oplus W_0\rightarrow \tilde{V}\oplus \tilde{V}^*$ by the compositions $W_0\xrightarrow {\eta _0} V_{1}\otimes K\rightarrow \tilde{V}\otimes K$ and $F^*\xrightarrow {\gamma ^T} E^*\otimes K\rightarrow \tilde{V}^*\otimes K$ gives a semistable $\mathrm {SO}(p,q)$-Higgs bundle $(\tilde{V}\oplus \tilde{V}^*,F\oplus F^*\oplus W_0,\tilde{\eta })$. However, this semistable Higgs bundle is ${\mathcal S}$-equivalent to the original Higgs bundle. To fix this, we also deform $F\oplus F^*\oplus W_0$.

First assume ${{\,\mathrm{rk}\,}}(F)=1$. Since $\gamma ^T$ is non-zero, we have a short exact sequence

$$\begin{aligned} 0\rightarrow F^*\xrightarrow {\gamma ^T}E^*\otimes K\rightarrow T\rightarrow 0, \end{aligned}$$

where T is a torsion sheaf. Since $V_1\otimes K$ is locally free, this yields the exact sequence

$$\begin{aligned} 0\rightarrow {{\,\mathrm{Hom}\,}}(E^*,V_1)\rightarrow {{\,\mathrm{Hom}\,}}(F^*,V_1\otimes K)\rightarrow {{\,\mathrm{Hom}\,}}(T,V_1\otimes K)\rightarrow 0, \end{aligned}$$

which implies that the map $H^1({{\,\mathrm{Hom}\,}}(E^*,V_1))\rightarrow H^1({{\,\mathrm{Hom}\,}}(F^*,V_1\otimes K))$, $\sigma \mapsto (\sigma \otimes {{\,\mathrm{Id}\,}}_K)\gamma ^T$ is surjective. For any $\delta \in H^1({{\,\mathrm{Hom}\,}}(F^*,W_0)){\setminus } 0$, we have $\eta _0\delta \in H^1({{\,\mathrm{Hom}\,}}(F^*,V_1\otimes K))$, and there exists $\sigma \in H^1({{\,\mathrm{Hom}\,}}(E^*,V_1))$ such that

$$\begin{aligned} \eta _0\delta -(\sigma \otimes {{\,\mathrm{Id}\,}}_K)\gamma ^T=0 \end{aligned}$$

(5.26)

in cohomology. Let $\mathbb {V}$ and $\mathbb {W}$ be the holomorphic orthogonal bundles, defined respectively by the $C^\infty$ bundles $V_1\oplus E\oplus E^*\oplus V_{-1}$ and $F\oplus W_0\oplus F^*$, together with the ${{\bar{\partial }}}$-operators

(5.27)

where $\alpha \in \Omega ^{0,1}({{\,\mathrm{Hom}\,}}(E,V_1))$, $\sigma \in \Omega ^{0,1}({{\,\mathrm{Hom}\,}}(E^*,V_1))$ and $\delta \in \Omega ^{0,1}({{\,\mathrm{Hom}\,}}(F^*,W_0))$ are (0, 1)-forms representing the cohomology classes $\alpha$, $\sigma$ and $\delta$ respectively. Notice that (5.26) implies that there is $\epsilon \in \Omega ^0({{\,\mathrm{Hom}\,}}(F^*,V_1\otimes K))$ so that the representatives $\sigma$ and $\delta$ satisfy

$$\begin{aligned} \eta _0\delta -(\sigma \otimes {{\,\mathrm{Id}\,}}_K)\gamma ^T=\epsilon {{\bar{\partial }}}_{F^*}-{{\bar{\partial }}}_{V_1}\epsilon . \end{aligned}$$

(5.28)

Finally, let $\tilde{\eta }:\mathbb {W}\rightarrow \mathbb {V}\otimes K$ be given, according to the above $C^\infty$ decompositions, by

$$\begin{aligned} \tilde{\eta }=\left( {\begin{matrix}0&{} \eta _0 &{} -\epsilon \\ 0 &{}0 &{}0 \\ 0&{} 0 &{} \gamma ^T\\ 0&{}0 &{}0 \end{matrix}}\right) . \end{aligned}$$

The Higgs field $\tilde{\eta }$ is holomorphic by (5.28). As in the previous propositions, $(\mathbb {V},\mathbb {W},\tilde{\eta })$ is a semistable $\mathrm {SO}(p,q)$-Higgs bundle which is not ${\mathcal S}$-equivalent to the first chain of (5.23). Using the gauge transformations $g^{\mathbb {V}}_t=\left( {\begin{matrix}t^{-1}&{}&{}&{}\\ {} &{}t^{-\frac{1}{2}}&{}&{}\\ &{}&{}t^{\frac{1}{2}}&{}\\ &{}&{}&{}t\end{matrix}}\right)$ and $g^{\mathbb {W}}_t=\left( {\begin{matrix}t^{-3/2}&{}&{}\\ {} &{}{{\,\mathrm{Id}\,}}_{W_0}&{}\\ &{}&{}t^{3/2}\end{matrix}}\right)$, one shows that $\lim _{t\rightarrow \infty }(\mathbb {V},\mathbb {W},t\tilde{\eta })$ is the first chain of (5.23). By Proposition 3.6, the first chain of (5.23) is not a local minimum.

Now suppose $n>1$. Since $\deg (F)\leqslant 0$, we have $\theta \in H^1(\Lambda ^2F){\setminus } 0$. Using $\theta$, we can the deformed Higgs bundle

in the $C^\infty$-decompositions $V_1\oplus E\oplus E^*\oplus V_{-1}$ and $F\oplus W_0\oplus F^*$. As above one uses suitably chosen gauge transformations and Proposition 3.6 to conclude that the first chain of (5.23) is not a local minimum.

An analogous argument, using $W_{p-2}$ and $V_{p-1}$ instead of $W_0$ and $V_1$, can be used to prove that a strictly polystable $\mathrm {SO}(p,q)$-Higgs bundle represented by the first chain of (5.24) is not a local minimum. The second chain in (5.25) is also dealt in a similar manner.

Consider the second chain of (5.23). Since ${{\,\mathrm{rk}\,}}(F)=1$ and $\beta \ne 0$, we have a short exact sequence $0\rightarrow F\xrightarrow {\beta }E\otimes K\rightarrow Q\rightarrow 0,$ where Q is the quotient sheaf. One sees that the map $H^1({{\,\mathrm{Hom}\,}}(E,V_1))\rightarrow H^1({{\,\mathrm{Hom}\,}}(F,V_1\otimes K))$, $a\mapsto a\beta$ is surjective. So, as in the previous case, by picking a non-zero element $c\in H^1({{\,\mathrm{Hom}\,}}(F,W_0))$, there exists $a\in H^1({{\,\mathrm{Hom}\,}}(E,V_1))$ such that $\eta _0c-a\beta =0$ in cohomology. Given this choice and given a non-zero element $b\in H^1({{\,\mathrm{Hom}\,}}(E^*,V_1))$, we construct a non-trivial deformation of the second chain of (5.23) in a similar manner to the case ${{\,\mathrm{rk}\,}}(F)=1$ in the first chain of (5.23).

An analogous argument can be used to prove that the second chain of (5.24) and the first chain in (5.25) are not a local minimum. $\square$

5.4 Summary of classification of minima of Hitchin function on ${\mathcal M}(\mathrm {SO}(p,q))$

Putting everything together, the following theorem classifies all polystable minima of the Hitchin function in the moduli space of $\mathrm {SO}(p,q)$-Higgs bundles for $p\leqslant q$.

Theorem 5.10

For $1\leqslant p\leqslant q$, let $f:{\mathcal M}(\mathrm {SO}(p,q))\rightarrow \mathbb {R}$ be the Hitchin function on the moduli space of polystable $\mathrm {SO}(p,q)$-Higgs bundles given by (3.1). A polystable $\mathrm {SO}(p,q)$-Higgs bundle $(V,W,\eta )$ is a local minimum of f if and only if $\eta =0$ or $(V,W,\eta )$ is isomorphic to a holomorphic chain of one of the following mutually exclusive types, where we have suppressed the twisting by K in the Higgs field from the notation:

• 1. $p=2$ and $(V,W,\eta )$ is of the form

  

  where $V=V_{-1}\oplus V_1$ with ${{\,\mathrm{rk}\,}}(V_{-1})=1$ and $0<\deg (V_{-1})< 2g-2$, $V_1=V_{-1}^*$ and $\eta _0$ is non-zero.

• 2. $p\geqslant 2$ and $(V,W,\eta )$ is of the form

  

  where $W_0'$ is a polystable $\mathrm {O}(q-p+1,\mathbb {C})$-bundle with $\det (W_0')=I$, $W=W_0'\oplus \bigoplus \limits _{i=1}^{p-1}W_{-p+2i}$ with $W_j=IK^{-j}$ for all j, $V=\bigoplus \limits _{i=0}^{p-1}V_{1-p+2i}$ with $V_j=IK^{-j}$ for all j, and each $\eta _j$ is non-zero.

• 3. $p=q$ and $(V,W,\eta )$ is of the form

  

  where I is a 2-torsion line bundle, $W=\bigoplus \limits _{i=0}^{p-1}W_{1-p+2i}$, $V=I\oplus \bigoplus \limits _{i=1}^{p-1}V_{-p+2i}$ with $W_j=IK^{-j}$ and $V_j=IK^{-j}$ for all j, and each $\eta _j$ is non-zero.

• 4. $q=p+1$ and $(V,W,\eta )$ is of the form

  

  where $V=\bigoplus \limits _{i=0}^{p-1}V_{1-p+2i}$ with $V_j=K^{-j}$ for all j, $W=\bigoplus \limits _{i=0}^{p-1}W_{-p+2i}$ with $W_j=K^{-j}$ for all $|j|<p$, ${{\,\mathrm{rk}\,}}(W_{-p})=1$ with $0<\deg (W_{-p})\leqslant p(2g-2)$ and each $\eta _j$ is non-zero.

Remark 5.11

In cases (2) and (3), $\det (V)= I^{p}=\det (W)$. Thus, such a Higgs bundle always reduces to $\mathrm {SO}_0(p,q)$ when p is even, and reduces to $\mathrm {SO}_0(p,q)$ only when $I={\mathcal O}$ for p odd.

Proof

If $\eta =0$, then we are done, so suppose $\eta \ne 0.$ By Theorem 5.3 and Propositions 5.6 and 5.7, the result holds if $(V,W,\eta )$ is a stable $\mathrm {SO}(p,q)$-Higgs bundle, so suppose it is a strictly polystable $\mathrm {SO}(p,q)$-Higgs bundle. By Proposition 2.17,

$$\begin{aligned} (V,W,\eta )\cong \Big (E\oplus E^*\oplus V',F\oplus F^*\oplus W',\left( {\begin{matrix}\gamma &{}&{}\\ {} &{}\beta ^*\\ &{}&{}\eta '\end{matrix}}\right) \Big ), \end{aligned}$$

where $(E,F,\beta ,\gamma )$ is a polystable $\mathrm {U}(p_1,q_1)$-Higgs bundle and $(V',W',\eta ')$ is a stable $\mathrm {SO}(p_2,q_2)$-Higgs bundle which does not necessarily have $p_2\leqslant q_2$. By Propositions 5.8 and 5.9, if such a Higgs bundle is a minimum of the Hitchin function, then one of the following hold

• (a) $\beta =\gamma =0$ and $(V',W',\eta ')$ is a minimum from Theorem 5.3,

• (b) $p_1=1$, $\beta =0$ or $\gamma =0$ and $\eta '=0$,

• (c) $q_1=1$, $\beta =0$ or $\gamma =0$ and $\eta '=0$.

For case (a), note that if $p_2=0$ or $q_2=0$ then the Higgs field is zero and we are at a minimum. Consider a holomorphic chain of the form

where $V_{-r}'$ and $W_{-r}'$ are holomorphic line bundles of positive degree. Since $\deg (E)=0$ and $\deg (V_r')<0$, $H^1({{\,\mathrm{Hom}\,}}(E,V_r'))$ and $H^1({{\,\mathrm{Hom}\,}}(E^*,V_r'))$ are both non-zero. For $\alpha \in H^1({{\,\mathrm{Hom}\,}}(E,V_r')){\setminus } 0$ and $\sigma \in H^1({{\,\mathrm{Hom}\,}}(E^*,V_r')){\setminus } 0$, take a deformation of V by fixing all the summands $V'_{2-r},\ldots ,V'_{r-2}$, and deforming $V'_{-r}\oplus E\oplus E^*\oplus V'_r$ to $\mathbb {V}$ as in (5.27). Keep W fixed. Keep also the Higgs field fixed, except that its restriction to $W'_{r-1}$ is composed with the inclusion of $V'_r\otimes K\rightarrow \mathbb {V}\otimes K$. As in the proofs of the previous propositions, this yields a polystable $\mathrm {SO}(p,q)$-Higgs bundle deforming the first chain above and decreasing f by Proposition 3.6. Similarly, the second chain does not define a minimum.

Since $q\geqslant p,$ the only way we can have a holomorphic chain

with ${{\,\mathrm{rk}\,}}(W_j')={{\,\mathrm{rk}\,}}(V_j')=1$ for all j is if $E=0$. Such a holomorphic chain is not strictly polystable. To finish case (a), consider holomorphic chains of the form

By Theorem 5.3 and Remark 5.4, such a Higgs bundle is a polystable minimum if and only if it satisfies the conditions of case (1) or case (2) in the statement of the theorem.

For case (b), we have ${{\,\mathrm{rk}\,}}(E)=1$ and up to switching E and $E^*$ the holomorphic chains are given by

(5.29)

where $0<\deg (E).$ As above, (with the roles of E and $V'$ switched) this does not define a local minimum if $V'\ne 0.$ When $V'=0,$ we have a local minimum satisfying case (1) of the statement of theorem.

For case (c), we have ${{\,\mathrm{rk}\,}}(F)=1$ and the holomorphic chain is given by (5.29) with E and F switched. As above, this is not a minimum if $W'=0.$ Since $p\leqslant q$ and ${{\,\mathrm{rk}\,}}(V)={{\,\mathrm{rk}\,}}(V')+2{{\,\mathrm{rk}\,}}(E)\leqslant 2,$ we have $V'=0$, giving a local minimum satisfying case (1) of the statement of theorem. $\square$

6 The connected components of ${\mathcal M}(\mathrm {SO}(p,q))$

In this section we use the results from the previous sections to count the number of connected components of the moduli space ${\mathcal M}(\mathrm {SO}(p,q))$, with $1\leqslant p\leqslant q$. If $p\ne 2$ or if $(p,q)=(2,2)$ or $(p,q)=(2,3)$ then we have enough information to give a precise count. In the remaining cases, namely $p=2$, $q\geqslant 4$, we give a lower bound on the number of connected components of ${\mathcal M}(\mathrm {SO}(2,q))$ and conjecture that this bound is sharp.

6.1 Connected components of ${\mathcal M}(\mathrm {SO}(p,q))$ for $2< p\leqslant q$

Recall from (2.14) that the moduli space of $\mathrm {SO}(p,q)$-Higgs bundles decomposes as

$$\begin{aligned} {\mathcal M}(\mathrm {SO}(p,q))=\coprod _{a,b,c}{\mathcal M}^{a,b,c}(\mathrm {SO}(p,q)), \end{aligned}$$

(6.1)

where the indices (a, b, c) are classes in $H^1(X,\mathbb {Z}_2)\times H^2(X,\mathbb {Z}_2)\times H^2(X,\mathbb {Z}_2)$ and a polystable $\mathrm {SO}(p,q)$-Higgs bundle $(V,Q_V,W,Q_W,\eta )$ is in ${\mathcal M}^{a,b,c}(\mathrm {SO}(p,q))$ if a is the first Stiefel–Whitney class of $(V,Q_V)$ and $(W,Q_W),$b is the second Stiefel–Whitney class of $(V,Q_V)$ and c is the second Stiefel–Whitney class of $(W,Q_W)$. Notice that each ${\mathcal M}^{a,b,c}(\mathrm {SO}(p,q))$ is not necessarily connected.

When $2< p\leqslant q$, the maximal compact subgroup $\mathrm {S}(\mathrm {O}(p)\times \mathrm {O}(q))\subset \mathrm {SO}(p,q)$ is semisimple. Thus by Proposition 2.20 each of the spaces ${\mathcal M}^{a,b,c}(\mathrm {SO}(p,q))$ is nonempty and has a unique connected component in which every Higgs bundle $(V,Q_V,W,Q_W,\eta )$ can be deformed to one with vanishing Higgs field. Such components account for $2^{2g+2}$ connected components of ${\mathcal M}(\mathrm {SO}(p,q))$. These are the ‘mundane’ components mentioned in the Introduction. Taking into account the ‘exotic’ components, we obtain the following precise count of the connected components of ${\mathcal M}(\mathrm {SO}(p,q))$ for $2< p\leqslant q$.

Theorem 6.1

Let X be a compact Riemann surface of genus $g\geqslant 2$ and denote the moduli space of $\mathrm {SO}(p,q)$-Higgs bundles on X by ${\mathcal M}(\mathrm {SO}(p,q)).$ For $2< p\leqslant q$, we have

$$\begin{aligned} |\pi _0({\mathcal M}(\mathrm {SO}(p,q)))|=2^{2g+2}+{\left\{ \begin{array}{ll} 2^{2g+1}-1+2p(g-1)&{}\text {if }q=p+1\\ 2^{2g+1}&{}\text {otherwise}. \end{array}\right. } \end{aligned}$$

Remark 6.2

We have often ignored the orientation of an $\mathrm {SO}(p,q)$-Higgs bundle. This is justified because the choice of orientation does not effect the component count of Theorem 6.1. Namely, every Higgs bundle can be deformed to a local minimum of the Hitchin function, and, for $2< p\leqslant q,$ such local minima either have zero Higgs field or are given by cases (2)–(4) of Theorem 5.10. The components corresponding to zero Higgs field are labeled by the topological invariants of $\mathrm {S}(\mathrm {O}(p)\times \mathrm {O}(q))$-bundles. For minimum of cases (2)–(4) of Theorem 5.10, there is a holomorphic orthogonal summand of either V or W with odd rank. Taking the isomorphism which is $-{{\,\mathrm{Id}\,}}$ on this summand and ${{\,\mathrm{Id}\,}}$ on the other summands reverses the orientation and acts on the Higgs field by $\eta \mapsto -\eta$. However, since the minimum is a $\mathbb {C}^*$-fixed point, there is a orientation preserving gauge transformation which sends $-\eta \mapsto \eta .$

Proof

By the above discussion we only need to determine the number of connected components of ${\mathcal M}(\mathrm {SO}(p,q))$ with the property that the Higgs field never vanishes. Recall that if $\mathrm {Min}({\mathcal M}(\mathrm {SO}(p,q)))$ is the subspace of ${\mathcal M}(\mathrm {SO}(p,q))$ where the Hitchin function (3.1) attains a local minimum, then

$$\begin{aligned} |\pi _0({\mathcal M}(\mathrm {SO}(p,q)))|\leqslant |\pi _0(\mathrm {Min}({\mathcal M}(\mathrm {SO}(p,q))))|. \end{aligned}$$

From Theorem 5.10, an $\mathrm {SO}(p,q)$-Higgs bundle $(V,W,\eta )$, with $2< p<q-1$, is a minimum of the Hitchin function with non-zero Higgs field if and only if the holomorphic chain is given by:

(6.2)

where the bundle $W_0'$ is a polystable $\mathrm {O}(q-p+1,\mathbb {C})$-bundle with $\det (W_0')=I$, $V_j=IK^{-j}$ and $W_j=IK^{-j}$ for all $j\ne 0,$ and each $\eta _j$ is non-zero. Such chains also define minimum when $q=p$. The other minimum when $q=p$ are given by holomorphic chains

(6.3)

where $I^2={\mathcal O},$$V_j=IK^{-j}$ and $W_j=IK^{-j}$ for all $j\ne 0,$ and each $\eta _j$ is non-zero. When $q=p+1$, in addition to minimum of the form (6.2) with ${{\,\mathrm{rk}\,}}(W_0')=2$, there are also minima of the form

(6.4)

where $V_j=K^{-j}$ and $W_j=K^{-j}$ for all $|j|<p,$$W_{-p}$ is a holomorphic line bundle with $0<\deg (W_{-p})\leqslant p(2g-2)$ and each $\eta _j$ is non-zero.

For $2< p=q$, each type of minimum is labeled by the choice of the 2-torsion line bundle I,  yielding $2^{2g+1}$ connected components. For $2<p<q$, the connected components of the minima subvarieties of the form (6.2) are labeled by the first and second Stiefel–Whitney class of the bundle $W_0'$ by Proposition 2.20. Thus, the number of connected components of these minima subvarieties is given by $|\mathrm {Bun}_X(\mathrm {O}(q-p+1))|=2^{2g+1}$ for $2< p<q-1$. For $2< p=q-1$, when the first Stiefel–Whitney class of $W_0'$ vanishes the second Stiefel–Whitney class also vanishes since $sw_1(W_0')=0$ implies $W_0'=L\oplus L^{-1}$ for some degree zero line bundle L. This gives $2^{2g+1}-1$ connected components of the minima subvarieties whose Higgs bundles are of the form (6.2). There are $p(2g-2)$ connected components of minima subvarieties of type (6.4) since its connected components are labeled by $\deg (W_{-p})\in (0,p(2g-2)]$.

Finally, by Theorem 4.1, each of the above minima are in a different connected component of the image the map $\Psi :{\mathcal M}_{K^p}(\mathrm {SO}(1,q-p+1))\times \bigoplus \nolimits _{j=1}^{p-1}H^0(K^{2j})\rightarrow {\mathcal M}(\mathrm {SO}(p,q)).$ Thus, each such minima subvariety defines a connected component. $\square$

The following is a direct corollary of the above proof. This formulation will be useful in Sect. 7. Recall notation (2.11).

Corollary 6.3

Suppose $2<p<q-1$. For polystable Higgs bundles $(V,W,\eta )\in {\mathcal M}(\mathrm {SO}(p,q))$ we have the following dichotomy:

• Either $(V,W,\eta )$ can be deformed to a polystable $(V',W',0)$,

• or $(V,W,\eta )$ can be deformed to $({\mathcal K}_{p-1}\otimes I,W_0'\oplus {\mathcal K}_{p-2}\otimes I,(0\ \ \eta _0))$, where $W_0'$ is a polystable rank $q-p+1$ orthogonal bundle with $\Lambda ^{q-p+1} W_0'=I$ and $({\mathcal K}_{p-1},{\mathcal K}_{p-2},\eta _0)$ is the unique minimum in the $\mathrm {SO}(p-1,p)$-Hitchin component.

For minima of the form (6.2), (6.3) or (6.4), the first and second Stiefel–Whitney classes of V and W are readily computed. The results are shown in the table.

Type of min.	$a=sw_1(W)$	$b=sw_2(V)$	$c=sw_2(W)$
(6.2)	$\begin{matrix} 0 &{}\,\text {if }p\text { is even}\\ sw_1(W_0')&{} \text {if }p\text { is odd} \end{matrix}$	0	$sw_2(W'_0)$
(6.3)	$\begin{matrix} 0&{}\quad \,\text {if }p\text { is even}\\ sw_1(I)&{}\quad \text {if }p\text { is odd}\end{matrix}$	0	0
(6.4)	0	0	$\deg (W_{-p})\pmod 2$

The following corollaries are immediate. Recall the notation of (6.1).

Corollary 6.4

For $2<p< q-1,$ we have

$$\begin{aligned} |\pi _0({\mathcal M}^{a,b,c}(\mathrm {SO}(p,q)))|={\left\{ \begin{array}{ll} 2&{}\text {if }p\text { is odd and }b=0\\ 2^{2g}+1&{}\text {if }p\text { is even, }a=0\text { and }b=0\\ 1&{}\text {otherwise.} \end{array}\right. } \end{aligned}$$

Corollary 6.5

For $2<p$ and $p=q-1$, we have

$$\begin{aligned}&|\pi _0({\mathcal M}^{a,b,c}(\mathrm {SO}(p,p+1)))|\\&\quad ={\left\{ \begin{array}{ll} 2&{}\text {if }p\text { is odd, }b=0\text { and }a\ne 0\\ 2+p(g-1)&{}\text {if }p\text { is odd and }a=b=c=0\\ 1+p(g-1)&{}\text {if }p\text { is odd and }a=b=0\text { and }c\ne 0\\ 2+2^{2g}+p(g-1)&{}\text {if }p\text { is even and }a=b=c=0\\ 1+2^{2g}+p(g-1)&{}\text {if }p\text { is even and }a=b=0\text { and }c\ne 0\\ 1&{}\text {otherwise.} \end{array}\right. } \end{aligned}$$

Corollary 6.6

For $2<p$ and $p=q$, we have

$$\begin{aligned} |\pi _0({\mathcal M}^{a,b,c}(\mathrm {SO}(p,p)))|={\left\{ \begin{array}{ll} 3&{}\text {if }p\text { is odd and }b=c=0\\ 2^{2g+1}+1&{}\text {if }p\text { is even and }a=b=c=0\\ 1&{}\text {otherwise.} \end{array}\right. } \end{aligned}$$

We observe finally that the following corollary is immediate since the map $\Psi$ is injective.

Corollary 6.7

For $p\geqslant 1$, the number of connected components of ${\mathcal M}_{K^p}(\mathrm {SO}(1,q))$ are given by

$$\begin{aligned} |\pi _0({\mathcal M}_{K^p}(\mathrm {SO}(1,q)))|={\left\{ \begin{array}{ll} 2^{2g}&{} q=1\\ 2^{2g+1}-1+p(2g-2)&{} q=2\\ 2^{2g+1}&{}q>2. \end{array}\right. } \end{aligned}$$

In particular, if $q>2$ then every polystable $K^p$-twisted $\mathrm {SO}(1,q)$-Higgs bundle can be deformed to one with zero Higgs field.

6.2 Connected components of ${\mathcal M}(\mathrm {SO}(2,q))$

In the previous section a complete component count of ${\mathcal M}(\mathrm {SO}(p,q))$ when $p\leqslant q$ and $p\ne 2$ was given. We now discuss the case $p=2$. In this special case the group $\mathrm {SO}(p,q)$ is a group of Hermitian type. Furthermore in this case the minima of type (1) from Theorem 5.10 appear. These are given by holomorphic chains of the form

(6.5)

where $0<\deg (V_{-1})<2g-2$ and $\eta _0$ is non-zero.

Let $(V,W,\eta )$ be an $\mathrm {SO}(2,q)$-Higgs bundle. As in the general case, the first and second Stiefel–Whitney classes of the orthogonal bundles provide primary topological invariants which help distinguish the connected components of the moduli space. However, when the first Stiefel–Whitney class vanishes, we have $(V,Q_V)\cong ( L\oplus L^{-1}, \left( {\begin{matrix}0&{}1\\ 1&{}0\end{matrix}}\right) )$ for some line bundle L. The natural number $|\deg (L)|$ satisfies $|\deg (L)|=sw_2(V)\pmod 2$ and provides a refinement of the second Stiefel–Whitney class invariant. This natural number is the absolute value of the so-called Toledo invariant of the $\mathrm {SO}(2,q)$-Higgs bundle. Moreover, if such an $\mathrm {SO}(2,q)$-Higgs bundle $(V,W,\eta )$ is polystable then

$$\begin{aligned} |\deg (L)|\leqslant 2g-2. \end{aligned}$$

This inequality is usually referred to as the Milnor-Wood inequality and was derived in the proof of Theorem 5.3 (see (5.15)). The special maximal case $|\deg (L)|=2g-2$ will be discussed in Sect. 7.3.

Examining the minima classification of Theorem 5.10 and using Theorem 4.1, in the case $2=p\leqslant q$ we see that the only obstruction to obtaining a full connected component count of ${\mathcal M}(\mathrm {SO}(2,q))$ is the connectedness of the fixed point set (6.5). In particular, for $2=p<q$, we get bounds, rather than precise values, namely

$$\begin{aligned} |\pi _0({\mathcal M}(\mathrm {SO}(2,q))|\geqslant {\left\{ \begin{array}{ll} 2^{2g+2}-4+4(g-1)+2^{2g+1}+4g-5 &{}\text {if }q=3\\ 2^{2g+2}-4+4(g-1)+2^{2g+1}&{} \text {if }q\geqslant 4 \end{array}\right. } \end{aligned}$$

It follows from [28], that the above inequality was shown to be an equality for $q=3$:

$$\begin{aligned} |\pi _0({\mathcal M}(\mathrm {SO}(2,3))|=3\times 2^{2g+1}+8g-13. \end{aligned}$$

(6.6)

We conjecture that equality also holds above for $q\geqslant 4$.

The complete count of components for ${\mathcal M}(\mathrm {SO}(2,2))$ has been deduced by different methods in [6, Corollary 7.1]. We obtain the same count, as we now briefly explain, leaving the details for the reader. By Proposition 3.12 and (3.10), any non-zero local minima reduces to $\mathrm {SO}_0(2,2)$. The allowed topological types of a polystable $\mathrm {SO}_0(2,2)$-Higgs bundle are given by a pair of integers (l, m) such that $l\geqslant 0$ and $l-2g+2\leqslant m\leqslant 2g-2-l$, and if $l=0$, then only |m| is an invariant. All the minima are connected subvarieties, except when (l, m) equals $(0,2g-2)$ or $(2g-2,0)$ each corresponding to $2^{2g}$ Hitchin components. Adding the zero minima which do not reduce to $\mathrm {SO}_0(2,2)$, yields the following.

Proposition 6.8

$|\pi _0({\mathcal M}(\mathrm {SO}(2,2))|=3(2^{2g+1}-1)+2g(2g-3)$.

7 Positive surface group representations and Cayley partners

In this section, we recall the Non-Abelian Hodge correspondence between the Higgs bundle moduli space and the moduli space of surface group representations. After proving some immediate consequences of Theorem 6.1, we discuss how the exotic components of Theorem 4.1 are related to recent work of Guichard and Wienhard on positive Anosov representations [32]. Finally, we show this relation with positive Anosov representations can be seen as a generalization of the phenomenon which produces the so-called Cayley partner of a $\mathrm {G}$-Higgs bundle with maximal Toledo invariant for $\mathrm {G}$ a Hermitian group of tube type.

7.1 Surface group representations

Let $\pi _1(S)$ be the fundamental group of a closed oriented surface S of genus $g\geqslant 2$ and let $\mathrm {G}$ be a real semisimple Lie group. A representation $\rho :\pi _1S\rightarrow \mathrm {G}$ is called reductive if the composition of $\rho$ with the adjoint representation of $\mathrm {G}$ is a completely reducible representation.

Denote the set of reductive representations by ${{\,\mathrm{Hom}\,}}^{red}(\pi _1S,\mathrm {G}).$ The conjugation action of $\mathrm {G}$ on ${{\,\mathrm{Hom}\,}}(\pi _1S,\mathrm {G})$ does not in general have a Hausdorff quotient. However, if we restrict to the set of reductive representations, the quotient will be Hausdorff.

Definition 7.1

The $\mathrm {G}$-representation variety${\mathcal R}(S,\mathrm {G})$ of a surface group $\pi _1S$ is the space of conjugacy classes of reductive representations of $\pi _1S$ in $\mathrm {G}$:

$$\begin{aligned} {\mathcal R}(S,\mathrm {G})={{\,\mathrm{Hom}\,}}^{red}(\pi _1S,\mathrm {G})/\mathrm {G}. \end{aligned}$$

Example 7.2

The set of Fuchsian representations$\mathrm {Fuch}(S)\subset {\mathcal R}(S,\mathrm {SO}(2,1))$ is defined to be the subset of conjugacy classes of faithful representations with discrete image. The space $\mathrm {Fuch}(S)$ defines one connected components of ${\mathcal R}(S,\mathrm {SO}(2,1))$ [27] and is in one to one correspondence with the Teichmüller space of isotopy classes of marked Riemann surface structures on the surface S. Since the surface S is assumed to be orientable, every Fuchsian representation reduces to $\mathrm {SO}_0(2,1).$

For $\mathrm {G}$ a split real form, there is a preferred class of embeddings

$$\begin{aligned} \iota :\mathrm {SO}_0(2,1)\longrightarrow \mathrm {G}\end{aligned}$$

(7.1)

called a principal embedding. When $\mathrm {G}$ is an adjoint group, the principal embedding is unique up conjugation. For the split real form $\mathrm {G}=\mathrm {SO}_0(p,p-1)$, the principal embedding is given by taking the $(p-1)^{st}$-symmetric product of the standard action of $\mathrm {SO}_0(2,1)$ on $\mathbb {R}^3.$ The principal embedding defines a map $\iota :{\mathcal R}(S,\mathrm {SO}_0(2,1))\rightarrow {\mathcal R}(S,\mathrm {G})$, and the Hitchin component $\mathrm {Hit}(S,\mathrm {G})\subset {\mathcal R}(S,\mathrm {G})$ is defined to be the connected component containing $\iota (\mathrm {Fuch}(S)).$

Each representation $\rho \in {\mathcal R}(S,\mathrm {G})$ defines a flat $\mathrm {G}$-bundle $E_\rho =(\widetilde{S}\times \mathrm {G})/\pi _1S.$ This gives a decomposition of the $\mathrm {G}$ representation variety:

$$\begin{aligned} {\mathcal R}(S,\mathrm {G})=\coprod \limits _{a\in \mathrm {Bun}_S(\mathrm {G})}{\mathcal R}^a(\mathrm {G}), \end{aligned}$$

where $a\in \mathrm {Bun}_S(\mathrm {G})$ is the topological type of the flat $\mathrm {G}$-bundle of the representations in ${\mathcal R}^a(\mathrm {G}).$ When $\mathrm {G}$ is a Hermitian Lie group $\mathrm {Bun}_S(\mathrm {G})$ is infinite. Such $\mathrm {G}$-Higgs bundles and surface group representations acquire a discrete invariant called the Toledo invariant. While the Toledo invariant has several different descriptions, they all yield a finite set of allowed rational values, and hence give a notion of maximality (see for example [7, 15, 19]). In particular, ${\mathcal R}^a(\mathrm {G})$ is nonempty for only finitely many values of $a\in \mathrm {Bun}_S(\mathrm {G}).$

The following theorem links the $\mathrm {G}$-representation variety and the $\mathrm {G}$-Higgs bundle moduli space. It was proven by Hitchin [34], Donaldson [21], Corlette [18] and Simpson [44] in various generalities. For the general statement below see [24].

Theorem 7.3

Let S be a closed oriented surface of genus $g\geqslant 2$ and $\mathrm {G}$ be a real semisimple Lie group. For each Riemann surface structure X on S there is a homeomorphism between the moduli space ${\mathcal M}_K(\mathrm {G})$ of $\mathrm {G}$-Higgs bundles on X and the $\mathrm {G}$-representation variety ${\mathcal R}(S,\mathrm {G})$. Moreover, for each $a\in \mathrm {Bun}_S(\mathrm {G})$, this homeomorphism identifies the spaces ${\mathcal M}_K^a(\mathrm {G})$ and ${\mathcal R}^a(\mathrm {G}).$

As in (6.1), for $(a,b,c)\in H^1(S,\mathbb {Z}_2)\times H^2(S,\mathbb {Z}_2)\times H^2(S,\mathbb {Z}_2),$ we have

$$\begin{aligned} {\mathcal R}(S,\mathrm {SO}(p,q))=\coprod {\mathcal R}^{a,b,c}(\mathrm {SO}(p,q)). \end{aligned}$$

Using Theorem 6.1 and the above correspondence we have a connected component count of ${\mathcal R}(S,\mathrm {SO}(p,q)).$

Theorem 7.4

Let S be a closed surface of genus $g\geqslant 2$. For $2< p\leqslant q$, the number of connected components of the representation variety ${\mathcal R}(S,\mathrm {SO}(p,q))$ is given by

$$\begin{aligned} |\pi _0({\mathcal R}(S,\mathrm {SO}(p,q)))|=2^{2g+2}+{\left\{ \begin{array}{ll} 2^{2g+1}-1+2p(g-1)&{}\text {if }q=p+1\\ 2^{2g+1}&{}\text {otherwise}. \end{array}\right. } \end{aligned}$$

Remark 7.5

The connected components of ${\mathcal R}^{a,b,c}(\mathrm {SO}(p,q))$ are given by corollaries 6.4, 6.5, and 6.6.

Corollary 6.3 can now be interpreted as a dichotomy in terms of the $\mathrm {SO}(p,q)$ representation variety.

Theorem 7.6

Let S be a closed surface of genus $g\geqslant 2$. For $2<p<q-1$, the representation variety ${\mathcal R}(S,\mathrm {SO}(p,q))$ is disjoint union of two sets

$$\begin{aligned} {\mathcal R}(S,\mathrm {SO}(p,q))={\mathcal R}^{cpt}(S,\mathrm {SO}(p,q))\sqcup {\mathcal R}^{ex}(S,\mathrm {SO}(p,q)), \end{aligned}$$

(7.2)

where

• $[\rho ]\in {\mathcal R}^{cpt}(S,\mathrm {SO}(p,q))$ if and only if $\rho$ can be deformed to a compact representation,

• $[\rho ]\in {\mathcal R}^{ex}(S,\mathrm {SO}(p,q))$ if and only if $\rho$ can be deformed to a representation

  $$\begin{aligned} \rho '=\alpha \oplus (\iota \circ \rho _{\mathrm {Fuch}})\otimes \det (\alpha ), \end{aligned}$$

  (7.3)

  where $\alpha$ is a representation of $\pi _1S$ into the compact group $\mathrm {O}(q-p+1)$, $\rho _{\mathrm {Fuch}}$ is a Fuchsian representation of $\pi _1S$ into $\mathrm {SO}_0(2,1)$, and $\iota$ is the principal embedding from (7.1).

Proof

For the first part, note that a representation $\rho :\pi _1S\rightarrow \mathrm {SO}(p,q)$ can be deformed to a compact representation if and only if the corresponding Higgs bundle can be deformed to one with vanishing Higgs field.

If $\rho$ cannot be deformed to a compact representation, then by Corollary 6.3, the associated $\mathrm {SO}(p,q)$-Higgs bundle $(V,W,\eta )$ can be deformed to (cf. (2.11))

$$\begin{aligned} ({\mathcal K}_{p-1}\otimes I,\widehat{W}\oplus {\mathcal K}_{p-2}\otimes I,(0\ \ \eta _0)), \end{aligned}$$

where $\widehat{W}$ is a polystable rank $q-p+1$ orthogonal bundle with $\Lambda ^{q-p+1}\widehat{W}=I$ and $({\mathcal K}_{p-1},{\mathcal K}_{p-2},\eta _0)$ is the unique minimum in the $\mathrm {SO}(p-1,p)$-Hitchin component. Through Theorem 7.3, the Higgs bundle description of the Hitchin component from (2.18) is identified with the representation variety from Example 7.2. In particular, if $s_H$ is the Hitchin section from (2.12), the representation associated to $s_H(0)$ is $\iota \circ \rho _{\mathrm {Fuch}}$ for a Fuchsian representation $\rho _{\mathrm {Fuch}}$ [35]. In particular, the representation associated to the unique minimum in the $\mathrm {SO}_0(p,p-1)$-Hitchin component $({\mathcal K}_{p-1},{\mathcal K}_{p-2},\eta _0)$ is given by $\iota \circ \rho _{\mathrm {Fuch}}$ for a Fuchsian representation $\rho _{\mathrm {Fuch}}.$

If $A\in \mathrm {SO}_0(p,p-1)$ and $B\in \mathrm {O}(q-p+1),$ then $(A,B)\mapsto \left( {\begin{matrix}\det (B)\cdot A&{}0\\ 0&{}B\end{matrix}}\right)$ defines an embedding

$$\begin{aligned} \mathrm {SO}_0(p,p-1)\times \mathrm {O}(q-p+1)\hookrightarrow \mathrm {SO}(p,q). \end{aligned}$$

If $\alpha :\pi _1S\rightarrow \mathrm {O}(q-p+1)$ is the representation associated to the polystable $\mathrm {O}(q-p+1,\mathbb {C})$-bundle $\widehat{W},$ then the representation associated to the $\mathrm {SO}(p,q)$-Higgs bundle $({\mathcal K}_{p-1}\otimes I,\widehat{W}\oplus {\mathcal K}_{p-2}\otimes I,(0\ \ \eta _0))$ is given by $\alpha \oplus (\iota \circ \rho _\mathrm {Fuch})\otimes \det (\alpha )$. $\square$

7.2 Positive Anosov representations

Anosov representations were introduced by Labourie [37] and have many interesting geometric and dynamic properties which generalize convex cocompact representations into rank one Lie groups. Important examples of Anosov representations include Fuchsian representations, quasi-Fuchsian representations, Hitchin representations into split real groups and maximal representations into Lie groups of Hermitian type. We will describe the necessary properties of Anosov representations and refer the reader to [29, 31, 36, 37] for more details.

Let $\mathrm {G}$ be a semisimple Lie group and $\mathrm {P}\subset \mathrm {G}$ be a parabolic subgroup. Let $\mathrm {L}\subset \mathrm {P}$ be the Levi factor (the maximal reductive subgroup) of $\mathrm {P}$, it is given by $\mathrm {L}=\mathrm {P}\cap \mathrm {P}^{opp}$, where $\mathrm {P}^{opp}$ is the opposite parabolic of $\mathrm {P}.$ The homogeneous space $\mathrm {G}/\mathrm {L}$ realized as the unique open $\mathrm {G}$ orbit in $\mathrm {G}/\mathrm {P}\times \mathrm {G}/\mathrm {P}$, and points $(x,y)\in \mathrm {G}/\mathrm {P}\times \mathrm {G}/\mathrm {P}$ in this open orbit are called transverse.

Definition 7.7

Let S be a closed orientable surface of genus $g\geqslant 2$. Let $\partial _\infty \pi _1S$ be the Gromov boundary of the fundamentatl group $\pi _1S$, topologically $\partial _\infty \pi _1S\cong \mathbb {R}\mathbb {P}^1$. A representation $\rho :\pi _1S\rightarrow \mathrm {G}$is$\mathrm {P}$-Anosov if there exists a unique continuous boundary map $\xi _\rho :\partial _\infty \pi _1S\rightarrow \mathrm {G}/\mathrm {P}$ which satisfies

• Equivariance: $\xi (\gamma \cdot x)=\rho (\gamma )\cdot \xi (x)$ for all $\gamma \in \pi _1S$ and all $x\in \partial _\infty \pi _1S$.

• Transversality: for all distinct $x,y\in \partial _\infty \pi _1S$ the generalized flags $\xi (x)$ and $\xi (y)$ are transverse.

• Dynamics preserving: see [29, 31, 36, 37] for the precise notion.

The map $\xi _\rho$ will be called the $\mathrm {P}$-Anosov boundary curve.

One important property of Anosov representations is that they define an open subset of the representation variety ${\mathcal R}(S,\mathrm {G})$. The set of Anosov representations is however not closed. For example, for the group $\mathrm {PSL}(2,\mathbb {C})$ the set of Anosov representations corresponds to the non-closed set quasi-Fuchsian representations of ${\mathcal R}(S,\mathrm {PSL}(2,\mathbb {C}))$. The special cases of Hitchin representations and maximal representations define connected components of Anosov representations. Both Hitchin representations and maximal representations satisfy an additional “positivity” property which is a closed condition. For Hitchin representations this was proven by Labourie [37] and Fock-Goncharov [22], and for maximal representations by Burger-Iozzi-Wienhard [14]. These notions of positivity have recently been unified and generalized by Guichard and Wienhard [32].

For a parabolic subgroup $\mathrm {P}\subset \mathrm {G}$, denote the Levi factor of $\mathrm {P}$ by $\mathrm {L}$ and the unipotent subgroup by $\mathrm {U}\subset \mathrm {P}$. The Lie algebra $\mathfrak {p}$ of $\mathrm {P}$ admits an $Ad_{\mathrm {L}}$-invariant decomposition $\mathfrak {p}=\mathfrak {l}\oplus \mathfrak {u}$ where $\mathfrak {l}$ and $\mathfrak {u}$ are the Lie algebras of $\mathrm {L}$ and $\mathrm {U}$ respectively. Moreover, the unipotent Lie algebra $\mathfrak {u}$ decomposes into irreducible $\mathrm {L}$-representation:

$$\begin{aligned} \mathfrak {u}=\bigoplus \mathfrak {u}_\beta . \end{aligned}$$

Recall that a parabolic subgroup $\mathrm {P}$ is determined by fixing a simple restricted root system $\Delta$ of a maximal $\mathbb {R}$-split torus of $\mathrm {G}$, and choosing a subset $\Theta \subset \Delta$ of simple roots. To each simple root $\beta _j\in \Theta$ there is a corresponding irreducible $\mathrm {L}$-representation space $\mathfrak {u}_{\beta _j}.$

Definition 7.8

([32, Definition 4.2]) A pair $(\mathrm {G},\mathrm {P}^\Theta )$admits a positive structure if for all $\beta _j\in \Theta ,$ the $\mathrm {L}^\Theta$-representation space $\mathfrak {u}_{\beta _j}$ has an $\mathrm {L}^\Theta _0$-invariant acute convex cone $c_{\beta _j}^\Theta$, where $\mathrm {L}^\Theta _0$ denotes the identity component of $\mathrm {L}^\Theta$.

If $(\mathrm {G},\mathrm {P}^\Theta )$ admits a positive structure, then exponentiating certain combinations of elements in the $\mathrm {L}^\Theta _0$-invariant acute convex cones give rise to a semigroup $\mathrm {U}^\Theta _{>0}\subset \mathrm {U}^\Theta$ [32, Theorem 4.5]. The existence of the semigroup $U_{>0}$ gives a well defined notion of positively oriented triples of pairwise transverse points in $\mathrm {G}/\mathrm {P}^\Theta .$ This notion allows one to define a positive Anosov representation.

Definition 7.9

([32, Definition 5.3]) If the pair $(\mathrm {G},\mathrm {P}^\Theta )$ admits a positive structure, then a $\mathrm {P}^\Theta$-Anosov representation $\rho :\pi _1S\rightarrow \mathrm {G}$ is called positive if the Anosov boundary curve $\xi :\partial _\infty \pi _1S\rightarrow \mathrm {G}/\mathrm {P}^\Theta$ sends positively ordered triples in $\partial _\infty \pi _1S$ to positive triples in $\mathrm {G}/\mathrm {P}^\Theta .$

Conjecture 7.10

([32]) If $(\mathrm {G},\mathrm {P}^\Theta )$ admits a notion of positivity, then the set $\mathrm {P}^\Theta$-positive Anosov representations is an open and closed subset of ${\mathcal R}(S,\mathrm {G}).$

In particular, the aim of this conjecture is to characterize the connected components of ${\mathcal R}(S,\mathrm {G})$ which are not labeled by primary topological invariants as being connected components of positive Anosov representations, such connected components are referred as higher Teichmüller components.

Remark 7.11

When $\mathrm {G}$ is a split real form and $\Theta =\Delta$, the corresponding parabolic is a Borel subgroup of $\mathrm {G}$. In this case, the connected component of the identity of the Levi factor is $\mathrm {L}^\Delta _0\cong (\mathbb {R}^+)^{rk(\mathrm {G})}$ and each simple root space $\mathfrak {u}_{\beta _i}$ is one dimensional. The $\mathrm {L}^\Delta _0$-invariant acute convex cone in each simple root space $\mathfrak {u}_{\beta _i}$ is isomorphic to $\mathbb {R}^+.$ The set of $\mathrm {P}^\Delta$-positive Anosov representations into a split group are exactly Hitchin representations. When $\mathrm {G}$ is a Hermitian Lie group of tube type and $\mathrm {P}$ is the maximal parabolic associated to the Shilov boundary of the Riemannian symmetric space of $\mathrm {G}$, the pair $(\mathrm {G},\mathrm {P})$ also admits a notion of positivity [15]. In this case, the space of maximal representations into $\mathrm {G}$ are exactly the $\mathrm {P}$-positive Anosov representations. In particular, the above conjecture holds in these two cases.

In general, the group $\mathrm {SO}(p,q)$ is not a split group and not a group of Hermitian type. Nevertheless, if $p\ne q$, then $\mathrm {SO}(p,q)$ has a parabolic subgroup $\mathrm {P}^\Theta$ which admits a positive structure. Here $\mathrm {P}^\Theta$ is the stabilizer of the partial flag $V_1\subset V_2\subset \cdots \subset V_{p-1},$ where $V_j\subset \mathbb {R}^{p+q}$ is a j-plane which is isotropic with respect to a signature (p, q) inner product with $p<q.$ Here the subgroup $\mathrm {L}^\Theta _{pos}\subset \mathrm {L}^\Theta \subset \mathrm {SO}(p,q)$ which preserves the cones $c^\Theta _{\beta _j}$ is isomorphic to $\mathrm {L}^\Theta _{pos}\cong (\mathbb {R}^+)^{p-1}\times \mathrm {SO}(1,q-p+1).$ We refer the reader to [32] and [17, Section 7] for more details.

To construct examples of $\mathrm {SO}(p,q)$ positive Anosov representations we have the following proposition.

Proposition 7.12

Let $p<q$. Consider the signature (p, q)-inner product $\langle x,x\rangle = \sum \limits _{j=1}^px_j^2-\sum \limits _{j=p+1}^{p+q}x_j^2$. If $A\in \mathrm {SO}_0(p,p-1)$ and $B\in \mathrm {O}(q-p+1),$ then the set matrices $\left( {\begin{matrix}\det (B)\cdot A&{}0\\ 0&{}B\end{matrix}}\right)$ defines an embedding

$$\begin{aligned} \mathrm {SO}_0(p,p-1)\times \mathrm {O}(q-p+1)\hookrightarrow \mathrm {SO}(p,q). \end{aligned}$$

If $\rho _{Hit}:\pi _1S\rightarrow \mathrm {SO}_0(p,p-1)$ is a Hitchin representation and $\alpha :\pi _1S\rightarrow \mathrm {O}(q-p+1)$ is any representation, then

$$\begin{aligned} \rho =\rho _{Hit}\otimes \det (\alpha )\oplus \alpha :\pi _1S\rightarrow \mathrm {SO}(p,q) \end{aligned}$$

is a $\mathrm {P}^{\Theta }$-positive Anosov representation.

This is proven for $q=p+1$ in [17, Section 7], and the proof for general q is the same. For the proof of the first part of the above proposition it suffices to show that the map $\mathrm {SO}(p,p-1)\rightarrow \mathrm {SO}(p,q)$ described above sends the positive semigroup $U^\Delta _{>0}\subset \mathrm {SO}(p,p-1)$ into the positive semigroup $U^\Theta _{>0}.$ The second part follows from the fact that a representation $\rho$ is a $\mathrm {P}$-Anosov representation if and only if the restriction of $\rho$ to any finite index subgroup is $\mathrm {P}$-Anosov, and the fact that the centralizer of an Anosov representation acts trivially on the Anosov boundary curve.

Using Proposition 7.12 and Theorem 7.6, we conclude that for $q>p+1$ the connected components of ${\mathcal R}(S,\mathrm {SO}(p,q))$ from Theorem 4.1 contain $\mathrm {P}^\Theta$-positive Anosov representations.

Proposition 7.13

Let $\mathrm {P}^\Theta \subset \mathrm {SO}(p,q)$ be the stabilizer of the partial flag $V_1\subset V_2\subset \cdots \subset V_{p-1},$ where $V_j\subset \mathbb {R}^{p+q}$ is a j-plane which is isotropic with respect to a signature (p, q) inner product with $p<q$. If $q>p+1$, then each connected component of ${\mathcal R}^{ex}(S,\mathrm {SO}(p,q))$ from (7.2) contains $\mathrm {P}^\Theta$-positive Anosov representations.

Remark 7.14

When $q=p+1$, this was shown in [17] for the analogous connected components which contain minima of the form (6.2). The components which contain minima of the form (6.4) are smooth, and one cannot use Proposition 7.12 to obtain positive representations in these components. However, we note that if Conjecture 7.10 holds, then each of the these smooth connected components of ${\mathcal R}(S,\mathrm {SO}(p,p+1))$ consists of positive representations since each component would be contained in a component of positive representations into $\mathrm {SO}(p,p+2).$

Another special feature of Hitchin representations and maximal representations is that they satisfy a certain irreducibility condition. Namely, if $\rho :\pi _1S\rightarrow \mathrm {G}$ is such a representation, then there is no proper parabolic subgroup $\mathrm {P}$ so that $\rho$ factors as $\rho :\pi _1S\rightarrow \mathrm {P}\hookrightarrow \mathrm {G}.$ For the Hitchin case, this follows from smoothness, and for the maximal case it follows from the from [15, Theorem 5]. For the components in ${\mathcal R}^{ex}(S,\mathrm {SO}(p,q))$, with $2<p<q-1$ (cf. (7.2)), it follows from Corollary 4.19. Let ${\mathcal R}^{cpt}(S,\mathrm {SO}(p,q))$ be the union of the connected components of ${\mathcal R}(S,\mathrm {SO}(p,q))$ containing compact representations.

Proposition 7.15

Let $2<p\leqslant q$ and $\rho \in {\mathcal R}(S,\mathrm {SO}(p,q)){\setminus }{\mathcal R}^{cpt}(S,\mathrm {SO}(p,q))$. Then $\rho$ does not factor through any proper parabolic subgroup of $\mathrm {SO}(p,q)$.

Proof

Suppose $\rho \in {\mathcal R}^{ex}(S,\mathrm {SO}(p,q))$ factors through a proper parabolic subgroup $\mathrm {P}$. Since points of ${\mathcal R}(S,\mathrm {SO}(p,q))$ consist of completely reducible representations, $\rho$ must factor through the Levi factor $\mathrm {L}$ of $\mathrm {P}.$ Consequently, the $\mathrm {SO}(p,q)$-Higgs bundle associated to $\rho$ must reduce to an $\mathrm {L}$-Higgs bundle. The Levi factors of parabolics of $\mathrm {SO}(p,q)$ are isomorphic to $\mathrm {GL}(n,\mathbb {R})\times \mathrm {SO}(p-n,q-n)$, for some n, embedded as

$$\begin{aligned} (A,B)\mapsto \left( {\begin{matrix}A\\ {} &{}B\\ &{}&{}A^{-1}\end{matrix}}\right) . \end{aligned}$$

But by Corollary 4.19, the Higgs bundles in the components associated to ${\mathcal R}^{ex}(S,\mathrm {SO}(p,q))$ do not reduce to such groups, leading to a contradiction. $\square$

Propositions 7.13 and 7.15 give further evidence for Conjecture 7.10, and it is thus natural to expect that all representations in the connected components from Theorem 4.1 are positive Anosov representations. Indeed, this would follow from Conjecture 7.10 and Proposition 7.13. Moreover, if Conjecture 7.10 is true, then the connected components of Theorem 4.1 correspond exactly to those connected components of ${\mathcal R}(S,\mathrm {SO}(p,q))$ which contain positive Anosov representations.

7.3 Positivity and a generalized Cayley correspondence

We conclude the paper by interpreting the parameterization of the ‘exotic’ connected components of the $\mathrm {SO}(p,q)$-Higgs bundle moduli space from Theorem 4.1 as a generalized Cayley correspondence.

Let $\mathrm {G}$ be a simple adjoint Hermitian Lie group of tube type and let $\mathrm {G}/\mathrm {P}$ be the Shilov boundary of the symmetric space of $\mathrm {G}.$ In [7], it is proven that if $\mathrm {L}$ is the Levi factor of $\mathrm {P}$, then the space of Higgs bundles with maximal Toledo invariant is isomorphic to ${\mathcal M}_{K^2}(\mathrm {L}).$ More generally, an analogous statement holds when $\mathrm {G}'\rightarrow \mathrm {G}$ is a finite cover such that a $\mathrm {G}$-Higgs bundle with maximal Toledo invariant lifts to a $\mathrm {G}'$-Higgs bundle. This correspondence between maximal $\mathrm {G}$-Higgs bundles and $K^2$-twisted $\mathrm {L}$-Higgs bundles is called the Cayley correspondence.

Remark 7.16

In [7], the above statement is stated differently. We use the above interpretation because it relates directly with the notions of positivity discussed in the previous section.

Note that the above parabolic and Levi factor are exactly the objects which appear in the notion of positivity when $\mathrm {G}$ is Hermitian Lie group of tube type. When $\mathrm {G}$ is a split real form the Hitchin components of $\mathcal {M}(G)$ admit an analogous interpretation. Namely, if $\mathrm {G}$ is such a split group, then $(\mathrm {G},\mathrm {P})$ admits a positive structure when $\mathrm {P}$ is a minimal parabolic subgroup. In this case, $\mathrm {L}\subset \mathrm {P}$ is $(\mathbb {R}^*)^{{{\,\mathrm{rk}\,}}(\mathrm {G})}$ and the identity component $\mathrm {L}_0$ is given by $(\mathbb {R}^+)^{{{\,\mathrm{rk}\,}}(\mathrm {G})}.$ Moreover, the moduli space of $K^j$-twisted $\mathbb {R}^+$-Higgs bundles is isomorphic to $H^0(K^j):$

$$\begin{aligned} {\mathcal M}_{K^j}(\mathbb {R}^+)\cong H^0(K^j). \end{aligned}$$

Thus, when the Hitchin base is $\bigoplus \nolimits _{j=1}^{{{\,\mathrm{rk}\,}}(G)}H^0(K^{m_j+1}),$ the Hitchin components are given by

$$\begin{aligned} {\mathcal M}_{K^{m_1+1}}(\mathbb {R}^+)\times \cdots \times {\mathcal M}_{K^{m_{{{\,\mathrm{rk}\,}}(G)}+1}}(\mathbb {R}^+). \end{aligned}$$

In particular, the Higgs bundles associated surface group to representations into split real groups which are positive with respect the minimal parabolic subgroup also satisfy a ‘Cayley correspondence’.

For the group $\mathrm {SO}(p,q),$ the Levi factor of the parabolic $\mathrm {P}^\Theta$ so that $(\mathrm {SO}(p,q),\mathrm {P}^\Theta )$ has a positive structure is $\mathrm {L}^\Theta =\mathrm {SO}(1,q-p+1)\times (\mathbb {R}^*)^{p-1}$. Moreover, the subgroup $\mathrm {L}^\Theta _{pos}$ which preserves the positive cones is

$$\begin{aligned} \mathrm {L}^\Theta _{pos}\cong \mathrm {SO}(1,q-p+1)\times \underbrace{\mathbb {R}^+\times \cdots \times \mathbb {R}^+}_{(p-1)\text {-times}}. \end{aligned}$$

Recall that the ‘exotic’ connected components in the image of $\Psi$ Theorem 4.1 are isomorphic to

$$\begin{aligned} {\mathcal M}_{K^p}(\mathrm {SO}(1,q-p+1))\times \prod \limits _{j=1}^{p-1}H^0(K^{2j}). \end{aligned}$$

Using ${\mathcal M}_{K^{2j}}(\mathbb {R}^+)=H^0(K^{2j})$, this is equivalent to

$$\begin{aligned} {\mathcal M}_{K^p}(\mathrm {SO}(1,q-p+1))\times \prod \limits _{j=1}^{p-1}{\mathcal M}_{K^{2j}}(\mathbb {R}^+). \end{aligned}$$

When $2=p\leqslant q$, we recover the Cayley correspondence for groups of Hermitian type [7, 12]. Hence, for $2<p\leqslant q$ we have established that the Higgs bundles associated to representations into $\mathrm {SO}(p,q)$ which cannot be deformed to compact representations satisfy a generalized Cayley correspondence. Moreover, when $p<q-1$ each such component of the representation variety contains positive representations by Proposition 7.13. This suggests a general theorem for positive representations which relates the connected components of the subgroup of $L^\Theta$ which preserves the cones with the product of moduli spaces of appropriately twisted $L_j$-Higgs bundles. Indeed this is consistent with results in [30] where topological invariants for $\theta$-positive representations are defined in terms of principal bundles with structure group given by the Levi subgroup we have identified as $L^{\Theta }$. It would be interesting to understand in more detail the relation between these two points of view.

Appendix A: Review of gauge theory and the Hitchin–Kobayashi correspondence

Details on points treated sketchily in the following can be found in [24, 34]. For simplicity we consider K-twisted Higgs bundles but analagous statements can be made for L-twisted Higgs bundles.

Let $\mathrm {G}$ be a real semisimple Lie group and $\mathrm {H}\subset \mathrm {G}$ a maximal compact subgroup. Let P be a $C^\infty$ principal $\mathrm {H}^\mathbb {C}$-bundle and fix a reduction to a principal $\mathrm {H}$-bundle $P_{\mathrm {H}}$. Hitchin’s self-duality equations are

$$\begin{aligned} \begin{aligned} F(A)-[\varphi ,\tau (\varphi )]&=0,\\ {\bar{\partial }}_A\varphi&=0. \end{aligned} \end{aligned}$$

(A.1)

Here A is a $\mathrm {H}$-connection on $P_{\mathrm {H}}$, ${\bar{\partial }}_A$ its associated ${\bar{\partial }}$-operator and $\varphi \in \Omega ^{1,0}(P_{\mathrm {H}}[\mathfrak {m}^\mathbb {C}])$. The map $\tau :\Omega ^{1,0}(P_{\mathrm {H}}[\mathfrak {m}^\mathbb {C}])\rightarrow \Omega ^{0,1}(P_{\mathrm {H}}[\mathfrak {m}^\mathbb {C}])$ is obtained by combining the compact real structure on $\mathfrak {g}^\mathbb {C}$ with conjugation on the form component.

A pair $(A,\varphi )$ gives a corresponding $\mathrm {G}$-Higgs bundle structure $({\bar{\partial }}_A,\varphi )$ on P; we denote the corresponding $\mathrm {G}$-Higgs bundle by $({\mathcal E}_A,\varphi )$. Conversely, given a $\mathrm {G}$-Higgs bundle $({\mathcal E}_A,\varphi )$, where the holomorphic bundle ${\mathcal E}_A$ is defined by ${\bar{\partial }}_A$, one obtains a pair $(A,\varphi )$ by taking A to be the Chern connection associated to ${\bar{\partial }}_A$ via the fixed reduction $P_{\mathrm {H}}\subset P$. The Hitchin–Kobayashi correspondence for $\mathrm {G}$-Higgs bundles [24] says that the $\mathrm {G}$-Higgs bundle is polystable if and only if there is a structure $({\bar{\partial }}_A,\varphi )$ in its ${\mathcal G}_{\mathrm {H}^\mathbb {C}}$-orbit such that the corresponding pair $(A,\varphi )$ solves Hitchin’s equations. Moreover, this pair is unique up to ${\mathcal G}_{\mathrm {H}}$-gauge transformations, where $\mathcal {G}_{\mathrm {H}}$ denotes the gauge group of $\mathrm {H}$-gauge transformations of $P_{\mathrm {H}}$.

We recall the following alternative point of view. Instead of fixing a reduction of the principal $\mathrm {H}^\mathbb {C}$-bundle P, we can consider a fixed structure of $\mathrm {G}$-Higgs bundle $({\bar{\partial }}_A,\varphi )$ and consider (A.1) as equations for a reduction of structure group to $\mathrm {H}\subset \mathrm {H}^\mathbb {C}$, usually known as a a harmonic metric . The Hitchin–Kobayashi correspondence then says that such a reduction exists if and only if $({\bar{\partial }}_A,\varphi )$ defines a polystable $\mathrm {G}$-Higgs bundle.

The space $\mathcal {A}$ of $\mathrm {H}$-connections on $P_{\mathrm {H}}$ is an affine space modeled on $\Omega ^1(P_{\mathrm {H}}[\mathfrak {h}])$. Let $\mathcal {C}\subset \mathcal {A}\times \Omega ^{1,0}(P_{\mathrm {H}}[\mathfrak {m}^\mathbb {C}])$ denote the configuration space of solutions to Hitchin’s equations (A.1). As a set, the moduli space of solutions to Hitchin’s self-duality equations is

$$\begin{aligned} \mathcal {M}^a_{\mathrm {H}}(\mathrm {G}) = \mathcal {C}/\mathcal {G}_{\mathrm {H}}, \end{aligned}$$

where a is the topological type. We shall denote by ${\mathcal M}_{\mathrm {H}}(\mathrm {G})$ the union of the moduli spaces $\mathcal {M}^a_{\mathrm {H}}(\mathrm {G})$ over all topological types a. In order to give the moduli space a topology, suitable Sobolev completions must be used in standard fashion; see [4], and also [33, Sec. 8] where the straightforward adaptation to Higgs bundles is discussed in the case $\mathrm {G}=\mathrm {GL}(n,\mathbb {C})$. The moduli space ${\mathcal M}_{\mathrm {H}}(\mathrm {G})$ then becomes a Hausdorff topological space.

The Hitchin–Kobayashi correspondence can now be stated as saying that the map

$$\begin{aligned} \begin{aligned} {\mathcal M}_{\mathrm {H}}(\mathrm {G})&\xrightarrow {\cong } \mathcal {M}(\mathrm {G}),\\ (A,\varphi )&\mapsto ({\bar{\partial }}_A,\varphi ) \end{aligned} \end{aligned}$$

(A.2)

is a bijection. It follows from the constructions that it is in fact a homeomorphism. Here and below, in analogy with Notation 3.1, we do not distinguish notationally between a pair $(A,\varphi )$ and its gauge equivalence class.

The moduli space ${\mathcal M}_{\mathrm {H}}(\mathrm {G})$ can be given additional structure by considering the deformation complex

$$\begin{aligned} \Omega ^0(P_{\mathrm {H}}(\mathfrak {h})) \xrightarrow {d_0} \Omega ^1(P_{\mathrm {H}}(\mathfrak {h}))\times \Omega ^{1,0}(P_{\mathrm {H}}[\mathfrak {m}^\mathbb {C}]) \xrightarrow {d_1} \Omega ^2(P_{\mathrm {H}}(\mathfrak {h}))\times \Omega ^{1,1}(P_{\mathrm {H}}[\mathfrak {m}^\mathbb {C}]). \end{aligned}$$

(A.3)

The operator $d_0$ is given by the infinitesimal action of the gauge group and the operator $d_1$ is obtained by linearizing Hitchin’s equations; the fact that $d_1\circ d_0=0$ follows because $(A,\varphi )$ is a solution. Denote the ith cohomology group of this complex by $H^i_{(A,\varphi )}$.

Proposition A.1

Let $(A,\varphi )$ be a solution to Hitchin’s equations and let $(\mathcal {E},\varphi )$ be the corresponding Higgs bundle. Then there are isomorphisms

$$\begin{aligned} H^0_{(A,\varphi )}\otimes \mathbb {C}&\cong \mathbb {H}^0(C^\bullet (\mathcal {E},\varphi )),\\ H^1_{(A,\varphi )}&\cong \mathbb {H}^1(C^\bullet (\mathcal {E},\varphi )),\\ H^2_{(A,\varphi )}&\cong \mathbb {H}^2(C^\bullet (\mathcal {E},\varphi )) \oplus H^0_{(A,\varphi )}, \end{aligned}$$

where $C^\bullet (\mathcal {E},\varphi )$ is the deformation complex (2.4).

Proof

The hypercohomology groups of the complex $C^\bullet (\mathcal {E},\varphi )$ can be calculated, using a Dolbeault resolution, as the cohomology groups of the complex

$$\begin{aligned} \Omega ^0(P[\mathfrak {h}^\mathbb {C}]) \xrightarrow {\delta _0} \Omega ^{0,1}(P[\mathfrak {h}^\mathbb {C}])\times \Omega ^{1,0}(P[\mathfrak {m}^\mathbb {C}]) \xrightarrow {\delta _1} \Omega ^{1,1}(P[\mathfrak {m}^\mathbb {C}]), \end{aligned}$$

(A.4)

where the differentials are constructed combining the adjoint action of $\varphi$ with ${\bar{\partial }}_A$. The proposition now follows essentially as in [20, Sec. 6.4.2] (which gives the analogous comparison between the deformation complexes for solutions to the anti-self duality equations and holomorphic vector bundles on a complex surface) using the Kähler identities and the bundle isomorphisms

$$\begin{aligned} \Omega ^{0,1}(P[\mathfrak {h}^\mathbb {C}])&\cong \Omega ^{1}(P_{\mathrm {H}}[\mathfrak {h}])\\ \Omega ^{0}(P[\mathfrak {h}^\mathbb {C}])&\cong \Omega ^{0}(P_{\mathrm {H}}[\mathfrak {h}])\oplus \Omega ^{2}(P_{\mathrm {H}}[\mathfrak {h}]). \end{aligned}$$

$\square$

Proposition A.2

Let $(A,\varphi )\in {\mathcal M}_{\mathrm {H}}(\mathrm {G})$ and let $(\mathcal {E}_A,\varphi )$ be the corresponding polystable $\mathrm {G}$-Higgs bundle. Then the following statements are equivalent:

• 1. $H^0_{(A,\varphi )} =0$ and $H^2_{(A,\varphi )}=0$.

• 2. $\mathbb {H}^0(C^\bullet (\mathcal {E}_A,\varphi ))=0$ and $\mathbb {H}^2(C^\bullet (\mathcal {E}_A,\varphi ))=0$.

• 3. $(\mathcal {E}_A,\varphi )$ is stable as a $\mathrm {G}^\mathbb {C}$-Higgs bundle.

Proof

The equivalence of the first two statements is immediate from Proposition A.1. The equivalence of the last two statements is also immediate in view of Remarks 2.13 and 2.14. $\square$

Definition A.3

Let $\mathcal {C}^s\subset \mathcal {C}$ denote the subspace of pairs $(A,\varphi )$ such that $(\mathcal {E}_A,\varphi )$ is stable as a $\mathrm {G}^\mathbb {C}$-Higgs bundle. Similarly, let $\mathcal {C}_{\mathbb {C}}^s \subset \mathcal {A}\times \Omega ^{1,0}(P_{\mathrm {H}}[\mathfrak {m}^\mathbb {C}])$ denote the subspace of pairs $(A,\varphi )$ such that ${\bar{\partial }}_A\varphi =0$ and $(\mathcal {E}_A,\varphi )$ is stable as a $\mathrm {G}^\mathbb {C}$-Higgs bundle. Define $\mathcal {M}_{\mathrm {H}}^s(\mathrm {G})\subset {\mathcal M}_{\mathrm {H}}(\mathrm {G})$ and $\mathcal {M}^s(\mathrm {G})\subset \mathcal {M}(\mathrm {G})$ analogously.

We note that $\mathcal {C}_{\mathbb {C}}^s$ is an infinite dimensional Kähler manifold whose Kähler structure is induced from the ambient space $\mathcal {A}\times \Omega ^{1,0}(P_{\mathrm {H}}[\mathfrak {m}^\mathbb {C}])$.

Let $\Gamma _{(A,\varphi )}\subset \mathcal {G}_{\mathrm {H}}$ denote the stabilizer of a solution $(A,\varphi )$ to Hitchin’s equations. This is a compact Lie group with Lie algebra $H^0_{(A,\varphi )}$ [24]. The standard gauge theoretic construction of the moduli space can now be summarized as follows.

Proposition A.4

The subspace of $\mathcal {C}$ where $H^2_{(A,\varphi )}=0$ is a smooth infinite dimensional manifold. Moreover, for $(A,\varphi )$ with $H^2_{(A,\varphi )}=0$ a neighbourhood of the corresponding point in the moduli space is modeled on a neighbourhood of zero in $H^1_{(A,\varphi )}$ modulo the action of $\Gamma _{(A,\varphi )}$. If additionally $H^0_{(A,\varphi )} =0$, then $\Gamma _{(A,\varphi )}$ is finite. Thus $\mathcal {M}^s_{\mathrm {H}}(\mathrm {G})$ is a Kähler orbifold with Kähler form induced from $\mathcal {C}_{\mathbb {C}}^s$.

Remark A.5

The action of $\mathcal {G}_{\mathrm {H}}$ on $\mathcal {C}_{\mathbb {C}}^s$ is Hamiltonian with moment map $\mu (A,\varphi )= F(A)-[\varphi ,\tau (\varphi )]$. Hence the moduli space $\mathcal {M}^s(G)$ can be viewed as the infinite dimensional symplectic quotient

$$\begin{aligned} \mathcal {M}^s(G) = \mu ^{-1}(0)/\mathcal {G}_{\mathrm {H}} \cong \mathcal {C}_{\mathbb {C}}^s / \mathcal {G}_{\mathrm {H}^\mathbb {C}}. \end{aligned}$$

The isomorphism comes from the Hitchin–Kobayashi correspondence, which can thus be viewed as an infinite dimensional Kempf–Ness correspondence. Note that the Kähler form on $\mathcal {C}_{\mathbb {C}}^s$ restricts to a 2-form on $\mathcal {C}^s$ which is non-degenerate in directions transverse to the $\mathcal {G}_{\mathrm {H}}$-orbits — indeed this is just the pullback of the Kähler form on $\mathcal {M}^s(\mathrm {G})$.

The following was proved in [24]. It is analogous to the decomposition of a polystable vector bundle into a direct sum of stable ones, and plays a central role in the proof of the Hitchin–Kobayashi correspondence.

Proposition A.6

Let $(\mathcal {E},\varphi )$ be a polystable $\mathrm {G}$-Higgs bundle. Then there is a real reductive subgroup $\mathrm {G}'\subset \mathrm {G}$ and a Jordan–Hölder reduction of $(\mathcal {E},\varphi )$ to a stable $\mathrm {G}'$-Higgs bundle $(\mathcal {E}',\varphi ')$. The Jordan–Hölder reduction is unique up to isomorphism. Moreover, the solution to Hitchin’s equations on $(\mathcal {E}',\varphi ')$ induces the solution on $(\mathcal {E},\varphi )$.

Next we recall Hitchin’s method [34, 35] for studying the topology of $\mathcal {M}(\mathrm {G})$ using gauge theoretic methods, and explain how to translate it to the holomorphic point of view. Alternatively one could work exclusively using the holomorphic point of view, using Simpson’s adaptation in [46, Sec. 11].

Similarly to the holomorphic action of $\mathbb {C}^*$ on $\mathcal {M}(G)$ defined in Sect. 3.1, there is an action of $S^1$ on $\mathcal {A}\times \Omega ^{1,0}(P_{\mathrm {H}}[\mathfrak {m}^\mathbb {C}])$ given by

$$\begin{aligned} e^{i\theta }\cdot (A,\varphi ) = (A,e^{i\theta }\varphi ). \end{aligned}$$

This action clearly preserves the subspaces $\mathcal {C}^s$, $\mathcal {C}$ and $\mathcal {C}^s_{\mathbb {C}}$, and it descends to ${\mathcal M}_{\mathrm {H}}(\mathrm {G})$.

Proposition A.7

Let $S^1$ act on $\mathcal {M}(G)$ by restriction of the $\mathbb {C}^*$-action defined above. Then the following statements hold.

• 1. The bijection ${\mathcal M}_{\mathrm {H}}(\mathrm {G}) \rightarrow \mathcal {M}(\mathrm {G})$ defined in (A.2) is $S^1$-equivariant.

• 2. The class of $(A,\varphi )$ in ${\mathcal M}_{\mathrm {H}}(\mathrm {G})$ is fixed under the $S^1$-action if and only if the class of the corresponding Higgs bundle $(\mathcal {E}_A,\varphi )$ in $\mathcal {M}(\mathrm {G})$ is fixed under the $\mathbb {C}^*$-action.

Proof

Statement (1) is clear. Statement (2) is a consequence of the Hitchin–Kobayashi correspondence. $\square$

Since the vector bundle $P[\mathfrak {m}^\mathbb {C}] \cong P_{\mathrm {H}}[\mathfrak {m}^\mathbb {C}]$ has a Hermitian metric coming from the reduction of structure group to $\mathrm {H}$, one can define the Hitchin function:

$$\begin{aligned} f:{\mathcal M}_{\mathrm {H}}(\mathrm {G})\rightarrow \mathbb {R},\quad (A,\varphi )\mapsto \int _X||\varphi ||^2. \end{aligned}$$

(A.5)

We shall abuse notation and denote by the same letter the map $f:{\mathcal M}(\mathrm {G})\rightarrow \mathbb {R}$ induced via the identification (A.2). Using Uhlenbeck’s weak compactness theorem, Hitchin [34] showed that the map f is proper. Thus, as noted in Sect. 3.1, the Hitchin function can be used to study the connected components of the moduli space of $\mathrm {G}$-Higgs bundles.

The following is central for identifying local minima of f.

Lemma A.8

Let $(A,\varphi )\in \mathcal {M}^s_{\mathrm {H}}(\mathrm {G})$. If $(A,\varphi )$ is a local minimum of f, then it is a fixed point of the $S^1$-action. Equivalently, the corresponding Higgs bundle $({\mathcal E}_A,\varphi )\in {\mathcal M}^s(\mathrm {G})$ is a fixed point of the $\mathbb {C}^*$-action.

Proof

On the smooth locus of ${\mathcal M}_{\mathrm {H}}(\mathrm {G})$, the $S^1$-action is Hamiltonian with respect to the Kähler form and the function f (suitably normalized) is a moment map for this action (see [34, 35]). This means that, when multiplied by $\sqrt{-1}$, the vector field generating the $S^1$-action is the gradient of f and, therefore, critical points of f are exactly the fixed points of the $S^1$-action. This proves the proposition when $\Gamma _{(A,\varphi )}$ is trivial.

For a general $(A,\varphi )\in \mathcal {M}^s_{\mathrm {H}}(\mathrm {G})$ we can argue on the smooth manifold $\mathcal {C}^s\subset \mathcal {C}^s_{\mathbb {C}}$. Indeed, by its very definition, the function f lifts to the infinite dimensional Kähler manifold $\mathcal {C}^s_{\mathbb {C}}$ and it is a moment map for the $S^1$-action there. Thus, in view of Remark A.5, and in a similar way to the argument of the preceding paragraph, it follows that $(A,\varphi )$ is a critical point of f restricted to $\mathcal {C}^s$ if and only if its $\mathcal {G}_{\mathrm {H}}$-gauge equivalence class is fixed by the $S^1$-action. $\square$

We have the following useful observation. Let $\mathrm {G}'\subset \mathrm {G}$ be a reductive subgroup (we take this to include the choice of compatible Cartan data). Then a solution $(A,\varphi )$ to Hithin’s equations for $\mathrm {G}'$ on a principal $\mathrm {H}'$-bundle induces a solution for $\mathrm {G}$ on the $\mathrm {H}$-bundle obtained by extension of structure group. Hence we have a well defined map

$$\begin{aligned} {\mathcal M}(\mathrm {G}')\longrightarrow {\mathcal M}(\mathrm {G}) \end{aligned}$$

which is clearly compatible with the respective Hitchin functions. This leads immediately to the following result.

Lemma A.9

Let $\mathrm {G}'\subset \mathrm {G}$ be a reductive subgroup. Suppose $({\mathcal E},\varphi )$ is a $\mathrm {G}$-Higgs bundle which reduces to a $\mathrm {G}'$-Higgs bundle. If $({\mathcal E},\varphi )$ is a minimum of the Hitchin function on ${\mathcal M}(\mathrm {G})$ then it is a minimum of the Hitchin function on ${\mathcal M}(\mathrm {G}')$.

A solution $(A,\varphi )$ to Hitchin’s equations is called simple if its stabilizer $\Gamma _{(A,\varphi )}$ is trivial. The following proposition is simple to check.

Proposition A.10

Suppose that $(A,\varphi )\in {\mathcal M}_{\mathrm {H}}(\mathrm {G})$ is a fixed point for the $S^1$-action. Then for each $e^{i\theta }$ there is a gauge transformation $g(\theta )\in \mathcal {G}_{\mathrm {H}}$ such that

$$\begin{aligned} g(\theta )\cdot (A,\varphi ) = (A,e^{i\theta }\varphi ). \end{aligned}$$

The gauge transformation $g(\theta )$ is determined up to an element of the stabilizer $\Gamma _{(A,\varphi )}$. Moreover, if $(A,\varphi )$ is simple, then $e^{i\theta } \mapsto g(\theta )$ defines a group homomorphism $S^1\rightarrow \mathcal {G}_{\mathrm {H}}.$

Proposition A.11

Suppose $(A,\varphi )\in {\mathcal M}_{\mathrm {H}}(\mathrm {G})$ is a fixed point for the $S^1$-action. If $(A,\varphi )$ is simple, then there is an induced action of $S^1$ on $H^1_{(A,\varphi )}$.

Proof

For each $e^{i\theta }$, the derivative of its action on $\mathcal {C}$ defines a map $H^1_{(A,\varphi )}\rightarrow H^1_{(A,e^{i\theta }\varphi )}$. Composing with the inverse of the derivative of the unique gauge transformation $g(\theta )$ from Proposition A.10 we get a well defined map $H^1_{(A,\varphi )}\rightarrow H^1_{(A,\varphi )}$. Using the fact that $\theta \rightarrow g(\theta )$ is a group homomorphism it is easy to see that this gives an action of $S^1$. $\square$

If $(A,\varphi )$ has discrete stabilizer, then for each $\theta _0$ and each choice of gauge transformation $g(\theta _0)$ as in Proposition A.10, there is a unique smooth family $g(\theta )$ defined in a neighborhood of $\theta _0$. Taking $\theta _0=0$ and g(0) to be the identity we get the following result, by an argument similar to the proof of the preceding proposition.

Proposition A.12

Suppose $(A,\varphi )\in {\mathcal M}_{\mathrm {H}}^s(\mathrm {G})$ is a fixed point for the $S^1$-action. Then there is an induced local action of a neighborhood of the identity in $S^1$ on $H^1_{(A,\varphi )}$. In particular, there is an inifinitesimal $S^1$-action on $H^1_{(A,\varphi )}$, and a well-defined infinitesimal gauge transformation $\psi =\frac{dg_\theta }{d\theta }\big \vert _{\theta =0}\in \Omega ^0(P_{\mathrm {H}}[\mathfrak {h}])$.

Remark A.13

Note that $[\psi ,\varphi ] = i\phi$ because $g(\theta )\cdot (A,\varphi ) = (A,e^{i\theta }\varphi )$.

Now fix a maximal torus $\mathfrak {t}\subset \mathfrak {h}$. Since any element of $\mathfrak {h}$ is conjugate to an element in $\mathfrak {t}$, there is a point $p_0\in P_{\mathrm {H}}$ with the property stated in the following proposition.

Proposition A.14

Let $(A,\varphi )\in {\mathcal M}_{\mathrm {H}}^s(\mathrm {G})$ be a fixed point and let $\psi \in \Omega ^0(P_H[\mathfrak {h}])$ be the infinitesimal gauge transformation provided by Proposition A.12. Let $p_0\in P$ be such that the infinitesimal gauge transformation provided by Proposition A.12 satisfies $\psi (p_0)\in \mathfrak {t}$. Define

$$\begin{aligned} \mathrm {H}_0 = Z_{\mathrm {H}}(\psi (p_0))\subset \mathrm {H}. \end{aligned}$$

Then there is a subbundle $P_{\mathrm {H}_0}\subset P_{\mathrm {H}}$ which gives a reduction of structure group to $\mathrm {H}_0$.

Proof

Define

$$\begin{aligned} P_{\mathrm {H}_0}=\{p\in P_{\mathrm {H}}\;\;|\;\;\psi (p)\in \mathfrak {t}\} \subset P_{\mathrm {H}}. \end{aligned}$$

Let $\psi (p)\in \mathfrak {t}$. A point ${{\,\mathrm{Ad}\,}}(h)(\psi (p))=\psi (ph^{-1})$ in the adjoint orbit of $\psi (p)$ lies in $\mathfrak {t}$ if and only if $h\in Z_{\mathrm {H}}(\psi (p))$. Moreover, this centralizer does not depend on the choice of $\psi (p)$ in the adjoint orbit, as long as $\psi (p)$ lies in $\mathfrak {t}$. We therefore have an identification of the fiber $P_{H_0,x}$ of $P_{H_0}$ over $x=\pi (p_0)\in X$:

$$\begin{aligned} \mathrm {H}_0&\xrightarrow {\cong } P_{H_0,x},\\ c&\mapsto p_0\cdot c \end{aligned}$$

where the action comes from the right action of H on $P_{H_0}$.

Now note that, since $d_A\psi =0$, the eigenvalues for the action of $\psi$ on $P_{\mathrm {H}}[\mathfrak {h}]$ are constant. Hence the orbit in $\mathfrak {h}$ of $\psi (p)$ under the adjoint action of $\mathrm {H}$ is independent of $p\in P_{\mathrm {H}}$. It follows that the centralizer used in the preceding paragraph is the same for all fibers of $P_{\mathrm {H}}$ and, therefore, the construction globalizes to show that $P_{\mathrm {H}_0}\subset P_{\mathrm {H}}$ defines a reduction of structure group, as we wanted. $\square$

Remark A.15

Since the reduction $P_{\mathrm {H}_0}\subset P_{\mathrm {H}}$ just constructed only depends on the choice of the maximal torus $\mathfrak {t}\subset \mathfrak {h}$, it is unique up to conjugation by $\mathrm {H}$.

Proposition A.16

Suppose $(A,\varphi )\in {\mathcal M}_{\mathrm {H}}^s(\mathrm {G})$ is a fixed point for the $S^1$-action. Then there is a weight decomposition into ik-eigenspaces for the adjoint action of $\psi$ on the Lie algebra bundles $P_H[\mathfrak {h}^\mathbb {C}]$ and $P_H[\mathfrak {m}^\mathbb {C}]$:

$$\begin{aligned} P_\mathrm {H}[\mathfrak {h}^\mathbb {C}] = \bigoplus _k P_\mathrm {H}[\mathfrak {h}^\mathbb {C}]_{k} \quad \text {and}\quad P_\mathrm {H}[\mathfrak {m}^\mathbb {C}] = \bigoplus _k P_\mathrm {H}[\mathfrak {m}^\mathbb {C}]_{k}, \end{aligned}$$

where $\varphi \in H^0(P_H[\mathfrak {m}^\mathbb {C}]_1\otimes K)$ and $P_\mathrm {H}[\mathfrak {h}^\mathbb {C}]_{0}$ is identified with the adjoint bundle $P_{\mathrm {H}_0}[\mathfrak {h}^\mathbb {C}_0]$.

Proof

This is immediate from Proposition A.14—indeed, taking the weight space decomposition $\mathfrak {h}^\mathbb {C}=\bigoplus \mathfrak {h}^\mathbb {C}_k$ for the adjoint action of $\psi (p_0)$ we have $P_\mathrm {H}[\mathfrak {h}^\mathbb {C}]_{k} = P_{\mathrm {H}_0}[\mathfrak {h}^\mathbb {C}_{k}]$, and similarly for $\mathfrak {m}^\mathbb {C}$. The fact that $\varphi$ has weight one follows from Remark A.13. $\square$

Remark A.17

For any fixed $(A,\varphi )$ in the moduli space we can use the Jordan–Hölder reduction to a stable $\mathrm {G}'$-Higgs bundle to get a reduction of structure group as in Proposition A.14. However, the weight decomposition of Proposition A.16 is, in general, no longer well defined. This is because the center of the maximal compact $\mathrm {H}'\subset \mathrm {G}'$ may act non-trivially on the complement of ${\mathfrak {g}'}^\mathbb {C}$ in $\mathfrak {g}^\mathbb {C}$.

For a $({\mathcal E}, \varphi )\in {\mathcal M}^ s(\mathrm {G})$ which is fixed under the $\mathbb {C}^*$-action, the weight decomposition from Proposition  A.16 translates into

$$\begin{aligned} {\mathcal E}[\mathfrak {g}^\mathbb {C}] = {\mathcal E}[\mathfrak {h}^\mathbb {C}] \oplus {\mathcal E}[\mathfrak {m}^\mathbb {C}] = \bigoplus {\mathcal E}[\mathfrak {h}^\mathbb {C}]_k \oplus \bigoplus {\mathcal E}[\mathfrak {m}^\mathbb {C}]_k \end{aligned}$$

(A.6)

with ${\mathcal E}[\mathfrak {h}^\mathbb {C}]_{k} = P_\mathrm {H}[\mathfrak {h}^\mathbb {C}]_{k}$ and ${\mathcal E}[\mathfrak {m}^\mathbb {C}]_{k} = P_\mathrm {H}[\mathfrak {m}^\mathbb {C}]_{k}$, and where $\varphi \in H^0({\mathcal E}[\mathfrak {m}^\mathbb {C}]_1\otimes K)$. This gives a decomposition $C^\bullet ({\mathcal E},\varphi ) = \bigoplus C^\bullet _k({\mathcal E},\varphi )$ of the deformation complex (2.4), where

$$\begin{aligned} C^\bullet _k({\mathcal E},\varphi ): {\mathcal E}[\mathfrak {h}^\mathbb {C}]_{k}\xrightarrow {{{\,\mathrm{ad}\,}}_\varphi } {\mathcal E}[\mathfrak {m}^\mathbb {C}]_{k+1}\otimes K. \end{aligned}$$

(A.7)

Proposition A.18

Suppose $(A,\varphi )\in {\mathcal M}_{\mathrm {H}}^s(\mathrm {G})$ is a fixed point for the $S^1$-action. Let $H^1_{(A,\varphi )} = \bigoplus H^1_{(A,\varphi ),k}$ be the decomposition into ik-eigenspaces for the infinitesimal $S^1$-action given by Proposition A.12. Then there are canonical isomorphisms

$$\begin{aligned} H^1_{(A,\varphi ),k} \cong \mathbb {H}^1(C^\bullet _k(\mathcal {E}_A,\varphi )). \end{aligned}$$

Proof

In a similar way to Proposition A.12, there is an infinitesimal $S^1$-action on $\mathbb {H}^1(C^\bullet (\mathcal {E}_A,\varphi ))$ and, clearly, the isomorphism $H^1_{(A,\varphi )} \cong \mathbb {H}^1(C^\bullet (\mathcal {E}_A,\varphi ))$ of Proposition A.1 is $S^1$-equivariant. Thus there is a weight space decomposition $\mathbb {H}^1(C^\bullet (\mathcal {E}_A,\varphi ))=\bigoplus \mathbb {H}^1(C^\bullet (\mathcal {E}_A,\varphi ))_k$ with $H^1_{(A,\varphi ),k} \cong \mathbb {H}^1(C^\bullet (\mathcal {E}_A,\varphi ))_k$. It remains to see that $\mathbb {H}^1(C^\bullet (\mathcal {E}_A,\varphi ))_k \cong \mathbb {H}^1(C^\bullet _k(\mathcal {E}_A,\varphi ))$ and this is an easy check using the induced weight decomposition of the Dolbeault resolution (A.4). $\square$

We shall use the subscript “+” for the direct sums of subspaces with $k>0$.

Lemma A.19

Let $(A,\varphi )\in \mathcal {M}^s_{\mathrm {H}}(\mathrm {G})$. If $(A,\varphi )$ is a fixed point of the $S^1$-action, then it is a local minimum of the Hitchin function if and only if $H^1_{(A,\varphi ),k}=0$ for all $k>0$. Equivalently, a fixed point $({\mathcal E},\varphi )\in {\mathcal M}^s(\mathrm {G})$ for the $\mathbb {C}^*$-action is a local minimum of the Hitchin function if and only if $\mathbb {H}^1(C^\bullet _k(\mathcal {E},\varphi ))=0$ for all $k>0$.

Proof

Hitchin [34, 35] showed that on the smooth locus of ${\mathcal M}_{\mathrm {H}}(\mathrm {G})$, the subspace $H^1_{(A,\varphi ),k}$ can be identified with the $-k$-eigenspace for the Hessian of f. The extension to points of ${\mathcal M}_{\mathrm {H}}(\mathrm {G})$ which are orbifold singularities follows as in the proof of Lemma A.8. The equivalence of the statement for $({\mathcal E},\varphi )\in {\mathcal M}^s(\mathrm {G})$ follows from Proposition A.18. $\square$

We shall also need to show that certain $\mathrm {G}$-Higgs bundles which do not satisfy the hypothesis of Proposition 3.4 are not local minima of f. To this end we have the following result, analogous to a criterion of Simpson [46, Lemma 11.8].

Lemma A.20

Let $({\mathcal E}_0,\varphi _0)\in \mathcal {M}(G)$ be a fixed point of the $\mathbb {C}^*$-action. Suppose there exists a semistable $\mathrm {G}$-Higgs bundle $({\mathcal E},\varphi )$, which is not ${\mathcal S}$-equivalent to $({\mathcal E}_0,\varphi _0)$, and such that $\lim _{t\rightarrow \infty }({\mathcal E},t\varphi )=({\mathcal E}_0,\varphi _0)$ in $\mathcal {M}(G)$. Then $({\mathcal E}_0,\varphi _0)$ is not a local minimum of f.

Proof

Replacing $({\mathcal E},\varphi )$ with the polystable representative of its ${\mathcal S}$-equivalence class, we may assume that it is polystable. Note also that $({\mathcal E},\varphi )$ cannot be a fixed point of the $\mathbb {C}^*$-action.

Consider first the case when $({\mathcal E},\varphi )$ is stable. Then, as in the proof of Lemma A.8, we can use the moment map interpretation of f to deduce that the function $\mathbb {R}_{>0}\rightarrow \mathbb {R}$ defined by $t\mapsto f({\mathcal E},t\varphi )$ is strictly increasing as t tends to infinity. For the general case, consider the Jordan–Hölder reduction of $({\mathcal E},\varphi )$ given by Proposition A.6. This is a stable $\mathrm {G}'$-Higgs bundle for some $\mathrm {G}'\subset \mathrm {G}$ and cannot be fixed under the $\mathbb {C}^*$-action, since otherwise $({\mathcal E},\varphi )$ would also be fixed. Since the natural map $\mathcal {M}(\mathrm {G}')\rightarrow \mathcal {M}(\mathrm {G})$ is $\mathbb {C}^*$-equivariant and compatible with the respective Hitchin functions, the result follows by the same argument as in the previous paragraph. $\square$