1 Introduction

A natural request of a continuous group action \(G \times X \longrightarrow X\) is that it be equivariantly formal, meaning the fiber inclusion in the Borel fibration \(X \rightarrow X_G \rightarrow BG\) induces a surjection of Borel equivariant cohomology upon singular cohomology. While the term was only coined in 1997 by Goresky, Kottwitz, and MacPherson [28], the condition had already been alighted upon by Borel in Chapter XII of his Seminar [7]. This condition makes available a comparatively tractable computation of \({H^*_G}(X;\mathbb {Q})\) in terms of G-orbits of dimensions zero and one in the case there are only finitely many of each, as well as, by definition, guaranteeing all classes of \(H^*(X;\mathbb {Q})\) have equivariant extensions in \(H^*_G(X;\mathbb {Q})\), to which, for example, the localization theorems of Berline–Vergne/Atiyah–Bott [3, 4] and Jeffrey–Kirwan [35] can be applied.

As any orbit of a continuous action of a Lie group G on a space X, is a homogeneous space \(G/{{\mathrm{Stab }}}_G(x)\), it is natural to ask about equivariantly formal actions on such spaces. The transitive G-action is only equivariantly formal if the isotropy group \(K = {{\mathrm{Stab }}}_G(x)\) is of full rank, but some restriction of this action to a subgroup H will always be equivariantly formal. For this to happen, H cannot contain a strictly larger maximal torus than K does, so that the left action of K is in some sense the “largest” action on G / K which could conceivably be equivariantly formal. Assuming that G is compact, it is known that the isotropy action of K on G / K is equivariantly formal if K is of full rank in G ([10], Proposition 1), if \(H^*(G;\mathbb {Q}) \longrightarrow H^*(K;\mathbb {Q})\) is surjective ([47]. Thm. A, Cor. 4.2), or if (GK) is a generalized symmetric pair with K connected  [27], but otherwise few examples of such actions seem to be known. Nevertheless, the full-rank case has found wide application in symplectic geometry (see, e.g., the book of Ginzburg, Guillemin, and Karshon [25], in which equivariant cohomology is already mentioned in the first page of the introduction and occupies a thirty-one–page Appendix).

We show this question can be reduced to the case K is a torus. For concision, if the isotropy action of K on G / K is equivariantly formal, we call the pair (GK) isotropy-formal.

Theorem 1.1

IfG is a compact Lie group, K a closed, connected subgroup, and S any torus maximal within K, then (GK) is isotropy-formal if and only if (GS) is.

This result reduces the question to a study of embeddings of tori in Lie groups, an already more feasible-looking endeavor. Further, the question reduces to the case the commutator subgroup of G is simply-connected.

Theorem 1.2

Let G be a compact, connected Lie group, K a closed, connected subgroup, \(\widetilde{G}\) a finite central covering of G, and \(\widetilde{K}_0\) the identity component of the preimage of K in \(\widetilde{G}\). Then (GK) is isotropy-formal if and only if \((\widetilde{G},\widetilde{K}_0)\) is.

The question is largely dependent on the case G where itself is simply-connected.

Theorem 1.3

Let G be a compact, connected Lie group, \(G'\) its commutator subgroup, K a closed connected subgroup of G, and S a maximal torus in K. Write \(K' = K \cap G'\) and \(S' = S \cap G'\) for the intersections with \(G'\) and \(K'_0\) and \(S'_0\) for their respective identity components. Then (GK) is isotropy-formal if and only if

  1. 1.

    the pair \((G',K'_0)\) is isotropy-formal and

  2. 2.

    the inclusion induces an isomorphism of component groups.

These reductions are proven in Sect. 3, with some additional partial reductions having to do with disconnected goups and general compact Hausdorff groups expounded in Appendix B. The reductions achieved, in Sect. 4, we are able to completely determine for G any compact, connected Lie group and S any circular subgroup whether (GS) is isotropy-formal. A key condition turns out to be that there exist an element of the G conjugation by which acts as \(s \longmapsto s^{-1}\) on S. We say such an element reflectsS.

Proposition 1.4

Let G be a compact, connected Lie group and S a circular subgroup of G. There are the following three mutually exclusive cases.

  1. 1.

    The inclusion surjects in cohomology and S is not reflected in G.

  2. 2.

    The inclusion is trivial in cohomology and

    1. 2a.

      S is reflected in G.

    2. 2b.

      S is not reflected in G.

Only in the last case is (GS) not isotropy-formal.

Reflected circles can classified entirely, and from Propositions 1.4, 4.2, 4.5, and 4.6, one assembles the following result.

Theorem 1.5

Let G be a compact, connected Lie group and S a circular subgroup of G. If S is not contained in the commutator subgroup \(G'\) of G, then (GS) is isotropy-formal. Otherwise, we may assume by Theorem 1.2 that \(G'\) is a product of simple Lie groups \(K_j\). Pick for each a maximal torus containing the image \(S_j\) of . Then (GS) is isotropy-formal if and only if for each \(K_j\) there is an element of the Weyl group \(W(K_j)\) reflecting \(S_j\), which is determined as laid out in Table 1.

Table 1 Reflected circles in simple Lie groups

This table is compiled in Sect. 4.2.

Remark 1.6

(Explanatory remarks on Table 1) The notation J in the \(A_n\) case is the multiset of exponents \(a_1,\ldots ,a_{n} \in \mathbb {Z}\) such that the injection realizing a conjugate of S as a circular subgroup of the block-diagonal maximal torus of \(\mathrm {U}(n)\) is given by \(z \longmapsto {{\mathrm{diag}}}(z^{a_1},\ldots ,z^{a_n})\). We write \({-J}\) for the multiset \(\{-a_j\}_{1 \leqslant j \leqslant n}\) whose entries are the opposites of those of J; that is to say, for each \(a \in \mathbb {Z}\), the element \(-a\) occurs in \(-J\) with the same multiplicity that a occurs in J. For example, \([-1 \ 0 \ 1] \in \mathbb {Z}^3\) meets the condition \(J = -J\) and \([ 2\ 1\ -3]\) does not. See Corollary 4.13.

In the \(D_{2n+1}\) case, S is contained in a \(D_{2n}\) subgroup just if it is conjugate into a subtorus \(T^{2n} \times \{1\}\) of the standard maximal torus \(T^{2n+1}\) whose Lie algebra is the block-diagonal subspace \(\mathfrak {so}(2)^{\oplus 2n+1}\) of \(\mathfrak {so}(4n+2)\). See Corollary 4.16.

The condition that a circle in \(E_6\) be contained in a \(D_4\) subgroup manifests, within a given maximal torus \(T^6\) of \(E_6\), in a more intricate fashion. Precise statements are Proposition 4.20 and Remark 4.22.

As an example of Theorem 1.5, we can recover Shiga’s characterization ([47], Prop. 4.3) of circles in the unitary group yielding isotropy-formality.

Example 1.7

If S is a circle in the unitary group \(\mathrm {U}(n)\), then \(\big (\mathrm {U}(n),S\big )\) is or is not isotropy-formal as indicated in Table 2.

Table 2 The classification for circles in \(\mathrm {U}(n)\)

Corollary 1.8

(anonymous referee) Let G be a compact, connected Lie group and K a subgroup isomorphic to \(\mathrm {SO}(3)\) or \(\mathrm {SU}(2)\). Then (GK) is isotropy-formal.

Proof

This follows from Theorem 1.5 because the maximal torus \(S^1\) of K is contained in the commutator subgroup \(G'\) of G and is already reflected in K and hence a fortiori in G. \(\square \)

Alternate proof

Koszul ([39], 2.2\({}^{\mathrm o}\)) and Stiefel (unpublished) showed \(H^*G \longrightarrow H^*K\) is always surjective in this case (Samelson [46] derives this from the fact the Cartan 3-form given at the identity by \((u,v,w) \longmapsto B\big (u,[v,w]\big )\) is natural up to a scalar factor) so it follows ([47], Thm. A, Cor. 4.2) that (GK) is isotropy-formal. \(\square \)

A crucial step of in obtaining the key Proposition 1.4 is the following structure theorem for \(H^*(G/S)\), which turns out to mildly extend a result which can be pieced together from two Comptes Rendus notes of Leray and Koszul, a complete proof of which seems never to have been published. In case the result may be of independent interest, we take the opportunity to provide a proof in Appendix A.

Theorem 1.9

Let G be a compact, connected Lie group and S a circular subgroup.

  1. 1.

    If \(H^1 G \longrightarrow H^1 S\) is surjective, then \(H^*(G/S) \longrightarrow H^*G\) is injective and its image is the exterior algebra \(\Lambda \hat{P}\) on the intersection \(\hat{P}\) of \(\ker \big (H^*G \rightarrow H^*S)\) with the graded vector space P of primtive elements of the exterior Hopf algebra \(H^*G = \Lambda P\). Noncanonically, there is a \(z_1 \in H^1 G\) whose image spans \(H^1 S\) and

  2. 2.

    If \(H^1 G \longrightarrow H^1S\) is zero, then the image of \(H^*(G/S) \longrightarrow H^*G\) is the exterior algebra on a codimension-one subspace \(\hat{P}\) of P and \(P / \hat{P}\cong \mathbb {Q}z_3\) is graded in degree 3. The image of \({H^*_S}\longrightarrow H^*(G/S)\) is the subalgebra \(\mathbb {Q}[s]/(s^2)\) generated by a nonzero \(s \in H^2(G/S)\), and there are noncanonical isomorphisms

    $$\begin{aligned} H^*(G/S) \,\cong \, \Lambda \hat{P}\, \otimes \, \frac{\mathbb {Q}[s]}{(s^2)} \,\cong \, \frac{H^*G}{(z_3)} \, \otimes \, \frac{\mathbb {Q}[s]}{(s^2)}. \end{aligned}$$

2 Background

Associated to a continuous action of a topological group K on a space X,  ([7] IV.3.3, p. 53) is the (Borel)equivariant cohomology \({H^*_K}(X)\), the rational singular cohomology \(H^*(X_K;\mathbb {Q})\) of the homotopy quotient ([7], Def. IV.3.1, p. 52) (or Borel construction)

$$\begin{aligned} {{}_K X} = {X_K} {:}{=}\frac{EK \times X}{(ek,x) \sim (e,kx)}, \end{aligned}$$

where \(EK \rightarrow BK\) is a universal principal K-bundle. Until the last appendix, all cohomology will be singular cohomology with rational coefficients, which will henceforth be suppressed in the notation. We write \({{H^*_K}}\) for the coefficient ring \(H^*(BK) = {H^*_K}(\mathrm {pt})\). Associated to the homotopy quotient is a fiber bundle \(X \rightarrow {}_S X \rightarrow BS\), the Borel fibration. As noted in the introduction, an action of a topological group S on a space X is said to be equivariantly formal if the fiber inclusion in this fibration surjects in cohomology.Footnote 1 This condition is equivalent to the spectral sequence of this bundle collapsing at the \(E_2\) page ([25], Lem. C.24, p. 208). Given a Lie group G and closed subgroup K, we refer to the natural left K-action on the homogeneous space G / K of left cosets as the isotropy action. For brevity, when the isotropy action of K on G / K is equivariantly formal we call the pair (GK) isotropy-formal.

Given a Lie group G, we write Z(G) for its center, \({G'}\) for its commutator subgroup, \({G^{{\mathrm {ab}}}} {:}{=}G/G'\) for its abelianization, \({W_G}\) for its Weyl group, and \({N_G(K)}\) and \({Z_G(K)}\) respectively for the normalizer and the centralizer of a subgroup K in G. If S is a torus in G, we write \( N {:}{=}\pi _0 N_G(S)\) for the component group of its normalizer. We write \({h^\bullet (X)} {:}{=}\sum _{n \geqq 0} \dim _\mathbb {Q}H^n X\) for the total Betti number, and denote subgroup containment by “\(\leqslant \)”, isomorphism “\(\cong \)”, homotopy equivalence “\(\simeq \)”, and homeomorphism “\(\approx \)”.

2.1 Earlier work

As noted in the introduction, the question we are interested in could be asked in the late 1950s but only received a name in the 1990s. As of the beginning of this work, there were only the three known classes of cases in the introduction and the following general results of Shiga and Takahashi.

Theorem 2.1

(Hiroo Shiga [47]) Let G be a compact Lie group, K a closed, connected subgroup, and \(N_G(K)\) the normalizer. If (GK) is a Cartan pair and the map induced by is injective, then K acts equivariantly formally on G / K.

The notion of Cartan pair ([16], (3) on p. 70) here is not the notion due to Élie Cartan describing symmetric spaces, but an algebraic condition on the (Henri) Cartan model for G / K described in Appendix A which amounts to the space G / K being formal in the sense of rational homotopy theory. Visually, it corresponds to the tensor factorization \(E_2 = E_2^{\bullet ,0}\otimes E_2^{0,\bullet }\) in the Serre spectral sequence of the Borel fibration \(G \rightarrow {}_K G \rightarrow BK\) persisting to the \(E_\infty \) page. Shiga’s theorem can be equivalently restated as follows.

Proposition 2.2

(Shiga) Let G be a compact Lie group, K a closed, connected subgroup, and \(N_G(K)\) the normalizer. If (GK) is a Cartan pair and the map \(H^*_G \longrightarrow ({H^*_K})^{N_G(K)}\) is surjective, then K acts equivariantly formally on G / K.

The result also has a partial converse. In a later-written but earlier-published technical report [48], Shiga and Hideo Takahashi prove a partial converse.

Theorem 2.3

(Shiga–Takahashi) Let G be a compact group, S a toral subgroup, and \(N_G(S)\) the normalizer. Suppose that S contains regular elements of G and (GS) is a Cartan pair. Then S acts equivariantly formally on G / S if and only if and the map \(H^*_G \longrightarrow ({H^*_S})^{N_G(S)}\) is surjective.

In work with Chi-Kwong Fok [15], we show that if \({(G,K)}\) is isotropy-formal, then G / K must be formal, so the “Cartan pair” hypothesis is redudant. The hypothesis on regular elements is also unnecessary, and in further unpublished work [14], we show that S can also be replaced by any closed, connected subgroup K in the result. Although we do not need it in what follows, we state the strong version for here for reference.

Theorem 2.4

Let G be a Lie group, K a closed, connected subgroup, and \(N_G(K)\) the normalizer. Then \({(G,K)}\) is isotropy-formal if and only if G / K is formal and \(H^*_G \longrightarrow ({H^*_K})^{N_G(K)}\) is surjective.

Our trichotomy Proposition 1.4 about the case \(K \cong S^1\) can actually be refactored through the Shiga–Takahashi result. Noting that the regular element condition is unneeded, and that G / S is always formal for S a circle by the classical results of Appendix A, the Shiga–Takahashi Theorem 2.3 reduces isotropy-formality of \({(G,S)}\) to study of the map \({H^*_G}\longrightarrow {H^*_S}\). In this language, Proposition 1.4 can be reproven as follows: one has \(N = \pi _0 N_G(S)\) either trivial or \(\{\pm 1\}\). If it is trivial, then isotropy-formality is just that \({H^*_G}\longrightarrow {H^*_S}\) is surjective, which happens if and only if \(H^*(G) \longrightarrow H^*(S)\) surjects ([16], \(1^{\circ }\), p. 69) ([5], Cor., p. 139). Otherwise \(N \cong \{\pm 1\}\), meaning exactly that S is reflected in G (Proposition 4.1), and N acts as \(s \longmapsto \pm s\) on \(\mathbb {Q}[s] \cong H^*(BS)\), so that \(H^*(BS)^N = \mathbb {Q}[s^2]\); then one proves Lemma A.4 to see \(H^*(BG) \longrightarrow \mathbb {Q}[s^2]\) is always surjective.

The way this is presented in Sect. 4.1, we use a well-known fixed point criterion for equivariant formality (Lemma 3.8) and a computation of the vector space dimension of the cohomology of the fixed point set due to Goertsches (Proposition 3.11). Whether reasoning through a dimension count or through Theorem 2.3, one way or another the crux of it is understanding the cohomology of the maps \(S \rightarrow G \rightarrow G/S \rightarrow BS \rightarrow BG\).

3 Reductions

In this section we undertake a series of reductions that ultimately localizes most of the difficulty in determining which pairs \({(G,K)}\) are isotropy-formal in the case where G is semisimple and K a torus. Two further reductions, from disconnected to connected groups and from connected compact groups to Lie groups, only go through partially and are sequestered in Appendix B.

3.1 Compact total group

Let G be connected pro-Lie group and H a closed, connected subgroup. By the Cartan–Iwasawa–Malcev theorem, there exists a maximal compact subgroup \({K_H}\) of H, unique up to conjugacy ([32], Cor. 12.77), which is necessarily connected, such that there is a homeomorphism \(H \approx K_H \times \mathbb {R}^{\kappa }\) for some cardinal \(\kappa \) ([32], Cor. 12.82). Likewise G contains a maximal compact subgroup \({K_G}\), which after conjugation can be chosen to contain \(K_H\). In case G is a Lie group, at least, this yields a reduction result.

Proposition 3.1

Suppose G is a connected Lie group and H a connected, closed subgroup, with respective compact, connected subgroups \(K_G\) and \(K_H\), the one containing the other. Then (GH) is isotropy-formal if and only if \((K_G,K_H)\) is.

Proof

To identify the maps \(H^*_{K_H}(K_G/K_H) \longrightarrow H^*(K_G/K_H)\) and \(H^*_H(G/H) \longrightarrow H^*(G/H)\), it will be enough to see that in the commutative diagram

the horizontal maps are homotopy equivalences. A left–\(K_G\)-equivariant deformation retraction of G to \(K_G\) induces deformation retractions from \(G/K_H\) to \(K_G/K_H\) and from \({}_{K_H}G/K_H\) to \({}_{K_H}K_G/K_H\). The fibers of the bundles \(\delta \) and \(\epsilon \) are \(H/K_H\) and \((H/K_H) \times (H/K_H)\) respectively, both homeomorphic to Euclidean space, and \(G/K_H\) and G / H have the homotopy type of a CW complex so the long exact sequences of homotopy groups and Whitehead’s theorem show \(\delta \) and \(\epsilon \) are homotopy equivalences. \(\square \)

Remark 3.2

This proof of Proposition 3.1 depends only on homotopy equivalence, so the statement remains the same if \(H^*\) is replaced in the definition of isotropy-formality by any contravariant homotopy functor.

3.2 Toral isotropy

To reduce to toral isotropy actions, we require some well-known isomorphisms and the rarely remarked fact these isomorphisms are natural.

Let \(\xi _0:E_0 \rightarrow B_0\) be a fibration with homotopy fiber F such that \(\pi _1 B_0\) acts trivially on \(H^*F\). We can form a slice category of fibrations over \(\xi _0\) with homotopy fiber F by taking as objects maps of fibrations \(\xi \rightarrow \xi _0\) with homotopy fiber F and as morphisms between \(\xi ' \rightarrow \xi _0\) and \(\xi \rightarrow \xi _0\) maps of fibrations \(\xi ' \rightarrow \xi \) making the expected triangle commute up to homotopy. Such a morphism entails a homotopy-commutative prism

(3.1)

Lemma 3.3

([49], Cor. 4.4, p. 88) Let \(\xi _0:E_0 \rightarrow B_0\) be a fibration such that the fiber inclusion is surjective in cohomology and \(\pi _1 B\) acts trivially on \(H^*B\). Then the fiber inclusion of any fibration \(\xi : E \rightarrow B\) over \(\xi _0\) with homotopy fiber F is surjective in cohomology, and there is an \(H^*E_0\)-algebra isomorphism

$$\begin{aligned} {{{H^*B }\underset{H^*B_0 }{\otimes }{H^*E_0 } } \overset{\sim }{\longrightarrow }H^*E} \end{aligned}$$

natural in the fibration \(\xi \) over \(\xi _0\).

We prove the result so as justify the naturality clause we will need, absent in the original.

Proof

Surjectivity of \(H^*E \longrightarrow H^*F\) is implied by that of \(H^*E_0 \longrightarrow H^*F\) since the fiber inclusion \(F \longrightarrow E_0\) factors up to homotopy as \(F \rightarrow E \rightarrow E_0\). For the isomorphism, note that because of these surjections, the Serre spectral sequences of these fibrations collapse at the \(E_2\) page. Thus the ring map \(H^*B \otimes _{H^*B_0 } H^*E_0 \longrightarrow H^*E \) induced by the maps in the right square of (3.1) is equivalent on the level of \(H^*B_0 \)-modules to the canonical isomorphism

$$\begin{aligned} {{H^*B }\underset{H^*B_0 }{\otimes }{\big (H^*B_0 \otimes H^*F\big )} } \overset{\sim }{\longrightarrow }H^*B \otimes H^*F, \end{aligned}$$

and so is itself an isomorphism. For naturality, note that the ring map \(h^*:H^*E \longrightarrow H^*E'\) is completely determined its restrictions to its tensor-factors \(H^*B\) and \(H^*E_0\) and that the commutative diagrams in cohomology induced by the left square and top triangle of (3.1) respectively imply these restrictions are \({\bar{h}}^* :H^*B \longrightarrow H^*B'\) and \({{\mathrm{id}}}_{H^*E_0 }\). \(\square \)

The naturality in the following lemma follows from the standard proof by noting that a K-equivariant map \(X \longrightarrow Y\) yields commutative squares

Lemma 3.4

([34], Lemma III.1.1, p. 35) Let K be a compact, connected Lie group with maximal torus S and Weyl group W, and X a free K-space. Then there is a ring isomorphism, natural in X,

$$\begin{aligned} H^*( X / K) \overset{\sim }{\longrightarrow }H^*(X/S)^W. \end{aligned}$$

Lemma 3.5

([34], Prop. III.1, p. 38) Let K be a compact, connected Lie group with maximal torus S and Weyl group W. Then there are the following ring isomorphisms natural in X:

$$\begin{aligned} {H^*_K}(X)&\overset{\sim }{\longrightarrow }&{H^*_S}(X)^W,\\ {{{H^*_S}}\underset{{H^*_K}}{\otimes }{{H^*_K}(X)} }&\overset{\sim }{\longrightarrow }&{H^*_S}(X). \end{aligned}$$

Proof

The first statement follows from Lemma 3.4 and the definitions. The second follows from Lemma 3.3, applied to the K / S-bundle \(X_S \rightarrow X_K\) viewed as a bundle over \(BS \rightarrow BK\); alternately, as \(W_K\) acts on \(H^2_S\) as a reflection group, \({H^*_S}\) is a free module over \({H^*_K}\cong ({H^*_S})^{W_K}\) by the Chevalley–Shephard–Todd theorem ([36], p. 192) and Corollary B.3 applies. \(\square \)

Corollary 3.6

Let K be a compact, connected Lie group with maximal torus S and \(X \rightarrow Y\) a K-equivariant map. Then \(\varkappa _K:{H^*_K}Y \longrightarrow {H^*_K}X\) is surjective if and only if \(\varkappa _S:{H^*_S}Y \longrightarrow {H^*_S}X\) is.

Proof

Lemma 3.5 identifies \(\varkappa _K\) with the map of Weyl-invariants \((\varkappa _S)^W\) and \(\varkappa _S\) with the base extension \({{\mathrm{id}}}_{{H^*_S}} \otimes _{{H^*_K}} \varkappa _S\). If \(\varkappa _S\) is surjective, then it follows by averaging that \(\varkappa _K\) is as well, since \(\varkappa _S\) is W-equivariant and |W| is invertible in \(\mathbb {Q}\). On the other hand if \(\varkappa _K\) is surjective, then since the functor \({H^*_S}\otimes _{{H^*_K}} -\) is right exact, \(\varkappa _K\) is surjective as well. \(\square \)

Finally, the following well-known lemma follows from the preceding ones.

Lemma 3.7

([25], Prop. C.26, p. 207) If K is a compact, connected Lie group and S a maximal torus, and K acts on a space X, then the action of K is equivariantly formal if and only if the restricted action of S is.

We can now prove the promised reduction.

Theorem 1.1IfGis a compact Lie group,Ka closed, connected subgroup, andSany torus maximal withinK, then (GK) is isotropy-formal if and only if (GS) is.

Proof

By Lemma 3.7, it is enough to show that K acts equivariantly formally on G / S if and only if it does on G / K. To do so, we may apply Corollary 3.6 to the map of right K-spaces \(G \longrightarrow {}_K G\). \(\square \)

3.3 The dimension criterion

Equivariant formality can be reduced to a condition on total Betti numbers.

Lemma 3.8

([7, Prop. XII.3.4, p. 164], [26, Prop. 3.1, p. 81]) An action of a torus S on a topological space X with finite total Betti number is equivariantly formal if and only if \(h^\bullet (X) = h^\bullet (X^S)\).

For later reference, note one inequality always holds:

Lemma 3.9

(Borel [7, IV.5.5, p. 62]) ([25, Lem. C.24]) If a torus S acts on a topological space X with finite total Betti number, then \(h^\bullet (X) \geqslant h^\bullet (X^S)\).

Let G be a compact Lie group and S a torus in G. As the fixed point set of the left action of S on G / S is the quotient group \(N_G(S)/S\) of the normalizer, we need to know when \(h^\bullet (G/S)= h^\bullet \big (N_G(S)/S\big )\). The latter number is easily expressed in terms of other quantities. Recall that we denote by \(Z_G(S)\) the centralizer of S in G, by \({W_K}\) the Weyl group of K, and by N the component group \(\pi _0 N_G(S)\).

Lemma 3.10

Conjugation induces a natural injection . This induces homeomorphisms \(N_G(S) \approx N\times Z_G(S)\) and \((G/S)^S = N_G(S)/S \approx N\times Z_G(S)/S\). If K is a closed, connected subgroup with maximal torus S, there is a further homeomorphism \((G/K)^S = N_G(S)K/K \approx (N / W_K) \times Z_G(S)/S\). Particularly, \(h^\bullet \big ((G/K)^S\big ) = 2^{{{\mathrm{rk }}}G - {{\mathrm{rk }}}K} \cdot |N|/|W_K|\).

Proof

The centralizer \(Z_G(S)\) is connected since it is the union of the maximal supertori of S in G. As \(Z_G(S)\) is the kernel of the continuous homomorphism \(n \longmapsto (x \mapsto nxn^{-1})\) from \(N_G(S)\) into the discrete group \({{\mathrm{Aut}}}S \cong {{\mathrm{Aut}}}{\mathbb {Z}^{{{\mathrm{rk }}}S}}\), it is the identity component of \(N_G(S)\). Thus \(N = N_G(S)/Z_G(S)\); the homeomorphisms follow because group components are homeomorphic. As for K, one notes \(\pi : (G/S)^S \longrightarrow (G/K)^S\) can be equivalently written as the surjection with fibers \(nN_K(S)/S = nW_K\). It follows \((G/K)^S\) has \(|N|/|W_K|\) components, each homeomorphic to \(Z_G(S)/\big (Z_G(S)\cap K\big ) = Z_G(S)/S\). But since \(Z_G(S)/S\) is a compact, connected Lie group, \(H^*\big (Z_G(S)/S\big )\) is an exterior algebra on \({{\mathrm{rk }}}G - {{\mathrm{rk }}}S\) generators by Hopf’s theorem ([33], Satz I, p. 23), \(\square \)

Proposition 3.11

(Goertsches–Noshari ([27, Props. 2.1, 3.1]) Let G be a compact, connected Lie group and K a closed, connected subgroup. Write \(N = \pi _0 N_G(S)\). Then (GK) is isotropy-formal if and only if

$$\begin{aligned} h^\bullet (G/K)\leqslant 2^{{{\mathrm{rk }}}G -{{\mathrm{rk }}}K} \cdot \frac{|N|}{|W_K|}. \end{aligned}$$

Proof

Let S be a maximal torus of K. By Lemma 3.7, we may replace the K-action on G / K with the S-action. By Lemmas 3.8 and 3.9 this action is equivariantly formal if and only if \(h^\bullet (G/K)\leqslant h^\bullet \big ((G/K)^S\big )\), which is \(2^{{{\mathrm{rk }}}G - {{\mathrm{rk }}}K} \cdot { |N|}/{|W_K|}\) by Lemma 3.10. \(\square \)

3.4 Torus–cross–simply-connected total group

The structure theorem for compact, connected Lie groups  ([11], Thm. V.(8.1) & Ex. V.(8.7).6, p. 233, 238) states that each admits a finite central extension \({p}:{\widetilde{G}} \longrightarrow G\) such that the abelianization exact sequence \(1 \rightarrow {\widetilde{G}'} \rightarrow \widetilde{G} \rightarrow \widetilde{G}^{{\mathrm {ab}}}\rightarrow 0\) splits on the level of topological groups. If the kernel of p is F, we can write \( G \cong \widetilde{G}/ F . \) The total space \(\widetilde{G}\) (but not p itself, if \(A \ne 0\)) is uniquely determined up to isomorphism.

In determining which toral isotropy actions are equivariantly formal, we will show we can replace G with \(\widetilde{G}\) and the connected isotropy subgroup K (which we can take to be a torus) with the identity component \({\widetilde{K}_0}\) of its preimage \({\widetilde{K}} = p^{-1}K = F\widetilde{K}_0\).

Proposition 3.12

These assumptions induce isomorphisms \({H^*(G/K) \overset{\sim }{\,}{\rightarrow } H^*(\widetilde{G}/\widetilde{K})}\)\({\overset{\sim }{\rightarrow }H^*(\widetilde{G}/\widetilde{K}_0)} \).

This is a result of the following lemma and the homeomorphism \(\widetilde{G}/\widetilde{K}\overset{\approx }{\longrightarrow }G/K\).

Lemma 3.13

Let \(\Gamma \) be a path-connected topological group, F a central subgroup, and H another subgroup such that \(FH/H\) is finite. Then the covering \(FH/H \rightarrow \Gamma /H \rightarrow \Gamma /FH\) induces an isomorphism \(H^*(\Gamma /FH) \overset{\sim }{\longrightarrow }H^*(\Gamma /H)\).

Proof

As F is central, the covering action of \(f H \in f H/H\) is given by \({\gamma }{H} \cdot f H = {\gamma }{} \textit{fH} = {f}{\gamma }{H}\), left multiplication by f. But \(\Gamma \) being path-connected, left translation by its any element is homotopic to the identity. Thus ([30], Prop. 3G.1)

$$\begin{aligned} H^*(\Gamma /FH) \cong H^*(\Gamma /H)^{FH/H} = H^*(\Gamma /H). \end{aligned}$$

\(\square \)

The components of the normalizer are also preserved under this substitution.

Proposition 3.14

Under the foregoing assumptions, the projection \(p:\widetilde{G}\longrightarrow G\) induces an isomorphism \(N_{\widetilde{G}}(\widetilde{K}_0)/Z_{\widetilde{G}}(\widetilde{K}_0) \overset{\sim }{\longrightarrow }N_G(K)/Z_G(K)\). Particularly, if S is a torus, \(\pi _0 N_{\widetilde{G}}(\widetilde{S}_0) {=}{:}{\widetilde{N}} \cong N = \pi _0 N_G(S)\).

Proof

As p is a homomorphism, it sends \({N_{\widetilde{G}}(\widetilde{K}) \longrightarrow N_G(K)}\). We show this restriction is surjective and the preimage of \(Z_G(K)\) is \({Z_{\widetilde{G}}(\widetilde{K})}\). For surjectivity, given \(\widetilde{w}\in p^{-1}N_G(\widetilde{K}_0)\), note \(\widetilde{w}1 \widetilde{w}^{-1}= 1\) and \(p(\widetilde{w}\widetilde{K}_0 \widetilde{w}^{-1}) = K\), so \({\widetilde{w}\widetilde{K}_0\widetilde{w}^{-1}= \widetilde{K}_0}\). For the preimage, note that if \(\widetilde{z} \in p^{-1}Z_G(K)\), then \(\widetilde{z}\widetilde{k}\widetilde{z}^{-1}\widetilde{k}^{-1}\in \ker p\) for each \(\widetilde{k} \in \widetilde{K}_0\); since \(\ker p\) is discrete and \(\widetilde{z} 1 \widetilde{z}^{-1}1^{-1}= 1\), such a \(\widetilde{z}\) centralizes \(\widetilde{K}_0\). \(\square \)

These facts in hand, we conclude the proof of Theorem 1.2.

Theorem 1.2LetGbe a compact, connected Lie group,Ka closed, connected subgroup,\(\widetilde{G}\)a finite central covering ofG, and\(\widetilde{K}_0\)the identity component of the preimage ofKin\(\widetilde{G}\). Then (GK) is isotropy-formal if and only if\((\widetilde{G},\widetilde{K}_0)\)is.

Proof

Let S be a maximal torus of K and \(\widetilde{S}_0\) its connected lift in \(\widetilde{K}\). We know from Proposition 3.11 that (GK) is isotropy-formal if and only if

$$\begin{aligned} h^\bullet (G/K) = 2^{{{\mathrm{rk }}}G - {{\mathrm{rk }}}S} |N| / |W_K|, \end{aligned}$$

and the analogous statement holds of \((\widetilde{G},\widetilde{K})\). But evidently \({{\mathrm{rk }}}\widetilde{G}= {{\mathrm{rk }}}G\) and \({{\mathrm{rk }}}\widetilde{K}= {{\mathrm{rk }}}K\) and \(W_K \cong W_{\widetilde{K}}\); from Proposition 3.12, we know \(h^\bullet (\widetilde{G}/\widetilde{K}) = h^\bullet (G/K)\); and from Proposition 3.14, we know \(\widetilde{N}\cong N\). \(\square \)

In what follows we can therefore replace G with a cover \(\widetilde{G}= \widetilde{G}' \times \widetilde{G}^{{\mathrm {ab}}}\). For later, when we specialize to circles, we note the following corollary of Proposition 3.14.

Corollary 3.15

Under these hypotheses, the torus S is reflected in G just if \(\widetilde{S}\) is reflected in \(\widetilde{G}\).

3.5 Semisimple total group

In this section, G is a connected, compact Lie group, \(G'\) again its commutator subgroup, and \(G^{{\mathrm {ab}}}\) its abelianization. To separate out information about \(G'\), we will need another covering lemma similar in spirit to Lemma 3.13.

Lemma 3.16

Let \(\Gamma \) be a compact, connected Lie group, \(\Xi \) an abelian subgroup, and S a torus in \(\Xi \) such that \(\Xi /S\) is finite. Then the covering \(\Xi /S \rightarrow \Gamma /S \rightarrow \Gamma /\Xi \) induces an isomorphism \(H^*(\Gamma /S) \overset{\sim }{\longrightarrow }H^*(\Gamma /\Xi )\).

Proof

As \(\Xi \) is abelian, it is contained in the centralizer \(Z_\Gamma (S)\), which is path-connected, so that its right action on \(\Gamma /S\) is cohomologically trivial. Thus \( H^*(\Gamma /\Xi ) \cong H^*(\Gamma /S)^{\Xi /S} = H^*(\Gamma /S). \)\(\square \)

Given subgroup H of G, the canonical short exact sequence \(G' \rightarrow G \rightarrow G^{{\mathrm {ab}}}\) descends to a fiber bundle .

Proposition 3.17

If H is connected, this bundle has the cohomology of a trivial bundle.

Proof

Consider a finite central cover of the form \(\widetilde{G} = \widetilde{G}' \times \widetilde{G}^{{\mathrm {ab}}}\). Let \(\widetilde{H}\) be the full preimage of H in \(\widetilde{G}\) and \(\widetilde{H}_0\) its identity component. We will show \(\widetilde{G}'/ (\widetilde{G}' \cap \widetilde{H}_0) \rightarrow \widetilde{G}/\widetilde{H}_0 \rightarrow {{\mathrm{coker }}}(\widetilde{H}_0 \rightarrow \widetilde{G}^{{\mathrm {ab}}})\) is a trivial bundle. Then the Künneth theorem will yield the desired ring decomposition, for \({{\mathrm{coker }}}(H \rightarrow G^{{\mathrm {ab}}})\) and \({{\mathrm{coker }}}(\widetilde{H}_0 \rightarrow \widetilde{G}^{{\mathrm {ab}}})\) are tori of the same dimension, and \(H^*(G/H) \cong H^*(\widetilde{G}/ \widetilde{H}_0)\) by Proposition 3.12, while is a normal covering with covering action induced by right translation by central elements of \(\widetilde{G}\), so by Proposition 3.12 again, \(H^*\big (G'/(G' \cap H)\big ) \cong H^*\big (\widetilde{G}'/(\widetilde{G}' \cap \widetilde{H}_0)\big )\).

The short exact sequence \({{\mathrm{im }}}(\widetilde{H}_0 \rightarrow \widetilde{G}^{{\mathrm {ab}}}) \rightarrow \widetilde{G}^{{\mathrm {ab}}}\rightarrow {{\mathrm{coker }}}(\widetilde{H}_0 \rightarrow \widetilde{G}^{{\mathrm {ab}}})\) of tori splits on the level of topological groups. Replacing \(\widetilde{G}^{{\mathrm {ab}}}\) with the product in the expression \(\widetilde{G}= \widetilde{G}' \times \widetilde{G}^{{\mathrm {ab}}}\), the projection of \(\widetilde{H}_0\) to the cokernel component is trivial, so \(\widetilde{G}/\widetilde{H}_0\) is the direct product of \({{{\mathrm{coker }}}(\widetilde{H}_0 \rightarrow \widetilde{G}^{{\mathrm {ab}}})}\) and \({\big (\widetilde{G}' \times {{\mathrm{im }}}(\widetilde{H}_0 \rightarrow \widetilde{G}^{{\mathrm {ab}}})\big )/\widetilde{H}_0}\). But the inclusion of \(\widetilde{G}'/ (\widetilde{G}' \cap \widetilde{H}_0)\) into the latter is a continuous bijection of compact Hausdorff spaces, hence a homeomorphism. \(\square \)

Now we can carry through the claimed near-reduction to the semisimple case.

Theorem 1.3LetGbe a compact, connected Lie group, \(G'\)its commutator subgroup, Ka closed connected subgroup ofG, andSa maximal torus inK. Write\(K' = K \cap G'\)and\(S' = S\, \cap \, G'\)for theintersections with\(G'\)and\(K'_0\)and\(S'_0\) for their respectiveidentity components. Then (GK) is isotropy-formal if and only if

  1. 1.

    the pair \((G',K'_0)\) is isotropy-formal and

  2. 2.

    the inclusioninduces an isomorphism of component groups.

Proof

Note that \(S'_0\) is a maximal torus in \(K'_0\), so by Theorem 1.1 it is enough to show (GS) is isotropy-formal if and only if \((G',S'_0)\) is and the condition on normalizers holds.

From the decomposition \(G = G' \cdot Z(G)\), it follows that \(N_G(\Gamma ) = N_{G'}(\Gamma )\cdot Z(G)\) and \(Z_G(\Gamma ) = Z_{G'}(\Gamma )\cdot Z(G)\) for any subgroup \(\Gamma \), so that particularly \(\pi _0 N_G(S'_0) \cong \pi _0 N_{G'}(S'_0) {=}{:}{N'}\). As \(G'\) is normal in G, there is also a containment \(N_G(S) \leqslant N_G(S'_0)\), and so an induced monomorphism . Thus from Lemma 3.16, Borel’s Lemma 3.9 for the action of \(S'_0\) on \(G'/S'_0\) and Lemma 3.10, we see

$$\begin{aligned} h^\bullet (G'/S') = h^\bullet (G'/S'_0) \geqslant |N'|\, 2^{{{\mathrm{rk }}}G' - {{\mathrm{rk }}}S'} \geqslant |N|\, 2^{{{\mathrm{rk }}}G' - {{\mathrm{rk }}}S'}. \end{aligned}$$
(3.2)

Because rank is additive under direct products,

$$\begin{aligned} {{\mathrm{rk }}}G - {{\mathrm{rk }}}S&= \big ({{\mathrm{rk }}}G' + {{\mathrm{rk }}}Z(G)\big ) - \big ( {{\mathrm{rk }}}S'_0 + {{\mathrm{rk }}}{{\mathrm{im }}}(S \rightarrow G^{{\mathrm {ab}}})\big )\\&= {{\mathrm{rk }}}G' - {{\mathrm{rk }}}S'_0 + {{\mathrm{rk }}}{{\mathrm{coker }}}(S \rightarrow G^{{\mathrm {ab}}}), \end{aligned}$$

so multiplying (3.2) by \(2^{{{\mathrm{rk }}}{{\mathrm{coker }}}(S \,\rightarrow \, G^{{\mathrm {ab}}})}\) yields, by Proposition 3.17,

$$\begin{aligned} h^\bullet (G/S) \geqslant |N'|\,2^{{{\mathrm{rk }}}G - {{\mathrm{rk }}}S} \geqslant |N|\,2^{{{\mathrm{rk }}}G - {{\mathrm{rk }}}S}. \end{aligned}$$
(3.3)

Proposition 3.11 states that (GS) is isotropy-formal if and only if the inequalities (3.3) are in fact equalities, which is equivalent to (3.2) being equalities. But by Proposition 3.11 again, this can only happen if \((G',S')\) is isotropy-formal and \(N' \leftrightarrow N\). \(\square \)

Remark 3.18

It can really happen that the inequality \(|N|\leqslant |N'|\) is strict. For instance, let \(G = A \times G'\) for \(A = S^1\) and \(G' = \mathrm {SU}(2)^{2}\), pick a circle \(S^1\) in \(\mathrm {SU}(2)\), and let T be the maximal torus \((S^1)^3\) of G and \( {S = \big \{(z,w,zw^{-1}) : z,w \in S^1\big \}} \) a rank-two subtorus, so that \(S' = S'_0 = {\big \{(1,w,w^{-1}) : w \in S^1 \big \}}\). Then \(N' = W_{\mathrm {SU}(2)} \cong \mathbb {Z}/2\) but \(N= 1\).

4 Circular isotropy

Now we can tackle the case S is a circle. This section demonstrates the statements of Theorem 1.5 and Table 1 regarding equivariant formality of circle actions.

4.1 The trichotomy

Let \(S \cong S^1\) be a circle subgroup of a compact, connected Lie group G.

Proposition 4.1

Then the cardinality of \(\pi _0N_G(S)\) is 2 if S is reflected in G and 1 otherwise.

Proof

This follows from Lemma 3.10 since \(s \longmapsto s^{-1}\) is the only nontrivial continuous automorphism of \(S^1\). \(\square \)

As \(\widetilde{H}^* S^1 = H^1 S^1\) is one-dimensional, \(H^*G \longrightarrow H^*S\) is either surjective or trivial.

Proposition 4.2

The inclusion is trivial in cohomology if and only if S is contained in the commutator subgroup \(G'\), if and only if the map induced in \(H^1\) by \(S \rightarrow G \rightarrow G^{{\mathrm {ab}}}\) is trivial.

Proof

Since \(G'\) is the kernel of \(G \longrightarrow G^{{\mathrm {ab}}}{=}{:}A\), it contains S just if the composition \(S \rightarrow G \rightarrow A\) is trivial. If so, then of course the map \(H^1 A \longrightarrow H^1 S\) is trivial. If \(S \rightarrow G \rightarrow A\) is nontrivial, then its image is a circle, so the induced map \(\pi _1 S \longrightarrow \pi _1 A\) is nonzero and hence injective, and so \(H^1 A \longrightarrow H^1 S\) is surjective. But this map is nontrivial just if \(H^1 G \longrightarrow H^1 S\) is since \(H^1 A \longrightarrow H^1 G\) is an isomorphism, as can be seen for example by using Proposition 3.12 to pass to a finite cover \(\widetilde{A}\times \widetilde{G'}\) with \(0 = \pi _1 \widetilde{G'} = H^1 \widetilde{G'} = H^1 G'\). \(\square \)

We can now prove Proposition 1.4.

Proposition 1.4

Let G be a compact, connected Lie group and S a circular subgroup of G. There are the following three mutually exclusive cases.

  1. 1.

    The inclusion surjects in cohomology and S is not reflected in G.

  2. 2.

    The inclusion is trivial in cohomology and

    1. 2a.

      S is reflected in G.

    2. 2b.

      S is not reflected in G.

Only in the last case is (GS) not isotropy-formal.

Proof

Recall from Proposition 3.11 that (GS) is isotropy-formal just when \(h^\bullet (G/S) \leqslant |N|\, 2^{{{\mathrm{rk }}}G -{{\mathrm{rk }}}S}\). Theorem 1.9 imposes the constraint that \({h^\bullet (G/S)\in }\)\({\big \{\frac{1}{2} h^\bullet (G), h^\bullet (G)\big \}}\) and Proposition 4.1 that \(|N|\in \{1,2\}\). By Lemma 3.9, it is impossible that both \(h^\bullet (G/S)= \frac{1}{2} h^\bullet (G)\) and \(|N|= 2\) simultaneously, so there are only the following three cases.

  1. 1.

    We have \(h^\bullet (G/S)= \frac{1}{2} h^\bullet (G)\) and \(|N|= 1\). The action is equivariantly formal.

  2. 2.

    We have \(h^\bullet (G/S)= h^\bullet (G)\), and

    1. 2a.

      \(|N|= 2\). The action is equivariantly formal.

    2. 2b.

      \(|N|= 1\). The action is not equivariantly formal. \(\square \)

It remains to determine when \(|N| = 2\), or in other words when S is reflected in G.

4.2 Classification of reflected circles

In this section, we determine what circular subgroups S of compact, connected Lie groups G are reflected. First, we may assume S lies in some fixed maximal torus of T, since all maximal tori are conjugate and for any \(g \in G\) one has \(g N_G(S)g^{-1}= N_G(gSg^{-1})\). Further, we may represent reflections by Weyl group elements, in that \(N\leqslant {{\mathrm{Aut}}}S\) is naturally a quotient of \(N_W(S) \leqslant W\).

Lemma 4.3

([9, Exercise IX.2.4, p. 391], [20, Lemma 9.7, p. 20]) Let G be a compact, connected Lie group, T a maximal torus, and S a subtorus. Any automorphism of S induced from conjugation by an element of \(N_G(S)\) is also induced by an element of \(N_G(T) \cap N_G(S)\).

Precisely, the inclusion induces maps

Corollary 4.4

A toral subgroup S is reflected in a compact, connected Lie group G if and only if some element of the Weyl group W of G acts as \(s \longmapsto s^{-1}\) on S.

From Corollary 3.15, we may replace G with the product \(A \times G'\) of a torus A and a simply-connected Lie group \(G'\), but A is irrelevant:

Proposition 4.5

A toral subgroup S is reflected in a compact, connected Lie group G if and only if it lies in and is reflected in the commutator subgroup \(G'\).

Proof

Since the conjugation action of A is trivial, circles reflected by G are already reflected by \(G'\). From Propositions 4.2 and 1.4, we know any reflected S in G is contained in \(G'\). \(\square \)

Reflectibility of a torus in a semisimple group H in turn depends only on simple factors.

Proposition 4.6

A toral subgroup S is reflected in a product \(\prod H_j\) of Lie groups if and only if each of its images \(S_j\) under the factor projections to \(H_j\) is reflected in \(H_j\).

Proof

Since the homomorphisms preserve conjugacy and inversion, if \((h_j) \in \prod H_j\) reflects S, then \(h_j\) reflects \(S_j\). On the other hand, if some \(h_j \in H_j\) reflects each \(S_j\), then \((h_j)\) reflects \(\prod S_j\), which contains S. \(\square \)

We can in fact restrict attention to a single element of the Weyl group.

Proposition 4.7

A circular subgroup S is reflected in a simple Lie group H if and only if it is reflected by the longest word \(w_0\) in the Weyl group W of H.

Proof

If C is the closed Weyl chamber containing a given nonzero element \(v \in {\mathfrak {s}}< {\mathfrak {t}}\), then \(-v\) lies the “opposite” closed Weyl chamber \(-C\). The orbit \(W\cdot v\) meets \(-C\) in exactly one point ([1], Thm. 5.16), which must be \(w_0 \cdot v\) since \(w_0 \cdot C = -C\), so \({\mathfrak {s}}\) is reflected if and only if \(w_0 \cdot v = - v\). \(\square \)

There is a representation-theoretic restatement of the same condition.

Corollary 4.8

A circular subgroup S is reflected in a simple Lie group H if and only if the irreducible representation of H determined by S is self-dual.

Proof

Identify \({\mathfrak {t}}\) with its dual \({\mathfrak {t}}^{\vee }\) through the W-invariant inner product and let \(\lambda \) be an additive generator of the intersection of \({\mathfrak {s}}\) with the weight lattice of H. Then S is reflected if and only if \(w_0\cdot \lambda = -\lambda \). But the dual to the irreducible representation with highest weight \(\lambda \) is that with highest weight \(-w_0\cdot \lambda \). \(\square \)

Remark 4.9

The original proof of the classification in Table 1 was unnecessarily intricate and involved a computer algebra verification at one point, and has been greatly simplified through the arguments in Proposition 4.7 and Corollary 4.8, due to Jay Taylor [51] and Chi-Kwong Fok (personal communication).

To construct Table 1 we march case by case through the Killing–Cartan classification.

Proposition 4.10

A maximal torus T of a simple compact Lie group G whose type is one of

$$\begin{aligned} B_n,\quad C_n,\quad D_{2n},\ \ \ G_2,\quad F_4,\quad E_7,\quad E_8 \end{aligned}$$

is reflected in G.

Proof

The longest word \(w_0\) acts as \(-{{\mathrm{id}}}\) on the vector space \({\mathfrak {t}}\) carrying the defining representation of W precisely for Coxeter groups W of these types  ([36], Lem. 27–2, p. 283) so T is reflected by Proposition 4.7. Alternately, but relatedly, central involutions of a Weyl group W reflect the maximal torus T  ([21], Thm. 1.8) and the center of W is isomorphic to \(\mathbb {Z}/2\) precisely for Coxeter groups W of these types  ([21], Rmk. 1.9). \(\square \)

In the remaining cases, the longest word \(w_0 \in W\) does not act as \(-{{\mathrm{id}}}\) on \({\mathfrak {t}}\), so more work is required.

Proposition 4.11

A circular subgroup S is reflected in a simple Lie group H whose Weyl group has trivial center (viz. one of type \(A_n\), \(D_{2n+1}\), or \(E_6\)) if and only if there is some \(w \in W\) such that \(w\cdot {\mathfrak {s}}\) lies in the fixed point subalgebra of the Cartan subalgebra under an automorphism induced by a nontrivial diagram automorphism of the Dynkin diagram of H.

Proof

From Proposition 4.7 we know S is reflected if and only if \({\mathfrak {s}}\) is fixed pointwise by the nontrivial automorphism \(-w_0 \in {{\mathrm{Aut}}}{\mathfrak {t}}\). As \(w_0 = {{\mathrm{Ad }}}(n_0)\) for some \(n_0 \in N_H(T)\), we can extend \(-w_0\) to \(-{{\mathrm{Ad }}}(n_0) \in {{\mathrm{Aut}}}{\mathfrak {k}}\). Outer automorphisms of \(\mathfrak k\) are induced ([24], Prop. D.40, p. 498) by graph automorphisms of the Dynkin diagram \(\Gamma \) of H in the sense that \(({{\mathrm{Aut}}}{\mathfrak {k}})/({{\mathrm{Ad }}}H) \cong {{\mathrm{Aut}}}\Gamma \). Since W acts simply transitively on Weyl chambers, and \(-w_0\) stabilizes but does not fix the positive closed Weyl chamber C, the automorphism \(-{{\mathrm{Ad }}}(n_0)\) of \({\mathfrak {k}}\) is not inner and hence its outer isomorphism class corresponds to a nontrivial automorphism \(\theta \) of \(\Gamma \). This means the induced \(\theta \in {{\mathrm{Aut}}}{\mathfrak {t}}\) is the restriction of \(-{{\mathrm{Ad }}}(n_0 k) \in {{\mathrm{Aut}}}{\mathfrak {k}}\) for some \(k \in N_H(T)\), so that \(\theta \) fixes \({{\mathrm{Ad }}}(k^{-1}) {\mathfrak {s}}\). \(\square \)

It thus remains to find the fixed point subalgebras of nontrivial diagram automorphisms for Lie algebras of type \(A_n\), \(D_{2n+1}\), and \(E_6\). In all of these proofs, we use the fact that the W-equivariant isomorphism \({\mathfrak {t}}^{\vee }\overset{\sim }{\longrightarrow }{\mathfrak {t}}\) induced by the invariant inner product is also equivariant with respect to \(\theta = -w_0\), and so identifies the fixed point subspaces \(({\mathfrak {t}}^{\vee })^\theta \) and \({\mathfrak {t}}^\theta \).

Fig. 1
figure 1

The graph involution of \(A_n\)

Proposition 4.12

In a Lie algebra of type \(A_n\), a point \(v \in {\mathfrak {t}}^{\vee }< \mathbb {R}^{n+1}\) of the dual Cartan algebra is fixed by the automorphism \(\theta \) of Figure 1 if and only if a permutation of the coordinates of v yields \(-v\).

Proof

The diagram automorphism \(\theta \) acts on simple roots of \(A_n\) by exchanging \(\alpha _j \longleftrightarrow \alpha _{n-j}\). The \(\theta \)–fixed point subspace of \({\mathfrak {t}}^{\vee }\) is spanned by the sums \(\alpha _j + \alpha _{n-j}\) and so consists of those vectors \(\sum c_j \alpha _j \in {\mathfrak {t}}^{\vee }\) for which \(c_j = c_{n-j}\). The \(\alpha _j\) are usually identified with \(e_j - e_{j+1} \in \mathbb {R}^{n+1}\), where \((e_\ell )_{1 \leqslant \ell \leqslant n+1}\) is the standard basis and the resulting embedding takes

$$\begin{aligned} \sum c_j \alpha _j \longmapsto \big [ c_1\ \ (c_2-c_1)\ \ \cdots \ \ (c_n-c_{n-1}) \ \ -c_n\big ] {=}{:}\sum v_\ell e_\ell , \end{aligned}$$

translating the symmetry requirement \(c_j = c_{n-j}\) to the antisymmetry condition \(v_\ell = -v_{n+1-\ell }\). \(\square \)

Corollary 4.13

A circular subgroup S is reflected in \(\mathrm {SU}(n)\) if and only if the exponent multiset J of the inclusion of any conjugate of S into the standard maximal torus T satisfies \(J = -J\).

Proof

Let v span the tangent space \({\mathfrak {s}}< {\mathfrak {t}}\). Recalling the Weyl group \(W_{A_n} = S_{n+1}\) acts on \(\mathbb {R}^{n+1}\) by permuting coordinates, by Proposition 4.12 a permutation of the entries of v yields \(-v\) just if some \(w \in W_{A_n}\) sends v into \({\mathfrak {t}}^\theta \), and by Proposition 4.11, S is reflected just if this occurs. \(\square \)

Remark 4.14

(a) The root subsystems of \(A_{2\ell }\) and \(A_{2\ell -1}\) fixed by \(\theta \) are respectively of types \(B_n\) and \(C_n\), corresponding to the inclusions and respectively induced by the ring injections and . These subgroups are fixed points of involutive automorphisms of \(\mathrm {SU}(n)\) yielding the symmetric spaces \(\mathrm {SU}(n)/\mathrm {SO}(n)\) and \(\mathrm {SU}(2n)/\mathrm {Sp}(n)\).

(b) In terms of the self-duality criterion Corollary 4.8, the representation \(\tau \) of S on \(\mathbb {C}^n\) given by restricting the defining representation of \(\mathrm {SU}(n)\) to S is a direct sum \({\bigoplus _{j=1}^n \rho ^{\otimes a_j}}\) of tensor powers of the defining representation , and the dual representation \(\tau ^{\vee }= {\bigoplus _{j=1}^n \rho ^{\otimes (-a_j)}}\), will be isomorphic to \(\tau \) just if \(J = -J\).

Fig. 2
figure 2

The graph involution of \(D_{2n + 1}\)

Proposition 4.15

In a Lie algebra of type \(D_{2n+1}\), a point \(\lambda \in {\mathfrak {t}}^{\vee }\) of the dual Cartan algebra is fixed by an automorphism of the Dynkin diagram if and only if the last coordinate of \(\lambda \) is zero.

Proof

The nontrivial graph automorphism \(\theta \) of the Dynkin diagram of \(D_{2n+1}\), shown in Fig. 2, fixes all simple roots except \(\alpha _{2n}\) and \(\alpha _{2n+1}\), which it exchanges. The fixed point subspace of \(({\mathfrak {t}}^{\vee })^\theta \) is spanned by \(\{\alpha _j\}_{j < 2n} \cup \{ \alpha _{2n} + \alpha _{2n+1} \} \). The roots \(\alpha _j\) for \(j \leqslant 2n\) are usually identified with \(e_j - e_{j+1} \in \mathbb {R}^{2n+1}\) and \(\alpha _{2n+1}\) with \(e_n + e_{n+1}\), where \((e_j)_{1 \leqslant j \leqslant n+1}\) is again the standard basis. The image of the composite embedding is \(\mathbb {R}^{2n} \times \{0\}\) since \(\alpha _{2n} + \alpha _{2n+1} = 2e_{2n}\). \(\square \)

Corollary 4.16

A circular subgroup S is reflected in \(\mathrm {Spin}(4n+2)\) if and only if it is conjugate into a \(\mathrm {Spin}(4n)\) subgroup.

Proof

Let v span the tangent space \({\mathfrak {s}}< {\mathfrak {t}}= \mathbb {R}^{2n+1}\). Recalling the Weyl group \(W_{D_{2n+1}} = \{\pm 1\}^{2n} \rtimes S_{2n+1}\) acts on \(\mathbb {R}^{2n+1}\) by permuting its coordinates and negating an even number of them, by Proposition 4.15 some entry of v is 0 just if some \(w \in W_{A_n}\) sends v into \({\mathfrak {t}}^\theta \), and by Proposition 4.11, S is reflected just if this occurs. \(\square \)

Remark 4.17

The sublattice of a \(D_{2n+1}\) lattice fixed by \(\theta \) is of type \(B_{2n}\) and corresponds to a \(\mathrm {Spin}(4n)\) subgroup of \(\mathrm {Spin}(4n+2)\), the fixed point set of an involutive automorphism of \(\mathrm {Spin}(4n+2)\) yielding the symmetric space \(V_2(\mathbb {R}^{4n+2}) = \mathrm {Spin}(4n+2)/\mathrm {Spin}(4n) = \mathrm {SO}(4n+2)/\mathrm {SO}(4n)\).

Fig. 3
figure 3

The graph involution of \(E_6\)

Proposition 4.18

In a Lie algebra of type \(E_{6}\), a point \(\lambda \in {\mathfrak {t}}^{\vee }\) of the dual Cartan algebra is fixed by the nontrivial automorphism of the Dynkin diagram if and only if it lies in a certain \(F_4\) sublattice.

Proof

The fixed-point subspace \(({\mathfrak {t}}^{\vee })^\theta \) of the nontrivial automorphism \(\theta \) of the Dynkin diagram of \(E_6\) depicted in Fig. 3 is spanned by \(\Delta = \{\alpha _1 + \alpha _6,\alpha _2+\alpha _5,\alpha _3,\alpha _4\}\). By assumption, we have \(\alpha _i \cdot \alpha _j = -2 |\alpha _i| |\alpha _j|\) for adjacent \(\alpha _i,\alpha _j\) and \(= 0\) otherwise, so \(\Delta \) is a simple root system of type \(F_4\) with \(a_1 + \alpha _6\) and \( \alpha _2+\alpha _5\) long and \(\alpha _3\) and \(\alpha _4\) short. \(\square \)

Proposition 4.19

A circular subgroup S is reflected in \(E_6\) or its universal cover \(\widetilde{E}_6\) if and only if it is conjugate into a \(\mathrm {Spin}(8)\) subgroup.

Proof

It follows from Proposition 4.11 and Proposition 4.18 that the tangent lines \({\mathfrak {s}}\) to reflected circles S are precisely those sent into \({\mathfrak {t}}^\theta \) by some \(w \in W_{E_6}\). As \(({\mathfrak {t}}^{\vee })^\theta \) is spanned by an \(F_4\) sublattice of the \(E_6\) root lattice, its dual \({\mathfrak {t}}^\theta \) is tangent to the maximal torus \(T^4\) of an \(F_4\) subgroup. In the series of inclusions \(\mathrm {Spin}(8)< F_4 < E_6\), the first two share a maximal torus \(T^4\), so \({\mathfrak {t}}^\theta \) is actually tangent to the maximal torus of a \(\mathrm {Spin}(8)\). \(\square \)

It may be of interest to count these four-dimensional tori.

Proposition 4.20

Within any given maximal torus \(T^6\) of \(E_6\) or \(\widetilde{E}_6\), there are forty-five distinct Weyl-conjugate maximal tori \(T^4\) of \(\mathrm {Spin}(8)\) subgroups, all reflected.

Proof

The \(\mathrm {Spin}(8)\) tangent to \(T^4\) corresponds to a \(D_4\) sublattice of \({\mathfrak {t}}\) spanning \({\mathfrak {t}}^\theta \). Within a set of positive roots for a root system of type \(D_4\), it is not hard to check there are precisely three spanning sets of mutually orthogonal roots, so the number of tori in question will be a third of the number of sets of four mutually orthogonal roots in the root system \(\Phi (E_6)\). Any given set\(\{\alpha ,\beta ,\gamma ,\delta \}\) of four mutually orthogonal positive roots in \(\Phi (E_6)\) corresponds to \(\left| \{\pm 1\}^4 \rtimes S_4 \right| = 384\) different mutually orthogonal ordered quadruples of arbitrary roots, so the number of tori \(T^4\) can be obtained by dividing the number of such quadruples by \(384\cdot 3 = 1152 = \big |W_{F_4}\big |\). We will then be done if we can show \(W_{E_6}\), which is of cardinality \(51,840 = 45\cdot 1152\), acts simply transitively on mutually orthogonal ordered quadruples \((\alpha ,\beta ,\gamma ,\delta )\) in \(\Phi (E_6)\).

For this, Carter observes ([17], Lem. 11.(i), p. 14) that \(W_{E_6}\) acts transitively on roots \(\alpha \in \Phi (E_6)\), that \({{\mathrm{Stab }}}_{W_{E_6}} \alpha \) acts transitively on the \(A_5\) subsystem of roots \(\beta \) orthogonal to \(\alpha \), and that \({{\mathrm{Stab }}}_{W_{E_6}} (\alpha ,\beta )\) acts transitively on the \(A_3\) subsystem the roots \(\gamma \) orthogonal to both \(\alpha \) and \(\beta \), so that \(W_{E_6}\) acts transitively on mutually orthogonal ordered triples \((\alpha ,\beta ,\gamma )\). From there we may further see \({{{\mathrm{Stab }}}_{W_{E_6}} (\alpha ,\beta ,\gamma )}\) acts transitively on the \(A_1\) subsystem \(\{\pm \delta \}\) of roots orthogonal to all of \(\alpha ,\beta ,\gamma \). That the transitivity on quadruples is simple follows, since \(\big |\Phi (A_1)\big | = 2\), from repeated applications of the orbit–stabilizer theorem:

$$\begin{aligned} \underbrace{51,840}_{{|W_{E_6}|}} = \underbrace{720}_{|{{\mathrm{Stab }}}\alpha |}\cdot \underbrace{72}_{|\Phi (E_6)|} = \underbrace{24}_{|{{\mathrm{Stab }}}(\alpha ,\beta )|} \cdot \, \underbrace{30}_{|\Phi (A_5)|} \cdot 72 = \underbrace{2}_{|{{\mathrm{Stab }}}(\alpha ,\beta ,\gamma )|} \cdot \, \underbrace{12}_{|\Phi (A_3)|} \cdot \; 30 \cdot 72. \end{aligned}$$

\(\square \)

Remark 4.21

If we view \(T^4\) as the maximal torus of \(F_4 < E_6\), it follows from the equation \(|W_{E_6}| = 45\cdot |W_{F_4}|\) that \(W_{F_4}\) injects into \(W_{E_6}\) as the normalizer of \(T^4\). The author is advised this result can be understood from Carter’s book ([18], Sec. 13.3).

Remark 4.22

A standard system of simple roots for \(E_6\) in \(\mathbb {R}^5 \times \mathbb {R}^3\) is given   ([8], Planche V, p. 260) by

These roots span the six-dimensional subspace \( \big (\mathbb {R}^5 \,\times \, \{ 0\}^{ 3}\big ) \; +\,\ \mathbb {R}\cdot [1 \ \ 1 \ \ 1 \ \ 1 \ \ 1; 1\ \ 1\ \ 1] \) of \(\mathbb {R}^8\) and one obtains a system \(\Phi \) of 72 roots obtained from permutation of the first five coordinates of

$$\begin{aligned} \begin{aligned} \zeta ,\quad \gamma _{12},\quad \delta _{12},\quad {\eta _{12}} {:}{=}\zeta -\gamma _{12},\quad {\epsilon _1}&{:}{=}\zeta - 2\gamma _{12} + 2\delta _{12} + 3\delta _{23} + 2 \delta _{34}+ \delta _{45}\\&\,= \textstyle \frac{1}{2}\,[1\ {-1}\ {-1}\ -1\ -1;\ 1\ \ 1\ \ 1]. \end{aligned} \end{aligned}$$

and multiplication by \(\pm 1\). We may choose the positive roots \(\Phi ^+\) to be the 36 in the union of the following 135 maximal mutually orthogonal sets:

$$\begin{aligned} \begin{array}{llll} (60) &{}&{} \{\epsilon _a,\eta _{ab},\gamma _{ac},\delta _{de}\}, &{} \qquad \text{ where } \big |\{a,b,c,d,e\}\big | = 5 \text{ and } d<e,\\ (30) &{}&{} \{\eta _{ab},\eta _{cd},\gamma _{ac},\gamma _{bd}\}, &{} \qquad \text{ where } \big |\{a,b,c,d\}\big | = 4,\\ (15) &{}&{} \{\eta _{ab},\eta _{cd},\delta _{ab},\delta _{cd}\}, &{} \qquad \text{ where } \big |\{a,b,c,d\}\big | = 4 \text{ and } a<b \text{ and } c<d,\\ (15) &{}&{} \{\gamma _{ab},\gamma _{cd},\delta _{ab},\delta _{cd}\}, &{} \qquad \text{ where } \big |\{a,b,c,d\}\big | = 4 \text{ and } a<b \text{ and } c<d,\\ (15) &{}&{} \{\zeta ,\epsilon _a,\delta _{bc},\delta _{de}\}, &{} \qquad \text{ where } \big |\{a,b,c,d,e\}\big | = 5 \text{ and } b<c \text{ and } d<e. \end{array} \end{aligned}$$

These 135, found by brute force, form bases of the tangent spaces to the 45 tori figuring in Proposition 4.20, and each torus is reflected by the product of the four corresponding root reflections.

For example, the span \(\mathbb {R}^4 \,\times \, \{0\}^{ 4}\) of \(\{\gamma _{12},\delta _{12},\gamma _{34},\delta _{34}\}\) meets \(\Phi ^+\) in \(\{\gamma _{ab}, \delta _{ab} :1 \leqslant a < b \leqslant 4\}\). Among these, the roots orthogonal to \(\delta _{ab}\) are \(\{\gamma _{ab},\gamma _{cd},\delta _{cd}\}\) (where \(\big |\{a,b,c,d\}\big | = 4\)) and likewise the roots orthogonal to \(\gamma _{ab}\) are \(\{\delta _{ab},\gamma _{cd},\delta _{cd}\}\), so the spanning quadruples are determined by the (three) partitions of \(\{1,2,3,4\}\) into pairs of pairs \(\big \{\{a,b\},\{c,d\}\big \}\).