Signature characters of invariant Hermitian forms on irreducible Verma modules and Hall–Littlewood polynomials

Abstract

The Unitary Dual Problem is one of mathematics’ most important open problems: classify the irreducible unitary representations of a group. The general approach has been to classify all representations admitting non-degenerate invariant Hermitian forms, compute the signatures of those forms, and then determine which forms are positive definite. Signature character algorithms and formulas arising from deforming representations and analysing changes at reducibility points, as in Adams et al. (Unitary representations of real reductive groups (ArXiv e-prints), 2012) and Yee (Represent Theory 9:638–677, 2005), produce very complicated formulas or algorithms from the resulting recursion. This paper shows that in the case of irreducible Verma modules all of the complexity can be encapsulated by the affine Hecke algebra: for compact real forms and for alcoves corresponding to translations of the fundamental alcove by a regular weight, signature characters of irreducible Verma modules are in fact “negatives” of Hall–Littlewood polynomial summands evaluated at $q=-\,1$ times a version of the Weyl denominator, establishing a simple signature character formula and drawing an important connection between signature characters and the affine Hecke algebra. Signature characters of irreducible highest weight modules are shown to be related to Kazhdan-Lusztig basis elements. This paper also handles noncompact real forms. The current state of the art for the unitary dual is a computer algorithm for determining if a given representation is unitary. These results suggest the potential to move the state of the art to a closed form classification for the entire unitary dual.

1 Introduction

In the 1930s, Gelfand introduced a programme in abstract harmonic analysis that is a grand generalization of Fourier analysis (see the series of survey articles at the end of [8] for the history of his work). The idea is to attach to a problem a corresponding algebraic problem which may be solved by decomposing it into simpler (possibly infinitely many) problems. This permits the solution of difficult problems in diverse areas of mathematics. Realizing Gelfand’s programme motivates the Unitary Dual Problem: classify the irreducible unitary representations of a group. The unitary dual is known only for a limited selection of groups.

A common approach to classifying unitary representations is to first classify the Hermitian representations (those admitting an invariant Hermitian form), then compute the signatures of the invariant Hermitian forms on the Hermitian representations, and then determine which of those forms are positive or negative definite. Signatures may be expressed using signature characters, and determining if a form is positive definite is equivalent to finding if the signature character and the character are the same. Important philosophies for computing signature characters were developed in [28] and [29].

Standard limit representations may be constructed from one-dimensional representations of the Cartan subalgebra by cohomological induction (see [1, Definition 8.18]). In many cases the construction is equivalent to applying a Bernstein functor to a Verma module [14, Theorem 0.50]. Irreducible Harish-Chandra modules are linear combinations of standard limit representations, thus it is important to study Verma modules. While it is known that under certain conditions in a more general setting, applying Zuckerman functors to unitary representations preserves unitarity (see [28, 29]), unitary representations may also arise from the application of Zuckerman functors to non-unitary representations. Thus it is important to know signature characters of non-unitary highest weight modules as well. Thus signature characters of invariant Hermitian forms on Hermitian irreducible Verma modules and Hermitian irreducible highest weight modules were computed in [29,30,31,32].

The papers [31] and [32] show that the signature character of an invariant Hermitian form on an irreducible highest weight module can be expressed in terms of the signature characters of invariant Hermitian forms on irreducible Verma modules and signed Kazhdan–Lusztig polynomials which are Kazhdan–Lusztig polynomials evaluated at $-1$ up to a sign. Similar formulas for Harish-Chandra modules are obtained in the impressive preprint [2]. Unfortunately, signature characters of invariant Hermitian forms on irreducible Verma modules are indexed by alcoves of the affine Weyl group and the signature character formula in [30] depends on a choice of alcove path and appears complicated: it involves powers of 2, products of signs, and a summation over subsets of reflections in the alcove path chosen. (Please see Theorem 3.16.) Fortunately, the main result Theorem 7.9 of this paper simplifies the description of the signature character by showing that when the real form is compact and the highest weight is regular and in a translation of the fundamental alcove, the signature character of the invariant Hermitian form on the irreducible Verma module is equal to the “negative” of a summand of a corresponding Hall–Littlewood polynomial evaluated at $q=-1$ times a version of the Weyl denominator. For alcoves of other forms, the signature character of the invariant Hermitian form on the irreducible Verma module can be expressed as a sum of Hall–Littlewood polynomial summands at $q=-1$ times a version of the Weyl denominator. Signature characters of invariant Hermitian forms on irreducible highest weight modules are sums of signature characters of invariant Hermitian forms on irreducible Verma modules. Therefore signature characters of invariant Hermitian forms on irreducible highest weight modules can be expressed in terms of sums of “negatives” of summands of Hall–Littlewood polynomials evaluated at $q=-1$ times a version of the Weyl denominator and Kazhdan–Lusztig polynomials evaluated at $-1$. The signature characters of irreducible highest weight modules are shown to be related to Kazhdan–Lusztig basis elements.

We use the following approach to prove the main theorem. First, we simplify the formula for the signs appearing in the signature character formula for invariant Hermitian forms on irreducible Verma modules. Then we study the formula for Hall–Littlewood polynomials in terms of alcove walks of Schwer and Ram [25, 27]. We establish a connection between the summands which appear in the signature character formula and the summands which appear in the Hall–Littlewood formula by establishing that the summands that appear in the former formula correspond to positively folded alcove walks. We then use a result from Matthew Dyer’s thesis [6] which allows us to use R polynomials to show that the signature character formula and the “negative” of a summand of the corresponding Hall–Littlewood polynomial evaluated at $q=-1$ are the same after multiplication of the latter by a version of the Weyl denominator.

The paper is structured as follows. We begin by introducing notation in Sect. 2. In Sect. 3, we provide background material on signature characters of invariant Hermitian forms on irreducible Verma modules. In Sect. 4, we simplify the signature character formula for irreducible Verma modules by simplifying the signs and products of signs that appear in it. Next, we review the affine Hecke algebra and Hall–Littlewood polynomials in Sect. 5. In Sect. 6, we discuss Schwer and Ram’s formula for Hall–Littlewood polynomials in terms of alcove walks and establish some basic relations to the signature character formula. In Sect. 7, we introduce a formula of Dyer and use it to show that when the real form is compact, for alcoves of the form $w(-\lambda + \text {dominant fundamental alcove})$, the signature character formula and the “negative” of a summand of the corresponding Hall–Littlewood polynomial are the same (up to multiplication by a version of the Weyl denominator). This is accomplished by evaluating a summation of R polynomials. In Sect. 8, we express signature characters for irreducible Verma modules of highest weight corresponding to alcoves of the form $w( -\lambda + x (\text {dominant fundamental alcove)})$ in terms of “negatives” of summands of Hall–Littlewood polynomials evaluated at $q=-1$. In Sect. 9, we discuss the case of singular highest weight. In Sect. 10, we treat the case where the real form is noncompact. We use the simplified formula for irreducible Verma modules to write down formulas for signature characters for irreducible highest weight modules in Sect. 11.

2 Notation

In [30,31,32], the focus of the papers was signature characters and it was convenient to index Verma modules using antidominant weights and the Weyl group and to use the antidominant fundamental Weyl chamber and alcove. In this paper, we also discuss Hall–Littlewood polynomials, where it is standard to use the dominant fundamental Weyl chamber and alcove. Therefore we introduce notation for both choices.

Notation 2.1

We fix the following notation for this paper:

• $\mathfrak {g}_0$ is a real semisimple Lie algebra

• $\theta$ is a Cartan involution on $\mathfrak {g}_0$ inducing the decomposition $\mathfrak {g}_0 = \mathfrak {k}_0 \oplus \mathfrak {p}_0$

• $\mathfrak {h}_0 = \mathfrak {t}_0 \oplus \mathfrak {a}_0$ is the Cartan decomposition of a $\theta$-stable Cartan subalgebra

• omitting the subscript 0 indicates complexification

• ${\bar{\cdot }}$ applied to elements of $\mathfrak {g}$ and $\mathfrak {h}^*$ denotes complex conjugation relative to the real form $\mathfrak {g}_0$

• $\mathfrak {b}= \mathfrak {h}\oplus \mathfrak {n}$ is a Borel subalgebra giving positive roots $\Delta ^+( \mathfrak {g}, \mathfrak {h})$ and $\mathfrak {g}= \mathfrak {n}\oplus \mathfrak {h}\oplus \mathfrak {n}^-$ is the corresponding triangular decomposition

• let $\Pi =\{\alpha _1, \cdots , \alpha _m\}$ be the simple roots of $\Delta ^+( \mathfrak {g}, \mathfrak {h})$ and let $\alpha _0$ be the root for which $\alpha _0^\vee$ is the sum of all the highest coroots for each simple component of $\mathfrak {g}$

• let $s_i = s_{\alpha _i}$ and $S = \{ s_1, \ldots , s_m \}$

• $\Lambda _r$ is the root lattice and $\Lambda _r^+$ is the set of non-negative integral linear combinations of the simple roots

• $\Lambda$ is the weight lattice and $\lambda _1, \ldots , \lambda _m$ the fundamental weights

• $\rho$ is one half the sum of the positive roots and $\rho ^\vee$ is one half the sum of the positive coroots

• W is the Weyl group, $\mathfrak {C}_0$ the antidominant Weyl chamber, and ${\underline{\mathfrak {C}}}_0$ the dominant Weyl chamber

• $w_0$ is the long element of W

• for $w \in W$, $\Delta (w^{-1}) = \{ \alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h}): w^{-1} \alpha < 0 \}$

• $\lambda \in \mathfrak {h}^*$ is a dominant weight

• for $\mu \in \mathfrak {h}^*$, $W_\mu$ is the stabilizer of $\mu$ in W and $W^\mu$ is the set of minimal length coset representatives of $W / W_\mu$ where $\mu$ is dominant or antidominant

• for $\mu \in \mathfrak {h}^*$, $M(\mu ) = U(\mathfrak {g}) \otimes _{U(\mathfrak {b})} \mathbb {C}_{\mu -\rho }$ is the Verma module of highest weight $\mu - \rho$ with canonical generator $v_{\mu -\rho }$

• $H_{\alpha ,n}$ denotes the affine hyperplane $H_{\alpha ,n} = \{ \mu \in \mathfrak {h}_0^* : (\mu ,\alpha ^\vee ) = n \}$ where $\alpha \in \Delta (\mathfrak {g},\mathfrak {h})$ and $n \in \mathbb {Z}$ and $s_{\alpha ,n}$ is the corresponding affine reflection

• $H_{\alpha ,n}^+$ and $H_{\alpha ,n}^-$ denote the half-spaces $H_{\alpha ,n}^+ = \{ \mu \in \mathfrak {h}_0^* : (\mu , \alpha ^\vee ) > n \}$ and $H_{\alpha ,n}^- = \{ \mu \in \mathfrak {h}_0^* : (\mu , \alpha ^\vee ) < n\}$

• $W_a$ is the affine Weyl group generated by the $s_{\alpha ,n}$, $A_\circ = \cap _{\alpha \in \Pi } H_{\alpha ,0}^- \cap H_{\alpha _0, -1}^+$, ${\underline{A}}_\circ = \cap _{\alpha \in \Pi } H_{\alpha ,0}^+ \cap H_{\alpha _0, 1}^-$. (It is the affine Weyl group for the dual root system $\Delta ^\vee ( \mathfrak {g}, \mathfrak {h}) = \{ \alpha ^\vee : \alpha \in \Delta ( \mathfrak {g}, \mathfrak {h}) \}$.)

• ${\bar{\cdot }}: W_a \rightarrow W$ is the group homomorphism induced by the semidirect product structure $W_a = \Lambda _r \rtimes W$. Note that ${\bar{s}}_{\alpha ,n} = s_\alpha$.

• $S_a = S \cup \{ s_{\alpha _0,-1}\}$ and ${\underline{S}}_a = S \cup \{ s_{\alpha _0, 1} \}$

3 Signature characters for irreducible Verma modules

We review background on invariant Hermitian forms, signatures, and signature characters of invariant Hermitian forms on irreducible Verma modules. A more detailed explanation of the material in this section may be found in [30].

Definition 3.1

Given a representation V of the complex semisimple Lie algebra $\mathfrak {g}$, a Hermitian form $\langle \cdot , \cdot \rangle : V \times V \rightarrow \mathbb {C}$ is an invariant Hermitian form on V if

$$\begin{aligned} \langle X \cdot v, w \rangle = -\langle v, {\bar{X}} \cdot w \rangle \end{aligned}$$

for all $v, w \in V$ and all $X \in \mathfrak {g}$. Note that invariance depends on the real form due to complex conjugation with respect to the real form.

Definition 3.2

Given a weight $\mu \in \mathfrak {h}^*$, define the complex conjugate${\bar{\mu }}$ of $\mu$ by ${\bar{\mu }}(H) = \overline{\mu ({\bar{H}})}$ for all $H \in \mathfrak {h}$. The weight $\mu$ is real if ${\bar{\mu }} = \mu$, imaginary if ${\bar{\mu }} = -\mu$, and complex if it is neither real nor imaginary.

Definition 3.3

Given $\mu \in \mathfrak {h}^*$, define $\theta \mu \in \mathfrak {h}^*$ by $(\theta \mu )(H) = \mu ( \theta ^{-1}(H))$ for all $H \in \mathfrak {h}$.

For $\alpha \in \Delta (\mathfrak {g},\mathfrak {h})$, because $\mathfrak {h}_0$ is $\theta$-stable, $\alpha$ is imaginary-valued on $\mathfrak {t}_0$ and real-valued on $\mathfrak {a}_0$. Thus $\theta \mu = -{\bar{\mu }}$ for $\mu \in \Lambda _r$, so imaginary roots are supported on $\mathfrak {t}$, real roots on $\mathfrak {a}$, and complex roots on both. (See [13] for details.)

Remark 3.4

If $\mathfrak {g}_0$ is equal rank, then $\mathfrak {h}$ may be chosen so that $\mathfrak {h}= \mathfrak {t}$ so all roots are imaginary.

Since $\theta \alpha = \alpha$ for $\alpha$ imaginary, $\theta \mathfrak {g}_\alpha = \mathfrak {g}_\alpha$ and we conclude that $\mathfrak {g}_\alpha \subset \mathfrak {k}$ or $\mathfrak {p}$.

Definition 3.5

Let $\alpha$ be an imaginary root. It is compact if $\mathfrak {g}_\alpha \subset \mathfrak {k}$ and it is noncompact if $\mathfrak {g}_\alpha \subset \mathfrak {p}$.

Definition 3.6

Let $\alpha$ be an imaginary root. Define $\epsilon (\alpha )=1$ if $\alpha$ is compact, and $\epsilon (\alpha )=-1$ if $\alpha$ is noncompact. If $\alpha$, $\beta$, and $\alpha +\beta$ are roots and if $\alpha$ and $\beta$ are imaginary, from $[\mathfrak {g}_\alpha ,\mathfrak {g}_\beta ] = \mathfrak {g}_{\alpha +\beta }$, $[\mathfrak {k}, \mathfrak {k}] \subset \mathfrak {k}$, $[\mathfrak {k}, \mathfrak {p}] \subset \mathfrak {p}$, and $[\mathfrak {p}, \mathfrak {p}] \subset \mathfrak {k}$, we see that $\epsilon ( \alpha + \beta ) = \epsilon (\alpha ) \epsilon ( \beta )$. Therefore $\epsilon$ may be extended to a $\mathbb {Z}_2$-grading on the imaginary root lattice.

Proposition 3.7

[30, p. 641] The Verma module $M( \mu )$ admits a non-trivial invariant Hermitian form if $\mathfrak {h}$ is maximally compact, $\theta ( \Delta ^+(\mathfrak {g}, \mathfrak {h}) ) = \Delta ^+(\mathfrak {g}, \mathfrak {h})$ (recall we selected $\mathfrak {h}$ to be $\theta$-stable), and $\mu$ is imaginary. A non-trivial invariant Hermitian form on a Verma module is unique up to a non-zero real scalar.

If $\mathfrak {h}$ is maximally compact, all roots are either imaginary or complex. For the rest of this paper, we assume we are in the setting where non-trivial invariant Hermitian forms exist on Verma modules.

We pick a canonical form on each Verma module admitting an invariant Hermitian form:

Definition 3.8

Given $\mu \in \mathfrak {h}^*$, if $M(\mu )$ admits a non-trivial invariant Hermitian form, then the unique invariant Hermitian form $\langle \cdot , \cdot \rangle _\mu$ on $M( \mu )$ for which $\langle v_{\mu - \rho }, v_{\mu - \rho } \rangle _\mu = 1$ is called the Shapovalov form.

Although Verma modules are infinite dimensional, we may discuss the signature of the Shapovalov form because we can decompose the Verma module into a direct sum of orthogonal finite dimensional spaces.

Proposition 3.9

[30, p. 643] If $\nu , \nu ' \in \Lambda _r^+$ and $-{\bar{\nu }} = \theta (\nu ) \ne \nu '$, then by invariance $\langle M(\mu )_{\mu - \nu \rho }, M( \mu )_{\mu - \nu '-\rho } \rangle _\mu = 0$.

If $\nu$ is imaginary, then the $\mu -\nu -\rho$ weight space is paired with itself. If $\nu$ is complex, then the $\mu -\nu -\rho$ weight space is paired with the $\mu -\theta (\nu )-\rho = \mu + {\bar{\nu }} - \rho$ weight space. In the complex case, it can be shown (see [28, Lemma 3.18] or [30, p. 644 and 645]) that the number of positive and negative eigenvalues of a matrix representing the Shapovalov form on $M( \mu )_{\mu -\nu -\rho } \oplus M(\mu )_{\mu -\theta (\nu )-\rho }$ are equal. Thus we can define:

Definition 3.10

The signature character of the Shapovalov form $\langle \cdot , \cdot \rangle _\mu$ on the Verma module $M(\mu )$ is the formal sum

$$\begin{aligned} ch_s M(\mu ) = \sum _{\begin{array}{c} \nu \in \Lambda _r^+\\ \nu \text {imaginary} \end{array}} (p(\nu ) - q(\nu ))e^{\mu -\nu -\rho } \end{aligned}$$

where a matrix representing the Shapovalov form on $M( \mu )_{\mu - \nu - \rho }$ has $p(\nu )$ positive eigenvalues and $q( \nu )$ negative eigenvalues.

In order to derive signature character formulas for irreducible Verma modules, we must understand reducibility of Verma modules. The maximal proper submodule of a Verma module is the radical of the Shapovalov form, so a Verma module is reducible precisely when the Shapovalov form is degenerate. By a modification of the classical (invariant bilinear) Shapovalov determinant formula, we have:

Proposition 3.11

[30, p. 644] When $\nu \in \Lambda _r^+$ is imaginary, up to a scalar, the determinant of a matrix representing the Shapovalov form on $M( \mu )_{\mu - \nu - \rho }$ is

$$\begin{aligned} \prod _{\alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h})} \prod _{n=1}^\infty \left( (\mu , \alpha ^\vee ) - n \right) ^{P(\nu - n \alpha )} \end{aligned}$$

where P is Kostant’s partition function.

When $\nu \in \Lambda _r^+$ is complex, up to a scalar, the determinant of a matrix representing the Shapovalov form on $M( \mu )_{\mu - \nu - \rho } \oplus M( \mu )_{\mu - \theta \nu - \rho }$ is

$$\begin{aligned} \prod _{\alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h})} \prod _{n=1}^\infty \left( (\mu , \alpha ^\vee ) - n \right) ^{P(\nu - n \alpha )} \left( (\mu , \alpha ^\vee ) - n \right) ^{P(\theta \nu - n \alpha )}. \end{aligned}$$

Therefore Verma modules are reducible precisely on the affine hyperplanes $H_{\alpha ,n}$ where $\alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h})$ and $n \in \mathbb {Z}^+$. Within any connected region avoiding these reducibility hyperplanes, the Shapovalov form remains nondegenerate so the signature of the form cannot change. This idea was introduced in [28]. In [29], Wallach noted that there is a large region containing the antidominant Weyl chamber where the signature of the Shapovalov form stays constant:

$$\begin{aligned} \left( \cap _{\alpha \in \Pi } H_{\alpha ,1}^{-} \right) \cap H_{\alpha _0, 1}^-. \end{aligned}$$

By an asymptotic argument, Wallach obtained:

Theorem 3.12

([29, Lemma 2.3], reformulated in [30, Theorem 2.10 and Theorem 6.12]) If $\mu \in \mathfrak {h}^*$ is imaginary and if $\mu \in \left( \cap _{\alpha \in \Pi } H_{\alpha ,1} \right) ^- \cap H_{\alpha _0, 1}^-$, then the signature character of the Shapovalov form on $M( \mu )$ is

$$\begin{aligned} ch_s M( \mu ) |_{\mathfrak {a}}= & {} e^{\mu |_{\mathfrak {a}}} \qquad \text {and} \\ ch_s M(\mu ) |_\mathfrak {t}= & {} \frac{e^{(\mu - \rho )|_{\mathfrak {t}}}}{\prod _{\alpha \in \Delta ^+( \mathfrak {p}, \mathfrak {t})} (1 - e^{-\alpha }) \prod _{\alpha \in \Delta ^+(\mathfrak {k}, \mathfrak {t})}( 1 + e^{-\alpha })}. \end{aligned}$$

Because signatures are constant in regions bounded by reducibility hyperplanes $H_{\alpha ,n}$, it makes sense to find a formula for signature characters of invariant Hermitian forms on irreducible Verma modules indexed by the affine Weyl group. Define:

Definition 3.13

Let $R(\mu ) := \sum _{\nu \in \Lambda _r^+} c_\nu e^{\mu - \nu - \rho }$ where the constants $c_\nu$ are such that $R(\mu )=ch_s M( \mu )$ when $\mu \in \left( \cap _{\alpha \in \Pi } H_{\alpha ,1} \right) ^- \cap H_{\alpha _0, 1}^-$ is imaginary. For an alcove A of the affine Weyl group $W_a$, let $R^A(\mu ) := \sum _{\nu \in \Lambda _r^+} c_\nu ^A e^{\mu - \nu - \rho }$ be the formal sum such that $R^A(\mu ) = ch_s M(\mu )$ when $\mu \in A$ is imaginary.

To arrive at a formula for $R^A$, the philosophy is as follows: first determine how signatures change as you cross a reducibility hyperplane, then take a path from A to $\left( \cap _{\alpha \in \Pi } H_{\alpha ,1}\right) ^- \cap H_{\alpha _0, 1}^-$ where the signature is known by Wallach’s work and apply induction on the number of reducibility hyperplanes crossed. Specifically:

Lemma 3.14

[30, Proposition 3.2 and Lemma 4.3] Let A and $A'$ be adjacent alcoves separated by the reducibility hyperplane $H_{\alpha ,n}$ where $\alpha$ is imaginary. Then

$$\begin{aligned} R^A( \mu ) = R^{A'}( \mu ) + 2 \varepsilon ( A, A' ) R^{A - n\alpha }( \mu - n \alpha ) \end{aligned}$$

where $\varepsilon ( A, A' ) = \pm 1$.

This is because as you cross a reducibility hyperplane at a point not intersecting any others, the signature changes by the signature of the radical $M(\mu - n\alpha )$ which is the signature of the Shapovalov form or its opposite since invariant Hermitian forms on Verma modules are unique up to a real scalar.

The formula for $\varepsilon ( A, A' )$ from [30] is complicated and its simplification forms the content of the following section. Note that $\varepsilon ( A, A' ) = - \varepsilon ( A', A )$.

Complex $\alpha$ are somewhat more complicated. Let $\mu \in \mathfrak {h}^*$ be imaginary and suppose $\mu \in H_{\alpha ,n}$ where $\alpha$ is complex. Then $\mu \in H_{\theta \alpha , n}$ as well. Then:

Lemma 3.15

[30, Propositions 3.7 and 3.8] Let the imaginary weights be partitioned into alcoves by the affine hyperplanes $H_{\alpha ,n}$. Let A and $A'$ be adjacent alcoves separated by the reducibility hyperplane $H_{\alpha ,n}$ where $\alpha$ is complex. Then they are also separated by $H_{\theta \alpha ,n}$.

If $\alpha$ and $\theta \alpha$ are orthogonal, then

$$\begin{aligned} R^A( \mu ) = R^{A'}(\mu ). \end{aligned}$$

If $\alpha$ and $\theta \alpha$ are not orthogonal so that $\alpha + \theta \alpha$ is a root (note that it is imaginary), then $H_{\alpha +\theta (\alpha ), 2n}$ also separates A and $A'$ and

$$\begin{aligned} R^A( \mu ) = R^{A'}(\mu ) + 2 \epsilon ( A, A') R^{A-2n(\alpha + \theta ( \alpha ))}(\mu - 2n( \alpha + \theta ( \alpha ) ) ). \end{aligned}$$

Thus crossing only reducibility hyperplanes corresponding to imaginary roots results in changes to the signature character.

Using these lemmas, by induction applied to an alcove path from A to the region $\left( \cap _{\alpha \in \Pi } H_{\alpha ,1}\right) ^- \cap H_{\alpha _0, 1}^-$,

Theorem 3.16

[30, Theorems 4.6 and 6.12] Let $\Delta _i(\mathfrak {g},\mathfrak {h})$ be the imaginary roots of $\Delta (\mathfrak {g},\mathfrak {h})$. Use subscripts and superscripts i to indicate that objects are associated with $\Delta _i(\mathfrak {g},\mathfrak {h})$. In this theorem, simple roots, roots, Weyl group, length, reducibility hyperplanes, fundamental alcove, alcoves, affine Weyl group and so on are associated with the imaginary root system (which is just the usual root system when $\mathfrak {g}_0$ is equal rank). The alcoves are the regions in the imaginary weights of $\mathfrak {h}^*$ partitioned by reducibility hyperplanes of the form $H_{\alpha ,n}$ where $\alpha \in \Delta _i^+(\mathfrak {g},\mathfrak {h})$ and $n \in \mathbb {Z}^+$. Let A be an alcove of $W_a^i$ and let ${\bar{\cdot }} : W_a^i \rightarrow W_i$ be the homomorphism arising from the semidirect product structure $W_a^i = \Lambda _r^i \rtimes W_i$. Let $w \in W_i$ be such that $A \subset w \mathfrak {C}_0^i$. Let $A=C_0 {\mathop {\rightarrow }\limits ^{r_1}} C_1 {\mathop {\rightarrow }\limits ^{r_2}} C_2 {\mathop {\rightarrow }\limits ^{r_3}} \cdots {\mathop {\rightarrow }\limits ^{r_\ell }} C_\ell = w A_\circ ^i$ be a (not necessarily reduced) alcove path (the $C_i$ are alcoves, the $r_i$ are affine reflections, and $C_i = r_i C_{i-1}$ for $1 \le i \le \ell$). Then for imaginary $\mu \in A$,

$$\begin{aligned} ch_s M( \mu )|_{\mathfrak {a}}= & {} e^{(\mu -\rho )|_{\mathfrak {a}}} \qquad \text {and} \\ ch_s M( \mu )|_{\mathfrak {t}}= & {} R^A( \mu |_{\mathfrak {t}} ) \\= & {} \sum _{\begin{array}{c} I = \{i_1< \cdots < i_k\} \\ I \subset \{1, \ldots , \ell \} \end{array}} \varepsilon (I)2^{|I|} \frac{e^{{\bar{r}}_{i_1}{\bar{r}}_{i_2} \cdots {\bar{r}}_{i_k} r_{i_k} \cdots r_{i_2} r_{i_1} (\mu - \rho )|_{\mathfrak {t}}}}{\prod _{\alpha \in \Delta ^+( \mathfrak {p}, \mathfrak {t})} (1 - e^{-\alpha }) \prod _{\alpha \in \Delta ^+(\mathfrak {k}, \mathfrak {t})}( 1 + e^{-\alpha })}. \end{aligned}$$

where $\varepsilon ( I ) = \varepsilon ( C_{i_1 - 1}, C_{i_1} ) \varepsilon ( {\bar{r}}_{i_1}C_{i_2-1}, {\bar{r}}_{i_1}C_{i_2}) \cdots \varepsilon ( {\bar{r}}_{i_1}\cdots {\bar{r}}_{i_{k-1}}C_{i_k-1}, {\bar{r}}_{i_1}\cdots {\bar{r}}_{i_{k-1}}C_{i_k})$, $\varepsilon ( \emptyset ) = 1$, $\varepsilon ( C, C' ) = 0$ if the hyperplane separating the alcoves $C,C'$ is not a reducibility hyperplane and the formula for $\varepsilon (C,C')$ when the alcoves are separated by a reducibility hyperplane will be stated in Theorem 3.17.

Theorem 3.17

[30, Theorems 6.12, 5.3.4] We maintain the notation and setting of the previous theorem. Let C and $C'$ be adjacent alcoves of $W_a^i$ separated by the reducibility hyperplane $H_{\gamma ,n}$ where $\gamma \in \Delta ^+_i( \mathfrak {g}, \mathfrak {h})$, $n \in \mathbb {Z}^+$, $C \subset H_{\gamma ,n}^+$, and $C' \subset H_{\gamma , n}^-$. Let $w \in W$ be such that $C, C' \subset w \mathfrak {C}_0^i$. Let $\gamma = s_{i_1} \cdots s_{i_{r-1}}\alpha _{i_r}$ where the $\alpha _{i_j} \in \Pi ^i$ are such that $ht( s_{i_j} \cdots s_{i_{r-1}} \alpha _{i_r})$ strictly decreases as j increases. Let $w_\gamma = s_{i_1} \cdots s_{i_r}$. Recall $\epsilon$ the $\mathbb {Z}_2$-grading on the imaginary root lattice. Then if $\gamma$ does not form a type $G_2$ root system with other roots:

• If $\theta$ does not fix any element of the component of $\Pi$ corresponding to $\gamma$, then:

  $$\begin{aligned} \varepsilon ( C, C' ) = -1. \end{aligned}$$

• If $\theta$ fixes some element of the component of $\Pi$ corresponding to $\gamma$, then:

  $$\begin{aligned} \varepsilon (C, C' )= & {} \epsilon ( n \gamma ) \times (-1)^{\#\{\beta \in \Delta _i( w_\gamma ^{-1}) : |\beta | = |\gamma |, \beta \ne \gamma , \text { and } \beta , s_\beta \gamma \in \Delta _i(w^{-1}) \}} \\&\times (-1)^{\#\{\beta \in \Delta _i( w_\gamma ^{-1}) : |\beta | \ne |\gamma | \text { and } \beta , -s_\beta s_\gamma \beta \in \Delta _i(w^{-1}) \}}. \end{aligned}$$

Let $\alpha _1$ and $\alpha _2$ be the long and short simple roots for a type $G_2$ root system, respectively. We have the following table of values for $\varepsilon ( C, C' )$:

Note that in the referenced results, we have $(-1)^{n \# \{ \text {noncompact } \alpha _{i_j}: |\alpha _{i_j}| \ge |\gamma | \}}$ as the first of three terms in the formula for $\varepsilon$ (second bullet point). We can replace it by $\epsilon ( n \gamma )$ by the proof of Theorem 3.7 (2) of [32].

Note that we have to treat type $G_2$ separately because Theorems 5.3.4 and 6.12 of [30] hold for roots $\gamma$ not forming a type $G_2$ root system with other roots in $\Delta _i^+(\mathfrak {g},\mathfrak {h})$.

See section 6 of [30] for more information on $\Delta _i(\mathfrak {g},\mathfrak {h})$ and $\Pi _i$.

Combining Theorems 3.16 and 3.17 gives a formula for $ch_s M( \mu )$ when $M( \mu )$ is irreducible.

4 Simplifying the formulas for $\varepsilon (C,C')$ and $\varepsilon (I)$

In [28], Vogan introduced the idea of computing the signature of a non-degenerate invariant Hermitian form on a finite length $(\mathfrak {g},K)$-module by studying how signatures change as you cross reducibility points. Computing changes across reducibility points required knowledge of a sign ($\varepsilon$, as in Sect. 3) which was unknown at the time. The sign was first computed in [30]. We prove a simple formula for $\varepsilon$ for Verma modules in this section.

We want the formula for $\varepsilon$ to hold for as general $\mathfrak {g}_0$ as possible, so we work in the setting of Theorem 3.16 when we work on the formula for $\varepsilon$; in other words, the root system we are working with is $\Delta _i( \mathfrak {g}, \mathfrak {h})$ and all attached objects and quantities are associated to that root system. (In the equal rank case, $\Delta _i(\mathfrak {g},\mathfrak {h}) = \Delta (\mathfrak {g},\mathfrak {h})$, etc.) We revert to a generic Weyl group and root system when working on definitions and lemmas that hold in general settings (i.e. not necessarily arising from the study of signature characters).

In [30], we showed that for adjacent alcoves $C, C'$, $\varepsilon ( C, C')$ only depends on the hyperplane separating the alcoves and the Weyl chamber containing the alcoves. Thus we defined:

Definition 4.1

For $\gamma \in \Delta ^+_i(\mathfrak {g},\mathfrak {h})$, $n \in \mathbb {Z}^+$, define $\varepsilon ( H_{\gamma , n}, w ) = \varepsilon ( C, C' )$ where $C,C'$ are adjacent alcoves separated by $H_{\gamma ,n}$, $C, C' \subset w \mathfrak {C}_0^i$, $C \subset H_{\gamma ,n}^+$ and $C' \subset H_{\gamma ,n}^-$.

Remark 4.2

Note that in the definition, we require that $\gamma$ hyperplanes are positive in $w \mathfrak {C}_0^i$; i.e. if $H_{\gamma ,n}$ intersects $w \mathfrak {C}_0^i$, then $n > 0$. This is equivalent to $\gamma \in \Delta _i( w^{-1} )$. See the proof of [30, Lemma 5.2.3] for details.

To simplify the formula for $\varepsilon ( C, C') = \varepsilon ( H_{\gamma ,n}, w )$, we need to reformulate the second and third terms in the product in Theorem 3.17. Thus we define (in a general setting):

Definition 4.3

Let $\gamma \in \Delta ^+(\mathfrak {g},\mathfrak {h})$ and let $w \in W$ be such that $\gamma \in \Delta ( w^{-1} )$. Recall $w_\gamma$, defined in Theorem 3.17. Let

$$\begin{aligned} \mathcal {S}_{w_\gamma ,w}^2 = \lbrace \beta \in \Delta (w_\gamma ^{-1} ) \, : \, |\beta | = |\gamma |, \beta \ne \gamma , \text { and } \beta , s_\beta \gamma \in \Delta ( w^{-1} ) \rbrace \end{aligned}$$

and let

$$\begin{aligned} \mathcal {S}_{w_{\gamma },w}^3 = \lbrace \beta \in \Delta (w_{\gamma }^{-1} ) \, : \, |\beta | \ne |\gamma | \text { and } \beta , -s_\beta s_{\gamma } \beta \in \Delta ( w^{-1} ) \rbrace . \end{aligned}$$

Lemma 4.4

Let $\gamma \in \Delta ^+(\mathfrak {g},\mathfrak {h})$ and let $w \in W$ be such that $\gamma \in \Delta (w^{-1} )$. Suppose $\gamma$ does not form a type $G_2$ root system with other roots. Then:

$$\begin{aligned} \mathcal {S}_{w_\gamma ,w}^2 = \{ \beta \in \Delta ( w_\gamma ^{-1} ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \} : |\beta | = |\gamma |\} \end{aligned}$$

and

$$\begin{aligned} \mathcal {S}_{w_\gamma ,w}^3 = \left\{ \beta \in \Delta ( w_\gamma ^{-1} ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \} : |\beta | \ne |\gamma | \right\} . \end{aligned}$$

Therefore $\mathcal {S}_{w\gamma ,w}^2 \cup \mathcal {S}_{w\gamma ,w}^3 = \Delta ( w_\gamma ^{-1} ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \}$.

Proof

Suppose $\beta \in \mathcal {S}_{w_\gamma ,w}^2$. Since $(\beta , \gamma ) > 0$ by Lemma 3.2 of [32] and since $|\beta | = |\gamma |$, therefore $s_\beta \gamma = \gamma - \beta = -s_\gamma \beta$. Then

$$\begin{aligned} s_\beta \gamma = -s_\gamma \beta \in \Delta (w^{-1}) \iff \beta \in -s_\gamma \Delta ( w^{-1}). \end{aligned}$$

If $\beta \in \mathcal {S}_{w_\gamma , w}^3$ then, again, $(\beta , \gamma ) > 0$. The root $s_\gamma \beta$ is orthogonal to $\beta$, so we have $-s_\beta s_\gamma \beta = -s_\gamma \beta$. Then

$$\begin{aligned} -s_\beta s_\gamma \beta = -s_\gamma \beta \in \Delta ( w^{-1}) \iff \beta \in -s_\gamma \Delta ( w^{-1} ). \end{aligned}$$

$\square$

Then since $\varepsilon ( H_{\gamma ,n}, w ) = \epsilon ( n \gamma ) (-1)^{\# \mathcal {S}_{w_\gamma ,w}^2} (-1)^{\# \mathcal {S}_{w_\gamma ,w}^3}$, therefore it follows from the lemma:

Proposition 4.5

Let $\gamma \in \Delta _i^+(\mathfrak {g},\mathfrak {h})$, let $w \in W_i$ be such that $\gamma$ is positive on $w \mathfrak {C}_0^i$, and suppose $\theta$ fixes some element of $\Pi$. Suppose $\gamma$ does not form a type $G_2$ root system with other roots. Then:

$$\begin{aligned} \varepsilon ( H_{\gamma ,n}, w ) = (-1)^{\# \{ \Delta _i( w_\gamma ^{-1} ) \cap \Delta _i( w^{-1} ) \cap -s_\gamma \Delta _i( w^{-1} ) \setminus \{ \gamma \} \}} \end{aligned}$$

where $w_\gamma$ is defined in Theorem 3.17.

We wish to remove the dependence of the formula for $\varepsilon$ on a choice of element $w_\gamma$.

The next lemma holds in a general setting.

Lemma 4.6

Given $\gamma \in \Delta ^+(\mathfrak {g},\mathfrak {h})$, let $\gamma = s_{i_1} s_{i_2} \cdots s_{i_r}$ where $ht( s_{i_j} \cdots s_{i_{r-1}}\alpha _{i_r})$ strictly decreases as j increases. Recall we defined $w_\gamma = s_{i_1}s_{i_2} \cdots s_{i_r}$. Then:

• (1) $w_\gamma = s_{i_1}s_{i_2} \cdots s_{i_r}$ is a reduced expression, and

• (2) $s_\gamma = s_{i_1}s_{i_2} \cdots s_{i_r} \cdots s_{i_2} s_{i_1}$ is a reduced expression.

Proof

The first result can be found in the proof of Theorem 5.3.4 of [30]. To prove the second, suppose the expression is not reduced. Then there exist indices $j_1$ and $j_2$ as close to k as possible such that $s_{i_1} \cdots s_{i_k} \cdots s_{i_1} = s_{i_1} \cdots {\hat{s}}_{i_{j_1}} \cdots s_{i_k} \cdots {\hat{s}}_{i_{j_2}} \cdots s_{i_1}$ by the deletion condition. The deleted terms straddle (and may include) $s_{i_k}$ since $s_{i_1} s_{i_2} \cdots s_{i_k}$ is a reduced expression for $w_\gamma$. Without loss of generality, suppose $j_1 = 1$. Then $s_{i_1} s_{i_2} \cdots s_{i_k} \cdots s_{i_{j_2+1}} s_{i_{j_2}} = s_{i_2} \cdots s_{i_k} \cdots s_{i_{j_2+1}}$ so by Theorem 1.7 of [11], $\alpha _{i_1} = s_{i_2} \cdots s_{i_k} s_{i_{k-1}} \cdots s_{i_{j_2+1}} \alpha _{i_{j_2}}$. From the height condition in the construction of $w_\gamma$, $( \alpha _{i_j}, s_{i_{j+1}} \cdots s_{i_{k-1}} \alpha _{i_k} ) < 0$ for $j = 1, \dots , k-1$. But then

$$\begin{aligned} 0> & {} (\alpha _{i_1}, s_{i_2} \cdots s_{i_{k-1}} \alpha _{i_k} ) \\= & {} ( s_{i_2} \cdots s_{i_k} s_{i_{k-1}} \cdots s_{i_{j_2 +1}} \alpha _{i_{j_2}}, s_{i_2} \cdots s_{i_{k-1}} \alpha _{i_k} ) \\= & {} -(s_{i_{k-1}} \cdots s_{i_{j_2+1}} \alpha _{i_{j_2}}, \alpha _{i_k} ) \\= & {} -(\alpha _{i_{j_2}}, s_{i_{j_2+1}} \cdots s_{i_{k-1}} \alpha _{i_k} ) \\> & {} 0 \end{aligned}$$

—contradiction. Therefore our expression for $s_\gamma$ must be reduced. $\square$

It follows that (again, we keep the next lemma general):

Lemma 4.7

Let $\gamma \in \Delta ^+(\mathfrak {g},\mathfrak {h})$ and let $w_\gamma$ be as defined in Lemma 4.6. Then

$$\begin{aligned} \Delta ( s_\gamma ) = \Delta ( w_\gamma ^{-1} ) \cup -s_\gamma \Delta ( w_\gamma ^{-1} ) \end{aligned}$$

where the only root common to both sets on the right hand side is $\gamma$.

Proof

By the reduced expressions for $w_\gamma$ and $s_\gamma$ in Lemma 4.6 and by [11, p. 14],

$$\begin{aligned} \Delta ( w_\gamma ^{-1} )= & {} \{ \alpha _{i_1}, s_{i_1} \alpha _{i_2}, \cdots , s_{i_1} \cdots s_{i_{r-1}}\alpha _{i_r} \} \qquad \text {and} \\ \Delta ( s_\gamma )= & {} \{ \alpha _{i_1}, s_{i_1} \alpha _{i_2}, \cdots , s_{i_1} \cdots s_{i_{r-1}}\alpha _{i_r}, s_{i_1} \cdots s_{i_{r-1}}s_{i_r}\alpha _{i_{r-1}}, \cdots , s_{i_1} \cdots s_{i_r} \cdots s_{i_2} \alpha _{i_1} \}. \end{aligned}$$

Since $s_{i_1} \cdots s_{i_r} \cdots s_{i_{j+1}} \alpha _{i_j} = (s_{i_1} \cdots s_{i_r} \cdots s_{i_1}) s_{i_1} s_{i_2} \cdots s_{i_j} \alpha _{i_j} = -s_\gamma s_{i_1} \cdots s_{i_{j-1}} \alpha _{i_j}$, therefore $\Delta ( s_\gamma ) = \Delta ( w_\gamma ^{-1} ) \cup - s_\gamma \Delta ( w_\gamma ^{-1} )$. Since $\gamma$ belongs to both sets in the right hand side and since $\ell ( s_\gamma ) = 2 \ell ( w_\gamma ) - 1$, we have proved the rest of the lemma. $\square$

Lemma 4.8

Let $\gamma \in \Delta ^+( \mathfrak {g}, \mathfrak {h})$ and let $w \in W$ be such that $\gamma \in \Delta ( w^{-1} )$. Let $w_\gamma$ be as defined in Lemma 4.6. Then $\Delta ( s_\gamma ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \}$ is the disjoint union of two sets:

$$\begin{aligned} \Delta ( w_\gamma ^{-1} ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \} {\dot{\cup }} -s_\gamma \Delta ( w_\gamma ^{-1} ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \}. \end{aligned}$$

Furthermore,

$$\begin{aligned} \Delta ( s_\gamma ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \} = \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \}. \end{aligned}$$

Therefore

$$\begin{aligned} \# \left\{ \Delta ( w_\gamma ^{-1} ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \} \right\}= & {} \frac{1}{2} \# \left\{ \Delta ( s_\gamma ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \} \right\} \\= & {} \frac{1}{2} \# \left\{ \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \} \right\} . \end{aligned}$$

Proof

The disjoint union follows from the previous lemma.

Note that $\beta \in \Delta ( w_\gamma ^{-1} ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \}$ if and only if $-s_\gamma \beta \in -s_\gamma \Delta ( w_\gamma ^{-1} ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \}$. Therefore

$$\begin{aligned} \# \left\{ \Delta ( w_\gamma ^{-1} ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \} \right\} = \frac{1}{2} \# \left\{ \Delta ( s_\gamma ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \} \right\} . \end{aligned}$$

Note that if $\beta \in \Delta ( w^{-1}) \cap -s_\gamma \Delta ( w^{-1}) \setminus \{ \gamma \}$, then $-s_\gamma \beta \in \Delta ( w^{-1} ) \subset \Delta ^+( \mathfrak {g}, \mathfrak {h})$, therefore $s_\gamma \beta < 0$ so $\beta \in \Delta ( s_\gamma )$. Therefore $\Delta ( s_\gamma ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \} = \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \}$. This proves the lemma. $\square$

The above lemma allows us to remove dependence on a choice of $w_\gamma$ from the formula for $\varepsilon ( H_{\gamma ,n}, w )$.

Proposition 4.9

Let $\gamma \in \Delta _i^+( \mathfrak {g}, \mathfrak {h})$, $w \in W_i$ be such that $\gamma$ hyperplanes are positive on $w \mathfrak {C}_0^i$, and suppose $\theta$ fixes some element of $\Pi$. Then

$$\begin{aligned} \varepsilon ( H_{\gamma ,n}, w )= & {} \epsilon ( n \gamma ) (-1)^{\frac{1}{2} \# \{ \Delta _i( s_\gamma ) \cap \Delta _i( w^{-1} ) \cap -s_\gamma \Delta _i( w^{-1} ) \setminus \{ \gamma \} \}} \\= & {} \epsilon ( n \gamma ) (-1)^{\frac{1}{2} \# \{ \Delta _i( w^{-1} ) \cap -s_\gamma \Delta _i( w^{-1} ) \setminus \{ \gamma \} \} }. \end{aligned}$$

Proof

The proposition is straightforward to verify for a type $G_2$ root system. Otherwise, we combine the previous lemma with Proposition 4.5. $\square$

We can simplify the formula for $\varepsilon$ even further. For that, we need the following (general) lemma:

Lemma 4.10

Let $\gamma \in \Delta ^+( \mathfrak {g}, \mathfrak {h})$ and $w \in W$ be such that $\gamma \in \Delta ( w^{-1} )$. Then

$$\begin{aligned} \# \{ \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \} = \ell ( w ) - \ell ( s_\gamma w ). \end{aligned}$$

Proof

This is straightforward to verify for type $G_2$, which must be treated separately since Lemmas 3.4 and 3.6 of [32] do not apply to roots $\gamma$ forming a type $G_2$ root system with other roots. Otherwise, we prove this result by induction on w.

For the base case, we consider when w is minimal with respect to the Bruhat order such that $\gamma \in \Delta ( w^{-1} )$. (Note that such w may not be unique.) Let $w = s_{i_1} s_{i_2} \cdots s_{i_k}$ be a reduced expression. Since w is minimal, therefore $\gamma \not \in \Delta ( (s_{i_1} \cdots s_{i_{k-1}})^{-1} )$. Thus

$$\begin{aligned} \gamma \in \Delta ( w^{-1} ) \setminus \Delta ( (s_{i_1} \cdots s_{i_{k-1}})^{-1} ) = \{ s_{i_1} s_{i_2} \cdots s_{i_{k-1}} \alpha _{i_k} \}. \end{aligned}$$

Let $x = ws_{i_k}$ and $s = s_{i_k}$ in Lemmas 3.4 and 3.6 of [32]. Then $\mathcal {S}_{w_\gamma ,w}^2 = \mathcal {S}_{w_\gamma ,w}^3 =\emptyset$ so $\Delta ( w_\gamma ^{-1} ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \} = \emptyset$ by Lemma 4.4. Then since by Lemma 4.8$\Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \}$ is the disjoint union of $\Delta ( w_\gamma ^{-1} ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \}$ and $-s_\gamma \left( \Delta ( w_\gamma ^{-1} ) \cap \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \} \right)$, therefore $\Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \setminus \{ \gamma \} = \emptyset$. Thus

$$\begin{aligned} \# \{ \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) \} = \# \{ \gamma \} = 1 = k - (k-1) = \ell ( w ) - \ell ( s_\gamma w ) \end{aligned}$$

proving the base case.

Now suppose w is not minimal with respect to the Bruhat order such that $\gamma \in \Delta ( w^{-1} )$. By induction, suppose that for all $x < w$ such that $\gamma \in \Delta ( x^{-1} )$, $\# \{ \Delta ( x^{-1} ) \cap -s_\gamma \Delta ( x^{-1} ) \} = \ell (x ) - \ell ( s_\gamma x )$. Take in particular $x =ws_\alpha$ where $\alpha$ is simple and $\ell (x) = \ell (w) - 1$.

• Case 1: $\gamma = x \alpha$. We can apply Lemmas 3.4 and 3.6 of [32], as in the base case. Again, we have $\Delta (w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ) = \{ \gamma \}$ and $\# \{\Delta ( w^{-1} \cap -s_\gamma \Delta ( w^{-1} ) \} = 1 = \ell (w) - \ell ( s_\gamma w )$.

• Case 2: $\gamma \in \Delta ( x^{-1} ) = \Delta ( w^{-1} ) \setminus \{ x \alpha \}$. By induction, we have

  $$\begin{aligned} \# \{ \Delta ( x^{-1} ) \cap -s_\gamma \Delta ( x^{-1} ) \} = \ell ( x ) - \ell ( s_\gamma x ). \end{aligned}$$

We also have

$$\begin{aligned} \Delta ( w^{-1} )= & {} \Delta ( x^{-1} ) \cup \{ x\alpha \} \\ -s_\gamma \Delta ( w^{-1} )= & {} -s_\gamma \Delta ( x^{-1} ) \cup \{ -s_\gamma x \alpha \}. \end{aligned}$$

We have $\ell (w) = \ell (x) + 1$ and $\ell ( s_\gamma w ) = \ell ( s_\gamma x s_\alpha ) = \ell ( s_\gamma x ) \pm 1$. Therefore

$$\begin{aligned} \ell (w) - \ell ( s_\gamma w ) = \left\{ \begin{array}{ll} \ell (x) - \ell ( s_\gamma x ) &{} \quad \text {if } s_\gamma x\alpha > 0 \\ \ell (x) - \ell ( s_\gamma x ) +2 &{} \quad \text {if } s_\gamma x\alpha < 0. \end{array} \right. \end{aligned}$$

(4.1)

First, we show that $x \alpha \ne -s_\gamma x \alpha$. If, by contradiction, $x \alpha = -s_\gamma x \alpha = -x\alpha + \frac{2( \gamma , x\alpha )}{(\gamma ,\gamma )}\gamma$, then $\frac{2( \gamma , x\alpha )}{(\gamma , \gamma )}\gamma = 2x\alpha$ which implies that $\gamma = x \alpha$. But $\{ \gamma \} = \{ x \alpha \} = \Delta ( w^{-1} ) \setminus \Delta ( x^{-1} )$ and in the current case, $\gamma \in \Delta ( x^{-1} )$—contradiction. Therefore $x\alpha \ne -s_\gamma x \alpha$.

Next, note that $x\alpha \in -s_\gamma \Delta ( x^{-1} )$ if and only if $-s_\gamma x\alpha \in \Delta ( x^{-1} )$. Therefore

$$\begin{aligned} \# \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} )= & {} \# \left( \Delta ( x^{-1} ) \cup \{ x\alpha \} \right) \cap \left( -s_\gamma \Delta ( x^{-1} ) \cup \{ -s_\gamma x\alpha \} \right) \nonumber \\= & {} \left\{ \begin{array}{ll} \# \Delta ( x^{-1} ) \cap -s_\gamma \Delta ( x^{-1} ) &{} \quad \text {if } x\alpha \not \in -s_\gamma \Delta ( x^{-1} ) \\ \# \Delta ( x^{-1} ) \cap -s_\gamma \Delta ( x^{-1} ) +2 &{} \quad \text {if } x\alpha \in -s_\gamma \Delta ( x^{-1} ) \\ \end{array} \right. \nonumber \\= & {} \left\{ \begin{array}{ll} \# \Delta ( x^{-1} ) \cap -s_\gamma \Delta ( x^{-1} ) &{} \quad \text {if } -s_\gamma x\alpha \not \in \Delta ( x^{-1} ) \\ \# \Delta ( x^{-1} ) \cap -s_\gamma \Delta ( x^{-1} ) +2 &{} \quad \text {if } -s_\gamma x\alpha \in \Delta ( x^{-1} ) \end{array} \right. \nonumber \\ \end{aligned}$$

(4.2)

We prove the lemma from Eqs. (4.1) and (4.2) by showing that the case conditions of the latter equation correspond to the case conditions of the former equation.

If $-s_\gamma x \alpha \in \Delta ( x^{-1} )$, then $s_\gamma x\alpha < 0$, which is the second condition in Eq. (4.1). Then combining the Eqs. (4.1) and (4.2), we have

$$\begin{aligned} \ell (w) - \ell ( s_\gamma w )= & {} \ell ( x ) - \ell ( s_\gamma x ) + 2 = \# \Delta ( x^{-1} ) \cap -s_\gamma \Delta ( x^{-1} ) + 2 \\= & {} \# \Delta ( w^{-1} ) \cap -s_\gamma \Delta ( w^{-1} ). \end{aligned}$$

If $-s_\gamma x \alpha \not \in \Delta ( x^{-1} )$, then the argument is more complex. We want to show that $s_\gamma x\alpha > 0$, giving us the first condition of Eq. (4.1). Suppose, by contradiction, that $s_\gamma x \alpha < 0$. Since $-s_\gamma x \alpha \not \in \Delta ( x^{-1} )$ and $-s_\gamma x \alpha > 0$, therefore $x^{-1} s_\gamma x \alpha < 0$. Observe that

$$\begin{aligned} x^{-1}s_\gamma x \alpha = s_{x^{-1} \gamma } \alpha = \alpha - \frac{2(\alpha ,x^{-1}\gamma )}{(x^{-1} \gamma , x^{-1} \gamma )} x^{-1} \gamma \end{aligned}$$

and $x^{-1} \gamma < 0$ since $\gamma \in \Delta ( x^{-1} )$. Since $\alpha$ is simple and $x^{-1} s_\gamma x \alpha < 0$, therefore $( \alpha , x^{-1} \gamma ) < 0.$

Now $s_\gamma x\alpha = x\alpha - \frac{2(\gamma ,x\alpha )}{(\gamma , \gamma )} \gamma = x\alpha - \frac{2(\alpha ,x^{-1} \gamma )}{(\gamma , \gamma )} \gamma > 0$. But $s_\gamma x\alpha < 0$—contradiction. Therefore we must have $s_\gamma x \alpha > 0$, which is the condition for the first case of Eq. (4.1). Thus when $-s_\gamma x \alpha \not \in \Delta ( x^{-1} )$,

$$\begin{aligned} \ell (w) - \ell (s_\gamma ) = \ell (x) - \ell (s_\gamma x ) = \# \Delta ( x^{-1} ) \cap -s_\gamma \Delta ( x^{-1} ). \end{aligned}$$

$\square$

Finally, applying the previous lemma to Proposition 4.9 and from Theorem 3.17, we see that:

Theorem 4.11

Let $\gamma \in \Delta _i^+( \mathfrak {g}, \mathfrak {h})$, $n \in \mathbb {Z}^+$, and let $w \in W_i$ be such that $\gamma$ hyperplanes are positive on $w \mathfrak {C}_0^i$. Then

$$\begin{aligned} \varepsilon ( H_{\gamma ,n}, w) = \left\{ \begin{array}{ll} -1 &{} \quad \text {if } \theta \text { does not fix any element of the} \\ &{} \quad \text {component of } \Pi \text { corresponding to } \gamma \\ \epsilon ( n \gamma ) (-1)^{\frac{1}{2}( \ell _i(w) - \ell _i(s_\gamma w ) - 1 )} &{} \quad \text {otherwise.} \end{array} \right. \end{aligned}$$

From this simplified formula for $\varepsilon ( C, C' ) = \varepsilon ( H_{ \gamma , n}, w )$, we get:

Theorem 4.12

We use the setting and notation of Theorem 3.16. Suppose $\mathfrak {g}_0$ is such that either $\theta$ fixes some element of each component of $\Pi$ or $\theta$ fixes no element of each component of $\Pi$. Let A be an alcove of $W_a^i$ and let $w \in W_i$ be such that $A \subset w \mathfrak {C}_0^i$. Let $A=C_0 {\mathop {\rightarrow }\limits ^{r_1}} C_1 {\mathop {\rightarrow }\limits ^{r_2}} C_2 {\mathop {\rightarrow }\limits ^{r_3}} \cdots {\mathop {\rightarrow }\limits ^{r_\ell }} C_\ell = w A_\circ ^i$ be a path that stays in the Weyl chamber $w \mathfrak {C}_0^i$. Let $I = \{ i_1< \cdots < i_k \} \subset \{ 1, \ldots , \ell \}$. Let $\chi (I) = {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k}$. Consider the translation ${\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k}r_{i_k} \cdots r_{i_1}$. Let $\nu (I) \in \Lambda _r$ be such that $\mu - \nu (I) = {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k}r_{i_k} \cdots r_{i_1} \mu$ for all $\mu \in \mathfrak {h}^*$. If $\varepsilon ( I ) \ne 0$, then

$$\begin{aligned} \varepsilon (I) = \left\{ \begin{array}{ll} (-1)^{|I|} &{} \quad \text {if } \theta \text { does not fix any element of } \Pi \\ \epsilon ( \nu (I) ) (-1)^{\frac{1}{2} ( \ell (w) - \ell ( \chi (I)w ) - |I| )} &{} \quad \text {otherwise.} \end{array} \right. \end{aligned}$$

Proof

The first case is clear, so we suppose we are in the second case. Let $r_j = s_{\beta _j, N_j}$ where $\beta _j \in \Delta _i^+( \mathfrak {g}, \mathfrak {h})$. Then we may show by induction on k that $\nu (I) = N_{i_1} \beta _{i_1} + N_{i_2} {\bar{r}}_{i_1} \beta _{i_2} + \cdots + N_{i_k} {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{k-1}} \beta _{i_k}$. Thus by Theorem 4.11,

$$\begin{aligned} \varepsilon (I)= & {} \varepsilon ( H_{\beta _{i_1, N_{i_1}}}, w) \varepsilon ( H_{{\bar{r}}_{i_1} \beta _{i_2}, N_{i_2}}, {\bar{r}}_{i_1}w ) \cdots \varepsilon ( H_{{\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{k-1}} \beta _{i_k, N_{i_k}}}, {\bar{r}}_{i_1}\cdots {\bar{r}}_{i_{k-1}}w ) \\= & {} \epsilon ( N_{i_1} \beta _{i_1} + N_{i_2} {\bar{r}}_{i_1} \beta _{i_2} + \cdots + N_{i_k} {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{k-1}} \beta _{i_k} ) \\&\times (-1)^{\frac{1}{2}( \ell (w)-\ell ({\bar{r}}_{i_1} w) -1 )} (-1)^{\frac{1}{2}(\ell ({\bar{r}}_{i_1}w)-\ell ({\bar{r}}_{i_1}{\bar{r}}_{i_2} w) -1 )} \cdots (-1)^{\frac{1}{2}( \ell ({\bar{r}}_{i_1}\cdots {\bar{r}}_{i_{k-1}}w)-\ell ( {\bar{r}}_{i_1}\cdots {\bar{r}}_{i_k} w) -1 )}\\= & {} \epsilon ( \nu (I) )(-1)^{\frac{1}{2} ( \ell (w) - \ell ( \chi (I)w ) - |I| )} \end{aligned}$$

$\square$

Corollary 4.13

We use the setting and notation of Theorems 3.16 and 4.12. Let $\Pi = \Pi _1 \cup \Pi _2$ where $\Pi _1$ consists of the components of $\Pi$ for which $\theta$ fixes some element and $\Pi _2$ consists of the components of $\Pi$ for which $\theta$ does not fix any element. Let $\mathfrak {g}_0 = \mathfrak {g}_1 \oplus \mathfrak {g}_2$ be the corresponding decomposition of $\mathfrak {g}_0$ and let $W = W_1 \times W_2$. Let A be an alcove of $W_a^i$ and let $w \in W_i$ be such that $A \subset w \mathfrak {C}_0^i$. Let $A=C_0 {\mathop {\rightarrow }\limits ^{r_1}} C_1 {\mathop {\rightarrow }\limits ^{r_2}} C_2 {\mathop {\rightarrow }\limits ^{r_3}} \cdots {\mathop {\rightarrow }\limits ^{r_{\ell _1}}} C_{\ell _1} {\mathop {\rightarrow }\limits ^{r_{\ell _1+1}}} \cdots {\mathop {\rightarrow }\limits ^{r_{\ell _2}}} C_{\ell _2} = w A_\circ ^i$ be a path that stays in the Weyl chamber $w \mathfrak {C}_0^i$ such that $r_1, \ldots , r_{\ell _1}$ belong to $W_1$ and $r_{\ell _1 + 1}, \ldots , r_{\ell _2}$ belong to $W_2$. Let $w = (w_1, w_2) \in W_1 \times W_2 = W$. Let $I = I_1 \cup I_2$ where $I_1 = \{ i_1< \cdots < i_{k_1} \} \subset \{ 1, \ldots , \ell _1 \}$ and $I_2 = \{i_{k_1 +1}< i_{k_1+2}< \cdots < i_{k_2} \} \subset \{ \ell _1 + 1, \ell _1 + 2, \cdots , \ell _2 \}$. If $\varepsilon (I) \ne 0$, then:

$$\begin{aligned} \varepsilon (I) =\epsilon ( \nu (I_1) ) (-1)^{\frac{1}{2} ( \ell (w_1) - \ell ( \chi (I_1)w_1 ) - |I_1| )} (-1)^{|I_2|}. \end{aligned}$$

Thus Theorem 3.16 becomes:

Theorem 4.14

Use the notation of Theorems 3.16 and 4.12. Suppose $\mathfrak {g}_0$ is such that either $\theta$ fixes some element of each component of $\Pi$ or $\theta$ fixes no element of each component of $\Pi$. Let A be an alcove of $W_a^i$ and let $w \in W_i$ be such that $A \subset w \mathfrak {C}_0^i$. Let $A=C_0 {\mathop {\rightarrow }\limits ^{r_1}} C_1 {\mathop {\rightarrow }\limits ^{r_2}} C_2 {\mathop {\rightarrow }\limits ^{r_3}} \cdots {\mathop {\rightarrow }\limits ^{r_\ell }} C_\ell = w A_\circ ^i$ be an alcove path that stays in the Weyl chamber $w \mathfrak {C}_0^i$. Then for imaginary $\mu \in A$,

$$\begin{aligned} ch_s M( \mu )|_{\mathfrak {a}} = e^{(\mu -\rho )|_{\mathfrak {a}}}. \end{aligned}$$

If $\theta$ does not fix any element of $\Pi$,

$$\begin{aligned}&ch_s M( \mu )|_{\mathfrak {t}} = R^A( \mu |_{\mathfrak {t}} ) \\&\quad = \frac{e^{-\rho } \displaystyle { \sum _{\begin{array}{c} I = \{i_1< \cdots < i_k\} \subset \{1, \ldots , \ell \} \\ w> {\bar{r}}_{i_1} w> {\bar{r}}_{i_1}{\bar{r}}_{i_2} w> \cdots > {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k} w \end{array}}} (-1)^{|I|} 2^{|I|} e^{\mu |_\mathfrak {t}- \nu (I)}}{\prod _{\alpha \in \Delta ^+( \mathfrak {p}, \mathfrak {t})} (1 - e^{-\alpha }) \prod _{\alpha \in \Delta ^+(\mathfrak {k}, \mathfrak {t})}( 1 + e^{-\alpha })}. \end{aligned}$$

Otherwise,

$$\begin{aligned}&ch_s M( \mu )|_{\mathfrak {t}} = R^A( \mu |_{\mathfrak {t}} )\\&\quad = \frac{e^{-\rho } \displaystyle { \sum _{\begin{array}{c} I = \{i_1< \cdots < i_k\} \subset \{1, \ldots , \ell \} \\ w> {\bar{r}}_{i_1} w> {\bar{r}}_{i_1}{\bar{r}}_{i_2} w> \cdots > {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k} w \end{array}}} \epsilon ( \nu (I) ) (-1)^{\frac{1}{2} ( \ell (w) - \ell ( \chi (I)w ) - |I| )} 2^{|I|} e^{\mu |_\mathfrak {t}- \nu (I)}}{\prod _{\alpha \in \Delta ^+( \mathfrak {p}, \mathfrak {t})} (1 - e^{-\alpha }) \prod _{\alpha \in \Delta ^+(\mathfrak {k}, \mathfrak {t})}( 1 + e^{-\alpha })}. \end{aligned}$$

Proof

To prove our formulas, we only need to prove that the condition $\varepsilon (I) \ne 0$ is equivalent to $w> {\bar{r}}_{i_1} w> {\bar{r}}_{i_1}{\bar{r}}_{i_2} w> \cdots > {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k} w$. It clearly holds if I is empty. Otherwise, recall $\varepsilon ( I ) = \varepsilon ( C_{i_1 - 1}, C_{i_1} ) \varepsilon ( {\bar{r}}_{i_1}C_{i_2-1}, {\bar{r}}_{i_1}C_{i_2}) \cdots \varepsilon ( {\bar{r}}_{i_1}\cdots {\bar{r}}_{i_{k-1}}C_{i_k-1}, {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{k-1}}C_{i_k})$. Then for $\varepsilon (I) \ne 0$, we need the affine hyperplane separating ${\bar{r}}_{i_1}\cdots {\bar{r}}_{i_{j-1}} C_{i_j-1}$ and ${\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{j-1}}C_{i_j}$ to be a reducibility hyperplane for each $1 \le j \le k$. The adjacent alcoves lie in the Weyl chamber ${\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{j-1}}w \mathfrak {C}_0^i$. Let $\beta _{i_j} \in \Delta _i^+(\mathfrak {g},\mathfrak {h})$ be the type of the affine hyperplane separating the two alcoves. Thus, we need

$$\begin{aligned} ( \beta _{i_j}, {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{j-1}}w( -\rho _i ) )> 0&\iff ({\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{j-1}}w)^{-1} \beta _{i_j} < 0 \\&\iff \beta _{i_j} \in \Delta (({\bar{r}}_{i_1}\cdots {\bar{r}}_{i_{j-1}})w^{-1}) \\&\iff {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{j-1}}w > s_{\beta _{i_j}} {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{j-1}}w = {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{j-1}} {\bar{r}}_{i_j} w \end{aligned}$$

where the last equality comes from the fact that $r_{i_j}$ corresponds to the affine reflection through the affine hyperplane separating $C_{i_j - 1}$ and $C_{i_j}$, which give us the formula $s_{\beta _{i_j}} = {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{j-1}} {\bar{r}}_{i_j} {\bar{r}}_{i_{j-1}} \cdots {\bar{r}}_{i_1}$. $\square$

Corollary 4.15

We use the setting and notation of Theorems 3.16 and 4.12. Let $\Pi = \Pi _1 \cup \Pi _2$ where $\Pi _1$ consists of the components of $\Pi$ for which $\theta$ fixes some element and $\Pi _2$ consists of the components of $\Pi$ for which $\theta$ does not fix any element. Let $\mathfrak {g}_0 = \mathfrak {g}_1 \oplus \mathfrak {g}_2$ and $\mathfrak {h}^* = \mathfrak {h}_1^* \oplus \mathfrak {h}_2^*$ be the corresponding decompositions of $\mathfrak {g}_0$ and $\mathfrak {h}^*$, respectively. Let $W = W_1 \times W_2$ and $W_a^i = (W_1)_a^i \times (W_2)_a^i$. Let subscript and superscript 1’s and 2’s refer to objects corresponding to $\mathfrak {g}_1$ and $\mathfrak {g}_2$, respectively. Let $\mu = \mu _1 + \mu _2 \in \mathfrak {h}_1^* \oplus \mathfrak {h}_2^*$ be imaginary with $M( \mu )$ irreducible. Let $\mu _1$ belong to the alcove $A_{(1)}$ under the action of $(W_1)_a^i$ on $\mathfrak {h}_1^*$ and $\mu _2$ belong to the alcove $A_{(2)}$ under the action of $(W_2)_a^i$ on $\mathfrak {h}_2^*$. Let $A_{(1)} \subset w_1 (\mathfrak {C}_1)_0^i$ and $A_{(2)} \subset w_2 (\mathfrak {C}_2)_0^i$. Let $A_{(1)}=C_0^1 {\mathop {\rightarrow }\limits ^{r_1^{(1)}}} C_1^1 {\mathop {\rightarrow }\limits ^{r_2^{(1)}}} C_2^1 {\mathop {\rightarrow }\limits ^{r_3^{(1)}}} \cdots {\mathop {\rightarrow }\limits ^{r_{\ell _1}^{(1)}}} C_{\ell _1}^1 = w_1 (A_1)_\circ ^i$ be an alcove path that stays in the Weyl chamber $w_1 (\mathfrak {C}_1)_0^i$. Let $A_{(2)} =C_0^2 {\mathop {\rightarrow }\limits ^{r_1^{(2)}}} C_1^2 {\mathop {\rightarrow }\limits ^{r_2^{(2)}}} C_2^2 {\mathop {\rightarrow }\limits ^{r_3^{(2)}}} \cdots {\mathop {\rightarrow }\limits ^{r_{\ell _2}^{(2)}}} C_{\ell _2}^2 = w_2 (A_2)_\circ ^i$ be an alcove path that stays in the Weyl chamber $w_2 (\mathfrak {C}_2)_0^i$. Then:

$$\begin{aligned} ch_s M( \mu ) |_\mathfrak {a}= e^{(\mu -\rho )|_\mathfrak {a}} \end{aligned}$$

and

$$\begin{aligned} ch_s M( \mu ) |_\mathfrak {t}= \left( ch_s M( \mu _1 ) |_{\mathfrak {t}_1} \right) \left( ch_s M( \mu _2 ) |_{\mathfrak {t}_2} \right) \end{aligned}$$

where

$$\begin{aligned}&ch_s M( \mu _1 )|_{\mathfrak {t}_1}\\&\quad = \frac{e^{-\rho _1} \displaystyle { \sum _{\begin{array}{c} I = \{i_1< \cdots < i_k\} \subset \{1, \ldots , \ell _1\} \\ w_1> {\bar{r}}_{i_1}^{(1)} w_1> {\bar{r}}_{i_1}^{(1)}{\bar{r}}_{i_2}^{(1)} w_1> \cdots > {\bar{r}}_{i_1}^{(1)} \cdots {\bar{r}}_{i_k}^{(1)} w_1 \end{array}}} \epsilon ( \nu (I) ) (-1)^{\frac{1}{2} ( \ell (w_1) - \ell ( \chi (I)w_1 ) - |I| )} 2^{|I|} e^{\mu _1|_{\mathfrak {t}_1} - \nu (I)}}{\prod _{\alpha \in \Delta ^+( \mathfrak {p}_1, \mathfrak {t}_1 )} (1 - e^{-\alpha }) \prod _{\alpha \in \Delta ^+(\mathfrak {k}_1, \mathfrak {t}_1)}( 1 + e^{-\alpha })} \end{aligned}$$

and

$$\begin{aligned} ch_s M( \mu _2 )|_{\mathfrak {t}_2} = \frac{e^{-\rho _2} \displaystyle { \sum _{\begin{array}{c} I = \{i_1< \cdots < i_k\} \subset \{1, \ldots , \ell _2\} \\ w_2> {\bar{r}}_{i_1}^{(2)} w_2> {\bar{r}}_{i_1}^{(2)}{\bar{r}}_{i_2}^{(2)} w_2> \cdots > {\bar{r}}_{i_1}^{(2)} \cdots {\bar{r}}_{i_k}^{(2)} w_2 \end{array}}} (-1)^{|I|} 2^{|I|} e^{\mu _2|_{\mathfrak {t}_2} - \nu (I)}}{\prod _{\alpha \in \Delta ^+( \mathfrak {p}_2, \mathfrak {t}_2 )} (1 - e^{-\alpha }) \prod _{\alpha \in \Delta ^+(\mathfrak {k}_2, \mathfrak {t}_2)}( 1 + e^{-\alpha })}. \end{aligned}$$

5 The affine Hecke algebra and Hall–Littlewood polynomials

For $\mu \in \Lambda ^+$, the Schur polynomial $s_\mu$ is the character of the irreducible highest weight module with highest weight $\mu$. This family of symmetric functions played a prominent role in early work on symmetric functions. It was important in the representation theory of the groups $S_n$ and $GL_n( \mathbb {C})$.

Subsequently (see [15]), Hall and Littlewood independently introduced a one-parameter generalization of the Schur polynomials in [10] and [17], respectively. These polynomials are now called Hall–Littlewood polynomials. In [9], Green drew a connection between Hall–Littlewood polynomials and characters of finite general linear groups. In [22], Macdonald derived an explicit formula for spherical functions of a Chevalley group generalizing the formula for Hall–Littlewood polynomials. This generalized Hall–Littlewood polynomials to all root systems and these polynomials are referred to as both Hall–Littlewood polynomials and as Macdonald spherical functions.

Hall–Littlewood polynomials are a basis for the algebra of symmetric functions. At $q=0$, they are Schur functions and at $q=1$, they are monomial symmetric functions ($m_\mu = \sum _{\nu \in W \mu } X^\nu$). Therefore they interpolate between two well-known bases of the ring of symmetric functions.

Hall–Littlewood polynomials are ubiquitous in mathematics. In addition to the areas already listed, they appear in the study of [16]: projective and modular representations of the symmetric group, Kostka–Foulkes polynomials, unipotent classes and Springer representations, statistical physics, and representations of quantum affine algebras and affine crystals.

We begin this section by recalling the definitions of the Hecke and affine Hecke algebras and related objects which we will need. This material can be found in more detail in [4, 6, 11, 24, 25, 27].

Definition 5.1

Let $\mathbb {K}$ be the field of fractions of $\mathbb {Z}[q^{\frac{1}{2}},q^{-\frac{1}{2}}]$. The Hecke algebra$\mathcal {H}$ is the $\mathbb {K}$-algebra with $\mathbb {K}$-basis $\{ T_w \}_{w \in W}$ such that for all $s \in S$ and $w \in W$:

$$\begin{aligned} T_s T_w = \left\{ \begin{array}{ll} T_{sw} \qquad &{}\quad \text {if } \ell (sw) > \ell (w) \\ (q-1) T_w + qT_{sw} \qquad &{}\quad \text {if } \ell (sw) < \ell (w). \end{array} \right. \end{aligned}$$

Let ${\tilde{T}}_w = q^{-\ell (w)/2} T_w$. We get another presentation of $\mathcal {H}$: $\{{\tilde{T}}_w\}_{w \in W}$ is a $\mathbb {K}$-basis of $\mathcal {H}$ with multiplication rules for $s \in S$ and $w \in W$:

$$\begin{aligned} {\tilde{T}}_s {\tilde{T}}_w = \left\{ \begin{array}{ll} {\tilde{T}}_{sw} \qquad &{}\quad \text {if } \ell (sw) > \ell (w) \\ {\tilde{T}}_{sw} +(q^{\frac{1}{2}}-q^{-\frac{1}{2}}) {\tilde{T}}_w \qquad &{}\quad \text {if } \ell (sw) < \ell (w). \end{array} \right. \end{aligned}$$

From the equations, it can be seen that if $w = s_{i_1} \cdots s_{i_k}$ is a reduced expression, then $T_w = T_{s_{i_1}} \cdots T_{s_{i_k}}$ (similarly for ${\tilde{T}}_w$), so the $T_s$ where $s \in S$ generate $\mathcal {H}$. From the multiplication rules, we have for $s \in S$:

$$\begin{aligned} T_s^{-1} = q^{-1} T_s - (1-q^{-1})T_1. \end{aligned}$$

Therefore $T_{w^{-1}}^{-1} \in \mathcal {H}$ for $w \in W$ and it can be written as a linear combination of $T_x$ where $x \le w$. This leads to the definition of R polynomials. First:

Definition 5.2

Let ${\overline{\cdot }} : \mathbb {Z}[q^{\frac{1}{2}},q^{-\frac{1}{2}}]\rightarrow \mathbb {Z}[q^{\frac{1}{2}},q^{-\frac{1}{2}}]$ be the ring involution defined by $\overline{q^{\frac{1}{2}}} = q^{-\frac{1}{2}}$. It extends to a ring involution of $\mathcal {H}$ such that ${\overline{T}}_w = T_{w^{-1}}^{-1}$ and $\overline{{\tilde{T}}}_w = {\tilde{T}}_{w^{-1}}^{-1}$.

Now we define R polynomials:

Definition 5.3

[12, p. 169] For $x, w \in W$, let $R_{x,w}=0$ if $x \not \le w$. Otherwise, let $R_{x,w} \in \mathbb {Z}[q]$ be the polynomials defined by

$$\begin{aligned} T_{w^{-1}}^{-1} = \sum _{x \in W }{\bar{R}}_{x,w}q^{-\ell (x)} T_x. \end{aligned}$$

Definition 5.4

Let $\alpha = q^{-\frac{1}{2}}-q^{\frac{1}{2}}$. Similarly, for $x, w \in W$ define ${\tilde{R}}_{x,w}=0$ if $x \not \le w$ and otherwise let ${\tilde{R}}_{x,w} \in \mathbb {Z}[\alpha ]$ be defined by

$$\begin{aligned} {\tilde{T}}_{w^{-1}}^{-1} = \sum _{x \le w} {\tilde{R}}_{x,w}(\alpha ) {\tilde{T}}_x. \end{aligned}$$

The R polynomials satisfy the recurrence formulas for $s \in S$, $w \in W$ where $ws < w$

$$\begin{aligned} R_{x,w} = \left\{ \begin{array}{ll} R_{xs,ws} \qquad &{}\quad \text {if } xs < x \\ qR_{xs,ws}+(q-1)R_{x,ws} \qquad &{}\quad \text {if } xs > x. \end{array} \right. \end{aligned}$$

The ${\tilde{R}}$ polynomials satisfy the recurrence formulas for $s \in S$, $w \in W$ where $ws < w$

$$\begin{aligned} {\tilde{R}}_{x,w} = \left\{ \begin{array}{ll} {\tilde{R}}_{xs,ws} \qquad &{}\quad \text {if } xs < x \\ {\tilde{R}}_{xs,ws} + \alpha {\tilde{R}}_{x,ws} \qquad &{}\quad \text {if } xs > x. \end{array} \right. \end{aligned}$$

The R and ${\tilde{R}}$ polynomials are related by the formula

$$\begin{aligned} {\tilde{R}}_{x,w} = q^{\frac{1}{2}(\ell (w)-\ell (x))} {\bar{R}}_{x,w}. \end{aligned}$$

To discuss the affine Hecke algebra, we must discuss the extended affine Weyl group. Elements of the Weyl group also map $\Lambda$ to $\Lambda$, so we can define:

Definition 5.5

The extended affine Weyl group is

$$\begin{aligned} W_e = W \ltimes \Lambda . \end{aligned}$$

The action of $W_e$ on the set of alcoves is no longer free. Let $\Omega$ be the stabilizer of ${\underline{A}}_\circ$ in $W_e$. Then $\Omega \cong \Lambda / \Lambda _r$ where the isomorphism is realized by sending $g \in \Omega$ to the coset $g(0)+ \Lambda _r$. Furthermore, $W_e = W_a \rtimes \Omega$. For every $w \in W_e$, $w = vg$ for unique $v \in W_a$ and $g \in \Omega$. Extend the length function from $W_a$ to $W_e$ by defining $\ell (w) = \ell (v)$. (Note that for $w \in W_a$, $\ell (w)$ is the number of hyperplanes $H_{\alpha ,n}$ separating ${\underline{A}}_\circ$ and $w {\underline{A}}_\circ$ where $\alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h})$ and $n \in \mathbb {Z}$. It is also equal to the length of a reduced expression for w in the generators ${\underline{S}}_a$ of $W_a$.)

Notation 5.6

For $\mu \in \Lambda$, let $\tau _\mu \in W_e$ be translation by $\mu$: $\tau _\mu ( \nu ) = \nu + \mu$.

We now define the affine Hecke algebra.

Definition 5.7

The affine Hecke algebra$\mathcal {H}_a$ is the $\mathbb {K}$-algebra with $\mathbb {K}$-basis $\{{\tilde{T}}_w\}_{w \in W_e}$ with the relations

$$\begin{aligned} {\tilde{T}}_v {\tilde{T}}_w&= {\tilde{T}}_{vw}&\text {if } \ell (vw) = \ell (v)+\ell (w) \\ {\tilde{T}}_s^2&= (q^{\frac{1}{2}}-q^{-\frac{1}{2}}) {\tilde{T}}_s + {\tilde{T}}_1&\text {for } s \in {\underline{S}}_a. \end{aligned}$$

It is also the $\mathbb {K}$-algebra with $\mathbb {K}$-basis $\{ T_w \}_{w \in W_e}$ with the relations

$$\begin{aligned} T_v T_w&= T_{vw}&\text {if } \ell (vw) = \ell (v)+\ell (w) \\ T_s^2&= (q-1) T_s + q T_1&\text {for } s \in {\underline{S}}_a. \end{aligned}$$

We define the following elements of $\mathcal {H}_a$.

Definition 5.8

For $\mu \in \Lambda ^+$, let

$$\begin{aligned} {\tilde{X}}^\mu := {\tilde{T}}_{\tau _\mu } = q^{-\ell (\tau _\mu )/2} T_{\tau _\mu } =: X^\mu = q^{-(\rho ^\vee , \mu )} T_{\tau _\mu }. \end{aligned}$$

For $\mu \in \Lambda$, there are $\mu _1, \mu _2 \in \Lambda ^+$ such that $\mu = \mu _1 - \mu _2$. Let

$$\begin{aligned} {\tilde{X}}^\mu := {\tilde{X}}^{\mu _1} ({\tilde{X}}^{\mu _2})^{-1} = X^{\mu _1}(X^{\mu _2})^{-1} =: X^\mu . \end{aligned}$$

It can be shown that ${\tilde{X}}^\mu {\tilde{X}}^\nu = {\tilde{X}}^{\mu +\nu }= {\tilde{X}}^\nu {\tilde{X}}^\mu$. Thus $\mathbb {K}[\Lambda ] = \mathrm {Span}_{\mathbb {K}} \{{\tilde{X}}^\mu : \mu \in \Lambda \}$ is a subalgebra of $\mathcal {H}_a$.

Proposition 5.9

[21, Prop. 3.7] Two $\mathbb {K}$-bases for $\mathcal {H}_a$ are

$$\begin{aligned} \lbrace {\tilde{T}}_{w^{-1}}^{-1} {\tilde{X}}^\mu : w \in W, \mu \in \Lambda \} \quad \text {and} \quad \lbrace {\tilde{X}}^\mu {\tilde{T}}_{w^{-1}}^{-1} : w \in W, \mu \in \Lambda \}. \end{aligned}$$

Definition 5.10

Let

$$\begin{aligned} {\mathbf {1}}_0 = \sum _{w \in W} T_w = q^{\ell (w_0)} \sum _{w \in W} T_{w^{-1}}^{-1} \end{aligned}$$

and let

$$\begin{aligned} \tilde{{\mathbf {1}}}_0 = \frac{1}{W_0(q^{-1})}\sum _{w \in W} q^{-\ell (w)/2} {\tilde{T}}_{w^{-1}}^{-1} = \frac{1}{W_0(q)}\sum _{w \in W}q^{\ell (w)/2} {\tilde{T}}_w \end{aligned}$$

where $W_0(t) = \sum _{w \in W} t^{\ell (w)}$ is the Poincaré polynomial of W.

Note that $\tilde{{\mathbf {1}}}_0^2 = \tilde{{\mathbf {1}}}_0$ and

$$\begin{aligned} {\tilde{T}}_{w^{-1}}^{-1} \tilde{{\mathbf {1}}}_0 = q^{-\ell (w)/2} \tilde{{\mathbf {1}}}_0. \end{aligned}$$

There is actually a second way $\mathbb {K}[\Lambda ] = \mathrm {Span}_{\mathbb {K}} \{ {\tilde{X}}^\mu : \mu \in \Lambda \}$ is a subalgebra of $\mathcal {H}_a$: by the above formula, $\{{\tilde{X}}^\mu \tilde{{\mathbf {1}}}_0 : \mu \in \Lambda \}$ is a basis of $\mathcal {H}_a {\mathbf {1}}_0$. The map $\Phi$ defined by

$$\begin{aligned} \begin{array}{cccc} \Phi : &{} \mathbb {K}[\Lambda ] &{} \rightarrow &{} \mathcal {H}_a \tilde{{\mathbf {1}}}_0 \\ &{} f &{} \mapsto &{} f \tilde{{\mathbf {1}}}_0 \end{array} \end{aligned}$$

is a vector space isomorphism.

According to a theorem of Bernstein, the centre of $\mathcal {H}_a$ is

$$\begin{aligned} Z(\mathcal {H}_a) = \mathbb {K}[\Lambda ]^W, \end{aligned}$$

the ring of symmetric functions.

The spherical Hecke algebra is $\tilde{{\mathbf {1}}}_0 \mathcal {H}_a \tilde{{\mathbf {1}}}_0$.

Theorem 5.11

The ring of symmetric functions and the spherical Hecke algebra are isomorphic $\mathbb {K}$-algebras via

$$\begin{aligned} \begin{array}{ccccc} \Phi : &{} \mathbb {K}[\Lambda ]^W &{} = Z(\mathcal {H}_a) &{}\rightarrow &{} \tilde{{\mathbf {1}}}_0 \mathcal {H}_a \tilde{{\mathbf {1}}}_0 \\ &{} f &{} &{} \mapsto &{} f \tilde{{\mathbf {1}}}_0. \end{array} \end{aligned}$$

The isomorphism is called the Satake isomorphism.

Thus $\{ \tilde{{\mathbf {1}}}_0 {\tilde{X}}^\mu \tilde{{\mathbf {1}}}_0 : \mu \in \Lambda \}$ is a basis for $\tilde{{\mathbf {1}}}_0 \mathcal {H}_a \tilde{{\mathbf {1}}}_0$. This leads us to the definition of Hall–Littlewood polynomials.

Definition 5.12

Given $\mu \in \Lambda$, recall that $W_\mu$ is the stabilizer of $\mu$ and $W^\mu$ be the set of minimal length coset representatives of $W / W_\mu$. The Hall–Littlewood polynomial or Macdonald spherical function$P_\mu ( X; q^{-1} ) \in \mathbb {K}[\Lambda ]^W$ is the polynomial defined by

$$\begin{aligned} P_\mu (X; q^{-1} ) \tilde{{\mathbf {1}}}_0 = \left( \sum _{w \in W^\mu } q^{-\ell (w)/2} {\tilde{T}}_{w^{-1}}^{-1} \right) {\tilde{X}}^\mu \tilde{{\mathbf {1}}}_0. \end{aligned}$$

In fact,

$$\begin{aligned} P_\mu (X; q^{-1}) \tilde{{\mathbf {1}}}_0 = \frac{W_0(q^{-1})}{W_\mu (q^{-1})} \tilde{{\mathbf {1}}}_0 {\tilde{X}}^\mu \tilde{{\mathbf {1}}}_0 \end{aligned}$$

where $W_\mu ( t ) = \sum _{w \in W_\mu } t^{\ell (w)}$ is the Poincaré polynomial of $W_\mu$.

According to Macdonald,

Theorem 5.13

[22, Theorem 4.1.2] For $\mu \in \Lambda$,

$$\begin{aligned} P_\mu ( X; q^{-1} ) = \frac{1}{W_\mu ( q^{-1} )} \sum _{w \in W} w \left( X^\mu \prod _{\alpha \in \Delta ^+( \mathfrak {g}, \mathfrak {h})} \frac{1 - q^{-1} X^{-\alpha }}{1-X^{-\alpha }} \right) . \end{aligned}$$

Thus we can see that Hall–Littlewood polynomials interpolate between characters of irreducible highest weight modules at $q=0$ for dominant $\mu$ and monomial symmetric functions at $q=1$. The ultimate goal is to classify unitary representations, so we have to determine when signature characters and characters are equal. Thus expressing signature characters in terms of Hall–Littlewood polynomials is useful. This is the purpose of the following two sections.

Remark 5.14

Note that when $q=-1$ and $W_\mu$ is non-trivial, the denominator in Macdonald’s formula, $W_\mu (q^{-1})$, is zero. The formula should be interpreted in the following way when $q=-1$ (see [23, p. 259]). First, substituting $\mu = 0$ in Macdonald’s formula gives the formula for the Poincaré polynomial

$$\begin{aligned} W_0(q^{-1}) = \sum _{w \in W}w \left( \frac{1-q^{-1}X^{-\alpha }}{1-X^{-\alpha }} \right) \end{aligned}$$

[23, p. 207]. Let $\Delta _\mu$ be the roots associated with $W_\mu$ and $\Delta _\mu ^+ = \Delta _\mu \cap \Delta ^+(\mathfrak {g},\mathfrak {h})$. Thus

$$\begin{aligned}&W_\mu (q^{-1}) P_\lambda (X; q^{-1})\\&\quad = \sum _{w \in W} w\left( X^\mu \prod _{\alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h})} \frac{1 -q^{-1} X^{-\alpha }}{1-X^{-\alpha }} \right) \\&\quad = \sum _{v \in W^\mu } \sum _{w \in W_\mu } X^{vw\mu } vw \left( \prod _{\alpha \in \Delta _\mu ^+} \frac{1-q^{-1}X^{-\alpha }}{1-X^{-\alpha }} \prod _{\alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h}) \setminus \Delta _\mu ^+} \frac{1-q^{-1}X^{-\alpha }}{1-X^{-\alpha }} \right) \\&\quad = \sum _{v \in W^\mu } X^{v\mu } \left[ v \sum _{w \in W_\mu } w \prod _{\alpha \in \Delta _\mu ^+} \frac{1-q^{-1}X^{-\alpha }}{1-X^{-\alpha }} \right] \left[ v \prod _{\alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h}) \setminus \Delta _\mu ^+} \frac{1-q^{-1}X^{-\alpha }}{1-X^{-\alpha }} \right] \\&\qquad \text {since }w \in W_\mu \text { permutes the roots in } \Delta ^+(\mathfrak {g},\mathfrak {h}) \setminus \Delta _\mu ^+ \\&\quad = \sum _{v \in W^\mu } X^{v\mu } \left[ v W_\mu (q^{-1}) \right] \left[ v \prod _{\alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h}) \setminus \Delta _\mu ^+} \frac{1-q^{-1}X^{-\alpha }}{1-X^{-\alpha }} \right] \\&\quad = W_\mu (q^{-1}) \sum _{v \in W^\mu } X^{v\mu } v \prod _{\alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h}) \setminus \Delta _\mu ^+} \frac{1-q^{-1}X^{-\alpha }}{1-X^{-\alpha }} \end{aligned}$$

giving us the formula

$$\begin{aligned} P_\lambda (X;q^{-1}) = \sum _{v \in W^\mu } X^{v\mu } v \prod _{\alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h}) \setminus \Delta _\mu ^+} \frac{1-q^{-1}X^{-\alpha }}{1-X^{-\alpha }} \end{aligned}$$

which still holds when $q=-1$.

6 Alcove walks and Hall–Littlewood polynomials

Determining Kostka numbers and Littlewood-Richardson coefficients are two difficult problems in representation theory solved by Littelmann’s path model [18]. In [7], Gaussent and Littelmann introduced galleries, a discrete version of Littelmann paths. They showed that the gallery model and the path model are equivalent. Further work on this discrete version of Littelmann paths appeared in [19] and [20]. (History from [27] and [16].)

In [27], Schwer developed a formula for Hall–Littlewood polynomials in terms of positively folded galleries. In [25], Ram reformulated Schwer’s formula in terms of alcove walks, which originate from Gaussent-Littelmann and Lenart–Postnikov’s work. Using Lenart’s version of Ram’s formula, we will relate signature characters of invariant Hermitian forms on irreducible Verma modules and what we call summands of Hall–Littlewood polynomials.

We will discuss alcove paths, alcove walks, Ram’s alcove walk algebra, and Ram’s presentation of the affine Hecke algebra in terms of alcove walks. We show that the signature character condition $\varepsilon (I ) \ne 0$ corresponds with the condition of positive folding. We establish other basic relationships between Ram’s formula for Hall–Littlewood polynomials and the signature character formula for irreducible Verma modules.

First, we need to expand our definition of alcove. In addition to thinking of $W_e$ as acting on the alcoves in $\mathfrak {h}_0^*$ by both translations and orientation changes, $W_e$ may be thought of in the following way. Tile $\Omega \times \mathfrak {h}_0^*$ with alcoves. (When we are studying signature characters, replace this with $\Omega _i \times$ the imaginary weights.) The extended affine Weyl group $W_e$ acts freely on $\Omega \times \mathfrak {h}_0^*$ so that $g^{-1}{\underline{A}}_\circ$ is in the same place as ${\underline{A}}_\circ$ except on the $g^{\text {th}}$ “sheet” of $\Omega \times \mathfrak {h}_0^*$. (Note that in the signature character work, the action of $W_a$ chosen was $w \in W_a$ acts on $A_\circ$ by $wA_\circ$. In Ram’s work, $w \in W_e$ acts on the alcove ${\underline{A}}_\circ$ in $\Omega \times \mathfrak {h}_0^*$ by $w^{-1} {\underline{A}}_\circ$. This is so that you can list from the left to the right the reflections corresponding to the affine hyperplanes crossed as you take an alcove path [definition below] from A to B.)

Definition 6.1

Two distinct alcoves A and B are adjacent if they share a common alcove wall. Lenart and Ram have two different ways of indexing alcove walls. If the separating wall is $H_{\beta ,k}$, Lenart in [16] writes $A {\mathop {\rightarrow }\limits ^{\beta }} B$ if $A \subset H_{\beta ,k}^-$ and $B \subset H_{\beta ,k}^+$. Ram’s labelling of alcove walls is $W_e$-equivariant. That is, he labels the walls of ${\underline{A}}_\circ$$0, 1, 2, \ldots , m$ corresponding to $H_{\alpha _0,1}, H_{\alpha _1,0}, \ldots , H_{\alpha _m,0}$. Then for $w \in W_e$, the walls of $w{\underline{A}}_\circ$ are also labelled $0, 1, \ldots , m$ in a $W_e$-equivariant way. That is, if the wall F of ${\underline{A}}_\circ$ is labelled i, then the wall wF of $w {\underline{A}}_\circ$ is also labelled i. Another way to state this is that the wall separating ${\underline{A}}_\circ$ and $s_i {\underline{A}}_\circ$ is labelled i. So is the wall separating $w {\underline{A}}_\circ$ and $w s_i {\underline{A}}_\circ$ for every $w \in W_e$.

Definition 6.2

An alcove path from A to B is a sequence of alcoves $A = A_0 {\mathop {\rightarrow }\limits ^{R_1}} A_1 {\mathop {\rightarrow }\limits ^{R_2}} \cdots {\mathop {\rightarrow }\limits ^{R_\ell }} A_\ell = B$ where for $1 \le i \le \ell$, $A_{i-1}$ and $A_i$ are adjacent. Also, $R_i$ is the affine reflection corresponding to the affine hyperplane separating $A_{i-1}$ and $A_i$ and $A_i = R_i A_{i-1}$.

We come to Ram’s definition of the alcove walk algebra from [25]. The vertices of ${\underline{A}}_\circ$ are labelled $0, 1, \ldots , m$ and for $g \in \Omega$, let $\lambda _i$ be the image of the origin under g, i.e. in $g^{-1}{\underline{A}}_\circ$. Let g(i) be the index such that $gs_i g^{-1} = s_{g(i)}$.

Definition 6.3

Let the alcove walk algebra$\mathcal {A}$ be the $\mathbb {K}$-algebra with generators $g \in \Omega$ and for $0 \le i \le m$

with relations (straightening laws)

and

(Graphics are from [25], used with the author’s permission.) Multiplication in the algebra is concatenation of words in the generators. These words may be viewed as a sequence of arrows with the first arrow having its tail in the fundamental alcove.

Definition 6.4

[25] An alcove walk is a word in the generators of $\mathcal {A}$ satisfying:

• (a) the first step has tail in the fundamental alcove ${\underline{A}}_\circ$

• (b) at each step, the head of the arrow and the tail of the subsequent arrow are in the same alcove.

Definition 6.5

The type of an alcove walk is the sequence of labels corresponding to the folds and wall crossings of the walk.

Lenart formulates alcove walks as a generalization of alcove paths in the following way in [16]. An alcove walk from A to B is a sequence $(A=A_0, F_1, A_1, F_2, \ldots , F_\ell , A_\ell = B )$ where the $A_i$ are alcoves and each $F_i$ is a codimension one common face of $A_{i-1}$ and $A_i$.

The difference between an alcove path and an alcove walk is that $A_{i-1}$ and $A_i$ might be the same alcove in an alcove walk.

Definition 6.6

If in an alcove walk $(A=A_0, F_1, A_1, F_2, \ldots , F_\ell , A_\ell = B )$ from A to B$A_{i-1}=A_i$, then i is said to be a folding position of the walk. The fold is positive if both $A_{i-1}$ and $A_i$ lie on the positive side of the affine hyperplane containing $F_i$. It is said to be a negative fold otherwise. If the set of folding positions is empty, then the alcove walk is said to be unfolded.

Lenart addresses orientation changes ($\Omega$) in his formulation by also specifying a weight of a vertex of the final alcove in the alcove walk as the weight of the walk. We will use Lenart’s formulation without specifying the alcove walk’s weight by assuming that any orientation changes are made initially, prior to any affine reflections that take us from A to B. The orientation of the final alcove will be clear as we will be taking paths from ${\underline{A}}_\circ$ to $\lambda + {\underline{A}}_\circ$.

Definition 6.7

An alcove walk is positively folded if all of its folds are positive.

Definition 6.8

For an alcove path p starting in an alcove containing the origin, the weight of p, $wt(p) \in \Lambda$, is the image of the origin under the path. The final direction of p is $\varphi (p) \in W$ where p ends in the alcove $wt(p) + \varphi (p) {\underline{A}}_\circ$. The initial direction is $\iota (p) \in W$ where the starting alcove is $\iota (p) {\underline{A}}_\circ$.

Notation 6.9

Ram uses the notation

• $c_i^+$ for a positive i-crossing,

• $c_i^-$ for a negative i-crossing,

• $f_i^+$ for a positive i-crossing, and

• $f_i^-$ for a negative i-crossing

so that an alcove walk may be written as a string in these symbols.

Note that an arbitrary product in the above generators is not necessarily an alcove walk as the heads of arrows may not meet tails of subsequent arrows when represented diagramatically (see [25, pp. 144–145] for an example). However, its straightening (first applying relations $f_i^{\mp } = - f_i^{\pm }$ and then working left to right applying the relations $c_i^{\pm } = c_i^{\mp } + f_i^{\pm }$) is a sum of alcove walks.

Ram proves that the set of alcove walks is a basis for the alcove walk algebra $\mathcal {A}$. He also describes the affine Hecke algebra in terms of the alcove walk algebra.

Theorem 6.10

[25, Proposition 3.2] Let $\mathcal {H}_{\mathcal {A}}$ be the quotient of the alcove walk algebra $\mathcal {A}$ by the relations

and $p=p'$ if p and $p'$ are nonfolded walks ending in the same alcove. Then $\mathcal {H}_{\mathcal {A}}$ and $\mathcal {H}_a$ are isomorphic via

$$\begin{aligned} {\tilde{T}}_{w^{-1}}^{-1}&\leftrightarrow&\text {image in } \mathcal {H}_{\mathcal {A}} \text { of a minimal length alcove walk from } {\underline{A}}_\circ \text { to } w{\underline{A}}_\circ \quad \text {and}\\ {\tilde{X}}^\mu&\leftrightarrow&\text {image in } \mathcal {H}_{\mathcal {A}} \text { of a minimal length alcove walk from } {\underline{A}}_\circ \text { to } \mu + {\underline{A}}_\circ \end{aligned}$$

for $w \in W$ and $\mu \in \Lambda$.

(Graphics used with modification from [25] with permission of the author.)

Ram proves the following formula for multiplication in the affine Hecke algebra in terms of alcove walks:

Proposition 6.11

[25, Theorem 3.3] For an alcove walk p, let $f^-(p)$ be the number of negative folds of p and f(p) be the total number of folds of p. For $\mu \in \Lambda$ and $w \in W$, fix a minimal length alcove walk $p_w = c_{i_1}^- c_{i_2}^- \cdots c_{i_r}^-$ from ${\underline{A}}_\circ$ to $w{\underline{A}}_\circ$ and a minimal length alcove walk $p_\mu = c_{j_1}^{\epsilon _1} c_{j_2}^{\epsilon _2} \cdots c_{j_s}^{\epsilon _s}$ from ${\underline{A}}_\circ$ to $\mu + {\underline{A}}_\circ$. Then

$$\begin{aligned} {\tilde{T}}_{w^{-1}}^{-1} {\tilde{X}}^\mu = \sum _p (-1)^{f^-(p)}(q^{\frac{1}{2}}- q^{-\frac{1}{2}})^{f(p)} {\tilde{X}}^{wt(p)} {\tilde{T}}_{\varphi (p)^{-1}}^{-1}, \end{aligned}$$

where the indices p are all alcove walks of the form $p = c_{i_1}^- c_{i_2}^- \cdots c_{i_r}^- p_{j_1} p_{j_2} \cdots p_{j_s}$ where $p_{j_k}$ is $c_{j_k}^{\epsilon _k}, c_{j_k}^{-\epsilon _k}$, or $f_{j_k}^{\epsilon _k}$.

This gives a combinatorial formula for the transition matrix between the bases $\{ {\tilde{T}}_{w^{-1}}^{-1} {\tilde{X}}^\mu : w \in W, \mu \in \Lambda \}$ and $\{{\tilde{X}}^\mu {\tilde{T}}_{w^{-1}}^{-1} : \mu \in \Lambda , w \in W \}$ of $\mathcal {H}_a$.

Note that if $\mu$ in the proposition is dominant, then all the $\epsilon _i$ in $p_\mu$ must be $+$ and so the only folds which appear in the straightening of $p_w p_\mu$ are positive folds. Thus it follows from the proposition:

Theorem 6.12

([27, Theorem 5.5], as formulated in [25, Theorem 4.2]) For $\lambda \in \Lambda ^+$, let $p_\lambda = c_{i_1}^+ \cdots c_{i_\ell }^+$ be a minimal length alcove walk from ${\underline{A}}_\circ$ to $\lambda + {\underline{A}}_\circ$. Letting

$$\begin{aligned} B_q( p_\lambda ) = \{ \text {positively folded alcove walks of type } (i_1, \ldots , i_\ell ) \text { beginning at } w {\underline{A}}_\circ | w \in W^\lambda \}, \\ P_\lambda (X ; q^{-1} ) = \sum _{p \in B_q( p_\lambda )} q^{-\frac{1}{2} (\ell (\iota (p))+ \ell (\varphi (p))-f(p))} (1-q^{-1})^{f(p)} X^{wt(p)}. \end{aligned}$$

Note that if p begins at $w {\underline{A}}_\circ$, then $\iota (p)=w$.

To relate Hall–Littlewood polynomials and signature characters of invariant Hermitian forms on irreducible Verma modules, we use Lenart’s analysis of alcove walks from [16].

Definition 6.13

Given an alcove walk $p = (A_0, F_1, A_1, \ldots , F_\ell , A_\ell )$, let the folding operator$\phi _i$ be defined by

$$\begin{aligned} \phi _i( p ) = ( A_0, F_1, A_1, \ldots , A_{i-1}, F_i, A_i', F_{i+1}', A_{i+1}', \ldots , A_\ell ' ) \end{aligned}$$

where for $j \ge i$$A_j' = R_i(A_j)$ and $F_j' = R_i( F_j )$ where $R_i$ is the affine reflection corresponding to the affine hyperplane through $F_i$. In other words, the folding operator leaves $A_0, \ldots , A_{i-1}$ alone and reflects the rest of the walk through the affine hyperplane containing $F_i$.

Lenart observed that any two folding operators commute.

Definition 6.14

Given an alcove walk p, let $fp(p) = \{ j_1< j_2< \cdots < j_s \}$ be the set of folding positions of p. Then the unfold operator is defined to be

$$\begin{aligned} \mathrm {unfold}( p ) = \phi _{j_1} \cdots \phi _{j_s}(p), \end{aligned}$$

which produces an unfolded alcove walk.

Notation 6.15

Let $A_0 = {\underline{A}}_\circ {\mathop {\rightarrow }\limits ^{R_1}} A_1 {\mathop {\rightarrow }\limits ^{R_2}} \cdots {\mathop {\rightarrow }\limits ^{R_\ell }} A_\ell = \lambda + {\underline{A}}_\circ$ be a minimal length alcove path from ${\underline{A}}_\circ$ to $\lambda + {\underline{A}}_\circ$ where $\lambda \in \Lambda ^+$. Denote it by $p_\lambda$.

For $J = \{ j_1< \cdots < j_s \} \subset \{ 1, \ldots , \ell \}$, define

$$\begin{aligned} \phi (J)= & {} {\bar{R}}_{j_1} \cdots {\bar{R}}_{j_s} \in W, \\ \mu (J)= & {} R_{j_1} \cdots R_{j_s} (\lambda ) \in \Lambda . \end{aligned}$$

For $w \in W$, let $w p_\lambda$ be the path $wA_0 \rightarrow wA_1 \rightarrow \cdots \rightarrow w A_\ell$.

Lemma 6.16

[16, Proposition 2.5] Let $B_q( p_\lambda )$ be as defined in Theorem 6.12 and let $J = \{ j_1< \cdots < j_s \}$ denote a subset of $\{ 1, \ldots , \ell \}$. Then:

• (1) The alcove path $p \in B_q( p_\lambda )$ if and only if $p = \phi _{j_1} \cdots \phi _{j_s} ( w p_\lambda )$ for some indices $j_1< \cdots < j_s$ and $w \in W^\lambda$ and

  $$\begin{aligned} w> w {\bar{R}}_{j_1}> w {\bar{R}}_{j_1} {\bar{R}}_{j_2}> \cdots > w {\bar{R}}_{j_1} \cdots {\bar{R}}_{j_s} = w \phi (J ). \end{aligned}$$

• (2) If $p \in B_q( p_\lambda )$, $p = \phi _{j_1} \cdots \phi _{j_s} ( w p_\lambda )$, then $wt( p ) = w \mu (J ).$

• (3) If $p \in B_q( p_\lambda )$, $p = \phi _{j_1} \cdots \phi _{j_s} ( w p_\lambda )$, then $\varphi ( p ) = w \phi (J ).$

• (4) If $p \in B_q( p_\lambda )$, $p = \phi _{j_1} \cdots \phi _{j_s} ( w p_\lambda )$, then $\iota (p)=w$.

Thus Lenart reformulates Theorem 6.12 as:

Theorem 6.17

[16, Theorem 2.7] For $\lambda \in \Lambda ^+$, let $A_0 = {\underline{A}}_\circ {\mathop {\rightarrow }\limits ^{R_1}} A_1 {\mathop {\rightarrow }\limits ^{R_2}} \cdots {\mathop {\rightarrow }\limits ^{R_\ell }} A_\ell = \lambda + {\underline{A}}_\circ$ denote a minimal length alcove path from ${\underline{A}}_\circ$ to $\lambda + {\underline{A}}_\circ$. Then the Hall–Littlewood polynomial corresponding to $\lambda$ may be expressed as

$$\begin{aligned} P_\lambda ( X; q^{-1} ) = \sum _{w \in W^\lambda } \sum _{\begin{array}{c} J = \{ j_1< \cdots < j_s \} \subset \{1, \ldots , \ell \} \\ w> w{\bar{R}}_{j_1}> \cdots > w {\bar{R}}_{j_1} \cdots {\bar{R}}_{j_s} \end{array}} q^{-\frac{1}{2}( \ell (w) + \ell ( w \phi (J)) - |J|)} (1-q^{-1})^{|J|}{\tilde{X}}^{w\mu (J)}. \end{aligned}$$

Note that $J = \emptyset$ is permitted in the summation.

For the purpose of comparison to signature character formulas, we define:

Definition 6.18

Given $\lambda \in \Lambda ^+$, a summand of a Hall–Littlewood polynomial$P_\lambda ( X; q^{-1} )$ is the polynomial $P_\lambda ^w(X,q^{-1}) \in \mathbb {Z}[q^{\frac{1}{2}},q^{-\frac{1}{2}}][\Lambda ]$ defined by

$$\begin{aligned} P_\lambda ^w(X,q^{-1}) \tilde{{\mathbf {1}}}_0 = q^{-\ell (w)/2} {\tilde{T}}_{w^{-1}}^{-1} {\tilde{X}}^\lambda \tilde{{\mathbf {1}}}_0 \end{aligned}$$

for $w \in W$.

Then, by Lenart’s work:

Proposition 6.19

Let $\lambda \in \Lambda ^+$ and let $A_0 = {\underline{A}}_\circ {\mathop {\rightarrow }\limits ^{R_1}} A_1 {\mathop {\rightarrow }\limits ^{R_2}} \cdots {\mathop {\rightarrow }\limits ^{R_\ell }} A_\ell = \lambda + {\underline{A}}_\circ$ denote a minimal length alcove path from ${\underline{A}}_\circ$ to $\lambda + {\underline{A}}_\circ$. Then for $w \in W^\lambda$,

$$\begin{aligned} P_\lambda ^w( X; q^{-1} ) = \sum _{\begin{array}{c} J = \{ j_1< \cdots < j_s \} \subset \{1, \ldots , \ell \} \\ w> w{\bar{R}}_{j_1}> \cdots > w {\bar{R}}_{j_1} \cdots {\bar{R}}_{j_s} \end{array}} q^{-\frac{1}{2}( \ell (w) + \ell ( w \phi (J)) - |J|)} (1-q^{-1})^{|J|}{\tilde{X}}^{w\mu (J)}. \end{aligned}$$

We need the following results to compare summands of Hall–Littlewood polynomials and signature characters of invariant Hermitian forms on irreducible Verma modules:

Proposition 6.20

Let $\lambda \in \Lambda ^+$ be regular and fix $w \in W$. Let

$$\begin{aligned} A_0 = {\underline{A}}_\circ {\mathop {\rightarrow }\limits ^{R_1}} A_1 {\mathop {\rightarrow }\limits ^{R_2}} \cdots {\mathop {\rightarrow }\limits ^{R_\ell }} A_\ell = \lambda + {\underline{A}}_\circ \end{aligned}$$

be a minimal length alcove path from ${\underline{A}}_\circ$ to $\lambda + {\underline{A}}_\circ$. From this path, define the alcove path

$$\begin{aligned} C_0 {\mathop {\rightarrow }\limits ^{r_1}} C_1 {\mathop {\rightarrow }\limits ^{r_2}} \cdots {\mathop {\rightarrow }\limits ^{r_\ell }} C_\ell \end{aligned}$$

where $C_i = w( -\lambda + A_i )$. Use the notation of 6.15 and use the notation $\chi (I)$ and $\nu (I)$ from Theorem 4.12 (although the settings are different: this theorem makes general statements about alcove paths from ${\underline{A}}_\circ$ to $\lambda + {\underline{A}}_\circ$ not necessarily arising from studying signature characters). Then: (1) The path from ${\underline{A}}_\circ$ to $\lambda + {\underline{A}}_\circ$ may be chosen so that the final $\ell (w_0)+1$ alcoves form a path from $\lambda + w_0 {\underline{A}}_\circ$ to $\lambda + {\underline{A}}_\circ$ traversing alcoves of the form $\lambda + x {\underline{A}}_\circ$ where the $x \in W$ appearing come from a reduced expression for $w_0$. Specifically, if $w_0 = s_{j_1} \cdots s_{j_{\ell (w_0)}}$ is a reduced expression, let the final $\ell (w_0)+1$ alcoves be $\lambda + s_{j_1} \cdots s_{j_{\ell (w_0)}} {\underline{A}}_\circ$, $\lambda + s_{j_1} \cdots s_{j_{\ell (w_0)-1}} {\underline{A}}_\circ$, $\cdots$, $\lambda +s_{j_1} {\underline{A}}_\circ$, $\lambda + {\underline{A}}_\circ$. For the remainder of this proposition, assume that we have fixed such a path. The affine hyperplanes crossed on the path from $A_{\ell - \ell (w_0)}$ to $A_\ell$ are of the form $\{ H_{\alpha , (\lambda , \alpha ^\vee )} | \alpha \in \Delta ^+( \mathfrak {g}, \mathfrak {h}) \}$. That is, the affine hyperplanes crossed all contain $\lambda$.

(2) The path from $C_0$ to $C_{\ell - \ell (w_0)}$ stays in the Weyl chamber $w \mathfrak {C}_0$. The affine hyperplanes crossed on the path from $C_{\ell -\ell (w_0)}$ to $C_\ell$ are $\{ H_{\alpha ,0} | \alpha \in \Delta ^+( \mathfrak {g}, \mathfrak {h}) \}$.

(3) The condition

$$\begin{aligned} w> {\bar{r}}_{i_1} w> {\bar{r}}_{i_1} {\bar{r}}_{i_2} w> \cdots > {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k} w \end{aligned}$$

holds if and only if

$$\begin{aligned} w> w {\bar{R}}_{i_1}> w {\bar{R}}_{i_1}{\bar{R}}_{i_2}> \cdots > w {\bar{R}}_{i_1} \cdots {\bar{R}}_{i_k}. \end{aligned}$$

(4) For $I \subset \{ 1, 2, \ldots , \ell - \ell (w_0) \}$,

$$\begin{aligned} \chi (I)w = w \phi (I). \end{aligned}$$

(5) For $I \subset \{ 1, 2, \ldots , \ell - \ell (w_0) \}$,

$$\begin{aligned} w(-\lambda ) - \nu (I) = -w \mu (I). \end{aligned}$$

(6) For $J = I \cup {\bar{J}}$ where $I \subset \{ 1, 2, \ldots , \ell - \ell (w_0) \}$ and ${\bar{J}} \subset \{ \ell - \ell (w_0) + 1, \ell - \ell (w_0) + 2, \cdots , \ell \}$,

$$\begin{aligned} w \mu (I) = w \mu (J). \end{aligned}$$

Proof

• (1) The affine hyperplanes separating ${\underline{A}}_\circ$ and $\lambda + {\underline{A}}_\circ$ are $\{ H_{\alpha , j} : \alpha \in \Delta ^+( \mathfrak {g}, \mathfrak {h}), 1 \le j \le (\lambda , \alpha ^\vee ) \}$. The affine hyperplanes separating $\lambda + {\underline{A}}_\circ$ and $\lambda + w_0 {\underline{A}}_\circ$ are $\{H_{\alpha , (\lambda , \alpha ^\vee )} : \alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h})\}$. Taking a reduced expression for $w_0$ gives a minimal length path between those alcoves. Since the affine hyperplanes separating $\lambda + w_0 {\underline{A}}_\circ$ and ${\underline{A}}_\circ$ are $\{ H_{\alpha , j} : \alpha \in \Delta ^+( \mathfrak {g}, \mathfrak {h}), 1 \le j \le (\lambda , \alpha ^\vee ) -1 \}$, we can take a minimal length path from $\lambda + w_0 {\underline{A}}_\circ$ to $\lambda + {\underline{A}}_\circ$ and extend it to a minimal length path from ${\underline{A}}_\circ$ to $\lambda + {\underline{A}}_\circ$.

• (2) We only need to show that $-\lambda + A_0, -\lambda + A_1, \ldots , -\lambda + A_{\ell - \ell (w_0)} = w_0 {\underline{A}}_\circ$ lie in $\mathfrak {C}_0$. Note that $w_0 {\underline{A}}_\circ = A_\circ \subset \mathfrak {C}_0$. The affine hyperplanes separating $-\lambda + A_0$ and $w_0 {\underline{A}}_\circ$ are $\{ H_{\alpha , j} : \alpha \in \Delta ^+( \mathfrak {g}, \mathfrak {h}), 1 - (\lambda , \alpha ^\vee ) \le j \le -1 \}$. Since no Weyl chamber walls are crossed on the path from $-\lambda + A_0$ to $-\lambda + A_{\ell - \ell (w_0)} = w_0 {\underline{A}}_\circ$, therefore the path from $C_0$ to $C_{\ell - \ell (w_0)}$ stays in the Weyl chamber $w \mathfrak {C}_0$. The path from $-\lambda + A_{\ell - \ell (w_0)}$ to $-\lambda + A_\ell$ traverses alcoves of the form $x {\underline{A}}_\circ$ where the $x \in W$ come from a reduced expression for $w_0$. Therefore the affine hyperplanes crossed on the path from $-\lambda + A_{\ell - \ell (w_0)}$ to $-\lambda + A_\ell$ are $\{ H_{\alpha ,0} : \alpha \in \Delta ^+( \mathfrak {g}, \mathfrak {h}) \}$. Since $w \in W$ sends this set of hyperplanes to itself, therefore the affine hyperplanes crossed on the path from $C_{\ell -\ell (w_0)}$ to $C_\ell$ are $\{ H_{\alpha ,0} | \alpha \in \Delta ^+( \mathfrak {g}, \mathfrak {h}) \}$.

• (3) This follows from the fact that ${\bar{r}}_j = w {\bar{R}}_j w^{-1}$ for $j = 1, \ldots , \ell$.

• (4) This also follows from ${\bar{r}}_j = w {\bar{R}}_j w^{-1}$ for $j = 1, \ldots , \ell$.

• (5) For $1 \le j \le \ell - \ell (w_0)$, let the affine hyperplane $H_{\beta _j, N_j}$ correspond to $R_j$ where $\beta _j \in \Delta ^+( \mathfrak {g}, \mathfrak {h})$ and $N_j \in \mathbb {Z}^+$. Let $I = \{ i_1< \cdots < i_k \} \subset \{ 1, \ldots , \ell - \ell (w_0)\}$.

  $$\begin{aligned} w \mu (I )= & {} w R_{i_1} R_{i_2} \cdots R_{i_k} \lambda \\= & {} w R_{i_1} \cdots R_{i_{k-1}}( {\bar{R}}_{i_k} \lambda + N_{i_k} \beta _{i_k} ) \\= & {} w R_{i_1} \cdots R_{i_{k-2}}( {\bar{R}}_{i_{k-1}} {\bar{R}}_{i_k} \lambda + N_{i_k} {\bar{R}}_{i_{k-1}} \beta _{i_k} + N_{i_{k-1}} \beta _{i_{k-1}} ) \\&\vdots \\= & {} w( {\bar{R}}_{i_1} \cdots {\bar{R}}_{i_k} \lambda + N_{i_k} {\bar{R}}_{i_1} \cdots {\bar{R}}_{i_{k-1}} \beta _{i_k} + N_{i_{k-1}} {\bar{R}}_{i_1} \cdots {\bar{R}}_{i_{k-2}} \beta _{i_{k-1}} + \cdots + N_{i_1} \beta _{i_1} ) \\= & {} (w{\bar{R}}_{i_1}w^{-1} ) \cdots (w {\bar{R}}_{i_k}w^{-1})(w \lambda ) + \sum _{j=1}^k N_{i_j}(w{\bar{R}}_{i_1}w^{-1}) \cdots (w{\bar{R}}_{i_{j-1}}w^{-1}) w \beta _{i_j} \end{aligned}$$

  For $1 \le j \le \ell - \ell (w_0)$, $r_j$ corresponds to the affine hyperplane $w H_{\beta _j, N_j -(\lambda , \beta _j^\vee )} = H_{w \beta _j, N_j - (\lambda , \beta _j^\vee )}$. Note that $w \beta _j$ may not necessarily be a positive root. Let $\gamma _j = w \beta _j$ and let $M_j = N_j - (\lambda , \beta _j^\vee )$.

  $$\begin{aligned} r_{i_1} w (-\lambda )= & {} {\bar{r}}_{i_1} w (-\lambda ) + M_{i_1} \gamma _{i_1} \\ r_{i_2} r_{i_1} w (-\lambda )= & {} {\bar{r}}_{i_2} {\bar{r}}_{i_1} w (-\lambda ) + M_{i_1} {\bar{r}}_{i_2} \gamma _{i_1} + M_{i_2} \gamma _{i_2} \\ \vdots&\vdots \\ r_{i_k} \cdots r_{i_1} w (-\lambda )= & {} {\bar{r}}_{i_k} \cdots {\bar{r}}_{i_1} w (-\lambda ) + M_{i_1} {\bar{r}}_{i_k} \cdots {\bar{r}}_{i_2} \gamma _{i_1} + M_{i_2} {\bar{r}}_{i_k} \cdots {\bar{r}}_{i_3} \gamma _{i_2} + \cdots + M_{i_k} \gamma _{i_k} \end{aligned}$$

  Thus:

  $$\begin{aligned} w (-\lambda ) - \nu (I)= & {} {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k} r_{i_1} \cdots r_{i_k} w (-\lambda ) \\= & {} w (-\lambda ) + M_{i_1} {\bar{r}}_{i_1} \gamma _{i_1} + M_{i_2} {\bar{r}}_{i_1} {\bar{r}}_{i_2} \gamma _{i_2} + \cdots + M_{i_k} {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k} \gamma _{i_k} \\= & {} w (-\lambda ) - \sum _{j=1}^k M_{i_j} {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{j-1}} \gamma _{i_j} \\= & {} {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k} w (-\lambda ) + (w (-\lambda ) -{\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k} w (-\lambda ) ) - \sum _{j=1}^k M_{i_j} {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{j-1}} \gamma _{i_j} \end{aligned}$$

  Now

  $$\begin{aligned} {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k} w (-\lambda )= & {} {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{k-1}} ( w (-\lambda ) - (w (-\lambda ), \gamma _{i_k}^\vee ) \gamma _{i_k} ) \\= & {} {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{k-2}}( w(-\lambda ) -( w (-\lambda ), \gamma _{i_k}^\vee ) {\bar{r}}_{i_{k-1}} \gamma _{i_k} - (w(-\lambda ), \gamma _{i_{k-1}}^\vee ) \gamma _{i_{k-1}} ) \\&\vdots \\= & {} w (-\lambda ) - (w (-\lambda ), \gamma _{i_k}^\vee ) {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{k-1}} \gamma _{i_k} - (w (-\lambda ), \gamma _{i_{k-1}}^\vee ) {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{k-2}} \gamma _{i_{k-1}} \\&- \cdots - (w(-\lambda ), \gamma _{i_1}^\vee ) \gamma _{i_1}, \end{aligned}$$

  so

  $$\begin{aligned} w (-\lambda ) - \nu (I)= & {} {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k} r_{i_1} \cdots r_{i_k} w (-\lambda ) \\= & {} {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k} w (-\lambda ) + \sum _{j=1}^k ((w(-\lambda ), \gamma _{i_j}^\vee ) - M_{i_j}) {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{j-1}} \gamma _{i_j} \\= & {} {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k} w (-\lambda ) + \sum _{j=1}^k ((-\lambda , \beta _{i_j}^\vee ) - M_{i_j}) {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_{j-1}} \gamma _{i_j} \\= & {} -(w{\bar{R}}_{i_1}w^{-1}) \cdots (w {\bar{R}}_{i_k}w^{-1} ) w \lambda - \sum _{j=1}^k N_{i_j} (w {\bar{R}}_{i_1} w^{-1} ) \cdots (w {\bar{R}}_{i_{j-1}} w^{-1} ) w \beta _{i_j} \\= & {} - w\mu (I). \end{aligned}$$

• (6) For $j \in {\bar{J}}$, by (1), $R_j$ corresponds to reflection through a hyperplane of the form $H_{\alpha , (\lambda , \alpha ^\vee )}$ where $\alpha \in \Delta ^+( \mathfrak {g}, \mathfrak {h})$, so it fixes $\lambda$. Therefore if $I = \{ i_1< \cdots < i_k \} \subset \{ 1, 2, \ldots , \ell - \ell (w_0) \}$ and ${\bar{J}} = \{ i_{k+1}< \ldots < i_r \} \subset \{ \ell - \ell (w_0)+1, \ell - \ell (w_0)+2, \ldots , \ell \}$, then

  $$\begin{aligned} w\mu (J) = w R_{i_1} \cdots R_{i_k} R_{i_{k+1}} \cdots R_{i_r} (\lambda ) = w R_{i_1} \cdots R_{i_k} (\lambda ) = w \mu ( I). \end{aligned}$$

$\square$

We observe that the path from $C_{\ell -\ell (w_0)}$ to $C_\ell$ does not cross any reducibility hyperplanes, so these hyperplane crossings do not alter the signature character. Thus we can use the path defined above in Theorem 4.14, although our path ends in $C_\ell =w {\underline{A}}_\circ$ rather than $w A_\circ$ as required by the theorem; however, by our remark on signature characters we can truncate the path to end in $C_{\ell -\ell (w_0)}=w A_\circ$. Then, instead of $I \subset \{ 1, \ldots , \ell \}$ we have $I \subset \{ 1, \ldots , \ell - \ell (w_0)\}$. Thus for $\mu \in {\underline{A}}_\circ$, if $\mathfrak {g}_0$ is compact, then:

$$\begin{aligned}&ch_s M( w(-\lambda + \mu ) ) = R^{-w\lambda + w {\underline{A}}_\circ } ( -w\lambda + w\mu ) \\&\quad = \frac{e^{-\rho + w \mu } \displaystyle { \sum _{\begin{array}{c} I = \{i_1< \cdots< i_k\} \subset \{1, \ldots , \ell - \ell (w_0) \} \\ w> {\bar{r}}_{i_1} w> {\bar{r}}_{i_1}{\bar{r}}_{i_2} w> \cdots> {\bar{r}}_{i_1} \cdots {\bar{r}}_{i_k} w \end{array}}} (-1)^{\frac{1}{2} ( \ell (w) - \ell ( \chi (I)w ) - |I| )} 2^{|I|} e^{w(-\lambda ) - \nu (I)}}{\prod _{\alpha \in \Delta ^+(\mathfrak {g}, \mathfrak {h})}( 1 + e^{-\alpha })} \\&\quad = \frac{e^{-\rho + w \mu } \displaystyle { \sum _{\begin{array}{c} I = \{i_1< \cdots < i_k\} \subset \{1, \ldots , \ell - \ell (w_0) \} \\ w> w{\bar{R}}_{i_1}> w{\bar{R}}_{i_1}{\bar{R}}_{i_2}> \cdots > w{\bar{R}}_{i_1} \cdots {\bar{R}}_{i_k} \end{array}}} (-1)^{\frac{1}{2} ( \ell (w) - \ell ( w\phi (I) ) - |I| )} 2^{|I|} e^{-w\mu (I)}}{\prod _{\alpha \in \Delta ^+(\mathfrak {g}, \mathfrak {h})}( 1 + e^{-\alpha })}. \end{aligned}$$

Compare this with

$$\begin{aligned} P_\lambda ^w(X;-1)= & {} \sum _{ \begin{array}{c} J = I \cup {\bar{J}} \text { where} \\ I = \{i_1< \cdots< i_k \} \subset \{ 1, \ldots , \ell - \ell (w_0) \}, \\ {\bar{J}} = \{ i_{k+1}< \cdots < i_r \} \subset \{ \ell - \ell (w_0)+1, \cdots , \ell \}, \\ w> w{\bar{R}}_{i_1}> w{\bar{R}}_{i_1}{\bar{R}}_{i_2}> \cdots > w{\bar{R}}_{i_1} \cdots {\bar{R}}_{i_r} \end{array}} (-1)^{\frac{1}{2} ( \ell (w) + \ell ( w\phi (J) ) - |J| )} 2^{|J|} {\tilde{X}}^{w\mu (J)} \\= & {} \sum _{ J = I \cup {\bar{J}} \subset \{1, \ldots , \ell \}} (-1)^{\frac{1}{2} ( \ell (w) - \ell ( w\phi (I) ) - |I| )} 2^{|I|} {\tilde{X}}^{w\mu (I)} (2^{|{\bar{J}}|}(-1)^{\frac{1}{2}(\ell (w\phi (I \cup {\bar{J}})) + \ell (w\phi (I)) - |{\bar{J}}|)} ). \end{aligned}$$

For all $I = \{ i_1< \cdots < i_k \} \subset \{ 1, \ldots , \ell - \ell (w_0) \}$ appearing in the two different summations, we will prove in the following section that .

$$\begin{aligned} \sum _{\begin{array}{c} {\bar{J}} = \{ i_{k+1}< \cdots < i_r \} \subset \{ \ell - \ell (w_0) + 1, \cdots , \ell \} \\ w\phi (I) = w {\bar{R}}_{i_1} \cdots {\bar{R}}_{i_k}> w \phi (I) {\bar{R}}_{i_{k+1}}> \cdots > w \phi (I) {\bar{R}}_{i_{k+1}} \cdots {\bar{R}}_{i_r} \end{array}} 2^{|{\bar{J}}|}(-1)^{\frac{1}{2}(\ell (w\phi (I \cup {\bar{J}})) + \ell (w\phi (I)) - |{\bar{J}}|)} =1.\nonumber \\ \end{aligned}$$

(6.2)

7 Signature characters of irreducible Verma modules and summands of Hall–Littlewood polynomials

Throughout this section, we assume that $\mathfrak {g}_0$ is compact. We deal with non-compact real forms in Sect. 10. We compare the Hall–Littlewood polynomial summand $P_\lambda ^w( X; -1 )$ to the signature characters of the irreducible Verma modules $M(w(-\lambda + \mu ) )$ and $M( w(-\lambda ) - \mu )$ where $\mu \in {\underline{A}}_\circ$. We show that the signature characters correspond to the “negative” of the Hall–Littlewood polynomial summand evaluated at $q=-1$ times a version of the Weyl denominator when $\lambda \in \Lambda ^+$ is regular. To do this, we reformulate our summations using material from Matthew Dyer’s thesis [6] on natural orderings and R polynomials.

Notation 7.1

Let $T = \cup _{w \in W} wSw^{-1}$ denote the reflections of W.

Definition 7.2

A subgroup $W'$ of W is a reflection subgroup if it is generated by the reflections it contains: $W' = \langle W' \cap T \rangle$.

Notation 7.3

Given $W'$ a subgroup of W, let $S(W') = \{ t \in T | N(t) \cap W' = \{ t \} \}$ where $N(w) = \{t \in T | \ell (wt ) < \ell (w) \}$.

Definition 7.4

A partial order $\preceq$ on T is a natural order if for every dihedral reflection subgroup $W'$ of W with $S(W') = \{ r, s \}$, either

$$\begin{aligned} r \prec rsr \prec \cdots \prec srs \prec s \qquad \text {or} \qquad s \prec srs \prec \cdots \prec rsr \prec r. \end{aligned}$$

Note that natural orders are also called reflection orders in the literature.

Proposition 7.5

[6, 6.16]

• (1) A natural order is a total order on T.

• (2) The reverse of a natural order is a natural order on T.

• (3) $t_1 \prec t_2 \prec \cdots \prec t_{\ell (w_0)}$ is a natural order on T if and only if there is a reduced expression $w_0 = s_{i_1} \cdots s_{i_{\ell (w_0)}}$ for $w_0$ such that $t_j = s_{i_1} \cdots s_{i_j} \cdots s_{i_1}$.

Definition 7.6

[6, 6.22] Fix a natural order $\preceq$ on T. We make the following definitions:

• (1) Let $C_1 = \{ (x, y ) \in W \times W : x < y, y = x s_\alpha \text { for some } \alpha \in \Delta ^+( \mathfrak {g}, \mathfrak {h}) \}$, the edge set of the Bruhat graph of (W, S).

• (2) For $n \in \mathbb {Z}^+$, $x, y \in W$, define

  $$\begin{aligned} C_n(x, y ) = \{ (x_0, \ldots , x_n) \in W^{n+1} | (x_{i-1}, x_i) \in C_1 \text { for } 1 \le i \le n, x_0 = x, x_n = y \}. \end{aligned}$$

  In other words, $C_n(x,y)$ corresponds to the set of paths of length n from x to y in the Bruhat graph.

• (3) For $x, y \in W$, define a polynomial $r^\prec _{x,y}( \alpha ) \in \mathbb {Z}[\alpha ]$ by setting the coefficient of $\alpha ^n$ in $r^{\prec }_{x,y}$ to be the cardinality of

  $$\begin{aligned} C^\prec _n(x,y) := \{ (x_0, \ldots , x_n) \in C_n( x, y) | x_0^{-1} x_1 \prec x_1^{-1} x_2 \prec \cdots \prec x_{n-1}^{-1} x_n \}. \end{aligned}$$

Theorem 7.7

[6, Theorem 6.23] Let $\preceq$ be a natural order on the reflections of W. Then for all $x, y \in W$,

$$\begin{aligned} r^\prec _{x,y}(\alpha ) = {\tilde{R}}_{x,y}(\alpha ). \end{aligned}$$

As in Proposition 6.20, suppose $\lambda \in \Lambda ^+$ is regular and consider the paths $A_0 = {\underline{A}}_\circ {\mathop {\rightarrow }\limits ^{R_1}} A_1 {\mathop {\rightarrow }\limits ^{R_2}} \cdots {\mathop {\rightarrow }\limits ^{R_\ell }} A_\ell = \lambda + {\underline{A}}_\circ$ and $C_0 {\mathop {\rightarrow }\limits ^{r_1}} C_1 {\mathop {\rightarrow }\limits ^{r_2}} \cdots {\mathop {\rightarrow }\limits ^{r_\ell }} C_\ell$ where $C_i = w( -\lambda + A_i )$. Fix a reduced expression $w_0 = s_{j_1} \cdots s_{j_{\ell (w_0)}}$. Suppose from the reduced expression we constructed a minimal length path from $A_{\ell - \ell (w_0)} = \lambda + w_0 {\underline{A}}_\circ$ to $A_\ell = \lambda + {\underline{A}}_\circ$: $A_{\ell - \ell (w_0)} = \lambda + w_0 {\underline{A}}_\circ$, $A_{\ell -\ell (w_0)+1} = \lambda + s_{i_1} s_{i_2} \cdots s_{i_{\ell (w_0)-1}} {\underline{A}}_\circ$, ..., $A_\ell = \lambda + 1 {\underline{A}}_\circ$. The reduced expression also gives us the natural order $\preceq$ on T:

$$\begin{aligned} {\bar{R}}_\ell \prec {\bar{R}}_{\ell - 1} \prec \cdots \prec {\bar{R}}_{\ell - \ell (w_0) + 2} \prec {\bar{R}}_{\ell - \ell (w_0)+1}. \end{aligned}$$

Fix $I = \{ i_1< \cdots < i_k \} \subset \{ 1, \ldots , \ell - \ell (w_0)\}$ appearing in the two different summations in the discussion after Proposition 6.20. The goal is to prove formula (6.2). Given some ${\bar{J}} = \{ i_{k+1}< \cdots < i_r \}$ appearing in the equation, let $y = x_0 = w {\bar{R}}_{i_1} \cdots {\bar{R}}_{i_k} {\bar{R}}_{i_{k+1}} \cdots {\bar{R}}_{i_r}$, $x_1 = w {\bar{R}}_{i_1} \cdots {\bar{R}}_{i_k} {\bar{R}}_{i_{k+1}} \cdots {\bar{R}}_{i_{r-1}}$, $\ldots$, $x = x_{|J|} = w\phi (I) = w{\bar{R}}_{i_1} \cdots {\bar{R}}_{i_k}$. Note that

$$\begin{aligned} x_0^{-1} x_1 = {\bar{R}}_{i_{r}}, x_1^{-1} x_2 = {\bar{R}}_{i_{r-1}}, \ldots , x_{|J|-1}^{-1}x_{|J|} = {\bar{R}}_{i_{k+1}}. \end{aligned}$$

Thus requiring $i_{k+1}< i_{k+2}< \cdots < i_{r}$ in the summation in (6.2) is equivalent to the requirement that $x_0^{-1} x_1 \prec x_1^{-1} x_2 \prec \cdots \prec x_{|J|-1}^{-1} x_{|J|}$ in the definition of $C_{|J|}^\prec (y,x)$.

Let r(I) be the left side of formula (6.2). Recall $x = w \phi (I)$.

$$\begin{aligned} r(I)= & {} \sum _{\begin{array}{c} {\bar{J}} = \{ i_{k+1}< \cdots< i_r \} \subset \{ \ell - \ell (w_0) + 1, \cdots , \ell \} \\ w\phi (I) = w {\bar{R}}_{i_1} \cdots {\bar{R}}_{i_k}> w \phi (I) {\bar{R}}_{i_{k+1}}> \cdots> w \phi (I) {\bar{R}}_{i_{k+1}} \cdots {\bar{R}}_{i_r} \end{array}} 2^{|{\bar{J}}|}(-1)^{\frac{1}{2}(\ell (w\phi (I \cup {\bar{J}})) + \ell (w\phi (I)) - |{\bar{J}}|)} \nonumber \\= & {} \sum _{\begin{array}{c} {\bar{J}} = \{ i_{k+1}< \cdots < i_r \} \subset \{ \ell - \ell (w_0) + 1, \cdots , \ell \} \\ w\phi (I) = w {\bar{R}}_{i_1} \cdots {\bar{R}}_{i_k}> w \phi (I) {\bar{R}}_{i_{k+1}}> \cdots > w \phi (I) {\bar{R}}_{i_{k+1}} \cdots {\bar{R}}_{i_r} \end{array}} (-2i)^{|{\bar{J}}|}(-1)^{\frac{1}{2}(\ell (w\phi (I \cup {\bar{J}})) + \ell (w\phi (I))} \nonumber \\= & {} \sum _{y \le x} \sum _{n \ge 0} \sum _{(x_0, x_1, \ldots , x_n) \in C_n^\prec (y,x)} (-2i)^n i^{\ell (x)+\ell (y)} \nonumber \\= & {} \sum _{y \le x} r^\prec _{y,x}(-2i) i^{\ell (x)+\ell (y)} \nonumber \\= & {} \sum _{y \le x} {\tilde{R}}_{y,x}(-2i) i^{\ell (x)+\ell (y)} \end{aligned}$$

(7.1)

by Theorem 7.7.

Lemma 7.8

Let $x \in W$. Then

$$\begin{aligned} \sum _{y \le x} {\tilde{R}}_{y,x}(-2i) i^{\ell (x)+\ell (y)} = 1. \end{aligned}$$

(Note that we are evaluating the ${\tilde{R}}_{y,x}$ at $\alpha = -2i$. Recall that $\alpha = q^{-\frac{1}{2}} - q^{\frac{1}{2}}$, so this is equivalent to evaluation at $q = -1$.)

Proof

We prove this result by induction on x. When $x=1$, the formula holds. Assume, by induction, that for $x \in W$,

$$\begin{aligned} \sum _{y \le x} {\tilde{R}}_{y,x}(-2i)i^{\ell (x)+\ell (y)} = 1. \end{aligned}$$

Consider $xs > x$ where $s \in S$.

$$\begin{aligned} \sum _{y \le xs} {\tilde{R}}_{y,xs}(-2i) i^{\ell (xs)+\ell (y)} = \sum _{y \le x} {\tilde{R}}_{y,xs}(-2i) i^{\ell (xs)+\ell (y)} + \sum _{\begin{array}{c} y \not \le x \\ y \le xs \end{array}} {\tilde{R}}_{y,xs}(-2i) i^{\ell (xs)+\ell (y)}. \end{aligned}$$

By Property Z (see [5]), for y appearing in the second sum on the right hand side, $y > ys$ and $ys \le x$. Thus:

$$\begin{aligned} \sum _{y \le xs} {\tilde{R}}_{y,xs}(-2i) i^{\ell (xs)+\ell (y)}= & {} \sum _{\begin{array}{c} y \le x \\ ys< y \end{array}} {\tilde{R}}_{ys,x}(-2i) i^{\ell (x)+ \ell (ys) + 2} \\&+ \sum _{\begin{array}{c} y \le x \\ y < ys \end{array}} \left( {\tilde{R}}_{ys,x}(-2i) i^{\ell (x)+\ell (ys)} -2i {\tilde{R}}_{y,x}(-2i) i^{\ell (x)+\ell (y)+1} \right) \\&+ \sum _{\begin{array}{c} y \not \le x \\ y \le xs \\ (\text {so } y > ys, ys \le x) \end{array}} {\tilde{R}}_{ys,x}(-2i) i^{\ell (x)+\ell (ys)+2} \end{aligned}$$

Combining the first and last sums on the right hand side:

$$\begin{aligned} \sum _{y \le xs} {\tilde{R}}_{y,xs}(-2i) i^{\ell (xs)+\ell (y)}= & {} \sum _{\begin{array}{c} y \le x \\ y< ys \end{array}} {\tilde{R}}_{y,x}(-2i) i^{\ell (x)+\ell (y)} (-1) \\&+ \sum _{\begin{array}{c} y \le x \\ y< ys \end{array}} \left( {\tilde{R}}_{ys,x}(-2i) i^{\ell (x)+\ell (ys)} + 2 {\tilde{R}}_{y,x}(-2i) i^{\ell (x)+\ell (y)} \right) \\= & {} \sum _{\begin{array}{c} y \le x, y < ys \end{array}} \left( {\tilde{R}}_{ys,x}(-2i) i^{\ell (x) +\ell (ys)} + {\tilde{R}}_{y,x}(-2i) i^{\ell (x)+\ell (y)} \right) \\= & {} \sum _{y \le x} {\tilde{R}}_{y,x}(-2i) i^{\ell (x)+\ell (y)} \end{aligned}$$

where the final equality comes from ${\tilde{R}}_{ys,x} = 0$ if $ys > x$. Thus, by induction, for all $x \in W$,

$$\begin{aligned} \sum _{y \le x} {\tilde{R}}_{y,x}(-2i) i^{\ell (x)+\ell (y)} = 1. \end{aligned}$$

$\square$

It follows from Theorem 4.14, Propositions 6.19, 6.20, the discussion following that proposition, the discussion around formula (7.1) and Lemma 7.8:

Theorem 7.9

Suppose $\mathfrak {g}_0$ is compact, $\lambda \in \Lambda ^+$ is regular, and $w \in W$. Then for all $\mu \in {\underline{A}}_\circ$:

$$\begin{aligned} ch_s M( w(-\lambda + \mu ))= & {} R^{w(-\lambda + {\underline{A}}_\circ )}( w(-\lambda + \mu ) ) \\= & {} \frac{e^{w\mu - \rho }}{\prod _{\alpha \in \Delta ^+( \mathfrak {g}, \mathfrak {h})} (1+e^{-\alpha })} \Phi ( P_\lambda ^w( X; -1) ) \end{aligned}$$

where $\Phi ( {\tilde{X}}^\nu ) = e^{-\nu }$ for $\nu \in \Lambda$. Recall that $P_\lambda ^w( X ; q^{-1} ) \tilde{{\mathbf {1}}}_0 = q^{-\ell (w)/2} {\tilde{T}}_{w^{-1}}^{-1} {\tilde{X}}^\lambda \tilde{{\mathbf {1}}}_0$ is a Hall–Littlewood polynomial summand. Also,

$$\begin{aligned} ch_s M( w(-\lambda ) - \mu ) = e^{-\mu - w \mu } ch_s M( w(-\lambda + \mu )) = \frac{e^{-\mu - \rho }}{\prod _{\alpha \in \Delta ^+( \mathfrak {g}, \mathfrak {h})} (1+e^{-\alpha })} \Phi ( P_\lambda ^w( X; -1) ). \end{aligned}$$

Proof

The only thing left to discuss is the final equation. $w(-\lambda ) - \mu$ belongs to the alcove $-w\lambda + w_0 {\underline{A}}_\circ$ while $w(-\lambda + \mu )$ belongs to the alcove $-w\lambda + w{\underline{A}}_\circ$. The alcoves $-w\lambda + {\underline{A}}_\circ$ and $-w\lambda + w {\underline{A}}_\circ$ are separated by hyperplanes of the form $H_{\alpha ,n}$ where $\alpha \in \Delta (w^{-1})$. The only $\beta \in \Delta ^+(\mathfrak {g},\mathfrak {h})$ such that $\beta$ hyperplanes are positive in $w \mathfrak {C}_0$ are $\beta \in \Delta (w^{-1})$. Therefore there are no reducibility hyperplanes separating the alcoves $-w\lambda + w_0 {\underline{A}}_\circ$ and $-w\lambda + w {\underline{A}}_\circ$, so the signatures are the same for weights in those alcoves. $\square$

Example 7.10

Let $\mathfrak {g}_0 = \mathfrak {su}(2)$, $\Delta (\mathfrak {g},\mathfrak {h}) = \{ \pm \alpha _1 \}$, $\Lambda = \mathbb {Z}\lambda _1$. Let $\lambda = n \lambda _1$ where $n \in \mathbb {Z}^+$. Then $A_0 = {\underline{A}}_\circ {\mathop {\rightarrow }\limits ^{s_{\alpha _1,1}}} A_1 {\mathop {\rightarrow }\limits ^{s_{\alpha _1,2}}} \cdots {\mathop {\rightarrow }\limits ^{s_{\alpha _1,n}}} A_n = \lambda + {\underline{A}}_\circ$ is a minimal length alcove path from ${\underline{A}}_\circ$ to $\lambda + {\underline{A}}_\circ$.

When $w=1$, the only set appearing in the summation of Proposition 6.19 is $J = \emptyset$. Thus $P_\lambda ^1(X; q^{-1}) = {\tilde{X}}^\lambda$. It follows from Theorem 7.9 that for $\mu \in {\underline{A}}_\circ$:

$$\begin{aligned} ch_s M(-\lambda + \mu ) = \frac{e^{-\lambda + \mu - \rho }}{1+e^{-\alpha _1}} \qquad \text {and} \qquad ch_s M(-\lambda - \mu ) = \frac{e^{-\lambda - \mu - \rho }}{1+e^{-\alpha _1}}. \end{aligned}$$

By Proposition 6.19,

$$\begin{aligned} P_\lambda ^{s_1}(X; q^{-1})= & {} \sum _{i = 1}^n (1-q^{-1})^1 {\tilde{X}}^{s_1 s_{\alpha _1,i} \lambda } + q^{-1}{\tilde{X}}^{s_1 \lambda } \\= & {} \sum _{i=1}^n (1-q^{-1})^1 {\tilde{X}}^{\lambda - i \alpha _1 } + q^{-1} {\tilde{X}}^{s_1 \lambda } \end{aligned}$$

so $P_\lambda ^{s_1}(X; -1 ) = {\tilde{X}}^{s_1 \lambda } + 2 {\tilde{X}}^{s_1 \lambda + \alpha _1} + 2 {\tilde{X}}^{s_1 \lambda + 2 \alpha _1 } + \cdots + 2 {\tilde{X}}^{s_1\lambda + (n-1)\alpha _1}$, so by Theorem 7.9

$$\begin{aligned} ch_s M(s_1(-\lambda + \mu )) = \frac{e^{s_1 \mu - \rho }}{1+e^{-\alpha _1}} \left( e^{-s_1 \lambda } + 2 e^{-s_1 \lambda - \alpha _1} + 2 e^{-s_1 \lambda - 2 \alpha _1} + \cdots + 2e^{-s_1 \lambda - (n-1) \alpha _1 } \right) \end{aligned}$$

and

$$\begin{aligned} ch_s M(s_1(-\lambda ) - \mu )) = \frac{e^{- \mu - \rho }}{1+e^{-\alpha _1}} \left( e^{-s_1 \lambda } + 2 e^{-s_1 \lambda - \alpha _1} + 2 e^{-s_1 \lambda - 2 \alpha _1} + \cdots + 2e^{-s_1 \lambda - (n-1) \alpha _1 } \right) . \end{aligned}$$

8 The signature character for alcoves that are not translations of the fundamental alcove

In the previous section, we saw that signature characters for weights in alcoves of the form $w(-\lambda + {\underline{A}}_\circ )$ are “negatives” of Hall–Littlewood polynomial summands evaluated at $q=-1$ times a version of the Weyl denominator. In this section, we show how for weights in alcoves of the form $w(-\lambda + x {\underline{A}}_\circ )$ the signature characters are sums of “negatives” of Hall–Littlewood polynomial summands evaluated at $q=-1$ times a version of the Weyl denominator.

Again, throughout this section, $\mathfrak {g}_0$ is compact.

Theorem 8.1

Suppose $\mathfrak {g}_0$ is compact, $\lambda \in \Lambda ^+$ is regular, and $w, x \in W$. Then for $\mu \in {\underline{A}}_\circ$:

$$\begin{aligned} ch_s M(w(-\lambda + x\mu ) ) = \frac{e^{wx\mu - \rho }}{\prod _{\beta \in \Delta ^+(\mathfrak {g},\mathfrak {h})}(1+e^{-\beta })} \sum _{y \in W} c^w_{y,x} \Phi ( P_\lambda ^y(X;-1)) \end{aligned}$$

where the constants $c^{w}_{y,x} \in \mathbb {Z}$ are defined by

• (1) $c^w_{y,1} = \delta _{w,y}$

• (2) $c^w_{y,xs} = c^w_{y,x}$ if $xs > x$ and $w < wxsx^{-1}$ where $s \in S$.

• (3) $c^w_{y,xs} = c^w_{y,x} + 2(-1)^{\frac{1}{2} (\ell (w) - \ell (wxsx^{-1}) -1)} c^{wxsx^{-1}}_{y,x}$ if $xs > x$ and $w > wxsx^{-1}$ where $s \in S$.

Proof

The proof of formulas (1), (2), and (3) shows that $ch_s M( w(-\lambda + x\mu ) )$ is a sum of “negatives” of Hall–Littlewood polynomial summands evaluated at $q = -1$ times the Weyl denominator.

(1) is just Theorem 7.9.

Suppose $x < xs$ where $s \in S$. We get a path for computing the signature character of $M(w(-\lambda + xs\mu ))$ by concatenating a minimal length path from $xs{\underline{A}}_\circ$ to ${\underline{A}}_\circ$ (where the first alcove crossing is from $xs{\underline{A}}_\circ$ to $x {\underline{A}}_\circ$) and a minimal length path from ${\underline{A}}_\circ$ to $\lambda + {\underline{A}}_\circ$, translating by $-\lambda$, and then multiplying by w.

Recall that for adjacent alcoves A and $A'$, $R^A = R^{A'}$ if the hyperplane separating A and $A'$ is not a reducibility hyperplane. Applying this to $A = w(-\lambda + xs {\underline{A}}_\circ )$ and $A' = w(-\lambda + x {\underline{A}}_\circ )$ gives (2).

Suppose $w(-\lambda + xs {\underline{A}}_\circ )$ and $w(-\lambda + x {\underline{A}}_\circ )$ are separated by the reducibility hyperplane $H_{\gamma ,n}$. Then by Lemma 3.14, for $\mu \in {\underline{A}}_\circ$,

$$\begin{aligned} ch_s M(w(-\lambda + xs \mu ))= & {} e^{wxs\mu -wx\mu } ch_s M( w(-\lambda + x\mu )) \\&+ 2\varepsilon ( H_{\gamma ,n}, w) e^{wxs\mu - wxs\mu } ch_s M( w(-\lambda + xs\mu ) - n \gamma ) \\= & {} e^{wxs\mu -wx\mu } ch_s M( w(-\lambda + x\mu )) \\&+ 2\varepsilon ( H_{\gamma ,n}, w) e^{wxs\mu - wxs\mu } ch_s M( wxsx^{-1}(-\lambda ) + wxs\mu ) \\= & {} e^{wxs\mu -wx\mu } ch_s M( w(-\lambda + x\mu )) \\&+ 2(-1)^{\frac{1}{2}(\ell (w) - \ell (wxsx^{-1}) - 1 )} e^{wxs\mu - wxs\mu } ch_s M( wxsx^{-1}(-\lambda + x \mu ) ) \end{aligned}$$

This gives (3). $\square$

9 The singular case

In the case of type $A_2$, all alcoves of the form $w( -\lambda + x {\underline{A}}_\circ )$ where $\lambda \in \Lambda ^+$ is regular include all alcoves except those of the form $w {\underline{A}}_\circ$. Signature characters for the latter alcoves are known since they fall in the region for which Wallach determined a formula. Therefore for type $A_2$, a signature character formula for $w(-\lambda + x {\underline{A}}_\circ )$ where $\lambda$ is singular does not matter. However, this is not true in general. Consider type $A_3$ and consider the alcove $-a \lambda _1 + A_\circ$ for some $a \in \mathbb {Z}^+$. The alcove is in the antidominant Weyl chamber. If $-a \lambda _1 + A_\circ$ is of the form $w(-\lambda + x {\underline{A}}_{\circ })$ where $\lambda \in \Lambda ^+$ is regular, then $w=1$. Then $-\lambda$ is a vertex of $-a \lambda _1 + {\underline{A}}_\circ$. But those vertices are $-a \lambda _1, (-a+1) \lambda _1, -a \lambda _1 + \lambda _2, -a \lambda _1 + \lambda _3$, none of which are regular. Therefore in general, we need to know signature characters for alcoves of the form $w(-\lambda + x {\underline{A}}_\circ )$ where $\lambda \in \Lambda ^+$ is singular in order to know $R^A$ for every alcove A.

Computations done using a computer program written by Dan Ursu suggest that if $\lambda$ is singular then Theorem 7.9 still holds. The singular case is important because, according to Salamanca-Riba [26], it is known that any irreducible unitary Harish-Chandra module with strongly regular infinitesimal character must be isomorphic to some $A_{\mathfrak {q}}( \lambda )$. Thus to classify unitary representations we must classify those of singular infinitesimal character. For alcoves of the form $w(-\lambda + {\underline{A}}_\circ )$, one starting point would be on the Hall–Littlewood side to take a minimal length alcove path from ${\underline{A}}_\circ$ to $\lambda +{\underline{A}}_\circ$ ending with an alcove path from $\lambda + w_0^\lambda w_0 {\underline{A}}_\circ$ to $\lambda + {\underline{A}}_\circ$ where $w_0^\lambda$ is the long element of the stabilizer $W_\lambda$.

10 The noncompact case

We obtain formulas for the noncompact case from the compact case as follows.

Recall $\epsilon : \Lambda _r^i \rightarrow \{ \pm 1 \}$ the $\mathbb {Z}_2$-grading on the imaginary root lattice. We extend it to the formal series $\mathbb {Z}[[\Lambda _r^i]]$:

Notation 10.1

Let $\xi = \sum _{\mu \in \Lambda _r^i} a_\mu e^\mu \in \mathbb {Z}[[\Lambda _r^i]]$. Then

$$\begin{aligned} \epsilon ( \xi ) := \sum _{\mu \in \Lambda _r^i} \epsilon (\mu ) a_\mu e^\mu . \end{aligned}$$

Theorem 10.2

Let $\Pi = \Pi _1 \cup \Pi _2$ where $\Pi _1$ consists of the components of $\Pi$ for which $\theta$ fixes some element and $\Pi _2$ consists of the components of $\Pi$ for which $\theta$ does not fix any element. Let $\mathfrak {g}_0 = \mathfrak {g}_1 \oplus \mathfrak {g}_2$ and $\mathfrak {h}^* = \mathfrak {h}_1^* \oplus \mathfrak {h}_2^*$ be the corresponding decompositions of $\mathfrak {g}_0$ and $\mathfrak {h}^*$, respectively. Let $W_i = W^1_i \times W^2_i$ and $W_a^i = (W_1)_a^i \times (W_2)_a^i$. Let subscript and superscript 1’s and 2’s refer to objects corresponding to $\mathfrak {g}_1$ and $\mathfrak {g}_2$, respectively. Let $\lambda = \lambda _{(1)} + \lambda _{(2)} \in \Lambda _1^+ \cap ({\underline{\mathfrak {C}}}_1)_0^i \oplus \Lambda _2^+ \cap ({\underline{\mathfrak {C}}}_2)_0^i$ be regular and imaginary, and let $\mu _1 \in ({\underline{A}}_1)_\circ ^i$, $\mu _2 \in ({\underline{A}}_2)_\circ ^i$ be imaginary and $\mu = \mu _1 + \mu _2$. Let $w = (w_1, w_2 ) \in W_i = W^1_i \times W^2_i$.

Let $w_2 (\lambda _{(2)} + \mu _2)$ belong to the alcove $A_{(2)}$ under the action of $(W_2)_a^i$ on $\mathfrak {h}_2^*$. Let $A_{(2)} =C_0^2 {\mathop {\rightarrow }\limits ^{r_1^{(2)}}} C_1^2 {\mathop {\rightarrow }\limits ^{r_2^{(2)}}} C_2^2 {\mathop {\rightarrow }\limits ^{r_3^{(2)}}} \cdots {\mathop {\rightarrow }\limits ^{r_{\ell _2}^{(2)}}} C_{\ell _2}^2 = w_2 (A_2)_\circ ^i$ be an alcove path that stays in the Weyl chamber $w_2 (\mathfrak {C}_2)_0^i$. Then:

$$\begin{aligned} ch_s M( w (-\lambda + \mu ) ) |_\mathfrak {a}= e^{(w(-\lambda + \mu ) -\rho )|_\mathfrak {a}} \end{aligned}$$

and

$$\begin{aligned} ch_s M( w(-\lambda + \mu ) ) |_\mathfrak {t}= \left( ch_s M( w_1( -\lambda _{(1)} + \mu _1) ) |_{\mathfrak {t}_1} \right) \left( ch_s M( w_2 ( -\lambda _{(2)} + \mu _2) ) |_{\mathfrak {t}_2} \right) \end{aligned}$$

where

$$\begin{aligned} ch_s M( w_1(-\lambda _{(1)}+\mu _1) )|_{\mathfrak {t}_1} {=} e^{w_1( -\lambda _{(1)}+ \mu _1) -\rho _1} \epsilon \left( \frac{e^{w_1 \lambda _{(1)}}}{\prod _{\alpha \in \Delta ^+( \mathfrak {g}_1, \mathfrak {t}_1 )} (1+e^{-\alpha })} \Phi (P_{\lambda _{(1)}}^{w_1}(X;-1)) \right) \end{aligned}$$

and

$$\begin{aligned}&ch_s M( w_2(-\lambda _{(2)}+\mu _2) )|_{\mathfrak {t}_2}\\&\quad = \frac{e^{-\rho _2} \displaystyle { \sum _{\begin{array}{c} I = \{i_1< \cdots < i_k\} \subset \{1, \ldots , \ell _2\} \\ w_2> {\bar{r}}_{i_1}^{(2)} w_2> {\bar{r}}_{i_1}^{(2)}{\bar{r}}_{i_2}^{(2)} w_2> \cdots > {\bar{r}}_{i_1}^{(2)} \cdots {\bar{r}}_{i_k}^{(2)} w_2 \end{array}}} (-1)^{|I|} 2^{|I|} e^{(w_2(-\lambda _{(2)}+\mu _2))|_{\mathfrak {t}_2} - \nu (I)}}{\prod _{\alpha \in \Delta ^+( \mathfrak {p}_2, \mathfrak {t}_2 )} (1 - e^{-\alpha }) \prod _{\alpha \in \Delta ^+(\mathfrak {k}_2, \mathfrak {t}_2)}( 1 + e^{-\alpha })}. \end{aligned}$$

Proof

This follows from Theorems 4.14 and 7.9. $\square$

Theorem 10.3

Use the notation and setting of Theorem 10.2. Let $w, x=(x_1, x_2) \in W_i$. Let $w_2 (\lambda _{(2)} + x_2 \mu _2)$ belong to the alcove $A_{(2)}$ under the action of $(W_2)_a^i$ on $\mathfrak {h}_2^*$. Let $A_{(2)} =C_0^2 {\mathop {\rightarrow }\limits ^{r_1^{(2)}}} C_1^2 {\mathop {\rightarrow }\limits ^{r_2^{(2)}}} C_2^2 {\mathop {\rightarrow }\limits ^{r_3^{(2)}}} \cdots {\mathop {\rightarrow }\limits ^{r_{\ell _2}^{(2)}}} C_{\ell _2}^2 = w_2 (A_2)_\circ ^i$ be an alcove path that stays in the Weyl chamber $w_2 (\mathfrak {C}_2)_0^i$. Then:

$$\begin{aligned} ch_s M( w (-\lambda + x \mu ) ) |_\mathfrak {a}= e^{(w(-\lambda + x\mu ) -\rho )|_\mathfrak {a}} \end{aligned}$$

and

$$\begin{aligned} ch_s M( w(-\lambda + x\mu ) ) |_\mathfrak {t}= \left( ch_s M( w_1( -\lambda _{(1)} + x_1 \mu _1) ) |_{\mathfrak {t}_1} \right) \left( ch_s M( w_2 ( -\lambda _{(2)} + x_2\mu _2) ) |_{\mathfrak {t}_2} \right) \end{aligned}$$

where

$$\begin{aligned}&ch_s M(w_1(-\lambda _{(1)} + x_1 \mu _1 ) )|_{\mathfrak {t}_1} \\&\quad = e^{w_1 x_1\mu _1 - \rho _1} \sum _{y \in W_i^1} {c}^{w_1}_{y,x_1} e^{-y\lambda _{(1)}} \epsilon \left( \frac{e^{y\lambda _{(1)}}}{\prod _{\beta \in \Delta ^+(\mathfrak {g}_1,\mathfrak {t}_1)}(1+e^{-\beta })} \Phi ( P_{\lambda _{(1)}}^y(X;-1)) \right) \end{aligned}$$

(the constants $c^{w_1}_{y,x_1}$ are defined in Theorem 8.1) and

$$\begin{aligned}&ch_s M( w_2(-\lambda _{(2)}+x_2 \mu _2) )|_{\mathfrak {t}_2}\\&\quad = \frac{e^{-\rho _2} \displaystyle { \sum _{\begin{array}{c} I = \{i_1< \cdots < i_k\} \subset \{1, \ldots , \ell _2\} \\ w_2> {\bar{r}}_{i_1}^{(2)} w_2> {\bar{r}}_{i_1}^{(2)}{\bar{r}}_{i_2}^{(2)} w_2> \cdots > {\bar{r}}_{i_1}^{(2)} \cdots {\bar{r}}_{i_k}^{(2)} w_2 \end{array}}} (-1)^{|I|} 2^{|I|} e^{(w_2(-\lambda _{(2)}+x_2 \mu _2))|_{\mathfrak {t}_2} - \nu (I)}}{\prod _{\alpha \in \Delta ^+( \mathfrak {p}_2, \mathfrak {t}_2 )} (1 - e^{-\alpha }) \prod _{\alpha \in \Delta ^+(\mathfrak {k}_2, \mathfrak {t}_2)}( 1 + e^{-\alpha })}. \end{aligned}$$

11 Signature characters for irreducible highest weight modules

In this section, we assume that $\mathfrak {g}_0$ is equal rank.

We begin this section by discussing the theory of signature characters and signed Kazhdan–Lusztig polynomials for irreducible highest weight modules. Note that this material is only required for the proofs of the results in this section and not their statements. For more details, see [31].

Definition 11.1

For $t \in (-\delta , \delta )$, let $\langle \cdot , \cdot \rangle _t$ be an analytic family of invariant Hermitian forms on a finite-dimensional vector space V where the forms are non-degenerate for $t \ne 0$. The Jantzen filtration is

$$\begin{aligned} V = V^{\langle 0\rangle } \supset V^{\langle 1\rangle } \supset \cdots \supset V^{\langle N\rangle } = \{ 0 \} \end{aligned}$$

where $V^{\langle j\rangle }$ consists of vectors v for which there exists an analytic map $\gamma _v : (-\epsilon , \epsilon ) \rightarrow V$ for some small $\epsilon > 0$ such that

• (1) $\gamma _v(0)=v$, and

• (2) for every $u \in V$, $\langle \gamma _v(t), u \rangle _t$ vanishes at least to order j at $t=0$.

Then

$$\begin{aligned} \langle u, v \rangle ^j := \lim _{t \rightarrow 0^+} \frac{1}{t^j} \langle \gamma _u(t), \gamma _v(t) \rangle _t \qquad \forall u, v \in V \end{aligned}$$

is an invariant Hermitian form on $V^{\langle j \rangle }$ with radical $V^{\langle j+1 \rangle }$. Thus this descends naturally to a non-degenerate Hermitian form $\langle \cdot , \cdot \rangle _j$ on $V^{\langle j \rangle } / V^{\langle j+1 \rangle }$. We refer to this quotient as the $j^{{th}}$level of the Jantzen filtration. Levels of the Jantzen filtration are semisimple [3].

Definition 11.2

Given $\lambda \in \mathfrak {h}^*$, the integral Weyl group is

$$\begin{aligned} W_{[\lambda ]} := \{ w \in W : w \lambda - \lambda \in \Lambda _r \}. \end{aligned}$$

Let $w_\lambda ^0$ denote the long element of $W_{[\lambda ]}$. Let the integral root system of$\lambda$ be

$$\begin{aligned} \Delta _{[\lambda ]} := \{ \alpha \in \Delta (\mathfrak {g},\mathfrak {h}) : (\lambda ,\alpha ^\vee )) \in \mathbb {Z}\}. \end{aligned}$$

The multiplicity of composition factors in a given level of the Jantzen filtration of a Verma module is given by coefficients of Kazhdan–Lusztig polynomials:

Theorem 11.3

[3] Jantzen’s Conjecture: Let $\lambda$ be regular antidominant and x, y belong to the integral Weyl group $W_{[\lambda ]}$. Then

$$\begin{aligned}{}[M(x\lambda )_{\langle j \rangle } : L(y\lambda )] = \text {coefficient of } q^{(\ell (x)-\ell (y)-j)/2} \text { in } P_{w_\lambda ^0 w, w_\lambda ^0 y}(q). \end{aligned}$$

The Jantzen filtration may also be used to study signatures of invariant Hermitian forms.

Lemma 11.4

[28, Proposition 3.3] We use the notation defined in Definition 11.1. Letting $(p_j, q_j)$ be the signature of $\langle \cdot , \cdot \rangle _j$,

$$\begin{aligned} \text {the signature of } \langle \cdot , \cdot \rangle _t \text {is } \left\{ \begin{array}{ll} (\sum _j p_j, \sum _j q_j ) &{} \quad \text {if } t > 0 \\ (\sum _{j \text { even}} p_j + \sum _{j \text { odd}} q_j, \sum _{j \text { odd}} p_j + \sum _{j \text { even}} q_j) &{} \quad \text {if } t < 0. \end{array} \right. \end{aligned}$$

This leads to the definition of signed Kazhdan–Lusztig polynomials. Where the coefficients of classical Kazhdan–Lusztig polynomials record composition factor multiplicity in a given level of the Jantzen filtration, the coefficients of signed Kazhdan–Lusztig polynomials record the contribution to the signature character of a particular type of composition factor to a given level of the Jantzen filtration. Note that composition factor multiplicities in levels of the Jantzen filtration do not depend on the filtration chosen or the highest weight, but these choices affect signatures, so signed Kazhdan–Lusztig polynomials have additional parameters.

Definition 11.5

Given $\lambda$ regular antidominant and $x, y \in W_{[\lambda ]}$, consider an analytic path equal to $x\lambda$ at $t=0$ whose direction as $t \rightarrow 0^+$ is $\delta \in \mathfrak {h}^*$, where $\delta \in w \mathfrak {C}_0$. Consider the invariant Hermitian forms $\langle \cdot , \cdot \rangle _j$ on the levels of the Jantzen filtration on $M(x \lambda )$ arising from that analytic path.

$$\begin{aligned} ch_s \langle \cdot , \cdot \rangle _j = \sum _{y \le x} a^{\lambda ,w}_{w_\lambda ^0 x, w_\lambda ^0 y,j} ch_s L(y \lambda ) \end{aligned}$$

for some integers $a^{\lambda ,w}_{w_\lambda ^0 x, w_\lambda ^0 y,j}$ and we define signed Kazhdan–Lusztig polynomials by

$$\begin{aligned} P^{\lambda ,w}_{w_\lambda ^0 x, w_\lambda ^0 y} (q ) := \sum _{j \ge 0} a^{\lambda ,w}_{w_\lambda ^0 x, w_\lambda ^0 y, j} q^{(\ell (x)-\ell (y)-j)/2}. \end{aligned}$$

Thus by Lemma 11.4:

Proposition 11.6

[31, p. 175] For $\lambda$ regular antidominant, $x \in W_{[\lambda ]}$, $w \in W$, $\mu \in A_\circ$,

$$\begin{aligned} e^{-w\mu } ch_s M( x\lambda + w \mu ) = \sum _{y \le x} P^{\lambda ,w}_{w_\lambda ^0 x, w_\lambda ^0 y}(1) ch_s L(y \lambda ). \end{aligned}$$

For certain values of $w \in W$, signed Kazhdan–Lusztig polynomials and classical Kazhdan–Lusztig polynomials are related in a simple way:

Theorem 11.7

[32, Theorem 4.6 and Remark 4.7] Let $\lambda$ be regular antidominant, and let $x, y \in W_{[\lambda ]}$. Then

$$\begin{aligned} P^{\lambda ,w_0}_{x,y}(q) = \epsilon ( w_\lambda ^0(x\lambda -y\lambda )) P_{x,y}(-q) \end{aligned}$$

and

$$\begin{aligned} P^{\lambda ,1}_{x,y}(q) = (-1)^{\ell (x)-\ell (y)} \epsilon ( w_\lambda ^0(x\lambda -y\lambda )) P_{x,y}(-q). \end{aligned}$$

Using this formula for signed Kazhdan–Lusztig polynomials, the formula of Proposition 11.6 may be inverted to give:

Theorem 11.8

[32, Theorem 5.1] Let $\lambda \in \mathfrak {h}^*$ be regular antidominant, and let $x \in W_{[\lambda ]}$. Then

$$\begin{aligned} ch_s L( x \lambda )= & {} e^{-\mu } \sum _{y \le x} (-1)^{\ell (x)-\ell (y)} P^{\lambda ,w_0}_{y,x}(1) ch_s M( y \lambda + \mu ) \\= & {} e^{-\mu } \sum _{y \le x} (-1)^{\ell (x)-\ell (y)} \epsilon (w_\lambda ^0 (x \lambda - y \lambda ) ) P_{y,x}(-1) ch_s M( y \lambda + \mu ) \end{aligned}$$

and

$$\begin{aligned} ch_s L( x \lambda )= & {} e^{\mu } \sum _{y \le x} (-1)^{\ell (x)-\ell (y)} P^{\lambda ,1}_{y,x}(1) ch_s M( y \lambda - \mu ) \\= & {} e^{\mu } \sum _{y \le x} \epsilon (w_\lambda ^0 (x \lambda - y \lambda ) ) P_{y,x}(-1) ch_s M( y \lambda - \mu ) \end{aligned}$$

where $\mu \in {\underline{A}}_\circ$.

Proof

The first set of equations is Theorem 5.1 of [32] along with Theorem 11.7. We need the second set of equations as well to apply the results of this paper since we have a simple formula for $ch_s M( x\lambda - \mu )$. An adaptation of the proof of Theorem 5.1 of [32] along with Theorem 11.7 prove the second set of equations. $\square$

This concludes our recollection of the theory of signature characters of irreducible highest weight modules and signed Kazhdan–Lusztig polynomials.

From Theorems 11.7 and 11.8, it follows:

Theorem 11.9

Let $\lambda \in \Lambda ^+$ be regular and $x \in W$. If $\mathfrak {g}_0$ is compact, then:

$$\begin{aligned} ch_s L( -x \lambda ) = \sum _{y \le x} P_{y, x}(-1) \frac{e^{-\rho }}{\prod _{\alpha \in \Delta ^+( \mathfrak {g}, \mathfrak {h})} (1 + e^{-\alpha })} \Phi ( P_\lambda ^y(X; -1) ). \end{aligned}$$

If $\mathfrak {g}_0$ is equal rank, recalling the definition of $\epsilon$ from Notation 10.1, then:

$$\begin{aligned} ch_s L( -x \lambda )= & {} \sum _{y \le x} \epsilon (w_0(x\lambda -y\lambda )) P_{y, x}(-1) e^{-y \lambda - \rho } \\&\times \epsilon \,\left( \frac{e^{y\lambda }}{\prod _{\alpha \in \Delta ^+( \mathfrak {g}, \mathfrak {h})} (1+e^{-\alpha })} \Phi (P_{\lambda }^{y}(X;-1)) \right) . \end{aligned}$$

Recall the Kazhdan–Lusztig basis element [12]

$$\begin{aligned} C_x' = q^{-\ell (x)/2} \sum _{y \le x} P_{y,x} T_y. \end{aligned}$$

.

Notation 11.10

Let $C_{\lambda ,x}'(X; q^{-1} ) \in \mathbb {Z}[q^{\frac{1}{2}},q^{-\frac{1}{2}}][\Lambda ]$ be the polynomial satisfying

$$\begin{aligned} C_{\lambda ,x}'(X;q^{-1}) {\tilde{1}}_0 = C_x' {\tilde{X}}^\lambda {\tilde{1}}_0. \end{aligned}$$

Theorem 11.11

Let $\lambda \in \Lambda ^+$ be regular and $x \in W$. If $\mathfrak {g}_0$ is compact, then:

$$\begin{aligned} ch_s L(-x \lambda ) = (-1)^{-\ell (x)/2} \frac{e^{-\rho }}{\prod _{\alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h})}(1 +e^{-\alpha })} \Phi ( C_{\lambda ,x}'(X;-1) ). \end{aligned}$$

If $\mathfrak {g}_0$ is equal rank, recalling the definition of $\epsilon$ from Notation 10.1, then:

$$\begin{aligned} ch_s L(-x \lambda ) =(-1)^{-\ell (x)/2} e^{-x\lambda - \rho } \epsilon \left( \frac{e^{x\lambda }}{\prod _{\alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h})}(1+e^{-\alpha })} \Phi ( C_{\lambda ,x}' (X; -1) ) \right) . \end{aligned}$$

Proof

The Kazhdan–Lusztig basis element $C_x'$ satisfies ${\bar{C}}_x' = C_x'$. Thus

$$\begin{aligned} C_x'= & {} q^{\ell (x)/2} \sum _{y \le x} {\bar{P}}_{y,x} T_{y^{-1}}^{-1} \\= & {} q^{\ell (x)/2} \sum _{y \le x} {\bar{P}}_{y,x} q^{-\frac{\ell (y)}{2}} {\tilde{T}}_{y^{-1}}^{-1}. \end{aligned}$$

The result now follows from the definition of $P_\lambda ^y$, Theorem 11.9, and the fact that $P_{y,w}(-1) = {\bar{P}}_{y,w}(-1)$. $\square$

Finally, we deal with non-integral regular $\lambda$.

Theorem 11.12

Recall that $\mathfrak {g}_0$ is equal rank. Let $\lambda$ be regular dominant. For $x \in W_{[\lambda ]}$, choose $\mu \in {\underline{A}}_\circ$ imaginary such that no affine hyperplanes of the form $H_{\alpha ,n}$ where $\alpha \in \Delta ^+(\mathfrak {g},\mathfrak {h})$ and $n \in \mathbb {Z}$ cross the interiors of the line segments joining $x(-\lambda )$ and $x(-\lambda ) \pm \mu$. Suppose either there exists $\lambda ' \in \Lambda ^+$ regular, $z' \in W$, and $\mu ' \in {\underline{A}}_\circ$ such that $\lambda -\lambda ' \in \{\nu \in \mathfrak {h}^*: \nu \in H_{\alpha ,0} \, \forall \,\alpha \in \Delta _{[\lambda ]}\}$ and $x(-\lambda ) -\mu = x(-\lambda ' + z' \mu ')$; or there exists $\lambda _0 \in \Lambda ^+$ regular, $z_0 \in W$, and $\mu _0 \in {\underline{A}}_\circ$ such that $\lambda -\lambda _0 \in \{\nu \in \mathfrak {h}^*: \nu \in H_{\alpha ,0} \, \forall \,\alpha \in \Delta _{[\lambda ]}\}$ and $x(-\lambda _0) + \mu = x(-\lambda _0 + z_0 \mu _0)$. (Please see the diagram that accompanies the proof.) Then:

$$\begin{aligned} ch_s L(-x\lambda )= & {} \sum _{y \le x} \epsilon ( w_\lambda ^0( x \lambda - y \lambda ) ) P_{y,x}(-1) \\&\times \, \left( \sum _{w \in W} {c}_{w,y^{-1}xz'}^{y} e^{-y\lambda '-\rho } \epsilon \left( \frac{e^{y\lambda '}}{\prod _{\beta \in \Delta ^+(\mathfrak {g},\mathfrak {h})}(1+e^{-\beta })} \Phi ( P_{\lambda '}^y(X;-1)) \right) \right) \end{aligned}$$

in the first case, and in the second,

$$\begin{aligned} ch_s L(-x\lambda )= & {} \sum _{y \le x} (-1)^{\ell (x)-\ell (y)} \epsilon ( w_\lambda ^0( x \lambda - y \lambda ) ) P_{y,x}(-1) \\&\times \, \left( \sum _{w \in W} {c}_{w,y^{-1}xz_0}^{y} e^{-y\lambda _0-\rho } \epsilon \left( \frac{e^{y\lambda _0}}{\prod _{\beta \in \Delta ^+(\mathfrak {g},\mathfrak {h})}(1+e^{-\beta })} \Phi ( P_{\lambda _0}^y(X;-1)) \right) \right) . \end{aligned}$$

Proof

We prove only the first equation as the proof of the second is similar. We begin by proving that if $x(-\lambda ) - \mu = x(-\lambda ' + z'\mu ')$, then $y(-\lambda )-\mu = y(-\lambda ' + y^{-1}xz'\mu ' )$. The interior of the line segment joining $x(-\lambda )$ and $x(-\lambda ) - \mu$ lies in the interior of an alcove, as does the interior of the line segment joining $x(-\lambda ')$ and $x(-\lambda ')+xz' \mu ' = x(-\lambda ) - \mu$. Therefore there is a path from $x(-\lambda )$ to $x(-\lambda ')$ such that the interior lies in the interior of an alcove. Thus $x(-\lambda )-x(-\lambda ') \in \overline{{\underline{A}}}_\circ \cap \{ \nu \in \mathfrak {h}^* : \nu \in H_{\alpha ,0} \, \forall \, \alpha \in \Delta _{[\lambda ]}\}.$ Since $yx^{-1} \in W_{[\lambda ]}$, therefore $x(-\lambda )-x(-\lambda ') = y(-\lambda ) - y(-\lambda ') = \mu + xz'\mu ' = \mu + y(y^{-1}xz'\mu ')$. Therefore $y(-\lambda ) - \mu = y(-\lambda '+y^{-1}xz' \mu ')$. Thus:

$$\begin{aligned} ch_s L(-x\lambda )= & {} \sum _{y \le x} \epsilon ( w_\lambda ^0( x \lambda - y \lambda ) ) e^\mu P_{y,x}(-1) ch_s M( -y \lambda - \mu ) \\= & {} \sum _{y \le x} \epsilon ( w_\lambda ^0( x \lambda - y \lambda ) ) e^\mu P_{y,x}(-1) ch_s M( y(-\lambda ' +y^{-1}xz' \mu ' )) \\= & {} \sum _{y \le x} \epsilon ( w_\lambda ^0( x \lambda - y \lambda ) ) e^\mu P_{y,x}(-1) \\&\times \, \left( \sum _{w \in W} {c}_{w,y^{-1}xz'}^{y} e^{-y\lambda '-xz'\mu '-\rho } \epsilon \left( \frac{e^{y\lambda '}}{\prod _{\beta \in \Delta ^+(\mathfrak {g},\mathfrak {h})}(1+e^{-\beta })} \Phi ( P_{\lambda '}^y(X;-1)) \right) \right) \\= & {} \sum _{y \le x} \epsilon ( w_\lambda ^0( x \lambda - y \lambda ) ) e^{\mu -xz'\mu '} P_{y,x}(-1) \\&\times \, \left( \sum _{w \in W} {c}_{w,y^{-1}xz'}^{y} e^{-y\lambda '-\rho } \epsilon \left( \frac{e^{y\lambda '}}{\prod _{\beta \in \Delta ^+(\mathfrak {g},\mathfrak {h})}(1+e^{-\beta })} \Phi ( P_{\lambda '}^y(X;-1)) \right) \right) . \end{aligned}$$

$\square$

Remark 11.13

Note that we deformed $x(-\lambda )$ in directions so that the signed Kazhdan–Lusztig polynomials would be easy to evaluate, but then we needed the constants ${c}_{y,x}^w$ in order to rewrite $ch_s M( y(-\lambda ' + y^{-1}xz' \mu ' ))$ in our formula for $ch_s L( -x\lambda )$. We could have chosen the deformation direction so that $z'=x^{-1}$ so that all of the signature characters appearing are of the form $ch_s M( y(-\lambda ' + y^{-1}xz' \mu ' )) = ch_s M( y \lambda ' - \mu ' )$ in the formula for $ch_s L(-x\lambda )$ which can be expressed without the constants $c^w_{y,x}$. However, we would then have had in our formula $P^{\lambda , w}_{y,x}$ evaluated at w not necessarily equal to 1 or $w_0$, for which at the moment we do not have a simple formula. This suggests that the constants ${c}^w_{y,x}$ and signed Kazhdan–Lusztig polynomials are related in some way.