Unitary Representations of Unitary Groups

Abstract

In this paper we review and streamline some results of Kirillov, Olshanski and Pickrell on unitary representations of the unitary group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ of a real, complex or quaternionic separable Hilbert space and the subgroup $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$, consisting of those unitary operators g for which g −1 is compact. The Kirillov–Olshanski theorem on the continuous unitary representations of the identity component $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ asserts that they are direct sums of irreducible ones which can be realized in finite tensor products of a suitable complex Hilbert space. This is proved and generalized to inseparable spaces. These results are carried over to the full unitary group by Pickrell’s theorem, asserting that the separable unitary representations of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$, for a separable Hilbert space $\mathcal{H}$, are uniquely determined by their restriction to $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$. For the 10 classical infinite rank symmetric pairs (G, K) of non-unitary type, such as $(\mathop{\mathrm{GL}}\nolimits (\mathcal{H}),\mathop{\mathrm{U}}\nolimits (\mathcal{H}))$, we also show that all separable unitary representations are trivial.

FormalPara Introduction

One of the most drastic difference between the representation theory of finite-dimensional Lie groups and infinite-dimensional ones is that an infinite-dimensional Lie group G may carry many different group topologies and any such topology leads to a different class of continuous unitary representations. Another perspective on the same phenomenon is that the different topologies on G lead to different completions, and the passage to a specific completion reduces the class of representations under consideration.

In the present paper we survey results and methods of A. Kirillov, G. Olshanski and D. Pickrell from the point of view of Banach–Lie groups. In the unitary representation theory of finite-dimensional Lie groups, the starting point is the representation theory of compact Lie groups and the prototypical compact Lie group is the unitary group $\mathop{\mathrm{U}}\nolimits (n,\mathbb{C})$ of a complex n-dimensional Hilbert space. Therefore any systematic representation theory of infinite-dimensional Banach–Lie groups should start with unitary groups of Hilbert spaces. For an infinite-dimensional Hilbert space $\mathcal{H}$, there is a large variety of unitary groups. First, there is the full unitary group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$, endowed with the norm topology, turning it into a simply connected Banach–Lie group with Lie algebra $\mathfrak{u}(\mathcal{H}) =\{ X \in B(\mathcal{H}): X^{{\ast}} = -X\}$. However, the much coarser strong operator topology also turns it into another topological group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})_{s}$. The third variant of a unitary group is the subgroup $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$ of all unitary operators g for which g −1 is compact. This is a Banach–Lie group whose Lie algebra $\mathfrak{u}_{\infty }(\mathcal{H})$ consists of all compact operators in $\mathfrak{u}(\mathcal{H})$. If $\mathcal{H}$ is separable (which we assume in this introduction) and $(e_{n})_{n\in \mathbb{N}}$ is an orthonormal basis, then we obtain natural embeddings $\mathop{\mathrm{U}}\nolimits (n,\mathbb{C}) \rightarrow \mathop{\mathrm{U}}\nolimits (\mathcal{H})$ whose union $\mathop{\mathrm{U}}\nolimits (\infty,\mathbb{C}) =\bigcup _{ n=1}^{\infty }\mathop{\mathrm{U}}\nolimits (n,\mathbb{C})$ carries the structure of a direct limit Lie group (cf. [Gl03]). Introducing also the Banach–Lie groups $\mathop{\mathrm{U}}\nolimits _{p}(\mathcal{H})$, consisting of unitary operators g, for which g −1 is of Schatten class p ∈ [1, ∞], i.e., $\mathop{\mathrm{tr}}\nolimits (\vert U -\mathbf{1}\vert ^{p}) < \infty$, we thus obtain an infinite family of groups with continuous inclusions

$$\displaystyle{\mathop{\mathrm{U}}\nolimits (\infty,\mathbb{C})\hookrightarrow \mathop{\mathrm{U}}\nolimits _{1}(\mathcal{H})\hookrightarrow \cdots \hookrightarrow \mathop{\mathrm{U}}\nolimits _{p}(\mathcal{H})\hookrightarrow \cdots \hookrightarrow \mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})\hookrightarrow \mathop{\mathrm{U}}\nolimits (\mathcal{H}) \rightarrow \mathop{\mathrm{U}}\nolimits (\mathcal{H})_{s}.}$$

The representation theory of infinite-dimensional unitary groups began with I. E. Segal’s paper [Se57], where he studies unitary representations of the full group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$, called physical representations. These are characterized by the condition that their differential maps finite rank hermitian projections to positive operators. Segal shows that physical representations decompose discretely into irreducible physical representations which are precisely those occurring in the decomposition of finite tensor products $\mathcal{H}^{\otimes N}$, $N \in \mathbb{N}_{0}$. It is not hard to see that this tensor product decomposes as in classical Schur–Weyl theory:

$$\displaystyle{ \mathcal{H}^{\otimes N}\mathop{\cong}\bigoplus _{ \lambda \in \mathop{\mathrm{Part}}\nolimits (N)}\mathbb{S}_{\lambda }(\mathcal{H}) \otimes \mathcal{M}_{\lambda }, }$$

(1)

where $\mathop{\mathrm{Part}}\nolimits (N)$ is the set of all partitions $\lambda = (\lambda _{1},\ldots,\lambda _{n})$ of N, $\mathbb{S}_{\lambda }(\mathcal{H})$ is an irreducible unitary representation of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ (called a Schur representation), and $\mathcal{M}_{\lambda }$ is the corresponding irreducible representation of the symmetric group S _N , hence in particular finite-dimensional (cf. [BN12] for an extension of Schur–Weyl theory to irreducible representations of C ^∗-algebras). In particular, $\mathcal{H}^{\otimes N}$ is a finite sum of irreducible representations of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$.

The representation theory of the Banach Lie group $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$, $\mathcal{H}$ a separable real, complex or quaternionic Hilbert space, was initiated by A. A. Kirillov in [Ki73] (which contains no proofs), and continued by G. I. Olshanski [Ol78, Thm. 1.11]. They showed that all continuous representations of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$ are direct sums of irreducible representations and that for $\mathbb{K} = \mathbb{C}$, all the irreducible representations are of the form $\mathbb{S}_{\lambda }(\mathcal{H}) \otimes \mathbb{S}_{\mu }(\overline{\mathcal{H}})$, where $\overline{\mathcal{H}}$ is the space $\mathcal{H}$, endowed with the opposite complex structure. They also obtained generalizations for the corresponding groups over real and quaternionic Hilbert spaces. It follows in particular that all irreducible representations $(\pi,\mathcal{H}_{\pi })$ of the Banach–Lie group $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$ are bounded in the sense that $\pi: \mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H}) \rightarrow \mathop{\mathrm{U}}\nolimits (\mathcal{H}_{\pi })$ is norm continuous, resp., a morphism of Banach–Lie groups. The classification of the bounded unitary representations of the Banach–Lie group $\mathop{\mathrm{U}}\nolimits _{p}(\mathcal{H})$ remains the same for 1 < p < ∞, but for p = 1, factor representations of type II and III exist (see [Boy80] for p = 2, and [Ne98] for the general case). Dropping the boundedness assumptions even leads to a non-type I representation theory for $\mathop{\mathrm{U}}\nolimits _{p}(\mathcal{H})$, p < ∞ (cf. [Boy80, Thm. 5.5]). We also refer to [Boy93] for an approach to Kirillov’s classification based on the classification of factor representations of $\mathop{\mathrm{U}}\nolimits (\infty,\mathbb{C})$ from [SV75].

These results clearly show that the group $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$ is singled out among all its relatives by the fact that its unitary representation theory is well-behaved. If $\mathcal{H}$ is separable, then $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$ is separable, so that its cyclic representations are separable as well. Hence there is no need to discuss inseparable representations for this group. This is different for the Banach–Lie group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ which has many inseparable bounded irreducible unitary representations coming from irreducible representations of the Calkin algebra $B(\mathcal{H})/K(\mathcal{H})$. It was an amazing insight of D. Pickrell [Pi88] that restricting attention to representations on separable spaces tames the representation theory of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ in the sense that all its separable representations are actually continuous with respect to the strong operator topology, i.e., continuous representations of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})_{s}$. For analogous results on the automatic weak continuity of separable representations of W ^∗-algebras see [FF57, Ta60]. Since $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ is dense in $\mathop{\mathrm{U}}\nolimits (\mathcal{H})_{s}$, it follows that $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ has the same separable representation theory as $\mathop{\mathrm{U}}\nolimits (\mathcal{H})_{s}$. As we shall see below, all these results extend to unitary groups of separable real and quaternionic Hilbert spaces.

Here we won’t go deeper into the still not completely developed representation theory of groups like $\mathop{\mathrm{U}}\nolimits _{2}(\mathcal{H})$ which also have a wealth of projective unitary representations corresponding to nontrivial central Lie group extensions [Boy84, Ne13]. Instead we shall discuss the regular types of unitary representation and their characterization. For the unitary groups, the natural analogs of the finite-dimensional compact groups, a regular setup is obtained by considering $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$ or the separable representations of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$. For direct limit groups, such as $\mathop{\mathrm{U}}\nolimits (\infty,\mathbb{C})$, the same kind of regularity is introduced by Olshanski’s concept of a tame representation. Here a fundamental result is that the tame unitary representation of $\mathop{\mathrm{U}}\nolimits (\infty,\mathbb{C})_{0}$ are precisely those extending to continuous representations of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ ([Ol78]; Theorem 3.20).

The natural next step is to take a closer look at unitary representations of the Banach analogs of noncompact classical groups; we simply call them non-unitary groups. There are 10 natural families of such groups that can be realized by ∗-invariant groups of operators with a polar decomposition

$$\displaystyle{G = K\exp \mathfrak{p},\ \ \text{ where }\ \ K =\{ g \in G: g^{{\ast}} = g^{-1}\}\ \ \text{ and }\ \ \mathfrak{p} =\{ X \in \mathfrak{g}: X^{{\ast}} = X\}}$$

(see the tables in Sect. 5). In particular K is the maximal unitary subgroup of G. In this context Olshanski calls a continuous unitary representation of G admissible if its restriction to K is tame. For the cases where the symmetric space G∕K is of finite rank, Olshanski classifies in [Ol78, Ol84] the irreducible admissible representations and shows that they form a type I representation theory (see also [Ol89]).

The voluminous paper [Ol90] deals with the case where G∕K is of infinite rank. It contains a precise conjecture about the classification of the irreducible representations and the observation that in general, there are admissible factor representations not of type I. We refer to [MN13] for recent results related to Olshanski’s conjecture and to [Ne12] for the classification of the semibounded projective unitary representations of hermitian Banach–Lie groups. Both continue Olshanski’s program in the context of Banach–Lie groups of operators.

In [Pi90, Prop. 7.1], Pickrell shows for the 10 classical types of symmetric pairs (G, K) of non-unitary type that for q > 2, all separable projective unitary representations are trivial for the restricted groups $G_{(q)} = K\exp (\mathfrak{p}_{(q)})$ with Lie algebra

$$\displaystyle{\mathfrak{g}_{(q)} = \mathfrak{k} \oplus \mathfrak{p}_{(q)}\quad \text{ and }\quad \mathfrak{p}_{(q)}:= \mathfrak{p} \cap B_{q}(\mathcal{H}),}$$

where $B_{q}(\mathcal{H}) \trianglelefteq B(\mathcal{H})$ is the qth Schatten ideal. This complements the observation that admissible representations often extend to the restricted groups G ₍₂₎ [Ol90]. From these results we learn that for q > 2, the groups G _(q) are too big to have nontrivial separable unitary representations and that the groups G ₍₂₎ have just the right size for a rich nontrivial separable representation theory. An important consequence is that G itself has no non-trivial separable unitary representation. This applies in particular to the group $\mathop{\mathrm{GL}}\nolimits _{\mathbb{K}}(\mathcal{H})$ of $\mathbb{K}$-linear isomorphisms of a $\mathbb{K}$-Hilbert space $\mathcal{H}$ and the group $\mathop{\mathrm{Sp}}\nolimits (\mathcal{H})$ of symplectic isomorphism of the symplectic space underlying a complex Hilbert space $\mathcal{H}$.

This is naturally extended by the fact that for the 10 symmetric pairs (G, K) of unitary type and q > 2, all continuous unitary representations of G _(q) extend to continuous representations of the full group G [Pi90]. This result has interesting consequences for the representation theory of mapping groups. For a compact spin manifold M of odd dimension d, there are natural homomorphisms of the group C ^∞(M, K), K a compact Lie group, into $U(\mathcal{H}\oplus \mathcal{H})_{(d+1)}$, corresponding to the symmetric pair $(\mathop{\mathrm{U}}\nolimits (\mathcal{H}\oplus \mathcal{H}),\mathop{\mathrm{U}}\nolimits (\mathcal{H}))$ (cf. [PS86, Mi89, Pi89]). For d = 1, the rich projective representation theory of $U(\mathcal{H}\oplus \mathcal{H})_{(d+1)}$ now leads to the unitary positive energy representations of loop groups, but for d > 1 the (projective) unitary representations of $U(\mathcal{H}\oplus \mathcal{H})_{(d+1)}$ extend to the full unitary group $U(\mathcal{H}\oplus \mathcal{H})$, so that we do not obtain interesting unitary representations of mapping groups. However, there are natural homomorphisms C(M, K) into the motion group $\mathcal{H} \rtimes \mathop{ \mathrm{O}}\nolimits (\mathcal{H})$ of a real Hilbert space, and this leads to the interesting class of energy representations [GGV80, AH78].

The content of the paper is as follows. In the first two sections we discuss some core ideas and methods from the work of Olshanski and Pickrell. We start in Sect. 1 with the concept of a bounded topological group. These are topological groups G, for which every identity neighborhood U satisfies $G \subseteq U^{m}$ for some $m \in \mathbb{N}$. This boundedness condition permits showing that certain subgroups of G have nonzero fixed points in unitary representations (cf. Proposition 1.6 for a typical result of this kind). We continue in Sect. 2 with Olshanski’s concept of an overgroup. Starting with a symmetric pair (G, K) with Lie algebra $\mathfrak{g} = \mathfrak{k} \oplus \mathfrak{p}$, the overgroup K ^♯ of K is a Lie group with the Lie algebra $\mathfrak{k} + i\mathfrak{p}$. We shall use these overgroups for the pairs $(\mathop{\mathrm{GL}}\nolimits (\mathcal{H}),\mathop{\mathrm{U}}\nolimits (\mathcal{H}))$, where $\mathcal{H}$ is a real, complex or quaternionic Hilbert space.

In Sect. 3 we describe Olshanski’s approach to the classification of the unitary representations of $K:=\mathop{ \mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$. Here the key idea is that any representation of this group is a direct sum of representations π generated by the fixed space V of the subgroup K _n fixing the first n basis vectors. It turns out that this space V carries a ∗-representation (ρ, V ) of the involutive semigroup $C(n,\mathbb{K})$ of contractions on $\mathbb{K}^{n}$ which determines π uniquely by a GNS construction.

Now the main point is to understand which representations of $C(n,\mathbb{K})$ occur in this process, that they are direct sums of irreducible ones and to determine the irreducible representations. To achieve this goal, we deviate from Olshanski’s approach by putting a stronger emphasis on analytic positive definite functions (cf. Appendix A).

This leads to a considerable simplification of the proof avoiding the use of zonal spherical functions and expansions with respect to orthogonal polynomials. Moreover, our technique is rather close to the setting of holomorphic induction developed in [Ne13b]. In particular, we use Theorem A.4 which is a slight generalization of [Ne12, Thm. A.7].

In Sect. 4 we provide a complete proof of Pickrell’s theorem asserting that for a separable Hilbert space $\mathcal{H}$, the groups $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ and $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$ have the same continuous separable unitary representations. Here the key result is that all continuous separable unitary representations of the quotient group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})/\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$ are trivial. We show that this result carries over to the real and quaternionic case by deriving it from the complex case.

This provides a complete picture of the separable representations of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ and the subgroup $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$, but there are many subgroups in between. This is naturally complemented by Pickrell’s result that for the 10 symmetric pairs (G, K) of unitary type for q > 2, all continuous unitary representations of G _(q) extend to continuous representations of G. In Sect. 5 we show that for the pairs (G, K) of noncompact type, all separable unitary representations of G and G _(q), q > 2, are trivial. This is also stated in [Pi90], but the proof is very sketchy. We use an argument based on Howe–Moore theory for the vanishing of matrix coefficients.

FormalPara Notation and Terminology

For the nonnegative half line we write $\mathbb{R}_{+} = [0,\infty [$.

In the following $\mathbb{K}$ always denotes $\mathbb{R}$, $\mathbb{C}$ or the skew field $\mathbb{H}$ of quaternions. We write $\{1,\mathcal{I},\mathcal{J},\mathcal{I}\mathcal{J}\}$ for the canonical basis of $\mathbb{H}$ satisfying the relations

$$\displaystyle{\mathcal{I}^{2} = \mathcal{J}^{2} = -\mathbf{1}\quad \text{ and }\quad \mathcal{I}\mathcal{J} = -\mathcal{J}\mathcal{I}.}$$

For a real Hilbert space $\mathcal{H}$, we write $\mathcal{H}_{\mathbb{C}}$ for its complexification, and for a quaternionic Hilbert space $\mathcal{H}$, we write $\mathcal{H}^{\mathbb{C}}$ for the underlying complex Hilbert space, obtained from the complex structure $\mathcal{I}\in \mathbb{H}$. For a complex Hilbert space we likewise write $\mathcal{H}^{\mathbb{R}}$ for the underlying real Hilbert space.

For the algebra $B(\mathcal{H})$ of bounded operators on the $\mathbb{K}$-Hilbert space $\mathcal{H}$, the ideal of compact operators is denoted $K(\mathcal{H}) = B_{\infty }(\mathcal{H})$, and for 1 ≤ p < ∞, we write

$$\displaystyle{B_{p}(\mathcal{H}):=\{ A \in K(\mathcal{H}): \mathop{\mathrm{tr}}\nolimits ((A^{{\ast}}A)^{p/2}) =\mathop{ \mathrm{tr}}\nolimits (\vert A\vert ^{p}) < \infty \}}$$

for the Schatten ideals. In particular, $B_{2}(\mathcal{H})$ is the space of Hilbert–Schmidt operators and $B_{1}(\mathcal{H})$ the space of trace class operators. Endowed with the operator norm, the groups $\mathop{\mathrm{GL}}\nolimits (\mathcal{H})$ and $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ are Lie groups with the respective Lie algebras

$$\displaystyle{\mathop{\mathfrak{g}\mathfrak{l}}\nolimits (\mathcal{H}) = B(\mathcal{H})\quad \text{ and }\quad \mathfrak{u}(\mathcal{H}):=\{ X \in \mathop{ \mathfrak{g}\mathfrak{l}}\nolimits (\mathcal{H}): X^{{\ast}} = -X\}.}$$

For $1 \leq p \leq \infty$, we obtain Lie groups

$$\displaystyle{\mathop{\mathrm{GL}}\nolimits _{p}(\mathcal{H}):=\mathop{ \mathrm{GL}}\nolimits (\mathcal{H}) \cap (\mathbf{1} + B_{p}(\mathcal{H}))\quad \text{ and }\quad \mathop{\mathrm{U}}\nolimits _{p}(\mathcal{H}):=\mathop{ \mathrm{U}}\nolimits (\mathcal{H}) \cap \mathop{\mathrm{GL}}\nolimits _{p}(\mathcal{H})}$$

with the Lie algebras

$$\displaystyle{\mathop{\mathfrak{g}\mathfrak{l}}\nolimits _{p}(\mathcal{H}):= B_{p}(\mathcal{H})\quad \text{ and }\quad \mathfrak{u}_{p}(\mathcal{H}):= \mathfrak{u}(\mathcal{H}) \cap \mathop{ \mathfrak{g}\mathfrak{l}}\nolimits _{p}(\mathcal{H}).}$$

To emphasize the base field $\mathbb{K}$, we sometimes write $\mathop{\mathrm{U}}\nolimits _{\mathbb{K}}(\mathcal{H})$ for the group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ of $\mathbb{K}$-linear isometries of $\mathcal{H}$. We also write $\mathop{\mathrm{O}}\nolimits (\mathcal{H}) =\mathop{ \mathrm{U}}\nolimits _{\mathbb{R}}(\mathcal{H})$.

If G is a group acting on a set X, then we write X ^G for the subset of G-fixed points.

1 Bounded Groups

In this section we discuss one of Olshanski’s key concepts for the approach to Kirillov’s theorem on the classification of the representations of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ for a separable Hilbert space discussed in Sect. 3. As we shall see below (Lemma 4.1), this method also lies at the heart of Pickrell’s theorem on the separable representations of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$.

Definition 1.1.

We call a topological group G bounded if, for every identity neighborhood $U \subseteq G$, there exists an $m \in \mathbb{N}$ with $G \subseteq U^{m}$.

Note that every locally connected bounded topological group is connected. The group $\mathbb{Q}/\mathbb{Z}$ is bounded but not connected.

Lemma 1.2.

If, for a Banach–Lie group G, there exists a c > 0 with

$$\displaystyle{ G =\exp \{ x \in \mathfrak{g}: \|x\| \leq c\}, }$$

(Ol)

then G is bounded.

Proof.

Let U be an identity neighborhood of G. Since the exponential function $\exp _{G}: \mathfrak{g} \rightarrow G$ is continuous, there exists an r > 0 with expx ∈ U for $\|x\| < r$. Pick $m \in \mathbb{N}$ such that mr > c. For g = expx with $\|x\| \leq c$ we then have $\exp \frac{x} {m} \in U$, and therefore g ∈ U ^m. □ 

Proposition 1.3.

The following groups satisfy  ( Ol ) , hence are bounded:

• (i) The full unitary group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ of an infinite-dimensional complex or quaternionic Hilbert space.

• (ii) The unitary group $\mathop{\mathrm{U}}\nolimits (\mathcal{M})$ of a von Neumann algebra $\mathcal{M}$ .

• (iii) The identity component $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$ for a $\mathbb{K}$ -Hilbert space  $\mathcal{H}$.¹

Proof.

• (i) Case $\mathbb{K} = \mathbb{C}$ : Let $g \in \mathop{\mathrm{U}}\nolimits (\mathcal{H})$ and let P denote the spectral measure on the unit circle $\mathbb{T} \subseteq \mathbb{C}$ with $g =\int _{\mathbb{T}}z\,dP(z)$. We consider the measurable function $L: \mathbb{T} \rightarrow ]-\pi,\pi ]i$ which is the inverse of the function $]-\pi,\pi ]i \rightarrow \mathbb{T},z\mapsto e^{z}$. Then

  $$\displaystyle{ X:=\int _{\mathbb{T}}L(z)\,dP(z) }$$

  (1)

  is a skew-hermitian operator with $\|X\| \leq \pi$ and e ^X = g (cf. [Ru73, Thm. 12.37]).

Case $\mathbb{K} = \mathbb{H}$ : We consider the quaternionic Hilbert space as a complex Hilbert space $\mathcal{H}^{\mathbb{C}}$, endowed with an anticonjugation (=antilinear complex structure) $\mathcal{J}$. Then

$$\displaystyle{\mathop{\mathrm{U}}\nolimits _{\mathbb{H}}(\mathcal{H}) =\{ g \in \mathop{\mathrm{U}}\nolimits (\mathcal{H}^{\mathbb{C}}): \mathcal{J}g\mathcal{J}^{-1} = g\}.}$$

An element $g \in \mathop{\mathrm{U}}\nolimits (\mathcal{H}^{\mathbb{C}})$ is $\mathbb{H}$-linear if and only if the relation $\mathcal{J}P(E)\mathcal{J}^{-1} = P(\overline{E})$ holds for the corresponding spectral measure P on $\mathbb{T}$.

Let $\mathcal{H}_{0}:=\ker (g + \mathbf{1}) = P(\{-1\})\mathcal{H}$ denote the (−1)-eigenspace of g and $\mathcal{H}_{1}:= \mathcal{H}_{0}^{\perp }$. If X is defined by (1) and $X_{1}:= X\vert _{\mathcal{H}_{1}}$, then

$$\displaystyle{ \mathcal{J}X_{1}\mathcal{J}^{-1} =\int _{\mathbb{T}\setminus \{ -1\}} - L(z)\,dP(\overline{z}) =\int _{\mathbb{T}\setminus \{-1\}}L(z)\,dP(z) = X_{1}. }$$

Then $X:=\pi \mathcal{J}\vert _{\mathcal{H}_{0}} \oplus X_{1}$ on $\mathcal{H} = \mathcal{H}_{0} \oplus \mathcal{H}_{1}$ is an element $X \in \mathfrak{u}_{\mathbb{H}}(\mathcal{H})$ with e ^X = g and $\|X\| \leq \pi$. Therefore ( Ol) is satisfied.

• (ii) If $g \in \mathop{\mathrm{U}}\nolimits (\mathcal{M})$, then $P(E) \in \mathcal{M}$ for every measurable subset $E \subseteq \mathbb{T}$, and therefore $X \in \mathcal{M}$. Now (ii) follows as (i) for $\mathbb{K} = \mathbb{C}$.

• (iii) Case $\mathbb{K} = \mathbb{C},\mathbb{H}$ : The operator X from (1) is compact if g −1 is compact. Hence the group $\mathop{\mathrm{U}}\nolimits _{\infty,\mathbb{K}}(\mathcal{H})$ is connected and we can argue as in (i).

Case $\mathbb{K} = \mathbb{R}$ : We consider $\mathop{\mathrm{U}}\nolimits _{\infty,\mathbb{R}}(\mathcal{H}) =\mathop{ \mathrm{O}}\nolimits _{\infty }(\mathcal{H})$ as a subgroup of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H}_{\mathbb{C}})$. Let σ denote the antilinear isometry on $\mathcal{H}_{\mathbb{C}}$ whose fixed point set is $\mathcal{H}$. Then, for $g \in \mathop{\mathrm{U}}\nolimits (\mathcal{H}_{\mathbb{C}})$, the relation σ g σ = g is equivalent to $g \in \mathop{\mathrm{O}}\nolimits (\mathcal{H})$. This is equivalent to the relation $\sigma P(E)\sigma = P(\overline{E})$ for the corresponding spectral measure on $\mathbb{T}$.

Next we recall from [Ne02, Cor. II.15] (see also [dlH72]) that the group $\mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H})$ has two connected components. An element $g \in \mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H})$ for which g −1 is of trace class is contained in the identity component if and only if $\det (g) = 1$. From the normal form of orthogonal compact operators that follows from the spectral measure on $\mathcal{H}_{\mathbb{C}}$, it follows that $\det (g) = (-1)^{\mathop{\mathrm{dim}}\nolimits \mathcal{H}^{-g} }$. Therefore the identity component of $\mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H})$ consists of those elements g for which the (−1)-eigenspace $\mathcal{H}_{0} = \mathcal{H}^{-g}$ is of even dimension. Let $J \in \mathfrak{o}(\mathcal{H}_{0})$ be an orthogonal complex structure. On $\mathcal{H}_{1}:= \mathcal{H}_{0}^{\perp }$ the operator $X_{1}:= X\vert _{\mathcal{H}_{1}}$ satisfies

$$\displaystyle{ \sigma X_{1}\sigma =\int _{\mathbb{T}\setminus \{-1\}} - L(z)\,dP(\overline{z}) =\int _{\mathbb{T}\setminus \{-1\}}L(z)\,dP(z) = X_{1}, }$$

so that $X:=\pi J \oplus X_{1} \in \mathfrak{o}_{\infty }(\mathcal{H})$ satisfies $\|X\| \leq \pi$ and e ^X = g. □ 

Example 1.4.

• (a) In view of Proposition 1.3, it is remarkable that the full orthogonal group $\mathop{\mathrm{O}}\nolimits (\mathcal{H})$ of a real Hilbert space $\mathcal{H}$ does not satisfy ( Ol). Actually its exponential function is not surjective [PW52]. In fact, if g = e ^X for $X \in \mathfrak{o}(\mathcal{H})$, then X commutes with g, hence preserves the (−1)-eigenspace $\mathcal{H}_{0}:=\ker (g + \mathbf{1})$. Therefore $J:= e^{X/2}$ defines a complex structure on $\mathcal{H}_{0}$, showing that $\mathcal{H}_{0}$ is either infinite-dimensional or of even dimension. Therefore no element $g \in \mathop{\mathrm{O}}\nolimits (\mathcal{H})$ for which $\mathop{\mathrm{dim}}\nolimits \mathcal{H}_{0}$ is odd is contained in the image of the exponential function.

• (b) We shall need later that $\mathop{\mathrm{O}}\nolimits (\mathcal{H})$ is connected. This follows from Kuiper’s theorem [Ku65], but one can give a more direct argument based on the preceding discussion. It only remains to show that elements $g \in \mathop{\mathrm{O}}\nolimits (\mathcal{H})$ for which the space $\mathcal{H}_{0}:=\ker (g + \mathbf{1})$ is of finite odd dimension are contained in the identity component. We write $g = g_{-1} \oplus g_{1}$ with $g_{-1} = g\vert _{\mathcal{H}_{0}}$ and $g_{1}:= g\vert _{\mathcal{H}_{0}^{\perp }}$. Then g ₁ lies on a one-parameter group of $\mathop{\mathrm{O}}\nolimits (\mathcal{H}_{0}^{\perp })$, so that g is connected by a continuous arc to $g':= -\mathbf{1}_{\mathcal{H}_{0}} \oplus \mathbf{1}$. This element is connected to $g'':= -\mathbf{1}_{\mathcal{H}_{0}} \oplus -\mathbf{1}_{\mathcal{H}_{0}^{\perp }} = -\mathbf{1}_{\mathcal{H}}$, and this in turn to $\mathbf{1}_{\mathcal{H}}$. Therefore $\mathop{\mathrm{O}}\nolimits (\mathcal{H})$ is connected.

Lemma 1.5 (Olshanski Lemma).

For a group G, a subset $U \subseteq G$ and $m \in \mathbb{N}$ with $G \subseteq U^{m}$ , we put

$$\displaystyle{\eta:= \sqrt{1 - \frac{1} {4(m + 1)^{2}}} \in ]0,1[.}$$

If $(\pi,\mathcal{H})$ is a unitary representation with $\mathcal{H}^{G} =\{ 0\}$ , then, for any non-zero $\xi \in \mathcal{H}$ , there exists u ∈ U with

$$\displaystyle{\frac{1} {2}\|\xi +\pi (u)\xi \| <\eta \|\xi \|.}$$

Proof ([Ol78, Lemma 1.3]).

If $\|\xi -\pi (u)\xi \| \leq \lambda \|\xi \|$ holds for all u ∈ U, then the triangle inequality implies

$$\displaystyle{\|\xi -\pi (g)\xi \| \leq m\lambda \|\xi \|\quad \text{ for }\quad g \in U^{m} = G.}$$

For $\lambda < \frac{1} {m}$ this implies that the closed convex hull of the orbit π(G)ξ does not contain 0, hence contains a nonzero fixed point by the Bruhat–Tits Theorem [La99], applied to the isometric action of G on $\mathcal{H}$. This violates our assumption $\mathcal{H}^{G} =\{ 0\}$. We conclude that there exists a u ∈ U with $\|\xi -\pi (u)\xi \| > \frac{1} {m+1}\|\xi \|$. Thus

$$\displaystyle{2\|\xi \|^{2} - 2\mathop{\mathrm{Re}}\nolimits \langle \xi,\pi (u)\xi \rangle =\|\xi -\pi (u)\xi \|^{2} > \frac{\|\xi \|^{2}} {(m + 1)^{2}},}$$

which in turn gives

$$\displaystyle{\|\xi +\pi (u)\xi \|^{2} = 2\|\xi \|^{2} + 2\mathop{\mathrm{Re}}\nolimits \langle \xi,\pi (u)\xi \rangle <\big (4 - \frac{1} {(m + 1)^{2}}\big)\|\xi \|^{2} =\eta ^{2}\|\xi \|^{2}.\qquad }$$

 □ 

The following proposition is an abstraction of the proof of [Ol78, Lemma 1.4]. It will be used in two situations below, to prove Kirillov’s Lemma 3.6 and in Pickrell’s Lemma 4.1.

Proposition 1.6.

Let G be a bounded topological group and $(G_{n})_{n\in \mathbb{N}}$ be a sequence of subgroups of G. If there exists a basis of 1 -neighborhoods $U \subseteq G$ such that either

• (a)

  (U1):

  $(\exists m \in \mathbb{N})(\forall n)\ G_{n} \subseteq (G_{n} \cap U)^{m}$ , and

  (U2):

  $(\forall N \in \mathbb{N})(G_{N} \cap U)\cdots (G_{1} \cap U) \subseteq U$ ,

  or

• (b)

  (V1):

  $(\exists m \in \mathbb{N})(\forall n)\ G_{n} \subseteq (G_{n} \cap U)^{m}$ , and

  (V2):

  there exists an increasing sequence of subgroups $(G(n))_{n\in \mathbb{N}}$ such that

  • (1) $(\forall \,n \in \mathbb{N})$ the union of $G(m)_{n}:= G(m) \cap G_{n}$ , $m \in \mathbb{N}$ , is dense in G _n .

  • (2) $(G(k_{1})_{1} \cap U)(G(k_{2})_{k_{1}} \cap U)\cdots (G(k_{N})_{k_{N-1}} \cap U) \subseteq U$ for $1 < k_{1} <\ldots < k_{N}$ .

Then there exists an $n \in \mathbb{N}$ with $\mathcal{H}^{G_{n}}\not =\{0\}$ .

Proof.

• (a) We argue by contradiction and assume that $\mathcal{H}^{G_{n}} =\{ 0\}$ for every n. Let $\xi \in \mathcal{H}$ be nonzero and let $U \subseteq G$ be an identity neighborhood with $\|\pi (g)\xi -\xi \| < \frac{1} {2}\|\xi \|$ for g ∈ U such that (U1/2) are satisfied. Let η be as in Lemma 1.5.

Since G ₁ has no nonzero fixed vector, there exists an element $u_{1} \in U \cap G_{1}$ with

$$\displaystyle{\|\frac{1} {2}(\xi +\pi (u_{1})\xi )\| \leq \eta \|\xi \|.}$$

Then $\xi _{1}:= \frac{1} {2}(\xi +\pi (u_{1})\xi )$ satisfies $\|\xi _{1}-\xi \| < \frac{1} {2}\|\xi \|$, so that $\xi _{1}\not =0$. Iterating this procedure, we obtain a sequence of vectors $(\xi _{n})_{n\in \mathbb{N}}$ and elements $u_{n} \in U \cap G_{n}$ with $\xi _{n+1}:= \frac{1} {2}(\xi _{n} +\pi (u_{n+1})\xi _{n})$ and $\|\xi _{n+1}\| \leq \eta \|\xi _{n}\|$.

We consider the probability measures $\mu _{n}:= \frac{1} {2}(\delta _{\mathbf{1}} +\delta _{u_{n}})$ on G and observe that (U2) implies $\mathop{\mathrm{supp}}\nolimits (\mu _{n} {\ast}\cdots {\ast}\mu _{1}) \subseteq U$ for every $n \in \mathbb{N}$. By construction we have π(μ _n ∗⋯ ∗μ ₁)ξ = ξ _n , so that $\|\xi _{n}-\xi \| < \frac{1} {2}\|\xi \|$. On the other hand,

$$\displaystyle{\|\pi (\mu _{n} {\ast}\cdots {\ast}\mu _{1})\xi \| =\|\xi _{n}\| \leq \eta ^{n}\|\xi \| \rightarrow 0,}$$

and this is a contradiction.

• (b) Again, we argue by contradiction and assume that no subgroup G _n has a nonzero fixed vector. Let $\xi \in \mathcal{H}$ be a nonzero vector and U an identity neighborhood with $\|\pi (g)\xi -\xi \| < \frac{1} {2}\|\xi \|$ for g ∈ U such that (V1) is satisfied.

Since G ₁ has no nonzero fixed point in $\mathcal{H}$ and $\bigcup _{n=1}^{\infty }G(n)_{1}$ is dense in G ₁, there exists a $k_{1} \in \mathbb{N}$ and some $u_{1} \in U \cap G(k_{1})_{1}$ with $\|\frac{1} {2}(\xi +\pi (u_{1})\xi )\| <\eta \|\xi \|$ (Lemma 1.5). For $\xi _{1}:= \frac{1} {2}(\xi +\pi (u_{1})\xi )$ our construction then implies that $\|\xi _{1}-\xi \| < \frac{1} {2}\|\xi \|$, so that $\xi _{1}\not =0$. Any $u \in U \cap G_{k_{1}}$ commutes with u ₁, so that we further obtain

$$\displaystyle\begin{array}{rcl} \|\pi (u)\xi _{1} -\xi _{1}\|& =& \frac{1} {2}\|\pi (u)\xi -\xi +\pi (u)\pi (u_{1})\xi -\pi (u_{1})\xi \| {}\\ & <& \frac{1} {2}(\frac{1} {2}\|\xi \| +\|\pi (u_{1})\pi (u)\xi -\pi (u_{1})\xi \|) {}\\ & =& \frac{1} {2}(\frac{1} {2}\|\xi \| +\|\pi (u)\xi -\xi \|) < \frac{1} {2}(\frac{1} {2}\|\xi \| + \frac{1} {2}\|\xi \|) = \frac{1} {2}\|\xi \|. {}\\ \end{array}$$

Iterating this procedure, we obtain a strictly increasing sequence (k _n ) of natural numbers, a sequence (ξ _n ) in $\mathcal{H}$ and $u_{n} \in G(k_{n})_{k_{n-1}} \cap U$ with

$$\displaystyle{\xi _{n+1}:= \frac{1} {2}(\xi _{n} +\pi (u_{n+1})\xi _{n})\quad \text{ and }\quad \|\xi _{n+1}\| <\eta \|\xi _{n}\|.}$$

We consider the probability measures $\mu _{n}:= \frac{1} {2}(\delta _{\mathbf{1}} +\delta _{u_{n}})$ on G. Condition (V2)(2) implies that

$$\displaystyle{\mathop{\mathrm{supp}}\nolimits (\mu _{n} {\ast}\cdots {\ast}\mu _{1}) \subseteq \{\mathbf{1},u_{n}\}\cdots \{\mathbf{1},u_{1}\} \subseteq U\quad \text{ for every }\quad n \in \mathbb{N}.}$$

By construction $\pi (\mu _{n} {\ast}\cdots {\ast}\mu _{1})\xi =\xi _{n}$, so that $\|\xi _{n}-\xi \| < \frac{1} {2}\|\xi \|$. On the other hand,

$$\displaystyle{\|\pi (\mu _{n} {\ast}\cdots {\ast}\mu _{1})\xi \| =\|\xi _{n}\| \leq \eta ^{n}\|\xi \| \rightarrow 0,}$$

and this is a contradiction. □ 

2 Duality and Overgroups

Apart from the fixed point results related to bounded topological groups discussed in the preceding section, another central concept in Olshanski’s approach are “overgroups”. They are closely related to the duality of symmetric spaces.

Definition 2.1.

A symmetric Lie group is a triple (G, K, τ), where τ is an involutive automorphism of the Banach–Lie group G and K is an open subgroup of the Lie subgroup G ^τ of τ-fixed points in G. We write $\mathfrak{g} = \mathfrak{k} \oplus \mathfrak{p} = \mathfrak{g}^{\tau } \oplus \mathfrak{g}^{-\tau }$ for the eigenspace decomposition of $\mathfrak{g}$ with respect to τ and call $\mathfrak{g}^{c}:= \mathfrak{k} \oplus i\mathfrak{p} \subseteq \mathfrak{g}_{\mathbb{C}}$ the dual symmetric Lie algebra.

Definition 2.2.

Suppose that (G, K, τ) is a symmetric Lie group and G ^c a simply connected Lie group with Lie algebra $\mathfrak{g}^{c} = \mathfrak{k} + i\mathfrak{p}$. Then $X + iY \mapsto X - iY$ ($X \in \mathfrak{k}$, $Y \in \mathfrak{p}$), integrates to an involution $\tilde{\tau }^{c}$ of G ^c. Let $q_{K}: \tilde{K}_{0} \rightarrow K_{0}$ denote the universal covering of the identity component K ₀ of K and $\tilde{\iota }_{K}: \tilde{K}_{0} \rightarrow G^{c}$ the homomorphism integrating the inclusion $\mathfrak{k}\hookrightarrow \mathfrak{g}^{c}$. The group $\tilde{\iota }_{K}(\ker q_{K})$ acts trivially on $\mathfrak{g}_{\mathbb{C}}$, hence is central in G ^c. If it is discrete, then we call

$$\displaystyle{(K_{0})^{\sharp }:= G^{c}/\tilde{\iota }_{ K}(\ker q_{K})}$$

the overgroup of K ₀. In this case $\tilde{\iota }_{K}$ factors through a covering map $\iota _{K_{0}}: K_{0} \rightarrow (K_{0})^{\sharp }$ and the involution τ ^c induced by $\tilde{\tau }^{c}$ on $(K_{0})^{\sharp }$ leads to a symmetric Lie group $((K_{0})^{\sharp },\iota _{K_{0}}(K_{0}),\tau ^{c})$.

To extend this construction to the case where K is not connected, we first observe that $K \subseteq G$ acts naturally on the Lie algebra $\mathfrak{g}^{c}$, hence also on the corresponding simply connected group G ^c. This action preserves $\tilde{\iota }_{K}(\ker q_{K})$, hence induces an action on $(K_{0})^{\sharp }$, so that we can form the semidirect product $(K_{0})^{\sharp } \rtimes K$. In this group $N:=\{ (\iota _{K_{0}}(k),k^{-1}): k \in K_{0}\}$ is a closed normal subgroup and we put

$$\displaystyle{K^{\sharp }:= ((K_{ 0})^{\sharp } \rtimes K)/N,\quad \iota _{ K}(k):= (\mathbf{1},k)N \in K^{\sharp }.}$$

The overgroup K ^♯ has the universal property that if a morphism α: K → H of Lie groups extends to a Lie group with Lie algebra $\mathfrak{g}^{c} = \mathfrak{k} + i\mathfrak{p}$, then α factors through $\tilde{\iota }_{K}: K \rightarrow K^{\sharp }$.

Example 2.3.

• (a) If $\mathcal{H}$ is a $\mathbb{K}$-Hilbert space, then the triple $(\mathop{\mathrm{GL}}\nolimits _{\mathbb{K}}(\mathcal{H}),\mathop{\mathrm{U}}\nolimits _{\mathbb{K}}(\mathcal{H}),\tau )$ with $\tau (g) = (g^{{\ast}})^{-1}$ is a symmetric Lie group. For its Lie algebra

  $$\displaystyle{\mathfrak{g} =\mathop{ \mathfrak{g}\mathfrak{l}}\nolimits _{\mathbb{K}}(\mathcal{H}) = \mathfrak{k} \oplus \mathfrak{p} = \mathfrak{u}_{\mathbb{K}}(\mathcal{H}) \oplus \mathop{\mathrm{Herm}}\nolimits _{\mathbb{K}}(\mathcal{H}),}$$

  the corresponding dual symmetric Lie algebra is

  $$\displaystyle{\mathfrak{g}^{c} = \mathfrak{k} + i\mathfrak{p} = \mathfrak{u}_{\mathbb{K}}(\mathcal{H}) \oplus i\mathop{\mathrm{Herm}}\nolimits _{\mathbb{K}}(\mathcal{H}) \subseteq \mathfrak{u}(\mathcal{H}_{\mathbb{C}}).}$$

  More precisely, we have

  $(\mathbb{R})$ :

  $\mathop{\mathfrak{g}\mathfrak{l}}\nolimits _{\mathbb{R}}(\mathcal{H})^{c} = \mathfrak{o}(\mathcal{H}) \oplus i\mathop{\mathrm{Sym}}\nolimits (\mathcal{H})\mathop{\cong}\mathfrak{u}(\mathcal{H}_{\mathbb{C}})$ for $\mathbb{K} = \mathbb{R}$.

  $(\mathbb{C})$ :

  $\mathop{\mathfrak{g}\mathfrak{l}}\nolimits _{\mathbb{C}}(\mathcal{H})^{c} = \mathfrak{u}(\mathcal{H}) \oplus i\mathop{\mathrm{Herm}}\nolimits (\mathcal{H})\mathop{\cong}\mathfrak{u}(\mathcal{H})^{2}$ for $\mathbb{K} = \mathbb{C}$.

  $(\mathbb{H})$ :

  $\mathop{\mathfrak{g}\mathfrak{l}}\nolimits _{\mathbb{H}}(\mathcal{H})^{c} = \mathfrak{u}_{\mathbb{K}}(\mathcal{H}) \oplus \mathcal{I}\mathop{\mathrm{Herm}}\nolimits _{\mathbb{H}}(\mathcal{H})\mathop{\cong}\mathfrak{u}(\mathcal{H}^{\mathbb{C}})$ for $\mathbb{K} = \mathbb{H}$.

  Here the complex case requires additional explanation. Let I denote the given complex structure on $\mathcal{H}$. Then the maps

  $$\displaystyle{\iota _{\pm }: \mathcal{H}\rightarrow \mathcal{H}_{\mathbb{C}},\quad v\mapsto \frac{1} {\sqrt{2}}(v \mp iIv)}$$

  are isometries to complex subspaces $\mathcal{H}_{\mathbb{C}}^{\pm }$ of $\mathcal{H}_{\mathbb{C}}$, where ι ₊ is complex linear and ι ₋ is antilinear. We thus obtain

  $$\displaystyle{\mathcal{H}_{\mathbb{C}} = \mathcal{H}_{\mathbb{C}}^{+} \oplus \mathcal{H}_{\mathbb{C}}^{-}\mathop{\cong}\mathcal{H}\oplus \overline{\mathcal{H}},}$$

  and $\mathcal{H}_{\mathbb{C}}^{\pm }$ are the $\pm i$-eigenspaces of the complex linear extension of I to $\mathcal{H}_{\mathbb{C}}$. In particular, $\mathop{\mathfrak{g}\mathfrak{l}}\nolimits (\mathcal{H})^{c}$ preserves both subspaces $\mathcal{H}_{\mathbb{C}}^{\pm }$. This leads to the isomorphism

  $$\displaystyle{\gamma: \mathop{\mathfrak{g}\mathfrak{l}}\nolimits (\mathcal{H})^{c} \rightarrow \mathfrak{u}(\mathcal{H}_{\mathbb{C}}^{+}) \oplus \mathfrak{u}(\mathcal{H}_{\mathbb{C}}^{-})\mathop{\cong}\mathfrak{u}(\mathcal{H}) \oplus \mathfrak{u}(\overline{\mathcal{H}}),\quad \gamma (X + iY ) = (X + IY,X - IY ).}$$

Lemma 2.4.

For a $\mathbb{K}$ -Hilbert space $\mathcal{H}$ , let $(G,K) = (\mathop{\mathrm{GL}}\nolimits _{\mathbb{K}}(\mathcal{H}),\mathop{\mathrm{U}}\nolimits _{\mathbb{K}}(\mathcal{H})_{0})$ and $n:=\mathop{ \mathrm{dim}}\nolimits \mathcal{H}$ . Then K is connected for $\mathbb{K}\not =\mathbb{R}$ and n = ∞, and

$$\displaystyle{(K_{0})^{\sharp }\mathop{\cong}\left \{\begin{array}{@{}l@{\quad }l@{}} \tilde{\mathrm{U}}(n,\mathbb{C}) \quad &\text{ for }\mathbb{K} = \mathbb{R},n < \infty \\ \tilde{\mathrm{U}}(n,\mathbb{C})^{2}/\varGamma \quad &\text{ for }\mathbb{K} = \mathbb{C},n < \infty,\ \ \varGamma:=\{ (z,z): z \in \pi _{1}(\mathop{\mathrm{U}}\nolimits (n,\mathbb{C}))\}. \\ \tilde{\mathrm{U}}(2n,\mathbb{C}) \quad &\text{ for }\mathbb{K} = \mathbb{H},n < \infty, \\ \mathop{\mathrm{U}}\nolimits (\mathcal{H}_{\mathbb{C}}) \quad &\text{ for }\mathbb{K} = \mathbb{R} \\ \mathop{\mathrm{U}}\nolimits (\mathcal{H}) \oplus \mathop{\mathrm{U}}\nolimits (\overline{\mathcal{H}})\quad &\text{ for }\mathbb{K} = \mathbb{C} \\ \mathop{\mathrm{U}}\nolimits (\mathcal{H}^{\mathbb{C}}) \quad &\text{ for }\mathbb{K} = \mathbb{H}. \end{array} \right.}$$

Here $\mathcal{H}^{\mathbb{C}}$ is the complex Hilbert space underlying a quaternionic Hilbert space $\mathcal{H}$ . For $\mathbb{K} = \mathbb{R},\mathbb{H}$ , the map ι _K is the canonical inclusion, and ι _K (k) = (k,k) for $\mathbb{K} = \mathbb{C}$ .

Proof.

First we consider the case where n < ∞. Recall that

$$\displaystyle\begin{array}{rcl} \tilde{\mathrm{U}}(n,\mathbb{C})\mathop{\cong}\mathop{\mathrm{SU}}\nolimits (n,\mathbb{C}) \rtimes \mathbb{R},\quad \mathop{\mathrm{O}}\nolimits (n,\mathbb{R})_{0} =\mathop{ \mathrm{SO}}\nolimits (n,\mathbb{R}) \subseteq \mathop{\mathrm{SU}}\nolimits (n,\mathbb{C}),\quad \mathop{\mathrm{U}}\nolimits (n,\mathbb{H}) \subseteq \mathop{\mathrm{SU}}\nolimits (2n,\mathbb{C}).& & {}\\ \end{array}$$

For $\mathbb{K} = \mathbb{R}$, this implies that $K =\mathop{ \mathrm{O}}\nolimits (n,\mathbb{R})_{0}$ embeds into $\tilde{\mathrm{U}}(n,\mathbb{C})$, so that $K^{\sharp }\mathop{\cong}\tilde{\mathrm{U}}(n,\mathbb{C})$. For $\mathbb{K} = \mathbb{H}$, we see that $K =\mathop{ \mathrm{U}}\nolimits (n,\mathbb{H})$ embeds into $\tilde{\mathrm{U}}(2n,\mathbb{C})$, which leads to $\tilde{K}\mathop{\cong}\tilde{\mathrm{U}}(2n,\mathbb{C})$.

For $\mathbb{K} = \mathbb{C}$, we have the natural inclusion

$$\displaystyle{i_{K}: K =\mathop{ \mathrm{U}}\nolimits (n,\mathbb{C}) \rightarrow \mathop{\mathrm{U}}\nolimits (n,\mathbb{C}) \times \mathop{\mathrm{U}}\nolimits (n,\mathbb{C})\mathop{\cong}\mathop{\mathrm{U}}\nolimits (\mathbb{C}^{n}) \times \mathop{\mathrm{U}}\nolimits (\overline{\mathbb{C}^{n}}),\quad g\mapsto (g,g).}$$

To determine K ^♯, we note that the image of

$$\displaystyle{\pi _{1}(i_{K}): \mathbb{Z}\mathop{\cong}\pi _{1}(\mathop{\mathrm{U}}\nolimits (n,\mathbb{C})) \rightarrow \mathbb{Z}^{2}\mathop{\cong}\pi _{ 1}(\mathop{\mathrm{U}}\nolimits (n,\mathbb{C})^{2}),\quad m\mapsto (m,m)}$$

is Γ. Therefore $K^{\sharp }\mathop{\cong}\tilde{\mathrm{U}}(n,\mathbb{C})^{2}/\varGamma$.

If n = ∞, then $K =\mathop{ \mathrm{U}}\nolimits _{\mathbb{K}}(\mathcal{H})$ is a simply connected Lie group with Lie algebra $\mathfrak{k}$ [Ku65], so that K ^♯ is the simply connected Lie group with Lie algebra $\mathfrak{k}^{\sharp }$ and we have a natural morphism $\iota _{K}: K \rightarrow K^{\sharp }$ integrating the inclusion $\mathfrak{k}\hookrightarrow \mathfrak{k}^{\sharp }$. □ 

3 The Unitary Representations of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$

In this section we completely describe the representations of the Banach–Lie groups $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ for an infinite-dimensional real, complex or quaternionic Hilbert space. In particular, we show that all continuous unitary representations are direct sums of irreducible ones and classify the irreducible ones (Theorem 3.17). Our approach is based on Olshanski’s treatment in [Ol78]. We also take some short cuts that simplify the proof and put a stronger emphasis on analytic positive definite functions. This has the nice side effect that we also obtain these results for inseparable Hilbert spaces (Theorem 3.21).

3.1 Tameness as a Continuity Condition

We start with a brief discussion of Olshanski’s concept of a tame representation that links representations of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ to representations of the direct limit group $\mathop{\mathrm{U}}\nolimits (\infty,\mathbb{K})$.

Let K be a group and $(K_{j})_{j\in J}$ a non-empty family of subgroups satisfying the following conditions

(S1):

It is a filter basis, i.e., for j, m ∈ J, there exists an $\ell\in J$ with $K_{\ell} \subseteq K_{j} \cap K_{m}$.

(S2):

$\bigcap _{j\in J}K_{j} =\{ \mathbf{1}\}$.

(S3):

For each g ∈ K and j ∈ J there exists an m ∈ J with $gK_{m}g^{-1} \subseteq K_{j}$.

Then there exists a unique Hausdorff group topology τ on K for which $(K_{j})_{j\in J}$ is a basis of 1-neighborhoods [Bou98, Ch. 4]. We call τ the topology defined by (K _j ) _j∈J.

Definition 3.1.

We call a unitary representation $(\pi,\mathcal{H})$ of K tame if the space

$$\displaystyle{\mathcal{H}^{\mathcal{T}}:=\sum _{ j\in J}\mathcal{H}^{K_{j} } =\bigcup _{j\in J}\mathcal{H}^{K_{j} },}$$

is dense in $\mathcal{H}$. Note that, for $K_{j} \subseteq K_{k} \cap K_{\ell}$, we have $\mathcal{H}^{K_{j}} \supseteq \mathcal{H}^{K_{k}} + \mathcal{H}^{K_{\ell}},$ so that $\mathcal{H}^{\mathcal{T}}$ is a directed union of the closed subspaces $\mathcal{H}^{K_{j}}$.

Lemma 3.2.

A unitary representation of K is tame if and only if it is continuous with respect to the group topology defined by the filter basis (K _j ) _j∈J .

Proof.

If $(\pi,\mathcal{H})$ is a tame representation, then $\mathcal{H}^{\mathcal{T}}$ obviously consists of continuous vectors for K since, for each $v \in \mathcal{H}^{\mathcal{T}}$, the stabilizer is open. Hence the set of continuous vectors is dense, and therefore π is continuous.

If, conversely, $(\pi,\mathcal{H})$ is continuous and $v \in \mathcal{H}$, then the orbit map $K \rightarrow \mathcal{H},g\mapsto gv$ is continuous. Let $B_{\varepsilon }$ denote the closed $\varepsilon$-ball in $\mathcal{H}$. Then there exists a j ∈ J with $\pi (K_{j})v \subseteq v + B_{\varepsilon }$. Then $C:= \overline{\mathop{\mathrm{conv}}\nolimits (\pi (K_{j})v)}$ is a closed convex invariant subset of $v + B_{\varepsilon }$, hence contains a K _j -fixed point by the Bruhat–Tits Theorem [La99]. This proves that $\mathcal{H}^{K_{j}}$ intersects $v + B_{\varepsilon }$, hence that $\mathcal{H}^{\mathcal{T}}$ is dense in $\mathcal{H}$. □ 

Remark 3.3.

• (a) For a unitary representation $(\pi,\mathcal{H})$ of a topological group, the subspace $\mathcal{H}^{c}$ of continuous vectors is closed and invariant. The representation π is continuous if and only if $\mathcal{H}^{c} = \mathcal{H}$.

• (b) For a unitary representation $(\pi,\mathcal{H})$ of K, by Lemma 3.2, the space of continuous vectors coincides with $\overline{\mathcal{H}^{\mathcal{T}}}$. In particular, it is K-invariant.

• (c) If the representation $(\pi,\mathcal{H})$ of K is irreducible, then it is tame if and only if $\mathcal{H}^{\mathcal{T}}\not =\{0\}$.

• (d) If the representation $(\pi,\mathcal{H})$ of K is such that, for some n, the subspace $\mathcal{H}^{K_{n}}$ is cyclic, then it is tame.

Definition 3.4.

Assume that the group K is the union of an increasing sequence of subgroups $(K(n))_{n\in \mathbb{N}}$. We say that the subgroups K(n) are well-complemented by the decreasing sequence $(K_{n})_{n\in \mathbb{N}}$ of subgroups of K if K _n commutes with K(n) for every n and $\bigcap _{n\in \mathbb{N}}K_{n} =\{ \mathbf{1}\}$. For k ∈ K and $n \in \mathbb{N}$, we then find an m > n with k ∈ K(m). Then $kK_{m}k^{-1} = K_{m} \subseteq K_{n},$ so that (S1-3) are satisfied and the groups $(K_{n})_{n\in \mathbb{N}}$ define a group topology on K.

Example 3.5.

• (a) If $K = \oplus _{n=1}^{\infty }F_{n}$ is a direct sum of subgroups $(F_{n})_{n\in \mathbb{N}}$, then the subgroups $K(n):= F_{1} \times \cdots \times F_{n}$ are well-complemented by the subgroups $K_{n}:= \oplus _{m>n}F_{m}$.

• (b) If $K =\mathop{ \mathrm{U}}\nolimits (\infty,\mathbb{K})_{0} =\bigcup _{ n=1}^{\infty }\mathop{\mathrm{U}}\nolimits (n,\mathbb{K})_{0}$ is the canonical direct limit of the compact groups $\mathop{\mathrm{U}}\nolimits (n,\mathbb{K})_{0}$, then the subgroups $K(n):=\mathop{ \mathrm{U}}\nolimits (n,\mathbb{K})_{0}$ are well-complemented by the subgroups

  $$\displaystyle{K_{n}:=\{ g \in K: (\forall j \leq n)\ ge_{j} = e_{j}\}.}$$

3.2 Tame Representations of $\mathop{\mathrm{U}}\nolimits (\infty,\mathbb{K})$

Let $\mathcal{H}$ be an infinite-dimensional separable Hilbert space over $\mathbb{K} \in \{\mathbb{R},\mathbb{C},\mathbb{H}\}$ and $(e_{j})_{j\in \mathbb{N}}$ an orthonormal basis of $\mathcal{H}$. Accordingly, we obtain a natural dense embedding $\mathop{\mathrm{U}}\nolimits (\infty,\mathbb{K})\hookrightarrow \mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$, so that every continuous unitary representation of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ is uniquely determined by its restriction to the direct limit group $\mathop{\mathrm{U}}\nolimits (\infty,\mathbb{K})_{0}$. Olshanski’s approach to the classification is based on an intrinsic characterization of those representations of the direct limit group $\mathop{\mathrm{U}}\nolimits (\infty,\mathbb{K})_{0}$ that extend to $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$. It turns out that these are precisely the tame representations (Theorem 3.20). This is complemented by the discrete decomposition and the classification of the irreducible ones (Theorem 3.17).

In the following we write $K:=\mathop{ \mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ for the identity component of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$ (which is connected for $\mathbb{K} = \mathbb{C},\mathbb{H}$, but not for $\mathbb{K} = \mathbb{R}$), and $K(n):=\mathop{ \mathrm{U}}\nolimits (n,\mathbb{K})_{0}\mathop{\cong}\mathop{ \mathrm{U}}\nolimits (\mathcal{H}(n))_{0}$ for $n \in \mathbb{N}$, where $\mathcal{H}(n) =\mathop{ \mathrm{span}}\nolimits \{e_{1},\ldots,e_{n}\}$. For $n \in \mathbb{N}$, the stabilizer of e ₁, …, e _n in K is denoted K _n , and we likewise write $K(m)_{n}:= K(m) \cap K_{n}$. We also write $K(\infty ):=\mathop{ \mathrm{U}}\nolimits (\infty,\mathbb{K})_{0}\mathop{\cong}\lim _{\longrightarrow }\ \mathop{ \mathrm{U}}\nolimits (n,\mathbb{K})_{0}$ for the direct limit of the groups $\mathop{\mathrm{U}}\nolimits (n,\mathbb{K})_{0}$.

We now turn to the classification of the continuous unitary representations of K. We start with an application of Proposition 1.6.

Lemma 3.6 (Kirillov’s Lemma).

Let $(\pi,\mathcal{H}_{\pi })$ be a continuous unitary representation of the Banach–Lie group $K =\mathop{ \mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ . If $\mathcal{H}_{\pi }\not =\{0\}$ , then there exists an $n \in \mathbb{N}$ , such that the stabilizer K _n of $e_{1},\ldots,e_{n}$ has a nonzero fixed point.

Proof.

We apply Proposition 1.6(b) with G: = K, $G_{n}:= K_{n}$ and G(n): = K(n). Then $U_{\varepsilon }:=\{ g \in K: \|g -\mathbf{1}\| <\varepsilon \}$ provides the required basis of 1-neighborhoods in G (Proposition 1.3(iii)). Condition (V1) follows from Proposition 1.3(iii), (V2)(1) is clear, and (V2)(2) follows from the fact that, for $1 < k_{1} <\ldots < k_{N}$, elements $u_{j} \in K(k_{j})_{k_{j-1}}$ act on pairwise orthogonal subspaces. □ 

Proposition 3.7.

Any continuous unitary representation $(\pi,\mathcal{H})$ of $K =\mathop{ \mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ restricts to a tame representation of the subgroup $K(\infty ) =\mathop{ \mathrm{U}}\nolimits (\infty,\mathbb{K})_{0}$ .

Proof.

Let $\mathcal{H}_{0} \subseteq \mathcal{H}$ denote the maximal subspace on which the representation of K(∞) is tame, i.e., the space of continuous vectors for the topology defined by the subgroups K(∞) _n (Remark 3.3). Lemma 3.6 implies that $\mathcal{H}_{0}\not =\{0\}$. If $\mathcal{H}_{0}\not =\mathcal{H}$, then Lemma 3.6 implies the existence of nonzero continuous vectors in $\mathcal{H}_{0}^{\perp }$, which is a contradiction. □ 

Example 3.8.

The preceding proposition does not extend to the nonconnected group $\mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H})$ which has 2-connected components. The corresponding homomorphism

$$\displaystyle{D: \mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H}) \rightarrow \{\pm 1\}}$$

is nontrivial on all subgroups $\mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H})_{n}$, $n \in \mathbb{N}$.

Lemma 3.9.

Let $\mathcal{H}$ be a $\mathbb{K}$ -Hilbert space and $\mathcal{F}\subseteq \mathcal{H}$ be a finite-dimensional subspace with $2\mathop{\mathrm{dim}}\nolimits \mathcal{F} <\mathop{ \mathrm{dim}}\nolimits \mathcal{H}$.²

We write $P_{\mathcal{F}}: \mathcal{H}\rightarrow \mathcal{F}$ for the orthogonal projection and

$$\displaystyle{C(\mathcal{F}):=\{ A \in B(\mathcal{F}): \|A\| \leq 1\}}$$

for the semigroup of contractions on $\mathcal{F}$ . Then the map

$$\displaystyle{\theta: K =\mathop{ \mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0} \rightarrow C(\mathcal{F}),\quad \theta (g) = P_{\mathcal{F}}gP_{\mathcal{F}}^{{\ast}}}$$

is continuous, surjective and open. Its fibers are the double cosets of the pointwise stabilizer $K_{\mathcal{F}}$ of  $\mathcal{F}$ .

In particular, we obtain for $\mathcal{F} =\mathop{ \mathrm{span}}\nolimits \{e_{1},\ldots,e_{n}\}$ a map

$$\displaystyle{\theta: K \rightarrow C(n,\mathbb{K}):=\{ X \in M(n,\mathbb{K}): \|X\| \leq 1\},\quad \theta (k)_{ij}:=\langle ke_{j},e_{i}\rangle,}$$

which is continuous, surjective and open, and whose fibers are the double cosets $K_{n}kK_{n}$ for k ∈ K.

Proof.

• (i) Surjectivity: For $C \in C(\mathcal{F})$, the operator

  $$\displaystyle{U_{C}:= \left (\begin{array}{*{10}c} C &\sqrt{\mathbf{1} - CC^{{\ast}}} \\ -\sqrt{\mathbf{1} - C^{{\ast} } C}& C^{{\ast}}\end{array} \right ) \in B(\mathcal{F}\oplus \mathcal{F})}$$

  is unitary. In view of $2\mathop{\mathrm{dim}}\nolimits \mathcal{F}\subseteq \mathcal{H}$, we have an isometric embedding $\mathcal{F}\oplus \mathcal{F}\hookrightarrow \mathcal{H}$, and each unitary operator on $\mathcal{F}\oplus \mathcal{F}$ extends to $\mathcal{H}$ by the identity on the orthogonal complement. To see that the resulting operator in contained in K, it remains to see that $\det U_{C} = 1$ if $\mathbb{K} = \mathbb{R}$. To verify this claim, we first observe that for $U_{1},U_{2} \in \mathop{\mathrm{U}}\nolimits _{n}(\mathbb{K})$, we have

  $$\displaystyle{U_{U_{1}CU_{2}} = \left (\begin{array}{*{10}c} U_{1} & 0 \\ 0 &U_{2}^{{\ast}}\end{array} \right )U_{C}\left (\begin{array}{*{10}c} U_{2} & 0 \\ 0 &U_{1}^{{\ast}}\end{array} \right ),}$$

  which implies in particular that $\det U_{C} =\det U_{U_{1}CU_{2}}$. We may therefore assume that C is diagonal, and in this case the assertion follows from the trivial case where $\mathop{\mathrm{dim}}\nolimits \mathcal{F} = 1$. This implies that θ is surjective.

• (ii) θ separates the double cosets of $K_{\mathcal{F}}$: We may w.l.o.g. assume that e ₁, …, e _n span $\mathcal{F}$. First we observe that for m < 2n, the subgroup K _m acts transitively on spheres in $\mathcal{H}(m)^{\perp }$.³

  Suppose that θ(k) = θ(k′), i.e., that the first n components of the vectors ke _j and k′e _j , j = 1, …, n, coincide. Let $P: \mathcal{H}\rightarrow \mathcal{F}^{\perp }$ denote the orthogonal projection. Then $\|Pke_{1}\| =\| Pk'e_{1}\|$, so that the argument in the preceding paragraph shows that there exists a k ₁ ∈ K _n with $k_{1}Pke_{1} = Pk'e_{1}$. This implies that $k_{1}ke_{1} = k'e_{1}$. Replacing k by k ₁ k, we may now assume that $ke_{1} = k'e_{1}$. Then $\|Pke_{2}\| =\| Pk'e_{2}\|$ and the scalar products of Pke ₂ and Pk′e ₂ with Pke ₁ coincide. We therefore find an element k ₂ ∈ K _n fixing Pke ₁, hence also ke ₁, and satisfying $k_{2}Pke_{2} = Pk'e_{2}$, i.e., $k_{2}ke_{2} = k'e_{2}$. Inductively, we thus obtain $k_{1},\ldots,k_{n} \in K_{n}$ with $k_{n}\cdots k_{1}ke_{j} = k'e_{j}$ for j = 1, …, n, and this implies that $k' \in k_{n}\cdots k_{1}kK_{n} \subseteq K_{n}kK_{n}$.

• (iii) It is clear that θ is continuous. To see that it is open, let O ⊆ K be an open subset. Then θ(O) = θ(K _n OK _n ), so that we may w.l.o.g. assume that O = K _n OK _n . From (i) and (ii) it follows that every K _n -double coset intersects K(2n + 1), so that $\theta (O) =\theta (O \cap K(2n + 1))$. Therefore it is enough to observe that the restriction of θ to K(2n + 1) is open, which follows from the compactness of K(2n + 1) and the fact that $\theta \vert _{K(2n+1)}: K(2n + 1) \rightarrow C(n,\mathbb{K})$ is a quotient map. □ 

For a continuous unitary representation $(\pi,\mathcal{H}_{\pi })$ of K, let $V:= \mathcal{H}_{\pi }^{K_{n}}$ denote the subspace of K _n -fixed vectors, $P: \mathcal{H}_{\pi }\rightarrow V$ be the orthogonal projection and π _V (g): = P ^∗ π(g)P. Then π _V is a B(V )-valued continuous positive definite function and Lemma 3.9 implies that we obtain a well-defined continuous map

$$\displaystyle{\rho: C(n,\mathbb{K}) \rightarrow B(V ),\quad \rho (\theta (k)):=\pi _{V }(k)\quad \text{ for }\quad k \in K.}$$

The operator adjoint ∗ turns $C(n,\mathbb{K})$ into an involutive semigroup, and we obviously have $\pi _{V }(k)^{{\ast}} =\pi _{V }(k^{{\ast}})$.

Olshanski’s proof of the following lemma is based on the fact that the projection of the invariant probability measure on $\mathbb{S}^{n}$ to an axis for n → ∞ to the Dirac measure in 0.

Lemma 3.10 ( [Ol78, Lemma 1.7]).

The map ρ is a continuous ∗-representation of the involutive semigroup $C(n,\mathbb{K})$ by contractions satisfying ρ( 1 ) = 1 .

Using Zorn’s Lemma, we conclude that π is a direct sum of subrepresentations for which the subspace of K _n -fixed vectors is cyclic for some $n \in \mathbb{N}$. We may therefore assume that $V = (\mathcal{H}_{\pi })^{K_{n}}$ is cyclic in $\mathcal{H}_{\pi }$. Then the representation π is equivalent to the GNS-representation of K, defined by the positive definite function π _V (Remark A.3). Since the subspace $V = (\mathcal{H}_{\pi })^{K_{n}}$ is obviously invariant under the commutant π(K)′ of π(K), the cyclicity of V implies that we have an injective map

$$\displaystyle{\pi (K)' \rightarrow \rho (C(n,\mathbb{K}))' \subseteq B(V )}$$

which actually is an isomorphism because $\pi _{V }(K) =\rho (C(n,\mathbb{K}))$ is a semigroup (Proposition A.6).

Therefore the structure of π is completely encoded in the representation ρ of the semigroup $C(n,\mathbb{K})$. We therefore have to understand the ∗-representations (ρ, V ) of $C(n,\mathbb{K})$ for which the B(V )-valued function ρ ∘θ: K → B(V ) is positive definite.

Definition 3.11.

We call a ∗-representation (ρ, V ) of $C(n,\mathbb{K})$ θ-positive if the corresponding function ρ ∘θ: K → B(V ) is positive definite.

If ρ is θ-positive, then we obtain a continuous unitary GNS-representation $(\pi _{\rho },\mathcal{H}_{\rho })$ of K containing a K-cyclic subspace V such that the orthogonal projection $P: \mathcal{H}_{\rho }\rightarrow V$ satisfies $P\pi _{\rho }(g)P^{{\ast}} =\rho (\theta (g))$ for g ∈ K (cf. Remark A.3). The following lemma shows that we can recover V as the space of K _n -fixed vectors in $\mathcal{H}_{\rho }$.

Lemma 3.12.

$(\mathcal{H}_{\rho })^{K_{n}} = V$ .

Proof.

Since ρ ∘θ is K _n -biinvariant, the subspace V consists of K _n -fixed vectors because K acts in the corresponding subspace $\mathcal{H}_{\rho \circ \theta }\subseteq V ^{K}$ by right translations (cf. Remark A.3). Let $W:= (\mathcal{H}_{\rho })^{K_{n}}$ and $Q: \mathcal{H}_{\rho }\rightarrow W$ denote the corresponding orthogonal projection. Then $\nu (\theta (g)):= Q\pi _{\rho }(g)Q^{{\ast}}$ defines a contraction representation (ν, W) of $C(n,\mathbb{K})$ (Lemma 3.10) and, for $s \in C(n,\mathbb{K})$, we have ρ(s) = P ν(s)P ^∗.

In $\mathcal{H}$ the subspace V is K-cyclic. Therefore the subspaces Q π(g)V, g ∈ K, span W. In view of Q π(g)V = ν(θ(g))V, this means that V ⊆ W is cyclic for $C(n,\mathbb{K})$. Now Remark A.9 implies that V = W. □ 

We subsume the results of this subsection in the following proposition.

Proposition 3.13.

Let (ρ,V ) be a continuous θ-positive ∗-representation of $C(n,\mathbb{K})$ by contractions and $\varphi:=\rho \circ \theta$ . Then the corresponding GNS-representation $(\pi _{\varphi },\mathcal{H}_{\varphi })$ of K is continuous with cyclic subspace $V \mathop{\cong}(\mathcal{H}_{\varphi })^{K_{n}}$ and $(\pi _{\varphi })_{V } =\varphi =\rho \circ \theta$ . This establishes a one-to-one correspondence of θ-positive continuous ∗-representation of $C(n,\mathbb{K})$ and continuous unitary representations $(\pi,\mathcal{H}_{\pi })$ of K generated by the subspace $(\mathcal{H}_{\pi })^{K_{n}}$ of K _n -fixed vectors. This correspondence preserves direct sums of representations.

3.3 θ-Positive Representations of $C(n,\mathbb{K})$

For $\mathbb{K} = \mathbb{R},\mathbb{H}$, let $Z:=]0,1]\mathbf{1} \subseteq C(n,\mathbb{K})$ be the central subsemigroup of real multiples of 1. Then the continuous bounded characters of Z are of the form χ _s (r): = r ^s, s ≥ 0. Any continuous ∗-representation (π, V ) of Z by contractions determines a spectral measure P on $\hat{Z}:= \mathbb{R}_{+}$ satisfying $\pi (r\mathbf{1}) =\int _{ 0}^{\infty }r^{s}\,dP(s)$ (cf. [BCR84], [Ne00, VI.2]).

For $\mathbb{K} = \mathbb{C}$, the subsemigroup $Z:=\{ z \in \mathbb{C}^{\times }\mathbf{1}: \vert z\vert \leq 1\}\mathop{\cong}\,\,]0,1] \times \mathbb{T}$ is also central in $C(n,\mathbb{C})$. Its continuous bounded characters are of the form $\chi _{s,n}(re^{it}):= r^{s}e^{int}$, $s \geq 0,n \in \mathbb{Z}$. Accordingly, continuous ∗-representation of Z by contractions correspond to spectral measures on $\hat{Z}:= \mathbb{R}_{+} \times \mathbb{Z}$.

Let (ρ, V ) be a continuous (with respect to the weak operator topology on B(V )) ∗-representation of $C(n,\mathbb{K})$ by contractions. Since the spectral projections for the restriction $\rho _{Z}:=\rho \vert _{Z}$ lie in the commutant of $\rho (C(n,\mathbb{K}))$, the representation ρ is a direct sum of subrepresentations for which the support of the spectral measure of ρ _Z is a compact subset of $\hat{Z}$. We call these representations centrally bounded. Then the operators ρ(r 1) are invertible for r > 0, and this implies that

$$\displaystyle{\hat{\rho }(rM):=\rho (r^{-1}\mathbf{1})^{-1}\rho (M)\quad \text{ for }\quad r > 1,M \in C(n,\mathbb{K}),}$$

yields a well-defined extension $\hat{\rho }$ of ρ to a continuous ∗-representation of the multiplicative ∗-semigroup $(M(n,\mathbb{K}),{\ast})$ on V. For a more detailed analysis of the decomposition theory, we may therefore restrict our attention to centrally bounded representations. Decomposing further as a direct sum of cyclic representations, it even suffices to consider separable centrally bounded representations.

The following proposition contains the key new points compared with Olshanski’s approach in [Ol78]. Note that it is very close to the type of reasoning used in [JN13] for the classification of the bounded unitary representations of $\mathop{\mathrm{SU}}\nolimits _{2}(\mathcal{A})$.

Proposition 3.14.

For every centrally bounded contraction representation (ρ,V ) of $C(n,\mathbb{K})$ , the following assertions hold:

• (i) The restriction of $\hat{\rho }$ to $\mathop{\mathrm{GL}}\nolimits (n,\mathbb{K})$ is a norm-continuous representation whose differential $\mathtt{d}\hat{\rho }: \mathop{\mathfrak{g}\mathfrak{l}}\nolimits (n,\mathbb{K}) \rightarrow B(V )$ is a representation of the Lie algebra $\mathop{\mathfrak{g}\mathfrak{l}}\nolimits (n,\mathbb{K})$ by bounded operators.

• (ii) If ρ is θ-positive, then $\hat{\rho }$ is real analytic on $M(n,\mathbb{K})$ and extends to a holomorphic semigroup representation of the complexification

  $$\displaystyle{M(n,\mathbb{K})_{\mathbb{C}}\mathop{\cong}\left \{\begin{array}{@{}l@{\quad }l@{}} M(n,\mathbb{C}) \quad &\text{ for }\mathbb{K} = \mathbb{R} \\ M(n,\mathbb{C}) \oplus M(n,\mathbb{C})\mathop{\cong}B(\mathbb{C}^{n}) \oplus B(\overline{\mathbb{C}^{n}})\quad &\text{ for }\mathbb{K} = \mathbb{C} \\ M(n,M(2,\mathbb{C}))\mathop{\cong}M(2n,\mathbb{C}) \quad &\text{ for }\mathbb{K} = \mathbb{H}.\\ \quad \end{array} \right.}$$

Proof.

• (i) We have already seen that $\hat{\rho }: M(n,\mathbb{K}) \rightarrow B(V )$ is locally bounded and continuous. Hence it restricts to a locally bounded continuous representation of the involutive Lie group $(\mathop{\mathrm{GL}}\nolimits (n,\mathbb{K}),{\ast})$. Integrating this representation to the convolution algebra $C_{c}^{\infty }(\mathop{\mathrm{GL}}\nolimits (n,\mathbb{K}))$, we see that the subspace V ^∞ of smooth vectors is dense. For the corresponding derived representation

  $$\displaystyle{\mathtt{d}\hat{\rho }: \mathop{\mathfrak{g}\mathfrak{l}}\nolimits (n,\mathbb{K}) \rightarrow \mathop{\mathrm{End}}\nolimits (V ^{\infty })}$$

  our construction immediately implies that the operator $\mathtt{d}\hat{\rho }(\mathbf{1})$ is bounded and $\mathtt{d}\hat{\rho }(X) \geq 0$ for X = X ^∗ ≥ 0 because we started with a contraction representation of $C(n,\mathbb{K})$. From $X \leq \| X\|\mathbf{1}$ we also derive $\mathtt{d}\hat{\rho }(X) \leq \| X\|\mathtt{d}\hat{\rho }(\mathbf{1})$, so that d ρ(X) is bounded. As $\mathfrak{u}(n,\mathbb{K}) \subseteq \mathfrak{z}(\mathop{\mathfrak{g}\mathfrak{l}}\nolimits (n,\mathbb{K})) + [\mathop{\mathrm{Herm}}\nolimits (n,\mathbb{K}),\mathop{\mathrm{Herm}}\nolimits (n,\mathbb{K})]$, we conclude that $\mathtt{d}\hat{\rho }$ is a ∗-representation by bounded operators on the Hilbert space V.

For $X \in \mathop{ \mathfrak{g}\mathfrak{l}}\nolimits (n,\mathbb{K})$, we then have the relation

$$\displaystyle{\hat{\rho }(\exp X) = e^{\mathtt{d}\hat{\rho }(X)}\quad \text{ for }\quad X \in \mathop{ \mathfrak{g}\mathfrak{l}}\nolimits (n,\mathbb{K}).}$$

This implies that $\hat{\rho }: \mathop{\mathrm{GL}}\nolimits (n,\mathbb{K}) \rightarrow \mathop{\mathrm{GL}}\nolimits (V )$ is norm-continuous.

• (ii) Now we assume that $\varphi:=\rho \circ \theta: K \rightarrow B(V )$ is positive definite. Since θ(1) = 1, there exists an open 1-neighborhood U ⊆ K with $\theta (U) \subseteq \mathop{\mathrm{GL}}\nolimits (n,\mathbb{K})$. For k ∈ U we then have $\varphi (k) =\hat{\rho } (\theta (k))$, and since the representation $\hat{\rho }$ of $\mathop{\mathrm{GL}}\nolimits (n,\mathbb{K})$ is norm continuous, hence analytic, $\varphi$ is analytic on U. Now Theorem A.5 implies that $\varphi$ is analytic.

Let $\varOmega:=\{ C \in C(n,\mathbb{K}): \|C\| < \mathbf{1}\}$ denote the interior of $C(n,\mathbb{K})$. On this domain we have an analytic cross section of θ, given by

$$\displaystyle{\sigma (C):= \left (\begin{array}{*{10}c} C &\sqrt{\mathbf{1} - CC^{{\ast}}}& 0 \\ -\sqrt{\mathbf{1} - C^{{\ast} } C}& C^{{\ast}} & 0 \\ 0 & 0 &\mathbf{1} \end{array} \right ).}$$

Now $\varphi (\sigma (C)) =\rho (\theta (\sigma (C))) =\rho (C)$ for C ∈ Ω implies that $\rho \vert _{\varOmega }$ is analytic. From $\hat{\rho }(rC) =\hat{\rho } (r)\rho (C)$ for r > 0 it now follows that $\hat{\rho }: M(n,\mathbb{K}) \rightarrow B(V )$ is analytic.

It remains to show that $\hat{\rho }$ extends to a holomorphic map on $M(n,\mathbb{K})_{\mathbb{C}}$. First, the analyticity of $\hat{\rho }$ implies for some $\varepsilon > 0$ the existence of a holomorphic map F on $B_{\varepsilon }:=\{ C \in M(n,\mathbb{K})_{\mathbb{C}}: \|C\| <\varepsilon \}$ with $F(C) =\hat{\rho } (C)$ for $C \in M(n,\mathbb{K}) \cap B_{\varepsilon }$. This map also satisfies $F(rC) = \hat{\rho }(r\mathbf{1})F(C)$ for r < 1, which implies that F extends to a holomorphic map on $r^{-1}B_{\varepsilon } = B_{r^{-1}\varepsilon }$ for every r > 0. This leads to the existence of a holomorphic extension of $\hat{\rho }$ to all of $M(n,\mathbb{K})_{\mathbb{C}}$. That this extension also is multiplicative follows immediately by analytic continuation. □ 

Theorem 3.15 (Classification of Irreducible θ-Positive Representations).

Put $\mathcal{F}:= \mathbb{K}^{n}$ . Then all irreducible continuous θ-positive representations of $C(\mathcal{F})\mathop{\cong}C(n,\mathbb{K})$ are of the form

$$\displaystyle{\left \{\begin{array}{@{}l@{\quad }l@{}} \mathbb{S}_{\lambda }(\mathcal{F}_{\mathbb{C}}) \subseteq (\mathcal{F}_{\mathbb{C}})^{\otimes N}, \quad &\text{ for }\mathbb{K} = \mathbb{R}, \\ \mathbb{S}_{\lambda }(\mathcal{F}) \otimes \mathbb{S}_{\mu }(\overline{\mathcal{F}}) \subseteq \mathcal{F}^{\otimes N} \otimes \overline{\mathcal{F}}^{\otimes M},\quad &\text{ for }\mathbb{K} = \mathbb{C}, \\ \mathbb{S}_{\lambda }(\mathcal{F}^{\mathbb{C}}) \subseteq (\mathcal{F}^{\mathbb{C}})^{\otimes N}, \quad &\text{ for }\mathbb{K} = \mathbb{H},\\ \quad \end{array} \right.}$$

where $\lambda \in \mathop{\mathrm{Part}}\nolimits (N,n),\mu \in \mathop{\mathrm{Part}}\nolimits (M,n)$ .

Proof.

Let (ρ, V ) be an irreducible θ-positive representation of $C(n,\mathbb{K})$. Then $\rho (Z) \subseteq \mathbb{C}\mathbf{1}$ by Schur’s Lemma, so that ρ is in particular centrally bounded and extends to a holomorphic representation $\hat{\rho }: M(n,\mathbb{K})_{\mathbb{C}} \rightarrow B(V )$ (Proposition 3.14).

For $\mathbb{K} = \mathbb{R},\mathbb{H}$, the center of $M(n,\mathbb{K})_{\mathbb{C}}$ is $\mathbb{C}\mathbf{1}$. Since the only holomorphic multiplicative maps $\mathbb{C} \rightarrow \mathbb{C}$ are of the form $z\mapsto z^{N}$ for some $N \in \mathbb{N}_{0}$, it follows that $\hat{\rho }(z\mathbf{1}) = z^{N}\mathbf{1}$ for $z \in \mathbb{C}$. We conclude that the holomorphic map $\hat{\rho }$ is homogeneous of degree N. Hence there exists a linear map

$$\displaystyle{\tilde{\rho }: S^{N}(M(n,\mathbb{K})_{\mathbb{C}}) \rightarrow B(V )\quad \text{ with }\quad \tilde{\rho }(A^{\otimes N}) = \hat{\rho }(A),\quad A \in M(n,\mathbb{K})_{\mathbb{C}}.}$$

The multiplicativity of $\hat{\rho }$ now implies that $\tilde{\rho }$ is multiplicative, hence a representation of the finite-dimensional algebra $S^{N}(M(n,\mathbb{K})_{\mathbb{C}})$.

For $\mathbb{K} = \mathbb{R}$, we have $M(n,\mathbb{R})_{\mathbb{C}} = M(n,\mathbb{C})$, and

$$\displaystyle{S^{N}(M(n,\mathbb{C})) = (M(n,\mathbb{C})^{\otimes N})^{S_{N} }\mathop{\cong}M(nN,\mathbb{C})^{S_{N} }\mathop{\cong}B((\mathbb{C}^{n})^{\otimes N})^{S_{N} }.}$$

We conclude that $S^{N}(M(n,\mathbb{C}))$ is the commutant of S _N in $M(nN,\mathbb{C})$, and by Schur–Weyl theory, this algebra can be identified with the image of the group algebra $\mathbb{C}[\mathop{\mathrm{GL}}\nolimits (n,\mathbb{C})]$ in $B((\mathbb{C}^{n})^{\otimes N})$. Therefore its irreducible representations are parametrized by the set $\mathop{\mathrm{Part}}\nolimits (N,n)$ of partitions of N into at most n summands. This completes the proof for $\mathbb{K} = \mathbb{R}$. For $\mathbb{K} = \mathbb{H}$, we have the same picture because $M(n,\mathbb{H})_{\mathbb{C}}\mathop{\cong}M(2n,\mathbb{C})$.

For $\mathbb{K} = \mathbb{C}$, $Z(M(n,\mathbb{C})_{\mathbb{C}})\mathop{\cong}\mathbb{C}^{2}$, and the inclusion of $Z(M(n,\mathbb{C})) = \mathbb{C}\mathbf{1}$ has the form $z\mapsto (z,\overline{z})$. Hence there exist $N,M \in \mathbb{N}_{0}$ with $\rho _{Z}(z\mathbf{1}) = z^{N}\overline{z}^{M}\mathbf{1}$. Therefore the restriction of $\hat{\rho }$ to the first factor is homogeneous of degree N and the restriction to the second factor of degree M. This leads to a representation of the algebra

$$\displaystyle{S^{N,M}(M(n,\mathbb{C})):= S^{N}(M(n,\mathbb{C})) \otimes S^{M}(M(n,\mathbb{C})),}$$

so that the same arguments as in the real case apply. □ 

Now that we know all irreducible θ-positive representations, we ask for the corresponding decomposition theory.

Theorem 3.16.

Every continuous θ-positive ∗-representation of $C(n,\mathbb{K})$ by contractions is a direct sum of irreducible ones, and these are finite-dimensional.

Proof.

We have already seen that ρ decomposes into a direct sum of centrally bounded representations. We may therefore assume that ρ is centrally bounded, so that ρ extends to a holomorphic representation $\hat{\rho }$ of $M(n,\mathbb{K})_{\mathbb{C}}$ (Proposition 3.14). In the C ^∗-algebra $\mathcal{A}:= M(n,\mathbb{K})_{\mathbb{C}}$, every holomorphic function is uniquely determined by its restriction to the unitary group $\mathop{\mathrm{U}}\nolimits (\mathcal{A})$, which is a totally real submanifold. Therefore a closed subspace W ⊆ V is invariant under $\rho (C(n,\mathbb{K}))$ if and only if it is invariant under $\hat{\rho }(\mathop{\mathrm{U}}\nolimits (\mathcal{A}))$. Since the group $\mathop{\mathrm{U}}\nolimits (\mathcal{A})$ is compact, the assertion now follows from the classical fact that unitary representations of compact groups are direct sums of irreducible ones. □ 

3.4 The Classification Theorem

We are now ready to prove the Kirillov–Olshanski Theorem [Ki73, Ol78].

Theorem 3.17 (Classification of the Representations of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$)).

Let $\mathcal{H}$ be an infinite-dimensional separable $\mathbb{K}$ -Hilbert space.

• (a) The irreducible continuous unitary representations of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ are

  $$\displaystyle{\left \{\begin{array}{@{}l@{\quad }l@{}} \mathbb{S}_{\lambda }(\mathcal{H}_{\mathbb{C}}) \subseteq (\mathcal{H}_{\mathbb{C}})^{\otimes N}, \quad &\text{ for }\mathbb{K} = \mathbb{R}, \\ \mathbb{S}_{\lambda }(\mathcal{H}) \otimes \mathbb{S}_{\mu }(\overline{\mathcal{H}}) \subseteq \mathcal{H}^{\otimes N} \otimes \overline{\mathcal{H}}^{\otimes M},\quad &\text{ for }\mathbb{K} = \mathbb{C}, \\ \mathbb{S}_{\lambda }(\mathcal{H}^{\mathbb{C}}) \subseteq (\mathcal{H}^{\mathbb{C}})^{\otimes N}, \quad &\text{ for }\mathbb{K} = \mathbb{H},\\ \quad \end{array} \right.}$$

  where $\lambda \in \mathop{\mathrm{Part}}\nolimits (N),\mu \in \mathop{\mathrm{Part}}\nolimits (M)$ .

• (b) Every continuous unitary representation of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ is a direct sum of irreducible ones.

• (c) Every continuous unitary representation of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ extends uniquely to a continuous unitary representations of the full unitary group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})_{s}$ , endowed with the strong operator topology.

Proof.

• (a) In Theorem 3.15 we have classified the irreducible θ-positive representations of $C(n,\mathbb{K})$. The corresponding representations $(\pi,\mathcal{H}_{\pi })$ of $K =\mathop{ \mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ can now be determined rather easily. Since the passage from ρ to π preserves direct sums, we consider the representations ρ _N of $C(n,\mathbb{K})$ on $\mathcal{F}_{\mathbb{C}}^{\otimes N}$ for $\mathbb{K} = \mathbb{R}$, on $(\mathcal{F}^{\mathbb{C}})^{\otimes N}$ for $\mathbb{K} = \mathbb{H}$, and the representation ρ _N, M on $\mathcal{F}^{\otimes N} \otimes \overline{\mathcal{F}}^{\otimes M}$ for $\mathbb{K} = \mathbb{C}$.

  We likewise have unitary representations π _N of K on $(\mathcal{H}_{\mathbb{C}})^{\otimes N}$ for $\mathbb{K} = \mathbb{R}$, on $(\mathcal{H}^{\mathbb{C}})^{\otimes N}$ for $\mathbb{K} = \mathbb{H}$, and a representation π _N, M on $\mathcal{H}^{\otimes N} \otimes \overline{\mathcal{H}}^{\otimes M}$ for $\mathbb{K} = \mathbb{C}$. These are bounded continuous representations of K.

  For $K = \mathbb{R},\mathbb{H}$, the space of K _n -fixed vectors in $(\mathcal{H}_{\mathbb{C}})^{\otimes N}$ obviously contains $(\mathcal{F}_{\mathbb{C}})^{\otimes N}$ and by considering the action of the subgroup of diagonal matrices, we obtain the equality $(\mathcal{F}_{\mathbb{C}})^{\otimes N} = ((\mathcal{H}_{\mathbb{C}})^{\otimes N})^{K_{n}}$. Therefore the representation π _N corresponds to the representation ρ _N of $C(n,\mathbb{K})$. A similar argument shows that for $\mathbb{K} = \mathbb{C}$, the K-representation π _N, M corresponds to ρ _N, M .

  Since the representations ρ _N and ρ _N, M decompose into finitely many irreducible pieces, the representations π _N and π _N, M decompose in precisely the same way. For $\mathbb{K} = \mathbb{R},\mathbb{H}$, we thus obtain the Schur modules $\mathbb{S}_{\lambda }(\mathcal{H}_{\mathbb{C}})$ and $\mathbb{S}_{\lambda }(\mathcal{H}^{\mathbb{C}})$ with $\lambda \in \mathop{\mathrm{Part}}\nolimits (N)$, respectively. For $\mathbb{K} = \mathbb{C}$, we obtain the tensor products $\mathbb{S}_{\lambda }(\mathcal{H}) \otimes \mathbb{S}_{\mu }(\overline{\mathcal{H}})$ with $\lambda \in \mathop{\mathrm{Part}}\nolimits (N)$ and $\mu \in \mathop{\mathrm{Part}}\nolimits (M)$.

  Here the restriction to partitions consisting of at most n summands corresponds to the K-invariant subspace generated by the K _n -fixed vectors. This subspace is proper if n is small.

• (b) From Theorem 3.16 we know that θ-positive contraction representations of $C(n,\mathbb{K})$ are direct sums of irreducible ones. This implies that all continuous unitary representations of K are direct sums of irreducible ones. Since the correspondence between π and ρ leads to isomorphic commutants, the irreducible subrepresentations of π and the corresponding subrepresentations of ρ have the same multiplicities.

• (c) The assertion is trivial for the irreducible representations of K described under (a). Since $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ is dense in $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ with respect to the strong operator topology,⁴ this extension is unique and generates the same von Neumann algebra.

 □ 

Remark 3.18 (Representations of $\mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H})$).

The above classification can easily be extended to the nonconnected group $\mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H})$ (for $\mathbb{K} = \mathbb{R}$). Here the existence of a canonical extension $\overline{\pi }_{\lambda }$ of every irreducible representations π _λ of $\mathop{\mathrm{SO}}\nolimits _{\infty }(\mathcal{H}):=\mathop{ \mathrm{O}}\nolimits _{\infty }(\mathcal{H})_{0}$ to $\mathop{\mathrm{O}}\nolimits (\mathcal{H})$ implies that there exist precisely two extensions that differ by a twist with the canonical character $D: \mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H}) \rightarrow \{\pm 1\}$ corresponding to the determinant.

For a general continuous unitary representations of $\mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H})$, it follows that all $\mathop{\mathrm{SO}}\nolimits _{\infty }(\mathcal{H})$-isotypic subspaces are invariant under $\mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H})$, hence of the form $\mathcal{M}_{\lambda }\otimes \mathcal{H}_{\lambda }$, where $\mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H})$ acts by $\varepsilon \otimes \overline{\pi }_{\lambda }$ and $\varepsilon$ is a unitary representation of the 2-element group $\pi _{0}(\mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H}))$, i.e., defined by a unitary involution.

In particular, all continuous unitary representations of $\mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H})$ are direct sums of irreducible ones, which are of the form $\overline{\pi }_{\lambda }$ and $D \otimes \overline{\pi }_{\lambda }$. Here the first type extends to the full orthogonal group $\mathop{\mathrm{O}}\nolimits (\mathcal{H})$, whereas the second type does not.

Remark 3.19 (Extension to Overgroups).

(cf. [Ol84, §1.11]) Let $\mathcal{H}$ be an infinite-dimensional separable $\mathbb{K}$-Hilbert space and $K:=\mathop{ \mathrm{U}}\nolimits _{\mathbb{K}}(\mathcal{H})$. We put

$$\displaystyle{\mathcal{H}^{\sharp }:= \left \{\begin{array}{@{}l@{\quad }l@{}} \mathcal{H}_{\mathbb{C}} \quad &\text{ for }\mathbb{K} = \mathbb{R}\\ \mathcal{H} \oplus \overline{\mathcal{H} } \quad &\text{ for } \mathbb{K} =\mathbb{C} \\ \mathcal{H}^{\mathbb{C}} \quad &\text{ for }\mathbb{K} = \mathbb{H}. \end{array} \right.}$$

For each $N \in \mathbb{N}$ we obtain a norm continuous representation

$$\displaystyle{\pi _{N}: (B(\mathcal{H}^{\sharp }),\cdot ) \rightarrow B((\mathcal{H}^{\sharp })^{\otimes N}),\quad \pi _{ N}(A):= A^{\otimes N}}$$

of the multiplicative semigroup $(B(\mathcal{H}^{\sharp }),\cdot )$ whose restriction to $\mathop{\mathrm{U}}\nolimits (\mathcal{H}^{\sharp })$ is unitary.

We collect some properties of this representation:

• (a) Let $C(\mathcal{H}^{\sharp }) =\{ S \in B(\mathcal{H}^{\sharp }): \|S\| \leq 1\}$ denote the closed subsemigroup of contractions. Then $C(\mathcal{H}^{\sharp })$ is a ∗-subsemigroup of $B(\mathcal{H}^{\sharp })$ and $\pi _{N}\vert _{C(\mathcal{H}^{\sharp })}$ is continuous with respect to the weak operator topology. In fact, $\pi _{N}(C(\mathcal{H}^{\sharp }))$ consists of contractions, and for the total subset of vectors of the form $v:= v_{1} \otimes \cdots \otimes v_{N}$, $w:= w_{1} \otimes \cdots \otimes w_{N}$, the matrix coefficient $S\mapsto \langle \pi _{N}(S)v,w\rangle =\prod _{ j=1}^{N}\langle Sv_{j},w_{j}\rangle$ is continuous.

• (b) $K:=\mathop{ \mathrm{U}}\nolimits _{\mathbb{K}}(\mathcal{H})$ is dense in $C_{\mathbb{K}}(\mathcal{H})$ with respect to the weak operator topology. It suffices to see that for every contraction C on a finite-dimensional subspace $\mathcal{F}\subseteq \mathcal{H}$, there exists a unitary operator $U \in \mathop{\mathrm{U}}\nolimits _{\mathbb{K}}(\mathcal{H})$ with $P_{\mathcal{F}}UP_{\mathcal{F}}^{{\ast}} = C$, where $P_{\mathcal{F}}: \mathcal{H}\rightarrow \mathcal{F}$ is the orthogonal projection. Since $\mathcal{F}\oplus \mathcal{F}$ embeds isometrically into $\mathcal{H}$, this follows from the fact that the matrix

  $$\displaystyle{U:= \left (\begin{array}{*{10}c} C &\sqrt{\mathbf{1} - CC^{{\ast}}} \\ -\sqrt{\mathbf{1} - C^{{\ast} } C}& C^{{\ast}}\end{array} \right ) \in M_{2}(B_{\mathbb{K}}(\mathcal{F})) = B_{\mathbb{K}}(\mathcal{F}\oplus \mathcal{F})}$$

  is unitary and satisfies $P_{\mathcal{F}}UP_{\mathcal{F}}^{{\ast}} = C$ (Lemma 3.9).

• (c) Combining (a) and (b) implies that $\pi _{N}(C_{\mathbb{K}}(\mathcal{H})) \subseteq \pi _{N}(K)''$, and hence that $\pi _{N}(B_{\mathbb{K}}(\mathcal{H})) =\bigcup _{\lambda >0}\lambda ^{N}\pi _{N}(C_{\mathbb{K}}(\mathcal{H})) \subseteq \pi _{N}(K)''$. For the corresponding Lie algebra representation

  $$\displaystyle{\mathtt{d}\pi _{N}: \mathop{\mathfrak{g}\mathfrak{l}}\nolimits _{\mathbb{K}}(\mathcal{H}) \rightarrow B(\mathcal{H}^{\otimes N}),\quad \mathtt{d}\pi _{ N}(X):=\sum _{ j=1}^{N}\mathbf{1}^{\otimes (j-1)} \otimes X \otimes \mathbf{1}^{\otimes (N-j)},}$$

  this implies that $\mathtt{d}\pi _{N}(\mathop{\mathfrak{g}\mathfrak{l}}\nolimits _{\mathbb{K}}(\mathcal{H})) \subseteq \pi _{N}(K)''$, and hence also that $\mathtt{d}\pi _{N}(\mathop{\mathfrak{g}\mathfrak{l}}\nolimits _{\mathbb{K}}(\mathcal{H}))_{\mathbb{C}} \subseteq \pi _{N}(K)''$. The connectedness of the group K ^♯ (Lemma 2.4) now implies that $\pi _{N}(K^{\sharp }) \subseteq \pi _{N}(K)''$. Since the subgroup $K_{\infty }^{\sharp }$, consisting of those elements g for which g −1 is compact, is strongly dense in K ^♯ and the representation of K ^♯ is continuous with respect to the strong operator topology, the representations π _N of K ^♯ thus decomposes into Schur modules, as described in Theorem 3.17(a).

• (d) The preceding discussion shows in particular that the representation π _N of K extends to the overgroup K ^♯ without enlarging the corresponding von Neumann algebra. If ρ is the corresponding representation of $C(n,\mathbb{K}) = C(\mathcal{F})$ on $V:= (\mathcal{F}^{\sharp })^{\otimes N}$, where $\mathcal{F} = \mathbb{K}^{n}$, then ρ extends to a holomorphic representation $\hat{\rho }$ of $M(n,\mathbb{K})_{\mathbb{C}}$ and the map $\theta: K \rightarrow C(n,\mathbb{K})$ likewise extends to a holomorphic map $\hat{\theta }: B(\mathcal{H})_{\mathbb{C}} \rightarrow M(n,\mathbb{K})_{\mathbb{C}}$. Now $\hat{\rho } \circ \hat{\theta }: B(\mathcal{H})_{\mathbb{C}} \rightarrow B(V )$ is a holomorphic positive definite function corresponding to the representation of $(B(\mathcal{H})_{\mathbb{C}},\cdot )$ on $(\mathcal{H}^{\sharp })^{\otimes N}$ whose restriction yields a unitary representation of the unitary group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})^{\sharp }$ of $B(\mathcal{H})_{\mathbb{C}}$.

We conclude this subsection with the following converse to Proposition 3.7.

Theorem 3.20.

A unitary representation of $\mathop{\mathrm{U}}\nolimits (\infty,\mathbb{K})_{0}$ is tame if and only if it extends to a continuous unitary representation of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ for $\mathcal{H} =\ell ^{2}(\mathbb{N},\mathbb{K})$ .

Proof.

We have already seen in Proposition 3.7 that every continuous unitary representation of $K =\mathop{ \mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ restricts to a tame representation of $K(\infty ) =\mathop{ \mathrm{U}}\nolimits (\infty,\mathbb{K})_{0}$.

Suppose, conversely, that $(\pi,\mathcal{H}_{\pi })$ is a tame unitary representation of K(∞). Then the same arguments as for K imply that it is a direct sum of representations generated by the subspace $V = (\mathcal{H}_{\pi })^{K(\infty )_{n}}$ and we obtain a representation (ρ, V ) of $C(n,\mathbb{K})$ for which ρ ∘θ is positive definite on K(∞). Since it is continuous and K(∞) is dense in K, it is also positive definite on K. Now the GNS construction, applied to ρ ∘θ, yields the continuous extension of π to K. □ 

3.5 The Inseparable Case

In this subsection we show that Theorem 3.17 extends to the case where $\mathcal{H}$ is not separable.

Theorem 3.21.

Theorem  3.17 also holds if $\mathcal{H}$ is inseparable.

Proof.

• (a) First we note that the Schur–Weyl decomposition

  $$\displaystyle{\mathcal{H}^{\otimes N}\mathop{\cong}\bigoplus _{ \lambda \in \mathop{\mathrm{Part}}\nolimits (N)}\mathbb{S}_{\lambda }(\mathcal{H}) \otimes \mathcal{M}_{\lambda }}$$

  holds for any infinite-dimensional complex Hilbert space [BN12] and that the spaces $\mathbb{S}_{\lambda }(\mathcal{H})$ carry irreducible representations of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ which are continuous with respect to the norm topology and the strong operator topology on $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$.

• (b) To obtain the irreducible representations of $K:=\mathop{ \mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$, we choose an orthonormal basis $(e_{j})_{j\in J}$ of $\mathcal{H}$ and assume that $\mathbb{N} =\{ 1,2,\ldots \}$ is a subset of J. Accordingly, we obtain an embedding $K(\infty ):=\mathop{ \mathrm{U}}\nolimits (\infty,\mathbb{K})\hookrightarrow \mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$ and define $K_{n}:=\{ k \in K: ke_{j} = e_{j},j = 1,\ldots,n\}$. For a subset M ⊆ J, we put $K(M):=\mathop{ \mathrm{U}}\nolimits _{\infty }(\mathcal{H}_{M})_{0}$, where $\mathcal{H}_{M} \subseteq \mathcal{H}$ is the closed subspace generated by $(e_{j})_{j\in M}$.

• (c) Kirillov’s Lemma 3.6 is still valid in the inseparable case and Lemma 3.10 follows from the separable case because θ(K) = θ(K(∞)).

• (d) With the same argument as in Sect. 3.2 it follows that π is a direct sum of subrepresentations for which $(\mathcal{H}_{\pi })^{K_{n}}$ is cyclic. These in turn correspond to θ-positive representations (ρ, V ) of $C(n,\mathbb{K})$. We claim that ρ ∘θ is positive definite on $K(\mathbb{N})$ if and only if it is positive definite on K(M) for any countable subset with $\mathbb{N} \subseteq M \subseteq J$. In fact, there exists a unitary isomorphism $U_{M}: \mathcal{H}(\mathbb{N}) \rightarrow \mathcal{H}(M)$ fixing $e_{1},\ldots,e_{n}$. For k ∈ K(M) we then have $\rho (\theta (k)) =\rho (\theta (U_{M}^{{\ast}}kU_{M}))$, so that ρ ∘θ is positive definite on K(M) if it is on $K(\mathbb{N})$.

  For every finite subset F ⊆ K, there exists a countable subset J _c  ⊆ J containing $\mathbb{N}$ such that F fixes all basis elements e _j , $j\not\in J_{c}$. Therefore ρ ∘θ is positive definite on K if and only if this is the case on K(M) for every countable subset and this in turn follows from the positive definiteness on the subgroup $K(\mathbb{N})$. We conclude that the classification of the unitary representations of K is the same as for $K(\mathbb{N})$. □ 

Remark 3.22.

If $\mathcal{H}$ is inseparable, then the classification implies that all irreducible unitary representations of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ are inseparable. In particular, all separable unitary representations of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ are trivial because they are direct sums of irreducible ones.

Problem 3.23.

It seems that the classification problem we dealt with in this section can be formulated in a more general context as follows. Let $\mathcal{A}$ be a real involutive Banach algebra and $P \in \mathcal{A}$ be a hermitian projection, so that we obtain a closed subalgebra $\mathcal{A}_{P}:= P\mathcal{A}P$. On the unitary group.

$$\displaystyle{\mathop{\mathrm{U}}\nolimits (\mathcal{A}):=\{ A \in \mathcal{A}: A^{{\ast}}A = AA^{{\ast}} = \mathbf{1}\}}$$

We consider the map

$$\displaystyle{\theta: \mathop{\mathrm{U}}\nolimits (\mathcal{A}) \rightarrow C(\mathcal{A}_{P}):=\{ A \in \mathcal{A}_{P}: \|A\| \leq 1\},\quad \theta (g):= PgP.}$$

For which ∗-representations (ρ, V ) of the semigroup $C(\mathcal{A}_{P})$ is the function $\rho \circ \theta: \mathop{\mathrm{U}}\nolimits (\mathcal{A})_{0} \rightarrow B(V )$ positive definite?

For $\mathcal{A} = B_{\infty }(\mathcal{H})$, the compact operators on the $\mathbb{K}$-Hilbert space $\mathcal{H}$ and a finite rank projection P, this problem specializes to the determination of the θ-positive representations of $C(n,\mathbb{K})$.

If P is central, then θ is a ∗-homomorphism, so that ρ ∘θ is positive definite for any representation ρ.

4 Separable Representations of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$

In this section we show that for the unitary group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ of a separable Hilbert space $\mathcal{H}$, endowed with the norm topology, all separable representations are uniquely determined by their restrictions to the normal subgroup $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$. This result of Pickrell [Pi88] extends the Kirillov–Olshanski classification to separable representations of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$.

4.1 Triviality of Separable Representations Modulo Compacts

Before we turn to the proof of Theorem 4.6, we need a few preparatory lemmas.

Lemma 4.1.

Let $\mathcal{H}$ be an infinite-dimensional Hilbert space, $(\pi,\mathcal{H}_{\pi })$ be a continuous unitary representation of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ , and $\mathcal{H} = V \oplus V ^{\perp }$ with $V \mathop{\cong}V ^{\perp }$ . Then $\mathcal{H}_{\pi }^{\mathop{\mathrm{U}}\nolimits (V )}\not =\{0\}$ .

Proof.

Put $\mathcal{H}_{1}:= V$ and write $V ^{\perp }$ as a Hilbert space direct sum $\widehat{\oplus }_{j=2}^{\infty }\mathcal{H}_{j}$, where each $\mathcal{H}_{j}$ is isomorphic to V or $\mathcal{H}$. This is possible because $\vert J\vert = \vert \mathbb{N}\times J\vert$ for every infinite set J. We claim that some $\mathop{\mathrm{U}}\nolimits (\mathcal{H}_{j})$ has nonzero fixed points in $\mathcal{H}_{\pi }$. Once this claim is proved, we choose $g \in \mathop{\mathrm{U}}\nolimits (\mathcal{H})$ with $gV = \mathcal{H}_{j}$. Then $\pi (g)\mathcal{H}_{\pi }^{\mathop{\mathrm{U}}\nolimits (V )} = \mathcal{H}_{\pi }^{\mathop{\mathrm{U}}\nolimits (\mathcal{H}_{j})}\not =\{0\}$ implies the assertion.

For the proof we want to use Proposition 1.6. In $G:=\mathop{ \mathrm{U}}\nolimits (\mathcal{H})$ we consider the basis of 1-neighborhoods given by $U_{\varepsilon }:=\{ g \in \mathop{\mathrm{U}}\nolimits (\mathcal{H}): \|g -\mathbf{1}\| <\varepsilon \}$ and the subgroups $G_{j}:=\mathop{ \mathrm{U}}\nolimits (\mathcal{H}_{j})$. Then the proof of Proposition 1.3(i) shows that there exists an $m \in \mathbb{N}$ with $G_{j} \subseteq (U_{\varepsilon } \cap G_{j})^{m}$ for every j, which is (U1). It is also clear that (U2) is satisfied. Therefore the assertion follows from Proposition 1.6. □ 

Lemma 4.2.

Let $\mathcal{F}\subseteq \mathcal{H}$ be a closed subspace of finite codimension. Then the natural morphism $\mathop{\mathrm{U}}\nolimits (\mathcal{F}) \rightarrow \mathop{\mathrm{U}}\nolimits (\mathcal{H})/\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$ is surjective, i.e., $\mathop{\mathrm{U}}\nolimits (\mathcal{H}) =\mathop{ \mathrm{U}}\nolimits _{\infty }(\mathcal{H})\mathop{\mathrm{U}}\nolimits (\mathcal{F})$ .

Proof.

Since the groups $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ and $\mathop{\mathrm{U}}\nolimits (\mathcal{F})$ are connected (Proposition 1.3(i)), it suffices to show that their Lie algebras satisfy

$$\displaystyle{\mathfrak{u}(\mathcal{H}) = \mathfrak{u}(\mathcal{F}) + \mathfrak{u}_{\infty }(\mathcal{H}).}$$

Let $P: \mathcal{H}\rightarrow \mathcal{H}$ be the orthogonal projection onto $\mathcal{F}$. Then every $X \in \mathfrak{u}(\mathcal{H})$ can be written as

$$\displaystyle{X = PXP + (\mathbf{1} - P)XP + X(\mathbf{1} - P),}$$

where $PXP \in \mathfrak{u}(\mathcal{F})$ and the other two summands are compact because 1 − P has finite range. □ 

Lemma 4.3.

Let $(\pi,\mathcal{H}_{\pi })$ be a continuous unitary representation of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ with $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H}) \subseteq \ker \pi$ and $\mathcal{H} =\widehat{ \oplus }_{j\in J}\mathcal{H}_{j}$ with $\mathcal{H}_{j}$ infinite-dimensional separable and J infinite. Then $\bigcap _{j\in J}\mathcal{H}_{\pi }^{\mathop{\mathrm{U}}\nolimits (\mathcal{H}_{j})}\not =\{0\}$ .

Proof.

Let $V \subseteq \mathcal{H}$ be a closed subspace of the form $V = \overline{\sum _{j}V _{j}}$, where each $V _{j} \subseteq \mathcal{H}_{j}$ is a closed subspace of codimension 1. Then $V ^{\perp }\mathop{\cong}\ell^{2}(J,\mathbb{C})\mathop{\cong}\mathcal{H}\mathop{\cong}V$ because $\vert J \times \mathbb{N}\vert = \vert J\vert$. According to Lemma 4.2, we then have

$$\displaystyle{\mathop{\mathrm{U}}\nolimits (\mathcal{H}_{j}) \subseteq \mathop{\mathrm{U}}\nolimits (V _{j})\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H}_{j}) \subseteq \mathop{\mathrm{U}}\nolimits (V )\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H}).}$$

In view of Lemma 4.1, $\mathop{\mathrm{U}}\nolimits (V )$ has nonzero fixed points in $\mathcal{H}_{\pi }$, and since $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H}) \subseteq \ker \pi$, any such fixed point is fixed by all the subgroups $\mathop{\mathrm{U}}\nolimits (\mathcal{H}_{j})$. □ 

From now on we assume that $\mathcal{H}$ is separable.

Lemma 4.4.

Let $(\pi,\mathcal{H}_{\pi })$ be a continuous unitary representation of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ with $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H}) \subseteq \ker \pi$ and $g \in \mathop{\mathrm{U}}\nolimits (\mathcal{H})$ . If 1 is contained in the essential spectrum of g, i.e., the image of g − 1 in the Calkin algebra $B(\mathcal{H})/K(\mathcal{H})$ is not invertible, then 1 is an eigenvalue of π(g).

Proof.

We choose an orthogonal decomposition $\mathcal{H} =\widehat{ \oplus }_{n=1}^{\infty }\mathcal{H}_{n}$ into infinite-dimensional g-invariant subspaces of $\mathcal{H}$ as follows.

Case 1: :

If 1 is an eigenvalue of g of infinite multiplicity, then we put $\mathcal{H}_{0}:=\ker (\mathbf{1} - g)^{\perp }$. If this space is infinite-dimensional, then we put $\mathcal{H}_{1}:= \mathcal{H}_{0}$, and if this is not the case, then we pick a subspace $\mathcal{H}_{0}' \subseteq \mathcal{H}_{0}^{\perp }$ of infinite dimension and codimension and put $\mathcal{H}_{1}:= \mathcal{H}_{0} \oplus \mathcal{H}_{0}'$. We choose all other $\mathcal{H}_{n}$, n > 1, such that $\mathcal{H}_{1}^{\perp } =\widehat{ \oplus }_{n=2}^{\infty }\mathcal{H}_{n}$ and note that $\mathcal{H}_{1}^{\perp }\subseteq \ker (\mathbf{1} - g)$.

Case 2: :

If $\ker (\mathbf{1} - g)$ is finite-dimensional, then let $P_{\varepsilon } \in B(\mathcal{H})$ be the spectral projection for g corresponding to the closed disc of radius $\varepsilon > 0$ about 1. Then

$$\displaystyle{g_{\varepsilon }:= P_{\varepsilon } \oplus (\mathbf{1} - P_{\varepsilon })g}$$

satisfies $\|g_{\varepsilon } - g\| \leq \varepsilon$. The noncompactness of g −1 implies that if $\varepsilon$ is small enough, then

$$\displaystyle{g_{\varepsilon } -\mathbf{1} = 0 \oplus (\mathbf{1} - P_{\varepsilon })(g -\mathbf{1})}$$

is noncompact, and hence that $P_{\varepsilon }\mathcal{H}$ has infinite codimension. Further $P_{\varepsilon }\mathcal{H}$ is infinite-dimensional because 1 is an essential spectral value of g. Hence there exists a sequence $\varepsilon _{1} >\varepsilon _{2} >\ldots$ converging to 0, for which the g-invariant subspaces

$$\displaystyle{\mathcal{H}_{1}:= (P_{\varepsilon _{1}}\mathcal{H})^{\perp }\quad \text{ and }\quad \mathcal{H}_{ j}:= P_{\varepsilon _{j-1}}\mathcal{H}\cap (P_{\varepsilon _{j}}\mathcal{H})^{\perp }}$$

are infinite-dimensional.

In both cases, we consider $g_{\varepsilon }$ as an element $(g_{\varepsilon,n})$ of the product group $\prod _{n=1}^{\infty }\mathop{\mathrm{U}}\nolimits (\mathcal{H}_{n}) \subseteq \mathop{\mathrm{U}}\nolimits (\mathcal{H})$ satisfying $g_{\varepsilon,n} = \mathbf{1}$ for n sufficiently large. If $v \in \mathcal{H}_{\pi }$ is a nonzero simultaneous fixed vector for the subgroups $\mathop{\mathrm{U}}\nolimits (\mathcal{H}_{n})$ (Lemma 4.3), we obtain $\pi (g_{\varepsilon })v = v$ for every $\varepsilon > 0$, and now $v =\pi (g_{\varepsilon })v \rightarrow \pi (g)v$ implies that π(g)v = v. □ 

As an immediate consequence, we obtain:

Lemma 4.5.

Let $(\pi,\mathcal{H}_{\pi })$ be a continuous unitary representation of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ with $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H}) \subseteq \ker \pi$ and $j \in \mathbb{Z}$ with $\pi (\zeta \mathbf{1}) =\zeta ^{j}\mathbf{1}$ for $\zeta \in \mathbb{T}$ . If λ is contained in the essential spectrum of g, then λ ^j is an eigenvalue of π(g).

Proof.

Lemma 4.4 implies that π(λ ⁻¹ g) has a nonzero fixed vector v, and this means that π(g)v = λ ^j v. □ 

Theorem 4.6.

If $\mathcal{H}$ is a separable Hilbert space over $\mathbb{K} \in \{\mathbb{R},\mathbb{C},\mathbb{H}\}$ , then every continuous unitary representation of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})/\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ on a separable Hilbert space is trivial.

Proof.

• (a) We start with the case $\mathbb{K} = \mathbb{C}$. Let $(\pi,\mathcal{H}_{\pi })$ be a separable continuous unitary representation of the Banach–Lie group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ with $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H}) \subseteq \ker \pi$.

  Step 1: $\mathbb{T}\mathbf{1} \subseteq \ker \pi$: Decomposing the representation of the compact central subgroup $\mathbb{T}\mathbf{1}$, we may w.l.o.g. assume that $\pi (\zeta \mathbf{1}) =\zeta ^{j}\mathbf{1}$ for some $j \in \mathbb{Z}$. Let $g \in \mathop{\mathrm{U}}\nolimits (\mathcal{H})$ be an element with uncountable essential spectrum. If $j\not =0$, then Lemma 4.5 implies that π(g) has uncountably many eigenvalues, which is impossible if $\mathcal{H}_{\pi }$ is separable. Therefore j = 0, and this means that $\mathbb{T}\mathbf{1} \subseteq \ker \pi$.

  Step 2: Let $P \in B(\mathcal{H})$ be an orthogonal projection with infinite rank. Then $\mathop{\mathrm{U}}\nolimits (P\mathcal{H})\mathop{\cong}\mathop{\mathrm{U}}\nolimits (\mathcal{H})$, so that Step 1 implies that $\mathbb{T}P + (\mathbf{1} - P) \subseteq \ker \pi.$ If P has finite rank, then

  $$\displaystyle{\mathbb{T}P + (\mathbf{1} - P) \subseteq \mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H}) \subseteq \ker \pi.}$$

  This implies that $\ker \pi$ contains all elements g with $\mathrm{Spec}(g) \subseteq \{ 1,\zeta \}$ for some $\zeta \in \mathbb{T}$. Since every element with finite spectrum is a finite product of such elements, it is also contained in $\ker \pi$. Finally we derive from the Spectral Theorem that the subset of elements with finite spectrum is dense in $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$, so that π is trivial.⁵

• (b) Next we consider the orthogonal group $\mathop{\mathrm{U}}\nolimits _{\mathbb{R}}(\mathcal{H}) =\mathop{ \mathrm{O}}\nolimits (\mathcal{H})$ of a real Hilbert space $\mathcal{H}$. Since $\mathcal{H}$ is infinite-dimensional, there exists an orthogonal complex structure $I \in \mathop{\mathrm{O}}\nolimits (\mathcal{H})$. Then $\tau (g):= IgI^{-1}$ defines an involution on $\mathop{\mathrm{O}}\nolimits (\mathcal{H})$ whose fixed point set is the unitary group $\mathop{\mathrm{U}}\nolimits (\mathcal{H},I)$ of the complex Hilbert space $(\mathcal{H},I)$.

  Let $\pi: \mathop{\mathrm{O}}\nolimits (\mathcal{H}) \rightarrow \mathop{\mathrm{U}}\nolimits (\mathcal{H}_{\pi })$ be a continuous separable unitary representation with $\mathop{\mathrm{SO}}\nolimits _{\infty }(\mathcal{H}):=\mathop{ \mathrm{O}}\nolimits _{\infty }(\mathcal{H})_{0} \subseteq N:=\ker \pi$. Applying (a) to $\pi \vert _{\mathop{\mathrm{U}}\nolimits (\mathcal{H},I)}$, it follows that $\mathop{\mathrm{U}}\nolimits (\mathcal{H},I) \subseteq N$ and hence in particular that I ∈ N.

  For $X^{\top } = -X$ and $\tau (X) = -X$ we then obtain

  $$\displaystyle{N \ni I\exp (X)I^{-1}\exp (-X) =\exp (-X)\exp (-X) =\exp (-2X).}$$

  This implies that $\mathop{\mathbf{L}}\nolimits (N) =\{ X \in \mathfrak{o}(\mathcal{H}): \exp (\mathbb{R}X) \subseteq N\} = \mathfrak{o}(\mathcal{H})$, and since $\mathop{\mathrm{O}}\nolimits (\mathcal{H})$ is connected by Example 1.4, it follows that $N =\mathop{ \mathrm{O}}\nolimits (\mathcal{H})$, i.e., that π is trivial.

• (c) Now let $\mathcal{H}$ be a quaternionic Hilbert space, considered as a right $\mathbb{H}$-module. Realizing $\mathcal{H}$ as $\ell^{2}(S,\mathbb{H})$ for some set S, we see that $\mathcal{K}:=\ell ^{2}(S,\mathbb{C})$ is a complex Hilbert space whose complex structure is given by left multiplication $\lambda _{\mathcal{I}}$ with the basis element $\mathcal{I}\in \mathbb{H}$ (this map is $\mathbb{H}$-linear) and we have a direct sum $\mathcal{H}^{\mathbb{C}} = \mathcal{K}\oplus \mathcal{K}\mathcal{J}$ of complex Hilbert spaces.

  Let $\sigma: \ell^{2}(S,\mathbb{H}) \rightarrow \ell^{2}(S,\mathbb{H})$ be the real linear isometry given by $\sigma (v) = \mathcal{I}v\mathcal{I}^{-1}$ pointwise on S, so that $\mathcal{H}^{\sigma } =\ell ^{2}(S,\mathbb{C}) = \mathcal{K}$ and $\mathcal{K}\mathcal{J} = \mathcal{H}^{-\sigma }$. Then τ(g): = σ g σ defines an involution on $\mathop{\mathrm{U}}\nolimits _{\mathbb{H}}(\mathcal{H})$ whose group of fixed points is isomorphic to the unitary group $\mathop{\mathrm{U}}\nolimits (\mathcal{K})$ of the complex Hilbert space $\mathcal{K}$, on which the complex structure is given by right multiplication with $\mathcal{I}$, which actually coincides with the left multiplication.

  Let $\pi: \mathop{\mathrm{U}}\nolimits _{\mathbb{H}}(\mathcal{H}) \rightarrow \mathop{\mathrm{U}}\nolimits (\mathcal{H}_{\pi })$ be a continuous separable unitary representation with $\mathop{\mathrm{U}}\nolimits _{\mathbb{H},\infty }(\mathcal{H}) \subseteq N:=\ker \pi$. Applying (a) to $\pi \vert _{\mathop{\mathrm{U}}\nolimits (\mathcal{K})}$, it follows that $\mathop{\mathrm{U}}\nolimits (\mathcal{K}) \subseteq N$ and hence in particular that $\lambda _{\mathcal{I}}\in N$. On the Lie algebra level, $\mathfrak{u}(\mathcal{K})$ is complemented by

  $$\displaystyle{\{X \in \mathfrak{u}_{\mathbb{H}}(\mathcal{H}): \sigma X = -X\sigma \} =\{ X \in \mathfrak{u}_{\mathbb{H}}(\mathcal{H}): \lambda _{\mathcal{I}}X = -X\lambda _{\mathcal{I}}\},}$$

  and for any element of this space we have

  $$\displaystyle{N \ni \lambda _{\mathcal{I}}\exp (X)\lambda _{\mathcal{I}}^{-1}\exp (-X) =\exp (-X)\exp (-X) =\exp (-2X).}$$

  This implies that $\mathop{\mathbf{L}}\nolimits (N) = \mathfrak{u}_{\mathbb{H}}(\mathcal{H})$, and since $\mathop{\mathrm{U}}\nolimits _{\mathbb{H}}(\mathcal{H})$ is connected by Proposition 1.3(i), $N =\mathop{ \mathrm{U}}\nolimits _{\mathbb{H}}(\mathcal{H})$, so that π is trivial. □ 

Remark 4.7.

For $\mathbb{K} = \mathbb{R}$, the group $\mathop{\mathrm{O}}\nolimits (\mathcal{H})/\mathop{\mathrm{SO}}\nolimits _{\infty }(\mathcal{H})$ is the 2-fold simply connected cover of the group $\mathop{\mathrm{O}}\nolimits (\mathcal{H})/\mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H})$. Therefore the triviality of all separable representations of $\mathop{\mathrm{O}}\nolimits (\mathcal{H})/\mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H})$ follows from the triviality of all separable representations of $\mathop{\mathrm{O}}\nolimits (\mathcal{H})/\mathop{\mathrm{SO}}\nolimits _{\infty }(\mathcal{H}) =\mathop{ \mathrm{O}}\nolimits (\mathcal{H})/\mathop{\mathrm{O}}\nolimits _{\infty }(\mathcal{H})_{0}$.

Problem 4.8.

If $\mathcal{H}$ is an inseparable Hilbert space, then we think that all separable unitary representations $(\pi,\mathcal{H})$ of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ should be trivial, but we can only show that $\ker \pi$ contains all operators for which $(g -\mathbf{1})\mathcal{H}$ is separable, i.e., all groups $\mathop{\mathrm{U}}\nolimits (\mathcal{H}_{0})$, where $\mathcal{H}_{0} \subseteq \mathcal{H}$ is a separable subspace.

The argument works as follows. From Remark 3.22 we know that all irreducible representations of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$ are inseparable. Theorem 3.21 implies that $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H}) \subseteq \ker \pi$. Now Theorem 4.6 implies that $\ker \pi$ contains all subgroups $\mathop{\mathrm{U}}\nolimits (\mathcal{H}_{0})$, where $\mathcal{H}_{0}$ is a separable Hilbert space, and this proves our claim.

4.2 Separable Representations of the Lie Group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$

Based on Pickrell’s theorem on the triviality of the separable representations of the quotient Lie groups $\mathop{\mathrm{U}}\nolimits (\mathcal{H})/\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$, we can now determine all separable continuous unitary representations of the full unitary group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$.

Theorem 4.9.

Let $\mathcal{H}$ be a separable $\mathbb{K}$ -Hilbert space. Then every separable continuous unitary representation $(\pi,\mathcal{H}_{\pi })$ of the Banach–Lie group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ has the following properties:

• (i) It is continuous with respect to the strong operator topology on $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ .

• (ii) Its restriction to $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ has the same commutant.

• (iii) It is a direct sum of bounded irreducible representations.

• (iv) Every irreducible separable representation is of the form

  $$\displaystyle{\left \{\begin{array}{@{}l@{\quad }l@{}} \mathbb{S}_{\lambda }(\mathcal{H}_{\mathbb{C}}) \subseteq (\mathcal{H}_{\mathbb{C}})^{\otimes N},\lambda \in \mathop{\mathrm{Part}}\nolimits (N), \quad &\text{ for }\mathbb{K} = \mathbb{R}, \\ \mathbb{S}_{\lambda }(\mathcal{H}) \otimes \mathbb{S}_{\mu }(\overline{\mathcal{H}}) \subseteq \mathcal{H}^{\otimes N} \otimes \overline{\mathcal{H}}^{\otimes M},\lambda \in \mathop{\mathrm{Part}}\nolimits (N),\mu \in \mathop{\mathrm{Part}}\nolimits (M),\quad &\text{ for }\mathbb{K} = \mathbb{C}, \\ \mathbb{S}_{\lambda }(\mathcal{H}^{\mathbb{C}}) \subseteq (\mathcal{H}^{\mathbb{C}})^{\otimes N},\lambda \in \mathop{\mathrm{Part}}\nolimits (N), \quad &\text{ for }\mathbb{K} = \mathbb{H}.\\ \quad \end{array} \right.}$$

• (v) π extends uniquely to a strongly continuous representation of the overgroup $\mathop{\mathrm{U}}\nolimits (\mathcal{H})^{\sharp }$ with the same commutant.

Proof.

(i)  From Theorem 3.17(c) we know that $\pi _{\infty }:=\pi \vert _{\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})}$ extends to a unique continuous unitary representation $\overline{\pi }$ of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})_{s}$ on $\mathcal{H}_{\pi }$. In particular, the action of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ on the unitary dual of the normal subgroup $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ is trivial. Hence all isotypic subspaces $\mathcal{H}_{[\lambda ]}$ for π _∞ are invariant under π. We may therefore assume that π _∞ is isotypic, i.e., of the form 1 ⊗ρ _λ , where (ρ _λ , V _λ ) is an irreducible representation of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ (cf. Theorem 3.17). Then $\overline{\pi }:= \mathbf{1} \otimes \overline{\rho }_{\lambda }$ is continuous with respect to the operator norm on $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ because the representations of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ on the spaces $\mathcal{H}_{\mathbb{C}}^{\otimes N}$ are norm-continuous (Theorem 3.17(c)).

Now

$$\displaystyle{\beta (g):=\pi (g)\overline{\pi }(g)^{-1} \in \pi (\mathop{\mathrm{U}}\nolimits _{ \infty }(\mathcal{H}))' = \overline{\pi }(\mathop{\mathrm{U}}\nolimits (\mathcal{H}))'}$$

implies that $\beta: \mathop{\mathrm{U}}\nolimits (\mathcal{H}) \rightarrow \mathop{\mathrm{U}}\nolimits (\mathcal{H}_{\pi })$ defines a separable norm-continuous unitary representation vanishing on $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})$. By Theorem 4.6 it is trivial, so that $\pi = \overline{\pi }$.

• (ii) follows from (i) and the density of $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ in $\mathop{\mathrm{U}}\nolimits (\mathcal{H})_{s}$.

• (iii), (iv) now follow from Theorem 3.17.

• (v) In view of (iii), assertion (v) reduces to the case of irreducible representations. In this case (v) follows from the concrete classification (iv) and the description of the overgroups $\mathop{\mathrm{U}}\nolimits (\mathcal{H})^{\sharp }$ in Lemma 2.4.

 □ 

Corollary 4.10.

Let K be a quotient of a product $K_{1} \times \cdots \times K_{n}$ , where each K _j is compact, a quotient of some group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ or $\mathop{\mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ , where $\mathcal{H}$ is a separable $\mathbb{K}$ -Hilbert space. Then every separable continuous unitary representation π of K is a direct sum of irreducible representations which are bounded.

The preceding corollary means that the separable representation theory of K is very similar to the representation theory of a compact group.

4.3 Classification of Irreducible Representations by Highest Weights

We choose an orthonormal basis $(e_{j})_{j\in J}$ in the complex Hilbert space $\mathcal{H}$ and write $T\mathop{\cong}\mathbb{T}^{J}$ for the corresponding group of diagonal matrices. Characters of this group correspond to finitely supported functions $\lambda: J \rightarrow \mathbb{Z}$ via $\chi _{\lambda }(t) =\prod _{j\in J}t_{j}^{\lambda _{j}}$. For the subgroup T(∞) of those diagonal matrices t for which t −1 has finite rank, any function $\lambda: J \rightarrow \mathbb{Z}$ defines a character. Accordingly, each $\lambda = (\lambda _{j})_{j\in J} \in \mathbb{Z}^{J}$ defines a uniquely determined unitary highest weight representation $(\pi _{\lambda },\mathcal{H}_{\lambda })$ of $\mathop{\mathrm{U}}\nolimits (\infty,\mathbb{C})$ [Ne04, Ne98]. This representation is uniquely determined by the property that its weight set with respect to the diagonal subgroup $T\mathop{\cong}\mathbb{T}^{(J)}$, whose character group $\widehat{T}\mbox{ is }\mathbb{Z}^{J}$, coincides with

$$\displaystyle{\mathop{\mathrm{conv}}\nolimits (\mathcal{W}\lambda ) \cap (\lambda +\mathcal{Q}),\quad \text{ where }\quad \mathcal{Q}\subseteq \widehat{ T}}$$

is the root group and $\mathcal{W}$ is the group of finite permutations of the set J.

Proposition 4.11.

A unitary highest weight representation $(\pi _{\lambda },\mathcal{H}_{\lambda })$ of $\mathop{\mathrm{U}}\nolimits (\infty,\mathbb{C})$ is tame if and only if $\lambda: \mathbb{N} \rightarrow \mathbb{Z}$ is finitely supported.

Proof.

If π _λ is a tame representation, then its restriction to the diagonal subgroup is tame. Since this representation is diagonalizable, this means that each weight has finite support. It follows in particular that λ has finite support.

If, conversely, λ has finite support, then we write $\lambda =\lambda _{+} -\lambda _{-}$, where λ _± are nonnegative with finite disjoint support. Then $\mathcal{H}_{\lambda }$ can be embedded into $\mathbb{S}_{\lambda _{+}}(\mathcal{H}) \otimes \mathbb{S}_{\lambda _{-}}(\overline{\mathcal{H}})$ [Ne98], hence it is tame. □ 

Example 4.12.

$\mathbb{K} = \mathbb{R}$: In the infinite-dimensional real Hilbert space $\mathcal{H}$ we fix a complex structure I. Then there exists a real orthonormal basis of the form $\{e_{j},Ie_{j}: j \in J\}$. Then the subgroup $T \subseteq \mathop{\mathrm{O}}\nolimits (\mathcal{H})$ preserving all the planes $\mathbb{R}e_{j} + I\mathbb{R}e_{j}$ is maximal abelian. In $\mathcal{H}_{\mathbb{C}}$ the elements $e_{j}^{\pm }:= \frac{1} {\sqrt{2}}(e_{j} \mp Ie_{j})$ form an orthonormal basis, and we write 2J: = J ×{±} for the corresponding index set. In $\mathop{\mathrm{O}}\nolimits (\mathcal{H})^{\sharp } =\mathop{ \mathrm{U}}\nolimits (\mathcal{H}_{\mathbb{C}})$, the corresponding diagonal subgroup $T^{\sharp }\mathop{\cong}\mathbb{T}^{2J}$ is maximal abelian. The corresponding maximal torus $T_{\mathbb{C}}$ of $\mathop{\mathrm{O}}\nolimits (\mathcal{H})_{\mathbb{C}} \subseteq \mathop{\mathrm{GL}}\nolimits (\mathcal{H}_{\mathbb{C}})$ corresponds to diagonal matrices d acting by $de_{j}^{\pm } = d_{j}^{\pm 1}e_{j}^{\pm }$.

For a character χ _μ of T ^♯ with $\mu: 2J \rightarrow \mathbb{Z}$, the corresponding character of T is given by the finitely supported function $\mu ^{\flat }: J \rightarrow \mathbb{Z}$ with $\mu _{j}^{\flat } =\mu _{j,+} -\mu _{j,-}$. If $\lambda: 2J \rightarrow \mathbb{N}_{0}$ has finite support, then the corresponding irreducible representation of $\mathop{\mathrm{U}}\nolimits (\mathcal{H}_{\mathbb{C}})$ occurs as some $\mathbb{S}_{\lambda }(\mathcal{H}_{\mathbb{C}})$ in $\mathcal{H}_{\mathbb{C}}^{\otimes N}$, where ∑ _j ∈ 2J λ _j  = N. From the Classification Theorem 3.17 it follows that the restriction of π _λ to $\mathop{\mathrm{O}}\nolimits (\mathcal{H})$ is irreducible. The corresponding highest weight is $\lambda ^{\flat }$. On the level of highest weights, it is clear that, for each finitely supported weight $\lambda: J \rightarrow \mathbb{N}_{0}$, we obtain by

$$\displaystyle{\lambda _{j,+}^{\sharp }:=\lambda _{ j}\quad \text{ and }\quad \lambda _{j,-}^{\sharp }:= 0}$$

a highest weight λ ^♯ with $(\lambda ^{\sharp })^{\flat } =\lambda$. The irreducible representations of $\mathop{\mathrm{O}}\nolimits (\mathcal{H})$ are classified by orbits of the Weyl group $\mathcal{W}$ in the set of finitely supported integral weights $\lambda: J \rightarrow \mathbb{Z}$ of the root system D _2J (cf. [Ne98, Sect. VII]). Each orbit has a nonnegative representative, and then λ ^♯ is the highest weight of the corresponding representation $\pi _{\lambda ^{\sharp }}$ of $\mathop{\mathrm{U}}\nolimits (\mathcal{H}_{\mathbb{C}})$.

Example 4.13.

$\mathbb{K} = \mathbb{C}$: Let $(e_{j})_{j\in J}$ be an ONB of $\mathcal{H}$. In $\mathop{\mathrm{U}}\nolimits (\mathcal{H})^{\sharp }\mathop{\cong}\mathop{ \mathrm{U}}\nolimits (\mathcal{H}) \times \mathop{\mathrm{U}}\nolimits (\overline{\mathcal{H}})$ we have the maximal abelian subgroup $T^{\sharp } = T \times T$, where $T\mathop{\cong}\mathbb{T}^{J}$ is the subgroup of diagonal matrices in $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ with respect to the ONB $(e_{j})_{j\in J}$.

Let 2J: = J ×{±}, so that $T^{\sharp }\mathop{\cong}\mathbb{T}^{2J}$. For a finitely supported function $\mu: 2J \rightarrow \mathbb{Z}$, the corresponding character of T is given by $\mu ^{\flat }: J \rightarrow \mathbb{Z}$, defined by $\mu _{j}^{\flat } =\mu _{j,+} -\mu _{j,-}$. If $\lambda: 2J \rightarrow \mathbb{N}_{0}$ has finite support, and $\lambda =\lambda _{+} -\lambda _{-}$ with nonnegative summands λ _± supported in J ×{±}, respectively, the corresponding irreducible representation π _λ lives on $\mathbb{S}_{\lambda _{+}}(\mathcal{H}) \otimes \mathbb{S}_{\lambda _{-}}(\overline{\mathcal{H}}) \subseteq \mathcal{H}^{\otimes N} \otimes \overline{\mathcal{H}}^{\otimes M}$, where $N =\sum _{\lambda _{j}>0}\lambda _{j}$ and $M = -\sum _{\lambda _{j}<0}\lambda _{j}$. From the Classification Theorem 3.17 it follows that the restriction of π _λ to $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ is irreducible. The corresponding highest weight is $\lambda ^{\flat } =\lambda _{+} -\lambda _{-}$. For each finitely supported weight $\lambda =\lambda _{+} -\lambda _{-}: J \rightarrow \mathbb{N}_{0}$, we obtain by $\lambda _{j,\pm }^{\sharp }:=\lambda _{\pm,j},j \in J$, a highest weight λ ^♯ with $(\lambda ^{\sharp })^{\flat } =\lambda$. The irreducible representations of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ are classified by orbits of the Weyl group $\mathcal{W}\mathop{\cong}S_{(J)}$ in the set of finitely supported integral weights $\lambda: J \rightarrow \mathbb{Z}$ of the root system A _J (cf. [Ne98, Sect. VII]).

Example 4.14.

$\mathbb{K} = \mathbb{H}$: In the quaternionic Hilbert space $\mathcal{H}$ we consider the complex structure defined by multiplication with $\mathcal{I}$, which leads to the complex Hilbert space $\mathcal{H}^{\mathbb{C}}$. Then there exists a complex orthonormal basis of the form $\{e_{j},\mathcal{J}e_{j}: j \in J\}$. We write $T^{\sharp } \subseteq \mathop{\mathrm{U}}\nolimits (\mathcal{H}^{\mathbb{C}})$ for the corresponding diagonal subgroup. Note that $T^{\sharp }\mathop{\cong}\mathbb{T}^{2J}$ for 2J: = J ×{±}. The subgroup $T:= T^{\sharp } \cap \mathop{\mathrm{U}}\nolimits (\mathcal{H}) = (T^{\sharp })^{\mathcal{J}}$ acts on the basis elements $e_{j,+}:= e_{j}$ and $e_{j,-}:= \mathcal{J}e_{j}$ by $de_{j,\pm } = d_{j}^{\pm }e_{j,\pm }$.

The classification of the irreducible representations by Weyl group orbits of finitely supported functions $\lambda: J \rightarrow \mathbb{Z}$ (weights for the root system B _J ) and their corresponding weights $\lambda ^{\sharp }: 2J \rightarrow \mathbb{Z}$ is completely analogous to the situation for $\mathbb{K} = \mathbb{R}$. The irreducible representation of $\mathop{\mathrm{U}}\nolimits (\mathcal{H}^{\mathbb{C}})$ corresponding to λ ^♯ is $\mathbb{S}_{\lambda ^{\sharp }}(\mathcal{H}^{\mathbb{C}})$.

Remark 4.15 (Segal’s Physical Representations).

In [Se57] Segal studied unitary representations of the full group $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$, called physical representations. They are characterized by the condition that their differential maps finite rank hermitian projections to positive operators. Segal shows that physical representations decompose discretely into irreducible physical representations which are precisely those occurring in the decomposition of finite tensor products $\mathcal{H}^{\otimes N}$, $N \in \mathbb{N}_{0}$. In view of Pickrell’s theorem, this also follows from our classification of the separable representations of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$. Since Segal’s arguments never use the separability of $\mathcal{H}$, the corresponding result remains true for inseparable spaces as well.

Problem 4.16.

Theorem 3.17 implies in particular that all continuous unitary representations of $K =\mathop{ \mathrm{U}}\nolimits _{\infty }(\mathcal{H})_{0}$ have a canonical extension to their overgroups K ^♯ with the same commutant. The classification in terms of highest weights further implies that the representations of K ^♯ obtained from this extension process are precisely those with nonnegative weights.

Conversely, it follows that all unitary representations of K ^♯ with nonnegative weights remain irreducible when restricted to K.

One may ask a similar question for the smaller group $K^{\sharp }(\infty ) \subseteq K^{\sharp }$ or its completion with respect to the trace norm. Is it true that for any unitary representation π of K ^♯(∞) whose weights on the diagonal subgroup are nonnegative, $\pi (\mathop{\mathrm{U}}\nolimits (\infty,\mathbb{K}))$ has the same commutant? As we explain below, this is not true.

For the special case $\mathbb{K} = \mathbb{R}$ and $\lambda =\lambda _{+} -\lambda _{-}$ finitely supported, the restriction of the representation $\pi _{\lambda } =\pi _{\lambda _{+}} \otimes \pi _{\lambda _{-}}^{{\ast}}$ of K ^♯(∞) on $S_{\lambda _{+}}(\mathcal{H}_{\mathbb{C}}) \otimes S_{\lambda _{-}}(\overline{\mathcal{H}_{\mathbb{C}}})$ to the subgroup $K(\infty ) =\mathop{ \mathrm{SO}}\nolimits (\infty,\mathbb{R})$ is equivalent to the representation $\pi _{\lambda _{+}} \otimes \pi _{\lambda _{-}}$, which decomposes according to the standard Schur–Weyl theory. In particular, we obtain non-irreducible representations if λ takes positive and negative values on K(∞). That this cannot be repaired by the positivity requirement on the weights of $K^{\sharp }(\infty )$ follows from the fact that the determinant $\det: K^{\sharp }(\infty ) \rightarrow \mathbb{T}$ restricts to the trivial character of K(∞), but tensoring with a power of $\det$, any bounded weight λ can be made positive.

Is it possible to characterize those irreducible highest weight representations π _λ of $K^{\sharp }(\infty )$ whose restriction to K(∞) is irreducible?

5 Non-existence of Separable Unitary Representations for Full Operator Groups

In this section we describe some consequences of the main results from [Pi90]. We start with the description of 10 symmetric pairs (G, K) of groups of operators, where G does not consist of unitary operators and $K \subseteq G$ is “maximal unitary”. They are infinite-dimensional analogs of certain noncompact real reductive Lie groups. The dual symmetric pairs (G ^c, K) have the property that G ^c consists of unitary operators, hence they are analogs of certain compact matrix groups.

One of the main result of this section is that all separable unitary representations of the groups G are trivial, but there are various refinements concerning restricted groups.

5.1 The 10 Symmetric Pairs

Below we use the following notational conventions. We write $\mathop{\mathrm{O}}\nolimits (n):=\mathop{ \mathrm{O}}\nolimits (n,\mathbb{R})$, $\mathop{\mathrm{U}}\nolimits (n):=\mathop{ \mathrm{U}}\nolimits (n,\mathbb{C})$ and $\mathop{\mathrm{Sp}}\nolimits (n):=\mathop{ \mathrm{U}}\nolimits (n,\mathbb{H})$ for $n \in \mathbb{N}\cup \{\infty \}$. For a group G, we write $\varDelta _{G}:=\{ (g,g): g \in G\}$ for the diagonal subgroup of G × G.

If $\mathcal{H}$ is a complex Hilbert space, then we write $I \in B(\mathcal{H}_{\mathbb{C}})$ for the $\mathbb{C}$-linear extension of the complex structure Iv = iv on $\mathcal{H}$. Then $D:= -iI$ is a unitary involution that leads to the pseudo-unitary group

$$\displaystyle{\mathop{\mathrm{U}}\nolimits (\mathcal{H}_{\mathbb{C}},D) =\{ g \in \mathop{\mathrm{GL}}\nolimits (\mathcal{H}_{\mathbb{C}}): Dg^{{\ast}}D^{-1} = g^{-1}\}}$$

preserving the indefinite hermitian form $\langle Dv,w\rangle$. For the isometry group of the indefinite form $h((v_{1},v_{2}),(w_{1},w_{2})):=\langle v_{1},w_{1}\rangle -\langle v_{2},w_{2}\rangle$ on $\mathcal{H}\times \mathcal{H}$, we write $\mathop{\mathrm{U}}\nolimits (\mathcal{H},\mathcal{H})$. Now the group

$$\displaystyle{\mathop{\mathrm{O}}\nolimits ^{{\ast}}(\mathcal{H}_{\mathbb{C}}):=\mathop{ \mathrm{U}}\nolimits (\mathcal{H}_{\mathbb{C}},D) \cap \mathop{\mathrm{O}}\nolimits (\mathcal{H})_{\mathbb{C}}}$$

is a Lie group. Its Lie algebra $\mathfrak{o}^{{\ast}}(\mathcal{H}_{\mathbb{C}})$ satisfies $\mathfrak{o}^{{\ast}}(\mathcal{H}_{\mathbb{C}}) \cap \mathfrak{u}(\mathcal{H}_{\mathbb{C}})\mathop{\cong}\mathfrak{u}(\mathcal{H})$ and it is a real form of $\mathfrak{o}(\mathcal{H})_{\mathbb{C}}$. It is easy to see that the symmetric pair $(\mathop{\mathrm{O}}\nolimits ^{{\ast}}(\mathcal{H}_{\mathbb{C}}),\mathop{\mathrm{U}}\nolimits (\mathcal{H}))$ is dual to $(\mathop{\mathrm{O}}\nolimits (\mathcal{H}^{\mathbb{R}}),\mathop{\mathrm{U}}\nolimits (\mathcal{H}))$.

Non-unitary Symmetric Pairs

Table 1

Unitary Symmetric Pairs

Table 2

Remark 5.1.

• (a) The unitary symmetric pairs (1)–(3) are of group type and their non-unitary duals are complex groups.

• (b) The non-unitary pairs (4)–(6) are the symmetric pairs associated to pseudo-unitary groups of indefinite hermitian forms β with the matrix $D = \left (\begin{array}{*{10}c} \mathbf{1}& 0\\ 0 &-\mathbf{1}\end{array} \right )$ on $\mathcal{H}^{2}$. Accordingly, the corresponding symmetric spaces can be considered as Graßmannians of “maximal positive subspaces” for β.

• (c) The symmetric spaces corresponding to (7) and (8) are spaces of complex structures on real spaces. The space $\mathop{\mathrm{Sp}}\nolimits (\mathcal{H})/\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ is the space of positive symplectic complex structures on the real symplectic spaces $(\mathcal{H},\omega )$, where $\omega (v,w) =\mathop{ \mathrm{Im}}\nolimits \langle v,w\rangle$. Likewise $\mathop{\mathrm{O}}\nolimits (\mathcal{H}^{\mathbb{R}})/\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ is the space of orthogonal complex structures on the real Hilbert space $\mathcal{H}^{\mathbb{R}}$.

• (d) The spaces (4), (7) and (8) are of hermitian type (cf. [Ne12]).

• (e) The spaces (1), (9) and (10) are those occurring naturally for overgroups of unitary groups (cf. Example 2.3).

5.2 Restricted Symmetric Pairs

For each symmetric pair (G, K) of non-unitary type and 1 ≤ q ≤ ∞, we obtain a restricted symmetric pair (G _(q), K), defined by

$$\displaystyle{G_{(q)}:=\{ g \in G: \mathop{\mathrm{tr}}\nolimits (\vert g^{{\ast}}g -\mathbf{1}\vert ^{q}) < \infty \}.}$$

If $\mathfrak{g} = \mathfrak{k} \oplus \mathfrak{p}$ with $\mathfrak{p} =\{ X \in \mathfrak{g}: X^{{\ast}} = X\}$, then the Lie algebra of G ^c is $\mathfrak{g}_{(q)} = \mathfrak{k} \oplus \mathfrak{p}_{(q)}$, where $\mathfrak{p}_{(q)} = \mathfrak{p} \cap B_{q}(\mathcal{H})$. The corresponding dual symmetric pair is (G _(q) ^c, K) with $\mathfrak{g}_{(q)}^{c} = \mathfrak{k} \oplus i\mathfrak{p}_{(q)}$. We also write

$$\displaystyle{G_{\infty,(q)}:= G_{(q)} \cap (\mathbf{1} + K(\mathcal{H})) = K_{\infty }\exp (\mathfrak{p}_{(q)})}$$

for the closure of G(∞) in G _(q).

Proposition 5.2.

Spherical representations of any pair (G(∞),K(∞)) of unitary or non-unitary type are direct integrals of irreducible ones.

Proof.

This is [Pi90, Prop. 2.4], but it also follows from the general Theorem B.3 below. □ 

Combining the preceding proposition with two-sided estimates on the behavior of spherical functions near the identity, Pickrell proved:

Proposition 5.3 ( [Pi90, Prop. 6.11]).

Irreducible real spherical functions of the direct limit pairs (G(∞),K(∞)) always extend to spherical functions of G ₍₂₎ and, for q > 2, all spherical functions on G _(q) vanish.

If v is a C ¹-spherical vector for the unitary representation $(\pi,\mathcal{H}_{\pi })$ of (G _(q), K), then $\beta (X,Y ):=\langle \mathtt{d}\pi (X)v,\mathtt{d}\pi (Y )v\rangle$ defines a continuous K-invariant positive semidefinite symmetric bilinear form on $\mathfrak{p}_{(q)}$. Therefore one can also show that v is fixed by the whole group G _(q) by showing that β = 0 using the following lemma.

Lemma 5.4.

The following assertion holds for the K-action on $\mathfrak{p}_{(q)}$:

• (i) $[\mathfrak{k},\mathfrak{p}] = \mathfrak{p}$ .

• (ii) $\mathfrak{p}_{(2)}$ is an irreducible representation.

• (iii) For q > 2, every continuous K-invariant symmetric bilinear form on $\mathfrak{p}_{(q)}$ vanishes.

Proof.

(i), (ii): We check these conditions for all 10 families:

• (1)–(3) Then $\mathfrak{p} = i\mathfrak{k}$ with $\mathfrak{k} = \mathfrak{u}(\mathcal{H})$. Since $\mathfrak{k}$ is perfect by [Ne02, Lemma I.3], we obtain $[\mathfrak{k},\mathfrak{p}] = i[\mathfrak{k},\mathfrak{k}] = i\mathfrak{k} = \mathfrak{p}$.

  Here $\mathfrak{p}_{(2)} = i\mathfrak{u}_{2}(\mathcal{H})$, and since $\mathfrak{u}_{2}(\mathcal{H})$ is a simple Hilbert–Lie algebra [Sch60], (ii) follows.

• (4)–(6) In these cases $\mathfrak{g}^{c} = \mathfrak{u}(\mathcal{H}\oplus \mathcal{H})$, $\mathfrak{k} = \mathfrak{u}(\mathcal{H}) \oplus \mathfrak{u}(\mathcal{H})$ and $\mathfrak{p}\mathop{\cong}\mathop{\mathfrak{g}\mathfrak{l}}\nolimits (\mathcal{H})$ with the $\mathfrak{k}$-module structure given by $(X,Y ).Z:= XZ - ZY$. Since $\mathfrak{u}(\mathcal{H})$ contains invertible elements X ₀, and (X ₀, 0). Z = X ₀ Z, it follows that $\mathfrak{p} = [\mathfrak{k},\mathfrak{p}]$.

  Here $\mathfrak{p}_{(2)}\mathop{\cong}\mathop{ \mathfrak{g}\mathfrak{l}}\nolimits _{2}(\mathcal{H})$ is the space of Hilbert–Schmidt operators on $\mathcal{H}$. This immediately implies the irreducibility of the representation of $\mathbb{K} = \mathbb{R},\mathbb{C}$. For $\mathbb{K} = \mathbb{H}$, we have $\mathfrak{p}_{(2),\mathbb{C}}\mathop{\cong}\mathop{ \mathfrak{g}\mathfrak{l}}\nolimits _{2}(\mathcal{H}^{\mathbb{C}})$, and since the representation of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ on $\mathcal{H}^{\mathbb{C}}$ is irreducible (we have $\mathop{\mathrm{U}}\nolimits (\mathcal{H})_{\mathbb{C}}\mathop{\cong}\mathop{\mathrm{Sp}}\nolimits (\mathcal{H}^{\mathbb{C}})$), (ii) follows.

• (7)–(8) In these two cases the center $\mathfrak{z}:= i\mathbf{1}$ of $\mathfrak{k}\mathop{\cong}\mathfrak{u}(\mathcal{H})$ satisfies $\mathfrak{p} = [\mathfrak{z},\mathfrak{p}]$, which implies (i). The Lie algebra $\mathfrak{g}_{(2)} = \mathfrak{k} \oplus \mathfrak{p}_{(2)}$ corresponds to the automorphism group of an irreducible hermitian symmetric space (cf. [Ne12, Thm. 2.6] and the subsequent discussion). This implies that the representation of K on the complex Hilbert space $\mathfrak{p}_{(2)}$ is irreducible.

• (9) Here $\mathfrak{g} =\mathop{ \mathfrak{g}\mathfrak{l}}\nolimits (\mathcal{H})$, $\mathfrak{k} = \mathfrak{o}(\mathcal{H})$ and $\mathfrak{p} =\mathop{ \mathrm{Sym}}\nolimits (\mathcal{H})$. For any complex structure $I \in \mathfrak{o}(\mathcal{H})$ we then obtain

  $$\displaystyle{\mathfrak{p} =\mathop{ \mathrm{Herm}}\nolimits (\mathcal{H},I) \oplus [I,\mathfrak{p}] = [\mathfrak{u}(\mathcal{H},I),\mathop{\mathrm{Herm}}\nolimits (\mathcal{H},I)] \oplus [I,\mathfrak{p}] \subseteq [\mathfrak{k},\mathfrak{p}].}$$

  This proves (i).

  Next we observe that $\mathfrak{p}_{(2)} =\mathop{ \mathrm{Sym}}\nolimits _{2}(\mathcal{H})$ satisfies $\mathfrak{p}_{(2),\mathbb{C}}\mathop{\cong}\mathop{ \mathrm{Sym}}\nolimits _{2}(\mathcal{H}_{\mathbb{C}})\mathop{\cong}S^{2}(\overline{\mathcal{H}_{\mathbb{C}}})$, hence is irreducible by Theorem 3.17.

• (10) Here $\mathfrak{g} =\mathop{ \mathfrak{g}\mathfrak{l}}\nolimits (\mathcal{H})$, $\mathfrak{k} = \mathfrak{u}(\mathcal{H})$ and $\mathfrak{p} =\mathop{ \mathrm{Herm}}\nolimits (\mathcal{H})$ for $\mathbb{K} = \mathbb{H}$. With the aid of an orthonormal basis, we find a real Hilbert space $\mathcal{K}$ with $\mathcal{H}\mathop{\cong}\mathcal{K}\otimes _{\mathbb{R}}\mathbb{H}$, where $\mathbb{H}$ acts by right multiplication. This leads to an isomorphism $B_{\mathbb{H}}(\mathcal{H})\mathop{\cong}B_{\mathbb{R}}(\mathcal{K}) \otimes _{\mathbb{R}}\mathbb{H}$ as real involutive algebras. In particular,

  $$\displaystyle{\mathop{\mathrm{Herm}}\nolimits (\mathcal{H})\mathop{\cong}\mathop{\mathrm{Sym}}\nolimits (\mathcal{K}) \otimes \mathbf{1} \oplus \mathop{\mathrm{Asym}}\nolimits (\mathcal{K}) \otimes \mathop{\mathrm{Aherm}}\nolimits (\mathbb{H}).}$$

  Therefore (i) follows from $\mathop{\mathrm{Aherm}}\nolimits (\mathbb{H}) = [\mathop{\mathrm{Aherm}}\nolimits (\mathbb{H}),\mathop{\mathrm{Aherm}}\nolimits (\mathbb{H})]$ and from $\mathop{\mathrm{Sym}}\nolimits (\mathcal{K}) = [\mathfrak{o}(\mathcal{K}),\mathop{\mathrm{Sym}}\nolimits (\mathcal{K})]$, which we derive from (9).

  To verify (ii), we observe that $\mathfrak{p}_{(2)} =\mathop{ \mathrm{Herm}}\nolimits _{2}(\mathcal{H})$. From Kaup’s classification of the real symmetric Cartan domains [Ka97] it follows that $\mathfrak{p}_{(2)}$ is a real form of the complex JH ^∗-triple $\mathop{\mathrm{Skew}}\nolimits (\mathcal{H}^{\mathbb{C}})$ of skew symmetric bilinear forms on $\mathcal{H}^{\mathbb{C}}$, labelled by $II_{2n}^{\mathbb{H}}$. Since the action of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ on $\mathfrak{p}_{(2),\mathbb{C}}\mathop{\cong}\mathop{ \mathrm{Skew}}\nolimits (\mathcal{H}^{\mathbb{C}})\mathop{\cong}\varLambda ^{2}(\overline{\mathcal{H}_{\mathbb{C}}})$ extends to the overgroup $\mathop{\mathrm{U}}\nolimits (\mathcal{H}^{\mathbb{C}})$ with the same commutant, the irreducibility of the resulting representation implies that the representation of $\mathop{\mathrm{U}}\nolimits (\mathcal{H})$ on the real Hilbert space $\mathfrak{p}_{(2)}$ is irreducible as well.

(iii) can be derived from (i). If $\beta: \mathfrak{p}_{(q)} \times \mathfrak{p}_{(q)} \rightarrow \mathbb{R}$ is a continuous invariant symmetric bilinear form, then the same holds for its restriction to $\mathfrak{p}_{(2)}$. The simplicity of the representation on $\mathfrak{p}_{(2)}$ now implies that it is a multiple of the canonical form on $\mathfrak{p}_{(2)}$ given by the trace. But this form does not extend continuously to $\mathfrak{p}_{(q)}$ for any q > 2. □ 

Proposition 5.5.

• (a) Separable unitary representations of G _(q) , q ≥ 1, are completely determined by their restrictions to G(∞).

• (b) Conversely, every continuous separable unitary representation of G _∞,(q) ,q ≥ 1, extends to a continuous unitary representation of G _(q) .

Proof ( cf. [Pi90, Prop. 5.1]).

(a) Since $\mathfrak{p}(\infty )$ is dense in $\mathfrak{p}_{(q)}$, this follows from Theorem 4.9(i), applied to K.

• (b) Since K acts smoothly by conjugation on G _∞, (q), we can form the Lie group G _∞, (q) ⋊ K and note that the multiplication map to G _(q) defines an isomorphism $(G_{\infty,(q)} \rtimes K)/K_{\infty }\rightarrow G_{(q)}$. Therefore the existence of the extension of G _(q) follows from the uniqueness of the extension from K _∞ to K (Theorem 3.17(c)).

 □ 

Theorem 5.6.

If (G,K) is one of the 10 symmetric pairs of non-unitary type, then, for q > 2, all separable projective unitary representations of G _(q) and all projective unitary representations of G _∞,(q) are trivial.

Proof.

If $\pi: G_{(q)} \rightarrow \mathop{\mathrm{PU}}\nolimits (\mathcal{H}_{\pi })$ is a continuous separable projective unitary representation, then composing with the conjugation representation of $\mathop{\mathrm{PU}}\nolimits (\mathcal{H}_{\pi })$ on the Hilbert space $B_{2}(\mathcal{H}_{\pi })$ leads to a separable unitary representation of G _(q) on $B_{2}(\mathcal{H}_{\pi })$. If we can show that this representation is trivial, then π is trivial as well. Therefore it suffices to consider unitary representations.

Since the group G _∞, (q) is separable, all its continuous unitary representations are direct sums of separable ones. Hence, in view of Proposition 5.5, the triviality of all continuous unitary representations of G _∞, (q) is equivalent to the triviality of all separable continuous unitary representations of G _(q). We may therefore restrict our attention to separable representations of G _(q).

Let $(\pi,\mathcal{H})$ be a continuous separable unitary representation of G _(q). In view of Theorem 4.9, it is a direct sum of representations generated by the subspace $\mathcal{H}^{K_{n}}$ for some $n \in \mathbb{N}$. Any $v \in \mathcal{H}^{K_{n}}$ generates a spherical subrepresentation of the subgroup

$$\displaystyle{G_{(q),n}:=\{ g \in G_{(q)}: ge_{j} = e_{j},j = 1,\ldots,n\}.}$$

Now Theorem 5.6 implies that v is fixed by G _(q), n .

It remains to show that G _(q) fixes v. In view of Proposition 5.5, it suffices to show that, for every m > n, G(m) fixes v. The group G(m) is reductive with maximal compact subgroup K(m), and G(m) _n is a non-compact subgroup.

Case 1: :

We first assume that the center of G(m) _n is compact, which is the case for $G(m)\not =\mathop{\mathrm{GL}}\nolimits (m,\mathbb{K})$ (this excludes 1,8 and 9). Then G(m) is minimal in the sense that every continuous bijection onto a topological group is open, and this property is inherited by all its quotient groups [Ma97, Lemma 2.2]. In view of [Ma97, Prop. 3.4], all matrix coefficients of irreducible unitary representations $(\rho,\mathcal{H}_{\rho })$ of quotients of G(m) vanish at infinity of G(m)∕kerρ. If $G(m)_{n}\not\subseteq \ker \rho$, then the image of G(m) _n in the quotient group is noncompact, so that the only vector in $\mathcal{H}_{\rho }$ fixed by G(m) _n is 0. Since every continuous unitary representation of G(m) is a direct integral of irreducible ones, it follows that every G(m) _n -fixed vector in a unitary representation is fixed by G(m).

Case 2: :

If $G(m) =\mathop{ \mathrm{GL}}\nolimits (m,\mathbb{K})$, then $Z = \mathbb{R}_{+}^{\times }\mathbf{1}$ is a noncompact subgroup of the center and the homomorphism $\chi: G \rightarrow \mathbb{R}_{+}^{\times },\chi (g):= \vert \det _{\mathbb{R}}(g)\vert$ is surjective. Therefore S(m): = kerχ has compact center and satisfies G(m) = ZS(m). The preceding argument now implies that every fixed vector for S(m) _n in a unitary representation of G(m) is fixed by S(m). Since $\chi \vert _{G(m)_{n}}$ is nontrivial, we conclude that every fixed vector for G(m) _n in a unitary representation is fixed by G(m).

Combining both cases, we see that in every unitary representation of G(m), the subgroup G(m) _n and G(m) have the same fixed vectors, and this implies that every G _(q), n fixed-vector is fixed by G _(q). □ 

Theorem 5.7 ([Pi90, Prop. 7.1]).

If (G,K) is one of the 10 symmetric pairs of unitary type, then, for q > 2, every separable continuous projective unitary representation of G _(q) extends uniquely to a representation of G that is continuous with respect to the strong operator topology on G. In particular, it is a direct sum of irreducible ones which are determined by Theorem  4.9 .

Proof.

With similar arguments as in the preceding proof, we see that every separable unitary representation of G _(q) is a direct sum of representations generated by the fixed point space of some subgroup G _(q), n . Therefore its restriction to G(∞) is tame, so that Theorems 3.20 and 3.17 apply. □ 

Theorem 5.8.

If (G,K) is one of the 10 symmetric pairs of non-unitary type, then all separable unitary representations of G are trivial.

Proof.

Let $(\pi,\mathcal{H})$ be a continuous separable unitary representation of G. We know already from Theorem 5.6 that $G_{(q)} \subseteq N:=\ker \pi$ holds for q > 2. Now N is a closed normal subgroup containing K and its Lie algebra therefore contains $[\mathfrak{k},\mathfrak{p}] = \mathfrak{p}$ as well (Lemma 5.4). This proves that N = G. □ 

Appendices

A Positive Definite Functions

In this appendix we recall some results and definitions concerning operator-valued positive definite functions.

Definition A.1.

Let $\mathcal{A}$ be a C ^∗-algebra and X be a set. A map $Q: X \times X \rightarrow \mathcal{A}$ is called a positive definite kernel if, for any finite sequence $(x_{1},\ldots,x_{n}) \in X^{n},$ the matrix $Q(x_{i},x_{j})_{i,j=1,\ldots,n} \in M(n,\mathcal{A})$ is a positive element.

For $\mathcal{A} = B(V )$, V a complex Hilbert space, this means that, for $v_{1},\ldots,v_{n} \in V$, we always have $\sum _{i,j=1}^{n}\langle Q(x_{i},x_{j})v_{j},v_{i}\rangle \geq 0$.

Definition A.2.

Let $\mathcal{K}$ be a Hilbert space, G be a group, and U ⊆ G be a subset. A function $\varphi: UU^{-1} \rightarrow B(\mathcal{K})$ is said to be positive definite if the kernel

$$\displaystyle{Q_{\varphi }: U \times U \rightarrow B(\mathcal{K}),\quad (x,y)\mapsto \varphi (xy^{-1})}$$

is positive definite. For U = G we obtain the usual concept of a positive definite function on G.

Remark A.3 (Vector-Valued GNS-Construction).

We briefly recall the bridge between positive definite functions and unitary representations.

• (a) If $(\pi,\mathcal{H})$ is a unitary representation of G, $V \subseteq \mathcal{H}$ a closed subspace and $P_{V }: \mathcal{H}\rightarrow V$ the orthogonal projection on V, then $\pi _{V }(g):= P_{V }\pi (g)P_{V }^{{\ast}}$ is a B(V )-valued positive definite function with π _V (1) = 1.

• (b) If, conversely, $\varphi: G \rightarrow B(V )$ is positive definite with $\varphi (\mathbf{1}) = \mathbf{1}$, then there exists a unique Hilbert subspace $\mathcal{H}_{\varphi }$ of the space V ^G of V -valued function on G for which the evaluation maps $K_{g}: \mathcal{H}_{\varphi }\rightarrow V,f\mapsto f(g)$ are continuous and satisfy $K_{g}K_{h}^{{\ast}} =\varphi (gh^{-1})$ for g, h ∈ G [Ne00, Thm. I.1.4]. Then right translation by elements of G defines a unitary representation $(\pi _{\varphi }(g)f)(x) = f(xg)$ on this space with $K_{xg} = K_{x} \circ \pi (g)$. It is called the GNS-representation associated to ρ. Now $K_{1}^{{\ast}}: V \rightarrow \mathcal{H}_{\varphi }$ is an isometric embedding, so that we may identify V with a closed subspace of $\mathcal{H}_{\varphi }$ and K ₁ with the orthogonal projection to V. This leads to $\varphi (g) = K_{g}K_{1}^{{\ast}} = K_{\mathbf{1}}\pi (g)K_{\mathbf{1}}^{{\ast}}$, so that every positive definite function is of the form π _V . The construction also implies that $V \mathop{\cong}K_{1}^{{\ast}}(V )$ is G-cyclic in $\mathcal{H}_{\varphi }$.

For the following theorem, we simply note that all Banach–Lie groups are in particular Fréchet–BCH–Lie groups.

Theorem A.4.

Let G be a connected Fréchet–BCH–Lie group and U ⊆ G an open connected 1 -neighborhood for which the natural homomorphism $\pi _{1}(U,\mathbf{1}) \rightarrow \pi _{1}(G)$ is surjective. If $\mathcal{K}$ is Hilbert space and $\varphi: UU^{-1} \rightarrow B(\mathcal{K})$ an analytic positive definite function, then there exists a unique analytic positive definite function $\tilde{\varphi }: G \rightarrow B(\mathcal{K})$ extending  $\varphi$ .

Proof.

Let $q_{G}: \tilde{G} \rightarrow G$ be the universal covering morphism. The assumption that $\pi _{1}(U) \rightarrow \pi _{1}(G)$ is surjective implies that $\tilde{U}:= q_{G}^{-1}(U)$ is connected. Now $\tilde{\varphi }:=\varphi \circ q_{G}: \tilde{U}\tilde{U}^{-1} \rightarrow B(\mathcal{K})$ is an analytic positive definite function, hence extends by [Ne12, Thm. A.7] to an analytic positive definite function $\tilde{\varphi }$ on $\tilde{G}$. The restriction of $\tilde{\varphi }$ to $\tilde{U}$ is constant on the fibers of q _G , which are of the form $g\ker (q_{G})$. Using analyticity, we conclude that $\tilde{\varphi }(gd) =\tilde{\varphi } (g)$ holds for all $g \in \tilde{ G}$ and $d \in \ker (q_{G})$. Therefore $\tilde{\varphi }$ factors through an analytic function $\varphi: G \rightarrow B(U)$ which is obviously positive definite. □ 

Theorem A.5.

Let G be a connected analytic Fréchet–Lie group. Then a positive definite function $\varphi: G \rightarrow B(V )$ which is analytic in an open identity neighborhood is analytic.

Proof.

Since $\varphi$ is positive definite, there exists a Hilbert space $\mathcal{H}$ and a $Q: G \rightarrow B(\mathcal{H},V )$ with $\varphi (gh^{-1}) = Q_{g}Q_{h}^{{\ast}}$ for g, h ∈ G. Then the analyticity of the function $\varphi$ in an open identity neighborhood of G implies that the kernel $(g,h)\mapsto Q_{g}Q_{h}^{{\ast}}$ is analytic on a neighborhood of the diagonal $\varDelta _{G} \subseteq G \times G$. Therefore Q is analytic by [Ne12, Thm. A.3], and this implies that $\varphi (g) = Q_{g}Q_{\mathbf{1}}^{{\ast}}$ is analytic. □ 

The following proposition describes a natural source of operator-valued positive definite functions.

Proposition A.6.

Let $(\pi,\mathcal{H})$ be a unitary representation of the group G and H ⊆ G be a subgroup. Let $V \subseteq \mathcal{H}$ be an isotypic H-subspace generating the G-module $\mathcal{H}$ and $P_{V } \in B(\mathcal{H})$ be the orthogonal projection onto V. Then V is invariant under the commutant $\pi (G)' = B_{G}(\mathcal{H})$ and the map

$$\displaystyle{\gamma: B_{G}(\mathcal{H}) \rightarrow B_{H}(V ),\quad \gamma (A) = P_{V }AP_{V }}$$

is an injective morphism of von Neumann algebras whose range is the commutant of the image of the operator-valued positive definite function

$$\displaystyle{\pi _{V }: G \rightarrow B(V ),\quad \pi _{V }(g):= P_{V }\pi (g)P_{V }.}$$

In particular, if the H-representation on V is irreducible, then so is π.

Proof.

That γ is injective follows from the assumption that V generates $\mathcal{H}$ under G. If the representation (ρ, V ) of H is irreducible, then $\mathop{\mathrm{im}}\nolimits (\gamma ) \subseteq \mathbb{C}\mathbf{1}$ implies that $\pi (G)' = \mathbb{C}\mathbf{1}$, so that π is irreducible.

We now determine the range of γ. For any $A \in B_{G}(\mathcal{H})$, we have

$$\displaystyle{P_{V }\pi (g)P_{V }P_{V }AP_{V } = P_{V }\pi (g)AP_{V } = P_{V }A\pi (g)P_{V } = P_{V }AP_{V }P_{V }\pi (g)P_{V },}$$

i.e., γ(A) = P _V AP _V commutes with π _V (G). Since γ is a morphism of von Neumann algebras, its range is also a von Neumann algebra of V commuting with π _V (G). If, conversely, an orthogonal projection $Q = Q^{{\ast}} = Q^{2} \in B_{K}(V )$ commutes with π _V (G), then

$$\displaystyle{P_{V }\pi (G)QV = P_{V }\pi (G)P_{V }QV = QP_{V }\pi (G)P_{V }V \subseteq QV }$$

implies that the closed G-invariant subspace $\mathcal{H}_{Q} \subseteq \mathcal{H}$ generated by QV satisfies $P_{V }\mathcal{H}_{Q} \subseteq QV$, and therefore $\mathcal{H}_{Q} \cap V = QV$. For the orthogonal projection $\tilde{Q} \in B(\mathcal{H})$ onto $\mathcal{H}_{Q}$, which is contained in $B_{G}(\mathcal{H})$, this means that $\tilde{Q}\vert _{V } = Q$. This shows that $\mathop{\mathrm{im}}\nolimits (\gamma ) =\pi _{V }(G)'.$ □ 

Remark A.7.

The preceding proposition is particularly useful if we have specific information on the set π _V (G). As $\pi _{V }(h_{1}gh_{2}) =\rho (h_{1})\pi _{V }(g)\rho (h_{2})$, it is determined by the values of π _V on representatives of the H-double cosets in G.

• (a) In the context of the lowest K-type (ρ, V ) of a unitary highest weight representation (cf. [Ne00]), we can expect that $\pi _{V }(G) \subseteq \rho _{\mathbb{C}}(K_{\mathbb{C}})$ (by Harish–Chandra decomposition), so that $\pi _{V }(G)' =\rho _{\mathbb{C}}(K_{\mathbb{C}})' =\rho (K)'$ and γ is surjective.

• (b) In the context of Sect. 3 and [Ol78], the representation (ρ, V ) of H extends to a representation $\tilde{\rho }$ of a semigroup S ⊇ H and we obtain $\pi _{V }(G)' =\tilde{\rho } (S)'$.

In both situations we have a certain induction procedure from representations of K and S, respectively, to G-representations which preserves the commutant but which need not be defined for every representation of K, resp., S.

Lemma A.8 ( [NO13, Lemma C.3]).

Let (S,∗) be a unital involutive semigroup and $\varphi: S \rightarrow B(\mathcal{F})$ be a positive definite function with $\varphi (\mathbf{1}) = \mathbf{1}$ . We write $(\pi _{\varphi },\mathcal{H}_{\varphi })$ for the representation on the corresponding reproducing kernel Hilbert space $\mathcal{H}_{\varphi }\subseteq \mathcal{F}^{S}$ by $(\pi _{\varphi }(s)f)(t):= f(ts)$ . Then the inclusion

$$\displaystyle{\iota: \mathcal{F}\rightarrow \mathcal{H}_{\varphi },\quad \iota (v)(s):=\varphi (s)v}$$

is surjective if and only if $\varphi$ is multiplicative, i.e., a representation.

Remark A.9.

The preceding lemma can also be expressed without referring to positive definite functions and the corresponding reproducing kernel space. In this context it asserts the following. Let $\pi: S \rightarrow B(\mathcal{H})$ be a ∗-representation of a unital involutive semigroup (S, ∗), $\mathcal{F}\subseteq \mathcal{H}$ a closed cyclic subspace and $P: \mathcal{H}\rightarrow \mathcal{F}$ the orthogonal projection. Then the function

$$\displaystyle{\varphi: S \rightarrow B(\mathcal{F}),\quad \varphi (s):= P\pi (s)P^{{\ast}}}$$

is multiplicative if and only if $\mathcal{F} = \mathcal{H}$.

B C ^∗-Methods for Direct Limit Groups

In this appendix we explain how to apply C ^∗-techniques to obtain direct integral decompositions of unitary representations of direct limit groups.

We recall that for a C ^∗-algebra $\mathcal{A}$, its multiplier algebra $M(\mathcal{A})$ is a C ^∗-algebra containing $\mathcal{A}$ as an ideal, and in every faithful representation $\mathcal{A}\hookrightarrow B(\mathcal{H})$, it is given by

$$\displaystyle{M(\mathcal{A}) =\{ M \in B(\mathcal{H}): M\mathcal{A} + \mathcal{A}M \subseteq \mathcal{A}\}.}$$

Let $G = \lim _{\longrightarrow }\ G_{n}$ be a direct limit of locally compact groups and $\alpha _{n}: G_{n} \rightarrow G_{n+1}$ denote the connecting maps. We assume that these maps are closed embeddings. Then we have natural homomorphisms

$$\displaystyle{\beta _{n}: L^{1}(G_{ n}) \rightarrow M(L^{1}(G_{ n+1}))}$$

of Banach algebras, and since the action of G _n on $L^{1}(G_{n+1})$ is continuous, β _n is nondegenerate in the sense that $\beta (L^{1}(G_{n})) \cdot L^{1}(G_{n+1})$ is dense in $L^{1}(G_{n+1})$. On the level of C ^∗-algebras we likewise obtain morphisms

$$\displaystyle{\beta _{n}: C^{{\ast}}(G_{ n}) \rightarrow M(C^{{\ast}}(G_{ n+1})).}$$

A state of G ( = normalized continuous positive definite function) now corresponds to a sequence $(\varphi _{n})$ of states of the groups G _n with $\alpha _{n}^{{\ast}}\varphi _{n+1}$=$\varphi _{n}$ for every $n \in \mathbb{N}$. Passing to the C ^∗-algebras C ^∗(G _n ), we can view these functions also as states of the C ^∗-algebras. Then the compatibility condition is that the canonical extension $\tilde{\varphi }_{n}$ of $\varphi _{n}$ to the multiplier algebra satisfies

$$\displaystyle{\beta _{n}^{{\ast}}\tilde{\varphi }_{ n+1} =\varphi _{n}.}$$

Remark B.1.

The $\ell^{1}$-direct sum $\mathcal{L}:= \oplus ^{1}L^{1}(G_{n})$ carries the structure of a Banach-∗-algebra (cf. [SV75]). Every unitary representation $(\pi,\mathcal{H})$ of G defines a sequence of nondegenerate representations $\pi _{n}: L^{1}(G_{n}) \rightarrow B(\mathcal{H}_{n})$ which are compatible in the sense that

$$\displaystyle{\pi _{n} =\alpha _{ n}^{{\ast}}\tilde{\pi }_{ n+1}.}$$

Conversely, every such sequence of representations on a Hilbert space $\mathcal{H}$ leads to a sequence $\rho _{n}: G_{n} \rightarrow \mathop{\mathrm{U}}\nolimits (\mathcal{H})$ of continuous unitary representations, which are uniquely determined by

$$\displaystyle{\rho _{n}(g) =\tilde{\pi } _{n}(\eta _{G_{n}}(g)),}$$

where $\eta _{G_{n}}: G_{n} \rightarrow M(L^{1}(G_{n}))$ denotes the canonical action by left multipliers. For $f \in L^{1}(G_{n})$ and $h \in L^{1}(G_{n+1})$ we then have

$$\displaystyle\begin{array}{rcl} & & \rho _{n+1}(\alpha _{n}(g))\pi _{n}(h)\pi _{n+1}(f) =\rho _{n+1}(\alpha _{n}(g))\pi _{n+1}(\beta _{n}(h)f) =\pi _{n+1}(\alpha _{n}(g)\beta _{n}(h)f) {}\\ & & =\pi _{n+1}(\beta _{n}(g {\ast} h)f) =\pi _{n}(g {\ast} h)\pi _{n+1}(f) =\rho _{n}(g)\pi _{n}(h)\pi _{n+1}(f), {}\\ \end{array}$$

which leads to

$$\displaystyle{\rho _{n+1} \circ \alpha _{n} =\rho _{n}.}$$

Therefore the sequence (ρ _n ) is coherent and thus defines a unitary representation of G on $\mathcal{H}$. We conclude that the continuous unitary representations of G are in one-to-one correspondence with the coherent sequences of nondegenerate representations (π _n ) of the Banach-∗-algebras $L^{1}(G_{n})$ (cf. [SV75, p. 60]).

Note that the nondegeneracy condition on the sequence (β _n ) is much stronger than the nondegeneracy condition on the corresponding representation of the algebra $\mathcal{L}$. The group G _n+1 does not act by multipliers on $L^{1}(G_{n})$, so that there is no multiplier action of G on $\mathcal{L}$. However, we have a sufficiently strong structure to apply C ^∗-techniques to unitary representations of G.

Theorem B.2.

Let $\mathcal{A}$ be a separable C ^∗ -algebra and $\pi: \mathcal{A}\rightarrow \mathcal{D}$ a homomorphism into the algebra $\mathcal{D}$ of decomposable operators on a direct integral $\mathcal{H} =\int _{ X}^{\oplus }\mathcal{H}_{x},\,d\mu (x)$ . Then there exists for each x ∈ X a representation $(\pi _{x},\mathcal{H}_{x})$ of $\mathcal{A}$ such that $\pi \mathop{\cong}\int _{X}^{\oplus }\pi _{x}\,d\mu (x)$ .

If π is nondegenerate and $\mathcal{H}$ is separable, then almost all the representations π _x are nondegenerate.

Proof.

The first part is [Dix64, Lemma 8.3.1] (see also [Ke78]). Suppose that π is nondegenerate and let $(E_{n})_{n\in \mathbb{N}}$ be an approximate identity on $\mathcal{A}$. Then π(E _n ) → 1 holds strongly in $\mathcal{H}$ and [Dix69, Ch. II, no. 2.3, Prop. 4] implies the existence of a subsequence $(n_{k})_{k\in \mathbb{N}}$ such that $\pi _{x}(E_{n_{k}}) \rightarrow \mathbf{1}$ holds strongly for almost every x ∈ X. For any such x, the representation π _x is non-degenerate. □ 

Theorem B.3.

Let $G = \lim _{\longrightarrow }\ G_{n}$ be a direct limit of separable locally compact groups with closed embeddings $G_{n}\hookrightarrow G_{n+1}$ and $(\pi,\mathcal{H})$ be a continuous separable unitary representation. For any maximal abelian subalgebra $\mathcal{A}\subseteq \pi (G)'$ , we then obtain a direct integral decomposition $\pi \mathop{\cong}\int _{X}^{\oplus }\pi _{x}\,d\mu (x)$ into continuous unitary representations of G in which $\mathcal{A}$ acts by multiplication operators.

Proof.

According to the classification of commutative W ^∗-algebras, we have $\mathcal{A}\mathop{\cong}L^{\infty }(X,\mu )$ for a localizable measure space $(X,\mathfrak{S},\mu )$ [Sa71, Prop. 1.18.1]. We therefore obtain a direct integral decomposition of the corresponding Hilbert space $\mathcal{H}$. To obtain a corresponding direct integral decomposition of the representation of G, we consider the C ^∗-algebra $\mathcal{B}$ generated by the subalgebras $\mathcal{B}_{n}$ which are generated by the image of the integrated representations $L^{1}(G_{n}) \rightarrow B(\mathcal{H})$. Then each $\mathcal{B}_{n}$ is separable and therefore $\mathcal{B}$ is also separable. Hence Theorem B.2 leads to nondegenerate representations $(\pi _{x},\mathcal{H}_{x})$ of $\mathcal{B}$ whose restriction to every $\mathcal{B}_{n}$ is nondegenerate.

In [SV75], the $\ell^{1}$-direct sum $\mathcal{L}:= \oplus _{n\in \mathbb{N}}^{1}L^{1}(G_{n})$ is used as a replacement for the group algebra. From the representation $\pi: \mathcal{L}\rightarrow \mathcal{B}$ we obtain a representation ρ _x of this Banach-∗-algebra whose restrictions to the subalgebras $L^{1}(G_{n})$ are non-degenerate. Now the argument in [SV75, p. 60] (see also Remark B.1 above) implies that the corresponding continuous unitary representations of the subgroups G _n combine to a continuous unitary representation $(\rho _{x},\mathcal{H}_{x})$ of G. □ 

Remark B.4.

Let $(\pi,\mathcal{H})$ be a continuous unitary representation of the direct limit $G = \lim _{\longrightarrow }\ G_{n}$ of locally compact groups. Let $\mathcal{A}_{n}:=\pi _{n}(C^{{\ast}}(G_{n}))$ and write $\mathcal{A}:=\langle \mathcal{A}_{n}: n \in \mathbb{N}\rangle _{C^{{\ast}}}$ for the C ^∗-algebra generated by the $\mathcal{A}_{n}$. Then $\mathcal{A}'' =\pi (G)''$ follows immediately from $\mathcal{A}_{n}'' =\pi _{n}(G_{n})''$ for each n.

From the nondegeneracy of the multiplier action of C ^∗(G _n ) on C ^∗(G _n+1) it follows that

$$\displaystyle{C^{{\ast}}(G_{ n})C^{{\ast}}(G_{ n+1}) = C^{{\ast}}(G_{ n+1}),}$$

which leads to

$$\displaystyle{\mathcal{A}_{n}\mathcal{A}_{n+1} = \mathcal{A}_{n+1}.}$$

We have a decreasing sequence of closed-∗-ideals

$$\displaystyle{\mathcal{I}_{n}:= \overline{\sum _{k\geq n}\mathcal{A}_{k}} \subseteq \mathcal{A}}$$

such that G _n acts continuously by multipliers on $\mathcal{I}_{n}$. A representation of $\mathcal{I}_{n}$ is non-degenerate if and only if its restriction to $\mathcal{A}_{n}$ is nondegenerate because $\mathcal{A}_{n}\mathcal{I}_{n} = \mathcal{I}_{n}$.

If a representation $(\rho,\mathcal{K})$ of $\mathcal{A}$ is nondegenerate on all these ideals, then it is non-degenerate on every $\mathcal{A}_{n}$, hence defines a continuous unitary representation of G.

Remark B.5.

Theorem B.3 implies in particular the validity of the disintegration arguments in [Ol78, Thm. 3.6] and [Pi90, Prop. 2,4]. In [Ol84, Lemma 2.6] one also finds a very brief argument concerning the disintegration of “holomorphic” representations, namely that all the constituents are again “holomorphic”. We think that this is not obvious and requires additional arguments.