The Density Theorem for Operator-Valued Frames via Square-Integrable Representations of Locally Compact Groups

Abstract

In this paper, we first prove a density theorem for operator-valued frames via square-integrable representations restricted to closed subgroups of locally compact groups, which is a natural extension of the density theorem in classical Gabor analysis. More precisely, it is proved that for such an operator-valued frame, the index subgroup is co-compact if and only if the generator is a Hilbert–Schmidt operator. Then we present some applications of this density theorem, and in particular establish necessary and sufficient conditions for the existence of such operator-valued frames with Hilbert–Schmidt generators. We also introduce the concept of wavelet transform for Hilbert–Schmidt operators, and use it to prove that if the representation space is infinite-dimensional, then the system indexed by the entire group is Bessel system but not a frame for the space of all Hilbert–Schmidt operators on the representation space.

1 Introduction

Gabor analysis is devoted to expanding complex signals as linear combinations of basic signals derived from a single window function by varying it in time and frequency over specific lattices. Classical harmonic analysis on locally compact abelian groups plays a crucial role in modern time–frequency analysis. Let G be a second countable, locally compact, abelian group, let ${\widehat{G}}$ be the Pontryagin dual group of G, and let $\pi$ be the Weyl-Heisenberg representation of $G\times {\widehat{G}}$ [5]. Jakobsen and Lemvig [12] showed the following density theorem for Gabor frames:

Theorem 1.1

Let $\Delta$ be a closed subgroup of $G\times {\widehat{G}}$ and let $g\in L^2(G)$. If the system of the form

$$\begin{aligned} \pi (\Delta )g:=\{\pi (\nu )g\}_{\nu \in \Delta } \end{aligned}$$

(1.1)

is a Gabor frame for $L^2(G)$ with bounds $0<\alpha \le \beta <\infty$, then the following statements hold:

• (i) The index subgroup $\Delta$ is co-compact.

• (ii) $\alpha {\textrm{vol}}(\Delta )\le \Vert g\Vert ^2\le \beta {\textrm{vol}}(\Delta )$, where the size of $\Delta$ is given by

  $$\begin{aligned} \textrm{vol}(\Delta ):=\mu _{(G\times {\widehat{G}})/\Delta }((G\times {\widehat{G}})/\Delta ). \end{aligned}$$

The more general setup is as in the following proposition, in which the essential orthogonality relation for the matrix coefficient functions of irreducible square-integrable projective unitary representations is the foundation for our deduction and this proposition will get repeated use in this paper.

Proposition 1.2

[16] Let $\pi$ be a $\sigma$-projective irreducible unitary representation of a unimodular locally compact group G on ${\mathcal {H}}_\pi$. Then the following are equivalent:

• (i) There exist nonzero vectors $\xi ,\eta \in {\mathcal {H}}_\pi$ such that $\int \limits _G |\langle \xi ,\pi (x)\eta \rangle |^2dx<\infty$.

• (ii) For every $\xi ,\eta \in {\mathcal {H}}_\pi$, we have that $\int \limits _G |\langle \xi ,\pi (x)\eta \rangle |^2dx<\infty$.

• (iii) $\pi$ is a sub-representation of the $\sigma$-twisted left regular representation of G.

If any of the above assumptions holds, then there exists a positive number ${\textrm{d}}_{\pi }>0$, called the formal dimension of $\pi$, such that

$$\begin{aligned} \int \limits _G\langle \xi ,\pi (x)\eta \rangle \langle \pi (x)\eta ',\xi '\rangle dx=\frac{\langle \xi ,\xi '\rangle \langle \eta ',\eta \rangle }{\mathrm{d_\pi }} \end{aligned}$$

for all $\xi ,\xi ',\eta ,\eta '\in {\mathcal {H}}_\pi$.

Representations satisfying the above equivalent conditions in Proposition 1.2 are called square-integrable. The formal dimension ${\textrm{d}}_{\pi }$ is related to the Haar measure on G. In some concrete settings, we can explicitly compute it; see [17, Section 9].

Throughout this paper, we will suppose that $(\pi ,{\mathcal {H}}_\pi )$ is a square-integrable representation of a unimodular locally compact group G with formal dimension $\mathrm{d_\pi }>0$ and $\Lambda$ is a closed subgroup of G. We point out that these groups may not be abelian and discrete. The research of spanning properties of $\Lambda$-indexed systems of the form

$$\begin{aligned} \pi (\Lambda )\eta :=\{\pi (\lambda )\eta \}_{\lambda \in \Lambda }, \end{aligned}$$

(1.2)

where $\eta \in {\mathcal {H}}_\pi$, is fundamental in some aspects of applied and computational harmonic analysis. It includes Gabor analysis, wavelet analysis and so on [1, 4, 5, 8, 12, 17]. Systems with some special structure is well worth being investigated because they have a strong influence on practical applications. The system of the form (1.1) is a special case of (1.2). Under suitable assumptions on G and $\pi$, many fundamental results, known as density theorems, give basic obstructions to the spanning properties of such systems which is closely related to density of $\Lambda$ in G.

Inspired by operator-valued frames [13] and frames of the form (1.2), in this paper we mainly concentrate on the study of $\Lambda$-indexed operator-valued systems of the form

$$\begin{aligned} A\pi (\Lambda ):=\{A\pi (\lambda )\}_{\lambda \in \Lambda }, \end{aligned}$$

(1.3)

where A is a bounded linear operator on ${\mathcal {H}}_\pi$. Based on Theorem 1.1, we consider the following density problem for operator-valued frames of the form (1.3):

Problem 1.3

Let $(\pi ,{\mathcal {H}}_\pi )$ be a square-integrable representation of G with formal dimension ${\textrm{d}}_{\pi }>0$, let $\Lambda$ be a closed subgroup of G and let A be a bound linear operator on ${\mathcal {H}}_\pi$. Suppose that $A\pi (\Lambda )$ is an operator-valued frame on ${\mathcal {H}}_\pi$ with bounds $0<\alpha \le \beta <\infty$. Under what condition is the index subgroup $\Lambda$ co-compact? In this case, what can we say about the density of $\Lambda$? Do we have

$$\begin{aligned} \alpha \textrm{vol}(\Lambda )\le \frac{\Vert A\Vert ^2}{\textrm{d}}_{\pi }\le \beta \textrm{vol}(\Lambda ) \end{aligned}$$

for some appropriate norm ||A|| of the generator A?

To the best of our knowledge, the problem in this general setting has not been considered yet in the literature. The main purpose of the paper is to address the answers to this problem. We remark the investigation of density theorems is essential in Gabor analysis; see [10] for a history. If G, $\Lambda$, $\pi$ and A are replaced by $G\times {\widehat{G}}$, a closed subgroup of $G\times {\widehat{G}}$, the Weyl-Heisenberg representation and a rank-one operator on $L^2(G)$, respectively, then this problem reduces to Theorem 1.1.

This paper is organized as follows: In Sect. 2, we collect some concepts, notations and properties on Hilbert–Schmidt operators on Hilbert spaces, harmonic analysis on locally compact groups and frame theory. In Sect. 3, we prove a density theorem that says that for an operator-valued frame $A\pi (\Lambda )$ with bounds $0<\alpha \le \beta <\infty$, the index subgroup $\Lambda$ is co-compact if and only if the generator A is a Hilbert–Schmidt operator on ${\mathcal {H}}_\pi$, which is a main theorem in this paper. In this case, we obtain the inequalities

$$\begin{aligned} \alpha \textrm{vol}(\Lambda )\le \frac{\Vert A\Vert _2^2}{\textrm{d}}_{\pi }\le \beta \textrm{vol}(\Lambda ), \end{aligned}$$

where $\Vert A\Vert _2$ is the Hilbert–Schmidt norm of A. In addition, the generator of an operator-valued Bessel system indexed by a closed co-compact subgroup is necessarily a Hilbert–Schmidt operator. Moreover, some applications of this theorem are given in this section. Especially, we establish necessary and sufficient conditions for the existence of a Hilbert–Schmidt operator A on ${\mathcal {H}}_\pi$ such that $A\pi (\Lambda )$ is an operator-valued frame. Finally, in Sect. 4, the concept of the wavelet transform for Hilbert–Schmidt operators is introduced to prove that the system (1.3) indexed by the entire group must be Bessel, but not a frame for the space of all Hilbert–Schmidt operators on ${\mathcal {H}}_\pi$.

2 Preliminaries

2.1 Hilbert–Schmidt Operators on Hilbert Spaces

Let $\{\xi _i\}_{i\in {\mathbb {I}}}$ be an orthonormal basis for a separable Hilbert space ${\mathcal {H}}$, where ${\mathbb {I}}$ is a finite or countable set. Denote by ${\mathcal {B}}({\mathcal {H}})$ the space of all bound linear operators on ${\mathcal {H}}$. Let $A\in {\mathcal {B}}({\mathcal {H}})$. Then the operator A is called a Hilbert–Schmidt operator on ${\mathcal {H}}$ if

$$\begin{aligned} \Vert A\Vert _2:=\left( \sum \limits _i\Vert A\xi _i\Vert ^2\right) ^{\frac{1}{2}}<\infty . \end{aligned}$$

Also, we have that $\Vert A^*\Vert _2=\Vert A\Vert _2$, where $A^*$ is the adjoint operator of A. We denote the class of all Hilbert–Schmidt operators on ${\mathcal {H}}$ by ${\mathcal {S}}_2({\mathcal {H}})$.

It is well known that ${\mathcal {S}}_2({\mathcal {H}})$ is a Hilbert space with the inner product

$$\begin{aligned} \langle A,B\rangle _2:=\textrm{tr}(B^*A) \end{aligned}$$

for all $A,B\in {\mathcal {S}}_2({\mathcal {H}})$, where $\textrm{tr}(\cdot )$ is the usual trace function. Moreover, ${\mathcal {S}}_2({\mathcal {H}})$ is a two-side ideal containing the set ${\mathcal {F}}({\mathcal {H}})$ of all finite-rank operators in ${\mathcal {B}}({\mathcal {H}})$. In addition, $\Vert A\Vert \le \Vert A\Vert _2$ and $\textrm{tr}(AB)=\textrm{tr}(BA)$ hold for all $A,B\in {\mathcal {S}}_2({\mathcal {H}})$. Further, $\Vert AB\Vert _2\le \Vert A\Vert \Vert B\Vert _2$ holds for all $A\in {\mathcal {B}}({\mathcal {H}})$ and $B\in {\mathcal {S}}_2({\mathcal {H}})$.

Fix $\eta ,\xi \in {\mathcal {H}}$. We can define the operator $\eta \otimes \xi$ on ${\mathcal {H}}$ as follows:

$$\begin{aligned} (\eta \otimes \xi )\rho =\langle \rho ,\xi \rangle \eta \end{aligned}$$

for all $\rho \in {\mathcal {H}}$. If $\eta ,\eta _1,\xi ,\xi _1\in {\mathcal {H}}$ and $A,B\in {\mathcal {B}}({\mathcal {H}})$, then the following equalities hold:

$$\begin{aligned} (\eta \otimes \eta _1)(\xi \otimes \xi _1)&=\langle \xi ,\eta _1\rangle (\eta \otimes \xi _1),\\ (\eta \otimes \xi )^*&=\xi \otimes \eta ,\\ A(\eta \otimes \xi )B&=(A\eta )\otimes (B^*\xi ). \end{aligned}$$

We can refer to [14] for more information about Hilbert–Schmidt operators.

2.2 Harmonic Analysis on Locally Compact Groups

In this subsection, we collect some elementary notions and properties about locally compact groups.

A 2-cocycle (or a multiplier) on G is a function $\sigma :G\times G \rightarrow {\mathbb {T}}$ satisfying

$$\begin{aligned} \sigma (x,y)\sigma (xy,z)=\sigma (x,yz)\sigma (y,z) \end{aligned}$$

for all $x,y,z\in G$ and $\sigma (e,e)=1$, where ${\mathbb {T}}:=\{z\in {\mathbb {C}}:|z|=1\}$ and e is the identity of G. Denote by $Z^2(G,{\mathbb {T}})$ the set of all 2-cocycles on G. Fix $x\in G$. We say that the element x is $\sigma$-regular if $\sigma (x,y)=\sigma (y,x)$ whenever y commutes with x. If x is $\sigma$-regular, then every element in the conjugacy class $C_x:=\{y^{-1}xy:y\in G\}$ of x is also $\sigma$-regular. So it makes sense to consider $\sigma$-regular conjugacy classes. We say that the pair $(G,\sigma )$ satisfies Kleppner’s condition if the only finite $\sigma$-regular conjugacy class is the trivial one. By a $\sigma$-projective unitary representation of G, we mean a strongly continuous mapping $\pi :G\rightarrow {\mathcal {U}}({\mathcal {H}}_\pi )$ into the group of all unitary operators on a separable Hilbert space ${\mathcal {H}}_\pi$ satisfying

$$\begin{aligned} \pi (x)\pi (y)=\sigma (x,y)\pi (xy) \end{aligned}$$

for all $x,y\in G$. The Hilbert space ${\mathcal {H}}_\pi$ is called the representation space of $\pi$. We will always write the pair $(\pi ,{\mathcal {H}}_\pi )$ instead of $\pi :G\rightarrow {\mathcal {U}}({\mathcal {H}}_\pi )$.

Let $\Lambda$ be a closed subgroup of G. Then $\Lambda$ and $G/\Lambda$ are also locally compact groups. There exists a non-zero positive regular Borel measure $\mu _G$ on G which is left-translation invariant. That is, $\mu _G(xB)=\mu _G(B)$ holds for any element $x\in G$ and any $B\in {\mathcal {B}}_G$, where ${\mathcal {B}}_G$ denotes the $\sigma$-algebra of Borel sets of G. This measure $\mu _G$ is called a left Haar measure on G [9]. Moreover, the integral over G is left-translation invariant in the following sense that

$$\begin{aligned} \int \limits _{G}f\big (y^{-1}x\big )d\mu _G(x)=\int \limits _{G}f(x)d\mu _G(x) \end{aligned}$$

for all $f\in C_c(G)$ and all $y\in G$, where $C_c(G)$ denotes the set of all continuous functions on G having compact support [6]. From this measure $\mu _G$, we can define the space $L^p(G),1\le p<\infty$ to be the space of equivalence classes of measurable functions f on G, modulo equality a.e., with the property that

$$\begin{aligned} \int \limits _G|f(x)|^pd\mu _G(x)<\infty . \end{aligned}$$

This Haar measure on G is unique up to multiplication with a positive constant. More precisely, there is a continuous homomorphism $\Delta _G:G\rightarrow (0,\infty )$ with respect to multiplication on $(0,\infty )$, called the modular function, such that

$$\begin{aligned} \int \limits _Gf(xy)d\mu _G(x)=\Delta _G\big (y^{-1}\big )\int \limits _Gf(x)d\mu _G(x) \end{aligned}$$

and

$$\begin{aligned} \int \limits _Gf\big (x^{-1}\big )d\mu _G(x)=\int \limits _Gf(x)\Delta _G\big (x^{-1}\big )d\mu _G(x) \end{aligned}$$

for any $f\in L^1(G,\mu _G)$ and any $y\in G$. This modular function is independent of the choice of Haar measures on G. If $\Delta _G\equiv 1$, then G is called unimodular. There are many unimodular locally compact groups. For example, ${\mathbb {R}}^d$, discrete groups, the general linear group $\textrm{GL}(d,{\mathbb {R}})$ of all $d\times d$ invertible matrices with real entries and connected nilpotent Lie groups. The subgroup $\Lambda$ also has a modular function $\Delta _\Lambda$ and a left Haar measure $\mu _\Lambda$, whose scale we also fix.

The relation between the Haar measures on the three groups can be given as follows: Once two out of three Haar measures have been chosen on $G,\Lambda$ and $G/\Lambda$, the last one can be chosen so that Weil’s formula:

$$\begin{aligned} \int \limits _Gf(x)d\mu _G(x)=\int \limits _{G/\Lambda }\int \limits _\Lambda f(xy)d\mu _\Lambda (y)d\mu _{G/\Lambda }(x\Lambda ) \end{aligned}$$

for all $f\in L^1(G)$; see [11, Section 3]. The subgroup $\Lambda$ is called co-compact if the quotient group $G/\Lambda$ is compact under the quotient topology. The volume of $\Lambda$ is defined by

$$\begin{aligned} \textrm{vol}(\Lambda ):=\mu _{G/\Lambda }(G/\Lambda ). \end{aligned}$$

Note that this volume $\textrm{vol}(\Lambda )$ depend on the Haar measure $\mu _G$. We also know that $\Lambda$ is co-compact if and only if $\textrm{vol}(\Lambda )<\infty$. The subgroup $\Lambda$ is called a uniform lattice if $\Lambda$ is a discrete co-compact subgroup of G.

A classical example of square-integrable representations is the Weyl-Heisenberg representation. Let G be a second countable, locally compact, abelian group. For any $\nu =(\lambda ,\gamma )\in G\times {\widehat{G}}$. Let $\pi (\nu )$ denote the time–frequency shift operator $E_\gamma T_\lambda$, where the translation operator $T_\lambda ,\;\;\lambda \in G$ as follows:

$$\begin{aligned} T_\lambda :L^2(G)\rightarrow L^2(G),\;\;(T_\lambda f)(x)=f\big (x\lambda ^{-1}\big ) \end{aligned}$$

and the modulation operator $E_\gamma ,\;\;\gamma \in {\widehat{G}}$ as follows:

$$\begin{aligned} E_\gamma :L^2(G)\rightarrow L^2(G),\;\;(E_\gamma f)(x)=\gamma (x)f(x) \end{aligned}$$

for all $f\in L^2(G)$ and all $x\in G$. It is easy to verify that each $\pi (\nu )$ is a unitary operator on $L^2(G)$. We use the essential equality commutator relation

$$\begin{aligned} \gamma (\lambda )T_\lambda E_\gamma =E_\gamma T_\lambda \end{aligned}$$

to obtain the following useful identities:

$$\begin{aligned} \pi (\nu _1)\pi (\nu _2)=\overline{\gamma _2(\lambda _1)}\pi (\nu _1\nu _2) \end{aligned}$$

for all $\nu _i=(\lambda _i,\gamma _i),i=1,2$. So the Weyl-Heisenberg representation $\pi$ [5] is a projective unitary representation with the 2-cocycle $\sigma \in Z^2(G\times {\widehat{G}},{\mathbb {T}})$ given by

$$\begin{aligned} \sigma (\nu _1,\nu _2)=\overline{\gamma _2(\lambda _1)},\;\;\nu _i=(\lambda _i,\gamma _i),i=1,2. \end{aligned}$$

By [2, Theorem 1.3] and [12, Lemma 4.2], this representation is square-integrable irreducible and ${\textrm{d}}_\pi =1$. Moreover, $(G\times {\widehat{G}},\sigma )$ satisfies Kleppner’s condition [5, Section 6.2].

For more information about harmonic analysis on locally compact groups, we refer the readers to the books [3, 9].

2.3 Frames

In this subsection, we recall some basic concepts about frames.

Definition 2.1

(Frames [12]) Let ${\mathcal {H}}$ be a complex Hilbert space and let $(\Omega ,\Sigma ,\mu )$ be a measure space. A family of $\{f_k\}_{k\in \Omega }$ is a frame for ${\mathcal {H}}$ with respect to $(\Omega ,\Sigma ,\mu )$ with bounds $0<\alpha \le \beta <\infty$ if the following statements hold:

• (i) The mapping $\Omega \rightarrow {\mathcal {H}},k\mapsto f_k$ is weakly measurable, i.e., for all $f\in {\mathcal {H}}$, the mapping $\Omega \rightarrow {\mathbb {C}},k\mapsto \langle f,f_k\rangle$ is measurable.

• (ii) The inequalities

  $$\begin{aligned} \alpha \Vert f\Vert ^2\le \int \limits _{\Omega }|\langle f,f_k \rangle |^2d\mu (k)\le \beta \Vert f\Vert ^2 \end{aligned}$$

  hold for all $f\in {\mathcal {H}}$.

Let $(\pi ,{\mathcal {H}}_\pi )$ be a square-integrable representation of a unimodular locally compact group G and $\Lambda$ is a closed subgroup of G. We say that the pair $(\pi ,\Lambda )$ admits a frame if there exists $\eta \in {\mathcal {H}}_\pi$ such that $\pi (\Lambda )\eta$ is a frame for ${\mathcal {H}}_\pi$. That is, there exist constants $0<\alpha \le \beta <\infty$ such that

$$\begin{aligned} \alpha \Vert \xi \Vert ^2\le \int \limits _\Lambda |\langle \xi ,\pi (\lambda )\eta \rangle |^2d\lambda \le \beta \Vert \xi \Vert ^2 \end{aligned}$$

for all $\xi \in {\mathcal {H}}_\pi$. In this case, this vector $\eta$ is called the generator of the system $\pi (\Lambda )\eta$.

Motivated by the concept of operator-valued frames in Hilbert spaces proposed by Kaftal et al. in [13], we introduce the notion of operator-valued frames with special structure.

Definition 2.2

(Operator-valued frames with special structure) Let $(\pi ,{\mathcal {H}}_\pi )$ be a square-integrable representation of G with formal dimension ${\textrm{d}}_{\pi }>0$, let $\Lambda$ be a closed subgroup of G and let A be a bounded linear operator on ${\mathcal {H}}_\pi$. The system $A\pi (\Lambda )$ is called an operator-valued frame on ${\mathcal {H}}_\pi$ with bounds $0<\alpha \le \beta <\infty$ if this system satisfies

$$\begin{aligned} \alpha \Vert \xi \Vert ^2\le \int \limits _{\Lambda }\Vert A\pi (\lambda )\xi \Vert ^2 d\lambda \le \beta \Vert \xi \Vert ^2 \end{aligned}$$

for all $\xi \in {\mathcal {H}}_\pi$.

If one can choose $\alpha =\beta$ in the above inequalities, then $A\pi (\Lambda )$ is called tight. The system $A\pi (\Lambda )$ is called Parseval if $\alpha =\beta =1$. If $A\pi (\Lambda )$ satisfies the upper bound inequality, then it is called Bessel. In this case, this operator A is called the generator of the system $A\pi (\Lambda )$.

We say that $(\pi ,\Lambda )$ admits an operator-valued frame on ${\mathcal {H}}_\pi$ if there exists an operator $A\in {\mathcal {B}} ({\mathcal {H}}_\pi )$ such that $A\pi (\Lambda )$ is an operator-valued frame. In particular, if $A\in {\mathcal {F}}({\mathcal {H}}_\pi )$, then we say that $(\pi ,\Lambda )$ admits a FR-operator-valued frame. if $A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$, then we say that $(\pi ,\Lambda )$ admits a HS-operator-valued frame.

3 A Density Theorem and Its Applications

In this section, we first provide a density theorem for operator-valued frames of the form (1.3), which is a main theorem in this paper. From this, we also give a partial answer to Problem 1.3 in the introduction. This theorem suggests that when we deal with such operator-valued frames, the co-compactness of the index subgroup is closely related to the special property of the generator of this system. Hence if we want to obtain properties of such operator-valued frames indexed by closed co-compact subgroups, then this special property of the generators brings the great benefit to our proof and deduction. So we will investigate properties of such operator-valued frames indexed by closed co-compact subgroups. Moreover, applications of this density theorem are listed later.

3.1 A Density Theorem for Operator-Valued Frames

For the proof of our main theorem, we will use famous Weil’s identity in harmonic analysis on locally compact groups and the orthogonality relation for square-integrable representations.

Theorem 3.1

Let $(\pi ,{\mathcal {H}}_\pi )$ be a square-integrable representation of G with formal dimension ${\textrm{d}}_{\pi }>0$, let $\Lambda$ be a closed subgroup of G and let A be a bounded linear operator on ${\mathcal {H}}_\pi$. If $A\pi (\Lambda )$ is an operator-valued frame with bounds $0<\alpha \le \beta <\infty$, then the following conditions are equivalent:

• (i) The generator A is a Hilbert–Schmidt operator on ${\mathcal {H}}_\pi$, i.e., $A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$.

• (ii) The index subgroup $\Lambda$ is co-compact, i.e., $\textrm{vol}(\Lambda )<\infty$.

If any of the above conditions holds, then

$$\begin{aligned} \alpha \textrm{vol}(\Lambda )\le \frac{\Vert A\Vert _2^2}{\mathrm{d_\pi }}\le \beta \textrm{vol}(\Lambda ). \end{aligned}$$

In particular, the generator of an operator-valued Bessel system indexed by a closed co-compact subgroup is necessarily a Hilbert–Schmidt operator.

Proof

Suppose that $A\pi (\Lambda )$ is an operator-valued frame with bounds $0<\alpha \le \beta <\infty$. Since ${\mathcal {H}}_\pi$ is separable, we can assume that $\{\xi _i\}_{i\in {\mathbb {I}}}$ is an orthonormal basis for ${\mathcal {H}}_\pi$, where ${\mathbb {I}}$ is a finite or countable set.

(i)$\Rightarrow$(ii): If the generator A is a Hilbert–Schmidt operator on ${\mathcal {H}}_\pi$, then $\Vert A\Vert _2<\infty$ and so $\Vert A\Vert _2^2=\sum \limits _i\Vert A^*\xi _i\Vert ^2$. Fix $\xi \in {\mathcal {H}}_\pi$ and $i\in {\mathbb {I}}$. By Proposition 1.2, we see that

$$\begin{aligned} \int \limits _G|\langle \xi ,\pi (x)A^*\xi _i\rangle |^2dx=\frac{\Vert A^*\xi _i\Vert ^2\Vert \xi \Vert ^2}{\mathrm{d_\pi }}<\infty , \end{aligned}$$

which implies that the function $x\mapsto |\langle \xi ,\pi (x)A^*\xi _i\rangle |^2$ lies in $L^1(G)$. By Weil’s identity, we obtain that

$$\begin{aligned} \int \limits _G|\langle \xi ,\pi (x)A^*\xi _i\rangle |^2dx=\int \limits _{G/\Lambda } \int \limits _\Lambda |\langle \xi ,\pi (x\lambda )A^*\xi _i\rangle |^2d\lambda d(x\Lambda ). \end{aligned}$$

We also have that

$$\begin{aligned} \int \limits _{G/\Lambda }\int \limits _\Lambda |\langle \xi ,\pi (x\lambda )A^*\xi _i\rangle |^2d\lambda d(x\Lambda )=\int \limits _{G/\Lambda }\int \limits _\Lambda |\langle \pi (\lambda ^{-1})\pi (x)^* \xi ,A^*\xi _i\rangle |^2d\lambda d(x\Lambda ). \end{aligned}$$

Since $\Lambda$ is a unimodular locally compact group, we get that

$$\begin{aligned} \int \limits _\Lambda |\langle \pi (\lambda ^{-1})\pi (x)^*\xi ,A^*\xi _i\rangle |^2d\lambda =\int \limits _\Lambda |\langle \pi (\lambda )\pi (x)^*\xi ,A^*\xi _i\rangle |^2d\lambda \end{aligned}$$

for all $x\in G$. By Tonelli’s Theorem, we know that

$$\begin{aligned} \sum \limits _i\int \limits _{G/\Lambda }\int \limits _{\Lambda }|\langle A\pi (\lambda ) \pi (x)^*\xi ,\xi _i\rangle |^2d\lambda d(x\Lambda )=\int \limits _{G/\Lambda } \int \limits _{\Lambda }\Vert A\pi (\lambda )\pi (x)^*\xi \Vert ^2d\lambda d(x\Lambda ). \end{aligned}$$

Combining the above formulas, we conclude that

$$\begin{aligned} \frac{\Vert A\Vert _2^2\Vert \xi \Vert ^2}{{\textrm{d}}_{\pi }}&=\int \limits _{G/\Lambda }\int \limits _{\Lambda } \Vert A\pi (\lambda )\pi (x)^*\xi \Vert ^2 d\lambda d(x\Lambda )\\&\ge \int \limits _{G/\Lambda }\alpha \Vert \pi (x)^*\xi \Vert ^2d(x\Lambda ) =\int \limits _{G/\Lambda }\alpha \Vert \xi \Vert ^2d(x\Lambda )\\&=\alpha \Vert \xi \Vert ^2\int \limits _{G/\Lambda }d(x\Lambda )=\alpha \Vert \xi \Vert ^2\textrm{vol}(\Lambda ), \end{aligned}$$

where we use the condition that the system $A\pi (\Lambda )$ satisfies the lower frame bound inequality. Thus

$$\begin{aligned} \textrm{vol}(\Lambda )\le \frac{\Vert A\Vert _2^2}{\alpha {\textrm{d}}_{\pi }}<\infty , \end{aligned}$$

which shows that $\Lambda$ is co-compact.

(ii)$\Rightarrow$(i): If the index subgroup $\Lambda$ is co-compact, then $\textrm{vol}(\Lambda )<\infty$. Fix $\xi \in {\mathcal {H}}_\pi$. Similar to the above proof, using the upper frame bound inequality for the system $A\pi (\Lambda )$ yields that

$$\begin{aligned} \sum \limits _i\frac{\Vert A^*\xi _i\Vert ^2\Vert \xi \Vert ^2}{\mathrm{d_\pi }}&=\int \limits _{G/\Lambda }\int \limits _{\Lambda }\Vert A\pi (\lambda )\pi (x)^*\xi \Vert ^2 d\lambda d(x\Lambda )\\&\le \beta \Vert \xi \Vert ^2\textrm{vol}(\Lambda ). \end{aligned}$$

Hence

$$\begin{aligned} \sum \limits _i\Vert A^*\xi _i\Vert ^2\le \beta {\textrm{d}}_{\pi }\textrm{vol}(\Lambda )<\infty . \end{aligned}$$

That is, $A^*\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$ and $\Vert A^*\Vert _2^2\le \beta {\textrm{d}}_{\pi }\textrm{vol}(\Lambda )$. Thus $A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$ and

$$\begin{aligned} \frac{\Vert A\Vert _2^2}{{\textrm{d}}_{\pi }}\le \beta \textrm{vol}(\Lambda ). \end{aligned}$$

This completes the proof. $\square$

Remark 3.2

By Theorem 3.1, the answer to Problem 1.3 (i) is sure if and only if A is necessarily a Hilbert–Schmidt operator on ${\mathcal {H}}_\pi$. Since $\Vert A\Vert \le \Vert A\Vert _2$, we see that

$$\begin{aligned} \frac{\Vert A\Vert ^2}{{\textrm{d}}_{\pi }}\le \beta \textrm{vol}(\Lambda ), \end{aligned}$$

which is the right hand inequality in Problem 1.3 (ii). However, we do not obtain the left hand inequality.

As a special case, we also obtain the classical density theorem for systems of the form (1.2) via square-integrable representations restricted to closed subgroups. The following density theorem generalizes [12, Theorem 5.1] and [17, Proposition 7.2].

Theorem 3.3

Let $\Lambda$ be a closed subgroup of G and let $\eta \in {\mathcal {H}}_\pi$. If $\pi (\Lambda )\eta$ is a frame for ${\mathcal {H}}_\pi$ with bounds $0<\alpha \le \beta <\infty$, then the following statements hold:

• (i) The index subgroup $\Lambda$ is co-compact, i.e., $\textrm{vol}(\Lambda )<\infty .$

• (ii) $\alpha \textrm{vol}(\Lambda )\le \frac{\Vert \eta \Vert ^2}{\textrm{d}}_{\pi }\le \beta \textrm{vol}(\Lambda )$.

Proof

We assume that $\pi (\Lambda )\eta$ is a frame for ${\mathcal {H}}_\pi$ with bounds $0<\alpha \le \beta <\infty$. Fix $\eta _0$ in ${\mathcal {H}}_\pi$ with $\Vert \eta _0\Vert =1$. Put

$$\begin{aligned} A:=\eta _0\otimes \eta . \end{aligned}$$

Then $A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$ and $\Vert A\Vert _2=\Vert \eta \Vert$. For any $\xi \in {\mathcal {H}}_\pi$, we compute that

$$\begin{aligned} \int \limits _\Lambda \Vert A\pi (\lambda )\xi \Vert ^2d\lambda =\int \limits _\Lambda |\langle \xi ,\pi (\lambda )\eta \rangle |^2d\lambda . \end{aligned}$$

Thus $A\pi (\Lambda )$ is a HS-operator-valued frame with same bounds $0<\alpha \le \beta <\infty$. Now this theorem follows from Theorem 3.1. $\square$

Based on operator-valued frames [13] and ordinary Gabor frames of the form (1.1), we introduce the notation of operator-valued Gabor frames on locally compact abelian groups. Let $\Delta$ be a closed subgroup of the phase space $G\times {\widehat{G}}$, let A be a bounded linear operator on $L^2(G)$ and let $\pi$ be the Weyl-Heisenberg representation of $G\times {\widehat{G}}$, where G is a second countable, locally compact, abelian group and ${\widehat{G}}$ is the Pontryagin dual group of G. We say that the system $A\pi (\Delta )$ is an operator-valued Gabor frame on $L^2(G)$ with bounds $0<\alpha \le \beta <\infty$ if it satisfies

$$\begin{aligned} \alpha \Vert f\Vert ^2\le \int \limits _{\Delta }\Vert A\pi (\nu )f\Vert ^2 d\nu \le \beta \Vert f\Vert ^2 \end{aligned}$$

for all $f\in L^2(G)$. If $A\pi (\Delta )$ satisfies the upper bound inequality, then it is called Bessel. In this case, this operator A is called the generator of the system $A\pi (\Delta )$. Since this representation $\pi$ is a square-integrable representation with formal dimension $\textrm{d}_\pi =1$, we can obtain the following density theorem for operator-valued Gabor frames, which is a generalization of [12, Theorem 5.1].

Theorem 3.4

If $A\pi (\Delta )$ is an operator-valued Gabor frame with bounds $0<\alpha \le \beta <\infty$, then the following conditions are equivalent:

• (i) The generator A is a Hilbert–Schmidt operator on $L^2(G)$, i.e., $A\in {\mathcal {S}}_2(L^2(G))$.

• (ii) The index subgroup $\Delta$ is co-compact, i.e., $\textrm{vol}(\Delta )<\infty$.

If any of the above conditions holds, then

$$\begin{aligned} \alpha \textrm{vol}(\Delta )\le \Vert A\Vert _2^2\le \beta \textrm{vol}(\Delta ). \end{aligned}$$

In particular, the generator of an operator-valued Bessel system indexed by a closed co-compact subgroup of the phase space is necessarily a Hilbert–Schmidt operator.

3.2 Analysis Operators and Synthesis Operators

One can define $L^2(\Lambda ,{\mathcal {H}}_\pi )$ to be the space of equivalence classes of strongly measurable functions $\Psi :\Lambda \rightarrow {\mathcal {H}}_\pi$ (in the sense that the scalar-valued function $\lambda \mapsto \Vert \Psi (\lambda )\Vert$ is measurable), modulo equality a.e., with the property that

$$\begin{aligned} \int \limits _\Lambda \Vert \Psi (\lambda )\Vert ^2d\lambda <\infty . \end{aligned}$$

Then it becomes a Hilbert space with the inner product

$$\begin{aligned} \langle \Psi _1,\Psi _2\rangle =\int \limits _\Lambda \langle \Psi _1(\lambda ),\Psi _2(\lambda )\rangle d\lambda \end{aligned}$$

for all $\Psi _1,\Psi _2\in L^2(\Lambda ,{\mathcal {H}}_\pi )$. Let $A\pi (\Lambda )$ be an operator-valued Bessel system on ${\mathcal {H}}_\pi$. We can define the operator $\Theta _A$ associated to $A\pi (\Lambda )$ as follows:

$$\begin{aligned} \Theta _A:{\mathcal {H}}_\pi \rightarrow L^2(\Lambda ,{\mathcal {H}}_\pi ),\;\;\Theta _A\xi (\lambda )=A\pi (\lambda )\xi \end{aligned}$$

for all $\xi \in {\mathcal {H}}_\pi$ and all $\lambda \in \Lambda$. This operator $\Theta _A$ is called the analysis operator of the system $A\pi (\Lambda )$. It is easy to show that $A\pi (\Lambda )$ is a Bessel system with a bound $\beta >0$ if and only if $\Theta _A$ is a bounded linear operator with a bound $\beta ^{\frac{1}{2}}$. The adjoint operator of $\Theta _A$,

$$\begin{aligned} \Theta _A^*:L^2(\Lambda ,{\mathcal {H}}_\pi )\rightarrow {\mathcal {H}}_\pi ,\;\;\Psi \mapsto \Theta _A^*\Psi \end{aligned}$$

satisfying

$$\begin{aligned} \langle \Theta _A^*\Psi ,\xi \rangle =\int \limits _\Lambda \langle \pi (\lambda )^*A^*\Psi (\lambda ),\xi \rangle d\lambda \end{aligned}$$

for all $\xi \in {\mathcal {H}}_\pi$, is called the synthesis operator of $A\pi (\Lambda )$. If $A\pi (\Lambda )$ and $B\pi (\Lambda )$ are two operator-valued Bessel systems on ${\mathcal {H}}_\pi$, where $A,B\in {\mathcal {B}}({\mathcal {H}}_\pi )$, then one can define the (mixed) frame operator for them in the weak sense by

$$\begin{aligned} \langle S_{A,B}\xi ,\eta \rangle =\int \limits _\Lambda \langle A\pi (\lambda )\xi ,B\pi (\lambda )\eta \rangle d\lambda \end{aligned}$$

for all $\xi ,\eta \in {\mathcal {H}}_\pi$. Then $S_{A,B}=\Theta _B^*\Theta _A$. If their generators are same, i.e., $A=B$, then we write the frame operator S. We say that operator-valued Bessel systems $A\pi (\Lambda )$ and $B\pi (\Lambda )$ are dual if they satisfy

$$\begin{aligned} \int \limits _\Lambda \langle A\pi (x)\xi ,B\pi (x)\eta \rangle dx=\langle \xi ,\eta \rangle \end{aligned}$$

for all $\xi ,\eta \in {\mathcal {H}}_\pi$. That is, the frame operator for them satisfies the equality $S_{A,B}=I$. We also say that operator-valued Bessel systems $A\pi (\Lambda )$ and $B\pi (\Lambda )$ are HS-dual if they satisfy $A,B\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$ and $S_{A,B}=I$.

Lemma 3.5

Let $\Lambda$ be a unimodular closed subgroup of G and let $A,B\in {\mathcal {B}}({\mathcal {H}}_\pi )$. If $A\pi (\Lambda )$ and $B\pi (\Lambda )$ are two operator-valued Bessel systems on ${\mathcal {H}}_\pi$, then

• (i) $S_{A,B}\pi (\lambda )=\pi (\lambda )S_{A,B},\;\;for\;\;all\;\;\lambda \in \Lambda$.

• (ii) If $A\pi (\Lambda )$ is an operator-valued frame, then

  $$\begin{aligned} S^{-1}\pi (\lambda )=\pi (\lambda )S^{-1},\;\; S^{-\frac{1}{2}}\pi (\lambda )=\pi (\lambda )S^{-\frac{1}{2}} \end{aligned}$$

  for all $\lambda \in \Lambda$.

Proof

(i) Fix $\kappa \in \Lambda$. For any $\xi ,\eta \in {\mathcal {H}}_\pi$, we can get

$$\begin{aligned} \langle S_{A,B}\pi (\kappa )\xi ,\eta \rangle&=\int \limits _\Lambda \langle A\pi (\lambda )\pi (\kappa )\xi ,B\pi (\lambda )\eta \rangle d\lambda \\&=\int \limits _\Lambda \langle A\pi (\iota {\kappa }^{-1})\pi (\kappa )\xi ,B\pi (\iota {\kappa }^{-1})\pi (\kappa )\pi (\kappa )^*\eta \rangle d\iota \\&=\int \limits _\Lambda \langle A\pi (\iota )\xi ,B\pi (\iota )\pi (\kappa )^*\eta \rangle d\iota \\&=\langle S_{A,B}\xi ,\pi (\kappa )^{*}\eta \rangle \\&=\langle \pi (\kappa )S_{A,B}\xi ,\eta \rangle , \end{aligned}$$

where we use the fact that the Haar measure on $\Lambda$ is transition invariant in the second identity and $\pi$ is a $\sigma$-projective unitary representation in the third identity. Hence

$$\begin{aligned} S_{A,B}\pi (\kappa )=\pi (\kappa )S_{A,B}. \end{aligned}$$

(ii) We suppose that $A\pi (\Lambda )$ is an operator-valued frame. Then S is an invertible operator on ${\mathcal {H}}_\pi$. Fix $\lambda \in \Lambda$. By (i), we obtain that $S\pi (\lambda )=\pi (\lambda )S$ and so

$$\begin{aligned} S^{-\frac{1}{2}}\pi (\lambda )=\pi (\lambda )S^{-\frac{1}{2}}. \end{aligned}$$

$\square$

Remark 3.6

If $A\pi (\Lambda )$ is a HS-operator-valued frame indexed by a unimodular closed subgroup $\Lambda$ of G, then $A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$ and $AS^{-1},AS^{-\frac{1}{2}}\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$ since ${\mathcal {S}}_2({\mathcal {H}}_\pi )$ is a two-side ideal in ${\mathcal {B}}({\mathcal {H}}_\pi )$. For any $\lambda \in \Lambda$, by Lemma 3.5, we have that $A\pi (\lambda )S^{-1}=AS^{-1}\pi (\lambda )$ and $A\pi (\lambda )S^{-\frac{1}{2}}=AS^{-\frac{1}{2}}\pi (\lambda )$. Thus $AS^{-1}\pi (\Lambda )$ is the canonical HS-dual of $A\pi (\Lambda )$. Moreover, $AS^{-\frac{1}{2}}\pi (\Lambda )$ is a Parseval HS-operator-valued frame. By Theorem 3.1, we see that

$$\begin{aligned} \Big \Vert AS^{-\frac{1}{2}}\Big \Vert _2^2={\textrm{d}}_{\pi }\textrm{vol}(\Lambda ). \end{aligned}$$

3.3 Applications of the Density Theorem

We study the properties of the operator-valued frames generated by square-integrable representations of the Euclidean spaces. To do this, we need to clarify the structure of the closed subgroups, co-compact subgroups and uniform lattices of the Euclidean spaces; see [3]. The following corollary is a generalized version of [12, Corollary 5.2].

Corollary 3.7

Let $(\pi ,{\mathcal {H}}_\pi )$ be a square-integrable representation of ${\mathbb {R}}^d$, and let $\Lambda$ be a closed subgroup of ${\mathbb {R}}^d$. If $(\pi ,\Lambda )$ admits a HS-operator-valued frame, then the index subgroup $\Lambda$ is of the form

$$\begin{aligned} Q\big ({\mathbb {Z}}^k\times {\mathbb {R}}^{d-k}\big ),\;\;0\le k\le d,\;\;Q\in \textrm{GL}(d,{\mathbb {R}}). \end{aligned}$$

Proof

By [3, Example 21.7.2], there exist $0\le k,l\le d$ with $0\le k+l\le d$ such that

$$\begin{aligned} \Lambda \cong \{0\}^l\times {\mathbb {Z}}^k\times {\mathbb {R}}^{d-k-l}. \end{aligned}$$

By Theorem 3.1, $\Lambda$ is co-compact. This implies that $l=0$ and so $\Lambda \cong {\mathbb {Z}}^k\times {\mathbb {R}}^{d-k}$. That is, $\Lambda =Q({\mathbb {Z}}^k\times {\mathbb {R}}^{d-k})$ for some $Q\in \textrm{GL}(d,{\mathbb {R}})$. $\square$

Gabardo and Han [7] proved the following proposition which shows that for a Parseval Gabor frame indexed by full rank lattices, the norm of the window function is closely related to the determinant of the invertible matrices.

Proposition 3.8

[7] Let ${{\mathcal {L}}}=A{\mathbb {Z}}^d,{{\mathcal {K}}}=B{\mathbb {Z}}^d$ be two full rank lattices in ${\mathbb {R}}^d$ and let $g\in L^2({\mathbb {R}}^d)$. If ${{\mathscr {G}}}(g,{{\mathcal {L}}}\times {{\mathcal {K}}})$ is a Parseval Gabor frame for $L^2({\mathbb {R}}^d)$, then

$$\begin{aligned} \Vert g\Vert ^2=|\textrm{det}(AB)|. \end{aligned}$$

Our aim is to generalize this proposition to the more general setting.

Corollary 3.9

Let $(\pi ,{\mathcal {H}}_{\pi })$ be a square-integrable representation of ${\mathbb {R}}^d$ with formal dimension ${\textrm{d}}_{\pi }>0$, let $A\in {\mathcal {B}}({\mathcal {H}}_\pi )$ and let $\Lambda =P{\mathbb {Z}}^d$ be a full-rank lattice in ${\mathbb {R}}^d$. If $A\pi (\Lambda )$ is an operator-valued frame with bounds $0<\alpha \le \beta <\infty$, then $A\pi (\Lambda )$ is a HS-operator-valued frame. Moreover, we have that

$$\begin{aligned} \alpha |\textrm{detP}|\le \frac{\Vert A\Vert _2^2}{\textrm{d}_{\pi }}\le \beta |\textrm{detP}|. \end{aligned}$$

In particular, if $A\pi (\Lambda )$ is a Parseval operator-valued frame, then

$$\begin{aligned} \Vert A\Vert _2^2={\textrm{d}}_{\pi }|\textrm{detP}|. \end{aligned}$$

Proof

Since $\Lambda$ is a full-rank lattice in ${\mathbb {R}}^d$, $\Lambda$ is a closed co-compact subgroup of ${\mathbb {R}}^d$. Also,

$$\begin{aligned} \textrm{vol}(\Lambda )=m({\mathbb {R}}^d/(P{\mathbb {Z}}^d))=|\textrm{detP}|, \end{aligned}$$

where m is the Lebesuge measure on the Euclidean space. The remaining part of the proof follows from Theorem 3.1. $\square$

Now we apply our density theorem to a class of number-theoretic groups, the field ${\mathbb {Q}}_p$ of p-adic numbers. We know that ${\mathbb {Q}}_p$ is a locally compact abelian group under addition. We can refer to [15] for more information about ${\mathbb {Q}}_p$. The following corollary shows that for the p-adic numbers, there exists no other index subgroups corresponding to operator-valued frames. Moreover, this is a generalized version of [12, Corollary 5.2].

Corollary 3.10

Fix a prime number p. Let $\widehat{{\mathbb {Q}}_p}$ be the Pontryagin dual group of ${\mathbb {Q}}_p$, let $(\pi ,{\mathcal {H}}_\pi )$ be a square-integrable representation of ${\mathbb {Q}}_p\times \widehat{{\mathbb {Q}}_p}$, and let $\Lambda$ be a closed subgroup of ${\mathbb {Q}}_p\times \widehat{{\mathbb {Q}}_p}$. If $(\pi ,\Lambda )$ admits a HS-operator-valued frame, then we necessarily have that

$$\begin{aligned} \Lambda ={\mathbb {Q}}_p\times \widehat{{\mathbb {Q}}_p}. \end{aligned}$$

Proof

By Theorem 3.1, the index subgroup $\Lambda$ is co-compact. Since the only co-compact subgroup of ${\mathbb {Q}}_p\times \widehat{{\mathbb {Q}}_p}$ is the entire group itself, $\Lambda ={\mathbb {Q}}_p\times \widehat{{\mathbb {Q}}_p}$. $\square$

The following theorem shows that the existence of a HS-operator-valued frame indexed by a discrete abelian subgroup implies that the index subgroup is necessarily a uniform lattice. Furthermore, the size of the index subgroup can not be too small and the operator norm of the corresponding frame generator can not be too big.

Theorem 3.11

Let $\Lambda$ be a discrete subgroup of G equipped with the counting measure. If $A\pi (\Lambda )$ is a HS-operator-valued frame with an upper bound $\beta >0$, then $\Lambda$ is a uniform lattice in G, $\Vert A\Vert ^2\le \beta$ and

$$\begin{aligned} \textrm{vol}(\Lambda )\ge \frac{\Vert A\Vert ^2}{\beta {\textrm{d}}_{\pi }}. \end{aligned}$$

Proof

By Theorem 3.1, $\Lambda$ is co-compact. Since $\Lambda$ is discrete, $\Lambda$ is a uniform lattice. Let S be the frame operator associated with $A\pi (\Lambda )$. Then $\Vert S\Vert \le \beta$. Noticing that $AS^{-\frac{1}{2}}\pi (\Lambda )$ is a Parseval operator-valued frame and $\Lambda$ is a discrete subgroup equipped with the counting measure, we have that for each $\xi \in {\mathcal {H}}_\pi$,

$$\begin{aligned} \Vert A\xi \Vert ^2&\le \sum \limits _{\lambda \in \Lambda }\Big \Vert AS^{-\frac{1}{2}} \pi (\lambda )S^{\frac{1}{2}}\xi \Big \Vert ^2=\Big \Vert S^{\frac{1}{2}}\xi \Big \Vert ^2\\&\le \Vert S\Vert \Vert \xi \Vert ^2\le \beta \Vert \xi \Vert ^2 \end{aligned}$$

and so $\Vert A\Vert ^2\le \beta$. Again using Theorem 3.1, we obtain that

$$\begin{aligned} \textrm{vol}(\Lambda )\ge \frac{\Vert A\Vert _2^2}{\beta {\textrm{d}}_{\pi }}\ge \frac{\Vert A\Vert ^2}{\beta {\textrm{d}}_{\pi }}. \end{aligned}$$

In particular, if $A\pi (\Lambda )$ is a Parseval HS-operator-valued frame, then $\Vert A\Vert \le 1$ and

$$\begin{aligned} \textrm{vol}(\Lambda )\ge \frac{\Vert A\Vert ^2}{{\textrm{d}}_{\pi }}. \end{aligned}$$

$\square$

For every frame $\{f_k\}_{k\in \Omega }$ for a Hilbert space ${\mathcal {H}}$, we know that $\{S^{-1}f_k\}_{k\in \Omega }$ is the canonical dual frame, where S is the frame operator associated with the frame $\{f_k\}_{k\in \Omega }$. The following proposition is celebrated in frame theory.

Proposition 3.12

[12] Let ${\mathcal {H}}$ be a Hilbert space and let $\alpha ,\beta >0$. Then the following statements are equivalent:

• (i) The system $\{f_k\}_{k\in \Omega }$ is a frame for ${\mathcal {H}}$ with bounds $\alpha$ and $\beta$.

• (ii) The system $\{f_k\}_{k\in \Omega }$ is a Bessel system for ${\mathcal {H}}$ with a bound $\beta$ and there exists another Bessel system $\{g_k\}_{k\in \Omega }$ for ${\mathcal {H}}$ with a bound $\frac{1}{\alpha }$ such that the equality

  $$\begin{aligned} \langle f,g\rangle =\int \limits _{\Omega }\langle f,g_k\rangle \langle f_k,g\rangle d\mu (k) \end{aligned}$$

  holds for all $f,g\in {\mathcal {H}}$.

Based on Proposition 3.12, we get the following theorem. A necessary and sufficient condition for an operator-valued system to be an operator-valued frame is given in this theorem. Further, it will provide ideas for obtaining some duality properties about operator-valued frames with special structure.

Theorem 3.13

Let $\Lambda$ be a unimodular closed co-compact subgroup of G, let $A\in {\mathcal {B}}({\mathcal {H}}_\pi )$ and let $0<\alpha \le \beta <\infty$. Then the following statements are equivalent:

• (i) The system $A\pi (\Lambda )$ is an operator-valued frame with bounds $\alpha$ and $\beta$.

• (ii) The system $A\pi (\Lambda )$ is a Bessel system with a bound $\beta$. Moreover, the pair $(\pi ,\Lambda )$ admits another HS-operator-valued Bessel system $B\pi (\Lambda )$ with a bound $\frac{1}{\alpha }$ and the equality

  $$\begin{aligned} \langle \xi ,\eta \rangle =\int \limits _\Lambda \langle A\pi (\lambda )\xi ,B\pi (\lambda )\eta \rangle d\lambda \end{aligned}$$

  holds for all $\xi ,\eta \in {\mathcal {H}}_\pi$.

Proof

(i)$\Rightarrow$(ii): We suppose that S is the frame operator associated with the operator-valued frame $A\pi (\Lambda )$. Then by Theorem 3.1, $A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$. Thus $A\pi (\Lambda )$ is a HS-operator-valued frame indexed by a unimodular subgroup $\Lambda$ of G. Set

$$\begin{aligned} B:=AS^{-1}. \end{aligned}$$

By Remark 3.6, $B\pi (\Lambda )$ is the canonical HS-dual of $A\pi (\Lambda )$.

(ii)$\Rightarrow$(i): We assume that the statement (ii) holds. For any $\xi \in {\mathcal {H}}_\pi$, we have that

$$\begin{aligned} \Vert \xi \Vert ^2\le \int \limits _\Lambda |\langle A\pi (\lambda )\xi ,B\pi (\lambda )\xi \rangle |d\lambda . \end{aligned}$$

By the Cauchy–Schwarz inequality, we see that

$$\begin{aligned} \Vert \xi \Vert ^2&\le \int \limits _\Lambda \Vert A\pi (\lambda )\xi \Vert \Vert B\pi (\lambda )\xi \Vert d\lambda \\&\le \left( \int \limits _\Lambda \Vert A\pi (\lambda )\xi \Vert ^2d\lambda \right) ^{\frac{1}{2}} \left( \int \limits _\Lambda \Vert B\pi (\lambda )\xi \Vert ^2d\lambda \right) ^{\frac{1}{2}}. \end{aligned}$$

Since $B\pi (\Lambda )$ is Bessel with a bound $\frac{1}{\alpha }$, we obtain that

$$\begin{aligned} \int \limits _\Lambda \Vert B\pi (\lambda )\xi \Vert ^2d\lambda \le \frac{1}{\alpha }\Vert \xi \Vert ^2. \end{aligned}$$

Hence we get the lower frame bound inequality

$$\begin{aligned} \alpha \Vert \xi \Vert ^2\le \int \limits _\Lambda \Vert A\pi (\lambda )\xi \Vert ^2d\lambda . \end{aligned}$$

Thus $A\pi (\Lambda )$ is an operator-valued frame with bounds $\alpha$ and $\beta$. $\square$

Next we prove that under certain assumptions on square-integrable representations, uniform lattices-indexed HS-operator-valued frames always exist. To see this, we need the following proposition.

Proposition 3.14

[5] Let $(\pi ,{\mathcal {H}}_\pi )$ be a square-integrable representation of G with formal dimension ${\textrm{d}}_{\pi }>0$, where $(G,\sigma )$ satisfies Kleppner’s condition. Let $\Lambda$ be a uniform lattice in G. Then $(\pi ,\Lambda )$ admits an n-multiwindow d-super frame if and only if ${\textrm{d}}_{\pi }\textrm{vol}(\Lambda )\le \frac{n}{d}$.

Theorem 3.15

Let $(\pi ,{\mathcal {H}}_\pi )$ be a square-integrable representation of G with formal dimension ${\textrm{d}}_{\pi }>0$, and let $\Lambda$ be a discrete subgroup of G equipped with the counting measure. Suppose that $(G,\sigma )$ satisfies Kleppner’s condition. Then the following statements are equivalent:

• (i) $(\pi ,\Lambda )$ admits a FR-operator-valued frame, i.e., there exist a finite-rank operator A such that $A\pi (\Lambda )$ is an operator-valued frame.

• (ii) $(\pi ,\Lambda )$ admits a HS-operator-valued frame, i.e., there exist a Hilbert–Schmidt operator A such that $A\pi (\Lambda )$ is an operator-valued frame.

• (iii) $\Lambda$ is co-compact, i.e., $\textrm{vol}(\Lambda )<\infty$.

In particular, there always exist a HS-operator-valued Gabor frame with the index subgroup being the uniform lattice as long as there exist a uniform lattice in the phase space.

Proof

(i)$\Rightarrow$(ii): This is clear.

(ii)$\Rightarrow$(iii): Suppose that $A\pi (\Lambda )$ is a HS-operator-valued frame. By Theorem 3.1, $\Lambda$ is co-compact.

(iii)$\Rightarrow$(i): If $\Lambda$ is co-compact, then $\textrm{vol}(\Lambda )<\infty$ and so we can choose a positive integer n satisfying

$$\begin{aligned} {\textrm{d}}_{\pi }\textrm{vol}(\Lambda )\le n. \end{aligned}$$

Since $\Lambda$ is discrete, $\Lambda$ is a uniform lattice in G. By Proposition 3.14, $(\pi ,\Lambda )$ admits a n-multiwindow frame $\{\pi (\lambda )\eta _i\}_{\lambda \in \Lambda ,1\le i\le n}$ for ${\mathcal {H}}_\pi$. Thus there exist constants $0<\alpha \le \beta <\infty$ such that

$$\begin{aligned} \alpha \Vert \xi \Vert ^2\le \sum \limits _i\sum \limits _\lambda |\langle \xi , \pi (\lambda )\eta _i\rangle |^2\le \beta \Vert \xi \Vert ^2 \end{aligned}$$

for all $\xi \in {\mathcal {H}}_\pi$. We can take an orthonormal set $\{\xi _i:1\le i\le n\}$ in ${\mathcal {H}}_\pi$. Set

$$\begin{aligned} A:=\sum \limits _i\xi _i\otimes \eta _i. \end{aligned}$$

Then $A\in {\mathcal {F}}({\mathcal {H}}_\pi )$. For any $\xi \in {\mathcal {H}}_\pi$, we compute that

$$\begin{aligned} \sum \limits _\lambda \Vert A\pi (\lambda )\xi \Vert ^2=\sum \limits _i \sum \limits _\lambda |\langle \xi ,\pi (\lambda )\eta _i\rangle |^2. \end{aligned}$$

Hence $A\pi (\Lambda )$ is a FR-operator-valued frame. This completes the proof. $\square$

4 Operator-Valued Systems Indexed by the Entire Group G

In this section, we study the operator-valued systems indexed by the entire group. To study their properties better, we introduce the notion of the wavelet transform for Hilbert–Schmidt operators and give some basic properties about this wavelet transform.

Fix $\eta \in {\mathcal {H}}_\pi$. By Proposition 1.2, the system $\pi (G)\eta$ indexed by the entire group is a tight frame for ${\mathcal {H}}_\pi$ with bound $\frac{\Vert \eta \Vert ^2}{{\textrm{d}}_{\pi }}$. As a generalization of this fact, the first theorem suggests that HS-operator-valued systems indexed by the entire group must be tight operator-valued frames.

Theorem 4.1

Let $A\in {\mathcal {B}}({\mathcal {H}}_\pi )$. Then the system $A\pi (G)$ is a tight operator-valued frame with bound $\frac{\Vert A\Vert _2^2}{{\textrm{d}}_{\pi }}$ if and only if $A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$.

Proof

For the proof of the necessary part, we use Theorem 3.1. Conversely, we suppose that $A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$ and $\{\xi _i\}_{i\in {\mathbb {I}}}$ is an orthonormal basis for ${\mathcal {H}}_\pi$, where ${\mathbb {I}}$ is a finite or countable set. Then $\Vert A\Vert _2<\infty$. For any $\eta \in {\mathcal {H}}_\pi$ and any $x\in G$, we have that

$$\begin{aligned} \Vert A\pi (x)\eta \Vert ^2=\sum \limits _i|\langle A^*\xi _i,\pi (x)\eta \rangle |^2. \end{aligned}$$

By Proposition 1.2, we obtain that

$$\begin{aligned} \int \limits _G|\langle A^*\xi _i,\pi (x)\eta \rangle |^2dx=\frac{\Vert A^*\xi _i\Vert ^2\Vert \eta \Vert ^2}{\mathrm{d_\pi }} \end{aligned}$$

for all $i\in {\mathbb {I}}$. Hence we conclude that

$$\begin{aligned} \int \limits _G\Vert A\pi (x)\eta \Vert ^2dx=\frac{\Vert A\Vert _2^2}{\mathrm{d_\pi }}\Vert \eta \Vert ^2. \end{aligned}$$

Thus $A\pi (G)$ is a tight HS-operator-valued frame with bound $\frac{\Vert A\Vert _2^2}{{\textrm{d}}_{\pi }}$. $\square$

Definition 4.2

(Wavelet transform [5]) Let $(\pi ,{\mathcal {H}}_\pi )$ be a $\sigma$-projective unitary representation of locally compact group G. Given $\eta ,\xi \in {\mathcal {H}}_\pi$. The function $\phi _{\eta ,\xi }:G\rightarrow {\mathbb {C}}$ is given by

$$\begin{aligned} \phi _{\eta ,\xi }(x)=\langle \xi ,\pi (x)\eta \rangle \end{aligned}$$

for all $x\in G$. This function $\phi _{\eta ,\xi }$ is called the wavelet transform of $\eta$ with respect to $\xi$.

Based on Definition 4.2, we introduce the wavelet transform for Hilbert–Schmidt operators. This wavelet transform is closely related to square-integrable representations of locally compact groups.

Definition 4.3

(Wavelet transform for Hilbert–Schmidt operators) Let $(\pi ,{\mathcal {H}}_\pi )$ be a square-integrable representation of G with formal dimension ${\textrm{d}}_{\pi }>0$, and let $A,B\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$. The function $\Phi _{A,B}:G\rightarrow {\mathbb {C}}$ is given by

$$\begin{aligned} \Phi _{A,B}(x)=\langle B,A\pi (x)\rangle _2 \end{aligned}$$

for all $x\in G$. This function $\Phi _{A,B}$ is called the wavelet transform of A with respect to B.

We remark that the notion of the wavelet transform for Hilbert–Schmidt operators is a generalization of the classical wavelet transform. Indeed, if $\rho ,\eta ,\xi \in {\mathcal {H}}_\pi$ and $\Vert \rho \Vert =1$, then the wavelet transform of $\rho \otimes \xi$ with respect to $\rho \otimes \eta$ is precisely the wavelet transform of $\eta$ with respect to $\xi$. To see this, for any $x\in G$, we have that

$$\begin{aligned} \Phi _{\rho \otimes \xi ,\rho \otimes \eta }(x)=\langle \rho \otimes \eta ,(\rho \otimes \xi )\pi (x)\rangle _2 =\langle \pi (x)^*\xi ,\eta \rangle =\phi _{\eta ,\xi }(x). \end{aligned}$$

Next we collect some basic properties about the wavelet transform for Hilbert–Schmidt operators as follows.

Proposition 4.4

Let $A,B,C,D\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$. Then

• (i) $\Phi _{AB,C}=\Phi _{B,A^*C}$ and $\Phi _{A,BC}=\Phi _{B^*A,C}$. Moreover, this wavelet transform is conjugate-linear in the first variable and linear in the second variable in the sense that

  $$\begin{aligned} \Phi _{\alpha A+B,C}=\overline{\alpha }\Phi _{A,C}+\Phi _{B,C},\;\;\Phi _{A,\beta C+D}=\beta \Phi _{A,C}+\Phi _{A,D} \end{aligned}$$

  hold for all $\alpha ,\beta \in {\mathbb {C}}$, where $\overline{\alpha }$ is the complex conjugate of $\alpha$.

• (ii) For any $x,y\in G$, we have that

  $$\begin{aligned} \Phi _{A,B\pi (x)}(y)=\overline{\sigma (xy^{-1},y)\Phi _{B,A}(xy^{-1})}. \end{aligned}$$

• (iii) The function $\Phi _{A,B}$ is a bounded function on G and

  $$\begin{aligned} \Vert \Phi _{A,B}\Vert _{\infty }\le \Vert A\Vert _2\Vert B\Vert _2, \end{aligned}$$

  where $\Vert \Phi _{A,B}\Vert _{\infty }$ is the sup-norm of $\Phi _{A,B}$.

• (iv) The function $\Phi _{A,B}$ belongs to $L^2(G)$ and

  $$\begin{aligned} \Vert \Phi _{A,B}\Vert ^2=\frac{\Vert A^*B\Vert _2^2}{{\textrm{d}}_{\pi }}. \end{aligned}$$

  More generally, we obtain a generalized Moyal identity:

  $$\begin{aligned} \langle \Phi _{A,B},\Phi _{C,D}\rangle =\frac{\langle A^*B,C^*D\rangle _2}{{\textrm{d}}_{\pi }}. \end{aligned}$$

Proof

(i) It is immediate from Definition 4.3.

(ii) For any $x,y\in G$, we see that

$$\begin{aligned} \Phi _{A,B\pi (x)}(y)=\textrm{tr}(B\pi (x)\pi (y)^*A^*). \end{aligned}$$

Since $\pi$ is a $\sigma$-projective unitary representation, we have that

$$\begin{aligned} \pi (x)\pi (y)^*=\overline{\sigma (xy^{-1},y)}\pi (xy^{-1}). \end{aligned}$$

Thus

$$\begin{aligned} \Phi _{A,B\pi (x)}(y)&=\overline{\sigma (xy^{-1},y)}\textrm{tr}(B\pi (xy^{-1})A^*)\\&=\overline{\sigma (xy^{-1},y)}\langle B\pi (xy^{-1}),A\rangle _2\\&=\overline{\sigma (xy^{-1},y)\Phi _{B,A}(xy^{-1})}. \end{aligned}$$

(iii) By the Cauchy–Schwarz inequality, we obtain that

$$\begin{aligned} |\Phi _{A,B}(x)|^2\le \Vert B\Vert _2^2\Vert A\pi (x)\Vert _2^2=\Vert A\Vert _2^2\Vert B\Vert _2^2<\infty \end{aligned}$$

for all $x\in G$. So $\Phi _{A,B}$ is a bounded function on G and

$$\begin{aligned} \Vert \Phi _{A,B}\Vert _{\infty }\le \Vert A\Vert _2\Vert B\Vert _2. \end{aligned}$$

(iv) By Definition 4.3, we know that

$$\begin{aligned} \int \limits _G\Phi _{A,B}(x)\overline{\Phi _{C,D}(x)}dx=\int \limits _G\langle B,A\pi (x)\rangle _2\langle C\pi (x),D\rangle _2 dx. \end{aligned}$$

We assume that $\{\xi _i\}_{i\in {\mathbb {I}}}$ is an orthonormal basis for ${\mathcal {H}}_\pi$, where ${\mathbb {I}}$ is a finite or countable set. For any $x\in G$, we see that

$$\begin{aligned} \langle B,A\pi (x)\rangle _2\langle C\pi (x),D\rangle _2=\sum \limits _{i,j}\langle B\xi _i,A\pi (x)\xi _i\rangle \langle C\pi (x)\xi _j,D\xi _j\rangle . \end{aligned}$$

By Proposition 1.2, we have that

$$\begin{aligned} \int \limits _G\langle A^*B\xi _i,\pi (x)\xi _i\rangle \langle \pi (x)\xi _j,C^*D\xi _j\rangle dx=\frac{\langle A^*B\xi _i,C^*D\xi _j\rangle \langle \xi _j,\xi _i\rangle }{{\textrm{d}}_{\pi }} \end{aligned}$$

for all $i,j\in {\mathbb {I}}$. Thus by Tonelli’s Theorem, we obtain that

$$\begin{aligned} \int \limits _G\Phi _{A,B}(x)\overline{\Phi _{C,D}(x)}dx&=\sum \limits _{i,j}\int \limits _G\langle B\xi _i,A\pi (x)\xi _i\rangle \langle C\pi (x)\xi _j,D\xi _j\rangle dx\\&=\frac{1}{{\textrm{d}}_{\pi }}\sum \limits _i\sum \limits _j\langle D^*CA^*B\xi _i,\xi _j\rangle \langle \xi _j,\xi _i\rangle . \end{aligned}$$

Since $\{\xi _j:j\in {\mathbb {I}}\}$ is an orthogonormal basis for ${\mathcal {H}}_\pi$, the equality

$$\begin{aligned} \sum \limits _j\langle D^*CA^*B\xi _i,\xi _j\rangle \langle \xi _j,\xi _i\rangle =\langle D^*CA^*B\xi _i,\xi _i\rangle \end{aligned}$$

holds for all $i\in {\mathbb {I}}$. Thus we conclude that

$$\begin{aligned} \int \limits _G\Phi _{A,B}(x)\overline{\Phi _{C,D}(x)}dx=\frac{1}{\mathrm{d_\pi }}\langle A^*B,C^*D\rangle _2. \end{aligned}$$

Hence

$$\begin{aligned} \int \limits _G|\Phi _{A,B}(x)|^2dx=\frac{\Vert A^*B\Vert _2^2}{\mathrm{d_\pi }}<\infty , \end{aligned}$$

which implies that $\Phi _{A,B}\in L^2(G)$ and

$$\begin{aligned} \Vert \Phi _{A,B}\Vert ^2=\frac{\Vert A^*B\Vert _2^2}{{\textrm{d}}_{\pi }}. \end{aligned}$$

Moreover, we also obtain that

$$\begin{aligned} \langle \Phi _{A,B},\Phi _{C,D}\rangle =\frac{\langle A^*B,C^*D\rangle _2}{{\textrm{d}}_{\pi }}. \end{aligned}$$

The proof is completed. $\square$

Remark 4.5

Fix an unit vector $\rho \in {\mathcal {H}}_\pi$. Then for any $\xi ,\xi ',\eta ,\eta '\in {\mathcal {H}}_\pi$, take

$$\begin{aligned} A=\rho \otimes \xi ,B=\rho \otimes \eta ,C=\rho \otimes \xi ',D=\rho \otimes \eta '. \end{aligned}$$

On the one hand, we see that

$$\begin{aligned} \langle \Phi _{\rho \otimes \xi ,\rho \otimes \eta },\Phi _{\rho \otimes \xi ',\rho \otimes \eta '}\rangle =\langle \phi _{\eta ,\xi },\phi _{\eta ',\xi '}\rangle =\int \limits _G\langle \xi ,\pi (x)\eta \rangle \langle \pi (x)\eta ',\xi '\rangle dx. \end{aligned}$$

On the other hand, by Proposition 4.4 (iv), we obtain that

$$\begin{aligned} \langle \Phi _{\rho \otimes \xi ,\rho \otimes \eta },\Phi _{\rho \otimes \xi ',\rho \otimes \eta '}\rangle =\frac{1}{\mathrm{d_\pi }}\langle (\rho \otimes \xi )^*(\rho \otimes \eta ),(\rho \otimes \xi ')^*(\rho \otimes \eta ')\rangle _2. \end{aligned}$$

We compute that

$$\begin{aligned} \langle (\rho \otimes \xi )^*(\rho \otimes \eta ),(\rho \otimes \xi ')^*(\rho \otimes \eta ')\rangle _2 =\langle \xi ,\xi '\rangle \langle \eta ',\eta \rangle . \end{aligned}$$

Hence this conclusion is a generalization of the classical Moyal identity.

Theorem 4.6

Let $A,B\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$. If they satisfy the relation

$$\begin{aligned} \langle A,B\rangle _2={\textrm{d}}_{\pi }, \end{aligned}$$

then the systems $A\pi (G)$ and $B\pi (G)$ are HS-dual. Furthermore, we have a reproducing formula:

$$\begin{aligned} \int \limits _G\langle \pi (x)^*B^*A\pi (x)\xi ,\eta \rangle dx=\langle \xi ,\eta \rangle \end{aligned}$$

for all $\xi ,\eta \in {\mathcal {H}}_\pi$.

Proof

By Theorem 4.1, the systems $A\pi (G)$ and $B\pi (G)$ are two tight operator-valued frames. We suppose that $\{\xi _i\}_{i\in {\mathbb {I}}}$ is an orthonormal basis for ${\mathcal {H}}_\pi$, where ${\mathbb {I}}$ is a finite or countable set.

For any $\xi ,\eta \in {\mathcal {H}}_\pi$ and any $x\in G$, we have that

$$\begin{aligned} \langle A\pi (x)\xi ,B\pi (x)\eta \rangle =\sum \limits _i\langle A\pi (x)\xi ,\xi _i\rangle \langle \xi _i,B\pi (x)\eta \rangle . \end{aligned}$$

Thus by Tonelli’s Theorem,

$$\begin{aligned} \int \limits _G\langle \pi (x)^*B^*A\pi (x)\xi ,\eta \rangle dx=\sum \limits _i\int \limits _G\langle \pi (x)\xi ,A^*\xi _i\rangle \langle B^*\xi _i,\pi (x)\eta \rangle dx. \end{aligned}$$

By Proposition 1.2, we see that

$$\begin{aligned} \int \limits _G\langle B^*\xi _i,\pi (x)\eta \rangle \langle \pi (x)\xi ,A^*\xi _i\rangle dx=\frac{\langle B^*\xi _i,A^*\xi _i\rangle \langle \xi ,\eta \rangle }{{\textrm{d}}_{\pi }}. \end{aligned}$$

We compute that

$$\begin{aligned} \langle A,B\rangle _2=\sum \limits _i\langle B^*\xi _i,A^*\xi _i\rangle . \end{aligned}$$

Thus we obtain that

$$\begin{aligned} \int \limits _G\langle \pi (x)^*B^*A\pi (x)\xi ,\eta \rangle dx=\langle A,B\rangle _2\frac{\langle \xi ,\eta \rangle }{\mathrm{d_\pi }}=\langle \xi ,\eta \rangle . \end{aligned}$$

This finishes the proof. $\square$

Theorem 4.7

If $A,B,C\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$, then we have a reproducing formula:

$$\begin{aligned} {\textrm{d}}_{\pi }\int \limits _G\langle C,A\pi (x)\rangle _2\langle B\pi (x),D\rangle _2 dx=\langle BA^*C,D\rangle _2 \end{aligned}$$

for all $D\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$.

Proof

For any $D\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$, by the generalized Moyal identity, we see that

$$\begin{aligned} {\textrm{d}}_{\pi }\int \limits _G\langle C,A\pi (x)\rangle _2\langle B\pi (x),D\rangle _2 dx=\langle A^*C,B^*D\rangle _2. \end{aligned}$$

We also have that

$$\begin{aligned} \langle A^*C,B^*D\rangle _2=\textrm{tr}(D^*BA^*C)=\langle BA^*C,D\rangle _2. \end{aligned}$$

$\square$

Set

$$\begin{aligned} \textrm{supp}\Phi _{A,B}:=\textrm{cl}(\{x\in G:\Phi _{A,B}(x)\ne 0\}), \end{aligned}$$

where $\textrm{cl}(K)$ is the closure of the subset K of G. The set $\textrm{supp}\Phi _{A,B}$ is called the support of the wavelet transform of A with respect to B.

Theorem 4.7 has the following straight corollary. This corollary shows that the measure of the support of the wavelet transform for Hilbert–Schmidt operators has a special lower bound.

Proposition 4.8

Let A, B be nonzero Hilbert–Schmidt operators on ${\mathcal {H}}_\pi$. If the set $\textrm{supp}\Phi _{A,B}$ has a strictly positive measure, then it satisfies the inequality

$$\begin{aligned} \mu _G(\textrm{supp}\Phi _{A,B})\ge \frac{1}{\mathrm{d_\pi }}\frac{\Vert A^*B\Vert _2^2}{\Vert A\Vert _2^2\Vert B\Vert _2^2}. \end{aligned}$$

Proof

We suppose that $\textrm{supp}\Phi _{A,B}$ has a strict positive measure. Then $\Vert \Phi _{A,B}\Vert _\infty >0$. By Theorem 4.7, we have that

$$\begin{aligned} \Vert A^*B\Vert _2^2={\textrm{d}}_{\pi }\Vert \Phi _{A,B}\Vert ^2 \end{aligned}$$

and

$$\begin{aligned} \Vert \Phi _{A,B}\Vert _\infty \le \Vert A\Vert _2\Vert B\Vert _2. \end{aligned}$$

Since $\textrm{supp}\Phi _{A,B}=\textrm{cl}(\{x\in G:\Phi _{A,B}(x)\ne 0\})$, we have that

$$\begin{aligned} \Vert \Phi _{A,B}\Vert ^2&=\int \limits _{\textrm{supp}\Phi _{A,B}}|\Phi _{A,B}(x)|^2dx\\&\le \Vert \Phi _{A,B}\Vert _\infty ^2\mu _G(\textrm{supp}\Phi _{A,B}). \end{aligned}$$

Thus we conclude that

$$\begin{aligned} \mu _G(\textrm{supp}\Phi _{A,B})\ge \frac{\Vert \Phi _{A,B}\Vert ^2}{\Vert \Phi _{A,B}\Vert _\infty ^2}\ge \frac{1}{\mathrm{d_\pi }}\frac{\Vert A^*B\Vert _2^2}{\Vert A\Vert _2^2\Vert B\Vert _2^2}. \end{aligned}$$

$\square$

5 Operator-Valued Systems in ${\mathcal {S}}_2({\mathcal {H}}_\pi )$

Interestingly, the system of the form (1.3) indexed by the entire group is a Bessel system for ${\mathcal {S}}_2({\mathcal {H}}_\pi )$. However, there are not such a frame in ${\mathcal {S}}_2({\mathcal {H}}_\pi )$ under some assumptions on the representation space ${\mathcal {H}}_\pi$.

Theorem 5.1

Suppose that B is a nonzero Hilbert–Schmidt operator on ${\mathcal {H}}_\pi$ and the representation space ${\mathcal {H}}_\pi$ is a separable infinite dimensional Hilbert space. Then $B\pi (G)$ indexed by the entire group is a Bessel system for ${\mathcal {S}}_2({\mathcal {H}}_\pi )$ with bound $\frac{\Vert B\Vert ^2}{\mathrm{d_\pi }}$ in the sense that there exists a positive number $\beta$ such that

$$\begin{aligned} \int \limits _G|\langle A,B\pi (x)\rangle _2|^2 dx\le \beta \Vert A\Vert _2^2 \end{aligned}$$

for all $\;A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$. Moreover, this system $B\pi (G)$ can not be a frame for ${\mathcal {S}}_2({\mathcal {H}}_\pi )$ in the sense of Definition 2.1. That is, there do not exist constants $0<\alpha \le \beta <\infty$ such that

$$\begin{aligned} \alpha \Vert A\Vert _2^2\le \int \limits _G|\langle A,B\pi (x)\rangle _2|^2 dx\le \beta \Vert A\Vert _2^2 \end{aligned}$$

for all $A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$.

Proof

By Theorem 4.7, we obtain that

$$\begin{aligned} \int \limits _G|\langle A,B\pi (x)\rangle _2|^2 dx=\frac{\Vert B^*A\Vert _2^2}{{\textrm{d}}_{\pi }}\le \frac{\Vert B\Vert ^2}{\mathrm{d_\pi }}\Vert A\Vert _2^2 \end{aligned}$$

for all $A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$. Since $0<\Vert B\Vert ^2<\infty$, $B\pi (G)$ is a Bessel system with bound $\frac{\Vert B\Vert ^2}{{\textrm{d}}}_{\pi }$.

For the proof of moreover part, we assume that there exists also a positive constant $\alpha$ such that

$$\begin{aligned} \int \limits _G|\langle A,B\pi (x)\rangle _2|^2 dx\ge \alpha \Vert A\Vert _2^2 \end{aligned}$$

for all $A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$. For any $\xi \in {\mathcal {H}}_\pi$, $\xi \otimes \xi \in {\mathcal {S}}_2({\mathcal {H}}_\pi )$. On the one hand, using the above lower bound inequality for the system $B\pi (G)$ yields that

$$\begin{aligned} \alpha \Vert \xi \otimes \xi \Vert _2^2\le \int \limits _G|\langle \xi \otimes \xi ,B\pi (x)\rangle _2|^2dx. \end{aligned}$$

On the other hand, by Theorem 4.7,

$$\begin{aligned} \int \limits _G|\langle \xi \otimes \xi ,B\pi (x)\rangle _2|^2dx=\frac{\Vert B^*\xi \otimes \xi \Vert _2^2}{\mathrm{d_\pi }}. \end{aligned}$$

Thus the inequality

$$\begin{aligned} \Vert B^*\xi \Vert ^2\ge \alpha {\textrm{d}}_{\pi }\Vert \xi \Vert ^2 \end{aligned}$$

holds for all $\xi \in {\mathcal {H}}_\pi$. This implies that B is surjective.

However, since $B\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$, B is a compact operator on ${\mathcal {H}}_\pi$. Then B is not surjective because ${\mathcal {H}}_\pi$ is a separable infinite dimensional Hilbert space. We get a contradiction so far. Hence there does not exist such a positive constant. So $B\pi (G)$ is not a frame for ${\mathcal {S}}_2({\mathcal {H}}_\pi )$. This completes the proof. $\square$

Finally, we give the density theorem for the ordinary frame of the following form

$$\begin{aligned} B\pi (\Lambda ):=\{B\pi (\lambda ):\lambda \in \Lambda \}, \end{aligned}$$

where the generator $B\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$ and the index subgroup $\Lambda$ is a closed subgroup of locally compact abelian group G.

Theorem 5.2

Let $\Lambda$ be a closed subgroup of locally compact abelian group G, and let $B\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$. If $B\pi (\Lambda )$ is a frame for $S_2({\mathcal {H}}_\pi )$ with bounds $0<\alpha \le \beta <\infty$, then the following statements hold:

• (i) The index subgroup $\Lambda$ is co-compact, i.e, $\textrm{vol}(\Lambda )<\infty$.

• (ii) $\alpha \textrm{vol}(\Lambda )\le \frac{\Vert B\Vert ^2}{\textrm{d}_{\pi }}\le \beta \textrm{vol}(\Lambda )$.

Proof

We suppose that $B\pi (\Lambda )$ is a frame for $S_2({\mathcal {H}}_\pi )$ with bounds $0<\alpha \le \beta <\infty$. Then it satisfies the inequalities

$$\begin{aligned} \alpha \Vert A\Vert _2^2\le \int \limits _\Lambda |\langle A,B\pi (\lambda )\rangle _2|^2d\lambda \le \beta \Vert A\Vert _2^2 \end{aligned}$$

for all $A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$.

(i) By Theorem 4.7, we have that

$$\begin{aligned} \int \limits _G|\langle A,B\pi (x)\rangle _2|^2dx=\frac{\Vert B^*A\Vert _2^2}{\mathrm{d_\pi }}\le \frac{\Vert B\Vert ^2\Vert A\Vert _2^2}{{\textrm{d}}_{\pi }}<\infty \end{aligned}$$

for all $A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$. By Weil’s identity, we get that

$$\begin{aligned} \int \limits _G|\langle A,B\pi (x)\rangle _2|^2dx=\int \limits _{G/\Lambda }\int \limits _\Lambda |\langle A,B\pi (\lambda x)\rangle _2|^2d\lambda d(x\Lambda ). \end{aligned}$$

For any $x\in G$ and any $\lambda \in \Lambda$, we compute that

$$\begin{aligned} |\langle A,B\pi (\lambda x)\rangle _2|=|\langle A\pi (x)^*,B\pi (\lambda )\rangle _2|. \end{aligned}$$

Using the lower frame bound inequality for the system $B\pi (\Lambda )$ yields that

$$\begin{aligned} \int \limits _\Lambda |\langle A\pi (x)^*,B\pi (\lambda )\rangle _2|^2d\lambda \ge \alpha \Vert A\Vert _2^2. \end{aligned}$$

Combining the above formulas, we conclude that

$$\begin{aligned} \frac{\Vert B\Vert ^2\Vert A\Vert _2^2}{\mathrm{d_\pi }}\ge \int \limits _{G/\Lambda }\alpha \Vert A\Vert _2^2d(x\Lambda )=\alpha \Vert A\Vert _2^2\textrm{vol}(\Lambda ). \end{aligned}$$

So

$$\begin{aligned} \textrm{vol}(\Lambda )\le \frac{\Vert B\Vert ^2}{\alpha {\textrm{d}}_{\pi }}<\infty . \end{aligned}$$

This implies that $\Lambda$ is co-compact.

(ii) For every $A\in {\mathcal {S}}_2({\mathcal {H}}_\pi )$, we also have that

$$\begin{aligned} \frac{\Vert B^*A\Vert _2^2}{\mathrm{d_\pi }}&=\int \limits _{G/\Lambda }\int \limits _\Lambda |\langle A\pi (x)^*,B\pi (\lambda )\rangle _2|d\lambda d(x\Lambda )\\&\le \int \limits _{G/\Lambda }\beta \Vert A\pi (x)^*\Vert _2^2d(x\Lambda )\\&=\beta \Vert A\Vert _2^2\textrm{vol}(\Lambda ), \end{aligned}$$

where we use the the upper frame bound inequality for the system $B\pi (\Lambda )$. For any $\xi ,\xi _0\in {\mathcal {H}}_\pi$ with $\Vert \xi _0\Vert =1$, we obtain that

$$\begin{aligned} \frac{\Vert B^*\xi \otimes \xi _0\Vert _2^2}{\mathrm{d_\pi }}\le \beta \Vert \xi \otimes \xi _0\Vert _2^2\textrm{vol}(\Lambda ). \end{aligned}$$

That is, $\frac{\Vert B^*\xi \Vert ^2}{{\textrm{d}}_{\pi }}\le \beta \Vert \xi \Vert ^2\textrm{vol}(\Lambda )$ and so

$$\begin{aligned} \alpha \textrm{vol}(\Lambda )\le \frac{\Vert B\Vert ^2}{{\textrm{d}}_{\pi }}\le \beta \textrm{vol}(\Lambda ). \end{aligned}$$

This completes the proof. $\square$