Invertibility in the Flag Kernels Algebra on the Heisenberg Group

Abstract

Flag kernels are tempered distributions which generalize these of Calderón–Zygmund type. For any homogeneous group $\mathbb {G}$ the class of operators which acts on $L^{2}(\mathbb {G})$ by convolution with a flag kernel is closed under composition. In the case of the Heisenberg group we prove the inverse-closed property for this algebra. It means that if an operator from this algebra is invertible on $L^{2}(\mathbb {G})$, then its inversion remains in the class.

1 Introduction

A subalgebra $\mathcal {A}$ of the algebra $\mathcal {B}(\mathcal {H})$ of all bounded operators on a Hilbert space $\mathcal {H}$ is said to be inverse-closed if every $a\in \mathcal {A}$ which is invertible in $\mathcal {B}(\mathcal {H})$ is also invertible in $\mathcal {A}$. The question whether an algebra of convolution operators on a Lie group, or simply ${\mathbb {R}^n}$ is inverse-closed is not new. In 1953, Calderón and Zygmund [4] showed that the class of convolution operators on $L^{2}({\mathbb {R}^n})$ whose kernels are homogeneous of degree $-n$ and are locally in $L^{q}({\mathbb {R}^n})$ away from the origin, has the property. Here $1<q<\infty$. Much later the result was generalized by Christ [5] who proved that similar algebras on a homogeneous group are inverse-closed. A homogeneous group $\mathbb {G}$ is a nilpotent Lie group with dilations, a very natural generalization of the homogeneous structure on ${\mathbb {R}^n}$.

Another direction has been taken by Christ and Geller [6] who dealt with the algebra of operators with kernels which are homogeneous of degree $-n$ and smooth away from the identity on a homogeneous group with gradation. This algebra is inverse-closed too. A step further has been made by Głowacki [9] who showed that this is so for any homogeneous group.

The kernels which are smooth away from the identity allow an interesting generalization. One can relax the homogeneity condition demanding only that the kernel satisfies the estimates

$$\begin{aligned} |\partial _{x}^{\alpha }K(x)|\lesssim |x|^{-Q-|\alpha |}, \end{aligned}$$

where Q is the homogeneous dimension of the group. The cancellation condition takes the form

$$\begin{aligned} |\langle K,\varphi \rangle |=|\int _{\mathbb {G}}\varphi (x)K(x)dx|\lesssim \Vert \varphi \Vert , \end{aligned}$$

for $\varphi \in \mathcal {S}(\mathbb {G})$, where $\Vert \cdot \Vert$ is a fixed seminorm in the Schwartz space $\mathcal {S}(\mathbb {G})$. Such kernels K are often called the Calderón–Zygmund kernels and the corresponding operators $\mathrm{Op}(K)$ the Calderón–Zygmund operators. The class is closed under the composition of operators (Coré-Geller [7]) so they form an algebra. It has been proved recently (Głowacki [13]) that this algebra is inverse-closed as well.

Let us specify the notion to the Heisenberg group which is the group under study in this paper. For the sake of simplicity, let us consider here only the one-dimensional case. The underlying manifold of $\mathbb {H}$ is $\mathbb {R}^{3}$ which we write down as

$$\begin{aligned} \mathbb {H}=\mathbb {H}_{1}\oplus \mathbb {H}_{2}=\mathbb {R}^{2}\oplus \mathbb {R}. \end{aligned}$$

In these coordinates the group law is

$$\begin{aligned} (w,t)\circ (v,s)=(w+v,t+s+w_{1}v_{2}), \end{aligned}$$

where $w=(w_{1},w_{2})$, $v=(v_{1},v_{2})\in \mathbb {R}^{2}$, $t,s\in \mathbb {R}$. There are many choices of compatible dilations, but the most natural is

$$\begin{aligned} \delta _{r}(w,t)=(rw,r^{2}t),\qquad r>0. \end{aligned}$$

In this setting the size condition for a Calderón–Zygmund kernel reads

$$\begin{aligned} |\partial _{w}^{\alpha }\partial _{t}^{\beta }K(w,t)|\lesssim (\Vert w\Vert +|t|^{1/2})^{-4-|\alpha |-2\beta }. \end{aligned}$$

The cancellation condition doesn’t get any simpler, so we do not repeat it here.

We are going to compare these conditions with the estimates that define flag kernels which are the main object of study in this paper. Flag kernels were introduced by Müller–Ricci–Stein [14] and Nagel–Ricci–Stein [15] in their study of Marcinkiewicz multipliers (the first paper) and CR manifolds (the other one). These kernels are much more singular than the Calderón–Zygmund kernels. Accordingly, the definition is more complex. We consider a tempered distribution K on $\mathbb {H}$ which is smooth for $w\ne 0$ and satisfies the estimates

$$\begin{aligned} |\partial _{w}^{\alpha }\partial _{t}^{\beta }K(w,t)|\lesssim \Vert w\Vert ^{-2-|\alpha |}(\Vert w\Vert +|t|^{1/2})^{-2-2\beta },\quad w\ne 0, \end{aligned}$$

as well as the following three cancellation conditions:

• 1) For every $\varphi \in \mathcal {S}(\mathbb {H}_{1})$, the distribution

  $$\begin{aligned} f\mapsto \int _{\mathbb {H}}K(w,t)\varphi (w)f(t)dwdt \end{aligned}$$

  is a Calderón–Zygmund kernel on $\mathbb {H}_{2}$,

• 2) For every $\varphi \in \mathcal {S}(\mathbb {H}_{2})$, the distribution

  $$\begin{aligned} f\mapsto \int _{\mathbb {H}}K(w,t)f(w)\varphi (t)dwdt \end{aligned}$$

  is a Calderón–Zygmund kernel on $\mathbb {H}_{1}$,

• 3) For every $\varphi \in \mathcal {S}(\mathbb {H})$,

  $$\begin{aligned} \sup _{R>0}|\int _{\mathbb {H}}K(w,t)\varphi (Rw,R^{2}t)dwdt|<\infty . \end{aligned}$$

Finally, for given $\alpha ,\beta$, the estimates are uniform with respect to $\varphi$ if $\varphi$ stays in a bounded set in the respective Schwartz space.

The operators with flag kernels share some properties with the Calderón–Zygmund operators. They are bounded on $L^{p}(\mathbb {G})$-spaces and form an algebra (see Müller–Ricci–Stein [14], Nagel–Ricci–Stein [15], Nagel–Ricci–Stein–Wainger [16], Głowacki [11], Głowacki [12]). We are, however, interested in the inversion problem for this class. Before going any further, let us pause for a moment and consider the simplest case, namely that of an Abelian group ${\mathbb {R}^n}$. Then, the Fourier transform $\widehat{K}$ is a function on ${\mathbb {R}^n}$ which is smooth away from the origin and satisfies the estimates

$$\begin{aligned} |\partial ^{\alpha }_{\xi }\widehat{K}(\xi )|\lesssim |\xi |^{-|\alpha |},\quad \xi \ne 0. \end{aligned}$$

These estimates are, equivalent to the ones defining the Calderón–Zygmund kernel. If the operator $\mathrm{Op}(K)$ is invertible, then

$$\begin{aligned} |\widehat{K}(\xi )|\geqslant c>0,\ \ \ 0\ne \xi \in {\mathbb {R}^n}, \end{aligned}$$

and it is directly checked that $\widehat{L}=1/\widehat{K}$ satisfies analogous estimates, so that L is a flag kernel such that $L\star K=K\star L=\delta _{0}$.

A similar idea works for the Heisenberg group $\mathbb {H}$. Let $\pi ^{\lambda }$ denote the Schrödinger representation of $\mathbb {H}$ with the Planck constant $\lambda \ne 0$, If K is a flag kernel on $\mathbb {H}$ such that the operator $\mathrm{Op}(K)$ is invertible, then, for every $\lambda \ne 0$, the operator $\pi _{K}^{\lambda }$ is invertible (Theorem 7.4) and can be regarded as a pseudodifferential operator in a suitable class (Section 6 and 8). By the Beals theorem, the inverse belongs to the same class (Theorem 8.1). Now, the estimates are uniform in $\lambda$, so one can recover the kernel of the inverse operator from the kernels of $(\pi _{K}^{\lambda })^{-1}$ and show that it is a flag kernel (Theorem 8.3). Thus the algebra of the operators with flag kernels on the Heisenberg group turns out to be inverse-closed (Theorem 3.3). We believe that similar method could be used in the case of a general 2-step nilpotent Lie group.

There is a technicality in the proof we want to comment on. There exists no universal definition of the extension of the unitary representation to a space of distributions. One has to rely on specific properties of the distribution space in question. Everything works fine for distributions with compact support. A Calderón–Zygmund kernel can be split into a compactly supported part and a part which is square integrable, so there is no problem with the definition of $\pi _{K}^{\lambda }$. No such splitting is available for flag kernels. Instead we modify the domain of the distribution. Originally, a distribution is a functional on the Schwartz space. We introduce two other spaces on which flag kernels can be regarded as continuous functionals (Definitions 3 and 4). The cancellation conditions are important here. We also take adventage of a functional calculus of Głowacki [10] (Lemma 6.1). Once $\pi _{K}^{\lambda }$ is defined for our flag kernel, we can follow the path outlined above.

As we learned recently, Bony–Chemin in the introduction to their work [3] mentioned a gap in the proof of the Lemma 4.6 in the work of Beals [2]. This lemma is a key to a thesis of Theorem 4.7. We have changed our argument so as to be able to use the Theorem 3.2 from Beals [1] instead.

2 Preliminaries

The main structure of this work is the Heisenberg group. As a set it is

$$\begin{aligned} \mathbb {H}^{n}={\mathbb {R}^n}\times {\mathbb {R}^n}\times \mathbb {R}. \end{aligned}$$

Elements of the group will usually be denoted by

$$\begin{aligned} \mathbb {H}^{n}\ni h=(x,y,t)=(v,t). \end{aligned}$$

The group multiplication is

$$\begin{aligned} (x,y,t)\cdot (x',y',t')=(x+x',y+y',t+t'+xy'). \end{aligned}$$

We define a homogeneous norm on ${\mathbb {H}^n}$ as

$$\begin{aligned} |h|=\Vert v\Vert +|t|^{\frac{1}{2}}=\sum _{i=1}^{2n}|v_{i}|+|t|^{\frac{1}{2}} \end{aligned}$$

with the corresponding family of dilations

$$\begin{aligned} \delta _{j}(h)=\delta _{j}(v,t)=(jv,j^{2}t),\qquad j>0, \end{aligned}$$

in the sense that $|\delta _{j}(h)|=j|h|$. The set $\{\delta _{j}\}_{j>0}$ actually forms a group of automorphisms. The homogeneous dimension is the number $Q=2n+2$. We will use the designations

$$\begin{aligned} \partial ^{\gamma }_{h}=\partial _{v}^{\alpha }\partial _{t}^{\beta }=\partial _{v_{1}}^{\alpha _{1}}\partial _{v_{2}}^{\alpha _{2}}...\partial _{v_{2n}}^{\alpha _{2n}}\partial _{t}^{\beta } \end{aligned}$$

and

$$\begin{aligned} |\gamma |=|\alpha |+2\beta =\sum _{i=1}^{2n}\alpha _{i}+2\beta , \end{aligned}$$

where $\alpha =(\alpha _{1},\alpha _{2},\dots ,\alpha _{2n}),\ \alpha _{k},\beta \in \mathbb {N}$. One of the main tools is the abelian Fourier transform defined by the formula

$$\begin{aligned} \widehat{f}(\zeta ):=\int _{{\mathbb {H}^n}}f(h)e^{-2\pi ih\zeta }dh, \end{aligned}$$

where

It can be first defined for Schwartz functions

$$\begin{aligned} \mathcal {S}(\mathbb {H}^{n})=\{f\in \mathcal {C}^{\infty }(\mathbb {H}^{n}):\forall N\in \mathbb {N}\sup _{h\in {\mathbb {H}^n}}\max _{|\gamma |\leqslant N}|\partial _{h}^{\gamma }f(h)|(1+|h|)^{N}<\infty \} \end{aligned}$$

and then lifted to the Lebesque space of the square-integrable functions

$$\begin{aligned} L^{2}({\mathbb {H}^n})=\{f:\int _{{\mathbb {H}^n}}|f(h)|^{2}dh<\infty \} \end{aligned}$$

or to the space of tempered distributions $\mathcal {S}'({\mathbb {H}^n})$ which is the space of all continuous linear functionals on $\mathcal {S}({\mathbb {H}^n})$ in the sense of the usual seminorm topology. Note that we use the non-isotropic norm to define the Schwartz space. Nevertheless, this space is equivalent to the standard one defined by isotropic estimates. For $S\in \mathcal {S}'({\mathbb {H}^n}),\ f\in \mathcal {S}({\mathbb {H}^n})$ one can put

$$\begin{aligned} \langle \widehat{S},f\rangle :=\langle S,\widehat{f}\rangle . \end{aligned}$$

Let also

$$\begin{aligned} f^{\star }(x)=\overline{f(x^{-1})},\quad \langle S^{\star },f\rangle :=\langle S,f^{\star }\rangle ,\quad S\in \mathcal {S}'({\mathbb {H}^n}),\ f\in \mathcal {S}({\mathbb {H}^n}). \end{aligned}$$

By $\delta _{0}$ we will denote the Dirac distribution.

3 Flag Kernels and Their Convolution Operators

Automorphisms $\{\delta _{j}\}_{j>0}$ decompose our group ${\mathbb {H}^n}$ into their eigenspaces

$$\begin{aligned} \mathbb {G}_{1}\oplus \mathbb {G}_{2}\ni (v,t). \end{aligned}$$

Theorem 2.3.9 of Nagel–Ricci–Stein [15] says that there is a one-to-one correspodence between flag kernels and their multipliers. It allows us to define flag kernels as follows.

Definition 1

Let ${\mathbb {H}^n}$ be the Heisenberg group and

$$\begin{aligned} \mathbb {H}_{n}=\mathbb {G}_{1}^{\star }\oplus \mathbb {G}_{2}^{\star }\ni (w,\lambda ) \end{aligned}$$

the dual vector space to ${\mathbb {H}^n}$. We say that a tempered distribution K is a flag kernel iff its Fourier transform $\widehat{K}$ agrees with a smooth function outside of hyperspace $\{(w,\lambda ):\lambda =0\}$ and satisfies the estimates

$$\begin{aligned} |\partial _{w}^{\alpha }\partial _{\lambda }^{\beta }\widehat{K}(w,\lambda )|\leqslant c_{\alpha ,\beta }(\Vert w\Vert +|\lambda |^{1/2})^{-|\alpha |}|\lambda |^{-\beta },\ \ \ all \alpha ,\beta . \end{aligned}$$

Observe that in particular $\widehat{K}$ belongs to $L^{\infty }(\mathbb {H}_{n})$. For $f\in \mathcal {S}({\mathbb {H}^n})$, $K\in \mathcal {S}'({\mathbb {H}^n})$ we define their convolution as

$$\begin{aligned} K\star f(h):=\langle K,\, _{h^{-1}}\!\widetilde{f}\rangle =\int _{{\mathbb {H}^n}}K(h')f(h'^{-1}h)dh', \end{aligned}$$

where $\widetilde{f}(h):=f(h^{-1})$ and $_{x}f(h):=f(xh)$. Our point of departure are the following two theorems:

Theorem 3.1

Let K be a flag kernel on the Heisenberg group. Then

$$\begin{aligned} \Vert K\star f\Vert _{2}\lesssim \Vert f\Vert _{2},\qquad f\in \mathcal {S}({\mathbb {H}^n}). \end{aligned}$$

Hence we have an $L^{2}$-bounded operator $\mathrm{Op}(K)f:=K\star f$.

Observe that our convolution operators are right-invariant even though we often refer the reader to obviously equivalent left-invariant results.

Theorem 3.2

Let K, S be flag kernels on the Heisenberg group ${\mathbb {H}^n}$ and

$$\begin{aligned} T:=\mathrm{Op}(K)\mathrm{Op}(S). \end{aligned}$$

Then, there exists a flag kernel L such that $T=\mathrm{Op}(L)$.

Thus the flag kernels give rise to convolution operators, bounded on $L^{2}({\mathbb {H}^n})$, which form a subalgebra of $\mathcal {B}(L^{2}({\mathbb {H}^n}))$. For convenience we will write $L=K\star S$.

These are theorems of Nagel–Ricci–Stein [15] who proved them for a class of homogeneous groups which includes all two-step homogeneous groups (Theorems 2.6.B and 2.7.2). Partial results can be found in an earlier paper of Müller–Ricci–Stein [14] (Theorem 3.1). They were subsequently generalized for all homogeneous group independently and virtually simultanously by Nagel–Ricci–Stein–Wainger [16] and Głowacki [11]. The aim of this paper is to prove the following theorem.

Theorem 3.3

Let ${\mathbb {H}^n}$ be the Heisenberg group. Let K be a flag kernel on ${\mathbb {H}^n}$. Suppose that the operator $\mathrm{Op}(K)$ is invertible on $L^{2}({\mathbb {H}^n})$. Then there exists a flag kernel L, such that for all $f\in L^{2}({\mathbb {H}^n})$

$$\begin{aligned} \mathrm{Op}(K)^{-1}f=L\star f=\mathrm{Op}(L)f. \end{aligned}$$

Observe that $\mathrm{Op}(K)$ is right-invariant. Further the same holds for its inversion. By general theory it follows that there exists a tempered distribution L such that $\mathrm{Op}(K)^{-1}f=L\star f$. Now it suffices to show that L is a flag kernel. In the following considerations we can assume that the flag kernel K is symmetric, i.e. $K=K^{\star }$. In fact if Theorem 3.3 is true for such kernels, let us pick an arbitrary flag kernel K. Then, we can consider kernels $K^{\star }\star K$ and $K\star K^{\star }$ which are symmetric. By Theorem 3.2 they are both flag kernels. Therefore, by Theorem 3.3, there exist flag kernels S, T such that

Again, by Theorem 3.2 and associativity, it follows that there exist flag kernels $L_{1},L_{2}$ such that

The identity

$$\begin{aligned} L_{1}=L_{1}\star (K\star L_{2})=(L_{1}\star K)\star L_{2}=L_{2} \end{aligned}$$

ends the proof of our theorem for an arbitrary flag kernel K.

4 Schrödinger Representation

Definition 2

For $\lambda \ne 0$ and $h\in {\mathbb {H}^n}$ we define the family of operators

$$\begin{aligned} \{\pi _{h}^{\lambda }:h\in {\mathbb {H}^n},\lambda \ne 0\}, \end{aligned}$$

all acting on the same $L^{2}({\mathbb {R}^n})$ by the following formula

$$\begin{aligned} \pi _{h}^{\lambda }f(s):=\left\{ \begin{array}{ll} e^{2\pi i\lambda t}e^{2\pi i\sqrt{\lambda }ys}f(s+\sqrt{\lambda }x);&{}\quad \lambda >0, \\ e^{2\pi i\lambda t}e^{2\pi i\sqrt{|\lambda |}ys}f(s-\sqrt{|\lambda |}x);&{}\quad \lambda <0. \end{array} \right. \end{aligned}$$

For a Hilbert space $\mathcal {H}$, denote by $\mathcal {U}(\mathcal {H})$, $\mathcal {B}(\mathcal {H})$ the spaces of all unitary and bounded operators, respectively. It is well-known (see, e.g. Folland [8], Section 1.3) that, for every $\lambda \ne 0$,

$$\begin{aligned} {\mathbb {H}^n}\ni h\longmapsto \pi _{h}^{\lambda }\in \mathcal {U}(L^{2}({\mathbb {R}^n})) \end{aligned}$$

is a unitary representation on the Hilbert space $L^{2}({\mathbb {R}^n})$.

$$\begin{aligned} L^{1}({\mathbb {H}^n})\ni f\longmapsto \pi _{f}^{\lambda }\in \mathcal {B}(L^{2}({\mathbb {R}^n})),\\ \pi _{f}^{\lambda }g(s):=\int _{{\mathbb {H}^n}}f(h)\pi _{h}^{\lambda }g(s)dh \end{aligned}$$

is a representation of $\star$-algebra $L^{1}({\mathbb {H}^n})$ on the Hilbert space $L^{2}({\mathbb {R}^n})$.

5 Useful Notation

For $f,g\in \mathcal {S}({\mathbb {R}^n})$ we define the function

$$\begin{aligned} c_{f,g}(x,y):=\int _{{\mathbb {R}^n}}e^{2\pi iyu}f(u+x)g(u)du. \end{aligned}$$

In particular

$$\begin{aligned} \widehat{c_{f,g}}(\xi ,\eta )=\widehat{f}(\xi )g(\eta )e^{2\pi i\xi \eta } \end{aligned}$$

(5.1)

Let also

$$\begin{aligned} C_{f,g}^{\lambda }(x,y,t):=\langle \pi _{h}^{\lambda }f,g\rangle \end{aligned}$$

Let $\lambda >0$. One can calculate that

$$\begin{aligned} C_{f,g}^{\lambda }(x,y,t)=\int _{{\mathbb {R}^n}}\pi _{(x,y,t)}^{\lambda }f(u)g(u)du=e^{2\pi it\lambda }c_{f,g}(\sqrt{\lambda }x,\sqrt{\lambda }y). \end{aligned}$$

We also have

$$\begin{aligned} \widehat{c_{f,g}\circ \delta _{\sqrt{\lambda }}}(\xi ,\eta )&=\int \int \int e^{-2\pi ix\xi }e^{-2\pi iy\eta }e^{2\pi i\sqrt{\lambda }yu}f(u+\sqrt{\lambda }x)g(u)dudxdy \\&=\int \int \int \!\lambda ^{-n/2}e^{-2\pi ix\frac{\xi }{\sqrt{\lambda }}}e^{2\pi iu\frac{\xi }{\sqrt{\lambda }}}e^{-2\pi iy\eta }e^{2\pi i\sqrt{\lambda }yu}f(x)g(u)dudxdy\\&=\int \int \lambda ^{-n/2}\widehat{f}\left( \frac{\xi }{\sqrt{\lambda }}\right) e^{2\pi iu\left( \frac{\xi }{\sqrt{\lambda }}+\sqrt{\lambda }y\right) }e^{-2\pi iy\eta }g(u)dudy\\&=\int \lambda ^{-n/2}\widehat{f}\left( \frac{\xi }{\sqrt{\lambda }}\right) g^{\vee }\left( \frac{\xi }{\sqrt{\lambda }}+\sqrt{\lambda }y\right) e^{-2\pi iy\eta }dy\\&=\int |\lambda |^{-n}\widehat{f}\left( \frac{\xi }{\sqrt{\lambda }}\right) g^{\vee }(y)e^{-2\pi iy\frac{\eta }{\sqrt{\lambda }}}e^{2\pi i\frac{\xi }{\sqrt{\lambda }}\frac{\eta }{\sqrt{\lambda }}}dy\\&=|\lambda |^{-n}\widehat{f}\left( \frac{\xi }{\sqrt{\lambda }}\right) g\left( \frac{\eta }{\sqrt{\lambda }}\right) e^{2\pi i\frac{\xi }{\sqrt{\lambda }}\frac{\eta }{\sqrt{\lambda }}}=|\lambda |^{-n}\widehat{c_{f,g}}\left( \frac{\xi }{\sqrt{\lambda }},\frac{\eta }{\sqrt{\lambda }}\right) . \end{aligned}$$

Moreover

$$\begin{aligned} \widehat{C_{f,g}^{\lambda }}(\xi ,\eta ,r)=\widehat{c_{f,g}\circ \delta _{\sqrt{\lambda }}}\otimes \widehat{e^{2\pi i(\cdot )\lambda }}(\xi ,\eta ,r)=|\lambda |^{-n}\widehat{c_{f,g}}\otimes \delta _{\lambda }\left( \frac{\xi }{\sqrt{\lambda }},\frac{\eta }{\sqrt{\lambda }},r\right) , \end{aligned}$$

(5.2)

where $\delta _{\lambda }$ is a Dirac distribution supported at $\lambda$. For $\lambda <0$ the above formula should be slightly modified. For such $\lambda$ one can get analogously

$$\begin{aligned} \widehat{C_{f,g}^{\lambda }}(\xi ,\eta ,r)=-|\lambda |^{-n}\widehat{c_{f,g}}\otimes \delta _{\lambda }\left( -\frac{\xi }{|\lambda |^{1/2}},\frac{\eta }{|\lambda |^{1/2}},r\right) . \end{aligned}$$

Suppose that a is a function on ${\mathbb {R}^n}\times {\mathbb {R}^n}$ which is bounded or square-integrable. Then, the weakly defined operator

$$\begin{aligned} \langle Af,g\rangle&=\int \int e^{2\pi i\xi \eta }a(\xi ,\eta )\widehat{f}(\xi )g(\eta )d\eta d\xi \\&=\int \int a(\xi ,\eta )\widehat{c_{f,g}}(\xi ,\eta )d\xi d\eta =\langle a,\widehat{c_{f,g}}\rangle \end{aligned}$$

is a continuous mapping from $\mathcal {S}({\mathbb {R}^n})$ to $\mathcal {S}' ({\mathbb {R}^n})$. It is often denoted by $A=a(x,D)$ and called a pseudodifferential operator with the Kohn–Nirenberg (KN) symbol a.

6 The Class $\mathcal {S}_{0}$ and the Operator $\pi _{K}^{\lambda }$

Let g be a function on $\mathbb {H}_{n}$ such that $g(w,\lambda )=\varphi (\lambda )$, where $\varphi \in \mathcal {C}_{c}^{\infty }(\mathbb {G}_{2}^{\star })$. Then $g^{\vee }(u,t)=\delta _{0}\otimes \varphi ^{\vee }(u,t)$, where the ”$\vee$” sign denotes the inverse Fourier transform. Observe that if for example $f\in \mathcal {S}({\mathbb {H}^n})$, then

$$\begin{aligned} f\star g^{\vee }(h)&=\int f(hr^{-1})dg^{\vee } (r)=\int f(h-r)dg^{\vee } (r) \\&=\int f(r^{-1}h)dg^{\vee } (r)=g^{\vee }\star f(h), \end{aligned}$$

so $g^{\vee }$ is a central measure. We will need a notion of the $\lambda$-support of a function f. By definition a real number $\lambda _{0}$ is not in the $\lambda$-$\mathrm {supp\,}(f)$ iff there exists $\varepsilon$ such that no point $(w,\lambda )$, where $\lambda \in (\lambda _{0}-\varepsilon ,\lambda _{0}+\varepsilon )$, is in the support of f.

Definition 3

We say that a Schwartz function f is in $\mathcal {S}_{0} ({\mathbb {H}^n})$ iff

$$\begin{aligned} (\exists \varepsilon >0)\forall \varphi \in \mathcal {C}_{c}^{\infty }((-\varepsilon ,\varepsilon )\cup (-1/\varepsilon ,-\infty )\cup (1/\varepsilon ,\infty ))\,\widehat{f}\varphi =0, \end{aligned}$$

that is, iff $\lambda$-$\mathrm {supp\,}(\widehat{f})$ is bounded and does not contain 0.

Lemma 6.1

Suppose $f\in \mathcal {S}_{0} ({\mathbb {H}^n})$ and K is a flag kernel. Then $K\star f$ is in $\mathcal {S}_{0} ({\mathbb {H}^n})$.

Proof

Let us define first $a\# b:=(a^{\vee }\star b^{\vee })^{\wedge }$. For the sake of convenience of the reader we will write here $*_{e}$ instead of $*$ whenever the convolution becomes abelian. As f is in $\mathcal {S}_{0}({\mathbb {H}^n})$ take $\varepsilon ,\varphi$ which satisfy the condition of Definition 3. We have

$$\begin{aligned} \widehat{K\star f}\varphi =(K\star f\star _{e}\varphi ^{\vee })^{\wedge }=\widehat{K}\#\widehat{f}\varphi =\widehat{K}\# 0=0, \end{aligned}$$

so the same $\varepsilon$ works also for $K\star f$. It remains to explain why $K\star f$ is an element of $\mathcal {S}({\mathbb {H}^n})$. Let $\psi \in \mathcal {C}_{c}^{\infty }(\mathbb {R}\setminus \{0\})$ be equal to 1 on $\lambda$-$\mathrm {supp\,}$ of $\widehat{f}$. Observe that $\psi ^{\vee }$ can be thought of as a central measure. Thus

$$\begin{aligned} K\star f=K\star f\star _{e}\psi ^{\vee }=K\star _{e}\psi ^{\vee }\star _{e} f=K_{1}\star f, \end{aligned}$$

where $\widehat{K_{1}}$ is smooth and

$$\begin{aligned} |\partial _{w}^{\alpha }\partial _{\lambda }^{\beta }\widehat{K_{1}}(w,\lambda )|\leqslant c_{\alpha ,\beta }(1+\Vert w\Vert +|\lambda |^{1/2})^{-|\alpha |}(1+|\lambda |)^{-\beta }. \end{aligned}$$

Now if we write that $a\in Sym^{N,M}({\mathbb {H}^n})$ iff

$$\begin{aligned} |\partial _{w}^{\alpha }\partial _{\lambda }^{\beta }\widehat{a}(w,\lambda )|\leqslant c_{\alpha ,\beta }(1+\Vert w\Vert +|\lambda |^{1/2})^{-|\alpha |-N}(1+|\lambda |)^{-\beta -M}, \end{aligned}$$

then $K_{1}\in Sym^{0,0}({\mathbb {H}^n})$, $f\in Sym^{N,M}({\mathbb {H}^n})$ for all N, M because it is a Schwartz function. We use here an application of the symbolic calculus of Głowacki governed by the metric

$$\begin{aligned} g_{(w,\lambda )}(u,r)=\frac{\Vert u\Vert }{1+\Vert w\Vert +|\lambda |^{1/2}}+\frac{|r|}{(1+|\lambda |^{1/2})^{2}};\qquad (w,\lambda )\in \mathbb {H}_{n},\ (u,r)\in \mathbb {H}_{n}. \end{aligned}$$

.

Theorem 6.2

(Głowacki [10], Corollary 5.2) Let

$$\begin{aligned} a\in Sym^{m_{1},n_{1}}({\mathbb {H}^n}),\ b\in Sym^{m_{2},n_{2}}({\mathbb {H}^n}). \end{aligned}$$

Then, $a*b\in Sym^{m_{1}+m_{2},n_{1}+n_{2}}({\mathbb {H}^n})$.

We have

$$\begin{aligned} K\star f&=K_{1}\star f\in Sym^{0,0}({\mathbb {H}^n})\star \bigcap _{N,M}Sym^{N,M}({\mathbb {H}^n})\\&\subseteq \bigcap _{N,M}Sym^{0,0}({\mathbb {H}^n})\star Sym^{N,M}({\mathbb {H}^n})\\&\subseteq \bigcap _{N,M}Sym^{N,M}({\mathbb {H}^n})\cong \mathcal {S}({\mathbb {H}^n}). \end{aligned}$$

$\square$

Lemma 6.3

The class $S_{0} ({\mathbb {H}^n})$ is dense in $L^{2}({\mathbb {H}^n}).$

Proof

Let $g\in L^{2}({\mathbb {H}^n})$ be such that $\forall f\in S_{0}({\mathbb {H}^n})\ \langle g,f\rangle =0.$ Then $\langle \hat{g},\hat{f}\rangle =0$. Hence $\mathrm {supp\,}\hat{g}\subseteq \mathbb {R}^{2n}\times \{0\}.$ But it implies that $g=0$ almost everywhere. $\square$

Lemma 6.4

The Gärding space

$$\begin{aligned} \mathcal {G}^{\lambda }:=\{\pi _{\varphi }^{\lambda }f: \varphi \in S_{0}({\mathbb {H}^n}), f\in L^{2}({\mathbb {R}^n})\} \end{aligned}$$

is dense in $L^{2}({\mathbb {R}^n}).$

Proof

Take any $g\in L^{2}({\mathbb {R}^n})$ such that for all $\varphi \in S_{0},\ f\in L^{2}({\mathbb {R}^n})$ we have $\langle g,\pi _{\varphi }^{\lambda }f\rangle =0.$ We will show that $g=0$ a.e. Consider only those functions $\varphi$ which can be decomposed as $\varphi (x,y,t)=\varphi _{1}(x,y)\varphi _{2}(t).$ Then

$$\begin{aligned} 0&=\langle g,\pi _{\varphi }^{\lambda }f\rangle =\langle g,\int _{{\mathbb {H}^n}}\varphi (h)\pi _{h}^{\lambda }fdh\rangle =\int _{{\mathbb {H}^n}}\varphi (h)\langle g,\pi _{h}^{\lambda }f\rangle dh\\&=\int _{\mathbb {R}}\left( \int _{\mathbb {R}^{2n}}\varphi _{1}(x,y)\langle g,\pi _{h}^{\lambda }f\rangle dxdy\right) \varphi _{2}(t)dt=\int _{\mathbb {R}}F(t)\varphi _{2}(t)dt. \end{aligned}$$

It follows that $\langle \hat{F},\hat{\varphi _{2}}\rangle =0$, which, by the structure of $\mathcal {S}_{0} ({\mathbb {H}^n})$, implies that

$$\begin{aligned} \mathrm {supp\,}\hat{F}\subseteq \{0\}. \end{aligned}$$

Hence $\hat{F}=\sum _{n=0}^{N}c_{n}\delta _{0}^{(n)}.$ By the Schwarz inequality $||F||_{\infty }\le ||f||_{2}||g||_{2}||\varphi _{1}||_{1}$. So $F\in L^{\infty }(\mathbb {R})$ and at the same time $F(t)=\sum _{n=0}^{N}c_{n}t^{n}.$ Consequently, there is no other option than $F=const$, which means that

$$\begin{aligned} \int _{\mathbb {R}^{2n}}\varphi _{1}(x,y)\langle g,\pi _{h}^{\lambda }f\rangle dxdy=c_{\varphi _{2}}. \end{aligned}$$

By the density of $S(\mathbb {R}^{2n})$ in $L^{2}(\mathbb {R}^{2n})$, we have that the expression $\langle g,\pi _{h}^{\lambda }f\rangle$ does not depend on the variable t. So $(\pi _{(0,0,t)}^{\lambda }-I)g=0$ for all t, which leads to a contradiction unless $g=0$ a.e. $\square$

Lemma 6.5

Let K, L be flag kernels such that $K|_{\mathcal {S}_{0}({\mathbb {H}^n})}=L|_{\mathcal {S}_{0}({\mathbb {H}^n})}$. Then, $K=L$.

Proof

If $\langle K,f\rangle =\langle L,f\rangle$ for $f\in \mathcal {S}_{0}({\mathbb {H}^n})$, then $\langle K-L,f\rangle =0$ and so $\langle \widehat{K-L},\widehat{f}\rangle =0$. Therefore from the definition of the class $\mathcal {S}_{0}({\mathbb {H}^n})$, we have that $\widehat{K-L}=0$ outside of the center of $\mathbb {H}_{n}$ which has measure 0. Thus $\widehat{K}=\widehat{L}$ as elements of $L^{\infty }(\mathbb {H}_{n})$, so $K=L$ in $\mathcal {S}'({\mathbb {H}^n})$. $\square$

Definition 4

We denote by $\mathcal {B}_{0}({\mathbb {H}^n})$ the class of all smooth functions such that their Fourier transforms are bounded measures whose $\lambda$-support does not contain 0. One can norm this space with $\Vert f\Vert _{\mathcal {B}_{0}}=\Vert \widehat{f}\Vert _{\mathcal {M}}$, where $\Vert \cdot \Vert _{\mathcal {M}}$ denotes the total variation of a measure.

Observe that $\mathcal {S}_{0}({\mathbb {H}^n})\subset \mathcal {B}_{0}({\mathbb {H}^n})$. $\mathcal {B}_{0}({\mathbb {H}^n})$ is convenient because it contains objects of type $C_{f,g}^{\lambda }$, as by 5.2 and 5.1, we have

$$\begin{aligned} \Vert C_{f,g}^{\lambda }\Vert _{\mathcal {B}_{0}}\leqslant \Vert \widehat{c_{f,g}}\Vert _{1}\leqslant \Vert \widehat{f}\Vert _{1}\Vert g\Vert _{1}. \end{aligned}$$

Now according to Lemma 6.5 we can restrict the algebra of flag kernels to $\mathcal {S}_{0}({\mathbb {H}^n})$ and then extend it to $\mathcal {B}_{0}({\mathbb {H}^n})$ by the formula

$$\begin{aligned} \langle K,f\rangle =\int _{\mathbb {H}_{n}}\widehat{K}(w,\lambda )d\widehat{f}(w,\lambda ). \end{aligned}$$

Continuity is gained for free as $|\langle K,f\rangle |\leqslant \Vert \widehat{K}\Vert _{\infty }\Vert \widehat{f}\Vert _{\mathcal {M}}$. Now we can define the representation of a flag kernel. Suppose first that $K\in \mathcal {S}_{0}({\mathbb {H}^n})$. Then,

$$\begin{aligned} \langle \pi _{K}^{\lambda }f,g\rangle =\langle \int _{{\mathbb {H}^n}} K(h)\pi _{h}^{\lambda }fdh,g\rangle =\int _{{\mathbb {H}^n}} K(h)\langle \pi _{h}^{\lambda }f,g\rangle dh=\langle K,C^{\lambda }_{f,g}\rangle , \end{aligned}$$

for $f,g\in \mathcal {S}({\mathbb {R}^n})$. Hence, for every flag kernel we put $\langle \pi _{K}^{\lambda }f,g\rangle :=\langle K,C^{\lambda }_{f,g}\rangle$ as a weak definition of its representation. Observe next that if $\lambda >0$

$$\begin{aligned} \langle \pi _{K}^{\lambda }f,g\rangle&=\int \int K(u,t)c_{f,g}(|\lambda |^{1/2}u)e^{2\pi it\lambda }dudt\\&=\int \int \int \widetilde{\widehat{K}}(\xi ,\eta ,r)|\lambda |^{-n}\widehat{c_{f,g}}\left( \frac{\xi }{|\lambda |^{1/2}},\frac{\eta }{|\lambda |^{1/2}}\right) d\xi d\eta d\delta _{\lambda }(r)\\&=\int \int \widetilde{\widehat{K}}(|\lambda |^{1/2}\xi ,|\lambda |^{1/2}\eta ,\lambda )\widehat{c_{f,g}}(\xi ,\eta )d\xi d\eta . \end{aligned}$$

Similar calculation for $\lambda <0$ leads to a conclusion that $\pi _{K}^{\lambda }$ is a pseudodifferential operator with the KN symbol

$$\begin{aligned} \widetilde{\widehat{K}}(\mathrm {sgn\,}(\lambda )|\lambda |^{1/2}\xi ,|\lambda |^{1/2}\eta ,\lambda ). \end{aligned}$$

Lemma 6.6

Let K be a flag kernel and $\varphi$ in $\mathcal {S}_{0} ({\mathbb {H}^n})$. Then, the operators $\pi _{K\star \varphi }^{\lambda }$ and $\pi _{K}^{\lambda }\pi _{\varphi }^{\lambda }$ are equal.

Proof

First one can calculate that

$$\begin{aligned} C_{\pi _{\varphi }^{\lambda }f,g}^{\lambda }(h)&=\int \pi _{h}^{\lambda }\pi _{\varphi }^{\lambda }f(s)g(s)ds=\int \pi _{h}^{\lambda }\int \varphi (h')\pi _{h'}^{\lambda }f(s)dh'g(s)ds\\&=\int \varphi (h')\int \pi _{hh'}^{\lambda }f(s)g(s)dsdh'\!=\!\!\int \widetilde{\varphi }(h')C_{f,g}^{\lambda }(hh'^{-1})dh' \!=\!C_{f,g}^{\lambda }\star \widetilde{\varphi }(h). \end{aligned}$$

Now using fact that $K\star \varphi$ is in $\mathcal {S}_{0} ({\mathbb {H}^n})$

$$\begin{aligned} \langle \pi _{K\star \varphi }^{\lambda }f,g\rangle&=\int K\star \varphi (h)\langle \pi _{h}^{\lambda }f,g\rangle dh=\int K\star \varphi (h)C_{f,g}^{\lambda }(h)dh\\&=\int K(h)C_{f,g}^{\lambda }\star \widetilde{\varphi }(h)dh=\int K(h)C_{\pi _{\varphi }^{\lambda }f,g}^{\lambda }(h)dh=\langle \pi _{K}^{\lambda }\pi _{\varphi }^{\lambda }f,g\rangle . \end{aligned}$$

$\square$

Corollary 6.7

Let $K_{1},K_{2}$ be flag kernels. The operators $\pi _{K_{1}\star K_{2}}^{\lambda }$ and $\pi _{K_{1}}^{\lambda }\pi _{K_{2}}^{\lambda }$ are equal.

Proof

As flag kernels form an algebra, the above lemma implies

$$\begin{aligned} \pi _{K_{1}\star K_{2}}^{\lambda }\pi _{\varphi }^{\lambda }f=\pi _{K_{1}\star K_{2}\star \varphi }^{\lambda }f=\pi _{K_{1}}^{\lambda }\pi _{K_{2}\star \varphi }^{\lambda }f=\pi _{K_{1}}^{\lambda }\pi _{K_{2}}^{\lambda }\pi _{\varphi }^{\lambda }f, \end{aligned}$$

(6.1)

where the second equality follows by the fact that $K_{2}\star \varphi \in \mathcal {S}_{0} ({\mathbb {H}^n})$. 6.1 proves that the operators agree on vectors of type $\pi _{\varphi }^{\lambda }f$ which are dense in $L^{2}({\mathbb {R}^n})$ when $\varphi \in \mathcal {S}_{0} ({\mathbb {H}^n})$, $f\in L^{2}({\mathbb {R}^n})$. $\square$

7 Representations of $L^{2}$

We start from a simple calculation of the KN symbol of $\pi _{f}^{\lambda }$, where f is a Schwartz function. Let $\lambda$ be positive. We have

$$\begin{aligned} \pi _{f}^{\lambda }u(s)&=\int f(x,y,t)e^{2\pi it\lambda }e^{2\pi i\sqrt{\lambda }ys}u(s+\sqrt{\lambda }x)dxdydt\\&=\int f(x,\sqrt{\lambda }s^{\vee },\lambda ^{\vee })u(s+\sqrt{\lambda }x)dx\\&=|\lambda |^{-n/2}\int f\left( \frac{x-s}{\sqrt{\lambda }},\sqrt{\lambda }s^{\vee },\lambda ^{\vee }\right) u(x)dx\\&=|\lambda |^{-n/2}\int \int f\left( \frac{x-s}{\sqrt{\lambda }},\sqrt{\lambda }s^{\vee },\lambda ^{\vee }\right) e^{2\pi ix\xi }\widehat{u}(\xi )d\xi dx\\&=\int \int f(x,\sqrt{\lambda }s^{\vee },\lambda ^{\vee })e^{2\pi i\sqrt{\lambda }x\xi }e^{2\pi is\xi }\widehat{u}(\xi )d\xi dx\\&=\int f^{\vee }(\sqrt{\lambda }\xi ,\sqrt{\lambda }s,\lambda )e^{2\pi is\xi }\widehat{u}(\xi )d\xi . \end{aligned}$$

Here the sign “$\vee$”, over arguments of f, has been used as a notation of a partial inverse Fourier transform in respective variables. As we can see in this case the symbol of $\pi _{f}^{\lambda }$ is also

$$\begin{aligned} a(\xi ,\eta )=\widetilde{\widehat{f}}(\mathrm {sgn\,}(\lambda )|\lambda |^{1/2}\xi ,|\lambda |^{1/2}\eta ,\lambda ). \end{aligned}$$

Let for a moment $x,y\in {\mathbb {H}^n}$ and A be a Hilbert–Schmidt operator on $\mathcal {S}({\mathbb {H}^n})$ with a kernel $\Omega$. One can calculate that

$$\begin{aligned} Au(x)\!=\!\int \Omega (x,y)u(y)dy\!=\!\int \Omega (x,y)\int e^{2\pi iy\xi }\widehat{u}(\xi )d\xi dy\!=\!\int \Omega (x,\xi ^{\vee })\widehat{u}(\xi )d\xi . \end{aligned}$$

It is easy to see that if a is the KN symbol of A, then

$$\begin{aligned} a(\xi ,\eta )=e^{-2\pi i\xi \eta }\Omega (\xi ,\eta ^{\vee }), \end{aligned}$$

and by the Plancharel formula,

The sign change on the first coordinate, in a situation where $\lambda$ is negative, have no impact on the obstacles with which we struggle. Thus from now on in all calculations we will disregard this difference.

Lemma 7.1

Let $f\in L^{2}({\mathbb {H}^n})$ and $\{f_{n}\}_{n}\subset \mathcal {S}_{0} ({\mathbb {H}^n})$ be such that $f_{n}\rightarrow f$ in $L^{2}$. For almost every $\lambda$, there exists a subsequence $\{f_{n_{k}(\lambda )}\}_{k}$ such that $\pi ^{\lambda }_{f_{n_{k}(\lambda )}}$ tend to an operator $A^{\lambda }$ in the Hilbert–Schmidt norm. $A^{\lambda }$ depends neither on the chosen sequence $f_{n}$ nor on its subsequence $f_{n_{k}(\lambda )}$. Moreover the KN symbol of $A^{\lambda }$ is $a_{\lambda }(w)=\widetilde{\widehat{f}}(|\lambda |^{1/2}w,\lambda )$.

Proof

By Plancherel’s formula

$$\begin{aligned} \Vert f_{n}-f\Vert _{2}^{2}&=\int |\widehat{f_{n}}(w,\lambda )-\widehat{f}(w,\lambda )|^{2}dwd\lambda \\&=\int |\lambda |^{n}\int |\widetilde{\widehat{f_{n}}}(|\lambda |^{1/2}w,\lambda )-\widetilde{\widehat{f}}(|\lambda |^{1/2}w,\lambda )|^{2}dwd\lambda \\&=\int |\lambda |^{n}\Vert \pi _{f_{n}}^{\lambda }-A^{\lambda }\Vert _{HS}^{2}d\lambda , \end{aligned}$$

where $A^{\lambda }$ is the Hilbert–Schmidt operator with the symbol $(w,\lambda )\mapsto \widetilde{\widehat{f}}(|\lambda |^{1/2}w,\lambda )$. By Fatou’s lemma $\liminf \Vert \pi _{f_{n}}^{\lambda }-A^{\lambda }\Vert _{HS}=0$ for almost every $\lambda$. Therefore, for almost every $\lambda$, there exists a subsequence $f_{n_{k}(\lambda )}$ such that

$$\begin{aligned} \Vert \pi _{f_{n_{k}(\lambda )}}^{\lambda }-A^{\lambda }\Vert _{HS}\rightarrow 0. \end{aligned}$$

$\square$

Let $\widehat{f}^{\lambda }(w):=\widehat{f}(|\lambda |^{1/2}w,\lambda )$. Lemma 7.1 says that for every $u,v\in \mathcal {S}({\mathbb {H}^n})$, $\langle \widehat{f}^{\lambda }_{n},\widehat{c_{u,v}}\rangle$ tends to $\langle \widehat{f}^{\lambda },\widehat{c_{u,v}}\rangle$ a.e. which implies that $\langle \pi _{f_{n}}^{\lambda }u,v\rangle$ must have a limit. This limit is $A^{\lambda }$ and it will be denoted by $\pi _{f}^{\lambda }$. Nevertheless, for a given $L^{2}$ function, the operator exists only for a.e. $\lambda$.

Lemma 7.2

Let K be a flag kernel on ${\mathbb {H}^n}$. Then, for every $f\in L^{2}({\mathbb {H}^n})$,

$$\begin{aligned} \pi _{K}^{\lambda }\pi _{f}^{\lambda }=\pi _{K\star f}^{\lambda }. \end{aligned}$$

Proof

Let $\{f_{n}\}_{n}\subset \mathcal {S}_{0} ({\mathbb {H}^n})$ tend to f in $L^{2}({\mathbb {H}^n})$. By definition

$$\begin{aligned} \pi _{K\star f}^{\lambda }=\lim \pi _{K\star f_{n_{k}(\lambda )}}^{\lambda }, \end{aligned}$$

for a.e. $\lambda$. As $f_{n_{k}(\lambda )}$ converges to f in $L^{2}({\mathbb {H}^n})$, by Lemma 7.1, $K\star f_{n_{k}(\lambda )}$ converges to $K\star f$ a.e. Hence

$$\begin{aligned} \pi _{K}^{\lambda }\pi _{f}^{\lambda }=\lim \pi _{K}^{\lambda }\pi _{f_{n_{k_{s}}(\lambda )}}^{\lambda }=\lim \pi _{K\star f_{n_{k_{s}}(\lambda )}}^{\lambda }=\pi _{K\star f}^{\lambda }. \end{aligned}$$

$\square$

Assume that $f\in L^{2}({\mathbb {H}^n})$. Let us continue with the calculation of kernel $\Omega _{f}^{\lambda }$ of the operator $\pi _{f}^{\lambda }$. As it has been said before, we have

$$\begin{aligned} \Omega _{f}^{\lambda }(\xi ,\eta ^{\vee })=e^{2\pi i\xi \eta }f^{\vee }(|\lambda |^{1/2}\xi ,|\lambda |^{1/2}\eta ,\lambda ). \end{aligned}$$

Therefore,

$$\begin{aligned} \Omega _{f}^{\lambda }(\xi ,y)&=\int e^{-2\pi i\eta (y-\xi )}f^{\vee }(|\lambda |^{1/2}\xi ,|\lambda |^{1/2}\eta ,\lambda )d\eta \\&=|\lambda |^{-n/2}f\left( |\lambda |^{1/2}\xi ^{\vee },\frac{y-\xi }{|\lambda |^{1/2}},\lambda ^{\vee }\right) . \end{aligned}$$

Furthermore, by Plancherel’s formula

$$\begin{aligned} \mathfrak {G}_{f}(\lambda ):&=|\lambda |^{n}\Vert \pi _{f}^{\lambda }\Vert _{HS}^{2}=|\lambda |^{n}\Vert \Omega _{f}^{\lambda }\Vert _{2}^{2}=\int \int |f\left( |\lambda |^{1/2}\xi ^{\vee },\frac{y-\xi }{|\lambda |^{1/2}},\lambda ^{\vee }\right) |^{2}d\xi dy\\&=\int \int |f(x,y,\lambda ^{\vee })|^{2}dxdy. \end{aligned}$$

The function $\mathfrak {G}_{f}$ is continuous when $f\in \mathcal {S}({\mathbb {H}^n})$. Let A be linear, bounded operator on $L^{2}({\mathbb {H}^n})$ (in particular a convolver), $\chi _{E}$ a characteristic function of a set $E\subset \mathbb {R}$. Then, from the above calculation we can conclude that

$$\begin{aligned} \chi _{E}(\lambda )\mathfrak {G}_{Af}(\lambda )=\mathfrak {G}_{A(\chi _{E}(\lambda )f)}(\lambda ). \end{aligned}$$

Let as recall here that $\Vert f\Vert _{2}^{2}=\int _{\mathbb {R}^{\star }}\mathfrak {G}_{f}(\lambda )d\lambda$.

Lemma 7.3

Let A be a linear, bounded operator on $L^{2}({\mathbb {H}^n})$. Suppose that, for every $f\in L^{2}({\mathbb {H}^n})$

$$\begin{aligned} \int _{\mathbb {R}^{\star }}\mathfrak {G}_{Af}(\lambda )d\lambda \geqslant c^{2}\int _{\mathbb {R}^{\star }}\mathfrak {G}_{f}(\lambda )d\lambda , \end{aligned}$$

then, for almost every $\lambda$, $\mathfrak {G}_{Af}(\lambda )\geqslant c^{2}\mathfrak {G}_{f}(\lambda ).$

Proof

Assume a contrario that for a function g and $\lambda$ in a set E of positive Lebesque measure we have that $\mathfrak {G}_{Ag}(\lambda )<c^{2}\mathfrak {G}_{g}(\lambda )$. Then, there exists $\varepsilon$>0 and a subset F of E of positive Lebesque measure such that $\mathfrak {G}_{Ag}(\lambda )\leqslant (1-\varepsilon )c^{2}\mathfrak {G}_{g}(\lambda )$ on F. Therefore,

$$\begin{aligned} c^{2}\int _{\mathbb {R}^{\star }}\mathfrak {G}_{\chi _{F}(\lambda )g}(\lambda )d\lambda&\leqslant \int _{\mathbb {R}^{\star }}\mathfrak {G}_{A(\chi _{F}(\lambda )g)}(\lambda )d\lambda =\int _{F}\mathfrak {G}_{Ag}(\lambda )d\lambda \\&\leqslant c^{2}(1-\varepsilon )\int _{F}\mathfrak {G}_{g}(\lambda )d\lambda =c^{2}(1-\varepsilon )\int _{\mathbb {R}^{\star }}\mathfrak {G}_{\chi _{F}(\lambda )g}(\lambda )d\lambda , \end{aligned}$$

which is obviously a contradiction. $\square$

The same holds true for the opposite inequality and the proof is analogous.

Theorem 7.4

Let K be a symmetric flag kernel, such that $\mathrm{Op}(K)$ is invertible. The family $\{\pi _{K}^{\lambda }\}_{\lambda }$ is uniformly invertible, that is all $\pi _{K}^{\lambda }$ are invertible and the family of operators $\{(\pi _{K}^{\lambda })^{-1}\}_{\lambda }$ is uniformly bounded on $L^{2}({\mathbb {R}^n})$.

Proof

As $\mathrm{Op}(K)$ is invertible there exists a constant $C_{K}$, such that for $f\in L^{2}({\mathbb {H}^n})$ $\Vert K\star f\Vert _{2}\geqslant C_{K}\Vert f\Vert _{2}$. Using Plancherel formula we have

$$\begin{aligned} \int _{\mathbb {R}^{\star }}|\lambda |^{n}\Vert \pi _{K\star f}^{\lambda }\Vert _{HS}^{2}d\lambda =\Vert K\star f\Vert _{2}^{2}\geqslant C_{K}^{2}\Vert f\Vert _{2}^{2}=C_{K}^{2}\int _{\mathbb {R}^{\star }}|\lambda |^{n}\Vert \pi _{f}^{\lambda }\Vert _{HS}^{2}d\lambda . \end{aligned}$$

Now by Lemma 7.3

$$\begin{aligned} C_{K}\Vert \pi _{f}^{\lambda }\Vert _{HS}\leqslant \Vert \pi _{K\star f}^{\lambda }\Vert _{HS}=\Vert \pi _{K}^{\lambda }\pi _{f}^{\lambda }\Vert _{HS}\leqslant \Vert \pi _{K}^{\lambda }\Vert _{2\rightarrow 2}\Vert \pi _{f}^{\lambda }\Vert _{HS}. \end{aligned}$$

Consider the operator $\mathcal {P}_{g,h}$; $g,h\in L^{2}({\mathbb {R}^n})$, where $\Vert h\Vert _{2}\ne 0$, which acts on vectors $u\in L^{2}({\mathbb {R}^n})$ by $\mathcal {P}_{g,h}(u):=\langle u,g\rangle h$. It is easy to see that the kernel of $\mathcal {P}_{g,h}$ is $\Omega _{\mathcal {P}}(x,y)=g(x)h(y)$, so $\Vert \mathcal {P}_{g,h}\Vert _{HS}=\Vert g\Vert _{2}\Vert h\Vert _{2}$. Now

$$\begin{aligned} \pi _{K}^{\lambda }\mathcal {P}_{g,h}u=\langle \pi _{K}^{\lambda }u,g\rangle h=\langle u,(\pi _{K}^{\lambda })^{\star }g\rangle h=\langle u,\pi _{K^{\star }}^{\lambda }g\rangle h=\langle u,\pi _{K}^{\lambda }g\rangle h=\mathcal {P}_{\pi _{K}^{\lambda }g,h}u. \end{aligned}$$

Hence

$$\begin{aligned} \Vert \pi _{K}^{\lambda }g\Vert _{2}\Vert h\Vert _{2}=\Vert \pi _{K}^{\lambda }\mathcal {P}_{g,h}\Vert _{HS}\geqslant C_{K}\Vert \mathcal {P}_{g,h}\Vert _{HS}=C_{K}\Vert g\Vert _{2}\Vert h\Vert _{2}. \end{aligned}$$

Dividing both sides by $\Vert h\Vert _{2}$ we obtain that $\Vert \pi _{K}^{\lambda }g\Vert _{2}\geqslant C_{K}\Vert g\Vert _{2}$ holds for every $g\in \ L^{2}({\mathbb {R}^n})$ which, together with the fact that $\pi _{K}^{\lambda }$ is self-adjoint, implies our claim. $\square$

8 The Beals Theorem and the Main Result

Summing up our previous results we conclude that, for every $\lambda$, a flag kernel K, gives rise to an operator $\pi _{K}^{\lambda }$ which acts on $L^{2}({\mathbb {R}^n})$ as a pseudodifferential operator with the Kohn-Nirenberg symbol $a_{\lambda }(w)=\widetilde{\widehat{K}}(|\lambda |^{1/2}w,\lambda )$. Moreover,

$$\begin{aligned} |\partial _{w}^{\alpha }a_{\lambda }(w)|&=|\partial _{w}^{\alpha }\{\widetilde{\widehat{K}}(|\lambda |^{1/2}w,\lambda )\}|\leqslant c_{\alpha }(\Vert |\lambda |^{1/2}w\Vert +|\lambda |^{1/2})^{-|\alpha |}|\lambda |^{|\alpha |/2}\\&=c_{\alpha }(1+\Vert w\Vert )^{-|\alpha |}. \end{aligned}$$

Observe that these estimates do not depend on $\lambda$. In particular, by the Calderón–Vaillancourt theorem, the family $\{\pi _{K}^{\lambda }\}_{\lambda }$ is uniformly bounded on $L^{2}({\mathbb {R}^n})$. Let us define

$$\begin{aligned} Sym^{0}(\mathbb {R}^{2n}):=\{a\in \mathcal {C}^{\infty }(\mathbb {R}^{2n}):|\partial _{w}^{\alpha }a(w)|\leqslant c_{\alpha }(1+\Vert w\Vert )^{-|\alpha |}\}. \end{aligned}$$

In this language the family of symbols $\{a_{\lambda }\}_{\lambda }$ is bounded in $Sym^{0}(\mathbb {R}^{2n})$ with the natural seminorm topology. The key point in our argument is the following application of a much more general theorem of Beals.

Theorem 8.1

(Beals [1], Theorem 3.2) Let $A=a(x,D)$, where a is a smooth function on $\mathbb {R}^{2n}$, which satisfies the estimates

$$\begin{aligned} |\partial _{w}^{\alpha }a(w)|\leqslant C_{\alpha }. \end{aligned}$$

Moreover let A be invertible on $L^{2}({\mathbb {R}^n})$. Then, $A^{-1}=b(x,D)$ with b satisfying the same properties as a. Each seminorm of b depends only on a finite number of seminorms of a and the operator norm of $A^{-1}$.

Let us denote the symbol of $(\pi _{K}^{\lambda })^{-1}$ by $b_{\lambda }$. In the language of symbols the equation $\pi _{K}^{\lambda }(\pi _{K}^{\lambda })^{-1}=Id$ corresponds to $a_{\lambda }\#b_{\lambda }=1$. Differentiating this formula we get

$$\begin{aligned} \partial _{w}^{\alpha }b_{\lambda }=\sum _{\gamma \in \Gamma }c_{\gamma }\partial _{w}^{\gamma _{1}}b_{\lambda }\#\partial _{w}^{\gamma _{2}}a_{\lambda }\#\partial _{w}^{\gamma _{3}}b_{\lambda }, \end{aligned}$$

where $\Gamma :=\{\gamma =(\gamma _{1},\gamma _{2},\gamma _{3})\in (\mathbb {N}^{2n})^{3}: |\gamma _{2}|>0,\gamma _{1}+\gamma _{2}+\gamma _{3}=\alpha \}$. By the standard symbolic calculus, we obtain that in fact $\{b_{\lambda }\}_{\lambda }\subset Sym^{0}(\mathbb {R}^{2n})$. Moreover one can conclude that the seminorms of $b_{\lambda }$ once again do not depend on $\lambda$. So the family $\{b_{\lambda }\}_{\lambda }$ also corresponds to a bounded family in $Sym^{0}(\mathbb {R}^{2n})$. Similar formulas will be explained in later arguments. Following Głowacki [10], we say that a is a weak limit of a bounded sequence $\{a_{n}\}_{n}$ in $Sym^{0}(\mathbb {R}^{2n})$ iff for every $\alpha$ the sequence $\{\partial ^{\alpha }a_{n}\}_{n}$ converges almost uniformly to $\partial ^{\alpha }a$. The twisted multiplication $\#$ is continuous in the weak sense.

Lemma 8.2

The family $\{b_{\lambda }\}_{\lambda }$ is weakly smooth in the parameter $\lambda$.

Proof

We proceed by induction. Let $\{\lambda _{n}\}_{n}$ converge to a nonzero $\lambda$. As $\{b_{\lambda _{n}}\}_{n}$ is bounded in $Sym^{0}(\mathbb {R}^{2n})$, we can use Arzeli-Ascoli theorem to find a weakly convergent subsequence. Let $\{b_{\lambda _{n_{k}}}\}_{k}$ tend to $b_{\lambda }(\{n_{k}\})$. We have

$$\begin{aligned} 1=b_{\lambda _{n_{k}}}\#a_{\lambda _{n_{k}}}\rightarrow b_{\lambda (\{n_{k}\})}\#a_{\lambda }. \end{aligned}$$

Hence for every convergent subsequence $\{b_{\lambda _{n_{k}}}\}_{k}$, the limit must be the same and equal to $b_{\lambda }$. Therefore, it also must be the limit of $\{b_{\lambda _{n}}\}_{n}$. Assume now that $\partial _{\lambda }^{N}b_{\lambda }$ is continuous for $N<M$. Observe that using continuity of $b_{\lambda }$ which we have just obtained, formally we have

$$\begin{aligned} \lim _{h\rightarrow 0}\frac{b_{\lambda +h}-b_{\lambda }}{h}=\lim _{h\rightarrow 0}b_{\lambda }\#\frac{a_{\lambda }-a_{\lambda +h}}{h}\#b_{\lambda +h}=-b_{\lambda }\#\partial _{\lambda }a_{\lambda }\#b_{\lambda }, \end{aligned}$$

(8.1)

where the right hand side is weakly continuous. Consider the set

$$\begin{aligned} \Xi :=\{m=(m_{1},m_{2},m_{3})\in \mathbb {N}^{3}:m_{2}>0,m_{1}+m_{2}+m_{3}=M\}. \end{aligned}$$

Iterating the decomposition (8.1) we obtain

$$\begin{aligned} \partial _{\lambda }^{M}b_{\lambda }=\sum _{m\in \Xi }c_{m}\partial _{\lambda }^{m_{1}}b_{\lambda }\#\partial _{\lambda }^{m_{2}}a_{\lambda }\#\partial _{\lambda }^{m_{3}}b_{\lambda }. \end{aligned}$$

(8.2)

By induction hypothesis the right hand side is again weakly continuous. Hence the proof is complete. $\square$

The decomposition 8.2 actually gives more. It turns out that $b(w,\lambda ):=b_{\lambda }(w)$ is also smooth if only $\lambda \ne 0$. It is a consequence of the fact that every derivative of $b(w,\lambda )$ has bounded partial derivatives outside of every set of type $\mathbb {R}^{2n}\times [-\varepsilon ,\varepsilon ]$.

Note that $\widetilde{\widehat{B}}(w,\lambda )=\widehat{B}(-w,-\lambda )$, so $\widetilde{\widehat{B}}$ is the Fourier transform of a flag kernel if and only if $\widehat{B}$ is.

Theorem 8.3

Let B be a distribution such that $\widehat{B}(w,\lambda )=b_{\lambda }(|\lambda |^{-1/2}w)$. Then, B is a flag kernel.

Proof

It is obvious from the definition that $\widehat{B}$ is smooth away from the hyperspace $\{(w,\lambda ):\lambda =0\}$. We have

$$\begin{aligned} |(\partial _{1,2,...,2n}^{\alpha }\widehat{B})(|\lambda |^{1/2}w,\lambda )||\lambda |^{|\alpha |/2}&=|\partial _{1,2,...,2n}^{\alpha }\{\widehat{B}(|\lambda |^{1/2}w,\lambda )\}|=|\partial _{w}^{\alpha }b_{\lambda }(w)|\\ \leqslant c_{\alpha }(1+\Vert w\Vert )^{-|\alpha |}. \end{aligned}$$

Therefore,

$$\begin{aligned} |(\partial _{1,2,...,2n}^{\alpha }\widehat{B})(|\lambda |^{1/2}w,\lambda )|\!\leqslant \! c_{\alpha }(1+\Vert w\Vert )^{-|\alpha |}|\lambda |^{-|\alpha |/2}\!=\!c_{\alpha }(|\lambda |^{1/2}+\Vert |\lambda |^{1/2}w\Vert )^{-|\alpha |}. \end{aligned}$$

Now putting w instead of $|\lambda |^{1/2}w$ we obtain

$$\begin{aligned} |(\partial _{w}^{\alpha }\widehat{B})(w,\lambda )|\leqslant c_{\alpha }(|\lambda |^{1/2}+\Vert w\Vert )^{-|\alpha |}. \end{aligned}$$

(8.3)

It sufficies now to get the estimates of the derivatives with respect to $\lambda$. We can treat inequality 8.3 as an initial step of an induction. First of all using the fact that K is a flag kernel one can calculate that

$$\begin{aligned} \big |\partial _{\lambda }^{M}a_{\lambda }(w)\big |&=\bigg |\partial _{\lambda }^{M}\left\{ \widetilde{\widehat{K}}(|\lambda |^{1/2}w,\lambda )\right\} \bigg |\\&=\Bigg |\sum _{1\leqslant |\beta |+j\leqslant M}\frac{c_{\beta ,j}w^{\beta }}{|\lambda |^{M-j-|\beta |/2}}(\partial _{1,2,...,2n}^{\beta }\partial ^{j}_{2n+1}\widetilde{\widehat{K})}(|\lambda |^{1/2}w,\lambda )\Bigg |\\&\leqslant \sum _{1\leqslant |\beta |+j\leqslant M}\frac{|c_{\beta ,j}|\Vert w\Vert ^{|\beta |}}{|\lambda |^{M-j-|\beta |/2}}(\Vert |\lambda |^{1/2}w\Vert +|\lambda |^{1/2})^{-|\beta |}|\lambda |^{-j}\\&\lesssim \frac{\Vert w\Vert ^{|\beta |}}{|\lambda |^{M}(1+\Vert w\Vert )^{|\beta |}}\leqslant |\lambda |^{-M}. \end{aligned}$$

As $\{b_{\lambda }\}_{\lambda }$ is a bounded family in $Sym^{0}(\mathbb {R}^{2n})$ let us assume that it is so for the families $\{|\lambda |^{N}\partial _{\lambda }^{N}b_{\lambda }\}_{\lambda }$, where $N<M$. Now using 8.2 we can write

$$\begin{aligned} |\lambda |^{M}\partial _{\lambda }^{M}b_{\lambda }=\sum _{m\in \Xi }c_{m}|\lambda |^{m_{1}}\partial _{\lambda }^{m_{1}}b_{\lambda }\#|\lambda |^{m_{2}}\partial _{\lambda }^{m_{2}}a_{\lambda }\#|\lambda |^{m_{3}}\partial _{\lambda }^{m_{3}}b_{\lambda }. \end{aligned}$$

As $m_{2}>0$ one can use an induction argument and the standard symbolic calculus to deduce that the right hand side is bounded in $Sym^{0}(\mathbb {R}^{2n})$. Therefore, the families $\{|\lambda |^{M}\partial _{\lambda }^{M}b_{\lambda }\}_{\lambda }$ are bounded, for all $M\in \mathbb {N}$. Thus,

$$\begin{aligned} |\partial _{w}^{\alpha }(|\lambda |^{M}\partial _{\lambda }^{M}b_{\lambda }(w))|\leqslant c_{\alpha , M}(1+\Vert w\Vert )^{-|\alpha |}, \end{aligned}$$

so

$$\begin{aligned} |\partial _{w}^{\alpha }\partial _{\lambda }^{M}b_{\lambda }(w)|\leqslant c_{\alpha , M}(1+\Vert w\Vert )^{-|\alpha |}|\lambda |^{-M}. \end{aligned}$$

One can calculate that

$$\begin{aligned} \partial _{w}^{\alpha }\partial _{\lambda }^{M}b_{\lambda }(w)=\sum _{\gamma +\delta =\alpha }\sum _{1\leqslant |\beta |+j\leqslant M}\frac{c_{\gamma ,\beta ,j}w^{\beta -\gamma }}{|\lambda |^{M-j-(|\beta |+|\delta |)/2}}(\partial _{1,2,...,2n}^{\beta +\delta }\partial ^{j}_{2n+1}\widehat{B})(|\lambda |^{1/2}w,\lambda ). \end{aligned}$$

The only component of the sum on the right hand side that includes $j=M$ is the one with $\beta =\gamma =0,\ \delta =\alpha$. So, by induction hypothesis, we have

$$\begin{aligned}&|(\partial _{1,2,...,2n}^{\alpha }\partial ^{M}_{2n+1}\widehat{B})(|\lambda |^{1/2}w,\lambda )||\lambda |^{|\alpha |/2}\\&\quad \lesssim \sum _{\gamma +\delta =\alpha }\sum _{1\leqslant |\beta |+j\leqslant M,\ \beta \ne 0}\frac{\Vert w\Vert ^{|\beta |-|\gamma |}}{|\lambda |^{M-j-(|\beta |+|\delta |)/2}(\Vert |\lambda |^{1/2}w\Vert +|\lambda |^{1/2})^{|\beta |+|\delta |}|\lambda |^{j}}\\&\qquad +|\partial _{w}^{\alpha }\partial _{\lambda }^{M}b_{\lambda }(w)|\\&\quad \lesssim \sum _{\gamma +\delta =\alpha }\sum _{1\leqslant |\beta |+j\leqslant M,\ \beta \ne 0}\frac{\Vert w\Vert ^{|\beta |-|\gamma |}}{(1+\Vert w\Vert )^{|\beta |-|\gamma |}(1+\Vert w\Vert )^{|\alpha |}|\lambda |^{M}}\\&\qquad +(1+\Vert w\Vert )^{-|\alpha |}|\lambda |^{-M}\lesssim (1+\Vert w\Vert )^{-|\alpha |}|\lambda |^{-M}. \end{aligned}$$

Thus,

$$\begin{aligned} |(\partial _{1,2,...,2n}^{\alpha }\partial ^{M}_{2n+1}\widehat{B})(|\lambda |^{1/2}w,\lambda )|&\leqslant c_{\alpha ,M}(1+\Vert w\Vert )^{-|\alpha |}|\lambda |^{-M}|\lambda |^{-|\alpha |/2}\\&= c_{\alpha ,M}(\Vert |\lambda |^{1/2}w\Vert +|\lambda |^{1/2})^{-|\alpha |}|\lambda |^{-M}. \end{aligned}$$

Again putting w instead of $|\lambda |^{1/2}w$ we obtain

$$\begin{aligned} |(\partial _{w}^{\alpha }\partial ^{M}_{\lambda }\widehat{B})(w,\lambda )|\leqslant c_{\alpha ,M}(\Vert w\Vert +|\lambda |^{1/2})^{-|\alpha |}|\lambda |^{-M}. \end{aligned}$$

$\square$

proof of Theorem 3.3

The KN symbol of $(\pi _{K}^{\lambda })^{-1}$ is $b_{\lambda }(w)=\widehat{B}(|\lambda |^{1/2}w,\lambda )$ which is the symbol of $\pi _{\widetilde{B}}^{\lambda }$ and $\widetilde{B}$ is a flag kernel. So $(\pi _{K}^{\lambda })^{-1}=\pi _{\widetilde{B}}^{\lambda }$. Now

$$\begin{aligned} \pi _{\delta _{0}}^{\lambda }=Id=(\pi _{K}^{\lambda })^{-1}\pi _{K}^{\lambda }=\pi _{K}^{\lambda }(\pi _{K}^{\lambda })^{-1}=\pi _{\widetilde{B}}^{\lambda }\pi _{K}^{\lambda }=\pi _{K}^{\lambda }\pi _{\widetilde{B}}^{\lambda }=\pi _{\widetilde{B}\star K}^{\lambda }=\pi _{K\star \widetilde{B}}^{\lambda }. \end{aligned}$$

So $K\star \widetilde{B}=\delta _{0}=\widetilde{B}\star K$. Now putting $L=\widetilde{B}$ for $f\in L^{2}({\mathbb {H}^n})$ we achieve $K\star L\star f=L\star K\star f=f$, which is equivalent to $\mathrm{Op}(K)\mathrm{Op}(L)f=\mathrm{Op}(L)\mathrm{Op}(K)f=f$ and finally $\mathrm{Op}(K)^{-1}=\mathrm{Op}(L)$. $\square$