Hardy’s Theorem for Gabor Transform on Nilpotent Lie Groups

Abstract

In this paper, we study the conjecture of Bansal, Kumar and Sharma, which is an analog of Hardy’s theorem for Gabor transform in the setup of connected nilpotent Lie groups. To approach this conjecture, we use the orbit method and the Plancherel theory. When the Lie group G is simply connected, we show that the conjecture is true.

1 Introduction

A classical version of the uncertainty principle states that an integrable function f defined on the real line and its Fourier transform ${\hat{f}}$ cannot be simultaneously and sharply localized unless $f=0$ almost everywhere. An important result making this precise, is the Hardy Theorem (see [13]):

Theorem 1

Let $\alpha , \beta , c$ be positive real numbers and f a measurable function on ${\mathbb {R}}$ such that:

• (i) $\vert f(t)\vert \le c \text {e}^{-\alpha \pi t^2}, \ \ t\in {\mathbb {R}}$,

• (ii) $\vert {\hat{f}}(k)\vert \le c \text {e}^{-\beta \pi k^2}, \ \ k\in {\mathbb {R}}$.

If $\alpha \beta > 1$, then $f= 0$ a.e. If $\alpha \beta = 1$ then $f(t)=b\text {e}^{-\alpha \pi t^2}$, for some constant b. If $\alpha \beta < 1$, then any finite linear combination of Hermite functions satisfies (i) and (ii).

For the Fourier transformation we use the normalization

$$\begin{aligned} {{\hat{f}}}(k)=\int _{{\mathbb {R}}}f(t)\hbox {e}^{-2i\pi t k}dt,\ k\in {\mathbb {R}}. \end{aligned}$$

Note that Hardy’s theorem is also valid on ${\mathbb {R}}^n$ (see [19,  Theorem 4]). Much efforts have been deployed to prove Hardy-like theorems for various classes of non-Abelian connected Lie groups. Specifically, analogues and variants of Hardy’s theorem have been shown for nilpotent Lie groups [1, 2, 14, 19, 21], some classes of solvable Lie groups [3], non-compact connected semisimple Lie groups G with finite center [9, 15, 17, 18] and motion groups [16, 20]. For more information and further references concerning the entire subject, we refer the readers to the excellent monograph by Thangavelu [22].

The continuous Gabor transform (also known as windowed Fourier transform) is a classical tool in mathematical signal processing. Roughly speaking, it is the Fourier transform of a signal f seen through a sliding window $\psi$. For $f\in L^2({\mathbb {R}}^n)$ and a nonzero function $\psi \in L^2({\mathbb {R}}^n)$ called a window function, the continuous Gabor transform with respect to $\psi$ is defined on ${\mathbb {R}}^n\times \widehat{{\mathbb {R}}}^n$ by

$$\begin{aligned} {\mathcal {G}}_\psi f(x,w): =\int _{{\mathbb {R}}^n} f(y){\overline{\psi }}(y-x)e^{-2i\pi \langle y,w \rangle }dy. \end{aligned}$$

According to [11], we have for all $f_1$, $f_2$, $\psi _1$, $\psi _2\in L^2({\mathbb {R}}^n)$ the functions ${\mathcal {G}}_{\psi _1}f_1$ and ${\mathcal {G}}_{\psi _2}f_2$ belong to $L^{2}({\mathbb {R}}^n\times \hat{{\mathbb {R}}}^n)$ and

$$\begin{aligned} \langle {\mathcal {G}}_{\psi _1}f_1,{\mathcal {G}}_{\psi _2}f_2\rangle _{L^2({\mathbb {R}}^n\times \hat{{\mathbb {R}}}^n)}=\langle f_1 , f_2\rangle _{L^2({\mathbb {R}}^n)}\overline{\langle \psi _1,\psi _2\rangle }_{L^2({\mathbb {R}}^n)}. \end{aligned}$$

(1)

This transform plays an important role in time-frequency analysis namely by providing an interesting way to study the local frequency spectrum of signals. For a detailed discussion of time-frequency analysis, we refer the readers to [11]. It has been shown in the early 2000s that many uncertainty principles for the Fourier transform have a counterpart for the continuous Gabor transform (see e.g. [7, 12]). We specify that a Hardy-type theorem has been established in [12,  Theorem 2.6.2].

Theorem 2

Let $f,\psi \in L^2({\mathbb {R}}^n)$. Assume that

$$\begin{aligned} \Big \vert {\mathcal {G}}_\psi f(x,w)\Big \vert \le C \text {e}^{-\pi (\alpha x^2+\beta w^2)/2}, \end{aligned}$$

for some constants $\alpha , \beta , C>0$. Then three cases can occur.

• (i) If $\alpha \beta > 1$, then either $f=0$ a.e. or $\psi = 0$ a.e.

• (ii) If $\alpha \beta = 1$ and ${\mathcal {G}}_\psi f$ is not zero almost everywhere, then both f and $\psi$ are multiples of some time-frequency shift of the Gaussian $\text {e}^{-\alpha \pi x^2}$.

• (iii) If $\alpha \beta < 1$, then the decay condition is satisfied whenever f and $\psi$ are finite linear combinations of Hermite functions.

On the other hand, the continuous Gabor transform has also been extended to separable locally compact unimodular group of type I (see [10]). One should notice that, in the Euclidean setting, the continuous Gabor transform has many symmetries which are lost in the Lie group setting (the dual of G does not identify with G) and this is then a serious obstacle for stating uncertainty principles for the continuous Gabor transform. However, some attempts to extend Theorem 2 on special classes of non-Abelian Lie groups have already been made. In particular, we cite here the work of Bansal, Kumar and Sharma [6], where the authors generalized Hardy’s theorem for the Gabor transform on locally compact abelian groups having noncompact identity component and groups of the form ${\mathbb {R}}^n\times K$, where K is a compact group having irreducible representations of bounded dimension. When it comes to connected nilpotent Lie groups, only a conjecture are now available. In the same reference, the previous authors conjecture that if $\alpha ,\beta$ and C are positive real numbers and $f,\psi$ are square integrable functions on connected nilpotent Lie group $G=\exp \mathfrak {g}$ such that $\Vert {\mathcal {G}}_\psi f (g,\pi _l)\Vert _{HS}\le C \text {e}^{-\pi (\alpha \Vert g\Vert ^2+\beta \Vert l\Vert ^2)/2}$ for all $(g,l)\in G\times {\mathcal {W}}$, then $f= 0$ a.e. or $\psi = 0$ a.e. provided that $\alpha \beta >1$. Here ${\mathcal {W}}$ is a suitable cross-section for the generic coadjoint orbits in $\mathfrak {g}^*$, the vector space dual of $\mathfrak {g}$, and $\Vert g\Vert$ and $\Vert l\Vert$ are substitutes (in terms of bases of $\mathfrak {g}$ and $\mathfrak {g}^*$) for the Euclidean norms on ${\mathbb {R}}^n$ and $\widehat{{\mathbb {R}}}^n={\mathbb {R}}^n$. For details and unexplained notation see Sect. 2. They also proved that this conjecture fails for a connected nilpotent Lie group G having a square integrable irreducible representation. This paper is the first attempt to establish analog of Hardy’s theorem for Gabor transform on nilpotent Lie groups. By exploiting Hardy’s Theorem for ${\mathbb {R}}$ and a localized version of the Plancherel formula, we show in Sect. 4 that the above-mentioned conjecture holds. Our main result is the following:

Theorem 3

Let G be connected, simply connected nilpotent Lie group. Let $f,\psi \in L^2(G)$ be such that

$$\begin{aligned} \Vert {\mathcal {G}}_\psi f (g,\pi _l)\Vert _{HS}^2\le C e^{-{\pi \over 2} (\alpha \Vert g\Vert ^2+\beta \Vert l\Vert ^2)}, \end{aligned}$$

(2)

for all $(g,l)\in G\times {{\mathcal {W}}}$, where $\alpha ,\beta$ and C are positive real numbers. If $\alpha \beta >1$, then either $f=0$ a.e. or $\psi =0$ a.e.

2 Backgrounds

2.1 Continuous Gabor Transform

Let G be a separable locally compact unimodular group of type I, and let dg be its Haar measure. We endow the unitary dual of G with the Mackey Borel structure. We denote by $L^p(G)$ the space of $L^p$-functions on G for $p\ge 1$, and we define

$$\begin{aligned} \pi (f)=\int _Gf(g)\pi (g)dg,\quad \pi \in {{\hat{G}}}, \, f\in L^1(G). \end{aligned}$$

Then by the abstract Plancherel theorem, there exists a unique Borel measure $\rho$ on ${{\hat{G}}}$ such that for any function $f\in L^1(G)\cap L^2(G)$,

$$\begin{aligned} \int _G \vert f(g)\vert ^2dg=\int _{{{\hat{G}}}}\Vert \pi (f) \Vert _{HS}^2d\rho (\pi ), \end{aligned}$$

where $\Vert \pi (f) \Vert _{HS}=\big (\text {tr}\big (\pi (f)^*\pi (f)\big )\big )^{1/2}$ denotes the Hilbert-Schmidt norm of $\pi (f)$.

Let $f \in C_c(G)$, the set of all continuous complex-valued functions on G with compact supports, and $\psi$ a fixed nonzero function in $L^2(G)$, usually called window function. For $(x,\pi )\in G\times {{\hat{G}}}$, the continuous Gabor transform of f with respect to the window function $\psi$ is defined as a measurable field of operators on $G\times {{\hat{G}}}$ by

$$\begin{aligned} {\mathcal {G}}_\psi f(x,\pi ) :=\int _Gf(g){\overline{\psi }}(x^{-1}g)\pi (g)dg. \end{aligned}$$

Let $f_\psi ^x$ be the function defined on G by

$$\begin{aligned} f_\psi ^x(g)=f(g){\overline{\psi }} (x^{-1}g),\qquad \forall g\in G. \end{aligned}$$

Then, $f_\psi ^x\in L^1(G)\cap L^2(G)$ and

$$\begin{aligned} \pi (f_\psi ^x)=\int _Gf_\psi ^x(g)\pi (g)dg=\int _Gf(g){\overline{\psi }}(x^{-1}g)\pi (g)dg={\mathcal {G}}_\psi f(x,\pi ). \end{aligned}$$

(3)

By the Plancherel theorem, ${\mathcal {G}}_\psi f(x,\pi )$ is a Hilbert-Schmidt operator for all $x \in G$ and for almost all $\pi \in {{\hat{G}}}$. Furthermore,

$$\begin{aligned} \int _G \int _{{{\hat{G}}}} \Vert {\mathcal {G}}_\psi f(x,\pi )\Vert _{HS}^2d\rho (\pi )\, dx=\Vert \psi \Vert _2^2\Vert f \Vert _2^2. \end{aligned}$$

(4)

Thus, the continuous Gabor transform ${\mathcal {G}}_\psi : f\mapsto {\mathcal {G}}_\psi f$ ($f\in C_c(G)$) is a multiple of an isometry. So, we can extend ${\mathcal {G}}_\psi$ uniquely to a bounded linear operator on $L^2(G)$ which we still denote by ${\mathcal {G}}_\psi$ and this extension satisfies (4) for each $f \in L^2(G)$.

2.2 Nilpotent Lie Groups

We begin this subsection by reviewing some useful facts and notations for nilpotent Lie group. This material is quite standard, we refer the reader to [8] for details. We assume henceforth that $G=\exp \mathfrak {g}$ is a connected, simply connected nilpotent Lie group.

Let ${\mathcal {B}}=\{X_1,...,X_n\}$ be a strong Malcev basis of $\mathfrak {g}$ passing through the center of $\mathfrak {g}$. We introduce a norm function on G by setting for $x = \exp (x_ 1X_1+ \cdots +x_ nX_n )\in G, \ {x_ j} \in {\mathbb {R}}$,

$$\begin{aligned} \Vert x\Vert = \sqrt{{({x_ 1^2}+ \cdots +{x_n^2})}}. \end{aligned}$$

The map:

$$\begin{aligned} {{{\mathbb {R}}}^n} \rightarrow G , \ ({x_ 1},...,{x_ n}) \mapsto \displaystyle \exp \big (\sum _ {j = 1}^{n }{x_ j} {X_ j}\big ) \end{aligned}$$

is a diffeomorphism and maps the Lebesgue measure on ${{\mathbb {R}}}^n$ to the Haar measure on G. In this setup, we shall identify G as set with ${{\mathbb {R}}}^{n}$. We consider the Euclidean norm of $\mathfrak {g}^*$ with respect to the basis ${\mathcal {B}}^*=\{X_1^*,...,X_n^*\}$, that is,

$$\begin{aligned} \Big \Vert { \displaystyle \sum _ {j = 1}^{n }{l_ j} {X_ j^*}}\Big \Vert =\sqrt{( l_1^2 + \cdots + l_n^2 )}= \Vert l \Vert , \ \ l_j\in {\mathbb {R}}. \end{aligned}$$

Let ${\mathcal {U}}$ denote the Zariski open subset of $\mathfrak {g} ^*$ consisting of all elements in generic orbits with respect to the basis ${\mathcal {B}}^*$. Let S be the set of jump indices, and set $T = \{1,...,n \}\backslash S$ and $V_T = {\mathbb {R}}\text {-span}\{X_i^*\ : \ i \in T \}$. Then, ${\mathcal {W}} ={{\mathcal {U}} \cap V_T}$ is a cross section of the generic orbits and ${\mathcal {W}}$ supports the Plancherel measure on ${\hat{G}}$. Let Pf(l) denote the Pfaffian of the skew-symmetric matrix $M_S(l)= ( l([X_i,X_j]))_{ i,j \in S} .$ Then, one has that:

$$\begin{aligned} {{\vert Pf(l) \vert }^2}= \det {M_S (l)}. \end{aligned}$$

If dl is the Lebesgue measure on ${\mathcal {W}},$ then $d \tau = \vert Pf(l) \vert dl$ is a Plancherel measure for ${\hat{G}}$. Let dg be the Haar measure on G. For $\varphi \in L^1 (G) \cap L^2 (G)$, the Plancherel formula reads:

$$\begin{aligned} {{\Vert \varphi \Vert }^2 _ 2} = \displaystyle \int _G {\vert \varphi (g) \vert }^2 dg = \displaystyle \int _ {{\mathcal {W}} } { \Vert \pi _l( \varphi ) \Vert }^2_{HS} d \tau (l). \end{aligned}$$

(5)

3 Some Lemmas

In this section we prove three results, Lemmas 1, 2 and 3 which are required to prove Theorem 3.

For every $x, w \in {\mathbb {R}}^n$, we denote by ${\mathcal {M}}_w$ and ${\mathcal {T}}_x$ the modulation and the translation operators defined respectively on $L^2({\mathbb {R}}^n)$ by

$$\begin{aligned}&\forall z\in {\mathbb {R}}^n,\quad {\mathcal {M}}_wf(z)=e^{2i\pi \langle z,w\rangle }f(z), \\&\forall z\in {\mathbb {R}}^n,\quad {\mathcal {T}}_xf(z)=f(z-x). \end{aligned}$$

Then we deduce that,

$$\begin{aligned} \forall z\in {\mathbb {R}}^n,\quad {\mathcal {M}}_w({\mathcal {T}}_xf)(z)=e^{2i\pi \langle z,w\rangle }f(z-x), \end{aligned}$$

(6)

and

$$\begin{aligned} \forall z\in {\mathbb {R}}^n,\quad {\mathcal {T}}_x({\mathcal {M}}_wf)(z)=e^{-2i\pi \langle x,w\rangle }e^{2i\pi \langle z,w\rangle }f(z-x). \end{aligned}$$

(7)

The results in the following lemma are quite standard.

Lemma 1

Let $f,\psi \in L^2({\mathbb {R}}^n)$ and $\xi , \lambda , y, z \in {\mathbb {R}}^n$. Then,

• (i) ${\mathcal {G}}_{({\mathcal {M}}_\xi {\mathcal {T}}_z\psi )}({\mathcal {M}}_\lambda {\mathcal {T}}_yf)(x,w) = e^{2i\pi \langle x,\xi \rangle } e^{-2i\pi \langle y,w-\lambda +\xi \rangle } {\mathcal {G}}_\psi f(x-y+z,w-\lambda +\xi ).$

  In particular, ${\mathcal {G}}_\psi ({\mathcal {M}}_\lambda {\mathcal {T}}_yf)(x,w)=e^{-2i\pi \langle y,w\rangle } e^{2i\pi \langle y,\lambda \rangle } {\mathcal {G}}_\psi f(x-y,w-\lambda )$.

• (ii) ${\mathcal {G}}_\psi f(-x,-w)=e^{-2i\pi \langle x, w\rangle }\overline{{\mathcal {G}}_f\psi (x,w)}.$

• (iii) Let $F(x,w)= {\mathcal {G}}_\psi f(x,w) {\mathcal {G}}_\psi f(-x,-w)e^{2i\pi \langle x,w\rangle }.$ Then,

  $$\begin{aligned} {\hat{F}}(\nu ,\theta )=F(-\theta ,\nu ),\quad \nu , \theta \in {\mathbb {R}}^n. \end{aligned}$$

Let’s fix as above a strong Malcev basis $\{X_1,...,X_n\}$ of $\mathfrak {g}$ such that $X_1$ is in the center of $\mathfrak {g}$. For $a= (a_2,...,a_n)\in {\mathbb {R}}^{n-1}$, let $(f^g_\psi )_a$ be the complex valued function defined on ${\mathbb {R}}$ by

$$\begin{aligned} (f^g_\psi )_a(t)=f^g_\psi (t,a) =f^g_\psi \Big (\exp \big (tX_1+\sum _{j=2}^na_jX_j\big )\Big ). \end{aligned}$$

For $k\in {\mathbb {R}}$ and $s=(s_2,...,s_n)\in {\mathbb {R}}^{n-1}$, let $g=\exp \big (kX_1+\sum _{j=2}^ns_jX_j\big )$ and $f^{k,s}_\psi =f^g_\psi$. It is easy to see that $f^g_\psi \in L^1(G)$ for all $g\in G$, it sufficient to use Cauchy–Schwarz inequality. Moreover by [5,  Lemma 3.1], we have

$$\begin{aligned} {\mathcal {G}}_\psi f(g,\pi _l)=\pi _l(f^g_\psi ), \end{aligned}$$

(8)

for all $g\in G$. We should also mention that $f^g_\psi \in L^2(G)$, for almost all $g\in G$. In fact,

$$\begin{aligned} \int _G\int _G\vert f^g_\psi (x)\vert ^2dx\, dg =\int _G\int _G\vert f(x)\vert ^2\vert \psi (g^{-1}x)\vert ^2dx\, dg =\Vert f \Vert _2^2\Vert \psi \Vert _2^2<\infty . \end{aligned}$$

Then obviously $\int _G\vert f^g_\psi (x)\vert ^2dx<\infty$, for almost all $g\in G$.

Lemma 2

Let $f,\psi \in L^2(G)$ meet the condition (2) of Theorem 3. Then

$$\begin{aligned} I(f^{k,s}_\psi ):= \int _{{\mathbb {R}}^{n-1}} \Bigg ( \int _{{\mathbb {R}}} \big \vert (f^{k,s}_\psi )_a(t)\big \vert dt\Bigg )^2 da <\infty , \end{aligned}$$

for all $k\in {\mathbb {R}}$ and almost all $s=(s_2,...,s_n)\in {\mathbb {R}}^{n-1}$.

Proof

By using (2), we have

$$\begin{aligned}&\int _G\int _{\mathcal {W}} (1+\Vert g \Vert ^2) \Vert {\mathcal {G}}_\psi f(g,\pi _l)\Vert _{HS}^2 \vert Pf(l)\vert dl\,dg\nonumber \\&\quad \le \int _G\int _{\mathcal {W}} (1+\Vert g \Vert ^2) e^{-\pi (\alpha \Vert g\Vert ^2+\beta \Vert l\Vert ^2)} \vert Pf(l)\vert dl\,dg. \end{aligned}$$

(9)

Assume that the degree of the polynomial function Pf(l) is equal to $\delta$. Then,

$$\begin{aligned} \vert Pf(l)\vert\le & {} \text {cst} \ (1+\Vert l\Vert ^2)^\frac{\delta }{2}.\\\le & {} \text {cst} \ (1+\Vert l\Vert ^2)^\delta \end{aligned}$$

Therefore, the integral on the right hand side of (9) converges. Hence,

$$\begin{aligned} \infty> & {} \int _G(1+\Vert g \Vert ^2)\left( \int _{\mathcal {W}} \Vert \pi (f_\psi ^g) \Vert ^2_{HS}\vert Pf(l)\vert dl\right) dg \\= & {} \int _G\int _G (1+\Vert g \Vert ^2)\vert f(x)\vert ^2 \vert \psi (g^{-1}x)\vert ^2dx\,dg \end{aligned}$$

(using Plancherel formula of G)

$$\begin{aligned}&\ge \int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}}\int _{{\mathbb {R}}^{n-1}} \int _{{\mathbb {R}}}(1+\vert k\vert ^{2}) \Big \vert f\Big (\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\Big )\Big \vert ^2\\&\quad \times \Big \vert \psi \Big ( \exp \Big (kX_1+\sum _{j=2}^{n}s_jX_j\Big )^{-1}\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\Big )\Big \vert ^2 dt\, da\, dk\, ds. \end{aligned}$$

Noting that,

$$\begin{aligned}&\exp \Big (kX_1+\sum _{j=2}^{n}s_jX_j\Big )^{-1}\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\nonumber \\&\quad =\exp \Big ( \big (t-k+Q_1(a,s)\big )X_1+ \sum _{j=2}^{n} Q_j(a,s)X_j \Big ), \end{aligned}$$

(10)

where, for $1\le j\le n$, $Q_j$ is a polynomial function depending on $a=(a_2,...,a_n)$ and $s=(s_2,...,s_n)$. Furthermore, one can write

$$\begin{aligned} Q_j(a,s)= a_j-s_j+Q_j'(a_{j+1},...,a_n,s_{j+1},...,s_n), \quad j=2,...,n. \end{aligned}$$

(11)

It follows that,

$$\begin{aligned} \infty> & {} \int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}}\int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}} (1+\vert k\vert ^{2}) \Big \vert f\Big (\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\Big )\Big \vert ^2\\&\times \Big \vert \psi \Big ( \exp \Big ( \big (t-k+Q_1(a,s)\big )X_1+ \sum _{j=2}^{n} Q_j(a,s)X_j \Big )\Big )\Big \vert ^2 dt\,da\,dk\, ds \\= & {} \int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}}\int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}}\Big (1+ \big \vert t- r+Q_1(a,s)\big \vert ^{2}\Big ) \Big \vert f\Big (\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\Big )\Big \vert ^2\\&\times \Big \vert \psi \Big ( \exp \Big ( rX_1+ \sum _{j=2}^{n} Q_j(a,s)X_j \Big )\Big )\Big \vert ^2 dt\,da\,dr\, ds \end{aligned}$$

(by substituting $r=t-k+Q_1(a,s)$ for k). Now let’s use the change of variable $\sigma _j=Q_j(a,s)$, $j=2,...,n$, for fixed value of a. Note that, from (11) this change of variable has Jacobian 1. We then obtain,

$$\begin{aligned} \infty> & {} \int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}}\int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}}\Big (1+\big \vert t- r+R(a,\sigma )\big \vert ^{2}\Big )\Big \vert f\Big (\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\Big )\Big \vert ^2\\&\times \Big \vert \psi \Big ( \exp \Big ( rX_1+\sum _{j=2}^{n} \sigma _jX_j \Big )\Big )\Big \vert ^2 dt\,da\,dr\, d\sigma , \end{aligned}$$

where $R(a,\sigma )$ is a polynomial function depending on $a=(a_2,...,a_n)$ and $\sigma =(\sigma _2,...,\sigma _n)$. Therefore,

$$\begin{aligned}&\Big \vert \psi \Big ( \exp \Big ( rX_1+\sum _{j=2}^{n} \sigma _jX_j \Big )\Big )\Big \vert ^2 \int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}}\Big (1+ \big \vert t- r+R(a,\sigma )\big \vert ^{2}\Big ) \\&\quad \times \Big \vert f\Big (\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\Big )\Big \vert ^2 dt\, da <\infty , \end{aligned}$$

for almost all $r,\sigma _2,..., \sigma _n\in {\mathbb {R}}$. As $\psi$ is non identically zero, there exists $r_0,\sigma _0=(\sigma _2^0, ..., \sigma ^0_n)$ such that

$$\begin{aligned} \psi \Big ( \exp \Big (r_0X_1+ \sum _{j=2}^{n} \sigma ^0_jX_j \Big )\Big ) \ne 0, \end{aligned}$$

and

$$\begin{aligned} \int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}} \Big (1+\big \vert t- r_0+R(a,\sigma _0)\big \vert ^{2}\Big ) \Big \vert f\Big (\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\Big )\Big \vert ^2 dt\, da <\infty .\nonumber \\ \end{aligned}$$

(12)

On the other hand, we have

$$\begin{aligned} I(f^{k,s}_\psi )= & {} \int _{{\mathbb {R}}^{n-1}}\Bigg (\int _{{\mathbb {R}}}\Big \vert f\Big (\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\Big ){\overline{\psi }}\\&\Big (g^{-1}\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\Big )\Big \vert dt\Bigg )^2 da \\= & {} \int _{{\mathbb {R}}^{n-1}}\Bigg (\int _{{\mathbb {R}}}\Big \vert f\Big (\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\Big )\Big \vert \\&\times \Big \vert \psi \Big (\exp \Big ( \big (t-k+Q_1(a,s)\big )X_1+ \sum _{j=2}^{n} Q_j(a,s)X_j\Big )\Big )\Big \vert dt\Bigg )^2 da \end{aligned}$$

(using (10))

$$\begin{aligned}&\le \int _{{\mathbb {R}}^{n-1}}\Bigg (\int _{{\mathbb {R}}}{\Big \vert \psi \Big (\exp \Big ( \big (t-k+Q_1(a,s)\big )X_1+ \sum _{j=2}^{n} Q_j(a,s)X_j\Big )\Big )\Big \vert ^2\over 1+\big \vert t- r_0+R(a,\sigma _0)\big \vert ^2}dt\Bigg ) \\&\quad \times \Bigg ( \int _{{\mathbb {R}}} \Big (1+\big \vert t- r_0+R(a,\sigma _0)\big \vert ^2\Big ) \Big \vert f\Big (\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\Big )\Big \vert ^2 dt\Bigg )da \end{aligned}$$

(using Cauchy–Shwartz inequality),

$$\begin{aligned}&\le \int _{{\mathbb {R}}^{n-1}}\Bigg (\int _{{\mathbb {R}}}\Big \vert \psi \Big (\exp \Big ( \big (t-k+Q_1(a,s)\big )X_1+ \sum _{j=2}^{n} Q_j(a,s)X_j\Big )\Big )\Big \vert ^2dt\Bigg ) \\&\qquad \times \Bigg ( \int _{{\mathbb {R}}} \Big (1+\big \vert t- r_0+R(a,\sigma _0)\big \vert ^2\Big ) \Big \vert f\Big (\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\Big )\Big \vert ^2 dt\Bigg )da \\&\quad = \int _{{\mathbb {R}}^{n-1}}\Bigg (\int _{{\mathbb {R}}}\Big \vert \psi \Big (\exp \Big (zX_1+ \sum _{j=2}^{n} Q_j(a,s)X_j\Big )\Big )\Big \vert ^2dz\Bigg ) \\&\qquad \times \Bigg ( \int _{{\mathbb {R}}} \Big (1+\big \vert t- r_0+R(a,\sigma _0)\big \vert ^2\Big ) \Big \vert f\Big (\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\Big )\Big \vert ^2 dt\Bigg )da \end{aligned}$$

(using the change of variable $z=t-k+Q_1(a,s)$). By substituting $Q_j(a,s)$ for $s_j$, $j=2,...,n$, using Eq. (11), we have

$$\begin{aligned}&\int _{{\mathbb {R}}^{n-1}} I(f^{k,s}_\psi )\, ds\le \Vert \psi \Vert _2^2\int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}} \Big (1+\big \vert t- r_0+R(a,\sigma _0)\big \vert ^2 \Big )\\&\quad \Big \vert f\Big (\exp \Big (t X_1+\sum _{j=2}^{n}a_jX_j\Big )\Big )\Big \vert ^2 dt\, da, \end{aligned}$$

which is finite by (12). This implies that, $I(f^{k,s}_\psi )$ is finite for all $k\in {\mathbb {R}}$ and almost all $s\in {\mathbb {R}}^{n-1}$. $\square$

Before stating the next lemma, we need a localized version of the Plancherel measure (see [4]). Let $Z=\exp {\mathfrak {z}}$ be the center of G and fix a nonzero vector $X_1$ of $\mathfrak {z}$. Let $A=\exp \mathfrak {a}= \exp {\mathbb {R}}X_1$ be the closed connected subgroup of Z and $\chi =\chi _\zeta$, $\zeta =l_1X_1^* \in \mathfrak {a}^{ * }$, be the unitary character of A, defined by

$$\begin{aligned} \chi _\zeta (\exp tX_1)=\text {e}^{-2i\pi tl _1}. \end{aligned}$$

Let ${{\hat{G}}}_\chi =\{ \pi \in {{\hat{G}}} \ : \ \pi _{ \vert A}=\chi \cdot I \}$. For $1\le p<+\infty$, let $L^p(G/A, \zeta )$ be the set of all measurable functions $\varphi :G\rightarrow {\mathbb {C}}$ such that $\varphi (xa)={\overline{\chi }}(a)\varphi (x)$ for almost all $x\in G$ and $a\in A$ and such that

$$\begin{aligned} \Vert \varphi \Vert _{L^p(G/A)}^p=\int _{G/A}\vert \varphi (x)\vert ^pd{\dot{x}}<+\infty . \end{aligned}$$

Moreover, let ${\mathfrak {g}^*_\zeta }=\zeta + \mathfrak {a}^{\perp }$ and ${\mathcal {W}}_{ \zeta } ={\mathcal {W}}\cap {\mathfrak {g}_\zeta ^*}$. In this case, the Plancherel formula reads: if

$$\begin{aligned} \pi (\varphi )=\int _{G/A}\varphi (x)\pi (x)d{\dot{x}},\ \pi \in {{\hat{G}}}_\chi , \end{aligned}$$

then, for $\varphi \in L^1( G/A, \zeta )\cap L^2( G/A, \zeta )$ we have:

$$\begin{aligned} \Vert \varphi \Vert _2=\Big (\int _{{{\mathcal {W}}}_{\zeta }}\Vert \pi _{ l}(\varphi ) \Vert ^2_{HS}\vert Pf(l)\vert dl\Big )^{1\over 2}. \end{aligned}$$

(13)

If d is the maximal dimension of coadjoint orbits in $\mathfrak {g}^*$, then T has $n-d$ elements and thus $V_T$ can be identified with ${\mathbb {R}}^{n-d}$. We can identify $V_T$ with ${\mathbb {R}}X_1^*\oplus {\mathbb {R}}^{n-d-1}$. We denote by

$$\begin{aligned} \ p_*:V_T \rightarrow {\mathbb {R}}X_1^*, \ \ \ l\mapsto l_1 X_1^* \end{aligned}$$

the canonical projection. As ${\mathcal {W}}$ is a Zariski open set of $V_T$, $p_*({\mathcal {W}})$ is also a nonempty Zariski open set of ${\mathbb {R}}$. Then it will be convenient to write elements $l\in {\mathcal {W}}$, as $(l_1,l')$ where $l_1\in \ p_*({\mathcal {W}})$ and $l'\in {\mathcal {W}}_{{l_1}}=\{ l' \in {\mathbb {R}}^{n-d-1} \ : \ (l_1,l') \in {\mathcal {W}} \}$. It turns out that ${\mathcal {W}}_{{l_1}}$ is also a Zariski open set of ${\mathbb {R}}^{n-d-1}$ for each $l_1\in p_*({\mathcal {W}})$. The set ${\mathcal {W}}_{{l_1}}$ corresponds obviously to the cross-section ${{\mathcal {W}}}_{\zeta }$ used in the localized version of the Plancherel formula in (13). On the other hand, we obtain a decomposition of the Plancherel measure: for a function $F\in C_c({\mathcal {W}})$, we have

$$\begin{aligned} \int _{\mathcal {W}}F(l)dl=\int _{{\mathbb {R}}}\int _{{\mathcal {W}}_{l_1}}F(l)dl'\, dl_1, \end{aligned}$$

(14)

where the measure $dl'$ is induced on ${\mathcal {W}}_{l_1}$ by the Lebesque measure on ${\mathcal {W}}$.

Lemma 3

Let $f,\psi \in L^2(G)$ satisfying condition (2) of Theorem 3 and $\gamma \in ]0,\beta [$. Then there exists $c>0$, such that

$$\begin{aligned} \int _{{\mathbb {R}}^{n-1}}\left| \widehat{(f_\psi ^{k,s})_a}(l_1) \right| ^2da \le c\exp (-\pi (\alpha k^2+\alpha \Vert s\Vert ^2+\gamma l_1^2)), \end{aligned}$$

for all $k, l_1\in {\mathbb {R}}$ and almost all $s\in {\mathbb {R}}^{n-1}$.

Proof

Let h(k, s) be the function defined on ${\mathbb {R}}$ by

$$\begin{aligned} h(k,s)(\lambda )=\int _{{\mathbb {R}}^{n-1}} ((f_\psi ^{k,s})_a*(f_\psi ^{k,s})_a^\star )(\lambda )da, \end{aligned}$$

where $\lambda \in {\mathbb {R}}$ and $(f_\psi ^{k,s})_a^\star (\lambda )=\overline{(f_\psi ^{k,s})_a(-\lambda )}$. Then, $h(k,s)\in L^1({\mathbb {R}})$, for all $k\in {\mathbb {R}}$ and almost all $s\in {\mathbb {R}}^{n-1}$. In fact, from Lemma 2

$$\begin{aligned} \int _{\mathbb {R}}\vert h(k,s)(\lambda )\vert d\lambda\le & {} \int _{\mathbb {R}}\int _{{\mathbb {R}}^{n-1}}\int _{\mathbb {R}}\big \vert (f_\psi ^{k,s})_a(t)\big \vert \big \vert (f_\psi ^{k,s})_a(t-\lambda )\big \vert dt\, da\, d\lambda \nonumber \\= & {} \int _{{\mathbb {R}}^{n-1}}\Bigg (\int _{\mathbb {R}}\big \vert (f_\psi ^{k,s})_a(t)\big \vert dt\Bigg )^2da<\infty , \end{aligned}$$

(15)

for all $k\in {\mathbb {R}}$ and almost all $s\in {\mathbb {R}}^{n-1}$. Thus,

$$\begin{aligned} \widehat{h(k,s)}(l_1)=\int _{{\mathbb {R}}^{n-1}}\big \vert \widehat{(f_\psi ^{k,s})_a}(l_1) \big \vert ^2da. \end{aligned}$$

(16)

Identifying $A=\exp {\mathbb {R}}X_1$ with ${\mathbb {R}}$. Following the idea of Kaniuth and Kumar [14], for $u\in L^1(A)\cap L^2(A)$ define $u*f_\psi ^{k,s}$ on G by

$$\begin{aligned} u*f_\psi ^{k,s}(x)=\int _{A}u(z)f_\psi ^{k,s}(z^{-1}x)\,dz \end{aligned}$$

and then $h(k,s)_u: {\mathbb {R}}\rightarrow {\mathbb {C}}$ by

$$\begin{aligned} h(k,s)_u(t)=\int _{{\mathbb {R}}^{n-1}}\Big ((u*f_\psi ^{k,s})_a* \big ((u*f_\psi ^{k,s})_a\big )^\star \Big )(t)\,da. \end{aligned}$$

It is not hard to see that

$$\begin{aligned} h(k,s)_u(t)=\int _{{\mathbb {R}}^{n-1}} \big ((u*(f_\psi ^{k,s})_a)*(u*(f_\psi ^{k,s})_a)^\star \big )(t)\,da. \end{aligned}$$

Therefore, for every $\eta _1\in {\mathbb {R}}$

$$\begin{aligned} \widehat{h(k,s)_u}(\eta _1)&=\int _{{\mathbb {R}}^{n-1}} \Big \vert \big (u*(f_\psi ^{k,s})_a\big )^{\widehat{}}(\eta _1)\Big \vert ^2\,da \nonumber \\&=\vert {\hat{u}}(\eta _1)\vert ^2\int _{{\mathbb {R}}^{n-1}} \Big \vert \widehat{(f_\psi ^{k,s})_a}(\eta _1)\Big \vert ^2\,da =\vert {\hat{u}}(\eta _1)\vert ^2\widehat{h(k,s)}(\eta _1) \end{aligned}$$

(17)

(using Eq. (16)). Hence,

$$\begin{aligned} \int _{\mathbb {R}}\big \vert \widehat{h(k,s)_u}(\eta _1)\big \vert d\eta _1&=\int _{{\mathbb {R}}}\vert {\hat{u}}(\eta _1)\vert ^2\big \vert \widehat{h(k,s)}(\eta _1)\big \vert d\eta _1 \\&\le \int _{\mathbb {R}} \vert {\hat{u}}(\eta _1)\vert ^2 \Vert h(k,s) \Vert _1d\eta _1=\Vert u\Vert _2^2\Vert h(k,s) \Vert _1, \end{aligned}$$

which is finite by (15). By the inversion formula for ${\mathbb {R}}$, we have

$$\begin{aligned}&\int _{{\mathbb {R}}}\widehat{h(k,s)_u}(\eta _1)\,d\eta _1 =h(k,s)_u(0)\nonumber \\&\qquad =\int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}} \big \vert u*(f_\psi ^{k,s})_a (t)\big \vert ^2\,dt\,da =\Vert u*f_\psi ^{k,s}\Vert _2^2. \end{aligned}$$

(18)

Now fix $l_1\in {\mathbb {R}}$ and let $u_m\in L^1(A)$, $m \in {\mathbb {N}}^*$ such that $\widehat{u_m}(\eta _1)=1$ for $\eta _1\in V_m(l_1) =[l_1-(1/ 2m),l_1+(1/ 2m)]$ and $\widehat{u_m}(\eta _1)=0$ on the complement of $V_m(l_1)$. Noting that, $u_m$ is also in $L^2(A)$. Indeed, since $\widehat{u_m}\in L^1(A)\cap L^2(A)$,

$$\begin{aligned} \Vert u_m\Vert _2=\Vert {\check{u}}_m\Vert _2=\big \Vert \widehat{ \widehat{u_m}}\big \Vert _2=\Vert \widehat{u_m}\Vert _2<\infty , \end{aligned}$$

where ${\check{u}}_m(z)=u_m(-z)$.

As $\widehat{h(k,s)}$ is continuous and $V_m(l_1)$ has length 1/m, we have: for all $k\in {\mathbb {R}}$ and almost all $s\in {\mathbb {R}}^{n-1}$,

$$\begin{aligned} \widehat{h(k,s)}(l_1)&=\lim _{m\rightarrow \infty }m \int _{V_m(l_1)}\widehat{h(k,s)}(\eta _1)\,d\eta _1 =\lim _{m\rightarrow \infty }m \int _{{\mathbb {R}}}\widehat{h(k,s)}(\eta _1) {{\textbf {1}}}_{V_m(l_1)}\,d\eta _1 \\&= \lim _{m\rightarrow \infty }m \int _{{\mathbb {R}}}\widehat{h(k,s)_{u_m}}(\eta _1) \,d\eta _1\qquad (\text {using Eq.} (17))\\&=\lim _{m\rightarrow \infty }m \Vert u_m*f_\psi ^{k,s} \Vert _2^2 \qquad (\text {using Eq.} (18)) \\ {}&=\lim _{m\rightarrow \infty }m \int _{{\mathbb {R}}} \int _{{\mathcal {W}}_{\eta _1}}\vert Pf(\eta )\vert \Vert \pi _\eta (u_m*f_\psi ^{k,s}) \Vert ^2_{HS} \,d\eta '\, d\eta _1 \\&\qquad \qquad \qquad \qquad \qquad \qquad \quad (\text {using Eqs.} (5)\, \hbox {and}\, (14)) \\&=\lim _{m\rightarrow \infty }m \int _{{\mathbb {R}}}\vert \widehat{u_m}(\eta _1) \vert ^2\biggl (\int _{{\mathcal {W}}_{\eta _1}}\vert Pf(\eta )\vert \Vert \pi _\eta (f_\psi ^{k,s}) \Vert ^2_{HS} \,d\eta '\biggl )\,d\eta _1\\&=\lim _{m\rightarrow \infty }m \int _{V_m(l_1)}I\eta _1\,d\eta _1, \end{aligned}$$

where $\eta =(\eta _1,\eta ')$ and $\displaystyle I\eta _1=\int _{{\mathcal {W}}_{\eta _1}}\vert Pf(\eta )\vert \Vert \pi _\eta (f_\psi ^{k,s}) \Vert ^2_{HS} \,d\eta '$.

Now by (2) and (8), we obtain

$$\begin{aligned} I\eta _1 \le C \int _{{\mathcal {W}}_{\eta _1}} \vert Pf(\eta )\vert \exp ({-\pi (\alpha k^2+\alpha \Vert s\Vert ^2+\beta \Vert \eta \Vert ^2) }) d\eta '. \end{aligned}$$

Since $Pf(\eta )$ is a polynominal function of $\eta$, there exists $R>0$ such that

$$\begin{aligned} \vert {Pf(\eta )}\vert \exp ({-\pi (\beta -\gamma ) \Vert \eta \Vert ^2 })\le 1 \end{aligned}$$

for all $\eta \in \mathfrak {g} ^*$ with $\Vert {\eta }\Vert \ge R$. Let $K\ge 1$ such that $\vert Pf(\eta )\vert \le K$ for all $\eta \in \mathfrak {g}^*$ with $\Vert {\eta }\Vert \le R$. It follows that,

$$\begin{aligned} I\eta _1&\le C K \exp (-\pi (\alpha k^2+\alpha \Vert s\Vert ^2))\int _{{\mathcal {W}}_{\eta _1}} \exp \big ({-\pi \gamma \Vert \eta \Vert ^2 }\big ) d\eta ' \\&= C K \exp ({-\pi (\alpha k^2+\alpha \Vert s\Vert ^2)})\int _{{\mathbb {R}}^{n-d-1}} \exp \big ({-\pi \gamma \Vert (\eta _1,\eta ')\Vert ^2}\big ) d\eta ' \\&= C K \exp (-\pi (\alpha k^2+\alpha \Vert s\Vert ^2+\gamma \eta _1^2))\int _{{\mathbb {R}}^{n-d-1}} \exp \big ({-\pi \gamma \Vert \eta '\Vert ^2}\big ) d\eta '\\&= c \exp (-\pi (\alpha k^2+\alpha \Vert s\Vert ^2+\gamma \eta _1^2)), \end{aligned}$$

for some $c>0$. Therefore,

$$\begin{aligned} \widehat{h(k,s)}(l_1)&\le c\lim _{m\rightarrow \infty }m\int _{V_m(l_1)} \exp ({-\pi (\alpha k^2+\alpha \Vert s\Vert ^2+\gamma \eta _1^2)})d\eta _1 \\ {}&=c \exp ({-\pi ( \alpha k^2+\alpha \Vert s\Vert ^2+\gamma l_1^2)}). \end{aligned}$$

Finally, Eq. (16) allows us to conclude. $\square$

4 Proof of Theorem 3

For $a=(a_2,...,a_n), s=(s_2,...,s_n)\in {\mathbb {R}}^{n-1}$, let $f_{a,s}$,$\psi _{a,s}$ be the complex-valued functions defined on ${\mathbb {R}}$ by

$$\begin{aligned} f_{a,s}(t)= & {} f\Big ( \exp \Big ( \big (t-Q_1(a,s)\big )X_1+ \sum _{j=2}^{n} a_jX_j \Big )\Big ),\\&\text {and} \ \ \psi _{a,s}(t)=\psi \Big (\exp \Big ( tX_1+ \sum _{j=2}^{n} Q_j(a,s)X_j \Big ) \Big ), \end{aligned}$$

where the polynomial functions $Q_j$ are defined as in (10). Then obviously $f_{a,s},\psi _{a,s}\in L^2({\mathbb {R}})$, for almost all $a\in {\mathbb {R}}^{n-1}$ and all $s\in {\mathbb {R}}^{n-1}$. For fixed $\lambda$, $y\in {\mathbb {R}}$, let $F_{\lambda ,y}$ and $K_{\lambda ,y}$ be the functions defined on ${\mathbb {R}}\times {\mathbb {R}}$ by

$$\begin{aligned} F_{\lambda ,y}(k,l_1)={\mathcal {G}}_{\psi _{a,s}}({\mathcal {M}}_\lambda {\mathcal {T}}_yf_{a,s})(k,l_1) {\mathcal {G}}_{\psi _{a,s}}({\mathcal {M}}_\lambda {\mathcal {T}}_yf_{a,s})(-k,-l_1)e^{2i\pi k l_1}, \end{aligned}$$

and

$$\begin{aligned} K_{\lambda ,y}(k,l_1)=\int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}^{n-1}}F_{\lambda ,y}(k,l_1) \phi (a,s) da\, ds, \end{aligned}$$

where $\phi \in {\mathcal {S}}({\mathbb {R}}^{n-1}\times {\mathbb {R}}^{n-1})$, the Schwartz space of ${\mathbb {R}}^{n-1}\times {\mathbb {R}}^{n-1}$. Now for fixed $\mu \in {\mathbb {R}}$, let $R_{\lambda ,y,\mu }$ be the function defined on ${\mathbb {R}}$ by

$$\begin{aligned} R_{\lambda ,y,\mu }(k)=K_{\lambda ,y}(k,.)^{\widehat{}}(\mu ), \end{aligned}$$

(19)

where $K_{\lambda ,y}(k,.)^{\widehat{}}$ is the partial Fourier transform of $K_{\lambda ,y}$ with respect the second variable $l_1$. It follows, using Lemma 1, that

$$\begin{aligned} {\widehat{R}}_{\lambda ,y,\mu }(w)={\widehat{K}}_{\lambda ,y}(w,\mu )= & {} \displaystyle \int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}^{n-1}}{\hat{F}}_{\lambda ,y}(w,\mu )\phi (a,s)da\, ds\nonumber \\= & {} \int _{{\mathbb {R}}^{n-1}}\displaystyle \int _{{\mathbb {R}}^{n-1}}F_{\lambda ,y}(-\mu ,w)\phi (a,s)da\, ds=K_{\lambda ,y}(-\mu ,w).\nonumber \\ \end{aligned}$$

(20)

Lemma 4

There exists a positive constant $C_1$ such that

$$\begin{aligned} \Big \vert R_{\lambda ,y,\mu }(k)\Big \vert \le C_1 \, e^{-\alpha \pi k^2}. \end{aligned}$$

Moreover, the constant $C_1$ does not depend on $\lambda$, $\mu$ and y.

Proof

From Eq. (19) we have,

$$\begin{aligned}&\vert R_{\lambda ,y,\mu }(k)\vert \\&\quad =\vert K_{\lambda ,y} (k,.)^{\widehat{}}(\mu )\vert \\&\quad \le \displaystyle \int _{{\mathbb {R}}}\int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}^{n-1}}\Big \vert {\mathcal {G}}_{\psi _a,s}({\mathcal {M}}_\lambda {\mathcal {T}}_yf_{a,s})(k,l_1)\Big \vert \\&\qquad \Big \vert {\mathcal {G}}_{\psi _a,s}({\mathcal {M}}_\lambda {\mathcal {T}}_yf_{a,s})(-k,-l_1)\Big \vert \Big \vert \phi (a,s)\Big \vert da\, ds\, dl_1\\&\quad \le \text {cst} \displaystyle \int _{{\mathbb {R}}}\int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}^{n-1}}\Big \vert {\mathcal {G}}_{\psi _a,s}({\mathcal {M}}_\lambda {\mathcal {T}}_yf_{a,s})(k,l_1)\Big \vert \\&\qquad \Big \vert {\mathcal {G}}_{\psi _a,s}({\mathcal {M}}_\lambda {\mathcal {T}}_yf_{a,s})(-k,-l_1)\Big \vert da\, ds\, dl_1 . \end{aligned}$$

By using Cauchy-Schwartz inequality, we obtain

$$\begin{aligned} \vert R_{\lambda ,y,\mu }(k)\vert\le & {} \text {cst}\left( \displaystyle \int _{{\mathbb {R}}}\int _{{\mathbb {R}}^{n-1}}\left( \int _{{\mathbb {R}}^{n-1}}\big \vert {\mathcal {G}}_{\psi _a,s}({\mathcal {M}}_\lambda {\mathcal {T}}_yf_{a,s})(k,l_1)\big \vert ^2da\right) ds\,dl_1\right) ^{\frac{1}{2}}\\&\times \left( \displaystyle \int _{{\mathbb {R}}}\int _{{\mathbb {R}}^{n-1}}\left( \int _{{\mathbb {R}}^{n-1}}\big \vert {\mathcal {G}}_{\psi _a,s}({\mathcal {M}}_\lambda {\mathcal {T}}_yf_{a,s})(-k,-l_1)\big \vert ^2da\right) ds\,dl_1\right) ^{\frac{1}{2}}. \end{aligned}$$

Remark that,

$$\begin{aligned} \Big \vert {\mathcal {G}}_{\psi _{a,s}}({\mathcal {M}}_\lambda {\mathcal {T}}_yf_{a,s})(k,l_1)\Big \vert =\Big \vert {\mathcal {G}}_{\psi _{a,s}}f_{a,s}(k-y,l_1-\lambda )\Big \vert \end{aligned}$$

(using i) in Lemma 1)

$$\begin{aligned}&=\Bigg \vert \displaystyle \int _{{\mathbb {R}}}f_{a,s}(t)\overline{\psi _{a,s}}(t-k+y)\text {e}^{-2i\pi t(l_1-\lambda )}dt\Bigg \vert \\&\qquad =\Bigg \vert \int _{{\mathbb {R}}}f\left( \exp \left( (t-Q_1(a,s))\right) X_1+ \sum _{j=2}^{n} a_jX_j \right) \\&\qquad \times \ {\overline{\psi }}\left( \exp \left( ( t-k+y)X_1+ \sum _{j=2}^{n} Q_j(a,s)X_j \right) \right) \text {e}^{-2i\pi t(l_1-\lambda )}dt\Bigg \vert \\&\quad =\Bigg \vert \int _{{\mathbb {R}}}f\left( \exp \left( rX_1+ \sum _{j=2}^{n} a_jX_j \right) \right) \\&\qquad \times {\overline{\psi }}\left( \exp \left( ( r-(k-y)+Q_1(a,s))X_1+ \sum _{j=2}^{n} Q_j(a,s)X_j \right) \right) \text {e}^{-2i\pi r(l_1-\lambda )}dr\Bigg \vert \end{aligned}$$

(by substituting $r=t-Q_1(a,s)$ for t)

$$\begin{aligned}&=\Big \vert \int _{{\mathbb {R}}}f\left( \exp \left( rX_1+ \sum _{j=2}^{n} a_jX_j \right) \right) \\&\quad \times {\overline{\psi }}\left( \exp \left( (k-y) X_1+ \sum _{j=1}^{n} s_jX_j \right) ^{-1}\exp \left( r X_1+\sum _{j=2}^{n} a_jX_j\right) \right) \text {e}^{-2i\pi r(l_1-\lambda )}dr\Big \vert \end{aligned}$$

(using Eq. (10))

$=\displaystyle \left| \int _{{\mathbb {R}}}f_\psi ^{k-y,s}\left( \exp \left( rX_1+ \sum _{j=2}^{n} a_jX_j \right) \right) \text {e}^{-2i\pi r(l_1-\lambda )}dr\right| =\left| \widehat{(f_\psi ^{k-y,s})_a}(l_1-\lambda )\right|$.

It results that,

$$\begin{aligned} \big \vert R_{\lambda ,y,\mu }(k) \big \vert\le & {} \text {cst}\left( \displaystyle \int _{{\mathbb {R}}}\int _{{\mathbb {R}}^{n-1}} \left( \int _{{\mathbb {R}}^{n-1}}\big \vert \widehat{(f_\psi ^{k-y,s})_a}(l_1 - \lambda )\big \vert ^2da\right) ds\,dl_1\right) ^{\frac{1}{2}}\\&\times \left( \displaystyle \int _{{\mathbb {R}}}\int _{{\mathbb {R}}^{n-1}} \int _{{\mathbb {R}}^{n-1}}\big \vert \widehat{(f_\psi ^{-k-y,s})_a}(-l_1 - \lambda )\big \vert ^2da\,ds\,dl_1\right) ^{\frac{1}{2}} \\\le & {} \text {cst} \left( \displaystyle \int _{{\mathbb {R}}}\int _{{\mathbb {R}}^{n-1}} e^{-\pi (\alpha \vert k-y\vert ^2+\gamma (l_1-\lambda )^2+\alpha \Vert s\Vert ^2)} ds\,dl_1\right) ^{\frac{1}{2}}\\&\times \left( \displaystyle \int _{{\mathbb {R}}}\int _{{\mathbb {R}}^{n-1}} e^{-\pi (\alpha \vert k+y\vert ^2+\gamma (l_1+\lambda )^2+\alpha \Vert s \Vert ^2)} ds\,dl_1\right) ^{\frac{1}{2}} \quad (\text {using Lemma}\, 3) \\\le & {} \text {cst}\, e^{-\frac{\pi }{2}\alpha (\vert k-y\vert ^2+\vert k+y\vert ^2)}\left( \int _{{\mathbb {R}}^{n-1}} e^{-\pi \alpha \Vert s\Vert ^2} ds\right) \left( \int _{{\mathbb {R}}}e^{-\pi \gamma l_1^2} dl_1\right) \\\le & {} \text {cst}\, e^{-\pi \alpha k^2}, \end{aligned}$$

which is the desired result. $\square$

Lemma 5

There exists a positive constant $C_2$ such that

$$\begin{aligned} \big \vert {\hat{R}}_{\lambda ,y,\mu }(w)\big \vert \le C_2\, e^{-\pi \gamma w^2}. \end{aligned}$$

Moreover, the constant $C_2$ does not depend on $\lambda$, y and $\mu$.

Proof

By (20), we have

$$\begin{aligned}&\big \vert {\hat{R}}_{\lambda ,y,\mu }(w)\big \vert =\big \vert K_{\lambda ,y}(-\mu ,w)\big \vert \\&\quad \le \text {cst}\int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}^{n-1}}\Big \vert {\mathcal {G}}_{\psi _a,s}({\mathcal {M}}_\lambda {\mathcal {T}}_yf_{a,s})(-\mu ,w)\Big \vert \Big \vert {\mathcal {G}}_{\psi _a,s}({\mathcal {M}}_\lambda {\mathcal {T}}_yf_{a,s})(\mu ,-w)\Big \vert dads. \end{aligned}$$

As in the proof of the Lemma 4 we can show that,

$$\begin{aligned}&\big \vert {\hat{R}}_{\lambda ,y,\mu }(w)\big \vert \le \text {cst}\, e^{-\frac{\pi \gamma }{2}(\vert w-\lambda \vert ^2+\vert w+\lambda \vert ^2)}e^{\frac{-\pi a}{2}(\vert \mu +y\vert ^2+\vert -\mu + y\vert ^2) } \times \left( \int _{{\mathbb {R}}^{n-1}} e^{-\pi \alpha \Vert s \Vert ^2 } ds\right) \\&\quad \le \text {cst}\, e^{-\pi \gamma w^2}, \end{aligned}$$

which is the desired result. $\square$

We have shown finally that $R_{\lambda ,y,\mu }$ verifies the decay conditions of Hardy theorem on ${\mathbb {R}}$. Since $\alpha \beta >1$, we can choose $0<\gamma < \beta$ such that $\alpha \gamma >1$. We conclude that $R_{\lambda ,y,\mu }=0$ a.e. and ${\hat{R}}_{\lambda ,y,\mu }=0$ for all $\lambda , y, \mu \in {\mathbb {R}}$. In (20), allowing $\phi$ to vary through the space of Schwartz functions on ${\mathbb {R}}^{n-1}\times {\mathbb {R}}^{n-1}$, we obtain $F_{\lambda ,y}(-\mu ,w)=0$ for all $\lambda , y, \mu$ in ${\mathbb {R}}$ and almost all $w \in {\mathbb {R}}$. As $F_{-\lambda ,-y}$ is continuous on ${\mathbb {R}}\times {\mathbb {R}}$,

$$\begin{aligned} \vert F_{-\lambda ,-y}(0,0)\vert =\vert G_{\psi _{a,s}f_{a,s}(y,\lambda )}\vert ^2=0 \end{aligned}$$

(using i) in Lemma 1). Hence, ${\mathcal {G}}_{\psi _{a,s}}f_{a,s}=0$ a.e. By using Eq. (4), we have

$$\begin{aligned} \Vert \psi _{a,s}\Vert _2^2 \Vert f_{a,s}\Vert _2^2 =0, \end{aligned}$$

which implies either $\psi _{a,s}=0$ a.e. or $f_{a,s}=0$ a.e. Observe that,

$$\begin{aligned}&\int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}^{n-1}} \Vert f_{a,s} \Vert _2^2\Vert \psi _{a,s} \Vert _2^2da\, ds \\&\quad =\int _{{\mathbb {R}}^{n-1}}\int _{{\mathbb {R}}^{n-1}}\left( \int _ {\mathbb {R}}\Big \vert f\left( \exp \left( \left( t-Q_1(a,s)\right) X_1+ \sum _{j=2}^{n} a_jX_j \right) \right) \Big \vert ^2 dt\right) \\&\qquad \times \left( \int _ {\mathbb {R}} \Big \vert \psi \left( \exp \left( tX_1+ \sum _{j=2}^{n} Q_j(a,s)X_j \right) \right) \Big \vert ^2 dt\right) da\, ds =\Vert f \Vert _2^2\Vert \psi \Vert _2^2 \end{aligned}$$

(by substituting $t-Q_1(a,s)$ for t and $Q_j(a,s)$ for $s_j$, $j=2,...,n$, using Eq. (11)). This allow us to achieve the proof.