Spectral Multipliers on 2-Step Stratified Groups, I

Abstract

Given a 2-step stratified group which does not satisfy a slight strengthening of the Moore–Wolf condition, a sub-Laplacian ${\mathcal {L}}$ and a family ${\mathcal {T}}$ of elements of the derived algebra, we study the convolution kernels associated with the operators of the form $m({\mathcal {L}}, -\,i {\mathcal {T}})$. Under suitable conditions, we prove that: (i) if the convolution kernel of the operator $m({\mathcal {L}},-\,i {\mathcal {T}})$ belongs to $L^1$, then m equals almost everywhere a continuous function vanishing at $\infty$ (‘Riemann–Lebesgue lemma’); (ii) if the convolution kernel of the operator $m({\mathcal {L}},-\,i{\mathcal {T}})$ is a Schwartz function, then m equals almost everywhere a Schwartz function.

1 Introduction

It is well-known that the Fourier transform on $\mathbb {R}^n$ (or the torus $\mathbb {T}$) provides an important tool for the study of translation-invariant (differential) operators, for instance because it simultaneously ‘diagonalizes’ all these operators. This tool has been extended in various ways during the twentieth century, enhancing the study of left- or right-invariant (differential) operators on compact (Lie) groups (cf. [59]), on abelian locally compact (Lie) groups (cf. [70]), and then on general locally compact (Lie) groups (cf., for instance, [23, 32, 45, 46] for a comprehensive review of the literature). Nonetheless, the so-extended Fourier transform on a non-commutative group is operator-valued, so that it is far less manageable than in the abelian case.

In some situations, further commutativity assumptions make it possible to work with an essentially scalar-valued ‘portion’ of the Fourier transform. This is the case, for instance, in the setting of Gelfand pairs (cf. [27, 39, 72]). Let us briefly review the basic aspects of this theory, with an eye towards the kind of calculi we shall consider in the body of this paper. Assume that there is a compact Lie group K which acts analytically on G by group automorphisms in such a way that the algebra $L^1_K(G)$ of K-invariant elements of $L^1(G)$ is commutative. Then, $(G\rtimes K,K)$ is a Gelfand pair and the associated homogeneous space, that is, the quotient of $G\rtimes K$ by K, can be identified with G. Under these assumptions, the (commutative) algebra ${\mathfrak {D}}_K$ of left- and K-invariant differential operators has a finite number of generators ${\mathcal {L}}_1,\ldots ,{\mathcal {L}}_n$. Then, the Gelfand spectrum of the Banach $*$-algebra $L^1_K(G)$, that is, the space of non-zero continuous multiplicative linear functionals on $L^1_K(G)$, can be identified in a natural way with the joint spectrum $\sigma ({\mathcal {L}}_1,\ldots , {\mathcal {L}}_n)$ of the commutative family of self-adjoint closures of the operators ${\mathcal {L}}_1,\ldots ,{\mathcal {L}}_n$ of $L^2(G)$ (with initial domains $C^\infty _c(G)$); in addition, every $f\in L^1_K(G)$ can be interpreted as the (right) convolution kernel of the operator $m({\mathcal {L}}_1,\ldots ,{\mathcal {L}}_n)$ on $L^2(G)$, where m is the Gelfand transform of f and $m({\mathcal {L}}_1,\ldots ,{\mathcal {L}}_n)$ is defined by means of the spectral theorem (cf. [48, Sect. 5.3]). The correspondence between f and m can then be extended to an isometry of the space of K-invariant endomorphisms of $L^2(G)$ onto the space $L^\infty (\beta )$, where $\beta$ is a suitable measure on $\sigma ({\mathcal {L}}_1,\dots ,{\mathcal {L}}_n)$, and also to an isometry between $L^2_K(G)$ (the space of K-invariant elements of $L^2(G)$) and $L^2(\beta )$.

A classical example of this situation is the additive group $\mathbb {R}^n$ endowed with the action of the special orthogonal group SO(n), in which case the algebra ${\mathfrak {D}}_{SO(n)}$ is generated by the standard Laplacian $\Delta$, and the resulting calculus is essentially related to the analysis of radial functions. Another thoroughly studied example is that of the Heisenberg group $\mathbb {H}^n$ endowed with the action of the n-torus $\mathbb {T}^n$ (by componentwise multiplication on the component $\mathbb {C}^n$ of $\mathbb {H}^n=\mathbb {C}^n\times \mathbb {R}$), in which case the algebra ${\mathfrak {D}}_{\mathbb {T}^n}$ is generated by $X_1^2+Y_1^2,\dots , X_n^2+Y_n^2, i T$, where $X_1,\dots , X_n, Y_1,\dots , Y_n,T$ is the standard basis of left-invariant vector fields on $\mathbb {H}^n$, while the resulting analysis is essentially related to the study of ‘polyradial’ functions (cf., for instance, [4]).

Nonetheless, the setting of Gelfand pairs can be too narrow for some purposes. For example, for there to exist a group K as above, the group G must be unimodular (cf. [39, Theorem IV.3.1]); if G is solvable, then it must have polynomial growth; if G is nilpotent, then it is necessarily of step 2 (cf. [8, Corollary 7.4 and Theorem A]). In addition, even when such K exists, dealing with families of (K-invariant) operators which do not generate the algebra ${\mathfrak {D}}_K$ still provides new kinds of issues. For example, when $G=\mathbb {H}^n$ and $K=\mathbb {T}^n$ as above, the study of a Laplacian of the form ${\mathcal {L}}=\sum _{j=1}^n a_j(X_j^2+Y_j^2)+b T^2$, with $a_1,\dots , a_n,b>0$, by means of the Gelfand calculus is not trivial; for instance, there are no general means to determine whether an element of $L^1_{\mathbb {T}^n}$ is the convolution kernel associated with some bounded function of ${\mathcal {L}}$ or not.

Besides that, the study of more general operators has proved interesting in its own right, such as the study of sub-Laplacians (as ‘local’ models of subelliptic operators on sub-Riemannian manifolds, cf., for example, [62]), Rockland operators (as higher-order analogues of homogeneus sub-Laplacians on graded groups, see Sect. 3), and, more generally, weighted subcoercive operators (as analogues of positive Rockland operators on general Lie groups, see Sect. 3). The study of the functional calculi associated with these kinds of operators can be pursued in various directions, spreading from the proof of multiplier theorems (cf., for instance, [20, 37, 49,50,51,52, 54, 58]) to the local or microlocal study of pseudo-differential operators (cf., for instance, [61, Sect. 8]).

In this paper, we shall focus our attention on the functional calculi arising from the spectral theorem for commutative families of self-adjoint operators on hilbertian spaces. Calculi of this kind have already been considered, more or less explicitly, in a number of works (cf. [4, 5, 20, 29, 31, 42, 47, 48, 53, 66, 69] to name only a few).

A very general class of operators for which this kind of analysis can be efficiently carried on is that of weighted subcoercive systems of differential operators on connected Lie groups (cf. Sect. 3 for more details), as the analysis pursued in [47, 48] shows. Then, assume that ${\mathcal {L}}_1,\dots , {\mathcal {L}}_k$ form a weighted subcoercive system of differential operators, and that G is a connected and unimodular Lie group to avoid technicalities. It follows that these operators are essentially self-adjoint on $L^2(G)$ (with initial domain $C^\infty _c(G)$) and that their self-adjoint extensions commute, so that there is a unique spectral measure $\mu$ on $\mathbb {R}^k$ such that

$$\begin{aligned} {\mathcal {L}}_j \varphi =\int _{\mathbb {R}^k} \lambda _j \,\text {d}\mu (\lambda ) \,\varphi \end{aligned}$$

for every $\varphi \in C^\infty _c(G)$. If $m:\mathbb {R}^k\rightarrow \mathbb {C}$ is bounded and $\mu$-measurable, we may then associate with m a distribution ${\mathcal {K}}(m)$ such that

$$\begin{aligned} m({\mathcal {L}}_1,\dots , {\mathcal {L}}_k) \, \varphi =\int _{\mathbb {R}^k} m(\lambda )\,\text {d}\mu (\lambda ) \varphi = \varphi *{\mathcal {K}}(m) \end{aligned}$$

for every $\varphi \in C^\infty _c(G)$. Thus, ${\mathcal {K}}$ generalizes both the inverse Fourier transform on the classical abelian case, and an ‘inverse Gelfand transform’ in the case of Gelfand pairs considered above. It is then natural to investigate which properties ${\mathcal {K}}$ shares with the preceding transforms. On the one hand, in full generality one may prove that the following hold (cf. [47, 48, 66]):

• there is a positive Radon measure $\beta$ on $\mathbb {R}^k$ such that ${\mathcal {K}}$ extends to an isometry of $L^2(\beta )$ into $L^2(G)$ (‘Plancherel measure’).

• ${\mathcal {K}}$ extends to a continuous linear mapping (of norm 1) from $L^1(\beta )$ into $C_0(G)$ (‘Riemann–Lebesgue lemma’).

• there is an ‘integral kernel’ $\chi \in L^\infty (\beta \otimes \nu _G)$ such that, for every $m\in L^1(\beta )$,

  $$\begin{aligned} {\mathcal {K}}(m)(g)=\int _{\mathbb {R}^k} m(\lambda )\chi (\lambda ,g)\,\text {d}\beta (\lambda ) \end{aligned}$$

  for almost every $g\in G$.¹

• if G is a group of polynomial growth, then ${\mathcal {K}}$ maps ${\mathcal {S}}(\mathbb {R}^k)$ into ${\mathcal {S}}(G)$ (suitably defined).

On the other hand, some questions are still open in full generality, such as:

(RL):

if $m\in L^\infty (\mu )$ and ${\mathcal {K}}(m)\in L^1(G)$, does m necessarily admit a continuous representative?

(S):

if G is a group of polynomial growth and ${\mathcal {K}}(m)\in {\mathcal {S}}(G)$ for some $m\in L^\infty (\mu )$, does m necessarily admit a representative in ${\mathcal {S}}(\mathbb {R}^k)$?

Property (RL) is another analogue of the classical Riemann–Lebesgue lemma, and, to the best of our knowledge, has only been addressed for sub-Laplacians on stratified groups and the plane motion group [53] and for homogeneous sub-Laplacians and invariant derivatives along the centre in $MW^+$ groups [19] (cf. Definition 6.1). In addition to that, property (RL) trivially holds when ${\mathcal {L}}_1,\dots , {\mathcal {L}}_k$ generate the algebra of invariant operators on a Gelfand pair, cf. [48, Corollary 5.7] for further details.

Concerning property (S), it has already been studied for sub-Laplacians on solvable Lie groups [53], on several Gelfand pairs [4, 5, 29, 31], and for homogeneous sub-Laplacians and invariant derivatives along the centre in $MW^+$ groups [19].

The aim of the present paper is to further develop the theory concerning properties (RL) and (S) and to provide further examples where these properties hold.

Even though some of our main results hold in greater generality, in order to keep the exposition as simple as possible we shall confine ourselves to a homogeneous setting, making only use of homogeneous differential operators on homogeneous groups. In this particular setting, the notion of a weighted subcoercive system reduces to that of a Rockland family (cf. Definition 3.6).

Recall that, if G is a homogeneous group and ${\mathcal {L}}$ is a homogeneous left-invariant differential operator, then ${\mathcal {L}}$ is said to be Rockland if it is hypoelliptic; by a slight abuse of notation, we shall not consider constant operators to be Rockland, for technical convenience. Recall that Rockland operators were introduced in [61], where Rockland proved that a homogeneous left-invariant differential operator ${\mathcal {L}}$ on a Heisenberg group G is hypoelliptic if and only if $\text {d}\pi ({\mathcal {L}})$ is one-to-one on smooth vectors for every non-trivial irreducible continuous unitary representation $\pi$ of G, and conjectured that the same held for more general groups.² Notice that this characterization of hypoellipticity by means of the group Fourier transform generalizes in a natural way the analogous characterization of hypoelliptic homogeneous differential operators with constant coefficients on the Euclidean spaces. After Rockland’s conjecture was solved in the affirmative by Helffer and Nourrigat [38], several mathematicians provided deeper insight into the properties of such operators, studying the corresponding functional calculi (cf., for instance, [33, 37, 42, 49]), the properties of the corresponding heat semi-groups (cf., for instance, [6]), and the associated Sobolev spaces (cf., for instance, [30]).

Observe that, if $k=1$ and ${\mathcal {L}}_1$ is a Rockland operator, then property (RL) holds trivially (cf. Theorems 3.21 and 3.26), while property (S) can be characterized in a simple way, at least on abelian groups (cf. [19, Theorem 3.2]). We shall therefore concentrate on the case of more operators, which is much more involved; in this situation, properties (RL) and (S) may fail even in relatively simple contexts like abelian groups or the Heisenberg groups (cf. [19, Propositions 5.5, 8.1, 8.5 and Theorem 7.4]).

In the first part of the paper, we introduce Rockland families on homogeneous groups, and some relevant objects such as the ‘kernel transform’ ${\mathcal {K}}$, the ‘Plancherel measure’ $\beta$, the ‘integral kernel’ $\chi$, and the ‘multiplier transform’ ${\mathcal {M}}$ (Sect. 3). Then, we discuss the possibility of transferring properties (RL) and (S) to products of groups (Sect. 4) or to polynomial images of the given families (Sect. 5). The former case is relatively simple, and we are able to prove the following result (cf. Theorems 4.4 and 4.6 ).

Theorem 1.1

Let $G_1$ and $G_2$ be two homogeneous groups, ${\mathcal {L}}_{1,1},\dots , {\mathcal {L}}_{1,k}$ a Rockland family of $G_1$, and ${\mathcal {L}}_{2,1},\dots , {\mathcal {L}}_{2,h}$ a Rockland family on $G_2$, both satisfying property (RL) (resp. (S)). Then the corresponding Rockland family ${\mathcal {L}}'_{1,1},\dots , {\mathcal {L}}'_{1,k},{\mathcal {L}}_{2,1}',\dots ,{\mathcal {L}}_{2,h}'$ on $G_1\times G_2$ satisfies property (RL) (resp. (S)).

For what concerns those families whose elements are polynomial functions in a given family, a wide range of situations can occur. We shall collect in Sect. 10 some technical results which can be applied in several situations.

In the second part of the paper, we focus on the case of sub-Laplacians and bi-invariant vector fields on a 2-step stratified group G. Even in this specific context, there are two classes of such groups where the families of the preceding kind behave quite differently:

• the groups G which have a homogeneous subgroup $G'$ contained in [G, G] such that the quotient of G by $G'$ is a Heisenberg group;

• the groups G which have no such quotients.

We call the groups of the first kind $MW^+$ groups, or groups satisfying the $MW^+$ condition, since the condition which defines these groups is a slight strengthening of the Moore–Wolf condition (cf. [56, 57]); in fact, the condition that was actually considered in [56] is related to the centre Z of G instead of [G, G]. Nevertheless, one may always factor out an abelian group so as to reduce to a group with $Z=[G,G]$ (cf. Remark 7.12). Since the treatment of these two classes of groups is quite different, we focus here on groups which do not satisfy the $MW^+$ condition; $MW^+$ groups are studied in [19].

Our main results in this direction can be summarized as follows (cf. Theorems 8.5, 8.4, 8.2, and 9.2 ).

Theorem 1.2

Let G be a 2-step stratified group which does not satisfy the $MW^+$ condition. Let ${\mathcal {L}}$ be a homogeneous sub-Laplacian on G and $T_1,\dots , T_n$ a basis of the the derived algebra of G. Then the following hold:

• if $0\leqslant n'<n$, then $({\mathcal {L}},i T_1,\dots , i T_{n'})$ satisfies property (RL);

• if G is a free 2-step stratified group, then $({\mathcal {L}},i T_1,\dots , i T_n)$ satisfies property (RL);

• if $W=\lbrace 0 \rbrace$ (cf. Definition 7.1), then $({\mathcal {L}},i T_1,\dots , i T_{n'})$ satisfies properties (RL) and (S) for every $n'=0,\dots , n$.

In Sect. 7, we give an expression for the Plancherel measure and the integral kernel. Sections 8 and 9 are devoted to the proof of Theorem 1.2.

2 Definitions and Notation

2.1 Homogeneous Groups

Here we recall some basic definitions and properties of homogeneous groups. Cf. [33] for a more detailed exposition.

A homogeneous Lie algebra ${\mathfrak {g}}$ is a finite-dimensional Lie algebra over $\mathbb {R}$ endowed with a family of automorphisms $(\delta _r)_{r>0}$ such that $\delta _r=r^{ A}$ for every $r>0$, where A is a diagonalizable endomorphism of the vector space ${\mathfrak {g}}$ with eigenvalues $>0$. We shall generally write $r\cdot x$ instead of $\delta _r(x)$ for every $r>0$ and for every $x\in {\mathfrak {g}}$. A homogeneous group is a connected and simply-connected Lie group whose Lie algebra is homogeneous.

Notice that a homogeneous Lie algebra ${\mathfrak {g}}$ is necessarily nilpotent (cf. [33, Proposition 1.3]), though not all nilpotent Lie algebras admit a homogeneous structure (cf. [25]). In addition, the Baker–Campbell–Hausdorff formula defines a Lie group structure on ${\mathfrak {g}}$ for which the exponential map is the identity. Conversely, if G is a homogeneous group, then the corresponding exponential map is an analytic diffeomorphism. For this reason, it is customary to identify G with its Lie algebra as manifolds. In particular, sometimes the identity e of G is denoted by 0.

Let G be a homogeneous group. Then, we shall denote by $\nu _G$ a fixed Haar measure on G; we also denote by Q the homogeneous dimension of G, that is, the unique real number such that $\nu _G(r\cdot B)=r^Q \nu _G(B)$ for every $\nu _G$-measurable subset B of G and for every $r>0$.

Notice that $\nu _G$ is both left- and right-invariant; in addition, it induces the Lebesgue measure on the Lie algebra of G by means of the exponential map (cf. [33, Proposition 1.2]). Therefore, with the previous notation, the homogeneous dimension Q of G is the trace of A.

A homogeneous norm on G is a proper mapping $|\,\cdot \, |:G\rightarrow \mathbb {R}_+=[0,+\infty [$ which is symmetric³ and homogeneous of degree 1.

Sometimes homogeneous norms are also required to be of class $C^\infty$ on the complement of the identity; we shall not generally need this further assumption, but we recall that such homogeneous norms always exist (cf. [33, p. 8]). Notice that a homogeneous norm $|\,\cdot \, |$ is quasi-subadditive; in other words, there is a constant $C>0$ such that $|xy |\leqslant C(|x |+|y |)$ for every $x,y\in G$ (cf. [33, Proposition 1.6]). A homogeneous group admits subadditive homogeneous norms if (and only if) all the eigenvalues of A are $\geqslant 1$, with the previous notation (cf. [36]).

If T is a distribution on G, then we define $\langle {\check{T}}, \varphi \rangle =\langle T, \varphi (\,\cdot \,)^{-1}\rangle$ and $\langle T^*,\varphi \rangle {:}{=}\overline{\langle T, \overline{\varphi (\,\cdot \,^{-1})}\rangle }$ for every $\varphi \in C^\infty _c(G)$. In particular, if $f\in L^1_\text {loc}(G)$, then ${\check{f}}(x)=f(x^{-1})$ and $f^*(x)=\overline{f(x^{-1})}$ for almost every $x\in G$.

A function $f\in L^\infty (G)$ is of positive type if $\langle \varphi *f\vert \varphi \rangle \geqslant 0$ for every $\varphi \in C^\infty _c(G)$. In this case, $f=f^*$ and f has a continuous representative $f_0$ such that $\Vert f\Vert _\infty$ (cf. [32, Corollaries 3.21 and 3.22]).

Let X a (linear) differential operator (with smooth coefficients) on G. The formal adjoint $X^*$ of X is the unique differential operator on G such that $\int (X \varphi ){\overline{\psi }}\,\text {d}\nu _G=\int \varphi \overline{(X^*\psi )}\,\text {d}\nu _G$ for every $\varphi ,\psi \in C^\infty _c(G)$. The operator X is formally self-adjoint if $X=X^*$.

The operator X is homogeneous of degree $d\in \mathbb {C}$ if $X [ \varphi (r\,\cdot \,) ]=r^d (X \varphi )(r\,\cdot \,)$ for every $\varphi \in C^\infty (G)$ and for every $r>0$. We say that X is Rockland if it is homogeneous, left-invariant, hypoelliptic (cf. Definition 3.4 below), and annihilates constants.

Definition 2.1

Let G be a homogeneous group. Then, ${\mathcal {S}}(G)$ is the space of $f\in C^\infty (G)$ such that $(1+|\,\cdot \, |)^k X f$ is bounded for every $k\in \mathbb {N}$ and for every homogeneous differential operator X on G, endowed with the topology induced by the semi-norms $f\mapsto \Vert (1+|\,\cdot \, |)^k X f \Vert _\infty$. The dual of ${\mathcal {S}}(G)$ is denoted by ${\mathcal {S}}'(G)$.

Notice that the exponential mapping of G induces an isomorphism of ${\mathcal {S}}(G)$ onto the Schwartz space on the Lie algebra of G (cf. [33, Section D of Chapter 1]); a similar statement holds for ${\mathcal {S}}'(G)$.

2.2 Representations

We denote by $\langle \,\cdot \,\vert \,\cdot \,\rangle _H$, or simply $\langle \,\cdot \,\vert \,\cdot \,\rangle$, the (hermitian) scalar product of a hilbertian space H; the notation $\langle \,\cdot \,,\,\cdot \,\rangle$ will be reserved to bilinear pairings only.

Let G be a homogeneous group, and $\pi :G\rightarrow {\mathcal {L}}(H)$ a unitary representation of G in a (complex) hilbertian space H. Then, $C^\infty (\pi )$ denotes the space of smooth vectors of the representation $\pi$, that is, the space of $v\in H$ such that the mapping $x\mapsto \pi (x) v$ is of class $C^\infty$.

If X is a left-invariant differential operator on G, then $\text {d}\pi (X)$ denotes the derived representation of X, that is, the unique linear mapping $C^\infty (\pi )\rightarrow H$ such that $\text {d}\pi (X) v=X(x\mapsto \pi (x)v)(e)$ for every $v\in C^\infty (\pi )$.

For every irreducible unitary representation $\pi$ of G, we denote by $[\pi ]$ its equivalence class with respect to unitary equivalence. The dual ${\widehat{G}}$ of G is the set of such equivalence classes.⁴

The dual of a homogeneous group can be described by means of Kirillov’s theory; see, for example, [21, 45]. The same references also provide the existence of a Plancherel measure on ${{\widehat{G}}}$ and a corresponding Plancherel formula. We shall describe the Plancherel measure more precisely for 2-step stratified groups in Sect. 7.

2.3 Measures

We shall now recall some general notation concerning measures. First of all, all measures are supposed to be Radon, with the exception of the Hausdorff measures ${\mathcal {H}}^k$ on $\mathbb {R}^n$, for $k<n$, and of the Plancherel measures on the dual of a homogeneous group (cf. Sect. 2.2). Recall that, denoting by $\text {diam}(A)$ the diameter of a set A,

$$\begin{aligned} {\mathcal {H}}^k(E)=\frac{\pi ^{\frac{k}{2}}}{ 2^k \Gamma \left( \frac{k}{2}+1 \right) }\sup _{\delta >0} \inf \left\{ \sum _{j\in \mathbb {N}} \text {diam}(E_j)^k:E\subseteq \bigcup _{j\in \mathbb {N}} E_j, \;\forall j\in \mathbb {N}\quad \text {diam}(E_j)<\delta \right\} \end{aligned}$$

for every $E\subseteq \mathbb {R}^n$ (cf., for instance, [2, Definition 2.46]).

For the sake of simplicity, we only deal with Radon measures on Polish spaces, that is, topological spaces with a countable base whose topology is induced by a complete metric. For example, every locally compact space with a countable base is a Polish space, but we shall need to deal with some Polish spaces which are not locally compact (cf. the proof of Theorem 8.5). If X is a Polish space, then a positive Borel measure $\mu$ on X is a Radon measure if and only if it is locally finite (cf. [15, Theorem 2 and Proposition 3 of Chapter IX, Sect. 3]).

If X and Y are Polish spaces, $\mu$ is a positive Radon measure on X, and $\pi :X\rightarrow Y$ is a $\mu$-measurable mapping, then $\pi$ is called $\mu$-proper if $\pi _*(\mu )$ is a Radon measure. Observe that, if $\pi$ is proper, then it is $\mu$-proper for every Radon measure $\mu$ (cf. [15, Remark 2 of Chapter IX, Sect. 2, No. 3]).

Let X and Y be Polish spaces, $\mu$ a positive Radon measure on X, and $\pi :X\rightarrow Y$ a $\mu$-proper mapping; define $\nu {:}{=}\pi _*(\mu )$. Then, a disintegration of $\mu$ relative to $\pi$ is a family $(\mu _y)_{y\in Y}$ of positive Radon measures on X such that the following hold:

• $X\setminus \pi ^{-1}(y)$ is $\mu _y$-negligible for $\nu$-almost every $y\in Y$;

• if $f:X\rightarrow [0,\infty ]$ is $\mu$-measurable, then f is $\mu _y$-measurable for $\nu$-almost every $y\in Y$, the mapping $y\mapsto \int _X f\,\text {d}\mu _y$ is $\nu$-measurable, and

  $$\begin{aligned} \int _X f\,\text {d}\mu =\int _Y \int _X f(x)\,\text {d}\mu _y(x)\,\text {d}\nu (y); \end{aligned}$$

• $\mu _y(X)=1$ for $\nu$-almost every $y\in Y$.

Notice that, under the stated assumptions, a disintegration of $\mu$ relative to $\pi$ always exists, and is unique up to modifications on a $\nu$-negligible subset of Y (cf. [15, Proposition 13 of Chapter IX, Sect. 2, No. 7]).

If $\mu$ is a Radon measure on a Polish space X, and $f\in L^1_\text {loc}(\mu )$, then we shall denote by $f\cdot \mu$ the Radon measure $E\mapsto \int _E f\,\text {d}\mu$. We say that two positive Radon measures on a Polish space are equivalent if they share the same negligible sets; in other words, if they are absolutely continuous with respect to one another.

If X is a locally compact space, then we denote by $C_0(X)$ the space of complex-valued continuous functions on X which vanish at the point at infinity, endowed with the maximum norm. We denote by ${\mathcal {M}}^1(X)$ the dual of $C_0(X)$, that is, the space of bounded (Radon) measures on X.

2.4 Distributions

We shall generally adopt Schwartz’s notation for the spaces of distributions. So, ${\mathcal {D}}'$ will denote the space of distributions and ${\mathcal {E}}'$ the space of distributions with compact support. In addition, we shall denote by ${\mathcal {E}}$ the space of functions of class $C^\infty$ endowed with the topology of locally uniform convergence of all derivatives. When no reference to the topological structure of ${\mathcal {E}}$ is needed, we shall prefer the symbol $C^\infty$. Analogously, we shall denote by $C^\infty _c$ the space of compactly supported functions of class $C^\infty$, endowed with no topological structures.

If $\pi :\mathbb {R}^n\rightarrow \mathbb {R}^m$ is a mapping of class $C^\infty$, $T\in {\mathcal {D}}'(\mathbb {R}^n)$ and $\pi$ is proper on $\text {Supp}\left( T\right)$, then we shall denote by $\pi _*(T)$ the push-forward of T, defined by $\langle \pi _*(T),\varphi \rangle =\langle T, \varphi \circ \pi \rangle$ for every $\varphi \in C^\infty (\mathbb {R}^m)$. Notice that the present notation agrees with the the one defined in Sect. 2.3 when T is a measure.

2.5 Topological Tensor Products

In Sect. 4 we shall make use of the theory of tensor products of locally convex spaces, as well as of the theory of nuclear spaces. We refer the reader to [35, 67] for a general treatment of these topics; here we shall only recall some basic facts and notation.

Definition 2.2

Let E and F be two locally convex spaces (over $\mathbb {C}$). We endow the algebraic tensor product $E\otimes F$ (over $\mathbb {C}$) with the finest locally convex topology for which the canonical mapping $E\times F\rightarrow E\otimes F$ is continuous. We denote by $E{\widehat{\otimes }} F$ the Hausdorff completion of $E\otimes F$.

Sometimes the locally convex space $E\otimes F$ defined above is denoted $E\otimes _\pi F$ and is called the projective tensor product; since we shall not use other topologies on the vector space $E\otimes F$, following [35] we simply write $E\otimes F$ instead of $E\otimes _\pi F$.

If $E_1,E_2,F_1,F_2$ are four Hausdorff locally convex spaces, and $T_1:E_1\rightarrow F_1$ and $T_2:E_2\rightarrow F_2$ are two continuous linear mappings, then $T_1\otimes T_2:E_1\otimes E_2\rightarrow F_1\otimes F_2$ is a continuous linear mapping and we denote by $T_1{\widehat{\otimes }} T_2:E_1{\widehat{\otimes }} E_2\rightarrow F_1{\widehat{\otimes }} F_2$ its canonical extension.

Even though the tensor product defined above has several important properties and interacts nicely with products and strict morphisms (homomorphisms, in the terminology of [67]), it is not compatible with isomorphisms onto subspaces (cf. [67, Exercise 43.8, Proposition 43.9, and Remark 43.2]). This problem does not occur if one of the two factors of both tensor products is nuclear (cf. [67, Proposition 43.7 and Theorem 50.1]).

Definition 2.3

Let E be a Hausdorff locally convex space. Then E is said to be nuclear if, for every Hausdorff locally convex space F, the space $E\otimes F$ carries the topology of uniform convergence on the sets of the form $B_E\times B_F$, where $B_E$ and $B_F$ are equicontinuous subsets of $E'$ and $F'$, respectively, under the standard duality between $E\otimes F$ and $E'\times F'$.

Nuclear spaces enjoy several useful properties. For example, their bounded subsets are precompact, so that every (quasi-)complete nuclear space is semi-reflexive (cf. [67, Proposition 50.2]). In addition, subspaces, quotients (modulo closed subspaces), products, countable inductive limits, and tensor products of nuclear spaces are nuclear (cf. [67, Proposition 50.1]).

Several spaces of distributions and test functions are nuclear: for example, ${\mathcal {E}}$, ${\mathcal {E}}'$, ${\mathcal {S}}$, ${\mathcal {S}}'$, and ${\mathcal {D}}'$ (as well as its dual ${\mathcal {D}}$, which is $C^\infty _c$ with a suitable topology); see [67, Corollary to Theorem 51.5].

We end this section with a definition and a few technical results which will be used in Sect. 4.

Definition 2.4

Take $n\in \mathbb {N}^*$ and let F be a Fréchet space. Define ${\mathcal {S}}(\mathbb {R}^n;F)$ as the subset of $\varphi \in C^\infty (\mathbb {R}^n;F)$ where the semi-norms

$$\begin{aligned} \varphi \mapsto \sup _{x\in \mathbb {R}^n} (1+|x |)^k\Vert \partial ^\gamma \varphi \Vert _\rho \end{aligned}$$

are finite for every $k\in \mathbb {N}$, for every $\gamma \in \mathbb {N}^n$, and for every continuous semi-norm $\rho$ on F; endow ${\mathcal {S}}(\mathbb {R}^n;F)$ with the corresponding topology.

Let C be a closed subset of $\mathbb {R}^n$, and let $N_{\mathbb {R}^n,C,F}$ be the set of $\varphi \in {\mathcal {S}}(\mathbb {R}^n;F)$ which vanish on C. Then, we define ${\mathcal {S}}_{\mathbb {R}^n}(C;F){:}{=}{\mathcal {S}}(\mathbb {R}^n;F)/N_{\mathbb {R}^n,C,F}$; we shall omit to denote $\mathbb {R}^n$ when it is clear from the context. We shall simply write ${\mathcal {S}}_{\mathbb {R}^n}(C)$ or ${\mathcal {S}}(C)$ instead of ${\mathcal {S}}_{\mathbb {R}^n}(C;\mathbb {C})$.

Proposition 2.5

Let F be a Fréchet space over $\mathbb {C}$, and take $n\in \mathbb {N}^*$.⁵ Then, the bilinear mapping ${\mathcal {S}}(\mathbb {R}^n)\times F\ni (\varphi ,v)\mapsto [h\mapsto \varphi (h) v]\in {\mathcal {S}}(\mathbb {R}^n;F)$ induces an isomorphism

$$\begin{aligned} {\mathcal {S}}(\mathbb {R}^n){\widehat{\otimes }} F\rightarrow {\mathcal {S}}(\mathbb {R}^n;F). \end{aligned}$$

The proof is similar to that of [67, Theorem 51.6] and is omitted.

Proposition 2.6

Take $n_1,n_2\in \mathbb {N}^*$, and let $C_1,C_2$ be two closed subspaces of $\mathbb {R}^{n_1},\mathbb {R}^{n_2}$, respectively. Then, ${\mathcal {S}}_{\mathbb {R}^{n_1}\times \mathbb {R}^{n_2}}(C_1\times C_2)$ is canonically isomorphic to ${\mathcal {S}}_{\mathbb {R}^{n_1}}(C_1){\widehat{\otimes }} {\mathcal {S}}_{\mathbb {R}^{n_2}}(C_2)$.

Proof

Define

$$\begin{aligned} \Psi _{n,C}:{\mathcal {S}}(\mathbb {R}^n)\ni \varphi \mapsto (\varphi (x))_{x\in C}\in \mathbb {C}^C \end{aligned}$$

for every $n\in \mathbb {N}^*$ and for every closed subspace C of $\mathbb {R}^n$. Then, with the notation of Definition 2.4, $N_{\mathbb {R}^n,C,\mathbb {C}}$ is the kernel of $\Psi _{n,C}$. Throughout the proof, we shall identify ${\mathcal {S}}(\mathbb {R}^{n_1}\times \mathbb {R}^{n_2})$ and ${\mathcal {S}}(\mathbb {R}^{n_1}){\widehat{\otimes }} {\mathcal {S}}(\mathbb {R}^{n_2})$ by means of the canonical isomorphism (cf. [67, Theorem 51.6]). Then, $\Psi _{n_1+n_2, C_1\times C_2}=\Psi _{n_1,C_1}{\widehat{\otimes }} \Psi _{n_2,C_2}$ (cf. [35, Proposition 6 of Chapter I, Sect. 1, No. 3]). Therefore, [35, Proposition 3 of Chapter I, Sect. 1, No. 2] implies that $N_{\mathbb {R}^{n_1}\times \mathbb {R}^{n_2}, C_1\times C_2, \mathbb {C}}$ is the closed vector subspace of ${\mathcal {S}}(\mathbb {R}^{n_1}){\widehat{\otimes }} {\mathcal {S}}(\mathbb {R}^{n_2})$ generated by the tensors of the form $\varphi _1\otimes \varphi _2$, with $\Psi _{{n_1},C_1}(\varphi _1)=0$ or $\Psi _{{n_2},C_2}(\varphi _2)=0$. By the same reference, we see that $N_{\mathbb {R}^{n_1}\times \mathbb {R}^{n_2}, C_1\times C_2, \mathbb {C}}$ is the kernel of the canonical projection ${\mathcal {S}}(\mathbb {R}^{n_1}){\widehat{\otimes }} {\mathcal {S}}(\mathbb {R}^{n_2})\rightarrow {\mathcal {S}}_{\mathbb {R}^{n_1}}(C_1){\widehat{\otimes }} {\mathcal {S}}_{\mathbb {R}^{n_2}}(C_2)$. The assertion follows. $\square$

3 Rockland Families and the Kernel Transform

We begin this section with a brief description of weighted subcoercive systems of differential operators on general connected Lie groups; we shall then specialize our discussion to Rockland families on homogeneous groups.

In Sect. 3.1, Gwill denote a connected Lie group; in Sect. 3.2, Gwill denote a homogeneous group of homogeneous dimensionQ.

3.1 Weighted Subcoercive Systems

Weighted subcoercive operators were introduced by ter Elst and Robinson in [65]. Roughly speaking, they may be considered as generalizations of (positive) Rockland operators to a non-homogeneous setting and as analogues of higher-order elliptic operators in a (weighted) sub-Riemannian setting. Weighted subcoercive operators enjoy several important properties, such as:

• if ${\mathcal {L}}$ is a weighted subcoercive operator and X is a differential operator of strictly lower (weighted) order, then ${\mathcal {L}}+X$ is (weighted subcoercive and) hypoelliptic;

• if ${\mathcal {L}}$ is a weighted subcoercive operator and X is a differential operator of lower (weighted) order, then there is a constant $C>0$ such that $\Vert X f \Vert _2\leqslant C(\Vert f \Vert _2+\Vert {\mathcal {L}}f \Vert _2)$ for every $f\in C^\infty _c(G)$ (where $L^2(G)$ is relative to a right Haar measure on G);

• if ${\mathcal {L}}$ is a weighted subcoercive operator, then it generates a holomorphic semigroup of operators on $L^p(G)$ whose kernel is independent of $p\in [1,\infty ]$ and satisfies ‘Gaussian’ bounds.

All the preceding properties are modelled on analogous properties of (positive) Rockland operators; see [6, 38, 65] for a precise statement and the proof of the preceding assertions. Actually, the very notion of a weighted subcoercive operator is modelled, by a contraction argument, on that of a Rockland operator. With the purpose of underlining this fact, we are now going to define weighted subcoercive operators in a way which is slightly different (but equivalent) to the one chosen in [65]. Before passing to that, let us note that weighted sub-coercive operators lack the properties of Rockland operators which are deeply related to homogeneity, such as the fact that homogeneous functions of the given operator are homogeneous, which holds for positive Rockland operators but may not even make sense in a general context. In addition, homogeneity arguments which refer to large radii may fail for general weighted subcoercive operators, for example when studying necessary conditions for the $L^p-L^q$ boundedness of Riesz potentials (when defined); on the contrary, the homogeneity arguments which refer to small radii might be extended in a suitable way, due to the ‘locality’ of the contraction procedure used to define weighted subcoercive operators. Further, arguments which require the use of dyadic decompositions may not be easily extended to weighted subcoercive operators without further assumptions; for example, to prove multiplier theorems such as [49, Theorem 4.6], ‘local’ estimates might no longer be sufficient, since they may not be extended by homogeneity.

Let G be a Lie group and let ${\mathfrak {g}}$ be its Lie algebra. Let $({\mathfrak {g}}_\lambda )_{\lambda \geqslant 0}$ be an increasing, right-continuous, separated, and exhaustive filtration of ${\mathfrak {g}}$; in other words, $({\mathfrak {g}}_\lambda )$ is an increasing sequence of vector subspaces of ${\mathfrak {g}}$ such that $\bigcap _{\mu >\lambda } {\mathfrak {g}}_\mu ={\mathfrak {g}}_\lambda$ for every $\lambda \geqslant 0$, ${\mathfrak {g}}_0=0$, $\bigcup _{\lambda \geqslant 0} {\mathfrak {g}}_\lambda ={\mathfrak {g}}$, and $[{\mathfrak {g}}_{\lambda _1},{\mathfrak {g}}_{\lambda _2}]\subseteq {\mathfrak {g}}_{\lambda _1+\lambda _2}$ for every $\lambda _1,\lambda _2\geqslant 0$. For every $X\in {\mathfrak {g}}$, define $\deg X{:}{=}\min \lbrace \lambda \geqslant 0:X\in {\mathfrak {g}}_\lambda \rbrace$. Define, for every $\lambda >0$, ${\mathfrak {g}}_{\lambda ^-}{:}{=}\bigcup _{\mu <\lambda } {\mathfrak {g}}_\mu$ and ${\mathfrak {g}}_{*,\lambda }{:}{=}{\mathfrak {g}}_\lambda /{\mathfrak {g}}_{\lambda ^-}$, and set

$$\begin{aligned} {\mathfrak {g}}_*{:}{=}\bigoplus _{\lambda >0}{\mathfrak {g}}_{*,\lambda }. \end{aligned}$$

Notice that $[\,\cdot \,,\,\cdot \,]$ induces bilinear mappings ${\mathfrak {g}}_{*,\lambda _1}\times {\mathfrak {g}}_{*,\lambda _2}\rightarrow {\mathfrak {g}}_{*,\lambda _1+\lambda _2}$ for every $\lambda _1,\lambda _2>0$, which then define a Lie algebra structure on ${\mathfrak {g}}_*$. More precisely, ${\mathfrak {g}}_*$ becomes a homogeneous Lie algebra with dilations defined by $r\cdot X{:}{=}r^\lambda X$ for every $X\in {\mathfrak {g}}_{*,\lambda }$ and for every $\lambda >0$. Denote by $G_*$ the corresponding homogeneous group; $G_*$ is then the (local) contraction of G.

Further, let $U({\mathfrak {g}})$ be the enveloping algebra of ${\mathfrak {g}}$, and define $U_\lambda$ as the vector space generated by the products (as left-invariant differential operators on G) of the form $X_1\cdots X_k$, for $k\geqslant 0$, $X_1,\dots , X_k\in {\mathfrak {g}}$ and $\deg X_1+\cdots + \deg X_k\leqslant \lambda$. Then, $(U_\lambda )$ is an increasing, right-continuous, and exhaustive filtration of $U({\mathfrak {g}})$ with $U_0=\mathbb {R}I$, where I denotes the identity operator. For every $X\in U({\mathfrak {g}})$, define $\deg X{:}{=}\min \lbrace \lambda \geqslant 0:X\in U_\lambda \rbrace$. Define $U_{\lambda ^-}{:}{=}\bigcup _{\mu <\lambda } U_\mu$ and $U_{*,\lambda }{:}{=}U_\lambda /U_{\lambda ^-}$ for every $\lambda >0$ (while $U_{*,0}{:}{=}U_0$), and set

$$\begin{aligned} U_*{:}{=}\bigoplus _{\lambda \geqslant 0} U_{*,\lambda }. \end{aligned}$$

As for ${\mathfrak {g}}_*$, we endow $U_*$ with an algebra structure, observing that the product of $U({\mathfrak {g}})$ induces bilinear mappings $U_{*,\lambda _1}\times U_{*,\lambda _2}\rightarrow U_{*,\lambda _1+\lambda _2}$ for every $\lambda _1,\lambda _2\geqslant 0$.

We leave to the reader the proof of the following simple result, which is basically a consequence of the Poincaré–Birkhoff–Witt theorem (cf. [17, Chapter I, Sect. 2, No. 7, Theorem 1]).

Proposition 3.1

The canonical inclusions ${\mathfrak {g}}_\lambda \subseteq U_\lambda$, $\lambda >0$, induce a linear mapping $\pi _* :{\mathfrak {g}}_*\rightarrow U_*$ such that $\pi _*([X,Y])=[\pi _*(X),\pi _*(Y)]$ for every $X,Y\in {\mathfrak {g}}_*$. The canonical extension $U(\pi _*):U({\mathfrak {g}}_*)\rightarrow U_*$ of $\pi _*$ is an isomorphism of algebras.

Definition 3.2

A left-invariant differential operator ${\mathcal {L}}$ on G is said to be weighted subcoercive (with respect to the filtration $({\mathfrak {g}}_\lambda )$) if the element ${\mathcal {L}}+{\mathcal {L}}^*+U_{(\deg {\mathcal {L}})^-}$ of $U_*$ induces a positive Rockland operator on $G_*$ through $U(\pi _*)$, with the notation of Proposition 3.1.

In the preceding definition, ${\mathcal {L}}^*$ denotes the formal adjoint of ${\mathcal {L}}$ with respect to a right Haar measure on G. Notice that, if we replace ${\mathcal {L}}^*$ with the formal adjoint of ${\mathcal {L}}$ with respect to another (non-zero) relatively invariant measure on G (for instance, a left Haar measure), then ${\mathcal {L}}+{\mathcal {L}}^*+U_{(\deg {\mathcal {L}})^-}$ is unchanged. In addition, observe that, if ${\mathcal {L}}$ is weighted subcoercive, then $\deg {\mathcal {L}}\in 2 \lambda \mathbb {N}^*$ for every $\lambda >0$ such that ${\mathfrak {g}}_{*,\lambda }\not \subseteq [{\mathfrak {g}}_*,{\mathfrak {g}}_*]$.

We may now recall the definition of a weighted subcoercive system of differential operators (cf. [48]).

Definition 3.3

A weighted subcoercive system on G (relative to the filtration $({\mathfrak {g}}_\lambda )$) is a finite family ${\mathcal {L}}_1,\dots , {\mathcal {L}}_k$ of formally self-adjoint, commuting, left-invariant differential operators on G such that the differential operator $P({\mathcal {L}}_1,\dots ,{\mathcal {L}}_k)$ is weighted subcoercive for some (real) polynomial P in k indeterminates.

3.2 Rockland Families

We recall that, from now until the end of this section, Gdenotes a homogeneous group of homogeneous dimension Q.

Definition 3.4

Let ${\mathcal {L}}_A=({\mathcal {L}}_\alpha )_{\alpha \in A}$ be a family of differential operators on G. We say that ${\mathcal {L}}_A$ is jointly hypoelliptic if the following hold: if V is an open subset of G and T is a distribution on V such that ${\mathcal {L}}_\alpha T\in C^\infty (V)$ for every $\alpha \in A$, then $T\in C^\infty (V)$.

The following result enriches [47, Proposition 3.6.3].

Theorem 3.5

Let ${\mathcal {L}}_A=({\mathcal {L}}_\alpha )_{\alpha \in A}$ be a non-empty commutative finite family of formally self-adjoint, homogeneous, left-invariant differential operators which annihilate constants on G. Then, the following conditions are equivalent:

• 1. ${\mathcal {L}}_A$ is jointly hypoelliptic;

• 2. for every continuous non-trivial irreducible unitary representation $\pi$ of G in a hilbertian space H, the family of operators $\text {d}\pi ({\mathcal {L}}_A)$ is jointly injective on $C^\infty (\pi )$;

• 3. the algebra generated by ${\mathcal {L}}_A$ contains a positive Rockland operator, possibly with respect to a different family of dilations on G with respect to which the ${\mathcal {L}}_\alpha$ are still homogeneous.

• 4. the ${\mathcal {L}}_\alpha$ are essentially self-adjoint on $C^\infty _c(G)$, their self-adjoint extensions commute, and for every $m\in {\mathcal {S}}(\mathbb {R}^A)$ the convolution kernel of the operator $m({\mathcal {L}}_A)$ belongs to ${\mathcal {S}}(G)$;

• 5. ${\mathcal {L}}_A$ is a weighted subcoercive system.

Proof

1 $\implies$2 This is a simple adaptation of the proof of [7, Theorem 1].

2 $\implies$3 This is the implication $(ii) \implies (i)$ of [47, Proposition 3.6.3].

3 $\implies$1 Take an open subset V of G and $T\in {\mathcal {D}}'(V)$ such that ${\mathcal {L}}_\alpha T\in C^\infty (V)$ for every $\alpha \in A$. Take $P\in \mathbb {C}[A]$ such that $P(0)=0$ and $P({\mathcal {L}}_A)$ is hypoelliptic. Then, $P({\mathcal {L}}_A)T\in C^\infty (V)$, so that $T\in C^\infty (V)$.

3 $\implies$4 This follows from [47, Propositions 1.4.4, 3.1.2, and 4.2.1].

3 $\iff$5 This follows from [47, Proposition 3.6.3].

4 $\implies$3 Notice first that, by [55, Proposition 1.1], we may reduce to the case in which G is graded. Hence, there is a positive homogeneous proper polynomial P on $\mathbb {R}^A$, e.g. a suitable sum of even powers of coordinate functions. Now, take $t\geqslant 0$ and let $p_t$ be the convolution kernel of the operator $e^{- t P({\mathcal {L}}_A)}$, so that $p_t\in {\mathcal {S}}(G)$ for $t>0$, while $p_0=\delta _e$. In addition, denoting by d the degree of $P({\mathcal {L}}_A)$,

$$\begin{aligned} p_t(g)=t^{-\frac{Q}{d}} p_1\left( t^{-\frac{1}{d}}\cdot g \right) \end{aligned}$$

for every $t>0$ and for every $g\in G$. Arguing as in the proof of [33, Proposition 1.68], we see that, if we define

$$\begin{aligned} p(t,g){:}{=}{\left\{ \begin{array}{ll} p_t(g) &{} \text {if }t>0\\ 0 &{} \text {if }t\leqslant 0 \end{array}\right. } \end{aligned}$$

for $(t,g)\in \mathbb {R}\times G$, then p is a fundamental solution of $\partial _t-P({\mathcal {L}}_A)$ and is of class $C^\infty$ on $(\mathbb {R}\times G)\setminus \lbrace (0,e) \rbrace$. Consequently, $p^*$ is a fundamental solution of the right-invariant differential operator associated with $\partial _t-P({\mathcal {L}}_A)$. Arguing as in the proof of [68, Theorem 2.1], we see that $\partial _t-P({\mathcal {L}}_A)$ is hypoelliptic, so that also $P({\mathcal {L}}_A)$ is hypoelliptic. $\square$

Definition 3.6

A Rockland family is a non-empty finite commuting family of homogeneous left-invariant differential operators annihilating constants which satisfies the equivalent conditions of Theorem 3.5.

Here we depart slightly from the notion of ‘Rockland system’ as defined in [47]. Indeed, a Rockland system is a Rockland family, while a Rockland family need not be a Rockland system, since the algebra it generates need not contain a Rockland operator. Nevertheless, the difference is only illusory: as Theorem 3.5 shows, given a Rockland family ${\mathcal {L}}_A$, one may change the dilations of G in such a way that ${\mathcal {L}}_A$ becomes a Rockland system. In other words, up to a change of dilations, there is no difference between Rockland families and Rockland systems.

Notice that, as a consequence of the results of Sect. 5, the properties we are going to investigate do not pertain to the chosen family ${\mathcal {L}}_A$, but actually to the (non-unital) algebra it generates. As a matter of fact, we can start with a commutative, finitely generated, formally self-adjoint and dilation-invariant sub-algebra of the complexification $U_\mathbb {C}({\mathfrak {g}})$ of the universal enveloping algebra of ${\mathfrak {g}}$, and require that it contains a hypoelliptic operator which annihilate constants. It is not hard to see that such algebras are generated by a Rockland family (use [60] to prove that dilation-invariant sub-algebras are graded, that is, generated by homogeneous elements), and that different Rockland families which generate the same algebra are equivalent in a natural sense.

Definition 3.7

Let ${\mathcal {L}}_A$ be a Rockland family. Then, we denote by $\mu _{{\mathcal {L}}_A}$ the spectral measure associated with the self-adjoint extensions of the ${\mathcal {L}}_\alpha$ and by $\sigma ({\mathcal {L}}_A)$ its support, that is, the joint spectrum of ${\mathcal {L}}_A$.

We say that a $\mu _{{\mathcal {L}}_A}$-measurable function $m:\sigma ({\mathcal {L}}_A)\rightarrow \mathbb {C}$ (or $m:\mathbb {R}^A\rightarrow \mathbb {C}$) admits a kernel if $C^\infty _c(G)$ is contained in the domain of $m({\mathcal {L}}_A)$. In this case, ${\mathcal {S}}(G)$ is contained in the domain of $m({\mathcal {L}}_A)$ and there is a unique $K\in {\mathcal {S}}'(G)$ such that $m({\mathcal {L}}_A)\varphi = \varphi * K$ for every $\varphi \in {\mathcal {S}}(G)$ (cf. [24, Theorem 7.2]); we shall denote K by ${\mathcal {K}}_{{\mathcal {L}}_A}(m)$.

We shall often need some dilations on $\sigma ({\mathcal {L}}_A)$ which reflect the homogeneity of the ${\mathcal {L}}_\alpha$. This leads to the following definition.

Definition 3.8

Let ${\mathcal {L}}_A$ be a Rockland family. For every $r>0$, denote by $\lambda \mapsto r \cdot \lambda$ the unique bijection of $\sigma ({\mathcal {L}}_A)$ onto itself such that ${\mathcal {K}}_{{\mathcal {L}}_A}(\text {pr}_\alpha (r\,\cdot \,))=(r\,\cdot \,)_*{\mathcal {L}}_\alpha$ for every $\alpha \in A$, where $\text {pr}_\alpha :\mathbb {R}^A\rightarrow \mathbb {R}$ is the projection onto the $\alpha$-th component. In other words, $r \cdot (\lambda _\alpha )=(r^{\delta _\alpha }\lambda _\alpha )$ for every $(\lambda _\alpha )\in \sigma ({\mathcal {L}}_A)$, assuming that ${\mathcal {L}}_\alpha$ is homogeneous of degree $\delta _\alpha$ for every $\alpha \in A$.

We denote by $|\,\cdot \, |$ a proper positive function on $\sigma ({\mathcal {L}}_A)$ which is homogeneous of degree 1 with respect to these dilations.⁶

Let us now introduce our main objects of study.

Definition 3.9

Let ${\mathcal {L}}_A$ be a Rockland family. We say that ${\mathcal {L}}_A$ satisfies property:

(RL):

(‘Riemann–Lebesgue’) if every $m\in L^\infty (\mu _{{\mathcal {L}}_A})$ such that ${\mathcal {K}}_{{\mathcal {L}}_A}(m)\in L^1(G)$ has a continuous representative;

(S):

(‘Schwartz’) if every $m\in L^\infty (\mu _{{\mathcal {L}}_A})$ such that ${\mathcal {K}}_{{\mathcal {L}}_A}(m)\in {\mathcal {S}}(G)$ has a representative in ${\mathcal {S}}(\sigma ({\mathcal {L}}_A))$ (cf. Definition 2.4).

Remark 3.10

Observe that we do not require that m should have a representative in $C_0(\sigma ({\mathcal {L}}_A))$ in the definition of property (RL). In fact, the general theory shows that m vanishes at the point at infinity (cf. [48, Proposition 3.14]).

Thanks to Theorem 3.5, we may take advantage of the study of weighted subcoercive systems pursued in [47, 48]. For the ease of the reader, we shall collect below some basic results from [47, 48, 66] which will be used repeatedly in the following sections. We shall briefly indicate how to extend such results to the present setting for completeness.

Lemma 3.11

Let m be a function on $\sigma ({\mathcal {L}}_A)$ which admits a kernel in the sense of Definition 3.7. Then, for every $r>0$,

$$\begin{aligned} {\mathcal {K}}_{{\mathcal {L}}_A}({\overline{m}})={\mathcal {K}}_{{\mathcal {L}}_A}(m)^* \qquad \text {and} \qquad {\mathcal {K}}_{{\mathcal {L}}_A}(m(r\,\cdot \,))=(r\,\cdot \,)_*{\mathcal {K}}_{{\mathcal {L}}_A}(m) \end{aligned}$$

If, in addition, $m'$ is a bounded function on $\sigma ({\mathcal {L}}_A)$ and ${\mathcal {K}}_{{\mathcal {L}}_A}(m')\in {\mathcal {S}}(G)$, then

$$\begin{aligned} {\mathcal {K}}_{{\mathcal {L}}_A}(m m')={\mathcal {K}}_{{\mathcal {L}}_A}(m)*{\mathcal {K}}_{{\mathcal {L}}_A}(m')={\mathcal {K}}_{{\mathcal {L}}_A}(m')*{\mathcal {K}}_{{\mathcal {L}}_A}(m). \end{aligned}$$

Proof

Apply [48, Lemma 3.4 and Proposition 5.1] to $m\chi _{S_j}$ and $m'$, where $S_j=\lbrace \lambda \in \sigma ({\mathcal {L}}_A):|m(\lambda ) |\leqslant j \rbrace$, and pass to the limit. $\square$

Proposition 3.12

Let $\pi$ be a homogeneous homomorphism of Gonto a homogeneous group $G'$. Then, the following hold:

• 1. $\text {d}\pi ({\mathcal {L}}_A)=(\text {d}\pi ({\mathcal {L}}_\alpha ))_{\alpha \in A}$ is a Rockland family on $G'$;

• 2. $\sigma (\text {d}\pi ({\mathcal {L}}_A))\subseteq \sigma ({\mathcal {L}}_A)$;

• 3. if $m:\mathbb {R}^A\rightarrow \mathbb {C}$ is $\mu _{{\mathcal {L}}_A}$-measurable and continuous on an open set which carries $\mu _{\text {d}\pi ({\mathcal {L}}_A)}$, and if ${\mathcal {K}}_{{\mathcal {L}}_A}(m)\in {\mathcal {M}}^1(G)+{\mathcal {E}}'(G)$, then

  $$\begin{aligned} \pi _*({\mathcal {K}}_{{\mathcal {L}}_A}(m))= {\mathcal {K}}_{\text {d}\pi ({\mathcal {L}}_A)}(m). \end{aligned}$$

Proof

1 :

The fact that $\text {d}\pi ({\mathcal {L}}_A)$ is Rockland follows from the fact that, if ${\widetilde{\pi }}$ is a continuous unitary representation of $G'$, then ${\widetilde{\pi }} \circ \pi$ is a continuous unitary representation of G, with $C^\infty ({\widetilde{\pi }})=C^\infty ({\widetilde{\pi }}\circ \pi )$ since $\pi$ is a submersion, and $\text {d}{\widetilde{\pi }}(\text {d}\pi ({\mathcal {L}}_A))= \text {d}({\widetilde{\pi }} \circ \pi )({\mathcal {L}}_A)$; finally, ${\widetilde{\pi }}\circ \pi$ is irreducible or trivial if and only if ${\widetilde{\pi }}$ is irreducible or trivial, respectively.

3 :

Let ${\widetilde{\pi }}$ be the right quasi-regular representation of G in $L^2(G')$, that is, $({\widetilde{\pi }}(g) f)(g')=f(g'\pi (g))$ for every $g\in G$, for every $f\in L^2(G')$, and for almost every $g'\in G'$. Then [48, Proposition 3.7], applied to ${\widetilde{\pi }}$, implies that our assertion holds if $m\in C_0(\sigma ({\mathcal {L}}_A))$ and ${\mathcal {K}}_{{\mathcal {L}}_A}(m)\in L^1(G)$. The general case follows by approximation.

2 :

This follows easily from 3.$\square$

The following definition will shorten the notation in the sequel.

Definition 3.13

Let F be a subspace of ${\mathcal {D}}'(G)$. We denote by $F_{{\mathcal {L}}_A}$ the set of ${\mathcal {K}}_{{\mathcal {L}}_A}(m)$ as m runs through the set of $\mu _{{\mathcal {L}}_A}$-measurable functions which admit a kernel in F.

Proposition 3.14

Let F be a Fréchet space which is continuously embedded in ${\mathcal {M}}^1(G)$. Then, $F_{{\mathcal {L}}_A}$ is closed in F.

In particular, this applies to $L^1(G)$ and ${\mathcal {S}}(G)$.

Proof

Let $(m_j)$ be a sequence in $L^\infty (\mu _{{\mathcal {L}}_A})$ such that the sequence $({\mathcal {K}}_{{\mathcal {L}}_A}(m_j))$ converges to some f in F. Then, $(m_j({\mathcal {L}}_A))$ is a Cauchy sequence in ${\mathcal {L}}(L^2(G))$, so that $(m_j)$ converges to some m in $L^\infty (\mu _{{\mathcal {L}}_A})$ by spectral theory, so that ${\mathcal {K}}_{{\mathcal {L}}_A}(m)=f$. $\square$

The following theorem is a particular case of [48, Lemma 3.9, Theorem 3.10, and Proposition 3.12].

Theorem 3.15

Let ${\mathcal {L}}_A$ be a Rockland family. Then, there is a unique positive Radon measure $\beta _{{\mathcal {L}}_A}$ on $\sigma ({\mathcal {L}}_A)$ such that the following hold:

• 1. $\mu _{{\mathcal {L}}_A}$ and $\beta _{{\mathcal {L}}_A}$ are equivalent;

• 2. ${\mathcal {K}}_{{\mathcal {L}}_A}$ induces an isometry of $L^2(\beta _{{\mathcal {L}}_A})$ onto $L^2_{{\mathcal {L}}_A}(G)$;

• 3. $(r\,\cdot \,)_*(\beta _{{\mathcal {L}}_A})=r^{-Q} \beta _{{\mathcal {L}}_A}$ for every $r>0$.

The following corollary has already been considered in [48, Proposition 3.14] for the case $p=1$. The general case follows by interpolation.

Corollary 3.16

Take $p\in [1,2]$. Then, ${\mathcal {K}}_{{\mathcal {L}}_A}$ induces a unique continuous linear mapping

$$\begin{aligned} {\mathcal {K}}_{{\mathcal {L}}_A,p}:L^p(\beta _{{\mathcal {L}}_A})\rightarrow L^{p'}(G). \end{aligned}$$

In addition, ${\mathcal {K}}_{{\mathcal {L}}_A,1}$ maps $L^1(\beta _{{\mathcal {L}}_A})$ into $C_0(G)$, has norm 1, and induces an isometry from the set of positive $\beta _{{\mathcal {L}}_A}$-integrable functions into the set of continuous functions of positive type on G.

From the preceding corollary we deduce the existence of an ‘integral kernel’ $\chi _{{\mathcal {L}}_A}$ for the ‘kernel transform’ ${\mathcal {K}}_{{\mathcal {L}}_A}$. This integral kernel was introduced in [66, Theorem 2.11] for a sub-Laplacian on a group of polynomial growth, but can be defined in greater generality.

Proposition 3.17

There is a unique $\chi _{{\mathcal {L}}_A}\in L^\infty (\beta _{{\mathcal {L}}_A}\otimes \nu _G)$ such that

$$\begin{aligned} {\mathcal {K}}_{{\mathcal {L}}_A}(m)(g)=\int _{\sigma ({\mathcal {L}}_A)} m(\lambda ) \chi _{{\mathcal {L}}_A}(\lambda ,g) \,\text {d}\beta _{{\mathcal {L}}_A}(\lambda ) \end{aligned}$$

for $\nu _G$-almost every $g\in G$. In addition, $\Vert \chi _{{\mathcal {L}}_A} \Vert _\infty =1$.

Proof

Apply the Dunford–Pettis theorem and Corollary 3.16 (cf. also [66, Theorem 2.11]). $\square$

We now pass to show some of the main properties of $\chi _{{\mathcal {L}}_A}$. In particular, we shall find some representatives of $\chi _{{\mathcal {L}}_A}$ which are particularly well-behaved. The following simple result generalizes [66, Theorem 2.33], and translates the second assertion of Lemma 3.11 in terms of $\chi _{{\mathcal {L}}_A}$; the proof is elementary and is omitted.

Proposition 3.18

For every $r>0$ and for $(\beta _{{\mathcal {L}}_A}\otimes \nu _G)$-almost every $(\lambda ,g)\in \sigma ({\mathcal {L}}_A)\times G$,

$$\begin{aligned} \chi _{{\mathcal {L}}_A}(r\cdot \lambda ,g)=\chi _{{\mathcal {L}}_A}(\lambda , r\cdot g). \end{aligned}$$

The following property is reminiscent of an analogous one concerning Gelfand pairs. It extends [66, Proposition 2.14] to our setting; since the original proof does not seem to extend to our situation, we shall present an alternative one.

Proposition 3.19

Take a $\beta _{{\mathcal {L}}_A}$-measurable function $m:\sigma ({\mathcal {L}}_A)\rightarrow \mathbb {C}$ which admits a kernel in ${\mathcal {M}}^1(G)+{\mathcal {E}}'(G)$. Then

$$\begin{aligned} {\mathcal {K}}_{{\mathcal {L}}_A}(m)*\chi _{{\mathcal {L}}_A}(\lambda ,\,\cdot \,)=\chi _{{\mathcal {L}}_A}(\lambda , \,\cdot \,)*{\mathcal {K}}_{{\mathcal {L}}_A}(m)= m(\lambda )\chi _{{\mathcal {L}}_A}(\lambda ,\,\cdot \,) \end{aligned}$$

for $\beta _{{\mathcal {L}}_A}$-almost every $\lambda \in \sigma ({\mathcal {L}}_A)$.

Proof

Notice first that, for every $\varphi _2\in C^\infty _c(G)$, the linear functional

$$\begin{aligned} L^\infty (G)\ni f \mapsto \langle {\mathcal {K}}_{{\mathcal {L}}_A}(m)* f, \varphi _2\rangle = \langle f, {\mathcal {K}}_{{\mathcal {L}}_A}(m)\check{\;} * \varphi _2\rangle \in \mathbb {C}\end{aligned}$$

is continuous with respect to the weak topology $\sigma (L^\infty (G),L^1(G))$. In addition, for every $\varphi _1\in C^\infty _c(\sigma ({\mathcal {L}}_A))$,

$$\begin{aligned} {\mathcal {K}}_{{\mathcal {L}}_A}(\varphi _1)= \int _{\sigma ({\mathcal {L}}_A)} \varphi _1(\lambda )\chi _{{\mathcal {L}}_A}(\lambda ,\,\cdot \,)\,\text {d}\beta _{{\mathcal {L}}_A}(\lambda ) \end{aligned}$$

in $L^\infty (G)$, endowed with the weak topology $\sigma (L^\infty (G),L^1(G))$. Therefore,

$$\begin{aligned} \begin{aligned}&\int _{\sigma ({\mathcal {L}}_A)} \langle {\mathcal {K}}_{{\mathcal {L}}_A}(m)*\chi _{{\mathcal {L}}_A}(\lambda ,\,\cdot \,), \varphi _2\rangle \, \varphi _1(\lambda )\,\text {d}\beta _{{\mathcal {L}}_A}(\lambda )= \langle {\mathcal {K}}_{{\mathcal {L}}_A}(m)* {\mathcal {K}}_{{\mathcal {L}}_A}(\varphi _1), \varphi _2\rangle \\&\quad =\langle {\mathcal {K}}_{{\mathcal {L}}_A}(m \varphi _1), \varphi _2\rangle \\&\quad =\int _{\sigma ({\mathcal {L}}_A)} (m \varphi _1)(\lambda ) \langle \chi _{{\mathcal {L}}_A}(\lambda ,\,\cdot \,), \varphi _2\rangle \,\text {d}\beta _{{\mathcal {L}}_A}(\lambda ), \end{aligned} \end{aligned}$$

whence the assertion by the arbitrariness of $\varphi _2$. The other equality is proved similarly. $\square$

Corollary 3.20

Let P be a polynomial on $\sigma ({\mathcal {L}}_A)$. Then

$$\begin{aligned} P({\mathcal {L}}_A)\chi _{{\mathcal {L}}_A}(\lambda ,\,\cdot \,)=P({\mathcal {L}}_A^R)\chi _{{\mathcal {L}}_A}(\lambda ,\,\cdot \,)=P(\lambda )\chi _{{\mathcal {L}}_A}(\lambda ,\,\cdot \,) \end{aligned}$$

for $\beta _{{\mathcal {L}}_A}$-almost every $\lambda \in \sigma ({\mathcal {L}}_A)$; here, ${\mathcal {L}}_A^R$ denotes the family of right-invariant differential operators which corresponds to ${\mathcal {L}}_A$.

The following result shows the existence of well-behaved representatives of $\chi _{{\mathcal {L}}_A}$. Its proof follows the lines of those of [66, Lemmas 2.12 and 2.15, and Propositions 2.17 and 2.18], with minor modifications.

Theorem 3.21

There is a representative $\chi _{0}$ of $\chi _{{\mathcal {L}}_A}$ such that the following hold:

• 1. $\chi _{0}(\lambda ,\,\cdot \,)$ is a function of positive type of class $C^{\infty }$ with maximum 1 for every $\lambda \in \sigma ({\mathcal {L}}_A)$;

• 2. for every homogeneous left- and right-invariant differential operators X and Y on G of degrees $\text {d}_X$ and $\text {d}_Y$, respectively, there is a constant $C_{X,Y}>0$ such that

  $$\begin{aligned} \Vert Y X \chi _{0}(\lambda ,\,\cdot \,) \Vert _\infty \leqslant C_{\gamma _1,\gamma _2} |\lambda |^{\text {d}_{X}+\text {d}_{Y}} \end{aligned}$$

  for every $\lambda \in \sigma ({\mathcal {L}}_A)$;

• 3. $\chi _{0}(\lambda ,\,\cdot \,)$ converges to $\chi _{0}(0,\,\cdot \,)=1$ in ${\mathcal {E}}(G)$ as $\lambda \rightarrow 0$;

• 4. $\chi _{0}(\,\cdot \,,g)$ is $\beta _{{\mathcal {L}}_A}$-measurable for every $g\in G$.

Proof

Observe first that, since ${\mathcal {K}}_{{\mathcal {L}}_A}(m)$ is a continuous function of positive type with maximum 1 for every positive function m in $L^1(\beta _{{\mathcal {L}}_A})$ with integral 1, and since $L^1(G)$ is separable, $\chi _{{\mathcal {L}}_A}(\lambda ,\,\cdot \,)$ is a continuous function of positive type with maximum 1 for $\beta _{{\mathcal {L}}_A}$-almost every $\lambda \in \sigma ({\mathcal {L}}_A)$.

Now, there is a unique (positive Radon) measure ${\widetilde{\beta }}$ on $S{:}{=}\lbrace \lambda \in \sigma ({\mathcal {L}}_A):|\lambda |=1 \rbrace$ (cf. Definition 3.8) such that $\int _{\sigma ({\mathcal {L}}_A)} m\,\text {d}\beta _{{\mathcal {L}}_A}= \int _{0}^\infty \int _{S} m(r \cdot \lambda )\,\text {d}{\widetilde{\beta }}(\lambda )\,r^{Q-1}\text {d}r$ (argue as in the proof of [33, Proposition 1.15], using 3 of Theorem 3.15). Similarly, by Proposition 3.18 there is a unique ${\widetilde{\chi }}\in L^\infty ({\widetilde{\beta }} \otimes \nu _G)$ such that $\chi _{{\mathcal {L}}_A}(r\cdot \lambda , r^{-1}\cdot g)={\widetilde{\chi }}(\lambda ,g)$ for $(\nu _{\mathbb {R}^*_+}\otimes {\widetilde{\beta }} \otimes \nu _G)$-almost every $(r,\lambda ,g)\in \mathbb {R}^*_+\times S\times G$, where $\nu _{\mathbb {R}^*_+}$ denotes a Haar measure on the multiplicative group $\mathbb {R}^*_+=]0,+\infty [$.

Fix a representative ${\widetilde{\chi }}_1$ of ${\widetilde{\chi }}$, and define

$$\begin{aligned} {\widetilde{\chi }}_0(\lambda ,g){:}{=}\frac{1}{m(\lambda )^2}{\mathcal {K}}_{{\mathcal {L}}_A}(m)*{\widetilde{\chi }}_1(\lambda ,\,\cdot \,)*{\mathcal {K}}_{{\mathcal {L}}_A}(m) \end{aligned}$$

for every $(\lambda ,g)\in S\times G$, where m is a nowhere vanishing Schwartz function on $\mathbb {R}^A$. Then, ${\widetilde{\chi }}_0$ is a representative of ${\widetilde{\chi }}$ by Proposition 3.19 and ${\widetilde{\chi }}_0(\lambda ,\,\cdot \,)$ is a function of positive type of class $C^\infty$ with maximum 1 for ${\widetilde{\beta }}$-almost every $\lambda \in S$; we may assume that this happens for every $\lambda \in S$. Then, define $\chi _0(\lambda ,g){:}{=}{\widetilde{\chi }}_0(|\lambda |^{-1}\cdot \lambda , |\lambda |\cdot g)$ for every $(\lambda ,g)\in \sigma ({\mathcal {L}}_A)\times G$ with $\lambda \ne 0$, so that 1, 2, and 4 follow. For what concerns 3, observe that the $\chi _0(\lambda ,\,\cdot \,)$, as $|\lambda |\leqslant 1$, stay in a bounded, hence relatively compact, subset of ${\mathcal {E}}(G)$ by 2. In addition, if h is a cluster point of $\chi _0(\lambda ,\,\cdot \,)$ for $\lambda \rightarrow 0$, then 2 implies that $X h=0$ for every left-invariant vector field on G, so that h is constant. By 1, $h(e)=1$, whence 3. $\square$

We conclude this section with some remarks concerning the adjoint of ${\mathcal {K}}_{{\mathcal {L}}_A}$ and the continuity of $\chi _{{\mathcal {L}}_A}$.

Definition 3.22

We denote by ${\mathcal {M}}_{{\mathcal {L}}_A}:{\mathcal {M}}^1(G)\rightarrow L^\infty (\beta _{{\mathcal {L}}_A})$ the transpose of the mapping

$$\begin{aligned} L^1(\beta _{{\mathcal {L}}_A})\ni m\mapsto {\mathcal {K}}_{{\mathcal {L}}_A,1}(m) \check{\;} \in C_0(G). \end{aligned}$$

Notice that ${\mathcal {M}}_{{\mathcal {L}}_A}$ coincides with the adjoint of ${\mathcal {K}}_{{\mathcal {L}}_A}:L^2(\beta _{{\mathcal {L}}_A})\rightarrow L^2(G)$ on $L^1(G)\cap L^2(G)$. By interpolation we then deduce that ${\mathcal {M}}_{{\mathcal {L}}_A}$ extends to a continuous linear mapping of $L^p(G)$ into $L^{p'}(\beta _{{\mathcal {L}}_A})$ for every $p\in [1,2]$. In addition, observe that the definition of $\chi _{{\mathcal {L}}_A}$ and routine arguments lead to the following result (cf. [66, Theorem 2.13]).

Proposition 3.23

Take a representative $\chi _{0}$ of $\chi _{{\mathcal {L}}_A}$ as in Theorem 3.21. Then, for every $\mu \in {\mathcal {M}}^1(G)$ we have

$$\begin{aligned} {\mathcal {M}}_{{\mathcal {L}}_A}(\mu )(\lambda )=\int _{G} \overline{\chi _{0}(\lambda ,g) }\,\text {d}\mu (g) \end{aligned}$$

for $\beta _{{\mathcal {L}}_A}$-almost every $\lambda \in \sigma ({\mathcal {L}}_A)$.

Thus, from Proposition 3.19 we deduce the following result.

Corollary 3.24

Take $m\in L^\infty (\beta _{{\mathcal {L}}_A})$ such that ${\mathcal {K}}_{{\mathcal {L}}_A}(m)\in {\mathcal {M}}^1(G)$ and $\mu \in {\mathcal {M}}^1(G)$. Then

$$\begin{aligned} {\mathcal {M}}_{{\mathcal {L}}_A}( {\mathcal {K}}_{{\mathcal {L}}_A}(m)*\mu )= {\mathcal {M}}_{{\mathcal {L}}_A}(\mu * {\mathcal {K}}_{{\mathcal {L}}_A}(m) )=m {\mathcal {M}}_{{\mathcal {L}}_A}(\mu ). \end{aligned}$$

Corollary 3.25

Take a function $m\in L^\infty (\beta _{{\mathcal {L}}_A})$ such that ${\mathcal {K}}_{{\mathcal {L}}_A}(m)\in {\mathcal {M}}^1(G)$. Then, $m={\mathcal {M}}_{{\mathcal {L}}_A}({\mathcal {K}}_{{\mathcal {L}}_A}(m))$. In particular, m is continuous at 0 and

$$\begin{aligned} m(0)=\int _{G} \text {d}{\mathcal {K}}_{{\mathcal {L}}_A}(m). \end{aligned}$$

Proof

The first assertion follows from Corollary 3.24, applied with $\mu {:}{=}\delta _e$. The second assertion follows from 3 of Theorem 3.21. $\square$

Theorem 3.26

The following conditions are equivalent:

• 1. $\chi _{{\mathcal {L}}_A}$ has a representative $\chi _0$ such that $\chi _0(\,\cdot \,, g)$ is continuous on $\sigma ({\mathcal {L}}_A)$ for $\nu _G$-almost every $g\in G$;⁷

• 2. ${\mathcal {M}}_{{\mathcal {L}}_A}$ induces a continuous linear mapping from $L^1(G)$ into $C_0(\sigma ({\mathcal {L}}_A))$;

• 3. ${\mathcal {M}}_{{\mathcal {L}}_A}$ induces a continuous linear mapping from ${\mathcal {M}}^1(G)$ into $C_b(\sigma ({\mathcal {L}}_A))$;

• 4. $\chi _{{\mathcal {L}}_A}$ has a continuous representative.

In particular, this shows that, if $\chi _{{\mathcal {L}}_A}$ has a continuous representative, then ${\mathcal {L}}_A$ satisfies property (RL). Nevertheless, the converse fails, as Remark 8.3 shows.

Proof

1 $\implies$2 Take $\varphi \in L^1(G)$. In order to prove that ${\mathcal {M}}_{{\mathcal {L}}_A}(\varphi )$ is continuous, it suffices to show that

$$\begin{aligned} {\mathcal {M}}_{{\mathcal {L}}_A}(\varphi )(\lambda )=\int _G \overline{\chi _0(\lambda ,g)}\,\varphi (g)\,\text {d}g \end{aligned}$$

for $\beta _{{\mathcal {L}}_A}$-almost every $\lambda \in \sigma ({\mathcal {L}}_A)$, and to apply the dominated convergence theorem. In order to prove that ${\mathcal {M}}_{{\mathcal {L}}_A}(\varphi )$ vanishes at $\infty$, it suffices to observe that, if $\tau \in C^\infty _c(\sigma ({\mathcal {L}}_A))$ and $\tau (0)=1$, then ${\mathcal {M}}_{{\mathcal {L}}_A}(\varphi )$ is the limit in $C_b(\sigma ({\mathcal {L}}_A))$ of ${\mathcal {M}}_{{\mathcal {L}}_A}(\varphi *{\mathcal {K}}_{{\mathcal {L}}_A}(\tau (2^{-j}\,\cdot \,)))$, which equals $\tau (2^{-j}\,\cdot \,) {\mathcal {M}}_{{\mathcal {L}}_A}(\varphi )$ by Corollary 3.24.

2 $\implies$4 Take $\tau \in {\mathcal {S}}(\sigma ({\mathcal {L}}_A))$ such that $\tau (\lambda )>0$ for every $\lambda \in \sigma ({\mathcal {L}}_A)$. Observe that the mapping $G\ni g\mapsto {\mathcal {K}}_{{\mathcal {L}}_A}(\tau )(g\,\cdot \,)\in L^1(G)$ is continuous, so that also the mapping $G\ni g\mapsto {\mathcal {M}}_{{\mathcal {L}}_A}({\mathcal {K}}_{{\mathcal {L}}_A}(\tau )(g\,\cdot \,))\in C_0(\sigma ({\mathcal {L}}_A))$ is continuous. Therefore, the mapping

$$\begin{aligned} \sigma ({\mathcal {L}}_A)\times G\ni (\lambda ,g)\mapsto {\mathcal {M}}_{{\mathcal {L}}_A}({\mathcal {K}}_{{\mathcal {L}}_A}(\tau )(g\,\cdot \,))(\lambda ) \in \mathbb {C}\end{aligned}$$

is continuous. Now, let $\chi _{1}$ be a representative of $\chi _{{\mathcal {L}}_A}$ as in Theorem 3.21. Then, Proposition 3.19 implies that

$$\begin{aligned} \begin{aligned} {\mathcal {M}}_{{\mathcal {L}}_A}({\mathcal {K}}_{{\mathcal {L}}_A}(\tau )(g\,\cdot \,))(\lambda )&= \int _G {\mathcal {K}}_{{\mathcal {L}}_A}(\tau )(g g')\chi _{1}(\lambda ,g'^{-1})\,\text {d}g'\\&= [ {\mathcal {K}}_{{\mathcal {L}}_A}(\tau )*\chi _{1}(\lambda ,\,\cdot \,) ](g)= \tau (\lambda ) \chi _{1}(\lambda ,g) \end{aligned} \end{aligned}$$

for $(\beta _{{\mathcal {L}}_A}\otimes \nu _G)$-almost every $(\lambda , g)\in \sigma ({\mathcal {L}}_A)\times G$. In particular, $\chi _{{\mathcal {L}}_A}$ has a representative which is continuous on $\sigma ({\mathcal {L}}_A)\times G$. By [16, Corollary to Theorem 2 of Chapter IX, Sect. 4, No. 2], $\chi _{{\mathcal {L}}_A}$ has a continuous representative.

4$\implies$1 Obvious.

4$\implies$3 The proof is similar to that of the implication 1 $\implies$2.

3$\implies$2 This follows from the proof of the implication 1 $\implies$2.$\square$

4 Products

In this section we deal with the following situation: we have a finite family of homogeneous groups $(G_A)_{A\in {\mathcal {A}}}$, and on each $G_A$ a Rockland family ${\mathcal {L}}_A$.⁸ Then, we shall consider $G{:}{=}\prod _{A\in {\mathcal {A}}} G_A$, endowed with the dilations

$$\begin{aligned} r\cdot (g_A){:}{=}(r\cdot g_A), \end{aligned}$$

for $r>0$ and $(g_A)\in G$. We shall denote by $A'$ the union of ${\mathcal {A}}$ and, for every $\alpha \in A'$, we shall denote by ${\mathcal {L}}'_\alpha$ the operator on G induced by ${\mathcal {L}}_\alpha$. We denote by ${\mathcal {L}}'_{ A'}$ the family $({\mathcal {L}}'_\alpha )_{\alpha \in A'}$. We shall investigate what we can say about ${\mathcal {L}}'_{ A'}$ on the ground of our knowledge of the families ${\mathcal {L}}_A$.

The following result is a consequence of [48, Theorem 5.4 and Corollary 5.5] except for the last point, which is an easy consequence of the preceding ones.

Proposition 4.1

The following hold:

• 1. ${\mathcal {L}}'_{ A'}$ is a Rockland family and $\sigma ({\mathcal {L}}'_{A'})=\prod _{A\in {\mathcal {A}}}\sigma ({\mathcal {L}}_A)$;

• 2. take a $\mu _{{\mathcal {L}}_A}$-measurable function $m_A:\sigma ({\mathcal {L}}_A)\rightarrow \mathbb {C}$ which admits a kernel for every $A\in {\mathcal {A}}$. Then, $\bigotimes _{A\in {\mathcal {A}}} m_A$ is $\mu _{{\mathcal {L}}'_{ A'}}$-measurable, admits a kernel, and

  $$\begin{aligned} {\mathcal {K}}_{{\mathcal {L}}'_{ A'}}\left( \bigotimes _{A\in {\mathcal {A}}} m_A \right) = \bigotimes _{A\in {\mathcal {A}}} {\mathcal {K}}_{{\mathcal {L}}_A}(m_A); \end{aligned}$$

• 3. $\beta _{{\mathcal {L}}'_{ A'}}= \bigotimes _{A\in {\mathcal {A}}} \beta _{{\mathcal {L}}_A}$;

• 4. for $(\beta _{{\mathcal {L}}'_{ A'}}\otimes \nu _G)$-almost every $((\lambda _\alpha ), (g_A))\in \sigma ({\mathcal {L}}'_{A'})\times G$,

  $$\begin{aligned} \chi _{{\mathcal {L}}'_{ A'}} ((\lambda _\alpha )_{\alpha \in A'}, (g_A)_{A\in {\mathcal {A}}})= \prod _{A\in {\mathcal {A}}} \chi _{{\mathcal {L}}_A}((\lambda _\alpha )_{\alpha \in A}, g_A ). \end{aligned}$$

Now we focus on property (RL). See Sect. 2.5 for the notation concerning topological tensor products.

Lemma 4.2

Assume that ${\mathcal {A}}=\lbrace A_1,A_2 \rbrace$. Then, for every $m\in L^1(\beta _{{\mathcal {L}}'_{ A'}})$ and for every $\mu \in {\mathcal {M}}^1(G_{A_2})$ there is $m_{\mu }\in L^1(\beta _{{\mathcal {L}}_{A_1}})$ such that

$$\begin{aligned} \int _{G_{A_2}}{\mathcal {K}}_{{\mathcal {L}}'_{ A'},1}(m)(\,\cdot \,,g_2)\,\text {d}\mu (g_2) = {\mathcal {K}}_{{\mathcal {L}}_{A_1},1}(m_{\mu }). \end{aligned}$$

Proof

Observe first that

$$\begin{aligned} L^1(\beta _{{\mathcal {L}}'_{ A'}})\cong L^1(\beta _{{\mathcal {L}}_{A_1}}; L^1(\beta _{{\mathcal {L}}_{A_2}}))\cong L^1(\beta _{{\mathcal {L}}_{A_1}}){\widehat{\otimes }} L^1(\beta _{{\mathcal {L}}_{A_2}}) \end{aligned}$$

thanks to Proposition 4.1 and [67, Theorem 46.2]. Therefore, [67, Theorem 45.1] implies that there are $(c_j)\in \ell ^1$ and two bounded sequences $(m_{j,1}), (m_{j,2})$ in $L^1(\beta _{{\mathcal {L}}_{A_1}})$ and $L^1(\beta _{{\mathcal {L}}_{A_2}})$, respectively, such that

$$\begin{aligned} m=\sum _{j\in \mathbb {N}} c_j (m_{j,1}\otimes m_{j,2}) \end{aligned}$$

in $L^1(\beta _{{\mathcal {L}}'_{ A'}})$. Hence, it suffices to define

$$\begin{aligned} m_{\mu }{:}{=}\sum _{j\in \mathbb {N}} c_j\, \int _{G_2}{\mathcal {K}}_{{\mathcal {L}}_{A_2},1}(m_{j,2})\,\text {d}\mu \, m_{j,1}. \end{aligned}$$

$\square$

Corollary 4.3

Assume that ${\mathcal {A}}=\lbrace A_1,A_2 \rbrace$. Take $f\in L^1_{{\mathcal {L}}'_{A'}}(G)$ and $h\in L^\infty (G_{A_2})$. Then,

$$\begin{aligned} \int _{G_{A_2}} f(\,\cdot \,,g_2) h(g_2)\,\text {d}\nu _{G_{A_2}}(g_2)\in L^1_{{\mathcal {L}}_{A_1}}(G_{A_1}). \end{aligned}$$

In addition, $f(\,\cdot \,,g_2)\in L^1_{{\mathcal {L}}_{A_1}}(G_{A_1})$ for almost every $g_2\in G_{A_2}$.

Proof

1 Assume first that $m{:}{=}{\mathcal {M}}_{{\mathcal {L}}'_{A'}}(f)\in L^\infty (\sigma ({\mathcal {L}}'_{A'}))$ is compactly supported. Let $(K_j)$ be an increasing sequence of compact subsets of $G_{A_2}$ whose union is $G_{A_2}$. Then,

$$\begin{aligned} \begin{aligned} \lim _{j\rightarrow \infty } \int _{K_j} f(\,\cdot \,,g_2) h(g_2)\,\text {d}&\nu _{G_{A_2}}(g_2)=\int _{G_{A_2}}f(\,\cdot \,,g_2) h(g_2)\,\text {d}\nu _{G_{A_2}}(g_2) \end{aligned} \end{aligned}$$

in $L^1(G_{A_1})$. The first assertion follows from Lemma 4.2 and Proposition 3.14, while the second assertion follows directly from Lemma 4.2.

2 Now, take $\tau \in C^\infty _c(\sigma ({\mathcal {L}}_A))$ such that $\tau (0)=1$, and define $\tau _j{:}{=}\tau (2^{-j}\,\cdot \,)$ for every $j\in \mathbb {N}$. Then, 1 above implies that

$$\begin{aligned} \int _{G_2}{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m \tau _j )(\,\cdot \,,g_2) h(g_2)\,\text {d}\nu _{G_{A_2}}(g_2)\in L^1_{{\mathcal {L}}_{A_1}}(G_{A_1}) \end{aligned}$$

and that ${\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m \tau _j)(\,\cdot \,,g_2)\in L^1_{{\mathcal {L}}_{A_1}}(G_{A_1})$ for every $j\in \mathbb {N}$ and for almost every $g_2\in G_{A_2}$. Since ${\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m \tau _j )=f*{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}( \tau _j )$ converges to f in $L^1(G_{ A'})$, both assertions follow from Proposition 3.14. $\square$

Theorem 4.4

If ${\mathcal {L}}_A$ satisfies property (RL) for every $A\in {\mathcal {A}}$, then ${\mathcal {L}}'_{ A'}$ satisfies property (RL).

Proof

1 Proceeding by induction, we may reduce to the case in which ${\mathcal {A}}=\lbrace A_1,A_2 \rbrace$. In order to simplify the notation, we shall write $G_j$ instead of $G_{A_j}$ for $j=1,2$. Now, take $m\in L^\infty (\beta _{{\mathcal {L}}'_{ A'}})$ such that ${\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)\in L^1(G)$. Then, Corollary 3.25, Proposition 4.1 and Fubini’s theorem imply that

$$\begin{aligned} {\mathcal {M}}_{{\mathcal {L}}_{A_1}}[g_1\mapsto {\mathcal {M}}_{{\mathcal {L}}_{A_2}}[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)](\lambda _2)](\lambda _1)=m(\lambda _1, \lambda _2) \end{aligned}$$

for $\beta _{{\mathcal {L}}_{A_1}}$-almost every $\lambda _1\in \sigma ({\mathcal {L}}_{A_1})$ and for $\beta _{{\mathcal {L}}_{A_2}}$-almost every $\lambda _2\in \sigma ({\mathcal {L}}_{A_2})$. Observe that Lemma 4.2 implies that ${\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)\in L^1_{{\mathcal {L}}_{A_2}}(G_{2})$ for almost every $g_1\in G_{1}$; in addition, observe that, by assumption, ${\mathcal {M}}_{{\mathcal {L}}_{A_2}}$ induces a continuous linear mapping from $L^1_{{\mathcal {L}}_{A_2}}(G_{2})$ into $C_0(\sigma ({\mathcal {L}}_{A_2}))$. Therefore, the mapping $g_1 \mapsto {\mathcal {M}}_{{\mathcal {L}}_{A_2}}[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)]$ defines an element of $L^1(G_{1}; C_0(\sigma ({\mathcal {L}}_{A_2})))$.

2 Let us prove that, for every $\mu \in {\mathcal {M}}^1(\sigma ({\mathcal {L}}_{A_2}))$, the mapping

$$\begin{aligned} g_1 \mapsto (\mu {\mathcal {M}}_{{\mathcal {L}}_{A_2}})[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)] \end{aligned}$$

belongs to $L^1_{{\mathcal {L}}_{A_1}}(G_{1})$. Indeed, the preceding considerations show that $\mu {\mathcal {M}}_{{\mathcal {L}}_{A_2}}$ defines an element of $L^1_{{\mathcal {L}}_{A_2}}(G_{2})'$, so that it can be represented by an element of $L^\infty (G_2)$; hence, the assertion follows from Corollary 4.3.

Now, let us prove that the mapping

$$\begin{aligned} {\mathcal {M}}^1(\sigma ({\mathcal {L}}_{A_1}))\ni \mu \mapsto \left[ g_1 \mapsto (\mu {\mathcal {M}}_{{\mathcal {L}}_{A_2}})[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)]\right] \in L^1_{{\mathcal {L}}_{A_1}}(G_1) \end{aligned}$$

is weakly continuous on the bounded subsets of${\mathcal {M}}^1(\sigma ({\mathcal {L}}_{A_1}))$. Indeed, [67, Theorem 46.2] implies that $L^1(G_1; C_0(\sigma ({\mathcal {L}}_{A_2})))\cong L^1(G_1){\widehat{\otimes }} C_0(\sigma ({\mathcal {L}}_{A_2}))$, so that [67, Theorem 45.1] implies that there are $(c_j)\in \ell ^1$ and two bounded sequences $(f_{j}), (\varphi _{j})$ in $L^1(G_1)$ and $C_0(\sigma ({\mathcal {L}}_{A_2}))$, respectively, such that

$$\begin{aligned} \left[ g_1\mapsto {\mathcal {M}}_{{\mathcal {L}}_{A_2}}[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)]\right] =\sum _{j\in \mathbb {N}} c_j (f_{j}\otimes \varphi _{j}) \end{aligned}$$

in $L^1(G_1; C_0(\sigma ({\mathcal {L}}_{A_2})))$. Since the series

$$\begin{aligned} \sum _{j\in \mathbb {N}} c_j \langle \mu , \varphi _j\rangle f_j \end{aligned}$$

converges uniformly to $g_1 \mapsto (\mu {\mathcal {M}}_{{\mathcal {L}}_{A_2}})[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)]$ as $\mu$ stays in a bounded subset of ${\mathcal {M}}^1(\sigma ({\mathcal {L}}_{A_2}))$, the assertion follows.

3 Observe that, by assumption, ${\mathcal {M}}_{{\mathcal {L}}_{A_1}}$ induces a continuous linear mapping from $L^1_{{\mathcal {L}}_{A_1}}(G_{1})$ into $C_0(\sigma ({\mathcal {L}}_{A_1}))$; hence, 2 above implies that the mapping

$$\begin{aligned} \begin{aligned} \sigma ({\mathcal {L}}_{A_2})\ni \lambda _2 \mapsto {\mathcal {M}}_{{\mathcal {L}}_{A_1}}\left( g_1\mapsto {\mathcal {M}}_{{\mathcal {L}}_{A_2}}[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)](\lambda _2) \right) \in C_0(\sigma ({\mathcal {L}}_{A_1})) \end{aligned} \end{aligned}$$

is continuous. Therefore, the mapping

$$\begin{aligned} m_0:\sigma ({\mathcal {L}}'_{ A'})\ni (\lambda _1,\lambda _2)\mapsto {\mathcal {M}}_{{\mathcal {L}}_{A_1}}\left( g_1\mapsto {\mathcal {M}}_{{\mathcal {L}}_{A_2}}[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)](\lambda _2) \right) (\lambda _1)\in \mathbb {C}\end{aligned}$$

is continuous. Now, 1 implies that $m_0(\lambda _1,\lambda _2)=m(\lambda _1,\lambda _2)$ for $\beta _{{\mathcal {L}}_{A_1}}$-almost every $\lambda _1\in \sigma ({\mathcal {L}}_{A_1})$ and for $\beta _{{\mathcal {L}}_{A_2}}$-almost every $\lambda _2\in \sigma ({\mathcal {L}}_{A_2})$. Since both m and $m_0$ are $\beta _{{\mathcal {L}}'_{ A'}}$-measurable, Tonelli’s theorem implies that $m=m_0$$\beta _{{\mathcal {L}}'_{ A'}}$-almost everywhere. $\square$

Now, we focus on property (S).

Lemma 4.5

Assume that ${\mathcal {A}}=\lbrace A_1,A_2 \rbrace$. Take $\varphi \in {\mathcal {S}}_{{\mathcal {L}}'_{A'}}(G_{ A'})$, and take $T\in {\mathcal {S}}'(G_{A_2})$. Then,

$$\begin{aligned}{}[g_1\mapsto \langle T, \varphi (g_1,\,\cdot \,)\rangle ]\in {\mathcal {S}}_{{\mathcal {L}}_{A_1}}(G_{A_1}). \end{aligned}$$

The proof is similar to that of Corollary 4.3,with the only difference that here one has to approximate T in ${\mathcal {S}}'(G_{A_2})$ by a sequence of measures with compact support.

Theorem 4.6

If ${\mathcal {L}}_A$ satisfies property (S) for every $A\in {\mathcal {A}}$, then ${\mathcal {L}}'_{ A'}$ satisfies property (S).

Proof

1 Proceeding by induction, we may reduce to the case in which ${\mathcal {A}}=\lbrace A_1,A_2 \rbrace$. In order to simplify the notation, we shall write $G_j$ instead of $G_{A_j}$, for $j=1,2$. Now, take $m\in L^\infty (\beta _{{\mathcal {L}}'_{ A'}})$ such that ${\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)\in {\mathcal {S}}(G)$. Then, Corollary 3.25, Proposition 4.1 and Fubini’s theorem imply that

$$\begin{aligned} {\mathcal {M}}_{{\mathcal {L}}_{A_1}}[g_1\mapsto {\mathcal {M}}_{{\mathcal {L}}_{A_2}}[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)](\lambda _2)](\lambda _1)=m(\lambda _1, \lambda _2) \end{aligned}$$

for $\beta _{{\mathcal {L}}_{A_1}}$-almost every $\lambda _1\in \sigma ({\mathcal {L}}_{A_1})$ and for $\beta _{{\mathcal {L}}_{A_2}}$-almost every $\lambda _2\in \sigma ({\mathcal {L}}_{A_2})$. Observe that Lemma 4.5 implies that ${\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)\in {\mathcal {S}}_{{\mathcal {L}}_{A_2}}(G_{2})$ for every $g_1\in G_{1}$, and that by assumption ${\mathcal {M}}_{{\mathcal {L}}_{A_2}}$ induces a continuous linear mapping from ${\mathcal {S}}_{{\mathcal {L}}_{A_2}}(G_{2})$ onto ${\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2}))$. Therefore, the map $g_1 \mapsto {\mathcal {M}}_{{\mathcal {L}}_{A_2}}[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)]$ defines an element of ${\mathcal {S}}(G_{1}; {\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2})))$.

2 Let us prove that the mapping $g_1 \mapsto {\mathcal {M}}_{{\mathcal {L}}_{A_2}}[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)]$ induces an element of ${\mathcal {S}}_{{\mathcal {L}}_{A_1}}(G_{1}){\widehat{\otimes }} {\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2}))$. Take $T\in {\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2}))'$; then, Lemma 4.5 implies that

$$\begin{aligned}{}[g_1 \mapsto (T{\mathcal {M}}_{{\mathcal {L}}_{A_2}})[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)]] \in {\mathcal {S}}_{{\mathcal {L}}_{A_1}}(G_{1}) \end{aligned}$$

since $T{\mathcal {M}}_{{\mathcal {L}}_{A_2}}$ defines an element of ${\mathcal {S}}_{{\mathcal {L}}_{A_2}}(G_{2})'$, which can be extended to an element of ${\mathcal {S}}'(G_2)$. Next, observe that [67, Proposition 50.4] implies that

$$\begin{aligned} {\mathcal {S}}_{{\mathcal {L}}_{A_1}}(G_{1}){\widehat{\otimes }} {\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2}))\cong {\mathcal {L}}({\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2}))'; {\mathcal {S}}_{{\mathcal {L}}_{A_1}}(G_{1})) \end{aligned}$$

since ${\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2}))$ is nuclear thanks to [67, Proposition 50.1]. Now, the mapping

$$\begin{aligned} g_1 \mapsto {\mathcal {M}}_{{\mathcal {L}}_{A_2}}[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)] \end{aligned}$$

belongs to ${\mathcal {S}}(G_1; {\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2})))$; arguing as above, we see that this latter space is the canonical image of ${\mathcal {S}}(G_1){\widehat{\otimes }} {\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2}))\cong {\mathcal {L}}({\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2}))'; {\mathcal {S}}(G_{1}))$, so that the preceding arguments imply our claim.

3 Now, by assumption ${\mathcal {M}}_{{\mathcal {L}}_{A_1}}$ induces a continuous linear map from ${\mathcal {S}}_{{\mathcal {L}}_{A_1}}(G_{1})$ into ${\mathcal {S}}(\sigma ({\mathcal {L}}_{A_1}))$, so that we have the continuous linear mapping

$$\begin{aligned} {\mathcal {M}}_{{\mathcal {L}}_{A_1}}{\widehat{\otimes }} I_{{\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2})) } :{\mathcal {S}}_{{\mathcal {L}}_{A_1}}(G_{1}){\widehat{\otimes }} {\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2}))\rightarrow {\mathcal {S}}(\sigma ({\mathcal {L}}_{A_1})){\widehat{\otimes }} {\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2})); \end{aligned}$$

in addition, for every $T\in {\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2}))'$ and for every $\lambda _1\in \sigma ({\mathcal {L}}_{A_1})$,

$$\begin{aligned} \begin{aligned}&\Big \langle T, \left( {\mathcal {M}}_{{\mathcal {L}}_{A_1}}{\widehat{\otimes }} I_{{\mathcal {S}}(\sigma ({\mathcal {L}}_{A_2}))}\right) ( g_1 \mapsto {\mathcal {M}}_{{\mathcal {L}}_{A_2}}[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)] )(\lambda _1)\Big \rangle \\&\quad = {\mathcal {M}}_{{\mathcal {L}}_{A_1}}[g_1 \mapsto T{\mathcal {M}}_{{\mathcal {L}}_{A_2}}[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)]](\lambda _1) \end{aligned} \end{aligned}$$

(reason as in 2). Choosing $T=\delta _{\lambda _2}$ for $\lambda _2\in \sigma ({\mathcal {L}}_{A_2})$, and taking into account Proposition 3.12, we see that the mapping

$$\begin{aligned} \sigma ({\mathcal {L}}'_{ A'})\ni (\lambda _1,\lambda _2)\mapsto {\mathcal {M}}_{{\mathcal {L}}_{A_1}}(g_1 \mapsto {\mathcal {M}}_{{\mathcal {L}}_{A_2}}[{\mathcal {K}}_{{\mathcal {L}}'_{ A'}}(m)(g_1,\,\cdot \,)](\lambda _2))(\lambda _1) \end{aligned}$$

induces an element $m_0$ of ${\mathcal {S}}(\sigma ({{\mathcal {L}}'_{ A'}}))$. Now, 1 implies that $m_0(\lambda _1,\lambda _2)=m(\lambda _1,\lambda _2)$ for $\beta _{{\mathcal {L}}_{A_1}}$-almost every $\lambda _1\in \sigma ({\mathcal {L}}_{A_1})$ and for $\beta _{{\mathcal {L}}_{A_2}}$-almost every $\lambda _2\in \sigma ({\mathcal {L}}_{A_2})$. Since both m and $m_0$ are $\beta _{{\mathcal {L}}'_{ A'}}$-measurable, Tonelli’s theorem implies that $m=m_0$$\beta _{{\mathcal {L}}'_{ A'}}$-almost everywhere. The assertion follows. $\square$

5 Image Families

In this section we fix a Rockland family ${\mathcal {L}}_A$ on a homogeneous group G; we consider ${\mathcal {L}}_A$ as ‘known’ and we study an ‘image family’ $P({\mathcal {L}}_A)$, where $P:\mathbb {R}^A\rightarrow \mathbb {R}^\Gamma$ is a polynomial mapping with homogeneous components, and $\Gamma$ is a finite set. We shall investigate what we can say about $P({\mathcal {L}}_A)$ on the base of our knowledge of ${\mathcal {L}}_A$.

Proposition 5.1

The following statements are equivalent:

• 1. $P({\mathcal {L}}_A)$ is a Rockland family;

• 2. the restriction of P to $\sigma ({\mathcal {L}}_A)$ is proper;

In addition, if $P({\mathcal {L}}_A)$ is a Rockland family, then:

• (i) $\mu _{P({\mathcal {L}}_A)}=P_*(\mu _{{\mathcal {L}}_A})$ and $\sigma (P({\mathcal {L}}_A))=P(\sigma ({\mathcal {L}}_A))$;

• (ii) a $\beta _{P({\mathcal {L}}_A)}$-measurable function $m:\sigma (P({\mathcal {L}}_A))\rightarrow \mathbb {C}$ admits a kernel if and only if $m\circ P$ admits a kernel; in this case,

  $$\begin{aligned} {\mathcal {K}}_{P({\mathcal {L}}_A)}(m)= {\mathcal {K}}_{{\mathcal {L}}_A}(m\circ P); \end{aligned}$$

• (iii) $\beta _{P({\mathcal {L}}_A)}=P_*(\beta _{{\mathcal {L}}_A})$.

Notice that saying that P is proper on $\sigma ({\mathcal {L}}_A)$ amounts to saying that $P(\lambda )\ne 0$ for every non-zero $\lambda \in \sigma ({\mathcal {L}}_A)$ (cf. the proof of [33, Lemma 1.4]).

Proof

By spectral theory, $\mu _{P({\mathcal {L}}_A)}=P_*(\mu _{{\mathcal {L}}_A})$ and $\sigma (P({\mathcal {L}}_A))=\overline{P(\sigma ({\mathcal {L}}_A))}$, without further assumptions on $P({\mathcal {L}}_A)$. If $P({\mathcal {L}}_A)$ is a Rockland family, then also (ii) holds by spectral theory again; as a consequence, also (iii) holds in this case. Then, we are reduced to proving the equivalence of 1 and 2.

1 $\implies$2 This follows from [47, Lemma 3.5.1].

2$\implies$1 Notice first that the union of the families ${\mathcal {L}}_A$ and $P({\mathcal {L}}_A)$ is clearly Rockland, so that the $P_\gamma (\mathcal {L}_A)$ are essentially self-adjoint on $C^\infty _c(G)$ with commuting closures. Endow $\mathbb {R}^A$ with dilations extending those of $\sigma ({\mathcal {L}}_A)$ and with a homogeneous norm $|\,\cdot \, |'$ which is of class $C^\infty$ on $G\setminus \lbrace e \rbrace$; let S be the corresponding unit sphere. Take $\tau _1\in C^\infty (S)$ such that $\tau _1=1$ on a neighbourhood of $\sigma ({\mathcal {L}}_A)\cap S$ and such that $\tau _1$ is supported in $\lbrace x\in S:2|P(x) |\geqslant \min _{S\cap \sigma ({\mathcal {L}}_A)} |P | \rbrace$. Then, extend $\tau _1$ to a homogeneous function of degree 0. In addition, take $\tau _2\in C^\infty _c(\mathbb {R}^A)$ so that $\tau _2=1$ on a neighbourhood of 0. Now, if $m\in {\mathcal {S}}(\mathbb {R}^\Gamma )$, then clearly $[\tau _2+(1-\tau _2)\tau _1] (m\circ P)\in {\mathcal {S}}(\mathbb {R}^A)$, so that ${\mathcal {K}}_{P({\mathcal {L}}_A)}(m)\in {\mathcal {S}}(G)$. The assertion follows. $\square$

Proposition 5.2

Assume that $P({\mathcal {L}}_A)$ is a Rockland family, and take a disintegration $(\beta _{\lambda '})_{\lambda '\in \sigma (P({\mathcal {L}}_A))}$ of $\beta _{{\mathcal {L}}_A}$ relative to P. Then,

$$\begin{aligned} \chi _{P({\mathcal {L}}_A)}(\lambda ',g)= \int _{\sigma ({\mathcal {L}}_A)} \chi _{{\mathcal {L}}_A}(\lambda ,g)\,\text {d}\beta _{\lambda '}(\lambda ) \end{aligned}$$

for $(\beta _{P({\mathcal {L}}_A)}\otimes \nu _G)$-almost every $(\lambda ',g)\in \sigma (P({\mathcal {L}}_A))\times G$.

See Sect. 2.3 for the definition of a disintegration. Observe that the proof amounts to showing that both sides of the asserted equality have the same integrals when multiplied by elements of $C^\infty _c(\sigma ({\mathcal {L}}_A))\otimes C^\infty _c(G)$, which is clear.

Now, consider property (RL). Assume that ${\mathcal {L}}_A$ satisfies property (RL), and take $m\in L^\infty (\beta _{P({\mathcal {L}}_A)})$ such that ${\mathcal {K}}_{P({\mathcal {L}}_A)}(m)\in L^1(G)$. Then, there is ${{\widetilde{m}}}\in C_0(\sigma ({\mathcal {L}}_A))$ such that $m\circ P={{\widetilde{m}}}$$\beta _{{\mathcal {L}}_A}$-almost everywhere. In Sect. 10, we shall study this situation in a general setting, seeking conditions under which ${{\widetilde{m}}}$ is constant on the fibres of P in $\sigma ({\mathcal {L}}_A)$. Since P is proper, this implies that ${{\widetilde{m}}}$ is the composite of a continuous function with P, at least on $\sigma ({\mathcal {L}}_A)$. If this happens for every m, then $P({\mathcal {L}}_A)$ satisfies property (RL). Notice, however, that sometimes it is more convenient to argue on proper subsets of the spectrum.

Property (S) is studied in a similar way, again making use of the results of Sect. 10.

6 Quadratic Operators on 2-Step Stratified Groups

A connected Lie group G is called 2-step nilpotent if $[{\mathfrak {g}},[{\mathfrak {g}},{\mathfrak {g}}]]=0$, where ${\mathfrak {g}}$ is the Lie algebra of G. The group G is 2-step stratified if, in addition, it is simply connected and ${\mathfrak {g}}={\mathfrak {g}}_1\oplus {\mathfrak {g}}_2$, with $[{\mathfrak {g}}_1,{\mathfrak {g}}_1]=[{\mathfrak {g}},{\mathfrak {g}}]={\mathfrak {g}}_2$.

Notice that, if G is a simply connected 2-step nilpotent group, then it is ‘stratifiable,’ that is, for every algebraic complement ${\mathfrak {g}}_1$ of ${\mathfrak {g}}_2{:}{=}[{\mathfrak {g}},{\mathfrak {g}}]$, the decomposition ${\mathfrak {g}}={\mathfrak {g}}_1 \oplus {\mathfrak {g}}_2$ turns G into a stratified group. Nevertheless, G may be endowed with many different structures of a stratified group; when we speak of a 2-step stratified group, we mean that an algebraic complement of $[{\mathfrak {g}},{\mathfrak {g}}]$ is fixed.

A 2-step stratified group is endowed with the canonical dilations, that is $r\cdot (X+Y)=r X+ r^2 Y$ for every $r>0$, for every $X\in {\mathfrak {g}}_1$ and for every $Y\in {\mathfrak {g}}_2$. Thus, G becomes a homogeneous group.

Definition 6.1

Let G be a 2-step stratified group. Then, for every $\omega \in {\mathfrak {g}}_2^*$ we define

$$\begin{aligned} B_\omega :{\mathfrak {g}}_1\times {\mathfrak {g}}_1\ni (X,Y)\mapsto \langle \omega , [X,Y]\rangle . \end{aligned}$$

We say that G is an $MW^+$ group if $B_\omega$ is non-degenerate for some non-zero $\omega \in {\mathfrak {g}}_2^*$ (cf. [56, 57]). A Heisenberg group is an $MW^+$ group with one-dimensional centre.

Definition 6.2

Take $d\in \mathbb {N}^*$, and let ${\mathfrak {g}}$ be the free Lie algebra on d generators. Then, the quotient ${\mathfrak {g}}'$ of ${\mathfrak {g}}$ by its ideal $[{\mathfrak {g}}, [{\mathfrak {g}},{\mathfrak {g}}]]$ is the free 2-step nilpotent Lie algebra on d generators. The simply connected Lie group with Lie algebra ${\mathfrak {g}}'$ is called the free 2-step nilpotent Lie group on d generators.

Now, to every symmetric bilinear form q on ${\mathfrak {g}}_1^*$ we can associate a differential operator on G as follows:

$$\begin{aligned} {\mathcal {L}}{:}{=}-\sum _{\ell ,\ell '} q(X_\ell ^*, X_{\ell '}^*) X_{\ell } X_{\ell '}, \end{aligned}$$

where $(X_\ell )$ is a basis of ${\mathfrak {g}}_1$ with dual basis $(X^*_\ell )$. As the reader may verify, ${\mathcal {L}}$ does not depend on the choice of $(X_\ell )$; actually, one may prove that $-{\mathcal {L}}$ is the symmetrization of the quadratic form induced by q on ${\mathfrak {g}}^*$ (cf. [39, Theorem 4.3]).

Lemma 6.3

Let q be a symmetric bilinear form on ${\mathfrak {g}}_1^*$, and let ${\mathcal {L}}$ be the associated operator. Then, ${\mathcal {L}}$ is formally self-adjoint if and only if q is real. In addition, ${\mathcal {L}}$ is formally self-adjoint and hypoelliptic if and only if q is non-degenerate and either positive or negative.

Proof

The first assertion follows from the fact that the formal adjoint of ${\mathcal {L}}$ is associated with ${\overline{q}}$. The last assertion then follows from [40]. $\square$

Next, we show how to put ${\mathcal {L}}$ in a particularly convenient form according to the chosen $\omega \in {\mathfrak {g}}_2^*$.

Definition 6.4

Let V be a vector space and $\Phi$ a bilinear form on V. Then, define

$$\begin{aligned} \text {s}_\Phi :V\ni v\mapsto \Phi (v,\,\cdot \,)\in V^*\qquad \text {and}\qquad \text {d}_\Phi :V\ni v \mapsto \Phi (\,\cdot \,,v)\in V^*. \end{aligned}$$

If $\Phi$ is non-degenerate, that is, if $\text {d}_\Phi$ (or, equivalently, $\text {s}_\Phi$) is an isomorphism, then denote by ${\widehat{\Phi }}$ the inverse of $\Phi$, that is, $\Phi \circ (\text {s}_\Phi ^{-1}\times \text {d}_\Phi ^{-1})$.

Notice that any algebraic complement of the radical of a skew-symmetric bilinear form on a finite-dimensional vector space is symplectic. Therefore, by [1, Corollary 5.6.3] we deduce the following result.

Proposition 6.5

Let V be a finite-dimensional vector space over $\mathbb {R}$, let $\sigma$ be a skew-symmetric bilinear form on V, and let q be a positive, non-degenerate bilinear form on V. Then, there are a q-orthogonal basis $(v_j)_{j=1,\dots ,m}$ of V and a positive integer $n\leqslant \frac{m}{2}$ such that the following hold:

• $q(v_j,v_j)=q(v_{n+j},v_{n+j})>0$ for every $j=1,\dots , n$;

• $q(v_j,v_j)=1$ for every $j=2 n+1,\dots ,m$;

• for every $j,k=1,\dots , m$,

  $$\begin{aligned} \sigma (v_j,v_k)={\left\{ \begin{array}{ll} 1 &{} \text {if }j\in \lbrace 1,\dots ,n \rbrace \hbox { and }k=n+j;\\ -1 &{} \text {if }j\in \lbrace n+1,\dots ,2 n \rbrace \hbox { and } k=j-n;\\ 0 &{} \text {otherwise}. \end{array}\right. } \end{aligned}$$

Observe that $q(v_j,v_j)^{-1}$ is the eigenvalue of $|\text {d}_{q}^{-1}\circ \text {d}_\sigma |$ corresponding to $v_j$ ($j=1,\dots , 2 n$), where the absolute value is computed with respect to q.

7 Plancherel Measure and Integral Kernel

In this section, G denotes a 2-step stratified group of dimension n which does not satisfy the $MW^+$ condition, q a symmetric bilinear form on ${\mathfrak {g}}_1^*$, and $(T_1,\dots , T_{n_2})$ a basis of ${\mathfrak {g}}_2$. We shall denote by ${\mathcal {L}}$ the sub-Laplacian induced by q and we shall assume that ${\mathcal {L}}_A{:}{=}({\mathcal {L}}, (-i T_k)_{k=1,\dots ,n_2})$ is a Rockland family, that is, that ${\mathcal {L}}$ is a hypoelliptic sub-Laplacian, up to a sign. Indeed, if $\pi _0$ is the projection of G onto its abelianization, then $\text {d}\pi _0({\mathcal {L}}_A)$ is a Rockland family by Proposition 3.12, so that ${\mathcal {F}}(\text {d}\pi _0({\mathcal {L}}_A))$ vanishes only at 0. Since $\text {d}\pi _0(T_k)=0$ for every $k=1,\dots , n_2$, this implies that q is non-degenerate and either positive or negative; hence, ${\mathcal {L}}$ is a hypoelliptic sub-Laplacian, up to a sign. We may then assume that q is positive and non-degenerate.

We shall endow ${\mathfrak {g}}$ with a scalar product for which ${\mathfrak {g}}_1$ and ${\mathfrak {g}}_2$ are orthogonal, and which induces ${\widehat{q}}$ on ${\mathfrak {g}}_1$. Then, we may endow ${\mathfrak {g}}$ with the translation-invariant measure ${\mathcal {H}}^n$ (the n-dimensional Hausdorff measure); up to a normalization, we may then assume that$(\exp _G)_*({\mathcal {H}}^n)$is the chosen Haar measure onG. We shall endow ${\mathfrak {g}}_2^*$ with the scalar product induced by that of ${\mathfrak {g}}_2$, and then with the corresponding Lebesgue measure.

Definition 7.1

Define

$$\begin{aligned} J_{q, \omega }{:}{=}\text {d}_{q}\circ \text {d}_{B_\omega }:{\mathfrak {g}}_1 \rightarrow {\mathfrak {g}}_1 \end{aligned}$$

for every $\omega \in {\mathfrak {g}}_2^*$, and define $d{:}{=}\min _{\omega \in {\mathfrak {g}}_2^*}\dim \ker \text {d}_{B_\omega }$, so that $d>0$ since G is not an $MW^+$ group. We denote by W the set of $\omega \in {\mathfrak {g}}_2^*$ such that $\dim \ker \text {d}_{B_\omega }>d$. Define $n_1{:}{=}\frac{1}{2}(\dim {\mathfrak {g}}_1-d)$, and observe that $n_1=0$ if and only if G is abelian.

Lemma 7.2

The set W is an algebraic variety.

Proof

Define $p_{\omega }$ so that $X^d p_{\omega }(X)$ is the characteristic polynomial of $-J_{q,\omega }^2$. Then, W is the zero locus of the polynomial mapping $\omega \mapsto p_{\omega }(0)$, so that it is an algebraic variety. $\square$

Proposition 7.3

Assume that $n_2>0$. Then, there are a non-empty Zariski open subset $\Omega$ of ${\mathfrak {g}}_2^*\setminus W$, a positive integer ${\overline{h}}$, and three continuous mappings

$$\begin{aligned}&\mu :{\mathfrak {g}}_2^* \rightarrow \mathbb {R}_+^{\overline{h}}&P:\Omega \rightarrow {\mathcal {L}}({\mathfrak {g}}_1)^{\overline{h}}&P_0:{\mathfrak {g}}_2^*\setminus W\rightarrow {\mathcal {L}}({\mathfrak {g}}_1) \end{aligned}$$

such that the following hold:

• $\mu$ is homogeneous of degree 1, while P and $P_0$ are homogeneous of degree 0;

• $\mu$ is analytic on $\Omega$ and $\mu (\Omega )\subseteq (\mathbb {R}_+^*)^{\overline{h}}$, while P and $P_0$ are analytic on their domains;

• $\mu _h(\omega )\ne \mu _{h'}(\omega )$ for every $h,h'=1,\dots , {\overline{h}}$ such that $h\ne h'$ and for every $\omega \in \Omega$;

• for every $h=0,\dots , {\overline{h}}$ and for every $\omega \in \Omega$ (for every $\omega \in {\mathfrak {g}}_2^*\setminus W$, if $h=0$), $P_h(\omega )$ is a $B_\omega$- and ${\widehat{q}}$-self-adjoint projector of ${\mathfrak {g}}_1$;

• if $h=1,\dots , {\overline{h}}$, then ${{\,\mathrm{Tr}\,}}P_h$ is constant;

• $\sum _{h=0}^{{\overline{h}}} P_h(\omega )=I_{{\mathfrak {g}}_1}$ and $\sum _{h=1}^{{\overline{h}}} \mu _h(\omega ) P_h(\omega )= |J_{q, \omega } |$ for every $\omega \in \Omega$;

• $P_0(\omega )({\mathfrak {g}}_1)=\ker \text {d}_{B_\omega }$ for every $\omega \in {\mathfrak {g}}_2^*\setminus W$.

This result is a consequence of [51, Lemmas 4 and 5]. Make use of the arguments of [44, Sects. 1.3–4 and Sect. 5.1 of Chapter II] to prove the analyticity of $P_0$.

Definition 7.4

We define $\Omega$, ${\overline{h}}$, $\mu$, P, and $P_0$ as in Proposition 7.3. In addition, we define $\mathbf{n} _\mathbf{1 }=(n_{1,h})_{h=1,\dots ,{\overline{h}}}$ so that $n_{1,h}$ is the constant value of ${{\,\mathrm{Tr}\,}}P_h$ for every $h=1,\dots , \overline{h}$. We define $\mathbf 1 _{{\overline{h}}}$ as the element of $\mathbb {R}^{{\overline{h}}}$ whose components are all equal to 1.

We shall often identify $\mu (\omega )$ with the linear mapping

$$\begin{aligned} \mathbb {R}^{\overline{h}} \ni \lambda \mapsto \sum _{h=1}^{\overline{h} }\mu _h(\omega ) \lambda _h\in \mathbb {R}\end{aligned}$$

for every $\omega \in {\mathfrak {g}}_2^*$.

With the above notation, we have $\mu (\omega )(\mathbf{n} _\mathbf{1 })=\sum _{h=1}^{\overline{h} }\mu _h(\omega ) n_{1,h}$. Observe that the index 1 in $\mathbf{n} _\mathbf{1 }$ refers to the first layer ${\mathfrak {g}}_1$, just as the index 2 in $n_2$ refers to the second layer ${\mathfrak {g}}_2$.

Recall that a subset of a (finite-dimensional) affine space V is semialgebraic if it belongs to the algebra of subsets of V generated by the sets of the form $P^{-1}(]0,\infty [)$, where P is a polynomial on V. A mapping between two semialgebraic sets is semialgebraic if its graph is semialgebraic. See [22] for a more detailed exposition of the basic theory of semialgebraic sets and semialgebraic mappings.

Corollary 7.5

The function $\omega \mapsto \mu (\omega )(\mathbf{n} _\mathbf{1 })$ is a semialgebraic norm on ${\mathfrak {g}}_2^*$ which is analytic on ${\mathfrak {g}}_2^*\setminus W$.

Proof

Observe that, denoting by $\Vert \,\cdot \, \Vert _1$ the trace-norm,

$$\begin{aligned} 2\mu (\omega )(\mathbf{n} _\mathbf{1 })= \Vert J_{q , \omega } \Vert _1=\Vert J_{q,\omega }+P_0(\omega ) \Vert _1-d \end{aligned}$$

for every $\omega \in {\mathfrak {g}}_2^*$, and that the linear mapping $\omega \mapsto J_{q, \omega }$ is one-to-one since G is stratified. Therefore, all assertions follow except for semialgebraicity. Now, denote by C the cone of positive endomorphisms of ${\mathfrak {g}}_1$, and observe that C is semialgebraic since it is the image of the polynomial mapping ${\mathcal {L}}({\mathfrak {g}}_1)\ni T\mapsto T^* T\in {\mathcal {L}}({\mathfrak {g}}_1)$ (cf. [22, Corollary 2.4]). In addition, the mapping $C\ni T\mapsto \sqrt{T}\in C$ is semialgebraic, since its graph is the set of $(T_1,T_2)\in C\times C$ which solve the polynomial equation $T_1-T_2^2=0$. Therefore, the function ${\mathcal {L}}({\mathfrak {g}}_1)\ni T\mapsto \Vert T \Vert _1={{\,\mathrm{Tr}\,}}(\sqrt{T^* T})$ is semialgebraic, since it is the composite of semialgebraic mappings (cf. [22, Corollary 2.9]). Hence, the function $\omega \mapsto \mu (\omega )(\mathbf{n} _\mathbf{1 })$ is semialgebraic, since it is the composite of the linear mapping $\omega \mapsto J_{q,\omega }$ and the semialgebraic function $\Vert \,\cdot \, \Vert _1$ (cf. [22, Corollary 2.9] again). $\square$

Definition 7.6

By an abuse of notation, we shall denote by (x, t) the elements of G, where $x\in {\mathfrak {g}}_1$ and $t\in {\mathfrak {g}}_2$, thus identifying (x, t) with $\exp _G(x,t)$. For every $x\in {\mathfrak {g}}_1$ and for every $\omega \in {\mathfrak {g}}_2^*\setminus W$, we define

$$\begin{aligned} x_0(\omega ){:}{=}P_0(\omega ) (x), \end{aligned}$$

while, for every $\omega \in \Omega$ and for every $h=1,\dots , \overline{h}$,

$$\begin{aligned} x_h(\omega ){:}{=}\sqrt{\mu _h(\omega )}P_h(\omega ) (x). \end{aligned}$$

By an abuse of notation, we shall write $x(\omega )$ instead of $\sum _{h=1}^{{\overline{h}}}x_h(\omega )=\sqrt{ |J_{q,\omega } | }(x)$, so that $|x(\omega ) |^2= \sum _{h=1}^{\overline{h} } |x_h(\omega ) |^2=\langle |J_{q,\omega } |x\vert x\rangle$.

Proposition 7.7

The mapping

$$\begin{aligned} {\mathfrak {g}}_1 \times \Omega \ni (x,\omega )\mapsto x(\omega ) \end{aligned}$$

extends uniquely to a continuous semialgebraic function on ${\mathfrak {g}}_1\times {\mathfrak {g}}_2^*$ which is analytic on ${\mathfrak {g}}_1 \times ({\mathfrak {g}}_2^*\setminus W)$.

Proof

Observe that, for every $\omega \in {\mathfrak {g}}_2^*$, $-J_{q,\omega }^2=J_{q,\omega }^*J_{q,\omega }$ is positive, and that

$$\begin{aligned} -J_{q,\omega }^2+P_0(\omega ) \end{aligned}$$

is positive and non-degenerate as long as $\omega \not \in W$. Therefore, the mapping

$$\begin{aligned} \omega \mapsto \root 4 \of {-J_{q,\omega }^2}=\root 4 \of {-J_{q,\omega }^2+P_0(\omega )}- P_0(\omega )\in {\mathcal {L}}({\mathfrak {g}}_1) \end{aligned}$$

is continuous and semialgebraic on ${\mathfrak {g}}_2^*$ and analytic on ${\mathfrak {g}}_2^*\setminus W$ thanks to [18, Proposition 10 of Chapter I, Sect. 4, No. 8] (argue as in the proof of Corollary 7.5).⁹ The conclusion follows, since

$$\begin{aligned} \root 4 \of {-J_{q,\omega }^2}(x)= \sum _{h=1}^{\overline{h}} x_h(\omega ) \end{aligned}$$

for every $\omega \in \Omega$ and for every $x\in {\mathfrak {g}}_1$. $\square$

Definition 7.8

Define $G_\omega$, for every $\omega \in {\mathfrak {g}}_2^*$, as the quotient of G by its normal subgroup $\exp _G(\ker \omega )$.

Then, $G_0$ is the abelianization of G, and we identify it with ${\mathfrak {g}}_1$. If $\omega \ne 0$, we shall identify $G_\omega$ with ${\mathfrak {g}}_1 \oplus \mathbb {R}$, endowed with the product

$$\begin{aligned} (x_1,t_1) (x_2,t_2){:}{=}\left( x_1+x_2, t_1+t_2+\frac{1}{2}B_\omega (x_1,x_2) \right) \end{aligned}$$

for every $x_1,x_2\in {\mathfrak {g}}_1$ and for every $t_1,t_2\in \mathbb {R}$. Hence,

$$\begin{aligned} \pi _\omega (x,t)=(x,\omega (t)) \end{aligned}$$

for every $(x,t)\in G$.

Definition 7.9

For every $\omega \in {\mathfrak {g}}_2^*\setminus W$, define $|{{\,\mathrm{Pf}\,}}(\omega ) |{:}{=}\prod _{h=1}^{\overline{h}} \mu _h(\omega )^{n_{1,h}}$, the Pfaffian of $\omega$ (cf. [3]).

We are now in position to find the Plancherel measure and the integral kernel associated with ${\mathcal {L}}_A$. This is done by means of the explicit knowledge of the Plancherel and inversion formulae of G (cf. [3]).

Let us first describe the Plancherel measure on ${\widehat{G}}$; we follow the construction of the Plancherel measure of [3] as in [47, 4.4.1]. Take $\omega \in \Omega$ and $\tau \in P_0(\omega )({\mathfrak {g}}_1)$. Then, Proposition 6.5 shows that there is a basis $(X_{\omega ,1},\dots , X_{\omega ,n_1},Y_{\omega ,1},\dots , Y_{\omega ,n_1}, U_1,\dots , U_d)$ of ${\mathfrak {g}}_1$ such that $(U_1,\dots , U_d)$ is a basis of the radical $P_0(\omega )({\mathfrak {g}}_1)$ of $B_\omega$, such that $B_\omega (X_{\omega ,h},Y_{\omega ,k})=\delta _{h,k}$ and $B_\omega (X_{\omega ,h},X_{\omega ,k})=B_\omega (Y_{\omega ,h},Y_{\omega ,k})=0$ for every $h,k=1,\dots ,n_1$, and such that

$$\begin{aligned} {\mathcal {L}}=-\sum _{k=1}^{n_1} {\widetilde{\mu }}_{k}(\omega )(X_{\omega ,k}^2+Y_{\omega ,k}^2)-\sum _{h=1}^d U_h^2, \end{aligned}$$

where ${\widetilde{\mu }}_{1}(\omega ),\dots , {\widetilde{\mu }}_{n_1}(\omega )$ are the eigenvalues $\mu _h(\omega )$, $h=1,\dots , {\overline{h}}$, each repeated $n_{1,h}$-times. Define H as the space of holomorphic functions in $L^2(\mathbb {C}^{n_1}, \nu )$, where $\nu {:}{=}e^{-2|\,\cdot \, |^2}\cdot {\mathcal {H}}^{2 n_1}$. Define $\pi _{\omega ,\tau }$ as the unique continuous unitary representation of G in the hilbertian space H such that

$$\begin{aligned} \text {d}\pi _{\omega ,\tau }(X_{\omega ,k}+i Y_{\omega ,k}) f(z)= 2 z_k f(z) \quad \text {and}\quad \text {d}\pi _{\omega ,\tau }(X_{\omega ,k}-i Y_{\omega ,k}) f(z)= -\partial _{z_k} f(z) , \end{aligned}$$

for every $k=1,\dots ,n_1$, for every $f\in C^\infty (\pi _{\omega ,\tau })$, and for every $z\in \mathbb {C}^{n_1}$, while

$$\begin{aligned} \text {d}\pi _{\omega ,\tau }(T_\ell ) =\omega (T_\ell ) I_H \qquad \text {and} \qquad \text {d}\pi _{\omega ,\tau }(U_h) = \langle \tau \vert U_h\rangle I_H \end{aligned}$$

for every $\ell =1,\dots ,n_2$ and for every $h=1,\dots , d$, where $I_H$ is the identity of H. Thus, $\pi _{\omega ,\tau }$ is a version of the ‘Bargmann(–Fock)’ representation, cf. [43]; in particular, it is irreducible. Let $\Sigma$ be the (closed) subset $\bigcup _{\omega \in \Omega }[\lbrace \omega \rbrace \times P_0(\omega )({\mathfrak {g}}_1)]$ of $\Omega \times {\mathfrak {g}}_1$, and let $\nu _\Sigma$ be the (positive Radon) measure $\frac{1}{(2\pi )^{n_1+n_2}}\int _{\Omega } |{{\,\mathrm{Pf}\,}}(\omega ) |\delta _{\omega }\otimes (\chi _{P_0(\omega )({\mathfrak {g}}_1)}\cdot {\mathcal {H}}^d)\,\text {d}\omega$ on $\Sigma$. Then, the mapping $p:\Sigma \ni (\omega ,\tau )\mapsto [\pi _{\omega ,\tau }]\in {\widehat{G}}$ is one-to-one (but not onto), and the Plancherel measure $\nu _{{\widehat{G}}}$ is $p_*(\nu _\Sigma )$ (cf. [3, Sect. 2]). In particular, for every $f\in L^1(G)\cap L^2(G)$,¹⁰

$$\begin{aligned} \Vert f \Vert _2^2= \frac{1}{(2\pi )^{n_1+n_2}} \int _{\Omega } \int _{P_0(\omega )({\mathfrak {g}}_1)} \Vert \pi _{\omega ,\tau } (f) \Vert _2^2\,\text {d}\tau |{{\,\mathrm{Pf}\,}}(\omega ) |\,\text {d}\omega . \end{aligned}$$

Now, fix $(\omega ,\tau )\in \Sigma$, and define $w_{\gamma '}(z){:}{=}\sqrt{\frac{2^{n_1+2|\gamma ' |}}{\pi ^{n_1}\gamma ' !}}z^{\gamma '}$ for every $z\in \mathbb {C}^{n_1}$ and for every $\gamma '\in \mathbb {N}^{n_1}$, so that $(w_{\gamma '})_{\gamma '\in \mathbb {N}^{n_1}}$ is an orthonormal basis of H (cf. [43]), and

$$\begin{aligned} \text {d}\pi _{\omega ,\tau }({\mathcal {L}}) w_{\gamma '}= (|\tau |^2+{\widetilde{\mu }}(\omega )(\mathbf 1 _{n_1}+2\gamma ')) w_{\gamma '} \end{aligned}$$

for every $\gamma '\in \mathbb {N}^{n_1}$. Therefore, the preceding considerations and the definitions of $\mu (\omega )$ and ${\widetilde{\mu }}(\omega )$ show that there is a commutative family $(p_{\omega ,\gamma })_{\gamma \in \mathbb {N}^{{\overline{h}}}}$ of self-adjoint projectors of H such that $I_H=\sum _{\gamma } p_{\omega ,\gamma }$, such that ${{\,\mathrm{Tr}\,}}(p_{\omega ,\gamma })= \left( {\begin{array}{c}\mathbf{n} _\mathbf{1 }+\gamma -\mathbf 1 _{{\overline{h}}} \\ \gamma \end{array}}\right)$, and such that

$$\begin{aligned} \text {d}\pi _{\omega ,\tau }({\mathcal {L}}) p_{\omega ,\gamma } = (|\tau |^2+\mu (\omega )(\mathbf{n} _\mathbf{1 }+2\gamma )) p_{\omega ,\gamma } \end{aligned}$$

for every $\gamma \in \mathbb {N}^{{\overline{h}}}$.

We shall now introduce some notation to simplify the forthcoming formulae.

Definition 7.10

For every $m,\gamma \in \mathbb {N}$, denote by $\Lambda ^m_\gamma (X)=\sum _{j=0}^\gamma \left( {\begin{array}{c}\gamma +m\\ \gamma -j\end{array}}\right) \frac{(-X)^j}{j!}$ the $\gamma$-th Laguerre polynomial of order m. Define, in addition,

$$\begin{aligned} \Phi _d:\mathbb {R}_+\ni x \mapsto \Gamma \left( \frac{d}{2} \right) \frac{ J_{\frac{d}{2}-1}(x)}{ \left( \frac{x}{2} \right) ^{\frac{d}{2}-1} }=\Gamma \left( \frac{d}{2} \right) \sum _{k\in \mathbb {N}} \frac{(-1)^k x^{2 k} }{ 4^{ k} k! \Gamma \left( k+\frac{d}{2} \right) } , \end{aligned}$$

where $J_{\frac{d}{2}-1}$ is the Bessel function (of the first kind) of order $\frac{d}{2}-1$.

Then,

$$\begin{aligned} {{\,\mathrm{Tr}\,}}(\pi _{\omega ,\tau }(x,t)^* p_{\omega ,\gamma } )= e^{-\frac{1}{4}|x(\omega ) |^2- i \langle \tau \vert x_0(\omega )\rangle -i \omega (t)} \prod _{h=1}^{\overline{h}} \Lambda ^{n_{1,h}-1}_{\gamma _{h}}\left( \frac{1}{2} |x_{h}(\omega ) |^2 \right) \end{aligned}$$

by [43, Proposition 2] and [26, 10.12 (41)], while

The following proposition is a consequence of the preceding considerations and of [48, Proposition 3.7].

Proposition 7.11

For every $\varphi \in C_c(\sigma ({\mathcal {L}}_A))$,

$$\begin{aligned} \begin{aligned} \int _{\sigma ({\mathcal {L}}_A)} \varphi \,\text {d}\beta _{{\mathcal {L}}_A}&= \frac{\pi ^{\frac{d}{2}}}{(2\pi )^{n_1+n_2+d} \Gamma \left( \frac{d}{2}\right) } \sum _{\gamma \in \mathbb {N}^{{\overline{h}}}} \left( {\begin{array}{c}\mathbf{n} _\mathbf{1 }+\gamma -\mathbf 1 _{{\overline{h}}} \\ \gamma \end{array}}\right) \times \\&\quad \times \int _{ \mathbb {R}_+\times {\mathfrak {g}}_2^* } \varphi (\lambda +\mu (\omega )(\mathbf{n} _\mathbf{1 }+2 \gamma ), \omega (\mathbf{T} )) |\lambda |^{\frac{d}{2}-1} |{{\,\mathrm{Pf}\,}}(\omega ) | \,\text {d}(\lambda ,\omega ). \end{aligned} \end{aligned}$$

In addition,

$$\begin{aligned} \begin{aligned}&\chi _{{\mathcal {L}}_A}((\lambda , \omega (\mathbf{T} )),(x,t))\\&\quad =\frac{1}{c_{\lambda ,\omega }} \sum _{\begin{array}{c} \gamma \in \mathbb {N}^{\overline{h}}\\ \mu (\omega )(\mathbf{n} _\mathbf{1 }+2\gamma ) < \lambda \end{array}} \left( \lambda - \mu (\omega )(\mathbf{n} _\mathbf{1 }+2\gamma ) \right) ^{\frac{d}{2}-1} e^{-\frac{1}{4}|x(\omega ) |^2- i \langle \tau \vert x_0(\omega )\rangle -i \omega (t)}\times \\&\qquad \times \prod _{h=1}^{\overline{h}} \Lambda ^{n_{1,h}-1}_{\gamma _{h}}\left( \frac{1}{2} |x_h(\omega ) |^2 \right) \Phi _d\left( \sqrt{ \lambda - \mu (\omega )(\mathbf{n} _\mathbf{1 }+2\gamma )}|x_0(\omega ) |\right) \end{aligned} \end{aligned}$$

$(\beta _{{\mathcal {L}}_A}\otimes \nu _G)$-almost everywhere, where $c_{\lambda ,\omega }{:}{=}\sum _{\mu (\omega )(\mathbf{n} _\mathbf{1 }+2\gamma ) < \lambda } \left( \lambda - \mu (\omega )(\mathbf{n} _\mathbf{1 }+2\gamma ) \right) ^{\frac{d}{2}-1}$.

Remark 7.12

Let $T'_1,\dots , T'_n$ be n homogeneous elements of the centre ${\mathfrak {z}}$ of ${\mathfrak {g}}$. Let us show that the study of the family $({\mathcal {L}},-i T_1',\dots , - i T_n')$ can be reduced to that of the families of the form considered above on suitable 2-step stratified groups.

Notice that we may assume that there is $n'\in \lbrace 0,\dots , n \rbrace$ such that $T_j'\in {\mathfrak {g}}_2$ if and only if $j\leqslant n'$; let ${\mathfrak {g}}''$ be the vector subspace of ${\mathfrak {g}}$ generated by $T_{n'+1}',\dots , T_{n}'$, and observe that ${\mathfrak {g}}''\subseteq {\mathfrak {g}}_1$ by homogeneity. Let ${\mathfrak {g}}'_1$ be the ${\widehat{q}}$-orthogonal complement of ${\mathfrak {g}}''$ in ${\mathfrak {g}}_1$; define ${\mathfrak {g}}'{:}{=}{\mathfrak {g}}'_1\oplus {\mathfrak {g}}_2$. Then, ${\mathfrak {g}}$ is the direct sum of its ideals ${\mathfrak {g}}'$ and ${\mathfrak {g}}''$. Let $G'$ and $G''$ be the Lie subgroups of G corresponding to ${\mathfrak {g}}'$ and ${\mathfrak {g}}''$, and let ${\mathcal {L}}'$ and ${\mathcal {L}}''$ be the sub-Laplacians on $G'$ and $G''$, respectively, corresponding to the restrictions of q to ${\mathfrak {g}}_1'^*$ and ${\mathfrak {g}}''^*$, respectively. By an abuse of notation, then, ${\mathcal {L}}={\mathcal {L}}'+{\mathcal {L}}''$, so that the family $({\mathcal {L}}, - i T_1',\dots , - i T_n')$ is (algebraically) equivalent to the family $({\mathcal {L}}', - i T'_1,\dots , - i T'_n)$. Now, the family $(- i T'_{n'+1},\dots , - i T'_n)$ on $G''$ satisfies property (RL) by classical Fourier analysis. Therefore, Theorem 4.4 implies that the family $({\mathcal {L}}, - i T_1',\dots , - i T_n')$ satisfies property (RL) if the family $({\mathcal {L}}', - i T_1',\dots , - i T_{n'}')$ does.¹¹ Since this latter family is equivalent to a family of the form $({\mathcal {L}}',-i T_1,\dots , - i T_{n_2'})$ for some $n_2'$ and for some choice of the basis $T_1,\dots , T_{n_2}$ of ${\mathfrak {g}}_2$, our assertion follows. Notice, however, that $G'$ may be an $MW^+$ group; we shall deal with $MW^+$ groups in a future paper.

Similar arguments apply to property (S) and the continuity of the integral kernel.

8 Property (RL)

In this section we shall present several sufficient conditions for the validity of property (RL); we keep the notation of Sect. 7. First of all, we observe that the spectrum of ${\mathcal {L}}_A$ is a closed convex semialgebraic cone, thanks to Corollary 7.5 and [22, Corollary 2.9]. In addition, we can basically ignore the Laguerre polynomials of higher order which appear in the Fourier inversion formula, thanks to (a suitable extension of) [53, Proposition 5.4]. Indeed, with reference to the notation of Sect. 7, the ‘ground state,’ that is, the first eigenvalue of $\text {d}\pi _{\omega ,\tau }({\mathcal {L}}_A)$, is sufficient to cover the whole of $\sigma ({\mathcal {L}}_A)$, as $\omega$ and $\tau$ vary. This fact leads to significant simplifications, as the basic Lemma 8.1 shows.

We need to distinguish between the ‘full’ family ${\mathcal {L}}_A$, for which we can prove continuity of the multipliers only on a dense subset of the spectrum in full generality (cf. Lemma 8.1), and the ‘partial’ family $({\mathcal {L}}, (-i T_1,\dots , - i T_{n_2'}))$ for $n_2'<n_2$, for which by means of a deeper analysis we are able to prove property (RL) in full generality (cf. Theorem 8.5). This latter result requires to deal with Radon measures defined on Polish spaces which are not necessarily locally compact.

Concerning the ‘full’ family ${\mathcal {L}}_A$, as we observed above, we can prove in full generality that every integrable kernel corresponds to a multiplier which is continuous on a dense subset of the spectrum. In addition, we can prove that property (RL) holds in the following cases: when $P_0$ extends to a continuous function on ${\mathfrak {g}}_2^*\setminus \lbrace 0 \rbrace$, for example when $W=\lbrace 0 \rbrace$ or when G is the product of an $MW^+$ group and a non-trivial abelian group (cf. Theorem 8.2); when G is a free 2-step stratified group on an odd number of generators (cf. Theorem 8.4). In both cases, we make use of the simplified ‘inversion formula’ for ${\mathcal {K}}_{{\mathcal {L}}_A}$ which is available in this case; in the second case, we employ the simple structure of free groups to prove that the $L^1$ kernels are invariant under sufficiently many linear transformations in order that the above-mentioned inversion formula give rise to a continuous multiplier.

Lemma 8.1

Take $f\in L^1_{{\mathcal {L}}_A}(G)$. Then, ${\mathcal {M}}_{{\mathcal {L}}_A}(f)$ has a representative which is continuous on

$$\begin{aligned} \lbrace ( \mu _\omega (\mathbf{n} _\mathbf{1 } ) ,\omega (\mathbf{T} ) ):\omega \in {\mathfrak {g}}_2^* \rbrace \cup \lbrace ( \lambda ,\omega (\mathbf{T} ) ):\omega \in {\mathfrak {g}}_2^*\setminus W ,\lambda \geqslant \mu _\omega (\mathbf{n} _\mathbf{1 }) \rbrace . \end{aligned}$$

Proof

Fix a representative m of ${\mathcal {M}}_{{\mathcal {L}}_A}(f)$, and keep the notation of the last part of Sect. 7. Arguing as in [53, Proposition 5.4], we see that there is a negligible subset $N_1$ of ${\mathfrak {g}}_2^*$ such that for every $\omega \in {\mathfrak {g}}_2^*\setminus N_1$ there is negligible subset $N_{2,\omega }$ of $P_0(\omega )({\mathfrak {g}}_1)$ such that

$$\begin{aligned} \pi _{\omega , \tau }^*(f)= m(\text {d}\pi _{\omega , \tau }({\mathcal {L}}_A)) \end{aligned}$$

for every $\tau \in P_0(\omega )({\mathfrak {g}}_1)\setminus N_{2,\omega }$. Notice that we may assume that $W\subseteq N_1$. Therefore, for every $\omega \in {\mathfrak {g}}_2^*\setminus N_1$ and for every $\tau \in P_0(\omega )({\mathfrak {g}}_1)\setminus N_{2,\omega }$,

$$\begin{aligned} \begin{aligned} m(|\tau |^2+\mu (\omega )(\mathbf{n} _\mathbf{1 }), \omega (\mathbf{T} ))&=\frac{1}{{{\,\mathrm{Tr}\,}}p_{\omega ,0}} {{\,\mathrm{Tr}\,}}(m(\text {d}\pi _{\omega , \tau }({\mathcal {L}}_A)) p_{\omega ,0})\\&= \int _{G} f(x,t) e^{- \frac{1}{4}|x(\omega ) |^2- i \omega (t)-i \langle \tau | x_0(\omega )\rangle }\,\text {d}(x,t). \end{aligned} \end{aligned}$$

Now, for every $\omega \in {\mathfrak {g}}_2^*\setminus N_1$ there is negligible subset $N_{3,\omega }$ of $\mathbb {R}_+^*$ such that, for every $\lambda \in \mathbb {R}_+^*\setminus N_{3,\omega }$, we have ${\mathcal {H}}^{d-1}\big (\partial B_{P_0(\omega )({\mathfrak {g}}_1)}\big (0,\sqrt{\lambda } \big )\cap N_{2,\omega }\big )=0$. Therefore, for every $\omega \in {\mathfrak {g}}_2^*\setminus N_1$ and for every $\lambda \in \mathbb {R}_+^*\setminus N_{3,\omega }$,

Now, the mapping

$$\begin{aligned} (\lambda ,\omega ) \mapsto \int _G f(x,t) e^{ -\frac{1}{4}|x(\omega ) |^2-i \omega (t) } \Phi _d\left( \sqrt{\lambda } |x_0(\omega ) | \right) \,\text {d}(x,t) \end{aligned}$$

is continuous on $[ \mathbb {R}_+\times ({\mathfrak {g}}_2^*\setminus W)]\cup [\lbrace 0 \rbrace \times {\mathfrak {g}}_2^*]$ by Proposition 7.7, so that by means of Tonelli’s theorem we see that it induces a representative of m which satisfies the conditions of the statement. $\square$

Theorem 8.2

Assume that $P_0$ can be extended to a continuous function on ${\mathfrak {g}}_2^*\setminus \lbrace 0 \rbrace$. Then, ${\mathcal {L}}_A$ satisfies property (RL).

Notice that, by polarization, $P_0$ has a continuous extension to ${\mathfrak {g}}_2^*\setminus \lbrace 0 \rbrace$ if and only if $|P_0(x) |$ has a continuous extension to ${\mathfrak {g}}_2^*\setminus \lbrace 0 \rbrace$ for every $x\in {\mathfrak {g}}_1$.

In addition, observe that the hypotheses of the proposition hold in the following situations:

• when $W=\lbrace 0 \rbrace$, for example when G is the free 2-step nilpotent group on three generators;

• when $P_0$ is constant on ${\mathfrak {g}}_2^*\setminus W$, for example when $G=G'\times \mathbb {R}^d$ for some $MW^+$ group $G'$, such as a product of Heisenberg groups.

Proof

1 Keep the notation of the proof of Lemma 8.1. Assume first that $n_2=1$, so that $W=\lbrace 0 \rbrace$. In addition, $\ker \text {d}_{\sigma _\omega }=\ker \text {d}_{\sigma _{-\omega }}$ for every $\omega \in {\mathfrak {g}}_2^*$, so that $P_0$ is constant on ${\mathfrak {g}}_2^*\setminus \lbrace 0 \rbrace$. The computations of the proof of Lemma 8.1 then lead to the conclusion.

2 Denote by ${{\widetilde{P}}}_0$ the continuous extension of $P_0$ to ${\mathfrak {g}}_2^*\setminus \lbrace 0 \rbrace$; observe that ${{\widetilde{P}}}_0(\omega )$ is a self-adjoint projector of ${\mathfrak {g}}_1$ of rank d for every non-zero $\omega \in {\mathfrak {g}}_2^*$. Take $f\in L^1_{{\mathcal {L}}_A}(G)$ and define, for every non-zero $\omega \in {\mathfrak {g}}_2^*$ and for every $\lambda \geqslant 0$,

$$\begin{aligned} m(\mu (\omega )(\mathbf{n} _\mathbf{1 })+\lambda , \omega (\mathbf{T} )){:}{=}\int _G f(x,t) e^{ -\frac{1}{4}|x(\omega ) |^2-i \omega (t) } \Phi _d\left( \sqrt{\lambda } |{{\widetilde{P}}}_0(\omega )(x) | \right) \,\text {d}(x,t), \end{aligned}$$

so that $f={\mathcal {K}}_{{\mathcal {L}}_A}(m)$ as in the proof of Lemma 8.1. Then, m is clearly continuous con $\sigma ({\mathcal {L}}_A)\setminus (\mathbb {R}\times \lbrace 0 \rbrace ^{n_2})$, and $m(\mu (r \omega )(\mathbf{n} _\mathbf{1 })+\lambda , r \omega (\mathbf{T} ))$ converges to

$$\begin{aligned} \int _G f(x,t) \Phi _d\left( \sqrt{\lambda } |{{\widetilde{P}}}_0(\omega )(x) | \right) \,\text {d}(x,t) \end{aligned}$$

as $r\rightarrow 0^+$, uniformly as $\omega$ runs through the unit sphere S of ${\mathfrak {g}}_2^*$. Therefore, it will suffice to prove that the above integrals do not depend on $\omega \in S$ for every $\lambda \geqslant 0$. Indeed, Proposition 3.12 implies that, for every $\omega \in S$ (cf. Definition 7.8),

$$\begin{aligned} (\pi _\omega )_*(f)={\mathcal {K}}_{\text {d}\pi _\omega ({\mathcal {L}}_A)}(m). \end{aligned}$$

Now, 1 above implies that the family $\text {d}\pi _\omega ({\mathcal {L}}_A)$ satisfies property (RL). Then, Proposition 3.12 implies that

$$\begin{aligned} (\pi _0)_*(f)\in L^1_{\text {d}\pi _0({\mathcal {L}}_A)}(G_0); \end{aligned}$$

in addition, $\text {d}\pi _0({\mathcal {L}}_A)$ is identified with $(\Delta , 0,\dots ,0)$, where $\Delta$ is the (positive) Laplacian associated with the scalar product ${\widehat{q}}$ on ${\mathfrak {g}}_1$. Then,

$$\begin{aligned} \int _G f(x,t) \Phi _d\left( \sqrt{\lambda } |{{\widetilde{P}}}_0(\omega )(x) | \right) \,\text {d}(x,t) =\int _{{\mathfrak {g}}_1}(\pi _0)_*(f)(x) \Phi _d\left( \sqrt{\lambda } |{{\widetilde{P}}}_0(\omega )(x) | \right) \,\text {d}x, \end{aligned}$$

whence the assertion since $(\pi _0)_*(f)$ is rotationally invariant. $\square$

Remark 8.3

Let G be $\mathbb {H}^1\times \mathbb {R}$, where $\mathbb {H}^1$ is the three-dimensional Heisenberg group. If ${\mathcal {L}}$ is the standard sub-Laplacian on $\mathbb {H}^1$, T is a basis of the centre of the Lie algebra of $\mathbb {H}^1$, and $\Delta$ is the (positive) Laplacian on $\mathbb {R}$, then $({\mathcal {L}}+\Delta , i T)$ satisfies property (RL) by Theorem 8.2, but it is easily seen that its integral kernel does not admit any continuous representatives (cf. Proposition 7.11).

When G is a free group, we can remove the assumption that $P_0$ has a continuous extension.

Theorem 8.4

Assume that G is a free 2-step stratified group on an odd number of generators. Then, ${\mathcal {L}}_A$ satisfies property (RL).

Proof

Take $f\in L^1_{{\mathcal {L}}_A}(G)$; by Lemma 8.1, ${\mathcal {M}}_{{\mathcal {L}}_A}(f)$ has a representative m which is continuous on $\sigma ({\mathcal {L}}_A)\setminus (\mathbb {R}\times W)$. Now, (cf. Definition 7.8)

$$\begin{aligned} (\pi _\omega )_*(f)(x,t) =\int _{\omega (t')= t} f(x, t')\,\text {d}t' \end{aligned}$$

for almost every $(x,t)\in G_\omega$; in addition, Proposition 3.12 implies that $(\pi _\omega )_*(f)\in L^1_{\text {d}\pi _\omega ({\mathcal {L}}_A)}$ for every $\omega \in {\mathfrak {g}}_2^*\setminus W$. Further, if U is an isometry of ${\mathfrak {g}}_1$ which restricts to the identity on $(\ker \text {d}_{B_\omega })^\perp$, then $U\times I_\mathbb {R}$ is an automorphism of the Lie group $G_\omega$ which leaves $\text {d}\pi _\omega ({\mathcal {L}}_A)$ invariant, hence also $(\pi _\omega )_*(f)$.

Now, take $\omega \in W$ and an isometry U of $G_\omega$ which restricts to the identity on $(\ker \text {d}_{B_\omega })^\perp$. Since $\dim \ker \text {d}_{B_\omega }$ is odd, there must be some $v\in \ker \text {d}_{B_\omega }$ such that $U\cdot v=\pm v$. Let V be the orthogonal complement of $\mathbb {R}v$ in $\ker \text {d}_{B_\omega }$, so that V is U-invariant. Now, let $\sigma _V$ be a standard symplectic form on the hilbertian space V,¹² and define $\omega _p$, for every $p\in \mathbb {N}$, so that

$$\begin{aligned} B_{\omega _p}=B_\omega + 2^{-p} \sigma _V; \end{aligned}$$

this is possible since G is a free 2-step stratified group, so that $\lbrace B_{\omega '}:\omega '\in {\mathfrak {g}}_2^* \rbrace$ is the set of all skew-symmetric bilinear forms on ${\mathfrak {g}}_1$. Then, $\omega _p$ belongs to ${\mathfrak {g}}_2^*\setminus W$ and converges to $\omega$. In addition, $(\pi _{\omega _p})_*(f)$ is U-invariant thanks to the above remarks. Now, it is easily seen that $(\pi _{\omega _p})_*(f)$ converges to $(\pi _\omega )_*(f)$ in $L^1({\mathfrak {g}}_1 \oplus \mathbb {R})$, so that $(\pi _\omega )_*(f)$ is U-invariant.

Then, taking into account the fact that the limit points of $P_0(\omega ')$ as $\omega '\rightarrow \omega$ are self-adjoint projectors of ${\mathfrak {g}}_1$ onto some subspace of $\ker \text {d}_{B_\omega }$, as well as the arbitrariness of $\omega$, we see that the continuous mapping

$$\begin{aligned} m_1:\mathbb {R}_+\times ({\mathfrak {g}}_2^*\setminus W) \ni (\lambda ,\omega ) \mapsto \int _G f(x,t) e^{ -\frac{1}{4}|x(\omega ) |^2-i \omega (t) } \Phi _1\left( \sqrt{\lambda } |x_0(\omega ) | \right) \,\text {d}(x,t), \end{aligned}$$

extends to a continuous function on ${\mathfrak {g}}_2^*\times \mathbb {R}_+$. Now, clearly $m(\lambda , \omega (\mathbf{T} ))=m_1(\lambda - \mu _\omega (\mathbf{n} _\mathbf{1 }),\omega )$ for every $(\lambda , \omega (\mathbf{T} ))\in \sigma ({\mathcal {L}}_A)$; the assertion follows. $\square$

Theorem 8.5

Take $n_2'<n_2$. Then, the family $({\mathcal {L}}, (-i T_j)_{j=1,\dots ,n_2'})$ satisfies property (RL).

Proof

Define ${\mathcal {L}}'_{A'}=({\mathcal {L}}, (-i T_j)_{j=1,\dots , n_2'})$, and let $L:\sigma ({\mathcal {L}}_A)\rightarrow \sigma ({\mathcal {L}}'_{A'})$ be the unique linear mapping such that ${\mathcal {L}}'_{A'}=L({\mathcal {L}}_A)$. Until the end of the proof, we shall identify ${\mathfrak {g}}_2^*$ with $\mathbb {R}^{n_2}$ by means of the mapping $\omega \mapsto \omega (\mathbf{T} )$. In addition, define $X{:}{=}(\sigma ({\mathcal {L}}_A) \setminus W) \cup \partial \sigma ({\mathcal {L}}_A)$, so that X is a Polish space by [16, Theorem 1 of Chapter IX, §6, No. 1]. Let $\beta$ be the (Radon) measure induced by $\beta _{{\mathcal {L}}_A}$ on X, so that $\text {Supp}\left( \beta \right) =X$. Let $L'$ be the restriction of L to X. Since $\sigma ({\mathcal {L}}_A)$ is a convex cone by Corollary 7.5 and since W is $\beta _{{\mathcal {L}}_A}$-negligible, Proposition 10.3 implies that $\beta$ is $L'$-connected.

Now, Proposition 10.4 implies that $\beta$ has a disintegration $(\beta _{\lambda '})_{\lambda '\in \sigma ({\mathcal {L}}'_{A'})}$ such that $\beta _{\lambda '}$ is equivalent to $\chi _{L'^{-1}(\lambda ')}\cdot {\mathcal {H}}^{n_2-n_2'}$ for $\beta _{{\mathcal {L}}'_{A'}}$-almost every $\lambda '\in \sigma ({\mathcal {L}}'_{A'})$. Observe that $L^{-1}(\lambda ')\cap \sigma ({\mathcal {L}}_A)$ is a convex set of dimension $n_2-n_2'$ for $\beta _{{\mathcal {L}}'_{A'}}$-almost every $\lambda '\in \sigma ({\mathcal {L}}'_{A'})$. In addition, $W\cap L^{-1}(\lambda ')$ is an algebraic variety of dimension at most $n_2-n_2'-1$ for $\beta _{{\mathcal {L}}'_{A'}}$-almost every $\lambda '\in \sigma ({\mathcal {L}}'_{A'})$, for otherwise ${\mathcal {H}}^{n_2+1}(W)$ would be non-zero, which is absurd. Therefore, $\text {Supp}\left( \beta _{\lambda '}\right) =L'^{-1}(\lambda ')$ for $\beta _{{\mathcal {L}}'_{A'}}$-almost every $\lambda '\in \sigma ({\mathcal {L}}'_{A'})$.

Now, take $m_0\in L^\infty (\beta _{{\mathcal {L}}_A})$ so that ${\mathcal {K}}_{{\mathcal {L}}'_{A'}}(m_0)\in L^1(G)$. Let us prove that $m_0$ has a continuous representative. Indeed, Lemma 8.1 implies that there is a continuous function $m_1$ on X such that $m_0\circ L'=m_1$$\beta$-almost everywhere. Hence, Proposition 10.2 implies that there is a function $m_2:\sigma ({\mathcal {L}}'_{A'})\rightarrow \mathbb {C}$ such that $m_2\circ L'=m_1$. Since the mapping $L:\partial \sigma ({\mathcal {L}}_A)\rightarrow \sigma ({\mathcal {L}}'_{A'})$ is proper and onto, and since $\partial \sigma ({\mathcal {L}}_A)\subseteq X$, it follows that $m_2$ is continuous. The assertion follows (cf. [16, Corollary to Theorem 2 of Chapter IX, Sect. 4, No. 2]). $\square$

9 Property (S)

The results of this section are basically a generalization of the techniques employed in [4, 5]; we keep the notation of Sect. 7.

Theorem 9.2 applies, for example, to the free 2-step nilpotent group on three generators. Notice that we need to impose the condition $W=\lbrace 0 \rbrace$ since our methods cannot be used to infer any kind of regularity on $W\setminus \lbrace 0 \rbrace$; for example, in general our auxiliary functions $|x(\omega ) |^2$ and $P_0$ are not differentiable on W. Nevertheless, this does not mean that property (S) cannot hold when $W\ne \lbrace 0 \rbrace$; as a matter of fact, Theorem 9.3 shows that this happens for a product of free 2-step stratified groups on 3 generators and a suitable sub-Laplacian thereon.

In order to simplify the notation, we define ${\mathcal {S}}(G,{\mathcal {L}}_A){:}{=}{\mathcal {K}}_{{\mathcal {L}}_A}({\mathcal {S}}(\sigma ({\mathcal {L}}_A)))$.

We begin with a lemma which will allow us to get some ‘Taylor expansions’ of functions whose kernel transform belong to the Schwartz space, under suitable hypotheses. Its proof is modelled on a technique due to Geller [34, Theorem 4.4]. We state it in a slightly more general context.

Lemma 9.1

Let ${\mathcal {L}}'_{A'}$ be a Rockland family on a homogeneous group $G'$, and let $T'_1,\dots , T'_{n'}$ be a free family of elements of the centre of the Lie algebra ${\mathfrak {g}}'$ of $G'$. Let $\pi _1$ be the canonical projection of $G'$ onto its quotient by the normal subgroup $\exp _{G'}(\mathbb {R}T'_1)$, and assume that the following hold:

• $({\mathcal {L}}'_{A'}, i T'_1,\dots , i T'_{n'} )$ satisfies property (RL);

• $\text {d}\pi _1({\mathcal {L}}'_{A'}, i T'_{2},\dots , i T'_{n'})$ satisfies property (S).

Take $\varphi \in {\mathcal {S}}_{({\mathcal {L}}'_{A'}, i T'_1,\dots , i T'_{n'} )}(G')$. Then, there are two families $({\widetilde{\varphi }}_\gamma )_{\gamma \in \mathbb {N}^{n'}}$ and $(\varphi _\gamma )_{\gamma \in \mathbb {N}^{n'}}$ of elements of ${\mathcal {S}}(G',{\mathcal {L}}'_{A'})$ and ${\mathcal {S}}_{({\mathcal {L}}'_{A'}, i T'_1,\dots , i T'_{n'} )}(G')$, respectively, such that

$$\begin{aligned} \varphi =\sum _{|\gamma |<h} \mathbf{T} '^\gamma {\widetilde{\varphi }}_\gamma +\sum _{|\gamma |=h} \mathbf{T} '^\gamma \varphi _\gamma \end{aligned}$$

for every $h\in \mathbb {N}$.

Proof

For every $k\in \lbrace 1,\dots , n' \rbrace$, let $G'_k$ be the quotient of $G'$ by the normal subgroup $\exp _{G'}(\mathbb {R}T'_k)$. Endow ${\mathfrak {g}}'$ with a scalar product which turns $(T'_1,\dots , T'_{n'})$ into an orthonormal family. Then, Proposition 3.12 implies that $(\pi _1)_*(\varphi )\in {\mathcal {S}}_{\text {d}\pi _1({\mathcal {L}}'_{A'}, i T'_{2},\dots , i T'_{n'})}(G'_1)$, so that there is ${{\widetilde{m}}}_1\in {\mathcal {S}}(\sigma (\text {d}\pi _1({\mathcal {L}}'_{A'}, i T'_{2},\dots , i T'_{n'})))$ such that $(\pi _1)_*(\varphi )={\mathcal {K}}_{\text {d}\pi _1({\mathcal {L}}'_{A'}, i T'_{2},\dots , i T'_{n'})}({{\widetilde{m}}}_1)$. If we define ${\widetilde{\varphi }}_{0,1}{:}{=}{\mathcal {K}}_{({\mathcal {L}}'_{A'}, i T'_{2},\dots , i T'_{n'})}({{\widetilde{m}}}_1)$, then Proposition 3.12 implies that $(\pi _1)_*(\varphi -{\widetilde{\varphi }}_{0,1})=0$. In other words,

$$\begin{aligned} \int _{\mathbb {R}} (\varphi -{\widetilde{\varphi }}_{0,1})(\exp _{G'}(x+s T'_1))\,\text {d}s=0 \end{aligned}$$

for every $x\in T_1'^\perp$. Identifying ${\mathcal {S}}(G')$ with ${\mathcal {S}}(\mathbb {R}T'_1;{\mathcal {S}}(T_1'^\perp ))$ and applying a simple consequence of the classical Hadamard’s lemma, we see that there is $\varphi _1 \in {\mathcal {S}}(G')$ such that

$$\begin{aligned} \varphi ={\widetilde{\varphi }}_{0,1}+ T'_1 \varphi _1. \end{aligned}$$

Now, let us prove that $\varphi _1\in {\mathcal {S}}_{({\mathcal {L}}'_{A'}, i T'_{1},\dots , i T'_{n'})}(G')$. Indeed,

$$\begin{aligned} T'_1 {\mathcal {K}}_{({\mathcal {L}}'_{A'}, i T'_{2},\dots , i T'_{n'})}{\mathcal {M}}_{({\mathcal {L}}'_{A'}, i T'_{2},\dots , i T'_{n'})}(\varphi _1)=\varphi -{\widetilde{\varphi }}_{0,1}=T'_1 \varphi _1. \end{aligned}$$

Since clearly ${\mathcal {K}}_{({\mathcal {L}}'_{A'}, i T'_{2},\dots , i T'_{n'})}{\mathcal {M}}_{({\mathcal {L}}'_{A'}, i T'_{2},\dots , i T'_{n'})}(\varphi _1)\in L^2(G')$, and since $T'_1$ is one-to-one on $L^2(G')$, the assertion follows. If $n'\geqslant 2$, then we can apply the same argument to ${\widetilde{\varphi }}_{0,1}$ considering the quotient $G_2'$: since we know that ${\widetilde{\varphi }}_{0,1}\in {\mathcal {S}}(G',({\mathcal {L}}'_{A'},i T'_2,\dots , i T'_{n'}))$, we do not need to impose the condition that $\text {d}\pi _2({\mathcal {L}}'_{A'},i T'_3,\dots , i T'_{n'})$ satisfies property (S). Then, we obtain ${\widetilde{\varphi }}_{0,2}\in {\mathcal {S}}(G',({\mathcal {L}}'_{A'}, i T'_{3},\dots , i T'_{n'}))$ and $\varphi _2\in {\mathcal {S}}_{({\mathcal {L}}'_{A'}, i T'_{1},\dots , i T'_{n'})} (G')$ such that

$$\begin{aligned} \varphi ={\widetilde{\varphi }}_{0,2}+ T'_1 \varphi _1+ T'_2 \varphi _2. \end{aligned}$$

Iterating this procedure, we eventually find ${\widetilde{\varphi }}_0\in {\mathcal {S}}(G',{\mathcal {L}}'_{A'})$ and $\varphi _1,\dots , \varphi _{n'}\in {\mathcal {S}}_{({\mathcal {L}}'_{A'}, i T'_{1},\dots , i T'_{n'})}(G')$ such that

$$\begin{aligned} \varphi ={\widetilde{\varphi }}_0+\sum _{k=1}^{n'} T'_k \varphi _k. \end{aligned}$$

The assertion follows proceeding inductively. $\square$

Notice that, if G is abelian and ${\mathcal {L}}$ is a Laplacian on G, then ${\mathcal {L}}$ satisfies properties (RL) and (S) (cf. [71] for property (S)).

Theorem 9.2

Assume that $W=\lbrace 0 \rbrace$. Then, $({\mathcal {L}}, (-i T_k)_{k=1}^{n_2'})$ satisfies property (S) for every $n_2'\leqslant n_2$.

Proof

Notice that Theorems 8.2 and 8.5 imply that $({\mathcal {L}}, (-i T_k)_{k=1}^{n_2'})$ satisfies property (RL). Therefore, by means of Corollary 10.7 we see that it will suffice to prove the assertion for $n_2'=n_2$. In addition, the abelian case, that is, the case $n_2=0$, has already been considered in the remark preceding the statement. We proceed by induction on $n_2\geqslant 1$.

1 Observe first that the abelian case, Theorem 8.2, and Lemma 9.1 imply that we may find a family $({\widetilde{\varphi }}_\gamma )$ of elements of ${\mathcal {S}}(G,{\mathcal {L}})$, and a family $(\varphi _\gamma )$ of elements of ${\mathcal {S}}_{{\mathcal {L}}_A}(G)$ such that

$$\begin{aligned} \varphi =\sum _{|\gamma |< h} (-i\mathbf{T} )^\gamma {\widetilde{\varphi }}_\gamma + \sum _{|\gamma |=h}(-i\mathbf{T} )^\gamma \varphi _\gamma \end{aligned}$$

for every $h\in \mathbb {N}$.

Define ${{\widetilde{m}}}_\gamma {:}{=}{\mathcal {M}}_{{\mathcal {L}}}({\widetilde{\varphi }}_\gamma )\in {\mathcal {S}}(\sigma ({\mathcal {L}}))$ and $m_\gamma {:}{=}{\mathcal {M}}_{{\mathcal {L}}_A}(\varphi _\gamma )\in C_0(\sigma ({\mathcal {L}}_A))$ for every $\gamma$. Then,

$$\begin{aligned} m_0(\lambda , \omega ) =\sum _{|\gamma |<h} \omega ^\gamma {{\widetilde{m}}}_\gamma (\lambda )+ \sum _{|\gamma |=h} \omega ^\gamma m_\gamma (\lambda , \omega ) \end{aligned}$$

for every $h\in \mathbb {N}$ and for every $(\lambda , \omega )\in \sigma ({\mathcal {L}}_A)$.

By a vector-valued version of Borel’s lemma (cf. [41, Theorem 1.2.6] for the scalar, one-dimensional case), we see that there is ${\widehat{m}}\in C^\infty _c(\mathbb {R}^{n_2}; {\mathcal {S}}(\mathbb {R}) )$ such that $\partial ^\gamma {\widehat{m}}(0)={\widetilde{m}}_\gamma$ for every $\gamma \in \mathbb {N}^{n_2}$. Interpret ${\widehat{m}}$ as an element of ${\mathcal {S}}(\sigma ({\mathcal {L}}_A))$. Reasoning on $m-{\widehat{m}}$, we may reduce to the case in which ${{\widetilde{m}}}_\gamma =0$ for every $\gamma$; then, we shall simply write m instead of $m_0$.

2 Consider the norm $N:\omega \mapsto \mu (\omega )(\mathbf{n} _\mathbf{1 })$ on ${\mathfrak {g}}_2^*$ and let S be the associated unit sphere. Define $\sigma (\omega ){:}{=}\frac{\omega }{N(\omega )}$ for every $\omega \in {\mathfrak {g}}_2^*\setminus \lbrace 0 \rbrace$. Then, the mapping (cf. Definition 7.8)

$$\begin{aligned} S\ni \omega \mapsto (\pi _\omega )_*(\varphi )\in {\mathcal {S}}({\mathfrak {g}}_1 \oplus \mathbb {R}) \end{aligned}$$

is of class $C^\infty$. Fix $\omega _0\in S$. It is not hard to see that we may find a dilation-invariant open neighbourhood U of $\omega _0$ and an analytic mapping $\psi :U\times ({\mathfrak {g}}_1\oplus \mathbb {R})\rightarrow \mathbb {R}^{2 n_1}\times \mathbb {R}\times \mathbb {R}^d$ such that, for every $\omega \in U$, $\psi _\omega {:}{=}\psi (\omega ,\,\cdot \,)$ is an isometry of ${\mathfrak {g}}_1 \oplus \mathbb {R}$ onto $\mathbb {R}^{2 n_1}\times \mathbb {R}\times \mathbb {R}^d$ such that $\psi _\omega (P_0(\omega )({\mathfrak {g}}_1))=\lbrace 0 \rbrace \times \mathbb {R}^d$ and $\psi _\omega (\lbrace 0 \rbrace \times \mathbb {R})=\lbrace 0 \rbrace \times \mathbb {R}\times \lbrace 0 \rbrace$. Take $\omega \in U$. By transport of structure, we may put on $\mathbb {R}^{2 n_1}\times \mathbb {R}$ a group structure for which $\mathbb {R}^{2 n_1}\times \mathbb {R}$ is isomorphic to $\mathbb {H}^{n_1}$ and which turns $\psi _\omega$ into an isomorphism of Lie groups; here, ${\mathfrak {g}}_1\oplus \mathbb {R}$ is endowed with the structure induced by its identification with $G_\omega$.¹³ Then, there is a sub-Laplacian ${\mathcal {L}}'_\omega$ on $\mathbb {R}^{2 n_1}\times \mathbb {R}$ such that, if T denotes the derivative along $\lbrace 0 \rbrace \times \mathbb {R}\subseteq \mathbb {R}^{2 n_1}\times \mathbb {R}$ and $\Delta$ is the standard (positive) Laplacian on $\mathbb {R}^d$, then

$$\begin{aligned} \text {d}(\psi _\omega \circ \pi _\omega )({\mathcal {L}}_A)=({\mathcal {L}}'_\omega +\Delta , \omega (\mathbf{T} ) T). \end{aligned}$$

Then, Proposition 3.12 and Lemma 4.5 imply that

$$\begin{aligned} (\psi _\omega \circ \pi _\omega )_*(\varphi _\gamma )(y,t,\,\cdot \,)\in {\mathcal {S}}_\Delta (\mathbb {R}^d) \end{aligned}$$

for every $(y,t)\in \mathbb {R}^{2 n_1}\times \mathbb {R}$ and for every $\gamma \in \mathbb {N}^{n_2}$. Define

$$\begin{aligned}&{\widehat{\varphi }}_\gamma :(U\cap S) \times \mathbb {R}_+\times ( \mathbb {R}^{2 n_1}\times \mathbb {R})\ni (\omega , \xi , (y,t)) \\&\qquad \qquad \mapsto {\mathcal {M}}_\Delta ((\psi _\omega \circ \pi _\omega )_*(\varphi _\gamma )(y,t,\,\cdot \,) )(\xi ), \end{aligned}$$

so that ${\widehat{\varphi }}_\gamma (\omega ,\,\cdot \,,(y,t))\in {\mathcal {S}}(\mathbb {R}_+)$ for every $\omega \in U\cap S$ and for every $(y,t)\in \mathbb {R}^{2 n_1}\times \mathbb {R}$, since $\Delta$ satisfies property (S). In addition, the mapping

$$\begin{aligned} \omega \mapsto [ (y,t)\mapsto (\psi _\omega \circ \pi _\omega )_*(\varphi _\gamma )(y,t,\,\cdot \,) ] \end{aligned}$$

belongs to ${\mathcal {E}}(S\cap U; {\mathcal {S}}(\mathbb {R}^{2 n_1}\times \mathbb {R}; {\mathcal {S}}_\Delta (\mathbb {R}^d) ))$, so that the mapping

$$\begin{aligned} \omega \mapsto [ (y,t)\mapsto {\widehat{\varphi }}_\gamma (\omega ,\,\cdot \,,(y,t)) ] \end{aligned}$$

belongs to ${\mathcal {E}}(S\cap U; {\mathcal {S}}(\mathbb {R}^{2 n_1}\times \mathbb {R}; {\mathcal {S}}(\mathbb {R}_+) ))$. Now, observe that the mapping

$$\begin{aligned} U\ni \omega \mapsto \psi _\omega ^{-1}\in {\mathcal {L}}(\mathbb {R}^{2 n_1}\times \mathbb {R}\times \mathbb {R}^d; {\mathfrak {g}}_1 \oplus \mathbb {R}) \end{aligned}$$

is of class $C^\infty$, so that also the mapping

$$\begin{aligned} f:U\times \mathbb {R}^{n_1}\ni (\omega ,y)\mapsto & {} | ( \psi _{\sigma (\omega )}^{-1}(y,0,0) )(\omega ) |^2\\&\quad =\langle |J_{q,\omega } |(\psi _{\sigma (\omega )}^{-1}(y,0,0)) \vert \psi _{\sigma (\omega )}^{-1}(y,0,0)\rangle \end{aligned}$$

is of class $C^\infty$, thanks to Proposition 7.7. In addition, as in the proof of Lemma 8.1, we see that

$$\begin{aligned} \begin{aligned} m_\gamma (\xi +N(\omega ), \omega (\mathbf{T} ))= \int _{\mathbb {R}^{2 n_1}\times \mathbb {R}} {\widehat{\varphi }}_\gamma (\sigma (\omega ), \xi , (y,t)) e^{ -\frac{1}{4} f(\omega , y)-i N(\omega )t }\,\text {d}(y,t) \end{aligned} \end{aligned}$$

for every $\gamma \in \mathbb {N}^{n_2}$, for every $\omega \in U$, and for every $\xi \geqslant 0$. Therefore, the preceding arguments and some integrations by parts show that, for every $p_3\in \mathbb {N}$,

$$\begin{aligned} \begin{aligned}&m(\xi +N(\omega ),\omega (\mathbf{T} ))\\\quad&=\sum _{|\gamma |=h}\sigma (\omega (\mathbf{T} ))^\gamma \int _{\mathbb {R}^{2 n_1}\times \mathbb {R}}(-i T)^h{\widehat{\varphi }}_\gamma (\sigma (\omega ),\xi ,(y,t)) e^{-\frac{1}{4} f(\omega ,y)-i N(\omega ) t }\,\text {d}(y,t)\\\quad&=\sum _{|\gamma |=h}\frac{\omega (\mathbf{T} )^\gamma }{N(\omega )^{p_3}}\int _{\mathbb {R}^{2 n_1}\times \mathbb {R}} (-i T)^{p_3}{\widehat{\varphi }}_\gamma (\sigma (\omega ),\xi ,(y,t)) e^{-\frac{1}{4} f(\omega ,y)-i N(\omega ) t }\,\text {d}(y,t) \end{aligned} \end{aligned}$$

for every $h\in \mathbb {N}$, for every $\omega \in U$, and for every $\xi \geqslant 0$. Now, fix $p_1,p_2,p_3\in \mathbb {N}$, and take $h\in \mathbb {N}$. Apply Faà di Bruno’s formula and integrate by parts $p_3$ times in the t variable. Then, there is a constant $C>0$ such that

$$\begin{aligned} \begin{aligned} | (\partial _1^{p_1} \partial _2^{p_2}m )(\xi ,\omega (\mathbf{T} )) |&\leqslant C N(\omega )^{h-p_2-p_3}(1+N(\omega ))^{2 p_2} \int _{\mathbb {R}^{2 n_1}\times \mathbb {R}} (1+|(y,t) |)^{2 p_2}\times \\&\qquad \times \max _{\begin{array}{c} |\gamma |=h\\ p'=0,\dots , p_2 \end{array}} | {\widehat{\varphi }}_\gamma ^{(p_1+p_3+p')}(\sigma (\omega ), \xi -N(\omega ), (y,t)) | \,\text {d}(y,t) \end{aligned} \end{aligned}$$

for every $(\xi ,\omega (\mathbf{T} ))\in \sigma ({\mathcal {L}}_A)^\circ \cap (\mathbb {R}\times U)$, where ${\sigma ({\mathcal {L}}_A)}^\circ$ denotes the interior of $\sigma ({\mathcal {L}}_A)$. Here, $|(y,t) |=|y |+\sqrt{|t |}$ is a homogeneous norm on $\mathbb {R}^{2 n_1}\times \mathbb {R}$.

Now, take a compact subset K of $U\cap S$. Then, the properties of the ${\widehat{\varphi }}_\gamma$ imply that for every $p_4\in \mathbb {N}$ there is a constant $C'$ such that

$$\begin{aligned} | {\widehat{\varphi }}_\gamma ^{(p')}(\omega , \xi , (y,t)) |\leqslant \frac{C'}{ (1+\xi )^{p_4}(1+|(y,t) |)^{2 p_2+ 2 n_1+3} } \end{aligned}$$

for every $\gamma$ with length h, for every $p'=0,\dots , p_1+p_2+p_3$, for every $\omega \in K$, for every $\xi \geqslant 0$, and for every $(y,t)\in \mathbb {R}^{2 n_1}\times \mathbb {R}$. Therefore, there is a constant $C''>0$ such that

$$\begin{aligned} | (\partial _1^{p_1} \partial _2^{p_2}m )(\xi ,\omega (\mathbf{T} )) |\leqslant C'' N(\omega )^{h-p_2-p_3}\frac{(1+N(\omega ))^{2 p_2}}{ (1+ \xi -N(\omega ) )^{p_4} } \end{aligned}$$

for every $(\xi ,\omega (\mathbf{T} ))\in \sigma ({\mathcal {L}}_A)^\circ \cap (\mathbb {R}\times U)$ such that $\sigma (\omega )\in K$. By the arbitrariness of U and K, and by the compactness of S, we see that we may take $C''$ so that the preceding estimate holds for every $(\xi ,\omega (\mathbf{T} ))\in \sigma ({\mathcal {L}}_A)^\circ \cap (\mathbb {R}\times (\mathbb {R}^{n_2}\setminus \lbrace 0 \rbrace ))$.

Now, taking $h-p_3>p_2$ we see that $\partial _1^{p_1} \partial _2^{p_2}m$ extends to a continuous function on $\sigma ({\mathcal {L}}_A)$ which vanishes on $\mathbb {R}_+\times \lbrace 0 \rbrace$. If $N(\omega )\leqslant \frac{1}{3}$, then observe that

$$\begin{aligned} \frac{1}{3} + \xi + N(\omega ) \leqslant \frac{2}{3}+\xi \leqslant 1+\xi -N(\omega ) \end{aligned}$$

for every $\xi \geqslant N(\omega )$. On the other hand, if $N(\omega )\geqslant \frac{1}{3}$, then take $p_3=p_2+p_4+h$ and observe that

$$\begin{aligned} 1+\xi +N(\omega ) \leqslant (1+2 N(\omega ))(1+\xi -N(\omega ))\leqslant 5 N(\omega ) (1+\xi -N(\omega )) \end{aligned}$$

for every $\xi \geqslant N(\omega )$. Hence, for every $p_4\in \mathbb {N}$ we may find a constant $C'''>0$ such that

$$\begin{aligned} | (\partial _1^{p_1} \partial _2^{p_2}m )(\xi ,\omega (\mathbf{T} )) |\leqslant C''' \frac{1}{ (1+ \xi +N(\omega ) )^{p_4} } \end{aligned}$$

for every $\xi \geqslant N(\omega )$. Now, extending [64, Theorem 5 of Chapter VI] to the case of Schwartz functions in the spirit of [5, Theorem 6.1], we see that $m\in {\mathcal {S}}(\sigma ({\mathcal {L}}_A))$. $\square$

Theorem 9.3

Assume that G is the product of a finite family $(G_\iota )_{\iota \in I}$ of 2-step stratified groups which do not satisfy the $MW^+$ condition; endow each $G_\iota$ with a sub-Laplacian ${\mathcal {L}}_\iota$ and assume that $({\mathcal {L}}_\iota , -i {\mathcal {T}}_\iota )$ satisfies property (RL) (resp. (S)) for some finite family ${\mathcal {T}}_\iota$ of elements of the second layer of the Lie algebra of $G_\iota$. Define ${\mathcal {L}}{:}{=}\sum _{\iota \in I} {\mathcal {L}}_\iota$ (on G), and let ${\mathcal {T}}$ be a finite family of elements of the vector space generated by the ${\mathcal {T}}_\iota$. Then, the family $({\mathcal {L}}, - i {\mathcal {T}})$ satisfies property (RL) (resp. (S)).

Proof

Observe first that Theorems 4.4 and 4.6 imply that the family $({\mathcal {L}}_I, (- i {\mathcal {T}}_\iota )_{\iota \in I})$ on G satisfies property (RL) (and also property (S) if each one of the families $({\mathcal {L}}_\iota , -i {\mathcal {T}}_\iota )$ does). Therefore, the assertion follows from Propositions 10.2, 10.3, 10.4, 7.11, and Corollary 10.7, since $\sigma ({\mathcal {L}}_I, (- i {\mathcal {T}}_\iota )_{\iota \in I})$ is a semialgebraic (hence Nash subanalytic) closed convex cone thanks to the remark at the beginning of Section 8. $\square$

Appendix: Composite Functions

We collect in this appendix a number of technical results used throughout the paper to establish properties (RL) and (S) of ‘image families.’

1.1 Continuous Functions

In this subsection, we consider the following problem: given three Polish spaces X, Y, Z, a positive measure $\mu$ on X, a $\mu$-measurable mapping $\pi :X\rightarrow Y$, and a function $m:Y\rightarrow Z$ such that $m\circ \pi$ equals $\mu$-almost everywhere a continuous function, does m equal $\pi _*(\mu )$-almost everywhere a continuous function?

To this end, we introduce the following definition.

Definition 10.1

Let X be a Polish space, Y a set, $\mu$ a positive Radon measure on X, and $\pi$ a mapping from X into Y. We say that two points $x,x'$ of $\text {Supp}\left( \mu \right)$ are $(\mu ,\pi )$-connected if $\pi (x)=\pi (x')$ and there are $x=x_1,\dots , x_k=x'\in \pi ^{-1}(\pi (x))\cap \text {Supp}\left( \mu \right)$ such that, for every $j=1,\dots , k$, for every neighbourhood $U_j$ of $x_j$ in $\text {Supp}\left( \mu \right)$, and for every neighbourhood $U_{j+1}$ of $x_{j+1}$ in $\text {Supp}\left( \mu \right)$, the set $\pi ^{-1}(\pi (U_j)\cap \pi (U_{j+1}))$ is not $\mu$-negligible. We say that $\mu$ is $\pi$-connected if every pair of elements of $\text {Supp}\left( \mu \right)$ having the same image under $\pi$ are $(\mu ,\pi )$-connected.

Observe that $(\mu ,\pi )$-connectedness actually depends only on the equivalence class of $\mu$ and the equivalence relation induced by $\pi$ on X. In addition, notice that, if Y is a topological space and $\pi$ is open at some point of each fibre (in the support of $\mu$), then $\mu$ is $\pi$-connected.

We emphasize that, in the definition of $(\mu ,\pi )$-connectedness, the points $x_1,\dots ,x_k$ are fixed before considering their neighbourhoods. In other words, if for every neighbourhood U of x in $\text {Supp}\left( \mu \right)$ and for every neighbourhood $U'$ of $x'$ in $\text {Supp}\left( \mu \right)$ we found $x=x_1,\dots , x_k=x'$ and neighbourhoods $U_j$ of $x_j$ in $\text {Supp}\left( \mu \right)$ so that $U=U_1$, $U'=U_k$ and, for every $j=1,\dots , k$, the set $\pi ^{-1}(\pi (U_j)\cap \pi (U_{j+1}))$ were not $\mu$-negligible, then we would not be able to conclude that x and $x'$ are $(\mu ,\pi )$-connected.

Now we can prove our main result. Notice that, even though its hypotheses are quite restrictive, it still gives rise to important consequences.

Proposition 10.2

Let X, Y, Z be three Polish spaces, $\pi :X\rightarrow Y$ a $\mu$-measurable mapping, and $\mu$ a $\pi$-connected positive Radon measure on X. Assume that $\pi$ is $\mu$-proper and that there is a disintegration $(\lambda _y)_{y\in Y}$ of $\mu$ relative to $\pi$ such that $\text {Supp}\left( \lambda _y\right) \supseteq \text {Supp}\left( \mu \right) \cap \pi ^{-1}(y)$ for $\pi _*(\mu )$-almost every $y\in Y$.

Take a continuous map $m_0:X\rightarrow Z$ and assume that there is map $m_1:Y\rightarrow Z$ such that $m_0(x)= (m_1\circ \pi )(x)$ for $\mu$-almost every $x\in X$. Then, there is a $\pi _*(\mu )$-measurable mapping $m_2:Y\rightarrow Z$ such that $m_0=m_2\circ \pi$pointwise on $\text {Supp}\left( \mu \right)$.

Notice that, if $\pi$ is also proper, then $m_2$ is actually continuous on $\pi (\text {Supp}\left( \mu \right) )$.

Proof

Observe first that there is a $\pi _*(\mu )$-negligible subset N of Y such that $m_1\circ \pi =m_0$$\lambda _y$-almost everywhere for every $y\in Y\setminus N$. Notice that we may assume that $\text {Supp}\left( \mu \right) =X$ and that, if $y\in Y\setminus N$, then the support of $\lambda _y$ contains $\pi ^{-1}(y)$. Since $m_0$ is continuous and since $m_1\circ \pi$ is constant on the support of $\lambda _y$, it follows that $m_0$ is constant on $\pi ^{-1}(y)$ for every $y\in Y\setminus N$.

Now, take $y\in \pi (X)\cap N$ and $x_1,x_2\in \pi ^{-1}(y)$. Let ${\mathfrak {U}}(x_1)$ and ${\mathfrak {U}}(x_2)$ be the filters of neighbourhoods of $x_1$ and $x_2$, respectively. Assume first that $\pi (U_1)\cap \pi (U_2)$ is not $\pi _*(\mu )$-negligible for every $U_1\in {\mathfrak {U}}(x_1)$ and for every $U_2\in {\mathfrak {U}}(x_2)$. Take $U_1\in {\mathfrak {U}}(x_1)$ and $U_2\in {\mathfrak {U}}(x_2)$. Then, there is $y_{U_1,U_2}\in \pi (U_1)\cap \pi (U_2)\setminus N$, and then $x_{h,U_1,U_2}\in U_h\cap \pi ^{-1}(y_{U_1,U_2})$ for $h=1,2$. Now, $m_0(x_{1,U_1,U_2})=m_0(x_{2,U_1,U_2})$ for every $U_1\in {\mathfrak {U}}(x_1)$ and for every $U_2\in {\mathfrak {U}}(x_2)$. In addition, $x_{h,U_1,U_2}\rightarrow x_h$ in X along the product filter of ${\mathfrak {U}}(x_1)$ and ${\mathfrak {U}}(x_2)$. Since $m_0$ is continuous, passing to the limit we see that $m_0(x_1)=m_0(x_2)$. Since $\mu$ is $\pi$-connected, this implies that $m_0$ is constant on $P^{-1}(y)$ for every$y\in \pi (X)$. The assertion follows. $\square$

In the following proposition we give sufficient conditions in order that a measure be connected.

Proposition 10.3

Let $E_1,E_2$ be two finite-dimensional vector spaces, $L:E_1\rightarrow E_2$ a linear mapping, C a closed convex subset of $E_1$ and $\mu$ a positive Radon measure on $E_1$ with support C. Take a Polish subspace X of $E_1$ such that $\mu (E_1\setminus X)=0$, and assume that either $X=C$ or C is a convex cone. Then, $\mu _X$, that is, the measure induced by $\mu$ on X, is -connected.

Actually, there is no need that X be a Polish space, but we did not consider Radon measures on more general Hausdorff spaces.

Proof

We may assume that C has non-empty interior. Then, we may find a non-empty bounded convex open subset U of C and an convex open neighbourhood V of 0 in $\ker L$ such that $U+V\subseteq C$. Take $r\in ]0,1]$ and $x,y\in C\cap X$ such that $y-x\in V$; take $R_x>0$ so that $U\subseteq B(x,R_x)$. Then, for every $u\in U$ we have $y+ r(u-x)\in B(y,r R_x)\cap [y,y-x+u]\subseteq B(y,r R_x)\cap C$; analogously, $x+r(U-x)\subseteq B(x,r R_x)\cap C$. Since $L(x)=L(y)$, we infer that

$$\begin{aligned} L^{-1}(L( B(x, r R_x)\cap C \cap X)\cap L(B(y,r R_x)\cap C\cap X))\supseteq [x+r(U-x)]\cap X. \end{aligned}$$

Now, $x+r(U-x)$ is a non-empty open subset of $C=\text {Supp}\left( \mu \right)$, so that $\mu _X([x+r(U-x)]\cap X)=\mu (x+r(U-x))>0$. The arbitrariness of r then implies that x and y are $(\mu ,L)$-connected. The assertion then follows easily if $X=C$.

Finally, assume that $X\ne C$, so that C is a convex cone; we may assume that C has vertex 0. Then, given $x,y\in C\cap X$ such that $L(x)=L(y)$, we may find $r_{x,y}>0$ such that, with the above notation, $y-x\in r_{x,y} V$. Then, $r_{x,y} U+r_{x,y} V\subseteq r_{x,y} V\subseteq C$, so that the above argument shows that x and y are $(\mu ,L)$-connected. The arbitrariness of x and y then implies that $\mu _X$ is -connected. $\square$

Now we present a result on the disintegration of Hausdorff measures, which is particularly useful to check the assumptions of Proposition 10.2, and is a straightforward consequence of the general coarea formula (cf. [28, Theorem 3.2.22]). Recall that a subset of $\mathbb {R}^n$ is said to be countably ${\mathcal {H}}^k$-rectifiable if it is the union of an ${\mathcal {H}}^k$-negligible set and a countable family of Lipschitz images of bounded subsets of $\mathbb {R}^k$. For example, any countable union of k-dimensional submanifolds (of class $C^1$) of $\mathbb {R}^n$ is ${\mathcal {H}}^k$-measurable and countably ${\mathcal {H}}^k$-rectifiable, as well as any countable union of images of $C^\infty$ functions (defined on $C^\infty$ manifolds with a countable base) of maximum rank k, thanks to [63, Theorem 1]. We refer the reader to [28, Theorem 3.2.22] for the definition of the approximate k-dimensional Jacobian $\text {ap}\,J_{k} P$ of a Lipschitz mapping P defined on an ${\mathcal {H}}^{k}$-measurable and countably ${\mathcal {H}}^{k}$-rectifiable subset E of $\mathbb {R}^{n}$ and taking values in $\mathbb {R}^m$. Notice that, if E is a submanifold (of class $C^1$) of $\mathbb {R}^{n}$ and P is of class $C^1$, then $\text {ap}\,J_{k}P(x)$ is simply $\Vert \bigwedge ^{k} T_x (P) \Vert$ for every $x\in E$, where $T_x (P)$ denotes the differential of P.

Proposition 10.4

Let, for $j=1,2$, $E_j$ be an ${\mathcal {H}}^{k_j}$-measurable and countably ${\mathcal {H}}^{k_j}$-rectifiable subset of $\mathbb {R}^{n_j}$. Assume that $k_2\leqslant k_1$, and let P be a locally Lipschitz mapping of $E_1$ into $E_2$. Take a positive function $f\in L^1_\text {loc}({\mathcal {H}}^{k_1})$ which vanishes on the complement of $E_1$, and assume that $f(x){ ap }J_{k_2} P(x)\ne 0$ for ${\mathcal {H}}^{k_1}$-almost every $x\in E_1$, and that P is $(f\cdot {\mathcal {H}}^{k_1})$-proper.¹⁴

Then, the following hold:

• 1. the mapping

  $$\begin{aligned} g:\mathbb {R}^{n_2}\ni y \mapsto \int _{P^{-1}(y)} \frac{f}{{ap } J_{k_2} P}\,\text {d}{\mathcal {H}}^{k_1-k_2} \end{aligned}$$

  is well-defined ${\mathcal {H}}^{k_2}$-almost everywhere and measurable; in addition,

  $$\begin{aligned} P_*(f\cdot {\mathcal {H}}^{k_1})= g\cdot {\mathcal {H}}^{k_2}; \end{aligned}$$

• 2. the measure

  $$\begin{aligned} \beta _y{:}{=}\frac{1}{g(y)}\frac{f}{{ap } J_{k_2} P} \chi _{P^{-1}(y)}\cdot {\mathcal {H}}^{k_1-k_2} \end{aligned}$$

  is well-defined and Radon for $P_*(f\cdot {\mathcal {H}}^{k_1})$-almost every $y\in \mathbb {R}^{n_2}$; in addition, $(\beta _y)$ is a disintegration of $f\cdot {\mathcal {H}}^{k_1}$ relative to P;

• 3. $\beta _y$ is equivalent to $\chi _{P^{-1}(y)}\cdot {\mathcal {H}}^{k_1-k_2}$ for $P_*(f\cdot {\mathcal {H}}^{k_1})$-almost every $y\in E_2$.

1.2 Schwartz Functions

In this subsection we shall extend some results on composite differentiable functions by Bierstone, Milman and Schwarz in [9, 11] to the case of Schwartz functions by means of the techniques developed by Astengo, Di Blasio and Ricci in [5].

We shall take advantage of the remarkable works of Bierstone, Milman and Schwarz about the composition of smooth functions on analytic manifolds. Recall that, if M is a (real, finite-dimensional) analytic manifold, then a subset A of M is called analytic if every $x\in M$ has a neighbourhood U such that $U\cap A$ is the zero locus of a (real) analytic function on U. The set A is semianalytic if every $x\in M$ has a neighbourhood U such that $U\cap A$ belongs to the algebra of subsets of U generated by the sets of the form $f^{-1}(]0,\infty [)$, where f is a (real) analytic function on U. The set A is subanalytic if every $x\in M$ has a neighbourhood U such that $U\cap A=\text {pr}_1(B)$ for some analytic manifold N and some relatively compact semianalytic subset B of $M\times N$. The set A is Nash subanalytic of pure dimension k if it is closed, subanalytic, and, for every $x\in A$, $\dim _x Y=\dim Z_x=k$, where $Z_x$ denotes the smallest germ of an analytic set at x containing the germ of A at x (cf. [11, 1.5]). A closed subanalytic set is Nash subanalytic if it is the locally finite union of Nash subanalytic sets of pure dimension. We refer the reader to [9,10,11] for an account of the main properties of semianalytic and (Nash) subanalytic sets. As a matter of fact, in the applications we shall only need to know that any closed convex subanalytic set is automatically Nash subanalytic, since it is contained in an affine space of the same dimension, and that closed semianalytic sets are Nash subanalytic (cf. [9, Proposition 2.3]).

Our starting point is the following result (cf. [9, Theorem 0.2] and [11, Theorem 0.2.1]). If C is a closed subset of $\mathbb {R}^n$, then we denote by ${\mathcal {E}}(C)$ is the quotient of ${\mathcal {E}}(\mathbb {R}^n)$ by the space of functions of class $C^\infty$ which vanish on C.

Theorem 10.5

Let C be a closed subanalytic subset of $\mathbb {R}^n$ and let $P:\mathbb {R}^n\rightarrow \mathbb {R}^m$ be an analytic mapping. Assume that P is proper on C and that P(C) is Nash subanalytic. Then, the canonical mapping

$$\begin{aligned} \Phi :{\mathcal {E}}(\mathbb {R}^m)\ni \varphi \mapsto \varphi \circ P\in {\mathcal {E}}(C) \end{aligned}$$

has a closed range, and admits a continuous linear section defined on $\Phi ({\mathcal {E}}(\mathbb {R}^n))$.

In addition, $\psi \in {\mathcal {E}}(C)$ belongs to the image of $\Phi$ if and only if for every $y\in P(C)$ there is $\varphi _y\in {\mathcal {E}}(\mathbb {R}^m)$ such that, for every $x\in C$ such that $P(x)=y$, the Taylor series of $\varphi _y\circ P$ and $\psi$ at x differ by the Taylor series of a function of class $C^\infty$ which vanishes on C.

In order to simplify the notation, we shall simply say that $\psi$ is a formal composite of P if the second condition of the statement holds.

We shall now describe how Theorem 10.5 can be extended to the case of Schwartz functions. The strategy developed in [5] is the following: first, decompose dyadically a given Schwartz function in a sum of dilates of a family of test functions with a suitable decay; then, apply the section given by Theorem 10.5, truncate the resulting functions (so that they are still test functions), and finally sum their dilates. In order to do that, however, we need homogeneity.

Theorem 10.6

Let $P:\mathbb {R}^n\rightarrow \mathbb {R}^m$ be a polynomial mapping, and assume that $\mathbb {R}^n$ and $\mathbb {R}^m$ are endowed with dilations such that $P(r\cdot x)=r\cdot P(x)$ for every $r>0$ and for every $x\in \mathbb {R}^n$. Let C be a dilation-invariant subanalytic closed subset of $\mathbb {R}^n$, and assume that P is proper on C and that P(C) is Nash subanalytic. Then, the canonical mapping

$$\begin{aligned} \Phi :{\mathcal {S}}(\mathbb {R}^m)\ni \varphi \mapsto \varphi \circ P\in {\mathcal {S}}(C) \end{aligned}$$

has a closed range and admits a continuous linear section defined on $\Phi ({\mathcal {S}}(\mathbb {R}^m))$. In addition, $\psi \in {\mathcal {S}}(C)$ belongs to the image of $\Phi$ if and only if it is a formal composite of P.

As a matter of fact, in our applications C is (a subset of) $\sigma ({\mathcal {L}}_A)$. Then, Theorem 10.6 gives sufficient conditions in order that some $f\in {\mathcal {S}}_{P({\mathcal {L}}_A)}(G)$ which also belongs to ${\mathcal {K}}_{{\mathcal {L}}_A}({\mathcal {S}}(\sigma ({\mathcal {L}}_A)))$ should belong to ${\mathcal {K}}_{P({\mathcal {L}}_A)}({\mathcal {S}}(\sigma (P({\mathcal {L}}_A))))$ (cf. Sect. 5).

Notice, however, that sometimes it is convenient to take C so as to be a subset of $\sigma ({\mathcal {L}}_A)$ such that $P(C)=\sigma (P({\mathcal {L}}_A))$, since $\sigma ({\mathcal {L}}_A)$ need not be subanalytic.

Proof

For the first assertion, simply argue as in the proof of [5, Theorem 6.1] replacing the linear section provided by Schwarz and Mather with that of Theorem 10.5.

As for the second part of the statement, notice first that it follows easily from Theorem 10.5 when $\psi$ is compactly supported; since the image of $\Phi$ is closed, it follows by approximation in the general case. $\square$

In the following result, we give a simple but very useful application of Theorem 10.6.

Corollary 10.7

Let V and W be two finite-dimensional vector spaces, C a subanalytic closed convex cone in V, and L a linear mapping of V into W which is proper on C. Take $m_1\in {\mathcal {S}}(V)$, and assume that there is $m_2:W\rightarrow \mathbb {C}$ such that $m_1=m_2\circ L$ on C. Then, there is $m_3\in {\mathcal {S}}(W)$ such that $m_1=m_3\circ L$ on C.

Proof

Observe first that we may assume that C has non-empty interior and vertex 0. In addition, observe that L(C) is subanalytic (cf. [10, Theorem 0.1 and Proposition 3.13]), hence Nash subanalytic. Now, fix $x\in C$. Since the interior of C is not empty, it is clear that C is a total subset of V, so that we may find a free family $(v_j)_{j\in J}$ in C which generates an algebraic complement $V'$ of $\ker L$ in V. In addition, since either $x=0$ or $x\not \in \ker L$, we may assume that $x\in V'$. Let $L':W\rightarrow V$ be a linear mapping such that $L'\circ L$ is the identity on $V'$ and such that $L\circ L'$ is the identity on L(V).

Define $m'{:}{=}m_1\circ L'$, so that $m'\in {\mathcal {E}}(W)$. Next, define $C'{:}{=}V'\cap C$, so that $C'$ is a closed convex cone with non-empty interior in $V'$, since it contains the non-empty open set $\sum _{j\in J} \mathbb {R}_+^* v_j$. Take $z\in C'$ and any $y\in C\cap (x+\ker L)$. Then, $x+z=(L'\circ L) (x+z)=(L' \circ L )(y+z)$, so that $m_1=m'\circ L$ on $y+C'$. Since $m_1$ is constant on the intersections of C with the translates of $\ker L$, the same holds on $C\cap (y+C'+\ker L)$. Now, denote by $C'^\circ$ the interior of $C'$ in $V'$. Then, $y+C'^\circ +\ker L$ is an open convex set and y is adherent to $C\cap (y+C'^\circ +\ker L)$, which has non-empty interior since it is not empty and C is the closure of its interior (cf. [12, Corollary 1 to Proposition 16 of Chapter II, §2, No. 6]). Hence, the Taylor polynomials of every fixed order of $m_1$ and $m'\circ L$ about y coincide on $C\cap (y+C'^\circ +\ker L)$, hence on V. Since this holds for every $y\in C\cap [x+\ker L]$, Theorem 10.6 implies that there is $m_3\in {\mathcal {S}}(W)$ such that $m_1=m_3\circ L$ on C. $\square$