Elliptic boundary value problems associated with isometric group actions

Abstract

Given a manifold with boundary endowed with an action of a discrete group on it, we consider the algebra of operators generated by elements in the Boutet de Monvel algebra of pseudodifferential boundary value problems and shift operators acting on functions on the manifold and its boundary. Provided that the group is of polynomial growth and its action is isometric, we construct a Chern character for elliptic elements in this algebra with values in a de Rham type cohomology of the fixed point manifolds for the group action and obtain an index formula in terms of this Chern character. Our index formula contains as special cases the index formula by Fedosov for boundary value problems in the Boutet de Monvel algebra and the index formula by Nazaikinskii, Savin and Sternin for operators on a closed manifold associated with an isometric group action.

1 Introduction

The aim of this work is to give a solution of the index problem for nonlocal elliptic boundary value problems associated with isometric actions of discrete groups on manifolds with boundary.

The index problem in elliptic theory (see [1]) consists in computing indices of elliptic operators in terms of topological invariants of the principal symbol of operators and topological invariants of manifolds on which the operators are defined.

Index formulas are known in many geometric situations. In particular, many authors developed index theory for elliptic boundary value problems in the framework of classical boundary value problems (see [2,3,4]) and for Boutet de Monvel algebra of pseudodifferential boundary value problems with boundary and coboundary operators (see [5,6,7,8,9,10]).

Firstly, we note that obtaining index formulas in the case of pseudodifferential boundary value problems was of significant difficulty because principal symbols of the boundary value problems in question are operator functions defined on the cotangent bundle of the boundary of the manifold and it was necessary to construct topological invariants which take into account this operator function. These difficulties were overcame in [8], where a topological index of elliptic problems in Boutet de Monvel algebra was defined using special regularized traces on the algebra of principal symbols.

Secondly, we note that all known proofs of the index formulas for elliptic boundary value problems use stable homotopies (these homotopies are constructed using K-theory of the $C^*$-algebra of symbols), which allow to reduce the boundary value problem to such a form that its index is equal to the index of an operator on the double of the manifold with boundary. In this case, the index is easy to calculate using Atiyah–Singer formula (see [3, 7, 8, 10]).

Noncommutative geometry of Connes [11] contributes greately to the development of index theory. In noncommutative geometry, one usually considers algebras of operators, the principal symbols of which generate essentially noncommutative algebras. The algebras of crossed product type associated with group actions on manifolds (see [12,13,14,15,16,17,18,19,20,21,22]) appear in many applications. The corresponding equations on the manifold in question include pseudodifferential operators as well as shift operators along the orbits of the group action. For such operators, ellipticity conditions were obtained. They provide Fredholm solvability of the problem in Sobolev spaces (see [15]). However, index formulas were obtained only in the case of smooth closed manifolds (see [19, 20, 22, 23]). As a first step towards obtaining index formulas for boundary value problems associated with group actions on manifolds with boundary, a classification (up to stable homotopies) of elliptic boundary value problems was obtained and K-groups of the corresponding symbol algebra were calculated in [24].

In the present paper, we construct the topological index for elliptic boundary value problems on manifolds with boundary endowed with an isometric action of a discrete group of polynomial growth in the sense of Gromov [25]. More precisely, for the principal symbol of such a problem we construct the Chern character with values in a de Rham type cohomology of the cotangent bundle of fixed point submanifolds. The definition of Chern character uses traces on the algebra of noncommutative differential forms from [20] and on regularized traces from [8] on the algebra of symbols of Boutet de Monvel operators. Unlike the paper [8], where a topological index is constructed as a number, we refine this construction and construct a Chern character for the symbol in suitable cohomology groups, and the topological index is obtained from it by taking product with the Todd class of the manifold and integrating over the fundamental cycle.

Let us briefly describe the contents of the paper. In Sect. 2, we recall the definitions related to the Boutet de Monvel algebra of pseudodifferential boundary value problems on a manifold with boundary. In Sect. 3, we study the algebra generated by Boutet de Monvel operators and shift operators associated with isometric actions of discrete groups of polynomial growth on manifolds with boundary. We state an ellipticity condition for the elements in this algebra, their Fredholm property is established. After that we solve the index problem. To this end, we construct (in Sect. 4) cohomology of de Rham type for manifolds, whose boundary is the total space of a fibration (the cotangent bundle of a manifold with boundary has this structure and this structure implicitly appears in [8]). Furthermore, in Sect. 5, we introduce noncommutative differential forms on the boundary and regularized traces on them. These constructions enable us to define the Chern character for elliptic problems as a cohomology class on the cotangent bundle of the manifold of fixed points of the group action, considered as a manifold with fibered boundary (in the sense of Sect. 4). Next, in Sect. 6, we give the definition of the Todd class of the manifold and state the index theorem. The proof of the index theorem is given in Sects. 7 and 8. To this end, we establish a homotopy classification of elliptic boundary value problems, which enables us to make a homotopy of the boundary value problem to a quite simple boundary value problem in a neighborhood of the boundary of the manifold. Next, for such a problem the index formula can be checked by a reduction of the operator to the double and application of the index formula in [20] on the double.

2 Boutet de Monvel algebra

Let us recall the main properties of the Boutet de Monvel algebra (for more detailed exposition see [5, 7, 26, 27, 33]).

Let M be a smooth compact manifold with boundary X. We suppose that M is endowed with a Riemannian metric. In a neighborhood of the boundary we use local coordinates $x=(x',x_n)$ on M, where $\mathrm{dim}M=n$, $x'=(x_1,\ldots ,x_{n-1})$ are local coordinates on X, and $x_n$ is a defining function of the boundary, i.e., the boundary is locally defined by the equation $x_n=0$, while M is defined by the inequality $x_n\ge 0$. Denote as $(\xi ',\xi _n)$ the variables dual to $(x',x_n)$. Then $(x',x_n,\xi ',\xi _n)$ define local coordinates for the cotangent bundle $T^*M$. We fix a Riemannian metric on M.

Let us consider Boutet de Monvel operators of zero order and type. We write such operators as follows

$$\begin{aligned} {\mathcal {D}}= \left( \begin{array}{cc} A+G &{} C\\ B &{} A_X \end{array} \right) : \begin{array}{c} L^2(M)\\ \oplus \\ L^2(X) \end{array} \longrightarrow \begin{array}{c} L^2(M)\\ \oplus \\ L^2(X) \end{array}, \end{aligned}$$

(1)

where

• A is a classical pseudodifferential operator ($\psi$DO) of order zero on M; its complete symbol satisfies the transmission property (see below);

• $A_X$ is a $\psi$DO of order zero on X;

• B, C and G are boundary, coboundary and Green operators, respectively (or trace, potential, singular Green operators in the terminology of Boutet de Monvel [5]).

Recall that a classical symbol $a=a(x',x_n,\xi ',\xi _n)$ with an asymptotic expansion

$$\begin{aligned} a\sim a_m+a_{m-1}+\cdots \end{aligned}$$

(2)

into its homogeneous components $a_l$ satisfies the transmission property, if its order $m\in {\mathbb {Z}}$ and for all $k\in {\mathbb {Z}}_+$ and arbitrary multi-index $\alpha =(\alpha _1,...,\alpha _{n-1})\in {\mathbb {Z}}^{n-1}_+$ the following equality holds:

$$\begin{aligned} D^k_{x_n}D^\alpha _{\xi '}a_l(x',0,0,\xi _n)=e^{-i\pi (l-|\alpha |)}D^k_{x_n}D^\alpha _{\xi '}a_l(x',0,0,-\xi _n),\quad \xi _n\ne 0, \end{aligned}$$

(3)

where

$$\begin{aligned} D^\alpha _{\xi '}=\left( -i\frac{\partial }{\partial \xi _1}\right) ^{\alpha _1}\cdot ... \cdot \left( -i\frac{\partial }{\partial \xi _{n-1}}\right) ^{\alpha _{n-1}}. \end{aligned}$$

The principal symbol of operator (1) is the pair $\sigma ({\mathcal {D}})=(\sigma _M({\mathcal {D}}),\sigma _X({\mathcal {D}}))$, where the first component is called the interior symbol and is a function

$$\begin{aligned} \sigma _M({\mathcal {D}})=\sigma (A)\in C^\infty (S^*M), \end{aligned}$$

(4)

where $S^*M=\{(x,\xi )\in T^*M,|\xi |=1\}$ is the cosphere bundle of M, where $\sigma (A)$ is the principal symbol of A (if a is the complete symbol of A, then $\sigma (A)$ is equal to the leading order term $a_m$ in (2)). The second component is called the boundary symbol and is an operator function

$$\begin{aligned} \sigma _X({\mathcal {D}}) \in C^\infty (S^*X,{\mathcal {B}}(L^2({\mathbb {R}}_+)\oplus {\mathbb {C}})) \simeq C^\infty (S^*X,{\mathcal {B}}({\overline{H}}_+\oplus {\mathbb {C}})) \end{aligned}$$

(5)

on the cosphere bundle $S^*X$ of the boundary, where ${\mathcal {B}}$ is the algebra of bounded operators,

$$\begin{aligned} H_+={\mathcal {F}}({\mathcal {S}}(\overline{{\mathbb {R}}}_+)) \end{aligned}$$

stands for the space of images with the respect to the Fourier transform ${\mathcal {F}}_{x_n\rightarrow \xi _n}$ of the Schwartz space ${\mathcal {S}}(\overline{{\mathbb {R}}}_+)$ of smooth rapidly decaying at infinity functions on $\overline{{\mathbb {R}}}_+$. Denote the norm closure of $H_+\subset L^2({\mathbb {R}})$ by $\overline{H}_+$. Similarly, we define the space $H_-$ as

$$\begin{aligned} H_-={\mathcal {F}}({\mathcal {S}}(\overline{{\mathbb {R}}}_-)). \end{aligned}$$

Next, we define the projection $\Pi _+:H_+\oplus H_-\rightarrow H_+$ to the first summand and the continuous functional

$$\begin{aligned} \begin{array}{ccc} \Pi ':H_+\oplus H_- &{} \longrightarrow &{} {\mathbb {C}},\\ u(\xi _n) &{} \longmapsto &{}\lim \limits _{x_n\rightarrow 0+} {\mathcal {F}}^{-1}_{\xi _n\rightarrow x_n}(u(\xi _n)). \end{array} \end{aligned}$$

Note that if $u\in L^1({\mathbb {R}})\cap (H_+\oplus H_-)$, then

$$\begin{aligned} \Pi 'u=\frac{1}{2\pi }\int _{\mathbb {R}}u(\xi _n)d\xi _n. \end{aligned}$$

(6)

Let us denote the algebra of boundary symbols (5) by $\Sigma _X\subset C^\infty (S^*X,{\mathcal {B}}({\overline{H}}_+\oplus {\mathbb {C}}))$. To describe explicitly the elements in $\Sigma _X$, we consider smooth functions

• $b(x',\xi ',\xi _n)\in C^\infty (S^*X,H_-)$;

• $c(x',\xi ',\xi _n)\in C^\infty (S^*X,H_+)$;

• $g(x',\xi ',\xi _n,\eta _n)\in C^\infty (S^*X,H_+\otimes H_-)$;

• $q(x',\xi ')\in C^\infty (S^*X)$.

Here the spaces $H_\pm$, their topological tensor products, and smooth functions on $S^*X$ are considered as Fréchet spaces. Then an arbitrary boundary symbol $a_X\in \Sigma _X$ is a smooth operator family

$$\begin{aligned} a_X (x',\xi ')= \begin{array}{c} \overline{H}_+\\ \oplus \\ {\mathbb {C}} \end{array} \longrightarrow \begin{array}{c} \overline{H}_+\\ \oplus \\ {\mathbb {C}} \end{array} \end{aligned}$$

(7)

with the parameters $(x',\xi ')\in S^*X$. Operator (7) acts on pairs $h\in {\overline{H}}_+, v\in {\mathbb {C}}$ as follows:

$$\begin{aligned}&a_X(x',\xi ') \left( \begin{matrix} h \\ v \end{matrix} \right) \nonumber \\&\quad = \left( \begin{matrix} \Pi _+(a(x',0,\xi ',\xi _n)h(\xi _n))+\Pi '_{\eta _n}(g(x',\xi ',\xi _n,\eta _n)h(\eta _n)) + c(x',\xi ',\xi _n)v \\[2mm] \Pi '_{\xi _n}(b(x',\xi ',\xi _n)h(\xi _n)) + q(x',\xi ')v \end{matrix} \right) .\nonumber \\ \end{aligned}$$

(8)

Here $a(x',0,\xi ',\xi _n)$ is the restriction to the boundary of a symbol homogeneous of degree zero with the transmission property on M. The function $a(x',0,\xi ',\xi _n)$ is called the principal symbol of the boundary symbol $a_X$. It is known that smooth families (7) form an algebra.

Let us denote the algebra of matrices (1) by $\Psi _B(M)\subset {\mathcal {B}}(L^2(M)\oplus L^2(X))$. This algebra is called the Boutet de Monvel algebra.

Theorem 1

([7], Section 2.2.4.4, Corollary 2) The symbol mapping

$$\begin{aligned} \begin{array}{ccc} \Psi _B(M) &{} \longrightarrow &{} C^\infty (S^*M)\oplus C^\infty (S^*X,{\mathcal {B}}(L^2({\mathbb {R}}_+)\oplus {\mathbb {C}}))\\ {\mathcal {D}} &{} \longmapsto &{}(\sigma _M({\mathcal {D}}),\sigma _X({\mathcal {D}})) \end{array} \end{aligned}$$

is well defined and continuously extends to a monomorphism of $C^*$-algebras

$$\begin{aligned} \overline{\Psi _B(M)}/{\mathcal {K}} \longrightarrow C(S^*M)\oplus C(S^*X,{\mathcal {B}}(L^2({\mathbb {R}}_+)\oplus {\mathbb {C}})), \end{aligned}$$

where ${\mathcal {K}}\subset \overline{\Psi _B(M)}$ is the ideal of compact operators.

Note that the mappings in Theorem 1 are not surjective: first, the interior and boundary symbols satisfy additional properties (3) and (7). Moreover, they satisfy certain compatibility conditions.

The regularized trace ${{\,\mathrm{Tr}\,}}'$ of a boundary symbol $a_X\in \Sigma _X$ is defined by the formula

$$\begin{aligned} {{\,\mathrm{Tr}\,}}'a_X(x',\xi ') = \Pi '_{\eta _n} g(x',\xi ',\eta _n,\eta _n) + q(x',\xi '). \end{aligned}$$

(9)

The mapping

$$\begin{aligned} {{\,\mathrm{Tr}\,}}':\Sigma _X\longrightarrow C^\infty (S^*X) \end{aligned}$$

does not possess the trace property. More precisely, the following trace defect formula was obtained in [8, Section 2.4, Lemma 2.1]: given $a_{X,1}, a_{X,2}\in \Sigma _X$, we have

$$\begin{aligned} {{\,\mathrm{Tr}\,}}'[a_{X,1}, a_{X,2}] = -i\Pi '\left( \frac{\partial a_1(\xi _n)}{\partial \xi _n}a_2(\xi _n)\right) = i\Pi '\left( a_1(\xi _n)\frac{\partial a_2(\xi _n)}{\partial \xi _n}\right) , \end{aligned}$$

(10)

where $a_1, a_2$ are the principal symbols of $a_{X,1}, a_{X,2}$ respectively.

3 $\Gamma$-Boutet de Monvel operators: Fredholm property

Group actions and shift operators. Let $\Gamma$ be a discrete finitely generated group of isometries $\gamma :M\rightarrow M$, which preserve the boundary $\gamma (X)=X$. We suppose that the local coordinates near the boundary are chosen such that $x_n$ is a $\Gamma$-invariant function. Given $\gamma \in \Gamma$, we define the shift operator

$$\begin{aligned}&T_\gamma :L^2(M) \oplus L^2(X)\longrightarrow L^2(M) \oplus L^2(X),\\&(u(x),v(x')) \longmapsto (u(\gamma ^{-1}(x)),v(\gamma ^{-1}(x'))). \end{aligned}$$

This operator is unitary if we equip the $L^2$-spaces with the norms, defined by the volume forms associated with the Riemannian metric. The mapping $\gamma \mapsto T_\gamma$ defines a unitary representation of $\Gamma$ on $L^2(M) \oplus L^2(X)$.

It is known that compositions $T_\gamma {\mathcal {D}}T_\gamma ^{-1}$, where ${\mathcal {D}}$ is a Boutet de Monvel operator and $\gamma \in \Gamma$, is also a Boutet de Monvel operator. Moreover, the interior and boundary symbols of $T_\gamma {\mathcal {D}}T_\gamma ^{-1}$ are equal to

$$\begin{aligned}&\sigma _M(T_\gamma {\mathcal {D}}T_\gamma ^{-1})(x,\xi )=\sigma _M({\mathcal {D}})(\partial \gamma ^{-1}(x,\xi )), \\&\sigma _X(T_\gamma {\mathcal {D}}T_\gamma ^{-1})(x',\xi ')=\sigma _X({\mathcal {D}})(\partial \gamma ^{-1}(x',\xi ')). \end{aligned}$$

Here the actions of $\Gamma$ on M and X are lifted to the bundles ${T}^*M$ and $T^*X$ using codifferentials $\partial \gamma =(d\gamma ^t)^{-1}$ of the corresponding diffemorphisms (here $d\gamma$ is the differential of $\gamma$, while $d\gamma ^t$ is its dual mapping of the cotangent bundle).

$\Gamma$-Boutet de Monvel operators. Let us recall the definition of smooth crossed products (see [28] or [22]). Let ${\mathcal {A}}$ be a Fréchet algebra with seminorms $\Vert \cdot \Vert _m$, $m\in {\mathbb {N}}$, and $\Gamma$ be a group of polynomial growth in the sense of Gromov (see [25]), acting on ${\mathcal {A}}$ by automorphisms $a\mapsto \gamma (a)$, where $a\in {\mathcal {A}}$ and $\gamma \in \Gamma$. Then the smooth crossed product denoted by ${\mathcal {A}}\rtimes \Gamma$ is the vector space of functions $f:\Gamma \rightarrow {\mathcal {A}}$, which rapidly decay at infinity in the sence of the following estimates:

$$\begin{aligned} \Vert f(\gamma )\Vert _m\le C_{m,N}(1+|\gamma |)^{-N} \end{aligned}$$

for all $N,m\in {\mathbb {N}}$ and $\gamma \in \Gamma$, where the constant $C_{m,N}$ does not depend on $\gamma$. Here $|\gamma |$ is the length of $\gamma$ in the word metric on $\Gamma$. Finally, we assume that the action of $\Gamma$ on ${\mathcal {A}}$ satisfies the following property: given $m\in {\mathbb {N}}$, there exists $k\in {\mathbb {N}}$ and a real polynomial P(z) such that the following inequality

$$\begin{aligned} \Vert \gamma (a)\Vert _m\le P(|\gamma |)\Vert a\Vert _k \end{aligned}$$

holds for all a and $\gamma$. The product in ${\mathcal {A}}\rtimes \Gamma$ is defined by the formula:

$$\begin{aligned} \{f_1(\gamma )\}\cdot \{f_2(\gamma )\}=\left\{ \sum _{\gamma _1\gamma _2=\gamma }f_1(\gamma _1)\gamma _1(f_2(\gamma _2))\right\} . \end{aligned}$$

(11)

It can be shown that the right hand side in (11) is an element of ${\mathcal {A}}\rtimes \Gamma$, i.e., this space is an algebra.

The action of $\Gamma$ on Fréchet algebras $C^\infty (S^*M), \Sigma _X$ and $\Psi _B(M)$ satisfies the conditions above, and the following smooth crossed products are defined: $C^\infty (S^*M)\rtimes \Gamma , \Sigma _X\rtimes \Gamma$ and $\Psi _B(M)\rtimes \Gamma$. Elements $\left\{ {\mathcal {D}}_\gamma \right\} _{\gamma \in \Gamma }$ in the smooth crossed product $\Psi _B(M)\rtimes \Gamma$ define operators

$$\begin{aligned} \left\{ {\mathcal {D}}_\gamma \right\} _{\gamma \in \Gamma }\quad \longmapsto \quad \sum _{\gamma \in \Gamma } {\mathcal {D}}_\gamma T_\gamma : L^2(M)\oplus L^2(X)\rightarrow L^2(M)\oplus L^2(X). \end{aligned}$$

(12)

Definition 1

Operators in (12) are called $\Gamma$-Boutet de Monvel operators.

Definition 2

The symbol of operator (12) is a pair $\sigma ({\mathcal {D}})=(\sigma _M({\mathcal {D}}),\sigma _X({\mathcal {D}}))$, which consists of the interior and the boundary symbols

$$\begin{aligned} \sigma _M({\mathcal {D}})=\{\sigma (A_\gamma )\}_{\gamma \in \Gamma }\in C^\infty (S^*M)\rtimes \Gamma , \quad \sigma _X({\mathcal {D}})=\{\sigma _X({\mathcal {D}}_\gamma )\}_{\gamma \in \Gamma }\in \Sigma _X\rtimes \Gamma .\nonumber \\ \end{aligned}$$

(13)

The operators (12) form an algebra and symbols (13) enjoy the composition formula: given ${\mathcal {D}}_1,{\mathcal {D}}_2\in \Psi _B(M)\rtimes \Gamma$, we have $\sigma _M({\mathcal {D}}_1{\mathcal {D}}_2)=\sigma _M({\mathcal {D}}_1)\sigma _M({\mathcal {D}}_2)$ and $\sigma _X({\mathcal {D}}_1 {\mathcal {D}}_2)=\sigma _X({\mathcal {D}}_1)\sigma _X({\mathcal {D}}_2)$.

Operators, acting between ranges of projections. A matrix $\Gamma$-Boutet de Monvel operator is a triple $({\mathcal {D}}, {\mathcal {P}}_1, {\mathcal {P}}_2)$ such that

• ${\mathcal {D}}\in \mathrm{Mat}_N(\Psi _B(M)\rtimes \Gamma )$ is a matrix with the components in $\Psi _B(M)\rtimes \Gamma$;

• ${\mathcal {P}}_j\in \mathrm{Mat}_N((C^\infty (M)\oplus C^\infty (X))\rtimes \Gamma ),\; j=1,2,$ are projections;

• the following relation is satisfied ${\mathcal {P}}_2{\mathcal {D}}{\mathcal {P}}_1={\mathcal {D}}.$

Then we define the operator

$$\begin{aligned} {\mathcal {D}}: {\mathcal {P}}_1(L^2(M,{\mathbb {C}}^N)\oplus L^2(X,{\mathbb {C}}^N)) \longrightarrow {\mathcal {P}}_2(L^2(M,{\mathbb {C}}^N)\oplus L^2(X,{\mathbb {C}}^N)) \end{aligned}$$

(14)

Acting between the ranges of the projections on the $L^2$-spaces. Denote operator (14) by $({\mathcal {D}}, {\mathcal {P}}_1, {\mathcal {P}}_2)$.

Definition 3

Operator $({\mathcal {D}}, {\mathcal {P}}_1, {\mathcal {P}}_2)$ is elliptic, if there exists a matrix operator $({\mathcal {R}}, {\mathcal {P}}_2, {\mathcal {P}}_1)$ such that the following equalities hold

$$\begin{aligned} \sigma ({\mathcal {D}})\sigma ({\mathcal {R}})=\sigma ({\mathcal {P}}_2),\qquad \sigma ({\mathcal {R}})\sigma ({\mathcal {D}})=\sigma ({\mathcal {P}}_1). \end{aligned}$$

(15)

Theorem 2

An elliptic operator (14) has Fredholm property.

Proof

The proof is standard (see, for instance, [15]). Indeed, let $({\mathcal {D}}, {\mathcal {P}}_1, {\mathcal {P}}_2)$ be elliptic. Denote the symbol of ${\mathcal {R}}$ by

$$\begin{aligned} \left( \{r_{\gamma ,M}\}_{\gamma \in \Gamma },\{r_{\gamma ,X}\}_{\gamma \in \Gamma }\right) \in \Bigl (C^\infty (S^*M) \oplus \Sigma _X\Bigr )\rtimes \Gamma . \end{aligned}$$

We represent the operators ${\mathcal {D}}, {\mathcal {R}}$ as

$$\begin{aligned} {\mathcal {D}}=\sum _{\gamma \in \Gamma }{{\mathcal {D}}_\gamma T_\gamma },\qquad {\mathcal {R}}=\sum _{\gamma \in \Gamma }{{\mathcal {R}}_\gamma T_\gamma }, \end{aligned}$$

where $\Vert {\mathcal {D}}_\gamma \Vert _m\rightarrow 0, \Vert {\mathcal {R}}_\gamma \Vert _m\rightarrow 0$ rapidly, for all $m\in {\mathbb {N}}$, and write their symbols as

$$\begin{aligned} \sigma ({\mathcal {D}})=\left\{ \sigma ({\mathcal {D}}_\gamma )\right\} ,\qquad \sigma ({\mathcal {R}})=\left\{ \sigma ({\mathcal {R}}_\gamma )\right\} . \end{aligned}$$

Then (15) is equivalent to

$$\begin{aligned} \sum _{\gamma _1\gamma _2=\gamma }{\sigma ({\mathcal {D}}_{\gamma _1})\gamma _1\sigma ({\mathcal {R}}_{\gamma _2})}=\sigma ({\mathcal {P}}_{2\gamma }),\qquad \sum _{\gamma _1\gamma _2=\gamma }{\sigma ({\mathcal {R}}_{\gamma _1})\gamma _1\sigma ({\mathcal {D}}_{\gamma _2})}=\sigma ({\mathcal {P}}_{1\gamma }).\nonumber \\ \end{aligned}$$

(16)

We obtain

$$\begin{aligned}&{\mathcal {R}}{\mathcal {D}} = \sum _{\gamma _1\in \Gamma }{{\mathcal {R}}_{\gamma _1} T_{\gamma _1}}\sum _{\gamma \in \Gamma }{{\mathcal {D}}_{\gamma }T_{\gamma }}\nonumber \\&\quad = \sum _{\gamma ,\gamma _1\in \Gamma }{{\mathcal {R}}_{\gamma _1}T_{\gamma _1}{\mathcal {D}}_{\gamma _1^{-1}\gamma }T_{\gamma _1^{-1}\gamma }}= \sum _{\gamma \in \Gamma }\left( \sum _{\gamma _1\in \Gamma }{\mathcal {R}}_{\gamma _1}\gamma _1({\mathcal {D}}_{\gamma _1^{-1}\gamma })\right) T_{\gamma }. \end{aligned}$$

(17)

It follows from (16) that

$$\begin{aligned} \sigma \left( \sum _{\gamma _1\in \Gamma }{\mathcal {R}}_{\gamma _1}\gamma _1({\mathcal {D}}_{\gamma _1^{-1}\gamma })\right) = \sum _{\gamma _1\in \Gamma }\sigma ({\mathcal {R}}_{\gamma _1})\gamma _1\sigma ({\mathcal {D}}_{\gamma _1^{-1}\gamma })= \sigma ({\mathcal {P}}_{1\gamma }). \end{aligned}$$

Hence, by Theorem 1 we have

$$\begin{aligned} \sum _{\gamma _1\in \Gamma }{\mathcal {R}}_{\gamma _1}\gamma _1({\mathcal {D}}_{\gamma _1^{-1}\gamma }) = {\mathcal {P}}_{1\gamma }+K_\gamma , \end{aligned}$$

(18)

where $K_\gamma$ is a compact operator and $\Vert K_\gamma \Vert _m\rightarrow 0$ as $|\gamma |\rightarrow \infty$. Therefore, (17) and (18) give

$$\begin{aligned} {\mathcal {R}}{\mathcal {D}}={\mathcal {P}}_1 + K_1, \end{aligned}$$

where $K_1$ is a compact operator.

Similarly one can prove that ${\mathcal {D}}{\mathcal {R}}={\mathcal {P}}_2+K_2$, where $K_2$ is a compact operator. Then by Atkinson’s theorem operator (14) has Fredholm property. $\square$

$\Gamma$-Shapiro-Lopatinskii condition. Consider a nonlocal boundary value problem

$$\begin{aligned} \left\{ \begin{matrix} \displaystyle {Du}&{}=&{}f&{}\text { on } M, \\ \displaystyle {\sum \limits _{0\le j <d}\left. B_j \frac{\partial ^j u}{\partial x^j_n} \right| _{x_n=0}}&{}=&{}g&{}\text { on } X, \end{matrix} \right. \end{aligned}$$

(19)

where $D=\sum \nolimits _{\gamma \in \Gamma }D_\gamma T_\gamma$, ${\text {ord}}D=d$, $B_j=\sum \nolimits _{\gamma \in \Gamma }B_{j\gamma } T_\gamma$, ${\text {ord}}B_j=b-j$. Here $\{D_\gamma \}$ are differential operators on M and $\{B_{j\gamma }\}$ are differential operators on X. The functions u, f, g are elements of spaces

$$\begin{aligned} u\in P_1H^s(M,{\mathbb {C}}^N),\quad f\in P_2H^{s-d}(M,{\mathbb {C}}^N),\quad g\in P_3H^{s-b-1/2}(X,{\mathbb {C}}^{N_d}), \end{aligned}$$

where $P_1,P_2$ are $N\times N$ matrix projections over $C^\infty (M)\rtimes \Gamma$ and $P_3$ is $N_d\times N_d$ matrix projection over $C^\infty (X)\rtimes \Gamma$. We realize problem (19) as the following operator (cf. (14))

$$\begin{aligned}&(D,B): P_1H^s(M,{\mathbb {C}}^N)\rightarrow P_2H^{s-d}(M,{\mathbb {C}}^N)\oplus P_3H^{s-b-1/2}(X,{\mathbb {C}}^{N_d}) \nonumber \\&u\longmapsto \displaystyle {\left( Du, \sum \limits _{0\le j <d}\left. B_j \frac{\partial ^j u}{\partial x^j_n} \right| _{x_n=0}\right) }. \end{aligned}$$

(20)

We suppose here that $D=P_2DP_1$ and $P_3B=B$, where $B=(B_0,...,B_{d-1})$.

Note that in the local case, i.e., when $D,B_j,P_j$ do not contain shift operators $T_\gamma$ for $\gamma \ne e$, problem (20) is a classical boundary value problem. We give the ellipticity condition of problem (20), which generalizes the Shapiro–Lopatinskii condition. To this end, we write (19) in a neighborhood of the boundary as

$$\begin{aligned} \left\{ \begin{matrix} \displaystyle {\sum \limits _{\gamma \in \Gamma }D_\gamma \left( x_n, -i\frac{\partial }{\partial x_n}, x', -i\frac{\partial }{\partial x'} \right) T_\gamma u}&{}=&{}f,\\ \displaystyle {\sum \limits _{j}\sum \limits _{\gamma \in \Gamma }\left. B_{j\gamma }\left( x', -i\frac{\partial }{\partial x'} \right) T_\gamma \frac{\partial ^j u}{\partial x^j_n} \right| _{x_n=0}}&{}=&{}g. \end{matrix} \right. \end{aligned}$$

(21)

To state an analogue of Shapiro–Lopatinskii condition for problem (21) let us define an analogue of the Calderon bundle in this situation. To this end, we define for simplicity ${\mathcal {A}}=C^\infty (S^*X)\rtimes \Gamma$ and consider the homogeneous system of linear ordinary differential equations

$$\begin{aligned} \sigma (D)\left( 0, -i\frac{d}{dx_n},x',\xi '\right) u(x_n)=0, \end{aligned}$$

(22)

where $u\in C^\infty ({\mathbb {R}}, {\mathcal {A}}\otimes {\mathbb {C}}^N)$. Suppose that the triple $(D,P_1,P_2)$ is interior elliptic (this means that (15) holds for the interior symbols of of these operators). Then the solutions $u\in P_1C^\infty ({\mathbb {R}}, {\mathcal {A}}\otimes {\mathbb {C}}^N)$ of (22) either tends to zero as $x_n\rightarrow +\infty$ or $x_n\rightarrow -\infty$. Moreover, only the solution $u(x_n)\equiv 0$ tends to zero as $x_n\rightarrow \infty$ and $x_n\rightarrow -\infty$. Denote by $L_\pm (D)$ the Cauchy data subspaces of solutions of (22), which tend to zero as $x_n\rightarrow \pm \infty$:

$$\begin{aligned}&L_\pm (D)=\Bigg \{ \bigg .W=(W_0,...W_{d-1})\in (P_1{\mathcal {A}}\otimes {\mathbb {C}}^N)\otimes {\mathbb {C}}^d \bigg | \exists u\in P_1C^\infty ({\mathbb {R}}, {\mathcal {A}}\otimes {\mathbb {C}}^N),\Bigg .\\&\quad \Bigg . \sigma (D)\left( 0, -i\frac{d}{dx_n},x',\xi '\right) u(x_n){=}0, u(x_n)\rightarrow 0 \text { as } x_n\rightarrow \pm \infty , W_j{=}\left. \frac{\partial ^j u}{\partial x^j_n}\right| _{x_n=0} \Bigg \}. \end{aligned}$$

Then $L_\pm (D)\subset {\mathcal {A}}\otimes {\mathbb {C}}^{N_d}$ are right ${\mathcal {A}}$-modules. Moreover, they are finitely generated and projective. Denote by $Q\in \mathrm{Mat}_{N_d}({\mathcal {A}})$ a projection defining $L_+(D)$.

Definition 4

Problem (21) with the interior elliptic triple $(D,P_1,P_2)$ satisfies the Shapiro–Lopatinskii condition if the triple $(\sigma (D),Q,P_3)$ is elliptic on $S^*M$, i.e., there exists triple $(\sigma (R),P_3,Q)$ such that $\sigma (R)\sigma (B)=\sigma (Q)$, $\sigma (B)\sigma (R)=\sigma (P_3)$ in $\mathrm{Mat}_{N_d}(C^\infty (S^*X)\rtimes \Gamma )$.

Theorem 3

Let the operator $D: P_1H^s(M,{\mathbb {C}}^N)\rightarrow P_2H^{s-d}(M,{\mathbb {C}}^N)$ in (19) be elliptic and the Shapiro–Lopatinskii condition be satisfied (in the sense of Definition 4). Then (20) has Fredholm property.

4 Two de Rham complexes for manifolds with fibered boundary

Given a smooth manifold M with boundary $\partial M$, we suppose that the boundary is the total space of a locally trivial fiber bundle $\pi :\partial M\rightarrow X$ with the fiber F. Then the pair $(M,\pi )$ is called a manifold with fibered boundary.

The embedding $i:\partial M\rightarrow M$ induces the restriction mapping $i^*:\Omega ^*(M)\rightarrow \Omega ^*(\partial M)$ of the differential forms to the boundary. The projection $\pi$ defines the induced mapping $\pi ^*:\Omega ^*(X)\rightarrow \Omega ^*(\partial M)$ and the mapping

$$\begin{aligned} \pi _*:\Omega _c^*(\partial M)\longrightarrow \Omega _c^{*-\nu }(X),\quad \nu ={\text {dim}}F, \end{aligned}$$

(23)

of integration of compactly supported differential forms over the fiber F (see, for instance, [29]). Here we suppose that the fibers of $\pi$ have orientation continuously depending on the point of the base. Let us recall the definition of the integral in (23).

Definition 5

Given a form $\omega \in \Omega _c^k(\partial M)$, its integral over the fibers of $\pi :\partial M\rightarrow X$ is the differential form denoted by $\pi _*{\omega }\in \Omega _c^{k-\nu }(X)$ and such that

$$\begin{aligned} \int \limits _{X}{\Big (\pi _*{\omega }\Big )\wedge \omega _1} = \int _{\partial M}{\omega }\wedge \pi ^*\omega _1 \end{aligned}$$

for all differential forms $\omega _1$ on X.

The following properties are valid:

• 1. $\pi _*(\omega \wedge \pi ^*\omega _1)=(\pi _*\omega )\wedge \omega _1,$ for all forms $\omega \in \Omega _c^k(\partial M), \omega _1\in \Omega _c^l(X)$;

• 2. $d(\pi _*\omega ) = (-1)^\nu \pi _*(d\omega )$, for all forms $\omega \in \Omega _c^k(\partial M)$.

For simplicity, we suppose that X and $\partial M$ are oriented, and the orientation of $\partial M$ is given by the orientations of the fibers and the base.

Let us consider the graded morphism

$$\begin{aligned} (\Omega _c^*(M),d) {\mathop {\longrightarrow }\limits ^{\pi _*i^*}}(\Omega _c^{*-\nu }(X),d),\quad d\pi _*i^*=(-1)^\nu \pi _*i^*d \end{aligned}$$

(24)

of the de Rham complexes on M and X. Denote the cone of $\pi _*i^*$ by $(\Omega ^*(M,\pi ),\partial )$, where

$$\begin{aligned} \Omega _c^j(M,\pi )=\Omega _c^j(M)\oplus \Omega _c^{j-\nu -1}(X), \quad \partial =\left( \begin{array}{cc} d &{} 0\\ \pi _*i^*&{} (-1)^{\nu +1} d \end{array}\right) . \end{aligned}$$

(25)

Note that

$$\begin{aligned}&\partial ^2= \left( \begin{array}{cc} d^2 &{} 0\\ \pi _*i^*d+(-1)^{\nu +1} d(\pi _*i^*)&{} (-1)^{2\nu +2} d^2 \end{array}\right) \\&\quad = \left( \begin{array}{cc} 0 &{} 0\\ \pi _*i^*d+(-1)^{\nu +1} d\pi _*i^*&{} 0 \end{array}\right) = 0. \end{aligned}$$

The last equality follows from (24).

We denote the cohomology groups of the complex $(\Omega _c^*(M,\pi ),\partial )$ by $H_c^* (M,\pi )$.

Also, we consider the complex $({\widetilde{\Omega }}^*(M,\pi ),{{\widetilde{\partial }}})$:

$$\begin{aligned} {\widetilde{\Omega }}^j(M,\pi )=\{(\omega ,\omega _X)\in \Omega ^j(M)\oplus \Omega ^j(X)\;|\; i^*\omega =\pi ^*\omega _X\}, \quad {\widetilde{\partial }}=\left( \begin{array}{cc} d &{} 0\\ 0&{} d \end{array}\right) .\nonumber \\ \end{aligned}$$

(26)

Note that $di^*\omega =d\pi ^*\omega _X$ since $i^*\omega =\pi ^*\omega _X$. Hence, $i^*d\omega =\pi ^*d\omega _X$, since $i^*$ and $\pi ^*$ are the induced mappings on differential forms. Hence, ${\widetilde{\partial }}$ is well defined. We denote the cohomology groups of the complex $({\widetilde{\Omega }}^*(M,\pi ),{\widetilde{\partial }})$ by ${\widetilde{H}}^*(M,\pi )$.

Component-wise exterior product of differential forms gives the product

$$\begin{aligned} \wedge : \Omega _c^j(M,\pi )\times {\widetilde{\Omega }}^k(M,\pi )\longrightarrow \Omega _c^{j+k}(M,\pi ). \end{aligned}$$

This operation enjoys the Leibniz rule

$$\begin{aligned} \partial (a\wedge b)=\partial a\wedge b+(-1)^{j}a\wedge {{\widetilde{\partial }}} b,\quad a\in \Omega _c^j(M,\pi ), b\in {\widetilde{\Omega }}^k(M,\pi ). \end{aligned}$$

(27)

Indeed, given

$$\begin{aligned} a= \left( \begin{array}{c} \omega \\ \omega _X \end{array}\right) \quad \text {and}\quad b= \left( \begin{array}{c} \omega '\\ \omega '_X \end{array}\right) , \end{aligned}$$

we have

$$\begin{aligned}&\partial (a\wedge b)=\partial \left( \begin{array}{c} \omega \wedge \omega '\\ \omega _X\wedge \omega '_X \end{array}\right) = \left( \begin{array}{c} d(\omega \wedge \omega ')\\ \pi _*i^*(\omega \wedge \omega ')+(-1)^{\nu +1}d(\omega _X\wedge \omega '_X) \end{array}\right) \\&\quad = \left( \begin{array}{c} d\omega \wedge \omega '+(-1)^j\omega \wedge d\omega '\\ \pi _*(i^*\omega \wedge i^*\omega ')+(-1)^{\nu +1}d\omega _X\wedge \omega '_X+(-1)^{\nu +1}(-1)^{j-\nu -1}\omega _X\wedge d\omega '_X \end{array}\right) \\&\quad = \left( \begin{array}{c} d\omega \wedge \omega '\\ \pi _*i^*(\omega )\wedge \omega '_X+(-1)^{\nu +1}d\omega _X\wedge \omega '_X \end{array}\right) + \left( \begin{array}{c} (-1)^j\omega \wedge d\omega '\\ (-1)^j\omega _X\wedge d\omega '_X \end{array}\right) \\&\quad =\partial \left( \begin{array}{c} \omega \\ \omega _X \end{array}\right) \wedge \left( \begin{array}{c} \omega '\\ \omega '_X \end{array}\right) + (-1)^j\left( \begin{array}{c} \omega \\ \omega _X \end{array}\right) \wedge {{\widetilde{\partial }}} \left( \begin{array}{c} \omega '\\ \omega '_X \end{array}\right) . \end{aligned}$$

It follows from the Leibniz rule (27) that $\wedge$ defines a product in cohomology

$$\begin{aligned} \wedge :H_c^j(M,\pi )\times {\widetilde{H}}_c^k(M,\pi )\longrightarrow H_c^{j+k}(M,\pi ). \end{aligned}$$

(28)

Finally, we suppose that M and X are oriented manifolds and their orientations are compatible with the orientation of the fibers in the following way. Denote $n=\mathrm{dim}M$ and choose as positively oriented the form $(-1)^n dt_1\wedge \cdots \wedge dt_\nu \wedge dy_1\wedge \cdots \wedge dy_k\wedge dx_n$, where $t_1,\ldots ,t_\nu$ are some positively oriented coordinates on the fiber, $y_1,\ldots ,y_{n-\nu -1}$ are some positively oriented coordinates on X, while $x_n\ge 0$ is a defining function of the boundary.

We define the linear functional

$$\begin{aligned} \begin{array}{rcc} \displaystyle \langle \cdot ,[M,\pi ]\rangle :H_c^* (M,\pi ) &{} \longrightarrow &{} {\mathbb {C}}\\ (\omega ,\omega _X) &{} \longmapsto &{} \displaystyle \int _M \omega - (-1)^n\int _X \omega _X. \end{array} \end{aligned}$$

To prove that this functional is well defined, it suffices to show that it vanishes on exact forms. We choose compactly supported forms

$$\begin{aligned} \omega =f(t,y,x_n)dt_1\wedge \cdots \wedge dy_{n-\nu -1},\qquad \omega _X=g(y)dy_1\wedge \cdots \wedge dy_{n-\nu -1} \end{aligned}$$

and compute

$$\begin{aligned}&\langle \partial (\omega ,\omega _X),[M,\pi ]\rangle = \int \limits _M d\omega -(-1)^n\int \limits _X \left( \pi _*i^*\omega +(-1)^{\nu +1}d\omega _X\right) \\&\quad =\int \limits _{\partial M} i^*\omega -(-1)^n\int \limits _X \left( \pi _*i^*\omega \right) \\&\quad =(-1)^n\!\!\!\!\int \limits _{{\mathbb {R}}^\nu \times {\mathbb {R}}^{n-\nu -1}}\!\!\!\!\!\!\!f(t,y,0)dt_1\\&\qquad ...dy_{n-\nu -1}-(-1)^n\!\!\!\!\int \limits _{{\mathbb {R}}^{n-\nu -1}}{\left( \int \limits _{{\mathbb {R}}^\nu }{f(t,y,0)dt_1...dt_\nu }\right) dy_1...dy_{n-\nu -1}}=0. \end{aligned}$$

5 Chern character of elliptic symbols

In this section, we define the Chern character for symbols of elliptic operators (14). The definition uses noncommutative differential forms on cotangent bundles $T^*M$ and $T^*X$ which we now define.

Noncommutative differential forms. Regularized trace. Let $C^\infty _{tr}(T^*M)\subset C^\infty (T^*M)$ be the subalgebra of classical symbols of order $\le 0$, which satisfy the transmission property (see (3)). Let $\Omega _{T^*M}\subset \Omega (T^*M)$ be the subalgebra of differential forms on $T^*M$ with coefficients in $C^\infty _{tr}(T^*M)$.

By ${{\widetilde{\Sigma }}}_X\subset C^\infty (T^*X,{\mathcal {B}}({\overline{H}}_+\oplus {\mathbb {C}}))$ we denote the subalgebra of operator families $a_X(x',\xi ')$, $(x',\xi ')\in T^*X$, such that the family $a_X(x',\xi ')$ is defined as in (7), where $b\in C^\infty (T^*X,H_-), c\in C^\infty (T^*X,H_+)$, $g\in C^\infty (T^*X,H_+\otimes H_-)$, $q(x',\xi ')\in C^\infty (T^*X)$, while function $a(x',0,\xi ',\xi _n)$ is the restriction to the boundary $\partial (T^*M)\simeq T^*X\times {\mathbb {R}}$ of a symbol $a(x',x_n,\xi ',\xi _n)\in C^\infty _{tr}(T^*M)$.

By $\Omega _{T^*X}\subset \Omega (T^*X,{\mathcal {B}}({\overline{H}}_+\oplus {\mathbb {C}}))$ denote the subalgebra of differential forms on $T^*X$ with coefficients in ${{\widetilde{\Sigma }}}_X$. Let us consider the action of $\Gamma$ on the Frechet algebras $\Omega _{T^*M}$ and $\Omega _{T^*X}$ of differential forms and corresponding smooth crossed products

$$\begin{aligned} \Omega _{T^*M}\rtimes \Gamma \text { and }\Omega _{T^*X}\rtimes \Gamma . \end{aligned}$$

These crossed products are differential graded algebras with respect to the gradings defined by the gradings of the differential forms and the differentials defined by the exterior differential on differential forms.

Given $\gamma \in \Gamma$, we define the mappings (cf. [20, 30])

$$\begin{aligned} \tau ^{\gamma }:\Omega _{T^*M}\rtimes \Gamma \longrightarrow \Omega _{T^*M^{\gamma }},\qquad \tau ^\gamma _{X}:\Omega _{T^*X}\rtimes \Gamma \longrightarrow \Omega _{T^*X^{\gamma }}. \end{aligned}$$

(29)

Here $M^\gamma \subset M$ and $X^\gamma \subset X$ are the submanifolds of fixed points of isometry $\gamma$. To define these mappings, we introduce auxiliary notations. Denote the closure of $\Gamma$ in the compact Lie group of isometries of M by ${\overline{\Gamma }}$. This closure is a compact Lie group. Let $C^\gamma \subset {\overline{\Gamma }}$ be the centralizer¹ of $\gamma$. The centralizer is a closed Lie subgroup in ${\overline{\Gamma }}$. The elements of centralizer are denoted by h, while the induced Haar measure on the centralizer is denoted by dh. Next, for an element $\gamma '\in \langle \gamma \rangle$ in the conjugacy class of $\gamma$, we fix an arbitrary element $z=z(\gamma ,\gamma ')$, which conjugates $\gamma$ and $\gamma '=z\gamma z^{-1}$. Any such element defines diffeomorphism $\partial z:T^*M^\gamma \rightarrow T^*M^{\gamma '}$ of the corresponding fixed point manifolds.

We define the first functional in (29) by

$$\begin{aligned} \tau ^\gamma (\omega )= \sum _{\gamma '\in \langle \gamma \rangle }\;\;\; \int _{C^\gamma } \Bigl .h^*\bigl ( {z}^*\omega (\gamma ') \bigr )\Bigr |_{T^*M^\gamma } dh,\quad \text {where }\omega \in \Omega _{T^*M}\rtimes \Gamma , \end{aligned}$$

(30)

and the second functional by

$$\begin{aligned} \tau ^\gamma _X(\omega _X)= \sum _{\gamma '\in \langle \gamma \rangle }\;\;\; \int _{C^\gamma } {{\,\mathrm{Tr}\,}}_X \left( \Bigl .h^*\bigl ( {z}^*\omega _X(\gamma ') \bigr )\bigr |_{T^*X^\gamma } \right) dh,\quad \text {where }\omega _X\in \Omega _{T^*X}\rtimes \Gamma .\nonumber \\ \end{aligned}$$

(31)

Here

$$\begin{aligned} {{\,\mathrm{Tr}\,}}_X \left( \sum _I \omega _I(t)dt^I\right) = \sum _I {{\,\mathrm{Tr}\,}}'(\omega _I(t))dt^I, \end{aligned}$$

where ${{\,\mathrm{Tr}\,}}':{{\widetilde{\Sigma }}}_X \rightarrow C^\infty (T^*X)$ is the regularized trace defined earlier in (9).

Proposition 1

The following assertions hold:

• 1. The summands in (30) and (31) do not depend on the choice of the elements z.

• 2. Functionals (30) and (31) have the following properties:

  $$\begin{aligned}&\tau ^\gamma (\omega _1\wedge \omega _2)=(-1)^{\deg \omega _1\deg \omega _2}\tau ^\gamma (\omega _2\wedge \omega _1),\text { for all } \omega _1,\omega _2\in \Omega _{T^*M}\rtimes \Gamma ,\nonumber \\ \end{aligned}$$

  (32)
  $$\begin{aligned}&d\tau ^\gamma _X(\omega ) = \tau ^\gamma _X(d\omega ),\quad \text {for all }\omega \in \Omega _{T^*X}\rtimes \Gamma .\nonumber \\ \end{aligned}$$

  (33)

Proof

Let us prove 1. Indeed, let $z_1$ be another element such that $\gamma '=z_1\gamma z_1^{-1}$. Then for the summands in (30) we have:

$$\begin{aligned}&\int _{C^\gamma } \Bigl .h^*\bigl ( z_1^*\omega (\gamma ') \bigr )\Bigr |_{T^*M^\gamma } dh= \int _{C^\gamma } \Bigl .(z_1h)^*\bigl ( \omega (\gamma ') \bigr )\Bigr |_{T^*M^\gamma } dh\\&\quad = \int _{C^\gamma } \Bigl .h^*\bigl (z^* \omega (\gamma ') \bigr )\Bigr |_{T^*M^\gamma } dh. \end{aligned}$$

Here in the last equality we made the change of variable $z_1h=zh'$ in the integral and used the invariance of the Haar measure. Equality (31) is proved similarly.

Now let us move on to part 2. Let us show that $\tau ^\gamma$ is a graded trace, i.e., equality (32) holds. It suffices to prove this property for the following forms $\omega _1, \omega _2$:

$$\begin{aligned} \omega _1(\gamma )=\left\{ \begin{matrix} a, &{}\gamma =\gamma _1,\\ 0, &{}\gamma \ne \gamma _1, \end{matrix} \right. \qquad \omega _2(\gamma )=\left\{ \begin{matrix} b, &{}\gamma =\gamma _2,\\ 0, &{}\gamma \ne \gamma _2. \end{matrix} \right. \end{aligned}$$

Then

$$\begin{aligned} (\omega _1\wedge \omega _2)(\gamma )=\left\{ \begin{matrix} 0, &{}\gamma \ne \gamma _1\gamma _2,\\ a\wedge {\gamma _1^*}^{-1}b, &{}\gamma =\gamma _1\gamma _2. \end{matrix} \right. \end{aligned}$$

Since (30) does not depend on the choice of z, we set $z=e\in \Gamma$.

On the one hand, for $\gamma =\gamma _1\gamma _2$ we have

$$\begin{aligned}&\tau ^\gamma (\omega _1\wedge \omega _2)= \int _{C^\gamma }\Bigl .h^*\Bigl ( a\wedge {\gamma _1^*}^{-1}b\Bigr )\Bigr |_{T^*M^\gamma }dh= \int _{C^\gamma }\Bigl .(h^* a\wedge h^*{\gamma _1^*}^{-1}b)\Bigr |_{T^*M^\gamma }dh\nonumber \\&\quad =\int _{C^\gamma }\Bigl .(h^* a\wedge \gamma ^*h^*{\gamma _1^*}^{-1}b)\Bigr |_{T^*M^\gamma }dh, \end{aligned}$$

(34)

since the form is integrated over ${T^*M^\gamma }$ and over this manifold we have $\gamma ^*={\mathrm{Id}}$. Since $h\in C^\gamma$ in (34), we have $h^*\gamma ^*=\gamma ^*h^*$. Hence, (34) gives us

$$\begin{aligned} \tau ^\gamma (\omega _1\wedge \omega _2)= \int _{C^\gamma }\Bigl .h^* a\wedge h^*\gamma ^*{\gamma _1^*}^{-1}b\Bigr |_{T^*M^\gamma }dh= \int _{C^\gamma }\Bigl .h^*\Bigl ( a\wedge \gamma _2^*b\Bigr )\Bigr |_{T^*M^\gamma }dh,\nonumber \\ \end{aligned}$$

(35)

since $\gamma =\gamma _1\gamma _2$ and $\gamma ^*{\gamma _1^*}^{-1}=\gamma _2^*$.

On the other hand, for $z=\gamma _2$, we have

$$\begin{aligned}&\tau ^\gamma (\omega _2\wedge \omega _1)= \int _{C^\gamma }\Bigl .h^*\Bigl (z^*\Bigl (b\wedge {\gamma _2^*}^{-1}a\Bigr )\Bigr )\Bigr |_{T^*M^\gamma }dh\\&\quad = (-1)^{\mathrm{deg}\omega _1\mathrm{deg}\omega _2}\int _{C^\gamma }\Bigl .h^*\Bigl (z^*\Bigl ( {\gamma _2^*}^{-1}a\wedge b\Bigr )\Bigr )\Bigr |_{T^*M^\gamma }dh \\&\quad =(-1)^{\mathrm{deg}\omega _1\mathrm{deg}\omega _2}\int _{C^\gamma }\Bigl .h^*\Bigl ( a\wedge \gamma _2^*b\Bigr )\Bigr |_{T^*M^\gamma }dh\\&\quad = (-1)^{\mathrm{deg}\omega _1\mathrm{deg}\omega _2}\tau ^\gamma (\omega _1\wedge \omega _2). \end{aligned}$$

Here in the last equality we used (35).

Now we prove equality (33):

$$\begin{aligned}&\tau ^\gamma _X(d\omega _X)= \sum _{\gamma '\in \langle \gamma \rangle }\; \int _{C^\gamma } {{\,\mathrm{Tr}\,}}_X \left( \Bigl .h^*\bigl ( {z}^*(d\omega _X(\gamma ')) \bigr )\bigr |_{T^*X^\gamma } \right) dh \\&\quad =\sum _{\gamma '\in \langle \gamma \rangle }\; \int _{C^\gamma } {{\,\mathrm{Tr}\,}}_X \left( \Bigl .d\bigl (h^*\bigl ( {z}^*(\omega _X(\gamma ')) \bigr )\bigr )\bigr |_{T^*X^\gamma } \right) dh\\&\quad = d\sum _{\gamma '\in \langle \gamma \rangle }\; \int _{C^\gamma } {{\,\mathrm{Tr}\,}}_X \left( \Bigl .h^*\bigl ( {z}^*(\omega _X(\gamma ')) \bigr )\bigr |_{T^*X^\gamma } \right) dh \\&\quad =d\tau ^\gamma _X(\omega _X). \end{aligned}$$

$\square$

Definition of the Chern character. Consider an elliptic operator $({\mathcal {D}}, {\mathcal {P}}_1, {\mathcal {P}}_2)$. For brevity, we frequently denote it simply by ${\mathcal {D}}$. We extend the interior symbols $\sigma _M({\mathcal {D}}), \sigma _M({\mathcal {R}})$ of the original operator and its almost inverse to $T^*M$ as smooth symbols, which have the transmission property. We extend the boundary symbols $\sigma _X({\mathcal {D}})$ and $\sigma _X({\mathcal {R}})$ to $T^*X$ as smooth symbols. We denote such extensions by

$$\begin{aligned} a,r\in C^\infty _{tr}(T^*M)\rtimes \Gamma , \qquad a_X,r_X\in {{\widetilde{\Sigma }}}_X \rtimes \Gamma . \end{aligned}$$

Suppose that these extensions are compatible, i.e., the principal symbol of the boundary symbol is equal to the restriction of the interior symbol to the boundary and the following equalities hold:

$$\begin{aligned} a=P_2aP_1,\quad r=P_1rP_2,\quad a_X=P'_2a_XP'_1,\quad r_X=P'_1r_XP'_2, \end{aligned}$$

(36)

where

$$\begin{aligned} P_j=\sigma _M({\mathcal {P}}_j),\qquad P'_j=\sigma _X({\mathcal {P}}_j). \end{aligned}$$

The desired extensions can be defined as follows. The interior symbols $\sigma _M({\mathcal {D}}), \sigma _M({\mathcal {R}})$ are extended to $T^*M$ by homogeneity of order zero and are then multiplied by a smooth cut-off function $\chi (|\xi |)$ equal to 0 for $|\xi |$ small and equal to 1 for $|\xi |\ge 1$.

Similarly, the boundary symbols $\sigma _X({\mathcal {D}}), \sigma _X({\mathcal {R}})$ are extended from $S^*X$ to the domain $|\xi '|\ge 1$ as twisted homogeneous functions of degree zero. Recall that a boundary symbol $a_X(x',\xi ')$ is twisted homogeneous of degree zero for $|\xi '|\ge 1$ if $\forall |\xi '|\ge 1, x'\in X$ and $\lambda \ge 1$ we have

$$\begin{aligned} a_X(x',\lambda \xi ')=\varkappa _\lambda a_X(x',\xi ')\varkappa ^{-1}_\lambda , \end{aligned}$$

where

$$\begin{aligned} \begin{array}{ccc} \varkappa _\lambda : H_+\oplus {\mathbb {C}} &{} \longrightarrow &{} H_+\oplus {\mathbb {C}}\\ (h(\xi _n),v) &{} \longmapsto &{} (\lambda ^{-1/2}h(\lambda ^{-1}\xi _n), v) \end{array} \end{aligned}$$

is the action of the dilation group.

Then we choose arbitrary extensions of the components of boundary, coboundary and Green components (b, c, g, q) in (8) to the domain $|\xi '|\le 1$ such that (36) holds.

We define the noncommutative connections

$$\begin{aligned}&\nabla _{P_j}=P_j\cdot d\cdot P_j\quad \text { on }\quad T^*M\quad \text { and }\quad \nabla _{P'_j}=P'_j\cdot d'\cdot P'_j\quad \text { on }\quad T^*X, \text { where } j\\&\quad =1,2, \end{aligned}$$

where $d'$ stands for the exterior derivative on $T^*X$. We also define a connection on $T^*M$

$$\begin{aligned} {\widetilde{\nabla }}_{P_1}=\nabla _{P_1}+r(\nabla a), \quad \text { where } \nabla a \equiv \nabla _{P_2}a-a\nabla _{P_1}, \end{aligned}$$

(37)

and a connection on $T^*X$

$$\begin{aligned} {\widetilde{\nabla }}_{P'_1}=\nabla _{P'_1}+r_X(\nabla ' a_X), \quad \text { where } \nabla ' a_X =\nabla _{P'_2}a_X-a_X\nabla _{P'_1}. \end{aligned}$$

Lemma 1

The curvatures of ${\widetilde{\nabla }}_{P_1}$ and ${\widetilde{\nabla }}_{P'_1}$ are equal to

$$\begin{aligned}&{\widetilde{\Omega }}_{P_1}\overset{\mathrm {def}}{=}({\widetilde{\nabla }}_{P_1})^2=\nabla _{P_1}^2+\nabla _{P_1}(r\nabla a) + (r\nabla a)^2, \\&{\widetilde{\Omega }}_{P'_1}\overset{\mathrm {def}}{=}({\widetilde{\nabla }}_{P'_1})^2=\nabla _{P'_1}^2+\nabla _{P'_1}(r_X\nabla ' a_X) + (r_X\nabla ' a_X)^2. \end{aligned}$$

Proof

Indeed, we have

$$\begin{aligned}&{\widetilde{\Omega }}_{P_1}u=({\widetilde{\nabla }}_{P_1})^2u=(\nabla _{P_1}+r\nabla a)^2u= (\nabla _{P_1}^2+\nabla _{P_1}r\nabla a + r(\nabla a)\nabla _{P_1} + (r\nabla a)^2)u\\&\quad =(\nabla _{P_1}^2+\nabla _{P_1}(r\nabla a) + (r\nabla a)^2)u,\\&{\widetilde{\Omega }}_{P'_1}v=({\widetilde{\nabla }}_{P'_1})^2v=(\nabla _{P'_1}+r_X\nabla ' a_X)^2v= (\nabla _{P'_1}^2+\nabla _{P'_1}r_X\nabla ' a_X + r_X(\nabla ' a_X)\nabla _{P'_1} \\&\qquad + (r_X\nabla ' a_X)^2)v\\&\quad =(\nabla _{P'_1}^2+\nabla _{P'_1}(r_X\nabla ' a_X) + (r_X\nabla ' a_X)^2)v. \end{aligned}$$

$\square$

Let us define the differential forms with compact supports

$$\begin{aligned} {\text {ch}}_{T^*M}^\gamma \sigma ({\mathcal {D}}) \in \Omega ^{ev}_c(T^*M^\gamma ),\quad {\text {ch}}^\gamma _{T^*X}\sigma ({\mathcal {D}})\in \Omega ^{ev}_c(T^*X^\gamma ) \end{aligned}$$

(38)

on the cotangent bundles of the submanifolds of fixed points by the formulas

$$\begin{aligned}&{\text {ch}}^\gamma _{T^*M}\sigma ({\mathcal {D}})=\tau ^{\gamma }\left( e^{-{\widetilde{\Omega }}_{P_1}/2\pi i}(P_1-ra)\right) \nonumber \\&\quad - \tau ^{\gamma }\left( P_2e^{-\nabla ^2_{P_2}/2\pi i}-ae^{-{\widetilde{\Omega }}_{P_1}/2\pi i}r\right) , \end{aligned}$$

(39)

$$\begin{aligned}&{\text {ch}}^\gamma _{T^*X}\sigma ({\mathcal {D}})=\tau ^\gamma _X\left( e^{-{\widetilde{\Omega }}_{P'_1}/2\pi i}(P'_1-r_Xa_X)\right) \nonumber \\&\quad - \tau ^\gamma _X\left( P'_2e^{-\nabla ^2_{P'_2}/2\pi i}-a_Xe^{-{\widetilde{\Omega }}_{P'_1}/2\pi i}r_X\right) . \end{aligned}$$

(40)

Here and below we denote the extensions of mappings (29) to the matrix algebras over the corresponding crossed products again by $\tau ^\gamma , \tau _X^\gamma$. The extensions are obtained as the compositions of the matrix trace and mappings (29). Using the fact that

$$\begin{aligned} ar=P_2, ra=P_1, a_Xr_X=P'_2, r_Xa_X=P'_1 \end{aligned}$$

at infinity in the cotangent bundles, one can show that the noncommutative differential forms in (39) and (40) have compact supports on $T^*M$ and $T^*X$. Hence, the Chern forms (39) and (40) have compact supports on $T^*M^\gamma$ and $T^*X^\gamma$ respectively.

Remark

Since $\tau ^\gamma$ is a graded trace, (39) can be written as

$$\begin{aligned} {\text {ch}}^\gamma _{T^*M}\sigma ({\mathcal {D}})=\tau ^{\gamma }\left( e^{-{\widetilde{\Omega }}_{P_1}/2\pi i}P_1 - P_2e^{-\nabla ^2_{P_2}/2\pi i}\right) . \end{aligned}$$

The boundary $\partial (T^*M^\gamma )\simeq T^*X^\gamma \times {\mathbb {R}}$ is fibered over $T^*X^\gamma$ with the fiber ${\mathbb {R}}$. We denote the corresponding projection by $\pi ^\gamma :\partial (T^*M^\gamma )\rightarrow T^*X^\gamma$ and the embedding $\partial (T^*M^\gamma )\subset T^*M^\gamma$ by $i_\gamma$. Hence, the pair $(T^*M^\gamma ,\pi ^\gamma )$ is a manifold with fibered boundary in the sense of Sect. 4.

Proposition 2

Given $\gamma \in \Gamma$, we have

$$\begin{aligned}&d\left( {\text {ch}}_{T^*M}^\gamma \sigma ({\mathcal {D}})\right) =0, \end{aligned}$$

(41)

$$\begin{aligned}&d'\left( {\text {ch}}_{T^*X}^\gamma \sigma ({\mathcal {D}})\right) =\pi ^\gamma _*i_\gamma ^*\left( {\text {ch}}_{T^*M}^\gamma \sigma ({\mathcal {D}})\right) . \end{aligned}$$

(42)

In other words, the pair $({\text {ch}}_{T^*M}^\gamma \sigma ({\mathcal {D}}),-{\text {ch}}_{T^*X}^\gamma \sigma ({\mathcal {D}}))$ is closed in the complex $(\Omega _c^*(T^*M^\gamma ,\pi ^\gamma ),\partial )$, see (25), and we denote its cohomology class by

$$\begin{aligned} {\text {ch}}^\gamma \sigma ({\mathcal {D}}) \in H^{ev} (T^*M^\gamma ,\pi ^\gamma ). \end{aligned}$$

This class does not depend on the choice of the elements $a,r,a_X,r_X$ and does not change under homotopies of elliptic symbols.

Proof

1. Equality (41) can be proven in a standard way (see, for instance, [31])

$$\begin{aligned}&d\left( {\text {ch}}_{T^*M}^\gamma \sigma ({\mathcal {D}})\right) = d\tau ^{\gamma }\left( e^{-{\widetilde{\nabla }}^2_{P_1}/2\pi i}P_1\right) - d\tau ^{\gamma }\left( P_2e^{-{\nabla ^2_{P_2}}/2\pi i}\right) \\&\quad =\tau ^{\gamma }\left( {\widetilde{\nabla }}_{P_1}\left( e^{-{\widetilde{\nabla }}^2_{P_1}/2\pi i}P_1\right) \right) - \tau ^{\gamma }\left( \nabla _{P_2}\left( P_2e^{-{\nabla ^2_{P_2}}/2\pi i}\right) \right) \\&\quad =\tau ^{\gamma }\left[ {\widetilde{\nabla }}_{P_1},e^{-{\widetilde{\nabla }}^2_{P_1}/2\pi i}P_1\right] - \tau ^{\gamma }\left[ \nabla _{P_2},P_2e^{-{\nabla ^2_{P_2}}/2\pi i}\right] = 0. \end{aligned}$$

The last equality holds, since the commutators in it are equal to zero.

2. Next, let us prove equality (42). Let us calculate the right hand side in (42). We denote the restriction of the curvature form ${\widetilde{\Omega }}_{P_1}$ to $\partial T^*M$ by $\Omega _1={\widetilde{\Omega }}_{P_1}|_{\partial T^*M}$. It is clear that $\Omega _1$ is equal to the curvature form for the pair of restrictions $(a|_{\partial T^*M},r|_{\partial T^*M})$. Also we denote by $\Omega _2=\Omega _2(\xi _n)={\widetilde{\Omega }}_{P_1}|_{\partial T^*X}\cap \{\xi _n=\mathrm {Const}\}$ the family of curvature forms for the restrictions $(a|_{\partial T^*M\cap \{\xi _n=\mathrm{const}\}},r|_{\partial T^*M\cap \{\xi _n=\mathrm{const}\}})$, where $\xi _n$ is considered as a parameter. We have

$$\begin{aligned}&i^*_\gamma {\text {ch}}^\gamma _{T^*M}\sigma ({\mathcal {D}}) = \tau ^\gamma _0\left( e^{-\Omega _1/2\pi i}P_1 - P_2e^{-{\nabla ^2_{P_2}}/2\pi i}\right) , \nonumber \\&\pi ^\gamma _*i^*_\gamma {\text {ch}}^\gamma _{T^*M}\sigma ({\mathcal {D}})=-\frac{1}{2\pi i}\int _{{\mathbb {R}}}{\tau ^\gamma \left( \left( \frac{\partial }{\partial \xi _n}\lrcorner \Omega _1\right) e^{-\Omega _2/2\pi i}\right) d\xi _n}. \end{aligned}$$

(43)

Here $\tau ^\gamma _0$ stands for the trace defined as in (30) but for the manifold $\partial T^*M=T^*X\times {\mathbb {R}}$, while $\frac{\partial }{\partial \xi _n}\lrcorner$ stands for the substitution of $\frac{\partial }{\partial \xi _n}$ into the differential form. Note that the integrand in (43) is compactly supported. Hence, its integral is well defined. By Lemma 1, we have

$$\begin{aligned} \frac{\partial }{\partial \xi _n}\lrcorner \Omega _1 = \frac{\partial }{\partial \xi _n}\lrcorner \left( \nabla _{P_1}^2+\nabla _{P_1}(r\nabla a) + (r\nabla a)^2\right) . \end{aligned}$$

(44)

We now substitute $\partial /\partial \xi _n$ into each of the summands in (44). To this end, we represent the connections in the following form:

$$\begin{aligned}&d = d\xi _n\frac{\partial }{\partial \xi _n} +d',\quad \nabla _{P_j}=P_jd'P_j + P_jd\xi _n\frac{\partial }{\partial \xi _n}\equiv \nabla '_{P_j} + P_jd\xi _n\frac{\partial }{\partial \xi _n},\quad \nabla \\&\quad =\nabla ' + d\xi _n\frac{\partial }{\partial \xi _n}. \end{aligned}$$

For the first summand in (44), we have

$$\begin{aligned} \frac{\partial }{\partial \xi _n}\lrcorner \nabla _{P_1}^2 = \frac{\partial }{\partial \xi _n}\lrcorner (P_1dP_1)^2=0, \end{aligned}$$

(45)

since $P_1$ does not depend on $\xi _n$. For the second summand in (44), we have

$$\begin{aligned}&\nabla _{P_1}(r\nabla a) = \left( \nabla '_{P_1} + P_1d\xi _n\frac{\partial }{\partial \xi _n}\right) \left( r\nabla 'a + r\frac{\partial a}{\partial \xi _n}d\xi _n\right) \nonumber \\&\quad =\nabla '_{P_1}(r\nabla 'a) + \nabla '_{P_1}\left( r\frac{\partial a}{\partial \xi _n}d\xi _n\right) + \left( P_1d\xi _n\frac{\partial }{\partial \xi _n}\right) (r\nabla 'a) \nonumber \\&\qquad + \left( P_1d\xi _n\frac{\partial }{\partial \xi _n}\right) \left( r\frac{\partial a}{\partial \xi _n}d\xi _n\right) \nonumber \\&\quad =\nabla '_{P_1}(r\nabla 'a) + \nabla '_{P_1}\left( r\frac{\partial a}{\partial \xi _n}\right) d\xi _n + d\xi _n\frac{\partial }{\partial \xi _n}(r\nabla 'a). \end{aligned}$$

(46)

We now substitute $\partial /\partial \xi _n$ into (46):

$$\begin{aligned}&\frac{\partial }{\partial \xi _n}\lrcorner \nabla _{P_1}(r\nabla a) = -\nabla '_{P_1}\left( r\frac{\partial a}{\partial \xi _n}\right) + \frac{\partial }{\partial \xi _n}(r\nabla 'a) \nonumber \\&\quad =-\left( \nabla 'r\right) \frac{\partial a}{\partial \xi _n} -r\nabla '\frac{\partial a}{\partial \xi _n}+ \frac{\partial r}{\partial \xi _n}\nabla 'a + r\nabla '\frac{\partial a}{\partial \xi _n}= -\left( \nabla 'r\right) \frac{\partial a}{\partial \xi _n}+ \frac{\partial r}{\partial \xi _n}\nabla 'a.\nonumber \\ \end{aligned}$$

(47)

For the third summand in (44), we get

$$\begin{aligned} \frac{\partial }{\partial \xi _n}\lrcorner \left( \left( r\nabla 'a + r\frac{\partial a}{\partial \xi _n}d\xi _n\right) \left( r\nabla 'a + r\frac{\partial a}{\partial \xi _n}d\xi _n\right) \right) = r\frac{\partial a}{\partial \xi _n}r\nabla 'a- (r\nabla 'a)r\frac{\partial a}{\partial \xi _n}.\nonumber \\ \end{aligned}$$

(48)

Substituting (45), (47) and (48) into (43), we obtain

$$\begin{aligned}&\pi ^\gamma _*i^*_\gamma {\text {ch}}^\gamma _{T^*M}\sigma ({\mathcal {D}})\nonumber \\&\quad =\frac{i}{2\pi }\int _{{\mathbb {R}}}{\tau ^\gamma _0\left( \left( \frac{\partial r}{\partial \xi _n}\nabla 'a -\left( \nabla 'r\right) \frac{\partial a}{\partial \xi _n}+ \left[ r\frac{\partial a}{\partial \xi _n},r\nabla 'a\right] \right) e^{-\Omega _2/2\pi i}\right) d\xi _n}.\qquad \end{aligned}$$

(49)

As in (43), the argument of $\tau ^\gamma _0$ in (49) is compactly supported.

3. To calculate the left hand side in (42), we first prove two auxiliary lemmas

Lemma 2

For each form $\omega '\in {\text {Mat}}_N(\Omega _{T^*X} \rtimes \Gamma )$ such that $\omega '=P'_1\omega 'P'_1$, we have

$$\begin{aligned} d'\tau ^\gamma _X(\omega ')=\tau ^\gamma _X(\nabla _{P'_1}\omega '). \end{aligned}$$

(50)

Proof

The difference between the left and right hand sides in (50) is equal to

$$\begin{aligned}&d'\tau ^\gamma _X(\omega ')-\tau ^\gamma _X(\nabla _{P'_1}\omega ')=\tau ^\gamma _X(d'(P'_1\omega ')-P'_1d'\omega ')=\tau ^\gamma _X((d'P'_1)\omega ') \nonumber \\&\quad =\tau ^\gamma _X((d'P'_1)P'_1\omega 'P'_1)=\tau ^\gamma _X(P'_1d'P'_1P'_1\omega ')=0, \end{aligned}$$

(51)

where in the last line we used the trace property: $\tau ^\gamma _X(\omega 'P'_1)=\tau ^\gamma _X(P'_1\omega ')$. The last equality holds, since $P'_1$ acts as a scalar operator in the variable $\xi _n$. In the last equality in (51) we used the identity $P'_1(d'P'_1)P'_1=0$ for the projection $P'_1$.

This completes the proof of Lemma 2. $\square$

Using Lemma 2, we obtain the following expression for the left hand side in (42)

$$\begin{aligned}&d'{\text {ch}}^\gamma _{T^*X}\sigma ({\mathcal {D}})=\tau ^\gamma _X\left( \nabla _{P'_1}\big (e^{-{\widetilde{\nabla }}^2_{P'_1}/2\pi i}P'_1\big )\right) - \tau ^\gamma _X\left( \nabla _{P'_2}\big (P'_2e^{-\nabla ^2_{P'_2}/2\pi i}\big )\right) \nonumber \\&\quad + d'\tau ^\gamma _X\left[ a_X, e^{-{\widetilde{\nabla }}^2_{P'_1}/2\pi i}r_X\right] . \end{aligned}$$

(52)

For the first summand in (52), we have

$$\begin{aligned}&\nabla _{P'_1}\left( e^{-{\widetilde{\nabla }}^2_{P'_1}/2\pi i}P'_1\right) = \big [\nabla _{P'_1},e^{-{\widetilde{\nabla }}^2_{P'_1}/2\pi i}P'_1\big ] \\&\quad = \big [(\nabla _{P'_1}+r_X\nabla 'a_X)-r_X\nabla 'a_X,e^{-{\widetilde{\nabla }}^2_{P'_1}/2\pi i}P'_1\big ] \\&\quad =\big [{\widetilde{\nabla }}_{P'_1},e^{-{\widetilde{\nabla }}^2_{P'_1}/2\pi i}P'_1\big ] - \big [r_X\nabla 'a_X,e^{-{\widetilde{\nabla }}^2_{P'_1}/2\pi i}P'_1\big ]\\&\quad = - \big [r_X\nabla 'a_X,e^{-{\widetilde{\nabla }}^2_{P'_1}/2\pi i}P'_1\big ]. \end{aligned}$$

For the second summand in (52), we obtain

$$\begin{aligned} \nabla _{P'_2}\left( P'_2e^{-\nabla ^2_{P'_2}/2\pi i}\right) = \big [\nabla _{P'_2},P'_2e^{-\nabla ^2_{P'_2}/2\pi i}\big ] = 0. \end{aligned}$$

Substituting the last two formulas into (52), we get

$$\begin{aligned} d'{\text {ch}}^\gamma _{T^*X}\sigma ({\mathcal {D}})=-\tau ^\gamma _X\left[ r_X\nabla 'a_X,e^{-{\widetilde{\nabla }}^2_{P'_1}/2\pi i}P'_1\right] + d'\tau ^\gamma _X\left[ a_X, e^{-{\widetilde{\nabla }}^2_{P'_1}/2\pi i}r_X\right] .\nonumber \\ \end{aligned}$$

(53)

The next lemma is a generalization of (10).

Lemma 3

Given forms $\omega _{X,1},\omega _{X,2}\in \Omega _{T^*X} \rtimes \Gamma$, we have

$$\begin{aligned} \tau ^\gamma _X[\omega _{X,1},\omega _{X,2}] = -i\Pi '\left( \tau ^\gamma \left( \frac{\partial \omega _{1}}{\partial \xi _n}\omega _{2}\right) \right) = i\Pi '\left( \tau ^\gamma \left( \omega _{1}\frac{\partial \omega _{2}}{\partial \xi _n}\right) \right) , \end{aligned}$$

(54)

where $\omega _1,\omega _2$ are the principal symbols of $\omega _{X,1}$ and $\omega _{X,2}$ while

$$\begin{aligned}{}[a,b] = ab - (-1)^{kl}ba, \quad k=\mathrm{deg}\,a, l=\mathrm{deg}\,b. \end{aligned}$$

Proof

Consider noncommutative forms

$$\begin{aligned} \omega _{X,1}=a_{X,1}\alpha _1,\,\omega _{X,2}=a_{X,2}\alpha _2,\quad \text {where }a_{X,1},a_{X,2}\in {{\widetilde{\Sigma }}}_X \rtimes \Gamma ,\, \alpha _1,\alpha _2\in \Omega (T^*X). \end{aligned}$$

As in the proof of Proposition 1, it suffices to consider the symbols $a_{X,1},a_{X,2}$ of the following form

$$\begin{aligned} a_{X,1}(g)=\left\{ \begin{matrix} b_{X,1}, &{}g=\gamma _1,\\ 0, &{}g\ne \gamma _1, \end{matrix} \right. \qquad a_{X,2}(g)=\left\{ \begin{matrix} b_{X,2}, &{}g=\gamma _2,\\ 0, &{}g\ne \gamma _2, \end{matrix} \right. \end{aligned}$$

such that $\gamma =\gamma _1\gamma _2$.

A computation shows that

$$\begin{aligned}&(a_{X,1}\alpha _1a_{X,2}\alpha _2)(g)=\left\{ \begin{matrix} b_{X,1}{\gamma _1^*}^{-1}(\alpha _1b_{X,2}){(\gamma _1\gamma _2)^*}^{-1}(\alpha _2), &{}\text { if }g=\gamma _1\gamma _2,\\ 0, &{}\text { if }g\ne \gamma _1\gamma _2, \end{matrix} \right. \nonumber \\ \end{aligned}$$

(55)

$$\begin{aligned}&(a_{X,2}\alpha _2a_{X,1}\alpha _1)(g)=\left\{ \begin{matrix} b_{X,2}{\gamma _2^*}^{-1}(\alpha _2b_{X,1}){(\gamma _2\gamma _1)^*}^{-1}(\alpha _1), &{}\text { if }g=\gamma _2\gamma _1,\\ 0, &{}\text { if }g\ne \gamma _2\gamma _1. \end{matrix} \right. \end{aligned}$$

(56)

Substituting (55) and (56) into $\tau ^\gamma _X[\omega _{X,1},\omega _{X,2}]$, we obtain

$$\begin{aligned}&\tau ^\gamma _X[\omega _{X,1},\omega _{X,2}] = \tau ^\gamma _X\left( \omega _{X,1}\omega _{X,2}-(-1)^{kl}\omega _{X,2}\omega _{X,1}\right) \nonumber \\&\quad =\int _{C^\gamma } {{\,\mathrm{Tr}\,}}_X \left( \left. h^* \left( {z}^* \left( a_{X,1}\alpha _1 a_{X,2}\alpha _2(\gamma ) \right) \right) \right| _{T^*X^\gamma } \right) dh \nonumber \\&\qquad - \int _{C^\gamma } {{\,\mathrm{Tr}\,}}_X \left( \left. h^* \left( {z'}^* \left( (-1)^{kl}a_{X,2}\alpha _2 a_{X,1}\alpha _1(\gamma ') \right) \right) \right| _{T^*X^\gamma } \right) dh \end{aligned}$$

(57)

Consider now the first summand in (57). Here we take $z=e$. We substitute (55) into the first summand in (57) and get

$$\begin{aligned}&\int _{C^\gamma } {{\,\mathrm{Tr}\,}}_X \left( \left. h^* \left( {z}^* \left( a_{X,1}\alpha _1 a_{X,2}\alpha _2(\gamma ) \right) \right) \right| _{T^*X^\gamma } \right) dh\nonumber \\&\quad = \int _{C^\gamma } {{\,\mathrm{Tr}\,}}_X \left( \left. h^* \left( b_{X,1}{\gamma _1^*}^{-1}(\alpha _1b_{X,2}){\gamma ^*}^{-1}(\alpha _2) \right) \right| _{T^*X^\gamma } \right) dh\nonumber \\&\quad = \int _{C^\gamma } {{\,\mathrm{Tr}\,}}_X \left( \left. (h^* \left( b_{X,1}{\gamma _1^*}^{-1}(b_{X,2}\alpha _1))(h^*{\gamma ^*}^{-1}(\alpha _2)) \right) \right| _{T^*X^\gamma } \right) dh. \end{aligned}$$

(58)

Since $h\in C^\gamma ,$ we have $h^*{\gamma ^*}^{-1} = {\gamma ^*}^{-1}h^*$, and ${\gamma ^*}^{-1}h^*=h^*$ since the form in (58) is considered over $T^*X^\gamma$. Thus, (58) is equal to

$$\begin{aligned}&\int _{C^\gamma } {{\,\mathrm{Tr}\,}}_X \left( \left. (h^* \left( b_{X,1}{\gamma _1^*}^{-1}(b_{X,2}\alpha _1)\alpha _2) \right) \right| _{T^*X^\gamma } \right) dh \nonumber \\&\quad = \int _{C^\gamma } h^* \left( {{\,\mathrm{Tr}\,}}_X \left( \left( \left. b_{X,1}{\gamma _1^*}^{-1}(b_{X,2}) \right) \right| _{T^*X^\gamma } \right) \left. \left( {\gamma _1^*}^{-1}(\alpha _1)\alpha _2\right) \right| _{T^*X^\gamma } \right) dh. \end{aligned}$$

(59)

Now we consider the second summand in (57). Here $\gamma '=\gamma _2\gamma _1$ and $z'=\gamma _2$. We substitute (56) into the second summand in (57)

$$\begin{aligned}&\int _{C^\gamma } {{\,\mathrm{Tr}\,}}_X \left( \left. h^* \left( {z'}^* \left( a_{X,2}\alpha _2 a_{X,1}\alpha _1(\gamma ') \right) \right) \right| _{T^*X^\gamma } \right) dh\nonumber \\&\quad = \int _{C^\gamma } {{\,\mathrm{Tr}\,}}_X \left( \left. h^* \left( {\gamma _2}^* \left( b_{X,2}{\gamma _2^*}^{-1}(\alpha _2b_{X,1}){(\gamma _2\gamma _1)^*}^{-1}(\alpha _1) \right) \right) \right| _{T^*X^\gamma } \right) dh \nonumber \\&\quad = \int _{C^\gamma } {{\,\mathrm{Tr}\,}}_X \left( \left. h^* \left( \gamma ^*{\gamma _1^*}^{-1}(b_{X,2})b_{X,1} \alpha _2{\gamma _1^*}^{-1}(\alpha _1) \right) \right| _{T^*X^\gamma } \right) dh \end{aligned}$$

(60)

since $\gamma _2^*=\gamma ^*{\gamma _1^*}^{-1}$. Equality $h^*\gamma ^*=h^*$ over $T^*X^\gamma$ implies that (60) is equal to

$$\begin{aligned}&\int _{C^\gamma } {{\,\mathrm{Tr}\,}}_X \left( \left. h^* \left( {\gamma _1^*}^{-1}(b_{X,2})b_{X,1} \alpha _2{\gamma _1^*}^{-1}(\alpha _1) \right) \right| _{T^*X^\gamma } \right) dh\nonumber \\&\quad = (-1)^{kl} \int _{C^\gamma } h^* \left( {{\,\mathrm{Tr}\,}}' \left( \left. ({\gamma _1^*}^{-1}(b_{X,2})b_{X,1})\right| _{T^*X^\gamma } \left. \left( {\gamma _1^*}^{-1}(\alpha _1)\alpha _2\right) \right| _{T^*X^\gamma } \right) \right) dh.\nonumber \\ \end{aligned}$$

(61)

Finally, substitute (59) and (61) into (57) and obtain

$$\begin{aligned}&\tau ^\gamma _X[\omega _{X,1},\omega _{X,2}] \nonumber \\&\quad = \int _{C^\gamma } h^* \bigl ( {{\,\mathrm{Tr}\,}}' \left( \left( \left. b_{X,1}{\gamma _1^*}^{-1}(b_{X,2}) - {\gamma _1^*}^{-1}(b_{X,2})b_{X,1} \right) \right| _{T^*X^\gamma } \right) \left. \left( {\gamma _1^*}^{-1}(\alpha _1)\alpha _2\right) \right| _{T^*X^\gamma } \bigr ) dh \nonumber \\&\quad = \int _{C^\gamma } h^* \bigl ( {{\,\mathrm{Tr}\,}}' \left( \left( \left. [b_{X,1},{\gamma _1^*}^{-1}(b_{X,2})] \right) \right| _{T^*X^\gamma } \right) \left. \left( {\gamma _1^*}^{-1}(\alpha _1)\alpha _2\right) \right| _{T^*X^\gamma } \bigr ) dh. \end{aligned}$$

(62)

Now we use (10) and show that (62) is equal to

$$\begin{aligned}&\int _{C^\gamma } h^* \left. \left( -i\Pi ' \left( \frac{\partial b_1}{\partial \xi _n}{\gamma _1^*}^{-1}(b_2) \right) {\gamma _1^*}^{-1}(\alpha _1)\alpha _2 \right) \right| _{T^*X^\gamma } dh \\&\quad = -i\Pi '\int _{C^\gamma } h^* \left. \left( \frac{\partial b_1}{\partial \xi _n}{\gamma _1^*}^{-1}(b_2) {\gamma _1^*}^{-1}(\alpha _1)\alpha _2 \right) \right| _{T^*X^\gamma } dh \\&\quad = -i\Pi '\int _{C^\gamma } h^* \left( \left. \left( \frac{\partial a_1}{\partial \xi _n} \alpha _1a_2\alpha _2\right) (\gamma ) \right| _{T^*X^\gamma }\right) dh= -i\Pi ' \left( \tau ^\gamma _0\left( \frac{\partial \omega _{X,1}}{\partial \xi _n} \omega _{X,2} \right) \right) , \end{aligned}$$

where $b_1,b_2$ are the principal symbols of $b_{X,1}, b_{X,2}$, respectively. Here the used (55). The second equality in (54) is proved similarly.

This completes the proof of Lemma 3. $\square$

Now let us calculate the traces in (53) using Lemma 3:

$$\begin{aligned}&d'{\text {ch}}^\gamma _{T^*X}\sigma ({\mathcal {D}})= i\Pi '\tau ^\gamma _0\left( \left( \displaystyle {\frac{\partial }{\partial \xi _n}}(r\nabla ' a)\right) e^{-\Omega _2/2\pi i}P_1\right) - id'\Pi '\tau ^\gamma _0\left( \displaystyle {\frac{\partial a}{\partial \xi _n}}e^{-\Omega _2/2\pi i}r\right) \nonumber \\&\quad = i\Pi '\tau ^\gamma _0\left( \left( \frac{\partial }{\partial \xi _n}(r\nabla ' a)\right) e^{-\Omega _2/2\pi i}\right) - i\Pi '\tau ^\gamma _0\left[ {\widetilde{\nabla }}'_{P_1},r\frac{\partial a}{\partial \xi _n}e^{-\Omega _2/2\pi i}\right] \nonumber \\&\quad = i\Pi '\tau ^\gamma _0\left( \left( \frac{\partial }{\partial \xi _n}(r\nabla ' a) - \left[ {\widetilde{\nabla }}'_{P_1},r\frac{\partial a}{\partial \xi _n}\right] \right) e^{-\Omega _2/2\pi i}\right) \nonumber \\&\quad = i\Pi '\tau ^\gamma _0\left( \left( \frac{\partial r}{\partial \xi _n}\nabla ' a + r\nabla '\frac{\partial a}{\partial \xi _n} - \nabla '_{P_1}\left( r\frac{\partial a}{\partial \xi _n}\right) - \left[ r\nabla 'a,r\frac{\partial a}{\partial \xi _n}\right] \right) e^{-\Omega _2/2\pi i}\right) \nonumber \\&\quad = i\Pi '\tau ^\gamma _0\left( \left( \frac{\partial r}{\partial \xi _n}\nabla ' a - \left( \nabla 'r\right) \frac{\partial a}{\partial \xi _n} + \left[ r\frac{\partial a}{\partial \xi _n},r\nabla 'a\right] \right) e^{-\Omega _2/2\pi i}\right) , \end{aligned}$$

(63)

where ${\widetilde{\nabla }}'_{P_1} = \nabla '_{P_1} + r\nabla 'a$ (we have $({\widetilde{\nabla }}'_{P_1})^2=\Omega _2$). Since the argument of $\tau ^\gamma _0$ in (63) coincides with that in (49) and hence is compactly supported, we use (6) and (63) to obtain

$$\begin{aligned} d'{\text {ch}}^\gamma _{T^*X}\sigma ({\mathcal {D}})= \frac{i}{2\pi }\int {\tau ^\gamma _0\left( \left( \frac{\partial r}{\partial \xi _n}\nabla ' a - \left( \nabla 'r\right) \frac{\partial a}{\partial \xi _n} + \left[ r\frac{\partial a}{\partial \xi _n},r\nabla 'a\right] \right) e^{-\Omega _2/2\pi i}\right) d\xi _n}.\nonumber \\ \end{aligned}$$

(64)

Since the expressions in (49) and (64) are equal, we have desired equality (42).

4. Consider compatible families of interior symbols $a_t,r_t$ over $T^*M\times [0,1]$ and boundary symbols $a_{X,t},r_{X,t}$ over $T^*X\times [0,1]$, which smoothly depend on t. For such pairs of symbols, we consider the Chern forms ${\text {ch}}^\gamma _{T^*M\times [0,1]}\sigma ({\mathcal {D}})$ and ${\text {ch}}^\gamma _{T^*X\times [0,1]}\sigma ({\mathcal {D}})$. We represent the form ${\text {ch}}^\gamma _{T^*M\times [0,1]}\sigma ({\mathcal {D}})$ as

$$\begin{aligned} {\text {ch}}^\gamma _{T^*M\times [0,1]}\sigma ({\mathcal {D}})=dt\wedge \alpha +\beta , \end{aligned}$$

(65)

where $\alpha (t),\beta (t)\in \Omega _{T^*M}$ are smooth families of forms. Here

$$\begin{aligned} \beta (t_0)={\text {ch}}^\gamma _{T^*M\times \{t=t_0\}}\sigma ({\mathcal {D}}),\qquad \alpha =\frac{\partial }{\partial t}\lrcorner {\text {ch}}^\gamma _{T^*M\times [0,1]}. \end{aligned}$$

By already proven item 1 of the theorem, we have $d{\text {ch}}^\gamma _{T^*M\times [0,1]}\sigma ({\mathcal {D}})=0$. Using (65), we obtain

$$\begin{aligned} d{\text {ch}}^\gamma _{T^*M\times [0,1]}\sigma ({\mathcal {D}}) = -dt\wedge d\alpha + d\beta +dt\wedge \frac{\partial \beta }{\partial t}=0. \end{aligned}$$

Therefore, we obtain

$$\begin{aligned} \frac{\partial \beta }{\partial t}=d\alpha , \end{aligned}$$

which gives us

$$\begin{aligned} \beta (1)-\beta (0)=d\int \limits ^1_0{\alpha (t)dt}. \end{aligned}$$

Now we use the expansion

$$\begin{aligned} {\text {ch}}^\gamma _{T^*X\times [0,1]}\sigma ({\mathcal {D}})=dt\wedge \alpha _X+\beta _X, \end{aligned}$$

(66)

where $\alpha _X(t),\beta _X(t)\in \Omega _{T^*X}$. Let us find $\pi ^\gamma _*i^*_\gamma {\text {ch}}^\gamma _{T^*M\times [0,1]}\sigma ({\mathcal {D}})$. We obtain

$$\begin{aligned}&i^*_\gamma {\text {ch}}^\gamma _{T^*M\times [0,1]}\sigma ({\mathcal {D}}) = dt\wedge i^*_\gamma \alpha + i^*_\gamma \beta . \nonumber \\&\pi ^\gamma _*i^*_\gamma {\text {ch}}^\gamma _{T^*M\times [0,1]}\sigma ({\mathcal {D}}) = -dt\wedge \pi ^\gamma _*i^*_\gamma \alpha + \pi ^\gamma _*i^*_\gamma \beta . \end{aligned}$$

(67)

Let us now find $d{\text {ch}}^\gamma _{T^*X\times [0,1]}\sigma ({\mathcal {D}})$. Using expansion (66), we obtain

$$\begin{aligned} d{\text {ch}}^\gamma _{T^*X\times [0,1]}\sigma ({\mathcal {D}})=-dt\wedge d'\alpha _X+dt\wedge \frac{\partial \beta _X}{\partial t} + d'\beta _X. \end{aligned}$$

(68)

By the proven item 1 of the theorem, the left hand sides in (67) and (68) differ by a sign. Hence, their right hand sides differ by a sign:

$$\begin{aligned} - dt\wedge \pi ^\gamma _*i^*_\gamma \alpha + \pi ^\gamma _*i^*_\gamma \beta = dt\wedge d'\alpha _X - dt\wedge \frac{\partial \beta _X}{\partial t} - d'\beta _X, \end{aligned}$$

which gives us

$$\begin{aligned} \frac{\partial \beta _X}{\partial t} = d'\alpha _X + \pi ^\gamma _*i^*_\gamma \alpha . \end{aligned}$$

Integrating this equation, we obtain

$$\begin{aligned} \beta _X(1) - \beta _X(0) = d'\int \limits _0^1{\alpha _X(t)dt} + \pi ^\gamma _*i^*_\gamma \int \limits _0^1{\alpha (t)dt}. \end{aligned}$$

Thus, we obtain

$$\begin{aligned}&{\text {ch}}^\gamma _{T^*M}\sigma ({\mathcal {D}})(1)-{\text {ch}}_{T^*M}\sigma ({\mathcal {D}})(0) = d\omega , \end{aligned}$$

(69)

$$\begin{aligned}&{\text {ch}}^\gamma _{T^*X}\sigma ({\mathcal {D}})(1) - {\text {ch}}_{T^*X}\sigma ({\mathcal {D}})(0) = d'\omega _X + \pi ^\gamma _*i^*_\gamma \omega , \end{aligned}$$

(70)

where

$$\begin{aligned} \omega = \int \limits _0^1{\alpha (t)dt},\qquad \omega _X = \int \limits _0^1{\alpha _X(t)dt}. \end{aligned}$$

Equalities (69) and (70) imply that the difference

$$\begin{aligned}&({\text {ch}}^\gamma _{T^*M\times [0,1]}\sigma ({\mathcal {D}})(1),-{\text {ch}}^\gamma _{T^*X\times [0,1]}\sigma ({\mathcal {D}})(1))\\&\quad - ({\text {ch}}^\gamma _{T^*M\times [0,1]}\sigma ({\mathcal {D}})(0),-{\text {ch}}^\gamma _{T^*X\times [0,1]}\sigma ({\mathcal {D}})(0)) \end{aligned}$$

is a coboundary in the complex $(\Omega (T^*M^\gamma ,\pi ^\gamma ),\partial )$. This proves the homotopy invariance of the Chern character.

Let now $a_1, r_1, a_{X,1}, r_{X,1}$ be different extensions of the elliptic symbols to $T^*M$ and $T^*X$. Then we consider the homotopies

$$\begin{aligned}&a_t=a(1-t)+a_1\cdot t,\qquad r_t=r(1-t)+r_1\cdot t, \\&a_{X,t}=a_X(1-t)+a_{X,1}\cdot t,\qquad r_{X,t}=r_X(1-t)+r_{X,1}\cdot t, \end{aligned}$$

where $t\in [0,1]$. At $t=0$ we have the set $a,r,a_X,r_X$, while at $t=1$ we have the set $a_1, r_1, a_{X,1}, r_{X,1}$. Thus the homotopy invariance gives independence of the choice of the extensions. $\square$

6 Index theorem

To state our index formula, we need to define the necessary equivariant characteristic classes. Firstly, we define the Todd forms on $M^\gamma$:

where $\Omega ^\gamma$ is the curvature form of the Levi-Civita connection on $M^\gamma$. The Todd form ${\text {Td}}(T^*X^\gamma \otimes {\mathbb {C}})$ on $X^\gamma$ is defined in a similar way. The pair of these forms is closed in the complex $({\widetilde{\Omega }}^*(M^\gamma ,\pi ^\gamma ),{{\widetilde{\partial }}})$ (see (26)) and its cohomology class is denoted by

$$\begin{aligned} {\text {Td}}^\gamma (T^*M\otimes {\mathbb {C}}) \in {\widetilde{H}}^{ev}(M^\gamma ,\pi ^\gamma ). \end{aligned}$$

(71)

Next, let $N^\gamma$ be the normal bundle of $M^\gamma \subset M$. Then we have the natural action of $\gamma$ on $N^\gamma$ and the following differential forms on $M^\gamma$:

$$\begin{aligned}&{\text {ch}}^\gamma \Lambda ( {N}^\gamma \otimes {\mathbb {C}})={{\,\mathrm{Tr}\,}}_{\Lambda ^{ev}(N^\gamma )}\left( \gamma \exp (-\Omega /2\pi i)\right) \\&\quad - {{\,\mathrm{Tr}\,}}_{\Lambda ^{odd}(N^\gamma )}\left( \gamma \exp (-\Omega /2\pi i)\right) \in \Omega ^{ev}(M^\gamma ), \end{aligned}$$

where $\Omega$ is the curvature form of the exterior bundle $\Lambda (N^\gamma )$, $\gamma$ is considered as an endomorphism of the subbundles $\Lambda ^{ev/odd}(N^\gamma )$ of even/odd forms and ${{\,\mathrm{Tr}\,}}_{\Lambda ^{ev/odd}(N^\gamma )}$ is the fiber-wise trace functional on endomorphisms of the bundles $\Lambda ^{ev/odd}(N^\gamma )$. Similarly, let $N^\gamma _X$ be the normal bundle of $X^\gamma \subset X$. Then one can define the form ${\text {ch}}^\gamma \Lambda ( {N}_X^\gamma \otimes {\mathbb {C}})$ on $X^\gamma$ along the same lines. The pair $({\text {ch}}^\gamma \Lambda ( {N}^\gamma \otimes {\mathbb {C}}),{\text {ch}}^\gamma \Lambda ( {N}_X^\gamma \otimes {\mathbb {C}}))$ is closed in the complex $({\widetilde{\Omega }}^*(M^\gamma ,\pi ^\gamma ),{{\widetilde{\partial }}})$. We denote its cohomology class by

$$\begin{aligned} {\text {ch}}^\gamma \Lambda ({\mathcal {N}}^\gamma \otimes {\mathbb {C}})\in {\widetilde{H}}^{ev}(M^\gamma ,\pi ^\gamma ). \end{aligned}$$

(72)

This class is invertible since its zero degree component is nonzero (see the proof in [1] or [20]).

Theorem 4

Let ${\mathcal {D}}$ be an elliptic operator in the sense of Definition 3. Then the following index formula holds:

$$\begin{aligned} {\text {ind}}{\mathcal {D}}=\sum _{\langle \gamma \rangle \subset \Gamma }\langle {\text {ch}}^\gamma \sigma ({\mathcal {D}})\wedge {\text {Td}}^\gamma (T^*M\otimes {\mathbb {C}})\wedge {\text {ch}}^\gamma \Lambda ({\mathcal {N}}^\gamma \otimes {\mathbb {C}})^{-1},[T^*M^\gamma ,\pi ^\gamma ]\rangle ,\nonumber \\ \end{aligned}$$

(73)

where the summation is over the conjugacy classes in $\Gamma$ and the series converges absolutely.

To prove this theorem, we need to establish some auxiliary statements.

7 Homotopy classification

${\text {Ell}}$ -groups.

Let us denote the Abelian group of stable homotopy classes of elliptic $\Gamma$-Boutet de Monvel operators (14) by ${\text {Ell}}(M,\Gamma )$. We recall (for details see [32]) that two operators $({\mathcal {D}},{\mathcal {P}}_1,{\mathcal {P}}_2)$ and $({\mathcal {D}}',{\mathcal {P}}'_1,{\mathcal {P}}'_2)$ are called stably homotopic, if there exists a smooth homotopy of elliptic operators $({\mathcal {D}}_t,{\mathcal {P}}_{1,t},{\mathcal {P}}_{2,t}), t\in [0,1]$ such that

$$\begin{aligned}&\left. ({\mathcal {D}}_t,{\mathcal {P}}_{1,t},{\mathcal {P}}_{2,t})\right| _{t=0}=({\mathcal {D}},{\mathcal {P}}_1,{\mathcal {P}}_2)\oplus \mathrm {Triv},\qquad \\&\left. ({\mathcal {D}}_t,{\mathcal {P}}_{1,t},{\mathcal {P}}_{2,t})\right| _{t=1}=({\mathcal {D}}',{\mathcal {P}}'_1,{\mathcal {P}}'_2)\oplus \mathrm {Triv}', \end{aligned}$$

where $\mathrm {Triv}, \mathrm {Triv}'$ are some trivial operators. Here a trivial operator is an elliptic operator $({\mathcal {D}},{\mathcal {P}}_1,{\mathcal {P}}_2)$ (see (14)), where ${\mathcal {D}}$ has components in the subalgebra

$$\begin{aligned} (C^\infty (M) \oplus C^\infty (X))\rtimes \Gamma \subset \Psi _B(M)\rtimes \Gamma . \end{aligned}$$

(74)

It can be proven in a standard way that stable homotopy is an equivalence relation on the set of elliptic operators (14). Then the set of elliptic operators (14) considered modulo stable homotopies is denoted by ${\text {Ell}}(M,\Gamma )$. This set is an Abelian group with respect to the direct sum of operators. The zero element of the group is defined by the equivalence class of operators $({\mathcal {D}},{\mathcal {P}}_1,{\mathcal {P}}_2)$, where ${\mathcal {D}}$ is a matrix operator over $(C^\infty (M) \oplus C^\infty (X))\rtimes \Gamma$.

The aim of this section is to obtain an analogue of the Boutet de Monvel theorem, see [5, 9, 10], which provides stable homotopies of elliptic boundary value problems to a simple form in a neighborhood of the boundary. To state this result, we introduce some notations.

First, denote by ${\text {Ell}}(M^\circ ,\Gamma )$ the group of stable homotopy classes of triples $({\mathcal {D}},{\mathcal {P}}_1,{\mathcal {P}}_2)$ (see above), where the components of ${\mathcal {D}}$ are in the subalgebra (74) in a neigborhood of the boundary $X\subset M$.

Second, to each projection $P\in {\text {Mat}}_N(C^\infty (X)\rtimes \Gamma )$, we assign a special boundary value problem. Namely, consider the matrix $N\times N$ operator (cf. [3, Corollary 20.3.1]) on M:

$$\begin{aligned} \Lambda _P=\chi \left( \frac{\partial }{\partial x_n}(2P-1)+ \Lambda _X\right) + ( 1-\chi ) \Lambda _M, \end{aligned}$$

(75)

where $\Lambda _M,\Lambda _X$ are non-negative elliptic $\psi$DOs of order 1 on M and X respectively, $\chi \in C^\infty (M)$ is a function such that $0\le \chi \le 1$ and identically equal to one in a neighborhood of the boundary and equal to zero outside a slightly larger neighborhood of the boundary. Here we suppose that $\Lambda _M$ and $\Lambda _X$ commute with the actions of $\Gamma$ on M and X. For instance, we can take $\Lambda _M$ and $\Lambda _X$ with principal symbols equal to those of the square roots of the Laplacians associated with the Riemannian metrics on M and X. Let us consider the following boundary value problem for operator (75)

$$\begin{aligned} \left( \begin{array}{c} \Lambda _P\\ Pi^* \end{array} \right) : H^1(M,{\mathbb {C}}^N)\longrightarrow \begin{array}{c} L^2(M,{\mathbb {C}}^N)\\ \oplus \\ P H^{1/2}(X,{\mathbb {C}}^N) \end{array}. \end{aligned}$$

(76)

Similarly to [7], one can show that this boundary value problem has Fredholm property and its Fredholm index is equal to zero. We now reduce problem (76) to a problem in $L^2$ spaces. To this end, we denote by $\Lambda _-$ operator (76) for $P=0$. Note that the latter operator is Fredholm without any boundary conditions. Now let us define the zero order boundary value problem:

$$\begin{aligned} \left( \begin{array}{c} \Lambda _P(\Lambda _-)^{-1}\\ \Lambda _X^{1/2}Pi^*(\Lambda _-)^{-1} \end{array} \right) : L^2(M,{\mathbb {C}}^N)\longrightarrow \begin{array}{c} L^2(M,{\mathbb {C}}^N)\\ \oplus \\ P L^{2}(X,{\mathbb {C}}^N). \end{array} \end{aligned}$$

(77)

This problem is Fredholm with zero index as the composition of $\Lambda _-^{-1}$, problem (76) and operator $\Lambda _X^{1/2}$. Moreover, it is elliptic in the sense of Definition 3. Note that similar to [3, 7], the problem (77) is not a $\Gamma$-Boutet de Monvel operator (its interior symbol is only continuous on $S^*M$, since the interior symbol of $\Lambda _X$ as a $\psi DO$ on M is not smooth). However, a small deformation of (77) as in [3, 7] gives a $\Gamma$-Boutet de Monvel operator with smooth symbols. We do not repeat this deformation here for brevity. We denote problem (77) by $[{\mathcal {D}}_P, {\mathcal {P}}_{1,P}, {\mathcal {P}}_{2,P}]$.

Theorem 5

(homotopy classification). The mapping

$$\begin{aligned} \begin{array}{ccc} &{}&{}{\text {Ell}}( M^\circ ,\Gamma )\oplus K_0(C^\infty (X)\rtimes \Gamma ) \longrightarrow {\text {Ell}}(M,\Gamma )\\ &{}&{}{}[{\mathcal {D}},{\mathcal {P}}_1,{\mathcal {P}}_2] \oplus [P] \longmapsto [{\mathcal {D}},{\mathcal {P}}_1,{\mathcal {P}}_2]\oplus [{\mathcal {D}}_P, {\mathcal {P}}_{1,P}, {\mathcal {P}}_{2,P}] \end{array} \end{aligned}$$

(78)

is an isomorphism of groups.

Proof

1. For the Fréchet algebras $C^\infty (M),C^\infty (X),\Sigma ,\Sigma _X,\Psi _B(M),\ldots$ we consider their $C^*$-closures denoted by $C (M),C (X),{\overline{\Sigma }},{\overline{\Sigma }}_X,\overline{\Psi _B}(M),\ldots$.

Denote by ${\overline{{\text {Ell}}}}(M,\Gamma )$ the group of stable homotopy classes of triples $({\mathcal {D}},{\mathcal {P}}_1,{\mathcal {P}}_2)$ as in (14), where the elements are chosen from $C^*$-algebras

$$\begin{aligned} {\mathcal {D}}\in {\text {Mat}}_N(\overline{\Psi _B(M)})\rtimes \Gamma , \quad {\mathcal {P}}_{1,2}\in {\text {Mat}}_N(C (M)\oplus C (X))\rtimes \Gamma . \end{aligned}$$

Similarly, we define the group ${\overline{{\text {Ell}}}}( M^\circ ,\Gamma )$.

Since our Fréchet algebras are spectral invariant in their $C^*$-closures, the natural mappings

$$\begin{aligned} {\overline{{\text {Ell}}}}( M ,\Gamma )\longrightarrow {{\text {Ell}}}( M ,\Gamma ) \quad \text {and}\quad {\overline{{\text {Ell}}}}( M^\circ ,\Gamma )\longrightarrow {{\text {Ell}}}( M^\circ ,\Gamma ) \end{aligned}$$

(79)

are isomorphisms of Abelian groups. Hence, to prove Theorem 5, it suffices to establish the group isomorphism

$$\begin{aligned} {\overline{{\text {Ell}}}}(M^\circ ,\Gamma )\oplus K_0(C (X)\rtimes \Gamma ) \longrightarrow {\overline{{\text {Ell}}}}(M,\Gamma ). \end{aligned}$$

(80)

2. Let us express ${\overline{{\text {Ell}}}}(M,\Gamma )$ in terms of the K-group of some $C^*$-algebra associated with the symbol algebra. Namely, by [32] we have the isomorphism of Abelian groups

$$\begin{aligned} {\overline{{\text {Ell}}}}(M,\Gamma )\simeq K_0\Bigl ({{\,\mathrm{Con}\,}}(C(M)\oplus C(X)\rightarrow {\overline{\Sigma }})\rtimes \Gamma \Bigr ) , \end{aligned}$$

(81)

where

$$\begin{aligned} {{\,\mathrm{Con}\,}}(A\rightarrow B)=\bigl \{(a,b(t))\in A\oplus C([0,1),B) \;\bigl |\; f(a)=b(0)\bigr \} \end{aligned}$$

is the cone of a homomorphism $f:A\rightarrow B$ of $C^*$-algebras A and B. The mapping $C(M)\oplus C(X)\rightarrow {\overline{\Sigma }}$ in (81) is a monomorphism, which takes a pair of functions f, g to the symbol $(f, \mathrm{diag}(f|_X,g))$.

For brevity, the $C^*$-algebra ${{\,\mathrm{Con}\,}}(C(M)\oplus C(X)\rightarrow {\overline{\Sigma }})$ is denoted by ${\mathcal {A}}$.

3. Denote by ${\overline{\Sigma }}_0\subset {\overline{\Sigma }}$ the ideal of all symbols with zero interior symbol. We consider the commutative diagram

$$\begin{aligned} \begin{array}{rcccccl} 0\longrightarrow &{}{\overline{\Sigma }}_0 &{} \longrightarrow &{} {\overline{\Sigma }} &{}{\mathop {\longrightarrow }\limits ^{\sigma _M}}&{} C(S^*_{tr}M)&{} \longrightarrow 0\\ &{}\uparrow &{} &{} \uparrow &{} &{} \uparrow \\ 0\longrightarrow &{}C(X) &{} \longrightarrow &{} C(M)\oplus C(X) &{}\longrightarrow &{} C(M) &{}\longrightarrow 0, \end{array} \end{aligned}$$

(82)

where the space $S^*_{tr}M$ is obtained from the cosphere bundle $S^*M$ by identifying pairs of points $(x',0,0,\pm 1)$ on its boundary. Note that the interior symbol $\sigma _M$ in (82) is well defined, since the interior symbols with the transmission property are continuous functions on $S^*_{tr}M$. The rows in (82) are exact. The diagram (82) gives the short exact sequence

$$\begin{aligned} 0\rightarrow \mathrm{Con}( C(X)\rightarrow {\overline{\Sigma }}_0)\rtimes \Gamma \longrightarrow {\mathcal {A}} \rtimes \Gamma \longrightarrow {{\,\mathrm{Con}\,}}(C(M) \rightarrow C(S^*_{tr} M))\rtimes \Gamma \rightarrow 0 \end{aligned}$$

of crossed products of cones of vertical mappings in (82) and the corresponding periodic exact sequence of K-groups

$$\begin{aligned}&\ldots \rightarrow K_0\bigl ({{\,\mathrm{Con}\,}}( C(X)\rightarrow {\overline{\Sigma }}_0)\rtimes \Gamma \bigr )\longrightarrow K_0 ({\mathcal {A}}\rtimes \Gamma )\nonumber \\&\quad \longrightarrow K_0({{\,\mathrm{Con}\,}}(C(M) \rightarrow C(S^*_{tr}M))\rtimes \Gamma )\nonumber \\&\quad \longrightarrow K_1\bigl ( {{\,\mathrm{Con}\,}}( C(X)\rightarrow {\overline{\Sigma }}_0)\rtimes \Gamma \bigr ) \rightarrow \ldots \end{aligned}$$

(83)

4. Now let us calculate the K-groups in (83).

Lemma 4

We have group isomorphisms

$$\begin{aligned}&K_*\bigl ({{\,\mathrm{Con}\,}}( C(X)\rightarrow {\overline{\Sigma }}_0)\rtimes \Gamma \bigr )\simeq K_*(C_0(T^*X)\rtimes \Gamma ), \end{aligned}$$

(84)

$$\begin{aligned}&K_*({{\,\mathrm{Con}\,}}(C(M) \rightarrow C(S^*_{tr}M))\rtimes \Gamma )\simeq K_*(C_0(T^*M)\rtimes \Gamma )\oplus K_*(C(X)\rtimes \Gamma ).\nonumber \\ \end{aligned}$$

(85)

Proof

Let us construct isomorphism (84). The isomorphism of $C^*$-algebras ${\overline{\Sigma }}_0\simeq C(S^*X,{\mathcal {K}})$ implies the desired isomorphism:

$$\begin{aligned}&K_*\bigl ({{\,\mathrm{Con}\,}}( C(X)\rightarrow {\overline{\Sigma }}_0)\rtimes \Gamma \bigr )\simeq K_*\bigl ({{\,\mathrm{Con}\,}}( C(X)\rightarrow C(S^*X,{\mathcal {K}}))\rtimes \Gamma \bigr ) \nonumber \\&\quad \simeq K_*\bigl ({{\,\mathrm{Con}\,}}( C(X)\rightarrow C(S^*X))\rtimes \Gamma \bigr ) \simeq K_*\bigl ( C_0(T^*X)\rtimes \Gamma \bigr ), \end{aligned}$$

where we used the isomorphism of $C^*$-algebras ${{\,\mathrm{Con}\,}}(C(X)\rightarrow C(S^*X))\simeq C_0(T^*X)$ endowed with $\Gamma$ actions.

Isomorphism (85) can be constructed similarly. $\square$

5. Using Lemma 4, we can write sequence (83) as

$$\begin{aligned}&\ldots \rightarrow K_1(C_0(T^*M)\rtimes \Gamma )\oplus K_1(C(X)\rtimes \Gamma ){\mathop {\longrightarrow }\limits ^{\partial }}K_0(C_0(T^*X)\rtimes \Gamma ) \nonumber \\&\quad \longrightarrow K_0({\mathcal {A}}\rtimes \Gamma )\nonumber \\&\quad \longrightarrow K_0(C_0(T^*M)\rtimes \Gamma )\oplus K_0(C(X)\rtimes \Gamma ){\mathop {\longrightarrow }\limits ^{\partial }}K_1(C_0(T^*X)\rtimes \Gamma ) \longrightarrow \ldots \nonumber \\ \end{aligned}$$

(86)

Here the boundary mappings $\partial$ are the compositions

$$\begin{aligned}&K_*(C_0(T^*M)\rtimes \Gamma )\oplus K_*(C(X)\rtimes \Gamma )\\&\quad \longrightarrow K_*(C_0(T^*M)\rtimes \Gamma ) \longrightarrow K_{*+1}(C_0(T^*X)\rtimes \Gamma ) \end{aligned}$$

of projections to the first summand and restriction to the boundary $T^*M|_X\simeq T^*X\times {\mathbb {R}}$.

6. Consider the exact sequence

$$\begin{aligned}&\rightarrow K_1(C_0(T^*M)\rtimes \Gamma )\oplus K_1(C(X)\rtimes \Gamma ){\mathop {\longrightarrow }\limits ^{\partial }}K_0(C_0(T^*X)\rtimes \Gamma ) \nonumber \\&\quad \rightarrow K_0(C_0(T^* M^\circ )\rtimes \Gamma ) \oplus K_0(C (X)\rtimes \Gamma ) \nonumber \\&\quad \longrightarrow K_0(C_0(T^*M)\rtimes \Gamma )\oplus K_0(C(X)\rtimes \Gamma ){\mathop {\longrightarrow }\limits ^{\partial }}K_1(C_0(T^*X)\rtimes \Gamma ) \longrightarrow \ldots ,\nonumber \\ \end{aligned}$$

(87)

which represents the direct sum of the exact sequence of the pair

$$\begin{aligned} C_0(T^* M^\circ )\rtimes \Gamma \subset C_0(T^*M )\rtimes \Gamma \end{aligned}$$

and the sequence $0\rightarrow K_*(C(X)\rtimes \Gamma ){\mathop {\rightarrow }\limits ^{id}}K_*(C(X)\rtimes \Gamma )\rightarrow 0$. We consider (87) as the upper row in the commutative diagram

$$\begin{aligned}&\begin{array}{ccccc} K_1(C_0(T^*(M\cup X))\!\rtimes \! \Gamma ) &{} {\mathop {\rightarrow }\limits ^{\partial }}K_0(C_0(T^*X)\!\rtimes \! \Gamma ) \rightarrow \!\!\!\!&{} K_0(C_0(T^*M^\circ \cup X )\!\rtimes \! \Gamma ) &{} \rightarrow \\ \downarrow &{} \downarrow &{} \downarrow &{} \\ K_1(C_0(T^*(M\cup X))\!\rtimes \! \Gamma ) &{} {\mathop {\rightarrow }\limits ^{\partial }}K_0(C_0(T^*X)\!\rtimes \! \Gamma ) \rightarrow \!\!\!\!&{} K_0({\mathcal {A}}\!\rtimes \! \Gamma ) &{} \rightarrow \end{array}\nonumber \\&\begin{array}{cc} \quad \rightarrow K_0(C_0(T^*(M\cup X))\!\rtimes \! \Gamma ) {\mathop {\rightarrow }\limits ^{\partial }}\!\!\!\!&{} K_1(C_0(T^*X)\!\rtimes \! \Gamma ) \\ \downarrow &{} \downarrow \\ \quad \rightarrow K_0(C_0(T^*(M\cup X))\!\rtimes \! \Gamma ) {\mathop {\rightarrow }\limits ^{\partial }}\!\!\!\!&{} K_1(C_0(T^*X)\!\rtimes \! \Gamma ). \end{array} \end{aligned}$$

(88)

The vertical mappings in this diagram (except the middle one) are identity mappings. Hence, using Lemma 4, we obtain from the diagram that the middle mapping is an isomorphism:

$$\begin{aligned} K_0 ( {\mathcal {A}}\rtimes \Gamma )\simeq K_0(C_0(T^*M^\circ )\rtimes \Gamma ) \oplus K_0(C (X)\rtimes \Gamma ). \end{aligned}$$

This isomorphism and (79), and (81) give the desired isomorphism in Theorem 5. $\square$

8 Proof of the index theorem

Let us now prove Theorem 4.

1. We claim that the left and right hand sides in (73) define homomorphisms of Abelian groups

$$\begin{aligned} {\text {ind}},{\text {ind}}_t: {\text {Ell}}(M,\Gamma ) \longrightarrow {\mathbb {C}}. \end{aligned}$$

(89)

Indeed, the analytical index is invariant with respect to homotopies of elliptic operators. It is equal to zero in the case of trivial elliptic operators, since trivial operators are invertible. Thus, the analytical index ${\text {ind}}$ does not change under stable homotopies and it defines a group homomorphism

$$\begin{aligned} {\text {ind}}: {\text {Ell}}(M,\Gamma )\rightarrow {\mathbb {Z}}. \end{aligned}$$

On the other hand, the topological index is also invariant with respect to homotopies of elliptic symbols. It is equal to zero for the trivial operators, since the Chern character of the symbol of such operators is equal to zero.

2. By Theorem 5, it is sufficient to prove the equality of the indices a) in the case of operators in ${\text {Ell}}(M^\circ ,\Gamma )$ (i.e., operators trivial in a neighborhood of the boundary); b) in the case of special boundary value problems (77).

3. Case a): since the operator is trivial in a neighborhood of the boundary, it can be extended to the double of the manifold preserving the analytical index. In this case (73) follows from the index theorem in [20]. Indeed, consider an elliptic triple $({\mathcal {D}},{\mathcal {P}}_1,{\mathcal {P}}_2)$, trivial in a neighborhood of the boundary $X\subset M$. Triviality means that ${\mathcal {D}}$ has components in subalgebra (74). Then we have

$$\begin{aligned} {\text {ind}}({\mathcal {D}},{\mathcal {P}}_1,{\mathcal {P}}_2) = {\text {ind}}(D,P_1,P_2), \end{aligned}$$

(90)

where ${P}_{1,2}\in {\text {Mat}}_N(C^\infty (M))\rtimes \Gamma$ are the components of ${\mathcal {P}}_1$ and ${\mathcal {P}}_2$ on M, while D is a pseudodifferential operator in ${\mathcal {D}}$ on M.

Denote by 2M the double of M. As a topological space, it is obtained by gluing two copies of M along the boundary. The space of $C^\infty$-functions is defined by the formula

$$\begin{aligned} C^\infty (2M)=\{(u,v)\in C^\infty (M_\varepsilon )\oplus C^\infty (M_\varepsilon ) | u(x',x_n)=v(x',-x_n) \text { for all } |x_n|<\varepsilon \}, \end{aligned}$$

where $M_\varepsilon$ is the manifold obtained by gluing M and the cylinder $X\times (-\varepsilon ,0]$ along the boundary.

We assume that the coefficients of $(D,P_1,P_2)$ do not depend on $x_n$ for small $|x_n|$. Under this assumption we consider symmetric extensions of $(D,P_1,P_1)$ to the double (as in [34]) and denote the extension by $({\tilde{D}},{\tilde{P}}_1,{\tilde{P}}_2)$. On the one hand, the operator ${\tilde{D}}:{\mathrm{Im}}{\tilde{P}}_1\rightarrow {\mathrm{Im}}{\tilde{P}}_2$ is isomorphic to the direct sum of two copies of the original operator $D:{\mathrm{Im}}P_1\rightarrow {\mathrm{Im}}P_2$. Thus, we get

$$\begin{aligned} {\text {ind}}({\tilde{D}},{\tilde{P}}_1,{\tilde{P}}_2) = 2{\text {ind}}(D,P_1,P_2). \end{aligned}$$

(91)

On the other hand, we apply the index formula in [20] to compute the index of $({\tilde{D}},{\tilde{P}}_1,{\tilde{P}}_2)$ and obtain

$$\begin{aligned}&{\text {ind}}({\tilde{D}},{\tilde{P}}_1,{\tilde{P}}_2)\nonumber \\&\quad = \sum _{\langle \gamma \rangle \subset \Gamma }\langle {\text {ch}}^\gamma \sigma (\tilde{{\mathcal {D}}})\wedge {\text {Td}}^\gamma (T^*2M\otimes {\mathbb {C}})\wedge {\text {ch}}^\gamma \Lambda (2{\mathcal {N}}^\gamma \otimes {\mathbb {C}})^{-1},[T^*2M^\gamma ,\pi ^\gamma ]\rangle .\nonumber \\ \end{aligned}$$

(92)

Since ${\tilde{D}}$ is trivial for small $|x_n|$, the integrand in (92) is identically equal to zero for small $|x_n|$. Further, since $({\tilde{D}},{\tilde{P}}_1,{\tilde{P}}_2)$ is defined by a symmetric extension of $(D,P_1,P_2)$, we obtain from (91) that

$$\begin{aligned}&{\text {ind}}({\tilde{D}},{\tilde{P}}_1,{\tilde{P}}_2)\nonumber \\&\quad = 2\sum _{\langle \gamma \rangle \subset \Gamma }\langle {\text {ch}}^\gamma \sigma ({\mathcal {D}})\wedge {\text {Td}}^\gamma (T^*M\otimes {\mathbb {C}})\wedge {\text {ch}}^\gamma \Lambda ({\mathcal {N}}^\gamma \otimes {\mathbb {C}})^{-1},[T^*M^\gamma ,\pi ^\gamma ]\rangle .\nonumber \\ \end{aligned}$$

(93)

Hence, (90), (91), (93) imply the desired index formula

$$\begin{aligned}&{\text {ind}}(D,P_1,P_2)\\&\quad = \sum _{\langle \gamma \rangle \subset \Gamma }\langle {\text {ch}}^\gamma \sigma ({\mathcal {D}})\wedge {\text {Td}}^\gamma (T^*M\otimes {\mathbb {C}})\wedge {\text {ch}}^\gamma \Lambda ({\mathcal {N}}^\gamma \otimes {\mathbb {C}})^{-1},[T^*M^\gamma ,\pi ^\gamma ]\rangle \end{aligned}$$

for our operator $(D,P_1,P_2)$.

4. Case b): the analytical index is equal to zero for special boundary value problems (77). The proof is similar to that in [3, Proposition 20.3.1] (cf. [7, Sec.3.1.2.1]) and we do not repeat it here. The topological index is also equal to zero.

Indeed, given projection P and corresponding problem $({\mathcal {D}}_P,{\mathcal {P}}_{1,P},{\mathcal {P}}_{2,P})$, see (77), the topological index in (4) is equal to the sum of contribution of conjugacy classes $\langle \gamma \rangle \subset \Gamma$. Each such contribution

$$\begin{aligned} \langle {\text {ch}}^\gamma \sigma ({\mathcal {D}}_P)\wedge {\text {Td}}^\gamma (T^*M\otimes {\mathbb {C}})\wedge {\text {ch}}^\gamma \Lambda ({\mathcal {N}}^\gamma \otimes {\mathbb {C}})^{-1},[T^*M^\gamma ,\pi ^\gamma ]\rangle \end{aligned}$$

(94)

is equal to the sum of integrals of the forms representing the cohomology classes over $T^*M^\gamma$ and $T^*X^\gamma$. We claim that each integral is equal to zero.

First, choose local coordinates $(y,\eta )$ in $T^*X^\gamma$ and introduce spherical coordinates

$$\begin{aligned} \eta =r\omega ,\quad \text {where}\quad r=|\eta |,\quad \omega =\frac{\eta }{|\eta |}. \end{aligned}$$

(95)

Then it follows from (75),(76), (77) and (71),(72) that the differential forms, which represent components on $T^*X^\gamma$ of the cohomology class in (94), have no differentials $d\omega$. Hence, the integral over $T^*M^\gamma$ is equal to zero.

Second, choose coordinates $(y,x_n,\eta ,\tau )$ in a neighborhood of the boundary of $T^*M^\gamma$. Here we also introduce spherical coordinates (95) and also note that the integrand in the integral over $T^*M^\gamma$ has no differentials $d\omega$. Hence, the integral is equal to zero.

5. By 3. and 4. functionals (89) are equal on the generators of ${\text {Ell}}(M,\Gamma )$. Hence, these functionals are equal on ${\text {Ell}}(M,\Gamma )$. This completes the proof of the index theorem.