1. Introduction

\(1^{0}.\) The paper is devoted to the study of \(n-\)dimensional (\(n\geq2)\) Dirac operators with singular \(\delta-\)type potentials supported on hypersurfaces in \(\mathbb{R}^{n}\). In dimension \(3,\) such operators arise in problems of confinement and transition of relativistic particles through surfaces that are supports of the singular potentials. The formal Dirac operators with singular potentials are realized as unbounded operators in a Hilbert space with the domain described by the interaction (transmission) conditions on the supports of the singular potentials. Moreover, one can associate with the formal Dirac operator the interaction (transmission) problem for the Dirac operator on the support of the singular potential.

The paper is a natural continuation of the previous author’s papers [39, 40] devoted to the Dirac operators with singular potentials for dimensions \(2\) and \(3,\) and the paper [41] for \(n\geq2\) where the Lopatinsky–Shapiro condition relating to the interaction problems has been obtained in the effective form independent of the dimension.

Here we consider the \(L_{p}\)-theory of the interaction problems generated by Dirac operators in \(\mathbb{R}^{n}\) with singular potentials with supports on smooth closed and nonclosed hypersurfaces \(\Sigma\) belonging to the wide class of non compact hypersurfaces in \(\mathbb{R}^{n}.\) We reduce the interaction problems to pseudodifferential equations on the interaction hypersurface \(\Sigma\), and study the Fredholm properties of these pseudodifferential operators in Sobolev and Besov spaces.

\(2^{0}.\) Let

$$D_{m,\Phi,\Gamma\delta_{\Sigma}}=\mathfrak{D}_{m,\Phi}+\Gamma \delta_{\Sigma}$$

be the formal Dirac operator with singular potentials where

$$\mathfrak{D}_{m,\Phi}u(x)=(\alpha\cdot D+m(x)\alpha_{n+1}+\Phi(x))u(x),x\in \mathbb{R}^{n}, $$
(1)
$$\alpha\cdot D=\sum_{j=1}^{n}\alpha_{j}\cdot D_{j},D=\left( D_{1} ,...,D_{n}\right) ,D_{j}=-i\partial_{x_{j}} $$
(2)

\(\alpha_{j},j=1,..,n+1\) are the Dirac matrices (see [18]) that is the \(N\times N\) Hermitian matrices satisfying the anti-commutative relations

$$\alpha_{j}\alpha_{k}+\alpha_{k}\alpha_{j}=2\delta_{jk}\mathbb{I}_{N};\text{ }j,k=1,...,n+1, $$
(3)

where \(\mathbb{I}_{N}\) is the unit \(N\times N\) matrix. The formal Dirac operator (2) is implemented as an unbounded operator \(\mathcal{D}\) in the Hilbert space \(L^{2}(\mathbb{R}^{n},\mathbb{C}^{N})\). The self-adjointness of the operators \(\mathcal{D}\) and its spectral properties in the dimension \(2,3\) have been studied last time in many papers, see for instanse [3, 9–12, 15, 5, 6, 16, 20, 33, 31] for smooth bounded surfaces in \(\mathbb{R}^{3}\) and curves in \(\mathbb{R}^{2},\) and for the bounded curves with angular points in the paper [30], and for bounded rough surfaces \(\Sigma\subset\mathbb{R}^{3}\) in [14].

We associate with the formal Dirac operator (2) the interaction (transmission) problem

$$\mathbb{D}_{m,\Phi,\mathfrak{B}_{\Sigma}}u=\left\{ \begin{array} [c]{c} \mathfrak{D}_{m,\Phi}u=f\text{ on }\mathbb{R}^{n}\diagdown\Sigma=\Omega _{+}\cup\Omega_{-}\\ \mathfrak{B}_{\Sigma}u=f_{1}\text{ on }\Sigma \end{array} \right. $$
(4)

where \(\Omega_{\pm}\) are open domain in \(\mathbb{R}^{n}\) with the common smooth boundary \(\Sigma.\) The interaction condition on \(\Sigma\) is:

$$\begin{aligned} \, \mathfrak{B}_{\Sigma}u(\upsilon) & =\left( a_{+}(\upsilon)\gamma_{\Sigma }^{+}+a_{-}(\upsilon)\gamma_{\Sigma}^{-}\right) u(\upsilon)=f_{1} (\upsilon),\upsilon\in\Sigma,\\ \text{where }a_{\pm}(\upsilon) & =\mp i\alpha\cdot\nu(\upsilon)+\frac{1} {2}\Gamma(\upsilon),\alpha\cdot\nu(\upsilon)=\sum_{j=1}^{n}\alpha_{j}\nu _{j}(\upsilon), \end{aligned}$$

\(\nu(\upsilon)=(\nu_{1}(\upsilon),...,\nu_{n}(\upsilon))\) is the unit normal vector to \(\Sigma\) at the point \(\upsilon\) directed to \(\Omega_{-} ,\gamma_{\Sigma}^{\pm}:H^{r,p}(\Omega_{\pm},\mathbb{C}^{N})\rightarrow B_{p,p}^{r-1/p}\left( \Sigma,\mathbb{C}^{N}\right) ,r>1/p,\) or \(\gamma_{\Sigma}^{\pm}:B_{p,q}^{r}(\Omega_{\pm},\mathbb{C}^{N})\rightarrow B_{p,q}^{r-1/p}\left( \Sigma,\mathbb{C}^{N}\right) ,r>1/p,p,q\in(1,\infty)\) are the trace operators, \(H^{r,p}(\Omega_{\pm},\mathbb{C}^{N})\) are Sobolev (Bessel potential) spaces on \(\Omega_{\pm},\) \(B_{p,q}^{r}(\Omega_{\pm },\mathbb{C}^{N})\) are the Besov spaces. We assume that the functions \(m,\Phi\) belong to the space \(SO^{\infty}(\mathbb{R}^{n})\) of slowly oscillating at infinity functions:

$$SO^{\infty}(\mathbb{R}^{n})=\left\{ a\in C_{b}^{\infty}(\mathbb{R} ^{n}):\text{ }\lim_{x\rightarrow\infty}\partial_{x_{j}} a(x)=0,j=1,..,n\right\} ,$$

\(C_{b}^{\infty}(\mathbb{R}^{n})\) is the space of infinitely differentiable functions on \(\mathbb{R}^{n}\) bounded with all their derivatives. We assume that \(\Sigma\) is a closed or non closed manifold of the class \(\mathcal{R(} n-1)\) introduced in the papers [34], [37].

Let the Dirac operator \(\mathfrak{D}_{m,\Phi}:H^{1,2}(\mathbb{R} ^{n},\mathbb{C}^{N})\rightarrow L^{2}/(\mathbb{R}^{n},\mathbb{C}^{N})\) is invertible. By the well known R. Beals theorem [4] \(\mathfrak{D} _{m,\Phi}^{-1}\) is a pseudodifferential operator (psdo) in the L. Hörmander class \(OPS_{1,0}^{-1}\left( \mathbb{R}^{n},\mathbb{C}^{N}\right) \) with the slowly oscillating at infinity symbol (see [23]).

The operator \(\mathfrak{D}_{m,\Phi}^{-1}\) has the integral representation

$$\mathfrak{D}_{m,\Phi}^{-1}\phi(x)=\int_{\mathbb{R}^{n}}g_{m,\Phi} (x,y)\phi(y)dy,x\in\mathbb{R}^{n},\phi\in C_{0}^{\infty}\left( \mathbb{R} ^{n},\mathbb{C}^{N}\right) $$
(5)

with the Schwartz kernel \(g_{m,\Phi}\in C^{\infty}(\mathbb{R}^{n} \times\mathbb{R}^{n}\diagdown\left\{ (x,y)\in\mathbb{R}^{2n}:x\neq y\right\} ),\) satisfying the estimate

$$\left\Vert \partial_{x}^{\alpha}\partial_{y}^{\beta}g_{m,\Phi}(x,y)\right\Vert _{\hom(\mathbb{C}^{N})}\leq C_{\alpha\beta}e^{-\varepsilon\left\vert x-y\right\vert },\varepsilon>0,\left\vert x-y\right\vert >0. $$
(6)

We introduce the potential operator

$$\begin{aligned} \, \mathcal{P}_{m,\Phi,\Sigma}\psi(x) & =\left( \mathfrak{D}_{m,\Phi}^{-1} \psi\otimes\delta_{\Sigma}\right) (x) \\&= \int_{\Sigma}g_{m,\Phi}(x,\upsilon)\psi(\upsilon)d\upsilon,x \in \mathbb{R}^{n}\diagdown\Sigma,\psi\in C_{0}^{\infty}(\Sigma,\mathbb{C} ^{N})\nonumber \end{aligned}$$
(7)

satisfying the following properties:

(i) jf the \(C^{\infty}\)-hypersurface \(\Sigma\) \(\in\mathcal{R}(n-1)\) the operators

$$\begin{aligned} \, \mathcal{P}_{m,\Phi,\Sigma} & :B_{p,p}^{r-\frac{1}{p}}(\Sigma,\mathbb{C} ^{N})\rightarrow H^{r,p}(\mathbb{R}^{n}\diagdown\Sigma,\mathbb{C}^{N}),\\ \mathcal{P}_{m,\Phi,\Sigma} & :B_{p,q}^{r-\frac{1}{p}}(\Sigma,\mathbb{C} ^{N})\rightarrow B_{p,q}^{r}(\mathbb{R}^{n}\diagdown\Sigma,\mathbb{C}^{N}) \end{aligned}$$

are bounded for \(r>\frac{1}{p}\),

(ii) \(\mathcal{P}_{m,\Phi,\Sigma}\psi(x),\) has no tangential limits

$$\mathcal{P}_{\Phi,m,\Sigma}^{\pm}\psi(\upsilon)=\lim_{\Omega_{\pm}\ni x\rightarrow\upsilon\in\Sigma}\mathcal{P}_{\Phi,m,\Sigma}\psi(x)=\pm\frac {i}{2}\alpha\cdot\nu(\upsilon)\psi(\upsilon)+\mathcal{K}_{m,\Phi,\Sigma} \psi(\upsilon),\upsilon\in\Sigma $$
(8)

where \(\nu(\upsilon)\) is the unit normal vector to \(\Sigma\) at the point \(\upsilon\in\Sigma\) directed to \(\Omega_{-},\) and \(\mathcal{K}_{m,\Phi,\Sigma }\) is the psdo in the class \(OPS_{1,0}^{0}(\Sigma,\mathbb{C}^{N})\) on \(\Sigma.\)

We are looking for the solution of the interaction problem

$$\begin{aligned} \, & \left\{ \begin{array} [c]{c} \mathfrak{D}_{m,\Phi}u=f\text{ on }\mathbb{R}^{n}\diagdown\Sigma\\ \mathfrak{B}_{\Sigma}u=f_{1}\text{ on }\Sigma \end{array} \right. , \\ f & \in X^{s}\left( \mathbb{R}^{n}\diagdown\Sigma,\mathbb{C}^{N}\right) ,f_{1}\in Y^{s-1/p}(\Sigma,\mathbb{C}^{N}),s>1/p\nonumber \end{aligned}$$
(9)

where \(Y^{s-1/p}(\Sigma,\mathbb{C}^{N})\) is one from the spaces \(B_{p,p} ^{s-1/p}(\Sigma,\mathbb{C}^{N}),B_{p,q}^{s-1/p}(\Sigma,\mathbb{C}^{N}).\) Substituting (8) in the interaction condition we obtain the pseudodifferential equation on the hypersurface \(\Sigma\)

$$\Xi_{m,\Phi,\Sigma}\psi(\upsilon)=\left( \mathbb{I}_{N}+\mathcal{K} _{m,\Phi,\Sigma}\right) \psi(\upsilon)=f_{2}(\upsilon)-\mathfrak{B}_{\Sigma }\mathfrak{D}_{\Phi,m}^{-1}f_{1}(\upsilon),\upsilon\in\Sigma$$

Note that the psdo \(\Xi_{m,\Phi,\Sigma}\) has the principle symbol

$$\sigma_{\Xi_{m,\Phi,\Sigma}}^{0}(\upsilon,\xi)=\mathbb{I}_{N}+\Gamma \mathcal{(}\upsilon)\frac{\alpha\cdot\xi}{\left\vert \xi\right\vert } ,\upsilon\in\Sigma,\xi\in T_{\upsilon}^{\ast}(\Sigma)\diagdown0$$

where \(T_{\upsilon}^{\ast}(\Sigma)\) is the cotangent space to \(\Sigma\) at the point \(\upsilon.\)

\(1^{0}.\) If \(\Sigma\) is a compact \(C^{\infty}-\)hypersurface then the operator \(\Xi_{m,\Phi,\Sigma}\) is a Fredholm operator in the spaces \(Y^{s-1/p} (\Sigma,\mathbb{C}^{N})\) if and only if

$$\det \sigma_{\Xi_{m,\Phi,\Sigma}}^{0}(\upsilon,\xi)\neq0,\forall(\upsilon,\xi)\in T_{\upsilon}^{\ast}(\Sigma)\diagdown0 $$
(10)

Condition (10) yields also the Fredholmness of the operator

$$\mathbb{D}_{m,\Phi,\mathfrak{B}_{\Sigma}}:X^{s}(\mathbb{R}^{n}\diagdown \Sigma,\mathbb{C}^{N})\rightarrow X^{s-1}(\mathbb{R}^{n}\diagdown \Sigma,\mathbb{C}^{N})\oplus Y^{s-1/p}\left( \Sigma,\mathbb{C}^{N}\right) ,s>1/p.$$

\(2^{0}.\) Let the \(C^{\infty}-\)hypersurface \(\Sigma\) has the structure of an unbounded manifold of the class \(\mathcal{R}(n-1)\). Then \(\Sigma\) admits the compactification \(\hat{\Sigma}\) by the set of the infinitely distant points \(\hat{\Sigma}_{\infty}\) (see [34, 37]). We study the Fredholm property of the psdo \(\Xi_{m,\Phi,\Sigma}\) on \(\hat{\Sigma}\) using the local principle and the limit operators method on \(\hat{\Sigma}\) following paper [37]) (note the paper [17] devoted to the Fredholm theory of psdo’s on some noncompact manifolds).

We assume that \(\mathbb{R}^{n}\) is equipped with the structure of a manifold of class \(\mathcal{R}(n)\) and \(\Sigma\) is its submanifold

The main result of this chapter is the following theorem.

Theorem 1.

Let (i) \(m,\Phi\in SO^{\infty}(\mathbb{R}^{n});\) (ii) \(\Sigma \in\mathcal{R}(n-1);\) (iii) \(\Gamma\in SO^{\infty}(\Sigma)\otimes \mathcal{B}(\mathbb{C}^{N});\) (iv) the operator \(\mathfrak{D}_{m,\Phi} :H^{1,2}(\mathbb{R}^{n},\mathbb{C}^{N})\rightarrow L^{2}(\mathbb{R} ^{n},\mathbb{C}^{N})\) is invertible; (v) condition (10) is satisfied at every point \(\upsilon,\xi_{v}\in T^{\ast}(\Sigma);\) (vi) condition

$$\liminf_{\Sigma\ni\upsilon\rightarrow\upsilon_{\infty}}\inf_{\xi_{\upsilon}\in T_{\upsilon}^{\ast}(\Sigma)}\left\vert \det\left( \mathbb{I}_{N} +\Gamma(\upsilon)\frac{\alpha\cdot\xi_{\upsilon}+m(\upsilon)\alpha_{n+1} +\Phi(\upsilon)\mathbb{I}_{N}}{2\left( \left\vert \xi_{\upsilon}\right\vert ^{2}+m^{2}(\upsilon)-\Phi^{2}(\upsilon)\right) ^{1/2}}\right) \right\vert >0 $$
(11)

is satisfied for every infinitely distant point \(\upsilon_{\infty}\in \hat{\Sigma}_{\infty}.\)

Then \(\Xi_{m,\Phi,\Sigma}\) is a Fredholm operator in each space \(Y^{s} (\Sigma,\mathbb{C}^{N}),s\in\mathbb{R}\), \(ind\) \(\Xi_{m,\Phi,\Sigma}\) is independent of the space \(Y^{s}(\Sigma,\mathbb{C}^{N}),\) and \(ind\) \(\Xi_{m,\Phi,\Sigma}=0\) if \(m,\Phi\) are real-valued functions, and \(\Gamma(\upsilon)\) is an Hermitian matrix for each \(\upsilon\in\Sigma.\) Moreover,

$$\mathbb{D}_{m,\Phi,\mathfrak{B}_{\Sigma}}:X^{s}(\mathbb{R}^{n}\diagdown \Sigma,\mathbb{C}^{N})\rightarrow X^{s-1}(\mathbb{R}^{n}\diagdown \Sigma,\mathbb{C}^{N})\oplus Y^{s-1/p}\left( \Sigma,\mathbb{C}^{N}\right)$$

is a Fredholm operator for each \(s>1/p,p,q\in(1,\infty).\)

The proof of this theorem is based on the local principle in the compactification \(\mathbb{\hat{R}}^{n},\hat{\Sigma}\) and the limit operators approach (see [35, 36]).

It should be noted that the method of the potential operators for reduction of the boundary and transmission problems has a wide applications in the partial differential equations theory, mathematical physics, and numerical analysis (see, for instance, [1, 2, 21, 28, 22] for enough smooth boundaries or interaction hypersurfaces, and for non smooth hypersurfaces, see [2, 26, 25, 47], and references cited there.

Transmission and boundary problems for the Helmholtz equations on unbounded smooth hypersurfaces in \(\mathbb{R}^{n}\) have been considered in the papers [36, 42].

The paper is organized as follows. In Chap.2 we give the necessary notations and auxiliary materials on the Sobolev and Besov spaces, and pseudodifferential operators on \(\mathbb{R}^{n}\) acting in these spaces. We also introduce a class of noncompact manifolds following to the papers [34, 37], pseudodifferential operators on these manifolds, and the Fredholm theory for them.

In the Chap.3 we consider the \(L_{p}-\)theory of interaction problems on unbounded hypersurfaces of the class \(\mathcal{R(}n-1)\) introduced in Chap.2, associated with the formal Dirac operators with singular potentials. We consider the Dirac operator \(\mathfrak{D}_{m,\Phi}\) with the slowly oscillating at infinity mass \(m\) and electrostatic potential \(\Phi,\) and we study the inverse operator\(\mathfrak{\ D}_{m,\Phi}^{-1}.\) Then applying the potential operator generated by \(\mathfrak{D}_{m,\Phi}^{-1}\) we reduce the interaction problem \(\mathbb{D}_{m,\Phi.\mathfrak{B}_{\Sigma}}u=(f,f_{1})\) to a pseudodifferential equation on the interaction hypersurface \(\Sigma.\) We study the Fredholm theory of these pseudodifferential operators and apply them to the study of the Fredholm theory of the interaction problems on unbounded hypersurfaces of the class \(\mathcal{R}(n-1).\) The conical at infinity and slowly oscillating at infinity hypersurfaces are important examples of the interaction hypersurfaces under consideration.

2. Notations and auxiliary material

  1. If \(X,Y\) are Banach spaces then we denote by \(\mathcal{B(}X,Y)\) the space of bounded linear operators acting from \(X\) into \(Y\) with the uniform operator topology, and by \(\mathcal{K}(X,Y)\) the subspace of \(\mathcal{B(} X,Y)\) of all compact operators. In the case \(X=Y\) we write shortly \(\mathcal{B}(X)\) and \(\mathcal{K}(X).\)

  2. We denote by \(C_{b}^{\infty}(\mathbb{R}^{n})\) the space of infinitely differentiable of functions on \(\mathbb{R}^{n}\) bounded with all their derivatives, and we set

    $$SO^{\infty}(\mathbb{R}^{n})=\left\{ a\in C_{b}^{\infty}(\mathbb{R}^{n})\text{ such that }\lim_{x\rightarrow\infty}\partial_{x_{j}}a(x)=0,j=1,...,n\right\} .$$

    The functions of the class \(SO^{\infty}(\mathbb{R}^{n})\) are called slowly oscillating at infinity.

    Let \(\Sigma\) be a \(C^{\infty}-\)hypersurface in \(\mathbb{R}^{n}.\)Then we denote by \(C_{b}^{\infty}(\Sigma)\) the class of infinitely differentiable functions \(f\) on \(\Sigma\) such that \(\sup_{x\in\Sigma}\left\vert \partial^{\alpha }f(x)\right\vert <\infty\) for all multi-indices \(\alpha\) and if \(\Sigma\) is an unbounded hypersurface we define the class of slowly oscillating functions on \(\Sigma\) as

    $$SO^{\infty}(\Sigma)=\left\{ a\in C_{b}^{\infty}(\Sigma):\lim_{\Sigma \ni\upsilon\rightarrow\infty}\partial_{x_{j}}a(\upsilon)=0.\right\}$$

2.1. Bessel potentials and Besov spaces

  1. As usual, \(H^{s,p}(\mathbb{R}^{n})\), \(s\in\mathbb{R},p\in(1,\infty)\) is the Sobolev space (the space of Bessel potentials) on \(\mathbb{R}^{n}\) that is the space of distributions \(u\in\) \(S^{\prime}(\mathbb{R}^{n})\) such that

    $$\left\Vert u\right\Vert _{H^{s,p}(\mathbb{R}^{n})}=\left( \int_{\mathbb{R} ^{n}}\left\vert \left\langle D\right\rangle ^{s}u\right\vert ^{p}dx\right) ^{1\backslash p}<\infty,$$

    where \(\left\langle D\right\rangle ^{s}=(I-\Delta)^{s/2}\) is a \(psdo\) with symbol \((1+\left\vert \xi\right\vert ^{2})^{\frac{s}{2}}\). In the case \(p=2\) we use the standard notation \(H^{s,2}(\mathbb{R}^{n}):=H^{s}(\mathbb{R}^{n}).\)

  2. We introduce the Littlewood–Paley partition of unity

    $$\sum_{k=0}^{\infty}\lambda_{k}(\xi)=1,\xi\in\mathbb{R}^{n} $$
    (12)

    with \(\lambda_{0}(\xi)=\eta_{0}(\xi),\lambda_{k}(\xi)=\eta_{k}(\xi)-\eta _{k-1}(\xi),k\in\mathbb{N}\) where \(\eta_{0}\in C_{0}^{\infty}(\mathbb{R} ^{N}),\) so that:

    (i) \(\eta_{0}(\xi)=1\) for \(\left\vert \xi\right\vert \leq1\) and \(0\) for \(\left\vert \xi\right\vert \geq2,\)

    (ii) \(\eta_{k}(\xi)=\eta_{0}(2^{-k}\xi).\)

    Note some properties of the partition of unity (46): (i) supp \(\lambda_{0}=\left\{ \xi\in\mathbb{R}^{n}:\left\vert \xi\right\vert \leq2\right\} ,\) supp \(\lambda_{k}=\left\{ \xi\in\mathbb{R} ^{n}:2^{k-1}\leq\left\vert \xi\right\vert \leq2^{k+1}\right\} ;\) (ii) Let \(\psi_{k}=\lambda_{k-1}+\lambda_{k}+\lambda_{k+1},k\in\mathbb{N} _{0},\lambda_{-1}=0.\) Then \(\psi_{k}(\xi)=1\) for \(\xi\in\) supp \(\lambda_{k}.\) Hence \(\lambda_{k}\psi_{k}=\lambda_{k}\) and supp \(\psi_{k}=\left\{ \xi\in\mathbb{R}^{n}:2^{k-2}\leq\left\vert \xi\right\vert \leq2^{k+2}\right\} ,k\geq2.\)

    The Besov space \(B_{p,q}^{s}(\mathbb{R}^{n}),s\in\mathbb{R},1\leq p,q\leq\infty\) is defined as the space of distributions \(u\in S^{\prime }(\mathbb{R}^{n})\) with the finite norm:

    $$\left\Vert u\right\Vert _{B_{p,q}^{s}(\mathbb{R}^{n})}==\left\{ \begin{array} [c]{c} \left( \sum_{k=0}^{\infty}\left\Vert 2^{sk}\lambda_{k}(D)u\right\Vert _{L^{p}(\mathbb{R}^{n})}^{q}\right) ^{\frac{1}{q}}<\infty,q\in\lbrack 1,\infty)\\ \sup_{k\in\mathbb{N}_{0}}\left\Vert 2^{sk}\lambda_{k}(D)u\right\Vert _{L^{p}(\mathbb{R}^{n})}<\infty,q=\infty. \end{array} \right.$$

    Note that the space \(B_{\infty,\infty}^{s}(\mathbb{R}^{n})\) coincides with the Hölder–Zigmund space \(\Lambda^{s}(\mathbb{R}^{n}).\)

  3. Let \(X^{s}(\mathbb{R}^{n})\) be one of the space \(H^{s,p}(\mathbb{R} ^{n}),s\in\mathbb{R},p\in(1,\infty),\) \(B_{p,q}^{s}(\mathbb{R}^{n} ),s\in\mathbb{R},p,q\in\left[ 1,\infty\right] ,\) and \(X^{s}(\Omega)\) be the spaces of the restrictions of distributions \(u\in\) \(X^{s}(\mathbb{R}^{n})\) on \(\Omega\) with the standard norm

    $$\left\Vert u\right\Vert _{X^{s}(\Omega)}=\inf_{lu\in X^{s}(\mathbb{R}^{n} )}\left\Vert lu\right\Vert$$

    where \(lu\in X^{s}(\mathbb{R}^{n})\) is a an extension of \(u\in X^{s}(\Omega) \) on \(\mathbb{R}^{n}.\)

  4. Let \(\Omega\) be a domain in \(\mathbb{R}^{n}\) with \(C^{\infty}-\)boundary \(\partial\Omega,\) let \(S(\Omega),S(\partial\Omega)\) be the spaces of restriction of functions in the Schwartz space \(S(\mathbb{R}^{n})\) to \(\Omega,\) and let \(\gamma_{\mathbb{\partial\Omega}}:S(\Omega)\rightarrow\mathcal{S} (\partial\Omega)\) be the trace operator. Then \(\gamma_{\partial\Omega}\) is extended to the trace operator acting from \(H^{s,p}(\Omega)\) into \(B_{p,p}^{s-\frac{1}{p}}(\partial\Omega),\) and from \(B_{p,q}^{s}(\Omega)\) into \(B_{p,q}^{s-1/p}(\partial\Omega)\) if \(1<p,q<\infty,\) \(s>\frac{1}{p}.\) Let \(Y^{s}(\partial\Omega)\) is one of the spaces \(B_{p,p}^{s-\frac{1}{p}} (\partial\Omega),B_{p,q}^{s-\frac{1}{p}}(\partial\Omega).\)

    In what follows \(X^{s}(\Omega.\mathbb{C}^{N})=X^{s}(\Omega)\otimes \mathbb{C}^{N},Y^{s}(\Omega.\mathbb{C}^{N})=Y^{s}(\Omega)\otimes\mathbb{C} ^{N}.\)

    For more detail definitions and properties of the Besov spaces see for instance [7, 8, 44, 48].

  5. We introduce the weighted spaces \(H^{s,p}(\mathbb{R}^{n},\left\langle x\right\rangle ^{L}),\) \(B_{p,q}^{s}(\mathbb{R}^{n},\left\langle x\right\rangle ^{L}),p,q\in\left[ 1,\infty\right] ,L\in\mathbb{R}\) defined by the norm

    $$\left\Vert u\right\Vert _{H^{s,p}(\mathbb{R}^{n},\left\langle x\right\rangle ^{L})}=\left\Vert \left\langle x\right\rangle ^{L}u\right\Vert _{H^{s,p} (\mathbb{R}^{n})},\left\Vert u\right\Vert _{B_{p,q}^{s}(\mathbb{R} ^{n},\left\langle x\right\rangle ^{L})}=\left\Vert \left\langle x\right\rangle ^{L}u\right\Vert _{B_{p,q}^{s}(\mathbb{R}^{n})}.$$

    We denote by \(X^{s,L}(\mathbb{R}^{n},\mathbb{C}^{N})\) one of the space \(H^{s,p}(\mathbb{R}^{n},\left\langle x\right\rangle ^{L})\otimes\mathbb{C} ^{N},p\in(1,\infty),B_{p,q}^{s}(\mathbb{R}^{n},\left\langle x\right\rangle ^{L})\otimes\mathbb{C}^{N},p,q\in\left[ 1,\infty\right] .\)

2.2. Pseudodifferential operators (psdo) on \(\mathbb{R}^{n}\)

  1. We say that a function \(a\in C^{\infty}(\mathbb{R}^{n}\times \mathbb{R}^{n},\mathcal{B}(\mathbb{C}^{N}))\) belongs to the class \(S_{1,0} ^{m}(\mathbb{R}^{n},\mathbb{C}^{N})=S^{m}(\mathbb{R}^{n},\mathbb{C}^{N})\) if

    $$|a|_{l_{1},l_{2}}:=\sum_{|\alpha|\leq l_{1},|\beta|\leq l_{2},}\sup _{(x,\,\xi)\in{\mathbb{R}}^{n}\times{\mathbb{R}}^{n}}\left\Vert (\partial _{x}^{\beta}\partial_{\xi}^{\alpha}a(x,\,\xi))\left\langle \xi\right\rangle ^{-m+\left\vert \alpha\right\vert }\right\Vert _{\mathcal{B}(\mathbb{C}^{N} )}<\infty $$
    (13)

    for every \(l_{1},l_{2}\in{\mathbb{N}}_{0}=\mathbb{N\cup}\left[ 0\right) \) The semi-norms \(|a|_{l_{1},l_{2}}\) define the Freshet topology on \(S^{m}(\mathbb{R}^{n},\mathbb{C}^{N})\). The functions in \(S^{m}(\mathbb{R} ^{n},\mathbb{C}^{N})\) are called symbols. We associate with each symbol \(a\in S^{m}(\mathbb{R}^{n},\mathbb{C}^{N})\) the pseudodifferential operator ( psdo ) \(A=Op(a)\)

    $$\begin{aligned} \, Op(a)u(x) & :=(2\pi)^{-n}\int_{{\mathbb{R}}^{n}}d\xi\int_{{\mathbb{R}}^{n} }a(x,\,\xi)u(y)e^{i(x-y)\cdot\xi}dy,\ \ \ \\ u & \in S({\mathbb{R}}^{n},\mathbb{C}^{N})=S(\mathbb{R}^{n})\otimes \mathbb{C}^{N}. \end{aligned}$$

    We denote by \(OPS^{m}(\mathbb{R}^{n},\mathbb{C}^{N})\) the class of pseudodifferential operators (psdo\(^{\prime}\)s) with symbols in \(S^{m}(\mathbb{R}^{n},\mathbb{C}^{N})\) and we denote by \(\mathcal{S} ^{m}(\mathbb{R}^{n},\mathbb{C}^{N})\) the class of the classical symbols \(a(x,\xi)\in S^{m}(\mathbb{R}^{n},\mathbb{C}^{N})\) such that there exists the principle symbol \(a^{0}(x,\xi)\)

    $$\lim_{t\rightarrow\infty}t^{-m}a(x,t\xi)=a^{0}(x,\xi)\text{ for every } \xi:\left\vert \xi\right\vert =1.$$

We say that the symbol \(a(x,\xi)\in\mathcal{S}^{m}(\mathbb{R}^{n} ,\mathbb{C}^{N})\) is slowly oscillating at infinity and belongs to the class \(\mathcal{S}_{sl}^{m}(\mathbb{R}^{n},\mathbb{C}^{N})\) if for all multi-indices \(\alpha\) and \(\beta\neq0\)

$$\lim_{x\rightarrow\infty}\sup_{\xi\in\mathbb{R}^{n}}\left\vert (\partial _{x}^{\beta}\partial_{\xi}^{\alpha}a_{kl}(x,\,\xi))\left\langle \xi \right\rangle ^{-m+\left\vert \alpha\right\vert }\right\vert =0,k,l=1,...,N $$
(14)

and we say that the symbol \(a(x,\xi)\) belongs to the class \(\mathcal{\mathring {S}}^{m}(\mathbb{R}^{n},\mathbb{C}^{N})\) if conditions (14) holds for all \(\alpha,\beta.\)

Proposition 2.

(see, [32, 43, 38, 8, 44] Let \(Op(a)\in OPS^{m}(\mathbb{R} ^{n},,\mathbb{C}^{N})\). Then: (i) for all \(s,L\in\mathbb{R},\) and \(p,q\in(1,\infty)\) the operator

$$A=Op(a):X^{s,L}(\mathbb{R}^{n},\mathbb{C}^{N})\rightarrow X^{s-m,L} (\mathbb{R}^{n},\mathbb{C}^{N})$$

is bounded, and

$$\left\Vert Op(a)\right\Vert _{\mathcal{B}(X^{s,L}(\mathbb{R}^{n} ,\mathbb{C}^{N}),X^{s-m,L}(\mathbb{R}^{n},\mathbb{C}^{N}))}\leq C\left\vert a\right\vert _{L_{1},L_{2}} $$
(15)

with \(C>0,L_{1},L_{2}\in\mathbb{N}\) independent of \(a.\)

(ii) Let \(a_{j}\in OPS_{sl}^{m_{j}}(\mathbb{R}^{n},\mathbb{C}^{N}),j=1,2.\) Then \(Op(a_{1})Op(a_{2})\subset S_{sl}^{m_{1}+m_{2}}(\mathbb{R}^{n} ,\mathbb{C}^{N})\) and

$$Op(a_{1})Op(a_{2})=Op(a_{1}a_{2})+Op(r)$$

where \(r\in\mathring{S}^{m_{1}+m_{2}-1}(\mathbb{R}^{n},\mathbb{C}^{N}),\)

(iii) The operator \(Op(a)\in OP\mathring{S}^{m-\varepsilon}(\mathbb{R} ^{n},\mathbb{C}^{N})\) is a compact operator from \(X^{s,L}(\mathbb{R} ^{n},\mathbb{C}^{N})\ \)into \(X^{s-m,L}(\mathbb{R}^{n},\mathbb{C}^{N}).\)

  1. We denote by \(\mathbb{\tilde{R}}^{n}\) the spherical compactification of \(\mathbb{R}^{n}\) obtain by adding to each ray \(l_{\omega}=\left\{ x\in\mathbb{R}^{n}:x=t\omega,t>0,\omega\in S^{n-1}\right\} \) the infinitely distant point \(x_{\infty}.\) The topology in \(\mathbb{\tilde{R}}^{n}\) is introduced such that \(\mathbb{\tilde{R}}^{n}\) is isomorphic to the closed unit ball \(\bar{B}_{1}(0).\) We denote by \(\mathbb{\tilde{R}}_{\infty}^{n}\) the set of all infinitely distant points in \(\mathbb{\tilde{R}}^{n}.\)

  2. Let \(A=Op(a)\in OP\mathcal{S}^{m}(\mathbb{R}^{n};\mathbb{C}^{N})\) is psdo with the symbol \(a(x,\xi)\in\mathcal{S}^{m}(\mathbb{R}^{n},\mathbb{C} ^{N})\), and a sequence \(\mathbb{R}^{n}\ni g_{k}\rightarrow x_{\infty} \in\mathbb{\tilde{R}}_{\infty}^{n}.\) Then Arcela–Ascoli theorem yields that the sequence \(a(x+g_{k},\xi)\) has a subsequence \(a(x+h_{k},\xi)\rightarrow a^{h}(x,\xi)\in\mathcal{S}^{m}(\mathbb{R}^{n},\mathbb{C}^{N})\) in the sense of the uniform convergence on the sets in \(\mathbb{R}^{n}\times\mathbb{R}^{n}.\) The operator \(Op(a^{h})\) is called the limit operators defined by the sequence \(h_{k}\rightarrow x_{\infty}.\) We denote by \(\text{Lim}_{x_{\infty}}Op(a)\) the set of all limit operators of \(Op(a)\) defined by such sequences, and we set

    $$\text{Lim}Op(a)= {\displaystyle\bigcup\limits_{x_{\infty}\in\mathbb{\tilde{R}}_{\infty}^{n}}} \text{Lim}_{x_{\infty}}Op(a). $$
    (16)

Proposition 3.

(see [35, 38]). The operator \(A=Op(a)\in \mathcal{S}^{m}(\mathbb{R}^{n},\mathbb{C}^{N})\) is a Fredholm operator from \(X^{s}(\mathbb{R}^{n},\mathbb{C}^{N})\) into \(X^{s-m}(\mathbb{R}^{n} ,\mathbb{C}^{N})\) if and only if the following conditions hold :

(i)

$$\det a^{0}(x,\xi)\neq0\textit{ for all }(x,\xi)\in\mathbb{R}^{n}\times S^{n-1} $$
(17)

(ii) all limit operators \(Op(a^{h})\in LimOp(a)\) are invertible from \(X^{s}(\mathbb{R}^{n},\mathbb{C}^{N})\) into \(X^{s-m}(\mathbb{R}^{n} ,\mathbb{C}^{N}). \)

Remark 4.

Let \(Op(a)\in OP\mathcal{S}_{sl}^{m}(\mathbb{R}^{n} ,\mathbb{C}^{N}).\) Then the limit operator \(Op(a^{h})\) has the symbol \(a^{h}(\xi)\) independent of \(x,\) and the condition of invertibility of \(Op(a^{h})\) is:

$$\inf_{\xi\in\mathbb{R}^{n}}\left\vert \det a^{h}(\xi)\left\langle \xi\right\rangle ^{-m}\right\vert >0.$$

Hence condition (ii) of Proposition 3 can be written as follows

$$\liminf_{x\rightarrow\infty}\inf_{\xi\in\mathbb{R}^{n}}\left\vert \det a(x,\xi)\left\langle \xi\right\rangle ^{-m}\right\vert >0. $$
(18)

2.3. Fredholmness of psdo on a class of noncompact manifolds

Let \(\mathfrak{X}\) be a \(C^{\infty}\) noncompact manifold of a finite dimension \(n,\) \(C^{\infty}(\mathfrak{X})\) the space of infinitely differentiable functions on \(\mathfrak{X},\) \(C_{b}^{\infty}(\mathfrak{X})\) the subspace of \(C^{\infty}(\mathfrak{X})\) consisting of functions bounded with all their derivatives, \(C_{0}^{\infty}(\mathfrak{X})\) the subspace of \(C^{\infty}(\mathfrak{X})\) consisting of functions with compact supports.

Definition 5.

(see [37, 34]). We say that a noncompact \(C^{\infty}\)-manifolds \(\mathfrak{X}\) of a finite dimension \(n\in\mathbb{N}\) belongs to the class \(\mathcal{R(}n\mathcal{)}\) if there exists a finite covering of \(\mathfrak{X}\) by open sets \(U_{j},j=1,...,J\) and for every \(j\in J\) there is a homeomorphism \(\varphi_{j}:\) \(U_{j}\rightarrow\varphi _{j}(U_{j})\subset\) \(\mathbb{R}^{N}\). Let \(J=J^{\prime}\cup J",\) \(J^{\prime }\cap J"=\varnothing.\) We assume that \(\varphi_{j}(U_{j}),j\in J^{\prime}\) are open bounded sets in \(\mathbb{R}^{n},\) and \(K_{j}=\varphi_{j}(U_{j}),j\in J"\) are open conical sets in \(\mathbb{R}^{n}.\) The transition functions

$$\varphi_{j_{2}}\circ\varphi_{j_{1}}^{-1}:\varphi_{j_{1}}(U_{j_{1}}\cap U_{j_{2}})\rightarrow\varphi_{j_{2}}(U_{j_{1}}\cap U_{j_{2}}),j_{1}\in J^{\prime}$$

are \(C^{\infty}-\)diffeomorphism , and

$$d\left( \varphi_{j_{2}}\circ\varphi_{j_{1}}^{-1}\right) (x)\in SO^{\infty }(K_{j_{1}})\otimes\mathcal{B(}\mathbb{C}^{N}),j_{1},j_{2}\in J_{2}$$

We denote by \(\mathfrak{\hat{X}}=\mathfrak{X}\cup\mathfrak{X}_{\infty}\) a compactification of \(\mathfrak{X}\) by adding the set \(\mathfrak{X}_{\infty}\) of ”infinitely distant points” such that the pair \((\mathfrak{X} ,\mathfrak{\hat{X}}_{\infty})\) is locally homeomorphic to the pair \((\mathbb{R}^{n},\mathbb{\tilde{R}}^{n})\). It means that every point \(\mathfrak{x}\in\mathfrak{X}\) has a fundamental system of neighborhoods homeomorphic to open bounded sets in \(\mathbb{R}^{n},\) and an infinitely distant point \(\mathfrak{x}_{\infty}\in\mathfrak{X}_{\infty}\) has a fundamental system of neighborhoods homeomorphic to open conical sets in \(\mathbb{R}^{n}.\)

In what follows we denote by \(\hat{U}\) the closure of \(U(\subset\mathfrak{X})\) in \(\mathfrak{\hat{X}}.\)

Definition 6.

(i) We say that \(\chi_{U}\in C_{0}^{\infty}(\mathfrak{X})\) is a cut-off function of a bounded open set \(U\subset\mathfrak{X}\) if \(0\leq \chi_{U}(x)\leq1,\) supp\(\chi_{U}\subset U,\) and there exists an open set \(U^{\prime}(\overline{U^{\prime}}\subset U)\) such that \(\chi_{U}(x)=1\) if \(x\in U^{\prime};\) (ii) We say that \(\chi_{U}\in C_{b}^{\infty}(\mathfrak{X})\) is a cut-off function of a neighborhood \(U\) of infinitely distant point \(\mathfrak{x}_{\infty}\) if \(0\leq\chi_{U} (\mathfrak{x})\leq1,\) supp\(\chi_{U}\subset U,\) there exists other neighborhood of \(U^{\prime}(\overline{U^{\prime}}\subset U)\) of \(\mathfrak{X}_{\infty},\) such that \(\chi_{U}(\mathfrak{x})=1\) if \(\mathfrak{x}\in U^{\prime};\)

It follows from the definition of a manifolds \(\mathfrak{X}\in\) \(\mathcal{R} (n)\) that there exists a finite partition of the unity

$$\sum_{j\in J}\chi_{j}(\mathfrak{x})=1,\mathfrak{x\in X,}\chi_{j}\in C_{b}^{\infty}(\mathfrak{X)}$$

subordinate to the covering of \(\mathfrak{X}\) by charts \(\left\{ U_{j},\varphi_{j}\right\} _{j\in J}\) and for every multi-index \(\alpha\)

$$\left\vert \partial^{\alpha}\left( \chi_{j}\circ\varphi_{j}^{-1}\right) (x)\right\vert \leq C_{\alpha}\left\langle x\right\rangle ^{-\left\vert \alpha\right\vert },x\in K_{j},j\in J. $$
(19)

Definition 7.

We denote by \(X^{s}(\mathfrak{X},\mathbb{C}^{N})\) one of the spaces \(H^{s,p}(\mathfrak{X},\mathbb{C}^{N}),B_{p,\mathfrak{q}}^{s} (\mathfrak{X},\mathbb{C}^{N})\) which in the local coordinates belongs to the spaces \(H^{s,p}(\mathfrak{X},\mathbb{C}^{N}),B_{p,\mathfrak{q}}^{s} (\mathfrak{X},\mathbb{C}^{N})\) with the norm

$$\left\Vert u\right\Vert _{X^{s}(\mathfrak{X},\mathbb{C}^{N})}=\sum_{j\in J}\left\Vert \left( \chi_{j}u\right) \circ\varphi_{j}\right\Vert _{X^{s}(\varphi_{j}(U_{j}),\mathbb{C}^{N})}. $$
(20)

It can be proven that another partition of unity leads to a norm equivalent to (20).

Definition 8.

We say that an operator \(A:C_{0}^{\infty}(\mathfrak{X} ,\mathbb{C}^{N})\rightarrow C^{\infty}(\mathfrak{X},\mathbb{C}^{N})\) is a pseudodifferential operator in the class \(OP\mathcal{S}^{m}(\mathfrak{X} ,\mathbb{C}^{N})\) \((OP\mathcal{S}_{sl}^{m}(\mathfrak{X},\mathbb{C}^{N}))\) if for every local chart \((U,\varphi)\) the operator

$$\mathcal{A}_{U}=\varphi^{\ast}r_{U}Ai_{U}\varphi_{\ast}:C_{0}^{\infty} (\varphi(U))\rightarrow C^{\infty}(\varphi(U)) $$
(21)

is a pseudodifferential operator in the class \(OP\mathcal{S}^{m} (\varphi(U),\mathbb{C}^{N})\) \(\left( OP\mathcal{S}_{sl}^{m}(\varphi (U),\mathbb{C}^{N})\right) \) where \(i_{U}:C_{0}^{\infty}(U,\mathbb{C} ^{N})\rightarrow C_{0}^{\infty}(\mathfrak{X},\mathbb{C}^{N})\) is the imbedding operator and \(r_{U}:C^{\infty}(\mathfrak{X},\mathbb{C}^{N})\rightarrow C^{\infty}(U,\mathbb{C}^{N})\) is the restriction operator, \(\varphi^{\ast }u=u\circ\varphi,\) and \(\varphi_{\ast}v=v\circ\varphi^{-1}.\)

We note some properties of the operators in the class \(OPS^{m}(\mathfrak{X} ,\mathbb{C}^{N})\) which follow from Proposition 2:

Proposition 9.

(i) A pseudodifferential operator \(A\in\) \(OPS^{m}(\mathfrak{X} ,\mathbb{C}^{N})\) is continued to a bounded operator from \(X^{s} (\mathfrak{X},\mathbb{C}^{N})\) into \(X^{s-m}(\mathfrak{X},\mathbb{C}^{N})\) for every \(s\in\mathbb{R};\) (ii) Let \(A_{j}\in\) \(OPS^{m_{j}}(\mathfrak{X} ,\mathbb{C}^{N}),j=1,2.\) Then the product \(A_{2}A_{1}\) is well defined and \(A_{2}A_{1}\in OPS^{m_{1}+m_{2}}\left( \mathfrak{X},\mathbb{C}^{N}\right) .\) If \(A_{j}\in OPS_{sl}^{m_{j}}(\mathfrak{X},\mathbb{C}^{N}),j=1,2,\) then \(A_{1}A_{2}-A_{2}A_{1}=\left[ A_{1},A_{2}\right] \in OP\mathcal{\mathring {S}}^{m_{1}+m_{2}-1}(\mathfrak{X,}\mathbb{C}^{N})\) is a compact operator from \(X^{s}(\mathfrak{X},\mathbb{C}^{N})\) into \(X^{s-(m_{1}+m_{2})}(\mathfrak{X} ,\mathbb{C}^{N});\) (iii) Let \(\mathcal{F}_{1},\mathcal{F}_{2}\) be two open sets in \(\mathfrak{X}\,\ \)such that \(\mathcal{\bar{F}}_{1}\cap\mathcal{\bar {F}}_{2}=\varnothing,\) and \(A\in OP\mathcal{S}^{m}(\mathfrak{X},\mathbb{C} ^{N}).\) Then \(\chi_{\mathcal{F}_{1}}A\chi_{\mathcal{F}_{2}}I\in\mathcal{K} (X^{s}(\mathfrak{X},\mathbb{C}^{N}),X^{s-m}(\mathfrak{X},\mathbb{C}^{N})),\) where \(\chi_{\mathcal{F}}\) is a cut-off function of a set \(\mathcal{F}.\)

We denote by \(\sigma_{A}(\mathfrak{x,\xi}_{\mathfrak{X}})\) the symbol of the operator \(A\) defined on the cotangent bundle \(T^{\ast}(\mathfrak{X)\otimes }\mathcal{B}(\mathbb{C}^{N})\) (see for instance [45, 46]) The symbol \(\sigma_{A}(\mathfrak{x,\xi}_{\mathfrak{X}})\) is unique up to a symbol \(q\) which in each local coordinate system belongs to \(\mathcal{\mathring{S}}^{m-1}(\mathbb{R}^{n},\mathbb{C}^{N}).\)

We say that the operator \(A\in OP\mathcal{S}^{m}(\mathfrak{X},\mathbb{C}^{N})\) is elliptic at the point \(\mathfrak{x}\in\mathfrak{X}\) if

$$\det a^{0}(\mathfrak{x},\xi_{\mathfrak{x}})\neq0,\forall(\mathfrak{x,} \xi_{\mathfrak{x}})\in S^{\ast}(\mathfrak{X)} $$
(22)

where \(a^{0}(\mathfrak{x},\xi)\) is the principal symbol of \(A\) defined on the spheric cotangent bundle \(S^{\ast}(\mathfrak{X})\) (see for instance [46]).

If the operator \(A\in OP\mathcal{S}^{m}(\mathfrak{X},\mathbb{C}^{N})\) is uniformly elliptic, that is

$$ \inf_{(\mathfrak{x},\xi)\in S^{\ast}(\mathfrak{X})}\left\vert \det a^{0}(\mathfrak{x},\xi)\right\vert >0,$$
(23)

then the a priory estimate holds

$$\left\Vert u\right\Vert _{X^{s,L}(\mathfrak{X},\mathbb{C}^{N})}\leq C(\left\Vert Au\right\Vert _{X^{s-m,L}(\mathfrak{X},\mathbb{C}^{N} )}+\left\Vert u\right\Vert _{X^{s-1,L-1}(\mathfrak{X},\mathbb{C}^{N})} ),s\in\mathbb{R} $$
(24)

with a constant \(C>0\) independent of \(u.\) It implies that every solution \(u\in X^{s}(\mathfrak{X},\mathbb{C}^{N})\) of the equation

$$Op(a)u=f\in S(\mathfrak{X},\mathbb{C}^{N})$$

belongs to the Schwartz space \(S(\mathfrak{X},\mathbb{C}^{N}).\)

Definition 10.

(see [38]) We say that an operator \(A\mathfrak{\in }\mathcal{B(}X^{s}(\mathfrak{X},\mathbb{C}^{N})\),\(X^{s-m}(\mathfrak{X} ,\mathbb{C}^{N}))\) is locally invertible at a point \(\mathfrak{x} \in\mathfrak{\hat{X}}\) if there exists a neighborhood \(U\) of \(\mathfrak{x},\) a cut-off function \(\chi_{U}\) of \(U,\) and operators \(\mathfrak{L}_{U} ,\mathfrak{R}_{U}\in\mathcal{B(}X^{s-m}(\mathfrak{X},\mathbb{C}^{N})\) ,\(X^{s}(\mathfrak{X},\mathbb{C}^{N}))\) such that

$$\mathfrak{L}_{U}A\chi_{U}I=\chi_{U}I,\chi_{U}A\mathfrak{R}_{U}=\chi_{U}I.$$

Definition 11.

We say that \(A\mathfrak{\in}\mathcal{B(}X^{s}(\mathfrak{X} ,\mathbb{C}^{N})\),\(X^{s-m}(\mathfrak{X},\mathbb{C}^{N}))\) is a local type operator on \(\mathfrak{X}\) if for every open sets \(\mathcal{F}_{1} ,\mathcal{F}_{2}\subset\mathfrak{X}\) such that \(\mathcal{\bar{F}}_{1} \cap\mathcal{\bar{F}}_{2}=\varnothing,\) the operator \(\chi_{\mathcal{F}_{1} }A\chi_{\mathcal{F}_{2}}I\in\mathcal{K}(X^{s}(\mathfrak{X},\mathbb{C}^{N} )\),\(X^{s-m}(\mathfrak{X},\mathbb{C}^{N})).\)

Proposition 12.

(see [38]) Let \(A\mathfrak{\in}\mathcal{B(} X^{s}(\mathfrak{X},\mathbb{C}^{N}),X^{s-m}(\mathfrak{X},\mathbb{C}^{N}))\) be a local type operator on \(\mathfrak{X}.\) Then \(A\) is a Fredholm operator if and only if \(A\) is locally invertible at every point \(\mathfrak{x}\in \mathfrak{\hat{X}}.\)

We consider now the Fredholm property of \(A\in OP\mathcal{S}^{m} (\mathfrak{X},\mathbb{C}^{N})\) as operator acting from \(X^{s}(\mathfrak{X} ,\mathbb{C}^{N})\) into \(X^{s-m}(\mathfrak{X},\mathbb{C}^{N}).\) Following to Proposition 12 we have to consider the local invertibility of \(A\) at the points of the compactification \(\mathfrak{\hat{X}}.\) It is well known (see for instance [45, 46]) that \(A\in OPS^{m}(\mathfrak{X} ,\mathbb{C}^{N})\) is locally invertible at the point \(\mathfrak{x} \in\mathfrak{X}\) if and only if the principal symbol \(\sigma_{A} ^{0}(\mathfrak{x,\xi}_{\mathfrak{x}})\) is invertible at this point. It implies condition (22) at every point \(\mathfrak{x}\in\mathfrak{X}.\)

Let \(A=Op(a)\in OP\mathcal{S}^{m}(\mathfrak{X},\mathbb{C}^{N})\) the chart \((U,\varphi)\) be such that \(U\) is a neighborhood of the infinitely distant point \(x_{\infty}\in U_{\infty},\) \(\varphi:U\rightarrow K\) be a diffeomorphism extended to a continuous mapping \(\tilde{\varphi}:\hat{U}\rightarrow\tilde {K},\) \(A_{U}\) be the restriction of \(A\) on the neighborhood \(U\ \) with the local symbol \(\sigma_{A_{U}}(\mathfrak{x},\xi_{\mathfrak{x}}),\mathfrak{x}\in U,\xi_{\mathfrak{x}}\in T_{\mathfrak{x}}^{\ast}(U)\) be the symbol of the operator \(A_{U}.\) Let \(\mathcal{A}_{U}\) be the operator defined by formula (21). Then

$$\sigma_{\mathcal{A}_{U}}(x,\eta)=\sigma_{A_{U}}(\varphi^{-1}(\boldsymbol{x} ),d\varphi(x)\eta),x\in K,\eta\in\mathbb{R}^{n},$$

where \(\sigma_{\mathcal{A}_{U}}(x,\eta)\in\mathcal{S}^{m}(K,\mathbb{C}^{N}).\) Let \(x_{\infty}\in\tilde{K}_{\infty},\) and the sequence \(K\ni h_{k}\rightarrow x_{\infty}\) defines the limit operator \(\mathcal{A}_{U}^{h}\) with symbol

$$\sigma_{\mathcal{A}_{U}}^{h}(x_{\infty},\eta)=\lim_{h_{k}\rightarrow x_{\infty}}\sigma_{A_{U}}(\varphi^{-1}(x+h_{k}),d\varphi(x+h_{k})\eta )=\sigma_{A_{U}}(x_{\infty},\left( d\varphi\right) ^{h}\eta).$$

The limit is understood in the sense of converges on compact sets in \(K\times\mathbb{R}^n.\) Therefore, the condition of the local invertibility at the point \(\mathfrak{x}_{\infty}\) is:

$$\liminf_{\mathfrak{x}\rightarrow\mathfrak{x}_{\infty}}\inf_{\xi_{\mathfrak{x} }\in T_{\mathfrak{x}}^{\ast}(\mathfrak{X)}}\det\left\langle \xi\right\rangle ^{-m}\sigma_{A_{U}}(\mathfrak{x,}\xi_{\mathfrak{x}})\text{=}\liminf _{y\rightarrow y_{\infty}}\inf_{\eta\in\mathbb{R}^{n}}\left\vert \det\left\langle \eta\right\rangle ^{-m}\sigma_{\mathcal{A}_{U}} (y,\eta)\right\vert >0. $$
(25)

Theorem 13.

An operator \(A\in OP\mathcal{S}_{sl}^{m}(\mathfrak{X} ,\mathbb{C}^{N})\) acting from \(X^{s}(\mathfrak{X},\mathbb{C}^{N})\) into \(X^{s-m}(\mathfrak{X},\mathbb{C}^{N})\) is a Fredholm operator if the following conditions hold:

(i) the principal symbol \(\sigma_{A}^{0}(\mathfrak{x},\xi_{\mathfrak{x}})\) is invertible at every point \(\mathfrak{x,}\ \mathfrak{\xi}_{\mathfrak{x}}\in T_{\mathfrak{x}}^{\ast}(\mathfrak{X\dot{)}:}\left\vert \xi_{\mathfrak{x} }\right\vert =1;\) where \(T_{\mathfrak{x}}^{\ast}(\mathfrak{X)}\) is the cotangent space to \(\mathfrak{X}\) at the point \(\mathfrak{x}\)

(ii) for every infinitely distant point \(\mathfrak{x}_{\infty}\in \mathfrak{\hat{X}}_{\infty}\)

$$\liminf_{\mathfrak{X}\ni\mathfrak{x}\rightarrow\mathfrak{x}_{\infty}}\inf _{\xi_{\mathfrak{x}}\in T_{\mathfrak{x}}^{\ast}(\mathfrak{X)}}\left\vert \det\left\langle \xi_{\mathfrak{x}}\right\rangle ^{-m}\sigma_{A_{U} }(\mathfrak{x,}\xi_{\mathfrak{x}})\right\vert >0. $$
(26)

Proof.

Following Proposition 12. we have to consider the local invertibility of \(A\) at every point of the compactification \(\mathfrak{\hat{X} }\) of \(\mathfrak{X}.\) It is well known (see, for instance, [46]) that \(A\in OPS^{m}(\mathfrak{X},E)\) is locally invertible at the point \(\mathfrak{x}\in\mathfrak{X}\) if the principal symbol \(\sigma_{A}^{0}\) is invertible at this point, that is condition (\refe1) holds. Let \(A=Op(a)\in OP\mathcal{S}_{sl}^{m}(\mathfrak{X},\mathbb{C}^{N})\) and the chart \((U,\varphi)\) be such that \(U\) is a neighborhood of the infinitely distant point \(\mathfrak{x}_{\infty}\in\tilde{U}_{\infty},\) \(\varphi:U\rightarrow K\) be the diffeomorphism extended to the continuous mapping \(\tilde{\varphi }:\tilde{U}\rightarrow\tilde{K},\) \(A_{U}\) be the restriction of \(A\) on the neighborhood \(U,\) and \(\sigma_{A_{U}}(\mathfrak{x},\xi_{\mathfrak{x} }),\mathfrak{x}\in U,\xi_{\mathfrak{x}}\in T_{\mathfrak{x}}^{\ast }(\mathfrak{X})\) be the symbol of the operator \(A_{U}\) where \(T_{\mathfrak{x} }^{\ast}(\mathfrak{X})\) is the cotangent space to \(\mathfrak{X}\) at the point \(\mathfrak{x.}\) Let \(\mathcal{A}_{U}\) be the operator defined by formula (21\().\) Note that

$$\sigma_{\mathcal{A}_{U}}(x,\eta)=\sigma_{A_{U}}(\varphi^{-1}(x),d\varphi (x)\eta),x\in K,\eta\in\mathbb{R}^{n},$$

where \(\sigma_{\mathcal{A}_{U}}(x,\eta)\in\mathcal{S}_{sl}^{m}(K,\mathbb{C} ^{N}).\) Let \(x_{\infty}\in\tilde{K}_{\infty},\) and the sequence \(K\ni h_{k}\rightarrow x_{\infty}\) defines the limit operator \(\mathcal{A}_{U}^{h}\) with symbol

$$\sigma_{\mathcal{A}_{U}}^{h}(x_{\infty},\eta)=\lim_{h_{k}\rightarrow x_{\infty}}\sigma_{A_{U}}(\varphi^{-1}(x+h_{k}),d\varphi(x+h_{k})\eta )=\sigma_{A_{U}}(x_{\infty},\left( d\varphi\right) ^{h}\eta).$$

The limit is understood in the sense of converges on compacts in \(K\times\mathbb{R}^n.\) By Proposition 3 the condition of local invertibility at the point \(\mathfrak{x}_{\infty}\) is written as

$$\liminf_{\mathfrak{x}\rightarrow\mathfrak{x}_{\infty}}\inf_{\xi_{\mathfrak{x} }\in T_{\mathfrak{x}}^{\ast}(\mathfrak{X)}}\left\vert \det\left\langle \xi\right\rangle ^{-m}\sigma_{A}(\mathfrak{x,}\xi_{\mathfrak{x}})\right\vert >0. $$
(27)

Therefore, if condition (26) holds, then condition (27) holds at every point \(\mathfrak{x}_{\infty}\in\mathfrak{X}_{\infty},\) that is \(A\) is locally invertible at every point \(\mathfrak{x}_{\infty}\in\mathfrak{X} _{\infty}\). By Proposition 12 the operator \(A:X^{s}(\mathfrak{X} ,\mathbb{C}^{N})\rightarrow X^{s-m}(\mathfrak{X},\mathbb{C}^{N})\) is a Fredholm operator.

Corollary 14.

Let \(A=Op(a)\in OPS_{sl}^{m}\left( \mathfrak{X,}\mathbb{C} ^{N}\right) \). Then \(A:X^{s}(\mathfrak{X},\mathbb{C}^{N})\rightarrow X^{s-m}(\mathfrak{X},\mathbb{C}^{N})\) is a Fredholm operator if conditions (i) holds, and condition (ii) is changed by the condition

$$\liminf_{\mathfrak{X}\ni\mathfrak{x}\rightarrow\mathfrak{\infty}}\inf _{\xi_{\mathfrak{x}}\in T_{\mathfrak{x}}^{\ast}(\mathfrak{X)}}\left\vert \det\left\langle \xi_{\mathfrak{x}}\right\rangle ^{-m}\sigma_{A_{U} }(\mathfrak{x,}\xi_{\mathfrak{x}})\right\vert >0. $$
(28)

3. Fredholm theory of interaction problems associated with Dirac operators with singular potentials

3.1. Realization of the Dirac operators with singular potential as the interaction problem

Let \(D_{m,,\Phi,\Gamma\delta_{\Sigma}}=\mathfrak{D}_{m,\Phi}+\Gamma \delta_{\Sigma}\) be the formal Dirac operator defined by formulas (1), (3). We assume that \(\Sigma\) is a \(C^{\infty} -\)hypersurface in \(\mathbb{R}^{n}\) which is a common boundary of domains \(\Omega_{\pm}\subset\mathbb{R}^{n},n\geq2,\Phi,m\in C_{b}^{\infty} (\mathbb{R}^{n}),\) \(\Gamma_{i,j}\in C_{b}^{\infty}(\Sigma),i,j=1,...,N.\) The product \(\Gamma\delta_{\Sigma}u\) where \(u\) \(\in X^{s}(\mathbb{R}^{n} \diagdown\Sigma,\mathbb{C}^{N}),s>\frac{1}{p},p\in(1,\infty)\) is defined as the distribution in \(\mathcal{D}^{\prime}(\mathbb{R}^{n},\mathbb{C} ^{N})=\mathcal{D}^{\prime}(\mathbb{R}^{n})\otimes\mathbb{C}^{N}\)

$$\left( \Gamma\delta_{\Sigma}u\right) (\varphi)=\frac{1}{2}\int_{\Sigma }\Gamma(s)\left( \gamma_{\Sigma}^{+}u(s)+\gamma_{\Sigma}^{-}u(\sigma)\right) \cdot\varphi(\sigma)d\sigma,\varphi\in C_{0}^{\infty}(\mathbb{R} ^{n},\mathbb{C}^{N}). $$
(29)

\(d\sigma\) is the hypersurface Lebesgue measure, \(\gamma_{\Sigma}^{\pm} :X^{s}(\Omega_{\pm},\mathbb{C}^{N})\rightarrow Y^{s-1/p}(\Sigma,\mathbb{C} ^{N})\) where \(Y^{s}(\Omega_{\pm},\mathbb{C}^{N})=B_{p,p}^{s-1/p,p}\left( \Sigma,\mathbb{C}^{N}\right) \) if \(X^{s}(\Omega_{\pm},\mathbb{C}^{N} )=H^{s,p}(\Omega_{\pm},\mathbb{C}^{N})\) and \(Y^{s}(\Omega_{\pm},\mathbb{C} ^{N})=B_{p,q}^{s-1/p,p}\left( \Sigma,\mathbb{C}^{N}\right) \) if \(X^{s}(\Omega_{\pm},\mathbb{C}^{N})=H^{s,p}(\Omega_{\pm},\mathbb{C}^{N}).\)

Integrating by parts and taking into account (29) we obtain that

$$\begin{aligned} \, & \left\langle D_{m,\Phi,\Gamma\delta_{\Sigma}}u,\varphi\right\rangle _{L^{2}(\mathbb{R}^{n},\mathbb{C}^{N})}=\int_{\Omega_{+}\cup\Omega_{-} }\mathfrak{D}_{m,\Phi}u(x)\cdot\varphi(x)dx \\ & -\int_{\Sigma}i\alpha\cdot\nu(\sigma)(\gamma_{\Sigma}^{+}u(\sigma )-\gamma_{\Sigma}^{-}u(\sigma))\cdot\varphi(\sigma))d\sigma\nonumber\\ & +\frac{1}{2}\int_{\Sigma}\left( \Gamma(\sigma)\left( \gamma_{\Sigma} ^{+}u(\sigma)+\gamma_{\Sigma}^{-}u(\sigma)\right) \cdot\varphi(\sigma )\right) d\sigma,\varphi\in C_{0}^{\infty}(\mathbb{R}^{n},\mathbb{C} ^{N})\nonumber \end{aligned}$$
(30)

where \(\nu(\sigma)=(\nu_{1}(\sigma),...,\nu_{n}(\sigma))\) is the unit normal vectors on \(\Sigma\) directed to \(\Omega_{-}\). Formula (30) yields that

$$\begin{aligned} \, & D_{A,\Phi,m,\Gamma\delta_{\Sigma}}u \\ & =\mathfrak{D}_{A,\Phi,m}u-\left[ i\alpha\cdot\nu\left( \gamma_{\Sigma }^{+}u-\gamma_{\Sigma}^{-}u\right) -\frac{1}{2}\Gamma\left( \gamma_{\Sigma }^{+}u+\gamma_{\Sigma}^{-}u\right) \right] \delta_{\Sigma},\nonumber \end{aligned}$$
(31)

in the distribution sense, where \(\mathfrak{D}_{m,\Phi}u\) is the regular distribution defined by the vector-valued function \(\mathfrak{D}_{m,\Phi}u.\) Hence \(D_{m,\Phi,\Gamma\delta_{\Sigma}}u\) is the regular distribution if and only if

$$-i\alpha\cdot\nu\left( \gamma_{\Sigma}^{+}u-\gamma_{\Sigma}^{-}u\right) +\frac{1}{2}\Gamma\left( \gamma_{\Sigma}^{+}u+\gamma_{\Sigma}^{-}u\right) =0\text{ on }\Sigma. $$
(32)

We write condition (32) of the form

$$\mathfrak{B}_{\Sigma}u(\sigma)=a_{+}(\sigma)\gamma_{\Sigma}^{+}u(\sigma )+a_{-}(\sigma)\gamma_{\Sigma}^{-}u(\sigma)=0,\sigma\in\Sigma $$
(33)

where \(a_{\pm}(\sigma)\) are \(N\times N\) matrices

$$a_{\pm}(\sigma)=\frac{1}{2}\Gamma(\sigma)\mp i\alpha\cdot\nu(\sigma),\sigma \in\Sigma. $$
(34)

Therefore, we can associate with the formal Dirac operator \(D_{m,\Phi ,\Gamma\delta_{\Sigma}}\) the bounded operator of the interaction (transmission) problem

$$\mathbb{D}_{m,\Phi,\mathfrak{B}_{\Sigma}}u=\left\{ \begin{array} [c]{c} \mathfrak{D}_{m,\Phi}u\text{ on }\mathbb{R}^{n}\diagdown\Sigma\\ \mathfrak{B}_{\Sigma}u(\sigma)=a_{+}\gamma_{\Sigma}^{+}u+a_{-}\gamma_{\Sigma }^{-}u\text{ on }\Sigma \end{array} \right. $$
(35)

acting from \(X^{s}(\mathbb{R}^{n}\mathbb{\diagdown}\Sigma,\mathbb{C} ^{N}\mathbb{)}\) into \(X^{s-1}(\mathbb{R}^{n}\diagdown\Sigma,\mathbb{C} ^{N}\mathbb{)}\oplus Y^{s-1/p}(\Sigma,\mathbb{C}^{N}),s>1/p.p\in(1,\infty).\)

3.2. Inverse operator to the Dirac operator on \(\mathbb{R}^{n}\)

We consider the Dirac operator

$$\mathfrak{D}_{m,\Phi}=\alpha\cdot D+m\alpha_{n+1}+\Phi\mathbb{I}_{N}$$

defined by (1),(2) where \(m,\Phi\in C_{b}^{\infty} (\mathbb{R}^{n}).\) We give some properties of \(\mathfrak{D}_{m,\Phi}\) which will be used below:

  1. \(\ \mathfrak{D}_{m,\Phi}\) is a psdo of the class \(OP\mathcal{S} ^{1}\left( \mathbb{R}^{n},\mathbb{C}^{N}\right) \) with symbol

    $$\sigma_{\mathfrak{D}_{m,\Phi}}(x,\xi)=\alpha\cdot\xi+m(x)\alpha_{n+1} +\Phi(x)\mathbb{I}_{N},\left( x,\xi\right) \in\mathbb{R}^{n}$$

    and the principle symbol \(.\alpha\cdot\xi=\sum_{j=1}^{n}\alpha_{j}\xi_{j} ,\xi=(\xi_{1},...,\xi_{n})\in S^{n-1}.\)

  2. \(\mathfrak{D}_{m,\Phi}\) is elliptic on \(\mathbb{R}^{n}\) since \(\left( \alpha\cdot\xi\right) ^{2}=\left\vert \xi\right\vert ^{2}\mathbb{I}_{N}.\) The ellipticity of \(\mathfrak{D}_{m,\Phi}\) yields the a priori estimate

    $$\begin{aligned} \, \left\Vert u\right\Vert _{X^{s,L}(\mathbb{R}^{n},\mathbb{C}^{N})} & \leq C\left( \left\Vert \mathfrak{D}_{m,\Phi}u\right\Vert _{X^{s-1,L} (\mathbb{R}^{n},\mathbb{C}^{N},\left\langle x\right\rangle ^{r})}+\left\Vert u\right\Vert _{X^{s-1,L-1}(\mathbb{R}^{n},\mathbb{C}^{N})}\right) , \\ s,L & \in\mathbb{R}\nonumber \end{aligned}$$
    (36)
  3. A priory estimate (36) yields that every solution \(u\) of the equation \(\mathfrak{D}_{m,\Phi}u=0\) belonging to the space \(X^{s} (\mathbb{R}^{n},\mathbb{C}^{N})\) , actually, belongs to the Schwartz space \(S(\mathbb{R}^{n},\mathbb{C}^{N}).\)

  4. It follows from Proposition 2 that \(\mathfrak{D}_{m,\Phi}\) is a Fredholm operator from \(X^{s}(\mathbb{R}^{n},\mathbb{C}^{N})\) into \(X^{s-1}(\mathbb{R}^{n},\mathbb{C}^{N})\) if and only if

    $$\liminf_{x\rightarrow\infty}\inf_{\xi\in\mathbb{R}^{n}}\left\vert \det\left\langle \xi\right\rangle ^{-1}\left( \alpha\cdot\xi+m(x)\alpha _{n+1}+\Phi(x)I_{N}\right) \right\vert >0, $$
    (37)

    and \(ind\) \(\mathfrak{D}_{m,\Phi}\) is independent from \(s\in\mathbb{R} ,\)and \(p,q\in(1,\infty).\) Moreover, if \(m,\Phi\) are real-valued functions, then \(ind\) \(\mathfrak{D}_{m,\Phi}=0.\)

  5. If the operator \(\mathfrak{D}_{m,\Phi}:H^{1,p}(\mathbb{R}^{n} ,\mathbb{C}^{N})\rightarrow L^{2}(\mathbb{R}^{n},\mathbb{C}^{N})\) is invertible, then \(\mathfrak{D}_{m,\Phi}^{-1}\in OP\mathcal{S}^{-1} (\mathbb{R}^{n},\mathbb{C}^{N})\) (see [4]). Hence \(\mathfrak{D} _{m,\Phi}^{-1}\) is invertible operator from \(X^{s}(\mathbb{R}^{n} ,\mathbb{C}^{N})\) into \(X^{s-1}(\mathbb{R}^{n},\mathbb{C}^{N}).\)

  6. Let \(m,\Phi\in SO^{\infty}(\mathbb{R}^{n}).\) Taking into account Proposition 2 we obtain that

    $$\begin{aligned} \, & \mathfrak{D}_{m,\Phi}\mathfrak{D}_{m,-\Phi}=\left( -\Delta+m^{2}-\Phi ^{2}\right) \mathbb{I}_{N}+Q_{1}, \\ \mathfrak{D}_{m,-\Phi}\mathfrak{D}_{m,\Phi} & =\left( -\Delta+m^{2} -\Phi^{2}\right) \mathbb{I}_{N}+Q_{2}\nonumber \end{aligned}$$
    (38)

    where \(Q_{j}(x)\ \)are \(N\times N\) matrices for every \(x,\) such that

    $$\lim_{x\rightarrow\infty}\partial^{\alpha}Q_{j}(x)=0,\text{ }\forall \alpha,j=1,2. $$
    (39)

    Proposition 3 yields that if condition

    $$\liminf_{x\rightarrow\infty}\operatorname{Re}(m^{2}(x)-\Phi^{2}(x)>0 $$
    (40)

    is satisfied, then the operator

    $$\mathcal{C=}\left( -\Delta+m^{2}-\Phi^{2}\right) \mathbb{I}_{N} :X^{s}(\mathbb{R}^{n},\mathbb{C}^{N})\rightarrow X^{s-2}(\mathbb{R} ^{n},\mathbb{C}^{N})$$

    is a Fredholm operator. Moreover, it follows from the uniqueness of continuation for the operator \(\mathcal{C}\) that condition (40) yields that \(\ker C=\ker\mathcal{C}^{\ast}=\left\{ 0\right\} \). That is if condition (40) holds, the operator

    $$C:X^{s}(\mathbb{R}^{n},\mathbb{C}^{N})\rightarrow X^{s-2}(\mathbb{R} ^{n},\mathbb{C}^{N})$$

    is invertible operator for each \(s\in\mathbb{R}\), and \(p,q\in(1,\infty).\)

  7. Let the operator \(\mathfrak{D}_{m,\Phi}^{-1}\) be invertible, and

    $$\inf_{x\in\mathbb{R}^{n}}\operatorname{Re}\left( m^{2}(x)-\Phi^{2}(x)\right) >0. $$
    (41)

    Then formulas (38) yield that

    $$\mathfrak{D}_{m,\Phi}^{-1}=\mathfrak{D}_{m,-\Phi}\mathcal{C}^{-1} +Q\mathcal{C}^{.-1}. $$
    (42)

    Therefore,

    $$\sigma_{\mathfrak{D}_{m,\Phi}^{-1}}(x,\xi)=\frac{\alpha\cdot\xi+m(x)\alpha _{n+1}-\Phi(x)\mathbb{I}_{N}}{\left\vert \xi\right\vert ^{2}+m^{2}(x)-\Phi ^{2}(x)}+\mathcal{T(}x,\xi) $$
    (43)

    where \(\mathcal{T(}x,\xi)\in\mathcal{\mathring{S}}^{-2}(\mathbb{R} ^{n},\mathbb{C}^{N})\).

  8. The operator \(\mathfrak{D}_{m,\Phi}^{-1}\) has the integral representation

    $$\mathfrak{D}_{m,\Phi}^{-1}\varphi(x)=\int_{\mathbb{R}^{n}}g_{m,\Phi }(x,y)\varphi(y)dy,x\in\mathbb{R}^{n},\varphi\in C_{0}^{\infty}(\mathbb{R} ^{n},\mathbb{C}^{N}),$$

    with the Schwartz kernel \(g_{m,\Phi}\) (the Green function of the operator \(\mathfrak{D}_{m,\Phi}),\) is the distribution \(g_{m,\Phi}\in S^{\prime }(\mathbb{R}^{n}\times\mathbb{R}^{n})\otimes\mathcal{B(}\mathbb{C}^{n})\) which is the solution of the equation

    $$\mathfrak{D}_{m,\Phi}g_{m,\Phi}(\cdot,y)=\delta(\cdot-y),y\in\mathbb{R}^{n}.$$

    The Green function \(g_{m,\Phi}(x,y)\) has the following properties:

    (i) \(g_{m,\Phi}\in C^{\infty}(\left( \mathbb{R}^{n}\times\mathbb{R} ^{n}\right) \diagdown\mathcal{F)\otimes B}(\mathbb{C}^{N})\) where \(\mathcal{F}=\left\{ (x,y)\in\mathbb{R}^{n}\times\mathbb{R}^{n}:x=y\right\} ,\) (ii) there exists \(\varepsilon>0\) and constants \(\ C_{\alpha,\beta}>0\) such that for all multi-indices \(\alpha,\beta\)

    $$\left\Vert \partial_{x}^{\alpha}\partial_{y}^{\beta}g_{m,\Phi}(x,y)\right\Vert _{\mathcal{B}(\mathbb{C}^{N})}\leq C_{\alpha,\beta}\exp(-\varepsilon\left\vert x-y\right\vert ),\text{ for }\left\vert x-y\right\vert >0, $$
    (44)

    and

    $$\left\Downarrow g(x,y)\right\Downarrow _{\mathcal{B(}\mathbb{C}^{N})}\leq C\left\vert x-y\right\vert ^{-(n-1)},\left\vert x-y\right\vert <\varepsilon. $$
    (45)

3.3. Reducing of interaction problems to pseudodifferential equations

\(1^{0}.\) We consider interaction problem (35)

$$\mathbb{D}_{m,\Phi,\mathfrak{B}_{\Sigma}}u=\left\{ \begin{array} [c]{c} \mathfrak{D}_{m,\Phi}u=\left( \alpha\cdot D+\alpha_{n+1}m+\Phi\mathbb{I} _{N}\right) u=f\text{ on }\mathbb{R}^{n}\diagdown\Sigma\\ \mathfrak{B}_{\Sigma}u=a_{+}\gamma_{\Sigma}^{+}u+a_{-}\gamma_{\Sigma} ^{-}u=f_{1}\text{ on }\Sigma \end{array} \right. $$
(46)

where \(m,\Phi\in SO^{\infty}(\Sigma),\) and

$$a_{\pm}(\upsilon)=\frac{1}{2}\Gamma(\upsilon)\mp i\alpha\cdot\nu (\upsilon),\Gamma(\upsilon)=\left( \Gamma_{kl}(\upsilon)\right) _{k,l=1} ^{N},\Gamma_{kl}\in SO^{\infty}(\Sigma), $$
(47)

\(\nu(\upsilon)\) is the unit normal vector to \(\Sigma\) at \(\upsilon\in\Sigma\) directed to \(\Omega_{-},\Sigma\subset\mathbb{R}^{n}\) is a \(C^{\infty}-\) manifolds in \(\mathbb{R}^{n}\) of the class \(\mathcal{R(}n-1).\)

We use the notation

$$X^{s}(\mathbb{R}^{n}\diagdown\Sigma,\mathbb{C}^{N})=X^{s}(\Omega _{+},\mathbb{C}^{N})\oplus X^{s}(\Omega_{-},\mathbb{C}^{N})$$

where \(X^{s}(\mathbb{R}^{n}\diagdown\Sigma,\mathbb{C}^{N})\) is the space of the restrictions of distributions in \(X^{s}(\mathbb{R}^{n},\mathbb{C}^{N})\) on \(X^{s}(\mathbb{R}^{n}\diagdown\Sigma,\mathbb{C}^{N})\) with the standard norm of the restriction. We consider the operator \(\mathbb{D}_{m,\Phi,\mathfrak{B}_{\Sigma}}\) as acting from \(X^{s}(\mathbb{R}^{n}\diagdown\Sigma,\mathbb{C}^{N})\) into \(X^{s-1} (\mathbb{R}^{n}\diagdown\Sigma,\mathbb{C}^{N})\oplus Y^{s-1/p}(\Sigma ,\mathbb{C}^{N}),s>1/p,p,q\in(1,\infty).\)

We assume that the operator \(\mathfrak{D}_{m,\Phi}\) is invertible, and condition

$$\inf_{x\in\mathbb{R}^{n}}\operatorname{Re}\left( m^{2}(x)-\Phi^{2}(x)\right) >0, $$
(48)

holds and we will look for solutions of the equation

$$\begin{aligned} \, \mathbb{D}_{\Phi,m,\mathfrak{B}_{\Sigma}}u & =(f,f_{1}),u=(u_{+},u_{-})\in X^{s}(\mathbb{R}^{n}\diagdown\Sigma,\mathbb{C}^{N}), \\ (f,f_{1}) & \in X^{s}(\mathbb{R}^{n}\diagdown\Sigma,\mathbb{C}^{N} )\oplus\partial Y^{s-1/p}\left( \Sigma,\mathbb{C}^{N}\right) .\nonumber \end{aligned}$$
(49)

as

$$\begin{aligned} \, u(x) & =\mathfrak{D}_{m,\Phi}^{-1}f(x)+\mathcal{P}_{m,\Phi,\Sigma} \psi(x),x\in\mathbb{R}^{n}\diagdown\Sigma=\Omega_{\ast}\cup\Omega _{-}, \\ \psi & \in\partial Y^{s-1/p}(\Sigma,\mathbb{C}^{N})\nonumber \end{aligned}$$
(50)

where \(\mathcal{P}_{m,\Phi,\Sigma}\) is the potential operator

$$\mathcal{P}_{m,\Phi,\Sigma}\psi(x)=\mathfrak{D}_{m,\Phi}^{-1}\left( \psi\otimes\delta_{\Sigma}\right) =\int_{\Sigma}g_{m,\Phi}(x,\upsilon )\psi(\upsilon)d\upsilon,x\in\mathbb{R}^{n}\diagdown\Sigma,$$

\(g_{m,\Phi}(x,y)\) is the Schwartz kernel of the operator \(\mathfrak{D} _{m,\Phi}^{-1}\) and \(d\upsilon\) is the Lebesgue measure on the \(C^{\infty} -\)hypersurface \(\Sigma.\)

The following proposition gives some of the properties of potential operators required below.

Proposition 15.

Let \(m,\Phi\in SO^{\infty}(\mathbb{R}^{n}),\) and \(\Sigma \subset\mathbb{R}^{n}\) be a manifold in \(\mathbb{R}^{n}\) of the class \(\mathcal{R(}n-1).\) Then:

(i) the potential operator \(\mathcal{P}_{m,\Phi,\Sigma}\) is a bounded operator from \(Y^{s-1/p}\left( \Sigma,\mathbb{C}^{N}\right) ,s>1/p\) into \(X^{s}(\mathbb{R}^{n}\diagdown\Sigma,\mathbb{C}^{N});\)

(ii) there exist no tangential limits

$$\begin{aligned} \, & \mathcal{P}_{m,\Phi,\Sigma}^{\pm}\psi(\upsilon) \\ & =\lim_{\Omega_{\pm}\ni x\rightarrow\upsilon\in\Sigma}\mathcal{P} _{m,\Phi,\Sigma}\psi(x)=\pm\frac{i}{2}\alpha\cdot\nu(\upsilon)\psi (\upsilon)+\mathcal{K}_{m,\Phi,\Sigma}\psi(\upsilon),\upsilon\in \Sigma\nonumber \end{aligned}$$
(51)

where \(\nu(\upsilon)\) is the unit normal vector to \(\Sigma\) at the point \(\upsilon\in\Sigma\) directed to \(\Omega_{-},\) and \(\mathcal{K}_{m,\Phi,\Sigma }\in\) \(OP\mathcal{S}_{sl}^{0}(\Sigma,\mathbb{C}^{N})\) with the symbol

$$\sigma_{\mathcal{K}_{m,\Phi,\Sigma}}(\upsilon,\xi_{\upsilon})=\frac {\alpha\cdot\xi_{\upsilon}+m(\upsilon)\alpha_{n+1}+\Phi(\upsilon )\mathbb{I}_{N}}{2\sqrt{\left\vert \xi_{\upsilon}\right\vert ^{2} +m^{2}(\upsilon)-\Phi^{2}(\upsilon)}} $$
(52)

and principle symbol

$$\sigma_{\mathcal{K}_{m,\Phi,\Sigma}}^{0}(\upsilon,\xi_{\upsilon})=\frac {\alpha\cdot\xi_{\upsilon}}{2\left\vert \xi_{\upsilon}\right\vert } ,\xi_{\upsilon}\in T_{\upsilon}^{\ast}(\Sigma)\diagdown0,v\in\Sigma, $$
(53)

\(T_{\upsilon}^{\ast}(\Sigma)\) is the cotangent space to \(\Sigma\) at the point \(\upsilon\in\Sigma.\)

Proof.

Statement (i) is well known for the potential operators if \(\Omega_{\pm }=\mathbb{R}_{\pm}^{n},\) that is

$$\mathcal{P}_{m,\Phi,\Sigma}\psi(x^{\prime})\rightarrow p(x,D)(\delta (x_{n})\otimes\psi)(x),x\in\mathbb{R}_{+}^{n}\cup\mathbb{R}_{-}^{n}$$

where the symbol \(p(x,\xi)\in\mathcal{S}^{-1}(\mathbb{R}^{n},\mathbb{C}^{N})\) and satisfying the transmission property with respect to \(\mathbb{R}^{n-1}\) (see [8, 19, 44]). If the common boundary \(\Sigma\) of domains \(\Omega_{\pm}\) is a manifold of the class \(\mathcal{R} (n-1)\), we use a finite partition of unity and the transition to local coordinates.

(ii) Formula (51) follows from [46], Proposition 3.4, page 232 for more general setting. However we will prove that \(\mathcal{K} _{\Phi,m,\Sigma}\) is a psdo of the class \(OP\mathcal{S}_{sl}^{0} (\Sigma,\mathbb{C}^{N})\) with the symbol and principal symbol given by formulas (52) and (53). First, we consider the case \(\Omega_{\pm}=\mathbb{R}_{\pm}^{n}\) and \(\Sigma=\mathbb{R}^{n-1}.\) Taking into account that

$$\begin{aligned} \, \mathfrak{D}_{m,\Phi}^{-1}\psi(x) & =(2\pi)^{-n}\int_{\mathbb{R}^{2n}} \sigma_{\mathfrak{D}_{m,\Phi}^{-1}}(x,\xi)e^{i(x-y)\cdot\xi}\psi(y)dyd\xi,\\ x & =(x^{\prime},x_{n})\in\mathbb{R}^{n},\psi\in C_{0}^{\infty} (\mathbb{R}^{n},\mathbb{C}^{N}) \end{aligned}$$

and formula (43) we will write the potential operator \(\mathcal{P} _{m,\Phi,\mathbb{R}^{n-1}}\) as follows

$$\begin{aligned} \, & \left( \mathcal{P}_{m,\Phi,\mathbb{R}^{n-1}}\varphi\right) (x) \\ & =(2\pi)^{-n}\int_{\mathbb{R}^{n}\times\mathbb{R}^{n}}\sigma_{\mathfrak{D} _{m,\Phi}^{-1}}(x,\xi)e^{i(x^{\prime}-y^{\prime})\cdot\xi^{\prime}+ix_{n} \xi_{n}}\psi(y^{\prime})dy^{\prime}d\xi\nonumber \end{aligned}$$
(54)

with

$$\begin{aligned} \, \sigma_{\mathfrak{D}_{\Phi,m}^{-1}}(x,\xi) & =\mathcal{I(}x,\xi )+q(x,\xi),\text{ } \\ \mathcal{I(}x,\xi) & =\frac{\alpha\cdot\xi+m(x)\alpha_{n+1}+\Phi (x)\mathbb{I}_{N}}{\left\vert \xi\right\vert ^{2}+m^{2}(x)-\Phi^{2} (x)},\nonumber\\ q(x,\xi) & \in\mathring{S}^{-2}(\mathbb{R}^{n},\mathbb{C}^{N}).\nonumber \end{aligned}$$
(55)

Formulas (54),(55) yield that

$$ \left( \mathcal{P}_{m,\Phi,\mathbb{R}^{n-1}}\varphi\right) (x^{\prime },x_{n}) $$
(56)
$$ =\left( 2\pi\right) ^{-(n-1)} {\displaystyle\iint\limits_{\mathbb{R}^{2(n-1)}}} p(x^{\prime},x_{n},\xi^{\prime})e^{i(x^{\prime}-y^{\prime})\cdot\xi^{\prime} }\varphi(y^{\prime})dy^{\prime}d\xi^{\prime} $$
(57)

where

$$p(x,\xi^{\prime})=p_{1}(x,\xi^{\prime})+p_{2}(x,\xi^{\prime}), $$
(58)
$$p_{1}(x,\xi^{\prime})=(2\pi)^{-1}\int_{\mathbb{R}}\mathcal{I}(x,\xi^{\prime },\xi_{n})e^{ix_{n}\xi_{n}}d\xi_{n} $$
(59)

and

$$p_{2}(x,\xi^{\prime})=(2\pi)^{-1}\int_{\mathbb{R}}q(x,\xi^{\prime},\xi _{n})e^{ix_{n}\xi_{n}}d\xi_{n}. $$
(60)

Applying the residua theorem to (59) we obtain that

$$p_{1}(x^{\prime},x_{n},\xi^{\prime})=e^{-\mu(x,\xi^{\prime})\left\vert x_{n}\right\vert }\left( sgn(x_{n})\frac{i\alpha_{n}}{2}+A(x^{\prime} ,\xi^{\prime})\right) $$
(61)

with \(\mu(x,\xi^{\prime})=\sqrt{\left\vert \xi^{\prime}\right\vert ^{2} +m^{2}(x)-\Phi^{2}(x)}\) and \(\inf_{x}\operatorname{Re}\left( m^{2} (x)-\Phi^{2}(x)\right) >0,\) and

$$A(x,\xi^{\prime})=\frac{\alpha^{\prime}\cdot\xi^{\prime}+m(x)\alpha_{n+1} +\Phi(x)\mathbb{I}_{N}}{2\mu(x,\xi^{\prime})}. $$
(62)

Taking into account that \(q(x,\xi)\in\mathring{S}_{sl}^{-2}(\mathbb{R} ^{n},\mathbb{C}^{N})\) we obtain the estimates

$$\begin{aligned} \, \left\Downarrow \partial_{x}^{\beta}\partial_{\xi}^{\alpha}p_{2}(x,\xi ^{\prime})\right\Downarrow _{\mathcal{B}(\mathbb{C}^{N})} & \leq C_{\alpha\beta}(x)\int_{\mathbb{R}}(1+\left\lfloor \xi^{\prime}\right\rfloor +\left\vert \xi_{n}\right\vert )^{-2-\left\vert \alpha\right\vert }d\xi _{n} \\ & \leq C_{\alpha\beta}^{\prime}(x)\left( 1+\left\lfloor \xi^{\prime }\right\rfloor \right) ^{-1-\left\vert \alpha\right\vert }\nonumber \end{aligned}$$
(63)

with \(\lim_{x\rightarrow\infty}C_{\alpha\beta}^{\prime}(x)=0\,\ \)for all multi-indeces \(\alpha,\beta.\) Formulas (61),(62), and (63) yield that there exist limits

$$\mathcal{P}_{m,\Phi,\mathbb{R}^{n-1}}^{\pm}\varphi(x^{\prime})=\lim _{x_{n}\rightarrow\pm0}\left( \mathcal{P}_{m,\Phi,\mathbb{R}^{n-1}} \varphi\right) (x^{\prime},x_{n})$$

where

$$\begin{aligned} \, & \mathcal{P}_{m,\Phi,\mathbb{R}^{n-1}}^{\pm}\varphi(x^{\prime} ) \\ & =\pm\frac{i\alpha_{n}}{2}\varphi(x^{\prime})+\mathcal{K}_{m,\Phi ,\mathbb{R}^{n-1}}\varphi(x^{\prime}),\varphi\in C_{0}^{\infty}(\mathbb{R} ^{n-1},\mathbb{C}^{N})\nonumber \end{aligned}$$
(64)

and

$$\mathcal{\sigma}_{\mathcal{K}_{m,\Phi,\mathbb{R}^{n-1}}}(x^{\prime} ,\xi^{\prime})=\frac{\alpha^{\prime}\cdot\xi^{\prime}+m(x^{\prime} ,0)\alpha_{n+1}+\Phi(x^{\prime},0)\mathbb{I}_{N}}{2\sqrt{\left\vert \xi^{\prime}\right\vert ^{2}+m(x^{\prime},0)^{2}-\Phi(x^{\prime},0)^{2}} }+\mathcal{R}(x^{\prime},\xi^{\prime}), $$
(65)

where \(\mathcal{R}(x^{\prime},\xi^{\prime})\in\mathcal{\mathring{S}}_{sl} ^{-1}(\mathbb{R}^{n-1},\mathbb{C}^{N})\), \(\alpha^{\prime}\cdot\xi^{\prime }=\sum_{n=1}^{n-1}\alpha_{i}\xi_{i}.\) Transition to the local coordinates at the point \(\nu\) \(\in\Sigma\) with the frame \(\{f_{1,}...,f_{n}\}\) where \(f_{n}\) is the unit normal vector \(\nu(\upsilon)\) to \(\Sigma\) at the point \(\upsilon\), directed to \(\Omega_{-},\) and applying formulas (64), (65) we obtain formulas (51), (52).

\(2_{.}^{0}.\) Now we are ready to reduce the interaction problem (49) to the pseudodifferential equation on \(\Sigma\). Substituting

$$u^{\pm}(s)=\mathfrak{D}_{m,\Phi}^{-1}f_{1}+\mathcal{P}_{m,\Phi,\Sigma}^{\pm }\psi,\text{ }\psi\in X^{s-1/p}(\mathbb{\Sigma},\mathbb{C}^{N}),s>1/p$$

in the interaction condition

$$\mathfrak{B}_{\Sigma}u(\upsilon)=a_{+}(\upsilon)\gamma_{\Sigma}^{+} u(\upsilon)+a_{-}(\upsilon)\gamma_{\Sigma}^{-}u(\upsilon)=f_{2}(\upsilon ),\upsilon\in\Sigma,$$

and applying formulas (51) and (52) we arrive to the pseudodifferential equation on \(\Sigma\) with respect to \(\psi\in Y^{s-1/p}(\Sigma,\mathbb{C}^{N})\)

$$\begin{aligned} \, \Xi_{m,\Phi,\Sigma}\psi(\upsilon)&=\frac{i\alpha\cdot\nu (\upsilon)}{2}(a_{+}(\upsilon)-a_{-}(\upsilon))\psi(\upsilon)+(a_{+} (\upsilon)+a_{-}(\upsilon))\left( \mathcal{K}_{m,\Phi,\Sigma}\psi\right) (\upsilon) \\ & =f_{2}(\upsilon)-\mathfrak{B}_{\Sigma}\mathfrak{D}_{\Phi,m} ^{-1}f_{1}(\upsilon)\in Y^{s-1/p}(\Sigma,\mathbb{C}^{N}).\nonumber \end{aligned}$$
(66)

Taking into account that

$$a_{+}(\upsilon)-a_{-}(\upsilon)=-2i\alpha\cdot\nu\mathfrak{(}\upsilon \mathfrak{)}\text{ and }a_{+}(\upsilon)+a_{-}(\upsilon)=\Gamma(\upsilon ),\upsilon\in\Sigma$$

we obtain that \(\Xi_{m,\Phi,\mathfrak{\Sigma}}=I+\Gamma\mathcal{K} _{m,\Phi,\Sigma}\) is the psdo on \(\mathbb{\Sigma}\) of the class \(OP\mathcal{S}_{sl}^{0}(\Sigma,\mathbb{C}^{N\text{ }}).\)with the symbol

$$\begin{aligned} \, \sigma_{\Xi_{m,\Phi,\mathfrak{\Sigma}}}(\upsilon,\xi_{\upsilon}) & =\mathbb{I}_{N}+\Gamma(\upsilon)\frac{\alpha\cdot\xi_{\upsilon}+m(\upsilon )\alpha_{n+1}+\Phi(\upsilon)\mathbb{I}_{N}}{2\sqrt{\left\vert \xi\right\vert ^{2}+m(\upsilon)^{2}-\Phi(\upsilon)^{2}}}, \\ v & \in\Sigma,\xi_{\upsilon}\in T_{\upsilon}^{\ast}(\Sigma).\nonumber \end{aligned}$$
(67)

The principal symbol of \(\Xi_{m,\Phi,\mathfrak{\Sigma}}\) is

$$\sigma_{\Xi_{m,\Phi}}^{0}(\upsilon,\xi)=\mathbb{I}_{N} +\frac{\Gamma(\upsilon)\left( \alpha\cdot\xi_{\upsilon}\right) }{2\left\vert \xi_{v}\right\vert },\upsilon\in\Sigma,\xi_{\upsilon}\in T_{\upsilon}^{\ast }(\Sigma)\diagdown0. $$
(68)

3.4. Interaction on closed \(C^{\infty}-\)hypersurfaces

We consider the interaction problem for a closed \(C^{\infty}-\)hypersurface \(\Sigma\) which is the common boundary of the bounded open domain \(\Omega_{+} \) and unbounded domain \(\Omega_{-}=\mathbb{R}^{n}\diagdown\bar{\Omega}_{+} \)

Theorem 16.

Let: (i) \(m,\Phi\in SO^{\infty}(\mathbb{R}^{n}\mathbb{)},\) (ii) \(\Sigma\) be a \(C^{\infty}-\)closed hypersurface in \(\mathbb{R}^{n},\) and \(\Gamma\in C^{\infty}(\Sigma)\otimes\mathcal{B}(\mathbb{C}^{N}),\) (iii) the Dirac operator \(\mathfrak{D}_{m,\Phi}:H^{1,p}(\mathbb{R}^{n},\mathbb{C} ^{N})\rightarrow L^{2}(\mathbb{R}^{n},\mathbb{C}^{N})\) is invertible, (iv)

$$\det\sigma_{\Xi_{m,\Phi}}^{0}(\upsilon,\xi_{\upsilon} )=\det\left( \mathbb{I}_{N}+\frac{\Gamma(\upsilon)\left( \alpha\cdot \xi_{\upsilon}\right) }{2\left\vert \xi_{\upsilon}\right\vert }\right) \neq0 $$
(69)

for each point \(\upsilon\in\Sigma\) and \(\xi_{\upsilon}\in T_{\upsilon}^{\ast }(\Sigma)\diagdown0.\) Then:

(a) \(\Xi_{m,\Phi,\mathfrak{\Sigma}}\) is a Fredholm operator in the spaces \(Y^{s}(\Sigma,\mathbb{C}^{N}),\) \(s\in\mathbb{R},,p,q\) \(\in(1,\infty)\); (b) \(\ker\Xi_{m,\Phi,\mathfrak{\Sigma}}\) and \(\ker\Xi_{m,\Phi,\mathfrak{\Sigma} }^{\ast}\in C^{\infty}(\Sigma,\mathbb{C}^{N}),\) hence \(ind\) \(\Xi_{m,\Phi ,\mathfrak{\Sigma}}\) is independent of \(s,p,q;\) (c) if the matrix \(\Gamma(\upsilon)\) is Hermitian for each \(\upsilon\in\Sigma,\) then \(ind\) \(\Xi_{m,\Phi,\mathfrak{\Sigma}}=0.\)

Proof.

The Fredholmness of \(\Xi_{m,\Phi,\mathfrak{\Sigma}}\) in all spaces \(Y^{s}(\Sigma,\mathbb{C}^{N})\) follows from Theorem 13, statements (b), follows from a priori estimate (49), and statement (c) follows from the equality \(\sigma_{\Xi_{m,\Phi,\mathfrak{\Sigma}}}^{0}=\sigma _{\Xi_{m,\Phi,\mathfrak{\Sigma}}^{\ast}}^{0}\) which follows from the Hermite property of the matrix \(\Gamma\).

Corollary 17.

Let conditions of Theorem 69 hold. Then

$$\begin{aligned} \, \mathbb{D}_{m,\Phi,\mathfrak{B}_{\Sigma}} & :X^{s}(\mathbb{R}^{n} \diagdown\Sigma,\mathbb{C}^{N})\rightarrow X^{s-1}(\mathbb{R}^{n} \diagdown\Sigma,\mathbb{C}^{N})\oplus Y^{s-1/p}(\Sigma,\mathbb{C}^{N}),\\ s & >1/p,p,q\in(1,\infty) \end{aligned}$$

is a Fredholm operator.

Proof.

Conditions \((i),(ii),(iii)\) yield the local invertibility of the operator \(\mathbb{D}_{m,\Phi,\mathfrak{B}_{\Sigma}}\) at the points of \(\mathbb{R} ^{n}\diagdown\Sigma,\) and (69) is the Lopatinsky–Shapiro condition at the points \(s\in\Sigma\) for the operator of interaction problem \(\mathbb{D} _{m,\Phi,\mathfrak{B}_{\Sigma}}\) (see [41, 39]). Hence the local principle yields the Fredholmness of the operator \(\mathbb{D} _{m,\Phi,\mathfrak{B}_{\Sigma}}.\)

3.5. Interaction on hypersurfaces of the class \(\mathcal{R}(n-1)\)

Theorem 18.

Assume that \(m,\Phi\in SO^{\infty}(\Sigma\mathbb{)}\), the hypersurface \(\Sigma\in\mathcal{R}(n-1),\) \(\Gamma\in SO^{\infty} (\Sigma)\otimes\mathcal{B}(\mathbb{C}^{N}),\) and the Dirac operator \(\mathfrak{D}_{m,\Phi}:H^{1,2}(\mathbb{R}^{n},\mathbb{C}^{N})\rightarrow L^{2}(\mathbb{R}^{n},\mathbb{C}^{N})\) is invertible. Then \(\Xi_{m,\Phi ,\mathfrak{\Sigma}}\) is a Fredholm operator in the space \(Y^{s}(\Sigma ,\mathbb{C}^{N})\) if : (i) condition (69) holds for each point \(\upsilon\in\Sigma,\) (ii)

$$\liminf_{\Sigma\ni\upsilon\rightarrow\infty}\inf_{\xi_{v}\in T_{\upsilon }^{\ast}(\Sigma)}\left\vert \det\sigma_{\Xi_{m,\Phi,\mathfrak{\Sigma}} }(\upsilon,\xi_{\upsilon})\right\vert >0 $$
(70)

where

$$\sigma_{\Xi_{m,\Phi,\Sigma}}(\upsilon,\xi_{\upsilon})=I_{N} +\frac{\Gamma(\upsilon)\left( \alpha\cdot\xi_{\upsilon}+m(\upsilon )\alpha_{n+1}+\Phi(\upsilon)\mathbb{I}_{N}\right) }{2\left( \left\vert \xi_{\upsilon}\right\vert ^{2}+m^{2}(\upsilon)-\Phi^{2}(\upsilon)\right) ^{1/2}},\xi_{\upsilon}\in T_{\upsilon}^{\ast}(\Sigma),\upsilon\in\Sigma. $$
(71)

The index of \(\Xi_{m,\Phi,\mathfrak{\Sigma}}\) is independent of \(s\in \mathbb{R},p,q\in(1,\infty).\) If \(m,\Phi\) are real-valued functions, and \(\Gamma\) is a Hermitian matrix, then \(Ind\) \(\Xi_{m,\Phi,\mathfrak{\Sigma}}=0.\)

Proof.

Let \(U\) be a neighborhood of the infinitely distant point \(\upsilon\in \tilde{U}\) where \(\hat{U}\) is the compactification of \(U\) in the topology \(\hat{\Sigma}.\) According formula (67) the symbol of the operator \(\Xi_{m,\Phi,\mathfrak{\Sigma}}\) in the neighborhood \(U\) is given by formula (71). Thus Theorem 18.. follows from Theorem 13. Statements regarding the index are proven using the considerations given earlier.

Theorem 19.

Let conditions of Theorem 18 be satisfied. Then the operator

$$\mathbb{D}_{m,\Phi,\mathfrak{B}_{\Sigma}}:X^{s}(\mathbb{R}^{n}\diagdown \Sigma,\mathbb{C}^{N})\rightarrow X^{s-1}(\mathbb{R}^{n}\diagdown \Sigma,\mathbb{C}^{N})\oplus Y^{s-1/p}(\Sigma,\mathbb{C}^{N})$$

is Fredholm for each \(s>1/p,p,q\in(1,\infty).\)

Proof.

Let \(\mathbb{R}^{n}\) be equipped the structure of the manifold of the class \(\mathcal{R(}n).\) Let \(\mathbb{\hat{R}}^{n}\) be a compactification of \(\mathbb{R}^{n}\) as a manifold of the class \(\mathcal{R(}n)\) and \(\hat{\Sigma }\) is the closure of \(\Sigma\) in the topology of \(\mathbb{\hat{R}}^{n}.\)The local principle on \(\mathbb{\hat{R}}^{n}\) states that \(\mathbb{D} _{m,\Phi,\mathfrak{B}_{\Sigma}}:X^{s}(\mathbb{R}^{n}\diagdown\Sigma ,\mathbb{C}^{N})\rightarrow X^{s-1}(\mathbb{R}^{n}\diagdown\Sigma ,\mathbb{C}^{N})\oplus Y^{s-1/p}(\Sigma,\mathbb{C}^{N})\) is a Fredholm operator if and only if \(\mathbb{D}_{m,\Phi,\mathfrak{B}_{\Sigma}}\) is a locally Fredholm operator at every point \(x\in\mathbb{R}^{n}\) and locally invertible at every infinitely distant point \(x_{\infty}\in\mathbb{\hat{R} }_{\infty}^{n}.\) Since the operator \(\mathbb{D}_{m,\Phi,\mathfrak{B}_{\Sigma} }\) concedes with the operator \(\mathfrak{D}_{m,\Phi}\) on the set \(\mathbb{\hat{R}}^{n}\diagdown\hat{\Sigma}\) the operator \(\mathbb{D} _{m,\Phi,\mathfrak{B}_{\Sigma}}\) is locally Fredholm at every point \(x\in\mathbb{R}^{n}\diagdown\Sigma\) and locally invertible at every point \(x_{\infty}\in\mathbb{\hat{R}}_{\infty}^{n}\diagdown\hat{\Sigma}_{\infty}.\) Therefore, according to the local principle we need to study the local Fredholmness of \(\mathbb{D}_{m,\Phi,\mathfrak{B}_{\Sigma}}\) at the points \(\upsilon\in\hat{\Sigma}.\) The local Fredholmness at the points \(\upsilon \in\Sigma\) follows from the Lopatinsky–Shapiro condition at the point \(\upsilon\) (see [41]) which concedes with condition (69), and local invertibility at the points \(\upsilon_{\infty}\in\hat{\Sigma}_{\infty}\) It folloes from condition

$$\liminf_{\Sigma\ni\upsilon\rightarrow\upsilon_{\infty}}\inf_{\xi_{\upsilon}\in T_{v}^{\ast}(\Sigma)}\left\vert \det\sigma_{\Xi_{m,\Phi,\mathfrak{\Sigma}} }(\upsilon,\xi_{\upsilon})\right\vert >0,\forall\upsilon_{\infty}\in \hat{\Sigma}_{\infty}. $$
(72)

But condition (72) is equivalent to condition (70).

  1. Important examples of the hypersurfaces in \(\mathbb{R}^{n}\) to which Theorems 18 and 19 apply are:

    1) \(C^{\infty}\)-hypersurface \(\Sigma\subset\mathbb{R}^{n}\) conical at infinity, that is such that the hypersurface \(\Sigma_{R}=\Sigma\cap B_{R\text{ }}^{\prime},B_{R}^{\prime}=\left\{ x\in\mathbb{R}^{n}:\left\vert x\right\vert >R\right\} \) is a conical set for some \(R>0;\)

2) \(C^{\infty}\)-hypersurface \(\Sigma\subset\mathbb{R}^{n}\) slowly oscillating at infinity, that is there exists of a finite covering of \(\Sigma_{R}\subset\cup_{j=1}^{l}\mathcal{F}_{j\text{ }}\) by open sets \(\mathcal{F}_{j\text{ }}\) such that

$$\Sigma_{R}\cap\mathcal{F}_{j\text{ }}=\left\{ x=\left( x^{\prime} ,x_{n}\right) \in\mathbb{R}^{n}:x_{n}=f_{j}(x^{\prime}),x^{\prime}\in K_{j}\right\}$$

where \(K_{j}\) are open conical sets in \(\mathbb{R}^{n-1},\) \(f_{j}\) are \(C^{\infty}-\)real-valued functions such that \(\partial_{x_{k}}f_{j}\in SO^{\infty}(K_{j}),k=1,...,n-1.\) Examples of functions satisfying these conditions are:

$$f(x^{\prime})=A\left\vert x^{\prime}\right\vert ^{\alpha}\cos\log^{\beta }\left\vert x^{\prime}\right\vert ,0\leq\alpha\leq1,\beta\geq0,A\in \mathbb{R}.$$