Regularization of ill-posed operator equations: an overview

Abstract

We discuss and illustrate the stability issues associated with ill-posed linear operator equations and the need to have regularization methods for obtaining stable approximate solutions. As illustration, we describe two simple regularization methods, namely, the Lavrentiev regularization in the setting of a Banach spaces and Tikhonov regularization in the setting of Hilbert spaces, and discuss on the corresponding error estimates. We also indicate procedures to obtain error estimates under milder general source conditions and also to obtain better error estimates under modified methods. The discussed procedures include some of the work of the author as well.

1 Solvability and stability

Consider the problem of solving the operator equation

$$\begin{aligned} Tx=y, \end{aligned}$$

(1.1)

where $T: X\rightarrow Y$ is a linear operator between normed linear spaces X and Y (over the same field ${\mathbb {K}}\in \{{\mathbb {R}}, {\mathbb {C}}\}$) and $y\in Y$. Equation (1.1) is said to be an ill-posed equation if for some $y\in Y$, either

• (1) it does not have a solution or

• (2) it has more than one solution or

• (3) there is a solution which does not depend continuously on the data y.

In applications, one has to deal with approximate data, say $\tilde{y}$ in place of y which may not be in R(T). In fact, if $R(T)\not =Y$ and $y\in R(T)$, then $\forall \, \delta >0$,   $\exists \, y^\delta \in Y\setminus R(T)$ such that $\Vert y-y^\delta \Vert \le \delta$.

There are many operators of practical importance for which R(T) is not even closed in Y. For instance, if T is a compact operator of infinite rank, then R(T) is not closed (cf.  [11]). In this connection, let us observe the following result.

Theorem 1

Let T be a bounded linear operator between Banach spaces X and Y such that R(T) is not closed in Y. Then for every $x\in X$ and for every sequence $(\varepsilon _n)$ of positive real numbers, there exists a sequence $(x_n)$ in X such that

$$\begin{aligned} \Vert Tx_n-Tx\Vert < \varepsilon _n\quad \text{ but }\quad \Vert x_n-x \Vert =\frac{1}{\varepsilon _n} \quad \forall \, n\in {\mathbb {N}}. \end{aligned}$$

Before giving the proof of the above theorem, let us recall that a linear operator $T: X\rightarrow Y$ between normed linear spaces X and Y is said to be bounded below if there exists $c>0$ such that $\Vert Tu\Vert \ge c\Vert u\Vert$ for all $u\in X$. We shall make use of part of the following lemma  (cf. [11]).

Lemma 2

Let X and Y be normed linear spaces and $T: X\rightarrow Y$ be bounded below. Then the following are true.

• (1) T is one-one and its inverse is continuous from its range.

• (2) If X is a Banach space and T is continuous, then R(T) is closed.

Proof of Theorem 1

Suppose R(T) is not closed and $x\in X$. Let $(\varepsilon _n)$ be a sequence of positive real numbers. Since X is a Banach space and T is not bounded below, there exists a sequence $(u_n)$ in X such that

$$\begin{aligned} \Vert Tu_n\Vert < \varepsilon _n^2\Vert u_n\Vert \quad \forall \, n\in {\mathbb {N}}. \end{aligned}$$

Let $v_n = \frac{u_n}{\varepsilon _n\Vert u_n\Vert }$. Then we have

$$\begin{aligned} \Vert Tv_n\Vert = \Vert T \left( \frac{u_n}{\varepsilon _n\Vert u_n\Vert } \right) \Vert < \varepsilon _n\quad \text{ and }\quad \Vert v_n\Vert = \frac{1}{\varepsilon _n}. \end{aligned}$$

Taking $x_n:= x + v_n$, we have $Tx_n = Tx+Tv_n$ so that

$$\begin{aligned} \Vert Tx_n-Tx\Vert = \Vert Tv_n\Vert < \varepsilon _n\quad \text{ and }\quad \Vert x_n-x \Vert = \Vert v_n\Vert = {1}/{\varepsilon _n} \end{aligned}$$

for all $n\in {\mathbb {N}}.$ $\square$

Remark 3

Note that, by Theorem 1, for each $n\in {\mathbb {N}}$, $\exists x_n\in X$ such that

$$\begin{aligned} \Vert Tx_n-Tx\Vert < e^{-c n^p}\quad \text{ and }\quad \Vert x_n-x\Vert = e^{c n^p}.\quad \qquad \qquad \lozenge \end{aligned}$$

Let us illustrate Theorem 1 in the context of general Hilbert spaces.

Example 4

Let X and Y be infinite dimensional Hilbert spaces, $(\varphi _n)$ and $(\psi _n)$ be orthonormal sequences in X and Y, respectively. Let $(\lambda _n)$ be a sequence of positive scalars such that $\lambda _n\rightarrow 0$. Let

$$\begin{aligned} Tx := \sum _{n=1}^\infty \lambda _n\left\langle x, \varphi _n \right\rangle \psi _n,\quad x\in X. \end{aligned}$$

Let $x\in X$ and $x_k:=x+ \frac{1}{\sqrt{\lambda _k}}\varphi _k$ for $k\in {\mathbb {N}}$. Then we have $Tx_k = Tx + {\sqrt{\lambda _k}}\psi _k .$ Thus,

$$\begin{aligned} \Vert Tx_k - Tx\Vert = {\sqrt{\lambda _k}} \rightarrow 0\quad \text{ but }\quad \Vert x_k - x\Vert = \frac{1}{\sqrt{\lambda _k}} \rightarrow \infty . \quad \quad \quad \lozenge \end{aligned}$$

Remark 5

In the above example, let

$$\begin{aligned} T_kx := \sum _{n=1}^k \lambda _n\left\langle x, \varphi _n \right\rangle \psi _n,\quad x\in X \end{aligned}$$

for $k\in {\mathbb {N}}$. Note that $T_k$ is a finite rank bounded linear operator. We may assume, without loss of generality, that $\lambda _1 \ge \lambda _2\ge \cdots$. Then we have

$$\begin{aligned} \Vert Tx - T_kx\Vert ^2= & {} \sum _{n=k+1}^\infty \lambda _n^2\,| \left\langle x, \varphi _n \right\rangle |^2\\\le & {} \lambda _{k+1}^2\sum _{n=k+1}^\infty | \left\langle x, \varphi _n \right\rangle |^2 \le \lambda _{k+1}^2 \Vert x\Vert ^2 \end{aligned}$$

so that

$$\begin{aligned} \Vert T-T_k\Vert \le \lambda _{k+1}\rightarrow 0\quad \text{ as }\quad k\rightarrow \infty . \end{aligned}$$

Hence T is a compact operator (cf.  [11]).

By spectral theorem, we know that every compact operator of infinite rank can be represented as in Example 4, where $\lambda _k$ are the singular values of T (cf.  [11]). Thus, the level of ill-posedness of the corresponding operator equation is closely related to the rate of decay of the singular values. $\lozenge$

A typical example of a compact operator is the Fredholm integral operator as in the following example (For the proof of the affirmative statements, one may refer, e.g.,  [11]).

Example 6

Let $\Omega := [a, b]$ and $k\in L^2(\Omega \times \Omega )$. For $x\in L^2(\Omega )$, let

$$\begin{aligned} (Tx)(s) := \int _\Omega k(s, t) x(t) dt. \end{aligned}$$

Then $Tx\in L^2(\Omega )$ and the operator $T: L^2(\Omega )\rightarrow L^2(\Omega )$ is a compact operator. The following facts are known:

• (1) The range of T is closed if and only if $k(\cdot , \cdot )$ is a degenerate kernel.

• (2) If $k(\cdot , \cdot )$ is degenerate, then T is not one-one.

• (3) If $k(\cdot , \cdot )$ is non-degenerate, then T is of infinite rank hence it is with non-closed range.$\lozenge$

In view of the above example, the problem of solving a Fredholm integral equations of the first kind, namely, the equation

$$\begin{aligned} \int _\Omega k(s, t) x(t) dt = y(s),\quad x\in L^2[a, b], \end{aligned}$$

with non-degenerate kernel $k(\cdot , \cdot )$ is ill-posed. Many problems in applications, such as

• (1) computerized tomography,

• (2) remote sensing,

• (3) geological prospecting

appear as a problem of solving a Fredholm integral equation of the first kind, which is ill-posed (cf. [1,2,3]).

Compact operators also appear in the inverse problems related to parameter identification problems in PDE through the compact imbeddings of Sobolev spaces (cf.  [8]). For instance, if $\Omega$ is a bounded open set in ${\mathbb {R}}^N$ for some $N\in {\mathbb {N}}$, then the inclusion operator from $H^1_0(\Omega )$ into $L^2(\Omega )$ is compact.

Ill-posed linear operator equations also appear while dealing with the problem of solving a non-linear operator equation,

$$\begin{aligned} F(x)=y, \end{aligned}$$

(1.2)

where $F: D\subseteq X\rightarrow Y$ is a non-linear operator between Hilbert spaces. For instance, a linearization of the above equation takes the form

$$\begin{aligned} F'(x_0)(x-x_0) = y- F(x_0). \end{aligned}$$

(1.3)

If F is a completely continuous, that is, if image of every bounded set under F is relatively compact, then $F'(x_0)$ can be shown to be a compact operator (cf.  [6]), and hence the linearized equation (1.2) of the non-linear equation (1.2) is also an ill-posed equation. Many inverse problems in PDE, such as parameter identification problems, are nonlinear and ill-posed (cf.  [1]), and they can be formulated as a non-linear equation (1.2). For instance, consider the boundary value problem

$$\begin{aligned} -\nabla \cdot (q\,\nabla u) &= f \text{ on } \Omega ,\\ u &= g \text{ on } \partial \Omega . \end{aligned}$$

There are problems in applications where one would like to find the function u on $\Omega$ knowing the functions q, f and g. Such problems are the forward problems which are generally well-posed. There are also many problems of practical interest where one would like to find the functions f or q from the knowledge of u on $\Omega$ or g. Such problems are inverse problems which are generally ill-posed. The inverse problems in which one would like to find f from the knowledge of u are called source identification problems, and those problems in which one would like to find q from the knowledge of u are called parameter identification problems. The source identification problem and parameter identification problem can be represented abstractly as

$$\begin{aligned} F(f)=u\quad \text{ and }\quad F(q) = u, \end{aligned}$$

respectively. It can be shown that the source identification problem is a linear problem, whereas the parameter identification problem is non-linear; both are ill-posed, in general  [1, 7].

In this paper, we shall be concerned about linear ill-posed problems only.

2 Regularization

Assume for a moment that $y\in R(T)$, so that Eq. (1.1) has a solution, say x. Suppose $y^\delta \in Y$ is such that

$$\begin{aligned} \Vert y-y^\delta \Vert \le \delta \end{aligned}$$

for some error level $\delta >0$. As Eq. (1.1) is, in general, ill-posed, we would like to obtain approximations for x using solutions of a well-posed problems. Such a procedure is called a regularization method. More precisely, we have the following definition.

Definition 7

A regularization method for (1.1) involves a family $\{R_\alpha \}_{\alpha >0}$ of bounded linear operators from Y to X with $\alpha >0$ such that

$$\begin{aligned} R_\alpha Tx \rightarrow x\quad \text{ as }\quad \alpha \rightarrow 0 \end{aligned}$$

(2.1)

for every $x\in X$, and for each $x\in X$ and $\delta >0$, if $y^\delta \in Y$ is such that $y^\delta \rightarrow Tx$ as $\delta \rightarrow 0$, then there is a parameter choice strategy, say $\alpha :=\alpha _\delta$, depending on $y^\delta$ and $\delta$, such that

$$\begin{aligned} \alpha _\delta \rightarrow 0\quad \text{ and }\quad R_{\alpha _\delta }y^\delta \rightarrow x\quad \text{ as }\quad \delta \rightarrow 0. \end{aligned}$$

The family $\{R_\alpha : \alpha >0\}$ satisfying (2.1) is called a regularization family or simply, a regularization of the operator equation (1.1).$\lozenge$

We describe two simple and well-studied regularization methods, namely, Lavrentiev regularization and Tikhonov regularization—one in the context of Banach spaces and the other in the context of Hilbert spaces.

2.1 Lavrentiev regularization

Let X be a Banach space and $T: X\rightarrow X$ be a bounded linear operator with non-closed range R(T). Throughout this section, we assume the following (cf.  [14]).

Assumption 8

For each $\alpha >0$, $T+\alpha I$ is bijective and there exists $M>0$ such that

$$\begin{aligned} \Vert (T+\alpha I)^{-1}\Vert \le \frac{M}{\alpha } \end{aligned}$$

(2.2)

for all $\alpha >0.$ $\lozenge$

Operators satisfying the above assumption are called weakly sectorial operators (cf. [20]). The class of weakly sectorial operators include:

• (1) Positive self adjoint operators on a Hilbert space.

• (2) Many of the Abel integral operators (cf. [20]), including

  $$\begin{aligned} (Tu)(s):=\int _0^s t^{\beta -1}u(t)dt,\quad t\in [0, a], \end{aligned}$$

  where $X=C[0, a]$ with sup-norm.

• (3) The multiplication operator

  $$\begin{aligned} (T_\varphi u)(s):= \varphi (s)u(s),\quad s\in [-1, 1], \end{aligned}$$

  where $\displaystyle \varphi (s):= \left\{ \begin{array}{ll} 0, &{} -1\le s\le 0,\\ s, &{} 0 <s\le 1,\end{array}\right.$ and $X=C[-1, 1]$ with sup-norm.

Suppose $y\in R(T)$ and $x\in X$ is such that

$$\begin{aligned} Tx=y. \end{aligned}$$

By Assumption 8 on T, for each $\alpha >0$,

$$\begin{aligned} (T+\alpha I) x_\alpha = y \end{aligned}$$

(2.3)

is a well-posed operator equation. Note that

$$\begin{aligned} (T+\alpha I) (x-x_\alpha ) = (y+\alpha x) - y = \alpha x. \end{aligned}$$

Hence,

$$\begin{aligned} x-x_\alpha = S_\alpha x, \end{aligned}$$

(2.4)

where

$$\begin{aligned} S_\alpha := \alpha (T+\alpha I)^{-1}. \end{aligned}$$

(2.5)

By Assumption 8, $\Vert S_\alpha \Vert \le M$. In other words, $\{S_\alpha : \alpha >0\}$ is a uniformly bounded family.

Clearly, if $x\in X$ such that $Tx=y$, then

$$\begin{aligned} \Vert x_\alpha - x\Vert \rightarrow 0\quad \text{ as }\quad \alpha \rightarrow 0 \iff \Vert S_\alpha x\Vert \rightarrow 0 \quad \text{ as }\quad \alpha \rightarrow 0. \end{aligned}$$

In view of the above, we specify a condition under which $S_\alpha x \rightarrow 0$ as $\alpha \rightarrow 0$. First let us prove the following general result on $S_\alpha ,\, \alpha >0$.

Theorem 9

The following are true.

• (1)   $\Vert S_\alpha T\Vert \le (1+M)\alpha$ for every $\alpha >0.$

• (2) If R(T) is dense in X, then

  • (a) $S_\alpha u \rightarrow 0$ as $\alpha \rightarrow 0$ for every $u\in X$, and

  • (b) T is injective.

Proof

(1) Observe that, for $\alpha >0$,

$$\begin{aligned} S_\alpha T &= \alpha (T+\alpha I)^{-1} T = \alpha (T+\alpha I)^{-1} [(T+\alpha I) - \alpha I] \\&= \alpha - \alpha ^2(T+\alpha I)^{-1} = \alpha (I-S_\alpha ). \end{aligned}$$

Thus, we obtain $\Vert S_\alpha T\Vert \le (1+M)\alpha .$

(2)  By (1), $S_\alpha u\rightarrow 0$ for every $u\in R(T)$. Now, suppose that R(T) is dense in X. Since $(S_\alpha )$ is uniformly bounded and since it converges pointwise to 0 on a dense subspace of a Banach space, by a result in functional analysis (cf.  [11]), $\Vert S_\alpha u\Vert \rightarrow 0$ for every $u\in X$.

Now, to see that T is injective, let $x\in X$ be such that $Tx=0$. Then we have

$$\begin{aligned} (T+\alpha I)x = \alpha x \end{aligned}$$

so that $x= \alpha (T+\alpha I)^{-1} x= S_\alpha x\rightarrow 0.$ $\square$

As a corollary to the above theorem we deduce the following.

Theorem 10

Let $y\in R(T)$ and let $x\in X$ be such that $Tx=y$. Then we have the following.

• (1) If $x\in R(T)$, then x is the unique element in R(T) such that $Tx=y$, and

  $$\begin{aligned} \Vert x-x_\alpha \Vert = O(\alpha ) \end{aligned}$$

• (2) If R(T) is dense in X, then x is the unique element in X such that $Tx=y$, and

  $$\begin{aligned} \Vert x-x_\alpha \Vert \rightarrow 0\quad \text{ as }\quad \alpha \rightarrow 0. \end{aligned}$$

Proof

Let $y\in R(T)$ and let $x\in X$ be such that $Tx=y$.

(1)  Suppose $x\in R(T)$. Let $u\in X$ be such that $x=Tu$. Then, from (2.4) and from Theorem 9 (1), we have

$$\begin{aligned} \Vert x-x_\alpha \Vert = \Vert S_\alpha Tu\Vert \le \alpha (1+M)\Vert u\Vert . \end{aligned}$$

Hence, $\Vert x-x_\alpha \Vert = O(\alpha )$.

(2)  Suppose R(T) is dense in X. Then, by Theorem 9 (2), we have $\Vert x-x_\alpha \Vert = \Vert S_\alpha x\Vert \rightarrow 0$ as $\alpha \rightarrow 0$. Again, by Theorem 9 (2), T is injective. Hence, x is the unique element in X such that $Tx=y$. $\square$

Remark 11

In Theorems 9 and 10, the assumption that R(T) is dense in X can be weakened as follows.

• (1) Theorem 9(2) can be replaced by:

  $S_\alpha u \rightarrow 0$ as $\alpha \rightarrow 0$ for every $u\in \overline{R(T)}$.

• (2) Theorem 10(2) can be replaced by:

  If $x\in \overline{R(T)}$, then x is the unique element in X such that $Tx=y$, and $\Vert x-x_\alpha \Vert \rightarrow 0$ as $\alpha \rightarrow 0.$ $\lozenge$

From Theorem 9, we also deduce the following.

Theorem 12

The family $\{R_\alpha : \alpha >0\}$ of bounded operators $R_\alpha : X\rightarrow X$ defined by

$$\begin{aligned} R_\alpha y := (T+\alpha I)^{-1}y,\quad y\in X \end{aligned}$$

for $\alpha >0$ is a regularization family.

Proof

We observe that for $x\in X$,

$$\begin{aligned} x-R_\alpha Tx = x-(T+\alpha I)^{-1}Tx = (T+\alpha I)^{-1}[(T+\alpha I) - T]x = S_\alpha Tx. \end{aligned}$$

By Theorem 9 (1), $\Vert S_\alpha T\Vert \le (1+M)\alpha$. Hence,

$$\begin{aligned} \Vert x-R_\alpha Tx\Vert = \Vert S_\alpha Tx\Vert \rightarrow 0\quad \text{ as }\quad \alpha \rightarrow 0, \end{aligned}$$

showing that $\{R_\alpha : \alpha >0\}$ is a regularization family. $\square$

Definition 13

Under the Assumption 8 on T, the family of operators $R_\alpha ,\, \alpha >0,$ defined in Theorem 12, that is,

$$\begin{aligned} R_\alpha := (T+\alpha I)^{-1},\quad \alpha >0, \end{aligned}$$

is called the Lavrentiev regularization of (1.1).$\lozenge$

If X is a Hilbert space and T is a positive self-adjoint operator, then the Assumption 8 on T is automatically satisfied. Also, we know that R(T) is dense in $N(T)^\perp$.

Thus, in this case, Theorem 10 leads to the following result.

Theorem 14

Let X be a Hilbert space and T be a positive, self-adjoint, bounded linear operator on X . Suppose $y\in R(T)$ . Then there exists a unique $x\in N(T)^\perp$ such that $Tx=y$ , and

$$\begin{aligned} \Vert x-x_\alpha \Vert \rightarrow 0\quad \text{ as }\quad \alpha \rightarrow 0. \end{aligned}$$

Further, if $x\in R(T)$, then

$$\begin{aligned} \Vert x-x_\alpha \Vert =O(\alpha ). \end{aligned}$$

In the above theorem, by T being a positive and self adjoint, we mean that $\left\langle Tx, x \right\rangle \ge 0$ for all $x\in X$ and $T^*=T$, where $T^*$ is the adjoint of T.

2.2 Error estimates under noisy data

Suppose that the data y is noisy. Thus, for $\delta >0$ we may have $y^\delta \in X$ in place of y such that such that

$$\begin{aligned} \Vert y-y^\delta \Vert \le \delta . \end{aligned}$$

Let $x_\alpha ^\delta \in X$ be such that

$$\begin{aligned} (T+\alpha I) x_\alpha ^\delta = y^\delta . \end{aligned}$$

(2.6)

Then we have

$$\begin{aligned} (T+\alpha I) (x_\alpha - x_\alpha ^\delta ) = y - y^\delta \end{aligned}$$

so that

$$\begin{aligned} \Vert x_\alpha - x_\alpha ^\delta \Vert = \Vert (T+\alpha I)^{-1} (y - y^\delta )\Vert = \frac{1}{\alpha } \Vert \alpha (T+\alpha I)^{-1} (y - y^\delta )\Vert . \end{aligned}$$

Hence,

$$\begin{aligned} \Vert x_\alpha - x_\alpha ^\delta \Vert \le M \frac{\delta }{\alpha }. \end{aligned}$$

(2.7)

Thus, we obtain the following theorem.

Theorem 15

Let $y\in R(T)$ and let $x\in X$ be such that $Tx=y$. Then, under the Assumption 8on T, we have the following.

• (1) $\displaystyle \Vert x - x_\alpha ^\delta \Vert \le \Vert x-x_\alpha \Vert + M \frac{\delta }{\alpha }.$

• (2) If R(T) is dense in X and $\alpha _\delta$ is such that $\alpha _\delta \rightarrow 0$ and ${\delta }/{\alpha _\delta } \rightarrow 0$ as $\delta \rightarrow 0,$ then

  $$\begin{aligned} \Vert x - x_{\alpha _\delta }^\delta \Vert \rightarrow 0\quad \text{ as }\quad \delta \rightarrow 0. \end{aligned}$$

• (3) If $x\in R(T)$ and $\alpha _\delta \sim \delta ^{1/2}$, then

  $$\begin{aligned} \Vert x - x_{\alpha _\delta } ^\delta \Vert = O( \delta ^{1/2}). \end{aligned}$$

Proof

(1)  This is a consequence of (2.7).

(2)  Follows from the fact that $\Vert x-x_\alpha \Vert \rightarrow 0$ as $\alpha \rightarrow 0$.

(3)  $x\in R(T)$ implies $\Vert x-x_\alpha \Vert = O(\alpha )$. Hence $\alpha \sim \delta ^{1/2}$ implies $\alpha +\frac{\delta }{\alpha } = O(\delta ^{1/2}).$ $\square$

2.3 Tikhonov regularization

Suppose X and Y are Hilbert spaces and $T: X\rightarrow Y$ is a bounded operator. In this case, $T^*T$ is a positive self adjoint operator. Let $y\in Y$ be such that

$$\begin{aligned} T^*Tx = T^*y. \end{aligned}$$

(2.8)

It is known that the above equation has a solution if and only if $y\in R(T)+R(T)^\perp$, equivalently,

$$\begin{aligned} u\mapsto \Vert Tu-y\Vert \end{aligned}$$

attains its minimum at x, which satisfies (2.8) (cf.  [11, 12]).

Definition 16

A solution of (2.8) is called a least-square solution or more appropriately, a least residual norm (LRN) solution of (1.1).$\lozenge$

It can be shown that, if there is an LRN-solution x, then there is an LRN-solution of minimal norm, say $x^\dagger$, and it is given by (cf.  [11, 12])

$$\begin{aligned} x^\dagger := Qx, \end{aligned}$$

where $Q: X\rightarrow X$ is the orthogonal projection onto $N(T)^\perp$. In particular,

$$\begin{aligned} T^*Tx^\dagger = T^*y. \end{aligned}$$

(2.9)

One would like to obtain regularized approximations for an LRN solution of minimal norm.

Let $x_\alpha \in X$ be the unique element such that

$$\begin{aligned} (T^*T+\alpha I) x_\alpha = T^*y; \end{aligned}$$

(2.10)

equivalently (cf.  [12]), $x_\alpha$ minimizes

$$\begin{aligned} u\mapsto \Vert Tu-y\Vert ^2+\alpha \Vert u\Vert ^2. \end{aligned}$$

(2.11)

From (2.9) and (2.10), we obtain

$$\begin{aligned} (T^*T+\alpha I)(x^\dagger - x_\alpha ) = \alpha x^\dagger . \end{aligned}$$

Thus,

$$\begin{aligned} x^\dagger - x_\alpha = \alpha (T^*T+\alpha I)^{-1} x^\dagger =S_\alpha x^\dagger , \end{aligned}$$

where

$$\begin{aligned} S_\alpha := \alpha (T^*T+\alpha I)^{-1}. \end{aligned}$$

Since $T^*T$ is a positive self-adjoint operator, we have the following theorem (cf.  [12]).

Theorem 17

Suppose $y\in R(T)$ and $x\in N(T)^\perp$ is the unique element such that $Tx=y$ . For $\alpha >0$ , let $x_\alpha \in X$ be the unique element such that (2.10) is satisfied. Then,

$$\begin{aligned} \Vert x-x_\alpha \Vert \rightarrow 0\quad \text{ as }\quad \alpha \rightarrow 0, \end{aligned}$$

and the operators

$$\begin{aligned} R_\alpha := (T^*T+\alpha I)^{-1}T^*,\quad \alpha >0, \end{aligned}$$

define a regularization family. Further, if $x\in R(T^*T)$, then

$$\begin{aligned} \Vert x-x_\alpha \Vert =O(\alpha ). \end{aligned}$$

2.4 Error estimates under noisy data

Now, let $y^\delta \in Y$ be such that

$$\begin{aligned} \Vert y-y^\delta \Vert \le \delta , \end{aligned}$$

and let $x_\alpha ^\delta \in X$ be such that

$$\begin{aligned} (T^*T+\alpha I) x_\alpha ^\delta = T^*y^\delta . \end{aligned}$$

Then we have

$$\begin{aligned} \Vert x_\alpha - x_\alpha ^\delta \Vert = \Vert (T^*T+\alpha I)^{-1} T^*(y - y^\delta )\Vert . \end{aligned}$$

It can be shown that  (cf.  [12])

$$\begin{aligned} \Vert (T^*T+\alpha I)^{-1} T^*\Vert \le \frac{1}{2\sqrt{\alpha }}, \end{aligned}$$

so that

$$\begin{aligned} \Vert x_\alpha - x_\alpha ^\delta \Vert \le \frac{\delta }{2\sqrt{\alpha }}. \end{aligned}$$

Thus, we obtain the following theorem (cf.  [12]).

Theorem 18

Let $x\in N(T)^\perp$ be such that $Tx=y$. Then the following results are true.

• (1) $\displaystyle \Vert x - x_\alpha ^\delta \Vert \le \Vert x-x_\alpha \Vert + \frac{\delta }{2\sqrt{\alpha }}.$

• (2) If $\alpha _\delta$ is such that $\alpha _\delta \rightarrow 0$ and $\delta /\sqrt{\alpha _\delta } \rightarrow 0$ as $\delta \rightarrow 0$, then

  $$\begin{aligned} \Vert x - x_{\alpha _\delta }^\delta \Vert \rightarrow 0\quad \text{ as }\quad \delta \rightarrow 0. \end{aligned}$$

• (3) If $x\in R(T^*T)$ and $\alpha _\delta \sim \delta ^{2/3}$,

  $$\begin{aligned} \Vert x - x_{\alpha _\delta }^\delta \Vert \le O( \delta ^{2/3}). \end{aligned}$$

Remark 19

(i)  In Theorem 18 (2), we may take $\alpha _\delta :=\delta ^s$ for some $0<s<2$ so that $\alpha _\delta \rightarrow 0$ and $\delta /\sqrt{\alpha _\delta }=\sqrt{\delta }\rightarrow 0$ as $\delta \rightarrow 0$.

(ii)  The rate $O(\delta ^{2/3})$ obtained in Theorem 18 (3) is optimal for Tikhonov regularization, in the sense that, for any choice of the regularization parameter $\alpha _\delta$, the rate $o(\delta ^{2/3})$ is not possible (cf.  [2, 12]).$\lozenge$

3 Illustration of source conditions

In the last section, we obtained error estimates for certain regularization methods under the assumption that the solution x belongs to R(T) or $R(T^*T)$ if the space is a Hilbert space so that it is of the forms

$$\begin{aligned} x=Tu\quad \text{ or }\quad x = T^*Tu, \end{aligned}$$

respectively. Such conditions are called source conditions. Let us illustrate these conditions when the operator involved is a a compact operator.

Case 1: Let X be a Hilbert space and $T:X\rightarrow X$ be a compact positive self-adjoint operator of infinite rank. Then

$$\begin{aligned} Tu = \sum _{n=1}^\infty \lambda _n \left\langle u, \varphi _n \right\rangle \varphi _n, \quad u\in X, \end{aligned}$$

(3.1)

where $(\lambda _n)$ is a null sequence of positive real numbers and $\{\varphi _n: n\in {\mathbb {N}}\}$ is an orthonormal basis of $N(T)^\perp$. Suppose $x\in N(T)^\perp = \overline{R(T)}$. Then

$$\begin{aligned} x\in R(T)\iff & {} x=\sum _{n=1}^\infty \lambda _n \left\langle u, \varphi _n \right\rangle \varphi _n\quad \text{ for } \text{ some }\quad u\in X\\\iff & {} \left\langle x, \varphi _n \right\rangle = \lambda _n \left\langle u, \varphi _n \right\rangle \quad \forall \, n\in {\mathbb {N}} \text{ for } \text{ some } \, u\in X. \end{aligned}$$

Thus,

$$\begin{aligned} x\in R(T) \iff \sum _{n=1}^\infty \frac{|\left\langle x, \varphi _n \right\rangle |^2}{\lambda _n^2}<\infty . \end{aligned}$$

(3.2)

Case 2: Let X and Y be a Hilbert spaces and $T:X\rightarrow Y$ be a compact operator of infinite rank. Then

$$\begin{aligned} Tu = \sum _{n=1}^\infty s_n \left\langle u, \varphi _n \right\rangle \psi _n, \quad u\in X, \end{aligned}$$

(3.3)

where $(s_n)$ is null sequence of positive real numbers, $\{\varphi _n: n\in {\mathbb {N}}\}$ and $\{\psi _n: n\in {\mathbb {N}}\}$ are orthonormal bases of $N(T)^\perp$ and $N(T^*)^\perp$, respectively. Suppose $x\in N(T)^\perp = N(T^*T)^\perp = \overline{R(T^*T)}$. Then, as in Case 1,

$$\begin{aligned} x\in R(T^*T) \iff \sum _{n=1}^\infty \frac{|\left\langle x, \varphi _n \right\rangle |^2}{s_n^2}<\infty . \end{aligned}$$

(3.4)

Analogous to (3.2) and (3.4) we have the following situations: Let $0<\nu \le 1$. (i) Suppose $T:X\rightarrow X$ is a compact, positive, self adjoint operator with spectral representation as in (3.1). Then we may define

$$\begin{aligned} T^{\nu }u = \sum _{n=1}^\infty \lambda _n^\nu \left\langle u, \varphi _n \right\rangle \varphi _n, \quad u\in X. \end{aligned}$$

From this, it follows that,

$$\begin{aligned} x\in R(T^\nu ) \iff \sum _{n=1}^\infty \frac{|\left\langle x, \varphi _n \right\rangle |^2}{\lambda _n^{2\nu }}<\infty . \end{aligned}$$

(3.5)

(ii) Suppose $T:X\rightarrow Y$ is a compact operator with singular value representation as in (3.3). Then, since $T^*T: X\rightarrow Y$ is a compact, positive, self adjoint operator, we obtain

$$\begin{aligned} (T^*T)^{\nu }u = \sum _{n=1}^\infty s_n^{2\nu } \left\langle u, \varphi _n \right\rangle \varphi _n, \quad u\in X. \end{aligned}$$

From this, it follows that

$$\begin{aligned} x\in R((T^*T)^\nu ) \iff \sum _{n=1}^\infty \frac{|\left\langle x, \varphi _n \right\rangle |^{2}}{s_n^{2\nu }}<\infty . \end{aligned}$$

(3.6)

Clearly, for $0<\nu <1$, the requirements in (3.5) and (3.6), are milder than the conditions in (3.2) and (3.4), respectively.

Observations:  Let $(\mu _n)$ be a null sequence of positive real numbers and let $(\varphi _n)$ be an orthonormal sequence in a Hilbert space H. For $0<\nu <1$, let

$$\begin{aligned} H_\nu := \left\{ x\in H: \sum _{n=1}^\infty \frac{|\left\langle x, \varphi _n \right\rangle |^2}{\mu _n^{2\nu }}<\infty \right\} . \end{aligned}$$

Then:

• (1) $H_\nu$ is a Hilbert space (a generalized Sobolev space) with inner product:

  $$\begin{aligned} \left\langle x, y \right\rangle _\nu :=\sum _{n=1}^\infty \frac{1}{\mu _n^{2\nu }}\left\langle x, \varphi _n \right\rangle \left\langle \varphi _n, y \right\rangle . \end{aligned}$$

• (2) If $\nu _1< \nu _2$, then $H_{\nu _2}\subset H_{\nu _1}$ is a proper inclusion, and it is a compact embedding.

Note that

$$\begin{aligned} \left\langle x, \varphi _k \right\rangle _\nu&=\sum _{n=1}^\infty \frac{1}{\mu _n^{2\nu }}\left\langle x, \varphi _n \right\rangle \left\langle \varphi _n, \varphi _k \right\rangle =\frac{1}{\mu _k^{2\nu }}\left\langle x, \varphi _k \right\rangle ,\\ \left\langle \varphi _m, \varphi _k \right\rangle _\nu&=\frac{1}{\mu _k^{2\nu }}\left\langle \varphi _m, \varphi _k \right\rangle , \end{aligned}$$

so that $\{\mu _n^\nu \varphi _n: n\in {\mathbb {N}}\}$ is an orthonormal basis for $H_\nu$. Hence, for $\nu _1< \nu _2$, if $x\in H_{\nu _2}$, then

$$\begin{aligned} x = \sum _{n=1}^\infty \left\langle x, \mu _n^{\nu _2}\varphi _n \right\rangle \mu _n^{\nu _2}\varphi _n = \sum _{n=1}^\infty \mu _n^{\nu _2-\nu _1} \left\langle x, \mu _n^{\nu _2}\varphi _n \right\rangle \mu _n^{\nu _1}\varphi _n. \end{aligned}$$

Thus, the inclusion operator ${{\mathcal {J}}}: H_{\nu _2}\rightarrow H_{\nu _1}$ is a compact operator with singular values $\mu _n^{\nu _2-\nu _1}$. For $\nu >0$,

$$\begin{aligned} \left\langle x, y \right\rangle _{-\nu } :=\sum _{n=1}^\infty \frac{1}{\mu _n^{-2\nu }}\left\langle x, \varphi _n \right\rangle \left\langle \varphi _n, y \right\rangle =\sum _{n=1}^\infty {\mu _n^{2\nu }}\left\langle x, \varphi _n \right\rangle \left\langle \varphi _n, y \right\rangle . \end{aligned}$$

defines an inner product on H, and its completion $H_{-\nu }$ is linearly isometric with $(H_\nu )'$, the dual of $H_\nu$ (cf.  [17]).

The triple $(H_\nu ,\, H,\, H_{-\nu })$ is called a Gelfand triple. The family $\{H_\nu :\, \nu \in {\mathbb {R}}\}$ is a Hilbert scale, a generalization of the Sobolev scale. More generally, a Hilbert scale is defined as follows  (cf.  [17]):

Definition 20

A Hilbert scale $(X_s)_{s\in {\mathbb {R}}}$ is a family of Hilbert spaces generated by some unbounded closed densely defined positive, self adjoint operator B in X with $D(B)\subseteq X$ such that

• (1) $X_0=X$,

• (2) For $s>0$, $X_s$ is the completion of ${{\mathcal {D}}}:= \bigcap _{k=1}^\infty D(B^k)$ w.r.t. the norm

  $$\begin{aligned} \Vert u\Vert _s:= \Vert B^su\Vert ,\quad u\in {{\mathcal {D}}}. \end{aligned}$$

• (3) For $s>0$, $X_{-s}$ is the completion of X w.r.t. the norm

  $$\begin{aligned} \Vert u\Vert _{-s} := \sup _{0\not = v\in X_s} \frac{|\left\langle u, v \right\rangle |}{\Vert v\Vert _s}.\quad \quad \quad \lozenge \end{aligned}$$

As in the case of bounded operators, in the above definition, by B being a positive and self adjoint operator, we mean that $\left\langle Bx, x \right\rangle \ge 0$ for all $x\in B$ and $T^*=T$ on D(B), where $T^*$ is the adjoint of B, which can be defined for closed densely defined operator as well (see, e.g.  Nair [11]).

Remark 21

It is known that (cf.  [17]), for $s>0$, $X_{-s}$ is linearly isometric with the dual of $X_s$.$\lozenge$

4 Error estimates under general source conditions

4.1 General source conditions for Lavrentiev regularization

In this section, we shall consider a more general form of a source condition for Lavrentiev regularization, namely,

$$\begin{aligned} x = \phi (T)u \end{aligned}$$

(4.1)

for some $u\in X$, where $\phi (\lambda ),\, \lambda >0$, is a continuous function such that

$$\begin{aligned} \phi (\lambda )\rightarrow 0\quad \text{ as }\quad \lambda \rightarrow 0. \end{aligned}$$

(4.2)

Assumption 22

Suppose there exists a constant $c_0>0$ such that

$$\begin{aligned} \sup _{\lambda>0}\frac{\alpha \phi (\lambda )}{\lambda +\alpha }\le c_0\, \phi (\alpha )\quad \forall \alpha >0. \end{aligned}$$

(4.3)

$\lozenge$

Here is a sufficient condition on $\phi$ to satisfy (4.3).

Lemma 23

Suppose $\phi$ is concave, monotonically increasing and $\phi (\lambda )\rightarrow 0$ as $\lambda \rightarrow 0$ . Then

$$\begin{aligned} \sup _{\lambda>0}\frac{\alpha \phi (\lambda )}{\lambda +\alpha }\le \phi (\alpha )\quad \forall \alpha >0. \end{aligned}$$

Proof

For $\alpha >0$, writing a point in $[0, \phi (\lambda )]$ as

$$\begin{aligned} \frac{\alpha }{\lambda +\alpha } \phi (\lambda ) + \frac{\lambda }{\lambda +\alpha } 0, \end{aligned}$$

we obtain

$$\begin{aligned} \phi \left( \frac{\alpha \lambda }{\lambda +\alpha }\right) \ge \frac{\alpha }{\lambda +\alpha } \phi (\lambda ). \end{aligned}$$

Since $\phi$ is monotonically increasing, we have

$$\begin{aligned} \frac{\alpha }{\lambda +\alpha } \phi (\lambda ) \le \varphi \left( \frac{\alpha \lambda }{\lambda +\alpha }\right) \le \phi (\alpha ). \end{aligned}$$

Thus, we obtain the required result. $\square$

Theorem 24

Suppose x satisfies the source condition as in (4.1) for some $u\in X$ . Let $\phi$ be as in Assumption22. Then

$$\begin{aligned} \Vert x-x_\alpha \Vert \le c_0\Vert u\Vert \phi (\alpha ). \end{aligned}$$

Further, if $y^\delta \in X$ is such that $\Vert y-y^\delta \Vert \le \delta$ and $x_\alpha ^\delta$ is the corresponding Lavrentiev regularized solution, then

$$\begin{aligned} \Vert x-x_\alpha ^\delta \Vert \le c_0\Vert u\Vert \phi (\alpha ) + \frac{\delta }{\alpha }. \end{aligned}$$

In particular, taking $\alpha _\delta := \psi ^{-1}(\delta )$ , where $\psi$ is the inverse of $\alpha \mapsto \alpha \phi (\alpha )$ , then

$$\begin{aligned} \Vert x-x_{\alpha _\delta }^\delta \Vert = O(\phi (\psi ^{-1}(\delta )). \end{aligned}$$

Proof

By (2.4), we have

$$\begin{aligned} x-x_\alpha = \alpha (T+\alpha I)^{-1} \phi (T)u. \end{aligned}$$

Hence,

$$\begin{aligned} \Vert x-x_\alpha \Vert\le & {} \alpha \Vert (T+\alpha I)^{-1} \phi (T)\Vert \, \Vert u\Vert \\\le & {} \Vert u\Vert \sup _{\lambda >0} \frac{\alpha \phi (\lambda )}{\lambda +\alpha } \\\le & {} c_0\Vert u\Vert \phi (\alpha ). \end{aligned}$$

To obtain the last estimate, note that

$$\begin{aligned} \phi (\alpha ) = \frac{\delta }{\alpha } \iff \alpha \phi (\alpha ) = \delta \iff \alpha = \psi ^{-1}(\delta ), \end{aligned}$$

where $\psi$ is the the function $\alpha \mapsto \alpha \phi (\alpha )$. $\square$

Remark 25

If T is a positive compact self adjoint operator T with spectral representation

$$\begin{aligned} Tu = \sum _{n=1}^\infty \lambda _n\left\langle u, \varphi _n \right\rangle \varphi _n,\quad u\in X, \end{aligned}$$

then

$$\begin{aligned} \phi (T) u = \sum _{n=1}^\infty \phi (\lambda _n) \left\langle u, \varphi _n \right\rangle \varphi _n,\quad u\in X. \end{aligned}$$

Hence, the requirement (4.1) is same as requiring x to belong to the (Hilbert) space

$$\begin{aligned} X_\phi := \left\{ x\in X: \sum _{n=1}^\infty \frac{|\left\langle x, \varphi _n \right\rangle |^2}{[\phi (\lambda _n)]^2}<\infty \right\} . \end{aligned}$$

(4.4)

The above general consideration is useful in applications.$\lozenge$

4.2 An application

Let us consider the problem of identifying the initial temperature

$$\begin{aligned} x(\cdot ):= u(\cdot , 0) \end{aligned}$$

from the knowledge of the final temperature

$$\begin{aligned} y(\cdot ):=u(\cdot , \tau ), \end{aligned}$$

where u(s, t) satisfies the one-dimensional heat equation:

$$\begin{aligned} \frac{\partial u}{\partial t} = \kappa ^2 \frac{\partial ^2 u}{\partial s^2},\quad 0<s<\ell ,\quad 0<t<\tau . \end{aligned}$$

We may recall that

$$\begin{aligned} u(\cdot , t) = \sum _{n=1}^\infty e^{-\mu _n^2t}\left\langle u(\cdot , 0), \varphi _n \right\rangle \varphi _n, \end{aligned}$$

where

$$\begin{aligned} \mu _n:=\frac{n\pi \kappa }{\ell },\quad \varphi _n(s):= \sqrt{\frac{2}{\ell }}\sin (\mu _n s). \end{aligned}$$

Thus, the problem is to solve the operator equation

$$\begin{aligned} \sum _{n=1}^\infty e^{-\mu _n^2\tau }\left\langle x, \varphi _n \right\rangle \varphi _n = u(\cdot , \tau ). \end{aligned}$$

Note that

$$\begin{aligned} Tx:= \sum _{n=1}^\infty e^{-\mu _n^2\tau }\left\langle x, \varphi _n \right\rangle \varphi _n \end{aligned}$$

defines a compact self-adjoint operator on $L^2[0, \ell ]$ with spectrum

$$\begin{aligned} \{\lambda _n:= e^{-\mu _n^2\tau }: n\in {\mathbb {N}}\}. \end{aligned}$$

Clearly, for a given $y\in L^2[0, \ell ]$, the above operator equation

$$\begin{aligned} Tx=y \end{aligned}$$

has a solution $x\in L^2[0, \ell ]$, i.e.,

$$\begin{aligned} y\in R(T) \iff \sum _{n=1}^\infty e^{2\mu _n^2\tau }|\left\langle y, \varphi _n \right\rangle |^2<\infty . \end{aligned}$$

Let us assume that y satisfies the above condition. Again

$$\begin{aligned} x\in R(T) \iff \sum _{n=1}^\infty e^{2\mu _n^2\tau } |\left\langle x, \varphi _n \right\rangle |^2 <\infty . \end{aligned}$$

A milder assumption on x than the above would be:

$$\begin{aligned} \sum _{n=1}^\infty \mu _n^2|\left\langle x, \varphi _n \right\rangle |^2 <\infty . \end{aligned}$$

(4.5)

Recall that $\lambda _n:= e^{-\mu _n^2\tau }$ are the non-zero spectral values (eigenvalues) of T. Note that

$$\begin{aligned} \lambda _n := e^{-\mu _n^2\tau }\iff & {} \frac{1}{\lambda _n} = e^{\mu _n^2\tau } \iff \mu _n^2 = \frac{1}{\tau }\log \left( \frac{1}{\lambda _n}\right) . \end{aligned}$$

Thus the (milder) condition (4.5) takes the form:

$$\begin{aligned} \sum _{n=1}^\infty \frac{|\left\langle x, \varphi _n \right\rangle |^2}{[\phi (\lambda _n)]^2} <\infty , \end{aligned}$$

where

$$\begin{aligned} \phi (\lambda ):= \left[ \frac{1}{\tau }\log (\frac{1}{\lambda })\right] ^{-1/2},\quad 0<\lambda <1. \end{aligned}$$

(4.6)

That is, x belongs to the space

$$\begin{aligned} X_\phi := \left \{x\in X: \sum _{n=1}^\infty \frac{|\left\langle x, \varphi _n \right\rangle |^2}{[\phi (\lambda _n )]^2}<\infty \right\} \end{aligned}$$

(4.7)

with $\phi$ as in (4.6). Thus, we can achieve the error estimate as in Theorem 23 under the assumption (4.5) on the initial temperature.

4.3 General source conditions for Tikhonov regularization

In the case of Tikhonov regularization, we may assume, in place of (4.1)

$$\begin{aligned} x=\phi (T^*T)u \end{aligned}$$

(4.8)

for some $u\in X$. Recall that, in Tikhonov regularization, one solves the equation

$$\begin{aligned} (T^*T+\alpha I)x_\alpha = T^*y \end{aligned}$$

in place of

$$\begin{aligned} T^*T x = T^*y. \end{aligned}$$

Then, in place of Theorem 23, we have the following theorem (cf.  [12]).

Theorem 26

Suppose the LRN solution x satisfies the source condition as in (4.8) for some $u\in X$ . Then

$$\begin{aligned} \Vert x-x_\alpha \Vert \le c_0\,\Vert u\Vert \phi (\alpha ). \end{aligned}$$

Further, if $y^\delta \in X$ is such that $\Vert y-y^\delta \Vert \le \delta$ and $x_\alpha ^\delta$ is such that

$$\begin{aligned} (TT^*+\alpha I)x_\alpha ^\delta = T^*y^\delta , \end{aligned}$$

then

$$\begin{aligned} \Vert x-x_\alpha ^\delta \Vert \le c_0\, \Vert u\Vert \phi (\alpha ) + \frac{\delta }{2\sqrt{\alpha }}. \end{aligned}$$

In particular, taking $\alpha := \psi ^{-1}(\delta )$ , where $\psi$ is the inverse of $\alpha \mapsto \sqrt{\alpha }\phi (\alpha )$ , then

$$\begin{aligned} \Vert x-x_\alpha ^\delta \Vert = O(\phi (\psi ^{-1}(\delta )). \end{aligned}$$

4.4 Remarks on parameter choice strategies

The parameter choice strategy we have considered so far are a priori. That is, with some a priori knowledge on the smoothness of the unknown solution.

There are many a posteriori parameter choice strategies available in the literature. Two of the simple strategies are the following:

Morozov’s type discrepancy principle (MDP):  $\alpha$ is chosen such that

$$\begin{aligned} c_1\delta \le \Vert Tx_\alpha ^\delta - y^\delta \Vert \le c_2\delta . \end{aligned}$$

(4.9)

Arcangeli’s type discrepancy principle (ADP):  $\alpha$ is chosen such that

$$\begin{aligned} c_1\frac{\delta }{\sqrt{\alpha }} \le \Vert Tx_\alpha ^\delta - y^\delta \Vert \le c_2\frac{\delta }{\sqrt{\alpha }}. \end{aligned}$$

(4.10)

The following results are known:

• (1) (Groetch and Guacaneme  [4]): For Lavrentiev regularization, MDP (4.9) does not lead to a convergent method; whereas ADP (4.10) gives convergence.

• (2) (Groetch  [2]):  For Tikhonov regularization the order $O(\delta ^{1/2})$ is obtained with $x\in R(T^*)$ under MDP (4.9), and this rate cannot be improved under MDP.

• (3) (Groetch & Schock  [5]):  For Tikhonov regularization the order $O(\delta ^{1/3})$ is obtained with $x\in R(T^*)$ under ADP (4.10), and this order cannot be improved under the condition of $x\in R(T^*)$.

However, it is known that for Tikhonov regularization the optimal order $O(\delta ^{2/3})$ is obtained under apriori choice $\alpha \sim \delta ^{2/3}$. The question of obtaining this rate under ADP was open right from its introduction in 1966.

The above problem was settled positively in 1992 by the author  [10]: For Tikhonov regularization the best optimal order $O(\delta ^{2/3})$ is attained under ADP (4.10) with $x\in R(T^*T)$. In fact, it was shown that $O(\delta ^{2/3})$ is attained under a general discrepancy principle:

$$\begin{aligned} \Vert Tx_\alpha ^\delta -y^\delta \Vert \sim \frac{\delta ^p}{\alpha ^q} \end{aligned}$$

whenever

$$\begin{aligned} \frac{p}{q+1} = \frac{2}{3}. \end{aligned}$$

4.5 Modification of the methods to obtain better estimates

One of the modifications for obtaining a better method is to modify the Tikhonov regularization, requiring additional smoothness for the solution as well as for the regularized approximations:

Tikhonov regularization using an unbounded operator:  In this, one looks for $x_\alpha$ which minimizes

$$\begin{aligned} u\mapsto \Vert Tu-y\Vert ^2 + \Vert Lu\Vert ^2,\quad u\in D(L), \end{aligned}$$

where $L: D(L)\subseteq X\rightarrow Z$ is a closed unbounded operator. Equivalently (cf.  [18]), solve:

$$\begin{aligned} (T^*T+\alpha L^*L)x_\alpha = T^*y. \end{aligned}$$

(4.11)

It is known that the operator $T^*T+\alpha L^*L: D(L^*L)\rightarrow X$ has continuous inverse if

$$\begin{aligned} \Vert Tu\Vert ^2+\Vert Lu\Vert ^2\ge \gamma \Vert u\Vert ^2\quad \forall \, u\in D(L), \end{aligned}$$

(4.12)

for some $\gamma >0$ (see, e.g., [18]).

Note that the above condition is satisfied if L is bounded below (that is the case with many of the differential operators).

For obtaining error estimates, we use the following assumption.

Assumption 27

There exist $a>0, \, b\ge 0,\, c>0,\, d>0$ such that

$$\begin{aligned} \Vert Tu\Vert\ge & {} c\Vert u\Vert _{-a}\quad \forall \, u\in X_a,\\ \Vert Lu\Vert\ge & {} d\Vert u\Vert _b \quad \forall \, u\in D(L)\cap X_b, \end{aligned}$$

where $(X_s)_{s\in {\mathbb {R}}}$ is a Hilbert scale.$\lozenge$

It can be shown that, if T is injective and if the Hilbert scale is generated by $B:=(T^*T)^{-1/2}$ and if $L:=B^b$, then

$$\begin{aligned} \Vert Tu\Vert =\Vert u\Vert _{-1}\quad \text{ and }\quad \Vert Lu\Vert =\Vert u\Vert _b. \end{aligned}$$

Under the Assumption 27, we have the following result (cf.  [13]):

Theorem 28

If ${\hat{x}}\in D(L)$ and $\alpha$ chosen according to the Morozov-type discrepancy principle MDP (4.9), then

$$\begin{aligned} \Vert {\hat{x}} - x_\alpha ^\delta \Vert = O(\delta ^{\frac{b}{b+a}}). \end{aligned}$$

Note that, larger the b better the rate.

4.6 Further improvements

Here are a few results improving the above results (cf.  [15]).

Theorem 29

If ${\hat{x}}\in D(L^*L)$ and $\alpha$ is chosen according to the Morozov-type discrepancy principle MDP (4.9), then

$$\begin{aligned} \Vert {\hat{x}} - x_\alpha ^\delta \Vert = O(\delta ^{\frac{2b}{2b+a}}). \end{aligned}$$

Theorem 30

If ${\hat{x}}\in D(L^*L)$ , $L^*L x = (T^*T)^\nu ,\, 0<\nu \le 1$ and $\alpha$ is chosen according to the Morozov-type discrepancy principle MDP (4.9), then

$$\begin{aligned} \Vert {\hat{x}} - x_\alpha ^\delta \Vert = O(\delta ^p), \end{aligned}$$

where $p:= {2(a\nu +b)}/[{2(a\nu +b)+a}].$

Recently  [16], the condition

$$\begin{aligned} \Vert Tu\Vert ^{\theta }\Vert Lu\Vert ^{1-\theta }\ge \eta \Vert u\Vert ,\quad u\in D(L), \end{aligned}$$

(4.13)

for some $0\le \theta <1$ is considered in place of the completion condition (4.12) and proved the following theorem which includes results in the Hilbert scale setting as special cases:

Theorem 31

Suppose (4.13) is satisfied in place of (4.12).

• (1) If ${\hat{x}}\in D(L)$ and $\alpha$ is chosen according to the Morozov-type discrepancy principle MDP (4.9), then

  $$\begin{aligned} \Vert {\hat{x}} - x_\alpha ^\delta \Vert = O(\delta ^\theta ). \end{aligned}$$

• (2) If ${\hat{x}}\in D(L^*L)$ and $\alpha$ is chosen according to the Morozov-type discrepancy principle MDP (4.9), then

  $$\begin{aligned} \Vert {\hat{x}} - x_\alpha ^\delta \Vert = O(\delta ^\frac{2\theta }{1+\theta }). \end{aligned}$$

Note that, taking $\theta = \frac{b}{a+b}$, we obtain results in Theorem 28 and 29 as special cases.