Ill-Posed Problems

Abstract

For problems in mathematical physics, Hadamard postulated three properties that he deemed to be of central importance:

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

For problems in mathematical physics, Hadamard postulated three properties that he deemed to be of central importance:

• 1. Existence of a solution

• 2. Uniqueness of a solution

• 3. Continuous dependence of the solution on the data.

A problem satisfying all three of these requirements is called well-posed. To be more precise, we make the following definition: let A : U → V be an operator from a subset U of a normed space X into a subset V of a normed space Y. The equation $A\varphi = f$ is called well-posed if A is bijective and A ⁻¹: V → U is continuous. Otherwise, $A\varphi = f$ is called ill-posed or improperly posed. Contrary to Hadamard’s point of view, in recent years it has become clear that many important problems of mathematical physics are in fact ill-posed! In particular, all of the inverse scattering problems considered in this book are ill-posed, and for this reason we devote a short chapter to the mathematical theory of ill-posed problems. But first we present a simple example of an ill-posed problem.

FormalPara Example 2.1.

Consider the initial-boundary value problem

$$\displaystyle\begin{array}{rcl} & & \quad \frac{\partial u} {\partial t} = \frac{{\partial }^{2}u} {\partial {x}^{2}}\quad \mathrm{in}\quad [0,\pi ] \times [0,T] {}\\ & & u(0,t) = u(\pi,t) = 0\quad \mathrm{,}\quad 0 \leq t \leq T {}\\ & & \quad u(x,0) =\varphi (x)\quad \mathrm{,}\quad 0 \leq x \leq \pi \,\mathrm{,} {}\\ \end{array}$$

where $\varphi \in C[0,\pi ]$ is a given function. Then, by separation of variables, we obtain the solution

$$\displaystyle\begin{array}{rcl} u(x,t)& =& \sum _{1}^{\infty }a_{ n}{e}^{-{n}^{2}t }\sin nx, {}\\ a_{n}& =& \frac{2} {\pi } \int _{0}^{\pi }\varphi (y)\sin ny\,dy\,\mathrm{,} {}\\ \end{array}$$

and it is not difficult to show that this solution is unique and depends continuously on the initial data with respect to the maximum norm, i.e.,

$$\displaystyle{\max _{[0,\pi ]\times [0,T]}\left \vert u(x,t)\right \vert \leq C\max _{[0,\pi ]}\left \vert \varphi (x)\right \vert }$$

for some positive constant C [43]. Now consider the inverse problem of determining $\varphi$ from f : = u(⋅, T). In this case,

$$\displaystyle\begin{array}{rcl} u(x,t)& =& \sum _{1}^{\infty }b_{ n}{e}^{{n}^{2}(T-t) }\sin nx\,\mathrm{,} {}\\ b_{n}& =& \frac{2} {\pi } \int _{0}^{\pi }f(y)\sin ny\,dy\,\mathrm{,} {}\\ \end{array}$$

and hence

$$\displaystyle{{\left \|\varphi \right \|}^{2} = \frac{2} {\pi } \sum _{1}^{\infty }{\left \vert b_{ n}\right \vert }^{2}{e}^{2{n}^{2}T }\,\mathrm{,}}$$

which is infinite unless the b _n decay extremely rapidly. Even if this is the case, small perturbations of f (and hence of the b _n ) will result in the nonexistence of a solution! Note that the inverse problem can be written as an integral equation of the first kind with smooth kernel:

$$\displaystyle{\int _{0}^{\pi }K(x,y)\varphi (y)\,dy = f(x)\quad \mathrm{,}\quad 0 \leq x \leq \pi \,\mathrm{,}}$$

where

$$\displaystyle{K(x,y) = \frac{2} {\pi } \sum _{1}^{\infty }{e}^{-{n}^{2}T }\sin nx\sin ny\quad \mathrm{,}\quad 0 \leq x,y \leq \pi.}$$

In particular, the preceding integral operator is compact in any reasonable function space, for example, L ²[0, π].

FormalPara Theorem 2.2.

Let X and Y be normed spaces, and let A : X → Y be a compact operator. Then $A\varphi = f$ is ill-posed if X is not of finite dimension.

FormalPara Proof.

Assume A ⁻¹ exists and is continuous. Then $I = {A}^{-1}A: X \rightarrow X$ is compact, and hence, by Theorem 1.20 X, is finite dimensional.

We will now proceed, again following [111], to present the basic mathematical ideas for treating ill-posed problems. For a more detailed discussion we refer the reader to [71, 98, 111], and, in particular, [68].

2.1 Regularization Methods

Methods for constructing a stable approximate solution to an ill-posed problem are called regularization methods. In particular, for A a bounded linear operator, we want to approximate the solution $\varphi$ of $A\varphi = f$ from a knowledge of a perturbed right-hand side with a known error level

$$\displaystyle{\left \|f - {f}^{\delta }\right \| \leq \delta.}$$

When f ∈ A(X), then, if A is injective, there exists a unique solution $\varphi$ of $A\varphi = f$. However, in general we cannot expect that f ^δ ∈ A(X). How do we construct a reasonable approximation ${\varphi }^{\delta }$ to $\varphi$ that depends continuously on f ^δ?

Definition 2.3.

Let X and Y be normed spaces, and let A : X → Y be an injective bounded linear operator. Then a family of bounded linear operators R _α : Y → X, α > 0, such that

$$\displaystyle{\lim _{\alpha \rightarrow 0}R_{\alpha }A\varphi =\varphi }$$

for every $\varphi \in X$, is called a regularization scheme for A. The parameter α is called the regularization parameter.

We clearly have that R _α f → A ⁻¹ f as α → 0 for every f ∈ A(X). The following theorem shows that for compact operators this convergence cannot be uniform.

Theorem 2.4.

Let X and Y be normed spaces, let A : X → Y be an injective compact operator, and assume X has infinite dimension. Then the operators R _α cannot be uniformly bounded with respect to α as α → 0 and R _α A cannot be norm convergent as α → 0.

Proof.

Assume $\left \|R_{\alpha }\right \| \leq C$ as α → 0. Then, since R _α f → A ⁻¹ f as α → 0 for every f ∈ A(X), we have that $\left \|{A}^{-1}f\right \| \leq C\left \|f\right \|$, and hence A ⁻¹ is bounded on A(X). But this implies $I = {A}^{-1}A$ is compact on X, which contradicts the fact that X has infinite dimension.

Now assume that R _α A is norm convergent as α → 0, i.e., $\left \|R_{\alpha }A - I\right \| \rightarrow 0$ as α → 0. Then there exists α > 0 such that $\left \|R_{\alpha }A - I\right \| < \frac{1} {2}$, and hence for every f ∈ A(X) we have that

$$\displaystyle\begin{array}{rcl} \left \|{A}^{-1}f\right \|& =& \left \|{A}^{-1}f - R_{\alpha }A{A}^{-1}f + R_{\alpha }f\right \| {}\\ & \leq & \left \|{A}^{-1}f - R_{\alpha }A{A}^{-1}f\right \| + \left \|R_{\alpha }f\right \| {}\\ & \leq & \left \|I - R_{\alpha }A\right \|\left \|{A}^{-1}f\right \| + \left \|R_{\alpha }\right \|\left \|f\right \| {}\\ & \leq & \frac{1} {2}\left \|{A}^{-1}f\right \| + \left \|R_{\alpha }\right \|\left \|f\right \|. {}\\ \end{array}$$

Hence $\left \|{A}^{-1}f\right \| \leq 2\left \|R_{\alpha }\right \|\left \|f\right \|$, i.e., A ⁻¹: A(X) → X is bounded and we again have arrived at a contradiction.

A regularization scheme approximates the solution $\varphi$ of $A\varphi = f$ by

$$\displaystyle{\varphi _{\alpha }^{\delta }:= R_{\alpha }{f}^{\delta }.}$$

Writing

$$\displaystyle{\varphi _{\alpha }^{\delta }-\varphi = R_{\alpha }{f}^{\delta } - R_{\alpha }f + R_{\alpha }A\varphi -\varphi \,\mathrm{,}}$$

we have the estimate

$$\displaystyle{\left \|\varphi _{\alpha }^{\delta }-\varphi \right \|\leq \delta \left \|R_{\alpha }\right \| + \left \|R_{\alpha }A\varphi -\varphi \right \|.}$$

By Theorem 2.4, the first term on the right-hand side is large for α small, whereas the second term on the right-hand side is large if α is not small! So how do we choose α? A reasonable strategy is to choose α = α(δ) such that $\varphi _{\alpha }^{\delta } \rightarrow \varphi$ as δ → 0.

Definition 2.5.

A strategy for a regularization scheme R _α , α > 0, i.e., a method for choosing the regularization parameter α = α(δ), is called regular if for every f ∈ A(X) and all f ^δ ∈ Y such that $\left \|{f}^{\delta } - f\right \| \leq \delta$ we have that

$$\displaystyle{R_{\alpha (\delta )}{f}^{\delta } \rightarrow {A}^{-1}f}$$

as δ → 0.

A natural strategy for choosing α = α(δ) is the discrepancy principle of Morozov [130], i.e., the residual $\left \|A\varphi _{\alpha }^{\delta } - {f}^{\delta }\right \|$ should not be smaller than the accuracy of the measurements of f. In particular, α = α(δ) should be chosen such that $\left \|AR_{\alpha }{f}^{\delta } - {f}^{\delta }\right \| =\gamma \delta$ for some constant γ ≥ 1. Given a regularization scheme, the question, of course, is whether or not such a strategy is regular.

2.2 Singular Value Decomposition

Henceforth X and Y will always be infinite-dimensional Hilbert spaces and A : X → Y, A ≠ 0, will always be a compact operator. Note that A ^∗ A : X → X is compact and self-adjoint. Hence, by the Hilbert–Schmidt theorem, there exists at most a countable set of eigenvalues $\{\lambda _{n}\}_{1}^{\infty }$, of A ^∗ A and if ${A}^{{\ast}}A\varphi _{n} =\lambda _{n}\varphi _{n}$ then $({A}^{{\ast}}A\varphi _{n},\varphi _{n}) =\lambda _{n}{\left \|\varphi _{n}\right \|}^{2}$, i.e., ${\left \|A\varphi _{n}\right \|}^{2} =\lambda _{n}{\left \|\varphi _{n}\right \|}^{2}$, which implies that λ _n ≥ 0 for n = 1, 2, ⋯ . The nonnegative square roots of the eigenvalues of A ^∗ A are called the singular values of A.

Theorem 2.6.

Let $\{\mu _{n}\}_{1}^{\infty }$ be the sequence of nonzero singular values of the compact operator A : X → Y ordered such that

$$\displaystyle{\mu _{1} \geq \mu _{2} \geq \mu _{3} \geq \cdots \,.}$$

Then there exist orthonormal sequences $\{\varphi _{n}\}_{1}^{\infty }$ in X and $\{g_{n}\}_{1}^{\infty }$ in Y such that

$$\displaystyle{A\varphi _{n} =\mu _{n}g_{n}\quad \mathrm{,}\quad {A}^{{\ast}}g_{ n} =\mu _{n}\varphi _{n}.}$$

For every $\varphi \in X$ we have the singular value decomposition

$$\displaystyle{\varphi =\sum _{ 1}^{\infty }(\varphi,\varphi _{ n})\varphi _{n} + P\varphi \,\mathrm{,}}$$

where P : X → N(A) is the orthogonal projection operator of X onto N(A) and

$$\displaystyle{A\varphi =\sum _{ 1}^{\infty }\mu _{ n}(\varphi,\varphi _{n})g_{n}.}$$

The system $(\mu _{n},\varphi _{n},g_{n})$ is called a singular system of A.

Proof.

Let $\{\varphi _{n}\}_{1}^{\infty }$ be the orthonormal eigenelements of A ^∗ A corresponding to $\{\mu _{n}\}_{1}^{\infty }$, i.e.,

$$\displaystyle{{A}^{{\ast}}A\varphi_{n} =\mu _{ n}^{2}\varphi _{ n}\,\mathrm{,}}$$

and define a second orthonormal sequence by

$$\displaystyle{g_{n}:= \frac{1} {\mu _{n}}A\varphi _{n}.}$$

Then $A\varphi _{n} =\mu _{n}g_{n}$ and ${A}^{{\ast}}g_{n} =\mu _{n}\varphi _{n}$. The Hilbert–Schmidt theorem implies that

$$\displaystyle{\varphi =\sum _{ 1}^{\infty }(\varphi,\varphi _{ n})\varphi _{n} + P\varphi \,\mathrm{,}}$$

where P : X → N(A ^∗ A) is the orthogonal projection operator of X onto N(A ^∗ A). But ψ ∈ N(A ^∗ A) implies that $(A\psi,A\psi ) = (\psi,{A}^{{\ast}}A\psi ) = 0$, and hence N(A ^∗ A) = N(A). Finally, applying A to the preceding expansion (first apply A to the partial sum and then take the limit), we have that

$$\displaystyle{A\varphi =\sum _{ 1}^{\infty }\mu _{ n}(\varphi,\varphi _{n})g_{n}.}$$

We now come to the main result that will be needed to study compact operator equations of the first kind, i.e., equations of the form $A\varphi = f$, where A is a compact operator.

Theorem 2.7 (Picard’s Theorem).

Let A : X → Y be a compact operator with singular system $(\mu _{n},\varphi _{n},g_{n})$ . Then the equation $A\varphi = f$ is solvable if and only if f ∈ N(A ^∗ ) ^⊥ and

$$\displaystyle{ \sum _{1}^{\infty } \frac{1} {\mu _{n}^{2}}{\left \vert (f,g_{n})\right \vert }^{2} < \infty. }$$

(2.1)

In this case a solution to $A\varphi = f$ is given by

$$\displaystyle{\varphi =\sum _{ 1}^{\infty }\frac{1} {\mu _{n}}(f,g_{n})\varphi _{n}.}$$

Proof.

The necessity of f ∈ N(A ^∗)^⊥ follows from Theorem 1.29. If $\varphi$ is a solution of $A\varphi = f$, then

$$\displaystyle{\mu _{n}(\varphi,\varphi _{n}) = (\varphi,{A}^{{\ast}}g_{ n}) = (A\varphi,g_{n}) = (f,g_{n}).}$$

But from the singular value decomposition of $\varphi$ we have that

$$\displaystyle{{\left \|\varphi \right \|}^{2} =\sum _{ 1}^{\infty }{\left \vert (\varphi,\varphi _{ n})\right \vert }^{2} +{ \left \|P\varphi \right \|}^{2}\,\mathrm{,}}$$

and hence

$$\displaystyle{\sum _{1}^{\infty } \frac{1} {\mu _{n}^{2}}{\left \vert (f,g_{n})\right \vert }^{2} =\sum _{ 1}^{\infty }{\left \vert (\varphi,\varphi _{ n})\right \vert }^{2} \leq {\left \|\varphi \right \|}^{2}\,\mathrm{,}}$$

which implies the necessity of condition (2.1).

Conversely, assume that f ∈ N(A ^∗)^⊥ and (2.1) is satisfied. Then from (2.1) we have that

$$\displaystyle{\varphi:=\sum _{ 1}^{\infty }\frac{1} {\mu _{n}}(f,g_{n})\varphi _{n}}$$

converges in the Hilbert space X. Applying A to this series we have that

$$\displaystyle{A\varphi =\sum _{ 1}^{\infty }(f,g_{ n})g_{n}.}$$

But, since f ∈ N(A ^∗)^⊥, this is the singular value decomposition of f corresponding to the operator A ^∗, and hence $A\varphi = f$.

Note that Picard’s theorem illustrates the ill-posed nature of the equation $A\varphi = f$. In particular, setting ${f}^{\delta } = f +\delta g_{n}$ we obtain a solution of ${A\varphi }^{\delta } = {f}^{\delta }$ given by ${\varphi }^{\delta } =\varphi +\delta \varphi _{n}/\mu _{n}$. Hence, if A(X) is not finite dimensional, then

$$\displaystyle{ \frac{\left \|{\varphi }^{\delta }-\varphi \right \|} {\left \|{f}^{\delta } - f\right \|} = \frac{1} {\mu _{n}} \rightarrow \infty }$$

since, by Theorem 1.14, we have that μ _n → 0. We say that $A\varphi = f$ is mildly ill-posed if the singular values decay slowly to zero and severely ill-posed if they decay very rapidly (for example, exponentially). All of the inverse scattering problems considered in this book are severely ill-posed.

Henceforth, to focus on ill-posed problems, we will always assume that A(X) is infinite dimensional, i.e., the set of singular values is an infinite set.

Example 2.8.

Consider the case of the backward heat equation discussed in Example 2.1. The problem considered in this example is equivalent to solving the compact operator equation $A\varphi = f$, where

$$\displaystyle{(A\varphi )(x):=\int _{ 0}^{\pi }K(x,y)\varphi (y)\,dy\quad \mathrm{,}\quad 0 \leq x \leq \pi \mathrm{,}}$$

and

$$\displaystyle{K(x,y):= \frac{2} {\pi } \sum _{1}^{\infty }{e}^{-{n}^{2}T }\sin nx\sin ny.}$$

Then A is easily seen to be self-adjoint with eigenvalues given by $\lambda _{n} = {e}^{-{n}^{2}T }$. Hence μ _n = λ _n , and the compact operator equation $A\varphi = f$ is severely ill posed.

Picard’s theorem suggests trying to regularize $A\varphi = f$ by damping or filtering out the influence of the higher-order terms in the solution $\varphi$ given by

$$\displaystyle{\varphi =\sum _{ 1}^{\infty }\frac{1} {\mu _{n}}(f,g_{n})\varphi _{n}.}$$

The following theorem does exactly that. We will subsequently consider two specific regularization schemes by making specific choices of the function q, which appears in the theorem.

Theorem 2.9.

Let A : X → Y be an injective compact operator with singular system $(\mu _{n},\varphi _{n},g_{n})$ , and let $q: (0,\infty ) \times (0,\left \|A\right \|] \rightarrow \mathbb{R}$ be a bounded function such that for every α > 0 there exists a positive constant c(α) such that

$$\displaystyle{\left \vert q(\alpha,\mu )\right \vert \leq c(\alpha )\mu \quad \mathrm{,}\quad 0 <\mu \leq \left \|A\right \|\,\mathrm{,}}$$

and

$$\displaystyle{\lim _{\alpha \rightarrow 0}q(\alpha,\mu ) = 1\quad \mathrm{,}\quad 0 <\mu \leq \left \|A\right \|.}$$

Then the bounded linear operators R _α : Y → X, α > 0, defined by

$$\displaystyle{R_{\alpha }f:=\sum _{ 1}^{\infty }\frac{1} {\mu _{n}}q(\alpha,\mu _{n})(f,g_{n})\varphi _{n}}$$

for f ∈ Y, describe a regularization scheme with

$$\displaystyle{\left \|R_{\alpha }\right \| \leq c(\alpha ).}$$

Proof.

Noting that from the singular value decomposition of f with respect to the operator A ^∗ we have that

$$\displaystyle{{\left \|f\right \|}^{2} =\sum _{ 1}^{\infty }{\left \vert (f,g_{ n})\right \vert }^{2} +{ \left \|Pf\right \|}^{2}\mathrm{,}}$$

where P : X → N(A ^∗) is the orthogonal projection of X onto N(A ^∗), we see that for every f ∈ Y we have that

$$\displaystyle\begin{array}{rcl}{ \left \|R_{\alpha }f\right \|}^{2}& =& \sum _{ 1}^{\infty } \frac{1} {\mu _{n}^{2}}{\left \vert q(\alpha,\mu _{n})\right \vert }^{2}{\left \vert (f,g_{ n})\right \vert }^{2} {}\\ & \leq &{ \left \vert c(\alpha )\right \vert }^{2}\sum _{ 1}^{\infty }{\left \vert (f,g_{ n})\right \vert }^{2} {}\\ & \leq &{ \left \vert c(\alpha )\right \vert }^{2}{\left \|f\right \|}^{2}\mathrm{,} {}\\ \end{array}$$

and hence $\left \|R_{\alpha }\right \| \leq c(\alpha )$. From

$$\displaystyle\begin{array}{rcl} (R_{\alpha }A\varphi,\varphi _{n})& =& \frac{1} {\mu _{n}}q(\alpha,\mu _{n})(A\varphi,g_{n}) {}\\ & =& q(\alpha,\mu _{n})(\varphi,\varphi _{n}) {}\\ \end{array}$$

and the singular value decomposition for $R_{\alpha }A\varphi -\varphi$ we obtain, using the fact that A is injective, that

$$\displaystyle\begin{array}{rcl}{ \left \|R_{\alpha }A\varphi -\varphi \right \|}^{2}& =& \sum _{ 1}^{\infty }{\left \vert (R_{\alpha }A\varphi -\varphi,\varphi _{ n})\right \vert }^{2} {}\\ & =& \sum _{1}^{\infty }{\left \vert q(\alpha,\mu _{ n}) - 1\right \vert }^{2}{\left \vert (\varphi,\varphi _{ n})\right \vert }^{2}. {}\\ \end{array}$$

Now let $\varphi \in X$, $\varphi \neq 0$, and let M be a bound for q. We first note that for every $\epsilon$ > 0 there exists N = N($\epsilon$) such that

$$\displaystyle{\sum _{N+1}^{\infty }{\left \vert (\varphi,\varphi _{ n})\right \vert }^{2} < \frac{\epsilon } {2{(M + 1)}^{2}}.}$$

Since lim _{α → 0} q(α, μ) = 1, there exists α ₀ = α ₀($\epsilon$) such that

$$\displaystyle{{\left \vert q(\alpha,\mu _{n}) - 1\right \vert }^{2} < \frac{\epsilon } {2{\left \|\varphi \right \|}^{2}}}$$

for n = 1, 2, ⋯, N and all α such that 0 < α ≤ α ₀. We now have that, for 0 < α ≤ α ₀,

$$\displaystyle\begin{array}{rcl}{ \left \|R_{\alpha }A\varphi -\varphi \right \|}^{2}& =& \sum _{ 1}^{N}{\left \vert q(\alpha,\mu _{ n}) - 1\right \vert }^{2}{\left \vert \left (\varphi,\varphi _{ n}\right )\right \vert }^{2} {}\\ & \quad +& \sum _{N+1}^{\infty }{\left \vert q(\alpha,\mu _{ n}) - 1\right \vert }^{2}{\left \vert \left (\varphi,\varphi _{ n}\right )\right \vert }^{2} {}\\ & \leq & \frac{\epsilon } {2{\left \|\varphi \right \|}^{2}}\sum _{1}^{N}{\left \vert \left (\varphi,\varphi _{ n}\right )\right \vert }^{2} + \frac{\epsilon } {2}\quad \mathrm{.} {}\\ \end{array}$$

But, since A is injective,

$$\displaystyle{{\left \|\varphi \right \|}^{2} =\sum _{ 1}^{\infty }{\left \vert (\varphi,\varphi _{ n})\right \vert }^{2}\mathrm{,}}$$

and hence ${\left \|R_{\alpha }A\varphi -\varphi \right \|}^{2} \leq \epsilon$ for 0 < α ≤ α ₀. We can now conclude that $R_{\alpha }A\varphi \rightarrow \varphi$ as α → 0 for every $\varphi \in X$ and the theorem is proved.

A particular choice of q now leads to our first regularization scheme, the spectral cutoff method.

Theorem 2.10.

Let A : X → Y be an injective compact operator with singular system $(\mu _{n},\varphi _{n},g_{n})$ . Then the spectral cutoff

$$\displaystyle{R_{m}f:=\sum _{\mu _{n}\geq \mu _{m}}\frac{1} {\mu _{n}}(f,g_{n})\varphi _{n}}$$

describes a regularization scheme with regularization parameter m →∞ and $\left \|R_{m}\right \| = 1/\mu _{m}$ .

Proof.

Choose q such that q(m, μ) = 1 for μ ≥ μ _m and q(m, μ) = 0 for μ < μ _m . Then, since μ _m → 0 as m → ∞, the conditions of the previous theorem are clearly satisfied with $c(m) = \frac{1} {\mu _{m}}$. Hence $\left \|R_{m}\right \| \leq \frac{1} {\mu _{m}}$. Equality follows from the identity $R_{m}g_{m} =\varphi _{m}/\mu _{m}$.

We conclude this section by establishing a discrepancy principle for the spectral cutoff regularization scheme.

Theorem 2.11.

Let A : X → Y be an injective compact operator with dense range in Y, and let f ∈ Y and δ > 0. Then there exists a smallest integer m such that

$$\displaystyle{\left \|AR_{m}f - f\right \| \leq \delta.}$$

Proof.

Since $\overline{A(X)} = Y$, A ^∗ is injective. Hence the singular value decomposition with the singular system $(\mu _{n},g_{n},\varphi _{n})$ for A ^∗ implies that for every f ∈ Y we have that

$$\displaystyle{ f =\sum _{ 1}^{\infty }(f,g_{ n})g_{n}. }$$

(2.2)

Hence

$$\displaystyle{{ \left \|(AR_{m} - I)f\right \|}^{2} =\sum _{\mu _{ n}<\mu _{m}}{\left \vert (f,g_{n})\right \vert }^{2} \rightarrow 0 }$$

(2.3)

as m → ∞. In particular, there exists a smallest integer m = m(δ) such that $\left \|AR_{m}f - f\right \| \leq \delta$.

Note that from (2.2) and (2.3) we have that

$$\displaystyle{{ \left \|AR_{m}f - f\right \|}^{2} ={ \left \|f\right \|}^{2} -\sum _{\mu _{ n}\geq \mu _{m}}{\left \vert (f,g_{n})\right \vert }^{2}. }$$

(2.4)

In particular, m(δ) is determined by the condition that m(δ) is the smallest value of m such that the right-hand side of (2.4) is less than or equal to δ ². For example, in the case of the backward heat equation (Example 2.1) we have that $g_{n}(x) = \sqrt{2/\pi }\sin nx$, and hence m is determined by the condition that m is the smallest integer such that

$$\displaystyle{{\left \|f\right \|}^{2} -\sum _{ 1}^{m}{\left \vert b_{ n}\right \vert }^{2} {\leq \delta }^{2}\mathrm{,}}$$

where the b _n are the Fourier coefficients of f.

It can be shown that the preceding discrepancy principle for the spectral cutoff method is regular (Theorem 15.26 of [111]).

2.3 Tikhonov Regularization

We now introduce and study the most popular regularization scheme in the field of ill-posed problems.

Theorem 2.12.

Let A : X → Y be a compact operator. Then for every α > 0 the operator αI + A ^∗ A : X → X is bijective and has a bounded inverse. Furthermore, if A is injective, then

$$\displaystyle{R_{\alpha }:= {(\alpha I + {A}^{{\ast}}A)}^{-1}{A}^{{\ast}}}$$

describes a regularization scheme with $\left \|R_{\alpha }\right \| \leq 1/2\sqrt{\alpha }$ .

Proof.

From

$$\displaystyle{\alpha {\left \|\varphi \right \|}^{2} \leq (\alpha \varphi +{A}^{{\ast}}A\varphi,\varphi )}$$

for $\varphi \in X$ we can conclude that for α > 0 the operator α I + A ^∗ A is injective. Hence, since A ^∗ A is a compact operator, by Riesz’s theorem we have that ${(\alpha I + {A}^{{\ast}}A)}^{-1}$ exists and is bounded.

Now assume that A is injective, and let $(\mu _{n},\varphi _{n},g_{n})$ be a singular system for A. Then for f ∈ Y the unique solution $\varphi _{\alpha }$ of

$$\displaystyle{\alpha \varphi _{\alpha } + {A}^{{\ast}}A\varphi _{\alpha } = {A}^{{\ast}}f}$$

is given by

$$\displaystyle{\varphi _{\alpha } =\sum _{ 1}^{\infty } \frac{\mu _{n}} {\alpha +\mu _{n}^{2}}(f,g_{n})\varphi _{n}\,\mathrm{,}}$$

i.e., R _α can be written in the form

$$\displaystyle{R_{\alpha }f =\sum _{ 1}^{\infty }\frac{1} {\mu _{n}}q(\alpha,\mu _{n})(f,g_{n})\varphi _{n}\,\mathrm{,}}$$

where

$$\displaystyle{q(\alpha,\mu ) = \frac{{\mu }^{2}} {\alpha +{\mu }^{2}}.}$$

Since 0 < q(α, μ) < 1 and $\sqrt{\alpha }\mu \leq \left (\alpha {+\mu }^{2}\right )/2$, we have that $\left \vert q(\alpha,\mu )\right \vert \leq \mu /2\sqrt{\alpha }$, and the theorem follows from Theorem 2.9.

The next theorem shows that the function $\varphi _{\alpha } = R_{\alpha }f$ can be obtained as the solution of an optimization problem.

Theorem 2.13.

Let A : X → Y be a compact operator, and let α > 0. Then for every f ∈ Y there exists a unique $\varphi _{\alpha } \in X$ such that

$$\displaystyle{{\left \|A\varphi _{\alpha } - f\right \|}^{2} +\alpha { \left \|\varphi _{\alpha }\right \|}^{2} =\inf _{\varphi \in X}\left \{{\left \|A\varphi - f\right \|}^{2} +\alpha { \left \|\varphi \right \|}^{2}\right \}.}$$

The minimizer is the unique solution of $\alpha \varphi _{\alpha } + {A}^{{\ast}}A\varphi _{\alpha } = {A}^{{\ast}}f$ .

Proof.

From

$$\displaystyle\begin{array}{rcl}{ \left \|A\varphi - f\right \|}^{2} +\alpha { \left \|\varphi \right \|}^{2}& =&{ \left \|A\varphi _{\alpha } - f\right \|}^{2} +\alpha { \left \|\varphi _{\alpha }\right \|}^{2} {}\\ & \quad +& 2\mathrm{Re}(\varphi -\varphi _{\alpha },\alpha \varphi _{\alpha } + {A}^{{\ast}}A\varphi _{\alpha } - {A}^{{\ast}}f) {}\\ & \quad +&{ \left \|A(\varphi -\varphi _{\alpha })\right \|}^{2} +\alpha { \left \|\varphi -\varphi _{\alpha }\right \|}^{2}\,\mathrm{,} {}\\ \end{array}$$

which is valid for every $\varphi,\varphi _{\alpha } \in X$, we see that if $\varphi _{\alpha }$ satisfies $\alpha \varphi _{\alpha } + {A}^{{\ast}}A\varphi _{\alpha } = {A}^{{\ast}}f$, then $\varphi _{\alpha }$ minimizes the Tikhonov functional

$$\displaystyle{{\left \|A\varphi - f\right \|}^{2} +\alpha { \left \|\varphi \right \|}^{2}.}$$

On the other hand, if $\varphi _{\alpha }$ is a minimizer of the Tikhonov functional, then set

$$\displaystyle{\psi:=\alpha \varphi _{\alpha } + {A}^{{\ast}}A\varphi _{\alpha } - {A}^{{\ast}}f}$$

and assume that ψ ≠ 0. Then for $\varphi:=\varphi _{\alpha } - t\psi$, with t a real number, we have that

$$\displaystyle\begin{array}{rcl}{ \left \|A\varphi - f\right \|}^{2} +\alpha { \left \|\varphi \right \|}^{2}& =&{ \left \|A\varphi _{\alpha } - f\right \|}^{2} +\alpha { \left \|\varphi _{\alpha }\right \|}^{2} \\ & \quad -& 2t{\left \|\psi \right \|}^{2} + {t}^{2}({\left \|A\psi \right \|}^{2} +\alpha { \left \|\psi \right \|}^{2}).{}\end{array}$$

(2.5)

The minimum of the right-hand side of (2.5) occurs when

$$\displaystyle{t = \frac{{\left \|\psi \right \|}^{2}} {{\left \|A\psi \right \|}^{2} +\alpha { \left \|\psi \right \|}^{2}}\,\mathrm{,}}$$

and for this t we have that ${\left \|A\varphi - f\right \|}^{2} +\alpha { \left \|\varphi \right \|}^{2} <{ \left \|A\varphi _{\alpha } - f\right \|}^{2} +\alpha { \left \|\varphi _{\alpha }\right \|}^{2}$, which contradicts the definition of $\varphi _{\alpha }$. Hence ψ = 0, i.e., $\alpha \varphi _{\alpha } + {A}^{{\ast}}A\varphi _{\alpha } = {A}^{{\ast}}f$.

By the interpretation of Tikhonov regularization as the minimizer of the Tikhonov functional, its solution $\varphi _{\alpha }$ keeps the residual ${\left \|A\varphi _{\alpha } - f\right \|}^{2}$ small and is stabilized through the penalty term $\alpha {\left \|\varphi _{\alpha }\right \|}^{2}$. This suggests the following two constrained optimization problems:

Minimum norm solution : for a given δ > 0 minimize $\left \|\varphi \right \|$ such that $\left \|A\varphi - f\right \| \leq \delta$.

Quasi-solutions : for a given ρ > 0 minimize $\left \|A\varphi - f\right \|$ such that $\left \|\varphi \right \| \leq \rho$.

We begin with the idea of a minimum norm solution and view this as a discrepancy principle for choosing $\varphi$ in a Tikhonov regularization.

Theorem 2.14.

Let A : X → Y be an injective compact operator with dense range in Y, and let f ∈ Y with $\left \|f\right \| >\delta > 0$ . Then there exists a unique α such that

$$\displaystyle{\left \|AR_{\alpha }f - f\right \| =\delta.}$$

Proof.

We must show that

$$\displaystyle{F(\alpha ):={ \left \|AR_{\alpha }f - f\right \|}^{2} {-\delta }^{2}}$$

has a unique zero. As in Theorem 2.11, we have that

$$\displaystyle{f =\sum _{ 1}^{\infty }(f,g_{ n})g_{n},}$$

and for $\varphi _{\alpha } = R_{\alpha }f$ we have that

$$\displaystyle{\varphi _{\alpha } =\sum _{ 1}^{\infty } \frac{\mu _{n}} {\alpha +\mu _{n}^{2}}(f,g_{n})\varphi _{n}.}$$

Hence

$$\displaystyle{F(\alpha ) =\sum _{ 1}^{\infty } \frac{{\alpha }^{2}} {{(\alpha +\mu _{n}^{2})}^{2}}{\left \vert (f,g_{n})\right \vert }^{2} {-\delta }^{2}.}$$

Since F is a continuous function of α and strictly monotonically increasing with limits F(α) → −δ ² as α → 0 and $F(\alpha ) \rightarrow {\left \|f\right \|}^{2} {-\delta }^{2} > 0$ as α → ∞, F has exactly one zero α = α(δ).

To prove the regularity of the foregoing discrepancy principle for Tikhonov regularizations, we need to introduce the concept of weak convergence.

Definition 2.15.

A sequence $\{\varphi _{n}\}$ in X is said to be weakly convergent to $\varphi \in X$ if

$$\displaystyle{\lim _{n\rightarrow \infty }(\psi,\varphi _{n}) = (\psi,\varphi )}$$

for every ψ ∈ X and we write $\varphi _{n} \rightharpoonup \varphi$, n → ∞.

Note that norm convergence $\varphi _{n} \rightarrow \varphi$, n → ∞, always implies weak convergence, but, as the following example shows, the converse is generally false.

Example 2.16.

Let ℓ ² be the space of all sequences $\{a_{n}\}_{1}^{\infty }$, $a_{n} \in \mathbb{C}$, such that

$$\displaystyle{ \sum _{1}^{\infty }{\left \vert a_{ n}\right \vert }^{2} < \infty. }$$

(2.6)

It is easily shown that, with componentwise addition and scalar multiplication, ℓ ² is a Hilbert space with inner product

$$\displaystyle{(a,b) =\sum _{ 1}^{\infty }a_{ n}\bar{b}_{n},}$$

where $a =\{ a_{n}\}_{1}^{\infty }$ and $b =\{ b_{n}\}_{1}^{\infty }$. In ℓ ² we now define the sequence $\{\varphi _{n}\}$ by $\varphi _{n} = (0,0,0,\cdots \,,1,0,\cdots \,)$, where the one appears in the nth entry. Then $\{\varphi _{n}\}$ is not norm convergent since $\left \|\varphi _{n} -\varphi _{m}\right \| = \sqrt{2}$ for m ≠ n, and hence $\{\varphi _{n}\}$ is not a Cauchy sequence. On the other hand, for ψ = { a _n } ∈ ℓ ² we have that $(\psi,\varphi _{n}) = a_{n} \rightarrow 0$ as n → ∞ due to the convergence of the series in (2.6). Hence $\{\varphi _{n}\}$ is weakly convergent to zero in ℓ ².

Theorem 2.17.

Every bounded sequence in a Hilbert space contains a weakly convergent subsequence.

Proof.

Let $\{\varphi _{n}\}$ be a bounded sequence, $\left \|\varphi _{n}\right \| \leq C$. Then for each integer m the sequence $(\varphi _{m},\varphi _{n})$ is bounded for all n. Hence by the Bolzano–Weierstrass theorem and a diagonalization process (cf. the proof of Theorem 1.17) we can select a subsequence $\{\varphi _{n(k)}\}$ such that $(\varphi _{m},\varphi _{n(k)})$ converges as k → ∞ for every integer m. Thus the linear functional F defined by

$$\displaystyle{F(\psi ):=\lim _{k\rightarrow \infty }(\psi,\varphi _{n(k)})}$$

is well defined on $U:=\mathrm{ span}\{\varphi _{m}\}$ and, by continuity, on $\bar{U}$. Now let $P: X \rightarrow \bar{ U}$ be the orthogonal projection operator, and for arbitrary ψ ∈ X write $\psi = P\psi + (I - P)\psi$. For arbitrary ψ ∈ X define F(ψ) by

$$\displaystyle\begin{array}{rcl} F(\psi ):=\lim _{k\rightarrow \infty }(\psi,\varphi _{n(k)})& =& \lim _{k\rightarrow \infty }\left [(P\psi,\varphi _{n(k)}) + \left (\left (I - P\right )\psi,\varphi _{n(k)}\right )\right ] {}\\ & =& \lim _{k\rightarrow \infty }\left (P\psi,\varphi _{n(k)}\right ), {}\\ \end{array}$$

where we have used the easily verifiable fact that P is self-adjoint. Thus F is defined on all of X. Furthermore, $\left \|F\right \| \leq C$. Hence, by the Riesz representation theorem, there exists a unique $\varphi \in X$ such that $F(\psi ) = (\psi,\varphi )$ for every ψ ∈ X. We can now conclude that $\lim _{k\rightarrow \infty }(\psi,\varphi _{n(k)}) = (\psi,\varphi )$ for every ψ ∈ X, i.e., $\varphi _{n(k)}$ is weakly convergent to $\varphi$ as k → ∞.

We are now in a position to show that the discrepancy principle of Theorem 2.14 is regular.

Theorem 2.18.

Let A : X → Y be an injective compact operator with dense range in Y. Let f ∈ A(X) and f ^δ ∈ Y satisfy $\left \|{f}^{\delta } - f\right \| \leq \delta < \left \|{f}^{\delta }\right \|$ with δ > 0. Then there exists a unique α = α(δ) such that

$$\displaystyle{\left \|AR_{\alpha (\delta )}{f}^{\delta } - {f}^{\delta }\right \| =\delta }$$

and

$$\displaystyle{R_{\alpha (\delta )}{f}^{\delta } \rightarrow {A}^{-1}f}$$

as δ → 0.

Proof.

In view of Theorem 2.14, we only need to establish convergence. Since ${\varphi }^{\delta } = R_{\alpha (\delta )}{f}^{\delta }$ minimizes the Tikhonov functional, we have that

$$ \displaystyle\begin{array}{rcl} {\delta }^{2} +\alpha {\left \|{\varphi }^{\delta }\right \|}^{2}& =&{ \left \|{A\varphi }^{\delta } - {f}^{\delta }\right \|}^{2} +\alpha { \left \|{\varphi }^{\delta }\right \|}^{2} {}\\ & \leq &{ \left \|A{A}^{-1}f - {f}^{\delta }\right \|}^{2} +\alpha { \left \|{A}^{-1}f\right \|}^{2} {}\\ & \leq & {\delta }^{2} +\alpha { \left \|{A}^{-1}f\right \|}^{2}, {}\\ \end{array}$$

and hence $\left \|{\varphi }^{\delta }\right \| \leq \left \|{A}^{-1}f\right \|$. Now let g ∈ Y. Then

$$\displaystyle\begin{array}{rcl} \left \vert ({A\varphi }^{\delta } - f,g)\right \vert & \leq & \left (\left \|{A\varphi }^{\delta } - {f}^{\delta }\right \| + \left \|{f}^{\delta } - f\right \|\right )\left \|g\right \| \\ & \leq & 2\delta \left \|g\right \| \rightarrow 0 {}\end{array}$$

(2.7)

as δ → 0. Since A is injective, A ^∗(Y ) is dense in X, and hence for every ψ ∈ X there exists a sequence {g _n } in Y such that A ^∗ g _n → ψ. Then

$$\displaystyle{ {(\varphi }^{\delta }-\varphi,\psi ) = {(\varphi }^{\delta }-\varphi,{A}^{{\ast}}g_{ n}) + {(\varphi }^{\delta }-\varphi,\psi -{A}^{{\ast}}g_{ n}) }$$

(2.8)

and, for every $\epsilon$ > 0,

$$\displaystyle{ \left \vert ({\varphi }^{\delta }-\varphi,\psi -{A}^{{\ast}}g_{ n})\right \vert \leq \left \|{\varphi }^{\delta }-\varphi \right \|\left \|\psi -{A}^{{\ast}}g_{ n}\right \| < \frac{\epsilon } {2} }$$

(2.9)

for all δ > 0 and N > N ₀ since $\left \|{\varphi }^{\delta }-\varphi \right \|$ is bounded. Hence for N > N ₀ and δ sufficiently small we have from (2.7)–(2.9) that

$$\displaystyle\begin{array}{rcl} \left \vert {(\varphi }^{\delta }-\varphi,\psi )\right \vert & \leq & \left \vert {(\varphi }^{\delta }-\varphi,{A}^{{\ast}}g_{ n})\right \vert + \left \vert {(\varphi }^{\delta }-\varphi,\psi -{A}^{{\ast}}g_{ n})\right \vert {}\\ &\leq & \left \vert ({A\varphi }^{\delta } - f,g_{n})\right \vert + \frac{\epsilon } {2} {}\\ & \leq & \epsilon \,\mathrm{,} {}\\ \end{array}$$

where we have set $f = A\varphi$. We can now conclude that ${\varphi }^{\delta } \rightharpoonup {A}^{-1}f$ as δ → 0. Then, again using the fact that $\left \|{\varphi }^{\delta }\right \| \leq \left \|{A}^{-1}f\right \|$, we have that

$$ \displaystyle\begin{array}{rcl} {\left \|{\varphi }^{\delta } - {A}^{-1}f\right \|}^{2}& =&{ \left \|{\varphi }^{\delta }\right \|}^{2} - 2\mathrm{Re}\left ({\varphi }^{\delta },{A}^{-1}f\right ) +{ \left \|{A}^{-1}f\right \|}^{2} \\ & \leq & 2\left ({\left \|{A}^{-1}f\right \|}^{2} -\mathrm{ Re}\left ({\varphi }^{\delta },{A}^{-1}f\right )\right ) \rightarrow 0{}\end{array}$$

(2.10)

as δ → 0, and the proof is complete.

Under additional conditions on f, which may be viewed as a regularity condition on f, we can obtain results on the order of convergence.

Theorem 2.19.

Under the assumptions of Theorem 2.18 , if f ∈ AA ^∗ (Y ), then

$$\displaystyle{\left \|{\varphi }^{\delta } - {A}^{-1}f\right \| = O\left ({\delta }^{1/2}\right )\quad \mathrm{,}\quad \delta \rightarrow 0.s}$$

Proof.

We have that ${A}^{-1}f = {A}^{{\ast}}g$ for some g ∈ Y. Then from (2.10) we have that

$$ \displaystyle\begin{array}{rcl}{ \left \|{\varphi }^{\delta } - {A}^{-1}f\right \|}^{2}& \leq & 2\left ({\left \|{A}^{-1}f\right \|}^{2} -\mathrm{ Re}\left ({\varphi }^{\delta },{A}^{-1}f\right )\right ) {}\\ & =& 2\mathrm{Re}\left ({A}^{-1}f -{\varphi }^{\delta },{A}^{-1}f\right ) {}\\ & =& 2\mathrm{Re}\left (f - {A\varphi }^{\delta },g\right ) {}\\ & \leq & 2\left (\left \|f - {f}^{\delta }\right \| + \left \|{f}^{\delta } - {A\varphi }^{\delta }\right \|\right )\left \|g\right \| {}\\ & \leq & 4\delta \left \|g\right \|\,\mathrm{,} {}\\ \end{array}$$

and the theorem follows.

Tikhonov regularization methods also apply to cases where both the operator and the right-hand side are perturbed, i.e., both the operator and the right-hand side are “noisy.” In particular, consider the operator equation $A_{h}\varphi = {f}^{\delta }$, A _h : X → Y, where $\left \|A_{h} - A\right \| \leq h$ and $\left \|f - {f}^{\delta }\right \| \leq \delta$, respectively. Then the Tikhonov regularization operator is given by

$$\displaystyle{R_{\alpha }:={ \left (\alpha I + A_{h}^{{\ast}}A_{ h}\right )}^{-1}A_{ h}^{{\ast}}\,\mathrm{,}}$$

and the regularization solution ${\varphi }^{\alpha }:= R_{\alpha }{f}^{\delta }$ is found by minimizing the Tikhonov functional

$$\displaystyle{\left \|A_{h}\varphi - {f}^{\delta }\right \| +\alpha \left \|\varphi \right \|.}$$

The regularization parameter α = α(δ, h) is determined from the equation

$$\displaystyle{{\left \|A_{h}\varphi _{\alpha } - {f}^{\delta }\right \|}^{2} = \left (\delta +h{\left \|\varphi _{\alpha }\right \|}^{2}\right ).}$$

Then all of the results obtained earlier in the case where A is not noisy can be generalized to the present case where both A and f are noisy. For details we refer the reader to [130].

We now turn our attention to the method of quasi-solutions.

Theorem 2.20.

Let A : X → Y be an injective compact operator and let ρ > 0. Then for every f ∈ Y there exists a unique $\varphi _{0} \in X$ with $\left \|\varphi _{0}\right \| \leq \rho$ such that

$$\displaystyle{\left \|A\varphi _{0} - f\right \| \leq \left \|A\varphi - f\right \|}$$

for all $\varphi$ satisfying $\left \|\varphi \right \| \leq \rho$ . The element $\varphi _{0}$ is called the quasi-solution of $A\varphi = f$ with constraint ρ.

Proof.

We note that $\varphi _{0}$ is a quasi-solution with constraint ρ if and only if $A\varphi _{0}$ is a best approximation to f with respect to the set $V:=\{ A\varphi: \left \|\varphi \right \| \leq \rho \}$. Since A is linear, V is clearly convex, i.e., $\lambda \varphi _{1} + (1-\lambda )\varphi _{2} \in V$ for all $\varphi _{1},\varphi _{2} \in V$ and 0 ≤ λ ≤ 1. Suppose there were two best approximations to f, i.e., there exist v ₁, v ₂ ∈ V such that

$$\displaystyle{\left \|f - v_{1}\right \| = \left \|f - v_{2}\right \| =\inf _{v\in V }\left \|f - v\right \|.}$$

Then, since V is convex, $\frac{1} {2}\left (v_{1} + v_{2}\right ) \in V$, and hence

$$\displaystyle{\left \|f -\frac{v_{1} + v_{2}} {2} \right \| \geq \left \|f - v_{1}\right \|.}$$

By the parallelogram equality we now have that

$$\displaystyle\begin{array}{rcl}{ \left \|v_{1} - v_{2}\right \|}^{2}& =& 2{\left \|f - v_{ 1}\right \|}^{2} + 2{\left \|f - v_{ 2}\right \|}^{2} {}\\ & \quad -& 4{\left \|f -\frac{v_{1} + v_{2}} {2} \right \|}^{2} {}\\ & \leq & 0\,\mathrm{,} {}\\ \end{array}$$

and hence v ₁ = v ₂. Thus if there were two quasi-solutions $\varphi _{1}$ and $\varphi _{2}$, then $A\varphi _{1} = A\varphi _{2}$. But since A is injective $\varphi _{1} =\varphi _{2}$, i.e., the quasi-solution, if it exists, is unique.

To prove the existence of a quasi-solution, let $\{\varphi _{n}\}$ be a minimizing sequence, i.e., $\left \|\varphi _{n}\right \| \leq \rho$, and

$$\displaystyle{ \lim _{n\rightarrow \infty }\left \|A\varphi _{n} - f\right \| =\inf _{\left \|\varphi \right \|\leq \rho }\left \|A\varphi - f\right \|. }$$

(2.11)

By Theorem 2.17, there exists a weakly convergent subsequence of $\{\varphi _{n}\}$, and without loss of generality we assume that $\varphi _{n} \rightharpoonup \varphi _{0}$ as n → ∞ for some $\varphi _{0} \in X$. We will show that $A\varphi _{n} \rightarrow A\varphi _{0}$ as n → ∞. Since for every $\varphi \in X$ we have that

$$\displaystyle{\lim _{n\rightarrow \infty }\left (A\varphi _{n},\varphi \right ) =\lim _{n\rightarrow \infty }\left (\varphi _{n},{A}^{{\ast}}\varphi \right ) = \left (\varphi _{ 0},{A}^{{\ast}}\varphi \right ) = \left (A\varphi _{ 0},\varphi \right )\,\mathrm{,}}$$

we can conclude that $A\varphi _{n} \rightharpoonup A\varphi _{0}$. Now suppose that $A\varphi _{n}$ does not converge to $A\varphi _{0}$. Then $\{A\varphi _{n}\}$ has a subsequence such that $\left \|A\varphi _{n(k)} - A\varphi _{0}\right \| \geq \delta$ for some δ > 0. Since $\left \|\varphi _{n}\right \| \leq \rho$ and A is compact, $\{A\varphi _{n(k)}\}$ has a convergent subsequence that we again call $\{A\varphi _{n(k)}\}$. But since convergent sequences are also weakly convergent and have the same limit, $A\varphi _{n(k)} \rightarrow A\varphi _{0}$, which is a contradiction. Hence $A\varphi _{n} \rightarrow A\varphi _{0}$. From (2.11) we can now conclude that

$$\displaystyle{\left \|A\varphi _{0} - f\right \| =\inf _{\left \|\varphi \right \|\leq \rho }\left \|A\varphi - f\right \|\,\mathrm{,}}$$

and since ${\left \|\varphi _{0}\right \|}^{2} =\lim _{n\rightarrow \infty }\left (\varphi _{n},\varphi _{0}\right ) \leq \rho \left \|\varphi _{0}\right \|$, we have that $\left \|\varphi _{0}\right \| \leq \rho$. This completes the proof of the theorem.

We next show that under appropriate assumptions the method of quasi-solutions is regular.

Theorem 2.21.

Let A : X → Y be an injective compact operator with dense range, and let f ∈ A(X) and $\rho \geq \left \|{A}^{-1}f\right \|$ . For f ^δ ∈ Y with $\left \|{f}^{\delta } - f\right \| \leq \delta$ , let ${\varphi }^{\delta }$ be the quasi-solution to $A\varphi = {f}^{\delta }$ with constraint ρ. Then ${\varphi }^{\delta } \rightharpoonup {A}^{-1}f$ as δ → 0, and if $\rho = \left \|{A}^{-1}f\right \|$ , then ${\varphi }^{\delta } \rightarrow {A}^{-1}f$ as δ → 0.

Proof.

Let g ∈ Y. Then, since $\left \|{A}^{-1}f\right \| \leq \rho$ and $\left \|{A\varphi }^{\delta } - {f}^{\delta }\right \| \leq \left \|A\varphi - {f}^{\delta }\right \|$ for $f = A\varphi$, we have that

$$\displaystyle\begin{array}{rcl} \left \vert \left ({A\varphi }^{\delta } - f,g\right )\right \vert & \leq & \left (\left \|{A\varphi }^{\delta } - {f}^{\delta }\right \| + \left \|{f}^{\delta } - f\right \|\right )\left \|g\right \| \\ & \leq & \left (\left \|A{A}^{-1}f - {f}^{\delta }\right \| + \left \|{f}^{\delta } - f\right \|\right )\left \|g\right \| \\ & \leq & 2\delta \left \|g\right \|. {}\end{array}$$

(2.12)

Hence $\left ({A\varphi }^{\delta } - f,g\right ) = \left ({\varphi }^{\delta } - {A}^{-1}f,{A}^{{\ast}}g\right ) \rightarrow 0$ as δ → 0 for every g ∈ Y. Since A is injective, A ^∗(Y ) is dense in X, and we can conclude that ${\varphi }^{\delta } \rightharpoonup {A}^{-1}f$ as δ → 0 (cf. the proof of Theorem 2.18).

When $\rho = \left \|{A}^{-1}f\right \|$, we have (using $\left \|{\varphi }^{\delta }\right \| \leq \rho = \left \|{A}^{-1}f\right \|$) that

$$ \displaystyle\begin{array}{rcl}{ \left \|{\varphi }^{\delta } - {A}^{-1}f\right \|}^{2}& =&{ \left \|{\varphi }^{\delta }\right \|}^{2} - 2\mathrm{Re}\left ({\varphi }^{\delta },{A}^{-1}f\right ) +{ \left \|{A}^{-1}f\right \|}^{2} \\ & \leq & 2\mathrm{Re}\left ({A}^{-1}f -{\varphi }^{\delta },{A}^{-1}f\right ) \rightarrow 0{}\end{array}$$

(2.13)

as δ → 0.

Note that for regularity we need to know a priori the norm of the solution to the noise-free equation.

Theorem 2.22.

Under the assumptions of Theorem 2.21 , if f ∈ AA ^∗ (Y ) and $\rho = \left \|{A}^{-1}f\right \|$ , then

$$\displaystyle{\left \|{\varphi }^{\delta } - {A}^{-1}f\right \| = O\left ({\delta }^{1/2}\right )\quad \mathrm{,}\quad \delta \rightarrow 0.}$$

Proof.

We can write ${A}^{-1}f = {A}^{{\ast}}g$ for some g ∈ Y. From (2.12) and (2.13) we have that ${\left \|{\varphi }^{\delta } - {A}^{-1}f\right \|}^{2} \leq 2\mathrm{Re}\left (f - {A\varphi }^{\delta },g\right ) \leq 4\delta \left \|g\right \|$, and the theorem follows.